table of contents
| STACD.CONF(5) | STACD.CONF(5) |
NAME¶
stacd.conf - stacd(8) configuration file
SYNOPSIS¶
/etc/nvme/stacd.conf
DESCRIPTION¶
When stacd(8) starts up, it reads its configuration from stacd.conf.
CONFIGURATION FILE FORMAT¶
stacd.conf is a plain text file divided into sections, with configuration entries in the style key=value. Spaces immediately before or after the = are ignored. Empty lines are ignored as well as lines starting with #, which may be used for commenting.
OPTIONS¶
[Global] section¶
The following options are available in the [Global] section:
tron=
ip-family=
Choices are ipv4, ipv6, or ipv4+ipv6.
Defaults to ipv4+ipv6.
ignore-iface=
There is no guarantee that there will be a route to reach that IOC. However, we can use the socket option SO_BINDTODEVICE to force the connection to be made on a specific interface instead of letting the routing tables decide where to make the connection.
This option determines whether stacd will use SO_BINDTODEVICE to force connections on an interface or just rely on the routing tables. The default is to use SO_BINDTODEVICE, in other words, stacd does not ignore the interface.
BACKGROUND: By default, stacd will connect to IOCs on the same interface that was used to retrieve the discovery log pages. If stafd discovers a DC on an interface using mDNS, and stafd connects to that DC and retrieves the log pages, it is expected that the storage subsystems listed in the log pages are reachable on the same interface where the DC was discovered.
For example, let's say a DC is discovered on interface ens102. Then all the subsystems listed in the log pages retrieved from that DC must be reachable on interface ens102. If this doesn't work, for example you cannot "ping -I ens102 [storage-ip]", then the most likely explanation is that proxy arp is not enabled on the switch that the host is connected to on interface ens102. Whatever you do, resist the temptation to manually set up the routing tables or to add alternate routes going over a different interface than the one where the DC is located. That simply won't work. Make sure proxy arp is enabled on the switch first.
Setting routes won't work because, by default, stacd uses the SO_BINDTODEVICE socket option when it connects to IOCs. This option is used to force a socket connection to be made on a specific interface instead of letting the routing tables decide where to connect the socket. Even if you were to manually configure an alternate route on a different interface, the connections (i.e. host to IOC) will still be made on the interface where the DC was discovered by stafd.
Defaults to false.
[I/O controller connection management] section¶
Connectivity between hosts and subsystems in a fabric is controlled by Fabric Zoning. Entities that share a common zone (i.e., are zoned together) are allowed to discover each other and establish connections between them. Fabric Zoning is configured on Discovery Controllers (DC). Users can add/remove controllers and/or hosts to/from zones.
Hosts have no direct knowledge of the Fabric Zoning configuration that is active on a given DC. As a result, if a host is impacted by a Fabric Zoning configuration change, it will be notified of the connectivity configuration change by the DC via Asynchronous Event Notifications (AEN).
Table 1. List of terms used in this section:
| Term | Description |
| AEN | Asynchronous Event Notification. A CQE (Completion Queue Entry) for an Asynchronous Event Request that was previously transmitted by the host to a Discovery Controller. AENs are used by DCs to notify hosts that a change (e.g., a connectivity configuration change) has occurred. |
| DC | Discovery Controller. |
| DLP | Discovery Log Page. A host will issue a Get Log Page command to retrieve the list of controllers it may connect to. |
| DLPE | Discovery Log Page Entry. The response to a Get Log Page command contains a list of DLPEs identifying each controller that the host is allowed to connect with. Note that DLPEs may contain both I/O Controllers (IOCs) and Discovery Controllers (DCs). DCs listed in DLPEs are called referrals. stacd only deals with IOCs. Referrals (DCs) are handled by stafd. |
| IOC | I/O Controller. |
| Manual Config | Refers to manually adding [Subsystem] sections to /etc/nvme/nvme-stas.conf (see nvme-stas.conf(5)). |
| Automatic Config | Refers to receiving configuration from a DC as DLPEs |
| External Config | Refers to configuration done outside of the nvme-stas framework, for example using nvme-cli commands |
DCs notify hosts of connectivity configuration changes by sending
AENs indicating a "Discovery Log" change. The host uses these AENs
as a trigger to issue a Get Log Page command. The response to this command
is used to update the list of DLPEs containing the controllers the host is
allowed to access. Upon reception of the current DLPEs, the host will
determine whether DLPEs were added and/or removed, which will trigger the
addition and/or removal of controller connections. This happens in real time
and may affect active connections to controllers including controllers that
support I/O operations (IOCs). A host that was previously connected to an
IOC may suddenly be told that it is no longer allowed to connect to that IOC
and should disconnect from it.
IOC connection creation. There are 3 ways to configure IOC connections on a host:
IOC connection removal/prevention. There are 3 ways to remove (or prevent) connections to an IOC:
The decision by the host to automatically disconnect from an IOC following connectivity configuration changes is controlled by the honor-fabric-zoning parameter.
honor-fabric-zoning=
With yes, the removal of a DLPE means the subsystem is no longer zoned for this host, and stacd disconnects the I/O controllers it had connected for it. With no, zoning is disregarded entirely and every connection stacd has made is kept, whatever the CDC says. Connections can then only be removed with the nvme-cli command "nvme disconnect", by adding an exclude= entry, or by rebooting.
This only ever applies to connections stacd made itself. A connection that belongs to another orchestrator, or one made at boot from the NBFT, is never disconnected by stacd: that is decided by libnvme's ownership registry and by the NBFT, not by this parameter.
Defaults to yes.
connect-attempts-on-ncc=
If a host is currently failing to connect to an I/O controller and if the NCC bit associated with that I/O controller is asserted, the host can decide to stop trying to connect to that subsystem until connectivity is restored. This will be indicated by the CDC when it clears the NCC bit.
The parameter connect-attempts-on-ncc= controls whether stacd will take the NCC bit into account when attempting to connect to an I/O Controller. Setting connect-attempts-on-ncc= to 0 means that stacd will ignore the NCC bit and will keep trying to connect. Setting connect-attempts-on-ncc= to a non-zero value indicates the number of connection attempts that will be made before stacd gives up trying. Note that this value should be set to a value greater than 1. In fact, when set to 1, stacd will automatically use 2 instead. The reason for this is simple. It is possible that a first connect attempt may fail.
Defaults to 0.
[Controllers] section¶
The following options are available in the [Controllers] section:
Note
The controller= keyword no longer belongs here. Which controllers to connect to, and with what parameters, is stated in /etc/nvme/nvme-stas.conf as of nvme-stas 3.0 — a Discovery Controller as a [Discovery Controller] section, an I/O subsystem as a [Subsystem] section. See nvme-stas.conf(5). A controller= entry left behind in this file is reported in the log, not silently ignored.
exclude=
Deprecated. Use libnvme's host-wide exclusion list instead: /etc/nvme/exclusions.conf and /etc/nvme/exclusions.conf.d/, which are managed with nvme exclusion. That list is honored by every NVMe-oF tool on the host, not just stafd and stacd, and it is read live: an entry added there takes effect on the next connection attempt, without reloading stafd or stacd.
exclude= still works and is honored in addition to libnvme's list — a controller is excluded if it matches either — but it will be removed in a future release.
An entry is a series of fields separated by semi-colons:
exclude=transport=[trtype];traddr=[traddr];trsvcid=[trsvcid];host-iface=[iface];nqn=[nqn]
Every field is optional. Multiple exclude= keywords may appear in the config file to specify more than 1 excluded controller.
Note 1: A minimal match approach is used to eliminate unwanted controllers. That is, you do not need to specify all the parameters to identify a controller. Just specifying the host-iface, for example, can be used to exclude all controllers on an interface.
Note 2: an exclusion takes precedence over the connectivity configuration. A controller configured in /etc/nvme/nvme-stas.conf can be eliminated by the exclude= keyword.
Examples:
exclude = transport=tcp;traddr=fe80::2c6e:dee7:857:26bb # Eliminate a specific address exclude = host-iface=enp0s8 # Eliminate everything on this interface
SEE ALSO¶
| nvme-stas 3.0 |