NAME¶

rds-info — display information from the RDS kernel module

SYNOPSIS¶

DESCRIPTION¶

The rds-info utility presents various sources of information that the RDS kernel module maintains. When run without any optional arguments rds-info will output all the information it knows of. When options are specified then only the information associated with those options is displayed.

The options are as follows:

-v

Requests verbose output. When this option is given, some classes of information will display additional data.

-c

Display global counters. Each counter increments as its event occurs. The counters may not be reset. The set of supported counters may change over time.

--E

Indicates pending failed notifications. The connection is not open for traffic until all pending failed notifications have been reaped.

CounterName

The name of the counter. These names come from the kernel and can change depending on the capability of the kernel module.

Value

The number of times that the counter has been incremented since the kernel module was loaded.

-k

Display all the RDS sockets in the system. There will always be one socket listed that is neither bound to nor connected to any addresses because rds-info itself uses an unbound socket to collect information.

BoundAddr, BPort

The IP address and port that the socket is bound to. 0.0.0.0 0 indicates that the socket has not been bound.

ConnAddr, CPort

The IP address and port that the socket is connected to. 0.0.0.0 0 indicates that the socket has not been connected.

SndBuf, RcvBuf

The number of bytes of message payload which can be queued for sending or receiving on the socket, respectively.

Inode

The number of the inode object associated with the socket. Can be used to locate the process owning a given socket by searching /proc/*/fd for open files referencing a socket with this inode number.

Cong

This field displays 1 if a rds socket is in a congested state and 0 otherwise. Needs kernel support and will display -1 if the the kernel does not support this field.

Pid

The pid of the process that intitially created the socket. Needs kernel support to be displayed and will be left blank otherwise.

Comm

The task_comm of the process associated with the pid. Needs kernel support to be displayed and will be left blank otherwise.

-n

Display all RDS connections. RDS connections are maintained between nodes by transports.

LocalAddr

The IP address of this node. For connections that originate and terminate on the same node the local address indicates which address initiated the connection establishment.

RemoteAddr

The IP address of the remote end of the connection.

NextTX

The sequence number that will be given to the next message that is sent over the connection.

NextRX

The sequence number that is expected from the next message to arrive over the connection. Any incoming messages with sequence numbers less than this will be dropped.

Flg

Flags which indicate the state of the connection.

s

A process is currently sending a message down the connection.

c

The transport is attempting to connect to the remote address.

C

The connection to the remote host is connected and active.

-N

Display the RDS information, as specified by any other command-line options, for all network namespaces. Information for the "home" namespace (the network namespace in which rds-info is executed) is printed first, followed by information for any other network namespaces in the order of their namespace identifiers. A header containing a given namespace's identifier is printed before the information for that namespace. If the underlying kernel RDS implementation does not support network namespaces, the global RDS information will appear for every network namespace.

In general, namespaces do not have human-oriented names; the kernel uses opaque numeric values as namespace identifiers. ip-netns(8) can create and operate on named network namespaces, but these names are for user convenience only, and are not available to the kernel or to ip-netns-unaware software. Container frameworks that isolate containers by using network (and other) namespaces typically create these namespaces directly and do not use or export names for them.

Thus, in the absence of a more user-friendly alternative, rds-info uses the numeric kernel namespace identifiers to identify namespaces in its output.

Users can obtain the identifier of the network namespace for a process as follows:

# echo $$
77712
# ls -l /proc/77712/ns/net
lrwxrwxrwx 1 root root 0 Jul 18 11:21 /proc/77712/ns/net -> 'net:[4026531840]'

In this example, the value 4026531840 is the identifier for the network namespace of the current shell process.

To locate the RDS information for a particular container in rds-info output:

1. Identify a process in the container, for example by using the container management software's ps or inspect command.
2. Find the network namespace identifier for the container process from the /proc entry for the process, as indicated above.
3. Look for the rds-info output section for the network namespace of the container's process.

Accessing other namespaces requires that the rds-info process has the CAP_SYS_ADMIN and CAP_SYS_PTRACE capabilities, and that the libdhash package is installed on the system.

-r, -s, -t

Display the messages in the receive, send, or retransmit queues respectively.

LocalAddr, LPort

The local IP address and port on this node associated with the message. For sent messages this is the source address, for receive messages it is the destination address.

RemoteAddr, RPort

The remote IP address and port associated with the message. For sent messages this is the destination address, for receive messages it is the source address.

Seq

The sequence number of the message.

Bytes

The number of bytes in the message payload.

The following information sources are dependent on specific transports which may not always be available.

-I

Display the IB connections which the IB transport is using to provide RDS connections.

LocalAddr

The local IP address of this connection.

RemoteAddr

The remote IP address of this connection.

LocalDev

The local IB Global Identifier, printed in IPv6 address syntax.

RemoteDev

The remote IB Global Identifier, printed in IPv6 address syntax.

SrcQPNo

The source QP number for this connection.

DstQPNo

The destination QP number for this connection.

If verbose output is requested, additional per-connection information will be displayed. These additional fields are documented below.

CacheAllocs

The total number of cache allocs used by this connection.

RecvAllocCtr

The RDS IB connection recv ring alloc counter.

RecvFreeCtr

The RDS IB connection recv ring free counter.

SendAllocCtr

The RDS IB connection send ring alloc counter.

SendFreeCtr

The RDS IB connection send ring free counter.

SendBytes

The total number of bytes sent by this connection.

RecvBytes

The total number of bytes received by this connection.

RReadBytes

The total number of RDMA bytes read by this connection.

RWriteBytes

The total number of RDMA bytes written by this connection.

TxPollTs

The time in msec from last scq[send completion queue] poll.

RxPollTs

The time in msec from last rcq[recv completion queue] poll.

TxPollCnt

The number of scq polls so far, by RDS on this connection.

RxPollCnt

The number of rcq polls so far, by RDS on this connection.

Scq_vec

The zero based vector number associated with the send CQ.

Rcq_vec

The zero based vector number associated with the Receive CQ.

The "-I" option also supports the use of the "-o" flag in addition to it to allow the user to selectively display the additional fields documented above.

Example: rds-info -I -o "rx_poll_cnt, r_read_bytes, recv_alloc_ctr" rds-info -I -o "cache_allocs, send_bytes, tx_poll_ts"

-T

Display the TCP sockets which the TCP transport is using to provide RDS connections.

LocalAddr, LPort

The local IP address and port of this socket.

RemoteAddr, RPort

The remote IP address and port that this socket is connected to.

HdrRemain

The number of bytes that must be read off the socket to complete the next full RDS header.

DataRemain

The number of bytes that must be read off the socket to complete the data payload of the message which is being received.

SentNxt

The TCP sequence number of the first byte of the last message that we sent down the connection.

ExpectedUna

The TCP sequence number of the byte past the last byte of the last message that we sent down the connection. When we see that the remote side has acked up to this byte then we know that the remote side has received all our RDS messages.

SeenUna

The TCP sequence number of the byte past the last byte which has been acked by the remote host.