1. Manuals
  2. Tumbleweed
  3. rds-tools
  4. rds-stress(1)

Scroll to navigation

RDS-STRESS(1) General Commands Manual RDS-STRESS(1)

NAME

rds-stress — send messages between processes over RDS sockets

SYNOPSIS

rds-stress [OPTIONS]

DESCRIPTION

rds-stress sends messages between groups tasks, usually running on seperate machines.

First a passive receiving instance is started.

$ rds-stress

Then an active sending instance is started, giving it the address and port at which it will find a listening passive receiver. In addition, it is given configuration options which both instances will use.

$ rds-stress -s recvhost -p 4000 -t 1 -d 1

The active sender will parse the options, connect to the passive receiver, and send the options over this connection. From this point on both instances exhibit the exact same behaviour.

They will create a number of child tasks as specified by the -t option. Once the children are created the parent sleeps for a second at a time, printing a summary of statistics at each interval.

Each child will open an RDS socket, each binding to a port number in order after the port number given on the command line. The first child would bind to port 4001 in our example. Each child sets the send and receive buffers to exactly fit the number of messages, requests and acks, that will be in flight as determind by the command line arguments.

The children then enter their loop. They will keep a number of sent messages outstanding as specified by the -d option. When they reach this limit they will wait to receive acks which will allow them to send again. As they receive messages from their peers they immediately send acks.

Every second, the parent process will display statistics of the ongoing stress test. The output is described in section OUTPUT below.

If the -T option is given, the test will terminate after the specified time, and a summary is printed.

Each child maintains outstanding messages to all other children of the other instance. They do not send to their siblings.

OPTIONS

Passive Receiver and Active Sender Configuration Options

The following options are available for use on either the passive receiver or active sender command lines, but are not shared between the two:

--port port_number
Each parent binds a TCP socket to this port number and their respective address. They will trade the negotiated options over this socket. Each child will bind an RDS socket to the range of ports immediately following this port number, for as many children as there are. The default port number value is 4000.
--recv-addr receive_address
This specifies the address that messages will be sent from. If -s is not specified then rds-stress waits for a connection on this address before proceeding.

If this option is not given, rds-stress will choose an appropriate address. The passive process will accept connections on all local interfaces, and obtain the address once the control connection is established. The active process will choose a local address based on the interface through.

Prints a round-trip time (RTT) histogram.
rtt-time
Reports on any RTT exceeding the rtt-time. The default is that there is no outlier checking.
If used, children processes will issue a "RDS_CANCEL_SENT_TO" for each task and report how long that operation took together with the destination IP-address and port plus the process-id of that child process. Since signal SIGINT is ignored by the children processes, a user can press Ctrl-C interactively to terminate the parent process early and observe the amount of time it took to complete "RDS_CANCEL_SENT_TO" in the midst of send operations.
seconds
If used, the parent process will arm a timer (alarm(seconds)) to terminate in the midst of operation. This is useful in combination with the "--cancel-sent-to" option in order to measure the amount of time spent in "RDS_CANCEL_SENT_TO" while send operations are ongoing.
If used, each task will not set RDS-INQ socket option to receive the number of backlog bytes pending to be read from the receive queue. Since application is now not aware of pending bytes in the receive queue, the number of recvmsg() syscalls can be higher leading to higher system load and lower cpu yield.

Active Sender Configuration Options

The following options are available for use on the active sender command line, but are shared with the passive receiver instance prior to the test starting.

--send-addr send_address
A connection attempt is made to this address. Once its complete and the options are sent over it then children will be created and work will proceed.

If this option is not given, rds-stress will choose an appropriate address. The passive process will accept connections on all local interfaces, and obtain the address once the control connection is established. The active process will choose a local address based on the interface through which it connects to the destination address.

--ack-bytes ack_bytes
This specifies the size of the ack messages, in bytes. There is a minimum size which depends on the format of the ack messages, which may change over time. See section "Message Sizes" below.
--req-bytes request_bytes
This specifies the size of the request messages, in bytes. It also has a minimum size which may change over time. See section "Message Sizes" below.
--one-way
Datagrams will be sent one way only. The default is that they are sent both ways.
--depth queue_depth
Each child will try to maintain this many sent messages outstanding to each of its peers on the remote address. The default depth is 1.
--tasks nr_tasks
Each parent will create this many children tasks. The default number of tasks is 1.
--time seconds
Specify the duration of the test run. After the specified number of seconds, all processes on both ends of the connection will terminate, and the active instance will print a summary. By default, rds-stress will keep on sending and receiving messages.
--tos tos
Type of Service. The default is 0.
retries
This can be used to specify the number of times an active sender instance should retry on a failed connect. The default is 0, meaning no reties if the connect fails.
enable
Enable/disable congestion monitoring. The default is 1, meaning enabled.
--verify
With this option enabled, packets are filled with a pattern that is verified by the receiver. This check can help detect data corruption occuring under high load.
--report-cpu
This causes rds-stress to create child tasks which just consume CPU cycles. One task is created for each CPU in the system. First each child observes the maximum rate at which it can consume cycles. This means that this option should only be given on an idle system. rds-stress can then calculate the CPU use of the system by observing the lesser rate at which the children consume cycles. This option is *not* shared between the active sender and passive receiver instances. It must be specified on each rds-stress command line.
--report-summary
This flag can be used in conjunction with -T. It suppresses the ongoing display of statistics, and prints a summary only.
--rtprio
This tells the rds-stress parent process to run with SCHED_RR priority, giving it precedence over the child processes. This is useful when running with lots of tasks, where there is a risk of the child processes starving the parent, and skewing the results.
--trace
Trace execution.
Print job parameters/options.
Generate perf data output for script parsing.
Reset the connection and exit.
Enable async sends.
Always flip-flop between send-only, then receive-only mode. This is to avoid the slower-is-faster paradox, which happens when the cost of a sendmsg becomes higher than the cost of a recvmsg making rds-stress transition from a bursty test-mode to a streaming test-mode.
Synchronizes the send-only and receive-only modes with other processes via a System V semaphore. Without this option, the send-only and receive-only modes of processes are not synchronized, and can drift from each other, especially with multiple invocations of "rds-stress". All processes sending at the same time, and subsequently hitting upon bottlenecks at the same time, is a different performance-test than processes sending at slightly different times. If that distinction is uncontrolled and left to chance, performance measurements are left to chance as well. This option implies and enables the --always-bursty.

RDMA specific Options:

--rdma-bytes rdma_bytes
RDSv3 is capable of transmitting part of a message via RDMA directly from application buffer to application buffer. This option enables RDMA support in rds-stress: request packets include parameters for an RDMA READ or WRITE operation, which the receiving process executes at the time the ACK packet is sent. See section "Message Sizes" below.
--rdma-buffers iovecs
Number of user buffers to target. The max supported is 512. The default is 1.
--rdma-mode nr
Mode (0=readwrite,1=readonly,2=writeonly). The default is 0.
enable
Enable/disable RDMA use_once buffers. The default is 0, meaning disabled.
use-get_mr
Use GET_MR to get RDMA cookie or use new RDMA_MAP control message which is more efficient and eliminates a system call. The default is 0, meaning disabled.
enable
Enable/disable RDMA read fencing. The default is 1, meaning enabled.
enable
Option is now obsolete as RDMA notifications are always enabled.
enable
Enable/disable reuse of MR handle for all RDMA operations. The default 0, meaning disabled.
alignment
Request a buffer alignment to test unaligned RDMA. The default is 0, meaning disabled.
Used to track whether RDS zerocopy code issues repeating R_Keys.

Message Sizes

Options which set a message size (such as -a) specify a number of bytes by default. By appending K, M, or G, you can specify the size in kilobytes, megabytes or gigabytes, respectively. For instance, the following will run rds-stress with a message and ACK size of 1024 bytes, and an RDMA message size of 1048576 bytes:

rds-stress ... -q 1K -a 1K -D 1M

OUTPUT

Each parent outputs columns of statistics at a regular interval:

tsks
The number of child tasks which are running.
tx/s
The number of sendmsg() calls that all children are executing, per second.
tx+rx K/s
The total number of bytes that are flowing through sendmsg() and recvmsg() for all children. This includes both request and ack messages.
rw+rr K/s
The total number of bytes that are being transferred via RDMA READs and WRITEs for all children.
tx us/c
The average number of microseconds spent in sendmsg() calls.
rtt us
The average round trip time for a request and ack message pair. This measures the total time between when a task sends a request and when it finally receives the ack for that message. Because it includes the time it takes for the receiver to wake up, receive the message, and send an ack, it can grow to be quite large under load.
cpu %
This is the percentage of available CPU resources on this machine that are being consumed since rds-stress started running. It will show -1.00 if -c is not given. It is calculated based on the amount of CPU resources that CPU soaking tasks are able to consume. This lets it measure CPU use by the system, say in interrupt handlers, that task-based CPU accounting does not include. For this to work rds-stress must be started with -c on an idle system.
May 15, 2007 Linux 6.4.0-150700.53.19-default