table of contents
CONNTRACKD.CONF(5) | File Formats Manual | CONNTRACKD.CONF(5) |
NAME¶
conntrackd.conf - configuration file for conntrackd daemon
DESCRIPTION¶
conntrackd.conf is the main configuration file for the conntrackd(8) daemon. It is loaded by calling `conntrackd -C conntrackd.conf'.
The format of this file is simple, using brackets for sections and key-value pairs for concrete configuration directives:
section1 { option1 value1 option2 value2 } section2 { option3 value3 subsection1 { option4 value4 } }
You should consider this file as case-sensitive. Empty lines and lines starting with the '#' character are ignored.
Before starting to develop a new configuration, you may want to learn the concepts behind this technology at http://conntrack-tools.netfilter.org/manual.html.
There are complete configuration examples at the end of this man page.
SYNC¶
This top-level section defines how conntrackd(8) should handle synchronization with other cluster nodes.
There are 3 main synchronization modes or protocols: NOTRACK, ALARM and FTFW.
There are 3 transport protocols as well: TCP, Multicast and UDP.
You have to choose one synchronization mode and one transport protocol.
Also, there are some general options in this section.
Mode FTFW¶
This mode is based on a reliable protocol that performs message tracking. Thus, the protocol can recover from message loss, re-ordering and corruption.
In this synchronization mode you may configure ResendQueueSize, CommitTimeout, PurgeTimeout, ACKWindowSize , DisableExternalCache and StartupResync.
- ResendQueueSize <value>
- Size of the resend queue (in objects). This is the maximum number of
objects that can be stored waiting to be confirmed via acknoledgment. If
you keep this value low, the daemon will have less chances to recover
state-changes under message omission. On the other hand, if you keep this
value high, the daemon will consume more memory to store dead objects.
Example: ResendQueueSize 131072
Default is 131072 objects.
- CommitTimeout <seconds>
- This parameter allows you to set an initial fixed timeout for the
committed entries when this node goes from backup to primary. This
mechanism provides a way to purge entries that were not recovered
appropriately after the specified fixed timeout. If you set a low value,
TCP entries in Established states with no traffic may hang. For example,
an SSH connection without KeepAlive enabled.
Example: CommitTimeout 180
By default, this option is not set (the daemon uses an approximate timeout value calculation mechanism).
- PurgeTimeout <seconds>
- If the firewall replica goes from primary to backup, the `conntrackd -t
command' is invoked in the script. This command schedules a flush of
the table in N seconds.
This is useful to purge the connection tracking table of zombie entries and avoid clashes with old entries if you trigger several consecutive hand-overs.
Default is 60 seconds.
- ACKWindowSize <value>
- Set the acknowledgement window size. If you decrease this value, the
number of acknowlegdments increases. More acknowledgments means more
overhead as conntrackd(8) has to handle more control messages. On
the other hand, if you increase this value, the resend queue gets more
populated. This results in more overhead in the queue releasing.
Example: ACKWindowSize 300
If not set, default window size is 300 (value is based on some practical experiments measuring the cycles spent by the acknowledgment handling with oprofile).
- DisableExternalCache <yes|no>
- This clause allows you to disable the external cache. Thus, the state
entries are directly injected into the kernel conntrack table. As a
result, you save memory in user-space but you consume slots in the kernel
conntrack table for backup state entries. Moreover, disabling the external
cache means more CPU consumption. You need a Linux kernel >=
2.6.29 to use this feature.
If you are installing conntrackd(8) for first time, please read the user manual and I encourage you to consider using the fail-over scripts instead of enabling this option!
By default this is set to no, meaning the external cache is enabled.
- StartupResync <yes|no>
- Order conntrackd to request a complete conntrack table resync against the
other node at startup. A single request will be made.
This is useful to get in sync with another node which has been running while we were down.
Example: StartupResync yes
By default, this clause is set to no.
Mode ALARM¶
This mode is spamming. It is based on a alarm-based protocol that periodically re-sends the flow state to the backup firewall replicas. This protocol consumes a lot of bandwidth but it resolves synchronization problems fast.
In this synchronization mode you may configure RefreshTime, CacheTimeout, CommitTimeout and PurgeTimeout.
- RefreshTime <seconds>
- If a conntrack entry is not modified in <= N seconds, then a message is
broadcasted. For example, this mechanism may be used to resynchronize
nodes that just joined the multicast group.
Example: RefreshTime 15
- CacheTimeout <seconds>
- If we don't receive a notification about the state of an entry in the
external cache after N seconds, then remove it.
Example: CacheTimeout 180
- CommitTimeout <seconds>
- Same as in FTFW mode.
- PurgeTimeout <seconds>
- Same as in FTFW mode.
Mode NOTRACK¶
Is the most simple mode as it is based on a best effort replication protocol, ie. unreliable protocol. This protocol sends and receives the state information without performing any specific checking.
In this synchronization mode you may configure DisableInternalCache, DisableExternalCache, CommitTimeout, PurgeTimeout and StartupResync.
- DisableInternalCache <yes|no>
- This clause allows you to disable the internal cache. Thus, the
synchronization messages are directly sent through the dedicated link.
This option is set to no by default.
- DisableExternalCache <yes|no>
- Same as in FTFW mode.
- CommitTimeout <seconds>
- Same as in FTFW mode.
- PurgeTimeout <seconds>
- Same as in FTFW mode.
- StartupResync <yes|no>
- Same as in FTFW mode.
MULTICAST¶
This section indicates to conntrackd(8) to use multicast as transport mechanism between nodes of the firewall cluster.
Please note you can specify more than one dedicated link. Thus, if one dedicated link fails, the daemon can fail-over to another. Note that adding more than one dedicated link does not mean that state-updates will be sent to all of them. There is only one active dedicated link at a given moment.
The Default keyword indicates that this interface will be selected as the initial dedicated link. You can have up to 4 redundant dedicated links.
Note: use different multicast groups for every redundant link.
Example:
Multicast Default { IPv4_address 225.0.0.51 Group 3781 IPv4_interface 192.168.100.101 Interface eth3 SndSocketBuffer 1249280 RcvSocketBuffer 1249280 Checksum on } Multicast { IPv4_address 225.0.0.51 Group 3782 IPv4_interface 192.168.100.102 Interface eth4 SndSocketBuffer 1249280 RcvSocketBuffer 1249280 Checksum on }
- IPv4_address <address>
- Multicast address: The address that you use as destination in the
synchronization messages. You do not have to add this IP to any of your
existing interfaces.
Example: IPv4_address 255.0.0.50
- Group <number>
- The multicast group that identifies the cluster.
Example: Group 3780
If any doubt, do not modify this value.
- IPv4_interface <address>
- IP address of the interface that you are going to use to send the
synchronization messages. Remember that you must use a dedicated link for
the synchronization messages.
Example: IPv4_interface 192.168.100.100
- Interface <name>
- The name of the interface that you are going to use to send the
synchronization messages.
Example: Interface eth2
- SndSocketBuffer <number>
- This transport protocol sender uses a buffer to enqueue the packets that
are going to be transmitted. The default size of this socket buffer is
available at /proc/sys/net/core/wmem_default.
This value determines the chances to have an overrun in the sender queue. The overrun results in packet loss, thus, losing state information that would have to be retransmitted. If you notice some packet loss, you may want to increase the size of the buffer. The system default size is usually around ~100 KBytes which is fairly small for busy firewalls.
Note: The NOTRACK protocol is best effort, it is really recommended to increase the buffer size.
Example: SndSocketBuffer 1249280
- RcvSocketBuffer <number>
- This transport protocol receiver uses a buffer to enqueue the packets that
the socket is pending to handle. The default size of this socket buffer is
available at /proc/sys/net/core/rmem_default.
This value determines the chances to have an overrun in the receiver queue. The overrun results in packet loss, thus, losing state information that would have to be retransmitted. If you notice some packet loss, you may want to increase the size of the buffer. The system default size is usually around ~100 KBytes which is fairly small for busy firewalls.
Note: The NOTRACK protocol is best effort, it is really recommended to increase the buffer size.
Example: RcvSocketBuffer 1249280
- Checksum <yes|no>
- Enable/Disable message checksumming. This is a good property to achieve fault-tolerance. In case of doubt, use it.
UDP¶
This section indicates to conntrackd(8) to use UDP as transport mechanism between nodes of the firewall cluster.
As in the Multicast configuration, you may especify several fail-over dedicated links using the Default keyword.
Example:
UDP { IPv4_address 172.16.0.1 IPv4_Destination_Address 172.16.0.2 Port 3781 Interface eth3 SndSocketBuffer 1249280 RcvSocketBuffer 1249280 Checksum on }
- IPv4_address <address>
- UDP IPv4 address that this firewall uses to listen to events.
Example: IPv4_address 192.168.2.100
- IPv6_address <address>
- UDP IPv6 address that this firewall uses to listen to events.
Example: IPv6_address fe80::215:58ff:fe28:5a27
- IPv4_Destination_Address <address>
- Destination IPv4 UDP address that receives events, ie. the other
firewall's dedicated link address.
Example: IPv4_Destination_Address 192.168.2.101
- IPv6_Destionation_Address <address>
- Destination IPv6 UDP address that receives events, ie. the other
firewall's dedicated link address.
Example: IPv6_Destination_Address fe80::2d0:59ff:fe2a:775c
- Port <number>
- UDP port used
Example: Port 3780
- Interface <name>
- Same as in the Multicast transport protocol configuration.
- SndSocketBuffer <number>
- Same as in the Multicast transport protocol configuration.
- RcvSocketBuffer <number>
- Same as in the Multicast transport protocol configuration.
- Checksum <yes|no>
- Same as in the Multicast transport protocol configuration.
TCP¶
You can also use Unicast TCP to propagate events.
If you combine this transport with the NOTRACK mode, it becomes reliable.
The TCP transport protocol can be configured in exactly the same way as the UDP transport protocol.
As in the Multicast configuration, you may especify several fail-over dedicated links using the Default keyword.
Example:
TCP { IPv6_address fe80::215:58ff:fe28:5a27 IPv6_Destination_Address fe80::215:58ff:fe28:5a27 Port 3781 Interface eth2 SndSocketBuffer 1249280 RcvSocketBuffer 1249280 Checksum yes }
OPTIONS¶
Other unsorted options that are related to the synchronization protocol or transport mechanism.
- TCPWindowTracking <yes|no>
- TCP state-entries have window tracking disabled by default, you can enable it with this option. As said, default is off. This feature requires a Linux kernel >= 2.6.36.
- ExpectationSync <on|{ list }>
- Set this option on if you want to enable the synchronization of
expectations. You have to specify the list of helpers that you want to
enable.
This feature requires a Linux kernel >= 3.5.
Example, sync all expectations:
ExpectationSync on
Example, sync given expectations:
ExpectationSync { ftp ras q.931 h.245 sip }
By default, this option is disabled.
GENERAL¶
This top-level section contains generic configuration directives for the conntrackd(8) daemon.
- Systemd <yes|no>
- Enable systemd(1) runtime support if conntrackd(8) is
compiled with the proper configuration. Then you can use a service unit of
Type=notify.
Obviously, this requires the init of your system to be systemd(1).
Note: systemd(1) watchdog is supported as well.
Example: Systemd yes
By default runtime support is enabled if conntrackd was built with the systemd feature. Otherwise is off.
- Nice <value>
- Deprecated. Conntrackd ignores this option and it will be removed in the future. Please note that you can run nice(1) and renice(1) externally. Also note that conntrackd(8) now uses by default a RT scheduler.
- HashSize <value>
- Number of buckets in the cache hashtable. The bigger it is, the closer it
gets to O(1) at the cost of consuming more memory. Read some
documents about tuning hashtables for further reference.
Example: HashSize 32768
- HashLimit <value>
- Maximum number of conntracks, it should be double of
/proc/sys/net/netfilter/nf_conntrack_max since the daemon may keep
some dead entries cached for possible retransmission during state
synchronization.
Example: HashLimit 131072
- LogFile <yes|no|filename>
- Enable conntrackd(8) to log to a file.
Example: LogFile no
Default is no. Default logfile is /var/log/conntrackd.log.
- Syslog <yes|no|facility>
- Enable connection logging via Syslog. If you set the facility, use the
same as in the Stats section, otherwise you'll get a warning
message.
Example: Syslog local0
Default is off.
- Lockfile <filename>
- Lockfile to be used by conntrackd(8) (absolute path).
Example: LockFile /var/lock/conntrack.lock
Default is /var/lock/conntrack.lock.
- NetlinkBufferSize <value>
- Netlink event socket buffer size. If you do not specify this clause, the
default buffer size value in /proc/sys/net/core/rmem_default is
used. This default value is usually around 100 Kbytes which is
fairly small for busy firewalls. This leads to event message dropping and
high CPU consumption.
Example: NetlinkBufferSize 2097152
- NetlinkBufferSizeMaxGrowth <value>
- The daemon doubles the size of the netlink event socket buffer size if it
detects netlink event message dropping. This clause sets the maximum
buffer size growth that can be reached.
Example: NetlinkBufferSizeMaxGrowth 8388608
- NetlinkOverrunResync <yes|no|value>
- If the daemon detects that Netlink is dropping state-change events, it
automatically schedules a resynchronization against the Kernel after 30
seconds (default value). Resynchronizations are expensive in terms of CPU
consumption since the daemon has to get the full kernel state-table and
purge state-entries that do not exist anymore.
Note: Be careful of setting a very small value here.
Example: NetlinkOverrunResync yes
The default value is 30 seconds. If not specified, the daemon assumes that this option is enabled and uses the default value.
- NetlinkEventsReliable <yes|no>
- If you want reliable event reporting over Netlink, set on this option. If
you set on this clause, it is a good idea to set off
NetlinkOverrunResync.
You need Linux Kernel >= 2.6.31 for this option to work.
Example: NetlinkEventsReliable yes
This option is off by default.
- PollSecs <seconds>
- By default, the daemon receives state updates following an event-driven
model. You can modify this behaviour by switching to polling mode with
this clause.
This clause tells conntrackd(8) to dump the states in the kernel every N seconds. With regards to synchronization mode, the polling mode can only guarantee that long-lifetime states are recovered. The main advantage of this method is the reduction in the state replication at the cost of reducing the chances of recovering connections.
Example: PollSecs 15
- EventIterationLimit <value>
- The daemon prioritizes the handling of state-change events coming from the
core. With this clause, you can set the maximum number of state-change
events (those coming from kernel-space) that the daemon will handle after
which it will handle other events coming from the network or userspace.
A low value improves interactivity (in terms of real-time behaviour) at the cost of extra CPU consumption.
Example: EventIterationLimit 100
Default (if not set) is 100.
UNIX¶
Unix socket configuration. This socket is used by conntrackd(8) to listen to external commands like `conntrackd -k' or `conntrackd -n'.
Example:
UNIX { Path /var/run/conntrackd.ctl }
- Path <filename>
- Absolute path to the Unix socket.
Example: Path /var/run/conntrackd.ctl
- Backlog <value>
- Deprecated option.
FILTER¶
Event filtering. This clause allows you to filter certain traffic.
There are currently three filter-sets: Protocol, Address and State. The filter is attached to an action that can be: Accept or Ignore. Thus, you can define the event filtering policy of the filter-sets in positive or negative logic depending on your needs.
You can select if conntrackd(8) filters the event messages from user-space or kernel-space. The kernel-space event filtering saves some CPU cycles by avoiding the copy of the event message from kernel-space to user-space. The kernel-space event filtering is preferred, however, you require a Linux kernel >= 2.6.29 to filter from kernel-space.
The syntax for this section is: Filter From <from> { }.
If you want to select kernel-space event filtering, use the keyword Kernelspace instead of Userspace.
Example:
Filter From Userspace { Protocol Accept { TCP SCTP DCCP } Address Ignore { IPv4_address 127.0.0.1 IPv6_address ::1 } State Accept { ESTABLISHED CLOSED TIME_WAIT CLOSE_WAIT for TCP } }
- Protocol <policy> { <protocols list> }
- Accept only certain protocols: You may want to replicate the state of
flows depending on their layer 4 protocol.
Policy is one of Accept or Ignore.
Protocols are: TCP, SCTP, DCCP, UDP, ICMP and IPv6-ICMP.
The ICMP and IPv6-ICMP protocols require a Linux kernel >= 2.6.31.
Example:
Protocol Accept { TCP SCTP DCCP }
- Address <policy> { <addresses list> }
- Ignore traffic for a certain set of IP's: Usually all the IP assigned to
the firewall since local traffic must be ignored, only forwarded
connections are worth to replicate.
Note that these values depends on the local IPs that are assigned to the firewall.
You may specify several IPv4_address and/or IPv6_address directives. You can also specify networks in CIDR format.
Policy is one of Accept or Ignore.
Example:
Address Ignore { IPv4_address 127.0.0.1 # loopback IPv4_address 192.168.0.100 # virtual IP 1 IPv4_address 192.168.1.100 # virtual IP 2 IPv4_address 192.168.100.100 # dedicated link ip IPv4_address 192.168.0.0/24 IPv6_address ::1 }
- State <policy> { <states list> for TCP }
- Filter by flow state. This option introduces a trade-off in the
replication: it reduces CPU consumption at the cost of having lazy backup
firewall replicas.
Note: only affects TCP flows.
The existing TCP states are: SYN_SENT, SYN_RECV, ESTABLISHED, FIN_WAIT, CLOSE_WAIT, LAST_ACK, TIME_WAIT, CLOSED and LISTEN.
Policy is one of Accept or Ignore.
Example:
State Accept { ESTABLISHED CLOSED TIME_WAIT CLOSE_WAIT for TCP }
SCHEDULER¶
Select a different scheduler for the daemon, you can select between RR and FIFO and the process priority.
Using a RT scheduler reduces the chances to overrun the Netlink buffer and conntrackd(8) uses by default RR unless FIFO is selected. See sched_setscheduler(2) for more information.
Example:
Scheduler { Type FIFO Priority 99 }
- Type <type>
- Supported values are RR or FIFO.
Default: RR
- Priority <value>
- Value of the scheduler priority. Minimum is 0, maximum is 99.
Default: 99 (as returned by sched_get_priority_max(2) for SCHED_RR)
STATS¶
This top-level section indicates conntrackd(8) to work as a statistic collector for the nf_conntrack linux kernel subsystem.
- LogFile <yes|no|filename>
- If you enable this option, the daemon writes the information about
destroyed connections to a logfile.
Default is no. Default filename is /var/log/conntrackd-stats.log.
- NetlinkEventsReliable <yes|no>
- If you want reliable event reporting over Netlink, set on this option. If
you set on this clause, it is a good idea to set off
NetlinkOverrunResync. This requires Linux kernel >=
2.6.31.
Default is no.
- Syslog <yes|no|facility>
- Enable connection logging via Syslog. If you set the facility, use the
same as in the General section, otherwise you'll get a warning
message.
Example: Syslog local0
Default is no.
HELPER¶
Note: this configuration is very advanced and has nothing to do with synchronization or stats collection.
This top-level section indicates conntrackd(8) to inject user-space helpers into the nf_conntrack linux kernel subsystem. It will result in the nf_conntrack engine sending connections to userspace for further processing.
Before this, you have to make sure you have registered the given user-space helper stub.
Example:
% nfct add helper ftp inet tcp
Each user-space helper should be registered using a Type section, which are named this way:
Type <name> <af> <transport>
Examples:
Helper { Type ftp inet tcp { QueueNum 0 QueueLen 10240 Policy ftp { ExpectMax 1 ExpectTimeout 300 } } Type rpc inet tcp { QueueNum 1 QueueLen 10240 Policy rpc { ExpectMax 1 ExpectTimeout 300 } } Type rpc inet udp { QueueNum 2 QueueLen 10240 Policy rpc { ExpectMax 1 ExpectTimeout 300 } } Type tns inet tcp { QueueNum 3 QueueLen 10240 Policy tns { ExpectMax 1 ExpectTimeout 300 } } Type dhcpv6 inet6 udp { QueueNum 4 QueueLen 10240 Policy dhcpv6 { ExpectMax 1 ExpectTimeout 300 } } Type ssdp inet udp { QueueNum 5 QueueLen 10240 Policy ssdp { ExpectMax 1 ExpectTimeout 300 } } }
Parameters inside the Type section:
- QueueNum <number>
- Set NFQUEUE number you want to use to receive traffic from the kernel.
Example: QueueNum 0
- QueueLen <number>
- Maximum number of packets waiting in the queue to receive a verdict from
user-space.
Rise value if you hit the following error message:
"nf_queue: full at X entries, dropping packet(s)"
Default is 1024.
Example: QueueLen 10240
- Policy <name> { }
- Set the expectation policy for the given helper.
This sub-section contains 2 directives: ExpectMax <number> (maximum number of simultaneous expectations) and ExpecTimeout <seconds> (maximum living time for one expectation).
COMPLETE EXAMPLES¶
Find below some real-life working examples.
STATS EXAMPLE¶
This configuration example tells conntrackd(8) to work as a stats collector.
Stats { LogFile yes NetlinkEventsReliable no Syslog yes } General { Systemd yes HashSize 8192 HashLimit 65535 Syslog yes LockFile /var/lock/conntrack.lock UNIX { Path /var/run/conntrackd.ctl } NetlinkBufferSize 262142 NetlinkBufferSizeMaxGrowth 655355 Filter { Protocol Accept { TCP UDP } Address Ignore { IPv4_address 127.0.0.1 IPv6_address ::1 } } }
SYNC EXAMPLE 1¶
This example configures synchronization in FTFW mode with Multicast transport.
It includes common general configuration as well.
Note: this is one of the recommended setups for conntrackd(8) in a firewall cluster environment.
Sync { Mode FTFW { ResendQueueSize 131072 PurgeTimeout 60 ACKWindowSize 300 DisableExternalCache no } Multicast { IPv4_address 225.0.0.50 Group 3780 IPv4_interface 192.168.100.100 Interface eth2 SndSocketBuffer 1249280 RcvSocketBuffer 1249280 Checksum yes } Multicast Default { IPv4_address 225.0.0.51 Group 3781 IPv4_interface 192.168.100.101 Interface eth3 SndSocketBuffer 1249280 RcvSocketBuffer 1249280 Checksum yes } Options { TCPWindowTracking no ExpectationSync yes } } General { Systemd yes HashSize 32768 HashLimit 131072 LogFile yes Syslog no LockFile /var/lock/conntrack.lock UNIX { Path /var/run/conntrackd.ctl } NetlinkBufferSize 2097152 NetlinkBufferSizeMaxGrowth 8388608 NetlinkOverrunResync yes NetlinkEventsReliable no EventIterationLimit 100 Filter From Userspace { Protocol Accept { TCP SCTP DCCP } Address Ignore { IPv4_address 127.0.0.1 IPv4_address 192.168.100.0/24 IPv6_address ::1 } } }
SYNC EXAMPLE 2¶
This example configures synchronization in NOTRACK mode with TCP transport.
It includes common general configuration as well.
Sync { Mode NOTRACK { DisableInternalCache yes DisableExternalCache yes } TCP { IPv4_address 192.168.2.100 IPv4_Destination_Address 192.168.2.101 Port 3780 Interface eth2 SndSocketBuffer 1249280 RcvSocketBuffer 1249280 Checksum yes } Options { TCPWindowTracking no ExpectationSync yes } } General { Systemd yes HashSize 32768 HashLimit 131072 LogFile yes Syslog no LockFile /var/lock/conntrack.lock UNIX { Path /var/run/conntrackd.ctl } NetlinkBufferSize 2097152 NetlinkBufferSizeMaxGrowth 8388608 NetlinkOverrunResync yes NetlinkEventsReliable no EventIterationLimit 100 Filter From Userspace { Protocol Accept { TCP SCTP DCCP } Address Ignore { IPv4_address 127.0.0.1 IPv4_address 192.168.0.0/16 IPv6_address ::1 } State Accept { ESTABLISHED CLOSED TIME_WAIT CLOSE_WAIT for TCP } } }
SEE ALSO¶
conntrackd(8), conntrack(8), nfct(8), http://conntrack-tools.netfilter.org/manual.html
AUTHOR¶
Pablo Neira Ayuso wrote and maintains the conntrackd tool.
This manual page was written by Arturo Borrero Gonzalez <arturo@debian.org> based on the conntrackd tarball config examples.
Please send bug reports to <netfilter-devel@lists.netfilter.org>. Subscription is required.
This documentation is free/libre under the terms of the GPLv2+.
January 20, 2021 |