The FSEvents Monitor¶

The FSEvents monitor, available only on Apple macOS, has no known limitations and scales very well with the number of files being observed. In fact, I observed no performance degradation when testing fswatch observing changes on a filesystem of 500 GB over long periods of time. On macOS, this is the default monitor.

The kqueue Monitor¶

The kqueue monitor, available on any *BSD system featuring kqueue, requires a file descriptor to be opened for every file being watched. As a result, this monitor scales badly with the number of files being observed and may begin to misbehave as soon as the fswatch process runs out of file descriptors. In this case, fswatch dumps one error on standard error for every file that cannot be opened. Beware that on some systems the maximum number of file descriptors that can be opened by a process is set to a very low value (values as low as 256 are not uncommon), even if the operating system may allow a much larger value.

If you are running out of file descriptors when using this monitor and you cannot reduce the number of observed items, either:

-

Consider raising the number of maximum open file descriptors (check your OS documentation).

-

Consider using another monitor.

The inotify Monitor¶

The inotify monitor, available on Linux since kernel 2.6.13, may suffer a queue overflow if events are generated faster than they are read from the queue. In any case, the application is guaranteed to receive an overflow notification which can be handled to gracefully recover. fswatch currently throws an exception if a queue overflow occurs. Future versions will handle the overflow by emitting proper notifications. However, the odds of observing a queue overflow on a default configured mainstream GNU/Linux distribution is very low.

The inotify API provides no information about the user or process that triggered an event, so applications cannot easily distinguish events they trigger themselves from events triggered by other processes. Inotify reports only events triggered through the filesystem API, so it does not catch remote events that occur on network filesystems. Furthermore, pseudo-filesystems such as /proc, /sys, and /dev/pts are not monitorable with inotify.

The inotify API does not report file accesses or modifications that may occur because of mmap(2), msync(2), and munmap(2). It identifies affected files by filename, but by the time the application processes the event, the filename may already have been deleted or renamed. Events are identified via watch descriptors, so applications must cache a mapping, if one is needed, between watch descriptors and pathnames. Be aware that directory renamings may affect multiple cached pathnames.

The inotify API sends events for the direct child elements of a watched directory and it scales pretty well with the number of watched items. For this reason, depending on the number of files to watch, it may sometimes be preferable to watch a common parent directory and filter received events rather than adding a huge number of file watches. Inotify monitoring of directories is not recursive, so additional watches must be created for subdirectories. That can take a significant amount of time for large directory trees.

If monitoring an entire directory subtree, and a new subdirectory is created in that tree or an existing directory is renamed into that tree, be aware that by the time you create a watch for the new subdirectory, new files and subdirectories may already exist inside the subdirectory. Therefore, you may want to scan the contents of the subdirectory immediately after adding the watch, and, if desired, recursively add watches for any subdirectories that it contains.

The event queue can overflow. In this case, events are lost. Robust applications should handle the possibility of lost events gracefully. For example, it may be necessary to rebuild part or all of the application cache.

If a filesystem is mounted on top of a monitored directory, no event is generated, and no events are generated for objects immediately under the new mount point. If the filesystem is subsequently unmounted, events will subsequently be generated for the directory and the objects it contains.

The fanotify Monitor¶

The fanotify monitor, available on Linux when the build system detects the required fanotify headers and symbols, uses file-handle based fanotify events. It is not the default Linux monitor; select it explicitly with:

The fanotify monitor is useful when Linux-specific process attribution is required. Custom format directive %K prints the identifier kind, such as `pid' or `tid', and %P prints the reported process or thread identifier. The monitor property fanotify.process-id accepts `pid' or `tid'. The fanotify.report-pidfd property requests pidfd reporting when supported by the build headers and kernel; the pidfd is exposed through the library event metadata only while the callback is executing.

Fanotify reports only events triggered through the filesystem API, so it does not catch remote events that occur on network filesystems. It also does not report accesses or modifications that may occur because of mmap(2), msync(2), and munmap(2). Directory events are created only if the directory itself is opened, read, and closed, so adding, removing, or changing children of a marked directory does not create events for the monitored directory itself. Fanotify monitoring of directories is not recursive: monitoring subdirectories requires additional marks, and using FAN_CREATE to detect them is racy because events may be lost before a new mark is added. Monitoring mounts or entire filesystems avoids that race. The event queue can also overflow, in which case events are lost.

The fanotify.unlimited-queue and fanotify.unlimited-marks properties request FAN_UNLIMITED_QUEUE and FAN_UNLIMITED_MARKS, respectively. These properties are opt-in because they require matching build headers and the capability expected by the running kernel, typically CAP_SYS_ADMIN, and fanotify initialization fails if the process lacks it.

Fanotify support depends on kernel, C library, filesystem, and permission support for the requested mode. Some filesystems do not support file handles, and some fanotify modes require additional privileges. Use the inotify monitor when broad Linux compatibility is more important than process attribution.

Fanotify directory marks are not equivalent to macOS FSEvents recursive tree watches. With ordinary directory marks, FAN_EVENT_ON_CHILD reports events for immediate children only, not grandchildren or deeper descendants. Recursive fanotify monitoring therefore still marks subdirectories explicitly. Avoiding recursive mark expansion requires fanotify mount or filesystem marks, which have broader scope and different permission constraints.

The Poll Monitor¶

The poll monitor was added as a fallback mechanisms in the cases where no other monitor could be used, including:

-

Operating system without any sort of file events API.

-

Situations where the limitations of the available monitors cannot be overcome (i.e.: observing a number of files greater than the available file descriptors on a system using the kqueue monitor).

The poll monitor, available on any platform, only relies on available CPU and memory to perform its task (besides the stat(2) function). The performance of this monitor degrades linearly with the number of files being watched. The authors' experience indicates that fswatch requires approximately 150 MB or RAM memory to observe a hierarchy of 500.000 files with a minimum path length of 32 characters. A common bottleneck of the poll monitor is disk access, since stat()-ing a great number of files may take a huge amount of time. In this case, the latency should be set to a sufficiently large value in order to reduce the performance degradation that may result from frequent disk access.

How to Choose a Monitor¶

fswatch already chooses the "best" monitor for your platform if you do not specify any. However, a specific monitor may be better suited to specific use cases. Please, read the MONITORS section to get a description of all the available monitors and their limitations.

Usage recommendations are as follows:

-

On macOS, use only the FSEvents monitor (which is the default behaviour).

-

On Linux, use the inotify monitor (which is the default behaviour).

-

On Linux, use the fanotify monitor only when its Linux-specific process attribution or fanotify semantics are required.

-

If the number of files to observe is sufficiently small, use the kqueue monitor. Beware that on some systems the maximum number of file descriptors that can be opened by a process is set to a very low value (values as low as 256 are not uncommon), even if the operating system may allow a much larger value. In this case, check your OS documentation to raise this limit on either a per process or a system-wide basis.

-

If feasible, watch directories instead of watching files. Properly crafting the receiving side of the events to deal with directories may sensibly reduce the monitor resource consumption.

-

If none of the above applies, use the poll monitor. The authors' experience indicates that fswatch requires approximately 150 MB or RAM memory to observe a hierarchy of 500.000 files with a minimum path length of 32 characters. A common bottleneck of the poll monitor is disk access, since stat()-ing a great number of files may take a huge amount of time. In this case, the latency should be set to a sufficiently large value in order to reduce the performance degradation that may result from frequent disk access.

Pruning Recursive Traversal¶

The --prune regexp option defines regular expressions that stop recursive traversal from descending into matching directories. This is different from --exclude: excluded paths are filtered from reported events, while pruned directories and their descendants are skipped during library-side recursive traversal.

Pruning only affects monitors that perform this traversal in fswatch. Native recursive backends may accept --prune, but may not be able to prevent the operating system from traversing matching subtrees in the same way.

Prune expressions use the same case-sensitivity and extended-regular-expression settings as normal path filters.

Pruning is applied only to non-root directories. A command-line root path remains eligible for monitoring even if it matches a prune expression.

For example, the following command recursively watches /home/me without descending into hidden directories:

Other options govern how regular expressions are interpreted:

-

Regular expressions can be extended if option -E is specified.

-

Regular expressions can be case insensitive if option -I is specified.

Basic Usage¶

fswatch syntax is the following:

fswatch will then output change events to standard output. By default, only the affected file name is printed. However, many options are available to format the event record, including:

-

The possibility of adding the event timestamp.

-

The possibility of adding the event mask in both textual and numerical form.

The following command listens for changes in the current directory and events are delivered every 5 seconds:

The following command listens for changes in the current user home directory and /var/log:

Piping fswatch Output to Another Process¶

Very often you wish to not only receive an event, but react to it. The simplest way to do it is piping fswatch output to another process. Since in Unix and Unix-like file system file names may potentially contain any character but NUL (\0) and the path separator (/), fswatch has a specific mode of operation when its output must be piped to another process. When the [-0] option is used, fswatch will use the NUL character as record separator, thus allowing any other character to appear in a path. This is important because many commands and shell builtins (such as read) split words and lines by default using the characters in $IFS, which by default contains characters which may be present (although rarely) in a file name, resulting in a wrong event path being received and processed.

Probably the simplest way to pipe fswatch to another program in order to respond to an event is using xargs:

-

fswatch -0 will split records using the NUL character.

-

xargs -0 will split records using the NUL character. This is required to correctly match impedance with fswatch.

-

xargs -n 1 will invoke command every record. If you want to do it every x records, then use xargs -n x.

-

xargs -I {} will substitute occurrences of {} in command with the parsed argument. If the command you are running does not need the event path name, just delete this option. If you prefer using another replacement string, substitute {} with yours.

On BSD-derived systems, including macOS, the system xargs implementation may limit the size of replacement strings used by -I. If commands fail when processing long path names, consider using a different consumer that reads fswatch's NUL-separated records directly, or install GNU findutils and use gxargs instead of xargs.

Bubbling Events¶

An often requested feature is being able to receive a single event "per batch", instead of receiving multiple events. This use case is implemented by the [-o, --one-per-batch] option which tells fswatch to dump a record containing the number of received events, without any other detail:

This is useful if, for example, you want to respond to change events in a way which is (or can easily be) path-independent (because you are not receiving any event detail) and you prefer to "bubble" events together to reduce the overhead of the command being executed. A typical case is a directory synchronisation job whenever some files change.

Receiving a Single Event¶

Another requested feature is the possibility of receiving a single event and exit. This is most useful when existing scripts processing events include the restart logic of fswatch This use case is implemented by the [-1, --one-event] option: