NAME¶

saptune-note - Note definition files for saptune version 3

DESCRIPTION¶

This man page documents the format of the Note definition files for saptune version 3.

The saptune Note definitions will be installed by the rpm in /usr/share/saptune/notes (the Package Area), but for the execution they will be searched in /var/lib/saptune/working/notes (the Working Area) for the saptune SAP Note definitions or in /etc/saptune/override for customer specific changes to the saptune SAP Note definitions or in /etc/saptune/extra for vendor or customer specific tuning definitions.
The Note definition files use the INI file format.
A comment line starts with #.
Inline comments are possible and start with a blank followed by a # followed by any non-# character. (' #anything except #')
If you need to add a character sequenze of ' #anything' to your description, please write it as ' ##anything' (double the #) to give saptune the chance to distinguish this text from an inline comment. saptune will strip the second # from the output result, so that the resulting text will appear as ' #anything'.
Lines starting with '[' indicate the begin of a new section.

SECTIONS¶

A section starts with a '[section_name]' keyword in the first line, followed by lines with options and comments.

It is possible to limit the scope of a section (e.g. to a special hardware architecture and/or a special os version, a special block device hardware or a cloud service provider) by adding tags to the section definition.
Tags are separated by ':' and can appear in any order. Unsupported tags are considered an error and the whole section is ignored.
It is possible to use more than one tag per section. The tags are concatenated by AND and not OR. That means using the same tag multiple times with different values inside a section will not work. The whole section will be ignored.
In such a case split your section into 2 (or more) consecutive sections.
For some tags the value of the tag is treated as a string and used as a regular expression to match the content of the respective source. This is mentioned in the description of the possible tags below. As a reference https://golang.org/pkg/regexp/#MatchString can be used to see, what additional expressions may be possible inside the tag value. But attention, only some basic ones are really supprted (like 'sd[ab]' for block devices).

Attention: as the IBM Power platform (hardware architecture 'ppc64le') does not support files in /sys/class/dmi (the directory is not available on this hardware architecture) some of the tags listed below are not usable on Power systems for now.

The syntax for such sections is:

[section_name:[tag=value]...]

Supported tags are:

os=<os_version>

to define a special os version
Valid values for os= must match the value from the VERSION= line of /etc/os-release, e.g. 12-SP5, 12, 15, 15-SP5 or 16.0

If VERSION= contains spaces, only the string until the first space is used for comparison. This excludes any suffixes - like for beta versions - from the comparison.

To mark an entire major release the wildcard * in combination with a separator - or . (regardless of the one present in VERSION) can be used. Valid patterns are: 12-*, 12.*, 15-*, 15.*, 16-* or 16.*

Additional there is the possibility to use a list or a range of valid service packs or dot releases. Again the separator can be - or . (regardless of the one present in VERSION). And for service packs (in 12/15) the SP prefix should be omitted.
e.g 15-[2,4-5,6-,-7] or 15.[2,4-5,6-,-7]

arch=<hardware_architecture>

to define a special hardware architecture:
Valid values for arch= are the output from uname -i

pmu_name=<CPUplatform>

to select a special CPU platform:
Valid values for pmu_name= are the content from /sys/devices/cpu/caps/pmu_name. So this tag is an Intel CPU only feature for now.
The tag is not limited to the cpu section, it can be used in any section

vendor=<hardwarevendor>

to define a special system hardware vendor
Valid values for vendor= are the values found in /sys/class/dmi/id/board_vendor or parts of the string available in this file.
The value of the vendor tag is used as a regular expression '.*<value>.*' to match the content of the vendor file.

For IBM Power platform (hardware architecture 'ppc64le') the value is set to 'IBM'.

model=<hardwaremodel>

to define a special system hardware model
Valid values for model= are the values found in /sys/class/dmi/id/product_name or parts of the string available in this file.
The value of the model tag is used as a regular expression '.*<value>.*' to match the content of the model file.

On IBM Power platform (hardware architecture 'ppc64le') the file /sys/firmware/devicetree/base/model is used to identify the hardware model.

csp=<cloudserviceprovider>

to define a special cloud service provider
Valid values for csp= are azure, aws, google and ibmVPC

virt=<virtualizationtype>

to define a special virtualization class or type
Valid class values for virt= are vm, chroot, container and bare-metal.
Valid type values for virt= are the output reported by /usr/bin/systemd-detect-virt. If no virtualization is detected it's concidered as bare-metal

DMIinterfacetag:<filename>=<filecontent>

Additional every filename from /sys/class/dmi/id/ can be used as a tag.
This way it's possible to restrict a section to a special hardware vendor or hardware model regarding the bios, the chassis, the board or anything else provided by the DMI interface.
The value of the DMI interface tag is used as a regular expression '.*<value>.*' to match the content of the file in /sys/class/dmi/id/<filename>.

Attention: if the pattern does not match any filename in /sys/class/dmi/id, the section will be skipped as unknown.

Attention: not usable on IBM Power platform (hardware architecture 'ppc64le')

For the following 3 tags we have special rules:
This tags have different effects regarding the section they are used. In all sections except the [block] section they are used the same way all other tags are used to restrict the section to be only valid on a system which includes this special sort of block device.
But used in the [block] section they mean that all the settings available in this section will only apply to block devices, which match the tag value. So you can restrict the settings to dedicated block devices.

blkvendor=<blockdevicehardwarevendor>

to define a hardware vendor to match a special block device
Valid values for blkvendor= are the values found in /sys/block/<dev>/device/vendor or parts of the string available in this file.
The value of the blkvendor tag is used as a regular expression '.*<value>.*' to match the content of the vendor file.

blkmodel=<blockdevicehardwaremodel>

to define a hardware model to match a special block device
Valid values for blkmodel= are the values found in /sys/block/<dev>/device/model or parts of the string available in this file.
The value of the blkmodel tag is used as a regular expression '.*<value>.*' to match the content of the model file.

Attention: if the tag does not match any block device, no block device will be handled by saptune. e.g. 'saptune note verify' will list NO block devices.

blkpat=<pattern>

to define a pattern to match a special block device in /sys/block/

Attention: if the pattern does not match any block device, no block device will be handled by saptune. e.g. 'saptune note verify' will list NO block device. There will be NO Warning or Error message logged in such a case.

For processing a section the following rules apply:

•

Only sections that match the system are processed. Sections without a tag are always used.

•

The order of the section within the file matter. Eache section and each line in a section gets processed from top to down.

The rules apply for shipped Note definition files as well as for customer defined Note definition files. Tagged sections can be used in override files.

ATTENTION: To be clear - if there are more sections with the same section_name containing the same parameters with different values, the last valid section will win.

So it's all about order.

The following section definitions are available and used in the saptune SAP Note definition files. Each of these sections can be used in a vendor or customer specific Note definition file placed in /etc/saptune/extra.

List of supported sections:
version, block, cpu, filesystem, grub, limits, login, mem, pagecache, reminder, rpm, service, sysctl, sys, vm

See detailed description below:

[version]¶

This section is a mandatory section and is used to display version, description and last change date of the underlying Note during saptune action 'list'.

New Syntax:

VERSION=<versionNo>


DATE=<release date of used note and related values>


DESCRIPTION=<description of the note>


REFERENCES=<list of URLs containing information regarding the Note separated by blank>
Example:


VERSION=5


DATE=16.04.2019


DESCRIPTION=VIP: this is VIP Note 1, which contains Very Important Parameters


REFERENCES=https://inter.net.addr.com/path/Note_Info http://inter.net.addr.com/another_path/A_second_Note_Info
The entries are treated as 'Key Value' pairs. The equal operator (=) is mandatory, but can be used with spaces around. The entries can be placed in any order inside the version section.
We skipped the fields 'CATEGORY' and '<noteId>' from the old syntax because these values are not in use. The '<noteId>' was always taken from the filename of the Note definition file and we stay with this behaviour for now.
VERSION is a number that should indicate how many changes are done for this Note definition in the past. Allowed are digits, upper-case and lower-case letters, dots, underscores, minus and plus signs.
DATE is the date of the last changes.
DESCRIPTION is the description of the Note, which will be displayed during the action 'saptune note list'.
REFERENCES is a list of URLs separated by blank, which contain additional information about the Note definition and the content. If you need to use a 'blank' inside the URL definition please mask it as '%20'.

[block]¶

The settings of the "[block]" section will be set on all block devices found in /sys/block, which are considered as valid.

The following rules apply for valid devices:

•

all multipath devices (dm-*, if mpath-, but not LVM- or other dm-)

•

all physical disks (indicated by device/type=0 or names like nvme*, vd*; for nvme* devices additional check for existence of file 'queue/scheduler' and 'queue/nr_requests')
except they are part of a device mapper construct (like mpath-).

The section "[block]" can contain the following options:

IO_SCHEDULER=STRING

The default I/O scheduler for single-queued block layer devices offers satisfactory performance for wide range of I/O task, however choosing an alternative scheduler may potentially yield better latency characteristics and throughput. "noop" is an alternative scheduler, in comparison to other schedulers it may offer more consistent performance, lower computation overhead, and potentially higher throughput. For most SAP environments (RAID, storage arrays, virtualizaton) 'noop' is the better choice.
With the new introduced multi-queue scheduler for block layer devices the recommended I/O scheduler is 'none' as an equivalent to 'noop' for single-queued block layer devices.

So IO_SCHEDULER can now contain a comma separated list of possible schedulers, which are checked from left to right. The first one which is available in /sys/block/<device>/queue/scheduler will be used as new scheduler setting for the respective block device.
The selection per device is logged.
When set, all block devices on the system will be switched to one of the chosen schedulers.
Valid values can be found in /sys/block/<device>/queue/scheduler.

NRREQ=INT

IO nr_requests specifies the maximum number of read and write requests that can be queued at one time. The default value is 128, which means that 128 read requests and 128 write requests can be queued before the next process to request a read or write is put to sleep.
When set, the number of requests for all block devices on the system will be switched to the chosen value

READ_AHEAD_KB=INT

disk readahead (queue/read_ahead_kb) defines the maximum number of kilobytes that the operating system may read ahead during a sequential read operation. As a result, the likely-needed information is already present within the kernel page cache for the next sequential read, which improves read I/O performance. Device mappers often benefit from a high read_ahead_kb value. Increasing the read_ahead_kb value might improve performance in environments where sequential reading of large files takes place.
When set, the value of read_ahead_kb for all block devices on the system will be switched to the chosen value

MAX_SECTORS_KB=INT

disk max_sectors_kb (queue/max_sectors_kb) defines the maximum number of kilobytes that the block layer will allow for a filesystem request. Must be smaller than or equal to the maximum size allowed by the hardware (queue/max_hw_sectors_kb).
When set, the value of max_sectors_kb for all block devices on the system will be switched to the chosen value.
If the value is higher than 'max_hw_sectors_kb' it will be limited to 'max_hw_sectors_kb' and a footnote is displayed.

[cpu]¶

The section "[cpu]" manipulates files in /sys/devices/system/cpu/cpu*.
This section can only contain the following options:

energy_perf_bias=STRING

Energy Performance Bias EPB (applies to Intel-based systems only)
supported values are: performance (0), normal (6) and powersave (15)
The command 'cpupower set -b <value>' is used to set the value, if the system supports Intel's performance bias setting. See cpupower(1) and cpupower-set(1) for more information.
If system does not support Intel's performance bias setting - 'all:none' is used in the column 'Actual' of the verify table and the footnote '[1] setting is not supported by the system' is displayed.

When set as 'energy_perf_bias=<performance|normal|powersave> in the Note definition file, the value will be set for all available CPUs.
The command 'cpupower -c all set -b <value>' or 'cpupower -c <cpu> set -b <value>' is used to set the value.

governor=STRING

CPU Frequency/Voltage scaling (applies to Intel-based systems only)
The clock frequency and voltage of modern CPUs can scale, in order to save energy when there's less work to be done. However HANA as a high-performance database benefits from high CPU frequencies.
supported values are: performance (0), normal (6) and powersave (15)
The command 'cpupower frequency-set -g <value>' is used to set the value, if the value is a supported governor listed in /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor' See cpupower(1) and cpupower-frequency-set(1) for more information.
If the governor settings of all available CPUs are equal, 'all:<governor>' is used in the column 'Actual' of the verify table. If not, each CPU with its assigned governor is listed (e.g. cpu1:powersave cpu2:powersave cpu3:powersave cpu4:powersave cpu5:powersave cpu6:powersave cpu7:powersave cpu0:performance)

When set as 'governor=<performance|powersave> in the Note definition file, the value will be set for all available CPUs.
The command 'cpupower -c all frequency-set -g <value>' or 'cpupower -c <cpu> frequency-set -g <value>' is used to set the value.

force_latency=STRING

force latency - configure C-States for lower latency (applies to Intel-based systems only)
Input is a string, which is internally treated as a decimal (not a hexadecimal) integer number representing a maximum response time in microseconds.
It is used to establish a latency upper limit by limiting the use of C-States (CPU idle or CPU latency states) to only those with an exit latency smaller than the value set here. That means only those states that require less than the requested number of microseconds to wake up are enabled, all the other C-States are disabled.
The files /sys/devices/system/cpu/cpu*/cpuidle/state*/latency and /sys/devices/system/cpu/cpu*/cpuidle/state*/disable are used to limit the C-States.

If system does not support force latency settings - 'all:none' is used in the column 'Actual' of the verify table and the footnote '[1] setting is not supported by the system' is displayed.

When set in the Note definition file for all available CPUs all CPU latency states with a value read from /sys/devices/system/cpu/cpu*/cpuidle/state*/latency >= (higher than) the value from the Note definition file are disabled by writing '1' to /sys/devices/system/cpu/cpu*/cpuidle/state*/disable

ATTENTION: not idling *at all* increases power consumption significantly and reduces the life span of the machine because of wear and tear. So do not use a too strict latency setting. For SAP HANA workloads a value of '70' microseconds (as a "light sleep") seems to be sufficient. And the impact on power consumption and life of the CPUs is less severe. But don't forget: The deeper the idle state, the larger is the exit latency.

[filesystem]¶

The section "[filesystem]" is checking filesystem mount options.
The values from the Note definition files are only checked against /proc/mounts and /etc/fstab. Changing the filesystem mount options is not supported by saptune.

This section can only contain the following parameter:

xfs_options=STRING

where STRING is a list of valid mount options separated by ','
A prefix '-' for the option indicates, that the option should NOT be available on any 'xfs' filesystem. A prefix '+' or no prefix for the option indicates, that the option should be available on any 'xfs' filesystem.

For the check first the mounted filesystems of the requested filesystem type (for now only 'xfs') will be read from /proc/mounts and separated in a list with mount points containing the option and another list with mount points NOT containing the option.
Then the defined filesystems of the requested filesystem type (for now only 'xfs') will be read from /etc/fstab, skipping the already mounted mount points and split the remaining entries in a list with mount points containing the option and another list with mount points NOT containing the option.
At least combine the lists from proc and fstab to get one list of mount points containing the option and another list with mount points NOT containing the option.

To decide, if a mount point contains the option or not, we use a simple string comparison between the value from the Note definition file and the option available on the system.
This can lead to a not-compliant result even everything is in order!
Because default options might not appear in the output of /proc/mounts, they are not found even if they are set internally. Also the content of the the 'defaults' entry of not-mounted filesystem is opaque.
Keep this in mind when crafting overrides or extra Notes!

[grub]¶

The section "[grub]" is checking kernel command line settings for grub. The values from the Note definition files are only checked against /proc/cmdline. Changing the grub configuration is not supported by saptune.

Some of these values are set by 'alternative' settings by saptune during runtime, so changing the grub configuration is possible but not needed.

This section can contain options like:

intel_idle.max_cstate=1 and processor.max_cstate=1

Configure C-States for lower latency in Linux (applies to Intel-based systems only) - see force_latency in section [cpu] as 'alternative' settings

numa_balancing=disable

Turn off autoNUMA balancing - see kernel.numa_balancing in section [sysctl] as 'alternative' settings

transparent_hugepage=never

Configure transparent hugepages - see THP in section [vm] as 'alternative' settings

[mem]¶

The section "[mem]" manipulates the size of TMPFS (/dev/shm).

With the STD implementation, the SAP Extended Memory is no longer stored in the TMPFS (under /dev/shm). However, the TMPFS is required by the Virtual Machine Container (VMC). For this reason, we still recommend the same configuration of the TMPFS:
75% (RAM + Swap) is still recommended as the size.
This section can contain the following options:

ShmFileSystemSizeMB=INT

Use ShmFileSystemSizeMB to set an absolute value for your TMPFS.
If ShmFileSystemSizeMB is set to a value > 0, the setting for VSZ_TMPFS_PERCENT will be ignored and the size will NOT be calculated.
If ShmFileSystemSizeMB is set to '0' the size will be calculated using VSZ_TMPFS_PERCENT

VSZ_TMPFS_PERCENT=INT

Size of tmpfs mounted on /dev/shm in percent of the virtual memory.
Depending on the size of the virtual memory (physical+swap) the value is calculated by (RAM + SWAP) * VSZ_TMPFS_PERCENT/100
If VSZ_TMPFS_PERCENT is set to '0', the value is calculated by (RAM + SWAP) * 75/100, as the default is 75.

As this parameter is only used to calculate the value of ShmFileSystemSizeMB it will not be checked and compared during the saptune operation 'verify'. A footnote is pointing this out.

[reminder]¶

The section "[reminder]" contains important information and all settings of a SAP Note, which can not set by saptune.

This section is displayed at the end of the saptune options 'verify' and 'apply'. It will be highlighted with red color to get the attention of the customer.

Sometimes this section may include lines with parameter settings commented out as the SAP Note only contains rough estimations as the settings are highly customer environment and workload dependend. Please be aware that these parameter settings can't be activated by an override file. If you need to set such parameters you need to create a 'custom' note containing these settings by using 'saptune note create'

[rpm]¶

The section "[rpm]" is checking rpm versions on the system. The values from the Note definition files are only checked against the installed rpm versions on the system. No other action is supported.
Only lower versions installed on the system are considered as non-compliant. A installed rpm version equal or higher is considered as compliant.
Package dependencies - if needed - are handled by the saptune package installation.

With the availability of tagged sections, we support 2 different types of rpm line syntax. The first one - our Old Syntax - only for compatibility reasons. The second one - our New Syntax - is our preferred syntax in combination with tagged rpm sections.

Old Syntax:
<rpm package name> <SLE Version> <rpm package version>
this syntax is mainly used for compatibility reasons and when using a 'non-tagged' rpm section.
Add one line for each SLE version a package should be checked for, even if the package version is the same.
The SLE version has to be noted in the same format as the 'VERSION=' entry in /etc/os-release.

To address all SLE versions and service packs the keyword 'all' can be used instead of a dedicated SLE version.

e.g
systemd 12-SP2 228-142.1
sapinit-systemd-compat 12 1.0-2.1
util-linux 16.0 2.40.4-160000.1.3
bzip2 all 1.0.8

Only the lines where the SLE version is matching the running system OS are checked and displayed during the 'verify' option.
Entries with non-matching SLE versions are reported in the saptune log file as INFO
That means, if there is no matching SLE version for the running OS no rpm entries are listed during the 'verify' operation.

New Syntax:
<rpm package name> <rpm package version>
this syntax is the preferred syntax when using a 'tagged' rpm section, where the targeted operating system and/or system architecture is defined by using the tags os= and/or arch=
Add one line for each package and package version to be checked.

e.g
systemd 228-142.1
util-linux 2.25-22.1

Only the lines where the tags of the section match the running system OS and/or the system architecture are checked and displayed during the 'verify' option.
That means, if there is no matching SLE version for the running OS and/or no matching system architecture in the tags of the rpm section no rpm entries are listed during the 'verify' operation.

[service]¶

The section "[service]" is dealing with starting, enabling, disabling and stopping services controlled by systemd.
The syntax for the entries are:

<servicename>=STRING

where STRING is a list of valid values separated by ',', which are checked from left to right. The first entry of the pair 'start'/'stop' or 'enable'/'disable' will be used as new settings for the service.
Valid services are those listed by the command 'systemctl list-unit-files'.
Valid values are 'start' or 'stop', 'enable' or 'disable'.

Exceptions and Warnings:

For the service uuidd.socket only 'start' is a valid value, because the uuidd.socket service is essential for a working SAP environment.

Concerning sysstat.service please be in mind: A running sysstat service can effect the system performance. But if there are real performance trouble with the SAP system, SAP service normally orders the sysstat reports collected in /var/log/sa.
See sar(1), sa2(8), sa1(8) for more information

If a service is enabled or disabled by default or admin choice, saptune will NOT disable or enable this service, if only 'start' or 'stop' is used. In this case it will only start/stop the service. If such a service is started by systemd during a system reboot after the start of saptune.service it will be possible that a service is stopped/running even if it was started/stopped by saptune. To change this, the service can be additional enabled or disabled by using 'enable' or 'disable' in the Note definition or Override file.

[sysctl]¶

The section "[sysctl]" can be used to modify kernel parameters. The parameters available are those listed under /proc/sys/.
Please write the section keyword '[sysctl]' in the first line and add the desired tunables in 'sysctl.conf' syntax.

sysctl.parameter=VALUE

There will be a detection of conflicting (system) sysctl entries.
When parsing the section '[sysctl]' in the Note definition file saptune additional collects all defined sysctl settings (parameter and value) availabel in "/etc/sysctl.conf", "/run/sysctl.d/", "/etc/sysctl.d/", "/usr/local/lib/sysctl.d/", "/usr/lib/sysctl.d/", "/lib/sysctl.d/", "/boot/" (list retrieved from the comment in /etc/sysctl.conf and man page sysctl.conf(5)). When this file list contains a directory (like /etc/sysctl.d/) the files located in this directory are read too.
saptune will now check, if the parameters from the section '[sysctl]' in the Note definition file are additional defined in one or more of the (system) sysctl config files. If yes, a warning is displayed and logged and a footnote will be prepared for the 'saptune verify' output. The info shown is the filename, where the parameter is additionally defined with it's value in brackets.
The central saptune configuration file contains a parameter SKIP_SYSCTL_FILES, which contains a comma separated list of sysctl.conf files or directories containing sysctl.conf files, which should be excluded from this warning message and the footnote.
Default is SKIP_SYSCTL_FILES="/boot" to skip the WARNINGS for /boot/sysctl.conf-<kernelversion>
The parameter can be adapted by 'saptune configure SKIP_SYSCTL_FILES <newValue>'

Hint: At the moment links are not recognized. So the linked files will be added both in the file list.

[sys]¶

The section "[sys]" can be used to modify parameters available under /sys/, if the related file is writable.
The syntax for the sys.parameters is following the 'sysctl.conf' syntax. So it's the absolute filename without the prefixed /sys/ and all remaining '/' exchanged by '.'
e.g. /sys/module/watchdog/parameters/open_timeout should be written as module.watchdog.parameters.open_timeout

sys.parameter=VALUE

ATTENTION: saptune is NOT validating the value before trying to apply.

[vm]¶

The section "[vm]" manipulates /sys/kernel/mm switches.
This section can to contain the following options:

THP=STRING

This option configures transparent hugepages by changing /sys/kernel/mm/transparent_hugepage/enabled
Possible values are 'never' to disable, 'always' to enable and 'madvise'.

KSM=INT

Kernel Samepage Merging (KSM). KSM allows for an application to register with the kernel so as to have its memory pages merged with other processes that also register to have their pages merged. For KVM the KSM mechanism allows for guest virtual machines to share pages with each other. In today's environment where many of the guest operating systems like XEN, KVM are similar and are running on same host machine, this can result in significant memory savings, the default value is set to 0.

FILES¶

/usr/share/saptune/notes

/var/lib/saptune/working/notes

/var/lib/saptune/staging/latests

/etc/saptune/extra

/etc/saptune/override

SEE ALSO¶

saptune-solution(5) saptune(7) saptune(8)

AUTHOR¶

Soeren Schmidt <soeren.schmidt@suse.com>, Angela Briel <abriel@suse.com>