1. Manuals
  2. Leap-16.0
  3. SAPHanaSR-angi
  4. SAPHanaController-scale-out(7)

Scroll to navigation

ocf_suse_SAPHanaController(7) OCF resource agents ocf_suse_SAPHanaController(7)

NAME

SAPHanaController - Manages takeover between two SAP HANA databases with system replication (scale-out).

SYNOPSIS

SAPHanaController [ start | stop | status | monitor | promote | demote | meta-data | validate-all | reload | methods | usage ]

DESCRIPTION

SAPHanaController is an resource agent (RA) for SAP HANA databases. It manages takeover for a SAP HANA database with system replication in an OCF promotable clone configuration. This manual page explains SAP HANA scale-out scenarios. For scale-up, see SAPHanaController-scale-up(7).

System replication will help to replicate the database data from one site to another site in order to compensate for database failures. With this mode of operation, internal SAP HANA high-availability (HA) mechanisms and the Linux cluster have to work together.

An HANA scale-out setup already is, to some degree, an HA cluster on its own. The HANA is able to replace failing nodes with standby nodes or to restart certain sub-systems on other nodes. As long as the HANA landscape status is not "ERROR" the Linux cluster will not act. The main purpose of the Linux cluster is to handle the takeover to the other site. Only if the HANA landscape status indicates that HANA can not recover from the failure and the replication is in sync, then Linux will act. As an exception, the Linux cluster will react if HANA moves the master nameserver role to another candidate. SAPHanaController is also able to restart former failed worker nodes as standby. In addition to the SAPHanaController and SAPHanaTopology RAs, the SAPHanaSR solution uses an "HA/DR provider" API provided by HANA to get informed about the current state of the system replication.

On initial cluster start, the cluster needs to detect a valid HANA system replication setup, including system replication status (SOK) and last primary timestamp (LPT). This is necessary to ensure data integrity.

The SAPHanaController RA performs the actual check of the SAP HANA database instances and is configured as promoatble clone resource. The Linux cluster resources' promoted status will follow the HANA's active master name server role. Resources (e.g. IP address) that need to run on the same node with that HANA role should be co-located with the promoted target. Managing the two SAP HANA instances in scale-up or scale-out setup means that the resource agent controls the start/stop of the instances. In addition the resource agent is able to monitor the SAP HANA databases on landscape host configuration level. For this monitoring the resource agent relies on interfaces provided by SAP. A third task of the resource agent is to also check the synchronisation status of the two SAP HANA databases. If the synchronisation is not "SOK", then the cluster avoids to takeover to the secondary side, if the primary fails. This is to improve the data consistency.

The resource agent uses the following five interfaces provided by SAP:

1. sapcontrol/sapstartsrv
The interface sapcontrol/sapstartsrv is used to start/stop a HANA database instance/system. In scale-out systems, the whole system is started or stopped at once by calling the sapcontrol function StartSystem or StopSystem on the master nameserver node. An exception is made on orphaned worker nodes. SAPHanaController also checks via this interface the instance status to allow a start of HANA standby nodes.

2. landscapeHostConfiguration
The interface is used to monitor an entire HANA system. The python script is named landscapeHostConfiguration.py. landscapeHostConfiguration.py has some detailed output about HANA system status and node roles. For our monitor the overall status is relevant. This overall status is reported by the return code of the script: 0: Internal Fatal, 1: ERROR, 2: WARNING, 3: INFO, 4: OK
The SAPHanaController resource agent will interpret return code 0 as FATAL, 1 as NOT-RUNNING (or ERROR) and return codes 2+3+4 as RUNNING. FATAL means the status can not be determined.
Note: Some conditions cause HANA stopping to work, but not reporting an error. E.g. filesystem filled up.

3. hdbnsutil
The interface hdbnsutil is used to check the "topology" of the system replication as well as the current configuration (primary/secondary) of a SAP HANA database instance. A second task of the interface is the posibility to run a system replication takeover (sr_takeover) or to register a former primary to a newer one (sr_register). In scale-out setups, hdbnsutil sometimes might take some time.

4. systemReplicationStatus
The interface systemReplicationStatus is a special query to check the HANA system replication status via a python library. This interface exists since SAP HANA 1.0 SPS09.

5. saphostctrl
The interface saphostctrl uses the function ListInstances to figure out the virtual host name of the SAP HANA instance. This is typically the hostname used during the HANA installation.

To make configuring the cluster as simple as possible, the additional SAPHanaTopology resource agent runs on all nodes of the Linux cluster and gathers information about the statuses and configurations of SAP HANA system replication. The SAPHanaTopology RA is designed as a normal (stateless) clone.

Please see also the REQUIREMENTS section below and manual page SAPHanaSR-angi-scenarios(7).

SUPPORTED PARAMETERS

This resource agent supports the following parameters:

SID

SAP System Identifier. Has to be same on both instances. Required. Example "SID=SLE".

InstanceNumber

Number of the SAP HANA database. Has to be same on both instances. For system replication also Instance Number+1 is blocked. Required. Example "InstanceNumber=00".

DIR_EXECUTABLE

The full qualified path where to find sapstartsrv and sapcontrol. Specify this parameter, if you have changed the SAP kernel directory location after the default SAP installation.
Optional, well known directories will be searched by default.

DIR_PROFILE

The full qualified path where to find the SAP START profile. Specify this parameter, if you have changed the SAP profile directory location after the default SAP installation.
Optional, well known directories will be searched by default.

HANA_CALL_TIMEOUT

Define timeout how long a call to HANA to receive information can take. This could be e.g. landscapeHostConfiguration.py. There are some specific calls to HANA which have their own timeout values. For example the sr_takeover command does not timeout (inf). If the timeout is reached, the return code will be 124. If you increase the timeouts for HANA calls you should also adjust the operation timeouts of your Linux cluster resources.
Optional. Default value: 120.

INSTANCE_PROFILE

The name of the SAP HANA instance profile. Specify this parameter, if you have changed the name of the SAP HANA instance profile after the default SAP installation. Normally you do not need to set this parameter.
Optional, well known directories will be searched by default.

ON_FAIL_ACTION

Defines how the RA escalates monitor failures on an HANA primary node. If srHook=SOK, in case of monitor failure an node fencing could be triggered. For srHook=SFAIL, the restart will be proceeded as usual. This option may speed up takeover on scale-up systems, depending on how long HANA needs for stopping. For scale-out see also SAPHanaSR-alert-fencing(8). Values: [ proceed | fence ].
- proceed: proceed the failure as usual, i.e. initiate demote-stop sequence.
- fence: trigger stop failure and node fencing, if conditions are matched.
Optional. Default value: proceed.

PREFER_SITE_TAKEOVER

Defines whether RA should prefer to takeover to the secondary database instead of restarting the primary site locally. However a takeover will only be triggered, if the SAP HANA landscape status is on "ERROR". For "FATAL" a local restart is initiated. PREFER_SITE_TAKEOVER usually is choosen for HANA system replication performance-optimised setups. On the other hand local restart of the master instead of takeover could be combined with HANA's persistent memory features. Example: "PREFER_SITE_TAKEOVER=true".
Optional. Default value: false.

DUPLICATE_PRIMARY_TIMEOUT

Time difference needed between two primary time stamps (LPTs), in case a dual-primary situation occurs. If the difference between both node's last primary time stamps is less than DUPLICATE_PRIMARY_TIMEOUT, then the cluster holds one or both instances in a "WAITING" status. This is to give an admin the chance to react on a failover. Note: How the cluster proceeds after the DUPLICATE_PRIMARY_TIMEOUT has passed, depends on the parameter AUTOMATED_REGISTER. See also the examples section below.
Optional. Default value: 7200.

AUTOMATED_REGISTER

Defines whether a former primary database should be registered as secondary automatically by the resource agent during cluster/resource start, if the DUPLICATE_PRIMARY_TIMEOUT condition is met. Registering a database as secondary will initiate a data synchronisation from primary and might overwrite local data. For multi-tier please consider whether the used HANA supports star topology replication. This is important, because an sr_takeover followed by an sr_register of the former primary will convert the multi-tier chain topology into a multi-target star topology. See also REQUIREMENTS section of SAPHanaSR-ScaleOut(7). Example: "AUTOMATED_REGISTER=true".
Optional. Default value: false.

SUPPORTED PROPERTIES

hana_${sid}_glob_filter

Global cluster property hana_${sid}_glob_filter . This property defines which messages are logged by the RA. It should only be set if requested by support engineers. The default is sufficient for normal operation. See also SAPHanaSR-showAttr(8).
Message Types: [ act | dbg | dec | flow | lpa | ra | score | top ]
ACT: Action. Start, stop, sr_takeover and others. See also section SUPPORTED ACTIONS.
DBG: Debugging info. Usually not needed at customer site. See SUSE TID 7022678 for maximum RA tracing.
DEC: Decision taken by the RA.
ERR: Error.
FLOW: Function calls and the respective return codes.
LPA: Last Primary Arbritration. Everything related to the LPT calculation.
RA: Resource Agent messages marking the start of an resource action and the stop with time needed for the action.
SCORE: Everything related to node scoring calculation. See also parameter PREFER_SITE_TAKEOVER.
TOP: Topology. Messages related to HANA SR topology, like site name and remote site.
Optional, advanced. Default value: ra-act-dec-lpa.

hana_${sid}_gra

The node attribute hana_${sid}_gra identifies what generation of the RA is running. The generation should be same on all nodes. See also SAPHanaSR-showAttr(8) and SAPHanaSR-manageAttr(8).
Optional.

SUPPORTED ACTIONS

This resource agent supports the following actions (operations):

start

Starts the HANA instance or brings the "clone instance" to a WAITING status. The correct timeout depends on factors like database size and storage performance. Large databases might require higher start timeouts, use of persistent memory might reduce the timeout needed. Suggested minimum timeout: 3600.

stop

Stops the HANA instance. The correct timeout depends on factors like database size. Starting with SAP HANA 2.0 SPS06, shutdown can be accelerated by optimizing memory de-allocation. See also manual page SAPHanaSR_basic_cluster(7). If HANA database memory de-allocation and internal timeouts have been tuned for fast shutdown, the RA timeout might be reduced. Suggested minimum timeout: 600.

promote

Either runs a takeover for a secondary or a just-nothing for a primary. The correct timeout depends on factors like system replication operation mode and current load on the database. Suggested minimum timeout: 900.

demote

Nearly does nothing and just marks the instance as demoted. Suggested minimum timeout: 320.

status

Reports whether the HANA instance is running. Suggested minimum timeout: 60.

monitor (promoted role)

Reports whether the HANA database seems to be working in replication primary mode. It also needs to check the system replication status. Suggested minimum timeout: 700. Suggested interval: 60.

monitor (demoted role)

Reports whether the HANA database seems to be working in replication secondary mode. It also needs to check the system replication status. The secondary role's monitor interval has to be different from the primary (promoted) role. Suggested minimum timeout: 700. Suggested interval: 61.

validate-all

Reports whether the parameters are valid. Suggested minimum timeout: 5.

meta-data

Retrieves resource agent metadata (internal use only). Suggested minimum timeout: 5.

methods

Reports which methods (operations) the resource agent supports. Suggested minimum timeout: 5.

reload

Changes parameters without forcing a recover of the resource. Suggested minimum timeout: 5.

RETURN CODES

The return codes are defined by the OCF cluster framework. Please refer to the OCF definition on the website mentioned below. In addition return code 124 will be logged if HANA_CALL_TIMEOUT has been exceeded.
In addition, log entries are written, which can be scanned by using a pattern like "SAPHanaController.*RA.*rc=[1-7,9]" for errors. Regular operations might be found with "SAPHanaController.*RA.*rc=0".

EXAMPLES

* Below is an example configuration for a SAPHanaController multi-state resource in an HANA scale-out performance-optimised scenario.
The HANA consists of two sites with five nodes each. An additional cluster node is used as majority maker for split-brain situations. In addition, a SAPHanaTopology clone resource is needed to make this work.

primitive rsc_SAPHanaCon_SLE_HDB00 ocf:suse:SAPHanaController \
op start interval="0" timeout="3600" \
op stop interval="0" timeout="3600" \
op promote interval="0" timeout="900" \
op demote interval="0" timeout="320" \
op monitor interval="60" role="Promoted" timeout="700" \
op monitor interval="61" role="Unpromoted" timeout="700" \
params SID="SLE" InstanceNumber="00" PREFER_SITE_TAKEOVER="true" \
DUPLICATE_PRIMARY_TIMEOUT="7200" AUTOMATED_REGISTER="true" \
HANA_CALL_TIMEOUT="120"

clone mst_SAPHanaCon_SLE_HDB00 rsc_SAPHanaCon_SLE_HDB00 \
meta clone-node-max="1" promotable="true" interleave="true"

location SAPHanaCon_not_on_majority_maker mst_SAPHanaCon_HAE_HDB00 -inf: vm-majority

* The following shows the filter for log messages set to defaults, pacemaker-1.0.
This property should only be set if requested by support engineers. The default is sufficient for normal operation. SID is HA1.

property $id="SAPHanaSR" \
hana_ha1_glob_filter="ra-act-dec-lpa"

* Remove log messages filter attribute from CIB, pacemaker-2.0.
Could be done once a specific filter is not needed anymore.

# SAPHanaSR-showAttr
# crm_attribute --delete -t crm_config --name hana_ha1_glob_filter
# SAPHanaSR-showAttr

* Search for log entries of the resource agent. Show errors only:

# grep "SAPHanaController.*RA.*rc=[1-7,9]" /var/log/messages

* Search for log entries of the resource agent. Show date, time, return code, runtime:

# grep "SAPHanaContoller.*end.action.monitor_clone.*rc=" /var/log/messages | awk '{print $1,$11,$13}' | colrm 20 32 | tr -d "=()rsc" | tr "T" " "

* Show and delete failcount for resource.
Resource is rsc_SAPHanaCon_HA1_HDB00, node is node22. Useful after a failure has been fixed and for testing. See also cluster properties migration-threshold, failure-timeout and SAPHanaController parameter PREFER_SITE_TAKEOVER.

# crm resource failcount rsc_SAPHanaCon_HA1_HDB00 show node22
# crm resource failcount rsc_SAPHanaCon_HA1_HDB00 delete node22

* Manually trigger an SAPHanaController probe action. Output goes to the usual logfiles. Number of nodes is 6, InstanceNr is 00.

# OCF_ROOT=/usr/lib/ocf/ OCF_RESKEY_SID=HA1 OCF_RESKEY_InstanceNumber=00 OCF_RESKEY_CRM_meta_clone_max=6 OCF_RESKEY_CRM_meta_clone_node_max=1 OCF_RESKEY_CRM_meta_interval=0 /usr/lib/ocf/resource.d/suse/SAPHanaController monitor

* Check for working NTP service on chronyd-based systems:

# chronyc sources

* Use of DUPLICATE_PRIMARY_TIMEOUT and Last Primary Timestamp (LPT) in case the primary node has been crashed completely.

Typically on each side where the RA detects a running primary a time stamp is written to the node's attributes (last primary seen at time: lpt). If the timestamps ("last primary seen at") differ less than the DUPLICATE_PRIMARY_TIMEOUT then the RA could not automatically decide which of the two primaries is the better one.

1. nodeA is primary and has a current time stamp, nodeB is secondary and has a secondary marker set:
nodeA: 1479201695
nodeB: 30

2. Now nodeA crashes and nodeB takes over:
(nodeA: 1479201695)
nodeB: 1479201700

3. A bit later nodeA comes back into the cluster:
nodeA: 1479201695
nodeB: 1479202000
You see while nodeA keeps its primary down the old timestamp is kept. NodeB increases its timestamp on each monitor run.

4. After some more time (depending on the parameter DUPLICATE_PRIMARY_TIMEOUT)
nodeA: 1479201695
nodeB: 1479208895
Now the time stamps differ >= DUPLICATE_PRIMARY_TIMEOUT. The algorithm defines nodeA now as "the looser" and depending on the AUTOMATED_REGISTER the nodeA will become the secondary.

5. NodeA would be registered:
nodeA: 10
nodeB: 1479208900

6. Some time later the secondary gets into sync
nodeA: 30
nodeB: 1479209100

* Use of DUPLICATE_PRIMARY_TIMEOUT and Last Primary Timestamp (LPT) in case the the database on primary node has been crashed, but the node is still alive.

Typically on each side where the RA detects a running primary a time stamp is written to the node's attributes (last primary seen at time: lpt). If the timestamps ("last primary seen at") differ less than the DUPLICATE_PRIMARY_TIMEOUT then the RA could not automatically decide which of the two primaries is the better one.

1. nodeA is primary and has a current time stamp, nodeB is secondary and has a secondary marker set:
nodeA: 1479201695
nodeB: 30

2. Now HANA on nodeA crashes and nodeB takes over:
nodeA: 1479201695
nodeB: 1479201700

3. As the cluster could be sure to properly stopped the HANA instance at nodeA it *immediately* marks the old primary to be a register candidate, if AUTOMATED_REGISTER is true:
nodeA: 10
nodeB: 1479201760

4. Depending on the AUTOMATED_REGISTER parameter the RA will also immediately regisiter the former primary to become the new secondary:
nodeA: 10
nodeB: 1479201820

5. And after a while the secondary gets in sync
nodeA: 30
nodeB: 1479202132

* Set parameter AUTOMATED_REGISTER="true". See SUPPORTED PARAMETERS section above for details.

# crm_resource -r rsc_SAPHanaCon_HA1_HDB00 -p AUTOMATED_REGISTER -v true
# crm_resource -r rsc_SAPHanaCon_HA1_HDB00 -g AUTOMATED_REGISTER

FILES

/usr/lib/ocf/resource.d/suse/SAPHanaController
the resource agent
/usr/lib/ocf/resource.d/suse/SAPHanaTopology
the also needed topology resource agent
/usr/lib/ocf/resource.d/suse/SAPHanaFilesystem
the filesystem monitoring resource agent
/usr/lib/SAPHanaSR-angi/
directory with function libraries
/usr/sap/$SID/$InstanceName/exe
default path for DIR_EXECUTABLE
/usr/sap/$SID/SYS/profile
default path for DIR_PROFILE

REQUIREMENTS

For the current version of the SAPHanaController resource agent that comes with the software package SAPHanaSR-angi, the support is limited to the scenarios and parameters described in the manual pages SAPHanaSR(7) and SAPHanaSR-ScaleOut(7).

BUGS

In case of any problem, please use your favourite SAP support process to open a request for the component BC-OP-LNX-SUSE. Please report any other feedback and suggestions to feedback@suse.com.

SEE ALSO

ocf_suse_SAPHanaTopology(7) , ocf_suse_SAPHanaFilesystem(7) , ocf_heartbeat_IPaddr2(8) , SAPHanaSR-showAttr(8) , SAPHanaSR(7) , SAPHanaSR-ScaleOut(7) , SAPHanaSR_maintenance_examples(7) , SAPHanaSR_basic_cluster(7) , SAPHanaSR-ScaleOut_basic_cluster(7) , SAPHanaSR-manageAttr(8) , SAPHanaSR-alert-fencing(8) , chrony.conf(5) , stonith(8) , crm(8)
https://documentation.suse.com/sbp/sap/ ,
https://www.suse.com/support/kb/doc/?id=000019138 ,
http://clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Pacemaker_Explained/s-ocf-return-codes.html ,
http://scn.sap.com/community/hana-in-memory/blog/2015/12/14/sap-hana-sps-11-whats-new-ha-and-dr--by-the-sap-hana-academy ,
http://scn.sap.com/docs/DOC-60334 ,

AUTHORS

F.Herschel, L.Pinne.

COPYRIGHT

(c) 2014 SUSE Linux Products GmbH, Germany.
(c) 2015-2017 SUSE Linux GmbH, Germany.
(c) 2018-2025 SUSE LLC
The resource agent SAPHanaController comes with ABSOLUTELY NO WARRANTY.
For details see the GNU General Public License at http://www.gnu.org/licenses/gpl.html

10 Mar 2025