BioSolveIT GmbH - knowledge base :: How To: Manage License Server Redundancy

Knowledgebase Home | Glossary | Favorites | Support

Search the Knowledgebase		Browse by Category

[Advanced Search]

Knowledgebase Home > Licenses

How To: Manage License Server Redundancy

Would you like to...

Print this page

Email this page

Post a comment

Subscribe me

Add to favorites

User Opinions

100%

(5 votes)

Ensuring License Availability

You can configure multiple license servers to allow FlexEnabled applications to continue to check our licenses if one of the license servers goes down. This failover protection for license servers can be provided using either of the following methods:
•     Redundancy using the license search path: configure and maintain multiple independent license servers, each with a subset of the total licenses available to the enterprise. Configure the FlexEnabled client with the license servers in the license search path. This provides load balancing capabilities and limited failover protection. You must manage different versions of the license rights on each license server. This configuration option is available when licenses are held in license files and in trusted storage.
•     Three-server redundancy: configure and maintain a set of three license server systems configured specifically for three-server redundancy. This provides failover protection only. You manage only one version of the license file and vendor daemon on all three license servers. This configuration option is only available when licenses are held in license files.

Do not store your license files on a single network file server (separate from the license servers) if you are using either of these methods of failover protection: The failure of the file server will cause all the license servers to fail.

Redundancy Using the License Search Path

In this configuration you install multiple license servers that each use a subset of the available licenses. Network machines are configured with a license search path that contains details of each license server. A FlexEnabled application tries each license server on the license search path in order until it succeeds or gets to the end of the list.

Example of Redundancy Using the License Search Path

This example demonstrates the use of two license servers, chicago and tokyo, that serve five licenses each for the features f1 and f2. The publisher supplies the following license files:
•     For chicago
    SERVER chicago 17007ea8 1700
    VENDOR sampled /etc/mydaemon
    FEATURE f1 sampled 1.000 01-jan-2010 5 SIGN=.....
    FEATURE f2 sampled 1.000 01-jan-2010 5 SIGN=.....
•      For tokyo
    SERVER tokyo 17007ea8 1700
    VENDOR sampled /etc/mydaemon
    FEATURE f1 sampled 1.000 01-jan-2010 5 SIGN=.....
    FEATURE f2 sampled 1.000 01-jan-2010 5 SIGN=.....

The license search path is set on the network machines using the LM_LICENSE_FILE environment variable so that machines in the US request licenses first from the license server chicago and machines in Japan request licenses first from the license server tokyo.
•      US machines set LM_LICENSE_FILE to - 1700@chicago:1700@tokyo
•      Japanese machines set LM_LICENSE_FILE to - 1700@tokyo:1700@chicago
This example uses Unix syntax (:) for separating entries on the license search path. See Setting the License Search Path using an Environment Variable for full details of the license search path syntax.

Limitations of Redundancy Using the License Search Path

The main limitation is that this method only provides limited protection: When a license server fails, the licenses it serves are no longer available.

All licenses must be checked out from a single server

By default, once a FlexEnabled application has successfully checked out a license from a license server, all subsequent license requests from that application must be served by the same license server. When an application makes subsequent license requests and no more licenses are available from that license server, the license request is denied even though licenses may exist on another server. However, this behavior is configurable by software publishers. Contact your software publisher to determine whether or not each new license request scans all the license servers.

Licenses are queued from a single server

If the application supports license queuing, all licenses are queued from the first license server on the list rather than the request moving to another license server.


Overview of Three-Server Redundancy

Using the three-server redundancy capability in FlexNet Publisher, all three license servers operate to form a triad. The license servers send periodic messages to each other to make sure that at least two servers are running and communicating. A quorum is formed when at least two of the three license servers are running and communicating with each other.

The license servers are identified as either primary, secondary, or tertiary. One license server is also designated as the master [m] and is responsible for:
•     serving licenses to FlexEnabled applications
•     recording information into the debug log.
•     recording information into the report log.
If the master fails, then another license server becomes the master. In the following figure, the primary license server is the master [m]. When a FlexEnabled application sends a checkout request for a license, the master responds and then serves the license to the FlexEnabled application.

Configuring License Servers for Three-Server Redundancy

If the master fails, then the secondary license server becomes the master (see the following figure) and will server licenses to FlexEnabled applications. The tertiary license server can never be the master. If both the primary and secondary license servers go down, licenses are no longer served to FlexEnabled applications. The master will not serve licenses unless there are at least two license servers in the triad running and communicating.

Understanding How License Servers Communicate

When started, each license server reads the license file and checks that it can communicate with the other license servers. Until each license server establishes this first connection with the others, it will continue to send messages periodically. Once the initial communication has been established, each license server periodically sends a heartbeat to the others. Heartbeats are messages sent over TCP/IP Each license server sends a heartbeat and waits for a response from the other license servers. If a license server does not receive a response, it shuts down the vendor daemon so that it cannot serve licenses. A publisher or license administrator can configure the amount of time a license server waits to receive a heartbeat using the HEARTBEAT_INTERVAL property. Poor network communication causes system performance to slow. Slow network communication can also cause a delay in the transmission of heartbeats between license servers.

Both the software publisher and the license administrator must perform certain configuration steps. This section describes the steps that each must perform.

Configuration for License Administrators

The license administrators should perform the following steps:
1.    Before the license administrator gets the license server software package, they should identify and set up the three systems. When selecting systems, make sure they are stable. Do not use systems that are frequently rebooted or shut down.
2.    Send the publisher the hostname and hostid values for these systems. Ask the publisher what system identifier they need for the hostid. This could be an Ethernet address, disk serial number, etc. The publisher will create license server components specifically for these systems.
3.    After receiving the license server package from the publisher, change the following SERVER line properties in the license file if necessary:
    •     port number the license servers uses to listen for communication
    •     PRIMARY_IS_MASTER keyword
    •     HEARTBEAT_INTERVAL property
Do not change the hostid values. If the hostid changes at any time, the license administrator must work with the software publisher to obtain a new license file.
4.    Perform any additional configuration as required by the software publisher.
5.    Copy or install the license server software package to each of the three systems.
6.    Start the license servers in the following order: primary, secondary, and then tertiary.

An Example License File

The following is an example of a license file that is configured for three-server redundancy.
SERVER pat 17003456 2837 PRIMARY_IS_MASTER
SERVER lee 17004355 2837
SERVER terry 17007ea8 2837
VENDOR demo
FEATURE f1 demo 1.0 1-jan-2018 10 SIGN=”<...>”
FEATURE f2 demo 1.0 1-jan-2018 10 SIGN=”<...>”

The following portions of the license file directly affect the three-server redundant configuration:
•     SERVER lines: These three lines define each of the systems involved.
    •     The host values: they are: pat, lee, and terry.
    •     The hostid values: they are: 17003456, 17004355, and 17007ea8. This example uses the value returned by the lmhostid utility default hostid type. The default hostid type is different for every platform.
•     The TCP/IP ports: All servers use the same port (2837, in the example) to listen for communication.
The following properties of the license file do not affect the three-server redundant configuration directly, but are used to define license rights or configure the license server.
•     VENDOR line: this is required and references the publisher’s vendor daemon.
•     FEATURE lines: The two features, f1 and f2, define the license rights. The SIGN value for each FEATURE line encodes the license server hostid values.

Using the lmstat Utility

The output message generated by the lmstat utility identifies which license server is the master. In the following example lmstat output, the secondary license server is the master.

[Detecting lmgrd processes...]
License server status: 30000@RMD-PRIMARY,30000@RMD-SECONDARY,
30000@RMD-TERTIARY
License file(s) on RMD-PRIMARY: C:\server\3.lic:
RMD-PRIMARY: license server UP v11.4
RMD-SECONDARY: license server UP (MASTER) v11.4
RMD-TERTIARY: license server UP v11.4

Starting and Stopping License Servers

To start the entire system, you must start each license server manager (lmadmin or lmgrd). Generally, it is good practice to start the primary license server before the secondary or tertiary license server. This allows the primary license server to become the master before the others start. If you start the secondary and tertiary before the primary, then the secondary will establish itself as master. If you do not set the PRIMARY_IS_MASTER keyword for the primary license server, then the order in which you start the license servers is important. If you do not set this property, when you start the primary license server after the secondary license server control will not transfer to the primary license server. By setting the PRIMARY_IS_MASTER keyword, you ensure that when the primary license server is running, it is always the master.
The lmdown utility will shut down all three license servers using a single command. You do not have to shut down each license server separately.
Running the License Server Manager as a Service on Windows. There are no dependencies or known issues related to running the license server manager as a service in this configuration.

Logging and the Debug Log

When using three-server redundancy, the master records information to its local debug log and report log (and the Windows event log if this is configured). If this system fails, another license server becomes the master and records information to its local debug log and report log. Subsequently, there may be different versions of the debug log and report log on the primary and secondary license server which each contains different information.

Using Other Capabilities with Three-Server Redundancy

The following section describe other capabilities available in FlexNet Publisher Licensing Toolkit and how they interact with three-server redundancy.

Configuring the License Search Path

This configuration can be performed by either the software publisher or the license administrator. Before a FlexEnabled application can check out a license, it must know where to locate the license rights. The license search path identifies the location of license rights. When connecting to a license server configured for three-server redundancy, the FlexEnabled application must use the port@host convention (and not a license file location) in the license search path. The license search path should list the license servers in the same order that they appear in the license file. This helps shorten the amount of time it takes to identify the master server and respond to the checkout request. Although the configuration will work if you include only one of the license servers in the license search path, this may lengthen the amount of time it takes for the license server to respond to the checkout request. This is because the license server must identify all other license servers and designate a master. Separate each port@host entry with a comma. Using the previous license file as an example, the license search path should be
    2837@pat,2837@lee,2837@terry

The FlexEnabled application will try to connect to each of the license servers in the list, in the order listed, until it either successfully connects to a license server or reaches the end of the list. This helps ensure that the FlexEnabled application can connect to the quorum.

Specifying Three-Server Redundancy in the License Finder

When the license search path has not been configured, the FLEXlm License Finder dialog is displayed on Windows platforms when a FlexEnabled application is run.
Task:         To specify a triad of license servers in the License Finder dialog:
1.    Select Specify the License File.
2.    Click Next.
3.    Type the path name or use the browse button to specify your three-server redundant license file. An Example License File shows a typical three-server redundant license file.
4.    Click Next.

Note that the License Finder dialog option, Specify the License Server System, allows you to only specify a single license server and not a triad of license servers.
Using License File Keywords

The following keywords and properties for the SERVER line allow you to modify the configuration.
•    Host: this is the hostname of the system. The publisher should know this information when generating the license file. This value can be changed after the license file has been signed.
•    Port: the port number that the license server uses to listen for communication. Unlike single license servers, each SERVER line must include a port number. This can be any number between 1024 and 64000 that is not used by another process running on the system. This value can be changed after the license file has been signed. If you are using lmadmin, you do not need to edit the license file: you can configure the port number using the interface. See on-line help for more details. To make it easier to administer the license server, we strongly recommended that you define the same port number for each SERVER line. This value can be changed after the license file has been signed.
•    PRIMARY_IS_MASTER: this keyword ensures that the primary server is the master whenever it is running and communicating with one of the other license servers.
    •     If this is set and the primary server goes down, when the primary server comes back up again, it will always become the master.
    •     If this is not set and the primary server goes down, the secondary server becomes the master and remains the master even when the primary server comes back up. The primary can only become the master again when the secondary license server fails. This parameter is optional and should be placed on the first SERVER line. This value can be changed after the license file has been signed. The license server must be running a version 10.8 or later vendor daemon to use this keyword.
•     HEARTBEAT_INTERVAL=seconds: this indicates how long the license servers wait to receive a heartbeat from another license server before shutting down the vendor daemon. This value is used in the following equation to calculate the actual timeout value: timeout = (3 * seconds) + (seconds – 1) The default value is 20, which equates to an actual timeout of 79 seconds. Valid values are 0 through 120. This parameter is optional and should be placed on the first SERVER line in the license file. This value can be changed after the license file has been signed. The license server must be running a version 10.8 or later vendor daemon to use this keyword.

Using Options File Keywords

None of the keywords in the options file affect three-server redundancy.

Troubleshooting Tips and Limitations for Three-Server Redundancy

Important: Three-server redundancy configurations require all three servers use the same platform type. You can use any of the following platform types in the configuration, but each server must use the same platform type: HPV_*, VMW_*, or PHY_*.

Separating the Contents of a License File

Because the hostid values in the SERVER lines are computed into the signature of each feature definition line, make sure you keep SERVER lines together with any feature definition lines as they were generated. This means that if you move a feature definition line to another file, you must also move the respective SERVER lines and VENDOR line.

Putting the License File on a Network File Server

Do not put the license file on a network file server. If you do this, you lose the advantages of having failover protection because the file server becomes a possible single point of failure.

Using License Servers in Heavy Network Traffic

On a network with excessive traffic, the license servers may miss heartbeats which causes them to shut down the vendor daemon. The master may then stop serving licenses. If you find that heavy network traffic causes this to occur, you should set the HEARTBEAT_INTERVAL to a larger value. Enterprises can experience a performance issue when there is slow network communication or if FlexEnabled clients are using a dialup link to connect to the network.

Using Multiple Vendor Daemons

The license server manager (lmadmin or lmgrd) can not start vendor daemons from multiple software publishers when configured for three-server redundancy. The license server manager can only manage one vendor daemon. If one of the systems runs more that one vendor daemon, then the license administrator must run separate instances of the license server on that system to support the other vendor daemons. Make sure that the port numbers do not clash.

Avoiding Undefined lmdown Behavior

If any two license servers in a three-server redundancy group are started with the -allowStopServer no option (lmadmin) or the -x lmdown option (lmgrd), then the behavior of lmdown is undefined for that system.

Visitor Comments

No visitor comments posted. Post a comment