Keeping an accurate time and date on a single server or across a group of servers is an important consideration for every customer. The following article provides details on how PowerVM manages the time on the server.
PowerVM Hypervisor and Service Processor
Each Power Systems server has a real-time hardware clock which begins running from 0 when power is first applied to the server at the factory. The value in the real-time clock is thus independent of Coordinated Universal Time (UTC). The real-time clock has a battery, so it ticks inexorably forward even when the server is disconnected from electrical power. The real-time clock has enough bits to run for many millennia before wrapping back to 0.
It is a good practice to set Coordinated Universal Time (UTC) on a server when the server is first deployed. UTC on a server can be set via the Advanced System Management Interface (ASMI) on the Flexible Service Processor (FSP) only when the server is powered off. The FSP's ASMI is usually accessed via a Hardware Management Console (HMC).
Setting a server's UTC does not change the server's real-time clock. Instead, a delta is saved by which UTC can be computed from the server's real-time clock. As each LPAR is created, the LPAR is configured with the same delta as the server so that the LPAR's virtual time of day (TOD) clock is consistent with UTC set on the server. The time zone on the operating system running on an LPAR can be configured so that the appropriate local time is displayed on the LPAR.
The real-time clock delta of the server and of each LPAR is saved in NVRAM on the FSP so the deltas survive server power loss. An LPAR's delta does not change unless/until the LPAR's virtual TOD clock is changed by (for example) using the AIX date command. The server's delta can be managed/changed as described below.
The enterprise baseboard management controller (eBMC) was introduced in Power10 and provides similar system management functionality as the FSP. With respect to time, the eBMC time-of-day clock cannot be synced with the Time Reference Partition (TRP) time-of-day clock. It is recommended that the Network Time Protocol (NTP) be enabled using the same NTP server on both the TRP and the eBMC to keep the time-of-day clocks in the TRP, the hypervisor and the eBMC all in sync.
Time Reference Partitions (TRP)
Since the service processor only allows a change to the TOD when the server is powered off, PowerVM allows a user designated Time Reference Partition (TRP) to control the TOD for the PowerVM Hypervisor and the service processor. Setting up a partition as the Time Reference Partition is as simple as clicking on the partition name on the HMC which will open the partition properties menu. From the Partition Properties menu, select the settings tab and chose Enabled for the Time Reference attribute
In the enhanced UI, you can select a partition and view its General properties. Select the Advanced option and the "Enable Time Reference" option can be seen which can be used to set the partition as Time Reference Partition
Because the TRP is responsible to maintaining the current time for the FSP and the PowerVM hypervisor, a TRP cannot use partition migration to move to another server. For this reason, you might want to create a partition just for this purpose or use partitions that cannot be migrated to other servers (for example VIOS partitions).
Once you have established a TRP, whenever the time is changed by the operating system, a notification is sent to the PowerVM Hypervisor to update its time based on the time set by the TRP. In addition, the hypervisor will forward the updated time to the FSP. PowerVM support allows multiple partitions to be designated as the TRP. If multiple partitions are designated as time reference partitions, the TRP that has been active the longest is used as the source for the TOD. If the current TRP becomes inactive, another partition designated as the TRP will take over the responsibility of maintaining the time on the server.
Management of Time within a Partition
Whenever you start a partition for the first time after power on of the server or anytime you create a new partition, the partition is started based on the current PowerVM hypervisor time and date. After this first time partition initialization, the operating system is in control of setting the correct time and date for the partition. Different operating systems have various time zone adjustments that can be applied on a partition basis so different partitions on the same server can be set up to run in different time zones. Some operating systems support the Network Time Protocol (NTP) where the partition can receive the current time from a known external source and synchronize the partition’s time and date to the values received from the NTP server. This would allow multiple partitions across multiple servers to stay in sync with regards to time of day. This may be especially important in the case of communications between partitions or in the situation where Live Partition Mobility (LPM) is used to seamlessly move a partition from one server to another.
HMC time
The HMC also allows for the setting of the time and date on the HMC. In Enhanced UI, You can view the Change Date & Time option under HMC Management -> Console Settings.
You are allowed to set the time based on the Coordinated Universal Time (UTC) standard, based on the local time or you can configure a NTP server. There are various public NTP servers that are available that transmit the current time. Some of these NTP servers also automatically handle events like leap seconds. Enabling NTP on the HMC requires a restart of HMC afterwards.
In summary, there are various options available to Power customers to ensure that the service processor, PowerVM hypervisor, partitions and the HMC all maintain an accurate time and date.
Example Behavior
Time Reference Protocol testing was performed on POWER8 by our System Test team. The permutations/options noted in the blog were explored/validated. The configuration and results of our “real-life” tests are noted below.
System - Power8 Firmware=FW840.40, HMC=8.8.4.0 SP2, VIOS=2.2.3.52
Initial State:
Dual VIOS configuration, both VIOS profiles have TRP disabled.
All Client profiles have TRP disabled
All clients and VIOS initially shut down
System is in POWER OFF
ASMI (Advanced System Management Interface) used to set back time by ONE HOUR (6:02:38 AM) real time: 7:02:38AM
NTP is initially disabled on VIOS and clients and HMC.
Use Case Sequence:
Use Case-A (no TRP)
Power on System with VIOS TRP disabled
Activate both VIOS with TRP disabledResult: When VIOS is activated, time was still OFF by ONE HOUR on BOTH VIOSes
Use Case-B (initial enablement of TRP)
Set first VIOS (via smittty) to be off of current time by -10 min
Set second VIOS (via smitty) to be off of current time by -5 min
Enable TRP on first VIOS running profile (this setting is dynamic and takes effect immediately).
Result: Hypervisor time and VIOS time set to current time - 10 min (longest running TRP enabled partition set to Hypervisor time)
Enable TRP on second VIOS running profile (this setting is dynamic and takes effect immediately).
Result: no change as the second VIOS, although TRP enabled, is NOT the longest running TRP-enabled partition.
Disable TRP on first VIOS (this setting is dynamic and takes effect immediately)
Result: Hypervisor time set to current time - 5 min (set from TRP on second VIOS).
Use Case-C (inclusion of NTP / ntpdate)
Usage of ntpdate to set time on first VIOS (recall: second VIOS has most uptime at this point)
Result: first VIOS has ntpdate "correct" time, Hypervisor and second VIOS are still at current time - 5 min
Usage of ntpdate to set time on SECOND VIOS
Result: Hypervisor time and VIOS time set to current time
General Comments and Recommendations:
TRP syncs Hypervisor time with LONGEST RUNNING TRP enabled partition. This setting is dynamic from the current running configuration from any partition on the server. Note that the TRP setting once set, is persistent on the partition until it is disabled.
If more than one partition has TRP enabled and the longest running TRP enabled partition goes down or is rebooted, TRP is used from the other lpar where TRP is enabled next immediate after the first one.
Using TRP on a dual VIOS configuration with ntp/ntpdate is sufficient (also keep HMC on NTP if possible) as a best practice.
Contacting the PowerVM Team
Have questions for the PowerVM team or want to learn more? Follow our discussion group on LinkedIn IBM PowerVM or IBM Community Discussions
#PowerVM
#powervmblog