KB2012.717
ADVISORY: Possible time issue due to erroneous leap second advertisement

This notice applies to hardware NTP or PTPv2 timeclocks (GPS, CDMA, etc.) and to software-based NTP servers such as those running NTPD on FreeBSD, Unix, or Linux.

Last updated: 7/18/2012

Advisory

Check your hardware-based and Linux-based primary NTP servers for erroneous pending leap second indicators.

NOTE: This advisory describes a newly discovered issue and is not related to the widely-publicized problems many systems had dealing with the most recent leap second applied last month (on 30 June 2012).

Background

The NTP and PTPv2 protocols advertise upcoming leap seconds by setting a flag in each packet. If the flag is set, it means that an extra second should be added (or, theoretically, removed) on the last second of the last minute of the last hour of the last day of the current month. There is no apply date information included in the packet -- a pending leap second simply assumes changes will be applied at the end of the of current month. Historically, leap seconds have been applied on either 30 June or 31 December, but this is not part of the specification.

Note that leap seconds are applied at midnight UTC, or everywhere in the world at the same moment. For most North American organizations, this means late afternoon to early evening local time. Last month, most public servers correctly advertised the upcoming leap second.

There is no standard for how far in advance a server should start advertising. At any time after the first of the month, if a leap indicator signal is retrieved from GPS satellites, NIST time files, or other means, servers should alert their clients. Conversely, once the leap second has been applied, servers should stop advertising a pending leap second.

The Problem

At the time of this writing, some NTP servers have not stopped advertising the 30 June 2012 leap second. This means that any standards-based client (such as Domain Time) will think another leap second is scheduled for the current month. This may result in incorrect/inconsistent time synchronization on time clients if the incorrectly-advertised leap second is applied.

Any server that is still advertising a pending leap second is wrong, and needs to be reset. It may also mean that the server will try to apply a leap second to itself at the end of the current month, making all of its timestamps after that time off by one second.

How to check your servers

How to correct the problem

If your NTP server or device is advertising a leap second, try restarting the system. Once it has reacquired its sources, it should no longer be advertising an upcoming leap second. (Be sure to re-check, in case it is using faulty NTP servers itself.)

Some NTP installations may benefit from NIST's leap second data file. Refer to your NTPD documentation on "leapfile" for further information.

If your server is hardware-based, such as a GPS receiver, you may need to contact the manufacturer.

For further information, please see David Mills' NTP documentation at http://www.eecis.udel.edu/~mills/ntp/html/leap.html

Mitigations for Domain Time users

All Users
Test and fix any NTP servers under your control (as described above).

If you use public time servers as your primary time source(s), you should test them for the leap second indicator as described above and avoid using any misreporting servers. Keep in mind that pool.ntp.org is a round-robin DNS alias for multiple servers, some of which may be advertising incorrectly.

Optional
On Domain Time version 5.1 and later, you can set Domain Time Clients and Servers to ignore pending leap seconds by unchecking the box labeled "Enable advance scheduling of leap second corrections" on the Advanced tab of the Control Panel applet. This will tell Domain Time that incoming leap second advertisements (from NTP, PTPv2, or DT2 sources) are not trusted. When this box is unchecked, Domain Time will not apply pending leap seconds, even if one is already scheduled or becomes scheduled after you check. (The log may still say a leap second has been scheduled; the enabled checkbox state is tested at the moment before applying the leap second.)

Versions prior to 5.1 do not do advance leap second scheduling; they merely adjust to any changes to the server's time at their next timecheck.

Domain Time Servers that are aware of a pending leap second will advertise it via the NTP, PTPv2, and DT2 protocols. If your network is arranged in a hierarchy, where Domain Time Server provides the time to all Domain Time Clients, it is NOT sufficient to uncheck the box on all of your Domain Time Server machines. The servers will continue passing the information to their clients, even if the leap second will not be applied by the server. You must uncheck the box on every client.

IMPORTANT: Unchecking this box will NOT prevent the problem entirely. Domain Time will not schedule and apply an advertised leap second on its own, but if your NTP source applies the scheduled leap second to itself, the time provided by that server will be off by one full second with no indication to clients that its time is incorrect.

Summary

The only way to be certain you won't have an issue is to fix any of your incorrectly advertising primary time sources (or use alternate sources).

For more information on Domain Time leap second scheduling, see http://www.greyware.com/kb/KB2012.717.asp