Skip to content

time: use monotonic clock to measure elapsed time #12914

Closed

Description

@tsuna

Go's standard library doesn't provide any API to access a monotonic clock source, which means one can't reliably time how long an operation takes without copy-pasting or importing some low-level platform-dependent code such as the clock package at https://github.com/davecheney/junk.

This API doesn't necessarily need to return a time.Time, just returning a number of nanoseconds as a uint64 would be enough, kinda like Java's System.nanoTime().

Activity

  1. adg commented on Oct 13, 2015

    @adg
    Contributor

    Do we have a monotonic clock source we can export?

    If not, why should this be in the standard library and not just an
    importable package?

    On 13 October 2015 at 12:42, Benoit Sigoure notifications@github.com
    wrote:

    Go's standard library doesn't provide any API to access a monotonic clock
    source, which means one can't reliably time how long an operation takes
    without copy-pasting or importing some low-level platform-dependent code
    such as the clock package at https://github.com/davecheney/junk.

    This API doesn't necessarily need to return a time.Time, just returning a
    number of nanoseconds as a uint64 would be enough, kinda like Java's
    System.nanoTime()
    http://docs.oracle.com/javase/8/docs/api/java/lang/System.html#nanoTime--
    .

    —
    Reply to this email directly or view it on GitHub
    #12914.

  2. tsuna commented on Oct 13, 2015

    @tsuna
    ContributorAuthor

    Why should this be in the standard library: because it's a pretty fundamental thing, especially in a language geared towards systems programming. I see a lot of people timing how long operations take using time.Now() and subtracting two time.Time objects, and I always end up saying "this is not a correct way of measuring how much time elapsed" but don't have a clear alternative to propose.

    Come on, even Java has it.

  3. adg commented on Oct 13, 2015

    @adg
    Contributor

    I see a lot of people timing how long operations take using time.Now() and subtracting two time.Time objects, and I always end up saying "this is not a correct way of measuring how much time elapsed"

    Got any concrete examples?

  4. tsuna commented on Oct 13, 2015

    @tsuna
    ContributorAuthor

    Any concrete examples of ..? People using time.Now() or it being wrong because it doesn't take into account changes made to the system time?

  5. adg commented on Oct 13, 2015

    @adg
    Contributor

    I suppose what I'm getting at is this: If there's a broad need for such a package, why isn't there one that people are already using?

  6. extemporalgenome commented on Oct 13, 2015

    @extemporalgenome
    Contributor

    This API doesn't necessarily need to return a time.Time, just returning a number of nanoseconds as a uint64 would be enough, kinda like Java's System.nanoTime().

    time.Duration happens to represent a 64-bit quantity nanoseconds, and, unless there's some large problem with it, should be favored as the return type for any monotonic duration handling functions.

  7. added this to the Unplanned milestone on Oct 13, 2015
  8. taruti commented on Oct 21, 2015

    @taruti
    Contributor

    I think there are two things:

    A) A proper monotonic clock package (there are multiple 3rd party ones)

    B) Measuring elapsed time on systems where system time may leap to either direction

    B is simpler. Either ensure and document that time.Since works
    even with clock shifts or define new API like:

    type MeasureStart struct { ... }
    func (*MeasureStart)Start()
    func (*MeasureStart)Elapsed() time.Duration
    
  9. crawshaw commented on Oct 23, 2015

    @crawshaw
    Member

    On what platforms is time.Now() not monotonic? For linux/amd64, time.Now is implemented by runtime·nanotime, which is calling clock_gettime (via VDSO) with CLOCK_MONOTONIC. I see similar calls being used on linux/arm, openbsd, freebsd, etc.

    Instead of a new interface, can we document time.Now as monotonic?

  10. ianlancetaylor commented on Oct 23, 2015

    @ianlancetaylor
    Contributor

    @crawshaw time.Now is not monotonic on any platform that supports settimeofday. On amd64 GNU/Linux time.Now does not actually use CLOCK_MONOTONIC, it uses CLOCK_REALTIME. It's runtime.nanotime that uses CLOCK_MONOTONIC.

  11. ianlancetaylor commented on Oct 23, 2015

    @ianlancetaylor
    Contributor

    Sorry, I somehow missed that you mentioned nanotime yourself. That's not what implements time.Now, though, that is implemented by time·now.

  12. crawshaw commented on Oct 23, 2015

    @crawshaw
    Member

    Ah right, thanks. I got confused by //go:linkname time_runtimeNano time.runtimeNano, which I misread as implementing time.Now

  13. alexbrainman commented on Oct 23, 2015

    @alexbrainman
    Member

    time.Now() is monotonic on windows. We use undocumented way to fetch time, but it is monotonic as far as I know. @dvyukov to confirm.

    Alex

  14. tsuna commented on Oct 23, 2015

    @tsuna
    ContributorAuthor

    On Mon, Oct 12, 2015 at 9:07 PM, Andrew Gerrand wrote:

    I suppose what I'm getting at is this: If there's a broad need for such a package, why isn't there one that people are already using?

    Because people incorrectly assume that time only moves forward? It's an extremely common mistake that even at Google we were trying to hammer in people's head. When you see the large impact that things such as leap seconds have (including at Google), it's easy to see this is a well known problem but also a recurring problem.

    How is it that a language geared towards systems programming that comes with such a rich standard library offers no standard way of measuring time correctly?

    Look at gRPC, every single use of time.Now() there is buggy because it implicitly assumes that time only moves forward and at a constant pace, neither of which is true for CLOCK_REALTIME. Pretty much any distributed systems code that uses time.Now() other than for printing the current time is subtly buggy. And there is a lot of it. And it's there chiefly because there is no alternative in the standard library to get monotonic time so people don't even think about it.

  15. rsc commented on Oct 24, 2015

    @rsc
    Contributor

    How is it that a language geared towards systems programming that comes with such a rich standard library offers no standard way of measuring time correctly?

    I expect that if you care that much about time you will run your system clocks correctly. On a well-run system, time does only move forward and at a constant (enough) pace.

    When you see the large impact that things such as leap seconds have (including at Google), it's easy to see this is a well known problem but also a recurring problem.

    I don't understand this comment. Leap seconds don't exist at Google.

    It is true that if you use time.Now to measure time during a leap smear then your measurements may be off by approximately 0.001%. But time will still be monotonic.

  16. 153 remaining items

  17. removed this from the Proposal milestone on Feb 3, 2017
  18. cespare commented on Feb 3, 2017

    @cespare
    Contributor

    Looks like the current code on master uses 1885-2157 as the range of wall times when monotonic time is present.

  19. rsc commented on Feb 3, 2017

    @rsc
    Contributor

    OK, this is checked in for Go 1.9. I moved the internal epoch to 1885 (max year 2157) to avoid any possible problem with NTP-epoch-derived times. I also adjusted the design doc to reflect this change and some minor encoding changes.

    Thanks for the constructive conversation everyone.

  20. orivej commented on Feb 3, 2017

    @orivej
    1. The merged implementation changes the meaning of time.Now().Sub(old) when now and old are separated by system suspend and resume: it used to calculate real time difference and now that difference is reduced by the duration the system was suspended. Shouldn't time use monotonic real time source when available, and if so, can runtime.nanotime be redefined to read from that source, or should there be a new clock specifically for time.Now() to read from?

    2. Is there an explanation for + 1 in nanotime() - startNano + 1 in the design doc? If it is needed, why startNano is not reduced by 1 instead?

  21. rsc commented on Feb 3, 2017

    @rsc
    Contributor

    @orivej,

    1. Whether time.Since(start) includes "system suspended" time depends on the underlying OS's monotonic clock, but in general it's tricky to add that time into the clock since you can't take it back if you later find out you are wrong. My preference here would be to do the same thing on all operating systems, whatever that is. My guess is that all operating systems can provide the current semantics but that not all can provide the "include suspended time" semantics.

      It seems to me that if you want to include time while the computer was off, you're really talking about wall times, so the fix would be to flip into wall mode by recording 'start := time.Now().AddDate(0, 0, 0)', putting up with the fact that time.Since(start) might then return negative elapsed times.

    2. The +1 is just to make sure that tests never see a 0 monotonic time, since I used GetMono(&t) == 0 to say that there's no monotonic time. It's only for testing, and even in that context it's probably unnecessary, but also harmless.

  22. gopherbot commented on Feb 5, 2017

    @gopherbot
    Contributor

    CL https://golang.org/cl/36334 mentions this issue.

  23. neild commented on Mar 1, 2017

    @neild
    Contributor

    I'm looking at some code which assumes that times truncated to the same time are equal. Simplified:

    func main() {
            t1 := time.Now().Truncate(1 * time.Second)
            t2 := time.Now().Truncate(1 * time.Second)
            h1, m1, s1 := t1.Clock()
            h2, m2, s2 := t2.Clock()
            if h1 == h2 && m1 == m2 && s1 == s2 {
                    if !t1.Equal(t2) {
                            fmt.Printf("BUG: times truncated to the same second, but are not equal:\n%v\n%v\n", t1, t2)
                    }
            }
    }

    The actual code is mapping times to windows; it assumes that all times within a given window will truncate to the same time.

    This doesn't seem unreasonable. Should Truncate discard the monotonic time component?

  24. dmitshur commented on Mar 1, 2017

    @dmitshur
    Member

    @neild A relevant open issue is #18991.

  25. locked and limited conversation to collaborators on Mar 1, 2018
  26. removed their assignment
    on Jun 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions