Roast me i'm having bad ideas (probably)

Ok so i’m curious why no one does this.

Here is what I’m trying to do : we all know that System.currentTimeMillis() is the clock time (more or less the time) and System.nanoTime() can do nothing with actual time but can be used to measure things up to few microseconds accuracy…

What i want is essentially a cheap unique id/timestamp generator that is going to be called very rapidly maybe multiple times per microsecond and should issue new unique timestamps/ids…

Its single threaded and i do not have to worry about clock drift on that hardware.
Think of it as the master node that assigns unique event ids to log of events.

Just copy paste into a blank kotlin file and run and you’ll see.


import java.util.Date
import kotlin.math.absoluteValue

typealias Nanos = Long
//typealias Nanos = ULong

object clock {

    var prevNanos: Nanos
    var prevTime: Nanos

    init {
        val offsetNanos = System.currentTimeMillis() * 1_000_000L
        val prevNanos = System.nanoTime()
        Thread.yield()
        val nextNanos = System.nanoTime()
        val duration = nextNanos - prevNanos
        val nextTime = offsetNanos + duration

        this.prevNanos = nextNanos
        this.prevTime = nextTime
    }

    /**
     * Supposedly more accurate time? and cheaper?
     */
    fun time(): Nanos {
        val prevNanos = this.prevNanos
        val prevTime = this.prevTime

        val nextNanos = System.nanoTime()
        assert(nextNanos >= prevNanos)
        val duration = nextNanos - prevNanos
        val nextTime = prevTime + duration

        this.prevNanos = nextNanos
        this.prevTime = nextTime

        return nextTime
    }

    /**
     * Same as the other but will increment one 1ns if you are calling it too frequently,
     * guaranteed to be unique (assuming thread confinement)
     */
    fun timeUnique(): Nanos {
        val prevNanos = this.prevNanos
        val prevTime = this.prevTime

        val nextNanos = System.nanoTime()
        assert(nextNanos >= prevNanos)
        var duration = nextNanos - prevNanos
        if (duration == 0L) duration++ // can be done without branches but whatever
        val nextTime = prevTime + duration

        this.prevNanos = nextNanos
        this.prevTime = nextTime

        return nextTime
    }

    /**
     * Subtracts two numbers as normal but if they are equal returns 1 without using branches.
     */
    fun subtractButIncrementIfEqual(a: Long, b: Long): Long {
        val result = a - b

        // Create a mask that is 1 if a == b, otherwise 0
        val isEqualMask = (result or -result).inv() ushr 63

        // Add the mask to the result, this increments the result if a == b
        return result + isEqualMask
    }
}

fun main() {
    while (true) {
        println("enter to print bunch of unique time/ids")
        readLine()
        val times = LongArray(100) { clock.timeUnique() }

        for (i in 1..times.lastIndex) {
            val diff = times[i] - times[i - 1]
            assert(times[i] > times[i - 1]) { "wtf" }
            val timeNs = times[i]
            val timeMs = (timeNs / 1_000_000L).toLong()


            val clockMs = System.currentTimeMillis()
            val differenceWithWallClock = (clockMs - timeMs).absoluteValue
            println("timeNanos : $timeNs, diffWithPreviousNs : $diff, timeMs : $timeMs, clockDiffMs : $differenceWithWallClock, date : ${Date(timeMs)}")
        }
    }
}

I’m not gonna use this anyway but it seems the only reason everyone is using Millis is to catch clock drift, which i often dont want to do…

So its not about time at all, you just need unique ids? What about some sort of sequence? Simplest solution is having a counter variable in your code. Should be unique if single threaded, however you’d have to either store the value for when your program starts again, or at any start, initialize with a unique starting value that is higher than everything before.

Why not about time? you get both time and unique id… more accurate time than system millis. To be fair i also thought nanotime is significantly faster for some reason but in benchmarks its exactly the same cost…

In the main method you can see MS time difference compared to system.millis,
the difference is always within 1ms even after hours of running it… but yes if it made no sense as time of course i’d just increment a counter.

I basically have nanoscale timestamps (which are not exactly nanosecond accurate) but are unique, you can make them uniqe across several machines too using the same approach as chronicle distributed timestamps, using last 2 digits as instance identifiers but sacrificing the precision of nanotime by 2 least significant digits…

I’ve actually improved this example so that its not just random singleton but implements TImePovider so in tests where you need to control time you can simply swap the time provider and advance time as you like and i removed the if (duration == 0L) from unique nanotime method.

I just dont know why no one does take system millis to bootstrap nanotime and then keep using nanotime… never saw anyone do it so maybe there is a good reason for it.

If i wanted to actually pick up time drift, I could easily introduce a rare “correction” that also captures system.millis and makes sure every N hour you used system.millis or even better you could check against system.millis more frequently but gradually get on par with it so that if system moved time backwards you do not ever see your nanosec timestamp decrement but still slow down enough to catch up with system.milllis.

As Robert says, this whole page looks like a big X-Y problem.

There are many ways of generating unique IDs, depending on things like how widely they need to be unique (a thread, a process, a machine, an organisation, or globally), whether uniqueness is guaranteed or probabilistic, size and format, whether equivalent objects should have the same ID, whether/how the ID needs to be stored/published, etc.

But if it needs to be unique only within a single thread (which is all a time-based one would provide), I can’t imagine why it wouldn’t be much simpler and quicker (as well as more portable, easier to test, etc.) to use e.g. a basic sequence number instead.

However, if there are other reasons for wanting to use the time, then please explain!

1 Like

because
a) counter is meaningless while timestamp carries valuable information and can be an id
b) if i want highest precision time information how else am i supposed to get it? jvm gives us system.millis which is millisecond accuracy while nanotime is accurate around 30ns which is miles better, but because nanotime is not primed with real timestamp we are doing it to have actually meaningful nanosecond timestamps…

so you get 3 things for free

  1. more accurate timesamps, 2. uniqueness across whatever set you need (this is not limited to a thread you can very cheaply do it across a vm and with last 2 digits sacrificed you can make it unique across 99 machines 3. your time will never go backwards

you literally pay the same cost as system.millis + (few cycles)

As others said, I’m a little confused about requirements for this. You don’t specify you require strictly a timestamp, you say more like: “it would be nice”. But for me if something is not guaranteed to be a timestamp, then it can’t be used as a timestamp, so it could be replaced by any arbitrary integer.

Also, I’m not sure why you don’t do it simply like this:

val offset = System.currentTimeMillis() * 1_000_000L - System.nanoTime()

Array(20) {
    offset + System.nanoTime()
//    Thread.yield()
}.forEach(::println)

This is without the uniqueness guarantee if we get the same nano. From your experiments, does it really happen we can get the same value twice from the System.nanoTime() or you just think it may happen? Nanos is the level of individual CPU cycles, so we shouldn’t be able to get so low. But I don’t know.

Simple implementation of uniqueness could be:

var counter = 0L

// later
offset + System.nanoTime() + counter++

After generating 1M IDs we drifted by just 0.1s. This could be a problem, but again, we don’t really generate precise timestamps anyway.

Answering why people don’t use this pattern. It doesn’t sound like a very common problem, it is quite custom.

1 Like

Well i said “What i want is essentially a cheap unique id/timestamp generator” but yes i could be more clear about it.

You dont need to experiment, if you simply run the example and keep ENTER pressed you will see a bunch of 1s (if you use the other methods you will see 0) so yes it will happen very frequently especially if your’e in a tight loop… which is why i separate printing and generating timestamps…

System.nanos is not just few cycles, it does go to kernel (Measuring Time: From Java to Kernel and back - JVM Advent).

My assumption of nanos being faster than millis was clearly wrong though.

Fair enough i guess its a custom problem, but you agree one could easily use system.milllis to prime nanos and then have nanotime actually as a usable timestamp?

Regarding your example, your counter will very quickly grow over time and now each next timestamp would start making huge gaps in time… you dont want to do that especially if you’re using a lot of those unique ids. Original problem with timestamp as id was that you may ask for so many ids that you race the clock and end up with the same ids…

all those 1s are really duplicated ids/timestamps because they are in tight loop that asks for 100 stamps…

im gonna keep this running overnight, see if clockDiffMs will ever go beyond few milliseconds… which would mean using millis + nanos from there on is accurate way to deal with time

Well, maybe. I think this is all about guarantees. System.nanoTime() doesn’t provide the same kind of guarantees as System.currentTimeMillis(), documentation is very clear it doesn’t represent a real time flow. I can tell you for sure if I suspend my computer, System.nanoTime() stops progressing, so timestamp drifts by the duration the OS was suspended. This is just one example. Maybe some operating systems (Android?) may suspend individual processes, so they lose the notion of time even if we don’t suspend the OS? Maybe System.nanoTime() drifts naturally with time? Maybe it may drift if the JVM or OS is overloaded? I don’t know answers to these questions.

I suspect in many/most cases we can assume this is a viable alternative for getting the current timestamp. But again, it is all about guarantees and if we need them or not.

1 Like

Good point, i was only thinking about server side.