Here is what I’m trying to do : we all know that System.currentTimeMillis() is the clock time (more or less the time) and System.nanoTime() can do nothing with actual time but can be used to measure things up to few microseconds accuracy…
What i want is essentially a cheap unique id/timestamp generator that is going to be called very rapidly maybe multiple times per microsecond and should issue new unique timestamps/ids…
Its single threaded and i do not have to worry about clock drift on that hardware.
Think of it as the master node that assigns unique event ids to log of events.
Just copy paste into a blank kotlin file and run and you’ll see.
import java.util.Date
import kotlin.math.absoluteValue
typealias Nanos = Long
//typealias Nanos = ULong
object clock {
var prevNanos: Nanos
var prevTime: Nanos
init {
val offsetNanos = System.currentTimeMillis() * 1_000_000L
val prevNanos = System.nanoTime()
Thread.yield()
val nextNanos = System.nanoTime()
val duration = nextNanos - prevNanos
val nextTime = offsetNanos + duration
this.prevNanos = nextNanos
this.prevTime = nextTime
}
/**
* Supposedly more accurate time? and cheaper?
*/
fun time(): Nanos {
val prevNanos = this.prevNanos
val prevTime = this.prevTime
val nextNanos = System.nanoTime()
assert(nextNanos >= prevNanos)
val duration = nextNanos - prevNanos
val nextTime = prevTime + duration
this.prevNanos = nextNanos
this.prevTime = nextTime
return nextTime
}
/**
* Same as the other but will increment one 1ns if you are calling it too frequently,
* guaranteed to be unique (assuming thread confinement)
*/
fun timeUnique(): Nanos {
val prevNanos = this.prevNanos
val prevTime = this.prevTime
val nextNanos = System.nanoTime()
assert(nextNanos >= prevNanos)
var duration = nextNanos - prevNanos
if (duration == 0L) duration++ // can be done without branches but whatever
val nextTime = prevTime + duration
this.prevNanos = nextNanos
this.prevTime = nextTime
return nextTime
}
/**
* Subtracts two numbers as normal but if they are equal returns 1 without using branches.
*/
fun subtractButIncrementIfEqual(a: Long, b: Long): Long {
val result = a - b
// Create a mask that is 1 if a == b, otherwise 0
val isEqualMask = (result or -result).inv() ushr 63
// Add the mask to the result, this increments the result if a == b
return result + isEqualMask
}
}
fun main() {
while (true) {
println("enter to print bunch of unique time/ids")
readLine()
val times = LongArray(100) { clock.timeUnique() }
for (i in 1..times.lastIndex) {
val diff = times[i] - times[i - 1]
assert(times[i] > times[i - 1]) { "wtf" }
val timeNs = times[i]
val timeMs = (timeNs / 1_000_000L).toLong()
val clockMs = System.currentTimeMillis()
val differenceWithWallClock = (clockMs - timeMs).absoluteValue
println("timeNanos : $timeNs, diffWithPreviousNs : $diff, timeMs : $timeMs, clockDiffMs : $differenceWithWallClock, date : ${Date(timeMs)}")
}
}
}
So its not about time at all, you just need unique ids? What about some sort of sequence? Simplest solution is having a counter variable in your code. Should be unique if single threaded, however you’d have to either store the value for when your program starts again, or at any start, initialize with a unique starting value that is higher than everything before.
Why not about time? you get both time and unique id… more accurate time than system millis. To be fair i also thought nanotime is significantly faster for some reason but in benchmarks its exactly the same cost…
In the main method you can see MS time difference compared to system.millis,
the difference is always within 1ms even after hours of running it… but yes if it made no sense as time of course i’d just increment a counter.
I basically have nanoscale timestamps (which are not exactly nanosecond accurate) but are unique, you can make them uniqe across several machines too using the same approach as chronicle distributed timestamps, using last 2 digits as instance identifiers but sacrificing the precision of nanotime by 2 least significant digits…
I’ve actually improved this example so that its not just random singleton but implements TImePovider so in tests where you need to control time you can simply swap the time provider and advance time as you like and i removed the if (duration == 0L) from unique nanotime method.
I just dont know why no one does take system millis to bootstrap nanotime and then keep using nanotime… never saw anyone do it so maybe there is a good reason for it.
If i wanted to actually pick up time drift, I could easily introduce a rare “correction” that also captures system.millis and makes sure every N hour you used system.millis or even better you could check against system.millis more frequently but gradually get on par with it so that if system moved time backwards you do not ever see your nanosec timestamp decrement but still slow down enough to catch up with system.milllis.
As Robert says, this whole page looks like a big X-Y problem.
There are many ways of generating unique IDs, depending on things like how widely they need to be unique (a thread, a process, a machine, an organisation, or globally), whether uniqueness is guaranteed or probabilistic, size and format, whether equivalent objects should have the same ID, whether/how the ID needs to be stored/published, etc.
But if it needs to be unique only within a single thread (which is all a time-based one would provide), I can’t imagine why it wouldn’t be much simpler and quicker (as well as more portable, easier to test, etc.) to use e.g. a basic sequence number instead.
However, if there are other reasons for wanting to use the time, then please explain!
because
a) counter is meaningless while timestamp carries valuable information and can be an id
b) if i want highest precision time information how else am i supposed to get it? jvm gives us system.millis which is millisecond accuracy while nanotime is accurate around 30ns which is miles better, but because nanotime is not primed with real timestamp we are doing it to have actually meaningful nanosecond timestamps…
so you get 3 things for free
more accurate timesamps, 2. uniqueness across whatever set you need (this is not limited to a thread you can very cheaply do it across a vm and with last 2 digits sacrificed you can make it unique across 99 machines 3. your time will never go backwards
you literally pay the same cost as system.millis + (few cycles)
As others said, I’m a little confused about requirements for this. You don’t specify you require strictly a timestamp, you say more like: “it would be nice”. But for me if something is not guaranteed to be a timestamp, then it can’t be used as a timestamp, so it could be replaced by any arbitrary integer.
Also, I’m not sure why you don’t do it simply like this:
This is without the uniqueness guarantee if we get the same nano. From your experiments, does it really happen we can get the same value twice from the System.nanoTime() or you just think it may happen? Nanos is the level of individual CPU cycles, so we shouldn’t be able to get so low. But I don’t know.
Simple implementation of uniqueness could be:
var counter = 0L
// later
offset + System.nanoTime() + counter++
After generating 1M IDs we drifted by just 0.1s. This could be a problem, but again, we don’t really generate precise timestamps anyway.
Answering why people don’t use this pattern. It doesn’t sound like a very common problem, it is quite custom.
Well i said “What i want is essentially a cheap unique id/timestamp generator” but yes i could be more clear about it.
You dont need to experiment, if you simply run the example and keep ENTER pressed you will see a bunch of 1s (if you use the other methods you will see 0) so yes it will happen very frequently especially if your’e in a tight loop… which is why i separate printing and generating timestamps…
My assumption of nanos being faster than millis was clearly wrong though.
Fair enough i guess its a custom problem, but you agree one could easily use system.milllis to prime nanos and then have nanotime actually as a usable timestamp?
Regarding your example, your counter will very quickly grow over time and now each next timestamp would start making huge gaps in time… you dont want to do that especially if you’re using a lot of those unique ids. Original problem with timestamp as id was that you may ask for so many ids that you race the clock and end up with the same ids…
all those 1s are really duplicated ids/timestamps because they are in tight loop that asks for 100 stamps…
im gonna keep this running overnight, see if clockDiffMs will ever go beyond few milliseconds… which would mean using millis + nanos from there on is accurate way to deal with time
Well, maybe. I think this is all about guarantees. System.nanoTime() doesn’t provide the same kind of guarantees as System.currentTimeMillis(), documentation is very clear it doesn’t represent a real time flow. I can tell you for sure if I suspend my computer, System.nanoTime() stops progressing, so timestamp drifts by the duration the OS was suspended. This is just one example. Maybe some operating systems (Android?) may suspend individual processes, so they lose the notion of time even if we don’t suspend the OS? Maybe System.nanoTime() drifts naturally with time? Maybe it may drift if the JVM or OS is overloaded? I don’t know answers to these questions.
I suspect in many/most cases we can assume this is a viable alternative for getting the current timestamp. But again, it is all about guarantees and if we need them or not.
I strongly believe no one should use any interpretation as storage format for time/date… store/transmit everything as a timestamp then deal with specific timezone/dates inplace.
I have seen enough examples of people shooting themselves in the foot because of storing some sort of interpretation of what date and time is…
kotlinx date time looks nice though (if i were to ever do multiplatform i’d use it as something that converts my timestamps).
btw you were right it will start coasting further and further away from system.millis
so if i were ever to use this it would actually have to periodically use system.millis like in the init so it keeps up with it.
Still i kinda like having nanoscale timestamps and being able to use them as both id and timestamps…
That’s what I like so much about Kotlinx-datetime. They have a very strong separation between instants of time (represented by the Instant class which is a timestamp) and local representations (represented by LocalDateTime and associated classes). The only way to go between the two is with a TimeZone.
yes it has one field but its still a class so instances in JVM (and frankly every other platform) will have all the overhead of an object with all the headers and everything… why not an inline class?
The worst part for me is that its immutable internal val value: jtInstant, another needless BS use of immutability (just like java records) that will force creation of bunch of objects and inefficient use of hardware.
In my own implementations i use a simple ULong (with a typealias so its very evident what the code is dealing with) and bunch of extension methods that allow me to do most time related compute with exception of complex timezone related crap.
I wrote it before inline classes, but i should look into converting into inline classes so extension methods do not apply to non timestamp primitives.
Because that doesn’t play well with multi-platform. We started out with Klock library which does use inline and for iOS we had to wrap it because inline types to don’t translate to iOS
No string is one good example how immutability should be used, it enables JVM to do insane amount of performance magic that is only possible because string is immutable. And no i dont write my own classes for string (smart people did already) and i use it when its needed, but what SDK devs should do is give me options and not force me to use immutable version for everything.
In performance critical systems you will never eeeever see people use String. SDK devs are fools for using String type directly everywhere, they should have been smarter and introduced CharSequence sooner and use that everywhere… their poor choices haunt us to this day with shitty apis.
Immutability is fine for message passing between different “actors”/processes almost nothing else should ever use it. Why? because all the hardware we have is designed for mutability, every time you instead of mutating create a copy of an object you pretty much flush your CPU caches that had everything you needed to do it lighting fast. You also make code less readable and more cumbersome to use. You also give GC more work to do…
For what benefit? concurrency/parallelism? it does not benefit that but everyone says how its useful for that, you still need to understand how to ensure visibility and happens before contracts properly.
Its great if you want to make cloud providers rich though, i’m sure they love it and promote immutability everywhere.
