Ask HN: We just had an actual UUID v4 collision...

477 points by mittermayr 10 days ago | 342 comments

I know what you're thinking... and I still can't believe it, but...

This morning, our database flagged a duplicate UUID (v4). I checked, thinking it may have been a double-insert bug or something, but no.

The original UUID was from a record added in 2025 (about a year ago), and today the system inserted a new document with a fresh UUIDv4 and it came up with the exact same one:

b6133fd6-70fe-4fe3-bed6-8ca8fc9386cd

We're using this: https://www.npmjs.com/package/uuid

I thought this is technically impossible, and it will never happen, and since we're not modifying the UUIDs in any way, I really wonder how that.... is possible!? We're literally only calling:

import { v4 as uuidv4 } from "uuid";

const document_id = uuidv4();

... and then insert into the database, that's it.

Additionally, the database only has about 15.000 records, and now one collision. Statistically... impossible.

Has that ever happened to anyone?! What in the...

jandrewrogers 9 days ago |

This is surprisingly common.

The security of UUIDv4 is based on the assumption of a high-quality entropy source. This assumption is invalidated by hardware defects, normal software bugs, and developers not understanding what "high-quality entropy" actually means and that it is required for UUIDv4 to work as advertised.

It is relatively expensive to detect when an entropy source is broken, so almost no one ever does. They find out when a collision happens, like you just did.

UUIDv4 is explicitly forbidden for a lot of high-assurance and high-reliability software systems for this reason.

LocalH 9 days ago | |

This is why CloudFlare has done what they did with the lava lamp wall. Not that the wall is such a great source of entropy on its own - I'm sure it's not their only source, but you can never have too many sources of entropy - but it makes it visible in a way that can grab those who don't fully understand the concepts of RNGs and how entropy plays into that.

The more sources of entropy, the more closely you approach "perfect" randomization. And a large chunk of those entropy sources need to be non-deterministic. Even on the small level, local applications running on local systems, like games, can use things like the mouse coordinates, the timings between button presses, the exact frame count since game start before the player presses Start to greatly enhance randomness while still using PRNGs under the hood

Yes, for the latter, that's technically deterministic (and the older the game considered, the more deterministic it is, see TAS runs of old games obliterating the "RNG"). But when you have fifty different parameters feeding into the initial seed, that's fifty things an attack would have to perfectly predict or replay (and there are other ways to avoid replay attacks that can be layered on top)

If CloudFlare had less than 100 different sources of entropy, I'd be disappointed. And that's assuming their algorithm for blending those entropy sources into a single seed value is good

greiskul 9 days ago | | |

> you can never have too many sources of entropy

This is so true. And the beauty is that with algorithms, we don't even need to know much about the entropy to be able to extract it.

There is the Von Neumann method of generating an unbiased coin from a biased coin. Of throwing it twice, and checking if you got HT or TH. And completely discarding all HH or TT results. It doesn't matter if the coin you are using is 20% or 80%, the result will be a true 50/50.

There are more modern algorithms that can be even better (in that they need less coin tosses if you have a very unbalanced coin).

And then there is modern cryptographic hashing. Feed it all the bits you can. Collisions end up only happening in the real world if every single one of those bits is identical. So if you have actual entropy being fed, that cannot be controlled, predicted, or replicated, modern cryptography tells you that the end result is unique.

victorbjorklund 9 days ago | | |

If I understand it the Lava lamps are 90% PR/fun. They have a lot of other sources for entropy that scales better.

dheera 9 days ago | | |

The lava lamps are just for show.

You can get entropy just by plugging an oscilloscope into a pile of dirt and cranking the gain up.

bmitc 9 days ago | | |

> This is why CloudFlare has done what they did with the lava lamp wall.

Interesting. I wonder how true it actually is that they use it like they claim here: https://www.cloudflare.com/learning/ssl/lava-lamp-encryption.... It's in one of their lobbies, so doesn't that make it susceptible to an attack in some way? I'm not knowledgeable enough to know, but I figured if they actually used that method, they'd have a more controlled environment.

I also don't fully understand it. A large part of that wall is static. And the camera isn't going to pick up on the stochastic properties of the lava as much as exists in the real world. So it feels like their images will be very statistically similar.

__s 9 days ago | | |

Old games are RTA viable to RNG manip: https://m.youtube.com/watch?v=Bgh30BiWG58

Groxx 9 days ago | |

Yep - I've seen legitimate-looking dups on bad hardware, and "there are a ton of trailing zeros" is also an incredibly common duplicate mode for some UUID libraries (like earlier Go ones that didn't validate the "requested N bytes, returned 3, you must re-request to get N-3 more" return values. it doesn't happen on most hardware or OSes, so people never check it, so it just comes up in production some day with tens of thousands of collisions).

thecloud 9 days ago | |

Thanks for the insight! Mind expanding on what alternatives are being used in high reliability systems instead of UUIDv4?

jandrewrogers 9 days ago | | |

In high-reliability systems a criterion for identifier design is easy detection of defective identifiers. This includes buggy systems and adversarial manipulation.

The problem with UUIDs that rely on entropy sources is that it is computationally expensive to detect if the statistical distribution of identifiers is diverging from what you would expect from a random oracle. I've written systems that can detect entropy source anomalies but you'll want to turn it off in production.

It is pretty cheap to sanity check most non-probabilistic identifier schemes. UUIDs that use broken hash algorithms (e.g. UUIDv3/5) or leak state (e.g. UUIDv7) are exposed to adversarial exploitation.

The identifier scheme is dependent on the use case. Does the uniqueness constraint apply to the instance of the object or the contents of the object? Is the generation of identifiers federated across untrusted nodes? How large is the potential universe of identifiers?

The basic scheme I've seen is a 128-bit structured value that has no probabilistic component. These identifiers can be encrypted with AES-128 when exported to the public, guaranteeing uniqueness while leaking no internal state. The benefit of this scheme is that it is usually drop-in compatible with standard UUID even though it is technically not a UUID and the internal structure can carry useful metadata about the identifier if you can decrypt it.

Federated generation across untrusted nodes requires a more complex scheme, particularly if the universe of identifiers is extremely large. These intrinsically have a collision risk regardless of how the identifiers are generated.

All of the standardized UUID really weren't designed with the requirements of scalable high-reliability systems in mind. They were optimized for convenience and expedience which is a perfectly reasonable objective. Most people don't need an identifier system engineered for extreme reliability, even though there is relatively little cost to having one.

filcuk 9 days ago | | |

The latest UUID (7?) Uses half random gen, half timestamp. This not only makes it sortable by creation, but would also make a collision like this impossible.

matt-p 9 days ago | | |

UUIDv7 is arguably better, because it is entropy plus time.

lazide 9 days ago | | |

Sequences, generally.

perching_aix 9 days ago | |

How is UUIDv4 to blame for a broken source of entropy? Or am I misinterpreting your words?

hmry 9 days ago | | |

I wouldn't say it's "to blame", but it is more susceptible to bad RNG.

If the RNG is bad, you'll get more benefit from adding non-random bits than you would from additional badly RNG'd bits.

The probability of future collisions also rises the more IDs you generate. If you incorporate non-random bits, you can alleviate that:

- timestamps make the collision probability not grow over time as you accumulate more existing UUIDs that could collide

- known-distinct machine IDs make the collision probability not grow as you add more machines

jandrewrogers 9 days ago | | |

I never blamed UUIDv4 for broken entropy sources. A broken entropy source breaks UUIDv4 even if you are using it correctly.

There is a long history of broken entropy sources showing up in real systems. No matter how hard people try to prevent this it keeps happening. Consequently, a requirement for high-quality entropy sources is correctly viewed as an unnecessary and avoidable foot-gun in high-reliability software systems.

hombre_fatal 9 days ago | | |

Presumably they mean using randomness as unique IDs.

ranger_danger 9 days ago | |

Reading the UUID spec leads me to believe that good entropy is not even a requirement for any version:

> Implementations SHOULD utilize a cryptographically secure pseudorandom number generator (CSPRNG) to provide values that are both difficult to predict ("unguessable") and have a low likelihood of collision ("unique").

From https://www.rfc-editor.org/rfc/rfc9562.html#unguessability

So I don't think technically we can say entropy or random numbers at all are even "required for UUIDv4 to work as advertised."

toast0 9 days ago | | |

Any PRNG, including a CSPRNG is simple to predict if you know its inputs.

You need entropy to seed your CSPRNG.

adonovan 9 days ago | |

For a while we’ve been fixing telemetry-reported crash bugs in the project I maintain, and now hardware bugs are showing up with some frequency. I was amazed how common they are. Sometimes data values (e.g. SP register) are corrupted, but other times even infallible operations (e.g loads of rodata constants) crash, indicating that the instruction itself was corrupted. So, yeah, I believe you’ll eventually see UUID collisions, but not because the underlying cryptanalysis was wrong.

Hizonner 9 days ago | |

> UUIDv4 is explicitly forbidden for a lot of high-assurance and high-reliability software systems for this reason.

Hmm. What do those systems do for cryptography? Just assume it won't work and not rely on it at all?

jandrewrogers 9 days ago | | |

In these kinds of systems the cryptographic components often aren't even accessible from the software. It isn't a thing you need to worry about.

This makes it easier to audit for use of entropy sources in the software since there really isn't a valid use case for it.

BrandoElFollito 9 days ago | |

> We just had an actual UUID v4 collision...

(...)

> This is surprisingly common

And there goes though the window my blind faith with UUID v4 :)

I recently read about UUID v7 (https://en.wikipedia.org/wiki/Universally_unique_identifier#...) that became my favourite random identifier.

iberator 7 days ago | |

what else so you suggest instead of uuuid4?

erikerikson 9 days ago | |

Super simple to detect and try again.

jandrewrogers 9 days ago | | |

A collision is simple to detect but it requires you to actually check, which is expensive at scale. The entire point of UUIDv4 is that you don't have to check for collisions because it should never happen. But if you don't check and it does happen you are in UB territory which is generally very bad.

A risk of collision before it happens is non-trivial to detect but this is really what you'd want.

squirrellous 9 days ago | | |

In this specific case. In the case of trace IDs (an example of which is [1]) where the equivalent of UUIDs are explicitly used to avoid coordination, it’s hard to imagine how you’d reliably detect and retry.

[1] https://news.ycombinator.com/item?id=48033853

throwaway_19sz 10 days ago |

Funny story no one will believe, but it’s true. A good friend of mine joined a startup as CTO 10 years ago, high growth phase, maybe 200 devs… In his first week he discovered the company had a microservice for generating new UUIDs. One endpoint with its own dedicated team of 3 engineers …including a database guy (the plot thickens). Other teams were instructed to call this service every time they needed a new ‘safe’ UUID. My pal asked wtf. It turned out this service had its own DB to store every previously issued UUID. Requests were handled as follows: it would generate a UUID, then ‘validate’ it by checking its own database to ensure the newly generated UUID didn’t match any previously generated UUIDs, then insert it, then return it to the client. Peace of mind I guess. The team had its own kanban board and sprints.

CodesInChaos 9 days ago |

This is usually caused by an insufficently seeded PRNG.

Are you generating the UUID in the backend, or the frontend? Frontend is fundamentally unreliable for many reasons, including deliberate collisions. So if that case you'll need to handle collisions somehow. Though you can still engineer around common sources of collisions, the specifics depend on the environment.

On the other hand making a backend reliable is feasible. What kind of environment is your code running in? Historically VMs sometimes suffered from this problem, though this should be solved nowadays. Heavily sandboxed processes might still run into this, if the RNG library uses an unsafe fallback. Forking processes or VMs can cause state duplication and thus collisions.

danpalmer 9 days ago | |

I remember hearing about Segment (analytics company) had their entire product based around UUIDs generated in web browsers. There were collisions all over the place, the product was seemingly incapable of producing useful data at a fundamental level because of it. Hopefully they've fixed that now.

_kst_ 9 days ago |

This reminds me of a passage from the book "Pro Git".

<https://git-scm.com/book/en/v2>

"Here’s an example to give you an idea of what it would take to get a SHA-1 collision. If all 6.5 billion humans on Earth were programming, and every second, each one was producing code that was the equivalent of the entire Linux kernel history (6.5 million Git objects) and pushing it into one enormous Git repository, it would take roughly 2 years until that repository contained enough objects to have a 50% probability of a single SHA-1 object collision. Thus, an organic SHA-1 collision is less likely than every member of your programming team being attacked and killed by wolves in unrelated incidents on the same night."

Deliberate collisions are addressed in the following paragraph.

SHA-1 hashes are not random, so the issue of poor pseudo-random number generation doesn't apply as it does to uuidv4. And SHA-1 hashes are 160 bits, vs. 128 for uuidv4.

But I love the idea of unrelated wolf attacks.

adyavanapalli 10 days ago |

What you're talking about is so extremely rare that it's much more likely that the entire Earth is destroyed by an asteroid right this inst...

thomasmg 9 days ago | |

It is not quite as rare. I calculated it to be less common than being hit by a meteorite, and added a section about that and the Birthday Paradox to Wikipedia, to the article about UUIDs. It got removed / replaced a few years ago however. (If my source was correct, there was actually a woman hit by a meteorite, but she survived, with a leg injury.)

If you do have a UUID collision, chances are extremely high that it's either a software bug, or glitch in the computer. It could be a cosmic ray. Cosmic rays messing with the computer memory or CPU are actually relatively common.

delichon 10 days ago | |

About as rare as an asteroid typing an ellipsis and clicking the add comment button.

throw0101c 9 days ago | | |

Well, this joke dates back to (at least) the dial-up days where {#`%${%&`+'${`%& NO CARRIER

xerox13ster 9 days ago | | |

That’s just a result of jounce from localized gravity effects and atmospheric pressure disturbances in the moments before impact.

Think the ultrasonic typing hacking scene in Pantheon combined with the keyboard bouncing due to rumbling.

sebazzz 10 days ago | |

Well it would be statistically even rarer for that UUID collision to happen and the earth to be destroyed by an asteroid.

spindump8930 9 days ago | |

It's very common if you improperly seed, as others in the thread brought up! Or in your framing, as rare as earth getting hit if it were surrounded by a sci-fi density asteroid field.

crazylogger 10 days ago | |

For a single database using UUIDs, yes, it's astronomically rare. But it's quite a different thing to say that no computer system on Earth has ever experienced a UUID collision. The number of systems out there is also astronomical.

nathanmills 9 days ago | | |

>The number of systems out there is also astronomical.

Not even close

juancn 10 days ago |

Something off on how the RNG is initialized? Lack of entropy?

If the rng is not customized it will use:

    const rnds8 = new Uint8Array(16);
    export default function rng() {
        return crypto.getRandomValues(rnds8);
    }

getRandomValues doesn't specify a minimum amount of entropy.

Hizonner 10 days ago | |

It's a near certainty that something is badly wrong with the RNG, and, yes, probably in how it's seeded.

It's probably messing up the cryptography, too.

Onavo 9 days ago | | |

But defaults should be sane and safe. RNG isn't the sort of thing you want to be messing up. Every JS dev was taught that Math.random is not safe by default, but the crypto package is.

Geee 10 days ago |

According to the many-worlds interpretation of quantum mechanics, there's bound to be one branch of universe where every UUID is the same. Can you imagine what those guys are thinking?

BobaFloutist 9 days ago | |

Not only that, there's vastly more where every UUID except one is the same, but they never got to that one because they didn't ever use them.

Or where the first two are unique, but every following one is one of the first two.

beng-nl 9 days ago | | |

One where all uuids are always different whenever the person who wrote the implementation tries it, regardless of computer. And whenever it’s someone else, always the same.

nyantaro1 10 days ago | |

This is why I am not a fan of the Everett approach

suprjami 9 days ago | |

They probably just generate "the UUID" then add an increasing number on the end so it's unique. Problem solved.

zeeveener 9 days ago | |

"Huh, this is just an identity function. Cool. Let's move on."

e12e 9 days ago |

Some discussion here:

https://github.com/uuidjs/uuid/issues/546

Eg:

> FWIW, I just tested crypto.getRandomValues() behavior on googlebot and it is also deterministic(!)

D2OQZG8l5BI1S06 9 days ago | |

That makes sense. I'm not sure why anybody would generate UUIDs in browsers though, it seems to defeat the purpose.

danpalmer 9 days ago | | |

Tell that to Segment. Hopefully they've fixed that, but they didn't seem to think it was a problem years ago (spoiler: it was a big problem).

perlgeek 9 days ago | | |

When you do CQRS, you basically have to generate some form of unique identifier in the client. Why not a UUID in the browser?

mittermayr 10 days ago |

I fully agree. It makes no sense. Yet...

The only guesses I'm having is that we originally generated UUIDv4s on a user's phone before sending it to the database, and the UUID generated this morning that collided was created on an Ubuntu server.

I don't fully know how UUIDv4s are generated and what (if anything) about the machine it's being generated on is part of the algorithm, but that's really the only change I can think of, that it used to generated on-device by users, and for many months now, has moved to being generated on server.

dweez 9 days ago |

Good moment to revisit this fun article: https://jasonfantl.com/posts/Universal-Unique-IDs/

If the entire universe were turned into a giant computer and did nothing but generate uuids until its heat death, how many bits would you need for the ID space?

CodeWriter23 9 days ago | |

If you're gonna go there, this is obligatory https://www.decisionproblem.com/paperclips/

ipaddr 9 days ago | |

"But are you worried that every human on Earth will be hit by a meteorite right now? That probability is also non-zero, yet it is so infinitesimally small that we treat it as an impossibility."

This might be a bad example because one meteorite could take out the world and given enough time is likely to.

beejiu 9 days ago |

Are your UUIDs generated client side or server side? If it's client side, it could be due to a crawling bot. Googlebot for example executes Javascript using deterministic "randomness".

adzm 9 days ago | |

Googlebot's lack of randomness was the conclusion of a previous incident for that package https://github.com/uuidjs/uuid/issues/546

AgentME 9 days ago | |

Yeah, the answer almost certainly has to be this, or that they were using an old version of the package which didn't use the system RNG correctly (the current version appears to do it correctly, but I didn't dive into older versions), or their project has loaded an old broken polyfill re-implementing the JS crypto API, or they were running this on a hosting setup that does something jank like resuming the same VM snapshot with its RNG state on multiple servers. This category of explanation is many orders of magnitude more likely than a true random collision.

arslantoor 1 day ago |

BTW, it never happened to me, I have been working for more than 7 years as a developer maintaining over 1M plus records, but the twist is that we used to generate UUID by timestamps, which is now automatically handled by uuidv7, where the first half is our timestamps and the second half random number, so collisions never happen. In your case, you must be shifted to UUIDv7 without making any change, or just generate using a timestamp

But yeah, collision is expected in random generation; a UUIDv4 is a 128-bit number. If you subtract the bits used to specify the version and variant, you are left with 122 bits of pure randomness.That means there are $2^{122}$ possible combinations, where each 100 combination has chances of collision by 50%

leni536 10 days ago |

It's not happening by chance, there is a bug somewhere.

From what I skimmed the package should just call to the js runtime's crypto.randomUUID(). I think it should always be properly seeded.

I think it is extremely unlikely that the runtime has a bug here, but who knows? What js runtime do you use?

jbverschoor 9 days ago |

Most plausible cause: uuid package depends on some random number generator package, which has recently been compromised in order to make “random” numbers predictable. As a result, many crypto (ssl + currency) projects are compromised due to a supplychain attack.

merlindru 10 days ago |

Gotta be a seeding issue. If it's not, and you can prove it, you're about to be a little famous probably :P

tumdum_ 10 days ago |

Poorly seeded prng.

jdthedisciple 10 days ago | |

most likely the culprit indeed

nswango 10 days ago | | |

But I used nonstandard nonces!

serf 10 days ago |

1 in 4.72 × 10²⁸

1 in 47.3 octillion.

i'd be suspecting a race condition or some other naive mistake, otherwise id be stocking up on lottery tickets.

(lol at the other user posting at the same time about the lottery ticket.. great minds and all that.)

petee 10 days ago | |

I've always looked at it the the other way - being that lucky would mean you have even less chance of something else lucky happening, good time to save your money

k4rli 10 days ago | |

The lottery ticket part makes no sense. Statistically if such an improbable event just happened to him, then chance of it happening again should be even more improbable.

sowbug 9 days ago | | |

This is probably (ha) a troll thread, but in case anyone here is among today's lucky (ha) 10,000, https://en.wikipedia.org/wiki/Independence_(probability_theo...

jaccola 9 days ago | | |

The chance of him winning the lottery is identical to before, however the reward if he wins is slightly greater.

He would win the lottery money + he gets to tell people who don’t understand independence this incredible story!

angoragoats 9 days ago | | |

No, the events are independent. If you have a UUID collide, your chance of winning the lottery if you enter it is exactly the same as it was before the UUID collision.

jordiburgos 10 days ago |

Please, do not use b6133fd6-70fe-4fe3-bed6-8ca8fc9386cd, I checked my database and I was using it already.

rich_sasha 10 days ago | |

I always thought generating UUIDs at random was insane. I now only use LLMs. The prompt is: "generate a UUID. Make sure no one ever used it anywhere in their code or database. Check your work and think hard about each step. Do not output any reasoning or plain English, only th UUID itself".

You're welcome.

mh2753 10 days ago | | |

Actually asking ChatGPT this query led it giving me this UUID "550e8400-e29b-41d4-a716-446655440000" which happens to be a very common example UUID

mittermayr 10 days ago | |

I knew it, we're all getting the same cheap UUIDs and the good ones are reserved for the big dogs.

Galanwe 10 days ago | | |

uuid.uuidv4() recently switched to "adaptive entropy" instead of "xmax entropy" in an effort to save costs on non-premium users.

antonvs 9 days ago | | |

You mean you’re not already entropymaxxing? n00b

robshep 10 days ago | |

I'm using 16b55183-1697-496e-bc8a-854eb9aae0f3 and probably some more too. I suppose if we all post our list here, then we can all check for duplicates?

jsnell 10 days ago | | |

You can check https://everyuuid.com/ for collisions.

mittermayr 10 days ago | | |

We should all send our already-generated UUIDs to a shared database, we could just put it on Supabase with a shared username/password posted on HN, so we can all ensure that after generating a UUIDv4 locally, it's not used by anyone else. If it's in the database, we know it's taken.

It's a super simple mechanism, check in common worldwide UUID database, if not in there, you can use it. Perhaps if we use a START TRANSACTION, we could ensure it's not taken as we insert. But that's all easy, I'll ask Claude to wire it up, no problem.

volemo 10 days ago | | |

A site previously posted here could be useful: https://everyuuid.com/

dist-epoch 9 days ago |

It's much more likely that you hit an "impossible bug" due to a bit flip somewhere.

Imagine the database having the old UUID in a memory buffer due to a recent index scan, and a bit flip happened somewhere in the logic which basically copied the old UUID into the memory location of the new UUID, or some buffer addresses got swapped, or the operation which allocated the new UUID received a memory buffer containing the old one, and due to a bit flip the memcpy operation was skipped, or something along that line.

Facebook wrote extensively about this, stuff like "if (false) {do_x(); )" and do_x being called. For example their critical RocksDB kv store has extensive redundant protections to defend against such "impossible bugs".

smokel 9 days ago |

Multiple times have I blamed compilers, cosmic rays, quantum effects, or at the very least an obscure kernel bug, before realizing that I was the source of a bug.

A collision at 15,000 records is so unlikely that I would first suspect something else. Duplicate processing, replayed requests, reused objects, misleading logs, or another code path reusing the identifier.

Could you share a bit more of the surrounding code so we can check?

8organicbits 9 days ago |

I wrote about real world collisions, including that particular library last year (https://alexsci.com/blog/uuid-oops/).

There are a bunch of constraints that must be strictly held for UUIDs to be collision resistant, I'd guess there is a problem with your random number generator.

glaslong 10 days ago |

Buy some lava lamps

pif 9 days ago |

All the comments I've been able to read are missing the elephant in the room: no high-quality entropy source can turn a "should" into a "must".

If you want something that is difficult to guess, ask the cryptography guys. But if you need something that is -_guaranteed_ unique, you must build it yourself.

sedatk 9 days ago |

> Duplicate UUIDs (Googlebot)

> This module may generate duplicate UUIDs when run in clients with deterministic random number generators, such as Googlebot crawlers. This can cause problems for apps that expect client-generated UUIDs to always be unique. Developers should be prepared for this and have a strategy for dealing with possible collisions, such as:

> - Check for duplicate UUIDs, fail gracefully

> - Disable write operations for Googlebot clients

https://github.com/uuidjs/uuid/commit/91805f665c38b691ac2cbd...

baq 9 days ago |

the vm you're running on virtualized all the entropy away.

Imustaskforhelp 9 days ago | |

This seems very likely to be the case.

Something tangentially cool which is related: https://eu.mouser.com/new/leetronics/leetronics-infinite-noi...

wazoox 8 days ago |

This didn't happen to me... yet. Here's what I found 2 days ago deep in a production PHP codebase:

  private static function createUUID(){
   $md5 = md5(uniqid('', true));
   return substr($md5, 0, 8 ) . '-' .
    substr($md5, 8, 4) . '-' .
    substr($md5, 12, 4) . '-' .
    substr($md5, 16, 4) . '-' .
    substr($md5, 20, 12);
  }

Holy cow, how didn't this horror come and bite us in the juicy parts ? I don't know.

sbuttgereit 9 days ago |

> I thought this is technically impossible

No, very technically possible... though, with good randomness, very, very unlikely.

But nothing technically prevents a UUIDv4 from generating a duplicate value.

nu11ptr 9 days ago |

Ultimately it comes down to your entropy source. I always generate and insert in a loop for this reason, if there is a collision, I therefore handle that gracefully.

beardyw 10 days ago |

Just a stupid question, but why not append the date, even in seconds as hex. It's just a few bytes and would guarantee that everything OK now will be OK in the future?

flohofwoe 10 days ago | |

You can just use a different UUID variant which includes timestamp data instead (e.g. v1 or v7), there are also variants which include the MAC address.

itsyonas 9 days ago | |

Might as well just use uuidv7

ASalazarMX 9 days ago | | |

But since the randomness is obviously borked, it was much better to use v4 and find out about it after just 15K records instead of X million records later.

mittermayr 10 days ago | |

yeah, any sort of additional semi-random data could've helped prevent this, I'm sure. That, however, is also kind of the idea of UUIDv4, it has lots of randomness and time built in already.

flohofwoe 10 days ago | | |

UUID v4 consists of only random bits, no timestamp info.

beardyw 10 days ago | | |

But surely hashing the date still allows for a future collision. Leaving the date as is means it will never collide after that one second has passed.

pan69 10 days ago | |

> but why not append the date

And use uuid v5 to hash it :)

OutOfHere 10 days ago |

This is why I prefer to use a random base32 string over UUID. At least you get a proper 128 bit entropy instead of just a 122 bit entropy as with UUIDv4. That's a 64x difference in collision probability. I always thought UUIDs were a toy, not for serious use. If you control the strings, you can even make a longer ID.

Also, numerous applications that use a unique ID per record frequently need to check for ID collisions. I know I do for a short URL generator.

athrowaway3z 9 days ago |

The rule of thumb is simple:

Consider if your ID can contain a timestamp besides a random value. The answer is usually yes. UUIDv7 is fine.

If you've spend the time to really work through the whole problem and have written down a proof how that leads to unacceptable info leak: Congratulations your system is complex and slow enough that you might as well take a strong cryptographic hash or UUIDv5 if you're lazy.

xyzzy123 9 days ago |

I had dup uuids causing soak test failures in a Linux based distributed system. After long investigation it turned out there was a kernel bug (race condition) that meant two processes on MP system reading from /dev/random at the same could (very rarely, like 1 in a million) get the same bytes when reading the device.

I'd look at rng initialisation first.

evnix 9 days ago |

All the probability mathematics aside, the real world we live in is probably a lot less random even with the best hardware random number generators.

I've moved on to something like TSID(where security isn't a factor) or uuidv7 to make sure this never really occurs in practice rather than over engineering the code with retries.

NKosmatos 10 days ago |

> I thought this is technically impossible

Actually it's not impossible, but very very improbable.

P.S. You should play a lottery/powerball ticket

P.P.S. Whenever I use the word improbable, the https://hitchhikers.fandom.com/wiki/Infinite_Improbability_D... comes in mind

sebazzz 10 days ago | |

> P.S. You should play a lottery/powerball ticket

Actually, they should not. That collision and winning the lottery would be even rarer.

lgeorget 9 days ago | | |

Assuming they are independent events, OP is not more nor less likely to win the lottery now that before running in the collision. I actually have more question if you claim the events in question are NOT independent!

rithdmc 10 days ago | |

Inconceivable!

abhishekiaex 3 days ago |

So, what the solution folks? I am also using the same in our network system.

wg0 10 days ago |

Would the UUID v7 be more collision proof? Hard to say because it takes time into account but then the number of entropy bits are reduced hence the UUID generated exactly at the same time have more chance of a collusion because number of entropy bits are a much smaller space hence could result in collusions more easily.

Thoughts?

_kst_ 9 days ago | |

UUID v7 relies on knowing what time it is.

Speculation: The most likely scenario for a UUID v7 collision is if UUIDs are generated during a system boot sequence, before the system clock is set to the current time. It's always 1970 somewhere. There are still 62 random bits, and optionally another 12 random bits, but those too could be problematic if the system hasn't generated enough entropy yet.

AntiUSAbah 10 days ago | |

You open up every millisecond a new block. Should be even more unlikely

mdavid626 9 days ago |

Or there is some other explanation, eg. somebody messed with the request manually, or with the db.

sudb 9 days ago |

This is first time I have experienced some vindication that choosing CUID2[1] for one of my projects was actually a good idea.

1. https://github.com/paralleldrive/cuid2

QuercusMax 9 days ago |

I lost all confidence in the infallability of software RNG when I was working on an assignment for Data Structures a million years ago (2000?). The assignment was simple: simulate a 2D random walk where you randomly go NSEW, and run 100 cases, collecting stats as to how long it takes to return to the origin.

Super easy assignment, wrote it up probably in C++ (maybe just C?), and ran it on my linux box (probably Debian potato). It finished super quick and gave me an average of like 5.6 steps to return to the origin or something. Cool!

I copied it over to my account on the department's HP-UX machines where I was supposed to run and submit it to my instructor. Compiled fine. And then it... just ran forever. I was doing rand() % 4 or something, and the HP-SUX RNG had crazy bias in its last 2 bits, and it just walked away forever, never returning to the origin. Well crap!

Got an A for my writeup, though!

0xfffafaCrash 9 days ago |

Is the uuid generated in the frontend or backend? If frontend, I’d wager the likeliest explanation is that the client code or request was messed with to inject a previously known uuid rather than an entropy issue.

rglover 9 days ago |

A check inside the generator function is the best way I've found to avoid this. Wrap uuid or whatever random generator with a check against an ID cache. If it already exists, just run the generator recursively.

naikrovek 10 days ago |

The chance of a UUIDv4 collision is very low, but it is never zero.

If everything is done properly, then this is very likely the one and only time anyone involved in the telling or reading of this account will ever experience this.

dalmo3 10 days ago | |

Classic gamblers fallacy!

jaccola 9 days ago | | |

Ironically one of the few comments in this thread that isn’t necessarily the gamblers fallacy!

The chance anyone involved saw or heard about the first one was near zero, now they’ve seen this one the chance they see another is still near zero (I.e unchanged).

not_math 10 days ago |

Reminds me of some code I saw running in production. Every time we added a new entry, we were pulling all the UUIDs from this table, generating a new UUID, and checking for collisions up to 10 times.

erlkonig 8 days ago |

I keep telling the dev teams I'm on that with enough data points, all those random numbers will (probably) eventually collide, and *then* we'll see how robust their software really is. At least your database flagged it, and hopefully nothing major exploded.

And yet, plenty of experienced devs, including team leads and CIOs, are convinced it's impossible. As in, they absolutely don't write code to deal with the condition. So a bad RNG can randomly destroy the system far sooner than expected at any time, and it won't be noticed, caught, re-genned, or anything, with concurrent corruption being entirely possible. They're fine with it. I feel like these are the same guys who don't check to see if malloc() succeeds.

I like to ask them, "If it's impossible, you're using too many bits, right?". I haven't talked any of them into hedging with a brownian motion detector, or a lava lamp or something for better randomness yet, but I'm still trying.

latentframe 9 days ago |

One of the most dangerous words in engineering is “statisticaly impossible” At enough scale edge cases stop to be theoretical and start become production events.

zie 9 days ago |

You forgot to use https://www.random.org/ as your source of randomness :)

nozzlegear 9 days ago |

> I thought this is technically impossible, and it will never happen,

In an eternal universe, even the most unlikely of events will happen an infinite number of times.

sqquima 9 days ago |

Meta, but if I had a question like this, I'd likely have asked on Twitter or Reddit first. I'll keep in mind using HN as an alternative Q&A site.

coldtea 9 days ago |

Were the chances than an npm package is crap factored in?

timbritt 8 days ago |

Why is nobody asking why the database wasn’t configured to enforce uniqueness on this column? This should have been a log discovery.

danfritz 9 days ago |

Always let your db generate uuids. On postgres this is easy since v18 it supports uuid v7!

There is no need to set uuids through javascript or node imo

hx8 9 days ago | |

There's plenty of reasons to set a unique identifier before database save, or to want a unique identifier that doesn't have a 1-to-1 relationship with your object.

For example, in the idempotent kafka consumer pattern we set a unique ID in the header of every kafka message at the time of message publishing. We then have our consumers do a quick check of the ID against their data store to see if they have processed the message before or not. This way there is no impact if a consumer sees the same message twice. This allows us more flexibility during rebalancing events or replaying old offsets.

Cantthink1029 9 days ago | |

Not every application uses a DB you know, there are other reasons to use a UUID

radial_symmetry 9 days ago |

Glad to be reading the comments here because I also had this happen to me once and thought I must have been going insane.

jabeer 5 days ago |

Would love feedback on my HTML-to-PDF API — pdfkitt.dev

BugsJustFindMe 9 days ago |

This is like one of the hardest things for people to understand. Even the best randomness guarantees fuck all. Entropy-based IDs are collision-resistant not collision-proof.

shortercode 9 days ago |

Fun thing about random is that these things happen. UUIDv7 is less prone to this as it includes both a time component and random. I’ve been using ULID in a few project which has similar attributes to uuidv7 but more space efficient.

lyfeninja 10 days ago |

Although incredibly rare, it's not impossible so probably best to just plan for collisions. A simply retry should suffice. But I agree I feel like something is going on somewhere else ...

swah 7 days ago |

const blacklisted = ['b6133fd6-70fe-4fe3-bed6-8ca8fc9386cd'];

pyuser583 9 days ago |

What’s the prefered uuid these days?

AndreyK1984 10 days ago |

Why not to have timestamp-uuid instead ?

dgellow 10 days ago | |

How confident are you that your machines clocks are in perfect sync? What about the risk of clock drift + correction, or hardware issues?

croon 10 days ago | | |

Not GP, but: not confident. How confident would I be to avoid a (slightly lower entropy) UUID collision while also avoiding a clock desync landing on the exact same logged millisecond? Very, which is how confident I was about not encountering an UUID collision before this thread, so very++ I guess.

kdps 9 days ago | | |

I get why sync of mutiple machines matters for ordering and causality, but why is it a problem for uniqueness?

darqis 8 days ago |

pg 18 supports uuidv7 natively and your default should be unique and uuid7()

byteflow 9 days ago |

That's insane..

nhumrich 9 days ago |

> technically impossible

Not at all! Just very unlikely. It's about odds and statistics. Not physics.

ASalazarMX 9 days ago | |

This undersells the word unlikely. It is very, very, very, very unlikely.

ares623 10 days ago |

Buy a lottery ticket

zuzululu 9 days ago |

just uuidv5

kittikitti 9 days ago |

Almost all pseudo-random number generators are absolute garbage. They need you believe they work because the NSA needs backdoors and to foolproof ransomware attacks. This isn't surprising at all to me.

MagicMoonlight 9 days ago |

This is why it’s stupid to assume a randomly generated ID is unique just because it is random.

sublinear 10 days ago |

> We're using this: https://www.npmjs.com/package/uuid

Why? There's a built-in for this.

https://nodejs.org/api/crypto.html#cryptorandomuuidoptions

OptionOfT 9 days ago | |

That's what the package uses. And if `crypto.randomUUID()` doesn't exist, it falls back to `crypto.getRandomValues()`, which per the documentation isn't AS strong:

https://developer.mozilla.org/en-US/docs/Web/API/Crypto/getR...

So by using the package you actually lose visibility of cases where `crypto.randomUUID()` would fail.

Lammy 9 days ago |

> I thought this is technically impossible, and it will never happen

I always hated this meme/mindset, because if you dig in to the history of them you'll see that their original purpose was to collide. They were labels to identify messages in Apollo's distributed computing architecture. UID and later UUIDs were a reversible way to mark an intersection point between two dimensions.

Any two nodes in a distributed system would generate the same UID/UUID for the same two inputs, and a recipient of an identified message could reverse the identifier back into the original components. They were designed as labels for ephemeral messages so the two dimensions were time and hardware ID (originally Apollo serial number, later 802.3 hwaddress etc).

I think a lot of the confusion can be traced to the very earliest AEGIS implementation where the Apollo engineers started using “canned” (their term, i.e. static or well-known) UIDs to identify filesystems. Over time the popular usage of UUID fully shifted from ephemeral identifiers where duplicates were intentional toward canned identifiers where duplicates were unwanted and the two dimensions were random-and-also-random.

function uuid4() { var temp_url = URL.createObjectURL(new Blob()); var uuid = temp_url.toString(); URL.revokeObjectURL(temp_url); return uuid.split(/[:\/]/g).pop().toLowerCase(); // remove prefixes }

import rng from './rng'; const a = rng(); console.log('a after first call: ', Array.from(a)); const b = rng(); console.log('a after second call:', Array.from(a)); console.log('b after second call:', Array.from(b)); console.log('a === b (same reference)? ', a === b); console.log('a equals b (same contents)? ', a.every((v, i) => v === b[i]));

a after first call: [ 101, 193, 125, 19, 142, 136, 181, 140, 209, 224, 176, 153, 179, 248, 246, 166 ] a after second call: [ 4, 29, 48, 215, 162, 60, 64, 23, 78, 137, 2, 186, 230, 249, 70, 224 ] b after second call: [ 4, 29, 48, 215, 162, 60, 64, 23, 78, 137, 2, 186, 230, 249, 70, 224 ] a === b (same reference)? true a equals b (same contents)? true

import rng from './rng'; async function getId() { const bytes = rng(); await new Promise(r => setTimeout(r, 0)); // yield to the event loop return Array.from(bytes); } const [id1, id2] = await Promise.all([getId(), getId()]); console.log('id1:', id1); console.log('id2:', id2); console.log('identical?', id1.every((v, i) => v === id2[i]));

id1 captured: [ 61, 116, 151, 35, 153, 75, 105, 15, 59, 235, 162, 215, 224, 115, 31, 122 ] id2 captured: [ 13, 3, 84, 28, 22, 176, 160, 70, 67, 246, 1, 37, 38, 61, 171, 23 ] id1 after await: [ 13, 3, 84, 28, 22, 176, 160, 70, 67, 246, 1, 37, 38, 61, 171, 23 ] id2 after await: [ 13, 3, 84, 28, 22, 176, 160, 70, 67, 246, 1, 37, 38, 61, 171, 23 ] --- final id1: [ 13, 3, 84, 28, 22, 176, 160, 70, 67, 246, 1, 37, 38, 61, 171, 23 ] final id2: [ 13, 3, 84, 28, 22, 176, 160, 70, 67, 246, 1, 37, 38, 61, 171, 23 ] identical? true