(Shameless plug to my ad-free blog: https://www.lelanthran.com/chap15/content.html)

1. Repeatable: the same inputs always lead to the same outputs 2. Algorithmic: the answer is arrived at by a set of steps that can clearly be shown to produce output with some desired property.

(2 is not a textbook definition of deterministic that I’ve ever heard, but I think it’s what people mean here)

LLMs don’t usually satisfy definition 1. They could, in principle, but it wouldn’t matter if they did. It wouldn’t tell you anything about whether the outputs are trustworthy.

What people seem to want is 2, but 2 is, I think, a contradiction. There cannot be an algorithmic process that can turns a natural language prompt into a working program, because natural language is too ambiguous to specify an output.

Nevertheless I agree with the main point of the article. Non-algorithmic processes (like intelligence) can produce algorithmic processes (like software). After all that’s what human programmers do!

https://thinkingmachines.ai/blog/defeating-nondeterminism-in...

But the determinism isn't the real distinction. LLM inference can be made deterministic (temparaute zero, etc.). And multi-threaded compiler can be non-dterministic.

And human developers are also non-deterministic. There is always the nondeterminism even before AI.

I think the real difference is how leaky the abstraction is. If you just need to understand language instead of compiler internal to write correct code, the compiler is non-leaky abstrction. If you rely on compiler to do auto SIMD optimization, and have to do hacks to trigger the optimization, compiler is leaky abstraction.

For AI, if you don't ever need to read the code and AI writes perfectly bug-free program that perfectly follows instruction, then AI is a non-leaky abstraction, but that's not the case for now. To make code maintainable and debuggable you still need to read the code if the project is complex enough.

They were saying one cannot replace programmers because any such attempt has led to something as complicated as code, and thus needed programmers.

I was telling them that the same way managers give them vague instructions and they turn out with code, managers could give LLMs vague instructions and they output code.

It's the uncertainty of "who knows what the machine will output this time" if you have no idea of what the input is going to be.

This tracks well with my experiences integrating AI into a software dev pipeline. Sure the output of an LLM is not deterministic in the same sense as a compiler (at least when the LLM has the typical configuration). However, that just means the LLM should not be solely responsible for the final product of your pipeline.

I think the reason it is so polarizing, is whether you like using your brain to problem solve or not. Those who don’t want to use their brain to problem solve and would rather just spend all their time in meetings, talking to customers or whatever else, are ecstatic with LLMs. It lets them skip problem solving entirely.

Those who like to think hard and solve problems, hate it because there’s no problem solving anymore when you use LLMs, and it moves so fast that it is hard to not use LLMs at most tech jobs (outside of managers forcing it down your throat). I never cared at all for customers, I definitely don’t care for shareholders, I only joined tech because I found it interesting to tax my mind to solve real problems and it paid well. Sadly now it only pays well, I think I’ll leave soon, once I make my money.

Unfortunately despite the author's indutrious optimism, this is equivalent to one or more open problems in theoretical CS. For example we don't know BPP = P or not. So are random algorithms tractable? We do not know. If you claim to know the answer, you could win a big math prize...

(It should be noted that despite the title, the author's main argument shows that output non-determinism does matter, since their entire effort is toward mitigating the stochastic behavior of LLMs.)

"Non-determinism" (i.e. the "N" in NP-Complete) does not mean "random", it means "following all paths at once" (there's a really elegant way to understand the List monad as representing non-deterministic computing). It's such a fundamental (and delightful) concept in computer science that refusing to understand what it means is a tragedy related to the loss of technical language itself.

But, even if we just sub "non-deterministic" for "inconsistent" or "random" or "stochastic" or whatever better term you come up with. Statements like the following just seem to not understand the basic architecture of LLMs:

> Given the same input prompt, a model's output is not guaranteed to be identical on every run, at least under normal running conditions.

"Normal running conditions"? The reproducibility of the output is 100% a property of how you choose to sample from an LLM. There are virtually endless ways you can define your sampler and whether or not the same result is reproducible is a fairly minor aspect of all the other choices you make when sampling.

At the end of the day, each step of the LLM gives you a distribution over next tokens, and you can do with that what you like.

In the end the article almost becomes a tautology: the output of an LLM represents a probability distribution over all the outputs it believes are valid with some variation around an expected output. Yes, that's literally what an LLM is.

Model outputs are deterministic. Your API provider may be injecting untunable amounts of randomness, but if you're not at the whim of an opaque cloud service, your model can be as deterministic as you please.

The unease that you describe isn't about models not being deterministic (because they are), it's that generating outputs from LLMs is an inherently inductive process (operating backwards by extrapolating from observations and examples) rather than a deductive process (operating forwards from careful logical reasoning founded on axiomatic principles). Deductive processes are more reliable, predictable, and analyzable than inductive processes, but they're also much harder to produce for many classes of problems.

And yet so many people expect deterministic responses from llms while they're entirely okay with non-deterministic responses from teammates. It's just not how it's been done in the past to always expect deterministic code generation from humans, and yet we expect that when we give work to agents.