C Is Not a Low-Level Language (2018)(queue.acm.org) |
C Is Not a Low-Level Language (2018)(queue.acm.org) |
int main() {
__m512i vecA = _mm512_setr_epi32(0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15);
__m512i vecB = _mm512_setr_epi32(0,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75);
unsigned short mask = 0;
__asm__ (
"vp2intersectd %[B], %[A], %%k2"
: "=@cck2" (mask)
: [A] "v" (vecA), [B] "v" (vecB)
: "k3"
);
printf("Intersection Mask: 0x%04X\n", mask);
return 0;
}
Obviously if you rely on implicit behavior from the compiler to optimize your code you are no longer "low level". But if you can quickly and easily drop into machine-level instructions to provide explicit implementation semantics, and the language indeed makes that relatively simple and easy to do, that sure seems "low level" to me
Yet one of the more interesting reasons, in my mind, is that C adds a tonne of abstractions. The roster of data types is one of those abstractions. Processors have a very weak notion of data types, and memory has absolutely no notion of memory types at all. For example: casting a `float` to an `int` has a very specific definition in C, and that definition involves altering the pattern of bits. While you can create a float and force the C compiler to regard that memory location as an int (via casting pointers), it isn't how the language is meant to be used (outside of rare cases).
If I recall correctly, some of the direct predecessors of C were typeless, which is closer to how the CPU and RAM treat data.
Practically speaking, we have a word for the kind of "abstractionless" model you're describing: machine code. I mean, even assembler is a bunch of abstractions about 'registers' and 'instructions' that are really just specific portions of the hardware or opcodes!
So we either descend endlessly into pedantry arguing that cosmic rays and electron tunnelling represent inexcusable deviations from the overly abstracted semantics that hardware vendors expose in their products or maybe we draw the line somewhere else.
You may not agree with mine, that "practical and simple interop with machine-level language impls across a high-level language interface is sufficiently close to the hardware as to be low level" but there has to be a limit somewhere between that and "technically the hardware's operating temperature is part of its logical semantics because if it exceeds a certain value for long enough it starts to degrade and yield incorrect results or terminate execution". I think eventually it just becomes unproductive nerd sniping, personally
This is absolutely not true, unless you mean to say that processors should somehow support composite (aka C struct) data types as an instruction primitive. Processor operations have to be strongly typed, by definition. For example, these are the data types supported by operations in the modern x86 instruction set (ignoring vector extensions):
- signed and unsigned integers of 8, 16, 32 and 64 bits
- floating-point decimals of 32, 64 and 80 bits (and 128 via sse)
- nul-terminated byte strings
> For example: casting a `float` to an `int` has a very specific definition in C, and that definition involves altering the pattern of bits
I don't understand this example. Casting a float to an int also has a very specific definition in IEEE-754 and is pretty much universally implemented as a hardware instruction. It has been in the x86 family since its inception: https://www.felixcloutier.com/x86/fisttp
The contemporary Basic dialects of that time I could argue were low level languages. They were really thin. Like bcpl.
What makes 'C' not really low level by a reasonable definition, is that the register allocation decisions are not yet made. Which, depending on how it works out, can effect ordering, inlining, unrolling etc, so the compiler can pretty much go to town on your code and create something unrecognizable.
Since registers aren't really allocated here, this is basically on the level of C code, and all that stuff can happen here, so this really isn't much lower level than C.
Not being elitist, it's just worth knowing what's going on under the hood of compilers, and the nature of the contract they uphold.
There is AFAIK no way to express or interact with speculative execution/branch prediction, for example.
TFA famously argues that Spectre/Meltdown et al break that abstraction. But note that they are quite literally exceptions to the rule: the only reason we know/care about them is that the "magic under the hood" that was supposed to make CPUs faster while maintaining that abstraction introduced a bug that caused the implementation details to leak to the end users.
Similarly even vp2intersectd took multiple cycles in its original Intel impl and even in the performant AMD Zen5 impl it still takes >1 cycle with 6 levels of pipelining or somesuch. Ok. If literally not even a chip's ISA is "low level" then the term is effectively meaningless.
The only way you could define a "low level" language capable of exercising that hardware's capabilities fully would be to have some kind of per-cycle, pipeline-aware annotation layer over the actual machine code... which really seems like quite a lot of noise/cruft you'd not typically want to add on top of everything, all in the name of still technically being low-level according to some dubiously pedantic criteria nobody would event want in practice.
Does that mean that C# is low-level language? Is Java then? Is adding intrinsic enough to turn a language from high-level to low-level?
This is fine, it’s a term of art and those don’t need to be immediately obvious.
I don’t like the title of this article for that reason, though. Really a better title would be something like “a modern x86 processor is not a PDP-11.” The subtitle is perfect basically.
Edit: also IMO it is not really fair to beat up on C for this, the problem is not really one of low-level-ness. A language that actually exposed the complexity of speculative execution and all that could be pretty high level. It would just be harder to read in a linear text editor, right? We’d be better off drawing the dependency graph or something.
I actually believe Mojo is the only modern language not designed to pretend every computer is a PDP-11. C has been so successful that many succeeding languages just did C things as a matter of course.
In Mojo, everything is designed with the complexity of the modern computer in mind, and at every stage the programmer has complete control of outcomes. You decide what gets inlined, what gets passed in registers, what gets unrolled, etc. The language has excellent ... ney, probably the best portable SIMD support there is; all integers are built on top of SIMD, and the scalar integers are just SIMD with length of 1. You get complete control of what gets compiled as well due to powerful compile-time programming that is similar to, but more powerful than Zig's (imo, because you can supply a lot more information). While C and Rust allow inline asm, Mojo goes further by letting you supply inline MLIR and LLVM as well, so in situations that warrant it, you can tell the compiler to compile to a specific LLVM intrinsic. The language also does not assume you're compiling to run on just one machine; every modern computer is heterogeneous by nature and may contain multiple programmable units, so the compilation pipeline is designed to allow compiling certain parts of code for one target and other parts for other targets... as one compilation unit.
#define array(T, N) struct array##T##N { T value[N]; }
void copy(array(int, 32)* x, array(int, 32)* y) {
*x = *y;
}
int main() {
array(int, 32) x;
array(int, 32) y = { 1, 2, 3, 4 };
copy(&x, &y);
}
What does this mean?
I think defer is good. It's a common type of control flow that's usually done with gotos and mistakes, and I'd say it's slightly less complex than the for loop syntax.
---
C is a general-purpose programming language with features economy of expression, modern flow control and data structures, and a rich set of operators. C is not a "very high level" language, nor a "big" one, and is not specialized to any particular area of application.
But its absence of restrictions and its generality make it more convenient and effective for many tasks than supposedly more powerful languages.
Reason being, I don't see any fundamental difference between an x86 CPU that translates x86 instructions into simpler microinstructions for execution and a machine that translates, say, Haskell code into simpler microinstructions for execution, and I'd still consider Haskell a higher-level language than C.
Current ISAs have so much machinery underneath that it’s hard to tell when you’re talking to the iron and when you’re talking to the microcode
You can still argue that Forth on a Forth CPU is a genuinely low-level language. :)
IMHO, C is the lowest level a procedural programming language can get.
I don’t think byte code qualifies as human readable but it is closer to the metal obviously
GPUs can expose more of their internals thanks to shader compilation. They don't have to emulate previous-gen chip, and instead every chip can expose exactly what it supports and rely on software being recompiled for it.
What level would you say this language was? Is it a low-level systems language, or is it also usable for writing web sites?
To me the definition of low-level vs high-level strictly comes from the indirection the language runtime provides for you. If the language compiles down to asm, it's low level. It literally does _not_ matter what it looks like. The only other constraint is possibly whether you can manipulate low-level CPU level constructors like memory, albeit it's not necessary. You can take python and write an LLVM frontend for it and it would instantly become a low-level language.
So the question is if we really want lower level ISAs. Probably not?
There are many ways in which our current ISAs are actually thoughtfully optimized for superscalar out-of-order processors. Just look at all of the big differences from 32 bit arm to 64 bit arm, which all exist to make execution faster on superscalar processors.
And yet they are still perfectly implementable in cheap microcontrollers. The Cortex-A53, available in boards for a little over $15, is a simple 2-wide perfectly in-order design, without a physical register page beyond the ISA register. Basically, it is a simple Pentium-type chip.
The Apple M chips are some of the most impressive feats of out-of-order superscalar micro-engineering ever. And yet both of these can run the same software with the same ISA. This is enormously valuable.
I fail to see how any sort of much lower level access to the machine would be portable across price ranges and microarchitecture generations. I also fail to see how it would provide a non-trivial speedup over C code pattern recommendations and targeted extensions (eg. vector extensions).
That's the assumption that can be removed. GPUs don't have stable ISAs, and their assembly-like code gets recompiled for each microarchitecture.
In the Intel's world of prebaked machine code adoption of a wider set of SIMD instructions takes a decade+. In GPUs it's just a driver update.
As for GPUs, while the ISA is not constant, it's STILL C-ish running over a dynamic hardware scheduling layer.
Edit: To clarify, I have nothing against a closed ISA, I just don't see how making that ISA non-C-ish is valuable.
I would say Verilog is very much NOT a low level language.
Metaphorically, it feels closer to SQL to me. I mean this in that you theoretically tell the system what it should do, and it builds it into the messy real world. However, the reality is that the planner (sql) or linker/placer/router/whatever (verilog) are very good, but you often end up needing to actually fully understand the problem anyway when things don't work in the abstract.
I know there is https://clash-lang.org/ for Verilog design, which sounds a little like what you are talking about (never really looked at it myself).
My argument is that verilog is to FPGAs as assembler is to pdp11s, and C is closer to assembler than Java or prolog. High level languages are easier for programmers, but interestingly, the high level language Haskel readily compiles to FPGA. What's happening here?
Verilog is not a programming language (because FPGAs are not programmed) it's a hardware description language. It's also very lossy (every vendor has reams of coding guides for using it just right with their synthetizer).
This is embarrassingly bad.
pre- and post- increment operators cleanly lined up with... the programmer's conceptualization and objectives--the index is/was frequently used in other contexts than loop bounds and indexing. if that's not your conceptualization, don't use that operator. whether you are on a PDP-11 makes no difference.
> GPUs achieve very high performance without any of this logic, at the expense of requiring explicitly parallel programs.
GPU cores are in some ways closer to "PDP-11", they're either acting as thousands of parallel simple processors, or expose pretty raw instructions for very parallel use-cases.
So no, Mojo wouldn't be low level. It can't be.
But even before you get to out-of-order/speculative execution, I think most languages lack good (i.e. non-intrinsic-based) support for wide registers or anything SIMD related. I know C++ and Rust are both working on this
This didn’t quite work out in the long term since hardware evolves faster than ISAs. Today’s “maps directly to the hardware” instruction is tomorrow’s “we add more hardware and play tricks to make this faster.” You explode all of the physical registers as logical registers, then a few years later you double the physical registers count and do clever mapping to extract more speed.
My favorite is the MIPS branch delay slot. Instead of complicated branch prediction to hide latency, expose the pipeline directly to the programmer. And then a couple of hardware generations down the line, the pipeline becomes much longer and more complicated and the CPU is back to playing tricks to hide latency, and the weird branch delay slot remains as essentially a vestige of bygone days.
do you mean low level but higher level than assembly language for those processors (like MIPS assembly for an R10k, for example) ?
The article mentions assembly once. But it's not an argument about how assembly is "low level" and C isn't, although it may sound like that, upon a first reading, given the article's contentious tone.
The article is really an argument about how C programmers believe, and constantly state, that they are programming "close to the metal", but what they are really programming is a very fast PDP-11 emulator with lots of implicit behavior.
Implicit behavior like speculative execution and asynchronous execution and lots and lots of caching.
It is, though:
> Think of programming languages as belonging on a continuum, with assembly at one end
The article explicitly states that assembly is the end of the continuum, that it is the lowest of the low-level. Therefore, it is not making the argument that assembly is not low-level. But the exact same arguments it makes to distinguish C as not low-level can be applied to assembly. The entire article is based on a fundamental logical error.
This is a 100% skill issue of the author, as is always the case. C does expose it fully, except it's implicitly implied by your code rather than explicitly declared. Same with all of the other arguments that always plague these type of articles.
Assembly expose instructions that C was never meant to work with. Compilers force C to do so anyway. If you had a compiler that converted 8086 x86 assembly to modern x86 or CUDA bytecode; I'd consider that pretty equivalent.
LLMV intermediate representation is probably more low level (closer to the real compute model it runs on) than that theoretical 8086 x86 compiler.
I understand what you are getting to here, and agree that we are getting into the domain of semantics. Yet it could be argued that (for the most part) there is a 1:1 mapping between assembly and the assembly processes is (mostly) reversible. Personally, this is where I would draw the line.
> So we either descend endlessly into pedantry arguing that cosmic rays and ...
I think I see where you're going, though I don't agree with the particular example. If you're saying that machine language is an abstraction in itself, that those sequences of 1's and 0's are a construct of electrical engineers to describe electrical pulses that control transistors in a chip, then I fully agree with you. And if you say that those electrical pulses and transistors are themselves abstractions of physical processes, then I fully agree with you. But I wouldn't really describe it as nerd sniping. These involve different disciplines that are examining the machine at fundamentally different levels.
The part I find tedious is C programmers who cling to the language by claiming it lets you understand and finely control what the machine is doing, when it clearly does not, because x86 assembly itself is being emulated by the cpu underneath. To me, anyway, that's where I find some credence in the "C is a high-level language" meme.
It can be "low-level", but it depends how you define what that means in the age of microcode abstractions.
Most people shouldn't write in C, but that is because their skills belong in Application user space. =3
Mojo has exactly this with the SIMD struct: https://mojolang.org/nightly/docs/std/builtin/simd/SIMD/
You can place LFENCE(x86)/CSDB(arm) around code blocks, but you can do that in C too.
Yes, Java and C# are low level languages.
Individual instructions assume the data they operate on is of a particular type, but it doesn't differentiate data types in memory. Here's an example where I forced the C compiler to treat the bit pattern of two floats as integers, then add those values as integers. The result is, of course, absolutely meaningless.
float dx = 1.0;
float dy = 1.0;
int *pix = &dx;
int *piy = &dy;
int isum = *pix + *piy;
float *pdsum = &isum;
printf("%d\n", isum);
printf("%f\n", *pdsum);
The bits are untyped, the choice of operation decides how the bits are interpreted. Nothing enforces the type of that result, you can always interpret it as something else. It may not be meaningful. Or it may be, like a fast inverse square root.
Strong typing means that each value has an intrinsic type, and there is no reinterpreting it. What CPUs do is not that, or rather the only types are "_n_-bits" (_n_ a power of 2).
btw a bit of nitpick: to be fair basically no one uses x87 anymore, it's https://www.felixcloutier.com/x86/cvttss2si and friends but yes :)
Otherwise, no.
We even have our Herb Sutter: Jens Gustedt
Again -- I think your impression is the result of the contentious tone of the article. Yes, the article explicitly states:
"Think of programming languages as belonging on a continuum, with assembly at one end and the interface to the Starship Enterprise’s computer at the other. Low-level languages are “close to the metal,” whereas high-level languages are closer to how humans think."
See the very next section "FAST PDP-11 EMULATORS"
"The root cause of the Spectre and Meltdown vulnerabilities was that processor architects were trying to build not just fast processors, but fast processors that expose the same abstract machine as a PDP-11. This is essential because it allows C programmers to continue in the belief that their language is close to the underlying hardware."
See also the section "IMAGINING A NON-C PROCESSOR", where the author explicitly discusses alternative processor designs (which would of course require new assembly languages!).
The author is actually trying something like a reductio on your mental model. When the author states "Think of programming languages as belonging on a continuum", the author is really saying "This is everyone's impression, but ... when you look a little deeper you see the cracks (which are actually contradictions)."
I'm really not certain that's the idea, and it certainly does not feel very charitable. Perhaps you are holding on a little too tightly to this high vs. low level distinction (the simple mental model being attacked)? I compared the author's argument to a reductio. A reductio is not intentionally misleading?
This paper reminds me of "It's Time for Operating Systems to Rediscover Hardware". See: https://www.youtube.com/watch?v=36myc8wQhLo
There, an argument is made that our simple model of "the machine" is also wrong. There, the speaker points out much of the software that is running on our complex SoCs, with multiple cores, is firmware. To which, I'd imagine you might argue: "But that firmware is not the OS?! This talk is misleading!"
> If the author knew this, and his central premise were that no low-level programming language existed anymore, the article would be titled differently and he wouldn't be making the arguments against C specifically.
I am not sure. I believe the reason C is targeted specifically is because C communities are where this myth, and its religiosity (!), is the strongest.
> But this is essentially packaged as clickbait
I'd agree that the article is provocative, but it would seem to have good reason to be. Lots and lots of people think both C and our processors must work one way. That there is or was some level of naturalism/determinism at play. The author is simply pointing out -- not so much.
> After writing out my charitable interpretation of the author's capabilities, this interpretation only leaves me more disgusted with the article as a writing output.
Yes, it was designed to make you mad. But if you were forced to write a rebuttal to the entire article, from a charitable POV, I think you'd see there is some value to the reader in realizing this tight coupling (C and processor design) is not a necessary condition.
C being a low-level language is not a myth if, as normal people do, you consider assembly languages to be low-level languages. The article itself offers assembly as the low-level language and makes no effort to redress this later.
If you want to argue that there are no low-level languages when it comes to programming modern CPUs, you are free to do so, but that is a different argument. And it is arguably not a fruitful one because then the terminology loses all meaning. Okay, you've defined assembly as a high-level language. Now what have you accomplished other than making it harder for people to articulate and categorise classes of languages? We'll still need an adjective for distinguishing between such wildly distinct languages assembly and Python, so we have to come up with something else to replace "low-level" and you haven't accomplished much of anything at all.
If you think low-level programming has gotten too far from the bare metal, or if you think processors are designed for C and that's a problem, just argue that directly instead of this really torturous detour singling out C and how people use terminology to communicate useful concepts.
I'd suggest you're holding that stick too tight!
> If you want to argue that there are no low-level languages when it comes to programming modern CPUs, you are free to do so, but that is a different argument.
Not if you read the whole article?
> And it is arguably not a fruitful one because then the terminology loses all meaning. Okay, you've defined assembly as a high-level language.
Again, what if the terminology isn't very important? I'd argue the distinction between high and low level languages is not an important distinction, because it is so crude. For example, C/C++/Rust have all been described as high level languages at one time or another. C people seem to be the only ones that take real offense to this.
Then also, where new programming methodologies such as CUDA are invented to allow new processor designs, it turns out that they can be quite successful despite moving away from C. But then it also turns out that this programming model was actually not that great to program in, and people try hard to move back again by making the hardware more capable.