Standardizing source maps

saagarjha 68 days ago |

I wonder if the source map people could learn a thing or two from the debuginfo maintainers and vice versa.

Probably. The RADDBG folks recently created their own debug format (RDI) to replace PDB and DWARF. Could be worth a gander.

https://github.com/EpicGamesExt/raddebugger?tab=readme-ov-fi...

VorpalWay 68 days ago | | |

While I have never worked with PDB, I have worked directly with DWARF. It is an insane format. It embeds (at least) three different byte code formats that need to be interpreted. One of them is even Turing complete.

First up is mapping from address to file, line and column. This one is basically a custom data compression scheme in the form a custom byte code. Strange but not too bad.

Second is “DWARF expressions", which is Turing complete and used for many things, such as figuring out where in memory or registers a given high level variable can br found at at any point of the program execution. It is baroque to say the least.

Then there is EH frames, which is used for unwinding (on exceptions in C++ or panics in Rust for example). This is used to specify how to find the base of the current stack frame given the current instruction pointer. This is needed if you don't use frame pointers. In itself it isn't Turing complete, but it can call out to Dwarf Expressions as subroutines, so it actually is TC. Except from what I have read, no compiler actually makes use of that capability, thankfully.

Surprisingly, the DWARF specification itself is actually reasonably readable and well written.

tliltocatl 67 days ago | | |

> One of them is even Turing complete.

> figuring out where in memory or registers a given high level variable

Isn't the task itself Turing-hard? Or at least complex enough so that coming up with a non-Turing-complete solution would be impractical?

VorpalWay 67 days ago | | |

Good question, that I don't fully know the answer to. The rest of the byte code (apart from the primitives that enable looping) already allow expressing a lot. From memory (it has been almost half a year since I last worked on this), you can specify things like "for this 32 bit value, the first two bytes can be found in the middle of RAX, the third byte is found following this chain of pointers, and the final byte is on the stack" without even touching the TC parts.

Basically, my impression was that that the format was flexible enough that I couldn't see why you would need the TC parts in practice. The compilers seemed to agree and not use it in practise (at least gcc and llvm).

This was of interest to me since I was generating BPF code from these (for user space trace points) and BPF is famously and intentionally not TC. I could translate many patterns that do show up in real world code, but not the general case.

saagarjha 65 days ago | | |

This is largely because debug info is not great and does not generate the Turing complete counterpart to your code so that variables do not get optimized out. In the general case this is required.

cadamsdotcom 68 days ago |

Very cool to see so much work going on to make the web platform even more awesome - and in the open!

Stuff like this makes me believe open wins over closed in the end :)

sureglymop 68 days ago |

This is a great endeavour. Recently I have been thinking about how to add syntax and metaprogramming extensions to programming languages without forking the compiler/interpreter. Source maps are needed there in order to have good editor support through e.g. an LSP server proxy. In researching it I was a bit let down I couldn't find too much research and specifications for the topic.

conorh 67 days ago |

Great article, never realized just how adhoc the source map 'standard' was!

skybrian 67 days ago | |

There was a well-specified design doc and Chrome implemented it. It worked. Sometimes that's all you need to get a defacto standard.

This new standards process is making some useful improvements, though.

indolering 67 days ago |

Glad to see this finally getting some much needed love and attention!