> Why Microsoft’s Motive Wasn’t Deliberate Sabotage

I absolutely do not agree.

Not only is the standard overly complex, Microsoft also indulged in all sorts of unscrupulous activities to corrupt various National Standards Organisations to get it approved through the ISO <https://en.wikipedia.org/wiki/Standardization_of_Office_Open...>, which is clear evidence of malicious intent.

This is a quote from Richard Stallman:

> The specifications document was so long that it would be difficult for anyone else to implement it properly. When the proposed standard was submitted through the usual track, experienced evaluators rejected it for many good reasons. Microsoft responded using a special override procedure in which its money buy the support of many of the voting countries, thus bypassing proper evaluation and demonstrating that ISO can be bought.

    If ISO were to give OOXML with its 6546 pages the same level of review that other standards have seen, it would take 18 years (6576 days for 6546 pages) to achieve comparable levels of review to the existing ODF standard (871 days for 867 pages) which achieves the same purpose and is thus a good comparison.

    Considering that OOXML has only received about 5.5% of the review that comparable standards have undergone, reports about inconsistencies, contradictions and missing information are hardly surprising.[118]

Microsoft seems to have known that they could ram basically anything through a standards body, so they presumably didn't bother to actually try and simplify the standard. Instead, it's basically an XML serialization of their older binary formats, complete with all of the quirks and bugs that have to be emulated for 100% compatibility.

To be fair, we're talking about a product line with over 35 years of history here. Cruft in the format builds up but can never be removed, so long as you commit to strong backwards compatibility - which Microsoft has always done.

Fun trivia: many of the old binary formats use a meta-format called OLE2 (Object Linking and Embedding). The file format is a FAT12 filesystem packed into a single file, with a FAT filesystem chain, file blocks aligned to a specific power-of-two size, etc. This made saving files very fast, but raised the possibility of internal fragmentation (where individual sub-files are scattered over many non-contiguous blocks); hence, users were recommended to "Save As..." periodically for large/complex files to optimize the internal storage.

I love this screen that shows you exactly why they named it “Office Open” XML: https://i.imgur.com/hnj3sdv.png

It was a pretty big deal when OpenOffice.org's 2.0 release came with OpenDocument as the default file format. Very easy for someone to misread this MSOffice screen and click on OOXML expecting it to mean OO.o.

> In my view, OOXML is indeed complex, convoluted, and obscure. But that’s likely less about a plot to block third-party compatibility and more about a self-interested negligence: Microsoft prioritized the convenience of its own implementation and neglected the qualities of clarity, simplicity, and universality that a general-purpose standard should have.

The author only provides arguments for "self-interested negligence". He provides no counterarguments to the claim that OOXML complexity was "a plot to block third-party compatibility". Therefore, he cannot compare "negligence" and "a plot". Therefore, his claim that "negligence" is a better explanation for OOXML complexity than "a plot" cannot follow.

To restate:

> If we dig into the context of OOXML’s creation, it can be argued that harming competitors was not Microsoft’s primary aim.

The author provides no evidence to support this claim. At most, the evidence provided in this section at most supports the claim that "negligence" played a role in OOXML complexity. From this evidence alone, no conclusions can be drawn about the "primariness" of "negligence" vs "harming competitors".

Worth keeping in mind that the native MSO formats were using "structured storage", a horrible binary chunked serialization and metadata format from an era where binary embedding of document streams in other application documents via "Object linking and embedding" (OLE, see also Apple's OpenDoc format) was deemed desirable, with zero consideration given to third-party apps and segment formats tied to C++ data structures. Compared to that, OOXML is still a huge progress, and while it's complex I wouldn't say it's maliciously so.

The Shakespeare example is a good one where the sentence is split into multiple spans to apply style rules yet the bare text content could be extracted by just removing all XML tags. Whereas the ODF variant is actually less recommendable as it relies on an unneccesarily complex formatting and text addressing language on top of XML.

The article says

> Even at a glance [ODF's markup] is more intelligible. Strip the text: namespaces and it’s nearly valid HTML. The only thing that needs explaining is that ODF doesn’t wrap To be with a dedicated “bold” tag. Instead, it applies an auto-style named T1 to a <text:span>, an act of separating content and presentation that mirrors established web practices.

but this definitely makes things more complex for data exchange compared to OOXML.

If you ever write some HTTP endpoint where tabulated data is returned, you could quite reasonably return RFC 4180 style CSV.

However, if your API ever interfaces with users in a corporate environment, parsing simple comma-separated UTF-8 CSV is suddenly quite beyond the reach of however is nibbling at your endpoint, so why not code up a simple little reusable bit of code where you can write any simple tabular data (string, numbers, and dates, in one or more sheets of data made up of rows and columns) that lets you choose the output format? A zip-archive of CSV-files (one per sheet), JSON, ODS, or XLSX; pick your poison.

I did just that, and while it is perfectly doable, any low-level, low-resources, low-dependency approach will mean actually touching the XML in LibreOffice's ODS (fine), and Microsoft's OOXML (…).

This is how you write a date in a cell in both.

ODS:

            <table:table-row table:style-name="ro1">
                <table:table-cell office:value-type="date" office:date-value="2021-04-10T12:34:56" calcext:value-type="date">
                    <text:p>10/4/2021, 12:34</text:p>
                </table:table-cell>
            </table:table-row>
            

OK, a bit verbose, but trivial to implement. Format the date however you like — you'll probably use two different formatters on the same datetime instant.

XLSX (OOXML):

            <row r="1" ht="12.8">
                <c r="A1" s="1" t="n">
                    <v>39448.5</v>
                </c>
            </row>

Obviously, as you can all plainly see, the date here is 2008-01-01T12:00:00…

And of course it makes perfect sense to hardcode the cell coordinate there. It's not like you would dynamically generate a bunch of cells (…).

The OOXML fight is near and dear to my heart because, when it happened, I was a baby developer, and I cared about the issue for some reason I can barely recall, and I found an expert on the issue on Twitter. That guy would regularly tweet about everything that was going on and the problems with the spec and the shenanigans, and I was one of the, like, 20 people who was hanging on his every word. And sometimes he'd talk about bee keeping instead. It was my first introduction to Twitter at its best. You got these unfiltered whole views of the lives and concerns of real people who were, in part, experts at what you cared about. So sometimes you had to listen to them talk about other random stuff they thought was neat. And that's great!

You want to see something "fun" check out Apple's Numbers XML format from before they switched to protobuf. A simple 1,2,3 resulted in over 1 megabyte of XML.

At a previous job I'd been tasked with developing support for importing Numbers files along side our existing Excel and CSV support. After a couple days we rightly gave up as the tiny fraction of people who actually wanted to import Numbers files was outweighed by it's massive complexity.

We ended up just adding instructions for Numbers users to export to CSV

It's as complex as it needs to be to losslessly convert old binary office files.

A better format would have made us geeks a lot happier, but the average user just wants things to work the way they always have.

I remember reading an early criticism of the spreadsheet side of OOXML, where a simple spreadsheet with three cells was created: A1 containing '1', B1 containing '2', and C1 containing the formula A1+B1. That spreadsheet was saved, the file opened in an editor which showed the values of the cells, and A1 changed to something else, say 3. This broke the spreadsheet, as there were all sorts of knock-on effects contained in the virtually opaque mess that followed the cell contents.

I've probably got the details wrong, but that was the gist of it. I'd love to rediscover the analysis, but my searches have not yielded it.

Dead on.

Microsoft is just dominant and exporting its 40 year old legacy codebase as a spec. LibreOffice team is frustrated that the for-profit model is beating the OSS model and crying foul over mostly necessary complexity. If LibreOffice started from scratch they’d probably appreciate how much Microsoft serializes because a sufficiently complicated document saved to .docx basically provides a reference implementation.

We do need for-profit alternatives to Word, and I’m working on one in legal.

[edit: I hope to put some real thoughts on this down soon, but most of the wonkiness emanates from evolving functionality and varying trends in best practices over the decades. I’ve implemented a fair bit of the spec here: https://tritium.legal, but most of the hard part is providing for bidi language support, fonts, real-time editing and re-rendering, UI and annotations like spellchecking and grammar, not conforming to the markup spec. Spec conformance is just polish and testing. A performant modern word processor of any spec, however, is a technological achievement on the order of a web browser.]

I once digged through the 5000 page specification. There was a lot of useless stuff that only old Microsoft Word supported like WordArt items.

People who were developing "office" programs in the early 1990s were thinking about the problem of serializing arbitrary object graphs into documents to support technologies like

https://en.wikipedia.org/wiki/Object_Linking_and_Embedding

where you could embed an Excel spreadsheet inside a Word document or actually embedded any of a large range of COM objects into a Word document which on one hand is a really appealing vision but on the other hand means you have to have and be able to run all the binaries for all the objects that live in a document which ties the whole thing to Windows.

PDF is a different sort of document format which privileges viewing over editing but it is also really about serializing an object graph when it comes down to it and then having various sorts of filters and transformations and a range of objects defined in the spec as opposed to open ended access to an object library.

This kind of system has a lot of overlap with the serdes problem you get with RPC frameworks that used to be under the files "Sun RPC sucks", "DCOM Sucks", "CORBA Sucks" and "WS-* Sucks" Those things are mostly forgotten these days because well... they sucked, and now the usual complaint is "protobuf sucks" but you rarely hear "JSON sucks" because it gave up on graphs for trees, if you don't have a type system people can't say the type system sucks, and the only thing that really sucks about it is that people won't just use ISO 8601 dates but you can always rise above that by just using ISO 8601 dates without asking for permission. But we all agree YAML sucks.

That points to any flexible document format sucking but also sucks because it has lots of poorly specified and obscure features that amount to "format this the same way Word 95 formatted it if you used a certain obscure option".

From a glass is half empty perspective it sucks because it's close to impossible to make a Microsoft Office replacement that renders 100% of documents 100% correctly.

From a glass is half empty perspective it rules because if you want to make a Python script that writes an Excel script with formulas it is easy. If you want to extract the images out of a Word document it is easy because a Word document is just a ZIP file. If you want to do anything with an OOXML document short of writing an Office replacement it's actually a pretty good situation.

I guess OP wasn't around back then when Microsoft openly bribed African countries to join ISO just to vote on this topic.

This says it all: https://cdn.imgpile.com/p/RppGj1l

I remember Spreadsheet ML, an older format compatible with Excel. It had a subset of features, I think, but it was a rather powerful subset: formatting, formulae, multiple sheets. And it was rather simple. (Had a silly design mistake: for some reason MS gave namespace to attributes, which is not necessary, only for rather specific purposes).

Another XML standard from MS that also seems relatively simple is XPS, a PDF alternative. But it uses Open Packaging and that is somewhat hard to read.

I have no idea what this article is intending to express. It is artificially complex to dump the exact implementations of your legacy products into a giant data structure and call it a standard. Nobody can implement that. Which is why they had to bribe, stuff committees and bully people to get it done.

I don't think anyone cares about debating the word "artificial," I don't think that was anyone's point. It's just not a standard. It was, as is made clear here, a way to head off a standard that would be possible to competitors to implement with a fake standard that Microsoft couldn't even implement.

I also don't think that it is "a counterproductive reflex that’s common in open-source circles: scolding users for accepting proprietary tech." I don't even know wtf that's supposed to mean. People are stuck with it because of corruption, they're not being scolded for using it.

edit: "LibreOffice itself, as ODF’s flagship, still suffers from rough edges in design, interaction, and performance. As a result, even as Office hobble itself with bloat, most people still find it easier."

Yeah, it'd be a lot easier if they didn't every have to deal with OOXML and could just work on their own product.

Microsoft Office has many features. Each feature must be reflected in the file format somehow.

(I wonder what the specification-pages-to-man-years ratio is...)

https://m.slashdot.org/story/78708 (2007)

Microsoft just took what they had and directly translated it to XML. It's not intentionally messy, it's just a big corporation with old product acting like it.

I think the last part is probably the biggest thing holding them back IMO... I tend not to install MS Office products on my personal devices, I haven't run Windows on a personal device in a few years. I've mostly maintained just my resume in word or libre-office format for well over a decade. I can't tell you how many times the LO format lost formatting, or just messed up between version upgrades. Same goes for opening a word version in LO.

That doesn't count the various times where it behaved weird, inconsistently had fields/tables that were impossible to edit, etc. I've had to completely recreate everything a couple times over the years. That's just one document, for one guy that I don't really touch that often.

Say what you will about Firefox vs Chrome in terms of usability, compared to MS Word using LibreOffice is worse than early betas of Netscape Navigator 4.0. It's both impressive and upsetting. OnlyOffice at least looks nicer, even if it doesn't really function any better. MS's online version of Word in the browser operates more consistently than either.

> Faced with demands for openness, Microsoft could have produced a clean, modern spec and keep the mass pile of legacy inside the application.

Very, very few people care about openness. Maybe a few hundred. Tens of millions care about docx capturing exactly what their doc files had.

Microsoft made the correct choice.

I spent a lot of time last year replicating every valid Excel number format. I've really struggled to find good documentation on the excel format when you really get into the weeds.

The use of namespaces is also incredibly annoying in so far as I can tell in every xml library I can find they really aren't well supported for that "human" readable component.

When you crack open the file it feels like you are going to be able to find everything you need with an xpath like //w:t but none of the xml parsers I've found cope well with the namespaces.

   result = doc.findall("//w:t", namespaces={"w": "..."})

    var navigator = doc.Root!.CreateNavigator();
    nsManager = new XmlNamespaceManager(navigator.NameTable);
    nsManager.AddNamespace("w", "...");
    var results = doc.Root?.XPathSelectElements("//w:t", nsManager);

sigh Just because it was not deliberately engineered to be prohitibively expensive to support does not mean that it can not be used to deliberately obstruct interoperability. It's really not that difficult a concept: if you want others to suffer, you can take a sad artifact of well-meant historical accidents, and say "welp, now it's a standard, you gotta support it!" There is nothing contradictory or conspirational.

Unless we ever get the full archive of Microsoft emails, meeting minutes and recordings from all the secret microphones they didn't have in their meeting rooms, I don't think you can ever disprove this claim. It's generally impossible to conclusively disprove conspiracy theories, because you could always claim you're only showing there are no documents proving the conspiracy, but there are no documents disproving it.

The author is just implicitly appealing to Occam's razor here, as people often in face of accusations of a plot. They can show that Microsoft has backed the ANSI accreditation of ODF[1] and eventually implemented support for ODF import and export in Office, but that's not enough to prove there was no conspiracy.

Instead, the article just provides a very plausible explanation for the complexity in OOXML. Does this explanation thoroughly disprove the accusations of a plot? Clear not. Is it more plausible than a great plot to crush a bunch of competitors that had no market share and kill a better standard document format that Microsoft did end up implementing in Office? Yes. This is probably as far as we can get.

[1] https://news.microsoft.com/source/2007/05/16/microsoft-votes...

I mean sometimes you gotta ship a product (and remember back then, that meant masters for CDs,) and it's perfectly possible that whatever team was in charge of handling 'conversion' stuff for old format (remember that old excel formats have OLE type cruft going on, the sorts of things that led to VBA viruses, imagine what other functionality needs to be implemented) just plain had to take shortcuts in uglifying the spec to support all the jank.

Can you explain what's wrong with the concept of a container format that allows embedding subdocuments of different types?

> zero consideration given to third-party apps and segment formats

The reality is the opposite. COM serialization was specifically built to allow for composing components (and serializations thereof) that didn't know about each other into a single document. That's why it leans so heavily on GUIDs for names: they avoid collisions without needing coordination. That's a laudable goal, not pointless bloat. And the COM people implemented it pretty efficiently too!

> C++ data structures

What gives you that idea? Yes, the OLE stream thing was a binary format, but so is DER for ASN.1. Every webpage you load goes over a binary tagged object format not too different from OLE/COM's.

But due to a persistence of myths from the 90s, people still think of the Office binary format as "horrible" when it's actually quite elegant, especially considering the problems the authors had to solve and their constraints in doing so.

In many ways, we've regressed.

> Markup

The author of the article nails it when he says ODF is meant to be a markup language and OOXML is the serialization of an object graph. So what? Do people write ODF by hand? There are countless JSON formats just as inscrutable as MSO's legacy streams.

Anyway, the idea that the MSO binary format was crap because it was binary, lazy, and represented a "memory dump" is an old myth that just won't die. It wasn't a memory dump, it wasn't lazy, and it wasn't crap. Yes, there are real problems with some of the things people put inside the OLE container, but it's facile and wrong to blame the container or the OLE stream composition model for the problem.

IIRC Adobe's PSD file format is similar, which made it very very complex to reverse engineer on one side - and vulnerable to exploits on the other side.

> However, if your API ever interfaces with users in a corporate environment, parsing simple comma-separated UTF-8 CSV is suddenly quite beyond the reach of however is nibbling at your endpoint

Excel can directly ingest a CSV file served over an URL as data source, with the Accept header manually set to text/csv.

I wrote a backend once that supported this feature so that management could pull whatever data they wanted off an internal application without pestering me. They could literally take the URL of a page and pull it as a CSV file as-is.

Anybody who knows a bit of Excel can pull that data themselves by following a set of simple instructions.

I was not aware there was an RFC for CSV, but the concept of "simple comma-separated UTF-8 CSV" is, in my experience, not something that exists. In a previous job, a chunk of my work was taking CSV files that were given to us and writing tooling to process them into a structured form for import elsewhere (typically we'd do a few test runs, and finally do a cut-over with final data, so it had to be scripted.)

During this I saw just about every variant of CSV and character encoding known to man, often inside the same file. Once I had a file that had UTF-8, MARC-8, Latin1, and (yes really) VT100 control codes. All in one file.

All in all, I'd prefer something that actually could be validated for some sort of correctness (this said, another time I got an XML export from some software that was invalid XML, so...)

Storing dates as numbers in a spreadsheet has been a thing since the first spreadsheet program I know of. Microsoft picked "days since 1900". If you're on UNIX, you may prefer using "1199188800" instead.

Other than that, the difference is pretty minor. ODS is very verbose and stores the content of the cell twice for some reason, but the XML trees are essentially the same.

The best way for corporate interaction is to export to whatever the hell Microsoft Excel accepts as an external data source, because .xslx files can natively import remote data that way. Hope your customers' computers are all configured for en_US mode, though, because CSVs aren't as universal as people pretend they are.

I feel like Libreoffice became largely irrelevant the day Google Docs came out. People put up with LO wonkyness because it was free and office was expensive.

Google completely flipped the game and then cloud collaboration became everything.

LibreOffice has versions that you pay for, with support. The most prominent is Collabora, which is a (if not the) biggest contributor to LibreOffice.

Where does the article say it’s a necessary complexity?

> Thus, the primary goal for this new format wasn’t to be elegant, universal, or easy to implement; it was to placate regulators while preserving Microsoft’s technological and commercial advantages.

That sounds quite anti-competitive to me

> We do need for-profit alternatives to Word, and I’m working on one in legal.

Wow, big undertaking!

What we really need, though, is a for-profit alternative to Excel, that's not Google. I think Excel is more of the Killer App than Word has ever been.

This may be nitpicking, but the complexity in OOXML is not "necessary", at least not in the sense of what Fred Brooks would call essential complexity. As OP clearly demonstrate, the complexity in OOXML is not artificial: there was never some grand conspiracy by Microsoft to create a format that competitors will find it hard to implement.

But very little of this complexity is necessary for a standard interoperable document file format. The background was that the EU started pushing for a standardized document exchange format, and several governments started implementing regulations requiring the use of this format — Microsoft now had some very big customers which urgently needed a feature: a standard document file format. Microsoft _could_ have implemented and submitted a new format that doesn't include slavishly reflect their in-memory object graph and legacy issues. Or they even could have just adopted ODF (shudder). But they've chosen the easy way, because, frankly, they probably just didn't have the time. They took the accidental complexity which was the hot mess Microsoft Office internals (like a buggy date format) and serialized it to disk. It was never an ideal solution, but this was quick to implement.

That's just a classic case of technical debt: Microsoft needed to deliver a feature fast, and they were willing to make compromises. The crazy political shenanigans Microsoft had executed to standardize their technical debt are ironically just another form of accidental complexity.

> If LibreOffice started from scratch

What do you mean though? Libreoffice wrote their application from scratch, did they not? And they managed to implement a superior serialization format, did they not? And they managed to get that format standardized without bribing and cheating, did they not?

What you're saying is akin to "those residents of banana republics are just frustrated capitalism (and a little help from the CIA) is beating democracy"

> We do need for-profit alternatives to Word

Why does it have to be for profit?

> LibreOffice team is frustrated that the for-profit model is beating the OSS model

Let's take a look at this "for-profit model" - is it just higher price outweighed by better product? lol:

Microsoft, after getting beat up in the press for making propietary extensions to the Kerberos protocol, has released the specifications on the web -- but in order to get it, you have to run a Windows .exe file which forces you agree to a click-through license agreement where you agree to treat it as a trade secret, before it will give you the .pdf file. Who would have thought that you could publish a trade secret on the web? - https://slashdot.org/story/00/05/02/158204/kerberos-pacs-and...

Back in 2001, Be, Inc. managed to get BeOS pre-installed on one computer model from Hitachi. Just one. On the entire PC market. Microsoft forced Hitachi to drop the bootloader entry to hide BeOS from customers buying it. They enforced their monopoly over the only possible niche BeOS could find on the PC market, crushing Be, Inc. in the process. - https://www.haiku-os.org/blog/mmu_man/2021-10-04_ok_lenovo_w...

So why aren't there any dual-boot computers for sale? The answer lies in the nature of the relationship Microsoft maintains with hardware vendors. More specifically, in the "Windows License" agreed to by hardware vendors who want to include Windows on the computers they sell. This is not the license you pretend to read and click "I Accept" to when installing Windows. This license is not available online. This is a confidential license, seen only by Microsoft and computer vendors. You and I can't read the license because Microsoft classifies it as a "trade secret." The license specifies that any machine which includes a Microsoft operating system must not also offer a non-Microsoft operating system as a boot option. In other words, a computer that offers to boot into Windows upon startup cannot also offer to boot into BeOS or Linux. The hardware vendor does not get to choose which OSes to install on the machines they sell -- Microsoft does. - https://birdhouse.org/beos/byte/30-bootloader/