I have learned to be scared of my MacBook. Seemingly safe behavior can cause permanent damage. It does completely unexpected things, apparently by design.

I do not put my pictures in the ~/Pictures directory for fear of what the newest app will do to “improve” them for me. I fully expect it to apply lossy compression to my files without asking. This is after Photos or whatever it was called at the time mangled the dates on a bunch of my vacation photos to 10 years before the actual trip.

Oh and have fun when your photos are automatically uploaded to iCloud to save space locally then silently deleted from iCloud to... save space? My sister lost her first year of baby pictures to that one.

Same with ~/Music after iTunes wiped out a bunch of carefully curated metadata. Yes, I did want that album art.

I fat-fingered some key combination in Messages recently and got a prompt confirming I wanted to delete the entire conversation history. I consider myself lucky it bothered to ask.

I can add “view a PDF” to the list of things likely to leave me holding the bag.

Why anyone treats PDF as anything but a write-once format is beyond me. It's so finicky that I'm not shocked bugs like this happen. The only programs I'd be reasonable sure wouldn't screw it up are Acrobat itself, and pdflatex and friends.

I think we need a multi-image container format. It could be something that's literally a bunch of jpgs/pngs/pick your poison in a tar container, and given a new extension. OSes would open it and present it as a gallery in order. There's no value in a non-ocr'd PDF existing. For OCR'd text that gets more complicated, but it feels like we should be able to come up with a common denominator that doesn't have the legacy of a binary format derived from postscript in the early 90s.

I work with a ton of PDF's between my Mac and iPad, and it mostly works but there are still just way too many bugs.

It's a lot of little things, like in Catalina where opening up the sidebar for annotations (comments) seemingly randomized their order. (Big Sur, fortunately, fixed it to be page-order again.)

Or how printing a PDF from a website (in Catalina, also seemingly fixed in Big Sur) would look right on the page... but if you copied and pasted the text from the PDF to somewhere else, something like 10% of the glyphs were scrambled ("lik3 thZs"), like some sort of character table corruption.

Or reading a PDF with Books on my iPad, maybe 10% of the time bookmarking a page... doesn't bookmark it. Or removing a bookmark... doesn't remove it. Or a handful of highlights you just made have inexplicably disappeared the next time you open the file.

Or whenever you open the PDF in Books it remembers which page you were on. Except sometimes it doesn't, so you can't really rely on that for saving your place.

Or in Books, if you select some text to copy but accidentally hit the adjacent "select all" in the pop-up menu, and you're dealing with a 400-page PDF, it just locks up and you have to restart it.

Or in Preview if you want to convert a PDF to black-and-white, there's an option for it but your PDF will balloon in filesize to 10x larger or something.

I mean, I could go on and on. It's weird, because Preview is an incredible app, really. But it really is like they build it and then never bother to test if basic workflows reliably work.

This is a clickbait, sensationalist headline. “Saving a PDF with Preview in Big Sur can corrupt OCR text added by a third-party program” is more accurate.

I find posts like this completely pointless when they include no details at all. This is just "there's an incompatibility between third-party software and a version of macOS that the third-party software says they don't support yet, so I'm going to publicly criticize Apple".

If you're not going to do the work to figure out what the corruption is, at least include the two PDFs so other people can look at them and see what happened.

My guess is that the output PDF is still valid, but that an embedded (subset) font has had its `ToUnicode` map stripped, so that there's no link between the character codes used in the text elements and the "actual" characters they represent (there are also other ways this corruption could happen, but dropping or mangling the `ToUnicode` map seems like a likely cause).

They are using software unsupported by the vendor and blaming Apple for the outcome.

“ABBYY says they don’t support Big Sur yet, that’s fine. But Apple didn’t tell me that I can’t upgrade to Big Sur when I use ABBYY. I’d be a lot less angry if there was a changelog or release notes from Apple where it says there is a known problem with OCR’ed PDFs in Preview. Their software is broken, they need to tell me. I don’t care if it only worked because they had workarounds for super shitty PDFs that ABBYY possibly produces, I just need my OS to keep working for me.”

I remember many years ago distributing PDFs as part of course material, that Adobe's official reader would open just fine, but Mac's built-in one wouldn't (and simply fail with a useless "an error occurred" message.) Only a small subset of the class was using Macs and the built-in reader, so it took a while to discover. The problem eventually turned out to be some oddity in the way it treats whitespace[1], that Adobe and a few other readers were perfectly fine with, but not Preview.

[1] PDF is one of the strangest file formats I've worked with. It is a bizarre mix of binary and text, and some of the other design decisions are also perplexing.

I think the HN crowd has forgotten that the entire legal system uses PDFs, and in addition uses the redaction features of the likes of Adobe Acrobat, as well as others trying to squeeze in like FoxIT.

Redaction is huge in governments that have gone digital. Gone are the days where you print the paper, black it out, and then photocopy it.

I have worked with PDFs for a long time, and if you ever wanted compatibility, you had to use Adobe Pro, since there were so many bad PDFs with weird embedded stuff that only Adobe could read properly...because it was initially created in Adobe sigh

All other products try to catch up, but they can't clean up the mess that Adobe has left behind.

Preview seems like a good example of something that's worth open sourcing. Not only will people end up doing work for you, you get eyes on the code and more direct issue tracking.

Consumers get a product and they still have to go on Mac to use it.

From what I can tell, there is no reason you can't just run the PDF through ABBYY FineReader again and get the exact same OCR you got the first time, so I think "irreversible" is a bit over-the-top.

Is it as easy as CMD+Z? No. Is it data you can never get back? Also no.

This happened to me in Catalina as well. This summer I was preparing the paper proceedings for a conference, which were made with InDesign. I had to remove a couple of pages from the output, did so with Preview, and from then on, the text was garbled on copy-pasting. Had to switch to using Acrobat for that step.

Preview for PDF manipulation was a nice try at first, until I realized I suddenly have unexpected problems with produced docs, trouble with drag-and-drop, overwritten files etc..

Now I am using PDFGenius and never looking back.

Let's be real. Every single macOS release, until it reaches x.y.4 or x.y.5 is just in beta and you are the tester.

I upgraded to Catalina when it hit 10.15.6, and I watched for the year since the release all the comments and posts about the horrible things it was doing to their computer, files, apps, etc.

Apple supports the latest 2 versions of macOS. Always be on the "previous" one is my advice. Since my family and friends started following it, they are much happier and more productive.

Let the masses beta test.

Apple has a lot of shit they need to fix in macOS and the accompanying apps.

That said, the author of this article is clearly an ass, and i have a hard time being sympathetic.

Assuming the pdf is actually in spec, which it's probably not, this shouldn't be happening. That said, if the 3rd party app vendor says the pdfs they generate are broken in big sur, that should tell you, they may be broken other places as well, and it's probably not apple's issue.

The text corruption doesn't appear to be random. The same word gets converted to the same corruption. It's more likely an encoding/decoding bug.

Preview used to be solid, but it has been increasingly fragile in recent years. I found PDF Expert to be a great replacement. I have no affiliation.

> You have to completely close the file and reopen it, only then will you realize that it has been destroyed.

At first glance, it’s a replacement cypher. Every ‘a’ becomes a filled square, every ‘b’ a ‘p’, every ‘c’ a ‘(‘, every ‘d’ a ‘)’, etc.

However, there are exceptions, for example the first ‘b’ on line 10. It becomes an ‘ä’ on line 21. I guess that’s because that is bold text, and thus a different font.

Once again, the Hacker News comments prove to be more useful and insightful than the article itself.

I had an issue recently where the form contents filled and saved with Preview.app would not show up in acrobat reader. I've encountered this in two cases so far, with two completely different documents.

I have encountered too man PDFs (mostly digital originals rather than OCRed scans) corrupted this way during the recent months. Now I see why...

I hate Preview's PDF editing features, I wish there was a way to turn them off.

I'm the kind of person who tends to randomly click on things as I read them. In other PDF readers, this is quite harmless. In Preview, it starts editing the PDF. 99.9% of the time I have zero interest in editing or annotating the PDF I am reading. And then when I quit it asks me if I want to save a copy. I never wanted to change it to begin with!

(Maybe it is time I found another PDF reader...)

anyone else not able to see sufficient details the tiny screenshots? What was the difference?

Using Apple devices in general seems like a total crap shoot to me nowadays because of the impossibility of down-grading the OS. Every "upgrade" comes with a considerable risk that something that had been working will stop working, and if that happens, you are pretty much SOL.

PDF is one of the best things IMO - it's like a docker container for documents. The way the original authors intended it to be, including fonts and all the things that go into making a document.

For example, I personally like to purchase books that are in PDF format, not epub/mobi. I want to rely on professional typesetting from the publishers, not some front-end engineer's vision of what the ebook should look like and how it should be presented to the user. It works only for novels and long form reading where typesetting is not critical. Basically any book that also has a good audiobook version works fine with epub/mobi since visual formatting is a non-issue. For everything else - programming books to research papers, PDF is great. Can PDF format be improved? Sure, but the level of adoption and its widespread use is more important than fixing copy paste and content migration aspects of PDFs.

What I absolutely DO NOT WANT - is web page like format with auto-flowing text and something that fits to the screen with user styling/typesetting. That IMO defeats the purpose of what the container is supposed to do, i.e. contain and not leak. It should most definitely have fixed physical dimensions (or pixels).

> Why anyone treats PDF as anything but a write-once format is beyond me.

sigh These kind of ignorant statemens really annoy me.

The massive part of PDF spec is dedicated to editable features (annotations, form filling and digital signatures) which are used by massive industries daily because it brings them a lot of value. It's REALLY not that hard to think for a second and remember why having a single file which can be:

- Edited, annotated and commented

- Approved with explicit markers

- Digitally signed by auditors and reviewers with tamper protection

- Rendered on anything that has a semblance of interactive UI with graceful degradation

- Archived on any kind of file storage

is insanely useful and valuable for massive worldwide industries.

In many industries, writable PDFs are commonplace and aren't seen to be a problem because people in those industries don't see any problem with using Acrobat. If you want to unseat Acrobat and PDF, I think you'll have to provide something that has equivalent or greater power.

As for multi-image container formats, I think we already have that: cbz/cbr. Just a zip or rar archive with images in it, to be displayed in order by name. This 'format' is in common use already for scans of comic books. There are numerous viewers you can use to open and browse these files. Something to consider though; accessibility. A screen reader needs to use OCR to read these files to people with impaired vision. PDF files aren't fantastic for screen readers either, but they're much better than just a JPEG. I'd love to see some sort of subtitle system for images (I think mpv could probably overlay a subtitle sidecar on a still image, but that's not widely supported. Text-based subtitles formats are easy to wire into text to speech though.)

The comic book piracy community in fact does have what you're asking for. The formats .cbz, .cbr, etc. are just zip and rar files respectively, with images inside and a standard format for the internal filenames so they're presented in order by your reader.

Would love something like that with svg allowed inside to support vector drawings.

Basic PDF editing (adding, removing, reordering, cropping and rotating) has been rock solid in Apple's Preview app. It's something that I dearly miss when I'm on Windows.

Which makes showstopper bugs like this a very unfortunate.

But then again, it's been common knowledge that you shouldn't upgrade to a .0 release...

Because you often have no choice.

You have a PDF of a book and you need to export the pages of a single chapter to a new PDF.

Or you have 30 different PDF's that you need to combine into a single one.

Or a full-color large-filesize scanned PDF that you need to convert to a smaller-sized black-and-white one.

Or you need to copy quotes from a PDF to paste somewhere else.

I could go on... PDF's are documents, and normal workflows involve all sorts of conversion and rearrangement and processing of documents. That's the whole point.

There are lots of image-container formats. DjVu is my favorite, and supports OCR text annotations also. But, outside of some niches, it has fallen out of use these days. Adobe closed the quality gap by adding jbig2 and jpeg2000 support to PDF, making it possible to build a "bunch of images" PDF with similar quality to DjVu.

Strangely enough (?) I have not had many problems with PDF, certainly no more than with other document formats. I often use tools like pdftk [0] (as found on many Linux distributions) to split sections out of PDFs, create new ones out of single-page PDFs created with ImageMagick, create odd-even page versions etc. I generally do not touch the more "advanced features" like embedded JS (which I have disabled in all readers which support it), I just use it as a document format which more or less guarantees the resulting document looks the way it was meant to, plus or minus a few fonts. For that purpose it works works well enough.

The "bunch of jpgs/pngs/pick your poison in a tar container" format you describe exists in a fashion: Comic Book Archive, a format meant for and mostly used for comics. It consists of a compressed archive which contains sequentially numbered "pages" which can be JPEG, PNG or other image file formats. For pure image documents it can be used as a replacement for PDF but since it does not support text it can not be used for scanned OCR'ed documents. DjVu [2] does support a text layer but that comes at the cost of complexity, it is far from the simple container you propose. Since an OCR'ed text layer needs to save not only the text itself but also the location on the image for each character I don't see any way to avoid complexity here.

[0] https://www.pdflabs.com/tools/pdftk-server/

[1] https://en.wikipedia.org/wiki/Comic_book_archive

[2] https://en.wikipedia.org/wiki/DjVu

Blind and shortsighted people may not like your idea. There are accessibility options in PDF, but even a regular one can be accessed with a screen reader. Your images won't.

> Why anyone treats PDF as anything but a write-once format is beyond me.

Because PDF is supposed to be like electronic paper and you normally can draw on a paper original.

In an ideal world OSes and apps would be better equipped to separate data and metadata and work with multi-stream files. Every file would have different kinds of pure data (e.g. scanned picture and OCRed text), metadata (e.g. title, author, ISBN/etc IDs, ToC etc) and user-generated annotations in separate streams. But our actual world is not ideal, we mix everything into one stream for every document and allow the apps modify it every time we view it.

We have that container format. It is called: TIFF

There already exist formats as you describe, but regardless, a change of format wouldn't fix anything. The same problem would just surface there, as Preview and whatever add support for those formats, and include editing features.

If you don't want to break something, just don't try to edit it. That can't be enforced anywhere but at the individual user level.

Even proprietary formats in first-party software have this problem. Hell, plain-text editors have this problem.

Making a PDF include OCR'd text seems to by definition require writing twice. It has to be edited to add the OCR'd text that was not in the original image-only PDF, no?

I mean, you can say that the second PDF is a different PDF that has only been written once, but you can say that about literally any edit, "write once" no longer means anything if you think about it that way.

> I think we need a multi-image container format. It could be something that's literally a bunch of jpgs/pngs/pick your poison in a tar container

You've just described the .cbt format!

https://en.m.wikipedia.org/wiki/Comic_book_archive

PDF is often called "electronic paper", meaning its primary use, typesetting documents on screen and paper alike.

I'd rather view that PDF is an electronic clay tablet. You can put any text or image on it, but once it's formed, you better not try to alter it, lest you break it.

What annoys me is that there are some .pdf manuals I want to refer to, and when I view them in preview and go to quit it asks if I want to keep my changes or revert them.

WTF? why does viewing a manual - and I only used navigation like page up / page down - cause modifications?

> Why anyone treats PDF as anything but a write-once format is beyond me.

Hahaha. Quite a lot of people around me think that PDF is a collection of JPEGs glued together (because they mostly see PDF as scanned non-OCRed docs).

"Why anyone treats PDF as anything but a write-once format is beyond me."

Some of use work in print publishing and being able to edit PDF's is critical.

Here's is one good reason: My company uses digital certificate signing on PDF. It is a very common usage. We have yet to migrate to DocuSign.

> The only programs I'd be reasonable sure wouldn't screw it up are Acrobat itself

Oh no, I've seen acrobat screw with it too

There are already a handful of multi-image container formats. They're all images inside zipfiles, of course.

We already have CBR/CBZ.

CBR and CBZ for comics fits this I think?

.PDF works just fine. Don't blame the shortcomings of crappy software implementations on the format itself.

We need a format that will still be readable 100 years or more from now, and .PDF serves that purpose.

> 10% of the glyphs were scrambled ("lik3 thZs"), like some sort of character table corruption.

As I understand it, that's a form copy protection trying to prevent exactly what you're doing.

That not remembering the last page you were at in Books drives me crazy.

My iPad would be the perfect device for reading PDF books especially technical and maths text books, if weren't for the Chinese torture of whether it will remember your page the next time you open it or will I have to scroll through until I recall where I left.

That's fair, and I honestly didn't enjoy posting the headline here, but as far as I know I have to use the original title? And the original title is from a personal blog where we talk about annoying things. We're not a professional tech blog, or a bug tracker, or… something other than our own little thing. I choose that headline not to be clickbait, but "sensationalist" is probably true because I'm personally really, really angry about this issue. I wanted to do my usual "scan all my documents for the month" routine 30 minutes before going to bed, and instead it turned into a two hour debugging session. And I can't even use my normal workflow now, possibly until March. I find it completely unacceptable that Apple would break Preview that way again. It's not even the first time. Just thinking about it now gets me going again. That's why the headline sounds like it sounds. I would have no problem at all if it was modified here, and as I said – your assessment is absolutely fair.

Kind of agree the current title is a bit misleading.

I've never used any editing features in Preview (I mean it's called "preview" so...) and reading the title I thought this meant it was mangling files even by opening them which would have been super scary.

As for non-Acrobat software mangling PDFs after editing... Well that's much less surprising. I've even had Acrobat mangle stuff in PDFs after editing...

In my very humble opinion, it's accurate: "Saving a PDF with Preview in Big Sur [Preview in Big Sur] can corrupt OCR text added by a third-party program [is irreversibly destroying PDFs]"

I understand the (fairly common, in these comments) viewpoint that this is the fault of the "third-party program", but since the PDF is readable up until Preview touches it...I find it hard to come around to the viewpoint the third-party program is relevant. Readable bytes -> Preview -> unreadable bytes is my mental model so far.

Edit: absolutely unacceptable this is downvoted to -4. I've observed for a couple months that participation in Apple-related threads, outside indignation that Apple was involved in the discussion at all, gets down to -5 before getting back to -1 a day or two later. No matter what tone is used, this happens, and it makes the problem even worse in the long run. Been here 10 years, always been a _slight_ problem, but over the last year, it's virtually impossible to participate without continuing to slowly destroy my 11 year old account. Not sure how much longer I can keep trying.

Did they save the file using Preview? If so that’s on them, they chose to write a pdf using Preview and that comes with all of the pitfalls of pdf compatibility. Does plain old PostScript have this problem?

So Preview opens a file, which is apparently valid per Preview (Preview handles and displays it correctly). After changes, preview then saves a file that is not valid per Preview (OCR text layer is corrupted).

It is very difficult for me construe a situation in which this would not be considered errata in Preview. Even if ABBYY is writing unusual PDFs, it's popular enough software that this issue will be encountered in the real world multiple times. Having to deal with unusually formed PDFs is just a general trait of writing PDF software. If you consider it a non-issue that your software writes corrupt PDFs when the same situation is handled correctly by Acrobat, no matter how unjust you feel the cruel world to be, you should not be in the PDF business. There's a reason most PDF viewers only present a highly constrained feature set, and it's because writing a capable PDF editor is very difficult. Apple has decided they are up to the challenge, and in this case has failed.

As a general rule, if your software package opens a file fine, then writes a broken version of it, seems to all the world like a bug in your software package.

The idea that this behavior in a PDF reader can be excused because the software that generated the PDF was not approved for the operating system the viewer is running on per the vendor... kind of stretches credulity. I don't usually inquire as the OS used to generate the PDF when I receive one.

Here's the rest of the statement that you left out that answers your point.

"...I just need my OS to keep working for me. This bug could hit me without even owning a scanner at all – someone sending me a PDF that I then unknowingly break before archiving it. That’s the part I’m mad about."

Read it again. This bug could hit you without ever using ABBYY yourself. Apple broke Preview.

Would be interesting to see if Preview is stripping OCRd text from PDFs not created by ABBYY FineReader.

Are you arguing that ABBYY Finereader is going to produce different PDF files once they support the new OS? Possibly they will, if only to work around this obvious bug in Preview.

How? I assume it's mostly using proprietary MacOS APIs, and, ignoring that it doesn't really matter beyond apple being an abusive partner, lawyers are a powerful tool (This software is under the Apple don't take the piss licence)

I wouldn't be to sure, Apple's applications usually rely heavily on proprietary Cocoa APIs not available anywhere else

There already is an equivalent feature in GNOME. Preview's code is of little value outside of Mac. It's also not rocket science, that the thing stopping other people from doing the same thing would be lack of access to original source code. :)