Why do we convert structured data to PDFs? Company A has structured data. They input this into a PDF (making it unstructured) and send it to Company B. Company B now has to use PDF parsing software to turn it back into structured data. Why? |
Why do we convert structured data to PDFs? Company A has structured data. They input this into a PDF (making it unstructured) and send it to Company B. Company B now has to use PDF parsing software to turn it back into structured data. Why? |
PDF has facilities for tagging documents such that they can be reflowed like HTML so they can be viewed on different sized screens. It is a boon for accessibility but framing the discussion around accessibility as opposed to a better experience for everyone, particularly automated tools, is hard. (e.g. in politics there is the analogy of how we "can't have good things" because policies that are good for everyone get framed as policies that benefit a racial or other group perceived as a "special interest")
I spoke w/ Larry Masinter at Adobe and he told me Adobe would like people who want structured data in their PDF documents to simply attach files to the PDF. A scientific paper could contain a CSV file of the data, for instance, or a business document could contain a JSON or XML document.
Note that "structured" is not a panacea because the structure might not be the same in the two organizations. For exchange of structured data to take place the organizations have to agree on some ontology, something that happens in some industries some of the time, but it isn't free, and when it is not in place people still have an excuse to continue using paper processes or processes that emulate paper processes.
http://www.bloechle.ch/jean-luc/pub/Bloechle_Thesis.pdf
I made a HN post on this here: https://news.ycombinator.com/item?id=33674525
Unfortunately I contacted the author via youtube and the work is proprietary, owned by the business he either created or sold-to.
1. PDF is considered tempered proof. Obviously, not true, but legal is ok with that.
2. PDF can be reviewed quickly by non-technical folks, and then parsed and store into databases.
3. PDF is flat file that can be archived easily per legal, other formats such as word documents are used for that as well.
In a sense, PDF is what people want. Structured data is what machines want.
On a more philosophical level, the PDF has structure which is probably the most generalized structure across all domains: paragraphs of text on a page. Consider that most people barely know how to search a text file for a given word, and a minuscule percent of those people who know how to query a SQL database. People simply do not have the time or resources to learn a separate domain (data structure design and interaction) apart from their own domain. In other words, there's very few people who understand or even have motivation to use tools that provide exponential return on their time (such as manipulating/filtering/working with structured data). Time passes uniformly, and you typically receive no reward other than more work for learning tools to improve your own workflow.
Software engineers have long noticed that we can successfully create "models", "view models", and "views" of data that achieve the separation of concerns that you are seeking. A PDF is nothing more than a "view" of data, which has passed through a professional who has created a "view model" of that data (he/she decided how best to organize the data on to the page), and then you read the document and "parse" the data with your intellect. There is a lot of expertise and professionalism embedded in crafting paragraphs (or other graphical representations) that you can't discredit.
There is very little software options to treat generalized, domain-specific data in this three-step manner.
PDF-based interchange of structured data as part of an ongoing relationship ... seems to reflect a poor business relationships management.
(And yes, there are all manner of organisations which fail to follow good practices whether on grounds of competence or malice, but generally, this is how I'd suggest addressing this issue. I'd also strongly suggest checking to see if such a data exchange option is already available.)
If only on the basis that PDF itself is a relatively recent data format.
"Legacy industry" to me would be IBM mainframe data formats and the like.
Which ... are their own flavour of fun.
Also interesting as a business case, because 1co to 1co requires one connector on every side. Adding another co will only require 1 (not 2 nor 3) connector on the new co side; since the others are still valid hooking into the sento.io platform.
Note: I'm not affiliated, I just came across them a few months ago and this reminded me of them.
When XMP first came out, Adobe tools would look at all the metadata in, say, an image file (such as EXIF) and re-express it in XMP format. I liked that a lot because I could read that XMP packet with my RDF tools and have complete access to all the metadata with very simple software.
At some point other people in the industry accused Adobe of undermining other metadata standards and Adobe was pressured to only use XMP for data that could not be expressed with EXIF and other formats. This takes away complete and easy-to-work-with metadata unless I write my own tools that can convert the EXIF metadata to XMP and merge it with the XMP which might be in the document.
The semantic web community also has some blame here as it never embraced XMP, if Adobe had had more industry support it might not have nerfed XMP. I very much like how XMP adopted solutions to problems like keeping track of the order of authors that communities like the one behind Dublin Core haven't had the moral fortitude to address... Keeping Dublin Core in the category of "metadata for an elementary school library" as opposed to the world beating solution that XMP and DC could have been.