You make a couple of great points.
Let me analogize the problem. Imagine that one day you’re reading books and working on computers and floppy disks.
So your information is limited. To be more fair, you’re searching paper copies of patents in “shoes” and using the Dialog database and now-arcane machines to actually look at images of patents (wow).
This is what I was doing in the early 90’s as a patent examiner, following working in industry as a software engineer.
Then along comes all the information in the entire world from the dawn of civilization to the present: the Internet. (Or think of it this way: Bill Cosby meets Twitter.) So now, you’ve got the USPTO, with access to all of human information.
Now, in reality, patents like most other creative work are typically incremental changes. If enough time is spent, it is possible to find reference that are very close to the “invention” being examined, especially if all information is available.
The next challenge is how close do the references have to be? When are the references close enough that when combined together, it would have been obvious to come up with your invention. Is your invention obvious, or not, over references. If it wouldn’t have been obvious to combine them, you get a patent, but not otherwise.
Well, who gets to decide this? It’s not really the patent examiner, though it really is – let me explain.
The patent examiner can’t say it would have been obvious to me, Mr. Patent Examiner, and so you don’t get a patent. That subjective nonsense.
We don’t really want this objective person to be a real person at all. Imagine if we let the president decide – this president – I can’t imagine anything worse.
If you’re familiar with contracts, we create the legal fiction of the “reasonable person” to decide how the contract should be interpreted, and the parties, judges, juries and everyone else must put themselves in the shoes of this fictitious person to decide how to interpret contract language.
In patent law, we’ve had to create the legal fiction of the “person of ordinary skill in the art (POSITA)” If you think about it, it’s logical, because the person has to be someone who works in the field of the patent subject matter, and this person’s doesn’t have a low level of knowledge or a genius level of knowledge, but instead an “ordinary” level of knowledge. And that’s when the patent application is filed, not later.
It’s the POSITA that the examiner pretends to be when issuing the patent, and whose shoes judges, juries and even Supreme Court justices must step in to make the determination.
So if you put 1000 people in a room, and gave them some good (perhaps not even great references) and asked them whether it would have been obvious to come up with an invention, how hard do you think it would be to get consensus? I think it’s a lot more likely that you couldn’t get half of them to agree one way or the other. When I did EE and math all those years ago, I remember a couple of grad students who though everything was rudimentary. I guess there's always some bonus points awarded with this kind of chest pounding.
Now, throw on top on top of that a few more facts. 1 - the examiner has perhaps a few hours or a day or two to find all of the best references, and make a rejection; 2 – the attorney responding for the applicant is limited really by the client’s dollars, so unlike a big corp a small company doesn’t have a budget to keep arguing over and over to get some decent claims allowed; 3 – the budget once we’re in licensing or litigation is much higher, so you can bet that a big company being sued will comb the earth to find the best references to invalidate the patent; 4 – that same big company has enormous marketing dollars to paint anyone who ever brings a suit as underhanded, bad, gaming the system, basically the same as the old lady who sued McDonald’s for giving her coffee that burned her; 5 – for these reasons, and that so many software engineers feel as you do, the USPTO and this particular examiner are extremely nervous to allow a software patents that’s got any teeth. Who does that hurt?
Well, I can assure you it's a much bigger problem for smaller companies and startups than for the big companies pushing out the marketing. That's one of the reasons I'm always fascinated by the vehement antipathy toward patents by startups in SiVy, in a Spock sort of way.
I think you get the scope of the problem. We have huge systemic issues that are exacerbated by big dollar interests. And it's not the best formula when everyone's completely convinced they're right about everything, if you know what I mean.