Was Google Earth Stolen?(avibarzeev.medium.com) |
Was Google Earth Stolen?(avibarzeev.medium.com) |
> However, I don’t believe anything was actually stolen from, copied from, or even directly inspired by ART+COM.
https://web.archive.org/web/20130605121646/http://www.busine...
(Source: Businessweek, June 5, 2013 - now part of Bloomberg)
CAFC affirms invalidity of geographic map visualization patent asserted against Google Earth:
https://www.ipwatchdog.com/2017/10/25/cafc-affirms-invalidit...
>In affirming the district court, the Federal Circuit found against Art+Com’s arguments that Google did not provide clear and convincing evidence that the invention of the asserted claims were put into public use. Google’s argument on this point revolved around a 1994 recording of a lecture given by Stephen Lau, then of the Stanford Research Institute (SRI), on a geographical visualization system known as SRI TerraVision. Lau also gave testimony during the Delaware trial on the operation of the SRI TerraVision system. Google also brought an expert witness who gave his opinion that the SRI TerraVision system anticipates the asserted claims.
ART+COM INNOVATIONPOOL GMBH, Plaintiff-Appellant v. GOOGLE LLC, Defendant-Appellee:
https://caselaw.findlaw.com/us-federal-circuit/1878050.html
>Google introduced several forms of evidence in support of this effort. First, it called Stephen Lau as a witness, who testified that, while he was employed at the federally funded, not-for-profit company Stanford Re-search Institute (“SRI”), he helped develop SRI TerraVision, “an earth visualization application” that “used a co[arse-to-fine] algorithm to retrieve images [sic] data across the network from multiple servers.” Trial Tr. 1029 ll. 9–18. He further testified that SRI TerraVision was part of the “MAGIC project,” an “umbrella federally funded research project” that focused on terrain visualization. Id. at 1030 ll. 9–12, 1043 ll. 5–10. He also testified both that he wrote about 89 percent of the source code underlying SRI TerraVision and that the project was meant to be put into the public domain. Id. at 1030–32, 1151. Lau further testified that SRI TerraVision allowed a user to navigate around a two- or three-dimensional representation of a graphical area and to zoom in and out to different levels of detail, and described how SRI TerraVision drew its image data from a network of multiple servers spread across the country. Id. at 1034–35, 1051.
>While Lau was on the stand, Google displayed a 1994 VHS tape in which the narrator walks the viewer through the operation of SRI TerraVision. J.A. 2565. In the tape, the narrator describes how a user can move from a low-resolution picture of a larger geographic area to a higher-resolution picture of a smaller geographic area using a “multi-resolution pyramid.” J.A. 2565, 3532–33. The narrator continues:
I did find this video of ART+COM's Terravision:
T-Vision aka TerraVision (1994):
https://www.youtube.com/watch?v=fjZ-wh5jykk
>This video is (C) Joachim Sauter, ART+COM and Deutsche Telekom. Found it on Joachim Sauter's website. Merely sharing it with the world, funnily on a Google platform :D
I was actually responsible for invalidating another maps-related patent from the 1990s, in Germany. I was in Patent Litigation and I also remember having a "ART+COM" folder in my email, but that's about all I can remember. I don't think the case was very active by 2014.
Anyhow, there's something Avi said that I have to comment on:
The test for patent-worthiness is: “would it be non-obvious to someone who is skilled in the art?” I’d been doing “stupid programmer tricks” like this since high school, as have many others in my field.
If only it were so simple! You would think that "hey, that's just a stupid programmer trick!" would be enough to render something obvious, but you'd be wrong. That's what my paper [1] is about.
No, to call "obvious" the combination of references A and B, there usually has to be some "teaching, suggestion, or motivation" in the literature that it would be a good idea to combine A and B. There are nuances to this statement, in case anyone's feeling pedantic.
[1] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2399580
There's the rub. Because patents get issued for things that are so obvious that you would never bother mentioning them in "the literature".
Take dragging a map with the mouse as an example. First, if you're doing online mapping, the first thing you think of is to just grab the map and drag it wit the mouse, so you implement that. (Even though some clown has a patent for the act of doing so).
But assuming you've gotten away with that somehow, now imagine the first day of implementing it, when you "drop" the map while your mouse is still moving. It'll stop in a jarring way and your first thought will be "That's ugly. What if it kept going with momentum and coasted to a stop." And that would get you sued out of existence.
Now, the patent system seems to suggest that the appropriate way to handle this would be to search the patent database for "dragging a map with a mouse", find the patent for "momentum", license that from whoever has it, read it and learn how to implement it, thus benefitting from some novel invention, saving yourself time and effort, and reimbursing the inventor for his effort.
But in actuality, you spend the time it takes to draw a breath thinking about the problem, then code it up in the next few minutes and carry on with your day. Because there are very few things in programming where describing the problem doesn't also describe how to solve it.
I've never seen an example of a software patent that covers an actual non-obvious "invention".
When online mapping was first around the javascript slippy map hadn't been invented. The page would show you a given map tile, and provide left/right/up/down/zoom buttons that would load a tile one step in the direction you chose. Sometimes with a complete pageload for every step.
For example: https://web.archive.org/web/20040805053745/http://getamap.or...
The latter case is a litmus test of non-obviousness -- all the pieces were there for some time but no one had put them together. It is distinct from when everyone puts the pieces together the second the pieces are available.
Patent clerks can't tell the difference.
I will say that the state-of-the-art for geospatial computing in the 1990s was pretty damn primitive, many useful things had not been invented yet. It isn't great today but it has come a long way.
Hence the focus on prior art.
You can laugh at "native common sense" as being meaningless when the law is involved, but actually it's not. In the right circumstances, the fact that most practitioners of the software art see things a certain way would be "dispositive" (as the lawyers like to put it). We haven't seen anyone create those circumstances.
I had a senior attorney review this paper for me, and he had a lot of interest in the concept. He called it a "toolkit" and found it intriguing that software engineers had a toolkit they applied to any problem. You know that we do.
The forces arrayed against this concept are formidable. Even Computer Science departments in universities aren't unified against software patents. I actually talked to some profs at Illinois, and one of them said "you're assuming that I agree with you!" He didn't. He had a startup company and the VCs had told him he had to have patents.
Rounded corners should not be patentable. The latest ARM core design should be.
A good idea will win on its merits or fail.
Legal ownership of ideas just tether us to paying Disney and Google’s of the world in perpetuity as they buy everything up.
We basically live in a state sanctioned system of quota and expropriation of effort. It’s not violent like the USSR, so we look beyond it.
And we spend a lot of energy policing it all.
Their patent was invalidated in 2017. Here's why:
https://www.ipwatchdog.com/2017/10/25/cafc-affirms-invalidit...
A tale of something that is "very obvious" only after it has been done. I mean MP3 is about reducing information for frequencies hidden by other frequencies right (I think that is roughly correct!) and that is a really obvious way to reduce bandwidth....once someone worked it out.
Novelty is also a requirement, in addition to nonobviousness.
The OP suggests there are more important issues for programers to focus on:
"I'm openly critical of their ad-driven business model, especially since the damage is clearer. That's broached in the show, but only to point out how much money Google would theoretically owe these guys if they'd agreed or been forced to license at 10 cents per use (which was very unlikely IMO).
Invading people's privacy for profit is not something to yearn for. It's one of the reasons I sold all my Google stock. I respect Google's engineering work and the great "free" information they provide. There are a lot of smart and dedicated people working there. But the biz model is not ok, IMO."
Sad that Google appropriated this guy's work for a "business model" that he does not approve of.
Perhaps there are many more folks like him, who have done solid, valuable work in some area that has nothing to do with invading people's privacy or online ads, only to have the product of that work usurped and given away for "free" in the interests of furthering Google's prime directive: selling online ad services.
But Tim is surely up to clarify any questions.
Well, nothing is harder to say than if something is "obvious". Locking backward, basically most things are "obvious".
"someone who is skilled in the art?"
This is another questions. What is "skilled"? Most patent courts assume an average person from the field of the invention and not an extraordinary specialist.
I've also noticed that many insights also seem "obvious" in hindsight (the proverbial "egg of Columbus" thing). In fact, the better the insight, the more "obvious" it seems!! Thus, "hindsight obviousness" is not an argument against an innovation
But the fact remains that Art+Com had a working system in 1994 and the Keyhole founders had learned about it working at Silicon Graphics.
As it looks like there were multiple teams in the US working on something like or comparable to Terravision. Heck, I wrote a program in 1984 showing the world as a ball with the contours of the continents at the ZX Spectrum. I bet hundred's if not thousands of people did that as well, it is not that hard and requires only a little trigonometry.
I have no further insights into the story, but folks here should at least respect that a small group of hackers and and artists build a working system a good decade before Google Earth was published. I have yet to see any demonstration or video that comes near to Art+Com's TerraVision in 1994. There are papers written at the same time, a DARPA project working for a single military installation and a lot of work done shortly after TerraVision had been publicly demonstrated, but so far I have yet to see a video from 1994 or before comparable to Art+Com's TerraVision.
https://en.wikipedia.org/wiki/Outlook_on_the_web
First with (of course) an Internet Explorer only plugin to asynchronously load XML, then standardized across browsers.
Ironically, John Hanke went off to found Niantic as a startup within Google and launched it's first augmented reality game called Ingress on this fictionalized premise.
Same goes for certain writers books. They are getting paid to write books and then pick something that sells and paint a narrative. And then people think this is true, wholly true or useful whereas it may not.
So sad, so scary.
I have one criticism: the story confounds stealing the idea (in itself a very fishy concept) and patent violation. It's a popular misconception that these two things are relatated and to the authors defense it probably is the same in the Netflix documentary. However I think it's important to be precise when talking about these things.
Building a PC app that does it much better is also worth recognition IMO, and the Netflix show denies that.
The IP issues were resolved, but I think it was important to point out why there was no infringement for people who might not understand the details as well.
1 2
3 4
1 3
2 4
00 01 (0x row)
10 11 (1x row)
(x0 (x1
col) col)(Illegal) Copying is not theft
‘The Billion Dollar Code’: Developers Who Sued Over Google Earth Algorithm - https://news.ycombinator.com/item?id=28900180 - Oct 2021 (61 comments)
The predecessor to Google Earth was clumsy, yet powerful - https://news.ycombinator.com/item?id=28626361 - Sept 2021 (52 comments)
And then in the final verdict, the jury decides the patent is invalid. Of course, if it's invalid, nobody can infringe it either, which makes the rest of the decisions obvious.
(I'm also a bit disappointed that it shows the two Germans as being close to the CCC, which was firmly against software patents, but doesn't even touch on the tension this must have caused).
Spoiler: They resolved the issue by getting a charter of CCC members, whereupon neither of them was listed and thus the defense (Google) was forbidden from even asking about it.
I understand that reasonable people differ on their understanding of the truth. If it's for entertainment, then no big deal. If it's for "justice" then that's what the courts are for: to declare one truth more truthy than all the others. And in fact, ART+COM did lose in court, so...
At the time, game developers were using idTech 2 and 3, Unreal Engine, Renderware, Vision and a slew of others. I think Alchemy simply lacked market presence.
https://avibarzeev.medium.com/was-google-earth-stolen-7d1b82...
Now I have an even better one for you, and I'll withhold my (undoubtedly correct) answer until later:
Wheels on luggage. Why were they so late in coming along? Was it not obvious?
We had aftermarket wheeled carriers for strapping your suitcase in, so people recognized the need. The arguments that "people hired porters or trolleys" and "railroad stations had cobblestone floors" do not hold up at all -- no, they didn't, and airports had hard floors.
You can find articles on this topic. I think they're all wrong. Go.
As their litigation backlog grows, and as resentment grows, they are increasingly impervious to the law. Too big to fail, too big to sue.
I mean, is it okay to feature and discuss paywalled information here and everybody good with it? To me it feels like an ad for medium.com
One of there earliest globes renderings* im aware off, all complete with weather data in late 60s:
https://twitter.com/wallacetim/status/908531916315471872
* for the sake of visual impression, 2D orthographic/stereographic projections are equivalent on-screen with proper 3d-renderings.
But I didn't think it was relevant to the patent case or the Netflix show, which hinged on whether two SGI execs copied anyone's code. They didn't. More relevant were actual systems like SRI's TerraView that were prior art in a legal sense.
configurations of roads, rivers or frontiers, satellite images, actual temperatures, historical views, CAD-models, actual camera shots, are called up, stored or generated in a spatially distributed fashion. [2] were all not in SRI but in Keyhole EarthViewer/ Google Earth and not covered by prior art.
That should be distinct enough in usual patent cases where any minute modification of a car engine is patentable.
Is that mentioned lecture video of Stephen Lau, which I'm sad to learn, was one of the early COVID-19 victim[3], somewhere publicly available?
[1] https://www.sri.com/wp-content/uploads/pdf/778.pdf
[2] https://patft.uspto.gov/netacgi/nph-Parser?patentnumber=RE44...
[3] https://www.dignitymemorial.com/obituaries/pelham-al/stephen...
---
Some Excursion: even 10 years later, the most relevant IP to current mesh based / photogrammetric modeled virtual globes in my opinion:
The Rapid 3D Mapping pipeline developed by the swedish miltary devision of Saab, spun off as C3, that got acquired by Apple in October 2011.
The completeness and vastness of surface features was quite the step up from the draped texture on 2.5D terrain plus some random models.
For those who don't remember. The Google Earth/Apple Maps of today is quite a different beast to anything before April 2011, first available in Nokia ovi Maps 3d
https://freegeographytools.com/2011/nokia-ovi-maps-in-3d
https://www.slashgear.com/apple-acquires-c3-technologies-for...
From reamde, by Neal Stephenson, 2011.
There is always a message and the line between fiction and nonfiction isn't always clear. Who is making the media and why, who did the research, who funded the project, etc.
What's sad and scary is that media literacy and critical thinking isn't a top priority for teaching in schools.
In the current era, I think we've lost confidence and taste for balanced narratives, neutral depictions and such. Journalism is storytelling. Every headline, choice of picture or example is a narrative selection made by an author. Objectivity is a delusion, best discarded.
That "postmodern" take might be true. It's hard to argue against, honestly. That said, untruths can and do play a role. There's a difference between a world where objectivity is a sham and one where objectivity is not even a sham.
Either way, It's hard to go back. Documentaries and books today are making an argument, not depicting one neutrally... They're not even pretending to. That may make the endeavour more honest, depending on your POV. Regardless, no one owes the whole truth anymore and no one expects it. It's up to you to synthesize yourself a makeshift whole.
Accounting for that so much misinformation, propaganda and opinions have been published in the media even back then, the outrage that "these days" things are worse feels a bit short-sighted.
Probably because the notion that we've ever had "balanced narratives, neutral depictions and such" is a bald-faced lie, per, "Negro lynched for assaulting white woman," et al.
The media landscape has balkanized enough that attempting to declare A Single Truth from on high (however fact-based, or not, it actually was) is no longer possible. To those constantly thrown under the steamroller of Stability and Civility, this is a good thing. At this point, people have to choose between convenience or truthz because a convenient truth is unattainable.
There is literally nothing (except defamation/libel laws, which is why all the characters are invented) that says any story presented as true by a publisher has to actually be true. There is no penalty for lying, or presenting a fictional story as true.
Any expectation that anything a publisher produces is even vaguely related to actual events is based on a now-outdated set of principles in journalism that are no longer followed.
my issue with his narrative and he doesnt really offer any viable alternatives execept suggesting government regulation (which is usually responsible for whatever situation he's ranting about in the first place)
After reading constant praise about simplicity here on HN it always sound like cognitive dissonance to then read when patents are mentioned that something 'simple' should not be protected.
I see the lineage from SGI's Clip-Map to Keyhole's Universal Texture to Carmack's "Mega Texture" to today's Unreal's Nanite for geometry.
(not claiming anyone infringed anyone's IP, just the natural evolution of a clever idea)
I'm talking about the way we even structure the question today. There's no expectation of neutrality, and trying to appear neutral will seem like hypocrisy. If you buy a book on an intellectual topic, you expect to get one half of an argument, not the whole.
Until you show us a transcript of the hearing in which the stakes were explained in these terms to the jury (which doesn’t exist, as it’d be sanctionable for the lawyer arguing it for several reasons), this is just gratuitous and uninformed criticism of a country that ostensibly isn’t yours — which you hedged by saying “arguably” behind one of the key points, because you know it’s mostly bullshit. I guarantee you the randomly selected jury barely knew where Germany was, much less the important lines to draw to preserve our sovereignty in intellectual property abroad. Ask any patent litigator how difficult it is to bring the jury to the facts, looooong before getting to the Sun Tzu realpolitik you’re cooking up here.
If you’re going to make the low key America-is-stupid argument (I heard you, don’t worry), you can’t immediately undermine your own argument alleging that said stupidity intentionally furthers a broader evil purpose. It’s just stupid.
We have so many problems with our legal system that I’m honestly amazed you were able to invent one I hadn’t heard before. I also think you’re imagining China here, not the United States, given how trade dress and patent litigation goes for foreign entities there (like BMW; hey, they’re from Germany too!). Also extremely undermining to the point of hilarity that you included that country’s specific take on intellectual property litigation in the “rest of the world” distinguishing point regarding objectivity.
[1] p1514:
According to these results, the patentee is significantly more likely to win a jury trial if: (1) the infringer is foreign; (2) the infringer is a corporation; and (3) the patentee is the plaintiff. The multivariate regression model further supports the conclusion that American juries favor domestic over foreign parties in patent trials.
[1] p1510:
Patentee win rate by Party Alienage:
Patentee-Accused Infringer / Jury / Judge
Domestic-Domestic / 71% / 29% Foreign-Domestic / 38% / 31%
[1] https://www.law.gmu.edu/assets/files/publications/working_pa...
Obviousness is the wrong caliper here, I think.
IF there's plenty of prior art, then it's easy to define why something is "obvious" - it had been done may times before.
Such innovation does not need the protection of the patent system any more than the obvious inventions do, it happens on its own.
I was kinda sad when I heard they replaced everything with Windows NT.
- you get the idea
In later revisions, they had terrain mapping but it required 8 gigabytes of ram for the texture mapper, at a cost of $60,000.
That was all ripped out and replaced with Dell hardware running Red Hat in 2005. The trusty O2 ran crawls and alerts, spooled WSI’s AWIPS data feed into iNews (how TV news produces the show) for another decade. It could still be there, running.
Google maps was still years away. And tile server tech wasn’t quite there to ship it yet. But it was the obvious thing to build, and not in any way a mystery as to how you’d do it.
Inventions are often that way -- think of the mousetrap. People knew springs pinched the crap out of you when released. But nobody had made a device to kill mice, yet untilthat was invented.
I don't know what the cognitive bias for thinking that because you thought of an idea it must be obvious is, but it exists. I say that as a patent inventor.
The technical "how" of the implementation is not really obvious, you could implement the same result in many possible ways, some better than others.
Technical and business limitations drove the design of MapQuest. That client computers had little computing power, bandwidth (on both sides of the pipe) was limited and expensive, and processing power on the server for a very low value user (per interaction) had to be meted. No one who used MapQuest didn't wish they could just smoothly scroll the map, but we were fine given many of us were on a terrible connection, ran on systems with limited graphics power (where even smoothly blitting a high resolution raster graphic was taxing), etc.
Every improvement (more computing power, memory, storage, bandwidth, or even business model, etc) in the industry invariably leads to many people all independently seeing the same obvious next step, many groups building the same eventual thing, and then the losers (from a market perspective) claiming that their ideas were stolen. It is the story of this industry.
Perhaps you could interpret the parent comment as "if you're doing online mapping [and mouse drag/drop is possible in the browser], the first thing you think of ..."
There were also other interesting navigation schemes that relied on multi button mice or tablets.
I still like the model where you hold down a button and the navigation speed and direction is controlled by the distance and angle from that click point to the mouse cursor.
I think "grab to move" is as old as our opposable thumbs. There is nothing innovative about it
That physics combination with drag and drop interaction was not obvious in the time period we are discussing. The closest thing I can remember to that sort of thing were side scrolling games that accelerated with the character centered on the screen according to the game's physics, and possibly paddle-based game physics dating to the Atari era.
Slowly increasing scroll was present in some interfaces. But not tweened stop.
My main point is that clever people often dismiss what they find trivial to be obvious to others, and while that's often the case once many examples have been observed, it isn't prior to the exposure to many examples.
Copying natural laws in our work is something that was obvious for humans for millennia. We've been imagining space ships 100 years before going to space but simulating inertia for an object "sliding" on a screen was "not obvious at the time"?
Of course it was! How to do it (well) may not have been obvious but the idea of doing it was there for anyone using a computer. The parallel with the real world was always obvious because this is directly analogous to how physical objects behave and you'd do naturally in the physical world. The world around you provides the "observed example", not other computer implementations.
I personally thought since decades ago about this kind of inertia while scrolling through lists or dragging things. I thought of transparency, skeuomorphism, writing on touchscreens, etc. long before they were implemented because computers could handle them. And I can assure you I wasn't the only one.
"Same thing that you've been doing for ages in real life but now on a computer".
P.S. I'm thinking of a holographic/VR 3D interface right now that replicates interactions with physical objects either by manipulating it with my body parts, with electronic input devices, or with a BCI. I hope I live long enough to read an internet discussion where someone claims this was just not obvious back in the dark ages of the early 2020s...
Your interface is just a fantasy. You haven't taken any steps towards making it a reality. There's no demonstrable prototype. There's no documented research into it's principles that you have uniquely contributed directly to.
That's the difference. You haven't undergone any inventive steps, because you haven't created anything. You have a vague idea that such a thing may be possible, but haven't undergone the rigor of understanding and implementing a basic example of what you have conceptualized.
This isn't to be confused with comercializing the idea, figuring out how to market and produce it to meet demand, package it, and deliver it. That's a wholly different process, parts of which itself may require further inventions.
> I'm speaking of the inertial scrolling [...] That physics combination with drag and drop interaction was not obvious
Now you pivoted to the point that the implementation is what makes it not obvious:
> You have to think of and implement the idea.
Despite the tone of disapproval you just landed in agreement with what I said in the very first comment:
> The "what" is obvious [...] The technical "how" of the implementation is not really obvious
Will it be a novel invention to implement the inertial scrolling of a map using an exceptionally overly-complicated and completely impractical method that nobody has used before? No. Will the idea of inertial scrolling be obvious despite my completely non-obvious implementation? Very much so.