Apple Maps Flyover Reverse Engineering

Apple Maps Flyover Reverse Engineering(github.com)

246 points by benzinschleuder 7 years ago | 26 comments

w-ll 7 years ago |

I don't think this is satellite data, most likely photos captured from planes.

And I'd kinda be curious to what Microsoft has been working on, I remember their birds eye was pretty good a few years ago.

Apple I'd believe the are selectively trying to improve their data in areas that are heavily visited by their users.

Also, I did for the first time see an Apple branded car with a photo sphere on the roof recently and kinda got excited that they are gonna try and bring competition into maps / street view that Google has dominated for the last decade.

ynniv 7 years ago | |

The 3d content in Apple Maps comes from their acquisition of C3:

https://9to5mac.com/2011/10/29/apple-acquired-mind-blowing-3...

The technique is detailed in the conference paper:

https://www.spiedigitallibrary.org/conference-proceedings-of...

Essentially they use a rig of DSLR cameras mounted to the bottom of a Cessna to take photographs in a 45 meter grid, and then do stereo reconstruction from that. Crucially they describe using fast phase based disparity maps, which at the time were a recently declassified technique for guiding cruise missiles. C3 was a part of the defense contractor Saab AB.

sorenjan 7 years ago | | |

Saab also owns Vricon, basically the same thing as C3 but not aimed at consumers. They sold part of it to DigitalGlobe, and they use both aerial and satellite imagery for 3D models.

https://www.vricon.com/

mempko 7 years ago | | |

Nokia also invested in C3 and had the first 3D maps available to consumers before anyone, completely in WebGL which used to be on https://wego.here.com. I guess Apple had more money to acquire them.

*Disclaimer - I worked for Nokia at the time.

gibolt 7 years ago | |

Most of the high resolution imagery is from planes.

Previous discussion on how Google Maps' Satellite View was almost named Bird Mode: https://news.ycombinator.com/item?id=19235017

rootusrootus 7 years ago | |

The Apple street view cars also have LIDAR, so I'm curious to see what their plan is for that.

madeofpalk 7 years ago | | |

https://techcrunch.com/2018/06/29/apple-is-rebuilding-maps-f...

tl;dr they're using it to make better maps

reaperducer 7 years ago | | |

Considering the expense, and especially time, involved in gathering street-level data, if I was Apple, I'd put every sensor I could think of on those cars and then let the data scientists figure out a use for it later.

bluthru 7 years ago | | |

Hopefully 3d street view. You'd want that data for good AR overlays, too.

gugagore 7 years ago | | |

That's probably for making maps used by localization on autonomous cars.

threeseed 7 years ago | | |

The most immediate use will likely be for the Apple Glasses which are expected to come out in 2020.

LIDAR could definitely assist with identifying what the user is looking at.

chrisseaton 7 years ago | |

> I don't think this is satellite data

It's a metaphor. Like 'bird's eye view'.

retroplasma 7 years ago | |

It's planes and I think also helicopters. I've written satellite mode in the repo because the Maps app button says "Satellite"

londons_explore 7 years ago | | |

It'll be planes, because the running cost per mile travelled is critical to making the whole thing cost effective.

Power consumption and weight of the rig isn't insubstantial, so the smallest planes and gliders can't be used, and after that cost per hour and cost per mile are the key metrics.

Planes can be 10x cheaper by that metric.

cmiller1 7 years ago | |

>I did for the first time see an Apple branded car with a photo sphere on the roof recently and kinda got excited that they are gonna try and bring competition into maps / street view that Google has dominated for the last decade.

Are you in Cali?

Hongwei 7 years ago |

I recently asked the founder of a pretty big GIS company how they handle plagiarism. The answer: "we don't bother. [we just monetize services on top of open data.]"

So it looks like the intentional errors method didn't work out in the end. [1]

Wonder if Apple will be as accommodating though :p

https://www.gislounge.com/map-traps-intentional-mapping-erro...

aaronbrethorst 7 years ago |

this is the same person who RE'd Google Earth a couple months ago: https://news.ycombinator.com/item?id=18900080

reaperducer 7 years ago |

Coincidence: Half an hour after reading this thread, a white Subaru with California tags stopped at the red light outside my office window. Big disco ball on a platform attached to the roof and "Apple Maps maps.apple.com" stenciled on the window.

I took a picture.

https://imgur.com/UwK2wYy

pducks32 7 years ago |

As someone who works with GIS data and map meshes this is really awesome. Hats off to the author.

saagarjha 7 years ago |

I’d be interested in hearing how they reverse engineered the data!

retroplasma 7 years ago | |

Mostly used popular tools like Hopper, LLDB, Hex Fiend. Xcode's debugger UI was surprisingly helpful too because it lists current memory allocations of obj-c instances nicely. Also learned about Frida along the way and would have used it more in retrospect.

zhoujianfu 7 years ago |

I just like how the example is the end of the Santa Monica pier!

marmshallow 7 years ago |

This is awesome. What are people using it for?