Interactive Dynamic Video(interactivedynamicvideo.com) |
Interactive Dynamic Video(interactivedynamicvideo.com) |
I.e. does this measure something previously unmeasurable, or is it just that the visualization provides an extra channel for interpreting the data?
Or is it that it would make this kind of tool available to anyone with a digital camera?
It might also make quality assurance/ product testing cheaper but that's less save-the-world :-p
This kind of analysis is already done and important, but usually it takes installing a lot of sensors and giving an impulse to the structure. I have actually worked on a company that did this, using strain gauges.
The fact that this is one using a commercial camera is kind of impressive. I'll take a look on the papers someday, but the obvious way to do this would be to have a framerate at least the double of the frequencies being measured.
Being able to determine frequencies much higher than sample rate not only is non-trivial, but also alleviates immensely the cost of these measurements.
A question for the authors. All the video is with vibrating but otherwise stationary objects. Must the object be stationary for these tools to work?
There's a company that makes a rig that's basically just an engine and 4 wheels. You film your entire commercial using it and they replace it with computer imagery of your car in post. I can't find the video at the moment, though.
They use optical flow to get displacement values for each pixel. This works well when you don't have very large movements between frames, no sudden illumination changes and no rotations.
Car commercials will probably have all of those and on top of that be highly edited (e.g. you have to deal with cuts in the video)
You could try to use some other approach for motion estimation, like identifying and tracking salient points, but that would require quite a lot of work on top of what this demo shows.
A few questions if the author is on HN: I wonder how long it takes to analyse a 5 second video? Also, it seems the algorithm only works with static images after the initial video analysis, or am I wrong? Also, how long does it take to render the "virtual" state of the object?
The simulation runs in realtime, and is interactive. Generating the simulation can take a while though.
The 5 second video took about a minute to process on my laptop. But longer videos, and higher-resolution videos, can take much longer. For instance, we've used the technique on high speed video to recover audible vibration modes, and this can take hours to process because the video itself is so big.
I wrote the processing code in Matlab, and the simulation in c++ and GLSL. The simulation is pretty well engineered because I wanted it to run in realtime. I didn't put a lot of time or effort into optimizing the video analysis part though, so it probably could be made faster.
Thanks for responding to my questions! The whole project is amazing and I hope you keep working on this (and future projects) with the same enthusiasm as you do now! :D
Yup, MIT.
I am also the first author of the paper described in these videos, which my colleagues and I published in the top academic journal in our field.
I also made all of the videos, did the voiceovers, and hand drew the animations (though I'm flattered you think they were professionally done). I wrote and presented the TED talk. I purchased the dedicated host name for a single algorithm (using money from my own grad student stipend no less) and created the webpage myself.
I am an academic first, and I take academic integrity very seriously. I also take education very seriously. I consider educating the public about research to be part of my job, and this is done best when people are excited about the research.
Also, this work IS exciting. If it weren't we wouldn't have spent so much time working on it and it wouldn't have been published in ACM TOG. But back to flashy videos and press...
Consider that in the past three days, nearly 100,000 people (and counting) sat through a video where I explain what vibration modes are. VIBRATION MODES. They may have been lured in by pokemon, but kids who remember that video won't have to ask their physics teachers "but why should I care about this? what is it useful for?" Hell, if it gets people excited I hope teachers show it to their students before they teach the topic. I'm not making money off of these videos, I'm just stoked this many people are getting excited about research. Our paper has 17 numbered equations in it - it's not exactly a page turner.
When scientists don't make an effort to communicate their work to the public, that responsibility falls on people outside of the academic community - people who often don't understand the work. When we make things harder for the press, we only encourage them to bastardize the work to make it more palatable to the general public. By taking an active role in how we present academic work to general audiences, we can better shape the message, manage expectations, and help prevent content from being sacrificed for click-bait.
Ok, I'm going to get off my soap box now. Cheers! -Abe Davis
To the advantage of being at MIT, they are very good about preparing a press release and contacting other news outlets to get exposure. But as a broader point, when it comes to reporting on science, I think the MIT/CSAIL set a good example.
We should encourage people to make their research accessible to a broader audience, not engage in catty sniping and whinging when they get recognized for their work.
I'm all for science being promoted/marketed, even if it's in a(n endearingly) dorky way. And there are some fun use cases for this to boot (see the Pokemon GO / AR vid below).
To me misleading the public is not the worst about it. Pretending scientific accomplishments can deter and demotivate other, 'competing' scientists in the field.
Yup, Hacker News Commenter.
;)
That said, it's always sort of amusing to see the MIT PR-machine at work, because it's so _obvious_ once you get familiar with how they work. At most universities the idea of hiring someone to make a professional animation with perfect sound-studio recorded narrative for a single project/paper would just... not happen. On the other hand I can see why they do it, because it works.. it really changes how people perceive the work, and the importance of it. On the other hand, yes, it can be a little frustrating to see something sort of publicized as completely game-changing as if the work came leaped forth from the vacuum of space and directly into the minds of MIT graduate students, when plenty of other people at other universities are doing related work. (How many citations does the video have, for example?) Not saying that's the case here, but I'll admit that it's sometimes been how I've initially reacted to things from MIT that were closer to my research domain. Kind of like, "hey that's pretty similar to what I'm doing, how come I don't have a dedicated web page and professional video and thousands of hits coming from the top of hacker news and reddit?" And then it hits me: "oh right, because I didn't make one.."
tldr; MIT has PR down to a science. They take publicity seriously, and it makes a huge difference for them. That is not necessarily a bad thing, although it does raise the bar for everyone else, which can be kind of annoying when you have papers and theses to write.
Why do you think professionals were hired? A basic microphone+blanket to damp echos, and a passing familiarity (perhaps a friend) with a video editor should be enough to put that together. (No offense to the team.) If you wanted to hire someone, I'm betting you could get it done for $50 on Fiverr. Another $10 for a domain, and a few bucks for hosting.
You're making it sound like this is some huge professional outfit with a coordinated marketing plan. It looks more like someone trying to show off their project that they spent a lot of time on.
But actually this is unlikely to make a big difference on battery drain. Pokemon GO appears to be built on Unity and Unity is pretty notorious for having poor battery management. Whether you're pegging CPU or not, Unity burns lots of power. So, the incremental battery life hit is likely to be low in doing image processing on top of a frame.
They can't be arsed to fix a simple VSync bug on Linux, meaning even simple sprite games, or even more insulting: TIS-100, a game about optimising assembly on fake retro-hardware, deliberately made to look like a terminal, always burns 25% CPU power on my laptop.
More importantly, as Paolo Pedercini[0] mentioned on Twitter recently:
> I can open my Flash crap from 15 years ago no problem, but a project I made last year in Unity 5.1 crashes silently if rebuilt in Unity 5.3 I'm telling you: today's indie game production is a Unity monoculture that will disappear at the next hardware architecture cycle
[0] https://twitter.com/molleindustria/status/760904169444274176