We do not send any webcam / audio data back to a server, all of the computation is totally client side. The storage API requests are just downloading weights of a pretrained model.
We're thinking about releasing a blog post explaining the technical details of this project, would people be interested?
And some quick questions:
What network topology do you use, and on what model is it based (e.g. "inception")?
What kind of data have you used to pretrain the model?
The application takes webcam frames and infers through SqueezeNet, producing a 1000D logits vector for each frame. These can be thought of as unnormalized probabilities for each of ImageNet's 1000 classes.
During the collection phase, we collect these vectors for each class in browser memory, and during inference we pass the frame through SqueezeNet and do k-nearest neighbors to find the class with the most similar logits vector. KNN is quick because we vectorize it as one large matrix multiplication.
I'll go deeper in a blog post soon :)
It's like when you turn on a camera and people can see themselves on a TV. A lot of people can't help but make faces at it.
One could probably map each gesture to a regular USB device that acts as a second keyboard and mouse? The hard part is identifying enough unique gestures?
https://www.engadget.com/2014/08/28/leap-motion-s-next-senso...
It's a really well put together demo & tutorial.
I held a pen up next to me and held the green button.
Then did the same with a mouse.
It would flick between the two if I was holding nothing, so I held the orange button for a bit while holding nothing.
Worked pretty much every time.
Training is fast enough with a few hundred images per class that I didn't notice any delay.
I can't run the demo here (browser not capable enough, and no camera) and I'm getting really curious what this is about.
I've solve regression, classification and recommendation problems with it and the best part is it deploys an web service with a few clicks.
1 a working phone
2 a valid credit card
To use azure which places a too high bar on students. I mean I've tried to argue for graduated restrictions so basically students with .edu emails should be able to do some things without entering a credit card number but the fact that it is not possible suggests this isn't a priority for azure.
Google says this finds on your browser so there's little infrastructure cost for this demo, right?
Is there an ML library that can easily start capturing images from the webcam so you can play around with training a model?
and you could do worse then this https://github.com/fchollet/keras/blob/master/examples/mnist...
(It is not a Python application but a Java application, but still as fun!)
Your unbridled optimism even in the face of reality has inspired me to give this a shot, thank you! It seems to be based on Weka, so it should be good.
On a first run, I don't really see how to record images from the webcam, it just says "waiting for samples". I'll play around some more and hopefully figure it out, thanks again.
EDIT: Ah, there's a detailed walkthrough which seems to work well!
I wish chrome would give the option to only give permission "this time" and I wish it didn't allow camera access from cross domain iframes.
Google could theoretically release compromised versions of Google Chrome and only use the permission on devices where webcam LEDs are unlikely (e.g. smartphones), but this is going deep into tin-foil-hat territory.
also this isn't about Google spying. it's about Chrome's bad camera permission model. any company can abuse it
Of course I'm not suggesting Google would actually do that but some other company might make seeamazingcamerameme.com to get users to turn on there camera for that domain and then after that make iframes for seeamazingcamerameme.com/spy
if you like this I would highly recommend looking at openframeworks.
the interactive browser part excites me want to try to make something with deeplearn.js
What value is there in taking care to store biometric data only locally, in a separate chip inaccessible even to the OS, if people will simply claim it's equivalent to keeping a remote database of millions of faces?
I don't know anything about Squeezenet, but it makes a lot of calls to storage.googleapis.com. I wouldn't be surprised if it's making some PUT requests. https://github.com/googlecreativelab/teachable-machine/blob/...
Also, I don't think that this sends any data to Google, since it trains the neural net in the browser. You could even verify this yourself by looking at the source code.
- Gmail / Google Plus / Google Apps profile pictures
- Google Street View
- Google Hangouts
- implementing a primitive Face ID or Snapchat-style camera on Google Android
- the large mass of face pictures that they index with Google Images
Also I think a lot of the processing is done in the browser using deeplearning.js, so I don't know how much is sent back to Google.
It's a pretty neat way of creating a reasonable training set of 3 classes.
Because if you assume an attacker to have control over DNS, the security model of giving permission on a per-domain basis is broken anyway, and the initial concern with granting google this access is already subsumed in your general paranoia.
> Are my images being stored on Google servers?
> No. All the training is happening locally on your device.
But I do agree people need to ask the question before making assumptions. Sadly, the two popular mindsets is either to not think about privacy at all, or believe that everything is infringing on your privacy.
2. Press "Start" or "Skip tutorial" (You don't have to give access to anything)
3. Scroll down to read the FAQ
https://h4labs.wordpress.com/2017/09/27/groundhog-day-amazon...
Also, my own personal privacy is less secure if it's a relative inconvenience for employers. If everyone but me gives up their privacy then there's more pressure on me to follow suit.
The argument even doubles back on itself. If these comments aren't interesting to you... don't read them. Embrace tree-style collapsible comments.
I'm curious why you've used a different classification algorithm on top of a neural network. I would expect that a neural network on top of a pretrained network could give similar results, with the benefit of simpler code. Is performance the reason?
Anyway, I'm looking forward to your blog post.
KNN also makes training "instant", and the code much much simpler.
By the way, I think your software could become very popular on the Raspberry Pi, because it would be very cheap and fun to use it for all sorts of applications (e.g. home automation).
I responded to the argument you linked. You're avoiding a more interesting discussion on the topic. Push the [-] button and move on. Your comment is blatantly hypocritical:
"Every time X is updated people complain about X; those people ignore the details of the update."
"Every time people complain about X other people complain about them complaining about X; those people ignore the details of the complaint."