Eager Execution: An imperative, define-by-run interface to TensorFlow(research.googleblog.com) |
Eager Execution: An imperative, define-by-run interface to TensorFlow(research.googleblog.com) |
Looking forward to your feedback as you try it out.
You can't please everybody, as if they listen or not to users people still complain. If both are making effort to improve themselves though, the community has only to benefit from this competitiveness.
Doing image classification, object localization, and homography (given an input image, which of my known template images is matches it and in what orientation).
There's a lot of work being done on this specific part. If you have a standard RNN architecture you want to run, you can probably use the cudnn code in tf.contrib.cudnn to get a super fast implementation.
There is some performance work that needs to be done on properly caching weights between time steps of an RNN if you use a tf.nn.RNNCell. Currently if you want to implement a custom architecture, or a seq2seq decoder, or an RL agent, this is the API you would want to use. Several of the eager benchmarks are based on this API; so that performance will only improve.
I'm hopeful that for the next major release, we'll also have support for eager in tf.contrib.seq2seq.
Eager is actually not as innocent as "open-source projects borrowing the best parts from each other", as some commenters here suggest.
Google is attempting to dominate the machine-learning API and the Python ecosystem for scientific computing.
The company that controls the API influences which apps are built on it and how. Think about how Google bundled Android services on top of Android, and how that posed an existential threat to other companies. That's what's coming for TensorFlow. Many developers are too naive to realize it, or too short-sighted to care.
I wouldn't compare a permissively licensed library to Android services at all.
I didn't compare Tensorflow to Android services. I said that Tensorflow would serve as the basis of a service bundle, much like Android did. Let's come back in a couple years and I'll tell you I told you so.
It has to be consistent and there has to be one way to do it.
I personally have a 10 message thread with Google cloud support on exporting a Cloud trained model to tensorflow and nobody could figure it out [Case #13619720].
In fact if you dig up the case, then even official support told me that savedmodel needs some freezing using bazel otherwise it doesn't work.
The github page and stackoverflow are full of these. If you can, please take the message to the other side :(
I don't think the cloud guys (where training will happen in distributed mode) talk to the android guys (where models will be used after quantization). There is a huge serialization problem that all of us are currently struggling with.
We can do better, however, and we're working on ways to leverage the hardware better (for example, if you have no data-dependent choices in your model we can enqueue kernels in parallel on all GPUs in your machine at once from a single python thread, which will perform much better than explicit python multithreading).
Stay on the lookout as we release new experimental APIs to leverage multiple GPUs and multiple machines.
Chainer started it, was around years earlier, and it still has more users. So Google is not copying PyTorch, it's copying Chainer.
I really pray the tensorflow teams give it due importance.
Insulting the reader
> but Google is not a charity
truism
> They are pouring commercial resources...
As opposed to "non-commercial resources"?
> ... for a reason.
Everything happens for a reason.
> That reason is Google Cloud.
> How do I know this?
Pray tell!
> Because Tensorflow works better on Google Cloud than anywhere else.
This is the only real argument in this conspiracy. And if "anywhere" includes the users' hardware, it's wrong: tensorflow runs flawlessly on any Linux/NVIDIA hardware. Maybe it works better with GCE than AWS, but that would once again fall into that "rather unsurprising" category of factoids.
> Google is making a concerted effort to catch up with AWS in cloud, mostly through machine learning.
This can be re-written as "Google has a cloud offering, which it tries to sell. And right now, machine learning is pretty hot". Throwing a "concerted effort" in there is just trying to jazz it up to something ominous. Which it isn't.
> I didn't compare Tensorflow to Android services. I said that Tensorflow would serve as the basis of a service bundle, much like Android did.
"The basis of a service bundle" actually doesn't sound that scary. Nobody is disputing that Google offers services build on tensorflow. It just isn't any sort of "Trojan horse" conspiracy, and it is somewhat limited by the fact the tensorflow is OSS licensed and could be forked by anybody people suddenly find out it's full of geek soldiers.
Maybe, be people in this thread treat Tensorflow's creation as an act of simple altruism.
> And if "anywhere" includes the users' hardware, it's wrong: tensorflow runs flawlessly on any Linux/NVIDIA hardware. Maybe it works better with GCE than AWS, but that would once again fall into that "rather unsurprising" category of factoids.
Sorry, Tensorflow is slow on GPUs compared to other frameworks. This is not just an early blip, its a consistent pattern that has been repeatedly demonstrated. Why is Tensorflow slow on commodity hardware? Why isn't Google with it's infinite resources making Tensorflow run as fast as other frameworks on GPUs? Because it needs to demonstrate an advantage on the Google Cloud with TPUs.
On that cloud, it surrounds Tensorflow with other functionality that makes it easy to build AI, which aren't part of the Tensorflow project. Tensorflow is hard and inefficient to serve for inference, for example.
Machine learning is Google cloud's only hope to salvage Diane Greene's efforts and extend their dominance to a new sector. They're running a distant fourth.
> actually doesn't sound that scary.
It sounds scary to a lot of companies that don't want to be controlled or destroyed by Google. But by all means, lend them a hand, geek soldier.
Sample of search results: https://cloud.google.com/ml-engine/ https://cloud.google.com/tpu/
Even their docs: https://cloud.google.com/tpu/
Marketing content/training: https://www.coursera.org/learn/serverless-machine-learning-g...
vs (1 link I found with googling) for AWS: https://aws.amazon.com/tensorflow/
If we push the amazon equivalent though, run this: site:amazon.com aws mxnet
Every cloud vendor has their own framework. Microsoft has CNTK on azure as well.
Google doesn't want a repeat of what happened with map reduce and hadoop: https://www.quora.com/What-is-the-relationship-between-MapRe...
That being said, as a user: Just take it. You benefit from vendors competing. Google would love to pay you to use their tools.
1. We don't know what to use and its very confusing. For example, now there is https://stackoverflow.com/questions/42216208/should-tensorfl.... Will freeze_graph become canonical and we forget about SavedModel? And everything else deprecated? It should be part of the core API and workable on CloudML, where we don't have a lot of control on running scripts and certainly not Bazel builds.
2. Android/ios story. Now you have the Pixel Visual Core as well... Please make it seamless all the way to Android or Ios or raspberry pi (whatever you guys support).