Playing Atari with Six Neurons(arxiv.org) |
Playing Atari with Six Neurons(arxiv.org) |
The staging of components in this paper (compressor/controller), where neuroevolution is only applied to a low-dimensional controller, reminds me of Ha and Schmidhuber's recent paper on world models (which is briefly cited) [1]. They employ a variational autoencoder with ~4.4M parameters, an RNN with ~1.7M parameters, and a final controller with just 1,088 parameters! Though it's recently been shown that neuroevolution can scale to millions of parameters [2], the technique of applying evolution to as few parameters as possible and supplementing with either autoencoders or vector quantization seems to be gaining traction. I hope to apply some of the ideas in this paper to multiple co-evolving agents...
care to elaborate?
"To the best of our knowledge, the only prior work using unsupervised learning as a pre-processor for neuroevolution is (cite)."
Just amazing how much low-hanging fruit there still is in the space.
I am tempted to blame inconsistency across terminology and implementations for this lack of understanding but I suspect it has more to do with approaching this field through the lens of a developer and not a researcher or academic. Trying to understand the code without grasping the "science" of the mechanisms completely.
Either way if you feel to be in a similar spot check out this resource: https://reinforce.io and their respective Github repo: https://github.com/reinforceio/tensorforce.
Just reading through their code, and documentation has made a lot of the concepts clearer.
And a few more resources I found really helpful: http://karpathy.github.io/2016/05/31/rl/ https://www.analyticsvidhya.com/blog/2017/01/introduction-to... https://www.oreilly.com/ideas/reinforcement-learning-with-te...
Edit: My point that I forgot to mention was that I always feel like I am playing catch-up to understand what is going on half the time as the amount of new content being released exceeds what I can absorb.
One fine day my boss came to me and said that he had an ask from Atari Marketing (in the Home Computer arm of the company).
The marketing drone came to my office (yes, we had offices in those days). "My idea is to pre-copyright all possible 8x8 bitmaps so that people can't use them without our permission. Can you print them out for me so we can submit them to the copyright office?" He actually meant all possible 8x8 bitmaps containing five colors, with colors chosen from an 7 or 8 bit space (I forget which).
I told him the story of the guy who supposedly invented chess, and was offered a choice of reward by his king. The fellow simply asked, "Just give me one grain of rice for the first square, two grains of rice for the second, four for the third, and so on." Most of you know how this ends, it's grade school math.
I explained to the marketing guy that the printout would probably outweigh the planet, maybe the solar system, maybe the galaxy. He went away, a little disgusted with those pesky engineers. (I don't know if he was the same oxygen waster who wanted me to write a 16K cartridge in just a couple of weeks, but he certainly was in the same department).
So I'm still sticking with three brain cells, despite all the downvotes :-)
(weight of a sheet of paper) * 5^64
.4 x Milky Way mass
Almost half the galaxy mass, that's a lot of grains of rice!
The Atari that brought out the Atari ST etc. was one of those, but that pretty much failed and Tramiel merged it into JTS and later sold the remains to Hasbro which then sold it to Infogrames Entertainment. The current Atari Inc. used to be Infogrames, and just licensed the name. Infogrames Entertainment itself then renamed itself to Atari SA.
The other part of the original Atari, Atari Games Inc. failed in 2003. The intellectual property of that division is as far as I know now owned by Warner.
[0] http://www.wolframalpha.com/input/?source=frontpage-immediat...
This means that Ruby does not need to know _anything_ about the Python object: whatever you call on the Ruby object is just forwarded to the Python one, and whatever result is passed back to Ruby.
About the overhead, I sincerely do not know; I expected to have some so my code does part of the image pre-processing directly in Python (`narray`) in order to pass a smaller object to Ruby, but besides that I could perceive none -- grain of salt advised, as that was possibly hidden from me because my computation in Ruby was orders of magnitude more complex/time consuming than what was going on on the Python interpreter.
Definitely ping Murata-san either on GitHub https://github.com/mrkn/pycall.rb/ or Twitter `@mrkn`, I will send him a link to this thread so he can contribute if he feels like it. Personally, I am a fan of his work and elegant approach, I owe him for enabling me to keep working in Ruby while everybody publishes code in Python :)