Munich 1991: The Roots of the Current AI Boom(people.idsia.ch) |
Munich 1991: The Roots of the Current AI Boom(people.idsia.ch) |
And while it is very true that often the research coming out of Academia is useless, what is always neglected are the roots of the research done in private labs.
When Jürgen Schmidhuber and team published their work on Neural Nets back in 1991 it was also useless. Unless you had a supercomputer and very, very deep pockets you were not going to do anything with what came out of their lab.
But still, 30 years later here we are, standing on top of the shoulders of this useless research.
The closest to that that I've seen is that traditional academia approaches are too far removed from practical applications for highly applied fields like software engineering, or too slow for fast-moving fields like modern day ML (thus, all the preprints).
Practically no one is against hard science research, properly conducted. The issues are rampant fraud / p-hacking / unreproducible garbage mixed with an unhealthy dose of ideological monoculture and indoctrination, garnished with rising tuition prices while sitting on huge endowments in case of the Ivy Leagues.
However I often see this going from "there's issues" to discounting academia altogether and positioning private labs as a good or only alternative.
After all, most people in the open science collaboration which published the seminal paper kicking off the replication crisis were from academia.
No, the research was made in USSR, however much Scmidhuber likes to think of "occupied Ukraine".
I mean, if one thinks it is his mission to establish the truth..
The truth, Scmidhuber, was never in your fuhrer's hands. Nor it is in the hands of the western fuhrers of today.
Just for the context, today is Russia's Commemoration Day of the victims of the Velikaya Otechestvennaya Voyna (your translations always feel wrong, sorry) of 1941-1945. (Yes, 1941, when the western fascist coalition of 5 million soldiers, invaded Soviet Union.)
26 and more million people of USSR perished, 13 million civilians. Of all nationalities.
In the Schmidhuber case their is 20 years and a chain of countless other works in between the two.
Indeed I remember buying a set of three conference-papers-as-books around that time, titled Artificial Neural Networks .. proceedings of the whatever the conference was.
No doubt Schmidhuber made important contributions, but I see him pop up claiming to be the 'root' of it all every couple of years.
related paragraph from Wikipedia:
Modern backpropagation was first published by Seppo Linnainmaa as "reverse mode of automatic differentiation" (1970)[26] for discrete connected networks of nested differentiable functions.[27][28][29]
In 1982, Paul Werbos applied backpropagation to MLPs in the way that has become standard.
Both papers are direct applications of the chain rule applied to estimate the gradient of a multivariate function.
And that's where Schmidhuber goes off the rails: publicly shaming published papers into citing you isn't good academic practice. It's bullying.
You can't claim independence from past work simply because you didn't look directly at it. The job of an academic researcher is to know the landscape of relevant ideas, where they come from, where they're going, and to hopefully contribute a few new good ones.
Citation chains should extend back from your work, along a reasonable line conceptual inheritance, back to a reasonable point of origin. Schmidhuber has different definitions for both of these reasonables than the bulk of the ML research community, to a point that makes him difficult to satisfy.
But if you build on them you should have read them. I don't know about the specifics and I don't know if Schmidhuber is out of line or not, and citations and impact factors are a terrible mess, but generally speaking, you are responsible for finding and reading and citing any related work that needs to be cited, and if you work on neural networks in an academic context you probably have been forced to read that particular one at some point. Citation obligations don't just disappear because you don't want to do the research.