I expected that around 10,000 people would have a perfect score in the class given my experience with Asian participation in online programming contests. There are a few countries in that part of the world where organizing a group to check each others' work and ensuring perfect scores is common and encouraged even when the official requirement is to do your own work. And those happen to be countries filled with millions of smart people. India, China, and Russia seem to be the heart of the phenomenon. Google, Facebook, and Topcoder have systems to deal with it but ai-class.org does not.
Anyway I'm neither surprised nor disappointed not to be in the top sliver. I'm not hireable anyway. It's funny that there is an identifiable top 1,000 at all instead of 10k+ perfect scores.
How do you know this?
Why? Because it is interesting. Find something to apply to that might add value to the business. If a enough people do that, AI techniques might become common place. Get on Kaggle, GitHub your next project etc. Putting what we've learned to practical use is how we can proliferate and disseminate this knowledge. This was the intent of the class in the first place anyway.
tl;dr Using this course as a gateway to your next job is shortsighted IMO.
You can't sit back and wait for AI to "happen". What would that even look like? The benefit of these techniques is in application to existing data; you don't have to quit and found some kind of computer vision/kinect/big data startup to take advantage of the knowledge.
You should be aiming to put yourself out of a job by using AI.
edit: why is this getting voted down? I would expect there are a fair number of people here who took the course with a casual approach, like I did. E.g., I usually watched the lectures early Sunday, then did the homework that night. Then, for those topics that I enjoyed and wanted in more depth, I read the sections in the book on them. This approach is relaxing and fun, but does increase the tendency to make silly mistakes and not catch them (which I in fact have done a few times now).
I am, it turns out, enjoying the material enough that I think I would like, if I ever find myself looking for a job, to find one where I can use this stuff.
That's at least partly due to an extremely effective teaching system and style, so I'd say it's a worthwhile tradeoff.
Personally I've always got a bit wrong in most of he homeworks but usually felt that while I understood the material I didn't recite enough time to rigorously check my answers and rewatch the materials to pick up on some of the subtle things you need to remember or some questions.
What struck me was how much importance he gave to this metric, which isn't that hard to game on an online offering.
I know a few people (who shall remain nameless) who collaborate and check each others answers and so on before submission, in direct violation of the Stanford policy (and have 100s or close to it), and so probably have received this mail, whereas more "deserving" (note quotes) people who honestly work through the course material may not because they have, say, a 85% or 90% score.
That said, my key takeaway from this is that professors are very impressed by perfect scores irrespective of how you got them. There must be something magical about that row of 100s. Once you set up a grading/ranking system, it is psychologically very hard not to admire people who end up at the top.
I am personally a little dubious that the people with the highest scores would make the best pool of employees, especially given that this is an online course without the programming component, but what do I know?
I wrote Java code for most of the algorithms in AIMA as a side project a few years ago [1], and after I read an online post by Peter Norvig saying a few of his students had tried and failed a few times (to implement the code in Java- Common Lisp code existed and the Python version was in its infancy), I sent him the code and this became the "official" java distribution for AIMA ( though I don't maintain it anymore- the immensely talented Ciaran O'Reilly of the Stanford Research Institute does) and no one ever invited me to Stanford or offered me a cool AI job[2], sob! :-P.
No I am not bitter I tell you, not even the teeniest bit :-p [3]
I wonder how this signalling will play into the upcoming courses? If there are tangential real world benefits to be gained by attempting a "perfect" score, then you can expect a lot more game playing wrt scores and exams.
[1] More about how Peter Norvig shredded my initial code etc here http://news.ycombinator.com/item?id=2405277
[2] though eventually, after a lot more work, it did lead to my working on good ML/robotics etc projects from Bangalore, which is a hard thing to do in the Great Outsourcing Wasteland.
[3] I am really not bitter.
I wrote the code for the hell of it, not to get a job. AIMA was my introduction to the fascinating field of AI. It is a great, great book and it has a lot more material than is covered in the course.
I once did want to go to Stanford and learn from the great profs there, but now in a "mountain comes to Mohammed" fashion, Stanford is coming to me. I don't care about the credentialling - I just want to learn. I took the AI online course and enjoyed Peter's and Sebastian's teaching immensely. Fwiw I should have a high 90's score, (I didn't add it all up) but nowhere near a perfect score.
Its wonderful that Sebastian & Peter are reaching out like this after all they have done so far. Congrats to everyone that slugged it out, and good luck on the final this weekend!
They will also probably look closely at that optional programming assignment as well.
I knew there was a reason Google let Peter Norvig take the time to do this ;)
Follow up is how we feel about that...
On the other hand, it was inevitable that as long as "grades" or equivalent are officially certified (even though the course is not for degree credit), that people would collect online courses as credentials. Wherever there's signaling value, there will be cheaters faking the signal.
Disclaimer: I got the email and missed two homework questions.
I made 6 100's, a 98, and a 97, and a 100 on the midterm and I got it.
:(
I was one of three researchers in a team at Yorktown Heights that did some of the best research in AI in the world, published a string of peer-reviewed papers, was the source of two commercial products, gave a paper at the AAAI IAAI conference in Stanford with for that year the 25 best AI applications in the world, personally won an award, etc.
I've taught computer science at Georgetown University and Ohio State University. My Ph.D. is in applied math.
Once in my career, just as a 'scientific programmer', in two weeks I sent a few resume copies, went on seven interviews, and got five offers.
But after my Ph.D. and AI work, I sent over 1000 resumes to Google, Microsoft, GE, FedEx, and hundreds more, got only five interviews, and no offers. I got a nice letter back from Fisher Black (as in Black-Scholes) saying that he saw no applications of applied math or AI at GS.
I ask you: Who will hire you in AI and why?
In business, hiring is because some manager has some work to do and a budget to do it. That manager believes that they know nearly all that is needed to do the work and otherwise would not be betting his career on the work. Thus, the manager is not hiring high technical expertise he doesn't have. Instead the manager is, as on a factory floor 100 years ago, hiring labor to add 'muscle' to his work.
In particular, unless the manager knows AI, he won't be hiring for AI. And there is at most only a tiny chance that the manager knows AI and even less chance that his project will depend on AI.
Moreover, the manager does not want competition from below and does not want his project 'disrupted' from below so really doesn't want technical expertise above what is needed just to get his project done.
Net, if you know some AI and want to use it in business, then find an application and start your own business. Then, since you know AI, you won't have to hire anyone in AI either.
What I've said here for AI holds for essentially all advanced academic topics.
Sorry 'bout that.
(Lest I seem overly harsh, I'm in the same boat.)
Basically your answer is like the classic "I could have got a 5.0, but I partied", which is a statement that says a number of things. 1. "I think all the kids who got 5.0 are boring and don't party" (untrue). 2. "I am not capable of getting a 5.0 and partying" (probably true).
That the person would even make the statement says something about their general level of intelligence. Hence when you say "If I had but known, I would have been in the top 100", you are saying "I had no idea that a hugely publicized course at a renowned university might lead to job opportunities". This lack of intelligence seems to contradict your first statement. Or it could be that you are not in the habit of thinking about such things. I hope you are either a kid, or an incredibly wealthy trust-fund baby living in isolation. Usually, however, I hear such remarks from people who are not smart enough to realize they are not smart enough, or they are quitters, or they are victims.
The "partying" I was doing instead of putting a non-casual effort into the class was dealing with some projects at work, and starting to read a book on analytic number theory because I'm tired of being someone who doesn't understand the proof of the prime number theorem.
> you are saying "I had no idea that a hugely publicized course at a renowned university might lead to job opportunities"
Yeah, I did not think it would lead to much in the way of job opportunities. That's because the class is just an introduction. If you compare almost any section of the class to the corresponding section in the Russell and Norvig book, you'll see that they are leaving out a lot of detail in the class. The homework in the class almost never pushes any limits of what was done in the lectures. Many of the homework problems are just things that were done in lecture with the numbers changed.
I would have expected the job opportunities to start at a deeper level than the class reaches.
I presume people are guessing how they are doing by just looking at their homework and midterm scores so far, weighted according to the appropriate weights.
Of course, I know it's their first class and you can't get everything perfect right off the bat. But I'd love to see the lab assignments being one larger project, some kind of task where you need to apply the principles learned to solve different problems. It would be also more effective in measuring/grading students.
On an unrelated note: I wonder what is the license for the classes/videos. I am working now in an adaptive e-learning platform. It would be cool to put their videos and questions in our system and see how effective the system is.
The course was awesome for what it was: a chance for the average programmer to get their feet wet in ML, but the fact that it was hosted by Stanford seems to lead to a somewhat humorous irony that it was pretty accessible whereas Stanford's real-life rigor is supposed to be nothing of the sort.
However, I doubt they have the bandwidth to manually grade thousands of assignments. Given the scale of the class, the grading is forces to be automated. It's hard to come up with such assignments and even harder to make them challenging, given an automated grading script.
So if ML numbers are in any way similar to this maybe it wouldn't be so bad. That is if there was such a letter.
I'll add that Professor Widom's style is quite engaging as well.
The correlation will be interesting. Undoubtedly the top 1000 is made of many of the type of people that must be the best at whatever they try (not exclusively of course) and with a decent amount of time on their hands, after all there is no disincentive whatsoever for not completing something and no previously known reward for perfection (I guess a piece of paper can count). Given some wording in some of the Qs (actually my only complaint about this course) a lot of care must be taken to ensure that every question is right, so being in the top 1000 is no small effort.
That said I have a job I like already, and plan on using what I learned here.
Those 1000 with more time and drive to be the best probably are some of the best job candidates though. Plus chances are they do need a job after all.
So their metric isn't ideal for identifying passionate poeple ;-)
Second, I registered for the Machine Learning course (I am not sure if the same applies to the AI course) and I compared it with the actual ML course at Stanford (CS229) (I mainly looked at Youtube videos of Andrew[1] as well as Assignments/Midterm[2]). The latter is by far more advanced and theoretical. The assignments tend to test more than basic comprehension of the material presented in the lectures, which is exactly what the online course reviews tend to evaluate. They require strong mathematical knowledge and obviously a minimum level of creativity/intelligence.
[1] http://www.youtube.com/results?search_query=machine+learning...
I don't. That part of the post was written with tongue firmly attached to cheek. If that tone didn't come through, that means I have to improve my writing.
The online ML course is CS 229A (which is also an actual course at Stanford. The online version is close to the Stanford course).
The "tough" version is CS 229 (no 'A' at the end). I registered for the ML course thinking it was an online version of CS 229 and dropped out when it was confirmed to be 229A. In my politically incorrect opinion, 229A is close to worthless. The math is important in real world ML. This course included gems such as "if you don't know what a derivative is, that is fine".
The online AI course is almost exactly the same course as Stanford (CS 221), minus, of course, the programming assignments. It is an introductory, broad based course, and it does the job well (imo)
The online DB course is almost (if not exactly) the same as Stanford CS 145. I think this was the best course of the three.
All courses track the corresponding Stanford courses.
Another interesting aspect of this "online ethics" is that there's no technical measures preventing you from pulling up past quiz results, and even if there was, you could still keep previous sessions open in a browser and even if there were countermeasures against that (javascript erasing), JS could be disabled, etc.
The Programming Projects that ML class had were slightly better metric of performance as there's more work that would have to be plagiarized, and if you're just going to go through life outsourcing all of your work then I guess that's your prerogative. However, I think that if you wanted to be very serious about actually testing for knowledge of material then the addition of some sort of interview component (phone/skype session), while time-consuming, could help.
1) you immediately know if you got it right or wrong when you submit, so you can to a lesser extent brute force the correct answer
2) with the exception of maybe the first assignment, they are all "fill in the blank" sort of programming assignments. You basically just have to find the equations they give you in the PDF, translate them directly to Octave, and bam you're done.
I personally have only scored straight-100%s in a single course (Python programming), and that was only because I was relatively an expert in the material before the course began.
The work you did strikes me as a far better measure of character and subject understanding.
Why would someone do this for a free online course that gives no credit for a degree? I mean, the whole point is to learn, not get the highest grade. I really fail to understand people sometimes.
* rather than just giving up on a problem, you can talk it out and learn together * you get the opportunity to teach the material that you think you know that others find hard (a good heuristic for problems you may have just barely understood, but gotten correct anyway). Teaching material is a great way to learn it, and expose any gaps you might have in your knowledge. * instant feedback on problems while they are still fresh in your memory
Your final score will be calculated as 30% of the score on the top 6 of your 8 homework assignments, 30% your score on the midterm exam, and 40% your score on the final exam. For those completing the advanced track you will receive your final score as a percentage as well as your percentile ranking within all those who completed the advanced track, and this will appear on your statement of accomplishment. The statement of accomplishment will be sent via e-mail and signed by Sebastian Thrun and Peter Norvig. We hope to have them digitally signed to verify their authenticity. It will not be issued by Stanford University.
So, since this thread was about 'job placement' for students who did well in an AI class, I posted about my experience getting hired where part of my background was some expertise in AI.
Net, I have to conclude that, in getting a "job", expertise in AI will be from very rarely helpful to often a serious disqualification as in "It appears that you are overqualified for our position and would not be happy in it".
Edit: for example some opportunities here http://www.kdnuggets.com/jobs/
The main issues were (1) I am a US citizen and, thus, not on an H1B and (2) I was over 50.
Generally a subordinate is supposed to be younger.
I'm at a 4-person startup and AI is the core of what we do. My e-mail is on my profile if you (or others) are interested.
You sound like a qualified candidate, if only you'd sent your resume our way.
Sorry 'bout that.
Finally I gave up on getting hired and decided to start a business.
I'm now deep into that effort and at least for now should continue it.
I.E., it would make sense they only want to hire people under certain age, or certain academic enrollment status... Who knows...
Aptitude and interest goes a long way.
Definitely learnt a lot, which surprised me because I did 3 classes that were based around AI and probability in my course. It has enlightened me a bit on the difference between a top of the range CS college and where I did my degree just it terms of how much ground can be covered and covered well.
Well, the only two people I personally know who would get all 100s are Peter Norvig and Sebastian Thrun, and I personally wouldn't mind hiring them!
Of course, in reality, Peter Norvig and Sebastian Thrun are working on projects that have long time horizons, e.g. self-driving cars and search. So perhaps you're still correct: The people you would hire to bang out code to meet a short deadline are probably different from the people you would want to work on your long-term technology bets.
In general, I disagree that knowing a topic incredibly well is necessarily overfitting. Deep knowledge can only aid new insights. You often hear about mathematicians and physicists who think by inhabiting their own mental world, composed of insights that they hold so deeply that they are _intuitive_.
"But in education circles, Mr. Khan’s efforts have captured imaginations and spawned imitators. Two Stanford professors have drawn on his model to offer a free online artificial intelligence class. Thirty-four thousand people are now taking the course, and many more have signed up."
Not sure if "taking the course" means advanced track and completed midterm or not.
It also included other gems like debugging models with learning curves, stochastic gradient descent, artificial data and ceiling analysis. I have not come across practical things like these in more mathematically oriented ML books that I have tried reading in the past.
Interestingly, your arrogance is in sharp contrast with the humility of the professor, where he admits in places that he went around using tools for a long time(like SVM) without fully understanding the mathematical details.
I'd hardly call it worthless myself. It lacks a deeper analysis of all the methods that are used, but using them can sometimes be a greater challenge.
I did the AI course and the ML course and find it a great way of getting a little overview of the subjects, so when I study on my own, I have a little direction.
A bit of me died when I heard prof. Ng say that. However, I had committed to finishing ml-class and I did. As of now, I'm glad I went through with it. I felt like I was learning all these cool AI techniques that I hadn't heard about. However, the proof is in the pudding. The question is will I be able to take a real world problem and apply what I learned in that class to come up with something interesting? If I can't you are probably right. My perfect record would only be worth the paper it's printed on and the money I paid for the course!
I'm not pointing fingers at Prof. Ng. or anyone here. It was an experiment for Stanford and an experiment for me. I know I am looking forward to the courses next year :).
Elsewhere on this thread I've outlined how business 'handles' new technology.
So, yes, maybe a big credit card company gets up on their hind legs about credit card fraud and wants to attack the problem with AI, big data, machine learning, optimization, etc. So they set up a group.
First problem: Some high executive looks at the group, doesn't see what he expects and respects, gets a headache, and kills the group.
Second problem: The group gets some really nice research done, writes a technical paper, and gives an executive briefing; some high executive gets a headache, concludes that the group is just engaging in 'theoretical nonsense', and kills the group.
Third problem: The group is making good progress, has some running software with good estimates that credit card fraud losses will go down by 75%, thus giving an ROI off the tops of the charts, with essentially no effect on non-fraud (false alarm) credit card operations. Some executives elsewhere in the company start to feel the heat of internal competition and work to shutdown the project.
Fourth problem: The project is rolled out in production and is fully successful. There is no more need for the group, and it is disbanded with everyone fired. The head of the group has his house foreclosed, and his wife leaves him. He sends 1000 resume copies and joins his brother's business mowing grass.
Lesson: Being a technical employee in a big company where 95% of the people and nearly all the executives are non-technical sucks, i.e., is a career long walk on a short pier.
Net, big, old companies are nearly always just unable to work effectively with new ideas. Exceptions are possible but rare in practice.
Larger lesson: Start a successful company and either run it and pass it down in the family or sell it for $50 million, $500 million, whatever. Don't be the factory floor worker. Instead start a company and be the CEO. Use special technical expertise as the advantage.
Can plug together one heck of a Web server computer for $1000 in parts. Heck, just in starting a pizza shop, one pizza oven might cost more than $1000. The pizza shop needs to pay attention to a long list of regulations, but a Web server doesn't,
One of the good things about the US is the relative ease of starting a new business. If want to use technical abilities to start a company and sell it for $50 million, then the US is the place to be. Thank the big, old companies because they are the ones willing to spend $50 million buying a company based on less than 100 KLOC!
You are "overqualified". People will read your resume and assume that you will be unhappy with a regular software engineering job, and therefore not hire you for such a position.
Like it or not, you'd need to find an AI job where people needed your particular skills. (Or something closely related.)
After my PHD I got a postdoc in a 3 year EU project just finished. Now I am craving to get out of "academia" and get into software development again. The problem is that I would consider my development skills as a "junion" or "mid level" developper but without hardcore expertise in a technology.
And the worst problem is that as you say, a lot of companies that see my Resume see "PhD" and think "overqualified".
Recently I tried appliying to a group some company that is doing Machine learning with the hope that they will see a PhD as a "feature" and not a bug.
At the time, 1995-2005 (see my post below) AI was not much on the radar of companies. It would, could, and should have been but was not.
But asking that a company need "particular skills" that are a bit advanced is, as I explained, fundamentally something of a long shot.
Net, if someone has some advanced expertise and sees an application, then they should just start a business and there be CEO-CTO-CIO, and Chief Scientist along with chief floor sweeper until they get funding and/or revenue and can hire people.
As with the homeworks, exams must be completed individually without the help of other people.
https://www.ai-class.com/overview
I think that lots of people probably did collaborate on the homeworks and the mid-term and likely will do so again on the final and it was and will be cheating.
It's a shame, especially given that the instructors do seem to be attaching some importance to students' scores and rankings, but I'm not letting it detract too much from my enjoyment of the class.
Near the end of the visit, a nice girl in their HR office walked me to a bulletin board they had with their legal announcement of their job openings and their claim that they had to hire H1Bs because no qualified US citizens were available. We didn't say anything to each other, but the scam was clear, Hope they didn't fire her.
A friend, who worked with me at Yorktown Heights, recently went on an interview for a programmer slot. Apparently all the programmers were on H1Bs from Taiwan, India, and Russia. My friend didn't get hired. He's terrific at C, C++, C#, Visual Basic, .NET, FoxPro, T-SQL, and system management and administration of Windows Server, SQL Server, and Exchange. And he's an expert in AI with an applied math Ph.D. from one of the world's best math departments. Yup, guess those guys from India, etc. were 'better qualified'.
The early part of my career was greatly helped by the Cold War and the Space Race. That's why in two weeks I could go on seven interviews and get five offers. Yes, that was near DC. At one time I was making six times what a new Camaro cost. I still have the Camaro!
But as has been documented, about then some executives from industry and government got together to see if they could change that 'horrible' situation. The result was that the NSF set up a team of economists that did some supply-demand calculations and estimated how many more 'tech workers' would be needed to 'solve the problem'.
Then to get the 'tech workers', the NSF wrote into academic research grant contracts that so many students had to be supported. And, hint, hint, hint, such students are available from Taiwan and India.
So, for some years freshman calculus classes were taught by graduate students with good understanding of Chinese but poor understanding of English. Of course, if are going to study a subject in English but don't know much English, then about the easiest subject to study is math since the vocabulary is very small and the terms are very well defined.
Soon US citizens in college walked into computer science classes and, on the first day, saw only a minority of US citizens, sensed something wrong, and walked out. So for some years during rapid growth of computing in the US, academic computer science was very short on US citizens.
Really, the H1B program was designed and intended to flood the US labor market for tech workers, and basically the program worked.
Congress was getting pushed from two sides on the H1B situation. But 9/11 provided an excuse to throttle all immigration (except from Mexico!), and the permitted H1B slots were shrunk.
Now some tech industry executives are back at claims that the US needs more immigrant entrepreneurs to get 'skills' in 'short supply' in the US. So, amazing situation: In the US families commonly have one heck of a time paying for college. Even in the US, a good computer, printer, Internet connection, work space, etc. for learning computing is somewhat expensive. A lot of bandwidth to US servers is needed for downloads. Yet somehow in countries with average family incomes 10% of those in the US and 10,000 miles farther away from US servers people are 'better trained' in 'technical skills'. Amazing.
No, it's just an old story: Economic activity needs land, labor, raw materials, capital, etc. Anyone with one of these likes to believe that their part is the most valuable. So, the people in the US with the capital, and who never wrote 100 lines of software or invented an algorithm, tend to believe that they have the most brains, the most valuable part, and should have the most power and that labor should be like workers on a factory floor 100 years ago. They wrap themselves in notions like "The US is a nation of immigrants". Yes, Mayor Bloomberg, you are one of those people.
To me these are now very old issues and there are some larger issues:
First, Moore's law and related 'laws' for other hardware have been charging along so fast that what can be assembled for a development computer or a first server for $1000 in parts is astounding.
Second, common US Internet bandwidth is beyond belief, even for a server. Just do some arithmetic assuming a Web page that sends for 200,000 bits, with three ads, with some reasonable 'charge per thousand ads displayed' (CPM), and an Internet connection with 15 Mbps upload bandwidth for less than $100 a month, assume half fill that bandwidth 24 x 7, and estimate the monthly revenue. THen can join the supercharged Corvette of the month club or the 50 foot yacht of the year club. Multiply it out and see.
Third, US technical graduate education still totally knocks the socks off nearly all the rest of the world. If have some such education and some research and also a good application, then get a computer, type in the code, go live on the Internet, get users, ads, ad revenue, a Corvette and a passenger about 5-4, 110 pounds, good figure, natural blond, cute, sweet, majored in art history, good at cooking, sewing, playing piano or violin, singing, wants to be a wife and mommy, ...!
Yes, I've published in mathematical statistics, that is, the more serious version of 'machine learning'. And I've published in optimization, that is, the more serious version of 'planning' in AI. And my Ph.D. research was in stochastic optimal control, that is, the more serious version of the AI 'planning over time under uncertainty'. And I've done some applied math research for my project. I recommend this path instead of 'computer science'.
That is, if going to grad school, I believe that there are some serious advantages in a carefully selected collection of topics in applied math instead of 'computer science'. Start with an undergraduate major in pure math. Sure, somewhere learn to write some code and then get, say, three hours of lectures on 'algorithms and data structures'. In graduate school, take seriously measure theory, functional analysis, probability based on these two, stochastic processes, optimization, and mathematical statistics, at least. My guess is that you will have the best tools for the future of computing and 'information technology' entrepreneurship and won't have much competition from outside the US or even inside. And those math classes are NOT crowded!
I need to get back to it!
Yes, I know that Google, Microsoft, etc. should be using AI for ad targeting, search ranking, scam and fraud detection, etc. Maybe they are.
Still from the tech news it does appear that the best way to get paid for such work from such companies is to do a startup with such work and then just sell your company to one of those companies.
I still believe that the ability of US big business, with its many 'traditions', will have one heck of a tough time running projects with anything advanced technically and not already common in the company. In simplest terms, the existing middle and upper management has essentially no ability to work productively with anything new.
Indeed, broadly the leading US research universities are terrific at working, for as far as they go, with new technical ideas, and business is just AWFUL at it. So, super tough to get a business to buy a technical idea, but they will buy a business based on a technical idea.
Oh there is a history: Off and on for decades business has tried to make statistics, optimization, AI, etc. work. The pattern is some new buzz words, some hype, some projects, some project failures, and then all's quiet again. Thing is, of course, eventually the ideas have to work. Business doesn't know how to make such ideas work. Parts of US research, in or close to academics, is from a little better at making ideas work to much better. E.g., look up the video of Eric Lander's lecture at Princeton 'Secrets of the Genome' or some such and see what he made work with new ideas and new hardware.
In simple terms, 'business' gets a 'business model' that is working and then hires 'line managers' to 'manage' the execution of this business model. Typically when the model dies, the business dies. Doing well with work that is new and advanced within the on-going business is not common. Yes, one of the reasons is part of the tax code -- it can look better for a company's financial condition just to buy a business for, say, $50 million, than to develop in-house.
The good news is, this is a GREAT time for information technology startup entrepreneurs. Don't struggle against your problems; instead, pursue your opportunities.
Back to it.
1) They provide some set of data and establish rules for the competition.
2) They implement their own solution to the challenge, and that is the benchmark.
3) A "passing grade" is obtained by getting any working system.
4) The actual grading is then given on a curve, compared against their benchmark.
That said, learning what the algos are and how they work is one thing; learning how to actually apply them to real life situations is another thing. I think the class leans quite heavily towards the former, but I really love the few glimpses of the latter.
Personally, as someone who is new to the field (didn't do maths at college) & is barely fitting the classes & exercises around a fulltime workload & other things, I am glad that the programming exercises are "easy". Some of them are ridiculously easy, agreed (where 1/2 the solution is given basically verbatim in the pdf notes, and the other 1/2 in the code comments) - but for most of them I think it's enough to wrap my head around what's actually happening, especially in terms of the multiclass neural network assignment. That gives me enough foundation to try to apply them to real-world situations on my own time.
And he knew all the tricks. I don't think anyone ever beat him. And he didn't show anyone any of the solutions until after the final.
I felt like I learned more from the few minutes I spent reading those solutions than I did during the rest of the course.