The Story Behind the Worst Movie on IMDb(fivethirtyeight.com) |
The Story Behind the Worst Movie on IMDb(fivethirtyeight.com) |
I'm thinking this movie otherwise gets forgotten in the trash bin of bad movies and the data would never tell you anything because it wouldn't exist.
To me that's just stating the obvious. Of course if there is such a thing as a worst movie then it will have a higher percentage of 1 star votes than other movies. So I don't know how that's evidence for anything except that the movie seems to be bad.
I though the majority of IMDB data was not downloadable?
Also you can buy a license for access to more complete datasets.
seriously, i do take IMDB ratings into account, but i consider them unreliable at best. Inception, when it came out, was the best movie of all time for a while, according to IMDB users. enough said.
For me the best place are meta sites that gather from many related sizes.
At Rotten Tomatoes they rated Seth Rogen-vehicle "Neighbors" as 100%, last I checked. Meaning only: no reviewer at that point had said it was awful. And yet, I saw it and it was awful.
At IMdB, I can see that it rates as 7.6/10.0, and also that teenager voters loved it (9.0/10.0), where older viewers hated it (5.0/10.0). Far more useful information.
Gunday (the film of the parent article) gets fairer treatment from the IMdB's top 1000 users, who rate it 4.9/10.0
Each site has its own user base, and those users have their own biases. A review from Rotten Tomatoes will vary greatly from those of sites that include only noteworthy critics or only crowdsourced opinion without the "community" aspect. Some communities will be more critical, while some will be less. Like most online reviews, the criteria for rating is completely subjective; one user gives it a 10 because it had their favorite actress in it, while another user gives it a 5 because there was a scene they didn't like. What's awful to you may not be awful to me, and crowd-sourced data or aggregate generalized polling isn't a great way to distinguish that.
If you look below that, you can see the actual "Average Score", which in the case of "Neighbors" is actually lower than IMdB's, at 7.1/10.
The release for that movie is the 9th, so its not out yet outside of limited distribution. Wait until all the reviews come in this weekend for a more complete score.
In general, I don't trust online reviews for anything because I don't know what 'normal' is or the motivations that people have for voting, as you say here. If I read any restaurant reviews or ratings, it's under the assumption that they're astroturfed.
But, IMdB provides the online reviews I trust the most. I know that the dataset is large enough to be reliable. I feel that people are motivated to vote for their own reference, with minimal outside agenda (this exception is newsworthy because it's rare). As an encyclopedic reference, the site is politically and culturally neutral, compared in particular to national newspaper reviews. I know from experience how the IMdB normally and consistently rates films/genres I might be interested to watch (I know its biases, compared to my own). The stats are openly broken down demographically, with the statistically crucial number of responses, which I feel confident in interpreting. I also know, for example, that new releases will be over-rated according to how close they are to their release date.
By averaging additional information from other sites, I believe it would be difficult to retain those subtle points, which are important to me. I certainly don't believe that Rotten Tomatoes contains the same nuance of information.
IMDb gives a 6.9/10 from 119,115 users. That's the only real statistic it gives us; there is no 'Metascore' for this movie. (Personally I think the Oscar nomination should be mentioned next to this score, but it's sort of buried further down the page)
Rotten Tomatoes, on the other hand, shows us several numbers. The Tomatometer is at 50% for "All Critics", with an average rating of 5.7/10 and 52 reviews. The "Top Critics" Tomatometer is at 36%, with an average rating of 5.9/10 and 14 reviews. The Audience rating, however, is 73%, with an average rating of 3.4/5 from 333,273 users.
--
The above example shows how you can sometimes use IMDb's stats of general ratings as a median reference between the audience numbers and critics numbers. But this doesn't always paint the best picture. Let's take another example: the recent (and generally accepted as a flop) 47 Ronin.
Here, IMDb gives us a 6.3/10 rating from 60,849 users. But it also gives a Metascore of 29/100. Yet the number in a gold star in bigger font is just the '6.3'. So even though this movie has a dramatically lower Metascore, they only feature the general user rating from IMDb.
The story is much more dramatic on Rotten Tomatoes. It has a 13% 'All Critics' Tomatometer rating from 72 reviews, with an average rating of 4.1/10. The 'Top Critics' Tomatometer shows 0% with 13 reviews and an average rating of 2.9/10; not one single top critic liked this film, out of 13 critics! Yet the audience rating shows 51% with an average of 3.3/5 with 53,921 ratings.
We can see how IMDb got its '6.3' rating here: both seem to show a middle-of-the-road rating from general audience ratings. But Rotten Tomatoes' critics stats show this movie isn't worth wasting two hours of your life on. The gamble of whether you may like the movie or not becomes more certainly less likely when you take the critics' reviews into consideration.
--
As we can see above, even within each website, sometimes they have not only greatly varying statistics, but sometimes missing or hidden information (Robin Hood not having a Metacritic rating and not prominently displaying the oscar nod). People weigh their options based on the data they have at hand, so the information you give people - along with its context - will change their minds greatly, regardless of the source.
This is why I think it's much more realistic to look at multiple different sites. You need as much information as you can get, and no one site gives you all the relevant data, as it varies from film to film and user community to user community. It would seem you can't just depend on 'the crowd' to give someone an accurate idea of whether they will like a film; a survey would probably be better, but nobody's going to survey every film they watch.
Even if you limit the field to movies that have seen a theatrical release, the same principal applies. Some movies are shockingly bad, but are made with few little effort by people who know they are making a bad movie. These seem less interesting, less sad and less funny than very bad movies made by people trying very hard and spending lots of money.
Personally, when rating the badness of work, I judge it against my own assessment of the potential of the work. The potential isn't directly measurable or quantifiable, but it consists of things like the budget, the vision of the creator, the skill and effort of the artists involved in the creation, etc. etc.
So, for example, I think of Star Wars: Episode 1 as being a worse film than a lot of 50s low-budget sci-fi B-movies, even though I might choose to watch it more readily. The B-movies were pumped out without much effort. They're boring and poorly made on a technical level, but they never really had a chance to be any good. Episode 1 was made on a vast budget, drew on extremely rich source material, and was the culmination of unimaginable amounts of time and effort by a large number of hard-working people. For me, that makes the failure of Episode 1 more profound and more egregious.
What about a blue screen?
But no, a game company is the worst because a bunch of fanboys rallied to say so.
It's really quite fascinating to me how a movie like that, so incredibly terrible with seemingly no irony in it's production, has somehow become a fantastic work of art. It's that piece in the modern museum you think looks absolutely so terrible but something about it appeals to you.
I've probably seen The Room a half dozen times in various settings (movie theater showing it in jest to a huge audience, group of friends over a few beers, etc), and it's always overwhelmingly entertaining because it's just THAT bad. It almost impresses me more that they pulled this off (unintentionally, of course) when compared to some of my actual favorite films.
It's fascinating because of some similarities to the truly worst movie in history, The Tango Lesson[0]: each movie has one person as writer, director, and star, and (apparently) focuses self-indulgently on the abortive romantic entanglements of the author, who is utterly unaware that nobody else could possibly care.
The Tango Lesson falls through the bottom of the badness scale into a realm where it can't even be enjoyed for its unredeemable awfulness. Everyone I was with thought it was dreadful, and what was left of the audience shuffled out shaking their heads.
The Nostalgia Critic reviews The Room: http://thatguywiththeglasses.com/videolinks/thatguywiththegl...
The 'best' bad movies are hard to quantify. I certainly can't think of any 'good' bad movies that planned or aimed to be regarded as bad. As another commenter said, you need the creators to believe that they were making something worthwhile...
I would be interested to see how its rating has changed over time.
There are certainly films that are "so bad they're good", but then there are just bad films.
Unexpectedly, however, "Plan 9 from Outer Space" is rated much more evenly for a film that's been considered to be the worst of all time.
I'm a die-hard MST3K fan and I've only seen Manos two or three times.
Even so, I Heart Huckabees tops my list of movies that I just can't watch. That one's actively grating rather than just incompetent.
R.O.T.O.R. - http://www.imdb.com/title/tt0098156/ratings?ref_=tt_ov_rt
My personal favorite bad movie has always been "Cool As Ice" [1], but even that seems to be rated higher than R.O.T.O.R.…
[1] http://www.imdb.com/title/tt0101615/ratings?ref_=tt_ov_rt
FiveThirtyEight has had a rough start so far so I may be more inclined to seek more nuance than I normally would as they've tended to be extraordinarily simplistic (Silver himself notwithstanding) to date.
I think the author calls data erroneous as the rating does not represent the actual quality of the movie - a thing which IMDB wants to achieve. He doesn't call it erroneous for actual errors in the process of collecting ratings.
I think, as you said, you are just inclined to seek more nuances.
In the (paraphrased) words of Andrew Gelman:
"I can't knock this guy for slamming my book without reading it. For example, I have never read the autobiography of Uri Geller, but I'm confident it's full of crap."
If you don't think Gonojagoron Moncho is lying to these people about the point they're taking offense over, how is it relevant that they haven't seen the movie?
I'm sure it's possible, but no rating represents the "actual quality of the movie." A rating is based on a subjective amalgam measurement that encompasses a myriad biases, even when watched and with total scrutiny.
Far too often do you find a movie (usually a somewhat recent release) that's just flat out terrible despite having a > 7 score on IMDB (and usually a Metacritic score in the 30's or 40's)
Example: "Spy Kids" got a 93% fresh rating from critics, but 45% from audience ratings.
Code attached should you want to investigate for yourself.
If I had unlimited time and money I would create a new movie review site that fights corruption. I would have a phone app for doing the reviews and it would require taking a photograph of your ticket stub at the theater and uploading it for proof (w GPS tag). It is far too easy and anonymous to participate in these online movie ratings sites that have a large impact on people's decision making. I have more ideas for the review site if anyone wanted to discuss it further.
If you're interested, there's a really interesting documentary about Troll 2 called Best Worst Movie, which covers the film's production and cult following. After watching the documentary, Troll 2 becomes an enjoyable movie because you know all the crazy backstories to the actors and scenes.
It's interesting that so many people gathered solely to express "we want this guy dead, not imprisoned." I wonder if there's more context?
Given the dynamics of Bangladeshi politics, a life sentence from the current government is actually a jailing until the opposition comes to power and pardons everyone convicted by its predecessor.
I'm against capital punishment, but I understand where the protesters are coming from: they want real justice for the atrocities in 1971, and see the death penalty as the only way to obtain lasting closure.
Politics in a democracy of 180MM people is really, really messy.
That, uh, seems like plenty of context to me? Throughout history, seriously bad dude plus nationalistic fervor equals lynch mob pretty darn often. The seriously bad dude part is usually even optional!
The most obvious effect is movies that drop in the rankings 30 days after release; people who just saw a movie give it 10/10 and then stop participating, so 30 days later they are no longer considered 'active' and their vote stops contributing to the overall ranking.
If someone only gives 10s or 1s to everything, their vote of 10, should probably have less weight, than someone who distributes their votes more evenly.
Did they start hanging out with the bad kids, take up cigarettes, drinking, gambling only to progress to crack and burglaries one of which ended with our Data shooting a home owner who returned unexpectedly?
I guess I don't understand what data is. I always thought it was a set of values. And I always thought that the problem when using data was in the interpretation, and that a prudent consumer of data would always be careful to distinguish between a random sample and self-selecting sample when drawing conclusions, and then would only state conclusions couched in the language of statistical inference.
Leaving aside the question of why I should give a fuck about this supposed outrage, why does the author expect there to be a strong correlation between movie quality and the ratings on a website devoted to providing entertainment by having users rate movies?
When The Matrix is purported to be better a better movie than Lawrence of Arabia, the problems of interpretation are systemic.
That tipped the romset into the spotlight - there were some leaderboards for highest recent activity and so on. Other people started downloading the romset and voting on it.
Suddenly this obscure romset was catapulted into most of the lists for "most active" and "best" etc.
I had rated the game honestly. I had fond memories playing the cab for a week on holiday in my youth. But I was surprised that so many other people felt the same, especially on the Mame platform where the game's controls made it tricky.
All my deliberate attempts at voting shennanigans failed miserably. (Although I haven't investigated MTurk or similar yet.)
I wish there was a site like Meatball wiki where people could share their vote-weighting methods.
Obscure? :-(
This was my favorite arcade game as a kid. I don't think it was very obscure. Though I have an actual Tron arcade machine not 10 feet from me, so perhaps I'm the wrong person to judge that…
This one has caused a stir on IMDB for reasons of moral integrity, or bending the truth. Rather odd, since this has been going on in films and similar media for a very long time.
"International Gorillay", a film from Pakistan depicting Salman Rushdie [1]. Coincidentally it was released a year or so after Rushdie's "The Satanic Verses" [2]. Ayatollah Ruhollah Khomeini [3] wanted Rushdie dead because of this novel. BTW Iranian films can be very good, like "Where is the Friend's Home?" [3].
[0] http://www.imdb.com/title/tt0052077/ [1] http://www.imdb.com/title/tt0251144/ [2] http://en.wikipedia.org/wiki/The_Satanic_Verses [3] http://en.wikipedia.org/wiki/Ruhollah_Khomeini [3] http://www.imdb.com/title/tt0093342/
The only movies that escape from the IMDB average are a) decent movies that are loved by the masses, b) great movies the masses don't watch (being in black and white or not in English alone pretty much guarantees 2 bonus points) and c) movies everyone agrees on are total crap.
What you may want is a personalized rating based on people with the same tastes as you.
I haven't used it in a while, but I used to find MovieLens, from the University of Minnesota, useful for that: http://movielens.umn.edu
It even has a feature where you can get joint personalized recommendations for you and a friend (assuming the friend also has a MovieLens account, of course), which is useful for brainstorming movies to rent with someone that might be mutually enjoyed.
Data analysis by Eugene Bialczak. Also, a disclaimer: the author wrote much of the IMDb Trivia App.
Besides, for us in small countries, we would probably have almost no ratings in any film besides blockbusters.
It's similar to the problem of an author's "best" or most notable works getting lower crowd-sourced reviews than their average work. What happens there is that generally people only bother to read, and subsequently review, books that they think there's a good chance they'll like. If you're a mediocre author putting out filler in your genre, your 6th, average, book will get pretty good reviews because everyone who bought it knew what they were getting. If your 7th book happens to be excellent, hype will induce a lot of people who don't like your genre to buy it and see what everyone's talking about -- and they'll take it out on you in their reviews afterwards.
It's difficult to mock properly when all you've given is a question mark. :)
"The next lowest-rated movie on IMDb — 1.8 stars overall ..."
I am not sure what the writer means by a "qualified" movie, but this one does rate less than 1.8: http://www.imdb.com/title/tt2094870/
It has votes from only 195 users as of this writing, though.
I thought it was indicative of a larger trend where crowdsourced data are used to illustrate a point. Like the Google flu trends articles, which have gone around HN at least twice, once when they were successful (https://news.ycombinator.com/item?id=5040204) and once when they were critiqued (e.g., https://news.ycombinator.com/item?id=7455307).
I work a lot with sampled data, and I have found that sampling issues can be some of the most difficult to appreciate and to quantify -- even for experts.
I guess it comes down to sampling from one distribution, P(x), when the situation you really care about samples according to a different distribution P'(x). If P is far from P', your conclusions from P can be arbitrarily bad. If you have an adversary moving P around deliberately, as here, it's even worse.
If there is an interesting statistical result it's that the movie's rating is entirely consistent with crowd sourced predictions. The theory is that 'wisdom of crowds' results directly from diversity among those making predictions.[1] In the case of the lowest rated movie, those making predictions were unusually homogeneous, and therefore an inaccurate prediction as to the quality is unsurprising.
Again, it's all in the interpretation, e.g. there's statistical evidence that a lot of morons ranked the The Matrix.
[1] Diversity Prediction Theorem: http://vserver1.cscs.lsa.umich.edu/~spage/ONLINECOURSE/predi...
Paradoxically, this video manages to be more irritating than the terrible film it talks about. It's particularly ironic that the reviewer complains about Tommy wiseau saying 'hi' too often, and then peppers the video with metallic pings every few seconds while making nonsensical unfunny asides to viewers.
I haven't seen The Room, but the ne plus ultra of that kind of self-indulgence for me is Garden State. 90 minutes of Zach Braff's daddy issues and romantic wish fulfilment - I've had better afternoons. (It amazes me that it picked up an audience more or less on the basis of having The Shins on the soundtrack.)
The Star Wars prequels fall into that category, even if Lucas isn't in them. Proves very well that he should be shot on sight if ever he approaches a typewriter again. Not a fanboy of the first films by any measure, but at least Lawrence Kasdan or whoever can string a couple of lines together. Left to his own devices, Lucas is capable only of bloated, soulless tech demos.
But if a group with an agenda comes in and rates the movie on a different set of criteria without letting the typical reader know renders the data useless.
Anyone can intentionally make a bad movie that's worse than a stinker like Battlefield Earth. That's nothing worth recognizing because the difference between the actual quality and the intended quality, the ΔQ, is so tiny.
"The worst movie ever" designation should go to the film that has the highest ΔQ.
really, profitability is the best "general goodness" measure ever created.
Snark serves no one well.
>> It seems messier than politics in a democracy of 300MM people or 70MM...
> Snark serves no one well.
FWIW I didn't see it as snark. You seemed to imply the size of the democracy mattered. His/her response was a couple counter-factuals. Then you seemed to acknowledge it's not the size, it's the age of the democracy (which is a great point).
In any case I found the rest of your original post really helpful context, explaining why a "life" sentence probably won't turn out to be that. Thanks!
People who are motivated to see a movie early are the same people who are likely to enjoy that movie. This makes it natural for movies to get rated very highly when they are first released. That's just natural.
If you forced people to photograph their ticket stub, I believe this could make the problem worse: mainly people who particularly loved the movie (or thought it was significantly worthy of comment) would be motivated enough to do that. You'll just collect the IMdB's 10s and 1s.
I know many people use the IMdB for their own record-keeping. These people would stop doing that if there was an administrative barrier that made it less convenient than a paper equivalent, e.g. writing in an address book. I strongly believe this makes their votes neutral/genuine compared to other survey methods.
Even a basic trimmed statistic would find a much more accurate average. Lazy people leaving 10s and 1s would be removed or given a lower weighting.
But you raise an interesting point, in that there should also be a penalty for novel, provocative work that, while perhaps well-executed, rests on principles that can never be used again, so they don't advance the state of the art or otherwise leave us with any enduring influence. Works like Blue and John Cage's 4'33 would fall into that category, I think.
I've seen the first ten minutes of Blue, it's actually really good. After ten minutes you start seeing stuff, but you can't work out if it is your own imagination or the film itself. The concept of Blue could be advanced: I was thinking of "Gamma" - viewers are exposed to intense gamma radiation for 90 minutes.
Those buzzkills at the FDA have already banned any artistically worthwhile amount of X-rays from CRT televisions though[1] :(
[1] http://www.gpo.gov/fdsys/pkg/CFR-2013-title21-vol8/xml/CFR-2...
To me, works like these are symptoms of art forms that are well on their way to exhausting their own possibilities. What next? Green? 4'34?
It's a joke, apologies I couldn't resist. Some art forms and certain works of art are genuinely dangerous [0].
Regarding cosmic rays, I thought it was caused my massive particles, and being able to produce them artificially would be a considerable scientific feat - its artistic relevance would pale in comparison.
> Those buzzkills at the FDA have already banned any artistically worthwhile amount of X-rays from CRT televisions though[1] :(
An old chemistry teacher of mine used to use an old TV screen as a cover for potassium+water reactions, it was about an inch in thickness.
As an artist, you could be more aware of the room noise as part of the performance of your work instead of getting in the way of it. As a patron, you could consciously choose a place that has noise to enhance your experience of a work.
http://www.ted.com/conversations/14056/why_is_absolute_silen...
Also, You can't just say 'The Market' as if it was all one bucket. Or some 'God' thing that punishes.