There’s something rotten in the state of online video streaming

There’s something rotten in the state of online video streaming(gigaom.com)

101 points by kfitchard 12 years ago | 61 comments

j_m_b 12 years ago |

As a comcast customer, I've noticed that if I am having a problem with streaming video that if I go to speedtest.net, the video streams fine after the initial ping test from speedtest.net. It will be fine for a few moments (seconds to minutes), but will slow down or stutter again eventually. A fresh visit to speedtest.net again corrects the problem. I have a hunch that a visit to speedtest.net will disable traffic shaping for a set period of time in order to convince people that their connection is fine.

It is interesting that in the article about Verizon's shaping of traffic to AWS that the tech support had the author go to speedtest.net and test his traffic in order to tell him "see no problems!". I've had similar responses from AT&T when I was a DSL subscriber.

I understand that traffic shaping should only affect particular sites and that,in theory,there would be no need for a temporary disabling of traffic shaping for a speedtest. My theory is that due to the high volume of packets needed in order to have a good result on speedtest.net, the analyses of packets at the router for traffic shaping is enough of a slowdown to affect the results.

Anyone else ever notice this? Any alternative explanations as to why going to speedtest.net would suddenly make paused streaming video run again? I notice this mainly when watching live streaming youtube events or sometimes when I am watching netflix during peak hours.

I would love to hear alternative hypothesis as to what is going so that they could be tested!

coffeedrinker 12 years ago | |

Netflix is unwatchable during prime time where I live.

However, I have also noted that on two separate occasions when I began examining the Frontier FIOS site, I received a personal knock on my door from my local Comcast representative. Wondering if there was a connection to my search for another provider and the person at my door, I asked her if she was talking to everyone in the neighborhood or just me. It was just me--they sent a person out to personally ask me about my Comcast service.

All they have to do is fix the evening streaming.

codezero 12 years ago | | |

And maybe stop monitoring your personal searches, right?

nitrogen 12 years ago | |

In my area, Comcast runs its own Speedtest.net server to hide the fact that the closest peering point that leads back to the same state is a few thousand miles away and is insanely oversubscribed.

If Speedtest had more integrity, ISP-owned servers would be deprioritized.

FireBeyond 12 years ago | | |

Not just that, Ookla /actively partners/ with the ISPs. Comcast's site is actually at speedtest.comcast.net.

There's a financial dis-incentive for them to do so.

corysama 12 years ago | |

I can't find it now, but I have read an article about a guy who wrote a script to ping speedtest once/minute. It made a clear, measurable difference in his download rate from other sites as long as he kept the script running.

j_m_b 12 years ago | | |

Yep! Another commenter mentioned writing a script to do this. I would very much appreciate it if you could try to find the search terms to locate that article and post the article as a reply. I would appreciate it very much!

jstalin 12 years ago | |

Yes, this happens to me as well. Netflix downloads at around 500kbps. Try using a VPN service. When I do, the throughput jumps to 8-10mbps.

vvvVVVvvv 12 years ago | | |

How about a script that'll randomly request Speedtest page while you stream ?

Hoff 12 years ago | |

Try some alternative performance-testing sites, and see if there's a difference with those; if there might be different QoS rules in play for specific web sites.

The speedof.me site is one I've occasionally used, and that also avoids the need for Adobe Flash.

kbenson 12 years ago | |

I would love to see some further investigation into this. My initial thought is that if it's shown likely, someone needs to make a little utility that just loads speedtest.net every minute or so in the background.

AJ007 12 years ago |

No one has pointed out that this is bad for the providers throttling traffic.

If Time Warner, Comcast, and AT&T want their names to be associated with budget brands then its ok. Customers accept poor quality when the price is really low and there is no lock in (pre-paid mobile phone service.) Good luck getting those people to purchase premium services from you years in to the future.

On Hacker News we complain a lot about things most people don't care about like privacy and security. Most big companies can disregard those issues with minimal visible impact to their business long term. Video streaming quality is different because its what basically all of your customers are doing. Anyone with an IQ over 70 knows something is wrong, and you can't excuse it away.

Its possible that the peak bandwidth doesn't exist. In that case these ISPs are overselling their "inventory" much like airlines oversell seats. It could be early signs of infrastructure issues to investors.

blueblob 12 years ago |

Make the cable companies common carriers!

https://petitions.whitehouse.gov/petition/restore-net-neutra...

Buttons840 12 years ago | |

I see that petition has already meet the required signature count. I haven't followed the state of petitions.whitehouse.gov; do we expect them to actually comment on this petition?

blueblob 12 years ago | | |

I am not sure. There seems to be evidence that they do not respond to a few petitions, even when they meet the 100K signatures (at least immediately)[1]. Since [1]'s release, it appears that they have responded to a few of the petitions mentioned, and if you look at the site itself[2] (assuming "popular" is actually sorting by the number of signatures) there are only 15 unanswered petitions that meet the 100K requirement (though they changed the limit a few times and some may have met the original requirement). I suspect that they only respond to petitions that are in the mutual interest of the public and the politicians themselves but I have no evidence to support it.

[1] http://www.nextgov.com/emerging-tech/2014/01/white-house-owe... [2] https://petitions.whitehouse.gov/petitions/popular/0/2/0

shigginbotham 12 years ago | |

This actually isn't a net neutrality issue because back in 2009 when the FCC was writing the open internet order it decided not to mess with peering and interconnection disputes. So designating ISPs as common carriers could be a first step here, but it wouldn't be the only step.

blueblob 12 years ago | | |

Could you explain in a little more detail? I was under the impression that the FCC would fix prices for common carriers, and therefore they would have to at least charge everyone in the same manner.

nas 12 years ago |

I wonder if publishing these metrics would go a long way to solving the problem. I know I'd must prefer to sign up with an ISP that doesn't throttle traffic I care about. I guess the big problem with that idea is that in many areas of North America there is very little competition in the ISP marketplace.

If the government was going to do something, I'd prefer it concentrate on increasing competition. Ideally, every home should have at least two or three options for high speed network access.

toast0 12 years ago | |

> If the government was going to do something, I'd prefer it concentrate on increasing competition.

Indeed, competition would help tremendously. Mandated wholesale access to the last mile delivery would be a great start (and something that used to be mandated for DSL from ILECs). I believe AT&T, Verizon, Comcast, etc can do a good job of connecting my house to their network, and deliver my data to their office; from there, I'd prefer to have someone else take responsibility.

Bonuses if new regulation in the same spirit covered all mediums (or at least more of them), and addressed unfair pricing practices by AT&T from the last time (direct to consumer offerings were priced significantly below the wholesale pricing, which was anti-competitive). Some parameters for dealing with congestion in the shared medium between the end user and the central office(s) might be appropriate too.

ForHackernews 12 years ago | |

Google is making performance metrics for YouTube public to help people when choosing an ISP (in places where there is a choice): http://www.ntca.org/new-edge/video/google-to-rate-isps-based...

JRobertson 12 years ago | | |

Direct link to the Google Reports site: http://www.google.com/get/videoqualityreport/#how_video_gets...

ItendToDisagree 12 years ago |

"But they built the last mile so they should be able to do whatever they want with it..." /sarcasm

You know that comment is coming to this discussion at some point.

Wish there was a good answer to this. Making internet connectivity a 'utility' has its downsides and upsides. It really is becoming a pressing issue though (and not just because of netflix being slowed).

ddorian43 12 years ago | |

weren't they payed with tax money to build the infrastructure?

skywhopper 12 years ago | | |

Varies by municipality, but pretty much everywhere the "last mile" is a local monopoly, granted by the town, county, or city with the rights to do so. And so long as that's the case there's no real competition and thus no way for Internet users to route around these abusive practices of their ISPs.

talmand 12 years ago | | |

Sort of, my understanding is that they mostly received tax breaks with possibly some subsidies.

Either way, they received an incentive to do so.

incision 12 years ago |

Reminds me of the Comcast situation with Bitorrent in 2007 [0].

I have little doubt that ISPs will continue to escalate this game of service degradation and denial without strict regulation.

0: http://consumerist.com/2007/10/27/damning-proof-comcast-cont...

jcampbell1 12 years ago | |

Many people don't realize much of the bit torrent debate ended because bit torrent switched from TCP to µTP. The problem was bit torrent clients caused massive congestion by making hundreds of TCP connections. µTP is extremely congestion/network friendly. It backs off based on latency, so it doesn't fill every buffer on the network.

zokier 12 years ago | | |

The BitTorrent discussion didn't end because of uTP. It ended because ISPs got bigger fish to fry. Fish with actual money, like Google/Youtube and Netflix.

tenpoundhammer 12 years ago |

This is fine let them keep screwing around with Netflix, Amazon, and Google(Youtube) it won't be long before they get pissed and build their own internet backbone/service providers(already happening).

I can't wait to get Amazon High Speed Internet.

So please, let them keep acting jerks.

EEGuy 12 years ago |

I wish ISPs business where completely out of the content business.

Slowing competing content could be justified as a "fiduciary duty", yet it's much simpler than that: Gatekeepers and highwaymen, extracting private taxes on content they did not create.

ogreyonder 12 years ago |

Is there any interest in a Chrome plugin that would passively report anonymized download speed for things like YouTube or Amazon? I can imagine that kind of info would be useful.

pronoiac 12 years ago | |

YouTube used to offer a view of your download speed, compared to other customers of your ISP, other ISPs, and other countries, but it appears to be closed down, sadly.

* youtube.com/my_speed

existencebox 12 years ago | |

I think if you could do some detection of what connectivity is being used, and other sorts of relevant connection data, it would be a valuable motivator to see, "We've detected you are 30% slower than other customers with x and y differences in service"

This is super hand wavy, but I think with proper data it could be a useful tool.

shigginbotham 12 years ago | | |

That's what many of the tests at M-Labs are trying to do (I encourage people to take a look http://www.measurementlab.net/) but packets don't traverse networks in a straightforward path. I think focusing on that data and trying to figure out ways to prove traffic manipulation and pinpoint consistent places/times/applications where it slows would be valuable. It's also worth understanding how the FCC currently measures speed and treats speed tests. It's not at all straightforward http://www.fcc.gov/measuring-broadband-america/2013/develope...

We should have a technically-dominated discussion about tests and the right way to look for problems, that we aren't having today because it's so wonky. We totally need data, but we also need to figure out what data will be most useful to prove bad behavior. For example, if my speed drops to 1 Mbps for 30 minutes during prime time but my speeds the rest of the day are 55 Mbps then that blip is heavily discounted, but shouldn't be given that I'm probably trying to stream video.

stox 12 years ago |

We've all known this was coming for years, actually, it is amazing it has taken this long.

kriswill 12 years ago |

If AT&T, Comcast, Verizon and other ISPs are intentionally limiting bandwidth to certain content providers, as the measurements seem to indicate, they may be actually doing us a favor. By doing this, they are causing people to discuss the issue and eventually lead to market changing trends or new regulations and potential legal hazards for ISPs that do this in the future.

If anything, Amazon is a smart company, there might be an opportunity here for last-mile internet delivery...Amazon Fiber Prime anyone?

hussong 12 years ago |

I wish ISPs could stay out of the content business since their content curation and presentation are usually subpar at best.

amurmann 12 years ago | |

I wish we could amend the constitution that they are not allowed to do anything at all but sell bits to consumers. No selling of content, access to consumers, devices or ice cream or whatever other scheme they come up with. Their sole interest has to be selling more bits! If they even as much as think about doing anything else there need to bd draconian penalties. I would even argue that they are endangering infrastructure and charge them with terrorism.

fredgrott 12 years ago |

The most interesting thing is the astroturffing comments form unnamed Verizon employees in the first 10 comments

steeve 12 years ago |

So now not only's the catalog is meh, the quality/stream too.

Why are they pushing people so much to piracy?

And don't get me started about the situation once you're outside the US.

goggles99 12 years ago |

>While I was seeing my episode of The Good Wife falter at what appeared to be 1.9 Mbps, I was able to measure connection speeds of 28 Mbps to my house using a Speedtest.net test from Ookla. This is exactly the dichotomy that the M-Lab data is showing, and my example is not an isolated one; Comcast users have been complaining for months.

How do we know that it is not just the provider being sapped for bandwidth? I notice that speed tests I run in the mid-day and evening come back pretty much the same, but Netflix takes noticeably longer to buffer in the evening. I always assumed this was because their servers were much busier then.

xur17 12 years ago | |

I'm guessing it's that link to Netflix from your provider that is busier at that time. I use a local provider for internet service, that despite offering crappy customer service, has given me consistent speeds, and no buffering issues with Netflix or Youtube at all.

shmerl 12 years ago |

The first rotten thing there is DRM. There are no DRM-free video services where you can buy anything (Headweb doesn't count because it's regionally restricted). So there simply is nothing to chose from. Failure at step 0. It's not just rotten - it's completely decomposed.