Why Your Netflix Traffic Is Slow and Why the FCC Won’t Necessarily Make It Fast(freedom-to-tinker.com) |
Why Your Netflix Traffic Is Slow and Why the FCC Won’t Necessarily Make It Fast(freedom-to-tinker.com) |
http://www.cogentco.com/en/news/press-releases/631-cogent-of...
Pretty sure Comcast didn't take them up on the offer. Doing so would have fixed the issue and Netflix wouldn't be writing checks to Comcast now.
http://knowmore.washingtonpost.com/2014/04/25/this-hilarious...
EDIT: Also worth noting that Netflix has to pay every month (which is what Comcast wants) rather than Cogent just giving them $10k once for new hardware (not as delicious as lots more money!)
https://freedom-to-tinker.com/blog/feamster/why-your-netflix...
I'm having trouble with this logic. Increased leverage over content providers seems like it would exactly exacerbate this situation.
The right way to solve the internet problem is "unbundling"; that is, the people who own the last mile lease it out to companies that sell you internet, video and other services. If Comcast customers were getting a bad netflix experience they could switch to an ISP that gives a good one.
That's terrible. The ISP should give me "internet access" which includes everything out there. They should not get to pick and chose what I can get.
>> If Comcast customers were getting a bad netflix experience they could switch to an ISP that gives a good one.
This is called competition. It's what you get with net neutrality. Since every ISP has to give you everything equally, the only way they can compete is by building better infrastructure and compete on ping time and bandwidth. This is how it should be. Allowing them to prioritize certain services for a fee (regardless of who pays) means allowing them to stagnate - in fact paying them to stagnate - which is bad for everyone.
You are stating that "Net Neutrality" will magically create competing ISPs in places where only a duopoly of <Cable Company> and <DSL Company> currently exist. Please explain to me how net neutrality will somehow affect things like local municipalities granting ISPs local monopolies. Or how "net neutrality" will allow new-comers to overcome the initial regulatory and capital investment hurdles to setting up a new ISP.
What you're missing is the "unbundling" is talking about forcing (e.g.) Comcast to lease it's "last-mile" lines out to competing ISPs. This means that competing ISPs don't need to invest in "last-mile" infrastructure. (If you don't understand what "last-mile" means, it would behoove you to look it up).
Net neutrality does play a part here in that if Comcast partners with Netflix, they could provide a better Netflix experience than a smaller ISP without bargaining power, but the lack of ISP competition is arguably a bigger problem and calling out "Net Neutrality" as the solution to all Internet connectivity woes doesn't help.
Access to everything equally is not physically possible; if we pretend it is physically possible, it's not logistically possible: no ISP is going to peer with all networks at all peering points; and no regulator is going to compel an ISP to peer with all networks at all peering points.
Unbundling is easy to define, and easy to regulate, and offers a way to get competition where it's most effective. Unfortunately, it seems to be not easy to put in place. The basic idea with unbundling is that last mile delivery is cost intensive, and has high barriers to entry, etc: a market solution isn't going to work: instead require that the last mile providers offer wholesale access to get packets to a centralized point (or points) where other providers can deliver the connectivity to rest of the networks. This actually makes sense: once you're at a carrier hotel, you have a large number of options to interconnect; getting from a central last-mile provider building to a carrier hotel is managable, but getting from each home to a specific building is capital intensive.
I don't know enough about unbundling to say if I support it or not but I believe it has been pretty popular in countries that have adopted it.
That is not what last mile means.
And customer service.
Unfortunately there are other problems (like "your DSL being down is not our fault") with unbundling.
The only way that would happen nationwide is via federal law. What chances do you think the Tea Party congress would have of passing this?
The FCC policy is pretty much our only shot here and makes a lot of sense. The industry will pivot itself around it now that they can't fight it. They are now forced to play nicely with each other. Yes, there is going to be an ugly interim period, but I think we're going to remember Wheeler as the man who saved the internet as we know it.
Bandwidth may very well be a limited resource in some cases but the dispute between Comcast and Netflix is not one of them. The Comcast network has the bandwidth available to serve the customers (since not all sites are congested) and Netflix is willing to peer for free wherever they want them to, so the slowdown is completely artificial and Comcast generated.
This problem seems much worse in the ISP world. Many years ago, ISPs could easily oversell their connection. Realistically only worrying about the small percentage of extreme users downloading high volumes of data via Usenet, BitTorrent, or other file sharing services. Life for an ISP was probably pretty great.
Now, with the sheer number of households streaming video, it seems to be apparent that ISPs are drastically overselling their connections. It does not even make logical sense to blame a single service if an ISPs upstream connection is saturated. If the ISP did not oversubscribe their connections at such ridiculous rates, this would not be an issue. Why does it matter that a single service is causing the largest portion of the traffic congestion?
Netflix recommends a 5Mbps connection to stream HD video. Most all devices today would stream at HD quality, if possible. How is it that an ISP such as Comcast doesn’t have the uplink capacity to sustain many users at a mere 5Mbps? I find it absurd that multiple ISPs are overselling their connection to the point where a speed of 1/7 of the average connection in the USA is too much of a strain. It really seems that the level of oversubscription is equivalent to having a data closet with a 48-port 100Mbps switch uplinked at 2Gbps . The whole situation just stinks of pure greed.
This does not follow. A larger ISP with more market share, regardless of the state of the competition, can extract larger peering agreements. So this merge would absolutely make the current situation for Netflix worse and other peering agreements more expensive.
The next step is stopping the TWC/Comcast merger.
Comcast claims that Netflix was sending traffic at such
high volumes as to intentionally congest the links
between different transit ISPs and Comcast, essentially
taking a page from Norton’s “peering playbook” and
forcing Comcast and its peers (i.e., the transit
providers, Cogent, Level 3, Tata, and others) to upgrade
capacity one-by-one, before sending traffic down a
different path, congesting that, and forcing an upgrade.
Both sides of this argument are reasonable and plausible?
Are you kidding me? If Netflix was intentionally shaping their traffic to disrupt Comcast, how is that not a denial of service attack? That sounds like a criminal accusation more than a bargaining position.
If Netflix actually was doing that, shouldn't comcast be sueing rather than asking for money for a peering arrangement?
also, and this is more to my own ignorance than it is to a counterpoint. I thought peering arrangements were things that happened between ISPs. Netflix is a customer of an ISP.
http://blog.level3.com/open-internet/verizons-accidental-mea...
http://publicpolicy.verizon.com/blog/entry/level-3s-selectiv...
Basically there are 4 different situations:
1) ContentProvider<->Level3<->Cogent<->ISP<->User
2) ContentProvider<->Cogent<->Level3<->ISP<->User
3) ContentProvider<->Level3->ISP<->User
4) ContentProvider<->Cogent->ISP<->User
In all cases the first network gets a payment by the content provider and the last network a payment by the user. In cases 1 and 2 the middle network is not getting a payment because neither side is their direct client. So the Cogent/Level3 dispute is about there being the same number of 1 and 2 cases so that it all evens out and no compensating payment is necessary. Cases 3 and 4 require no additional payments no matter the upload/download ratio because both network links have been paid for, one by the user the other by the content provider.
This whole thing is about the ISPs noticing that the content providers are making money offering content over the internet (shocking I know) and trying to reach across the network and grab some of that revenue, even though everyone has already been compensated for the actual transfer of bits.
http://blog.level3.com/open-internet/observations-internet-m...
[1]http://gizmodo.com/this-box-can-hold-an-entire-netflix-15925...
This is really "Game of Thrones" vs. "The Internet" and GoT won.
First I pay for internet access. 50Mbps "theoretical max" but it's fairly reasonable to expect to get say 10% of that most of the time.
Then Netflix has to pay again, in order for me to get the internet traffic that I requested. If Netflix doesn't pay, then Comcast schedules upgrades more slowly than they normally would such that my Netflix experience is degraded. They're not reducing capacity, they're just growing capacity in some areas much, much more slowly than in the rest. To get extra cash.
"Comcast claims that Netflix was sending traffic at such high volumes as to intentionally congest the links between different transit ISPs and Comcast, essentially taking a page from Norton’s “peering playbook” and forcing Comcast and its peers (i.e., the transit providers, Cogent, Level 3, Tata, and others) to upgrade capacity one-by-one, before sending traffic down a different path, congesting that, and forcing an upgrade. Their position was that Netflix was sending more traffic through these transit providers than the transit providers could handle, and thus that Netflix or their transit providers should pay to connect to Comcast directly. Comcast also implies that certain transit providers such as Cogent are likely the source of congested paths, a claim that has been explored but not yet conclusively proved one way or the other, owing to the difficulty of locating these points of congestion (more on that in a future post)."
Now, peering negotiations are an ugly, unpleasant activity that I'm quite happy never to have been a part of. But I don't find this particular claim to be "reasonable and plausible", in the author's words. Where is the congestion?
* Inside Netflix? Nope.
* Between Netflix and their transit providers? Seems unlikely; Netflix is paying them, after all. Such congestion would effect many of Netflix's customers and be against Netflix's interests.
* In the transit providers? Possibly, but again, Netflix is paying them. Further, Netflix connecting directly to Comcast is the first thing Netflix would do, if the transit providers were trying to squeeze more money out of them.
* Between the transit providers and Comcast? That seems to me to be the most likely interpretation of that paragraph. But if Netflix is doing this intentionally; say, by directing 90% of their traffic to Comcast customers through Tata, those customers are most likely to be unhappy with Netflix, not Comcast---any traffic from a customer out that doesn't go through Tata is fine, so they wouldn't likely perceive any problems with anything but Netflix. Further, that seems to contradict the statement that "[customers] routinely experienced high latency to many Internet destinations every evening during 'prime time'". And if the congestion is not hitting one specific transit provider, then the idea that it is intentional on Netflix's part is silly and Comcast's peering relationships are simply inadequate for the traffic its customers are using.
* Inside Comcast? Well, I could see almost anyone wanting to have an unrelated third-party pay for their infrastructure upgrades, but....
Then, there's this: "The best technical solution (and what ultimately happened) is that Netflix and Comcast should interconnect directly." The best technical solution? The best technical solution? (Here's a hint: a fully-connected network is not a good technical solution.)
But the ultimate problem here is: "This is where market leverage comes into play: Because most consumers do not have choice in broadband Internet providers, Comcast arguably (and, empirically speaking, as well) has more market leverage: They can afford to ask Netflix to pay for that direct link—a common Internet business relationship called paid peering—because they have more market power."
Comcast has more market power because it is a monopoly and the problem with monopolies is precisely that they use their market power to squeeze money out of counter-parties.
As an aside, I find Feamster's comment reply, "What is false in Oliver’s presentation (and many of the claims in the popular press) is his claim about what causes* that slowdown. The cause of the slowdown is congestion, not intentional throttling,*" to be simply disingenuous. What exactly is the difference between intentional throttling and intentionally introducing congestion by not upgrading inadequate peering relationships?
You pay for internet. Netflix has to pay for internet too. To deliver packets to Comcast's network, Netflix can chose to pay any number of transit companies, or they can cut out the middle man and pay Comcast. Why is it ok for Netflix to pay a transit company, but not Comcast?
As some ISPs got bigger, they were able to negotiate peering agreements instead of paid service since it was in both party's best interest. A could bill B $10k/mo for service and B could bill A $10k/mo for service and in the end it was a wash. This makes sense too. Once this happened your bill largely reflected the cost of the last mile, since the transit was now free. Bandwidth has gotten cheaper.
What we're seeing now is that some ISPs have gotten so big that instead of paying transit providers for access to the greater internet, or simply getting free peering, they're charging others for peering to them. How are they able to do this? They have an effective monopoly on their customers and they're "leveraging" this to get even more money. Now my monthly fee goes to pay for the last mile, and someone else also has to pay for the last mile. Am I getting a discount? No. Bandwidth isn't getting cheaper, but the last mile is getting paid for twice. Is the last mile getting faster? Also no.
So this feels very much like a monopoly abusing their position of power to get extra money.
Because transit providers are actually providing a service to Netflix. They allow them to dump all the traffic in one location and then the transit provider will build out a network to take care of moving the traffic to anywhere in the globe where there's an ISP requesting it. The transit provider will then not pay the ISP to take the traffic and deliver it to their requesting user. When Netflix connects directly to Comcast that's not what's happening, Netflix has to get the content to Comcast's network by itself and the Comcast network is already paid for by the user. So if Netflix pays Comcast then Comcast is getting paid to create the network by both the user and Netflix.
He discusses that in a later comment. The point I think he's making is that it wouldn't qualify as "paid prioritization", and shouldn't be characterised as such.
It was paid; it succeeded in getting Netflix to write them a check. And it may not be strictly "prioritization", but to me the intent means more than the letter.
It doesn't. Local monopoly support from the government is a big problem, but it's also not the only problem. I actually have 3 options at my home and it doesn't seem to be causing an increase in service or a reduction in prices.
That said, as video moves to the internet and as the ISPs lose differentiation, we can hope for actual competition on access to what FCC regulation will make a commodity service. But I'm not entirely convinced that will happen either.
Cable built up the outdoor wiring plant by offering a compelling new service (great reception for local TV and exclusive national programming) without significant competition (you could get a giant satellite dish, but compact dishes were not available until after cable was mostly entrenched).
It's not easy to overlay additional infrastructure wiring to compete with existing services (most areas already have good enough internet and video services from one or two providers; especially when a new entrant prompts upgrades from the incumbents). On the other hand, it's very easy to use wholesale access (if available) to provide middle mile competition. You would just need a couple routers (one at last mile provider building; one in a local carrier hotel/colo), a leased line between them, and a transit account.
The "not pay" part is, I assume, based on the prevalent use of settlement-free peering in the industry. These agreements were based on a understanding that, over time, the traffic would wash out even between the two, so let's just not keep track and call it even from the start. This is not necessarily a given in all situations. Comcast asserted that the quantity of Netflix traffic was breaking this understanding.
I don't think that's the case. Transit providers will always tend to deliver more traffic to the ISP than the ISP delivers to them, because users download more than they upload. That's still fine because the transit provider's network is paid for by the content provider and the ISPs network is paid for by the user so the settlement free peering always makes sense. Without it the ISP wouldn't have content so no users and the transit provider wouldn't be able to deliver traffic so would have no clients. The fact that Netflix is now a significant single content provider in the transit network doesn't change this dynamic at all, it just makes it easier for ISPs to squeeze them for money. Compensation for unbalanced peering only makes sense between transit providers since for each connection only one of them is being compensated (the one whose client sent the traffic) so the other one will only accept that peering if it is balanced.
I've explained this more clearly elsewhere in the thread:
Then we need new--and more stringent--regulation.
Unbundling has a clear model that works (or is at least workable), look at the US between 1996 and 2004, or most of Europe now.
I had to buy the DOCSIS 3 modem for my house, so they're not paying for that. I suspect that they had to buy new modems on their end, but my modem was only about $100 so I can't imagine that theirs was more than $100 especially if they're buying millions of them.
They haven't upgraded the coax that runs to my house, or the pulls that run along the telephone poles. Those are all the same. So it's an equipment upgrade? $100 once so that they can charge $20/mo more forever? They made their money back on the last mile well inside of a year.
And they haven't had to put more fiber into the ground either. Fiber transceiver keep getting faster and cheaper every year. As slower ones break you can replace them with faster ones for less money. Even if you upgrade early, it's not as though it's hugely expensive.
Here's a 10gig, 10km transceiver for $400: http://www.newegg.com/Product/Product.aspx?Item=N82E16816523...
And here's a 10gig, 40km transceiver for $4k: http://www.newegg.com/Product/Product.aspx?Item=N82E16816401...
I pay for 50/5 but I suspect that Comcast only allocates 5/0.5 or so for me, since it can't manage to keep up with Netflix all that well. So that means that Comcast can do the last mile for $100 or less of new investment and the last 25 miles for $4 (2000 people per pair, $8k per pair) of incremental investment. This is not a lot of money relative to the $60/mo that I pay.
> Is it valid to look at this as paying for that, and if CDNs can't pay, then end user prices will have to go up?
So first I disagree that the CDN should have to pay anything, again since I'm paying Comcast to connect me to the CDN!!
But even if I admit your point (which again, I emphatically DO NOT!) the amount of money to make an upgrade happen is TRIVIAL relative to yearly revenues. $104 in one-time hardware versus $720 in yearly, recurring revenue.
Speedtest gives me around 50/10.
If extra money has to come from somewhere, I think it's better for the Internet ecosystem for all those costs to be passed on to the customer as price increases, but obviously a lot of customers are more concerned about their own bill than the health of innovation in the Internet.
Edit: I'm just explaining why a hardware upgrade was besides the points that were being made. These are the facts regardless of who you think is in the right. The argument was over bandwidth usage, not hardware upgrades. Is anyone disagreeing with that?
All Internet traffic has more download than upload (for the end-consumer). This is literally why ISPs offer faster download speeds in end-consumer packages than upload speeds - they know that consumers will need to download more than they upload.
In vector calculus, this is known as divergence. Traffic on the highway has divergence of 0, meaning that if you draw a closed loop of any shape, over the course of a day/week, the number of cars crossing that loop in both directions (in/out) will be the same.[0][1] Internet traffic does not have zero divergence, because bits are interchangeable and easy to create, destroy, and copy (whereas cars and humans are not).
Comcast isn't dumb. They know this is the way the Internet works. They just also know that it's more profitable to pretend they don't and extort money based on a false definition of "equality".
[0] https://www.physicsforums.com/threads/just-what-does-it-mean...
[1] It's not exactly the same, due to births, deaths, and migration, but you get the idea.
Yes, but they also have commercial customers that send out lots of data. Netflix is an outlier all by itself, outweighing the connection.
Trying to boil this down to "Netflix is sending a lot more than it's receiving" is to completely ignore WHY this is happening and really makes the debate useless. Symmetric (or nearly symmetric) peering agreements made sense when everyone had symmetric connections like modems, ISDN, T1, T3, etc.
When the majority of connections are not symmetric, it makes no sense at all.
Again, remember that Netflix isn't DOSing Comcast. Any packets that Netflix sends to Comcast's network was at the request of one of Comcast's (and Netflix's!) customers.
In that case many of those would be Comcast customers, and peering would end up equal.
Netflix has a problem because it's an outlier. I'd like to see the chart without Netflix and see if comcast still receives more than it sends then.
Netflix is one of the first ContentProviders to be pumping so much traffic that they will imbalance the peering traffic ratios of any almost any Tier 1 they buy transit/CDN from, except for probably Google, which ended up doing what Netflix is doing now: paid peering.
That's only relevant if the transit/CDN is actually peering with another Tier 1 provider to deliver the Netflix traffic (cases 1/2). What's been in the news has been cases 3/4 where there's a single transit provider in the picture and there's no reason for the ISP not to accept the content with no compensation for the asymmetry as it has a client paying him for that service.
> except for probably Google, which ended up doing what Netflix is doing now: paid peering.
Do you mean Google is also paying ISPs for access to users like Netflix has had to? Because if you mean Google is having to do paid peering with transit providers because their traffic is asymmetric that would just be normal payment for transit, as if Level3/Cogent/etc where providing transit to them.
The issue is really that the idea of transit providers in general are becoming obsolete. So much fiber was laid, with DWDM and further subdivision abilities, it costs comparatively nothing to get a nationwide network these days. That was Cogent's entire business model, buy cheap fiber, sell transit for almost nothing to content providers, practically (in some cases, actually) give away transit to ISPs/businesses to keep their traffic ratios as balanced as possible.
Now, we're in a situation where it's not technically or financially hard to have a presence in every regional peering point. The major content providers don't need transit providers (or in the case of Level3's CDN, are already provider) except out of convenience and their access to settlement free peering. The major ISPs have no need for transit providers(or are already providers), as all the content providers wants to peer with them, and most of them have nationwide networks to do so. (see comcast: http://business.comcast.com/images/default-source/about-us/t... )
Unregulated peering has lead to the cost of transit plummeting to almost nothing. ( http://drpeering.net/white-papers/Internet-Transit-Pricing-H... ) We've just hit an interesting point where 2 contentproviders (Netflix and Youtube) are half of all internet traffic, and the eyeball networks are all basically tier 1 transit providers. There's basically unlimited choice for transit providers for ContentProviders, but each eyeball only has 1-3 ISPs to choose from.
My thoughts are rather than attempting to regulate transit/peering, regulate that last mile network and let people choose their own transit provider. Back in the DSL days, I used a local ISP that provided cheaper/better service than our telco. If I could pay a flat $x/mo for 1gb connection to a transit provider of my choice, I'm sure Google or Level3 would offer cheap uncongested access.
Are you sure?
http://en.m.wikipedia.org/wiki/Tier_1_network#List_of_tier_1...
Is there a relevant difference between that and what Netflix wanted?
I'm not saying that an average consumer is sending as much as they receive, but Comcast doesn't only have consumers, they also serve websites. That should balance out for small websites in and out of their network.
They are. They're buying 50/5 or 50/10 internet and then asking their ISP to (horror of horrors!) make good on their claim. If the ISP isn't DRAMATICALLY overselling their capacity, things are totally fine. If they can actually deliver on 10% of what they're "promising" (but have cleverly worded the contract to avoid having to actually make good on those claims, a shady practice also), there are no problems with Netflix whatsoever.
Level3 offered to pay for the direct cost of the upgrades, namely a few grand for the fiber transceivers on Comcast's side of the peering point in the carrier hotel. That would mean it's a "free" upgrade, and no end-users are paying for it. But Comcast didn't bite.
That makes your objection COMPLETELY nonsensical.
The question is whether all consumers' prices should go up, or Netflix prices should go up. In the first case, consumers are subsidising netflix for those that use it.
>Level3 offered to pay for the direct cost of the upgrades, namely a few grand for the fiber transceivers on Comcast's side of the peering point in the carrier hotel. That would mean it's a "free" upgrade, and no end-users are paying for it. But Comcast didn't bite.
This is meaningless. As I said elsewhere, they are not complaining about the cost of the upgrade. They're complaining about the cost of taking the extra data. That cost is what would either be passed on to users, or to Netflix. Talking about the direct cost is missing the point.
ISP can't actually deliver said data, and so, can't deliver on their promise.
I can't see how this is Netflix's fault. ISP makes a promise, ISP can't deliver.
Yes it does happen to be coming from Netflix, but Netflix didn't FORCE the ISP's customers to become Netflix customers, did they? Or does Netflix not only have some magical ability to FORCE data upon people, but also mind control to MAKE certain people sign up?
You're arguing that the ISP should be allowed to overstate their capacity as much as they want, and if it causes any problems anywhere, that the counterparty is at fault no matter what. That's completely nonsensical.
> The question is whether all consumers' prices should go up, or Netflix prices should go up.
> They're complaining about the cost of taking the extra data.
Basically they're complaining that they have to make good on the promises that they've made to customers. They were hoping that they could way, way, way oversell their capacity (and charge more for a theoretical peak capacity that you can never use) and get more money from their customers as a result.
Now what's happening is that many of their customers are asking them to make good on 10% of their promises all at the same time. This causes problems for the ISP, so they flail about trying to blame other entities than themselves.
Nobody is asking the ISP to give them a HIGHER data rate than they've already paid for.
They're asking the ISP to give them a mere 10% of what they've already paid for.
Who?
From what I can tell even their commercial offerings are heavily asymmetric to downloading: http://business.comcast.com/internet/business-internet/plans...
Download / Upload (in Mbps)
16 / 3
50 / 10
75 / 15
100 / 20
150 / 20
So who are these companies that use Comcast to send a lot?
And as I mentioned elsewhere in this thread, I'd like to see a chart of data in and out excluding Netflix and large companies. If the problem only affects them or other large companies (like Youtube, Apple, Microsoft, etc, which all pay for direct connections), but not small websites, then I think that would support my argument.
Storage / Transfer / Price
10MB / 100MB / Free
300GB / Unlimited / $20
400GB / Unlimited / $40
500GB / Unlimited / $80
You must realize that these are absolutely NOT true commercial offerings. They don't even list what your commit bandwidth is!
Also you can't run your own applications there! Do you really think Netflix runs off of a shared server like this? They're a HUGE AWS customer running many thousands of VMs. How do you access control things, how do you deploy your code, how do you do basically ANYTHING with this offering from Comcast? I suspect that you don't.
Here are some legitimate commercial offerings: http://www.hostgator.com/dedicated
If you notice, they have a page dedicated to their network. What familiar names do we find on this page? Cogent and Level3! http://www.hostgator.com/network
Notice that there is no mention ANYWHERE of ANY COMPANY that might even a LITTLE BIT be considered a consumer ISP. Comcast, AT&T, Verizon, Cox, etc, none of them are mentioned.
NOT HARDLY!
They would mostly be customers of small datacenters in various places all around the country, and thus, customers of Cogent and Level3.
Datacenters generally go for tier1 bandwidth, not consumer ISP bandwidth.
Look at the plans that they offer on their website. The fastest upload I can find is 20Mbps for $200/mo. http://business.comcast.com/internet/business-internet/plans...
That's $10/meg which is A LOT.
Three years ago if you were willing to buy in bulk, you could get bandwidth for as low as $0.65/meg as this HN user points out. https://news.ycombinator.com/item?id=3892854 Based on his comments I suspect that you could get it for less than $0.50/meg today, maybe less than $0.25/meg.
Sure looks like Comcast wanting to have its cake and eat it too.
Why should Netflix be able to do that and not smaller sites?
Comcast sells asymmetric connectivity that's skewed as far as 10:1 download vs upload. And then they claim that peering should be symmetric as if they aren't a huge contributor to the asymmetry.
Yes if they are willing to interconnect with that ISP at every exchange point. In practice that's expensive and what transit providers are for. Here's my proposal for a simple set of rules that would solve all these disputes:
https://news.ycombinator.com/item?id=7644339
>Why can't I host my website for free, just paying an upfront cost for a server, and get it hosted in an ISP's facility?
You should be able to do something similar to that to provide service for only that ISP's users. In fact Netflix does exactly that with some ISPs:
https://openconnect.itp.netflix.com/
This doesn't allow you to host your website for free because it only adds connectivity to the users of that specific ISP, not the whole internet. For that you have to buy transit.
How many times would they need to interconnect with a particular ISP? Once?
It's right there in the link. You'd have to interconnect with Verizon/Comcast at every one of their regional interchange points. It would become very expensive very fast for a single small website, but it would be reasonable for a transit provider or a very large content provider that already has servers everywhere (like Google). There would be no free riding, everyone would need to pay for their own way, either by directly physically connecting to the interchange points or by paying a transit provider that already does.
From the page:
>We utilize a wide array of network carriers including Level 3, nLayer, Comcast, Cogent Communications, and Hurricane Electric.
Take that back?
I think it is telling, however, that Comcast's logo isn't anywhere on the bottom of the page with all their partner logos. And I think it is also telling that it's only Comcast and not all the consumer ISPs.
Whether ISPs should need to build out enough network to satisfy peak usage completely is a separate issue, but irrelevant here.
No, that's exactly the issue. If the ISP doesn't massively overstate their capacity, there is no congestion. If there is no congestion, there are no complaints. If there are no complaints, there is no problem!
Remember, customers aren't asking the ISP to allow them to utilize their connection 100%. They're asking for 10%. Asking an ISP to not overstate their capacity by more than 10x is a pretty fucking reasonable thing to do, Net Neutrality or not.
And my question was serious. If you think that what Netflix was asking for should be given as a matter of principle, then it shouldn't matter how large they are, and absolutely every website should never have to pay for bandwidth again. What exactly am I missing?
When a Comcast customer uses their Comcast internet connection to use Netflix, or Google, or HN, or any other site, it is that customer who is using Comcast's bandwidth. It is not Netflix, Google, or HN using it.
And Netflix has its own CDN, which wants to connect directly to last-mile ISPs. In that setup, they aren't paying anyone other than the end ISP.
>When a Comcast customer uses their Comcast internet connection to use Netflix, or Google, or HN, or any other site, it is that customer who is using Comcast's bandwidth. It is not Netflix, Google, or HN using it.
This is pointless semantics. If I send you something, we can either call it me using the bandwidth, or you using it. Doesn't change my point here.
Not really. If I mail something to you, I pay the postage. That's how mail works.
When I connect to the internet, if I request something from you, we both pay our respective ISPs to make it happen. That's how the internet works. And the limit to connection speed has historically always been the last mile on one side or the other.
So what Comcast did is pushed the congestion from the last mile to a peering point by purposefully not upgrading the peering point. And predictably, what happened is that the last mile wasn't the limiting factor anymore. According to 20-30 years of internet history, that's breaking the internet. Maybe not Comcast's contract with their customers (since they can unilaterally change that at any time) but definitely in terms of expectations.
That would be like gas stations suddenly advertising the pre-tax price on their signs only once you're done pumping $30 worth of gas they charge your card for $40 because of the taxes. Might not be explicitly illegal since so many other products are advertised on the pre-tax price. But it would be breaking with decades of convention. People might get really pissed as a result.
Sadly few of us have options to get internet aside from Comcast, where as in the gas station scenario we could easily stop patronizing BP or Texaco or Shell or whoever. That also gets people riled up. And rightly so!
The data was paid for by Comcast's customers. I pay Comcast for the service of transport of packets from my network to the internet, and from the internet to my network. That's the whole point of an ISP.
And why shouldn't it be free? I, the Comcast customer, requested this data. Why should Netflix have to pay for the last mile when that's what I pay for already?
Why does Comcast get to charge for both ends of the pipe when they only own one side?
https://help.netflix.com/en/node/306
So you can see that they recommend 3Mbps for SD and 5Mbps for HD.
The problem is that the ISP has promised 50Mbps per customer to said customer, but then they are not allocating enough resources at various points throughout their network to actually deliver on 5Mbps per customer to Cogent or Level3. Ostensibly Cogent and Level3 can handle the upgrade and increased bandwidth, and ostensibly the ISPs internal network can handle the increased bandwidth, the congestion was at a number of peering points only. Those were the bottlenecks.
This doesn't JUST affect Netflix customers, but in fact ANY traffic that runs from Cogent or Level3 to the ISPs customer or vice versa. It just so happens that A LOT of that traffic is Netflix, but it's not EXCLUSIVELY Netflix. There are a great many people that buy transit from Cogent and Level3, meaning that ISPs are harming more than simply Netflix customers but ALL their customers.
Please understand that if the ISP isn't making an effort to ensure that their peering arrangements are sufficient to satisfy the demands of their customers, that is the ISPs fault, and not the counterparty's fault. Especially when the counterparty is willing to make the necessary upgrades to relieve the congestion, and the ISP is not because the ISP wants to get paid for it.
Remember that the ISP is the one that's offered highly asymmetric internet access to their customers. They should not be surprised that their customers utilize said service in an asymmetric fashion. Failing to upgrade the peering connection isn't "standard operating procedure" for a consumer-facing ISP. It's that ISP doing something highly unusual because they think they can get away with it.
This seems to be the main point of departure for us. If you could show that large consumer-facing ISPs used to give free peering even when the other side was sending much more data than they were, then I'd agree with you. The ISPs claim it is standard for someone sending more data to pay. Can you show otherwise?
For a commercial, content producers, the model is "sender pays" which makes sense. That's why hosting companies buy transit from Level3, Cogent, etc. They're paying to send.
What's happened is that Netflix has already paid to send, and ISP customers have already paid to receive, and ISPs have said "yeah that's fine, but if you REALLY want to send, you're going to have to pay AGAIN"
That's the sticking point.
Nobody's arguing that anyone should get a free ability to send, or a free ability to receive. All traffic is generally double billed, once for the sender's connection and once for the receiver's connection.
The "innovation" here is that some ISPs have decided that SOME traffic should be TRIPLE billed. Once at the sender, once at the receiver, and a third time when it crosses from the sender's network to the receiver's network.
Why, if triple billing is OK, should only the consumer ISPs be allowed to do it? Why doesn't Cogent or Level3 charge consumer ISPs extra for receiving more than they're sending? Under the consumer ISPs model of "receiver pays" it would be entirely reasonable for Cogent or Level3 to charge extra, wouldn't it?
You're basically arguing that the way consumer ISPs handle this is acceptable. OK. Why isn't the exact opposite also acceptable? It's not as though most consumer ISPs have a global transit network! They're the ones that desperately need more download than upload to satisfy their highly asymmetrically provisioned customer connections, right?
Ultimately the problem here is that there are several analytical frameworks that one can use to decide who is the asshole here and who is the saint.
Given that I'm held captive by my consumer ISP and have a fair amount of choice over what content I subscribe to, I'm more inclined to believe that the consumer ISPs are abusing their power over the content providers. I don't have pleasant interactions with them basically ever so I can't see why I would extend them the benefit of the doubt.
Right now, they want to send to the ISP more than the ISP wants the data. So they have to pay. If they wanted to charge, the ISPs would say "sorry".
And you sort of ignored my question. You claimed above that paid peering for asymmetrical connections was not standard practice. I want you to source that, because the ISPs claim it always was standard practice.
>Ultimately the problem here is that there are several analytical frameworks that one can use to decide who is the asshole here and who is the saint.
That's why it's better to let everyone bargain it out amongst themselves. If it's important to netflix to have a faster connection, they'll pay for it. If it's important for the ISP to have a faster connection, they'll pay. In a free market, the resources should flow to whoever needs them most. (I believe that's the Coase theorem.)
If I had the opportunity to ditch my ISP who wants to throttle Netflix through underhanded, shady tactics I WOULD ABSOLUTELY DO SO!
But the various entities were granted regional monopolies on certain wire plants like coax and utp and those huge, huge, huge advantages have yet to be leveled by the same entities that enacted the laws which caused them.
If I had the option to choose from at least say 5-10 different ISPs and some of them did what Comcast, Verizon and AT&T choose to do and other did not, I would vote with my wallet and choose a provider who does not engage in such tactics. This would cause those providers to lose customers and eventually, those providers to reconsider their "paid peering" stance. Or not! But I wouldn't be subject to the whims of a few business folk at a company that I have no practical way to influence.
As it stands, I do not have that choice. So I'm not terribly opposed to various ISPs being told through various means to play nice or have their toys taken away.
But given that I don't have any kind of real choice (duopoly at best and I'm in a major metro area!) it's kind of disingenuous to say "the free market knows best!" when there is in fact not a free market. Free the market up and let people duke it out. But don't pretend that the un-free-ness of the market somehow doesn't exist just because there are multiple long-haul transit providers.
I've argued elsewhere that throttling based on content shouldn't be allowed for reasons similar to what you're saying, but I don't want to extend that to paid peering or prioritization.
Right, but those two issues are for me, inseparable. If I don't have the freedom to choose my ISP and my ISP doesn't take my opinion into account when making business decisions, then I have no recourse.
If there were other ISPs for me to choose from, then I'm all for absolutely ZERO laws about net neutrality because then actual market forces can actually go to work.
Market forces on the consumer side of consumer ISPs can directly influence the happenings on the peering side of consumer ISPs.
> I think every participant should be allowed to make their own decision whether to pay, and the other side shouldn't be forced to carry traffic it isn't paid for from that side.
Sure, so now Cogent and Level3 can demand $100/Mbps/mo from Comcast right? Because without buying global transit, it's not really INTERNET, it's just Comcast. What if all the transit providers decided that those were the new rules? Would Comcast have any alternative?
What if these transit providers all got together and agreed that none of them would offer Comcast global transit for less than $100/Mbps/mo? Would that be OK? I mean, Comcast has highly asymmetric flows right? They should be allowed to do this, shouldn't they?
Because as it stands right now, I've got a million Comcasts I can choose from, but they've all agreed with one another that the pricing is the same no matter which one I pick. If Comcast can hold me hostage, why can't the transit providers all agree to hold Comcast hostage?
