Https hurts users far away from the server(antoine.finkelstein.fr) |
Https hurts users far away from the server(antoine.finkelstein.fr) |
This blog post is a nice example: 30 requests (ublock origin blocked another 12, with those enabled the time to load increases to a whopping 28 seconds), 2.5M transferred, 7 seconds load time. And all that for 4K payload + some images.
The number of connections to one host isn't relevant in HTTP2. As ublock is blocking some ~20 connections these are going to different hosts. Connecting to a different host in HTTP2 is no different then HTTP1.1
If your HTTP2 is terminating at MANY boxes within your infrastructure you are failing to understand how HTTP2 works. Connections within a single TLS/TCP/IP connection are free, new TLS/TCP/IP connections cost exactly as much as before.
I am currently working as a dev in an SEO-Agency (in Austria), and we never believed this hypothesis - so we tested this once with a bunch of our sites:
When moving sites with a German speaking audience to a VPS in America, your rankings at google.de/google.at will decrease (slightly - the effect is not that big) - the other way around your rankings will improve (slightly).
However - even if your rankings would improve when moving to America I would recommend keeping your sites hosted in Europe: The increase in rankings will not offset the decrease in user satisfaction and therefore the decline in your conversion rates.
A bit off-topic, but out of curiosity, have you run any other interesting experiments like this? I would love to read a blog post about them.
And here is the article on budget https://webmasters.googleblog.com/2017/01/what-crawl-budget-...
Although doesn't help for all types of requests, it has its uses.
I would also assume that Google is smart enough to take the physical location of your server into account when calculating how much penalty to apply in which searches. Sites that load fast in Germany should have higher ranks in searches from Germany.
Is this the initial handshake which understandably introduces latency?
After that, times should be similar. What could be killing users far away is requiring multiple handshakes because multiple things requiring handshakes are being introduced at the same time.
For reference, I'm physically located in China so requests have to go through a bunch of filtering-oriented routers, and get 150-180ms from US, 200ms Japan and 180ms Singapore (yay geography) and around 200-250ms from Europe - this is SSL requests and not from a connections hub like Shanghai or Shenzhen close to domestic exit-points. Double to triple these times for first handshake.
From a VPS in Sydney, with a Good Enough bandwidth:
root@sydney:~# speedtest-cli 2>&1 | grep -e Download: -e Upload:
Download: 721.20 Mbits/s
Upload: 117.89 Mbits/s
root@sydney:~# ./rg-diag -json https://www.theregister.co.uk/ | grep -e elapsed_time -e cloudflare_time -e origin_response_time
"elapsed_time": "0.539365s",
"origin_response_time": "0.045138s",
"cloudflare_time": "0.494227s",
Why does railgun help? Because this is what a user would get otherwise; the "whitepapers" site is hosted in the UK, and doesn't use Cloudflare or Railgun – it only uses Cloudflare for DNS:
./rg-diag -json http://whitepapers.theregister.co.uk/ | grep elapsed_time
"elapsed_time": "0.706277s",
How much would https add, if it were done without Cloudflare's https and Railgun? That's easy to check, as our the whitepapers site has TLS (although admittedly not http/2):
root@sydney:~# ./rg-diag -json https://whitepapers.theregister.co.uk/ | grep elapsed_time
"elapsed_time": "1.559860s",
It's a bit older, but here's some info, much of it is still valid: https://istlsfastyet.com/
Without OCSP browser makes slow request to CA, but caches results for a long time so slow request happens not often.
With OCSP stapling enabled more data is transferred between client and server on each TLS handshake.
Main proponents of OCSP stapling are CA, because it saves them bandwidth/hardware.
I'm not certain how session resumption plays into this either. If OCSP is skipped for resumed session as well (which would be my guess), you'd probably not take that small bandwidth hit all that often.
As an aside, OCSP stapling improves your user's privacy quite a bit as well, by not giving your CA a list of all IP addresses connecting to a domain.
It's probably controversial, but I'd love to see a yellow security icon in browsers when sites are using well known https relays that can see plaintext (or are doing other obviously bad things, like running software with known zero day exploits, etc)
Nice part about cloudflare though is that they can use anycast to determine location and then send the closest server IPs. For sub-$200/mo, you're not able to do that, you'd have to find a provider that could do it for you, I'm not sure anyone offers country-based anycast DNS alone.
EDIT: Looks like easyDNS enterprise may be able to do it, https://fusion.easydns.com/Knowledgebase/Article/View/214/7/... for about $12.75/mo too. Might be a decent way to brew your own mini caching CDN for fairly cheap.
Anycast doesn't determine location or send the closet IPs, it's all the same IP address announced using BGP (border gateway protocol) to automatically route to the closest (in network travel) server.
First that almost every firewall out there right now supports https snooping via MITM. Example: https://www.paloaltonetworks.com/features/decryption
Second, I just got back from rural China where most unblocked american webpages take between 5-15 seconds to load on my mobile phone many of them take upwards of a minute to load fully. This seems to be a fun combo of network latency, smaller than expected bandwidth, and pages using javascript with a series of different load events to display content. That dompageloaded->xmlhttprequest -> onreadystatechanged chain can ad some serious time on a 500ms round trip, and that's without talking about the css, the images, and the javascript.
I forgot to pay me electric bill before I flew out and it took me nearly an hour to login, push pay my bill, accept the terms, and confirm payment. I was not a happy camper.
It seems to me that while https is a very good thing, in some cases http and low bandwidth solutions might be worth implementing. It seems to me that one might actually want to tailor this to your audience, no one in their right mind is going to waste 5 minutes loading your web page. If they are so desperate they need to wait, they are going to hate you every minute they do it.
Seems prudent to mention that this requires cooperation of the client bein MitMed. Specifically, the client needs to install a root certificate.
That sucks but I don't see how having a site where you may have to enter payment information on an unsecured connection would be a solution.
$ curl -o /dev/null -s -w "@time-format.txt" http://rest.ably.io/time
time_namelookup: 0.012
time_connect: 0.031
time_appconnect: 0.000
time_pretransfer: 0.031
time_total: 0.053
$ curl -o /dev/null -s -w "@time-format.txt" https://rest.ably.io/time
time_namelookup: 0.012
time_connect: 0.031
time_appconnect: 0.216
time_pretransfer: 0.216
time_total: 0.237
Luckily not that much of an issue for us as when using an actual client library (unlike with curl) you get HTTP keep-alive, so at least the TCP connection doesn't need to be renewed for every request. And most customers who care about low latency are using a realtime library anyway, which just keeps a websocket, so sidesteps the whole issue. Certainly not enough to make us reconsider using TLS by default.
Still, a bit annoying when you get someone who thinks they've discovered with curl that latency from them to us is 4x slower than to Pubnub, just because the Pubnub docs show the http versions of their endpoints, wheras ours show https, even though we're basically both using the same set of AWS regions...
The Cloudflare Railgun is an interesting solution, and one that could be implemented in the context of an SPA over a websockets connection. Or conceivably some other consumer of an API.
https://tools.ietf.org/html/draft-thomson-http-bc-00, and Ericsson's article on it https://www.ericsson.com/thecompany/our_publications/ericsso...
[0]: https://hpbn.co/
Why? Did it hurt user engagement? Were people complaining the site was slow?
If it’s no, then clearly we should improve.
To their credit this post talks about improving the performance, instead of just using it to complain about they can't use https because of a difference in a metric that may or may not actually cause end users any pain.
I understand that by using the generic CF free cert, https terminates at CF and the connection CF->Origin is over unencrypted HTTP. Is this why there is latency overhead? Because CF cannot connect to origin via https so it cannot open a persistent tunnel? Or is it because the overhead of keeping an open https tunnel per origin server is prohibitively expensive to maintain for every free customer?
I assume that even though there is no persistent tunnel, CF still must still use persistent TCP connections?
That would be interesting.
root@sydney:~# ./rg-diag -json https://thereglabs.com/ | grep -e elapsed_time
"elapsed_time": "0.863677s",
root@sydney:~# time curl -sH 'Host: thereglabs.com' -H 'Cf-Visitor: {"scheme":"https"}' http://THE_ORIGIN_SERVER/ -o/dev/null
real 0m0.821s
We've got another domain on the business plan… so let's try that one.
This is an _image_ request, which is _cached by cloudflare at the edge_:
root@sydney:~# ./rg-diag -json https://regmedia.co.uk/2016/11/09/hypnotist_magician_smaller.jpg | grep elapsed_time
"elapsed_time": "0.239641s",
… compared to if we were to request the same file from the ORIGIN_SERVER over HTTP:
root@sydney:~# ./rg-diag -json http://ORIGIN_SERVER/2016/11/09/hypnotist_magician_smaller.jpg | grep elapsed_time
"elapsed_time": "0.704458s",
… and that image was likely in their cache; so… let's add a parameter so they _will_ have to ask the origin server:
$ pwgen 30 2
Eehacoh2phoo1Ooyengu6ohReWic2I Zeeyoe8ohpeeghie3doyeegoowiCei
root@sydney:~# ./rg-diag -json 'https://regmedia.co.uk/2016/11/09/hypnotist_magician_smaller.jpg?Eehacoh2phoo1Ooyengu6ohReWic2I=Zeeyoe8ohpeeghie3doyeegoowiCei' | grep elapsed_time
"elapsed_time": "1.198940s",
All in all, from the point of view of being able to _easily_ serve people on the other side of the world with a "good enough" (not great, mind you!) response time, both "standard" Cloudflare, the "pro" offering _and specifically_ the "business" offering are just effin AWESOME.
Most websites are on virtual servers (hardware in general) that is not owned by them. For example, Amazon could easily let the NSA look into your AWS server directly. IMO, the url lock should just be an encryption auditor. The end website is using acceptable algorithms and has a currently valid certificate? That's good enough.
Almost any HTTPS site can be forged/"broken" (unless they're using preloaded HPKP), if the attacker has root certificates (or even just a bug in a CA website), which the NSA certainty does.
Nation state adversaries just aren't really within the typical TLS threat model. I do concede that it does make agencies jobs much harder if used correctly, however.
CloudFlare's "Flexible SSL" offering means a CloudFlare "https://" site is quite likely to not even have that level of security though. They send supposedly HTTPS data unencrypted and unauthenticated across the open Internet; if that doesn't warrant a yellow/red icon then I don't know what does.
Hm. Good idea, why not go a step further and turn the 'no server signatures' advice on it's head: full disclosure, server signatures on, in fact, list each and every component in the stack so that end users can (through some plug-in) determine whether or not a site is safe to use.
Of course nothing bad could ever come from that. /s
I'm all for making the use of for instance Cloudflare less transparent so that users know who they are really talking to, but I'm confused about how you'd want to establish what a site is running without giving a potential attacker a lot of valuable information.
FWIW, my personal website uses let's encrypt, so it would be yellow or worse.
Anyway, I like the idea of tying the security color in the url bar to an attacker model, since it at least gets people to think about attack models.
Does anyone remember a few years ago when Google found out through leaks that the govt was wiretapping it's private traffic between datacentres?
What makes you so naive to think that the govt isn't sniffing every single page on cloudflare?
The risk here is real, but it's much more pervasive than one data handler.
If your risk profile is outside the boundaries of normal internet use then you likely already know what to do - and we now have a multitude of tools for more private communications.
This analysis seems flawed. If you care about mass surveillance, you want their top-tier security and legal teams working for you.
In an ideal world you'd want Youtube's (Google's) legal teams working for you, protecting you against DMCA abuses and alike... but they don't. Not unless you're a top-tier YouTuber and even then it's laughable dice roll as to whether they feel the bad press is worth their time to do anything.
And I don't believe from a bottom-line perspective the shareholders believe it's worth their time to do anything more than provide a platform (no matter how problematic) and market it.
Will Cloudflare do everything they can to keep your content accessible? Sure. Anything above and beyond that? lol. Good luck with that...
Among other reasons, not encrypting traffic gives an opportunity for bad actors to replace content in transit to your end users when your end users are on compromised connections, such as rogue "free" wifi networks in airports or coffee shops, or even legitimate networks which have in some way been compromised, e.g. the ISPs of the world who decide to inject other content e.g. their own ads into unencrypted traffic.
The next question is usually "what could they possibly do, change a few pictures?"
They could inject malicious payloads, and for all your users would know, it would appear to them that it came from your site.
> I can't use LetsEncrypt with my hosting provider
Consider switching. For a static site, consider Gitlab; they do a good job of permitting LetsEncrypt.
---
I sincerely appreciate the question, though. I have marketing people ask me this question all the time in private who hesitate to do so in public because quite a few security types berate them for not doing something "obviously" more secure. It's not at all obvious to most of the world's web designers and content creators that a static site should be TLS'd until it's framed (heh) in this manner. The fact that you asked brings about a massive educational moment.
Anyway, consider switching hosts. :)
don't know the answer myself here.. there are good technical reasons, I agree..
but it is a logical fact that if google search was always 100%, there would be no need for adwords and site ads...
The Internet is not a safe place. We should aim for HTTPS EVERYWHERE.
I think that's a bit sensationalist.
They're not as easy to get away from as you think.
You should think about https for sites like yours the way you think about vaccines. SSL everywhere makes everyone safer, even though it doesn't have a tremendous impact on your own site.
Also, shameless plug, if you want really easy SSL you can use our new startup: https://fly.io. I'm not sure what country you're in, but we have a bunch of servers all over to help make it fast. :)
The second is more moral. Making https the default means more and more of the web will be encrypted and authenticated. This is a good thing.
You forgot about the great firewall of China playing merry MITM with your connections.
I wonder if it would be lower latency to open a single websocket tunnel on page load and download assets over the tunnel. Although at that point I suppose you're just replicating the functionality of http/2.
We regularly test different things, but few are as extensive as this "server location test".
This one was quite easy to do - and to revert - even when doing it for a lot of sites: just duplicate your sites on another continent and change your dns-settings.
Sadly we do not blog about this stuff. As our customers are not particularly fond of sharing their data - and blog posts without precise data are not useful at all...
Additionally, most of our assumptions and hypotheses were wrong. So most of our blogsposts would sound like:
"We thought google would work like this, but sadly we were wrong"
SEOs might like these posts - but potential customers probably not so much :D
If you have a RTT of seconds, 10 additional roundtrips can cost an entire minute.
These things become very noticeable very quickly if the system is in non-perfect environments.
Additionally, even with modern browsers it takes far too long to open a website — it should be 100% instant, less than one or two frames (16 or 33ms). That's not possible, as RTT is usually around 18ms between users and CDN edges, but at least it should be below perceivable delays (100ms).
EDIT:
My best websites hover around 281ms to start of transfer, and 400ms to the site being finished.
That's improvable, but most sites out there take literally half a minute to load.
Now go on a 64kbps connection, and try again. Handshake takes seconds, start of transfer is almost after 30sec, and by the time your website arrives your coffee has gone cold (a few minutes for a google search).
Years ago, Google was usable on dial-up. Now even the handshake takes as long as an entire search used to take. Notice anything?
They sell you a widget/service. That's where your relationship ends.
You can email me if you want to avoid cluttering HN: mrkurt at gmail.
I think we can safely say that this would be a very weird way to go about earning a few bucks through CA investments.
[1]: https://w3techs.com/technologies/history_overview/ssl_certif...
[2]: http://pki.goog/
Handle your TLS/HTTP2 termination on the same boxes and use HTTP1.1 within the internal network.
Checking some of my sites I see:
- values ranging vom 320 - 410 for a bunch of German speaking sites hosted in Europe
- and values of 221 and 240 for my two English speaking sites hosted in America (via firebase - on googles own infrastructure)
So if you are concerned with your crawl budget, I think you better focus on things like:
- On-site duplicate content
- Soft error pages
- and Low quality and spam content
Plus you could also get some high quality backlinks.
And please be aware that the crawl frequency does not directly influence your rankings. So, as long as your do not really have a big problem regarding your crawl budget, you may spend your time wiser if you focus on other metrics.
PS: You may already know the tools, but others could be interested:
If you want to optimize you sites performance in respect to SEO use the following tools:
https://developers.google.com/speed/pagespeed/insights/?hl=d...
The most important aspect is "Reduce server response time" - you have to pass this!
Choose a server near you and aim for a Speed-Index of maximum 3000 - I personally target 1000, but depending on your influence regarding the website's frontend you will not be able to achieve this.
For some comparison results: Uncached, with my own CMS indexing and analyzing a 6GB database of crashdumps and providing an overview with graphs over that, I get a score of 623. (This running on a 9€/mo dedicated server).
That problem would not exist on 'any 3rd party host'.
For example, if you're using a ccTLD for the domain, or if it's a generic-TLD and you declare a country in Google Webmaster Tools, that will be a much stronger weighting.
Of course, if that's wrong I'd love to know!
They did recently release some features called traffic manager that lets you control the origin server based on geo. If you just need geo-balanced DNS though, AWS Route 53, Azure Traffic Manager, and NSOne offer DNS based routing.
I seem to remember getting different IPs from different locations, but it could just be random or I could be mistaken.
EDIT: Tried now and it seems I'm getting the same IPs from Canada and Australia, so you are indeed correct.
Is there any solution other than totally killing HTTP that protects from HTTPS stripping attacks? HSTS won't protect first visit and STS preload lists can only be so large.
When I popped my SIM into my iPhone it forced me to download a configuration profile with a self-signed Vodafone cert, which means they can mitm any connection. I think this is required by the government so they can block adult websites by default? (I've also seen torrent websites also fail silently with misleading "server not found" errors)
I haven't looked into if they're doing the filtering via DNS or mitm, but I avoid the censorship by connecting to a vpn.
The filtering in the UK is by inspecting HTTP requests, so when a single image on wikipedia.org was blocked, every request to Wikipedia ended up going through each ISPs hidden proxy. [3]
According to [2], HTTPS sites aren't filtered -- but it references a page from 2004. I suspect HTTPS sites are now simply blocked outright at either DNS or IP level, but I don't have a way to verify this, and can't find any details.
[1] https://wiki.openrightsgroup.org/wiki/Internet_censorship
[3] https://en.wikipedia.org/wiki/Child_abuse_image_content_list...
[2] https://wiki.openrightsgroup.org/wiki/Cleanfeed#cite_note-LI...
The "server not found" errors sound like DNS blocking, which they can do without MITM.
Pretty much all major CDNs use anycast today for load balancing, rolling downtime and security/ddos protection. Http/tcp connections are usually short-lived, relatively cheap to setup (since it's an edge network anyway) and BGP route updates don't happen that often.
LinkedIn switched to anycast too after testing: https://engineering.linkedin.com/network-performance/tcp-ove...
And infact reducing the number of requests using things like spritemaps, bundling js and css is actually an antipattern with http2.
Those are just ways to lose some of the impact of bloat without addressing the bloat itself.
If you address bloat directly it will benefit all users.
With http2 it's all multiplexed into one connection. So you have the one ssl connection, but that one connection has multiplexed connections inside it. And since it's one tcp, the tcp sliding window has opened up and is actually faster then opening a new tcp connection.
Correct me if I'm wrong but I'm pretty sure this enables complete MITM by Vodafone when using cellular network.
http://imgur.com/b0il5xb http://imgur.com/3mw5ZGZ http://imgur.com/6ehhfuZ
This shouldn't effect your security stance.
There's a common misconception that you trust your private keys with your CA and they can somehow transparently MITM you. But they only have your public key, not your private keys, so they can't do that.
The security threat from trusted CAs is that they can MITM anyone, regardless of if you use them or not. BUT the attack isn't transparent, and things like cert pinning are effective in the real world from preventing attacks.
It's an attack that's difficult to deploy because it's easy to detect if you're looking in the right places, and as soon as it's detected, you know the CA has been compromised, and the attacker loses a large investment.
In particular, the company operating the data center your server is in can reliably do this, and so can the backbone provider they use, and probably the server's local government. The DNS provider that controls your domain can mitm the ca process too (though with a higher chance of detection).
The argument for making domain validation yellow (and not red) is that domain validation protects against attacks from residential ISPs / coffee shops, and it would also be hard for a foreign government to launch the attack against their own citizens. They basically have to compromise the CA, tamper with your browser, or just randomly break https with "bad certificate warnings".
Over time, I'd hope more bad security practices (crypto related or not) would lead to yellow bars.
For instance, intel secure enclaves help cloud security a lot, but they are still exotic. If they catch on, and you're at a vps that doesn't offer something like that, then you get a yellow bar starting in 2027.
Otherwise it doesn't matter if you use Let's Encrypt as the attacker could just initiate the validation regardless of your CA and end up with a valid certificate (which would still fail cert pinning)
Edit: Oh I see, it's a more about if DV should ever be green.
Right, but if I go to PayPal.com and the address bar says "Comcast [US]" or "National Security Agency [US]", I'll know something's up.
Though, that just shifts the potential problem towards anyone in the DNS-sec chain of trust. Most notably, the controller of the TLD (most often a government) and the registrar.
Thing is, those are also an issue with normal (http based) DV. After all, they could also change the A record for the domain they want a cert for.
I believe the current solution to all this is focused on detection rather than prevention (through certificate transparency and similar proposals). The idea being that any organization that isn't trustworthy will only get to pull of this hack once, in a short time frame before having their trust revoked.