Mitmproxy 11: Full HTTP/3 Support(mitmproxy.org) |
Mitmproxy 11: Full HTTP/3 Support(mitmproxy.org) |
Back in 2011, I was using it to learn API development by intercepting mobile app requests when I discovered that Airbnb’s API was susceptible to Rails mass assignment (https://github.com/rails/rails/issues/5228). I then used it to modify some benign attributes, reached out to the company, and it landed me an interview. Rest is history.
Sometimes it's easier to use mitmproxy with an existing implementation than to read the documentation!
;)
> Chrome does not trust user-added Certificate Authorities for QUIC.
Interesting. In linked issue chrome team says:
> We explicitly disallow non-publicly-trusted certificates in QUIC to prevent the deployment of QUIC interception software/hardware, as that would harm the evolvability of the QUIC protocol long-term. Use-cases that rely on non-publicly-trusted certificates can use TLS+TCP instead of QUIC.
I don't follow evolution of those protocols, but i am not sure how disallowing custom certificates has anything with "evolvability" of protocol ...
Anyone knows are those _reasons_?
They can easily release a sightly tweaked QUIC version in Chrome and support it on e.g Youtube, and then use metrics from that to inform proposed changes to the "real" standard (or just continue to run the special version for their own stuff).
If they were to allow custom certificates, enterprises using something like ZScaler's ZIA to MITM employee network traffic, would risk to break when they tweak the protocol. If the data stream is completely encrypted and opaque to middleboxes, Google can more or less do whatever they want.
Kinda related: https://en.wikipedia.org/wiki/Protocol_ossification
One of the reasons for developing HTTP 2 and 3 was because it was so difficult to make changes to HTTP 1.1 because of middleware that relied heavily on implementation details, so it was hard to tweak things without inadvertently breaking people. They're trying to avoid a similar situation with newer versions.
I think because of KZ case browsers and Chrome especially went for using only their own cert store instead of operating system one.
Those proxies tend to be strict in what they accept, and slow to learn new protocol extensions. If Google wants to use Chrome browsers to try out a new version of QUIC with its servers, proxies make that harder.
Maybe more accurately “chrome is designed to work for you in so far as that also works for google”. I share the long standing dismay that so many willingly surrendered their data and attention stream to an ad company.
Evidently Firefox is not a typical browser anymore.
That said, for many people who care about this stuff, this could be an option. There's nothing preventing you from doing this technically speaking.
There's a small risk of triggering subresource integrity checks when rewriting Javascript files, but you can probably rewrite the hashes to fix that problem if it comes up in practice.
And even when it is HTTP, as other commenters said, the reverse proxy is able to handshake connections to the backend much more quickly than an actual remote client would, so it's still advantageous to use http/2 streams for the slower part of the connection.
That's just called a web server and not a reverse proxy then. Both are just evolutions of CGI.
At a startup I was working on a few years ago, I set up mitmproxy in dev and eventually if memory serves right I also sometimes enabled it in prod to debug things.
That being said, we did not have a lot of users. We had in fact very very few users at the time.
The servers which run the frameworks have to http/3. In most cases the advantages should be transparent to the developers.
I’d totally pay a cloud provider to just do this and forward requests to my port 80 or 443 with self signed certificates.
Https+acme is already open to this attack vector, so why inconvenience myself by pretending it is not?
So inserting mitmproxy into the setup was just a case of putting it between the Nginx or Caddy that did TLS termination, and the web server that served the backend API. So to mitmproxy it was all plain HTTP traffic passing through it, locally on the same machine.
I bound the mitmweb web UI to the VPN interface so that us devs could connect to the dev server with VPN and then have access to the mitmweb web UI to inspect requests and responses.
Since http/3 uses udp to send the fragments, duplicate packet information doesn’t have to be sent.
Kind of funny the newer protocol effectively works in the opposite direction of GraphQl.
But for doing split VPN with IP addresses they need to create an IP route in the VPN client. If you just have a couple IPs, it's fine, but if you have a couple hundred targets, you're gonna break some guys Windows or Mac machine sending that huge routing table.
Also, there are targets that change IP addresses. For example, AWS Elastic Load Balancers change IP addresses sometimes (if nothing have changed in the last years, haven't deployed ELBs in a while...).
If you're going to say that Firefox doesn't care about having users, point out its just stunningly stupid memory usage, blatantly stale developer tools (that one hurts me the worst because the Chrome dev-tooling is actually open source, so there's nothing stopping them from actually having Best In Class dev tooling other than no-fucks-given), or the har-de-har-har that comes up periodically of bugs that have been open longer than a lot of developers have been alive
1: https://github.com/mozilla-services/syncstorage-rs#running-v...
Of course one can just use Firefox Sync out of the box <https://support.mozilla.org/en-US/kb/sync>; even Mozilla has not yet stooped so low as to require opening a terminal just to use Firefox or its Sync component
You can use QUIC with custom certs without any trouble.
And while you can use QUIC with custom certs in a technical sense if you do the compile flags and build your own universe, 99.9999% of the people on Earth with their standard QUIC lib implementations (most use the same two) will be unable to connect to it.
For Chrome at least..!
Yes, the protocol is weird, compared to TCP. It has many extra features and one restriction, which is mandatory TLS, which I wouldn't even consider skipping anyway. Still nothing to do with ads.
If you compare that to the graceful Connection: Upgrade handshake in http/1.1 and websockets, for example, this would've been much better because there is no isolation based on tools and libraries, only based on trust chain of the certificates. If a new version of the protocol breaks, it automatically falls back with both parties knowing about it. If QUIC protocol has changes on the client side, good luck finding that out.
Either way, I still fail to see how this relates to the original complaint that QUIC somehow leads to ads.
"HTTP/3 uses QUIC, a transport layer network protocol which uses user space congestion control over the User Datagram Protocol (UDP). The switch to QUIC aims to fix a major problem of HTTP/2 called "head-of-line blocking": because the parallel nature of HTTP/2's multiplexing is not visible to TCP's loss recovery mechanisms, a lost or reordered packet causes all active transactions to experience a stall regardless of whether that transaction was impacted by the lost packet." (Wikipedia)
I'm a text-only browser user; I also use TCP clients with a localhost TLS forward proxy. When I visit a website I only request resources manually and only from a single domain at a time. This works great for me. But obviously it is not compatible with ads. This is because ads requires websites that cause so-called "modern" browsers to automatically request resources (ads) from multiple domains "simulataneously", i.e., "in parallel". As such, I see no ads. Not a problem, I can live without them.
However, for those who let these browsers request ads automatically, there are problems. "The web", as they experience it, gets slower. Because although the requests to multiple domains may be executed "simultaneously", and the HTTP/1.1 protocol allows for pipelining requests, these browsers cannot process the responses that arrive out of order.
HTTP/1.1 pipelining does work. Outside the browser, when requesting non-advertising resources, for example. I have used HTTP/1.1 pipelining outside the browser for over 20 years. It works beautifully for me. I am only requesting resources from a single domain and I want the responses in order. So, right there, we can see that HTTP/3 is addressing the needs of advertising companies and their partners rather than web users who are reqesting non-advertising resources like me.
As the web became infested with ads/tracking and became slower as a result, the ad companies and CDNs, e.g., Google and Akamai, sought to "make the web faster" by introducing a new HTTP protcocol for the so-called "modern" browser. (This browser, as we know, is more or less under the control of an advertising company, the same one introducing the protocol.)
Now, a web user (cf. institutional player in the online advertising industry) might conclude the easiest way to "make the web faster" is to remove what is making the web slow: ads.
But the "solution" chosen by the advertising company was to keep the ads and try to make it easier for the so-called "modern" browser to process out-of-order responses from multiple domains, e.g., ad servers, faster.
Will HTTP/3 and its use of QUIC "lead to ads". That is for the reader decide. I think it is safe to say it will not lead away from them and it could help cement the ad infestation of the web even more.
tl;dr HTTP/1.1 is optimal for sequential information retrieval from a single domain. HTTP/3 is optimal for resource, e.g., advertising, retrieval from multiple domains "simultaneously".
In practice http/1.1 with 1 connection per request even encouraged making more domains/subdomains if you wanted to have more simultaneous requests
The ad company created QUIC to make HTTP/3 possible. That's also certainly true.
What follows: the ad company created QUIC because they wanted to make ads easier.
But "QUIC exists to improve ad deliverability" is true only in the shallowest of senses, similar to "nuclear power plants exist to drop bombs on civilians" just because research on nuclear power was driven in large part by war needs. In reality, nuclear power has its own uses beyond the military.
Similarly, QUIC taken on its own merits does not have anything to do with ads. It's just another general purpose protocol.
BTW, multiple streams will not make it any faster to load ads from third parties. Head-of-line blocking only affects resources within a single TCP connection, which can only ever be to one server. That means QUIC's streams do nothing to make loading Google's third party ads easier.