What would happen if we didn't use TCP or UDP?(github.com) |
What would happen if we didn't use TCP or UDP?(github.com) |
We could've had it as a QUIC replacement if it weren't for terrible middleboxes and IPv4 NAT screwing everyone over once again. Hell, NAT wouldn't even have been an issue had SCTP support been widespread before consumer NAT devices started being implemented.
It's a prime example of how new network protocols now have to lie and deceive to work over the internet. TLS needs to pretend to be TLS 1.2, QUIC needs to pretend to be an application on top of UDP while reimplementing SCTP and TCP, and even things like secure DNS are now piped over HTTPS just in case a shitty middlebox can't deal with raw TLS.
It would have forced us to adopt V6 and… my god… the network would be so much cleaner and superior in every way. Things would just have addresses, and protocol innovation at L3 would be possible.
Because QUIC uses UDP, which is supported by most/all intermediate routing equipment.
more importantly though, it transmits multiple independent streams of message chunks in parallel.
similarity with UDP ends at message oriented nature of the protocol. closest equivalent for TCP would be MPTCP I suppose ?
SCTP you're talking about runs on top of DTLS on top of UDP. DTLS has issues on its own, but even if it didn't it wouldn't beat QUIC in TTFB.
QUIC is supposed to be faster than SCTP by combining layers and eliminate round trips. Also, QUIC is a stream protocol like TCP. SCTP makes messages explicit. Both have multiplexing which is why seem different.
Related: there's a parallel Internet with a different root of number allocation called GRX/IPX (GPRS Roaming Exchange/Internetwork Packet Exchange)
SCTP does this.
A shim on top of TCP socket receive could also do this also, as long as there is a convention to prefix each message with a length field, say 16 bits, with the MSB indicating that the message is incomplete and is continued in the next length delimited segment.
One interesting case for a particular Netgear family of routers was that, while the traffic would survive end-to-end it was not without corruption - very specific corruption which took the form of zeroing the first 4 bytes. Given this aligns with where the src/dst ports would normally be, I suspect it was being treated as TCP/UDP albeit without the actual translation path taken.
One would have a hard time communicating with anyone. The internet has standardized around TCP and UDP. There are too many devices in most paths that will not be able to handle other protocols. Replacing all the hardware on the internet would take even longer than deprecating IPv4 in my pessimistic opinion. To get around this there would have to be some significant gains of the new protocol that would warrant big expenses in every corporation and government and all the hardware manufacturers would all have to agree on implementing support and further agree on interpretation of the new RFC.
> Granted, the server was just two hops away from the client, and it didn't have to pass through the scary sea of the internet
If you have CGNAT you're still screwed. Get one of those free ipv6 tunnels.
sock_raw = socket(AF_PACKET , SOCK_RAW , htons(ETH_P_ALL));
IP networks should forward anything, but NAT is a major problem. Would be interesting to try with IPv6.And this brings me to why I love networking on Plan 9. First off the dial string, net!address!service, passes the network along with the address and service port letting you easily switch protocols as needed. e.g. a program could listen on IL port 666 using the dial string il!*!666 and the client dials il!10.1.1.1!666. Second, that lets you dial and listen on any protocol from any program. If one wanted to use raw Ethernet you use the dial string ether0!aabbccddeeff. If I wanted to add a protocol like quic or a dead one like ipx/spx I just need to mount it to /net and keep the semantics the same as other services in net and any program can dial that protocol - ezpz user space networking stacks. Powerful abstraction.
Why did a single packet of the custom protocol get through with all later packets dropped? Does anyone know?
subsequent packets then got blocked because the firewall has no way of matching this to an existing flow.
I guess this is a security feature, since a process cannot even listen on some ports without having elevated privileges. I wouldn't expect another process being able to capture all this traffic anyway. This would also require a mechanism of sending the same stream to multiple processes (TCP listeners and all-protocol listeners).
But I didn't even know it was possible to capture traffic from multiple transport layer protocols using a syscall, perhaps that syscall requires elevated privileges itself..?
You are exactly right
We had the option to make every object addressable and we chose not to. Why keep sweating over it, just accept that software only works on the devs machine and ship that.
There was a time when IPX/SPX was a contender. Xerox pitched XNS directly at TCP. DECNet/OSI was around. There were a lot of others...lot's of experimenting going on at the time.
The way the NATs (network address translators) are sharing the scarce public IPv4 addresses is by multiplexing on the transport level fields (ports in case of TCP/UDP and IDs or inner packet transport level fields in case of ICMP).
Since they are unaware of your protocol, they get into a “special case mode”, which on a naive translator might consume a whole IP address (so you would really make a network admin with a few of those, because you exhaust all their available addresses :-) ; but on the carrier grade NAT there are safeguards against it and the packets are just dropped.
Still lame that in 2024 major services like Steam and Quest basically require IPv4.
I want to be able to use the internet without silly things like exhausting a network admins ipv4s.
A raw Ethernet socket sends packets at the Ethernet level while a raw IP socket sends packets at the IP level. If you want to play on your local network you want Ethernet (maybe). If you want to send weird packets across the Internet you probably want IP, so that you don't have to waste effort doing route lookups and MAC address lookups and your code will still work on non-Ethernet networks, including VPNs.
There are some protocols that run on top of Ethernet but Internet-compatible protocols all run on IP by definition.
This comment was delayed several hours due to HN rate limiting.
I'll try out IPv6
Header integrity was so far off of consideration during IPv4's implementation because the internet was a dozen universities and DoD sites that it was overkill (and possibly a waste of limited CPU cycles at the time).
What's far more likely to have happened is that we'd see more proxies instead of NATs (SOCKS, etc). I don't think that'd be better than NAT.
Now we have a slew of protocols that either implement TLS, or roll their own custom thing, or have X-over-HTTPS protocols, including SSTP and DoH.
Today V6 has been stripped down almost to what it should have been: bigger addresses, then stop. All that's left to get there is to deprecate SLAAC or make it optional.
When a machine receives a packet, after the Ethernet and IP layers have made sure the packet is addressed to this machine, the very next thing that happens is checking what transport layer implementation should receive the packet - and this is done based on the "transport" bits in the IP header. If the packet is, say, TCP, then the TCP layer starts reading its own header and finds out the port number, and proceeds from there.
OH woah woah. Okay that's the root of my confusion then. Nevermind. I thought the OS network stack exposes ports and packets come on to ports
Nevermind then :)
Networks are built in layers. There's a physical layer underneath IP, then there's IP, and then there's TCP and UDP on top of IP.
The OS network stack has components that handle all of these layers. That's why it's called a stack.
Port numbers are part of the individual protocol (TCP or UDP) because there are a lot of things you can do with networking, and port numbers don't necessarily make sense with all of them.
For example, when you ping another computer, that uses ICMP, and there is no need for ports with ICMP. You're pinging the whole computer, not trying to connect with one of several applications running on it. So ports are not really needed.
If you mean socket as in `my_socket = socket(AF_INET, SOCK_STREAM, 0);` that's TCP/IP, not raw IP, so it will have port numbers in the TCP part of the packet. `SOCK_STREAM` and `SOCK_DGRAM` respectively correspond to TCP and UDP. Raw IP sockets on Linux can be created by `socket(AF_INET, SOCK_RAW, protocol);` and that will have no TCP/UDP header attached by the Kernel after the IP header in the packet.
(googling a bit as I write, forgive errors)
The socket API has different types of sockets, typically you use stream (TCP) or dgram (UDP), but you can also get a completely raw socket where you need to construct your own ethernet and IP headers, or create your own replacement of IP protocol. So socket does not mean only TCP/UDP or something with ports, unix sockets are another example of this.
All I can think of is that an "IP address" does not have a port component and that's all that IP deals with.
That, among other things, is a problem. SLAAC is too limited.
You can use DHCPv6 but then you can't use Android because Android, and I think they're alone here, stubbornly and dogmatically refuses to implement it. I guess you could go around and statically assign V6 IPs to Android devices, or run NATv6 with SLAAC for those and DHCPv6 for everything else, but that's annoying as hell.
These are completely invented problems in your head. SLAAC can absolutely advertise a single /64 internally. It can advertise any /64 you tell it to.
DHCPv6 can absolutely respond with DNS servers (and nothing else) in parallel. Configuring your SLAAC daemon to tell clients to get DNS servers via DHCPv6 is a 15 minute exercise with google.
Oh, summer child...
The standard requiring an entire /64 for SLAAC is also just a poor design.
Suppose your ISP is doing the right thing and giving you at least a /56. You have a complex network with your own subnets and you're not sure how many you'll need in the future, but the spec says you only have to give each one a /64 and that seems like plenty, so the person setting up the network does that. Time passes and you get devices in various subnets with fixed addresses you need to stay fixed. Then you want to hook up a VM host or anything else that wants to further subdivide the address space, but the spec says you can't even though there are still scads of addresses. And for what, so they can use EUI-64 which in practice is only 48 bits anyway and is effectively deprecated because it's a privacy fail?
The real pressure we need are to corporate networks. Too many of them think they can use 10.0.0.0/8 forever. Too many of them own giant chunks of public IPv4 space and think they are immune to the address exhaustion. At least the prices for IPv4 addresses are going up at major clouds like AWS and Hetzner. But it still seems too slow of a price rise to hit enough bottom lines that major corporations are feeling the pressure yet.
Many LAN operators conclude that the pragmatic impossiblility of blocking DoH is a net-negative for both network security and censorship avoidance.
https://version6.ru/en/dividing-the-indivisible
If you're just dealing with a single network at home, a /64 is otherwise fine. But there's a reason the recommended handout from ISPs is a /56; the inventors of IPv6 (or more specifically SLAAC) just didn't take into consideration how intransigent big telecoms would be.
Basically, giving out a /64 is the modern equivalent to ISPs saying "you have to pay extra to have more computers on the internet and you're not allowed to use NAT" that was actually a thing up until the mid 2000s.
The frustrating this is that there's no reason to do a /64 as IPv6 was literally designed to hand out huge IP ranges.
"In particular, we recommend:
- Home network subscribers, connecting through on-demand or always-on connections should receive a /48.
- Small and large enterprises should receive a /48.
- Very large subscribers could receive a /47 or slightly shorter prefix, or multiple /48's.
There was some walking that back in later RFCs, arguing that home customers could get by with just a /56.
SHOULD This word, or the adjective "RECOMMENDED", mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.
Not following recommendation from RFC6177 by allocating a perfectly valid /64 (it being inconvenient to some is another story) is not "violates the documented standards"
Of course there is. Blocking all traffic with destination port 443 is virtually impossible. Conversely, blocking port 853 is trivial, and it forces all clients to either not resolve DNS, or downgrade to un-encrypted DNS.
Of course, if DoH had not been encrypted, it wouldn't have mattered that it uses port 443. But being encrypted yet easily identifiable would have also defeated half the point.
Both DoH and DoT achieve actual concealment (and therefore privacy and censorship avoidance) through encryption. That one is more obscure than the other doesn't change the fact that the whole point of both protocols is encrypted DNS queries, not obscured DNS queries.
And again, if I'm the network operator and a host can obscure/obfuscate its DNS queries, then I've lost some measure of control over my network and the hosts that connect to it. I can trivially redirect all TCP/853 traffic to my DoT-capable resolver of choice. I can't do the same for all TCP/443 traffic (i.e. redirect it to my DoH-capable resolver of choice).
I don't care that an eavesdropper can observe discrete TCP/853 traffic because it's encrypted. The whole point is maintained, and I've maintained control over my private network.
You're mostly commenting on the negative effect that DoH has over a private network administrator's legitimate need to control DNS resolution in their own private network.
I was discussing how DoH has positive effects on a network user trying to evade illegitimate control over their DNS resolution on the Internet, such as legally enforced DNS-based censorship of certain sites. Several countries have legally mandated ISPs log and prevent resolution of, say, thepiratebay.com; and some include a requirement to prevent attempts at circumvention of these bans, such as DoT traffic (they might also ban DoH traffic to well known resolvers, which is where own proxies come in).
Regardless, I think we can both agree that DoH was not created to work around ossification, the way QUIC was built on top of UDP instead of being a separate transport.
> I don't care that an eavesdropper can observe discrete TCP/853 traffic because it's encrypted.
Also, this is another level of miscommunication. I agree you don't need DoH to protect from eavesdropping, DoT works just as well. DoH protects from ISPs dropping easily-identifiable DoT packets to force a downgrade to regular plaintext DNS.
SCTPs mistake was it wasn't implemented as a userland library on top of UDP to begin with.
Edit: a quick search showed that NAT traversal is an issue (of course!)
AFAIU, Apple has flexed their muscle to improve MPTCP support on networks. I've never seen numbers, though, regarding success and usage rates. Google has published alot of data for QUIC. It would be nice to be able compare QUIC and MPTCP. (Maybe the data is out there?) I wouldn't presume MPTCP is less well supported by networks than QUIC. For one thing, it mostly looks like vanilla TCP to routers, including wrt NAT. And while I'd assume SCTP is definitely more problematic, it might not be as bad as we think, at least relative to QUIC and MPTCP.
I suspect the real thing holding back MPTCP is kernel support. QUIC is, for now, handled purely in user land, whereas MPTCP requires kernel support if you don't want to break application process security models (i.e. grant raw socket access). Mature MPTCP support in the Linux kernel has only been around for a few years, and I don't know if Windows even supports it, yet.
Maybe its me being stupid but why don't we use quic always instead of tcp?
I think it has to do with something that I read that tcp can do upto 1000 connections simultaneously no worries and they won't interfere with each other's bandwidth / impact each other , but udp does make it possible for one service being very high to impact other.
There was this latest test by anton putra with udp vs tcp and the difference was IIRC negligible. Someone said that he should probably use udp in kernel mode to essentially get insane performance I amnot sure
A big reason is because QUIC is a lot younger than TCP and it will take a while for all the use cases of TCP to decide (if they are actively maintained and looking at possible upgrades) if QUIC is a good option worth testing.
QUIC's rollout so far hasn't been entirely without bugs/controversies/quirks/obstacles/challenges. You still see a lot more HTTP/2 than HTTP/3 connections in the current wild and that doesn't seem to be changing near as fast as major providers upgraded HTTP/1.x to HTTP/2. There's still a bunch of languages and contemporary OSes without strong QUIC support. (Just the other day on HN was a binding for Erlang to msquic, IIRC, for a first pass at QUIC support in that language.)
Some point soon QUIC might start feeling as rock solid as TCP, but today TCP is (decades of) rock solid and QUIC is still a lot new and a little quirky.
I think it is to the point that if your user base doesn't warrant it, (i.e. you are targeting well connected devices with minimal latency/packetloss) it's not even worth turning HTTP/3 on
That is kind of nice to know actually. The support will come considering its built on top of UDP. You just need people pushing and google is already pushing it hard .
The main problem is quic's support in languages. But support will come.So after reading this comment of yours , I am pretty optimistic about quic overall
(On the diversity of implementations front: So far we've got Google's somewhat proprietary implementation, Apple's kind of broken entirely proprietary implementation, and Microsoft's surprisingly robust and [also surprisingly to some] entirely open source C implementation. General language support would be even worse without msquic and the number of languages binding to it. Microsoft seems to be doing a lot more for faster/stronger/better QUIC adoption than Google today, which I know just writing that sentence will surprise a lot of people.)
There will be trade-offs to be found with TCP. For instance, a lot of discussion elsewhere in these threads is on the overbearing/complicated/nuanced congestion control of TCP, but that's as much a feature as a bug and when TCP congestion control works well it quietly does the internet a wealth of good. QUIC congestion control is much more a binary: dropped packets or not. That's a good thing as an application author, especially if you are expecting the default case to be "not" on dropped packets, but it doesn't give the infrastructure a lot of options and when pressure happens and those "allow UDP packet" switches are turned off and most of your packets as an application developer are dropped how do you expect to route around that? At least for now most of the webservers built to support HTTP/3 still fallback to HTTP/2 on request, go back to the known working congestion control of TCP that most of the internet and especially the web was built on top of.
I'm not a pessimist on QUIC, I think it has great potential. I also am not an optimist about it 100% replacing TCP in our near future, and maybe not even in our lifetime. As an application developer, it will be a great tool to have in your toolbelt as a "third" compromise option between TCP and UDP, but deciding between TCP and QUIC is probably going to be an application-by-application pros/cons list debate, at least in the short term and I think probably in the long term too.
IMO it's a structural problem that blocks a lot of innovation. The same thing happens when a popular open source project that's author led switches to an external maintainer. When the incentives to block innovation are stronger than the incentives to allow it, you get ossification.
If you approach to security is that only square tiles are allowed because your security framework is a square grid, and points just break your security model, maybe it was never a valid thing to model in the first place.
I'm not saying security should not exist, but to use an analogy the approach should be entirely different - we have security guards, less so fences, not because fences don't provide some security, but because the agent can make the proper decision, and a lot of these enterprise models are more akin to fences with a doorman, not an professional with a piece and training...