eBPF – The Future of Networking and Security

eBPF – The Future of Networking and Security(cilium.io)

195 points by genbit 5 years ago | 36 comments

The future ought to be capabilities. All this policy scripting stuff is just drudgery make-work that gets us nowhere.

alexgartrell 5 years ago | |

Can you provide more context on why you feel that's true (or even possible)?

For the last few years, I managed the Container Runtime group at Facebook. My experience has been:

1. `if (has_capability(..., X)) { ... }` gets put into code pretty haphazardly in a way that's not necessarily super well structured. Once it's there, it's ABI, and you're screwed if you want to iterate on it. That's why cap_sys_admin is /almost/ root.

2. If you wanted to do the right thing from the jump (e.g. for bpf itself), you'd have to add a new capability. This is a heavy lift for something that might not actually get any traction. It requires changing a bunch of common tools, and you likely end up breaking a bunch of applications.

3. Debugging capability failures is a pain in the ass. We ended up building and deploying capability tracing infrastructure just to figure out what people are actually using.

4. For gradual roll outs of enforcement/changes, you need the flexibility to warn first, enforce second. We did large scale monitoring of all such changes to make sure we didn't break the workloads.

5. Even if you nail all of the above, the ability to make finer-than-capability-grained decisions (i.e. binding to port 20 or 80 is okay but not port 22) is really valuable.

I'm all for kernel abstractions that just work and solve all problems for all people, but I think the overwhelming trend has been towards kernel interfaces that provide a lot of flexibility and then more opinionated libraries/tools that kind of let us have our cake and eat it to (io_uring => liburing, bpf => libbpf, btrfs => btrfstools).

wmf 5 years ago | | |

Are we talking about POSIX capabilities or object capabilities?

codys 5 years ago | | |

What POSIX and the linux kernel calls "capabilities" unfortunately result in quite a bit of confusion, which I believe is the cause of your post. POSIX capabilities bear little resemblance to actual capability based security (where a capability is a send/recv-able token that references an object and a set of rights for interacting with that object).

tptacek 5 years ago | |

BPF wasn't originally conceived of as a reference monitor or ACL system; in fact, originally, it was believed that operating systems would use BPF-style packet filters to do pretty much all their demuxing.

Ericson2314 5 years ago | | |

That's all true. I'm worried about what people will do with this stuff in practice (more rope to hang themselves) not eBPF fundamentally is.

layoutIfNeeded 5 years ago | | |

Are you referring to STREAMS? https://en.m.wikipedia.org/wiki/STREAMS

trasz 5 years ago | | |

AFAIK BPF wasn’t conceived as anything security-related, it was just an optimization.

ryanmarsh 5 years ago |

I'm not an expert in BPF by any means. My gut tells me that the hype of eBPF is an example of Hyrum's law. That is, eBPF will be leveraged beyond its design intent, as an in-kernel JIT engine. This is more a comment on human nature than the technology itself.

joestringer 5 years ago | |

Even looking at the original BPF which focused on filtering packets as they are forwarded to userspace (think tcpdump)[1] and looking at the extensions that eBPF provides on top to hook into various subsystems[2,3], it's clear that this is going far beyond the use cases originally envisioned. I'd love to see an eBPF paper to follow up / contrast with the '93 USENIX BPF paper.

[1]: https://www.tcpdump.org/papers/bpf-usenix93.pdf

[2]: https://ebpf.io/what-is-ebpf#hook-overview

[3]: http://www.brendangregg.com/BPF/bpf_performance_tools_book.p...

tptacek 5 years ago | | |

FWIW: I just wrote a long-ish post on the history from BPF (and before BPF) to eBPF and XDP:

https://fly.io/blog/bpf-xdp-packet-filters-and-udp/

An interesting fact is that packet filtering as a problem domain has been dominated by in-kernel virtual machines going back into the 1980s; it's an idea that comes all the way from Xerox.

tgraf 5 years ago | | |

The shift from BPF to eBPF was less of an evolutionary step as the name might indicate. The overlap with the name BPF is primarily due to the requirement for eBPF to be a superset of BPF in order to avoid having to maintain two virtual machines long-term. This was one of the conditions for eBPF to be merged and in that context, the name eBPF made sense.

tptacek 5 years ago | |

Easy to predict something that's already happening. :)

https://github.com/xdp-project/xdp-tutorial

It's a good thing, I think! Compared to loading new unmanaged C code into the kernel, BPF is a really nice way to add functionality to Linux.

tgraf 5 years ago |

Disclaimer: I wrote the post.

Happy to answer any questions.

_vufv 5 years ago | |

First of all, congrats. The tech is great and I hope you'll be able to make a company around it.

As for the question: How are you looking to make money?

miohtama 5 years ago |

eBPF is also used high throughput blockchain, Solana

https://github.com/solana-labs/rbpf

Unlike more common Rust + LLVM + WASM toolchain, Solana smart contracts use Rust + LLVM + eBPF.

chc4 5 years ago | |

Solana uses a custom Rust re-implementation of a custom C re-implementation of the Linux BPF VM for what appears to be licensing reasons. Notably, it's jitting all bytecode without a verifier or emitting runtime bounds checks[0]. I suspect you can pop a shell on every single computer on their testnet somewhere between "trivially" and "extremely trivially".

They appear to be running some kind of "open security test"[1] but are only paying out their own imaginary funny money. I'd suggest you run for the hills as fast as you can instead of considering Solana.

0: https://github.com/solana-labs/rbpf/blob/f7007d6ae8728e61401... 1: https://forums.solana.com/t/tour-de-sol-stage-1-details/317

miohtama 5 years ago | | |

Interesting. I am not sure if your comment Without a verifier make sense. Because AFAIK you need to verify the contract only once, when it is deployed. Not every time it is invoked. Verifying a contract should be super cheap compared to executing it, unless eBPF verification is somehow super expensive.

ninjha 5 years ago |

I was under the impression that Cilium was one of the more common choices for Kubernetes CNI but judging by the other comments... maybe not?

We’re currently moving to Kubernetes for our infrastructure at the Berkeley OCF (https://ocf.berkeley.edu/), and picked Cilium for all the networking things.

It’s good to see that there’s a company backing it now!