These DPI companies always make me smile because 20 years ago I was the inventor on this patent: https://patents.google.com/patent/US6182146B1/en It describes a way of doing DPI to identify protocols that are not running on standard ports. We used this for a protocol analysis product [0] that did network monitoring (for accounting purposes inside companies and led to companies discovering what people were mis-using their network connections for---hello PointCast[1]) and for prediction of network scaling needs. And all that was based on stuff I'd been doing from about 1984 [2].

Bottom line: scooping up packets is easy; encrypt your shit.

[0] https://www.cnet.com/au/news/tool-gauges-web-apps/

[1] https://en.wikipedia.org/wiki/PointCast_(dotcom)

[2] http://blog.jgc.org/2011/01/network-protocol-analysis-prior-...

Ponder: Is your available ISP speed restricted, through backdoor channels and red letters, to not overcome the throughput bandwidth of these devices?

Or, another way, is the maximum throughput of these monitoring setups limiting ISP maximum offered speeds in the countries that use them?

US, UK, Aus, Canada, the eyes, all have unusually low maximum consumer speeds vs. non implicated countries such as Japan, korea, even China, given the technology available today.

This article perfectly illustrates a major flaw in surveillance journalism.

As luck would have it, I'm pretty familiar with Endace --- or was, back in 2003-2005. I was at Arbor Networks then. Arbor does large-scale network instrumentation for anti-DDoS and performance monitoring. By the time I left, every major ISP in the world had their network instrumented with Arbor gear.

We'd had lots of conversations with Endace. We were as a firm extremely interested in any technology we could buy off the rack to get performant access to raw packets and telemetry data --- Arbor had no hardware engineers, and everything they shipped at the time shipped on COTS X86 rackmounts running OpenBSD. My point here is not just that there are multiple uses for the kind of stuff Endace makes, but also that I vividly remember Endace because very few companies made products in this space at all.

Obviously, any company that can facilitate efficient access to, storage of, and analysis of raw traffic data is going to have multiple markets to sell to. And we should not make apologies for companies that take the extra money --- sell their souls, so to speak --- by offering their products to facilitate dragnet surveillance. We would all do well to keep in mind that the problem with selling to this market is far worse than NSA's abuses, which are trivial compared to the abuses perpetrated by countries in the Middle East and Asia. Point being: packaging and selling for the global surveillance market is ethically hazardous in the extreme.

No, the problem here is that this kind of story is unintentionally deceptive about who the real enablers of large-scale surveillance are. They're not the dinky little company in New Zealand selling packet capture technology. They're the networking and database giants, the companies our parents automatically have their retirement accounts invested in because they're huge components of the stock market, who have entire teams of people, euphemistically named (maybe something like "public sector" or "APAC public sector" or "GSA" or "defense"), packaging and selling 8-9 figure "solutions" to government around the world. Compared the giants, Endace is a gnat. They're not the enablers. We know who the real enablers are.

You can tell, because of the article's lurid descriptions of Endace's major transactions with GCHQ --- the focus of the article. They've got smoking gun proof: invoices for $300,000 and $160,000. Or: less than SourceFire would have charged Chick-Fil-A† to install commercial Snort boxes.

† I have no idea if Chick-Fil-A was a SourceFire customer.

"[..]Alongside its government clients, Endace has many major corporate customers.

Endace’s sales lists include finance industry giants such as Morgan Stanley, Reuters, and Bank of America. [..]"

What for do finance companies need systems that intercept data?

Can someone ELI5 how they actually capture all these data ?

Are they basically cracking encryption ? I thought the kind of encryption provided by VPN services (256-bit AES/CBC) was strong enough? If that's what they do, aren't they violating privacy laws ? Aren't they breaching companies such as Google, Facebook etc... T&C ? Are they installing some kinds of trojans, keyloggers and stuff on 3rd parties computers ? Isn't what they are selling black hat hacking solutions ? Or are they only capturing clear traffic ? which is not necessarily very meaningful.

It says : "extract information about people’s usage of services such as Gmail, Hotmail, WhatsApp, and Facebook"

The latest terms and conditions you had to acknowledge recently to continue using watsapp (yes, I read them!) mentionned that they don't keep a record of the content being exchanged via watsapp. So, is watsapp lying ? Or what does this Endace system records ? Watsapp T&C also say that they use a strong encryption. so, FTW?

Gigamon, who has both an office in Milpitas and a government sales division (including for overseas governments), makes photonic tapping systems that are also pretty useful for this mass surveillance kind of stuff.

Five years ago, they had a box capable of handling 1 Tbps and assured us that bigger, beefier ones were coming. US DoD was a customer then and, I'm sure, still is.

Are we still safe using Tor?

[slightly offtopic] Does anyone have a driver for Endace DAG 9.2X and is willing to share? )