Serving 200M requests per day with a CGI-bin(jacob.gold) |
Serving 200M requests per day with a CGI-bin(jacob.gold) |
People/orgs do tend to get kind of addicted to certain technologies that can interact poorly with the one-shot model, though. E.g., high start up cost Python interpreters with a lot of imports are still pretty slow, and people get addicted to that ecosystem and so need multi-shot/persistent alternatives.
The one-shot model in early HTTP was itself a pendulum swing from other concerns, e.g. ftp servers not having enough RAM for 100s of long-lived, often mostly idle logins.
No lingering state, very easy to dump a core and debug, nice mostly-linear request model (no callback chains, etc.) and trivially easy to scale. You're just reading from stdin and writing to stdout. Glorious. Websockets adds a bit of complexity but almost none.
The big change in how we build things was the rise of java. Java was too big, too bloated, too slow, etc. so people rapidly moved into multi-threaded application servers, all to avoid the cost of fork() and the dangers of C. We can Marie Kondo this shit and get back to things that are simple if we want to.
I don't even like Rust and this sounds like heaven to me. Maybe someone will come up with a way to make writing the kind of web-tier backend code in Rust easy by hiding a lot of the tediousness and/or complexity in a way that makes this appealing to node/js, php and python programmers.
Part of the Java rise was C/C++ being error prone and syntax similarity with such, but this was surely intermingled with a full scale marketing assault by Sun Microsystems who at the time had big multi-socket SMP servers they wanted to sell with Solaris/etc. and part of that was the Solaris/Java threading. Really for a decade or two prior to that the focus was on true MMU-based hardware-enforced isolation with OS kernel clean-up (more like CHERI these days) not the compiler-enforced stuff like Rust does.
I think you could have something more ergonomic than Perl/Python ever was and as practically fast as C/Rust with Nim (https://nim-lang.org/). E.g., I just copied that guy's benchmark with a Nim stdlib std/cgi and got over 275M CGI/day to localhost on a 2016 CPU doing only 2 requesters & 2 http server threads. With some nice DSL easily written if you don't like any current ones you could get the "coding overhead" down to a tiny footprint. In fairness I did zero SQLite whatever, but also he was using a computer over 4x bigger and probably a GHz faster with some IPC lift as well. So, IF you had the network bandwidth (hint - usually you don't!), you could probably support billions of hits/day off a single server.
To head off some lazy complaints, GC is just not an issue with a single threaded Nim program whose lifetime is hoped/expected to be short anyway. In many cases (just as with CLI utilities!) you could probably just let the OS reap memory, but, of course, it always "all depends" on a lot of context. Nim does reference counting anyway whereas most "fighting the GC" is actually fighting a "separate GC thread" (Java again, Go, D, etc.) trashing CPU caches or consuming DIMM bandwidth and so on. For this use, you probably would care more about a statically linked binary so you don't pay ld.so shared library set up overhead on every `exec`.
There's websocketd, which makes your program just a matter of reading from stdin and writing stdout. http://websocketd.com/
We invented PHP and FastCGI mainly to get away from the performance hit of starting a new process just to handle a web request!
It was only a few years ago that I realized that modern hardware means that it really isn't prohibitively expensive to do that any more - this benchmark gets to 2,000/requests a second, and if you can even get to a few hundred requests a second it's easy enough to scale across multiple instances these days.
I have seen AWS Lambda described as the CGI model reborn and that's a pretty fair analogy.
With this hardware, if you reach for Kestrel you can easily do a few trillion requests per day. The development experience would be nearly identical - You can leverage the string interpolation operator for a PHP-like experience. LINQ and String.Join() open the door to some very terse HTML template syntax for tables and other nested elements.
The hard part is knowing how to avoid certain landmines in the ecosystem (MVC/Blazor/EF/etc.). The whole thing can live in one top-level program file that is ran on the CLI, but you need to know the magic keywords - "Minimal APIs" - or you will find yourself in the middle of the wrong documentation.
I just did a `time perl -e ''` (starting perl, executing an empty program), it took 5ms. 33ms with python3, 77ms with ruby.
Besides your two slower examples, Julia and Java VMs and else thread PHP also have really big start up times. As I said up top, people just get addicted to "big environments". Lisp culture would do that with images and this is part of where the "emacs is bloated" meme came from.
Anyway, at the time getline wasn't even standardized (that was 2008 POSIX - and still not in Windows; facepalm emoji), but you could write a pretty slick little library for CGI in a few hundred..few thou' lines of C. Someone surely even did.
But things go by "reputation" and people learn what their friends tell them to, by and large. So, CGI was absolutely the thing that made the mid to late 90s "Perl's moment".
I wouldn't be surprised to learn that Perl startup time has drifted. Need benchmark.
One shared JVM for maximum performance!
It can also share db connection pools, caches, etc. among those applications!
Wow!
[0]: https://www.ibrahimdiallo.com/reqvis
Note: works best on desktop browsers for now.
I've been asked about architecture of a stocks ticker that would serve millions of clients to show them on their phone the current stock price. First thought was streams, Kafka, pubsub etc but then I came up with static files on a server.
I wonder how much would it cost though
If all you need is to return rarely-changing data, especially without wasting time on authorization, you can easily approach the limits of your NIC.
The 3700X is an 8 Core Zen 2 CPU. Or about 150 RPS per vCPU. By EOY we will have 256 Core Zen 6c EPYC, on a Dual Socket that is 512 Core or 1024 Thread. 153,600 RPS.
And in the old days a RPS is a single pageview, which isn't the case in the modern world.
CPU Core still have a healthy 5 - 10 years cost reduction roadmap. I wonder if CGI will make a comeback someday.
- Keep-alive/pooled connections to remote services can significantly reduce average latency for making those calls.
- In-memory caches that allow amortizing repeated lookups across requests.
Just those two alone mean that a serious high performance server probably couldn’t get away with just CGI even ignoring startup time.
This is all internal use though, I don’t need to scale to hundreds of concurrent users let alone thousands. Apache and cgi bin is fine.
https://github.com/Jacob2161/cgi-bin/blob/main/gohttpd/main....
See as "Benchmarking (writes|reads) using Go net/http"
It was faster but not by very much. Running CGI programs is just forking processes, so Apache's forking model works just about as well as anything else.
If this refers to https://github.com/six-ddc/plow then -- oops! lots of issues in that repo, no tests, etc. etc.
The results in the README are also pretty clearly unsound! In both scenarios, writes were faster than reads?
_edit_: I guess because the writes all returned 3xx, oops again!
Probably don't take this article's claims at face value...
plow may not be the best tool that exists but it does make concurrent HTTP requests and generate metrics for them successfully.
The writes returned 3xx because the handler returns a redirect, so this is expected.
HTTP load testing is a problem area that is much more subtle than it seems. I've no doubt that plow does what you're saying here, but, without any tests whatsoever, I have serious doubts that it does so correctly, particularly if/when the load test starts bumping up against any of the numerous bottlenecks that can affect results and their measurements.
> The writes returned 3xx because the handler returns a redirect, so this is expected.
Yeah, but, unless `plow` actually follows that redirect, it's not really measuring the actual end-to-end latency, and further the guestbook.cgi returns 301 See Other for both valid requests (that performed a write) and invalid requests (that didn't).
Could someone please explain what the better options are?
At the very least go for FastCGI, for christ’s sake…
The "performance hit of starting a new process" is bigger if the process is a dynamically-linked php interpreter with gobs of shared libraries to load, and some source file, reading parsing compiling whatever, and not just by a little bit, always has been, so what the author is doing using go, I think, would still have been competitive 25 years ago if go had been around 25 years ago.
Opening an SQLite database is probably (surprisingly?) competitive to passing a few sockets through a context switch, across all server(ish) CPUS of this era and that, but both are much faster than opening a socket and authenticating to a remote mysql process, and programs that are not guestbook.cgi often have many more resource acquisitions which is why I think FastCGI is still pretty good for new applications today.
Yes! Note that the author is using a technology that wasn't available when I too was writing cgi_bin programs in the 00's: Go. It produces AOT compiled executables but is also significantly easier to develop in and safer than trying to do the same with C/C++ in the 00's. Back then we tended to use Perl (now basically dead). Perl and Python would incur significant interpreter startup and compilation costs. Java was often worse in practice.
> I have seen AWS Lambda described as the CGI model reborn and that's a pretty fair analogy.
Yes, it's almost exactly identical to managed FastCGI. We're back to the challenges of deployment: can't we just upload and run an executable? But of course so many technologies make things much, much more complicated than that.
But imagine having to host 50 small applications each serving a couple of hundreds requests per day. In that case, the memory overhead of Tomcat with 50 war files is much bigger than a simple Apache/Nginx server with a CGI script.
Not saying that can't happen with CGI, but since Tomcat is a shared environment, it's much more susceptible to it.
This is why shared, public Tomcat hosting never became popular compared to shared CGI hosting. A rogue CGI program can be managed by the host accounting subsystem (say, it runs too long, takes up too much memory, etc.), plus all of the other guards that can be put on processes.
The efficiency of CGI, specifically for compiled executables, is that the code segments are shared in virtual memory, so forking a new one can be quite cheap. While forking a new Perl or PHP process shares that, they still need to repeatedly go through the parsing phase.
The middle ground of "p-code" can work well, as those files are also shared in the buffer cache. The underlying runtime can map the p-code files into the process, and those are shared across instances also.
So, the fork startup time, while certainly not zero, can be quite efficient.
Because a lot of production software is half-baked. If you have to hand over an application to an operations team you need documentation, instrumentation, useful logging, error handling and a ton of other things. Instead software is now stuffed into containers that never receive security updates, because containers make things secure apparently. Then the developers can just dump whatever works into a container and hide the details.
To be fair most of that software is also way more complex today. There are a ton of dependencies and integrations and keeping track of them is a lot of work.
I did work with an old school C programmer that complained that a system we deployed was a ~2GB war file, running on Tomcat and requiring at least 8GB of memory and still crashed constantly. He had on multiple occasions offered to rewrite the how thing in C, which he figured would be <1MB and requiring at most 50MB of RAM to run. Sadly the customer never agreed, I would have loved to see if it had worked out as he predicted.
Not to mention endless frustration any upgrades would cause since we had to get all teams onboard with "Hey, we are upgrading PHP 5, you ready?" and there was always that abandoned app that couldn't be shut down because $BusinessReasons.
Containers have greatly helped with those frustration points and languages self-hosting HTTP have really made stuff vastly better for us Ops folks.
I develop and deploy Flask + Rails + Django apps regularly and the deploy process is the same few Docker Compose commands. All of the images are stored the same with only tiny differences in the Dockerfile itself.
It has been a tried and proven model for ~10 years. The core fundamentals have held up, there's new features but when I look at Dockerfiles I've written in 2015 vs today you can still see a lot of common ideas.
> Yeah, but, unless `plow` actually follows that redirect, it's not really measuring the actual end-to-end latency...
Not in this case, since the purpose here was to measure the POST (write) not the subsequent GET (read) that a browser would do after the redirect.
Some perl5 lover should take the time to compile all those 5.6 to 5.42 versions on the same host OS/CPU and do a performance comparison and create a nice chart for the world to trap and maybe correct such performance regressions. I just tried getting 5.8.9 to compile on modern Linux with gcc-15, and it seemed like a real PITA. (Earlier didn't even ./Configure -des right.)
zsh# rei=(chrt 99 taskset -c 2-3 env -i HOME=$HOME PATH=/bin)
zsh# tim=(-k2 -o14 -n14 -m14)
# Set 1
zsh# $rei tim $tim './perl-5.8.9</n' 'perl-5.42.0</n'
218.5 +- 1.9 μs (AlreadySubtracted)Overhead
626 +- 14 μs ./perl-5.8.9</n
953.5 +- 8.6 μs perl-5.42.0</n
# Set 2
zsh# $rei tim $tim './perl-5.8.9</n' 'perl-5.42.0</n'
247 +- 15 μs (AlreadySubtracted)Overhead
633 +- 26 μs ./perl-5.8.9</n
989 +- 25 μs perl-5.42.0</n
# Consistency check - well within 2 sigma for both
zsh# a (989 +- 25)-(953.5 +- 8.6)
36 +- 26
zsh# a (626 +- 14)-(633 +- 26)
-7 +- 30
# Final report; weighted means to maximize precision
zsh# a mean(989 +- 25,953.5 +- 8.6)/mean(626 +- 14,633 +- 26)
1.525 +- 0.033
By the way if you're using nginx, then instead of FastCGI you might prefer SCGI, which does one connection per request and no multiplexing, so it's much simpler.
I mean, it's not like people ran Internet servers on such vastly different CPUs/OSes or that diversity was such a disadvantage. DEC Alpha was probably the most different for its 64-bitness, but I ran all those open source Linux/C things on that by 1996..97. But we may just have to agree to disagree that it made a lot of sense for that reason. I have disagreements with several high profile SiValley "choices" and I know I'm a little weird.
Anyway, I don't mean to be arbitrarily disputatious. Focusing on what we do agree on, I agree 100% early Java stdlib's being bigger than C/C++ & early STL/template awfulness was a huge effect. :-) C++ like keywords and lexical sensibilities mattered, too. PLang researchers joked upon Java's success that C's replacement sure had to "look like C". But I think programmers having all their lib needs met with very little work matters even more and network-first package managers were just getting going. A perceived-as-good stdlib absolutely helps Go even today. Human network effects are a very real driver even among very talented engineers.
Maybe since CTAN/CPAN?, that often comes more from popularity and the ecosystem than "what's in the stdlib". Even before then there was netlib/fortran algorithm distribution, though. Node/Rust/Python worlds today show this.
How to drive popularity in a market of ideas competing for it is hard/finicky or the biggest marketing budget would always win and people could also always just "buy reputation" which empirically does not happen (though it sure happens sometimes which I guess shows ad spend is not wasted). Even so, "free advertising", "ecosystem builds going exponential", etc. - these are just tricky to induce.
The indisputable elephant in the room is path dependence. Fortran, still somewhat reflective of people "stacking punch card decks" to "link" programs in the 1950s, is still used by much modern scientific research either directly or indirectly. Folks were literally just figuring out what a PLang should be and how interacting with these new things called "computers" might work.
But path dependence is everywhere all around us.. in institutions, traditions, and technology. It's all really a big Humanity Complete discussion that spirals into a cluster of Wicked Problems. Happens so fast on so many topics. :-) If you happen to make something that catches on, let us hope you didn't make too many mistakes that get frozen in! Cheers!
I’ve never used Fortran.. I started out with basic and wanted to do C/C++ because of gamedev. I loved the syntax. Perhaps bc it made me feel smart. The C-style syntax is also big part what made Java popular.
However, I think more and more than the popularity is because Java was the de facto standard to teach at universities. The reason for that was that it was a very clear language, and easy to compile, unlike C, which has so many behavioral quirks, and ofc memory management.
These days Python is being taught in both CS and statistics, which creates a broader market and broader usage spectrum.
Ruby, in my opinion, got big because of rails, but loved because of the stdlib
Look at qmail, which has the best track record of any piece of software I am aware of in wide distribution, and it was written in C.
Also: Memory leaks go away when you exit(), so they are actually more common in dynamic languages in my experience, although they manifest as fragmentation that the interpreter simply lacks the ability to do anything out.
Buffer overflows seem pretty common to people who do a lot of dynamic memory allocation: I would recommend not doing that in response to user-input.
The result is that your C-based guestbook CGI is probably written very differently than a PHP-based guestbook. Mine basically just wrote to a logfile because since 2.6.35 we have been able to easily make a 1mb PIPE_BUF and get lock-free stores with no synchronisation and trivial recovery, and thus know exactly where each post began and end. I'm not sure I want more than 1mb of user input back in those days, but the design made me very confident there were no memory leaks or buffer overflows in what was like 5 system calls. No libraries.
You could do this.
You can do this.
But you want more? That C-based guestbook also only ever needs to write to one file, so permissions could be (carefully) arranged to make that the only file it can write to. A PHP-based guestbook needs read (and possibly write-access) to lots of files. Some of those things can be shared objects. It is so much easier to secure a single static binary than a dynamic language with dynamic loading that if you actually care about security, you could focus on how to make those static binaries easier.
I had no real debugging environment. I was probably writing all my code in vi and then just compiling and deploying. I guarantee there were buffer overflows and off-by-ones etc.
Web app code was so simple back then, though. The most complex one I wrote was a webmail app, which I was so pleased with, and then HoTMaiL was released three weeks later, with this awesome logo:
https://tenor.com/view/hotmail-outlook-microsoft-outlookcom-...
You can use it with any language that can read stdin and write stdout. Yes, printing and reading.
Memory leaks are considered a feature in short lived programs, since not freeing memory in favor of relying on the kernel to free it at program exit lets them run faster.
I used Perl instead, which worked way better in that regard (+ taint based security was welcome in handling untrusted user input), and an enormous (for the time) CPAN ecosystem, but had other problems.
Python web ecosystem was a mess, so PHP3 it was (ah the “good” ol days of mysql_real_escape_string()) … until some enterprising individuals wrote Django and I happily switched. Thank you :)
One additional bit of context is the person you’re replying to, simonw, is also the creator of Django, which at the time was the world’s defacto standard Python web framework, and was created at a time (2005) that long predates either Go (2009) or Rust (2012).
We'd use this approach not just for webapps, but versions of applications we'd build in house, bundles of scripts, whatever.
I've always been a little surprised Nim wasn't more popular with its Python like syntax, Lisp like meta-power, C-like speed and usually automatic memory management. Ah well.