A Posix Queue Implementation

A Posix Queue Implementation(zinascii.com)

88 points by rzezeski 11 years ago | 13 comments

bcantrill 11 years ago |

This is a great piece, and it reminds how rare it is to have excellent expository systems writing. It's certainly fitting that Ryan cites the late Richard Stevens, who was the exemplar of such writing. Those who like Stevens and Ryan's piece should also look for work by Elliott Organick: he must be considered the pioneer of expository systems monographs, and while his books are very hard to find (and the systems they describe long-since powered off), they remain rewarding reads.

On a more personal note, it was fun to reminisce about code that I hadn't been in in a long time -- I had actually forgotten how much surgery I did on the message queue code back in 1997[1], and I had the opportunity to chuckle at some of the comments made by my 23-year-old self.[2] I also marveled at how little this software has changed in the nearly two decades since; I've written about this before[3], but it's always inspiring to be reminded that we in software have the luxury of creating useful things that do not wear out -- may we only have more writing like Ryan's to learn about them!

[1] https://github.com/illumos/illumos-gate/blob/a9e987e05eeb8cc...

[2] https://github.com/illumos/illumos-gate/blob/a9e987e05eeb8cc...

[3] http://dtrace.org/blogs/bmc/2004/08/28/the-economics-of-soft...

tedunangst 11 years ago |

Does anything use posix queues? Lack of support has never been an issue for openbsd afaik. Looks like somebody snuck it into FreeBSD, which I wasn't aware of until I just looked it up. Seems like a forgotten feature.

rdtsc 11 years ago | |

I thought about them a few times wanted to use them. But every time I thought "Hmm, wonder if anyone else is using them... I better stay away if I hit some bug or untested corner case".

jzwinck 11 years ago | | |

I used POSIX message queues in production once. They worked fine, as a replacement for TCP when on the same machine, with reduced latency (roughly 75%). But eventually I switched to Unix Domain Sockets which offered similar good performance with a more familiar operating model (e.g. no worries about cleaning up old queues, and it's just another type of socket). This was on Linux 2.6.32-ish, and the POSIX MQ implementation had no bugs that I noticed over some millions of messages. Which is more than I can say of the UDP implementation.

chubot 11 years ago |

Is this secure? That is, can you receive messages from an untrusted process without worrying that it will somehow crash or corrupt your process?

There are two processes mmap'ing the same memory, and operating on it with a mutex. It seems like the one side has to perform elaborate checks to make sure the other side is obeying the protocol. Or am I mistaken?

Animats 11 years ago |

Compare QNX message passing, which has a much simpler API and runs faster.

The UNIX/Linux world has never gotten message passing right. It's inevitably built on top of files, sockets, or shared memory, which adds complexity and reduces speed.

bcantrill 11 years ago | |

Ironically, in addition to working on the code described by the linked piece, I also implemented POSIX.1b message queues for QNX. (Namely, for QNX4 in 1995.) While I'm not necessarily in love with .1b message queues (see the commit referred to by my earlier comment, above), one cannot compare blocking message passing (Send/Receive/Reply) to message queueing -- they have entirely different semantics. (The QNX primitives are more analogous to doors.[1]) I would also caution about making blanket statements about performance with respect to different IPC mechanisms on entirely different systems: aside from the lack of quantification in your assertion, different IPC mechanisms often solve slightly different problems, and each must be considered within the context of its own system.

[1] http://en.wikipedia.org/wiki/Doors_(computing)

skywhopper 11 years ago | |

Using files and sockets as common abstractions for a variety of underlying mechanisms is one of the most powerful things about UNIX/Linux. Yes, there are tradeoffs, but in return, it means you can interoperate with a tremendous number of utilities and tools (which don't even need to know anything about the underlying mechanism) in unplanned and unexpected ways... for free.

frou_dh 11 years ago | | |

Trigger Warning: Memories of reading about Plan 9

pjmlp 11 years ago | | |

As long as you happen to understand what goes thru the pipe, which isn't always the case.

tedunangst 11 years ago | |

Building message queues on top of shared memory seems like a pretty natural high performance solution. What would you suggest?

umanwizard 11 years ago | |

FreeBSD pipes, for one, haven't been based on sockets in over ten years.

clumsysmurf 11 years ago |

This reminds me of a Java project called Chronicle started by Peter Lawrey

https://github.com/OpenHFT/Chronicle-Queue