Noise Mapping with KSQL, a Raspberry Pi and a Software-Defined Radio

Noise Mapping with KSQL, a Raspberry Pi and a Software-Defined Radio(confluent.io)

115 points by h3throw 7 years ago | 30 comments

antirez 7 years ago |

<JOKE> As the author of dump1090 I'm going to change the license to require that it is always used together with Redis Streams and not with Kafka. </JOKE>

Topgamer7 7 years ago | |

Us hackernewsians have no humor. I'm taking this seriously and forking dump1090 and calling it dump1337

bigiain 7 years ago | | |

Bug Report: I'm not receiving any aircraft data on 1337MHz...

ihuman 7 years ago | |

OP's python script manually parses dump1090's output[0]. Is this the recommended way to get information with this level of detail from dump1090's output into another program? In the past I've used the SBS1 port because I didn't think to run dump1090 as a sub-process. Next time I use dump1090, I'd rather not use a SBS1 parser if I don't have to.

[0] https://github.com/saubury/plane-kafka/blob/master/raspberry...

antirez 7 years ago | | |

Both works but perhaps using TCP port 30001 makes more sense in the context of a serious project:

1. Simpler to upgrade to a distributed setup.

2. Format is stable but with enough pre-parsed information, like the coordinates which are hard to extract.

3. The process doing the data collection can avoid to die together with dump1090 in case of crashes or USB errors. You just need a script to restart dump1090, and the ability to reconnect in the other side.

However I don't remember exactly how the SBS1 port data are ready-to-use exactly, and if there is too much work to do compared to just parsing the raw text output of the program.

Anechoic 7 years ago |

I'd be curious to see how this compares with a real-time flight track/noise monitoring system ( https://webtrak.emsbk.com/lax4 at LAX for example).

jcims 7 years ago | |

That’s really cool! I wonder if you can see claps of thunder roll across the sensors. Would also be interesting to use this to detect planes that aren’t squitting ADSB.

whalesalad 7 years ago |

This is killer. I’ve been wanting to do something similar to figure out why I have Bluetooth and WiFi connectivity issues at random parts throughout the day. I’m thinking there is some kind of interference and would like to prove that.

This post will serve as a model for that. I’ve had an SDR and a few old Pis lying around for years... time to dust them off.

penagwin 7 years ago | |

I've wanted to do related things with wifi/bluetooth, just a heads up though that the typical rtl-sdr you find for 20/30$ can't see the 2.4ghz spectrum which both wifi/bluetooth use (typically)

simcop2387 7 years ago | | |

May not need to worry about that, the interference can be in other bands if it's strong enough to blind the other receivers (There's no such thing as a perfect filter). That said I'd expect that it's going to be someone's microwave, or similar device, that's just flooding the spectrum around 2Ghz also.

Cynddl 7 years ago |

> The Raspberry Pi is a little underpowered for serious computation, so I need to offload processing to my local Kafka cluster.

Couldn't this be done on the Raspberry Pi with a O(n) search in the OpenFlights data? Probably less sexy, but a simple grep with location and time range should be enough.

jhpankow 7 years ago | |

That wouldn't show you the real power of KSQL though.

okigan 7 years ago | | |

hmm, still seems like odd usage.

given that in the end the query goes for specific time window, wouldn't "regular" sql work fine (vs event processing) ?

dingaling 7 years ago | |

Or with an old surplus Core2 laptop.

Using an RPi with an SDR dongle is already pushing its limits, let alone piling on more load. I can't understand why he'd use it other than for buzzword recognition.

Of course the lower-effort solution would have beem simply to run an FR24 playback for his location at 06:00...

JaimeThompson 7 years ago | | |

Often it is easier to put the RPi, SDR, and antenna in a good place to receive signals then it is to run high quality antenna cable inside. It also helps isolate your main network from lightning strikes.

towawayzone 7 years ago |

I know I'm completely missing the point, but how does knowing which plane is upsetting your cat help with the situation? Are you going to report them to the FAA or something?

P.S. Coming up with examples for things in computer science that aren't absurd is hard. I had to think of an example when I was explaining junction tables for many-to-many relationships to someone the other day. After a minute or so of thinking, I just fell back on that old standard of books and authors.

minitoar 7 years ago |

At first I thought this was about mapping noise around the San Carlos airport, identifier "KSQL".

tjohns 7 years ago | |

Same. I skimmed the article, and was really confused I saw the author talking about 787s and A380s, because there's no way one of those would fit on the runway at KSQL.

nimish 7 years ago |

> The Raspberry Pi is a little underpowered for serious computation, so I need to offload processing to my local Kafka cluster.

The RPI includes purpose built hardware for DSP in the VideoCore so, probably not.

Poe's law in action I suppose.

I'm all for big data but this is a case of nuking a fly.

inetknght 7 years ago | |

My understanding was that everything on the Pi except for the video processor was open source. Am I mistaken? Or, are you talking of something different? Or, do you know of an open source library or demonstration of use of the VideoCore for DSP?

Rebelgecko 7 years ago | | |

There's docs and source for some aspects of the VideoCore. I think someone put together an OpenCL implementation, but I don't know how well it works

Edit: That said, I think any sort of OpenCL/GPGPU solution is overkill for this problem. I run Flightaware's dump1090 fork on my Pi and it seems to only use around 30% of one CPU core.

3xblah 7 years ago |

When I hear the term "ksql" I think of Arthur Whitney's ksql and the early days of kdb, which dates back to the 90's.

jo-wol 7 years ago |

Everything is great in this post except ksql license.

halbritt 7 years ago | |

The Confluent Community License?

legionof7 7 years ago |

I've been seeing so many Raspberry Pi projects on HN lately. It's great! I just got a Zero W and have a B+ lying around that I don't know what to do with.