Real time operating systems. Less fancy than they sound but the devil is in the details. Robots need things to happen at a certain speed and at the right time so we have a type of scheduler (that can be patched into the linux kernel) that sacrifices absolute throughput to try and guarantee tasks start inside a particular window. Funny enough, if you've done game development and recognize that everything needs to happen inside 1/60th of a second or better, you know some of the hard parts here.
Memory mapped addresses. C is scary but ultimately fairly simple. Once you get the hang of doing silly things with pointers and arrays, the next step is dealing with microcontrollers. You probably wonder how they do anything without an operating system though, and the answer is memory mapped IO. They have a fully flat memory space, starting with 0x0 and going up from there. That space usually contains basically everything from your stack, heap, flash storage, and all the peripherals like GPIO, I2C, SPI, serial, and so on. You can literally do things like int* x = 0x12345678; *x = 0x1; to turn on an LED because the device is listening for changes to that address to set the output state.
There's a ton of other stuff, but these are the gateways to understanding the space you're dealing with at a basic level.
I remember reading a book that explains the low level Arduino schematic paired with C code that does exactly what you are describing. Is there any such book for modern microcontrollers?
I'm only half kidding, the programming manuals are pretty good these days. Honestly there's more in common with the arduino and a modern uC than there is different. If you want to use the knowledge in anger you need to learn linker scripts, makefiles, gcc, gdb and jtag so you can deploy and debug code on boards that don't have self-flashing bootloaders and FTDI baked in. There's also some tribal knowledge about how ISRs work like keeping ISRs as short as possible (no print statements) but I don't know what books cover that.
* Agile for software-hardware is hard if not impossible. For software it can be reasonable, but it’s really hard to iterate on both hardware and software at the same time. It’s easier if the hardware designed is locked or the process involves waterfall on the hardware side. * I often found that people that come from pure software to robotics don’t have control experience and something that can be easily solved with a PID controller end up being a custom code that ends up being way more complicated than needed.
If you have any custom hardware, you are basically stuck with a turn time of at minimum a day or two for any changes (often more, weeks + for new PCBs is common if you're not throwing huge amounts of money at people).
In this context, any process that depends on rapid small iterations is basically impossible, because each iteration just takes too much time.
Pieces of code interact more heavily than a linux machine. Testing requires more hand-holding/baby-sitting. Cross-platform architectures don't scale down well. There are many types of comms busses with no/few standard embeddings.
Some teams know how to make it a lot easier. Some CTOs know this, but most find out the hard way. Embedded practices lag webdev by 5-10 or more years bc they were good enough for a long time or for small projects. Expectations are rising, but there is less leverage than adtech so salaries are ok but not explosive.
Then once you know how it works and have done most of your optimizations, you can trim down the BOM and get rid of hardware that proved useless. You'd probably want another round of tuning at this point, but at least you know basically how it works, and have tried out many different combinations of hardware in your software iterations.
But quite a lot of CS people -- well at least I hope it is not just me -- end up on the web development trajectory despite or because of having less than good enough understanding of e.g. trigonometry, geometry and calculus... all of which start to matter a lot when you are starting to make things; especially things that move and consume electricity.
I am a good programmer but a weak mathematician. I have a 3D printer now, and I'm starting to learn FreeCAD and microcontroller stuff actively rather than just reading about it, and run up against my maths weakness all the time.
I have recently discovered the three books Joan Horvath and Rich Cameron wrote:
https://www.makershed.com/products/make-trigonometry-print
So I do physical things in my personal projects and then come to realize that the potential return on investment (time spent vs. expected money) would be peanuts in comparison to pure software. I would love to know of hardware related work that has decent margins. Or at least enough margins to justify paying a good salary to enough engineers to sustain them.
This cannot be said enough, IMO. The web software world's endless quest for the shallow and lucrative is exhausting.
My own shitty first foray into robotics is an RPi that I can talk to [2].
If you didn't see "the coolest robot I've ever built" you gotta watch that... so inspiring [3]
Latent Space has a robotics demo day coming up, pretty curious to see what comes out of that [4]
Some stream-of-conscious thoughts about why I'm drawn to robotics:
* The maker / hacker / homebrew communities that are basically just using robots to express art. Maker Faire, Burning Man, etc.
* The satisfaction of writing code and seeing something physical happen. Last week I was trying to figure out how to get a shitty third party Amazon robot hat [5] to actually do something useful so I was iterating through the GPIOs and I somehow made it actually smoke. I'm weirdly proud of messing up so badly that my hardware actually smoked!
* The joy of demystifying hardware and learning all the layers of abstraction just within hardware
[2] https://www.biodigitaljazz.net/blog/STTTGTS.html
[3] https://news.ycombinator.com/item?id=38162881
[4] https://lu.ma/latent-space-final-frontiers
[5] A Xmas present from my sweet wife, really touching that she's encouraging me to actually pursue my interest in robotics
source: I work in robotics. AMA.
That may be the most number of times I've typed "etc." in a paragraph.
You just need to pop in 3x18650 batteries in the chassis and connect a Jetson nano to the UART pins. It has an ESP32 inside, so you can also program that, but it comes programmed out of the box.
You can control it with this python library https://github.com/msanterre/wave_rover_serial
Isaac SIM is a terrific simulator, you won't get much better.
https://www.allesblinkt.com/projects/round-about-four-dimens...
Hope this is not considered too off-topic here.
That cube is amazing. Thanks for sharing. The aesthetic is so polished. Some true 21st century art right there.
This made the rounds a few months ago but worth posting here again. So inspiring. https://www.youtube.com/watch?v=bO-DWWFolPw
The article suggests playing around with a Raspberry Pi Pico, but this is a bit bare bones. Are there any kits, or entry level programmable drones on the market with a reasonable toolchain to program sequences?
https://www.bitcraze.io/products/crazyflie-2-1/
https://www.bitcraze.io/documentation/repository/
It was mentioned in this IEEE Spectrum article: https://spectrum.ieee.org/drone-quadrotor-2667196800
I have looked at the Ryze Tello [0] before, and it seems like a decent entry-level device, but I'm not sure whether it's actually a huge pain to develop for. I'd like something to be extendable _like_ a Pico, for instance if I want to install additional sensors, but all of these platforms are quite foreign to me.
https://www.coppeliarobotics.com/
But take your pick there are many to choose from.
https://www.sciencedirect.com/science/article/abs/pii/S15691...
I think it was an issue from 2022 or 2023 (not enough energy currently to track down the issue in their catalog)
Here's an article on their site about the topic:
A lot of SEs leave industries like semiconductors for web dev jobs that pay waay better
The components are much better. Batteries are better. Motors are better. Radio communications work. Cameras are cheap. Short-range LIDAR is affordable. Navigation systems work. Robotics work used to require using a lot of time building custom solutions for those problems. Now you can just order components.
Here's an idea I'd like to see revisited. Back in the 1980s, someone built a pair of small forklifts that operated as a team. These were little things, about half a meter cubed, with maybe 50cm of lift. Individually, they couldn't do much. But a pair working together could pick up and move a couch, with one robot lifting each end. In the 1980s, the researchers had trouble coordinating two mobile robots. Communications alone were a big problem. Not today.
There are many material handling tasks where one small robot isn't enough, and a big machine the size of a forklift is too bulky. But teams of small robots might work.
I still have a small robot arm on my desk, but it's not connected to anything.
Meh, maybe in a decade. ROS has no official support for it either. Robotics is still very much a C++ town and will probably remain as such for a long while as long as people remain stubbornly set in their ways. If you apply as a Rust only dev to a robotics company nobody will take you seriously.
Yes it can and we can.
Software has been a critical component of much progress we have made over the last few decades.
I understand the difficulty with the abstract many people have. Some may think Hilbert spaces are not real. For many they are as real as any space around them. Some live in them !
The other part that’ll catch you pretty hard if you don’t have strong trig and linear algebra going for you is that debugging is going to be really hard. I admittedly have been deep in positioning-and-attitude control for 5 years now, but the ability to look at a transformation matrix or quaternion and mentally grok exactly what it means odd going to be incredibly useful when you’re trying to figure out why your system isn’t doing what you think it should be.
My calculus is strongest; I can deal with that basically. My geometry is so poor that I am surprised I passed my maths A level.
These books excite me because they give back that, er, constructivist thing that has always appealed to me -- Piaget, Papert, Minsky.
I enjoyed your article, thank you for writing it.
But the pay/perks situation is dependent on putting up with a lot of corporate BS; if you're a solo developer, or the only developer in a small team, web stuff is hard, the task shrinks over time as things commoditise and you become more dependent on hosted offerings, and pays relatively poorly. I am tired and seeking a direction change before it is too late.
Literally the only topic on which I agree with Elon Musk is that more developers need to be persuaded to do something more interesting than getting dragged into web dev BS. And I wish I'd paid a lot, lot more attention in maths lessons away on topics that didn't have to do with logic.
(It is, nevertheless, amusing that it is web dev BS where Musk is getting his posterior handed to him)
The number of people who both know C++ (not C) and how to use an oscilloscope is surprisingly small.
Electronics at robotic speeds isn't that hard. Most signals are audio bandwidth or lower. You rarely need all the elaborate design techniques needed when you get into the MHz-Ghz range.
Statements like these make my eyes roll.
The difficult part in embedded, and what makes the difference between a n00b spending 2 months finding a hardware/firmware issue and a graybeard finding it in under 2 days, is not how to use an oscilloscope or how to write "clean" c++, it's knowing where exactly to put the oscilloscope probes and what exactly to look for on the oscilloscope screen, and how to make the bug/edge case reproducible.
You don't have unlimited scope probes, trace memory and time to probe every single signal on the board and stare at blank signals not knowing what to look for, for weeks, while th clock is ticking.
Tinkering with STM32s and Raspberry Pis at home for fun, doesn't prepare you for the issues you'll encounter in debugging production devices (especially battery powered or RF) you'll need to ship on time and on budget.
Knowing how to use a scope and how to write c++ is only 10% of a successful product. The other 90% is blood, sweat and tears hunched over for hours/days over boards and breakpoints to find out your race condition comes from a cheap crystal oscillator operating out of spec and not from your software.
Especially that at small companies and start-ups you don't have the budgets and HW lab equipment that the likes of Apple or Qualcomm can afford to brute force your way to the solution, so you need to be very shrewd and clever with your debugging to make the most out of your limited resources.
I recommend using Arduino and/or Wokwi (https://wokwi.com/) to get started.
Using simulation to train a DIY spot on something novel and see how good I could get the performance in the real world.
Of course that's a dream of mine one day and not an active project.
Chicken, meet egg. If a web crawler can’t find plenty of reference examples, then LLMs trained on web crawled data aren’t going to be very useful.
It seems like the publication turned what is industry standard into a sensationalistic article.
The only thing Nvidia did different was rolling directly to tapeout with their simulated design without any intermediate prototypes which was indeed a risky move but not unheard of for cash strapped semi startups.
You certainly can work with visual slam in the industry, but the job pool for that is not very big, I would say.
It gets interesting when these systems need to be productionized and scaled. A strong software engineer who can speak the math is invaluable. My first year on the job was basically removing N^2 loops and making 3D visualizations.
Depending on what kind of drone you want to build (powered fixed wing, glider, quad, hex or octocopter) that might be more or less of an issue. Given the power draw and weight of an octocopter I don't think it would be much of an issue there but on a quadcopter, especially a very small one that might be prohibitive. But I'd focus on getting it to work before optimizing for size and weight.
Plenty of tutorials and libraries (often “working but not high-quality”) are available as well.
Even if c/c++ scares you a little, I’d still start there (I came from a long c/c++ background, but I’ve seen my kids be able to do it with not nearly as much help as I expected to give them).
Dirt cheap and readily available as well. Clones are fine; you don’t need the name brand Arduino boards.
My grasp of trig has relapsed to 13-year-old level. It's an embarrassment!
I feel like lots of companies have ditched ROS 1 only because of support being cut and cargo culting about how ROS 2 is better for unclear reasons, but in practice it feels anything but production ready to me. Everyone talks big how Zenoh will solve everything, but the thing is a damn prototype and really goes against the whole idea that having a DDS will be somehow better instead of just adding absurd overhead in both CPU and networking.
I've touched ROS 2 back in 2019 during the Crystal Clemmys era but it still felt experimental back then (and I believe still was considered beta), I plan to re-evaluate later this year/early 2025 how stable ROS 2 is now.
ROS 2 is better in some aspects that weren't accounted for back when ROS 1 was developed (which started as research platform), mainly distributed systems (multi-robot environments) and good tech developed elsewhere that the system could benefit from.
My personal take is that there is a lot of hit and miss between versions, no guarantee that what works now will work on the next one due to the decentralized nature of the community and the fact the team developing all this is relatively small. I would love to have a way to do small iterations on my own stack to bring it to the latest and greatest ROS release, but the combo of ROS 2 + major Ubuntu upgrade every 2 years is maybe too much for my own peace of mind.
Feels like everything is server programming.
I am steering myself away from web dev tools (I'm a Laravel/PHP/Rails/Node.js/Vue/React/GraphQL/DB/server/Perl/HTML/CSS/front-end/back-end/every-bloody-end guy, jack-of-all-trades, master-of-some).
I've been doing it too long and I'm a solo/design agency worker now. I've built some big stuff (and some early semi-famous stuff) and I've also ended up doing the crappy-little-WP-site shuffle.
I am aiming specifically towards tools that reach out a little further.
Basically I am finally, after all these years, properly learning Python, which I have never once actually needed. Because it reaches further than the programming languages on that list. Into desktop app development. Into scientific computing. Into embedded. More towards 3D printing, GIS etc.
And it still works for all the web things, so I can bridge.
And perhaps most importantly, more towards teaching and education, so I can actually be some use before I die.
I started in game mechanics programming and graphics programming when I was a kid.
But that isn’t the same as knowing how to get a job in those fields.
Coming from software, if you are interested, I would suggest either:
- Backend platform development (Python, C++ as main programming languages with a strong focus on ROS[1]).
- Frontend development (nothing too different from what's out there).
As small projects I would suggest playing with ROS to learn it and getting a running simulation with a simple robot that you can teleoperate, most of the stack already exists, it's just connecting everything together [2].
Another venue is open source contribution [1] to get known within the community and potentially attract interest from companies. ROS has multiple packages, from cloud infrastructure to drivers and simulation, if you see anything there you could contribute to, they will gladly take contributions.
In general robotics greatly benefits of good technologies from other areas, if there is a tool we use you believe could be better or a lack of good tooling in a specific area, it will get noticed.
So this would be my suggested path: learn C++/Python if you're not familiar with, learn ROS and watch which specialties appear more often in robot related jos posts [3]. If you are really invested, maybe go to a robotics conference as ROSCon to meet other enthusiasts, which companies are engaged with the community, etc.
Good luck!
Note: not everything robot related is done in ROS, but it's almost a standard within the field save for a few exceptions.
[1]: https://www.ros.org/ [2]: http://wiki.ros.org/ROS/Tutorials [3]: https://discourse.ros.org/c/jobs/15
In academia, yes. ROS is a piece of middleware for passing messages around, and a standard for talking to it. Funding agencies pushed academic robotics projects to talk to ROS so that results from different projects could interoperate. Which they sort of do. You get a lot of tooling for logging, user interfaces, wiring things up, etc. Think of it as a solderless breadboard for robotics software. The final product probably doesn't use it.
I disagree here, usually projects that are not keen on relying on ROS end up re-implementing a lot of the groundwork (shared memory, message distribution, parameter server, logging, etc.), usually they are older products (10 or 15+ years old), where ROS wasn't really a thing and migrating to ROS after implementing their own tailored stack is not worth it.
The real trouble comes during the next step. You have to integrate all these disparate boards into one unit. You can copy schematics and layouts from the dev boards, but that takes a lot of time. Once you've done that, you need to re-validate your entire stack, electronics, firmware, and software. You will inevitably find that you've missed a connection or the routing introduced some noise or you got this component value wrong. So you spin a new board, wait a week, and do it all over again several more times.
Debugging electronics is a lot like debugging software. Except that the code is physical objects and the bugs are emergent properties of the whole system and are inherently chaotic. You (generally) can't step through the system like you would with code, you can only observe a small number of nodes in the system at any one time.
It's harder than you expect because you're dealing with invisible forces and effects of the fundamental laws of physics. It requires either very deep knowledge or a lot of time in iterations to solve some of these problems.
- when your function calls don't have enough white space between them, they'll some times mix up their return value (crosstalk)
- the more deeply your if/else statements are nested, the more random their results end up being (voltage/IR drop)
- when the user makes a selection, your bound function is called somewhere between once and a dozen times. Each time with a different value (bounce)
- their is no `true` or `false`. You just are given a float between one and zero, where one is true and zero is false. Sometimes `true` is as low as 0.3 and sometimes `false` is as high as 0.7
- the farther apart your variable declaration is from it's use, the less it represents what you actually declared. If it's two lines apart, a 'foo' string is still 'foo'. Hundred lines apart though, and 'foo' might become 5.00035 (attenuation)And don't forget that you have to balance the physical shape and arrangement of components and traces with frequency and surrounding components otherwise you've created a transmitter spewing out noise at tens of MHz.
Or the corollary: if you aren't careful you can receive radio signals that will corrupt your data.
Oh, you think your wire can handle the 0.5A your widget needs? Let me tell you about transients that spike to tens of amps for a few hundred nanoseconds. But it's okay, that problem can be solved with a bit of trigonometry.
On the plus side, if you forget to connect your ADC to something, you now have a surprisingly decent random number source.
I love the absolute chaotic insanity of electronics. On the surface things make sense, but one level deeper and nothing makes sense. If you go further than that, at the bottom you'll find beautiful and pure physics and everything makes sense again.
I feel the same way about software. It's a hot mess, but under everything there's this little clockwork machine that simply reads some bits, then flips some other bits based on a comparison to another set of bits. There's no magic, just pure logic. I find it a very beautiful concept.
No updates. No bugfixes. What you pressed to CD (as much as a buck per CD qty 1000), was what your customers got. Period.
And it hurt to republish, because those updates were via mail. The box/sleeve, the labelling, the labour, the shipping.
DEVs today have no idea how easy their push is.
Get it right or you fail.
> mainly distributed systems (multi-robot environments) and good tech developed elsewhere that the system could benefit from
Yeah that was the theory anyway, I'm not sure how much of that had really worked out as intended. Automatic discovery without a roscore has proven... unfeasible. Especially over wifi since the required amount of multicasts basically runs a continuous ddos attack and now we're moving back to the same old concept with FastDDS Discovery Server, Zenohd, and the ros2 daemon. Sure the DDS technically has encryption now, but I've yet to hear of anybody using it. Overall it's just a mess.
When you're doing that evaluation, take a good look at python nodes and their performance compared to Noetic. Currently it's bad, like horrendously bad, 30x less efficient in some cases that I've tested. And they've also ported the CLI parts from bash to python so tab autocomplete and parameter fetching take like actual seconds. I just don't know...
Maybe there's lots of systems out there already running on Cyclone or FastRTPS that make it all worthwhile, but I haven't really heard of any such cases.
> the combo of ROS 2 + major Ubuntu upgrade every 2 years is maybe too much for my own peace of mind
Yeah 2 years seems a long time, but it never turns out that way. It really pains me to see useful packages made obsolete and unusable again and again for no reason but moving to the next release which usually brings effectively nothing by itself.
Ouch. Back when I played with it efficiency wasn't even under discussion, as the basics had just gotten there (topic/service/action support), last I paid attention to this matter was during to some performance/inconsistencies of FastDDS (during Foxy, maybe?) from which the default was changed to Cyclone, but I must admit I'm a bit out of the loop.
> Yeah 2 years seems a long time, but it never turns out that way. It really pains me to see useful packages made obsolete and unusable again and again for no reason but moving to the next release which usually brings effectively nothing by itself.
Yup. Two years makes sense for cloud, but with hardware involved and a fleet deployed in varying conditions around the world, 2 years to migrate a major version of ROS and Ubuntu is maybe too short (not even considering packages that you depend on that might not be ported at all). I would love to see other distros with longer spans (Alma/RHEL/Debian) becoming 1st class citizens and ROS releases turning into incremental versions running on the same platform until they reach EOL, at "robotics pace" that would be fantastic.
The reality is that computers are comprised of several PCBs running with thousands of multi-GHz signals. So all of the foregoing engineering design principles had to be observed to make our systems as reliable as they are.