Hey HN,

We’ve been on a wild journey building Aux Machina, and think it’s fun to share a bit of the ride with you. It all starts with a “what if.” What if we make AI-powered visuals without needing a PhD in prompting? Sounds straightforward, right? Turns out, it’s more like a series of sleepless nights and way more Python than we bargain for .

Picture this: We met at Harrison.ai in 2019, knee-deep in CNNs in a pre-ChatGPT world. We were building healthcare AI that saves lives — serious stuff. But between all the daily stand-ups and coffee refills , we realised, “Hey, we actually like working together!” (Shocking, I know.) Naturally, after life-saving AI, we took on the next obvious challenge ... helping people create pretty pictures with way less hassle . Same level of world-changing impact, right?

So, with a “let’s make this happen” attitude, we got to work. We’re a small, execution-focused team that just wanted to make creating custom visuals way easier for small businesses, marketers, and creators who don’t want to deal with complicated AI prompts just to get something halfway decent. “How hard could it be?” we thought. Spoiler: it’s harder than we thought. We went with a Python backend and a TypeScript/React front end, nothing too fancy, but it gets the job done (mostly without crashing) . Although, we had one glorious moment where everything went down in flames for a time ... because we didn’t update the .env file. Yup, just one tiny oversight that took hours to discover - like finding a needle in a haystack, if the haystack was also on fire. Fun times.

On the model side, we fine-tuned a latent diffusion model and added some low-rank adaptations to handle the image generation process. The best way to describe it? You feed in your ideas, and the model refines them through layers of noise reduction and reconstruction, bringing something (hopefully) that doesn’t look like abstract art . Sometimes it’s spot on, sometimes... not so much. But hey, we’ve all had our Picasso moments.

Modularity is a big deal for us. We don’t want to build one of those all-in-one, Swiss Army knife disasters that no one touches because it’s too overwhelming — and racks up tech debt faster than you can say ‘refactor’ . I’ll never forget the day we almost accidentally wipe out half the codebase because of a misplaced function. David, our solutions architect, still talks about it like a near-death experience. So yeah, we break things down into modular components. The result? We iterate quickly and improve individual features without breaking everything else — or turning the codebase into a spaghetti monster . We also can’t resist adding some large language model (LLM) magic for tasks like interpreting user inputs or the visual vocabulary tied to reference images. The LLM makes sure what you type turns into something that’s, well, pretty close to what you’re imagining. Most of the time, anyway . There were a few days when the output was stranger than a cat riding a Roomba.

A few things we’ve learned on the way: 1. Building AI tools is like trying to herd cats — except the cats are on fire, and you’re trying not to get burned . 2. React is great for making things slick and modular, but if you’re not careful, it’ll eat your lunch and your weekend. 3. Latent diffusion models are super cool, but they need a lot of fine-tuning unless you want everything to look like Picasso had a rough day .

We’re super excited to see how people use Aux Machina as we head into the go-to-market phase . We were all sizzle and no steak, but now we’re as ready as a bride’s nightie on the big day. We’d love your thoughts (or even better, your brutally honest feedback).

Give it a try: https://www.auxmachina.com

user:	jameswan
created:	February 14, 2022
karma:	2
about:	Co-founder of Aux Machina