Show HN: High-Res Neural Cellular Automata

Show HN: High-Res Neural Cellular Automata(cells2pixels.github.io)

99 points by esychology 3 hours ago | 15 comments

Neural CAs model self-organizing pattern formation.

Now they can generate patterns at HD resolution in real-time, enabled by turning each CA cell into a Neural Field.

Try 3 demos: grow a pattern from a seed (and damage it, it heals), synthesize PBR textures that can regenerate, or create 3D textures like clouds.

whilenot-dev 1 hour ago |

The automata just completely destroys the image if I draw too much over the stabilized image with the brush. 5 horizontal swipes are enough to destroy the kitty, is that to be expected?

EDIT: video here: https://imgur.com/a/ItZGd5X

WhiteNoiz3 2 minutes ago | |

With the old model (and I suspect this one too) it's trained to generate from a single 'seed' pixel in the center of the image. If you erase the center of the image, that's when it completely collapses.

esychology 32 minutes ago | |

The NeuralCA both generates and maintains the pattern. Because the NCA was not exposed to damage or erasure during training, its regeneration capability is a purely emergent phenomenon. However, this ability remains somewhat brittle, particularly when the central regions of the pattern are erased.

hidelooktropic 25 minutes ago |

For the unfamiliar, could someone explain what I'm looking at? The abstract was a little too concrete (heh) for me to follow.

esychology 21 minutes ago | |

If you're familiar with CAs (e.g. Conway's Game of Life), you can think of a NeuralCA as a CA where the update rule is given by a neural network. Here we optimize the neural net weights so that it behaves a certain way (e.g. grow a lizard from a single seed).

flir 7 minutes ago | | |

What are the inputs to the NN? The whole grid, or just nearby cells? What happens if two NNs overlap on the same grid? (Gonna go read the paper).

jekude 1 hour ago |

The abstract implies that strictly local updates are a hinderance to high res, however i would have thought there would be an interesting way to get speed up gains from neighbor-only traffic on GPUs CAM-style. am i making that up?

embedding-shape 1 hour ago |

Really interesting demo, nicely done :) Would be fun if switching the "Target Image" when using the second brush mode in the Growing Demo didn't erase/reset the existing canvas, so we could "stamp" new things on top of other images. Small thing perhaps but I got sad when it disappeared when I wanted to merge a kitten on top of the chameleon but couldn't :(

bfmalky 39 minutes ago | |

You can, just enable the 'transition' switch.

mirekrusin 1 hour ago |

So the goal is to evaporate it with minimum number of shots?

amelius 1 hour ago |

Why are the images always generated in the same orientation (upright)? Do the cells have awareness of what is "up"?

esychology 1 hour ago | |

yeah normally NCAs have a sense of up and left. There are some isotropic variants that make the perception fully rotation-invariant.

WithinReason 1 hour ago |

You can make the centipede grow longer, which makes sense given how this works. Or grow a 2nd centipede for extra points.

esychology 1 hour ago | |

haha yes, also the same with the worm