Generative Flow Networks

Generative Flow Networks(yoshuabengio.org)

168 points by selimonder 4 years ago | 18 comments

cs702 4 years ago |

For a quick, easy-to-digest introduction to the topic, there's a fantastic video with explanations and an interview with Bengio that uses Galton peg boards as a starting point for understanding generative flow networks. Highly recommended if you're a researcher or practitioner interested in learning about this:

https://www.youtube.com/watch?v=M49TMqK5uCE

manux 4 years ago |

Hi, first author of the original NeurIPS paper here! Someone just shared this HN post with me. I'll go through the comments but happy to try to answer questions as well.

cs702 4 years ago | |

I'm still wrapping my head around this (it seems rather magical; can it really be just that?)... but I find the idea of "sampling from a distribution of trajectories that reach a terminal state whose probability is proportional to a positive reward function, by minimizing the difference between the flow coming into the trajectories and the flow coming out of them, which by construction must be equal" to be both beautiful and elegant -- like all great ideas.

How did you and your coauthors come up with this? Trial and error? Or was there a moment of serendipitous creative insight?

To the moderators: My comment is now at the top of the page, but manux's comment above is more deserving of the top spot. I just upvoted it. Please consider pinning it at the top.

manux 4 years ago | | |

I think the inspiration came to me from looking at SumTrees and from having worked on Temporal Difference learning for a long time. The idea of flows came to Yoshua and I from the realization that we wanted some kind of energy conservation/preservation mechanism from having multiple paths lead to the same state.

But, yes in the moment it felt like some very serendipitous insight!

igorkraw 4 years ago | |

Could you highlight the difference between this and training a permutation invariant or equivariant policy network using standard supervised or RL methods? Assuming I also have a way of having an invariant/equivariant loss function

manux 4 years ago | | |

What the permutation invariance gets you is that the model doesn't arbitrarily prefer one (graph) configuration over another, but this seems tangential. The difference between this and RL is in what we do with the reward:

- RL says, give me a reward and I'll give you its max.

- GFlowNet says, give me a reward and I'll give you all its modes (via p(x) \propto R(x)).

Yes you would ideally have a loss (well, a reward/energy) that is invariant and operates e.g. directly on the molecule rather than on some arbitrary ordering of the nodes.

soraki_soladead 4 years ago |

Here's the referenced tutorial: https://milayb.notion.site/GFlowNet-Tutorial-919dcf0a0f0c4e9...

(Some of the links appear to require signing in to Notion but this one doesn't.)

mark_l_watson 4 years ago |

I started reading through the linked tutorial, but I think I need to first back up and review reinforcement learning (I took 2 Coursera classes in RL, and helped with one project at a previous job, so I am at a beginner stage).

I love deep learning, a practical way to solve many problems (and most of my work over the last 6 years) but DL seems incomplete as far as explainability, causation, etc. Anyway, I just retired last week (mostly to take care of my wife who is having a health crisis) so I have plenty of time to study GFN, probably starting with re-taking the 2 Coursera classes on RL and then finishing the RL specialization.

awelkie 4 years ago |

Having just skimmed the linked tutorial, it looks like GFlowNets could be useful for automated theorem proving. Anyone with more knowledge know if this is possible?

manux 4 years ago | |

Yes, although they may not be ideal unless you're able to define a "distance" between a proof that the agent is proposing and whether that proof is correct (i.e. unless you're able to define a reward or energy function).

It may be possible to infer/learn a score from existing proofs though. We have a paper that manages to both learn a flow and an energy function (the score) from data: https://arxiv.org/abs/2202.01361

I don't know much about theorem proving though. Can some value be attributed to partial proofs?

andrewnc 4 years ago | |

I don't see why not. I think they can be used for anything a GNN / Transformer could be used for, but with better probabilistic qualities.

mcbuilder 4 years ago | | |

What do you see as the application to GNN / Transformer tasks? Both of those networks are best suited for different tasks, and GFlowNetworks seem well suited to quickly approximating the partition function of say an Deep RL reward distribution, something that MCMC is ill suited for because of the incredibly high dimensional space. I see these all as different techniques, perhaps the ideas will combine at some point but GFlowNetworks are not an evolution of GNNs or Transformers.

ramesh31 4 years ago |

Correct me if I'm wrong, but it seems like SOTA for AI now is entirely in developing more and more complex sets of convolutions for deep learning?

MathYouF 4 years ago | |

And what is the SoTA for biological intelligence? While not exactly this, it seems to be pretty similar (even between members of the same species).

Snark aside, with a few nuances, you're mostly correct.