Ask HN: What are the most promising RL fields for a new master student?

18 points by zecice 2 hours ago | 8 comments

i am an incoming msc student thinking about a research direction in reinforcement learning. which rl subfields do you think have the most potential right now? i am particularly interested in emboddied ai and bcis.

gessha 1 hour ago |

General advice from a CV PhD dropout:

Lean onto the interests of assistant professors at your school instead of trying to find promising RL subtopics.

This way you can get time with someone who’s deep in the field and can be a guiding light for your projects.

If you’re really into embodied AI, you can add that to an existing project but doing it by yourself can be extremely tough especially if your topic requires a lot of compute and your last name is not Rockefeller.

jvanderbot 40 minutes ago | |

Concur! (CS PhD)

And in general, depending on "funding", simply tapping latent ideas from profs will be an instant win-win. They've wanted to try it, and you've wanted to do work in the field. Approach it that way, as a meaningful collaboration for both of you - and if you have more than one option, then bias towards personality match and "fun" factor.

Tryk 56 minutes ago | |

It is especially important to work with early-stage researchers, as they are more eager to publish and obtain career stability than full professors.

Hendrikto 1 hour ago |

My advice, as somebody who has a master’s degree, is to pick a topic that you are really interested in, and that captures your imagination. You will have to dedicate yourself to this work for a considerable amount of time.

Doing a CV-driven master thesis will be miserable and more likely to fail. It should be something that is fun and captivating.

m_ke 43 minutes ago |

On Policy Self Distillation and Active Learning. Anything that increases sample efficiency by providing a richer more dense feedback signal and is more efficient at exploration / sampling.

murzynalbinos 1 hour ago |

RL for closed-loop adaptive BCIs and sim-to-real robotics still look like the strongest practical directions right now.

porridgeraisin 59 minutes ago |

Depends on the structure of your Msc, the funding, the compute available, etc, you need to give those details if you want a detailed answer. If you choose something at the limits of the compute available to you, then you make things 10x as difficult.

The easy answer that applies everywhere is, choose a topic that has a good volume of phds, postdocs, or professors at the college you're joining. The deep knowledge that arises from this no matter which subfield in RL it is in will later help you transition to your desired subfield much better than a surface-level effort directly in your desired subfield.

The most _likely_ answer is sim2real. It has just the right mix of generative AI (funding), robotics usecases (there is an easy "end" for your thesis to hook onto as the "impact"), and lots of unexplored paths (you won't be chasing a common frontier competing with hundred others). But like I said this depends on the previous two points.

Of course there's what you're passionate about and so on, but at the end of the day objectively deep knowledge matters more than passion, unless you're so deep into something, but then your choice is made and you wouldn't have asked this question, so prioritise deep knowledge. Try to actively reduce breadth, IMO the purpose of a research-oriented program is to learn to study depth-wise to the end.

As for RL in BCI, as far as I know, we still use established "RL 101" algorithms directly applied to the brain signals. Any research deep enough (but short-term enough for a MSc) here will be less RL theory/algorithms and more about the details of this specific application. If that's what you want, go for it.

dominotw 1 hour ago |

things that are hard to verify.

ai didnt takeover the world like it was supposed to because its only good at verifiable stuff like coding but vast majority of economy isnt verifiable.