RL Is Bottlenecked by Inference. Scale It Independently

11 points by alex000kim 43 days ago | 2 comments

efiop 43 days ago |

what did gpu hours look like here? with 3 replicas for a 1.8x speedup, the cost tradeoff isn’t obvious.

efiop 43 days ago | |

ah, nevermind. 3 engines seem cheaper overall too: 7x661s vs 5x1200s of allocated H100 time per step. Nice.

efiop 43 days ago |

what did gpu hours look like here? with 3 replicas for a 1.8x speedup, the cost tradeoff isn’t obvious.

efiop 43 days ago | |

ah, nevermind. 3 engines seem cheaper overall too: 7x661s vs 5x1200s of allocated H100 time per step. Nice.