Ways to think about token pricing

Ways to think about token pricing(ben-evans.com)

24 points by mercutio2 3 hours ago | 7 comments

I know there's a lot of reasons to think that everyone will just use AI inference in the cloud, but I think if everyone had access to a dgx gb400 class machine with 512gb of hbm4 vram and 1tb of lpddr8x a lot of people are going to be running finetunes of models locally. Like the dgx gb300 is $94k now, but i bet this class of machine will come down to $20kish in the next 2-3 years.

Analemma_ 34 minutes ago | |

You are waaaaaay too optimistic about RAM prices. This is not a one-quarter crisis and you need to settle in for the long haul: the era of "hardware always comes down in price" is over, definitely for the next 3 years and possibly the next 5-8. The new DRAM fabs aren't even coming online until 2028 and there's not much reason to think they'll make much of a dent in the supply-demand mismatch even when they do.

Legend2440 23 minutes ago | | |

DRAM prices have been highly cyclical for a long time. They'll come down again.

cyanydeez 1 hour ago | |

sure, if there werent capitalists digging their moat. in 1 year local inferwnce doubled cost from ram alone because cloud vendors cornered markets on their circular cash flow expectations.

mips_avatar 1 hour ago | | |

Yeah but Samsung/Micron/SK hynix are building new fabs, and Chinese Ram will be able to fully supply the lower end ddr5 ram market within 2 years. The crunch is temporary and on the other side of it there will be incredible hardware at the $10-20k price point.

nekusar 2 hours ago |

1. Slot machine

You put in minutes of time combined with token costs. You pull the lever and HOPE something good comes back.

And it gets tantalizingly close, but for 100% gotta pull the arm again!