Dark Hacker News
new
|
best
|
ask
|
show
|
jobs
Frontier Pre-Training at Microsoft AI | Dark Hacker News
Frontier Pre-Training at Microsoft AI
(microsoft.ai)
26 points
by
psoulos
5 hours ago
| 3 comments
adroitboss
1 hour ago
|
next
[−]
Wait, I don't get it. Is the model not open? If not, why are they comparing it to Open Weight models instead of the state of the art?
leothetechguy
1 hour ago
|
parent
|
next
[−]
Because SOTA Closed Base Models are not available to compare to.
Bolwin
2 hours ago
|
next
[−]
Very similar to Kimi K3 it seems, but fewer active params (80b vs 104b). I don't think they mentioned what form of attention they use