15:[[47507666,{"by":"felix089","descendants":98,"id":47507666,"kids":[47508505,47509457,47515524,47515726,47511835,47514072,47591872,47577161,47516688,47509481,47517991,47515692,47508466,47524068,47509256,47530154,47542113,47514047,47515300,47517164,47553160,47525715,47512268,47515349,47513494,47513941,47520249,47527301,47517046,47507931,47515277,47510172,47557319,47521704,47512684,47514394,47529219,47511017,47509557,47522911,47516716,47511278,47513816,47517331,47508926,47559923,47538501,47514930,47517251,47513216,47513982,47513095,47514693,47508186,47509372,47526999,47526719,47518583,47522695],"score":118,"text":"Hey HN! After the Car Wash Test post got quite a big discussion going (400+ comments, https://news.ycombinator.com/item?id=47128138), I spent the past few weeks building a tool so anyone can run these kinds of questions and get structured results. No signup and free to use.

You type a question, define answer options, pick up to 50 models at a time from a pool of 200+, and they all answer independently under identical conditions. No system prompt, structured output, same setup for every model.

You can also run a debate round where models see each other's reasoning and get a chance to change their minds. A reviewer model then summarizes the full transcript. All models are routed via my startup Opper. Any feedback is welcome!

Hope you enjoy it, and would love to hear what you think!","time":1774379727,"title":"Show HN: AI Roundtable – Let 200 models debate your question","type":"story","url":"https://opper.ai/ai-roundtable/"}],[47494696,{"by":"noemit","descendants":22,"id":47494696,"kids":[47498409,47496895,47569769,47540979,47518087,47561130,47540200,47499371,47499342,47499000,47548754,47508348,47532964,47526878,47508063,47498203,47500805,47522625,47497365,47498274],"score":41,"text":"I ran an experiment to see if CLI actually was the most intuitive format for tool calling. (As claimed by a ex-Manus AI Backend Engineer) I gave my model random scenarios and a single tool "run" - i told it that it worked like a CLI. I told it to guess commands.

it guessed great commands, but it formatted it always with a colon up front, like \n:help\n:browser\n:search\n:curl

It was trained on how terminals look, not what you actually type (you don't type the ":")

I have since updated my code in my agent tool to stop fighting against this intuition.

LLMs they learn what commands look like in documentation/artifacts, not what the human actually typed on the keyboard.

Seems so obvious. This is why you have to test your LLM and see how it naturally works, so you don't have to fight it with your system prompt.

This is Kimi K2.5 Btw.","time":1774297908,"title":"LLMs learn what programmers create, not how programmers work","type":"story"}]]

user:	infosecphoenix
created:	October 5, 2012
karma:	2
about:	Co-founder and CTO of an AI/ML startup in the Bay Area