Making Claude a Chemist(anthropic.com) |
Making Claude a Chemist(anthropic.com) |
[1] https://www.ginkgo.bio/autonomous-lab
Step 1. Make it so Claude can do anything — the whole point of AGI
Step 2. Wait, if the user can do Anything, that would be Very Bad!
Step 3. Err on the safe side with blanket bans of entire fields
The latter actually seems to me a sensible reaction to e.g. the compartmentalization used in the large scale cyber attack using Claude last year. Where they were able to do Bad Thing by dividing it into many, many Small, Seemingly Harmless Things.
Gated access sounds bad (and I agree it sounds bad!) but it might actually be the only sensible response to such a set of conditions. I'm not sure though.
--
I saw some studies recently which showed LLMs provide much more detailed information to expert users. So we can distinguish between competence and incompetence based on use of language, and that is a reasonable metric for harm reduction.
But I don't think we can reliably detect "user has harmful intentions", at least not at a sufficient level of sophistication of the attacker.
They are following closely and the best offer 80-90% of the performance and come with a very small fraction of the costs.
Feels complex like solving a Rubik's cube to write down synthesis steps but it is all a sequence of memorized tricks. Do Cannizaro if you want this, Bergmann to do that.
But the synthesis plan is only 10% of the actual work.
The gap between writing down the synthesis step and actually doing it is also extremely large.
Even if you get the right molecule, it might be the wrong way around or just clump up into a useless mess.
The Ritonavir episode of Veritasium is a great example of how all chemistry on paper is a mere shadow of what actually happens in real life.
I remember two years ago, when I actually got into using graph data structures, wondering if maybe the "space" of available reactions for any given starter and target molecules could be mapped as a graph, with intermediates as nodes and reactions as weighted directed edges, so synthesis becomes pathfinding through chemical space.
Turns out, it’s a thing! [^0]
Edit: Makes you wonder how much interesting stuff is sitting in plain sight, waiting for someone with the right cross-domain awareness / knowledge / whatever to notice it.
Sounds a lot like vibe coding lol
You can use those and they probably won't intentionally sabotage you.