It's Hard to Eval Is a Product Smell

It's Hard to Eval Is a Product Smell(hamel.dev)

2 points by _pdp_ 8 hours ago | 1 comment

This is super interesting, and I like the idea of verifiable artifacts that an agent can produce, i.e. notebooks for analysis, links to the source for some claims. Building for scale, it would be interesting to know how the author thinks about automating that and building benchmarks to automate testing the quality

brammertottens 8 hours ago |

This is super interesting, and I like the idea of verifiable artifacts that an agent can produce, i.e. notebooks for analysis, links to the source for some claims. Building for scale, it would be interesting to know how the author thinks about automating that and building benchmarks to automate testing the quality