TinyToT – Tree of Thoughts Inference Server(github.com) |
TinyToT – Tree of Thoughts Inference Server(github.com) |
It scores 97% on a 35-question benchmark spanning graduate-level science, medicine, law, finance, and software engineering. It runs on a laptop CPU in under 1ms per query.
Fun fact: This was able to summarize the Lord of the Rings well and work on a variety of different problems.