I recently expanded a project that allows for easy benchmarking of local language models, specifically for Named Entity Recognition (NER) tasks. By using a simple JSON configuration, you can now evaluate and compare the performance of different models installed through Ollama without the need for extensive coding.

The idea is to make it easier for developers and data scientists to test various models and see how they perform on specific tasks, all within a streamlined framework.

If you're interested in learning more about the approach and trying it out, you can check out the full details in my blog post: Creating a JSON Framework to Test Local Language Models for NER — https://datasignal.uk/blog/ner-framework.html

The code is open source and available on GitHub: NER-llm-blog Repo — https://github.com/DataSignalOrg/NER-llm-blog/tree/master

I’d love to hear your thoughts or suggestions for improvements!