I wanted to share a project I have been working on recently called "plomp". It is intended to be a drop-in way to gain visibility into python programs which are prompting LLMs many times or using complicated contexts while remaining unopinionated on how you are doing the prompting.

Basically you add some instrumentation to your program via decorators or small function calls and it produces a visualization of the program execution.

Personally, I am not particularly interested in using a heavy-duty orchestration or chaining framework for managing context, chaining prompts (Langchain etc.) and I typically just interact directly with the LLM providers via their clients or standard HTTPS. I had noticed that having this layer for debugging would be useful for some of my other side projects so I decided to pull it out into something standalone.

https://github.com/michaelgiba/plomp

Let me know what you think