Here is a little pet project of mine, it is a standalone electron js app meant to run a packaged version of Mistral 7B with a focus on loading and interacting with docs in a fully local environment, kinda like a mini chat with RTX.

Runs well on Apple Silicon and on Windows with GPU acceleration although file loading is slower on windows. Current supported files are: pdf, docx, xslx, pptx, and markdown. 16gb of RAM are recommended and might actually be necessary.

Needs some work and will likely open source it soon. Main issue right now is the file loading system as it is rather counterintuitive, it only allows to select directories with the files rather than just choosing the files themselves. Windows installer is also quite janky as it is my first time working on such a project and Inno setup refused to work properly.

Anyway hope you find it interesting! Please let me know what you think :)

PS: website looks BAD on mobile