Hey HN,

I built this tool to solve a problem that drove me crazy during my studies: course materials are often in formats that are terrible for quick review and reference. I was spending hours watching long video lectures before finals, trying to make sense of sparse PDF slides, or deciphering the ancient hieroglyphs in my friends' (or my own) notes.

The Lore Engine is a Python-based tool that processes content (from a single file or an entire folder) and generates comprehensive markdown files. You can feed it PDFs (slides, scanned notes), or transcripts with optional video context.

What makes it different is that you can control the output based on your needs. Through the CLI, you can choose: - Content Type: (e.g., Slides, Textbook, Videos) - Output Format: Detailed notes, practice problems, or a concise pre-exam revision sheet. - Detail Level: From short-hand summaries to a deep-dive.

(Note: The practice problem and revision sheet modes are still experimental.)

The goal is to turn 10 hours of passive video watching into a ~2-hour focused read, while preserving all the necessary detail.

On the technical side: It's a multimodal pipeline using the Gemini API (I chose it for the free tier and native multimodal support). The prompt system is modular, allowing for fine-grained control over the output style.

For video processing, it uses video_reader-rs (Rust FFmpeg bindings) instead of OpenCV for a significant performance boost and better memory management (it doesn't load entire videos into RAM).

To avoid redundant images (like 50 identical slides), it uses perceptual hashing (pHash) combined with a custom diversity scoring algorithm to select visually distinct frames.

The pipeline is multi-processed and uses round-robin API keys to scale throughput—which is essential when you're trying to process an entire course the night before an exam. It also has built-in rate limiting and exponential backoff for retries.

The project is open source and runs locally. Here is the link to the repository: https://github.com/Slydite/lore-engine

I'd love to get your feedback. Thanks for checking it out!

user:	Slydite
created:	June 5, 2022
karma:	19