About a year ago I had an idea and I've been very slowly iterating since then. Imagine you play a game in a foreign language that you don't know very well (e.g. you're learning). You see an unknown word in a dialog. You hit a hot key and then:
1. The game is paused.
2. Over the game window an overlay is drawn. On this overlay, the text from the screen is recognized.
3. You can select a word or a phrase and get their translation, ask ChatGPT to explain it, or open Wiktionary.
4. Then you press Esc and continue playing. There's only one catch: you need to give it a Google Cloud API key, because it uses its OCR and language services. However, what a single person may use, it'd most likely be in the free tier range and would cost $0.00/month. So it's mostly an inconvenience. It works on Windows and also X11 Linux. Not on Wayland or Mac, for now. Nothing fundamentally impossible there, I just need to test and polish the rough edges. It's open source, GPL-3.0. And if you care, it's not Electron, it's Qt. Nowadays we should frugal with RAM :) |
No comments yet