15:[[40528192,{"by":"phodo","descendants":90,"id":40528192,"kids":[40529996,40529501,40529146,40537612,40531138,40529193,40535585,40529380,40530990,40530859,40608694,40533883,40530530,40529607,40617855,40528985,40533534,40531173,40529239,40532665,40559760,40530614,40531714,40532241,40530921,40533472,40528956,40532485,40529216,40529689,40535649,40543878,40533017,40530116,40530343,40543076,40529092,40528532,40531037],"score":257,"text":"As the title says, I have many PDFs - mostly scans via Scansnap - but also non-scans. These are sensitive in nature, e.g. bills, documents, etc. I would like a local-first AI solution that allows me to say things like: "show me all tax documents for August 2023" or "show my home title". Ideally it is Mac software that can access iCloud too, since that where I store it all. I would prefer to not do any tagging. I would like to optimize on recall over precision, so False Positives in the search results are ok. What are modern approaches to do this, without hacking one up on my own?","time":1717100645,"title":"Ask HN: I have many PDFs – what is the best local way to leverage AI for search?","type":"story"}],[38949005,{"by":"lxe","descendants":3,"id":38949005,"kids":[38949187,38949402],"score":4,"text":"llm-companion is a rather fast and high-quality (I think) TTS push-to-talk chat interface with OpenAI-compatible LLM APIs, such as OpenAI, Oobabooga, vllm, llama.cpp, etc.

It uses Whisper for transcription (local), StyleTTS2 for TTS (also local).

Here's a little demo video if it in action: https://twitter.com/lxe/status/1745348827983560991

It uses streaming for both LLM results and TTS, which significantly decreases the interaction latency as compared to something like ChatGPT Voice mode.

The goal was to create a fully self-contained locally-running AI chat app, and this is the result.

Enjoy!","time":1704958899,"title":"Show HN: LLM-Companion: push-to-talk + TTS web chat app for OpenAI-like APIs","type":"story","url":"https://github.com/lxe/llm-companion"}],[38157524,{"by":"lxe","descendants":19,"id":38157524,"kids":[38161183,38159572,38157949,38171814,38159915,38161000,38159481,38161903],"score":154,"text":"A simple mobile web app inspired by Fuzzy-Search/realtime-bakllava that uses llama.cpp server backend with multimodal mode to describe and narrate what the phone camera sees.

I built this thing in a few hours using a single ChatGPT thread to generate most things for me and iterate on this project. Here's the workflow: https://chat.openai.com/share/ea84ec69-5617-45e8-8772-ac2dcf...","time":1699232108,"title":"Show HN: LLaVaVision: An AI \"Be My Eyes\"-like web app with a llama.cpp backend","type":"story","url":"https://github.com/lxe/llavavision"}],[35256769,{"by":"lxe","descendants":98,"id":35256769,"kids":[35257901,35263883,35258153,35258186,35257190,35258239,35258435,35257101,35261893,35257654,35260007,35258106,35260037,35267214,35267025,35257021,35257948,35304071,35304065,35257245],"score":449,"text":"I've been playing around with https://github.com/zphang/minimal-llama/ and https://github.com/tloen/alpaca-lora/blob/main/finetune.py, and wanted to create a simple UI where you can just paste text, tweak the parameters, and finetune the model quickly using a modern GPU.

To prepare the data, simply separate your text with two blank lines.

There's an inference tab, so you can test how the tuned model behaves.

This is my first foray into the world of LLM finetuning, Python, Torch, Transformers, LoRA, PEFT, and Gradio.

Enjoy!","time":1679458529,"title":"Show HN: Finetune LLaMA-7B on commodity GPUs using your own text","type":"story","url":"https://github.com/lxe/simple-llama-finetuner"}],[35144494,{"by":"yousnail","descendants":1,"id":35144494,"kids":[35144495],"score":4,"time":1678748934,"title":"PLC code generation using ChatGPT","type":"story","url":"https://www.lucidautomationsecurity.com/2023/03/13/chatgpt-generated-plc-code/"}]]

user:	yousnail
created:	June 28, 2017
karma:	11