Ask HN: How to unit test AI responses? I am tasked to build a customer support chat. The AI should be trained with company docs. How can I be sure the AI will not hallucinate a bad response to a customer? |
Ask HN: How to unit test AI responses? I am tasked to build a customer support chat. The AI should be trained with company docs. How can I be sure the AI will not hallucinate a bad response to a customer? |
If you 100% can not tolerate "bad" answers, only use the LLM in the front end to map the user's input onto a set of templated questions with templated answers. In the worst case, the user gets a right answer to the wrong question.
This involves having another prompt, and possibly another model, evaluate the quality of the first response. Then you write your code to try again in a loop and raise an alert if it keeps failing.
Quality Training Data: Train the model on high-quality, up-to-date company documents, ensuring it reflects accurate information.
Fine-tuning: Regularly fine-tune the model on specific support use cases and real customer interactions.
Feedback Loops: Implement a system for human oversight where support agents can review and correct the AI's responses.
Context Awareness: Design the system to ask clarifying questions if uncertain, avoiding direct false information.
Monitoring: Continuously monitor and evaluate the AI’s performance to catch and address any issues promptly.