It's about the hardest agentic problem that has measurable results and will be adopted (because it's an engineering culture) that I can think of. RunWare have some good insights on this - much of the problem is context layer improvisation and repetition, and the only way out is an awful lot of diagnostic development.
And anything less than 90% accuracy on causal analysis is more work than doing everything by hand.
Yeah the context is the key to make the LLM really useful for oncall. But honestly the SRE agent companies today are trying to keep users in their platforms, not focusing on solving the context problem. That's what I saw.
Some lessons learned when using the SRE agents to handle incidents. We eventually ended up to a semi automation way to get more accurate LLM results by providing domain knowledge and context to LLM. Wondering if you encountered the similar inaccurate or sometimes nonsense response from SRE agents. How would you solve it?