One design pattern for LLM tools that I'm very interested in exploring is having an LLM only engage when the user (human) is not in a flow state.
Take writing for example. There are times when text auto-complete is actually a welcome feature because you might be experiencing writer's block. An LLM tool providing proactive guidance then becomes a way to get unstuck. But if you're in flow and flying away at the keyboard, auto-complete becomes a distraction and can take you away from your train of thought.
There are obviously some signals that we could programmatically make use of in this scenario (words per minute, pauses between keystrokes, etc.) but I do think defining "flow" in those terms is nebulous. I can imagine a world where stuff like skin conductance tracking a la Oura Ring becomes part of the mix. Would be pretty cool.