The Age of Agent Experience(stytch.com) |
The Age of Agent Experience(stytch.com) |
We are early enough in this evolution to help direct the ship in a way that serves the end user, web owners/creators, and the agent.
It’s almost inevitable since everyone wants more growth and advertising is almost always seen as free money left on the table by decision makers.
This is fine until the agent decides to order something the customer did not want. This is inherent to the concept of an agent. Due to the probabilistic nature of LLMs, and the fact that no agent will ever be perfectly able to predict exactly what you want at the time you want, this scenario is inevitable.
As the shop owner, this would result in an increased numbers of returns. You could recommend that the user must approve the purchase, but given that you do not define these agents, there is no way for you to ensure that the user is actually following your advice.
My entire extended family has two yubikeys: My key and my spare key.
I realize there's a strong impulse to not "reinvent the wheel," but what we have currently is unsustainable. Specifically, the fact that every API uses a slightly different REST API and its own unique authentication & authorization workflow. It worked fine for the days when application developers would spend a few weeks on each new integration, but it totally breaks down when you want to be able to orchestrate an agent across many user-defined services.
I think a simple protocol based on JSON and bog-standard public key encryption could allow agents to coordinate and spend credits/money based on human-defined budgets.
1) long tail of websites that don't have APIs, so the only way for an agent to interact with them on the user's behalf is to log in more conventionally, and
2) even if a website has APIs, there may be tasks to be done that are outside the scope of the provided APIs.
Thoughts?
I was an early engineer at Plaid and I think it's an interesting parallel, financial data aggregators used to use more of a screenscraping model of integration but over the past 5+ years, it's moved almost fully to OAuth integrations. would expect the adoption curve here to be much steeper than that, banks are notoriously slow so would expect tech companies to move even more quickly towards OAuth and APIs for agents.
another dimension of this, is that it's quite easy to block ai agents screenscraping, we're able to identify with almost 100% accuracy open ai's operator, anthropic's computer use api, browswerbase, etc. so some sites might choose to block agents from screenscraping and require the API path.
all of this is still early too, so excited to see how things develop!
I've tried making a Firefox extension that fills webforms using an LLM and the things website makers come up with the break their own forms for both humans and agents are just insane.
There are probably over a 1000 different ways to ask for someone's address that an agent (and/or human) would struggle to understand. Just to name an example.
I think agents will be able to get through them easily, but NOT because the websites makers are going to do a better job at being easier to use.
You can find its dogfooding demo on the Show HN [2].
- The human wouldn't need to share their password information with the agent
- Services would be able to block or ask for approval when agents take sensitive actions. Maybe an e-commerce site is happy to let an agent browse and add items to a cart, but wants a human in the loop for checkout.
- Services would be able to attribute any actions taken to the agent on behalf of the user. Did Joe approve this expense report, or did Joe's agent approve this expense report?
But, parsing documentation? And, believing it blindly? hah. Maybe ressurect Semantic web as well..
This gave me a chuckle. I believe the current hype term along this line is "ontologies".
This shows we need to build better approaches to agent interactions that are not at the level of "run a virtual browser", but that encodes much more of the workflows available than raw API's do today.
If you think AI has agency then you must think all software has agency. AI is just software.
To those of you who say humans are just software: try deactivating a human and see what happens. Note that this is a different experience than deactivating AI.
Tools in this space rely a lot on human use of a computer being much slower, less precise, and more variable than machine use of a computer.
behavioral data like mouse movements, shortest path, etc is helpful but likely to result in less of a deterministic signal compared to device intelligence based on those signals of where and how the request is being made.
we'll have a more in depth blog post on what we're seeing with this next week too.
The economic incentives, the brand control needs, etc. are important dynamics and I don't think it's all in their court alone. It's a combination of where the market goes (the platforms and systems they prefer) and the capabilities unlocked by those platforms.
With that, this evolution will follow the propagation of agent usage. So we will see a lot more initial adoption of AX principles and patterns from developer tools because the software industry has be the most infiltrated by the rise of agentic workflows. As that expands, the nature of markets and meeting user needs will drive adoption of AX.
- In this world the information delivered to agents should align with content delivered visibly to the human web. This is essentially how the bulk of SEO overloading is detected. There needs to be a way to validate this and establish trust - completely solvable. These techniques penalize these schemes from the outset. (this is probably not the best forum to go too deep into that)
- We're assuming agents have full buying decisions here. I do not believe we will see that as common place for a long time. Even if we did, the same systems for PCA compliance are in play and the interfaces pushed by both payment gateways and shopping carts protect against duplicate purchase attempts. Those attempting to abuse this fall more into the malicious actor camp.
- phishing and malicious actors are going to do what they have always done. There are some very important security, access control, and compliance measures we should put in place for the most sensitive of actions - as we always have where most existing ones still apply. The agent experience and the ecosystem in general will have to evolve to have verifiable trust patterns. So that when a human delegates to an agent to do something, the human can have confidence and ways to validate interactions.
I'll be the first to admit that I don't have all of the answers here but with agents becoming the new entry point or delegation tool for the next generation of digital users, these are questions we have to answer and solve for. It starts by focusing the industry around the domain of this problem, that is AX. How to do it effectively and what needs to evolve to achieve it... that's where the work is.