This is an important issue when thinking ahead of the technological curve and where we are headed with Law and Governance both in amd out of the tech world. Personally I believe we are pretty clearly heading towards an inevitable 'Goovernment' which would be a merger of Big Tech and government itself.

New Hampshire Employment Security (NHES) has launched an "AI Adjudication Assistant" to process unemployment claims, aiming to streamline fact-gathering and speed up the benefits process. The system engages claimants in a conversational interface to collect details about their job separation.

Recent screenshots here; https://imgur.com/a/34FtrCC from the live public portal (adjudication.assistant.nhuis.nh.gov) confirm the system is "Powered by Gemini" (by Google). This represents one of the first direct integrations of a commercial, public-facing LLM into a core state-level government administrative function ( afaik )

While the stated goal is efficiency, this implementation has significant technical and governance implications:

    Outsourcing of Core Logic:

The state is no longer just using a cloud provider for infrastructure (IaaS) or software (SaaS), but is now outsourcing a component of its adjudicative logic to Google's third-party AI model (MaaS - Model as a Service). The initial process of summarizing a citizen's case for a human reviewer is being handled by a commercial, closed-source system in Gemini.

    Data Sovereignty and Security:

The system processes a high volume of sensitive Personally Identifiable Information (PII). The data flow now extends beyond state-controlled servers to Google's API endpoints. This raises critical questions about data handling, retention policies for API calls containing PII, and whether this data could be used for training future models. The attack surface expands from a state-run web site component to the complex security environment of a commercial LLM.

    Accountability and the "Black Box":

The NHES FAQ confirms that a human adjudicator makes the final decision. However, this human-in-the-loop is acting on a summary generated by the AI. If a flawed or biased summary from the Gemini model leads to a wrongful denial of benefits, the chain of accountability is now unclear for anyone in such a position, as this is all still in such early phases.

    Vendor Lock-In:

By building the workflow around a specific proprietary model, the state creates a significant dependency. Migrating to a different model provider or an "in-house solution" (the old way) in the future would require substantial technical effort and retraining, granting the vendor considerable long-term leverage.

This new system being tested in New Hampshire serves as a case study in the direct merger of Big Tech and state governance. It highlights a trade-off between the immediate efficiency gains of using powerful, pre-existing models and the long-term risks associated with dependency, security, and public accountability.

[Sources]: Route Fifty article on the modernization push: https://www.route-fifty.com/artificial-intelligence/2025/03/new-hampshires-benefits-program-embraces-ai-amid-modernization-push/403448/