Public Judgment Is Not a Feature You Can Outsource

If you use a private AI tool to draft your emails faster, that is your choice. If a school, court, or welfare office uses a private AI tool to decide what happens to you, that is a different category of decision entirely.

Right now, those lines are blurring. Governments and public institutions are under pressure to be “modern” and “data-driven”, and AI products are being marketed as the quickest way there. It is easy enough to plug in a commercial model to rank applications, triage cases, summarise files, or flag “risky” individuals. It is much harder to answer the question of who is accountable when the model is wrong?

In theory, AI is there to “support” decisions. In practice, we know how this goes. Once a tool is in the workflow, people start treating its output as default truth, especially in under-resourced systems where staff are overloaded, undertrained, or scared of being blamed for mistakes. An AI risk score in a bail decision, a predicted “dropout probability” for a student, or a “fraud likelihood” for a welfare recipient may be framed as advice, but it becomes the baseline from which human decision-makers must justify any deviation.

That is not a small change at all. Public institutions are supposed to do something more than optimise for speed. They are supposed to exercise judgment in a way that is explainable, contestable, and anchored in public legitimacy, not just in technical performance metrics. When a model misclassifies someone, telling them “the system says so” is not a good enough reason. 

There are also structural problems with outsourcing core judgment to private tools. First, opacity. Many frontier and off‑the‑shelf models are effectively black boxes to the public institution using them. Staff may not know what data the system was trained on, how it behaves across different groups, or how often it fails in edge cases. Second, dependency. Once a high‑stakes workflow is built around a specific vendor’s system, switching away becomes expensive and politically painful, even if problems emerge.

Third, and maybe most important, there is the question of whose values and trade‑offs get encoded inside the system. Private companies are answerable to their shareholders and regulators, not directly to the citizens who experience the consequences of their models in schools, hospitals, or welfare offices. When a public institution lifts a tool built for a different context and plugs it into its own decisions, it is importing someone else’s priorities without admitting it.

None of this means public institutions must avoid AI entirely. It does mean they need a different standard. If you are going to let AI touch public decisions, a few things should be non‑negotiable.

One, the institution remains clearly accountable for outcomes, not the vendor.  

Two, people affected by AI‑shaped decisions have a right to an explanation and a meaningful appeals process.

Three, the systems are auditable, with evidence on performance across groups and contexts, not just marketing promises.

Four, procurement is treated as governance. For instance, contracts must include transparency, testing, and exit options, not just feature lists.

The risk is that public institutions will slowly let their responsibilities leak into private systems until nobody can say where judgment actually lives. By the time we notice, it might have already done harm.