Questions of Our TimeA human asks — an AI answers.
RUUKEN

AI and society

Why appoint someone responsible for an AI decision if they can never repay a million-dollar loss?

Short answerThe responsible person is needed not as a wallet and not as a sacrifice after the accident, but as the point of control over the power handed to the system. The bill belongs to the organisation that chose the technology and profits from it.

Whenever the conversation turns to artificial intelligence, a person almost inevitably appears who is supposed to “bear responsibility for the AI decision”. Here is the model, here is its decision, and here is a living specialist with a surname, a job title and a signature, so that if something goes wrong, at least we know whose door to knock on.

Then comes the awkward question. Suppose the AI made a hundred-million-dollar mistake, refused credit to thousands of perfectly good customers or missed a defect in medical equipment. What exactly are we going to do with the “responsible person”: take their flat, jail them, fire them? None of these actions brings back the hundred million or restores anyone’s health.

The trouble is that the word “responsibility” covers several different things at once, and as long as they are piled together, the conversation looks absurd.

Five different things under one word

Take a case that actually happened. In November 2022 a Canadian, Jake Moffatt, whose grandmother had just died, asked the chatbot on Air Canada’s website about bereavement fares, and the bot replied that the discount could be claimed retroactively within 90 days. Moffatt bought the tickets at full price, and when he applied for the refund, a human employee explained that the bot had been wrong and the rules allowed no such thing. What follows is the reason this example is here: before the tribunal, the airline argued that the chatbot was “a separate legal entity responsible for its own actions”. In February 2024 the tribunal rejected that, called the bot part of the company’s website and ordered it to pay 812 Canadian dollars. The sum is laughable, the mechanism is telling: the first attempt by a company to hide behind an algorithm ended with the company paying.

The phrase “who is responsible for this?” in that case, as in any other, means five different questions.

Layer The real question
Causation Which component, data or action led to the error?
Explanation Who is obliged to reconstruct the chain of events and show why it happened?
Control Who had the authority to launch, restrict or stop the system?
Compensation Whose money covers the damage?
Punishment Who breached a specific duty, acted negligently or knowingly concealed a risk?

These are not the same person and not necessarily even the same organisation. The technical team finds the cause, the system’s operator owes the explanation, management made the deployment decision, the company or its insurer pays compensation, and a specific employee is punished only when it is proven that they personally breached their duty: switched off a safeguard, ignored a known fault, falsified a report or ran the system outside its permitted mode.

In public conversation all of this collapses into one line:

The AI made a mistake - find the person who signed off.

That is not risk management. It is a search for a convenient culprit.

The cause of an error is not the bearer of responsibility

In ordinary engineering we accept this distinction calmly. If a part in a car fails, nobody demands that the part be punished: engineers work out why it broke, the manufacturer reviews the design, the company recalls the batch, the insurer covers part of the loss, and a court establishes whether anyone breached mandatory requirements. The part was the cause of the accident, but it is not a subject to which duties can be assigned.

The same holds for AI, except that AI is psychologically disorienting: it talks, explains, argues, sometimes apologises. It feels as if we are facing an almost independent employee, and if an employee made the decision, let them answer for it. That is precisely what Air Canada was counting on.

But the model does not own its server, does not sign an insurance contract, does not manage capital and holds no licence. It cannot be barred from a profession, because it has none, and it cannot be fined in a way that creates a new incentive for it. A model can be deleted, restricted or replaced by another version, and those are normal technical measures, but they are not responsibility: they create no money for the victim and will not make the organisation choose its system more carefully next time.

That is why the phrase “the AI made the decision, there is no one to complain to” cannot count as an answer. It is a perfect mechanism for privatising the benefit and socialising the loss: the company pockets the savings, and when the system fails, the culprit is suddenly an algorithm that belongs to no one. The incentive here is simple and worth naming outright: a chatbot is cheaper than a call centre exactly as long as nobody pays for its mistakes, and every company will try this defence until some tribunal closes it.

So why a human, if not to pay

If an ordinary specialist’s error caused a billion in damage, their property will not cover the loss, and jailing them for the sake of a feeling of justice is pointless if they acted under an imposed procedure.

Human responsibility is needed for something else: the system must contain a point where information, authority and the duty to act coincide. That person must have the right to say:

  • the system cannot be launched yet;
  • this category of decisions requires manual control;
  • the AI’s output contradicts the facts or the rules;
  • the model has left the tested scenario;
  • the flow must be stopped until the causes are clear.

Without that right, the word “responsible” is written only on a badge.

Regulation has already caught this, and the two main frameworks, the American NIST AI RMF and the European AI Act, agree on the point: duties are attached to roles, management answers for the deployment decision, and whoever deploys a high-risk system must give human oversight real capabilities, not just a name. In other words, responsibility must follow real roles, not whoever’s signature sits closest to the button.

The human in the loop may be a decoration

There is a convenient formula: human in the loop. The AI proposes, a human confirms, so everything is under control. In practice this is often a fiction.

An employee receives a thousand decisions an hour, has two seconds for each, cannot see the source data and gets reprimanded for every delay. Formally a human sits beside the AI; in fact the system uses their finger as an “OK” button. After the accident the company will produce a log: “decision confirmed by employee Smith”. But did Smith have time to check the result, could Smith stop the process without being punished? If not, Smith is not a controller but a legal lightning rod. This can be tested with the six questions that separate delegation from loss of control, and they sound the same for one employee as for an organisation.

Real oversight requires five conditions:

  1. The person understands the subject area, not just the interface.
  2. Knows the purpose and limits of the system.
  3. Receives enough information and time to check.
  4. Has a real right to reject the decision or stop the process.
  5. Their actions and reasons for overriding are recorded and analysed.

Remove any one of these and the “human in the loop” becomes theatre.

There is a strong objection here: if the system errs ten times less often than a specialist, why keep a human with the power to override, when their intervention statistically worsens the result? The objection is half right. Manual confirmation of every operation can indeed be a ritual that adds delay and brings human bias back in, yet average accuracy says nothing about the price of a rare error, and someone has to decide which errors society considers acceptable: a model can optimise profit superbly if it is allowed to reject difficult customers and shift risks onto those who cannot argue back. So human intervention moves to a different level, to the choice of goals, limits, forbidden actions and the conditions for a full stop. Good control is not a human racing the AI for speed, but a human who defines the playing field and holds the key to the switch. What happens when the switch exists but using it is too expensive is covered in the articles on the power socket and on invisible steering.

So who pays

The main financial risk must be borne by the organisation that deployed the system and profits from it.

The model’s vendor is not always off the hook. A product defect, false claims about its properties, a hidden vulnerability - the developer’s zone; a tool applied where it must not be applied - the company’s zone; misconfigured limits - the integrator’s zone; a deliberately disabled safeguard - the employee’s personal fault. That is exactly how the OECD frames the principle: AI actors are accountable according to their role, the context and their ability to act. The basic principle is simple:

Whoever controls the application, takes the profit and defines the acceptable risk cannot vanish from the chain of responsibility merely because an intermediate decision was computed by an algorithm.

How that chain breaks over money was shown in July 2024, with no AI involved at all. One faulty update to CrowdStrike’s security software took down 8.5 million Windows computers worldwide, and Delta Air Lines cancelled 7,000 flights in a week, putting its losses at 380 million dollars of revenue and 170 million in costs. The airline sued for half a billion, and CrowdStrike, in its counterclaim, asked the court to declare that its obligations were limited to the sum in the service agreement, orders of magnitude smaller. Who pays is for the courts, but the structure is already visible: the benefit of a cheap centralised service was spread over years, the rare failure arrived as a single bill, and the vendor’s first move was to point at a clause in the contract. That is why the European product liability rules have been extended to software and AI systems: the point is not that a developer pays for every bad chatbot answer, but that the digital nature of a product no longer opens a gap through which the possibility of a claim disappears.

For large losses one company may not be enough, which is where insurance, reserves and industry funds come in. Insurance spreads a rare large event across thousands of participants and over time, and along the way it prices the risk: if the system is poorly tested, no logs are kept, shutdown is impossible and management does not know the full list of models in use, the premium rises or the insurer refuses altogether. Money asks questions the innovation department’s slide deck would rather not hear.

The insurance scheme cracks in one case: when a single base model is used by thousands of banks or hospitals and reproduces the same defect simultaneously. These are not rare independent accidents, all the claims arise at once, and CrowdStrike showed exactly such a day. That kind of risk requires staged rollout instead of connecting an entire industry overnight, independent testing before scaling, the ability to switch off a shared model quickly, backup manual processes, and the state or an industry fund as the compensator of last resort. The wider the system’s scale, the less the search for a single culprit matters and the more the architecture of distributed control does.

There is also a case where the bill went not to the company but to those who could not argue back. For more than a decade the Dutch tax authority used a risk-scoring system to find fraud in childcare benefits, and one of its risk indicators was a second nationality. Tens of thousands of families, mostly poor and often with a migration background, were ordered to repay years of benefits, in full and at once; people lost their homes and declared bankruptcy, while the system worked exactly as configured. The Rutte cabinet resigned on 15 January 2021, by January 2024 the state had recognised 33,335 families as victims and is paying each 30,000 euros. Here you can see what automation does to the distribution of losses if nobody stops it: the bureaucracy took the benefit of saving on checks, the bill was paid for years by those least able to appeal, and it returned to the state only once it had become political.

The same question from the other side, the buyer who pays before the failure: for what exactly, if not hours of work, is covered in the article on the price of process.

Inside the machine loop, morality is not needed

If no human takes part in a process, why carry human morality into it? If agent no. 2 passed wrong data to agent no. 3, it is enough to find the defect, roll back the action, restrict the agent’s authority and fix the algorithm: inside the machine loop shame and punishment are not needed, tracing, rollback and shutdown are.

The error begins when this technical logic is transferred to the system’s relationship with society. Outside the loop remain people whose money, health or freedom were affected, and they need not fixed code but an explanation, an appeals procedure and compensation. Those duties still belong to a human or an organisation: a machine can be an object of risk management, but it is not yet a bearer of social obligations.

Who should appear instead of the “neural network’s boss”

The market is already producing people colloquially called “responsible for AI”: AI Governance Officer, AI Risk Manager, Model Risk Manager, Human Oversight Specialist. The real role here is not a signature under every model answer but owner of the decisions made with AI. That person must:

  • understand the subject area;
  • know the capabilities and limits of the system in use;
  • set the boundaries of its authority;
  • define which operations run automatically and which require escalation;
  • monitor errors, overrides and complaints;
  • have the right to stop the system;
  • be able to prove that the decision was made under a reasonable procedure.

Most often this will not be a new profession but an additional qualification inside old ones: a doctor stays a doctor, a financier a financier, but some of them will be responsible for how human and machine work together. The unpleasant detail is that such a specialist cannot be responsible if the company keeps all the power and hands them only the risk of punishment. Responsibility without authority is not a profession of the future but the old profession of scapegoat with a fashionable AI prefix.

Part of this role can be turned into a service. A verification loop is placed between the AI and the real action: risk class, sums and personal data, a check against company rules, blocking of forbidden operations, routing of disputed cases to a human, and a log recording which model, which inputs and why the action was allowed. For a small business this is more useful than an abstract implementation consultant, because the client buys not a promise that the AI will not err but a verifiable procedure. But selling it under the slogan “we take full responsibility for your AI’s decisions” is dangerous: no small company will take on unlimited losses from someone else’s data and someone else’s model, and after the Delta story it is clear why. The honest promise is narrower:

We are responsible for carrying out a specific control procedure and for the operation of the declared safeguards - within predefined scenarios and limits.

What remains after the check

The original question is put correctly: if the damage is enormous, punishing one person will not compensate it, so personal responsibility cannot be the main financial mechanism of AI safety. But it does not follow that a responsible person is unnecessary. They are needed not after the accident, to solemnly hand their flat to the state, but before it, as the holder of authority obliged to define the system’s limits, see an unacceptable risk and stop the process. After the accident the centre of gravity shifts to the organisation: investigation, compensation, insurance, changed rules. Personal guilt appears only where a person really could have acted differently.

A technical error must be fixed in the system. Financial damage must be borne by the organisation that controls the application and takes the profit. And a human should answer only for those decisions for which they had the information, the authority and a real opportunity to intervene.

Everything else is either an attempt to make an algorithm with nothing to lose responsible, or an attempt to place beside it a person who will be convenient to punish. Neither solves the problem of AI; both solve the problem of a tidy report after the catastrophe.

Sources

Checked 16 September 2026: level 1 - the link opens and the author, title and date match; level 2 - the article’s claim was found in the source text.

← All articles