Definition & Explanation
The tendency of people to trust the output of an automated system and evaluate it less critically than information from other sources, even when there are signals that the output is wrong. The EU AI Act explicitly names automation bias in Article 14 on human oversight: anyone overseeing a high-risk AI system must remain aware of this tendency.
Article 14(4)(b) EU AI Act requires that persons overseeing a high-risk AI system remain aware of the possible tendency to automatically or excessively rely on the output produced by the system. This is one of the few places where the legislator wrote a psychological phenomenon directly into the legal text. Providers must design their systems so that effective oversight is possible, and deployers must, under Article 26, assign oversight to people with the necessary competence, training, and authority. AI literacy under Article 4 is also relevant: staff need to know when to distrust AI output.
A recruiter receives a ranked shortlist of candidates from an AI system and adopts the ranking without reviewing the underlying CVs. Or a doctor follows an AI diagnosis even though the clinical picture points in another direction. In both cases a human is formally part of the decision, but that human functions as a rubber stamp. Research in fields such as law enforcement and medical imaging shows the effect grows stronger as the system is right more often and workload increases.
The misconception is that putting a "human in the loop" solves automation bias by itself. Without training, sufficient time per decision, and the organizational freedom to deviate from the AI output, human oversight becomes a formality. Effective oversight requires measurable pushback: how often does the reviewer overrule the system? An override rate of nearly zero is usually not a sign of a perfect system, but of automation bias.
