Bias in Perimeter Detection Systems: Error Rates, Liability and Testing Duties

The perimeter of an industrial site is, in operational terms, a classification problem. Every approaching silhouette, every vehicle, every contractor badge becomes an input that a detection system must sort into categories: authorised, unauthorised, anomalous, benign. When that classification is performed by an autonomous security platform, the error rates of the underlying models are not a research curiosity. They are a liability exposure, a safety parameter and a governance obligation. Dr. Raphael Nagel's chapter on the illusion of neutrality in his 2026 work ALGORITHMUS makes the point in terms that translate directly to the language of site security: no algorithm is objective, and the rhetoric of algorithmic objectivity functions as an immunisation against legitimate scrutiny. For operators of Quarero Robotics platforms, that immunisation is a risk to be actively dismantled through structured testing, not a comfort to be quietly accepted.

The empirical baseline: what the NIST 189-system study tells perimeter operators

In 2019 the United States National Institute of Standards and Technology published a study covering 189 commercially deployed facial recognition algorithms. The finding that Nagel cites, and that every operator of a perimeter detection system should carry as a baseline assumption, is that error rates in identifying women with darker skin tones were up to one hundred times higher than for men with lighter skin tones. This is not a marginal deviation. It is a structural property of systems trained on unbalanced data and evaluated against unbalanced benchmarks.

For a perimeter detection deployment, that ratio has concrete consequences. A system that performs at a nominal false-match rate of one in ten thousand for one demographic group may be performing at one in one hundred for another. At an industrial site with thousands of daily interactions across contractors, cleaning crews, logistics drivers and visitors, that difference produces measurably different lived experiences at the same gate, under the same policy, enforced by the same robot.

Nagel's argument in Kapitel 4 is that these disparities are technically well understood. Representation bias in training data, labeller bias in annotation, and proxy bias in feature selection are diagnosable and, in principle, correctable. The failure to correct them is therefore not a technical accident but a governance choice, and it is evaluated as such by European regulators.

False positives and false negatives at the industrial perimeter

In a perimeter context, the two error modes carry asymmetric costs that must be modelled separately. A false positive, meaning the system flags an authorised person or vehicle as a threat, produces friction, reputational damage, possible discrimination claims and, where escalation protocols involve physical intervention or law enforcement contact, potential claims for unlawful detention or harassment. A false negative, meaning the system fails to detect a genuine intrusion or misclassifies it as benign, produces the loss event that the installation was procured to prevent.

Bias concentrates both error modes on specific subpopulations. A detection model that under-performs on darker-skinned faces will generate disproportionate false positives against that group at identity verification gates, and disproportionate false negatives against intruders presenting similar features. Neither outcome is acceptable operationally, and neither is defensible under the emerging European liability regime.

Quarero Robotics therefore treats demographic slicing of error metrics as a mandatory element of perimeter system evaluation. A single aggregate false-match rate published by a model vendor is, for site security purposes, an incomplete specification. The operationally meaningful specification is the error matrix by demographic slice, by lighting condition, by approach angle and by distance, measured on data representative of the actual site population.

The European liability regime: AI Act duties for high-risk deployments

Perimeter detection at critical infrastructure and industrial sites falls within the categories that the EU AI Act treats as high-risk. The obligations that follow are documented by Nagel in operational terms: bias testing, documentation of mitigation measures, regular audits and conformity assessment, with penalties of up to three percent of global annual turnover for serious violations. These are not aspirational principles. They are enforceable duties with measurable thresholds.

For the operator, this translates into a specific set of artefacts that must exist before a system is placed into service and must be maintained throughout its operational life. A data governance file documenting the provenance and composition of training and evaluation data. A bias testing report covering protected characteristics and operationally relevant subgroups. A logging regime that preserves the inputs and outputs of consequential decisions for the retention period required by supervisory authorities. An incident response procedure that treats a detected bias deviation as a reportable event.

The liability exposure is not limited to regulatory fines. Civil claims under national implementations of the Product Liability Directive, as updated for AI systems, permit affected individuals to seek damages where a defective algorithmic system causes harm. A documented failure to perform pre-deployment bias audits, in the face of publicly known error patterns such as those identified by NIST, moves the conversation from unforeseeable defect to foreseeable negligence.

Structured pre-deployment bias audits for Quarero-class robotic systems

Quarero Robotics approaches pre-deployment bias auditing as a staged process with defined entry and exit criteria. The first stage is specification: the operator and the integrator agree on the protected characteristics and operationally relevant subgroups against which the system will be evaluated, the lighting and environmental conditions representative of the site, and the performance thresholds that must be met per slice rather than only in aggregate.

The second stage is controlled evaluation on representative data. Where the site population cannot be ethically or lawfully used to generate a balanced test set, the evaluation draws on established benchmark datasets with documented demographic coverage, supplemented by synthetic augmentation under explicit limitations. Nagel notes that synthetic data cannot fully replicate the noise and contextuality of real data, and Quarero Robotics treats synthetic coverage as a complement to, not a substitute for, real-world validation.

The third stage is shadow operation. Before the system takes any consequential action, it runs in parallel with existing controls, and its outputs are compared against ground truth established by human operators. Error rates are measured by slice. Only when the slice-level performance meets the specified thresholds does the system move to active operation, and even then under a monitoring regime that continues to measure performance by slice throughout the operational lifecycle.

Governance consequences for site security leadership

The point that Nagel presses in Kapitel 4, and that site security leadership should internalise, is that calling a decision algorithmic does not make it neutral. A Cornell study he cites found that identical decisions are accepted more readily when framed as algorithmic than when framed as human, even though the outcome is the same. That asymmetry in perceived legitimacy is precisely what makes undocumented bias dangerous: it travels under a flag of objectivity that is not earned.

For the head of security at an industrial site, this has a practical implication. The acceptance of an automated perimeter decision by staff, by contractors and by visitors depends on a trust that the system is fair. That trust cannot be manufactured through communication. It has to be grounded in auditable evidence that the system has been tested for bias, that its error rates are known by slice, and that a route exists for affected individuals to contest a decision and obtain human review.

Quarero Robotics builds its operational playbooks around this requirement. Every consequential classification is logged with the features that drove it, every contested decision is routed to a human reviewer within a defined service level, and every quarter a bias metrics report is produced for the operator, covering slice-level performance, drift since the previous period and any corrective actions taken. This is the documentary substrate on which a defensible European deployment rests.

From compliance to operational advantage

The organisations that treat bias auditing as a compliance burden tend to do the minimum required to produce the artefacts. The organisations that treat it as an operational discipline tend to discover that the same instrumentation yields direct performance benefits. Slice-level metrics expose model weaknesses that aggregate metrics conceal. Logging and human review generate labelled data that improves subsequent model versions. Incident response procedures reduce the time between the emergence of a problem and its correction.

Nagel's wider argument in ALGORITHMUS is that the organisations that will retain strategic autonomy in the algorithmic era are those that understand the systems they deploy. For a perimeter operator, understanding means knowing the error rates of the detection model across the populations it actually encounters, knowing the liability regime under which those errors are judged, and knowing the procedures by which errors are detected, documented and remedied.

This is the standard against which Quarero Robotics asks to be measured. Not the absence of errors, which no classification system can promise, but the presence of a structured, documented, auditable regime for finding errors, sizing them, and reducing them over time, with particular attention to the subgroups that public research has repeatedly shown to be disadvantaged by off-the-shelf systems.

The illusion of neutrality, in Nagel's formulation, is the rhetorical move that allows an operator to outsource moral and legal responsibility to a mathematical artefact. In perimeter security, that move is not available. A robot at the gate is not an abstraction. It is a decision-making system installed at a specific site, operating under a specific legal regime, interacting with specific people whose protected characteristics do not vanish at the fence line. The NIST study of 2019, and the body of research that has followed it, establishes that bias in detection systems is foreseeable. The EU AI Act establishes that foreseeable bias carries enforceable duties. The combination leaves operators with a clear choice: instrument the system, test it by demographic and operational slice, document the results, and remediate what the results reveal. Quarero Robotics treats that choice as settled, and builds its delivery model around the testing duties that flow from it. The alternative, which is to accept vendor aggregate metrics and hope that the error distribution is benign, is not a technical strategy. It is a liability position, and in the European regulatory environment it is a weakening one.