Uber Facial Recognition: Driver Concerns & Privacy Issues

Facial Recognition System Under Scrutiny at Uber in the U.K.

Uber’s deployment of facial recognition technology as part of a driver identification process is facing legal challenges in the United Kingdom. The App Drivers & Couriers Union (ADCU) and Worker Info Exchange (WIE) are requesting that Microsoft halt Uber’s utilization of its business-to-business facial recognition system, citing numerous instances of driver misidentification.

Misidentification and Licence Revocations

The ADCU reports identifying seven cases where inaccurate facial recognition and other identity verification procedures resulted in drivers losing their jobs and having their licenses revoked by Transport for London (TfL).

The “Real Time ID Check” System

Uber initially launched the “Real Time ID Check” system in the U.K. in April 2020. The company stated its intention was to ensure driver accounts were exclusively used by licensed individuals who had undergone Enhanced DBS checks. Drivers were given the option of having their selfies verified either through photo comparison software or by human reviewers.

Cases of Incorrect Identification

In one specific instance, the ADCU stated a driver was dismissed by Uber and subsequently had their license revoked by TfL. The union successfully intervened, establishing the driver’s correct identity and prompting Uber and TfL to reverse their decisions. This highlights concerns regarding the precision of the Microsoft facial recognition technology.

Concerns About Accuracy and Bias

The ADCU points to Microsoft’s temporary suspension of sales of the facial recognition system to U.S. police departments following the Black Lives Matter protests. Research indicates that facial recognition systems can exhibit higher error rates when identifying people of color. A 2018 MIT study revealed Microsoft’s system could have an error rate as high as 20%, with the lowest accuracy observed for dark-skinned women.

Calls for Review of TfL Revocations

The union has written to the mayor of London, requesting a comprehensive review of all TfL private-hire driver license revocations based on reports from Uber utilizing evidence from its Hybrid Real Time Identification systems.

Microsoft’s Response

Microsoft has acknowledged the concerns and stated its commitment to ongoing testing and improvement of the Face API, with a particular focus on fairness and accuracy across diverse demographic groups. The company also provides guidance and tools to customers for assessing fairness within their systems.

Uber’s Implementation and Regulatory History

The ADCU asserts that Uber swiftly implemented the workforce surveillance and identification system as part of efforts to regain its operating license in the U.K. capital.

Previous Licence Issues with TfL

In 2017 and 2019, TfL declined to renew Uber’s license, citing safety and security failures and deeming the company “not fit and proper” to hold a private hire vehicle license. Uber successfully appealed these decisions in court, but the most recent renewal was for only 18 months and included numerous conditions.

Lack of Regulatory Standards

Labor activists are now raising concerns that no regulatory standards have been established regarding the workplace surveillance technology that the ADCU claims TfL encouraged Uber to implement. Furthermore, TfL has not conducted an equalities impact assessment.

Discrimination Claim Filed

WIE has confirmed it is filing a discrimination claim on behalf of Imran Raja, a driver who was dismissed after Uber’s Real ID check and subsequently had his license revoked by TfL. His license was later reinstated following union intervention.

Further Appeals Planned

Additional Uber drivers who were misidentified by the facial recognition checks are preparing to appeal TfL’s license revocations through the U.K. courts, as per WIE.

TfL’s Position

A TfL spokesperson clarified that implementing facial recognition technology is not a condition of Uber’s license renewal. The requirement is that Uber maintains adequate safety systems.

Data Protection Impact Assessment

Requests have been made to TfL and the U.K.’s Information Commissioner’s Office for a copy of the data protection impact assessment Uber claims was conducted before launching the Real-Time ID Check.

Uber’s Defense

Uber disputes the union’s claim that its use of facial recognition technology for driver identity checks poses a risk of automating discrimination. The company maintains it has a manual review system in place to prevent such failures.

System Details

Uber’s Real-Time ID system utilizes an automated “picture matching” check, comparing a driver’s selfie with a single photo on file. If no machine match is found, the query is sent to a three-person human review panel for manual verification. A second panel is used if the first cannot reach a consensus.

Human Error and Appearance Changes

In some cases, Uber attributes misidentifications to human error by its review teams. One instance involved a driver whose appearance had changed (growing a beard), and staff failed to recognize him in the older, clean-shaven photo on file.

Data Transparency Concerns

Uber declined to provide details regarding the outcomes of the other five identity check failures cited by the union, or the ethnicities of the misidentified drivers.

Geolocation and Driver Deactivations

WIE has evidence suggesting that facial recognition checks are integrated into geolocation-based deactivations carried out by Uber. In one case, a driver received an explanation solely related to location, but a witness statement accidentally sent by TfL to WIE revealed the inclusion of facial recognition evidence.

Questions About Human Oversight

Concerns remain regarding the extent to which Uber’s human review staff can override machine suggestions, particularly considering the company’s business imperatives and need to demonstrate regulatory compliance.

The Role of Human Bias

James Farrer, founder of WIE, questions the quality of Uber’s human checks, given the known discrimination problems associated with facial recognition technology. He suggests that humans may be biased to confirm machine findings and lack the confidence or support to overrule them.

Driver Evidence and Appeals

Farrer also points out that Uber has previously prioritized customer complaints over driver concerns, raising doubts about its ability to make balanced decisions using facial recognition.

Governance and Responsibility

Farrer emphasizes the importance of intelligent and responsible governance of facial recognition technology, acknowledging Microsoft’s recognition of its limitations.

Broader Legal Battles

This latest pressure on Uber’s processes follows recent legal victories for Farrer and other labor rights activists regarding the company’s classification of drivers as “self-employed” rather than workers under U.K. law.

Uber’s Response to the Supreme Court Ruling

Following the Supreme Court’s ruling, Uber announced it would treat drivers as workers and expand the benefits it provides. However, litigants argue that Uber’s offer still underpays drivers by 40%-50% of their legal entitlements and that the legal fight will continue.

EU-Level Advocacy

At the EU level, Uber is advocating for an employment law carve-out for platform work and has been accused of attempting to lower legal standards for workers.

Data Disclosure Orders

In a separate case in the Netherlands, a court ordered Uber to disclose more data on its drivers, following a challenge by the ADCU and WIE. The court also allowed drivers to collectively seek data to support collective bargaining efforts.

Algorithmic Decision-Making

The issue of meaningful human intervention in algorithmic decisions is emerging as a key battleground in the regulation of powerful platforms and their societal impacts.

Ongoing Scrutiny

The challenges to Uber’s use of facial recognition-linked terminations demonstrate that scrutiny of its automated decisions is far from over. The interrogation of the limits and legality of these decisions is only just beginning.

Further Legal Challenges

Uber’s use of geolocation for driver suspensions is also facing legal challenge, and pan-EU legislation is being negotiated to increase platform transparency requirements.

Similar Case in the Netherlands

A court in the Netherlands also ordered India-based ride-hailing company Ola to disclose data about its facial-recognition-based “Guardian” system.

Tenacity in Pursuit of Fairness

Farrer remains confident that workers will achieve transparency, citing his years of fighting Uber through U.K. courts and his unwavering commitment to rebalancing platform power.