EU sets strict limits on the use of AI-biometric surveillance by police

It has been revealed that in most cases, police and national security agencies in the EU are not allowed to use real-time data generated by artificial intelligence without a judicial authorization.

This measure is part of an historic agreement that was reached on Friday between the European Parliament and EU member countries after three days’ negotiations. Officials only revealed operational details Monday as the final text won’t be released until a “cleaning-up process” has been completed.

In public and private areas, from sports fields to parks, the ban on “Big Brother’s” surveillance will be in effect, except when there are specific serious crimes committed, terrorist threats or urgent searches of victims. The police will still need to get approval from an independent administrative authority or judge before they can begin surveillance.

Police will only be able turn on AI biometrics in exceptional circumstances such as an actual terrorist threat.

According to a EU official familiar with the text, the new rules still require that they obtain authorization within 24 hours, and submit a “prior impact assessment on fundamental rights” to the relevant authority.

The authorities must also inform the market surveillance authority as well as the data protection authority. The official stated that if the authority or judge does not grant permission, then the AI tool should be turned off, and any data collected on the suspect or suspects must be deleted immediately.

Officials claim that these safeguards would prevent what is called “predictive police”, which MEPs were concerned could be used in conjunction with racial profiles to discriminate individuals. According to an EU official, law enforcement agencies would no longer be allowed “to pursue someone as suspect just because an algorithm says that you are a crime”.

The EU and MEPs agreed on a list of 16 specific types of serious crimes where this exception may be applied.

Excepted from the list were crimes that fall under the jurisdiction and authority of the International Criminal Court, such as terrorism, murder or rape, organized or armed robbery or kidnapping. Other exceptions include crimes committed by the International Criminal Court, sabotage or unlawfully seizing an aircraft or ship, trafficking in human body parts or illegal drugs or weapons, or participation in criminal organisations involved in these offences.

The EU Gazette or the Official Journal, which is the EU’s official gazette, will publish the new laws governing artificial intelligence in six months. The EU’s new laws on artificial intelligence will include additional bans that are designed to protect society from the dangers AI poses.

Included are AI systems and applications that “manipulate user behaviour in order to circumvent their free will”. For example, toys with voice-assistance encourage dangerous behavior among minors. The ban also includes AI systems which allow governments and companies to “score” socially, similar China’s system of “social credit”.

In the final text, the EU made sure to include this clause, as it was worried that AI-derived data could be exploited to exploit people’s vulnerabilities based on their age, disabilities, social or economic status. This could have meant that, for example, certain customers would be barred from certain restaurants because of their “social score”, or status. Or that certain employees might be denied jobs.

Artificial intelligence that can assess stress and fatigue by reading facial expressions will be banned in the workplace.

MEPs who led the fight against prohibitions wanted to make sure that the EU didn’t become a surveillance state, like China, in which the traffic police could intervene, say, if the believe a truck driver is fatigued.