16 July 20
ationwide protests against racist policing have brought new scrutiny onto big tech companies like Facebook, which is under boycott by advertisers over hate speech directed at people of color, and Amazon, called out for aiding police surveillance. But Microsoft, which has largely escaped criticism, is knee-deep in services for law enforcement, fostering an ecosystem of companies that provide police with software using Microsoft’s cloud and other platforms. The full story of these ties highlights how the tech sector is increasingly entangled in intimate, ongoing relationships with police departments.
Microsoft’s links to law enforcement agencies have been obscured by the company, whose public response to the outrage that followed the murder of George Floyd has focused on facial recognition software. This misdirects attention away from Microsoft’s own mass surveillance platform for cops, the Domain Awareness System, built for the New York Police Department and later expanded to Atlanta, Brazil, and Singapore. It also obscures that Microsoft has partnered with scores of police surveillance vendors who run their products on a “Government Cloud” supplied by the company’s Azure division and that it is pushing platforms to wire police field operations, including drones, robots, and other devices.
With partnership, support, and critical infrastructure provided by Microsoft, a shadow industry of smaller corporations provide mass surveillance to law enforcement agencies. Genetec offers cloud-based CCTV and big data analytics for mass surveillance in major U.S. cities. Veritone provides facial recognition services to law enforcement agencies. And a wide range of partners provide high-tech policing equipment for the Microsoft Advanced Patrol Platform, which turns cop cars into all-seeing surveillance patrols. All of this is conducted together with Microsoft and hosted on the Azure Government Cloud.
Last month, hundreds of Microsoft employees petitioned their CEO, Satya Nadella, to cancel contracts with law enforcement agencies, support Black Lives Matter, and endorse defunding the police. In response, Microsoft ignored the complaint and instead banned sales of its own facial recognition software to police in the United States, directing eyes away from Microsoft’s other contributions to police surveillance. The strategy worked: The press and activists alike praised the move, reinforcing Microsoft’s said position as a moral leader in tech.
Yet it’s not clear how long Microsoft will escape major scrutiny. Policing is increasingly done with active cooperation from tech companies, and Microsoft, along with Amazon and other cloud providers, is one of the major players in this space.
Because partnerships and services hosting third party vendors on the Azure cloud do not have to be announced to the public, it is impossible to know full extent of Microsoft’s involvement in the policing domain, or the status of publicly announced third party services, potentially including some of the previously announced relationships mentioned below.
Microsoft declined to comment.
Microsoft: From Police Intelligence to the Azure Cloud
In the wake of 9/11, Microsoft made major contributions to centralized intelligence centers for law enforcement agencies. Around 2009, it began working on a surveillance platform for the NYPD called the Domain Awareness System, or DAS, which was unveiled to the public in 2012. The system was built with leadership from Microsoft along with NYPD officers.
While some details about the DAS have been disclosed to the public, many are still missing. The most comprehensive account to date appeared in a 2017 paper by NYPD officers.
The DAS integrates disparate sources of information to perform three core functions: real-time alerting, investigations, and police analytics.
Through the DAS, the NYPD watches the personal movements of the entire city. In its early days, the system ingested information from closed-circuit TV cameras, environmental sensors (to detect radiation and dangerous chemicals), and automatic license plate readers, or ALPRs. By 2010, it began adding geocoded NYPD records of complaints, arrests, 911 calls, and warrants “to give context to the sensor data.” Thereafter, it added video analytics, automatic pattern recognition, predictive policing, and a mobile app for cops.
By 2016, the system had ingested 2 billion license plate images from ALPR cameras (3 million reads per day, archived for five years), 15 million complaints, more than 33 billion public records, over 9,000 NYPD and privately operated camera feeds, videos from 20,000-plus body cameras, and more. To make sense of it all, analytics algorithms pick out relevant data, including for predictive policing.
The NYPD has a history of police abuse, and civil rights and liberties advocates like Urban Justice Center’s Surveillance Technology Oversight Project have protested the system out of constitutional concerns, with little success to date.
While the DAS has received some attention from the press — and is fairly well-known among activists — there is more to the story of Microsoft policing services.
Over the years, Microsoft has grown its business through the expansion of its cloud services, in which storage capacity, servers, and software running on servers are rented out on a metered basis. One of its offerings, Azure Government, provides dedicated data hosting in exclusively domestic cloud centers so that the data never physically leaves the host country. In the U.S., Microsoft has built several Azure Government cloud centers for use by local, state, and federal organizations.
Unbeknownst to most people, Microsoft has a “Public Safety and Justice” division with staff who formerly worked in law enforcement. This is the true heart of the company’s policing services, though it has operated for years away from public view.
Microsoft’s police surveillance services are often opaque because the company sells little in the way of its own policing products. It instead offers an array of “general purpose” Azure cloud services, such as machine learning and predictive analytics tools like Power BI (business intelligence) and Cognitive Services, which can be used by law enforcement agencies and surveillance vendors to build their own software or solutions.
Microsoft’s Surveillance-Based IoT Patrol Car
A rich array of Microsoft’s cloud-based offerings is on full display with a concept called “The Connected Officer.” Microsoft situates this concept as part of the Internet of Things, or IoT, in which gadgets are connected to online servers and thus made more useful. “The Connected Officer,” Microsoft has written, will “bring IoT to policing.”
With the Internet of Things, physical objects are assigned unique identifiers and transfer data over networks in an automated fashion. If a police officer draws a gun from its holster, for example, a notification can be sent over the network to alert other officers there may be danger. Real Time Crime Centers could then locate the officer on a map and monitor the situation from a command and control center.
According to this concept, a multitude of surveillance and IoT sensor data is sent onto a “hot path” for fast use in command centers and onto a “cold path” to be used later by intelligence analysts looking for patterns. The data is streamed along through Microsoft’s Azure Stream Analytics product, stored on the Azure cloud, and enhanced by Microsoft analytics solutions like Power BI — providing a number of points at which Microsoft can make money.
While the “Connected Officer” was a conceptual exercise, the company’s real-world patrol solution is the Microsoft Advanced Patrol Platform, or MAPP. MAPP is an IoT platform for police patrol vehicles that integrates surveillance sensors and database records on the Azure cloud, including “dispatch information, driving directions, suspect history, a voice-activated license plate reader, a missing persons list, location-based crime bulletins, shift reports, and more.”
The MAPP vehicle is outfitted with gear from third-party vendors that stream surveillance data into the Azure cloud for law enforcement agencies. Mounted to the roof, a 360-degree high-resolution camera streams live video to Azure and the laptop inside the vehicle, with access also available on a mobile phone or remote computer. The vehicle also sports an automatic license plate reader that can read 5,000 plates per minute — whether the car is stationary or on the move — and cross-check them against a database in Azure and run by Genetec’s license plate reader solution, AutoVu. A proximity camera on the vehicle is designed to alert the officers when their vehicle is being approached.
Patrolling the skies is a drone provided by Microsoft partner Aeryon Labs, the SkyRanger, to provide real-time streaming video. (Aeryon Labs is now part of surveillance giant FLIR Systems.) According to Nathan Beckham of Microsoft Public Safety and Justice, the vehicle’s drones “follow it around and see a bigger view of it.” The drones, writes DroneLife, can “provide aerial views to the integrated data platform, allowing officers to assess ongoing situations in real time, or to gather forensic evidence from a crime scene.”
Police robots are also part of the MAPP platform. Products from ReconRobotics, for example, “integrat[ed] with Microsoft’s Patrol Car of the Future Program” in 2016. Microsoft says ReconRobotics provides their MAPP vehicle with a “small, lightweight but powerful robot” that “can be easily deployed and remotely controlled by patrol officers to provide real-time information to decision-makers.”
Another Microsoft partner, SuperDroid Robots, has also announced they will provide the Microsoft MAPP vehicle with two compact remote-controlled surveillance robots, the MLT “Jack Russell” and the LT2-F “Bloodhound,” the latter of which can climb stairs and obstacles.
Although it sports a Microsoft insignia on the hood and door, the physical vehicle the company uses to promote MAPP isn’t for sale by Microsoft, and you probably won’t see Microsoft-labeled cars driving around. Rather, Microsoft provides MAPP as a platform through which to transform existing cop cars into IoT surveillance vehicles: “It’s really about being able to take all this data and put it up in the cloud, being able to source that data with their data, and start making relevant information out of it,” said Beckham.
Indeed, Microsoft says “the car is becoming the nerve center for law enforcement.” According to Beckham, the information collected and stored in the Azure cloud will help officers “identify bad actors” and “let the officers be aware of the environment that is going on around them.” As an example, he said, “We’re hoping with machine learning and AI in the future, we can start pattern matching” with MAPP vehicles providing data to help find “bad actors.”
Last October, South African police announced Microsoft partnered with the city of Durban for “21st century” smart policing. Durban’s version of the the MAPP solution includes a 360-degree ALPR to scan license plates and a facial recognition camera from Chinese video surveillance firm Hikvision for use when the vehicle is stationary (e.g., parked at an event).
According to South African news outlet ITWeb, the metro police will use the MAPP solution “to deter criminal activities based on data analysis through predictive modeling and machine learning algorithms.” The vehicle has already been rolled out in Cape Town, where Microsoft recently opened a new Azure data center — an extension of the digital colonialism I wrote about in 2018.
Much like the U.S. (albiet with some different dynamics), South Africa faces the scourge of police brutality that disproportionately impacts people of color. The country had its own George Floyd moment during the recent Covid-19 lockdown when the military and police brutally beat 40-year-old Collins Khosa in the poor Alexandra township, leading to his death — over a cup of beer. (A military inquiry found that Khosa’s death was not linked to his injuries at the hands of authorities; Khosa’s family and many others in South Africa have rejected the review as a whitewash.)
The MAPP solution will be used for “zero tolerance” policing. For example, Durban Metro Police spokesperson Parboo Sewpersad said the rollout aims to punish “littering, drinking and driving, and drinking and walking” during summer festivities.
It is difficult to determine where else the MAPP vehicle may be deployed. The rollout in South Africa suggests Microsoft sees Africa as a place to experiment with its police surveillance technologies.
Microsoft: Powering CCTV and Police Intelligence in the City
Beyond wiring police vehicles, video surveillance provides another lucrative source of profits for Microsoft, as it is loaded with data packets to transmit, store, and process — earning fees each step of the way.
When building a CCTV network packed with cameras, cities and businesses typically use a video management system, or VMS, to do things like display multiple camera feeds on a video wall or offer the option to search through footage. A leading VMS provider, Genetec, offers the core VMS integrated into Microsoft’s Domain Awareness System. A close partner of Microsoft for over 20 years, the two companies work together on integrating surveillance services on the Azure cloud.
Some of the most high-profile city police forces are using Genetec and Microsoft for video surveillance and analytics.
Through a public-private partnership called Operation Shield, Atlanta’s camera network has grown from 17 downtown cameras to a wide net of 10,600 cameras that officials hope will soon cover all city quadrants. Genetec and Microsoft Azure power the CCTV network.
On June 14, Atlanta’s Chief of Police, Erika Shields, resigned after APD cops shot and killed a 27-year-old Black man, Rayshard Brooks. Last month, six Atlanta police officers were charged for using excessive force against protesters of police violence.
In 2019, Atlanta Police Foundation COO Marshall Freeman told me the foundation had just completed a “department-wide rollout” for Microsoft Aware (Domain Awareness System). Freeman said the Atlanta Police Department uses Microsoft machine learning to correlate data, and plans to add Microsoft’s video analytics. “We can always continue to go back to Microsoft and have the builders expand on the technology and continue to build out the platform,” he added.
In Chicago, 35,000 cameras cover the city with a plug-in surveillance network. The back-end currently uses Genetec Stratocast and Genetec’s Federation service, which manages access to cameras across a federated network of CCTV cameras — a network of camera networks, so to speak.
In 2017, Genetec custom-built their Citigraf platform for the Chicago Police Department — the second-largest police force in the country — as a way to make sense of the department’s vast array of data. Powered by Microsoft Azure, Citigraf ingests information from surveillance sensors and database records. Using real-time and historical data, it performs calculations, visualizations, alerts, and other tasks to create “deep situational awareness” for the CPD. Microsoft is partnering with Genetec to build a “correlation engine” to make sense of the surveillance data.
The city of Detroit uses Genetec Stratocast and Microsoft Azure to power their controversial Project Green Light. Launched in 2016 in tandem with a new Real Time Crime Center, the project allows local businesses — or other participating entities, such as churches and public housing — to install video cameras on their premises and stream surveillance feeds to the Detroit Police Department. Participants can place a “green light” next to the cameras to warn the public — which is 80 percent Black — that “you are being watched by the police.”
In 2015, the DPD stated, “the day is coming where police will have access to cameras everywhere allowing the police to virtually patrol nearly any area of the city without ever stepping foot.”
DPD Assistant Chief David LeValley explained to me that prior to creating the new command center, the department sent a team of people to several other U.S. cities, including New York, Chicago, Atlanta, Boston, and the Drug Enforcement Administration center in El Paso, Texas, to scope out their intel centers. “Our Real Time Crime Center is an all-encompassing intelligence center, it’s not just Project Green Light,” he explained.
The expansion of police surveillance in Detroit has been swift. Today, Project Green Light has around 2,800 cameras installed across over 700 locations, and two smaller Real Time Crime Centers are being added, a development trending in cities like Chicago. LeValley told me those RTTCs will do things like “pattern recognition” and “reports for critical incidents.”
In the wake of George Floyd’s murder, activists in Detroit have recharged their efforts to abolish Project Green Light in the fight against police surveillance, which local community advocates like Tawana Petty and Eric Williams deem racist. This year, two Black men, Robert Julian-Borchak Williams and Michael Oliver, were wrongfully arrested after being misidentified by the DPD’s facial recognition technology.
Nakia Wallace, a co-organizer of Detroit Will Breathe, told me Project Green Light “pre-criminalizes” people and “gives the police the right to keep tabs on you if they think you are guilty” and “harass Black and brown communities.” “Linking together cameras” across wide areas is “hyper-surveillance” and “has to be stopped,” she added.
The “function that the [DPD] serve,” Wallace said, is “the protection of property and white supremacy.” “They’re hyper-militarized, and even in the wake of that, people are still dying in the city” because “they have no interest in the livelihood of Detroit citizens.” Instead of militarizing, we need to “stop pretending like poor Black people are inherently criminals, and start looking at social services and things that prevent people from going into a life of crime.”
In a 2017 blog post, Microsoft boasted about the partnership with Genetec for the DPD, stating that Project Green Light is “a great example of how cities can improve public safety, citizens’ quality of life, and economic growth with today’s technologies.”
Microsoft Actually Does Supply Facial Recognition Technology
While Microsoft has been powering intelligence centers and CCTV networks in the shadows, the company has publicly focused on facial recognition regulations. On June 11, Microsoft joined Amazon and IBM in saying it will not sell its facial recognition technology to police until there are regulations in place.
This is a PR stunt that confused how Microsoft’s relationship to policing works technically and ethically, in a number of ways.
First, while the press occasionally criticizes Microsoft’s Domain Awareness System, most attention to Microsoft policing focuses on facial recognition. This is mistaken: Microsoft is providing software to power a variety of policing technologies that undermine civil rights and liberties — even without facial recognition.
Second, facial recognition is a notable feature of many video surveillance systems and Real Time Crime Centers that Microsoft powers. The cities of New York, Atlanta, Chicago, and Detroit are among those utilizing Microsoft services to collect, store, and process the visual surveillance data used for facial recognition. Microsoft services are part and parcel of many police facial recognition surveillance systems.
Third, at least one facial recognition company, Veritone, has been left out of the conversation. A Microsoft partner, the Southern California artificial intelligence outfit offers cloud-based software called IDentify, which runs on Microsoft’s cloud and helps law enforcement agencies flag the faces of potential suspects.
In a 2020 keynote at the Consumer Electronics Show, speaking alongside executives from Microsoft, Deloitte, and Oracle, Veritone CEO Chad Steelberg claimed that thanks to Veritone’s IDentify software on Azure, cops have helped catch “hundreds and hundreds of suspects and violent offenders.” Veritone’s Redact product expedites prosecutions, and Illuminate allows investigators to “cull down evidence” and obtain anomaly “detection insights.”
In a recent webinar, Veritone explained how IDentify leverages data police already have, such as arrest records. If a person is detected and has no known match, the IDentify software can profile suspects by creating a “person of interest database” that “will allow you to simply save unknown faces to this database and continuously monitor for those faces over time.”
Veritone claims to deploy services in “about 150 locations,” but does not name which ones use IDentify. It launched a pilot test with the Anaheim Police Department in 2019.
Microsoft lists Veritone IDentify as a facial recognition law enforcement product offering in its app repository online. The promotional video on the Microsoft website advertises IDentify’s ability to:
… compare your known offender and person of interest databases with video evidence to quickly and automatically identify suspects for investigation. Simply upload evidence from surveillance systems, body cameras, and more. … But best of all, you’re not chained to your desk! Snap a picture and identify suspects while out on patrol, to verify statements, and preserve ongoing investigations.
Veritone has been a staunch defender of its facial recognition technology. In May 2019, the company tweeted:
In a promotional video featuring Microsoft, Veritone’s Jon Gacek said, “You can see why at Veritone we’re excited to be tightly partnered with Microsoft Azure team. Their vision and our vision is very common.”
Smoke, Mirrors, and Misdirection
Despite claims to the contrary, Microsoft is providing facial recognition services to law enforcement through partnerships and services to companies like Veritone and Genetec, and through its Domain Awareness System.
Microsoft’s public relations strategy is designed to mislead the public by veering attention away from its wide-ranging services to police. Instead, Microsoft president and chief legal officer Brad Smith urges the public to focus on facial recognition regulation and the issue of Microsoft’s own facial recognition software, as if their other software and service offerings, partnerships, concepts, and marketing are not integral to a whole ecosystem of facial recognition and mass surveillance systems offered by smaller companies.
Esteemed Microsoft scholars, such as Kate Crawford, co-founder of the Microsoft-funded think tank, AI Now Institute, have followed this playbook. Crawford recently praised Microsoft’s facial recognition PR and criticized companies like Clearview AI and Palantir, while ignoring the Microsoft Domain Awareness System, Microsoft’s surveillance partnerships, and Microsoft’s role as a cloud provider for facial recognition services.
Crawford and AI Now co-founder Meredith Whittaker have condemned predictive policing but haven’t explained the fact that Microsoft plays a central role in predictive policing for police. Crawford did not respond to a request for comment.
Microsoft and its advocates may claim that it is a “neutral” cloud provider and it’s up to other companies and police departments to decide how they use Microsoft software. Yet these companies are partnering with Microsoft, and Microsoft is getting paid to run their mass surveillance and facial recognition services on the Azure cloud — services that disproportionately affect people of color.
If these Microsoft clients were offering sex trafficking services on the Azure cloud, Microsoft would surely close their accounts. And because law enforcement agencies purchase surveillance technologies using taxpayer dollars, the public is actually paying Microsoft for its own police surveillance.
If activists force corporations like Microsoft, Amazon, Google, IBM, and Oracle to terminate partnerships and infrastructure services for third parties conducting police surveillance, then cloud providers would have to acknowledge they are accountable for what is done on their clouds. Moving forward, activists could press to replace corporate ownership of digital infrastructure and data with community ownership at the local level.
There is a lot at stake in this moment.
