Philadelphia Predictive Policing Experiment

The Philadelphia Predictive Policing Experiment was a city-wide, place-based, randomized experiment to study the impact of different police strategies on violence and property crime in predicted criminal activity areas.

The experiment was designed to test two theoretically-relevant operational questions about police patrol. If police are able to dedicate a car to predicted crime areas, would it be better to use a marked car or an unmarked car? The visible police car would emphasize deterrence and prevention. The plain-clothes car would allow officers to conduct surveillance and approach criminal activity undetected. The experiment also examined if it was effective to simply tell officers on roll call where the predicted grids were for the day without having a car dedicated to the task. These three interventions were compared to control areas where 'business-as-usual' policing was conducted.

Research design

The experiment ran in two phases; a three month property crime phase, and a three month violent crime phase. For each day during these phases, predictive software identified three small grid cells in each district for an eight-hour tour. Districts responded to these predicted crime areas depending on their experimental assignment. The Research and Analysis section of the Philadelphia Police Department worked with the academic researchers to randomly assigned 20 Philadelphia Police Department districts into one of four experimental conditions:

In five districts, officers were made aware of the predicted high crime activity area at roll call and asked to concentrate there when able (the "awareness" treatment condition).
Five districts received the awareness model treatment as well as a patrol car solely dedicated to the predicted crime area ("marked car" treatment condition).
Finally, five districts received an intelligence-led, investigative response with an unmarked unit dedicated to the predicted area ("unmarked car" treatment condition).
Control districts were areas where police personnel did not have access to the crime prediction software, so they maintained a standard patrol strategy.

Results

When examining both predicted high-crime grid cells and the grids cells immediately surrounding them, the marked car patrols resulted in a 31% reduction in property crime counts, or a 36% reduction in the number of cells experiencing at least one crime. While this sounds substantial, the specific numbers are small. Small micro-grids will never generate a massive volume of crime in a short time frame, therefore the raw numbers will remain small. This translates to a reduction in three crimes over 3 months for an average city district patrolling around three grids. To extrapolate, if each of the 21 geographic districts dedicated a marked car to three grids for an 8 hour shift each day, we estimate a reduction in 256 Part I property crimes per year.

There were also signs of a temporal diffusion of benefits to the eight hours after the property crime marked car patrols. While the percentages were substantial, the results were not statistically significant due to floor effects. There were no crime reduction benefits associated with the violent phase of the experiment, nor were there any benefits with the property crime awareness or unmarked car interventions.

And at the district level?

We also explored what happened at the district level. During the property experiment, expected weekly property crime counts for the entire district were between three and eight percent lower in the marked car districts compared to the control districts.

In summary, because of statistical limitations we should recognize the limitations in categorically stating that the implementation of a dedicated marked car to property crime predicted micro-grid areas reduced crime; however, both the micro-level grid analysis and the broader district analysis are supportive of this implication. It appears that marked police cars dedicated to predictive policing areas were effective at reducing some property crime. Unmarked cars, and efforts to combat violence, were not shown to be effective in the Philadelphia Predictive Policing Experiment.

Publications

Ratcliffe, J. H., Taylor, R. B., Askey, A. P., Thomas, K., Grasso, J., Bethel, K., Fisher, R., Koehnlein, J. (2021) The Philadelphia Predictive Policing Experiment. Journal of Experimental Criminology, 17(1): 15-41. Available here.

Abstract:

Objectives This place-based, randomized experiment explored the impact of different patrol strategies on violent and property crime in microscale predicted crime areas. The experiment aimed to learn whether different but operationally realistic police responses to crime forecasts, estimated by a predictive policing software program, could reduce crime.

Methods Twenty Philadelphia city districts were randomized to three interventions and one control condition. The three interventions comprised awareness districts (where officers were made aware of predicted areas on roll-call), marked car districts (where a marked patrol police car was dedicated to treatment areas), and unmarked car districts (a plain-clothes vehicle was dedicated to treatment areas). A business-as-usual approach represented the control condition in districts where staff had no access to the predictive software program. Two distinct 3-month phases examined crime outcomes for property and violent crime, respectively.
Results The marked car treatment showed substantial benefits for property crime (31% reduction in expected crime count), as well as temporal diffusion of benefits to the subsequent 8-h period (40% reduction in expected crime count). No other intervention demonstrated meaningful crime reduction. These reductions were probably not substantial
enough to impact city or district-wide property crime. Some violent crime results ran contrary to expectations, but this happened in a context of extremely low crime counts in predicted areas. The small grid size areas hampered achieving statistical power.

Conclusions The experiment found reductions in property crime resulting from the marked car focused patrols. It also demonstrated the real-world challenges of estimating and preventing crime in small areas.

Ratcliffe, J. H., Taylor, R. B., & Fisher, R. (2020) Conflicts and congruencies between predictive policing and the patrol officer’s craft. Policing and Society, 30(6): 639–655. Available here.

Abstract:

This publication reports on the qualitative findings from the study based on over 100 ride-alongs with officers. Limitations of the technology, including spatial, temporal, and spatiotemporal inaccuracies and/or unresponsiveness conflicted with officers’ craft-based knowledge. Concerns about the technology marginalizing their expertise and interfering with peer-based responsiveness norms surfaced as well. Notwithstanding those concerns, some officers pointed out how the prediction technology helped deepen their craft-based knowledge.

Taylor, RB, Ratcliffe, JH and Fisher, R (2020) Was the Pope to blame? Statistical powerlessness and the predictive policing of micro-scale randomized control trials, Criminology & Public Policy, 19(3): 965-996. Available here.

Abstract:

Research Summary
Hinkle et al. (2013) highlighted a statistical powerlessness problem in hot‐spots policing experiments in midsized cities with moderate property crime rates. The current work demonstrates that this problem is less readily resolved than previously suspected. It reviews results from a predictive policing randomized control trial in a large city with property crime rates higher than Chicago or Los Angeles. It reports, for the first time, a graphical analysis indicating the marked car patrol intervention, practically effective at the 500′ by 500′ (mission) grid level, three grids per shift, likely had a district‐wide impact on reducing reported property crime. In addition, it reviews results of a series of thought experiments exploring statistical power impacts of four modified experimental designs. Only one alternative design, with spatially up‐scaled predictive policing mission areas and concomitantly higher property crime prevalence rates, produced acceptable statistical power levels. Implications follow for current theoretical confusion in community criminology about concentration effects and units of analysis, and how models organize impacts across those different units. Implications follow for practice amid ongoing concerns about whether predictive policing works and, if it did, how to gauge its impacts and social justice costs.

Policy Implications
The current work brings to the fore important questions beyond “does predictive policing work?” Can we design predictive policing randomized experiments capable of showing statistical effectiveness? Furthermore, if we can, and if those studies include larger mission areas than the micro‐scaled geographic grids used so far, how do we integrate social justice concerns into effectiveness metrics given the broader segments of communities likely affected?

Ratcliffe, J. H., Taylor, R. B., & Askey, A.P., Fisher, R. and Koehnlein, J. (2018) The Philadelphia Predictive Policing Experiment: Final report. National Institute of Justice. January 2018. Download here