Philadelphia Predictive Policing Experiment
The Philadelphia Predictive Policing Experiment was a city-wide, place-based, randomized experiment to study the impact of different police strategies on violence and property crime in predicted criminal activity areas.
The experiment was designed to test two theoretically-relevant operational questions about police patrol. If police are able to dedicate a car to predicted crime areas, would it be better to use a marked car or an unmarked car? The visible police car would emphasize deterrence and prevention. The plain-clothes car would allow officers to conduct surveillance and approach criminal activity undetected. The experiment also examined if it was effective to simply tell officers on roll call where the predicted grids were for the day without having a car dedicated to the task. These three interventions were compared to control areas where 'business-as-usual' policing was conducted.
The experiment ran in two phases; a three month property crime phase, and a three month violent crime phase. For each day during these phases, predictive software identified three small grid cells in each district for an eight-hour tour. Districts responded to these predicted crime areas depending on their experimental assignment. The Research and Analysis section of the Philadelphia Police Department worked with the academic researchers to randomly assigned 20 Philadelphia Police Department districts into one of four experimental conditions:
In five districts, officers were made aware of the predicted high crime activity area at roll call and asked to concentrate there when able (the "awareness" treatment condition).
Five districts received the awareness model treatment as well as a patrol car solely dedicated to the predicted crime area ("marked car" treatment condition).
Finally, five districts received an intelligence-led, investigative response with an unmarked unit dedicated to the predicted area ("unmarked car" treatment condition).
Control districts were areas where police personnel did not have access to the crime prediction software, so they maintained a standard patrol strategy.
When examining both predicted high-crime grid cells and the grids cells immediately surrounding them, the marked car patrols resulted in a 31% reduction in property crime counts, or a 36% reduction in the number of cells experiencing at least one crime. While this sounds substantial, the specific numbers are small. Small micro-grids will never generate a massive volume of crime in a short time frame, therefore the raw numbers will remain small. This translates to a reduction in three crimes over 3 months for an average city district patrolling around three grids. To extrapolate, if each of the 21 geographic districts dedicated a marked car to three grids for an 8 hour shift each day, we estimate a reduction in 256 Part I property crimes per year.
There were also signs of a temporal diffusion of benefits to the eight hours after the property crime marked car patrols. While the percentages were substantial, the results were not statistically significant due to floor effects. There were no crime reduction benefits associated with the violent phase of the experiment, nor were there any benefits with the property crime awareness or unmarked car interventions.
And at the district level?
We also explored what happened at the district level. During the property experiment, expected weekly property crime counts for the entire district were between three and eight percent lower in the marked car districts compared to the control districts.
In summary, because of statistical limitations we should recognize the limitations in categorically stating that the implementation of a dedicated marked car to property crime predicted micro-grid areas reduced crime; however, both the micro-level grid analysis and the broader district analysis are supportive of this implication. It appears that marked police cars dedicated to predictive policing areas were effective at reducing some property crime. Unmarked cars, and efforts to combat violence, were not shown to be effective in the Philadelphia Predictive Policing Experiment.
A note on software efficacy
We used the HunchLab program designed by Azavea and subsequently purchased by Shotspotter Inc. HunchLab is a web-based predictive policing system that accesses real-time Philadelphia Police data to produce crime forecasts for the city. It incorporates statistical modeling that considers seasonality, risk terrain modeling, near repeats, and collective efficacy. Azavea adapted the software at the request of the Philadelphia Police Department and the research team to generate three predicted 500 feet square grids per district per shift. They also included a slight randomization component to reduce the possibility that the same grid cells were predicted every day. It is important to note therefore that the experiment artificially reduced the efficiency of the software, because it forced the software to choose grids in low crime districts, and limited the number of grids it could assign in high crime districts.
The software was able to predict twice as much crime as we would expect if crime were spread uniformly across the districts, even when artificially constrained by our experiment to be less effective than designed. Predictive Accuracy Index values were calculated for the property crime and violent crime phases, and are reported in the Ratcliffe et al paper published in the Journal of Experimental Criminology.
Publications related to the experiment
Ratcliffe, J. H., Taylor, R. B., Askey, A. P., Thomas, K., Grasso, J., Bethel, K., Fisher, R., Koehnlein, J. (in press) The Philadelphia Predictive Policing Experiment. Journal of Experimental Criminology.
This publication has the primary results from the experiment, charts with the implementation evaluation, predictive accuracy index values for the software efficacy, full Bayes results from the experiment, and results from the analysis of temporal displacement to the subsequent shift for both the property and violent crime phases of the experiment. The full article is available here. We also have a short three-page research summary that explains the experiment in brief. Scroll further down for the final report.
Objectives This place-based, randomized experiment explored the impact of different patrol strategies on violent and property crime in microscale predicted crime areas. The experiment aimed to learn whether different but operationally realistic police responses to crime forecasts, estimated by a predictive policing software program, could reduce crime.
Methods Twenty Philadelphia city districts were randomized to three interventions and one control condition. The three interventions comprised awareness districts (where officers were made aware of predicted areas on roll-call), marked car districts (where a marked patrol police car was dedicated to treatment areas), and unmarked car districts (a plain-clothes vehicle was dedicated to treatment areas). A business-as-usual approach represented the control condition in districts where staff had no access to the predictive software program. Two distinct 3-month phases examined crime outcomes for property and violent crime, respectively.
Results The marked car treatment showed substantial benefits for property crime (31% reduction in expected crime count), as well as temporal diffusion of benefits to the subsequent 8-h period (40% reduction in expected crime count). No other intervention demonstrated meaningful crime reduction. These reductions were probably not substantial
enough to impact city or district-wide property crime. Some violent crime results ran contrary to expectations, but this happened in a context of extremely low crime counts in predicted areas. The small grid size areas hampered achieving statistical power.
Conclusions The experiment found reductions in property crime resulting from the marked car focused patrols. It also demonstrated the real-world challenges of estimating and preventing crime in small areas.
Ratcliffe, J. H., Taylor, R. B., & Fisher, R. (in press) Conflicts and congruencies between predictive policing and the patrol officer’s craft. Policing and Society.
This publication reports on the qualitative findings from the study based on over 100 ride-alongs with officers. Limitations of the technology, including spatial, temporal, and spatiotemporal inaccuracies and/or unresponsiveness conflicted with officers’ craft-based knowledge. Concerns about the technology marginalizing their expertise and interfering with peer-based responsiveness norms surfaced as well. Notwithstanding those concerns, some officers pointed out how the prediction technology helped deepen their craft-based knowledge.
Ratcliffe, J. H., Taylor, R. B., & Askey, A.P., Fisher, R. and Koehnlein, J. (2018) The Philadelphia Predictive Policing Experiment: Final report. National Institute of Justice. January 2018.