Category Archives: General ramblings

Classic experiments in evidence-based policing

So far, it’s been a fun semester teaching evidence-based policing for the first time. We have covered everything from evidence-based medicine to research design and the Maryland Scientific Methods Scale, and even some basic stats so that we can understand confidence intervals. It’s been particularly rewarding to see students who have spent years in policing exploring and learning about the world of research evidence that supports and helps their world, a world of which many have been until now unaware.

What I am also learning is that those of us in the police education field have done a lousy job of explaining what we do and why it is important to advancing policing and the practice of law enforcement. There is a range of classic studies that are not well known, and an absence of knowledge around these – and other important works – fuels the never-ending cycle of operational decisions that fly in the face of all we know about what works, and what doesn’t. Police still support strategies and crime reduction tactics that are known to not work.

In light of this, I started putting together a list of experiments of which that I thought my students should be aware. The original studies are described in a range of works from academic journal articles to long-winded reports. All pretty impenetrable for most folk, especially busy cops. So I have copied and pasted the key pieces of information into a single page per study, copied from the original sources directly. I cite them at the bottom of each page so you know the source.

This isn’t an exhaustive list, and I intend for it to grow, but for now the list comprises:

  1. The Kansas City Preventive Patrol Experiment
  2. The Newark Foot Patrol Experiment
  3. The Philadelphia Foot Patrol Experiment
  4. The Minneapolis Domestic Violence Experiment
  5. The Minneapolis Hot Spots Policing Experiment
  6. The Philadelphia Policing Tactics Experiment
  7. The Sacramento Hot Spots Policing Experiment
  8. The Queensland Procedural Justice Experiment

I will add to these over time, but for now if you want a copy, download a pdf of the one page summaries.

Note: If you are using these summaries to write a college paper, you should refer to the original study and cite it appropriately. All I have done is edit a copy-and-paste, but I’m 1) not writing a term paper and 2) not passing this off as my own work. If you do, that’s plagiarism. 

The Modifiable Areal Unit Problem

The Modifiable Areal Unit Problem (MAUP) is a potential source of error that can affect spatial studies which utilize aggregate data sources (Unwin, 1996). Geographical data are often aggregated in order to present the results of a study in a more useful context, and spatial objects such as census tracts or police beat boundaries are examples of the type of aggregating zones used to show results of some spatial phenomena. These zones are often arbitrary in nature and different areal units can be just as meaningful in displaying the same base level data. For example, it could be argued that census tracts containing comparable numbers of houses are better sources of aggregation than police beats (which are often based on ancient parish boundaries in the UK) when displaying burglary rates.Preview

Large amounts of source data require a careful choice of aggregating zones to display the spatial variation of the data in a comprehensible manner. It is this variation in acceptable areal solution that generates the term ‘modifiable’. Only recently (well, the last 30 years) has this problem been addressed in the area of spatial crime analysis, where ‘the areal units (zonal objects) used in many geographical studies are arbitrary, modifiable, and subject to the whims and fancies of whoever is doing, or did, the aggregating.’ (Openshaw, 1984 p.3).

As the study area for crime incident locations has effectively infinite resolution, there exists a potentially infinite number of different options for aggregating the data. Numerous administrative boundaries exists, such as enumeration districts, wards, counties, health authority areas, etc. Within modern GIS, it is an elementary task to automatically generate a huge variety of non-overlapping boundaries. Regular, often square, grids are common, though polygons have been used in other studies of crime distribution (Hirschfield et al., 1997). The number of different combinations of areal unit available to aggregate data is staggering. Openshaw (1984) calculated that if one was to attempt to aggregate 1,000 objects into 20 groups, you would be faced with 101,260 different solution combinations. Although there are a large number of different spatial objects and ways in which a large geographical area can be sub-divided, the choices of areal units tend to be dominated by what is available rather than what is best. Police crime data is often mapped to police beats, even when the information is passed to outside agencies such as neighborhood watches or local councils who might benefit from more relevant boundary structures.

The MAUP consists of both a scale and an aggregation problem, and the concept of the ecological fallacy should also be considered (Bailey and Gatrell, 1995). The scale problem is relatively well known. It is the variation which can occur when data from one scale of areal units is aggregated into more or less areal units. For example, much of the variation in census areas changes or is lost when the data are aggregated to the ward or county level.

The aggregation problem is less well known and becomes apparent when faced with the variety of different possible areal units for aggregation. Although geographical studies tend towards aggregating units which have a geographical boundary, it is possible to aggregate spatial units which are spatially distinct. Aggregating neighbors improves the problem to a small degree but does not get round the quantity of variation in possibilities which remains.

For a paper that discusses the MAUP and possible solutions, see:
Ratcliffe, J. H. and McCullagh, M. J. 1999 ‘Hotbeds of crime and the search for spatial accuracy’, Geographical Systems 1(4): 385-398. Paper available here.

Also see the Ecological Fallacy.


Bailey, T. C. and Gatrell, A. C. 1995 Interactive Spatial Data Analysis, Second Edition: Longman.

Hirschfield, A., Yarwood, D. and Bowers, K. 1997 ‘Crime Pattern Analysis, Spatial Targeting and GIS: The development of new approaches for use in evaluating Community Safety initiatives.’, in N. Evans-Mudie (ed) Crime and health data analysis using GIS, Sheffield: SCGISA.

Openshaw, S. 1984 ‘The modifiable areal unit problem’, Concepts and Techniques in Modern Geography 38: 41.

Unwin, D. J. 1996 GIS, spatial analysis and spatial statistics’, Progress in Human Geography 20(4): 540-441.

Why we shouldn’t fixate on homicide numbers

There are some certainties in life. Death, taxes, the Eagles snatching defeat from the jaws of victory. And the annual January media fixation with homicide rates as the barometer of everything from a city’s moral compass to the effectiveness of the police chief.

I spent a couple of days speaking to various reporters about the homicide numbers in Philadelphia, and how they were significantly down on a few years ago, but had remained largely unchanged since last year. ‘What could we gather from this?’ ‘What were the implications?’ ‘Were police department strategies starting to falter?’ ‘What does it mean for the mayor and police commissioner?’

Taking more time than I really had, given I am trying to update ‘Intelligence-Led Policing’ for a second edition, I tried to explain that the homicide figures are a really bad choice of metric. For just about anything. For example, a not insubstantial number of homicides occur between people who know each other, and often take place indoors. How are the police department supposed to anticipate and prevent those homicides? Even if they develop a ‘Minority Report’ predictive capacity, we have a reactive legal and criminal justice system: it isn’t keen on letting the police just wander into your house and lock you up for pondering murder. And for the homicides that take place on the street? Sitting in on numerous Philadelphia Police Department crime briefings and listening to the homicide reports, it is clear that many are the result of minor disputes that flared up with little-to-no warning or are the result of disputes between participants in gangs or drug organizations who conceal their business and would never seek the intervention of the police.

The difference between a homicide and an aggravated assault is also largely outside of police control. Could be the shooter has lousy aim or is firing gangster style, there is a delay in getting the victim to the hospital, or simply medical mismanagement. Once a person decides to shoot someone else, they are easily able to in the US because we allow them the opportunities to do so. Our legislators seem unwilling to help the police with this, so again, little chance for police influence here.

I examined a summary of every incident recorded by the Philadelphia Police for the last available full year (2013) to estimate how much police patrol energy is expended on responding to homicide incidents. In Philadelphia, the city receives millions of calls for service, and from these – as well as police-generated activity – an INCT database is created. This database contains every incident where a police officer was required to act, and ranges from dog bites and graffiti to shootings and homicides, and from assistance to city agencies and delivering messages, to removing debris from the interstate or arresting a drunk driver. In 2013 there were in excess of 1.65 million incidents. What percentage of these related to homicide? 0.021%. Less than one quarter of one tenth of one percent.

I explained to the reporters that aggravated assaults and robberies were also down, and due to their greater number generally, this was a much better way to indicate the crime health of the city. They said they got it, but their hands were tied: “the public interest is in homicides”.  So we still got story after story about the homicide rate. Not a major grumble: reporters have to make a call and write what they think is the story. But I wonder if the fascination with homicides is really driven by public demand, or by the media? I can’t believe there was massive public outcry that drove the claim that “Philly’s Murder Rate Is Skyrocketing Again in 2014”… especially only two weeks into the year (you had to go back and checked the post date didn’t you?).

Traditionally, homicides have been used because they are easily comparable between cities, because police departments have recorded other incidents in different ways, or because in the past (sometimes not so distant) the police have distorted the crime figures. But homicides comprise so little of the work of a police agency, and the chances of most people being a victim of homicide are so low, that they tell us little about the experienced crime rate or the quality of life for city residents.

We need to start moving to more holistic measures if oversight and strategy are to be more data driven and evidence based. Harm-focused policing that examines and weighs all incidents, and includes other harms to communities, such as traffic accidents or the potentially deleterious impact of unrestrained pedestrian investigations, is increasingly possible with the big data sets that public agencies generate. We need to evolve beyond our fixation with homicide if we are to move the discussion about safety and harm forward.

But in the meantime, Philadelphia, be glad that shootings and robberies are also down.

(This post was updated shortly after posting to correct the homicide incident rate)