It’s time for Compstat to change

If we are to promote more thoughtful and evidence-based policing, then Compstat has to change. The Compstat-type crime management meeting has its origins in Bill Bratton’s need to extract greater accountability from NYPD precinct commanders in late 1990s New York. It was definitely innovative for policing at the time, and instigated many initiatives that are hugely beneficial to modern policing (such as the growth of crime mapping). And arguably it has been successful in promoting greater reflexivity from middle managers; however these days the flaws are increasingly apparent.

Over my years of watching Compstat-type meetings in a number of departments, I’ve observed everyone settle into their Compstat role relatively comfortably. Well almost. The mid-level local area commander who has to field questions is often a little uneasy, but these days few careers are destroyed in Compstat. A little preparation, some confidence, and a handful of quick statistics or case details to bullshit through the tough parts will all see a shrewd commander escape unscathed.

In turn, the executives know their role. They stare intently at the map, ask about a crime hot spot or two, perhaps interrogate a little on a case just to check the commander has some specifics on hand, and then volunteer thoughts on a strategy the commander should try—just to demonstrate their experience. It’s an easy role because it doesn’t require any preparation. In turn, the area commander pledges to increase patrols in the neighborhood and everyone commits to reviewing progress next month, safe in the knowledge that little review will actually take place because by then new dots will have appeared on the map to absorb everyone’s attention. It’s a one-trick pony and everyone is comfortable with the trick.

There are some glaring problems with Compstat. The first is that the analysis is weak and often just based on a map of dots or, if the department is adventurous, crime hot spots. Unfortunately, a map of crime hot spots should be the start of an analysis, not the conclusion. It’s great for telling us what is going on, but this sort of map can’t really tell us why. We need more information and intelligence to get to why. And why is vital if we are to implement a successful crime reduction strategy.

We never get beyond this basic map because of the second problem: the frequent push to make an operational decision immediately. When command staff have to magic up a response on the spot, the result is often a superficial operational choice. Nobody wants to appear indecisive, but with crime control it can be disastrous. Too few commanders ever request more time to do more analysis, or time to consider the evidence base for their operational strategies. It’s as if asking to think more about a complex problem would be seen as weak or too ‘clever’. I concede that tackling an emerging crime spike might be valuable (though they often regress to the mean, or as Sir Francis Galton called it in 1886, regression towards mediocrity). Many Compstat issues however, revolve around chronic, long-term problems where a few days isn’t going to make much difference. We should adopt the attitude that it’s better to have a thoughtfully considered successful strategy next week than a failing one this week.

Because of the pressure to miracle a working strategy out of thin air, area commanders usually default to a limited set of standard approaches, saturation patrol with uniform resources being the one that I see at least 90 percent of the time. And it’s applied to everything, regardless of whether there is any likelihood that it will impact the problem. It is suggested by executives and embraced by local area commanders because it is how we’ve always escaped from Compstat. Few question saturation patrols, there is some evidence it works in the short term, and it’s a non-threatening traditional policing approach that everyone understands. Saturation patrol is like a favorite winter coat, except that we like to wear it all year round.

Third, in the absence of a more thoughtful and evidence-based process, too many decisions and views lack any evidential support and instead are driven by personal views. There is a scene in the movie Moneyball where all the old baseball scouts are giving their thoughts on which players the team should buy, based only on the scouts’ experience, opinion and personal judgment. They ignore the nerd in the corner who has real data and figures … and some insight. They even question if he has to be in the room. In the movie, the data analyst is disparaged, even though he doesn’t bring an opinion or intuition to the table. He brings data analysis, and the data don’t care how long you have been in the business.

Too many Compstat meetings are reminiscent of this scene. The centerpiece of many Compstat meetings is a map of crime that many are viewing for the first time. A room full of people wax lyrical on the crime problem based on their intuitive interpretation of a map of crime on the wall, and then they promote solutions for our beleaguered commander, based too often on opinion and personal judgement and too little on knowledge of the supporting evidence of the tactic’s effectiveness. Because everyone knows they have to come back in a month the strategies are inevitably short-term in nature and never evaluated. And without being evaluated, they are never discredited, so they become the go-to tactical choice ad infinitum.

So the problems with Compstat are weak analysis, rushed decision-making, and opinion-driven strategies. What might the solutions be?

The U.K.’s National Intelligence Model is a good starting point for consideration. It has a strategic and a tactical cycle. The strategic meeting attendees determine the main strategic aims and goals for the district. At a recent meeting a senior commander told me “We are usually too busy putting out fires to care about who is throwing matches around.” Any process that has some strategic direction to focus the tactical day-to-day management of a district has the capacity to keep at least one eye on the match thrower. A monthly meeting, focused on chronic district problems, can generate two or three strategic priorities.

A more regular tactical meeting is then tasked with implementing these strategic priorities. This might be a weekly meeting that can both deal with the dramas of the day as well as supervise implementation of the goals set at the strategic meeting. It is important that the tactical meeting should spend some time on the implementation of the larger strategic goals. In this way, the strategic goals are not subsumed by day-to-day dramas that often comprise the tyranny of the moment. And the tactical meeting shouldn’t set strategic goals—that is the role of the strategic working group.

I’ve previously written that Compstat has become a game of “whack-a-mole” policing with no long-term value. Dots appear, and we move the troops to the dots to try and quell the problem. Next month new dots appear somewhere else, and we do the whole thing all over again. If we don’t retain a strategic eye on long-term goals, it’s not effective policing. It’s Groundhog Day policing.

Policing explained in a few graphs

As a new semester starts and I look forward to teaching a new group of undergraduates, I am reminded that some of them have not had extensive exposure to law enforcement or policing research. How then to encapsulate the essence of what is useful to know about policing in a brief enough format? These are some of the graphs I use to help my new students understand the challenges of policing in the 21st century.

1. Policing is overwhelmingly a social service

Graph no. 1. This is from the second edition of my book “Intelligence-Led Policing“. The area of each box represents the volume of incidents in 2015 in the City of Philadelphia (about 1.5m in total). These incidents can come from verified calls for service from the public (something really took place as confirmed by a police officer), or from officer-initiated events (such as drug incidents). 

What is clear from the graphic is that violent crime plays such a small part in the day-to-day demands on police departments, even in Philadelphia, one of the more troubled cities in the U.S. While the media frets over homicide, it can be seen in the lower right as one of the least noticeable boxes in the graph. The majority of the police department’s workload is the day-to-day minutiae of life in a big city.

2. Impacting early in the crime funnel is the key to public safety

Graph no. 2 is another image from my Intelligence-Led Policing book. The crime funnel represents what happens to a random selection of 1,000 crimes that affect the public (top bar). It shows the loss of cases through the criminal justice system. These are British national data derived from public records, but the comparisons to the U.S. are very similar. If you take a random selection of 1,000 crimes actually suffered by the public (violence, robbery, vehicle theft, residential burglary, theft and criminal damage) you can see that they only report 530 to the police, who in turn record just 43 percent of the original total.

Of these 429 events, 99 are detected (solved or cleared in some way) and of these, 60 end up with a day in court. The majority of those are found guilty or plead the same, but in the end only four of the events from the original 1,000 end up in a custodial sentence for the offender. This is an incarceration rate of 0.4% based on crime suffered by the community.

The main point here is that impacting higher in the crime funnel will be more effective because it affects the numbers below and affects a larger number of actual cases. Improving the detection rate will have an impact on prosecutions, pleas and incarceration, but only to a minimal level. Being prevention focused and changing the higher numbers is much more impactful. Consider if you could have a 10% change on one level. Where would it be most effective?

3: Public perception of the police has little to do with crime

Graph no. 3. Going back nearly 20 years, it is clear from the graph that violent crime in the U.S. has continued a decline right through to at least 2013. Things have changed in the last year or two but that isn’t the point here. Public perception in terms of confidence in the police does not seem to be tied to the crime rate. Confidence in the police is pretty high, especially compared to nearly every other occupation (except the medical field). After all in June this year, it was found that only 27% of Americans say they have “a great deal” or “quite a lot” of confidence in newspapers, so police are doing pretty well. The graph shows however that there isn’t a direct and easy correlation between the crime rate and confidence in the police. It is more complicated. Public confidence is more complicated than just crime reduction. This graph is from a forthcoming book I am in the middle of writing called “Fighting Crime” and it will be a crime reduction guide for mid-level police command staff.

4: Policing is changing drastically

Graph no. 4 isn’t even about policing, but it is about external policy impacts on policing. When I joined the police service in 1984 and started policing on the old H district in East London, we had to deal with a few behavioral health patients because we had St. Clement’s hospital mental health hospital a stone’s throw from the nick. So perhaps we were immune to the gradual change in the social structure of society, a change that has affected the U.S. as well. Graph no. 4 is from the U.S. Substance Abuse and Mental Health Services Administration.

30 years ago, nearly two-thirds of U.S. mental health spending was on inpatient and residential care: in other words, professionally trained carers. Over the last 30 years this has reduced to only be about one-third of national spending. As graph number 4 shows on the right, the corresponding increase has not been in outpatient treatment, but in retail prescription drugs. This has essentially shifted the supervision of people having a behavioral health crisis onto the community – and the police.

It is indicative of just one change in society that has made demands on the police. Police officers are not really trained to be mental health professionals, and yet society has demanded they take a role in the care and control of people with behavioral health crises because society seems unwilling to pay for appropriate professional care. Behavioral health is just one factor, and students are often able to consider many other areas where the police are now the lead agency in areas that were never originally police issues.

Source: This is from SAMHSA data and screen captured from a PowerPoint I use in when consulting and training police departments on intelligence-led policing.

5. Policing is increasingly a safer occupation

This is one graphic that I confess I didn’t create myself, but it tracks with my research on this area and mimics many similar charts online. It is sourced from the Officer Down memorial page. While things have changed in the last couple of years, the good news is that policing has become increasingly safer (as has society) in the U.S. over the last 30 years.

What I find interesting is that increases in police fatalities are linked to government policy changes that alter the role of police in society. The overarching point remains that policing has become a much safer occupation over the last 30 years or more. It now does not feature in the top 10 list of dangerous occupations (This next graph from WaPo isn’t counted in my list – it’s a freebie).

6: Policing is most effective when focused on specific people and places

OK, this one takes a bit to get your head around. Graph no. 6 is the Evidence-Based Policing Matrix from Cynthia Lum and colleagues has three dimensions and four symbols. The key symbol is the black circle – this indicates an effective intervention (white is ineffective, grey is a mixed result, and the red triangle is a harmful intervention). Where the black circles are most concentrated are in the areas where interventions focus proactive activity on places that are at the neighborhood level or smaller, or on people. This (and other evidence) suggests that general interventions that are applied across whole jurisdictions or cities are less effective.

The bottom line? Police interventions that actually reduce crime are locally focused and tailored to specific problems, not citywide interventions.

Are there more graphics that depict the reality of 21st Century policing? Of course. But these ones seem to be significant. Ping me if you have others you think are relevant and important.

Harm-focused policing

On 28th January, 2015 I gave the Police Foundation‘s Ideas in American Policing lecture on the topic of harm-focused policing. This brief blog provides some background details to the talk. Please note that for a number of reasons (including photograph copyright) I am not distributing copies of the PowerPoint slides.

Harm-focused policing weighs the social harms of criminality and disorder with data from beyond crime and antisocial behavior, in order to focus police priorities and resources in furtherance of both crime and harm reduction.

Example information and data sources could include drug overdose information that could help triage drug markets for interdiction, traffic fatality data to guide police patrol responses, and community impact assessments to prioritize violent street gangs. For a summary of the core of the presentation and a grey-scale version of some of the graphics, please see:

Ratcliffe, J. H. (2015). Towards an index for harm-focused policing. Policing: A Journal of Policy and Practice, 9(2), 164-182.

You can visit the journal site and access the paper here (or here) and watch my annotated video of the lecture below (you might want to make it full screen so you can read the slides).

During the presentation, I had a couple of quotes. Here are the quotes and their sources.

“to establish priorities for strategic criminal intelligence gathering and subsequent analysis based on notions of the social harm caused by different sorts of criminal activity”. The source for this is page 262 of Ratcliffe, J. H. & Sheptycki, J. (2009) Setting the strategic agenda. In J. H. Ratcliffe (ed.) Strategic Thinking in Criminal Intelligence (2nd edition) Sydney: Federation Press.

“Weighting crimes on the basis of sentencing guidelines can be justified on good democratic grounds as reflecting the will of the people. …… it remains far closer to the will of the people than any theoretical or even empirical system of weighting that academics might develop.” The source for this is Sherman, L. W. (2013). Targeting, testing and tracking police services: The rise of evidence-based policing, 1975-2025. In M. Tonry (Ed.), Crime and Justice in America, 1975-2025. Chicago: University of Chicago Press. Page 47.

 

Why we shouldn’t fixate on homicide numbers

There are some certainties in life. Death, taxes, the Eagles snatching defeat from the jaws of victory. And the annual January media fixation with homicide rates as the barometer of everything from a city’s moral compass to the effectiveness of the police chief.

I spent a couple of days speaking to various reporters about the homicide numbers in Philadelphia, and how they were significantly down on a few years ago, but had remained largely unchanged since last year. ‘What could we gather from this?’ ‘What were the implications?’ ‘Were police department strategies starting to falter?’ ‘What does it mean for the mayor and police commissioner?’

Taking more time than I really had, given I am trying to update ‘Intelligence-Led Policing’ for a second edition, I tried to explain that the homicide figures are a really bad choice of metric. For just about anything. For example, a not insubstantial number of homicides occur between people who know each other, and often take place indoors. How are the police department supposed to anticipate and prevent those homicides? Even if they develop a ‘Minority Report’ predictive capacity, we have a reactive legal and criminal justice system: it isn’t keen on letting the police just wander into your house and lock you up for pondering murder. And for the homicides that take place on the street? Sitting in on numerous Philadelphia Police Department crime briefings and listening to the homicide reports, it is clear that many are the result of minor disputes that flared up with little-to-no warning or are the result of disputes between participants in gangs or drug organizations who conceal their business and would never seek the intervention of the police.

The difference between a homicide and an aggravated assault is also largely outside of police control. Could be the shooter has lousy aim or is firing gangster style, there is a delay in getting the victim to the hospital, or simply medical mismanagement. Once a person decides to shoot someone else, they are easily able to in the US because we allow them the opportunities to do so. Our legislators seem unwilling to help the police with this, so again, little chance for police influence here.

I examined a summary of every incident recorded by the Philadelphia Police for the last available full year (2013) to estimate how much police patrol energy is expended on responding to homicide incidents. In Philadelphia, the city receives millions of calls for service, and from these – as well as police-generated activity – an INCT database is created. This database contains every incident where a police officer was required to act, and ranges from dog bites and graffiti to shootings and homicides, and from assistance to city agencies and delivering messages, to removing debris from the interstate or arresting a drunk driver. In 2013 there were in excess of 1.65 million incidents. What percentage of these related to homicide? 0.021%. Less than one quarter of one tenth of one percent.

I explained to the reporters that aggravated assaults and robberies were also down, and due to their greater number generally, this was a much better way to indicate the crime health of the city. They said they got it, but their hands were tied: “the public interest is in homicides”.  So we still got story after story about the homicide rate. Not a major grumble: reporters have to make a call and write what they think is the story. But I wonder if the fascination with homicides is really driven by public demand, or by the media? I can’t believe there was massive public outcry that drove the claim that “Philly’s Murder Rate Is Skyrocketing Again in 2014”… especially only two weeks into the year (you had to go back and checked the post date didn’t you?).

Traditionally, homicides have been used because they are easily comparable between cities, because police departments have recorded other incidents in different ways, or because in the past (sometimes not so distant) the police have distorted the crime figures. But homicides comprise so little of the work of a police agency, and the chances of most people being a victim of homicide are so low, that they tell us little about the experienced crime rate or the quality of life for city residents.

We need to start moving to more holistic measures if oversight and strategy are to be more data driven and evidence based. Harm-focused policing that examines and weighs all incidents, and includes other harms to communities, such as traffic accidents or the potentially deleterious impact of unrestrained pedestrian investigations, is increasingly possible with the big data sets that public agencies generate. We need to evolve beyond our fixation with homicide if we are to move the discussion about safety and harm forward.

But in the meantime, Philadelphia, be glad that shootings and robberies are also down.

(This post was updated shortly after posting to correct the homicide incident rate)

Schrödinger’s crime hotspot

Attendance at the recent 2014 American Society of Criminology conference brought a chance to catch up with friends and observe a couple of splendid presentations (and quite a few awful ones). A couple of sessions in particular reaffirmed to me the gulf between some academic criminology and public policy. I watched as speakers attempted to parse in ever-increasing detail the boundaries of crime hotspots. Discussions continued around the efficacy of street blocks as potentially more accurate units of analysis compared to census block groups, and I could see the accuracy issue being a hot topic in predictive policing workshops. Hotspot boundary definition appeared to be the end in itself.

As my colleague Ralph B. Taylor has argued, “hot spots exist in the data world but not the real world” (Taylor, 2009) ¹. In this they are unlike land use parcels, behavior settings or street blocks – places that exist in the data and real worlds. He goes on to contend that “hot spots are amalgams of different types of locations” and we therefore have a construct validity problem. Simply because we see a cluster of events, does not mean we have a new entity (a crime hotspot) but rather a collection of events that exist as a cohesive entity only in the abstract world. To think otherwise is to commit a reification fallacy (Gould, 1981). When we move to the real world, and think about operationalizing a strategy to address our hotspots, things can unravel as it becomes clear that this collection of points exists for different reasons, each of which need addressing.

I thought of Ralph Taylor’s comments as I sat in the audience, and pondered the analogy between crime hotspots and Schrödinger’s cat. Erwin Schrödinger’s feline thought experiment was designed to explain why the Copenhagen interpretation of quantum superposition was flawed, because the cat exists in a paradoxical state of being neither alive nor dead. The hypothetical animal is placed in a steel box with a Geiger counter, a vial of poison, a hammer, and a small amount of radioactive substance, small enough to have only a 50/50 chance of being detected over the course of an hour. If the radioactive substance decay is detected by the counter, the hammer is triggered to smash the vial, release the poison and kill the cat. Only by looking in the box can the observer determine whether the cat is alive or dead. Prior to opening the box, the cat’s health is unknown and could be considered simultaneously alive and dead. The observer opens the box with the express intent of confirming the wellbeing of the cat. Before opening the box, the cat’s condition is unresolved and abstract. In the same way, crime hotspots are in a largely abstract state until we look at them from a particular viewpoint.

This brings me to two points that appeared rather lost on some of the conference speakers. First, crime mapping and the application of GIS to crime problems is not the end of the analysis – it is the start of it. Digital cartography is a necessary abstraction of the real world, and to think otherwise is to be oblivious to the classification, simplification and symbolization that takes place. It is through these processes that unrelated events can often be made apparently similar. The nighttime beating and robbery of a drug dealer will often be classified in the same manner as the punch a school child receives as they are relieved of their smart phone by a classmate. In a map of crime for the year, these events will likely be cartographically identical, but unlikely to be prevented in the future with the same response. Just because two crimes share geographic proximity, doesn’t mean they necessarily share a common cause (a point I’ve made elsewhere).

This brings me to a related second point. Crime hotspots (in the abstract world) are only made real when they are mapped for a purpose. When a police captain asks for a map of robbery hotspots, the captain is bringing a purpose to the analysis. He or she wants to deploy a surveillance team, task a crime prevention officer, or know where to assign more foot patrol officers. An academic, meanwhile, might want to seek underlying causal factors and understand why crime concentrates in certain areas. With the knowledge of the eventual purpose, a proficient analyst can create tailored hotspots that map to the parameters of the user’s needs. The maps would be different, but no less useful. Our captain brings a lens through which he or she interrogates the hotspot, and by looking at a map of crime hotspots they are made real and are understood. It is the captain, not the analyst, who opens the box.²

Both the captain and the academic bring to the analysis a predetermined purpose, and although different, the crime hotspots that each uncovers are equally valid. In opening the box, and staring at the map through the lens of a proposed application, crime hotspots are made real and understood. At this point, they can serve a purpose. But until then, they remain in an abstract state and their accuracy, or even their state as being alive or dead as viable entities, remains unknown. Like Schrödinger’s cat.

Works cited

Taylor, R. B. (2009). Hot spots do not exist, and other fundamental concerns about hot spots policing. In N. Frost, J. Freilich & T. Clear (Eds.) Contemporary Issues in Criminal Justice Policy: Policy Proposals from the American Society of Criminology Conference (pp. 271-278). Belmont, CA: Cengage/Wadsworth.

Gould, S. J. (1981). The Mismeasure of Man. New York: Norton.

Note

¹ And continues to discuss in greater detail in his forthcoming book, Taylor, R. B. (2015) Community Criminology. New York: New York University Press.

² In discussing this with John Eck he added the suggestion of a Schrödinger’s policy, where two policies sit in a box and are both in a state of existence, until someone looks at the data. Then only one policy becomes viable and exists.