Data-Driven Analysis of Problems and Assessment of Problem

a. In the Model

Just as the old adage says, “If it isn’t counted, it doesn’t count.” Compstat relies on the availability of timely, accurate, and useful data. Without this, commanders are severely handicapped in their ability to identify the emergence of hot spots, crime patterns, or trends. This data can come in various forms and from various sources but the model expects that data sets will be standardized, accurate, and comparable. Mapping is a key component of the model and represents a significant departure from simple counting of crimes. The graphic representation of criminal complaints, and the deployment of

police assets, helps commanders readily identify changes in activity and where resources are needed. While the former standard of excellence in this method of data collection and analysis was the pin map, readily available software makes mapping various events, finding relationships, and making time based comparisons significantly easier. The NYPD placed an early emphasis on mapping in the Compstat model. Jack Maple once likened Compstat without mapping to Generals deploying troops in wartime without a map (Silverman, 1999).

While different geographic commands may find different crimes and crime clusters, the data and methods they use must facilitate the comparison of apples to apples when deciding how to allocate resources especially if inter precinct collaboration is proposed. Data in the model is the building block from which most decisions are made.

What data gets collected speaks to the enforcement priorities of the agency. Data in the model also represents the scorecard by which performance is measured. The data collected during and after a response plan, is useful in determining when a response should be altered or when it has achieved the goal. Post-intervention data represents the building blocks from which the department can evaluate and communicate the lessons learned. Commanders rely on data in the model so that they can demonstrate the effectiveness of their interventions. In the absence of data, a commander neither knows a crime problem is emerging nor whether it has waned.

b. In the National Survey

Results indicated that the vast majority of responding SLTLEs are using mapping to help them identify clusters and emergent crime concerns (85.2 percent).

Almost all, (93.4 percent) conduct some type of crime analysis to identify trends. It is of interest to note that far fewer SLTLEs (57 percent) use statistical analysis in the Compstat meetings. (Weisburd et al.) The responding SLTLEs also asserted that they had access to relevant data in a timely manner. The clear majority could access incident reports, arrest reports, field interviews, call for service and citation information within seven days. This is the baseline data from which trend analysis can occur and it is a clear reflection of the model that this element seems well reflected in the responding SLTLEs. None of the

responding agencies collected and reported in Compstat on data that was reflective of police priorities other than crime fighting. One can surmise from this general omission, that Compstat, practically applied, did not place as great a priority on traffic enforcement, the diminishment of complaints, disaster preparedness, or quality of life issues as they did on crime suppression as measured in criminal complaint data. There was also a general agreement in the responding agencies that the speed and accuracy of the data was emphasized over the value of the analysis applied to that data. There was a great deal of emphasis on commanders being familiar with the data in minute detail while there was relatively little emphasis on expanding the quality or academic rigor in the analysis of that data.

c. In the Three Cities Study

The LPD, MPD, and the NPD were like the vast majority of responding agencies in the National Survey in that they had developed systems for record keeping that made UCR data available to decision makers quickly. The three agencies however made use of that data in different ways. In the LPD, clerks insured offense reports were entered into a database daily. Bulletins were distributed to the various districts on a regular basis but, these bulletins contained little in the way of analysis. The bulletins did not make it easier to identify or act on the emergence of crime patterns, trends, or hot spots and thus were seen as not useful (Willis et al., 2003). In the MPD, crime reports were entered into a mapping program on a regular basis. Commanders could regularly view maps to quickly identify hot spots and emergent crime patterns. Narrative debrief information was also collected and made available to investigators investigating particular offenders or unknown subjects linked to crime patterns. In the NPD, crime information was regularly entered into a database and then aggregated by crime type.

Commanders and ranking subordinates, rather than analysts, regularly met to discuss the data and to collaborate in an effort to identify emergent patterns, trends, and hot spots.

While NPD did not focus on mapping between Compstat presentations as the MPD did, the NPD focus on pattern identification was functional and selective when compared to the LPD method of simply listing crime. In addition to crime reports, complaints were

also regularly collected and reviewed in NPD’s Compstat process. Collecting and reporting on complaint data in the context of crime and enforcement reports signals an attempt to communicate the priority of balancing crime fighting against maintaining good relations with the public.

Referring again to the adage that what gets counted is what counts, it is clear that the Compstat data collection process communicated senior leader priorities.

Each agency collected data on the Part 1 UCR crimes. This is in line with the majority of agencies nationwide that use Compstat. What is notable is what is absent regarding data collection and analysis. With respect to collection, no agency appeared to collect data on traffic infractions as a component of crime suppression. None of the three agencies had employed a method of significantly reducing errors in reporting or data entry. None of the agencies had emphasized the collection of data related to quality of life and nuisance issues. None of the cities had developed a formal mechanism to collect data on enforcement priorities from law abiding members of the communities. These omissions could contribute to an informal and perhaps unintentional communication that traffic enforcement, error reduction in reporting and data entry, quality of life issues, and partnership with the community, are not high priorities to the senior leadership.

Another facet interesting in its omission centers on mapping as a component of Compstat in the three agencies studied. While each used mapping in the Compstat presentation, researchers assessed that mapping and statistical analysis played a very small role in how commanders made decisions regarding response strategies and resources allocation (Willis et al., 2003). In each of the cities, district commanders conducted simplistic analysis often relying on simple counting, anecdotal information, and factors distinct from evidence to plan their responses. Willis et al. attribute the gap between what analysis could be conducted by each agency’s Crime Analysis Unit (CAU) and what was used by district commanders to a sense of urgency and a lack of training.

(Willis et al., 2003). In most cases, district commanders were not trained in, nor sought a deep understanding of statistical analysis. Commanders also perceived in each of the cities that there was a premium placed on rapidity over deliberation and the slow development of an evidence based approach to a particular crime problem. While this

contradiction to the model was evident in all three cities it was most prevalent in the LPD. Commanders there read every offense report written in their districts before a Compstat meeting. The commanders spent a significant amount of their time on this task, more to avoid the embarrassment of being caught unaware of a particular event, than to glean trend or pattern information. Even in the MPD, where mapping technology and analytical capacity was the most widely used, commanders focused on simple characteristics such as time and place in their analysis (Willis et al., 2003) While Compstat brought clear gains to each agency in terms of making more data, more rapidly available, there were limited gains in the rigor of the analysis applied to that data. Willis et al. found that commanders in the LPD, MPD, and NPD, were not seeking more in depth analysis. The focus in each Department was on how to make more data more accessible more quickly as opposed to how to produce more insightful, predictive analysis to guide their separate and collective efforts. This can be attributed to the fact that commanders were judged more on their understanding of and familiarity with incidents than on the depth of their analysis of those incidents. Furthermore, it was clear, in each of the departments, that having a plan to address those incidents was more critical to avoiding sanction, than was demonstrating that the plan was the best option, focused on efficiency and effectiveness.