Multicriteria Ranking for Police Efficient and Effective Analysis

Nonparametric assessments of police technical and scale efficiency is challenging because of the stochastic nature of criminal behavior and because of the subjective dependence on multiple decision criteria, which can lead to a more or less efficiency prospect depending on the regulation, necessity, or organizational objective. There is a trade-off between efficiency and effectiveness in many police performance assessments, i.e., efficient departments (producing more clear-ups with a given resource) are crime-specialized or cannot reproduce those good results effectively on more severe or complex occurrences. This study proposes a combined methodology for carrying out efficiency and effectiveness analysis of Police departments. A conditional non-parametric approach, which allows to include crime as an external factor in the analysis, is combined with a noncompensatory ranking based on the PROMETHEE II methodology for the approach illustrated on the multidimensional efficiency and effectiveness comparison of 145 Pernambuco (Brazil)'s police departments. The application results offer compelling perspectives for public administrations concerning the strategic prioritization of units for rewards or interventions.


Introduction
Ranking Decision Making Units (DMUs) according to their productive performance has been the objective of Data Envelopment Analysis (DEA) applications in many sectors of economic activities for classifying both efficient and inefficient units [1,2,3]. Ranking service units provide valuable discriminations that support strategic decision-making by creating incentive structures for rewarding efficient managers, teams, resource allocations, recognizing prospective policies and best practices, changing misleading business competencies, operations and activities, and developing sustainable directions for continuous improvement [4]. It also offers clear information for taxpayers and society on investments' returns regarding public and state companies.
Ranking police units under the influence of different environments, subjective value judgments, contexts and exogenous potentials of policing and criminality is challenging in the field of nonparametric efficiency analysis due to the stochastic nature of criminal occurrences. According to Nepomuceno et al. [5], crime cannot be considered an input for or output from police departments' production system in the short-run because 1. it cannot be directly controlled (reduced), 2. stochastic characteristics (increasing police efficiency, e.g., by solving more cases and still having a more than proportional increase in crime), 3. influence of exogenous determinants (income, educational attainment, religious preference, population density, social status, age, a race and ethnicity, among others) which are neither in control nor can be coherently measured by police; 4. dependence and strict correlation with other results, 5. simultaneity bias and 6. impossibility to measure the number of crimes that have been prevented. The authors suggest using Nonparametric Robust Estimators [6,7,8], which account for extreme values in the data and permit measuring the effect of exogenous variables on the efficiency of Decision Making Units. This methodology is explained in detail in the first part of the next section.
Ranking service units also requires much effort in defining quality standards for the organization's products and services. Such a prospect is not limited to measuring decision units' technical efficiency with projections for how much outputs can be expanded and inputs contracted toward the industry's production capacity. It also extends to measuring how effective the decision unit is in achieving predefined objectives, which is strictly related to the quality of products and services. There is a recurrent trade-off between quality and efficiency [9,10,11]. Lo Storto [12], investigating the relationship between efficiency and effectiveness of public expenditure in 108 major Italian municipalities, suggests shreds of evidence for this trade-off involving indicators of public service quality (expenditure effectiveness) and DEA measures for cost-efficiencies. Nepomuceno et al. [13] investigating 88 public and private health service units in Pernambuco, Brazil, also offer some support in addition to this discussion. According to the authors, most hospitalization-efficient units are crowded public hospitals working at full capacity most of the year, which can only meet all the demand for hospitalizations (the output in the analysis) by compromising the service's quality.
Multicriteria Decision Aid (MCDA) methods are a valuable source for systematic ranking multiple alternatives based on decision criteria weighted and evaluated by one or many decisionmakers and stakeholders. In particular, Outranking methodologies such as the ELECTRE (Elimination and Choice Translating Algorithm) [14,15] and PROMETHEE (Preference Ranking Organization Method for Enrichment of Evaluations) [16. 17, 18] families offer outranking relations on the set of alternatives considering the decision marker's preferences so that an alternative x (e.g., university, bank, hospital, police department, vendor, city, region, person), for instance, is claimed to outrank another alternative y if, and only if, x is at least as good as y and there is no strong argument to contradict this assertion [19]. In this paper, we combine the Full Disposal Hull (FDH) [20] directional efficiency measure for policing conditional to crime as an exogenous factor with the PROMETHEE outranking for a complete classification of Pernambuco's municipalities based on the efficiency of each police department in solving three types of crimes violent crimes (CVLI), street robberies (mugging) and carjack (or more generally motor vehicle theft and robbery) using officers as input and based on the effectiveness of reaching Pact for Life state goal on reducing homicides [5,21,22]. Some assessments of crime and policing in Pernambuco has used multicriteria and geospatial tools for ranking, clustering and classifying units and regions according to the vulnerability to homicides [23,24,25], preference learning [26,27], and property crimes [28,29]. Despite providing valuable support for policymakers, to the best of our knowledge, ranking regions or police units for public security purposes based on a multicriteria combination of Nonparametric Robust Estimators for the technical efficiency with measures for effectiveness of results is not featured in the current literature. In the next section, this combined methodology is presented in detail, highlighting the potentials that are not limited to this application. The third section is dedicated to evaluating and discussing the compensatory and non-compensatory rankings generated from different perspectives, which can aid substantial subsidies for strategic decision-making. The conclusion summarizes the proposal, empirical evaluation and contributions.

Methodology
Many DEA ranking methods in the Productive and Efficiency Analysis literature consider postanalysis approaches [3]. The framework illustrated in Figure 1 can be situated in this classification. Four sub-ranks are constructed through pairwise comparisons. Compensations between efficiency and effectiveness are restricted with vetos' imposition, which works to cluster effective/ineffective and efficient/inefficient alternatives (municipalities). The municipality is top-ranked when it is sufficiently effective according to the predefined objective and efficient in using the available resources to produce clear-ups for the specified felonies and misdemeanors. The second sub-rank has effective but not efficient municipalities, i.e., excellent efficiency prospects cannot offset poor effectiveness. If the municipality is efficient in using the available resources to solve crimes but is not effective in reaching the specified institutional goal, it is located in the third sub-rank with similar municipalities. The last sub-rank has both ineffective and inefficient units. The PROMETHEE II net flow coefficient outranks the units in each sub-rank of this framework.

Effectiveness: The Pact for Life Initiative
The Pact for Life program (Programa Pacto pela Vida) [21,22] was Pernambuco's state policy for police repression and social prevention of crime and violence. The initiative based the preventive and repressive actions on reducing impunity of violent crime through the integration of the public security system, readjustment and modernization of processes, protocols and routines directly related to the police operational capacity the Criminal Justice system, consolidation and integration of information, dissemination and democratization of data, and strategic training new police officers. Focusing on reducing homicides prior to any other criminal typification, the Pact for Lige program was one of the most successful state-level projects in their first years of execution The program's goal is to reduce homicides by 12%. Pernambuco's municipalities a re considered effective when they reach this coefficient. Because crime is a stochastic concept which depends on many objective and subjective determinants external to the police department field of action, the interesting perspective in this program is trying to integrate the many many sectors and public institutions (police courts, justice courts, the prison system, the community) which is part of the solution. In this analysis, the effectiveness veto of 12% is used to improve DEA's discriminant power, supporting a more appropriate ranking of both efficient and inefficient police units in the state.

Efficiency: Conditional Frontier Analysis
Conditional Frontier Analysis [7,8] with its directional version [30,31] is part of the Nonparametric Robust Approach proposed to overcome some drawbacks in the traditional DEA measures for the technical efficiency. The application of this methodology permits an assessment of police performance by considering criminal data as exogenous variables (Z) that are not under the control of the service units but may influence their performance. Consider a particular DMU j from a set of m decision units using i = 1, 2, 3 …, n inputs to produce r = 1, 2, 3 …, s outputs. The directional efficiency is obtained by choosing a feasible direction g(x,y) ≥ 0 for contracting resources (inputs) and/or the expanding results (outputs) to reach the industry efficient frontier: The evaluation is sensitive to the selected directions. The choice for the data direction g(x,y) = (X,Y) or for the unit vector g(x,y) = (1, 1) is the most appropriate in many circumstances due to the simplicity for aggregations. In this assessment, we choose the unit vector. See a brief discussion on the direction's choice in [32] and a review of methodologies in [33]. Exogenous factors can affect police production by changing the input/output configuration or by affecting the distribution of inefficiencies. Including potential exogenous determinants of efficiency is based on the premise that the joint probability distribution H(XY) = prob(X ≤ x, Y ≥ y) when conditional to Z = z can define an attainable production such that x can produce y when Z = z. For the directional efficiency, the conditional directional distance function is defined as [30,31]: is the joint conditional probability that a given unit j with production (X, Y) dominates the unit under evaluation with production configuration (x, y). This is empirically developed using a non-parametric estimator that smooths the multiple exogenous factors Z in the neighborhood of z from a sample q = 1, 2, 3, …, p of observations based on an appropriate kernel estimation and choice for a bandwidth vector. Three models were developed to derive FDH directional efficiency measures which compose the decision criteria in the ranking of municipalities: a model for Violent Crimes clear-ups as output (CVLI -Crimes Violentos Letais e Intencionais), a model for Street Mugging as output (Assalto a Transeúnte) and the last one for Carjacking as output (Roubo de Veículos), all conditional to the occurrences, using sworn and administrative officers as input.

Outranking: PROMETHEE II Net Flow
According to Brans et al. [17] the PROMETHEE method offers a preference function for each decision criterion in the elicitation process and prioritizing multiple alternatives. The intensity of preference for an alternative x over another option y, Pi(x, y), is developed through differences in performance levels on that criterion for the two alternatives fi(x) -fi(y) ∀ i = 1, 2, 3… I decision criteria.
This takes on values between 0 and 1. The decision-maker defines a standard function shape (usual, u-shape, level, linear, Gaussian), which specifies potential parameters according to their evaluation. The preference index P(x, y) is defined as a weighted average of preferences on the individual criteria: This elicitation's weights represent a notion of global importance in the decision -maker's perspective, instead of scaling factors (trade-offs). When the decision-maker cannot provide or is not willing to provide details on how critical are each decision criter ia, an alternative is using ROC weights [34] for evaluating criteria with partial information about attribute weights. In this methodology, ranking the least to the most important criteria in the decision-maker's perspective is sufficient to obtain the optimal weights to be considered in the pairwise comparisons for definitions of outranking flows. The weights for each criterion are determined by: Where L is the total of criteria from w = k to L. After defining weights and potential threshold parameters (for preference and indifference comparisons), the PROMETHEE preference index provides a valued outranking relation for ordering alternatives. The positive outranking flow and the negative outranking flow (over all alternatives under consideration) are defined as follows [19]: The Thus, an alternative a outranks b if Q(x) > Q(y), and it is indifferent when Q(x) = Q(y). This complete preorder is more disputable than partial orders derived from the individual positive and negative flows due to some information loss. Nevertheless, they are attractive in providing a DEA post-analysis non-compensatory ranking of DMUs.

Results and Discussion
The following tables report the overall Policing Effectiveness-Efficiency application for a noncompensatory ranking of 145 Pernambuco's municipalities described in the methodology. According to the results, none of the three completely efficient units (i.e., efficient in all three output models) are ineffective. For this reason, we have 3 sub-rankings instead of 4, as illustrated in Figure 1. The tables provide information on the non-compensatory compared to the compensatory ranking position, i.e., when the municipalities are all outranked in the same group without the imposition of effectiveness or efficiency vetos. We adopted the usual standard function shape by not considering indifference or preference thresholds during the elicitation process. Roc weights were used considering the four decision metrics resulted from Nepomuceno et al. [5]: Pact for Life effectiveness in reducing 12% homicides (the most important criterion, w = 0,5208333), relative directional efficiency in violent crimes (CVLI, the second most important criterion, w = 0,2708333), relative directional efficiency in carjacking (the third most important criterion, w = 0,145833) and relative directional efficiency in street mugging (the least important criterion, w = 0,0625).
The Net Flow parameter is used to outranking the units in each sub-rank. The effectiveness is measured in how much the municipality has reached the target of 12% reduction in homicides (so more is preferable, but 0.12 is sufficient). The last column for the relative inefficiency aggregates each unit's relative inefficiency scores for all the three models considering the slacks (so less is preferable and zero means the unit is efficient in all three models, with no slack for police officers). It is interesting how different the non-compensatory top-ranked municipalities would feature in a compensatory evaluation. Jucati, the first top-ranked municipality, for instance, is a small city in the agreste pernambucano (rural / wasteland region) of about 11 thousand residents and a population density of 87.92 per km². It had 4 officers as input along the year, 4 homicides occurrences (all solved), 12 street mugging (10 solved) and 15 carjacking (8 recovered). The municipality reduced from 9 homicides in 2015 to 6 homicides in 2016 (about 33% reduction) and from 6 homicides in 2016 to 4 homicides in 2017 (about 33% reduction).
When compared to the first effective but not efficient unit (Cumaru, Table 2) we can observe the compensation effect: because Cumaru, another small city in Pernambuco, could reduce completely the homicides in the year of evaluation (from 2 to zero, 100% effectiveness, w = 0,5208333) this more than compensates a poor efficiency performance (25% relative inefficiency), locating this municipality in the second position in the Compensatory Ranking, and Jucati in the 18º. Compansations of this nature can be observed all over the rankings. Non-compensatory / Compensatory ranking inversions are even bigger for Saloá (2 compared to 37) and Camocim de São Félix (3 compared to 69). The Noncompensatory ranking of units in this assessment tends to provide a fairer evaluation in line with what is expected by the policy maker.

Conclusions
Ranking alternatives is a topic of importance and relevant discussion in many decision problems due to the consequences for the strategic management of human resources, and each particular ranking reflects specific values, preferences and criteria for specific design purposes. This work has highlighted the importance of a fair non-compensatory judgment on police units' performance evaluations using multiple decision criteria based on efficiency and effectiveness. Different types of felonies have different degrees of severity, complexity and social importance. Traditional DEA measures for technical efficiency fail to consider those clear-up specifications and the quality of results in ranking police units and defining benchmarks for best practices. In this work, we have combined the advances in Robust Non-parametric Frontier estimations with Multiple Criteria Outranking to provide a more realistic and valuable public security ranking of municipalities. By including crime as an exogenous factor affecting the regular distribution of efficiencies in three police performance models, we derived more robust technical efficiency measures to be used with effectiveness goals as decision criteria for classifying the units.
Perhaps a more appropriate avenue to follow the current proposal is to consider intervals of efficiency or including indifference and preference thresholds in the decision matrix based on an elicitation of the society's values reflected by a public authority. Intervals of efficiency instead of an absolute measure of zero slacks for multiple performance models (when the directional vector diverges from the data direction) can be an exciting alternative for permitting service units to discount relative small and irrelevant differences in the efficiency score that would drastically lead to ranking reversal because of eventual unconventional weighting structures. In addition, efficiencybased models for re-allocating sworn officers and administrative police staff such as [35] can offer an additional post-analysis value added to identify efficient strategies to improve the current police productivity. All these extensions are left for future research.
Author Contributions: Conceptualization: first author; methodology: first and second authors; software: first and second authors; validation: second and third authors; formal analysis: first, second and third authors; investigation: first author; resources: first, second and third authors; data curation: first author; writingoriginal draft preparation: first author.; writing-review and editing: second and third authors; visualization: first author; supervision: second and third authors; project administration: first author; funding acquisition: second author. All authors have read and agreed to the published version of the manuscript.

Funding:
The APC for this research was funded by the Dipartimento di Ingegneria Meccanica e Aerospaziale (DIMA) from Sapienza Università di Roma.