Collaborative Reform Completion Memorandum

PROJECT PLAN #4

Date: October 16, 2024

DEPARTMENT OF JUSTICE (DOJ) FINDINGS

Finding # 28: The SFPD’s failure to fully and adequately address incidents of biased misconduct contributed to a perception of institutional bias in the department.

Finding # 30: The weight of the evidence indicates that African-American drivers were disproportionately stopped compared to their representation in the driving population.

Finding # 35: The SFPD does not have sufficient systems, tools, or resources needed to integrate and develop the appropriate data required to support a modern, professional police department.

Finding # 79: Evaluation of employee performance is not an institutionalized practice in the SFPD.

Project Plan #4 addresses the above findings and associated compliance measures that were not previously completed under recommendations 28.1, 28.4, 28.5, 30.3, 30.4, 35.3., 79.1, 79.2, 79.3. This list of recommendations will be addressed through the development and implementation of the Management Dashboard (MD) and an electronic performance appraisal system respectively. 

Background:

The SFPD has demonstrated a commitment to understanding root causes and addressing disparities. This commitment was described in detail within Collaborative Reform Initiative (CRI) Recommendation, 1.1, which received substantial compliance. As a reminder, SFPD outlined four “Contributing Factors” – individual officer actions, policies and culture, management and leadership, and relationship with community – over which SFPD has direct ability to influence, and which are likely causes of disparities. “Mitigating Actions” are those activities that SFPD employs to reduce the impact or likelihood of “Contributing Factors” to generate disparities.

Within that larger framework, the Management Dashboard will be used as one of the tools that the Department can use to identify where individual focus may be necessary and subsequent specific mitigating actions employed.

Recommendation 1.1 references the organizational approach that defines this framework in a police commission presentation from 2021 provided by Chief Scott and Executive Director Catherine McGuire (See Attachment #1: Solutions to Disparities in Policing).

Contained among the responses to other recommendations going back to 2018, implementation of mitigating actions to address the four contributing factors over which SFPD has some influence, have been presented. Below is a list of the contributing factors and associated mitigating actions, along with parenthetical notes providing just some examples of the form that mitigating actions have taken, as a reminder:

1. SFPD organizational culture or policies:
   1. Add friction (Use of Force policy and Critical Mindset Coordinated Response training (Critical Mindset/Coordinated Response training)
   2. Use objective standards (Policy on curtailing pretextual stops; 4^th amendment training)
   3. Reduce threat (time, distance, de-escalation efforts in CMCR training and policies)
   4. Institute egalitarian norms (accountability recommendations providing transparency surrounding disciplinary outcomes, improved promotions and assignment processes, and improved access to training opportunities)
2. Relationship among community and police:
   1. Increase Trust (transparency efforts and community engagement with follow- up/follow-through)
   2. Increase empathy (Sojourn, Race and Reconciliation)
   3. Increase positive contact (Sojourn and community engagement activities)
3. Individual officer characteristics or behaviors:
   1. Institute training (BiasSync, Sojourn)
   2. Build awareness and interventions using the MD. (This Project Plan)
4. Management and Leadership direction:
   1. Increase accountability (improving timeliness of disciplinary results and transparency around disciplinary outcomes)
   2. What direction about enforcement priorities is given to personnel (dashboard provides data about directions officers receive, i.e., BOLOs & other alerts)
   3. Reduce fatigue (overtime-limiting policies and enforcement)

SFPD is unaware of any academic literature which has identified conclusively that these contributing factors are root causes of police contact disparities, nor the degree to which they influence disparities. However, these factors are broad in scope and commonly understood to be contributors to the issue. With this in mind, SFPD seeks to move forward to articulate and document its approach to addressing disparities in enforcement, such that, if or when improvements are realized, they can be studied and replicated.

The SFPD has tailored the work articulated in each mitigating action to SFPD’s operating environment and continues to improve upon that work. As noted in Recommendation 1.1, “Officer characteristics” is defined as the “attitudes or beliefs, known or unknown, of individual officers that can result in disparities in the data or disparate treatment in police action.” The Management Dashboard is intended to assist supervisors in understanding whether those disparate data exist among their team and assist their team in making improvements in the delivery of police services and community interactions.

Known negative attitudes or beliefs about a particular demographic group that are deemed explicit, also considered “explicit bias,” are addressed in policy (a combination of Department General Orders (DGO) 5.17 and 2.01, among others), The disciplinary system addresses those behaviors that are explicit, thus violating the requirements stated in DGOs.

The Dashboard can also provide supervisors with the ability to review other groups of officers, providing the supervisor the ability to determine if members under their command are performing, in the aggregate, at comparable levels and whether they are engaging with community members consistently and appropriately.

For instance, the dashboard can provide information on all call responses and enforcement activity that might be influencing the officer’s discretionary work. With this context and by discussing with the officer, supervisors can understand which of the “mitigating actions” (Dr. Eberhardt’s work previously noted) should be re-emphasized, re-trained, or improved. Supervisors can also review what types of actions are taken and whether officers are applying the same objective standard in conducting their enforcement actions.

Management Dashboard (MD) – Finding 28, 30, & 35:

The Management Dashboard (MD) will utilize stop data that is collected pursuant to the California Racial and Identity Profiling Act (RIPA), as well as information from the Human Resource Management System (HRMS), and Body Worn Camera (BWC) metadata.

Supervisors will be required to review the stop data of the members under their direct supervision quarterly, while also being able to electronically review these data at any time.

The quarterly review will involve analyzing members’ stop activity at the individual officer level compared to the following 5 data groups:

• Their assigned Performance Improvement Plan (PIP) group
• Half of their assigned watch
• The entire watch at their assignment
• Their entire station or command
• The entire watch citywide

This comparative analysis will help supervisors determine if there is a disparate pattern that may be within the member’s control. This will enable supervisors to have the greatest “wide angle” lens to be able to assess individual member enforcement patterns.

To get the most relevant information for supervisors and ensure a procedurally-just process for the members, the Department ceased collection of RIPA data through the State-provided Stop Data Collection System (SDCS). This change allowed for data collection in additional fields which provide more context about the circumstances or information that led to the stop’s initiation.

SFPD partnered with Benchmark Analytics to develop a system that complies with the requirements of RIPA and enables the Department to collect data critical to understanding discretion members had when taking enforcement action (Attachment #2 DN 23-108 Transition to Benchmark Stop Data System).

To assist with the integration of data and reporting with the state system, SFPD’s Business Analysis Team (BAT) worked with Benchmark and Cal DOJ to ensure successful transfer of data required under RIPA. Initial transfers have been completed and system improvements are ongoing as needed.

Benchmark and BAT have established a good working relationship to make updates to the stop data reporting system as required by regulatory or legal changes. This was demonstrated when a required field (officer gender) was removed by way of court order in quarter 2 of 2024 (Attachment #3: RIPA Data Collection).

The SFPD CRI team and Business Analysis Team (BAT), in partnership with Benchmark, are currently managing the oversight of data accuracy related to issues such as compliance trends, anomalies, the review of selection for reasons for stops and other such measures. This oversight will later be conducted by the Dashboard Review Unit (DRU) in conjunction with the Staff Inspections Unit (SIU) through dip-sampling audits. Compliance with RIPA reporting requirements are also reviewed by the Department of Police Accountability (DPA) if a citizen complaint is initiated or through their audit authority.

An example of oversight related to data accuracy in the stop data system was established when the Department amended the term “On-View” as well as the definition to “Required Action” as it had led to some confusion among members (Attachment #4 DN 24-071 Benchmark Stop Data System Change).

Categories of Stops:

The Department has worked with Benchmark to modify the RIPA data element of whether the stop was in response to a call for service. While this data element is helpful for law enforcement agencies and the public to know, it does not assist in determining how much discretion an officer had in what the 2024 RIPA Report describes as “Officer-Initiated Stops.” Officer-Initiated Stops accounted for nearly ninety percent of all stops, however the level of discretion for these stops was not previously known.

The following table represents the “Reason for Contact” area in the benchmark system, and each category depicted here is described in detail below:

Image

The Department has modified the question of whether a stop was in response to a call for service, to “stop made in response to” with a series of options from which to choose. The six options represent varying degrees of discretion. They are as follows:

1. Call For Service: When a stop is made by a member in response to a call for service.
2. Required Action: When a stop is made by a member for a felony crime in progress, misdemeanor crime of violence, misdemeanor property crime, or misdemeanor where a member can articulate a threat to public safety (reckless driving, driving under the influence), member of the public alerting them to an incident, or a person is in medical distress (physical or psychological).
3. Self-Initiated Activity (SIA): When a stop is made by a member for conduct that they believe there is reasonable suspicion or probable cause that a violation is, was, or about to be committed that does not fall within the guidelines of a “required action.” This would include a consensual encounter that becomes a detention.
4. Command Directed: When a stop is made by a member when they are instructed, deployed, or detailed to specific locations or districts by a higher-ranking member. These directives can be communicated to individual members, entire watches, stations, units, divisions, bureaus or Department wide. They include operations orders which contain an enforcement plan. This category also applies when a high-ranking member directs personnel to focus on a specific type of enforcement including but not limited to, auto burglary, narcotics, illegal vending, etc.
   1. This does not apply to being assigned as a sector car in a district.
5. Investigative Intelligence, Bulletin, or Broadcast (IIBB): When a stop is made by a member using specific information developed in a follow-up investigation, disseminated in a crime bulletin, or radio broadcast. A crime bulletin can request an arrest, stop and identify only, or develop independent reasonable suspicion to detain.
6. 10B-Police Law Enforcement Services: When a stop is made by a member related to their 10B assignment.
   1. A 10B assignment is an overtime assignment at the request of (and paid for) by a private person, corporation, firm or organization for the purpose of additional law enforcement services.

The breakdown of stops in this manner affords the Department greater insight into the stop data. In the case of the Command Directed category, it provides information on whether and how much influence that deployment strategies resulting from higher-level decisions or community requests have on the outcomes of enforcement. The Department, in partnership with Benchmark, developed a Command Directive database to track directives given to officers (Attachment #5DN 23-109 Benchmark Command Directive System).

A quote from 2003 report by the National Research Council (NRC), Fairness and Effectiveness in Policing: The Evidence, states:

“Although scholars continually deplore the absence of research about the behavior of police agencies as a whole, most analysis continues to focus on the behavior of individual police officers.”

By capturing the command directives issued by supervisors and above, coupled with the ability for officers to identify when enforcement action was taken based on these command directives, the Department will gain valuable insight regarding the impact of such directives on enforcement. This will enable analysis on enforcement activities that goes beyond the decision of the individual officer. This represents progress in this area as evidenced by the above quote from the NRC, highlighting the lack of fulsome analysis.

These categorizations also allow for the Department to fully capture the reality that “officer initiated” enforcement is not always at the officer’s discretion to conduct a stop.

Additions to RIPA-Required Stops Information:

Additional fields have been added to the new MD system. These additions to the required RIPA data are an attempt to mitigate limitations of the data and subsequent analysis of that data. The following link is the 2024 RIPA appendix, which highlights the limitations:

Appendix - 2024 - RIPA Board - Annual Report - Racial and Identity and Profiling Advisory Board (ca.gov)

These additions also allow SFPD to better compare stops data to various benchmarking data sets. For instance, by being able to de-duplicate individuals stopped multiple times and to remove non-San Franciscans, SFPD gets closer to data set appropriate for comparison to census data. The fields added to the new MD are as follows and are described in detail below:

1. Additional members involved in a stop:
   1. For members who were involved in the stop beyond the one entering the stop data, SFPD added the ability to list their identifying information. The Department works almost exclusively in pairs, and where only one member is entering data for the RIPA information, stop data would be attributable to only about half of the personnel involved in conducting stops. Therefore, not capturing additional officers would result in an approximate 50% loss of stop data related to their enforcement activities
2. Unique identifier of those stopped:
   1. A field to record the unique arrest number (known as an “SF number”) was added to the Benchmark Stop Data System for stopped individuals discovered or known to have been arrested previously. This number enables the Department to identify when somebody who has been arrested in San Francisco has been stopped multiple times. This data element will allow for SFPD to examine whether individuals are being stopped with a frequency that could be considered harassing. In addition, this will allow for the de-duplication of individuals subject to stop when benchmarking against data sets in which those individuals are counted only once. The inclusion of this information in the data will allow us not only to understand how many stops in total are being conducted, but also how many unique individuals are being stopped rather than just the number of people who have been stopped. This has shifted to become a mandatory entry field once the SF number is known to exist.
   2. Basis for search sequencing: The basis for search field in RIPA has also been modified to allow for members to report the order of the searches that they undertook in a stop. This is an important data element update because in the SDCS, all the searches conducted were captured but not the order in which they occurred. Officers may conduct multiple searches in a stop for different reasons. For example, officers conduct a search that they are required to do. If that search yielded evidence of a crime, officers may then conduct a type of search which would not be technically required, based on the evidence of a crime. In these instances, it is important for us to know that the mandatory search preceded a search that would usually be classified as discretionary.
3. San Francisco residency field:
   1. The Department added a field to capture whether the person stopped was a resident of San Francisco. This field has three possible selections (yes, no or unknown). If the person is not a resident of San Francisco, the officer identifies what California city they reside in or they select “out of state”. If the city of residence is unknown, officers select that option but must provide a reason for why they were unable to obtain the data. This information collectively is valuable and demonstrates the Departments commitment to data validity.

Supporting the need for additions to RIPA-required stops information is the National Academy of Sciences study in 2018 on Racial Bias and Disparities in Proactive Policing stating:

“Studies that seek to benchmark citizen-police interactions against simple population counts or broad publicly available measures of criminal activity do not yield conclusive information regarding the potential for racial bias in proactive policing efforts” (Weisburd, Majmundar, Aden, et al., 2019)

It is important to note that the above study when referring to “proactive policing” does so regarding policing strategies where a goal is preventing or reducing crime, rather than in reference to individual officer pro-activity.

As RIPA regulations are ever evolving, future changes to the Benchmark Stop Data Collection System are likely. These changes will be made whenever legislation related to RIPA occurs, requiring modification to the system. As an example, for how these changes will be implemented, the following Department Notice was issued to reflect changes to RIPA that took effect in January of 2024. (Attachment #6 DN 23-198 Stop Data Regulatory Update)

Training and Mitigating Action:

The training curriculum for the MD includes the Departments commitment to obtaining the most relevant contextualized data to ensure procedurally just reviews of officer activity. This is consistent with Departmental goals of reducing disparities that are within the control of the member or the department. This is accomplished by collecting the most relevant data to make informed assessments.

The SFPD MD Training addresses both data entry in addition to guidance on employee management, coaching and intervention. The training curriculum emphasizes the critical role of the supervisor in combining review of data with their own observations of members and review the supervisor in combining review of data with their own observations of members and review of their overall work and activity. Specific to data entry, In order to ensure completeness of each entry, and compliance with RIPA, BSDS mandates data entry by preventing form submission when mandatory fields go unpopulated. In addition, as with other data sets, SFPD BAT members perform data validation steps to confirm compliance and internal consistency of logic.. When issues arise, clarification is sought by BAT from the entering officer. CADOJ also has a tool for agencies to use for data validation that SFPD uses prior to submission to ensure completeness and logical consistency. To assist supervisors with the review of member stop data and to ensure they understand their roles and responsibilities as supervisors, training has been, and will continue to be, provided. The training includes relevant laws, California Peace Officers Standards & Training (POST) expectations regarding bias-free policing, Department Notices, Department policy on bias-free policing, and training videos. Some training has already been provided, and includes the following:

• Roll-call trainings for DGO 5.17; Bias-Free Policing Policy, and DGO 11.07; Prohibiting Discrimination, Harassment and Retaliation were provided to all department members via the PowerDMS platform.
• Training via Department Notice (DN) and video were provided in the PowerDMS platform for DGO 9.07; Restricting the Use of Pretext Stops, and DGO 9.01; Traffic Enforcement, via 24-098 and DN 23-154 respectively.
• Each sworn member completes Bi-Annual Advanced Officer Training (AO) which includes a POST course titled: Beyond Bias: Racial and Identity Profiling.
• DN #23-108 Transition to Benchmark Stop Data System (Revisit Attachment #2 DN 23- 108)
  • Related videos: This DN included two training videos (from Benchmark and from the Department) related to the use of this new stop data system.
  • Stop Form QuickStart Guide: This DN also included a written guide on how to complete a stop report.
• DN #23-109; Benchmark Command Directive System (Revisit Attachment #5 DN 23- 109)
  • Related videos: This DN included a Benchmark Training Video.
  • Forms: This DN also served to release a new SFPD form #621 and provided training on completion of the new form.
• DN 23-159; Benchmark Stop Data System (Attachment #7: DN 23-159)
  • This DN provided additional training on AB953 regulations.
  • This DN provided a Cal DOJ generated Frequently Asked Questions (FAQ) document.
• DN #23-198; Stop Data Regulatory Update (Revisit Attachment #6 DN 23-198)
  • This DN included a Power DMS training video.
  • This DN provided a link to a virtual POST course at the following link:
    • California POST Course Catalog
  • DN #24-071; Benchmark Stop Data System Change (Revisit Attachment #4: DN 24- 071)

Supervisor specific training will also provide supervisors with options for action when a supervisor believes that there is a disparate enforcement pattern that is within the control of the member. The CRI Team, Bias Recommendation Group, will closely monitor and provide support for the supervisors at Southern Station during the pilot period. This oversight will include identification of appropriate courses of action on the behaviors of officers. This will be determined by the CRI team (to become the DRU), and feedback from field supervisors engaging in the pilot. Options for action may include, but are not limited to:

• One on one counseling session by a supervisor with the member
• Variation in daily assignment (car sector, foot beat, traffic car, CED detail)
• Additional training (BiasSync, POST implicit bias training, tactical communication, report writing, defensive tactics)
• Exposure to non-enforcement duties in collaboration with Community Engagement Division (CED)
• Changing of watch or assignment (with approval of the member due to MOU)

If a supervisor believes that the member’s enforcement pattern indicates inappropriate consideration of race, ethnicity, national origin, religion, sex, sexual orientation, gender identity or expression, socio-economic status, age, cultural group, disability or affiliation with any noncriminal group, that will constitute a violation of Department policy regarding DGO 5.17 Bias Free Policing and would be forwarded to the Risk Management Office for investigation.

The policies and procedures for the MD will include supervisory roles and expectations, as well as how to interpret and analyze the information provided by the MD. There will be a Department General Order (DGO) created pertaining to this policy. This will not occur until the pilot period rollout of the MD has been completed. Prior to the MD related DGO, the Department will continue to utilize Department Notices to facilitate the distribution of MD information and training.

Because the Dashboard is still in the developmental and pilot phase, official approaches to mitigate disparate results have not yet been fully established. Further, these approaches may need to be tailored to the officer, the needs of the assignment, and the actual circumstances of the behavior that resulted in disparities. DRU will use multiple resources to create and continuously improve mitigating approaches, such as the Center for Policing and Equity report and partner guidance, other academic research, and input from the San Francisco Police Academy. Proposed strategies currently include change of watch or assignment, change of partnership, change of PIP group, or additional training.

Dashboard Review Unit (DRU):

The Management Dashboard (MD) will provide the SFPD with valuable data that can assist in policy development, member performance, and contextualizing the work that the Department does every day for the public they protect.

The Chief of Police is proposing a staffed unit to supervise and manage this effort, known as the Dashboard Review Unit (DRU). Currently, the unit consists of members from the CRI Team, Bias Recommendation Group. This CRI group includes a Lieutenant, sergeant and officer who are working together to create MD related policies, procedures and training in preparation for the upcoming pilot phase. Of the DRU responsibilities that are listed below, the following are those that the CRI group will also perform during the pilot phase:

• Serve as a resource for supervisors if disparities exist.
• Ensure supervisor compliance in reviewing subordinate stop data.
• Review of organizational trends.
• Serve as a liaison between the SFPD and Benchmark Analytics.

The CRI team continues to address and support all deficiencies as they present themselves during the buildout and testing phases of the MD. The CRI team will continue to address and support the pilot phase. Southern Station was identified as the ideal place to rto initiate the MD, given the high work volume, and the variety of calls in the district.

In advance of the Department-wide rollout, the SFPD will staff the DRU with the necessary personnel to establish a cyclical process to determine the needs for system updates and/or additional training for members or other process and program improvements. The DRU will be responsible for the following after the Management Dashboard is established:

• Review of feedback from officers/supervisors regarding the MD system.
• Serve as a resource for supervisors if disparities exist.
• Ensure supervisor compliance in reviewing subordinate stop data.
• Interpret data relevance.
• Determine effectiveness of overall procedures.
• Determine effectiveness of and/or need for training.
• Determine effectiveness of interventions.
• Review of organizational trends.
• Serve as a liaison between the SFPD and Benchmark Analytics.

Overall, the DRU will provide supportive assistance to members as we navigate the use of this new tool.

Supervisory Responsibilities:

The Dashboard Review Unit will be responsible for monitoring data as described in the prior section of this document under “Dashboard Review Unit.” Access to the MD will be provided to commissioned officers and executive staff for the purpose of reviewing organizational trends. At the district station level, data will be reviewed by district station captains who will then go over this data with their patrol lieutenants. Each station captain will ensure that supervisors review data for their respective PIP groups. Captains and lieutenants will then meet with sergeants to discuss data and ensure unit compliance of data review.

Supervisors will be required to review the stop data of the members under their direct supervision quarterly. This review will involve analyzing members’ stop activity at the individual officer level compared to the following: their Performance Improvement Plan (PIP) group, half of their assigned watch, the entire watch at their assignment, their station or command, and the entire watch citywide. This comparative analysis will help supervisors determine if there is a disparate pattern that may be within the member’s control. As described above, under “Training and Mitigating Action”, future training will include information on appropriate intervention if disparities in data are identified.

Bi-annual reviews will be conducted at the same time as the Department’s Performance Review schedule. The review of data will allow for ongoing engagement, discussion and mentorship between supervisors and members. The MD will provide a detailed view of each officer’s stop history and enable a supervisor to address the question of how each officer is performing overall and relative to their station and citywide. The second bi-annual review will provide additional supporting data for supervisory review from a larger amount of data over the course of a full year.

The CRI team is working with Benchmark to create an audit log capability within the MD to review supervisory compliance of stop data review.

Pilot Period Rollout:

A pilot period rollout of the MD is essential to ensure its success. The pilot period will enable the Department to assess the MD and respond accordingly if any changes are needed. The pilot period is set to officially begin in the first quarter of 2025 and will run through the end of 2025 at one district station. Southern Station has been identified as the pilot station.

In Quarter 2 of 2024, access to the MD was enabled for all supervisors at Southern Station. All supervisors at that station underwent preliminary training for the purpose of introducing this new tool and informing them of the choice to pilot the MD at that location. Initial training was introductory pending the pilot rollout and provided in-person at Southern Station over the course of several days to account for contact with all supervisory personnel. Considering the pilot rollout, additional training will be conducted for all Southern Station supervisors in the first quarter of 2025. During the year of 2025, Southern Station supervisors will be able to navigate the MD and review the data capture related to the people they supervise to become familiar with the system.

The procedures for the MD during the pilot period rollout will be governed by Unit Order that will be approved by Chief Scott. Supervisory review will be conducted on a quarterly basis, as described in policy in the Management Dashboard Unit Order draft (Attachment #8 Management Dashboard Unit Order - Draft).

Prior to transitioning to a department-wide rollout, the CRI Team will evaluate the effectiveness, the strengths, and the weaknesses of the pilot phase and make the needed modifications to the upcoming phase based on that evaluation. The measure will be qualitative feedback from supervisors and leadership of Southern Station regarding the utility of the MD for interpreting officer’s enforcement patterns. Deficiencies will be analyzed and addressed by the team to change and alter the system and process as needed. Since the SFPD is breaking ground in reviewing data in this way, best practices and promising approaches don’t exist and SFPD will be learning by doing.

User input and the ability to course correct will also be the responsibility of the CRI Team, Bias Recommendation Group. This team will be in close contact with supervisors and members of Southern Station and will be soliciting feedback in an ongoing manner during the pilot period. This will enable the Department to assess the MD and respond accordingly if any changes are needed.

During the pilot phase, supervisors will review the data of the members they supervise on a quarterly basis. This review will be in partnership with the Professional Standards and Principled Policing Unit (CRI Team, Bias Recommendation Group) and Benchmark Analytics. This pilot rollout review will occur both for second and third quarters of 2025. This is to ensure that progress is made, and deficiencies are addressed within the MD.

Ongoing training and support for supervisors tasked with making key decisions during the pilot phase will also be addressed. In addition, the CRI Team, Bias Recommendation Group will serve as a resource, working closely with supervisors to facilitate making those key decisions.

In the fourth quarter of 2025, using feedback from members who participated in the pilot period rollout, Departmentwide supervisory training will be conducted to implement the MD for the entire Department effective during the first quarter of 2026. To ensure a timely Departmentwide rollout, the policy and procedures will be implemented through Department Notice, while the permanent DGO is in the process of being approved.

The MD pilot program and ongoing quarterly review of data will occur for an entire year between Quarter 1 of 2025 through quarter 1 of 2026. During this time, the SFPD will assess the need for improvements in data accuracy, data entry, compliance and training for the MD. Identified items will be considered and drafted into policy and training prior to the department wide MD launch in quarter 1 of 2026.

The integration of training and policy and the need for specificity and tools for engagement will continue during the pilot program. SFPD training and policy will highlight the expectations for engagement of supervisors with the data and interaction with individual members, including the tools at their disposal. Any need to course correct or improve upon training/policies will be identified and addressed during the pilot period.

As with any new system, the SFPD anticipates continuous improvements will be necessary. The goal is to identify any major deficiencies prior to the departmentwide rollout. The pilot program will allow for a thoughtful and thorough MD development phase with member input for this new system. The purpose of the pilot rollout of the MD, rather than implementing it Departmentwide, is to test and get feedback ensuring that this process is procedurally just for all.

MD Progress Reporting:

The SFPD has established a self-reporting mechanism for the continued advancement of Project Plan #4. With respect to reporting on compliance, the SFPD will share updates on this project plan, as requested, with the Police Commission. In addition, for transparency, major updates and communications will also be provided to the public via the SFPD website. The reporting will include actions undertaken, what was accomplished, what was not completed, remediation procedures and updated tasking and timelines. Reporting will cover current success and barriers to completion and will continue until the project is finished.

The department has established an ongoing partnership between the CRI Team, Bias Recommendation Group and Benchmark Analytics. The CRI team and Benchmark meet weekly and have created a Project Plan timeline for all MD and electronic Performance Appraisal updates to ensure that these systems are on target for both the pilot phases and the departmentwide rollouts. This timeline is a living document that will allow for updates, additions, and modifications to the detailed tasking proposed to get both systems up and running.

The SFPD has created and shared this timeline with CRI Partners. The timeline and ongoing working relationship with Benchmark Analytics clearly define the path forward for the development of these systems. This timeline serves to identify a strategy and goals for implementation of project tasking that remains. There will be site visits, as needed, with Benchmark pertaining to the MD, which will include on-site system testing, review of proposed timelines and to uncover any additional actionable tasks.

Management Dashboard Timeline:

The following timeline identifies the Departments proposed schedule for the MD rollout:

09/17/2024 – Completed an onsite SFPD meeting with Benchmark pertaining to the MD (Attachment #9: Benchmark MD Site visit email invite)

Q4 2024 – Benchmark/SFPD completion of Dashboard timeline and actionable tasks

Q1 2025 – Unit order (policy) released for MD Southern Station pilot program 

Q1 2025 – In -person MD supervisory training for Southern Station personnel 

Q2 2025 – Official launch of Southern Station MD pilot program

Q3 2025 – Completion of review of Southern Station enforcement with supervisors and patrol officers (AKA pilot roll-out review) from quarter 2, 2025

Q4 2025 – Completion of review of Southern Station enforcement with supervisors and patrol officers (AKA pilot roll-out review) from quarter 3, 2025

Q4 2025 – Completion of training for all Department supervisors on the MD

Q4 2025 – Department Notice (policy) – for rollout of MD Department-wide.

Q4 2025 – Rollout of MD Department-wide with Chief Scott’s video message and training videos to all sworn personnel.

Q1 2026 – Department-wide MD use commences.

Performance Appraisals – Finding 79:

In the past, the Department utilized the Performance Improvement Program (PIP), which involved paper documents and record retention of performance evaluations in PIP binders for all sworn personnel. This antiquated practice was difficult to track, making auditing for program compliance a challenge. The Department found that the practice of storing appraisals in physical PIP binders leaves them vulnerable to misplacement or loss.

The SFPD Chain of command has defined roles and responsibilities in ensuring compliance and sufficiency of tasking with overarching program goals for Performance Appraisals. The Department recognizes that appraisal of performance is critical in the relationship with members, their supervisors and members interaction with the overall organization. The Chain of Command is responsible for ensuring that the appraisals are completed in the appropriate manner and that the feedback provided will benefit the individual member and the organization, by developing and enhancing each members performance. The defined roles and responsibilities in ensuring compliance are established are outlined in DGO 1.04, DGO 1.06, DGO 1.07 and DGO 3.18.

Shift to Electronic Performance Appraisals:

Effective in quarter 1 of 2025, officer performance evaluations will become fully electronic in the Benchmark system. This will enable the Department to easily determine whether performance appraisals are being conducted, as required by Department policy. Additionally, by centralizing PIP evaluations in a computerized network, there is no need to physically transport and secure these binders every time a member is transferred to a new assignment. Instead, Performance Evaluations will be accessible remotely by supervisors. A computerized system will ultimately enable the Department to hold its members more accountable.

The review periods for bi-annual performance appraisals, in the past, were from February to July and August to January. Upon the transition to Benchmark, the performance appraisal cycle will be in line with the semi-annual seniority sign-up as codified in the current Memorandum of Understanding (MOU) between the Department and the Police Officer’s Association (POA).

Currently, the semi-annual seniority sign-up takes effect at the start of the first pay period in March and September. As such, the appropriate appraisal periods should be March-August and September-February. Therefore, these rating periods will be updated to match the sign-up periods. The transition to these new rating periods will improve supervisors’ ability to rate subordinates since they will have a full 6-month period with the members they manage prior to rating them.

The rating period of January 2025 to February 2025 will serve as the “soft launch” for this new electronic Performance Appraisal system. During the soft launch period, improvements for the new electronic Performance Appraisal system will be communicated to Benchmark in an ongoing manner. Feedback will be based on user experience and workflow. As with any new system we do anticipate the need for a continuous improvement loop. We will remain in constant communication with Benchmark to correct any issues as they arise. In March of 2025, the new Performance Appraisal system will enter a full launch departmentwide for the full rating period. At this time, audits will be conducted and any lack in compliance will be addressed.

Performance Appraisal Policy:

The procedure for completion of electronic performance appraisals has been drafted in the new “Performance Appraisal Guide”. This new guide will provide direction to members on the transition to the electronic system and explain the new process for performance appraisals. The new Performance Appraisal Guide has been reviewed by the SFPD Labor Relations Director. (Attachment #10: Performance Appraisal Guide revision 2024 – 10.7.24).

The new Performance Appraisal Guide provides instructions for completion of the new electronic Performance Appraisal form in the Benchmark System. The electronic performance appraisal form and a revised Performance Appraisal Guide will be used by all sworn staff of the Department beginning in quarter 1 of 2025 for the rating period of September 2024 to February 2025. The distribution of the Performance Appraisal Guide will be via Department Notice (DN) (Attachment #11: Draft DN Performance Appraisal Transition to Benchmark). That same DN will include training videos for the use of the electronic Performance Appraisals. The rating period of September 2024 to February 2025 will serve as the “soft launch” for this new electronic Performance Appraisal system. The full launch will occur in March of 2025.

Training for Electronic Performance Appraisals:

Training on the new performance evaluation system will be included in the Unit Order (Revisit Attachment #11: Draft DN Performance Appraisal Transition to Benchmark), and in the new Performance Appraisal Guide (Revisit Attachment #10: Performance Appraisal Guide revision 2024 – 10.7.24). This new guide provides instruction for completion of the new electronic Performance Appraisal form in the Benchmark System. The electronic performance appraisal form and a revised Performance Appraisal Guide will be used by all sworn staff of the Department beginning in quarter 1 of 2025 for the rating period of September 2024 to February 2025. The distribution of the Performance Appraisal Guide will be via Department Notice (DN) (Revisit Attachment #11: Draft DN Performance Appraisal Transition to Benchmark). That same DN will include two Benchmark training videos for the use of the electronic Performance Appraisals. The rating period of September 2024 to February 2025 will serve as the “soft launch” for this new electronic Performance Appraisal system.

Rating Criteria:

The new performance appraisal form will simplify the rating criteria. The new rating criteria is described in the new Performance Appraisal Guide (Revisit Attachment #10: Performance Appraisal Guide Revision 2024). The new form will also allow for flexibility in rating officers, as not all assignments are the same and therefore require the ability to rate members according to a variety of duties and responsibilities.

The form will have the following free text fields:

• Duties and Responsibilities
• Performance Objectives / Expectations
• Training Plan

The above will be completed consistently with the workflow at the start of a rating period and will be signed by the evaluator and the person receiving the evaluation.

The ratings for completing the performance appraisal will be:

• Exceeds Expectations
• Meets Expectations
• Below Expectations

The following fields will be completed once the rating period has concluded:

• One of three rating options of “Exceeds Expectations”, “Meets Expectations”, or “Below Expectations” will be selected.
• A section for “General Comments”
• A section for “Areas for Enhancement”
• A section for “Areas of Development”

The last section of the performance evaluation will have an area for signatures once the Performance Appraisal is approved and accepted. There will also be an area for the person being evaluated to request to review/discuss the evaluation with the next level of command in lieu of signing. The person being evaluated can also enter comments to rebut an evaluation in this section, as is their right per California law.

The performance appraisals are based upon the expectations that were set at the beginning of the rating period. If a member is rated “Below Expectations”, an additional field will appear that requests a date from the member completing the appraisal to input the date that the member was informed their performance was not meeting expectations. This additional field will ensure that members completing the appraisal will attempt to elevate the underperforming member’s performance prior to the end of the rating period.

Auditing of Performance Appraisals:

In March of 2024, the Staff Inspection Unit (SIU) completed an Annual Inspection Plan (AIP) for the year 2024 (Attachment #12: 2024 Annual Inspection Plan). This AIP was written before the Benchmark system went live, and thus the methodology will change a bit, but the purpose of the audit will remain the same. The SIU will audit compliance of performance appraisals to ensure that supervisors have completed the most recent performance appraisal.

The Officer-in-Charge of SIU will continue to conduct annual audits for compliance within the new electronic performance appraisal system. The first electronic audit will occur in September of 2025, after the transition to Benchmark, and will be based upon the rating period of March to August of 2025 for Performance Appraisals completed in the new Benchmark System. This audit will review a random sample of supervisors to determine the level of compliance for electronic sign off on Performance Appraisals in the Benchmark System during that period.

The SFPD anticipates that this first audit will show improved compliance as compared to PIP binders. However, the Department recognizes that results may not be perfect, as there is a learning curve with any new system and corrective action with system use is anticipated. Another electronic audit will occur in quarter 1 of 2026 after the full rating period of September 2025 to February 2026. The SFPD anticipates that at that time, the audit will reveal a significant increase in compliance departmentwide for Performance Appraisals.

During the initial launch period, improvements for the new electronic Performance Appraisal system will be communicated to Benchmark in an ongoing manner. Feedback will be based on user experience and workflow. As with any new system we do anticipate the need for a continuous improvement loop. We will remain in constant communication with Benchmark to correct any issues as they arise.

The SFPD Chain of Command will also play a key role in ensuring compliance and sufficiency of responses with the overarching program goals. The duty of compliance in completion of performance appraisals will be modeled through the chain of command, since performance appraisals will be done for all ranks. The chain of command will respond to electronic reports that detail if members of their command have not completed performance appraisals. Additionally, the new system will provide electronic sign-off reminders, coupled with built-in auditing systems through the Staff Inspections Unit as part of their Annual Inspection Plan (Revisit Attachment #12: 2024 Annual Inspection Plan).

Response to Non-Compliance:

A non-compliance review and training for supervisors will occur if noncompliance of performance evaluations is identified. Electronic reports will be generated to determine if any member has not received a performance appraisal. This will serve as the first review of potential non-compliance. Additionally, a non-compliance review will be conducted in regular auditing by the Staff Inspection Unit, as established in our Annual Inspection Plan (Revisit Attachment #12: 2024 Annual Inspection Plan). If non-compliance is identified, a retraining will be required for members out of compliance.

Promotional Factors:

As part of their findings, the DOJ recommended that the SFPD use performance evaluations as a factor in promotions. In response to that, the Department is now working with the San Francisco Department of Human Resources (DHR) to factor in performance evaluations in promotions. In August of 2024, the SFPD initiated conversations related to a request that the DHR amend the San Francisco Civil Service Rules to include performance appraisals in “secondary criteria” for candidates for promotion. If approved, the Department would request that DHR include the last two years of performance appraisals (from Sept. 2024 on), total of (4) appraisals since they are conducted bi-annually in subsequent job announcement applications for the promoted civil service ranks.

The timeline for introduction of performance appraisals into the promotional process rests with the Department of Human Resources (DHR). This requires approval from the Civil Service Commission and an amendment to the Civil Service rules. Additionally, all active promotional lists are in effect through the end of 2024. The current sergeant list expires in June of 2025, and the lieutenant and captain lists expire in quarter 4 of 2025, if they are not extended. Since these lists are still active, the inclusion of performance appraisals cannot be incorporated in this promotional cycle for all three ranks. However, we anticipate inclusion in the next promotional process for all civil service ranks, if approved by DHR.

Performance Appraisal Timeline:

The following timeline identifies the Departments proposed schedule for the rollout of the new electronic Performance Appraisal System.

Q1-Q4, 2024 – Sandbox testing of Performance Appraisals (Attachment # 13 – P.A. Sandbox testing emails).

Q4, 2024 – Revised Performance Appraisal Guide provided to Labor Relations Director for review/editing. COMPLETED (Attachment #14 - Email for meet and confer draft of PA Guide)

Q4, 2024 – Department Notice for training/communication to all members related to new performance appraisal system in Benchmark. 
NOTE: This will include training videos and will serve to release the new Performance Appraisal Guide.

Q1, 2025 – Launch begins: Transition to the new performance appraisal form consistent with the revised performance appraisal guide.

January to February, 2025 – Soft launch

March, 2025 - Full launch.

March, 2025 - Request that Department of Human Resources amend the Civil Service Rules to add performance appraisals for inclusion in secondary criteria for future promotional examinations. The request will be for the wording to reflect the inclusion of (4) previous performance appraisals (2-year period) completed after the 2024 revision of the performance appraisal guide. NOTE: This is contingent upon the full launch phase.

Q3, 2025 – First audit (focusing on rating period March 2025 – August 2025).

Q1, 2026 – Second audit (focusing on rating period September 2025 – February 2026)

References:

National Research Council, 2003. Fairness and effectiveness in policing: The evidence.
National Academies of Sciences, Engineering, and Medicine. 2018. Proactive Policing: Effects on Crime and Communities. Washington, DC: The National Academies Press.