Assessment of Performance Factors to Aid in the Identification of Strategies to Improve Inspection Uniformity

Gretchen Boyd

Louisville Metro Department of Public Health and Wellness

International Food Protection Training Institute

2010 Fellow in Applied Science, Law and Policy:  Fellowship in Food Protection

Abstract

               In today’s complex food supply chain with food transported from across the globe, Food Safety Inspection Officers (FSIOs) are charged with the important task of protecting the food supply for consumers. FSIOs at state and local agencies collectively conduct more inspections, test more food samples, and bring more food safety enforcement actions than federal food safety agencies(Davis, S.E., & Taylor, M.R., 2009).. Particular emphasis should be placed on FSIOs competence during inspections (Davis, S.E., & Taylor, M.R., 2009). While standardization provides some assurance to program management and industry that an employee is capable of performing inspections, assurance of long-term performance and consistency must be a consideration. This project will analyze inspection results from Louisville Metro Department of Public Health and Wellness (LMPHW) against an FSIO’s profile of factors to determine if trends can be identified. The analysis will lead the discussion of how to address trends that can aid in improved standardization within a local food safety program resulting in FSIOs meeting a standard supporting an integrated food safety system.

Background

            The mission of food safety programs is to prevent a foodborne illness outbreak. The Centers for Disease Control and Prevention (CDC) states that 1 in 6 Americans have succumbed to a foodborne illness and approximately 164 billion dollars per year is spent on foodborne illnesses victims (Neal Fortin, 2009). According to the Food and Drug Administration’s (FDA’s) website, “It is critical that food safety personnel become standardized and subsequently certified through this process to ensure that retail foods are safe, unadulterated, and honestly presented at retail throughout the United States” (Food and Drug Administration, 2007). The need for standardization within food safety programs is imperative. One of the primary concerns regarding government regulators is the lack of uniformity of FSIOs and the actual inspections they conduct. Existing tools to address uniformity can be found in the Conference for Food Protection (CFP) document entitled “A Guide to Conducting an Assessment of Training Needs in Retail Food, Restaurant, and Institutional Foodservice for Regulatory Food Safety Inspection Officers,” (Certification of Foods Safety Regulation Professionals Work Group, 2008) in the FDA Retail and Manufactured Food Standards, and in various accreditation programs. Despite these existing sources, a constant challenge to food program management is to ensure that staff members apply uniformity when conducting inspections.

            To address non-uniformity, the Kentucky State Food Safety Branch provides food core training to local departments. This training lasts one week and encompasses both classroom instruction and field inspections for entry-level environmentalists. Kentucky mandates that all environmentalists pass a State Board examination to become certified as a Registered Sanitarian within their first year of employment. The State Food Safety Branch incorporates a standardization program based on FDA guidance, which is better defined and followed than at the local level. Local agencies are responsible for internal standardization of their staff, and each agency has discretion regarding the dissemination of information to staff members. However, t no steadfast guidelines or mandates exist to ensure that this process is being followed, which has resulted in a lack of compliance at the local level. For instance, the Louisville Metro Department of Public Health and Wellness (LMPHW), the largest district in the State of Kentucky, Food Hygiene program lacks such a formal standard. The LMPHW Food Hygiene program is charged with regulating approximately 3800 permanent establishments. The program consists of 13 full-time FSIOs and 4 seasonal/part-time FSIOs. Typically, three individuals are invited to go through the standardization process each year. In the past, no official procedure was in place to ensure that FSIOs received formal training from the standardized FSIOs. The frequency of risk factor violation debited varies significantly, thus demonstrating that the need for a standardization approach is clearly warranted (Figure 1).

Problem Statement

            By identifying variables that impact the knowledge, skills, and abilities of FSIOs, agencies can focus on the most effective way to address the performance of an FSIO in the field, which could lead to control strategies to help agencies monitor and identify trends as well as proactively improve standardization to achieve the goal of inspection uniformity. This project will examine various factors that may influence performance through analysis of FSIO data. 

Research Questions

·       How does an FSIO’s years of service correlate with critical violation debits and scores of B  and C  issued during inspections?

·       How does the amount of entry-level training that an FSIO receives correlate with critical violations debited and B and Cs issued during an inspection?

·       Do other factors show any trends that identify inconsistencies?

Methodology

Each FSIO completed a questionnaire to collect profile data and to identify barriers the FSIOs believe exist within the food hygiene program with regards to uniformity. Dependant variables (outcome measures) consisted of an analysis of electronic FSIOs’ inspection data collected from 2010 and January 2011 at the LMPHW. The Environmental Health Management Information System (EHMIS), the Kentucky State reporting database, was used to acquire the data. Potential independent profile factors were identified for reported associations with LMPHW’s performance variability measures. The factors used and the data collected include:

Each FSIO’s number of inspections, number of critical violations debited, and number of B and Cs issued were totaled from 2010 to January 2011. (The LMPHW employs a placarding program that constitutes facilities earning A, B, or C placards at the time of the inspections for public view). The total number of critical violations and B(passed - last inspection score of 85-89% or 90% and above with one correctible critical violation) and Cs (failed - last inspection score of either 84% or below, or at least one critical violation that was not immediately correctable) issued, was divided by the total number of inspections to calculate a proportion in the specified time period for each FSIO. For the purpose of this project, only an FSIO’s years of service and entry-level training was analyzed. The data was then placed into a histogram chart for analysis. A range of variation 3% or greater was used as a measure of significance and indicated an acceptable variance in the reported differences. 

Results

The data analyzed illustrates a number of trends with significant ranges of variation that can potentially lead to non-uniform inspections conducted by FSIOs. An assumption cannot be made that debiting more or less critical violations or issuing more or less B and Cs is a singular indicator of performance. However, coupled with the variability shown within certain factors, further scrutiny is warranted to verify non-uniformity that will in turn heighten awareness of the need for a standardization process. Figure 1 illustrates the vast range of variation between all FSIOs and the percent critical violations debited. Of particular note, the results for critical violation debits concluded a range of variation of 1% to 39%. This dramatic variation should indicate a need for a supervisor to look more closely at other factors. Figure 2 illustrates that the average percent critical violations debited decreased as FSIOs’ years of service reached 11-20+ years. Using this data, a supervisor might consider whether career-spanning training is sufficiently offered to staff. Figure 3 indicates that the more entry-level training an FSIO receives, the more critical violations are debited. 

          Potential exists to have successful compliance rates with a placarding program, however, this approach can create stress for an FSIO due to psychological pressures during interactions between operators and FSIOs. Figure 4 exhibits the total number of B and Cs issued by the FSIOs (citing no critical violations or overall violations would lead to an A rating). The range of 0 to 53 among FSIOs is significant to show a need for standardization. Figure 5 illustrates an FSIO with 0-2 years of service issued significantly more B and Cs than the rest of the FSIOs. More study would need to be conducted to see if there is any effect on FSIO decision-making as the FSIOs progress through their careers. Figure 6 illustrates the same result in both analyses, concluding that FSIOs who receive more training debit more critical violations and issue more B and C placards during inspections than FSIOs who receive less training.

            The additional data analyses conducted also illustrated inconsistency among other factors and demonstrate the inconsistencies present that potentially lead to non-uniformity. Conducting these analyses and using “course correction” tools will help agencies fulfill CDC’s Essential Public Health Service No. 9 (Core Public Health Functions Steering Committee, 1994) and aid in the implementation of FDA’s Voluntary National Retail Food Regulatory Program Standards, Standard 4, Uniform Inspection Program (Food and Drug Administration, 2007).

Conclusion/Recommendations

The findings of this project will lend themselves to additional study through applied research grants; more study is needed using data from an extended time period to identify possible trends in FSIO’s performance. Data from other agencies should be analyzed to seek trends across agencies. Researchers should seek grants to elaborate on the work started here, such as linking Meyers-Briggs evaluations or other personality indicators to performance. CFP and other groups should identify tools that can be used by agency management and supervisors to better evaluate, motivate, assess, train, and improve staff performance and uniformity. Models should be developed to show the attribution of standardization and training to foodborne illness reduction. When updating Program Standard #4 of FDA Voluntary National Retail Food Regulatory Program Standards, these results and future research findings should be considered. A toolkit for performance evaluations on field staff should be created in order to accurately assess FSIOs while working in the field.

Acknowledgments

The following individuals and entities are to be recognized for their invaluable contributions and support to the development of this project to better identify the training needs in the Louisville Metro Department of Public Health and Wellness (LMPHW) Food Hygiene Program: The International Food Protection Training Institute (IFPTI); The Association of Food and Drug Officials (AFDO); The Louisville Metro Department of Public Health and Wellness, Louisville KY; my mentor, Gerald Wojtala, Executive Director, IFPTI; Dr. Craig Kaml, Director, Curriculum Delivery, IFPTI; Dr. Kieran Fogarty, Acting Director of Evaluation, IFPTI and Associate Professor, Western Michigan University; Kelly Monahan, Environmental Health Manager, LMPHW; Joel Sparks, Quality Assurance Coordinator, LMPHW; the IFPTI staff; the IFPTI Fellowship in Food Protection Fellows; and Wayne Boyd, my husband.

Corresponding Author:

Gretchen Boyd, Louisville Metro Department of Public Health and Wellness. 

Email: Gretchen.Boyd@louisvilleky.gov

 References

Certification of Foods Safety Regulation Professionals Work Group. (2008, January 7).        Assessment of training needs pilot project report.  Conference for Food Protection.      Retrieved from http://www.foodprotect.org/media/guide/ATNPilotProjectReport-4-15-     08.pdf

Core Public Health Functions Steering Committee. (1994). National public health performance    standards; 10 Essential public health services.  Centers for Disease Control and                             Prevention.  Retrieved from http://www.cdc.gov/nphpsp/essentialServices.html

Davis, S. E., & Taylor, M. R. (2009). Stronger partnerships for safer food: An agenda for                               strengthening  state and local roles in the nation’s food safety system.  Food Safety   Research Consortium.

Food and Drug Administration. (2007). Voluntary national retail food regulatory program                                  standards.  Food and Drug Administration.  Retrieved from    http://www.fda.gov/Food/FoodSafety/RetailFoodProtection/ProgramStandards

Fortin, N. D. (2009). Food Regulation: Law, Science, Policy, and Practice.  Hoboken, New                 Jersey: John Wiley & Son Inc.

Kentucky Cabinet for Health and Family Services. (2010). Environmental Health Management

Information System.  Kentucky: Frankfort.