Active Managerial Control: Using Data to Drive Intervention Strategies in a Retail Food Safety Program
Shawna Johnson
Food Safety Lead, Boulder County Public Health
Abstract
Active managerial control (AMC) takes a proactive approach to managing risk within retail food establishments (RFE). In 2007, Boulder County Public Health (BCPH) noticed a trend between the presence of AMC practices and the reduction of foodborne illness risk-factor violations. BCPH implemented an AMC evaluation as a tool to assist RFEs to take a proactive approach to manage risk within their facility, however, the intervention strategy has not yet been fully evaluated. The purpose of this study was to determine if AMC practices contribute to inspection score and overall food safety risk within RFEs in Boulder County. Data were analyzed using inspection and AMC practices from RFEs in Boulder County through 2019. The study found that the presence of AMC practices in RFEs had an influence on inspection scores rated as “good.” As AMC scores goes up (e.g., more food safety practices in place), overall inspection scores went down (e.g., fewer violations observed). As a result of these findings, more analysis may shed light on which AMC practices contribute to the reduction of specific foodborne illness risk-factor violations. BCPH findings suggest that AMC evaluation is a useful intervention strategy, and its use could assist other food safety programs with managing risk within RFEs in their community.
Key words: Active managerial control, food program intervention strategies, foodborne illness risk factors, reduction of foodborne illness.
Active Managerial Control: Using Data to Drive Intervention Strategies in a Retail Food Safety Program
Background
Boulder County Public Health (BCPH) is the local public health agency serving Boulder County, CO. The county includes several dense, urban centers surrounded by rural mountain communities. It is the eighth-most populous of the 64 counties in Colorado with a population of 322,510 (United States Census Bureau May 2020). The BCPH Environmental Health Division Food Safety Program is the primary regulatory agency responsible for regulating approximately 1,800 retail food establishments (RFEs) within Boulder County. The program prides itself in taking a collaborative approach with the RFEs, and is at the forefront of innovation in advancing food safety. In 2017, the food safety team was awarded the Samuel J. Crumbine Consumer Protection Award for Excellence in Food Protection for its groundbreaking work, including an oral culture training program, the development of Active Managerial Control (AMC) tools, and the creation of the After a Disaster Guide for Retail Food Establishments (Samuel J Crumbine 2021).
One of the innovative approaches implemented by the BCPH food safety team is the AMC Program. AMC is used by the food industry to prevent, manage, and control foodborne illness (FBI) risk factors. By reviewing and analyzing food outbreak data, the U.S. Food and Drug Administration (FDA) identified five major risk factors related to employee behaviors and preparation practices in retail food establishments that contribute to foodborne illness. These risk factors include poor personal hygiene; improper food holding/time and temperature; contaminated equipment/protection from contamination; inadequate cooking; and food obtained from unsafe sources. (U.S. Food & Drug Administration Retail Food Code 2017).
With these risk factors in mind, in 2007 the BCPH food safety team noticed a pattern: facilities with AMC practices in place had fewer FBI risk-factor violations overall, whereas facilities with no AMC practices in place had more FBI risk-factor violations. Seeing the value of AMC practices first-hand, BCPH responded by designing and implementing an AMC evaluation (BCPH AMC Evaluation and Scoring Guide 2012). AMC evaluations then were conducted during routine inspections at facilities needing help implementing AMC practices.
Participating in an AMC evaluation also allowed facilities to become recognized for their excellence in food safety as part of the BCPH Partners for Food Safety Program (PFSP). PFSP facilities with AMC practices in place typically had fewer significant FBI risk-factor violations observed. Those facilities subsequently had better inspection scores and were thus eligible to apply for the PFSP per 2016 BCPH FDA grant results.
An initial analysis of the BCPH intervention strategies showed that AMC evaluations were not being conducted at all facilities and instead were only being used to evaluate PFSP candidates and a handful of underperforming facilities. Because of this, the data showing the effectiveness of the AMC evaluation as an intervention to reduce foodborne illness did not accurately represent AMC facilities (BCPH initial AMC results).
To evaluate the intervention strategy more comprehensively, in 2012, BCPH began conducting AMC evaluations, in conjunction with all RFE routine inspections. Findings of subsequent inspections further illustrated that RFEs with AMC practices in place had fewer violations and better overall inspection scores than RFEs with limited to no AMC practices in place. (BCPH Partners for Food Safety Program).
At the same time, the FDA was conducting the Retail Food Risk Factor Study looking at the occurrence of FBI risk factors in retail food store deli departments. The study looked at the relationship between risk-factor violations and AMC practices or “the presence of food safety management systems (FSMS)” from 2015-2016. One goal of the study was to determine the most effective intervention strategies to address out-of-compliance risk factors.
FDA findings were very similar to BCPH’s; they showed that AMC practices reduce the overall occurrence of FBI risk-factor violations. Along with identifying the common out-of-compliance risk factors of improper holding time/temperature and poor personal hygiene, a well-established FSMS was a strong predictor of compliance; establishments with a well-developed FSMS had significantly fewer out-of-compliance risk factors than those with less developed FSMS. FDA findings suggested that a well-developed and documented FSMS is a useful tool in the reduction of foodborne illness risk factors (U.S. Food & Drug Administration: Retail Food Risk Factor Study, 2021).
Generally, BCPH’s AMC evaluation data and the FDA’s risk-factor study data show that AMC practices, or the presence of a FSMS, reduce FBI risk-factor violations. However, BCPH data has not yet been fully analyzed to determine the extent to which AMC practices contribute to food safety risk.
Problem Statement
The extent to which active managerial control (AMC) practices contribute to inspection scores and food safety risk in Boulder County retail food establishments (RFE) is unknown.
Research Questions
1. What are the effects of AMC score on inspection score among different facility types in Boulder County retail food establishments?
2. What is the correlation between inspection score and AMC score for Boulder County retail food establishments?
3. What is the correlation between inspection score and AMC score for Boulder County retail food establishments over time?
Methodology
Inspection and AMC data was obtained from the BCPH Envision Connect database for the period of January 01, 2012 to December 31, 2018. Data obtained included facility ID, facility name, type of facility, inspection date, AMC evaluation date, inspection score, AMC score, out of compliance violations, and AMC practices observed, and included FBI risk-factor violations from regular and routine inspections conducted by inspectors, following the standardized Colorado Retail Food Code and standardized marking procedures. Data after 2018 was not used due to changes in regulations and code citations.
Statistical analysis of the data was performed utilizing SAS statistical software version 9.4. The data was analyzed by running descriptive statistics to describe the sample population. Wilcoxon Rank Sum was used to identify if the presence of AMC practices resulted in an overall lower inspection score (fewer FBI risk factor violations). Correlation analysis was conducted to identify the relationship between inspection score and AMC score, and to determine if the implementation of AMC practices over time resulted in an improved inspection score.
Results
The study population of 1,267 unique facilities is representative of facility types in the community. Of the 2,206 unique inspection events, 1,099 (49.9%) resulted in an AMC score of 50 or higher. In general, an AMC score of zero means no food safety practices are in place whereas an AMC score of 100 means all food safety practices are in place. Graph 1 shows the median inspection and AMC scores based on facility type. An inspection score of zero means there were no violations observed; an inspection score of 100 means there were several violations observed.
Graph 1
Median Inspection and AMC Scores by Facility Type
Facilities with a high (above 50) AMC score had a statistically significantly better median inspection score than those with a lower (less than 50) AMC score. Of those facilities with a high AMC score, the mean inspection score was 15.1 and of those with a low AMC score, the mean inspection score was 33.6.
To further illustrate the relationship between AMC and inspection scores, Graph 2 displays AMC scores negatively correlated with median inspection scores. As AMC scores increase (e.g., more food safety practices are in place), the inspection score decreases (e.g., fewer violations are observed). The correlation coefficient is (-.40150), indicating a moderate negative linear relationship between AMC score and inspection score.
Graph 2
Inspection Score and AMC Score with Trend Lines
As shown in Chart 1, a negative correlation is also noted between the difference in AMC score and the difference in inspection score over time. Among the 223 facilities that had at least three AMC inspections, the mean inspection score increased by 3.1 (e.g., more food safety violations were observed) and the mean AMC score decreased by 2.5 (e.g., fewer AMC practices were observed). This highlights the fact that when facilities stopped using AMC practices, inspection scores increased.
Chart 1
Correlation Between AMC Score and Inspection Score Among BCPH Facilities Over Time.
Conclusions
Based on the results the following conclusions were made:
1. The effects of AMC scores on inspection scores were different among facility types. In comparison to other facility types, schools and grocery finished foods had the highest median AMC scores and significantly lower inspection scores.
2. AMC practices had an influence on a good inspection score; as AMC scores go up (e.g., more food safety practices in place), overall inspection score goes down (e.g., fewer violations observed).
3. There was a negative correlation between the implementation of AMC practices and inspection scores. When facilities stopped using AMC practices there was an increase in inspection score (e.g., more violations observed)
Recommendations
Conclusions lead to recommend the following for future research:
1. Repeat the study in other jurisdictions to determine if outcomes are reproducible.
2. Conduct data analysis to determine the correlation between specific AMC practices and specific FBI risk-factor violations.
3. Further research the differences among facility types when it comes to the presence or absence of AMC practices.
4. Provide outreach to other jurisdictions on implementation of AMC evaluation as a tool for agency use.
Acknowledgments
I would like to acknowledge IFPTI for this opportunity and other IFPTI fellows for providing support. I would also like to acknowledge Emily Payne, MSPH, an epidemiologist who assisted in data analysis, and the BCPH food safety team and Boulder County for supporting me and participating in data collection.
References
Boulder County Public Health. (2012) Active Managerial Control evaluation and scoring guide.
Boulder County Public Health. (2016) U.S. Food & Drug Administration grant results.
Boulder County Public Health. Initial AMC results.
Boulder County Public Health. (2012) Partners for Food Safety Program.
International Association for Food Protection, Samuel Crumbine Award for Excellence in Food Safety (2021). https://www.foodprotection.org/about/past-awardees/samuel-j-crumbine-award/
U.S. Food & Drug Administration Retail Food Code (2017). https://www.fda.gov/media/110822/download
U.S. Food & Drug Administration: Retail Food Risk Factor Study (2021). https://www.fda.gov/media/148247/download
United States Census Bureau. Annual Estimates of Resident Population: April 1, 2010, to July 1, 2019. U.S. Census Bureau, Population Division. Web May 2020 http://www.census.gov/
Author Note
Shawna Johnson MPH, Food Safety Lead
Boulder County Public Health
This research was conducted as part of the International Food Protection Training Institute’s Fellowship in Food Protection, Cohort X
Correspondence concerning this article should be addressed to:
Shawna Johnson, MPH Boulder County Public Health
3450 Broadway, Boulder Colorado 80304
Funding for the IFPTI Fellowship in Food Protection Program was made possible by the Association of Food and Drug Officials.