Assessment of Inspector Confidence in Conducting Manufactured Food Inspections in Virginia
Rashelly Bland, CP-FS
Food Safety Manager - Virginia Department of Agriculture and Consumer Services
Abstract
This research project sought to assess the relationship between an inspector’s manufactured food training curriculum and their confidence levels in conducting manufactured food inspections. The Food Safety Modernization Act’s goal of transitioning from a reactive approach to preventive control standards to food safety necessitated significant changes both in how manufactured food inspections were conducted, and how manufactured food regulators were trained. This research project was conducted in three phases including data collection from reports generated by the Virginia Department of Agriculture and Consumer Services (VDACS) Food Safety Program’s inspectional database, a Microsoft Forms survey sent to the VDACS Food Safety Program inspectors, and data analysis using Microsoft Power BI. Analysis of the data collected revealed high levels of confidence in conducting manufactured food inspections overall, although confidence levels varied significantly based on the type of training completed or the type of manufactured food inspection conducted. Additional training and continuing education were recommended, as well as further research in assessing a potential relationship between inspector knowledge and inspector confidence levels as they relate to conducting manufactured food inspections.
Keywords: Manufactured food, training, preventive controls, inspections, confidence
Assessment of Inspector Confidence in Conducting Manufactured Food Inspections in Virginia
Background
The signing of the Food Safety Modernization Act (FSMA) into law in January 2011 (U.S. GPO, 2011) aimed to reform the food safety system that was previously in place. The Preventive Controls Rule for Human Food, one of the seven major rules published to implement the FSMA, necessitated significant changes both in how manufactured food inspections were conducted and in how manufactured food regulators were trained. While inspections under 21 CFR 110 Current Good Manufacturing Practice in Manufacturing, Packing, or Holding Human Food (Federal Register, 1986) primarily focused on the evaluation of a food facility’s good manufacturing practices (GMPs), inspections under 21 CFR 117 Current Good Manufacturing Practice, Hazard Analysis, and Preventive Controls for Human Food (Federal Register, 2015) expanded that focus to include both good manufacturing practices and preventive controls. A limited-scope preventive controls for human food (LS-PCHF) inspection involves a broad assessment of process, sanitation, and allergen preventive controls, while a full-scope PCHF (FS-PCHF) inspection involves the inspector conducting his or her own hazard analysis and a full evaluation/review of the firm’s written food safety plans, including the facility’s hazard analysis, preventive control programs, supply-chain programs, or recall plan (FDA, 2020). Although both limited-scope and full-scope PCHF inspections require more specialized training than the GMP inspections conducted prior to the implementation of the FSMA, data on whether regulators have been provided with sufficient training to confidently conduct these, relatively new, types of manufactured food inspections, could not be found.
Ensuring food safety inspectors are provided with appropriate food safety education and resources is a necessary component towards achieving an integrated food safety system. Standard 2 of the Manufactured Food Regulatory Program Standards (MFRPS) provides regulatory agencies with a uniform framework upon which to build their respective training programs to ensure all inspectors receive the training required to adequately perform their duties. As part of this framework, MFRPS Standard 2: Training Program provides examples of training curricula for multiple types of inspections, such as good manufacturing practice, limited-scope preventive controls for human food, and full-scope preventive controls for human food (FDA, 2022). This research project sought to assess the relationship between an inspector’s manufactured food training curriculum and their confidence levels in conducting manufactured food inspections.
Problem Statement
The relationship between manufactured food inspectors’ training and their confidence levels in conducting manufactured food inspections in Virginia is unknown.
Research Questions
What food safety training have inspectors completed?
What resources are inspectors familiar with to assist them with conducting manufactured food inspections?
What resources do inspectors believe are needed to assist them in conducting manufactured food inspections?
How confident are inspectors in their ability to conduct manufactured food inspections at various types of firms?
Methodology
This research project was conducted in three phases:
Phase 1: The VDACS Food Safety Program uses a digital database called the Virginia Inspection Program Reporting System (VIPRS) to maintain its firm inventory, inspection reports, complaints, samples, and other information. The reporting functionality in VIPRS was used to identify the five most common manufactured food types in Virginia. These five commodities were included in the survey to assess whether the inspectors’ confidence levels varied significantly between commodity types.
Phase 2: A Microsoft Forms survey was created and distributed to the VDACS Food Safety Program inspectors. The survey included a combination of multiple choice, Likert scale, and open-ended questions that asked inspectors about their years of experience with the agency, years of experience conducting inspections, food safety training they’ve completed, resources they were familiar with to assist them in their duties, and resources they would like to be provided with to assist them in their duties. Inspectors also were asked to rate their levels of confidence in conducting different inspection types and inspectional tasks as they related to each of the five most common manufactured food types in Virginia. Survey responses were anonymous.
Phase 3: The survey results were aggregated using Microsoft’s Power BI data visualization software. Inspectors who had not completed any manufactured food training, and thus did not conduct manufactured food inspections, were filtered out of the dataset. The data then was analyzed to assess inspector confidence levels in conducting manufactured food inspections based on completed training, years of experience, type of inspection, type of inspectional task, and type of commodity. The open-ended survey questions were analyzed for trends and potential areas of improvement for the manufactured food inspection training program.
Results
The VIPRS database report of manufactured food firm types inspected by the VDACS Food Safety Program listed 3,127 active manufactured food firms that covered more than 25 different commodity types. As shown in Figure 1, the five most common types of food manufactured in Virginia were baked goods (1,277 firms), alcoholic beverages (653 firms), snack foods (160 firms), coffee (136 firms), and condiments (113 firms). These five manufactured food commodity types accounted for 75% of all manufactured food firms in Virginia and were included in the survey to examine whether the inspectors’ levels of confidence in conducting manufactured food inspections varied significantly between commodities.
Figure 1
Manufactured Food Firm Types in Virginia
The study population for this project was the VDACS Food Safety Program’s field inspection team was comprised of 33 inspectors during the survey data collection period in November 2022. Of these 33 inspectors, 31 (94%) responded to the survey and represented a wide range of experience as VDACS Food Safety Program inspectors: 12 inspectors reported 0-5 years of experience, three inspectors reported 6-10 years of experience, four inspectors reported 11-15 years of experience, and 12 inspectors reported 16+ years of experience. As outlined in Figure 2, survey respondents reported completing several different types of online, field, and instructor-led food safety training. This project’s assessment of inspector confidence was conducted using the results from inspectors who, at minimum, had completed both manufactured food online training and manufactured food field training.
Figure 2
Types of Food Safety Training Completed
In addition to reporting what types of food safety training they have completed, inspectors reported familiarity with a multitude of resources to assist them in conducting manufactured food inspections such as the 21 CFR 117 regulation, the FDA website, the AFDO website, the FDA’s Preventive Controls for Human Food guidance documents, training materials from completed coursework, internal meetings and internal documents, such as the Program’s 21 CFR 117 Code Reference document and 21 CFR 117 training presentations. When asked what resources they would like to be provided with to assist in conducting manufactured food inspections, 81% of inspectors suggested a combination of additional guidance documents, literature, additional coursework, additional on-the-job experience, and/or periodic refresher training.
Inspectors were asked to rate their levels of confidence in completing inspectional tasks and conducting Current Good Manufacturing Practice (cGMP) and Limited-Scope Preventive Controls for Human Food (LS-PCHF) manufactured food inspections using a five-point Likert scale of ranking responses from 1 (not at all confident) to 5 (highly confident). For this project, confidence was defined as the combination of Likert scale ratings 4 and 5. As outlined in Figure 3, inspectors reported a 90% overall confidence level in conducting cGMP inspections and a 75% overall confidence level in conducting LS-PCHF inspections. When this data was further analyzed by commodity type, the inspection type confidence levels were: 96% cGMP and 79% LS-PCHF for baked goods; 89% cGMP and 79% LS-PCHF for alcoholic beverages; 96% cGMP and 77% LS-PCHF for snack foods; 96% cGMP and 85% LS-PCHF for coffee; and 92% cGMP and 71% LS-PCHF for condiments. When filtered by training curriculum completed, the data showed that the confidence levels for inspectors who reported completing the LS-PCHF training curriculum were 88% cGMP and 70% LS-PCHF, while the confidence levels for inspectors who reported completing the FS-PCHF training curriculum inspectors were 95% cGMP and 88% LS-PCHF. When the data was filtered by years of experience, it showed that approximately one half of inspectors (0-10 years of experience) reported confidence levels of 83% cGMP and 65% LS-PCHF, while the other one-half of inspectors (11+ years of experience) reported confidence levels of 96% cGMP and 83% LS-PCHF.
Figure 3
Confidence Levels by Percentage
In addition to rating their overall confidence in conducting cGMP and LS-PCHF inspections, inspectors also were asked to rate their level of confidence related to completing certain inspectional tasks required during a LS-PCHF inspection. These inspectional tasks included determining which subparts of 21 CFR 117 apply to the firm (82% confidence), evaluating the potential hazards of the commodity type being inspected (80% confidence), assessing if the firm has the necessary components of a food safety plan (64% confidence), assessing if the firm has conducted a hazard analysis (75% confidence), assessing if the firm has the necessary components of a recall plan (71% confidence), assessing if the firm has an environmental monitoring program (78% confidence), assessing if the firm has implemented adequate allergen controls (79% confidence), and assessing if the firm has implemented adequate process controls to control significant hazards (71% confidence).
Conclusions
The following conclusions were developed based on the results of this research project:
Inspectors who completed a manufactured food training curriculum modeled after MFRPS Standard 2: Training Program reported high levels of confidence in conducting cGMP inspections and moderate levels of confidence in conducting LS-PCHF inspections.
Completing the FS-PCHF training curriculum resulted in significantly higher levels of confidence in conducting manufactured food inspections (95% cGMP and 88% LS-PCHF confidence) than completing the LS-PCHF training curriculum (88% cGMP and 70% LS-PCHF confidence).
A total of eight inspectors reported completing the FS-PCHF training curriculum in the survey although training records show that three inspectors actually completed this curriculum. Whether due to misremembrance of the courses they’ve completed, or confusion while answering the survey, the incorrect responses interfered with a true comparison of confidence level results between the inspectors who had and who had not completed this training.
In addition to the training they’ve completed, the inspectors’ years of experience also had a substantial impact on their confidence level ratings. The VDACS Food Safety Program’s 20+ year history with cGMP inspections compared to its approximately four-year history with LS-PCHF inspections played an important role in the inspection type confidence levels, yet inspectors with more overall years of experience were significantly more confident in conducting the newer inspection type than those with fewer years of experience.
Though inspectors were familiar with a variety of reputable resources to assist them in conducting manufactured food inspections, most inspectors (81%) expressed an unambiguous desire for additional resources and continuing education, particularly in relation to PCHF.
While VDACS Food Safety Program inspectors were confident in conducting manufactured food inspections, overall, there was a significant difference in their confidence levels, depending on the inspection type.
Overall, the high confidence level results for cGMP inspections compared to the low/moderate confidence level results for LS-PCHF inspections indicated a distinct need for additional training, education, and experience in PCHF.
Recommendations
The VDACS Food Safety Program should continue using the most current version of MFRPS Standard 2: Training Program as the foundation for its training program, while also supplementing it with additional training, particularly for PCHF and commodities such as condiments.
In addition to its standardized training curriculum, the Program should provide consistent opportunities for continuing education such as coursework, refresher training, and streamlined guidance documents on topics such as the multiple components of LS-PCHF inspections.
Furthermore, conducting periodic evaluations of the training program’s effectiveness and continued assessments of inspector confidence levels would be beneficial.
The Program also may benefit from further research involving an assessment of inspector knowledge and comparison of that knowledge to confidence levels in conducting manufactured food inspections.
Acknowledgements
I would like to acknowledge and thank the IFPTI team, my fellow members of Cohort XI, the VDACS Food Safety Program, my mentor Kathy Fedder, and my husband, Fenton L. Bland III, for each of their contributions to my research project and their support of my participation in this Fellowship. I am very grateful for this experience and to everyone involved for being there for me every step of the way and supporting me throughout this rewarding journey.
References
U.S. Food and Drug Administration (October 15, 2020). Compliance program guidance manual. Chapter 03: Foodborne Microbiological Hazards. Program 7303.040. https://www.fda.gov/media/131744/download
U.S. Food and Drug Administration (September 30, 2022). Standard 2: Training Program. Manufactured Food Regulatory Program Standards. https://www.fda.gov/media/131392/download
U.S. Government Publishing Office (January 4, 2011). Public law 111-353. Federal Register, Volume 76, 124 STAT. 3885. https://www.govinfo.gov/content/pkg/PLAW-111publ353/pdf/PLAW-111publ353.pdf
Author Note
Rashelly Bland, CP-FS, Food Safety Manager
Virginia Department of Agriculture and Consumer Services
This research was conducted as part of the International Food Protection Training Institute’s Fellowship in Food Protection, Cohort XI.
Correspondence concerning this article should be addressed to:
Rashelly Bland, Virginia Department of Agriculture and Consumer Services
PO Box 1163, Richmond, VA 23218
Funding for the IFPTI Fellowship in Food Protection Program was made possible by the Association of Food and Drug Officials.