Enhancing the Quality and Oversight of Federal Facilities Data for Environmental Cleanup

On this page

• Background
• Implementation
• Recommendations
• Agency Comments
  • OSWER Response
  • ORD Response
  • OECA Response
• Notes

MEMORANDUM

SUBJECT:
Enhancing the Quality and Oversight of Federal Facilities Data for Environmental Cleanup

FROM:
James E. Woolford, Director
Federal Facilities Restoration and Reuse Office (5101)

Nancy Wentworth, Director
Quality Assurance Division (8724R

TO:
Regional Superfund National Policy Managers, Regions I-X
Regional Science & Technology Directors, Regions I-X

Thank you for your continued support of activities that should enable us to address perceived and real problems with the quality of data supporting our environmental restoration decisions. In an effort to move forward, this memorandum establishes specific actions that Regions must implement to enhance the quality and oversight of our data collection and environmental decision making processes. We also want to reiterate the concept that ensuring reliable data quality is the responsibility of all personnel, not just one program or the quality assurance staff.

Background

In 1997, the EPA Office of the Inspector General (0IG) identified various environmental data quality oversight activities needing greater coordination and integration at the Regional and project specific level.¹ Also, the OIG identified a set of recommendations to build more robust and reliable data collection systems; to strengthen the capabilities of our data collection processes to generate consistently data of known quality; and to increase the certainty of the data behind our environmental restoration decisions.

We have discussed the OIG findings and recommendations with you and your Federal Facilities Program Managers, as well as the Regional Science and Technology Divisions, on numerous occasions. One of the recommendations of the audit was for the Federal Facilities Restoration and Reuse Office (FFRRO) to develop a policy to be implemented by EPA regions that assures an effective quality management system and project specific planning and documentation.

Pending finalization of such a policy, FFRRO and QAD, in consultation with the regions, have identified in this guidance, specific actions Regions must implement as interim measures.

Implementation

Based on the recommendations of the OIG Audit, there are immediate actions Regions should implement if they have not already done so. (See Attachment 1 for a full list of the OIG Recommendations.) These actions address specific environmental data quality oversight activities or functional management controls that will allow EPA to: 1) clarify the roles and responsibilities of all the participants involved in federal facility response projects; 2) institutionalize more systematic and consistent approaches to the design, implementation and evaluation of CERCLA and RCRA response projects; 3) enhance the certainty and quality of environmental data collected and overseen by EPA's hazardous waste programs; and 4) enhance our ability to meet short and longterm challenges to our processes for assessing and implementing response decisions.

Regional federal facility program managers and staff must routinely consult with Regional quality assurance personnel throughout the cleanup process. (OIG Recommendation # 2). EPA Regional programs must clearly define and establish the roles and responsibilities of Regional QA staff and federal facility staff relative to the processes by which the lead federal (non-EPA) agency develops and implements data quality requirements for a cleanup agreement, permit, order, etc. Particular attention must be given to role and responsibilities relative to site-specific documentation on environmental data provided by other federal agencies that support response decisions. Typically, other federal agencies use QA Project Plans (QAPPs) to define environmental data requirements for a particular response action. (OIG Recommendation # 3). Please note that although a QAPP or similar document may not be specifically included as a deliverable in, for example, a Federal Facility Agreement (FFA) under the CERCLA National Contingency Plan (NCP), QAPPs are required and the lead federal agency is required to obtain EPA concurrence on them where the CERCLA process is being followed.²

At a minimum, QA Officers and federal facility Superfund staff should work together to ensure that requirements are based on well defined data quality objectives (DQOs) and the appropriate documentation for each data collection activity is gathered to support the implementation of environmental response decisions (OIG Recommendations # 1 & 13). Working together, we can ensure the appropriate site-specific documentation is consistent with Agency regulations, policies, and guidance. For guidance on the development of QAPPs, including examples of QAPP elements, see the document titled EPA QA/G-5. Guidance on Quality Assurance Project Plans.

Regional QA Managers must assess the effectiveness of the hazardous waste Quality System for federal facility cleanups implemented according to the Regional Quality Management Plan (QMP). A recommended approach to accomplish this requirement is through annual management system reviews (MSRs) of federal facility response programs. Also, the MSR can be used as an opportunity to set clear expectations and controls on core management and QA and quality control activities that can improve the coordination and integration of oversight of data quality activities. In particular, this would be appropriate action when an internal or external audit of the federal facility program identifies problems with the quality of environmental data collection systems or shortcoming with compliance with the QMP. For example, a Region should establish corrective measures on the management controls for environmental data collection systems to include requirements for quality management plans that discuss the nature, frequency, and the roles and accountability of cross-functional managers/staff oversight of cleanup projects (OIG Recommendations # 3 & 14).

The recommendation for using a MSR can help Regional managers meet reporting requirements under the May 13, 1998, EPA Order 1000.24 CHG 1, Management Integrity and the July 16, 1998, EPA Order 5360.1 CHG 1, Policy Program ReQuirements for the Mandatory Agency-Wide Quality System. (As a reference, see the September 29, 1998, letter from Henry Longest II, Acting Assistant Administrator for the Office of Research and Development to Assistant and Regional Administrators requesting that they review internal quality systems implementation and that they designate QA/QC individuals within their organization.) For guidance on the development of possible managerial controls, see recommended managerial controls in the July 1998 document titled: EPA Quality Manual for Environmental Programs. 5360 USEPA (Agency Directive).

At the facility/site level, EPA federal facility RPMs are expected to work with the Regional QA staff and the federal agency lead counterparts to identify measures to detect data integrity problems (GIG Recommendation # 5). Although other federal agencies are designated authorities under Executive Order 12580 to address cleanup and related data integrity requirements, EPA has responsibility to establish data quality oversight procedures or practices that ensure that unreliable data are not used in decision making at federal facility cleanups and property transfers. For example, the systematic use of effective data verification and validation protocols; the use of data quality review tools/checklists; the training of data reviewers to heighten their awareness of indicators of fraudulent practices; and the use of qualified labs may also help in reducing the likelihood of both data integrity problems and fraudulent practices. (GIG Recommendations # 5, 9 and 10)

If you have any questions about this guidance, please contact Mike Carter in the Federal Facilities Restoration and Reuse Office, at (202) 260-5686, or via E-mail at [email protected]. Again, thank you for your commitment and support to these efforts.

Attachment

Recommendations

We recommend that the Assistant Administrator for Solid Waste and Emergency Response

1. Work with Regions to ensure that Federal facility Superfund QAPPs:
   1. Include QAPP requirements that are based on well defined data quality objectives.
   2. Are prepared for each data collection activity: that is used for decision making.
2. Ensure that regional quality assurance personnel are involved in the entire QAPP process, from development of the QAPP to compliance with the QAPP.
3. Issue guidance that specifies regional oversight responsibilities for Federal facility Superfund cleanups-Ensure this guidance addresses the oversight of laboratory data quality and includes a requirement for site-specific plans that discuss the nature, frequency, and responsibility for data quality oversight activities.
4. Assess the adequacy of DoD's and DOE's environmental data management systems.
5. Establish procedures for ensuring fraudulent or poor quality data is not used at Federal facility cleanups.
6. Fully identify the impacts of the Eureka Laboratories fraudulent and poor laboratory practices on Federal Facility cleanups.
7. Develop a national quality management plan.
8. Develop performance measures for the environmental data quality system, and compare actual performance of the system to these measures.
9. Issue program-specific QAPP guidance to ensure that the following quality assurance measures are included when high-quality data is required by data quality objectives:
   1. The use of EPA national functional guidelines or their equivalent for data validation. The data validation should represent all matrices, analysis types, and laboratory decision points, and be based on the data quality objectives.
   2. Data validation performed by a party independent of both the laboratory and its parent company.
   3. On-site laboratory audits before work is started and periodically throughout the project. Also, the guidance should specify that the audits will be conducted by an activity independent of the laboratory and will include both announced and unannounced audits.
   4. Magnetic data maintained and made available to regions. In addition, magnetic tape audits should be required if major deficiencies are found by other quality assurance methods, such as data validation or performance evaluation samples.
10. Continue the development of electronic data validation, expand its capabilities, and encourage its use.
11. Create a forum for sharing environmental laboratory evaluations, such as laboratory audits, among Federal agencies.
12. Publicize best practices used in Federal facility agreements, QAPPs, and laboratory contracts to make EPA regions and other Federal facilities aware of them.

The Assistant Administrator for Research and Development

1. Refine the data quality objectives process by:
   1. Ensuring the early involvement of key decision makers.
   2. Using checklists to identify all necessary activities
   3. Identifying specific documentation requirements.
   4. Using the model developed at the Hanford site as a guide.
2. Work with the Federal Facilities Restoration and Reuse Office and regions to develop acceptable quality management plans.

The Assistant Administrator for Enforcement and Compliance Assurance

1. Request that Executive Order 12580 be modified to expressly identify EPA's oversight role for environmental data quality.

Agency Comments

The Offices of Solid Waste and Emergency Response, Research and Development, and Enforcement and Compliance Assurance generally agreed With the findings and recommendations. Their complete responses are at Appendix A.

OSWER Response

OSWER agreed improvements needed to be made in the current quality assurance oversight process. However, it cautioned that EPA must not undermine recent initiatives to streamline the Superfund process. It also stated "... We believe that having sound information to base cleanup decisions is critical, but we also must recognize the responsibilities delegated under Executive order 12580 to other Federal agencies. While we must improve our efforts, so too must other Federal agencies improve their accountability... "

OSWER agreed to coordinate with EPA regions and the Departments of Defense and Energy to assess the adequacy of DaD's and DOE's environmental data quality systems by November 30, 1997. OSWER, EPA regions, DoD, and DOE will also develop a framework for the minimum quality assurance program that the Federal facilities should have in place. This framework will incorporate currently available quality assurance guidance and information. EPA will complete the design of the quality assurance framework and the initial implementation by May 31, 1998.

Each EPA region will be responsible for verifying that its Superfund Federal facilities have established and are maintaining the quality assurance program. OSWER also agreed to develop a quality management plan, encourage the use of electronic data validation, develop a mechanism to share laboratory audit information, and encourage the dissemination quality assurance best practices.

ORD Response

The Office of Research and Development agreed to amend its DQO guidance and work with OSWER's Federal Facility Restoration and Reuse Office and regions as they develop and implement quality management plans. ORD will also provide training in the Agency's quality management system.

ORD pointed out that "... There is no system, no matter how well conceived and documented, that cannot be circumvented by unexpected environmental conditions, unintentional mistakes by staff", or intentional malfeasance. "

OECA Response

The Office of Enforcement and Compliance Assurance and OSWER, working with EPA regions and other Federal departments, will undertake a program to improve the quality of RIFFS work the Federal departments conduct. "Consistent with our long-held enforcement first principles, we applaud the OIG for supporting the need for strong EPA oversight."

OECA viewed the best approach to improving data quality at Federal facilities as the cooperative, yet aggressive, approach detailed in OSWER's comments. OECA said it remains ready to pursue amending the Executive Order if EPA fails to secure the improvements OSWER actions seek.

Notes

1. See March 20, 1997, OIG's audit report titled. Laboratory Data Quality at Federal Facilities Superfund Sites. 

2. Federal Register, Vol. 55, No. 46, March 8, 1990, Sections 300.415(b)(4)(ii),300.430(b)(8)and 3OO.435 (b).