Friday, 4 January 2013

Better Together An audit is meaningless without effective follow-up

Better Together

An audit is meaningless without effective follow-up

by J.P. Russell

Conducting an audit correctly is important because if it’s done wrong, the results may not be credible. But even if everything was done appropriately, following up on the findings is just as important because it gives meaning to the audit.

The two actions—the audit and the follow-up—must go together for an audit program to be effective. While the audit gathers information about the existing process or system, the follow-up audit gathers data about auditee (or process owner) actions to fix problems found during the audit.

Consistent conclusions

In a process or system audit, there are findings and an audit conclusion that should be consistent with the findings.
Findings may include conformity and nonconformity statements, opportunities for improvement and systemic issues regarding the effectiveness and efficiency of the management system. Conclusions may require maintaining the status quo, adjusting the amount of oversight or recommending awards such as a certification or license to operate.
Findings of conformity may require no follow-up. Conformity indicates the system operates according to the requirements and is effective. This gives management confidence the organization goals and objectives will be achieved.
If there are nonconformities, the auditee is responsible for taking action. A nonconformity (noncompliance) may result in the auditee taking remedial or corrective action. The audit should be followed up to verify the agreed-on corrective actions were taken.
It is the responsibility of the auditee to keep management informed of the status of actions taken. Corrective actions can be followed up as a separate audit or as part of a subsequent regularly scheduled audit.
Timing to address nonconformities may depend on the importance or urgency of implementing a solution. The follow-up may be conducted by a member of the audit team or other person assigned by management.
If an audit is conducted on a prospective supplier to determine its capability to meet customer requirements, following up on actions to address nonconformities may not be necessary.
If remedial actions were taken, it may be appropriate to stop or discontinue them at an agreed-on time. Some remedial actions—such as sorting, containing, selecting and extra checking—increase costs and should be stopped when the corrective action is implemented. If remedial actions become part of the solution, the extra costs should be justified.

Making the correction

In many cases, nonconformities and findings are systemic and require corrective action to eliminate the cause of the nonconformity, defect or undesirable situation.
Eliminating the cause will ensure the problem does not recur as a result of the same cause. It’s necessary to verify that corrective actions are effectively implemented and reviewed to ensure the action taken was suitable, adequate and effective.
Use the following checklist to verify implementation of the corrective action:

Are people aware of the change and the reason behind it?
Was there training?
Were documents changed?
Were responsibilities defined?
If affected, was the customer notified?
Is the change being followed consistently?

We want the corrective action to fix the problem. This is where the audit program pays back the organization. The person reviewing the implemented corrective action must verify the cause was eliminated by ensuring the output process goals and objectives have been achieved.
The request for corrective action should be assigned to the process owner or a team to determine the root cause and actions to eliminate the cause of the problem. The approved corrective action plan should include measures to determine whether the corrective action was effective.
The measures should include output or deliverables of the process, as well as process capability (effectiveness and efficiency). The person reviewing the corrective action should use the measures to determine whether the corrective action was effective. Quantifiable measures must be used to determine effectiveness.
Output measures may include product characteristics, delivery of a service, product use, service deliverables and customer feedback in the form of complaints or surveys. Process measures may include monitoring efficiency (cost), consistency and variation.
The output of a process may include product and service deliverables, such as treatment, product or venue rental, planning, delivery of a product, application of expert knowledge and benefit plans. It’s important to look at the product and the process because a solution to eliminate causes of a problem should not come at the expense of process effectiveness and efficiency.
To verify the effectiveness of the corrective action, use this checklist:

Were output measures defined and achieved?
Were process measures defined and achieved?
Are processes capable?
Are there records?

Don’t let it happen again

Depending on the purpose of the audit, opportunities for improvement—including potential nonconformities, problems or undesirable situations—may be observed and reported. Preventive action eliminates the cause of the potential nonconformity, problem or undesirable situation.
Preventive action is proactive, and corrective action is reactive. The person checking preventive actions should follow a path similar to the one for checking corrective actions.
Many organizations do not require improvement opportunities be addressed by the auditee (process owner). Because it is not a nonconformity (noncompliance), management has the option of addressing it or not. But why not require improvement opportunities be addressed? Not addressing opportunities for improvement is minimal thinking rather than maximum thinking.
If a person conducting an audit or observing a process thinks there is an opportunity to improve it, why not follow up? It is interesting that organizations develop programs to nurture feedback from employees but dismiss improvement opportunities from an audit because it is not required to be compliant with a standard.

It’s about time

Timing is everything. This is true when it comes to checking corrective actions. If you are auditing a management system for the first time and find the procedures have only recently been issued, what can you conclude based on your assessment? Do you have confidence the procedures are effective? Corrective and preventive actions should be implemented and reviewed at agreed-on times.
Timing the review of corrective or preventive actions after implementation will vary, depending on the nature of the actions, industry and organization culture. I would start with a minimum of three months for the initial review of the actions taken. The final review could be scheduled for 12 months from the initial implementation.
Why perform a second review? Because you must ensure the actions taken are sustained. You want to ensure the actions taken are fully integrated into the management system. It may seem like extra work, but after you truly eliminate causes of problems and potential problems, resources are freed to advance and improve the organization.
You may want to follow the same steps to review innovative actions taken to change a process or system to better meet organization objectives.¹ For example, innovative actions may address objectives to control costs, improve safety, lower risk or improve customer satisfaction.
After the final review is complete, close out the corrective or preventive action request and celebrate the team’s success and the benefits realized for the organization. It is time for interviews, news releases and awards. Perhaps an annual meeting would be a good time to recognize contributions.

Uh-oh

Occasionally, the cause of the problem or undesirable situation is not eliminated. This shouldn’t happen, but it does.
Either the approved corrective or preventive action plan did not eliminate all the causes, or it didn’t eliminate the right cause. If the corrective action is not effective, it needs to be recycled. If you recycle several corrective actions each year, you should revalidate the corrective action and problem-solving processes.
As an auditor, I have reviewed approved corrective actions that were nothing more than remedial action. The so-called corrective action may appear to be effective because the measurements are wrong. What should an auditor do? Is the auditor there to verify that the planned corrective actions were implemented? Or should the auditor report that the corrective action plan was wrong from the very beginning?
On one hand, ISO 19011 and the ASQ certified quality auditor body of knowledge emphasize the importance of auditor competency. A competent auditor should report his or her assessment of the situation and request that the corrective action be recycled.
On the other hand, if an auditor (or other person assigned follow-up responsibilities) questions the creditability of the corrective action, the auditee (process owner) may become defensive. In response, management of the auditing organization (or department that approved the corrective action plan) may also become defensive.
In the midst of this back and forth, the auditor (or person assigned to follow-up) feels like the target. If that individual is assumed to be competent, his or her judgment should stand without question. At that point, there should be a process to address this situation, such as a higher-level review by people who are free of bias.
Results come from checking, not expecting.² All agreed-on action items must be checked, which requires minimal time and resources for well-managed organizations with a sharing and trusting culture. Verification of corrective actions closes the loop on actions to fix problems or implementation of improvements. Closing the loop ensures your organization benefits from your audit program and other assessment activities.

References

J.P. Russell, Continual Improvement Assessment Guide, ASQ Quality Press, 2004, p. 38.
Ibid., p. 2.