Lessons Learned and Corrective Action from IAQG’s RMC Ove - PowerPoint Presentation

Slide1

Lessons Learned and Corrective Action from IAQG’s RMC Oversight Assessment in Montreal

1

Other Party (OP) Assessor WorkshopPalm Beach Gardens, FloridaJanuary 16, 2014

Will Tate / Robert Flaharty

Triumph Group / Spirit AeroSystems

January 16, 2014Slide2

ObjectivesOverview of the Americas Sector Audit conducted in Montreal, October 2013

FindingsOpportunity for ImprovementsStrengthsGeneral Comments for AAQGFeedback for IAQG OPMT

2Slide3

Oversight Requirement

Complete an oversight of each SMS, as defined in the 9104/2 standard (AS9104/1, sec. 13.2a)Provide a mechanism for a periodic review of the lessons learned by each of the IAQG sectors. (AS9104/1, sec. 13.2b)

West Palm Beach, FL- January 16, 2014

Oversight of all entities of the

ICOP (

Industry Controlled Other Party

)

process is required as

defined

in 9104/1 and 9104/2 Aerospace StandardsSlide4

SUMMARY OF OVERSIGHT

Overall the AAQG SMS Oversight was successfully completed as required1x Major and 4x Minor Non-conformances were raisedIn addition, a further 16x Opportunity For Improvements (OFI’s) were identifiedPage 4Slide5

Summary of Findings

Minor – Delegation of responsibility for Auditor Re-AuthenticationMinor – Timely closure of oversight NCR’sMajor – Correct application of the definition of Aerospace Audits to all AEA Re-authentication applicantsMinor – Oversight reports are not correctly documented or that records of oversight are maintainedMinor – Conflicts of interest are not assessed and effectively managed for oversight teams

5Slide6

AAQG # 01 (Minor):It was not demonstrated that the currently documented arrangements effectively reflect the current arrangements for the control of the AAQG RMC auditor authentication activity as follows:

Auditor re-authentication review and decision-making are subcontracted from the RMC Auditor Review Committee to the respective AAB’s, RABQSA and INMETRO however the arrangement is not documented (ref: RMC Proc. 103)Arrangements for the retention of auditor authentication or re-authentication records are not documented.Procedure 103 was not demonstrated to be effectively implemented as the date of release was indicated as a draft In considering Procedure 103, the AAQG RMC may wish to review the division of responsibilities between the industry and the subcontract AAB to clearly document that the responsibility of auditor authentication and re-authentication decision-making is defined as being with either the RMC Auditor Review Committee voting members or with the subcontract AABPage 6Slide7

AAQG # 01 (Minor): cont

Corrective Action:The RMC 103 Procedure will be corrected to reflect the current revision’s release date by 1, Dec 2013 and a memo will be sent to the America’s AABs (INMETRO and Exemplar Global, formally RABQSA) defining the delegation of re-authentication approval of existing auditors by the auditor review team chair by 18 Dec 2013.Page 7Slide8

AAQG # 02 (Minor):The AAQG RMC was not able to demonstrate the timely closure of oversight NCR’s in all cases (ref: 9104/2 Appendix J timescales)

Corrective Action:Update RMC Procedure 102 to include escalation process and more clearly define the roles and responsibilities of the process for escalation and oversight of the RMC for status of open issues. With the release of AS9104-002 (2013) there will be a documented requirement for escalation after 90 days when issued during independent oversight in paragraph 7.15.2 b: for “shared” or supplemental. The subject of closing and elevating NCRs over 90 days will be presented as a topic for future OP Assessor Training Workshops. Oversight Chair will monitor for closure/status of open issues and present during closed RMC meetings. ANAB documented an internal NCR in August for having 2 AB issued NCR’s (to CB’s) that went beyond 90 days and we did not move to suspend, as required.  ANAB’s corrective action was to modify our process and require a 75 day response from our CB’s on all AQMS minor NCR’s (All Major NCR’s are already at 60 days for response), this allows 2 weeks for any “back and forth” that may be needed.  This is documented in their newly released Accreditation Rule 29 (AR29).   Their EQM System workflow now flags its coordinators at day 75 and they in turn notify Bob Cruse of any that are nearing the 90 day mark so they can begin assembling an appeal panel to convene on (or as near as possible) the 90 day deadline.Page

8Slide9

AAQG#03 (Major):It was not demonstrated that in all cases that RABQSA, as the AAB for the AAQG RMC, is correctly applying the definition of Aerospace Audits to all AEA Re-authentication applicants.

This finding is a recurrence of AAQG Oversight finding from May 2011. No evidence was available to demonstrate that corrective action from the May 2011 finding had been effectively implemented. Corrective Action:The RMC auditor review team will provide a follow-up review with Exemplar Global personnel on the correct type of audits to be included with auditor authentication and re-authentication documentation packages. The Auditor review team will perform periodic random checks of re-authenticating auditors, by requesting that their re-authentication documentation be forwarded to the auditor review team for review of their audit logs. The Auditor review team will request that AB auditors that have been recently re-authenticated, have their documentation reviewed by the auditor review sub team of at least 2 RMCAR members. Page 9Slide10

AAQG # 04 (Minor):It was not demonstrated that in all cases oversight reports are correctly documented or that records of oversight are maintained and made available for oversight of the SMS.

Corrective Action:Review documents to verify if the records requirement is clearly stated and the location and process for storing is identified.Communicate the records requirement at the upcoming OP Assessor workshop. Established a secure RMC website for records retention. Access limited to RMC voting members.Review current process and update RMC procedure 102.Page 10Slide11

AAQG # 05 (Minor):It was not demonstrated in all cases that conflicts of interest are assessed and effectively managed during the deployment of oversight teams conducting oversight on behalf of the AAQG RMC.

Corrective Action:1. Revise RMC procedure 102 to be clear on definition of "Conflict of Interest" as it relates to auditing AAB, CB, TPAB or AB. Procedure 102 to match requirements defined in to-be released AS9104-002. 2. Declare in advance to the auditee any potential conflicts and obtain permission to proceed prior to conducting the audit.Page 11Slide12

SUMMARY OF FINDINGS – OFI’s

OFI#1: Guidance for Travel Policy (for OP Assessor Oversight) is informally documented but may benefit from being formally covered in Procedure 102 Oversight or stand-alone procedureOFI#2: AAQG RMC may benefit from maintaining a record of the approved AAB/TPAB procedures (with Rev status) for their approved AAB & TPAB subcontractorsOFI#3: Upon implementation of 9104-002 , the AAQG RMC may benefit from re-issuing current Form A & B’s to reflect any changes associated with implementationOFI#4: AAQG RMC may benefit from updating Procedure 102 to reflect the current operating practice i.e. excluding CB, TPAB and AAB representatives from the Surveillance Sub TeamOFI#5: The AAQG RMC benefit from introducing a documented process to ensure that 9104-002 Form A’s (previously Annex I’s) are renewed after 3 yearsPage 12Slide13

SUMMARY OF FINDINGS – OFI’s

OFI#6: It was not demonstrated how the AAG RMC recognise stakeholders from other CBMC’s and IAQG Sectors e.g. AB’s, AQMS Auditors, TPAB’s, CB’s, TP’s and Training CoursesOFI#7: The recorded Objective evidence for CB Head Office ASA April 2013 was considered minimal and would benefit from including more detailed objective evidenceOFI#8: The AAQG RMC may benefit from maintaining a record of ICOP related implementation fees e.g. Auditor Authentication fees etcOFI#9: Procedure 102 should clearly define the process for the management and escalation of NCR’s for independent OP Assessor Oversights e.g. OP Assessor contacts AB to escalate etcOFI#10: Procedure 103 needs to reflect that RMC Auditor Review Committee makes a recommendation which is then endorsed by the full RMCPage 13Slide14

SUMMARY OF FINDINGS – OFI’s

OFI#11: Procedure 101 should include a statement to ensure ‘Rights of Access’ to all stakeholder dataOFI#12: The AAQG RMC should ensure that it maintains documented records of the decision making process (vote) for suspension or withdrawal of approval for a stakeholder e.g. AAB etcOFI#13: AAQG RMC should consider introducing a Document Register that records all relevant scheme procedures and forms, including revision statusOFI#14: The AAQG RMC may benefit from increasing the duration for the 2013 Oversight to increase the effectiveness of the Office Oversight of INMETRO as the 2012 was 1 day covering both AB & AAB activityOFI#15: The AAQG RMC may benefit from maintaining a list of attendees for OP Assessor Workshop Training courses.Page 14Slide15

SUMMARY OF FINDINGS – OFI’s

OFI#16: The current AAQG TP List published in OASIS, managed by RABQSA on behalf of AAQG RMC, contains a number of approved Training Courses which are not listed in 9104-003 e.g. ‘AQMS Foundation (Tailored)’ performed by SAI Global, AS9100 Standard Blended performed by Gladhill Associates International etc; the AAQG RMC may benefit from updating this List to only reflect approved Training Courses relevant to the ICOP schemePage 15Slide16

STRENGTHS IDENTIFIED

Strong committed team of RMC members actively supporting the ICOP scheme in AAQGUse of CB Report Cards to focus CB improvementStrong working relationship with ANAB performing joint CB OversightsCo-operation and assistance of the AAQG Auditee’s throughout this oversight with a genuine commitment to improve their processes.Page 16Slide17

GENERAL COMMENTS FOR AAQG

AAQG RMC must ensure they apply the lessons learned with RABQSA to future AAB/TPAB activities – ultimately the RMC is responsible for the performance of its subcontracted activities!Consider performing an annual self SMS assessment using an independent OP AssessorBeware of potential Conflicts of Interest and how you manage them effectivelyFor future AAQG Oversight, it is requested that the RMC ensures greater support from their stakeholders e.g. ANAB, RABQSA and INMETROIt was identified that the 9104-001 Transition approval of INMETRO was not completed in accordance with SR002 timescales and suspension was not initiated in due time; however since the transition has now been completed, no NCR is being raised at this time.Page 17Slide18

General feedback for IAQG OPMT

Needs to be stronger in monitoring SMS Oversight NCR’s and ensuring these are closed in a timely mannerMust consider performing SMS Oversight in the week before the IAQG OPMT F2F meetings or as a totally separate activityConsider having a more dedicated OP Assessor team to conduct the Oversights – this would ensure a consistent approach and might help to introduce common practicesSMS Oversight provides an excellent opportunity to learn from other SMS on different approaches, identify best practice etcOASIS changes – functionality to be amended to show Suspension date for AAB/TPABPage 18Slide19

General observations by the RMC team

Validate representation by all Americas required entities for the entire duration of the auditRMC should establish a records administrator for RMC records managementValidate availability of all records 2 weeks prior to the audit dateEstablish a minimum of 2 audit teams and designate a facilitator to track down items as requiredTrack closure of audit CAs as part of our Action itemstime logRMC should conduct semi-annual internal audits (Jan/Jul)Establish a timeframe for submitting completed oversight records for retentionEnsure all entities approved by the RMC have records retained that show evidence of the approval (specify where all records will be retained)Page 19Slide20

20

Robert

Flaharty – Spirit Aerosystems Wichita, KS30 Years working in Supplier Quality

Supplier Audits, Source Inspection, Corrective Action

Systems Development

.

Will Tate – Triumph Group, Dallas, TX

35 Years working in Aerospace & Defense

15 Years Engineering, 20 Years QA and Process Improvement

Supplier QA, QMS, QA information systems

.Slide21

21