Military science, Law enforcement | High school » Bob Fritzsche - Metrology and Calibration Overview

WARFARE CENTERS CORONA WARFARE CENTERS CORONA Metrology and Calibration (METCAL) Overview NAVAL SURFACE WARFARE CENTER, CORONA DIVISION MEASUREMENT SCIENCE DEPARTMENT Bob Fritzsche (MS 20) 951.2735244 Robert.Fritzsche@NavyMil 1 Keeping America’s Navy #1 in the World 1 Measurement WARFARE CENTERS CORONA “When you can measure what you are speaking about and express it in numbers, you know something about it; and when you cannot measure it, you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind. It may be the beginning of knowledge, but you are scarcely in your thought advanced to the stage of a science.” Lord Kelvin You don’t know anything about anything unless you can measure it. Keeping America’s Navy #1 in the World 2 Overview WARFARE CENTERS CORONA • NSWC Corona Background • Why Calibrate? • Metrology and Calibration Program • Measurement Reliability • Measurement-Related Standards • Conclusion Keeping America’s

Navy #1 in the World 3 NSWC Corona Mission WARFARE CENTERS CORONA To serve as the Navy’s independent assessment agent throughout the lifecycle: "To gauge the warfighting capability of ships and aircraft, from unit to battlegroup level, by assessing the suitability of design, the performance of equipment and weapons, and the adequacy of training." Ref: OPNAVNOTE 5450 Keeping America’s Navy #1 in the World 4 Navy Organization WARFARE WARFARECENTERS CENTERS CORONA CORONA MISSION Secretary Secretary of of the the Navy Navy Operate the Navy's full spectrum research, development, test and evaluation, engineering and fleet support Center for ship hull, mechanical and electrical systems, surface ship combat systems, coastal warfare systems, and other offensive and defensive systems associated with surface warfare. Chief Chief of of Naval Operations Naval Operations Naval Naval Sea Sea Systems Command Systems Command Naval Naval Surface Surface Warfare

Warfare Center Center Reference: OPNAVNOTE 5450 Ser 09B22/1U510577 of 23 Dec 1991 SHIPS AND SHIP SYSTEMS Commandant Commandant of of the Marine Corps the Marine Corps SURFACE SHIP COMBAT SYSTEMS Carderock Division Dahlgren Division Crane Division Pt. Pt. Hueneme Division NAVSSES, NAVSSES NAVSSES,, Philadelphia Philadelphia CSS, CSS, Panama Panama City City Det., Det., Louisville Louisville CDSA, CDSA, Dam Dam Neck Neck Det., Det., Virginia Virginia Beach Keeping America’s Navy #1 inBeach the World ORDNANCE ASSESSMENT Indian Head Division Corona Division NAVSSES, NAVSSES, Seal Seal Beach Beach Philadelphia Philadelphia Det. Det. 5 NSWC Corona Division WARFARE CENTERS CORONA Commanding Commanding Officer Officer CAPT CAPT RR. Shafer Shafer Technical Technical Director Director Vacant Vacant Deputy Deputy Technical Technical Director Director JJ. Stone ( Act) ) Act Stone (Act) Measurement Measurement Science Science A. A. Hovakemian Hovakemian Product

Product Engineering Engineering Assessment Assessment DD. Sugg Sugg Performance Performance Assessment Assessment DD. Bergstrom Bergstrom Quality Quality Assessment Assessment Force Force Training Training Assessment Assessment DD. Costlow Costlow M Act) Act) M. Gammon Gammon ((Act) Command Command JJ. Cardany Cardany Keeping America’s Navy #1 in the World Readiness Readiness Assessment Assessment JJ. Watson Watson 6 Measurement Science WARFARE CENTERS CORONA Measurement Science Department Arman Hovakemian MS00 Measurement Requirements Assessment Ken Felix MS10 Air Programs MS11 A. Fritzsche FMS Programs MS12 I. Garcia Ground Programs MS13 L. Poling METCAL In-Service Assessment Bob Fritzsche MS20 METCAL Systems MS21 R. Greenhalge Vacant MS30 MS22 L. Rice Fleet METCAL In-Service MS23 R. Zaja SSP METCAL In-Service MS14 MS24 Advanced Measurement Systems John Griffith MS40 DC/Low Frequency MS31 S. Buckman Microwave/ Electro-Optics NAVSEA

METCAL In-Service Requirements Analysis M. Doerr Metrology Engineering Assessment MS32 Ed Trovato Physical/Mechanical MS33 R. Figuerres B. Heglar R&D Coordinator MS40A J. Walden CBNR Warfare Systems Metrology MS41 M. Bishop Advanced Automated Calibration MS42 R. Schumacher Sensors and Special Projects MS43 A. Luh Analytical Metrology MS25 J. Davis Keeping America’s Navy #1 in the World 7 Definitions WARFARE CENTERS CORONA Metrology: The Science of Measurement Calibration: VIM: Set of operations that establish, under specified conditions, the relationship between values of quantities indicated by a measuring instrument or measuring system, or values represented by a material measure or a reference material, and the corresponding values realized by standards. Navy: Calibration is the comparison of a measurement system or device of unverified accuracy, to a measurement system or device of known or greater accuracy, to detect and correct any variation from

required performance specifications. Keeping America’s Navy #1 in the World 8 Measurement Possibilities WARFARE CENTERS GO AL CORONA Not Precise/ Precise / Precise / Not Accurate Not Accurate Accurate Keeping America’s Navy #1 in the World 9 Why Calibrate? WARFARE CENTERS CORONA • Ensure the readiness of test equipment to perform accurate measurements and provide valid data • Limit the number of erroneous test decisions • Maintain overall measurement integrity and traceability Keeping America’s Navy #1 in the World 10 Why Calibrate? – Data Integrity WARFARE CENTERS CORONA • How do we know the data is good? – – – – Accurate? Precise? Stable? Can you trust the person collecting the data? Can you trust the measurement instrument? What proof do you have that the numbers reflect reality? • Can you trust the decision you are about to make using the collected data? • What is the cost if you are wrong? In Tolerance Out of Tolerance

Difference? Keeping America’s Navy #1 in the World 11 Uniform Measurement Accuracies WARFARE CENTERS CORONA Keeping America’s Navy #1 in the World 12 NSWC Corona – METCAL Role WARFARE CENTERS CORONA NSWC Corona serves as the Scientific and Technical Agent for the Navy Metrology and Calibration (METCAL) Program per OPNAVINST 3960.16A, Navy Test and Monitoring Systems (TAMS) and NAVAIRINST 13640.1B NAVSEAINST 4734.1B SSPINST 4355.8B MARCOR ORDER 4733.1B Keeping America’s Navy #1 in the World 13 Navy Calibration Program WARFARE CENTERS CORONA METCAL Executive Director TAMS Executive Board NAVSEA TMDE SISCAL NAVAIR SSP USMC METCAL NPSL Shore Calibration Laboratories Afloat Calibration Laboratories Keeping America’s Navy #1 in the World 14 METCAL Processes and Products WARFARE CENTERS CORONA Navy METCAL Program NIST METCAL Technical Agent Acquisition Support Acquisition • Cal Rqmts Analysis • Cal Logistics NPSL In-Service Test Equipment

Navy Cal Lab Engineering • Procedures • Intervals • Standards • Training • Database Mgmt • Traceability • Technical Support R&D Maintenance • Standards (State-of-the-art) Mission Ensure that TMDE used to make measurements on Navy Systems is accurate and traceable to National Standards to reduce or eliminate the safety and cost impacts of wrong test decisions. Keeping America’s Navy #1 in the World 15 METCAL Products and Services WARFARE CENTERS CORONA Metrology Engineering Products and Services Keeping America’s Navy #1 in the World 16 METCAL Program Scope WARFARE CENTERS CORONA Test Equipment Different Models: ~ 140K Models: 140K Different S/ ~ 1,850K S/N: 1,850K Approved Calibration Standards (Different Model #s) Annual Calibrations ~ 1.85M ~5,000 ~800K Approved Calibration Procedures Annual Engineering Inquiries/Actions Calibration Laboratories Shore 165 Ship 244 ~5,000 ~2,000 Keeping America’s Navy #1 in the World 409 17

Measurement Traceability WARFARE CENTERS CORONA Weapon System (F/A-18) National Institute of Standards & Technology Support Equipment Custodians Fleet “I” Level Calibration Labs Navy Primary Standards Lab Depot “D” Level Calibration Labs Keeping America’s Navy #1 in the World 18 METCAL Acquisition Life Cycle WARFARE CENTERS CORONA A B Concept & Technology Development • Metrology Guidance/ Consideration C System Development & Demonstration • Requirements Analyses • Verify Contractor Traceability • ILSMT • International/ National Measurement Technology Assessment • LSA • Technology Validation • Logistics Document Reviews • CMRS Development • CMRS Guidance • CSP Development • TEMP Inputs • Metrology R&D • Evaluation Criteria • SERD • CMRS Review, Approval • Calibration Support Plan Approval • Cal Standard Design, Develop, Procurement • Evaluation Criteria • Cal Procedure Development, Approval

(Contractor, DoD) IOC Production & Deployment • Contractor’s Calibration System Evaluation (Follow-on) • Calibration Std ILS • Deploy Calibration Support FOC Operational Support • TMDE Calibration • Feedback Responses • Technical/Engrg/ Training Assists • Interval Reliability Analysis • Measure Process Control • Joint Service Support • Modification/Rqmt Planning • Establish Cal Support Level • Tech, Mgt, Pol. Docs • Facility (Incl Environ) Requirements • Training Requirements • Manpower, Workload Requirements • Interval Analyses Keeping America’s Navy #1 in the World 19 Calibration Logistics Lifecycle WARFARE CENTERS CORONA Maintenance Maintenance Requirement Requirement Identified Identified TMDE TMDE Requirement Requirement Identified Identified • Testing • Preventive Maintenance • Inspection • Repair • Assist in identifying TMDE to support: – Testing – Preventive Maintenance – Inspection – Repair • Begin

developing calibration support requirements Calibration Calibration Requirements Requirements Analysis Analysis • Determine if current calibration support is in place • If not, determine proper calibration support for application • Identify R&D rqmts • Maintain calibration support • Identify replacement / replenishment for obsolete TMDE and Standards • Technical Support • Retire support Keeping America’s Navy #1 in the World Initial Initial Calibration Calibration Support Support Calibration Calibration Support Support Sustainment Sustainment 20 Navy Metrology R&D Program WARFARE CENTERS CORONA Provide Advanced Measurement Technology for Emerging Weapons Systems / Improve Effectiveness and Efficiency of Existing Measurement Systems NAVY / MARINE CORPS • Electrical SEA AIR LAND SPACE TECHNOLOGY AREAS • Physical/Mechanical Calibration Standards & Traceability • Electro-Optic • mm/Microwave • Analytic Metrology • Radiation •

Chemical/Biological Defense Keeping America’s Navy #1 in the World 21 Joint Service Metrology WARFARE CENTERS CORONA Joint Chiefs – Joint Logistics Commanders OFFICES OFFICESOF OF PRIMARY PRIMARYRESPONSIBILITY RESPONSIBILITY (OPRs) (OPRs) JOINT JOINTTECHNICAL TECHNICAL COORDINATING COORDINATING GROUP GROUPon onCALIBRATION CALIBRATIONAND ANDMEASUREMENT MEASUREMENT TECHNOLOGY TECHNOLOGY (JTCG-CMT) (JTCG-CMT) TMDE TMDEPROGRAM PROGRAM STANDARDS STANDARDS GROUP GROUP CALIBRATION CALIBRATION COORDINATION COORDINATION GROUP GROUP(CCG) (CCG) STANDARDS STANDARDSLAB LAB OPERATIONS OPERATIONS WORKING WORKINGGROUP GROUP ANALYTICAL ANALYTICAL METROLOGY SUBGROUP METROLOGYSUBGROUP ENGINEERING ENGINEERING WORKING WORKING GROUP GROUP ELECTRICAL ELECTRICAL//ELECTRONIC ELECTRONIC SUBGROUP SUBGROUP CHEMICAL CHEMICAL//BIOLOGICAL BIOLOGICAL SUBGROUP SUBGROUP INTERSERVICE INTERSERVICE SUPPORT SUPPORT GROUP GROUP(IS) (IS) CALIBRATION CALIBRATION PROCEDURES PROCEDURES GROUP GROUP CHEM/BIO

CHEM/BIO SUPPORT SUPPORT WORKING WORKINGGROUP GROUP AUTOMATED AUTOMATED PROCESSES PROCESSES WORKING WORKINGGROUP GROUP RF RF//MICRO MICRO//MM-WAVE MM-WAVE SUBGROUP SUBGROUP PHYSICAL PHYSICAL//MECHANICAL MECHANICAL SUBGROUP SUBGROUP Keeping America’s Navy #1 in the World ELECTRO-OPTIC ELECTRO-OPTIC SUBGROUP SUBGROUP RADIOLOGICAL RADIOLOGICAL SUBGROUP SUBGROUP 17 Jan22 2008 Industry/Professional Involvement WARFARE CENTERS CORONA Represents the Navy in the Measurement Science Field National Conference of NATIONAL DEFENSE INDUSTRIAL ASSOCIATION International National Standards Laboratories International DoD Navy METCAL JTCG-CMT Joint Technical Coordinating Group Calibration and Measurement Technology ASME Keeping America’s Navy #1 in the World 23 False Accept/False Reject WARFARE CENTERS CORONA Keeping America’s Navy #1 in the World 24 Calibration Testing WARFARE CENTERS CORONA Calibration Tests Test Instrument Test Steps ICP • A Test Instrument is

considered In Tolerance if it passes all the steps of an Instrument Calibration Procedure (ICP) Keeping America’s Navy #1 in the World 25 True In-Tolerance WARFARE CENTERS CORONA A Unit Under Test (UUT) is truly in tolerance if: Lower Tolerance Limit < UUT Bias < Upper Tolerance Limit UUT Bias (True) Lower Tolerance Limit (-L) 0 Upper Tolerance Limit (L) The UUT Bias is unknown Keeping America’s Navy #1 in the World 26 True Out Of Tolerance WARFARE CENTERS CORONA A Unit Under Test (UUT) is truly out of tolerance if: UUT Bias < Lower Tolerance Limit UUT Bias > Upper Tolerance Limit or UUT Bias (True) Lower Tolerance Limit (-L) 0 Upper Tolerance Limit (L) The UUT Bias is unknown Keeping America’s Navy #1 in the World 27 Observed In Tolerance WARFARE CENTERS CORONA A Unit Under Test (UUT) is observed in tolerance if: Lower Tolerance Limit < Deviation < Upper Tolerance Limit Deviation (Observed) Lower Tolerance Limit (-L) 0 Upper

Tolerance Limit (L) The Deviation is the observed difference between the UUT and the CAL Keeping America’s Navy #1 in the World 28 False Accept WARFARE CENTERS CORONA Deviation (Observed) Calibration System Error UUT Bias (True) Acceptance -L 0 L False Accept (FA): • The Deviation is observed in tolerance [ -L < Deviation < L ] • The UUT Bias is out of tolerance [ Bias > L or Bias < -L ] • The decision to accept the UUT is incorrect Keeping America’s Navy #1 in the World 29 Measurement Reliability WARFARE CENTERS CORONA Measurement Reliability •• Measurement MeasurementReliability Reliabilityisisthe theprobability probabilityof ofbeing beinginintolerance tolerance(the (the probability probabilityall allthe thecalibration calibrationprocedure proceduresteps stepspass). pass). •• Measurement MeasurementReliability Reliabilitydecreases decreaseswith withtime timeafter aftercalibration calibration(which (whichisis why whywe wecalibrate

calibrateperiodically). periodically). 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 0 6 12 18 24 30 36 Time Since Calibration (Months) Keeping America’s Navy #1 in the World 30 WARFARE CENTERS Calibration Intervals And Test Decisions CORONA • Good test decisions: – Detect bad items and repair or replace them – Verify good items and return them to the Fleet • Good test decisions require accurate test equipment • Accuracy in test equipment is maintained by periodic calibration – TMDE accuracy, or measurement uncertainty degrades with time – TMDE “Out of Tolerance” (OOT) not obvious to user - insidious error – TMDE OOT can only be determined by calibration – Periodic calibration keeps measurement uncertainty at acceptable levels • Accurate calibration intervals keep false accept risk (acceptance of bad test equipment) low Keeping America’s Navy #1 in the World 31 Calibration Intervals WARFARE CENTERS CORONA •• To Toobtain

obtainaaCalibration CalibrationInterval, Interval,you youneed: need: –– AAMeasurement MeasurementReliability ReliabilityCurve Curve(Estimated (Estimatedusing usingCalibration CalibrationResults) Results) –– AAMeasurement MeasurementReliability ReliabilityTarget Target 100 % Measurement Reliability vs. Calibration Interval Measurement Reliability 90 % 80 % 70 % 60 % 50 % 40 % 30 % 20 % 10 % 0% 0 6 12 18 24 30 36 42 48 Calibration Interval (Months) Keeping America’s Navy #1 in the World 32 ISO 9001 – Calibration Requirements WARFARE CENTERS CORONA 7.6 Control of monitoring and measuring devices • Requires organization to assess/understand measurement needs • Requires that measuring equipment – be accurate to ensure product conformity to specification – be calibrated or verified at specified intervals, or prior to use, against measurement standards traceable to international or national measurement standards Keeping America’s Navy #1 in the World

33 Z540-1 Withdrawal WARFARE CENTERS CORONA Test & Measurement Equipment/Cal Lab Accreditation 1994 2005 2006 2007 ANSI/NCSL Z540.1 ANS/ISO/IEC 17025:2005 ANSI/NCSL Z540.1 Approved Approved ANSI/NCSL Z540.32006 Approved Withdrawn Z540.1 is replaced by Z5403 and 17025 Keeping America’s Navy #1 in the World 34 9001, 17025, & Z540.3 Relationship WARFARE CENTERS CORONA ENTERPRISE Quality System ISO 9001 Independent Calibration Lab Products / Services ISO/IEC 17025 Measurement and Test Equipment ANSI/NCSL Z540.3 Internal Calibration Laboratory ISO/IEC 17025 (or Z540.3) Keeping America’s Navy #1 in the World 35 Distribution of Z540.1 Requirements WARFARE CENTERS CORONA • 17025 Z540.1 Part I – Cal Lab – Cal lab requirements – Cal lab competency demonstration (ISO G25) • – – General requirements for the competence of testing and calibration laboratories Calibration service component competency demonstration Z540.3 Part II –

Cal System – M&TE cal requirements – Cal system requirements Keeping America’s Navy #1 in the World – – Cal service component technical requirements Optional use of ANS/ISO 17025 with SC 5.3 for Cal service component competency demonstration – – M&TE cal requirements Cal system requirements 36 Roles of New Standards WARFARE CENTERS CORONA • ANSI/ISO/IEC 17025: – Competence of test and calibration laboratories – Demonstration requirements: • Able to operate a management system • Are technically competent • Able to generate technically valid results Laboratory Competence Requirements • ANSI/NCSL Z540.3 – Technical requirements for calibration of M&TE – System for control of the accuracy of M&TE: • Calibration system performance requirements • Calibration laboratory performance requirements • Link to use of 17025 accredited cal labs METCAL Program Requirements – ISO 17025 accredited cal lab “smart customer” Keeping

America’s Navy #1 in the World 37 WARFARE CENTERS Calibration Program Functions CORONA • Calibration Requirements – • Calibration Procedure – • • • Development, Review, Approval TMDE/Calibration Standards – Inventory Control – Reliability, Interval Assessment – Specification, Evaluation, and Management Calibration Laboratory – Facility/Technical/Management Requirements – Assessment (Accreditation) Calibration Data – • Determination, Assessment Collection, Management, Assessment Training – Development / Assessment • Technical Support • Professional Society Involvement Keeping America’s Navy #1 in the World 38 So, Why Calibrate? WARFARE CENTERS CORONA Test, Measurement, and Diagnostic Equipment Must Be Supportable Compatible • Repair parts • Requirements – Form-fit-function – Accuracy – Traceability – Repeatability • Repair procedures • Allowances • Training • Inventories • ICPs

• Standards • Documentation Affordable • Control proliferation • Prevent unauthorized substitution • Ensure adequate support To operate and maintain safe and reliable systems Keeping America’s Navy #1 in the World 39