Calibration Fundamentals
An Introduction to Metrology
Presented by Joël Rameaux Service & Support Unit Agilent Technologies Technologies Europe
Calibration Fundamentals Today’s topics Contents
An Introduction to Metrology
There’s much more beneath the surface !
• Metrology definition • Measurements confidence • What is calibration? • Cal comparison • Iceberg ahead ! • How to calibrate • Calibration of Agilent equipment • Errors and uncertainty • Summary
The Importance of Measurement “Metrology” is the science of measurement British scientist Lord Kelvin (William Thomson 1824-1907) is quoted from his lecture to the Institution of Civil Engineers, 3 May 1883... "I often say that when you can measure what you are speaking about and express it in numbers you know something about it; but when you cannot express it in numbers your knowledge is a meagre and unsatisfactory kind"
“You cannot manage what you cannot measure.” “You HP co-founder Bill Hewlett
Metrology : Wiki Definition Metrology is the science of measurement. Metrology includes all theoretical and practical aspects of measurement. Applied metrology concerns the application of measurement science to manufacturing and other processes and their use in society, ensuring the suitability of measurement instruments, their calibration and quality control of measurements. Although the emphasis is on the measurements themselves, traceability of the calibration of the measurement devices is necessary to ensure confidence in the measurements. Metrologist : Perform metrology
The Need for Good Measurement We all make and use measurements every day • • • • • •
How much fuel will I get? What weight of fruit does the pie need? Will this nut fit the bolt? Does your mobile ‘phone reliably connect to any network? Which drug is most effective? What is the time?
We have expectations about the correctness of these measurements…
What are Specifications? Define a product’s expected performance An example using a familiar product… Manufacturer’s view • Frequency: 900 MHz • Channel spacing: 200 kHz • Max power: 2 W • Rate/Modulation: 270 kbps/GMSK • Sensitivity: -97 dBm • etc. − Reason: Conforms with legal criteria (safety, interference); compatible with operators’ networks.
Consumer’s view • Reception quality − Reason: Can hear clearly when driving (*) a car, while in an elevator or walking through an underpass.
(*) Via
Bluetooth® hands-free, of course!
Why Measure? We take measurements to get information about the manufactured or used product so that we can make good decisions. Measure the product to see if it conforms with specs or requirements
Measurement decisions are made during: • Research • Product development • Production • Final testing • Operation
Does it comply?
Yes
We have a good product
No
But how do we know our measurement tools are giving good results?
Measurement Confidence Measurements are made using a wide variety of instruments, ranging from very simple to extremely sophisticated. We must have confidence that all those measuring instruments are giving valid, useful data. Because measured values are the basis of decisions. That confidence is provided by an effective program of calibration
Calibration Confidence Reference Equipment used during calibration should be calibrated and should match or exceed accuracy expectation. Procedures used during calibration should be clear, validated and repeatable. Measurement results should be given at minimum as a Pass/Fail result and preferably with measurement data and uncertainties associated Existing Traceability toward reference standards People should know what they are doing Preference should be given to automated calibration as it increases confidence in overall calibration procedure
So, what is calibration ? (1) A graphic illustration – search Google Images for ‘meal’ and see the variety… Are they all identical ?
So, what is calibration ? (2) Definition “Operation that, under specified conditions, in a first step establishes a relation between the quantity values with measurement uncertainties provided by measurement standards and corresponding indications with associated measurement uncertainties and, in a second step, uses this information to establish a relation for obtaining a measurement result from an indication.” From the ISO/IEC Guide 99 : 2007 -- International Vocabulary of Metrology
• Or more simply... “The process of comparing a device with unknown performance to measurement standards traceable to the SI units and with determined uncertainties to enable obtaining thereafter useful measurement results from the device.” There is no detail on what should be done during calibration.
Calibration Provides Confidence Confidence that a device’s past performance was valid and that its future performance will continue to be good within recommended calibration interval. • Unacceptable past performance may necessitate customer alert and product recall. • Due to drift, measuring equipment needs to be regularly calibrated to contain error to acceptable limits. • Knowledge of the value of a calibrated parameter becomes less certain as time elapses since calibration. • Some products require annual optimisation (adjustment) for high confidence of compliance in future.
Calibration & adjustment
e u l a v r e t e Performance m drift a r a P
Calibration & adjustment
Spec limit
Time
Verification, Calibration, Adjustment, Repair Specification Limit of normal equipment operation, defined by original manufacturer
Verification Test versus specifications. Result is ‘Pass’ or ‘Fail’. Verification should include a data report with failed points, if any, and possibly measurement uncertainty.
Adjustment Mechanical or electrical tuning of equipment to bring it back within specifications. Adjustment can be done in case of ‘Failed’ verification or ‘Passed’ verification (adjust always equipment). Adjustment should include a as-received data report and a postadjusted data report.
Calibration Verification + Adjustment combined (if needed)
Repair If equipment fail verification and cannot be adjusted, then unit is faulty and should be repaired. Adjustment is not a repair !
Calibration Performance Real data comparison MXG RF Digital Signal Generator Max
e u l a V d e r u s a e M d e z i l a m r o N
820 POINTS TESTED Min Max
257 POINTS TESTED
t r u e p w t u o O P N5182A
l a C t n e l i g A
d r o e M w o l P a t i g i D
r g e o w l o a n P A
r l a e t w i g o i P D
c s i n u o o i r m r u a p H S
c b i u n S o m r a H
c n i o n N o m r a H
l n a o t i i g t i a D l u d o M
z y H c M n e 0 u 1 q e r F
A r e M w D o C - P W
Min
l a C t n e l i g A n o N
Calibration Performance Real data comparison MXG RF Digital Signal Generator Max
l a C t n e l i g A
e u l a V d e r u s a e M d e z i l a m r o N
Min Max
Not measured (long or difficult tests)
t r u e p w t u o O P N5182A
d r o e M w o l P a t i g i D
r g e o w l o a n P A
r l a e t w i g o i P D
c s i n u o o i r m r u a p H S
c b i u n S o m r a H
c n i o n N o m r a H
l n a o t i i g t i a D l u d o M
z y H c M n e 0 u 1 q e r F
A r e M w D o C - P W
Min
l a C t n e l i g A n o N
Calibration -- The Iceberg Analogy The calibration certificate and measurement data report • What the customer sees
Calibration -- The Iceberg Analogy More to it than meets the eye The calibration certificate and measurement data report • What the customer sees • What the customer doesn’t see – – – – – – – – – –
Calibration procedure, software Measurement adequacy Traceability Equipment recall system Records Labelling for identity & status Interval management Environment Supplier evaluation Technical competence
How to deliver Calibration? (1) Calibration is verifying instrument specifications by measuring actual performance using external lab standards [instruments] that in turn have better performance, and are traceable to International Standard (NIST, NPL, PTB and NMIJ). You need : - Systems/ETEs used as reference - To follow OEM recommendations - Appropriate Procedures - Trained People - Cal Laboratory - To archive results Agilent measures the actual performance of every warranted specification, for every installed option, every time using same procedure and including adjustments
How to deliver Calibration ? (2) • Use OEM Calibration Labs • Built your own cal lab • Bring Cal Lab on your site(s)
• Mix of all above…
How to deliver Calibration ? (3) TME Software Calibration Platform • Check ETEs (reference) needed • Check ETEs cal due date • Compute uncertainties as a function of ETEs used
• Define tests to be run as a function of configuration (option, frequency, etc…), test operator setup • Give Pass/Fail summary and data reports with points out of specs • Print clean data report • Archive data and traceability • Validated by Agilent (operation, data points, uncertainties)
Calibration of Agilent Equipment? Automatic Calibration with commercialy available software 30 %
• Reliable & Repeteable • Automatic configuration check and uncertainty measurement • Most new equipment
Automatic Calibration with proprietary software 40 %
• Reliable & Repeteable • Automatic configuration check and uncertainty measurement • Older equipment
Manual Calibration 20 %
10 %
• • •
Rely on strickly following recommended test procedure Highly operator dependant Very old equipment, no computer interface
No-Calibration required • Accessories, cables, adapters
What Needs Calibrating? • Any equipment used in incoming, final inspection, in-process or quality control (SPC), product calibration/testing. • Equipment used to perform lengthy or costly machine set-up.
• Items whose indications impact upon controlled conditions (e.g. environment). • Any situation where the failure costs are too high (safety, one off testing, etc…).
• Equipment used in R&D where design decisions are made. • Used during failure or returned material analysis, where critical determinations could be affected (including repair troubleshooting).
• Whenever someone acts upon an instrument's reading. • Equipment used for calibration (traceability) of any of the equipment used in the above situations. • In general, all equipment who are used as reference for other items
Error & Uncertainty Error is the result of a measured value minus the expected or nominal value of the quantity. The purpose of calibration is to determine the magnitude of (and possibly correct for) errors.
Measured value
Expected value
Negative error
Measured value
Positive error
Outlier Average error
Error is by how much we miss the target (i.e. the red bulls-eye). The dispersion (spread of shots) represents repeatability – a component of uncertainty.
Shots at a target – another illustration of error
Error, Precision & Accuracy Note: In reality we need lots of data to determine the distribution (5 shots is not enough).
Accurate = True Precise = Repeatable Bad approach
Bad reference equipment e u l a v e u r T
Inaccurate and imprecise
Bad methodoloy
Quite accurate but imprecise
e u l a v e u r T
Precise but inaccurate e u l a v e u r T
State of the art
e u l a v e u r T
Accurate and precise
Concepts of Uncertainty Example : How long is a piece of string?
Contributors Tape measure • Manufacturing tolerance
Method & Personnel
• Bent or stretched
• Lining-up string & tape measure
• Temperature / humidity
• Horizontal or vertical
• Graduation resolution
• Repeatability
String • Lying straight / flat • Under / Over-stretched • Temperature / humidity • Frayed ends
• Eye report on graduation
Concepts of Uncertainty Sources of error & uncertainty Measuring equipment • Specifications inc. “imported” uncertainty (traceability) • Readability
Properties of the thing being measured • Stability • Sensitivity to input change • Readability
Measurement method • Use of correctional data • Configuration • Data processing
Operator skill • Reaction time • Manual dexterity • Visual ability (meter parallax)
Environment • Temperature • Humidity • Altitude / Barometric pressure • EMI
Impedance effects • Mismatch • Loading (e.g. local / remote sensing)
Cables and connections • Torque • Materials • Cleanliness • Mechanical stress
Measurement non-repeatability • Due to properties of the test equipment and unit-under-test and also the variable influences of test environment and operator.
Mistakes ! • Can’t be accounted for but can be statistically identified (outliers).
Uncertainty Example : Power sensor cal factor Measurement equation
Mi1 Mt (1 ( e t ))2 Kb ( Ks Ks) Er 2 Ms Mi 2 ( 1 ( e s ))
Uncertainty calculation for a device
• One spreadsheet/calculation to be created for each measurement setup and adjusted (uncertainty) for each measured point -> Give uncertainty per measured point -> Imply Pass/Fail criteria • Same analysis to be done for each measurement point • Same analysis to be done for each setup (alternate replacement) Remember the MXG : 820 measurement points for 10 performance setup ! Not including alternate ETEs analysis.
-> This analysis and validation is the pure role of the metrologist
Basic Model for Product Specification Where does uncertainty fit in?
Calibration & adjustment
e u l a v r e t Guardband e Performance m drift a r a P
Production margin
Spec limit
Production process distribution (manufacturing reproducibility)
Time
Calibration uncertainty
Mean
Delta environment and drift
Production line test limit
Specification
Calibration Intervals • May be expressed as a calendar period, usually months or years between calibrations. • Or might be an interval of usage time such as 1000 hours of operation. • Or possibly the number of usage operations, especially mechanical items subject to degradation through wear.
• Although a manufacturer may recommend an interval (the spec implicitly has time dependency owing to expected drift), the equipment user is responsible for assigning the appropriate interval. Most of users follow manufacturer’s recommended CI
• Aim is to balance the cost of calibration (how often and the extent of the testing) with the risk of bad measurement (cost of product recall, safety, etc.) • Better do a complete calibration every 3 years than a bad calibration every year
Calibration Summary What you need to remember • Only manufacturer can define specs and what need to be tested. • All calibration are not equivalent. Compare what is really done to have a complete picture : function, number of points and accuracy. • Calibration certify past performance was correct and future performance is expected to be good • Some calibrations can be done by yourselves, some cannot.
• Agilent is ready to help, in performing on-Site calibration for reduced down time and minimize risk in shipment • You can tune calibration interval. It is your decision, but you should be able to justify in front of auditors.
• Metrology is really a specific and full time job reserved to specialists • Any question :
[email protected] or contact Boris at IBIS
Calibration Summary
Questions ?
Calibration Summary
Thanks for your attention !
Metrological [measurement] Traceability Definition “Property of a measurement result whereby the result can be related to a reference through a documented unbroken chain of calibrations, each contributing to the measurement uncertainty.” From the ISO/IEC Guide 99 : 2007 -- International Vocabulary of Metrology
• Or more simply as... − A series of calibrations of known accuracy that link measurements to recognised standards through an unbroken chain of comparisons. • The need for records is to link a product or service event to a process, status or time.
− If the process is measurement, detail about the equipment, procedure, test conditions and personnel are all relevant to demonstrating traceability.
Traceability International System of Units (SI units) Every measurement is linked to one or more of just 7 base units ! Definitions (at NIST site) The kilogramme is the only unit defined by a “prototype” (a specific physical item held at BIPM in Paris, France).
http://physics.nist.gov/cuu/Units/index.html
Measurement Traceability A representation of the “national measurement system” Unc = 0.008%
International comparisons via BIPM
International comparisons via regional accreditation organisation Unc = 3% TUR 10:1
Spec = 15% TUR 5:1
PRODUCT SPEC
Unc = 0.04% Unc = 0.1% TUR 2.5:1
Unc = 0.3% TUR 3:1
MANUFACTURING
NATIONAL METROLOGY INSTITUTE
Another stds lab
STANDARDS LABORATORY
CAL FACILITY
Another production line
NATIONAL REALISATION OF SI DEFINITION
Another cal lab
And another
Another factory
And another
Uncertainty is the potential inaccuracy. The levels do not represent consistent values of uncertainty. Test uncertainty ratio (TUR) is the ratio of the unit-under-test's specification to the measurement uncertainty.
Traceability – Why Bother? Summary • Common measurement references are critical to the world-wide exchange and acceptance of products, services and technology. • The aim is to transfer these references in a controlled manner to the billions of individual measurements made daily. • Traceable measurements ensure uniformity of manufactured goods and industrial processes.
• It supports equity in trade, compliance to regulatory laws and standards.
• If the calibration paper trail was followed from the unit under test up through the working, reference and primary standards, it would eventually end at the record of an experiment made to establish the quantity in terms of one or more of the SI base units. • It is essential to science and the development of technology.
Recommended Resources (Free) Find out more
NPL Good Practice On-line Modules on: • Measurement http://www.npl.co.uk/publications/good-practice-online-modules/measurement/
• Uncertainty http://www.npl.co.uk/publications/good-practice-online-modules/measurement-uncertainty/
NPL -- A Beginner's Guide to Uncertainty of Measurement http://resource.npl.co.uk/cgi-bin/download.pl?area=npl_publications&path_name=/npl_web/pdf/mgpg11.pdf
UKAS M3003 -- The Expression of Uncertainty and Confidence in Measurement http://www.ukas.com/library/Technical-Information/Pubs-Technical-Articles/Pubs-List/M3003.pdf
