Calibration of Linear and Angular Measuring Instruments (Metrology)

Every measuring instrument must be provable, i.e. it must be caused to prove its ability reliably. The procedure for this is calibration. The variation in any observation on a product depends upon the variation in the product due to process of manufacture and variation due to measuring process, i.e. it can be expressed as : oobservation = oprocess + ameasurement.
In order to keep a0&seroaton minimum so that product as a whole is reliable, omeasurement should be kept minimum which in other words means that measurement process or the instruments used should be precise with minimum of variation in the measured values. In order to maintain the precision accuracy of measuring device its periodical calibration is essential as from the moment an instrument is put into use it begins to deteriorate in accuracy and its precision. To a degree, this takes place even if the instrument is not being used. Regarding calibration it is said that in a plant where accuracy is not properly organised, 30% to 50% of the measuring equipment used do not give true results.
In order to maintain accuracy of measuring instruments, following procedure should be followed:
(i) Each instrument should be numbered. It serves the easy location of the instrument. (ii) A card record should be established for each instrument.

Table 10.1 Calibration Card

Defects Found (if any) Errors / Defects After repairs (if any) Details of
repairs carried out
Remarks Calibrated by Place of use Next calibration due on Initial Lab. IIC

Instrument………………….. Type and Class………………….. Inventroy No………………………

Table 10.2

Annual Calibration Programme For General Measurement Instruments
Empty—Indicates that calibration is due.
Indicates that calibration has been completed for all the instruments due for calibration in the month.

Months SHOPS
Calipers, Micrometers, Protractors, dial gauges etc. Sine bars Limit gauges Slip gauges Levels, surface plates etc. Others
January O 0
February O 0 0 0 0
March o 0
April o o O
May o 0 0
June o 0 0
July o 0 0
August 0 0 0
September 0 0 0
October 0 0 0
November 0 0 0
December 0 0

(iii) Checking interval should be established.
(iv) Some system should be adopted for providing adherence to the checking schedule.
(v) The record of the findings of the check should be maintained.
(vi) Record of checks should be further studied and analysed so as to improve upon the system.
It shall be preferable to have individual history card for each instrument. History card can be of type given in Table 10.1. These types of history cards can be prescribed using Kardex cabinets.
As regards checking interval, this mainly depends upon the frequency of use of the instruments, and the precision requirement of the measurements. For example in machine shop and tool room the measurements made are of more precise nature as compared to those made in casting and fabrication shops, so instruments used in these shops require more frequent calibration. It is always better to prepare annual calibration programme for instruments used in various shops, so as per the programme the shops send the instruments for calibration. The annual programme can be of the type as suggested in Table 10.2 which is for general measuring instruments such as vernier calipers, micrometers, protectors, limit gauges, slip gauges, dial gauges etc. For optical measuring instruments like universal microscopes, tool maker’s microscopes etc. annual programme can be made for periodical calibration as well as for general preventive maintenance which besides general cleaning and lubrication will also consist of carrying visual checks like relative movements of moving parts, presence of corrosion, scratch marks, visibility and correct working of optical system etc. Such visual checks should be carried out more frequently. In Table 10.3 is shown such chart wherein the annual schedule for calibration and preventive maintenance of optical instruments is drawn. Table 10.4 also shows the history card to be made for limit gauges. For every gauge or gauge

Table 10.3

. Annual Schedule for Preventive Maintenance and Calibration of Optical Metrological Instruments
A Indicates about preventive maintenance being due. □ Indicates about calibration being due.

Instruments Months
Jan. Feb. Mar. Apr. May June July Aug. Sept. Oct. Nov. Dec.
Universal Microscope
SI. No. …
A O A A A □ A A A □ A A A □ A A
Tool Makers Microscope
SI. No….
A □ A A A A □ A A A A □ A A A
Interferometer SI. No. … A A A A A A
Shadow graph SI. No…. A A □ A A A A □ A A A A □ A A

Table 10.4


Gauge Major Dia
and Tolerance
Effective Dia and Tolerance Tolerance Minor Dia and Tolerance
For New Gauge Transfer to Insp. Wear Limit Pitch HalfProfile Angle
Go W
ire size
Not Go ictor X


About Stability
Place of Use Next Checking Due Checked By
Go No Go
Major Dia Effective Dia HalfProfile Angle Pitch Minor Dia Major Dia Effective Dia HalfProfile Angle Pitch Minor Dia
Left Right Left Right Go Not Go

Type……………………… Size…………………,….. Card……………………… Instt. No……………………..
set an individual history card should be maintained. For gauges of any particular size, sometimes it becomes difficult for shop or inspection personnel to know whether the gauges are available or not.
For this purpose the charts could be maintained which indicate the maximum/minimum inventory stock quantity level for these gauges so that at any time when the quantity available in the stores goes down the minimum value, action should be taken for the procurement.
Here are described the methods of calibration of some of the important metrological instruments.

10.2.1. General Metrological Instruments,

(a) Vernier Calipers and other Vernier Instruments.
The following table gives the allowable deviation in the parameters.

Parameters Permissible Error
Least count 0.02 mm Least count 0.05 mm
Zero error 0.02 0.05
Flatness of measuringjaws 0.003 0.004
Parallelness of measuring jaws 0.010 0.015
Error in readings 0.02 0.05
Spherical portion size of inside ±0.02 ±0.03
measuring jaws

The zero error is checked by bringing in contact the jaws, and the shift of zero of main scale is observed with respect to zero of vernier scale. The flatness of the measuringjaws is checked using a straight edge having sharp edge of class 1 accuracy. The straight edge is put over the surface and the light gap observed between the straight edge and the surface and compared with standard light gap formed between another straight edge and an optical flat. The parallelness is checked by inserting a slip gauge of any value between the jaws at various positions and determining the out of parallelness using slip gauges.
Error in readings along the entire range is also found out using slip gauges. In case of vernier calipers having spherical inside measuring jaws, the width of spherical portion is checked using a passmeter or dial type micrometric comparator.
(b) Dial Gauges. The dial indicators of least count 0.01 mm can be conveniently calibrated using passmeter or micrometer dial comparator of least count (L.C.) 0.002 mm. Here dial indicator is inserted in place of fixed right hand side flat jaw and the dial tip rests on the movable jaw. The dial indicators can also be calibrated using slip gauges and by fixing the dial gauge in a comparator stand.
The following table gives the permissible errors in the dial indicators :

Parameter Permissible Error
L.C. 0.01 mm L.C. 0.001 mm L.C. 0.002 mm
Maximum error along entire range and in any one turn 0.02 mm along entire range 0.003 mm 0.004 mm
0.006 if any one turn.
Variation in readings along entire range. 0.02 mm. 0.003 mm 0.004 mm

(c) Micrometers. In case of micrometers the following are the main points to be checked:
(i) General appearance and relative movement of moving parts.
(ii) Checking initial zero setting for micrometers of size 25—50 mm or more. (Hi) Flatness of measuring surfaces.
(iv) Parallelness of measuring surfaces.
(v) Error in readings.
In general appearance, the micrometer is thoroughly checked for presence of scratches, dents etc. on measuring jaws, as well as, for corrosion marks, scratches, dents etc. on the surfaces of measuring drums, for proper working of ratchet system. The relative movement of moving parts is also checked which should be smooth. The working of lock system is also checked. The zero error of micrometer is checked and if it is found wrong it is adjusted easily for micrometers of size 25—50 mm and more. The size of the setting piece is checked on interferometer or any other comparator set to read upto 0.0001 mm. The permissible error allowed in its size 0.001 mm for micrometers upto size of 100 mm. The flatness error is checked by keeping optical flat on each jaw. The maximum permissible error is 0.0009 mm. The parallelness error is also checked using four optical flats of different width so that one complete turn of micrometer drum is made. The optical flat is set in such a way that total fringes on both sides are minimum. The permissible error in parallelity is 0.002 mm for micrometers upto 100 mm size and 0.004 mm for micrometers above 100 mm and upto 200 mm size.
The error in readings is checked using slip gauges so as to cover the entire range. The maximum permissible error is 0.004 mm for class I micrometers and 0.008 mm for class II micrometers.

10.2.2. Optical Measuring Instruments

Calibration of Universal Microscope. The calibration of the microscope is done as follows :
(i) General visual checking. Under this, levelling, movement of carriages, micrometric drums and vertical column, clear visuality of optical systems focusing etc. is checked. The measuring surfaces should be free from corrosion marks, scratches dents, etc.
(ii) Straight movement of longitudinal and transverse tables. The permissible error in these movements is 0.002 mm in horizontal plane and 0.005 mm in vertical plane.
This is checked using a rectangular straight edge and a dial indicator of least count 0.001 mm alongwith attachment for fixing dial indicator. The straight edge is put over the work table of microscope and dial tip is set at its top working face. The table is moved longitudinally and the dial tip deviation is the required error in vertical plane. For determining error in horizontal plane the dial tip is set horizontally touching the vertical side of straight edge and again table is moved and deviation is noted. Similarly, the straight edge is put in transverse direction over the table and deviations of movements in horizontal and vertical planes atre noted.
(iii) Relative perpendicular motion of the longitudinal and transverse carriage. The permissible error allowed is 0.005 mm in 1100 mm length. This is done using a precision square of class 0 or class I accuracy having maximum error of ± 10″ in squareness. The movement of one of the carriages is set first coinciding with the shadow of one of the sides of square. Then the other side of square is brought in coincidence with transverse line of ocular microscope and the transverse table is moved. The deviation of the shadow and the square from the transverse line as observed through microscope is noted when the transverse carriage is moved.
(iv) Perpendicular motion of ocular microscope column with respect to the work stage. The permissible error in this motion is 0.060 mm over 100 mm movements. This is checked using a precision try square of class I accuracy and dial indicator 90.01 mm) with fixing attachment. The try square is put with its base over the work stage and the other side is held vertical with dial tip touching on it. The column is moved up and down and the deviation of dial readings is noted.
(v) Checking reading of column rotation used for setting of helix angles. The permissible error allowed in the entire range of column rotation is ± 5′. This is checked by using a clinometer of 1′ least count. The clinometer is put over column top and inclination is given which is further checked with clinometer.
(vi) Change of ocular lines ‘position due to column rotation. This is checked using a special control shaft where a right angled blade is provided at right angle to shaft diameter line. The ocular line is set coinciding with the blade and the column is titled and the shift in position of ocular line is noted which is the required error. The maximum error permissible is 0.010 mm in entire ± 12.5° rotation of column from centre position.
(vii) Checking of main ocular microscope magnification. This is checked using a special glass scale where graduation lines in millimetres and tenth of millimetres are marked very accurately. The permissible error in this case is 0.0005 mm.
(viii) Change of job image due to minor shift of focusing ring. For this a ground sheet is taken with a X mark engraved and it is focused by the microscope stage using reflecting light. The position of mark is observed through the ocular and then an optical flat of nearly 20—25 mm thickness is put over the mark and refocusing is done. In this position, the shift of mark from the original position is noted which should not exceed 0.005 mm.
(ix) Parallelness of the longitudinal axis of millimetre scale to the movement of carriages. For this test, the millimetre line is brought in contact with any of micron line and its position is observed. Now the carriage (longitudinal or transverse) is moved to its other extreme and the position of test millimetre line is noted. Its shift from the original position of test millimetre line is the required error which should not be more than half division of micron scale as seen visually.
(x) Change of reading on micron scale by repeated 3—4 settings of the millimetre line in the centre of spiral double lines. The error should not exceed 0.0005 mm.
(xi) Checking of centres and centre stocks.
(a) Wear of centres. The centres are set and checked for straightness which is observed while rotating the centres. The maximum error permissible is 0.01 mm.
(b) Radial eccentricity of the centre sleeves (Fitted in stocks). This is done using a dial indicator of least count 0.001 mm and measured by touching the dial tip on inside surface of sleeves where centres are fitted and the sleeve is rotated in the stock. The maximum variation in the reading of the dial indicator is the required error which should not exceed 0.002 mm.
(c) Radial eccentricity of each centre when the sleeve is rotated. In this case the dial tip is put over the centre surface with the centre fitted in the sleeve and the sleeve is rotated. The maximum deviation of dial readings is the required error which should not exceed 0.005 mm.
(d) Coaxiality of both centres in horizontal and vertical plane. For checking the coaxiality in horizontal plane both the centres are brought in field of view of main microscope and viewed for coaxiality. For determining coaxiality in vertical plane, a precision ground mandrel of 200 mm length is held between the centres. The dial indicator (L.C. 0.001 mm) is clamped with a suitable attachment to the microscope tube with its tip resting over the mandrel at one end. By moving the transverse carriage the maximum dial reading over one end is noted and the
longitudinal carriage is moved by approximately 200 mm and again the maximum dial reading is noted at this end over the mandrel. The difference in these two readings is the required error in vertical plane which should not exceed 0.001 mm.
(xii) Error in readings. The error in readings is checked in two ways :
(a) Error in readings while measuring on flat glass measuring table.
(b) Error in readings while measuring in between centres using measuring knives.
(a) Error in measurement on flat tables. The permissible error in measurement is given as follows : Error = ± (1 + 1/10) microns ; where L is the length being measured in millimetres.
This error is determined with the help of a standard scale which is flat transparent glass scale on which graduations in millimetres are made from 0 to 200 mm with an accuracy of ± 0.0002 mm. Along the longitudinal movement of the measuring carriage, measurements are made in the range 0—200 mm in the intervals of every 5 mm. Along the transverse movement of measuring carriage, measurements are made in the range 0—50 mm in the interval of every 1 or 2 mm. Every time the difference between actual interval distance and the measured value with the microscope is made which is the error. The maximum error in the entire measurement range in both longitudinal and transverse movement should not exceed the value as calculated from the abore mentioned formula. From these data of errors the standard deviation of the error variation can also be found which should remain within 1/3 to 1/5 of the least count of the instrument, and should in no case exceed the value of least count of the instrument even in case of an old used instrument.
(b) Error in measurement between centre using measuring knives. The error in measurement should not exceed the values given in the following table :

Check Parameter to be Checked Permissible error
1. D iameter of precision plug whose diameter is certified using interferometer. Diameter ± 0.0015 mm
2. Thread plug gauge of size around M 48 x 5 and M 90 x 5 with certified effective diameter pitch and half profile angle. (1) Effective diameter
(2) Pitch
(3) Half profile angle
± 0.0025 mm ± 0.0025 mm ±3′

For all the above measurements, measuring knives supplied in set alongwith the instrument are used.
Every parameter is measured three times and average of these three measurements is taken as the required parameter.

Next post:

Previous post: