In mobile phone cameras, usually a voice coil motor (VCM) is used as a micro-positioning device for the image autofocus (AF) because of its low cost, simplicity, and reliability. Measuring the actual displacement of the VCM is important when we assemble the camera and test the AF performance for distant objects. In this paper, we propose using a confocal displacement sensor for calibrating the VCM displacement, where the axial chromatic aberration of a confocal objective lens is used to measure the target position. The tolerance angle for the dynamic tilt of a VCM increased up to ±15o because of the large numerical aperture of the confocal objective lens, which increased the stability of the repeatable in-line inspection. We compared the measurement robustness of the confocal displacement sensor with that of the laser displacement sensor in a mass production line to verify its performance superiority.