AprilTags

Now, let's extrapolate this a bit.

Think about building a demo. You set up your robots, cameras, lighting, and the entire environment. Then you train your robot and everything is awesome. So awesome that you're asked to take the setup across the world and demonstrate it at a big event. Heck, it could just be down the street, or in the lobby of your office. The key is, you need to physically move your demo outside of the lab.

You pack the robot, camera, lighting, and everything else and head off to the great yonder. But when you get there, your robot is behaving differently. It seems to be missing it's mark 80% of the time, when back in the lab it was accurate over 80% of the time. That's a huge difference. What happened?

In actuality, so much could have happened. The robot's motor calibration could be off. The lighting could be different. Something could have been bumped or bent during transportation. The camera could be sitting a few centimeters away from where it was in the lab. Or it could be any combination of these possibilities, and more. But one of these things we can eliminate with AprilTags.

Had we placed a tag on, or near, the robot in a known location and recorded the camera's extrinsics prior to training, we could use that information to replicate the camera's position and orientation after we moved the setup. This is a hard lesson to learn, trust me.

As those familiar with my posts here at Cloutier.engineer, as well as on LinkedIn, you are familiar with the infamous story of The Backpack, so I won't repeat too much. But suffice to say, after an entire day of setup at Hannover Messe 2026, the world's leading industrial transformation trade fair in Hannover, Germany, my robots were working superbly and fulfilling their goal of being fantastic conversation starters.

Then, on day three, I left my post. I needed a croissant and espresso for breakfast. I was gone approximately 5 minutes, but when I came back, my camera and boom were laying prone on the table. It came to my attention later that a conference attendee had swiftly turned around and hit the boom with a large backpack, crashing the camera to the ground. The how was not important, but the demo was nonfunctional. I could just grab the Vision Calibration library and Bob's your uncle, right? Right?

Wrong....

I had so little time to prepare the demo that I settled on manually calibrating the setup with a tape measure, bubble level, string, and some weights to make the string hang straight. I had an entire day before the show to ensure everything was calibrated, I just hadn't planned on having to do it during the show too! It wasn't until that very moment that I envisioned the Vision Calibration library. What is the expression, something about necessity being the mother of invention?

In the end, I was back up and running in about 30 minutes, but that's about 25 minutes too long. Lesson learned, and an idea was born.

    
# Build the image
docker build -t vision_calibration-base .

# Create the container
docker run -it \
    --name vision_calibration \
    --privileged \
    --network host \
    -v ~/code/robotics/vision_calibration/test:/app/test \
    -v ~/code/robotics/vision_calibration/src:/app/src \
    -v /dev:/dev \
    vision_calibration-base bash
    

Once your initial training rig is setup, place an apriltag somewhere in view of the camera. Note, you MUST be able to replicate this later. So attach it properly to the robot, table, whatever, as long as you can put it in the same exact location relative to your robot later. The acquire the current camera extrinsics into a zip file:

    
python3 -m vision_calibration.calibrator --tag-id 1 --output-path /path/where/you/store/camera_extrinsics.npz
    

Assuming there could be more than one AprilTag in the camera view, we tell the program which we are using for calibration by passing the --tag-id which represents the numerical value of the tag, such as 1.

Once your training is done and you have flown across the globe to demonstrate the system and re-setup your demonstration... now your camera is nowhere near where it was during training. Simple enough, give your best effort to get it close, then place the apriltag in the same spot from your first acquisition (e.g. attached to the robot, etc) and run a live calibration session. Running the below will load your original camera_extrinsics.npz file and compare it to the current camera input in realtime, providing you instructions on what direction to move the camera.

    
# Note: I find it is easiest to adjust the pitch, roll, 
# and yaw first, followed by Y, then Z. 
python3 -m vision_calibration.calibrator --tag-id 1 --pose-file /app/test/data/camera_extrinsics.npz --live
    

Here is the sample output:

    
========================================
        CAMERA LIVE ALIGNMENT
========================================

POSITION
----------------------------------------
  X:    -30.6 mm   Target:    -33.2 mm   Error:     +2.6 mm
  Y:    -81.8 mm   Target:    -79.3 mm   Error:     -2.5 mm
  Z:    539.5 mm   Target:    542.0 mm   Error:     -2.5 mm

  Distance from target:      4.4 mm

========================================
           MOVE DIRECTIONS
========================================
  X: CENTERED
  Y: CENTERED
  Z: CENTERED

========================================
             ROTATION
========================================
  PITCH: CENTERED
  ROLL:  CENTERED
  YAW:   -1.45 deg

ORIENTATION ERROR
----------------------------------------
  Rx:    -0.01 deg
  Ry:    -0.81 deg
  Rz:    -1.45 deg

  Total rotation error:     1.66 deg