Build a ROS 2 Robot That Sees and Drives to a Marker

What you are building

A robot that is put down somewhere in a room, turns until it finds a printed marker on a wall, and drives up to it — stopping half a metre away, square to the marker’s face rather than merely close to it.

Every other build on this site runs one program on a microcontroller. This one is a computer with wheels, and that changes what the project is about. The hard parts are no longer timing and wiring; they are knowing where things are relative to each other, and keeping several pieces of software honest with one another while they all run at once.

Why this robot needs a computer

An Arduino cannot do this, and the reason is worth being precise about. It is not raw speed — it is that a camera frame is a quarter of a megabyte, ArUco detection wants floating-point linear algebra, and the whole thing has to happen while the wheels are still being commanded at 50 Hz. That is a job for an operating system with real memory and real scheduling.

What you get in exchange for that Linux box is the ability to look inside a running robot. Every message between every part is a topic you can subscribe to from a laptop. When the robot drives past the marker instead of to it, you do not add print statements and reflash — you ros2 topic echo the pose and watch it be wrong in real time.

The shape of the node graph

Five responsibilities, one node each:

The interfaces are what make this worth structuring. The controller never touches a camera or a motor — it consumes a transform and produces a Twist. That means you can drive the same robot with keyboard teleop, test the controller against a fake transform with no hardware at all, and later drop in a navigation stack that publishes the identical topic.

Read nodes, topics, services, and actions before drawing your own version of that table. The most common mistake at this stage is making the docking controller a service — it is a long-running goal you must be able to cancel, which is exactly what an action is for.

Bill of materials

This is the two-brain robot, and the parts list reflects that: a computer for perception and a microcontroller for the things a computer cannot do reliably.

Total: roughly $145–175. This is the most expensive project on the site by a wide margin, and most of it is the Pi.

Power is where these robots fail, and the failure is expensive. A brown-out during an SD card write corrupts the whole system image. Use a switching regulator rated well above the Pi’s draw, never share the rail with the motors, and check vcgencmd get_throttled — anything non-zero means the supply is inadequate and a corrupted card is coming.

Measure your printed markers. An ArUco pose solution scales directly with the marker size you declare. A marker printed at 95% by a printer’s “fit to page” gives poses 5% wrong at every distance, and nothing in the pipeline will tell you.

Build order

1 — Get ROS 2 running and prove the network. Follow installing ROS 2 on a Raspberry Pi. Do not move on until ros2 topic echo on your laptop shows a topic published on the robot. Discovery over Wi-Fi is the single most likely thing to eat an evening, and you want it eaten now rather than while debugging vision.

2 — Make the wheels obey a Twist. Wire the Arduino, encoders and TB6612FNG, and build the cmd_vel bridge. Milestone: keyboard teleop from the laptop drives the real robot. Calibrate wheel separation here — everything downstream inherits it.

3 — Calibrate the camera. Properly, with a real target, following camera calibration and ArUco pose. An uncalibrated camera produces poses that look plausible and are wrong by tens of percent at the edges of the frame, which is the worst failure mode available to you.

4 — Publish the marker as a transform. Not as a custom message with an x and a y in it. Publishing camera_link → marker_7 into TF2 means the controller can ask “where is the marker, in base_link?” and the answer accounts for exactly how your camera is mounted — including the fact that it is tilted, which it is.

5 — Drive at the marker. A proportional controller on bearing: turn toward it, drive forward, stop at the standoff distance. This works, and it arrives at an angle, which is the point of the next step.

6 — Approach along the normal. Instead of steering at the marker, compute a waypoint standing off from the marker’s face along its normal, drive to that, and only then drive straight in. This is the difference between reaching the dock and docking. It is also where the marker’s orientation — the part of the pose you did not need until now — starts to matter, and where you will meet pose ambiguity.

7 — Wrap it in a state machine. Search, approach, align, final, done, plus the transitions out of each when the marker is lost. Finite state machines for robot behaviour covers the structure; the specific thing this robot needs is a lost-marker timeout that returns to search rather than continuing on the last known pose.

Hardware notes that matter

Mount the camera rigidly, and measure how. Every pose the robot computes is relative to the camera, and TF2 turns that into a robot-relative pose using the transform you declare from base_link to camera_link. A camera on a flexible bracket makes that declared transform a lie that varies with acceleration.

Tilt the camera down, and say so. A camera pointed at the horizon loses the marker exactly when the robot gets close. Ten to fifteen degrees down keeps it in frame through the final approach — and that tilt goes into the static transform, not into a fudge factor in the controller.

Measure the marker with calipers. Pose scale is directly proportional to the marker size you tell the solver. Printing at 96% because the printer scaled to fit makes every distance 4% wrong, silently and consistently.

Feed the Pi separately from the motors. A stalling motor drops the rail, and where an Arduino would reset harmlessly, a Pi corrupts its filesystem. Either a separate regulator or a separate pack — this is not the place to save a component.

Keep the camera ribbon away from motor leads. It is an unshielded high-speed link. Motor noise coupled into it appears as corrupted frames, which appear as detections that flicker for no visible reason.

What good looks like

Two rows carry the lesson of the whole project.

Resolution is not free, and higher is often worse. Going from 640×480 to 1080p costs 4–6× the processing for roughly 1.7× the detection range. Unless you specifically need to see a marker further away, the smaller frame gives you a faster, fresher answer — and freshness matters more than precision in a control loop.

The latency row is why the two-brain split exists. At 0.3 m/s, 100 ms of vision latency is 3 cm of travel; turning at 90°/s it is 9° of rotation. Every vision measurement describes where the robot was. Close a fast loop directly on that and it will oscillate. The working structure is the one this robot uses: the Arduino runs the fast loop on encoders, and vision corrects it slowly — exactly the same shape as fusing GPS with odometry on the waypoint rover, and for exactly the same reason.

Rolling shutter, if the robot moves while it looks

The standard Pi camera modules have a rolling shutter: rows are exposed one after another, so the top of a frame is captured a few milliseconds before the bottom. Stationary, this is invisible. Turning at 90°/s, the four corners of an ArUco marker were captured at four different robot orientations — and the pose solved from them is wrong in a way calibration cannot fix.

If the robot must do vision while moving, that is what the Global Shutter module is for. If it can afford to stop, look, and then move, the cheaper module is entirely fine — and stopping to look is a legitimate design choice, not a workaround.

When it goes wrong

The distribution in that table is worth noticing before you start: almost none of it is computer vision. It is networking, frames, units and control — which is what building a ROS 2 robot actually teaches, and why the ROS 2 foundations roadmap spends its time on communication patterns and coordinate frames rather than on OpenCV.

If this is a step too far, the micromouse teaches mapping and planning on a microcontroller with no Linux involved, and the phone-controlled robot covers the “commands arrive over a network” half of this problem for a fraction of the parts.