The robot sees the part but misses the grasp: how much does half a degree matter?

The problem: recognition does not guarantee reaching

Imagine a gripper picking up a small component from a table. The camera recognizes it and the software draws the right box, yet the fingers descend several millimetres to one side. We naturally suspect the vision model. The fault may instead lie in translating between two descriptions of space: the camera’s and the robot’s. This article explains how an almost invisible angular error can become a noticeable grasping error, and how to distinguish that problem from image noise.

Our precise question is: how much does the commanded position change when the camera–robot geometric relationship is slightly wrong? We first build a planar example, then estimate uncertainty. The main result needs no matrix algebra to understand: the same orientation error matters more for points farther from the transformation’s origin. At half a metre, half a degree creates about 4.36 mm of displacement. These are reproducible educational calculations, not measurements on a real robot or EL-AI industrial results.

Two maps of the same table

A coordinate frame has an origin, directions and units. Saying “40 centimetres to the right” is insufficient unless we know to the right of what. Let p_C describe the point in the camera frame and p_B in the robot-base frame. The physical point stays still; only its description changes. We assume that a metric position on the plane is already available. A box in pixels alone does not provide that position: depth, or a known plane with calibrated projection geometry, is needed.

Moving between the maps requires a rotation R to align directions and a translation t to connect origins. Thus p_B = R p_C + t means: orient the vector measured from the camera, then add the displacement of its origin relative to the base. Modern Robotics explains the distinction between describing a point in another frame and physically moving an object. Confusing these operations, or using the inverse transformation, can cause much larger errors than those studied here.

Here θ is the angle between the frames’ axes; trigonometric calculations use radians. R has no units, while p_C and t are in metres. For a readable example choose θ = 0 and t = (0.20, 0.10) m: the frames share an orientation but have different origins. The target p_C = (0.40, 0.30) m becomes p_B = (0.60, 0.40) m. Its distance from the camera origin is 0.50 m, following the familiar 3–4–5 right triangle.

Half a degree becomes millimetres

Now suppose the software mistakenly uses θ = 0.5° rather than zero, while the translation remains correct. The point rotates slightly about the camera origin. Think of the tip of a clock hand: for the same angle, a long hand travels farther than a short one. This analogy concerns geometry only; we are not modelling robot inertia or velocity. Subtracting the correct and computed positions measures the displacement without simulating motion.

I denotes the transformation that leaves the point unchanged; double bars denote error-vector length. The first component places the estimated target about 2.63 mm to the left, the second about 3.48 mm upward in our plane. We do not add these numbers: they are perpendicular displacements, so distance follows Pythagoras. A practical consequence is already clear: perfect visual recognition would not compensate for this incorrect geometric relationship.

We can generalize without listing cases. Two radii of length r separated by angle δ form an isosceles triangle; the segment connecting their tips is the chord. Halving the triangle gives r·sin(|δ|/2) for each half of that chord. Doubling gives the exact distance. For small angles, the sine of an angle in radians is nearly the angle itself, yielding the simple rule “distance times angular error”.

The table compares two calculations of the same experiment, not two robots. Even at five degrees the chord and approximation differ little here, yet the positional error is already large relative to many components. Approximation accuracy and process acceptability are different questions. Also, r is neither distance from the robot base nor arm length: it is distance from the origin about which we perturb the rotation.

Errors have direction

If translation is also wrong, the effects combine as vectors. Add 2 mm along the vertical axis to the half-degree example: the result is about 6.079 mm, not 4.363 + 2 = 6.363 mm. An opposite error could reduce displacement at that point and create the illusion of good calibration. Moving the target changes this accidental cancellation. One successful point therefore does not establish that the transformation is correct across the workspace.

For small errors we can separate three contributions: error in the observed point, translation error and angular error. Call them δp, δt and δθ. Matrix J rotates a vector by ninety degrees: it identifies the tangent direction in which the point initially moves as the angle changes. We neglect products of small errors. This is a linearization, a local description rather than a new exact law.

This formula also indicates where to look for the cause. Wrong translation displaces all points in the same way. Wrong rotation creates a pattern that depends on their positions. Localization noise may instead vary between images. These are diagnostic signatures of the model, not an automatic diagnosis: optical distortion, structural flex and depth changes may overlap and require a richer model.

How much can we trust the estimated point?

So far we imposed a specific error. In practice we often know a spread: repeated measurements return slightly different values. Standard deviation, denoted σ, describes the scale of this variability; it is not a guaranteed maximum. Assume independent, zero-centred errors: 0.5 mm for each point coordinate, 0.7 mm for each translation component and 0.2° for the angle. These are synthetic assumptions chosen to illustrate the mechanism, not camera specifications.

The covariance matrix Σ collects coordinate variances and how coordinates vary together. If that language is unfamiliar, picture a cloud of possible positions: a circular cloud means equal spread in all directions; an elongated one indicates a more uncertain direction. The following formula adds independent contributions. Superscript T denotes transposition, exchanging rows and columns; vvᵀ builds the contribution directed along v.

The last two lines apply to our isotropic errors: equal spread in both coordinates. The radial direction follows the origin-to-target line; the tangential direction is perpendicular. Angular error mainly widens the latter. At r = 0.5 m we obtain 0.860 mm radially and 1.946 mm tangentially. Replacing them with a single “camera accuracy” would be misleading: they concern the whole transformation under specific assumptions.

We checked the calculation using 100,000 Gaussian realizations with seed 20260925 and the exact, unlinearized rotation. Observed standard deviations are 0.858 mm and 1.949 mm, close to local predictions. The 95th percentile of total distance is 3.930 mm and the 99th is 5.096 mm, obtained by sorting samples and selecting the 95,000th and 99,000th. These are simulation percentiles, not guarantees that 95% of real grasps succeed: contact, gripper orientation and component dimensions are absent.

More images do not erase wrong calibration

The simulation represents possible errors in the complete estimate, like an ensemble of hypothetical calibrations and observations. In a system with fixed calibration, some error remains common to every image. Averaging N independent observations can reduce visual-noise variance by a factor of N; it does not automatically reduce shared rotation or translation error. A gripper can therefore be highly repeatable and always hit the wrong point. Repeatability means low spread; accuracy means closeness to the correct reference.

Calibration geometry also matters. In an even simpler example, estimate a direction from two points separated by L, each with independent transverse noise of standard deviation σ. Their difference has deviation √2·σ; dividing by L estimates angular uncertainty in radians. With σ = 1 mm, a 100 mm baseline gives about 0.810°, while 500 mm gives about 0.162°. This is a two-point linearization, not an advertised hand–eye algorithm accuracy; it shows why nearby references may provide poor orientation information.

From explanation to a useful check

A check consistent with this analysis uses targets independent of those used to estimate the transformation, distributed across the workspace. Record the residual vector at each position, not merely average length. Nearly parallel residuals suggest examining translation; residuals that rotate or grow with distance suggest examining orientation. This is a proposed protocol, not one executed on hardware. The measurement reference needs known uncertainty, otherwise we might attribute the checking system’s own error to the camera.

Intrinsic camera calibration describes projection and distortion; extrinsic calibration describes relationships between frames. OpenCV 4.13.0 documentation distinguishes these objects and provides calibrateHandEye for the camera–gripper relationship. With a robot-mounted camera, the base–camera relationship changes with arm pose and several transformations must be composed. Our fixed-relationship example does not solve that full problem: it isolates propagation of an error already present in one transformation.

The model excludes the third dimension, correlated errors, deformation, acquisition-to-action delay and tool uncertainty. A moving object can be localized correctly yet be elsewhere when the gripper arrives. A non-Gaussian distribution may have heavier tails than our simulation. We have therefore established neither safety, compliance nor grasp success. We have obtained a way to formulate measurable questions before replacing the AI model or purchasing more expensive components.

What we have learned

A gripper can miss a correctly recognized object because seeing and reaching require a reliable geometric relationship. Half a degree has no operational meaning on its own: target distance and direction also matter. Separating systematic error, spread and geometry helps identify the missing measurement. Industrial and collaborative robotics is a stated direction of interest for EL-AI; this article develops its foundations without announcing undocumented products, installations or company tests.

Sources and reproducibility

Kevin M. Lynch, Frank C. Park — Modern Robotics, §3.3.1, Homogeneous Transformation Matrices.

OpenCV 4.13.0 — Camera Calibration and 3D Reconstruction; calibrateHandEye.

The sources support geometric conventions and calibration distinctions; numbers, planar derivation and simulation are the educational analysis executed here. The snippet below checks deterministic displacement. The archive contains the full experiment, including error generation and percentiles. Running code is unnecessary to follow the conclusion, but makes the numerical core checkable.

import math
p = (0.4, 0.3)  # metres in the camera frame
a = math.radians(0.5)
q = (math.cos(a)*p[0]-math.sin(a)*p[1],
     math.sin(a)*p[0]+math.cos(a)*p[1])
error_mm = 1000 * math.hypot(q[0]-p[0], q[1]-p[1])
print(round(error_mm, 6))  # 4.363309

Code, data, and instructions · JSON. Educational calculations executed with Python 3.14.0; figures with Matplotlib 3.11.2. AI-assisted analysis, without claiming peer review or human review. Original illustrative ImageGen cover: it does not document EL-AI people, premises, or installations. Sources accessed 25 September 2026.