HD Map Quality Assurance & Certification

Every stage before this one asks whether the map is internally correct — whether the topology closes, whether the geometry is continuous, whether the schema validates. Certification asks a different question: how do you know the map matches the world, and can you prove it to somebody who was not there. Those are measurement and evidence problems, and passing the gates in topological validation rules contributes almost nothing to either.

This stage sits alongside HD map version control — it is what turns a signed release into a release with a claim attached. Its constraints are that the accuracy figure must be measured against ground truth the map was not built from, that the gate must refuse rather than warn, and that every shipped lane must resolve back to the inputs that produced it.

The four artefacts a certified release carries, and what each one answers:

A useful way to hold the distinction: the earlier stages establish that the map is a consistent description of something, and certification establishes that the something is the road. Neither implies the other. A pipeline with an unnoticed datum shift produces a map that passes every topology and continuity check and is a metre from the ground everywhere; a pipeline with a correct datum and a broken junction expansion produces a map that sits exactly on the road and cannot be driven. Assurance therefore has to reach outside the pipeline for its ground truth, which is the single structural property that makes it different from the gates before it — and the reason the control-point split has to happen before the build rather than after.

Assurance Strategy Comparison #

Four strategies are in common use. They differ in what they can catch and in how much of the map they touch.

The strategies are complementary rather than ranked, and a credible safety case uses all four: the gates give total coverage of a narrow class, the control points give a defensible absolute number, cross-source agreement catches attribution mistakes that no geometric check sees, and drives catch the things nobody thought to check for. What matters is being explicit about the blind column — the safety case has to state what the assurance did not cover.

Stage-by-Stage Implementation #

Stage 1 — Withhold control before you build #

The constraint: a control point used to build the map cannot measure the map. Split the surveyed control set before the pipeline runs and keep the withheld half out of every stage, including the datum fit in coordinate reference systems for AVs.

import numpy as np

def split_control(points: np.ndarray, ids: list[str], holdout_frac: float = 0.3,
                  seed: int = 20260809) -> tuple[dict, dict]:
    """Deterministically split surveyed control into fit and holdout sets."""
    rng = np.random.default_rng(seed)
    order = rng.permutation(len(ids))
    cut = int(len(ids) * (1.0 - holdout_frac))
    fit = {ids[i]: points[i] for i in order[:cut]}
    holdout = {ids[i]: points[i] for i in order[cut:]}
    return fit, holdout

The seed is pinned and recorded in the validation record, so the split is reproducible and an auditor can confirm the holdout was not chosen after the fact. Stratify by road class if the control set is uneven — a holdout that is all motorway measures a motorway map.

Stage 2 — Measure absolute accuracy against the holdout #

The constraint: report a distribution, not a mean. A map with 0.04 m RMSE and a 0.9 m tail is not a 0.04 m map for anything that matters.

def absolute_accuracy(map_points: dict[str, np.ndarray],
                      holdout: dict[str, np.ndarray]) -> dict[str, float]:
    """Positional error statistics against withheld survey control."""
    common = sorted(set(map_points) & set(holdout))
    if not common:
        raise ValueError("no holdout point resolves in the map")
    err = np.linalg.norm(
        np.array([map_points[k] for k in common]) -
        np.array([holdout[k] for k in common]), axis=1)
    return {
        "n": len(common),
        "rmse_m": float(np.sqrt((err ** 2).mean())),
        "p95_m": float(np.percentile(err, 95)),
        "max_m": float(err.max()),
    }

Expected output: four numbers per release. The gate is written against p95_m and max_m; RMSE alone hides exactly the tail a hazard analysis cares about.

What a mean-only accuracy claim conceals:

Stage 3 — Measure coverage in lane-metres, not in area #

The constraint: coverage must be comparable with the operational design domain the safety case claims, and that domain is described as routes and road classes, not as square kilometres.

def coverage_lane_metres(lanes, meets_bar) -> dict[str, float]:
    """Fraction of drivable lane length that clears the full quality bar."""
    total = sum(l.length for l in lanes)
    good = sum(l.length for l in lanes if meets_bar(l))
    by_class: dict[str, float] = {}
    for cls in {l.road_class for l in lanes}:
        cls_lanes = [l for l in lanes if l.road_class == cls]
        cls_total = sum(l.length for l in cls_lanes)
        by_class[cls] = sum(l.length for l in cls_lanes if meets_bar(l)) / cls_total
    return {"overall": good / total, **by_class}

Breaking coverage down by road class is what turns the number into something actionable: 97% overall with 62% on residential streets is a very different release from 97% flat, and only one of them supports a claim about urban operation.

Stage 4 — Gate, and refuse #

The constraint: the gate's thresholds are fixed before the run, and the gate's outcome is binary. Anything softer is a warning, and warnings are not evidence.

from dataclasses import dataclass

@dataclass(frozen=True)
class ReleaseCriteria:
    p95_m: float = 0.10
    max_m: float = 0.25
    coverage_overall: float = 0.98
    coverage_min_class: float = 0.90
    open_defects: int = 0

def gate(accuracy: dict, coverage: dict, defects: int,
         crit: ReleaseCriteria = ReleaseCriteria()) -> list[str]:
    fails = []
    if accuracy["p95_m"] > crit.p95_m:
        fails.append(f'p95 {accuracy["p95_m"]:.3f} m > {crit.p95_m} m')
    if accuracy["max_m"] > crit.max_m:
        fails.append(f'max {accuracy["max_m"]:.3f} m > {crit.max_m} m')
    if coverage["overall"] < crit.coverage_overall:
        fails.append(f'coverage {coverage["overall"]:.3f} < {crit.coverage_overall}')
    worst = min(v for k, v in coverage.items() if k != "overall")
    if worst < crit.coverage_min_class:
        fails.append(f'worst road class {worst:.3f} < {crit.coverage_min_class}')
    if defects > crit.open_defects:
        fails.append(f'{defects} open defect(s)')
    return fails

ReleaseCriteria is frozen and version-controlled next to the pipeline, so relaxing a threshold is a reviewable change rather than an argument in a release meeting.

Stage 5 — Emit traceability with the artefact #

The constraint: traceability must be queryable per lane, not per release, because the question an incident investigation asks is about one lane.

def traceability_record(lane, build) -> dict:
    return {
        "lane_id": lane.id,
        "tile": lane.tile_quadkey,
        "survey_pass": lane.provenance.pass_id,
        "captured": lane.provenance.captured_iso,
        "operator": lane.provenance.operator,
        "toolchain": build.toolchain_digest,
        "validation_run": build.validation_run_id,
        "feature_digest": lane.digest,
    }

Writing the record next to the lane, keyed by the same digest the version-control layer uses, means a lane and its provenance cannot drift apart: change the lane and the digest changes, so a stale record is detectable rather than merely wrong.

Validation & QC Automation #

The assurance pipeline itself needs a gate, because a gate that never rejects anything proves nothing.

def assert_gate_rejects_known_bad(gate_fn, fixtures) -> None:
    """A gate that cannot fail is not a gate. Feed it defects it must refuse."""
    for name, args in fixtures.items():
        assert gate_fn(*args), f"gate accepted the known-bad fixture {name!r}"

Keep a fixture set of releases that must be refused — a shifted datum, a tile with a withdrawn lane, a coverage regression on one road class — and run it on every pipeline change. The enforced thresholds elsewhere: holdout ≥30% of control, never reused across releases without re-survey; p95 ≤0.10 m and max ≤0.25 m against the holdout; coverage ≥98% overall and ≥90% in every road class; and zero open defects at the release severity.

The four evidence sources, and the fact that none of them is a superset of another:

Edge Cases & Failure Patterns #

The holdout leaks into the build. The commonest way is a shared datum fit that quietly consumes every control point. Pass the fit set explicitly through the pipeline and assert that no holdout identifier appears in any build input.

Accuracy improves every release and the map does not. The holdout is being re-used, so the pipeline has slowly been tuned against it. Rotate the holdout on a survey cadence and record which split each release used.

Coverage is 100% and vehicles still meet unmapped roads. Coverage is being computed over the mapped extent rather than over the claimed operational domain. The denominator has to come from the domain definition, not from the map.

A lane's provenance points at a survey pass that no longer exists. Provenance was recorded by reference to mutable storage. Record the pass digest, not its path, and treat survey data as immutable.

The gate passes on a release whose tiles fail in the field. The gate measured the build output and the field runs the served tiles. Add a post-publication check that re-derives accuracy from tiles fetched through the serving path in map tile serving and distribution.

Performance & Scale Notes #

Accuracy measurement is cheap — a few thousand control points against a spatial index — and coverage is a single pass over lane geometry, so the whole assurance run is minutes even on a national map. The expensive part is survey, and it is amortized: control points are re-observed on a multi-year cycle, not per release.

Traceability is the part that scales badly if done naively. One record per lane over a national map is hundreds of millions of rows, so keep the record as a columnar sidecar keyed by feature digest rather than as a row per lane per release; unchanged lanes share a record across releases exactly as unchanged features share a content-addressed object.

Frequently Asked Questions #

What is the difference between absolute and relative map accuracy? #

Relative accuracy is how well the map agrees with itself — whether two lanes 30 metres apart are really 30 metres apart. Absolute accuracy is how well the map agrees with the ground, measured against surveyed control points in a known datum. A map can have excellent relative accuracy and be a metre off in absolute terms, which is fine for a vehicle localizing against map features and fatal for one fusing map geometry with raw GNSS. Both must be measured, and the accuracy claim in a safety case has to say which one it means.

Why must a release gate refuse rather than warn? #

A warning is a decision deferred to whoever is looking at the build output, which under release pressure is a decision to ship. A gate that refuses converts a quality question into a scheduling question, which is the only form in which it reliably gets answered. It also makes the evidence trail meaningful: a release that exists is a release that passed, so the artefact itself is the record, and nobody has to reconstruct from logs whether a warning was reviewed.

How is map coverage measured? #

In lane-metres that meet the full quality bar, not in area or in tile count. A region is not covered because a tile exists for it; it is covered to the extent that the drivable lanes inside it carry validated geometry, topology and attributes. Reporting coverage as a percentage of route lane-metres over the operational design domain makes it directly comparable with the domain the safety case claims, which area-based figures never are.

What does ISO 26262 actually require of a map artefact? #

It requires that the artefact be traceable, that its verification be planned and evidenced, and that changes be controlled and re-verified. In practice that means every shipped lane resolves back to the survey pass, toolchain version and validation run that produced it; that the acceptance criteria were written before the run rather than fitted to it; and that a change to any input invalidates the evidence until the affected artefacts are re-verified. The standard does not name a map accuracy number — that comes from your own hazard analysis.

Related #

• Measuring HD Map Accuracy Against Survey Control — the holdout split and the error statistics in full.
• Building a Map Release QA Gate in Python — wiring the gate into CI so it refuses rather than warns.
• Producing ISO 26262 Traceability for Map Artefacts — the provenance record and how it survives a re-build.
• HD Map Version Control — the signed release these artefacts are attached to.
• Topological Validation Rules — the internal-consistency half that certification sits on top of.

Up one level: HD Mapping Architecture & Spatial Data Standards — the pipeline whose output this stage certifies.