VehicleMapping HD AV Spatial Data

Round-Tripping HD Map Formats Without Losing Attributes

A conversion is trustworthy when you can state what it costs. That statement has two halves — how far the geometry moved and which attributes did not survive — and both are only measurable if a feature can be recognised at the far end of the loop. Identity is therefore the precondition, not a detail.

This task builds the gate for map format interoperability: a stable identity derived from geometry, a full loop with no intermediate inspection, geometric deviation measured against the source, and an attribute diff that separates dropped from changed.

Why the comparison has to close on the original, drawn as the two ways three hops can combine:

Compounding and Cancelling Hop Residuals with Identical Per-Hop Numbers Two vertex trajectories through three hops, one accumulating displacement and one returning near the origin, both with the same per-hop residual magnitude. source +0.04+0.04+0.04 compounding → 0.12 m from source source +0.04−0.04+0.04 cancelling → 0.04 m from source per-hop measurement reports 0.04 m for both so a hop-wise gate passes the left case, which is the one that fails in the field

Prerequisites #

  • Python 3.10+, NumPy 1.24+, shapely 2.0+.
  • Input: a source map, the conversion chain under test, and the sidecar convention from mapping OpenDRIVE, Lanelet2 and NDS.Live schemas.
  • Upstream stage: each individual converter passes its own tests; this gate measures the chain.
  • Output: a report of maximum geometric deviation and a three-way attribute diff.

Step-by-Step #

1. Derive an identity that survives every hop #

python
import hashlib
import numpy as np

def lane_identity(centreline: np.ndarray, ordinal: int, grid_m: float = 0.05) -> str:
    """A geometry-derived identity: quantized midpoint plus lane ordinal."""
    mid = centreline[len(centreline) // 2]
    q = np.round(mid / grid_m).astype(np.int64)
    payload = f"{q[0]}:{q[1]}:{ordinal}".encode()
    return hashlib.sha256(payload).hexdigest()[:16]

Key parameter: grid_m is coarse enough that resampling noise cannot change the identity and fine enough that two distinct lanes cannot share one. Quantizing at 0.05 m against a 3.5 m lane width leaves ample separation. Expected output: a 16-character identity that is identical before and after the loop.

2. Run the loop without looking inside #

python
def round_trip(source_map, chain):
    """Apply every converter in order and return the final map plus the sidecars."""
    current, sidecars = source_map, []
    for convert in chain:
        current, side = convert(current)
        sidecars.append(side)
    return current, sidecars

Inspecting intermediates is how hop-wise gating creeps back in. The chain is the unit under test.

3. Measure geometry against the source #

python
from shapely.geometry import LineString

def geometric_deviation(source, returned) -> dict[str, float]:
    """Max and 95th-percentile Hausdorff distance per lane, source vs returned."""
    out = {}
    for ident, src in source.items():
        got = returned.get(ident)
        if got is None:
            out[ident] = float("inf")             # lost entirely
            continue
        a, b = LineString(src), LineString(got)
        out[ident] = max(a.hausdorff_distance(b), b.hausdorff_distance(a))
    return out

Hausdorff rather than vertex-wise distance, because the loop legitimately resamples: the returned lane may have a different vertex count and still describe the same curve. An infinite value marks a lane that did not come back at all, which is a different failure from one that came back displaced.

4. Diff the attributes three ways #

python
def attribute_diff(src_attrs: dict, ret_attrs: dict) -> dict[str, list]:
    dropped = sorted(k for k in src_attrs if k not in ret_attrs)
    added   = sorted(k for k in ret_attrs if k not in src_attrs)
    changed = sorted(k for k in src_attrs
                     if k in ret_attrs and src_attrs[k] != ret_attrs[k])
    return {"dropped": dropped, "added": added, "changed": changed}

added is not noise: a converter that synthesizes a default where the source was silent has changed the map's meaning, and that shows up here and nowhere else.

The three diff buckets and what each one means for the map:

Dropped, Added and Changed Attributes After a Round Trip Three panels, each naming a diff bucket, what it indicates, and the action it implies. dropped in the source, gone after unmappable with no sidecar, or a converter bug must equal the declared set exactly — not merely be a subset added absent in the source, present after a converter invented a default where the source was silent the map now asserts something nobody surveyed changed present in both, different value a bug for a direct field; tolerable for a derived one gate on the field's class, never on the bucket alone a converter that reports zero in all three buckets and a non-zero geometric residual has done exactly what it should

Verification & Acceptance Criteria #

python
DECLARED_LOSS = {"user_data"}          # what we accept losing, exactly

def assert_round_trip(source, chain, tol_m=0.2) -> None:
    returned, sidecars = round_trip(source, chain)

    dev = geometric_deviation(source.geometry, returned.geometry)
    worst = max(dev.values())
    assert worst <= tol_m, f"worst lane deviates {worst:.3f} m > {tol_m} m"
    assert not [k for k, v in dev.items() if v == float("inf")], "lanes were lost"

    for ident, src in source.attributes.items():
        d = attribute_diff(src, returned.attributes[ident])
        assert set(d["dropped"]) == DECLARED_LOSS, f"{ident}: {d['dropped']}"
        assert not d["added"], f"{ident}: invented {d['added']}"
        for k in d["changed"]:
            assert classify(k) == "derivable", f"{ident}: direct field {k} changed"

Acceptance gate: worst-case geometric deviation ≤0.2 m against the source; zero lanes lost; the dropped set exactly equal to the declared loss set; zero invented attributes; and no direct field changed. The equality on DECLARED_LOSS is what turns "we know it is lossy" into a check that still fails when a new field starts being lost.

What each of the three diff buckets should contain in a healthy conversion, and what a non-empty one means:

Expected Contents of Each Diff Bucket Three rows pairing a diff bucket with its expected contents and the meaning of anything beyond that. a healthy round trip is not an empty diff — it is a diff whose contents were predicted dropped present before, absent after exactly the declared unmappable set anything else is a bug added absent before, present after empty — a default is an invented fact any entry is a bug changed present in both, different value derivable fields, inside tolerance a direct field is a bug the gate asserts equality on the first bucket rather than containment, so a newly lost field still fails the run

Common Errors & Fixes #

Identity does not match after the loop, so everything looks lost. The identity was derived from a source-schema id that got renumbered. Derive it from quantized geometry as in step 1.

Deviation is fine per hop and fails end to end. That is the gate working. Find which hop contributes most by running the chain with one hop replaced by the identity function.

A field appears in added on every lane. A converter is filling a default — usually a lane type or a speed limit. Make it emit nothing where the source was silent; a missing attribute and a defaulted one mean different things downstream.

Hausdorff distance is large but the lane looks identical. The returned lane extends past the source's end, usually because a boundary-crossing feature was reassembled from two tiles. Compare over the common station range, or fix the tiling seam.

The gate passes and a consumer still rejects the output. The output is lossless and non-conformant — an unmappable field was encoded as a target extension. Check that the target still validates against the unextended schema.

FAQ #

Why measure against the original rather than hop by hop? #

Because per-hop errors are not independent and can cancel. A resampling that shifts a vertex outward on the first hop and a smoothing that pulls it back on the second produce two small hop residuals and one small total, which is fine — but the same two hops can also compound, giving two small residuals and a total twice as large. Only the comparison against the source measures the quantity a consumer actually experiences.

What makes an identity stable across a conversion? #

It must be derivable from something every schema preserves, which in practice means the geometry itself rather than a numeric identifier. A hash of the quantized centreline midpoint plus the lane's ordinal within its section survives renumbering, reordering and re-encoding, and collides only where two lanes really do occupy the same place. Numeric ids from the source schema are renumbered by almost every converter and cannot be used.

Should a round trip ever be lossy on purpose? #

Yes, when the target genuinely cannot express something and the sidecar is not being carried — for example when handing a map to a third-party simulator that takes one file. The requirement is that the loss be declared rather than discovered: the gate should assert the dropped set equals the expected set exactly, so a newly lost field fails the run even though losses in general are tolerated.

Up one level: Map Format Interoperability: OpenDRIVE, Lanelet2 & NDS.Live — the stage this gate certifies.