The pharmacist grounds each ideal request in a realizable scene; the generator renders it with exact labels. The pharmacist uses a Perceiver IO encoder and a DETR-style set decoder. Motion vectors enter only as keys and values, so the read is permutation-invariant and linear in set size; each object query emits placement and motion as one committed six-vector. Trained on near-uniform renders so it carries minimal bias toward any motion regime. Round-trip A→B→A median error is exactly zero at 1 px quantization.
The label guarantee. Deformation bends the space, not the object. Every deformed point maps back to one point on the undeformed surface, so the labels stay exact under bending, and a one-line cycle check rules out folds.
This flat puzzle uses |Δx| + |Δy| + 2(|Δmotion-x| + |Δmotion-y|), in scene units. The training model uses full 3D placement and motion vectors; this is a simplified 2D illustration.
The highlighted cells form the cheapest one-to-one assignment, found by checking all six pairings. Reordering rows cannot change that minimum.
Why this matters: a scene has no natural “first object.” The pharmacist's object queries predict placement and motion together. Assignment pairs the predictions with the true objects before computing their loss, so output order does not define object identity. The separate camera prediction is outside this object matching.
A request goes in as a bag of motion vectors; the pharmacist emits a scene; the generator renders it with exact labels. Teal = requested, orange = delivered, measured from the rendered pair's flow.
"Copy" requests take a real benchmark pair's own annotated motion as the bag; the round trip shows how faithfully a fabricated scene can reproduce it (the labels are exact either way). "Gap-fill" requests are what the doctor actually writes: a fabricated bag aimed at deficient cells of the coverage table (a placement, a magnitude band, a direction sector, and a camera kind). The pharmacist never sees an image; it reads the vectors as keys and values, so the read is permutation-invariant, and emits six committed numbers per object. Delivered magnitude tracks the request; direction spread is realised by the camera kind.
