Project page · 2026

UniQuery4R: Unified 4D Scene Reconstruction from a Single Query

Tiancheng Chen¹, Sheng Tang¹, Wenhua Jin^1,3, Weiqi Zhang², Juntong Fang², Junsheng Zhou², Zesong Li¹

¹ Kosmo Research ² School of Software, Tsinghua University ³ Automotive Engineering Department, Jilin University

Paper Code soon BibTeX Gallery

Given a continuous source-pixel query over a jointly encoded clip, UniQuery4R predicts target correspondence, target-time 3D position, scene flow, and source depth, while estimating camera parameters per view.

Visualization

Gallery

Select a sequence, then drag to orbit, pan, and scroll to zoom around the reconstructed 3D trajectories.

Choose a sequence

Preparing interactive viewer

Loading compact 3D reconstruction data.

Frame – / –

Visualization powered by Three.js. Drag to orbit, right-drag to pan, and scroll to zoom.

Benchmark

Interactive Comparison

Compare OpenD4RT, 4RC, V-DPM, and UniQuery4R under synchronized time and camera controls.

Choose a sequence

OpenD4RT

Waiting to load…

4RC

Waiting to load…

V-DPM

Waiting to load…

UniQuery4ROurs

Waiting to load…

Drag any panel to update all four viewpoints. Timeline playback and frame seeking are shared.

Paper

Abstract

Reconstructing dynamic 4D scenes requires jointly estimating correspondence, geometry, object motion, and camera motion. Existing feed-forward methods typically predict dense task-specific maps or independently process source–target pairs, leading to unnecessary computation for sparse queries and limited feature reuse across different frame pairs.

We present UniQuery4R, a query-conditioned framework that encodes a multi-frame clip once and selects the source view, target view, and continuous source-image coordinate only at decoding time via source-to-target cross-attention. Each query jointly predicts target correspondence, target-time 3D position, and scene flow, along with source depth, while camera parameters are estimated per view.

This design allows the encoded clip to be reused across arbitrary source–target selections and supports both sparse inference and dense reconstruction through batched queries, without learned temporal embeddings tied to a fixed clip length. We further introduce a direction–magnitude parameterization of scene flow with separate supervision for moving and static points. Across four evaluation datasets, UniQuery4R achieves the best macro-average results among the evaluated methods for WorldTrack dynamic-point tracking and for our scene-flow evaluation derived from the same datasets.

Approach

Method Overview

**UniQuery4R architecture.** A clip is jointly encoded into multi-scale features. At decoding time, a continuous query *q = (u, v, s, t)* selects the source and target views. The sampled source query attends to the full target feature map and jointly drives correspondence, geometry, and motion prediction.

Encode once

All views are jointly contextualized before the query is known, allowing the same encoded clip to serve arbitrary source–target selections.

Query continuously

Bilinear multi-scale sampling preserves sub-pixel coordinates and provides one interface for sparse on-demand inference and dense reconstruction.

Attend source to target

The source query attends over the full selected target field, recovering correspondence without fixed temporal embeddings or predefined matches.

Evaluation

Results

UniQuery4R achieves the best four-dataset macro-average results among evaluated methods for dynamic-point tracking under the WorldTrack protocol.

WorldTrack Dynamic-Point Tracking

APD (%) ↑ and EPE (m) ↓ across four datasets and their equal-weight macro-average.

Method	PStudio		PO		DR		ADT		Macro Avg.
Method	APD ↑	EPE ↓	APD ↑	EPE ↓	APD ↑	EPE ↓	APD ↑	EPE ↓	APD ↑	EPE ↓
SpatialTrackerV2	49.93	0.4639	53.45	0.5173	54.34	0.4368	69.39	0.2764	56.78	0.4236
St4RTrack	70.80	0.2479	67.38	0.314	74.24	0.2594	72.90	0.2958	71.33	0.2793
TraceAnything	70.86	0.278	39.37	1.0462	60.89	0.5642	75.69	0.2515	61.70	0.535
Any4D	74.72	0.2361	65.99	0.3783	76.72	0.3463	78.09	0.2378	73.88	0.2996
V-DPM	78.95	0.1757	80.82	0.1857	75.29	0.2438	85.47	0.1725	80.13	0.1944
4RC	69.08	0.259	79.81	0.2353	84.65	0.1667	84.16	0.177	79.43	0.2095
OpenD4RT	78.63	0.1811	66.03	0.3398	76.78	0.2407	69.91	0.2966	72.84	0.2646
UniQuery4R (ours)	84.64	0.1333	83.74	0.1604	80.00	0.2078	87.17	0.1387	83.89	0.1601

DAVIS Visualization

Long-sequence DAVIS tracking comparison. — **Long-sequence DAVIS tracking.** Each color denotes one fixed source query over time. UniQuery4R qualitatively exhibits less drift and fragmentation under occlusion and deformation.

Reference

BibTeX

If you find UniQuery4R useful, please cite our work.

@article{chen2026uniquery4r,
  title   = {UniQuery4R: Unified 4D Scene Reconstruction from a Single Query},
  author  = {Chen, Tiancheng and Tang, Sheng and Jin, Wenhua and Zhang, Weiqi
             and Fang, Juntong and Zhou, Junsheng and Li, Zesong},
  note    = {Project page; publication details forthcoming},
  year    = {2026}
}