NeurIPS-2024 - Template-free NeRF for Semantic 3D Reconstruction of Dynamic Scene

Sep 2, 2026

Template-free NeRF is evaluated for semantic 3D reconstruction in dynamic scenes, focusing on multi-entity reconstruction with semantic outputs. Experimental comparisons cover prior dynamic NeRF methods and variants with or without pretraining features, reporting geometric accuracy using distance/compensation metrics and overlap-based precision, recall, and F-score. Results also analyze hand, object, and scene reconstruction, including ablations on initialization and head-sharing, alongside evaluation under ground-truth versus predicted masks and pose, demonstrating improved reconstruction quality for the proposed method.

LanguageEnglish

CategoryGeneral

FormatPDF

Pages20

Reading Time7 min

AccessFree to download

Methods and Training Settings

Template-free vs Pretrained Features

Evaluation Metrics and Comparative Results

Distance/Compensation, Precision/Recall, F-score, Chamfer

Reconstruction Tasks and Ablation Studies

Hand, Object, and Scene Reconstruction

Ground-Truth vs Predicted Mask/Pose

| Methods | Template
free | No pre
trained
features | Reconstructs | | --- | --- | --- | --- | | Vid2Avatar [39] | ✗ | ✓ | single entity | | AnimatableNeRF [7] | ✗ | ✓ | single entity | | SDF-PDF [8] | ✗ | ✓ | single entity | | HumanNeRF [40] | ✗ | ✓ | single entity | | HOSNeRF [13] | ✗ | ✓ | multiple entities | | NDR [15] | ✓ | ✓ | single entity | | HyperNeRF [27] | ✓ | ✓ | single entity | | D-NeRF [14] | ✓ | ✓ | single entity | | BANMO [16] | ✓ | ✗ | single entity | | RAC [17] | ✓ | ✗ | single entity | | TAVA [28] | ✓ | ✓ | single entity | | Ours | ✓ | ✓ | multiple entities
with semantic | | ❳❳❳ Metric ❳
❳
Method ❳❳❳❳ | Trained with
ResField [45] | Dist. Acc. ↓
(cm) | Comp. ↓
(cm) | Prec. ↑
(%) | Recal. ↑
(%) | F-score ↑
(%) | Chamfer ↓
(cm) | | --- | --- | --- | --- | --- | --- | --- | --- | | Tensor4D [25] | ✗ | 4.152 | 2.441 | 69.413 | 91.642 | 78.993 | 3.297 | | NDR [15] | ✗ | 4.203 | 3.527 | 73.048 | 78.846 | 75.599 | 3.865 | | HyperNeRF [27] | ✗ | 4.125 | 3.362 | 73.510 | 80.683 | 76.661 | 3.742 | | D-NeRF [14] | ✗ | 7.074 | 7.301 | 48.324 | 45.781 | 46.301 | 7.188 | | NDR [15] | ✓ | 3.591 | 3.193 | 78.564 | 82.472 | 80.385 | 3.399 | | HyperNeRF [27] | ✓ | 3.879 | 3.313 | 76.099 | 81.578 | 78.619 | 3.596 | | D-NeRF [14] | ✓ | 3.927 | 3.364 | 75.774 | 81.453 | 78.398 | 3.646 | | Ours | ✗ | 2.721 | 2.142 | 89.120 | 93.853 | 91.343 | 2.431 | | ❳❳❳ Metric ❳
❳
Method ❳❳❳❳ | Trained with
ResField [45] | Dist. Acc. ↓
(cm) | Comp. ↓
(cm) | Prec. ↑
(%) | Recal. ↑
(%) | F-score ↑
(%) | Chamfer ↓
(cm) | | --- | --- | --- | --- | --- | --- | --- | --- | | Tensor4D [25] | ✗ | 4.409 | 2.419 | 69.402 | 91.680 | 79.000 | 4.414 | | NDR [15] | ✗ | 4.865 | 3.546 | 69.653 | 76.994 | 72.728 | 4.205 | | HyperNeRF [27] | ✗ | 4.794 | 3.363 | 70.276 | 79.531 | 74.202 | 4.078 | | D-NeRF [14] | ✗ | 6.515 | 6.132 | 53.013 | 52.745 | 52.071 | 6.324 | | NDR [15] | ✓ | 4.681 | 3.186 | 72.934 | 81.025 | 76.553 | 3.931 | | HyperNeRF [27] | ✓ | 4.265 | 3.394 | 73.864 | 79.642 | 76.331 | 3.831 | | D-NeRF [14] | ✓ | 4.729 | 3.403 | 71.211 | 79.244 | 74.706 | 4.066 | | Ours | ✗ | 1.761 | 1.863 | 97.225 | 93.624 | 95.343 | 1.812 | | Tensor4D [25] ✗ 4.390 | | | 2.523 | 56.953 | 91.683 | 70.260 | 3.956 | | NDR [15] | ✗ | 3.747 | 3.607 | 76.526 | 75.534 | 75.675 | 3.677 | | HyperNeRF [27] | ✗ | 3.647 | 3.508 | 78.171 | 76.892 | 77.144 | 3.586 | | D-NeRF [14] | ✗ | 4.675 | 5.529 | 64.88 | 54.095 | 57.748 | 5.102 | | NDR [15] | ✓ | 3.442 | 3.531 | 81.114 | 76.612 | 78.641 | 3.485 | | HyperNeRF [27] | ✓ | 3.451 | 3.282 | 80.551 | 80.720 | 80.174 | 3.379 | | D-NeRF [14] | ✓ | 3.565 | 3.362 | 79.379 | 79.155 | 78.875 | 3.464 | | Ours ✗ 3.571 | | | 2.121 | 82.762 | 92.410 | 86.991 | 2.741 | | | | Hand reconstruction | | | Object reconstruction | | | | --- | --- | --- | --- | --- | --- | --- | --- | | ❳❳❳ Metric ❳
❳ Method ❳❳❳
❳

Dist. Acc. ↓(cm) | | | F-score ↑ | Chamfer ↓ Dist. Acc. ↓ | | F-score ↑ | Chamfer ↓ | | | | | % | (cm) | (cm) | % | (cm) | | NDR [15] | ✓ | 1.419 | 94.051 | 1.217 | 1.154 | 93.782 | 1.279 | | HyperNeRF [27] | ✓ | 1.435 | 93.491 | 1.198 | 1.159 | 97.988 | 1.042 | | Ours | ✗ | 1.373 | 95.396 | 1.294 | 0.530 | 99.980 | 0.463 | | ❳❳❳ Metric ❳
❳
Method ❳❳❳❳ | Dist. ↓
Acc.(cm) | Comp. ↓
(cm) | Prec. ↑
(%) | Recal. ↑
(%) | F-score ↑
(%) | Chamfer ↓
(cm) | | --- | --- | --- | --- | --- | --- | --- | | TAVA [28] 2.79 | | 2.14 | 90.40 | 95.67 | 92.95 | 2.47 | | AnimatableNeRF [7] | 3.63 | 2.39 | 81.57 | 93.32 | 88.30 | 2.81 | | Ours | 2.47 | 2.01 | 92.15 | 95.99 | 93.97 | 2.24 | | TAVA [28] | 0.92 | 0.53 | 99.65 | 100.00 | 99.83 | 0.73 | | Ours | 0.81 | 0.61 | 99.99 | 100.00 | 99.99 | 0.71 | | ❤❤Me❤th❤od❤❤❤❤❤❤M❤et❤i | Dist. ↓ F-score ↑ Chamfer ↓
Acc. (cm) (%) (%) | | | | --- | --- | --- | --- | | W/o xinitc (Section 3, B)
Same geometry heads (Section 3, A,B,C) | 3.92
7.83 | 83.50
41.18 | 3.51
8.42 | | | |

FAQs

What does the document propose for dynamic-scene semantic reconstruction?

It presents a template-free NeRF approach aimed at semantic 3D reconstruction in dynamic scenes, including multi-entity reconstruction with semantic information.

How are the reconstruction results evaluated?

Evaluation reports geometric distance/compensation errors and overlap-based precision, recall, F-score, along with Chamfer distance to quantify reconstruction quality.

What ablations or evaluation conditions are included?

The document compares variants such as training with or without pretrained features, initialization/head-sharing choices, and evaluation using ground-truth versus predicted mask and pose inputs.

Sep 2, 2026

Upload by Arica Lee