SINGLE IMAGE. EXPLICIT SCENE. EDITABLE LIGHT.

Beyond appearance.

Evidence-Guided Programmatic 3D Scene Reconstruction
with Agentic Lighting Reasoning

Zeting Luo · Yifan Li · Shuai Yang

Wangxuan Institute of Computer Technology, Peking University

Geometry×Materials×Illumination

01 / THE SCENE IS JUST THE BEGINNING

One image. Many possibilities.

Input photograph of a bedroom with striped sunlight on the wall and bedINPUT IMAGE
It starts with a single observation.
LightHarness reconstructed bedroom rendered in BlenderRECONSTRUCTED SCENE
Explicit geometry, materials, and lighting. Ready to edit.
EXPLORE AN EDIT

From Figure 1 of the paper. Select an edit to explore the rendered scene variants.

THE IDEA

A shadow is more
than a dark pixel.

A dark surface can mean dark material, a hidden occluder, or a missing light. Matching appearance alone does not tell us which scene property to change.

LightHarness reasons about them together. Complementary visual evidence and a persistent lighting memory guide a VLM agent to reconstruct an executable Blender scene—without additional training or differentiable rendering.

Read the method

02 / HOW IT WORKS

Observe. Reason. Render. Refine.

A closed loop, grounded in evidence.
A lighting memory that lasts across iterations.

Figure 3. Evidence-guided generation and verification, connected by Lighting Reasoning Memory.
01

Gather the evidence

Masks, depth, surface orientation, albedo, shading, and an HDR lighting prior constrain plausible scene explanations.

02

Keep a lighting memory

Track each light’s identity, supporting evidence, parameters, and revision history—including off-frame source hypotheses.

03

Verify, then refine

Decomposed render passes, rubric scores, and VLM diagnosis guide revisions. Export the best valid scene and its matching light state.

Look inside the visual evidence
Semantic masks for scene objects
Masks
Relative scene depth
Depth
Surface normal evidence
Normals
Estimated albedo
Albedo
Estimated diffuse shading
Shading
HDR lighting prior
HDR prior

03 / RECONSTRUCTION

Structure and light, in agreement.

Explore all six scene comparisons
shown in Figure 4.

LightHarness reconstruction of the backlit sceneReference image of the backlit sceneINPUTLIGHTHARNESS

Drag to compare · Use arrow keys for precise control

COMPARE METHODS

Recovering backlit structure

Inspect how each method reconstructs the foreground against a bright window.

VIGA reconstruction of the backlit sceneView all methods together ↗
15.03 dBPSNR ↑
0.216DreamSim ↓
61.9%Lighting preference ↑
74.6%Structure preference ↑

Reported in Table 1 of the supplied paper. User preference is measured across the five compared methods.

Full quantitative comparison
Table 1 · Source-view reconstruction. Best values are bold.
MethodPSNR ↑SSIM ↑LPIPS ↓DreamSim ↓
3DCodeVerse10.210.4280.6770.363
GPT-6-Sol13.030.5030.5480.318
VIGA14.280.5250.4460.242
SceneRig11.860.4550.6650.420
LightHarness15.030.4860.4370.216

Reason beyond what is visible.

A diagonal cast shadow constrains both the surrounding structure and the illumination.

Each source of evidence matters.

Removing evidence, geometric or lighting assessment, or memory affects layout and illumination.

EXPLORE IN 3D / THE SCENES BEHIND FIGURE 4

Step beyond the reference view.

Orbit the actual reconstructed geometry.
Compare LightHarness with four baselines.

LightHarness Ours
Paper rendering of the LightHarness scene before the interactive model is loaded
Original scene geometry · Reference camera
Paper rendering of the baseline scene before the interactive model is loaded
Original scene geometry · Reference camera
Drag to orbit · Scroll to zoom · Right-drag to pan

Models load on demand. Select a scene and a baseline, then open the 3D viewer.

Reference image & paper renderings
Reference image from Figure 4
Input image
LightHarness paper rendering from Figure 4
LightHarness · Paper rendering
Baseline paper rendering from Figure 4
VIGA · Paper rendering

Interactive previews use neutral studio lighting and approximate web materials; they do not reproduce Blender/Cycles light transport. Use the paper renderings to compare illumination. Cutaway hides recognized walls and ceilings. Linked navigation applies relative movements around each method’s own camera and scene scale.

04 / BEYOND RECONSTRUCTION

Edit the scene.
Let the light respond.

Change materials, insert objects, or move light.
The reconstructed scene makes the edits explicit.

Texture editing performed directly by GPT-Image
Direct GPT-Image editing
Texture edit rendered from the reconstructed Blender scene
Edit on our reconstructed scene
Rendered texture edit with GPT-Image post-processing
Our edit + GPT-Image post-process

Replace multiple surface textures while retaining an explicit scene structure. The paper also explores GPT-Image post-processing of the rendered edit.

05 / TAKE A CLOSER LOOK

From pixels to something editable.

Read the paper or open a scene in Blender.

Scene downloads correspond to the Figure 1 examples. The page displays pre-rendered results; downloaded scenes can be inspected and re-rendered in Blender.

Limitations and open questions

Inferring unseen objects from sparse or overlapping shadows remains difficult. Figure 11 shows incomplete reconstruction of shadows cast by off-frame people and a sign. More challenging scene examples and improved reasoning about unseen geometry remain directions for future work.

Reference scene with shadows of off-frame people and a sign
Input image
Reconstructed scene with incomplete shadows from unseen objects
Rendered reconstruction