ViewRec3D
Learning to Recommend 3D Viewpoints
for AI Photography

Chengzhi Zhang1,*, Yunzhong Hou1,*,†, Zizheng Sun1, Ming-Hsuan Yang2, Chi Harold Liu1

1Beijing Institute of Technology    2University of California, Merced

*Equal contribution.    †Corresponding author.

ViewRec3D overview: expert photographs, 3D scene reconstruction, dataset generation, and viewpoint recommendation
Overview of ViewRec3D: generating viewpoint recommendation data from expert photographs and learning to predict expert viewpoints with ViewRecNet.

Abstract

Viewpoint dictates the photo composition and plays a vital role in AI photography. Existing 3D viewpoint recommendation methods are often limited in viewpoint adjustments, which is mainly due to their lack of high-quality training data with accurate and large 3D view changes. Ideally, the training data should contain suboptimal images and expert-selected optimal viewpoints, but this can be very expensive to curate. To solve this, we build an automatically-generated 3D viewpoint recommendation dataset from expert photos. Specifically, we first consider these expert photos as optimal and outpaint them to provide more information on scene layouts. Next, we explicitly reconstruct the 3D scene. We then render suboptimal images from random viewpoints. Lastly, we introduce a hierarchical data filtering scheme and acquire high-quality data pairs for our ViewRecDB-100K dataset. On top of this, we further introduce a view recommendation model, ViewRecNet, that can predict expert viewpoints from any suboptimal image inputs using a ray-based dense supervision. The proposed method achieves strong results both quantitatively and qualitatively, supported by LLM-based viewpoint preference and human studies.

Data Generation

Data generation pipeline. Starting from expert photographs, we generate suboptimal-to-optimal viewpoint pairs through image outpainting, 3D Gaussian reconstruction, random-view rendering, and image restoration.

Data generation pipeline
Figure 2: Data generation method. Given an optimal image (a), we first determine whether orientation adjustment is required and outpaint the image (b). We then reconstruct the complete 3D scene (c) and render multiple Gaussian splatting (GS) images (d) from random viewpoints. Finally, we restore holes or other artifacts in GS images to get corresponding suboptimal images.

Hierarchical filtering. We apply hierarchical filtering to outpainted images, rendered views, and restored images, using image similarity, LLM-based visual checks, and VGGT-based pose verification to retain high-quality pairs.

Hierarchical filtering for outpainting, novel views, and image restoration
Figure 3: Hierarchical filtering for outpainting, novel views, and image restoration (left to right).

visualize

Select an original image below to explore its visualization. Translation: (x, y, z). Rotation: (pitch, yaw, roll), in degrees. Angular errors compare estimated and target poses. Move the pointer over the images, or swipe on touchscreens, to compare views. Use the arrow keys when a comparison is focused.

View Recommendation Model

Model Architecture. Given a single photograph, ViewRecNet recommends a camera rotation, translation, field of view, and image orientation (portrait or landscape). It predicts these adjustments from the camera token of a pretrained VGGT backbone. During training, an auxiliary head predicts camera rays from patch tokens, providing dense geometric supervision.

ViewRecNet architecture showing the camera prediction and ray supervision branches

Quantitative analysis. On ViewRecEval, ViewRecNet achieves the highest AUC@30 (0.5271). Human raters select its recommendation as the best in 44.48% of comparisons, the highest share among the five methods.

Table 2: Quantitative comparison on ViewRecEval.
MethodImage-level ConsistencyPose-level ConsistencyPreference
SSIM ↑LPIPS ↓PSNR ↑CLIP-I ↑AUC@30 ↑Dist. ↓LLM-Better ↑LLM-Imp. ↑Human-Best ↑
CACNet0.36160.717814.890.86080.31570.484334.40%–20.50%
Venus-Q0.35840.733014.730.86230.29900.447525.60%–20.20%
PPC0.39150.716715.100.69770.30600.596132.10%53.10%4.22%
SPAS0.37510.712015.050.84210.46860.458735.70%52.00%10.60%
ViewRecNet0.39290.691915.310.88120.52710.3080Ref.67.80%44.48%

Arrows indicate whether higher or lower values are better. “Ref.” marks the reference method in pairwise LLM comparisons; “–” indicates that the metric does not apply.

Qualitative analysis. Figure 5 compares the recommended views with the expert photographs. Compare the subject’s position and the amount of surrounding context to see how each method changes the framing.

Visual comparison of viewpoint recommendations and expert compositions