Surface-shape maps help an AI video model keep a subject consistent from every side

When an AI video model is given a few photographs of an object and asked to swing the camera round to its far side, it has to guess at surfaces it has never seen. Researchers at KAIST AI describe a system called OmniShowcase that addresses this in two ways. It gives the video model an explicit description of the subject's 3D shape, built from the reference images. It is also trained on synthetic videos in which every camera movement is tied to a real underlying 3D object. The work appears in a preprint posted to arXiv and has not yet been peer reviewed.

The task is known as subject-to-video, or S2V. It means generating a video that features a specific subject, such as an object or a character, supplied as reference images. According to the abstract, early S2V methods worked from a single reference image. As the viewpoint shifted, the model had to infer the subject's unseen appearance and geometry. More recent methods accept several reference views, which gives them more information from different angles. The authors argue that these methods still do not specify the 3D structure that connects the views. They also point out that existing training data do not guarantee that a moving camera keeps showing the same underlying 3D subject.

OmniShowcase has two components. The first is a fully automated pipeline for synthesising training videos. It produces clips along a wide range of camera paths, with viewpoint changes driven by the subject's geometry and extending to complete 360-degree orbits. The second is a method for conditioning the video model on geometry, meaning that extra input is supplied to steer what it generates. From the reference views, the system reconstructs surface-normal maps. These are images that record, for each point, which direction the subject's surface is facing, so they capture shape without colour or texture. The maps are arranged as a turntable, a set of views showing the subject from all the way round. The abstract notes that this turntable does not need to be aligned in time with the generated frames. In other words, the maps do not have to match the output video frame by frame.

To support the method, the team built two resources. Showcase-4K is a collection of videos rendered from 3D assets, which are digital models of objects. It covers twelve kinds of camera motion across 88 trajectory configurations, and each video is paired with surface-normal conditioning derived from reconstruction. ShowcaseBench is a separate, held-out evaluation set. It includes the source meshes and camera parameters used to render each video, so the true geometry behind every clip is known. In the authors' experiments, OmniShowcase achieved state-of-the-art results on subject identity, which measures how faithfully the subject's appearance is preserved, and on 3D consistency. They report that its video quality and motion remained competitive.

Several questions remain open. The authors describe video quality and motion as competitive rather than leading, so the gains are concentrated on identity and geometric consistency. Both the training corpus and the benchmark are built from rendered 3D assets. The abstract does not say how the approach performs on subjects photographed in the real world, where geometry must be reconstructed from imperfect images. All results are reported by the authors on a benchmark they constructed, and outside evaluation will follow peer review.