Tip:
Highlight text to annotate it
X
This demo demonstrates frame rate upsampling, extrapolation forward in time to reduce perceived delay in an input->render->display loop, and anaglyph stereo pair generation.
The technique used generates a keyframe every N frames, storing the color, velocity, current time, and transform of the keyframe. Extrapolated frames are reprojected from the last keyframe by scattering the keyframe's pixels as variable-sized points to their new position in the output image, which is somewhere forward in time and possibly under a different transform due to user input after the keyframe. It's important that the points have their size adjusted to compensate for changes in screen-space area that arise when an object is under rotation, otherwise we'd end up with gaps.
The black regions trailing the cubes are areas of disocclusion, pixels that were occluded in the keyframe but have become visible in the reprojected frame. These areas should be covered up with some sort of hole-filling algorithm, possibly using data from previous keyframes. For this demo, we won't try to address that problem, and the black regions will actually be helpful to illustrate how the algorithm works and its shortcomings.
What we're currently looking at is rendering extrapolated into the future by 50ms, and a keyframe generated every 4 frames. Let's pause the demo and look at what adjusting the extrapolation time does.
Now, let's resume the animation, and pan around a bit.
The black regions at the edge of the screen can be fixed by rendering with a larger field of view that extends outside the visible window, giving us extra border pixels to sample from when rotating the screen. The framerate dips a bit when the entire screen is undergoing motion, due to the increased number of pixels that need to be scattered (there's a fast path for static pixels).
Let's look at what happens when I set the keyframe interval to something very large. Keep an eye on the black disocclusion regions to get an idea of how often the keyframe is being regenerated. You can see that for most of the cubes, even in this extreme case, motion holds up well, despite the keyframes being quite a bit off.
Finally, here's anaglyph stereo pair generation. The artifacts in this mode are pretty strong due to the large separation from the keyframe at all times.