Making YouTube VR180 captures with NVIDIA ShadowPlay, ffmpeg, and Valve Index

I recently picked up a Valve Index headset and am having some fun doing VFR flights. I’ve found that I can record the binocular stereo fisheye rendering using NVIDIA’s ShadowPlay capture tool, just like with regular 2d video, and that it’s possible to convert that to a format that YouTube accepts for VR180 playback, which works both in VR/3D and on flat 2D screens with a scrollable projection in the YouTube player.

4K recording attempting a landing at Long Beach (may still be processing at post time):

For best results, run the 2d screen at 4K and capture that – 1080p will produce a somewhat fuzzy picture, but 4K requires more system resources and produces larger uploads that take longer to process.

To avoid distortion, you’ll want to convert the stereo VR projection from the capture into an equirectilinear projection for VR180 video, switch from left/right stereo to top/bottom stereo, and correct for the Index’s eye cant (the displays for each eye are not parallel but cant outwards, in order to provide more peripheral vision). This can be done with ffmpeg, the swiss-army knife tool of video processing. It’s a command-line program with a very complex command interface, but good luck for you! You can just paste this into a .bat file once you’ve managed to install ffmpeg and it’ll convert a single input file to a single output file:

ffmpeg -i "%~f1" ^
-vf scale=4320:2160,^
setsar=1,^
split[left_in][right_in];^
[left_in]^
crop=^
w=2160:^
h=2160:^
x=0:^
y=0,^
v360=^
input=flat:^
ih_fov=104:^
iv_fov=110:^
h_fov=180:^
v_fov=180:^
w=2160:^
h=2160:^
yaw=10:^
output=he^
[left_out];^
[right_in]^
crop=^
w=2160:^
h=2160:^
x=2160:^
y=0,^
v360=^
input=flat:^
ih_fov=104:^
iv_fov=110:^
h_fov=180:^
v_fov=180:^
w=2160:^
h=2160:^
yaw=-10:^
output=he^
[right_out];^
[left_out]^
pad=^
w=2160:^
h=4320:^
x=0:^
y=0^
[left_padded];^
[left_padded][right_out]^
overlay=^
x=0:^
y=2160 ^
-c:v libx264 ^
-crf 18 ^
-c:a copy ^
"%~n1-vr180-canted.mp4"

I’ll try and clean this batch file up to make the variables easier to change! Things you might want to adjust:

  • the cant angle (10 degrees for the Index)
  • horizontal and vertical FOV (104 & 110, from a spreadsheet I found someone collecting data in, will link later)
  • the resolution (2160 pixels on a side for 4K recording, 4320 is double that)
  • the compression quality (CRF level 18 chosen arbitrarily in the scripts I based this script on)

You can then take this reprojected video and send it into the next step:

Google’s VR180 Creator tool can add the appropriate stereo/VR formatting metadata for YouTube to recognize it as a VR180 3D video:

https://arvr.google.com/vr180/apps/

For settings, select top/bottom stereo input and 180-degree field of view:

The output file can now be uploaded directly to YouTube and will play back in the VR player mode with no further intervention on your part. Pretty neat!

Unfortunately I can’t find a way to play YouTube VR videos on the Index headset (SteamVR), but I’ve confirmed they look ok on my phone in a Google Cardboard viewer. :wink: Note the cant correction for the Valve Index is important for 3D output, or the eyes don’t line up correctly.

I’ve had trouble finding other software that comfortably plays them back locally on Windows; VLC and the default Windows video player both seem to figure out it’s VR video but distort the view weirdly. But it’s playing back fine in YouTube, so I must be doing something right!

I’m considering doing some ‘stable-camera’ tests where I set the headset down on a desk to provide a more stable fixed camera view that the viewer can comfortably look around in, but with only 130 degrees FOV you don’t have a lot of peripheral vision to pan into. :slight_smile:

4 Likes

Good information. Remember to post YouTube videos in the #community:content-creators section. You can link to the video from your subject category.

1 Like

Thanks! I’ll post video-primary posts over there. :slight_smile: (Feel free to change the category if you need to.)