Mike Swanson's Blog

Spatial Video Tool

Spatial is a free macOS command-line tool to process MV-HEVC video files (currently produced by iPhone 15 Pro and Apple Vision Pro). It exports from MV-HEVC files to common stereoscopic formats (like over/under, side-by-side, and separate left- and right-eye videos) that can be used with standard stereo/3D players and video editors. It can also make MV-HEVC video from the same stereoscopic formats to be played on Apple Vision Pro and Meta Quest.

For a deeper dive into Apple’s spatial and immersive formats, read my post about Spatial Video.

Download

This tool uses Apple frameworks to encode/decode the MV-HEVC format and work with associated metadata. As such, it is Mac-only. It requires macOS Sonoma (14.0) or later and an Apple silicon Mac (though the ‘info’ and ‘metadata’ commands do work on Intel Macs).

If you have Homebrew installed:

brew install spatial

When you install spatial with Homebrew, there is no need to prefix spatial commands with ./

If you don’t want to use Homebrew, you can download the latest release and unzip it to a folder of your choice.

The zip file includes a small MV-HEVC-encoded sample file called spatial_test.mov for experimentation.

Usage

You can run ./spatial at a Terminal prompt for help. The tool has five commands:

  • info: Shows spatial media information for the first video track and its extensions. Can also be used to probe for a spatial video track.
  • export: Exports from a MV-HEVC spatial video to a stereo-formatted output video or two separate videos (first for the left eye and second for the right eye).
  • make: Makes a MV-HEVC spatial video from a stereo-formatted input video or two separate videos (first for the left eye and second for the right eye).
  • combine: Combines (muxes) already-encoded audio and MV-HEVC video files into a single output file.
  • metadata: Reports and modifies spatial video metadata in an input file and writes it to an output file.

Here’s an example command-line to export a spatial video to a standard over/under video:

./spatial export -i spatial_test.mov -f ou -o over_under.mov

Likewise, to make a spatial video from an over/under video (with some common iPhone-15-Pro-like spatial values):

./spatial make -i over_under.mov -f ou -o new_spatial.mov --cdist 19.24 \
--hfov 63.4 --hadjust 0.02 --primary right --projection rect

There are other command-line options to control the encoding format and bitrate, along with settings that are related to the spatial video format. You can find them in the help for each command.

Documentation

For a more in-depth description of the command-line parameters and options, read the Spatial Video Tool Documentation.

Tips

  • For Apple Photos to recognize a MV-HEVC output video as SPATIAL (and display that indicator), specify at least hfov and either cdist or hadjust on the command line. You can also use the info command to discover these values (for example, from an iPhone 15 Pro spatial video capture) if you’re working with a spatial source. Or, just use ‐‐cdist 19.24 ‐‐hfov 63.4 ‐‐hadjust 0.02 ‐‐primary right to experiment iPhone-15-Pro-ish values.
  • If your video is displaying with the wrong aspect ratio (perhaps ~1:1, square) in Apple Vision Pro, you need to tag the output video as rectilinear. Add ‐‐projection rect to your make command line.
  • There’s an issue that negative numbers on the command-line can be misinterpreted as arguments (e.g. ‐‐hadjust -0.02). To work around this for now, use an equal sign: ‐‐hadjust=-0.02
  • I’ve been able to produce 8K MV-HEVC spatial output with correctly-formatted source video: over/under source = 7680×8640, side-by-side source = 15360×4320.
  • According to Apple, iPhone 15 Pro and iPhone 15 Pro Max spatial videos are recorded at 1080p at 30 fps in SDR. One minute of spatial video is approximately 130 MB (one minute of regular 1080p 30 fps video is approximately 65 MB).
  • There can be issues with ffmpeg HEVC-encoded video and apps that process video with AVFoundation (e.g. spatial and Apple Preview, among many others). You may need to add -tag:v hvc1 after -c:v libx265 to encode more Apple-friendly source video in ffmpeg. This is likely the problem if you can’t use Preview to watch videos on your Mac (yet they play fine with VLC). To easily tag an already-encoded video without a re-encode:
    ffmpeg -i input.mov -c copy -tag:v hvc1 output.mov

  • To learn more about Apple spatial video, watch Deliver video content for spatial experiences from WWDC23. Also check out these spec documents: Apple HEVC Stereo Video Interoperability Profile, ISO Base Media File Format and Apple HEVC Stereo Video, and Video Contour Map Payload Metadata within the QuickTime Movie File Format.

Other Resources

If you’re not a command-line user, don’t have an Apple silicon Mac, or if you just prefer a UI, here are other tools that perform similar tasks:

If you’re a developer who’s wondering how to process MV-HEVC/spatial video on your own:

As always, I’d love your feedback.