Foveated rendering and foveated streaming on Apple Vision Pro

Two independent ways to spend pixels where they matter — Varjo Native Foveated Rendering in Autodesk VRED cuts the workstation's rendering load, and Apple's foveated streaming sends the sharpest part of the stream where the wearer is looking.

Foveation uses the fact that the eye only sees sharply in a small area around where it is looking. Innoactive Spatial can use it at two different points in the pipeline, and the two are independent of each other:

Varjo Native Foveated RenderingApple foveated streaming
Where it runsIn Autodesk VRED, on the workstationIn the stream between the workstation and the headset
What it savesGPU time: the periphery is rendered at lower resolutionStream quality budget: the area you look at gets the most detail
Who it applies toVRED onlyAny application streamed with Innoactive Spatial
Where you turn it onVRED preferencesInnoactive Spatial app settings on the Vision Pro

Use the first when the workstation cannot hold the frame rate. Use the second to get the sharpest image where the wearer is looking. You can use both at the same time.

Varjo Native Foveated Rendering in VRED

VRED can render the centre of the view at full resolution and the periphery at a lower one. On scenes where the GPU is the bottleneck — raytracing, dense geometry — this frees up a large part of the frame time.

In an internal test with the Genesis sample scene on a ThinkPad P16 (RTX 3500 Ada) with raytraced reflections, the streamed frame rate went from 5 FPS to 22.5 FPS with Native Foveated Rendering on. Results depend heavily on the scene and the workstation: simple scenes that are not GPU-bound gain little.

Turn it on

Requires Autodesk VRED 2027.2 or later. On earlier versions, keep it off — it produces a visible square or stripe artifact in the headset (see Lessons learnt).

  1. In VRED, open Preferences → Extended Reality → XR Options → Varjo.
  2. Enable Native Foveated Rendering.
  3. Restart the streaming session.

The periphery is rendered at lower resolution, so look at the edges of the view on your own content and decide whether the trade-off works for your review.

Apple foveated streaming on the Vision Pro

The Innoactive Spatial app can stream through Apple's foveated streaming framework. With it, the headset shares limited information about where the wearer is looking with the streaming session, and the stream concentrates its detail there. When the session starts, visionOS asks the wearer to allow this.

When Apple's framework is not used, the stream can still use static foveated streaming: the centre of the view gets the most detail, independent of where the wearer looks.

Turn it on

On the Apple Vision Pro, open Innoactive Spatial and go to Settings → CloudXR:

  • Use Apple Foveated Streaming Framework — on by default. The app uses it whenever the workstation's Spatial Runtime supports it.
  • Static Foveated Streaming — on by default. Shown only while Use Apple Foveated Streaming Framework is off. Leave it on: turning it off can make the image look softer or more pixelated.

Changes apply from the next session.

Before you use it

  • Firewall. Apple's foveated streaming uses an extra port on the workstation for session management (TCP 55000). See Ports to whitelist.
  • Pairing. The first connection with Apple's framework is paired by scanning a QR code shown on the workstation.
  • Eye calibration. Because the stream follows the wearer's gaze, an eye calibration made by someone else — or made while wearing glasses — puts the sharp area in the wrong place. On a shared headset, calibrate without glasses; see Lessons learnt.