Skip to content
Snippets Groups Projects

Qt Quick 3D (Xr Module)

The purpose of this module is to demonstrate the usage of Qt Quick 3D with OpenXR.

OpenXR dependency setup (Windows)

Download this repository: https://github.com/KhronosGroup/OpenXR-SDK-Source

Build and install somewhere, for example I installed mine to:

D:/SDK/OPENXR

To build, create e.g. a build directory and do something like

cmake -GNinja -DCMAKE_INSTALL_PREFIX=d:/sdk/openxr -DCMAKE_BUILD_TYPE=Release ..

followed by

ninja install

Then for CMake to be able to find the SDK with findPackage you need to setup and environment variable (replacing the path with your installation prefix)

OPENXR_ROOT=D:/SDK/OPENXR

Then you should be able to build this module.

Note that on Windows the build type is of significance since the SDK builds a static library which is then linked to the Qt libraries. The above is sufficient for a Release or RelWithDebInfo build of Qt Quick 3D XR, but for Debug the OpenXR library must also be built with build type Debug. The debug build results in openxr_loaderd.lib (with a 'd' suffix), this is also what gets picked up by the SDK's CMake integration.

Using the Meta XR Simulator (Windows)

Base rendering/input functionality and passthrough simulation has been confirmed to work with v57, 59, and 60.

  • Download from https://developer.oculus.com/downloads/package/meta-xr-simulator and extract just the MetaXRSimulator directory somewhere, e.g. c:\MetaXRSimulator.

  • Build and install the loader and the Khronos openxr headers, set OPENXR_ROOT as described above.

  • Now, to use the Simulator, set the environment variable XR_RUNTIME_JSON=c:\MetaXRSimulator\meta_openxr_simulator.json

  • Alternatively, the setting can be made permanent via the registry. Check/change the value of ActiveRuntime under Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\OpenXR\1

The Simulator supports D3D11, D3D12, and Vulkan. Support for all these, incl. D3D12 is now present in Qt Quick 3D XR as well. Use QSG_RHI_BACKEND (or QQuickWindow::setGraphicsApi()) to change the QRhi backend Qt Quick uses. In practice D3D11/12 seem to work the best. E.g. when simulating passthrough, Vulkan seems to be a bit broken. (even after switching ses_texture_format from "gpu_handle" to "jpg", see docs)

For passthrough, launch the Simulator's bundled synth_env_server from MetaXRSimulator/.synth_env_server and leave it running. Applications attempting to use passthrough will not work without this. See the XR Simulator docs for details. https://developer.oculus.com/documentation/native/xrsim-native/

Input simulation seems to work well. Use [ and ] to toggle what to control (head, left controller, right controller). See the Simulator UI for mouse/keyboard/controller bindings.

Targeting Android with Quest 3 (using a Windows or macOS host)

Download the Oculus OpenXR Mobile SDK https://developer.oculus.com/downloads/package/oculus-openxr-mobile-sdk

Extract it somewhere and set the OCULUS_OPENXR_MOBILE_SDK environment variable to point to it.

This includes the OpenXR loader, so the OpenXR dependency setup step described above is not needed when building for Android.

The module and examples can now be built against an Android build of Qt.

One may need to target android-31 (to be checked if this is relevant, Qt dev branch currently defaults to 33) Both recent Android NDKs (25.2.9519653 and 26.1.10909125 at the time of writing) have been seen working.

It can be highly useful to install the Quest Developer Hub https://developer.oculus.com/downloads/package/oculus-developer-hub-win/ as well. This allows installing, starting, and stopping applications (from .apk). During our testing ADB over Wifi seemed unstable. Prefer using a USB connection for adb and MQDH.

To launch from the Application List on the device, switch to Unknown Sources to see the Qt applications installed for development purposes.

When creating new applications, check the examples' CMakeLists.txt and the custom Android manifest (adapt the names and labels as necessary). Calling qt_xr_setup() and using the custom manifest is mandatory for Android builds.

Both OpenGL ES and Vulkan are supported. OpenGL is the default with Qt on Android. Use QSG_RHI_BACKEND (or QQuickWindow::setGraphicsApi()) to request using Vulkan.

Running on Windows while streaming to a Quest 3 headset with Quest Link, Air Link, Steam Link, or Virtual Desktop

Same steps as for Windows and the Simulator described above. The same (regular Windows) build of Qt and applications can be used as with the Simulator. The only difference is the OpenXR runtime used.

  • If using Link (via cable or Air Link), check the Oculus application's settings (Settings->General->OpenXR Runtime) to ensure the Oculus OpenXR runtime is set as default in the registry. If you are also using the Simulator, make sure XR_RUNTIME_JSON is not set at this point. Then launch Link on the headset. Once the Link UI is up, launch a Qt application.

  • If using Virtual Desktop 1.29.7 or newer, use the VD streamer app's settings to switch the OpenXR runtime (SteamVR, VDXR). This has been confirmed to work with both runtimes using VD 1.29.9. See https://github.com/mbucchia/VirtualDesktop-OpenXR/wiki for more information on this and the different OpenXR runtimes.

Note that these runtimes may not provide a number of OpenXR extensions you'd normally have when running natively on Android on the Quest, or if they do, they may be limited in various ways. (esp. for MR, think passthrough, anchors, etc.; the Oculus OpenXR runtime does actually support passthrough, but the performance can be quite bad when streaming via Air Link)

  • During our testing the Quest Link seemed highly unstable, with issues ranging from hating USB 3 ports to failing to bring up the Link UI, although Air Link at least seems to be improved in version 59 of the system software.

  • Virtual Desktop works very well. Note that VD switches the OpenXR runtime in the registry automatically based on the streamer app's settings. (to SteamVR or VDXR)

  • For SteamVR, using the recently released Steam Link (VR) is also an option.

  • Using SteamVR has the downside of auto-launching the SteamVR UI and treating the Qt application as a game. (although this can be handy e.g. for quitting the application from the Steam menu) On the upside all the tools from that ecosystem (e.g. fpsVR and co.) are available to use in Qt apps as well.

  • VDXR provides a basic but pretty good experience since one can effectively run/stop/change/rebuild applications just like one would with non-VR fullscreen applications on the PC screen. (because running an app takes over the headset "screen", but quitting or Control+C for console apps gives back the VD remote display right away, which is very handy)

Support for the above streaming solutions and OpenXR runtimes is provided on a best effort basis since Qt cannot commit to offer guaranteed support for all the possible 'streaming tech + OpenXR runtime + 3D API' permutations.

For reference, with the default installation locations the OpenXR runtime JSON files are at:

  • C:\Program Files\Oculus\Support\oculus-runtime\oculus_openxr_64.json
  • C:\Program Files (x86)\Steam\steamapps\common\SteamVR\steamxr_win64.json
  • C:\Program Files\Virtual Desktop Streamer\OpenXR\virtualdesktop-openxr.json

The Oculus and VD applications alter the value of ActiveRuntime under Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\OpenXR\1. For development purposes, it is often more convenient to use the XR_RUNTIME_JSON environment variable to quickly change which runtime is used.

Support for 3D APIs (and the related bugs and quirks) may vary depending on the OpenXR runtime. On Windows, prefer using D3D12 or Vulkan as these tend to work best. Use QSG_RHI_BACKEND (or QQuickWindow::setGraphicsApi()) to request something other than the default D3D11.

Troubleshooting

Always check the logs printed on the debug output. When running non-console Windows applications, use the DebugView tool, or Creator's application output pane, or something similar in the IDE of your choice. With Android, adb logcat.

Both Qt Quick 3D XR and the OpenXR runtimes (and sometimes the streaming solution such as Quest Link) print a lot of stuff there, and without these logs it is impossible to know what is going on when something fails.

When streaming from Windows, setting QT_WIN_DEBUG_CONSOLE to 'new' or 'attach' can be very handy since being able to Ctrl+C is very useful in many cases (as there is no window to close, and Qt apps and examples are typically built as non-console by default).

Legacy: Getting OpenXR to work with an Oculus HMD (not Quest 3)

The OpenXR SDK just provides a loader, but individual XR device providers provide the actual OpenXR runtime that is loaded. In the case of Oculus HMD(s) there isn't yet support for OpenXR out of the box. So you need to do a bit of setup first.

First create a file called openxr.json somewhere on your system. I put mine in D:/SDK/openxr.json The add the following content:

{
    "file_format_version": "1.0.0",
    "runtime":
    {
        "api_version": "1.0",
        "name": "Oculus OpenXR",
        "library_path": "c:\\Program Files\\Oculus\\Support\\oculus-runtime\\LibOVRRT64_1.dll"
    }
}

If you installed the Oculus software somewhere other than the default you may need to adjust the library_path value.

Then you need to add a registry values in the following location: Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Khronos\OpenXR\1 Inside this area add a REG_SZ value called ActiveRuntime with the value of the location of the openxr.json file you created. This will enable the use of OpenXR with the Oculus runtime.