NA-MIC Project Weeks

Back to Projects List

That Rendering Thing

I’ve been experimenting with wgpu-py, an implementation of WebGPU that you can pip_install directly into Slicer.

The work in progress in SlicerWGPU, which is now working pretty nicely in demo mode.

Easy to install, can work with any recent Slicer release. Don’t even need to restart Slicer.
Can write shader code easily that works on Vulkan, DirectX, and Metal.
It supports multi-volume rendering with full quality compositing
It can directly render segmentations and provides instant updates in response to edits
It supports interactive rendering of volumes under non-linear transforms
Interactive elements like transform handles and markups are still handled by VTK and seamlessly composited

Currently in demo mode. Some things are integrated to work with the MRML scene while, others work alongside.
The VTK OpenGL render window is tapped into so that most rendering works cleanly, but SlicerWGPU uses a different method to composite multiple translucent surfaces (A-Buffer compared to Depth Peeling) so those aren’t currently combined.
Performance seems about the same as VTK’s GPU renderer, but no extensive testing has been done.

Clone the SlicerWGPU repository
Drag and drop the folder onto a running Slicer (5.10.0 or really any version should work).
Be sure to load the Scene Rendering module (you can ignore the others)
Go to the Scene Rendering module and try the buttons. The top block of buttons (the “VTK Injection” block) are the current demos and the “Dual View” bock are some older experiments.

Give demos to anyone who might be interested and collect feedback
Collect use-cases where existing Slicer rendering doesn’t fully meet requirements but the SlicerWGPU code could
Consider what would be needed for a more complete integration (i.e. possible custom display nodes and dedicated GUI).

Experiment with an updated Volume Rendering module to take advantage of advanced features
Try implementing any new features or use cases people suggest
Think about how to render volumes that are too large for GPU memory (i.e. paging texture data)
Consider how to make this more easily available and useful

(TBD)