Filament Renderer — fluxvideotex
fluxvideotex uses Filament
— Google's physically-based real-time rendering engine — as a headless offscreen compositor.
Each incoming RGBA video frame is uploaded as a GPU texture, set as the
baseColorMap of every material instance in the loaded GLB scene,
and the result is rendered and read back as a GStreamer buffer.
The element runs entirely in software: no display server, no window, no GPU display output.
The OpenGL context is headless (Engine::Backend::OPENGL) with an offscreen
SwapChain configured CONFIG_READABLE for DMA readback.
FLUX uses a fork of Filament —
jesusluque/filament
(PR #1,
commit a7b0837)
— which adds Rec.709/Rec.2020/PQ/HLG color spaces and Y'CbCr output to
ColorGrading. These are not in upstream google/filament.
- gltfio UbershaderProvider — pre-built PBR shaders, no
matcstep - Multi-axis rotation animated from buffer PTS via
rotation-period-*props (software only — GLB animation tracks not played) - Async
readPixelswithpumpMessageQueues()spin (macOS) - Vertical flip on readback (Filament is bottom-up; GStreamer top-down)
- Thread-safe owner-thread model — all Filament calls on one dedicated thread
- Live property changes tear down and rebuild the scene cleanly
Color space — color-space property
Filament's ColorGrading post-processing is configured at scene creation time.
The color-space property maps directly to the Filament
(Gamut - TransferFunction - WhitePoint) DSL:
| Value | Filament expression | Use |
|---|---|---|
| srgb (default) | Rec709 - sRGB - D65 | Standard web / display |
| bt709 | Rec709 - BT709 - D65 | HD broadcast OETF |
| rec709-linear | Rec709 - Linear - D65 | Linear light compositing |
| rec2020-linear | Rec2020 - Linear - D65 | Wide-gamut linear |
| rec2020-pq | Rec2020 - PQ - D65 | HDR10 / SMPTE ST.2084 |
| rec2020-hlg | Rec2020 - HLG - D65 | HLG / ARIB STD-B67 |
Y'CbCr output — ycbcr-output property
When ycbcr-output=true, Filament's ColorGrading::ycbcrOutput()
encodes the colour-graded result as packed Y'CbCr into the RGBA8 readback buffer
(R=Y', G=Cb, B=Cr, A=1).
The element advertises AYUV on its src pad so downstream encoders and
muxers receive the correct signal with no extra videoconvert step.
Custom GLB — glb-file property
The default scene is a unit cube with KHR_materials_unlit and
baseColorTexture.uri = "flux://channel/0", generated at build time
by gen_cube.py and embedded as a C byte array via xxd -i.
Set glb-file to any GLB path to use a different mesh — any material
whose baseColorTexture URI is flux://channel/0 receives
the live video feed; other materials are untouched.
GLB animation tracks are not played. The gltfio animator
is not invoked — all motion comes from the software rotation driven by
rotation-period-*. The GLB provides mesh geometry and materials only.
Run with gst-launch-1.0
// Built-in cube, SDR, default rotation GST_PLUGIN_PATH=/path/to/flux/tools/gstreamer/target/release:\ /path/to/flux/tools/filament/build/gst-fluxvideotex \ gst-launch-1.0 \ videotestsrc pattern=smpte is-live=true \ ! videoconvert \ ! "video/x-raw,format=RGBA,width=1280,height=720,framerate=30/1" \ ! fluxvideotex width=1280 height=720 \ color-space=srgb ycbcr-output=false \ ! "video/x-raw,format=RGBA,width=1280,height=720" \ ! videoconvert \ ! glimagesink sync=false // Custom GLB mesh, fast rotation, HLG Y'CbCr ! fluxvideotex width=1280 height=720 \ glb-file=/path/to/scene.glb \ rotation-period-x=10 rotation-period-y=15 rotation-period-z=20 \ color-space=rec2020-hlg ycbcr-output=true \ ! "video/x-raw,format=AYUV,width=1280,height=720" \ ! videoconvert ! ...