Combing physical LED panels with a virtual set up with real-time lightning to develop the smallest LED stage on earth.
We integrated a real-time video feed into the LED panels, while sending the signal to the virtual production environment.

The custom LED Controller is utilising 3 ESP32, input is taking from Touchdesigner which is being sent to the LED panels via DMX and to Unreal via NDI.
A metahuman character, receiving the realtime lights from NDI inside of Unreal Engine 5.1 is then being projected onto the scene blending in naturally.

With this minified controlled LED mix up, we could experiment with real time light reflections in the real world and in the virtual production.