Be able to add an overlay (a webcam feed with greenscreen processing) to an existing VR capable flight simulator so that the user can see their physical controls and the sim utilizing green screen tech.


I need a way to run a flight simulator (XPlane, MSFSX, etc) within Unreal Engine 4. Any advice/direction is greatly appreciated. Also a solution that doesn't involve using Unreal is also valid, but I prefer to stick with Unreal.

What I Have So Far

I currently have a way to display a webcam stream on a plane surface in Unreal with the green screen processing occurring in a blueprint.

Ideas/Examples That I Would Need Advice On

-Running the flight simulator directly


~Working Example in UE4

-Running the simulator on a monitor and using some form of screen cast software to send to Unreal

-Using an existing free flight simulator UE4 project (not prefered)

  • \$\begingroup\$ Your question is confusing. You talk more about solutions than the actual problem. What are you trying to achieve? (And not how are you trying to achieve it.) \$\endgroup\$ – Vaillancourt Jul 19 '17 at 20:04
  • \$\begingroup\$ @AlexandreVaillancourt I am looking for any advice on how to run a flight simulator with a webcam overlay on it. I tried to orient the post by providing of solutions I had brainstromed but needed help getting started on. \$\endgroup\$ – Collin R. Jul 19 '17 at 20:09
  • \$\begingroup\$ So you want something like a live instructor being seen by a new pilot in VR? \$\endgroup\$ – Vaillancourt Jul 19 '17 at 20:11
  • \$\begingroup\$ Incredibility similar to that. I'm trying to add a pass-through camera ability so that the pilot can see the physical controls in front of them but the sim where the green screen is \$\endgroup\$ – Collin R. Jul 19 '17 at 20:14
  • \$\begingroup\$ Ah, then you should add that to your question, this would make it a bit clearer :) \$\endgroup\$ – Vaillancourt Jul 19 '17 at 20:17

A good streaming software can do what you are describing. Having multiples video inputs and processing green screen filters...

OBS can certainly do that and it's free and open source. You can send the output of OBS to a second monitor (or in your case vr headset) using projector mode:

Right click your preview window -> Projector -> select monitor

OBS website


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.