One thing you'll need to contend with is making sure both the 2D video and the 3D scene produce a frame in a timely manner. Typically video is kept in compressed form on disk/network and is read a little at a time to decompress each frame and then display it. Likewise, 3D scenes may have what's currently visible in memory, and then load more geometry/textures/etc. from disk/network/etc. when needed. This could cause I/O contention if both geometry and video need to be loaded at the same time. If you can keep much of it in memory, you'll reduce I/O contention between the 2 tasks.
Additionally, video will need to be decompressed usually and that takes up CPU time. If you know your target machine, you may be able to optimize your threading strategy to have 1 (or a few) thread(s) doing video decompression while another thread (or threads) deals with your 3D needs. Furthermore, different codecs have different amounts of CPU requirements. You can experiment with different compression schemes to see which gives you a good trade-off of compressed size vs. decompression cost. Usually more highly compressed video will reduce IO overhead as fewer bytes need to be read from disk or network, but high quality, highly compressed video may be more expensive to decompress.
The best thing to do is come up with an implementation and profile it. Without having any specifics on type of video, complexity of 3D scene, etc., nobody can even begin to guess where your bottlenecks will be. Running real-world tests and measuring is your best bet for figuring this out.