I want to play an RTSP video stream on a Texture2D. The stream is coming from an IP camera and is encoded with H.264, MJPEG or MPEG4.

I know about VLC's libraries, ffmpeg, direct show, media foundation and such, but how do I connect them to Mono develop?


Well this is not a minor task. Basically you need to build a streaming server that supports RTP and also RTSP if you want playback options, or use an existing streaming solution; ffmpeg has one, but unfortunately it doesn't work on Windows. If you plan to use ffmpeg you also need to make Unity connect with native code dynamic libraries which I am not sure if all versions support it.

Building a streaming server needs a lot of work, but it is not really hard. You need to start by understand RTP which is used for data transfer and RTSP which is used for negotiation and playback options.

You can read the packets via sockets, sniffing, or if the camera SDK gives you the packets. Once you get the packets, often given as a blob of char* and a length buffer, you need to start by segmenting the packet into UDP, RTP/RTSP and RTP payload. Once you extract the RTP payload you can pass it into an ffmpeg decoder which will return a decoded image usually encoded as RGBA once you get there you need to march your image data into Unity's managed code and pass it to a texture.

Well if that sounds too much work, there is an open source library that do that for you, it's called live555 note that I don't know much about it but it can get the job done.


There is also net7mma.

It has a c# rtsp client and server and also has implementations of rtp and rtcp.

There are sdp classes and stream classes for dealing with certain types of packetization and depacketization.

The library also works in mono and has no external dependencies.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.