I'm trying to make an augmented reality game for advertisement, using the kinect sdk 1.6 and xna 4.0. I need to use the color image format rgb 1280X960, in spite of the fact that it runs at 12fps, because the quality of the image is important for this project. The problem is that when I call the function to build a color image frame, the fps of the game drops a lot. When I use only skeleton frame information to build a game scene, it runs at 50/60 fps, but when I call the function for the color frame the frames drop to 10/14 fps. The purpose of this function is the transformation of the stream to bgra format. If I comment out this loop, I get 30/40 fps. Is there any other way to make this more efficient?
public Texture2D Video()
{
try
{
if (video == null)
video = new Texture2D(gd, Sensor.ColorStream.FrameWidth, Sensor.ColorStream.FrameHeight);
ColorImageFrame framecolor = Sensor.ColorStream.OpenNextFrame(100);
if (framecolor != null)
{
byte[] bgraPixelData = new byte[framecolor.PixelDataLength];
framecolor.CopyPixelDataTo(bgraPixelData);
//Convert RGBA to BGRA
byte[] colorbuffer = new byte[framecolor.PixelDataLength];
for (int i = 0; i < bgraPixelData.Length; i += 4)
{
colorbuffer[i] = bgraPixelData[i + 2];
colorbuffer[i + 1] = bgraPixelData[i + 1];
colorbuffer[i + 2] = bgraPixelData[i];
colorbuffer[i + 3] = (byte)255; //The video comes with 0 alpha so it is transparent
}
video.SetData(colorbuffer);
}
}
catch
{
video = null;
}
return video;
}
-----How do i ressolve it?
I finally decide to use a shader to make the gpu deal with the process of tranform between formats, as it was suggested by @Seth Battin. this is the shader:
sampler sprite : register(s0);
/*
* Convert from BGRX to RGBA
*/
float4 BGR2RGB(float2 texCoord : TEXCOORD0) : COLOR0
{
float4 tex = tex2D(sprite, texCoord);
return float4(tex.b, tex.g, tex.r, 1.0);
}
technique KinectVideo
{
pass KinectVideo
{
PixelShader = compile ps_2_0 BGR2RGB();
}
}
and the code i used before, as it no longer need to make the transform, now looks like this:
public RenderTarget2D Video()
{
try
{
if (Sensor != null)
{
if (Sensor.ColorStream != null)
{
using (var frame = Sensor.ColorStream.OpenNextFrame(0))
{
if (frame != null)
{
if (buffer == null || buffer.Length != frame.PixelDataLength)
{
buffer = new byte[frame.PixelDataLength];
}
frame.CopyPixelDataTo(buffer);
//GraphicsDevice.Textures[0] = null;
videoframe.SetData<byte>(buffer);
}
}
}
}
}
catch
{
}
return videoframe;
}
I know that the try/catch its expensive, but now i have a good rate of fps, and its woorking fine. After that, i just apply the shader in the draw call, like this:
sb.Begin(SpriteSortMode.Texture, null, null, null, null, kinectcolorvisualizer);
sb.Draw(video, pantalla, null, Color.White, 0, Vector2.Zero, SpriteEffects.None, 1);
sb.End();
where "sb" is spritebatch, "kinectcolorvisualizer" is the shader and "video" is the RenderTarget2D returned by the video function
Thanks to @Seth Battin and to @Mustafa Karakuş for their help.