c# Smoothing edges while removing background with Kinect V2

Question

I am working on removing the background and leaving only the bodies with Kinect V2 and c#/WPF in real time. Removing the background works fine, but the edges of the bodies are very rough with Jaggies on the edges.

I need to smooth the edges in real-time (30 frames per second). I would appreciate any advice on that.

I am able to select the edges (similar to Photoshop's magic wand). I tried to use something like Gaussian blur, but it seems to be too slow for a real-time application. Probably I am missing something because it seems to be a standard problem for many applications like games etc. Thank you!

Answer 1

You probably need to look into implementations of depth image enhancements or smoothing that fill holes around the edges of the silhouettes. For starters; maybe you can look into Kinect Depth Smoothing .This should work in real time since its just based on calculating modes. For more accurate implementation, there are research papers that address the same issue such as the ones below:

[a] Chen, L., Lin, H.., Li, S., “Depth image enhancement for Kinect using region growing and bilateral filter,” Pattern Recognition (ICPR), 2012 21st International Conference on, 3070–3073 (2012).

[b] Le, AV, Jung, S.-W.., Won, CS, “Directional joint bilateral filter for depth images,” Sensors 14(7), 11362–11378, Multidisciplinary Digital Publishing Institute (2014).