Find chest size using Kinect v2

Question

I need to find out the front measure of chest for any individual using Kinect while facing the camera. My current solution is:

1. When a MultiFrameSource arrives get the color (to display the body in the ui) body (to get the Joints), and bodyIndex frames.

2. copy the BodyIndexFrame to an byte[] _bodyData by using:

    bodyIndexFrame.CopyFrameDataToArray(_bodyData); 

3. I get the Joint objects for: spineShoulder and spineMid. I have assumed that the chest will always be between those points .

4. I convert both Joints to CameraSpacePoint (x,y,z) and from CameraSpacePoint to DepthSpacePoint (x,y) by using

    _sensor.CoordinateMapper.MapCameraPointToDepthSpace(jointPosition); 

I still keep a reference to the z value of spineShoulder.

5. Second assumption => Starting from spineShoulderY to spineMidY I try to find the widest point which is in the player area. in order to do so I will try to find the longest segment between spineShoulderX and the first left region found which does not belong to the player and the longest segment between spineShoulderX and first right side region found which does not belong to the player. Both x segments found must be in the same y coordinate.

    /*************** * Returns the distance between 2 points */ private static int getDistanceToMid(int pointX, int midX) { if (midX > pointX) { return (midX - pointX); } else if (pointX > midX) { return (pointX - midX); } else { return 0; } } /********* * Loops through the bodyData array * It will look for the longest x distance from midX to the last left x value * which still belongs to a player in the y coordinate */ private static int findFarLeftX(byte[] bodyData,int depthWidth, int midX, int y) { int farLeftX = -1; for (int x = midX; x >= 0; --x) { int depthIndex = (y * depthWidth) + x; if (depthIndex > 0 && depthIndex < bodyData.Length) { byte player = bodyData[depthIndex]; if (player != 0xff){ if (farLeftX == -1 || farLeftX > x) { farLeftX = x; } } else{ return farLeftX; } } } return farLeftX; } /********* * Loops through the bodyData array * It will look for the longest x distance from midX to the last right x value * which still belongs to a player in the y coordinate */ private static int findFarRightX(byte[] bodyData, int depthWidth, int midX, int y) { int farRightX = -1; for (int x = midX; x < depthWidth; ++x) { int depthIndex = (y * depthWidth) + x; if (depthIndex > 0 && depthIndex < bodyData.Length) { byte player = bodyData[depthIndex]; if (player != 0xff) { if (farRightX == -1 || farRightX < x) { farRightX = x; } else{ return farRightX; } } } } return farRightX; } private static BodyMember findElement(byte[] bodyData, int depthHeight, int depthWidth, int startX, int startY, int endY) { BodyMember member = new BodyMember(-1, -1, -1, -1); int totalMaxSum = 0; int farLeftX = -1; int farRightX = -1; int selectedY = -1; for (int y = startY; y < depthHeight && y <= endY; ++y) { int leftX = findFarLeftX(bodyData, depthWidth, startX, y); int rightX = findFarRightX(bodyData, depthWidth, startX, y); if (leftX > -1 && rightX > -1) { int leftToMid = getDistanceToMid(leftX, startX); int rightToMid = getDistanceToMid(rightX, startX); int sum = leftToMid + rightToMid; if (sum > totalMaxSum) { totalMaxSum = sum; farLeftX = leftX; farRightX = rightX; selectedY = y; } } } member.setFarLeftX(farLeftX); member.setFarLeftY(selectedY); member.setFarRightX(farRightX); member.setFarRightY(selectedY); return member; } 

6. findElement will return a BodyMember object which contains farLeftX, farRightX, farLeftY and farRightY.

7. I create 2 DepthSpacePoint objects: DepthSpacePoint chestX1 = new DepthSpacePoint(); chestX1.X = bodyMemberObj.getFarLeftX(); chestX1.Y = bodyMemberObj.getFarLeftY();

    DepthSpacePoint chestX2 = new DepthSpacePoint(); chestX2.X = bodyMemberObj.getFarRightX(); chestX2.Y = bodyMemberObj.getFarRightY(); 

8. In order to get real world coordinates in meters these points must be converted to CameraSpacePoint object. In order to do so I will use the joint's z value that I kept a reference to back in point 4.

    CameraSpacePoint chestLeft = _sensor.CoordinateMapper.MapDepthPointToCameraSpace(chestX1,spineShoulderZ); CameraSpacePoint chestRight = _sensor.CoordinateMapper.MapDepthPointToCameraSpace(chestX1,spineShoulderZ); 

9. Now, If my code and assumptions are right I should be able to get the correct distance in meters for the front chest.

    double chestLength = (chestLeft.X > chestRight.X) ? chestLeft - chestRight : chestRight - chestLeft; 

However this does not seem to be returning the correct values. I have been looking into a solution for this during weeks but I seem to be stuck.

Answer 1

I have worked with Kinect V2, and can say that skeleton data alone wild be insufficient to get reliable results. Even clothes has impact how Kinect interprets body parts, so you will have to combine results from other sensors data.

Additionally I suggest you to be creative about how you approach this, for example, you could investigate some possible anatomic correlations about human body, most likely height is a proxy indicator, maybe age from face-recognition and height is another hint, etc.

Answer 2

Are you using using kinect One? Measurements in the old kinects are probably not accurate enough to fit your requirements.

However, from my point of view it would be also worth to do extraction of the moving object inside the frames (if there is only one person in front of the Kinect you will receive this person's contour, but to be sure you can compare its position with Kinect skeleton).

You can tell person to rise their hands for a short period of time. Than use Kinect distance to the persons skeleton and based on the size of the contour and given distance - calculate your final chest measurement just below the shoulders (position taken from the skeleton). I'm also not fully aware how accurately Kinect handles "bigger" people.

Here is described one of the methods: http://onlinelibrary.wiley.com/doi/10.1002/scj.10237/abstract You can also google many other papers with given subject (for free).

What do you think about this idea? Cheers