Select Points in Directx11

Question

I'm using DirectX11 and DXUT to show point cloud and now want to interactively select points using mouse movement. Specifically when I drag my mouse to draw a rectangle, points in this rectangle would be selected. My method is first convert mouse position to a box, range(-1,1). Then transform the points to projection space using mvp matrix. At last, compare these points' x and y with the box. If in the box, the point is selected.Here is some segments of my code.

CModelViewerCamera  g_Camera;

XMMATRIX mView = g_Camera.GetViewMatrix();
XMMATRIX mProj = g_Camera.GetProjMatrix();
XMMATRIX mvp= mWorld * mView * mProj;

XMFLOAT4 p_ext( p.x,p.y,p.z, 1 );
XMVECTOR tmp_p = XMLoadFloat4(&p_ext);
tmp_p = XMVector4Transform( tmp_p, XMMatrixTranspose(mvp) );
XMStoreFloat4( &p_ext, tmp_p );
float x = p_ext.x;
float y = p_ext.y;

if ( x > bottom_left.x && x < up_right.x &&
        y > bottom_left.y && y < up_right.y )
{
    //In box, Mark using other color
}

However when I run my code, I find that points transformed to projection space is not in range(-1,1). Can anyone point out my mistake, or just give me another way to achieve the goal. Thank you.

Answer 1

The mvp matrix is supposed to apply the model transformation, the view transformation and the projection transformation in that order. For the mvp matrix to do this it needs to be calculated like this:

mvp = mProj * mView * mWorld

Second why are you transposing the matrix mvp ? DirectX is using row-major matrices and XMMatrix is also row-major so there shouldn't be a reason to transpose the mvp matrix.