Picking via DirectX12 Tool Kit

Question

I am facing problems trying to make 3d objects clickable by mouse. For intersection checking I use ray casting. Code I found , ported for my solution:

Exactly picking

bool RaySphereIntersect(Vector3, Vector3, float);

bool TestIntersection(Matrix projectionMatrix, Matrix viewMatrix, Matrix worldMatrix, Vector3 origin, float radius, int m_screenWidth, int m_screenHeight, int mouseX, int mouseY)
{
float pointX, pointY;
Matrix inverseViewMatrix, translateMatrix, inverseWorldMatrix;
Vector3 direction, rayOrigin, rayDirection;

bool intersect, result;

// Move the mouse cursor coordinates into the -1 to +1 range.
pointX = ((2.0f * (float)mouseX) / (float)m_screenWidth) - 1.0f;
pointY = (((2.0f * (float)mouseY) / (float)m_screenHeight) - 1.0f) * -1.0f;

// Adjust the points using the projection matrix to account for the aspect ratio of the viewport.
pointX = pointX / projectionMatrix._11;
pointY = pointY / projectionMatrix._22;

// Get the inverse of the view matrix.
inverseViewMatrix=XMMatrixInverse(NULL, viewMatrix);

// Calculate the direction of the picking ray in view space.
direction.x = (pointX * inverseViewMatrix._11) + (pointY * inverseViewMatrix._21) + inverseViewMatrix._31;
direction.y = (pointX * inverseViewMatrix._12) + (pointY * inverseViewMatrix._22) + inverseViewMatrix._32;
direction.z = (pointX * inverseViewMatrix._13) + (pointY * inverseViewMatrix._23) + inverseViewMatrix._33;

// Get the origin of the picking ray which is the position of the camera.
// Get the world matrix and translate to the location of the sphere.
// Now get the inverse of the translated world matrix.
inverseWorldMatrix= XMMatrixInverse(NULL, worldMatrix);

// Now transform the ray origin and the ray direction from view space to world space.
rayOrigin=XMVector3TransformCoord(origin, inverseWorldMatrix);
rayDirection=XMVector3TransformNormal(direction, inverseWorldMatrix);

// Normalize the ray direction.
rayDirection=XMVector3Normalize(rayDirection);



// Now perform the ray-sphere intersection test.
intersect = RaySphereIntersect(rayOrigin, rayDirection, radius);

if (intersect == true)
 return true;
else
 return false;

}



 bool RaySphereIntersect(Vector3 rayOrigin, Vector3 rayDirection, float radius)
 {
 float a, b, c, discriminant;
 // Calculate the a, b, and c coefficients.
 a = (rayDirection.x * rayDirection.x) + (rayDirection.y * rayDirection.y) + (rayDirection.z * rayDirection.z);
 b = ((rayDirection.x * rayOrigin.x) + (rayDirection.y * rayOrigin.y) + (rayDirection.z * rayOrigin.z)) * 2.0f;
 c = ((rayOrigin.x * rayOrigin.x) + (rayOrigin.y * rayOrigin.y) + (rayOrigin.z * rayOrigin.z)) - (radius * radius);

// Find the discriminant.
discriminant = (b * b) - (4 * a * c);

// if discriminant is negative the picking ray missed the sphere, otherwise it intersected the sphere.
 if (discriminant < 0.0f)
 return false;
 else
 return true;

 }

How do I create sphere

D3DSphere(float x, float y, float z, float radius, float r, float g, float b, float a)  
{
this->x = x;
this->y = y;
this->z = z;

this->radius = radius;

this->shape = GeometricPrimitive::CreateSphere(radius*2.0f);

this->world = Matrix::Identity;
this->world = XMMatrixMultiply(this->world, Matrix::CreateTranslation(x, y, z));
this->index = vsphere;

d3dsphere[vsphere] = this;
vsphere++;
}

How do I call raycaster

void Game::LButtonUp(int x, int y)
{
Vector3 eye(camx, camy, camz);
Vector3 at(Vector3::Zero);
m_view = Matrix::CreateLookAt(eye, at, Vector3::UnitY);
for (int i = 0; i < vsphere; i++)
{
if (TestIntersection(m_projection, m_view, d3dsphere[i]->world, eye, d3dsphere[i]->radius, 800, 600, x, y))
{
MessageBoxW(NULL, L"LOL", L"It works", MB_OK);

break;
}
}

}

Nothing happens by clicking, but if I rotate camera, perpendicularly to XOY, sometimes, clicking near the sphere, message box appears.

---

Update

MessageBox appears independently on camera angle, and it seems, that it detects intersection correctly, but mirrored, relatively to the window center. For example, if sphere is at (0, window.bottom-20) point then I will get MessageBox if I click at (0, 20) point.

What if calculation of the direction of the picking ray is wrong, if it was wrote for left-handed system, and I use right-handed?

Answer 1

Probably, because of the right-handed system, that is used by default in DirectX Tool Kit the next section from caster

pointX = ((2.0f * (float)mouseX) / (float)m_screenWidth) - 1.0f;
pointY = (((2.0f * (float)mouseY) / (float)m_screenHeight) - 1.0f) * -1.0f; 

Should be changed to

pointX = (((2.0f * (float)mouseX) / (float)m_screenWidth) - 1.0f) * -1.0f;
pointY = (((2.0f * (float)mouseY) / (float)m_screenHeight) - 1.0f); 

Important

That code also will work wrong because of depth independence, ie you may select sphere that is situated behind the sphere you clicking. For solve that I changed the code:

float distance3(float x1, float y1, float z1, float x2, float y2, float z2)
{
float dx=x1-x2;
float dy=y1-y2;
float dz=z1-z2;
return sqrt(dx*dx+dy*dy+dz*dz);
}

void Game::LButtonUp(int x, int y)
{
Vector3 eye(camx, camy, camz);
Vector3 at(Vector3::Zero);
m_view = Matrix::CreateLookAt(eye, at, Vector3::UnitY);
int last_index=-1;
float last_distance=99999.0f;//set the obviously highest value, may happen in your scene

for (int i = 0; i < vsphere; i++)
{
if (TestIntersection(m_projection, m_view, d3dsphere[i]->world, eye, d3dsphere[i]->radius, 800, 600, x, y))
{
float distance=distance3(camx,camy,camz, d3dsphere[i]->x, d3dsphere[i]->y, d3dsphere[i]->z);
if(distance<last_distance)
{
last_distance=distance;
last_index=i;
}
}
}

d3dsphere[last_index];//picked sphere 
}