Cannot move view matrix and direction view matrix is pointing in simultaneously

Question

When I move around the scene (without moving my 'head', which is where my view matrix is pointing) everything works just fine. When I look around without moving, everything works just fine. But it is when I combine the two that my head (view matrix) starts going wild. I'm assuming the issue is in my math/logic, but I just cannot figure it out.

Here is my code (simplified, of course):

#define MOVE_DISTANCE 0.15f
#define LOOK_SENSITIVITY 0.01f

static GLFWwindow* window;

static glm::mat4 viewMatrix;
static glm::vec3 look_at;


static void mouseCallback(GLFWwindow* window, double x, double y){
  static double prev_x = 0;
  static double prev_y = 0;
  static float pitch = 0, yaw = 0;

  pitch += (y-prev_y) * LOOK_SENSITIVITY;
  yaw += (x-prev_x) * LOOK_SENSITIVITY;

  if(pitch > 89){
    pitch = 89;
  }
    if(pitch < -89){
      pitch = -89;
  }

  look_at = glm::vec3(cos(yaw)*cos(pitch), sin(pitch), sin(yaw)*cos(pitch) - 1.0f);

  prev_x = x;
  prev_y = y;
}

int main(){
  window = glfwCreateWindow(800, 600, "OpenGL", NULL, NULL); //I left the rest of the initialization out

  glfwSetInputMode(window, GLFW_CURSOR, GLFW_CURSOR_DISABLED);
  glfwSetCursorPosCallback(window, mouseCallback);

  look_at = glm::vec3(0.0f, 0.0f, -1.0f);
  glm::vec3 position(0.0f, 0.0f, 0.0f);

  glm::mat4 projectionMatrix = glm::perspective(45.0f, (float) 800/600, 0.1f, 100.0f); //fov, screen aspect ratio, near render distance, far render distance (z axis)
  glm::mat4 modelMatrix = glm::translate(glm::mat4(1.0f), glm::vec3(0.0f, 0.0f, -5.0f));
  viewMatrix = glm::lookAt(position, look_at, glm::vec3(0, 1, 0));


  while(!glfwWindowShouldClose(window)){
    if(glfwGetKey(window, GLFW_KEY_W) == GLFW_PRESS){
      position.z -= MOVE_DISTANCE;
      look_at.z -= MOVE_DISTANCE;
    }
    if(glfwGetKey(window, GLFW_KEY_S) == GLFW_PRESS){
      position.z += MOVE_DISTANCE;
      look_at.z += MOVE_DISTANCE;
    }
    if(glfwGetKey(window, GLFW_KEY_D) == GLFW_PRESS){
      position.x += MOVE_DISTANCE;
      look_at.x += MOVE_DISTANCE;
    }
    if(glfwGetKey(window, GLFW_KEY_A) == GLFW_PRESS){
      position.x -= MOVE_DISTANCE;
      look_at.x -= MOVE_DISTANCE;
    }
    if(glfwGetKey(window, GLFW_KEY_SPACE) == GLFW_PRESS){ //FLy up
      position.y += MOVE_DISTANCE;
      look_at.y += MOVE_DISTANCE;
    }
    if(glfwGetKey(window, GLFW_KEY_LEFT_CONTROL) == GLFW_PRESS){ //Fly down
      position.y -= MOVE_DISTANCE;
      look_at.y -= MOVE_DISTANCE;
    }

    viewMatrix = glm::lookAt(position, look_at+position, glm::vec3(0, 1, 0));

    glClear(GL_COLOR_BUFFER_BIT);

    glDrawElements(GL_TRIANGLES, NUM_OF_INDICES, GL_UNSIGNED_BYTE, 0);

    glfwSwapBuffers(window);
    glfwPollEvents();
  }
}

Whenever I move around while moving the mouse, things get out of hand and my head (the direction my view matrix is pointing in) starts flicking around. I am aware that the movement isn't impacted by the direction I am looking in. I plan on implementing this sometime in the future.

Could someone help me put my finger on my error?

Answer 1

Your problem is that regardless of where you look at, pressing W will always subtract MOVE_DISTANCE from your world-space position.z coordinate, even though moving "forward" or "backward" would not necessarily mean "moving along the world's Z axis".

So, you must determine the axes of "forward" (in order to move forward and backward) as well as "right" (or "left") and "up" or "down". The latter might not be necessary when you want SPACE and CTRL to always move up/down along the world's Y axis.

For the other two axes, when you have any arbitrary view matrix m (such as your matrix built via glm::lookAt ) then you can easily extract the right , up and forward vectors from it, like so:

#include <glm/gtc/matrix_access.hpp>
...
glm::vec4 right(glm::row(m, 0));
glm::vec4 up(glm::row(m, 1));
glm::vec4 forward(-glm::row(m, 2));

Like @3Dave said above in the comment, you already know where "forward" is, because you are manipulating that vector yourself and computing the lookat transformation from it, so you could omit that.

Now, when you want to move left/right you subtract/add right to your world-space position and likewise for the other axes.

Another thing that is incorrect is that you also add to / subtract from your look_at direction when moving with the keyboard keys. This is wrong, because your look_at vector is a direction and you already use it as a direction vector when you compute the arguments for the glm::lookAt call.

Incorporating all of this into your code (together with some slight refactorings), will lead to following while-loop:

while (!glfwWindowShouldClose(window)) {
  glm::vec3 right(glm::row(viewMatrix, 0)),
            up(glm::row(viewMatrix, 1)),
            forward(-glm::row(viewMatrix, 2));
  glm::vec3 v(0, 0, 0);
  if (glfwGetKey(window, GLFW_KEY_W) == GLFW_PRESS)
    v += forward;
  if (glfwGetKey(window, GLFW_KEY_S) == GLFW_PRESS)
    v -= forward;
  if (glfwGetKey(window, GLFW_KEY_D) == GLFW_PRESS)
    v += right;
  if (glfwGetKey(window, GLFW_KEY_A) == GLFW_PRESS)
    v -= right;
  position += v * MOVE_DISTANCE;
  viewMatrix = glm::lookAt(position, position + look_at, glm::vec3(0, 1, 0));
}