Generating smooth normals for terrains in OpenGL

Question

I'm implementing a system to generate terrains using perlin noise. This is how I generate the vertices:

int arrayIdx = 0;

    for(float x = offset.x - CHUNK_WIDTH / 2.0f; x < float(CHUNK_WIDTH) + offset.x - CHUNK_WIDTH / 2.0f; x += TRIANGLE_WIDTH) {
        for(float y = offset.y - CHUNK_WIDTH / 2.0f; y < float(CHUNK_WIDTH) + offset.y - CHUNK_WIDTH / 2.0f; y += TRIANGLE_WIDTH) {
            float height0 = noise->octaveNoise(x + 0.0f * TRIANGLE_WIDTH, y + 0.0f * TRIANGLE_WIDTH),
            height1 = noise->octaveNoise(x + 1.0f * TRIANGLE_WIDTH, y + 0.0f * TRIANGLE_WIDTH),
            height2 = noise->octaveNoise(x + 0.0f * TRIANGLE_WIDTH, y + 1.0f * TRIANGLE_WIDTH),
            height3 = noise->octaveNoise(x + 1.0f * TRIANGLE_WIDTH, y + 1.0f * TRIANGLE_WIDTH);

            mapVertices[arrayIdx + 0] = glm::vec3(x + 0.0f * TRIANGLE_WIDTH, height0, y + 0.0f * TRIANGLE_WIDTH);
            mapVertices[arrayIdx + 1] = glm::vec3(x + 1.0f * TRIANGLE_WIDTH, height1, y + 0.0f * TRIANGLE_WIDTH);
            mapVertices[arrayIdx + 2] = glm::vec3(x + 0.0f * TRIANGLE_WIDTH, height2, y + 1.0f * TRIANGLE_WIDTH);
            mapVertices[arrayIdx + 3] = glm::vec3(x + 1.0f * TRIANGLE_WIDTH, height3, y + 1.0f * TRIANGLE_WIDTH);
            mapVertices[arrayIdx + 4] = glm::vec3(x + 1.0f * TRIANGLE_WIDTH, height1, y + 0.0f * TRIANGLE_WIDTH);
            mapVertices[arrayIdx + 5] = glm::vec3(x + 0.0f * TRIANGLE_WIDTH, height2, y + 1.0f * TRIANGLE_WIDTH);

            mapUVs[arrayIdx + 0] = glm::vec2(x + 0.0f * TRIANGLE_WIDTH, y + 0.0f * TRIANGLE_WIDTH);
            mapUVs[arrayIdx + 1] = glm::vec2(x + 1.0f * TRIANGLE_WIDTH, y + 0.0f * TRIANGLE_WIDTH);
            mapUVs[arrayIdx + 2] = glm::vec2(x + 0.0f * TRIANGLE_WIDTH, y + 1.0f * TRIANGLE_WIDTH);
            mapUVs[arrayIdx + 3] = glm::vec2(x + 1.0f * TRIANGLE_WIDTH, y + 1.0f * TRIANGLE_WIDTH);
            mapUVs[arrayIdx + 4] = glm::vec2(x + 1.0f * TRIANGLE_WIDTH, y + 0.0f * TRIANGLE_WIDTH);
            mapUVs[arrayIdx + 5] = glm::vec2(x + 0.0f * TRIANGLE_WIDTH, y + 1.0f * TRIANGLE_WIDTH);



            glm::vec3 normal0 = -1.0f * glm::triangleNormal(mapVertices[arrayIdx + 0], mapVertices[arrayIdx + 1], mapVertices[arrayIdx + 2]),
            normal1 = +1.0f * glm::triangleNormal(mapVertices[arrayIdx + 3], mapVertices[arrayIdx + 4], mapVertices[arrayIdx + 5]);

            mapNormals[arrayIdx + 0] = normal0;
            mapNormals[arrayIdx + 1] = (normal0 + normal1) / 2.0f;
            mapNormals[arrayIdx + 2] = (normal0 + normal1) / 2.0f;
            mapNormals[arrayIdx + 3] = normal1;
            mapNormals[arrayIdx + 4] = (normal0 + normal1) / 2.0f;
            mapNormals[arrayIdx + 5] = (normal0 + normal1) / 2.0f;


            arrayIdx += 6;
        }
    }

Not using lighting produces these quite smooth results,

The only thing left to do is to generate normals for the triangles, that will make the terrain look smooth. Just using glm::triangleNormal yields this result,

As you can see, lighting really destroys the illusion of a smooth surface.

I tried using an average value of normals on the colliding vertices of the triangles like this:

arrayIdx = 0;

    for(float x = offset.x - CHUNK_WIDTH / 2.0f; x < float(CHUNK_WIDTH) + offset.x - CHUNK_WIDTH / 2.0f; x += TRIANGLE_WIDTH) {
        for(float y = offset.y - CHUNK_WIDTH / 2.0f; y < float(CHUNK_WIDTH) + offset.y - CHUNK_WIDTH / 2.0f; y += TRIANGLE_WIDTH) {

            if((x == offset.x - CHUNK_WIDTH / 2.0f && y == offset.y - CHUNK_WIDTH / 2.0f) ||
               (x == float(CHUNK_WIDTH) + offset.x - CHUNK_WIDTH / 2.0f - TRIANGLE_WIDTH && y == offset.y - CHUNK_WIDTH / 2.0f) ||
               (x == offset.x - CHUNK_WIDTH / 2.0f && y ==  float(CHUNK_WIDTH) + offset.y - CHUNK_WIDTH / 2.0f - TRIANGLE_WIDTH) ||
               (x == float(CHUNK_WIDTH) + offset.x - CHUNK_WIDTH / 2.0f - TRIANGLE_WIDTH && y ==  float(CHUNK_WIDTH) + offset.y - CHUNK_WIDTH / 2.0f - TRIANGLE_WIDTH)) {
                //Special case
            }
            else if(x ==  float(CHUNK_WIDTH) + offset.x - CHUNK_WIDTH / 2.0f - TRIANGLE_WIDTH ||
                    y ==  float(CHUNK_WIDTH) + offset.y - CHUNK_WIDTH / 2.0f - TRIANGLE_WIDTH) {
                 //Special case
            }
            else {
                glm::vec3 averageNormals = (mapNormals[arrayIdx + 3 + 0] + //This triangle
                                            mapNormals[arrayIdx + 0 + int(CHUNK_WIDTH * (1.0f / TRIANGLE_WIDTH)) * 6 + 6] + //Triangle after and this one
                                            mapNormals[arrayIdx + 2 + 6] + //Triangle in the right
                                            mapNormals[arrayIdx + 5 + 6] + //Triangle in the right
                                            mapNormals[arrayIdx + 1 + int(CHUNK_WIDTH * (1.0f / TRIANGLE_WIDTH)) * 6] + //Triangle after this one
                                            mapNormals[arrayIdx + 4 + int(CHUNK_WIDTH * (1.0f / TRIANGLE_WIDTH)) * 6])  //Triangle after this one
                                            / 6.0f;

                mapNormals[arrayIdx + 3 + 0] = averageNormals;
                mapNormals[arrayIdx + 2 + 6] = mapNormals[arrayIdx + 3 + 0];
                mapNormals[arrayIdx + 5 + 6] = mapNormals[arrayIdx + 3 + 0];
                mapNormals[arrayIdx + 1 + int(CHUNK_WIDTH * (1.0f / TRIANGLE_WIDTH)) * 6] = mapNormals[arrayIdx + 3 + 0];
                mapNormals[arrayIdx + 4 + int(CHUNK_WIDTH * (1.0f / TRIANGLE_WIDTH)) * 6] = mapNormals[arrayIdx + 3 + 0];
                mapNormals[arrayIdx + 0 + int(CHUNK_WIDTH * (1.0f / TRIANGLE_WIDTH)) * 6 + 6] = mapNormals[arrayIdx + 3 + 0];
            }

            arrayIdx += 6;
        }
    }

which yielded this result,

but this doesn't look much better.

Using the normals as fragment color give this result:

Rendering the normals as lines yields this, this is before the optimization and with larger triangles, to reduce the number of lines:

This is with my optimization:

Somehow, two normals dont get set.

The blue lines here are the average normals, the green lines are the individual normals before optimizing, they look good:

This is with wireframe:

Maybe some normals aren't set to the average value?

How can I generate normals that are smooth?

Answer 1

The way to compute the per-vertex normal is as follows: You have to consider each polygon that the vertex also belongs to, for instance:

As you mentioned, the problem seems to be related to lighting. Having the proper per-vertex normal computed, you should be also using the correct shading technique, which in this case would be Phong or Gouraud shading instead of Flat shading, as it appears you are using.

Answer 2

I can see the images I requested...

Now look below the terrain if there are not some normals pointing downwords. If yes it means wrong order of multiplication in cross or not the same winding in the terrain geometry.

Then check if your normals are normalized (the same size) but from a quick look it look OK to me.

If you render your averaged normal (with different color) it should be enclosed by the other normals (in the middle of them)

Anyway this does not look right to me:

looks like one of the normal is going in reflected direction. Are you sure you are computing the normal from correct vertexes ?

Your geometry looks like triangulated QUAD grid so you just may be use wrong diagonal ...