Is this Edge of a 3D solid Concave or Convex?

Question

Given a 3D solid model and an edge, I am testing whether that edge is concave or convex.

What is the best way to do this? I'd like to minimize the assumptions regarding the input geometry. My first pass at the problem takes the average of the vertices of the two adjacent faces to generate center points, offsets one of those points by the face normal at that point, and tests whether the offset point is closer or farther to the opposing face than the original. This works for pairs of simple faces, approximately the same size. It fails, for example, with small faces far from the center of larger faces.

I'm doing this in Revit, but I imagine the problem is the same in Rhino, Catia, any solid modeler. From the edge I can extract the adjacent faces. I know the faces are oriented correctly such that the normals point outward. I can project 3D points to the faces, calculate normals of the faces at those points, etc.

Here's the code for the naive version:

  public static Boolean AreFacesConcave(Face face_0, Face face_1, Document doc)
    {
        UV uvMid_0 = VertexAverageUV( face_0 ); //3D average of the vertices
        UV uvMid_1 = VertexAverageUV( face_1 ); // approximates the center of the face

        XYZ pt_0 = face_0.Evaluate(uvMid_0);
        XYZ pt_1 = face_1.Evaluate(uvMid_1);

        // normals at those points
        XYZ normal_0 = face_0.ComputeNormal(uvMid_0);
        XYZ normal_1 = face_1.ComputeNormal(uvMid_1);

        // third point, offset from face 2 by normal
        XYZ pt_2 = pt_1.Add(normal_1.Normalize());


        Double d0 = pt_0.DistanceTo(pt_1);
        Double d1 = pt_0.DistanceTo(pt_2);

        return (d1 < d0);

    }

Answer 1

If you know that normal vectors always are outward, get two points A and B inside two adjacent faces (either mean of three non-collinear vertices or your 'center points').

Then check the sign of dot (scalar) product of vectors AB and nA (normal to the face containing point A).

 Result = DotProduct(AB, nA)

Negative sign denotes 'convex' edge, positive - 'concave' one.

2D example: nA is outward normal, D-edge for CDF is concave, D-edge for CDE is convex