在Android上通过OpenGL ES显示Blender 3D角色的方法

Question

I'd like to create my personal light-weight Android 3D engine to show 3D animation
configured by Blender.

I have successfully loaded the Blender-created 3D models by OpenGL ES on Android. Now, I'd like to further step into animate the 3D models - character animated by Blender by OpenGL ES on Android.

Below are my steps:
[1]Use Blender 2.69 to create a 3D character and animate it in distinct frame, totally
250 frames, where the 3D character is applied onto with a metarig armature object and
guarantee that each vertex contains at least one vertex group and dedicated weighting.
[2]Modify the ./2.69/scripts/addons/io_scene_obj/export.py and add the necessary
Blender python defined function to export the whole bone system of the armature.
Actually I export the 3D character in .obj format and add my personal defined tag
for the bone system, I name the tag - bonelib. Store it in file *.bones
[3]Modify the ./2.69/scripts/addons/io_scene_obj/export.py so that it can export the
animation in the unit of frame where each frame contains the transformation - transition,
rotation, scale of each distinct bone in the armature. I add my personal defined tag
for the animation frames - framelib. Store it in file *.fms.
[4]I write a java application to translate *.obj, *.bones, *.fms to hex binary format
*.bin, *.frames files and can successfully load the 3D model and restore every detail
pertaining to construct the 3D model by OpenGL ES in Android.
[5]I add some function to calculate the correct transformation matrix of each bone and
apply it onto each vertex with weighting value where the vertex group it is belong to.

After so much effort, I failed to display the correct animation of my 3D character, it
is distorted and could not be told what exactly the 3D model it is.

I list my code in two parts:
[Part-1]Blender python to export 3D model, bone system, animation data

To export armature, I define two below functions:

def matrix_to_string(m, dim=4):
s = ""

for i in range(0, dim):
    for j in range(0, dim):
        s += "%+.2f "%(m[i][j])
return(s)

def export_armature(path_dir, objects):
#Build up the dictionary for the armature parented by all mesh objects
ArmatureDict = {}vertex coordinate

for ob in objects:
    b_export_armature = True

    if ob.parent and (ob.parent_type == 'OBJECT' or ob.parent_type == 'ARMATURE' or ob.parent_type == 'BONE'):
        if ob.parent_type == 'OBJECT':
            if ob.parent.type != 'ARMATURE':
                b_export_armature = False

        if b_export_armature == True:
            p = ArmatureDict.get(ob.parent.name)

            if p is None:
                ArmatureDict[ob.parent.name] = ob.parent

#print("Total %d armatures to be exported\n" %(len(ArmatureDict)))

for key in ArmatureDict.keys():
    a_obj = ArmatureDict.get(key)
    filename = a_obj.name + ".bones"
    bonesfilepath = path_dir + '/' + filename
    file = open(bonesfilepath, "w", encoding="utf8", newline="\n")
    fw = file.write

    #Write Header
    fw('#Blender v%s BONS File: %s\n' %(bpy.app.version_string, filename))
    fw('#Author: mjtsai1974\n')
    fw('\n')

    #Armature architecture portion
    list_bones = a_obj.data.bones[:]

    fw('armature %s\n' %(a_obj.name))

    for bone in list_bones:
        fw('bonechain %s' %(bone.name))
        child_bones = bone.children
        for child in child_bones:
            fw(' %s' %(child.name))
        fw('\n')

    fw('\n')

    #Armature and its bone chain restpose portion
    mat_rot = mathutils.Matrix.Rotation(math.radians(90.0), 4, 'X')
    m = mat_rot * a_obj.matrix_world
    fw('restpose armature %s %s\n' %(a_obj.name, matrix_to_string(m, 4)))

    for bone in list_bones:
        m = mat_rot * bone.matrix_local
        fw('restpose bone %s %s\n' %(bone.name, matrix_to_string(m, 4)))

    file.close()

To export from the graph editor, I define below functions:

def get_bone_action_location(action, bonename, frame=1):
loc = Vector()
if action == None:
    return(loc)
data_path = 'pose.bones["%s"].location'%(bonename)
for fc in action.fcurves:
    if fc.data_path == data_path:
        loc[fc.array_index] = fc.evaluate(frame)
return(loc)

def get_bone_action_rotation(action, bonename, frame=1):
rot = Quaternion( (1, 0, 0, 0) )  #the default quat is not 0
if action == None:
    return(rot)
data_path = 'pose.bones["%s"].rotation_quaternion'%(bonename)
for fc in action.fcurves:
    if fc.data_path == data_path: # and frame > 0 and frame-1 <= len(fc.keyframe_points):
        rot[fc.array_index] = fc.evaluate(frame)
return(rot)

def export_animation_by_armature(filepath, frames, objects):
#Build up the dictionary for the armature parented by all mesh objects
ArmatureDict = {}

mat_rot = mathutils.Matrix.Rotation(math.radians(90.0), 4, 'X')

for ob in objects:
    b_export_armature = True

    if ob.parent and (ob.parent_type == 'OBJECT' or ob.parent_type == 'ARMATURE' or ob.parent_type == 'BONE'):
        if ob.parent_type == 'OBJECT':
            if ob.parent.type != 'ARMATURE':
                b_export_armature = False

        if b_export_armature == True:
            p = ArmatureDict.get(ob.parent.name)

            if p is None:
                ArmatureDict[ob.parent.name] = ob.parent

#print("Total %d armatures to be exported\n" %(len(ArmatureDict)))

path_dir = os.path.dirname(filepath) 

for key in ArmatureDict.keys():
    a_obj = ArmatureDict.get(key)
    filename = a_obj.name + ".fms"
    bonesfilepath = path_dir + '/' + filename
    file = open(bonesfilepath, "w", encoding="utf8", newline="\n")
    fw = file.write

    #Write Header
    fw('#Blender v%s FRAMES File: %s\n' %(bpy.app.version_string, filename))
    fw('#Author: mjtsai1974\n')
    fw('\n')

    #Use armature pose bone chain for 
    list_posebones = a_obj.pose.bones[:]

    fw('animator %s\n' %(a_obj.name))

    for frame in frames:
        bpy.context.scene.frame_set(frame)

        action =a_obj.animation_data.action
        #action = bpy.data.objects[a_obj.name].animation_data.action

        if action == None:
            print("Armature %s has no action" %(a_obj.name))

        fw('frame %d\n' %(frame))

        for bone in list_posebones:
            m = get_bone_action_rotation(action, bone.name, frame) #read the fcurve-animation rotation
            l = get_bone_action_location(action, bone.name, frame) #read the fcurve-animation location

            #m = m.to_matrix().to_4x4()
            #m = mat_rot * m
            q =  Quaternion((m.w, m.x, -m.z, m.y))

            tl = mathutils.Matrix.Translation(l)
            tr = mat_rot * tl
            loc = tr.to_translation()

            fw('bone %s t %+.2f %+.2f %+.2f\n' %(bone.name, loc[0], loc[1], loc[2]))
            fw('bone %s r %+.2f %+.2f %+.2f %+.2f\n' %(bone.name, q.w, q.x, q.y, q.z))
            fw('bone %s s %+.2f %+.2f %+.2f\n' %(bone.name, 1.0, 1.0, 1.0))

        fw('\n')

    file.close()

#[mjtsai@20140517]append the .frames information at the end of .obj file
file = open(filepath, "a+", encoding="utf8", newline="\n")
fw = file.write

for key in ArmatureDict.keys():
    a_obj = ArmatureDict.get(key)
    filename = a_obj.name + ".fms"

    fw('\nframelib %s\n' %(filename))

file.close()

With all above I think that I have correctly export all my bones in the armature
from Blender coordinate system to OpenGL coordinate system. Can anyone point where
exactly possibly the code segment I made mistake?

[Part-2]Android APK to figure out the correct OpenGL coordinate

This is the function I calculate the transformation data of the distinct bine:

public void buildTransformationDataByBoneNameAtFrame(String sz_Name, Armature ar_Obj, FrameLib fl_Obj, int idx_Frame) {
        float [] f_ary_Matrix_Inverted = new float[16];
        float [] f_ary_Matrix_Total = new float[16];
        float [] f_ary_Matrix_Self = new float[16];
        float [] f_ary_Data_Matrix_Parent = null;
        float [] f_ary_Data_Self = null;
        float [] f_ary_Data_Parent = null;
        AnimatorUnit au_Obj = fl_Obj.getAnimatorUnit();
        ArrayList<String> ary_list_Strings = ar_Obj.buildFromRootToBoneByName(sz_Name);
        FrameUnit fu_Obj = au_Obj.getFrameByIndex(idx_Frame);

        if (ary_list_Strings == null) {
            LoggerConfig.Log(String.format("Failed in building array list for Bone[%s] ", sz_Name));

            //new RuntimeException(String.format("Failed in building array list for Bone[%s] ", sz_Name));
            return;
        }

        if (fu_Obj == null) {
            LoggerConfig.Log(String.format("FrameUnit - %d doesn't exist ", idx_Frame));

            return;
        }

        String sz_LastBoneName = ary_list_Strings.get(ary_list_Strings.size() - 1);

        //Suppose the very last one in the array list should be the same bone name to sz_Name
        if (!sz_LastBoneName.equals(sz_Name)) {
            LoggerConfig.Log(String.format("LAST_BONE_NAME[%s] != Bone[%s]", sz_LastBoneName, sz_Name));

            return;
        }

        for (int Index = 0; Index < ary_list_Strings.size(); Index++) {
            String sz_ParentBoneName = "";
            String sz_BoneName = ary_list_Strings.get(Index);
            AnimationUnit amu_Obj = fu_Obj.getAnimationUnitByName(sz_BoneName);
            AnimationUnit amu_ParentObj = null;
            Restpose rp_Obj = ar_Obj.getRestposeByName(sz_BoneName);
            Restpose rp_ParentObj = null;

            if (Index != 0) {
                //This means that it is child bone
                //Get parent bone
                sz_ParentBoneName = ary_list_Strings.get(Index - 1);
                rp_ParentObj = ar_Obj.getRestposeByName(sz_ParentBoneName);
                amu_ParentObj = fu_Obj.getAnimationUnitByName(sz_ParentBoneName);
                f_ary_Data_Matrix_Parent =  amu_ParentObj.getTransformationData();
                f_ary_Data_Parent = rp_ParentObj.getData();  //parent bone's local matrix

                Matrix.invertM(f_ary_Matrix_Inverted, 0, f_ary_Data_Parent, 0);  //parent bone's inverse local matrix

                //Get child bone itself
                f_ary_Data_Self = rp_Obj.getData();  //child bone's local matrix

                //Multiply child bone's local matrix by parent bone's inverse local matrix
                Matrix.multiplyMM(f_ary_Matrix_Self, 0, f_ary_Matrix_Inverted, 0, f_ary_Data_Self, 0);

                //Multiply (child bone's local matrix by parent bone's inverse local matrix) by parent bone's transformation matrix
                Matrix.multiplyMM(f_ary_Matrix_Total, 0, f_ary_Data_Matrix_Parent, 0, f_ary_Matrix_Self, 0);

                amu_Obj.inflateTransformationData();
                amu_Obj.finalizeTransformationData(f_ary_Matrix_Total);
            }   else {
                //This means that it is root bone
                f_ary_Data_Self = rp_Obj.getData();

                amu_Obj.inflateTransformationData();
                amu_Obj.finalizeTransformationData(f_ary_Data_Self);
            }
        }

        //Before we return float array, free the arraylist just returned from ar_Obj.buildFromRootToBoneByName(sz_Name);
        ary_list_Strings.clear();

        //For garbage collection
        ary_list_Strings = null;
        f_ary_Matrix_Inverted = null;
        f_ary_Matrix_Total = null;
        f_ary_Matrix_Self = null;
    }

Below I list the caller code snippet to build the transformation data:
FrameLibInfoWavefrontObjectToBinary framelibInfo = new FrameLibInfoWavefrontObjectToBinary(m_WavefrontObject);

i_ary_Statistics[0] = i_ary_Statistics[1] = 0;

framelibInfo.read(dis, i_ary_Statistics);

FrameLib fl_Obj = m_WavefrontObject.getFrameLib();
AnimatorUnit au_Obj = null;
BoneLib bl_Obj = null;
Armature ar_Obj = null;
ArrayList<BoneChain> bc_Objs = null;
BoneChain bc_Obj = null;
Bone b_Obj = null;
int count_Frames = 0;
String sz_BoneName = ""; 

if (fl_Obj != null) {
    au_Obj = fl_Obj.getAnimatorUnit();

    count_Frames = au_Obj.getFrameCount();

    if (count_Frames > 0) {
        bl_Obj = m_WavefrontObject.getBoneLibs().get(0);  //By default, we have only one bonelib

        ar_Obj = bl_Obj.getArmature();

        bc_Objs = ar_Obj.getBoneChains();

        for (int i_Frame = 0; i_Frame < count_Frames; i_Frame++)  {
            for (int i_BC = 0; i_BC < bc_Objs.size(); i_BC++) {
                bc_Obj = bc_Objs.get(i_BC);

                b_Obj = bc_Obj.getParentBone();

                framelibInfo.buildTransformationDataByBoneNameAtFrame(b_Obj.getName(),  ar_Obj, fl_Obj, i_Frame);
            }
        }
    }
}

framelibInfo.dispose();
framelibInfo = null;

[Question]I could not display the 3D animation I applied by Blender. I don't know where
goes wrong. For the coordinate system transformation issue, I think that I have fully
implemented in the python function. To calculate the correct transformation data for
each bone in distinct frame, I have also restored in the same order the matrix_local
of a bone object and the matrix_world of the armature object are exported.
The algorithm is inspired from below http://blenderartists.org/forum/showthread.php?209221-calculate-bone-location-rotation-from-fcurve-animation-data

I have spend almost one month to evaluate this python script with the rigging applied in
my 3D character and found that the calculated final coordinate of bone fully matches
with the Blender native build-in bone coordinate.

But, why can't I display the animation I applied by Blender in my 3D character in Android
OpenGL ES???

Answer 1

Why create your own game engine when there are several really good, free and open source implementations of this on the web that work well on Android. I highly recommend you take a good look at the jMonkeyEngine SDK or the LWJGL for Java or the PowerVR SDK or Assimp for C++. I use the PowerVR plugin for Blender to export to their POD and Collada formats, which works really well.

Some of these open source solutions would probably give you an idea of what is missing in your code. There is a complete list of game engines for Android here .