对于另一个模块中定义的符号，insmod 失败并显示“模块中的未知符号”

Question

I am working in Ubuntu. I am trying to make two kernel modules which uses each other functions. My problem is that I got modules properly compiled, but the symbol is not resolved for one of them.

To make things simple, let's call these modules as m1 and m2 .

m2 is exporting function void func_m2(void) . The m1 is calling this function. Both modules properly compile.

After it all compiles, I need to load first the m2 module (because it has exported func_m2 function) and afterwards m1 module. So, let's make it:

volodymyr@sv1:~/development/kmodules/m2$ sudo insmod ./m2.ko

Now, lets load m1 module which is trying to use func_m2 :

volodymyr@sv1:~/development/kmodules/m1$ sudo insmod ./m1.ko
insmod: error inserting './m1.ko': -1 Unknown symbol in module

Following is what I see in logs:

volodymyr@sv1:~/development/kmodules/m1$ dmesg | tail
[ 3938.166616] Loading m2 module ...
[ 3963.078055] m1: no symbol version for func_m2
[ 3963.078059] m1: Unknown symbol func_m2

So, it seems like the references to symbol func_m2 is not resolved. Interesting. Let's check if it is present in symbol table:

volodymyr@sv1:~/development/kmodules$ cat /proc/kallsyms | grep 'func_m2'
ffffffffa00530d0 r __ksymtab_func_m2    [m2]
ffffffffa00530e8 r __kstrtab_func_m2    [m2]
ffffffffa00530e0 r __kcrctab_func_m2    [m2]
ffffffffa0053000 T func_m2      [m2]
000000004edd543f a __crc_func_m2        [m2]

As you can see, the func_m2 is actually present in symbol table. So why m1 can't be loaded?

I have installed properly Linux headers for my kernel and Linux sources. I did not make any modifications in kernel, it is untouched, and it's version is: 2.6.31-16-generic (I run x64)

Now, to show you full picture I am putting here the source code and Makefile I used for this test for both m1 and m2 modules.

m1 module:

m1.c:

#include <linux/module.h>
#include <linux/kernel.h>

extern void func_m2(void);

int hello_start(void)
{
    printk(KERN_INFO "Loading m1 module ...\n");

    func_m2();

    return 0;
 }

 void hello_end(void)
 {
    printk(KERN_INFO "Unloading m1 ...\n");
 }

module_init(hello_start);
module_exit(hello_end);

MODULE_LICENSE("GPL");

Makefile:

obj-m := m1.o

all:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

clean:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

m2 module:

m2.c:

#include <linux/module.h>
#include <linux/kernel.h>

int hello_start(void)
{
    printk(KERN_INFO "Loading m2 module ...\n");

    return 0;
}

void hello_end(void)
{
    printk(KERN_INFO "Unloading m2 ...\n");
}

void func_m2(void)
{
    printk(KERN_INFO "This a function in m2\n");
}

module_init(hello_start);
module_exit(hello_end);

MODULE_LICENSE("GPL");
EXPORT_SYMBOL(func_m2);

Makefile:

obj-m := m2.o
export-objs := m2.o

all:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

clean:
    make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

Basically my question is: Why can't m1 be loaded?

It would be helpful if someone could answer.

Answer 1

When you build m2, it creates a Module.symvers file.

Copy this file to where you are building m1. Then make m1, and insmod it.

You probably had a warning when you were building m1 before, something like:

WARNING: "func_m2" [/tmp/m1/m1.ko] undefined!

This should disappear once you use the Module.symvers from the m2 module.

From http://www.kernel.org/doc/Documentation/kbuild/modules.txt :

--- 6.2 Symbols and External Modules

When building an external module, the build system needs access to the symbols from the kernel to check if all external symbols are defined. This is done in the MODPOST step. modpost obtains the symbols by reading Module.symvers from the kernel source tree. If a Module.symvers file is present in the directory where the external module is being built, this file will be read too. During the MODPOST step, a new Module.symvers file will be written containing all exported symbols that were not defined in the kernel.

And this is also worth reading, from the same file:

--- 6.3 Symbols From Another External Module

Sometimes, an external module uses exported symbols from another external module. kbuild needs to have full knowledge of all symbols to avoid spitting out warnings about undefined symbols. Three solutions exist for this situation.

NOTE: The method with a top-level kbuild file is recommended but may be impractical in certain situations.

Use a top-level kbuild file If you have two modules, foo.ko and bar.ko, where foo.ko needs symbols from bar.ko, you can use a common top-level kbuild file so both modules are compiled in the same build. Consider the following directory layout:

./foo/ <= contains foo.ko ./bar/ <= contains bar.ko

The top-level kbuild file would then look like:

$ ./Kbuild (or ./Makefile): obj-y := foo/ bar/

And executing

$ make -C $KDIR M=$PWD

will then do the expected and compile both modules with full knowledge of symbols from either module.

Use an extra Module.symvers file When an external module is built, a Module.symvers file is generated containing all exported symbols which are not defined in the kernel. To get access to symbols from bar.ko, copy the Module.symvers file from the compilation of bar.ko to the directory where foo.ko is built. During the module build, kbuild will read the Module.symvers file in the directory of the external module, and when the build is finished, a new Module.symvers file is created containing the sum of all symbols defined and not part of the kernel.

Use "make" variable KBUILD_EXTRA_SYMBOLS If it is impractical to copy Module.symvers from another module, you can assign a space separated list of files to KBUILD_EXTRA_SYMBOLS in your build file. These files will be loaded by modpost during the initialization of its symbol tables.

Answer 2

Here are some issues I found with your code:

(a). Your initialization and termination functions should be declared static and properly identified. For example, in m1.c -

static int __init hello_start(void)
{
     printk(KERN_INFO "Loading m1 module ...\n");

    func_m2();

    return 0;
}

static void __exit hello_end(void)
{
    printk(KERN_INFO "Unloading m1 ...\n");
}

Repeat this for m2.c

(b). Build both of your modules together, using the same Makefile. I bet if you look closely at the output from your existing Makefile for m1.c, you will see a warning indicating that func_m2() is undefined. Anyhow, the consolidated Makefile should look like -

SRCS   = m1.c m2.c
OBJS   = $(SRCS:.c=.o)

obj-m += $(OBJS)

EXTRA_CFLAGS = -O2


all:
    $(MAKE) -C /lib/modules/`uname -r`/build M=$(PWD) modules

clean:
    $(MAKE) -C /lib/modules/`uname -r`/build M=$(PWD) clean
    $(RM) Module.markers modules.order

After both modules are built, run insmod on 'm2.ko' before issuing the insmod for 'm1.ko'. Check results via dmesg.

Also, over here I am assuming that both m1.c and m2.c are in the same directory. Even if they are in different directories, this technique will work, but it will be messy. If they are in different directories, do the following.

I did little research and found a way to build modules in separate directories. The example I used is much simpler than what you have, but perhaps it is adaptable.

I have following manifest of files in a directory called ExportSymbol ...

$ ls -CFR
.:
include/  Makefile  mod1/  mod2/

./include:
m2_func.h

./mod1:
Makefile  module1.c

./mod2:
Makefile  module2.c

The m2_func.h appears as:

#ifndef M2_FUNC_H
#define M2_FUNC_H

void m2_func(void);

#endif

The top-level Makefile appears as:

obj-y := mod1/ mod2/

all:
    $(MAKE) -C /lib/modules/`uname -r`/build M=$(PWD) modules

clean:
    $(MAKE) -C /lib/modules/`uname -r`/build M=$(PWD) clean
    $(RM) Module.markers modules.order

The Makefile and module1.c, which are in mod1/, appear as:

SRCS   = module1.c
OBJS   = $(SRCS:.c=.o)

obj-m += $(OBJS)

EXTRA_CFLAGS += -I${PWD}/include

all:
    $(MAKE) -C /lib/modules/`uname -r`/build M=$(PWD) modules

clean:
    $(MAKE) -C /lib/modules/`uname -r`/build M=$(PWD) clean
    $(RM) Module.markers modules.order

---

#include <linux/module.h>
#include <linux/kernel.h>

static int __init hello_start(void)
{
 printk(KERN_INFO "Loading m1 module ...\n");

 m2_func();

 return 0;
}

static void __exit hello_end(void)
{
 printk(KERN_INFO "Unloading m1 ...\n");
}

module_init(hello_start);
module_exit(hello_end);

MODULE_LICENSE("GPL");

The Makefile and module2.c, which are in mod2/, appear as:

SRCS   = module2.c
OBJS   = $(SRCS:.c=.o)

obj-m += $(OBJS)

EXTRA_CFLAGS += -I${PWD}/include

all:
    $(MAKE) -C /lib/modules/`uname -r`/build M=$(PWD) modules

clean:
    $(MAKE) -C /lib/modules/`uname -r`/build M=$(PWD) clean
    $(RM) Module.markers modules.order

---

#include "m2_func.h"
#include <linux/module.h>
#include <linux/kernel.h>

static int __init hello_start(void)
{
 printk(KERN_INFO "Loading m2 module ...\n");

 return 0;
}

static void __exit hello_end(void)
{
 printk(KERN_INFO "Unloading m2 ...\n");
}

void m2_func(void)
{
 printk(KERN_INFO "This a function in m2\n");
} 

module_init(hello_start);
module_exit(hello_end);

MODULE_LICENSE("GPL");
EXPORT_SYMBOL(m2_func);

NOTE: I can't use your makefile as it generates *.ko per each c file. The Makefile is doing its job. A 'ko' file is a kernel object file; you will have one for each .c source file. There's no way around this. If you do not want multiple ko-files, then put all of your code in one source file.

Answer 3

The easiest way is to create a Makefile for two modules at the same time

example :

 obj-m += hello.o
 obj-m += world.o
 
 all:
      make -C /lib/modules/`uname -r`/build M=$(PWD) modules
install: 
      make -C /lib/modules/`uname -r`/build M=$(PWD) modules_install
clean:
      make -C /lib/modules/`uname -r`/build M=$(PWD) clean

First Module hello.c

#inluce  <linux/module.h>
#include <linux/init.h>

int hello_print(){
 printk("<0> Hello -> %s : %d",__FUNCTION__,__LINE__);
return 0;
}
EXPORT_SYMBOL(hello_print),

static int __init open(void)
{
    printk("<0> Module Hello Start!");
    return 0;
}

static void __exit close(void) {
    printk("<0> Module Hello Stop!");
}

module_init(open);
module_exit(close);
MODULE_LICENSE("GPL v2");

Second Module world.c

#inluce  <linux/module.h>
#include <linux/init.h>

extern int hello_print();

static int __init open(void)
{
    printk("<0> Init World Module");
    printk("<0> Call hello_print() from Hello Module");
    hello_print();
    return 0;
}

static void __exit close(void) {
    printk("<0> Module Hello Stop!");
}

module_init(open);
module_exit(close);
MODULE_LICENSE("GPL v2");

then

$ make 
$ insmod hello.ko
$ insmod world.ko