Helping with Linux kernel dump crash: Unable to handle kernel NULL pointer dereference at virtual address 00000001

Question

I'm a newbie in Linux driver Field.Now, I has just done with writing Linux driver and I'm testing the accuracy of the function of this driver. The goal of my driver is using FPGA card with CPU ARMv7 through PCIe communication and doing both encryption and decryption in this card. When I test my kernel module, I see this kernel panic. In my test case, I send multiple packets continuously to FPGA card to implement encryption/decryption. However, after doing several packets well, Linux kernel was crash. At first, I think that this kernel crash bug is related to allocating kernel memory and freeing this memory (kzalloc function and kfree function). May be memory allocated for some pointer does not be freed immediately. Can anyone suggests the cause and the solution for this kernel panic?

[  532.593938] Unable to handle kernel NULL pointer dereference at virtual address 00000001
[  532.602069] pgd = ecb8c000

[  532.604780] [00000001] *pgd=2ca83831, *pte=00000000, *ppte=00000000
 [532.611066] Internal error: Oops: 17 [#1] SMP ARM
[  532.615777] Modules linked in: testcrypto(+) huy_crypto xdma ath9k ath9k_common pppoe ppp_async ath9k_hw ath10k_pci ath10k_core ath pppox ppp_generic nf_conntrack_ipv6 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_recent xt_policy xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_helper xt_esp xt_ecn xt_dscp xt_conntrack xt_connmark xt_connlimit xt_connbytes xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CT xt_CLASSIFY slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache iptable_raw iptable_mangle iptable_filter ipt_ah ipt_ECN ip_tables crc_ccitt compat sch_cake
[  532.686456]  act_connmark nf_conntrack sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_tcindex cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred cryptodev nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 ifb ip6_vti ip_vti xfrm6_mode_tunnel xfrm6_mode_transport xfrm6_mode_beet ipcomp6 xfrm6_tunnel esp6 ah6 xfrm4_tunnel xfrm4_mode_tunnel xfrm4_mode_transport xfrm4_mode_beet ipcomp esp4 ah4 ip6_tunnel tunnel6 tunnel4 ip_tunnel mpls_iptunnel mpls_router mpls_gso xfrm_user xfrm_ipcomp af_key xfrm_algo algif_skcipher algif_hash af_alg sha512_generic md5 echainiv cbc authenc gpio_button_hotplug [last unloaded: testcrypto]
[  532.748684] CPU: 0 PID: 5563 Comm: insmod Not tainted 4.14.176 #0
[  532.754789] Hardware name: Marvell Armada 380/385 (Device Tree)
[  532.760721] task: ef3bde00 task.stack: ef182000
[  532.765263] PC is at __kmalloc_track_caller+0x100/0x144
[  532.770499] LR is at 0x89a5
[  532.773297] pc : [<c01ee8b4>]    lr : [<000089a5>]    psr: 20000013
[  532.779576] sp : ef183d98  ip : a0000013  fp : ffffee4b
[  532.784811] r10: 00000009  r9 : 00008124  r8 : 00000002
[  532.790046] r7 : 00000001  r6 : ecfe2a40  r5 : 014000c0  r4 : ef001e40
[  532.796586] r3 : 00000000  r2 : ef7d6a34  r1 : 2ee8d000  r0 : 000089a6
[  532.803128] Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[  532.810278] Control: 10c5387d  Table: 2cb8c04a  DAC: 00000051
[  532.816035] Process insmod (pid: 5563, stack limit = 0xef182210)
[  532.822054] Stack: (0xef183d98 to 0xef184000)
[  532.826419] 3d80:                                                       00000008 c024eb40
[  532.834616] 3da0: ecfe24c0 00000124 00000002 c01c73b8 ec94b540 ef0181c0 ec94b540 c024eb40
[  532.842812] 3dc0: 00000000 00000124 bf6780c8 ec94b540 c07088c8 ec94b540 00000124 bf6780c8
[  532.851008] 3de0: 00000000 c024fcc8 00000000 c07088c8 ec94b540 c025173c 00000000 edfed95c
[  532.859204] 3e00: ec94b540 c025207c 00001000 00000000 c07088c8 edfed95c 00000000 00000000
[  532.867401] 3e20: edfed800 edfeda58 ec94b540 c0252a5c 00000000 00000000 c07e21ec c07e21d8
[  532.875597] 3e40: 00000008 ef183f40 edfed800 edfeda34 edfeda70 0000002c 014000c0 bf678080
[  532.883793] 3e60: c0a03c48 c0193a30 bf67808c 00007fff bf678080 c0191330 c0a5dce8 bf6780c8
[  532.891990] 3e80: c0190a50 bf67808c c0702ce4 bf678170 c082dfd8 c07da244 c07da3a0 c0a03c48
[  532.900186] 3ea0: c07da250 f1523fff ffe00000 ef0e8600 fffff000 c0a5dcc0 014002c0 00000011
[  532.908382] 3ec0: 00000000 00000000 00000000 00000000 00000000 00000000 6e72656b 00006c65
[  532.916578] 3ee0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[  532.924774] 3f00: 00000000 00000000 00000000 00000000 00000000 c9929c40 00000080 00000fe4
[  532.932970] 3f20: 00000000 00642ff4 f1523fe4 ffffe000 b6f74230 00000051 00000000 c0194110
[  532.941166] 3f40: f1515bcc f1513000 00010fe4 f1523904 f1523748 f151f760 00004000 00004080
[  532.949362] 3f60: 00000000 00000000 00000000 000036d0 00000029 0000002a 00000017 00000000
[  532.957558] 3f80: 00000012 00000000 00000000 00000000 00000003 00000080 c01077a4 ef182000
[  532.965755] 3fa0: 00000080 c01075a0 00000000 00000000 00632010 00010fe4 b6f74230 00000700
[  532.973951] 3fc0: 00000000 00000000 00000003 00000080 00010fe4 00000000 00000020 00000000
[  532.982148] 3fe0: beca6d1c beca6d00 00011d50 b6f2abac 60000010 00632010 00000000 00000000
[  532.990350] [<c01ee8b4>] (__kmalloc_track_caller) from [<c01c73b8>] (kstrdup+0x30/0x54)
[  532.998378] [<c01c73b8>] (kstrdup) from [<c024eb40>] (__kernfs_new_node+0x28/0x130)
[  533.006055] [<c024eb40>] (__kernfs_new_node) from [<c024fcc8>] (kernfs_new_node+0x1c/0x38)
[  533.014340] [<c024fcc8>] (kernfs_new_node) from [<c025173c>] (__kernfs_create_file+0x18/0xa4)
[  533.022885] [<c025173c>] (__kernfs_create_file) from [<c025207c>] (sysfs_add_file_mode_ns+0x13c/0x194)
[  533.032213] [<c025207c>] (sysfs_add_file_mode_ns) from [<c0252a5c>] (internal_create_group+0x194/0x2e8)
[  533.041630] [<c0252a5c>] (internal_create_group) from [<c0193a30>] (load_module+0x1b74/0x2118)
[  533.050262] [<c0193a30>] (load_module) from [<c0194110>] (SyS_init_module+0x13c/0x174)
[  533.058201] [<c0194110>] (SyS_init_module) from [<c01075a0>] (ret_fast_syscall+0x0/0x54)
[  533.066312] Code: ea00000f e121f00c eaffffd2 e5943014 (e7973003) 
[  533.072445] ---[ end trace bb93ca4b64a48f93 ]---
[  533.079218] Kernel panic - not syncing: Fatal exception
[  533.084457] CPU1: stopping
[  533.087173] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G      D         4.14.176 #0
[  533.094497] Hardware name: Marvell Armada 380/385 (Device Tree)
[  533.100436] [<c010ecf8>] (unwind_backtrace) from [<c010a9b0>] (show_stack+0x10/0x14)
[  533.108202] [<c010a9b0>] (show_stack) from [<c0636974>] (dump_stack+0x94/0xa8)
[  533.115442] [<c0636974>] (dump_stack) from [<c010db38>] (handle_IPI+0xe4/0x190)
[  533.122769] [<c010db38>] (handle_IPI) from [<c0101494>] (gic_handle_irq+0x8c/0x90)
[  533.130357] [<c0101494>] (gic_handle_irq) from [<c010b64c>] (__irq_svc+0x6c/0x90)
[  533.137855] Exception stack(0xef067f80 to 0xef067fc8)
[  533.142919] 7f80: 00000001 00000000 00000000 c01145a0 ffffe000 c0a03cb8 c0a03c6c 00000000
[  533.151115] 7fa0: 00000000 414fc091 00000000 00000000 ef067fc8 ef067fd0 c0107f68 c0107f6c
[  533.159309] 7fc0: 60000013 ffffffff
[  533.162807] [<c010b64c>] (__irq_svc) from [<c0107f6c>] (arch_cpu_idle+0x34/0x38)
[  533.170224] [<c0107f6c>] (arch_cpu_idle) from [<c015f6d4>] (do_idle+0xdc/0x19c)
[  533.177551] [<c015f6d4>] (do_idle) from [<c015f9f0>] (cpu_startup_entry+0x18/0x1c)
[  533.185139] [<c015f9f0>] (cpu_startup_entry) from [<0010182c>] (0x10182c)
[  533.193813] Rebooting in 3 seconds..

(Updated) Moreover, the kernel has been crashed after several packets processed. In the log, I see some bug information related to Insmod function. Here is my code in the Init function module: ( In my Init function, I implement encryption/decryption with one packet request many times by using a loop for and module_param value. )

static int __init test_init(void)
{
        for (i = 0; i < req_num; i ++)
        {
        if (cipher_choice == 3)
            {
                test_esp_rfc4106(test_choice,endec);
                mdelay(1000);
                pr_err("--------------------------%d-------------------: 
                %s - PID:%d\n",__LINE__ , __func__ ,  current->pid);
                pr_err("------------------------Number of req----------- 
                --------: %d\n",i);
            }

          }
    return 0;
}

Answer 1

You can tell from the log that this happened right at soon as the module was loaded (well, during loading) because insmod is still running. It was creating an entry in '/sys'. This gives you some idea what may have been happening before the crash.

Since the crash is inside kstrdup/kmalloc and not directly related to your code, the most likely cause is either a double free or a buffer overflow in your module's code. Since it was soon after loading the module, probably the problem is in your module's init. No one is going to be able to tell you exactly what went wrong because you didn't post any code.