eBPF tools - skb_network_header crashes in a BPF Kernel trace function

Question

I am looking to trace ip_forward_finish . The intent is to trace latency of all TCP connections going through a linux based gateway router. Hence thought of tracing ip_forward_finish kernel function. And capture the time-stamp of SYN , SYN-ACK and ACK messages at the router.

The issue is accessing iphdr inside the trace function makes the verifier complain with the following error:

bpf: Failed to load program: Permission denied
0: (79) r6 = *(u64 *)(r1 +96)
1: (b7) r1 = 0
2: (6b) *(u16 *)(r10 -24) = r1
3: (bf) r3 = r6
4: (07) r3 += 192
5: (bf) r1 = r10
6: (07) r1 += -24
7: (b7) r2 = 2
8: (85) call bpf_probe_read#4
9: (69) r1 = *(u16 *)(r10 -24)
10: (55) if r1 != 0x8 goto pc+7
 R0=inv(id=0) R1=inv8 R6=inv(id=0) R10=fp0
11: (69) r1 = *(u16 *)(r6 +196)
R6 invalid mem access 'inv'

HINT: The invalid mem access 'inv' error can happen if you try to dereference
    memory without first using bpf_probe_read() to copy it to the BPF stack.
    Sometimes the bpf_probe_read is automatic by the bcc rewriter, other times
    you'll need to be explicit.

The code fragment I originally had was as below and the crash occurs when an access to ip_Hdr->protocol is made. And I also checked that ip_Hdr is not null.

int trace_forward_finish(struct pt_regs *ctx,struct net *net,
                         struct sock *sk, struct sk_buff *skb)
{
    if (skb->protocol != htons(ETH_P_IP))
        return 0;

    struct iphdr* ip_Hdr = (struct iphdr *) skb_network_header(skb);

    if (ip_Hdr->protocol != IPPROTO_TCP)
        return 0;

    /// More code
}

Per the HINT in the message, I did try to change to bpf_probe_read but still the same outcome

int trace_forward_finish(struct pt_regs *ctx,struct net *net,
                         struct sock *sk, struct sk_buff *skb)
{
    if (skb->protocol != htons(ETH_P_IP))
        return 0;

    struct iphdr ip_Hdr;
    bpf_probe_read(&ip_Hdr, sizeof(ip_Hdr), (void*)ip_hdr(skb)); 

    if (ip_Hdr.protocol != IPPROTO_TCP)
        return 0;

    return 0;
}

Any help would be appreciated.

Answer 1

bcc will try to transform your dereferences of kernel pointers into calls to bpf_probe_read . You can see that happening by passing debug=4 to the BPF() call.

In your case, I suspect that you would need to include function skb_network_header in your code so that bcc is able to rewrite it.

If that's not sufficient, then you might need a manual call to bpf_probe_read to retrieve structure struct iphdr from skb_network_header 's pointer.