Socket option IP_MULTICAST_IF with static multicast route switches from multicast to unicast MAC addressing

Question

I would like experts' advice on the usage of the socket option IP_MULTICAST_IF ("set multicast interface") combined with static multicast routes.

On a LAN, a multicast IP datagram is commonly sent in a multicast Ethernet frame (IP/MAC multicast destination address mapping). On a multi-homed Linux system (kernel 5.11), I have noticed that the socket option IP_MULTICAST_IF modifies the behavior as follow:

• Without static route, the multicast IP datagram is always sent in a multicast Ethernet frame, with or without IP_MULTICAST_IF .
• With a static route, without IP_MULTICAST_IF , the multicast IP datagram is sent in a multicast Ethernet frame.
• With a static route, with IP_MULTICAST_IF , the multicast IP datagram is sent in a unicast Ethernet frame to the gateway.

First question: With a static route for the multicast packet, should the multicast IP datagram be sent in a multicast Ethernet frame or in a unicast Ethernet frame to the gateway?

Second question: Whatever is the answer to the first question, why does the socket option IP_MULTICAST_IF switch from multicast to unicast MAC addressing?

The Linux man page ("man 7 ip") is not very explicit:

IP_MULTICAST_IF (since Linux 1.2)
    Set  the  local device for a multicast socket.  The argument for setsockopt(2) is an ip_mreqn or (since Linux 3.5)
    ip_mreq structure similar to IP_ADD_MEMBERSHIP, or an in_addr structure.  (The kernel determines  which  structure
    is being passed based on the size passed in optlen.)  For getsockopt(2), the argument is an in_addr structure.

Here is a sample configuration to reproduce this between two Linux virtual machines.

First system, "vmubuntu", receiver running Wireshark on interface ens38:

ens33: 00:0C:29:46:B7:CE  192.168.98.3   / 24  -> NAT, default route
ens38: 00:50:56:39:F0:03  192.168.233.11 / 24  -> host local vmware

Second system, "vmfedora", sender system:

ens33: 00:0C:29:B6:33:8D  192.168.98.2   / 24  -> NAT, default route
ens37: 00:50:56:29:34:37  192.168.233.10 / 24  -> host local vmware

We declare a static route on "vmfedora" for multicast traffic, using "vmubuntu" as gateway. Note that the default route for general IP traffic is on the other interface.

$ uname -a
Linux vmfedora 5.11.15-200.fc33.x86_64 #1 SMP Fri Apr 16 13:41:20 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
$
$ sudo route add -net 224.0.0.0 netmask 240.0.0.0 gw 192.168.233.11 dev ens37
$
$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.98.1    0.0.0.0         UG    20100  0        0 ens33
192.168.98.0    0.0.0.0         255.255.255.0   U     100    0        0 ens33
192.168.233.0   0.0.0.0         255.255.255.0   U     101    0        0 ens37
224.0.0.0       192.168.233.11  240.0.0.0       UG    0      0        0 ens37

Let's send multicast packets from "vmfedora" to 239.230.2.44:3044, in the range of the route. We bind the socket to local address 192.168.233.10, which is also the outgoing interface for the static route. We do not use the socket option IP_MULTICAST_IF (sample code below).

On "vmubuntu", Wireshark reports this:

Internet Protocol Version 4, Src: 192.168.233.10, Dst: 239.230.2.44
Ethernet II, Src: VMware_29:34:37 (00:50:56:29:34:37), Dst: IPv4mcast_66:02:2c (01:00:5e:66:02:2c)
    Destination: IPv4mcast_66:02:2c (01:00:5e:66:02:2c)
        Address: IPv4mcast_66:02:2c (01:00:5e:66:02:2c)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)
    Source: VMware_29:34:37 (00:50:56:29:34:37)
        Address: VMware_29:34:37 (00:50:56:29:34:37)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Type: IPv4 (0x0800)

We can see that the IP datagram with a multicast destination address is sent in an Ethernet frame with the corresponding multicast destination address.

Now, let's add socket option IP_MULTICAST_IF after bind() . Wireshark reports this:

Internet Protocol Version 4, Src: 192.168.233.10, Dst: 239.230.2.44
Ethernet II, Src: VMware_29:34:37 (00:50:56:29:34:37), Dst: VMware_39:f0:03 (00:50:56:39:f0:03)
    Destination: VMware_39:f0:03 (00:50:56:39:f0:03)
        Address: VMware_39:f0:03 (00:50:56:39:f0:03)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Source: VMware_29:34:37 (00:50:56:29:34:37)
        Address: VMware_29:34:37 (00:50:56:29:34:37)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Type: IPv4 (0x0800)

Now, we see that the IP datagram with a multicast destination address is sent in an Ethernet frame with the unicast MAC address of the gateway as destination address.

And the only difference is the socket option IP_MULTICAST_IF .

If you want to reproduce this, see the code below. It takes one mandatory command line parameter: the IP address of the local interface to which the socket is bound. The second optional command line parameter is "-f" to force the socket option IP_MULTICAST_IF (not set by default).

Thanks for your explanations on this option.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <ctype.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <netinet/in.h>

int main(int argc, char* argv[])
{
    struct sockaddr_in dest;
    dest.sin_family = AF_INET;
    dest.sin_port = htons(3044);
    inet_pton(AF_INET, "239.230.2.44", &dest.sin_addr);

    struct sockaddr_in local;
    local.sin_family = AF_INET;
    local.sin_port = 0;
    local.sin_addr.s_addr = 0;

    int use_mcast_if = 0;

    for (int arg = 1; arg < argc; ++arg) {
        if (strcmp(argv[arg], "-f") == 0) {
            use_mcast_if = 1;
        }
        else if (inet_pton(AF_INET, argv[arg], &local.sin_addr) != 1) {
            fprintf(stderr, "invalid local address: %s\n", argv[arg]);
            return EXIT_FAILURE;
        }
    }

    const int sock = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
    if (sock < 0) {
        perror("socket()");
        return EXIT_FAILURE;
    }
    
    if (bind(sock, (struct sockaddr*)&local, sizeof(local)) < 0) {
        perror("bind()");
        return EXIT_FAILURE;
    }

    if (use_mcast_if && setsockopt(sock, IPPROTO_IP, IP_MULTICAST_IF, &local.sin_addr, sizeof(local.sin_addr)) < 0) {
        perror("setsockopt(IP_MULTICAST_IF)");
        return EXIT_FAILURE;
    }

    const int data = 0x12345678;
    for (int i = 0; i < 10; ++i) {
        sendto(sock, &data, sizeof(data), 0, (struct sockaddr*)&dest, sizeof(dest));
    }

    close(sock);
    return EXIT_SUCCESS;
}

Answer 1

Linux doesn't normally handle multicast routing without special software such as mrouted or pimd. I tried this on a CentOS 7 VM (3.10 kernel) and saw unicast MAC addresses with the static route whether or not I used IP_MULTICAST_IF .

The following post on ServerFault goes into this in more detail:

https://serverfault.com/questions/814259/use-ip-route-add-to-add-multicast-routes-to-multiple-interfaces

TL;DR -- Don't use static multicast routes. Stick with setting IP_MULTICAST_IF .