How to deploy an IPv6 container with Docker Swarm Mode or Docker Compose

Question

In the end I'd like to have a pure IPv6 network deployed via compose or swarm mode. For now, I'd just like to have a single container deployed with IPv6 (only). I am not currently interested in routing (just container to container connectivity).

My setup:

• OS: Centos 7
• dockerd --ipv6 --fixed-cidr-v6=2001:db8:1::/64 --iptables=true --ip-masq=true --mtu=1600 --experimental=true
• docker-engine-17.05.0.ce-1.el7.centos.x86_64.rpm
• Host has IPv4 and IPv6 addresses. Forwarding is on for both (not that it matters for me).

I've tried what seems to be every combination (I'm only listing a couple)

Self-contained Docker stack with container and network:

version: '3'

networks:
  app_net:
    driver: overlay
    driver_opts:
      com.docker.network.enable_ipv6: "true"
    ipam:
      driver: default
      config:
      -
        subnet: 172.16.238.0/24
      -
        subnet: 2001:3984:3989::/64        

services:
  app:
    image: alpine
    command: sleep 600
    networks:
      app_net:
        ipv4_address: 0.0.0.0
        ipv6_address: 2001:3984:3989::10

Result: Only IPv4 address in container, 0.0.0.0 is ignored.

---

Externally pre-created network (as per https://stackoverflow.com/a/39818953/1735931 )

docker network create --driver overlay --ipv6 --subnet=2001:3984:3989::/64 --attachable ext_net

version: '3'

networks:
  ext_net:
    external:
      name: ext_net

services:
  app:
    image: alpine
    command: ifconfig eth0 0.0.0.0 ; sleep 600
    cap_add:
     - NET_ADMIN
    networks:
      ext_net:
        ipv4_address: 0.0.0.0
        ipv6_address: 2001:3984:3989::10

Result: Both IPv4 and IPv6 addresses in container, but cap_add is ignored (not supported in Swarm Mode), and thus the ifconfig disable ipv4 attempt above does not work.

I don't currently have docker-compose installed, and will probably try that next, but is there a way to run pure IPv6 containers in Docker Swarm Mode?

Note: I am able to run and configure a few IPv6-only containers manually without swarm/compose: (Create network as above or even just use the default bridge)

$ docker run --cap-add=NET_ADMIN --rm -it alpine
$$ ifconfig eth0 0.0.0.0
$$ ping6 other-container-ipv6-address # WORKS!

or shorthand:

$ docker run --cap-add=NET_ADMIN --rm -it alpine sh -c "/sbin/ifconfig eth0 0.0.0.0 ; sh"

Answer 1

I was able to hack it with docker-compose via severe ugliness. If you're desperate, here it is. (This method can never work for Swarm Mode due to privilege escalation).

The Plan

1. Grant containers rights to manage IP's
2. Remove IPv4 IP address from within each container on startup.
3. Use a volume to improvise a hosts file in place of DNS (DNS is IPv4-only in docker).

Steps

1. Enable IPv6 in Docker daemon .
2. Create a docker-compose.yml file that creates an ipv6 network, a volume for shared files, and two containers
3. Run an entrypoint script in each container that performs the aforementioned steps.

Files

docker-compose.yml

# Note: enable_ipv6 does not work in version 3!
version: '2.1'

networks:
  app_net:
    enable_ipv6: true
    driver: overlay
    ipam:
      driver: default
      config:
      -
        subnet: 172.16.238.0/24
      -
        subnet: 2001:3984:3989::/64

services:
  app1:
    build: ./server 
    hostname: server1
    command: blablabla # example of arg passing to ipv6.sh
    cap_add:
     - NET_ADMIN
    volumes:
     - ipv6stuff:/ipv6stuff
    networks:
      - app_net

  app2:
    build: ./server 
    hostname: server2
    command: SOMETHING # example of arg passing to ipv6.sh
    cap_add:
     - NET_ADMIN
    volumes:
     - ipv6stuff:/ipv6stuff
    networks:
      - app_net

volumes:
  ipv6stuff:

server/Dockerfile

FROM alpine:latest
ADD files /
RUN apk --update add bash  #simpler scripts
# Has to be an array for parameters to work via command: x in compose file, if needed
ENTRYPOINT ["/ipv6.sh"]

server/files/ipv6.sh

#!/bin/bash
# Optionally conditional logic based on parameters here...
# (for example, conditionally leave ipv4 address alone in some containers)
#
# Remove ipv4
ifconfig eth0 0.0.0.0

IP6=$(ip addr show eth0 | grep inet6 | grep global | awk '{print $2}' | cut -d / -f 1)

echo "Host $HOSTNAME has ipv6 ip $IP6" 

# Store our entry in the shared volume
echo "$IP6   $HOSTNAME" > /ipv6stuff/hosts.$HOSTNAME

# Remove existing ipv4 line from /etc/hosts just to be thorough
# Docker does not allow removal of this file and thus simple sed -i isn't going to work.
cp /etc/hosts /tmp/1 ; sed -i "s/^.*\s$HOSTNAME//" /tmp/1 ; cat /tmp/1 > /etc/hosts

# Wait for all containers to start
sleep 2

# Put everyone's entries in our hosts file.
cat /ipv6stuff/hosts.* >> /etc/hosts

echo "My hosts file:"
cat /etc/hosts

# test connectivity (hardcoded)
ping6 -c 3 server1
ping6 -c 3 server2