Hyperledger Composer - Docker Swarm

Question

I've been experimenting with Hyperledger Composer and with the official multi org tutorial. I was successful in modifying the given demo, adding a third organisation and finally installing my own bna.

The next step was to fully understand how to deploy the Fabric network and Composer on multiple physical machines. And I went through all the available info about deploying such a process but without much luck.

Let suppose:

PC1: 1 Orderer, 1 Organisation, One cli container;

PC2: 1 Organisation;

PC3: 1 Organization;

I'm able to put the 3 machines in a swarm.

I know that I need to generate the certificates for all the machines and that they should be identical.

But from there I don't fully understand how to continue, or how to add references to the swarm network inside the compose files...

#docker-compose-cas-template-0 - This is for PC 1
version: '2'

networks:
  example:
services:
  ca0:
    image: hyperledger/fabric-ca
    environment:
      - FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server
      - FABRIC_CA_SERVER_CA_NAME=ca-manager
      - FABRIC_CA_SERVER_TLS_ENABLED=true
      - FABRIC_CA_SERVER_TLS_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.manager.example.com-cert.pem
      - FABRIC_CA_SERVER_TLS_KEYFILE=/etc/hyperledger/fabric-ca-server-config/CA1_PRIVATE_KEY
    ports:
      - "7054:7054"
    command: sh -c 'fabric-ca-server start --ca.certfile /etc/hyperledger/fabric-ca-server-config/ca.manager.example.com-cert.pem --ca.keyfile /etc/hyperledger/fabric-ca-server-config/CA1_PRIVATE_KEY -b admin:adminpw -d'
    volumes:
      - ./crypto-config/peerOrganizations/manager.example.com/ca/:/etc/hyperledger/fabric-ca-server-config
    container_name: ca_peerManager
    networks:
      - example



# docker-compose-base-1.yaml - This is for PC1
version: '2'

services:

  orderer.proa.com:
    container_name: orderer.proa.com
    image: hyperledger/fabric-orderer:$IMAGE_TAG
    environment:
      - ORDERER_GENERAL_LOGLEVEL=INFO
      - ORDERER_GENERAL_LISTENADDRESS=0.0.0.0
      - ORDERER_GENERAL_GENESISMETHOD=file
      - ORDERER_GENERAL_GENESISFILE=/var/hyperledger/orderer/orderer.genesis.block
      - ORDERER_GENERAL_LOCALMSPID=OrdererMSP
      - ORDERER_GENERAL_LOCALMSPDIR=/var/hyperledger/orderer/msp
      # enabled TLS
      - ORDERER_GENERAL_TLS_ENABLED=true
      - ORDERER_GENERAL_TLS_PRIVATEKEY=/var/hyperledger/orderer/tls/server.key
      - ORDERER_GENERAL_TLS_CERTIFICATE=/var/hyperledger/orderer/tls/server.crt
      - ORDERER_GENERAL_TLS_ROOTCAS=[/var/hyperledger/orderer/tls/ca.crt]
    working_dir: /opt/gopath/src/github.com/hyperledger/fabric
    command: orderer
    volumes:
    - ../channel-artifacts/genesis.block:/var/hyperledger/orderer/orderer.genesis.block
    - ../crypto-config/ordererOrganizations/proa.com/orderers/orderer.proa.com/msp:/var/hyperledger/orderer/msp
    - ../crypto-config/ordererOrganizations/proa.com/orderers/orderer.proa.com/tls/:/var/hyperledger/orderer/tls
    - orderer.proa.com:/var/hyperledger/production/orderer
    ports:
      - 7050:7050

  peer0.manager.proa.com:
    container_name: peer0.manager.proa.com
    extends:
      file: peer-base.yaml
      service: peer-base
    environment:
      - CORE_PEER_ID=peer0.manager.proa.com
      - CORE_PEER_ADDRESS=peer0.manager.proa.com:7051
      - CORE_PEER_GOSSIP_BOOTSTRAP=peer1.manager.proa.com:7051
      - CORE_PEER_GOSSIP_EXTERNALENDPOINT=peer0.manager.proa.com:7051
      - CORE_PEER_LOCALMSPID=ManagerMSP
    volumes:
        - /var/run/:/host/var/run/
        - ../crypto-config/peerOrganizations/manager.proa.com/peers/peer0.manager.proa.com/msp:/etc/hyperledger/fabric/msp
        - ../crypto-config/peerOrganizations/manager.proa.com/peers/peer0.manager.proa.com/tls:/etc/hyperledger/fabric/tls
        - peer0.manager.proa.com:/var/hyperledger/production
    ports:
      - 7051:7051
      - 7053:7053

  peer1.manager.proa.com:
    container_name: peer1.manager.proa.com
    extends:
      file: peer-base.yaml
      service: peer-base
    environment:
      - CORE_PEER_ID=peer1.manager.proa.com
      - CORE_PEER_ADDRESS=peer1.manager.proa.com:7051
      - CORE_PEER_GOSSIP_EXTERNALENDPOINT=peer1.manager.proa.com:7051
      - CORE_PEER_GOSSIP_BOOTSTRAP=peer0.manager.proa.com:7051
      - CORE_PEER_LOCALMSPID=ManagerMSP
    volumes:
        - /var/run/:/host/var/run/
        - ../crypto-config/peerOrganizations/manager.proa.com/peers/peer1.manager.proa.com/msp:/etc/hyperledger/fabric/msp
        - ../crypto-config/peerOrganizations/manager.proa.com/peers/peer1.manager.proa.com/tls:/etc/hyperledger/fabric/tls
        - peer1.manager.proa.com:/var/hyperledger/production

    ports:
      - 8051:7051
      - 8053:7053

UPDATED with docker-compose-cli.yaml - for ORG2 and PC2

# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#

version: '2'

volumes:
  peer0.sponsor.example.com:
  peer1.sponsor.example.com:

networks:
  example:
    external:
      name: example

services:

  peer0.sponsor.example.com:
    container_name: peer0.sponsor.example.com
    extends:
      file:  base/docker-compose-base-2.yaml
      service: peer0.sponsor.example.com
    networks:
      - example

  peer1.sponsor.example.com:
    container_name: peer1.sponsor.example.com
    extends:
      file:  base/docker-compose-base-2.yaml
      service: peer1.sponsor.example.com
    networks:
      - example

  cli2:
    container_name: cli2
    image: hyperledger/fabric-tools:$IMAGE_TAG
    tty: true
    stdin_open: true
    environment:
      - GOPATH=/opt/gopath
      - CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
      #- CORE_LOGGING_LEVEL=DEBUG
      - CORE_LOGGING_LEVEL=INFO
      - CORE_PEER_ID=cli
      - CORE_PEER_ADDRESS=peer0.sponsor.example.com:7051
      - CORE_PEER_LOCALMSPID=SponsorMSP
      - CORE_PEER_TLS_ENABLED=true
      - CORE_PEER_TLS_CERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/sponsor.example.com/peers/peer0.sponsor.example.com/tls/server.crt
      - CORE_PEER_TLS_KEY_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/sponsor.example.com/peers/peer0.sponsor.example.com/tls/server.key
      - CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/sponsor.example.com/peers/peer0.sponsor.example.com/tls/ca.crt
      - CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/sponsor.example.com/users/Admin@sponsor.example.com/msp
    working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer
    command: /bin/bash
    volumes:
        - /var/run/:/host/var/run/
        - ./../chaincode/:/opt/gopath/src/github.com/chaincode
        - ./crypto-config:/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/
        - ./scripts:/opt/gopath/src/github.com/hyperledger/fabric/peer/scripts/
        - ./channel-artifacts:/opt/gopath/src/github.com/hyperledger/fabric/peer/channel-artifacts
    depends_on:
      - peer0.sponsor.example.com
      - peer1.sponsor.example.com

    networks:
      - example

Answer 1

You're on right track. I'll list down the steps below:

1. We create a docker swarm and connect these hosts. Since you've already created a swarm, I assume your hosts are connected.

2. We create an overlay network from one of the hosts. In our case, it is "example" network.

    docker network create --attachable --driver overlay example 

   Now, this overlay network will be available in all the hosts. You can run the following command in each of the hosts:

    docker network ls 

   Here, you'll be able to see the network with the name "example" that is an overlay network.

   Also, you can inspect the network to see which all hosts (peers) are connected to this network using:

    docker network inspect example 

3. Spin up the containers. However, in this step, we need to make these containers join the existing overlay network ie "example". So your compose files will be like:

    version: '2' networks: example: external: name: example services: ca0: image: hyperledger/fabric-ca environment: - FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server - FABRIC_CA_SERVER_CA_NAME=ca-manager - FABRIC_CA_SERVER_TLS_ENABLED=true - FABRIC_CA_SERVER_TLS_CERTFILE=/etc/hyperledger/fabric-ca-server-config/ca.manager.example.com-cert.pem - FABRIC_CA_SERVER_TLS_KEYFILE=/etc/hyperledger/fabric-ca-server-config/CA1_PRIVATE_KEY ports: - "7054:7054" command: sh -c 'fabric-ca-server start --ca.certfile /etc/hyperledger/fabric-ca-server-config/ca.manager.example.com-cert.pem --ca.keyfile /etc/hyperledger/fabric-ca-server-config/CA1_PRIVATE_KEY -b admin:adminpw -d' volumes: - ./crypto-config/peerOrganizations/manager.example.com/ca/:/etc/hyperledger/fabric-ca-server-config container_name: ca_peerManager networks: - example 

   This configuration will remain similar for all your docker containers, be it, peer, orderer, ca or cli. Also, this configuration will make sure that your container will join the existing network instead of creating a new one.

Note: Running the docker in swarm mode requires few ports to be opened up. You can find those references in this article .