初始化集群 Postgres-XL

Question

I'm having issues initialising a Postgres-XL cluster despite (I think) following steps exactly. A lot of Googling doesn't seem to be bringing up the answer or not, not sure if I'm being stupid or not.

I'm trying to initialise across three VMs, say - 127.168.453.83, 127.168.453.84, 127.168.453.85

My pgxc_conf.conf is as follows:

# pgxcInstallDir variable is needed if you invoke "deploy" command from pgxc_ctl utility.
# If don't you don't need this variable.
pgxcInstallDir=$HOME/pgxc
#---- OVERALL -----------------------------------------------------------------------------
#
pgxcOwner=postgres      # owner of the Postgres-XC databaseo cluster.  Here, we use this
                        # both as linus user and database user.  This must be
                        # the super user of each coordinator and datanode.
pgxcUser=$pgxcOwner     # OS user of Postgres-XC owner

tmpDir=/tmp                 # temporary dir used in XC servers
localTmpDir=$tmpDir         # temporary dir used here locally

configBackup=n                  # If you want config file backup, specify y to this value.
configBackupHost=pgxc-linker    # host to backup config file
configBackupDir=$HOME/pgxc      # Backup directory
configBackupFile=pgxc_ctl.bak   # Backup file name --> Need to synchronize when original changed.

dataDirRoot=$HOME/DATA/pgxl/nodes

#---- GTM ------------------------------------------------------------------------------------

# GTM is mandatory.  You must have at least (and only) one GTM master in your Postgres-XC cluster.
# If GTM crashes and you need to reconfigure it, you can do it by pgxc_update_gtm command to update
# GTM master with others.   Of course, we provide pgxc_remove_gtm command to remove it.  This command
# will not stop the current GTM.  It is up to the operator.

#---- Overall -------
gtmName=()

#---- GTM Master -----------------------------------------------

#---- Overall ----
gtmMasterServer=(127.168.453.83)
gtmMasterPort=(6666)
gtmMasterDir=($dataDirRoot/gtm)

#---- Configuration ---
gtmExtraConfig=() # Will be added gtm.conf for both Master and Slave (done at initilization only)
gtmMasterSpecificExtraConfig=() # Will be added to Master's gtm.conf (done at initialization only)

#---- GTM Slave -----------------------------------------------

# Because GTM is a key component to maintain database consistency, you may want to configure GTM slave
# for backup.

#---- Overall ------
gtmSlave=n                  # Specify y if you configure GTM Slave.   Otherwise, GTM slave will not be configured and
                            # all the following variables will be reset.
gtmSlaveName=()
gtmSlaveServer=()       # value none means GTM slave is not available.  Give none if you don't configure GTM Slave.
gtmSlavePort=()         # Not used if you don't configure GTM slave.
gtmSlaveDir=()  # Not used if you don't configure GTM slave.
# Please note that when you have GTM failover, then there will be no slave available until you configure the slave
# again. (pgxc_add_gtm_slave function will handle it)

#---- Configuration ----
gtmSlaveSpecificExtraConfig=() # Will be added to Slave's gtm.conf (done at initialization only)

#---- GTM Proxy -------------------------------------------------------------------------------------------------------
# GTM proxy will be selected based upon which server each component runs on.
# When fails over to the slave, the slave inherits its master's gtm proxy.  It should be
# reconfigured based upon the new location.
#
# To do so, slave should be restarted.   So pg_ctl promote -> (edit postgresql.conf and recovery.conf) -> pg_ctl restart
#
# You don't have to configure GTM Proxy if you dont' configure GTM slave or you are happy if every component connects
# to GTM Master directly.  If you configure GTL slave, you must configure GTM proxy too.

#---- Shortcuts ------
gtmProxyDir=()

#---- Overall -------
gtmProxy=()             # Specify y if you conifugre at least one GTM proxy.   You may not configure gtm proxies
                        # only when you dont' configure GTM slaves.
                        # If you specify this value not to y, the following parameters will be set to default empty values.
                        # If we find there're no valid Proxy server names (means, every servers are specified
                        # as none), then gtmProxy value will be set to "n" and all the entries will be set to
                        # empty values.
gtmProxyNames=()    # No used if it is not configured
gtmProxyServers=()          # Specify none if you dont' configure it.
gtmProxyPorts=()                # Not used if it is not configured.
gtmProxyDirs=() # Not used if it is not configured.

#---- Configuration ----
gtmPxyExtraConfig=none      # Extra configuration parameter for gtm_proxy.  Coordinator section has an example.

#---- Coordinators ----------------------------------------------------------------------------------------------------

#---- shortcuts ----------
coordMasterDir=$dataDirRoot/coord_master
coordSlaveDir=$HOME/coord_slave
coordArchLogDir=$HOME/coord_archlog

#---- Overall ------------
coordNames=(coord1 coord2)      # Master and slave use the same name
coordPorts=(5432 5432)          # Master server listening ports
poolerPorts=(6668 6669)         # Master pooler ports
coordPgHbaEntries=(127.168.453.0/24 127.168.453.0/24)   # Assumes that all the coordinator (master/slave) accepts
                                                # the same connection
                                                # This entry allows only $pgxcOwner to connect.
                                                # If you'd like to setup another connection, you should
                                                # supply these entries through files specified below.
#coordPgHbaEntries=(127.0.0.1/32)   # Same as above but for IPv4 connections

#---- Master -------------
coordMasterServers=(127.168.453.84 127.168.453.85)      # none means this master is not available
coordMasterDirs=($coordMasterDir/coord1 $coordMasterDir/coord2)
coordMaxWALsender=0 # max_wal_senders: needed to configure slave. If zero value is specified,
                        # it is expected to supply this parameter explicitly by external files
                        # specified in the following.   If you don't configure slaves, leave this value to zero.
coordMaxWALSenders=(0 0)
                        # max_wal_senders configuration for each coordinator.

#---- Slave -------------
coordSlave=n            # Specify y if you configure at least one coordiantor slave.  Otherwise, the following
                        # configuration parameters will be set to empty values.
                        # If no effective server names are found (that is, every servers are specified as none),
                        # then coordSlave value will be set to n and all the following values will be set to
                        # empty values.
coordSlaveSync=n        # Specify to connect with synchronized mode.
coordSlaveServers=()            # none means this slave is not available
coordSlavePorts=()          # coordinator slave listening ports
coordSlavePoolerPorts=()            # coordinator slave pooler ports
coordSlaveDirs=()
coordArchLogDirs=()

#---- Configuration files---
# Need these when you'd like setup specific non-default configuration
# These files will go to corresponding files for the master.
# You may supply your bash script to setup extra config lines and extra pg_hba.conf entries
# Or you may supply these files manually.
coordExtraConfig=coordExtraConfig   # Extra configuration file for coordinators.
                        # This file will be added to all the coordinators'
                        # postgresql.conf
# Pleae note that the following sets up minimum parameters which you may want to change.
# You can put your postgresql.conf lines here.
cat > $coordExtraConfig <<EOF
#================================================
# Added to all the coordinator postgresql.conf
# Original: $coordExtraConfig
log_destination = 'stderr'
logging_collector = on
log_directory = 'pg_log'
listen_addresses = '*'
max_connections = 100
hot_standby = off
max_pool_size = 300
EOF

# Additional Configuration file for specific coordinator master.
# You can define each setting by similar means as above.
coordSpecificExtraConfig=(none none none)
coordSpecificExtraPgHba=(none none none)

#---- Datanodes -------------------------------------------------------------------------------------------------------

#---- Shortcuts --------------
datanodeMasterDir=$dataDirRoot/dn_master
datanodeSlaveDir=$dataDirRoot/dn_slave
datanodeArchLogDir=$dataDirRoot/datanode_archlog

#---- Overall ---------------
primaryDatanode=data1 # Primary Node.
datanodeNames=(data1 data2)
datanodePorts=(15432 15433) # Master and slave use the same port!
datanodePoolerPorts=(6671 6672) # Master and slave use the same port!
datanodePgHbaEntries=(127.168.453.0/24 127.168.453.0/24)    # Assumes that all the coordinator (master/slave) accepts
                                        # the same connection
                                        # This list sets up pg_hba.conf for $pgxcOwner user.
                                        # If you'd like to setup other entries, supply them
                                        # through extra configuration files specified below.
#datanodePgHbaEntries=(127.0.0.1/32)    # Same as above but for IPv4 connections

#---- Master ----------------
datanodeMasterServers=(127.168.453.84 127.168.453.85)   # none means this master is not available.
                                                    # This means that there should be the master but is down.
                                                    # The cluster is not operational until the master is
                                                    # recovered and ready to run.
datanodeMasterDirs=($datanodeMasterDir/data1 $datanodeMasterDir/data2)
datanodeMaxWalSender=0                              # max_wal_senders: needed to configure slave. If zero value is
                                                    # specified, it is expected this parameter is explicitly supplied
                                                    # by external configuration files.
                                                    # If you don't configure slaves, leave this value zero.
datanodeMaxWALSenders=(0 0)
                        # max_wal_senders configuration for each datanode

#---- Slave -----------------
datanodeSlave=n         # Specify y if you configure at least one coordiantor slave.  Otherwise, the following
                        # configuration parameters will be set to empty values.
                        # If no effective server names are found (that is, every servers are specified as none),
                        # then datanodeSlave value will be set to n and all the following values will be set to
                        # empty values.
datanodeSlaveServers=() # value none means this slave is not available
datanodeSlavePorts=()   # Master and slave use the same port!
datanodeSlavePoolerPorts=() # Master and slave use the same port!
#datanodeSlaveSync=y        # If datanode slave is connected in synchronized mode
datanodeSlaveDirs=()
datanodeArchLogDirs=()

# ---- Configuration files ---
# You may supply your bash script to setup extra config lines and extra pg_hba.conf entries here.
# These files will go to corresponding files for the master.
# Or you may supply these files manually.
datanodeExtraConfig=datanodeExtraConfig
cat > $datanodeExtraConfig <<EOF
#================================================
# Added to all the datanode postgresql.conf
# Original: $datanodeExtraConfig
log_destination = 'stderr'
logging_collector = on
log_directory = 'pg_log'
listen_addresses = '*'
max_connections = 100
hot_standby = off
EOF
# Additional Configuration file for specific datanode master.
# You can define each setting by similar means as above.
datanodeSpecificExtraConfig=()
datanodeSpecificExtraPgHba=()

So fairly basic setup to trial. But when running pgxc_ctl init all, I consistently get the below. I'm not sure what is driving the FATAL GTM error when initialising the datanodes. Have done a lot of Googling to no avail - Any help would be greatly appreciated.

Installing pgxc_ctl_bash script as /var/lib/postgresql/pgxc_ctl/pgxc_ctl_bash.
Installing pgxc_ctl_bash script as /var/lib/postgresql/pgxc_ctl/pgxc_ctl_bash.
Reading configuration using /var/lib/postgresql/pgxc_ctl/pgxc_ctl_bash --home /var/lib/postgresql/pgxc_ctl --configuration /var/lib/postgresql/pgxc_ctl/pgxc_ctl.conf
Finished reading configuration.
   ******** PGXC_CTL START ***************

Current directory: /var/lib/postgresql/pgxc_ctl
Initialize GTM master
The files belonging to this GTM system will be owned by user "postgres".
This user must also own the server process.


fixing permissions on existing directory /var/lib/postgresql/DATA/pgxl/nodes/gtm ... ok
creating configuration files ... ok
creating control file ... ok

Success.
waiting for server to shut down.... done
server stopped
Done.
Start GTM master
server starting
Initialize all the coordinator masters.
Initialize coordinator master coord1.
Initialize coordinator master coord2.
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_GB.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/DATA/pgxl/nodes/coord_master/coord1 ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... creating cluster information ... ok
syncing data to disk ... ok
freezing database template0 ... ok
freezing database template1 ... ok
freezing database postgres ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success.
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_GB.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/DATA/pgxl/nodes/coord_master/coord2 ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... creating cluster information ... ok
syncing data to disk ... ok
freezing database template0 ... ok
freezing database template1 ... ok
freezing database postgres ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success.
Done.
Starting coordinator master.
Starting coordinator master coord1
Starting coordinator master coord2
2019-02-28 16:53:22.290 GMT [56601] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2019-02-28 16:53:22.290 GMT [56601] LOG:  listening on IPv6 address "::", port 5432
2019-02-28 16:53:22.294 GMT [56601] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
2019-02-28 16:53:22.306 GMT [56601] LOG:  redirecting log output to logging collector process
2019-02-28 16:53:22.306 GMT [56601] HINT:  Future log output will appear in directory "pg_log".
2019-02-28 16:53:22.303 GMT [56394] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2019-02-28 16:53:22.303 GMT [56394] LOG:  listening on IPv6 address "::", port 5432
2019-02-28 16:53:22.307 GMT [56394] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
2019-02-28 16:53:22.318 GMT [56394] LOG:  redirecting log output to logging collector process
2019-02-28 16:53:22.318 GMT [56394] HINT:  Future log output will appear in directory "pg_log".
Done.
Initialize all the datanode masters.
Initialize the datanode master data1.
Initialize the datanode master data2.
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_GB.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/DATA/pgxl/nodes/dn_master/data1 ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... creating cluster information ... ok
syncing data to disk ... ok
freezing database template0 ... ok
freezing database template1 ... ok
freezing database postgres ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success.
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_GB.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/DATA/pgxl/nodes/dn_master/data2 ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... creating cluster information ... ok
syncing data to disk ... ok
freezing database template0 ... ok
freezing database template1 ... ok
freezing database postgres ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success.
Done.
Starting all the datanode masters.
Starting datanode master data1.
Starting datanode master data2.
2019-02-28 16:53:32.000 GMT [58828] LOG:  listening on IPv4 address "0.0.0.0", port 15432
2019-02-28 16:53:32.000 GMT [58828] LOG:  listening on IPv6 address "::", port 15432
2019-02-28 16:53:32.003 GMT [58828] LOG:  listening on Unix socket "/tmp/.s.PGSQL.15432"
2019-02-28 16:53:32.019 GMT [58828] LOG:  redirecting log output to logging collector process
2019-02-28 16:53:32.019 GMT [58828] HINT:  Future log output will appear in directory "pg_log".
2019-02-28 16:53:32.003 GMT [58632] LOG:  listening on IPv4 address "0.0.0.0", port 15433
2019-02-28 16:53:32.003 GMT [58632] LOG:  listening on IPv6 address "::", port 15433
2019-02-28 16:53:32.006 GMT [58632] LOG:  listening on Unix socket "/tmp/.s.PGSQL.15433"
2019-02-28 16:53:32.020 GMT [58632] LOG:  redirecting log output to logging collector process
2019-02-28 16:53:32.020 GMT [58632] HINT:  Future log output will appear in directory "pg_log".
Done.
psql: FATAL:  Could not obtain a transaction ID from GTM. The GTM might have failed or lost connectivity
psql: FATAL:  Could not obtain a transaction ID from GTM. The GTM might have failed or lost connectivity
Done.
psql: FATAL:  Could not obtain a transaction ID from GTM. The GTM might have failed or lost connectivity
psql: FATAL:  Could not obtain a transaction ID from GTM. The GTM might have failed or lost connectivity
Done.

Answer 1

I know this is late for your setup but for people that are seeing this issue.

Only do this on New Clusters where you don't have any data

• If this is a cluster that you had tried to initialize a few times and things are not working. Stop all components, rename your GTM directory to something else.

• Do an init all

Your cluster should stop getting the GTM errors.