为什么我在 MariaDB Galera Clutser 中收到此错误？

Question

I am using Galera Cluster, and recently encountered the following error while restarting MariaDB. It consists of a total of 3 nodes. When only the first node is running, DB access is possible, but when the second node is executed, the following error message is displayed and the cluster is not formed. Why?

MariaDB Version: 10.4.20

mariadb | =====================================
mariadb | 2022-08-11 12:00:11 0x7f37339cf700 INNODB MONITOR OUTPUT
mariadb | =====================================
mariadb | Per second averages calculated from the last 60 seconds
mariadb | -----------------
mariadb | BACKGROUND THREAD
mariadb | -----------------
mariadb | srv_master_thread loops: 3 srv_active, 0 srv_shutdown, 74 srv_idle
mariadb | srv_master_thread log flush and writes: 76
mariadb | ----------
mariadb | SEMAPHORES
mariadb | ----------
mariadb | OS WAIT ARRAY INFO: reservation count 20
mariadb | --Thread 139884474328832 has waited at dict0dict.cc line 880 for 232.00 seconds the semaphore:
mariadb | Mutex at 0x556c68691100, Mutex DICT_SYS created dict0dict.cc:824, lock var 2
mariadb |
mariadb | --Thread 139874768164608 has waited at srv0srv.cc line 2011 for 242.00 seconds the semaphore:
mariadb | X-lock on RW-latch at 0x556c68691130 created in file dict0dict.cc line 833
mariadb | a writer (thread id 139875057460992) has reserved it in mode  exclusive
mariadb | number of readers 0, waiters flag 1, lock_word: 0
mariadb | Last time write locked in file dict0stats.cc line 2486
mariadb | OS WAIT ARRAY INFO: signal count 13
mariadb | RW-shared spins 19, rounds 487, OS waits 13
mariadb | RW-excl spins 17, rounds 103, OS waits 2
mariadb | RW-sx spins 0, rounds 0, OS waits 0
mariadb | Spin rounds per wait: 25.63 RW-shared, 6.06 RW-excl, 0.00 RW-sx
mariadb | ------------
mariadb | TRANSACTIONS
mariadb | ------------
mariadb | Trx id counter 2908999983
mariadb | Purge done for trx's n:o < 2908999981 undo n:o < 0 state: running but idle
mariadb | History list length 4
mariadb | LIST OF TRANSACTIONS FOR EACH SESSION:
mariadb | ---TRANSACTION 421359460385048, COMMITTED IN MEMORY flushing log
mariadb | 0 lock struct(s), heap size 1128, 0 row lock(s), undo log entries 2
mariadb | ---TRANSACTION 421359460380824, not started
mariadb | 0 lock struct(s), heap size 1128, 0 row lock(s)
mariadb | --------
mariadb | FILE I/O
mariadb | --------
mariadb | I/O thread 0 state: waiting for completed aio requests (insert buffer thread)
mariadb | I/O thread 1 state: waiting for completed aio requests (log thread)
mariadb | I/O thread 2 state: waiting for completed aio requests (read thread)
mariadb | I/O thread 3 state: waiting for completed aio requests (read thread)
mariadb | I/O thread 4 state: waiting for completed aio requests (read thread)
mariadb | I/O thread 5 state: waiting for completed aio requests (read thread)
mariadb | I/O thread 6 state: waiting for completed aio requests (write thread)
mariadb | I/O thread 7 state: waiting for completed aio requests (write thread)
mariadb | I/O thread 8 state: waiting for completed aio requests (write thread)
mariadb | I/O thread 9 state: waiting for completed aio requests (write thread)
mariadb | Pending normal aio reads: [0, 0, 0, 0] , aio writes: [0, 0, 0, 0] ,
mariadb |  ibuf aio reads:, log i/o's:, sync i/o's:
mariadb | Pending flushes (fsync) log: 0; buffer pool: 0
mariadb | 129125 OS file reads, 323 OS file writes, 57 OS fsyncs
mariadb | 0.00 reads/s, 0 avg bytes/read, 0.00 writes/s, 0.00 fsyncs/s
mariadb | -------------------------------------
mariadb | INSERT BUFFER AND ADAPTIVE HASH INDEX
mariadb | -------------------------------------
mariadb | Ibuf: size 1, free list len 4026, seg size 4028, 0 merges
mariadb | merged operations:
mariadb |  insert 0, delete mark 0, delete 0
mariadb | discarded operations:
mariadb |  insert 0, delete mark 0, delete 0
mariadb | Hash table size 2212699, node heap has 0 buffer(s)
mariadb | Hash table size 2212699, node heap has 0 buffer(s)
mariadb | Hash table size 2212699, node heap has 0 buffer(s)
mariadb | Hash table size 2212699, node heap has 0 buffer(s)
mariadb | Hash table size 2212699, node heap has 2 buffer(s)
mariadb | Hash table size 2212699, node heap has 2 buffer(s)
mariadb | Hash table size 2212699, node heap has 2 buffer(s)
mariadb | Hash table size 2212699, node heap has 3 buffer(s)
mariadb | 0.00 hash searches/s, 0.00 non-hash searches/s
mariadb | ---
mariadb | LOG
mariadb | ---
mariadb | Log sequence number 1202450201026
mariadb | Log flushed up to   1202450200636
mariadb | Pages flushed up to 1202450196728
mariadb | Last checkpoint at  1202450196719
mariadb | 1 pending log flushes, 0 pending chkp writes
mariadb | 35 log i/o's done, 0.00 log i/o's/second
mariadb | ----------------------
mariadb | BUFFER POOL AND MEMORY
mariadb | ----------------------
mariadb | Total large memory allocated 8606711808
mariadb | Dictionary memory allocated 407480
mariadb | Buffer pool size   513728
mariadb | Free buffers       384737
mariadb | Database pages     128982
mariadb | Old database pages 47768
mariadb | Modified db pages  136
mariadb | Percent of dirty pages(LRU & free pages): 0.026
mariadb | Max dirty pages percent: 75.000
mariadb | Pending reads 0
mariadb | Pending writes: LRU 0, flush list 20, single page 0
mariadb | Pages made young 0, not young 0
mariadb | 0.00 youngs/s, 0.00 non-youngs/s
mariadb | Pages read 128851, created 131, written 282
mariadb | 0.00 reads/s, 0.00 creates/s, 0.00 writes/s
mariadb | No buffer pool page gets since the last printout
mariadb | Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
mariadb | LRU len: 128982, unzip_LRU len: 0
mariadb | I/O sum[0]:cur[0], unzip sum[0]:cur[0]
mariadb | ----------------------
mariadb | INDIVIDUAL BUFFER POOL INFO
mariadb | ----------------------
mariadb | ---BUFFER POOL 0
mariadb | Buffer pool size   64216
mariadb | Free buffers       48110
mariadb | Database pages     16104
mariadb | Old database pages 5964
mariadb | Modified db pages  20
mariadb | Percent of dirty pages(LRU & free pages): 0.031
mariadb | Max dirty pages percent: 75.000
mariadb | Pending reads 0
mariadb | Pending writes: LRU 0, flush list 20, single page 0
mariadb | Pages made young 0, not young 0
mariadb | 0.00 youngs/s, 0.00 non-youngs/s
mariadb | Pages read 16100, created 4, written 31
mariadb | 0.00 reads/s, 0.00 creates/s, 0.00 writes/s
mariadb | No buffer pool page gets since the last printout
mariadb | Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
mariadb | LRU len: 16104, unzip_LRU len: 0
mariadb | I/O sum[0]:cur[0], unzip sum[0]:cur[0]
mariadb | ---BUFFER POOL 1
mariadb | Buffer pool size   64216
mariadb | Free buffers       48038
mariadb | Database pages     16177
mariadb | Old database pages 5991
mariadb | Modified db pages  5
mariadb | Percent of dirty pages(LRU & free pages): 0.008
mariadb | Max dirty pages percent: 75.000
mariadb | Pending reads 0
mariadb | Pending writes: LRU 0, flush list 0, single page 0
mariadb | Pages made young 0, not young 0
mariadb | 0.00 youngs/s, 0.00 non-youngs/s
mariadb | Pages read 16177, created 0, written 8
mariadb | 0.00 reads/s, 0.00 creates/s, 0.00 writes/s
mariadb | No buffer pool page gets since the last printout
mariadb | Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
mariadb | LRU len: 16177, unzip_LRU len: 0
mariadb | I/O sum[0]:cur[0], unzip sum[0]:cur[0]
mariadb | ---BUFFER POOL 2
mariadb | Buffer pool size   64216
mariadb | Free buffers       48125
mariadb | Database pages     16089
mariadb | Old database pages 5959
mariadb | Modified db pages  0
mariadb | Percent of dirty pages(LRU & free pages): 0.000
mariadb | Max dirty pages percent: 75.000
mariadb | Pending reads 0
mariadb | Pending writes: LRU 0, flush list 0, single page 0
mariadb | Pages made young 0, not young 0
mariadb | 0.00 youngs/s, 0.00 non-youngs/s
mariadb | Pages read 16089, created 0, written 0
mariadb | 0.00 reads/s, 0.00 creates/s, 0.00 writes/s
mariadb | No buffer pool page gets since the last printout
mariadb | Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
mariadb | LRU len: 16089, unzip_LRU len: 0
mariadb | I/O sum[0]:cur[0], unzip sum[0]:cur[0]
mariadb | ---BUFFER POOL 3
mariadb | Buffer pool size   64216
mariadb | Free buffers       48078
mariadb | Database pages     16137
mariadb | Old database pages 5976
mariadb | Modified db pages  64
mariadb | Percent of dirty pages(LRU & free pages): 0.100
mariadb | Max dirty pages percent: 75.000
mariadb | Pending reads 0
mariadb | Pending writes: LRU 0, flush list 0, single page 0
mariadb | Pages made young 0, not young 0
mariadb | 0.00 youngs/s, 0.00 non-youngs/s
mariadb | Pages read 16137, created 0, written 65
mariadb | 0.00 reads/s, 0.00 creates/s, 0.00 writes/s
mariadb | No buffer pool page gets since the last printout
mariadb | Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
mariadb | LRU len: 16137, unzip_LRU len: 0
mariadb | I/O sum[0]:cur[0], unzip sum[0]:cur[0]
mariadb | ---BUFFER POOL 4
mariadb | Buffer pool size   64216
mariadb | Free buffers       48092
mariadb | Database pages     16124
mariadb | Old database pages 5972
mariadb | Modified db pages  46
mariadb | Percent of dirty pages(LRU & free pages): 0.072
mariadb | Max dirty pages percent: 75.000
mariadb | Pending reads 0
mariadb | Pending writes: LRU 0, flush list 0, single page 0
mariadb | Pages made young 0, not young 0
mariadb | 0.00 youngs/s, 0.00 non-youngs/s
mariadb | Pages read 16124, created 0, written 46
mariadb | 0.00 reads/s, 0.00 creates/s, 0.00 writes/s
mariadb | No buffer pool page gets since the last printout
mariadb | Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
mariadb | LRU len: 16124, unzip_LRU len: 0
mariadb | I/O sum[0]:cur[0], unzip sum[0]:cur[0]
mariadb | ---BUFFER POOL 5
mariadb | Buffer pool size   64216
mariadb | Free buffers       48143
mariadb | Database pages     16071
mariadb | Old database pages 5952
mariadb | Modified db pages  1
mariadb | Percent of dirty pages(LRU & free pages): 0.002
mariadb | Max dirty pages percent: 75.000
mariadb | Pending reads 0
mariadb | Pending writes: LRU 0, flush list 0, single page 0
mariadb | Pages made young 0, not young 0
mariadb | 0.00 youngs/s, 0.00 non-youngs/s
mariadb | Pages read 16071, created 0, written 1
mariadb | 0.00 reads/s, 0.00 creates/s, 0.00 writes/s
mariadb | No buffer pool page gets since the last printout
mariadb | Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
mariadb | LRU len: 16071, unzip_LRU len: 0
mariadb | I/O sum[0]:cur[0], unzip sum[0]:cur[0]
mariadb | ---BUFFER POOL 6
mariadb | Buffer pool size   64216
mariadb | Free buffers       48087
mariadb | Database pages     16129
mariadb | Old database pages 5973
mariadb | Modified db pages  0
mariadb | Percent of dirty pages(LRU & free pages): 0.000
mariadb | Max dirty pages percent: 75.000
mariadb | Pending reads 0
mariadb | Pending writes: LRU 0, flush list 0, single page 0
mariadb | Pages made young 0, not young 0
mariadb | 0.00 youngs/s, 0.00 non-youngs/s
mariadb | Pages read 16066, created 63, written 63
mariadb | 0.00 reads/s, 0.00 creates/s, 0.00 writes/s
mariadb | No buffer pool page gets since the last printout
mariadb | Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
mariadb | LRU len: 16129, unzip_LRU len: 0
mariadb | I/O sum[0]:cur[0], unzip sum[0]:cur[0]
mariadb | ---BUFFER POOL 7
mariadb | Buffer pool size   64216
mariadb | Free buffers       48064
mariadb | Database pages     16151
mariadb | Old database pages 5981
mariadb | Modified db pages  0
mariadb | Percent of dirty pages(LRU & free pages): 0.000
mariadb | Max dirty pages percent: 75.000
mariadb | Pending reads 0
mariadb | Pending writes: LRU 0, flush list 0, single page 0
mariadb | Pages made young 0, not young 0
mariadb | 0.00 youngs/s, 0.00 non-youngs/s
mariadb | Pages read 16087, created 64, written 68
mariadb | 0.00 reads/s, 0.00 creates/s, 0.00 writes/s
mariadb | No buffer pool page gets since the last printout
mariadb | Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s
mariadb | LRU len: 16151, unzip_LRU len: 0
mariadb | I/O sum[0]:cur[0], unzip sum[0]:cur[0]
mariadb | --------------
mariadb | ROW OPERATIONS
mariadb | --------------
mariadb | 0 queries inside InnoDB, 0 queries in queue
mariadb | 0 read views open inside InnoDB
mariadb | Process ID=1, Main thread ID=139874768164608, state: enforcing dict cache limit
mariadb | Number of rows inserted 0, updated 0, deleted 0, read 200
mariadb | 0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
mariadb | Number of system rows inserted 5, updated 0, deleted 4, read 9
mariadb | 0.00 inserts/s, 0.00 updates/s, 0.00 deletes/s, 0.00 reads/s
mariadb | ----------------------------
mariadb | END OF INNODB MONITOR OUTPUT
mariadb | ============================

Answer 1

Well, unfortunately, you did not include an error. Or that's what it looks like. The output that you have included is nothing out of ordinary - this is a result of a SHOW ENGINE INNODB STATUS; command, which exposes some of the internals. Based on this, it shows that you have a fairly unused server with very limited traffic. There were two transactions running, one of them got locked on some of the rows. You still have free memory in the InnoDB buffer pool. Nothing that would point towards any issues.

In general, the problem with the Galera cluster could be that the second node did not join the cluster properly. Unless all nodes are in "Primary" state, as shown in the output of SHOW STATUS LIKE 'wsrep_cluster_status'; If the node is not in the primary state, you won't be able to run queries against that node (unless you have set wsrep_dirty_reads , then you can run SELECTs , but that's another story). This is intended to ensure that you won't be getting stale reads (dataset on the non-Primary node is outdated as it is not part of the cluster).