PHP & MySQL - SQLSTATE[HY000] [2003] 無法連接到 MySQL 服務器 - 通過 TCP 的許多連接

Question

我有很多 PHP 實例（250 到 500 和更多），它們打開一個 DB-Connection 並執行 SELECT、UPDATE 和 INSERT。 幾秒鍾后，我收到以下錯誤：

PHP Fatal error:  Uncaught exception 'PDOException' with message 'SQLSTATE[HY000] [2003] Can't connect to MySQL server on 'xxx.xxx.xxx.xxx' (4)' in /var/www/xxx/vendor/propel/propel/src/Propel/Runtime/Connection/PdoConnection.php:49

我至少花了 30 個小時來尋找解決方案。 在我看來，故障在於MySQL或Debian的錯誤配置。 我找到了各種配置選項，但它們都不起作用。

---

附加信息

• 該錯誤僅發生在遠程服務器上 - 當我直接在 DB-Server 上啟動腳本時，我沒有收到任何錯誤。

---

當前配置

配置文件

net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
# Disable netfilter on bridges.
#net.bridge.bridge-nf-call-ip6tables = 0
#net.bridge.bridge-nf-call-iptables = 0
#net.bridge.bridge-nf-call-arptables = 0

net.ipv4.ip_forward = 1
net.ipv4.neigh.default.gc_thresh1 = 4096
net.ipv4.neigh.default.gc_thresh2 = 8192
net.ipv4.neigh.default.gc_thresh3 = 16384
net.ipv4.neigh.default.gc_interval = 5
net.ipv4.neigh.default.gc_stale_time = 120
net.core.netdev_max_backlog = 262144
#net.core.rmem_default = 16777216
net.core.rmem_max = 108544
net.core.somaxconn = 32768
net.core.wmem_max = 108544

net.netfilter.nf_conntrack_max = 10000000
net.netfilter.nf_conntrack_tcp_timeout_established = 40
net.netfilter.nf_conntrack_tcp_timeout_close = 10
net.netfilter.nf_conntrack_tcp_timeout_close_wait = 10
net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 10
net.netfilter.nf_conntrack_tcp_timeout_last_ack = 10
net.netfilter.nf_conntrack_tcp_timeout_syn_recv = 10
net.netfilter.nf_conntrack_tcp_timeout_syn_sent = 10
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 10

net.ipv4.tcp_fin_timeout = 1
net.ipv4.tcp_max_orphans = 262144
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_max_syn_backlog = 262144
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_sack = 0
net.ipv4.tcp_syn_retries = 2
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syncookies = 0
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_wmem = 4096 16384 16777216

等/mysql/my.cf

[client]
port        = 3306
socket      = /var/run/mysqld/mysqld.sock

[mysqld_safe]
socket      = /var/run/mysqld/mysqld.sock
nice        = 0

[mysqld]
user        = mysql
pid-file    = /var/run/mysqld/mysqld.pid
socket      = /var/run/mysqld/mysqld.sock
port        = 3306
basedir     = /usr
datadir     = /var/lib/mysql
tmpdir      = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking

bind-address        = {local server ip}

key_buffer      = 16M
max_allowed_packet  = 16M
thread_stack        = 192K
thread_cache_size       = 1100
myisam-recover         = BACKUP
max_connections        = 1000
#table_cache            = 64
#thread_concurrency     = 10

query_cache_limit   = 2G
query_cache_size        = 32M

#general_log_file        = /var/log/mysql/mysql.log
#general_log             = 1
log_error = /var/log/mysql/error.log
slow_query_log_file = /var/log/mysql/mysql-slow.log
slow_query_log      = 1
long_query_time = 2
log_queries_not_using_indexes
#server-id      = 1
#log_bin            = /var/log/mysql/mysql-bin.log
expire_logs_days    = 10
max_binlog_size         = 100M
#binlog_do_db       = include_database_name
#binlog_ignore_db   = include_database_name

innodb_buffer_pool_size = 16G
innodb_buffer_pool_instances = 16

[mysqldump]
quick
quote-names
max_allowed_packet  = 16M

[mysql]
#no-auto-rehash # faster start of mysql but no tab completition

[isamchk]
key_buffer      = 16M

!includedir /etc/mysql/conf.d/

加載模塊 - modules.conf

nf_conntrack
nf_conntrack_ipv4

啟動時啟動腳本

echo "1" > /proc/sys/net/ipv4/tcp_tw_reuse
echo "1" > /proc/sys/net/ipv4/tcp_tw_recycle
echo "15" > /proc/sys/net/ipv4/tcp_fin_timeout
/etc/init.d/networking restart

---

我的系統

• Debian 8.2
• MySQL 5.5.44-0+deb8u1
• php5-mysql 5.6.13+dfsg-0+deb8u1
• 具有 64GB RAM、48 個 CPU 的虛擬機（Intel Xeon E7540 at 2.0 GHz）
• 在 ESxi-Host v. 6.0.0 上運行 2494585

負載測試

啟動 x PHP-instances / loadTester.php

<?php
    require_once __DIR__ . "/autoloader.php";

    $numberOfInstances = 100;
    $index = 0;
    $numberOfDataset = 500;
    while($index < $numberOfInstances) {
        @system( "php " . __DIR__ . "/loadTest.php $index $numberOfDataset > /dev/null &" );

        $index++;
    }

一個實例 SELECT / loadTest.php

<?php
    require_once __DIR__ . "/autoloader.php";

    $skip = $argv[ 1 ];
    $numberOfDataset = $argv[ 2 ];

    $loops = 100;
    $index = 0;
    $skip = $skip * $numberOfDataset;
    while ( $index < $loops ) {
        $adresses = AddressQuery::create()->offset( $skip )->limit( $numberOfDataset )->find();
    }

網絡圖

網絡圖

Answer 1

我找到了“可能是 TCP/IP 問題”的解決方案：

我為 MySQL 服務器創建了一個 SSH 隧道 - 遠程服務器的 CPU 負載現在降低了 10 倍，錯誤不再出現並且連接現在已加密。

在遠程服務器上設置

我創建了一個 bash 腳本，它在每次啟動時運行：

ssh -fNg -L 3307:{local server ip}:3306 -p {redirect port} root@{public ip}

在我的路由器上，我配置了到虛擬數據庫服務器的轉發 -> {public ip} {redirect port}