简体   繁体   中英

PHP socket_read/recv blocking python's socket.send

I'm building a Firebird DB transaction manager in Python on Linux with JS+PHP clients. Javascript sends all necessary information to PHP; PHP encodes this and sends it via socket to Python, which has a socket bound to a port constantly listening and creates a new thread using threading to handle that request asynchronously.

The accept loop in python:

while 1:
    conn, addr = s.accept()

    req = conn.recv(1024)
    ret = read_headers(req)

    threading.Thread(target=client_thread, args=(conn, addr, ret, smphr,)).start()
s.close()

The send/read block in php:

$sock = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
$sockconnect = socket_connect($sock, $host, $port);
$msg = urldecode(http_build_query($params));
socket_write($sock, $msg, strlen($msg));
$received;
while(socket_recv($sock, $buf, 1024, 0) >= 1){
    $received .= $buf;
}
echo $received;
socket_close($sock);

It all seems to be working properly until we started testing with larger number of connections. I have a loop in the JS client that sends several (25-100 are the numbers I've used so far) queries request with a select first random-number-of-lines from a large table.

The first few requests that the server receives are processed simultaneously but then it seems to become synchronous.

After much logging, found out that only 7/8 threads are active at any given time. New requests are only accepted and processed after one of the 7 current ones finishes.

If I comment the socket_recv while loop in php, python will then run everything simultaneously and return as soon as is available, which is exactly what I want, but since I've commented the block that actually gets the result, nothing is shown (obviously).

Every request/queries is logged as a different script call (according to chrome's network dev tool) so I don't know why they're blocking each other.

I'm fairly new at php/python and I can't for the life of me figure out what's going on.

Any suggestions?

Edit: I've also tried different bits of code to read the response in php (none worked as intended):

$buf = 'buffer';
socket_recv($sock, $buf, 1024, MSG_WAITALL);
echo $buf;

Same as previous implementation, 7/8 thread 'limit'

$buf = 'buffer';
socket_recv($sock, $buf, 1024, MSG_DONTWAIT);
echo $buf;

As the flag implies, doesn't wait for a response, therefore has no response

while ($out = socket_read($sock, 1024, PHP_NORMAL_READ)) {
    echo $out;
}

Same thread 7/8 thread limit.

Second edit:

Added Python prints, in case it helps.

With read in php :

starting select first 3000 * from receb_quotas on tr1
starting select first 1 * from receb_quotas on tr0
starting select first 1 * from receb_quotas on tr2
starting select first 1 * from receb_quotas on tr4
starting select first 3000 * from receb_quotas on tr3
starting select first 3000 * from receb_quotas on tr5
finishing tr4 (count: 1) | remaining threads: 7
finishing tr0 (count: 1) | remaining threads: 7
starting select first 150 * from receb_quotas on tr8
starting select first 3000 * from receb_quotas on tr6
finishing tr2 (count: 1) | remaining threads: 7
starting select first 1 * from receb_quotas on tr7
finishing tr7 (count: 1) | remaining threads: 7
starting select first 3000 * from receb_quotas on tr9
finishing tr8 (count: 150) | remaining threads: 7
finishing tr1 (count: 3000) | remaining threads: 6
finishing tr3 (count: 3000) | remaining threads: 5
finishing tr6 (count: 3000) | remaining threads: 4
finishing tr5 (count: 3000) | remaining threads: 3
finishing tr9 (count: 3000) | remaining threads: 2

Without php read :

starting select first 3000 * from receb_quotas on tr1
starting select first 15 * from receb_quotas on tr0
starting select first 15 * from receb_quotas on tr3
starting select first 3000 * from receb_quotas on tr4
starting select first 1500 * from receb_quotas on tr2
starting select first 150 * from receb_quotas on tr5
starting select first 1 * from receb_quotas on tr6
starting select first 1500 * from receb_quotas on tr7
starting select first 150 * from receb_quotas on tr8
starting select first 15 * from receb_quotas on tr9
finishing tr0 (count: 15) | remaining threads: 11
finishing tr3 (count: 15) | remaining threads: 10
finishing tr6 (count: 1) | remaining threads: 9
finishing tr9 (count: 15) | remaining threads: 8
finishing tr8 (count: 150) | remaining threads: 7
finishing tr5 (count: 150) | remaining threads: 6
finishing tr7 (count: 1500) | remaining threads: 5
finishing tr2 (count: 1500) | remaining threads: 4
finishing tr1 (count: 3000) | remaining threads: 3
finishing tr4 (count: 3000) | remaining threads: 2

It really does seem that without the read in php, the queries are all started at the same time and returned as soon as ready.

Turns out that this was a browser issue. Chrome (maybe other modern browsers) only support up to 6 ajax calls at the same time.

Since I had that 7 thread 'cap', one of them being the main thread. This lines up perfectly.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM