Apache 2.4: php_network_getaddresses: getaddrinfo failed: - no DNS lookups being made

Question

Mostly posting this as a rubber duck method and for my future self.

We have a RHEL 7.4 server running Apache 2.4 (httpd24-httpd-2.4.27-8.el7.x86_64) with mod_php (no php-fpm). Today, all of the sudden about 20 of our servers started failing in our PHP app with php_network_getaddresses: getaddrinfo failed:. Doing a systemctl restart httpd24-httpd resolves the issue, (reload does not (opcache). If I monitor DNS resolution with tcpdump -i eth0 -vvv -nn udp port 53 I don't even see the DNS lookups being made, however I do see others. It appears that Apache isn't even making the DNS requests that PHP wants. These servers reboot daily and had all turned on around 7am or later so they weren't running that long (not some kind of buildup) and had been operating fine for months.

Our main concern now is that our production servers are running the same stack and although production has been fine I want to avoid this issue reaching production.

Why would Apache all of the sudden stop making DNS lookups? (It doesn't even make it to /etc/hosts)

Answer 1

I traced the issue to the fact that system patching was performed and glibc was updated and the processes (and system) were not rebooted. I rebooted the httpd service and DNS lookups were working again. I suspect it had something to do with the old glibc still being in memory along with bits of the new glibc.