We are seeing some strange behavior and we are unsure whether it's an issue with apache, php, mysql or even the OS, so to the large brains of stackoverflow!
We now have Apache and mod_php speaking to some mysql5 server. Sometimes, a procedure will decide to hang, attempting to read from the file descriptor.
Firing up strace one on of these (all hanging processes demonstrated exactly the same results) gave this :
[root@prweb133v ~]# strace -p 8450 Process 8450 attached - interrupt to quit read(57, <unfinished ...>
What exactly could it have been attempting to read?
[root@prweb133v ~]# lsof -p 8450 ... ... httpd 8450 apache 57u IPv4 5546599 TCP prweb133v.local:36615->hadat.local:mysql (ESTABLISHED)
That's our mysql server! Ok, so perhaps it had been attempting to browse the outcomes of a question, I figured. Checking the processlist around the mysql server, the bond was established but in a condition of SLEEP.
So i quickly checked netstat to determine who had been attempting to send/receive what.
Around the webserver :
[root@prweb133v ~]# netstat -t -n -a | grep 36615 tcp 0 5 172.23.179.6:36615 172.23.179.67:3306 ESTABLISHED
as well as on the mysql server there is a recognised connection but within the send or receive queues.
Any idea what these mysterious 5 bytes might be, or why they at random do not get the the mysql server?
What mysql-engine are you currently using (myisam, innodb, ...)? Would you use mysql or mysqli interface around the php side?
I'd provide the "log" and "log_slow_queries" within the mysql config file an attempt (possibly to some ramdisk) together with wading trough the output for "SHOW GLOBAL STATUS" within the mysql spend (every server variable that finishes in "*_waits" or perhaps is connection related).
Have you got changed any kind within the "Fine tuning" portion of the mysql config file? Transformed some buffers?
Within the php.ini, have you got a default value (60) for mysql.connect_timeout? Setting "mysql.trace_mode" to "on" will not hurt for some time.
Additionally you may want to stress-test, if at all possible, various partsOrWeb addresses of the application having a tool like "ab" to be able to narrow it lower.
Or: When utilizing apache with prefork module, in your area start just one server ("Startservers 1", "MaxSpareServers ", something of that nature) and stress test until it dangles. Then your logs might be more value.