Monthly Archives: September 2016

LAMP with https causes issues with slow requests

Hey,

So I ran into a really interesting issue the other day.

I was developing an app in PHP and was preparing it for https. In this project I use sessions and cookies with the secure flag set and when I enabled https I noticed that all requests to the server took a very long time. One request took about 8-10 seconds and I could not figure out why.

I was using these versions on CentOS 7 running on the main machine.

This project is a little special because I am running the mysql-server on Remnux in a virtual machine in KVM on the main machine. Remnux is running this version of mysql-server:

I doubt you would have the issue if the mysql server ran on the same machine, I.e localhost. On this scenario the machine used 10.1.1.2 and the Remnux instance where the database is running is using 192.168.10.250.

So, as you may understand I am using 192.168.10.250 as the database in my php code.

I started to debug with strace by attaching to the httpd process:

This will print a lot of nice stuff when you suspect something is really strange. For instance, below you see the web server reading a file called dbfunctions.php.

This is how a connection to the mysql server looks like. You can see the connect() and setsocketopt() functions being called to establish a connection. It was also here I noticed a delay.

It can be useful to troubleshoot delays.

I also found a thread on stack overflow which suggested that the cause was DNS recursive requests. So I fired up tcpdump with this command

And I saw that the machine was trying to do reverse lookups of 192.168.x.x which took forever (Going out to google)

I had a discussion with a friend and after trying nginx I realised that the problem was not apache or nginx. He suggested I try this setting in the mysql-server on Remnux.

http://dev.mysql.com/doc/refman/5.7/en/server-options.html#option_mysqld_skip-name-resolve

I edited the file /etc/mysql/my.cnf and added the setting “skip-name-resolve”. Here is a snippet from my configuration file on Remnux:

After a restart of everything the issue was gone.

I hope I can help someone else by making this documented and the steps to troubleshoot public. This was really a very strange issue and took some time to troubleshoot.

Good luck!