Excuse me while I let off some steam. I’ve just spent many hours debugging an authentication issue that was preventing Google Chrome users from logging into a PHP web app we’re currently working on. Here are the gory details for your amusement.

The app uses OAuth to authenticate against the client’s central auth service. That means instead of logging in to our app directly, the process is more like signing into Twitter from a third party – we send the user away to log in, they authenticate on the central server as normal and get sent back to us with a token which we can then use to log them in to our app.

The issue we were seeing is that Chrome users were clicking the login link, being sent out to authenticate, coming back and still being told they weren’t logged in. In all other browsers, returning from a successful login allowed access the app as expected. Puzzling.

As I went through debugging, it became clear that the point at which the process was failing was when the CSRF tokens were being compared. When we create the log in link, we add a CSRF token to the URL and at the same time store the same token in the user’s PHP session. On returning from authenticating, the token is handed back to us and we can compare it to the copy in the session storage to make sure there’s no funny business going on.

Poking further, it became clear that the reason the CSRF tokens didn’t match was due to the user being given a new PHP session when they returned. So when we compared the token, there was no stored value to compare it to. Very odd indeed.

Usually, if sessions go awry, the best thing to do is turn to the raw HTTP headers. A PHP session is of course based on a cookie – the server issues a session ID in a cookie and the browser presents it back to the server with each request. If the session ID was changing, that had to mean that either the browser wasn’t sending the cookie, or the server was issuing a new one. Either way, that would show up in the request and response headers.

Using the Network tab of the Chrome web inspector, I could see the headers for each file. I could see the cookie being set with the page response, and then presented back with the stylesheet request. Clicking out to log in and coming back to my page, I could see the cookie being presented again… but wait! With a different session ID. When did that change?

Double checking the headers again, I could see that no new cookie was being sent in the response to any of the items in the Network inspector. But there are requests that happen that the network inspector doesn’t show you. Like requesting a favicon.

As this app is in development, it doesn’t have a favicon, so the request should result in a 404. Except in this case, it didn’t. The server is configured (and to be fair, misconfigured) to route any requests that are not for real files or directories to my index.php page. This is so the app can have /any/url/it/pleases/ and it just gets routed through to my app to parse internally. A front controller.

This meant that instead of getting a 404 from Apache, the favicon request was being handed back the app’s home page. Niiice. But that shouldn’t be an issue, as even a request for a favicon is sent along with the cookie information, so PHP should pick up the session and carry on as normal. But the plot thickens. Apache is sitting behind Varnish, which I’d configured to strip cookies from any request for static files such as images, CSS, JavaScript, and, you know, favicons.

That meant that by the time the request got passed through to Apache, the cookie had been stripped and the request looked like a new user. Who was then given a brand new PHP session. As soon as I put a favicon in place, bingo, Chrome users could log in.

I’m not sure if Chrome requests the favicon at the beginning of the request, or right at the end, but either way, it was enough to change the user’s session cookie between the end of one page load and the beginning of the next.

A perfect storm of a bit of misconfiguration on my part, a lack of transparency from Chrome’s network inspector, and a missing favicon.