Recently I was developing a new site for a client that had exceptionally high processor usage for every page load. Now I have other Drupal sites on this server that use a similar module setup but use a lot less processor. My interest was piqued.
After going through all of my Devel Module output (mainly the query output) I decided to fire up my Firefox add-on, livehttp headers. It turns out that the Flash that was present on every page was pulling an XML file I was generating from Drupal. Now the problem didn’t lie with the XML file I was creating, it was the fact that Flash was looking for a crossdomain.xml file when I didn’t have one. When Flash requested the non-existent crossdomain.xml file Drupal would serve the 404 page essentially causing every page request to serve up two pages at the same time with all of the overhead of the CMS (twice!).
Now in all fairness this isn’t a Drupal issue or a Flash issue, it’s a developer issue. Make sure that when your site goes live you double check your livehttp headers for 404s and you use the Firebug “net” feature to see any files that weren’t found as well. The fix for this problem was simple; I just added a crossdomain.xml file in the sites root.
Today’s moral of the story… having your .htaccess serve up a full CMS page on 404s is really bad if your going to have one or more 404s on every page. On the flip side, your Quality Assurance should catch 404s before the site goes live (even if you QA it yourself).