The Key to ExpressionEngine Performance

Eric Lamb is the developer of professional and enterprise grade ExpressionEngine add-ons. Founded in 2009 Eric’s company mithra62 aims to be a leader in ExpressionEngine add-on development and has a reputation for stability, usefulness, and being highly configurable. Check out Eric’s popular ExpressionEngine add-on Backup Pro.

Imagine you were to build a house. After all the years of building web sites and being a slave to “the man” the pay day has finally come; you’ve sold out and you’re finally gonna hire a contractor to design and build you the house of your dreams (or your wife’s/husband’s/whatever’s dreams). You want to get everything; all the bells and whistles anyone could ever want in a house. Doors and windows and all the other house “things”. No expense is spared and it’s gonna be a marvel of engineering to be proud of.

And then you decide to build it on top of a swamp that floods every year with intermittent droughts, earthquakes, tornadoes, and hurricanes, and wasps and mosquitos. And bed bugs. And that really gross smell tanneries have hanging around. That too.

This is what a lot of web developers do. It’s really weird when you think about it too. They spend so much time putting together their project, crafting the perfect solution to whatever problem they’re trying to solve, and then just throw it up onto any old web server that’s not equipped for it. A website needs a literal solid foundation. A home to live and work that is as specific to it’s needs as possible. Yet so many web developers just throw the site up on a host and consider the project done.

The big problem is the fallacy that “all you need is PHP” a lot of clients (and devs) have in their mind. It’s not really the client’s fault though; how could they understand the nuance of using a tool like ExpressionEngine, that’s built with a PHP framework like CodeIgniter? Nope, the problem lies with us, the web developers. A lot of us just don’t know what the hell’s going on with the server.

It’s the FRAMEWORK Keeping You Down

PHP, by itself, is pretty lightweight. If you want to display a simple PHP script that connects to a database, compiles information about something, and then display the HTML, using native inline procedural PHP is pretty efficient. It’s also time consuming, likely to be insecure, tough to maintain while keeping ones joy and sanity, and just an all around pain in the ass. So, smart developers use frameworks like CodeIgniter (the framework used for ExpressionEngine).

As a programmer, frameworks are great. They’re pretty much the antithesis of the above. As ExpressionEngine is to building a website, frameworks are to building tools like ExpressionEngine. Most frameworks take security seriously, are easy(ish) to maintain, and are a pleasure to work with (or at least make an attempt to not be an absolute pain). But it usually comes at a cost; performance. It’s a given rule that any time you use a framework you’re going to need more server resources made available to your program.

The biggest reason for this is because frameworks just have a lot of PHP files that do a lot of complicated stuff behind the scenes. Pretty much everything is abstracted and object oriented and polymorphic and buzzword, buzzword, buzzword. It’s a lot of processing. And when you take into account that PHP is an interpreted language that has to compile the PHP on every request it starts to add up.

Introducing the PHP Opcode Cache

To get around dealing with the large number of PHP scripts a framework all but requires a PHP Opcode Cache (or PHP Accelerator). An Opcode Cache is a PHP module that takes your PHP scripts and caches the compiled bytecode so later requests don’t have that overhead. For those that don’t what that means, suffice it to say that using a PHP Opcode Cache will improve your PHP performance by, in my experience, anywhere from 60 to 90%. This is serious improvement I’m talking about. More than that, it’s also the most cost effective in terms of time investment and overall implementation pain.

For example, the process of getting setup is ridiculously simple using the Alternative PHP Cache (APC). On my new Ubuntu VM, it was literally 1 command:

aptitude install php-apc

That’s it. I run that single command on my server and all my ExpressionEngine sites are running at an average of 4MB on the homepages and at a tenth of a second request time using APC (down from 25MB and 1.5 seconds).

In terms of PHP Opcode Caches, I have 3 personal favorites (though there are lots of others): APC (obviously), eAccelerator, and XCache.

APC

APC is the one I’m currently using for my server. Not because it’s the best Opcode Cache out there but because, as the above demonstrates, it’s crazy simple to get up and running (I’m lazy) and my current needs don’t require more than the “good enough” it offers.

The really nice thing about APC (outside of crazy simple setup) is the maturity. It used to be implemented by Facebook (who also contributed to the project) and has been endorsed by the the people behind Zend. In fact, APC is supposed to be integrated into the core of PHP no later than 6.0 (though I think they’ve moved this to 5.5 though I could be wrong).

eAccelerator

eAccelerator has been around for about 9 years now and is my go to when shaving a couple extra MBs of memory matters. The performance return on eAccelerator is really impressive and in my experience it gets no better. eAccelerator is also the Opcode Cache use by cPanel and WHM managed servers (by default) which I think says a lot.

The big downside to eAccelerator is that it’s sort of a pain in the ass to get setup on the command line though. It requires building things custom to the platform and it also needs to be rebuilt every time you update PHP. Definitely an advanced option.

XCache

XCache is a really nice Opcode Cache in my experience. For Windows hosting it’s an absolute must and I’ve used it in production on quite a few projects. Setup is simple and straightforward and I have yet to experience any issues on installation. The performance gains are similar to APC though it does depend on what you’re doing and how it’s being done (both work differently so some things work better on one and not the other).

I have had issues with it though. I’m not entirely sure what the root cause of the issue was/is but with Zend Framework based sites XCache didn’t work out quite so well. It’s been a couple years since I’ve used it so this may be a non issue.

Additional Considerations

There are lots of other techniques you can use to improve the performance of your PHP site outside of an Opcode Cache but none can give you the return quite like it. Adding things like MySQL query caching, or memcached, or a number of other things can help improve things further but you’ll never get the return like on an Opcode Cache.

One thing to keep in mind is that an Opcode Cache can also become a crutch. It’s crucial that one not be used during development. If what you’re building needs an Opcode Cache in order to function you’ve likely made some serious errors elsewhere. Using an Opcode Cache during development will hide this from you and possibly hurt things later.

For production though, an Opcode Cache is an absolute requirement. If you have a dedicated server you’re sort of on your own in terms of implementation. Google is your friend. Things like “Ubuntu install eAccelerator” will do you good (or whatever OS you’re using). For managed hosting though, just send an email to your host and ask them to install the Opcode Cache you want. You’ll be seriously amazed at the speeds and footprint of your sites.

Eric Lamb is the developer of professional and enterprise grade ExpressionEngine add-ons. Founded in 2009 Eric’s company mithra62 aims to be a leader in ExpressionEngine add-on development and has a reputation for stability, usefulness, and being highly configurable. Check out Eric’s popular ExpressionEngine add-on Backup Pro.

Solspace on Importing Tweets in ExpressionEngine

If you’ve tried to use the Twitter API to display tweets on a website, you probably have come across a situation where tweets just didn’t display, especially if the Twitter account wasn’t regularly updated.

Solspace ran into this same problem:

Pretty quickly we learned that depending on Twitter’s search API creates a fragile system. The API limits requests per IP. If you are on a shared server then you may be sharing this limit. Additionally, Twitter’s API changes occasionally. These issues can potentially leave an empty spot on our site or even a page that won’t load. I’ve seen a situation on a client site where the client was displaying their own tweets, but they tweeted infrequently and the search API wouldn’t pull their older content, sometimes leaving the space on their site devoted to tweets empty.

and created a workaround by importing the tweets into the EE database. To follow along their tutorial you’ll need the Twitter Timeline plugin (free from EllisLab) and the Solspace Importer ($79).

The process is simple enough: create an XML feed using Twitter Timeline and then pull that in using the Importer module. You can also use DataGrab, if you prefer that add-on.

Read their tutorial: Importing Tweets Into ExpressionEngine for Reliability and Control

Streamlined Asset Serving

Aaron Gustafson of Easy! Designs posted how they handle serving up multiple asset files (CSS, javascript) in the most efficient way possible. They are particular about organizing their CSS and javascript into multiple templates but serving that up as a dozen individual files isn’t very efficient.

A while back, it was not uncommon for us to include each of these assets into the document separately, but, as website optimization and performance folks will tell you, all of that separation leads to a lot of additional overhead because the browser must request each of those files individually. In the interest of streamlining the download process, we decided to merge all of the stylesheets together at the template level before sending them over the wire.

I keep all of the assets of sites I build outside of EE templates, but I really like this way of leveraging EE templates. It’s almost slick enough for me to rethink my ways.

Read the whole article: Template-based Asset Munging in ExpressionEngine

Weightshift on Making Client Sites Faster

Scott Robbin from Weightshift posted a spot-on article about how front-end developers can improve the performance of the websites they build. He has three basic concepts you should follow (fewer file requests, download files concurrently and keep cache as long as you can) and shows examples for each one. His write-up came out of his talk at WordCamp Chicago but the techniques apply to any CMS.

Though, the talk itself was WordPress-specific, the underlying concepts are applicable to any client site, whether it be coded in WordPress, Expression Engine, MovableType or none-of-the-above. At Weightshift, we have made performance tweaking a standard part of our development cycle on all projects. I’d like to take a moment to share our process, and some of the tools that we use.

It’s not just up to backend developers and server admins to make sites run faster. The performance concerns should be addressed throughout the entire project—from IA to design to development.

Read the whole article: Making Client Sites Faster

EE Gets Smart About Managing High Traffic

Those of us that have built high traffic websites on ExpressionEngine know that you can’t just build the site, put it up on a server and hope it doesn’t buckle under the crush of visitors. All of us, I assume, implement some kind of caching in EE along with lean templates, as a way to keep the site nimble and quick. Well, we now have another tool at our disposal.

In the latest EE build, Ellis Lab has included a new feature: Tracking Preferences.  There are four preferences for disabling some of the tracking that EE does on every page load: Online User Tracking, Template Hit Tracking, Weblog Entry View tracking and Referrer Tracking.

Derek Jones from Ellis Lab explains why these new preferences are so important:

MySQL’s default storage engine (MyISAM) is designed for efficiency with reading, which is good since ExpressionEngine is a read-heavy application in terms of database usage, not write-heavy.  When writes are made to tables with the MyISAM storage engine, it must establish a lock on the entire table for the write to occur.  ExpressionEngine has a few things that it tracks to provide bits of information to the site owner.  How many times a template has been accessed.  How many users are currently online.  Where inbound links are coming from.  How many times a particular entry has been viewed.  On high traffic sites, or under extreme and unusual traffic events (i.e. Digg), these normally innocuous bits of tracking suddenly become a problem, as you have thousands of visitors hitting the site simultaneously, and for each one of those visitors, MySQL is locking the associated table to try to write to it.  Before you know it, there’s a queue of table locks and MySQL just can’t keep up.

Unless you need these tracking features, you should disable them. You will save one query per feature disabled per page load. And while that doesn’t sound like a lot, multiply those queries times a large surge in traffic and it could mean the difference between a site that is up and one that crashes and burns. I’m pretty sure we’ve all had sites that have crashed and burned.

In my day-to-day use of EE, I’ve never used Template Hit Tracking or had a client use it, so it will be the first thing I disable on every project. I will also disable Referrer Tracking (although I’m guessing just uninstalling the module will have the same effect) since I and my clients normally use a third party website statistics system.