Using Cache Wisely with ExpressionEngine
by Ryan Battles
Whenever ExpressionEngine renders a page, a variety of things happen. A parsing engine runs through the code, database calls are made, variables are replaced, conditional statements are evaluated, third party add-on scripts are run and more. While servers are amazingly fast at doing all of this, sometimes all of these processes can make a page take too many valuable seconds to load.
While it is always best practice to write your code so that the page loads as fast as possible, there are times where caching can come in and save the day by providing lighting-fast load times even for pages with a lot of processing needs. However, there are also times where caching can work against you and actually make the page slower than loading the page without it. I’ll cover the basics of caching in ExpressionEngine, including when not to do it.
What follows is a description of five types of caching:
- Tag Caching
- Query Caching
- Template Caching
- “Morsel” Caching
- Static Page Caching
We will look at how to use each of them and investigate their pros and cons.
What is Caching?
Before we begin, I’d like to quickly explain what caching is. Whatever is being cached (an entire page, parts of a page, just a few tags) is first run through the normal ExpressionEngine parsing engine and the output is stored either in a text file on the server, or within a database entry. Later, when the page is requested again, the stored output is used to render the page (or parts of the page) in the browser instead of the page being rendered completely from scratch. This avoids the obvious overhead of MySQL queries and other processor-intensive actions.
Tag Caching
Tag caching is used to cache a particular loop of code. This caches part of the page, but not all of it. The ExpressionEngine docs state that it can be used on any tag, but it’s most common use would probably be on the Channel Entries tag:
{exp:channel:entries channel="blog" limit="10" cache="yes" refresh="30"}
The above example would save the output of that tag for 30 minutes. This is most useful if your Channel Entries loop is causing a lot of processing time (you can see the time used to execute it by enabling template debugging).
A creative use of this template tag is to combine it with a random parameter in the Channel Entries tag. For example, if you want your entries to display in a random order that changes every hour, you could construct a tag like this:
{exp:channel:entries channel="blog" limit="10" cache="yes" refresh="60" orderby="random"}
The only caveat with this approach is that the tag needs to be called in order for the cache to begin. This makes it very difficult to make something change at the top of the hour with this method because the caching won’t start until someone visits that page, but it does give you an example of how you can start using this parameter to achieve some interesting effects.
PROS: Allows you to save a portion of the page as static while allowing the rest of the page to be dynamic.
CONS: Might not save time, may even increase it depending on your server environment and complexity of tag loop. Also, if using more than one on a page, perhaps template caching is more appropriate (see below).
Query Caching
SQL Query caching is used to cache the output of each query to the database. It can be turned on at CP > System Administration > Database Settings. It cannot be used with the Channel Entries loop because each call to that loop has to check for the current time and check it against the entry date or expiration date of entries. If you don’t post-date or expire your entries, you can manually turn on this feature called “Cache Dynamic Channel Queries” in CP > Admin > Channel Administration > Global Channel Preferences.
Because this type of caching involves reading and writing files to the server’s disk, it may end up being counter-productive as database calls may actually be faster. Greg Aker (former employee of EllisLab, current employee of EngineHosting, all-around brilliant guy) has stated in a blog post that “[query caching] should not be used unless you have done a cost / benefit analysis on actual db server load vs file system I/O.” Again, the output profiler and template debugger are your friends here.
PROS: Provides a significant reduction in database load.
CONS: May cause more harm than good due to frequent reading and writing of files.
Template Caching
Template caching will cache the entire template’s output as a cached file. It is important to note that template caching will supersede individual tag caching, and will also be refreshed every time certain events happen, such as the editing of an entry or addition of a comment. Template caching is enabled by clicking on “Edit Preferences” next to each template’s name within your template manager.
A major drawback of this type of caching is that many types of dynamic content will no longer be dynamic. The exception seems to be with using conditionals and variables related to usernames. In a test template that I created and cached, I noticed that the username would change depending on who was logged in, and the “if logged_out” conditional was respected, however the “current_time” variable remained the same throughout the caching duration. I couldn’t find any official documentation on what is cached and what isn’t at a template level, so you may just have to experiment with your own setup. Generally, however, all tags will be cached within the template.
PROS: All tags are cached in one file, potentially superseding the performance benefit of multiple tag caching calls. Preserves active user status and conditionals.
CONS: Greatly reduces the possibility of dynamic content, may not be as efficient as dynamic queries due to filesystem reading and writing.
“Morsels” Caching
At this point we will diverge from some default ExpressionEngine functionality and delve into third-party solutions. “Morsels” is a term that is being borrowed from Solspace’s “Template Morsels” add-on, which pre-parses portions of template code and stores them as a database entry. It also eliminates some dynamic features of the code block. It can be extremely powerful when you know that something isn’t going to be very dynamic, but is processor intensive to run. I have used Template Morsels on a few sites now and can attest to it’s immediate performance gain when used in the right situation.
Another add-on that is in the same genre, yet has a few different feature options is Causing Effect’s CE Cache, while I have not yet had the pleasure of using this add-on, its documentation states that you can set a few more options, including specifying how you want the caching to work (file-based, database, APC, Memcache, or Memcached caching). Another interesting note is that you can escape certain parts of the caching to remain dynamic. This fact alone could make CE Cache one of the most powerful tools for improving site performance on many levels.
Both of these add-ons are commercially available through their respective sites linked to above.
PROS: Offers a more powerful, feature-rich method of caching. Avoids writing files to the hard disk, almost always improving performance.
CONS: They are not a core part of EE, and require a separate purchase.
Static Page Caching
This is the grand-daddy of all caching. It completely bypasses ExpressionEngine altogether by storing a rendered template as a single file on the server, and uses .htaccess rules to redirect the URL to the file. It doesn’t get any faster than this.
Static Page Caching is an add-on by Solspace, and is commercially available from their website. Because it completely bypasses ExpressionEngine, the performance gains are immense, but this also means that absolutely nothing is dynamic about it, and the entire page is cached, so there can be no active user data displayed on the template. For some sites, this simply will never work, but for others, it is a godsend of efficiency.
PROS: Fastest loading available.
CONS: Nothing is dynamic on the page, not a core part of EE, requires a separate purchase.
Conclusion
Caching can dramatically improve the loading of a page in ExpressionEngine, and is therefore a necessary tool to understand for any ExpressionEngine developer. However, many types of caching can simply be band-aids for poorly performing code, and should not be used as a first-line of defense when optimizing a site. First, try to optimize your code, there are many great articles(http://ee-spotlight.com/tag/optimization) on how to do that. Secondly, whenever using caching, always use diagnostic tools like the output profiler and template debugger to see exactly what caching is doing for or against you.
Post to Twitter
Erik Reagan — 10:21 on 01.04.2012
Hey Ryan,
Nice rundown of EE-centric caching options.
I’ve recently been playing with HTTP caching mechanisms like Varnish and Squid. Like the EE options, they have both Pros and Cons to them. The next iteration of our company site will be served up with via Varnish.
It’s blazingly fast but limits your front-end dynamic (member-based) details to either not being cached URLs or being pulled in via Ajax.
Bjørn Børresen — 11:20 on 01.04.2012
Nice writeup! When it comes to query caching it is also EllisLab’s official stance that it should be disabled on the majority of environments as MySQL’s native caching is almost always superior to it:
http://expressionengine.com/forums/viewthread/170479/#812281
Spamschlucker — 13:41 on 01.04.2012
Hi,
first thank you for this very useful and detailed article! I immediately bookmarked it.
There are two things I would like to add.
1st: The view-counter does not work when using query-caching or template-caching. This is a great handicap for me, because it is important for me to get the stats how often each entry was viewed. I would be glad if Ellislab would be able to implement this feature working when using cache.
2nd: Maybe the caching-issue is in some particular cases overestimated. Today server-hardware is so astonishing cheap. In most (!) cases the problem is not the serverpower but the transmitting and rendering of the frontend. Try Google Pagespeed (as Firefox-Plugin or via e.g. http://gtmetrix.com/ ). The value you get has nothing to do with your backend-performance. If you are interested in frontend-engineering I recommend reading “High Performance Web Sites” (O’Reilly Media) by Steve Souders, former frontend-engineer at yahoo.com. It’s one of the most important books for me I read in the last 10 years, really. At the beginning of the book he states: Only 20% of speed-issues have their reason in a bad backend-performance. 80% are induced by bad frontend-design.
Isko — 15:27 on 01.04.2012
What about a plug-in like CE Cache? I’ve used Static Page Caching plug-in previously and it’s excellent. Made the site fly! But, if I understand correctly, with CE Cache you can make parts of the page dynamic and this would help in instances were you need the dynamic part as well.
I’m currently pondering between these two for a project I’m working on and when I saw the title of this post, was hoping for some more insight on both.
Ryan Battles — 15:34 on 01.04.2012
@Isko - I mentioned CE Cache under the “Morsels” section above. It works much differently from Static Page Caching (STP), in that STP uses .htaccess rules to bypass EE entirely and simply load a file. This turns off all dynamic aspects of the page, hence the “Static” in page caching. CE Cache works more like a morsel that is controlled via EE capabilities, and yes, it does allow parts of the cache to be dynamic. Unless you have a good reason to use STP, I would start with CE Cache.
Isko — 15:38 on 01.04.2012
@Ryan Battles - Oh snap.. totally skipped the Morsel part as I thought it was only about the Template Morsels plug-in. Re-reading it! Thanks!