A Debugger’s Tale

Eric Lamb is the developer of professional and enterprise grade ExpressionEngine add-ons. Founded in 2009 Eric’s company mithra62 aims to be a leader in ExpressionEngine add-on development and has a reputation for stability, usefulness, and being highly configurable. Check out Eric’s popular ExpressionEngine add-on Backup Pro.

When I first released the EE Debug Toolbar to the ExpressionEngine community back in November I was supposed to be done with it. I figured the community had it so they’d pick it up. Initially, I put it together, honestly, as an experiment, something I did just to see if I could. I’d been on a Github kick for way too long and figured it would be a cool last “thing” to put out there before I got back to my commercial products and projects (a man’s gotta eat). I had absolutely no intention of the Toolbar becoming an actual project for me though.

That was 3 months ago and hundreds of man hours since (yeah, plural) split between 2 developers on this Toolbar (Christopher Imrie has matched me blow for blow since I put the Toolbar on Github). To date, we’ve had a couple major releases, a few minor releases, and are prepping to release what is now a debugging platform for ExpressionEngine websites.

The Toolbar hasn’t exactly been the smoothest of projects and, as mentioned above, I’m honestly surprised just how much of a personal investment this has taken to get where it is. There’s a lesson here I think.

Now, don’t get me wrong; the Toolbar, as a tool in it’s current state, has some “wow” to it and I’m proud and thrilled at the reception (to put it mildly). Making the ExpressionEngine debug output easier to read and presenting the data in that format is clearly useful and, considering the default method of just throwing up the output on the page, definitely needed. The problem though is that once you get past the “wow” and start to look at it as a useful debugger you start to notice a few issues that keep the Toolbar from being a really useful ExpressionEngine tool.

The first big problem (after the initial round of bug fixes) was the needle in a haystack approach CodeIgniter takes with the template debugger data. It just dumps it all together so the Toolbar, at the time, did the same. Just a long list of “stuff” that happened and it’s up to the developer to dig through and make sense of things. As a programmer, let me tell you, it kinda sucks trying to use that sea of data to find a specific issue.

So Christopher put together a neat canvas graph to highlight the progress over time of template execution. No talk or discussion, just something Christopher felt needed a’ doin’. So he did it and sent a pull request. And it was really good at showing if an issue existed.

After the graph update came time to pay the technical debt built up over having no plan or structure with the code whatsoever when it was initially built. Remember, this was an experiment first, so the couple initial builds were just made up as I went along. Not painful to work on but also not pleasant and if this was going to be an actual project the code needed to be cleaned up and structured a little better.

Again Christopher stepped up and took lead on this, making the code much, much, cleaner and easier to work on. He set things up so the jQuery was much more efficient and things were laid out inline with all good ExpressionEngine add-ons. The importance of this step should not be minimized as it set the ground work for everything to come.

Still, this question of “how useful it is” bugged me. We had a good base to work on, thanks to Christopher, but the project itself was still lacking in the “useful” department. The lowest hanging fruit to make the Toolbar usable was revisiting the graph so that’s what we did next. A static canvas graph is basically an image so what we currently had wasn’t all that helpful.

Adding Google Charts for interactivity made a lot of sense. It was an external library so we could keep the core code clean, it’s a robust library and easy to work with, and it’s distributed so the code is likely to be cached by end users. Full of win. Christopher and I have made it a point to keep the core as minimal as possible so an external and distributed library was perfect. This feature, alone, took us over 3 weeks to get just right though due to our schedules and all the other tweaks that went into the pull request.

Christopher and I started discussing possible directions and what to do next almost immediately. We were both in that heady state programmers get into when a project is humming along and wanted to get the project where it needed to be. We talked about adding in Warnings and Alerts and a Memory History profiler to track resource usage over multiple requests. Thinking about what would be cool and useful for an ExpressionEngine debugger brings up all sorts of interesting directions and approaches. And all sorts of time sinks and opportunities for code bloat.

This is when things really started to dawn on me as to the enormity of the project. How do we build out the “everything” debugger ExpressionEngine needed. How could we possibly fill all the debugging needs for an abstract tool like EE? The Toolbar is potentially going to require a huge amount of work to do right. Way more than I could possible donate on my own. No, in order to do this right we’d need the help of way, way, more developers. Bummer.

We need people like the Nerdery and Cotter Interactive to write killer themes and styles like their CP Themes so users have the option of how their debugging experience looks. We need devs like Exp:resso, ObjectiveHTML, Solspace, and Low to write custom debugging experiences for their own add-ons and tools. And we need all devs to think about their debugging experience and what tools they need. Basically, we need a community for debugging inside the ExpressionEngine community.

Christopher and I discussed this problem and decided our best course of action was to move the Toolbar into a platform model instead of the simple add-on it was. By this I mean we had to create a base for other developers and designers to customize and create their own debugging experience specific to them. A way for the Toolbar to change depending on what add-ons a site has installed or what specific needs a particular project may have. This way, Securit:ee (only $45 on devot:ee!), FreeForm, and Postmaster, and all the other ExpressionEngine add-ons can create their own panels or custom tools specific to their products debugging needs.

Our first, and easiest, move was to get the Toolbar the ability to be themed. Even from the first day of release requests were coming in for how to style things so creating themes made sense anyway. The idea was that theme designers could create a completely new and different Toolbar using CSS (and even being able to replace the entire toolbar paradigm by changing the JS if desired). Overall, this took around 5 weeks of pretty consistent development split between both Christoper and I but the end result was a Toolbar that you could completely customize in terms of the creative.

Now, up till this point (about 3 weeks at the time I’m writing this), the pull requests were all pretty straight forward. Each request iterated over the last to continue building a coherent idea in a logical and progressive manner. The Hookah branch spit in the face of that bliss and has caused a 90% rewrite of everything (I think the images were the only files not changed).

Christopher and I started the Hookah branch 3 weeks ago and it’s been an interesting direction to go with a project. If you’ve never attempted to convert an existing project to a platform let me tell you, it’s a pain in the ass. It would have been much smarter to go in this direction at initial design but, as mentioned: experiment. Still, now that we’re about 99% complete with this branch it’s clear this was the path to go.

So now it’s possible, through the Hookah branch until merge (and it ain’t ready for merge), to customize 100% of the EE Debug Toolbar. You can style it any way you want. You can create new panels, change existing panels, remove panels and create custom widgets outside the panel paradigm.

Here are a just couple examples:

Performance Alerts / Memory History

Note the highlighted buttons in the Toolbar; that’s a new extension shipped with the Toolbar to highlight where performance isn’t acceptable (and yes, those values are configurable).

Another extension that ships with the Toolbar is EEDT Memory History. This extension, shown in the top right, tracks your page performance over multiple pages so you can get an idea, visually, of which pages are a priority for performance tweaks.

Minimal / Hot Dog Stand Theme / Settings

You can also position the Toolbar however you see fit regardless of theme (previously, the position was controlled at the theme level). Don’t like it in the bottom right? Cool; go to the Toolbar Settings page and set your location.

The Toolbar uses normal ExpressionEngine extensions to do everything and Christopher and I have worked really hard to make sure the API for managing panels is as easy and intuitive as possible so programmers would actually want to write helpful extensions for their products. We’re still adding some finishing touches to the branch and we are just about done but hit us up if you have any requests.

In the meantime, now’s the time to get involved. If you have any needs for ExpressionEngine debugging I highly suggest you hit up Github and take a look at the project.

Eric Lamb is the developer of professional and enterprise grade ExpressionEngine add-ons. Founded in 2009 Eric’s company mithra62 aims to be a leader in ExpressionEngine add-on development and has a reputation for stability, usefulness, and being highly configurable. Check out Eric’s popular ExpressionEngine add-on Backup Pro.