Archive for December, 2010

Concrete5 vs WordPress: Benchmarking Load Time

December 14th, 2010

I just discovered Concrete5 CMS recently when another developer in my area launched a site with it.  Always up for learning something new, I went to the website, read the sales pitch, and decided to give it a whirl.  Before I spend time learning yet another CMS, for kicks I thought I would benchmark it for speed against WordPress, my current go-to solution.  Here we go.

Preliminaries
Hardware: All tests will be conducted on my Desktop Computer; a custom built PC with a 3.5Ghz Core2 Duo, 4GB of Ram, and a 10,000 RPM hard drive.  Not identical, but similar to many server setups on the market.

Software: I’m running Ubuntu 10.04 with Apache 2.2, PHP 5.3, and MySQL 5.1.  Once again, apart from using a Desktop OS, this is almost identical to your usual LAMP server software. For benchmarking, I will be using Siege 2.68.

Step 1: Fresh Installs
I downloaded and installed the latest version of Concrete5 (5.4.1.1) and WordPress (3.03)  Here are the screenshots of the home pages out of the box:

WordPress Fresh Install

Concrete5 Fresh Install

a

Step 2: Balancing
Okay, First thing, to be fair we need to balance the page weights.  Siege will not load all the linked resources (like CSS and Javscript), I actually only care about the html page weights.  Out of the box Concrete is 1771 bytes and WordPress is 2015 bytes.  Pretty close.  After removing several widgets from the WordPress sidebar (Those extra queries weren’t fair anyway) and adding the right amount of Lorem Ipsum, the WordPress page is now exactly 1771 bytes as well.  Perfect.

Step 3: Attack!
To stress test my desktop server I am using the following siege command:

 siege -c 50 -r 40 http://localhost/[siteurl]

This will attempt to make 50 concurrent requests to the website, and will repeat each request 40 times.  This is a total of 2000 requests to each site.

Step 4: Results
Here is the raw data from the tests:

WordPress Concrete5
Total Requests: 2000 2000
Average Response Time (seconds): 2.52 1.62
Transactions per second: 16.23 22.46
Longest Transaction (seconds): 4.8 3.13
Shortest Transaction (seconds): .10 .07
Elapsed Time 123.21 89.04

And the corresponding screenshots:

Results for WordPress

Results for Concrete5

a

Step 5: Conclusions
As we can see from the data Concrete5 outperformed WordPress by 20-30% in every measure. This is a significant amount.  What does this tell us?  For small sites with little content, Concrete5 will scale to additional concurrent users better than WordPress. What does this not tell us?  For one thing, the sites may not scale to additional content equally well.  This test also ignored all the static content which will download from either CMS with equal speed.  Finally, WordPress also has some excellent caching plugins that may have closed the gap.

Other Considerations
Am I suggesting you ditch WordPress and port your sites to Concrete?  Not at all. WordPress has many good things going for it; it is the #1 blogging tool and is a finely tuned engine that powers millions of websites.  What I CAN say for sure, is that if you can outperform that, then you’re doing something right.  Hats off to the team at Concrete.

——————————————————-

Update (1 Day Later)
Okay, After posting this, I had a lot of people point out to me that this is not a fair fight.  Concrete has caching enabled by default, and WordPress does not.   After installing wp-super-cache, the elapsed time for 2000 requests from WordPress fell from 123 seconds to 27 seconds.  Wow, crazy plugin.  Either way, my original results stand when you consider out of the box performance.  Wp-super-cache is neither bundled with WordPress nor do it’s version numbers suggest it is stable.  It’s technical options are overwhelming to all but advanced users.  Kudos still go to Concrete5 for integrating a simple, stable caching system into the framework.

Code Complete

December 12th, 2010

Whew!  This book took a little while to get through.  That was partly because it’s a healthy 862 pages, and partly because I became addicted to StarCraft II halfway through reading it 😉  Nonetheless, it is an awesome book.  Five out of five stars.  Jeff Atwood classifies this book as “The Joy of Cooking for software developers.” I would have to agree.  If there was such a thing as a modern programming Bible, this is it.

This book reads like a how-to for all aspects of software development.  It begins with preliminary things like gathering specifications, defining prerequisites, and planning.  It moves through very high-level decisions such as choice of language for a project, and how much project infrastructure to use (build tools, version control, automated testing, etc).  It covers estimation, and different techniques for planning the development and integration of large projects.

The author then moves through high level decisions within your code.  Things like class structure, interfaces, and guidelines for creating routines are covered.  He weighs pros and cons of breaking routines into sub-routines, choosing parameters to pass effectively, and vital concepts such as abstraction and encapsulation.  There is a very informative chapter on defensive programming that covers topics such as error-handling, assertions, exceptions, and debugging.

The bulk of the book is on the nitty-gritty of writing code.  Variables are covered in depth. He thoroughly covers topics such as variable scope, initialization, placement, and persistence.  He devotes an entire chapter to variable names.  Following variables is an excellent discussion of data types.  Moving right along, conditionals, loops, and other control structures are covered in great detail.

Next McConnell takes the reader through various other aspects of programming, including collaboration, testing, debugging, refactoring, and code tuning.  Finally, the book addresses topics such as the effect of project size on construction, using programming tools effectively, code documentation, and the considerations of programmer personalities themselves.

What stuck out most to me about this book (and what makes it stand out from the crowd) is the author’s effective merging of both professional experience and academic knowledge.  For things like variable names: he won’t just tell you a variable name should be between 7 and 20 letters, he’ll cite a study showing that code using variables of that length were shown to have fewer bugs.  The book is filled with pieces of “hard data” that are usually a result of a formal study that confirms what he has learned through experience.  McConnell’s bibliography is extensive, and his command of sources is impressive.

Also, what impressed me about this book is how the small topics reinforce larger themes.  One of the primary tenets of good software construction (and one that is thoroughly covered here)  is managing complexity.  Concepts such as good class structure, good routine names, and good commenting are not effective for their own sake.  They are effective because they make code less complex and easier to understand.  Programs that are broken down into smaller, less complex pieces are programs that are easier to develop, debug, and maintain.

All in all, I would strongly recommend this book to any software developer.  Even if you don’t feel like reading through a 2″ thick book with thin pages and no pictures, you can jump to any particular topic of interest and absorb what tidbit you need.  The book is also structured well enough to serve as a reference.  It will likely serve as such for me until a year or two from now when I plan on re-reading it.  It’s that good.