Rethinking Blogs at The New York Times

The New York Times

See Also: The Technology Behind the NYTimes.com Redesign

The Blogs at the Times have always run on WordPress. The New York Times, as an ecosystem, does not run on one platform or one technology. It runs on several. There are over 150 developers at the Times split across numerous teams: Web Products, Search, Blogs, iOS, Android, Mobile Web, Crosswords, Ads, BI, CMS, Video, APIs, Interactive News, and the list goes on. While PHP is frequently used, Elastic Search and Node make an appearance, and the Newspaper CMS, “Scoop,” is written in Java. Interactive likes Ruby/Rails.

The “redesign,” which launched last week, was really a re-platform: where Times development needs to head, and a rethinking of our development processes and tools. The customer-facing redesign was 2 main pieces:

  • a new Article “app” that runs inside of our new platform
  • the “reskinning” of our homepage and section fronts

What is launching today is the re-platform of Blogs from a WordPress-only service to Blogs via WordPress as an app inside of our new platform.

The Redesign

Most people who use the internet have visited an NYTimes article page –

the old design:
http://www.nytimes.com/2013/12/29/arts/music/lordes-royals-is-class-conscious.html

Lorde

the new:
http://www.nytimes.com/2014/01/15/arts/music/jay-z-offers-a-view-of-his-legacy-at-barclays-center.html?ref=music

Jay-Z at Barclay's

What is not immediately obvious to the reader is how all of this works behind the scenes.

Non-Technical

To skip past all of the technical details, click here:

How Things Used to Work

For many years at the Times, article pages were generated into static HTML files when published. This was good and bad. Good because: static files are lightning fast to serve. Bad because: those files point at static assets (CSS, JavaScript files) that can only change when the pages are re-generated and re-published. One way around this was to load a CSS file that had a bunch of @import statements (eek), with a similar loading scheme for JS (even worse).

Blogs used to load like any custom WordPress project:

  • configured as a Multisite install (amassing ~200 blogs over time)
  • lots of custom plugins and widgets
  • custom themes + a few child themes

A lot of front-end developers also write PHP and vice versa. At the Times, in many instances, the team working on the Blogs “theme” was not the same team working on the CSS/JS. So, we would have different Subversion repos for global CSS, blogs CSS; different repos for global JS, blogs JS; and a different repo for WordPress proper. When I first started working at the Times, I had to create a symlink farm of 7 different repos that would represent all of the JS and CSS that blogs were using. Good times.

On top of that, all blogs would inherit NYTimes “global” styles and scripts. A theme would end up inheriting global styles for the whole project, global styles for all blogs, and then sometimes, a specific stylesheet for the individual blog. For CSS, this would sometimes result in 40-50 (sometimes 80!) stylesheets loading. Not good.

WordPress would load jQuery, Prototype, and Scriptaculous with every request (I’m pretty sure some flavor of jQuery UI was in there too). As a result, every module within the page would just assume that our flavor of jQuery global variable NYTD.jQuery was available anywhere, and would assume that Prototype.js code could be called at will. (Spoiler alert: that was a bad idea.)

WordPress does not use native WP comments. There is an entire service at the Times called CRNR (Comments, Ratings, and Reviews) that has its own user management, taxonomy management, and community moderation tools. Modules like “CRNR” would provide us with code to “drop onto the page.” Sometimes this code included its own copy of jQuery, different version and all.

Widgets on blogs could be tightly coupled with the WordPress codebase, or they could be some code that was pasted into a freeform textarea from some other team. The Interactive News team at the Times would sometimes supply us code to “drop into the C-Column” – translation: add a widget to the sidebar. These “interactives” would sometimes include their own copy jQuery (what version…? who knows!).

How Things Work Now

The new platform has 2 main technologies at its center: the homegrown Madison Framework (PHP as MVC), and Grunt, the popular task runner than runs on Node. Our NYT codebase is a collection of several Git repos that get built into apps via Grunt and deployed by RPMs/Puppet. For any app that wants to live inside of the new shell (inherit the masthead, “ribbon,” navigation automatically), they must register their existence. After they do, they can “inherit” from other projects. I’ll explain.

Foundation

Foundation is the base application. Foundation contains the Madison PHP framework, the Magnum CSS/Responsive framework, and our base JavaScript framework. Our CSS is no longer a billion disparate files – it is LESS manifests, with plenty of custom mixins, that compile into a few CSS files. At the heart of our JS approach is RequireJS, Hammer, SockJS and Backbone (authored by Times alum Jeremy Ashkenas).

Madison is an MVC framework that utilizes the newest and shiniest OO features of PHP and is built around 2 main software design patterns: the Service Locator pattern (via Pimple), and Dependency Injection. The main “front” of any request to the new stack goes through Foundation, as it contains the main controller files for the framework. Apps register their main route via Apache rewrite rules, Madison knows which app to launch by convention based on the code that was deployed via the Grunt build.

Shared

Shared is collection of reusable modules. Write a module once, and then allow apps to include them at-will. Shared is where Madison’s “base” modules exist. Modules are just PHP template fragments which can include other PHP templates. Think of a “Page” module like so:

Page
- load Top module
- load Content module
- load Bottom module

Top (included in Page)
- load Styles module
- load Scripts module
- load Meta module

...

In your app code, if you try to embed a module by name, and it isn’t in your app’s codebase, the framework will automatically look for it in Shared. This is similar to how parent and child themes work in WordPress. This means: if you want to use ALL of the default modules, only overriding a few, you need to only specify the overriding modules in your app. Let’s say the main content of the page is a module called “PageContent/Thing” – you would include the following in your app to override what is displayed:

// page layout
$layout = array(
    'type' => 'Page',
    'name' => 'Page',
    'modules' => array(
        array(
            'type' => 'PageContent',
            'name' => 'Thing'
        ),
        .....
    )
);

// will first look in
nyt5-app-blogs/Modules/PageContent/Thing.tpl.php
// if it doesn't find it
nyt5-shared/PageContent/php/src/Thing.tpl.php

So there’s a lot happening, before we even get to our Blogs app, and we haven’t even really mentioned WordPress yet!

App-specific

Each app contains a build.json file that explains how to turn our app into a codebase that can be deployed as an application. Each app might also have the following folder structure:

js/
js/src
js/tests
less/
php/
php/src
php/tests

Our build.json files lists our LESS manifests (the files to build via Grunt) and our JS mainifests (the files to parse using r.js/Require). Our php/src directory contains the following crucial pieces:

Module/ <-- contains our Madison override templates
WordPress/ <-- contains our entire WP codebase
ApplicationConfiguration.php <-- optional configuration
ApplicationController.php <-- the main Controller for our app
wp-bootstrap.php <-- loads in global scope to load/parse WordPress

The wp-bootstrap.php file is the most interesting portion of our WordPress app, and where we do the most unconventional work to get these 2 disparate frameworks to work together. Before we even load our app in Madison proper, we have already loaded all of WordPress in an output buffer and stored the result. We can then access that result in our Madison code without any knowledge of WordPress. Alternately, we can use any WP code inside of Madison. Madison eschews procedural programming and enforces namespace-ing for all classes, so collisions haven’t happened (yet?).

Because we are turning WP content in Module content, we no longer want our themes to produce complete HTML documents: we only to produce the “content” of the page. Our Madison page layout gives us a wrapper and loads our app-specific scripts and styles. We have enough opportunities to override default template stubs to inject Blog-specific content where necessary.

In the previous incarnation of Blogs, we had to include tons of global scripts and styles. Using RequireJS, which leans on Dependency Injection, we ask for jQuery in any module and ensure that it only loads once. If we in fact do need a separate version somewhere, we can be assured that we aren’t stomping global scope, since we aren’t relying on global scope.

Using LESS imports instead of CSS file imports, we can modularize our code (even using 80 files if we want!) and combine/minify on build.

Loading WordPress in our new unconventional way lets us work with other teams and other code seamlessly. I don’t need to include the masthead/navigation markup in my theme. I don’t even need to know how it works. We can focus on making blogs work, and inherit the rest.

What I Did

For the first few months of the project, I was able to work in isolation and move the Blogs codebase from SVN to Git. I was happy that we were moving the CSS to LESS and the JS to Require/Backbone, so I took all of the old files and converted them into those modern frameworks. The Times had 3 themes that I was given free reign to rewrite and squish into one lighter, more flexible theme. Since the Times has been using WordPress since 2005, there was code from the dark ages of the internet that I was able to look at with fresh eyes and transition. Once a lot of the brute force initial work was done, I worked with a talented team of people to integrate some of the Shared components and make sure we had stylistic parity between the new Article pages and Blogs.

To see some examples in action, a sampling:

Dealbook

Bits

Well

The Lede

City Room

ArtsBeat

Public Editor’s Journal

Paul Krugman

Installing PHP 5.4 (like a boss) with MacPorts

PHP 5.4.7 is the latest stable release of PHP. WordPress has a minimum required version of 5.2.6. Most developers aren’t using the PHP 5.4 branch. Actually, most aren’t even rocking PHP 5.3. This disgusts me.

PHP 5.3 added support for closures. If you come from the world of JavaScript, you know how useful they can be. If you have used PHP 5.3 and closures in classes, you will be happy to know that PHP 5.4 allows you to use $this in closures in class methods.

If you haven’t messed around with PHP 5.3, you can install these MacPorts to get started:

php5 +apache2+fastcgi+pear
php5-apc
php5-curl
php5-gd
php5-http
php5-iconv
php5-imagick
php5-mbstring
php5-mcrypt
php5-memcached
php5-mysql +mysqlnd
php5-openssl
php5-tidy

If you already have PHP 5.3 and want to upgrade to PHP 5.4, these are some tricks to get you on the right path:

sudo -s // use sudo mode throughout

port uninstall php5
// won't work if you have extensions installed,
// so uninstall everything that has PHP5 as a dependency first

port install php54
cd /opt/local/etc/php54 && sudo cp php.ini-development php.ini

Install a bunch of PHP extensions:

port install php54-apc php54-curl php54-gd php54-http php54-iconv php54-imagick php54-mbstring php54-mcrypt php54-memcached php54-mysql php54-openssl php54-tidy

To use mysqlnd with a local MySQL server, edit /opt/local/etc/php54/php.ini and set

mysql.default_socket, mysqli.default_socket and pdo_mysql.default_socket
to

/opt/local/var/run/mysql5/mysqld.sock

Make sure PHP 5.4.6 is the default PHP binary:

which php

If it’s something like /usr/bin/php:

cd /usr/bin && sudo rm -rf php
sudo ln -s /opt/local/bin/php54 php

You now have PHP 5.4.6 and your extensions, but you no longer have the apache variant.

port install php54-apache2handler

cd /opt/local/apache2/modules
sudo /opt/local/apache2/bin/apxs -a -e -n php5 mod_php54.so

vi /opt/local/apache2/conf/httpd.conf (remove the old php5.so)

You now have PHP 5.4 and the apache handler, but you no longer have the PEAR variant. You can try to make this work:

port install pear-PEAR

Or you can do the following:

cd #
curl http://pear.php.net/go-pear.phar -o go-pear.phar
sudo php go-pear.phar

You will prompted to specify config vars, we want to change #1 and #4.

Press 1 – Installation base ($prefix) – and enter:

/opt/local/lib/php54

Press 4 – Binaries directory – and enter:

/opt/local/bin

More checks for PEAR:

pear info pear && rm go-pear.phar
pear config-set auto_discover 1

// make sure PEAR is in the PHP include path
pear config-get php_dir

// if you don't see "/opt/local/lib/php54/share/pear" in there
php54 -i|grep 'php.ini'
// you should see "/opt/local/etc/php54" - if you don't:
sudo vi php.ini
// change include_path to:
include_path = ".:/opt/local/lib/php54/share/pear"

PEAR is installed, let’s install some PEAR stuffs:

// Unit tests
pear install pear.phpunit.de/PHPUnit
// Documentation generator
pear install pear.apigen.org/apigen

Restart Apache:

sudo /opt/local/apache2/bin/apachectl restart

You can start Apache and Memcached, et al by using commands like:

sudo port load apache2
sudo port unload apache2

sudo port load memcached
sudo port unload memcached

// memcached debugging, start with:
memcached -vv

WP + You + OOP

A lot of the programming associated with WordPress is inherently not object-oriented. Most of the theme mods or plugins that developers write for their blog(s) are one-off functions here or there, or a small set of filters and actions with associated callback functions. As WordPress matures into an “application framework” (everyone else’s words, not mine), the need for better code organization, greater maintainability, and the self-documenting powers of Object Oriented Programming become immediately apparent.

Because a majority of WordPress sites aren’t complex, the audience for discussions like this are small. Probably 99% of WordPress installs don’t need extra PHP code for them to work how their site owner(s) want. A majority of sites don’t use Multisite. A majority of sites have no need for Web Services, and when they do: they just install some Twitter widget or the like that does the heavy lifting for them. But I don’t think anyone involved with WordPress core wants that to be the future of WordPress. When people talk about the future of WordPress, they talk about how it can run any web application, but there aren’t a large number of compelling examples of WP doing that yet.

Almost by accident, I think eMusic has become a great example of how to not only run WordPress at scale, but how to write a site using WordPress as an application framework, and I have many examples of how we organize our code that can help anyone else who is struggling to make sense of a pile of code that an entire team needs to decipher and maintain.

Before we dig into how to write better OO code, we need to first figure out how we are going to organize our codebase.

Some Setups Tips

  • Run WordPress as an svn:external: This should almost be mandatory. You want your directory structure to look like so:
    /index.php
    /wp-config.php
    /wordpress
    /wp-content
    /wp-content/themes
    /wp-content/plugins
    /wp-content/mu-plugins
    /wp-content/requests
    /wp-content/site-configs
    /wp-content/sunrise.php
    
    // in the root of your site
    svn propedit svn:externals .
    
    // add this code
    wordpress http://core.svn.wordpress.org/branches/3.4/

    This is important so that you never overwrite core, and so you can’t check-in whatever hacks you have added while debugging core code.

    Because we are using an external, you need to add these lines to wp-config.php:

    define( 'WP_CONTENT_URL', 'http://' . DOMAIN_CURRENT_SITE . '/wp-content' );
    define( 'WP_CONTENT_DIR', $_SERVER['DOCUMENT_ROOT'] . '/wp-content' );

    You also need to alter index.php to look like this:

    define( 'WP_USE_THEMES', true );
    
    /** Loads the WordPress Environment and Template */
    require( './wordpress/wp-blog-header.php' );
  • Use Sunrise:
    If you are using Multisite, Sunrise is the best super-early place to hook in and alter WordPress. In wp-config.php:

    define( 'SUNRISE', 1 );

    You then need to a file in wp-content called sunrise.php.

  • Use Site Configs:
    One of the main things we use sunrise.php for is site-specific configuration code. I made a folder in wp-content called site-configs that houses files like global.php (all sites), emusic.php (site #1), emusic-bbpress.php (site #2), etc
  • Separate HTTP Requests:
    I made a folder in wp-config called requests that houses site and page-specific HTTP logic. Because a big portion of our main site is dynamic and populates its data from Web Services, it makes sense to organize all of that logic in one place.
  • Use “Must Use” plugins:
    If you have classes or code that are mandatory for your application, you can autoload them by simply placing each file in your wp-content/mu-plugins folder. If your plugin requires a bunch of extra files: it is not a good candidate for mu-plugins.

Use classes to encapsulate plugins and theme configs

MOST plugins in the WordPress plugin repository are written using procedural code – meaning, a bunch of function and global variables (whadup, Akismet!). Hopefully you know enough about programming to know that is a bad idea. 1) Global variables suck and are easily over-writable and 2) PHP will throw a fatal error if you try to overload a function (declare a function with the same name twice).

Because you have to protect your function names against this, most procedural plugin authors namespace their functions by prepending an identifier to their function names:

function my_unique_plugin_name_woo_hoo( ) {
    return 'whatever';
}
// call me maybe
my_unique_plugin_name_woo_hoo( );

If you declare a bunch of function callbacks for actions and filters in your plugin, you can see how this would be gross:

function my_unique_plugin_name_alter_the_content() { ... }
add_filter( 'the_content', 'my_unique_plugin_name_alter_the_content' );

function my_unique_plugin_name_red() { ... }
add_filter( 'the_content', 'my_unique_plugin_name_red' );

function my_unique_plugin_name_green() { ... }
add_filter( 'the_content', 'my_unique_plugin_name_green' );

function my_unique_plugin_name_blue() { ... }
add_filter( 'the_content', 'my_unique_plugin_name_blue' );

If we instead use an OO approach, we can add all of our functions to a class as methods, ditch the namespacing, and group our filters and actions together into one “init” method.

class MyUniquePlugin {
    function init() {
        add_filter( 'the_content', array( $this, 'alter_the_content' ) );
        add_filter( 'the_content', array( $this, 'red' ) );
        add_filter( 'the_content', array( $this, 'green' ) );
        add_filter( 'the_content', array( $this, 'blue' ) );
    }

    function alter_the_content() { ... }
    function red() { ... }
    function green() { ... }
    function blue() { ... }
}

How we call this class is up for debate, and I will discuss this at length later. But for right now, let’s call it like this:

$my_unique_plugin = new MyUniquePlugin;
$my_unique_plugin->init();

The init method is used like a constructor, but it ditches any ambiguity between __construct() and class MyPlugin { function MyPlugin() {} }. Could you use __construct() in any of my examples throughout instead of init() now that we are all in PHP5 syntax heaven? Probably. However, we don’t really want to use either, because we don’t want to give anyone the impression that our plugin classes can be called at will. In almost every situation, plugin classes should only be called once, and this rule should be enforced in code. I’ll show you how.

The Singleton Pattern

The Singleton Pattern is one of the GoF (Gang of Four) Patterns. This particular pattern provides a method for limiting the number of instances of an object to just one.

class MySingletonClass {
    private static $instance;
    private function __construct() {}
    public static function get_instance() {
        if ( !isset( self::$instance ) )
            self::$instance = new MySingletonClass();

        return self::$instance;
    }
}

MySingletonClass::get_instance();

Why do we care about limiting the number of instances to one? Think about a class that encapsulates code used to connect to a database. If the database connection is made in the constructor, we should share that connection across all instances of the class, we shouldn’t try to open a connection every time we need to make a SQL query.

For WordPress plugin classes, we want to store all of our actions and filters in a constructor or a class’s init() method. We don’t want to register those filters and actions more than once. We also don’t need or want multiple instances of our plugin class. This makes a plugin class a perfect candidate to implement the Singleton pattern.

class MyUniquePlugin {
    private static $instance;
    private function __construct() {}
    public static function get_instance() {
        if ( !isset( self::$instance ) ) {
            $c = __CLASS__;
            self::$instance = new $c();
        }

        return self::$instance;
    }

    function init() {
        add_filter( 'the_content', array( $this, 'alter_the_content' ) );
        add_filter( 'the_content', array( $this, 'red' ) );
        add_filter( 'the_content', array( $this, 'green' ) );
        add_filter( 'the_content', array( $this, 'blue' ) );
    }

    function alter_the_content() { ... }
    function red() { ... }
    function green() { ... }
    function blue() { ... }
}

MyUniquePlugin::get_instance();

// or, store the value of the class invocation
// to call public methods later
$my_plugin = MyUniquePlugin::get_instance();

Ok, so great, we implemented Singleton and limited our plugin to only one instance. We want to do this for all of our plugins, but it would be great if there was a way to not repeat code in every plugin, namely all of the class members / methods needed to implement Singleton:

    private static $instance;
    private function __construct() {}
    public static function get_instance() {
        if ( !isset( self::$instance ) ) {
            $c = __CLASS__;
            self::$instance = new $c();
        }

        return self::$instance;
    }

To do so, we are going to need a base or intermediate class that can be extended. Here is a base class, but we have a few problems:

class BaseSingleton {
    // this won't work, since $instance will get overwritten
    // every time BaseSingleton is instantiated by a sub-class
    private static $instance;

    // this won't work, because the child class
    // needs to be able to call parent::__construct,
    // meaning the parent constructor has to be as visible
    // as the child - the child has to have >= visibility
    private function __construct() {}

    public static function get_instance() {
        if ( !isset( self::$instance ) ) {
            // this won't work, because __CLASS__ refers
            // to BaseSingleton,
            // not the class extending it at runtime
            $c = __CLASS__;
            self::$instance = new $c();
        }

        return self::$instance;
    }
}

Let’s try to fix is:

class BaseSingleton {
    // store __CLASS__ = (instance of class) as key => value pairs
    private static $instance = array();

    // let the extending class call the constructor
    protected function __construct() {}

    public static function get_instance( $c = '' ) {
        if ( empty( $c ) ) 
            die( 'Class name is required' );
        if ( !isset( self::$instance[$c] ) )
            self::$instance[$c] = new $c();

        return self::$instance[$c];
    }
}

We’re getting closer, but we have some work to do. We are going to use OO features of PHP and some new stuff in PHP 5.3 to make a base class that implements Singleton and works the way we want (we don’t want to do this hack: ClassName::get_instance( 'ClassName' ) ).

Abstract Classes

Abstract classes are kinda like Interfaces:

Classes defined as abstract may not be instantiated, and any class that contains at least one abstract method must also be abstract. Methods defined as abstract simply declare the method’s signature – they cannot define the implementation.

When inheriting from an abstract class, all methods marked abstract in the parent’s class declaration must be defined by the child; additionally, these methods must be defined with the same (or a less restricted) visibility.

Here’s an example:

abstract class BaseClass {
    protected function __construct() {}
    abstract public function init();
}

As we can see, BaseClass does nothing except provide a blueprint for how to write our extending class. Let’s alter it by adding our Singleton code:

abstract class BasePlugin {
    private static $instance = array();
    protected function __construct() {}

    public static function get_instance( $c = '' ) {
        if ( empty( $c ) ) 
            die( 'Class name is required' );
        if ( !isset( self::$instance[$c] ) )
            self::$instance[$c] = new $c();

        return self::$instance[$c];
    }

    abstract public function init(); 
}

Our base class now has the following properties:

  • Declared abstract, cannot be called directly
  • Encapsulates Singleton code
  • Stores class instances in key => value pairs (we’re not done with this)
  • Instructs the extending / child class to define an init() method
  • Hides the constructor, can only be called by itself or a child class

Here’s an example of a plugin extending BasePlugin:

class MyPlugin extends BasePlugin {
    protected function __construct() {
        // our parent class might
        // contain shared code in its constructor
        parent::__construct();
    }

    public function init() {
        // implemented, but does nothing
    }
}
// create the lone instance
MyPlugin::get_instance( 'MyPlugin' );

// store the instance in a variable to be retrieved later:
$my_plugin = MyPlugin::get_instance( 'MyPlugin' );

Here’s is what is happening:

  • MyPlugin is extending BasePlugin, inheriting all of its qualities
  • MyPlugin implements the required abstract function init()
  • MyPlugin cannot be instantiated with the new keyword, the constructor is protected
  • MyPlugin is instantiated with a static method, but because an instance is created, $this can be used throughout its methods

We’re almost done, but we want to call get_instance() without our hack (passing the class name).

Late Static Binding in PHP 5.3

get_called_class() is a function in PHP 5.3 that will give us the name of the child class that is calling a parent class’s method at runtime. The class name will not be resolved using the class where the method is defined but it will rather be computed using runtime information. It is also called a “static binding” as it can be used for (but is not limited to) static method calls.

Here’s an easy example to explain how this works:

class A {
    public static function name() {
        echo get_called_class();
    }
}

class B extends A {}

Class C extends B {}

C::name(); // outputs "C"

get_called_class() is new to PHP 5.3. WordPress only requires PHP 5.2, so you might need to upgrade to be able to implement this class like so:

abstract class BasePlugin {
    private static $instance = array();
    protected function __construct() {}
    public static function get_instance() {
        $c = get_called_class();
        if ( !isset( self::$instance[$c] ) ) {
            self::$instance[$c] = new $c();
            self::$instance[$c]->init();
        }

        return self::$instance[$c];
    }

    abstract public function init();
}

Now we can instantiate our plugin like so:

// create the lone instance
MyPlugin::get_instance();

// store the instance in a variable to be retrieved later:
$my_plugin = MyPlugin::get_instance();

Base Classes for Plugins and Themes

Just like it says: USE BASE CLASSES for plugins AND themes. If you aren’t using Multisite, you probably don’t have the problem of maintaining code across parent and child themes that might be shared for multiple sites. Each theme has a functions.php file, and that file, in my opinion, should encapsulate your theme config code (actions and filters, etc) in a class. Parent and Child themes are similar to base and sub classes.

Because we don’t want to repeat ourselves, code that would be copied / pasted into another theme to inherit functionality should instead be placed in a BaseTheme class. Our theme config classes should also implement Singleton.

One of the pieces of functionality we need to share across themes at eMusic is regionalization. Regionalization is accomplished by using a custom taxonomy “region” and some custom actions and filters. For a theme to be regionalized, it needs to override some class members and call BaseTheme::regionalize().

Here’s part of our BaseTheme class:

abstract class BaseTheme implements Singleton {
    var $regions_map;
    var $regions_tax_map;
    var $post_types = array( 'post' );

    private static $instance = array();

    public static function get_instance() {
        $c = get_called_class();
        if ( !isset( self::$instance[$c] ) ) {
            self::$instance[$c] = new $c();
            self::$instance[$c]->init();
        }

        return self::$instance[$c];
    }

    protected function __construct() {}
    abstract protected function init();

    protected function regionalize() {
	add_filter( 'manage_posts_columns', array( $this, 'manage_columns' ) );
	add_action( 'manage_posts_custom_column', array( $this, 'manage_custom_column' ), 10, 2 );
	add_filter( 'posts_clauses', array( $this, 'clauses' ), 10, 2 );
	add_filter( 'manage_edit-post_sortable_columns',array( $this, 'sortables' ) );
	add_filter( 'pre_get_posts', array( $this, 'pre_posts' ) );
    }

    ...
}

Here is a theme extending it in its functions.php file:

if ( ! isset( $content_width ) )
    $content_width = 448;

class Theme_17Dots extends BaseTheme {
    function __construct() {
        global $dots_regions_tax_map, $dots_regions_map;

        $this->regions_map = $dots_regions_map;
        $this->regions_tax_map = $dots_regions_tax_map;

        parent::__construct();
    }

    function init() {
        $this->regionalize();

        add_action( 'init', array( $this, 'register' ) );
        add_action( 'after_setup_theme', array( $this, 'setup' ) );
        add_action( 'add_meta_boxes_post', array( $this, 'boxes' ) );
        add_action( 'save_post', array( $this, 'save' ), 10, 2 );
        add_filter( 'embed_oembed_html', '_feature_youtube_add_wmode' );
    }

    ....
}

“Must Use” Plugins

Let’s assume we stored our BaseTheme and BasePlugin classes in files called class-base-theme.php and class-base-plugin.php. The next question is: where should these  files go? Probably the best place for them to go is wp-content/mu-plugins. The WordPress bootstrap routine will load files in that directory automatically. Because our BaseTheme class is not immediately invoked within the file, the loading of the file just makes the class available for our theme child class in functions.php to extend.

Sunrise

sunrise.php can be viewed as a functions.php for your whole network. Aside from letting you switch the context of $current_blog and $current_site, you can also start adding actions and filters.

In the eMusic sunrise.php, I take this opportunity to load “site configs,” mainly to filter which plugins are going to load for which site, and which plugins will load for the entire network of sites (plugins that load for every site). To accomplish this, we need a “global” config and then a config for each site like so:

require_once( 'site-configs/global.php' );

if ( get_current_blog_id() > 1 ) {
    switch ( get_current_blog_id() ) {
    case 2:
        require_once( 'site-configs/bbpress.php' );
	break;

    case 3:
        require_once( 'site-configs/dots.php' );
	break;

     case 5:
        require_once( 'site-configs/support.php' );
	break;
    }

    add_filter( 'pre_option_template', function () {
	return 'dark';
    } );
} else {
    require_once( 'site-configs/emusic.php' );
}

Site Configs

In our global site config, we want to filter our active network plugins. As I have said in previous posts and presentations, relying on active plugins being correct in the database is dangerous and hard to keep in sync across environments.

We can filter active network plugins by filtering the active_sitewide_plugins site option:

add_filter( 'pre_site_option_active_sitewide_plugins', function () {
    return array(
        'batcache/batcache.php'                         => 1,
        'akismet/akismet.php'                           => 1,
        'avatar/avatar.php'                             => 1,
        'bundle/bundle.php'                             => 1,
        'cloud/cloud.php'                               => 1,
        'download/download.php'                         => 1,
        'emusic-notifications/emusic-notifications.php' => 1,
        'emusic-ratings/emusic-ratings.php'             => 1,
        'emusic-xml-rpc/emusic-xml-rpc.php'             => 1,
        'johnny-cache/johnny-cache.php'                 => 1,
        'like-buttons/like-buttons.php'                 => 1,
        'members/members.php'                           => 1,
        'minify/minify.php'                             => 1,
        'movies/movies.php'                             => 1,
        'shuffle/shuffle.php'                           => 1,
        'apc-admin/apc-admin.php'                       => 1,
        //'debug-bar/debug-bar.php'                     => 1
    );
} );

Filtering plugins in this manner allows to do 2 main things:

  • We can turn plugins on and off by commenting / uncommenting
  • We can visually see what our base set of plugins is for our network without going to the admin or the database

We have a lot of other things in site-configs/global.php, but I mainly want to demonstrate how we can load classes in an organized manner. In a site-specific config, we will filter what plugins are active for that site only.

For the main eMusic theme, we use these plugins:

add_filter( 'pre_option_active_plugins', function () {
    return array(
        'artist-images/artist-images.php',
        'catalog-comments/catalog-comments.php',
        'emusic-post-types/emusic-post-types.php',
        'discography.php',
        'emusic-radio/emusic-radio.php',
        'gravityforms/gravityforms.php',
        'super-ghetto/super-ghetto.php'
        //,'theme-check/theme-check.php'
    );
} );

This is just another example of us not repeating ourselves. When we think about everything in an object-oriented manner, everything can be cleaner and more organized.

HTTP Requests

Another area where we need to organize our code is around HTTP requests. We have “pages” in WordPress that aren’t populated via content from the database, but from a combination of Web Service calls based on request variables.

We use the following files:

// base class
/wp-content/mu-plugins/request.php

// extended by theme classes implementing the Factory pattern
// "dark" is a theme
/wp-content/requests/dark.php

// the theme class, where applicable,
// loads a request class when required by a certain page
/wp-content/requests/home.php
/wp-content/requests/album.php
/wp-content/requests/artist.php
... etc ...

As an example, an album page loads AlbumRequest:

class AlbumRequest extends RequestMap { ... }
class RequestMap extends API { ... }

// the API class implements CURL
class API {}

// CURL implements the cURL PHP extension, curl_multi(), etc
class CURL {}

Our organization of HTTP files is example of OO principles in action, and includes ways to organize our codebase in a way that is not specified by WordPress out of the box.

This is also an example of the self-documenting nature of object-oriented programming. Rather than show you each line of AlbumRequest, we can assume that it deals with requests specific to the Album page, inheriting functionality from RequestMap, which calls methods available in API, which at its core is implementing cURL.

Conclusion

Hopefully after reading all of this, you can clearly see some of the benefits of organizing your code in an object-oriented manner. The basics of OOP are outside the scope of this post but are essential for all developers to learn, especially those who work on teams whose members have varying degrees of skill.

Organized and readable code is essential, and the patterns available to us from the history of computer programming so far should be used to help get it there.

Couchbase

After reading an article on Tumblr’s architecture, I was intrigued by their affinity for Redis. I know lots of other projects use it, so I did some reading to find out if it was a suitable replacement for Memcached (it is marketed as “Memcached on Steroids”).

Redis is not an LRU Cache out of the box, it requires configuration for that. Redis is a NoSQL solution that provides a replication solution and can store key-values with expiration similar to Memcached.

A little about Membase:

Membase was developed by several leaders of the memcached project, who had founded a company, NorthScale, expressly to meet the need for an key-value database that enjoyed all the simplicity, speed, and scalability of memcached, but also provided the storage, persistence and querying capabilities of a database.

A then Membase became Couchbase:

As of February 8, 2011, the Membase project founders and Membase, Inc. announced a merger with CouchOne (a company with many of the principal players behind CouchDB) with an associated project merger. The merged project will be known as Couchbase. In January of 2012, Couchbase released a product building on Membase, known as Couchbase Server 1.8.

Most of the benchmarks I saw comparing Memcached and Redis: Memcached is still way faster yet “lacks the featureset of Redis.” Most of the benchmarks I saw for Couchbase used phrases like “sub-millisecond responses” / “twice as fast as Memcached.” I understand it’s cool to use sets and append / prepend to them in Redis, but other than those features, what makes it ultimately the better choice? I have concluded it just isn’t. If you like the tools it provides, awesome! If you want something that’s fast, has great tools, and is powerful all day long – why aren’t we all using Couchbase?

Major bug with WordPress + New Relic

If you haven’t seen New Relic yet, definitely take a look, it’s awesome.

That being said… if you are running New Relic monitoring on a huge production site, your logs might be spinning out of control without you knowing it, and I’ll explain why.

We use New Relic at eMusic to monitor our WordPress servers and many other application servers – Web Services, Databases etc – and we started noticing this in the logs like whoa: failed to delete New Relic auto-RUM

In New Relic’s FAQ section:

Beginning with release 2.6 of the PHP agent, the automatic Real User Monitoring (auto-RUM) feature is implemented using an output buffer. This has several advantages, but the most important one is that it is now completely accurate for all frameworks, not just Drupal and WordPress. The old mechanism was fraught with problems and highly sensitive to things like extra Drupal modules being installed, or customization of the header format. Using this new scheme, all of the problems go away. However, there is a down-side, but only for specific PHP code. This manifests itself as a PHP notice that PHP failed to delete buffer New Relic auto-RUM in…. If you do not have notices enabled, you may not see this and depending on how your code is written, you may enter an infite loop in your script, which will eventually time out, and simply render either an empty or a partial page.

To understand the reason for this error and how it can create an infinite loop in code that previously appeared to work, it is worth reading the PHP documentation on the ob_start() PHP function. Of special interest is the last optional parameter, which is a boolean value called erase that defaults to true. If you call ob_start() yourself and pass in a value of false for that argument, you will encounter the exact same warning and for the same reason. If that variable is set to false, it means that the buffer, once created, can not be destroyed with functions like ob_end_clean(), ob_get_clean(), ob_end_flush() etc. The reason is that PHP assumes that if a buffer is created with that flag that it modifies the buffer contents in such a way that the buffer cannot be arbitrarily stopped and deleted, and this is indeed the case with the auto-RUM buffer. Essentially, inside the agent code, we start an output buffer with that flag set to false, in order to prevent anyone from deleting that buffer. It should also be noted that New Relic is not the only extension that does this. The standard zlib extension that ships with PHP does the same thing, for the exact same reasons.

We have had several customers that were affected by this, and in all cases it was due to problematic code. Universally, they all had code similar to the following:

while (ob_get_level()) {
  ob_end_flush ();
}

The intent behind this code is to get rid of all output buffers that may exist prior to this code, ostensibly to create a buffer that the code has full control over. The problem with this is that it will create an infinite loop if you use New Relic, the zlib extension, or any buffer created with the erase parameter set to false. The reason is pretty simple. The call to ob_get_level() will eventually reach a point where it encounters a non-erasable buffer. That means the loop will never ever exit, because ob_get_level() will always return a value. To make matters worse, PHP tries to be helpful and spit out a notice informing you it couldn’t close whatever the top-most non-erasable buffer is. Since you are doing this in a loop, that message will be repeated for as long as the loop repeats itself, which could be infinitely.

So basically, you’re cool if you don’t try to flush all of the output buffers in a loop, because you will end up breaking New Relic’s buffer. Problematic as well if you are managing several of your own nested output buffers. But the problem might not be you, the problem is / could be WordPress.

Line 250 of wp-includes/default-filters.php:

add_action( 'shutdown', 'wp_ob_end_flush_all', 1 );

What does that code do?

/**
 * Flush all output buffers for PHP 5.2.
 *
 * Make sure all output buffers are flushed before our singletons our destroyed.
 *
 * @since 2.2.0
 */
function wp_ob_end_flush_all() {
	$levels = ob_get_level();
	for ($i=0; $i

So that’s not good. We found our culprit (if we were having the problem New Relic describes above). How to fix it?

I put this in wp-content/sunrise.php

<?php
remove_action( 'shutdown', 'wp_ob_end_flush_all', 1 );

function flush_no_new_relic() {
	$levels = ob_get_level();
	for ( $i = 0; $i < $levels - 1; $i++ )
		ob_end_flush();
}

add_action( 'shutdown', 'flush_no_new_relic', 1, 0 );

This preserves New Relic’s final output buffer. An esoteric error, but something to be aware of if you are monitoring WordPress with New Relic.