At the Post we make a lot of great interactive graphics. Most of these are built as static html pages. It’s a great way for our incredibly talented graphics team to work. They have flexibility to design and build the approach that is best for telling the story.

However, there is a drawback to this approach – it makes it difficult to work with content from our CMS. Our reporters and photographer's workflow is built around having their work printed in the paper and displayed on washingtonpost.com, both of which are powered by a CMS. As we transform the way we tell stories online, our interactive graphics must be integrated with great writing and awesome photography. How can we do that without making our graphics team hack a template to death or inject raw code into the CMS? Gulp provides an answer.

Gulp Glue

On our recent Black Route project we tried a new approach that allowed our reporters and copy editors to work on the article and the graphics team to build interactive features without breaking either team's workflow.

We use Gulp on the graphics team for our build tasks – SASS compilation, JavaScript modules etc. We really like the speed and flexibility it provides. And in this case it was the glue to stick our graphics work together with the article being written in our CMS.

Beyond our normal build tasks we came up with an approach to combine our interactive graphics, developed by the graphics editors, with the article being written by the reporters. What we needed was a way to intersperse interactive graphics throughout the article in a way that was familiar for our graphics editors. We came up with what we called the “article transform.” This, along with a few other Gulp tasks, allowed us to take interactive graphics, photos and the article stored in a CMS and combine them into a single HTML file which we could then publish online. Let’s dive into the article transforms and other tasks in detail.

Article Transforms

The article transform is the most interesting and important part of what we had Gulp do on this project. What if you could write jQuery selectors for where you wanted to insert your HTML graphics into article content and get the result as static html at the end? This is what the transforms allowed.

Each transformation was stored as a file in a directory. The selector location and the transform was specified as front matter and the HTML injection we wanted applied was the body of the file like this:


—
selector: 'article'
action: 'prepend'
—
<div class="wrap—right wrap—map">
  <figure class="locator-map map" id="follow-locator"></figure>
</div>

Our Gulp task runs through a directory of these files and saves them in memory:


var memory = []; //array of files contents etc... 

gulp.task('load-insertions', function() {
  //setup the objects we need... 
  memory = [];
  return gulp.src(config.injections_src)
  .pipe(frontMatter({ 
      property: 'fm',
      remove: true 
    })) 
  	// tap into the stream to get each file's data
  .pipe( tap(function(file) {
  	console.log('Loading Injections: ' + path.basename(file.path));
  	var newinsertion = {};
  	for(var i in file.fm){
  		newinsertion[i] = file.fm[i];
  	}
    newinsertion.content = file.contents.toString();

    memory.push(newinsertion);

  }));
});

Now that we have all the transforms in memory we can download the article and apply them.

Download and transform

We have an internal JSON api for all the stories in the CMS. Using gulp-download we would pull down the story, turn it into a JS object using gulp-json-transform and pull out the article html content. Then we parse that using Cheerio and apply our injections.


gulp.task('download-transform', ['load-insertions'], function () {
    return download(config.src_url)
        .pipe(jsonTransform(function(data) {
        //we only need the html node from story adapter
            return data.html;
        }))
        .pipe(cheerio(function ($, file) {

            var $paragraphs = $('p'); //get all the paragraphs

            for(var i=0; i< memory.length; i++){
                var insertion = memory[i];
                if(insertion['offset'] && insertion['selector'] == 'p'){
                    $paragraphs.eq(insertion['offset'])[insertion['action']](insertion['content']);
                }else{
                    $(insertion['selector'])[insertion['action']](insertion['content']);
                }
            }
        }))
        .pipe(rename(config.file_name))
        .pipe(gulp.dest(config.dest));
});

Note that this task uses the injections stored in memory by the previous task and runs that previous task as a dependency. This approach was inspired by this Gulp recipe. We kept the configuration in the front matter where the injections were specified so the Gulp task wouldn't have to be constantly updated – even as the article or injection changed. The huge upside of doing it this way is the syntax for the injections is ultra familiar: it's just jQuery. And because we can use any css selectors we want the interactive graphics won't break even as the reporters copy edit the article and move content around.

All together

So in the end our complete article transform pipeline performs the following tasks:

  1. Load into memory article transformations that will apply graphic specific markup to the article
  2. Download the article from the JSON api endpoint
  3. Parse the html from the downloaded article using Cheerio
  4. Apply those transforms to the parsed html
  5. Inject the transformed article html into the graphics template
  6. Save the file and serve it locally (ready for production)

It’s probably even clearer as a picture:

Connected Newsroom

Being able to connect Gulp tasks like legos allows us take things we are familiar with—our normal build system and javascript—and extend them to make functional, cross-newsroom workflows. These workflows allow our reporters and graphics editors to collaborate and tell amazing stories in inventive ways on tight deadlines using workflows everyone is comfortable with.