Headless Chrome with Testem on VSTS-Hosted Agents

I previously wrote about building a Node app on VSTS Windows agents. Since then, we’ve started using headless Chrome on those agents. Here’s how.

Background

Browser test automation has come a long way over the years. We used to accept having a browser flashing around in an interactive login session. Then we made browsers headless-ish with Xvfb. Then we used truly headless browsers like PhantomJS, even though they tended to be weaker than their desktop counterparts.

Last year, Firefox and Chrome added headless support, giving us the best of both worlds: the performance, stability, and feature set of mass-market browsers with the logistical convenience of command line applications.

The JavaScript ecosystem has embraced them. Ember.js, the front-end framework I’m currently working in, switched to headless Chrome by default last year, and it recently dropped support for Phantom.

Approaches

At a high level, I need to 1) download Chrome, and 2) make it available to my test framework.

Get Chrome

We’re using VSTS-hosted agents, which don’t yet have Chrome preinstalled. Where should we get it?

There’s a VSTS task and a few npm packages, but nothing seemed widely used or well-maintained. I suppose the JavaScript community doesn’t have a great need for such a package, as competing CI systems offer Chrome out-of –the –box.

Luckily, I stumbled across a blog post by Benjamin Spencer, in which he pulls it out of Puppeteer, a Chrome automation project maintained by Google. We’re not using Puppeteer, but as a source, it seems more likely to stick around (and get updated) than the other, lesser-used projects.

Provide it to Testem

Testem’s approach of looking for specific executables works well for most situations, but it won’t magically find Puppeteer’s chrome.exe deep inside node_modules. I experimented with adding it on the executable path, but then I decided to try Testem’s custom launchers instead.

Implementation

Here’s how to download and use headless Chrome for an Ember-CLI project on VSTS Windows agents:

1. Add the Puppeteer package with e.g. yarn add --dev puppeteer. This puts the browser at a path like node_modules/puppeteer/.local-chromium/win64-536395/chrome-win32/chrome.exe.
2. Wrap Chrome for easy access. We can avoid using this long (and dynamic) path with a short Node script that asks Puppeteer where to find Chrome, then runs it:

   
   const { executablePath } = require("puppeteer");
   const { execFileSync } = require("child_process");
   
   let exePath = executablePath();
   let args = process.argv.slice(2);
   execFileSync(exePath, args);
   

   Now we can run Chrome with e.g. node run-chrome.js google.com.

3. Add a custom launcher to Testem’s configuration. Here’s my Testem.js:

   
   /* eslint-env node */
   module.exports = {
     "test_page": "tests/index.html?hidepassed",
     "report_file": "tmp/results.xml",
     "reporter": "xunit",
     "xunit_intermediate_output": true,
     "disable_watching": true,
     "launch_in_ci": [
       "LocalChrome"
     ],
     "launchers": {
       "LocalChrome": {
         "exe": "node",
         "args": [
           'run-chrome.js',
           '--disable-gpu',
           '--headless',
           '--remote-debugging-port=9222',
           '--window-size=1440,900'
         ],
         "protocol": "browser"
       },
     }
   };
   

Conclusion

With Chrome in CI, our builds are faster and more reliable, and we’re able to raise our target language level, which had been held back for PhantomJS. VSTS may soon offer Chrome out-of-the-box, but with a little effort, we can enjoy it today.