Programming►node.js
Programming

Remove Bower from your build script

The mysterious broken build

This morning, our QA told us that knockout, a javascript library that we used in our web app is missing on staging environment. Then we checked the package she got from CI server, and the javascript library was indeed not included. But when we tried to generate the package on our local dev box, we found that knockout is included.

It is a big surprise to us, because we share the exact same build scripts and environment between dev-boxes and CI agents and because we manage the front-end dependencies with bower. In our gulp script, we ask bower to install the dependencies every time to make sure they are up to date.

The root cause of the broken build

After spending hours on diagnosing the CI agents, we finally figure out the reason, a tricky story:

When the Knockout maintainer released the v3.1 bower package, they made a mistake in bower.json config file, which packaged the spec folder instead of the dist folder. So this package is actually broken, because the main javascript file dist/knockout.js , described in bower.json doesn’t exist.

Later, the engineers realized they made a mistake, and they fixed the issue by releasing a new package. Maybe they think they haven’t changed any script logic, so they release the new package under the same version number, which is the criminal who broke our builds.

We’re so unlucky that the broken package is downloaded on our CI server when our build script was executed there for the first time. And the broken package is stored in bower cache at that time.

Because of Bower’s cache mechanism, the broken package is used unless the version is bumped or cache is expired. This is the reason why our build is broken on the CI server.

But on our dev box, for some reason, we had run bower cache clean, which invalidated the cache. So we have a good build on our local dev box. This is the reason why we can generate good package on our dev box.

It is a very tricky issue when using bower to manage dependencies. Although it is not completely our fault, but it is kind of the worst case then we can face. The build broke silently, there were no error logs or messages that helped to figure out the reason. (Well, we haven’t got a chance to setup the smoke test for our app yet, so it could be kind of our fault.)

We thought we had been careful enough to clean the bower_components folder every time, but that prevented us from figuring out the real cause.

After fixing this issue, discussed with my pair Rafa and we came up some practices that could be helpful to avoid this kind of issue:

Best practices

  • Avoid bower install or any equivalent step (such as gulp-bower, grunt-bower, etc.) in the build script
  • Check bower_components into the code repository or download the dependencies from our self managed repository for large projects.
  • When dependencies are changed, manually install them and make sure they’re good.

After doing this, our build script runs even faster, because we don’t need to check all dependencies are up-to-date every time. This is a bonus from removing bower install from our build script.

Some thoughts on the package system

Bower components are maintained by the community, and there is no strict quality control to ensure the package is bug-free or being released in an appropriate way. So it could be safer if we can check them manually, and lock them down across environments.

This could be common issue for all kind of community managed package system. Not just Bower, it could be Maven, Ruby Gem, Node.js package, Python pip package, nuget package or even Docker containers!

Practice►Continues Integration
Practice

Multiple Project Summary Reporting Standard - cctray xml feed

CCTray.xml is an RSS-like CI server build status xml feed, which is originally developed for CruiseControl.net.
ThoughtWorks declared it in a standard called “Multiple Project Summary Reporting Standard”, which now have become some kind of unofficial standard of CI server feed that is widely supported by all kind of popular CI servers.

You can find the feed as described below:

And according to cc_dashboard, there are some exceptions that are not included in the document.

  • An additional “Pending” activity
  • An additional “Unknown” status. I’ve seen Unknown reported by CruiseControl.rb when project builds are serialized (“Configuration.serialize_builds = true” set in .cruise/site_config.rb) and one build is waiting for another build to finish. I’ve seen Unknown reported by Hudson when a project is disabled.

The following is a patched version of Multiple Project Summary Reporting Standard.

Multiple Project Summary Reporting Standard

Introduction

Various Continuous Integration monitoring / reporting tools exist. Examples are:

These tools work by polling Continuous Integration servers for summary information and presenting it appropriately to users.

If a Continuous Integration server can offer a standard summary format, and a reporting tool can consume the same, then we get interoperability between reporting tools and CI Servers.

Description

Summary information will be available as a plain XML string retrievable through an http GET request.

The format of the XML will be as follows:

Summary

A single node, the document root, which contains 0 or many node.

Each may have the following attributes:






































































































name description type required
name The name of the project string yes
activity The current state of the project string enum : Sleeping, Building, CheckingModifications
yes
lastBuildStatus A brief description of the last build string enum : Pending, Success, Failure, Exception, Unknown yes
lastBuildLabel A referential name for the last build string no
lastBuildTime When the last build occurred DateTime yes
nextBuildTime When the next build is scheduled to occur (or when the next check to see whether a build should be performed is scheduled to occur) DateTime no
webUrl A URL for where more detail can be found about this project string (URL) yes

Clients that consume this XML should not rely on any optional attribute being present, and should degrade their functionality gracefully.

Example

CCTray.xml Sample
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
<Projects>
    <Project
        name="SvnTest"
        activity="Sleeping"
        lastBuildStatus="Exception"
        lastBuildLabel="8"
        lastBuildTime="2005-09-28T10:30:34.6362160+01:00"
        nextBuildTime="2005-10-04T14:31:52.4509248+01:00"
        webUrl="http://mrtickle/ccnet/"/>
    <Project
        name="HelloWorld"
        activity="Sleeping"
        lastBuildStatus="Success"
        lastBuildLabel="13"
        lastBuildTime="2005-09-15T17:33:07.6447696+01:00"
        nextBuildTime="2005-10-04T14:31:51.7799600+01:00"
        webUrl="http://mrtickle/ccnet/"/>
</Projects>

Schema

cctray.xml Schema
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
<?xml version="1.0" encoding="UTF-8" ?>
<xs:schema elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="Projects">
<xs:complexType>
<xs:sequence>
<xs:element name="Project" maxOccurs="unbounded">
<xs:complexType>
<xs:attribute name="name" type="xs:NMTOKEN" use="required" />
<xs:attribute name="activity" use="required">
<xs:simpleType>
<xs:restriction base="xs:NMTOKEN">
<xs:enumeration value="Sleeping" />
<xs:enumeration value="Building" />
<xs:enumeration value="CheckingModifications" />
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="lastBuildStatus" use="required">
<xs:simpleType>
<xs:restriction base="xs:NMTOKEN">
<xs:enumeration value="pending"/>
<xs:enumeration value="Exception"/>
<xs:enumeration value="Success"/>
<xs:enumeration value="Failure"/>
<xs:enumeration value="Unknown"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="lastBuildLabel" type="xs:NMTOKEN" use="required" />
<xs:attribute name="lastBuildTime" type="xs:dateTime" use="required" />
<xs:attribute name="nextBuildTime" type="xs:dateTime" use="optional" />
<xs:attribute name="webUrl" type="xs:string" use="required" />
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>