Continuous Integration with TypeScript + Mocha + Istanbul (NYC) + CircleCI

Date posted: 2018-05-01

This post is about joining TypeScript (programming language) with Mocha (test framework), Istanbul (code coverage), and CircleCI (continuous integration).

I created a simple TypeScript project with the following structure (the files are all empty for now, except for package.json, which contains the initial code from npm):

.
|-- .circleci
|   |-- config.yml
|-- dist
|-- package.json
|-- src
|   |-- print.ts
|   `-- transform.ts
|-- test
|   |-- mocha.opts
|   `-- unit
|       `-- transform.test.ts
`-- tsconfig.json

First, I made a tsconfig.json to configure how TypeScript will be compiled:

{
  "compilerOptions": {
    "module": "commonjs",
    "removeComments": false,
    "sourceMap": true,
    "baseUrl": "types",
    "typeRoots": ["node_modules/@types"],
    "target": "es6",
    "lib": ["es2016", "dom"],
    "rootDir": "src",
    "outDir": "dist",
    "types": [
      "mocha"
    ]
  },
  "include": [
    "src"
  ]
}

The "removeComments": false is very important. We will see why later!

I also made a little script to compile the TypeScript code in the package.json file:

"compile": "./node_modules/.bin/tsc"

Let's start with print.ts and transform.ts:

// print.ts
// This is just a dummy function. We won't do anything interesting with it
export function print(v: any) {
    console.log(v);
}
// transform.ts
// This extremely over-complicated function will receive an array of numbers
// and return 0 if the sum of the numbers is 0, 1 if the sum is > 0, and -1
// if the sum is -1
// I made it complicated so we will have lots of branches to test
export function transform(input: number[]): number {
    if (!input || input.constructor !== Array)
        throw new Error('Input must be an array of numbers!');

    try {
        const total = input.reduce((acc: number, n: number) => acc + n, 0);

        if (total === 0) {
            console.log('The input is equal to zero');
            return 0;
        } else if (total > 0) {
            console.log('The input is greater than zero');
            return 1;
        } else {
            console.log('The input is greater than zero');
            return -1;
        }
    } catch (e) {
        console.error(`Unknown error occurred: ${e}`);
        return 0;
    }
};

Alright. We have the code, now we need to make unit tests for it!

First, I will install the following packages:

And now I am going to write the test cases for transform.ts:

import { transform } from '../../src/transform';
import { expect } from 'chai';

describe('transform', () => {

    it('should fail if non-array is passed', () => {
        expect(() => transform('Bad input!' as any)).to.throw();
    });

    it('should return 0', () => {
        const result = transform([1, -1, 2, -2]);
        expect(result).to.eql(0);
    });

    it('should return 1', () => {
        const result = transform([1, -1, 2, -2, 3]);
        expect(result).to.eql(1);
    });

    it('should return -1', () => {
        const result = transform([1, -1, 2, -2, -3]);
        expect(result).to.eql(-1);
    });

});

Perfect! We have the test cases done.

Now, here is one problem: should we compile the tests? They are written in TypeScript, so they should be compiled, right? Well, you don't have to. Luckily, ts-node is here to help! Ts-node is a TypeScript interpreter! Although I would not recommend actually using it to run the main script, it is great for running the test cases!

First, installing the packages we need:

Now let's configure mocha to use ts-node:

# test/mocha.opts
--require ./node_modules/ts-node/register
--require ./node_modules/source-map-support/register
--recursive
--exit

Here is what these lines mean:

And let's make an NPM script to run the tests (files that end in .test.ts) in package.json:

...
  "scripts": {
    "test": "./node_modules/.bin/mocha test/**/*.test.ts",
    "compile": "./node_modules/.bin/tsc"
  },
...

That's it. Whenever we run npm test, mocha will run all the tests for us. Let's try it:

  transform
    ✓ should fail if non-array is passed
The input is equal to zero
    ✓ should return 0
The input is greater than zero
    ✓ should return 1
The input is greater than zero
    ✓ should return -1

  4 passing (7ms)

But that's not all! Writing tests is not torture enough - we need to make sure we write enough tests to cover all our code. This is what code coverage does.

Istanbul (also known as NYC - I actually don't get why the two names) will make this very easy. I will install the following packages:

Easy. Now we can modify the test script so Istanbul will check our code coverage:

...
  "scripts": {
    "test": "./node_modules/.bin/nyc ./node_modules/.bin/mocha test/**/*.test.ts",
    "coverage": "./node_modules/.bin/nyc report",
    "compile": "./node_modules/.bin/tsc"
  },
...

Whenever we run the tests, we will get the coverage for our files. I also added a separate script (coverage) for when we just want to see the coverage, without running the tests again.

I will also add some settings for Istanbul in the package.json file:

...
  "nyc": {
    "extension": [ // <- Extensions to be covered
      ".ts"
    ],
    "include": [ // <- Which directories should be covered?
      "src"
    ],
    "reporter": [ // <- Reporters used *1
      "text",
      "html"
    ],
    "all": true, // <- Check all files? *2
    "check-coverage": true, // <- Enforce a coverage threshold?
    "statements": 90, // <- Minimum coverage for statements (%)
    "functions": 90, // <- Minimum coverage for functions (%)
    "branches": 90, // <- Minimum coverage for branches (%)
    "lines": 90 // <- Minimum coverage for lines (%)
  },
...
  1. Reporters are how the coverage is reported to us. In this case, I am asking for two types of reports: text in the terminal, and html files (useful for CircleCI)
  2. If all is set to false, it will only check the coverage of the files used by the test files. If you have a file that was not tested at all, it will not show up in the reports.

Let's take a look at the output of npm test:

ERROR: Coverage for lines (81.25%) does not meet global threshold (90%)
ERROR: Coverage for functions (66.67%) does not meet global threshold (90%)
ERROR: Coverage for statements (82.35%) does not meet global threshold (90%)
--------------|----------|----------|----------|----------|-------------------|
File          |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
--------------|----------|----------|----------|----------|-------------------|
All files     |    82.35 |      100 |    66.67 |    81.25 |                   |
 print.ts     |        0 |      100 |        0 |        0 |                 2 |
 transform.ts |     87.5 |      100 |      100 |    86.67 |             19,20 |
--------------|----------|----------|----------|----------|-------------------|

Cool! But there is a problem there: we still haven't fully tested transform.ts:

    } catch (e) {
        console.error(`Unknown error occurred: ${e}`);
        return 0;
    }

I put that catch there as an example of something I can't really test. Nothing will throw an error there, but sometimes we are using something that can fail under circumstances out of our control, and they are failures that we can not reproduce.

What can we do then? We can tell Istanbul to ignore lines, like this:

    } catch (e) {
        /* istanbul ignore next */
        console.error(`Unknown error occurred: ${e}`);
        /* istanbul ignore next */
        return 0;
    }

This will only work if "removeComments": false is set in tsconfig.json, otherwise the compiler will remove the comment.

Let's try it now:

ERROR: Coverage for lines (85.71%) does not meet global threshold (90%)
ERROR: Coverage for functions (50%) does not meet global threshold (90%)
ERROR: Coverage for statements (85.71%) does not meet global threshold (90%)
--------------|----------|----------|----------|----------|-------------------|
File          |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
--------------|----------|----------|----------|----------|-------------------|
All files     |    85.71 |      100 |       50 |    85.71 |                   |
 print.ts     |        0 |      100 |        0 |        0 |                 2 |
 transform.ts |      100 |      100 |      100 |      100 |                   |
--------------|----------|----------|----------|----------|-------------------|

Sweet!

I won't bother making the test case for print.ts because that file was there only to show you what "all": true does: even if we are not testing that file, it will show up in the coverage report! Let's just jump into integration with CircleCI.

CircleCI is very easy to set up. Most of the times, continuous integration systems have their own separate environment (such as a container), which bad news for people who can't get their code running even on their own machine. CircleCI is no exception. All we need to do is describe how the environment should be and how to run our tests (find more information here).

Here is my ./circleci/config.yml that describes how to run my tests:

version: 2
jobs:
  build:
    working_directory: ~/app
    docker:
      - image: circleci/node:10.0.0
    steps:
      - checkout

      - run:
          name: Installing packages
          command: npm install

      - save_cache:
          key: dependency-cache-{{ checksum "package.json" }}
          paths:
            - ./node_modules

      - run:
          name: Running tests
          command: npm test

      - store_artifacts:
          path: coverage
          prefix: coverage

In this case, I am asking for a container with Node 10.0. Then I follow these steps:

  1. Install my npm packages
  2. Cache my npm packages (will make the jobs a lot faster)
  3. Run the tests
  4. Save the html files with the coverage (remember the html reporter?) as an artifact, which we can access after the tests are done

As long as our project is set up on CircleCI, it will test anything we push into our repository.

All done!

Repository with the code