Continuous Integration with TypeScript + Mocha + Istanbul (NYC) + CircleCI

Writing unit and integration tests is the bane of my existence. The sheer amount of boredom produced by this practice would easily make me rich if I somehow were paid to get bored. I would love to meet someone who genuinely enjoys writing tests for a hobby so I would allow them to write all my tests for free, although my self-preservation instinct tells me that such person cannot be trusted and will eventually try to stab me with a fish or some unusual object that will make people chuckle when they read the news. Anyway, writing unit tests is torture, but it has to be done. Other things that should be done, on top of writing unit tests is: 1. Check the coverage of these tests to make sure you did not miss any lines, branches, functions, files, etc 2. Continuously test the code pushed into a repository with a continuous integration system. This way, we can easily know if the tests are broken for a pull request This post will be about joining TypeScript (programming language) with Mocha (test framework), Istanbul (code coverage), and CircleCI (continuous integration). I created a simple TypeScript project with the following structure (the files are all empty for now, except for package.json, which contains the initial code from npm): ```json . |-- .circleci | |-- config.yml |-- dist |-- package.json |-- src | |-- print.ts | `-- transform.ts |-- test | |-- mocha.opts | `-- unit | `-- transform.test.ts `-- tsconfig.json ``` First, I made a **tsconfig.json** to configure how TypeScript will be compiled: ```json { "compilerOptions": { "module": "commonjs", "removeComments": false, "sourceMap": true, "baseUrl": "types", "typeRoots": ["node_modules/@types"], "target": "es6", "lib": ["es2016", "dom"], "rootDir": "src", "outDir": "dist", "types": [ "mocha" ] }, "include": [ "src" ] } ``` The `"removeComments": false` is very important. We will see why later! I also made a little script to compile the TypeScript code in the **package.json** file: ``` "compile": "./node_modules/.bin/tsc" ``` Let's start with **print.ts** and **transform.ts**: ```TypeScript // print.ts // This is just a dummy function. We won't do anything interesting with it export function print(v: any) { console.log(v); } ``` ```TypeScript // transform.ts // This extremely over-complicated function will receive an array of numbers // and return 0 if the sum of the numbers is 0, 1 if the sum is > 0, and -1 // if the sum is -1 // I made it complicated so we will have lots of branches to test export function transform(input: number[]): number { if (!input || input.constructor !== Array) throw new Error('Input must be an array of numbers!'); try { const total = input.reduce((acc: number, n: number) => acc + n, 0); if (total === 0) { console.log('The input is equal to zero'); return 0; } else if (total > 0) { console.log('The input is greater than zero'); return 1; } else { console.log('The input is greater than zero'); return -1; } } catch (e) { console.error(`Unknown error occurred: ${e}`); return 0; } }; ``` Alright. We have the code, now we need to make unit tests for it! First, I will install the following packages: - **chai** - Has useful tools that will make asserting the results easier - **mocha** - Our test framework - **@types/chai** - TypeScript types for the chai module - **@types/mocha** - TypeScript types for the mocha module And now I am going to write the test cases for **transform.ts**: ```TypeScript import { transform } from '../../src/transform'; import { expect } from 'chai'; describe('transform', () => { it('should fail if non-array is passed', () => { expect(() => transform('Bad input!' as any)).to.throw(); }); it('should return 0', () => { const result = transform([1, -1, 2, -2]); expect(result).to.eql(0); }); it('should return 1', () => { const result = transform([1, -1, 2, -2, 3]); expect(result).to.eql(1); }); it('should return -1', () => { const result = transform([1, -1, 2, -2, -3]); expect(result).to.eql(-1); }); }); ``` Perfect! We have the test cases done. Now, here is one problem: should we compile the tests? They are written in TypeScript, so they should be compiled, right? Well, you don't have to. Luckily, ts-node is here to help! Ts-node is a TypeScript interpreter! Although I would not recommend actually using it to run the main script, it is great for running the test cases! First, installing the packages we need: - **source-map-support** - **typescript** - **ts-node** Now let's configure mocha to use ts-node: ```json # test/mocha.opts --require ./node_modules/ts-node/register --require ./node_modules/source-map-support/register --recursive --exit ``` Here is what these lines mean: - `require ./node_modules/ts-node/register` - Here we are telling Mocha to use ts-node as the interpreter - `require ./node_modules/source-map-support/register` - Support for source maps. Will be useful later with Istanbul - `--recursive` - Test all the files in the directory, not individual files - `--exit` - Force exit after the tests are done (will kill any pending promises) And let's make an NPM script to run the tests (files that end in *.test.ts*) in package.json: ```json ... "scripts": { "test": "./node_modules/.bin/mocha test/**/*.test.ts", "compile": "./node_modules/.bin/tsc" }, ... ``` That's it. Whenever we run `npm test`, mocha will run all the tests for us. Let's try it: ```json transform ✓ should fail if non-array is passed The input is equal to zero ✓ should return 0 The input is greater than zero ✓ should return 1 The input is greater than zero ✓ should return -1 4 passing (7ms) ``` But that's not all! Writing tests is not torture enough - we need to make sure we write enough tests to cover all our code. This is what code coverage does. Istanbul (also known as NYC - I actually don't get why the two names) will make this very easy. I will install the following packages: - **nyc** Easy. Now we can modify the `test` script so Istanbul will check our code coverage: ```json ... "scripts": { "test": "./node_modules/.bin/nyc ./node_modules/.bin/mocha test/**/*.test.ts", "coverage": "./node_modules/.bin/nyc report", "compile": "./node_modules/.bin/tsc" }, ... ``` Whenever we run the tests, we will get the coverage for our files. I also added a separate script (`coverage`) for when we just want to see the coverage, without running the tests again. I will also add some settings for Istanbul in the **package.json** file: ``` ... "nyc": { "extension": [ // <- Extensions to be covered ".ts" ], "include": [ // <- Which directories should be covered? "src" ], "reporter": [ // <- Reporters used *1 "text", "html" ], "all": true, // <- Check all files? *2 "check-coverage": true, // <- Enforce a coverage threshold? "statements": 90, // <- Minimum coverage for statements (%) "functions": 90, // <- Minimum coverage for functions (%) "branches": 90, // <- Minimum coverage for branches (%) "lines": 90 // <- Minimum coverage for lines (%) }, ... ``` 1. Reporters are how the coverage is reported to us. In this case, I am asking for two types of reports: **text** in the terminal, and **html** files (useful for CircleCI) 2. If **all** is set to **false**, it will only check the coverage of the files used by the test files. If you have a file that was not tested at all, it will not show up in the reports. Let's take a look at the output of `npm test`: ```json ERROR: Coverage for lines (81.25%) does not meet global threshold (90%) ERROR: Coverage for functions (66.67%) does not meet global threshold (90%) ERROR: Coverage for statements (82.35%) does not meet global threshold (90%) --------------|----------|----------|----------|----------|-------------------| File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s | --------------|----------|----------|----------|----------|-------------------| All files | 82.35 | 100 | 66.67 | 81.25 | | print.ts | 0 | 100 | 0 | 0 | 2 | transform.ts | 87.5 | 100 | 100 | 86.67 | 19,20 | --------------|----------|----------|----------|----------|-------------------| ``` Cool! But there is a problem there: we still haven't fully tested **transform.ts**: ```TypeScript } catch (e) { console.error(`Unknown error occurred: ${e}`); return 0; } ``` I put that **catch** there as an example of something I can't really test. Nothing will throw an error there, but sometimes we are using something that *can* fail under circumstances out of our control, and they are failures that we can not reproduce. What can we do then? We can tell Istanbul to ignore lines, like this: ```TypeScript } catch (e) { /* istanbul ignore next */ console.error(`Unknown error occurred: ${e}`); /* istanbul ignore next */ return 0; } ``` This will **only** work if `"removeComments": false` is set in **tsconfig.json**, otherwise the compiler will remove the comment. Let's try it now: ```json ERROR: Coverage for lines (85.71%) does not meet global threshold (90%) ERROR: Coverage for functions (50%) does not meet global threshold (90%) ERROR: Coverage for statements (85.71%) does not meet global threshold (90%) --------------|----------|----------|----------|----------|-------------------| File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s | --------------|----------|----------|----------|----------|-------------------| All files | 85.71 | 100 | 50 | 85.71 | | print.ts | 0 | 100 | 0 | 0 | 2 | transform.ts | 100 | 100 | 100 | 100 | | --------------|----------|----------|----------|----------|-------------------| ``` Sweet! I won't bother making the test case for **print.ts** because that file was there only to show you what `"all": true` does: even if we are not testing that file, it will show up in the coverage report! Let's just jump into integration with CircleCI. CircleCI is very easy to set up. Most of the times, continuous integration systems have their own separate environment (such as a container), which bad news for people who can't get their code running even on their own machine. CircleCI is no exception. All we need to do is describe how the environment should be and how to run our tests (find more information here). Here is my **./circleci/config.yml** that describes how to run my tests: ```yml version: 2 jobs: build: working_directory: ~/app docker: - image: circleci/node:10.0.0 steps: - checkout - run: name: Installing packages command: npm install - save_cache: key: dependency-cache-{{ checksum "package.json" }} paths: - ./node_modules - run: name: Running tests command: npm test - store_artifacts: path: coverage prefix: coverage ``` In this case, I am asking for a container with Node 10.0. Then I follow these steps: 1. Install my npm packages 2. Cache my npm packages (will make the jobs a lot faster) 3. Run the tests 4. Save the *html* files with the coverage (remember the *html* reporter?) as an artifact, which we can access after the tests are done As long as our project is set up on CircleCI, it will test anything we push into our repository. All done! Repository with the code