blog

Benchmark: NodeJS x Perfect (Swift)

Yesterday (October 27th, 2016) I went to a presentation called "How to Completely Fail at Open-Sourcing", presented by Sean Stephens, at FSOSS - Free Software and Open Source Symposium hosted at Seneca College. Sean Stephens is the CEO of PerfectlySoft Inc, the company that developed Perfect, a library for server-side Swift development. This immediately caught my attention: I've been thinking about server-side development in Swift for a while, and it seems that it finally happened.

During the presentation, Sean showed us some benchmarks where Swift (using the Perfect framework) beat NodeJS in several fronts. You can see more details in this post. Since I recently benchmarked PHP x NodeJS (around a month ago), I decided to use a similar scenario and test Perfect x NodeJS. This is how I set it up:

I wanted 2 servers: one with Perfect, and the other one with pure NodeJS. For every request, they would go to MongoDB, fetch all results, append some text to the response, and send it back. I used siege as the stress tester, in order to simulate concurrent connections.

I set up a virtual machine with 1 processor core, 512Mb of RAM and 20Gb of storage; the machine was running Debian Jessie. In this machine, I installed Docker and made 3 images:

#1st image: MongoDB
FROM ubuntu:16.04
MAINTAINER Docker
RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv EA312927
RUN echo "deb http://repo.mongodb.org/apt/ubuntu $(cat /etc/lsb-release | grep DISTRIB_CODENAME | cut -d= -f2)/mongodb-org/3.2 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-3.2.list
RUN apt-get update && apt-get install -y mongodb-org
VOLUME ["data/db"]
WORKDIR "data/"
EXPOSE 27017
ENTRYPOINT ["/usr/bin/mongod"]
# 2nd image: NodeJS
FROM node:wheezy
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY package.json /usr/src/app/
RUN npm install
COPY . /usr/src/app
EXPOSE 8000
CMD [ "node", "index.js" ]
// index.js
var http = require('http');
var mongodb = require('mongodb');

mongodb.connect('mongodb://10.46.52.207:27017/test', function (err, db) {
  if (err) { console.log(err); }
  http.createServer(function (req, res) {
    var s = "";
    for (var i = 1; i <= 1000; i++) {
      s += '' + i;
    }
    db.collection("test").find({}).toArray(function(err, doc) {
      res.end("Hello world" + JSON.stringify(doc) + s);
    });
  }).listen(8000);

});
// package.json
{
  "name": "node",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "mongodb": "^2.2.10"
  }
}
# Copyright (C) 2016 PerfectlySoft Inc.
# Author: Shao Miller <swiftcode@synthetel.com>

FROM perfectlysoft/ubuntu1510
RUN /usr/src/Perfect-Ubuntu/install_swift.sh --sure
RUN apt-get install libtool -y
RUN apt-get install dh-autoreconf -y
RUN git clone https://github.com/mongodb/mongo-c-driver.git
WORKDIR ./mongo-c-driver
RUN ./autogen.sh --with-libbson=bundled
RUN make
RUN make install
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY . /usr/src/app
RUN swift build
CMD .build/debug/App --port 8000
// Package.swift
import PackageDescription
let package = Package(
  name: "App",
  targets: [],
  dependencies: [
    .Package(
      url: "https://github.com/PerfectlySoft/Perfect-HTTPServer.git",
      majorVersion: 2,
      minor: 0),
    .Package(
      url:"https://github.com/PerfectlySoft/Perfect-MongoDB.git", 
      majorVersion: 2,
      minor: 0)
  ]
)
// Sources/main.swift
import PerfectLib

import PerfectHTTP
import PerfectHTTPServer
import MongoDB

let server = HTTPServer()

var routes = Routes()
routes.add(method: .get, uri: "/", handler: {
  (request, response)->() in
    response.setHeader(.contentType, value: "text/html")

    let client = try! MongoClient(uri: "mongodb://10.46.52.207:27017")
    let db = client.getDatabase(name: "test")
    guard let collection = db.getCollection(name: "test") else { return }

    let fnd = collection.find(query: BSON())

    var arr = [String]()
    for x in fnd! {
      arr.append(x.asString)
    }

    defer {
      collection.close()
      db.close()
      client.close()
    }

    var s = ""

    for x in 1...1000 {
      s += String(x)
    }

    response.appendBody(string: "Hello world {\(arr.joined(separator: ","))}\(s)")
    response.completed()
  }
)

server.addRoutes(routes)

server.serverPort = 8000

do {
  try server.start()
} catch PerfectError.networkError(let err, let msg) {
  print("Network error thrown: \(err) \(msg)")
}

By the way, I'm sorry if my Swift code is not Swifty enough - I am just a JavaScript peasant. But anyway, these are the results I got:

500 users 1,000 users 1,500 users
NodeJS Perfect NodeJS Perfect NodeJS Perfect
Number of hits 1284 1273 2293 2284 3641 3556
Availability (%) 100 100 100 100 100 100
Data transferred (Mb) 4.08 4.26 7.28 7.64 11.56 11.9
Reponse time (s) 0.04 0.07 0.41 0.44 0.41 0.12
Transaction rate (/s) 84.89 86.25 161.37 161.19 250.76 250.78
Concurrency 3.85 5.84 65.67 71.08 102.82 30.17
Shortest transaction (s) 0 0 0 0 0 0
Longest transaction 0.22 0.27 7.12 7.16 7.13 0.36

The results were remarkably similar, I actually double checked to make sure I wasn't making requests to the same container. There are some discrepancies, but I would attribute them to statistical error.

Giving that we chose NodeJS for our project because of its resiliency, I think it is safe to say that Perfect is also a very good choice for APIs that are constantly under heavy load.