blog

Benchmark: NodeJS x Perfect (Swift)

Yesterday (October 27th, 2016) I went to a presentation called "How to Completely Fail at Open-Sourcing", presented by Sean Stephens, at FSOSS - Free Software and Open Source Symposium hosted at Seneca College. Sean Stephens is the CEO of PerfectlySoft Inc, the company that developed Perfect, a library for server-side Swift development. This immediately caught my attention: I've been thinking about server-side development in Swift for a while, and it seems that it finally happened. During the presentation, Sean showed us some benchmarks where Swift (using the Perfect framework) beat NodeJS in several fronts. You can see more details in this post. Since I recently benchmarked PHP x NodeJS (around a month ago), I decided to use a similar scenario and test Perfect x NodeJS. This is how I set it up: I wanted 2 servers: one with Perfect, and the other one with pure NodeJS. For every request, they would go to MongoDB, fetch all results, append some text to the response, and send it back. I used siege as the stress tester, in order to simulate concurrent connections. I set up a virtual machine with 1 processor core, 512Mb of RAM and 20Gb of storage; the machine was running Debian Jessie. In this machine, I installed Docker and made 3 images: ```Dockerfile #1st image: MongoDB FROM ubuntu:16.04 MAINTAINER Docker RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv EA312927 RUN echo "deb http://repo.mongodb.org/apt/ubuntu $(cat /etc/lsb-release | grep DISTRIB_CODENAME | cut -d= -f2)/mongodb-org/3.2 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-3.2.list RUN apt-get update && apt-get install -y mongodb-org VOLUME ["data/db"] WORKDIR "data/" EXPOSE 27017 ENTRYPOINT ["/usr/bin/mongod"] ``` ```Dockerfile # 2nd image: NodeJS FROM node:wheezy RUN mkdir -p /usr/src/app WORKDIR /usr/src/app COPY package.json /usr/src/app/ RUN npm install COPY . /usr/src/app EXPOSE 8000 CMD [ "node", "index.js" ] ``` ```javascript // index.js var http = require('http'); var mongodb = require('mongodb'); mongodb.connect('mongodb://10.46.52.207:27017/test', function (err, db) { if (err) { console.log(err); } http.createServer(function (req, res) { var s = ""; for (var i = 1; i <= 1000; i++) { s += '' + i; } db.collection("test").find({}).toArray(function(err, doc) { res.end("Hello world" + JSON.stringify(doc) + s); }); }).listen(8000); }); ``` ```json // package.json { "name": "node", "version": "1.0.0", "description": "", "main": "index.js", "scripts": { "test": "echo "Error: no test specified" && exit 1" }, "author": "", "license": "ISC", "dependencies": { "mongodb": "^2.2.10" } } ``` ```Dockerfile # Copyright (C) 2016 PerfectlySoft Inc. # Author: Shao Miller FROM perfectlysoft/ubuntu1510 RUN /usr/src/Perfect-Ubuntu/install_swift.sh --sure RUN apt-get install libtool -y RUN apt-get install dh-autoreconf -y RUN git clone https://github.com/mongodb/mongo-c-driver.git WORKDIR ./mongo-c-driver RUN ./autogen.sh --with-libbson=bundled RUN make RUN make install RUN mkdir -p /usr/src/app WORKDIR /usr/src/app COPY . /usr/src/app RUN swift build CMD .build/debug/App --port 8000 ``` ```swift // Package.swift import PackageDescription let package = Package( name: "App", targets: [], dependencies: [ .Package( url: "https://github.com/PerfectlySoft/Perfect-HTTPServer.git", majorVersion: 2, minor: 0), .Package( url:"https://github.com/PerfectlySoft/Perfect-MongoDB.git", majorVersion: 2, minor: 0) ] ) ``` ```swift // Sources/main.swift import PerfectLib import PerfectHTTP import PerfectHTTPServer import MongoDB let server = HTTPServer() var routes = Routes() routes.add(method: .get, uri: "/", handler: { (request, response)->() in response.setHeader(.contentType, value: "text/html") let client = try! MongoClient(uri: "mongodb://10.46.52.207:27017") let db = client.getDatabase(name: "test") guard let collection = db.getCollection(name: "test") else { return } let fnd = collection.find(query: BSON()) var arr = [String]() for x in fnd! { arr.append(x.asString) } defer { collection.close() db.close() client.close() } var s = "" for x in 1...1000 { s += String(x) } response.appendBody(string: "Hello world {(arr.joined(separator: ","))}(s)") response.completed() } ) server.addRoutes(routes) server.serverPort = 8000 do { try server.start() } catch PerfectError.networkError(let err, let msg) { print("Network error thrown: (err) (msg)") } ``` By the way, I'm sorry if my Swift code is not Swifty enough - I am just a JavaScript peasant. But anyway, these are the results I got:  
500 users 1,000 users 1,500 users
NodeJS Perfect NodeJS Perfect NodeJS Perfect
Number of hits 1284 1273 2293 2284 3641 3556
Availability (%) 100 100 100 100 100 100
Data transferred (Mb) 4.08 4.26 7.28 7.64 11.56 11.9
Reponse time (s) 0.04 0.07 0.41 0.44 0.41 0.12
Transaction rate (/s) 84.89 86.25 161.37 161.19 250.76 250.78
Concurrency 3.85 5.84 65.67 71.08 102.82 30.17
Shortest transaction (s) 0 0 0 0 0 0
Longest transaction 0.22 0.27 7.12 7.16 7.13 0.36
The results were remarkably similar, I actually double checked to make sure I wasn't making requests to the same container. There are some discrepancies, but I would attribute them to statistical error. Giving that we chose NodeJS for our project because of its resiliency, I think it is safe to say that Perfect is also a very good choice for APIs that are constantly under heavy load.