hcoelho.com

my blog

Title only : Full post

Benchmark: NodeJS x Perfect (Swift)

:

Yesterday (October 27th, 2016) I went to a presentation called "How to Completely Fail at Open-Sourcing", presented by Sean Stephens, at FSOSS - Free Software and Open Source Symposium hosted at Seneca College. Sean Stephens is the CEO of PerfectlySoft Inc, the company that developed Perfect, a library for server-side Swift development. This immediately caught my attention: I've been thinking about server-side development in Swift for a while, and it seems that it finally happened.

During the presentation, Sean showed us some benchmarks where Swift (using the Perfect framework) beat NodeJS in several fronts. You can see more details in this post. Since I recently benchmarked PHP x NodeJS (around a month ago), I decided to use a similar scenario and test Perfect x NodeJS. This is how I set it up:

I wanted 2 servers: one with Perfect, and the other one with pure NodeJS. For every request, they would go to MongoDB, fetch all results, append some text to the response, and send it back. I used siege as the stress tester, in order to simulate concurrent connections.

I set up a virtual machine with 1 processor core, 512Mb of RAM and 20Gb of storage; the machine was running Debian Jessie. In this machine, I installed Docker and made 3 images:

1st image: MongoDB

Dockerfile
------------------
FROM ubuntu:16.04
MAINTAINER Docker
RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv EA312927
RUN echo "deb http://repo.mongodb.org/apt/ubuntu $(cat /etc/lsb-release | grep DISTRIB_CODENAME | cut -d= -f2)/mongodb-org/3.2 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-3.2.list
RUN apt-get update && apt-get install -y mongodb-org
VOLUME ["data/db"]
WORKDIR "data/"
EXPOSE 27017
ENTRYPOINT ["/usr/bin/mongod"]

2nd image: NodeJS

Dockerfile
------------------
FROM node:wheezy
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY package.json /usr/src/app/
RUN npm install
COPY . /usr/src/app
EXPOSE 8000
CMD [ "node", "index.js" ]


index.js
------------------
var http = require('http');
var mongodb = require('mongodb');

mongodb.connect('mongodb://10.46.52.207:27017/test', function (err, db) {
  if (err) { console.log(err); }
  http.createServer(function (req, res) {
    var s = "";
    for (var i = 1; i <= 1000; i++) {
      s += '' + i;
    }
    db.collection("test").find({}).toArray(function(err, doc) {
      res.end("Hello world" + JSON.stringify(doc) + s);
    });
  }).listen(8000);

});


package.json
------------------
{
  "name": "node",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "mongodb": "^2.2.10"
  }
}

Perfect (Swift)

Dockerfile
------------------

# Copyright (C) 2016 PerfectlySoft Inc.
# Author: Shao Miller 

FROM perfectlysoft/ubuntu1510
RUN /usr/src/Perfect-Ubuntu/install_swift.sh --sure
RUN apt-get install libtool -y
RUN apt-get install dh-autoreconf -y
RUN git clone https://github.com/mongodb/mongo-c-driver.git
WORKDIR ./mongo-c-driver
RUN ./autogen.sh --with-libbson=bundled
RUN make
RUN make install
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY . /usr/src/app
RUN swift build
CMD .build/debug/App --port 8000


Package.swift
------------------
import PackageDescription
let package = Package(
  name: "App",
  targets: [],
  dependencies: [
    .Package(
      url: "https://github.com/PerfectlySoft/Perfect-HTTPServer.git",
      majorVersion: 2,
      minor: 0),
    .Package(
      url:"https://github.com/PerfectlySoft/Perfect-MongoDB.git", 
      majorVersion: 2,
      minor: 0)
  ]
)


Sources/main.swift
------------------
import PerfectLib

import PerfectHTTP
import PerfectHTTPServer
import MongoDB

let server = HTTPServer()

var routes = Routes()
routes.add(method: .get, uri: "/", handler: {
  (request, response)->() in
    response.setHeader(.contentType, value: "text/html")

    let client = try! MongoClient(uri: "mongodb://10.46.52.207:27017")
    let db = client.getDatabase(name: "test")
    guard let collection = db.getCollection(name: "test") else { return }

    let fnd = collection.find(query: BSON())

    var arr = [String]()
    for x in fnd! {
      arr.append(x.asString)
    }

    defer {
      collection.close()
      db.close()
      client.close()
    }

    var s = ""

    for x in 1...1000 {
      s += String(x)
    }

    response.appendBody(string: "Hello world {\(arr.joined(separator: ","))}\(s)")
    response.completed()
  }
)

server.addRoutes(routes)

server.serverPort = 8000

do {
  try server.start()
} catch PerfectError.networkError(let err, let msg) {
  print("Network error thrown: \(err) \(msg)")
}

By the way, I'm sorry if my Swift code is not Swifty enough - I am just a JavaScript peasant. But anyway, these are the results I got:

500 users 1,000 users 1,500 users
NodeJS Perfect NodeJS Perfect NodeJS Perfect
Number of hits 1284 1273 2293 2284 3641 3556
Availability (%) 100 100 100 100 100 100
Data transferred (Mb) 4.08 4.26 7.28 7.64 11.56 11.9
Reponse time (s) 0.04 0.07 0.41 0.44 0.41 0.12
Transaction rate (/s) 84.89 86.25 161.37 161.19 250.76 250.78
Concurrency 3.85 5.84 65.67 71.08 102.82 30.17
Shortest transaction (s) 0 0 0 0 0 0
Longest transaction 0.22 0.27 7.12 7.16 7.13 0.36

The results were remarkably similar, I actually double checked to make sure I wasn't making requests to the same container. There are some discrepancies, but I would attribute them to statistical error.

Giving that we chose NodeJS for our project because of its resiliency, I think it is safe to say that Perfect is also a very good choice for APIs that are constantly under heavy load.

cdot node perfect swift 

JavaScript Generators

:

Asynchronous programming in JavaScript seems to be a double edged sword: on one side, you have a program that does not block the I/O, on the other side, you have Callback Hell and promises. People seem to love promises, I absolutely hate them. Yes, I know they simplify Callback Hell, but it they don't do it well - they are still ugly and messy. There. I said it.

So, what exactly is the problem again? The problem is: suppose you have a series of asynchronous functions that you want to execute sequentially:

function asyncFunction1() {
    setTimeout(function () {
        console.log('- 1');
    }, 1000);
}

function asyncFunction2() {
    setTimeout(function () {
        console.log('- 2');
    }, 500);
}

function asyncFunction3() {
    setTimeout(function () {
        console.log('- 3');
    }, 0);
}

function run() {
    asyncFunction1();
    asyncFunction2();
    asyncFunction3();
}

run();

/*
I wanted:
- 1
- 2
- 3
Result:
- 3
- 2
- 1
*/

This didn't work because the last function executed much faster than the first one. Ok, what is the solution? Callbacks:

function asyncFunction1(cb) {
    setTimeout(function () {
        console.log('- 1');
        cb();
    }, 1000);
}

function asyncFunction2(cb) {
    setTimeout(function () {
        console.log('- 2');
        cb();
    }, 500);
}

function asyncFunction3(cb) {
    setTimeout(function () {
        console.log('- 3');
        cb();
    }, 0);
}


function run() { // EEEEWWW -v
    asyncFunction1(() => {
        asyncFunction2(() => {
            asyncFunction3(() => {});
        });
    });
}

run();

/*
- 1
- 2
- 3
*/

It solved the problem, but now the run function looks messy and weird. So how exactly generators can solve this problem?

First, I want to explain what is a generator: it is a function that can be stopped and resumed. Useless, right? This is how it looks like:

// The notation function* () indicates a generator
function* Sequence() {
    // The yield keyword stops the function at that point
    yield 1;
    yield 2;
    yield 3;
}

// Here we are instantiating the generator - it did not run yet!
const sequence = Sequence();

// When we call .next(), the generator resumes
console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next());

/*
What we get from a .next is an object with 2 memebers: value,
which is the value that was yielded, and done, a boolean
indicating if we are done or not. In this example, we called
.next() 4 times:
{ value: 1, done: false }
{ value: 2, done: false }
{ value: 3, done: false }
{ value: undefined, done: true }
*/

It doesn't seem very useful yet, but we can get some interesting functionality from it. In the case below, the generator will never be completed, but every time you call .next(), it will increment a counter in 1:

function* Sequence() {
    let i = 0;
    while (true) { yield i++; }
}

const sequence = Sequence();

console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next());

/*
{ value: 0, done: false }
{ value: 1, done: false }
{ value: 2, done: false }
{ value: 3, done: false }
{ value: 4, done: false }
*/

Or maybe we could use it to square the number, instead of incrementing:

function* Sequence() {
    let i = 2;
    while (true) { yield i, i *= i; }
}

const sequence = Sequence();

console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next());

/*
{ value: 2, done: false }
{ value: 4, done: false }
{ value: 16, done: false }
{ value: 256, done: false }
{ value: 65536, done: false }
*/

Another important aspect of a generator is that the 'yield' keyword returns a value: the value is whatever is passed as a parameter for the .next() function. In the next example, I'm using the value passed to .next() to reset my sequence:

function* Sequence() {
    let i = 1;
    while (true) {
        const reset = yield i++;
        if (reset !== undefined) { i = reset; }
    }
}

const sequence = Sequence();

console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next());
console.log(sequence.next(100));
console.log(sequence.next());
console.log(sequence.next());

/*
{ value: 1, done: false }
{ value: 2, done: false }
{ value: 3, done: false }
{ value: 100, done: false }
{ value: 101, done: false }
{ value: 102, done: false }
*/

It is important to notice that in order to reach a yield, you need to call a .next() first, otherwise the parameter will simply get tossed away. This example will not print the first parameter:

function* Sequence() {
    while (true) { console.log(yield); }
}

const sequence = Sequence();

sequence.next("a");
sequence.next("b");
sequence.next(1);
sequence.next(2);

/*
b
1
2
*/

So, where exactly do generators solve the problem with asynchronous functions? Ok. Let's start easy. Suppose we have a sequence of 3 typical functions, being the one in the middle asynchronous. This is how we would make them execute in sequence:

function asyncFunction(value, cb) {
    setTimeout(function () {
        console.log(value);
        cb();
    }, 1000);
}

function syncFunction(value) {
    console.log(value);
}

// This function executes them in the order we want
function run() {
    syncFunction('- 1');
    asyncFunction('- 2', function () {
        syncFunction('- 3');
    });
}
run();

/*
- 1
- 2
- 3
*/

Notice the pattern: when we have an asynchronous function (asyncFunction), we give it a callback (syncFunction) to execute when it is done. Now here is the magic: if the function "run" was a generator instead of a normal function, we could make it yield right after the asynchronous function call, and instead of the asynchronous function calling the next function when it is done, it would resume the generator!

This is how it could look like:

let run;

function asyncFunction() {
    setTimeout(function () {
        console.log('- 2');
        run.next();
    }, 1000);
}

function syncFunction(value) {
    console.log(value);
}

// Notice how cleaner the block of this function is
function* Run() {
    syncFunction('- 1');
    yield asyncFunction('- 2');
    syncFunction('- 3');
}

run = Run();
run.next();

/*
- 1
- 2
- 3
*/

A bit messy, right? But you probably agree that the function "run" (now the generator "Run") looks a lot nicer. We can make a wrapper in order to make it look much cleaner.

In this case, I made a function called executeGenerator, which takes care of the ugly part. This is how nice a sequence of 5 function (3 of them asynchronous) would look like using this wrapper:

function syncFunction1() {
    console.log('- 1');
}

// Notice how the asynchronous functions still think they are
// using classic callbacks - this means that libraries that
// rely on callbacks would still be compatible
function asyncFunction2(cb) {
    setTimeout(function () {
        console.log('- 2');
        cb();
    }, 1000);
}

function asyncFunction3(cb) {
    setTimeout(function () {
        console.log('- 3');
        cb();
    }, 500);
}

function asyncFunction4(cb) {
    setTimeout(function () {
        console.log('- 4');
        cb();
    }, 0);
}

function syncFunction5() {
    console.log('- 5');
}

function* run() {
    syncFunction1();
    yield asyncFunction2; // <- no function call here, we
    yield asyncFunction3; //    are yielding the asynchronous
    yield asyncFunction4; //    functions themselves*
    syncFunction5();
}

executeGenerator(run);

/*
- 1
- 2
- 3
- 4
- 5
*/
  • If you had to pass parameters in that case, you could just use .bind to prepare the function.

It not only looks a lot simpler and cleaner, it is still compatible with asynchronous functions that use callbacks. This is how my wrapper looks like:

function executeGenerator(Gen) {
    const gen = Gen();

    function cb() {
        executeTask();
    }

    function executeTask() {
        const nextTask = gen.next();
        const nextAsyncFunction = nextTask.value;
        const isDone = nextTask.done;
        if (isDone) { return; }
        nextAsyncFunction(cb);
    }

    executeTask();
}

There are 2 sub-functions there: 1- cb is a fake callback (I'll explain what it does later) 2- executeTask gets the yielded values from the generator (the asynchronous functions) and executes them, sending the fake callback we just made as the callback

We first start the generator, and then resume it to get the first yielded value (the asynchrnous function), we then execute it passing a callback that will execute the task again: it will resume the generator (which will get the next yielded asynchronous function) and execute the next asynchronous function with the fake callback, which will resume the generator, and so on until our generator is done.

There are several libraries that simplify asynchronous programming with generators, like "co". Despite being hard to understand in the beginning, they are a great alternative to classic callbacks and promises (at least until "async" and "await" are implemented, which will probably happen for the next version of JavaScript).

cdot javascript es7 generators 

Making a User Control module for DotNetNuke

:

I had to take a small break from JavaScript and Node.js in the past few days and work with ASP.NET: our client module in the front-end needed some information from the server, but there was no way we could retrieve it from the DOM, so I had to develop a small User Control module for DotNetNuke (an ASP.NET framework) that passes this information to our module in the frontend. A User Control module is a module that can be embedded in the skin of the website, so it can be called for every page.

This is how it works: in the backend, we get the information, put it in a stringified JSON, and include it in the DOM of the webpage. For instance(I had to use square brackets instead of angle brackets because WordPress was breaking my page - again, use your imagination):

View.ascx
<%=getServerInfo(); %>

View.ascx.cs
String getServerInfo() {
    return "[script]window.info = 1;[/script]";
}

With this, the window instance will have an "info" object, which can be accessed by the JavaScript in the frontend:

<html>
<head>
    <script>window.info = 1;</script>
    <script>alert(window.info);</script>
</head>
<body>...</body>
</html>

This is very simple, but my C# skills were a bit rusty, and I've never worked with DotNetNuke before. These are the files (with pseudocode, of course) that I had to create in order to get a module running:

View.ascx
<%@ Control Language="C#" CodeBehind="View.ascx.cs" Inherits="MyClass.Controller" %>
[asp:Literal ID="placeholder" runat="server"][/asp:Literal]

The file above is responsible for creating the View of the module, as well as linking it to its "codebehind" (the logic behind the view), specifying which classes it implements, and making a placeholder for our JSON to be inserted.

View.ascx.cs
using ...;

namespace MyClass
{
    public partial class Controller : System.Web.UI.UserControl
    {
        protected void Page_Load(object sender, EventArgs e)
        {
            placeholder.Text = "";
        }
    }
}

The file above is the "codebehind" of the module - as soon as the page loads, it will replace our placeholder.

View.ascx
<dotnetnuketype="Package">
  <packages>
    <package name="View" type="SkinObject">
      <friendlyName>My Module</friendlyName>
      <components>
        <component type="SkinObject">
          <moduleControl>
            <controlKey>Bridge</controlKey>
            <controlSrc>/DesktopModules/MyClass/View.ascx</controlSrc>
          </moduleControl>
        </component>
        <component type="Assembly">
          <assemblies>
            <assembly>
              <path>bin</path>
              <name>View.dll</name>
            </assembly>
          </assemblies>
        </component>
        <component type="File">
          <files>
            <basePath>DesktopModules\MyClass</basePath>
            <file>
              <name>View.ascx</name>
            </file>
          </files>
        </component>
      </components>
    </package>
  </packages>
</dotnetnuke>

The file above is responsible for defining the module: it tells DotNetNuke what it is.

View.ascx
<%@ Register TagPrefix="dnn" TagName="MYCLASS" Src="~/DesktopModules/MyClass/View.ascx" %>
<dnn:MYCLASS ID="dnnMyClass" runat="server" />

The snippet above is inserted in the skin of the portal, so it can be called in all pages.

After the module is compiled into a DLL, it can be used by the website.

cdot asp.net dnn dotnetnuke