WebSocket, Big Data, and Non-Blocking Server-Side I/O with Kaazing, MongoDB, and Node.js

In this example, we look at how real-time Web communications can work well with a non-blocking JavaScript back-end server and a NoSQL database. Our demo is a simple drawing app. If all you use is WebSocket with simple pub-sub messaging, you can already achieve pretty amazing things with drawing. For example, you can share a drawing board among multiple browser clients and while viewing each other’s drawings in real time.

Screen Shot 2013-06-03 at 12.05.19 PM

If you add a server-side component to it, you can do some central coordination and processing. And if on top of that you use a persistent storage, you can record all your real-time interactions to the database, and if needed play it all back. Icing on the cake is the ability of setting the replay speed: speeding it up or slowing it down.

Here’s what we did:

Read on to see how we did it.

The Architecture

Now let’s see how it’s all done.

The browsers connect to the Kaazing WebSocket Gateway. We use a higher level pub/sub API, JMS; this keeps our JavaScript code, running in the browser, simple. The JMS Edition of Kaazing comes out-of-the-box with an open source message broker, Apache ActiveMQ. The primary role of the message broker is to route the messages between the message publishers and the message subscribers. In addition to the browsers, we have Node.js as well connecting to the message broker. To achieve all this, we use a STOMP (Simple or Streaming Text Oriented Protocol) connector package for Node.js. By doing so Node.js will receive the same messages that the browsers receive. As the messages arrive to Node.js, they’re quickly passed on to MongoDB, which in turn will persist them. When the playback time comes, the messages are retrieved from MongoDB, and Node.js schedules and plays them back to the message broker, so they can be sent to the browsers for display.

Installing Kaazing and the Multi-Client WebSocket Drawing App

The simplest way of getting started is by installing the WebSocket app. To do so, you need to get a WebSocket server. You can download, install, and start a fully functional unlimited developer licensed Kaazing WebSocket Gateway in a few minutes. The simple steps are documented and shown here. Kaazing comes in various flavors (editions). As mentioned above, we use the JMS Edition, bundled with the open source Apache ActiveMQ JMS message broker, giving us the power of messaging on top of HTML5 WebSocket. Once you have the WebSocket server and the message broker up and running, all you need to do is copy the application code into a sub-directory: $GATEWAH_HOME/web/extras. We called ours draw. You can download the application from github.

Screen Shot 2013-06-04 at 9.51.28 AM

After starting up the Kaazing WebSocket Gateway, you’ll be able to access the application through the following URL:
http://YOUR_IP_ADDRESS:8001/draw
For example:
http://192.168.4.106:8001/draw

This will invoke the index.html file, pulling in the required JavaScript and CSS files from the /js and /css directories.

At this point, you have a collaborative mini drawing app. The app uses simple color coding: red is used to display drawing happening in this browser, and yellow is used when the drawing is originating from somewhere else, that is another browser window or played back from the database.

Installing MongoDB

mongodbThe next step is installing MongoDB, an open source NoSQL document database. It’s as simple as it gets, and is all documented here.

Installing Node.js

nodeThe last component is Node.js, an event-driven, non-blocking I/O server, allowing you to build data-intensive real-time apps in a lightweight manner. Node.js uses JavaScript on the server side.

The Application

Now, let’s walk through how the app was built.

HTML5 App in the Browser

The client-side HTML5 app consists of a couple of simple files: index.html, js/draw.js, and css/draw.css.

The HTML file renders the visible objects (canvas, button, and speed slider) on the page, the CSS file sets the visual attributes for them.

The draw.js file contains all the application logic. On the one hand it detects mouse gestures and takes care of the drawing, on the other hand it handles the communication with the message broker: it publishes messages when you’re drawing on the canvas, and consumes messages when somebody else is drawing or when a previously recorded drawing is played back from MongoDB and Node.js.

The browser can send two “types” of messages: control messages and drawing messages. Control messages are startRecordingstopRecordingplayBack, and deleteRecording.

If you’d like to learn more how you can use WebSocket with pub/sub messaging to build collaborative apps, this Tutorial is a great starting point.

Server-Side Code in Node.js

The app running in Node.js communicates with the message broker through STOMP. The server-side code is located in server.js. Before starting it, make sure you have Apache ActiveMQ running, as the server is attempting to connect to upon start. To start the server, simply run:

$>node server.js

After successfully connecting to it, the app will subscribe to drawTopic (/topic/drawTopic) that the browsers are publishing to. When the browser sends a control message, Node.js works hand-in-hand with MongoDB.

Recording

When recording, we save the coordinates of the pixels, along with the timestamp. Saving the timestamp will allow us to play back the recording in a properly timed manner (longer pauses in the drawing are recorded and played back as such).

Playing Back

The play-back functionality is done by Node.js querying the contents of the database (pixels to be drawn), and then scheduling the messages for publishing through the message broker. The messages will then eventually result in actual pixels drawn on the canvas(es) in the browser(s). We use the timestamp to set the delay between the messages.

Deleting the Recording

We can very easily delete the previous recording by deleting the collection from MongoDB.

More Information

  • Check out the project on github.
  • Try following the tutorial for building a simple peer-to-peer application yourself.
  • Learn more about Kaazing WebSocket Gateway – JMS Edition, which is a messaging over WebSocket solution using the Java Message Service API.
  • Check out the API documentation and other how-tos for building clients using Kaazing WebSocket Gateway – JMS Edition.
This entry was posted in html5, JMS, Kaazing, WebSocket. Bookmark the permalink.

6 Responses to WebSocket, Big Data, and Non-Blocking Server-Side I/O with Kaazing, MongoDB, and Node.js

  1. Alexey says:

    what about webrtc vs websocket?

  2. Maximos says:

    Just to make it a little bit easier for others that want to install the components in order to hands on checkout this tutorial:
    First install the node modules required npm install XXX where replace xxx with mongodb, rsvp and stomp (a package.json would be really useful :D )
    Then on at least in my case there was a change in the RSVP module so there where some minnor changes in the gateway.js:
    change “new RSVP.promise()” with “new RSVP.defer()” and “gateway.initPromise.then” with “gateway.initPromise.promise.then”
    Also the github code is missing the /lib/StompJms required in the index.html, change the last with “http://demo.kaazing.com/lib/client/javascript/StompJms.js” in the code and then everything works like a charm!
    Really nice tutorial and the sample is really impressive, keep up the good work.

  3. Maurya says:

    Thanks for the nice tutorial, but i am quite new to node.js. Trying to figure out where do we run the server.js from? node.js–node_modules – HOME ? . Also tested node.js installation works for test.js..example

    @Maximos – where did you install the npm installs? I have not used the default directory have customized the install ..Any

    • Maximos says:

      Hello Maurya,

      you need either to provide a package.json file to node, or use node -g install MODULE_NAME (please read some related documentation for more info, for example this one https://blog.nodejitsu.com/package-dependencies-done-right/).
      After you install all the dependencies that I wrote on the previous comment, then cd into the directory where the server.js is and run: node server.js.

      best,
      Maximos

  4. I am using TIBCO EMS for kaazing. In node js STOMP is not connecting to TIBCO which is running in my local (port=7222).
    Is there any node module like we have for Javascript. (jmsClient.js)? Or is there any alternative way to connect to kaazing in node js.

    new stomp.Stomp(config.stomp_args); // Note working when I change active MQ to TIBCO EMS

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s