Extreme HTML5 Video Interactivity: Sending WebSocket Messages with Popcorn.js

One of our most popular demos at Kaazing is using a Web browser on a smartphone to control a physical toy truck from continents away. The truck has a Raspberry Pi attached to it, connecting to a WebSocket server, and listening to control command messages: drive forward or backward, turn left or right, or turn headlight on and off. You can learn more about the project in our Remote Controlling a Car over the Web. Ingredients: Smartphone, WebSocket, and Raspberry Pi blog post.

The most interesting way to demonstrate the truck is by having a remote person control the truck, and join in over a video conference. Here’s the recording of us doing just this. Fast forward to 4:08 for the truck demo.

Now, there are certain circumstances, when running Skype, or other live video chat apps is not an option. You may be off the grid, or simply not have anybody handy controlling your truck remotely.

To address this challenge, we wanted to create a self-contained environment where the same dialog and experience can be presented, but without all the above mentioned dependencies. To achieve this, we decided to record a video of someone operating the remote control that the presenter could use as the “Skyped-in” portion of the presentation. There are a number of ways we could hack this, for example by pretending to control the car by emulating the controls in the room. How cool would it be to instead have the recorded video actually trigger the remote controls using WebSocket messages? Instead of a real person controlling the car in real time, we could have the video control the car in real time.

First, we recorded the video. In the video, David Witherspoon operates the remote control. (Aside: David is a software engineer at Kaazing who, along with colleague Prashant Khanal, was instrumental in dreaming up and building the truck). David followed the script of the dialog very precisely. Knowing the script was not sufficient, we had to do it with exact timing, as it was specified to be run during the actual demo.

After processing the video, I embedded it in a web page, and overlaid the video with a live video feed of the presenter’s laptop camera. This is an important step to make the experience more realistic; after all, every video conference does this.

Here’s the HTML code:

	<head>
		<script src="http://popcornjs.org/code/dist/popcorn-complete.min.js"></script>
		<script src="http://demo.kaazing.com/lib/client/javascript/StompJms.js"></script>
		<script>document.addEventListener( "DOMContentLoaded", function() {doConnect();}, false );</script>
		<link rel="stylesheet" href="css/truck.css"</style>
	</head>
	<body>
		<video id="selfVideo" autoplay width="256"></video>
		<video id="truckVideo" width="1024">
			<!-- <source src="videos/PeterTruck.mp4" type="video/mp4"> -->
			<source src="http://localhost/videos/DavidTruck.mp4" type="video/mp4">
		</video>
	</body>
	<script src="js/truck.js"></script>	

And here is the JavaScript code:

var errorCallback = function(e) {
console.log('Reeeejected!', e);
};

// Not showing vendor prefixes.
navigator.getUserMedia  = navigator.getUserMedia ||
              navigator.webkitGetUserMedia ||
              navigator.mozGetUserMedia ||
              navigator.msGetUserMedia;

var video = document.getElementById ("selfVideo");

if (navigator.getUserMedia) {
  navigator.getUserMedia({audio: false, video: true}, function(stream) {
    video.src = window.URL.createObjectURL(stream);
  }, errorCallback);
} else {
  video.src = 'somevideo.webm'; // fallback.
}

The simple CSS snippet ensures that the presenter’s “self” video overlays the remote person’s (recorded) video.

#selfVideo
	{position:fixed;
	top:30px;
	left:30px;}

Also, browsers are required to prompt for permission before Web apps can start using the built-in camera. First, you have to select Allow, for the “little video” in the top left corner to appear. Here’s what the permission request bar looks like in Chrome:

usecamera

And here’s the end result:

popcornvideo

Whenever my actor friend in the main video uses his remote control, we must trigger a corresponding WebSocket message. The messages are sent by the Web app hosting the video at the exact time when the control is touched in the video. I used popcorn.js, an open source media library, to get the timing right:

Popcorn.js is an HTML5 media framework written in JavaScript for filmmakers, web developers, and anyone who wants to create time-based interactive media on the web. Popcorn.js is part of Mozilla’s Popcorn project.

I created an array with the timing and the messages that needed to be sent. The timing is measured in seconds.

var davidTruckMsgs = [
[33,"frontlight;on"],
[35,"frontlight;off"],
[36,"frontlight;on"],
[38,"frontlight;off"],
[42,"steering;right : thrust;off"],
[43,"steering;left : thrust;off"],
[44,"steering;right: thrust;off"],
[45,"steering;left : thrust;off"],
[48,"steering;off : thrust;forward"],
[50,"steering;off : thrust;backward"],
[51,"steering;off : thrust;forward"],
[52,"steering;off : thrust;backward"],
[86,"steering;left : thrust;forward"]
];

Then, we have to schedule the WebSocket messages, as defined in the array specified above. Note: The above array is called davidTruckMsgs, and down below we iterate over the truckMsgs array. As you can see in the completed source code, I have multiple arrays for various videos/actors. Whichever is the one used at the moment is referenced as truckMsgs later on.

for (var truckMsg in truckMsgs) {
  var obj = truckMsgs[truckMsg];
  pop.cue( obj[0], makeCallback( obj ) );
}
pop.play();

The makeCallback function invokes the actual logic sending the WebSocket message. If you’re wondering why this is all needed in the first place, check out this question on Stack Overflow.

function makeCallback(obj) {
    return function() {
    	doSend(session.createTextMessage(obj[1]));
    };
}

For usability, I added pause/continue functionality whenever the main video is clicked. This gives the presenter more control, allowing him/her to preload the page with the main video paused on it.

vid = document.getElementById ("truckVideo");
vid.addEventListener ('click', function() {
     vid.paused ? vid.play() : vid.pause();
});

For the WebSocket communication we used the JMS edition of the Kaazing WebSocket Gateway, allowing us to leverage simple pub/sub messaging concepts. With the help of popcorn.js, from the HTML5 web app we publish WebSocket messages to a so called topic, and whoever is interested in (read: subscribed to) it will receive it.

This way the video is driving the truck, simply by having the WebSocket messages sent out properly timed to the pre-recorded video.

Here’s the end result. Isn’t it awesome?

You can see the entire source code on github.

Posted in IoT, JMS, Kaazing, Uncategorized, WebSocket | Tagged | 3 Comments

The Internet of “Many Different Things”

by Vikram Mehta, Chief Executive Officer, Kaazing

Kevin Ashton coined the term the “Internet of Things” (IoT) in 1999 while working at Proctor & Gamble. At that time, the idea of everyday objects with embedded sensors or chips that communicate with each other had been around for over a decade, going by terms such as ‘ubiquitous computing’ and ‘pervasive computing’. What was new was the idea that everyday objects – such as a refrigerator, a car or a pallet – could connect to the Internet, enabling autonomous communication with each other and the environment.My new blog series – “Accelerating the Web for the Internet of Things” is inspired by Kevin Ashton’s more recent observation in The Economist’s informative IoT report.


giga-om-logo“What we have right now is a lot of IoT-type technology that is heavy on ‘things’ and light on Internet. That’s the bit that needs to change. ”
~ GigaOM


vikram-mehtaWhat does one the world’s largest energy-producing and trading companies have in common with three of the world’s top ten banks, one of America’s top three professional sports franchises, one of Europe’s most sophisticated rail transportation networks, one of the top three transportation companies in the USA, one of the world’s top five commercial airlines, one of the world’s best known cable and satellite networks, and one of the world’s most valuable retail brands?

They have all have seen the future – an always-on, always-connected, and always-informed world, a world in which every imaginable thing is connected to every other thing. More importantly, they’ve realized that being connected in such a manner to their suppliers, customers, employees, and machines that are at the center of their business, is central to their success.

There is something else these global enterprises have in common. They all rely on technology, products, and services from Kaazing, specially engineered to give them a head start on their competition in this brave new world, a world that has aptly been called the “Internet of Things” (IoT).

The Internet of Things is the next big thing for it will, amongst other things, help build intelligent cities; enable better management of the earth’s scarce resources; reduce energy consumption by giving us Internet control of the furnace at home; warn you that the transmission on your car is about to fail and scheduling a visit to the car service facility; help reduce the incidence of SIDS (Sudden Infant Death Syndrome); provide better and preventive care for the elderly; provide insurance companies with the ability to offer pay-as-you-go insurance to customers based on their driving behavior; provide retailers with new and unique ways to connect with and engage customers; enable payments in real time; and much more…

This brave new world will include millions of embedded electronic measuring devices: thermostats, pressure gauges, pollution detectors, cameras, microphones, glucose sensors, EKG machines, Electroencephalograph machines, and equipment that will probe and monitor everything from cities to endangered species. Sensors will monitor the atmosphere, our ships, highways and fleets of trucks, and even – in a movement called the Quantified Self (QS) – a person’s daily life in terms of inputs (e.g. food consumed, quality of surrounding air), states (e.g. mood, arousal, blood oxygen levels), and performance (mental and physical).

Everything and everyone will be connected and always communicating; a gigantic Web connecting Mobile users, Marketplaces, and Machines – in real time. A Web that is never fails to get information to its destination at lightning speed and securely.

When I think of IoT, I picture a gigantic freeway system that serves as a means of communication between many different things – machines communicating with other machines (M2M), humans communicating with other humans (P2P), and humans communicating with machines (P2M).

Some like GE have referred to an IoT world as the Industrial Internet, while others like Cisco have coined the term Internet of Everything.

I prefer to call it the “Internet of Many Different Things.” because while unwavering in its form, the Internet must for the first time since its invention accommodate many different types of communications, and do so at unprecedented scale. Some communications will be more sensitive to latency, speed, security, scale, delivery guarantees, and timeliness than others. And the Internet and systems that ride the Internet must have the ability to cope with it all.

___

In forthcoming blogs, I will explore how KAAZING helps enterprises brings this ability to life – for mobility, marketplaces and machines.

Posted in Featured, IoT, Uncategorized, WebSocket | Leave a comment

Kaazing at SFHTML5: All about WebGL

Kaazing was invited to speak at the upcoming San Francisco HTML5 User Group meeting on January 23, 2014 at Google’s San Francisco office. The title of the event is All about WebGL. I (Peter Moskovits) am representing Kaazing, sharing a speaking slot with Victor Sand from Goo Technologies. If you cannot make it to the event in person, the event is televised live streamed on Google Developers Live.

victorsand
Victor Sand – Goo Technologies

To get an idea what our talk is about, here’s an excerpt from our abstract:

Victor and Peter will demonstrate how WebGL experiences created with Goo Engine can be elevated to the next level by making them even more immersive and engaging. They will show how you can use any smartphone connected to the public Internet to remotely control WebGL applications and games. Last, Peter and team will demonstrate how this communication pattern can be applied to control Web connected physical objects from continents away.

As Vanessa Wang, co-organizer of the meetup, who not by the way runs documentation at Kaazing, stated in her meetup invitation:

One of the highlights of this event is a deep dive talk from a technical expert from Sony Entertainment Network/Playstation 4 on how the Playstation 4 UX was built with WebGL. This talk is part of a jam-packed technology event centered on WebGL, starting with a high-level introduction to WebGL, WebGL and Playstation UX, WebGL and Real-Time Communication, and finally a comprehensive talk on developing with WebGL, LEAP Motion, Three.js, WebRTC, and the Web Audio API.

If you’re at the event, remember to stop by the Kaazing table. If for nothing else, to drop your business card, as at the event we’ll be raffling off an iPad Mini Retina.

display_hero

Posted in html5, Kaazing, Uncategorized, WebSocket | Tagged | Leave a comment

Kaazing Powers the Internet of Things at HTML5DevConf October 2013 (San Francisco)

On 21-24 October 2013, HTML5DevConf returned to San Francisco, this time at the enormous Moscone Center North. As always, the sold-out conference attracted web developers, architects, business development leads, CTOs, CEOs, designers, students, QA engineers, technical writers — and many more — from all over the world. With the move from the Palace Hotel to Moscone North, the conference also saw a huge increase in attendance with filled rooms for all ten (yes, 10) tracks. The conference also held three days of training, all of which sold out weeks before the event.
HTML5DevConf logo
Kaazing was proud to host a booth on both days of the conference in Moscone’s famous Exhibit Hall. We showed lots of demos of using WebSockets and HTML5 to power objects (including a radio controlled truck), super-fast financial applications, as well as controlling Goo Technologies’ Pearl Boy demo with WebSockets. Here is one of our engineers, Sanjay, with the Android mascot …
sanjay-android-intel
… and our booth in the Exhibit Hall.
kaazing-booth
It was a fun event, and we even raffled off an iPad mini. Congratulations to the winner, Steve!
iPad mini winner
Our own Head of Training, Richard Clark (@rdclark), led a day-long training course on Native HTML5 Development. Kaazing’s Director of Technology, Frank Greco (@frankgreco) and Director of Product Management, Robin Zimmerman (@robinzim) gave a presentation to hundreds of attendees on WebSocket Perspectives and Visions for the Future. Frank’s slides are online; the recording will be available soon. Frank and Robin also showed some cool demos on using WebSockets and messaging (pub/sub) with a Raspberry Pi to control objects, including turning on a light using a light switch remotely over WebSockets. This demo was built by Kaazing engineers David Witherspoon (@DPW_Spoon) and Prashant Khanal (@ipras); check out how they built it.
frank-robin-demosfrank-robin-crowd
After Frank and Robin’s presentation, we returned to our booth and enjoyed more conversation with attendees on using messaging and WebSockets.
robin-boothfrank-booth
This was a really exciting and fun conference. Thanks so much to Ann and her awesome team of organizers. We’re definitely looking forward to the next one!

Posted in html5, training, Uncategorized, WebSocket | Tagged , , , , , , , | Leave a comment

Controlling Physical Devices on the Real Time Web: Kaazing IoT Talk at JavaOne 2013

Two visionary Kaazing engineers, David Witherspoon and Prashant Khanal, gave an exciting presentation at JavaOne this year: JMS, WebSocket, and the Internet of Things — Controlling Physical Devices on the Real Time Web.

DSC_1196

David and Prashant led the audience through the steps of building connected Things by combining open source hardware, a Raspberry Pi, with real-time Web communication and messaging, powered by Kaazing.

They started off by demonstrating Kaazing’s remote controlled truck. A remote member of the team joined the presentation over Skype and used a Web browser on his smart phone to control the truck in the conference room.

IMG_6014

Next, they walked the audience through building similar M2M systems. David and Prashant used the simplest “machines” for their demonstrations: a light bulb and a switch, each connected to a Raspberry Pi. All it took was three basic steps.

DSC_1197

1. Connecting Hardware with Software – the Magic of the Raspberry Pi

The software stack installed on the two Raspberry Pis:

This step consisted of two sub-steps. First, Prashant and David demonstrated how to leverage software running on the Pi to turn the light on/off. Then, they showed how the Raspberry Pi can detect the on/off state changes of the switch. It’s important to note that in this step, a switch was connected to a Pi and a lightbulb was connected to another Pi; the switch and the lightbulb (the two Things) are not yet connected.

Watch David demonstrate how software controls the two hardware pieces, the light and the switch.

Source Code for the Light

light

package com.kaaazing.demo.light;

import com.pi4j.io.gpio.GpioController;
import com.pi4j.io.gpio.GpioFactory;
import com.pi4j.io.gpio.GpioPinDigitalOutput;
import com.pi4j.io.gpio.PinState;
import com.pi4j.io.gpio.RaspiPin;

public class Light {

	private final GpioController gpio;
	private final GpioPinDigitalOutput gpioPin;

	public Light() {
		gpio = GpioFactory.getInstance();
		gpioPin = gpio.provisionDigitalOutputPin(RaspiPin.GPIO_01,
				"MyLight", PinState.LOW);
	}

	public void on() {
		gpioPin.high();
		System.out.println("Light on");
	}

	public void off() {
		gpioPin.low();
		System.out.println("Light off");
	}

	public void shutdown() {
		gpio.shutdown();
	}

	public static void main(String[] args) {
		Light lightController = new Light();
		System.out.println("Light controller initialized");
		try {
			Thread.sleep(2000);
			lightController.on();
			Thread.sleep(2000);
			lightController.off();
			Thread.sleep(2000);
			lightController.on();
			Thread.sleep(2000);
			lightController.off();
		} catch (InterruptedException e) {
			e.printStackTrace();
		} finally {
			lightController.shutdown();
		}
	}

}

First, in lines 16-17 a General Purpose I/O (GPIO) output pin is initialized. Then, in the on() and off() methods we set its value to high (line 21) and low (line 26). Eventually, we invoke the on() and off() methods.

Note: For the entire source code, scroll to the bottom of this post.

Source Code for the Switch

switch

package com.kaaazing.demo.toggle;

import java.util.concurrent.CopyOnWriteArrayList;

import com.pi4j.io.gpio.GpioController;
import com.pi4j.io.gpio.GpioFactory;
import com.pi4j.io.gpio.GpioPinDigitalInput;
import com.pi4j.io.gpio.PinPullResistance;
import com.pi4j.io.gpio.PinState;
import com.pi4j.io.gpio.RaspiPin;
import com.pi4j.io.gpio.event.GpioPinDigitalStateChangeEvent;
import com.pi4j.io.gpio.event.GpioPinListenerDigital;

public class Toggle {
	private final GpioController gpio;
	private final GpioPinDigitalInput gpioPin;
	private final CopyOnWriteArrayList listeners;

	public Toggle() {
		listeners = new CopyOnWriteArrayList();

		gpio = GpioFactory.getInstance();
		gpioPin = gpio.provisionDigitalInputPin(RaspiPin.GPIO_00,
				PinPullResistance.PULL_DOWN);
		gpioPin.addListener(new GpioPinListenerDigital() {

			@Override
			public void handleGpioPinDigitalStateChangeEvent(
					GpioPinDigitalStateChangeEvent event) {
				notifyListeners(event.getState());
			}
		});
	}

	private void notifyListeners(PinState state) {
		if (state == PinState.HIGH) {
			System.out.println("Toggle on");
		} else {
			System.out.println("Toggle off");
		}

		for (ToggleListener listener : listeners) {
			if (state == PinState.HIGH) {
				listener.on();
			} else {
				listener.off();
			}
		}
	}

	public void registerListener(ToggleListener listener) {
		listeners.add(listener);
		// send initial state
		if (gpioPin.getState() == PinState.HIGH) {
			listener.on();
		} else {
			listener.off();
		}
	}

	public void shutdown() {
		gpio.shutdown();
	}

	public static void main(String[] args) {
		Toggle toggle = new Toggle();
		System.out.println("Running toggle for 20 seconds");
		try {
			Thread.sleep(20000);
		} catch (InterruptedException e) {
			e.printStackTrace();
		} finally {
			toggle.shutdown();
		}
		System.out.println("Exiting");
	}
}

 

package com.kaaazing.demo.toggle;

public interface ToggleListener {
	public void on();
	public void off();
}

For the switch, first we initialize an input pin, and then write the listener code waiting for the toggle event.

Note: For the entire source code, scroll to the bottom of this post.

2. Pub-Sub Messaging to Build a Loosely Coupled System – Introducing the Message Broker

Next, David and Prashant demonstrated how to connect the lightbulb and its Pi with the switch and its Pi. While it’s easy enough to connect the switch and the lightbulb directly, they wanted to show the power of the publish-subscribe (pub-sub) messaging model, which lets you build a real-life system. By using a pub-sub model, one Thing can send a message to another Thing via a message broker to send signals to each Thing. To illustrate this, David and Prashant used the open source Apache ActiveMQ JMS message broker that ships pre-packaged with the JMS Edition of Kaazing WebSocket Gateway, but you can use a variety of JMS message brokers to achieve the same result. Kaazing WebSocket Gateway integrates with a number of different JMS message brokers, including TIBCO Enterprise Message Service, Informatica Ultra Messaging, IBM WebSphere MQ, JBoss Messaging, Open MQ Messaging, and Oracle WebLogic JMS.

In this case, the Pi attached to the switch is the publisher, and the Pi attached to the light is the subscriber.

message-broker

Watch David demonstrate the connected scenario.

Source Code for the Light

package com.kaaazing.demo.light;

import java.net.URI;

import javax.jms.Connection;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageConsumer;
import javax.jms.MessageListener;
import javax.jms.Session;
import javax.jms.TextMessage;
import javax.jms.Topic;

import org.apache.activemq.ActiveMQConnectionFactory;

import com.kaaazing.demo.util.AbstractJmsMessenger;
import com.kaaazing.demo.util.DefaultExceptionListener;

public class LightJmsTcpController extends AbstractJmsMessenger {

    private final ActiveMQConnectionFactory connectionFactory;
    private final Connection connection;
    private final Session session;
    private final Topic topic;
    private final MessageConsumer consumer;
    private final Light light;
    private boolean running = true;

    public LightJmsTcpController() {
        try {
            light = new Light();

            connectionFactory = new ActiveMQConnectionFactory(URI.create("tcp://"
                    + BROKER_HOSTNAME + ":61616"));
            connection = connectionFactory.createConnection();
            connection.setExceptionListener(new DefaultExceptionListener());
            session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
            topic = session.createTopic(LIGHT_TOPIC);
            consumer = session.createConsumer(topic);
            consumer.setMessageListener(new MessageListener() {

                @Override
                public void onMessage(Message message) {
                    try {
                        TextMessage textMessage = (TextMessage) message;
                        String messageData = textMessage.getText();
                        System.out.println("message received: " + messageData);
                        if (messageData.equals(ON_MESSAGE)) {
                            light.on();
                        } else if (messageData.equals(OFF_MESSAGE)) {
                            light.off();
                        } else if (messageData.equals(SHUTDOWN)) {
                            System.out.println("shutting down");
                            shutdown();
                        }
                    } catch (JMSException e) {
                        e.printStackTrace();
                        shutdown();
                    }
                }
            });

            connection.start();
            System.out.println("Light initialized");
        } catch (JMSException e) {
            shutdown();
            throw new RuntimeException(e.getMessage());
        }
    }

    private void shutdown() {
        running = false;
        if (light != null) {
            light.shutdown();
        }
        if (connection != null) {
            try {
                System.out.println("Cleaning up resources");
                connection.close();
            } catch (JMSException e) {
                throw new RuntimeException(e.getMessage());
            }
        }
    }

    public static void main(String[] args) throws JMSException, InterruptedException {
        LightJmsTcpController lightController = new LightJmsTcpController();
        while (lightController.isRunning()) {
            Thread.sleep(1000);
        }
        System.out.println("Exiting Application");

    }

    public boolean isRunning() {
        return running;
    }

}

In this demo first we create an instance of the Light class (line 31), then a TCP connection to ActiveMQ (lines 33-35). Then, we create a JMS session (line 37), a topic (line 38), and a consumer (line 39), and define the message listener (line 40) that’s invoked when a new message arrives. Depending on the message, the light is turned on or off (lines 48-55).

Note: For the entire source code, scroll to the bottom of this post.

Source Code for the Switch

package com.kaaazing.demo.toggle;

import java.net.URI;

import javax.jms.Connection;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageConsumer;
import javax.jms.MessageListener;
import javax.jms.MessageProducer;
import javax.jms.Session;
import javax.jms.TextMessage;
import javax.jms.Topic;

import org.apache.activemq.ActiveMQConnectionFactory;

import com.kaaazing.demo.util.AbstractJmsMessenger;
import com.kaaazing.demo.util.DefaultExceptionListener;

public class ToggleJmsTcpListener extends AbstractJmsMessenger implements
        ToggleListener {

    private final ActiveMQConnectionFactory connectionFactory;
    private final Connection connection;
    private final Session session;
    private final Topic topic;
    private final MessageProducer producer;
    private final Toggle toggle;
    private final MessageConsumer consumer;
    private boolean running = true;

    public ToggleJmsTcpListener() {

        try {
            toggle = new Toggle();

            connectionFactory = new ActiveMQConnectionFactory(URI.create("tcp://"
                    + BROKER_HOSTNAME + ":61616"));
            connection = connectionFactory.createConnection();
            connection.setExceptionListener(new DefaultExceptionListener());
            session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
            topic = session.createTopic(LIGHT_TOPIC);
            producer = session.createProducer(topic);

            // clean shutdown listener
            consumer = session.createConsumer(topic);
            consumer.setMessageListener(new MessageListener() {
                @Override
                public void onMessage(Message message) {
                    try {
                        TextMessage textMessage = (TextMessage) message;
                        String messageData = textMessage.getText();
                        if (messageData.equals(SHUTDOWN)) {
                            System.out.println("shutting down");
                            shutdown();
                        }
                    } catch (JMSException e) {
                        e.printStackTrace();
                        shutdown();
                    }
                }
            });

            connection.start();
            System.out.println("Toggle initialized");
            toggle.registerListener(this);

        } catch (JMSException e) {
            shutdown();
            throw new RuntimeException(e.getMessage());
        }
    }

    @Override
    public void on() {
        try {
            System.out.println("Sending on message");
            producer.send(session.createTextMessage(ON_MESSAGE));
        } catch (JMSException e) {
            e.printStackTrace();
            shutdown();
        }
    }

    @Override
    public void off() {
        try {
            System.out.println("Sending off message");
            producer.send(session.createTextMessage(OFF_MESSAGE));
        } catch (JMSException e) {
            e.printStackTrace();
            shutdown();
        }
    }

    private void shutdown() {
        running = false;
        if (toggle != null) {
            toggle.shutdown();
        }
        if (connection != null) {
            try {
                System.out.println("Cleaning up resources");
                connection.close();
            } catch (JMSException e) {
                throw new RuntimeException(e.getMessage());
            }
        }
    }

    public static void main(String[] args) throws JMSException,
            InterruptedException {
        ToggleJmsTcpListener toggleJmsTcpListener = new ToggleJmsTcpListener();
        while (toggleJmsTcpListener.isRunning()) {
            Thread.sleep(1000);
        }
        System.out.println("Exiting Application");
    }

    public boolean isRunning() {
        return running;
    }

}

Similar to the lightbulb, for the switch we create a TCP connection to ActivemMQ (line 37-39). After creating the JMS session (line 41), a topic (line 42), we create a JMS message producer (line 43). When the switch is toggled, we create and send a text message (lines 78 and 89) on the topic.

Note: For the entire source code, scroll to the bottom of this post.

3. Extending the Reach to the Web – WebSocket Transport with Kaazing WebSocket Gateway

The challenge with a TCP-based system, like the one outlined in Step 2, is that TCP has difficulty connecting isolated networks. To use TCP connections, you must open ports on firewalls and address the challenges of penetrating network intermediaries. Also, connectivity with mobile and Web applications raises issues. If you want to build a truly global system, you must ensure that your Things are always connected, no matter what.

The good news is that you can very easily extend your JMS-based applications to the Web, simply by introducing Kaazing WebSocket Gateway in the picture. With Kaazing, you can connect your devices in a secure fashion with low latency using open industry standard Web communication: HTML5 WebSocket.

WebSocket is a natural choice for asynchronous scenarios with event-driven architecture, such as the switch and the light. As the switch is toggled (event), the light (subscriber) is notified of the event asynchronously (without explicitly polling). It’s easy to see that using traditional synchronous Web programming models, like REST, don’t fit the bill as well as the asynchronous pattern that WebSocket supports.

For connected devices, reliability is critical. The moment a connection is lost due to network disruption or other errors, the Kaazing client will automatically try to restore that connection. The Kaazing client libraries will also re-subscribe to any active subscriptions prior to the connection failure.

In this demo, the switch and its Pi can be in Spokane, WA, while the light and its Pi can reside in Kathmandu, Nepal. As long as they’re connected to the public Internet, the JMS messages can traverse the Web, and the switch can control the light.

message-broker-kaazing

Watch David demonstrate the WebSocket connected scenario.

Source Code for the Light

package com.kaaazing.demo.light;

import java.net.URI;

import javax.jms.Connection;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageConsumer;
import javax.jms.MessageListener;
import javax.jms.Session;
import javax.jms.TextMessage;
import javax.jms.Topic;

import com.kaaazing.demo.util.AbstractJmsMessenger;
import com.kaaazing.demo.util.DefaultExceptionListener;
import com.kaazing.gateway.jms.client.stomp.StompConnectionFactory;

public class LightJmsWsController extends AbstractJmsMessenger {

	private final StompConnectionFactory connectionFactory;
	private final Connection connection;
	private final Session session;
	private final Topic topic;
	private final MessageConsumer consumer;
	private final Light light;
	private boolean running = true;

	public LightJmsWsController() {
		try {
			light = new Light();

			connectionFactory = new StompConnectionFactory(URI.create("ws://"
					+ GATEWAY_HOST + "/jms"));
			connection = connectionFactory.createConnection();
			connection.setExceptionListener(new DefaultExceptionListener());
			session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
			topic = session.createTopic(LIGHT_TOPIC_WS);
			consumer = session.createConsumer(topic);
			consumer.setMessageListener(new MessageListener() {

				@Override
				public void onMessage(Message message) {
					try {
						TextMessage textMessage = (TextMessage) message;
						String messageData = textMessage.getText();
						System.out.println("websocket message received: " + messageData);
						if (messageData.equals(ON_MESSAGE)) {
							light.on();
						} else if (messageData.equals(OFF_MESSAGE)) {
							light.off();
						} else if (messageData.equals(SHUTDOWN)) {
							System.out.println("shutting down");
							shutdown();
						}
					} catch (JMSException e) {
						e.printStackTrace();
						shutdown();
					}
				}
			});

			connection.start();
			System.out.println("Light initialized");
		} catch (JMSException e) {
			shutdown();
			e.printStackTrace();
			throw new RuntimeException(e.getMessage());
		}
	}

	private void shutdown() {
		running = false;
		if (light != null) {
			light.shutdown();
		}
		if (connection != null) {
			try {
				System.out.println("Cleaning up resources");
				connection.close();
			} catch (JMSException e) {
				throw new RuntimeException(e.getMessage());
			}
		}
	}

	public static void main(String[] args) throws JMSException,
			InterruptedException {
		LightJmsWsController lightController = new LightJmsWsController();
		while (lightController.isRunning()) {
			Thread.sleep(1000);
		}
		System.out.println("Exiting Application");

	}

	public boolean isRunning() {
		return running;
	}

}

Notice that when switching from TCP to WebSocket, the only change you have to make to your code is replace the connection factory class, the protocol, and the port number.

Note: For the entire source code, scroll to the bottom of this post.

Source Code for the Switch

package com.kaaazing.demo.toggle;

import java.net.URI;

import javax.jms.Connection;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageConsumer;
import javax.jms.MessageListener;
import javax.jms.MessageProducer;
import javax.jms.Session;
import javax.jms.TextMessage;
import javax.jms.Topic;

import com.kaaazing.demo.util.AbstractJmsMessenger;
import com.kaaazing.demo.util.DefaultExceptionListener;
import com.kaazing.gateway.jms.client.stomp.StompConnectionFactory;

public class ToggleJmsWsListener extends AbstractJmsMessenger implements
        ToggleListener {

    private final StompConnectionFactory connectionFactory;
    private final Connection connection;
    private final Session session;
    private final Topic topic;
    private final MessageProducer producer;
    private final Toggle toggle;
    private final MessageConsumer consumer;
    private boolean running = true;

    public ToggleJmsWsListener() {

        try {
            toggle = new Toggle();
			connectionFactory = new StompConnectionFactory(URI.create("ws://"
                    + GATEWAY_HOST + "/jms"));
            connection = connectionFactory.createConnection();
            connection.setExceptionListener(new DefaultExceptionListener());
            session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
            topic = session.createTopic(LIGHT_TOPIC_WS);
            producer = session.createProducer(topic);

            // clean shutdown listener
            consumer = session.createConsumer(topic);
            consumer.setMessageListener(new MessageListener() {
                @Override
                public void onMessage(Message message) {
                    try {
                        TextMessage textMessage = (TextMessage) message;
                        String messageData = textMessage.getText();
                        if (messageData.equals(SHUTDOWN)) {
                            System.out.println("shutting down");
                            shutdown();
                        }
                    } catch (JMSException e) {
                        e.printStackTrace();
                        shutdown();
                    }
                }
            });

            connection.start();
            System.out.println("Toggle initialized");
            toggle.registerListener(this);

        } catch (JMSException e) {
            shutdown();
            e.printStackTrace();
            throw new RuntimeException(e.getMessage());
        }
    }

    @Override
    public void on() {
        try {
            System.out.println("Sending websocket on message");
            producer.send(session.createTextMessage(ON_MESSAGE));
        } catch (JMSException e) {
            e.printStackTrace();
            shutdown();
        }
    }

    @Override
    public void off() {
        try {
            System.out.println("Sending websocket off message");
            producer.send(session.createTextMessage(OFF_MESSAGE));
        } catch (JMSException e) {
            e.printStackTrace();
            shutdown();
        }
    }

    private void shutdown() {
        running = false;
        if (toggle != null) {
            toggle.shutdown();
        }
        if (connection != null) {
            try {
                System.out.println("Cleaning up resources");
                connection.close();
            } catch (JMSException e) {
                throw new RuntimeException(e.getMessage());
            }
        }
    }

    public static void main(String[] args) throws JMSException,
            InterruptedException {
        ToggleJmsWsListener toggleJmsWsListener = new ToggleJmsWsListener();
        while (toggleJmsWsListener.isRunning()) {
            Thread.sleep(1000);
        }
        System.out.println("Exiting Application");
    }

    public boolean isRunning() {
        return running;
    }

}

Interestingly, the source code used for the WebSocket-enabled scenario is almost identical to the one used in Step 2. As the highlighted code snippet indicates (lines 35-36), all you need to change is the connection information: simply modify the connection factory, the protocol, and the port number.

Note: For the entire source code, scroll to the bottom of this post.

To keep the audience engaged (and to emphasize the “color” of the WebSocket server used for the demos – Kaazing’s color is orange), for the WebSocket demo Prashant replaced the white light bulb with an orange one.

DSC_1202

Slides

You can also review the entire deck as presented at JavaOne.

Source Code & Download

The entire source code for the demonstration is published under David’s GitHub account. Star it, fork it, tweet it!

You can also download a free, fully functional version of the Kaazing WebSocket Gateway – JMS Edition that we used. To learn more, check out these resources:

Finally, you can meet David and Prashant in person and learn more about Kaazing and our platform at the upcoming HTML5DevConf (22-23 October at Moscone North in San Francisco, CA).

Tell us about your Raspberry Pi project – and drop a comment if you’re (interested in) building Internet (or Web) of Things applications.

Posted in Events, html5, JMS, Kaazing, WebSocket | Tagged , , , , , , | Leave a comment

Real Time Data Panel Discussion at Edge Conference 2013

At the Edge Conference in New York City last week a panel of 5 industry experts discussed some of the exciting questions and challenges of real time data. Here’s the abstract of the discussion:

WebSockets have now been with us for several years and enjoy near universal support. WebRTC is just starting out. Chat, live blogging, telemetry/dashboards, remote assist and video conferencing are all proven and popular use cases for real time data. Developers are becoming increasingly comfortable integrating real time elements into new projects, but problems remain with the complexity of scaling a real-time backend, as well as implementing APIs that are practical and useful.

john-fallowsBesides John Fallows, CTO and co-founder of Kaazing, Henrik Joreteg (&yet), Martyn Loughran (Pusher), Wesley Hales (Apigee), and Rob Hawkes (ViziCities) were invited to be on the panel, moderated by Scott Jenson (Jenson Design).

Here are a few deep links into interesting questions and topics addressed during the panel:

Question (14:26): WebSocket and other real-time protocols are commonly blocked by corporate proxies and content inspection firewalls; and that’s a particular problem for the sort of customers we have at the FT. How much is this stifling adoption and what do we do about it?

Question (22:42): Will the WebSocket protocol replace Server-Sent Events in the future? Why do we have to have both specs if WebSocket can accomplish the same tasks that SSE does, and more?

Question (35:30): We see a rise of security issues; people are trying to open too many TCP connections to a Web server and are killing the Web server as a result of it. Therefore, people tend to disable keep-alive connections to deal with this issue. What do you think about WebSocket security related to this problem and how can we solve it?

John on enabling the Web of Things (49:21)

Posted in Events, Kaazing, Security, WebSocket | Leave a comment

Kaazing Presentation at JavaZone

javazoneKaazing co-founder and CEO, Jonas Jacobi, gave a talk at JavaZone in Norway, titled 6 Years Later: One Line of Code That Changed the Web Forever. Jonas and his co-founder, John Fallows gave a presentation at JavaZone 5 years ago, and talked about the WebSocket standard the very first time in public.

In his talk, Jonas touches upon a number of interesting topics, including:

  • The challenges of HTTP
  • How the WebSocket standard addresses those challenges
  • The WebSocket protocol handshake and protocol
  • Protocol layering on top of WebSocket
  • Deployment topologies
  • Scalability
Posted in html5, JMS, Kaazing, Security, Uncategorized, WebSocket | Leave a comment