Introducing the web client

Let’s take a break from low-level details and talk about using a web client as a stream-ing client. Usstream-ing a web client, we no longer need to think about a lot of those details because the web client isn’t used to interact with a third-party API that may be trans-actional and is perhaps not idempotent. Instead the web client is primarily used for displaying the results of a stream. We will often need to keep state and potentially

perform some computations. Although there is and will continue to be a tremendous amount of growth in the industry around building clients that act on data, being able to use a browser to display the results of a streaming application is today and will for quite some time be a useful tool because it allows people to understand their business.

Some may question the use of a browser for rendering a stream, calling it “eye-candy.”

At times I can agree with that, although recently I was working with two clients that used a browser-based dashboard for the following two reasons:

 It was the first time they could understand how people used their website and understand how their business was doing. You may think you can get this from a typical analytics report, but when you see the stream of data, you see that it’s alive and lets you see a different story unfold.

 A major news publisher used a browser-based dashboard in its editorial bull-pen so editors could see in real time how their articles were performing. They could use the tool to help make decisions on what types of social promoting to try and drive ad revenue for their articles. This is powerful and may be hard to automate.

If you have a stream flowing at a high velocity, a browser-based client will undoubtedly fall behind in reading it. That may sound like a terrible thing, but I equate it to decid-ing which HD TV to buy. When I compare the picture of a lower-end HD TV to a high-end HD TV, I may notice subtle differences in the color saturation or sharpness, but the picture doesn’t change. This same idea applies to a dashboard in a browser that can’t keep up with a stream. It may be missing some of the data points, but the overall picture of the business and trends doesn’t change. In working with hundreds of cus-tomers I’ve seen this concept proved out over and over. Everyone says they want abso-lute 100% accuracy in the numbers, but when it comes down to it, being able to tell the correct story and show the correct trends is more than satisfactory.

That doesn’t mean we don’t need to worry about anything. We will undoubtedly have to keep state and may have to perform some basic computations. Earlier I talked about maintaining state using web storage and IndexedDB in a browser. Without get-ting too bogged down by the code, let’s see how we would want to use those technolo-gies to store state and perform basic computations. Figure 8.11 illustrates a web-based client that’s showing a stacked bar graph of running sales per rep.

In figure 8.11, as the data is flowing from the streaming API, we’ll be using JavaScript in the browser to update the in-browser local storage. There are two options when storing the current state of the graph in local storage: store the latest values received from the streaming API or the current state of the entire graph. The benefit to storing the entire state of the graph is that if the user closes the page and comes back to it, we have some state to start off with, and if we use local storage and not the session storage, then the state will persist after the user closes the browser.

There are two things to keep in mind with this. First, before deciding to store the data in local storage and having it persist after the browser is shut down, you will want

to check with the appropriate security personnel to ensure it’s appropriate. Secondly, there may be many times when the streaming API is not sending the running totals but only the incremental changes. When this occurs you want to be able to keep track of the changes and increment the values accordingly. If you find you have to do this for a lot of different data points, consider using the IndexedDB storage option and not local storage.

8.3.1 Integrating with the streaming API service

When it comes to integrating a browser-based client with a streaming API there’s a lot of flexibility. Out of the box you should be able to integrate with APIs that expose a stream using HTTP Long Polling, WebSockets, and server-sent events. The next listing and listings 8.2 and 8.3 illustrate these technologies.

var source = new EventSource("<SSE Service URL>");

source.onmessage = function(event) {

document.getElementById("event").innerHTML += event.data + <br>";

};

Listing 8.1 Server-Sent Events

Web based streaming client

Events Streaming

data API Process

messages

Local storage

On receipt of the message update the localStorage to keep track of the current

state of the graph. Then update the graph.

25 50 100

0 75

Apple Orange Peach Mango Blueberry Banana Grapes

Running Orders by Rep

John Jane Mary

Figure 8.11 Web-based client using local storage to maintain state of a graph

Connects to the server-sent event server

The function called every time a new message is received

(function pollServer() { setTimeout(function() {

$.ajax({ url: "<Server To Talk To>", success: function(data) { document.getElementById("event").innerHTML += data.value + <br>";

}, dataType: "json", complete: pollServer });

}, 30000);

})();

function webSocket() {

if ("WebSocket" in window) { // open a web socket

var ws = new WebSocket("ws://<url to service>");

ws.onmessage = function (event) {

document.getElementById("event").innerHTML += event.data + <br>";

};

} }

Listings 8.1 through 8.3 should give you a sense for how each of the integrations would work. Both server-sent events and WebSockets work on a message-by-message basis, allowing you to easily and cleanly apply your business logic. Arguably, the Long Polling does as well, but remember, with the Long Polling example it’s the browser that’s initiating the request by setting a timeout to execute the request each time.

Regardless of which technology is used by a streaming API, you should be able to integrate with it using JavaScript and a browser. Chapter 9 walks through this in more detail, and we’ll build out a client application.