Comet in Erlang with Mochiweb and a Finite State Machine
<Comet is a pretty nebulous technology, encompassing everything from really-frequent XHR requests to the cutting-edge WebSocket protocol. Sadly, WebSocket is a bit too cutting-edge for my tastes, so for my app, I’ve chosen a long-polling approach based loosely on the Bidirectional-streams Over Synchronous HTTP protocol. By loosely I mean really loosely; I’ve really only kept the notion of request ids and session ids.
The comet connection consists of two HTTP connections, only one of which is long-polling at any one time. It’ll wait a maximum of a minute for any server-side information. Whenever the client needs to send data to the server, there’s another available connection. I’ve implemented the comet client in Cappuccino, but you can do it with any JavaScript framework.
Mochiweb is a lightweight HTTP toolkit for Erlang. I chose Erlang for this project because it’s rock solid with easy scaling, and the lightweight process / message passing system makes coding Comet easy.
Here’s the Mochiweb request handler:
The io:format calls are unnecessary; they’re just there for me to keep an eye on things while I develop the system. The important bit is the two calls to inet:setops/2. Internet sockets in Erlang can be in either active mode, in which incoming data is sent as messages to the associated Erlang process, or passive mode, which means that process needs to manually request data when it’s ready for it. Mochiweb works in passive mode, which has one major disadvantage; you don’t get notified if the socket dies. So, before we switch into waiting for data, we set the socket into {active, once}. This means that one message will be allowed to be sent to the process before setting it right back into passive mode. We can’t get additional data from the client on this socket at this point in time; we’ve already received the entire request and have yet to send any response. So, the only message we can receive from the socket is {tcp_closed, Socket}. That way we avoid trying to send messages from the server to a socket that’s died.
I use this gen_server (I’ve only shown the internals; the rest of the gen_server is pretty basic) to make session ids both unique and unpredictable.
Finally, here’s the meat of the system: the connection FSM:
I’ll break it down function by function.
handle_json/1 is called by the comet request handler I showed above. It decodes the JSON request body and determines if this is a request to set up a comet connection, or a request on a connection that already exists. handle_setup/1 is pretty obvious, but handle_request/3 is a little more complex. It sends an event to the FSM that there’s a new request and sits down to wait for a message — forever. It also traps that {tcp_closed, Socket} message I mentioned earlier; if that happens it tells the FSM it went away and dies. But otherwise it returns the message back to the client.
Finally, handle_packet/2 will be called by the app logic, once I write it, to deliver messages to the client.
So much for the FSM interface. The internals of the FSM live in the same module, in the callback functions.
start/0 and init/1 are pretty straightforward. I want to mention the dietimer here, though. If 45 seconds elapses without a waiting request from the client, we assume the client’s gone offline and tear down the corresponding FSM. We’ll kill or reset the timer in plenty of event handlers, though.
The next bunch of functions handle events sent with gen_fsm:send_event/2. This is the heart of the FSM. There are three states: waiting (for both requests and packets), have_request (but no packet), and have_packet (but no request). They should also all be pretty straightforward.
This took a bit of time to set up, but I think it’s easy to follow, which is the important part. There’s still a bit of work to be done; I haven’t set up the actual app logic that the messages will be going to or coming from. But this is a good start.