FOSDEM: The Challenges and Secrets of the Realtime World
This is a simple, high-level introduction to “realtime”. Many apps have “realtime” aspects: Periscope, Gett (taxis), various IoT things, drones. How can we do realtime? Not by polling!! There are many “realtime protocols” - HTTP streaming, HTTP long polling, HTTP/2, Kafka, WebSocket, XMPP. The common element is TCP - they are all a layer on top of a TCP socket. HTTP long polling is very popular. In usual implementation, the client makes a request signalling interest in something, and the server only responds once an interesting thing happens. The client then immediately makes another request. To avoid dropping messages, the re-request should include a timestamp or sequence number. This also accounts for connectivity issues - especially on mobile, connections drop frequently, and ideally should be re-connected transparently. Most/all of these have a concept of “channels”, or “topics”, or “interests” - a thing which clients can “publish” and “subscribe” to. Scaling pub/sub (in PubNub’s case) means operating multiple servers, and connecting clients to their closest server using GeoDNS. You then need some way to communicate publishes between those servers.
More by Jim
- Project C-43: the lost origins of asymmetric crypto
- How Hacker News stays interesting
- My parents are Flat-Earthers
- The dots do matter: how to scam a Gmail user
- The sorry state of OpenSSL usability
- I hate telephones
- The Three Ts of Time, Thought and Typing: measuring cost on the web
- Granddad died today
- Your syntax highlighter is wrong
I wrote this because I felt like it. This post is not associated with my employer. Found an error? Edit this page.