Have you ever noticed that if you start a timer in your web browser, your timesheet will show that fresh timer in your phone, or in any of our desktop apps? No need for reloading or manual refreshes, it just takes a moment for any of your Harvest apps to be in sync.
If you are an Administrator viewing or editing another user’s timesheet, you will also see live updates to that timesheet, whether they’re made by the owner of the timesheet, or yourself. And it would be confusing otherwise! These live timesheet updates prevent incorrect data, and remove a lot of friction from time tracking.
Our need to implement live updates appeared shortly after Harvest was initially released 15 years ago, since the same timesheet could be viewed and edited from different browsers or browser tabs. Once we released our iPhone app, there were no excuses — we needed to implement a way to keep our timesheet data consistent across any apps and browsers ‘connected’ to it.
Our first approach for keeping timesheets fresh across devices was to use polling. Polling was straightforward: we just needed to make browsers and our mobile apps ask for timesheet updates from the server on a given interval. But if we make apps ask for updates every few seconds, that’s going to put an extra load on our servers, and the devices themselves. If we set the polling interval to a minute or more, the application will feel unresponsive.
As we released apps for Android, Mac and Windows, and added integrations through the Harvest Button and Widget, our customers started to use a higher number of devices to track time through the day. Polling was inefficient and unsustainable if we wanted to support our ecosystem of apps and integrations
We moved onto pushing updates to browsers and apps using WebSockets, which we built as a separate service using Node.js. Over time, all the platforms we use added good support for Server Sent Events (SSE). WebSockets allow bidirectional communication between server and client, while with SSE only servers can push events to connected clients.
The features in SSE are enough for our use case, and managing the life cycle of SSE connections is a bit simpler than with WebSockets. This is how we currently generate, transmit and push the events that support live timesheet updates:
- Harvest generates an event for each action on the app: starting or stopping a timer, creating a project, editing a user, etc.
- Those events are pushed through a Pub/Sub service. We use this service and a set of Pub/Sub consumers to support several internal services, as well as some of our integrations, like Slack.
- For the specific case of live updates, a consumer reads events and makes them available to the SSE / WebSockets service.
- The SSE / WebSockets service relays the event to all connected browsers / apps that need the update. This includes the timesheet owner, and any Administrators, if any of these users are currently connected to receive live timesheet updates.
The SSE API has a few features that make pushing events to our connected clients a lot easier:
- When a client (browser or app) loads a timesheet, it also initiates an SSE connection using our WebSockets / SSE services as event source.
- Clients will automatically reconnect to the event source if the connection is lost.
- The SSE API is just HTTP, so we can use an existing session in Harvest to authenticate connected clients.
- Good old HTTP means that we don’t need external libraries, we can send events to clients with a few lines of our own code.
Most of the logic in our SSE / WebSockets service focuses on whether we should forward a given event to a given connected client, depending on:
- The event type: for live updates, we need to receive timer start/stop events, timer edits and deletions, etc., but also events for projects, tasks, project assignments, because they affect the available choices for tracking time.
- The timesheet a client is connected to: if an Administrator is viewing another user’s timesheet on a given client, there’s no point in forwarding events for the Administrator’s own timesheet to that client.
- The permissions of the user linked to the connected client. Currently, only Administrators can view other users’ timesheets. But in a possible future, Managers might have partial access to other users’ timesheets. In that case, only events from projects where the Manager is a PM on should be forwarded to the client.
Live timesheet updates have been in production for a few years now, and we have had a smooth ride since we switched to SSE a couple of years ago. Because live updates just work most of the time, maybe you hadn’t noticed them before. But if you do, now you’ll know what’s happening under the hood!