Your browser is no longer supported! Please upgrade your web browser now.

A Gift of Time

Seasons greeting! As the year winds down, I’d like to share some improvements I’ve made to our database this year. You may not have noticed them, but you’ve probably felt them when you run reports or load pages in Harvest. Many Harvest pages should feel much snappier!

Last year we were unhappy with how long it took for several of our reports to retrieve data from the database. We have monster database servers, so we knew hardware wasn’t the issue. But still, we saw large projects could wait on data up to 12 seconds, making page renders and refreshes noticeably slow. This issue grew worse for our customers when we redesigned our Projects section to incorporate more reporting, which ran several slow queries next to management actions.

Over the years we’ve tried several strategies to improve our performance, including caching report information. However, we prefer to show you the most up-to-date data whenever possible, and some projects are so active that the cached data doesn’t last long anyway.

The problems with data retrieval and poor load times boiled down to the way we store and retrieve your reporting data. There was no simple fix, like adding a missing database index or moving to a different technology stack. Instead, we precomputed key project report details and eliminated all SQL JOINs from our report. In the coming months I’ll post more details about these changes to our technical blog, Techtime, so stay tuned to that blog if you’re interested in the nitty gritty details!

Remember that 12-second report I mentioned earlier? With our database improvements it now runs in about 1.5 seconds. That may not seem like a lot, but when you’re a manager going through different projects, and each project page can run several queries, that time adds up. Saving 10 seconds for a project manager who checks on only two project reports a day saves them an extra two hours a year! That’s two hours they would have wasted waiting on a spinner for the page to load.

You may not have noticed these improvements to make your reports snappier unless you were paying close attention. But, we consider that a success! One of the key goals of this project has been to make improvements without disrupting you. This means our database maintenance, report verification, and performance rollout have taken significant thought and planning to ensure you could continue to use Harvest without interruption

This was a difficult post to write, because for you, my work has been nearly invisible. There are no new features to use, buttons to press, or reports to run. But, your experience in Harvest is now faster and smoother. Though you may not see a difference, you may feel one. We’re happy you’ll be spending far less time in Harvest, and more time doing the things that matter.

Thoughts or questions about this post? Need some help?
Get in touch →

This was posted in Product News.
  • This post just warmed my heart. Seriously. Happy holidays!

  • Thanks Andrew, that really means a lot!

  • This is a very welcome upgrade. Thanks!

    Out of curiosity, do you precompute that data on a timer, or when new data comes in?

    Is it being saved to a different table properly indexed for the report queries?

  • Hey Ben,

    I’ll get into more in detail on Techtime, but we’re precomputing some key details of projects, like the billable rate, when time entries are saved. We do calculations when different reports run, but we don’t have to have SQL JOINS against our largest tables to find all the key components needed in those calculations.

    > Is it being saved to a different table properly indexed for the report queries?

    Nope! That was something I researched and considered early on, but I decided against maintaining a separate table in order to run reports. The cost of regenerating that data when something big changed just felt too high for the benefits.

    We may reconsider that design again in the future, but until then, the new system should scale nicely as a project grows.

  • Steve Gosling on December 19, 2016

    Thanks for this upgrade – appreciated!

  • Isaiah Macadam on December 19, 2016

    Thank you Zach!

    Merry Christmas!

Leave a comment…

Your email address will not be published. Required fields are marked. *