When Do You Need a Data Warehouse?

A question we often receive from clients is "when do we need to invest in a data warehouse?" (and therefore the skills required to build and maintain the data warehouse). These are generally organizations that are familiar with the value of their data, may be in the early stages of their digital transformation journey and are currently building or growing their analytics practice. They may be using a combination of Excel, Google Data Studio and Google Analytics for reporting.

Though every company is different and we absolutely must evaluate your unique scenario, I generally start with the following factors to guide the conversation.

Are you analyzing data from multiple sources?

For example, if you're attempting to analyze data from your CRM, ERP, Paid Media Campaigns and Web Analytics you could benefit from a data warehouse. Establishing a single source of truth will benefit your analytics capalities in various ways.

Do you need to main separate security of your application data from you analytics data? G

enerally, we'd recommend a data wareshouse when maintaining granuluar permissions and security are necessary.

Do you need to separate your application (transactional) data from your analytics data?

As your business grows, the volume of transactions increase and the complexity of your queries increase, you'll want to process your analytics workloads in a different system than your transactions. The assist the performance of your queries while avoiding decreasing the peformance of your app.

Do you need to increase the performance of your analytics queries?

Data Warehouse servcies such as Azure Datawarehouse (now Synapse), AWS Redshift, Snowflake, Google BiqQuery, etc. are designed specifically to provide excellent query performance. Additionally, you'll often need to aggregate your log level data into summary data that can be more efficiently queried.

Again, every client and scenario are different, but we generally start with a discussion of the factors in helping clients navigate this decision.