AtScale Blog

We Told You It Would Be Cloudy

You have probably already heard all about it, read all about it and know all about it, but if you are anything like me, (or Einstein) you have already figured out that the more we learn about BI on Big Data the more we realize how little we know. Or, do we?

The Forecast Calls for Cloudy Weather

You don’t have to be a clairvoyant to know that there is an ever-increasing trend in cloud adoption among start-ups and enterprises alike.  In the world of Big Data, the past few years have shown a significant increase in cloud adoption.  While Amazon initially led the way with cloud data products - with Amazon Elastic MapReduce (EMR) for Hadoop and Redshift for data warehousing - the past 12 months have seen new entrants on the scene.

Is it October already?

It’s hard to believe that October is here. It feels like only a few days ago that we released AtScale 5.0 and AtScale 5.5.  Both releases contained a number of great new features that I was excited to share with the Big Data and Business Intelligence communities. Although I may be a little biased, I really do believe that the release of AtScale 6.0 marks one of the biggest releases in the history of the company. 

Despite the challenges associated with data warehousing, enterprise IT leaders have accepted it as a necessary evil of deriving value from information within Hadoop and other Big Data ecosystems. How much does it cost to create data warehouses or datamarts that extract data out of Hadoop? Is there a better way to do BI on Big Data?!

Industry leaders know that their challenges with data analytics are spread across 4 areas: confirmation (‘the things they know they know’); intuition (‘the things they don’t know they know’); inspection (‘the things they know they don’t know’); and revelation (‘the things they don’t know they don’t know). 

This last category is often the most tragic for organizations.  Luckily, we now have helpful resources from Tom Davenport's latest webinar on Data Strategy to key research data points on chief data executive's priorities.

The rapidly exploding demand for business intelligence on big data is nothing new - this trend is clearly indicated in the latest Big Data Maturity surveys (2015 and 2016).  As shown in the graphic below, 75% of respondents are planning on deploying BI workloads on their big data platforms (with 73% of respondents already with some BI use cases deployed).

Every once in awhile, the ultimate question comes up: "What is the best analysis tool for BI on Hadoop?!"  AtScale is not in the business of favoring one tool versus the other.  We are in the business of making all of them work.  There are indeed many reasons why business users and IT departments choose particular analysis tools.   Here are a few things to consider. 

Conducting exploratory data analysis or even basic business intelligence on Hadoop often requires input from data scientists who: 

• Create models for related information across Hadoop.
• Structure databases to contain that data.
• Ensure different BI tools generate consistent results when referencing identical data elements.

How many data scientists you'll have to hire depends on the size of your organization and the composition of the data under its control. In this post we review how much it typically costs to hire a data scientist, the factors you'll have to consider when assembling a team and ways you can alleviate the workload placed on data scientists. 

In the world of Business Intelligence and Big Data there continue to be a number of exciting innovations as new and improved options for processing large data sets appear on the market.  You may be familiar with AtScale’s BI-on-Hadoop Benchmarks - where we focus on evaluating the top SQL-on-Hadoop engines and their fitness to support traditional BI-style queries.  As we continue to work with customers who are navigating their journey to BI on Big Data, we are increasingly getting questions about the emerging cloud-based data processing engines.

 In this blog post, we will take a deeper look at BigQuery from Google, and how it stacks up in the BI-on-Big Data ecosystem.

I’ve asked it before and I’ll ask it again. Wouldn’t it be great if you could easily analyze ALL your data from a Excel single file? We all know this isn’t feasible; especially when dealing with big data and complex business analytics needs.

In working at the intersection of Big Data and traditional Business Intelligence, the AtScale team has encountered a number of complex business analytics use cases that are difficult, if not near-impossible, to solve using typical table-based data models and SQL. Today, I’m going to share why and how complex analysis, like for multi-level metrics, is no longer as ‘difficult’ nor ‘near-impossible’ as it once was.