AtScale Blog

 The joy of working as a Customer Success Solution Architect is that I have the opportunity to work with many different customers and each challenges us with a different Big Data use case.

I've worked with enterprises that offload their Netezza database into the cloud. I've seen companies analyze social media data in real-time. I've helped teams streamline operational processes and increase efficiency in production lines. Big Data provides enterprises a competitive advantage and reduces operational costs across a these varied scenarios. However, setting up a big data environment is not for the faint-hearted - or is it?

Often times, customers approach me with questions around AtScale’s ability to integrate into the customer’s operational stack.  Today, I want to highlight a component of AtScale’s Development Extensions called Webhooks.

 A webhook (also called a web callback or HTTP push API) is a way for an application to provide other applications with real-time information. A webhook delivers data to other applications as certain events occur, meaning you get data immediately as opposed to a REST API which you would need to poll for data very frequently in order to get it close to real-time. This makes webhooks much more efficient for both provider and consumer.

March is gone and Spring has arrived, at least for many of us. A lot happened in March, and we certainly don't want you to miss out on what’s big on big data. Without further ado, here is what you might have missed in March.

Congratulations!  Your data is controlled, aggregated and turbocharged in your AtScale virtual cube.  You have Tableau to create remarkable visualizations. Your data is happy! But are your cube designers and business users too? For instance, did you know that centralizing calculations in your AtScale Virtual Cube eliminates TDE perpetuation, 3rd party ETL processes and version control headaches?  For an enhanced AtScale experience, here are 5 Best Practices you should be implementing in order to maximize AtScale on Tableau.

Data Lake Intelligence with AtScale

In my recent Data Lake 2.0 article I described how the worlds of big data and cloud are coming together to reshape the concept of the data lake. The data lake is an important element of any modern data architecture, and the data lake footprint will continue to expand. However, the data lake investment is only one part of delivering a modern data architecture. At Yahoo!, in addition to building a Hadoop-based data lake, we also needed to solve the problem of connecting traditional business intelligence workloads to this Hadoop data. Although the term “Data Lake” didn’t exist back then, we were solving the problem of: “How can you deliver an interactive BI experience on top of a scale-out Data Lake” - it turns out we were pioneers in delivering Data Lake Intelligence.

Our experiences and learnings from those initial efforts led to the architecture that sits at the core of the AtScale Intelligence Platform. Because AtScale has been built from the ground up to deliver business-friendly insights from the vast amounts of information in data lakes, AtScale has experienced tremendous success and adoption in enterprises ranging from financial services, to retail to digital media. With the release of AtScale 6.5, we’ve continued to build on and expand AtScale’s ability to uniquely deliver on the promise of Data Lake Intelligence. If this sounds like something you might be interested in knowing more about… keep reading!

While it may be tempting to focus our efforts only on self-service BI in terms of security and access control mechanisms, it is important to also place emphasis on economies to achieve success.  When an enterprise develops a self-service BI environment, it undoubtedly means that their IT team adopts the role of a service provider. Data and services become available to internal business users for a price. What are these hidden costs?

In the world of Business Intelligence and Big Data there continue to be a number of exciting innovations as new and improved options for processing large data sets appear on the market.  You may be familiar with AtScale’s BI-on-Hadoop Benchmarks - where we focus on evaluating the top SQL-on-Hadoop engines and their fitness to support traditional BI-style queries.  As we continue to work with customers who are navigating their journey to BI on Big Data, we are increasingly getting questions about the emerging cloud-based data processing engines.

 In this blog post, we will take a deeper look at BigQuery from Google, and how it stacks up in the BI-on-Big Data ecosystem.

I’ve asked it before and I’ll ask it again. Wouldn’t it be great if you could easily analyze ALL your data from a Excel single file? We all know this isn’t feasible; especially when dealing with big data and complex business analytics needs.

In working at the intersection of Big Data and traditional Business Intelligence, the AtScale team has encountered a number of complex business analytics use cases that are difficult, if not near-impossible, to solve using typical table-based data models and SQL. Today, I’m going to share why and how complex analysis, like for multi-level metrics, is no longer as ‘difficult’ nor ‘near-impossible’ as it once was.

Wouldn’t it be great if you could load all of your data from a single file into an Excel pivot table for easy analysis? 

Unfortunately, this approach isn’t usually viable when dealing with complex business analytics and big data.  Take for example a typical use case found inthe world of healthcare insurance.  A large insurance provider has 10s of millions of members, and processes 100s of millions of claims a year.  As flexible as Excel is, we all know it won’t handle this volume or velocity of data. 

As a result, more and more enterprises store  large data sets in big data platforms like Hadoop.  And while Hadoop provides a low-cost and performant approach to store and process this information, there is still the challenge of supporting the many types of analytics required on claims and member data sets.  But why? Why and how, with all of the advances in technology, can a simple calculation cause so much complexity?

Digital transformation is a broad term that has various meanings by application, but in general, it means that more and more of what organizations, people, governments do is happening in computers, mobile devices and networks. As a result, the way things are done is changing, especially in the way things are connected. So in this new world of data flying everywhere, being generated and consumed, where does one stop for a second to take a look at what’s going on?