Often times, customers approach me with questions around AtScale’s ability to integrate into the customer’s operational stack. Today, I want to highlight a component of AtScale’s Development Extensions called Webhooks.
A webhook (also called a web callback or HTTP push API) is a way for an application to provide other applications with real-time information. A webhook delivers data to other applications as certain events occur, meaning you get data immediately as opposed to a REST API which you would need to poll for data very frequently in order to get it close to real-time. This makes webhooks much more efficient for both provider and consumer.
Consuming Webhooks in AtScale
Webhooks are sometimes referred to as “reverse APIs" because they give you what amounts to an API spec, and you must design an API for the webhook to use. The webhook will make an HTTP request to your app (typically a POST), and you will then be responsible for interpreting it.
AtScale provides two categories of webhooks that you can configure to notify your applications of events that happen in your AtScale system.
- The first category is comprised of webhooks for events that take place in the Design Center. You can configure webhooks for events that involve organizations and projects.
- The second category is comprised of webhooks for events that take place in the AtScale engine.
When it comes to configurating webhooks, there are two paths you can take. If you don’t see either one, please contact your AtScale Administrator.
To configure Webhooks for Design Center events, you need to do that at a particular project level as follows:
1. To navigate to the Webhooks page for a project, click Settings -> Webhooks at the top of the project's Overview page.
2. Provide the URL for the endpoint (ie. http://sandbox.atscale.com:10555/publish) and pick the "EVENT" that this Webhook will trigger on (In this case ‘Project published’).
To configure webhook for Engine, open Manage -> Engine Overview and select Webhooks from Settings tab
As you can see, in both cases you specify an event within AtScale that you want to subscribe to and act upon accordingly with another application you might have in your stack or perhaps just a small web service. Given that information, what can we really do with Webhooks?
What Are Some of The Uses of Webhooks?
DESIGN CENTER WEBHOOKS
As mentioned earlier, Design Center Webhooks are triggers on events that occur within Design Center application, where you create projects, cubes, and other items. Here is a list of events currently supported for Design Center for Webhooks:
- Snapshot created
- Snapshot restored
- Project published
- Project promoted
- Project imported
- Project deleted
- Project delete and unpublishing
One of the most common uses customers ask me about is the ability to check projects into source control repository like Git. The concept of projects within AtScale encapsulates metadata regarding all the elements of the cubes (Facts, Dimensions, Calculations) and their security. As such, when looking to utilize a source code repository, the Project's representation as XML document is what you want to capture. This can be easily triggered by ‘Snapshot’ or ‘Publish’ events as they occur when users click ‘Take Snapshot’ or ‘Publish’ buttons on the UI. In each case, AtScale will check for any webhooks that are registered to act on these events and fire off a payload to these web services with XML representing the project currently being acted on. From here on, you have an option to either take that XML and commit to Git or other repository source control or perhaps do some other analysis, even analyzing contents of the document to populate your common metadata dictionary with information about the published cube.
ENGINE WEBHOOKS
ENGINE webhooks are triggered on events specific to operating engine components such as aggregate management.
Here is the list of currently supported events:
- Schema Unpublish
- Schema Publish
- Aggregate Batch Success/Failure
- Environment Created
- Environment Deleted
- Aggregate Batch Started
- Environment Updated
One of the most common uses of Engine Webhooks revolves around notifications regarding aggregate refreshes:
1. Aggregate Batch Started - within AtScale, aggregate management is done on batches of aggregates pertaining to a cube. Therefore when data refresh is triggered for a particular published cube, the back-end kicks off a refresh process on a batch of aggregates. As such, the notification of this event triggers a webhook, which can notify an external system (healthchecks, ETL, etc.) that data refresh started.
2. Aggregate Batch Success/Failure - once all the aggregates are built in a batch (or failed), as talked about in section (a), the Engine calls this webhook, which at this point can notify an external system you may have setup internally to notify an administrator or business users that data is ready to be used in new aggregates.
In conclusion, Webhooks provide a useful mechanism for integration and operational support of AtScale. While the webhooks listed above are provided in the current version, watch this space for updates as more webhooks become available and new use cases for these webhooks are provided.
We invite you to learn more about AtScale today!