Introduction to Azure Data Factory Alerts

Enabling Azure Data Factory alerts can allow organizations to closely monitor their systems so they can appropriately respond to any problems that may arise. We’ll discuss the different kinds of alerts and walk you through how we implemented them for one of our clients.

Operations require steady support to run smoothly and communicate both success and failure. Configuring alerts to monitor the system and parameters help organizations to respond and take the required actions based on critical events. Azure provides alerting capabilities for all their resources. In this article, we will be discussing how to enable alerts in Azure Data Factory using a quick and easy interface.

Main Types of Alerts

Metric Alerts

Metric alert rules are specified with a target criteria for a metric within the resource to be monitored. Based on the condition, notifications are sent to an action group when the alert rule fires. Here are a few more attributes of metric alerts:

azure data factory alerts - azure logo

Smartbridge is a Microsoft Azure Partner

  • Monitoring and alerting happens for the current state of time.

  • Mostly based on performance, usage, and status (Success/Failure/Cancelled) driven metrics.

  • Can be checked from the Metrics section in the Azure portal.

Log Analytics Alerts

These alerts are triggered based on log searches that are automatically run at periodic internals. Advanced alerting for non-fatal errors, warnings, and business logic errors can be created in Azure Monitor and Log Analytics. A few more attributes of log analytics alerts are:

  • Provides long term storage of logs (default ADF logging period is 45 days) which can enable more sophisticated analytics. For example, trending analysis using historical comparisons of pipeline performance and activities.

  • Ability to merge various metrics and observe the relationship between them.

  • Able to analyze all types of logs including any custom logs written for a specific business case.

  • Can be checked from the Log Analytics workspace in the Azure portal.

azure data factory alerts

Azure Data Factory Alerts

At Smartbridge, we implemented native ADF Alerts as part of an Azure implementation for a client. We created two alert rules: one to monitor pipeline failures and the other for trigger failures. Below is an overview of the steps we took and how the alerts work.

Start by navigating to the Monitor tile in Data Factory and click “New Alert Rule”. Below are the parameters used to create an alert rule to monitor pipeline failures:

  • Metric: Failed Pipeline Run

  • Severity: 0 to 4

  • Dimension: Select the Pipelines and Failure Type to be associated

  • Alert Logic: Greater than or Equal to 1(threshold count) based on Count aggregation

  • Evaluation Period: Over the last 1 min

  • Frequency: Every 1 min

  • Notification: Configure Email/SMS/Voice and use an action group for notifications to be sent

  • Check Enable Rule Upon Creation

From the time of the rule being enabled, the system looks for any pipeline failure over the past 60 seconds then continues to check every minute. During this period if the count of pipeline failure is exceeding or equal to 1, the alert rule fires. Depending on the mode of notification configured, an Email/SMS/Voice call is sent to the action group notifying that the alert was activated.

If there is a consecutive pipeline failure during the evaluation window, the alert condition remains fired. If the pipeline execution is successful, the value becomes less than the threshold count. This condition should be satisfied for three healthy executions which will move the alert to the resolved condition. A notification will then be sent for Alert Deactivation.

The alert condition must change to resolved before further alerts can be triggered for that specified metric. The alert condition is controlled by the system whereas the state of the alert can be modified by the user.

Looking for more on Azure and Systems Modernization?

Explore more insights and expertise at smartbridge.com/modernization

There’s more to explore at Smartbridge.com!

Sign up to be notified when we publish articles, news, videos and more!