Introduction
Foundational's observability features help you monitor your data quality and ensure the reliability of your data pipelines. A key part of effective monitoring is setting appropriate thresholds for your tests. Custom thresholds allow you to tailor alerts to your specific data monitoring needs, reducing noise and ensuring you're notified of the issues that truly matter. This page will guide you through how to configure custom thresholds for observability metrics within Foundational.
Why Configure Custom Thresholds?
By default, monitoring systems often use generic thresholds that may not be suitable for your specific data. Custom thresholds enable you to:
Reduce Alert Fatigue: Avoid being overwhelmed by alerts for minor fluctuations that are normal in your data.
Focus on Critical Issues: Ensure you are promptly notified when metrics deviate significantly from expected values, indicating a potential problem.
Align with Business Needs: Set thresholds based on the specific requirements and tolerances of your data-driven applications and decisions.
Tailor Monitoring Granularity: Define different thresholds for different columns or tables, allowing for more precise control over your monitoring.
How to Configure Custom Thresholds
Foundational allows you to configure custom thresholds for various observability metrics, such as null counts, zero counts, and unique values. Here's how:
Navigate to Table Monitoring Settings: From the information panel for any specific table, click on the settings button
Choose a Metric: Select the metric for which you want to configure thresholds (e.g., Null Count, Zero Count, Distinct Values Count).
Select Columns: Choose the specific columns you want to apply the custom thresholds to.
Set Minimum and Maximum Thresholds: For each selected column, define the minimum and maximum acceptable values or percentages for the chosen metric.
Foundational allows you to set minimum and maximum thresholds (0-100) for the selected metric rate. Alerts will trigger when the rate falls outside this range.
Save Your Settings: Save the configured thresholds.