Freshness Checks
Monitor data freshness by tracking when tables were last updated.
Freshness checks are the core monitoring feature in FreshGuard. They alert you when a table hasn’t received new data within a specified time window. Free+
How It Works
A freshness check runs a query against your table to find the most recent timestamp value:
SELECT MAX(updated_at) FROM your_tableFreshGuard compares this timestamp to the current time. If the difference exceeds your configured tolerance, the check fails and an alert is triggered.
Configuration
When creating a freshness rule, you configure:
Timestamp Column
The column FreshGuard uses to determine when the table was last updated. This should be a TIMESTAMP, TIMESTAMPTZ, or DATETIME column that reflects when rows were inserted or modified.
Common column names: updated_at, created_at, loaded_at, _etl_loaded_at.
Tip
Choose a column that updates reliably with every data load. If your ETL only inserts new rows, created_at works well. If it also updates existing rows, use updated_at.
Tolerance
The maximum acceptable age of the most recent row. If the newest row is older than this tolerance, the check fails.
Examples:
- 30 minutes — for near-real-time pipelines.
- 2 hours — for hourly batch loads.
- 24 hours — for daily pipelines.
Set the tolerance to be comfortably longer than your pipeline’s normal schedule. If your pipeline runs every hour, a tolerance of 2 hours gives you a buffer before alerting.
Example Scenarios
Hourly ETL Pipeline
Your pipeline loads new rows into orders every hour. You set:
- Timestamp column:
loaded_at - Tolerance: 2 hours
If the pipeline fails and no new rows arrive for 2 hours, the freshness check fires an alert.
Daily Batch Load
A warehouse table daily_revenue is updated once per day around 6 AM. You set:
- Timestamp column:
report_date - Tolerance: 26 hours
This gives a 2-hour buffer past the expected 24-hour cycle, so a slight delay doesn’t trigger a false alarm.
Event Stream
A user_events table receives events continuously. You set:
- Timestamp column:
event_time - Tolerance: 15 minutes
If events stop flowing for 15 minutes, you’re alerted immediately.
Recovery Notifications
When a freshness check transitions from failing back to passing (i.e. new data arrives), FreshGuard sends a recovery notification to the same alert destinations. This lets your team know the issue has resolved without needing to manually check the dashboard.