LogoLogo
API DocsDeveloper PortalSystem StatusTry for Free
  • Quickstart Guide
    • Introduction
    • Get started as an Account Owner
    • Get started as a User
    • Glossary
    • FAQs
  • Manage Users
    • Types of Users
    • Add and Delete Users
    • Import Users
    • User Permissions - Access Controls
    • Manage Your Profile
    • Notification Rules
    • On-Call Reminder Rules
    • Change Account Owner
  • Manage Teams
    • Understanding Teams
    • Role Based Access Control
    • Owner Based Access Control
    • Create and Delete Teams
    • Add and Remove Team Members
    • Squads
    • Stakeholder Groups
  • Services
    • Adding a Service
    • Service Overview
    • Service Graph
    • Maintenance Mode
    • Alert Deduplication Rules
      • Alert Deduplication Rules
      • Incident Status Based Deduplication
      • Service Dependency Based Deduplication
      • Key Based Deduplication
    • Event Tagging
    • Alert Routing
    • Alert Suppression
    • Custom Content Templates
    • Intelligent Alert Grouping (IAG)
    • Auto Pause Transient Alerts (APTA)
    • Delayed Notifications
  • Schedules
    • Schedules (New)
      • Adding a Schedule
      • Schedules Overview
      • Who is On-Call?
      • My On-Call Shifts
      • Overrides
      • Videos: How to set up common use cases?
  • Escalation Policies
    • Create and Manage Escalation Policy
    • Round Robin & Advanced Escalations
    • Reassign an Incident
  • Notifications
    • Understanding Incident Notifications
  • Dashboards
    • Incident Management Dashboard
    • Dashboard Metrics
    • Take Bulk Actions
    • Squadcast Search
  • Incident List
    • Incident List View
    • Incident Priorities
    • Filter Incidents
    • Save Filter View
    • Merge Incidents
    • Snooze Incidents
  • Incidents Page
    • Incidents Details
    • Incident Activity Timeline
    • Communication Channels
    • Create Incident Manually
    • Incident Notes
    • Incident Watchers
    • Past Incidents
    • Additional Responders
    • Incident Summaries
    • Incident Suggestions
  • Runbooks
    • Runbooks
  • Postmortems
    • Postmortem Templates
    • Create Postmortems
    • Accessing Postmortem
  • Status Page
    • Status Page
    • Status Page Overview
    • Components and Groups
    • Issues
    • Subscribers
    • Maintenance
  • SLO Tracker
    • SLO Basics
    • Configure and Monitor your SLOs
  • Webforms
    • Webforms
  • Global Event Rulesets
    • Global Event Rulesets
  • Workflows
    • Workflows
    • Workflows Overview
    • Actions
  • Live Call Routing
    • Live Call Routing
  • Analytics
    • Analytics (New)
    • Organization Level Analytics
    • On Call Hours Per User
    • Weekly Reports
  • Integrations
    • Incident Webhook (Incident Webhook/API)
    • Outgoing Webhooks
    • ServiceNow Extension
    • Extensions
      • Jira Cloud Integration
      • Jira DC (Data Center)
      • CircleCI
      • Google Chat
      • Freshdesk
      • Freshservice
      • Asana
      • ClickUp
      • Trello
      • Zendesk
      • Hubspot
    • Alert Source Integrations (Native)
      • Admin Labs
      • Airbrake
      • Amazon EventBridge
      • Amazon GuardDuty
      • Amazon Opensearch
      • APImetrics
      • AppDynamics
      • AppSignal
      • Auvik
      • AWS CloudTrail Logs
      • AWS CloudTrail via CloudWatch
      • Amazon Cloudwatch (AWS) Integration
      • AWS CloudWatch Event Rules
      • AWS Elastic Beanstalk via CloudWatch
      • Amazon RDS (AWS)
      • Amazon SNS (AWS)
      • Azure Monitor
      • Better Uptime
      • Bitbucket
      • Bitrix 24
      • Blue Matador
      • Bugsnag
      • Buildkite
      • Checkly
      • Checkmk
      • CircleCI Integration
      • Cisco DNAC
      • Cisco Meraki
      • ClickUp Integration
      • CloudAMQP
      • Cloudflare
      • Conviva
      • CopperEgg
      • Coralogix
      • Cronitor
      • Crowdstrike Falcon
      • Datadog
      • Databricks
      • Dead Man's Snitch
      • Domotz
      • Dotcom Monitor
      • Dynatrace
      • ElastAlert
      • Elastic
      • Elecard Boro
      • Email Integration
      • Endtest
      • Errorception
      • Freshdesk Integration
      • Freshping
      • Freshservice
      • Ghost Inspector
      • GitHub Integration
      • GitLab
      • Grafana 8
      • Grafana
      • Graylog v4
      • Graylog
      • HaloPSA
      • Healthchecks
      • Heroku
      • HetrixTools
      • Honeybadger
      • Honeycomb
      • Humio
      • Hund
      • Hydrozen
      • Hyperping
      • Icinga2
      • InsightOps (LogEntries)
      • Instana
      • Intercom
      • Jenkins Integration
      • Jira Cloud Alert Source
      • Jira Server Alert Source
      • Kapacitor
      • Kentik
      • Komodor
      • Kibana
      • LibreNMS
      • Linear
      • Loggly
      • Logstash
      • Logz.io
      • ManageEngine Application Manager
      • ManageEngine Opmanager
      • Mezmo (formerly LogDNA)
      • MongoDB Atlas / Cloud Manager
      • Nagios
      • New Relic
      • Nixstats
      • NodePing
      • Observium
      • Oh Dear
      • Oracle Cloud Infrastructure
      • OSNexus QuantaStor
      • OverOps
      • Papertrail
      • Pingdom
      • Plesk 360
      • Postman
      • Postmark
      • Powercode
      • Progress WhatsUp Gold
      • Prometheus
      • PRTG Network Monitor
      • Rapid7 InsightIDR
      • RapidSpike
      • Redash
      • Redgate SQL Monitor
      • Rollbar
      • Rundeck
      • Runscope
      • Salesforce Cloud
      • Scout APM
      • Sematext
      • Sensu Go
      • Sensu
      • Sentry.io
      • Server Density
      • ServerGuard24
      • ServiceNow Integration
      • Shortcut (Clubhouse)
      • SignalFx
      • SigNoz
      • Site24x7
      • Slack
      • SolarWinds AppOptics
      • SolarWinds Observability SaaS (SWO)
      • SolarWinds Observability Self Hosted
      • Sonar
      • Splunk
      • Sqreen
      • Stackdriver
      • Stackify Retrace
      • StatHat
      • StatusCake
      • ServiceDesk Plus OD
      • Sumo Logic
      • Sysdig Monitor
      • Threat Stack
      • Trello
      • Twilio
      • Uptime
      • Uptime Robot
      • Uptrends
      • Wavefront
      • Zabbix 5.0
      • Zabbix 6.2
      • Zabbix
      • Zendesk Integration
      • Zoho Desk
      • Zoho Desk via Zoho Flow
      • LogicMonitor
  • ChatOps
    • Google Chat
    • Microsoft Teams
    • Slack for Incident Management
      • Using the Integration
  • Single Sign-On (SSO)
    • AWS SSO
    • Azure Active Directory SSO
    • Google SSO
    • Microsoft ADFS SSO
    • Okta SSO Integration
    • SAML 2.0 based SSO
  • Mobile App
    • Using the Mobile App
  • Terraform & API Documentation
    • Terraform Provider
    • Public API - Refresh Token
    • API Documentation
    • Getting Started with Squadcast GraphQL
      • Schedules
        • Create Schedule
        • Update Schedule
        • Delete Schedule
        • Pause Schedule
        • Get Schedules
        • Get Schedule by ID
        • Resume Schedule
        • Clone Schedule
        • Get Gaps
      • Rotations
        • Create Rotation
        • Update Rotation
        • Delete Rotation
        • Get Rotation by ID
        • Get Rotation Events by ID
      • Overrides
        • Create Override
        • Update Override
        • Delete Override
        • Get Override by ID
      • Calendar URLs
      • Who is On-Call
    • Developer Portal
    • Incident Rate Limiting
  • Managing your Squadcast Account
    • Audit Logs
    • Organizations
    • Billing FAQs
    • Deactivate your Squadcast Account
    • Delete your Squadcast Account
Powered by GitBook
On this page
  • How it works
  • Prerequisites
  • Create Key Based Deduplication
  • Delete Key Based Deduplication

Was this helpful?

  1. Services
  2. Alert Deduplication Rules

Key Based Deduplication

Define dedup keys using customizable templates for configured alert sources. Auto-group similar incidents for efficient incident management and grouping of duplicates.

PreviousService Dependency Based DeduplicationNextEvent Tagging

Last updated 1 year ago

Was this helpful?

Key Based Deduplication is an efficient way to avoid duplicate entries when processing incoming Events alongside existing Incidents. It works by generating a Deduplication Key using a user-defined template specific to events from an Alert Source. This key helps identify and group duplicates.

Note: This feature is available across all pricing plans. For all older accounts (created before Aug 21, 2023) and Enterprise accounts:

The introduction of this feature does not affect any existing Deduplication Rules you may have set up for your Services. Your current rules will remain unchanged and continue to function as expected, providing you with the same level of control and efficiency in managing your alerts.

If you switch to Key-Based Deduplication by setting it to active, Deduplication Rules will become inactive.

Important: Automation rule CRUD operations have a 5-minute caching delay before changes take effect.

How it works

  1. To use the Key Based Deduplication feature, the user must opt-in to a specific Service.

  2. They can define a template to generate dedupe keys for each alert source within the Service. The user also specifies a duration (x) for the Deduplication Window.

  3. For an incoming Event, the Deduplication Key is calculated based on the defined template.

  4. This Key is then compared (using equality) against any previous Incidents within the Deduplication Window (last 5 minutes or the specified duration).

  5. If an Incident with the same Deduplication Key is found, the current Event is deduplicated against that Incident.

  6. However, if no matching Incident is found, a new Incident is created.

  7. Once the Deduplication Duration (x) elapses, the system recalculates the Deduplication Key using the defined template. This process continues for ongoing Deduplication.

Prerequisites

The User Role associated with the user in the Team must have required permissions to manage Services (ability to manage Key Based Deduplication).

Create Key Based Deduplication

To add Key Based Deduplication:

  1. Navigate to Services -> Service Overview -> Select or search for your desired Service.

  2. On the extreme right, expand the accordion -> In the Automation section, View All.

  3. In the Key Based Deduplication section, Add Dedup Key.

  4. Select an alert source to begin creating Deduplication Keys for your incoming Events.

  5. On the right, you can view the payload of the latest alert for the chosen Alert Source.

Note: You can configure one Dedup Key per Alert Source.

Here are some illustrative examples that demonstrate how to define a template for generating Deduplication keys.
  1. Define the Deduplication Time, in min(s) or hour(s).

Regular Expression-Based Extraction in Go Template

Our system supports regular expression-based extraction using regex rules. It allows for multiple name captures but only retains the first match for a specific named group. Additionally, when the passed expression is not valid, the function returns empty match results.

{{- with $matches := ("(?m)^Container: (?P<container>.*)|Alertname: (?P<alertname>.*)|Summary: (?P<summary>.*)$" | reExtract .description) -}}
    {{$matches.container}}-{{$matches.alertname}}
{{- end -}}

This code snippet is a template written in Go's text templating language, used to parse alert descriptions. It extracts specific details like container name, alert name, and summary from the descriptions using regular expressions and presents them in a concise format.

Here's a breakdown:

  1. Regular Expression ((?m)^Container: (?P<container>.*)|Alertname: (?P<alertname>.*)|Summary: (?P<summary>.*)$):

    • (?m) enables multiline matching.

    • ^Container: (?P<container>.*) captures everything after "Container: " into a named group "container".

    • |Alertname: (?P<alertname>.*) captures everything after "Alertname: " into a named group "alertname".

    • |Summary: (?P<summary>.*)$ captures everything after "Summary: " into a named group "summary" and ensures it matches the end of the line ($).

  1. Template Processing ({{$matches := ...}}):

    • The code defines a variable $matches using the regular expression to extract details from the description string .description.

  1. Output Formatting ({{$matches.container}}-{{$matches.alertname}})

    • The template accesses the captured container name ($matches.container) and alert name ($matches.alertname) from the $matches variable.

    • It combines them with a hyphen (-) for a one-line description format.

Note: The maximum time limit is 48 hours.

  1. Click Save.

Delete Key Based Deduplication

To delete a Key Based Deduplication config,

  1. Click on the Key Based Deduplication Rule for a selected Service.

  2. On the right-hand side, click More Options -> Delete Key

  3. In the Configuration page, click Delete. A confirmation modal will appear.

  4. Click Delete anyway to confirm.

Note: Kindly note that the Deduplication will cease once the Key is deleted.

The user needs to define a template to generate Deduplication Keys using the variables from the payload referenced on the right, for a particular alert source of a Service. For additional information on how to write templates, please refer to A Deduplication Key is calculated for the incoming event based on the template defined by the user.

Have any questions? .

Go's standard library.
Ask the community
Image. Flow Diagram for Key Based Deduplication
Flow Diagram for Key Based Deduplication