Configure and Monitor your SLOs

With Squadcast, you can define and monitor Service Level Objects for your services.

Note:

This feature is available as part of the Product Trial and Premium and Enterprise Plan.

Before configuring your SLOs, we recommend you read our SLO Basics documentation.

Before you begin to define your SLO, you should have an expectation of what percent of your SLI (availability, latency, etc.) is needed to pass the SLO. For example, you may want your service to be available 99.99% of the time to pass the SLO. In this case, 99.99% is the Target SLO and “availability” is your SLI.

Creating New SLO

Navigate to SLO from the left sidebar. To create a new SLO, click on +Create New SLO button on the top right.

Define SLO

You begin by defining your SLO.

1. Enter the SLO Name of your choice (this needs to be unique across SLOs)

2. Enter the SLO Description detailing out specifics of the SLO

3. Enter Tags (key-value pairs) specifying information such as the Owner, Environment and Type of SLO

You can additionally add your own tags by clicking the +Add Tag button.

Once done, click on Next to Configure SLO.

Configure SLO

1. Under the Services Associated with this SLO tab, you can select multiple Services to link it to the SLO. Only incidents from these linked Services can then be mapped to the SLO

2. Enter the SLIs that affect the SLO. There could be one or more SLIs - like availability, response time, etc - that map to this SLO

3. Enter the percentage or ratio under Target SLO in %. This sets the target percentage to define compliance

4. Error Budget is auto-calculated based on the values entered for target SLO and duration. It is calculated in minutes and cannot be edited

5. Enter the Duration for this SLO, by choosing between Rolling Period or Fixed Duration (Calendar Duration)

• Under Rolling, you can select the period in days. This option is used when you want the SLO calculated continuously for a defined number of days (for example, continuously over a 7-day period). This can be a maximum of 90 days.

• Under Fixed Duration, you can select the start and end dates from the drop-down. This option is used when the SLO has to be calculated over a fixed duration - for example, over one quarter at a time. The fixed duration can be a maximum of one year. Once done, click on Next to configure Error Budget Policy.

Important: When an SLO reaches the end of its configured time duration, it transitions into an inactive state. Once inactive, the SLO becomes uneditable, signaling that it has fulfilled its purpose and achieved the intended goal for which it was established.

Error Budget Policy

The Error Budget Policy defines the conditions based on which to notify one or more Users or Squads or when incidents have to be created when a condition is breached.

Choose the conditions you want to be alerted for, out of the following options:

• Alert when there is a breach of allocated Error Budget

• Alert when there is an Unhealthy Burn Rate. An unhealthy burn rate is determined when the error budget is burning faster than what’s expected. For example, for an SLO of 99.99% over the course of a year, the error budget works out to be about 52 mins 35 seconds - or approximately about 4 mins 30 sec per month. If the error budget is being burnt faster than than, then its considered unhealthy

• Alert when the number of False Positives exceeds the set limit

• Alert when the Error Budget decreases below the set limit

Choose the mode of delivery of the alerts

• Email Alerts, wherein an email notification is sent to the Users or Squads you specify

• Incident Alerts, wherein an alert is created for the specified Service

Once done, click on Create, and your SLO is created!

Monitor Your SLOs

Once created, you can access all your SLOs for the current Team from the SLO dashboard.

The SLO list view shows information for each SLO, including:

Field	Description
Target SLO (%)	Shows the percentage or ratio to target, which is the target value for compliance
Current SLO (%)	Shows the current historical compliance with the SLO
SLO Health	Indicates the health of the SLO, either as Healthy or Needs Attention
Service	Shows the services related to the SLO
Status	Indicates whether the SLO is Active or Inactive
Incidents Reported	Indicates the # of incidents reported + the false positives for this SLO
Time Window	Indicates the type of time window you have configured, either Rolling or Fixed
Duration (Days)	Indicates the duration (in days) for which the slo is configured
Updated On	Indicates the latest date of update
Tags	Indicates the tags associated with the SLO

SLO Detail Page

To view the details of a particular SLO, select the SLO from the SLO list in the SLO dashboard.

The SLO details view shows information for each SLO, including:

Field	Description
Target SLO (%)	Indicates the percentage value set to target performance compliance.
Total Error Budget (mins)	Indicates the entire time period for which a system can fail without violating the SLO.
Time Window	Indicates the extent of time for which the SLO has been set. It can be on a rolling (or continuous) basis or on a fixed basis (eg, once a quarter).
Duration	Indicates the duration (in days) for which the SLO is configured.

The fields hereon are time-range sensitive.

Field	Description
Current SLO (%)	Indicates the current historical compliance with the SLO in the specified time range.
Error Budget Consumed	Indicates the Error Budget consumed in the specified time range.
MTTA (mins)	Indicates the mean time taken to acknowledge the SLO-violating incidents, for the specified time range.
MTTR (mins)	Indicates the mean time taken to resolve the SLO-violating incidents, for the specified time range.
Error Budget Consumed by SLIs	Indicates the consumption of error budget across different SLIs, including the number of incidents affecting each of the SLIs. Expand the accordion to view a further breakdown of the error budget consumed by each SLI, and a list of services associated with the SLO.

Marking an Incident as False Positive

This is useful if an incident was previously marked as one that affects an SLO and has been subsequently determined that it does not. This acts like an “undo” button.

Through the SLO Details page, checkmark the incident(s) -> Click on Mark as False Positive button

Note:

If you marked an incident as False Positive by mistake, you can undo this. Navigate to False Positives tab on SLO Details Page -> Check the incident -> Click on SLO Affected

Deleting an SLO

To delete an SLO, click on the icon on the right of the SLO from the SLO list, and click on the Delete icon, as shown in the image below.

FAQs

Please refer to the Frequently Asked Questions below that might help you fix any issues/answer your queries.

1. Can I delete services associated with an SLO?

Yes, you can delete a service associated with an SLO, the SLO and its Incidents will still be intact.

2. How is the Error Budget calculated?

An error budget is 1 minus the SLO of the service. A 99.9% SLO service has a 0.1% error budget. If our service receives 1,000,000 requests in four weeks, a 99.9% availability SLO gives us a budget of 1,000 errors over that period.

3. How is the Burn Rate calculated?

• First, we calculate the error budget allocated for a day. (for eg: if slo is 99.99% for a year, you get a 55.5min/365 error budget for a day)

• Then, we fetch the total error budget spent till today’s date

• Subsequently, we see how many days its been since the slo started and later check if the user has consumed more error budget than they were supposed to, to calculate the burn rate

4. What determines a Healthy or Unhealthy SLO?

Healthy or unhealthy is based on how rapidly the error budget is getting depleted. Slo burn rate indicates how fast your error budget is getting consumed relative to your SLO’s target length.

For example, if have a 99.9% target for a month, then you will get 43.12 min of downtime, Which means you can burn 1.43 Min(43.12/30) of error budget every day. If you burn a total of 30 min of error budget within the first 10 days of your SLO duration then it will turn to Needs Attention (unhealthy).

So the SLO is Healthy today because it’s consumed less error budget than allocated for the days it's been since the SLO started.

5. Can I automatically associate incidents with an SLO?

We’re working on something that can help you do this in the near future.

6. Are there any rate limits for SLOs?

There exists a rate limit on the number of SLO promotions that can take place (manually or via Workflows). Not more than 50 incidents per hour can be promoted to SLO violations in a Team.

7. How many incidents are considered for error budget calculation when they are overlapping in a given time window?

The current incident and 49 preceding incidents (a total of 50 incidents) are considered for calculating the error budget consumed in case there is an overlapping observed by the system (i.e., calculate the cumulative error budget consumed for multiple "open" SLO-violating incidents).

Have any questions? Ask the community.