PagerDuty
Our PagerDuty integration enables various functions to improve the efficiency of your incident management. You can:
- Automatically create FireHydrant incidents from PagerDuty alerts
- Notify different channels based on incoming alert content
- Page out to teams and service in PagerDuty from FireHydrant
- Automatically pull in on-call responders from PagerDuty's escalation policies to incident channels
- Importing services from PagerDuty to FireHydrant's Service Catalog
- ...and more!
Prerequisites
- You will need Owner permissions to configure integrations in FireHydrant
- You'll need access to PagerDuty's Webhooks and Integrations settings (usually admin).
Installation
First, you need a PagerDuty API key.
- Go to your PagerDuty dashboard and navigate to the page where you can set up API keys. (This will be something like Integrations > API Access Keys ).
API Access Keys menu item in PagerDuty
- On the API Access page, click Create New API Key and provide a description for the API key.
- Click Create Key.
- Copy the API Key that PagerDuty generates. Then go to FireHydrant's integrations page.
- Click on the PagerDuty integration card, then click Setup PagerDuty.
- Enter the PagerDuty API key you copied into the API Token field.
- For Domain, we default to showing
https://api.pagerduty.com, which is the domain for PagerDuty's US service region. If you are a company based out of the EU and using PagerDuty's EU service region, then you will need to modify that value tohttps://api.eu.pagerduty.com. For more information, visit PagerDuty's documentation. - Click Authorize Application.
Setup Outgoing Webhook
Note
If you plan to import or link services from PagerDuty, do not configure this outgoing webhook and skip this section. When you link or import PagerDuty Services to FireHydrant services, we automatically create service-scoped webhooks in PagerDuty.
After configuring the connection, if you do not plan to import or link services from PagerDuty, you should configure an outgoing webhook from PagerDuty so FireHydrant can both sync incident states and receive webhooks on alerts.
Webhook URL generated after installing PagerDuty
- In the integration settings page for PagerDuty on FireHydrant, copy the webhook address provided.
- In PagerDuty, Head to Integrations > Generic Webhooks and click "+ New Webhook."
- In Webhook URL, paste in the value you copied from FireHydrant.
- For Scope Type choose "Account".
- For Event Subscription, you can uncheck all the
service.*events, leaving only the incident updates. - Click "Add Webhook".
To test this, go ahead and create an incident in FireHydrant. With default rules in FireHydrant, this will log an alert processing message.
Webhook logged after PagerDuty alert created
PagerDuty Alert Routing
Configuring alert routes in PagerDuty
Once your PagerDuty instance is configured, you can set up Alert Routes to take action on your alerts based on the data included in the alert. You can automatically open new incidents, send alerts to any Slack channel, log an alert in FireHydrant, or simply ignore it. To learn more, visit Alert Routing documentation. The rest of this page details the parameter mappings for PagerDuty.
Parameter Mappings
Here is the table of routable parameters on FireHydrant and the corresponding key/value from the inbound PagerDuty webhook(s). The $ refers to the webhook body content as a JSON object.
An explanation of PagerDuty's Webhook content can be found in their docs here.
| Parameter Name | PagerDuty Webhook Body | Notes |
|---|---|---|
| PagerDuty: Event Type | $.event.event_type | The type of event sent by PagerDuty |
| PagerDuty: Occurred At | $.event.occurred_at | The datetime of when the event occurred according to PagerDuty |
| PagerDuty: Alert ID | $.event.data.id | The ID of the alert in PagerDuty |
| PagerDuty: Incident ID | $.event.data.incident.id | The ID of the incident in PagerDuty |
| PagerDuty: Incident Summary | $.event.data.title | Short summary of the PagerDuty incident |
| PagerDuty: Data Type | $.event.data.type | The type of data, e.g., 'incident' or 'incident_note', present in this message |
| PagerDuty: Alert Web URL | $.event.data.html_url | URL to alert page in PagerDuty |
| PagerDuty: Incident Web URL | $.event.data.incident.html_url | URL to incident page in PagerDuty |
| PagerDuty: Incident Priority | $.event.data.priority.summary | Incident priority in PagerDuty. Must enable priorities in PagerDuty to use this. |
| PagerDuty: Incident Status | $.event.data.status | The status of the incident in PagerDuty. Is one of triggered, acknowledged, and resolved. |
| PagerDuty: Incident Note Content | $.event.data.content | The content of a note posted to an incident in PagerDuty. Only shows up for incident.annotated events. |
| PagerDuty: Incident Title | $.event.data.title | Name of the incident. Essentially the same as Incident Summary. |
The following table maps our overall Alert Routing mapping object - these parameters are standard across all Alerting/Monitoring integrations.
| Parameter Name | PagerDuty Webhook Body | Notes |
|---|---|---|
| Alert Summary | $.event.data.title | The same as PagerDuty: Incident Summary above. |
| Alert Description | N/A | Maps to nothing for PagerDuty. PagerDuty has a Description field in their alerts, but oddly they do not surface it in the webhook, so FireHydrant has no way of accessing it. |
| Alert Priority | $.event.data.priority.summary | The same as PagerDuty: Incident Priority above. |
| Alert Status | $.event.data.status | Almost the same as PagerDuty: Incident Status above. However, FireHydrant expects opened instead of triggered to align with internal terminology. |
| Alert Associated Infrastructure | N/A | The associated Service/Functionality in FireHydrant. For us to detect this, you need to import and/or link a service in PagerDuty to FireHydrant. |
Next Steps
Now that you've configured the PagerDuty integration, you can take the following steps to get the full power of FireHydrant + PagerDuty:
- Import and Link Components into your Service Catalog from PagerDuty
- See how you can page and lookup on-call personnel in PagerDuty
- Automatically create PagerDuty incidents via Runbook step
- Configure teams to pull personnel from on-call schedules
Updated 3 months ago