Notifications and alerting — channel setup¶

Navigate: Notifications (/notifications) → Channels → Add channel (/notifications/channels/new).

Paridity sends outbound alerts — a run failed a proof, schema drift was detected, a terminal status was reached — to the tools your team already watches. A channel is a destination (a Slack workspace, a PagerDuty service, a Jira project); a rule decides which events go to which channel. This page is the per-channel setup reference: what credential each channel needs, the configuration fields it reads, how to create the credential in the vendor tool, and how to verify delivery before you rely on it.

Notifications are outbound only. Paridity never reads back from these tools except where a channel returns a ticket URL it just created (ServiceNow and Jira).

How credentials are stored¶

You never paste a raw secret into a channel. Every channel stores a credential_ref — a URI pointer into your secrets manager, exactly like a system connection (aws-secrets://…, azure-keyvault://…, gcp-secretmanager://…, infisical://…, or vault://…). The secret itself is resolved through the SecretsClient at the moment a notification is sent and is never written to the platform database. The non-secret routing information (a Slack channel name, a Jira project key, a ServiceNow instance URL) lives in the channel's channel_config and is not sensitive.

So for every channel below there are two things to provide:

The credential — created in the vendor tool, stored in your secrets manager, referenced by credential_ref.
The channel_config — the non-secret fields, entered in the channel form.

Verify a channel before you trust it¶

Every channel supports a test send. Use it immediately after creating a channel — a wrong token or a blocked URL surfaces here, not during your first real incident.

In the UI: open the channel and use its Send test action. It fires a synthetic "Paridity test notification" straight at the adapter (bypassing rules) and reports sent or failed with the provider's reason, cut to its first line (at most 200 characters). If the channel's credential pointer cannot be resolved, the reason says so without the secrets backend's own text; that is in the server log under the request's reference (X-Request-ID).
By API: POST /api/v1/notification-channels/{channel_id}/test. Requires notification_rule.create (roles platform_admin, tenant_admin, or operator).
ServiceNow and Jira only — POST /api/v1/notification-channels/{channel_id}/test-ticket opens a real ticket titled [TEST] and returns its URL, so you can confirm the ITSM integration end to end (and then close the test ticket).

A failed test returns the real vendor error (an invalid Slack token, an AADSTS7000215 bad client secret, an SSRF-blocked URL), which is almost always enough to fix the setup on the spot.

Who may set a channel's credential and destination¶

A channel pairs a secret (the credential_ref) with the place it is sent (a URL or host). Whoever sets both decides where a secret goes, so:

Setting or changing credential_ref, or a destination field, needs platform_admin. The destination fields are base_url (Jira), instance_url (ServiceNow), url (webhook), and smtp_host, smtp_port and tenant_id (email). Slack and Teams incoming webhooks carry their destination inside the secret, so for them the credential_ref is the destination. Anyone else gets HTTP 403 CHANNEL_DESTINATION_REQUIRES_ADMIN.
An operator can still rename a channel, turn it on or off, edit its other settings (recipients, project key, issue type) and run Send test. Resending the stored value unchanged is not a change.
A channel's credential_ref must be its own. A ref that names a system's connection_ref, an AI provider or identity-provider secret, a decrypt key, a signing or encryption-key ref, a replication tool credential or a platform signing setting is refused with HTTP 422 CHANNEL_CREDENTIAL_CROSS_USE, and a stored channel that names one is refused at send time.
Every change of credential_ref or a destination writes a notification_channel.destination_changed audit entry with the actor, the old and new destination hosts, and a SHA-256 fingerprint of the ref (never the ref itself or the secret).

All seven channels below were live-delivery verified end to end on 2026-08-24 (#1712 / PR #1961) — each one both delivering a real message and surfacing a deliberately bad credential as a failure.

Slack¶

Credential: either a Slack Incoming Webhook URL or a Bot OAuth token (xoxb-…). The adapter auto-detects which you gave it: a value starting with xoxb- is treated as a bot token and posted via chat.postMessage; anything else is treated as the Incoming Webhook URL and posted directly.

channel_config:

Field	Required	Default	Notes
`channel`	Only with a bot token	`#general`	The channel to post into. Ignored on the Incoming Webhook path — a webhook URL already encodes its channel.

Set up an Incoming Webhook (simplest):

Go to api.slack.com/apps → Create New App → From scratch (Slack's current app-creation dialog; the old "From an app manifest" split is gone — pick From scratch, name the app, choose the workspace).
Open Incoming Webhooks, toggle Activate Incoming Webhooks on.
Add New Webhook to Workspace, choose the target channel, and copy the generated https://hooks.slack.com/services/… URL.
Store that URL in your secrets manager and reference it as the channel's credential_ref. Leave channel blank.

Or use a Bot token (needed if one app must post to several channels): add the chat:write OAuth scope, install the app to the workspace, copy the Bot User OAuth Token (xoxb-…), store it, and set channel to the target (e.g. #recon-alerts). Invite the bot to any private channel it must post to.

Note: the Slack Web API returns HTTP 200 even when a post fails ({"ok": false}); Paridity parses ok and reports those as failures, so a bad bot token shows up correctly in the test send.

Microsoft Teams¶

Credential: a Teams Workflows (Power Automate) webhook URL — the trigger URL of a "Post to a channel when a webhook request is received" flow.

Microsoft has retired the classic Office 365 "Incoming Webhook" connector (deprecation began August 2024, completing through end-2025). Do not use a classic Incoming Webhook — it will stop working. Paridity posts the Workflows-compatible Adaptive Card envelope ({"type":"message","attachments":[{ …adaptive card… }]}), which is what the Power Automate flow expects.

channel_config: none. (The adapter's docstring shows a card_color example, but the code does not read it — card accent colour is fixed per severity. Setting card_color has no effect; leave channel_config empty.)

Set up the Workflows webhook:

In the target Teams channel, ⋯ → Workflows (or start from Power Automate) and pick the template "Post to a channel when a webhook request is received."
Complete the flow (choose the team and channel it posts to) and copy the generated HTTP POST URL.
Store that URL in your secrets manager and reference it as credential_ref.

The card shows the severity, trigger, mapping, and run ID as a fact list.

PagerDuty¶

Credential: a PagerDuty Events API v2 Integration Key (also called the routing key). Events are sent to https://events.pagerduty.com/v2/enqueue.

channel_config:

Field	Required	Default	Notes
`severity_map`	No	`{}`	Maps Paridity severity (`info`/`warning`/`error`/`critical`) to a PagerDuty severity, which must be one of `critical`/`error`/`warning`/`info`. Unmapped values pass through unchanged.

Set up the integration:

In PagerDuty, open the target service → Integrations → Add integration, choose Events API v2.
Copy the Integration Key and store it in your secrets manager as the channel's credential_ref.

Deduplication: when an event carries a run_id, Paridity sets dedup_key = recon-<run_id>, so multiple events for the same run collapse into a single incident rather than paging repeatedly (verified live: 2 events → 1 incident).

Recommendation: keep PagerDuty Intelligent Alert Grouping ON for the service. (Our internal test rig runs it off to make each event individually visible; in production, grouping reduces noise and complements the per-run dedup_key.)

ServiceNow¶

Credential: either a Base64-encoded user:password string (sent as Authorization: Basic …) or a value beginning with Bearer (sent verbatim as a bearer token). Records are created via {instance_url}/api/now/table/{table}.

channel_config:

Field	Required	Default	Notes
`instance_url`	Yes	—	e.g. `https://yourinstance.service-now.com`. Missing → the channel fails with "instance_url is required."
`table`	No	`incident`	Target table.
`impact_map`	No	`{critical:1, error:2, warning:3, info:3}`	Paridity severity → ServiceNow impact (1–3).
`urgency_map`	No	`{critical:1, error:2, warning:3, info:3}`	Paridity severity → ServiceNow urgency (1–3).
`assignment_group`	No	—	Added to the record only if set.
`category`	No	—	Added to the record only if set.

Set up:

Create (or reuse) a ServiceNow integration user with permission to insert into your target table, or mint an OAuth bearer token.
For Basic auth, Base64-encode user:password and store that string in your secrets manager; for a token, store Bearer <token>. Reference it as credential_ref.
Set instance_url and, if you want them, impact_map/urgency_map, assignment_group, and category.

The created record's URL is returned, and you can dry-run the whole path with the test-ticket action, which opens a [TEST] incident.

Jira¶

Credential:

Jira Cloud (*.atlassian.net): [email protected]:api_token (sent as Authorization: Basic, REST API v3, description in Atlassian Document Format).
Jira Data Center / Server: a Personal Access Token (sent as Authorization: Bearer, REST API v2, plain-text description).

The adapter picks Cloud vs DC automatically by whether atlassian.net appears in base_url.

channel_config:

Field	Required	Default	Notes
`base_url`	Yes	—	e.g. `https://acme.atlassian.net`.
`project_key`	Yes	—	e.g. `RECON`. Missing `base_url` or `project_key` → the channel fails.
`issue_type`	No	`Bug`	Issue type name.
`labels`	No	`[]`	Added only if non-empty.
`assignee_account_id`	No	—	Cloud account ID (Cloud) or username (DC).
`priority_map`	No	`{critical:Highest, error:High, warning:Medium, info:Low}`	Unmapped severity → `Medium`.

Set up (Cloud):

Create an API token at id.atlassian.com → Security → API tokens for a user who can create issues in the project.
Store email:api_token in your secrets manager; reference it as credential_ref.
Set base_url and project_key (and issue_type if not Bug).

Set up (Data Center): create a Personal Access Token for the service user, store the raw PAT, and set the same channel_config.

The created issue's URL ({base_url}/browse/{KEY}) is returned; use the test-ticket action to open a [TEST] issue end to end.

Jira channels require platform build ≥ migration 184 (which added jira to the channel-type enum). Earlier builds could run the adapter but not persist a Jira channel.

Email¶

Email has two auth modes, chosen by channel_config.auth_mode. Pick graph for Microsoft 365 / Exchange Online (which has disabled basic SMTP AUTH tenant-wide), and smtp_basic for any other SMTP server.

Mode `smtp_basic` (default)¶

Credential: the SMTP password (or app password).

Field	Required	Default	Notes
`auth_mode`	No	`smtp_basic`	Omit for SMTP.
`smtp_host`	Yes	—	e.g. `smtp.example.com`.
`smtp_port`	No	`587`	—
`smtp_user`	No	`""`	The login user; also the default `from_addr`.
`from_addr`	No	= `smtp_user`	Envelope sender.
`to`	Yes	`[]`	Recipient list. Empty `to` → the channel fails.
`cc`	No	`[]`	—
`use_tls`	No	`true`	`true` → STARTTLS on `smtp_port`; `false` → implicit TLS (SMTP over SSL).

Mode `graph` (Microsoft 365 / Exchange Online)¶

Credential: the Azure app registration's client secret (never in channel_config).

Field	Required	Default	Notes
`auth_mode`	Yes	—	Set to `graph`.
`tenant_id`	Yes	—	Azure directory (tenant) ID.
`client_id`	Yes	—	App registration (client) ID.
`from_addr`	Yes	—	The sending mailbox UPN.
`to`	Yes	—	Recipient list.
`cc`	No	`[]`	—

Set up Graph:

In Entra ID → App registrations → New registration, create an app.
Under API permissions, add Microsoft Graph → Application permissions → Mail.Send, then Grant admin consent (this is an application permission, not delegated).
Under Certificates & secrets, create a client secret; store its value in your secrets manager and reference it as credential_ref.
Set auth_mode: graph, tenant_id, client_id, from_addr (the mailbox to send as), and to.

Production tenants — scope the app. Mail.Send as an application permission lets the app send as any mailbox in the tenant. Restrict it with an Exchange application access policy so the app may only send as the specific from_addr mailbox(es) you intend. Do this before enabling the channel in production.

Paridity uses the OAuth2 client-credentials flow and calls Graph sendMail (expecting 202 Accepted); it does not read the mailbox back. Auth failures surface the Azure error (e.g. AADSTS7000215 for a bad client secret) in the test send.

Webhook (generic)¶

Credential: a bearer token / API key, sent as Authorization: Bearer <credential> (your own static headers can override).

channel_config:

Field	Required	Default	Notes
`url`	Yes	—	Must be `https://`. Missing → the channel fails.
`method`	No	`POST`	Upper-cased.
`headers`	No	`{}`	Static headers merged into every request.

The adapter posts a fixed JSON envelope:

{
  "trigger": "schema_drift",
  "severity": "critical",
  "title": "...",
  "body": "...",
  "run_id": "...",
  "mapping_id": "...",
  "mapping_name": "...",
  "tenant_id": "...",
  "details": { }
}

SSRF protection (important for self-hosted receivers). Every channel adapter that sends a credential - webhook, Jira, ServiceNow, Slack, Teams, PagerDuty and Microsoft Graph email - is deliberately restrictive so it cannot be turned into a request forgery primitive (#3365 extended the webhook policy to all of them; SMTP email gets the address check below, without the https rule):

The URL scheme must be https — plain HTTP is rejected.
The hostname is DNS-resolved and rejected if it resolves to a private, loopback, link-local, or CGNAT address (10/8, 172.16/12, 192.168/16, 127/8, 169.254/16, 100.64/10, and the IPv6 equivalents). Point the webhook at a publicly resolvable HTTPS endpoint. This applies to a self-hosted Jira Data Center, an on-premises ServiceNow or an internal SMTP relay too: each is refused unless a platform admin has added it to the private destination allowlist (next section).
HTTP redirects are not followed, to prevent redirect-based bypass.
After connecting, the address actually connected to is checked again before any response is read (for SMTP: before the password is sent), so a DNS answer that changes between the check and the connection cannot reach a blocked address.

There is a deployment-level dev/CI escape hatch (NOTIFY_WEBHOOK_ALLOW_PRIVATE=1 in the worker environment) that disables the SSRF check so a docker-network sink can receive deliveries. It is environment-only — it can never be enabled through channel_config or the API — and must never be set outside a local dev or CI stack.

Since #2142 this policy is shared with the SSO paths (OIDC discovery and the JWKS fetch), and the escape hatch has a policy-wide name: OUTBOUND_URL_ALLOW_PRIVATE=1 relaxes it for every outbound surface in the process, which is what a dev stack running Keycloak on plain HTTP over a private docker network needs. NOTIFY_WEBHOOK_ALLOW_PRIVATE continues to work unchanged; either variable being set relaxes the check. Both are environment-only, both are dev/CI only, and the Helm chart sets neither — a production deployment has no way to turn the policy off.

Private destinations (internal systems)¶

A customer-hosted installation often sends alerts to systems on its own network: an internal SMTP relay, Jira Data Center, an on-premises ServiceNow, an internal webhook receiver. A platform admin can permit such a destination for the tenant under Notifications -> Private destinations (API: POST /api/v1/notification-private-destinations). Everyone who can read notification channels (notification_rule.read) sees the list; only a platform admin can add or remove entries (notification_channel.configure_destination, the same permission that already decides where a channel secret is sent).

Each entry is exactly one host and one port:

Host form	Example	Permits
Hostname	`jira.corp.example`	A channel URL or SMTP host that is exactly this name (case and a trailing dot ignored). No wildcards; `x.jira.corp.example` needs its own entry.
IP address or CIDR	`10.20.0.5`, `10.20.0.0/24`, `fd12:3456::/64`	A channel URL or SMTP host written as an IP address inside it. A CIDR never permits a hostname that happens to resolve into it.

The port is the URL's port (443 when https:// names none) or the email channel's smtp_port. A different port is refused.

What an entry permits. A destination whose host and port match an entry may resolve to an address in 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 100.64.0.0/10 or IPv6 fc00::/7. Nothing else about the policy changes:

HTTP channels still require https, and redirects are still not followed.
The connection must land on one of the addresses the name resolved to when it was checked. If DNS answers with a different private address at connect time, the send is refused (DNS rebinding cannot move an allowlisted name to another internal host).
The entry applies only to that tenant's notification channels. It does not apply to sign-in (OIDC / JWKS) or any other outbound path.

What an entry can never permit. Loopback (127.0.0.0/8, ::1), link-local (169.254.0.0/16, fe80::/10, which includes the AWS, Azure, GCP and OCI metadata address 169.254.169.254), the AWS IPv6 metadata address fd00:ec2::254, the Alibaba Cloud metadata address 100.100.100.200, unspecified (0.0.0.0/8, ::) and multicast addresses, and the names localhost / *.localhost / metadata.google.internal. Such an entry is refused when it is added (HTTP 422 PRIVATE_DESTINATION_NEVER_PERMITTED; a CIDR that overlaps one of these ranges is refused the same way), and such an address is refused at send time whatever the list says. A public address needs no entry and is refused as one (PRIVATE_DESTINATION_NOT_PRIVATE).

A tenant holds at most 200 entries. Each addition and removal writes a notification_private_destination.added / .removed audit entry (notice severity) with the actor and the host, port and note. Removing an entry takes effect on the next send.

An entry does not relax certificate checks. An internal endpoint whose certificate is issued by a private CA needs that CA trusted - see the next section.

Internal CA certificates¶

An HTTPS channel validates the server certificate against the public CA bundle shipped with the platform (certifi), and SMTP STARTTLS / SSL validates against the container's system trust store. An internal Jira, ServiceNow, webhook receiver or SMTP relay whose certificate comes from your own private CA fails that check until the CA is trusted.

Set TLS_CA_BUNDLE_PATH to a PEM file holding your internal CA certificate(s) (the root, plus any intermediates the server does not send). The platform adds these CAs to the default trust store; it does not replace it, so public destinations such as Slack, PagerDuty and Microsoft Graph keep working. It applies to every notification channel (HTTPS and SMTP) and to the other outbound fetches that share the same client, including OIDC discovery and JWKS for sign-in.

Certificate verification is never turned off. The server certificate must still chain to a trusted CA and match the host name; a certificate from any CA that is in neither the default store nor your bundle is still refused.

With the Helm chart, put the bundle in a ConfigMap or Secret and enable tls.caBundle. The chart mounts it read-only into the API and every worker and sets TLS_CA_BUNDLE_PATH for you:

kubectl create configmap corp-root-ca --from-file=ca-bundle.pem=./corp-root-ca.pem

tls:
  caBundle:
    enabled: true
    configMapName: corp-root-ca    # or secretName: <a Secret> - exactly one
    key: ca-bundle.pem             # the key that holds the PEM
    mountPath: /etc/paridity/tls   # TLS_CA_BUNDLE_PATH = <mountPath>/<key>

Naming both secretName and configMapName, or neither, fails the helm install / helm upgrade with a message saying so. With Docker Compose, mount the file into the api and worker-notification services and set TLS_CA_BUNDLE_PATH on both (the shipped docker-compose.yml has a commented example).

If the file is missing or is not a PEM certificate bundle, the API and the notification worker log an ERROR at start naming the path, and every outbound HTTPS / SMTP send refuses with the same reason until it is fixed. Nothing falls back to an unverified connection. A changed bundle is picked up when the pods restart.

Delivery history — was anyone told?¶

The Delivery history tab on the Notifications page is the platform's own answer to the question every operator asks after an incident: was anyone told, and if not, why not? It reads the append-only delivery log — every decision the platform made about a notification, newest first — and needs no database access.

Each row shows when the decision was made, the trigger and severity, the mapping, the channel, the status, and the run it belongs to (the run link opens the run's evidence). The status is the point of the page:

Status	What it means	What to look at
Sent	The provider accepted the message. `Sent at` is the acceptance time.	Nothing — unless the recipient still did not see it, in which case the problem is downstream of Paridity (mailbox rules, a muted channel).
Failed	The provider refused the message. The provider's error is shown in full in the row's details — for example Mailtrap's `550 5.7.0 Too many emails per second. Please upgrade your plan`.	The error text. It names the fix: a rate limit, a revoked webhook, an unreachable host.
Suppressed	Paridity decided not to send. This is not a failure — the row records the reason so a deliberate silence is never mistaken for a lost message.	The reason (below).
Pending	Recorded but not yet attempted. A row that stays pending is a worker that has not run.	Worker health.

Suppression reasons, in plain words:

cooldown — a notification for this rule, trigger and mapping was already sent inside the rule's cooldown window (set per rule; 300 s by default).
flag_off — automatic ticketing is switched off for this mapping.
no_channel — automatic ticketing is on for this mapping, but no auto-ticket channel is configured.
channel_disabled — the configured channel is disabled or no longer exists.
no_adapter — the configured channel's type cannot open tickets.
not_applicable — the mapping was not visible to the tenant when the ticket was evaluated.
resend_exhausted — a lost auto-ticket was re-sent the maximum number of times and the platform stopped retrying.

A quiet period is different from all four statuses: a rule that never matched leaves no row. If you expected a notification and see nothing, the rule did not fire — check its triggers, minimum severity and mapping scope.

Filter by status, trigger and channel, or open the tab from a run's Notifications card to see only that run. The history is read-only: it is evidence, and nothing on this page edits or deletes it. Visible to every role that can read runs, including analysts and auditors.

Access, roles and audit — who may create channels and rules, and how notification actions are audited.
Runs and Evidence — the events that trigger notifications.
Secrets are stored the same way as system connections — see your deployment's secrets-manager setup for creating a credential_ref.

Notifications and alerting — channel setup¶

How credentials are stored¶

Verify a channel before you trust it¶

Who may set a channel's credential and destination¶

Slack¶

Microsoft Teams¶

PagerDuty¶

ServiceNow¶

Jira¶

Email¶

Mode smtp_basic (default)¶

Mode graph (Microsoft 365 / Exchange Online)¶

Webhook (generic)¶

Private destinations (internal systems)¶

Internal CA certificates¶

Delivery history — was anyone told?¶

Related¶

Mode `smtp_basic` (default)¶

Mode `graph` (Microsoft 365 / Exchange Online)¶