> ## Documentation Index
> Fetch the complete documentation index at: https://docs.chronosphere.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Convert Datadog monitors

[Datadog monitors](https://docs.datadoghq.com/api/latest/monitors/) actively check
metrics of the infrastructure and manage alerts on alert platforms. Chronosphere
metrics of the infrastructure and manage alerts on alert platforms. Chronosphere
Observability Platform uses [monitors and alerts](/investigate/alerts) for the same
purposes.

Datadog creates monitoring and notification in one longer file, while Chronosphere
separates monitors and notifications into smaller logical configuration files. These
smaller files enable users to target and update specific changes without risking the
entire configuration.

## Before you begin

A single Datadog monitor becomes several Observability Platform resources: a `Monitor`, a
`NotificationPolicy`, and one or more `Notifier` definitions. Before you convert a
monitor, gather the following details from the Datadog definition so you can map them
to the correct resource:

* **Query**: The Datadog `query` becomes the `Monitor` PromQL query. Review
  [querying metrics](/ingest/metrics-traces/collector/mappings/datadog/dogstatsd) for
  how the query syntax differs.
* **Thresholds and severities**: The Datadog `options.thresholds` (critical, warning)
  and any no-data behavior become `series_conditions` on the `Monitor`.
  Observability Platform doesn't support separate recovery thresholds.
* **Notification routes**: Each `@`-mention in the Datadog `message` (such as a Slack
  channel or PagerDuty service) becomes a `Notifier`, and the routing logic becomes a
  `NotificationPolicy`.
* **Tags**: Datadog `tags` become `Monitor` labels, which you can then use to route
  alerts with notification policy overrides.

Review [migration considerations](/ingest/metrics-traces/collector/mappings/datadog/migration-issues)
before you start, then use the [field mapping](#field-mapping) tables to translate
each field. Apply the resulting files with [Chronoctl](/tooling/chronoctl).

## Compare configurations

These are examples of matching configurations for Datadog and
Observability Platform.

<Tabs>
  <Tab title="Datadog" id="datadog-monitor">
    This is an example of a Datadog monitor definition.

    ```text theme={null}
    "id": 1234567,
        "org_id": 12345,
        "type": "metric alert",
        "name": "IOWAIT is high ({{value}})",
        "message": "{{#is_alert}}\Load is too high, check and lower load immediately (use AWS console for {{pod.name}} to scale tasks to 1 and investigate)\n@slack-ops-bots \n{{/is_alert}} \n\n{{#is_alert_recovery}}\n@slack-ops-bots \n@pagerduty-resolve \n{{/is_alert_recovery}}{{#is_warning}}\nLoad is reaching the limit.\n@slack-ops-warning-bots\n@pagerduty{{/is_warning}}",
        "tags": [
            "high-load",
            "team:platform"
        ],
        "query": "min(last_30m):max:system.cpu.iowait{function:cassandraevents} by {pod,name} > 20",
        "options": {
            "notify_audit": false,
            "locked": false,
            "timeout_h": 0,
            "include_tags": true,
            "no_data_timeframe": 30,
            "require_full_window": true,
            "notify_by": ["pod"],
            "notify_no_data": true,
            "new_group_delay": 60,
            "renotify_interval": 30,
            "renotify_occurrences": 1,
            "renotify_statuses": [
                "alert",
                "no data",
            ],
            "scheduling_options": {
                "evaluation_window": {
                    "hour_starts": 30
                }
            },
            "thresholds": {
                "critical": 20,
                "critical_recovery": 10,
                "warning": 15
            },
            "timeout_h": 12,
            "escalation_message": "{{#is_alert}}\nEscalated to pagerduty - \nLoad is too high, check and lower load immediately (use AWS console for {{pod.name}} to scale tasks to 1 and investigate)\n@slack-ops-bots \n@pagerduty \       n{{/is_alert}}",
            "evaluation_delay": 300,
            "min_failure_duration": 120,
            "silenced": {}
        },
        "multi": true,
        "created_at": 1479858941000,
        "created": "2016-11-22T15:55:41.80188-08:00",
        "modified": "2021-10-14T09:23:36.750186-07:00",
        "deleted": null,
        "restricted_roles": null,
        "priority": 1,
        "overall_state_modified": "2022-07-05T06:13:14-07:00",
        "overall_state": "OK",
        "creator": {
            "name": "Jane Smith",
            "handle": "janesmith@example.com",
            "email": "janesmith@example.com",
            "id": 18219
        },
        "matching_downtimes": []
    ```
  </Tab>

  <Tab title="Chronosphere" id="chronosphere-monitors">
    Observability Platform configuration breaks down into smaller, more strictly defined
    files covering the same functions. These file definitions use the `v1/config`
    resource format that you apply with [Chronoctl](/tooling/chronoctl).

    Configure the monitor:

    ```yaml Chronoctl example icon="square-terminal" theme={null}
    api_version: v1/config
    kind: Monitor
    spec:
      slug: cluster-iowait-high
      name: IOWAIT is high
      labels:
        datadog_id: "1234567"
        tag_iowait: "true"
        team: platform
      annotations:
        message_critical: Load is too high, check and lower load immediately (use AWS console for {{pod.name}} to scale tasks to 1 and investigate)
        message_warning: Load is reaching the limit
      prometheus_query: <promql query>
      signal_grouping:
        label_names:
          - name
      series_conditions:
        defaults:
          critical:
            conditions:
              - op: GT
                value: 20
                sustain_secs: 120
          warn:
            conditions:
              - op: GT
                value: 15
                sustain_secs: 120
      notification_policy_slug: iowait-notification-policy
      interval_secs: 60
    ```

    This is the notification policy:

    ```yaml Chronoctl example icon="square-terminal" theme={null}
    api_version: v1/config
    kind: NotificationPolicy
    spec:
      slug: iowait-notification-policy
      name: iowait notification policy
      routes:
        defaults:
          critical:
            notifier_slugs:
              - default-route-slack-bot
          warn:
            notifier_slugs:
              - default-route-slack-bot
        overrides:
          - alert_label_matchers:
              - name: tag_iowait
                type: EXACT
                value: "true"
            notifiers:
              critical:
                notifier_slugs:
                  - slack-ops-bots
                  - pagerduty-critical
              warn:
                notifier_slugs:
                  - slack-ops-warning-bots
                  - pagerduty-warning
    ```

    And separate notifier routes:

    Slack, for an Alerts channel:

    ```yaml Chronoctl example icon="square-terminal" theme={null}
    api_version: v1/config
    kind: Notifier
    spec:
      name: Default route slack bot
      slug: default-route-slack-bot
      skip_resolved: false
      slack:
        api_url: <slack url>
        channel: alerts
        username: oncall-bot
    ```

    Slack, for an on-call channel:

    ```yaml Chronoctl example icon="square-terminal" theme={null}
    api_version: v1/config
    kind: Notifier
    spec:
      name: slack-ops-bots
      slug: slack-ops-bots
      skip_resolved: false
      slack:
        api_url: <slack url>
        channel: oncall
        username: oncall-bot
    ```

    Critical alerts sent to PagerDuty:

    ```yaml Chronoctl example icon="square-terminal" theme={null}
    api_version: v1/config
    kind: Notifier
    spec:
      name: pagerduty-critical
      slug: pagerduty-critical
      skip_resolved: false
      pagerduty:
        routing_key: <routing_key>
        url: https://events.pagerduty.com/v2/enqueue
        severity: critical
    ```

    Warnings sent to PagerDuty:

    ```yaml Chronoctl example icon="square-terminal" theme={null}
    api_version: v1/config
    kind: Notifier
    spec:
      name: pagerduty-warning
      slug: pagerduty-warning
      skip_resolved: false
      pagerduty:
        routing_key: <routing_key>
        url: https://events.pagerduty.com/v2/enqueue
        severity: warning
    ```
  </Tab>
</Tabs>

## Field mapping

Chronosphere and Datadog fields have many equivalent functions. Use the following
tables to map fields between these apps.

<Note>
  Names of Chronosphere equivalents are subject to change as the conversion process
  improves.
</Note>

### Configuration mapping

This table matches Datadog fields to their Chronosphere equivalents for monitor
specification.

| Datadog field        | Chronosphere equivalent                                                                                                                                                                                                                           |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `created`            | N/A                                                                                                                                                                                                                                               |
| `creator`            | N/A                                                                                                                                                                                                                                               |
| `id`                 | Add to `Monitor.labels`.                                                                                                                                                                                                                          |
| `message`            | Add to `Monitor.annotations` and create `Notifier` resources. See details.                                                                                                                                                                        |
| `modified`           | N/A                                                                                                                                                                                                                                               |
| `multi`              | `Monitor.spec.signal_grouping.signal_per_series`                                                                                                                                                                                                  |
| `name`               | `Monitor.name` - This can also contain variables.                                                                                                                                                                                                 |
| `options`            | [Monitor options](#monitor-options)                                                                                                                                                                                                               |
| `threshold_windows`  | N/A - Used only for `anomalies`.                                                                                                                                                                                                                  |
| `thresholds`         | `Monitor.spec.series_conditions.defaults.<severity>.conditions`                                                                                                                                                                                   |
| `timeout_h`          | N/A                                                                                                                                                                                                                                               |
| `overall_state`      | For monitors with an `Ignored / Skipped / Unknown` state, still create the monitor but have it either go to a black hole route or create it as muted.                                                                                             |
| `priority`           | Can support as a message annotation.                                                                                                                                                                                                              |
| `query`              | `Monitor.spec.prometheus_query`                                                                                                                                                                                                                   |
| `restricted_roles`   | N/A                                                                                                                                                                                                                                               |
| `state`              | N/A                                                                                                                                                                                                                                               |
| `matching_downtimes` | Equivalent to schedules.                                                                                                                                                                                                                          |
| `tags`               | An arbitrary list of strings that fits the tag format (which can be single word tags). Chronosphere can support this using `Monitor.labels`, if the field requires a key-value format. Tags are used as label names with the value set to `true.` |
| `type`               | The type of monitor. Chronosphere supports query alert and metric alerts.                                                                                                                                                                         |

### Monitor options

Use these values in the specification's `options` field.

| Datadog field                  | How to map                                                                                                                                                                               |
| ------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `aggregation`                  | N/A - For log alerts only.                                                                                                                                                               |
| `enable_logs_sample`           | N/A - For log alerts only.                                                                                                                                                               |
| `enable_samples`               | N/A - Per [Datadog docs](https://docs.datadoghq.com/api/latest/monitors/#get-all-monitor-details). This is used only by CI Test and Pipeline monitors.                                   |
| `escalation_message`           | No separate message for renotify notifications; can append this to the generic alert message.                                                                                            |
| `evaluation_delay`             | Can support by using offset in the query.                                                                                                                                                |
| `group_retention_duration`     | N/A - Not for metrics monitors.                                                                                                                                                          |
| `groupby_simple_monitor`       | N/A - For log alerts only.                                                                                                                                                               |
| `include_tags`                 | Use Prometheus  `{{ $value }}` template.                                                                                                                                                 |
| `min_failure_duration`         | `Monitor.spec.series_conditions.defaults.<severity>.conditions.sustain_secs`                                                                                                             |
| `min_location_failed`          | Can support by adding thresholds to the PromQL expression.                                                                                                                               |
| `new_group_delay`              | N/A                                                                                                                                                                                      |
| `new_host_delay`               | N/A - Deprecated, use `new_group_delay` instead.                                                                                                                                         |
| `no_data_timeframe`            | Threshold for a `no data` alert. See [severity](#severity) section for details.                                                                                                          |
| `notification_preset_name`     | N/A - Datadog [docs](https://docs.datadoghq.com/api/latest/monitors/#get-all-monitor-details).                                                                                           |
| `notify_audit`                 | N/A                                                                                                                                                                                      |
| `notify_by`                    | Equivalent to `Monitor.spec.signal_grouping`, except the inverse. Note: This can be set to `*`, which is the same as setting `Monitor.spec.signal_grouping.signal_per_series`.           |
| `notify_no_data`               | Add a `NOT_EXISTS` series condition in the MonitorSpec. Review [severity](#severity) for details.                                                                                        |
| `on_missing_data`              | N/A - Not for metrics alerts.                                                                                                                                                            |
| `renotify_interval`            | `NotificationPolicy.routes.overrides.notifiers.<severity>.repeat_interval_secs`                                                                                                          |
| `renotify_occurrences`         | N/A                                                                                                                                                                                      |
| `renotify_statuses`            | Only renotify on status X. Create overrides using `NotificationPolicy.routes.overrides.notifiers.<severity>.repeat_interval_secs` for each severity listed here.                         |
| `require_full_window`          | Only evaluate if there's a full window of data. Datadog recommends setting this to `false`. Supportable using the `count_over_time` function.                                            |
| `scheduling_evaluation_window` | [Cumulative time windows](https://docs.datadoghq.com/monitors/configuration/?tab=thresholdalert#cumulative-time-windows). For example, "evaluate this alert every hour on the :00 mark." |
| `silenced`                     | Dictionary of [muted tags to end timestamp](https://docs.datadoghq.com/monitors/guide/monitor_api_options/#common-options). Create MutingRule objects for each tag.                      |
| `thresholds`                   | Thresholds for severity. Can map to `Monitor.spec.series_conditions.defaults` for warning and critical. No support for separate thresholds for recovery.                                 |
| `variables`                    | N/A                                                                                                                                                                                      |

## Severity

Chronosphere supports both critical and warning severities by implementing different
thresholds for the metric values. In addition to this, Datadog also supports alerting
on no data for a particular metric as a distinct `severity`. Although this state
isn't a true severity, the state is treated the same as critical and warning alerts
for configuration.

Chronosphere supports alerting on `no data` conditions using a series condition in
the MonitorSpec:

```yaml Chronoctl example icon="square-terminal" theme={null}
api_version: v1/config
kind: Monitor
spec:
  prometheus_query: <promql query>
  series_conditions:
    defaults:
      critical:
        conditions:
          - op: NOT_EXISTS
            sustain_secs: 60
```

## Message and route

Datadog allows different messages and routing endpoints for the different severity
levels (critical, warning, no data). Chronosphere can support different messages by
using separate annotations:

```yaml Chronoctl example icon="square-terminal" theme={null}
api_version: v1/config
kind: Monitor
spec:
  annotations:
    message_critical: This is the critical threshold message
    message_warning: This is the warning threshold message
    message_no_data: This is the message for no data
```

To support different routes, users must use a separate monitor with different labels,
set using notification policies.

## Notification policy resources

Link a `Monitor` resource to a `Notifier` resource by defining a
[notification policy](/investigate/alerts/notifications/policies). Each unique
route in the Datadog message field maps to a `Notifier` resource. The `Monitor`
contains a `label` specifying the notification route it links to, and the
`NotificationPolicy` defines overrides that point to each `Notifier`.

For example:

```yaml Chronoctl example icon="square-terminal" theme={null}
api_version: v1/config
kind: Monitor
spec:
  labels:
    datadog_id: "1234567"
    route_slack_ops_bots_critical: "true"
    route_slack_ops_bots_warning: "true"
    route_pagerduty_critical: "true"

---
api_version: v1/config
kind: NotificationPolicy
spec:
  routes:
    overrides:
      - alert_label_matchers:
          - name: route_slack_ops_bots_critical
            type: EXACT
            value: "true"
        notifiers:
          critical:
            notifier_slugs:
              - slack-ops-bots
```

## Evaluation frequency

Datadog doesn't support the use of different evaluation frequencies per monitor, but
instead relies on a hard-coded interval dependant on the
[evaluation window](https://docs.datadoghq.com/monitors/configuration/?tab=thresholdalert#evaluation-frequency).
For windows of less than `24h`, the window defaults to `1m`. Set this to a value you
want to use with the `Monitor.spec.interval_secs` field (in seconds), or default to
`15` to receive faster alerts.
