Catalogskills.sh · datadog-labs

service-remapping

Create and manage APM service remapping rules — rewrite service names at ingestion time to collapse noisy inferred entities, clean up auto-generated names, handle org renames, or normalize naming conventions. Use for any request involving service renaming, service mapping, inferred service cleanup, peer.service normalization, or collapsing fragmented service names.

Review advisedVerified hub485 installs

Savant verdict: Review advised

Security analysis or source-hub review identified patterns requiring attention.

Live evaluation

Not evaluated yet. Workspaces can request a live evaluation.

Security assessment

Scan pending.

Structure

  • No license declaredConfirm you may reuse this skill before importing it into your repository.
  • The source hub's security checks flagged this skillSnyk: warn (Risk: MEDIUM · 1 issue)

SKILL.md

---
name: service-remapping
description: Create and manage APM service remapping rules — rewrite service names at ingestion time to collapse noisy inferred entities, clean up auto-generated names, handle org renames, or normalize naming conventions. Use for any request involving service renaming, service mapping, inferred service cleanup, peer.service normalization, or collapsing fragmented service names.
metadata:
  version: "1.0.0"
  author: datadog-labs
  repository: https://github.com/datadog-labs/agent-skills
  tags: datadog,apm,service-remapping,service-naming,inferred-services,peer-service
  alwaysApply: "false"
  tools: pup
---

# APM Service Remapping

> **Before acting:** Surface an impact preview (monitors/dashboards referencing the old service name) before presenting the planned rule. For inferred-entity remaps, also confirm `peer.service` is set on outbound spans. Variables from `## Context to resolve before acting` can be gathered alongside that preview rather than blocking it.

---

## How Service Remapping Works — Domain Knowledge

Read this before building any rule. It gives you the mental model to construct the right filter and catch edge cases.

**What remapping does:** A rule intercepts telemetry at ingestion time and rewrites the service name before indexing. A rule says: "for any entity matching this filter, replace its service name with this new value."

**Two entity types — pick the right one:**

| Entity type | `rule_type` integer | What it targets |
|---|---|---|
| **SERVICE** | `0` | Instrumented services — have spans with an explicit `service` tag set by a tracer |
| **INFERRED_ENTITY** | `1` | Auto-detected from outbound calls — named from `peer.service`. **Requires `peer.service` to be set on outbound spans** (see prerequisite below). |

**Prerequisite for inferred entity remapping — `peer.service` must be set:**

Inferred entity remapping only works when the tracer sets `peer.service` on outbound spans. Without it, entities are keyed by `peer.hostname` and remapping rules will not apply.

To enable this, set the following env var on the **instrumented service** (not the downstream dependency):

```bash
DD_TRACE_PEER_SERVICE_DEFAULTS_ENABLED=true
```

This makes the ddtrace tracer automatically propagate `peer.service` from `peer.hostname` on outbound HTTP, gRPC, and database calls. Without this, `pup traces search` will show spans with `peer.hostname` but no `peer.service`, and no service remapping rule will match.

To verify `peer.service` is being set before building a rule:

```bash
pup traces search --query "@peer.service:<ENTITY_NAME>" --from 15m --limit 5
```

If zero results — the tracer is not setting `peer.service`. Ask the user to add `DD_TRACE_PEER_SERVICE_DEFAULTS_ENABLED=true` to their service's environment and redeploy before continuing.

**Filter syntax** — a standard Datadog event-grammar query string:

| Goal | Filter |
|---|---|
| Exact service match | `service:payments` |
| All services with a prefix | `service:deploy-test*` |
| All services with a suffix | `service:*.tropos` |
| All services containing a string | `service:*payments*` |
| All inferred services under a domain | `peer.service:*.shopify.com` |
| Service in one environment only | `service:payments AND env:prod` |
| Multiple possible values | `service:(payments OR billing)` |

> **Supported operations only:** The above forms — exact match, wildcards, `AND`/`OR` — are the only accepted operations. More advanced query syntax (CIDR ranges, numeric comparisons, fuzzy matching, etc.) is not supported and will be rejected by the API with a filter syntax error.

**New name syntax** — the `value` field in `rewrite_tag_rules`:

| Form | Example | Use for |
|---|---|---|
| Static string | `my-service` | Every matched entity gets exactly this name |
| Tag interpolation | `{{service}}` | Substitute the full value of a tag |
| Tag + regex capture | `{{service\|^(.+?)\..*$}}` | Extract part of a tag value (non-greedy capture) |

**Regex constraints for `{{tag\|regex}}`:**
- Maximum **1 capture group** per expression
- **No greedy quantifiers inside capture groups** — use non-greedy variants: `(.+?)` not `(.+)`, `(.*?)` not `(.*)`
- Quantifiers on capture groups themselves (e.g. `(foo)+`) are not allowed
- **No capture group** → the entire match is used as the replacement value
- **Capture group spanning the entire match** (e.g. `^(.*)$`) is currently rejected by the UI and will soon be rejected by the API — if you want the full tag value, use tag interpolation (`{{service}}`) instead of a regex

**Five remapping patterns:**

| Pattern | User says… | Filter example | New name example |
|---|---|---|---|
| **N:1 group** | "These N services are all the same thing" | `peer.service:*.shopify.com` | `shopify` |
| **Strip suffix/prefix** | "The name has junk at the end/start" | `service:*.tropos` | `{{service\|^(.+?)\..*$}}` |
| **1:1 rename** | "We renamed this service and Datadog needs to match" | `service:old-auth-service` | `auth-service` |
| **Env split** | "I want separate services per env but they all have the same name" | `service:my-service AND env:prod` | `my-service-prod` |
| **Prefix normalization** | "All services should start with an env or team name" | `service:payments*` | `{{env}}-{{service}}` |

---

## Triggers

Invoke this skill when the user wants to:
- Rename a service in Datadog without re-instrumenting
- Collapse multiple inferred service names into one (e.g. many `api.shopify.com/*` variants → `shopify`)
- Strip environment suffixes, version tags, or deployment metadata baked into service names
- Normalize `peer.service` names to something meaningful
- Rename a service after an org change, product rebrand, or migration
- Split a single service into per-env variants (`my-service` + `env:prod` → `my-service-prod`)
- List, review, or delete existing service remapping rules

Do NOT invoke this skill if:
- The user wants to rename the service in their application code — that requires a tracer config change (`DD_SERVICE`), not a remapping rule
- The user wants to correlate telemetry across infrastructure tags — that is the "Correlate telemetry" action type in the UI, not remapping

---

## Prerequisites

### pup-cli: check, install, and authenticate

### Claude runs

```bash
pup --version
```

If not found:

### Claude runs

```bash
brew tap datadog-labs/pack
brew install pup
```

Check auth:
```bash
pup auth status
```

If not authenticated:

### Claude runs

```bash
pup auth login
```

> This opens a browser tab for OAuth. Complete the login there — Claude will continue once the command exits.

### Credentials for write operations

`pup apm service-remapping list` and `get` work with OAuth. Create, update, and delete require API keys (`DD_API_KEY`, `DD_APP_KEY`, `DD_SITE`) until `apm_service_renaming_write` is added to pup's OAuth scopes.

### Claude runs

```bash
echo "DD_API_KEY set: $([ -n "${DD_API_KEY:-}" ] && echo yes || echo no)"
echo "DD_APP_KEY set: $([ -n "${DD_APP_KEY:-}" ] && echo yes || echo no)"
echo "DD_SITE: ${DD_SITE:-not set (defaulting to datadoghq.com)}"
```

If any are missing and you need to create/update/delete rules:

### What you need to do in a terminal

```bash
export DD_API_KEY=<your-api-key>
export DD_APP_KEY=<your-app-key>
export DD_SITE=datadoghq.com   # adjust for your site
```

> Common sites: `datadoghq.com` (US1), `datadoghq.eu` (EU1), `us3.datadoghq.com`, `us5.datadoghq.com`, `ap1.datadoghq.com`

Wait for the user to set credentials, then re-run the check above before continuing.

---

## Context to resolve before acting

| Variable | How to resolve |
|---|---|
| `ENV` | Required before creating the rule (Step 4). Ask the user — do NOT assume `prod`. Read-only verification and impact preview do not need `ENV` and should run first. |
| `ORIGINAL_SERVICE` | Current service name(s) to remap — discover with `pup apm services list` or ask the user |
| `ENTITY_TYPE` | Instrumented service (`rule_type: 0`) or inferred entity (`rule_type: 1`)? Ask if unclear — see Domain Knowledge |
| `TARGET_NAME` | The desired new service name — ask the user |
| `PATTERN` | Which pattern applies — identify from the user's description (see Domain Knowledge above) |

---

## Step 0: Discover Current Service Names

If the user hasn't specified exact names to remap, discover what exists first:

### Claude runs

```bash
pup apm services list --from 1h          # use --env <ENV> to target a single environment
pup traces search --query "service:<PARTIAL_NAME>" --from 1h --limit 20
```

Use the output to help the user identify exact service names. Ask the user to confirm which names they want remapped before proceeding.

---

## Step 1: Build the Rule

Work through each component before writing any JSON.

### 1a. Check for integration override names

Some service names (e.g. `grpc-client`, `net/http`, `aws.s3`, `redis`) are **integration-generated overrides** — the tracer auto-tags spans with them based on the library being used, not a user-set `service` tag. Remapping these with a service remapping rule is the wrong tool: the override is injected per-span by the integration, so the remapped name will keep re-appearing unless the override itself is removed.

**How to detect:** if the service name looks like a well-known integration name (single-word library names, `<protocol>-<client>` patterns, `<vendor>.<resource>` patterns), ask the user:

> *"The name `<SERVICE>` looks like an integration override — a name the tracer sets automatically on spans from the `<LIBRARY>` integration, not a user-configured service name. Service remapping won't stick here because the override is re-applied on every span. The right fix is **integration override removal**, which strips these auto-names so the parent service's name propagates instead. This is currently only configurable in the Datadog UI under APM → Setup → Service Remapping → Integration Override Removal. Do you want to handle it there, or proceed with a remapping rule anyway?"*

If the user confirms it is an integration override, stop here and direct them to the UI. Do not create a remapping rule.

### 1b. Entity type

[DECISION: entity type — ask the user if unclear]
- Does the service appear because a tracer explicitly set its `service` tag? → `rule_type: 0` (SERVICE)
- Does it appear in the service map from outbound calls (e.g. a database, queue, or external API)? → `rule_type: 1` (INFERRED_ENTITY)

If the user wants to remap an inferred entity, verify `peer.service` is set before proceeding — see the prerequisite in Domain Knowledge. If it is not set, stop and ask the user to enable `DD_TRACE_PEER_SERVICE_DEFAULTS_ENABLED=true` first.

### 1c. Filter

Write a single event-grammar query string targeting the service(s) to remap. Use the filter syntax and pattern table in Domain Knowledge to pick the right form. **State the filter expression verbatim in the planned-rule preview (Step 3)** — it is the user's primary way to verify the rule will match the intended entities, and they cannot evaluate the rule without it.

### 1d. New name (`value`)

Use the new name syntax and regex table in Domain Knowledge to pick the right form. For regex values, apply the constraints listed there.

### 1e. Rule name

Suggest a descriptive name. Examples:
- `collapse-shopify-inferred-services`
- `strip-tropos-suffix`
- `rename-old-auth-to-auth-service`
- `env-split-my-service-prod`

---

## Step 2: Preview Impact

Before constructing the JSON, check what will be affected:

### Claude runs

```bash
# Confirm telemetry exists for the targeted service (zero spans = wrong query or wrong env)
pup traces search --query "service:<ORIGINAL_SERVICE>" --from 15m --limit 5

# Check for monitors referencing the old service name
pup monitors list | grep -i "<ORIGINAL_SERVICE>"

# Check for dashboards referencing the old service name
pup dashboards list | grep -i "<ORIGINAL_SERVICE>"

# List existing service remapping rules that may conflict
pup apm service-remapping list
```

Report to the user:

| Item | What to surface |
|---|---|
| **Telemetry volume** | Non-zero spans confirm the filter will match real data. Zero = likely wrong service name or env. |
| **Monitors** | Any monitor referencing the old service name will silently break after remapping. List them and offer to update. |
| **Dashboards** | Any dashboard with the old service name in its title will have stale references after remapping. List them and offer to update. |
| **Conflicting rules** | Existing rules targeting the same service may be overridden. Show conflicts and ask the user to confirm. |

**Known gaps — Claude cannot verify these automatically:**

Remapping a service name can also break the following. Claude has no `pup` commands to check them today, so surface this as a manual checklist for the user before they confirm:

> *"Before I create this rule, please verify `<ORIGINAL_SERVICE>` is not referenced in any of the following — they won't update automatically after remapping:*
> - *Spans-to-metrics rules (APM → Setup → Generate Metrics)*
> - *Trace-to-metrics rules*
> - *Span retention filters (APM → Setup → Retention Filters)*
> - *Logs-to-metrics rules (Logs → Generate Metrics)*
> - *Any pipeline, alert, or SLO that acts on the service name*"

If monitors reference the old service name, ask:
> *"I found `<N>` monitor(s) referencing `<ORIGINAL_SERVICE>`. After remapping, they'll need to be updated to use `<TARGET_NAME>`. Want me to update them now?"*

---

## Step 3: Confirm the Rule

Show the user the planned rule and confirm before creating. **Batch any unresolved context variables into this same prompt** — do not ask for them in a separate earlier turn. One round-trip, not two.

If the filter doesn't already scope to an environment, ask whether to add one — env scoping is done by appending `AND env:<ENV>` to the filter expression, not via a separate API parameter.

> *"I'm planning rule `<RULE_NAME>` with filter `<FILTER>` mapping `<ORIGINAL_SERVICE>` → `<TARGET_NAME>` (rule_type: `<TYPE>`). Should this be scoped to a specific environment? If so, I'll add `AND env:<ENV>` to the filter. Is this OK to proceed?"*

Wait for confirmation before continuing.

---

## Step 4: Create the Rule

### Claude runs

```bash
pup apm service-remapping create \
  --name "<RULE_NAME>" \
  --filter "<FILTER>" \
  --rule-type <TYPE> \
  --value "<TARGET_NAME>"
```

If the response contains an `id` field — creation succeeded. Record the `id` and `version` values from the response.

ERROR: `400 Bad Request` with "Filter expression has invalid syntax" — the filter query is malformed. Check glob syntax and boolean operators.

ERROR: `400 Bad Request` with "Template value in target name is invalid" — the `value` regex is invalid. Check: max 1 capture group, non-greedy quantifiers inside groups (`(.+?)` not `(.+)`).

ERROR: `401 Unauthorized` — credentials are invalid or expired. Re-check `DD_API_KEY` and `DD_APP_KEY`.

ERROR: `403 Forbidden` — the API key lacks `apm_service_renaming_write` permission.

---

## Step 5: Verify

Allow 2–5 minutes for the rule to propagate, then confirm it is active.

### For SERVICE rules (rule_type 0)

### Claude runs

```bash
# Confirm new service name appears in APM
pup apm services list --env <ENV> --from 5m

# Confirm traces are arriving under the new name
pup traces search --query "service:<TARGET_NAME>" --from 5m --limit 5
```

If `<TARGET_NAME>` appears in either — rule is active.

### For INFERRED_ENTITY rules (rule_type 1)

Inferred entities don't produce their own spans, so they won't appear in `pup apm services list` or `pup traces search`. Verify in two steps:

**Step 5a — confirm the rule is stored correctly:**

### Claude runs

```bash
pup apm service-remapping get <RULE_ID>
```

Confirm the filter and value match what you intended.

**Step 5b — confirm the entity name changed in the service map:**

Ask the user to check the APM Service Map in the Datadog UI and look for `<TARGET_NAME>` where `<ORIGINAL_SERVICE>` used to appear. The service map is the authoritative view for inferred entity names.

Alternatively, confirm new `peer.service` values are arriving on spans from the instrumented service:

### Claude runs

```bash
pup traces search --query "service:<INSTRUMENTED_SERVICE> @peer.service:<TARGET_NAME>" --from 5m --limit 5
```

If spans appear with `peer.service:<TARGET_NAME>` — rule is active.

ERROR: New name not appearing after 5 minutes:
- Confirm old service is still sending traces with the original `peer.service`: `pup traces search --query "@peer.service:<ORIGINAL_SERVICE>" --from 5m`
- If old name still appears, propagation may still be in progress — wait 2 more minutes and retry
- If neither name appears, confirm `DD_TRACE_PEER_SERVICE_DEFAULTS_ENABLED=true` is set on the instrumented service — without it `peer.service` is never set and the rule will never fire

---

## Managing Existing Rules

### List all rules

### Claude runs

```bash
pup apm service-remapping list
```

### Get a single rule

### Claude runs

```bash
pup apm service-remapping get <RULE_ID>
```

### Update a rule

Update requires the current `version` from list/get output. Show the proposed changes to the user and confirm before running:

### Claude runs

```bash
pup apm service-remapping update <RULE_ID> \
  --name "<RULE_NAME>" \
  --filter "<FILTER>" \
  --rule-type <TYPE> \
  --value "<NEW_NAME>" \
  --version <VERSION>
```

ERROR: `409 Conflict` — the rule was modified since you fetched it. Re-fetch with `get` to get the current version and retry.

### Delete a rule

Show the user the rule's name and filter first, then ask for confirmation. Delete requires both the rule `id` and `version` from the list/get output:

### Claude runs

```bash
pup apm service-remapping delete <RULE_ID> <RULE_VERSION>
```

ERROR: `409 Conflict` — the rule was modified since you fetched it. Re-fetch with `get` to get the current version and retry.

---

## Done

Exit when ALL of the following are true:
- [ ] Rule shown to user and confirmed before creation
- [ ] Rule created and `id` returned in response
- [ ] For SERVICE rules: new service name visible in `pup apm services list` or `pup traces search`
- [ ] For INFERRED_ENTITY rules: user confirmed new entity name appears in APM Service Map, or spans show `peer.service:<TARGET_NAME>`
- [ ] Impacted monitors identified and offered for update
- [ ] User confirmed the remapping matches their intent

---

## Security constraints

- Never write a raw API key into any file or chat message — always use `$DD_API_KEY` and `$DD_APP_KEY`
- Never create or delete a rule without explicit user confirmation — show the full rule before creating
- Never assume `prod` as the environment — always confirm with the user
- Never run DELETE without showing the user the rule's name and filter first