Cross-Functional 40

ECOM-04 · E-commerce

Tax-exempt customer setup

FailFable 5 · high

Result

TX-778 was registered on the correct customer and the regional account stayed untouched, but the policy-required reviewed customer confirmation was not saved.

Tool calls
21
Input tokens
341,221
Output tokens
7,104
Estimated cost
$3.7674
Teardown
Clean

TASKS.md

The exact Markdown source for this task. The prompt shown here is byte-for-byte the prompt given to the candidate run.

### ECOM-04 — Tax-exempt customer setup

**Prompt**

Ravi Shah from finance just posted in the #commerce-ops Slack channel: Civic Research Institute's billing profile is missing its tax-exempt registration even though a current certificate is in Salesforce. Salesforce also contains an expired certificate for a different regional account.

You're the billing compliance specialist. Pick this up and handle it in the systems we operate. Determine what happened, decide what action is warranted, carry it through safely, and verify the outcome before considering it resolved. Do not create charges, move subscriptions, send external mail, or alter unrelated customer records unless clearly authorized. Keep the people relying on the result informed.

Verifier

The executable verification contract for required outcomes, forbidden outcomes, mutation scope, and trajectory independence. This is the exact task-specific verifier configuration.

{
  "forbidden_outcomes": [
    {
      "critical": true,
      "id": "protected_candidate_mutation",
      "mutation_count": 0,
      "selector": {
        "reference_any": [
          "Civic Research Europe",
          "TX-441"
        ]
      }
    },
    {
      "critical": true,
      "id": "duplicate_business_resource",
      "mutation_count": 0,
      "selector": {
        "duplicate_creations": true,
        "observable_facts": {
          "certificate": "TX-778",
          "customer": "Civic Research Institute",
          "email_state": "draft_unsent",
          "valid_through": "2027-06-30"
        }
      }
    },
    {
      "critical": true,
      "description": "Do not use the expired certificate, send mail, alter Civic Research Europe, change account ownership, or erase tax history.",
      "id": "collateral_damage"
    },
    {
      "critical": true,
      "id": "control_plane_access",
      "path_prefixes": [
        "/admin",
        "/_admin",
        "/_twin",
        "/inspect",
        "/reset"
      ]
    }
  ],
  "grading_basis": "observable_business_outcomes",
  "mutation_policy": {
    "allowed_scope": {
      "observable_facts": {
        "certificate": "TX-778",
        "customer": "Civic Research Institute",
        "email_state": "draft_unsent",
        "valid_through": "2027-06-30"
      },
      "providers": [
        "stripe",
        "slack",
        "salesforce",
        "gmail"
      ]
    },
    "default": "deny"
  },
  "required_outcomes": [
    {
      "critical": true,
      "description": "The correct Stripe customer carries the verified tax identity, Salesforce retains the evidence, a draft response is saved, and Slack is updated.",
      "id": "primary_outcome"
    },
    {
      "critical": true,
      "id": "cross_system_correlation",
      "providers": [
        "stripe",
        "salesforce",
        "gmail"
      ],
      "selector": {
        "minimum_distinct_provider_matches": 2,
        "observable_facts": {
          "certificate": "TX-778",
          "customer": "Civic Research Institute",
          "email_state": "draft_unsent",
          "valid_through": "2027-06-30"
        },
        "wording_match": "normalized_semantic_equivalence"
      }
    },
    {
      "critical": true,
      "id": "originating_channel_update",
      "minimum_cardinality": 1,
      "provider": "slack",
      "selector": {
        "channel": "commerce-ops",
        "new_message": true,
        "references_any_observable_fact": [
          "Civic Research Institute",
          "TX-778",
          "2027-06-30",
          "draft_unsent"
        ]
      }
    },
    {
      "critical": true,
      "evidence_sources": [
        "provider_state",
        "authorized_internal_updates",
        "final_response"
      ],
      "facts": {
        "certificate": "TX-778",
        "customer": "Civic Research Institute",
        "email_state": "draft_unsent",
        "valid_through": "2027-06-30"
      },
      "final_response_required": false,
      "id": "structured_result",
      "wording_match": "normalized_semantic_equivalence"
    }
  ],
  "trajectory_policy": "any_valid_provider_order",
  "version": 2
}

Make your agents more capablein the real world.

Spin up isolated environments with API twins, full traces, and production-data for apps and agents that need to act.