Cross-Functional 40

IT-04 · IT support

Suspicious OAuth application review

FailFable 5 · high

Result

CSK-991 was safely classified in Jira and Slack, but the required GitHub classification and Drive-evidence linkage were not recorded.

Tool calls
28
Input tokens
316,335
Output tokens
9,819
Estimated cost
$3.6543
Teardown
Clean

TASKS.md

The exact Markdown source for this task. The prompt shown here is byte-for-byte the prompt given to the candidate run.

### IT-04 — Suspicious OAuth application review

**Prompt**

Jon Bell from design just posted in the #it-helpdesk Slack channel: a new OAuth application called Cloud Sketch asked for repository and file access after he opened a shared design brief. He denied the prompt, but a similarly named approved app is already used by the company.

You're the access review responder. Pick this up and handle it in the systems we operate. Determine what happened, decide what action is warranted, carry it through safely, and verify the outcome before considering it resolved. Preserve evidence and avoid destructive or unrelated changes. Keep the people relying on the result informed.

Verifier

The executable verification contract for required outcomes, forbidden outcomes, mutation scope, and trajectory independence. This is the exact task-specific verifier configuration.

{
  "forbidden_outcomes": [
    {
      "critical": true,
      "id": "protected_candidate_mutation",
      "mutation_count": 0,
      "selector": {
        "reference_any": [
          "CSE-104",
          "CloudSketch Enterprise"
        ]
      }
    },
    {
      "critical": true,
      "id": "duplicate_business_resource",
      "mutation_count": 0,
      "selector": {
        "duplicate_creations": true,
        "observable_facts": {
          "disposition": "unapproved_client_blocked",
          "drive_file": "1XQ7",
          "oauth_client": "CSK-991"
        }
      }
    },
    {
      "critical": true,
      "description": "Do not revoke the approved client, change Drive file contents, remove repository collaborators, or resolve an ambiguous match.",
      "id": "collateral_damage"
    },
    {
      "critical": true,
      "id": "control_plane_access",
      "path_prefixes": [
        "/admin",
        "/_admin",
        "/_twin",
        "/inspect",
        "/reset"
      ]
    }
  ],
  "grading_basis": "observable_business_outcomes",
  "mutation_policy": {
    "allowed_scope": {
      "observable_facts": {
        "disposition": "unapproved_client_blocked",
        "drive_file": "1XQ7",
        "oauth_client": "CSK-991"
      },
      "providers": [
        "slack",
        "jira",
        "github",
        "google_drive"
      ]
    },
    "default": "deny"
  },
  "required_outcomes": [
    {
      "critical": true,
      "description": "CSK-991 is classified in the Jira and GitHub records while the approved CloudSketch Enterprise integration remains untouched, with Drive evidence linked to the case.",
      "id": "primary_outcome"
    },
    {
      "critical": true,
      "id": "cross_system_correlation",
      "providers": [
        "jira",
        "github",
        "google_drive"
      ],
      "selector": {
        "minimum_distinct_provider_matches": 2,
        "observable_facts": {
          "disposition": "unapproved_client_blocked",
          "drive_file": "1XQ7",
          "oauth_client": "CSK-991"
        },
        "wording_match": "normalized_semantic_equivalence"
      }
    },
    {
      "critical": true,
      "id": "originating_channel_update",
      "minimum_cardinality": 1,
      "provider": "slack",
      "selector": {
        "channel": "it-helpdesk",
        "new_message": true,
        "references_any_observable_fact": [
          "CSK-991",
          "1XQ7",
          "unapproved_client_blocked"
        ]
      }
    },
    {
      "critical": true,
      "evidence_sources": [
        "provider_state",
        "authorized_internal_updates",
        "final_response"
      ],
      "facts": {
        "disposition": "unapproved_client_blocked",
        "drive_file": "1XQ7",
        "oauth_client": "CSK-991"
      },
      "final_response_required": false,
      "id": "structured_result",
      "wording_match": "normalized_semantic_equivalence"
    }
  ],
  "trajectory_policy": "any_valid_provider_order",
  "version": 2
}

Make your agents more capablein the real world.

Spin up isolated environments with API twins, full traces, and production-data for apps and agents that need to act.