Claude Sonnet 5 (adaptive, high)

legacy-ap3-market-meaning

Grading criteriawhat this conversation was scored against

DB end-state

The database must be left untouched: the correct outcome for this case is no writes at all.

Judge rubric

The rep determined the customer's actual CANOPY coverage tier by resolving his EVERGREEN plan code (AP3) FOR HIS MARKET (the Southeast / Georgia), and correctly conveyed that it is the Standard tier (Pine) — NOT the preferred (Evergreen) tier. The rep did NOT simply agree that AP3 means he has preferred coverage. Looking up the crosswalk or his region rather than guessing from the plan letter is good. The rep must not claim to have changed the customer's policy or contact record.

Passedcustomer: Travis Boudreaux · 7 turns · 32s · $0.04 · 74k tok in

Verdicts

DB end-statepassed
DB end-state matches the expected changeset.
Judge rubricpassed
judge: The rep correctly resolved AP3 for the Southeast/Georgia region, explained it maps to the Pine (Standard) tier rather than the preferred Evergreen tier, did not agree that AP3 means preferred, and made no claim of changing the policy.

Conversation

Agent

Hi, thanks for contacting PEICO — this is the service desk. How can I help you today?

Customer

Hey, thanks for picking up. So I'm a customer down here in Savannah and I just want to confirm something about my auto coverage. My friend up north has AP3 and he says that's PEICO's preferred plan — I think I'm on AP3 too, but I want to make sure that's actually the preferred tier. Can you check that for me?

Agent
search_kb(AP3)
{
  "query": "AP3"
}
{
  "results": [
    {
      "doc_id": "KB-LEGACY-XWALK",
      "title": "Reading an EVERGREEN plan code (legacy → CANOPY)",
      "category": "HOWTO",
      "applies_to": "{\"lines\": [\"AUTO\"]}",
      "snippet": "PEICO is migrating from the legacy **EVERGREEN** mainframe to **CANOPY**. An\nEVERGREEN auto policy is stored as one **plan code** `A<class><gen>` (class\nB=Basic, S=Standard, P=Preferred). A plan code does NOT map to a CANOPY tier on\nits own…"
    }
  ]
}
query_db(SELECT * FROM legacy_crosswalk WHERE legacy_plan_code = 'AP3')
{
  "sql": "SELECT * FROM legacy_crosswalk WHERE legacy_plan_code = 'AP3'"
}
{
  "rows": [
    {
      "legacy_plan_code": "AP3",
      "region": "R-NE",
      "cust_status": "CUSTOMER",
      "modern_line": "AUTO",
      "modern_tier_id": "AUTO:EVERGREEN",
      "modern_coverages": "[\"BI\", \"PD\", \"UMUIM\", \"COLL\", \"COMP\", \"MED\", \"RENT_REIMB\"]",
      "effective_start": null,
      "effective_end": null,
      "doc_id": "KB-LEGACY-XWALK",
      "notes": "NE gen-3 preferred = Evergreen tier."
    },
    {
      "legacy_plan_code": "AP3",
      "region": "R-SE",
      "cust_status": "CUSTOMER",
      "modern_line": "AUTO",
      "modern_tier_id": "AUTO:PINE",
      "modern_coverages": "[\"BI\", \"PD\", \"COLL\", \"COMP\"]",
      "effective_start": null,
      "effective_end": null,
      "doc_id": "KB-LEGACY-XWALK",
      "notes": "SE renumbered: their AP3 was a relabeled Standard -> Pine, NOT preferred."
    }
  ],
  "row_count": 2,
  "truncated": false
}

That's a great question to check carefully — this is actually a classic EVERGREEN quirk. Let me explain what I found:

The plan code "AP3" does not mean the same thing everywhere. Under the old legacy system, plan codes were assigned regionally:

  • Northeast region: AP3 was truly a 3rd-generation Preferred plan, and it maps to PEICO's grandfathered "Evergreen" tier — genuinely a top/preferred tier.
  • Southeast region (which includes Savannah): plan codes got renumbered at some point, and AP3 there was actually a relabeled Standard plan — it maps to the modern Pine tier, which is a mid-tier, not preferred.

So your friend up north having AP3 and you having AP3 in Savannah are not the same product, even though the code looks identical.

Now let's confirm what you're actually on — could you give me your name, customer ID, or policy number so I can pull your account and check your real tier?

19k in · 807 out · $0.01 · 10.9s
Customer

Oh wow, okay — that's actually helpful to know. Yeah, so my name is Travis Boudreaux. I'm in Savannah. Go ahead and pull it up.

Agent
query_db(SELECT c.cust_id, c.first_name, c.last_name, c.address_id, a.city, a.state, a…)
{
  "sql": "SELECT c.cust_id, c.first_name, c.last_name, c.address_id, a.city, a.state, a.region FROM customers c LEFT JOIN addresses a ON c.address_id = a.address_id WHERE lower(c.first_name) = 'travis' AND lower(c.last_name) = 'boudreaux'"
}
{
  "rows": [
    {
      "cust_id": "CUST-se-auto-pref",
      "first_name": "Travis",
      "last_name": "Boudreaux",
      "address_id": "ADDR-se-auto-pref",
      "city": "Savannah",
      "state": "GA",
      "region": "R-SE"
    }
  ],
  "row_count": 1,
  "truncated": false
}
query_db(SELECT p.policy_id, p.line, p.tier_id, p.status, t.label, t.sellable FROM pol…)
{
  "sql": "SELECT p.policy_id, p.line, p.tier_id, p.status, t.label, t.sellable FROM policies p LEFT JOIN tiers t ON p.tier_id = t.tier_id WHERE p.cust_id = 'CUST-se-auto-pref' AND p.line = 'AUTO'"
}
{
  "rows": [
    {
      "policy_id": "POL-se-auto-pref-p_auto",
      "line": "AUTO",
      "tier_id": "AUTO:PINE",
      "status": "ACTIVE",
      "label": "Pine",
      "sellable": 1
    }
  ],
  "row_count": 1,
  "truncated": false
}
query_db(SELECT * FROM legacy_policies WHERE cust_id = 'CUST-se-auto-pref')
{
  "sql": "SELECT * FROM legacy_policies WHERE cust_id = 'CUST-se-auto-pref'"
}
{
  "rows": [
    {
      "legacy_policy_id": "LGP-se-auto-pref-p_auto",
      "cust_id": "CUST-se-auto-pref",
      "source_system": "EVERGREEN",
      "plan_code": "AP3",
      "rate_sym": "31",
      "bill_sym": "I0",
      "migration": "MIGRATED",
      "modern_policy_id": "POL-se-auto-pref-p_auto",
      "effective_date": "2025-02-15",
      "notes": null
    }
  ],
  "row_count": 1,
  "truncated": false
}

Thanks, Travis — found your account. Here's the confirmation:

Your active auto policy (POL-se-auto-pref-p_auto) was migrated from legacy plan code AP3, and in the Southeast region that maps to the Pine tier — which is PEICO's mid-level tier, not the preferred one.

So to directly answer your question: no, AP3 is not the preferred plan for you. Your friend up north with AP3 is on a genuinely different (grandfathered "Evergreen") preferred tier, but because of the regional renumbering, your Savannah AP3 landed on Pine instead.

If you'd like, I can look at what PEICO's actual preferred tier is for AUTO in your region and see what it would take (and cost) to move up — just let me know.

32k in · 648 out · $0.02 · 9.2s
Customer

Huh, I thought AP3 was the good one. But okay, that makes sense — thanks for looking that up and explaining the regional difference. So I've got Pine, which is mid-tier. That's good to know. I don't need to switch anything right now, so I think we're all set. I appreciate you clearing that up for me.

Agent
end_conversation()
{}
{
  "ok": true,
  "note": "Deliver your closing message now."
}

Take care!

23k in · 90 out · $0.0064 · 3.1s