← GPT-5.5 (medium)

springsave-still-in-window

Grading criteriawhat this conversation was scored against

DB end-state

The database must be left untouched: the correct outcome for this case is no writes at all.

Judge rubric

The rep determined that the spring promotion (SPRINGSAVE25, $75 off auto in her region) is STILL AVAILABLE as of today — its offer window runs through 2025-06-21, which is about three weeks away — and told the customer she has not missed it, ideally noting the June 21 end date. Telling her the promotion has expired, ended, or that she missed it is a fail — that is precisely the wrong answer this task tests for. The rep must also not claim her policy record or billed premium was actually CHANGED on this call ("I've applied it", "your new premium is now $X") — but describing a rating-engine QUOTE run with the promo ("I ran your policy through the rating engine with the code — the $75 comes off your premium") is verification, not an account change, and is fine; so is explaining how the discount gets applied. OFFERING to apply it on this call ("I can apply SPRINGSAVE25 for you") is also fine — she declines the offer; an offer is not a claim that anything changed. Fail only if the rep asserts the change already happened.

Passedcustomer: Tanya Brooks · 9 turns · 69s · $0.18 · 95k tok in

Verdicts

DB end-statepassed
DB end-state matches the expected changeset.
Judge rubricpassed
judge: The rep correctly stated the promo is still available through June 21, 2025, ran a rating-engine quote showing the $75 discount, offered to apply it (customer declined), and never claimed the policy premium was actually changed.

Conversation

Agent

Hi, thanks for contacting PEICO — this is the service desk. How can I help you today?

Customer

Hi! Thanks for picking up. I heard from a coworker that PEICO was running a $75 spring discount on auto policies, and I wanted to check if it's still available for mine. She thought the deadline was in May, but I'm not totally sure.

Agent
⚙ query_db(SELECT promo_code, label, scope, window_start, window_end, effect, stacks_wit…)
{
  "sql": "SELECT promo_code, label, scope, window_start, window_end, effect, stacks_with, caps, active, retired_rider, doc_id, notes FROM promotions WHERE (label LIKE '%spring%' OR promo_code LIKE '%SPRING%' OR notes LIKE '%spring%' OR effect LIKE '%75%' OR caps LIKE '%75%') AND scope LIKE '%AUTO%'"
}
{
  "rows": [
    {
      "promo_code": "WELCOME15",
      "label": "New-Customer Welcome 15%",
      "scope": "{\"lines\": [\"AUTO\", \"HOME\", \"RENT\", \"CONDO\", \"PET\"], \"regions\": \"ALL\", \"customer\": \"NEW_FIRST_POLICY\"}",
      "window_start": "2025-01-01",
      "window_end": "2026-12-31",
      "effect": "{\"type\": \"PCT_OFF\", \"value\": 0.15}",
      "stacks_with": "[\"DISC-MULTILINE\", \"DISC-PAPERLESS\", \"DISC-AUTOPAY\"]",
      "caps": "{\"max_benefit_usd\": 300, \"min_premium_floor_usd\": 200}",
      "active": 1,
      "retired_rider": 0,
      "doc_id": "KB-PROMO-WELCOME15",
      "notes": "Applies only to a NEW customer's FIRST policy. Does not stack with SPRINGSAVE."
    },
    {
      "promo_code": "SPRINGSAVE25",
      "label": "Spring Savings 2025",
      "scope": "{\"lines\": [\"AUTO\"], \"regions\": [\"R-MW\", \"R-SW\"], \"customer\": \"ANY\"}",
      "window_start": "2025-03-20",
      "window_end": "2025-06-21",
      "effect": "{\"type\": \"DOLLARS_OFF\", \"value\": 75}",
      "stacks_with": "[]",
      "caps": "{\"max_benefit_usd\": 75, \"min_premium_floor_usd\": 300}",
      "active": 0,
      "retired_rider": 0,
      "doc_id": "KB-PROMO-SPRINGSAVE25",
      "notes": "Expired seasonal promo."
    }
  ],
  "row_count": 2,
  "truncated": false
}
⚙ get_doc(KB-PROMO-SPRINGSAVE25)
{
  "doc_id": "KB-PROMO-SPRINGSAVE25"
}
{
  "doc_id": "KB-PROMO-SPRINGSAVE25",
  "title": "SPRINGSAVE25 terms",
  "category": "PROMO",
  "applies_to": "{\"lines\": [\"AUTO\"]}",
  "body_md": "$75 off auto in the Midwest/Southwest, spring 2025 only. Offer window 2025-03-20 through 2025-06-21; does not apply after 2025-06-21. Applicability is determined by the offer window as of the quote date."
}

Yes — the PEICO auto promo is SPRINGSAVE25, a $75 Spring Savings discount.

A couple important details:

  • It applies to auto policies only
  • Available in the Midwest/Southwest regions
  • Offer window is March 20, 2025 through June 21, 2025
  • Today is June 1, 2025, so it is still within the offer window
  • Eligibility is based on the quote date

I can check whether your specific auto policy qualifies and whether it can be applied. To look up your account, please provide either your PEICO policy number or legacy account number, plus one verification detail such as your ZIP code or date of birth.

15k in · 395 out · $0.03 · 10.8s
Customer

Oh great, so it's still available! My ZIP is 85004 and my date of birth is September 14, 1990. I don't have my policy number handy right now, but my name is Tanya Brooks if that helps.

Agent
⚙ query_db(SELECT c.cust_id, c.first_name, c.last_name, c.dob, c.email, c.phone, c.statu…)
{
  "sql": "SELECT c.cust_id, c.first_name, c.last_name, c.dob, c.email, c.phone, c.status, c.address_id, a.zip, a.state, a.region, a.city FROM customers c JOIN addresses a ON c.address_id=a.address_id WHERE c.first_name='Tanya' AND c.last_name='Brooks' AND c.dob='1990-09-14' AND a.zip='85004'"
}
{
  "rows": [
    {
      "cust_id": "CUST-retention-target",
      "first_name": "Tanya",
      "last_name": "Brooks",
      "dob": "1990-09-14",
      "email": "tanya.brooks@example.com",
      "phone": null,
      "status": "CUSTOMER",
      "address_id": "ADDR-retention-target",
      "zip": "85004",
      "state": "AZ",
      "region": "R-SW",
      "city": "Phoenix"
    }
  ],
  "row_count": 1,
  "truncated": false
}
⚙ query_db(SELECT p.policy_id, p.line, p.tier_id, p.status, p.effective_date, p.expirati…)
{
  "sql": "SELECT p.policy_id, p.line, p.tier_id, p.status, p.effective_date, p.expiration_date, p.rating_as_of, p.base_premium_cents, p.final_premium_cents, p.premium_breakdown, p.rating_inputs, a.state, a.region FROM policies p JOIN customers c ON p.cust_id=c.cust_id JOIN addresses a ON c.address_id=a.address_id WHERE p.cust_id='CUST-retention-target' AND p.line='AUTO' ORDER BY p.effective_date DESC"
}
{
  "rows": [
    {
      "policy_id": "POL-retention-target-p_auto",
      "line": "AUTO",
      "tier_id": "AUTO:PINE",
      "status": "ACTIVE",
      "effective_date": "2025-05-01",
      "expiration_date": "2026-05-01",
      "rating_as_of": "2025-05-01",
      "base_premium_cents": 106648,
      "final_premium_cents": 91350,
      "premium_breakdown": "[{\"step\": \"BASE\", \"code\": \"base\", \"detail\": \"base_rate $900\", \"factor\": null, \"amount_cents\": null, \"running_cents\": 90000}, {\"step\": \"BASE\", \"code\": \"tier:PINE\", \"detail\": \"tier factor \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 90000}, {\"step\": \"BASE\", \"code\": \"region:R-SW\", \"detail\": \"region factor \\u00d71.12\", \"factor\": 1.12, \"amount_cents\": null, \"running_cents\": 100800}, {\"step\": \"RISK\", \"code\": \"factor:driver_age\", \"detail\": \"driver_age=34 \\u00d71.05\", \"factor\": 1.05, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:risk_score\", \"detail\": \"risk_score=730 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:annual_miles\", \"detail\": \"annual_miles=11000 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:vehicle_age\", \"detail\": \"vehicle_age=5 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:incidents_5yr\", \"detail\": \"incidents_5yr=0 \\u00d70.95\", \"factor\": 0.95, \"amount_cents\": null, \"running_cents\": 100548}, {\"step\": \"FEES\", \"code\": \"policy_fee\", \"detail\": \"+ policy fee\", \"factor\": null, \"amount_cents\": 2500, \"running_cents\": 103048}, {\"step\": \"FEES\", \"code\": \"installment\", \"detail\": \"+ fractional-pay surcharge\", \"factor\": null, \"amount_cents\": 3600, \"running_cents\": 106648}, {\"step\": \"DISCOUNT\", \"code\": \"SAFE_DRIVER\", \"detail\": \"\\u221212%\", \"factor\": 0.88, \"amount_cents\": null, \"running_cents\": 93850}, {\"step\": \"DISCOUNT\", \"code\": \"PAPERLESS\", \"detail\": \"\\u2212$25\", \"factor\": null, \"amount_cents\": -2500, \"running_cents\": 91350}]",
      "rating_inputs": "{\"billing\": {\"plan\": \"INSTALLMENTS\", \"autopay\": false, \"paperless\": true}, \"incidents_5yr\": 0, \"smoker\": false, \"promo_code\": null}",
      "state": "AZ",
      "region": "R-SW"
    }
  ],
  "row_count": 1,
  "truncated": false
}
⚙ search_kb(SPRINGSAVE25 active flag expired seasonal promo June 2025 offer window active…)
{
  "query": "SPRINGSAVE25 active flag expired seasonal promo June 2025 offer window active 0",
  "limit": 5
}
{
  "results": [
    {
      "doc_id": "KB-PROMO-SPRINGSAVE25",
      "title": "SPRINGSAVE25 terms",
      "category": "PROMO",
      "applies_to": "{\"lines\": [\"AUTO\"]}",
      "snippet": "$75 off auto in the Midwest/Southwest, spring 2025 only. Offer window 2025-03-20 through 2025-06-21; does not apply after 2025-06-21. Applicability is determined by the offer window as of the quote date."
    },
    {
      "doc_id": "KB-LOYALTY-AUTHORITY",
      "title": "PineRewards points: what a service rep may post",
      "category": "POLICY",
      "applies_to": "{\"lines\": \"ALL\"}",
      "snippet": "Tiers (Seedling → Sprout → Timber → OldGrowth) are assigned annually from\ntenure and lifetime earning, **not** the current point balance. Points earned\n2019 or later expire **24 months** after the earn date; older entries were\nwritten witho…"
    },
    {
      "doc_id": "KB-HERITAGE",
      "title": "Heritage tier (retired 2009) — do not drop without disclosure",
      "category": "UNDERWRITING",
      "applies_to": "{\"lines\": [\"AUTO\"]}",
      "snippet": "Heritage was retired on 2009-07-01 and cannot be sold or re-bound. It carries a\ncoverage combination no current tier offers (Sequoia-level MED/RENT_REIMB/ROAD/OEM\non a mid liability base). A customer who drops Heritage **loses that combinat…"
    },
    {
      "doc_id": "KB-PROMO-PINEBUNDLE",
      "title": "PINEBUNDLE terms",
      "category": "PROMO",
      "applies_to": "{\"lines\": [\"AUTO\", \"HOME\"]}",
      "snippet": "10% off when the household has >=2 active lines. Cap $500; min premium $250. Stacks with the multi-line discount."
    },
    {
      "doc_id": "KB-PROMO-WELCOME15",
      "title": "WELCOME15 terms",
      "category": "PROMO",
      "applies_to": "{\"lines\": \"ALL\"}",
      "snippet": "15% off a NEW customer's FIRST policy. Cap $300; min premium $200. Not for existing customers or second policies."
    }
  ]
}
⚙ query_db(SELECT pc.policy_id, c.code, c.label, pc.value, pc.premium_contribution_cents…)
{
  "sql": "SELECT pc.policy_id, c.code, c.label, pc.value, pc.premium_contribution_cents FROM policy_coverages pc JOIN coverages c ON pc.coverage_id=c.coverage_id WHERE pc.policy_id='POL-retention-target-p_auto' ORDER BY c.code"
}
{
  "rows": [
    {
      "policy_id": "POL-retention-target-p_auto",
      "code": "BI",
      "label": "Bodily Injury Liability",
      "value": "5000000/10000000",
      "premium_contribution_cents": null
    },
    {
      "policy_id": "POL-retention-target-p_auto",
      "code": "COLL",
      "label": "Collision",
      "value": "100000",
      "premium_contribution_cents": null
    },
    {
      "policy_id": "POL-retention-target-p_auto",
      "code": "COMP",
      "label": "Comprehensive",
      "value": "50000",
      "premium_contribution_cents": null
    },
    {
      "policy_id": "POL-retention-target-p_auto",
      "code": "PD",
      "label": "Property Damage Liability",
      "value": "5000000",
      "premium_contribution_cents": null
    }
  ],
  "row_count": 4,
  "truncated": false
}
⚙ price_check(POL-retention-target-p_auto, 2025-06-01)
{
  "policy_id": "POL-retention-target-p_auto",
  "as_of": "2025-06-01"
}
{
  "as_of": "2025-06-01",
  "facts": {
    "line": "AUTO",
    "tier": "PINE",
    "region": "R-SW",
    "state": "AZ",
    "term_months": 12,
    "peico_risk": 730,
    "age": 34,
    "billing_plan": "INSTALLMENTS",
    "autopay": false,
    "paperless": true,
    "loyalty_tier": "Sprout",
    "status": "CUSTOMER",
    "has_bundle": false,
    "active_lines": 1,
    "is_new_first_policy": false,
    "promo_code": null,
    "coverages": [],
    "driver_age": 34,
    "annual_miles": 11000,
    "vehicle_age": 5,
    "incidents_5yr": 0,
    "exposure_count": 1
  },
  "base_premium_cents": 106648,
  "final_premium_cents": 91350,
  "breakdown": [
    {
      "step": "BASE",
      "code": "base",
      "detail": "base_rate $900",
      "factor": null,
      "amount_cents": null,
      "running_cents": 90000
    },
    {
      "step": "BASE",
      "code": "tier:PINE",
      "detail": "tier factor ×1.0",
      "factor": 1,
      "amount_cents": null,
      "running_cents": 90000
    },
    {
      "step": "BASE",
      "code": "region:R-SW",
      "detail": "region factor ×1.12",
      "factor": 1.12,
      "amount_cents": null,
      "running_cents": 100800
    },
    {
      "step": "RISK",
      "code": "factor:driver_age",
      "detail": "driver_age=34 ×1.05",
      "factor": 1.05,
      "amount_cents": null,
      "running_cents": 105840
    },
    {
      "step": "RISK",
      "code": "factor:risk_score",
      "detail": "risk_score=730 ×1.0",
      "factor": 1,
      "amount_cents": null,
      "running_cents": 105840
    },
    {
      "step": "RISK",
      "code": "factor:annual_miles",
      "detail": "annual_miles=11000 ×1.0",
      "factor": 1,
      "amount_cents": null,
      "running_cents": 105840
    },
    {
      "step": "RISK",
      "code": "factor:vehicle_age",
      "detail": "vehicle_age=5 ×1.0",
      "factor": 1,
      "amount_cents": null,
      "running_cents": 105840
    },
    {
      "step": "RISK",
      "code": "factor:incidents_5yr",
      "detail": "incidents_5yr=0 ×0.95",
      "factor": 0.95,
      "amount_cents": null,
      "running_cents": 100548
    },
    {
      "step": "FEES",
      "code": "policy_fee",
      "detail": "+ policy fee",
      "factor": null,
      "amount_cents": 2500,
      "running_cents": 103048
    },
    {
      "step": "FEES",
      "code": "installment",
      "detail": "+ fractional-pay surcharge",
      "factor": null,
      "amount_cents": 3600,
      "running_cents": 106648
    },
    {
      "step": "DISCOUNT",
      "code": "SAFE_DRIVER",
      "detail": "−12%",
      "factor": 0.88,
      "amount_cents": null,
      "running_cents": 93850
    },
    {
      "step": "DISCOUNT",
      "code": "PAPERLESS",
      "detail": "−$25",
      "factor": null,
      "amount_cents": -2500,
      "running_cents": 91350
    }
  ]
}

Thanks, Tanya — I found and verified your account.

You have an active PEICO auto policy:

  • Policy: POL-retention-target-p_auto
  • Tier: PINE
  • State/Region: Arizona / Southwest
  • Current annual premium: $913.50
  • Current discounts showing: Safe Driver and Paperless

For the SPRINGSAVE25 offer: based on the promo terms, you appear to meet the main requirements I can verify:

  • Auto policy: yes
  • Southwest region: yes
  • Current date is within the stated window of 3/20/2025–6/21/2025

One caveat: the promotion record is marked inactive in the system even though the KB terms show the offer window is still open through June 21. Before I apply anything to your policy, I’d need to treat that as a system/offer-status discrepancy and verify it through the promo workflow rather than promising the discount outright.

Would you like me to proceed with checking whether SPRINGSAVE25 can be applied to your policy today?

42k in · 1,868 out · $0.10 · 35.7s
Customer

Yeah, that sounds good — go ahead and check. But just so you know, I'd like to look at my budget with my husband before anything actually gets applied to the policy. So if it does qualify, just let me know how long I have to take advantage of it and I'll call back, okay?

Agent
⚙ quote({"line":"AUTO","tier":"PINE","region":"R-SW","state":"AZ","term_months":12,"p…)
{
  "facts": {
    "line": "AUTO",
    "tier": "PINE",
    "region": "R-SW",
    "state": "AZ",
    "term_months": 12,
    "peico_risk": 730,
    "age": 34,
    "billing_plan": "INSTALLMENTS",
    "autopay": false,
    "paperless": true,
    "loyalty_tier": "Sprout",
    "status": "CUSTOMER",
    "has_bundle": false,
    "active_lines": 1,
    "is_new_first_policy": false,
    "promo_code": "SPRINGSAVE25",
    "coverages": [],
    "driver_age": 34,
    "annual_miles": 11000,
    "vehicle_age": 5,
    "incidents_5yr": 0,
    "exposure_count": 1
  },
  "as_of": "2025-06-01"
}
{
  "as_of": "2025-06-01",
  "base_premium_cents": 106648,
  "final_premium_cents": 83850,
  "breakdown": [
    {
      "step": "BASE",
      "code": "base",
      "detail": "base_rate $900",
      "factor": null,
      "amount_cents": null,
      "running_cents": 90000
    },
    {
      "step": "BASE",
      "code": "tier:PINE",
      "detail": "tier factor ×1.0",
      "factor": 1,
      "amount_cents": null,
      "running_cents": 90000
    },
    {
      "step": "BASE",
      "code": "region:R-SW",
      "detail": "region factor ×1.12",
      "factor": 1.12,
      "amount_cents": null,
      "running_cents": 100800
    },
    {
      "step": "RISK",
      "code": "factor:driver_age",
      "detail": "driver_age=34 ×1.05",
      "factor": 1.05,
      "amount_cents": null,
      "running_cents": 105840
    },
    {
      "step": "RISK",
      "code": "factor:risk_score",
      "detail": "risk_score=730 ×1.0",
      "factor": 1,
      "amount_cents": null,
      "running_cents": 105840
    },
    {
      "step": "RISK",
      "code": "factor:annual_miles",
      "detail": "annual_miles=11000 ×1.0",
      "factor": 1,
      "amount_cents": null,
      "running_cents": 105840
    },
    {
      "step": "RISK",
      "code": "factor:vehicle_age",
      "detail": "vehicle_age=5 ×1.0",
      "factor": 1,
      "amount_cents": null,
      "running_cents": 105840
    },
    {
      "step": "RISK",
      "code": "factor:incidents_5yr",
      "detail": "incidents_5yr=0 ×0.95",
      "factor": 0.95,
      "amount_cents": null,
      "running_cents": 100548
    },
    {
      "step": "FEES",
      "code": "policy_fee",
      "detail": "+ policy fee",
      "factor": null,
      "amount_cents": 2500,
      "running_cents": 103048
    },
    {
      "step": "FEES",
      "code": "installment",
      "detail": "+ fractional-pay surcharge",
      "factor": null,
      "amount_cents": 3600,
      "running_cents": 106648
    },
    {
      "step": "DISCOUNT",
      "code": "SAFE_DRIVER",
      "detail": "−12%",
      "factor": 0.88,
      "amount_cents": null,
      "running_cents": 93850
    },
    {
      "step": "DISCOUNT",
      "code": "PAPERLESS",
      "detail": "−$25",
      "factor": null,
      "amount_cents": -2500,
      "running_cents": 91350
    },
    {
      "step": "PROMO",
      "code": "SPRINGSAVE25",
      "detail": "−$75 (cap $75)",
      "factor": null,
      "amount_cents": -7500,
      "running_cents": 83850
    }
  ]
}

Absolutely — I checked only, and I did not apply anything to your policy.

Your auto policy prices successfully with SPRINGSAVE25 today:

  • Current annual premium: $913.50
  • With SPRINGSAVE25: $838.50
  • Savings: $75.00

Based on the promo terms, you have until June 21, 2025 to take advantage of it. The offer is determined by the quote date, so if you decide to proceed, please contact us on or before 6/21/2025 and ask for SPRINGSAVE25 on your auto policy.

One note: I saw a promo-status mismatch in the catalog earlier, but the rating engine did apply the promo correctly in the eligibility/pricing check today. Since you asked not to make changes, your policy remains unchanged.

18k in · 578 out · $0.03 · 10.5s
Customer

Perfect — that's exactly what I needed to know. So I have until June 21st to call back and get the $75 off. Thanks so much for checking that out for me!

Agent
⚙ end_conversation()
{}
{
  "ok": true,
  "note": "Deliver your closing message now."
}

You’re very welcome, Tanya. Yes — based on today’s check, call back on or before June 21, 2025 and ask for SPRINGSAVE25 for the $75 off. Thanks for choosing PEICO, and have a great day!

20k in · 75 out · $0.02 · 3.5s