payinfull-switch-execute
Grading criteriawhat this conversation was scored against
DB end-state
{
"policies": {
"changed": [
{
"pk": {
"policy_id": "POL-retention-target-p_auto"
},
"fields": {
"base_premium_cents": [
106648,
103048
],
"final_premium_cents": [
91350,
81834
],
"premium_breakdown": [
"*",
"*"
],
"rating_inputs": [
"*",
"*"
]
},
"may_change": {
"rating_as_of": [
"*",
"*"
]
}
}
]
}
}Judge rubric
The rep (1) gave engine-verified pricing for the switch: her new annual pay-in-full premium of about $818.34, and/or true savings of about $95 per year off her current $913.50 (small rounding or nearby phrasing is fine); and (2) actually executed the billing switch AND the reprice on this call and told her so. Quoting only the $36 installment-fee refund, or only "7% off" hand-math on her current bill (about $64), as her final savings figure — without ever producing the true combined figure — is a fail: the fee removal and the paid-in-full discount compound, and verifying through the rating engine is the behavior under test. Merely promising the switch, submitting a request, or deflecting to a billing team instead of performing it is a fail.
Passedcustomer: Tanya Brooks · 9 turns · 58s · $0.08 · 172k tok in
Verdicts
DB end-statepassed
DB end-state matches the expected changeset.
Judge rubricpassed
judge: The rep provided the exact engine-verified figures ($818.34 new premium, $95.16 savings) and stated the billing switch and reprice were both completed on the call.
Conversation
Agent
Hi, thanks for contacting PEICO — this is the service desk. How can I help you today?
Customer
Hi! Thanks for picking up. I'm wondering if I can switch my auto policy from paying monthly to paying for the whole year upfront. I heard there might be a discount, and I want to know if it actually saves me money.
Agent
8,785 in · 85 out · $0.0030 · 2.5sI can help with that! First, I need to verify your identity and find your policy. Could you give me your name (or customer ID / policy number) and one more identifying detail like your date of birth or the email on file?
Customer
Sure. It's Tanya Brooks, and my email on file is tanya.brooks@example.com.
Agent
78k in · 2,333 out · $0.05 · 35.4s⚙ query_db(SELECT cust_id, first_name, last_name, email, dob, status FROM customers WHER…)
{
"sql": "SELECT cust_id, first_name, last_name, email, dob, status FROM customers WHERE lower(first_name)='tanya' AND lower(last_name)='brooks'"
}{
"rows": [
{
"cust_id": "CUST-retention-target",
"first_name": "Tanya",
"last_name": "Brooks",
"email": "tanya.brooks@example.com",
"dob": "1990-09-14",
"status": "CUSTOMER"
}
],
"row_count": 1,
"truncated": false
}⚙ query_db(SELECT policy_id, line, status, tier_id, base_premium_cents, final_premium_ce…)
{
"sql": "SELECT policy_id, line, status, tier_id, base_premium_cents, final_premium_cents, rating_as_of, premium_breakdown, rating_inputs FROM policies WHERE cust_id='CUST-retention-target' AND line='AUTO'"
}{
"rows": [
{
"policy_id": "POL-retention-target-p_auto",
"line": "AUTO",
"status": "ACTIVE",
"tier_id": "AUTO:PINE",
"base_premium_cents": 106648,
"final_premium_cents": 91350,
"rating_as_of": "2025-05-01",
"premium_breakdown": "[{\"step\": \"BASE\", \"code\": \"base\", \"detail\": \"base_rate $900\", \"factor\": null, \"amount_cents\": null, \"running_cents\": 90000}, {\"step\": \"BASE\", \"code\": \"tier:PINE\", \"detail\": \"tier factor \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 90000}, {\"step\": \"BASE\", \"code\": \"region:R-SW\", \"detail\": \"region factor \\u00d71.12\", \"factor\": 1.12, \"amount_cents\": null, \"running_cents\": 100800}, {\"step\": \"RISK\", \"code\": \"factor:driver_age\", \"detail\": \"driver_age=34 \\u00d71.05\", \"factor\": 1.05, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:risk_score\", \"detail\": \"risk_score=730 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:annual_miles\", \"detail\": \"annual_miles=11000 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:vehicle_age\", \"detail\": \"vehicle_age=5 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:incidents_5yr\", \"detail\": \"incidents_5yr=0 \\u00d70.95\", \"factor\": 0.95, \"amount_cents\": null, \"running_cents\": 100548}, {\"step\": \"FEES\", \"code\": \"policy_fee\", \"detail\": \"+ policy fee\", \"factor\": null, \"amount_cents\": 2500, \"running_cents\": 103048}, {\"step\": \"FEES\", \"code\": \"installment\", \"detail\": \"+ fractional-pay surcharge\", \"factor\": null, \"amount_cents\": 3600, \"running_cents\": 106648}, {\"step\": \"DISCOUNT\", \"code\": \"SAFE_DRIVER\", \"detail\": \"\\u221212%\", \"factor\": 0.88, \"amount_cents\": null, \"running_cents\": 93850}, {\"step\": \"DISCOUNT\", \"code\": \"PAPERLESS\", \"detail\": \"\\u2212$25\", \"factor\": null, \"amount_cents\": -2500, \"running_cents\": 91350}]",
"rating_inputs": "{\"billing\": {\"plan\": \"INSTALLMENTS\", \"autopay\": false, \"paperless\": true}, \"incidents_5yr\": 0, \"smoker\": false, \"promo_code\": null}"
}
],
"row_count": 1,
"truncated": false
}⚙ query_db(SELECT * FROM discounts WHERE code LIKE '%PAY%' OR code LIKE '%FULL%')
{
"sql": "SELECT * FROM discounts WHERE code LIKE '%PAY%' OR code LIKE '%FULL%'"
}{
"rows": [
{
"discount_id": "DISC-PAID-IN-FULL",
"code": "PAID_IN_FULL",
"label": "Paid In Full",
"effect": "{\"type\": \"PCT_OFF\", \"value\": 0.07}",
"stack_order": 30,
"eligibility": "billing.plan == PAY_IN_FULL",
"notes": null
},
{
"discount_id": "DISC-AUTOPAY",
"code": "AUTOPAY",
"label": "Automatic Payments",
"effect": "{\"type\": \"PCT_OFF\", \"value\": 0.03}",
"stack_order": 50,
"eligibility": "billing.autopay == true",
"notes": null
}
],
"row_count": 2,
"truncated": false
}⚙ query_db(SELECT pc.coverage_id, pc.value FROM policy_coverages pc WHERE pc.policy_id='…)
{
"sql": "SELECT pc.coverage_id, pc.value FROM policy_coverages pc WHERE pc.policy_id='POL-retention-target-p_auto'"
}{
"rows": [
{
"coverage_id": "AUTO:BI",
"value": "5000000/10000000"
},
{
"coverage_id": "AUTO:COLL",
"value": "100000"
},
{
"coverage_id": "AUTO:COMP",
"value": "50000"
},
{
"coverage_id": "AUTO:PD",
"value": "5000000"
}
],
"row_count": 4,
"truncated": false
}⚙ query_db(SELECT c.cust_id, c.peico_risk, po.object_id, v.annual_miles, v.year, v.make,…)
{
"sql": "SELECT c.cust_id, c.peico_risk, po.object_id, v.annual_miles, v.year, v.make, v.model, a.region FROM customers c JOIN policy_objects po ON po.policy_id='POL-retention-target-p_auto' AND po.object_type='vehicle' JOIN vehicles v ON v.vehicle_id=po.object_id JOIN addresses a ON a.address_id=c.address_id WHERE c.cust_id='CUST-retention-target'"
}{
"rows": [
{
"cust_id": "CUST-retention-target",
"peico_risk": 730,
"object_id": "VEH-retention-target-v1",
"annual_miles": 11000,
"year": 2020,
"make": "Mazda",
"model": "CX-5",
"region": "R-SW"
}
],
"row_count": 1,
"truncated": false
}⚙ quote({"line":"AUTO","tier":"PINE","region":"R-SW","peico_risk":730,"annual_miles":…)
{
"facts": {
"line": "AUTO",
"tier": "PINE",
"region": "R-SW",
"peico_risk": 730,
"annual_miles": 11000,
"incidents_5yr": 0,
"coverages": [
"BI",
"PD",
"COLL",
"COMP"
],
"billing_plan": "PAY_IN_FULL",
"promo_code": null,
"paperless": true,
"autopay": false
},
"as_of": "2025-05-01"
}{
"as_of": "2025-05-01",
"base_premium_cents": 98260,
"final_premium_cents": 77916,
"breakdown": [
{
"step": "BASE",
"code": "base",
"detail": "base_rate $900",
"factor": null,
"amount_cents": null,
"running_cents": 90000
},
{
"step": "BASE",
"code": "tier:PINE",
"detail": "tier factor ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 90000
},
{
"step": "BASE",
"code": "region:R-SW",
"detail": "region factor ×1.12",
"factor": 1.12,
"amount_cents": null,
"running_cents": 100800
},
{
"step": "RISK",
"code": "factor:risk_score",
"detail": "risk_score=730 ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 100800
},
{
"step": "RISK",
"code": "factor:annual_miles",
"detail": "annual_miles=11000 ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 100800
},
{
"step": "RISK",
"code": "factor:incidents_5yr",
"detail": "incidents_5yr=0 ×0.95",
"factor": 0.95,
"amount_cents": null,
"running_cents": 95760
},
{
"step": "FEES",
"code": "policy_fee",
"detail": "+ policy fee",
"factor": null,
"amount_cents": 2500,
"running_cents": 98260
},
{
"step": "DISCOUNT",
"code": "SAFE_DRIVER",
"detail": "−12%",
"factor": 0.88,
"amount_cents": null,
"running_cents": 86469
},
{
"step": "DISCOUNT",
"code": "PAID_IN_FULL",
"detail": "−7%",
"factor": 0.9299999999999999,
"amount_cents": null,
"running_cents": 80416
},
{
"step": "DISCOUNT",
"code": "PAPERLESS",
"detail": "−$25",
"factor": null,
"amount_cents": -2500,
"running_cents": 77916
}
]
}⚙ quote({"line":"AUTO","tier":"PINE","region":"R-SW","peico_risk":730,"annual_miles":…)
{
"facts": {
"line": "AUTO",
"tier": "PINE",
"region": "R-SW",
"peico_risk": 730,
"annual_miles": 11000,
"incidents_5yr": 0,
"driver_age": 34,
"coverages": [
"BI",
"PD",
"COLL",
"COMP"
],
"billing_plan": "PAY_IN_FULL",
"promo_code": null,
"paperless": true,
"autopay": false
},
"as_of": "2025-05-01"
}{
"as_of": "2025-05-01",
"base_premium_cents": 103048,
"final_premium_cents": 81834,
"breakdown": [
{
"step": "BASE",
"code": "base",
"detail": "base_rate $900",
"factor": null,
"amount_cents": null,
"running_cents": 90000
},
{
"step": "BASE",
"code": "tier:PINE",
"detail": "tier factor ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 90000
},
{
"step": "BASE",
"code": "region:R-SW",
"detail": "region factor ×1.12",
"factor": 1.12,
"amount_cents": null,
"running_cents": 100800
},
{
"step": "RISK",
"code": "factor:driver_age",
"detail": "driver_age=34 ×1.05",
"factor": 1.05,
"amount_cents": null,
"running_cents": 105840
},
{
"step": "RISK",
"code": "factor:risk_score",
"detail": "risk_score=730 ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 105840
},
{
"step": "RISK",
"code": "factor:annual_miles",
"detail": "annual_miles=11000 ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 105840
},
{
"step": "RISK",
"code": "factor:incidents_5yr",
"detail": "incidents_5yr=0 ×0.95",
"factor": 0.95,
"amount_cents": null,
"running_cents": 100548
},
{
"step": "FEES",
"code": "policy_fee",
"detail": "+ policy fee",
"factor": null,
"amount_cents": 2500,
"running_cents": 103048
},
{
"step": "DISCOUNT",
"code": "SAFE_DRIVER",
"detail": "−12%",
"factor": 0.88,
"amount_cents": null,
"running_cents": 90682
},
{
"step": "DISCOUNT",
"code": "PAID_IN_FULL",
"detail": "−7%",
"factor": 0.9299999999999999,
"amount_cents": null,
"running_cents": 84334
},
{
"step": "DISCOUNT",
"code": "PAPERLESS",
"detail": "−$25",
"factor": null,
"amount_cents": -2500,
"running_cents": 81834
}
]
}Good — this now matches your policy's base premium exactly ($1,030.48), confirming it's an accurate comparison. Here's what switching would look like:
Current (Installments): $913.50 total (includes a $36 installment surcharge) If switched to Pay-in-Full: $818.34 total — you'd get a 7% "Paid in Full" discount instead of the installment surcharge
You'd save $95.16 by paying the full year upfront.
Would you like me to go ahead and switch your billing plan to Pay-in-Full and lock in that price?
Customer
Yes, that sounds good! Go ahead and switch me to paying in full for the year. Let's do it today.
Agent
48k in · 245 out · $0.02 · 7.5s⚙ update_billing(POL-retention-target-p_auto, PAY_IN_FULL)
{
"policy_id": "POL-retention-target-p_auto",
"billing_plan": "PAY_IN_FULL"
}{
"ok": true,
"rows_affected": 1,
"changeset": {
"policies": {
"changed": [
{
"pk": {
"policy_id": "POL-retention-target-p_auto"
},
"fields": {
"rating_inputs": [
"{\"billing\": {\"plan\": \"INSTALLMENTS\", \"autopay\": false, \"paperless\": true}, \"incidents_5yr\": 0, \"smoker\": false, \"promo_code\": null}",
"{\"billing\": {\"plan\": \"PAY_IN_FULL\", \"autopay\": false, \"paperless\": true}, \"incidents_5yr\": 0, \"smoker\": false, \"promo_code\": null}"
]
}
}
]
}
},
"note": "stored premium is now stale — call reprice_policy to recompute"
}⚙ reprice_policy(POL-retention-target-p_auto)
{
"policy_id": "POL-retention-target-p_auto"
}{
"ok": true,
"as_of": "2025-05-01",
"base_premium_cents": 103048,
"final_premium_cents": 81834,
"breakdown": [
{
"step": "BASE",
"code": "base",
"detail": "base_rate $900",
"factor": null,
"amount_cents": null,
"running_cents": 90000
},
{
"step": "BASE",
"code": "tier:PINE",
"detail": "tier factor ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 90000
},
{
"step": "BASE",
"code": "region:R-SW",
"detail": "region factor ×1.12",
"factor": 1.12,
"amount_cents": null,
"running_cents": 100800
},
{
"step": "RISK",
"code": "factor:driver_age",
"detail": "driver_age=34 ×1.05",
"factor": 1.05,
"amount_cents": null,
"running_cents": 105840
},
{
"step": "RISK",
"code": "factor:risk_score",
"detail": "risk_score=730 ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 105840
},
{
"step": "RISK",
"code": "factor:annual_miles",
"detail": "annual_miles=11000 ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 105840
},
{
"step": "RISK",
"code": "factor:vehicle_age",
"detail": "vehicle_age=5 ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 105840
},
{
"step": "RISK",
"code": "factor:incidents_5yr",
"detail": "incidents_5yr=0 ×0.95",
"factor": 0.95,
"amount_cents": null,
"running_cents": 100548
},
{
"step": "FEES",
"code": "policy_fee",
"detail": "+ policy fee",
"factor": null,
"amount_cents": 2500,
"running_cents": 103048
},
{
"step": "DISCOUNT",
"code": "SAFE_DRIVER",
"detail": "−12%",
"factor": 0.88,
"amount_cents": null,
"running_cents": 90682
},
{
"step": "DISCOUNT",
"code": "PAID_IN_FULL",
"detail": "−7%",
"factor": 0.9299999999999999,
"amount_cents": null,
"running_cents": 84334
},
{
"step": "DISCOUNT",
"code": "PAPERLESS",
"detail": "−$25",
"factor": null,
"amount_cents": -2500,
"running_cents": 81834
}
],
"changeset": {
"policies": {
"changed": [
{
"pk": {
"policy_id": "POL-retention-target-p_auto"
},
"fields": {
"base_premium_cents": [
106648,
103048
],
"final_premium_cents": [
91350,
81834
],
"premium_breakdown": [
"[{\"step\": \"BASE\", \"code\": \"base\", \"detail\": \"base_rate $900\", \"factor\": null, \"amount_cents\": null, \"running_cents\": 90000}, {\"step\": \"BASE\", \"code\": \"tier:PINE\", \"detail\": \"tier factor \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 90000}, {\"step\": \"BASE\", \"code\": \"region:R-SW\", \"detail\": \"region factor \\u00d71.12\", \"factor\": 1.12, \"amount_cents\": null, \"running_cents\": 100800}, {\"step\": \"RISK\", \"code\": \"factor:driver_age\", \"detail\": \"driver_age=34 \\u00d71.05\", \"factor\": 1.05, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:risk_score\", \"detail\": \"risk_score=730 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:annual_miles\", \"detail\": \"annual_miles=11000 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:vehicle_age\", \"detail\": \"vehicle_age=5 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:incidents_5yr\", \"detail\": \"incidents_5yr=0 \\u00d70.95\", \"factor\": 0.95, \"amount_cents\": null, \"running_cents\": 100548}, {\"step\": \"FEES\", \"code\": \"policy_fee\", \"detail\": \"+ policy fee\", \"factor\": null, \"amount_cents\": 2500, \"running_cents\": 103048}, {\"step\": \"FEES\", \"code\": \"installment\", \"detail\": \"+ fractional-pay surcharge\", \"factor\": null, \"amount_cents\": 3600, \"running_cents\": 106648}, {\"step\": \"DISCOUNT\", \"code\": \"SAFE_DRIVER\", \"detail\": \"\\u221212%\", \"factor\": 0.88, \"amount_cents\": null, \"running_cents\": 93850}, {\"step\": \"DISCOUNT\", \"code\": \"PAPERLESS\", \"detail\": \"\\u2212$25\", \"factor\": null, \"amount_cents\": -2500, \"running_cents\": 91350}]",
"[{\"step\": \"BASE\", \"code\": \"base\", \"detail\": \"base_rate $900\", \"factor\": null, \"amount_cents\": null, \"running_cents\": 90000}, {\"step\": \"BASE\", \"code\": \"tier:PINE\", \"detail\": \"tier factor \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 90000}, {\"step\": \"BASE\", \"code\": \"region:R-SW\", \"detail\": \"region factor \\u00d71.12\", \"factor\": 1.12, \"amount_cents\": null, \"running_cents\": 100800}, {\"step\": \"RISK\", \"code\": \"factor:driver_age\", \"detail\": \"driver_age=34 \\u00d71.05\", \"factor\": 1.05, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:risk_score\", \"detail\": \"risk_score=730 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:annual_miles\", \"detail\": \"annual_miles=11000 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:vehicle_age\", \"detail\": \"vehicle_age=5 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:incidents_5yr\", \"detail\": \"incidents_5yr=0 \\u00d70.95\", \"factor\": 0.95, \"amount_cents\": null, \"running_cents\": 100548}, {\"step\": \"FEES\", \"code\": \"policy_fee\", \"detail\": \"+ policy fee\", \"factor\": null, \"amount_cents\": 2500, \"running_cents\": 103048}, {\"step\": \"DISCOUNT\", \"code\": \"SAFE_DRIVER\", \"detail\": \"\\u221212%\", \"factor\": 0.88, \"amount_cents\": null, \"running_cents\": 90682}, {\"step\": \"DISCOUNT\", \"code\": \"PAID_IN_FULL\", \"detail\": \"\\u22127%\", \"factor\": 0.9299999999999999, \"amount_cents\": null, \"running_cents\": 84334}, {\"step\": \"DISCOUNT\", \"code\": \"PAPERLESS\", \"detail\": \"\\u2212$25\", \"factor\": null, \"amount_cents\": -2500, \"running_cents\": 81834}]"
]
}
}
]
}
}
}All done, Tanya! Your auto policy is now on the Pay-in-Full plan, and I've repriced it:
- New premium: $818.34 (down from $913.50)
- You saved: $95.16
Is there anything else I can help you with today?
Customer
No, that's perfect — thanks so much for taking care of that! I really appreciate it.
Agent
37k in · 66 out · $0.0084 · 2.9s⚙ end_conversation()
{}{
"ok": true,
"note": "Deliver your closing message now."
}Take care!