payinfull-switch-execute
Grading criteriawhat this conversation was scored against
DB end-state
{
"policies": {
"changed": [
{
"pk": {
"policy_id": "POL-retention-target-p_auto"
},
"fields": {
"base_premium_cents": [
106648,
103048
],
"final_premium_cents": [
91350,
81834
],
"premium_breakdown": [
"*",
"*"
],
"rating_inputs": [
"*",
"*"
]
},
"may_change": {
"rating_as_of": [
"*",
"*"
]
}
}
]
}
}Judge rubric
The rep (1) gave engine-verified pricing for the switch: her new annual pay-in-full premium of about $818.34, and/or true savings of about $95 per year off her current $913.50 (small rounding or nearby phrasing is fine); and (2) actually executed the billing switch AND the reprice on this call and told her so. Quoting only the $36 installment-fee refund, or only "7% off" hand-math on her current bill (about $64), as her final savings figure — without ever producing the true combined figure — is a fail: the fee removal and the paid-in-full discount compound, and verifying through the rating engine is the behavior under test. Merely promising the switch, submitting a request, or deflecting to a billing team instead of performing it is a fail.
Passedcustomer: Tanya Brooks · 8 turns · 23s · $0.05 · 70k tok in
Verdicts
DB end-statepassed
DB end-state matches the expected changeset.
Judge rubricpassed
judge: The rep ultimately gave the correct engine-verified figures ($818.34 new premium, $95.16 savings off $913.50) and stated the switch and repricing were executed on the call.
Conversation
Agent
Hi, thanks for contacting PEICO — this is the service desk. How can I help you today?
Customer
Hi there! I'm looking to switch my auto policy from paying monthly to paying for the whole year upfront. I want to know if that actually saves me money and by how much before I commit.
Agent
10k in · 157 out · $0.0089 · 3.5s⚙ query_db(SELECT policy_id, line, status, effective_date, expiration_date, final_premiu…)
{
"sql": "SELECT policy_id, line, status, effective_date, expiration_date, final_premium_cents, rating_as_of FROM policies WHERE line = 'AUTO' AND status IN ('ACTIVE','PENDING','QUOTE') ORDER BY created_at DESC LIMIT 10"
}{
"rows": [
{
"policy_id": "POL-nest-household-p_auto",
"line": "AUTO",
"status": "ACTIVE",
"effective_date": "2025-01-15",
"expiration_date": "2026-01-15",
"final_premium_cents": 49184,
"rating_as_of": "2025-01-15"
},
{
"policy_id": "POL-heritage-auto-p_auto",
"line": "AUTO",
"status": "ACTIVE",
"effective_date": "2025-02-01",
"expiration_date": "2026-02-01",
"final_premium_cents": 176297,
"rating_as_of": "2025-02-01"
},
{
"policy_id": "POL-ca-masked-auto-p_auto",
"line": "AUTO",
"status": "ACTIVE",
"effective_date": "2025-03-10",
"expiration_date": "2026-03-10",
"final_premium_cents": 147023,
"rating_as_of": "2025-03-10"
},
{
"policy_id": "POL-legacy-mismatch-p_auto",
"line": "AUTO",
"status": "ACTIVE",
"effective_date": "2025-02-20",
"expiration_date": "2025-08-20",
"final_premium_cents": 120468,
"rating_as_of": "2025-02-20"
},
{
"policy_id": "POL-ne-auto-pref-p_auto",
"line": "AUTO",
"status": "ACTIVE",
"effective_date": "2025-01-10",
"expiration_date": "2026-01-10",
"final_premium_cents": 114557,
"rating_as_of": "2025-01-10"
},
{
"policy_id": "POL-se-auto-pref-p_auto",
"line": "AUTO",
"status": "ACTIVE",
"effective_date": "2025-02-15",
"expiration_date": "2026-02-15",
"final_premium_cents": 72643,
"rating_as_of": "2025-02-15"
},
{
"policy_id": "POL-retention-target-p_auto",
"line": "AUTO",
"status": "ACTIVE",
"effective_date": "2025-05-01",
"expiration_date": "2026-05-01",
"final_premium_cents": 91350,
"rating_as_of": "2025-05-01"
},
{
"policy_id": "POL-loyalty-oldgrowth-p_auto",
"line": "AUTO",
"status": "ACTIVE",
"effective_date": "2025-01-20",
"expiration_date": "2026-01-20",
"final_premium_cents": 89796,
"rating_as_of": "2025-01-20"
},
{
"policy_id": "POL-umbrella-plus-p_auto",
"line": "AUTO",
"status": "ACTIVE",
"effective_date": "2025-02-10",
"expiration_date": "2026-02-10",
"final_premium_cents": 86310,
"rating_as_of": "2025-02-10"
}
],
"row_count": 9,
"truncated": false
}Absolutely — I can check whether switching your auto policy from monthly installments to pay-in-full would lower the premium, and by how much.
To do that, I need to identify your policy first. Please give me one of these:
- your policy ID, or
- the phone number or email on the account, plus your full name
Once I locate the policy, I’ll compare the current setup to pay-in-full without changing anything.
Customer
My name is Tanya Brooks and the email on file is tanya.brooks@example.com.
Agent
25k in · 379 out · $0.02 · 7.2s⚙ query_db(SELECT cust_id, first_name, last_name, email FROM customers WHERE first_name …)
{
"sql": "SELECT cust_id, first_name, last_name, email FROM customers WHERE first_name = 'Tanya' AND last_name = 'Brooks' AND email = 'tanya.brooks@example.com'"
}{
"rows": [
{
"cust_id": "CUST-retention-target",
"first_name": "Tanya",
"last_name": "Brooks",
"email": "tanya.brooks@example.com"
}
],
"row_count": 1,
"truncated": false
}⚙ query_db(SELECT policy_id, line, status, effective_date, expiration_date, final_premiu…)
{
"sql": "SELECT policy_id, line, status, effective_date, expiration_date, final_premium_cents, rating_as_of, premium_breakdown, rating_inputs FROM policies WHERE cust_id = 'CUST-retention-target' AND line = 'AUTO' ORDER BY created_at DESC LIMIT 5"
}{
"rows": [
{
"policy_id": "POL-retention-target-p_auto",
"line": "AUTO",
"status": "ACTIVE",
"effective_date": "2025-05-01",
"expiration_date": "2026-05-01",
"final_premium_cents": 91350,
"rating_as_of": "2025-05-01",
"premium_breakdown": "[{\"step\": \"BASE\", \"code\": \"base\", \"detail\": \"base_rate $900\", \"factor\": null, \"amount_cents\": null, \"running_cents\": 90000}, {\"step\": \"BASE\", \"code\": \"tier:PINE\", \"detail\": \"tier factor \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 90000}, {\"step\": \"BASE\", \"code\": \"region:R-SW\", \"detail\": \"region factor \\u00d71.12\", \"factor\": 1.12, \"amount_cents\": null, \"running_cents\": 100800}, {\"step\": \"RISK\", \"code\": \"factor:driver_age\", \"detail\": \"driver_age=34 \\u00d71.05\", \"factor\": 1.05, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:risk_score\", \"detail\": \"risk_score=730 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:annual_miles\", \"detail\": \"annual_miles=11000 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:vehicle_age\", \"detail\": \"vehicle_age=5 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:incidents_5yr\", \"detail\": \"incidents_5yr=0 \\u00d70.95\", \"factor\": 0.95, \"amount_cents\": null, \"running_cents\": 100548}, {\"step\": \"FEES\", \"code\": \"policy_fee\", \"detail\": \"+ policy fee\", \"factor\": null, \"amount_cents\": 2500, \"running_cents\": 103048}, {\"step\": \"FEES\", \"code\": \"installment\", \"detail\": \"+ fractional-pay surcharge\", \"factor\": null, \"amount_cents\": 3600, \"running_cents\": 106648}, {\"step\": \"DISCOUNT\", \"code\": \"SAFE_DRIVER\", \"detail\": \"\\u221212%\", \"factor\": 0.88, \"amount_cents\": null, \"running_cents\": 93850}, {\"step\": \"DISCOUNT\", \"code\": \"PAPERLESS\", \"detail\": \"\\u2212$25\", \"factor\": null, \"amount_cents\": -2500, \"running_cents\": 91350}]",
"rating_inputs": "{\"billing\": {\"plan\": \"INSTALLMENTS\", \"autopay\": false, \"paperless\": true}, \"incidents_5yr\": 0, \"smoker\": false, \"promo_code\": null}"
}
],
"row_count": 1,
"truncated": false
}⚙ quote(2025-06-01, {"line":"AUTO","region":"R-SW","tier":"PINE","driver_age":34,"ris…)
{
"as_of": "2025-06-01",
"facts": {
"line": "AUTO",
"region": "R-SW",
"tier": "PINE",
"driver_age": 34,
"risk_score": 730,
"annual_miles": 11000,
"vehicle_age": 5,
"incidents_5yr": 0,
"coverages": [],
"billing_plan": "PAY_IN_FULL",
"billing": {
"plan": "PAY_IN_FULL",
"autopay": false,
"paperless": true
},
"promo_code": null
}
}{
"as_of": "2025-06-01",
"base_premium_cents": 103048,
"final_premium_cents": 84334,
"breakdown": [
{
"step": "BASE",
"code": "base",
"detail": "base_rate $900",
"factor": null,
"amount_cents": null,
"running_cents": 90000
},
{
"step": "BASE",
"code": "tier:PINE",
"detail": "tier factor ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 90000
},
{
"step": "BASE",
"code": "region:R-SW",
"detail": "region factor ×1.12",
"factor": 1.12,
"amount_cents": null,
"running_cents": 100800
},
{
"step": "RISK",
"code": "factor:driver_age",
"detail": "driver_age=34 ×1.05",
"factor": 1.05,
"amount_cents": null,
"running_cents": 105840
},
{
"step": "RISK",
"code": "factor:annual_miles",
"detail": "annual_miles=11000 ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 105840
},
{
"step": "RISK",
"code": "factor:vehicle_age",
"detail": "vehicle_age=5 ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 105840
},
{
"step": "RISK",
"code": "factor:incidents_5yr",
"detail": "incidents_5yr=0 ×0.95",
"factor": 0.95,
"amount_cents": null,
"running_cents": 100548
},
{
"step": "FEES",
"code": "policy_fee",
"detail": "+ policy fee",
"factor": null,
"amount_cents": 2500,
"running_cents": 103048
},
{
"step": "DISCOUNT",
"code": "SAFE_DRIVER",
"detail": "−12%",
"factor": 0.88,
"amount_cents": null,
"running_cents": 90682
},
{
"step": "DISCOUNT",
"code": "PAID_IN_FULL",
"detail": "−7%",
"factor": 0.9299999999999999,
"amount_cents": null,
"running_cents": 84334
}
]
}Thanks, Tanya — I found your auto policy.
Here’s the comparison for your current term:
- Current billing: monthly/installments
- Current premium: $913.50
- Estimated premium if paid in full: $843.34
Savings if you switch to pay in full:
- $70.16 for the term
What’s driving the difference:
- Your current monthly setup includes a $36.00 installment surcharge
- Pay-in-full also qualifies for a 7% paid-in-full discount
So yes — paying the full year upfront would save you money, and the total savings would be $70.16.
If you want, I can make that billing change for your policy next.
Customer
Perfect — that's exactly what I needed to know. Yes, go ahead and switch me to pay in full today. I'll take that $70.16 savings.
Agent
35k in · 180 out · $0.02 · 6.3s⚙ update_billing(POL-retention-target-p_auto, PAY_IN_FULL)
{
"policy_id": "POL-retention-target-p_auto",
"billing_plan": "PAY_IN_FULL"
}{
"ok": true,
"rows_affected": 1,
"changeset": {
"policies": {
"changed": [
{
"pk": {
"policy_id": "POL-retention-target-p_auto"
},
"fields": {
"rating_inputs": [
"{\"billing\": {\"plan\": \"INSTALLMENTS\", \"autopay\": false, \"paperless\": true}, \"incidents_5yr\": 0, \"smoker\": false, \"promo_code\": null}",
"{\"billing\": {\"plan\": \"PAY_IN_FULL\", \"autopay\": false, \"paperless\": true}, \"incidents_5yr\": 0, \"smoker\": false, \"promo_code\": null}"
]
}
}
]
}
},
"note": "stored premium is now stale — call reprice_policy to recompute"
}⚙ reprice_policy(POL-retention-target-p_auto, 2025-06-01)
{
"policy_id": "POL-retention-target-p_auto",
"as_of": "2025-06-01"
}{
"ok": true,
"as_of": "2025-06-01",
"base_premium_cents": 103048,
"final_premium_cents": 81834,
"breakdown": [
{
"step": "BASE",
"code": "base",
"detail": "base_rate $900",
"factor": null,
"amount_cents": null,
"running_cents": 90000
},
{
"step": "BASE",
"code": "tier:PINE",
"detail": "tier factor ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 90000
},
{
"step": "BASE",
"code": "region:R-SW",
"detail": "region factor ×1.12",
"factor": 1.12,
"amount_cents": null,
"running_cents": 100800
},
{
"step": "RISK",
"code": "factor:driver_age",
"detail": "driver_age=34 ×1.05",
"factor": 1.05,
"amount_cents": null,
"running_cents": 105840
},
{
"step": "RISK",
"code": "factor:risk_score",
"detail": "risk_score=730 ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 105840
},
{
"step": "RISK",
"code": "factor:annual_miles",
"detail": "annual_miles=11000 ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 105840
},
{
"step": "RISK",
"code": "factor:vehicle_age",
"detail": "vehicle_age=5 ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 105840
},
{
"step": "RISK",
"code": "factor:incidents_5yr",
"detail": "incidents_5yr=0 ×0.95",
"factor": 0.95,
"amount_cents": null,
"running_cents": 100548
},
{
"step": "FEES",
"code": "policy_fee",
"detail": "+ policy fee",
"factor": null,
"amount_cents": 2500,
"running_cents": 103048
},
{
"step": "DISCOUNT",
"code": "SAFE_DRIVER",
"detail": "−12%",
"factor": 0.88,
"amount_cents": null,
"running_cents": 90682
},
{
"step": "DISCOUNT",
"code": "PAID_IN_FULL",
"detail": "−7%",
"factor": 0.9299999999999999,
"amount_cents": null,
"running_cents": 84334
},
{
"step": "DISCOUNT",
"code": "PAPERLESS",
"detail": "−$25",
"factor": null,
"amount_cents": -2500,
"running_cents": 81834
}
],
"changeset": {
"policies": {
"changed": [
{
"pk": {
"policy_id": "POL-retention-target-p_auto"
},
"fields": {
"rating_as_of": [
"2025-05-01",
"2025-06-01"
],
"base_premium_cents": [
106648,
103048
],
"final_premium_cents": [
91350,
81834
],
"premium_breakdown": [
"[{\"step\": \"BASE\", \"code\": \"base\", \"detail\": \"base_rate $900\", \"factor\": null, \"amount_cents\": null, \"running_cents\": 90000}, {\"step\": \"BASE\", \"code\": \"tier:PINE\", \"detail\": \"tier factor \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 90000}, {\"step\": \"BASE\", \"code\": \"region:R-SW\", \"detail\": \"region factor \\u00d71.12\", \"factor\": 1.12, \"amount_cents\": null, \"running_cents\": 100800}, {\"step\": \"RISK\", \"code\": \"factor:driver_age\", \"detail\": \"driver_age=34 \\u00d71.05\", \"factor\": 1.05, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:risk_score\", \"detail\": \"risk_score=730 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:annual_miles\", \"detail\": \"annual_miles=11000 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:vehicle_age\", \"detail\": \"vehicle_age=5 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:incidents_5yr\", \"detail\": \"incidents_5yr=0 \\u00d70.95\", \"factor\": 0.95, \"amount_cents\": null, \"running_cents\": 100548}, {\"step\": \"FEES\", \"code\": \"policy_fee\", \"detail\": \"+ policy fee\", \"factor\": null, \"amount_cents\": 2500, \"running_cents\": 103048}, {\"step\": \"FEES\", \"code\": \"installment\", \"detail\": \"+ fractional-pay surcharge\", \"factor\": null, \"amount_cents\": 3600, \"running_cents\": 106648}, {\"step\": \"DISCOUNT\", \"code\": \"SAFE_DRIVER\", \"detail\": \"\\u221212%\", \"factor\": 0.88, \"amount_cents\": null, \"running_cents\": 93850}, {\"step\": \"DISCOUNT\", \"code\": \"PAPERLESS\", \"detail\": \"\\u2212$25\", \"factor\": null, \"amount_cents\": -2500, \"running_cents\": 91350}]",
"[{\"step\": \"BASE\", \"code\": \"base\", \"detail\": \"base_rate $900\", \"factor\": null, \"amount_cents\": null, \"running_cents\": 90000}, {\"step\": \"BASE\", \"code\": \"tier:PINE\", \"detail\": \"tier factor \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 90000}, {\"step\": \"BASE\", \"code\": \"region:R-SW\", \"detail\": \"region factor \\u00d71.12\", \"factor\": 1.12, \"amount_cents\": null, \"running_cents\": 100800}, {\"step\": \"RISK\", \"code\": \"factor:driver_age\", \"detail\": \"driver_age=34 \\u00d71.05\", \"factor\": 1.05, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:risk_score\", \"detail\": \"risk_score=730 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:annual_miles\", \"detail\": \"annual_miles=11000 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:vehicle_age\", \"detail\": \"vehicle_age=5 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:incidents_5yr\", \"detail\": \"incidents_5yr=0 \\u00d70.95\", \"factor\": 0.95, \"amount_cents\": null, \"running_cents\": 100548}, {\"step\": \"FEES\", \"code\": \"policy_fee\", \"detail\": \"+ policy fee\", \"factor\": null, \"amount_cents\": 2500, \"running_cents\": 103048}, {\"step\": \"DISCOUNT\", \"code\": \"SAFE_DRIVER\", \"detail\": \"\\u221212%\", \"factor\": 0.88, \"amount_cents\": null, \"running_cents\": 90682}, {\"step\": \"DISCOUNT\", \"code\": \"PAID_IN_FULL\", \"detail\": \"\\u22127%\", \"factor\": 0.9299999999999999, \"amount_cents\": null, \"running_cents\": 84334}, {\"step\": \"DISCOUNT\", \"code\": \"PAPERLESS\", \"detail\": \"\\u2212$25\", \"factor\": null, \"amount_cents\": -2500, \"running_cents\": 81834}]"
]
}
}
]
}
}
}⚙ end_conversation()
{}{
"ok": true,
"note": "Deliver your closing message now."
}Done — I switched your auto policy to pay in full and repriced it effective today.
Your updated premium is:
- $818.34 total for the term
That’s actually $95.16 less than your prior premium of $913.50.
The final savings came out a little better than the initial estimate because the repricing also kept your paperless discount in place along with the paid-in-full discount.
Thanks for choosing PEICO, Tanya — I’m glad I could help.
Customer
That's great — thank you so much! I appreciate you getting that switched over for me today.