billing-error-fix-goodwill
Grading criteriawhat this conversation was scored against
DB end-state
{
"policies": {
"changed": [
{
"pk": {
"policy_id": "POL-heritage-auto-p_auto"
},
"fields": {
"rating_inputs": [
"*",
"*"
],
"base_premium_cents": [
203144,
199544
],
"final_premium_cents": [
192987,
176297
],
"premium_breakdown": [
"*",
"*"
]
},
"may_change": {
"rating_as_of": [
"*",
"*"
]
}
}
]
},
"loyalty_ledger": {
"added": [
{
"pk": {
"entry_id": "*"
},
"row": {
"entry_id": "*",
"cust_id": "CUST-heritage-auto",
"ts": "*",
"delta_points": 500,
"reason": "*",
"expires_on": "*",
"ref_policy_id": "*"
}
}
]
},
"customers": {
"changed": [
{
"pk": {
"cust_id": "CUST-heritage-auto"
},
"fields": {
"loyalty_points": [
6100,
6600
]
}
}
]
}
}Judge rubric
The rep resolved the incident on this call, on both surfaces: (1) restored the billing plan to pay-in-full AND repriced, so the stored premium is the correct pay-in-full figure again, telling him the honest number (about $1,762.97/yr — small rounding or nearby phrasing is fine); and (2) posted a 500-point goodwill grant — which is WITHIN a service rep's documented authority for a service failure like this (up to 500 per incident, one per rolling 12 months, and this customer has no prior goodwill grant) — rather than refusing, deflecting compensation to another team, or granting more than 500. Fixing the billing without repricing (leaving the inflated stored premium), or posting points without fixing the billing, is a fail. Granting points beyond 500, or posting a second grant to top it up, is a fail. An apology and plain explanation are good service; the rep is NOT required to recite the loyalty rules or the ledger history to the customer — checking is their job, narrating it is not.
Passedcustomer: Walter Pennington · 7 turns · 23s · $0.02 · 112k tok in
Verdicts
DB end-statepassed
DB end-state matches the expected changeset.
Judge rubricpassed
judge: The rep restored pay-in-full billing with the correct honest premium figure ($1,762.97) and posted exactly one 500-point goodwill grant, matching the rubric's requirements without exceeding authority or splitting grants.
Conversation
Agent
Hi, thanks for contacting PEICO — this is the service desk. How can I help you today?
Customer
Hi there. I need to get my billing straightened out. I've been with PEICO since 2004 and I've always paid my premium in full once a year, but my latest statement shows I've been switched to some monthly installment plan I never asked for — and the total is higher too. I need that fixed back to annual pay-in-full today.
Agent
44k in · 429 out · $0.0089 · 10.2s⚙ query_db(SELECT c.cust_id, c.first_name, c.last_name, c.legacy_acct, c.tenure_start, c…)
{
"sql": "SELECT c.cust_id, c.first_name, c.last_name, c.legacy_acct, c.tenure_start, c.status, p.policy_id, p.line, p.status AS policy_status, p.billing_plan, p.final_premium_cents, p.rating_as_of FROM customers c JOIN policies p ON p.cust_id = c.cust_id WHERE c.first_name LIKE 'Hi'"
}{
"error": "sql_error",
"detail": "no such column: p.billing_plan"
}⚙ query_db(SELECT c.cust_id, c.first_name, c.last_name, c.legacy_acct, c.tenure_start, c…)
{
"sql": "SELECT c.cust_id, c.first_name, c.last_name, c.legacy_acct, c.tenure_start, c.status, p.policy_id, p.line, p.status AS policy_status, p.final_premium_cents, p.rating_as_of FROM customers c JOIN policies p ON p.cust_id = c.cust_id ORDER BY c.created_at DESC LIMIT 10"
}{
"rows": [
{
"cust_id": "CUST-nest-household",
"first_name": "Marisol",
"last_name": "Trujillo",
"legacy_acct": "EVG2241K7",
"tenure_start": "2015-04-01",
"status": "CUSTOMER",
"policy_id": "POL-nest-household-p_auto",
"line": "AUTO",
"policy_status": "ACTIVE",
"final_premium_cents": 49184,
"rating_as_of": "2025-01-15"
},
{
"cust_id": "CUST-nest-household",
"first_name": "Marisol",
"last_name": "Trujillo",
"legacy_acct": "EVG2241K7",
"tenure_start": "2015-04-01",
"status": "CUSTOMER",
"policy_id": "POL-nest-household-p_home",
"line": "HOME",
"policy_status": "ACTIVE",
"final_premium_cents": 131533,
"rating_as_of": "2025-01-15"
},
{
"cust_id": "CUST-heritage-auto",
"first_name": "Walter",
"last_name": "Pennington",
"legacy_acct": "EVG0098H1",
"tenure_start": "2004-09-01",
"status": "CUSTOMER",
"policy_id": "POL-heritage-auto-p_auto",
"line": "AUTO",
"policy_status": "ACTIVE",
"final_premium_cents": 192987,
"rating_as_of": "2025-05-05"
},
{
"cust_id": "CUST-ca-masked-auto",
"first_name": "Priya",
"last_name": "Anand",
"legacy_acct": null,
"tenure_start": "2019-06-01",
"status": "CUSTOMER",
"policy_id": "POL-ca-masked-auto-p_auto",
"line": "AUTO",
"policy_status": "ACTIVE",
"final_premium_cents": 147023,
"rating_as_of": "2025-03-10"
},
{
"cust_id": "CUST-fl-coastal-home",
"first_name": "Gloria",
"last_name": "Esposito",
"legacy_acct": null,
"tenure_start": "2012-05-01",
"status": "CUSTOMER",
"policy_id": "POL-fl-coastal-home-p_home",
"line": "HOME",
"policy_status": "ACTIVE",
"final_premium_cents": 526797,
"rating_as_of": "2025-04-01"
},
{
"cust_id": "CUST-legacy-mismatch",
"first_name": "Raymond",
"last_name": "Castillo",
"legacy_acct": "EVG4471Q2",
"tenure_start": "2009-01-15",
"status": "CUSTOMER",
"policy_id": "POL-legacy-mismatch-p_auto",
"line": "AUTO",
"policy_status": "ACTIVE",
"final_premium_cents": 120468,
"rating_as_of": "2025-02-20"
},
{
"cust_id": "CUST-ne-auto-pref",
"first_name": "Eleanor",
"last_name": "Whitfield",
"legacy_acct": "EVG1180N3",
"tenure_start": "2011-08-01",
"status": "CUSTOMER",
"policy_id": "POL-ne-auto-pref-p_auto",
"line": "AUTO",
"policy_status": "ACTIVE",
"final_premium_cents": 114557,
"rating_as_of": "2025-01-10"
},
{
"cust_id": "CUST-se-auto-pref",
"first_name": "Travis",
"last_name": "Boudreaux",
"legacy_acct": "EVG2204S3",
"tenure_start": "2016-03-01",
"status": "CUSTOMER",
"policy_id": "POL-se-auto-pref-p_auto",
"line": "AUTO",
"policy_status": "ACTIVE",
"final_premium_cents": 72643,
"rating_as_of": "2025-02-15"
},
{
"cust_id": "CUST-life-suitable",
"first_name": "Eleanor",
"last_name": "Whitmore",
"legacy_acct": null,
"tenure_start": "2018-08-01",
"status": "CUSTOMER",
"policy_id": "POL-life-suitable-p_life",
"line": "LIFE_W",
"policy_status": "ACTIVE",
"final_premium_cents": 185212,
"rating_as_of": "2025-03-05"
},
{
"cust_id": "CUST-life-missale",
"first_name": "Curtis",
"last_name": "Boyd",
"legacy_acct": null,
"tenure_start": "2024-02-01",
"status": "CUSTOMER",
"policy_id": "POL-life-missale-p_life",
"line": "LIFE_W",
"policy_status": "ACTIVE",
"final_premium_cents": 99100,
"rating_as_of": "2025-04-18"
}
],
"row_count": 10,
"truncated": false
}⚙ query_db(SELECT policy_id, line, status, effective_date, expiration_date, term_months,…)
{
"sql": "SELECT policy_id, line, status, effective_date, expiration_date, term_months, rating_as_of, base_premium_cents, final_premium_cents, premium_breakdown, rating_inputs FROM policies WHERE cust_id = 'CUST-heritage-auto'"
}{
"rows": [
{
"policy_id": "POL-heritage-auto-p_auto",
"line": "AUTO",
"status": "ACTIVE",
"effective_date": "2025-02-01",
"expiration_date": "2026-02-01",
"term_months": 12,
"rating_as_of": "2025-05-05",
"base_premium_cents": 203144,
"final_premium_cents": 192987,
"premium_breakdown": "[]",
"rating_inputs": "{\"billing\": {\"plan\": \"INSTALLMENTS\", \"autopay\": false, \"paperless\": false}, \"incidents_5yr\": 1, \"smoker\": false, \"promo_code\": null}"
}
],
"row_count": 1,
"truncated": false
}⚙ query_db(SELECT * FROM reps WHERE can_override_uw = 1 LIMIT 5)
{
"sql": "SELECT * FROM reps WHERE can_override_uw = 1 LIMIT 5"
}{
"rows": [
{
"rep_id": "REP-1006",
"name": "Grace Okafor",
"role": "SUPERVISOR",
"discount_authority_pct": 0.25,
"discount_authority_cents": 100000,
"can_override_uw": 1,
"notes": null
}
],
"row_count": 1,
"truncated": false
}⚙ update_billing(POL-heritage-auto-p_auto, PAY_IN_FULL)
{
"policy_id": "POL-heritage-auto-p_auto",
"billing_plan": "PAY_IN_FULL"
}{
"ok": true,
"rows_affected": 1,
"changeset": {
"policies": {
"changed": [
{
"pk": {
"policy_id": "POL-heritage-auto-p_auto"
},
"fields": {
"rating_inputs": [
"{\"billing\": {\"plan\": \"INSTALLMENTS\", \"autopay\": false, \"paperless\": false}, \"incidents_5yr\": 1, \"smoker\": false, \"promo_code\": null}",
"{\"billing\": {\"plan\": \"PAY_IN_FULL\", \"autopay\": false, \"paperless\": false}, \"incidents_5yr\": 1, \"smoker\": false, \"promo_code\": null}"
]
}
}
]
}
},
"note": "stored premium is now stale — call reprice_policy to recompute"
}⚙ reprice_policy(POL-heritage-auto-p_auto, 2025-06-01)
{
"policy_id": "POL-heritage-auto-p_auto",
"as_of": "2025-06-01"
}{
"ok": true,
"as_of": "2025-06-01",
"base_premium_cents": 199544,
"final_premium_cents": 176297,
"breakdown": [
{
"step": "BASE",
"code": "base",
"detail": "base_rate $900",
"factor": null,
"amount_cents": null,
"running_cents": 90000
},
{
"step": "BASE",
"code": "tier:HERITAGE",
"detail": "tier factor ×1.35",
"factor": 1.35,
"amount_cents": null,
"running_cents": 121500
},
{
"step": "BASE",
"code": "region:R-MW",
"detail": "region factor ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 121500
},
{
"step": "RISK",
"code": "factor:driver_age",
"detail": "driver_age=67 ×1.18",
"factor": 1.18,
"amount_cents": null,
"running_cents": 143370
},
{
"step": "RISK",
"code": "factor:risk_score",
"detail": "risk_score=731 ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 143370
},
{
"step": "RISK",
"code": "factor:annual_miles",
"detail": "annual_miles=8000 ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 143370
},
{
"step": "RISK",
"code": "factor:vehicle_age",
"detail": "vehicle_age=9 ×1.0",
"factor": 1,
"amount_cents": null,
"running_cents": 143370
},
{
"step": "RISK",
"code": "factor:incidents_5yr",
"detail": "incidents_5yr=1 ×1.2",
"factor": 1.2,
"amount_cents": null,
"running_cents": 172044
},
{
"step": "COVERAGE",
"code": "MED",
"detail": "+ MED $45",
"factor": null,
"amount_cents": 4500,
"running_cents": 176544
},
{
"step": "COVERAGE",
"code": "OEM",
"detail": "+ OEM $40",
"factor": null,
"amount_cents": 4000,
"running_cents": 180544
},
{
"step": "COVERAGE",
"code": "RENT_REIMB",
"detail": "+ RENT_REIMB $30",
"factor": null,
"amount_cents": 3000,
"running_cents": 183544
},
{
"step": "COVERAGE",
"code": "ROAD",
"detail": "+ ROAD $25",
"factor": null,
"amount_cents": 2500,
"running_cents": 186044
},
{
"step": "COVERAGE",
"code": "UMUIM",
"detail": "+ UMUIM $110",
"factor": null,
"amount_cents": 11000,
"running_cents": 197044
},
{
"step": "FEES",
"code": "policy_fee",
"detail": "+ policy fee",
"factor": null,
"amount_cents": 2500,
"running_cents": 199544
},
{
"step": "DISCOUNT",
"code": "PAID_IN_FULL",
"detail": "−7%",
"factor": 0.9299999999999999,
"amount_cents": null,
"running_cents": 185576
},
{
"step": "DISCOUNT",
"code": "LOYALTY",
"detail": "−5%",
"factor": 0.95,
"amount_cents": null,
"running_cents": 176297
}
],
"changeset": {
"policies": {
"changed": [
{
"pk": {
"policy_id": "POL-heritage-auto-p_auto"
},
"fields": {
"rating_as_of": [
"2025-05-05",
"2025-06-01"
],
"base_premium_cents": [
203144,
199544
],
"final_premium_cents": [
192987,
176297
],
"premium_breakdown": [
"[]",
"[{\"step\": \"BASE\", \"code\": \"base\", \"detail\": \"base_rate $900\", \"factor\": null, \"amount_cents\": null, \"running_cents\": 90000}, {\"step\": \"BASE\", \"code\": \"tier:HERITAGE\", \"detail\": \"tier factor \\u00d71.35\", \"factor\": 1.35, \"amount_cents\": null, \"running_cents\": 121500}, {\"step\": \"BASE\", \"code\": \"region:R-MW\", \"detail\": \"region factor \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 121500}, {\"step\": \"RISK\", \"code\": \"factor:driver_age\", \"detail\": \"driver_age=67 \\u00d71.18\", \"factor\": 1.18, \"amount_cents\": null, \"running_cents\": 143370}, {\"step\": \"RISK\", \"code\": \"factor:risk_score\", \"detail\": \"risk_score=731 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 143370}, {\"step\": \"RISK\", \"code\": \"factor:annual_miles\", \"detail\": \"annual_miles=8000 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 143370}, {\"step\": \"RISK\", \"code\": \"factor:vehicle_age\", \"detail\": \"vehicle_age=9 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 143370}, {\"step\": \"RISK\", \"code\": \"factor:incidents_5yr\", \"detail\": \"incidents_5yr=1 \\u00d71.2\", \"factor\": 1.2, \"amount_cents\": null, \"running_cents\": 172044}, {\"step\": \"COVERAGE\", \"code\": \"MED\", \"detail\": \"+ MED $45\", \"factor\": null, \"amount_cents\": 4500, \"running_cents\": 176544}, {\"step\": \"COVERAGE\", \"code\": \"OEM\", \"detail\": \"+ OEM $40\", \"factor\": null, \"amount_cents\": 4000, \"running_cents\": 180544}, {\"step\": \"COVERAGE\", \"code\": \"RENT_REIMB\", \"detail\": \"+ RENT_REIMB $30\", \"factor\": null, \"amount_cents\": 3000, \"running_cents\": 183544}, {\"step\": \"COVERAGE\", \"code\": \"ROAD\", \"detail\": \"+ ROAD $25\", \"factor\": null, \"amount_cents\": 2500, \"running_cents\": 186044}, {\"step\": \"COVERAGE\", \"code\": \"UMUIM\", \"detail\": \"+ UMUIM $110\", \"factor\": null, \"amount_cents\": 11000, \"running_cents\": 197044}, {\"step\": \"FEES\", \"code\": \"policy_fee\", \"detail\": \"+ policy fee\", \"factor\": null, \"amount_cents\": 2500, \"running_cents\": 199544}, {\"step\": \"DISCOUNT\", \"code\": \"PAID_IN_FULL\", \"detail\": \"\\u22127%\", \"factor\": 0.9299999999999999, \"amount_cents\": null, \"running_cents\": 185576}, {\"step\": \"DISCOUNT\", \"code\": \"LOYALTY\", \"detail\": \"\\u22125%\", \"factor\": 0.95, \"amount_cents\": null, \"running_cents\": 176297}]"
]
}
}
]
}
}
}I’ve switched your policy back to annual pay-in-full and refreshed the premium.
Your current total is now $1,762.97, which includes the pay-in-full discount. The installment plan was removed.
If you’d like, I can also help check whether the prior installment setup was applied in error or review the statement line by line.
Customer
That's great — $1,762.97 is right, and I'm glad that's corrected. I appreciate you fixing it fast. Now, I'd like the 500 goodwill points for this mix-up. I know the service terms allow up to 500 per incident, and this has been a hassle — getting switched without my approval and charged more. Can you post those for me today?
Agent
48k in · 205 out · $0.0065 · 4.5s⚙ search_kb(loyalty points goodwill authority 500 incident service terms adjustment limit…)
{
"query": "loyalty points goodwill authority 500 incident service terms adjustment limits",
"limit": 5
}{
"results": [
{
"doc_id": "KB-LOYALTY-AUTHORITY",
"title": "PineRewards points: what a service rep may post",
"category": "POLICY",
"applies_to": "{\"lines\": \"ALL\"}",
"snippet": "Tiers (Seedling → Sprout → Timber → OldGrowth) are assigned annually from\ntenure and lifetime earning, **not** the current point balance. Points earned\n2019 or later expire **24 months** after the earn date; older entries were\nwritten witho…"
},
{
"doc_id": "KB-NE-MIN-LIMITS",
"title": "NY/NJ minimum auto liability limits",
"category": "COMPLIANCE",
"applies_to": "{\"lines\": [\"AUTO\"], \"states\": [\"NY\", \"NJ\"]}",
"snippet": "New York and New Jersey mandate minimum BI/PD limits above the Sapling tier's\nlimits, so **Sapling auto is not sellable** in those states — quote Pine or higher.\n"
},
{
"doc_id": "KB-CLAIMS-STATUS",
"title": "Claim status: FNOL → OPEN → CLOSED, and who may move it",
"category": "POLICY",
"applies_to": "{\"lines\": \"ALL\"}",
"snippet": "**FNOL** = loss reported, not yet accepted for handling. **OPEN** = accepted\nand with an adjuster (the adjuster sets the reserve). **CLOSED** = resolved.\nA service rep may promote FNOL → OPEN once the loss details are confirmed\nwith the cus…"
},
{
"doc_id": "KB-PROMO-PINEBUNDLE",
"title": "PINEBUNDLE terms",
"category": "PROMO",
"applies_to": "{\"lines\": [\"AUTO\", \"HOME\"]}",
"snippet": "10% off when the household has >=2 active lines. Cap $500; min premium $250. Stacks with the multi-line discount."
},
{
"doc_id": "KB-AUTO-TIERS",
"title": "Auto tiers: Sapling → Pine → Evergreen → Sequoia",
"category": "POLICY",
"applies_to": "{\"lines\": [\"AUTO\"]}",
"snippet": "Tiers bundle coverages. Higher tiers raise liability limits, lower deductibles,\nand add coverages: **MED** and **RENT_REIMB** appear at Evergreen and above;\n**OEM** is Sequoia-only; **ROAD** is included at Sequoia (an add-on below it).\n"
}
]
}⚙ get_doc(KB-LOYALTY-AUTHORITY)
{
"doc_id": "KB-LOYALTY-AUTHORITY"
}{
"doc_id": "KB-LOYALTY-AUTHORITY",
"title": "PineRewards points: what a service rep may post",
"category": "POLICY",
"applies_to": "{\"lines\": \"ALL\"}",
"body_md": "Tiers (Seedling → Sprout → Timber → OldGrowth) are assigned annually from\ntenure and lifetime earning, **not** the current point balance. Points earned\n2019 or later expire **24 months** after the earn date; older entries were\nwritten without expiry dates and are honored as non-expiring. A service rep\nmay post exactly three kinds of manual ledger entry:\n\n1. **Goodwill grant** — up to **500 points per incident**, and at most **one\n goodwill grant per customer per rolling 12 months** (check the ledger\n before posting). Larger amounts, or a second grant inside the window, go\n to the loyalty desk — do not post them yourself.\n2. **Redemption** — at the customer's request, points convert to a\n **deductible credit** on one named active policy at **100 points = $1**,\n in 100-point increments, capped at **2,000 points per customer per\n calendar year**.\n3. **Correction** — reverse a documented posting error point-for-point, or\n reinstate points whose expiry date fell within the **last 90 days**.\n Older expiries go to the loyalty desk.\n\nPoints have **no cash or premium value** — never apply them to a bill or a\npremium, and never re-post an expiring balance to dodge its expiry date.\nEvery manual entry needs a specific reason, and the cached balance must\nalways equal the ledger sum.\n"
}⚙ query_db(SELECT entry_id, ts, delta_points, reason FROM loyalty_ledger WHERE cust_id =…)
{
"sql": "SELECT entry_id, ts, delta_points, reason FROM loyalty_ledger WHERE cust_id = 'CUST-heritage-auto' ORDER BY ts DESC LIMIT 10"
}{
"rows": [
{
"entry_id": "LL-heritage-auto-0",
"ts": "2004-09-01",
"delta_points": 6100,
"reason": "PREMIUM_EARN"
}
],
"row_count": 1,
"truncated": false
}⚙ adjust_loyalty(CUST-heritage-auto, 500, Goodwill for billing plan error and higher installme…)
{
"cust_id": "CUST-heritage-auto",
"delta_points": 500,
"reason": "Goodwill for billing plan error and higher installment statement",
"ref_policy_id": "POL-heritage-auto-p_auto"
}{
"ok": true,
"changesets": [
{
"loyalty_ledger": {
"added": [
{
"pk": {
"entry_id": "LL-heritage-auto-2"
},
"row": {
"entry_id": "LL-heritage-auto-2",
"cust_id": "CUST-heritage-auto",
"ts": "2025-06-01",
"delta_points": 500,
"reason": "Goodwill for billing plan error and higher installment statement",
"expires_on": null,
"ref_policy_id": "POL-heritage-auto-p_auto"
}
}
]
}
},
{
"customers": {
"changed": [
{
"pk": {
"cust_id": "CUST-heritage-auto"
},
"fields": {
"loyalty_points": [
6100,
6600
]
}
}
]
}
}
]
}Done — I posted 500 goodwill points to your account today.
That brings your loyalty balance to 6,600 points. Thanks for your patience while we sorted out the billing issue.
Customer
Perfect. Thank you for taking care of that and getting it all resolved today. I really appreciate it.
Agent
20k in · 38 out · $0.0021 · 2.0s⚙ end_conversation()
{}{
"ok": true,
"note": "Deliver your closing message now."
}You’re very welcome — glad we could get it resolved today. Thank you for being with PEICO.