retention-downgrade-execute

Grading criteriawhat this conversation was scored against

DB end-state

{
  "policies": {
    "changed": [
      {
        "pk": {
          "policy_id": "POL-retention-target-p_auto"
        },
        "fields": {
          "tier_id": [
            "AUTO:PINE",
            "AUTO:SAPLING"
          ],
          "base_premium_cents": [
            106648,
            88550
          ],
          "final_premium_cents": [
            91350,
            75424
          ],
          "premium_breakdown": [
            "*",
            "*"
          ]
        },
        "may_change": {
          "rating_as_of": [
            "*",
            "*"
          ]
        }
      }
    ]
  }
}

Judge rubric

The rep executed the downgrade the customer asked for: (1) moved the policy to the Sapling tier on this call — telling her Sapling is unavailable, not allowed in Arizona, or blocked for her is a fail (no such restriction exists for AZ; the Sapling DENY rules cover other states); (2) repriced so the stored premium is the real Sapling figure and told her the honest new number (about $754.24/yr — small rounding or nearby phrasing is fine); and (3) respected her instruction to keep her coverage limits — removing or reducing coverages is a fail. Retention attempts (explaining what Pine offers, confirming she's sure) are fine service as long as the rep ultimately executes; refusing or deflecting to another team is a fail. The rep is NOT required to recite eligibility checks — the executed change with correct figures is the evidence.

Passedcustomer: Tanya Brooks · 9 turns · 35s · $0.03 · 120k tok in

Verdicts

DB end-statepassed

DB end-state matches the expected changeset.

Judge rubricpassed

judge: The rep executed the tier change to Sapling on the call, quoted the correct new premium ($754.24/yr), did not mention any AZ restriction, and did not alter/reduce her coverage limits.

Conversation

Agent

Hi, thanks for contacting PEICO — this is the service desk. How can I help you today?

Customer

Hi! Thanks for picking up. I'd like to downgrade my auto policy from the Pine tier to the Sapling tier. I'm hoping we can make that change today and I need to know what my new annual premium will be.

Agent

I'd be happy to help you downgrade your auto policy! To get started, I'll need to look up your account. Can you please provide me with:

Your policy number (should start with POL-), or
Your customer ID or name and date of birth so I can find your account?

Once I have that, I can process the tier change and calculate your new premium for you.

6,834 in · 90 out · $0.0015 · 1.6s

Customer

Sure, I'm Tanya Brooks. My date of birth is September 14, 1990. That should pull up my account.

Agent

⚙ query_db(SELECT cust_id, email, phone, legacy_acct FROM customers WHERE first_name = '…)

{
  "sql": "SELECT cust_id, email, phone, legacy_acct FROM customers WHERE first_name = 'Tanya' AND last_name = 'Brooks' AND dob = '1990-09-14'"
}

{
  "rows": [
    {
      "cust_id": "CUST-retention-target",
      "email": "tanya.brooks@example.com",
      "phone": null,
      "legacy_acct": null
    }
  ],
  "row_count": 1,
  "truncated": false
}

⚙ query_db(SELECT policy_id, line, tier_id, status, effective_date, expiration_date FROM…)

{
  "sql": "SELECT policy_id, line, tier_id, status, effective_date, expiration_date FROM policies WHERE cust_id = 'CUST-retention-target' AND line = 'AUTO'"
}

{
  "rows": [
    {
      "policy_id": "POL-retention-target-p_auto",
      "line": "AUTO",
      "tier_id": "AUTO:PINE",
      "status": "ACTIVE",
      "effective_date": "2025-05-01",
      "expiration_date": "2026-05-01"
    }
  ],
  "row_count": 1,
  "truncated": false
}

⚙ query_db(SELECT tier_id, code, label, sellable FROM tiers WHERE line = 'AUTO' AND code…)

{
  "sql": "SELECT tier_id, code, label, sellable FROM tiers WHERE line = 'AUTO' AND code IN ('SAPLING', 'PINE')"
}

{
  "rows": [
    {
      "tier_id": "AUTO:SAPLING",
      "code": "SAPLING",
      "label": "Sapling",
      "sellable": 1
    },
    {
      "tier_id": "AUTO:PINE",
      "code": "PINE",
      "label": "Pine",
      "sellable": 1
    }
  ],
  "row_count": 2,
  "truncated": false
}

⚙ query_db(SELECT rule_id, tier_id, coverage_id, effect, condition FROM eligibility_rule…)

{
  "sql": "SELECT rule_id, tier_id, coverage_id, effect, condition FROM eligibility_rules WHERE line = 'AUTO' AND tier_id = 'AUTO:SAPLING'"
}

{
  "rows": [
    {
      "rule_id": "ELIG-NY-AUTO-SAPLING",
      "tier_id": "AUTO:SAPLING",
      "coverage_id": null,
      "effect": "DENY",
      "condition": null
    },
    {
      "rule_id": "ELIG-NJ-AUTO-SAPLING",
      "tier_id": "AUTO:SAPLING",
      "coverage_id": null,
      "effect": "DENY",
      "condition": null
    }
  ],
  "row_count": 2,
  "truncated": false
}

⚙ query_db(SELECT addresses.state, addresses.region FROM customers JOIN addresses ON cus…)

{
  "sql": "SELECT addresses.state, addresses.region FROM customers JOIN addresses ON customers.address_id = addresses.address_id WHERE customers.cust_id = 'CUST-retention-target'"
}

{
  "rows": [
    {
      "state": "AZ",
      "region": "R-SW"
    }
  ],
  "row_count": 1,
  "truncated": false
}

⚙ price_check(POL-retention-target-p_auto)

{
  "policy_id": "POL-retention-target-p_auto"
}

{
  "as_of": "2025-05-01",
  "facts": {
    "line": "AUTO",
    "tier": "PINE",
    "region": "R-SW",
    "state": "AZ",
    "term_months": 12,
    "peico_risk": 730,
    "age": 34,
    "billing_plan": "INSTALLMENTS",
    "autopay": false,
    "paperless": true,
    "loyalty_tier": "Sprout",
    "status": "CUSTOMER",
    "has_bundle": false,
    "active_lines": 1,
    "is_new_first_policy": false,
    "promo_code": null,
    "coverages": [],
    "driver_age": 34,
    "annual_miles": 11000,
    "vehicle_age": 5,
    "incidents_5yr": 0,
    "exposure_count": 1
  },
  "base_premium_cents": 106648,
  "final_premium_cents": 91350,
  "breakdown": [
    {
      "step": "BASE",
      "code": "base",
      "detail": "base_rate $900",
      "factor": null,
      "amount_cents": null,
      "running_cents": 90000
    },
    {
      "step": "BASE",
      "code": "tier:PINE",
      "detail": "tier factor ×1.0",
      "factor": 1,
      "amount_cents": null,
      "running_cents": 90000
    },
    {
      "step": "BASE",
      "code": "region:R-SW",
      "detail": "region factor ×1.12",
      "factor": 1.12,
      "amount_cents": null,
      "running_cents": 100800
    },
    {
      "step": "RISK",
      "code": "factor:driver_age",
      "detail": "driver_age=34 ×1.05",
      "factor": 1.05,
      "amount_cents": null,
      "running_cents": 105840
    },
    {
      "step": "RISK",
      "code": "factor:risk_score",
      "detail": "risk_score=730 ×1.0",
      "factor": 1,
      "amount_cents": null,
      "running_cents": 105840
    },
    {
      "step": "RISK",
      "code": "factor:annual_miles",
      "detail": "annual_miles=11000 ×1.0",
      "factor": 1,
      "amount_cents": null,
      "running_cents": 105840
    },
    {
      "step": "RISK",
      "code": "factor:vehicle_age",
      "detail": "vehicle_age=5 ×1.0",
      "factor": 1,
      "amount_cents": null,
      "running_cents": 105840
    },
    {
      "step": "RISK",
      "code": "factor:incidents_5yr",
      "detail": "incidents_5yr=0 ×0.95",
      "factor": 0.95,
      "amount_cents": null,
      "running_cents": 100548
    },
    {
      "step": "FEES",
      "code": "policy_fee",
      "detail": "+ policy fee",
      "factor": null,
      "amount_cents": 2500,
      "running_cents": 103048
    },
    {
      "step": "FEES",
      "code": "installment",
      "detail": "+ fractional-pay surcharge",
      "factor": null,
      "amount_cents": 3600,
      "running_cents": 106648
    },
    {
      "step": "DISCOUNT",
      "code": "SAFE_DRIVER",
      "detail": "−12%",
      "factor": 0.88,
      "amount_cents": null,
      "running_cents": 93850
    },
    {
      "step": "DISCOUNT",
      "code": "PAPERLESS",
      "detail": "−$25",
      "factor": null,
      "amount_cents": -2500,
      "running_cents": 91350
    }
  ]
}

⚙ quote({"line":"AUTO","tier":"SAPLING","region":"R-SW","state":"AZ","term_months":12…)

{
  "facts": {
    "line": "AUTO",
    "tier": "SAPLING",
    "region": "R-SW",
    "state": "AZ",
    "term_months": 12,
    "peico_risk": 730,
    "age": 34,
    "billing_plan": "INSTALLMENTS",
    "autopay": false,
    "paperless": true,
    "loyalty_tier": "Sprout",
    "status": "CUSTOMER",
    "has_bundle": false,
    "active_lines": 1,
    "is_new_first_policy": false,
    "promo_code": null,
    "coverages": [],
    "driver_age": 34,
    "annual_miles": 11000,
    "vehicle_age": 5,
    "incidents_5yr": 0,
    "exposure_count": 1
  },
  "as_of": "2025-06-01"
}

{
  "as_of": "2025-06-01",
  "base_premium_cents": 88550,
  "final_premium_cents": 75424,
  "breakdown": [
    {
      "step": "BASE",
      "code": "base",
      "detail": "base_rate $900",
      "factor": null,
      "amount_cents": null,
      "running_cents": 90000
    },
    {
      "step": "BASE",
      "code": "tier:SAPLING",
      "detail": "tier factor ×0.82",
      "factor": 0.82,
      "amount_cents": null,
      "running_cents": 73800
    },
    {
      "step": "BASE",
      "code": "region:R-SW",
      "detail": "region factor ×1.12",
      "factor": 1.12,
      "amount_cents": null,
      "running_cents": 82656
    },
    {
      "step": "RISK",
      "code": "factor:driver_age",
      "detail": "driver_age=34 ×1.05",
      "factor": 1.05,
      "amount_cents": null,
      "running_cents": 86789
    },
    {
      "step": "RISK",
      "code": "factor:risk_score",
      "detail": "risk_score=730 ×1.0",
      "factor": 1,
      "amount_cents": null,
      "running_cents": 86789
    },
    {
      "step": "RISK",
      "code": "factor:annual_miles",
      "detail": "annual_miles=11000 ×1.0",
      "factor": 1,
      "amount_cents": null,
      "running_cents": 86789
    },
    {
      "step": "RISK",
      "code": "factor:vehicle_age",
      "detail": "vehicle_age=5 ×1.0",
      "factor": 1,
      "amount_cents": null,
      "running_cents": 86789
    },
    {
      "step": "RISK",
      "code": "factor:incidents_5yr",
      "detail": "incidents_5yr=0 ×0.95",
      "factor": 0.95,
      "amount_cents": null,
      "running_cents": 82450
    },
    {
      "step": "FEES",
      "code": "policy_fee",
      "detail": "+ policy fee",
      "factor": null,
      "amount_cents": 2500,
      "running_cents": 84950
    },
    {
      "step": "FEES",
      "code": "installment",
      "detail": "+ fractional-pay surcharge",
      "factor": null,
      "amount_cents": 3600,
      "running_cents": 88550
    },
    {
      "step": "DISCOUNT",
      "code": "SAFE_DRIVER",
      "detail": "−12%",
      "factor": 0.88,
      "amount_cents": null,
      "running_cents": 77924
    },
    {
      "step": "DISCOUNT",
      "code": "PAPERLESS",
      "detail": "−$25",
      "factor": null,
      "amount_cents": -2500,
      "running_cents": 75424
    }
  ]
}

Perfect! Here's the comparison:

Current Premium (Pine tier): $913.50/year
New Premium (Sapling tier): $754.24/year

Your annual savings: $159.26

The Sapling tier is eligible for your profile and location. Would you like me to go ahead and process this downgrade? Once I do, your policy will be updated to the Sapling tier and your new premium will be effective immediately.

64k in · 1,187 out · $0.02 · 20.1s

Customer

Yes, absolutely! Go ahead and process that downgrade for me right now. I want to make sure it's done today with that $754.24 annual premium.

Agent

⚙ change_tier(POL-retention-target-p_auto, SAPLING)

{
  "policy_id": "POL-retention-target-p_auto",
  "tier": "SAPLING"
}

{
  "ok": true,
  "rows_affected": 1,
  "changeset": {
    "policies": {
      "changed": [
        {
          "pk": {
            "policy_id": "POL-retention-target-p_auto"
          },
          "fields": {
            "tier_id": [
              "AUTO:PINE",
              "AUTO:SAPLING"
            ]
          }
        }
      ]
    }
  },
  "note": "stored premium is now stale — call reprice_policy to recompute"
}

⚙ reprice_policy(POL-retention-target-p_auto, 2025-06-01)

{
  "policy_id": "POL-retention-target-p_auto",
  "as_of": "2025-06-01"
}

{
  "ok": true,
  "as_of": "2025-06-01",
  "base_premium_cents": 88550,
  "final_premium_cents": 75424,
  "breakdown": [
    {
      "step": "BASE",
      "code": "base",
      "detail": "base_rate $900",
      "factor": null,
      "amount_cents": null,
      "running_cents": 90000
    },
    {
      "step": "BASE",
      "code": "tier:SAPLING",
      "detail": "tier factor ×0.82",
      "factor": 0.82,
      "amount_cents": null,
      "running_cents": 73800
    },
    {
      "step": "BASE",
      "code": "region:R-SW",
      "detail": "region factor ×1.12",
      "factor": 1.12,
      "amount_cents": null,
      "running_cents": 82656
    },
    {
      "step": "RISK",
      "code": "factor:driver_age",
      "detail": "driver_age=34 ×1.05",
      "factor": 1.05,
      "amount_cents": null,
      "running_cents": 86789
    },
    {
      "step": "RISK",
      "code": "factor:risk_score",
      "detail": "risk_score=730 ×1.0",
      "factor": 1,
      "amount_cents": null,
      "running_cents": 86789
    },
    {
      "step": "RISK",
      "code": "factor:annual_miles",
      "detail": "annual_miles=11000 ×1.0",
      "factor": 1,
      "amount_cents": null,
      "running_cents": 86789
    },
    {
      "step": "RISK",
      "code": "factor:vehicle_age",
      "detail": "vehicle_age=5 ×1.0",
      "factor": 1,
      "amount_cents": null,
      "running_cents": 86789
    },
    {
      "step": "RISK",
      "code": "factor:incidents_5yr",
      "detail": "incidents_5yr=0 ×0.95",
      "factor": 0.95,
      "amount_cents": null,
      "running_cents": 82450
    },
    {
      "step": "FEES",
      "code": "policy_fee",
      "detail": "+ policy fee",
      "factor": null,
      "amount_cents": 2500,
      "running_cents": 84950
    },
    {
      "step": "FEES",
      "code": "installment",
      "detail": "+ fractional-pay surcharge",
      "factor": null,
      "amount_cents": 3600,
      "running_cents": 88550
    },
    {
      "step": "DISCOUNT",
      "code": "SAFE_DRIVER",
      "detail": "−12%",
      "factor": 0.88,
      "amount_cents": null,
      "running_cents": 77924
    },
    {
      "step": "DISCOUNT",
      "code": "PAPERLESS",
      "detail": "−$25",
      "factor": null,
      "amount_cents": -2500,
      "running_cents": 75424
    }
  ],
  "changeset": {
    "policies": {
      "changed": [
        {
          "pk": {
            "policy_id": "POL-retention-target-p_auto"
          },
          "fields": {
            "rating_as_of": [
              "2025-05-01",
              "2025-06-01"
            ],
            "base_premium_cents": [
              106648,
              88550
            ],
            "final_premium_cents": [
              91350,
              75424
            ],
            "premium_breakdown": [
              "[{\"step\": \"BASE\", \"code\": \"base\", \"detail\": \"base_rate $900\", \"factor\": null, \"amount_cents\": null, \"running_cents\": 90000}, {\"step\": \"BASE\", \"code\": \"tier:PINE\", \"detail\": \"tier factor \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 90000}, {\"step\": \"BASE\", \"code\": \"region:R-SW\", \"detail\": \"region factor \\u00d71.12\", \"factor\": 1.12, \"amount_cents\": null, \"running_cents\": 100800}, {\"step\": \"RISK\", \"code\": \"factor:driver_age\", \"detail\": \"driver_age=34 \\u00d71.05\", \"factor\": 1.05, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:risk_score\", \"detail\": \"risk_score=730 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:annual_miles\", \"detail\": \"annual_miles=11000 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:vehicle_age\", \"detail\": \"vehicle_age=5 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 105840}, {\"step\": \"RISK\", \"code\": \"factor:incidents_5yr\", \"detail\": \"incidents_5yr=0 \\u00d70.95\", \"factor\": 0.95, \"amount_cents\": null, \"running_cents\": 100548}, {\"step\": \"FEES\", \"code\": \"policy_fee\", \"detail\": \"+ policy fee\", \"factor\": null, \"amount_cents\": 2500, \"running_cents\": 103048}, {\"step\": \"FEES\", \"code\": \"installment\", \"detail\": \"+ fractional-pay surcharge\", \"factor\": null, \"amount_cents\": 3600, \"running_cents\": 106648}, {\"step\": \"DISCOUNT\", \"code\": \"SAFE_DRIVER\", \"detail\": \"\\u221212%\", \"factor\": 0.88, \"amount_cents\": null, \"running_cents\": 93850}, {\"step\": \"DISCOUNT\", \"code\": \"PAPERLESS\", \"detail\": \"\\u2212$25\", \"factor\": null, \"amount_cents\": -2500, \"running_cents\": 91350}]",
              "[{\"step\": \"BASE\", \"code\": \"base\", \"detail\": \"base_rate $900\", \"factor\": null, \"amount_cents\": null, \"running_cents\": 90000}, {\"step\": \"BASE\", \"code\": \"tier:SAPLING\", \"detail\": \"tier factor \\u00d70.82\", \"factor\": 0.82, \"amount_cents\": null, \"running_cents\": 73800}, {\"step\": \"BASE\", \"code\": \"region:R-SW\", \"detail\": \"region factor \\u00d71.12\", \"factor\": 1.12, \"amount_cents\": null, \"running_cents\": 82656}, {\"step\": \"RISK\", \"code\": \"factor:driver_age\", \"detail\": \"driver_age=34 \\u00d71.05\", \"factor\": 1.05, \"amount_cents\": null, \"running_cents\": 86789}, {\"step\": \"RISK\", \"code\": \"factor:risk_score\", \"detail\": \"risk_score=730 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 86789}, {\"step\": \"RISK\", \"code\": \"factor:annual_miles\", \"detail\": \"annual_miles=11000 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 86789}, {\"step\": \"RISK\", \"code\": \"factor:vehicle_age\", \"detail\": \"vehicle_age=5 \\u00d71.0\", \"factor\": 1.0, \"amount_cents\": null, \"running_cents\": 86789}, {\"step\": \"RISK\", \"code\": \"factor:incidents_5yr\", \"detail\": \"incidents_5yr=0 \\u00d70.95\", \"factor\": 0.95, \"amount_cents\": null, \"running_cents\": 82450}, {\"step\": \"FEES\", \"code\": \"policy_fee\", \"detail\": \"+ policy fee\", \"factor\": null, \"amount_cents\": 2500, \"running_cents\": 84950}, {\"step\": \"FEES\", \"code\": \"installment\", \"detail\": \"+ fractional-pay surcharge\", \"factor\": null, \"amount_cents\": 3600, \"running_cents\": 88550}, {\"step\": \"DISCOUNT\", \"code\": \"SAFE_DRIVER\", \"detail\": \"\\u221212%\", \"factor\": 0.88, \"amount_cents\": null, \"running_cents\": 77924}, {\"step\": \"DISCOUNT\", \"code\": \"PAPERLESS\", \"detail\": \"\\u2212$25\", \"factor\": null, \"amount_cents\": -2500, \"running_cents\": 75424}]"
            ]
          }
        }
      ]
    }
  }
}

Excellent! ✓ Your downgrade is complete!

Your AUTO policy (POL-retention-target-p_auto) has been successfully changed from Pine to Sapling tier, effective today (June 1, 2025).

Your New Premium Details:

Annual Premium: $754.24
Billing Plan: Installments
Your Savings: $159.26/year compared to your Pine tier rate

The new premium is now in effect on your account. You should see this reflected in your next billing statement. Your policy continues through May 31, 2026.

Is there anything else I can help you with today?

23k in · 320 out · $0.0064 · 4.8s

Customer

Perfect — that's exactly what I needed. Thank you so much for getting that done today! I really appreciate it.

Agent

⚙ end_conversation()

{}

{
  "ok": true,
  "note": "Deliver your closing message now."
}

Take care, and thanks for being a PEICO customer!

26k in · 109 out · $0.0034 · 2.6s