Partners in Evaluation
Financial Coaching Bench is an effort supported by NextLadder Ventures, advised by the Financial Health Network.
Key Takeaways
- Popular chatbots rank near the bottom. The default models in ChatGPT and Google’s free AI products rank 25th to 28th of 28, 11 to 19 points below the leader. Newer GPT-6 and Muse Spark models also score below their predecessors, while free Claude ranks 7th.
- Claude Opus 5.5 leads, and the top twelve models are within 2.4 points of each other. MiMo V2.6 Pro, fifth, costs $0.013 per scenario, under 1% of Claude Opus 5.5’s $1.63.
- About 95% of failed checklist items are advice the AI coach left out or only partly gave. The other 5% are items the coach contradicted. Within a model, each doubling of response length is associated with about 3 more points; at equal response length, GPT-5.6 Sol ranks first.
About the Benchmark
Financial Coaching Bench measures how well AI models coach a household through a financial problem. According to State of the Prompt, 5.4% of adults’ AI chatbot conversations are mainly about money, and 14.4% of those are about credit, debt and loans.
A collections notice, a rent increase or a defaulted loan can leave a household with urgent decisions. Useful coaching depends on the client’s income, debts and deadlines, including details they may not mention without being asked.
Each scenario has a client’s opening message, a hidden fact sheet (income, debts, deadlines, savings, household) and a practitioner-written checklist of what a good coach recommends, warns about or gets right. The model being tested acts as the coach and can ask questions and search the web. Claude Sonnet 4.6 plays the client and answers the coach’s questions using the fact sheet. The conversation ends when the coach delivers a plan or reaches the exchange limit.
Scenarios follow a Vals taxonomy that draws on the Financial Health Network’s Financial Health Pulse and FinHealth Spend reports: the four FinHealth pillars (Borrow, Spend, Save, Plan), two cross-cutting domains (Life Events & Shocks, and Emerging & Digital-First Behaviors) and Long-Term Wealth & Security, based on the Aspen Institute’s New Wealth Agenda. The 256 scored scenarios are Borrow 28%, Spend 16%, Save 15%, Life Events & Shocks 13%, Long-Term Wealth & Security 11%, Plan 9% and Emerging & Digital-First Behaviors 8%. The test set is private except two scenarios, shown under Example Scenarios.
Results
Rankings
The top twelve are Claude Opus 5.5, Muse Spark 1.2, Claude Fable 5.1, Qwen 3.8 Max, MiMo V2.6 Pro, Wonder Full, Claude Sonnet 5.5, GPT-5.6 Sol, Kimi K3, Grok 4.7, GLM 5.3 and Wonder.
GPT-6 Luna has served free ChatGPT since Oct 8, and Gemini 3.5 Flash-Lite most free Gemini prompts since Oct 9. Gemini 3.5 Flash has been the default in Google Search’s AI Mode since May. They replaced GPT-5.6 Luna (69.3%) and Gemini 3.6 Flash (65.8%), which score higher. ChatGPT and Gemini are the chatbots US adults use most: 44% and 24% have used them, against 6% for Claude (Pew Research Center, February 2026).
Failed checklist items and high-cost credit
Contradictions make up 3% to 8% of each model’s failed items, often where the coach recommended a different approach.
GPT-5.6 models mention payday, title, pawn or cash-advance lenders in 26–31% of scenarios, Gemini models in 2–4%; none of 24 randomly sampled passages recommends one. In one scenario a $40 payday loan costs less than the client’s $100 late rent fee, and practitioners recommend the loan; 5 of 28 coaches did. None of 20 randomly sampled debt-settlement mentions recommends a for-profit settlement firm, and none of 20 buy-now-pay-later mentions promotes it.
Questions
Muse Spark 1.2 opens with about eight questions and GPT-5.6 Sol with one; they rank 2nd and 8th.
Length
Coaches write 900 to 2,500 words per conversation, or four to ten minutes of reading. At equal response length, GPT-5.6 Sol moves from 8th to 1st and Muse Spark 1.2 from 2nd to 11th.
Domains and clients
| Models (28) | ||||||||
|---|---|---|---|---|---|---|---|---|
| 74.2% | 67.0% | 83.7% | 78.7% | 78.7% | 76.8% | 78.9% | 76.2% | |
| 76.8% | 68.3% | 79.6% | 71.1% | 77.5% | 78.7% | 81.4% | 75.7% | |
| 74.3% | 71.9% | 73.3% | 80.0% | 78.1% | 79.4% | 77.9% | 75.7% | |
| 78.4% | 69.8% | 75.7% | 75.1% | 69.5% | 77.7% | 82.2% | 75.6% | |
| 72.7% | 73.9% | 74.5% | 74.7% | 79.5% | 77.3% | 83.6% | 75.4% | |
| 71.3% | 69.0% | 74.6% | 76.0% | 82.9% | 78.5% | 84.9% | 75.0% | |
| 73.5% | 70.4% | 73.0% | 77.3% | 80.1% | 71.1% | 88.6% | 74.9% | |
| 73.0% | 68.8% | 73.6% | 71.1% | 82.5% | 83.7% | 82.0% | 74.9% | |
| 76.1% | 65.4% | 78.3% | 76.9% | 75.1% | 69.5% | 82.6% | 74.5% | |
| 75.1% | 67.9% | 72.1% | 74.8% | 75.8% | 73.6% | 84.6% | 74.1% | |
| 73.1% | 66.5% | 75.0% | 75.3% | 76.7% | 74.0% | 85.7% | 74.1% | |
| 75.4% | 67.7% | 70.8% | 73.0% | 77.3% | 75.3% | 81.4% | 73.8% | |
| 70.0% | 68.3% | 75.0% | 72.0% | 79.9% | 75.3% | 81.6% | 73.1% | |
| 73.1% | 64.4% | 74.7% | 69.9% | 75.7% | 77.7% | 80.4% | 72.8% | |
| 68.0% | 66.9% | 70.3% | 68.6% | 78.9% | 73.2% | 83.0% | 71.0% |
Coaches met 73.5% of checklist items for the 53 clients earning under $30,000 and 69.1% for clients earning more. No model scored reliably worse for those clients after correcting for 28 comparisons.
Web search and cost
Web search did not measurably change scores for the three models tested without it: GPT-6 Luna, GPT-5.6 Luna and Muse Spark 1.2.
Referrals to free help
Where a practitioner expected a referral to free help, the models behind free ChatGPT and free Gemini named one in about 70% and 60% of conversations, compared with 75–94% for the leading models. Counselors reviewing AI advice should check referrals and deadlines against the client’s circumstances, because an answer can omit them and still sound complete.
Methodology
Practitioners wrote and reviewed every checklist. One reviewer marked each item must-have or path-dependent: must-have items are graded strictly, and path-dependent items also pass with a coherent, justified alternative.
DeepSeek V4.1 Flash grades each item against the full transcript; it is also on the board as a coach, ranking 14th. On 182 items that a practitioner also graded, it agreed with the practitioner 85% of the time and passed about the same share of items (53%).
A scenario’s score is the share of items met; an unfinished scenario (timeout or context overflow) scores zero. The Length-Controlled ranking estimates each model’s score at the average response length across all models, using the relationship between length and score within each model. Jev 1.13, a separate classification model, grades whether the coach asked for each hidden fact, assumed it or never raised it; the Elicitation ranking shows the share asked for.
The benchmark scores 256 of the 269 test scenarios. They were selected to keep each domain’s share within five percentage points of the taxonomy’s target distribution, with Borrow capped at 28%. A separate validation set is private.
Example Scenarios
In one example, a client with about $30,000 in credit card debt and a new raise asks whether to take out a consolidation loan or a home equity loan. In the other, a client paid on a prepaid card asks what to look for in a first checking account for direct deposit. Each shows four models’ conversations with the checklist and the judge’s verdicts.
Scenario
- client age
- 40-44 years
- household child dependents
- 0
- household marital status
- Married
Client#1
Client
I have about $30k in debt across a few credit cards. I am able to make my payments every month, but not much more than the minimum. I just got a decent raise at work that I want to use to take care of getting this debt paid off. Should I apply for a loan and which type - consolidation or home equity?
Coach#1
Response
A raise is a real chance to get ahead of this, but a loan isn’t automatically the right move — and a home-equity loan is a very different bet from an unsecured consolidation loan. Before recommending either, I need a few numbers so we don’t trade expensive credit-card debt for something worse, or put your house on the line for no real gain.
Please share what you can:
- The raise: old take-home pay vs. new take-home pay, and roughly how much extra you’ll have each month after taxes.
- The cards: for each card, balance, APR, and minimum payment. Total is about $30k — the rates matter more than the total.
- Credit score (or a rough range, if you know it). That’s what decides whether a consolidation loan’s rate would actually beat your cards.
- Housing: do you own, and if so, roughly what the home is worth, what you still owe, and your mortgage rate and payment? A home-equity loan or HELOC uses the house as collateral — miss payments and you can lose it.
- Cash cushion: any emergency savings, or are you basically at zero?
- Other debts: car, student loans, personal loans — balances, rates, and payments.
While you pull that together, the decision rule is simple:
- Don’t borrow against the house just to tidy up credit cards unless the rate savings are large and you can afford the payment even if income drops. Unsecured card debt can be negotiated or discharged in bankruptcy; a home-equity loan generally cannot, and the house is the collateral.
- A personal consolidation loan only helps if the new rate is meaningfully lower than your card APRs, the term doesn’t stretch the payoff so long that you pay more interest overall, and you stop using the cards. If your score is fair or poor, the “consolidation” rate is often not much better — sometimes worse once fees are counted.
- Often the cleanest use of a raise is to keep the cards, automate a fixed extra payment aimed at the highest-APR balance, and avoid a new loan entirely. That only works if the raise is big enough to move the payoff date.
Reply with those numbers (even rough ones) and I’ll run the math on “extra payments only” vs. a consolidation loan vs. home equity, and tell you which one actually fits.
Client#2
Client
ok that's a lot but let me try to answer what I can
the raise was 20% and my new take-home is around $5,500 a month for our household. so yeah it's a pretty solid bump, and I'm planning to put all of that extra toward debt and maybe savings.
on the cards, I honestly don't have every rate memorized right now. I know they're pretty high interest rates, like not great at all. I have a few cards and total it's around $30k. I don't have all the individual balances in front of me.
credit score is 660. I know it's taken a hit because the cards are pretty much maxed out.
we do own our home. the value is around $140k and we only have about 15% equity in it. I'm not sure offhand what our exact mortgage rate is. the mortgage payment is around 25% of our take-home.
savings... yeah that's kind of embarrassing, we've got about $200 in savings right now. basically nothing.
and we also have 2 car loans on top of everything. I don't know the exact numbers on those off the top of my head.
so based on all that, what do you think? does a consolidation loan make sense or should I just attack the cards directly with the extra money each month?
Coach#2
Searches
Response
Skip both loans for now. Use the raise to pay the cards down directly. A home-equity loan is a clear no, and a consolidation loan only makes sense later if a real offer beats your card rates on a short term.
Your house can't carry this debt. At about $140k value and 15% equity, you owe roughly $119k already. Paying off $30k of cards with home equity would put more debt on the house than the house is worth. Even a lender willing to go to 90–95% of value might free something like $7k–$14k before closing costs, not $30k. You'd pay fees, keep most of the card debt, and turn unsecured debt into debt that can cost you the house. With $200 in savings, that risk isn't worth it.
A personal loan is a weaker maybe, not a yes. With a 660 and maxed-out cards, offers often land in the mid-to-high teens, sometimes over 20%, plus an origination fee. The average card rate nationally is about 21% right now; maxed cards are often higher. Rough math on $30k:
- At 22% for 3 years, the payment is about $1,150 a month and interest is about $11,300.
- A 17% consolidation loan for 3 years is about $1,070 a month and about $8,500 in interest. A 5% fee rolled into the loan wipes out most of that savings.
- Stretch it to 5 years around 18% and the payment drops to about $760, but interest climbs to roughly $15,700. That's more than just paying the cards off faster with the raise.
So a loan only wins if a soft-pull offer is several points below your actual card rates, the term is 3 years (4 at the outside), and the fee is small. Don't apply yet. You don't know the rates you'd be replacing, and a new loan while the cards stay open is how people end up with the loan and the cards.
Your one move this month: don't apply for anything. On your next paycheck, move money to savings until that account hits $1,000, and send the rest of the raise as an extra payment to the highest-APR card. Minimums on everything else, including the mortgage and both cars. Housing comes before the cards.
$200 is not a moral failure, but it is an emergency. One car repair and you're back on the cards or short on the mortgage. A 20% raise on a $5,500 take-home is roughly $900 a month before lifestyle creep, but check the actual paycheck difference — taxes mean it may be less. Even $400–$500 extra, aimed at the highest rate, changes the payoff date. Paying only minimums on $30k mostly feeds interest.
Tonight, open each card account and write down balance, APR, and minimum. Same for both car loans if you can. Once you have those, I can tell you which card to hit first and whether any loan quote is actually worth taking.
Client#3
Client
ok that actually makes a lot of sense, especially the home equity part. I didn't really think about the fact that we barely have any equity - that would have been a bad move.
and yeah I get the savings thing, $200 is scary when I think about it. so build to $1k first, then attack the highest rate card with the rest. got it.
before I go though, can you walk me through what the full plan looks like start to finish? like once I have $1k in savings and I'm throwing the extra at the cards, what's the overall strategy? do I just keep doing that until they're all paid off, or is there a point where I should revisit the consolidation loan idea?
Coach#3
Response
Yes. Same strategy the whole way: protect the house and the minimums, keep a small cash cushion, and throw every extra dollar at the highest-rate card until the cards are gone. A consolidation loan is a checkpoint, not the plan.
This week. List every card and both car loans: balance, APR, minimum, due date. Automate every minimum, including the mortgage, so a busy month can’t create a late fee. If work matches retirement contributions, contribute just enough to get the full match before any extra card payments. That’s an instant 50–100% return, which beats even a brutal card rate. No match? Cards get the extra money.
Next one or two paychecks. Savings goes from $200 to $1,000. Still pay the minimums. Don’t put new charges on the cards to “make room” for that transfer. If a real emergency spends the $1,000 later, refill it before you resume extra debt payments. Don’t build a big emergency fund yet while the cards are this expensive.
Then attack the cards, in rate order. The day after payday, send a fixed extra payment to the highest-APR card. Minimums on everything else. When that card hits zero, freeze it in the app so you can’t casually reuse it, leave the account open, and roll that card’s old minimum plus the extra onto the next-highest APR. Same payment, new target, until they’re all at zero. Don’t close them as you go — losing the limits can hurt your score, and you may need the history. Just don’t spend on them.
A raise only works if it isn’t sitting in checking. If the take-home bump is about $700–$900, that’s the extra payment once the $1,000 is in place. Check the actual paycheck difference; taxes mean it may be less than 20%.
Rough pace, so you can see it. On $30,000 at 22% — close to today’s average card rate, and yours may be higher — interest alone is about $550 a month. Pay $1,050 total and you’re looking at roughly 3.5 years and about $13,000 in interest. Pay $1,450 and it’s closer to 2 years and about $8,000 in interest. Minimums-only is how this stays a decade-long problem. Your real date depends on the APRs and what you’re already paying, which is why the list matters.
Revisit a consolidation loan twice, not every month.
- First checkpoint: as soon as you have the APRs. Soft-pull prequalification only, so it doesn’t ding the 660. Take it only if the rate is at least about 5 points below the average rate on the cards you’d pay off, the term is 3 years (4 at the outside), and the fee is small — under about 3%. Then keep paying at least what you were already sending to the cards. A lower payment feels easier and usually costs more interest.
- Second checkpoint: when the card balances are down about 30%, or your score is up around 700. Maxed cards are a big reason you’re at 660. Lower balances often unlock a genuinely better rate. Same test. If you’re already on a 2–3 year pace, skip the loan. The hassle and the temptation to run the cards back up aren’t worth a small savings.
Home equity stays off the table. You still don’t have the equity, and the house shouldn’t back this debt.
After the cards hit zero. Grow savings from $1,000 toward about 3 months of must-pay bills — mortgage, both cars, insurance, utilities, food. Then put extra on whichever car loan has the higher rate. Don’t prepay the mortgage while a car loan is expensive or savings are still thin.
If money gets tight at any point, cut the extra card payment before you touch the mortgage, car payments, insurance, utilities, or food. Those keep the household standing. The cards can wait a month. A nonprofit credit-counseling agency (not a debt-settlement company) is the backup if the rates turn out to be near 30% or the minimums stop being affordable — not the first move.
Send me the balances, APRs, and minimums when you have them, plus the two car payments. I’ll name the first card to hit and a monthly dollar amount that fits the $5,500 take-home.
- The coach never sees the scenario details above. It starts with the client's opening message and learns the rest in conversation.
- Select a criterion to read the judge's rationale and highlight the coach turns it cites. The judge cites turns only for passed criteria.
- The score includes Must have and Path dependent criteria.
Citation
Vals AI, Financial Coaching Bench: Can AI Models Coach a Household Through a Real Financial Problem?, November 2026. https://www.vals.ai/benchmarks/financial-coaching-bench
Citation (BibTeX)
@misc{financialcoaching2026,
title = {Financial Coaching Bench: Can AI Models Coach a Household
Through a Real Financial Problem?},
author = {{Vals AI}},
year = {2026},
month = nov,
howpublished = {Vals AI},
url = {https://www.vals.ai/benchmarks/financial-coaching-bench}
}