eqlx.org / eqlx-credit
Python library · v0.2 · Apache-2.0
PD model validation, the way supervisors ask for it.
A bank's rating model gives every borrower a PD, the chance of default within a year, and once a year someone must check that model against what really happened. eqlx-credit computes those checks, the same way every time, each one from a public source.
What goes in, what comes out
No language model inside. Same input, same answer. For people who build, validate or review credit-risk models, and for their agents.
You give
- a rating scale: borrowers and defaults per grade
- the PD each grade had at the start of the year
- default rates of earlier years
- your thresholds
eqlx-credit computes
- how well the model ranks, with its margin of error
- defaults seen against defaults expected
- a traffic light per grade
- drift over the years and in the mix
You get
- every figure with its inputs
- observed and expected values side by side
- rows ready for a report table
- the public source behind each test
One yearly check, start to finish
A made-up corporate portfolio of 6,350 borrowers in six grades. It ships in the repository, with two problems planted in it.
1You enter the rating scale
One row per grade, safest first. PD is what the model promised at the start of the year.
| Grade | PD | Borrowers | Defaults |
|---|---|---|---|
| 1 | 0.12 % | 820 | 1 |
| 2 | 0.35 % | 1,450 | 4 |
| 3 | 0.70 % | 1,910 | 11 |
| 4 | 1.40 % | 1,320 | 19 |
| 5 | 3.00 % | 640 | 34 |
| 6 | 7.00 % | 210 | 12 |
2You set the policy
Your institution's choices. The library does not pick them.
| A result counts as a finding below | 5 % |
| How strongly defaults move together | 0.12 |
| Traffic light turns yellow at | 95 % |
| and red at | 99.9 % |
| Ranking score when the model was approved | 0.860 |
| Shift in the mix: watch / material | 0.10 / 0.25 |
3Does the model still rank well?
AUC: how often the model rates a defaulter as riskier than a non-defaulter. 0.5 is a coin flip, 1 is perfect.
4Defaults per grade: seen against expected
Expected = borrowers × PD. The bar is what happened.
5Is the PD of grade 5 too low?
Jeffreys test, the one the ECB asks for: how well do the defaults fit the PD? Run for every grade.
6Traffic light per grade
Read two ways: as if every default were separate, and allowing for bad years when defaults rise together.
| Grade | Separate | Together (0.12) |
|---|---|---|
| 1 to 4 | green | green |
| 5 | yellow | green |
| 6 | green | green |
Grade 5 is yellow only in the first reading. Bad year or wrong PD? That is for the validator to judge.
7Right over several years?
Share of the whole portfolio that defaulted each year, against a PD of 1.10 %.
8Has the mix of borrowers moved?
PSI: one number for how far the spread over grades shifted since last year. 0 means no change.
What comes back
The ranking score fell from 0.860 to 0.795 (p = 0.003). Grade 5 had 34 defaults against 19.2 expected (p = 0.0009) and is yellow if defaults are separate. A test across all grades at once, Hosmer-Lemeshow, gives p = 0.0437. No opinion is given: that stays with the validator.
Each figure above is the library's output for this portfolio. Both planted problems, the weaker ranking and grade 5, were found.
What it covers
Grouped by the question a validator asks.
| Question | Functions | Public source |
|---|---|---|
| Does the model still rank well? | auc_ci, auc_ci_from_counts, auc_decline_test, gini, ks | DeLong et al. (1988); ECB instructions for IRB validation reporting (2019) |
| Are the grade PDs right this year? | jeffreys_test, binomial_test(rho=), traffic_light, hosmer_lemeshow | ECB (2019); BCBS Working Paper 14 (2005); Tasche (2003) |
| Are they right over several years, or per borrower? | normal_test, spiegelhalter | BCBS WP14 (2005); Spiegelhalter (1986) |
| What were the default rates? | cohort_default_rates, long_run_average | EBA/GL/2017/16 |
| Has the portfolio moved or bunched up? | psi, hhi(exposure=) | Siddiqi (2006) |
| How do I set the PD level and curve? | anchor_to_central_tendency, fit_ct_ar, apply_pd_floor, master scale | Tasche (2013); Basel II, CRR, Basel III floors |
| Too few defaults to estimate a PD? | pluto_tasche, van_der_burgt | Pluto & Tasche (2005); van der Burgt (2008) |
| What capital does it need? | risk_weight, capital_requirement | Basel II ¶272–330 and Annex 5 |
Capital: the Basel risk weight for any PD
The risk weight is the share of a loan a bank must count when it works out its capital. Loss rate 45 % in both charts.
It rises with the PD
Loans to large companies.
It depends on the kind of loan
The same PD of 0.50 % in five loan classes.
All twelve figures are the library's output and equal the figures printed in Basel II Annex 5.
Too few defaults to trust the count
Six grades with 8 defaults among 1,750 borrowers. The Pluto-Tasche method gives a cautious upper limit for each PD: 90 % confidence, defaults moving together at 0.12.
A grade with no defaults still gets a PD above zero. The limits never fall as the grades get riskier.
How an agent uses it
The agent finds the data, picks the test and writes the summary. It does not produce a number itself: a Gini or a risk weight written by a language model can look right and be wrong.
These are the steps of the Agent Skill annual-pd-validation in the repository; the example above is its bundled portfolio. Today eqlx-credit is a Python library with structured results that any agent framework can call. An MCP server, so agents can call it directly, is next.
How it was tested
Every function is compared with something it was not derived from.
Published figures, reproduced
Why "defaults move together" matters
500 borrowers, PD 1 %, 12 defaults. How likely is a year this bad if the PD is right?
And on every change
What it does not do
pip install eqlx-credit