Each method has advantages; in practice, however, all of these methods generally produce a similar prediction of the relative credit quality of account holders by capturing the underlying correlation between risk characteristics and delinquency behavior. These models use those factors correlating most strongly with good or bad performance. A sound practice for scoring models used for underwriting includes data from rejected applications to correct for estimation bias that arises if only approved accounts are used. If rejected applicants are systematically excluded from a model’s development, sample correlation between the applicants’ characteristics and delinquency will reflect only the behavior of the relatively good segment of the population. When the model is applied to the general population, it will overestimate the relative quality of the accounts with characteristics similar to those that were rejected, increasing the likelihood that lower-quality applicants will be approved.
These models predict the behavior of new applicants based on the performance of previous applicants. If the distribution of characteristics in the through-the-door population shifts (due, for example, to a change in marketing strategy that successfully attracts applicants outside the bank’s current market), the model’s ability to discriminate between “good’ and “bad” accounts may deteriorate. Other elements affecting a model’s ability to rank-order risk arise from using different sources to select sample applicants, using data from new market areas, and changing credit policy. Economic or regulatory changes also can affect the reliability of a model.
Scoring models generally become less predictive over time because they are typically developed without explicitly capturing the time-sensitive impact of changing economic and market conditions. Applicant characteristics, such as income, job stability, and age, change, as do overall demographics. These changes result in significant shifts in the profile of the through-the-door applicants. Once a fundamental change in the profile occurs, the model may be less able to identify potentially good and bad applicants.
For systems developed by third parties, examiners should review the third party’s guidelines in conjunction with bank management’s system for periodically assessing the model and the frequency of such assessments.
