scorecard development process is statistically valid.
Data Scientist
The data scientist is the person who helps source and extract the required records and fields of information in order to populate the scorecard development database. This person usually has:
● An in-depth knowledge of the various databases in the company, and the data sets being used.
● Proficiency in the tools and systems to determine and document data lineage, to perform field-specific code mappings to common values and definitions from a variety of internal legacy transaction systems and external data reporters.
● Ability to merge/combine information from disparate sources and perform necessary preprocessing to deal with data issues, such as undefined codes, missing information, or extreme/suspect values.
● Familiarity with file formats and fields of information available from the different credit bureaus, rating agencies, and other third-party data providers.
A good example of the required knowledge for data sourcing and extraction is in mortgage lending, where there can be up to four co-applicants, and records for each must be found and joined into a single complete applicant record with individual and combined characteristics derived. These include characteristics such as combined loan-to-value ratio, combined income, payment to combined income, combined debt-to-income ratio, and payment shock to combined current housing payments. Even in a data warehouse, the co-applicant records may reside in a different table that the primary applicant record and matching logic must be used to associate related records. Typical scorecard developers do not possess this type of in-depth knowledge, especially in the larger, more complex financial institutions.
Product or Portfolio Risk Manager/Credit Scoring Manager
Risk managers are responsible for the management of the company’s portfolio and usage of scorecards. They are usually responsible for creating policies and strategies for approvals, credit limit setting, collections treatment, and pricing. In most companies, this person would be the business owner of the scorecard. This person usually has:
● Subject matter expertise in the development and implementation of risk strategies using scores.
● An in-depth understanding of corporate risk policies and procedures.
● An in-depth understanding of the risk profile of the company’s customers and applicants for products/services.
● A good understanding of the various implementation platforms for risk scoring and strategy implementation in the company.
● Knowledge of legal issues surrounding usage of particular characteristics/processes to adjudicate credit applications.
● Knowledge of credit application processing and customer management processes in the company.
● Knowledge of roll rate models; delinquency trends by product, region, and channel; and reports and the average time to charge-off.
When a modeler is asked to build a model (typically a process initiated by the business area), the first question they should ask the businessperson is “why?” That businessperson is typically the risk manager. The answer to that question determines everything else that is done from that point forward, including deciding the target, variable mix in the model, picking the best model, conditions, appropriate model fit measures, and, of course, the final cutoff for any decisions. This person ensures that business considerations are given sufficient thought in the design and implementation of scorecards. Early on in the process, the risk manager can tap their knowledge of the portfolio risk dynamics and performance experience to help with determining the definition of what constitutes “bad” performance for the population of interest. A good practice is to involve risk managers (or a representative) in each phase of the scorecard development process, and get their approval at the end of each one. Risk managers should be able to use some of their experience to point scorecard developers in a particular direction, or to give special consideration to certain data elements. For example, in cases where data is weak or biased, risk managers may use their experience to adjust weight of evidence (WOE) curves or to force certain variables (weak but logical) into the model. Experienced risk managers are also aware of historical changes in the market, and will be able to adjust expected performance numbers if required. They would also contribute heavily to the development of strategies and to gauging possible impacts of those strategies on customers and the various areas of the organization. Scorecards are developed to help in decision making – and anticipating change is key.
Product Manager(s)
The product manager is responsible for the management of the company’s product(s) from a marketing or customer retention perspective. Their main objectives are usually revenue related, and they would have:
● Subject matter expertise in the development and implementation of product-marketing strategies.
● An in-depth knowledge of the company’s typical client base and target markets, including its best/most valued customers.
● Knowledge of future product development and marketing direction.
Product managers can offer key insights into the client base and assist during segmentation selection, selection of characteristics, and gauging impact of strategies. They also coordinate design of new application forms where new information is to be collected. Segmentation offers the opportunity to assess risk for increasingly specific populations – the involvement of marketing in this effort can ensure that scorecard segmentation is in line with the organization’s intended target markets. This approach produces the best results for the most valued segments and harmonizes marketing and risk directions. In other words, the scorecard is segmented based on the profile that a product is designed for, or the intended target market for that product, rather than based on risk considerations alone.
We will cover the gauging of impact on key customer segments post-cutoff selection in later chapters. This involves, for example, measuring metrics like expected approval rates for high-net-worth and similar premium customers. Marketing staff should be able to provide advice on these segments. The product managers typically do not have a say in the selection of final models or variables.
Operational Manager(s)
The operational manager is responsible for the management of departments such as Collections, Application Processing, Adjudication (when separate from Risk Management), and Claims. Any strategy developed using scorecards, such as changes to cutoff levels, will impact these departments. Operational managers have direct contact with customers, and usually have:
● Subject matter expertise in the implementation and execution of corporate strategies and procedures.
● An in-depth knowledge of customer-related issues.
● Experience in lending money.
Operational managers can alert the scorecard developers on issues such as difficulties in data collection and interpretation by frontline staff, impact on the portfolio of various strategies, and other issues relating to the implementation of scorecards and strategies.
A best practice that is highly recommended is to interview operational staff before starting the modeling project. For example, if analysts are looking to develop a mortgage application scorecard, they should go talk to adjudicators/credit analysts who approve mortgages, or other senior staff who have lending experience. Similarly, talking to collections staff is useful for those developing collections models. I normally ask them some simple questions. For applications models, I typically get about 8 to 10 approved and another 8 to 10 declined applications, and ask the adjudicator to explain to me why they were approved or declined. I often ask collectors if they can identify which debtors are likely to pay before talking to them, and which specific variables they use to decide. Staff from Adjudication, Collections, and Fraud departments can offer experience-based insight into factors that are predictive of negative behavior (i.e., variables that they think are predictive), which helps greatly when selecting characteristics for analysis, and constructing the “business optimal” scorecard. This is particularly useful for analysts who don’t have a lot of banking/lending