Customized Natural Language Processing Model for a Large Financial Institution

Challenge

The client had a high-risk regulatory issue related to sales practices potentially impacting customers that was initiated by the use of a third party. The bank identified a significant number of complaints related to this product, compared to similar products.

To better understand their potential exposure, the client hired Guidehouse to review a sample of these complaints. Analysis of the sample results led the client to initiate a full review of the remaining complaints. However, due to the time needed to complete the review, the sensitive regulatory nature of this issue, and the cost, manual review of the population was not a desirable option.
 

Solution

Guidehouse developed an NLP model that was able to efficiently analyze data for tens of thousands of complaints. The NLP model is a supervised machine-learning model that analyzes key words, assigning prediction weightages based on the presence of words in each of the classification groups’ (restitution or no restitution) training data. Guidehouse leveraged the NLP model output to group complaints by likelihood of restitution, performing thorough and comprehensive manual reviews on the cases with the highest likelihood of restitution and strategically sampling cases with less likelihood of restitution to validate model accuracy.

To better explain the weighted impact of individual words on model predictions, Guidehouse leveraged SHapley Additive exPlanations (SHAP). SHAP provided insights into each prediction and overall key words that helped determine why a complaint had a higher or lower likelihood of needing restitution. The model accuracy exceeded 90%, which allowed the client to confidently decide more than 50% of complaints without having to perform manual reviews, resulting in review time and cost savings of more than 70%.

Impact 

The use of the NLP model reduced the total time and cost to review this matter by more than 70%.

The Guidehouse NLP model provided the client an accurate assessment of the likelihood each case required restitution, enabling the client to avoid full manual review of the complaint population. As a result, the client was able to focus time and energy on reviewing the complaints with the highest likelihood of needing restitution and sample the less likely buckets to validate the model’s accuracy. The Guidehouse model allowed the client to restitute their impacted customers quickly, completely and efficiently.