What is it about?
This research investigates how to build a Foundation Model for financial transactions. In a nutshell, it is a large neural network model that has been trained on vast amount of data to learn patterns in peoples' transaction habits. The authors designed a novel algorithm that teaches the model to learn behavioural features that can predict future behaviour, and also enhances its memory capacity to remember past information. They used this algorithm to train a large Foundation Model on over 5.1 billion real transactions from over 100 issuing banks. This model was able to significantly improve performance on multiple downstream transaction understanding tasks, particularly on fraud detection in card payments.
Featured Image
Photo by naipo.de on Unsplash
Why is it important?
Deep learning produces the most powerful models, but using deep learning solutions requires expertise and can pose significant engineering challenges, especially in real-time applications. Many financial institutions that analyse transactional data still heavily rely on hand-engineered features and simple models based on decision trees. A pretrained Foundation Model that is accessible through an API call-out can enable these simple models to use the power of deep learning, without the associated challenges of pretraining and model serving. The same unified Foundation Model can be used in various applications, from fraud detection to customer churn prediction. This approach of using embeddings from a pretrained neural network can accelerate the shift towards more powerful and highly capable models. That will have an impact not only on the financial institutions but also directly on their customers. Improved fraud models means better protection against fraudsters and fewer losses, while improved churn models mean better understanding of customer needs and improved customer satisfaction.
Perspectives
I think there's a genuine demand in industry for a unified Foundation Model for transactional data. When we presented this work at ICAIF'23, we got a lot of interest not only from academics but also from innovation teams at banks and other financial institutions. The biggest challenge that everyone has faced is obtaining a large and diverse dataset for pretraining. Working with a team that has access to transactional histories from so many institutions has put us in a unique position where we can focus on research and push the boundaries of AI in the financial industry, getting us closer to what’s become a standard in the space of Large Language Models.
Piotr Skalski
Featurespace
Read the Original
This page is a summary of: Towards a Foundation Purchasing Model: Pretrained Generative Autoregression on Transaction Sequences, November 2023, ACM (Association for Computing Machinery),
DOI: 10.1145/3604237.3626850.
You can read the full text:
Contributors
The following have contributed to this page
