.png)
AI is revolutionising the financial sector, but with concerns over ethics, how can firms ensure proprietary and embedded tech stays secure, accurate and unbiased?
Key takeaways:
- Firms should rightly approach AI with caution, especially within the financial sector.
- Bias, security concerns, model drift and regulatory issues all present all major ethical concerns for the adoption of AI.
- At Ophelos we implement a stringent ethical framework for handling our models to ensure their accuracy.
- This includes pre-processing data right through to auditing and optimisation.
- Embedding ethics into our wider operation is also vital for ensuring our values are reflected in our products.
Moral dilemmas surrounding the use of AI have been debated for decades, dating all the way back to the inception of the very first computer program in the 1940s.
However, conversations about AI’s true potential for harm have been recently invigorated by the proliferation of generative AI — visions of a Matrix-style machine apocalypse once again being thrust into public consciousness.
Whilst we might not be running headlong towards the red pill just yet, taking a cautionary approach to AI is certainly more than warranted, especially across the financial space.
Data bias, security concerns, copyright conundrums and model inaccuracies all present serious threats to companies and customers alike — especially within a sector that services vulnerable individuals on a daily basis.
Can we trust AI to make decisions in these high-stakes environments, where even the smallest model inaccuracies or biases have the potential to drastically affect customer livelihoods? Across such a (rightly) regulated industry, how can we leverage AI in a way that minimises risk whilst staying compliant?
In this article, we’ll explore some of the most pressing ethical concerns surrounding the current use of AI within the debt industry. Additionally, we’ll detail some of the effective mitigation strategies we employ at Ophelos to ensure our AI-native platform is continually functioning in line with an ethical framework.
AI’s ethical concerns
Fundamentally, models are only ever as good as the data that they’re trained on and the monitoring put in place to ensure their continued accuracy.
Even the smallest dip in training input quality or lackadaisical approaches to monitoring practices can have big repercussions on the quality of model outputs.
Let’s take a look at some of the key concerns for AI’s application within the context of debt collection.
Data bias
Training data with even the smallest inbuilt bias has the potential to exacerbate pre-existing inequalities. Already marginalised demographics are particularly at risk of these inequalities and there have been reports of bias finding its way into training data.
This is true across both implicit, or subconscious, bias and explicit, or conscious, bias. One of the most famous cases of algorithmic bias is the COMPAS algorithm, used to measure the likelihood of US defendants committing reoffences. COMPAS has been found to disproportionately rate Black defendants as more likely to re-offend than White defendants.
In systems that employ machine learning, it’s easy to see how echo chambers entrenched with existing prejudices can very quickly develop.
Security and privacy
Owing to the vast amount of data AI models utilise, customer and client privacy is continually at risk from model errors and security breaches. For example, data poisoning is a form of cyber attack that intentionally compromises training data, modifying model outputs.
Data breaches can also be of particular concern when firms use third-party tools to embed AI within preexisting platforms. Third-party integrations can often sit as a weak touchpoint in tech stacks, being vulnerable to threats or data leaks.
Regulatory compliance
Outdated training data runs the risk of quickly falling behind on the latest financial regulation. In addition to models being frequently updated and tested, continuous monitoring is key to ensuring model outputs reflect the latest compliance.
Model inaccuracies
Errors within training data and inputs can have huge implications for output quality. Inaccuracies can occur if proper data cleansing practices and testing are not followed.
Within models that employ machine learning, model inaccuracies may develop over time if inputs vary considerably. This is known as model drift and can result in increasingly inaccurate outputs if models aren’t properly monitored and optimised.
Model drift poses a significant risk for systems that employ machine reasoning, like decision engines, or next-action summaries — where an algorithm makes a ‘choice’ based on historical data.
Building ethical AI frameworks
The good news? All of these concerns can be mitigated with ethical framework approaches that include watertight testing, continuous monitoring and data balancing.
At Ophelos, we follow several different processes that ensure our platform and models are operating as robustly as possible, from pre-processing data right through to auditing and optimisation.
Pre-processing data balancing
We use data pseudonymisation, which includes replacing PII with artificial identifiers. This process allows us to remove any sensitive attributes from datasets at the data pre-processing stage.
This step allows us to mitigate potential bias within training data, ensuring we’re cleaning and balancing datasets even before they’ve been utilised for training.
Robust testing
We operate on a three-line defence framework for implementing any new training sets or features — both requiring numerous levels of approval before being employed. We also test using sandboxes and controlled environments to ensure the quality of outputs and the accuracy of models.
Continuous monitoring and feedback
This is especially important for machine learning and any models making ‘decisions’. Our model outputs are monitored by human agents across customer interactions, decision making and high-impact outcomes.
For example — sample communications written by our proprietary model OphelosGPT are checked weekly by a human agent to monitor their quality against set benchmarks.
We also have guardrails on the frequency and volume of communications the model is able to send to customers with QAs of the responses also regularly conducted, with feedback fed back into the model weekly.
Data cleansing
To combat the potential for model drift or output inaccuracies we regularly cleanse data and run quality checks with alerts on ingestion. We also run regular audits on data sets and outputs which are cross-checked by different teams.
Any issues that are flagged are addressed with our rigorous escalation procedure, including discussion with our Risk Committee before an action plan is put in place to address any concerns.
Regulatory horizon scanning and monitoring
We conduct horizon scanning to ensure we are not missing any regulatory changes that could impact the way we use our AI. Horizon scanning allows us to stay ahead of the curve and mitigate compliance risk by anticipating regulatory updates.
The importance of organisational transparency and training
With AI being at the heart of what we do, it’s important that as an organisation, we are always operating in line with this ethical framework across all aspects of our operations.
Firstly it’s important that all of our staff are kept up to date with the latest developments in AI, regardless of their role. We run regular training sessions to ensure our team understand the tech they are using, with clearly defined roles and responsibilities within our engineering team to ensure there is always 100% accountability.
Upholding transparency with both our clients and customers is also incredibly important. We always strive for clear communication that discloses data collection and how we are using data in accordance with the latest regulations.
Embedding ethics into our everyday work environment enables us to stay true to values when building new products, and features and implementing any changes to existing models.
Embracing concerns and upholding best practice
Used correctly with the right guardrails in place, AI is the future of finance — including the credit landscape.
With the ability to employ intelligent vulnerability detection, streamlined customer service and end-to-end personalisation — AI, and especially AI-native platforms, hold the ability to improve everything from operational efficiency to cutting costs.
But even as an organisation that has readily adopted AI, it’s important not to shy away from the issues presented by this developing technology.
Any company utilising AI has a responsibility to uphold best practices to ensure its ethical use — a concern which extends far beyond engineering processes.
Want to discover more about Ophelos’ AI-native platform and how we could help you revolutionise your collections process? Book a demo with our team today.
