EXCLUSIVE: Where Multimodal Meets Mobile: Inside a Bold New Phase of Fintech Innovation

By Sarat Pediredla, CEO at digital product consultancy hedgehog lab

The next big bang in financial services lies in the merging of next-gen AI and mobile tech.

According to The McKinsey Global Institute, generative AI could unlock between $200 billion and $340 billion in annual new value to the sector. Meanwhile, 73% of UK bank account holders used mobile banking in Q1 2024. Combined, the two technologies hold out the promise to unlock hyper-personalised banking experiences, products and interfaces.

However, even fintech trailblazers such as Monzo, Starling and Revolut aren’t there yet. Partly, this is because AI technologies such as Open AI’s ChatGPT need to be seen as a foundational layer for the technology. For innovation on an unprecedented scale and scope, fintechs should look to the next chapter of multimodal AI tools, with systems trained on a variety of complex data inputs, offering limitless potential for fintech product innovation.

Making the most of multimodal AI

As multimodal AI transforms our relationship with machines, it will unleash deeply immersive tech, allowing human-like management of live conversations and complex problems. For example, new virtual assistants could imitate human gestures or chat using images, videos, and financial documents. California software company Infer.so recently created what it calls the world’s first multimodal AI voice bot coupled with an AI voice generator from ElevenLabs. By synthesising speech in real-time, it improves user experience with seamless, authentic conversations. Fintech would be a prime use case for this innovation, supporting consumers with personalised advice.

Similarly, Germany’s Commerzbank joined with Microsoft, using the Azure OpenAI Service to create a new virtual assistant in its mobile app. The software, featuring integrated avatar tech, offers business customers ‘natural and engaging conversations’. A similar solution from AI platform Gupshup for Philippines-based neobank, Tonik, lends accuracy and a human touch to customer queries on the Tonik mobile app – enhancing customer care and saving the fintech an estimated $20 million over the next three years.

New opportunities in mobile tech

Mirroring the rise of multimodal AI has been the increasing ubiquity of mobile technologies. Mobile tech developments – not just handsets, but smart speakers, wearables, Connected TVs – offer fertile ground for experimental growth in the fintech sector.

The wearable payments sector alone – with an estimated market value of more than $55 billion in 2023 – means customers can use rings, key fobs or bracelets to pay on the move. Advances in biometrics, meanwhile, allow shoppers to use their face and voice for authentication, accessing payment methods with no manual input.

And the uptick in in-car payments systems, used to store secure payment data and enabled by fingerprint tech, is an example of yet more transformative mobile UX.

The personalised future of fintech

Most financial services companies have yet to leverage mobile and AI, and ChatGPT is just the tip of the iceberg. New levels of personalisation can be augmented by mobile tech, and the combination of the two is key to fintech’s evolution.

In a new era of multimodal AI, fintechs can reinvent themselves as vital lifestyle companions to the mobile-centric, time-shy consumer. Whether it’s voice notifications for budgeting or financial tips via smartwatch, AI-fuelled innovation can leverage the interest in new features such as Apple Intelligence to not just preempt but exceed expectations for next-generation consumers.

Augmented reality can also play a role. Its crossover with AI paves the way for new, scalable experiences that are highly personalised and responsive. Banks including BNP Paribas and Westpac Banking are already experimenting with smartphone-based AR features, such as allowing customers to review account activity in a VR environment, or robot advisors that provide visual cash flow analysis. Add in real-time info, or customised design, and it’s clear to see how this type of innovation can gain traction in fintech and with customers.

There are caveats. Regulators are increasingly looking at AI, though the hope is that with banking already such a regulated environment, innovation here will not need to be stifled. Multimodal AI is also a cost-intensive journey, involving complex data management, so going in with a clear vision is key.

However, the potential in the overlap of emerging genAI and mobile tech is still huge. Many fintechs right now don’t fully understand the scale of the challenge, or lack the talent or ability to invest. But those willing to be the first to ride the wave will reap the rewards, with a new class of highly personalised, responsive solutions for customers with big expectations.