IT Architecture: event-streaming for data liberation

In the banking industry, whether to advise and personalize the customer experience, assess credit risk or meet regulatory requirements, data is at the heart of products and relationships. At Ditto, we are building a brand-new data-centric platform that will put our banking and technology services at the service of companies that want to enrich their offering and/or improve their customer experience.

Our product and development teams have set themselves a very specific requirement in order to achieve this objective: to use the latest technologies (as the web giants such as Instagram, LinkedIn or Netflix do), and to base this platform on the method best able to guarantee, by design, resilience, scalability, flexibility and compliance: event-streaming. 

1 – The “event”: the reference unit of IT architecture in event-streaming

Before we look at what an “event-based architecture” is, let’s look at the definition of an event in an IT context. Any new information (new client created in the system, client that connects, changeover to daylight saving time), update, request … within a platform is considered as an event. In simple terms, an event occurs when “something happens” on the platform and these events have repercussions throughout the system. 

Thus, setting the event at the heart of the architecture implies tracking additions or new information in a very precise way and keeping track of evolutions, just like an accounting ledger. Each event is stored in the same place which keeps the history of each previous entry. The value ? To guarantee  real-time information sharing for all the different entities of the platform, and thus its users. This accurate tracking of all changes to the platform also ensures that no data is lost, making the platform much more reliable and repositionable at any time in the past to audit, detect the origin of potential errors or simulate different scenarios.

2 – An event-based architecture: a “giant chat” to liberate the use of data

To be able to put the event at the heart of a tech architecture, it is necessary to be able to register them in a given place, which will serve as an official and universal source.

So, imagine a “giant chat”, like corporate instant messaging, organized by major themes and informing all the entities of the platform via push notifications.  

The revolution is profound. If data was previously available in the operated service and closed to other entities of the system, it is now part of the system itself and is therefore accessible to all. In this state of mind, the sharing and above all the use of data is widely encouraged since this is what will enrich it and bring added value, in addition to the time saved thanks to easy accessibility. It is indeed the door open to new and more efficient services, developed faster and more autonomously.  It is also a guarantee of greater scalability: as the data is centralised, the creation of a new service can easily capitalise on everything that already exists in the system.

3 – Kafka: the original and most widely used event streaming architecture management platform 

At the origin of its creation is an amazing story. Back in 2009, at … LinkedIn! The teams, faced with problems integrating data from the various systems in place within the company’s infrastructure, set out to find an efficient way to centralize these data flows. This is how Kafka, a distributed messaging system, was born! Ten years later, it is THE most advanced and proven solution on the subject of data streaming. At Ditto, it was the natural choice to build our new platform. 

Not only LinkedIn, but also Microsoft, Netflix, Twitter, Spotify, Uber … the number of companies using Kafka is growing, sending trillions of messages per day in the dedicated cluster at LinkedIn. 

*******

This release of the use of the data does not in any way prevent the imposition of access rights. Even better, such an architecture allows intrinsically, (or by design) the auditability and traceability of events.

In business terms, the new opportunities for value creation are almost infinite and some of them can already be imagined in a very concrete way.