Yet, while this customer relies on data for all lines of business, they found that batch reporting with data at rest was unreliable, costly, and didn't allow for real-time reactivity. Their system often offered outdated reporting on inventory, affecting their global supply chain, logistical operations, and ultimately, their customers.
They decided to adopt a Kafka framework on-site to leverage real-time data. They quickly found, however, that open-source Kafka is not only challenging to implement, but also requires full-time maintenance and infrastructure management. This resulted in difficulty scaling and no Kafka support for their path to the cloud.
An IT Architect and CTO of one of the customer’s service lines explained, "One of the challenges with Kafka was its operational complexity. We had to allocate a lot of our valuable technical resources and expertise to babysit it and keep it running. Nor is it a cloud-native data system, so it could not scale and handle the volumes of data that we needed to process."