Not too long ago, anything to do with analytics was assumed to be a batch job that would be processed sometime well after any given set of events occurred. These days, however, within the context of a digital business, organizations want access to analytics applications in real time. To enable IT organizations to achieve that goal, Streamlio today unveiled a namesake event-driven platform that unifies multiple open source technologies.
Streamlio CEO Lewis Kaneshiro says Streamlio integrates Apache Pulsar messaging software originally built by Yahoo with a Heron data streaming engine originally developed by Twitter. To provide a set of storage services, Streamlio has also incorporated Apache BookKeeper, streaming storage software that Streamlio has infused with the DistributedLog system also developed by Twitter. All three core capabilities are being integrated on top of an instance of the open source Kubernetes container orchestration engine, says Kaneshiro.
Beyond simply integrating multiple open source projects to create a real-time, event-driven platform to drive Big Data analytics applications, Kaneshiro says Streamlio is also incorporating machine learning algorithms originally from Microsoft to continually optimize the overall platform.
The goal, says Kaneshiro, is to eliminate much of the integration work an IT organization would have to do on its own to create an equivalent real-time platform based on modern microservices architecture.
“There’s too much friction right now when it comes to working with these technologies,” says Kaneshiro.
Obviously, Streamlio is not the only vendor making a case for an event-driven platform capable of support real-time processing. There are a host of options ranging from legacy monolithic platforms developed by Oracle and IBM to platforms based on open source Apache Kafka messaging software. Apache Pulsar itself is still an incubation project. But based on its usage at a web-scale company such as Yahoo, Streamlio is clearly betting that Pulsar may supplant rival approaches such as Kafka.
Whatever the outcome, the sophistication of event-driven platforms based on open source software would appear to be about to make a significant leap forward.
