As a technology, enterprise search sounds like something that would be a great idea. But as experience has taught many an IT organization, it often winds up being a lot harder to implement than most IT organizations bargained for. Alas, circumstances often conspire to make the need for some technologies almost unavoidable, which is increasingly becoming the case with enterprise search thanks mainly to the rise of Big Data.
IT organizations are beginning to collect orders of magnitude more data. Collecting data is one thing; however, making actual use of it is another. Enterprise search clearly has a role to play in terms of making Big Data accessible; the challenge is doing it in a way that other applications can utilize.
To address that specific issue, LucidWorks this week formally released LucidWorks Big Data, an application development environment that brings together Hadoop, Mahout, Hive and Lucene/Solr enterprise search technology in a way that is designed to make these technologies accessible to the average developer by using a single RESTful application programming interface.
According to LucidWorks CEO Paul Doscher, most of the frameworks available for building Big Data applications are incomplete, arcane and don’t particularly scale well. LucidWorks Big Data is designed to make building Big Data applications that embed enterprise search functionality available without developers having to be worried about all the underlying plumbing.
LucidWorks is not the only company looking at ways to make Big Data more programmable. But it is one of the few committed to doing it around an open source technology model, which at the very least means that when it comes to the cost of experimenting with Big Data, the price is probably right.
