The rapid growth of consumer technology and steadily falling unit costs for processors, storage and communications have resulted in a major disruption to the data that is potentially available to organizations. In big data infrastructure, file systems need a layer of abstraction over them to allow for ease of scale and rapid data processing. These technologies include file systems such as GFS, HDFS and Lustre, which increasingly form the core of these new environments. Management is needed, as so-called worker nodes in file systems like HDFS can come and go, and checks need to be made to see if the rate of node failure is too high to get the necessary work done.
Future IT leaders will need to seek technologies that eliminate silos in order to deliver the right information to the right person within the right application environment at the right time. ... More >>