The Challenges of Running Containers at Scale

    Slide Show

    Data Back Up, Recovery and Erasure: A Guide for Corporate IT

    It seems the enterprise is approaching container technology with a mixture of anticipation and trepidation as it seeks to establish architectures that offer broader scalability and are more suitable to microservices than standard virtualization.

    But the growing number of deployments is starting to point out the challenges inherent in container-based data environments, although it appears that most of the issues can be overcome by a proper management stack and a reasonably good understanding of what containers can and cannot do.

    At the moment, much of the momentum behind containers comes from developers, says’s Clint Boulton, while CIOs and other C-suite executives are a little more wary. At a recent Wall Street Journal gathering, Docker CEO Ben Golub focused primarily on the technology’s ability to support cloud-based app development and testing even as an online poll showed a fair amount of skepticism of containers’ value proposition and whether it could do anything that simple virtualization or platforms like Red Hat’s OpenShift could not. One key advantage that containers bring to the table is that they do not rely on a guest operating system, which in turn should provide a more integrated change management structure to enable the kind of continuous delivery and integration required of cloud-based apps and services.

    Still, enterprise executives should be prepared to face three key challenges when deploying containers, says Red Hat’s Lars Herrmann. The most potentially disruptive is fragmentation as services and app components become disseminated across disparate infrastructure. Fortunately, this is also the most easily addressed by uniform image formatting and standards-based orchestration. As well, organizations should beware of partially open or fully closed (fauxpen) platforms that contain proprietary code despite their use of the Linux label, and even full-blown “container washing” in which stripped-down VMs are presented as containers. In both cases, the full truth of the technology might not become apparent until after they are deployed, so CIOs need to check carefully before buying into a particular platform.

    Containers are only marginally useful until they are deployed at scale, so it is important to create a robust management and orchestration layer at the outset, says Enterprise Tech’s Doug Black. The problem here is that orchestration software can become overly complex in itself as the container environment scales from a few small clusters to hundreds or even thousands. Commercial management platform companies like Avi Networks and Mesosphere are at the forefront of this new class of software, but organizations should still keep an eye on how they intend to manage tasks like load balancing, performance monitoring, service discovery and micro-segmentation, even when the container environment is relatively small. If you think you have problems with the multiple management stacks that currently populate the data environment, you ain’t seen nothing yet.

    All of this means that while containers are very handy in the lab, the real challenge lies in scaling them into production environments, says Forrester’s Dave Bartoletti. At the moment, only about 8 percent of developers are using the technology for production workloads, and many are already struggling with security, performance consistency, integration and other issues. It’s important to remember that containers are only one way of engaging the modern data user. To accomplish a complete digital transformation, organizations will have to meld containers with dev/ops, cloud services and altogether new application architectures to provide the kind of functionality that customers and knowledge workers demand.

    Problems aside, however, it seems clear that containers are the future of application and service delivery. This isn’t to say they are the answer to everything, but organizations that wish to remain on the cutting edge of the new digital economy will have a tough time of it without a far-reaching container strategy.

    But as with any technology initiative, understanding the potential downside is often more important than anticipating all the advantages.

    Arthur Cole writes about infrastructure for IT Business Edge. Cole has been covering the high-tech media and computing industries for more than 20 years, having served as editor of TV Technology, Video Technology News, Internet News and Multimedia Weekly. His contributions have appeared in Communications Today and Enterprise Networking Planet and as web content for numerous high-tech clients like TwinStrata and Carpathia. Follow Art on Twitter @acole602.

    Arthur Cole
    Arthur Cole
    With more than 20 years of experience in technology journalism, Arthur has written on the rise of everything from the first digital video editing platforms to virtualization, advanced cloud architectures and the Internet of Things. He is a regular contributor to IT Business Edge and Enterprise Networking Planet and provides blog posts and other web content to numerous company web sites in the high-tech and data communications industries.

    Latest Articles