Big Data generated by retail, social media, and the Internet of Things (IoT), as well as the challenges of storing and analyzing such data, are now commonplace ideas within the IT landscape. What is less common is the discussion about Big Data’s impact on networks, both today and in the future.
Understanding the interplay between Big Data and networks, including the Internet, is an important part of planning your approach to network architecture and network operations management.
In this slideshow, Kentik has identified five ways Big Data is changing how networks work.
Big Data’s Effect on IT Networks
Click through for five ways Big Data is changing IT networks, as identified by Kentik.
Traffic Patterns
Network traffic patterns are going more horizontal.
Big Data is primarily processed in large batches and involves huge data transfers between server clusters within and often between data centers. This means that the classic north-south network traffic pattern stemming from client-server interactions is increasingly being altered by large east-west traffic patterns.
Big Data can create dramatically more demand on your network infrastructure. Make sure that you’ve factored Big Data flows into your network capacity plan. Understand how your switches and routers behave under more frequent traffic peaks and valleys, since those big traffic flows will come intermittently.
Traffic Management
Network traffic management becomes more important.
Preparing for Big Data means more than just increasing network bandwidth. Very large traffic flows can dominate all other traffic, similar to the effect of backup routines. Unlike backup routines, which should run off-hours, Big Data processing will be a normal business routine that will be performed at any time of the day. It will be critical to manage the interaction of Big Data flows with other traffic to prevent undue impact on latency-sensitive applications such as unified communications.
Update your class-of-service (CoS) management design so that it is appropriate for the presence of business hours Big Data flows.
IoT and DDoS
IoT and DDoS will force you to interact with Internet Big Data.
Big Data isn’t just inside your data centers. Big Data is flowing into your network from all over the Internet due to sensors that monitor and control far-flung assets, facilities and processes. In addition, the ever-growing presence of botnets and other Internet predation means that you are probably already sustaining DDoS attacks, whether you know it or not.
Plan for both IoT and DDoS inbound traffic to increase dramatically. Train networking staff on Internet routing so that they can better manage (and mitigate) legitimate and attack vector traffic coming in from the Internet.
Cloud Networking
Cloud networking expertise is increasing in importance.
Data is becoming so big, and periodic batch processing is frequent enough in nature that it’s increasingly infeasible to equip private and on-premises data centers with the capacity to handle the peaks of demand and gain a reasonable return on capital. This and the general rise of public cloud and SaaS applications are creating a hybrid cloud reality. Along with that comes the necessity for greater cloud networking expertise.
Understand how you are connected to your cloud providers. If you don’t have redundant Internet connections via different ISPs (multi-homing), you should strongly consider doing so. If your business is large enough and critical applications are increasingly residing on public clouds, you should also consider developing a direct peering plan for your Internet traffic.
Network Monitoring
Network monitoring data is Big Data.
Big Data can look like massive batches of retail data, but it can also look like millions of flows of IoT data and web interactions. As the networking industry has increasingly relied on traffic flow data to understand these dynamics, the growth of that monitoring data has soared with the capacity of the networks themselves. Network monitoring such as NetFlow can generate millions of data records per minute in some cases. Accumulated over any period of time, this in itself becomes Big Data. Networking groups need to have a strategy in place for handling these massive volumes of data, while retaining enough detail to enable effective forensics and informed operational decisions.
Ensure that you have sufficient capacity to have both summary and detailed views of your network monitoring data.
