There are a ton of enterprise security products, from intrusion detection to data loss prevention to zero-day attack monitoring, that are based on monitoring logs for weird behavior. In fact, any serious discussion of enterprise security always boils down to carefully monitoring your server logs. You can employ high-end applications to do this for you, but ultimately your IT team needs to know how to protect and utilize the logs that spell out virtually every aspect of your network's health.
The National Institutes of Standards and Technology has prepared a Guide to Computer Security Log Management that covers the full gamut of issues relating to monitoring and - perhaps more importantly - securing your log data. The 72-page report is available for free to IT Business Edge members here in the IT Downloads library.
Like all NIST reports, the PDF includes an executive summary (in this case, three pages) that covers the broad range of topics covered in the document. The report's high-level recommendations include:
Organizations should establish policies and procedures for log management. Seems obvious, but some shops simply do not address log management at the policy level. Generally, organizations should require logging and analyzing of critical data, with other log analysis being optional. Policies should also account for specific regulations, such as Sarbox and HIPAA, to which they may be subject.
Organizations should prioritize log management appropriately throughout the organization. This will likely fall to IT in most cases, but depending on your corporate environment, legal or another department might share in the ownership of some log analysis.
Organizations should establish standard log management operational processes. As with all ongoing business processes, there's a right way and a wrong way to tackle log monitoring. Among tactics to consider are ensuring that each logging host's clock is synched to a common time source and ensuring that the process keeps up to date with regulatory requirements.
The report then goes into 60-plus pages of deep detail on the basics of log structure and some advanced management tactics, including advice on how long to retain logged data, as you can see in the image below.
It's important to note that since so much of what you rely on log data for is operational, you can archive it fairly quickly when compared to other data types, such as email. The report stresses the importance of verifying log transfers and, of course, keeping data in a secure location with appropriate redundancies.