In his post On the difficulties of event correlation, Ben talks about how hard event correlation is - and I couldn't agree with him more. In addition, I am often surprised about how many organizations blindly run down the path to adding more event collection to their environment before they understand the ones they already have.
A great example is IDS deployment. Along with the rest of the infosec establishment, I generally agree that enterprise-wide IDS can be an important part of a comprehensive infosec program. However, I often see that such deployments don't succeed in the long run. Everyone is excited about it initially, but after a few months their interest wanes and the platform falls into a state of disrepair. Why? Is it because IDS data isn't useful? Not at all. Instead, I think these are the drivers:
- Poor event correlation. Ben makes some suggestions as to why, but the bottom line is that it's really hard to use data that isn't correlated with the rest of the environment.
- Failure to budget staff resources to maintain the platform -- there is an additive staff cost of deploying IDS, even if 7x24 monitoring is outsourced.
- IDS events are not on the top of the list of "event value." By "event value," I mean that eventually, folks realize that there are more important events that they're not capturing. Events like "server down," "disk full," or "network linked failed." If they're not reporting/handling these higher value events already, adding a high quantity of lower value events results in them being perceived as noise.
This last item is really important and apparently not obvious. The bottom line is this: before deploying a platform like an IDS, first make sure that you're already capturing and managing the interesting, high-value events that currently exist in the environment. OS and infrastructure device logs are already easily available, take time to capture them centrally and use them. One that's been mastered, then it's reasonable to take on the IDS event stream.