15+ Years of Loading Threat Intel into SIEM: Why Does This Still Suck?

Unfortunately, I am old enough to remember how SIEM was done before the arrival of threat intelligence feeds. We had to write broad behavioral (well, “behavioral-ish”, if I am totally honest) rules without relying on any precise knowledge of attacker infrastructure and details of their operations (IF event_type=exploit FOLLOWED BY event_type=config_change ON the same machine THEN alert).

Another choice was to write simple atomic rules on obviously bad single events (IF event_type = logs_deleted THEN alert). Detections involved the patterns we observed (rarely, but we did have honeypots and IR), hypothesized (more often) and sometimes made up in the lab. Back then, detections rarely were built from precise and detailed knowledge of malicious activity.

Arrival of threat intelligence (TI) feeds in 2005–2010 has helped with some SIEM problems, but possibly made others worse. It definitely made some people lazier and made them rely on “OMG, look mom, a bad IP!” or other fragile indicators (you can hash this out on your own) from the bottom layers of the pyramid of pain. Their detection became a bit faster, a lot more fragile, a bit broader (with some luck, and with enough TI vendor spend), a lot noisier (“hi, feeds with 97% false positive rates!”) but also a bit more… I dunno.. “narrow-minded” (“not in my threat feeds? So not a threat!”)

Thus here is a notable point: technical TI have pushed many SIEM operators to lower levels of the pyramid, perhaps unintentionally. As a sidenote, when smart people offered them “strategic TI”, they could not find an obvious hole in their SIEM where it could be shoved, so they didn’t use it…

Obviously, TI brought obvious good news too: it made things better by relying on rapidly developing data about the attackers and their activities provided by the threat feeds. In a sense, TI “corrupted” SIEM, and as we know, many corrupt systems continue to function, and occasionally improve. Then again, maybe this metaphor is a bit much, as TI+SIEM did bring a lot of good news (especially on the alert triage and investigation side, TI as context, etc)

Finally, I also noticed that this area is where the “romantic dreams” of many SIEM operators clash with reality on the ground the most. Naive “get TI -> shove into SIEM -> detect threats” crowd shows up at the bar, drinks all the vodka, and leaves disappointed … ok, my metaphors suck today, Gemini, help?

So, Anton, the point? Enough with the rants already!

How to make real work closer to the perfect world?

What can we do to make TI work better inside your SIEM?

Specifically, how can we detect better using TI?

Before we get to the current answer, let me present my 2019 answer: “Detecting Threats by Matching Threat Intel to Logs — Oh Really?” (Jul 2019). I suggest you go read it, especially if you are at the “tieing my shoelaces seems really hard” stage of your development with SIEM and TI (or if you like my jokes…). We don’t want people creatively repeat 2015 mistakes in 2025… especially because there are 2005 mistakes to be repeated?

So I want to push this forward. And I will do this with a table, an ugly table that Anna used to love so much… Here it is: threat intel in SIEM in the real world vs the unicorn utopia.

OK, what’s next? How do we journey from the real world to a magical world where TI is beautiful and effective? This will be covered in the next blog!

Related posts:

• “Detecting Threats by Matching Threat Intel to Logs — Oh Really?”
• “About Threat Intel Retro-Matching”
• “Focus Threat Intel Capabilities at Detection Engineering (Part 4)” and the rest of the series
• “Blueprint for Threat Intel to Detection Flow (Part 7)”
• “Back to Cooking: Detection Engineer vs Detection Consumer, Again?”
• “2021 Threat Intelligence Use Cases”

Useful reading:

• Awesome Threat Detection
• How Google Does It: Making threat detection high-quality, scalable, and modern
• Implementing a Modern Detection Engineering Workflow (Part 2)
• On Detection: Tactical to Functional: Part 11: Functional Composition
• Using Google Threat Intelligence to create behavioral detections as a detection engineer