The Future of Threat Intelligence With Industry Veteran Errol Weiss

Errol: Right. I think as we evolved that team at Citigroup starting back in 2007 and 2008, the thing that became really apparent to me was about building an organization that looked like the intelligence management lifecycle. When they need to build out a function like this, a lot of people think they need to just staff up a lot of analysts to consume all of this information and then kick it out to their internal consumers or the management. But it’s not just about analysts — it’s everything else that we have in that cycle. You can look in any military intelligence or any threat intelligence periodical and you’ll see that continuous lifecycle of requirements: collecting, analyzing, reporting, disseminating, and add the feedback loop in there, as well.

The other piece of it is, with the type of work that we do, it tends to be very tactical at times, very event-driven. So, learn the incident management process, become an incident responder, use those incident response skills. Those were always really helpful for us. But in terms of that lifecycle, the analysis team is my production line. They are the ones that I want focused on producing that intelligence. Everything from these tactical reports that we do and then, as you evolve and mature your organization, you’re going to be getting to more strategic items, too. And you’ll have consumers internally that are demanding these kinds of things.

I mentioned collection. As the organization grows, you might need a full-time collection manager, and this person is the one, for me, who is responsible for understanding what all of our intelligence requirements are, managing them, and tasking all of our intelligence assets. And then, the secret to success for me over at Citi was building out what I call the “Client Services Organization.” The whole idea was to really connect the intelligence that we were producing to the internal lines of business and internal consumers inside the company widely.

The whole idea, again, is to ask, “How do I turn and make that intelligence actionable and relevant for these internal consumers?” When you look at the skill sets, analysts are very different than the type of people that I need who are doing personal engagement and making those connection points, and working with them to understand what their needs are and how to get intelligence to them. We put a lot of work in for that.

Errol: A really tricky problem, right? And a whole notion of metrics. It’s been a tough problem to show the value of what we are delivering. I’d say, there are a few things you can do. One, in the banking and finance sector, we can use threat intelligence to mitigate risk for our customer accounts, shut them down if necessary, or reissue credentials. We can actually take a dollar amount and translate that into a potential fraud-loss avoidance.

The harder part is trying to quantify literally everything else that we do. And I started off, in the beginning, doing things like looking at malicious IP addresses, or malicious URLs that are embedded in emails and reporting those to our defense organizations, and they implement blocks on them. And to me, it always felt kind of funny — I get more sources, and so those numbers go up. Summer comes and people go on vacation, and then, numbers go down. Or stop paying for a source, it dries up and numbers go down even more. Is that good or bad? I’d have management asking me, “What would that mean?” and I have to answer “I don’t know.”

What I do measure, now that we’ve gotten a little smarter about it, is trying to get feedback from the defense organizations saying, “Hey, I sent you 10,000 IPs last month. What was the result of that?” So, now I can get statistics from them on how many blocks they actually implemented or how many blocks they were able to do based on the information that I told them. To me, that’s one of the value propositions. And again, I’m trying to stay away from the, “Is it going up or down?” Doesn’t really tell me anything, but it just shows the efficiency there.

The other measure for success is all about the narrative. I use that client services organization to gather information from our intel customers to understand, “Hey, what did we do for you that was great this last month?” And if I can get one or two a quarter, that to me means we’re doing pretty good. These are hard. These are long-term issues.

Errol: So, good news, bad news here. When I was thinking about this question, I broke it up into two major pieces. One, thinking about the intel market itself, and then, the threat. Thinking about the market, there’s a lot of companies out there that say they do threat intelligence, and I certainly see a lot of consolidation coming up in that market. I think the reason for that is, the ones who are using and embracing automation are the ones who will survive. And the ones who when they have to hire somebody because they just got another client, or they have to hire 10 more people because they got a big client, they’re gonna have problems. That’s the dynamic we’re going to see out there. And it’s unfortunate because I love this field, and I love some of the niche players that are out there. They have things that I can’t get anywhere else, but I definitely see it moving toward a commoditization type of service, because they can’t scale through that automation.

But what we need to do better, back to the automation piece. We’re always looking for more automation — how can we make this more efficient, more effective? We’ve got to get the humans out of the cut, copy, paste business, and believe me, as many resources as we have, and as many people as I’ve got on the development side that are trying to help us with workflow and automation, we still suffer from cut, copy, and paste.

I’m hoping that things like machine learning will be a factor for us. And then, I also see a lot of threat intelligence organizations strictly focused on protecting the perimeter, protecting the infrastructure, and that’s all well and good. The issue I see there is, as we see better and smarter networks coming out, software-defined networks, some of the reality around that, we’ll still need indicators of compromise and all of the issues that we’re handling today to deal with that intelligence.

Errol: I think it’s kind of what I was alluding to earlier. I’m hoping that we’ll be able to get away from this massive indicators-of-compromise problem, as well as the issue of getting tons of different feeds from many different sources. We’re processing this stuff all day long and we’ve got quality problems in that list, where we’ll have ton of IPs in there and we’ll even see our own IPs sometimes. Reliance on that information is really tough. Whether we ever get to the point where we can use that information confidently and actually take block activity based on that — I don’t know, I kind of doubt it.

But I think as the technology evolves and we do see the network hardware getting better at being self-filling and blocking, I’m hoping we can rely more on that. Threat intel’s never going to go away. We’ll just have a different focus.