Jacob Leverich
Yeah, I mean, so I guess one of the, maybe one of the mistakes and it's sort of near and dear to my heart is, like 10 or 15 years ago, it was a case that logs were so expensive and just dealing with like large scale, like event management was so expensive, it's absurdly expensive that naturally people are like, this isn't going to work, I can't do that. And so it went like hardcore onto metrics. Makes perfect sense. Like it just makes so much sense to like, like particularly for a lot of just kind of real-time monitoring use cases. Yeah, I mean, I don't need the raw strings anymore. I can have these aggregated sort of signals to work off of when I'm doing my learning. It makes perfect sense. But then I think people kind of maybe swung the pendulum too hard. And now I can only see things at an aggregate level, but I can no longer figure out what's going on at the individual user level or at the individuals host level or individual. Yes, just anything like that. And so people talk about the cardinality problem, but I think when people are talking about the Cardinality Problem, it's because they swung the pendulum so hard towards metrics because at their scale, there was no event management system that could possibly serve their needs. And I think the truth is the right answer is somewhere in the middle. You want to use metrics for some things, but then for the deep analysis, the deep troubleshooting, you actually want to look at things at a more granular level. You need something that can handle the events as well. I think that's just something that I've seen a bunch of times is folks have just like swung the pendulum over so hard and just sort of abandoned sort of the more granular data because they assumed that it was too expensive, but like that, I think their dynamics that have changed significantly in the past decade. And so that's a big one. And then I guess the other thing is that, people tend to like when a group in teams grow and when lots of different job functions get involved, everyone kind of brings in a different tool to kind of do their thing makes perfect sense. But it like kind of at some point, it sort of precipitates like kind of some of the problems I was talking about earlier, it's like, Hey, the site's down, but there's like 10 different like tools we have to like, look at this thing. I unfortunately don't know how to use nine of them. So like, what do I do? I got to page my friend and tell him, get him to wake up and help me out. And it's not just like on the technical operations side of things. I mean, this is like support and usage analytics, and it could be FinOps, it could be security analytics. Like it's all these different like parties that actually have invested interest in the operations of the system. They all have like their own needs and they tend to, we need to like actually work with kind of monitoring the systems or monitoring this data. And so you end up like duplicating the data, you end up like sort of making just like just a hard trade-offs and sort of compromises. Like one team sort of is at the mercy of the other. And so people run into a lot of trouble with that kind of stuff. And so I think that's just a fact, right? That happens. I'd like to think that what we're building is sort of the start of the answer to a problem like that. If we can load all this data into a data lake, load it once, and we can give a lot of different people access to that data and for what it's worth, like, Hey, if it's in the data lake like, hey, bring your tool to the data, like this should actually like ideally be an ecosystem around it. And I guess what I see is, kind of long-term, this leads to a somewhat of a disaggregation of this market. And we've already seen this with the data collection and instrumentation, for the longest time, I use Fluent D for years before I even heard hearing this Splunk. And yeah, thank God open telemetry has like started to gain traction. And now like everyone can sort of see like, oh, this is my way out of kind of proprietary vendor specific instrumentation and data collection and sort of like, we're seeing a commoditization of like the data instrumentation data collection layer. And that's great because that kind of means like, okay, so cool. The data collection side, I can actually send the data to two places. And so if I have two different people that want to consume this data, that's fine. And I think that the next shoe to drop there is on the data management side. As well, it still is a very expensive to load this data into all these different systems. And so like that kind sucks. I'd rather economize that and load it into one system, load it once. By the way, like I'd rather not load it onto a silo. I'd prefer to load it someplace where I still actually own the data. I can still actually do whatever I want with them. I can sort of apply my own governance policies on top of it. It's sort of like that seems to be kind of what most people would want. And so the last piece of the puzzle to fall into place is like, okay, so now we have open data, like open data collection. We have it stored in like data lakes, ideally in open data formats. And then sort of now it's like, okay, now it a place that other all the point solutions for the engineering operations or for customer support or for security, they could all come to it. And sort of kind of put just their own, like, personalized view on top of it. But it's not like the only one person has access to that data. that's kind of, I guess, the way I see that unfolding over time.