Tune in to this episode of Ask A CISO to learn:
- What do you do with cloud visibility?
- Understanding the attack surface
- Cyber hygiene
- Cloud sprawl and high cloud costs
- Top 5 cloud cost drivers
- How to help everyone in the organization understand cloud costs
About The Guest: Nick Lumsden
Nick Lumsden is Co-Founder and CTO of Tenacity Cloud.
Nick has more than 20 years of experience as a leader and technologist and is adept at rapidly growing, transforming, creating, and running organizations in cloud and healthcare technology companies.
He also has deep experience in operations, product management, and engineering, especially in Stage 2, rapid-growth companies.
Besides his professional hats, Nick is also an athlete, father, musician, woodworker, gardener, bookworm, geek, late-night coder, and frequent outdoor adventurer.
About The Host: Paul Hadjy
Paul Hadjy is co-founder and CEO of Horangi Cyber Security.
Paul leads a team of cybersecurity specialists who create software to solve challenging cybersecurity problems. Horangi brings world-class solutions to provide clients in the Asian market with the right, actionable data to make critical cybersecurity decisions.
Prior to Horangi, Paul worked at Palantir Technologies, where he was instrumental in expanding Palantir’s footprint in the Asia Pacific.
He worked across Singapore, Korea, and New Zealand to build Palantir's business in both the commercial and government space and grow its regional teams.
He has over a decade of experience and expertise in Anti-Money Laundering, Insider Threats, Cyber Security, Government, and Commercial Banking.
Transcript
Jeremy
Hello, and welcome to another episode of the Ask A CISO podcast.
My name is Jeremy Snyder. I'm the founder and CEO of FireTale.io, and I'm the host for today's episode.
Today I'm delighted to be joined by Nick Lumsden from Tenacity Cloud.
Nick is the co-founder and CTO of Tenacity Cloud. Nick has more than 20 years of experience as a leader and technologist, and is adept at rapidly growing, transforming, creating, and running organizations in the cloud and healthcare technology companies.
Nick also has deep experience in operations, product management, and engineering, especially in stage two rapid-growth companies.
Besides his professional hats, Nick is also an athlete, father, musician, woodworker, gardener, bookworm, geek, late-night coder, and frequent outdoor adventurer. Bit of a Renaissance man, I think it would be fair to say, Nick, and thank you so much for taking the time to join us today.
Nick
Thanks, Jeremy. I'm a bit, uh, eclectic in my hobbies, so thanks so much for having me on the show.
Great, great to be here.
Jeremy
I think it's awesome to be eclectic. Fantastic. I think it's awesome to have eclectic interests, and to kind of, you know, branch outside of the day-to-day of what we all do.
But I guess on that topic, I just wanna ask for our audience: Tenacity Cloud, tell us the story. You're the co-founder of the organization. What's the mission? What's the problem that you guys set out to solve with Tenacity Cloud?
Nick
Yeah, absolutely. So this journey started with myself and my business partner, who I have worked with for the better part of a decade now. And his name is Jason Yaeger. And it turns out, we, we found each other in a past organization where we're successful. We had an exit, and we're, we're yin and yang. We're the exact opposite people.
So where, where I'm a bookworm, he's an avid golfer, right? And where he likes fast cars. I like, you know, my old pickup and going fishing. So we're just a really complementary to one another.
What's interesting is that in our journeys, our separate journeys in our career, we both noticed a handful of issues that just kept recurring over and over and over throughout our career, no matter what the technology stack was, no matter where we were. And those issues, you know, all the way back into, even for me, the, the nineties, were around just really understanding everything in your environment and having your arms around it.
Jeremy
Mm-hmm.
Nick
Do you have visibility? Do you have transparency into it. Doesn't matter if you have a hardware stack in a closet somewhere, or if you're now executing code against AWS to quickly build up and tear down your infrastructures. Do you know what you actually have? Do you know how it's configured? Are you aware of it? Do you know where potentially there are vectors of vulnerability, commonly are hidden just due to the scale of an environment, even modest scale environments, it's hard to understand all the potential points of issue or interest.
And then, you know, it. how does that map to all the things I'm obligated to do as a CIO a CTO a CISO, or anyone throughout the organization. that's, you know, helping protect, whether it's healthcare data or someone's sensitive information, or maybe it's just trade secrets about the business. Those problems have existed over and over and over, and we
Jeremy
For sure.
Nick
found Tenacity to go solve some of those problems.
Jeremy
Okay.
So you kind of started that journey with that focus on visibility and I
Nick
Mm-hmm.
Jeremy
I, by the way, I kind of a hundred percent echo that sentiment, and I've had that same challenge from my early days of running data centers back in '97. You know, I went through the whole transition from data centers in my office to data centers at a co-lo provider to managed kind of servers with Rackspace.
And then, you know, I went to work at AWS in the early 2010s, and I've seen the transition on, but really throughout that entire journey, I would a hundred percent agree with what you're saying there. If you don't have visibility onto it, how can you have any level of confidence as to its security posture as to how vulnerable it may be, et cetera.
So when you kind of started that journey, what were the first kind of elements that you said, you know, okay, we're gonna focus on visibility, but what's the next phase after that? Because visibility's important, but you know, it just gives you a list. What do you do with that list?
Nick
Yeah. Yeah.
So we headed down a couple of paths and I'll talk about that and I'll talk about our pivots.
So the very first thing we said is we came up with, you know what, well, what are kind of the, what are the pillars that your infrastructure is built on? And when we kind of took apart whether it's working in large organizations, you're a part of enterprise architecture or if you're in a small organization, you're just kind of thinking about your CCoE or your governance, risk, all the areas you have to look at.
There's kind of four pillars, and those four pillars are your asset inventory, just what do you have? Do you auto-detect it? Do you know what it is?
Second was, you know, really around cost and attack surface and optimization of the environment. Do you know what's actually out there? And is it supposed to be there? That's a big component.
And then the third part was around posture. How is it configured? And the fourth part is, can you map all of that out to your compliance obligations, your regulatory requirements, or your best practice obligations, maybe even your own internal policies and procedures.
So that's where we started. We started with actually
Jeremy
Okay.
Nick
building out a solution for those four pillars.
Jeremy
Okay. Okay.
And you said there's been some pivots over the years. Talk us through that journey. cause I'm sure there was some really interesting lessons learned along the way.
Nick
We did, we learned that some, you know, not all problems are equal.
And so we found out during this journey that the hardest part to get right and it's one of the first tenets really, when I go back to operating in highly secure environments, it's one of the first tenets is I have to get my actual base infrastructure correct. I have to understand my attack surface.
Jeremy
Okay.
Nick
And the biggest enemy to running a great cybersecurity program is having a whole lot of junk in the environment that creates noise and buries the actual signal.
And so we decided to narrow our focus. We've actually pivoted into focusing on how do you optimize your actual cloud environment? How do you get rid of legacy infrastructure? How do you find what's been abandoned and orphaned? How do you find what's been over-provisioned in? And how do you narrow that all the way down to just what you need and then keep it operating in that prime status.
Jeremy
And so, you know, a term that really comes to my mind as you describe that is hygiene. And I've heard, I've talked to a lot of CISOs on this podcast and elsewise throughout my career, and it's one of these kind of overlooked, neglected practices that I think a lot of organizations kind of think of as not being high value-add.
Everybody likes to create, nobody likes to go clean up after themselves. And yet, you know, not having good cyber hygiene can lead to a lot of risk for the organization.
So when you kind of thought about approaching it, was that a concept that came into a lot of your conversations or a lot of your own kind of product designs? And how did you kind of, you know, convey that message to customers who were looking at addressing some of these problems?
Nick
Yeah. It's absolutely about hygiene, and I like to use this analogy.
It's about something, you know, something we don't even think about personally. You and I don't even think about it anymore because it's so fundamental to our career that we get up in the morning, and we take a shower, and we brush our teeth, and we comb our hair, and we get our haircut once a month or whatever it is.
Those things don't even enter our thought anymore because they're foundational to having success and being successful in a society. And we haven't yet recognized that in cybersecurity or more broadly, just in technology, in general, of having good hygiene should be as routine as brushing our teeth.
Jeremy
Mm-hmm.
And so you make that analogy to customers. I can imagine that they don't always pick up on that right away. Do you think that the transition to the cloud with everything kind of being on a utility pricing model has started to make that pain a little bit more acute?
Because nowadays if you've got bad hygiene, you start to get sprawl in your cloud accounts, your bill goes up.
Nick
Yeah.
Jeremy
Right?
And so all of a sudden, from a fixed monthly cost, you're now looking at variable pricing and you're looking at like, Oh crap, this month is 20% more expensive than last month. What's going on here? Has that helped the conversation at all, or has it hurt, or some of both?
Nick
It's a little bit of both. And so the absolute motivator for an organization is the fact that they're missing out on upwards of 50% savings on their bill, that they're paying twice as much as they need to. That's the absolute motivator if you just brush your teeth in your cloud environment, you can save a lot of money.
Now a lot of them, you know, hit the easy button and they will use some sort of discount vehicle or commitment vehicle to save themselves 10 or 15% on the cloud bill, they'll feel really good about it.
Jeremy
Sure.
Nick
They can now tell a story about it, but they missed the entire attack surface conversation of really understanding sprawl and what's going on in the environment. And that's an ongoing journey.
Jeremy
Yeah.
Nick
Optimization's not a one-time thing. You have to keep doing it. How the problem has gotten worse is that back in 1997, in the data center era, that you and I were both in, changes to that environment came quarterly, maybe? And when we bought hardware, we bought for four to five years of capacity which has its own problems.
Now, the change rate is dozens, sometimes hundreds of times a day, and that creates its own sorts of, of complexity, both from, you know, understanding what the change is or the impact of those nett changes on the environment, but also just in that your focus is on what do I need to change?
Because cloud infrastructure as code has become strategically advantageous to businesses and they're gonna leverage it to have actual power in their market where they're competing, then that's where their focus is. Their focus isn't on well, you know, what QA resources did I not spin down and now are sitting out there
Jeremy
Yeah.
Nick
in our, you know, ignored surface.
Jeremy
Yeah, and it strikes me, you know, I spent some time at AWS in the early days of cloud in 2010, you know, running around talking to CIOs and CTOs about this cloud thing and trying to convince 'em that it was real and I would, you know, get a lot of pushback.
And that I kind of saw some of that early cloud customers adoption journey going from, you know, not believing it necessarily. I call that, you know, month zero. Month one is the let me get a developer to kick the tires.
Month two is let me try to migrate one low-risk internal workload.
And then my month three problem that I used to get, you know, I get that call in month three from a customer saying, Hey, I don't understand my bill, or my bill is higher than I expected it to be. Are we still seeing a lot of those same behaviors now in 2022? We're still seeing a lot of that kind of friction as customers ramp up on cloud?
Nick
There is.
It is a different world and it's a different way of thinking about kind of your cost structures or your budget. Something that's highly variable in, in IT and in technology is not friendly to the CIO who's trying to plan their budget, right?
Jeremy
Yeah.
Nick
We tend to think in CapEx and amortization and, you know, it's a very fixed model. So that brings its own problems.
Now, the advantage now over 2010 is that there's tons of tools out there to help. Giving a tool that aligns to more of a financial way of looking at cloud. So meaning that it has business context. I think that's really critical. That's really important. Understanding, at the end of the day, this infrastructure's allocated back to some sort of revenue stream and being able to actually match that, understand it and have the business make decisions about it is really critical.
Jeremy
And do you think customers are starting to get that?
Cuz that's a promise that I think has been out there for a long time, which is to say like, Hey, I can directly attribute the cost of this workload against the revenue produced by that application. And I can calculate, kind of, you know, an application-specific ROI, positive or negative or what have you.
And yet, you know, a lot of the experience that I had from five years spent in the CSPM world is that customers have workloads all mixed up and they're not tagged properly, and they can't differentiate the bill from workload A to workload B. So it's always been kind of a promise rather than something that is a reality.
Are we starting to see any change there or any improvement or is it still kind of a mixed bag? What are you guys observing?
Nick
It's a mixed bag, but it is improving.
And I'll say, Oh, there's two things that are, are driving that improvement.
One is in the best, most disciplined customers and organizations we've seen, they understand why tagging is important. They understand that being the driver of business context. Now, there's still some issues in that tagging doesn't solve the problem of shared infrastructure, right? And sometimes there's just infrastructure that's really, really hard to attribute.
And so they still have to kind of solve those problems. I would say that those organizations, what they see is that the tools in the market that they've used historically, let's say, the past five years really aren't helping them with that need, right?
And so they're looking to the next generation of optimization tools, of cloud management tools to help them there. So there needs to be some intelligence, some AI, some sophistication around cost allocation and understanding infrastructure and being able to potentially where those costs should be aligned if they're untagged or if they're kind of ambiguous infrastructure, we need to support that.
On the other end of the spectrum are organizations that went to the cloud. They're either in the midst of a long term migration strategy or they're stuck, or maybe they've completed it, but the environment from, kind of, that discipline perspective has largely decayed and I call this the ball of spaghetti problem, right? It's
Jeremy
Yeah, sure.
Nick
There is nothing to do but implement it. You either have to unwind the whole thing, which is, is you just can't do from a business context, or you have to have tools that help you identify each noodle and understand whose plate it's supposed to go on.
Jeremy
Yeah.
And would you say ... What are some of the behaviors you see from customers who get themselves to that point where they are kind of stuck or where they have that ball of spaghetti, you know, what are some of the, what are some of the behaviors you observe and maybe what are some of the best practices for people who find themselves in that situation with their organization?
What would you recommend?
Nick
Yeah.
Those who are there who have that complete lack of understanding, overspending, don't understand their cost and their drivers. They are most often ... The characteristics or attributes they have is they're paralyzed. They're sometimes talking about repatriation, they're,
Jeremy
Bringing it back in-house or going back to a co-lo?
Nick
Basically. Yeah. Yeah.
We've actually seen customers having those conversations. There's some small margin, I don't know what it is, 5% or something, or workloads are being repatriated.
Jeremy
Sure.
Nick
But if you're going down that path, then they're missing just all the opportunities that are actually cost advantageous over on-prem. And they're really not understanding the power that cloud gives them from the ability to respond and be strategically more competitive in their market, in their app development, et cetera, the deliver to market.
And so I would say that they are stuck in kind of analysis paralysis and don't know what to do. And those who are doing something about it, are starting with, you know, how do I actually find the waste in my environment?
I'll tell you, we did, we did an install with an organization, I won't name the organization, but they started a free trial with us. Within 48 hours we had identified over 10,000 resources in AWS. They were mostly EBS volumes that were just abandoned. They just, they weren't connected to anything. They, they weren't doing anything.
Jeremy
Yep.
Nick
And their initial reaction to us was, No, your tool's wrong. It's totally impossible. There's no way that that is sitting in our environment.
And two days later the guy reaches back out to us and goes, Oh my God, you are right. There are all these resources. And they ended up saving almost 20 grand a month and reduced significantly the amount of surface in this particular area. And by the way, these were costs that couldn't be attributed. Like they were just overhead, which a lot of times their bill is more than 50% overhead.
They just don't know where it's supposed to go back to. You wanna be able to take that and eliminate all that. And so just, just being able to get insight. Just that glimpse that told them, here's the first place to go look, and it's the biggest impact. It got them excited. It got them moving down the path of like, Well, wait a minute, what else can we clean up?
Okay, well let's look at lifecycle policies. Let's go look at your EC2 instances you've over-provisioned. Let's go look at all these NAT gateways that are just hanging out there and doing nothing. So it got them moving down that path of actually reducing attack surface.
Jeremy
And so along those lines, I'm curious, you know, now in 2022, aside from EBS volumes, and you mentioned, you know, NAT Gateways, what are the other big kind of cost drivers that are also creating exposed attack surface for customers? Like do you have, let's say, a top five list that you could share with the audience of things to look out for in their own accounts?
Nick
The top five things that are happening or that we find are abandoned database instances. A lot of times they're legacy database instances, so they're running on an EC2 servers. They're very expensive. And then when you go look at 'em, they're actually publicly exposed because they were probably used for some project or some migration and somebody at the end of the migration, the classic phrase is, Well, let's just leave this hangout for a month just in case.
Jeremy
Just just in case, Later. That's right.
Nick
Yeah. Right. Just in case.
Jeremy
Yeah.
Nick
And a month later, nobody cleaned it up, and you know, we found one particular instance to the tune of about $70,000 a year was just hanging out, right? It needed to be eliminated and it was publicly accessible on, you know, SQL, port 1433. So that's one.
S3 buckets are another one where they get rampantly created data gets put on 'em. They don't really understand the objects that are inside of it, and they don't really understand the permission set. And so we find over and over and over publicly exposed S3 buckets probably have data that shouldn't be exposed publicly. So, yeah, you know, goes hand in hand with, you know, that needs to be eliminated.
And then of course, your biggest players are always EC2 instances. That's, that's the vast majority of the bill. Even though more and more organizations are making use of containers and serverless infrastructure, EC2 still takes the cake for largest amount of spend, largest amount of over-provisioning, largest abandonment, and resources just basically left hanging out, exposed, unupgraded et cetera, not managed.
So those would be the top hit list for me.
Jeremy
Gotcha. That's a great list.
And so I think for anybody listening today, you know, if you are facing the same question, either you feel like your bill is too high or you're not sure, you know, there you've got a list of things that you can go quickly check through.
Unfortunately AWS doesn't make it super easy for you cuz you do have to cycle through the regions that you're using to go look for those things, regions and services. But you know, this is where the value of a tool like Tenacity Cloud or any number of others in the market can help you bring that visibility into one central location to make some of those assessments.
And I imagine that's a big part of what you end up doing on a day-to-day basis with your customers. Nick is kind of having those assessments and those conversations, and so I'm curious, you know, when you have those conversations, one of the things I've observed is that even in the cloud, organizations still tend to behave like organizations, which is to say that finance people care about finance stuff and security people care about security stuff.
Do you have any tips or any kind of insights as to how we bridge that gap and we make people understand that, you know, things like proper hygiene actually have applicability on both sides, and they should work together to try to, you know, reduce these sprawl problems that we're all seeing?
How do we get people to realize it's not just their silo?
Nick
It's the hardest problem in technology is getting the people to talk. You know, it's like putting a bunch of introverts in a room and telling 'em all to mingle. So, you know, as with that analogy, you need a little bit of lubricant and you can have cocktails at an event, but for us it's giving them a common set of data and a common platform that they all can understand. That it doesn't have its own secret language, that it isn't hieroglyphics to someone.
And so when we first started, and I'm glad you asked this question, cuz when we first started this is the problem we stated is in every organization somebody's pointing at someone else and saying it's their problem.
Security's pointing at the cloud guys and saying, they're the problem. They're the ones they, you need to fix this right now. And that doesn't help anything. What helps is when a security engineer can go to a cloud engineer and provide context and say, Hey, listen, we're seeing this particular issue. And you know, there's a number of ways that maybe we can solve this, but here's why it's a problem.
And then you get this conversation that happens, goes like, Well wait a minute, well I have to write the app to do these things, so maybe we could solve it these six other ways. And suddenly that conversation starts. And so as we have matured our platform, we have listened to the different users, whether it's folks from the finance department or folks from the cloud team, or folks from the security team, and tried to write with context, not with a secret elite language that only a few people know.
You will not see language in our platform that's riddled with acronyms and must have no in the environment. We've tried to make it very contextual, including the way that you look at financial data. Does, is the finance person looking at it because they're gonna wanna look at like it's a financial statement, whereas is it the cloud person or security person looking at it? Cause they're gonna wanna look at anomalies and outliers.
They're gonna want to know, well, wait a minute, I just wanna know this one point out here. And so we always make sure that there's views that support the way they want to look at the data in that create collaboration throughout those teams.
Jeremy
Yeah. I think it's a really subtle, but such an important point that you raised there, which is really the contextualization of this information.
We talked at the beginning of this episode about visibility and how important it is to start with that as a baseline, and maybe the second step in that journey is to say, Okay, I know what's there. Let me understand what each of those things that is out there is a part of, to put that context around it so that you can start to address it properly.
I mean, just if in simple terms, if you think about, Hey, you know, here's my development environment, here's my production environment. Like, in our data center world, that was very easy because, you know, we didn't put non-production in the data center. Everything that was in the data center was production by definition. And anything pre-production was in a rack in the closet, in the office hallway or what have you with no SLA on it, right?
But now in the cloud, we've got pre-prod and prod out there, and you would never mix those two things, or you wouldn't think about mixing those two things from a contextual perspective because one has sensitive data and the other one shouldn't.
I won't say doesn't, I'll say shouldn't, right?
But this point is subtle but important as I said, you know, it's getting that context around it is super relevant and important. And I think the way you expressed it there about kind of using the context to bridge the gap is something that a lot of us could keep in mind. Cuz I think we as people who focus on cloud security have a little bit of a challenge to overcome in helping people understand how this transition actually does benefit the organization.
I do see a lot of organizations that say like, Hey, well I'm just gonna go to the cloud. Now I can rack up expenses at scale, you know, and I can do it more rapidly and I can do it with all these great tools around it.
So I guess, you know, one of the questions that comes to my mind is when you work with an organization, and let's say you've gone three, four months in. You gave that example of how you can kind of find all these problems within a day or two, right?
Once you've seen them go through, let's say a couple cycles of finding stuff, reducing their sprawl and so on, does that really help them to reestablish a confidence in moving forward in the cloud, and does that really kind of unlock more potential for the organization?
What's been your experience there?
Nick
Yeah, very much so. Once you sort of see the environment for maybe how dirty it was or how much waste was sitting there, it's hard to not see it again.
And so just having that experience of having found those research and I'm gonna take this to a second level cuz this is the next level of maturity. Once you clean up the environment, it's hard to not see it again. Well then the very next thing you get to move into is actually then managing your committed use discounts, getting to take advantage of saving plans.
Jeremy
Okay.
Nick
And ROIs and spot instances and eventually, enterprise discount programs. It frees you up to start thinking about those things. And when you have to think about those things, optimizing them and understanding them, you just have a far better grasp on your environment and how you're actually using it.
So, you know, let's just say the ongoing use of, of, let's just take a really simple example, the ongoing use of, of a reserved instance plan. And where you might do some analysis in your environment or, or Tenacity does some analysis for you and it tells you, hey, you're using, you know, 10 of this particular instance type.
And you know, you can buy a plan that aves you whatever, 50, 60, 70% on those. And, well then you start to pay attention actually to the use of those. Are they staying updated? Are you moving to the newest instance family? There are new, sometimes potentially new discounts for actually doing the upgrade and switching plans, selling an old plan and buying a new one.
When you get to that sort of proactive level of management, that waste conversation, it really does become, um, what's the right term? You get your arms around it, right? You never optimizing once, but once you've optimized once, it's really hard to go back to living in that world of kind of sprawl and, and squalor. Be a little bit like moving into a house, remodeling it, it's now beautiful. It'd be really hard to un-remodel it and try to live in it that way again
Jeremy
Yeah. Yeah. Yeah.
That's awesome. That's a really great kind of story, I think, that captures, or it's a great analogy, I should say, that captures the potential of what you can do once you do wrap your arms around the sprawl problem.
Well, Nick, thank you so much. This has been a great conversation. Any final tips you'd like to leave with the audience regarding sprawl, hygiene, cost containment, anything from your years of experience with cloud technologies and from Tenacity? Any final pointers, tips, anything you wanna share?
Nick
The only thing I would say is, sometimes people are hesitant to talk to someone else about the problem of, of what's going on in their environment. And I usually start with telling people there's no mistake you've ever made that I haven't already made in the many, many infrastructures I've run in the hundreds of clients that I've, I've helped manage.
I've made some doosies in my career. It's really just about taking those lessons and applying them to how we're gonna solve the problems of the future. So reach out. I'm always available if, if you all reach out on, on LinkedIn, find me, or just email me. I am very vocal, very responsive. I'll hop on a call with anyone. Doesn't even have to be related to Tenacity, just related to, Man, how do I do this? What's your experience in this? I'll tell you all about my failures, so that you can learn from 'em.
It's the only thing I had to take away, and I hope that people feel that same way with everyone else because we learn from each other, from our, more from our failures than our successes.
Jeremy
Absolutely. I could not agree with that last point more. We learn much more from our failures than our successes. I may or may not in, let's say my second year out of undergrad, I may or may not have completely wiped our entire CRM database. That may or may not have happened, so we've all been there.
But your point, I think, it's so well spoken. Everybody's made mistakes. We can only learn when we collaborate and we're open to talking about them and learning from people around us. Getting best practices and tips, tricks from people like Nick, from those that are out there in the community doing the good work.
Nick, thank you again for taking the time to talk with us today, and helping our audience understand the importance of good hygiene in their cloud environment and how that helps to reduce attack surface and keep them safe.
All right, we'll see you next time in the ASk A CISO podcast.
Nick
Thanks so much, Jeremy.
