Welcome, listeners, to the Cloud and Culture Podcast. If you're tuning in for the first time, we're only in the third episode after all, here's a quick background. I'm Derek Harris from VMware. And I'm Danielle Burrow from VMware. And this podcast is focused on VMware Pivotal Labs and the work done by our team of experts to help organizations, ranging from startups to Fortune 500s, shift their software development into high gear. As the title suggests, that entails modernizing and sometimes just kickstarting software development and tooling, as well as helping clients create a culture that will allow them to run efficient, self-sufficient software development teams for the long term. And our guest this week is Josh Rosso, a staff architect at VMware, who's been around Kubernetes for about as long as anyone, first at CoreOS, then at Heptio, and now at VMware. He spends a lot of time working with customers on their Kubernetes environments, and he's seen a lot with regard to what works, what doesn't, and where the gotchas are, especially as organizations move from testing Kubernetes to deploying it in production and then to scaling it. He also co-hosts the Podlets and TGIK podcast, along with other VMware Kubernetes experts. And if you're thinking about using Kubernetes to modernize your app infrastructure, sorry, your app infrastructure, you should learn a lot in this episode. And although we talk a lot about Kubernetes in the general sense, don't forget that VMware offers a whole collection of products to improve and simplify the Kubernetes experience. Our Tanzu lineup features tools that make it easier to manage Kubernetes clusters, manage multi-cluster and multi-cloud Kubernetes environments, and everything in between. VMware has also developed a range of open source projects to advance the Kubernetes experience, including Harbor, Valero, Octent, and Entrea. And of course, VMware Pivotal Labs has an entire team of Kubernetes experts, including Josh, who are dedicated to helping organizations design and implement a successful path to production at scale with Kubernetes. So listen to Josh now, and then explore our Kubernetes story at tanzu.vmware.com. And if you want to learn more about how VMware Pivotal Labs can help you on your modernization
2:02and Kubernetes journeys, visit tanzu.vmware.com slash labs. So Josh, thanks for joining us. Could you start by just giving us a little bit of background on your experience working with Kubernetes? Absolutely. So I've been doing Kubernetes for what I perceive to be a while, given that Kubernetes is still a bit of a newer project. I started out doing Kubernetes full-time at a company called CoreOS that a lot of the listeners might be familiar with. CoreOS worked on things like etcd, which is a data store of Kubernetes, and built out our own Kubernetes distribution, which was called Tectonic. And now a lot of those bits are relevant in OpenShift. When Red Hat bought us, I spent a very micro amount of time there before joining Heptio with Craig McLucky and Joe Beta and was helping bring Kubernetes to the enterprise, specifically kind of open source, upstream compliant Kubernetes, when that was a really big deal back in the day and still is today to some degree. And now I'm at VMware. And my day-to-day job has been pretty similar throughout that whole kind of progression of companies in that I go to fairly large organizations, you know, Fortune 500, Fortune 100, and help them establish a Kubernetes strategy and instantiate it, architect it, engineer around it, all that good stuff. So a lot of big banks, a lot of big retailers that some of the listeners probably interact with daily had some amount of involvement in helping them build out their platform. And I've seen a lot of the good, bad, and ugly that comes along with that process. I like how you say back in the day, I still hung up on that. Yeah, it's funny in Kubernetes years, right? It's like, you know, most people could say, oh, yeah, I've been doing virtual machines for
4:02this decades or whatever it might be, or a decade. And with Kubernetes, it's like five years makes you a extreme veteran, you know? Yeah, that's funny. You mentioned some of the types of organizations you're working with. I'm curious, like, what, I mean, this was maybe jumping into the deep end too fast, but the one thing I'm curious about is just what types of applications you see running inside those, like, what kind of things are running on Kubernetes at the moment? Yeah, absolutely. There is a ton of diversity to this answer. You know, if we were to go back a couple years ago, it was definitely less common for serious stateful workloads to be run inside of Kubernetes. So, you know, a lot of folks who are just getting started, and this includes the big organizations too, they were getting started by making sure their stateless web services could run really well in Kubernetes, applications that they had containerized and running really well would go into Kubernetes. And the idea would be perhaps they've got stateful services in a cloud provider like AWS or in their on-prem data center. They've been running Cassandra reliably for the last forever, as forever as far as Cassandra is concerned, and they would just kind of take that model of not trying to put everything in Kube, but trying to put the containerized stateless apps in Kube. Nowadays, it's definitely a bit different. You know, not only are people transitioning from their Kubernetes strategy being a bit of like a science fair project, if you will, right? You know, we're starting to see a lot of folks actually transition into getting workloads into production, and this includes workloads that are stateless and many times stateful. So when you go to, you know, a large financial services org, there's a good chance they might be running something like CockroachDB, and CockroachDB is something that they want to run as a service in the cluster so that as they onboard applications, their applications can consume those CockroachDB
6:06instances as well. So the short answer is all the applications, really. You know, the great thing about Kube is you can run most application types in there. I mean, at the end of the day, containers are wrapping a process and doing a couple simple tricks to, not really simple, but doing a couple tricks to make it run well. And it's just a matter of, you know, changing the paradigm a little bit of your application, like how it approaches high availability, how it approaches its ability to be rescheduled, just to make sure it can run well in Kube. And so, I mean, you kind of alluded to like, well, actually, it's not so simple. I mean, it's getting easier. But how would you say the experience has been for these organizations overall? Like, is it worth the effort and the change to move to a Kubernetes-based platform? Yeah, it's a great question. And as a VMware employee and someone who adores Kubernetes, my answer should probably be yes. But it is way more nuanced than that. You know, one of my first conversation points with a lot of folks with Kubernetes is just making sure that they should be using it in the first place. You know, a common, I don't know if this is a cliche at this point, but I'll call it out just in case if you're a startup that is attempting to acquire market share for your product. And, you know, at this point in time, there's nothing more important than shipping something, gaining users, so on and so forth. Like, I've seen so many startups that both friends and customers work at where, you know, they have this affinity towards, well, we really need to solve for scale and this thing and that thing and Kubernetes is great, so we should introduce that and use it. But the thing that they lose sight of is every second they're operating Kubernetes or working on kind of their infrastructure platform is the second they're taking away from shipping software that helps them get their market share.
8:08The thing I tell them is, like, your customers don't care whether you're running on Kubernetes, right? Unless it's me. If I'm your customer, I'll be like, oh, that's really cool. But aside from me, most people aren't really going to care. So if you're that kind of persona, maybe something like Heroku, you know, a platform as a service type offering is way more appropriate for you. Now, a lot of these big organizations in regards to your question, Danielle, about like the worth, obviously the answer is it depends. But a lot of them are really thinking through, you know, how do I kind of standardize on a application platform where we can have this kind of consistent experience to onboard apps and get things deployed and scale them out and have really solid declarative APIs and so on and so on and so forth. And I think what a lot of these larger organizations are seeing is Kubernetes is kind of a natural evolution for them to get to that state. And to be clear, Kubernetes is part of the puzzle for these groups, right? Kubernetes is the container orchestrator layer. It's not like they should be going in and saying, and we're going to introduce Kubernetes and our problems will be solved. You know, more so the problem is, oh, we need to think about how we schedule and think about workloads. So we'll use Kubernetes as kind of that foundational orchestration layer. And then for monitoring, we'll add this piece in. And for source control management and continuous delivery, we'll add this GitOps type flow in with these tools and so on. So, you know, I think the super honest answer is it's a bit to be determined for a lot of these orgs whether the investment is worth it. But in my case, I'm very lucky to say I've seen a lot of organizations see this process all the way through. And for many of them, they've gone from developers putting in a ticket and getting a virtual machine 14 days later that's misconfigured, and then they have to open up another ticket so that they can finally get their app on it. They've gone from some of those really old legacy models to this completely API-driven
10:09container-based system that has enabled them to ship software a lot faster. It has enabled them to have easier primitives for scaling and monitoring and traceability. So, you know, it's a really inspiring direction for most enterprises, for most organizations. But I think timing is everything, is really kind of what I'm trying to say in too many words. You really have to figure out, is this the right time for you? Josh, can you go into it? You mentioned this and you mentioned scale several times. So it does seem, and I could be mistaken here, right? Like there's a kind of double-edged sword with Kubernetes where on the one hand, it'd say, well, if you're not operating at a big scale, it might not be worth the effort. On the other hand, you know, it seems to be that it gets a little, it seems to be a little more complex the more you get, the more you scale out, right? As most things probably do. So, I mean, does that kind of match? It sounds like maybe that matches your experience. And how do you find that balance between like, okay, well, now we're at scale. Now this makes a lot of sense, but also how do we manage this thing if we're actually doing it at, you know, let's say bank or mega retailer scale? Yeah, absolutely. So the universal principle that scale makes things more complicated, I am a hundred percent on board with. You said something really interesting at the first part of that question that I'll call out, which is like, you know, unless you're at a certain scale, should you consider Kubernetes? Now, you know, in a lot of the things I'm saying now I should mention are Josh's opinion. So listeners, please know that you might disagree and that's awesome. But my perspective is very much going a couple years back, you know, if you weren't in a state where like, I heard someone say once, like, you have so many machines that you can't remember their names, right? Like, or you have so many machines you can't, like, that could be a really good inflection point for adding an orchestrator that abstracts them like Kubernetes and on and on. Today, it's a little different. Like, you have to think a couple years ago, we didn't have managed services per se, like EKS for Amazon, TKG from VMware, AKS from Microsoft, and of course, GKE, I should call
12:15out from Google. And these have lowered the bar a bit. You know, what you can do here is you can go to a provider and you can basically say, give me a cluster and handle some of the bootstrapping and management concerns for me. So I think Kubernetes is at a place today where you don't have to be at mega scale to use it. You can still justify its use, especially if you've got expertise, especially if the container-based models and flows are things that make sense for you. So I'm not saying everyone should use Kube, but you don't necessarily have to be at a massive scale because no longer is it as onerous to operate Kubernetes. Now, the second part of your question, I'm super on board with. Like, what ends up happening is, you know, especially when you're one of these large organizations, as you gain adoption, right? And you actually get people onboarding, you get applications scaling out. Things do become way more complicated, which I feel like is probably true for almost anything. And with Kubernetes, how this kind of shows is, yeah, there's like, there's some potential limitations at like technical levels, like our deep lower level concerns, like people using the service abstraction in Kubernetes. It by default uses this thing called IP tables. There have been experiences where you have an insane amount of services with a lot of churn. And that particular piece in Linux called IP tables doesn't do so well with like mutable changes over time. It can cause some scaling issues with services. So like you end up like at the Linux level having to think through, oh my gosh, do I replace this thing called Kube Proxy that like handles this really low level implementation detail? Do I tune Kube Proxy or use it in a different mode? You get a lot of those lower level concerns. Now, there's also higher level concerns. There's like, all right, we're scaling now. Should we, you know, this is the most common one I'd say, should we have one mega cluster that just grows and grows and grows and puts stress on these edges of like how much the
14:19API server can handle and how much kind of communication can occur between the state of all these nodes as you go up and up and up? Or should we take this like multi-cluster federated model where we have things where we can just instantiate clusters, you know, clusters like cattle, if you will, and just start bootstrapping more and more and more and more. And that model is one that we've actually seen a lot of success with with customers at scale. The big trade-off with that multi-cluster model is that you end up with a federation problem. And there's open source projects. There's companies that try to solve the federation problem. But I'd say largely, it's still pretty difficult where basically you say, now I'm no longer in a world where I have this one scheduler that understands a singular cluster. Now I've got 27 clusters that I want my developers to be able to deploy to. And most of us don't want our developers to have to worry about which of the 27 clusters to deploy to, and then how to make it highly available. We want to build an abstraction or kind of a federation system on top of that. Right. So, um, so yeah, it's, it's really just kind of like, as, as you scale up, you end up having to think a little bit about, you know, what areas should we scale and how should we approach this? Um, and, and there's a lot of ways to kind of, uh, uh, to use a crude analogy, kind of skin that cat. Yeah. And I, I feel like you, you just kind of covered my next question a little bit, but maybe, you know, kind of a different angle. Like if you think about the things that you've really got to think about, um, kind of the gotchas as you move through different stages of setting up a, you know, a production, like getting to that production platform at scale. So like, what are the things you have to think about when you're moving from like desktop to like test development to like production to production at scale, like going through, you know, that process. Are there different concerns along the way?
16:20Yeah. I hope this won't feel like a cop-out answer, Danielle. Like one of the places that I usually start from, because a lot of times it's like, okay, let's, let's talk about what are, what are the things that I should be? I'm speaking for a company here. What are the things I should be worried about when I get to scale? And the first thing that I tell them is first, let's worry about needing to get to scale. Because so many times we don't even get to that point. And we've covered a lot of that already, right? It's like, do I actually benefit from running Kubernetes? Is it beneficial to me? Another really common thing is if, especially if you're building out, you know, an application platform on top of cube where like you're going to be having monitoring, maybe a service mesh, maybe X, Y, and Z thing to, to benefit the apps. Make sure you're engaged with your development teams during that process. It might seem obvious, but it's super sad how many times I go to large organizations that have a build it and they will come mentality. And then they wonder why platform adoption kind of lags. It's because you built something in a silo and you misalign some of your decisions with what developers actually need and want. So let's assume you did those two things right and you built a great platform and you got great adoption. So then that gets to, I think, what maybe you were more so getting at. What are some of the kind of key gotchas? I'll throw a couple off the top of my head that I commonly see. One of them, as you start kind of getting adoption, is really around, well, I'll use this one as an example. It's kind of random, but choosing your open source tooling wisely. And this one is interesting because we've had this paradigm shift from, you know, you have a vendor or an enterprise supply you with something to, you know, a lot more of a, like, open source. I can pull it off GitHub type model. Now, open source has always been a thing, don't get me wrong, but it's becoming even more prevalent. And if you look at something like the cloud native landscape, there's tons of logos for all types
18:23of projects and products out there, right? So the one thing that I'm oftentimes telling folks as they scale and choose these different tools is, you know, really think wisely and honestly about the onus it puts on you to run some of these projects. You know, the common thing, I don't know who came up with it, that people say, like, free as in puppy, not free as in beer for open source, right? It's very true. Open source is free as in puppy. Sure, it's free for you to download and deploy your cluster. But when, you know, your container networking solution, let's say you're using Cilium as an example, if it breaks down on you, who's on the hook for solving that problem? Who's on the hook if a critical bug comes up and sends your production system spiraling? You know, in cases like Cilium or Calico or Antreya, in the case of VMware, a lot of these open source projects have enterprise support. So not that you need enterprise support, but the question really is, do you have the operational maturity that when you scale out and something goes wrong, you understand how to work the project, how to work the code, how to fix bugs, how to solve problems? Those are really kind of honest and big things to solve for. And then I guess the last thing I would say, just to give one more random example is, you know, really commonly when you start scaling out, it's just really scaling out how you manage and how you do SRE like things to the many, many different entities you have to worry about. So like in that multi-cluster model, you know, let's say you end up with hundreds and hundreds of clusters. How are you going to have an SRE practice around all of these different units that you have to worry about? How are you going to ensure reliability, not just for one thing, which might be where you started, but now a hundred things? And how are you going to kind of federate and think about kind of the management plane on top of all of that stuff? And I think, I think a lot of this is maturing in the cloud native space right now.
20:23There's such great stuff happening upstream in products and projects that are out there, but it's very new because if I were to predict, I think from, at least from Josh's experience, I think, um, a lot of folks are just now reaching scale in production. Now there's tons of people who have been in production with Kubernetes forever, but I think we're kind of hitting more of an inflection point. And now we're starting to see more of the needs for how do I kind of scale out my management capabilities over, over many of these clusters. All right. I think that open source discussion is interesting too. And I should say, you know, we're working on, uh, publishing at some point soon there, and maybe it'll be published by the time this airs. So you can go back and look for it. But a buyer's guide to open source. And then one of the things where I think we talk about many of those things in terms of what does this actually cost you to run? Right. And yeah, who's, what's the community look like behind this project? There's just so much happening right now that to pick something out of the, what is it, what's been approved by your legal department? You know, there's, cause we do actually have to push it to production. Now you have a whole lot, whole other set of concerns, I think. So, right. There's a, yeah, there's a lot with the open source that, um, that maybe it's taken for granted, but, uh, sticking specifically to Kubernetes, I want to ask too, like because you, because you mentioned you were working, if you're building out a Kubernetes platform, the importance of working with your develop, with your developers. And I've heard this from customers and other people, like, you know, there's a, from both directions. One is developers coming and saying, Hey, we want Kubernetes. And the other is this platform team or the ops team coming and saying, Hey, we deployed Kubernetes. So how have you seen that balance play out in the field? I mean, is this something driven by developers, by the infrastructure folks? I guess there's benefits to both sides, but. Definitely both in my experience. And here, here's my current working theory. And I'm still learning about how all this plays out as we see more and more people adopt cube. If you can operate at the intersection of apps and platform, that is the gold place to operate at. Cause here's, here's, here's the thing, right?
22:23The, um, the developers to get their apps into production rarely find Kubernetes to be solely enough. Um, some app teams run their own Kubernetes clusters. That's, that's a bit of a different persona here, but the app teams, when they come to the develop, to the platform teams, especially in large orgs and say, Oh, we'd really love Kubernetes. What they actually mean is they'd love Kubernetes with a certain ingress, uh, controller set up, maybe a service mesh monitoring pre-baked in so they can access stuff, so on and so forth. And then the platform team, you know, they have desires of making sure they have great customer satisfaction of their developers. Like, you know, at the end of the day, the platform teams, uh, customers are those development teams. So what I mean by the intersection of apps and platform is here's, here's what I think we're largely doing a lot of the time. Um, there is this, uh, there is a desire to put apps on a more modern platform, right? So we start talking about adoption and understanding the apps and we start understanding kind of common patterns like, Oh, these apps put into their Java code, this mutual TLS system that they bake in. And then, Oh, this, these Python apps that we're pulling in over here for something machine learning related. They also do service to service communication over like some mutual TLS thing. And they all have to bake it in via some library that they import and so on and so forth. So then the platform team kind of learns about this common pattern and they're like, okay, this is great. So it sounds like maybe some kind of like service identity system that we could bake into the platform and offer developers would enable them to ship code without having to be worried about identity services, intra code. That's kind of interesting. Okay, cool. So let's think about, um, some type of technology. Maybe it's console connect, or maybe it's some service service mesh technology like Istio, which of course solves a bunch of other problems too, but we bake this into the platform as a platform
24:25service. And now we've relieved our app developers of that concern. That is one of the, again, where you can really make your platform super valuable is not just shipping random features, but shipping things that make a lot of sense with your understanding of the app persona. I'll give you one more super random example. I see this quite a bit. So app developers are going in and they're like, oh, you know, we, we so often we have to like talk to this external vault system. Vault is a secret management store by a, by a company called HashiCorp. It's a great open source project. And, um, and what we find is, okay, every application is having to like import the spring cloud library thing that allows them to communicate with vault. And then the Python developers are having to do the same thing. And the go developers are having to do the same thing. What if as a platform service, we set up Kubernetes to integrate with vault so that a application could come in, ask for a secret that it is allowed to have access to in vault. And by a, just like a little annotation on a YAML file, Kubernetes goes out, does the magic and just makes the secret show up in the file system of the container. Like, boom, what an amazingly powerful platform service, right? So it's those kinds of things where like, don't just guess, work together at that intersection. And what you're going to end up delivering is something that is insanely valuable for both the infrastructure team and also the app developers. So, um, Josh, I'm kind of wondering, just going back to talking about open source and the Kubernetes community, are there holes in the Kubernetes that you think, you know, still ought to be filled by the community? Like what's, what's missing or where, or where are we headed? Yeah, that's a great question. Yeah, I think, I think a lot of things are really in flight right now.
26:27So I'll, I'll include gaps to include things that like people are actively working on, but I think we're still pretty early days. Like you heard me mentioned many times. I think federation is a pretty big one. Um, there's been great traction. I know there's a bunch of humans doing awesome work here, so I'm not trying to discount that. It's just, it's pretty early days. And I don't think we've really seen something kind of, uh, fill that gap completely just yet. Um, you know, kind of, kind of like just to give an analogy, like kind of like service mesh, like we all knew maybe we needed to fill this service mesh gap for people who have the right need to implement a service mesh. Um, and we saw players kind of come up like Istio and Linkerdia and now that's got its own thriving ecosystem. I think, I think federation is definitely one of those big ones. Um, I think a lot on kind of like the, I guess a bit on the kind of app side where we're, we're really, we're really kind of moving up the stack at this point, in my opinion, as, as Kubernetes becomes more and more boring. Um, there's, there's a lot of really interesting things happening on the app. side that are filling gaps. Like how do we appropriately abstract Kubernetes to make sure developers can be successful without becoming a Kubernetes expert? That's a super interesting space because it's a balancing act, right? A lot of times it's, it's a little naive to say, well, we're just going to completely build a full abstraction. Developers will never know they're using Kubernetes and we'll keep exposing the right knobs to the developers over time. Like some really solid engineering orgs are successful at that. That model, but many can't do that. And Kubernetes offers so many great features that a lot of times we want to expose a little bit of that to developers. So like, you know, I think where I'm going with this is examples like GitOps. GitOps has become this really interesting model now where we're starting to see it play out, where developers can focus more on, I commit the thing to Git. And maybe I don't really understand all the things that get picked up in order to kind of make it happen, right?
28:27As we see things like things inspired by Cloud Foundry kind of come into the Kubernetes ecosystem, that's also a really interesting place that I think we'll see some traction. Like if those of you who have ever used Cloud Foundry have probably heard of things like CF push, like how could I, from my source control repo, say, push this app and all the different things kind of happen and it just becomes available. Like that's a really interesting developer experience too. So I would say if I were to hedge my bets on just two, because I could talk forever about this, I'd say federation of clusters is a really interesting space. And I just think upstack, like developer onboarding, you know, we're getting really good at Kubernetes. How do we make it frictionless for the people whose day job isn't Kubernetes to benefit from all the awesomeness that Kubernetes offers? Yeah, on a related note, I wanted to ask too, is there a, do you see like a standard application, like a Kubernetes application architecture, right? Or a standard development experience, developer experience? I mean, are these things, are we heading in that direction as you talk about all just all the, you know, the tools that are kind of getting critical mass and the way that they might be exposed to developers? Is there, you know, are we heading toward a point where if I'm a developer, if I'm building an application that's going to run a Kubernetes, I kind of, you know, I'm kind of, I'm architecting in a certain way, right? I'm, I'm expecting a certain experience and I'm building in a certain way. Cause that's just like, you know, maybe it's like the lamp stack for something. My answer is not really. I mean, it is, it's super interesting. Like speaking to like the application architecture, I help people run monoliths on Kubernetes all the time. It works great. There's not, you can run a monolith on Kubernetes. You don't, you don't have to re-architect into a 97 factor microservice architecture that is compliant with X, Y, and Z standard. You know, there are definitely things you got to do to get your app in Kubernetes. It's maybe possibly got to run in a container, although there's some really interesting projects to make virtual machines, first class citizens.
30:27You probably want it to be able to be highly available, resilient to scheduling events. There's, there's, you know, exposing probes for its readiness and liveness of the application. Like there are these small things, but largely we can make things run pretty well in cube. So as far as like how people are architecting and building apps, I'm seeing an insane amount of diversity. I mean, when I go customer to customer, I feel like I'm not even talking about the same Kubernetes anymore. Cause like the use cases and the application types and their, their stacks vary so much, which I think speaks volumes to Kubernetes. Like while Kubernetes is a lot of an investment to build out and determine which knobs to turn, it also can handle a lot because it did a beautiful job of making things extensible, customizable, and so on. Um, and then as far as kind of the, the more downstream thing, like, is there common kind of like build and deploy patterns? Um, I see a lot of diversity here too, like, uh, or, or variants, I should say, where some folks are just linking their old Jenkins system up to cube, uh, that runs outside of cube and just deploying. And that's perfectly fine. I think the GitOps model is gaining a lot of traction. Cause like, it's such a good intersection. Git is something that developers understand platform people understand, you know, it's like, Hey, commit this thing. And then all the other things will happen is super compelling for a lot of organizations. And, and, you know, and as far as like containerizing and on all the interesting stuff that's happening, there's so much going on right now, like build packs. Um, and I know, uh, red hat has like a source to image system too, where like with build packs, let's look at the source code and figure out how to make the container image without you having to understand how to write a Docker file. But then I go to another organization. It's like, no, we really liked the declarative nature of Docker files. Like it declares what's going to be built. It's super clear. Like we like this, like, and that's perfectly fine. So, um, I, I think, I think there's definitely a lot of fragmentation to answer your question as far as like how all these things fit together.
32:28But at the same time, that fragmentation is like a lot of healthy variants because there are simply so many styles to deploying and building software. And I think Kubernetes can support the, the 90 percentile of those, of those approaches. Okay. Yeah. So given all of that, do you think like, like what, how mature of an organization, I guess, technologically, and, and because maybe some of the diversities of this, the function of the, the, the technological maturity of companies that are adopting this right now, but how, like in order to do Kubernetes well right now, I mean, what kind of, you know, if we look at digital transformation or throw or cloud native or throw or modernization or any of these kinds of bigger picture things, how far along that journey does someone, does a company need to be to really be able to take advantage of, of Kubernetes effectively today? Right. I think, I think a good starting point, and I'm sorry to generalize here for everyone who's listening, but I think a really good starting point is like thinking about the things that Kubernetes solves and whether it's actually aligned with problems you need to solve today. If you look at your current setup and your software is, I don't know, let's say buggy, it has stability issues, it has X, like Kubernetes is probably not going to fundamentally solve that problem. In fact, it might distract you from actually solving real problems. You might actually need application work and re-architecture of apps, and maybe you need to put a microservice back into a monolith or a monolith needs to be broken up into a microservice. Those are things that Kubernetes inherently shouldn't be viewed as, in my opinion, as a forcing function to solve. So the way I'd say it is like, be weary of that really ambitious engineer that comes into your org all of a sudden and says, oh, and we should add Kubernetes to our roadmap, unless there's really good alignment for why that should occur. Not to state the obvious, but that why question is super, super critical. Now, as far as maturity goes, you know, maturity can mean so many things and has so many different
34:30axes to it. But, you know, again, the first starting point is like, will the things Kubernetes solves actually help us out? And then you take a step up from that and say, do we actually have organizational experience running Kubernetes? And if the answer is no, how quickly could we get spun up on it? Are we cool with using managed services like Google, Amazon, Microsoft, VMware services that give us Kubernetes in a much easier way that negate the need for us to manage the control plane and otherwise? That'll take a lot of operational burden off you. But if you're evaluating like, okay, we have a lot of operational maturity and we want to run our own cube. Are we ready to handle lifecycle management concerns of the cluster? Are we ready to handle building on top of Kubernetes? Because remember, it's rare that Kubernetes in and of itself is enough. It's usually a foundation we build upon, right? So it's really just kind of being honest about what would that future state look like? And what's the engineering effort involved? And I'm not going to say you're going to be able to get that number right. Like we need this many humans and this many hours. Obviously, that's naive to think it's that simple of algebra, but trying to just have that kind of sit down moment where you really think about the benefit to cost analysis and really understanding your team. You know, going back to that startup Heroku example I gave where I said, if market shares your biggest thing, maybe cube isn't right for you. On the flip side, if I land on that team and there are a bunch of really amazing system engineers and Kubernetes experts, and heck, maybe they know more about Kubernetes than they do about Heroku, I'd be open to having that conversation about Kubernetes. Heck yeah. Let's spin up something on a cloud provider. Let's run our app stack on Kubernetes. Done deal. So all of these have gotchas and kind of corner cases, but generally speaking, I hope that gives kind of a sense of how I would at least view that question about maturity and timing. Yeah, absolutely. I think it's a matter of having heard customers talk about this too in various formats, right?
36:32There's this idea of, right, sometimes like their applications just are not ready, right? They need to be like, you know, from an architectural standpoint, you're just in your team. You don't have people who can work with. So that's right. That's kind of what's getting at. It's just like, it's just not, if you're a fortune 500 company, like you might have some fantastic, fantastically talented people and modern stuff in some parts of the org, but in other parts, right, it might be looking like it's the 1980s. And so I think that's perfectly put. And if I could just say something, it's like, we all have an insanely long list of problems. And I just want to make sure that you know that Kubernetes solves the important ones up top. And if it doesn't, maybe injecting it is actually going to cause more work and pain for you than it's going to benefit you. Right. Okay. So anything else, Josh? Or is that a good ending place? That was, I think, a good ending place, at least for me. I super appreciate y'all having me. It was great to kind of throw some thoughts out here. And at least, again, a lot of this is just opinion. So it was great to be able to kind of voice my opinion. And I'm super interested to hear what people think. No, we love having you, Josh. We're going to have you back for a GitOps discussion. Let's do it.