Code to Production: From Cloud to DevOps to Platform Engineering, with Purnima Padmanabhan, VMware Tanzu by Broadcom

Episode 281 · Wed, 18 Sep 2024 12:23:55 +0200 · 30:53 · SRT · archive.org · all Tanzu Talk

0:00

Why don't you introduce yourself? Of course, I'm happy to do that. Purnima Padmanabhan, I am the general manager of the Tanzu division at Broadcom, and it is great to be here to chat with you. Yeah, today, I've been looking forward to this because, you know, I don't get the chance to talk with a lot of people who have kind of a parallel career that I've been through, let alone worked at BMC Software. So we should get to that at some point, those times. But I thought, like, you know, I wanted to talk about, like, what you've done over your career and kind of like the jobs that you've had and the groups that you've worked in, the products you've worked on, the people you've worked with, they've all been focused on, you know, we might call it like DevOps or platform engineering now, but there's this kind of continuous thread of, I mean, I don't want to put too many words in your mouth, but I feel like there's this continuous thread of we have infrastructure and we have developers that want to write applications and run on this infrastructure and we should make that better. And, like, I think over the past, I don't know, you tell me what you think it is, but 20 or so years, there's been a couple different types of innovations and stages that we've gone through. And to kind of do it in reverse, how do you think we are now? Like, compared to, like, what you've seen in the past, how do you think we're doing? The stage of DevOps? Yeah, yeah. Like, this kind of, like, platform DevOps thing, like, is everything perfect for the application developers? Is it going out? Is it going well for the operations people? Like, what's the state when you go out and talk with people? Well, that's a good question on how platform engineering and DevOps has, is it solving the problem today? You know, is it, are we at the final stage? And I don't think so, right? It's never, it has gotten a lot better over the last 25 years, but it's not perfect yet. And there is still continuous room for improvement. And I think it is good to go back and ask a question. The terminologies change. Today, we talk about platform engineering and DevOps. But what is the goal that we have been striving for? And when you think about the goal, even when I started on early in my, in the software business,

2:02

it has always been about, hey, you have written your applications, you have written your business logic. How do you speed the time, the amount of time it takes to take that code into production, that business logic, right? Right. And often there's like a lot of pitfalls. And when I think back about my career, right, early on, I was with a company called Loud Cloud, Mark Andreessen and Ben Horowitz had started it. And it was, I always say, cloud before cloud. But there, the whole thing was, hey, developers build their business logic, but often use things like standard middleware, infrastructure software, or infrastructure middleware, databases, web servers, app servers. And the thought for Loud Cloud was, can you give that on a tap, right? Very easily provision it and provide it so that the infrastructure automation pieces are there and the developers can build their software soon or faster. Then that was a very important step and eventually led to the public cloud boom that we have seen and making infra, you know, much easier. And then I moved to BMC and that is why both of us were there. That's great. Probably even at the same time. And a lot of it at BMC was about process automation, where infrastructure automation is a piece, but often there are many, many teams that work together. There is database teams, networking teams, compute teams, infrastructure, monitoring teams, right? Application teams, services team. And there was always a confusion between all of these, right? Anything went wrong, anybody could have made the change. Things would go wrong and then you'd take hours and hours and hours in war rooms trying to find out problems. And what BMC focused on was process automation and ITIL, you know, now we think ticket is a bad word, but ITIL really transformed a lot of the processes there and made things simpler, made the connectivity, the idea of a common view of the environment with CMDB,

4:08

the idea that you could have consistent configuration and change management. That automation was a really big, you know, and we still see that, right? Today, now the evolution of what we did at BMC, you see with ServiceNow and other companies, right? So now you've automated the process piece, you've automated the infrastructure piece. And as I kind of think about my next steps in care careers, a few startups, and then I was at VMware. And VMware is where, you know, we have had this interesting relationship with that platform. Pivotal, right? Pivotal said, okay, even those are disparate pieces. And can you make that even simpler for the developer? And Pivotal was, of course, started with VMware, spun out. And then again, we bought it, Pivotal back again. But the concept was, why should the developer even think about these processes and tickets separately? Why should they think about automation of infrastructure separately? Can they just build their code and define their intent? And the platform takes care of everything else and makes it much faster for developers. And so that was kind of the next evolution. To some extent, Pivotal pioneered the notion of platform engineering. And from there onwards, we are here today, where the Pivotal assets and a lot of the organic Kubernetes assets that we have within the organization have come together as the Tanzu division. But again, the singular goal of Tanzu division is to reduce that time it takes from code to having a running application and project production that can be maintained, that is resilient, that is safe. And we have done a lot. We can talk more, but the work is not done. Yeah, you know, I mean, there's a lot of little interesting, let's say, side journeys and cul-de-sacs and what you were just saying to kind of explore there. And I think, you know, one of them, I think I was a little more familiar with the, I mean, tell me if I'm wrong, but I think there was kind of like two states of loud cloud. There was like the first go at it and then kind of automation, if I remember. And I think, I guess you're like the first one, what, I mean, you would know, was kind of like, I think we would call it a managed service provider, maybe something like that back then.

6:14

And it's interesting that one of the notions was there's all this like middleware that developers use. Why don't we take care of that for developers? Right. Like kind of standardize that because that's a very similar, I'm always trying to pull out like two different paths through all of this that you were just going over. And one of them is standardize what developers use, a platform, right? Just, just like here we have like, you know, at Tanzu platform, we've got like Spring as a standard framework and data services as a thing that's all taken care of. And I think, I think that comes up over and over again. Like we're still always trying to just get developers to be happy about using some, some services we've packaged up for them instead of having to do it on their own. Yeah. Standardized Lego blocks. And, and look that, that part of loud cloud business, which is basically like a managed service and like a cloud service, really. So just like you get, and, and the notion of, it was in fact called utility computing, the notion. Oh, that's right. Yeah. VM on a tap, right. Is what later we started getting with AWS is essentially the same thing with loud cloud, which is, hey, I'll provision the infrastructure. I'll provision the virtual machine. I will provision the, the, the stack, basic stack, like operating system and maybe a web server or app server. So that is the notion like RDS. That is the precursor to RDS for a database. Right, right. But the managed service part of it ran into one problem, which is at that time, people still were not comfortable with shared infrastructure. So people, we would have customers who would come and walk in and say, hey, can you, can I see my physical server? Can you show me a demo? Can you show me, walk me through the Equinix or wherever we were sitting, our, kept our infrastructure and pinpoint the exact server that my software is running? Yeah. So that was still there. And so the evolution, maybe it was a cloud before cloud or maybe a little early in its times for the cloud.

8:16

And so the evolution of it was then to say, okay, instead of giving you a service, like a cloud service, how about we give you the software? And that's what became Opsware, cloud cloud, Opsware. And it is an infrastructure automation. So the same automation software that was used to build the cloud, let me give that to you so that you can build your services. You can automate the provisioning of infrastructure, of servers, boot up of that, secure the, put the secure OS and put your secure packages and then give it over to the developer. But again, the net goal was same. Either you do it yourself or we do it for you, a set of consistent building blocks for the developers. And whether it doesn't have to be the person, developer doesn't just mean the person who's writing code. It could also be the application team, right? There are multiple roles within that that is setting up that much. So do you think there's, especially, well, kind of informed or kind of colored by the various jobs you've had throughout your career, do you think there's a pattern of someone wants to basically have a hosted thing and then someone wants to see the server? And then eventually you package up the software to give to them in basically a private cloud form, right? Like I ask this because like you see this in a lot of the, even, even if you, you know, like the automation vendors, like Puppet and Chef, like they kind of start started off as like public cloud things. And then eventually they all get, they all find out that customers actually want to run stuff on their own. Just like you were saying, I don't know. That seems like a near universal pattern that, that universal pattern. But I think the shift is that there is enough. There are enough people who wanted, who are willing to accept the hosted form factor. There is enough maturity, enough consistency APIs and SLAs. So what I see is if you look at the patterns, I see both continuing to coexist, which is, can you make things as turnkey as possible for me?

10:20

And then package up everything, host everything and give it so that all I have to do is a little, little bit. But even there, again, you still have to do work to make that ready for your application. Or the second one is give me the software that allows me to offer that as a central team, IT team or a platform team, all those services as a utility for my developers, right? I think both models will continue to coexist. And both, those models are actually, I think, independent, a little bit independent of private cloud or public cloud. See, one is where do you host your software? Where do you host your environment itself, right? Is it a hosted environment or is it a private environment? But the second thing is, hey, where is the management software that you use to build out those environments? And that could be either offered as a software that you manage yourself or as a SaaS. Right, right, right. So there are multiple vectors there. Yeah, there's an interesting distinction there is, and especially drawing from what you're saying is, are we talking about, like, I want to be able to go touch my server or I want to be able to control and manage the stack, right? Like, you know, I don't really care where the platform is running, probably, but I want control over what the platform does versus kind of like a take it or leave it kind of situation. Correct. And so you, that's why I keep saying now we have enough viable choices that there's a spectrum. Right. And you pick and choose based on the needs, whether it is resiliency needs, security needs, data privacy needs, control needs. You pick where in the spectrum you want to be. But again, all these are just permutation combinations of the stack below. The goal still stays the same. So then, you know, I was saying there's kind of, there's kind of two threads to pick out. One of them is, is, is that, that kind of, let's call it like the way of business runs, like it's applications, the developers, like, you know, the, the software that you're writing.

12:26

And it kind of, as, as a, whatever we used to call it at BMC, like IT business alignment, we had all sorts of fantastic charts about how, how IT was actually running the business. And there, there's, there's another path that comes up a lot in, in your, your career, which it's kind of like more internal facing IT, like automating the infrastructure, doing that service management service delivery. That's, that's a phrase I haven't used in a long time, you know, kind of all, all the ITIL stuff. What do you think, like, why do we not talk about like ITIL and IT service management as, as much as we used to? Cause that used to be, I feel like the thing, right? Like everything rotated around, like, let's map out what your service is. Let's have the tickets around it. Like you're saying service delivery and a CMDB, but I don't know, that doesn't seem to come up. Maybe those people are just doing their job and they don't do conference talks about it. I think it is the latter. What happens is you don't talk about Linux now. Right, right. You don't talk about operating systems. You don't talk about us, batching operating systems, even though that's a fundamental problem and everybody's still doing it. And similarly, there are basic change management processes that you don't talk about. But that said, they have to exist. You cannot automate something that does not exist. Right. Right. You have to have a process that you can do manually two times in a row, then you can automate it. And that is kind of what, so a lot of those underlying discipline that was built over the ages, right? Many, the last few decades has allowed us to actually do automation at a much higher level of fidelity. So a lot of times I see customers, they still have the idle process of ticketing process, but instead of a human generating a ticket and closing a ticket and updating a ticket, now systems do it. As we have done more process automation at the DevOps level and at the platform level, these things just automatically get opened and closed so that the tracking is there, the logging is there, the discipline around, you know, the troubleshooting and monitoring and logging, that all is still a very much, very much a very critical discipline.

14:41

The remediation approaches have changed as you have gone on a platform approach, but also as you have swapped out the underlying virtual machines, which are persistent and long running to more of these ephemeral containers, what has happened is you don't, for many basic remediation steps, you don't need a human in the loop. You just throw out, throw the thing out and you reinstantiate it or you reboot it and the automated platform can do it, right? As you know, one of the things that Pivotal pioneered is this idea of repave. Now, in an idle world, the repave would have required, repave, what does repave mean? Essentially, you're taking your entire infrastructure that is running the app and with no downtime to the app, no downtime to the app, you're continuously fixing and repaving the underlying infrastructure. That would have required a lot of tickets, manual work, a lot of process coordination. But if you think about it, it is still there. There's a very strict sequence of repave. But the advances in technology with containerization allows us to do repave by just throwing away some instances or moving them to easily to some other location, fixing the host, updating it, upgrading it, and then bringing it back into configuration. Now, all those, instead of being individual steps, are consistent steps that have been done in the past that can now be automated as part of platform. So all you do is issue a single command called repave the infrastructure for me, and it happens. So that is really the power – platform engineering doesn't live in isolation. It's built on top of these subsequent layers of automation and consistency that we have built. Like you were saying, to repave your infrastructure in the olden times would require a lot of, like, you know, runbook thinking and kind of – Runbooks. Yeah, tickets and approval meetings. And it's almost – you know, there's maybe two causes of that.

16:44

One of them is just didn't exist. Whether you're using VMs or, like, containers, right? Like, it's just so much faster to do what you were describing. But there's also – the other side of that feels like a certain amount of trust that it will work, right? I feel like a lot of the times there is a governance in sort of, like, some service delivery or an application. It's because some people don't trust the computers enough to do it on their own. Like, they want to have a checkpoint in there. And, yeah, I mean, like you're saying, I think over the years people have just come to trust that it will work. And the way it happens is that it's thanks to some of this process automation, right? Yeah. Again, as I say, if you have seen a runbook work 10 times in a row and consistently in a row and consistently deliver results, now you say, hey, I can maybe call that runbook automatically. Well, maybe I can now automate it as part of my subsystem. So these don't just happen overnight, but the fact that you can do something consistently multiple times in a row gives the trust to automate it. And that is where what platform has benefited from. And that's what we are trying to leverage as part of the platform engineering process, which is, hey, can we automate things that you know are consistent? To some extent, boring and easy. It doesn't mean governance goes away, right? It means that instead of doing the, when you do repave, if there were 20 steps and each of them was a separate ticket with its own approvals, now there might be just one approval at the top saying, hey, do you know what is the time window? Do you know it's a low risk time? And have you got the right approvals? And then you issue the repayment, right? That would be an example of an approval that might be still needed for that repave action. And that would still go through the ticketing system. But you're not trying to orchestrate every little piece of the runbook. Through a ticketing system because you now have enough, we have enough comfort in the automation. We understand the, each of the layers have been automated enough.

18:46

Right. And proven itself to be consistent, right? Like that's, if you have inconsistency, there's no way you can trust it. It has to behave the way you assume it would. You cannot automate something that you can't do manually two times in a row. Right, right. Yeah. It's a good rule of thumb. All right. So, so after the, let's call it the ITEL era, the BMC era, where, where do you go? Where do you go after that? This is like 2011 or so, or 2008? Like what, what year do you go to the next? 2008, 2008. And, you know, sometimes it's one of those things where you say, okay, you know, you, you think of yourself as a startup person or a big company person. And I always thought of myself as a startup person, but I have been in big companies now for a long time. And so after BMC, went to a company called Mocha 5 and pioneering started by a professor from Stanford, Monica Lam, and Vinod Khosla's company. So very interesting, a desktop virtualization. And the idea was how can you give customers a desktop practically on a USB key that you just carry anywhere. Yeah. Put it into the system and be up and running. And so it was both offline yet available all the time, always synced when it was up and running. And so VDI was already in play, but still, if you think about it, you know, I think when you think about innovation, right, just like the first phase of loud cloud, the idea that I can have a shared server and still be secure was mind bending. Right. But now it doesn't matter, right? Everybody, fast forward, now it's very, everybody's comfortable. When we were doing Mocha 5, VDI was there already, but the connectivity was not as good always. So people, the need for a desktop that you could have with you with all the data that you could carry around was very valuable and without necessarily having a device. And so, but now if you think about it, fast forward again, with 4G, 5G, et cetera, the connectivity and disconnected mode of operations is less critical, I would say.

20:52

Yeah. Most people understand, right? Especially with phones and apps, all of them require connectivity all the time. So you're a lot, lot more comfortable. And so that was the premise of Mocha 5, I mean, really a fabulous time. And after Mocha 5, did a cloud security startup and then a container security startup. And so different, so when you think about, it's not just about automation of infrastructure, but just various aspects. How does the automation power the user experience, how does it power the employee experience, how does it power the partner and customer experience, how do you make sure that all that experiences are then secure? So those were, and container, especially Kubernetes had started to build up in popularity and so did a container security startup. And then I joined VMware, primarily doing cloud management or automation. While we've gone through many phases of that cloud management journey at VMware from 2016, I would say onwards, synthesizing it down to, it's not about just delivering a VM. It's not about doing individual network storage and compute virtualization. But early on itself, the role of cloud management, which was we realized and later branded ARIA, was how can you bring all those pieces together so that you can give a set of services, public cloud-like services that were complete and comprehensive? And how can you ship a set of management software or provide that management software as a SaaS so that customers can do it themselves? That was the big part, right? Which is, hey, you're not beholden to just the set of services that you may have from a private or public cloud, but can you curate your own set of services on-prem or on your captive infrastructure, whether it's on-prem or not, that have a similar type of experience, easy for the developers to consume? Yeah, that makes sense. So I have two last questions. And one of them is, now that we've kind of gone over various places you've worked, I wonder if, like, what are some ways of, like, working and managing a startup that you find helpful to apply in a larger organization, like we're in now and at VMware?

23:08

Like, what are things that you bring over? And, you know, if you also want to make it interesting, what are things you don't bring over? That's a wonderful question. So there's some aspects of startup that you should absolutely keep. And it is relentless focus on product market fit. Just because you have a mature product that customers are paying a lot of money for doesn't mean you can rest on your laurels. Right. You have to see how are the needs for that particular service that you're offering customer software or service evolving and how the ecosystem is evolving and constantly test and make sure that you're driving that innovation that is needed to help them with the goal. It doesn't mean you're changing your goal every time. Right. The goals actually stay pretty consistent. Like in our case, my goal is singular goal. I need to accelerate quote production. It's a very simple goal. If it used to take three months, today it has to take three days and then three hours and three minutes. That is the goal. But what new technologies are getting introduced, new approaches are getting introduced, new services are getting introduced, how do those change the landscape and how can you continuously keep not only up to date with it, but take advantage of that. Leapfrog. And that curiosity is something that is always inherent in a startup. So definitely you have to keep that curiosity going. The other thing is, generally you tend to in a startup have less red tape in the sense the different silos, whether it is marketing and sales and product and engineering and, you know, support tend to not talk as much or even if they talk, they tend to talk as much. They tend to be a lot more siloed and fiefdoms that are established in a larger company. In a startup, that's not the case. You really, all of you just are so focused on survival and getting the first land and making it happen that you work together. Actually, there is nothing that prevents you from doing the same thing in a larger company. That's what I've realized, especially, I think, and with Hawk's approach.

25:25

The Broadcom Hawk creates these divisions almost like a mini company. And so you have all the functions fully aligned within a division. And those functions better roll together in concert towards a singular goal. And when you do that, really the movement is much faster, it's a lot more efficient, and you actually get better results. So that is one thing that I would translate, might seem counterintuitive, but I would definitely translate from a startup to a large company. Yeah, I like the, you know, if I were to, I don't know how I would file it in my head, the startup aspect you were saying is, in a startup, you're trying to get product market fit, like figure out the product. And like you're saying, you know, you got to land a deal and, you know, it's, it's constantly getting to that initial, like getting to that initial phase. Whereas in, in contrast, it's not like you don't do that at a large organization, but you more shift into continuous product market fit. We are always going back and revisiting, like, do we still have product market fit and like, has this changed around and, and yeah, I mean, I agree with you. It's like, I always think unhelpful bureaucracy is a choice that, that organizations make, like, they're not always aware that they're making it. And you just kind of like flow into that river of bureaucracy, but eventually you can just decide, we don't want to have all this red tape and, you know, do something about it, which, which is nice. Well, so the last question I was going to ask, so we spent, you know, we kind of started off talking about like, um, where's DevOps and platform engineering, how we progressed with that. And do you think we'll get to the point where we don't really have much separation between development and operations anymore? Like, because a lot of what we've been talking about is it's almost like if there's operations at the, at the bottom of the stack and developers are at the top or the business, you know, running at the top, a lot of what we talked about is kind of like bringing them ever closer and closer together. And kind of having them help each other, but it seems like there's still plenty of operations roles and plenty of development roles instead of it just being kind of fully automated and only application developers running around.

27:33

I don't think that that is, that that is going to happen because it's not a feasible approach. When you think about a customer's environment, one of these large healthcare firms, financial services firms, doesn't matter, retail, and they are building their own environment. There will always be the role of operations and the role of platform engineering doesn't go away. And that is fundamentally different from the role of the team that is writing business logic to do, let us say, grocery transactions. And so the goal of infra and platform is to remove as much of the complexity and present simple primitives to the developers. Right. But that role doesn't change. Somebody has to do it. If developers themselves have to do it, that doesn't help. So there are only two scenarios then, you know. One is you could go to a cloud-like offering where everything is set up for you and those primitives are all you offer and you buy them by the SIP or something like that. Right. As a developer, as an application team in an organization, which means, of course, you have no control of the infrastructure, no control of the policies, no control, but you're just getting those primitives. And that is a valid model. And you will see the hyperscalers are, you know, offering those models. Other model is you do the same thing in-house with your own infrastructure or with the public cloud infrastructure and curate that experience. But ultimately, somebody has to curate that experience so that the developers can have that speed. So it's not that operations goes away or developers do everything. That's the wrong thing. This idea, oh, I'm giving DevOps and I'm giving developers YAML code. Developers don't want to write YAML code. The whole goal of what we are doing is developers should just write business code. And everything else should be taken care of by operations and platform teams. Right. And someone has to do that to build the platforms and maintain them and run them. So the infra operations, you know, and platform operations.

29:37

So the IT organizations, as you see, are evolving from, you know, OK, now they have private cloud teams and more and more so platform teams. And this is critical because these teams are the ones who offer the set of services with the right set of SLAs and visibility to these application teams. And ultimately, the goal is so that the application teams add at a drop of a hat based on market needs. They can quickly make the changes to their software as soon as possible and get it out as soon as possible. Reliably, safely, consistently. Yeah. Yeah. Well, that sounds good. I mean, that's what we're always shooting for. I think that's good stuff. Well, I appreciate getting the chance to talk with you. It's fun going down memory lane a little bit and use the word runbook again after all these years. That was good stuff. All right. Well, well, well, thanks for being on. And I think I mean, I think the obvious thing is if, you know, we do a lot of great work over here at Tonsu. We've got the Tonsu platform, which is easy enough to find if you go to tonsu.vmware.com. And we're around here and there doing things. But all sorts of platform talk to be had as well. And we'll have to we'll have to have you back to have kind of more discussion about actually Tonsu platform and the technology. That would be wonderful. Yeah. All right. Well, thanks again. That was great.

Tanzu Talk · 🍕 Get more pizza.