The people side of platform engineering, and, what is platform engineering?

Episode 244 · Wed, 02 Nov 2022 15:13:28 +0100 · 49:37 · SRT · archive.org · all Tanzu Talk

You're in for a great discussion about, I don't know, platform engineering, platform as a product, thinking about how you manage the team and even recruit the team of people that run the platform you have. But I wanted to remind you that come December 6th to 8th, we have SpringOne coming up in 2022, in case you're listening to this in the future. It's going to be a great conference where we talk about software development, especially in the Java and the net world, people using Spring, setting up golden paths to production, all the kind of stuff that platform engineer people are concerned about and interested in. A little bit of Kubernetes stuff here and there, DevOps-y things, basically all the stuff you would expect from the cloud native world. And the stuff that I look forward to most, kind of the type of discussion Darren and I have here, is talking with the people and organizations about how they're managing, running that stuff, changing the organization, the skills and practices, the culture, if you will. But we've got all the sessions posted in the workshops. If you go to spring1.io, you can browse Zoom and check them out. And you should register. And I've got a code you can use to get $200 off. And that's Coté 200. That's C-O-T-E 200. So if you're interested in this discussion you're about to hear, these kinds of topics, definitely go to spring1.io and check out the stuff we have. It's probably a good conference for you to come to. And you'll be able to talk with the actual practitioners, do it, not just see the talk that they give. Ask questions, talk with your peers. So check that out. Use the code Coté 200 at spring1.io to check it out and save $200. And with that, let's go see what Darren's take on platform engineering is, the experience that he's had in large organizations putting it in place, and some advice for kind of the long-term viability and scaling up. And really one of the parts I like about the conversation a lot is we kind of think about what it means to product manage a platform platform for, for operations people, which I think is a big shift in what's going on there. Enjoy. I'm going to continue in my series of, I was, maybe, maybe you have a good way of putting it. I'm going to, I'm going to christen this a series called what is platform engineering or whatever we used to call it. I don't, I don't know. I don't know if that's a good one. It seems a

little too forceful, but to that end, you've worked in this area for many years and helped run and build and be on these teams. Why don't you introduce yourself? Yeah. Thanks for having me. My name's Darren Rice. I'm based near, near London in Essex in England. I joined VMware at the start of this year as part of a team called the Tanzu Value Advisory Team. It's a team that had been sort of quite successful in the U.S. for about three or four years, back even when, you know, when we were pivotal. But we decided to expand the function into Maya. I was the first one to join and now we have three members. I know you know them, Brian Ross and Mark Zotner. So we're now at full strength. And basically our team is a sort of executive advisory team. You know, we've come from industry, we've sat in these people's shoes and we can sort of talk through their problems at a business level and outcome level. You know, having seen what to do, what not to do, you know, and we can advise them beyond just sort of, you know, the products that we offer and the services we offer, but also around, you know, how do you drive some of this cultural change? How do you, how do you actually really do this in practice? Totally. Before, before VMware, I was in, you know, finance for 25 years, 20 years of that was in banks. And my most recent role, I sort of ran the global middleware team for, for UBS. Middleware is a bit of an old fashioned term now, I know, but I haven't found a better word for it yet. I mean, it's the. Oh, exactly. That's the, I mean, that's, that's the problem I always have is like, yeah, I know this is an old term, but we don't have a new one yet. Yeah, exactly. So I could make up lots of fancy names, but it's the bit that sits in the middle of the app and, and the operating system, right? So, so I ran that. So, but, and as a part of that, my team brought Tanzu into the, into the bank and we ran it at scale. So, you know, we, when we obviously tried to introduce a lot of these platforms of product concepts, you know, having had Tanzu labs in and to talk to us about all of that.

So, so, so let's, let's, well, well, first of all, yeah. I mean, Mark just told me this morning that he, he, he moved over to your team, which is exciting. He'll, he'll be great there. He, he, of course, he's been in this, I don't know, channel of podcast video stuff at some point talking about our legacy trap stuff and the work we did there, which is, which is always good stuff. And then you have, in addition to English, you've got German and French covered. You can do some, some TV and other things. Well, he not only increases the team by 50%, he increases the languages by 300%. Exactly. There you go. So like, like, let's start with like, I mean, I don't know what, without getting too like hung up on like, you know, the last, like, like six months or so of the phrase platform engineering. And, you know, as you were calling it platform as a product or running the platform or just running that like centralized thing that, that your developers use and that they run their apps on inside an organization, like, like, how do you think about what that is? Let's just call it the platform team. Like, like what, what is, what's going on there? What, what, what's the roles and responsibilities it has in the organization? Yeah, I think, so I was at, you know, banks for a long time and I saw things evolve from, um, let's say traditionally how people did it is people tended to order, you know, infrastructure and middleware in a kind of, you know, you would put something on a shelf, you'd say, okay, this is a web server package with some scripts around it, off you go and install it. And each person would go and individually order their server and their operating system on their database and so on. Right. You know, that's how, and our job in, you know, go back 15 years was to create these kind of generic sorts of self installable sort of pieces that really was impossible to put any standards around. It was not centrally hosted. It was, so you were more sort of providing a product in a very kind of put it on the shelf and away you go. Good luck. I think the, the, the change for when

you start talking about platforms and the concepts existed for a long time and, and people built things like web farms or, you know, shared database clusters or whatever it is, you know, I think we just sort of added new terms to it to call it, you know, platforms as a product or whatever. It just means that the sort of, it's that multi tenant, you know, someone's going to centrally own the responsibility for certain things. So the developer doesn't have to care about it. Right. The developer doesn't have to order all the pieces. Right. And so you worry about things like getting the base infrastructure, the capacity around it, right. How the platform is generally running, how someone consumes it and so on. Right. That for me is what it's all about. Um, a bit like, you know, it's having a landlord rather than you owning your own complete, how I use sort of house analogies quite a lot, by the way. So it may not be the first time we go into a house analogy, but, um, but that's the important thing for me is that suddenly you've got experts who can concentrate on what they're good at, which is hosting, right. Which is running, whether it's Java or web apps or databases or whatever it is, providing that for you and it started a long time ago, but then we got better at it. We, we had to do it originally using our own homegrown tools and just sort of sticking this stuff all together, sticking load balances over it and stuff like that. And then of course we brought products along like, like Tanzu. Um, and of course that cloud does a lot of that for you as well. Right. But the concept I think has existed for quite a long time. And, and I think, I mean, I think the, the, so one evolution of the idea of the, uh, let's call it the, the, the centralized services team, as, as, as we were joking, you know, like, I think, I think it's, it's a very, in a good way, a very mature, well-known thing that you, I mean, to go back further into history, the point of middleware is like, these are reusable common services that many different, uh, developers inside an organization

use that generally you don't want to code yourself like a database most famously, right? Like, or, you know, a message bus, or it's just sort of like, I don't know, the plumbing that would be ridiculous for you to write on your own. And so generally whether, I mean, I guess there's three, there's three things like one, you might actually write it on your own, but then it's used by everyone, right? Like it's, it's, so it's, it's not like every team doesn't rewrite their own stuff, or you might use like an open source thing and you kind of like manage that and run it on their own, or you like, similar to that, you buy, you buy it from a vendor. And, you know, I think, I think, you know, most, well, at least I, you know, I stopped coding in 2005. So, so all my knowledge is based on J2EE, right? And like, you know, that was a huge stack of middleware that basically aspired to just do everything. Or as you were saying, web service, uh, farms or clusters, whatever. So anyways, it seems like, like about, I don't know, around, at least this is when I heard about things about like 2015 or, or whatever in, in the, the platform as a service days. And this is when I joined Pivotal. Like we would talk a lot about as, as a phrase you used, uh, a little while ago, like platform as a product. And, and I think it was that as a product that was kind of like the evolution of the centralized middleware, which is what if we like product manage what we're providing here, which, which is a shift from, and I mean, tell me if I'm getting this wrong, like I'm going over it to kind of validate my fanciful notions, but it seems like with the traditional way of doing middleware, your focus is more on we're delivering some middleware or services that have to meet a set of requirements. Uh, that they, they need these, these features and they need like this uptime and they need this blah, blah, blah. But their, their job is to meet the needs that we have specified.

Whereas the slight, slightly, but, but constant, very consequently, more important thing of the, as a product part is let's go find out what the developers need. And like, let's continuously kind of like gather the requirements from developers by working with them and evolve what we're doing by figuring out what makes, let's treat them as the primary customer in, instead of the enterprise architecture as the primary customer, I, I guess, I don't know. I mean, does that seem like, is that a fair characterization? That is absolutely what happened. Um, because I think traditionally these platforms were just built by engineers and it was the same people that had engineered things in a traditional way that we're also providing, you know, the platforms for people to run on. I think what we did across the board, and this is not just true of middleware, we did it across the, you know, I wasn't the, uh, the genius inventor of the, this change of approach, right? But it was, but I was one of the implements of it. Um, it was, it was that change in mindset. You know, we changed our titles overnight from engineering leads to service and product leads, right? And that was a significant sort of mind shift. And we also started putting people into the positions, um, that maybe the traditional engineers didn't feel comfortable doing, but it was that interface to the customer, which would typically the application owner or developers, right? Um, I think in large organizations, when we talk about developers, actually, we talk about lots of different types of people, because there's people that are bashing out code, there's actually people that can actually write a line of code, but they're actually owning the applications and responsible for how the application gets, you know, its roadmap and how it gets deployed and what infrastructure it uses. But anyway, I'm sort of digressing there on what is an app owner, right? Sure. But certainly in terms of the roles of those service and product managers were much more around, you know, have you published your catalog of what you're actually offering and all the different t-shirt sizes and how, how do you consume it, right? You know, what does it cost for the internal user to use it? Are there other options? Have you got a roadmap from where you are

to where you need to be? Have you got information about what's the top priorities that the you'd like us to focus on? Is it end of life? Is it compliance? And then how are you running regular review sessions with your customers to find out what they want? So you can give the information two way. And again, I'll make sure it's a two way street, because we were often saying to the customers, by the way, we do want you to still worry about end of life and, and compliance and things like that, which is what I'll come on to in a minute, right? But it was them telling us, yeah, but we really need X, Y and Z. Whereas I think you're, I think you're right in the past, it was very much probably architectural CTO led saying we should go to technology X and our platform should be based on Tomcat, not web sphere or, you know, yeah, yeah. Tomcat, not Oracle. Yeah. You know, we, we didn't, we, we changed it because of those kinds of drivers, not because, oh, we need this particular feature. We need to start introducing different ways of doing this stuff. Right. Yeah. Yeah. No, and, and, and I mean, you, you went through a great list there of, uh, like, like thinking about your, your software or, or whatever the, the, the technology thing you're creating as a product, right. With roadmaps and like, like I, I, you know, in, in a, in a comical, but very serious way, so to speak. Right. Like just publishing the catalog of services you have and making sure people can access it is often overlooked. Right. Like, like, and, you know, it's just a form of marketing basically. Right. Like I, I remember, uh, I remember hearing, I forget where this was or who it was, but I remember someone was talking with, uh, uh, like, you know, the infrastructure people and, and they, they were, they were talking with who, I think this was one of our, our people who was talking with them and they were saying like, oh, you know, we, the developers keep asking us to like set up a public cloud access. And so we need some way to do that. And, and then they kind of went down to the people who are providing the service

and they were like, yeah, we have that. Like you totally have public cloud now. It's not a big deal. You can just do that. And then, and then, so the person asked them like, well, what, like, how do you do it? Cause the developers don't know about it. And they're like, oh, well you file a ticket and then we'll open it up for you. And it was kind of like, well, that's why they don't know about it. Cause there's not like, there's not like a catalog of offerings that, or even worse, there is a catalog of offerings, but there wasn't enough time spent to make sure the developers knew about it. Right. And I think like, like to kind of embellish or, or like add kind of a, whatever to the list you were saying, like, I think a couple of years ago, like I kind of, the way I also started thinking about it is like, well, I've always worked for software vendors and we sell our software as a product. So I know all the activities you go through to do that. And it's kind of like, well, you just use those activities for internal it now, which, which I think is a whole lot of different stuff that you add in, uh, as, as you were listening there. Yeah. Um, I think when you, the marketing and, um, sort of advertising almost internally of what you do, I think is a very important thing that often gets missed with traditional engineering teams. I mean, people will tell you we have, oh, we have documentation. Like exactly. It's not that it's not that you don't have something written down, but typically, I mean, and I was guilty of this when I was an engineer, right. Your documentation consisted of a, of a confluence page that started with a Unix command, right? It's like, well, no, you're, you're starting down here and that, that's still very important stuff that might be needed by technical people. Once you get into the, you know, how do I actually install this stuff? But no one really started at that bit before saying, you've already assumed that I'm, you know, I'm inside the car and I'm asking you, I've already picked the car. I know how to drive it. Now I've got detailed instructions on how to turn the key and what the buttons do. He's like, no, stop, step back from that. You know, how do I even know what I want? How do I get to

that confluence page in the first place? How do I know these are the different 12 things that you're offering? And also more importantly, how do I know that these are the ones that we, you want us to go, you know, that sort of order of preference, right? You know, decision trees that we then, as we got more mature, we started putting decision trees, not just here's a service catalog, do what you like. Cause I think one of the dangers of cloud is there's so many services. As soon as you open it up to people, there's, there's 27 different ways of doing it that you don't necessarily want to do it in 27 different ways. You want, you might want to offer two or three, but you also might want to say some kind of guidance and decision tree on if I'm doing this, then we really, you should take this one. Right. But if you really need that, then go to this one and then that one, not just here's the cloud, knock yourself out. Right. Yeah. Yeah. So, I mean, there's, there's a lot of like threads to pull on there, but one, one that, that I'm curious to hear, you know, having, having done this, this is your take on is how do you, um, how do you make sure that you keep doing the work to make things better? Like, like that's, that's kind of a ridiculous question. Right. But, but another, traditionally, like another thing that I think people encounter with, let's call it the middleware, like service delivery thing is that especially in enterprise architectures is it's, it's hard to kind of like gather the feedback. If the decisions you've made are the right ones and then make sure that you go and fix it. So for example, something as simple as you were saying of like, well, we posted a page about how to use, uh, whatever, how to package your stuff up to deploy. Like that's a thrilling thing. And, or, or like, you know, also like how to, how to set up a, a developer and a testing environment. Right. Like that's a common thing to get started with. And like in, in the ideal state, um, I don't

know what the period is, but like the person who the P the team will maintain that page would go and say like, well, is this working? Right. Like, is this good? And how would we know if it's good? And maybe we should change it. Right. And then, I mean, that goes on even further, like, oh, now we need to, uh, we found a need for developers to have a new type of service or a new type of configuration. And now we need to go do that. Right. And this is kind of like the product type is to use a way that some people say is like, you know, a project ends, but a product never ends until you kill it. Right. But like, and, and so it seems like for us vendors, it's easy because like, we want more money. So we keep paying attention to building it and we keep doing it. But it seems like when you're delivering in a project delivery mindset, like it's a new skill to learn that like, we've got to go back and revisit and revise this stuff, not just sort of be done with it. Well, I think the first thing you just hit the nail on the head, which is you don't run everything just as a project. Right. That's why we had to shift to sort of an ongoing concept of you own this platform or products forever. Someone once said to me, when's this done? And I said, Oh, never. Well, when the users don't need it anymore is the answer to when this is done and we'll move on to something else. Right. But until then, then it's never done. I think that's why we were encouraged and I'll get on to how you do this. But, you know, having those reviews with with the users, having internal, you know, we used to, we used to have internal chat channels or something like that, whether it's Slack or Skype or whatever it is, right, to encourage feedback. And we had, you know, part of that marketing side is, you know, leave feedback on, whether it's on Confluence or whatever it is, you know, have ways of having feedback. And yeah, you can't do everything, but then work out ways of, you know, prioritizing it. Obviously you will have multiple customers. Yeah. Because you'll have different lines of business and different maybe objectives. But if there are often, if they're saying, if you get multiple

lines of business or customers saying the same thing, then that's a driver to improve. And then you do that by setting the goals and objectives of the team managing that platform to be that improvement. You just deliver this project and you'll get your tick in the box at the end of your annual review, actually, you have to set those goals at the start of the year to say, well, you're a product owner. Your goal is the following and whether that's, you know, improvements in surveys or whether it's, you know, you have to make it up what, how you measure it, but it's got to be something that's measurable in terms of the platform. How many service reviews did you run? How much positive feedback did you get from those? Are your roadmaps in place for the next three years and always being kept? And as long as you sort of people get motivated by what their goals are, right? Yeah, that would be great. The goal is to finish the project and you're done. Yeah, that'd be a great thing to contemplate at some point is like, what are management metrics for platform engineering and platform as a product, right? And centered around like, I mean, I think you're identifying it is how do you, to make a verb out of it, how do you metric size product delivery? And, and, and I think, I think the, you know, I mean, as with so many metrics, right? Like whatever metrics, they're great and bad, who knows, just like sugar, like sugar can be terrible or it can be wonderful, right? Like just depending on what you do with it. And like, it seems like there's two types you're going over there. One is like the, I don't know, the trailing indicators, kind of like the static things of like, we had every quarter, we had like a customer review meeting and we generated like a backlog from it, which is just like, these are things that should be happening. It's like the blood test, right? Like, and, and, but then the other thing is more trying to capture like the flow and improvement of things, uh, as far

as like, like somehow, as you're saying, you want it, you want to capture that, like, yes, we're improving this. And not only are we improving it, but we're, we're learning what's better. And, uh, you know, figuring out how you capture that stuff, I'm sure is a bit of an art in some organizations of how you map it to how that organization functions. But as with most cases, it's probably pretty standard, the types of things you would, you would start measuring, which, um, I'm always a fan of giving like management, some new metrics to do stuff. Cause like, there's only so much like work they can do beyond like meetings, setting the tone and metrics, like everything else is, you know, it doesn't really scale so well. So they, they need the same tools or similar things. Yeah, that's, that's, that's right. I mean, you know, we, um, I think it's also about trying to tie into actually what the end goals are, because you could have specific goals about what the platform is, right. Yeah. Which will be very specific to the platform. But ultimately that's why I can't give complete examples now, because it could be, it just depends on what's the biggest driver from the customer's side. Right. And if the customer is saying, you know, what our measurement is time to market. And at the moment it's still even with the platform taking too long. Well, then you can put a measurement around that. If it's, it costs too much, you can put a measurement around how you can trim those costs. If it's, um, I need more or whatever it is, you could then put a measurement around that. The platform then becomes the enabler to those other KPIs or whatever it is that you, that you then address. Have, have you improved end of life? The platform isn't going to do that on its own. You've got to sort of keep that platform getting better and better. So it then has the knock on effect that other things in the organization are getting better. Right. Yeah. But I think it does come from a cultural mindset. You know, we had people come into the organization and, you know, and say, you are service and product

managers now, right? Your job is to continually improve your, your, your platform. That is your job. Not did you create a 10 Linux packages last month? Right. Yeah. Yeah. Yeah. You don't, you don't just do like a, like a, an inventory of the assets and services created, right? Like you, you want to, you want to shift towards like, uh, how, how are, how are you learning and, and, and adapting them to, uh, to be useful essentially? Well, so, so next week, uh, speaking of the people, like we have, uh, we have a talk at, uh, VMware Explorer in Barcelona with, uh, with another, another set of people who've been doing this for a while, uh, our, our friends from Mercedes, the ones who run the, um, who I think for like seven years now, uh, have been, have been running the, uh, the cloud foundry instance, the, I mean, started off being pivotal cloud foundry, and now it's the Tanzu application service. And they've been using that as the stack, the platform for doing all this that we've been talking about. Um, and so it's going to be great to hear from them. Like what I think, you know, what we've been trying to put together in this talk is not so much a definitional thing, but like, what are the lessons learned, uh, from, from those seven years and what does it look like after seven years? Right. Which, which I think, I think will be fun stuff. Um, but, you know, as a little preview of that, the part that you're going to talk about, I think that will be great is the, um, the, the people in the platform engineering team, right? Because as, as, as you and I have talked and you've commented on, like, there's, there's a lot of emphasis on the developer side of stuff, which is cool. And, and I think what I'm always interested in is not, not only how you sort of build the new role and, and even hire for a platform engineer on the team, but, you know, usually in a large organization, you have a lot of great skilled, uh, you know, good people already doing operations stuff and like, I don't know, maybe they can move on to doing this.

Right. And, and so the, so the question becomes like, so how do you, I mean, it's hard to phrase this without it sounding like negative, but I don't mean it to be, but like, how do you just work with what you have instead of like coming up with an entirely new team and kind of motivate people and train them and all of that. So what's, what's been your experience with that? Like working with the people that you have, I mean, I think there's always a bit of a mix because I mean, I've seen it done badly both ways and I've seen it done well. Right. I think what I've seen it done badly is either you don't bring anyone in at all. I'm not necessarily saying that has to be new permanent members of staff, but if you don't engage some kind of consultant or vendor or someone to work with you, that's done it before, I think that's a mistake. If you just try and you can, as you say, you've got some good people, but they need some guidance and steer. Right. Teach them to fish kind of thing. Right. I've also seen it the other way where projects have, they just brought in a completely new superstar team that work goes off into a lab and builds it all on their own. And they don't know anything about the organization and they come out and go, what do you reckon? And then everyone goes, this just won't work. And it also builds resentment in the old team saying, why do these guys get to work on the fancy shiny stuff? And we're stuck doing, you know, the traditional stuff. So either those two things don't work. I think you need a blend. You need some new people, some new ideas, and at least bringing in from maybe places that have already done it. But I think the vast majority, as you say, you need to then educate and train and mold your existing team to work with those new people and make them part of it, make them feel that they are part of it, not that they're being replaced. Right. So that you don't end up with, you're going to get a few Luddites, right? You're going to get a few and they're probably the people that won't be working for your organization for much longer anyway, because they'll say, you know, we've always done it like this. What's wrong with the physical server? You know, it works perfectly fine,

you know, and they'll put, so those people, you know, but the vast majority of people are willing to change. It's just, maybe they just need a bit of a steer. Right. I think the most important thing is get your own team to buy into the concept first, right? Before you start going, selling it to the app teams and developers. Um, obviously you probably might buy in from your manager as well, but before you start going out to wider stakeholders, you've got to try and convince your own team. And there's a few different ways of doing it. There's obviously incentives and there's also not so threats, but you've got to sort of make people realize that you've got to move on. The industry moves on the world moves on. Right. So the first thing we started with was from an ops perspective, um, understanding what is now expected of an ops team. The bar has been raised, right? Dining out on uptime was something we did in the nineties and maybe early two thousands. Right. But no one now says, um, Oh, what was great about your iPhone? And he went, Oh, it worked. Right. No one says that anymore. Right. Oh, it was on 99.9% of the time. You know, well, it probably wasn't because of the battery, but that's a different story. Right. But, uh, but, uh, you know what I mean? For, for banks, for retail, for any of these important government, you know, everyone just expects it. You know, the retention, I think there was a survey. I mean, you probably know better than I do about what the expectation now of an average user is. Right. And it's less than 30 seconds. If you're all right, yes. And, or if it is, even if it's slow or unusable, they'll move to something else. Now banks are a bit stickier than other organizations, particularly retail banks. We were talking about it last night, actually. Um, so if you've got a, a retail app and it doesn't deliver your groceries, you'll probably switch, right? If your bank, if you can't get your balance, it's going to take quite a lot for you to uproot all of your,

you know, account and move. But, but if you look at younger generations, they are doing that. They are getting the new, you know, the new banks. So, so banks have also can't rest on their laurels. So it's also around understanding we have to move to survive. And, and within that an operations team cannot just think the stuff that used to be important, uptime, capacity, you know, and, and that sort of stuff isn't going to cut it. Cloud gives it, gives you that out of the box. Right. So, so, and if you don't want to be replaced, you need to move on and be a, you know, a different sort of, you have a different mindset, get involved in, in, in some of the more interesting stuff. And so let's face it, producing graphs on uptime and how much capacity is left is pretty dull as well. Right. Yeah. Whereas, yeah. Right. Whereas, whereas writing automation or, you know, making the platform better, that's what engineers actually want to do. So that's my next point is traditionally in an organization that doesn't think in this mindset, you probably got a very, you mentioned it earlier, a ticketed approach. You've got very ad hoc requests coming your way because you haven't got a standard service catalog. Your life is probably not too happy because you get beaten over the head because you're not delivering these bespoke requirements every day. Right. And in some cases you just, you just can't do what the customer's wanting you or asking you to do. If you move that mindset to you, you work on automation, you work on constant improvements and you are making the customer happy. Well, that's a mind shift change, right? Then your day life, your day job will become happier. Right. But the other, the other example was rather than, I mean, I probably answered the same question a hundred times because we hadn't got it documented. People didn't know where to go. So compare that doing that all day long to creating a community, creating a chat channel.

The best thing ever was when a different customer answered the question that another customer put on that chat channel and our team. Oh, right, right. You got, you got the internal stack overflow effect where you can, you can do support by users talking with each other. Absolutely. Because I think you've probably answered that question 500 times. To me, if you've answered that question 500 times and people used to complain about it all day long to me saying, my day is just swamped with customer queries. And I'm like, have you written it down? Have you written down the answer? Have you pointed them at the site each time? Oh, do you just see what I mean? So it's that, it's that change. But, but again, building that community, getting, getting people to answer for you. Then if you want to, if you want to grow, you're not just adding headcount to your team, you grow by building champions within the, the app teams. Right. And then, and people resonate with that. They say, you're right. I don't want to be spending time troubleshooting the same problems, answering the same questions, doing ad hoc requests all day. I want to be working on cool new stuff. Right. And that's, that's how you incentivize engineers. I think the other thing is people care about their reputation. Right. I think traditionally infrastructure and ops teams, are considered expensive and a bottleneck. Right. That's how they're viewed by the business. A necessary evil almost. Right. Which I think is grossly unfair. What people want to do is they are, you know, celebrated for being experts in their field. It's, you know, and you, you know, what people actually like is, Oh, you've really helped me there because you really know your stuff. Yeah. You know how to use this platform. You've helped me steer my application in a way. Right. And that's the kind of stuff that people actually really like. And when they start doing it and seeing it in anger, right. Their daily life changes for the better. Right. And I've actually seen it happen. Right. Of course, you've still got some of the old stuff that you have to occasionally deal with because these organizations still have a lot of legacy, but you want people to be working in that new way. And I found that most people bought into that. Right.

yeah other area is also about sorry but i'm sort of in my in my flow now and the other area is about tying it back to some of the things that operations people do care about i mentioned capacity and end of life and and compliance right um i think unfortunately infrastructure teams and operations teams have traditionally been sort of beaten up by auditors saying you've got all this end of life you've got lots of you know stuff that isn't patched you've got non-compliant systems what are you going to do about it and often the answer is well i can't do anything about it because it's the applications that need to make the change to either upgrade or you know i can't necessarily depends on where you are in the stack but when it gets into sort of middle areas i just can't upgrade the java underneath someone right without them sort of needing to test their application i think if you can say but if we help the developers they can help us right if you if you deliver a platform that's always evergreen that's always compliant this that's always you know you you know adhering to our standards when it comes to an audit your life is not going to be miserable going around a thousand servers trying to show an auditor what on earth state that server is in so it's that sort of cyclical thing which is what i think devops really should be about it's yeah ops helping devs help ops yeah and and that's something that people who've even with a traditional infrastructure mindset have bought into i mean you know we had a situation where we're talking to a customer and they didn't want to go any faster from the infrastructure side because the data center was full and we realized that the data center is full because developers are hugging the infrastructure because they don't trust the infrastructure team to deliver the capacity when they need it right so it gets into this cyclical problem and we said you do realize that if you built the

trust up in the developers that the platform could deliver you what you need in a day they would give up all the capacity that they're hugging because they might need it once a year for some i don't know some regulatory test that they need and otherwise it sits completely idle right so it's it's trying to get that sort of we all help each other we're all wearing the same badge we're all in the same company kind of mindset yeah yeah and i remember i mean to highlight that i remember i remember us talking about that uh last time we talked and and i thought it was and i still do uh i think it's a great example of like you know in the old devops western western thing it's a great example uh practical example of what a pathological organization is like which is to say but you don't want to be that organization no one is like wakes up in the morning says like i'm going to put a pathological organization in place but it's exactly like the kind of unintended negative consequences that kind of occur when both sides aren't trusting and friendly with each other right and and it's it's kind of more lays on the developer side they're hoarding these resources it's not always the operations people but it does turn into this vicious pathological cycle where um bad things are happening you're you're wasting resources because of really like so speak the culture that you have in place like the people relations stuff where the ops people kind of assume developers are going to be wasteful and they're not making it easy for them to get resources and then the developers are like i don't want to work with those ops people they're terrible to talk with and so i'm just going to hoard this thing and make a an under under the desk data center and like whereas if you had a better relationship between them both of them would benefit i mean it's a typical like prisoner's dilemma kind of sort of of uh of capacity stuff but i mean anyways i i want to point out that that's a great example of of the effect of being a pathological

organization but it's human nature as well i mean we saw it with covid how if you don't think you can get hold of pasta or toilet rolls you buy a hundred yeah yeah it's the same thing with with uh with infrastructure if you don't think you can get a vm when you need it you order a hundred and then you sit with them until and then then you think you can use them when you need them and people used to do that right it wasn't until we had to introduce things like you know um monitoring usage and telling people you're one percent utilized we are going to auto turn you off but to counter that we will give you free vms for 30 days so we gave that trust that and and they knew that you didn't have to go through the any approvals or anything to get a free vm for 30 days you build that trust and and we proved that it worked so there you go we can give it to you if you need it on demand therefore we're going to take away everything that you're not using but yeah you had to do the the trust thing first before you got any agreement to do the auto delete thing second right yeah it's human nature it's human nature it it reminds me i mean this is this is speaking of seven years ago is one of the things that one of your other teammates tony uh from from the home depot i remember in one of his early presentations when he was at the home depot going over how they were setting up this platform stuff he was saying uh i don't know maybe it was six seven months into this uh they had set up their platform and then developers weren't using it you know as much as they thought they were which is which is as you know not cool because you still have this business case you need to make like if you've spent all this effort building up a platform and then six months later it's not being used your your spreadsheet's not going to look good as far as as far as your business case um and and you know the idea that they had was like well what they found out is that people like didn't want to go through the hassle of

accounting for and paying for the usage of that platform and so they were kind of like well so what if they don't have to pay for it because like it's kind of sort of all funny money in the in the first place it's all of our money and as an organization we benefit when developers are running on this platform so maybe we should just remove that barrier for them to do it and you know and we'll sort it out later but the important thing is to get people using it and and to you know again avoid that pathological thing where it's just like you're hoarding stuff because it seems because it's artificially expensive is is what what you want to you want to remove so you're absolutely right we went through the same sort of thing and we did change the way we did internal cost recovery initially people wanted us to have fair and transparent cost recovery based on what it really cost us to run it which is why the new stuff was very expensive and the old stuff was cheap because end of life is cheap right because oh yeah you didn't have to load in all the upfront fees for it yeah we've already written down the servers you're probably not paying license fees anymore because it's end of life yeah probably not got any people left that know anything about it or if you have this one guy like who's who spends a third of his time on it at best so if you think the real cost of running that that end of life system is the electricity right now there's a massive risk to it and unfortunately you know it was a concept that i touted for a while to try and put an internal sort of insurance premium because in my early days i worked in insurance right to try and get the idea that's interesting insurance premium charge against an end-of-life server because of the risk that it then came because if you had an outage the cost to the bank would be hard it never quite flew but i at least tried to to do it in the end we did manage to get the chain the the agreement to say within your area as you

say it's all internal money sloshing around i then have ended up with the ability within my area to say i'm going to make the uh the new platform keepers chips i'm going to make it's a loss leader it's come and stay at our hotel the first three weeks is free and then i started loading up the cost on the end of life to drive people off those platforms even though it was artificial charges now it's not the only lever because internal money doesn't completely drive people off platforms but it's a it's one of a few things you can do right totally totally and and you know i mean i think at the very least it's just like more crap you got to put up with right like as a user right like it's just you know another another barrier and bottleneck to just getting on and doing the work right which and so it's good i mean that's a that's a a good product way to think is just like uh like how how quickly can someone just use this platform and like what's all the stuff not never mind like even before they get to the command line or a button that they click like what do they have to do and then i think you know that that almost seems like the let's call it the intermediate platform uh team's job is to like think about what's everything that has to happen before someone touches a keyboard and and like let's let's make that as easy as possible because that's going to be the next you know i mean it seems like you're constantly trying to figure out the bottleneck and and eliminate it and uh get get people moving along faster well and also removing things that stop when people gain the system because of internal reasons like so you know internal chargebacks and you know isn't it is a fascinating topic uh wow hilarious topic a lot of time because you know i even had the conversation where because we hadn't managed to sort it in the early days of virtualization internally two vms cost more than one physical server and because you were splitting up a monolith and you're often running say your web

layer on one vm and your database on another internally people say well this is more expensive because it looked more expensive to them because they had the same cross charge right but you don't want people to be doing that because you know in real life that isn't the case it's definitely not more expensive to run two vms than one physical but you know you've got to start that's the continuous improvements you have to look at as well what what silly stuff is in the way of people using the platform or not using the platform rather right coming up yeah because or how do you stop people gaming the system to sort of say oh well you know your your platform is more expensive because you haven't sorted out the uh the pricing properly whatever it is yeah yeah yeah well great well this has been a good uh uh you know we'll have to uh we'll have to find some uh schedule some time to talk about all sorts of other aspects because there's there's many other things and we just kind of i don't know took some little digs here and there into it not not in the uh but it's uh and it'll be it'll be fun next week to see what uh what we end up doing in that talk and uh with with the mercedes people um but yeah i mean so uh like like like if people i mean do you have anything you want to point people to about yourself i mean uh any any uh any stuff you want to do a call to action about what what are you up to there on the internet yeah i mean so obviously i'm if people want to get hold of me or or see what i'm liking then obviously i'm on linkedin um just be cautious of the spelling of my name get his daran rice as we discussed just before yeah yeah because uh because everyone spells it wrong or as i said to you before maybe everyone spells it right except my parents exactly so it's daran rice okay on linkedin uh you know and i've recently also along these sort of lines wrote a blog um around how kpis drive behavior which is sort of right a classic sort of tale of um how transformations

can go wrong if you set the wrong behavior kpis or okay to be honest it doesn't really matter it's about the fact that people will they'll achieve something based on what their their goal is right yeah yeah sure whether that is the right goal or not so you know if that blog can be shared that's a published tanzu blog that i wrote that maybe along these sort of lines would be interesting for people to read other than i think i think especially you know i i have i have three kids in the heat of becoming wonderful people right in the middle and uh it's easy to forget when when you're you're you're you're parenting that in a corporate setting people are generally very good at doing what you ask them to do as as represented by metrics so to your to your to your point you should make sure your metrics are correct because otherwise they'll give you exactly what you asked for it even though you didn't want it which is uh which is good well so i'll i'll put a link to that and uh in in the show notes we have or which you can find by going to uh tanzu talk.com and finding this episode in in the list and um you know there's also uh i'll also put a link to there's two other things one uh you know we mentioned uh platform as a product we have a great paper that was updated i think last year kind of just going over the concept of platform as a product and a little more detail on what we were talking about and also we have a there's a there's a great gartner report that we uh licensed that goes over i think it fits in this space it kind of goes over the idea of an internal developer portal which is which is something adjacent and overlapping to a lot of the platform stuff we were talking about that would be another thing to talk fun to talk about at some point kind of narrowing down to more of the the idp backstage role in here which i think is always fascinating to uh figure out what the venn diagram is but i i think that's a great report to look through because it's very um it is the style of gartner report that is very prescriptive and lists a bunch of stuff

which i think is often useful in thinking about it just that you know the other style which is always nice is just like here's a fascinating great new idea isn't it awesome but you you sometimes are left thinking about what do i do now after that which is totally fine but this one has a little more practical uh and with that you know if if uh if you don't subscribe to the podcast you should go to tonsutalk.com and add it into your uh your podcast listener just search around your podcast thing for tonsu talk and subscribe to it i also recommend uh anytime you get access to someone's phone or device whether it's family friends or enemies you should subscribe them to it too so it downloads and give them a little surprise uh to follow there and with that we'll see everyone next time bye bye

Tanzu Talk · 🍕 Get more pizza.