”Running like Google,” the CRE Program & Pivotal, with Andrew Shafer (Ep. 56)

Episode 43 · 47:44 · SRT · archive.org · all Tanzu Talk

Well, we have a little special episode here. No Richard about. I don't know where he went to. Maybe he tried to find some umbrellas or whatever up there in the Pacific Northwest. I have a much-returning guest on. Why don't you introduce yourself? My name is Andrew Clay Schaefer, and I've been here before. Yes, many times. It's a pivotal thing. So you had a blog post last week going over us being involved in Google's CRE program. And I think if I understood it correctly, we're basically – well, Pivotal Cloud Foundry is the first platform that's in that program. And I thought it would be good to have you on to go over what that is. And I think more importantly and for me more interestingly than just being in the program, there's something about a lot of opinions and time-tested advice about how you run and operate and think about how you run your platform and infrastructure that all of that implies. And it would be an interesting one just to hear, like, what this CRE stuff is and how it kind of – I don't know, the philosophy of operations and running a platform that's embedded in it and it embodies. So why don't you start off with just kind of covering the basics of what CRE is and what this program is and what it means for us to be involved in it? Absolutely. There's a few things we need to build up to before you can really talk about CRE. So the CRE was announced by Google towards the end of last year. And the program is an extension of the SRE. And there's a lot of acronyms involved already. So let's start explaining this stuff. So there's the storied internal process and technology that runs Google. And there's a group of people, although you're not – you shouldn't call them system administrators, although a lot of them have those types of skills called the SREs, which is Site Reliability Engineering. And there's a book that came out, and it was recently made available to read for free.

You can read it – if you just search for Google SRE, you can read it for free on web or you can buy it or whatever. And it kind of walks through in long form, and there's a lot of different contributors, a lot of different chapters, all these different aspects of SRE and how they think of themselves and how they relate to the business. And how they are different or the same as anyone else doing operations. And, of course, there's this parallel conversation or maybe even subsuming conversation around DevOps and the trends. And my personal career, especially if you've ever heard me or seen me before, is sort of taking these ideas from the big web and the communities like the DevOps days and the Velocity Conference and trying to bring them into the enterprise minds and bodies so that they can benefit from this kind of emergent, dominant paradigm. And when I talk about cloud native, I always talk about the DevOps processes and continuous delivery and microservice and the rest of that. And that's all related. So this SRE framing, which is now very, very explicit, it's basically Google walking through their mindset, their process. You know, there's obviously not every detail, but it's a 500-page book. And they'll explicitly, in fact, when I gave a talk at the Google Next last week, one of the SREs came up to me because I made this comparison of SRE to DevOps and sort of how those are the same and connect. And some of the SREs were thanking me for doing that, and I thought it was interesting to see their mindset. And in the book, it calls out the word DevOps a little bit, but they also, I think there's certain people at Google that, and probably lots of places, that like to think that they invented everything.

So watching that evolve and having that be really explicit in public is exciting in and of itself. And then the CRE program is taking the process by which the SREs accept applications onto the Google infrastructure and manage it and operate it, and extending that to the Google Cloud platform customers. So they're, and I think it's worth walking through some of the stuff Pibble's done to kind of frame that. And it's not easy. It's not a checklist. You know, you don't just get a rubber stamp. And we're in the process of going through and doing the work with Google. But you get a review from the Google CRE team. And they're basically looking for the things that would potentially contribute or be detrimental to reliability and making the recommendations that they would make knowing what they know about distributed systems and the Google infrastructure to help you meet your service level objectives. So that's, to me, the most interesting thing. And this is where I think that a lot of the DevOps community would benefit from this is that the SRE process is highly quantitized. And in particular, I think that there's a couple great lessons about monitoring and alerting and all that stuff. But some of that is shared by, you know, a lot of the larger DevOps community. But there's this air budget idea that's somewhat unique to Google or at least initiated at Google, which is instead of having these emotional disagreements about when and why and how someone should be able to deploy,

So what Google does is between the software engineers and the site reliability engineers is work through what the business needs from a service level objective. So there's sort of this, you know, three-legged stool that figures out what this service needs to be. And they want to be realistic because every nine costs ten times more than the last nine. So you set some service level objective for your application. And then as long as you're within the air budget. So they have a bunch of monitoring that is giving them, you know, all the stuff and more about their applications. And one of those things is the errors and the availability. And if you exceed your error budget, you can't deploy anymore. And it's not really, there's no argument. It's not emotional. It's like, well, you're not meeting your service level objective. So instead of adding new features to this service, what we all need to do is we need to push the reliability higher. So we're going to do that work instead of new features. And when you're meeting your error budget, you could deploy at will. And I think that transforms a lot of the sort of squishy, oh, we want to continuously deliver. But, you know, we're trying to balance the life of the developer, productivity developer against the life of the operator. And gives you this really nice, elegant solution that is ambiguous. So that's one of the things I'm excited about. Yeah, that is interesting because it's a way of, I mean, like you're saying, it marries together one of the concepts in DevOps, which is if we're going to be running this stuff in production and expect it to work, we should probably have the programmers write things in a way so that it works in production. And scaling up and then also to that point is like, and no one really, well, on the business side, people often don't like working on spending their development time on features that don't have like a direct user facing thing.

So you've got to figure out a way to force them how to, I don't know, they even have fanciful terms like functional requirements and things like that. But it's interesting to institutionalize a, I don't know what you would call it, like a trigger that's basically like, you need to go and work on fixing this stuff so that works in production and versus like introducing even more features and a stopgap for that. Absolutely. There's a few other interesting things about their version of DevOps, which is in the, so the Amazon example, which is very famous, if you build it, you run it. So it's like you have these two pizza teams on a service and they're kind of responsible for the full life cycle of that service. At Google, they took a slightly different version, which is you basically run it until you prove that it won't ruin the carpet. And then the SREs will run it for you. And if you exceed your air budget, you're probably going to get woken up. But for the most part, if you build to these reliability standards, then the operational overhead is negligible. Right. So there's, there's this partnership between the software engineers and the SREs where the, where the SREs sort of adopt the operations. And that's, that's the process that the, that the CRE program is making, is making publicly available to the, to the Google Cloud customers. Oh, I see. That was going to be my next question is like, I still don't quite understand the difference, but tell me if this, if this, so, so the SRE books is, is a little more like internal facing. Like how we operate to run things and the CRE, what's that? It's how Google runs Google. Right, right. And whereas the CRE one is like, if, if you want an SRE to run your stuff, to use some, some language that you like to use every now and then, here's sort of like the contract of stuff you have to do. Like here's, here's the way you need to make your application work and, and the way that it has to operate so that you get to that point.

Like you're saying that, that the, uh, there's, there's not so much, there's not a whole bunch of management things that you have to do that are unexpected or out of the ordinary. So, so the aspiration for the Google CRE program is that they're going to run your applications for you. Like, like the SREs would, right? That's what the blog post talks about. They're going to be on smortems with you. And, and in order to do that, what they're, what they're asking you to do is meet these, these service level objectives with them, go through and review your code. If they want to, if they want to be the ones that are going to be, you know, jointly responsible for the proper care and feeding of that application, they, they want to have some understanding and ownership of what's there. And it goes right back into the, the contracts and promises line of, of, of reasoning that I've, you know, and I'm borrowing from Mark Burgess and like some of these other people are inspired by them. Um, and, and I just think, I just think it's really a gift to see the way that they're framing this stuff and be explicitly in conversations with them about why they think this, you know, will fail this way or, or, or you can expect these kinds of service level objectives with this stuff. Um, and then there's also this implicit, and this connects back to the other thing that I've been talking about for the last year, which is a transition in my mindset that happened a while ago from automate all the things to what you automate has as big of impact on your, on your operational overhead and cost as the fact that you automate. And so they're, they're saying if you, if you automate these things this way, if you have these types of architectural characteristics, then with these kinds of infrastructures and these kinds of architectures that you're operating cost is effectively negligible. And that's pretty exciting.

Right. And, and so, and so like, I mean, like, what are some examples of, of what the, the, the contract is or the things you have to do or things you, you, you shouldn't do? Like, I don't know. I mean, the, the easiest thing as an example, I don't know if it applies here at all. It was like, you know, you got your, your 12 factor example and, and the one on the tip of everyone's tongue is like, don't use local storage, which, which is all simple and makes sense. But like, what are, what are some of the things and ways you have to like architect and, and write your applications and services that they, that they eludicate? So there's, there's a, a, a bunch of things that are kind of, you know, straight off of some of the things that the platform's trying to provide. And that's why it gets really interesting when you start thinking about what the service level objective should be for something like Pivotal Cloud Foundry on, on Google Cloud Platform. So there's, there's a, a bit about telemetry, right? So the, if you read the Google SRE book, they have this pyramid of reliability and the bottom part of the, of the pyramid is monitoring. And I mean, it's really easy argument to make that if you're not monitoring, if you don't have a baseline, you don't understand what's happening in your systems, then, then how do you know if you're doing anything good or bad? And, and, and that's the first thing that they, they might fix. So there's a bunch of baseline around being able to get logs in and out, being able to do, you know, that kind of telemetry. And then there's, there's just sort of a methodical taking apart of the, of the services. How are they set up? How will they fail? If, if this is redundant, you know, what's the, what's the contract that needs to be made about consistency? What's the contract that needs to be made about, you know, these different pieces. So it's kind of a, all, all the things that you imagined architects kind of did when they're designing systems, except for Google's doing it with you. Right.

Yeah. What, I mean, to that point, uh, without getting too grim, I mean, if, if, if you were, I don't know if this is the right framing, but if you were to separate out at least two buckets, one of them is kind of what you were describing is like, uh, here's all of the stuff that you thought was happening, but it turns out people weren't actually doing it. Or technologically it didn't work out very well. And, and then, so there's that bucket of stuff, but then there's another bucket. Like, are there any sort of like new or novel things that come into play that, that like, I don't know, in the early 2000s, some enterprise architect wouldn't have to worry about? Like a way of thinking about how they write their software and architect it? In some ways, I, I don't know. I wouldn't say there's anything that's especially novel to me. Um, at least with the exposure I've had to the people that built the big web over the last 10 years and the conversations that happen at a velocity conference or DevOps days. But, but, but the, the thing that Google brings to the table is this, is this methodological or, um, methodical kind of disintegration and then, and then re reintegration of everything. Right. So, so like they, they want to, and they're, and they're quite adept. I mean, that was one of the things when you, when you see the talks that Onsi gave, we should get recorded, um, at some point soon, uh, about the process with the, with the, the cloud foundry engineering working with, with, um, the Google so that the assessment itself was, was impressive just in, in terms of the speed and the depth that, that the, the Google side took apart cloud foundry and, and had a bunch of meaningful rep, um, recommendations. Right. It's not, it's not like a superficial thing. It's like, oh yeah, because they're, they're the ones in some sense. The, the, the long, the long-term goal is that, that Google is going to be at least partially responsible for operating your, your, um, applications with you.

And they, they don't want to do that. They don't want to take that lightly because it's, it's them right there. It's their, they're the one getting woken up. Yeah. Yeah. And that, that's another thing that like, I think, I mean, it's, it sort of comes through there, but basically, and, and as you were alluding to, some people don't like using existing terms, but they're to some extent offering some managed service stuff in, in this, right? Like you're saying, you can, you'll actually be working with people to help you run your stuff, at least at the, the, the defined level that they'll help you out at, at. I, I would, I would, I would be reluctant to call it managed services, right? In a sense, it's, um, like, I, I kind of feel like managed services implies this thing that you, you've outsourced and kind of handed off. Right. And then you don't have to worry about it. I think what Google's offering is, is we're going to worry about this together and, and we're going to, we're going to think about this together and we're going to be involved in, in the postmortem. And we're going to be, so, so it's not quite managed services and it's, and it's offered essentially for free, right? So it's not like you're paying this thing, but Google's, Google's got a lot of experience running a lot of infrastructure and applications, you know, at scale with a low, um, operating cost. And, and this is why I think it's really interesting. I think it's exciting to see this method, methodological, uh, approach to building infrastructure, architecture and applications being brought into Pivotal because the, the next phase of this partnership and goal is to be able to have that, that kind of rigor be part of the labs process as well. Right. But I would, I would be, I would be hard pressed to call it managed services and I would, you know, if Google wanted to call it that at some point, maybe, but to me, it's much more of a, of a partnership. You're meeting, um, kind of at the minds and, and at the, at the level of, uh, responsibility together.

You're not, it's not handoff. Right, right, right, right. But the point being. Actually, that's an interesting point. Let me digress one sec. So the, I would say managed services is, is in some ways the way that, that most people have implemented that and the connotation that has to me is a manifestation of the old model of IT ops. Like the pre DevOps model, you just outsourced it, right? So you outsource your old style where, what SRE and CRE are bringing is this new collaboration between engineering and operations. And Google is willing to be, you know, part of that equation with you. Yeah, yeah. Yeah, no, I, I mean, it's sort of a pain metaphor, but, but classic managed servicing services is a lot more like outsourcing in the sense that there's a, and now the ITIL stuff is kind of built around this. It's always a, a fraught thing to bring up, but, uh, it's sort of like a black box of a service that you get. Whereas what you're describing is a lot more, I don't even know if it's a box, but it's a lot more transparent in a white box where you're, you're, you're getting, uh, you're getting people to help. As the customer, you're getting people to help you run the application, not totally run it for you and just provide a service or whatever. The key to keep in mind here, and I'm going to keep going back to it. And I've been trying to get people to see this in all the DevOps conversations I had is it's, it's actually that front loaded step that is pushing the operational overhead down, down, down. The reliability is going up, up, up. So now the cost of running that is so low. If you have, if you have all these kind of architectural practices, all this telemetry, then yeah, there's some fault detection, fault remediation that happens. But, but, but your break fix cycles are short and, and, and infrequent. And, and so it's not so much, oh yeah, they're going to run this with you. It's like, we're going to push the cost of running this down to, to nothing.

Right. And, and, and so do you, do you think, do you think it's fair to say, I mean, you, you still hear this all the time that people, uh, people want to like run like Google. And do you, do you think this is like a way to, uh, to like do that essentially? Oh, that's a, that's a, that's such a loaded term. Well, exactly. Hints it being a, a, a good conversation topic. There are things about Google that are uniquely Google. Yeah. Right. And, and, and some ways that was, that was a unique opportunity that Google has had to solve the problems that only Google has and, and couldn't be Google if they didn't solve them. So do you need to run like Google? Can you conceivably run like Google? I'm, I'm not sure you can. At the same time, there is a lot of information that's available to you that will drastically improve how you do these things that Google's made available. Just like you're saying, you can be inspired by how they run, but it may not be, uh, it may not be perfect to get to the point where you run like Google. I mean, that's, that's the implicit question. And that is like, is it actually a good idea to run like Google or run like Amazon or run like any of these, these hyperscale companies? So let's, let me frame it to two different ways. So there's a bunch of the, the SRE book, which is explicitly calling out to these Google backed services that do things like distributed locks and, and, you know, you're saying, uh, storage, right? Like that's not a easy problem to solve at scale. Distributed storage is, is going to be the bane of all the, the cloud builders until they give up. But the, the, the, those services are not available to you unless you're inside Google, right? So some of the stuff that Google does because they're Google, you're not going to be able to do unless you're Google, right?

Then, then the other side of this is all, all these, all these big web companies have operational excellence or they wouldn't be, they wouldn't be able to do this stuff, right? So I think Google has probably the farthest developed technology. They, they didn't necessarily productize it. They didn't necessarily understand how to bring it to market like Amazon did, but they, they probably have, at least in my opinion, some of the, some of the best technology. But those, those things are, are, are beneficial to you to see, to think, and you're going to see more and more. And we're already seeing this in the, in the pivotal framing of these enterprises that want to build what I'll call web scale applications, right? Especially when you start doing the math for what it's going to take to do some of these IoT projects, right? So when you start to have a lot of data that you need to ingest and analyze, then, then you start to look a little bit like, like Google probably did in the year 2000. And, and if you don't kind of think and act like they did, then, then you're probably not going to, it's probably not going to work very well. If you're, if you're going to, if you're trying to ITIL your way to, uh, web scale applications, you're, you're probably in trouble. No, that makes sense. It's sort of like there, there's a, you have to have a, uh, a Google class application to really need and get the benefits of running like Google rather than just adopting, you know, good practices and, uh, and, uh, operational advancements they've made into your existing things. Absolutely. Yeah. Well, what, what, what else, uh, I mean, what are some other like highlights from, from the book? I mean, it is 500 pages. I think you've read it, right? I did read it.

So, so. I did read it and I published a bunch of, um, quotes from it on Twitter. Yeah. I, I think you had the suggestion that you should have some, uh, some open mic dramatic readings from the SRE book, which, which would probably be fun. Maybe with some of that in the year 2000 music behind it. Yeah, we should, we should make those videos, but like, like the, the idea of the air budget, right? Like that's an interesting concept that I think, uh, whether you want to run like Google or not could be adapted into the way you think about, uh, doing, doing your, your operations and your software. So there's a, there's a value to bringing that kind of dispassionate numerical version of the decision about what you can deploy and when. Yeah. The, the other thing, there's so many good things actually. Here, here's another thing that sticks out is this notion of toil. So the, the, the Google SRE teams are, are highly capable software engineers as well as system administrators. And, and they're very explicit about how they hire. So they, they, they put the SREs through the same interview that they put their software engineers through. And you have to be, and you know, the, you can go watch all the Google, um, YouTube videos about their, their, um, interviews and, and, you know, that's, uh, that's a whole nother topic. It could be another hour long, uh, podcast, but these, these interviews are, are extensive and they're intense. And, you know, people that pass them are, I don't know, they're, they, they kind of met a high bar to be part of that Google software effort. And there, there's this explicit thing that's like, well, if they're close as a software engineer, but they also have deep understanding of some other thing like, uh, systems or networks or storage, then, then they're, then they're eligible for SRE. So they, they, they, they maybe don't have quite the same standard for the software side, but they have this deep understanding of how computers actually work, which some software people don't always have.

And, and the mix in their team is about 50, 50, these sort of system specialists and, and then the, the software engineers. And I've, I've, I've heard one of my favorite talks from DevOps days is Todd Underwood, who, who's one of the SRE leaders in Pittsburgh talking about you, you get a lot of the benefit of SRE. If you just took, you know, a third of your developers and told them to go make things better. Right. Like, don't, don't worry about, right. Don't worry about just make things better, make, make things more redundant. Think about how they're going to fail. Think about how they're scale and then apply software, software development to that process. So this, this whole setup is to get to this notion of toil in, in the SRE work week, month, quarter, whatever, there's a goal, explicit goal. That there's no more than 50% of the time spent on toil. And, and toil is what most, most of us would consider traditional operations work, right? So break, fix, incident management, that kind of stuff. Yeah. And the SRE team, according to the book, when their, when their toil budget is exceeded is, has explicit mandate from, from Google to push the, the, whatever kind of window for responsibility, operational responsibilities and toil back onto the, the dev teams. So it's just like, we'll run this for you until, until you're making our life hard. And then you're back, you're back on it with us. Right. Yeah. And, and then it's not explicit. Like it doesn't have a, a story about this happening to anyone. There's no names, but it says that it's even possible for you to get kicked out of the SREs basically. Like if your application is not well-behaved, so, so just like you had to prove that it was well-behaved to get into the SRE program in the first place from the Google side.

And it's a big deal inside of Google when you do, you talk to the software engineers there. When, when, when you get attention from the SREs, it's a, it's kind of a milestone. You can, you can actually get kicked out if you're, if you're causing, if you're making their life hard. Right. So it's sort of a, it's, it's a true partnership between the developers and operations, but it's also managed by the, these numerical contracts. It's like, if you, if you make my toil go higher than this, like, then you're here with me. Like, I'm not, I'm not doing this for you. We're doing it for together. Yeah. I mean, I guess it's a, it's an indication of, uh, the, the, the quality of your software, if there's a lot of toil to have to deal with it. And, and it's sort of like the, uh, uh, one of the things we talked about early on, like, like kind of the error budget. Like if there's, if there's, obviously you need to go back and make your software better, right? Like it doesn't, it doesn't fit the way that we want to operate. And I assume sort of profitably are operating. So it forces the, in, in that kind of cool headed way that numbers can do, uh, it, it forces you to go back and work on quality versus other things. I, I just liked that. It makes it really explicit to all parties that the goals in a numerical way, a measurable way. So, so you don't have the, in some ways it kind of eliminates some of the finger pointing and, and, you know, that's another chapter. They, they definitely talk about blameless cultures and trying to fix and do these things. But by removing some, I think you remove some of the possibility for that, but getting to this like really strict numerical value for, for how you're going to think about that. Yeah. Yeah. I mean, in a very, uh, in a very small wormhole between the two concepts kind of way, I just finished reading that. Um, did I finish reading it? I think I did that book by, uh, Mark Schwartz, you know, who's the CIO of, of one of the, the, uh, uh, DHS groups, uh, called the art of business value and art of business value.

Yeah. I just saw Mark. Yeah. You know, you know, usually when you ask, let's, let's call them a, uh, I don't know what to call these people anymore. Like the sort of DevOps, agile, cognoscenty, you know, all the people who are sort of like trying to the intelligentsia of that. And you ask them anything associated with metrics and proving business value. They pull a Obi-Wan Kenobi on you and you're like, Whoa, I don't even realize what happened. Suddenly I don't care about that. Like they never really directly answer the question. And to be honest, he doesn't really give a super direct answer to it, but he has one of the best. These are not the droids you're looking for answers in that book. And he, um, the best, not the droids you're looking for. That's right. Like he basically goes over. So finding business value is really hard. Right. And if you're just focused on ROI and all these other things, like the problem is to try to summarize what his point. Business value is unique to every single person and group and company that asked this question. I mean, more or less, it's unique. There's some common patterns. Like if you're a retailer, you probably would like to increase sales, right? Like, so there are some things to do that, but what the business finds valuable in terms of it can be very difficult to figure out. And he actually offers towards the end of the book, some, some, uh, systems of thinking. So you can find out what your business value is. But one of the interesting points he makes, and you can kind of see his government background peeking out through this is like, um, in the literature, uh, business value is often associated with what a user wants and the end business, like transacting money between a business and whoever their customer is. But it turns out that business value can be a whole lot of internal things as well. So like I, one of the, my favorite part of the book is, is in, in the government, at least in the U S government, I don't know how it is elsewhere. We value transparency and agencies doing their work almost above everything else because we just really don't want corruption and all sorts of things. So transparency is a prime deliverable thing and therefore something that has high business

value that you wouldn't necessarily think about. And so that's why we tend to have almost too much compliance and auditing and things like that. Cause transparency is extremely important to the citizens and the legislature, all this stuff. So, and, and so the way this, that thin wormhole is, is that clearly in, in Google SRE land, one of the business values they have is that we shouldn't have to muck with all this toil driven operation stuff so much. Like that's codified as something that you need to deliver on that you may not traditionally think as a type of business value. And so the point that you keep coming back to is, is they've figured out a way to institutionalize it, to wrap a bureaucracy and a governance around it. That's driven by numbers. And that has very, uh, very apparent and, and apparently enforced rules. Um, again, but then all the way connect back to the business value that we want is a side effect. Almost is like, we don't want to have to spend a lot of time operating this. Cause if we're doing that, it probably indicates there's not good quality to this software, which even goes back to the business value of, you know, making the end users happy and running the business well. But it's, it's, it's a good example of tracing that confusing business value thing down to actual, like probably numbers and rules somewhere. I totally agree. And I think very few people, and I'll include myself in this statement, really comprehend the unfathomable scale that Google operates at. And, and at that scale, the, the smallest inefficiency is literally millions of dollars. Right. And so they, they've had a, um, kind of philosophical methodological approach to driving those inefficiencies out of the system, uh, since the inception. Yeah. So there's another thing that we can kind of close with that I think is particularly interesting from a, from a pivotal narrative and also plays into the kind of the CRE partnership

that we're doing with Pivotal Cloud Foundry. There, there's a bunch of chapters that, that talk about all this other stuff to build up to this, but there's kind of this, this closing where it's framing the, you know, sort of the end of the book, but it's framing everything essentially in this language around platforms. Right. And, and I posted some of these on, on, uh, Twitter and maybe you want to link to them, but there's these quotes from the book where it says things like in order to ensure that the SREs, you know, don't, don't get the, the snowflake of the month to show up and then kind of start from, from first principles, trying to understand what's going on with it every time that, that they built these, these infrastructure and architectural components that the rest of the Google software engineers, uh, can take advantage of so that when, when they, when they show up built from these sort of platform components, then it's kind of pre audited. It's kind of pre, you know, pre, um, not, not approved is not the right word, but they, they, they, they built them. The SREs built many of these components and, and that's another quote, or that's another observation I had is that the, the SREs are, are largely architects as much as they are operations. And so when you take that framing around the platforms and the quotes from the book, they're essentially arguing that they built something that's, you know, Google specific, obviously like, like a, like a pivotal cloud foundry where you're, you're basically giving the, the software developers, the infrastructure and application components that make doing the right thing with respect to right reliability, simple and easy. Yeah. I mean, I mean, that gets, that gets back to those kind of two buckets I was talking about earlier. And, uh, I think, I think recently when I've been talking with, with organizations and enterprise about, you know, cloud native and pivotal cloud foundry, one of the things that falls in that first bucket, things you thought were happening is like, well, if we have a standard way of doing

all of this, things will run a lot more smoothly. Right. And, and, and, and in the, in the, what you were just talking about, I mean, the standard way is here's a bunch of tools that the people who operate it and architectures and ways of right running your software that we've written is that therefore we understand. And therefore things will go a lot more smoothly if you just use those. And I mean, there's, there's, there are a series of reasons that are more legit and understandable than others and enterprises, particularly like M and a and, uh, things where you've got, uh, two companies colliding and their architectures, uh, like, of course they're going to be different, but you know, it's projects. They're so fun. Oh yeah. Yeah. But, but, you know, it definitely seems like, I mean, this is almost, it's one of those things where like, when we said you should have a standardized platform that everyone in your organization uses, we actually meant that it's, it's my old, uh, you should actually floss and eat more fruits and vegetable things. And I think, I don't know. I mean, there probably is some interesting subjective studies of this, but it almost seems like if you're not paying attention to it very closely, there's a natural rate of decay where you just lose track of all of that. And it's good every, every decade or so to probably go back to that basic principle and try to introduce something. And so in the point of like, yeah, in the point of not exactly running like Google, but taking ideas for them, it's like, it turns out this principle is extremely handy for all sorts of reasons that if you standardize on something, you'll, uh, you'll win a lot better. So the, the idea you just expressed is, is spot on. The thing that I think has changed in, and, and let's be like, if you go through the history of Cloud Foundry, it was Google inspired from its inception. What's changed in the, in the new world is that these standards are actually enforced by the software itself. Right. I think in many organizations that, that we work with, they've, they've adopted standards, standards, but those standards are more about checklists and process and signatures and,

and they're very, very few are, are enforced. The policies are not enforced by the software itself. And I think we're moving to a software defined platform enforcing itself with respect to all these things. And there's a, there's another, we're not, we don't have time to go through, um, some of this, but it's very interesting, um, developments around how these platforms make secure security easier and security more, more feasible. Yeah. It's all, it's almost like, uh, software defined governance. Right. And I was, I was rewatching, um, one of our coworkers, Cornelia Davis, and she gives a good talk about, uh, uh, uh, how, how a traditional siloed organization, um, the, the way that it transforms and what it looks like when you're operating and, uh, whatever you want to call it, DevOps or cloud native way. And you have your, your product and set of project teams and all that kind of great stuff. But there's, there's one instant moment that I, I, uh, I, I caught this time listening to it where, where she essentially goes over this point that you're saying, and like, whether it's security or compliance or governance, if something's like so important that you have to do it, you should just have the platform do it for you. And it kind of, you know, you should, you should basically to use this term kind of inaccurately, you should automate it or make sure it's enforced by the actual technology instead of the people. And it kind of goes back to one of the, the, the, the themes running through a lot of what we've been talking about is, um, uh, I don't know how to phrase this right. But if we wanted, if we have something that we want to do and enforce, we need to make sure to write some software that forces us to do that metaphorically in the same way that a compiler will tell you like, no, that doesn't work and just won't compile your code. You have these stop gaps for all this governance that used to be, uh, more manual and kind of done on a smile, a handshake and a PowerPoint. And instead you have the system doing it all for you. I say this having spent many years of my life as a software developer that if you don't make doing the right thing, the easy thing, then many developers being forced to choose between

doing something right and doing something right now will not do the right thing. Absolutely. That's almost the credo of the software developer is do the fastest thing to be done. Or I think there's a, there's a footnote on that. The names have been changed to protect the guilty. Yeah. I think, I think there's a footnote that if you're in your twenties, do the slowest thing that maximizes my enjoyment, like building your own platforms and everything and frameworks. So there's that, those are the competing desires of, I think, software developers, which, uh, we could make a chart how that changes over maybe not age, but experience, uh, that you have. Here is a, is a great hobby to have in your time. Yeah. Well, great. Well, before we wrap up, is there anything else you want to, uh, make sure to call out here? I mean, we covered a lot of ground. I think that there's, uh, more to this story. You know, obviously we're, we're working through this stuff. We're kind of in the middle of it with Google, but I'm excited to see the, the, the language around service level objectives being, being created and published for, for cloud foundry. And, and then seeing that mentality and mindset permeate all through the rest of the, uh, the pivotal practice. Yeah. Well, great. Well, uh, thanks for being, if people, if I'll try to find links to the, uh, I think they put repost posted the recordings of all their talks and I'll, I'll put in the show notes, uh, a lot of the stuff we talked about, you can always get the show notes at pivotal.io slash podcast. But if people want to, uh, check in on you more, what's the, what's the best place to go on the internet? The easiest way to get my attention is little idea on Twitter and I have open DM. So if you want to tell me something, um, I'd probably respond. Uh, that, that's the, that's the thing that I have, you know, some different talks coming up. I'm doing, uh, DevOps days in Denver. There's the spring days that Pivotal is planning. There's probably a couple of trips to, you know, Cloud boundary summits a little ways out, but that should be a, some people to think about if you're so inclined. Oh yeah.

I, I forget, I forget to put that little ad in here. We do have spring days. I mean, that's over, uh, the next like three or four months, right? And I don't know, 10 cities or so. Yeah, we're, we're starting to do some grassroots, you know, organic spring. I mean, it's not, it's not, it's like a different topic than CRE, but it's. It's the Pivotal, uh, Java application framework and there's a ton of community and adoption. And so we're just trying to kind of give back and invest in, in having those community events where they don't have to go through the whole year and they don't have to travel so much. So we're, we're targeting, um, a couple of cities and then we also have a form where people can request to, to organize spring days locally and Pivotal is involved and we're kind of trying to provide a, a, a logistic support, but it's really locally organized. That's nice. And yeah, I mean, it's very similar to some of these other local conferences, but the, the spring communities exploding the interest in, you know, spring boot and the, the microservices and spring cloud and being able to do the integrations and, you know, service discovery and the rest of it. But, and then the new reactive stuff, it's, it's really interesting to see that kind of re, re, uh, I don't know, reinvigoration of Java. Like it's a, it's a renaissance of, of, um, uh, we'll, we'll, we'll say like there's this dark ages of Java. We won't name any technologies, but I think we're, we're through that and we're, we're, we're in a, a true renaissance with the kind of cloud native architectural patterns that are happening. Yeah. Well, as I was trying to write up in this column I had last month at the register, like the great thing about Java and other languages have this is that it's, it, uh, I don't know how it happens, but to anthropomorphize it, it allows itself to constantly evolve and shift around. It doesn't kind of get stuck in one way of doing things. So while there, while there may be like, uh, bummers that happened here and there for several years, like eventually Java's pretty trustworthy as an overall community of like catching

up and evolving and, uh, if not leading on, on various things. So it's, it's a very, uh, very thriving sort of ecosystem. But, uh, also before, before I wrap up, uh, we also have the, uh, not, not we, but the entire cloud foundry, uh, community has the cloud foundry summit coming up on June 13th and 15th. I think, you know, there's a lot of like, uh, low level nerd stuff where they show off technical things and you can do that. But what I really like about it, uh, is, is that there's a lot of actual users and customers of all the various cloud foundries who come and they not only tell you how awesome they are and how things are working, but many of those talks go over things they've tried out that didn't work well. And, uh, you know, the, the, the failures and the successes as they've tried to switch over their more or less large organization. That's usually what they are to, uh, doing things in, uh, in, in a cloud native kind of way. And if you want to get 20% off, I guess it's cloud foundry, Silicon Valley. So they have this weird code that CFSV 17 Coté. That's my name, C O T E. And, and you can get, uh, you can get a fifth of your registration fee off. I'll probably end up going there. I haven't finalized my travel, but it's, it's, it's a good conference. So, uh, you should check that out if you're interested in all of this. It'd be a cloud native good time. Exactly. And you know, it's that high. It's fun that it's attached to over there in Santa Clara and you're next to the Levi stadium. Go, uh, I don't think they sell jeans there. They should really look into that. So as always, this has been Pivotal Conversations. If you want to get the, uh, the most up to date episodes, when they come out, what you should do is go over to a soundcloud.com slash Pivotal Conversations. You can find the RSS feed there, or you can listen to it in soundcloud. If you want, they have delightful visuals. Uh, but you just put that in your, uh, your iTunes or your overcast or whatever podcast listening thing you have, and it'll, it'll download them as soon as I upload them. Uh, and we, you can also find it in iTunes and things like that. And we have, uh, if you search around for pivotal in iTunes, we have a two or three other podcasts, particularly one on, uh, the world of data and stuff that that's very interesting

and, uh, frequently comes out with good stuff. Uh, and also while you're in there, if you want to leave a review or leave some little stars or whatever kind of coins or resources you want to give us, uh, it's always nice to see that in there and it helps other people, uh, find and discover this content. And, uh, I'll put the show notes for this and many other stuff. As I mentioned earlier, many other episodes at pivotal.io slash podcast. And with that, we'll see everyone next time. Bye-bye.

Tanzu Talk · 🍕 Get more pizza.