1 00:00:00,000 --> 00:00:02,880 Well, we have a little special episode here. 2 00:00:03,120 --> 00:00:04,120 No Richard about. 3 00:00:04,380 --> 00:00:05,280 I don't know where he went to. 4 00:00:05,360 --> 00:00:08,780 Maybe he tried to find some umbrellas or whatever up there in the Pacific Northwest. 5 00:00:09,360 --> 00:00:11,520 I have a much-returning guest on. 6 00:00:11,600 --> 00:00:12,580 Why don't you introduce yourself? 7 00:00:13,140 --> 00:00:16,880 My name is Andrew Clay Schaefer, and I've been here before. 8 00:00:18,100 --> 00:00:19,740 Yes, many times. 9 00:00:19,740 --> 00:00:20,860 It's a pivotal thing. 10 00:00:21,740 --> 00:00:28,240 So you had a blog post last week going over us being involved in Google's CRE program. 11 00:00:28,400 --> 00:00:35,260 And I think if I understood it correctly, we're basically – well, Pivotal Cloud Foundry is the first platform that's in that program. 12 00:00:35,960 --> 00:00:39,320 And I thought it would be good to have you on to go over what that is. 13 00:00:39,320 --> 00:00:56,440 And I think more importantly and for me more interestingly than just being in the program, there's something about a lot of opinions and time-tested advice about how you run and operate and think about how you run your platform and infrastructure that all of that implies. 14 00:00:56,960 --> 00:01:09,640 And it would be an interesting one just to hear, like, what this CRE stuff is and how it kind of – I don't know, the philosophy of operations and running a platform that's embedded in it and it embodies. 15 00:01:10,240 --> 00:01:17,580 So why don't you start off with just kind of covering the basics of what CRE is and what this program is and what it means for us to be involved in it? 16 00:01:18,240 --> 00:01:18,640 Absolutely. 17 00:01:18,840 --> 00:01:23,760 There's a few things we need to build up to before you can really talk about CRE. 18 00:01:23,980 --> 00:01:29,040 So the CRE was announced by Google towards the end of last year. 19 00:01:29,540 --> 00:01:34,100 And the program is an extension of the SRE. 20 00:01:34,420 --> 00:01:36,360 And there's a lot of acronyms involved already. 21 00:01:36,480 --> 00:01:38,180 So let's start explaining this stuff. 22 00:01:38,260 --> 00:01:45,240 So there's the storied internal process and technology that runs Google. 23 00:01:46,120 --> 00:01:57,340 And there's a group of people, although you're not – you shouldn't call them system administrators, although a lot of them have those types of skills called the SREs, which is Site Reliability Engineering. 24 00:01:58,480 --> 00:02:03,360 And there's a book that came out, and it was recently made available to read for free. 25 00:02:03,780 --> 00:02:08,880 You can read it – if you just search for Google SRE, you can read it for free on web or you can buy it or whatever. 26 00:02:09,580 --> 00:02:25,000 And it kind of walks through in long form, and there's a lot of different contributors, a lot of different chapters, all these different aspects of SRE and how they think of themselves and how they relate to the business. 27 00:02:25,000 --> 00:02:30,460 And how they are different or the same as anyone else doing operations. 28 00:02:30,820 --> 00:02:38,160 And, of course, there's this parallel conversation or maybe even subsuming conversation around DevOps and the trends. 29 00:02:38,160 --> 00:03:03,680 And my personal career, especially if you've ever heard me or seen me before, is sort of taking these ideas from the big web and the communities like the DevOps days and the Velocity Conference and trying to bring them into the enterprise minds and bodies so that they can benefit from this kind of emergent, dominant paradigm. 30 00:03:03,680 --> 00:03:13,620 And when I talk about cloud native, I always talk about the DevOps processes and continuous delivery and microservice and the rest of that. 31 00:03:13,680 --> 00:03:14,560 And that's all related. 32 00:03:15,680 --> 00:03:24,140 So this SRE framing, which is now very, very explicit, it's basically Google walking through their mindset, their process. 33 00:03:24,620 --> 00:03:28,620 You know, there's obviously not every detail, but it's a 500-page book. 34 00:03:29,920 --> 00:03:46,020 And they'll explicitly, in fact, when I gave a talk at the Google Next last week, one of the SREs came up to me because I made this comparison of SRE to DevOps and sort of how those are the same and connect. 35 00:03:46,020 --> 00:03:53,380 And some of the SREs were thanking me for doing that, and I thought it was interesting to see their mindset. 36 00:03:53,380 --> 00:04:05,840 And in the book, it calls out the word DevOps a little bit, but they also, I think there's certain people at Google that, and probably lots of places, that like to think that they invented everything. 37 00:04:06,660 --> 00:04:13,820 So watching that evolve and having that be really explicit in public is exciting in and of itself. 38 00:04:14,160 --> 00:04:34,300 And then the CRE program is taking the process by which the SREs accept applications onto the Google infrastructure and manage it and operate it, and extending that to the Google Cloud platform customers. 39 00:04:36,300 --> 00:04:44,280 So they're, and I think it's worth walking through some of the stuff Pibble's done to kind of frame that. 40 00:04:44,480 --> 00:04:46,820 And it's not easy. 41 00:04:47,000 --> 00:04:47,660 It's not a checklist. 42 00:04:48,280 --> 00:04:50,000 You know, you don't just get a rubber stamp. 43 00:04:50,420 --> 00:04:55,700 And we're in the process of going through and doing the work with Google. 44 00:04:55,800 --> 00:05:02,840 But you get a review from the Google CRE team. 45 00:05:03,120 --> 00:05:25,260 And they're basically looking for the things that would potentially contribute or be detrimental to reliability and making the recommendations that they would make knowing what they know about distributed systems and the Google infrastructure to help you meet your service level objectives. 46 00:05:25,260 --> 00:05:28,600 So that's, to me, the most interesting thing. 47 00:05:28,600 --> 00:05:38,820 And this is where I think that a lot of the DevOps community would benefit from this is that the SRE process is highly quantitized. 48 00:05:40,600 --> 00:05:47,320 And in particular, I think that there's a couple great lessons about monitoring and alerting and all that stuff. 49 00:05:47,540 --> 00:05:51,580 But some of that is shared by, you know, a lot of the larger DevOps community. 50 00:05:51,580 --> 00:06:11,360 But there's this air budget idea that's somewhat unique to Google or at least initiated at Google, which is instead of having these emotional disagreements about when and why and how someone should be able to deploy, 51 00:06:12,360 --> 00:06:23,260 So what Google does is between the software engineers and the site reliability engineers is work through what the business needs from a service level objective. 52 00:06:24,360 --> 00:06:30,640 So there's sort of this, you know, three-legged stool that figures out what this service needs to be. 53 00:06:30,740 --> 00:06:36,500 And they want to be realistic because every nine costs ten times more than the last nine. 54 00:06:36,880 --> 00:06:40,620 So you set some service level objective for your application. 55 00:06:41,440 --> 00:06:44,500 And then as long as you're within the air budget. 56 00:06:44,620 --> 00:06:53,660 So they have a bunch of monitoring that is giving them, you know, all the stuff and more about their applications. 57 00:06:54,100 --> 00:06:57,360 And one of those things is the errors and the availability. 58 00:06:57,360 --> 00:07:01,800 And if you exceed your error budget, you can't deploy anymore. 59 00:07:02,940 --> 00:07:05,760 And it's not really, there's no argument. 60 00:07:05,960 --> 00:07:06,640 It's not emotional. 61 00:07:06,920 --> 00:07:09,280 It's like, well, you're not meeting your service level objective. 62 00:07:09,500 --> 00:07:19,520 So instead of adding new features to this service, what we all need to do is we need to push the reliability higher. 63 00:07:19,680 --> 00:07:22,100 So we're going to do that work instead of new features. 64 00:07:22,100 --> 00:07:25,280 And when you're meeting your error budget, you could deploy at will. 65 00:07:27,360 --> 00:07:34,180 And I think that transforms a lot of the sort of squishy, oh, we want to continuously deliver. 66 00:07:34,380 --> 00:07:41,640 But, you know, we're trying to balance the life of the developer, productivity developer against the life of the operator. 67 00:07:42,580 --> 00:07:48,900 And gives you this really nice, elegant solution that is ambiguous. 68 00:07:49,100 --> 00:07:50,660 So that's one of the things I'm excited about. 69 00:07:51,100 --> 00:08:08,540 Yeah, that is interesting because it's a way of, I mean, like you're saying, it marries together one of the concepts in DevOps, which is if we're going to be running this stuff in production and expect it to work, we should probably have the programmers write things in a way so that it works in production. 70 00:08:08,540 --> 00:08:25,720 And scaling up and then also to that point is like, and no one really, well, on the business side, people often don't like working on spending their development time on features that don't have like a direct user facing thing. 71 00:08:25,920 --> 00:08:32,520 So you've got to figure out a way to force them how to, I don't know, they even have fanciful terms like functional requirements and things like that. 72 00:08:32,520 --> 00:08:46,700 But it's interesting to institutionalize a, I don't know what you would call it, like a trigger that's basically like, you need to go and work on fixing this stuff so that works in production and versus like introducing even more features and a stopgap for that. 73 00:08:47,420 --> 00:08:47,720 Absolutely. 74 00:08:47,860 --> 00:09:01,520 There's a few other interesting things about their version of DevOps, which is in the, so the Amazon example, which is very famous, if you build it, you run it. 75 00:09:01,520 --> 00:09:07,720 So it's like you have these two pizza teams on a service and they're kind of responsible for the full life cycle of that service. 76 00:09:08,640 --> 00:09:19,140 At Google, they took a slightly different version, which is you basically run it until you prove that it won't ruin the carpet. 77 00:09:20,820 --> 00:09:23,180 And then the SREs will run it for you. 78 00:09:24,120 --> 00:09:27,420 And if you exceed your air budget, you're probably going to get woken up. 79 00:09:27,520 --> 00:09:35,660 But for the most part, if you build to these reliability standards, then the operational overhead is negligible. 80 00:09:36,220 --> 00:09:36,320 Right. 81 00:09:36,360 --> 00:09:44,340 So there's, there's this partnership between the software engineers and the SREs where the, where the SREs sort of adopt the operations. 82 00:09:44,340 --> 00:09:53,420 And that's, that's the process that the, that the CRE program is making, is making publicly available to the, to the Google Cloud customers. 83 00:09:53,420 --> 00:09:53,860 Oh, I see. 84 00:09:53,920 --> 00:10:03,400 That was going to be my next question is like, I still don't quite understand the difference, but tell me if this, if this, so, so the SRE books is, is a little more like internal facing. 85 00:10:03,400 --> 00:10:08,200 Like how we operate to run things and the CRE, what's that? 86 00:10:08,660 --> 00:10:09,820 It's how Google runs Google. 87 00:10:09,960 --> 00:10:10,360 Right, right. 88 00:10:10,440 --> 00:10:21,580 And whereas the CRE one is like, if, if you want an SRE to run your stuff, to use some, some language that you like to use every now and then, here's sort of like the contract of stuff you have to do. 89 00:10:21,580 --> 00:10:28,760 Like here's, here's the way you need to make your application work and, and the way that it has to operate so that you get to that point. 90 00:10:28,760 --> 00:10:37,380 Like you're saying that, that the, uh, there's, there's not so much, there's not a whole bunch of management things that you have to do that are unexpected or out of the ordinary. 91 00:10:37,640 --> 00:10:44,900 So, so the aspiration for the Google CRE program is that they're going to run your applications for you. 92 00:10:44,900 --> 00:10:46,940 Like, like the SREs would, right? 93 00:10:46,940 --> 00:10:48,280 That's what the blog post talks about. 94 00:10:48,500 --> 00:10:50,420 They're going to be on smortems with you. 95 00:10:50,420 --> 00:11:01,400 And, and in order to do that, what they're, what they're asking you to do is meet these, these service level objectives with them, go through and review your code. 96 00:11:01,400 --> 00:11:13,720 If they want to, if they want to be the ones that are going to be, you know, jointly responsible for the proper care and feeding of that application, they, they want to have some understanding and ownership of what's there. 97 00:11:13,720 --> 00:11:25,640 And it goes right back into the, the contracts and promises line of, of, of reasoning that I've, you know, and I'm borrowing from Mark Burgess and like some of these other people are inspired by them. 98 00:11:26,020 --> 00:11:44,760 Um, and, and I just think, I just think it's really a gift to see the way that they're framing this stuff and be explicitly in conversations with them about why they think this, you know, will fail this way or, or, or you can expect these kinds of service level objectives with this stuff. 99 00:11:45,520 --> 00:12:08,820 Um, and then there's also this implicit, and this connects back to the other thing that I've been talking about for the last year, which is a transition in my mindset that happened a while ago from automate all the things to what you automate has as big of impact on your, on your operational overhead and cost as the fact that you automate. 100 00:12:09,180 --> 00:12:26,700 And so they're, they're saying if you, if you automate these things this way, if you have these types of architectural characteristics, then with these kinds of infrastructures and these kinds of architectures that you're operating cost is effectively negligible. 101 00:12:26,880 --> 00:12:28,260 And that's pretty exciting. 102 00:12:29,200 --> 00:12:29,340 Right. 103 00:12:29,720 --> 00:12:39,080 And, and so, and so like, I mean, like, what are some examples of, of what the, the, the contract is or the things you have to do or things you, you, you shouldn't do? 104 00:12:39,080 --> 00:12:40,260 Like, I don't know. 105 00:12:40,280 --> 00:12:43,960 I mean, the, the easiest thing as an example, I don't know if it applies here at all. 106 00:12:43,960 --> 00:12:53,140 It was like, you know, you got your, your 12 factor example and, and the one on the tip of everyone's tongue is like, don't use local storage, which, which is all simple and makes sense. 107 00:12:53,140 --> 00:13:00,920 But like, what are, what are some of the things and ways you have to like architect and, and write your applications and services that they, that they eludicate? 108 00:13:02,840 --> 00:13:10,220 So there's, there's a, a, a bunch of things that are kind of, you know, straight off of some of the things that the platform's trying to provide. 109 00:13:10,220 --> 00:13:17,840 And that's why it gets really interesting when you start thinking about what the service level objective should be for something like Pivotal Cloud Foundry on, on Google Cloud Platform. 110 00:13:18,320 --> 00:13:22,300 So there's, there's a, a bit about telemetry, right? 111 00:13:22,300 --> 00:13:31,360 So the, if you read the Google SRE book, they have this pyramid of reliability and the bottom part of the, of the pyramid is monitoring. 112 00:13:32,360 --> 00:13:44,000 And I mean, it's really easy argument to make that if you're not monitoring, if you don't have a baseline, you don't understand what's happening in your systems, then, then how do you know if you're doing anything good or bad? 113 00:13:44,660 --> 00:13:47,380 And, and, and that's the first thing that they, they might fix. 114 00:13:47,460 --> 00:13:55,000 So there's a bunch of baseline around being able to get logs in and out, being able to do, you know, that kind of telemetry. 115 00:13:55,620 --> 00:14:03,900 And then there's, there's just sort of a methodical taking apart of the, of the services. 116 00:14:04,420 --> 00:14:05,980 How are they set up? 117 00:14:06,340 --> 00:14:07,620 How will they fail? 118 00:14:07,940 --> 00:14:13,220 If, if this is redundant, you know, what's the, what's the contract that needs to be made about consistency? 119 00:14:13,640 --> 00:14:17,000 What's the contract that needs to be made about, you know, these different pieces. 120 00:14:17,000 --> 00:14:28,040 So it's kind of a, all, all the things that you imagined architects kind of did when they're designing systems, except for Google's doing it with you. 121 00:14:28,280 --> 00:14:28,640 Right. 122 00:14:29,640 --> 00:14:29,960 Yeah. 123 00:14:30,040 --> 00:14:46,500 What, I mean, to that point, uh, without getting too grim, I mean, if, if, if you were, I don't know if this is the right framing, but if you were to separate out at least two buckets, one of them is kind of what you were describing is like, uh, here's all of the stuff that you thought was happening, but it turns out people weren't actually doing it. 124 00:14:46,500 --> 00:14:48,960 Or technologically it didn't work out very well. 125 00:14:49,560 --> 00:14:51,940 And, and then, so there's that bucket of stuff, but then there's another bucket. 126 00:14:52,020 --> 00:15:01,080 Like, are there any sort of like new or novel things that come into play that, that like, I don't know, in the early 2000s, some enterprise architect wouldn't have to worry about? 127 00:15:01,080 --> 00:15:04,800 Like a way of thinking about how they write their software and architect it? 128 00:15:06,320 --> 00:15:08,600 In some ways, I, I don't know. 129 00:15:08,740 --> 00:15:12,400 I wouldn't say there's anything that's especially novel to me. 130 00:15:12,980 --> 00:15:23,100 Um, at least with the exposure I've had to the people that built the big web over the last 10 years and the conversations that happen at a velocity conference or DevOps days. 131 00:15:23,520 --> 00:15:39,780 But, but, but the, the thing that Google brings to the table is this, is this methodological or, um, methodical kind of disintegration and then, and then re reintegration of everything. 132 00:15:39,780 --> 00:15:40,140 Right. 133 00:15:40,140 --> 00:15:43,600 So, so like they, they want to, and they're, and they're quite adept. 134 00:15:43,680 --> 00:15:53,500 I mean, that was one of the things when you, when you see the talks that Onsi gave, we should get recorded, um, at some point soon, uh, about the process with the, with the, 135 00:15:53,520 --> 00:16:12,740 the cloud foundry engineering working with, with, um, the Google so that the assessment itself was, was impressive just in, in terms of the speed and the depth that, that the, the Google side took apart cloud foundry and, and had a bunch of meaningful rep, um, recommendations. 136 00:16:15,780 --> 00:16:16,260 Right. 137 00:16:16,700 --> 00:16:18,740 It's not, it's not like a superficial thing. 138 00:16:18,800 --> 00:16:22,240 It's like, oh yeah, because they're, they're the ones in some sense. 139 00:16:23,100 --> 00:16:32,940 The, the, the long, the long-term goal is that, that Google is going to be at least partially responsible for operating your, your, um, applications with you. 140 00:16:33,300 --> 00:16:34,860 And they, they don't want to do that. 141 00:16:34,960 --> 00:16:37,660 They don't want to take that lightly because it's, it's them right there. 142 00:16:37,660 --> 00:16:39,300 It's their, they're the one getting woken up. 143 00:16:39,640 --> 00:16:39,760 Yeah. 144 00:16:39,960 --> 00:16:40,160 Yeah. 145 00:16:40,220 --> 00:16:52,680 And that, that's another thing that like, I think, I mean, it's, it sort of comes through there, but basically, and, and as you were alluding to, some people don't like using existing terms, but they're to some extent offering some managed service stuff in, in this, right? 146 00:16:52,680 --> 00:17:01,260 Like you're saying, you can, you'll actually be working with people to help you run your stuff, at least at the, the, the defined level that they'll help you out at, at. 147 00:17:03,120 --> 00:17:07,700 I, I would, I would, I would be reluctant to call it managed services, right? 148 00:17:07,720 --> 00:17:16,100 In a sense, it's, um, like, I, I kind of feel like managed services implies this thing that you, you've outsourced and kind of handed off. 149 00:17:16,180 --> 00:17:16,420 Right. 150 00:17:16,900 --> 00:17:18,500 And then you don't have to worry about it. 151 00:17:19,340 --> 00:17:30,020 I think what Google's offering is, is we're going to worry about this together and, and we're going to, we're going to think about this together and we're going to be involved in, in the postmortem. 152 00:17:30,020 --> 00:17:36,720 And we're going to be, so, so it's not quite managed services and it's, and it's offered essentially for free, right? 153 00:17:36,720 --> 00:17:49,680 So it's not like you're paying this thing, but Google's, Google's got a lot of experience running a lot of infrastructure and applications, you know, at scale with a low, um, operating cost. 154 00:17:50,100 --> 00:17:52,240 And, and this is why I think it's really interesting. 155 00:17:52,240 --> 00:18:16,980 I think it's exciting to see this method, methodological, uh, approach to building infrastructure, architecture and applications being brought into Pivotal because the, the next phase of this partnership and goal is to be able to have that, that kind of rigor be part of the labs process as well. 156 00:18:17,220 --> 00:18:17,740 Right. 157 00:18:18,900 --> 00:18:28,080 But I would, I would be, I would be hard pressed to call it managed services and I would, you know, if Google wanted to call it that at some point, maybe, but to me, it's much more of a, of a partnership. 158 00:18:28,620 --> 00:18:35,700 You're meeting, um, kind of at the minds and, and at the, at the level of, uh, responsibility together. 159 00:18:35,700 --> 00:18:37,360 You're not, it's not handoff. 160 00:18:37,700 --> 00:18:38,620 Right, right, right, right. 161 00:18:38,960 --> 00:18:40,480 But the point being. 162 00:18:40,820 --> 00:18:42,160 Actually, that's an interesting point. 163 00:18:42,300 --> 00:18:43,540 Let me digress one sec. 164 00:18:43,540 --> 00:18:57,260 So the, I would say managed services is, is in some ways the way that, that most people have implemented that and the connotation that has to me is a manifestation of the old model of IT ops. 165 00:18:57,260 --> 00:19:01,440 Like the pre DevOps model, you just outsourced it, right? 166 00:19:01,480 --> 00:19:11,380 So you outsource your old style where, what SRE and CRE are bringing is this new collaboration between engineering and operations. 167 00:19:11,860 --> 00:19:14,940 And Google is willing to be, you know, part of that equation with you. 168 00:19:15,500 --> 00:19:15,780 Yeah, yeah. 169 00:19:15,860 --> 00:19:25,960 Yeah, no, I, I mean, it's sort of a pain metaphor, but, but classic managed servicing services is a lot more like outsourcing in the sense that there's a, and now the ITIL stuff is kind of built around this. 170 00:19:26,220 --> 00:19:32,380 It's always a, a fraught thing to bring up, but, uh, it's sort of like a black box of a service that you get. 171 00:19:32,480 --> 00:19:41,040 Whereas what you're describing is a lot more, I don't even know if it's a box, but it's a lot more transparent in a white box where you're, you're, you're getting, uh, you're getting people to help. 172 00:19:41,040 --> 00:19:48,420 As the customer, you're getting people to help you run the application, not totally run it for you and just provide a service or whatever. 173 00:19:48,420 --> 00:19:52,920 The key to keep in mind here, and I'm going to keep going back to it. 174 00:19:53,120 --> 00:20:07,620 And I've been trying to get people to see this in all the DevOps conversations I had is it's, it's actually that front loaded step that is pushing the operational overhead down, down, down. 175 00:20:07,900 --> 00:20:10,140 The reliability is going up, up, up. 176 00:20:10,220 --> 00:20:14,040 So now the cost of running that is so low. 177 00:20:14,680 --> 00:20:24,940 If you have, if you have all these kind of architectural practices, all this telemetry, then yeah, there's some fault detection, fault remediation that happens. 178 00:20:25,640 --> 00:20:30,360 But, but, but your break fix cycles are short and, and, and infrequent. 179 00:20:30,520 --> 00:20:34,300 And, and so it's not so much, oh yeah, they're going to run this with you. 180 00:20:34,300 --> 00:20:38,240 It's like, we're going to push the cost of running this down to, to nothing. 181 00:20:39,440 --> 00:20:39,840 Right. 182 00:20:40,300 --> 00:20:47,940 And, and, and so do you, do you think, do you think it's fair to say, I mean, you, you still hear this all the time that people, uh, people want to like run like Google. 183 00:20:48,180 --> 00:20:52,140 And do you, do you think this is like a way to, uh, to like do that essentially? 184 00:20:54,640 --> 00:20:56,660 Oh, that's a, that's a, that's such a loaded term. 185 00:20:58,100 --> 00:20:59,020 Well, exactly. 186 00:20:59,100 --> 00:21:02,160 Hints it being a, a, a good conversation topic. 187 00:21:03,920 --> 00:21:07,040 There are things about Google that are uniquely Google. 188 00:21:07,280 --> 00:21:07,520 Yeah. 189 00:21:07,600 --> 00:21:07,840 Right. 190 00:21:08,140 --> 00:21:19,580 And, and, and some ways that was, that was a unique opportunity that Google has had to solve the problems that only Google has and, and couldn't be Google if they didn't solve them. 191 00:21:20,200 --> 00:21:23,340 So do you need to run like Google? 192 00:21:23,600 --> 00:21:26,400 Can you conceivably run like Google? 193 00:21:26,880 --> 00:21:28,600 I'm, I'm not sure you can. 194 00:21:28,980 --> 00:21:40,160 At the same time, there is a lot of information that's available to you that will drastically improve how you do these things that Google's made available. 195 00:21:41,920 --> 00:21:49,020 Just like you're saying, you can be inspired by how they run, but it may not be, uh, it may not be perfect to get to the point where you run like Google. 196 00:21:49,080 --> 00:21:50,920 I mean, that's, that's the implicit question. 197 00:21:50,920 --> 00:21:57,100 And that is like, is it actually a good idea to run like Google or run like Amazon or run like any of these, these hyperscale companies? 198 00:21:57,800 --> 00:22:00,640 So let's, let me frame it to two different ways. 199 00:22:00,760 --> 00:22:15,740 So there's a bunch of the, the SRE book, which is explicitly calling out to these Google backed services that do things like distributed locks and, and, you know, you're saying, uh, storage, right? 200 00:22:15,760 --> 00:22:18,400 Like that's not a easy problem to solve at scale. 201 00:22:18,940 --> 00:22:25,040 Distributed storage is, is going to be the bane of all the, the cloud builders until they give up. 202 00:22:25,040 --> 00:22:32,200 But the, the, the, those services are not available to you unless you're inside Google, right? 203 00:22:32,200 --> 00:22:40,260 So some of the stuff that Google does because they're Google, you're not going to be able to do unless you're Google, right? 204 00:22:42,340 --> 00:22:54,540 Then, then the other side of this is all, all these, all these big web companies have operational excellence or they wouldn't be, they wouldn't be able to do this stuff, right? 205 00:22:54,540 --> 00:23:02,180 So I think Google has probably the farthest developed technology. 206 00:23:02,180 --> 00:23:04,820 They, they didn't necessarily productize it. 207 00:23:04,900 --> 00:23:13,840 They didn't necessarily understand how to bring it to market like Amazon did, but they, they probably have, at least in my opinion, some of the, some of the best technology. 208 00:23:15,200 --> 00:23:23,220 But those, those things are, are, are beneficial to you to see, to think, and you're going to see more and more. 209 00:23:23,220 --> 00:23:34,540 And we're already seeing this in the, in the pivotal framing of these enterprises that want to build what I'll call web scale applications, right? 210 00:23:34,540 --> 00:23:42,180 Especially when you start doing the math for what it's going to take to do some of these IoT projects, right? 211 00:23:42,180 --> 00:23:51,120 So when you start to have a lot of data that you need to ingest and analyze, then, then you start to look a little bit like, like Google probably did in the year 2000. 212 00:23:52,520 --> 00:24:01,080 And, and if you don't kind of think and act like they did, then, then you're probably not going to, it's probably not going to work very well. 213 00:24:01,140 --> 00:24:08,980 If you're, if you're going to, if you're trying to ITIL your way to, uh, web scale applications, you're, you're probably in trouble. 214 00:24:10,240 --> 00:24:11,220 No, that makes sense. 215 00:24:11,680 --> 00:24:27,720 It's sort of like there, there's a, you have to have a, uh, a Google class application to really need and get the benefits of running like Google rather than just adopting, you know, good practices and, uh, and, uh, operational advancements they've made into your existing things. 216 00:24:30,980 --> 00:24:31,460 Absolutely. 217 00:24:31,980 --> 00:24:32,220 Yeah. 218 00:24:33,620 --> 00:24:37,780 Well, what, what, what else, uh, I mean, what are some other like highlights from, from the book? 219 00:24:37,820 --> 00:24:39,040 I mean, it is 500 pages. 220 00:24:39,280 --> 00:24:40,280 I think you've read it, right? 221 00:24:41,100 --> 00:24:42,020 I did read it. 222 00:24:42,140 --> 00:24:42,520 So, so. 223 00:24:42,620 --> 00:24:46,080 I did read it and I published a bunch of, um, quotes from it on Twitter. 224 00:24:46,380 --> 00:24:46,600 Yeah. 225 00:24:46,700 --> 00:24:53,820 I, I think you had the suggestion that you should have some, uh, some open mic dramatic readings from the SRE book, which, which would probably be fun. 226 00:24:53,860 --> 00:24:56,740 Maybe with some of that in the year 2000 music behind it. 227 00:24:56,760 --> 00:25:01,720 Yeah, we should, we should make those videos, but like, like the, the idea of the air budget, right? 228 00:25:01,720 --> 00:25:12,400 Like that's an interesting concept that I think, uh, whether you want to run like Google or not could be adapted into the way you think about, uh, doing, doing your, your operations and your software. 229 00:25:12,400 --> 00:25:22,580 So there's a, there's a value to bringing that kind of dispassionate numerical version of the decision about what you can deploy and when. 230 00:25:23,000 --> 00:25:23,100 Yeah. 231 00:25:23,880 --> 00:25:27,100 The, the other thing, there's so many good things actually. 232 00:25:27,280 --> 00:25:30,100 Here, here's another thing that sticks out is this notion of toil. 233 00:25:30,880 --> 00:25:38,580 So the, the, the Google SRE teams are, are highly capable software engineers as well as system administrators. 234 00:25:38,580 --> 00:25:40,760 And, and they're very explicit about how they hire. 235 00:25:41,180 --> 00:25:47,780 So they, they, they put the SREs through the same interview that they put their software engineers through. 236 00:25:48,720 --> 00:26:00,160 And you have to be, and you know, the, you can go watch all the Google, um, YouTube videos about their, their, um, interviews and, and, you know, that's, uh, that's a whole nother topic. 237 00:26:00,160 --> 00:26:07,720 It could be another hour long, uh, podcast, but these, these interviews are, are extensive and they're intense. 238 00:26:07,720 --> 00:26:16,040 And, you know, people that pass them are, I don't know, they're, they, they kind of met a high bar to be part of that Google software effort. 239 00:26:16,460 --> 00:26:33,040 And there, there's this explicit thing that's like, well, if they're close as a software engineer, but they also have deep understanding of some other thing like, uh, systems or networks or storage, then, then they're, then they're eligible for SRE. 240 00:26:33,040 --> 00:26:44,060 So they, they, they, they maybe don't have quite the same standard for the software side, but they have this deep understanding of how computers actually work, which some software people don't always have. 241 00:26:45,020 --> 00:26:55,020 And, and the mix in their team is about 50, 50, these sort of system specialists and, and then the, the software engineers. 242 00:26:56,280 --> 00:27:07,440 And I've, I've, I've heard one of my favorite talks from DevOps days is Todd Underwood, who, who's one of the SRE leaders in Pittsburgh talking about you, you get a lot of the benefit of SRE. 243 00:27:08,020 --> 00:27:13,640 If you just took, you know, a third of your developers and told them to go make things better. 244 00:27:14,300 --> 00:27:14,740 Right. 245 00:27:14,760 --> 00:27:16,420 Like, don't, don't worry about, right. 246 00:27:16,820 --> 00:27:21,220 Don't worry about just make things better, make, make things more redundant. 247 00:27:21,220 --> 00:27:22,480 Think about how they're going to fail. 248 00:27:22,620 --> 00:27:27,360 Think about how they're scale and then apply software, software development to that process. 249 00:27:28,280 --> 00:27:41,120 So this, this whole setup is to get to this notion of toil in, in the SRE work week, month, quarter, whatever, there's a goal, explicit goal. 250 00:27:41,120 --> 00:27:45,940 That there's no more than 50% of the time spent on toil. 251 00:27:47,340 --> 00:27:53,580 And, and toil is what most, most of us would consider traditional operations work, right? 252 00:27:53,660 --> 00:27:57,440 So break, fix, incident management, that kind of stuff. 253 00:27:58,100 --> 00:27:58,260 Yeah. 254 00:27:59,020 --> 00:28:21,840 And the SRE team, according to the book, when their, when their toil budget is exceeded is, has explicit mandate from, from Google to push the, the, whatever kind of window for responsibility, operational responsibilities and toil back onto the, the dev teams. 255 00:28:22,560 --> 00:28:26,160 So it's just like, we'll run this for you until, until you're making our life hard. 256 00:28:26,240 --> 00:28:29,120 And then you're back, you're back on it with us. 257 00:28:29,360 --> 00:28:29,540 Right. 258 00:28:29,840 --> 00:28:30,060 Yeah. 259 00:28:30,480 --> 00:28:32,840 And, and then it's not explicit. 260 00:28:32,840 --> 00:28:36,360 Like it doesn't have a, a story about this happening to anyone. 261 00:28:36,500 --> 00:28:43,600 There's no names, but it says that it's even possible for you to get kicked out of the SREs basically. 262 00:28:43,600 --> 00:28:55,160 Like if your application is not well-behaved, so, so just like you had to prove that it was well-behaved to get into the SRE program in the first place from the Google side. 263 00:28:55,200 --> 00:28:59,420 And it's a big deal inside of Google when you do, you talk to the software engineers there. 264 00:29:00,020 --> 00:29:04,300 When, when, when you get attention from the SREs, it's a, it's kind of a milestone. 265 00:29:04,580 --> 00:29:09,900 You can, you can actually get kicked out if you're, if you're causing, if you're making their life hard. 266 00:29:09,900 --> 00:29:10,200 Right. 267 00:29:10,620 --> 00:29:19,160 So it's sort of a, it's, it's a true partnership between the developers and operations, but it's also managed by the, these numerical contracts. 268 00:29:19,160 --> 00:29:25,280 It's like, if you, if you make my toil go higher than this, like, then you're here with me. 269 00:29:25,280 --> 00:29:27,040 Like, I'm not, I'm not doing this for you. 270 00:29:27,240 --> 00:29:28,340 We're doing it for together. 271 00:29:29,680 --> 00:29:30,040 Yeah. 272 00:29:30,060 --> 00:29:37,000 I mean, I guess it's a, it's an indication of, uh, the, the, the quality of your software, if there's a lot of toil to have to deal with it. 273 00:29:37,260 --> 00:29:42,340 And, and it's sort of like the, uh, uh, one of the things we talked about early on, like, like kind of the error budget. 274 00:29:42,340 --> 00:29:48,060 Like if there's, if there's, obviously you need to go back and make your software better, right? 275 00:29:48,060 --> 00:29:52,660 Like it doesn't, it doesn't fit the way that we want to operate. 276 00:29:52,660 --> 00:29:55,060 And I assume sort of profitably are operating. 277 00:29:55,400 --> 00:30:05,680 So it forces the, in, in that kind of cool headed way that numbers can do, uh, it, it forces you to go back and work on quality versus other things. 278 00:30:06,140 --> 00:30:07,280 I, I just liked that. 279 00:30:07,320 --> 00:30:13,480 It makes it really explicit to all parties that the goals in a numerical way, a measurable way. 280 00:30:14,100 --> 00:30:21,780 So, so you don't have the, in some ways it kind of eliminates some of the finger pointing and, and, you know, that's another chapter. 281 00:30:21,960 --> 00:30:25,480 They, they definitely talk about blameless cultures and trying to fix and do these things. 282 00:30:25,480 --> 00:30:37,640 But by removing some, I think you remove some of the possibility for that, but getting to this like really strict numerical value for, for how you're going to think about that. 283 00:30:37,640 --> 00:30:38,040 Yeah. 284 00:30:38,660 --> 00:30:39,020 Yeah. 285 00:30:39,020 --> 00:30:45,540 I mean, in a very, uh, in a very small wormhole between the two concepts kind of way, I just finished reading that. 286 00:30:45,540 --> 00:30:47,540 Um, did I finish reading it? 287 00:30:47,600 --> 00:30:59,140 I think I did that book by, uh, Mark Schwartz, you know, who's the CIO of, of one of the, the, uh, uh, DHS groups, uh, called the art of business value and art of business value. 288 00:30:59,140 --> 00:30:59,440 Yeah. 289 00:30:59,440 --> 00:31:00,000 I just saw Mark. 290 00:31:00,480 --> 00:31:00,920 Yeah. 291 00:31:01,060 --> 00:31:07,340 You know, you know, usually when you ask, let's, let's call them a, uh, I don't know what to call these people anymore. 292 00:31:07,340 --> 00:31:14,140 Like the sort of DevOps, agile, cognoscenty, you know, all the people who are sort of like trying to the intelligentsia of that. 293 00:31:14,280 --> 00:31:18,180 And you ask them anything associated with metrics and proving business value. 294 00:31:18,180 --> 00:31:22,860 They pull a Obi-Wan Kenobi on you and you're like, Whoa, I don't even realize what happened. 295 00:31:22,940 --> 00:31:24,240 Suddenly I don't care about that. 296 00:31:24,240 --> 00:31:26,360 Like they never really directly answer the question. 297 00:31:27,120 --> 00:31:32,540 And to be honest, he doesn't really give a super direct answer to it, but he has one of the best. 298 00:31:32,540 --> 00:31:35,180 These are not the droids you're looking for answers in that book. 299 00:31:35,480 --> 00:31:38,940 And he, um, the best, not the droids you're looking for. 300 00:31:39,520 --> 00:31:40,180 That's right. 301 00:31:40,280 --> 00:31:42,040 Like he basically goes over. 302 00:31:42,500 --> 00:31:45,280 So finding business value is really hard. 303 00:31:45,280 --> 00:31:45,760 Right. 304 00:31:45,760 --> 00:31:51,840 And if you're just focused on ROI and all these other things, like the problem is to try to 305 00:31:51,840 --> 00:31:53,320 summarize what his point. 306 00:31:54,180 --> 00:31:59,380 Business value is unique to every single person and group and company that asked this question. 307 00:31:59,380 --> 00:32:00,680 I mean, more or less, it's unique. 308 00:32:00,680 --> 00:32:01,840 There's some common patterns. 309 00:32:01,840 --> 00:32:05,740 Like if you're a retailer, you probably would like to increase sales, right? 310 00:32:05,740 --> 00:32:10,500 Like, so there are some things to do that, but what the business finds valuable in terms of 311 00:32:10,500 --> 00:32:13,220 it can be very difficult to figure out. 312 00:32:13,220 --> 00:32:17,140 And he actually offers towards the end of the book, some, some, uh, systems of thinking. 313 00:32:17,140 --> 00:32:19,860 So you can find out what your business value is. 314 00:32:19,860 --> 00:32:24,320 But one of the interesting points he makes, and you can kind of see his government background 315 00:32:24,320 --> 00:32:30,820 peeking out through this is like, um, in the literature, uh, business value is often associated 316 00:32:30,820 --> 00:32:36,660 with what a user wants and the end business, like transacting money between a business and 317 00:32:36,660 --> 00:32:37,720 whoever their customer is. 318 00:32:37,840 --> 00:32:41,980 But it turns out that business value can be a whole lot of internal things as well. 319 00:32:42,160 --> 00:32:46,460 So like I, one of the, my favorite part of the book is, is in, in the government, at least 320 00:32:46,460 --> 00:32:48,420 in the U S government, I don't know how it is elsewhere. 321 00:32:48,660 --> 00:32:54,020 We value transparency and agencies doing their work almost above everything else because we 322 00:32:54,020 --> 00:32:56,560 just really don't want corruption and all sorts of things. 323 00:32:56,560 --> 00:33:02,580 So transparency is a prime deliverable thing and therefore something that has high business 324 00:33:02,580 --> 00:33:04,540 value that you wouldn't necessarily think about. 325 00:33:04,540 --> 00:33:09,640 And so that's why we tend to have almost too much compliance and auditing and things like 326 00:33:09,640 --> 00:33:09,840 that. 327 00:33:09,840 --> 00:33:14,960 Cause transparency is extremely important to the citizens and the legislature, all this 328 00:33:14,960 --> 00:33:15,180 stuff. 329 00:33:15,180 --> 00:33:22,780 So, and, and so the way this, that thin wormhole is, is that clearly in, in Google SRE land, 330 00:33:22,920 --> 00:33:28,180 one of the business values they have is that we shouldn't have to muck with all this toil driven 331 00:33:28,180 --> 00:33:29,700 operation stuff so much. 332 00:33:29,700 --> 00:33:34,520 Like that's codified as something that you need to deliver on that you may not traditionally 333 00:33:34,520 --> 00:33:36,260 think as a type of business value. 334 00:33:36,440 --> 00:33:41,920 And so the point that you keep coming back to is, is they've figured out a way to institutionalize 335 00:33:41,920 --> 00:33:44,920 it, to wrap a bureaucracy and a governance around it. 336 00:33:44,920 --> 00:33:46,080 That's driven by numbers. 337 00:33:46,260 --> 00:33:52,080 And that has very, uh, very apparent and, and apparently enforced rules. 338 00:33:52,540 --> 00:33:58,580 Um, again, but then all the way connect back to the business value that we want is a side effect. 339 00:33:58,580 --> 00:34:01,460 Almost is like, we don't want to have to spend a lot of time operating this. 340 00:34:01,460 --> 00:34:05,720 Cause if we're doing that, it probably indicates there's not good quality to this software, 341 00:34:05,720 --> 00:34:09,520 which even goes back to the business value of, you know, making the end users happy and 342 00:34:09,520 --> 00:34:10,460 running the business well. 343 00:34:10,660 --> 00:34:15,100 But it's, it's, it's a good example of tracing that confusing business value thing down to 344 00:34:15,100 --> 00:34:19,500 actual, like probably numbers and rules somewhere. 345 00:34:19,800 --> 00:34:20,960 I totally agree. 346 00:34:20,960 --> 00:34:26,700 And I think very few people, and I'll include myself in this statement, really comprehend 347 00:34:26,700 --> 00:34:29,660 the unfathomable scale that Google operates at. 348 00:34:30,100 --> 00:34:37,500 And, and at that scale, the, the smallest inefficiency is literally millions of dollars. 349 00:34:38,160 --> 00:34:38,720 Right. 350 00:34:38,760 --> 00:34:48,160 And so they, they've had a, um, kind of philosophical methodological approach to driving those inefficiencies 351 00:34:48,160 --> 00:34:50,380 out of the system, uh, since the inception. 352 00:34:51,140 --> 00:34:51,540 Yeah. 353 00:34:51,700 --> 00:34:56,160 So there's another thing that we can kind of close with that I think is particularly interesting 354 00:34:56,160 --> 00:35:02,000 from a, from a pivotal narrative and also plays into the kind of the CRE partnership 355 00:35:02,000 --> 00:35:03,800 that we're doing with Pivotal Cloud Foundry. 356 00:35:04,460 --> 00:35:09,600 There, there's a bunch of chapters that, that talk about all this other stuff to build up to 357 00:35:09,600 --> 00:35:15,700 this, but there's kind of this, this closing where it's framing the, you know, sort of the end 358 00:35:15,700 --> 00:35:19,720 of the book, but it's framing everything essentially in this language around platforms. 359 00:35:20,380 --> 00:35:20,740 Right. 360 00:35:20,740 --> 00:35:25,680 And, and I posted some of these on, on, uh, Twitter and maybe you want to link to them, 361 00:35:25,680 --> 00:35:30,480 but there's these quotes from the book where it says things like in order to ensure that 362 00:35:30,480 --> 00:35:35,800 the SREs, you know, don't, don't get the, the snowflake of the month to show up and 363 00:35:35,800 --> 00:35:40,420 then kind of start from, from first principles, trying to understand what's going on with it 364 00:35:40,420 --> 00:35:46,000 every time that, that they built these, these infrastructure and architectural components 365 00:35:46,000 --> 00:35:52,600 that the rest of the Google software engineers, uh, can take advantage of so that when, when 366 00:35:52,600 --> 00:35:58,480 they, when they show up built from these sort of platform components, then it's kind of pre 367 00:35:58,480 --> 00:35:58,940 audited. 368 00:35:58,940 --> 00:36:05,180 It's kind of pre, you know, pre, um, not, not approved is not the right word, but they, 369 00:36:05,180 --> 00:36:06,580 they, they, they built them. 370 00:36:06,740 --> 00:36:11,720 The SREs built many of these components and, and that's another quote, or that's another 371 00:36:11,720 --> 00:36:17,040 observation I had is that the, the SREs are, are largely architects as much as they are 372 00:36:17,040 --> 00:36:17,620 operations. 373 00:36:18,560 --> 00:36:22,120 And so when you take that framing around the platforms and the quotes from the book, 374 00:36:22,120 --> 00:36:27,240 they're essentially arguing that they built something that's, you know, Google specific, 375 00:36:27,240 --> 00:36:32,580 obviously like, like a, like a pivotal cloud foundry where you're, you're basically giving 376 00:36:32,580 --> 00:36:40,220 the, the software developers, the infrastructure and application components that make doing the 377 00:36:40,220 --> 00:36:43,960 right thing with respect to right reliability, simple and easy. 378 00:36:44,220 --> 00:36:44,700 Yeah. 379 00:36:44,820 --> 00:36:48,280 I mean, I mean, that gets, that gets back to those kind of two buckets I was talking 380 00:36:48,280 --> 00:36:48,760 about earlier. 381 00:36:48,760 --> 00:36:54,180 And, uh, I think, I think recently when I've been talking with, with organizations and enterprise 382 00:36:54,180 --> 00:36:58,740 about, you know, cloud native and pivotal cloud foundry, one of the things that falls in that 383 00:36:58,740 --> 00:37:04,060 first bucket, things you thought were happening is like, well, if we have a standard way of doing 384 00:37:04,060 --> 00:37:06,680 all of this, things will run a lot more smoothly. 385 00:37:07,360 --> 00:37:07,760 Right. 386 00:37:07,840 --> 00:37:11,380 And, and, and, and in the, in the, what you were just talking about, I mean, the standard 387 00:37:11,380 --> 00:37:16,300 way is here's a bunch of tools that the people who operate it and architectures and ways of 388 00:37:16,300 --> 00:37:19,500 right running your software that we've written is that therefore we understand. 389 00:37:19,840 --> 00:37:23,180 And therefore things will go a lot more smoothly if you just use those. 390 00:37:23,700 --> 00:37:29,000 And I mean, there's, there's, there are a series of reasons that are more legit and understandable 391 00:37:29,000 --> 00:37:33,780 than others and enterprises, particularly like M and a and, uh, things where you've got, 392 00:37:33,920 --> 00:37:39,000 uh, two companies colliding and their architectures, uh, like, of course they're going to be 393 00:37:39,000 --> 00:37:42,060 different, but you know, it's projects. 394 00:37:42,260 --> 00:37:42,800 They're so fun. 395 00:37:42,920 --> 00:37:43,300 Oh yeah. 396 00:37:43,420 --> 00:37:43,640 Yeah. 397 00:37:43,700 --> 00:37:48,100 But, but, you know, it definitely seems like, I mean, this is almost, it's one of those things 398 00:37:48,100 --> 00:37:53,360 where like, when we said you should have a standardized platform that everyone in your organization 399 00:37:53,360 --> 00:37:58,420 uses, we actually meant that it's, it's my old, uh, you should actually floss and eat more 400 00:37:58,420 --> 00:37:59,660 fruits and vegetable things. 401 00:37:59,920 --> 00:38:01,600 And I think, I don't know. 402 00:38:01,660 --> 00:38:08,140 I mean, there probably is some interesting subjective studies of this, but it almost seems 403 00:38:08,140 --> 00:38:12,520 like if you're not paying attention to it very closely, there's a natural rate of decay 404 00:38:12,520 --> 00:38:14,840 where you just lose track of all of that. 405 00:38:14,840 --> 00:38:20,000 And it's good every, every decade or so to probably go back to that basic principle and 406 00:38:20,000 --> 00:38:20,960 try to introduce something. 407 00:38:20,960 --> 00:38:26,100 And so in the point of like, yeah, in the point of not exactly running like Google, but 408 00:38:26,100 --> 00:38:30,820 taking ideas for them, it's like, it turns out this principle is extremely handy for all 409 00:38:30,820 --> 00:38:34,900 sorts of reasons that if you standardize on something, you'll, uh, you'll win a lot better. 410 00:38:36,300 --> 00:38:39,880 So the, the idea you just expressed is, is spot on. 411 00:38:39,980 --> 00:38:45,080 The thing that I think has changed in, and, and let's be like, if you go through the history 412 00:38:45,080 --> 00:38:48,680 of Cloud Foundry, it was Google inspired from its inception. 413 00:38:50,080 --> 00:38:55,040 What's changed in the, in the new world is that these standards are actually enforced by 414 00:38:55,040 --> 00:38:55,940 the software itself. 415 00:38:55,940 --> 00:38:56,360 Right. 416 00:38:56,420 --> 00:39:02,220 I think in many organizations that, that we work with, they've, they've adopted standards, 417 00:39:02,220 --> 00:39:07,580 standards, but those standards are more about checklists and process and signatures and, 418 00:39:07,580 --> 00:39:11,600 and they're very, very few are, are enforced. 419 00:39:11,760 --> 00:39:13,840 The policies are not enforced by the software itself. 420 00:39:13,840 --> 00:39:21,700 And I think we're moving to a software defined platform enforcing itself with respect to all 421 00:39:21,700 --> 00:39:22,100 these things. 422 00:39:22,100 --> 00:39:26,320 And there's a, there's another, we're not, we don't have time to go through, um, some of 423 00:39:26,320 --> 00:39:32,200 this, but it's very interesting, um, developments around how these platforms make secure 424 00:39:32,220 --> 00:39:35,840 security easier and security more, more feasible. 425 00:39:36,200 --> 00:39:36,640 Yeah. 426 00:39:36,700 --> 00:39:40,000 It's all, it's almost like, uh, software defined governance. 427 00:39:40,600 --> 00:39:41,040 Right. 428 00:39:41,080 --> 00:39:45,740 And I was, I was rewatching, um, one of our coworkers, Cornelia Davis, and she gives a good 429 00:39:45,740 --> 00:39:53,000 talk about, uh, uh, uh, how, how a traditional siloed organization, um, the, the way that it 430 00:39:53,000 --> 00:39:56,640 transforms and what it looks like when you're operating and, uh, whatever you want to call it, DevOps 431 00:39:56,640 --> 00:39:57,620 or cloud native way. 432 00:39:57,620 --> 00:40:01,280 And you have your, your product and set of project teams and all that kind of great stuff. 433 00:40:01,280 --> 00:40:05,540 But there's, there's one instant moment that I, I, uh, I, I caught this time listening 434 00:40:05,540 --> 00:40:09,280 to it where, where she essentially goes over this point that you're saying, and like, whether 435 00:40:09,280 --> 00:40:14,560 it's security or compliance or governance, if something's like so important that you have 436 00:40:14,560 --> 00:40:17,060 to do it, you should just have the platform do it for you. 437 00:40:17,200 --> 00:40:20,680 And it kind of, you know, you should, you should basically to use this term kind of 438 00:40:20,680 --> 00:40:24,900 inaccurately, you should automate it or make sure it's enforced by the actual technology 439 00:40:24,900 --> 00:40:25,740 instead of the people. 440 00:40:25,960 --> 00:40:29,540 And it kind of goes back to one of the, the, the, the themes running through a lot of what 441 00:40:29,540 --> 00:40:34,320 we've been talking about is, um, uh, I don't know how to phrase this right. 442 00:40:34,400 --> 00:40:39,080 But if we wanted, if we have something that we want to do and enforce, we need to make 443 00:40:39,080 --> 00:40:42,840 sure to write some software that forces us to do that metaphorically in the same way 444 00:40:42,840 --> 00:40:47,580 that a compiler will tell you like, no, that doesn't work and just won't compile your code. 445 00:40:47,740 --> 00:40:53,000 You have these stop gaps for all this governance that used to be, uh, more manual and kind 446 00:40:53,000 --> 00:40:55,180 of done on a smile, a handshake and a PowerPoint. 447 00:40:55,480 --> 00:40:58,280 And instead you have the system doing it all for you. 448 00:40:58,620 --> 00:41:05,000 I say this having spent many years of my life as a software developer that if you don't make 449 00:41:05,000 --> 00:41:13,000 doing the right thing, the easy thing, then many developers being forced to choose between 450 00:41:13,000 --> 00:41:17,160 doing something right and doing something right now will not do the right thing. 451 00:41:17,700 --> 00:41:17,920 Absolutely. 452 00:41:18,080 --> 00:41:23,160 That's almost the credo of the software developer is do the fastest thing to be done. 453 00:41:23,160 --> 00:41:25,580 Or I think there's a, there's a footnote on that. 454 00:41:25,720 --> 00:41:27,360 The names have been changed to protect the guilty. 455 00:41:27,600 --> 00:41:27,840 Yeah. 456 00:41:27,920 --> 00:41:32,080 I think, I think there's a footnote that if you're in your twenties, do the slowest thing 457 00:41:32,080 --> 00:41:37,000 that maximizes my enjoyment, like building your own platforms and everything and frameworks. 458 00:41:37,200 --> 00:41:42,580 So there's that, those are the competing desires of, I think, software developers, which, uh, we 459 00:41:42,580 --> 00:41:47,600 could make a chart how that changes over maybe not age, but experience, uh, that you have. 460 00:41:47,660 --> 00:41:50,920 Here is a, is a great hobby to have in your time. 461 00:41:51,760 --> 00:41:52,120 Yeah. 462 00:41:52,320 --> 00:41:52,860 Well, great. 463 00:41:52,920 --> 00:41:56,140 Well, before we wrap up, is there anything else you want to, uh, make sure to call out 464 00:41:56,140 --> 00:41:56,400 here? 465 00:41:57,460 --> 00:41:58,760 I mean, we covered a lot of ground. 466 00:41:58,760 --> 00:42:02,060 I think that there's, uh, more to this story. 467 00:42:02,060 --> 00:42:05,320 You know, obviously we're, we're working through this stuff. 468 00:42:05,380 --> 00:42:09,860 We're kind of in the middle of it with Google, but I'm excited to see the, the, the language 469 00:42:09,860 --> 00:42:15,980 around service level objectives being, being created and published for, for cloud foundry. 470 00:42:15,980 --> 00:42:21,740 And, and then seeing that mentality and mindset permeate all through the rest of the, uh, the 471 00:42:21,740 --> 00:42:22,500 pivotal practice. 472 00:42:22,660 --> 00:42:23,100 Yeah. 473 00:42:23,600 --> 00:42:24,220 Well, great. 474 00:42:24,300 --> 00:42:27,960 Well, uh, thanks for being, if people, if I'll try to find links to the, uh, I think they 475 00:42:27,960 --> 00:42:32,020 put repost posted the recordings of all their talks and I'll, I'll put in the show notes, 476 00:42:32,020 --> 00:42:36,020 uh, a lot of the stuff we talked about, you can always get the show notes at pivotal.io 477 00:42:36,020 --> 00:42:37,080 slash podcast. 478 00:42:37,320 --> 00:42:41,040 But if people want to, uh, check in on you more, what's the, what's the best place to 479 00:42:41,040 --> 00:42:41,860 go on the internet? 480 00:42:43,520 --> 00:42:49,180 The easiest way to get my attention is little idea on Twitter and I have open DM. 481 00:42:49,180 --> 00:42:52,420 So if you want to tell me something, um, I'd probably respond. 482 00:42:52,840 --> 00:42:57,400 Uh, that, that's the, that's the thing that I have, you know, some different talks coming 483 00:42:57,400 --> 00:42:57,580 up. 484 00:42:57,660 --> 00:42:59,720 I'm doing, uh, DevOps days in Denver. 485 00:43:00,000 --> 00:43:02,940 There's the spring days that Pivotal is planning. 486 00:43:03,660 --> 00:43:08,540 There's probably a couple of trips to, you know, Cloud boundary summits a little ways 487 00:43:08,540 --> 00:43:12,380 out, but that should be a, some people to think about if you're so inclined. 488 00:43:12,860 --> 00:43:13,060 Oh yeah. 489 00:43:13,140 --> 00:43:15,160 I, I forget, I forget to put that little ad in here. 490 00:43:15,200 --> 00:43:16,140 We do have spring days. 491 00:43:16,140 --> 00:43:18,940 I mean, that's over, uh, the next like three or four months, right? 492 00:43:19,020 --> 00:43:21,000 And I don't know, 10 cities or so. 493 00:43:22,960 --> 00:43:26,600 Yeah, we're, we're starting to do some grassroots, you know, organic spring. 494 00:43:26,720 --> 00:43:29,660 I mean, it's not, it's not, it's like a different topic than CRE, but it's. 495 00:43:30,000 --> 00:43:37,960 It's the Pivotal, uh, Java application framework and there's a ton of community and adoption. 496 00:43:37,960 --> 00:43:42,820 And so we're just trying to kind of give back and invest in, in having those community events 497 00:43:42,820 --> 00:43:46,860 where they don't have to go through the whole year and they don't have to travel so much. 498 00:43:46,860 --> 00:43:53,200 So we're, we're targeting, um, a couple of cities and then we also have a form where people 499 00:43:53,200 --> 00:44:00,280 can request to, to organize spring days locally and Pivotal is involved and we're kind of 500 00:44:00,280 --> 00:44:05,720 trying to provide a, a, a logistic support, but it's really locally organized. 501 00:44:06,400 --> 00:44:07,060 That's nice. 502 00:44:07,060 --> 00:44:11,980 And yeah, I mean, it's very similar to some of these other local conferences, but the, 503 00:44:11,980 --> 00:44:18,540 the spring communities exploding the interest in, you know, spring boot and the, the microservices 504 00:44:18,540 --> 00:44:22,280 and spring cloud and being able to do the integrations and, you know, service discovery and the rest 505 00:44:22,280 --> 00:44:22,600 of it. 506 00:44:22,600 --> 00:44:26,340 But, and then the new reactive stuff, it's, it's really interesting to see that kind of 507 00:44:26,340 --> 00:44:31,780 re, re, uh, I don't know, reinvigoration of Java. 508 00:44:31,920 --> 00:44:37,120 Like it's a, it's a renaissance of, of, um, uh, we'll, we'll, we'll say like there's this 509 00:44:37,120 --> 00:44:38,200 dark ages of Java. 510 00:44:38,200 --> 00:44:44,400 We won't name any technologies, but I think we're, we're through that and we're, we're, 511 00:44:44,400 --> 00:44:48,820 we're in a, a true renaissance with the kind of cloud native architectural patterns 512 00:44:48,820 --> 00:44:49,340 that are happening. 513 00:44:49,600 --> 00:44:49,700 Yeah. 514 00:44:49,800 --> 00:44:54,660 Well, as I was trying to write up in this column I had last month at the register, like the 515 00:44:54,660 --> 00:44:59,180 great thing about Java and other languages have this is that it's, it, uh, I don't know 516 00:44:59,180 --> 00:45:04,280 how it happens, but to anthropomorphize it, it allows itself to constantly evolve and shift 517 00:45:04,280 --> 00:45:04,660 around. 518 00:45:04,660 --> 00:45:07,520 It doesn't kind of get stuck in one way of doing things. 519 00:45:07,520 --> 00:45:11,780 So while there, while there may be like, uh, bummers that happened here and there for 520 00:45:11,780 --> 00:45:17,360 several years, like eventually Java's pretty trustworthy as an overall community of like catching 521 00:45:17,360 --> 00:45:20,960 up and evolving and, uh, if not leading on, on various things. 522 00:45:20,960 --> 00:45:24,240 So it's, it's a very, uh, very thriving sort of ecosystem. 523 00:45:25,160 --> 00:45:30,560 But, uh, also before, before I wrap up, uh, we also have the, uh, not, not we, but the 524 00:45:30,560 --> 00:45:35,240 entire cloud foundry, uh, community has the cloud foundry summit coming up on June 13th 525 00:45:35,240 --> 00:45:35,840 and 15th. 526 00:45:35,920 --> 00:45:39,980 I think, you know, there's a lot of like, uh, low level nerd stuff where they show off technical 527 00:45:39,980 --> 00:45:41,440 things and you can do that. 528 00:45:41,440 --> 00:45:46,080 But what I really like about it, uh, is, is that there's a lot of actual users and customers 529 00:45:46,080 --> 00:45:50,100 of all the various cloud foundries who come and they not only tell you how awesome they 530 00:45:50,100 --> 00:45:53,740 are and how things are working, but many of those talks go over things they've tried out 531 00:45:53,740 --> 00:45:54,780 that didn't work well. 532 00:45:55,240 --> 00:46:00,120 And, uh, you know, the, the, the failures and the successes as they've tried to switch over 533 00:46:00,120 --> 00:46:02,380 their more or less large organization. 534 00:46:02,380 --> 00:46:06,860 That's usually what they are to, uh, doing things in, uh, in, in a cloud native kind of way. 535 00:46:06,860 --> 00:46:10,880 And if you want to get 20% off, I guess it's cloud foundry, Silicon Valley. 536 00:46:11,080 --> 00:46:15,020 So they have this weird code that CFSV 17 Coté. 537 00:46:15,240 --> 00:46:16,980 That's my name, C O T E. 538 00:46:17,220 --> 00:46:20,740 And, and you can get, uh, you can get a fifth of your registration fee off. 539 00:46:20,900 --> 00:46:22,280 I'll probably end up going there. 540 00:46:22,340 --> 00:46:25,160 I haven't finalized my travel, but it's, it's, it's a good conference. 541 00:46:25,160 --> 00:46:27,740 So, uh, you should check that out if you're interested in all of this. 542 00:46:28,200 --> 00:46:30,020 It'd be a cloud native good time. 543 00:46:30,420 --> 00:46:30,780 Exactly. 544 00:46:31,000 --> 00:46:32,360 And you know, it's that high. 545 00:46:32,420 --> 00:46:36,280 It's fun that it's attached to over there in Santa Clara and you're next to the Levi 546 00:46:36,280 --> 00:46:36,840 stadium. 547 00:46:37,000 --> 00:46:39,160 Go, uh, I don't think they sell jeans there. 548 00:46:39,220 --> 00:46:40,320 They should really look into that. 549 00:46:41,180 --> 00:46:43,520 So as always, this has been Pivotal Conversations. 550 00:46:43,520 --> 00:46:48,360 If you want to get the, uh, the most up to date episodes, when they come out, what you 551 00:46:48,360 --> 00:46:52,320 should do is go over to a soundcloud.com slash Pivotal Conversations. 552 00:46:52,680 --> 00:46:56,000 You can find the RSS feed there, or you can listen to it in soundcloud. 553 00:46:56,420 --> 00:46:58,420 If you want, they have delightful visuals. 554 00:46:58,720 --> 00:47:03,380 Uh, but you just put that in your, uh, your iTunes or your overcast or whatever podcast 555 00:47:03,380 --> 00:47:06,880 listening thing you have, and it'll, it'll download them as soon as I upload them. 556 00:47:07,200 --> 00:47:10,140 Uh, and we, you can also find it in iTunes and things like that. 557 00:47:10,140 --> 00:47:13,720 And we have, uh, if you search around for pivotal in iTunes, we have a two or three other 558 00:47:13,720 --> 00:47:18,320 podcasts, particularly one on, uh, the world of data and stuff that that's very interesting 559 00:47:18,320 --> 00:47:20,300 and, uh, frequently comes out with good stuff. 560 00:47:20,720 --> 00:47:24,220 Uh, and also while you're in there, if you want to leave a review or leave some little 561 00:47:24,220 --> 00:47:29,380 stars or whatever kind of coins or resources you want to give us, uh, it's always nice to see 562 00:47:29,380 --> 00:47:32,580 that in there and it helps other people, uh, find and discover this content. 563 00:47:33,100 --> 00:47:35,840 And, uh, I'll put the show notes for this and many other stuff. 564 00:47:35,840 --> 00:47:39,600 As I mentioned earlier, many other episodes at pivotal.io slash podcast. 565 00:47:39,820 --> 00:47:41,960 And with that, we'll see everyone next time. 566 00:47:42,160 --> 00:47:42,760 Bye-bye.