1 00:00:00,000 --> 00:00:04,980 Hello, everyone. This week, we have Tony on, who's always a good guest, fun to talk with. 2 00:00:05,240 --> 00:00:09,260 There was some weird internet connection problems. So every now and then, you'll hear some little 3 00:00:09,260 --> 00:00:14,840 glitchy robot sounds, and it might even cut out some of the things he was saying. But I think it's 4 00:00:14,840 --> 00:00:20,820 all pretty much in there. But I've left those robot things in there so that you can know when 5 00:00:20,820 --> 00:00:31,180 it goes, robot. And with that, enjoy the show. So I had carrots. And I had one carrot I sliced 6 00:00:31,180 --> 00:00:37,620 lengthwise. And then I had, we got some egg salad. Egg salad? I was going to say a sandwich, 7 00:00:37,740 --> 00:00:42,600 but it's just egg salad. And we need to eat that because we're going out of town for a few days 8 00:00:42,600 --> 00:00:49,120 soon, tomorrow. But now I put a mystery ingredient in there that I learned from my wife. I want to 9 00:00:49,120 --> 00:00:54,240 give each of you one guess. What is something that would go with carrots? Mystery ingredient. 10 00:00:55,280 --> 00:00:59,440 How about you go first, Richard? So it's not an ingredient in the egg salad, 11 00:00:59,560 --> 00:01:02,980 just something that goes with carrots, period. Well, it could go with carrots on its own, 12 00:01:03,040 --> 00:01:10,860 but I did put it into the egg salad. And it's not like pepper or anything like that. Like it's its 13 00:01:10,860 --> 00:01:17,420 own thing. Pickles? Pickles. That's a good guess. Egg salad with pickles is delicious. However, 14 00:01:17,420 --> 00:01:21,680 that is not what it is. How about, how about special guest first? Why don't you introduce yourself 15 00:01:21,680 --> 00:01:27,920 and then take a guess? Tony McCauley from Home Depot. Uh, so not, not a condiment, something. So 16 00:01:27,920 --> 00:01:32,880 not pepper, not a spice. Well, it's not a spice. It is a condiment, but it's, it's, uh, you see. 17 00:01:33,380 --> 00:01:41,440 Okay. Um, let's go with, um, I have no idea. No idea. You just totally give up. That's fine. 18 00:01:41,520 --> 00:01:46,580 Okay. I mean, if they catch up, we're, we're stopping this podcast. Oh, that's close. That's 19 00:01:46,580 --> 00:01:52,980 close. The secret ingredient, the secret ingredient was mustard. Oh, I almost said mustard. I think 20 00:01:52,980 --> 00:01:59,780 mustard's okay. Yeah. Yeah. Mustard come in all egg salad. I don't know. Well, it doesn't feel 21 00:01:59,780 --> 00:02:04,780 very edgy. Okay. Okay. I, you know, this, this, uh, you know, I got, I got the Albert Heine, 22 00:02:04,920 --> 00:02:09,800 the sort of, uh, grocery store that is, I like to think of it, the H-E-B, uh, of the Netherlands 23 00:02:09,800 --> 00:02:15,620 and, uh, their egg salad is a little mild. Maybe there's like a Indonesian or Surinamese egg salad 24 00:02:15,620 --> 00:02:19,880 I can get. I don't know if there's mustard. Okay. Okay. You got me there. Normally I would put 25 00:02:19,880 --> 00:02:25,720 mustard in there, but my whole point to this story is, uh, uh, I learned about a year ago that you could 26 00:02:25,720 --> 00:02:30,040 actually put mustard, just regular mustard on carrots. And that just like blew my mind. 27 00:02:30,460 --> 00:02:36,120 That seemed just crazy. But, uh, apparently that's totally normal. In some parts. I'm gonna, 28 00:02:36,120 --> 00:02:42,480 yeah, I'm gonna take your word for it and try it. Yeah. It's pretty good. Like I had a, I had a, 29 00:02:42,480 --> 00:02:47,180 I had a Dijon mustard, a little spicy, a little horseradishy, even regular mustard. It's, 30 00:02:47,260 --> 00:02:52,480 it's a strange combination, but, but, uh, but it works out. It's pretty good. See, if you do, 31 00:02:52,480 --> 00:02:58,100 if you do the spicy mustard, I, I can do that on almost everything. So yeah. Yeah. See, exactly. 32 00:02:58,240 --> 00:03:01,660 That's, that's, I normally, I, I, that's what I was thinking with the Dijon mustard. It's like, 33 00:03:01,680 --> 00:03:07,280 I like, I like some spicy food and I have a theory that like, if you have, you know, I can only eat 34 00:03:07,280 --> 00:03:13,080 like one carrot before I just like am really bored, you know, like carrots really good. And then like 35 00:03:13,080 --> 00:03:18,460 you get to that second carrot and it's just like, oh man, I feel like I should just put a bunch of 36 00:03:18,460 --> 00:03:22,080 dirt in my mouth. Not that it tastes like dirt, but it's just like, there's nothing going on. 37 00:03:22,680 --> 00:03:28,100 And, uh, I thought if I had some, some spiciness, it would, it would, uh, I guess metaphorically 38 00:03:28,100 --> 00:03:33,100 speaking, spice up the carrot. That's right. Maybe it goes on your tuna pizza from a couple episodes 39 00:03:33,100 --> 00:03:41,940 back. Oh no, I still am not into that. That's, that's no good. That is no good. I don't, I don't 40 00:03:41,940 --> 00:03:46,260 think fish goes on pizza. That's well, I guess anchovies. That's it. But is that really 41 00:03:46,260 --> 00:03:54,540 fish? Anchovies. Yeah. Yeah. It's like the McDonald's of the fish. Yeah. McDonald's version 42 00:03:54,540 --> 00:03:58,840 of fish. That's true. Well, we'll, uh, we'll say, you know, I think there's a lengthy fish 43 00:03:58,840 --> 00:04:03,840 conversation we could have about anchovies where I think I feel like an anchovy is sort 44 00:04:03,840 --> 00:04:09,200 of like a chip and in that a chip is often just an excuse to eat a bunch of queso, you know? 45 00:04:09,200 --> 00:04:13,220 And I don't know, maybe anchovy is just an excuse for salt, but that's, that's a category 46 00:04:13,220 --> 00:04:17,940 of food we could dive into, uh, on the next episode you're on, perhaps Tony. Well, why 47 00:04:17,940 --> 00:04:21,140 don't you, uh, once you introduce yourself, like you've already introduced yourself, but 48 00:04:21,140 --> 00:04:26,820 like, what do, uh, what do you work on? What's your deal? What is my deal? My deal is to, 49 00:04:26,820 --> 00:04:32,440 uh, so, uh, we have a bunch of developers at Home Depot who would have thought, and, uh, I 50 00:04:32,440 --> 00:04:37,740 think my deal is to make sure they are happy and productive. That is kind of my deal. 51 00:04:38,180 --> 00:04:43,000 You know, and I don't know if we discussed this in past conversations, but how did, uh, 52 00:04:43,340 --> 00:04:44,520 how did you end up doing that? 53 00:04:44,520 --> 00:04:49,820 Well, we started out, uh, running a platform team. So we had this old platform we built and 54 00:04:49,820 --> 00:04:55,260 then, uh, we signed a deal with you guys a few years ago and started doing CF. Um, and 55 00:04:55,260 --> 00:05:00,360 it turned out CF was really cool for empowering all of our developers and they loved it and they 56 00:05:00,360 --> 00:05:06,000 could choose the languages they wanted to work in. And, uh, I guess they enjoyed the way 57 00:05:06,000 --> 00:05:11,800 that we ran the platform. Uh, I think we did it a little untraditionally, uh, from typical 58 00:05:11,800 --> 00:05:16,000 infrastructure side of things. And then, uh, yeah, so they wanted us to do more and more 59 00:05:16,000 --> 00:05:21,900 things. So, um, you know, we started running some dev tool stuff, um, and now we're doing 60 00:05:21,900 --> 00:05:27,460 trainings and, uh, enablement stuff and now we're changing it controls. And so we're doing 61 00:05:27,460 --> 00:05:28,540 all sorts of cool things. 62 00:05:28,540 --> 00:05:32,640 Is, is, is kind of that collection of stuff different than what, what you, uh, you would 63 00:05:32,640 --> 00:05:33,840 have anticipated it being? 64 00:05:34,000 --> 00:05:39,500 Yes. Yeah, absolutely. Uh, yeah. I mean, when we started four years ago, it was, uh, very, 65 00:05:39,500 --> 00:05:44,580 um, like, Hey, we have some things that you need to go run and operate and make sure it doesn't 66 00:05:44,580 --> 00:05:50,880 fall down. Uh, and, and that was great. And we did that. Um, and then, uh, I think just the 67 00:05:50,880 --> 00:05:56,460 community aspect of things, we built up a pretty big Slack community. Um, and now, I mean, 68 00:05:56,460 --> 00:06:01,340 they just like, Hey, why don't you run the API gateway for us? Or, Hey, why don't you, uh, do 69 00:06:01,340 --> 00:06:06,440 something about these terrible IT controls? And so we just, we just started taking on more and 70 00:06:06,440 --> 00:06:06,960 more things. 71 00:06:07,440 --> 00:06:11,400 Yeah. You know, I, and that's, that's, uh, as, as you're going over that, I was, I was 72 00:06:11,400 --> 00:06:16,380 realizing, you know, one, one aspect I've noticed it, uh, with, with groups like yourself, 73 00:06:16,380 --> 00:06:21,260 right. Who are the, uh, as you add to summarize what you're saying, making the developers happy 74 00:06:21,260 --> 00:06:27,560 and enabling them and all that, uh, like that, uh, that doesn't seem like a traditional 75 00:06:27,560 --> 00:06:32,740 operations infrastructure thing to do. And, and I think, I think when I've talked with people 76 00:06:32,740 --> 00:06:40,020 who operations, people who are running a platform or who are in that role nowadays, uh, they all, 77 00:06:40,140 --> 00:06:45,880 I, force is the wrong word, but I'll use the passive voice. They all end up, uh, sort of doing 78 00:06:45,880 --> 00:06:51,140 what you described essentially being like an internal provider and having to, uh, having 79 00:06:51,140 --> 00:06:56,260 to getting to like talk with people and, uh, you know, have a community around it and work 80 00:06:56,260 --> 00:07:01,140 more closely with developers. And many of them have that same sort of, uh, posture, if you 81 00:07:01,140 --> 00:07:04,920 will, that you had that, like, we're, we're trying to do stuff that, uh, makes your life 82 00:07:04,920 --> 00:07:09,340 easier, but that, that, uh, people seem to evolve to that position. 83 00:07:09,700 --> 00:07:12,800 Yeah. I don't think we set out. I mean, it definitely wasn't in the cards for us and 84 00:07:12,800 --> 00:07:19,560 we sort of lucked into it. Um, you know, some of it with some, uh, uh, corralling and, uh, 85 00:07:19,560 --> 00:07:24,480 suggestion from, you know, some of the folks at Pivotal we were working with, but, um, yeah, 86 00:07:24,540 --> 00:07:28,580 I mean, I think, you know, also for us, we were all software developers, like none of us 87 00:07:28,580 --> 00:07:33,560 had systems backgrounds or infrastructure backgrounds. And so, um, for us, it was the 88 00:07:33,560 --> 00:07:35,100 only way we knew how to run things. 89 00:07:35,440 --> 00:07:37,800 How big is this team, Tony, that that's doing this function? 90 00:07:38,500 --> 00:07:45,980 Yeah. So the, uh, the platform team themselves, um, I think there are seven people now. Um, uh, 91 00:07:45,980 --> 00:07:50,820 we have some part-time help from some infrastructure people here and there, uh, for capacity build 92 00:07:50,820 --> 00:07:55,500 outs. And then, and now I have three other teams. So we have, um, a DevTools team, which 93 00:07:55,500 --> 00:08:01,780 is about seven or eight people, and they run things like Artifactory, GitHub, um, Slack. Uh, 94 00:08:01,780 --> 00:08:07,200 they do some of the Pivotal Tracker API stuff, um, Confluence. Um, and then we have another 95 00:08:07,200 --> 00:08:11,520 team, a pipeline engineering team. So they're doing some Spinnaker work and then they're also 96 00:08:11,520 --> 00:08:16,820 looking at, um, some sort of metrics and insight stuff. Um, and then we have another team that 97 00:08:16,820 --> 00:08:21,980 is software engineers and they're sort of working on, uh, filling all the gaps. So anything that 98 00:08:21,980 --> 00:08:27,200 we have to build custom, uh, to either automate some security stuff or compliance stuff for 99 00:08:27,200 --> 00:08:33,020 collaboration and sharing, that's another seven or eight. So there's about 45 to 50 people now, 100 00:08:33,020 --> 00:08:38,080 um, that work with me. Wow. That's awesome. So is that pipeline engineering team, are they 101 00:08:38,080 --> 00:08:42,100 helping build pipelines for specific teams or just building kind of the infrastructure patterns 102 00:08:42,100 --> 00:08:46,240 or models and other teams then build their pipelines for? Yeah, it's a good question. We have a, 103 00:08:46,360 --> 00:08:51,040 um, I think we were taking a slightly different approach. So the original approach was to sort of 104 00:08:51,040 --> 00:08:57,040 build one pipeline to rule them all. Um, and I think what we landed on was, um, 105 00:08:57,040 --> 00:09:01,820 and having done this at a couple of other companies, um, we didn't want to be too prescriptive 106 00:09:01,820 --> 00:09:06,920 with the CI portion of that. There's just too many languages and testing utilities and a bunch 107 00:09:06,920 --> 00:09:11,380 of other things. So we still let teams build their own CI environments. Uh, we help share 108 00:09:11,380 --> 00:09:16,100 best practices, but, um, but what we're working on now with Spinnaker is more of the CD part 109 00:09:16,100 --> 00:09:20,160 of it. So, um, actually getting your things into production, that's what that team is working 110 00:09:20,160 --> 00:09:24,600 on. Yeah. So speaking of that, so you're a, uh, you know, you at Home Depot, you yourself 111 00:09:24,600 --> 00:09:29,660 have been in cloud found your world for a long time, you know, good success, good stories. 112 00:09:29,660 --> 00:09:33,660 You've told us over the years now, I know you're doing some more Kubernetes sort of stuff 113 00:09:33,660 --> 00:09:37,240 at Home Depot. I remember when we first talked to you, like, I don't know, you know, a couple 114 00:09:37,240 --> 00:09:41,380 of years ago, I was like, you know, containers, we don't need any of that. We're doing, we're 115 00:09:41,380 --> 00:09:44,260 focused on apps and things like that, but it seems like some of the use cases have expanded. 116 00:09:44,380 --> 00:09:48,400 You're doing some Kubernetes work. Can you tell us about how you kind of see those two 117 00:09:48,400 --> 00:09:53,300 things living? Yeah, it's actually a really, uh, it's a very spirited conversation going 118 00:09:53,300 --> 00:10:00,000 on right now. Um, so we've had some people interested in Kubernetes for a while. Um, one 119 00:10:00,000 --> 00:10:05,420 of the things that's happened recently is, um, functions. Um, so we have a lot of teams 120 00:10:05,420 --> 00:10:10,600 now with, uh, SRE groups and, um, you know, a lot of these SRE groups, I think want Kubernetes 121 00:10:10,600 --> 00:10:15,020 and we have a lot of engineers that want Kubernetes. I've been like, well, what, what are we going 122 00:10:15,020 --> 00:10:19,280 to be using this for that we can't use CF for? Um, and there are some valid use cases that 123 00:10:19,280 --> 00:10:24,000 have come up. So we're seeing more and more, um, some vendored software that we buy, um, 124 00:10:24,060 --> 00:10:28,900 that, that comes packaged as a container and needs to run somewhere. Um, and we're also 125 00:10:28,900 --> 00:10:33,680 looking to migrate our old platform to something a little bit more modern and Kubernetes seems 126 00:10:33,680 --> 00:10:38,920 to be a good fit for that. Um, but in terms of, uh, uh, we still have a lot of engineers, 127 00:10:38,920 --> 00:10:43,560 I think that expect that when Kubernetes gets rolled out, it's going to be, you know, you 128 00:10:43,560 --> 00:10:46,760 get a cluster and you get it. It's like the Oprah Winfrey of containers. You know, it's 129 00:10:46,760 --> 00:10:50,260 like everyone gets a cluster and I'm not sure if that's the right pattern that we want to 130 00:10:50,260 --> 00:10:57,100 go. So we're still having some conversations, but as a whole, I feel, um, for 80, 85% of 131 00:10:57,100 --> 00:11:00,740 our workloads that we would still use CF. And I still think that's the better option. 132 00:11:00,740 --> 00:11:06,440 So, so how do you, how are you thinking through the, uh, uh, the, you get a cluster sort of 133 00:11:06,440 --> 00:11:13,540 strategy? Like that, uh, like I'm thinking of there, there's a presentation that, uh, that 134 00:11:13,540 --> 00:11:19,160 Rabobank gives where, uh, they, they kind of go over there, like, here's, here's the, the 135 00:11:19,160 --> 00:11:23,380 infrastructure of the platform that we think we should provide. Like there's four types and 136 00:11:23,380 --> 00:11:29,100 here's like, here's, you know, and now that I think of it, he doesn't really go into a detailed 137 00:11:29,100 --> 00:11:35,680 analysis, but it's sort of like across, uh, VMs, containers, and like, uh, pivotal, as pivotal 138 00:11:35,680 --> 00:11:41,880 cloud foundry and like functions. Here's, we kind of like figured, thought about what developers 139 00:11:41,880 --> 00:11:48,720 need. And, and this, this option fits these scenarios for them. Uh, and kind of like you 140 00:11:48,720 --> 00:11:54,500 were saying, most of their scenarios fit into, uh, using CF for it, but like, I don't know, 141 00:11:54,500 --> 00:12:00,460 how do you put together, like figuring out if every development team should run on like 142 00:12:00,460 --> 00:12:07,220 a Kubernetes cluster versus be a function or run in VMs or be on CF? What's the, what's the 143 00:12:07,220 --> 00:12:13,440 analysis there last year where we're like, uh, what's our platform of the future? Um, what 144 00:12:13,440 --> 00:12:16,760 is that going to be based on? And we had a lot of technical thought leaders that were 145 00:12:16,760 --> 00:12:21,380 like, uh, it has to be Kubernetes. It's the future. Um, and then I think when you stop 146 00:12:21,380 --> 00:12:26,020 focusing on the technology and all the buzzwords and you're like, yeah, but in terms of our engineers 147 00:12:26,020 --> 00:12:30,040 and how they're going to interact with this platform, what are those things? And when you 148 00:12:30,040 --> 00:12:34,240 abstract all that away, it's like the list they came up with. It was like, well, that's 149 00:12:34,240 --> 00:12:39,460 cloud foundry. Like, um, and so I think that's where we struggled is for me is that we have 150 00:12:39,460 --> 00:12:43,720 a lot of people that want to use Kubernetes for Kubernetes sake, um, versus like, well, 151 00:12:43,720 --> 00:12:47,840 what problems are we actually solving? Um, what do you, what do we want the interactions 152 00:12:47,840 --> 00:12:52,100 and the patterns and the day-to-day lives of our engineers to be like? Um, and I think 153 00:12:52,100 --> 00:12:56,140 where we're struggling is there's a lot of what I call resume building. There's a lot 154 00:12:56,140 --> 00:13:00,420 of people that just want to work with whatever this thing is that's eating the social media 155 00:13:00,420 --> 00:13:06,280 in the technology world, uh, versus, versus trying to get work done. Um, and so I think 156 00:13:06,280 --> 00:13:10,000 that's where we're struggling is, um, there are some valid use cases for Kubernetes and I 157 00:13:10,000 --> 00:13:14,420 think that want to hire, you want to the best talent at your company and they come here and 158 00:13:14,420 --> 00:13:18,880 they're like, yeah, I can't wait to work with Kubernetes. And you're like, well, uh, we 159 00:13:18,880 --> 00:13:24,780 kind of want you to sell hammers and fix like, you know, two day delivery. Um, so, so 160 00:13:24,780 --> 00:13:26,460 there's a, that's, that's part of the challenge. 161 00:13:26,460 --> 00:13:32,820 Well, you know, along those lines, uh, you know, one of the, it's almost, uh, uh, what's 162 00:13:32,820 --> 00:13:38,800 the word, uh, slightly like cliche at this point, but you often hear in the container in the 163 00:13:38,800 --> 00:13:43,820 Kubernetes world, right? That like, like Kubernetes is a platform for platforms. And I guess that 164 00:13:43,820 --> 00:13:50,280 means it's not a platform itself or something, but like, what, uh, what, what do you take that 165 00:13:50,280 --> 00:13:56,500 phrase to mean that it's like, and does that kind of fit, fit into like your, your own view 166 00:13:56,500 --> 00:13:59,720 of like where you would fit it into the, the platform you're providing? 167 00:14:00,080 --> 00:14:05,120 Yeah. I mean, that's the way I see it is. I, I see Kubernetes sort of as a infrastructure 168 00:14:05,120 --> 00:14:12,240 as a service, uh, plus or star. Right. Um, so I, I, if there were services or platforms 169 00:14:12,240 --> 00:14:16,940 that we wanted to expose to our engineers, um, you know, we would back those with Kubernetes 170 00:14:16,940 --> 00:14:21,660 when it made sense. Uh, and so the example would be, um, if a team wants a pipeline, right, 171 00:14:21,660 --> 00:14:25,520 they wouldn't get a cluster and build their own pipeline. They would get pipeline as a service 172 00:14:25,520 --> 00:14:31,340 backed by Kubernetes. Um, I don't think it would be Kubernetes itself or a cluster itself. Like 173 00:14:31,340 --> 00:14:37,020 I don't think cluster as a service is the right, uh, the right pattern to expose to teams for the 174 00:14:37,020 --> 00:14:41,480 most part. Is there certain types of package, whereas Tony you're seeing, is it databases or 175 00:14:41,480 --> 00:14:45,020 you started to do more with things like Kafka and messaging? I mean, what kind of commercial 176 00:14:45,020 --> 00:14:50,260 or package software do you find yourself throwing at these? Um, well, there's not a lot. I mean, 177 00:14:50,260 --> 00:14:54,520 there's only been a couple, that's the thing. So there's only been a couple that have come up so 178 00:14:54,520 --> 00:15:01,680 far. Um, you know, to be honest, my head is, I think, um, I, I still think we're figuring out a 179 00:15:01,680 --> 00:15:07,440 lot of, um, I, I personally, I think it's really early and, um, what I'm trying to, uh, is figure out 180 00:15:07,440 --> 00:15:12,340 the right balance of, there's a lot of really smart, talented engineering and companies that are going 181 00:15:12,340 --> 00:15:17,960 after solving these problems and, and how involved do we want to be in that? Or, um, do we just need to 182 00:15:17,960 --> 00:15:23,440 play with it enough to kind of stay, you know, up to speed with the landscape so that once it is 183 00:15:23,440 --> 00:15:29,060 figured out, you know, we can be a fast follower. So, um, you know, to be honest, I think, um, you 184 00:15:29,060 --> 00:15:35,000 know, I, I've always felt that, um, you know, the developer experience on Cloud Foundry is awesome. 185 00:15:35,420 --> 00:15:41,380 And, you know, I think it's sort of ironic that, um, you know, the, the developer experience friendly 186 00:15:41,380 --> 00:15:46,200 version of Kubernetes, like might actually be CF. Right. And it's, uh, so I think, you know, 187 00:15:46,200 --> 00:15:51,220 I think it's more of us just preparing for the future, um, staying up to speed, but it's tampering 188 00:15:51,220 --> 00:15:57,480 expectations and desires of a large technology base that, that just wants to kind of like, you know, 189 00:15:57,480 --> 00:16:03,300 play with things and tweet things and, uh, and put things on their resume and sections and, and, uh, 190 00:16:03,600 --> 00:16:07,860 and not get people upset and want to leave and do other things. So that's, that's the hard part. 191 00:16:08,640 --> 00:16:12,800 Yeah. One of the things I was thinking about when you were describing that, that challenge is, uh, 192 00:16:12,800 --> 00:16:17,260 I think it was in Paradox of Choice or a book I read recently that it's tough for humans because 193 00:16:17,260 --> 00:16:21,460 things lose their novelness with us fairly quickly. Like something that was super awesome. 194 00:16:22,000 --> 00:16:26,600 I mean, you know, the new car, the, whatever, you know, a month ago is now like, oh, that's cool. 195 00:16:26,940 --> 00:16:30,040 So do you feel like as you're further down this journey, some of the things that were just 196 00:16:30,040 --> 00:16:36,180 amazing at the beginning, whether it's getting apps to production on pipelines or CF push or some of 197 00:16:36,180 --> 00:16:39,920 these things now can maybe take for granted, are there things you think you take for granted 198 00:16:39,920 --> 00:16:44,320 because you've just been doing them a long time? What gets boring after a while? 199 00:16:44,520 --> 00:16:49,540 Oh, that is a really good question. Um, I, I think there are certainly things that we take 200 00:16:49,540 --> 00:16:55,180 for advantage, uh, take it or, uh, we don't appreciate. Um, I mean, I think scaling still is 201 00:16:55,180 --> 00:17:00,580 a big one. And a lot of the, uh, in the CF world, at least a lot of the sort of, uh, health 202 00:17:00,580 --> 00:17:05,660 checks and corrections of things that still have a lot of apps, you know, with memory leaks or issues 203 00:17:05,660 --> 00:17:09,740 here and there. And, and I think, you know, cloud foundries made it sort of easy to sweep some of 204 00:17:09,740 --> 00:17:15,480 that stuff under the rug for some teams. Um, not our most mature teams, but, um, but yeah, I think 205 00:17:15,480 --> 00:17:22,960 just the, I think the API itself, um, and how you can sort of abstract away a bunch of the things that 206 00:17:22,960 --> 00:17:27,200 engineers need to do on a day-to-day basis and to like, you know, eight API calls or something. 207 00:17:27,880 --> 00:17:32,980 Um, I think we take that for granted. And as we look to other platforms, even Google, for example, 208 00:17:33,660 --> 00:17:38,680 um, and the fact that they don't have that sort of interaction, um, it makes it really difficult 209 00:17:38,680 --> 00:17:43,300 to do some of the things that we've done the last couple of years with compliance and security and 210 00:17:43,300 --> 00:17:49,720 stuff, uh, because we don't have that, uh, that sort of front facing, uh, API proxy to everything, 211 00:17:49,720 --> 00:17:54,780 uh, underneath. What about that backend too? Cause I mean, you, you, I think started, 212 00:17:54,840 --> 00:17:59,540 as you mentioned running a platform team. So you also dealt with the good and the bad of trying to 213 00:17:59,540 --> 00:18:04,660 roll updates out to a multi-tenant form and things like that. Did that also mess with your 214 00:18:04,660 --> 00:18:08,240 expectations moving forward of what it, what does it mean? Now, hopefully if you're running something 215 00:18:08,240 --> 00:18:13,240 as a service entirely, you don't even see the muck of updating operating systems and things like that. 216 00:18:13,240 --> 00:18:17,400 But assuming you're not entirely in that world for quite a while, has that messed with your 217 00:18:17,400 --> 00:18:19,660 expectations of what you want a platform to do for you? 218 00:18:19,720 --> 00:18:24,340 Yeah, it has. And I think actually now that we're starting to assess, um, other things like 219 00:18:24,340 --> 00:18:30,340 Kubernetes or, you know, data service offerings and things, we keep finding ourselves, um, comparing 220 00:18:30,340 --> 00:18:35,720 everything, I think subconsciously to CF. It's like, oh, there was 3000 people and we're running it with 221 00:18:35,720 --> 00:18:42,240 a team of eight and, uh, we do updates and upgrades during the day when zero downtime. Um, and so now as 222 00:18:42,240 --> 00:18:45,560 we're starting to compare to other things, we're like, well, it doesn't do this and it doesn't do that. 223 00:18:45,560 --> 00:18:51,140 Or, uh, you know, we could do this, but we'd need like, you know, 15 SRE teams. And, and so, yeah, 224 00:18:51,140 --> 00:18:57,080 it is jading. It's jading sort of our assessment of everything else, uh, to some degree, at least 225 00:18:57,080 --> 00:19:02,460 me internally. Um, I know that we have some folks that are like, oh yeah, we, we should have an SRE 226 00:19:02,460 --> 00:19:07,860 team on every team. But, um, I don't feel that way personally, but, um, our platform team has been 227 00:19:07,860 --> 00:19:13,940 really good. Um, the fact that they can upgrade, you know, OSs and VMs and stuff without really even 228 00:19:13,940 --> 00:19:19,000 having to tell us most of the time is, is pretty incredible. Um, but yeah, I, yeah, the, the, the 229 00:19:19,000 --> 00:19:23,000 zero downtime, you know, during the day upgrades, that was a first for us at Home Depot. That was 230 00:19:23,000 --> 00:19:27,760 not something that we did before. Uh, you know, you were mentioning, uh, you know, uh, hammers and 231 00:19:27,760 --> 00:19:32,780 fixing second day delivery, uh, which, which sounds nice. So, you know, I've been, I've been trying to 232 00:19:32,780 --> 00:19:38,700 figure out like, uh, linking those two things together. So you do, you do like, uh, uh, a 233 00:19:38,700 --> 00:19:43,580 zero, you know, zero downtime upgrade or, or whatever you might pick as sort of a platform 234 00:19:43,580 --> 00:19:50,460 function you provide. And how do you like figure out how that links up to like fixing two day 235 00:19:50,460 --> 00:19:56,840 delivery? And then further on, how do you figure out like why two day delivery matters? I mean, 236 00:19:57,120 --> 00:20:02,360 intuitively it makes sense, but that's, that's kind of like a corny way of putting it. 237 00:20:02,780 --> 00:20:08,520 But to put it from another angle, like I've, I, I keep waiting for like, I, you know, in 238 00:20:08,520 --> 00:20:14,440 IT, we have this phrase, the business. And, uh, I hear very few stories about how the business 239 00:20:14,440 --> 00:20:20,800 is like, Whoa, we've got this crazy IT function now. So let's just have them like do things 240 00:20:20,800 --> 00:20:26,300 and we're not going to harass them about how much it costs all the time. Like, like we're 241 00:20:26,300 --> 00:20:31,080 just more going to focus on like, we want to do this new type of business. And so sure, we 242 00:20:31,080 --> 00:20:34,940 don't want to overpay for it, but like more of what we want to do is figure out how to provide 243 00:20:34,940 --> 00:20:40,200 this new thing. Right. And, uh, I don't know. So that I'm really avoiding like cliche terms, 244 00:20:40,200 --> 00:20:46,540 like IT business alignment or ROI or all that stuff. But like, I'm sort of realizing the, 245 00:20:46,540 --> 00:20:54,440 the spreadsheet driven, just like cost accounting way of doing like a business IT alignment, 246 00:20:54,440 --> 00:21:00,100 like that never really goes anywhere. Like it's maybe it works for like desktop operating 247 00:21:00,100 --> 00:21:04,440 systems, but for everything else, it doesn't seem very useful. So how, how, uh, when it 248 00:21:04,440 --> 00:21:08,680 comes to the, you know, selling more hammers and the two day shipping, like, what does that 249 00:21:08,680 --> 00:21:13,800 look like when the business people and the, the nerd people get together and try to work 250 00:21:13,800 --> 00:21:14,500 together on it? 251 00:21:16,300 --> 00:21:20,020 Yeah, I would, I would say, I would love, I would love to say that we've figured that 252 00:21:20,020 --> 00:21:26,440 out. Uh, the, so I think recently a big change for us is we just hired, um, so we have our 253 00:21:26,440 --> 00:21:31,720 own, uh, product organization now. So we just hired our first chief product officer. Um, and 254 00:21:31,720 --> 00:21:36,040 there is a group that's being built out. Um, and I think there's been a couple of big 255 00:21:36,040 --> 00:21:42,500 initiatives that have been embraced and funded, um, that, uh, one of those being sort of common 256 00:21:42,500 --> 00:21:48,700 services where we're trying to take, you know, 14 or 15, uh, large domains from the retail 257 00:21:48,700 --> 00:21:55,420 experience and sort of funding those, um, as, uh, you know, sort of ongoing, uh, things that 258 00:21:55,420 --> 00:22:00,020 we should always be caring about and in supply chain, uh, Hey, you know, we, we need these 259 00:22:00,020 --> 00:22:04,700 set of features, right. And then how do we know whether or not, um, we're, we're making 260 00:22:04,700 --> 00:22:09,620 money or we're saving money by doing them. Um, I can say from our part, what we're trying 261 00:22:09,620 --> 00:22:15,220 to do is be much more data driven. So we have all of these, you know, tools, um, uh, we have 262 00:22:15,220 --> 00:22:19,500 all these tools, right. So we have tracker and GitHub and cloud foundry and we have pipelines 263 00:22:19,500 --> 00:22:24,680 and it's, um, and, uh, another book that I read recently accelerate, or I guess not so 264 00:22:24,680 --> 00:22:28,700 recently, but you know, there's, there's some really good meaty things in there, but, um, 265 00:22:28,700 --> 00:22:32,380 it's like, how do you know if it's, if it's true? And so I think one of the big things, 266 00:22:32,380 --> 00:22:36,220 one of the big things that we're, that I'm, you know, sort of driving this year is 267 00:22:36,220 --> 00:22:40,380 like, Hey, let's be a much more data driven organization. So we have all these metrics 268 00:22:40,380 --> 00:22:46,360 and rich data, but how do we know if pushing more frequently has a positive impact on our 269 00:22:46,360 --> 00:22:51,240 product teams? Yeah. And so, um, so we're working with our product organization. Um, they 270 00:22:51,240 --> 00:22:55,520 have a bunch of things that they care about. That's different philosophy of like, okay, ours 271 00:22:55,520 --> 00:23:01,080 where it's what I call like TDD for product. Um, it's, it's, uh, they sort of come up with 272 00:23:01,080 --> 00:23:04,680 these things or like, Hey, we want to move some needle by X percent or some, you know, 273 00:23:04,720 --> 00:23:09,180 Y value. Um, and we're going to release some things and see if that needle moves. And so 274 00:23:09,180 --> 00:23:14,480 I think that's really interesting. And my theory is that when we pair that up with, um, sort 275 00:23:14,480 --> 00:23:19,140 of these rich metrics around how frequently teams go to production, um, or how, you know, 276 00:23:19,140 --> 00:23:24,700 uh, how many changes they go that don't, that don't cause problems that those have a positive 277 00:23:24,700 --> 00:23:29,500 impact on product teams being able to move their needles. So, um, so that's the theory. 278 00:23:29,500 --> 00:23:33,920 We'll see. Um, there's a lot of work to do there and we're just starting to, to kick some 279 00:23:33,920 --> 00:23:37,460 of that off. But, uh, I think that's an interesting case study for us. 280 00:23:37,600 --> 00:23:41,980 Yeah. Yeah. That'd be fun to see how that pans out. Like I think, uh, to, to cite another 281 00:23:41,980 --> 00:23:46,560 book. So I've been reading, um, I forget what it's called the flow framework or something. 282 00:23:46,560 --> 00:23:51,400 This, the new book from, uh, Mick Kirsten or Kirsten, I always forget how to say his name. 283 00:23:51,760 --> 00:23:55,840 Uh, who's, who's like the CEO of task top, but he's done all sorts of other stuff. And, 284 00:23:55,840 --> 00:24:02,180 uh, you know, he's trying to figure out essentially like how to, how to apply like lean thinking 285 00:24:02,180 --> 00:24:08,340 and value stream maps to the entire process of a software driven business, um, more or less. 286 00:24:08,340 --> 00:24:13,880 And, uh, he's got this little like, um, uh, interludes of, of lucky him being at like 287 00:24:13,880 --> 00:24:18,920 the BMW plant and seeing how they make cars and everything. Uh, which is fun if you're 288 00:24:18,920 --> 00:24:24,580 into that kind of, uh, interstitial stories, but it towards the end of the book, which I'm 289 00:24:24,580 --> 00:24:28,580 not quite finished with. He, he finally makes this interesting point about thinking about value 290 00:24:28,580 --> 00:24:35,760 streams where like in manufacturing, uh, doing a value stream is pretty straightforward because 291 00:24:35,760 --> 00:24:42,240 I guess you literally have a line, right? So it's a very, like, it's a very straight, uh, 292 00:24:42,280 --> 00:24:47,240 series of things. Now I'm sure there's, there's some nuance, but there's even more nuance. And 293 00:24:47,240 --> 00:24:51,640 what he does, what he says is he realizes that if you look at all the activities to get software 294 00:24:51,640 --> 00:24:55,820 out of the door, and I don't know if it's him, it's like two other people he was working with 295 00:24:55,820 --> 00:25:01,700 kind of brought this to him is that it's actually just like a network of things. And so like figuring 296 00:25:01,700 --> 00:25:07,460 out how to not only like optimize the way a network flows, but I think back to the point I was meandering 297 00:25:07,460 --> 00:25:15,380 about like how to find causation in a network can be very difficult. Right. And so it's, yes. And, 298 00:25:15,380 --> 00:25:20,940 and so like, it's, it's making me think that's part of the issue we always have is like intuitively, 299 00:25:20,940 --> 00:25:27,340 we know it'd be better to ship software more frequently, but then like, if you do the quarterly, 300 00:25:27,340 --> 00:25:32,040 like, you know, how many hammers did we sell? It's it, I don't know how you would do it, 301 00:25:32,040 --> 00:25:37,420 but it seems like it might be hard to say like, well, we ship software once a day for this last 302 00:25:37,420 --> 00:25:44,980 quarter. And therefore we sold 50% more hammers like, and, and like, and, and maybe wanting to 303 00:25:44,980 --> 00:25:50,040 answer that question is like the wrong question in the first place, but I'm pretty sure it's somewhere 304 00:25:50,040 --> 00:25:56,980 in there, right? Like somehow that the time and money spent on increasing releases has to link back 305 00:25:56,980 --> 00:26:01,020 up to something like that. And I have no idea how that happens in the real world. 306 00:26:01,360 --> 00:26:06,580 Yeah. I don't think it does. I think I like that answer. Move on to the next quandary. 307 00:26:08,240 --> 00:26:12,440 Yeah, exactly. It's a, yeah, I don't, I don't know if anyone's figured that out yet, but I think 308 00:26:12,440 --> 00:26:18,600 ideally, I think theoretically, if you break down these metrics and they, they sort of create a tree 309 00:26:18,600 --> 00:26:23,260 of metrics. And so I think as long as the app teams are able to move needles at their level that 310 00:26:23,260 --> 00:26:28,760 move needles above them, but it's all theory. Um, and I think, uh, I think the more important thing 311 00:26:28,760 --> 00:26:34,400 is capturing, um, what doesn't work. Right. And I think if you're doing these fast feedback loops, 312 00:26:34,400 --> 00:26:40,480 I think what we need to celebrate more often is the things that we didn't invest in that we learned 313 00:26:40,480 --> 00:26:46,580 was the wrong thing to do. And I think that, that bringing that to light is as valuable or more valuable. 314 00:26:46,580 --> 00:26:51,400 And I don't think we put enough focus on that. So, um, I think there's something there, um, and, 315 00:26:51,400 --> 00:26:55,960 and celebrating that sometimes. Yeah. No, you're reminding me. That's another little thing I had 316 00:26:55,960 --> 00:27:01,260 squirreled away on this topic is, uh, and, and I liked the way you put it. Like if it's, if it's 317 00:27:01,260 --> 00:27:08,400 impossible to show positive causation or not impossible, if it's not, if it's not profitable, 318 00:27:08,400 --> 00:27:16,240 so to speak, right. It's, it's not worth your time. Uh, maybe something good to do is, is to avoid 319 00:27:16,240 --> 00:27:22,400 things blowing up. Right. Which is, is kind of like a cynical way of looking at it. But I always 320 00:27:22,400 --> 00:27:27,660 thought that was one of the better, like, uh, sort of rules of thumb from the, the black Swan guy, 321 00:27:27,740 --> 00:27:33,360 Nassim Taleb, which is like, success is great, but what's really going to cause you to not succeed 322 00:27:33,360 --> 00:27:40,780 is blowing up. So like, if you can avoid over time, if you can avoid blowing up, you'll be 323 00:27:40,780 --> 00:27:48,380 successful, which, which, um, I, you know, which that, I think that's, that is a, again, easy and 324 00:27:48,380 --> 00:27:53,320 hard is the wrong word, but that's a much more profitable exercise to go through. Right. And that's 325 00:27:53,320 --> 00:27:57,240 why having stability and all these things is good. And not that you shouldn't try the other thing, 326 00:27:57,240 --> 00:28:04,800 but it definitely, uh, it has a clear effect. Yeah. Not blowing up. What's the, uh, I mean, 327 00:28:04,800 --> 00:28:09,300 so in that, what's the hardest part today about being, getting to production, Tony, what we struggle 328 00:28:09,300 --> 00:28:14,420 with is, uh, you know, honestly, just this whole socks thing, man, if you, if you got rid of socks 329 00:28:14,420 --> 00:28:22,820 and, and, uh, PCI and compliance, you know, or, um, uh, or, or found some ways to, to, I think, 330 00:28:22,900 --> 00:28:28,020 uh, align your company differently so that those things aren't as big of an issue. But, um, yeah, 331 00:28:28,020 --> 00:28:32,500 I think most of it is just us kind of getting in our own way. We have a lot of process and, and, uh, 332 00:28:32,780 --> 00:28:36,900 in regulatory things and, you know, that's, there's a finance group and then there's a, 333 00:28:37,060 --> 00:28:40,660 there's an audit group and then there's, you know, a technology group and a security group 334 00:28:40,660 --> 00:28:45,280 and they're all getting pulled in different directions. Right. And it's, uh, how do you kind 335 00:28:45,280 --> 00:28:50,460 of get them all on the same page that, Hey, our goal is to, you know, uh, release software more 336 00:28:50,460 --> 00:28:52,880 frequently so that we can figure out what works and doesn't work 337 00:28:52,900 --> 00:28:56,960 for our business faster. So, um, getting everyone on the same page about that mission, 338 00:28:56,960 --> 00:29:00,820 I think is, is the hardest part. You know, we, we, we kind of scheduled doing this, 339 00:29:00,920 --> 00:29:05,700 this talking when we're thinking about, uh, SpringOne platform talks and various stuff 340 00:29:05,700 --> 00:29:09,800 that y'all have going on. So y'all have been working on that problem, uh, of automating 341 00:29:09,800 --> 00:29:12,960 compliance and, uh, tell us about that. What's, what's that looking like? 342 00:29:13,240 --> 00:29:16,580 Yeah, I think, um, we thought it was pretty cool at the time. So a couple of years ago, 343 00:29:16,580 --> 00:29:20,600 uh, and it's, we've been iterating on it for a while. Um, what we did, I mean, 344 00:29:20,600 --> 00:29:24,660 so I think one of the things that we do differently, um, well, I'll, I'll, I'll just 345 00:29:24,660 --> 00:29:29,540 be transparent. So we, so we had an audit internal audit. Um, and it turns out that 346 00:29:29,540 --> 00:29:32,960 our, our controls were, you know, just do the right thing, right? Have it, have a change 347 00:29:32,960 --> 00:29:37,660 record for everything. And it turns out that, um, when you just do that, there's a percentage 348 00:29:37,660 --> 00:29:42,120 of your folks that don't do it. So, um, so it turns out that, uh, you know, we had a handful 349 00:29:42,120 --> 00:29:47,840 of teams that were not doing their change records. And so, um, you know, our, our leadership 350 00:29:47,840 --> 00:29:53,160 sort of removed everyone from production. So for us at Home Depot, we had people as space 351 00:29:53,160 --> 00:29:57,920 developers in production cloud foundry. Um, so we, you know, we didn't do anything to block 352 00:29:57,920 --> 00:30:02,400 their production access. And so we had teams, you know, pushing things to production without 353 00:30:02,400 --> 00:30:06,880 change records. And, and so we had to remove everyone from space developer, which caused 354 00:30:06,880 --> 00:30:12,440 a bunch of problems. We had issues around GitHub access and cloud foundry access. And they 355 00:30:12,440 --> 00:30:16,320 were like, Hey, if you're a developer and you can write code and cloud and GitHub, then you 356 00:30:16,320 --> 00:30:20,400 can't push your own code and cloud foundry. Um, I was like, well, you know, the whole DevOps 357 00:30:20,400 --> 00:30:25,680 thing is to, to write code and deploy code. Right? So what we did is we came up with this 358 00:30:25,680 --> 00:30:30,720 thing. Um, we called it locksmith. We put it, um, essentially what we did is we proxied 359 00:30:30,720 --> 00:30:37,440 all of the requests coming into the cloud foundry API and we allowed teams to do whatever they 360 00:30:37,440 --> 00:30:43,320 want, as long as it wasn't, you know, something that would change code. And then, so for those API 361 00:30:43,320 --> 00:30:49,400 calls, we would go through and make a bunch of service calls to make people feel comfortable 362 00:30:49,400 --> 00:30:54,520 that they had done all the right things. So we would look up change records on service. Now, 363 00:30:54,520 --> 00:30:58,120 uh, we would see if they've scanned their code and tested and a bunch of stuff. 364 00:30:58,120 --> 00:31:04,520 Um, and that worked, um, uh, except that we still had an issue with this sort of segregation 365 00:31:04,520 --> 00:31:08,840 of duties concern. Um, have you guys heard of segregation of duties, by the way, is that 366 00:31:08,840 --> 00:31:13,320 like a thing? Any, Oh yeah. Yep. All right. So I just want to make sure it wasn't just us, 367 00:31:13,320 --> 00:31:18,600 but, um, so to, to get around segregation of duties, what we did is we said, Hey, like, um, 368 00:31:18,600 --> 00:31:24,120 it doesn't really matter who pushes the code as long as they can't do it themselves. And so instead 369 00:31:24,120 --> 00:31:29,080 of, uh, separating, um, for us internally, the people that write code from the people that push 370 00:31:29,080 --> 00:31:35,720 code, uh, we said, Hey, like, uh, anyone, any code that gets deployed has to sort of be approved by 371 00:31:35,720 --> 00:31:41,160 two people. And if it is, then, then you can't go to production by yourself. And so we redefined 372 00:31:41,160 --> 00:31:45,960 segregation of duties for us internally. So, um, so essentially what this locksmithing 373 00:31:45,960 --> 00:31:51,240 does is it, is it looks at the artifact being pushed, make sure that multiple people have signed 374 00:31:51,240 --> 00:31:55,400 off on it. Um, so that's how we got around segregation of duties. Um, it's essentially 375 00:31:55,400 --> 00:32:02,200 a route service for the cloud foundry API. Um, if that makes sense. Um, but, and we've seen, 376 00:32:02,200 --> 00:32:07,240 and so now everyone has production access again in cloud foundry. So everyone's back to being space 377 00:32:07,240 --> 00:32:12,280 developers, um, in production, which is, I think, I think kind of rare, but yeah, that's, that's 378 00:32:12,280 --> 00:32:17,560 interesting. I mean, I mean, true to the description, you, uh, you, you sort of automated 379 00:32:17,560 --> 00:32:26,440 the enforcement of compliance, which, which, which, which seems, uh, seems like it would be better than 380 00:32:26,440 --> 00:32:31,960 sort of, uh, well, as you were saying, it's better than having sort of procedure that people need to 381 00:32:31,960 --> 00:32:36,120 follow. I mean, I guess, I guess you just lock down the thing and don't, don't let them do it, 382 00:32:36,120 --> 00:32:42,280 but like, it's probably even a richer source of, uh, compliance information, hopefully, and less 383 00:32:42,280 --> 00:32:47,560 tedious, which, and I was just thinking it's a, that's a great example of how like, uh, software 384 00:32:47,560 --> 00:32:51,800 is a network instead of a factory line, right? Like having to go out to service now to check 385 00:32:51,800 --> 00:32:56,360 something and look at all these other things. There's like multiple, like spaghetti bowls that 386 00:32:56,360 --> 00:33:01,880 you have to go dig around in and not just, uh, not just sort of like a compliance person sitting 387 00:33:01,880 --> 00:33:07,000 there on the line. Yeah. I mean, me personally, I'd rather us not have any of that and just, 388 00:33:07,000 --> 00:33:12,520 you know, hold people accountable. And, uh, you know, it, the, the joke I make is if you get audited 389 00:33:12,520 --> 00:33:17,800 and there's no one doing anything wrong, do you need it controls? And then, okay, this is what you 390 00:33:17,800 --> 00:33:23,240 should do and shouldn't do. Um, I think the problem is like, as a big company, what's the right mix 391 00:33:23,240 --> 00:33:28,520 of, uh, like, Hey, we're going to prevent you from doing a couple of things, uh, to show that we're not 392 00:33:28,520 --> 00:33:34,280 negligent. But, um, you know, ideally I would just, I would wish that we could just kind of, uh, hold 393 00:33:34,280 --> 00:33:38,680 people accountable so that you need less process, but that's hard to do in a big company. 394 00:33:38,840 --> 00:33:43,720 Yeah. Yeah. Yeah. There's probably some chart that shows a size of company and amount of process 395 00:33:43,720 --> 00:33:49,880 required. You know, years ago when you presented at different conferences, you talk about apps you've 396 00:33:49,880 --> 00:33:53,940 got in prod or how many, how much you're pushing. So even amongst all this, is there a stat now that 397 00:33:53,940 --> 00:33:59,080 you're proud of? Like even with all the craziness of pushing to prod and using different clouds, 398 00:33:59,180 --> 00:34:01,800 what's that thing that is celebrated internally right now? 399 00:34:01,940 --> 00:34:08,420 Yeah. So I think there's a few things. So, um, I think in production, you know, we're doing, um, uh, 400 00:34:08,420 --> 00:34:19,380 we do 17 billion HTTP. Um, and you know, we're still around 400 to 500 production changes a week, um, 401 00:34:19,380 --> 00:34:25,460 in production and of those, the fail rate. So, so now with locksmith, we can see, uh, the code changes 402 00:34:25,460 --> 00:34:30,780 that are people making. Is it, is it based on a change or is it based on an incident? Um, and the changes 403 00:34:30,780 --> 00:34:37,720 made from incidents is that, uh, less than 1%. So, um, if you look at those, you know, 400 to 500 404 00:34:37,720 --> 00:34:49,000 production changes each, um, 0.6% of those are due to incidents. So, um, yeah, so there's, uh, um, 405 00:34:49,000 --> 00:34:50,480 some really cool things going on. 406 00:34:50,480 --> 00:34:54,100 And you, uh, you garbled out there a little bit. How much, what did you say at first? How many 407 00:34:54,100 --> 00:34:55,720 deploys I think you do? 408 00:34:56,440 --> 00:35:02,860 Uh, we do four, yeah, 400, yeah, 400 to 500 a week in production. Yep. And the other number I 409 00:35:02,860 --> 00:35:08,020 think was API requests, right? I think you were saying, yeah, there's 17 billion, uh, service 410 00:35:08,020 --> 00:35:13,700 requests on each month and production. Those are some big numbers. What, what, like, what are, 411 00:35:13,980 --> 00:35:18,680 what are those? Like when it comes to, like, when you're, when you're deploying all those times a 412 00:35:18,680 --> 00:35:22,380 week, like what are you actually deploying and what are those calls for? Like, what are, 413 00:35:22,920 --> 00:35:27,480 I don't know what, what happens that frequently? Yeah, that's a good question. I don't think we 414 00:35:27,480 --> 00:35:34,620 know anymore. Uh, I mean, you know, four years ago I used to know every developer that was on the 415 00:35:34,620 --> 00:35:39,960 platform and, you know, would talk to them, you know, for two to three hours a day. And, uh, I mean, 416 00:35:40,020 --> 00:35:46,920 today it's kind of taken on a shape of its own. Um, uh, our Slack community, um, you know, 417 00:35:46,920 --> 00:35:52,100 the cloud foundry community that we have on Slack internally is, you know, by far probably one of the 418 00:35:52,100 --> 00:35:56,460 most, the busiest, except for the one that tells you where free food is. I think that one might be 419 00:35:56,460 --> 00:36:01,900 beating us, but, um, the, uh, yeah, the Slack community has been great. I mean, we, our team 420 00:36:01,900 --> 00:36:07,840 doesn't really even handle a lot of the sort of newbie questions or, um, you know, some of the, 421 00:36:07,840 --> 00:36:11,380 the first level of support questions. Our community does a lot of that for us. 422 00:36:11,380 --> 00:36:17,380 So, so finally, um, you know, like you're saying you have a, a chief product officer and, uh, you're, 423 00:36:17,460 --> 00:36:21,600 you're, you're helping make the developers lives easier and so forth and so on. 424 00:36:21,600 --> 00:36:28,100 All the great things like how, uh, like, like on your, in your group, how do you like product 425 00:36:28,100 --> 00:36:32,980 manage what you do? Cause I assume that's part of it, right? In the same way that a, uh, a chief 426 00:36:32,980 --> 00:36:38,060 product officer of, of Home Depot comes up with what their customers and I would imagine suppliers 427 00:36:38,060 --> 00:36:41,840 and other people, but you know, you've got a product management going on, uh, in the business 428 00:36:41,840 --> 00:36:45,780 side, but do, do y'all do that, uh, for the platform that you run? 429 00:36:46,620 --> 00:36:51,260 Yeah. So that's what's new. So that's sort of what I'm heading up. So we just hired a couple of product 430 00:36:51,260 --> 00:36:57,180 folks from this product group. Um, and we're looking at our engineers as users. So we're starting 431 00:36:57,180 --> 00:37:03,120 to do formal, uh, sort of UX research around that. Um, we're creating journey maps for the different 432 00:37:03,120 --> 00:37:08,120 demographics. So another big thing that we're seeing is whether or not the teams are in scope 433 00:37:08,120 --> 00:37:14,200 for socks or PCI, that seems to be a big factor into sort of their workflow and journey map. So, 434 00:37:14,760 --> 00:37:19,800 so we're doing a lot of formal research around, uh, what the different teams need and what their 435 00:37:19,800 --> 00:37:25,080 pain points are and their processes and stuff. And that's interesting to have a, uh, uh, speaking 436 00:37:25,080 --> 00:37:32,780 of segregations, like a segregation of compliance complexity, right? Like this, this profile of things 437 00:37:32,780 --> 00:37:38,300 needs to deal with all this stuff and this other one, you know, just do whatever, I mean, responsibly, 438 00:37:38,300 --> 00:37:43,260 but, but that is, I mean, that's a, that's an interesting way of like dividing things up. So how many, 439 00:37:43,260 --> 00:37:49,900 how many, uh, like teams do you have working on apps? Oh, that is a good question. So I think, 440 00:37:49,900 --> 00:37:55,500 um, um, we did go through a big hiring spree that you guys may have heard of. I think we ended up hiring 441 00:37:55,500 --> 00:38:03,880 900, uh, engineers last year. So I think, you know, um, for, for aprons as we call them associates, 442 00:38:03,880 --> 00:38:10,560 I think we're up to, um, and I think there's about 2,700 of them on the platform. So, um, 443 00:38:10,560 --> 00:38:15,520 and we've had a couple of acquisitions and stuff, but, uh, yeah, I mean, it's a, it's, it's a large 444 00:38:15,520 --> 00:38:21,060 group. Um, uh, I don't remember what was the question again? Oh, just, just like, you know, 445 00:38:21,060 --> 00:38:25,960 just to get a sense of like, so you have all the, the, uh, the astounding numbers of, 446 00:38:25,960 --> 00:38:32,900 of kind of the, uh, velocity or movement each week, right. And API calls and deploys. And so like, 447 00:38:33,320 --> 00:38:38,580 I mean, how many like applications and therefore how many teams and sort of how many people that 448 00:38:38,580 --> 00:38:45,200 equals, right? Like it's, it's always interesting, um, to think about how much, as you were kind of 449 00:38:45,200 --> 00:38:50,960 joking about earlier, how much software development large organizations actually do on their own. 450 00:38:51,060 --> 00:38:55,840 It's sort of like, right. I always, I was still, after everything that's been going on in the last 451 00:38:55,840 --> 00:39:00,920 five years, like it always seems like the, uh, the sort of hidden pool of software developers out 452 00:39:00,920 --> 00:39:06,840 there. Uh, so I'm always curious for a sizing of them. Yeah. So I, I don't know the number of, 453 00:39:06,840 --> 00:39:17,240 you know, in terms of CF, uh, things we have 4,000, over 4,000, um, services and production now 454 00:39:17,240 --> 00:39:23,400 on CF. Um, and then from teams, um, you know, we, we see some services on there that haven't, 455 00:39:24,080 --> 00:39:29,900 but then we still see new things being added all the time. So, um, yeah, I don't know the number of 456 00:39:29,900 --> 00:39:36,800 teams, but, uh, we're up to 4,000 apps or services. Yeah. Um, that's, that's a lot from like a couple 457 00:39:36,800 --> 00:39:43,240 of years ago, if I remember. Internally, um, was that, um, in terms of, of adoption, 458 00:39:43,240 --> 00:39:51,300 we saw nearly double the adoption in the last year. So, um, so that hockey, that, that little curve, 459 00:39:51,300 --> 00:39:54,940 um, went up a lot last year. It's starting to smooth out a little bit. 460 00:39:55,660 --> 00:40:03,140 So, so like, what, uh, like, like, what do you see being kind of next year? Like what are the, 461 00:40:03,140 --> 00:40:07,060 what's the next set of things that you think you'll be working on? Like, do you, do you get the chance 462 00:40:07,060 --> 00:40:12,320 to sort of like dream up things that the platform team might want to do? Or are you still like too 463 00:40:12,320 --> 00:40:18,360 busy, uh, you know, making sure all the pipes aren't bursting, keeping? Yeah. Um, so our team, 464 00:40:18,420 --> 00:40:23,900 I think there's a lot of, uh, there right now, uh, one of the big things that we're looking to help 465 00:40:23,900 --> 00:40:29,380 out with is, um, this sort of hybrid cloud thing, right? Which I'm not entirely sure what that means 466 00:40:29,380 --> 00:40:34,880 still, but, um, but we do know that we have some things that we want to run, you know, in Google or 467 00:40:34,880 --> 00:40:40,260 Microsoft, and we have something that we want to run on prem for lots of various reasons. Uh, but we're not 468 00:40:40,400 --> 00:40:47,360 good at, um, sort of that connectivity either from a networking perspective, um, openly. So I think, 469 00:40:47,500 --> 00:40:53,000 uh, or seamlessly, I should say. So I think there's some work to do there. Um, I would say that, um, 470 00:40:54,000 --> 00:40:59,280 to, to sum it up, I would say that we've sort of been lucky that we have this big firewall around 471 00:40:59,280 --> 00:41:04,420 everything that we do on prem. And so I think that allows us to, um, take certain liberties, 472 00:41:04,420 --> 00:41:09,160 you know, or like, Hey, we're behind a firewall, so we don't have to be, you know, really stringent 473 00:41:09,160 --> 00:41:13,640 about this or that in terms of service to service stuff. Um, but as you go into the hybrid cloud 474 00:41:13,640 --> 00:41:18,760 world, um, and you're sort of going from on prem to the public and then back, uh, there's some, 475 00:41:18,900 --> 00:41:24,280 you have to think about those things. So, so API management and sort of, um, and security things 476 00:41:24,280 --> 00:41:29,600 are, and, and also being data driven or some of the, the big, probably two top two to three 477 00:41:29,600 --> 00:41:35,640 things that we're working on, uh, over the next year. Yeah. And that, that seems like a fruitful area 478 00:41:35,640 --> 00:41:44,580 of, uh, seeing if there is a, a, some, some good useful ways of doing like, you know, business to 479 00:41:44,580 --> 00:41:50,420 it value mapping stuff in the sense of kind of like you were saying is like, well, if, if everyone 480 00:41:50,420 --> 00:41:56,840 around here says we need to do hybrid cloud things, then they must have a reason or they must have a 481 00:41:56,840 --> 00:42:00,940 reason is the wrong way of putting it, but like, it will allow us to like explore the reasoning that 482 00:42:00,940 --> 00:42:07,060 comes behind, like, we must do this thing and, uh, maybe kind of surface up the type of analysis and 483 00:42:07,060 --> 00:42:11,740 sort of, uh, rating you would do for things. And then maybe you can apply that type of, of thinking 484 00:42:11,740 --> 00:42:16,660 backwards, but, uh, cause you know, at the end of the day, it might just be sort of like I was talking 485 00:42:16,660 --> 00:42:22,360 about earlier. It's just like an intuitive benefit that you just want that, uh, without it being sort 486 00:42:22,360 --> 00:42:29,480 of like excruciatingly analyzed with numbers. Right. Well, I think it's interesting is going 487 00:42:29,480 --> 00:42:33,800 back to that whole ROI thing. I think for us, we're looking at it slightly differently as well. 488 00:42:33,800 --> 00:42:39,740 Um, for me anyway, um, I definitely think there's a business side to that, you know, pushing two times 489 00:42:39,740 --> 00:42:43,720 more frequently to production, sells more hammers. But I think what we're interested in learning is, 490 00:42:43,720 --> 00:42:49,320 um, uh, Hey, like, does the tech stack that you choose, how does that impact or the size of your 491 00:42:49,320 --> 00:42:53,480 team? We're trying to look at is, you know, what is, uh, you know, if you're in, you know, 492 00:42:53,540 --> 00:42:59,020 app engine or CF or Kubernetes, how does that impact your team's ability to deliver? And unless you have 493 00:42:59,020 --> 00:43:04,160 a really good business case for choosing one technology over the other, let's be very data 494 00:43:04,160 --> 00:43:10,780 driven about the decisions that we make, uh, from team sizing and silos and, uh, and the tech stack 495 00:43:10,780 --> 00:43:16,020 choices and stuff. Yeah, no, that makes sense. That, that, that's, that seems like a good, uh, uh, 496 00:43:16,020 --> 00:43:20,480 is it a solve or a bomb? It seems like a good way to address like a, uh, data driven question, 497 00:43:20,480 --> 00:43:23,200 like, like finances is, uh, get some data. 498 00:43:25,500 --> 00:43:28,640 Right. Well, when you have a bunch of people in the room and they're like, Hey, we should 499 00:43:28,640 --> 00:43:33,080 go Kubernetes and we should go this. It's, you know, um, instead of just sort of debating 500 00:43:33,080 --> 00:43:37,020 I'd rather just like, well, let's just look at the data and let that decide. Um, I think 501 00:43:37,020 --> 00:43:38,460 that's, that will be more fruitful. 502 00:43:38,840 --> 00:43:43,580 Yeah, yeah, for sure. All right. Well, great. Well, uh, thanks for coming back on and, 503 00:43:43,620 --> 00:43:47,740 uh, uh, checking in on us going, going through stuff, or I guess we checked in on you, but 504 00:43:47,740 --> 00:43:52,960 whatever. Uh, if, uh, if people wanted to, uh, see what you're up to online or, you know, 505 00:43:52,960 --> 00:43:57,120 do you get a Twitter account or any other things you would point them at for a further study? 506 00:43:58,480 --> 00:44:04,020 Yeah. Yeah. Um, it's, it's mostly dad stuff, but, uh, you can check me on Twitter at, uh, 507 00:44:04,020 --> 00:44:10,900 epic nerd, um, on Twitter. So, and, uh, and LinkedIn, uh, just Anthony McCauley on LinkedIn. 508 00:44:11,420 --> 00:44:15,580 That would be great if you were just epic nerd in LinkedIn too. I think that would fit well. 509 00:44:16,080 --> 00:44:21,320 Just own the name. Yep. Yeah. Yeah. I think, you know, maybe, was it about five years ago 510 00:44:21,320 --> 00:44:26,780 that there was some like, uh, data absurdist ism in, in LinkedIn when they, uh, introduced 511 00:44:26,780 --> 00:44:31,260 endorsements and people would get endorsed for like, you know, hot dog eating and things 512 00:44:31,260 --> 00:44:36,740 like that. Yeah. I mean, I just endorsed you for mustard on, on carrots like an hour ago. 513 00:44:38,000 --> 00:44:39,680 Carrot condiment innovation. 514 00:44:42,800 --> 00:44:49,420 I, uh, yeah, those things were hilarious. I never, yeah. Yeah. I'm going to add on the 515 00:44:49,420 --> 00:44:54,600 Coté and Richard that I'm glad to both of yours. That's good. You know, when I, when I enter 516 00:44:54,600 --> 00:44:59,640 my, I don't know if it's third or fourth career as a, uh, consumer packaged goods and condiment, 517 00:44:59,640 --> 00:45:04,740 uh, marketeer, these are really going to pay off these, these endorsements. That's going 518 00:45:04,740 --> 00:45:08,600 to be good. That's right. And then, and then, uh, I might write you, I'm going to write one 519 00:45:08,600 --> 00:45:13,240 for you. Okay. I'm going to look forward to it. And are y'all, uh, are y'all still hiring 520 00:45:13,240 --> 00:45:19,280 or is 900 enough? Are you still trying to find trash cans for them all? No, no, no. We're 521 00:45:19,280 --> 00:45:24,020 definitely hiring. I think, uh, I think we're filling up in Atlanta, but they're still hiring, 522 00:45:24,020 --> 00:45:29,120 uh, folks, anyone interested in working in Austin, we still have, uh, some openings there for sure. 523 00:45:29,120 --> 00:45:34,020 Oh yes. That's a good, that's a good place to go work. All right. Well, great. Well, uh, thanks 524 00:45:34,020 --> 00:45:39,400 again. And, uh, as always, this has been Pivotal Conversations. If you want to, uh, I guess you're 525 00:45:39,400 --> 00:45:43,320 already listening to this one, but if you want to go check out some of the other episodes, if you go 526 00:45:43,320 --> 00:45:49,040 to soundcloud.com slash Pivotal Conversations, you can see them there. And about every Thursday, 527 00:45:49,040 --> 00:45:54,000 if I remember to click the publish button as I did not last week, but eventually did about 528 00:45:54,000 --> 00:45:59,660 every Thursday, uh, you can see the full show notes over at pivotal.io slash podcast. And we'll put 529 00:45:59,660 --> 00:46:04,580 some links to some other relevant things and, uh, and stuff Tony has done as in the past. And with 530 00:46:04,580 --> 00:46:07,400 that, we'll see everyone next time. Bye-bye.