1 00:00:00,000 --> 00:00:03,680 Hey, listeners, thanks for tuning in to another episode of Pivotal Conversations. 2 00:00:04,100 --> 00:00:09,040 This week, we talked to Brian Kroger, founder of Kessel Run, an effort inside the U.S. Air 3 00:00:09,040 --> 00:00:12,680 Force to revolutionize the way the Air Force builds software. 4 00:00:13,340 --> 00:00:17,340 Now, the thing that was really the theme of this episode was the recommit. 5 00:00:17,720 --> 00:00:22,380 We often hear about how folks on their digital transformation journeys, or whatever you want 6 00:00:22,380 --> 00:00:27,840 to call it, how getting that buy-in is so important up front and how they get started. 7 00:00:27,840 --> 00:00:32,960 But we don't always hear about what happens when the project has already started and a 8 00:00:32,960 --> 00:00:34,660 recommit is required. 9 00:00:40,080 --> 00:00:43,480 All right, here we are at CF Summit, Philadelphia. 10 00:00:44,400 --> 00:00:51,300 I'm with Captain Brian Kroger of Kessel Run fame. 11 00:00:52,300 --> 00:00:57,820 And so one of the questions that, you know, some of my observations from the event here 12 00:00:57,820 --> 00:01:04,200 have been some interesting kind of tactics that I'm seeing emerge. 13 00:01:04,200 --> 00:01:10,380 A couple of years ago, we had this sort of notion of the frozen middle start to kind of 14 00:01:10,380 --> 00:01:11,860 come out in some of our conversations. 15 00:01:12,660 --> 00:01:18,220 And just to unpack that quickly, that's where, okay, senior leadership is sort of bought in 16 00:01:18,220 --> 00:01:20,100 at a high level of like, we need to do things differently. 17 00:01:20,220 --> 00:01:22,120 We need to digitally transform, whatever you want to call it. 18 00:01:22,600 --> 00:01:27,480 At the ground level, developers are basically in revolt because they can't get the systems 19 00:01:27,480 --> 00:01:32,160 that they want to use, and they can't deploy fast enough, and everything is very painful 20 00:01:32,160 --> 00:01:32,600 for them. 21 00:01:33,160 --> 00:01:38,520 But then you have this group that's sort of between them, that's where things tend to get 22 00:01:38,520 --> 00:01:38,920 stuck. 23 00:01:39,400 --> 00:01:43,060 And I have a lot of empathy for those people because I'm in the middle of management myself. 24 00:01:43,060 --> 00:01:47,360 So I know what it's like where you're sort of, you're a player coach, you're, you still have 25 00:01:47,360 --> 00:01:52,380 to get work done, but you also have to manage teams and deal with reports and all that fun 26 00:01:52,380 --> 00:01:52,680 stuff. 27 00:01:53,420 --> 00:01:57,000 So that frozen middle is in a tough, tough spot. 28 00:01:57,820 --> 00:02:02,700 But here at this conference, there were some interesting things that, you know, the West 29 00:02:02,700 --> 00:02:08,940 Corp talk highlighted this notion of recommitting and like making sure to empower those senior 30 00:02:08,940 --> 00:02:11,980 leaders with the evidence that they need to recommit. 31 00:02:12,720 --> 00:02:20,080 So this is something that as a, as someone who's kind of like in a similar role, I thought 32 00:02:20,080 --> 00:02:24,540 that you might have some interesting perspectives on to continue to unpack and maybe some other 33 00:02:24,540 --> 00:02:30,960 tactics to share that you found in terms of, you know, navigating that, the, that middle 34 00:02:30,960 --> 00:02:36,740 layer between, you know, the ground level execution and the senior leadership who's, you know, 35 00:02:36,740 --> 00:02:41,340 come to sometimes sort of, you know, seagulling in and out of the, of the situation. 36 00:02:42,620 --> 00:02:43,020 Yeah. 37 00:02:43,180 --> 00:02:47,780 So I think the first thing I like to do here is I want to change the narrative a little 38 00:02:47,780 --> 00:02:47,940 bit. 39 00:02:48,000 --> 00:02:50,280 I don't love calling it the frozen middle. 40 00:02:51,140 --> 00:02:55,460 And we, we, we have very hierarchical organizations, none more so than mine. 41 00:02:56,720 --> 00:03:02,200 And oftentimes those people, the resistors exist in the middle of that hierarchy. 42 00:03:02,200 --> 00:03:06,560 And I think, you know, that's where the term comes from, but in reality, I found that those 43 00:03:06,560 --> 00:03:07,680 people exist all over. 44 00:03:07,800 --> 00:03:09,060 They exist lateral to me. 45 00:03:09,180 --> 00:03:10,340 They exist below me. 46 00:03:11,240 --> 00:03:12,960 And even in some of the senior leadership. 47 00:03:13,300 --> 00:03:17,080 And so I really like, I reread the book recently. 48 00:03:17,080 --> 00:03:19,560 I hadn't read it in a long time, Crossing the Chasm. 49 00:03:20,300 --> 00:03:24,740 And I like to think of that, you know, we think about that in terms of technology adoption, 50 00:03:24,740 --> 00:03:28,120 but there's also a way to think about that in terms of cultural adoption. 51 00:03:28,120 --> 00:03:33,500 And that what you're seeing, that resistance that you're seeing to this new culture that 52 00:03:33,500 --> 00:03:39,260 we're trying to bring about in digital transformation is you're getting to the point of the chasm and 53 00:03:39,260 --> 00:03:43,940 you've got these early and late majority adopters that are just across the chasm. 54 00:03:44,320 --> 00:03:45,720 They have a different level. 55 00:03:45,860 --> 00:03:52,080 In fact, more in the book, he talks about you shouldn't try to market to more than one 56 00:03:52,080 --> 00:03:58,080 group at once and that you should only market to that next group, that next set of adopters 57 00:03:58,080 --> 00:04:00,560 when you've met their expectations. 58 00:04:00,560 --> 00:04:03,700 And it's really important to understand what their expectations are, right? 59 00:04:03,760 --> 00:04:10,220 So for them to press the I believe button requires a very different set of expectations than some 60 00:04:10,220 --> 00:04:12,380 of your pioneers and your early adopters. 61 00:04:12,920 --> 00:04:17,580 And I think it's really important to remember that and not try to thaw the frozen middle 62 00:04:17,580 --> 00:04:23,600 so much as understand what expectations they have before they're going to push the I believe 63 00:04:23,600 --> 00:04:32,940 button and go meet those expectations and don't bother them or have them bother you while you're 64 00:04:32,940 --> 00:04:34,060 going about that process. 65 00:04:35,020 --> 00:04:40,880 I think it's unfair to both the change agent to expect them to thaw that person without having 66 00:04:40,880 --> 00:04:46,220 proven the things that need to be proven for that person to believe, nor is it fair for 67 00:04:46,220 --> 00:04:49,320 that person to press the I believe button before their expectations have been met. 68 00:04:49,440 --> 00:04:53,640 And I think we set up this whole weird dynamic where we force one side or the other to do 69 00:04:53,640 --> 00:04:54,760 unrealistic things. 70 00:04:55,520 --> 00:05:00,620 And like an example of that is, you know, like I can take a pioneering individual and put them 71 00:05:00,620 --> 00:05:04,860 on one product team and they can believe that, you know, after being on one product team and 72 00:05:04,860 --> 00:05:07,700 building one app that you can build a whole system of systems this way. 73 00:05:09,000 --> 00:05:12,000 We haven't even done our first app integration, but they believe, right? 74 00:05:12,240 --> 00:05:12,460 Okay. 75 00:05:12,960 --> 00:05:16,880 But then some of these people that we might refer to as the frozen middle, they want to 76 00:05:16,880 --> 00:05:21,940 see four or five, six applications integrated in prod, like doing some like system level 77 00:05:21,940 --> 00:05:26,420 workflow before they can really press the I believe button that we can now turn that into 78 00:05:26,420 --> 00:05:27,340 a system of systems. 79 00:05:27,340 --> 00:05:31,320 And then there's some really late adopters that until the whole thing's built, they're 80 00:05:31,320 --> 00:05:32,120 just not going to believe. 81 00:05:32,720 --> 00:05:38,360 And we waste a lot of time, effort, resources, I believe, trying to convince people too early. 82 00:05:38,660 --> 00:05:45,060 How does that play out in terms of, you know, maybe can you map this a little bit to the 83 00:05:45,060 --> 00:05:46,240 experience that you've had? 84 00:05:46,540 --> 00:05:50,880 But if you were to go back and look at your efforts, you know, where did you actually do 85 00:05:50,880 --> 00:05:56,180 that well in terms of, hey, you know, not waste the resources trying to convince the 86 00:05:56,180 --> 00:05:58,420 people who just really weren't going to be convinced at that time? 87 00:05:59,860 --> 00:06:03,360 And where did maybe you kind of like looking back and like, oh yeah, we probably shouldn't 88 00:06:03,360 --> 00:06:04,180 have done it that way. 89 00:06:05,560 --> 00:06:05,920 Yeah. 90 00:06:06,780 --> 00:06:14,180 One area where we nailed this is, and this was a senior leader that we found, but I would 91 00:06:14,180 --> 00:06:16,400 say among her peers, not everybody was in agreement. 92 00:06:16,840 --> 00:06:20,160 Lauren Knausenberger, who signed our continuous authority to operate. 93 00:06:21,800 --> 00:06:24,120 She saw the early work that we did. 94 00:06:24,300 --> 00:06:27,960 Some people think that we just magically jumped to this continuous authority to operate. 95 00:06:28,540 --> 00:06:33,040 We started out, you know, the first time around, it took us two months, which was a huge change, 96 00:06:33,100 --> 00:06:33,220 right? 97 00:06:33,240 --> 00:06:34,720 It used to take 13 months. 98 00:06:35,160 --> 00:06:36,780 We got it down to four weeks. 99 00:06:36,880 --> 00:06:37,900 We got it down to two weeks. 100 00:06:37,980 --> 00:06:39,000 We got it down to one week. 101 00:06:39,440 --> 00:06:43,520 And that's when she came on board and she saw what we had done, saw that progress and 102 00:06:43,520 --> 00:06:48,920 really believed that we had met all of the conditions to be able to have continuous authority 103 00:06:48,920 --> 00:06:49,460 to operate. 104 00:06:51,000 --> 00:06:53,760 And so that's one area where we got it right. 105 00:06:54,880 --> 00:06:59,760 I think I could name a bunch where we got it wrong. 106 00:07:00,640 --> 00:07:03,180 Very early on, the budgets one comes up. 107 00:07:03,240 --> 00:07:08,020 There were a lot of conversations about budgets and how do you prove that you're producing, 108 00:07:08,160 --> 00:07:10,820 you're giving the ROI to go back and ask for more money. 109 00:07:11,700 --> 00:07:13,980 It's a theme at every one of these conferences. 110 00:07:15,640 --> 00:07:17,320 We get our budgets from Congress. 111 00:07:18,940 --> 00:07:21,160 That's quite a bureaucratic process at times. 112 00:07:21,660 --> 00:07:25,960 And I think very early on, we were trying to convince them not only of a whole new way of 113 00:07:25,960 --> 00:07:29,780 doing software development, but a whole new way of doing budgeting and cost estimating. 114 00:07:30,500 --> 00:07:33,340 And we still hadn't really proved anything. 115 00:07:34,080 --> 00:07:35,780 At least that's of note to them, right? 116 00:07:35,820 --> 00:07:38,140 They deal with billions and billions of dollars. 117 00:07:39,040 --> 00:07:44,800 And, you know, even the tanker planner story, as great as that was, it didn't really sell 118 00:07:44,800 --> 00:07:46,120 the message we needed to sell. 119 00:07:46,540 --> 00:07:50,620 And so it wasn't until very recently that we started to make a lot more headway. 120 00:07:50,620 --> 00:07:57,500 And, hey, there's a new way to contract, to think about the way that we cost model these 121 00:07:57,500 --> 00:07:58,000 efforts. 122 00:07:59,620 --> 00:08:02,100 So two very different scenarios. 123 00:08:02,700 --> 00:08:05,120 We've gotten it wrong more often than right, unfortunately. 124 00:08:05,600 --> 00:08:09,680 I think that until recently, it wasn't something that I really realized. 125 00:08:09,820 --> 00:08:13,840 And something else that you mentioned that I think is really important, you talk about 126 00:08:13,840 --> 00:08:18,060 getting the senior leader buy-in, but I think sometimes the story gets recast. 127 00:08:18,420 --> 00:08:23,940 So as soon as these things get successful, all the people that were detractors are all 128 00:08:23,940 --> 00:08:27,480 of a sudden like the heroes of the story, which is OK. 129 00:08:27,620 --> 00:08:31,020 I love to let those people take credit if it means they're going to be our champions. 130 00:08:31,420 --> 00:08:35,640 But it does this unfortunate thing for all of the other innovators and change agents that 131 00:08:35,640 --> 00:08:42,020 are out there in that the story gets retold as, you know, a senior leader comes down and 132 00:08:42,020 --> 00:08:43,480 like they're the change agent. 133 00:08:43,600 --> 00:08:44,660 They start this thing. 134 00:08:44,780 --> 00:08:45,740 They sponsor it. 135 00:08:45,740 --> 00:08:49,620 You know, they get all the barriers out of the way, some of which is true. 136 00:08:49,960 --> 00:08:54,980 But the real truth is almost all these efforts that I've seen that are successful are truly 137 00:08:54,980 --> 00:08:59,700 grassroots with leadership sponsors and servant leaders. 138 00:09:00,040 --> 00:09:04,120 Usually they have really great champions that are truly servant leaders. 139 00:09:04,960 --> 00:09:11,500 And when we tell the story otherwise, everybody else is sitting around waiting for their sponsor 140 00:09:11,500 --> 00:09:13,480 to tell them, yes, you can go do this. 141 00:09:13,700 --> 00:09:14,580 Here's all the resources. 142 00:09:14,740 --> 00:09:19,120 And I'm going to get the barriers out of the way instead of just, you know, like seizing 143 00:09:19,120 --> 00:09:21,480 the day and going and doing the thing yourself. 144 00:09:21,880 --> 00:09:22,040 Yeah. 145 00:09:22,700 --> 00:09:29,100 Yeah, I think that that's, that is another theme that I've seen where both from a leadership 146 00:09:29,100 --> 00:09:33,540 message of like, you need to get out of the way of your people and empower them to come 147 00:09:33,540 --> 00:09:39,200 to you with the ideas that they're, they're ready to go do, not just go off into a cave 148 00:09:39,200 --> 00:09:41,580 or wherever you do your strategic planning. 149 00:09:41,920 --> 00:09:47,060 Cave seems like a really, like maybe with a, you know, like an evil layer in it or something. 150 00:09:47,320 --> 00:09:51,420 Anyways, you go in your cave for like, you know, two weeks or three months or whatever, however 151 00:09:51,420 --> 00:09:55,560 long it takes to come up with this big plan while everyone is sort of like eagerly waiting 152 00:09:55,560 --> 00:09:59,140 outside for you to drop this amazing PowerPoint on them. 153 00:10:00,660 --> 00:10:05,520 And how it's like, you know, that, that whole model needs to change just, you know, almost 154 00:10:05,520 --> 00:10:11,780 anywhere as like, well, the ideas should be bubbling up and, you know, to your point, 155 00:10:11,920 --> 00:10:18,580 the servant leaders, uh, have figured that out and are really more just like making sure 156 00:10:18,580 --> 00:10:24,400 that those ideas happen and that they have the air cover to course correct, uh, when things 157 00:10:24,400 --> 00:10:26,980 don't go exactly according to the initial plan. 158 00:10:27,420 --> 00:10:27,520 Yeah. 159 00:10:27,700 --> 00:10:31,360 And your, your senior leadership is going to be on that bell curve of adoption as well. 160 00:10:31,720 --> 00:10:36,100 So if you're really fortunate, you get the one that's like the pioneer on the far left 161 00:10:36,100 --> 00:10:39,680 side of that, um, that's willing to take on any crazy idea. 162 00:10:39,820 --> 00:10:46,540 But if you think that you need the leader to start the effort and then that leader is somebody 163 00:10:46,540 --> 00:10:53,420 that's like a, like early majority adopter, uh, you're going to have a really hard time 164 00:10:53,420 --> 00:10:55,420 getting your project sponsored to get kicked off. 165 00:10:55,780 --> 00:11:02,600 So I think, um, it's really important to understand where everybody in your organization falls on 166 00:11:02,600 --> 00:11:05,960 that adoption cycle in terms of culture and what you're going to have to prove because it 167 00:11:05,960 --> 00:11:06,720 might be the case. 168 00:11:06,800 --> 00:11:08,260 Cause this was the case with Kessel Run, right? 169 00:11:08,660 --> 00:11:12,200 We didn't get, um, it wasn't a go do. 170 00:11:12,480 --> 00:11:17,460 It wasn't, uh, there wasn't even a lot of buy-in in the early days and that didn't come 171 00:11:17,460 --> 00:11:19,680 until we proved some things, right? 172 00:11:19,700 --> 00:11:24,260 So it might be the case that you have to go out on your own completely grassroots, you 173 00:11:24,260 --> 00:11:29,420 know, side hustle project, uh, prove some things and then you'll get your leadership buy-in. 174 00:11:29,520 --> 00:11:31,220 The leadership buy-in has to come at some point. 175 00:11:31,280 --> 00:11:36,320 There's no doubt about it, but you, you shouldn't expect that it's going to come at the start 176 00:11:36,320 --> 00:11:38,160 of your project or before you start. 177 00:11:38,800 --> 00:11:40,940 Um, you might have to bootstrap it. 178 00:11:44,240 --> 00:11:49,960 So like, what about, what do you think about that notion of the kind of the recommit that 179 00:11:49,960 --> 00:11:50,960 needs to happen, right? 180 00:11:50,960 --> 00:11:56,920 Where we have intentions and we're constantly sort of setting these intentions, uh, but we're 181 00:11:56,920 --> 00:12:00,000 not like, that's different from a real commitment. 182 00:12:00,140 --> 00:12:05,660 Commitment is what you're actually doing, um, as opposed to what you plan to do. 183 00:12:06,100 --> 00:12:13,600 So, uh, you know, once you get an initial commitment and that initial, like you've got the buy-in 184 00:12:13,600 --> 00:12:18,960 and you've been side hustling to get to even just that point, how often, like, do you see 185 00:12:18,960 --> 00:12:23,060 that where it's like, then things drift and you've, you've got to kind of, you know, you're 186 00:12:23,060 --> 00:12:26,700 not, you're not starting from the same position you were before, like when it was a complete 187 00:12:26,700 --> 00:12:32,160 side hustle, but you still have to be prepared to sort of get that recommitment. 188 00:12:32,160 --> 00:12:39,320 What are the things that trigger those moments for recommitment and, um, you know, what do 189 00:12:39,320 --> 00:12:45,360 you see in terms of when they happen or when they don't, like, you know, what, what sort 190 00:12:45,360 --> 00:12:47,000 of things help that process? 191 00:12:47,820 --> 00:12:53,820 Yeah, we, we have extensive reporting requirements and, uh, we have quarterly reporting that we 192 00:12:53,820 --> 00:13:00,060 have to do up to senior leadership and we use that as a forum for, um, also saying what we 193 00:13:00,060 --> 00:13:05,100 learned this quarter and, uh, what we still need to move forward in the next quarter to 194 00:13:05,100 --> 00:13:05,700 meet our goals. 195 00:13:06,220 --> 00:13:13,520 So I think the first thing is really having buy-in on what the shared outcomes are that 196 00:13:13,520 --> 00:13:15,320 the senior, senior leadership wants. 197 00:13:15,400 --> 00:13:18,860 I think sometimes I've seen it happen one way or the other, where it's either what senior 198 00:13:18,860 --> 00:13:25,600 leadership wants or what the innovation team wants, uh, versus mutual, uh, mutually agreed 199 00:13:25,600 --> 00:13:33,580 two objectives, but once you have those objectives, you can't be scared to lay out, hey, we, we 200 00:13:33,580 --> 00:13:35,300 tried to do all of these things last quarter. 201 00:13:35,640 --> 00:13:36,880 Uh, here's our learnings. 202 00:13:36,880 --> 00:13:38,420 Sometimes we did the wrong things. 203 00:13:38,540 --> 00:13:39,920 Sometimes things got in our way. 204 00:13:40,020 --> 00:13:40,980 Here's our blockers. 205 00:13:41,280 --> 00:13:43,100 Uh, and here's what we need to move forward. 206 00:13:43,640 --> 00:13:46,380 And, uh, or like, hey, we need to pivot entirely. 207 00:13:46,800 --> 00:13:53,000 Um, so we internally, we, we don't do this with senior leadership yet. 208 00:13:53,040 --> 00:13:56,580 At least we don't call the forum this and I don't know if we ever will, but we use growth 209 00:13:56,580 --> 00:14:01,980 boards internally as a means to do this, uh, where it's not just the, necessarily the product 210 00:14:01,980 --> 00:14:07,640 team coming and reporting, uh, what they've done in the last quarter or it's not an accountability 211 00:14:07,640 --> 00:14:08,300 mechanism. 212 00:14:08,520 --> 00:14:13,280 It's a two way conversation, uh, for how we're meeting our objectives for the, for the 213 00:14:13,280 --> 00:14:13,560 year. 214 00:14:14,020 --> 00:14:16,240 So recommitting has to happen there. 215 00:14:16,560 --> 00:14:23,920 And I have seen, um, like one issue that we really struggled with was hiring and talent 216 00:14:23,920 --> 00:14:24,380 management. 217 00:14:24,580 --> 00:14:26,820 So we can't just go hire people. 218 00:14:26,920 --> 00:14:27,220 Right. 219 00:14:27,300 --> 00:14:30,460 Um, we have tons and tons of restrictions on that. 220 00:14:30,660 --> 00:14:34,340 And we were, we were having trouble getting the billets in place and then getting people 221 00:14:34,340 --> 00:14:35,060 to fill those. 222 00:14:35,660 --> 00:14:40,360 And, uh, we brought that up to very, very senior leadership, Dr. Rober. 223 00:14:40,520 --> 00:14:44,500 So he is like the senior acquisition executive for all air force. 224 00:14:44,900 --> 00:14:52,400 Um, and he was, uh, very frequently, I would say monthly, maybe even more often than that 225 00:14:52,400 --> 00:14:56,100 checking in, uh, to see if there was movement. 226 00:14:56,200 --> 00:14:56,640 Right. 227 00:14:56,680 --> 00:15:02,400 So you talk about that frozen middle, everybody between us and him, uh, is, he just asked the 228 00:15:02,400 --> 00:15:05,760 question, like, Hey, how's the, how's the hiring coming? 229 00:15:06,120 --> 00:15:06,220 Right. 230 00:15:06,220 --> 00:15:08,080 And it just forced a complete shift. 231 00:15:08,740 --> 00:15:10,820 Um, but that was something we had to ask. 232 00:15:10,860 --> 00:15:13,940 And it was, uh, I think sometimes people are scared to ask those things. 233 00:15:14,080 --> 00:15:19,800 And I found that people like Dr. Roper, like more than willing, all too willing to, to jump 234 00:15:19,800 --> 00:15:22,220 in and get their hands dirty with some of those things. 235 00:15:22,660 --> 00:15:26,960 So that kind of asking for like, you know, this, Hey, keep us honest. 236 00:15:26,960 --> 00:15:33,160 Like we're, we're, we're raising this as a concern and an area that we're challenged with, 237 00:15:33,160 --> 00:15:36,900 not like I'm raising this to you cause I want you to go fix it. 238 00:15:36,960 --> 00:15:39,200 I'm raising it to you to put it on your radar. 239 00:15:39,360 --> 00:15:43,560 And now I want you to kind of keep me honest by checking in on it. 240 00:15:43,560 --> 00:15:49,140 But the process of doing that effectively maybe helps, uh, unblock some things. 241 00:15:49,380 --> 00:15:49,740 Absolutely. 242 00:15:50,160 --> 00:15:50,400 Okay. 243 00:15:50,940 --> 00:15:56,640 That's, um, yeah, I think that that's kind of an interesting, uh, model to be like asked 244 00:15:56,640 --> 00:15:58,340 to be kept honest in a way. 245 00:15:58,680 --> 00:16:05,240 Um, so, and the, the growth boards, like how often you may be mentioned and I would just 246 00:16:05,240 --> 00:16:07,840 catch it in my notes, but how often is that happening? 247 00:16:08,180 --> 00:16:09,000 Uh, quarterly. 248 00:16:09,320 --> 00:16:09,720 Okay. 249 00:16:09,960 --> 00:16:12,400 And we do them at various levels. 250 00:16:12,520 --> 00:16:17,000 And I will say we weren't as disciplined about doing them at the higher levels. 251 00:16:17,380 --> 00:16:22,300 These things like growth boards and OKRs, uh, they sound really nice. 252 00:16:22,400 --> 00:16:26,880 They're easy to put into PowerPoint decks, the discipline around them and making them good 253 00:16:26,880 --> 00:16:28,540 and making them work for you is really hard. 254 00:16:28,820 --> 00:16:29,020 Yeah. 255 00:16:29,120 --> 00:16:33,780 So I would say our first two quarters of OKRs were abysmal. 256 00:16:34,080 --> 00:16:37,420 Our growth boards, uh, were haphazard at best. 257 00:16:37,660 --> 00:16:44,740 Um, and I would say last quarter, um, is when we really got into the swing of things and 258 00:16:44,740 --> 00:16:49,360 started using them effectively where we have all of the product teams meeting with their, 259 00:16:49,360 --> 00:16:52,720 uh, portfolio managers on a quarterly basis. 260 00:16:52,720 --> 00:16:57,580 And then now we have this quarter coming up, we're getting ready to have all of the portfolio 261 00:16:57,580 --> 00:17:02,500 managers meet at that higher level, uh, to report up to the line of business. 262 00:17:02,880 --> 00:17:08,760 And so that process, again, that, that is a perfect, perfect forum. 263 00:17:09,120 --> 00:17:13,000 It's kind of like feedback, you know, we say like, oh yeah, feedback culture, but instead 264 00:17:13,000 --> 00:17:16,820 you actually, until you get into the habit of doing it and you have a forum for it, it 265 00:17:16,820 --> 00:17:17,460 doesn't happen. 266 00:17:17,680 --> 00:17:18,760 It doesn't happen magically. 267 00:17:19,100 --> 00:17:20,900 There's all sorts of barriers to it. 268 00:17:21,380 --> 00:17:25,800 Um, so I always like to use things like speed back where you can get high volume and high 269 00:17:25,800 --> 00:17:29,420 frequency, uh, with these, it's really difficult. 270 00:17:29,420 --> 00:17:36,440 Um, so just being super disciplined, even the first, I would say the first quarter, they won't 271 00:17:36,440 --> 00:17:37,520 seem valuable to you. 272 00:17:37,620 --> 00:17:40,120 You'll have meetings and they seem like they're going nowhere. 273 00:17:40,520 --> 00:17:42,700 The OKRs, you look at them and you kind of groan. 274 00:17:42,860 --> 00:17:43,500 You're like, I don't know. 275 00:17:43,540 --> 00:17:44,460 Those don't seem actionable. 276 00:17:44,980 --> 00:17:46,900 And all I can say is stick it out. 277 00:17:47,440 --> 00:17:53,060 Um, every, everything I've read about OKRs, whole books written about how you'll get it 278 00:17:53,060 --> 00:17:54,500 wrong your first three or four times. 279 00:17:55,120 --> 00:17:57,820 Um, I feel pretty good cause we got it on our third try. 280 00:17:58,020 --> 00:18:03,740 So, uh, that's pretty good, but sticking those things out and being disciplined about them 281 00:18:03,740 --> 00:18:05,500 gives you the forum, right? 282 00:18:05,520 --> 00:18:09,900 You don't want it to be that when you have a problem, then you go ask, you always want 283 00:18:09,900 --> 00:18:12,600 to have a forum where that's just a normal thing. 284 00:18:12,900 --> 00:18:15,800 It's the, it's the normal you ask for a recommit. 285 00:18:16,180 --> 00:18:16,580 Yeah. 286 00:18:17,020 --> 00:18:17,420 Yeah. 287 00:18:17,420 --> 00:18:20,180 I mean, it's, it, it sounds like a retro, right. 288 00:18:20,180 --> 00:18:25,680 In a lot of ways where, you know, teams might be having a retro on a weekly basis and you 289 00:18:25,680 --> 00:18:31,240 normalize the process of getting, airing out the kind of the, the not so good stuff. 290 00:18:31,860 --> 00:18:37,140 Um, and so, you know, this seems like kind of a similar equivalent at just like a different 291 00:18:37,140 --> 00:18:44,180 sort of level of, uh, the organization, different sort of, um, objective, but yeah, you're normalizing 292 00:18:44,180 --> 00:18:49,820 the process of keeping each other honest and making sure that you're checking in on what 293 00:18:49,820 --> 00:18:50,440 you've learned. 294 00:18:50,820 --> 00:18:55,220 Um, I like this kind of tree of like, you know, what you, what you've learned, what you 295 00:18:55,220 --> 00:18:57,340 need and do you need to pivot? 296 00:18:57,600 --> 00:19:04,680 And like, if you can answer those three questions collectively, um, you know, or be able to share 297 00:19:04,680 --> 00:19:08,960 that with that group, then you've got something to kind of have those conversations around. 298 00:19:09,420 --> 00:19:10,880 So, okay. 299 00:19:10,880 --> 00:19:13,620 Well, this has been enlightening and a very fun discussion. 300 00:19:13,620 --> 00:19:18,340 Um, any, any other thoughts you want to share on the topic or, or, you know, just learnings 301 00:19:18,340 --> 00:19:19,900 and observations from the conference? 302 00:19:20,340 --> 00:19:24,380 No, I, the biggest theme that I saw come out over and over again, and it's something we're 303 00:19:24,380 --> 00:19:27,880 still struggling with is the, the platform as a product discussion. 304 00:19:27,880 --> 00:19:33,420 And it's interesting that I, this, the conversation has been the same for two years and it seems 305 00:19:33,420 --> 00:19:37,060 like, uh, it's a battle everybody's still struggling through. 306 00:19:37,700 --> 00:19:40,900 Um, I had a really interesting speaker come to Kessel Run. 307 00:19:41,140 --> 00:19:45,860 Uh, he's, uh, he was the Amazon Black Friday guru, right? 308 00:19:45,900 --> 00:19:50,600 So like Amazon was always crashing on Black Friday and after he showed up and it didn't 309 00:19:50,600 --> 00:19:51,200 crash again. 310 00:19:51,340 --> 00:19:52,540 So that's his claim to fame. 311 00:19:52,540 --> 00:19:55,860 But he, uh, he said this thing that really struck me. 312 00:19:55,900 --> 00:19:59,580 He said, there was an in-house service that Amazon offered. 313 00:19:59,580 --> 00:20:04,120 Um, but I used Akamai instead and it was big scandal, right? 314 00:20:04,140 --> 00:20:05,660 Like everybody's like, how can you do that? 315 00:20:05,700 --> 00:20:07,340 How are you not using the Amazon service? 316 00:20:07,340 --> 00:20:09,160 And he said, Akamai is better. 317 00:20:09,360 --> 00:20:10,580 Like, it's just a fact. 318 00:20:10,640 --> 00:20:12,400 If you want my business, you have to earn it. 319 00:20:12,780 --> 00:20:16,040 You don't get just to have my business because you have an Amazon label. 320 00:20:16,840 --> 00:20:23,820 And I think that all of these teams, um, it's, it's really hard. 321 00:20:23,900 --> 00:20:26,260 It's easy to say that it's hard. 322 00:20:26,260 --> 00:20:33,360 I think for the platform teams, uh, the, there's so much technical work that has to be done. 323 00:20:33,520 --> 00:20:39,220 What I'm seeing missing is a lot of the product management role that would 324 00:20:39,220 --> 00:20:45,480 typically be involved in, in the product design roles that would help market the platform to 325 00:20:45,480 --> 00:20:46,100 the customer. 326 00:20:46,440 --> 00:20:51,320 So not, not a pivotal per se, but, uh, like at all, with all the clients as they're building 327 00:20:51,320 --> 00:20:54,980 out their platforms is there's not really a service offering. 328 00:20:54,980 --> 00:20:59,840 That's like, Hey, how do you make your platform a product and market it to your business? 329 00:21:00,200 --> 00:21:03,560 It's just kind of assumed that since the business is investing in this thing, that 330 00:21:03,560 --> 00:21:04,420 people are going to use it. 331 00:21:04,500 --> 00:21:07,620 And that's not the case with any customer I've seen to date. 332 00:21:08,180 --> 00:21:11,640 So it's a, it's something that we're still struggling through. 333 00:21:12,020 --> 00:21:16,580 And, um, you run into these weird things where people, if they don't start to think about 334 00:21:16,580 --> 00:21:19,680 the platform as a product, they start to build platform for platform sake. 335 00:21:19,940 --> 00:21:20,300 Right. 336 00:21:20,420 --> 00:21:24,440 And it becomes very technology driven discussion instead of a user driven discussion. 337 00:21:25,020 --> 00:21:26,600 And, uh, yeah. 338 00:21:26,680 --> 00:21:30,720 So that was just interesting to me that, um, it's a theme that keeps popping up. 339 00:21:30,780 --> 00:21:31,820 I don't know if I have a solution. 340 00:21:31,940 --> 00:21:35,840 I'm just admiring the problem at this point, but, uh, I like to throw it out there as it's 341 00:21:35,840 --> 00:21:36,980 something we struggle with too. 342 00:21:37,100 --> 00:21:39,420 And I feel like people should be more open and honest about it. 343 00:21:39,420 --> 00:21:45,700 And, um, as of, as in doing that, hopefully people come out of the woodwork with solutions 344 00:21:45,700 --> 00:21:46,960 to, to this problem. 345 00:21:47,580 --> 00:21:47,940 Yeah. 346 00:21:48,140 --> 00:21:53,500 I, and I think there's, you know, the, that notion that you touched on of basically even 347 00:21:53,500 --> 00:22:01,240 as an internal, um, consumer, you like your message to these teams is like, you got to 348 00:22:01,240 --> 00:22:02,420 earn my business, right. 349 00:22:02,440 --> 00:22:03,880 Or I'm going to take my business somewhere else. 350 00:22:03,880 --> 00:22:09,000 Cause that's the visceral when you are actually in say the software business or you're, you're 351 00:22:09,000 --> 00:22:13,120 just in the business world, you have to live that day in and day out. 352 00:22:13,120 --> 00:22:17,400 And like every day you're getting tested for like, you know, is the cash register ringing 353 00:22:17,400 --> 00:22:17,900 or not. 354 00:22:18,420 --> 00:22:26,380 Um, and so, yeah, but if you're offering this kind of internal service, but you don't 355 00:22:26,380 --> 00:22:32,780 necessarily think of it that way, you, um, yeah, you can get relaxed in your incumbency. 356 00:22:33,840 --> 00:22:35,000 Um, okay, well, great. 357 00:22:35,080 --> 00:22:38,120 That's, uh, that's a very fun topic. 358 00:22:38,200 --> 00:22:39,960 Um, near and dear to my heart as well. 359 00:22:39,960 --> 00:22:46,640 So thank you for joining us and, uh, look forward to talking to you more and captain and beyond. 360 00:22:47,600 --> 00:22:48,200 Thank you.