1 00:00:00,000 --> 00:00:01,800 Here's something I wanted to ask your advice on. 2 00:00:01,860 --> 00:00:05,120 So you're up there in Seattle, as I'm fond of reminding people. 3 00:00:05,200 --> 00:00:07,280 And maybe we have several other guests in the room. 4 00:00:07,340 --> 00:00:08,200 Maybe they can chime in. 5 00:00:08,200 --> 00:00:18,960 But I was at Costco recently, and I bought a new bag of coffee, which was roasted in the Cuban style, except they said that in Spanish, which I should probably be able to pronounce. 6 00:00:19,640 --> 00:00:23,040 And I looked at it, and it was espresso coffee. 7 00:00:23,940 --> 00:00:26,180 And now I am not making espresso out of it. 8 00:00:26,720 --> 00:00:34,260 And what I want to know is that if I'm making regular brew coffee, like, how many years am I taking off of my life each time I drink several cups of that? 9 00:00:34,260 --> 00:00:37,620 Like, is there something about it being espresso coffee that's, like, ruining me? 10 00:00:37,760 --> 00:00:40,120 Like, do I need to just not drink those two bags of coffee? 11 00:00:40,660 --> 00:00:42,120 What would your Seattle friends say? 12 00:00:42,840 --> 00:00:46,100 I mean, I'm sure it's, like, smoking two packs of cigarettes a day or something like that. 13 00:00:46,240 --> 00:00:50,560 But, you know, believe it or not, I'm whatever, 40-ish, and I've never had a cup of coffee. 14 00:00:51,020 --> 00:00:51,300 What? 15 00:00:52,360 --> 00:00:52,720 No. 16 00:00:52,920 --> 00:00:53,240 What? 17 00:00:53,320 --> 00:00:55,260 I'm dropping a truth bomb on me right now. 18 00:00:55,720 --> 00:00:56,080 Wow. 19 00:00:56,560 --> 00:00:58,520 You don't all have to make any kinds of confessions. 20 00:00:58,620 --> 00:00:59,760 No, I don't really like hot drinks. 21 00:01:00,080 --> 00:01:03,980 And so at this point, I've gone this far, and at this point, it's only out of principle. 22 00:01:04,220 --> 00:01:05,360 Like, I'm halfway to death. 23 00:01:05,500 --> 00:01:05,780 I might as well finish it. 24 00:01:05,780 --> 00:01:09,020 I tried drinking coffee when I used to, I worked construction artists in high school. 25 00:01:09,200 --> 00:01:12,880 But I had to be, you know, at the work site at times, like, 4th, 3rd in the morning. 26 00:01:13,360 --> 00:01:17,040 So I get there, I try to drink coffee, and I'd be like, this tastes like crap, and I'm still tired. 27 00:01:17,600 --> 00:01:18,700 And then it's just like, you know what? 28 00:01:18,720 --> 00:01:19,580 This is not worth it. 29 00:01:19,780 --> 00:01:19,980 Yeah. 30 00:01:20,180 --> 00:01:20,740 Just, yeah. 31 00:01:21,060 --> 00:01:22,640 Coffee just doesn't do anything for me, so. 32 00:01:22,640 --> 00:01:23,080 Yeah. 33 00:01:23,360 --> 00:01:24,220 That is a hurdle. 34 00:01:25,200 --> 00:01:26,080 Killing you quickly. 35 00:01:26,400 --> 00:01:27,780 I drink decaf. 36 00:01:28,100 --> 00:01:28,280 Okay. 37 00:01:28,840 --> 00:01:29,240 Wow. 38 00:01:29,440 --> 00:01:29,680 Wow. 39 00:01:29,920 --> 00:01:32,560 I asked the wrong crowd of people for advice on this. 40 00:01:32,660 --> 00:01:32,840 Kenny. 41 00:01:33,000 --> 00:01:33,140 Yeah. 42 00:01:33,200 --> 00:01:33,740 Kenny, come on. 43 00:01:33,780 --> 00:01:34,560 Save me. 44 00:01:34,720 --> 00:01:35,660 I drink coffee. 45 00:01:36,100 --> 00:01:37,040 I don't have an answer. 46 00:01:37,180 --> 00:01:37,300 Yeah. 47 00:01:37,300 --> 00:01:38,140 He drinks motor oil. 48 00:01:38,560 --> 00:01:38,980 So I don't know. 49 00:01:39,240 --> 00:01:43,800 Yeah, he goes over to that cold brew stand and just, like, wraps his lips around it and pulls 50 00:01:43,800 --> 00:01:45,640 on the lever, which, uh... 51 00:01:45,640 --> 00:01:46,380 Take stamps. 52 00:01:47,000 --> 00:01:54,080 I've tried to tell him that's not hygienic, but I just keep chasing around with one of those 53 00:01:54,080 --> 00:01:55,020 little Purell bottles. 54 00:01:55,020 --> 00:01:55,660 And he's like, what? 55 00:01:55,660 --> 00:01:55,700 It's really funny. 56 00:01:55,700 --> 00:01:56,520 I don't drink it. 57 00:01:56,840 --> 00:01:59,700 You gotta get, like, one of the little water bottles you use with cats. 58 00:02:00,500 --> 00:02:01,120 No, Kenny. 59 00:02:01,380 --> 00:02:01,640 No. 60 00:02:03,340 --> 00:02:08,100 Why don't we introduce the people we have as guests on this episode? 61 00:02:08,960 --> 00:02:12,060 You're in the room with them, Richard, so why don't you rattle it off? 62 00:02:12,160 --> 00:02:15,260 I mean, don't introduce them, but, you know, point to them so they know which order to go 63 00:02:15,260 --> 00:02:15,440 in. 64 00:02:15,760 --> 00:02:16,800 I'm getting the wave. 65 00:02:17,220 --> 00:02:22,640 Um, this is Dormain Draywitz on the, uh, the product team, product marketing team. 66 00:02:23,200 --> 00:02:23,740 Thank you. 67 00:02:23,880 --> 00:02:24,400 Here at Pivotal. 68 00:02:24,920 --> 00:02:25,600 I'm Steve O'Grady. 69 00:02:25,680 --> 00:02:26,240 I'm from Redbook. 70 00:02:26,800 --> 00:02:27,680 I'm Kenny Bastani. 71 00:02:27,720 --> 00:02:29,200 I'm a spring developer advocate here at Pivotal. 72 00:02:29,200 --> 00:02:34,280 And then, as always, we have Mr. No Fun, who doesn't drink coffee all of his life, Richard. 73 00:02:34,740 --> 00:02:35,880 Just, it's such a letdown. 74 00:02:36,280 --> 00:02:36,560 I don't know. 75 00:02:36,800 --> 00:02:39,600 And, and, and, and myself, Cody. 76 00:02:39,720 --> 00:02:45,440 So I think, uh, you know, so Steven wrote a, a post a while ago, which, which I think 77 00:02:45,440 --> 00:02:50,460 I have to say for myself, sort of like, uh, typed down, so to speak, something I've been 78 00:02:50,460 --> 00:02:55,340 thinking about a lot is like, uh, people don't talk about like data and databases in, in the 79 00:02:55,340 --> 00:02:59,500 context of, uh, DevOps very much, which, which I think is, is sort of curious. 80 00:02:59,500 --> 00:03:03,100 And I think, I think what with the collection of people we have there, I think we could have 81 00:03:03,100 --> 00:03:05,480 an interesting brief conversation about that. 82 00:03:05,480 --> 00:03:09,540 But let's start off with, uh, with you, Steven, can you give us kind of like a, uh, 83 00:03:09,720 --> 00:03:13,160 sort of an overview of like what's up with databases and DevOps? 84 00:03:13,660 --> 00:03:13,840 Yeah. 85 00:03:13,920 --> 00:03:14,940 The Reader's Digest version. 86 00:03:15,040 --> 00:03:15,160 Sure. 87 00:03:15,640 --> 00:03:19,860 So basically the idea is, is that we spent a lot of time looking at, uh, a variety of 88 00:03:19,860 --> 00:03:24,720 technologies and application development processes and just anything that a developer will use, 89 00:03:24,720 --> 00:03:28,020 uh, to build and construct applications is something that we're interested in. 90 00:03:28,280 --> 00:03:33,020 And what we've noticed, you know, certainly over the past, oh, I don't know, at least a couple 91 00:03:33,020 --> 00:03:37,460 of years, you know, say four or five years, um, the application development processes evolved 92 00:03:37,460 --> 00:03:42,220 quite significantly, you know, uh, you know, through the introduction of lots and lots of 93 00:03:42,220 --> 00:03:46,440 technologies, you know, from, uh, things like decentralized version control, all the way out 94 00:03:46,440 --> 00:03:49,340 to sort of modern CI/CD systems and pipelines and everything else. 95 00:03:49,680 --> 00:03:54,040 And, you know, the funny thing is, is that sort of, as you, as you noted in lead in, when 96 00:03:54,040 --> 00:03:58,360 you look at the way these applications are developed, the database is basically completely absent, 97 00:03:58,360 --> 00:03:59,360 right? 98 00:03:59,360 --> 00:04:02,700 You know, nobody talks about the database, nobody talks about the DBA. 99 00:04:02,700 --> 00:04:08,460 So, you know, one of the things that, you know, we're trying to figure out is, you know, 100 00:04:08,460 --> 00:04:14,260 is that appropriate, you know, is there an opportunity, uh, you know, potentially to try to bridge 101 00:04:14,260 --> 00:04:18,660 that divide between application developer and DBA, you know, very much as we have seen the 102 00:04:18,660 --> 00:04:23,360 industry do, uh, between two previously distinct categories in dev and ops. 103 00:04:23,360 --> 00:04:25,360 So that's kind of the idea that I don't. 104 00:04:25,360 --> 00:04:28,360 Did you, why, why do you think there's that divide? 105 00:04:28,360 --> 00:04:31,160 Is it because it was the same divide we had with dev and ops, where it's like, I don't 106 00:04:31,160 --> 00:04:32,280 like dealing with that team. 107 00:04:32,280 --> 00:04:35,160 So therefore app teams said, I don't want to deal with DBAs. 108 00:04:35,160 --> 00:04:38,960 So we'll just ignore that and ship software itself in its own way. 109 00:04:38,960 --> 00:04:42,660 I think there's a variety of reasons, honestly, some of it's structural, some of it's cultural, 110 00:04:42,660 --> 00:04:45,460 um, you know, and some of it's just sort of a matter of focus. 111 00:04:45,460 --> 00:04:45,660 Right. 112 00:04:45,660 --> 00:04:51,160 So, you know, taking, you know, taking a couple of those, uh, in many cases, it is a, uh, you 113 00:04:51,160 --> 00:04:53,760 know, very much sort of a cultural divide in the way that dev and ops was. 114 00:04:53,760 --> 00:04:55,760 Right. 115 00:04:55,760 --> 00:05:00,760 Um, all the scenarios, uh, when I was an SI and, you know, sort of, uh, uh, on client 116 00:05:00,760 --> 00:05:03,760 sites and buildings, um, I had to deal with ops. 117 00:05:03,760 --> 00:05:04,760 I hated ops. 118 00:05:04,760 --> 00:05:08,760 Um, I had to deal with DBAs, uh, I hated them slightly less, but they were a pain in the 119 00:05:08,760 --> 00:05:09,760 ass to deal with. 120 00:05:09,760 --> 00:05:10,760 And, you know, vice versa. 121 00:05:10,760 --> 00:05:13,760 I'm sure they hated us, you know, because we wanted to do things and then sort of move 122 00:05:13,760 --> 00:05:17,760 quickly and do basically, you know, things that didn't work for them. 123 00:05:17,760 --> 00:05:19,760 Um, so there's that, I think that's part of it. 124 00:05:19,760 --> 00:05:24,760 Um, particularly for many developers to, uh, any developers ever been on call is probably 125 00:05:24,760 --> 00:05:28,760 not super fond of ops cause they're the ones who typically call you in the middle of the 126 00:05:28,760 --> 00:05:29,760 night cause you're broke. 127 00:05:29,760 --> 00:05:35,760 Uh, so you have, you know, sort of those issues, but also just from a structural standpoint, 128 00:05:35,760 --> 00:05:36,760 right? 129 00:05:36,760 --> 00:05:39,760 Many of these very much like dev and ops were, were separate entities, you know, sort 130 00:05:39,760 --> 00:05:42,760 of, you know, Hey, I do my thing and you do your thing. 131 00:05:42,760 --> 00:05:44,760 You know, we don't really talk to each other. 132 00:05:44,760 --> 00:05:46,760 That's structurally been the way that many organizations have been set up, right? 133 00:05:46,760 --> 00:05:51,760 In other words, you have DBAs, uh, people responsible for securing, protecting, you know, 134 00:05:51,760 --> 00:05:54,760 designing the scheme is associated with a particular database. 135 00:05:54,760 --> 00:05:57,760 Um, and then you have people who do the app development. 136 00:05:57,760 --> 00:06:01,760 And to the extent that those are involved, you basically will say, here's what I want my 137 00:06:01,760 --> 00:06:02,760 application to look like. 138 00:06:02,760 --> 00:06:05,760 Um, figure the database out. 139 00:06:05,760 --> 00:06:06,760 And then that's kind of it. 140 00:06:06,760 --> 00:06:07,760 Right. 141 00:06:07,760 --> 00:06:12,760 And you know, they're done and, um, they're just not sort of integrate, integrally, uh, 142 00:06:12,760 --> 00:06:13,760 involved in that process. 143 00:06:13,760 --> 00:06:17,760 What do you think makes up the data DevOps story? 144 00:06:17,760 --> 00:06:20,760 I mean, I think there's, you mentioned some things like source control that became a big 145 00:06:20,760 --> 00:06:21,760 part of the dev life cycle. 146 00:06:21,760 --> 00:06:27,760 And, but I mean, automated builds and a different approach to maybe service abstractions, data 147 00:06:27,760 --> 00:06:28,760 replication, caching. 148 00:06:28,760 --> 00:06:33,760 I mean, what are those things that you think to put, bring it tangibly, if I'm a DBA listening 149 00:06:33,760 --> 00:06:36,760 person's going, what do you really mean when we think of DevOps and databases and everything 150 00:06:36,760 --> 00:06:42,760 from blue green deployments to schema changes and source control and CI builds, bring this 151 00:06:42,760 --> 00:06:43,760 down to a. 152 00:06:43,760 --> 00:06:44,760 Sure. 153 00:06:44,760 --> 00:06:45,760 Tangible level, right. 154 00:06:45,760 --> 00:06:46,760 Uh, yeah, I think there's a number of different things. 155 00:06:46,760 --> 00:06:48,760 So when you first, you know, when you design an application, right, you know, think about, 156 00:06:48,760 --> 00:06:49,760 let's take something simple. 157 00:06:49,760 --> 00:06:50,760 Let's say you're a rails developer. 158 00:06:50,760 --> 00:06:51,760 All right. 159 00:06:51,760 --> 00:06:55,760 How do you set up, you know, the, the database typically you're doing it in the YAML file 160 00:06:55,760 --> 00:06:56,760 or something like that. 161 00:06:56,760 --> 00:07:01,760 Um, what a lot of applications find out sort of as they grow up is that that maybe isn't 162 00:07:01,760 --> 00:07:03,760 ideal for what you want to do. 163 00:07:03,760 --> 00:07:06,760 Um, maybe you need to scale, maybe you need to shard it and so on. 164 00:07:06,760 --> 00:07:08,760 And all of a sudden you're out of your depth. 165 00:07:08,760 --> 00:07:10,760 Um, all of a sudden you're out of your expertise, right? 166 00:07:10,760 --> 00:07:16,760 So having, you know, deep gaze intimately involved in that developer process can help 167 00:07:16,760 --> 00:07:19,760 you either at the outset or as your application scales up. 168 00:07:19,760 --> 00:07:22,760 But there's also lots of sort of trivial things that we don't think about, right. 169 00:07:22,760 --> 00:07:27,760 In terms of, oh, uh, you know, as you're developing sort of an application, you know, 170 00:07:27,760 --> 00:07:31,760 one of the things that people can sort of gloss over frequently is test data, right? 171 00:07:31,760 --> 00:07:32,760 Where do you get that test data? 172 00:07:32,760 --> 00:07:34,760 How does it, you know, how is it generated, right? 173 00:07:34,760 --> 00:07:37,760 In many cases there are, you know, there are vendors out there who do nothing more than 174 00:07:37,760 --> 00:07:42,760 generate essentially the sort of lorem ipsum equivalent for, um, you know, that kind of 175 00:07:42,760 --> 00:07:47,760 data where DBA in many cases can basically say, all right, I can, you know, fork a copy of 176 00:07:47,760 --> 00:07:48,760 our database, salt it sufficiently. 177 00:07:48,760 --> 00:07:50,760 So we're not exposing anything we shouldn't be. 178 00:07:50,760 --> 00:07:56,760 And boom, now you have essentially real live test data that is a safe, but B is going to 179 00:07:56,760 --> 00:08:00,760 work, you know, basically going to give you the opportunity to test and build your application 180 00:08:00,760 --> 00:08:02,760 as if it was something in production. 181 00:08:02,760 --> 00:08:04,760 So I think there's lots of different ways this can play out. 182 00:08:04,760 --> 00:08:09,760 Frankly, we, you know, we can think of a couple of them, but, you know, very much as we've 183 00:08:09,760 --> 00:08:13,760 seen with DevOps, where this has taken lots and lots of different pathways, uh, people, 184 00:08:13,760 --> 00:08:16,760 you know, involved at the early days could not have anticipated. 185 00:08:16,760 --> 00:08:18,760 Basically, I think it's the same thing here. 186 00:08:18,760 --> 00:08:22,760 Dormain, you, uh, covered a lot of stuff with Greenplum as well, as we think of just beyond 187 00:08:22,760 --> 00:08:23,760 the database to the data warehouse. 188 00:08:23,760 --> 00:08:27,760 I mean, is this a complete, I mean, if you were introducing this to a Greenplum audience 189 00:08:27,760 --> 00:08:32,760 saying, Hey, it's time to start doing some CI and CD to your data warehouse and artifacts and source 190 00:08:32,760 --> 00:08:36,760 control, would you be, you know, burned at the stake or is this something where that kind 191 00:08:36,760 --> 00:08:41,760 of crowd you think would welcome this idea of maybe a little more rigor around the maintenance 192 00:08:41,760 --> 00:08:45,760 and the setup and the on-demand nature and all those things around warehouses? 193 00:08:45,760 --> 00:08:51,760 Well, I mean, like when you're looking at the data side of things, like, you know, jumping 194 00:08:51,760 --> 00:08:56,760 to the data warehouses, now you're talking about something that is more than a couple degrees 195 00:08:56,760 --> 00:08:58,760 removed away from the developer. 196 00:08:58,760 --> 00:08:59,760 Sure. 197 00:08:59,760 --> 00:09:01,760 I mean, there's a lot of stuff that's a lot closer to the development teams that would 198 00:09:01,760 --> 00:09:04,760 probably be a more logical starting place. 199 00:09:04,760 --> 00:09:10,760 And I think as those things, you know, potentially evolve and change in this direction, right? 200 00:09:10,760 --> 00:09:17,760 And to the hypothesis you've got, then that, that might have like downstream implications 201 00:09:17,760 --> 00:09:19,760 for teams like the data warehousing team. 202 00:09:19,760 --> 00:09:24,760 I mean, right now, like what's kind of interesting that, you know, we're, we're seeing a lot of 203 00:09:24,760 --> 00:09:31,760 already was, you know, the, the data that people want to take a look at and sort of apply data warehousing 204 00:09:31,760 --> 00:09:34,760 to is sort of, it's all over the place. 205 00:09:34,760 --> 00:09:35,760 Right. 206 00:09:35,760 --> 00:09:37,760 So different teams just went down different paths. 207 00:09:37,760 --> 00:09:41,760 You know, there was a lot of like, okay, we're going to create a Hadoop cluster because 208 00:09:41,760 --> 00:09:43,760 that seems like the cool thing to do. 209 00:09:44,760 --> 00:09:50,760 But some data is going in there, but it's not necessarily usable in the way that we want it. 210 00:09:50,760 --> 00:09:53,760 And then, oh, this team over here is building stuff in Amazon. 211 00:09:53,760 --> 00:09:56,760 So they're, you know, they've got a bunch of data accumulating in S3. 212 00:09:56,760 --> 00:10:03,760 And so, I mean, the last year, Greenplum actually started to, I mean, not started to, but took 213 00:10:03,760 --> 00:10:08,760 some additional steps down being able to just read from all these different places as external 214 00:10:08,760 --> 00:10:14,760 tables, because I think the notion of, okay, we're going to create a data lake and it's 215 00:10:14,760 --> 00:10:16,760 going to be one data lake to rule them all. 216 00:10:16,760 --> 00:10:19,760 And like data will flow in and unicorns will flow out. 217 00:10:19,760 --> 00:10:23,760 It just has been proving over and over again to not happen. 218 00:10:23,760 --> 00:10:26,760 And so it's for various reasons, right? 219 00:10:26,760 --> 00:10:30,760 Like people, different parts within an organization don't want to give up their data, right? 220 00:10:30,760 --> 00:10:31,760 The data huggers. 221 00:10:31,760 --> 00:10:34,760 And they've got, they're like, no, we're special. 222 00:10:34,760 --> 00:10:38,760 And, you know, they're, they're putting it into Hadoop. 223 00:10:38,760 --> 00:10:41,760 Like there's a shortage of skills to be able to do anything with it. 224 00:10:41,760 --> 00:10:47,760 So the natural, you know, gravity of data is just not actually that focused. 225 00:10:47,760 --> 00:10:49,760 But it seems like it's also coming to the developer more, Kenny. 226 00:10:49,760 --> 00:10:54,760 I mean, as you look at things like streaming data and Spring Cloud stream, and like all 227 00:10:54,760 --> 00:10:57,760 of a sudden we're getting closer to the warehouse than we probably have been before as a dev, 228 00:10:57,760 --> 00:11:00,760 because I might actually be streaming data from something front end to that. 229 00:11:00,760 --> 00:11:02,760 Or do you think this is forcing the issue finally? 230 00:11:02,760 --> 00:11:07,760 I mean, the things that Steve's bringing up here, because devs are now integrating data 231 00:11:07,760 --> 00:11:10,760 in different ways and moving data in different ways, they can't just stand back and say, 232 00:11:10,760 --> 00:11:11,760 eh, the database. 233 00:11:11,760 --> 00:11:15,760 Like that's a big part of their app and their service or microservice or whatever, right? 234 00:11:15,760 --> 00:11:19,760 Yeah, so this is a really hot topic, kind of near and dear to my heart in a way. 235 00:11:19,760 --> 00:11:22,760 I was a developer evangelist over at Neo4j a couple years back. 236 00:11:22,760 --> 00:11:28,760 And this is something that we talked about a lot is what is the future of the database really 237 00:11:28,760 --> 00:11:33,760 look like when you're going to a cloud environment where you have all these application components, 238 00:11:33,760 --> 00:11:36,760 infrastructure that's ephemeral, but that data isn't ephemeral, right? 239 00:11:36,760 --> 00:11:39,760 That data has to be somewhere at one time, right? 240 00:11:39,760 --> 00:11:42,760 It has to have backups and there's a process there and you need expertise. 241 00:11:42,760 --> 00:11:47,760 And so what I found is the DBA really has an opportunity, and this comes from personal experience 242 00:11:47,760 --> 00:11:52,760 developing microservices, that the DBA kind of becomes a little bit more of an architect. 243 00:11:52,760 --> 00:11:57,760 The developer is still doing the work and they should be competent enough to do that work, 244 00:11:57,760 --> 00:12:02,760 but the DBA is there to do reviews with the developers and make sure that the standards 245 00:12:02,760 --> 00:12:05,760 are being set forth in the application development process. 246 00:12:05,760 --> 00:12:06,760 So that's what I found. 247 00:12:06,760 --> 00:12:07,760 Yeah. 248 00:12:07,760 --> 00:12:09,760 So you're seeing that come closer. 249 00:12:09,760 --> 00:12:12,760 You know, I had a question for Dormain. 250 00:12:12,760 --> 00:12:20,760 I think, like as you were describing the world of data people, I don't know, I'd be interested 251 00:12:20,760 --> 00:12:24,760 what Kenny and Steven think if this is the same calibration that I was having. 252 00:12:24,760 --> 00:12:30,760 It's like, oh, structurally sounds like similar problems that the apps, the app dev and the ops 253 00:12:30,760 --> 00:12:31,760 people have, right? 254 00:12:31,760 --> 00:12:34,760 Like you're sort of like, uh, you're sort of like chasing all these trend things and you 255 00:12:34,760 --> 00:12:37,760 got to worry about things actually working and all of that. 256 00:12:37,760 --> 00:12:43,760 But, but I'm curious, like, I mean, ultimately a lot of what we're shooting for with all of 257 00:12:43,760 --> 00:12:46,760 the, like a few episodes ago, we talked about what cloud native is, but it's sort of like, 258 00:12:46,760 --> 00:12:49,760 I would like to more quickly innovate and do things right. 259 00:12:49,760 --> 00:12:53,760 Like currently the way that I'm running it doesn't move fast enough and I can't do enough 260 00:12:53,760 --> 00:12:54,760 with it. 261 00:12:54,760 --> 00:12:58,760 So I would like to change things around so that, uh, I can change stuff up. 262 00:12:58,760 --> 00:13:03,760 And, and when, when you look at like what DevOps has been doing a lot of what the application 263 00:13:03,760 --> 00:13:09,760 developers and the operations people have been figuring out past constraints they had in 264 00:13:09,760 --> 00:13:11,760 their life that caused them to need to move slow. 265 00:13:11,760 --> 00:13:16,760 So for example, uh, usually when we deployed a production stuff breaks, so we're going to try 266 00:13:16,760 --> 00:13:20,760 to avoid deploying to production, which obviously means you move slow. 267 00:13:20,760 --> 00:13:26,760 And I'm curious similarly in the data world, like what are some constraints that data people 268 00:13:26,760 --> 00:13:30,760 have that they would need to like figure out if they want to move fast. 269 00:13:30,760 --> 00:13:34,760 And, and, and as an example, cause my questions are often statements and very confusing, but 270 00:13:34,760 --> 00:13:39,760 like, you know, I often hear from data people, like you can't lose data, right? 271 00:13:39,760 --> 00:13:44,760 Which, which seems like a simple answer, but if you think about that mentality in the application 272 00:13:44,760 --> 00:13:48,760 world, I mean, like you can kind of lose an application cause you can just restart it. 273 00:13:48,760 --> 00:13:49,760 Right. 274 00:13:49,760 --> 00:13:51,760 So it's not like such a big deal. 275 00:13:51,760 --> 00:13:55,760 So like what are these kinds of constraints that data people have that are almost like these 276 00:13:55,760 --> 00:14:00,760 huge constraints that change how they think about how fast they can move and how devopsy 277 00:14:00,760 --> 00:14:01,760 they could be. 278 00:14:01,760 --> 00:14:02,760 Yeah. 279 00:14:02,760 --> 00:14:06,760 I mean, data loss is definitely one of them, but that's also kind of to that point of, you 280 00:14:06,760 --> 00:14:12,760 know, where in this sort of the spectrum of data practices, you know, when you're, when 281 00:14:12,760 --> 00:14:18,760 you're dealing with something like a gem fires and other product that I've worked on, that's, 282 00:14:18,760 --> 00:14:21,760 that is often in a position where like data loss is not an option, right? 283 00:14:21,760 --> 00:14:25,760 It's, it's a try, it's doing something that's transactional, et cetera. 284 00:14:26,760 --> 00:14:33,760 Um, whereas like certainly data loss is not acceptable, you know, for even something like a green 285 00:14:33,760 --> 00:14:36,760 plum, but it's much more tolerant to downtime. 286 00:14:36,760 --> 00:14:40,760 Um, whereas something like gem fire is not tolerant to downtime. 287 00:14:40,760 --> 00:14:44,760 Like what it's, it's doing, doing something that you've got to keep it up and you can't 288 00:14:44,760 --> 00:14:45,760 lose any data. 289 00:14:45,760 --> 00:14:50,760 So there's just some really tight constraints, but there are other systems that don't. 290 00:14:50,760 --> 00:14:52,760 So it's not the same as what I'm trying to say. 291 00:14:52,760 --> 00:14:55,760 It's like, there's, there is a spectrum in data. 292 00:14:55,760 --> 00:14:56,760 Yeah. 293 00:14:56,760 --> 00:15:00,760 Um, and there was something else in the first part of your question slash statement that 294 00:15:00,760 --> 00:15:06,760 was confusing that I had another comment on, but you've managed to totally. 295 00:15:06,760 --> 00:15:07,760 Wrap you up into a pretzel. 296 00:15:07,760 --> 00:15:08,760 Yeah. 297 00:15:08,760 --> 00:15:11,760 Like it was like a weird Jedi mind trick or something. 298 00:15:11,760 --> 00:15:12,760 I don't know. 299 00:15:12,760 --> 00:15:20,760 Um, um, so yeah, come back to me, but it's, um, the, the notion of what Kenny was referring 300 00:15:20,760 --> 00:15:25,760 to in terms of bringing, bringing the DBA in as that, that consultant architect and the 301 00:15:25,760 --> 00:15:30,760 notion of like, we've sort of touched on this with a couple of teams, like the balance team, 302 00:15:30,760 --> 00:15:31,760 right. 303 00:15:31,760 --> 00:15:37,760 And how do you start to incorporate data on that balance team so that they're, uh, they're 304 00:15:37,760 --> 00:15:42,760 present for not only like decision-making, but also for the discovery of requirements. 305 00:15:42,760 --> 00:15:49,760 Um, so, you know, like this is one of our data scientists, Ian, uh, Houston or Huston, 306 00:15:49,760 --> 00:15:51,760 or however you say his name, he's British. 307 00:15:51,760 --> 00:15:59,760 So I don't trust any pronunciation as soon as I start thinking about people from there. 308 00:15:59,760 --> 00:16:04,760 But nonetheless, um, he, he actually wrote about this and, uh, we, we interviewed him 309 00:16:04,760 --> 00:16:07,760 on, on the pivotal insights podcast. 310 00:16:07,760 --> 00:16:11,760 And, you know, he, he talked about a lot of really interesting points from a design perspective, 311 00:16:11,760 --> 00:16:17,760 which is like, Hey, to be sitting there in the user interviews that like the agile team 312 00:16:17,760 --> 00:16:22,760 is doing is amazing from a design perspective, instead of just getting like pitch something 313 00:16:22,760 --> 00:16:25,760 over the wall to be like, Hey, we need a predictive algorithm for this. 314 00:16:25,760 --> 00:16:30,760 But it's like, Oh, it really helps if I get to sit there and actually ask the user some 315 00:16:30,760 --> 00:16:32,760 questions about like what they're trying to do. 316 00:16:32,760 --> 00:16:35,760 Cause that's going to change maybe what I go do. 317 00:16:35,760 --> 00:16:40,760 Like from a data science perspective, which probably also ultimately influences 318 00:16:40,760 --> 00:16:42,760 what's going to happen for how are we going to structure the database? 319 00:16:42,760 --> 00:16:44,760 What database are we going to use for this? 320 00:16:44,760 --> 00:16:45,760 Um, but yeah. 321 00:16:45,760 --> 00:16:50,760 I want to piggyback that you may ask the question about constraints that kind of might impact 322 00:16:50,760 --> 00:16:51,760 this. 323 00:16:51,760 --> 00:16:55,760 I want to ask you, Steve, are there other constraints around people who like throwing logic in the 324 00:16:55,760 --> 00:16:56,760 database? 325 00:16:56,760 --> 00:16:59,760 And does that, does this change in DevOps where it says maybe I don't do store procedures 326 00:16:59,760 --> 00:17:04,760 or I don't use views or user defined functions and maybe even some things that exist in NoSQL 327 00:17:04,760 --> 00:17:05,760 databases. 328 00:17:05,760 --> 00:17:09,760 Is that a constraint that teams have to actually change their behavior or do you just have to 329 00:17:09,760 --> 00:17:12,760 adjust to using those things in a DevOps fashion? 330 00:17:12,760 --> 00:17:14,760 I would imagine in many cases you would just adjust. 331 00:17:14,760 --> 00:17:19,760 Um, I'm sure that there are, I'm positive that there are exceptions where in other words, 332 00:17:19,760 --> 00:17:24,760 it's like, like you crammed this full of so much procedural logic and so many stored procedures 333 00:17:24,760 --> 00:17:29,760 that it just is not, it's not going to be compatible with essentially a modern, you know, 334 00:17:29,760 --> 00:17:33,760 life cycle, you know, you know, particularly as quickly as we move and so on. 335 00:17:33,760 --> 00:17:39,760 But my guess is, is that in, in a lot of the cases, you know, you can, you can basically 336 00:17:39,760 --> 00:17:40,760 adapt. 337 00:17:40,760 --> 00:17:43,760 Um, and in some cases, you know, that may be pulling things out. 338 00:17:43,760 --> 00:17:48,760 Like I've had to do that before, um, years and years and years ago, pulling stored procedures 339 00:17:48,760 --> 00:17:53,760 out because, uh, the way that we were doing it in the content management deployment, um, was 340 00:17:53,760 --> 00:17:58,760 not compatible with some new operational standard they had, you know, for the data center. 341 00:17:58,760 --> 00:18:05,760 So, you know, obviously, you know, infrastructure people will have a history of, um, uh, dealing 342 00:18:05,760 --> 00:18:06,760 with changes just like this. 343 00:18:06,760 --> 00:18:12,760 Um, but yeah, I think I would, I'd be willing to bet in, in a lot of cases you can, you know, 344 00:18:12,760 --> 00:18:14,760 basically deal with that kind of logic. 345 00:18:14,760 --> 00:18:18,760 The only other one of the constraint I wanted to bring up for Kenny was, you know, that question 346 00:18:18,760 --> 00:18:19,760 of data loss. 347 00:18:19,760 --> 00:18:24,760 And, you know, we all love in this world of app CICD, this idea of like, I'm going to ship 348 00:18:24,760 --> 00:18:27,760 10 times an hour and I can do zero downtime because cloud foundry. 349 00:18:27,760 --> 00:18:31,760 You'll do blue green or I can do other ways to do rolling updates. 350 00:18:31,760 --> 00:18:33,760 What does it mean in a data world? 351 00:18:33,760 --> 00:18:38,760 How in the world do I do continuous delivery of a database where there is actually schema 352 00:18:38,760 --> 00:18:39,760 and state? 353 00:18:39,760 --> 00:18:43,760 I mean, what patterns do you see for actually not losing data or not taking downtime and 354 00:18:43,760 --> 00:18:45,760 actually swapping a schema or things like that? 355 00:18:45,760 --> 00:18:46,760 It's a great question. 356 00:18:46,760 --> 00:18:51,760 Um, and it's one that kind of keeps me up at night lately a little bit, especially recently. 357 00:18:51,760 --> 00:18:56,760 Um, I think a lot of people here probably heard everyone about getting the labs data loss. 358 00:18:56,760 --> 00:19:00,760 Um, now when you think about that, the microservice level, right, you have all of these independent 359 00:19:00,760 --> 00:19:04,760 agile teams who are delivering continuously into production and they're delivering features. 360 00:19:04,760 --> 00:19:05,760 They're delivering business logic, right? 361 00:19:05,760 --> 00:19:09,760 Um, so they're adding business capabilities to the whole, but they're all integrated together 362 00:19:09,760 --> 00:19:14,760 and each might have, and this is the pattern that they have their own database and the developers 363 00:19:14,760 --> 00:19:17,760 are managing that database, um, in a lot of cases. 364 00:19:17,760 --> 00:19:23,760 But what happens when one of those microservices that's has, um, relationships to other services, 365 00:19:23,760 --> 00:19:28,760 uh, in data there and they have data loss in one service and you can't get it back. 366 00:19:28,760 --> 00:19:31,760 But all of the other data was added to the other services. 367 00:19:31,760 --> 00:19:36,760 Now you have kind of an inconsistency issue, uh, where you're not able to recover without 368 00:19:36,760 --> 00:19:37,760 pulling everything back. 369 00:19:37,760 --> 00:19:40,760 So that's really, I think a problem that, um, most people should be worried about. 370 00:19:40,760 --> 00:19:44,760 And I think that DBA is a really, uh, kind of critical to this process, or at least 371 00:19:44,760 --> 00:19:48,760 on the platform side to make sure that they're handling these concerns across the board, 372 00:19:48,760 --> 00:19:50,760 uh, to make sure that this doesn't happen. 373 00:19:50,760 --> 00:19:53,760 Have you seen solutions, Steve, around that? Do people, again, do that work? 374 00:19:53,760 --> 00:19:57,760 It's people skittish and that's why they do stage, do they do plan deployments? 375 00:19:57,760 --> 00:19:59,760 I think the latter. Yeah, I really do. 376 00:19:59,760 --> 00:20:03,760 I think people, and this is this, you know, kind of circles us back around to the original point. 377 00:20:03,760 --> 00:20:08,760 Like so much of, um, you know, so much of we see, you know, of what we see today, you know, 378 00:20:08,760 --> 00:20:12,760 from a, from a model standpoint, from a procedural standpoint is due to the skittishness 379 00:20:12,760 --> 00:20:15,760 or the concerns around data, right. 380 00:20:15,760 --> 00:20:18,760 Um, in terms of, well, okay, you know, it's kind of hands off and I don't really talk to those 381 00:20:18,760 --> 00:20:22,760 people and I'm not really sure how to, you know, sort of, uh, um, you know, go back and 382 00:20:22,760 --> 00:20:25,760 forth. So yeah, I would say it's very much the latter. 383 00:20:25,760 --> 00:20:29,760 Is that where service abstractions or caches come in handy to insulate me a little bit 384 00:20:29,760 --> 00:20:33,760 from changes underneath? Like I can do V1, V2 of services that have different schemas 385 00:20:33,760 --> 00:20:36,760 and maybe I don't remove fields. I just make them additive. 386 00:20:36,760 --> 00:20:39,760 I don't know. Have you seen things domain where a cache or some other abstraction 387 00:20:39,760 --> 00:20:42,760 protects me a little bit from these database level changes? 388 00:20:42,760 --> 00:20:48,760 Yeah, definitely. We, I mean, we've seen caching come up a lot as customers are sort of going 389 00:20:48,760 --> 00:20:55,760 down a, Hey, we want to refactor an application on Pivotal Cloud Foundry. And, you know, this 390 00:20:55,760 --> 00:21:01,760 is going to lead to potentially a lot more calls into a legacy database. Oh, wait, that legacy 391 00:21:01,760 --> 00:21:06,760 database is a mainframe and it's really expensive or, you know, to have all those calls or there's 392 00:21:06,760 --> 00:21:12,760 going to be performance constraints, what have you. And so putting a caching layer, uh, 393 00:21:12,760 --> 00:21:18,760 between, you know, the, the sort of refactored or in process of being refactored application 394 00:21:18,760 --> 00:21:24,760 and that legacy database sort of suddenly helps lift some of those constraints and, and lets 395 00:21:24,760 --> 00:21:30,760 those, you know, new microservices call to something that's caching that data. They can, they can 396 00:21:30,760 --> 00:21:36,760 they can, but it's not going to stress that legacy database, which folks can still continue 397 00:21:36,760 --> 00:21:39,760 to run their legacy processes around, et cetera. 398 00:21:39,760 --> 00:21:43,760 Yeah. I think it's, it's interesting to see too, where you, you begin to have, it's certainly 399 00:21:43,760 --> 00:21:49,760 not a, a common, uh, practice today, but you do begin in some scenarios and, you know, some 400 00:21:49,760 --> 00:21:54,760 vendors and, um, uh, some organizations begin to have a notion of forking databases, right. 401 00:21:54,760 --> 00:21:59,760 You know, be able to say, okay, we're making a decision. We're doing something at this particular 402 00:21:59,760 --> 00:22:02,760 point in time and, you know, basically, okay, we're going to fork it. So we're going to take 403 00:22:02,760 --> 00:22:06,760 a copy of this database. We're going to go down this road. If this proves to be the wrong 404 00:22:06,760 --> 00:22:11,760 path, fine. You know, we have what is effectively a backup. Um, but in other words, it's, you know, 405 00:22:11,760 --> 00:22:19,760 very much a notion of, you know, trying to take a software development like approach, uh, you 406 00:22:19,760 --> 00:22:24,760 know, kind of with data, right. You know, having the ability to essentially have different versions 407 00:22:24,760 --> 00:22:28,760 and fork one and okay. Um, you know, how do I reconcile these things if necessary after 408 00:22:28,760 --> 00:22:33,760 the fact? So it's really not there. It's really not common. You very, very rarely see this in 409 00:22:33,760 --> 00:22:38,760 the wild, but it is a notion that is there, you know, has been sort of floated in, in certain 410 00:22:38,760 --> 00:22:41,760 corners and I would not be surprised to see that gain currency over time. Yeah. 411 00:22:41,760 --> 00:22:45,760 I mean, that's, I mean, and I'm starting to see that as I, I talked to a number of the 412 00:22:45,760 --> 00:22:51,760 ISVs that are building integrations with Cloud Foundry and there's a, there's a range of what 413 00:22:51,760 --> 00:22:56,760 types of integration. I mean, the most basic is usually, Hey, we want to have a, a service 414 00:22:56,760 --> 00:23:03,760 broker so that people can just connect to an externally managed version of this, um, database, 415 00:23:03,760 --> 00:23:07,760 right. Because they're, there's, they're already existing and we just want to be able to help 416 00:23:07,760 --> 00:23:12,760 developers get up and running. Um, maybe we were going to give them sort of more privileges 417 00:23:12,760 --> 00:23:19,760 and capabilities in terms of, Oh, create a new table or something. Um, but there's also, 418 00:23:19,760 --> 00:23:22,760 I don't want to like steal any thunder because I know some of these guys are going to be making 419 00:23:22,760 --> 00:23:27,760 announcements soon, guys and gals, but yeah, looking at ways to, you know, clone databases, 420 00:23:27,760 --> 00:23:32,760 make them, you know, discoverable, um, as well as sort of making them, you know, Hey, you want 421 00:23:32,760 --> 00:23:37,760 a new one, you can have it on demand, right. Fully managed, uh, type of thing. So there's, 422 00:23:37,760 --> 00:23:43,760 there's a lot of interesting work being done, but it's definitely, that's all as far as I can tell 423 00:23:43,760 --> 00:23:46,760 pretty early days, at least from my vantage point. 424 00:23:46,760 --> 00:23:50,760 That's a great point. I think just to add what Steven was saying, um, I think that there's 425 00:23:50,760 --> 00:23:56,760 a place in this microservice cloud native world for change data capture where every time you're 426 00:23:56,760 --> 00:24:00,760 writing some of the database, you have some kind of replica set that you can go back to. 427 00:24:00,760 --> 00:24:05,760 That's not really a database back. It's not like a monolithic backup, but it's a way to stream to 428 00:24:05,760 --> 00:24:09,760 somewhere else where you can keep that data safe. Um, that way, if you experience data loss, 429 00:24:09,760 --> 00:24:12,760 there's always going to be something that you can go back to to restore. 430 00:24:13,760 --> 00:24:17,760 Yeah. Sorry. Historian or just even having a good, I don't know, event sourcing model or something there. 431 00:24:17,760 --> 00:24:18,760 Yeah. 432 00:24:18,760 --> 00:24:21,760 So one thing I want to add one more question, I guess each of you get a shot at it. 433 00:24:21,760 --> 00:24:24,760 So for people who are trying to think about, all right, some of this sounds really interesting, 434 00:24:24,760 --> 00:24:29,760 science fiction, future stuff. How did, how did the company start to get started with some of this? 435 00:24:29,760 --> 00:24:34,760 Is it putting their database scripts in source control or trying to make it actually possible to get a database on demand? 436 00:24:34,760 --> 00:24:39,760 So a dev doesn't have to go through hoops to just get a local copy with some anonymized data. 437 00:24:39,760 --> 00:24:49,760 Like how do we recommend that a company does try to get started with treating databases as a little more of an agile asset versus these static things that we're terrified of domain first. 438 00:24:49,760 --> 00:24:56,760 I mean, I, I think some of it comes down to the people. So the notion of the balance team and bringing DBAs, data scientists, analysts, 439 00:24:56,760 --> 00:25:05,760 whoever is going to ultimately need to work with that data, like into the team that's building the application where the data is going to be produced. 440 00:25:05,760 --> 00:25:08,760 I think that's going to have, that's going to be some part of it. 441 00:25:08,760 --> 00:25:15,760 I agree with Dorman 100%. It starts with culture, right? And then everything else comes back to that. I think that's the first step. 442 00:25:15,760 --> 00:25:27,760 Yeah. So they stole my answer. I was going to go with culture. You know, basically integrating the teams as quickly as you can. And, you know, as patiently as you can, because it's not obviously an overnight process, I think is important. 443 00:25:27,760 --> 00:25:43,760 But beyond that, you know, really, I think one of the things that would be, you know, helpful beyond just sort of the initial association is really trying to, you know, get a, get an understanding of the data side of your business, right? 444 00:25:43,760 --> 00:25:51,760 Because, you know, many of the people listening to this probably have quite sophisticated, as we talked about, application development, you know, processes and policies. 445 00:25:51,760 --> 00:26:09,760 And you'd be very useful to begin not just sort of integrating teams, but taking, you know, having architects or people from an app dev standpoint, at least think about, and again, you can't parachute in and start telling people what to do with their data, but at least start thinking about, you know, from a, from an outsider's perspective, right? 446 00:26:09,760 --> 00:26:13,760 All right. What are the kinds of things that we've learned building applications that can be applied to this world? 447 00:26:13,760 --> 00:26:25,760 Because obviously they're apples and oranges, right? It's not as if everything's going to transfer, you know, with full fidelity, but there'll be a lot of lessons learned, which is, you know, hey, this is what we used to do in sort of building applications and waterfall. 448 00:26:25,760 --> 00:26:35,760 And this is dumb. Like we just don't do things this way anymore. So, like I said, I think trying to, trying to extract and distill some of those lessons learned, you know, and impart that. 449 00:26:35,760 --> 00:26:45,760 And again, how you do this is critical because if you parachute in, like I said, and start telling everybody what they've been doing wrong for years and years and years, you're not going to make many friends. You're going to set your effort back. 450 00:26:45,760 --> 00:26:55,760 But if you can do it diplomatically, if you can begin to have that conversation with the data teams within your organization, I think you, it can at least kickstart the process. 451 00:26:55,760 --> 00:27:07,760 Because what you want ultimately is for the data teams to start thinking about, cool, you know, hey, the application development, men and women have been doing a great job of, you know, sort of modernizing what they do and constantly innovating and so on. 452 00:27:07,760 --> 00:27:15,760 You know, this is cool. We can do the same thing. You know, you really want them bought into, you know, how do we make our lives easier? 453 00:27:15,760 --> 00:27:17,760 Because that's ultimately what all this is about. We want to ship things faster. 454 00:27:17,760 --> 00:27:23,760 We want to do it with, you know, sort of not just better velocity, better efficiency. 455 00:27:23,760 --> 00:27:28,760 So we're not going to call in the middle of the night and on the data side that you can move just as quickly without losing data, right? 456 00:27:28,760 --> 00:27:29,760 You know, because that's the key here. 457 00:27:29,760 --> 00:27:40,760 Well, speaking of making people's lives easier in the middle of the night, if you haven't already subscribed to this podcast, what you should do is find the RSS feed over at soundcloud.com/pivotalconversations. 458 00:27:40,760 --> 00:27:43,760 And you can just subscribe to it and it'll automatically download. 459 00:27:43,760 --> 00:27:49,760 Now, I would recommend that you go to all of your devices and subscribe to it there so we can goose our download numbers. 460 00:27:49,760 --> 00:28:00,760 That's always appreciated. But if you just want to see the show notes for this, you can go to cote.io/conversations, probably 53 or two, just sort it out. 461 00:28:00,760 --> 00:28:06,760 And also, if you go to pivotal.io/podcast, you can see a list of all of the podcasts that we have, including this episode. 462 00:28:06,760 --> 00:28:12,760 And it's always great if you want to leave a review in iTunes or recommend it in Overcast. 463 00:28:12,760 --> 00:28:16,760 And with that, we'll see everyone next time. Bye bye.