1 00:00:00,000 --> 00:00:06,880 Well, we've got a super special episode this week, Richard, and I have to say, I have not 2 00:00:06,880 --> 00:00:08,020 prepared for it at all. 3 00:00:08,240 --> 00:00:12,980 So what I'm going to do is take the reins sort of like patinaed and withered and warm 4 00:00:12,980 --> 00:00:15,200 and comfortable as they are and just give them over to you. 5 00:00:15,420 --> 00:00:19,020 And then at the end of this, I know that with the August panel that we have, I'm going to 6 00:00:19,020 --> 00:00:21,180 be fully educated and ready to go back to coding. 7 00:00:21,380 --> 00:00:24,800 So I'm looking forward to it because I'm getting tired of slides. 8 00:00:24,980 --> 00:00:26,100 It's not working out for me. 9 00:00:26,100 --> 00:00:30,820 So with that, why don't you take the reins over and have everyone introduce themselves? 10 00:00:31,860 --> 00:00:32,320 Yeah, thank you. 11 00:00:32,340 --> 00:00:34,160 I'm going to be like the engineering whisperer. 12 00:00:34,160 --> 00:00:34,780 It's going to be great. 13 00:00:34,880 --> 00:00:36,580 They're going to do all their thing. 14 00:00:36,680 --> 00:00:37,820 And yeah, this is a very special episode. 15 00:00:37,940 --> 00:00:40,520 You're going to gather your kids around afterwards and explain it to them. 16 00:00:40,780 --> 00:00:41,980 So I'm glad we get to do one of these. 17 00:00:42,060 --> 00:00:45,960 We have some smart people on the line, so I'll quick call you out and ask you to do just a 18 00:00:45,960 --> 00:00:46,880 couple second intro. 19 00:00:47,140 --> 00:00:49,720 So Mr. Schaefer, can you tell everyone who you are? 20 00:00:50,700 --> 00:00:54,680 I'm Andrew, often known as Andrew Clay Schaefer on the internet. 21 00:00:55,680 --> 00:00:57,820 I work for Pivotal. 22 00:00:57,940 --> 00:00:59,580 I think everyone here works for Pivotal today. 23 00:00:59,700 --> 00:01:03,820 We're just going to talk about some of the things we see happening, things people are 24 00:01:03,820 --> 00:01:09,720 struggling with, with respect to managing deployments of or continuously deploying microservices 25 00:01:09,720 --> 00:01:10,660 or die trying. 26 00:01:11,860 --> 00:01:12,340 Awesome. 27 00:01:12,700 --> 00:01:13,660 Kenny, you are here. 28 00:01:14,240 --> 00:01:14,600 Say hello. 29 00:01:14,800 --> 00:01:15,060 Hey, guys. 30 00:01:15,740 --> 00:01:16,740 I'm Kenny Pistani. 31 00:01:16,840 --> 00:01:18,760 I'm a Spring Developer Advocate at Pivotal. 32 00:01:19,280 --> 00:01:23,820 I'm writing a book with Josh Long called Cloud Native Java, which has many of the topics 33 00:01:23,820 --> 00:01:25,420 we're going to be discussing today in the book. 34 00:01:25,920 --> 00:01:27,800 So it's a topic near and dear to my heart. 35 00:01:28,720 --> 00:01:29,020 Awesome. 36 00:01:29,280 --> 00:01:29,980 And Ali, you're on. 37 00:01:30,300 --> 00:01:30,640 Say hi. 38 00:01:31,280 --> 00:01:31,500 Yeah. 39 00:01:31,580 --> 00:01:31,780 Hi. 40 00:01:31,900 --> 00:01:32,820 I'm Oliver Gierke. 41 00:01:33,020 --> 00:01:36,000 I'm the project leader of the Spring Data Project at Pivotal. 42 00:01:37,440 --> 00:01:43,280 And outside my engineering role, oftentimes involved into customer projects, software architecture, 43 00:01:44,300 --> 00:01:46,000 REST, and distributed systems. 44 00:01:47,100 --> 00:01:47,200 Yeah. 45 00:01:48,480 --> 00:01:48,840 Fantastic. 46 00:01:49,320 --> 00:01:55,140 So we are going to talk a little bit, as Schaefer teed up, this idea of how do I deal with 47 00:01:55,140 --> 00:01:59,560 these, obviously, dynamic distributed systems, updating them, contracts, things like that. 48 00:01:59,620 --> 00:02:03,740 But before we jump into it, I do want to have us define a few terms so that those listening, 49 00:02:04,340 --> 00:02:06,220 you know, we're all working on the same set of definitions. 50 00:02:06,580 --> 00:02:09,260 I think we should actually back up a little bit. 51 00:02:09,260 --> 00:02:15,720 We should back up one step, which is the reason this podcast is happening is the three 52 00:02:15,720 --> 00:02:20,280 of us had a little bit of an open discussion on Twitter about some of these things. 53 00:02:20,460 --> 00:02:26,380 And I wouldn't say we were arguing, but we didn't necessarily agree on how some of this 54 00:02:26,380 --> 00:02:26,700 should work. 55 00:02:26,780 --> 00:02:30,160 So I think it'll be fun to talk through this with the whole group. 56 00:02:30,580 --> 00:02:31,080 No, that's great. 57 00:02:31,180 --> 00:02:34,740 Kenny was on all caps, which when we knew it was done, we weren't going to, we had to 58 00:02:34,740 --> 00:02:36,640 take it into the podcast because he was yelling. 59 00:02:36,960 --> 00:02:37,780 So no, this was great. 60 00:02:38,260 --> 00:02:42,300 We wanted to, you know, it was great that we had this sort of forum because 140 characters 61 00:02:42,300 --> 00:02:43,760 never let you get the nuance out. 62 00:02:43,760 --> 00:02:48,640 And Ali wrote a fantastic blog post, which also was about 1,400 characters or more to 63 00:02:48,640 --> 00:02:49,580 get some things out there. 64 00:02:49,740 --> 00:02:51,360 So yeah, thanks for teeing that up. 65 00:02:51,420 --> 00:02:54,080 So let's, let's talk about some of this first. 66 00:02:54,160 --> 00:02:55,740 Who wants to define contract for me? 67 00:02:58,340 --> 00:03:00,100 We should, we should go right to the dictionary. 68 00:03:00,680 --> 00:03:01,460 There you go. 69 00:03:02,160 --> 00:03:02,620 There you go. 70 00:03:02,680 --> 00:03:03,480 Break out your Websters. 71 00:03:04,740 --> 00:03:07,280 Ali, do you want to take that one since you wrote that post? 72 00:03:08,240 --> 00:03:10,640 Yeah, of course I can take, I can take it. 73 00:03:10,640 --> 00:03:18,360 So we were basically like getting into a discussion about like how systems are, interact or have 74 00:03:18,360 --> 00:03:23,780 to interact with each other, how they're basically coming up with an agreement of how they actually 75 00:03:23,780 --> 00:03:24,880 interact with each other. 76 00:03:25,020 --> 00:03:31,380 That means like besides defining the protocols they want to use to talk to each other, decide 77 00:03:31,380 --> 00:03:36,520 what kind of semantics they share, what kind of shared understanding about like the data 78 00:03:36,520 --> 00:03:42,080 they share and the business processes that are actually implemented through that communication. 79 00:03:42,680 --> 00:03:48,200 And contracts and especially consumer driven contacts, the topic that we're going to get 80 00:03:48,200 --> 00:03:54,360 into are well or getting kind of a lot of intention these days. 81 00:03:55,160 --> 00:04:02,000 And we basically got into some argument about the nuance in terms of the effects of applying 82 00:04:02,000 --> 00:04:11,400 that kind of contract definition on the overall architecture and on the teams, how they communicate 83 00:04:11,400 --> 00:04:12,060 with each other. 84 00:04:12,340 --> 00:04:19,180 And that's what basically struck a chord to let me write that blog post, which is basically 85 00:04:19,180 --> 00:04:27,520 about evolving distributed systems in general, and the forces that are like applied to that 86 00:04:27,520 --> 00:04:35,120 to that topic in general, why you actually distribute systems or separate or divide systems into parts 87 00:04:35,120 --> 00:04:40,240 and what approaches to take to evolve them then independently. 88 00:04:40,660 --> 00:04:42,960 And contracts, of course, play a role in that, as I said. 89 00:04:43,200 --> 00:04:47,820 I mean, Kenny, how would you take a contract for someone to say, OK, when we talk contracts, 90 00:04:47,820 --> 00:04:49,580 we're not talking legal terms necessarily. 91 00:04:49,800 --> 00:04:53,180 But what's the contract to you when you have a service definition? 92 00:04:54,140 --> 00:04:58,700 So the contract is something that a service is going to publish that tells all of its potential 93 00:04:58,700 --> 00:05:05,040 consumers what exactly is the interface for communicating with that service. 94 00:05:05,240 --> 00:05:11,180 Now, you can take it further and say that in the case of consumer driven testing, that a certain 95 00:05:11,180 --> 00:05:14,900 method, an API method, has an expected response. 96 00:05:15,060 --> 00:05:21,140 Now, the producer can publish that as a stub to let its consumers know exactly how to interact 97 00:05:21,140 --> 00:05:24,680 with that API and for what conditions it can expect certain responses. 98 00:05:27,200 --> 00:05:28,380 And so that's the main. 99 00:05:28,480 --> 00:05:30,580 So if somebody says, who cares about contracts? 100 00:05:31,780 --> 00:05:35,040 They would say that because they don't want to deal with that or they don't like, you know, 101 00:05:35,040 --> 00:05:39,800 they have scars from soap and wisdom times or, you know, for someone who doesn't look to 102 00:05:39,800 --> 00:05:41,920 those things, why look at contracts? 103 00:05:42,420 --> 00:05:48,120 Yeah, I think I think the core of the nuance that we were just having our Twitter discussion 104 00:05:48,120 --> 00:05:55,200 about is what's the implications with respect to coupling and what can be changed independently 105 00:05:55,200 --> 00:05:56,120 and what can't. 106 00:05:57,140 --> 00:06:00,800 And just to just to pull because I did pull up dictionary dot com. 107 00:06:01,620 --> 00:06:06,280 The number one definition under contract is an agreement between two or more parties for 108 00:06:06,280 --> 00:06:08,660 doing or not doing something specified. 109 00:06:08,660 --> 00:06:14,800 And I think that kind of fits under, you know, an umbrella for a lot of the stuff we're talking 110 00:06:14,800 --> 00:06:15,180 about. 111 00:06:15,600 --> 00:06:22,140 But, you know, Kenny went right to this thing that would get exposed by an API that would 112 00:06:22,140 --> 00:06:29,080 be able to, in some cases, even negotiate what the contract will be for that particular 113 00:06:29,080 --> 00:06:30,940 request, that particular session. 114 00:06:32,260 --> 00:06:36,440 But in general, there's lots of things that are kind of contracts that go that go unspecified 115 00:06:36,440 --> 00:06:42,700 and unspoken about, you know, just the fact that it's even at HTTP requests or some of 116 00:06:42,700 --> 00:06:47,380 these other like protocol decisions have implications about the contract that you could keep. 117 00:06:48,640 --> 00:06:52,800 Now, I mean, this comes up more and Ali, your post hit on this, because when we talk about 118 00:06:52,800 --> 00:06:57,640 microservices, whether we want to use that term or system of systems, as you framed it, that 119 00:06:57,640 --> 00:07:02,040 as you start teasing apart these monoliths, you end up introducing a lot of new communication 120 00:07:02,040 --> 00:07:05,620 if you don't refactor or make these sort of self-inclusive systems. 121 00:07:05,620 --> 00:07:10,820 And if you want to reduce some of that communication, so maybe these contracts matter more as you have 122 00:07:10,820 --> 00:07:12,400 more services, more communication. 123 00:07:12,680 --> 00:07:17,320 Do you want to summarize just quickly kind of the position you put out there around teasing 124 00:07:17,320 --> 00:07:20,200 apart systems and considering what that communication is? 125 00:07:20,280 --> 00:07:25,780 And then to Andrew's point, what happens now when you have these systems with contracts between 126 00:07:25,780 --> 00:07:26,000 them? 127 00:07:26,080 --> 00:07:27,740 How do you start to deal with deployment concerns? 128 00:07:28,860 --> 00:07:29,040 Yeah. 129 00:07:29,380 --> 00:07:31,440 So let me start with the first thing. 130 00:07:31,440 --> 00:07:39,300 So what we started discussing was, OK, what if you if you like if you're dividing a monolith 131 00:07:39,300 --> 00:07:45,200 into into separate systems, right, no matter if it's like two or 10 or something, the one 132 00:07:45,200 --> 00:07:51,240 thing that that fundamentally changes is the amount of effort that you have to communicate 133 00:07:51,240 --> 00:07:52,180 with the other system. 134 00:07:52,620 --> 00:07:56,980 So if you have a monolithic Java application, like talking to a different part of the system 135 00:07:56,980 --> 00:07:58,120 is basically a method call. 136 00:07:58,200 --> 00:07:59,360 There's no network in between. 137 00:07:59,500 --> 00:08:00,020 There is nothing. 138 00:08:00,300 --> 00:08:05,860 If you have like that split up into systems, you now have a network in between which can 139 00:08:05,860 --> 00:08:06,200 fail. 140 00:08:06,340 --> 00:08:07,500 The other system can fail. 141 00:08:08,200 --> 00:08:13,680 You have to marshal your data into some JSON or some some serialization format. 142 00:08:13,820 --> 00:08:21,760 So if you keep the same communication patterns around for like this in systems, inter systems 143 00:08:21,760 --> 00:08:28,680 communication, then you'll run into issues because you you accumulate a lot of latency and you 144 00:08:28,680 --> 00:08:30,740 generally want to communicate less, right. 145 00:08:30,740 --> 00:08:35,360 So it's like in a distributed systems world, you want to have more coarse grained requests 146 00:08:35,360 --> 00:08:39,820 and you want to communicate less often, really. 147 00:08:40,000 --> 00:08:47,680 So that might already change the the approaches to architecture quite a lot because you might even 148 00:08:47,680 --> 00:08:53,300 want to design your system in a way that it doesn't actively reach out to other systems, but rather publish 149 00:08:53,300 --> 00:08:54,540 events or something. 150 00:08:54,540 --> 00:09:00,320 But even if you go with events and the same messaging, right, you still the systems have 151 00:09:00,320 --> 00:09:07,460 to agree on on on something because as Kenny already mentioned, even if you don't formally go ahead and define 152 00:09:07,460 --> 00:09:13,720 like the shared semantics, you will end up with something that's just an implicitly defined. 153 00:09:13,860 --> 00:09:30,680 Right. So the fields that that adjacent document has, for example, and then you, of course, do yourself a favor to to negotiate that between the involved parties and and define those define those things and then be able to to make some guarantees around that. 154 00:09:31,340 --> 00:09:43,500 The thing that I was kind of slightly disagreeing or not even disagreeing, I was kind of questioning or just wanted to to put an attention on the on the topic is that it's quite important. 155 00:09:43,500 --> 00:10:03,680 So the level of coupling you create with that is totally is very much depending on what exactly you put into that contract and what what aspects you focus on when defining the contract and different approaches to that basically having different consequences in the in the amount of coupling you create. 156 00:10:04,620 --> 00:10:16,680 So I want to point out one quick thing, and then I heard you had a question, Kenny, but the the things that all of us have to kind of reconcile across all this is there's tension. 157 00:10:16,680 --> 00:10:29,680 And you mentioned this in the blog post that what you would do to optimize solely for performance and latency might be different than what you would do to optimize for, you know, this this flexibility and decoupling. 158 00:10:30,440 --> 00:10:39,780 And just kind of like upfront recognized that there's definitely tension and compromises have to be made when when you're when you're engineering these things. 159 00:10:39,780 --> 00:10:42,940 So there's not necessarily a right answer or a wrong answer. 160 00:10:42,940 --> 00:10:47,300 It's a question of what are you optimizing for in this specific case? 161 00:10:48,120 --> 00:10:48,380 Definitely. 162 00:10:48,680 --> 00:10:48,780 Yeah. 163 00:10:48,860 --> 00:11:00,100 So that was it was definitely one of the core drivers that we also got into discussion on Twitter about like what's the or actually we didn't get into discussion about that on Twitter. 164 00:11:00,260 --> 00:11:04,500 Not explicitly, but it's sort of flew around in between the tweets. 165 00:11:04,500 --> 00:11:09,000 So like the question of like, why do you actually separate the systems in the first place? 166 00:11:09,100 --> 00:11:10,260 What's your actual goal? 167 00:11:10,740 --> 00:11:14,820 And how does your your your your architecture actually support that? 168 00:11:14,900 --> 00:11:21,480 And I think with with microservices, it's usually that you want to to enable the different teams to ship features independently. 169 00:11:22,040 --> 00:11:28,540 And you the teams get more independently, the less they have to talk to. 170 00:11:28,640 --> 00:11:31,660 And that's probably then something that Andrew is going to disagree with me. 171 00:11:31,740 --> 00:11:33,900 The less you have to you have to talk to others. 172 00:11:33,900 --> 00:11:36,260 That's probably the nuance I want to emphasize here. 173 00:11:36,380 --> 00:11:39,140 I'm not saying you don't want to communicate. 174 00:11:39,620 --> 00:11:46,680 But if if you're forced to talk to to other parties in in in that scenario, that's that's not it. 175 00:11:46,700 --> 00:11:48,500 That's not a good thing in the first place. 176 00:11:48,500 --> 00:11:50,020 That's rather something you want to avoid. 177 00:11:50,020 --> 00:11:50,340 Right. 178 00:11:50,820 --> 00:12:01,040 But of course, if you if you if you have to agree or have to talk at some at some points, because you you have to define like the shared semantics of APIs. 179 00:12:01,040 --> 00:12:04,760 But yeah, I think that's it. 180 00:12:04,920 --> 00:12:08,180 So I think I want to add one thing there, Andrew. 181 00:12:08,900 --> 00:12:17,460 Just to make sure that we kind of follow that thread of what you're talking about, the expectations between certain microservices that expose REST APIs. 182 00:12:17,460 --> 00:12:27,320 Now, Ali, you having worked on and led the Spring Data REST project, you found yourself quite familiar with hypermedia, specifically Hot OS. 183 00:12:27,560 --> 00:12:34,320 Can you speak to how Hot OS can help alleviate the problem of having these contracts between services? 184 00:12:35,720 --> 00:12:58,340 Yeah, it's also I mentioned that in the in the blog post or it's more in the end where I basically get to the point where what I propose like in a maybe not as an alternative, but what I propose to to focus more on when it comes to the definition of these kind of contracts in the broader sense of like, what do we agree on like between systems? 185 00:12:58,340 --> 00:12:58,640 Right. 186 00:12:59,040 --> 00:13:05,400 So the the aspect of hypermedia in REST systems is often a very overlooked one. 187 00:13:05,720 --> 00:13:13,120 But it's actually a very, very effective means to reduce the amount of coupling between systems. 188 00:13:13,320 --> 00:13:19,440 And the way it does that is, I mean, there's like it's a two level thing, basically. 189 00:13:19,440 --> 00:13:27,480 The first thing is adding hypermedia means to the to your representation, which means you would add links to your JSON, just like a web page has links. 190 00:13:27,480 --> 00:13:33,540 You would go ahead and use one of the of the of the of the formats that that that are around. 191 00:13:33,540 --> 00:13:38,040 There's a couple of them in JSON hell and collection JSON and Siren and whatnot. 192 00:13:38,940 --> 00:13:47,220 But it basically enables you to put links into the responses and the client could actually follow them that in the first place basically only links data to data. 193 00:13:47,220 --> 00:13:51,120 So that's there's nothing really, really fancy, but already helpful. 194 00:13:51,120 --> 00:14:15,560 And the tricky part and the one that's that's actually one of the most important ones in REST in terms of the the evolvability aspect of APIs is the so-called hypermedia as the engine of application state, which basically means that depending on the resources state, you would add a link to to the representation. 195 00:14:15,560 --> 00:14:19,080 And when that state changes, you might even remove it. 196 00:14:19,080 --> 00:14:27,560 And that can be used to basically signal the client that it can do something under certain conditions. 197 00:14:28,560 --> 00:14:31,860 And you might ask, ask yourself, OK, why? 198 00:14:32,020 --> 00:14:44,960 How does that help in terms of evolvability that helps because the client doesn't have to look at the JSON content, maybe the individual fields in too much detail and then basically replicate logic. 199 00:14:44,960 --> 00:14:52,380 That is already on the server itself, which the replication actually would lead to the coupling. 200 00:14:52,480 --> 00:14:52,640 Right. 201 00:14:52,740 --> 00:14:53,980 So let me give you an example. 202 00:14:53,980 --> 00:15:04,300 There's like let's say you have this this kind of order in your in your system and there's a business rule that says the order can only be canceled if it's not paid yet. 203 00:15:04,480 --> 00:15:04,860 Right. 204 00:15:04,920 --> 00:15:08,480 And there's a human readable status field in the representation. 205 00:15:08,480 --> 00:15:10,920 It says payment expected or something. 206 00:15:11,260 --> 00:15:11,380 Right. 207 00:15:11,500 --> 00:15:13,580 And while that's the case, it can be canceled. 208 00:15:13,580 --> 00:15:24,600 So if you now bake the logic into the client and literally program it like if the status field is payment expected, then present the payment button, let's say for a mobile client or something. 209 00:15:25,000 --> 00:15:29,380 Then you've added you've added you've added additional semantics to that field. 210 00:15:29,380 --> 00:15:29,700 Right. 211 00:15:29,860 --> 00:15:42,020 Stuff that's not even in a in a contract somewhere, not not something it's probably documented somewhere, but it's not not technically in a in a in a in a JSON schema or something because you just can't define it that way. 212 00:15:42,420 --> 00:15:48,660 If you then have to, for some reason, change the the the the field or the the condition. 213 00:15:49,020 --> 00:15:49,180 Right. 214 00:15:49,760 --> 00:15:52,840 Let's say we can now cancel orders if they are paid already. 215 00:15:52,840 --> 00:15:53,220 Right. 216 00:15:53,640 --> 00:16:05,680 Then that that client replicating that logic becomes a problem because now we have to communicate the that change to the client and then potentially negotiate a new contract or whatnot. 217 00:16:05,980 --> 00:16:19,000 And if you're instead introduce hypermedia means and only tell the client if there is a cancel link in the representation, you can issue a delete request to issue the cancellation. 218 00:16:19,000 --> 00:16:19,440 Right. 219 00:16:19,840 --> 00:16:25,220 Then that's canceling can appear in different in different other scenarios. 220 00:16:25,220 --> 00:16:35,980 And the change in when that cancellation is actually out is basically condensed into a Boolean yes, no question on the client. 221 00:16:36,100 --> 00:16:36,220 Right. 222 00:16:36,240 --> 00:16:38,700 It just needs to look for is the link there or not. 223 00:16:39,040 --> 00:16:40,420 And if so, I could do something. 224 00:16:40,520 --> 00:16:43,700 If not something else, I can't do it. 225 00:16:43,700 --> 00:16:43,900 Right. 226 00:16:44,000 --> 00:16:54,220 So and that helps helps the to to basically to keep a lot of the the logic on the on the server or the details of that logic, basically. 227 00:16:55,220 --> 00:17:13,640 Um, and actually allows it to or makes the the this the scenario of a breaking change in a in in some in some in some in some logic less likely or that it yeah, makes it less likely that that you run into that breaking change scenario. 228 00:17:15,120 --> 00:17:16,140 Does that make sense? 229 00:17:16,440 --> 00:17:19,160 Yeah, I agree with everything you said and it definitely makes sense. 230 00:17:19,160 --> 00:17:37,380 Because the the thing that is kind of interesting, or what I was reacting to with respect to communication is when when you introduce something and you gave this example of the cancel, and that's going to be, you know, in there's more than one way to solve this problem. 231 00:17:37,380 --> 00:17:51,740 But, um, I think it's actually pronounced hatios, but whatever, um, you know, going forward, you still have to tell like someone else on the client to implement any logic to be able to use that. 232 00:17:51,740 --> 00:18:02,140 So, so certainly introducing those things and not breaking, I think generally, there's a bunch of tricks you can use with respect to how fields appear and what the logic is. 233 00:18:02,140 --> 00:18:16,100 And, you know, that, that could be, um, some implicit, um, but, but the kind of general Postel's law robustness principle still applies that you want to be conservative in what you do, and you want to be liberal in what you accept from others. 234 00:18:16,100 --> 00:18:21,100 And that solves like a bunch of problems people run into with really strong coupling. 235 00:18:21,100 --> 00:18:38,380 And I think what you were reacting to is when, when you do things in such a way that it creates a lot of unnecessary communication, it creates a lot of overhead that is, it's not inherently adding any value. 236 00:18:38,600 --> 00:18:46,640 It's just there because it's part of the, the quote unquote process of the contract that, that I also have a visceral reaction against. 237 00:18:47,280 --> 00:18:47,380 Yeah. 238 00:18:47,660 --> 00:19:08,120 I mean, the, the, what, what, um, um, um, one aspect coming up here is that if you, if you, if you just from the example I mentioned, you, uh, you realize it's, it's an, it's an, it's an, it's been some functionality that's like actually implementing more, more business value. 239 00:19:08,120 --> 00:19:11,520 Let's say like the, the, the, the, the cancellation of an order, right. 240 00:19:11,540 --> 00:19:17,640 That that's not just like an, an arbitrary, oh, we get some Jason, we change some values and we put it back. 241 00:19:17,660 --> 00:19:17,900 Right. 242 00:19:17,900 --> 00:19:20,040 So we poked at, at the data basically. 243 00:19:20,320 --> 00:19:27,080 It's more looking at the API from a, okay, the API allows me to actually implement business functionality. 244 00:19:27,080 --> 00:19:33,680 So it's not a, not a, not really a sort of a mimicking a data access API over HTTP really. 245 00:19:33,680 --> 00:19:49,480 Um, and that's probably, um, that's a, that's a, that's a crucial point because if you're not writing these kinds of, um, these kinds of APIs, then that benefit might not even be as, um, as significant to you. 246 00:19:49,480 --> 00:19:49,740 Right. 247 00:19:49,740 --> 00:19:56,480 And I just read up the, the, the, the original, uh, article from Ian about consumer driven context. 248 00:19:56,640 --> 00:19:58,480 He has this notion of a conversation. 249 00:19:58,480 --> 00:20:02,700 So when between the client and the server, there's more of a high level conversation. 250 00:20:02,700 --> 00:20:09,320 Um, then that's kind of approach is kind of, um, is, is, is, is very useful. 251 00:20:09,420 --> 00:20:13,600 So, whereas in other scenarios, which are, which might be perfectly fine, right. 252 00:20:13,600 --> 00:20:18,000 Where you just want to have an HTTP interface to your, to your data, right. 253 00:20:18,260 --> 00:20:20,820 Uh, just because you want to add security to it or whatnot. 254 00:20:20,820 --> 00:20:32,200 Um, and that, that, that those high level conversations are not the main driver, um, then the hypermedia aspect might not be as important to you, but then you create more coupling between the services. 255 00:20:32,840 --> 00:20:41,980 Um, but if you just, if you're aware of that and, um, um, deal, deal with it accordingly, then, then that's just fine. 256 00:20:41,980 --> 00:20:48,660 It's just, um, I just don't want to like create the impression that there's like a black and white and here's the right thing to do always. 257 00:20:49,140 --> 00:20:54,560 Uh, it's, it's more of a, you have to find out what kind of, of API you're actually creating. 258 00:20:54,920 --> 00:21:09,920 And I'd argue the more coarse grained your systems are and the more coarse grained the systems that interact are, uh, the more you actually want to expose real business functionality via the API and the more important those, those aspects actually get. 259 00:21:09,920 --> 00:21:16,280 The more, the more, the more you get into the smaller and basically let's say have a, like your, your shopping system. 260 00:21:16,280 --> 00:21:21,440 And then you realize you have to like scale your, your reads and writes differently. 261 00:21:21,440 --> 00:21:36,620 You might even want to split up them internally, like, like have the, the, the business split on the, on the macro level and then internally even, um, create different, uh, different services, uh, just to be able to scale them differently. 262 00:21:36,620 --> 00:21:47,560 And then it's a totally different scenario because you're, you're, you're controlling the scope, like on a, on a different level and, um, um, can then also live with a, with a bit more stronger company. 263 00:21:47,560 --> 00:22:00,480 You might not even want to use rest in that case, but like a more, uh, more, um, efficient protocol here because like on the, on the, on the smaller scale, the, the performance implications might be different ones as well. 264 00:22:00,480 --> 00:22:04,180 So, um, yeah, trade-offs as always, right? 265 00:22:06,120 --> 00:22:06,520 Exactly. 266 00:22:10,520 --> 00:22:10,880 Good. 267 00:22:11,040 --> 00:22:16,280 So, yeah, I mean, Kenny, I want to make sure Kenny Schaefer, you have other thoughts on that before we dig in. 268 00:22:16,280 --> 00:22:17,460 Yeah, I think I'm fine with that. 269 00:22:17,560 --> 00:22:22,980 I, I, I think we can, if Andrew doesn't have anything to add to that, I think we can move on to kind of the next topic. 270 00:22:23,240 --> 00:22:32,420 Well, actually, I want to, I don't know if this is moving on to the next topic, but I kind of want to introduce a scenario and, and, and we'll throw it back and forth about what people think you should do. 271 00:22:32,420 --> 00:22:45,620 Uh, so, so you have some service, has APIs, it's, it's, I don't know if we need to get more specific about, you know, functionality or canceling or whatever, um, that functionality is, but things talk to things over network. 272 00:22:45,620 --> 00:22:46,800 There's, there's, there's clients. 273 00:22:47,240 --> 00:22:53,680 And at some point you're successful and you have a bunch of clients and maybe some of those clients are third party. 274 00:22:54,980 --> 00:22:58,440 And, and you, you need to make changes or you want to introduce functionality. 275 00:22:59,020 --> 00:23:08,120 What, what is your general strategy to, uh, either support backwards compatibility or, you know, kind of broadcast, negotiate. 276 00:23:08,460 --> 00:23:14,140 And people have tried a bunch of different things and you brought up in your blog post, which is also one of my pet peeves. 277 00:23:14,140 --> 00:23:19,020 Uh, people start putting, you know, slash V1, slash V2 in the endpoints. 278 00:23:19,220 --> 00:23:23,620 They do like a bunch of, I don't know, there's a bunch of like weird version juggling things. 279 00:23:23,680 --> 00:23:25,620 People try to do this, solve this problem. 280 00:23:25,620 --> 00:23:33,280 But I'm, I'm curious to see or hear what you, what you think the options are or what you would advocate, uh, to solve this problem. 281 00:23:34,360 --> 00:23:40,600 Um, to be completely honest, the, the, the, the point I'm, I'm, that's not the point that I made really. 282 00:23:40,660 --> 00:23:41,740 I've just referred to it. 283 00:23:41,860 --> 00:23:47,380 Uh, it's, it's getting back to one of the interviews with the Roy, actually Roy guy behind the rest dissertation. 284 00:23:47,880 --> 00:24:02,660 Um, is that it, the, the problem is not really like which means to choose, but that you're actually better off if you're not thinking of versions, but rather, uh, consider the thing that you build. 285 00:24:02,660 --> 00:24:16,680 If you, if you have a breaking change or, and you need to, to ship it, um, think of it as a completely new service that by actually doesn't actually have any relationship to the, to the, to the, to others. 286 00:24:16,680 --> 00:24:29,040 Because then all of a sudden you, you're not creating these, these implications about like, okay, uh, what, what, what parts of, or which, which resources of that API do actually change? 287 00:24:29,100 --> 00:24:36,000 What about, but the, the caching state, uh, can I actually negotiate between, between the two quote unquote versions? 288 00:24:36,240 --> 00:24:43,280 Um, or, um, is it, is it just better to, to think of it as a, as a completely different, different service? 289 00:24:43,440 --> 00:24:49,340 So, I mean, there, you, you already can just like go ahead and use different like host names or something. 290 00:24:49,340 --> 00:24:49,720 Right. 291 00:24:49,780 --> 00:25:03,080 So that's, again, this is, again, uh, rather considering it a different service than a version of the same thing, because, um, then the clients just like get, get all confused about, okay, which version do I request? 292 00:25:03,140 --> 00:25:04,460 And what kind of answer do I get? 293 00:25:04,580 --> 00:25:16,400 And in the end, it doesn't really matter if you're using them like an HTTP header or a, a, a path segment or a subdomain for, for, um, for, uh, your services. 294 00:25:16,640 --> 00:25:32,960 Uh, because yeah, the client will have to just detect it either way and then, uh, talk to it and potentially be, is potentially able to, to reuse part of the, of the logic, but it will also have to just like rewrite other parts of it. 295 00:25:33,020 --> 00:25:34,860 Basically the ones that, that you broke. 296 00:25:34,860 --> 00:25:51,480 But just to play devil's advocate, that's kind of at odds with, uh, the recommendation from the Restafarians that you should just have that one be a canonical, you know, like the, the argument people often make is there's, there's only one website, right? 297 00:25:51,480 --> 00:25:54,240 And that, that website's going to change over time. 298 00:25:54,540 --> 00:25:54,980 Right. 299 00:25:54,980 --> 00:26:04,540 So, so like you don't, you wouldn't all of a sudden have, you know, another, another website for your company just because you wanted to update the, the thing and like keep that one. 300 00:26:04,760 --> 00:26:06,620 So like old clients would go to the old one. 301 00:26:07,200 --> 00:26:07,620 Yeah. 302 00:26:07,700 --> 00:26:18,820 I mean, of course the, the, the, the further you can get away with, let's say a content negotiation or something with, with version media types and something that's, that's something you can do. 303 00:26:18,820 --> 00:26:35,600 But at the same time, um, if you're, if you're, if you're, if you're deploying a new website of your company, you're actually not doing a breaking or you're not, you're not introducing a breaking change because it's still just the website. 304 00:26:35,700 --> 00:26:43,020 You just go to the root site and you get something and then you, you find your ways through, through the website by following the links and stuff. 305 00:26:43,100 --> 00:26:53,400 So, I mean, having, having been, uh, the victim of a new website, uh, more than once in my life, there's definitely breaking changes from a human usability perspective. 306 00:26:53,400 --> 00:27:00,980 When you've got a routine that you're, you're doing on a, on a website that has a certain user interface and that changes it from under you. 307 00:27:01,200 --> 00:27:01,580 Right. 308 00:27:01,900 --> 00:27:11,120 I'm not sure that that principle is that much different than, than the, the, the, the adding a cancel button to the, to the, the, the payload. 309 00:27:11,600 --> 00:27:15,100 The point I'm trying to make is that what do you do as a human being? 310 00:27:15,240 --> 00:27:23,560 If you find yourself like with going to eBay, like after a decade or something and not finding the buttons at the places that, that you used to find them, they're either gone, right? 311 00:27:23,840 --> 00:27:29,180 I mean, they, they, they basically just dropped functionality or they moved it to some other place. 312 00:27:29,280 --> 00:27:33,100 And then you have to find some, discover it to some, to some degree. 313 00:27:33,640 --> 00:27:40,620 Uh, but it's not that you, you, you, you, you, you exchange the, the eBay website with a eBay website. 314 00:27:41,300 --> 00:27:45,920 And then you all of a sudden need to be reprogrammed completely as a, as a user. 315 00:27:45,920 --> 00:27:47,880 You just like go ahead and still use it. 316 00:27:48,520 --> 00:27:57,220 And it, it, it, it basically, uh, because from, from the web point of view, from the website point of view, nothing has changed. 317 00:27:57,220 --> 00:28:00,660 It's still HTML and HTTP and get and post requests. 318 00:28:00,660 --> 00:28:17,980 It's really, um, the point is actually that that's a, that that's, it's, it's not a good, a website change is not a good example for breaking the API because a breaking change in the API is usually, um, semantics of the fields changing, right? 319 00:28:17,980 --> 00:28:22,280 Or, uh, some, some, some, the structure may be in the JSON change. 320 00:28:22,840 --> 00:28:28,260 Um, but that's not going to happen in the, uh, in HTML. 321 00:28:28,260 --> 00:28:30,580 And also the HTML is still HTML. 322 00:28:30,780 --> 00:28:32,220 So the media type is still the same. 323 00:28:33,120 --> 00:28:36,200 Um, do you see the supportability change there though? 324 00:28:36,300 --> 00:28:44,960 I mean, is it realistic to have teams maintain three or four separate services instead of a, I mean, maybe it's the same as maintaining versions, but it seems like an operational burden as well goes on there. 325 00:28:45,360 --> 00:28:45,700 Exactly. 326 00:28:45,780 --> 00:28:49,780 I mean, that's exactly the point that they, um, there is a great blog post. 327 00:28:49,820 --> 00:29:09,060 I think we can, we can link to that in the show note, uh, the, the, the hidden costs of versioning APIs where someone, uh, took a really like, um, he did some, some scientific analysis about what or in how far the effort of maintaining API, API versions in parallel actually increases. 328 00:29:09,060 --> 00:29:13,220 It, it, it, it's, it's not increasing linearly as you can probably imagine. 329 00:29:13,720 --> 00:29:27,000 And, uh, he's basically making the point that it's, it's, it's worth investing, um, the time to, uh, to try to avoid that because it's, because, because of the fact that it's so much effort. 330 00:29:27,000 --> 00:29:29,800 And it's, it's, it's actually replicating into the clients, right? 331 00:29:29,800 --> 00:29:44,940 Because, um, they might, might have to, um, support it, have to be upgraded as well to the, uh, to the, uh, to the new version, because it's not only the effort to create a new version of the API, but also, uh, the, all the clients. 332 00:29:44,940 --> 00:29:48,200 And sometimes you're not even aware of, of, of clients. 333 00:29:48,200 --> 00:30:02,640 And, um, um, if you, let's say you have a public API, right, you don't even know how many clients are out there that are actually using the API and, and, and all these things complicate the matter. 334 00:30:02,780 --> 00:30:05,840 So that's, that's, that's, that's all arguments. 335 00:30:05,840 --> 00:30:12,740 Why I'd argue you, you want to avoid that as much as possible, uh, those breaking changes. 336 00:30:12,740 --> 00:30:21,080 And, uh, oftentimes the APIs I see, or there's, there's a lot of potential or would have been a lot of potential to actually avoid that in the, in the, in the first place. 337 00:30:21,080 --> 00:30:29,920 Um, but yeah, what, Kenny, what do you think about versioning, uh, sort of system of systems or microservices world? 338 00:30:30,680 --> 00:30:40,720 Yeah, I think in theory, you can say a lot of things about versioning and how backwards compatibility is going to affect the rate at which your consumers are going to be able to upgrade. 339 00:30:40,720 --> 00:30:43,540 But in reality, things are messy in enterprise software. 340 00:30:43,680 --> 00:30:53,900 So you can't necessarily, um, kind of put a standard forth that says we're not going to version our APIs because you'll always run into this problem where you have this brand being changed. 341 00:30:54,400 --> 00:31:07,480 Maybe it's a, uh, thinking incorrectly about the design of the API and you want to be able to support future changes from, from more consumers to be able to utilize that functionality, uh, in the way that was intended. 342 00:31:07,480 --> 00:31:13,600 And you're just always going to have those clients that, that need, just need that V1 compatibility and they don't need to upgrade. 343 00:31:13,600 --> 00:31:17,360 So in, in practice, versioning your APIs is fine. 344 00:31:19,820 --> 00:31:28,640 I, I think that the way I would frame this right now, and this is the way I think about all these problems is there's no such thing as a free lunch. 345 00:31:29,120 --> 00:31:41,000 And, and, and all these are points of, of what you're going to, so, so this notion of the hidden cost, if you, if you think you're going to save something in one regard, you often end up paying for it somewhere else. 346 00:31:41,380 --> 00:31:46,680 And, and, and so just be thoughtful about what you're trying to optimize and whose life you're trying to optimize, right? 347 00:31:46,680 --> 00:32:00,180 So I definitely have this pet peeve for, uh, the versioning in the API or the versioning in the endpoints or, you know, the thing people do if they put a .m in front and redirect you for your mobile device. 348 00:32:00,180 --> 00:32:22,320 Uh, I, I don't personally, like, it's annoying to me as a user, um, but someone did that because they're optimizing for their local, you know, lifestyle for their, for their experience as a developer or whatever, and had some, some operational or developer benefit that they were optimizing for, um, that might make someone else pay that price in another way. 349 00:32:22,660 --> 00:32:30,660 Right. So, Kenny, so let's, you know, a lot of our customers are starting to do some pretty big re-platforming efforts. 350 00:32:30,880 --> 00:32:38,040 I mean, we're, I tweeted out something today, one of our customers doing a, something with millions of lines of code that it's worth a billion dollars a year to them as an application. 351 00:32:38,040 --> 00:32:41,480 They're still refactoring it into microservices, doing some other things. 352 00:32:42,120 --> 00:32:46,380 How, how do you advise people to, you know, what are those tangible things up front? 353 00:32:46,400 --> 00:32:48,380 Should they use Spring Cloud Contract if they're using Java? 354 00:32:48,500 --> 00:32:50,520 Should they be, you know, what are they supposed to be doing? 355 00:32:50,520 --> 00:32:54,880 You know, where are those pitfalls that you say, gosh, especially up front, don't do this right away. 356 00:32:55,180 --> 00:32:57,900 And it might feel like premature optimization or things like that. 357 00:32:57,960 --> 00:33:07,480 Help people who are starting to tackle some pretty big efforts to get into more microservices or getting more into cross-service communication, not make mistakes that they're going to regret 12 months from now. 358 00:33:07,900 --> 00:33:09,780 So, I think this is a really interesting topic. 359 00:33:10,180 --> 00:33:10,920 It's a good question. 360 00:33:11,560 --> 00:33:18,080 I think right now there's just a lot of confusion around how do you effectively test a microservice architecture. 361 00:33:18,080 --> 00:33:24,320 I don't think we've seen the patterns emerge yet for how that's done, especially cross-platform. 362 00:33:24,520 --> 00:33:27,960 If you're using different languages, if you're running in a polyglot environment. 363 00:33:28,240 --> 00:33:48,020 So, what contracts give you, in a way, and I think directly to how we implement them in Spring Cloud contracts, which is a little bit different than what Ian Robinson had posted with Martin Fowler on his website, which really takes this perspective of, I'm going to publish the functionality of my service as a contract stub. 364 00:33:48,020 --> 00:33:55,860 I'm going to, as a producer, assert against that functionality so you know that in production, this is how I intend to use that API. 365 00:33:56,900 --> 00:34:01,100 Now, the consumers don't have to implement the same exact test. 366 00:34:01,220 --> 00:34:05,640 They can implement just certain parts of that test or they can assert against different conditions. 367 00:34:06,100 --> 00:34:21,120 But essentially, you're going to be able to publish that contract as a producer, as a test, that your consumers can just download as they go through their build and test environment and then run integration tests against that functionality. 368 00:34:21,820 --> 00:34:25,520 Now, this doesn't couple you to that service's expectations. 369 00:34:25,820 --> 00:34:28,460 You're just coupled to how you expect to use it. 370 00:34:28,620 --> 00:34:33,600 And not exactly in production, but your advancement into the production environment. 371 00:34:33,740 --> 00:34:35,920 So, you have to go through that promotion process to get there. 372 00:34:41,320 --> 00:34:42,700 Schaefer, do you have war wounds from this? 373 00:34:43,100 --> 00:34:48,340 How are you going to keep somebody from shaking their head 12, 18 months from now going, that was a poor choice up front? 374 00:34:49,460 --> 00:34:56,720 I think, honestly, if you're going to be involved in computers and software, you're going to have that problem, no matter what you do. 375 00:34:57,000 --> 00:35:02,720 Because today's best practices are always tomorrow's legacy code. 376 00:35:02,720 --> 00:35:06,860 But where do you see premature optimization, that people sometimes start too quick? 377 00:35:07,840 --> 00:35:08,480 Do you see any here? 378 00:35:08,900 --> 00:35:15,500 I mean, I think that the, kind of going back to some of all these original responses, 379 00:35:16,340 --> 00:35:26,700 I think that there's things that are strategically decomposed to enable teams to deliver, continuously deliver. 380 00:35:27,440 --> 00:35:30,520 And then there's, oh, buzzword compliance. 381 00:35:30,680 --> 00:35:37,620 Let's, like, make everything small and separated by high latency network communication. 382 00:35:38,320 --> 00:35:40,320 Those things are not necessarily the same. 383 00:35:41,140 --> 00:35:47,980 And I think that, you know, outside of the contracts and whether you're going to go to a tool to enforce them and test them, 384 00:35:48,040 --> 00:35:54,420 I do, I love some of the stuff that's going on with the contract testing as a mindset. 385 00:35:55,200 --> 00:35:59,100 At the same time, I feel like every generation re-implements CORBA. 386 00:35:59,720 --> 00:36:01,780 Or then, you know, then they call it SOAP. 387 00:36:01,780 --> 00:36:04,180 Or, and then, like, let's put XML in the middle or whatever. 388 00:36:05,580 --> 00:36:15,320 But in general, what you really want to be able to do is have the ability to have a high confidence that what you're going to do is going to work. 389 00:36:15,580 --> 00:36:22,420 And the tooling can certainly help enforce the properness of that communication. 390 00:36:23,120 --> 00:36:32,860 I also, going back to some of the things I was reacting to, Folly, don't think you ever get out of the fact that smart people solve problems. 391 00:36:32,860 --> 00:36:41,560 And that if you have smart people on both sides of an equation talking to each other, just going back to the kind of truism that is Conway's law, 392 00:36:42,040 --> 00:36:51,240 if there's problems between two groups of people working on the subsystem of the larger system, then that's going to be reflected in the API. 393 00:36:51,400 --> 00:36:59,540 And whether you're going to enforce that with code that has, you know, hard enforcement of the policy as a contract, 394 00:36:59,540 --> 00:37:07,440 or you're going to enforce that by convention as human beings cooperating with each other, is kind of left to, as an exercise for the reader. 395 00:37:07,940 --> 00:37:09,760 But to me, it all goes back to people. 396 00:37:09,900 --> 00:37:15,680 This is all, software's about people optimizing the experience and the performance of other people. 397 00:37:15,800 --> 00:37:20,540 And you can never get away from that high value of communication. 398 00:37:21,400 --> 00:37:23,120 That's well said. 399 00:37:23,200 --> 00:37:26,260 I wanted to quick click on that one thing because you tweeted something. 400 00:37:26,400 --> 00:37:29,660 Not that it's not always sarcastic, but you tweeted something today about that. 401 00:37:29,740 --> 00:37:31,520 Once you add message queues, you have two problems. 402 00:37:32,180 --> 00:37:33,540 You know, when you think about... 403 00:37:33,540 --> 00:37:39,320 I said some, because it's actually, there's a very famous quote about regular expressions. 404 00:37:39,600 --> 00:37:40,020 Right, right. 405 00:37:40,080 --> 00:37:41,000 Now you have... 406 00:37:41,000 --> 00:37:42,080 Now you have two problems. 407 00:37:42,260 --> 00:37:47,320 And so I said, sometimes when people have a problem, they think, I know I'll solve this with messaging. 408 00:37:47,800 --> 00:37:49,440 Now they have queue problems. 409 00:37:49,900 --> 00:37:50,920 I appreciate the pun. 410 00:37:52,300 --> 00:37:56,220 So when you think of inter-service communication, do you think that that's... 411 00:37:56,220 --> 00:37:59,340 Queuing, again, it's probably based on scenario, at least from my experience. 412 00:37:59,480 --> 00:38:05,660 But other cases where you see a crawl, walk, run of, hey, first, maybe you even have synchronous web service communication. 413 00:38:05,660 --> 00:38:09,340 Then maybe you track latency and things go to async or eventually go to messaging. 414 00:38:09,640 --> 00:38:12,840 Or, you know, do you have any sort of decision criteria you'd like to make with those? 415 00:38:13,480 --> 00:38:17,360 I mean, I think that there is this missing language. 416 00:38:17,680 --> 00:38:22,560 This is one of my favorite topics, actually, around architecting systems. 417 00:38:23,460 --> 00:38:32,580 And often when people talk about software architecture, they start talking about this kind of ivory tower of abstractions and how they communicate together. 418 00:38:32,580 --> 00:38:52,920 But when you sit and think about the system and start especially mapping out what people will euphemistically refer to as nonfunctional requirements, the way that you queue, the way that you cache, the way that you do all these things can have a huge impact on the qualities of that system. 419 00:38:53,380 --> 00:39:01,720 And I think that there's kind of like a missing pattern language that the industry doesn't have to discuss this properly. 420 00:39:01,920 --> 00:39:16,740 So kind of as a rule of thumb, like I look at a problem and it's like I can think about the throughput, I can think about the numbers, and that starts to kind of have an intuitive sense of like where I would queue or where I would cache or where you could get away with synchronous or whatever. 421 00:39:17,460 --> 00:39:29,020 But I don't feel – and I'd love to hear what Kenny and Oliver – I don't feel like we got that proper language to discuss those patterns across the industry. 422 00:39:29,160 --> 00:39:31,060 There's like people that have an intuitive sense of it. 423 00:39:31,840 --> 00:39:41,280 And, you know, there's little patterns and people, you know, pull out specific things and, you know, whatever, say things like CQRS or whatever, and that's great. 424 00:39:41,460 --> 00:39:47,100 But there's not really like, hey, when you see this thing, you should solve it exactly this way. 425 00:39:47,180 --> 00:39:50,720 I don't feel like we really have well-defined patterns. 426 00:39:51,980 --> 00:39:53,740 Kenny, what do you think? 427 00:39:54,340 --> 00:39:59,380 Yeah, I mean, I just want to reiterate what Andrew just said about us not having a pattern. 428 00:39:59,380 --> 00:40:12,340 So if you look at the last 15 years of software development, it's really about companies who have innovated, have been pioneers, who have led the way with a certain approach that was seen to be successful in the long run. 429 00:40:12,780 --> 00:40:17,640 And then you'll have these thought leaders or experts come along and write books and establish the vocabulary. 430 00:40:18,460 --> 00:40:20,580 We've seen that a lot, especially with Agile. 431 00:40:21,540 --> 00:40:23,260 But I just wanted to reiterate that point. 432 00:40:23,320 --> 00:40:24,620 I think that's a great point. 433 00:40:25,320 --> 00:40:40,820 I feel like we're at the phase where it's kind of a pop culture and there's pockets of people who've got the experience and the intuition, but the language is yet to emerge where we're really able to have kind of semantically dense conversations about these patterns yet. 434 00:40:42,440 --> 00:40:42,840 Yeah. 435 00:40:43,260 --> 00:40:43,660 Yeah. 436 00:40:43,860 --> 00:40:46,860 Ollie, thoughts on are we missing patterns? 437 00:40:46,960 --> 00:40:51,400 I mean, I've got enterprise integration patterns up here on my bookshelf and read that for the last 10 years. 438 00:40:51,400 --> 00:40:52,300 But I know what you're saying. 439 00:40:52,380 --> 00:40:57,280 I mean, the integration space has been talking about some of these just straight up messaging patterns for a while. 440 00:40:57,320 --> 00:41:03,780 But I understand we're talking differently as you factor in all the different communication aspects that involve connecting systems. 441 00:41:03,840 --> 00:41:10,020 Well, I think there's certainly patterns that are referenceable in the book with respect to queuing, caching, whatever. 442 00:41:10,400 --> 00:41:20,280 I think those are – I wouldn't say they're well-established, certainly not to the degree that some quote-unquote patterns got established in other aspects of software. 443 00:41:20,820 --> 00:41:33,220 But the thing that's absolutely missing is being able to kind of qualify and quantify the needs of a system and then map that into those patterns. 444 00:41:34,000 --> 00:41:35,200 That's well said. 445 00:41:35,480 --> 00:41:40,720 I think that's what's missing is some of that when is this the right fit versus like I'm just going to use this sort of wire tap pattern here. 446 00:41:40,840 --> 00:41:42,240 I'm going to use this pub sub here. 447 00:41:42,740 --> 00:41:52,560 Some of the worst things I've ever seen in my technical career were people misapplying patterns to solutions that they were never intended to be. 448 00:41:54,500 --> 00:42:01,220 Actually, it's one of the aspects that I like about the REST dissertation so much. 449 00:42:02,300 --> 00:42:05,980 Who of you was it who wanted me to define REST, actually? 450 00:42:07,120 --> 00:42:11,780 I threw that as throwing it down the gauntlet to really rile people up. 451 00:42:12,280 --> 00:42:14,160 I don't have to define it because it is defined. 452 00:42:14,520 --> 00:42:27,380 I especially like the way it's defined because that sort of plays into what Andrew just said is people usually think of it like URIs and there's HTTP methods and whatnot. 453 00:42:27,380 --> 00:42:44,580 And that's actually already like the implementation step to some degree because fundamentally it's about like there's certain characteristics or here's a set of constraints that were applied to the thing called the internet or the web and HTTP as its protocol implementation. 454 00:42:44,860 --> 00:42:48,620 There were some constraints about, let's say, the uniform interface, right? 455 00:42:48,700 --> 00:42:53,980 Or there's a well-defined set of methods that really have to get proposed to the internet. 456 00:42:54,060 --> 00:42:55,300 It could be something else. 457 00:42:55,440 --> 00:42:57,900 But you constrain yourself. 458 00:42:57,900 --> 00:43:07,000 You put constraints onto yourself and by that you get some effects, right, of the scalability, evolvability, and whatnot. 459 00:43:07,220 --> 00:43:14,060 And certain constraints imply or have a much bigger impact on certain effects on the system. 460 00:43:14,440 --> 00:43:23,160 Just like I mentioned, the hypermedia or HATOS aspect being the most important one of all of these in terms of evolvability of APIs. 461 00:43:23,160 --> 00:43:28,260 It also plays into scalability, but maybe not as much as statelessness or what have you. 462 00:43:29,300 --> 00:43:44,160 So fundamentally, the idea of defining the constraints and the effects that those apply or adhering to those constraints then causes might be an approach. 463 00:43:44,440 --> 00:43:56,640 To actually define the same thing for the things we just discussed, like, okay, do I want to avoid the inter-systems communication, right? 464 00:43:56,780 --> 00:44:02,200 Like, while I'm serving a user request, I don't actually call out to other systems. 465 00:44:02,200 --> 00:44:13,980 What effects does that have on the way I need to deal with data, the way I have to approach consistency within the overall system and whatnot? 466 00:44:14,980 --> 00:44:24,180 Maybe one example that we can give our listeners is that the self-contained systems approach. 467 00:44:25,380 --> 00:44:28,320 There's even a website about that, scs-architecture.org. 468 00:44:28,760 --> 00:44:48,400 It's basically coming up with a set of these constraints or some, what is it, heuristics or – I'm lacking a good term for that – but a set of rules, basically, that you force yourself into when designing a system of systems. 469 00:44:48,400 --> 00:44:56,060 And then giving you, like, or describing the consequences that that has on the system that you will be creating. 470 00:44:57,280 --> 00:45:02,240 It's a special kind of flavor or granularity of microservices to some degree. 471 00:45:02,480 --> 00:45:13,340 But I found it to be a good example for that kind of, okay, let's agree on a set of rules and see what kind of consequences do we get from them. 472 00:45:13,960 --> 00:45:20,580 I agree with Oliver, and I'm just trying to tie it back to what we were saying. 473 00:45:21,020 --> 00:45:29,040 I feel like the people who've had experience over the last 10 years are starting to have those heuristics. 474 00:45:29,460 --> 00:45:37,080 But what we haven't done is taken that next step where those are codified and that we can have rich conversations about them. 475 00:45:38,080 --> 00:45:40,340 And who do you – I'll be interested in your take. 476 00:45:40,400 --> 00:45:50,040 I mean, this isn't just an architecture discussion, especially Ivory Tower, but not even – so is this engineering – every developer should start to be internalizing some of these things? 477 00:45:50,140 --> 00:45:57,920 Does this stuff emerge from an architecture over time, or do you not really believe in that and think that these are some overarching things that should factor into a system design? 478 00:45:58,280 --> 00:46:02,280 Kind of who plays the role on projects to make sure that some of these things are done in a good fashion? 479 00:46:03,060 --> 00:46:03,700 Kenny, what do you think? 480 00:46:05,120 --> 00:46:09,820 Yeah, so I would say the architect still has a place. 481 00:46:09,960 --> 00:46:24,800 Maybe not the ivory tower architect, but an architect who can maybe embed themselves on a team and kind of give a set of rules for how you're able to test, especially, against microservices that you consume. 482 00:46:24,800 --> 00:46:46,280 Because at the end of the day, if we have 500 microservices in an architecture, and sure, there may be modularity in a certain set of microservices, but how do you really think about going into a testing environment and testing against maybe 20 or 30 microservices without having to strap an entire environment to test one-to-one between each service? 483 00:46:46,280 --> 00:46:52,460 So I think that's what we're seeing with things like Spring Cloud Contract, where you can simulate a service based off of a contract. 484 00:46:52,960 --> 00:47:02,160 But yeah, I think that you still just need knowledgeable people on your team to be able to set forth these, I guess, experiences that they've had in the past developing in this architecture. 485 00:47:03,460 --> 00:47:09,080 Yeah, Oliver, thoughts on who plays the role in this to make sure that you are thinking of the right principles? 486 00:47:09,460 --> 00:47:10,820 Everybody or certain folks? 487 00:47:11,680 --> 00:47:13,200 I'm totally with Kenny here. 488 00:47:13,320 --> 00:47:30,380 So I guess it's for some of these aspects, we just have to learn from the people that built these kind of systems, that tried different approaches, and found out what was working for them. 489 00:47:30,380 --> 00:47:41,880 But at this point, it's kind of important to abstract from the concrete situation they were in to the fundamental context. 490 00:47:42,180 --> 00:47:44,700 So not every company is a Netflix, right? 491 00:47:44,740 --> 00:47:52,720 So what might work for Netflix will only work for you if you're in the same situation, if you have the same driving forces. 492 00:47:54,200 --> 00:47:56,980 And what else did I want to get to? 493 00:47:58,720 --> 00:48:01,020 That thing, the... 494 00:48:01,020 --> 00:48:03,180 I'm sorry, I totally lost it now. 495 00:48:03,800 --> 00:48:04,060 Anyway. 496 00:48:05,280 --> 00:48:06,020 No problem. 497 00:48:06,100 --> 00:48:07,620 Schaefer, your thoughts on that? 498 00:48:07,920 --> 00:48:09,340 Who takes responsibility here? 499 00:48:09,820 --> 00:48:15,940 I think that the proper way for an organization to answer that question is it's everyone's responsibility. 500 00:48:16,340 --> 00:48:21,440 At the same time, everyone probably has a little bit of a specialization role. 501 00:48:21,700 --> 00:48:33,180 And the role of the PowerPoint jockey architect should probably be an endangered species at this point, although I know they're still thriving in certain pockets of the enterprise. 502 00:48:34,780 --> 00:48:41,480 The things that have been successful, if you just want to look at examples, you can see the things that happen at Amazon. 503 00:48:41,780 --> 00:48:44,880 You can see the things that happen at Google, Netflix, whatever. 504 00:48:45,000 --> 00:49:00,020 As an example, there's often a role of the architects or SREs or whatever who they don't have a project that they're trying to deliver the functionality for the business. 505 00:49:00,020 --> 00:49:09,480 So much as the mandate to roam around and look at how things are working and focus on making them better. 506 00:49:10,500 --> 00:49:20,920 And then another thing I would add, and this is something I think people learn the hard way one way or another, sometimes by accident, is that there's a high value to experimentation. 507 00:49:22,320 --> 00:49:40,460 And one of my mentors early as an engineer always said you should never implement the first thing you think of, that you should force yourself to think of a few solutions and articulate, at least in your own kind of mental process, what the tradeoffs of those solutions are before you make a decision. 508 00:49:41,000 --> 00:49:47,320 But a lot of things you can never really know how things will work until you make them work. 509 00:49:47,860 --> 00:49:52,820 And that is sort of undervalued in a lot of pockets. 510 00:49:52,820 --> 00:50:12,780 But going back to – and I also feel very strongly that a lot of what's happening with microservices and trying to build reliable systems can learn a ton of lessons by studying Erlang and the thought processes of Joe Armstrong in particular around reliability. 511 00:50:13,780 --> 00:50:30,880 And one of the things that the Erlang said that I think just resonates, and I say this all the time in meetings with customers, is that if you don't experiment before you put something into production, then production is always an experiment. 512 00:50:31,660 --> 00:50:31,960 Mm-hmm. 513 00:50:33,120 --> 00:50:33,860 Very good. 514 00:50:34,520 --> 00:50:37,200 Oli, did you get your idea back? 515 00:50:37,500 --> 00:50:38,320 Yeah, I got it. 516 00:50:38,320 --> 00:50:55,440 The thing I wanted to mention is that the reason we're actually talking about stuff like that, like about microservices and systems of systems, is that we've all made experience with the other approach to that, to bigger systems, which were monoliths, right? 517 00:50:55,500 --> 00:51:06,640 And we tried, and some succeeded, and some drew the necessary conclusions for new projects from that, and we drew conclusions from the failure. 518 00:51:06,640 --> 00:51:18,360 And that's basically what we just have to go through for the applied distributed systems area that we're – or era that we're actually in right now. 519 00:51:18,740 --> 00:51:19,220 Absolutely. 520 00:51:20,140 --> 00:51:23,500 So that's just – I guess we're just going to have to learn. 521 00:51:23,700 --> 00:51:25,320 That's probably it, and that's a good thing, I guess. 522 00:51:25,320 --> 00:51:38,060 So just to add the way that I say the same thing is that the way that we got to these microservice architectures was not because someone set out and said, oh, I need to build microservices. 523 00:51:38,420 --> 00:51:43,720 It was a Darwinian evolution that was solving the problem, right? 524 00:51:43,760 --> 00:51:45,620 Like Netflix didn't set out to build microservices. 525 00:51:46,180 --> 00:51:52,060 It was the way that they could work that solved all the constraints of the problem that they were trying to solve. 526 00:51:53,020 --> 00:52:03,460 To piggyback off that, so having done some research on the topic for the book, kind of looking at this evolution of where did microservice architectures originate from, 527 00:52:03,460 --> 00:52:22,400 the earliest reference that I could find went back to 2005 or 2006 when Werner Vogel's CTO of Amazon explained the processes for how internally their engineers were thinking about delivering Amazon.com without having to have this one monolithic application. 528 00:52:22,960 --> 00:52:25,000 And it came down for him when he explained in this article. 529 00:52:25,000 --> 00:52:34,120 He explained that it really was about control where different teams were unable to control their own destiny because they were sharing resources, 530 00:52:34,120 --> 00:52:40,360 and that gave them a constraint which prevented them from being able to control how they got features into production. 531 00:52:40,560 --> 00:52:42,020 So I think that's interesting, too. 532 00:52:44,040 --> 00:52:44,760 Very good. 533 00:52:44,900 --> 00:52:49,120 Well, hopefully for our listeners, they've got lots of chances to try to keep tapping into this stuff. 534 00:52:49,120 --> 00:52:53,840 Kenny, I think you're part of this Refresh Cloud Native Roadshow that's kicking off. 535 00:52:54,140 --> 00:52:54,480 Is that right? 536 00:52:55,660 --> 00:52:56,480 Yes, yes. 537 00:52:56,600 --> 00:52:58,140 With Casey West and Andrew. 538 00:52:58,520 --> 00:53:00,600 Yeah, so we'll be able to find you on the road. 539 00:53:00,680 --> 00:53:04,980 And obviously, Pivotal folks love helping teams do some of these decomposition exercises. 540 00:53:05,160 --> 00:53:06,200 Hey, we're always learning, too. 541 00:53:06,280 --> 00:53:08,160 It's not like we come in always with the exact answers. 542 00:53:08,300 --> 00:53:11,120 I think we learn with our customers of what fits their environment. 543 00:53:11,660 --> 00:53:14,700 But hopefully they can work with us or continue to try things out on their own. 544 00:53:14,780 --> 00:53:17,100 And the most important thing is get out there and start doing this stuff. 545 00:53:18,020 --> 00:53:18,740 All right, fellas. 546 00:53:18,740 --> 00:53:22,780 I appreciate you coming on for this very special edition of Pivotal Conversations. 547 00:53:24,360 --> 00:53:26,640 Well, and thanks for listening. 548 00:53:27,480 --> 00:53:33,000 As always, this has been Pivotal Conversations, which you can find if you go to pivotal.io slash podcast. 549 00:53:33,000 --> 00:53:38,940 Or if you're feeling really sprightly with your fingers, you can go to pivotal.io slash pivotal hyphen conversations. 550 00:53:39,280 --> 00:53:42,480 You might be feeling lonely that you haven't talked to the hyphen for a while. 551 00:53:42,640 --> 00:53:48,860 You can find our not-so-secret back end also at SoundCloud, which is at soundcloud.com slash Pivotal Conversations. 552 00:53:49,480 --> 00:53:54,460 And Richard, what's the thing that we like to ask everyone to do if they've enjoyed this episode? 553 00:53:54,840 --> 00:53:57,400 I mean, other than send us raw bags of cash. 554 00:53:58,280 --> 00:53:58,900 Right, right. 555 00:53:59,160 --> 00:54:00,340 Small unmarked bills. 556 00:54:00,640 --> 00:54:01,400 Loose change, Dakota. 557 00:54:02,000 --> 00:54:06,800 We do love if you add some reviews, you know, rate the podcast if you find this interesting. 558 00:54:06,900 --> 00:54:11,000 It always helps us and helps our boss justify our time spent doing podcasts. 559 00:54:11,460 --> 00:54:12,060 That's right. 560 00:54:12,220 --> 00:54:13,220 Well, we'll see everyone next time. 561 00:54:13,260 --> 00:54:14,740 And thanks again to everyone for being on. 562 00:54:14,900 --> 00:54:15,360 This is fun.