1 00:00:00,000 --> 00:00:11,360 Okay. Welcome back, folks, for another episode of Pivotal Conversations. Today, I'm joined 2 00:00:11,360 --> 00:00:17,400 by Viraj Naik from Travelers Insurance and Rohit Kalapur, one of our app modernization 3 00:00:17,400 --> 00:00:21,940 experts here at Pivotal. Viraj, go ahead and why don't you introduce yourself? 4 00:00:22,860 --> 00:00:29,600 Hi. My name is Viraj, and I am a lead architect with Travelers Insurance, working in personal 5 00:00:29,600 --> 00:00:34,980 insurance line of business. I generally am involved in many applications, but recently 6 00:00:34,980 --> 00:00:42,480 I've been involved in cloud applications and specifically around cloud transformation, 7 00:00:42,800 --> 00:00:49,620 app transformation. So that's just me. Awesome. Rohit, how about you go ahead and 8 00:00:49,620 --> 00:00:56,000 introduce yourself? You've been on the podcast before. Yeah, this is my second time. My name 9 00:00:56,000 --> 00:01:03,340 is Rohit Kalapur. I am one of the app transformation practice leads at Pivotal Labs. What that means 10 00:01:03,340 --> 00:01:10,340 in simple English is I help customers migrate workloads and their applications onto Pivotal 11 00:01:10,340 --> 00:01:16,540 platforms so that they can get the value of the cloud and realize the outcomes that they 12 00:01:16,540 --> 00:01:23,880 want from application modernization. I'm really, I'm pretty hot to try on app 13 00:01:23,880 --> 00:01:30,760 modernization these days. And of course, in classic confirmation bias mode, I come across 14 00:01:30,760 --> 00:01:36,600 these reports from like Forrester that says application modernization is like the most important work 15 00:01:36,600 --> 00:01:41,880 you're going to do for the next 10 years. And Gartner has their recommendations out. So I'm 16 00:01:41,880 --> 00:01:47,380 constantly like sniffing out and getting all this feedback from analysts in the market that, 17 00:01:47,900 --> 00:01:52,260 yeah, this is really important. But I think it's all also challenging because it's not always, 18 00:01:52,540 --> 00:01:58,300 it's not always the sexy new stuff, but it's like the, just the critical work that needs to be done. 19 00:01:58,300 --> 00:02:04,820 So I'm really excited to have the two of you on because I think this is, this is just like the 20 00:02:04,820 --> 00:02:10,600 stuff people need to hear about and know that it's, it's, it's not a no-fly zone. It's doable. 21 00:02:11,460 --> 00:02:16,380 Um, you know, here's how to do it successfully, et cetera. So I'm really excited. Um, and in 22 00:02:16,380 --> 00:02:23,580 particular this, this, this kind of, uh, uh, topic came about because, um, we've got an upcoming 23 00:02:23,580 --> 00:02:33,080 webinar, uh, with the two of you and it's focused on modernizing a .NET application off of mainframe. 24 00:02:33,080 --> 00:02:41,240 Um, and so that's another, like two other areas, two other kind of keywords popping up on my radar. 25 00:02:42,020 --> 00:02:47,100 Um, so we look at, we look at mainframe modernization, Rohit, you and I did a webinar 26 00:02:47,100 --> 00:02:55,620 earlier this year on this topic. And, um, it's sort of, you know, as, as much as people have said, 27 00:02:55,620 --> 00:03:01,640 you know, mainframe is dead, right? Like these rumors of the death of the mainframe have been 28 00:03:01,640 --> 00:03:09,060 greatly exaggerated and it's still around. And so, um, you know, there's been a long time when folks 29 00:03:09,060 --> 00:03:13,920 have, you know, Hey, we want to look at how do we, how do we get off of mainframes, et cetera. 30 00:03:14,580 --> 00:03:21,060 And yet that, that story is not new, right? Like you've, you've heard this for years. I've heard this 31 00:03:21,060 --> 00:03:27,540 for years, decades. So I'm curious and Viraj, maybe this is really more of a question for you. 32 00:03:27,540 --> 00:03:34,420 I'm curious, like what kind of was the catalyst that, that really made it happen this time? 33 00:03:34,420 --> 00:03:41,060 Because I'm sure that this was not the first time that travelers are like any company. It's not the 34 00:03:41,060 --> 00:03:46,540 first time folks have looked at, okay, no, for serious this time, we're going to, we're going to 35 00:03:46,540 --> 00:03:51,040 modernize off this mainframe. What happened? What was different this time around? 36 00:03:52,520 --> 00:03:59,160 Um, multiple things, right? Like I think recently we see that there are a lot of market disruptors, 37 00:03:59,160 --> 00:04:04,600 like there are a lot of things that forces, uh, change in strategy, change in direction. Uh, 38 00:04:04,600 --> 00:04:10,680 so our business also is not, uh, insulated from those things. So they are also looking at ways that 39 00:04:10,680 --> 00:04:17,640 we can adapt to the market. Um, we also want to grow in the market space. Uh, and, and sometimes, 40 00:04:17,640 --> 00:04:25,080 you know, you just want to try out things like you, you want a quick experimentation and then also a quick 41 00:04:25,080 --> 00:04:31,560 implementation. And, and in those cases, you also want to make sure that you, you can scale, you can 42 00:04:31,560 --> 00:04:36,840 expand, you can contract, uh, you know, uh, you don't want to build something and just leave it because 43 00:04:36,840 --> 00:04:41,480 you don't know if that, that is the model that we would want to continue for next two years or six 44 00:04:41,480 --> 00:04:48,200 months or whatever. So, so there is that flexibility that, uh, we wanted. So what you described about 45 00:04:48,200 --> 00:04:53,800 mainframe is true that has not changed. It's still, you know, uh, alive taking, in fact, performing really 46 00:04:53,800 --> 00:05:00,440 well, but that's the part like flexibility, scaling, you know, in, in terms of, uh, going up and down 47 00:05:00,440 --> 00:05:06,680 in terms of capacity, uh, you know, that was something that we said we have to have, uh, we cannot really 48 00:05:06,680 --> 00:05:14,280 always afford to have like six months of planning or, you know, uh, having infrastructure, uh, you know, 49 00:05:14,280 --> 00:05:19,320 ready, uh, for, for these things or growth opportunities and opportunities seldom wait, 50 00:05:19,320 --> 00:05:26,760 right? So you just have to be ready. Uh, so, so that was one of the key drivers. The other was like, 51 00:05:26,760 --> 00:05:32,200 as we all know, the mainframe has been around, uh, for a long time, uh, the skillset, I mean, 52 00:05:32,200 --> 00:05:35,880 in, in terms of the resources that you have, who can work on it, who can maintain it, 53 00:05:35,880 --> 00:05:40,280 who can develop it are very difficult to find. It's very hard to, you know, kind of, 54 00:05:41,720 --> 00:05:46,840 uh, get hold of that. And if you had a mainframe based organization, you don't really, uh, you know, 55 00:05:46,840 --> 00:05:52,120 can ingest new talent and, uh, you know, like no one really wants to come and say, yeah, 56 00:05:52,120 --> 00:05:57,160 interesting mainframe. I want to work on it. Right. So it's just, uh, it was hard. So that was, 57 00:05:57,160 --> 00:06:03,240 and what that leads to is like, okay, it's total cost of ownership is also going up. So you have this 58 00:06:03,240 --> 00:06:09,080 flexibility issue, you have this, uh, you know, talent issue, and then you also have, uh, the cost 59 00:06:09,080 --> 00:06:14,840 that goes up. So for every business opportunity that you see, you have to weigh in against, oh, 60 00:06:14,840 --> 00:06:21,000 do I have resources? Do I have, uh, you know, do I have infrastructure? Uh, so we said, we can't wait. 61 00:06:21,000 --> 00:06:27,160 We had done a lot of POCs, uh, you know, and, and we said, no, this has to happen as we grow, 62 00:06:27,160 --> 00:06:33,960 as we move along, this has to get at risk. We want to make sure that, uh, we have a distributed 63 00:06:33,960 --> 00:06:39,400 platform, which is not just distributed onto.net, uh, you know, technology standpoint, but from an 64 00:06:39,400 --> 00:06:45,080 infrastructure, uh, you know, standpoint also, we wanted to go to a scalable infrastructure that can 65 00:06:45,080 --> 00:06:50,680 go on and scale up, scale down and it is disposable. So tomorrow, if we say, Hey, you know what, 66 00:06:50,680 --> 00:06:57,800 this doesn't look good, get rid of it. That's fine. It's easy. It's so it's, it's just a marriage 67 00:06:57,800 --> 00:07:01,800 of convenience. So, you know, so as to say, we business was of the operation and looking for the 68 00:07:01,800 --> 00:07:08,280 opportunities, uh, we had this, uh, we had this and then we had a, like a cloud platform that, um, 69 00:07:09,240 --> 00:07:15,240 we thought is, is, is a good choice for us to make it happen. So that was different this time. 70 00:07:15,960 --> 00:07:19,080 Okay. I mean, you would, you mentioned some things in terms of, 71 00:07:19,800 --> 00:07:24,920 you know, wanting to be able to try things out, quick experimentation, quick implementation, 72 00:07:25,560 --> 00:07:31,400 which definitely that sounds like the antithesis of mainframe and like how anything gets done 73 00:07:31,400 --> 00:07:38,600 on the mainframe. So, uh, just to make sure I'm understanding this, you're saying that this, 74 00:07:38,600 --> 00:07:42,680 that was a blocker to being able to do that kind of quick experimentation and implementation. 75 00:07:43,880 --> 00:07:47,880 Yes. Yeah, you're right. Yeah. And that, that was one of the things that, that would slow you down 76 00:07:47,880 --> 00:07:53,240 is to kind of say, Hey, yes, we can, we can experiment on that. That's not, not a problem, 77 00:07:53,240 --> 00:07:57,320 but you know, to experiment and to keep the lights on. I mean, you have systems, 78 00:07:57,320 --> 00:08:01,000 you have enhancements that are ongoing, and then you want to do this experiment. Do I have 79 00:08:01,000 --> 00:08:06,920 resources? Uh, do I have the infrastructure to kind of say, okay, this experimentation would take, 80 00:08:06,920 --> 00:08:11,560 you know, this many amount of this much of MIPS, have I planned for it? Uh, you know, 81 00:08:11,560 --> 00:08:18,520 can I accept, uh, can I handle load if I see that experiment, uh, succeed, you know, succeed and, 82 00:08:18,520 --> 00:08:22,600 and so on and so forth. So that, that always was the challenge. And yeah. 83 00:08:23,560 --> 00:08:29,800 Yeah, no, that's, that's interesting. Like the sort of questions around say, you know, capacity 84 00:08:29,800 --> 00:08:34,680 planning and capacity management with something that you don't know if it's going to work or not. 85 00:08:34,680 --> 00:08:40,360 So that's a, that's a, almost a business level hindrance. Rohit, does this sort of jive with 86 00:08:40,360 --> 00:08:47,720 what you see? Cause you've, you worked with lots and lots of customers, um, some modernizing off of 87 00:08:47,720 --> 00:08:53,880 mainframe, some just modernizing off of other middleware stacks, but did this kind of constraints 88 00:08:53,880 --> 00:08:58,600 sort of jive with, with what you see, or is there anything else that you, you see kind of come up with, 89 00:08:58,600 --> 00:09:05,480 with other users? Yeah. The primary two drivers are cost and agility. And in case of mainframe 90 00:09:05,480 --> 00:09:12,120 modernization skills is like a third aspect, which adds itself. So cost agility for most, uh, 91 00:09:12,120 --> 00:09:17,240 legacy workloads or classic workloads, and then skills is an added component in mainframe modernization. 92 00:09:18,440 --> 00:09:24,360 Travelers is kind of lucky in that they were able to escape the gravitational pull of the mainframe, 93 00:09:24,360 --> 00:09:30,600 but a lot of the customers, um, are still stuck there because all of their critical 94 00:09:31,160 --> 00:09:39,240 transactional systems are in mainframe and they are not just in COBOL. They are in COBOL assembler. 95 00:09:39,240 --> 00:09:45,240 They interface with a lot of external systems. They leverage a lot of the secret sauce the mainframe 96 00:09:45,240 --> 00:09:51,320 provides in terms of resiliency. So I would say that we were kind of lucky in that this was a compute 97 00:09:51,320 --> 00:09:58,680 intensive workload, which was kind of, which was, I wouldn't say easy, but we were able to pull that 98 00:09:58,680 --> 00:10:05,960 out. Um, we had, I think initially we did a whole bunch of travelers did a bunch of POCs to validate 99 00:10:05,960 --> 00:10:13,080 that this is possible. And then, uh, I guess when the engagement formally began, uh, on a proper note, 100 00:10:13,080 --> 00:10:19,400 then we were able to kind of leverage that work and then pull out these vertical slices of functionality. 101 00:10:20,520 --> 00:10:24,920 I like the way that you described that as the, the ability to escape the gravity, 102 00:10:24,920 --> 00:10:29,960 the gravitational pull of the mainframe. And, and if I'm understanding this correctly, 103 00:10:29,960 --> 00:10:36,520 you're sort of describing that, you know, not all mainframe workloads are created equal and some 104 00:10:36,520 --> 00:10:43,880 of them are going to be a lot easier to modernize than others. And looking at how many other dependencies 105 00:10:43,880 --> 00:10:49,560 on other systems, et cetera, are one of those maybe indicators that, that folks can look at. Am I 106 00:10:49,560 --> 00:10:55,560 getting that right? Or is there another way to think about, you know, okay, uh, if the, if the workload 107 00:10:55,560 --> 00:11:01,640 has this type of criteria, it's in a better position to look at, to modernize off the mainframe. 108 00:11:01,640 --> 00:11:06,920 Yeah. I mean, this, this space. So you are absolutely right. It depends on the workload and the criteria you use 109 00:11:06,920 --> 00:11:12,680 to evaluate the workload. So this is one where, you know, as Rohit said, this is important and this is 110 00:11:12,680 --> 00:11:19,080 critical. And it was, uh, somehow the most important. I mean, if I were to call it again, I'm not, uh, you know, 111 00:11:19,080 --> 00:11:26,360 I'm not able to describe it. Everything is important. I mean, in all the parts of that mainframe system 112 00:11:26,360 --> 00:11:31,720 are important. This is like a giant or a big monolith that we're talking about, but this is where, you know, 113 00:11:31,720 --> 00:11:37,160 this has impact on customers where they can come. Uh, they want to see like, what is the rate that 114 00:11:37,160 --> 00:11:42,520 travelers is going to offer? And we want more and more customers to come try, uh, or get a rate. And, 115 00:11:42,520 --> 00:11:50,920 and as a result, yes, this is very critical, very important, uh, function. And it is in, in that sense, 116 00:11:50,920 --> 00:11:57,480 if, if, if a part of it had to be modernized and scaled, this was one, there are other parts where, 117 00:11:57,480 --> 00:12:01,800 yeah, you know what, we don't need to scale that. That is not as much used or that's not, 118 00:12:01,800 --> 00:12:07,560 you know, uh, the things that get, so if you can write it or migrate it on, on a cloud platform, 119 00:12:07,560 --> 00:12:12,360 great. But if not, um, that's fine. There is nothing, you know, where you need to scale up, 120 00:12:12,360 --> 00:12:17,400 scale down, or, you know, there are seasonalities to it. Um, but this was one where you, 121 00:12:17,400 --> 00:12:22,280 you had those variations. It was super critical, uh, and you wanted something which is robust. I 122 00:12:22,280 --> 00:12:27,720 mean, if this clicks or if the business, you know, decides to do certain things, and if they want to 123 00:12:27,720 --> 00:12:34,520 continue doing that, then we wanted to be, uh, stable enough to sustain the workloads. 124 00:12:35,080 --> 00:12:40,760 And I think what was cool was how we implemented it. I remember now where we, when a request for 125 00:12:40,760 --> 00:12:46,680 rating a policy came in, we sent it both to the new system that we modernized and the 126 00:12:46,680 --> 00:12:52,360 old mainframe system to compare and make sure that the rates that we get are equivalent across the 127 00:12:52,360 --> 00:12:57,800 two. And then once we had enough confidence, we switched off the, we, we switched off the 128 00:12:57,800 --> 00:13:02,360 processing on the mainframe so that, and then all of the traffic went to the newer distributed 129 00:13:02,360 --> 00:13:07,560 event based architecture, right? So because it was critical, we had to introduce that safeguard 130 00:13:07,560 --> 00:13:13,640 to make sure that we were not regressing any logic or making sure. And we, we functionally got the 131 00:13:13,640 --> 00:13:20,440 exact same age that we got on the older mainframe system. Yeah. Okay. Okay. And this is kind of 132 00:13:20,440 --> 00:13:26,200 actually perfect transition into my next question, which is about, you know, what is the workload 133 00:13:26,200 --> 00:13:32,680 or the application itself and what, what role does it play? And so this is the, like kind of a, 134 00:13:33,640 --> 00:13:41,800 for someone's getting a quote for insurance. There's a rate rating algorithm essentially that has to run. 135 00:13:41,800 --> 00:13:46,920 And then Rohit, what you kind of described where in the process of making this transition, one of the 136 00:13:46,920 --> 00:13:51,800 important kind of safeguards was these things are going to run in parallel for a while. We've, we've 137 00:13:51,800 --> 00:13:57,800 rebuilt it on a, a new platform, but it's still running on the mainframe and we can now run these in 138 00:13:57,800 --> 00:14:04,360 parallel and check to see if they're actually delivering the same results before we shut down the, the mainframe 139 00:14:04,360 --> 00:14:12,040 one. So, and then, um, but you kind of described that this was a particular opponent that needed to 140 00:14:12,040 --> 00:14:17,320 scale. And that was one of the drivers for identifying it as a good candidate to look at moving off of the 141 00:14:17,320 --> 00:14:22,520 mainframe because it had this sort of, you know, I don't know if it was seasonal, but just variation 142 00:14:22,520 --> 00:14:31,080 in the scaling demands. So, uh, so kind of come coming back to, you know, the, the original catalysts, 143 00:14:31,080 --> 00:14:39,720 right. As a business trying to be more agile, um, what were some of the things that were going to place 144 00:14:39,720 --> 00:14:46,200 more, uh, demand on this particular application that you were anticipating these, these sort of scaling 145 00:14:46,200 --> 00:14:53,400 events were going to start to happen. I mean, um, one of the things that we really wanted to try out 146 00:14:53,400 --> 00:15:00,760 was, you know, kind of integrate a different, um, uh, analytical, you know, capabilities. So in order 147 00:15:00,760 --> 00:15:08,440 to do that, we wanted to see, Hey, if we stand up or we create, you know, this, um, rating, you know, 148 00:15:08,440 --> 00:15:14,520 engine, uh, how would it, uh, you know, kind of, we can do what if scenarios, right. We could, you know, 149 00:15:14,520 --> 00:15:20,520 run, we could do it, we could do it at a scale. So, so just to make sure that, okay, this is something 150 00:15:20,520 --> 00:15:24,760 that is working as Rohit described, you know, we want to make sure that the rate is accurate. 151 00:15:24,760 --> 00:15:31,960 Um, and again, as we kind of integrate, as we kind of, you know, say, okay, I want to tap into like, 152 00:15:31,960 --> 00:15:39,080 or when I say, I, the business wants to tap into a new channel, new market. Um, this was the part 153 00:15:39,080 --> 00:15:46,520 where, you know, what I am not going to plan for weeks, months and say, okay, do I have infrastructure 154 00:15:46,520 --> 00:15:52,760 ready? Do I have things? I'm just going to go and say, okay, just tap into that channel or, 155 00:15:52,760 --> 00:16:00,440 you know, kind of go into that, you know, uh, today we may have, you know, kind of had some artificial, 156 00:16:00,440 --> 00:16:09,160 uh, you know, limitations in terms of self-imposed so as to stay within our, uh, um, plans or budgets, 157 00:16:09,160 --> 00:16:15,880 uh, in terms of infrastructure. Now we can, you know, remove those, uh, you know, uh, blockers and say, 158 00:16:15,880 --> 00:16:22,200 okay, open. We want to do everything and anything that we really feel we should, we shouldn't have any, 159 00:16:22,200 --> 00:16:29,000 you know, limitations, uh, impositions. Uh, and, and that was the, the thing that we, yes, we, we had 160 00:16:29,000 --> 00:16:32,920 to be ready. We had to kind of make sure no one said, Hey, this is the max that you, you could get, 161 00:16:33,480 --> 00:16:38,600 you could get anything. I mean, I mean, in terms of the workload or as long as you are accurate and we 162 00:16:38,600 --> 00:16:46,360 trust, um, that was the part, the scaling, uh, or the scaling. I mean, when I say scaling, it's not 163 00:16:46,360 --> 00:16:50,440 just upward, but you know, there are days when, okay, it's not, we don't have demands. Like on weekends, 164 00:16:50,440 --> 00:16:56,280 you know, typically you don't see as many. Right. We, we can scale down, um, or in some 165 00:16:56,280 --> 00:17:01,480 cases in worse scenario, if we see issues, we could easily turn it off and say, Hey, I, I don't want, 166 00:17:01,480 --> 00:17:08,920 um, to kind of use this, uh, you know, any, any more or tweak it, uh, that should be possible. 167 00:17:10,200 --> 00:17:15,560 Another thing that was very important that for the success of this program, which is basically 168 00:17:15,560 --> 00:17:20,360 has spanned multiple years now was the organizational commitment from the traveler side to 169 00:17:20,360 --> 00:17:26,200 right. Because in some sense, I mean, it, it got worse before it became better. 170 00:17:26,200 --> 00:17:32,200 We started migrating the workloads off the mainframe. We, I remember we created the distributed 171 00:17:32,200 --> 00:17:37,160 architecture. There were a whole bunch of microservices. There was some initial pain 172 00:17:37,160 --> 00:17:42,680 that we had to go through to land on the right, uh, set of services and the right target architecture, 173 00:17:42,680 --> 00:17:47,960 the right granularity of the services, uh, that will do all the processing. I mean, that's, 174 00:17:47,960 --> 00:17:53,640 uh, so it's, uh, in some ways you have to stick with it because the initially there is that pain 175 00:17:53,800 --> 00:18:00,360 in where it feels much worse and it feels like you're losing a lot of money, but then like 176 00:18:00,360 --> 00:18:05,800 the short term negative pain then, then results in a much more long-term benefit. 177 00:18:05,800 --> 00:18:09,560 So you definitely see some of that in mainstream modernization. 178 00:18:10,440 --> 00:18:15,560 Okay. That's, um, well, you know, first just to kind of close out on some of Viraj's points, 179 00:18:16,200 --> 00:18:23,480 it sounds like there really were kind of two factors with this specific application, uh, that were 180 00:18:24,200 --> 00:18:29,720 sort of drivers around one being able to want to try out sort of different, you know, 181 00:18:29,720 --> 00:18:34,840 there's an analytical component to this particular application and wanting to be able to try out 182 00:18:34,840 --> 00:18:40,040 different things, uh, you know, just within the team that's operating that I think of it as like a, 183 00:18:40,040 --> 00:18:45,560 I'm sure it's like a service into other, other applications and endpoints. 184 00:18:46,680 --> 00:18:49,400 Folks want to be able to iterate there. And then the second thing was 185 00:18:50,360 --> 00:18:55,240 wanting to remove those artificial gatekeepers so that any of those other parts of the business 186 00:18:55,240 --> 00:19:02,120 that are like, Hey, we want to run this promotion, um, but not have to have weeks and weeks of capacity 187 00:19:02,120 --> 00:19:09,240 planning and, and process and bureaucracy to appropriately request the capacity to run that promotion. 188 00:19:09,240 --> 00:19:13,960 I can empathize with that. Right. As a marketing person, it's like, Hey, can we turn this on like 189 00:19:13,960 --> 00:19:20,360 tomorrow? Um, you know, like that's the, the, you rarely want the answer to be like, we need six weeks 190 00:19:20,360 --> 00:19:26,280 notice. Um, so cause all my best ideas are like shower ideas and I just immediately come in and be like, 191 00:19:26,280 --> 00:19:34,200 yes, let's do this right now. Um, I'm sure it works the same in insurance, like shower ideas prevail. 192 00:19:36,920 --> 00:19:43,800 Um, but yeah, you know, Rohit, your point here about, I think this is a great thing for folks to, 193 00:19:43,800 --> 00:19:47,800 to have on their radar and be prepared for, right. The whole, like, it's going to get worse before 194 00:19:47,800 --> 00:19:52,600 it gets better. And you talked about finding the right granularity of the services. Like, 195 00:19:52,600 --> 00:19:56,840 I think this is something that, um, I know, you know, talking to some of the Gartner analysts, 196 00:19:56,840 --> 00:20:00,600 they, they, they, they, of course got a bunch of different names for all these different things. 197 00:20:00,600 --> 00:20:05,800 Like, you know, I don't know, macro services or nano services or mini service. I don't know. 198 00:20:05,800 --> 00:20:11,480 They're like using all of the terms. Um, but I think the point that they're trying to make is 199 00:20:11,480 --> 00:20:16,200 like, you sometimes need these different granularities and you don't want to break 200 00:20:16,200 --> 00:20:23,160 things down too far because then you do introduce excessive overhead. So, but it's a bit of a journey 201 00:20:23,160 --> 00:20:30,600 to find that right granularity. Are there, are there any shortcuts that you think, um, exist out there 202 00:20:30,600 --> 00:20:33,640 in terms of, you know, finding the right granularity? Is it better to just, 203 00:20:34,600 --> 00:20:38,520 you know, go really fine grained and then build back up? Or is it better to kind of 204 00:20:38,520 --> 00:20:45,320 go, you know, course to slightly less course and, and sort of get, get more fine grain from there? 205 00:20:45,320 --> 00:20:49,720 Or is it really something that everyone kind of has to figure out on their own? Because every workload is 206 00:20:49,720 --> 00:20:57,880 different. I think, uh, let Viraj answer part of this, but, uh, why don't you go first on this one? 207 00:20:59,240 --> 00:21:03,800 Yeah. I mean, I think I can, you know, tell you what we did was we started, I mean, again, we had 208 00:21:04,600 --> 00:21:10,360 two constraints, right? One, as Rohit said, um, this was something that is not a new business capability 209 00:21:10,360 --> 00:21:16,120 that, you know, we had a luxury of saying, okay, this is how it would be. It had to match in order to, 210 00:21:16,120 --> 00:21:21,720 so that this fits into the existing ecosystem, the output had to match, not just the output, 211 00:21:21,720 --> 00:21:27,240 but the response time also had to match, right? So all your decisions that you are making, uh, 212 00:21:28,120 --> 00:21:33,480 is like, no one is going to give this application, like, okay, I'll give you extra, you know, 500 213 00:21:33,480 --> 00:21:39,720 milliseconds to produce your response. Uh, so you kind of have those constraints to start with. So you say, 214 00:21:39,720 --> 00:21:44,200 okay, yeah, no matter which I'm, and in this case we started with, like, I, I, I remember it was all, 215 00:21:44,200 --> 00:21:49,480 it was almost two years ago, but we had about 12 or 13, uh, you know, services because we were 216 00:21:49,480 --> 00:21:55,480 like true microservices, like, okay, only, you know, single function or single thing that it will do. 217 00:21:56,200 --> 00:22:01,000 And then we realized that, you know what, this is fine. It's, it's really good. You know, it looks 218 00:22:01,000 --> 00:22:08,920 good. Um, we are pure microservices, but then it had overheads like serializing, deserializing. And, 219 00:22:08,920 --> 00:22:13,400 and then when you look at it and say, okay, if I were to do it at scale, if I were to say I'm running 220 00:22:13,400 --> 00:22:19,400 hundreds and thousands of transactions, um, I'm going to see some degradation and can I, can I, 221 00:22:19,400 --> 00:22:26,200 I cannot afford to go over the response time that I have, or I have to meet that SLA. And, and that kind of 222 00:22:26,200 --> 00:22:31,400 drove us to kind of say, okay, can we consolidate, you know, some of those things? So, you know, 223 00:22:31,400 --> 00:22:37,400 Rohit helped us with this domain driven design where we can say, okay, what is it that we can collapse? How can we not put, 224 00:22:37,400 --> 00:22:43,080 like, again, you know, create a monolith, but, um, just say, okay, these services that you have 225 00:22:43,080 --> 00:22:48,200 split apart, they can belong together. I think they, they have, you know, things that are common. 226 00:22:48,200 --> 00:22:53,160 They get, at least they are of the same domain. Can we put them together? And, you know, so we came 227 00:22:53,160 --> 00:22:59,160 down from 13 to, I think six that we have now. Yeah, we did. We remember we did that exercise of 228 00:22:59,160 --> 00:23:05,960 microservice rationalization. And what we did is we basically put every, each one of those 13 microservices 229 00:23:05,960 --> 00:23:12,120 through this six item checklist that Nate Chuta came up. Like he, there is an article that 230 00:23:12,840 --> 00:23:19,720 Nathan, should I put up the six factors of a microservice or some, I forget the exact title of 231 00:23:19,720 --> 00:23:25,080 that. But basically you, you take that, you take each microservice, you evaluate it on those six 232 00:23:25,080 --> 00:23:30,200 criteria. And if it stands on its own, it stands on its own. But if it doesn't stand on its own, 233 00:23:30,200 --> 00:23:35,480 you merge it with the other service on which it is dependent on. So we kind of use that six factor 234 00:23:35,480 --> 00:23:42,360 rubric to, to compress and compact as 13 microservices into six, six or seven odd services. 235 00:23:42,360 --> 00:23:46,280 The second thing I remember is that this was a very latency sensitive application, 236 00:23:46,280 --> 00:23:51,000 because when someone is really asking for a quote, he can't really, you can't really make him wait for 237 00:23:51,000 --> 00:23:56,920 so long. So there was a latency budget that really we had across all our individual microservices. 238 00:23:56,920 --> 00:24:02,200 We couldn't exceed that target SLA, which was, I don't remember, it was around like 400, 239 00:24:02,200 --> 00:24:07,560 500, 500 milliseconds. But we definitely had this notion of there was a latency budget and 240 00:24:08,840 --> 00:24:14,600 each microservice kind of had a portion of that and all of them could never exceed that latency 241 00:24:14,600 --> 00:24:17,720 budget to meet the overall service level objective of that. 242 00:24:17,720 --> 00:24:24,440 Yeah. Yep. And that's something that, I mean, whether it's something that a lot of folks will run 243 00:24:24,440 --> 00:24:29,560 into, or just if you understand the business use case for this particular application, then you 244 00:24:29,560 --> 00:24:35,000 understand that the, the latency budget becomes a determining factor in the architecture. 245 00:24:35,800 --> 00:24:41,640 Um, either way that's, uh, yeah, absolutely. And because of that, we made certain choices in 246 00:24:41,640 --> 00:24:49,640 the architecture too, where we decided not to pass these big message bodies. And instead we just passed 247 00:24:49,640 --> 00:24:58,120 IDs to the actual, uh, code or the X, the XML payload. And so we, we, we, we dramatically reduce the cost of 248 00:24:58,120 --> 00:25:04,200 syncs, uh, of, uh, serializing and deserializing this large payload. So we optimized as a result of that 249 00:25:04,200 --> 00:25:10,200 choice. We optimized the architecture to an engineer to achieve, to achieve that, uh, objective. 250 00:25:10,200 --> 00:25:16,760 Yeah. Okay. I think that that's, that's helpful. And then also starts to get us to our next question, 251 00:25:16,760 --> 00:25:23,320 which is around, you know, if you were to do this differently, do it all again, you know, 252 00:25:23,320 --> 00:25:26,920 what would you do differently? And, and what, what do you think you would do the same? 253 00:25:30,440 --> 00:25:36,520 I mean, I, I think I would pretty much do, it would be just repeat. I mean, it was good, but 254 00:25:36,520 --> 00:25:41,240 the only, only difference is, uh, we have that experience with us. We have that, uh, you know, 255 00:25:41,240 --> 00:25:47,880 what we did, so that will help us. So we don't really have to, you know, be in that position that 256 00:25:47,880 --> 00:25:51,400 Rohit was describing, right, where it gets uncomfortable. So at least we know some of those 257 00:25:51,400 --> 00:25:56,520 pains we can, uh, you know, upfront address, but some things that I would want to call out that we 258 00:25:56,520 --> 00:26:02,280 would continue to do, uh, you know, is, is we would definitely want to do as driven development. 259 00:26:02,280 --> 00:26:08,920 We definitely want to do their programming, uh, which we have, you know, a kind of practice, uh, 260 00:26:08,920 --> 00:26:15,880 and, you know, not just, uh, survive, but tried as well. Um, and, and of course, you know, 261 00:26:15,880 --> 00:26:22,360 we discussed, we talked about it in the earlier segment where, um, the granularity or, so just keep 262 00:26:22,360 --> 00:26:26,360 it simple. Like, so we said, okay, let's do it. Uh, what we used to call it as a 263 00:26:26,360 --> 00:26:33,640 shoddy viable product SVP, just keep it, you know, get something, uh, to be there, start, 264 00:26:33,640 --> 00:26:39,400 start building that, and then, you know, improve iteratively, um, and then build a lot of automation. 265 00:26:39,400 --> 00:26:44,840 So those things I would definitely would, uh, do it no matter what project, uh, uh, it is just, 266 00:26:45,400 --> 00:26:50,920 not just legacy transformation, but any project. So if it means I have to redo this project, I would, 267 00:26:50,920 --> 00:26:58,760 you know, pretty much do it, uh, again. And there is, there is things like, okay, uh, one thing is, 268 00:26:58,760 --> 00:27:05,000 as Rohit said, the thing that I would do differently, uh, yes, we did testing to make sure the rates match, 269 00:27:05,000 --> 00:27:12,200 uh, but there were things, uh, you know, like, how does this fit overall into the ecosystem? So as we 270 00:27:12,200 --> 00:27:18,600 kind of started integrating this with different applications, different things, uh, we had to kind 271 00:27:18,600 --> 00:27:24,360 of, you know, figure things out or they were like, Oh, gotcha moment. Like, okay. Uh, we, we had to 272 00:27:24,360 --> 00:27:29,400 go back, you know, tweak things, come back. Um, if we were to do it again, I would probably, 273 00:27:29,400 --> 00:27:35,640 you know, kind of have some focus on making sure that we do, uh, overall, I mean, we have a lot of, 274 00:27:35,640 --> 00:27:40,520 uh, you know, unit testing and lot of integration testing within this immediate, uh, you know, 275 00:27:40,520 --> 00:27:46,040 application vicinity, but make sure that the application or the product that you are kind of, 276 00:27:46,040 --> 00:27:51,240 you know, fitting in, uh, especially in legacy transformation, we do a lot of end to end regression 277 00:27:51,240 --> 00:27:58,040 testing as, uh, and often as you can, and as early as you can. So it may not be your system that has an 278 00:27:58,040 --> 00:28:04,760 issue or that that would create problems, but how those things are consumed or, you know, the moments 279 00:28:04,760 --> 00:28:09,080 and sometimes they are like, so innocuous that you don't even realize that, Oh, this is what, you know, 280 00:28:09,080 --> 00:28:14,840 how the others are consuming it, or they are kind of reading it. That is the only thing that I would 281 00:28:14,840 --> 00:28:19,800 be a little more, uh, you know, put more emphasis on, but there is not much that I would change from 282 00:28:19,800 --> 00:28:25,960 what we had done. Okay. I would like to someone like Rohit being there. And if I could repeat, 283 00:28:25,960 --> 00:28:30,120 like, I would want, you know, someone like him to be there next time we start as well. But that's, 284 00:28:30,120 --> 00:28:34,680 that's one thing to see our team also underwent some churn, right? We actually remember like, 285 00:28:34,680 --> 00:28:42,680 as the project went on, we lost a few key members. If I had to redo it again, then we, we, we wouldn't 286 00:28:42,680 --> 00:28:53,240 allow them to go. We had an awesome team to begin to start with too. Um, as an, but testament to the 287 00:28:53,240 --> 00:28:58,200 architecture, even when we lost some talented people, like new people came in and it's still, 288 00:28:58,200 --> 00:29:04,600 it's still grew stronger from there. But, but when some of the critical folks leave, it does leave that 289 00:29:04,600 --> 00:29:11,560 vacuum for a little bit. So I guess that's just life. But, um, there is kind of just a statement on, 290 00:29:11,560 --> 00:29:17,160 on, on, when a project runs, runs long like that, your team composition will also change. 291 00:29:17,160 --> 00:29:22,600 As long as there is, there is an anchor, strong anchor, uh, like Viraj, then things work out. 292 00:29:22,600 --> 00:29:27,720 Uh, yeah, but we also had like the, you know, the, the biggest thing that, you know, we had was the 293 00:29:27,720 --> 00:29:32,120 pair programming because even though people leave, they don't take knowledge with them because the 294 00:29:32,120 --> 00:29:37,160 knowledge is spread across, right? So everyone knows about everything that they may not really be 295 00:29:37,160 --> 00:29:40,840 like super experts in it. Like, oh, I know, but everyone knew, everyone knew, 296 00:29:40,840 --> 00:29:47,400 uh, enough about it to kind of go into it, fix it, or, uh, look into it and understand it. 297 00:29:47,400 --> 00:29:51,080 Uh, so that was the biggest thing. Everyone understood different components. Everyone 298 00:29:51,080 --> 00:29:56,360 understood the decisions. Uh, it, it is a team, you know, effort where we were not saying, okay, 299 00:29:56,360 --> 00:30:00,680 you know what, we are making a decision. Go ahead. Just developers just implemented. We kind of 300 00:30:00,680 --> 00:30:05,320 collectively, you know, discuss decisions. Uh, it was a collective responsibility. So as to say, 301 00:30:05,320 --> 00:30:11,880 hey, and as a result, a lot of, uh, you know, team members, when the experienced folks or those who 302 00:30:11,880 --> 00:30:16,680 have been, uh, you know, around left, new came in, it was not difficult because others were there to 303 00:30:16,680 --> 00:30:24,200 onboard them. Uh, and it, it just felt like the, the anchors, you know, help, um, in, in that sense. 304 00:30:24,200 --> 00:30:30,280 So, yeah, it, it sounds like from the things that you listed, you would, you would, you would keep 305 00:30:30,280 --> 00:30:36,760 were some of these factors that really help build resiliency against things like departures from the 306 00:30:36,760 --> 00:30:43,080 team. And, and I think that that's something worth calling out because like no one's plan for building 307 00:30:43,080 --> 00:30:49,560 an app or modernizing an app, no one's plan includes X amount of churn on the team doing it. Like 308 00:30:49,560 --> 00:30:56,520 that's, that's sort of this, it's always a surprise. And yet if you were to go and measure how all of 309 00:30:56,520 --> 00:31:04,040 these projects work out, you would probably find a statistical measure of you should anticipate, 310 00:31:04,040 --> 00:31:09,160 depending on this amount of time, you should anticipate this amount of churn. And, you know, 311 00:31:09,160 --> 00:31:14,200 it's not necessarily good or bad. It's just that this is statistically what you should anticipate. 312 00:31:14,200 --> 00:31:21,160 And like, as the time goes on, then, you know, it's not necessarily linear, right? You can probably 313 00:31:21,160 --> 00:31:28,520 expect a slightly increasing rate of churn over duration, which just has to do with life, as you 314 00:31:28,520 --> 00:31:33,800 said. And then if, you know, if you're statistically, you know, if you're out of bounds of that sort of 315 00:31:33,800 --> 00:31:39,400 standard deviation, then you, you might have a problem. But no, the point is no one plans for 316 00:31:39,400 --> 00:31:45,240 that. And yet it's a reality. And so having things like, you know, pair programming and, you know, 317 00:31:45,240 --> 00:31:51,560 test driven development helps sort of bake in resiliency against it. And then just to make sure 318 00:31:51,560 --> 00:31:56,360 I capture everything else, sort of keeping it simple with sort of the shoddiest viable products, 319 00:31:56,360 --> 00:32:04,200 MVP approach, building a lot of automation, and then adding in the end-to-end integration testing 320 00:32:04,760 --> 00:32:10,120 early and often would be kind of like the, these are the things you don't want to skimp on. 321 00:32:12,040 --> 00:32:17,960 So, and this, I've got one more question that I want to sort of shift gears to, which is 322 00:32:17,960 --> 00:32:25,160 now starting to look at the stack, the technology side, and understanding, you know, how you made 323 00:32:25,160 --> 00:32:32,520 decisions at selecting different parts of the technology stack from, you know, .NET and, you know, 324 00:32:32,520 --> 00:32:40,840 were you using the steel toe components? And you mentioned you had a cloud platform. I assume it's 325 00:32:40,840 --> 00:32:46,360 Pivotal platform, you know, how did that get selected? You know, what infrastructure is it running on 326 00:32:46,360 --> 00:32:52,360 underneath? Is it in-house? Is it on a cloud provider? And how, how did those decisions kind of get made? 327 00:32:55,240 --> 00:33:05,720 Well, as far as .NET is concerned, within at least personal lines, we have .NET as a standard platform. 328 00:33:05,720 --> 00:33:10,760 Though we have Java, we work on, you know, different things, but looking at the skill set, 329 00:33:10,760 --> 00:33:15,720 looking at, you know, the team strength, .NET is our go-to platform. If it is new thing, 330 00:33:16,200 --> 00:33:21,960 unless it's something else fits in better, like if it is something that we should develop it 331 00:33:22,440 --> 00:33:26,920 based on the availability of either tools or whatever, you know, kind of blends it into node 332 00:33:26,920 --> 00:33:31,640 or something else, we tend to, you know, kind of, so .NET is given. And we, in this case, we decided to go 333 00:33:31,640 --> 00:33:36,600 .NET Core because we wanted to, you know, kind of deploy it on the cloud. And, you know, tomorrow, 334 00:33:36,600 --> 00:33:40,200 if it is a multi-cloud platform, we can't always say, oh, we have to be on a Windows 335 00:33:40,920 --> 00:33:47,960 stack. So .NET Core was given. Steel 2 was, again, something, you know, as a best practice that we 336 00:33:47,960 --> 00:33:55,400 learned from, you know, working with Pivotal. It's a great, you know, tool, something that will help 337 00:33:55,400 --> 00:34:00,280 you build your microservices. I mean, you don't want to reinvent the wheel, you know, just to describe 338 00:34:00,280 --> 00:34:04,360 Steel 2. Like, there are things that it provides that otherwise I would have had to write, like, 339 00:34:04,360 --> 00:34:09,480 you know, all this connection, you know, health check, what have you, right? So why would I write 340 00:34:09,480 --> 00:34:15,640 that if it is already, you know, provided? It makes my life easy. And if it is, you know, something that 341 00:34:15,640 --> 00:34:23,800 is constantly enhanced and updated. So that was also one thing that we definitely wanted to, we started 342 00:34:23,800 --> 00:34:29,720 with this. And we wanted to now kind of continue that all the new things that come up, we say Steel 2 is 343 00:34:29,720 --> 00:34:38,600 must, you should use it. Other things, you know, in terms of the Pivotal Cloud, 344 00:34:40,120 --> 00:34:46,200 it was a thing that we had in-house. I don't know when it was chosen or how that decision got made, 345 00:34:46,200 --> 00:34:50,520 but it was not specific like, oh, we want to do this and this has to be on Pivotal. But it was a, 346 00:34:50,520 --> 00:34:56,920 you know, on-prem cloud platform that we have. It had been, you know, kind of, we had been working on 347 00:34:56,920 --> 00:35:04,280 slowly to kind of move application, harden it and kind of make sure that we can leverage it the way it was, 348 00:35:04,920 --> 00:35:10,840 you know, intended to be. So a lot of things that we did were like, kind of no brainer, like, okay, 349 00:35:10,840 --> 00:35:18,360 yeah, we have to, there has to be some of those things that we had to use. A, because they were 350 00:35:18,360 --> 00:35:26,360 the existing tool set, or because, you know, they were the right tools for the things that we were, 351 00:35:26,360 --> 00:35:30,360 you know, trying to do. Okay. I don't know if you want to add anything, 352 00:35:30,360 --> 00:35:36,520 because you have been involved in, with Pivotal and with Travelers for a long time, so. 353 00:35:36,520 --> 00:35:41,640 Well, and if there's any other kind of technology selections that were made, you know, 354 00:35:41,640 --> 00:35:46,760 that I didn't mention just because I don't know about them, you know, what else kind of went into it 355 00:35:46,760 --> 00:35:54,120 that you can share? So we used like, we used .NET Core for most of our development. We used Steeltoe 356 00:35:54,120 --> 00:35:59,560 as one of our frameworks for connecting to services on the platform for, for health checks, 357 00:35:59,560 --> 00:36:09,960 for service discovery, for actuators. Then we used the tools that the Pivotal platform at Travelers has 358 00:36:09,960 --> 00:36:19,080 for logging and metrics and monitoring. So yeah, we also used like Rabbit and Redis. Like, these are like, 359 00:36:19,720 --> 00:36:27,160 you know, marketplace services that we have within Pivotal at Travelers. So. Rabbit was our, Rabbit was our messaging 360 00:36:27,160 --> 00:36:33,640 broker, right, where we exchanged all our messages for one of the rate codes. All our microservices communicated 361 00:36:33,640 --> 00:36:42,440 through, on the RabbitMQ messaging, on the queues to communicate across one another. And then Redis was 362 00:36:42,440 --> 00:36:48,520 the cache to hold the big payloads. And so that we could only send references across. We didn't have to 363 00:36:48,520 --> 00:36:57,400 transfer the whole payload. It was Rabbit, Redis, Steeltoe, and .NET Core, kind of stack. 364 00:36:58,120 --> 00:37:03,240 And so I imagine, oh, go ahead. No, I'm sorry. I mean, that's pretty much the large, you know, 365 00:37:03,240 --> 00:37:10,760 part of what the toolset we had. Then we had something like, for some of these, you know, like when we had 366 00:37:12,840 --> 00:37:19,640 some abstract rules that we had to build and validate, we used, you know, it's an old framework called fitness. 367 00:37:20,360 --> 00:37:25,320 But we just wanted to make sure that we have automation all around. So we just wanted to make sure that 368 00:37:25,320 --> 00:37:31,320 we do use that to validate some of these rules. So anyone like, you know, a business analyst could say, 369 00:37:31,320 --> 00:37:36,360 hey, I want to try, you know, this variable, this variable, and see that rule is applied or 370 00:37:36,360 --> 00:37:42,520 it is able to, you know, produce the same outcome that I'm intending it to. And we had that as a part 371 00:37:42,520 --> 00:37:47,880 of our build pipeline where, you know, we would want to make sure that it passes the fitness, you know, 372 00:37:47,880 --> 00:37:52,840 test. And it's a wiki-waste, you know, tool. It's old. I mean, I don't think so a lot of people 373 00:37:52,840 --> 00:37:59,160 use it or know it, but it is something that we had. We used it as well. 374 00:37:59,160 --> 00:38:01,400 And what were you using for the build pipeline? 375 00:38:02,680 --> 00:38:14,360 We were using Jenkins for our CI. And for CD, we have a tool called UCD, urban code deploy. So 376 00:38:14,920 --> 00:38:22,760 Jenkins was hooked on to UCD and Git was our repository. So we used code in our check-in in Git. 377 00:38:24,040 --> 00:38:30,600 And then whenever we checked in the CI/CD pipeline, we had like other things that were looking at code 378 00:38:30,600 --> 00:38:36,360 quality. Like we had SonarQ, which was looking at what it is. And then we have BlackDuck, which looks 379 00:38:36,360 --> 00:38:41,480 at vulnerabilities and stuff like that. So those were built in. Those are standard. There was nothing 380 00:38:41,480 --> 00:38:48,120 specific to our project, but they are like a part of our overall pipeline. 381 00:38:48,120 --> 00:38:55,640 Yeah. And I mean, just even kind of what you mentioned about, was it called fitness? 382 00:38:56,680 --> 00:38:59,240 Yes. F-I-T-N-E-S-S. Yeah. 383 00:38:59,240 --> 00:39:02,360 Right. Like the Nike fitness training app or something. 384 00:39:02,360 --> 00:39:10,120 Yeah. I think if you search on fitness, F-I-T-N-E-S-S-E or something, it's a wiki-based, 385 00:39:10,120 --> 00:39:13,560 you know, thing. So it's very simple to configure. You just need certain setup. 386 00:39:15,720 --> 00:39:22,040 Like one of my team member had that experience. He suggested, hey, let's use this for, you know, 387 00:39:22,040 --> 00:39:26,600 this rule testing, like rather than building something or rather than, you know, kind of 388 00:39:26,600 --> 00:39:32,680 having someone go through the code or run unit tests just to validate, you know, like if he or she 389 00:39:32,680 --> 00:39:39,240 is able to test the rules independently. So we use that, like, you know, just because we had that 390 00:39:39,240 --> 00:39:44,200 tool, you know, or somebody knew about it. Yeah. I mean, I think, and also like the idea of being 391 00:39:44,200 --> 00:39:49,080 able to put that in as part of your built pipeline so that any of these, a lot of folks think, okay, 392 00:39:49,080 --> 00:39:53,880 well, yeah, we've got to run our unit tests and our integration tests and we have that automated. 393 00:39:53,880 --> 00:39:59,800 And it's like, well, if you think about the full path to production, there's these other points of 394 00:39:59,800 --> 00:40:05,160 validation and the, to the extent that you can automate those, you're going to make subsequent 395 00:40:05,160 --> 00:40:11,000 code changes that much more frictionless. So I think it's always interesting to hear like, okay, 396 00:40:11,000 --> 00:40:16,120 what are, what are some of the different steps and checkpoints people do have to make sure that 397 00:40:16,120 --> 00:40:23,640 they've got production ready code and that they can make those changes easily. So, okay. Thanks for 398 00:40:23,640 --> 00:40:29,720 the link. I'll, I'll include that in the show notes. And of course I have to ask a little 399 00:40:29,720 --> 00:40:36,760 bit about rabbit. Cause I spent a lot of time with that team, you know, you, you, you mentioned 400 00:40:36,760 --> 00:40:45,720 the, um, kind of the latency sensitivity, uh, and how making sure that, uh, the different microservices, 401 00:40:45,720 --> 00:40:52,040 they're all communicating via rabbit. Were you tuning that, uh, the, the, that sort of 402 00:40:52,680 --> 00:40:59,080 set up with rabbit so that you were getting kind of the, the, the lowest latency and highest performance results or 403 00:40:59,080 --> 00:41:03,960 what were you doing on the, on the, on that side, if, if you're willing to dive into that? 404 00:41:06,840 --> 00:41:12,920 Um, I don't think we did a whole lot of tuning. There are certain things that we did is, uh, you know, 405 00:41:12,920 --> 00:41:20,280 like the application that we have is important, is critical. Uh, so we, we had to make sure that what we use, 406 00:41:20,280 --> 00:41:27,080 uh, is on demand or, you know, kind of dedicated in, you know, in an, uh, instance of rabbit, 407 00:41:27,080 --> 00:41:31,400 it's not a shared instance, uh, so that we don't have a noisy neighbor, you know, who kind of just 408 00:41:32,120 --> 00:41:39,080 influences, uh, you know, or kind of impacts. Um, we did use like the three node cluster so that 409 00:41:39,080 --> 00:41:46,360 there is some amount of resiliency, uh, you know, into that. So if one server goes down, one VM goes down, 410 00:41:46,360 --> 00:41:53,080 and we, we are still able to carry on and, and as you know, kind of Rohit alluded, uh, we were not 411 00:41:53,080 --> 00:41:59,640 really putting in a big message, uh, or like the, the message size was big, but we were not putting 412 00:41:59,640 --> 00:42:04,280 that into rabbit. So all we were doing is just a notification with the ID saying, Hey, you know, 413 00:42:04,280 --> 00:42:11,960 this is the ID, you know, go grab it from the cash and work on it. So that way we kind of kept it, uh, 414 00:42:11,960 --> 00:42:18,360 minimal. Uh, and, and then we also made sure that our pre-fetch count is set enough so that you don't, 415 00:42:18,360 --> 00:42:23,240 you know, constantly hit, you know, rabbit to get work. So you would say, okay, my pre-fetch count is 416 00:42:23,240 --> 00:42:29,320 10 and, you know, take work on 10 things and then, you know, go back and get some more. Um, 417 00:42:29,320 --> 00:42:33,240 that pre-fetch count is something that we had to try out, you know, a few times, or, you know, 418 00:42:33,240 --> 00:42:39,560 kind of make sure that it's, it's appropriate, uh, and then we are not, uh, hitting. So these are 419 00:42:39,560 --> 00:42:45,960 some of those things that, that I am aware of that we, you know, kind of, uh, had to learn, had to work 420 00:42:45,960 --> 00:42:52,440 on. But, um, other than that, I can't, I'm still thinking, but I don't think we did anything, uh, 421 00:42:52,440 --> 00:42:55,800 differently. I mean, Rohit, do you know of anything that you? 422 00:42:55,800 --> 00:43:00,600 Yeah. So, uh, Viraj, remember we used the dead letter queue handling from Rabbit? 423 00:43:00,600 --> 00:43:01,560 Oh, dead letter, yeah. 424 00:43:01,560 --> 00:43:07,080 We had the DLQ handling, which we used. Then we also measured end-to-end latency 425 00:43:07,080 --> 00:43:13,720 with distributed tracing with Zipkin, right? So that we could tell for each where, what portion of the 426 00:43:13,720 --> 00:43:20,280 entire latency was being spent in each, uh, microservice. What was that span worth? What did 427 00:43:20,280 --> 00:43:25,080 we do distributed tracing? Or am I not? I think it, it was not, Zipkin was not, uh, available 428 00:43:25,080 --> 00:43:32,440 for .NET Core. So we, you know, we were hoping that it would be, but it was not. So we never got 429 00:43:32,440 --> 00:43:38,520 to that point. Um, but, but we use DLQ. I remember that, right? Yeah. Yeah. Dead letter queue for sure. 430 00:43:38,520 --> 00:43:43,720 Uh, the other thing that we, you know, since, uh, you talked about dead letter queue, one of the things 431 00:43:43,720 --> 00:43:51,320 that we had seen in some cases, uh, uh, is if the message stays long in the queue and we could not 432 00:43:51,320 --> 00:43:57,160 really, you know, always reproduce that or say, Hey, this is why it happens. Um, we would lose or, 433 00:43:57,160 --> 00:44:03,560 you know, sometimes the transaction would just time out after like 20 seconds or 30 seconds because it 434 00:44:03,560 --> 00:44:09,640 was just in the queue and every, you know, then the calling system would time out. So what we did was 435 00:44:09,640 --> 00:44:15,000 we implemented time to live, you know, kind of on the, on the messages. So we didn't want to really, 436 00:44:15,000 --> 00:44:20,120 you know, clutter in, even in, in, in accidentally that, Oh, messages leaving in there or, and that 437 00:44:20,120 --> 00:44:26,840 they're not getting cleaned up. So we just make sure that, um, beyond a certain time, the messages 438 00:44:26,840 --> 00:44:31,640 should not be in, they just get moved out to dead letter queue and they handle, they are handled as an 439 00:44:31,640 --> 00:44:37,480 error. So we also had great support from the business team, right? Remember they, they looked at all 440 00:44:37,480 --> 00:44:43,080 the validations and made sure that the, like all the business rules for mapping of all those policy 441 00:44:43,080 --> 00:44:48,440 things were correct. So I want to give a shout out to that as well, because like there was a lot of 442 00:44:48,440 --> 00:44:55,720 business rules in terms of just like mapping and derivations that were on the ZOS side that we 443 00:44:55,720 --> 00:45:02,600 ported to distribute it. And without their help, I mean, that would not have been do a practical, uh, 444 00:45:02,600 --> 00:45:10,200 possible. Right. Yeah. Sorry. That's another team inside, uh, travelers that was just good to 445 00:45:10,200 --> 00:45:15,960 collaborate with. Yeah. These, these, this was the, you can think of these domain as like the 446 00:45:15,960 --> 00:45:25,320 business analysts, the, the, the ZOS domain experts of that system that knew and had that had in some 447 00:45:25,320 --> 00:45:34,520 ways worked with the system for a very long time. Yeah, no, I think you're right. We had at least two 448 00:45:34,520 --> 00:45:42,120 individuals, uh, who were part of our team, um, who provided that it was one who was mainframe, you know, 449 00:45:42,120 --> 00:45:49,720 expert, not just mainframe, but, uh, she had worked on a lot of, you know, legacy apps, um, including, you know, 450 00:45:49,720 --> 00:45:56,520 the one that we were, you know, kind of moving from legacy. So she was working with us all throughout 451 00:45:56,520 --> 00:46:01,240 just to kind of help us, you know, kind of answer our questions or in some case, you know, guide us 452 00:46:01,240 --> 00:46:05,640 saying, Hey, and in, as we said, you know, validating certain things that we were doing, making sure 453 00:46:05,640 --> 00:46:10,760 that, okay, yes, they indeed are the right way to do. So there was that, uh, you know, tremendous 454 00:46:10,760 --> 00:46:15,320 collaboration and not over the email. That was, you know, the thing, like, it's just not like we were 455 00:46:15,320 --> 00:46:19,960 sending emails and, you know, we got, we got response. We were sitting right next to each 456 00:46:19,960 --> 00:46:24,920 other. Uh, we were sitting in the same, you know, uh, area where we could just walk over 457 00:46:24,920 --> 00:46:29,240 or we could, you know, uh, request for a quick huddle and say, Hey, let's talk about this. We 458 00:46:29,240 --> 00:46:34,680 discovered this. How do we fix it? So we could, you know, address things quickly. Interesting. Yeah. 459 00:46:34,680 --> 00:46:39,880 I think that that's also something that, you know, sometimes folks thinking about co-locating, 460 00:46:39,880 --> 00:46:45,000 co-locating teams and whatnot, that that's a good piece of feedback for folks. Um, 461 00:46:45,000 --> 00:46:49,560 you know, the value of that. And one more thing that I just want to quickly add was, 462 00:46:50,120 --> 00:46:55,000 um, since we were moving at a fast pace and, you know, we were kind of doing things, 463 00:46:55,720 --> 00:47:01,480 uh, we also had a focus on automating things, right? So as we described earlier, uh, that we 464 00:47:01,480 --> 00:47:06,200 used to compare rates on, you know, the old and the new system, again, you know, we are talking about 465 00:47:06,200 --> 00:47:11,240 not one, two, or, you know, 10, we were talking about like thousands of such transactions running, 466 00:47:11,240 --> 00:47:15,560 and there is no way you could do it without having some automation. So we wrote a tool, 467 00:47:15,560 --> 00:47:21,800 uh, you know, which would kind of automate. So we just had to say, Hey, here are the, you know, 468 00:47:21,800 --> 00:47:27,800 here is a database, go get that, uh, you know, payload, uh, fire these, uh, you know, uh, requests, 469 00:47:27,800 --> 00:47:34,200 compare the output and, you know, uh, spit out the result as to what is the accuracy. Like it's 99% 470 00:47:34,200 --> 00:47:40,280 matching a 5% and use that feedback. Like, okay, if it didn't match, what were those transactions 471 00:47:40,280 --> 00:47:44,920 and why did they not match? Uh, and maybe, you know, in, in many cases we discovered things that, 472 00:47:44,920 --> 00:47:52,120 Oh, which we missed or which we didn't do. Uh, so it helped us, you know, kind of fine tune and not 473 00:47:52,120 --> 00:47:56,760 fine tune, but in some many cases, fix things that we would have missed and, you know, went in and 474 00:47:56,760 --> 00:48:01,720 we could have gone in production. Uh, where I'm coming is automation is critical, you know, 475 00:48:01,720 --> 00:48:07,400 especially when you are building something, um, like this is an, in legacy, uh, you know, thing 476 00:48:07,400 --> 00:48:13,480 where you need to compare, uh, you can compare at scale. And if you have automation, it just helps 477 00:48:13,480 --> 00:48:22,200 you, you know, kind of deliver things, um, if not faster on time that you promised. Yeah. Um, yeah, 478 00:48:22,200 --> 00:48:26,600 no, that's a great point. And that's, you know, in this case, it was very specific to the application 479 00:48:26,600 --> 00:48:32,040 that you were working with. Uh, but it was worthwhile to build some automation specific 480 00:48:32,040 --> 00:48:37,000 to that just so that you could make these changes with confidence. So in some ways it's, it's, you 481 00:48:37,000 --> 00:48:41,800 know, to me, it's not unlike how you need to write some of those tests first to know that you haven't 482 00:48:41,800 --> 00:48:47,240 broken the functionality. You need to automate those things so that you can know that you're doing it 483 00:48:47,240 --> 00:48:56,840 efficiently. Um, okay. Well, I'm just, uh, trying to be mindful of time. This has been really, really 484 00:48:56,840 --> 00:49:03,400 interesting and useful to kind of understand the, the process and the tooling and, uh, the challenges 485 00:49:03,400 --> 00:49:08,440 along the way, the trials and tribulations of modernizing an application off of the mainframe 486 00:49:08,440 --> 00:49:16,600 into a .NET core stack, um, that's, you know, ready to scale on demand and making use of a lot of modern 487 00:49:16,600 --> 00:49:23,240 technologies. So, um, do you have any last words on, you know, how this is delivering value back to the 488 00:49:23,240 --> 00:49:33,080 business so far? Um, yeah, I mean, they're happy. I mean, at least from what I see that they are making 489 00:49:33,080 --> 00:49:39,560 plans around kind of saying, how do we use this, you know, uh, not just in like today we are doing 490 00:49:39,560 --> 00:49:43,240 it for one line of business, but they are already saying, Hey, how can we do this for the next line 491 00:49:43,240 --> 00:49:48,680 of business? Uh, you know, we want something similar. Um, and we are starting with some other 492 00:49:48,680 --> 00:49:55,320 initiatives where we are going to do, uh, what we have done in this case, uh, build a brand new, 493 00:49:55,320 --> 00:49:59,320 you know, thing in this case, in that case, it is not going to be a legacy transformation. So as to say, 494 00:49:59,880 --> 00:50:06,520 um, but it is modeled on, on the same thing. So again, in a day, if they were not confident that 495 00:50:06,520 --> 00:50:12,840 this is working or if this is not something that is producing value, um, no one would have really, 496 00:50:12,840 --> 00:50:19,720 you know, kind of, uh, supported that, accepted that. Um, and, and then I can, I can see that they 497 00:50:19,720 --> 00:50:26,600 are making plans for what we have built to be used, uh, more effectively and in, you know, in a much wider, 498 00:50:26,600 --> 00:50:30,600 you know, thing we started with something small only for a few states and only for, you know, 499 00:50:30,600 --> 00:50:36,520 certain things. Um, now, you know, we are kind of expanding it gradually, you know, making sure that 500 00:50:36,520 --> 00:50:44,360 we soak in all those things that are discovered, uh, and then we are going. So I think I do see a lot 501 00:50:44,360 --> 00:50:52,280 of excitement and a lot of, uh, you know, um, uh, opportunities that business, uh, can leverage 502 00:50:52,280 --> 00:50:56,840 using what we have built. So, yeah, I mean, that sounds like there's a lot of confidence in the 503 00:50:56,840 --> 00:51:04,840 process and the results. So, you know, success breeds success. Um, Rohit, any, any last words from you 504 00:51:04,840 --> 00:51:10,040 on folks who are kind of looking at making these types of changes and what you would recommend for 505 00:51:10,040 --> 00:51:12,280 how they get started on those engagements? 506 00:51:12,280 --> 00:51:18,760 Yeah. Get started. I mean, start, that's the big thing. Like, uh, don't expect it to, to be easy, 507 00:51:18,760 --> 00:51:26,200 but if the organizational commitment is there, uh, you will make progress and over time, the, 508 00:51:26,200 --> 00:51:32,600 the results will be manifold. So also understand that it's a journey, what the discussion about the 509 00:51:32,600 --> 00:51:38,280 results and outcomes that we are having here is a result of a two year project, right? That we started. 510 00:51:38,280 --> 00:51:43,800 Um, this is something like, it's like, we're discussing, we're reviewing a movie here, which 511 00:51:43,800 --> 00:51:48,680 is just two hours long, but it takes like eight, eight, nine months to make, right? So yeah, uh, 512 00:51:48,680 --> 00:51:55,880 it's a, it's a process. It's well worth it. Um, and you realize real costs. You realized all those 513 00:51:55,880 --> 00:51:59,720 three things that we discussed at the beginning of the call, right? Uh, you realize cost savings, 514 00:51:59,720 --> 00:52:05,960 you become more agile. Uh, you now have a set of technologies that your more modern workforce can 515 00:52:05,960 --> 00:52:13,080 work with. So, uh, if cost, uh, skills and agility is what you need, then you need to put forth a 516 00:52:13,080 --> 00:52:21,720 program like this to get more details. Sure. Sure. Listen in on our June, not June, January 23rd, uh, 517 00:52:21,720 --> 00:52:29,960 webinar. So we'll get into the deep details and cover all the gory, uh, bits of the promotion. 518 00:52:29,960 --> 00:52:36,440 And with visuals, those are always fun, you know, nothing like a good, uh, you know, flow chart or 519 00:52:36,440 --> 00:52:42,680 network diagram or something so that we can wrap our heads around what the heck is going on with, uh, 520 00:52:43,640 --> 00:52:50,280 these microservices everywhere. All right, gentlemen, thank you so much for, for joining us and, and 521 00:52:50,280 --> 00:52:56,280 sharing your experience. This has been, uh, tremendously insightful. So thanks. And we'll look forward to the, 522 00:52:56,280 --> 00:53:05,880 uh, the, the webinar in January 23rd. Thank you. Thank you.