Beyond ”Survival Is Not Mandatory,” the Cloud-Native Cookbook (Ep.54)

Episode 41 · 44:59 · SRT · archive.org · all Tanzu Talk

Now, you seem like the kind of person that if you woke up on the wrong side of the bed, you would have a couple of remediation programs that you would run. So let me ask you that. When you wake up and you're like, I'm not feeling it today. Like, how do you pump yourself up? What do you do? Man, if you mean pump me up myself up for work, like I just have a routine. So I start my morning. Or I just mean like life. Just life. You just like, you know, you feel like you want to pull a Garfield and get back in the cat box, right? Yeah. But you're like, you want to be more like an Odie, maybe. Yeah. Well, usually every morning it starts with a dog walk. So I get to clear my head a bit as I watch my dogs carefree sprinkle the yard. A little afternoon spray. Well, you know, that's a good metaphor for just like getting on with improving yourself that I find myself. I was even writing this in some document. And I guess we'll talk about this in the second half. But as I often tell my seven-year-old son, you know, he's doing this project, a living museum, which is popular with the kids nowadays. I don't know if you have that up in Donut Land there. But like, you know, he pretends to be a person and you go up and he like, I guess he's supposed to, he's too young to be able to jolt like an old animatronic robot. But if you imagine a kid doing that, he's supposed to, he's supposed to talk about John Glenn. So we had to look up some facts and do things. And, you know, we're working on that this weekend. And he struggles to not just play Minecraft all the time, which is understandable. There's, you gotta, you gotta get the, get the, get the gold and the iron. I don't know what else. I found this cave in there that has an infinite series of zombies coming out. I don't know if that's a bug or a feature, but it's really annoying. Anyhow, I was telling him that like, you know, if you don't want to do work, the best thing to do is just hurry up and do it so you can be done. That's right. But I don't think that's getting through. You know, it's sort of like, it's sort of like getting out in the morning and taking a walk. You should just wake up 30 minutes earlier and go take a walk.

It'll probably turn things around for you. Just do it. I'll tell you, I wrote my best blog post on dog walks. You know, heads clear. Just, you know, I don't know. Secret to life is perspective for me. Sometimes watching dogs carefree run around really helps me out. This reminds me, I've considered many times rewatching all of the West Wing, just so I can find this one scene where it's during the State of the Union address they're working on. There's, I think his name is Toby. He's like the chief of communications and kind of chief writer, very moody, like brooding person. Yeah, and it's like, I think it's maybe like two days before the State of the Union speech and he's in a bar playing pool and someone walks up to him and says, shouldn't you be working on the speech? And he just looks up to them and sighs and he says, I am working on it. There you go. That's my life in a nutshell, without the pool. So speaking of waking up in a poor state, there was a big Amazon outage last week. Yeah, that was great for brand awareness. Yeah, for Amazon brand awareness. Yes, yes. Yeah, you know, I was reading through the great brief write-up they had. First of all, I think I was telling you there's certain people that we work with who we need to introduce paragraphs to, but that's fine. I think maybe we should sit down, whoever wrote that, and the same thing. It's just put a break in there. It's good. Have a little paragraph. But it was nice to read their write-up. I like the kind of honesty in it. There was a bit of like, you know, as you would do, assuring that things are good. But it was basically just like, yeah, we just entered a command wrong, and then more importantly, we're going to make sure they went over, I'm imagining, a safeguard they were going to put in. I mean, basically what they're doing is they're updating some storage stuff on S3, and then because they mistyped a command, they brought down some other ways of managing S3 things. I forget what they did exactly, but the effect was that it made a lot of the S3 things like reboot probably to fix themselves automatically or something, and then, you know, problems. Right.

But I like the pretty simple explanation that even a, you know, a old country lawyer like myself can understand that, like, it's just like, they're like, yeah, we made it so that it actually slows down to do that, or we put some governors in there so that if we enter that in, it will catch it or something like that. I like the, I don't know, sort of just like pragmatic down-home-iness of it. Yeah, I can't. I still try to put myself in the shoes of the person who did it, though. Like, I took down the internet. Like, it'd be amazing to be that, you know, operations person who typed that in. But, you know, I think a few things caught people off guard. A, everyone realized how much everyone was using S3, but then it also cascaded to other stuff in the data center. Like, there were a bunch of Amazon services that were also experiencing a lot of latency. And even if you weren't using S3, if you're using these other Amazon services, you were getting hit. So probably good for Amazon themselves to think about, all right, did I have the right sort of circuit breaker stuff in there? Am I failing gracefully among these other services? Why did everything collapse? But at the same time, they recovered in two to three hours, which can feel like a lot. But show me most enterprises who would recover from what they did in less than a week. And I'll be surprised. And this is also going back to the, you should just floss your teeth or walk your dogs or do good behavior. And when you look at methodologies, right, like whether it's agile or DevOps or cloud native, there's this, I'm sure there's some like bias that this is in, in that gigantic thinking fast, thinking slow tone. But it's sort of like, the first mistake is assuming that you're doing something. So the thing that we're telling you you should change to is better or worse than what you're currently doing. And in my experience, you're currently not really doing anything. And so like, it's, it's kind of that point that like, sure, it goes down two hours, but like, what's your uptime? So it is, it's nice that they recovered. And it was, and I tweeted something last night. So for some reason, I felt like spending a half hour yesterday checking out some of the sites that went down.

I went to their status pages to see how did they describe the outage? Did they say like, hey, sorry, our service messed up? Or did they just flat out blame S3? Which again, when I worked for a cloud company and we ran a service, I was taught very clearly, you don't ever blast your, your technology because these are choices you made that either create a single point of failure or whatever that don't blame your technology, you know, blame the, don't blame the tool, blame the person using it. So when I looked, you know, Slack did a good job. They said, look, this service for uploading files is offline. Didn't say the technology, their service was offline. But then you jumped to Trello, you jumped to a bunch of others. Yep, S3 is down. Sorry about that. And for me, that, that, that's a cop out. I think you got to take more ownership of your architectural choices that made your system collapse because one of your dependent services was unavailable. But that's just me. And, you know, not to be overly dismissive, I was reading through some, some stuff, some, some, some description of what's going on out in the industry earlier this morning. And there's, there's, you know, I came across one of the usual stories where sometime in the past, some servers went down and people lost like a million dollars a minute or something while they were down. So, you know, there's stuff that can happen that is definitely terrible. But I think, I think, I think, I think more of the, again, like it's, it's figuring out the right baseline or the floor and realizing that things over time improve if you're operating in a fact-based world. Yeah. And at the end, and this is always the reminder too, because it's easy for, especially now, I guess, marketing people to say, well, you should just build everything super highly available and shame on you for, for hitting outage. But look, you, you get the availability you pay for, and some people don't need to pay for extreme nine nines of uptime that they can, their business can tolerate an hour of being offline versus paying, you know, millions of dollars for redundant infrastructure. I get that. But we're still doing amazing things with technology we weren't doing five years ago. You can probably tolerate a little downtime here and there. Yeah, that, that would be a fun, you know, IT sort of financial planning thing to do over the course of five years is like write down the, the cheapness that you built into the system.

And then when you get to the point where you're losing like a million dollars a minute, be like, yeah, but we recouped that back at the beginning when we didn't pay like $10 million over what we should have. So really not a big deal, but, uh, of course no one really thinks like that. No, it's that pesky perspective again. Yeah. So, so speaking of that, of, of reliability now, one, you know, uh, you can explain what the, what this Google cloud spanner thing is to me. And two, I forgotten what the P and cap is, but there was a lot of more fanciful stuff about reliability where they're like, well, it's like you get all threes of cap, but not really, but like, sort of. Which, which I think, I think, yeah, I think, I think to channel one of my favorite, uh, phrases from Matt Iglesias of the Weeds podcast, I don't, well, that's fine, but I don't think that's how words work. Yeah, I mean, spanners, some neat stuff. So it's a public beta service from Google. It's, it's more or less a database that acts like a regular relational database, you know, SQL syntax, regular asset transactions, schemas, things like that. But that it also supposedly horizontally scales really awesome, which is typically what you equate with a NoSQL database. It does sort of consistency algorithms and does, you know, uh, eventual consistency and replication between nodes that that's not typically how relational databases work. You scale relational by still having, you know, share data files or having read replicas that are slightly out of date, but not too bad. So it's been hard to mung these relational worlds and those SQL worlds if you're trying to get that great consistency story, but also partition tolerance and availability and those sorts of things. So allegedly, spanner kind of gives you all things in the cap theorem. And again, I think there are people saying, well, you're not totally getting this or that, but this does look like kind of a big deal. And so definitely encourage people to read that up. There's some databases kind of based on this model that have also come out. CockroachDB is out that also is like this model.

So pretty cool stuff. They spanner can do some things within the Google service. So this is an open source. This isn't run this in your data center. This is run this in Google. One of the reasons they can do it is because Google owns the kind of synchronized clock that lets these nodes day to date. They can have a greater consistency because they own the clock. I might not have that in my data center. So I think it's one of these you probably won't see their service become an on-prem sort of thing. But Google can do it, which means if you like Google Cloud, you have a really awesome database service that I think gives you the best of all worlds. Yeah. And much like the S3 thing, it's probably better than what you're already using, even though it has the weird pee. And then meanwhile, I think since last we recorded, there's like a whole series of things. Like there was, they call this the cloud bleed. Like is it, people pretty much call every security problem a bleed nowadays. Wasn't there a heart bleed or something? There was. It gets a logo, which is awesome. Like I guess if you have a bad enough vulnerability, you actually get a logo, which is pretty wild. Yeah. Yeah. So what's the name of the flayed man people on House of Thrones? Those are like the security brand people. Is it Frey? No, Frey is the bridge people. Anyways. Yeah, you're right. Yeah. Now I'm blanking on who slices people up. So yeah, cloud bleed, you know, cloud flare proxy problem. Things were apparently leaked for a while. I haven't seen necessarily what this might have, you know, how people got impacted yet. I think it's more of a risk thing. But I think another one of these usual things of, hey, we pass a lot of things through a lot of proxies nowadays through caches and DNS and data services and clouds that, you know, you're going to be at risk for these things. Even among providers, it should be even amazingly awesome. Yeah. You know, I mean, I mean, in my mind, if you like summarize these three things to bring in another, I don't know if it's lateral thinking or something, but like it's sort of like there's still a lot of things IT people need to do. Right. And it's a little it's like I'm always like a little cautious of it being self-serving, being at a vendor. But it's like a lot of what we talk about nowadays is not so much like eliminating things and bringing in robots, but it's just like improving and making better the way things operate and introducing productivity by changing the way things are operating.

But there's still like time and time again, if only in security. But I think also like like if you were switching all your stuff over to using Spanner, whatever that may be, you'd want to like figure out what's going on there. And then if you were if you were designing a storage thing, you would want to have some some partition tolerance in your global storage mechanisms. And all of that stuff still requires people to figure stuff out. It's not like the robots just gleefully take over and you're dandy. No, I wrote a blog post this last week on the Pivotal blog about enterprise architecture and cloud and how those two may look in the future together. And some of the points still is cloud doesn't negate the need for enterprise architects any more than it does these other roles. But the roles change and the needs change, but I still need people to make good decisions on connecting stuff and deciding what to use stuff and securing things. So it's it's adopting to the new world, as I think we're going to talk about today and the rest of the chat as well. What I like about that piece is like you gave actual recommendations of stuff enterprise architects should do. Right. Which I think I think I think a lot, you know, when we when we go in and talk with people at large organizations, I don't know, just to give a useless but like rhetorically useful. Like number, I think about a third of the conversations are basically like, so what you're telling me is I'm going to get fired. Right. And and like I think I think enterprise architects, QA to a certain extent, operations people definitely like like PMO people and and sometimes programmers, but very rarely are sort of all they kind of have that fear. And I think I think what we as an overall cloud native with a hyphen between the two words community like are just starting to do, but that we need to do a lot more of in the next few years is basically explain like what things all the different roles need to do. Right. Not not sort of like the the desired in state that an organization should look like and not even necessarily how you transition, but like what are people doing every day?

Like what does an enterprise architect do and how do they fit in doing things? And I think that, you know, if I'm remembering your post correctly, I mean, to unhelpfully summarize it, it's basically like, well, what an enterprise architect should do is help the application teams do better software. Right. Right. Like, yeah, it's actually representing the customer a bit. Yeah. Yeah. Just what your standards look like. Instead of focusing on a governance role. Right. It's more important to get involved in the process of discovering new and better ways of doing things that are properly risk managed from a technical standpoint to the to the organization. Right. So you don't want to enforce standards. You want to come up with new ways and new practices and technologies to operate and do, which I think is a very like not in the social political phrase meaning, but it's a very privileged position in an organization that that you get to be the applied shiny object seeker rather than the the UML toiler and enforcer and there are even coder who's doing something. Yeah. I mean, I wrote UML standards when I was an enterprise architect. So I know that I know that world well. But I think this as you mentioned, it's we have to be better about showing what these things look like, not just saying here's this magical new world adjust like, all right. How do I adjust architecture, data processes, operations? So we can, I think, as an industry do much better at that. But I think as you talk to customers, as we all hear from what real businesses are doing, this isn't, you know, fantasy stuff. These are how companies are trying to become more software driven by having more roles that are invested in business outcomes. You know, what architects aren't just running architecture review boards to tell you if you've, you know, checked all the boxes, it's, hey, we made life better for our customers. Will we increase revenue because we built better software? Hopefully, yes. Then your job still matters. Yeah. As one of my, there's many favorite lines of mine in this talk that Mark at HCSC gave at SpringOne Platform. But, but as, as he said in one line, and is repeated in that, that CIO panel that John Grosshandler did with him and Allstate people.

But he said, if we release 35 times a week, how am I going to get an architectural review? Right, right. It's just like the, the nature of the cabs and everything's has to change if, if that's the in-state that you want to get to. So before we get to the topic, you, you also, uh, added in, uh, that, that we released, uh, Spring Cloud Services, which I, which I, tell me if I'm wrong. I, and this is in, in Pivotal Web Services, uh, I think, but it's basically a way of controlling your, uh, I don't know how to describe it. Your intra-application communications, how inside your, your application, all the services and nodes and everything can talk with each other. Did I get that right? You're, you're, uh, you're like 48%, right? That's good. That's good. Solid. You know, it's a good batting average. So, uh, yeah, so I've had that some pivotal news here cause I thought this was interesting is so Spring Cloud Services is a, and this is available on Pivotal Web Services or Pivotal Cloud Foundry, but it's us packaging up some of the core Spring Cloud projects like Eureka, which is a registry, you know, the phone book for the cloud services, microservices. It packages up the config server, which lets you store configs outside of your app. Also does hysterics for circuit breaker stuff. And so we bundle it all up and then when you deploy it, we build the quick infrastructure. It's not hard to do in Spring itself, but we simplify it to actually deploy a config server or deploy an Eureka server. And so where that comes in handy is we also added container to container networking to Pivotal Cloud Foundry and Pivotal Web Services. What this means is that a container of an app, so let's say I have app A with six instances, six containers, right now if that app A needs to talk to app B, I go back to a routing tier and I kind of lose who I am so I can't do, you know, just anybody can call app B. What container to container to container networking does is say I can have a policy that says app A is allowed to talk to app B over this port and this sort of protocol. That's cool. And so how would I know the freaking IPs of an individual container? That's kind of obscure.

So when I have something like Spring Cloud Services and Eureka, when I register my service B, I'm telling it where my services are, what my container IPs are. So service A can say, Eureka, I want to call this container directly. What is its IP address? Eureka can tell you that. So it's a nice way to actually have a phone book for all your services that's live updated and cached and things like that. And then it's all bundled up with Spring Cloud Services. And then container to container networking gives you a really secure, nice way to bypass any extra proxies and do kind of secure policy-based communication between different containers. Makes sense. Yep. Yeah, fun stuff. That's the other 52% that was needed. To fill it in there. That's good. That's right. I'll take 48. That sounds accurate. You know, I don't want a controlling interest in being right. That sounds exhausting. Well, so I brought the topic that we'll have. We don't have a guest this week. So as I was alluding to earlier with the post that you wrote and what I liked about it, giving actual advice, I've been asked a few times recently to come and give a talk that's basically kind of follows the following premises of like, yes, we've read We've read the internet and we know all the stuff that we're supposed to do DevOps and cloud native and we're supposed to have a unified team and deploy once a week and all of this stuff. But we're currently in a large organization that collectively basically doesn't want to do any of that and thinks that it's impossible. And clearly the adamation that I wrote last year, the stop hitting yourself anti-pattern, where I just said, don't do that. Obviously, that didn't work. We kept hitting ourselves. That's right. So, so I, and, and, and I, I think, I think this is a fair ask. And the ask is basically like, so can you tell us how other companies like actual things and tactics that they've done, how they have gotten better and like moved closer towards doing, you know, DevOps is the thing people always want to say. So, so they say DevOps, despite all the, all the time we've spent on cloud native with a hyphen sort of like hyping, they just say DevOps, which is fine, whatever.

And they're like, so how do people actually switch over to doing DevOps when, you know, the company, the organization doesn't want to like, what, what do you do? And, you know, for, because it's, because it's, is it still popular? I assume so to put a new phrase in front of DevOps. I've been calling this my impossible DevOps project, you know, when you're operating in a caustic corporate culture and I'll, I'll, I'll clean it up and share it publicly. Like I did my other booklet, but I started writing down these things in, in a little booklet and I'll share it in the show notes here. I think I'll set it up so that if you go to cote.io slash cloud three, that, that you can look it up. But I don't know, I thought, I thought one, I would say like, you know, I'm, I'm working on this project and I, I'm curious to hear from people who have struggled through like doing the right thing in their organizations. And, and more importantly, it's not only good to know like what tactic you actually did to make it possible, but it's also good to know, I don't know what you would call it, a heuristic for giving up. Like, like I think, I think in a lot of the O'Reilly circuit rainbow and sandals, like even the, the DevOps enterprise summit, there's, there's this maniacal focus on like, just keep working on it and doing it right. But sometimes you should just give up and like move on to other things, right? They, if to use a fanciful term, you should prioritize. So it's good to identify when something, some effort that you're working on or change isn't going to work. And it's better to just desert that and move on to something else that you can, you can change. So one, it'd be great to hear in any, any anecdotes people have on that. Yeah. So this is a good topic. You know, I think that's what a lot of us like going to conferences for is to hear stories of other people and how they've kind of gone down these journeys. They're just talking about how their technology will make you awesome when you're frustrated, you can't get the last technology and shop. And so you're just showing the future I can't have. So there is a lot of, of great stuff there on, we do see some of our customers talk about to your point too. There's sometimes giving up and there's also just accepting that you'll never move beyond incubation.

So we'll always be a departmental thing that goes fast. Other departments have too much just resistance to change. And either you have to go work somewhere else or accept that you're going to, your work stays local. That's probably a tough realization, but I think we do see that. The Lord's prayer of cloud native, if I remember my Bible stuff correctly, or the fried egg theory. Yeah. Like as, as an example of it, like, like there's one, I think this is the first one that I started writing down is, is I've come up with a trite way to put it. Is, and is that, uh, basically like you can't change organizations. You can only start new ones, which, which is a mixture of like, you know, my delightful cynicism and actual stuff that, that, uh, people we work with have done. And then a bit of like what we were just talking about, like accepting things that can't work. And so one of the things that I've noticed, uh, organizations do that they talk about or ones who don't talk about it is they, uh, to use this word, literally, they literally set up a separate organization, uh, that's doing things in the new devopsy cloud native agile way. So they've got their, their hundreds or thousands of people operating in their traditional way, their waterfall way or wagil fall or whatever. And the first notion, uh, and also what you would assume is that like, we should fix that organization or we should change or transform that organization into doing things in a new way. But I think, and this probably isn't universally true, just like anything I say, but like, it seems like the more successful path is you set up a brand new organization, you know, still under the same, like corporate paperwork and everything. Uh, but we, we, we would call it a labs model and you, you, you slowly move people over to this new way of doing things and responsibility over there. And, uh, you end up like just having your old organization, either it dies out or it stays there, but you have a new organization. That's often physically located somewhere in examples, like with all state, it has a different name and logos to it.

And it has different operating procedures and probably different reporting staff. Like everything is just different. So it's a, it's a tricky way to basically start a fresh, if you will. And I think, I think, you know, also, you know, as more example of the kind of stuff I'm writing up. Um, so like I said, all state is the chief people who have talked about this, but there's, you know, there's certain other large insurance companies and, and most other customers, they end up setting up this. Like if, if you see some of the, uh, the talks from Humana, they talk about doing things, setting up a labs as well. But the other thing I think, and this is from, uh, uh, Doug's talk at, um, I guess it was SpringOne platform. He, he threw out this number very quickly that was basically like, at least in his experience, about 30% of your staff won't make it over to the new way of doing things. And so it is like, you know, and, and, uh, I've, uh, I, I think I did a little write up, uh, in my own blog about, about the idea of the frozen middle. After we, we podcasted about a while ago, like every now and then, uh, and it seems to be in bars or in parking garages after talks. Like I'll talk with other executives and their numbers are usually higher, you know, around 50, 60% or something. And, and it is kind of like depressing, but, but that, that seems to be what a lot of organizations do is they just like, if people don't change, then change the people to put another trite thing around it. And, and you can't always do that. So you often need to just set up a, uh, a different organization that isn't, uh, doesn't get infected by the old way of doing things. Yeah. And then sometimes, yeah, you know, accidentally or on purpose have to starve that old organization. I've had colleagues tell me about those as consultants that, you know, there's a new way of working. They were brought in to do it. The other group didn't want to change their application. So the new team just kind of built some services around it. They didn't have to interface with the old one and slowly and slowly that old services got marginalized and people, you know, you can't always have the antibodies infecting everybody else. And I'm mixing medical metaphors, but in my old job, we used to say, you'd have the people who didn't like the change.

If you'd be injecting these changes, they would all swarm it and try to kill it. And unless you either have amazing proof points or fantastic executive sponsorship, they're going to win. Because, you know, I don't know from your experience, I haven't seen yet what the average runway is for some of those incubated teams. Like, it's great to go spin off this other group. How much time do they get? You know, if there's no big results for a couple of years while they set things up, is that going to work? Or do you have to show results really quickly? What is that? Unless you have some really good air cover from an executive who believes in it. Because I've seen the opposite where, again, you think you have a long runway and it starts to shrink and shrink as your sponsors depart. And all of a sudden now you're left just hanging there and eventually things disappear. So what kind of sponsorship do you need? How do you build it broad and deep, not just one person that goes, I love cloud native. And then they go take another job and you're hosed. So how do you build up a bunch of people who believe in what you're doing? And most importantly, maybe how do you show proof points right away so that no one wants to shut this thing down? I don't know what you've seen there. Yeah, no, I mean, that's one of my standard talking points is the pace of change that you do. And I think, I mean, I'm repeating a little bit of what you're saying. But I think the main things that I found and I like to talk with people about is, one, like, it's probably going to take a year before you're ready to, like, say if something worked or didn't work. And sometimes it's a lot faster and sometimes it's slower. But, you know, the sort of case that I use is long ago by episodes, we had an interview with Tony at Home Depot. And so he's, because he's given several talks on this topic, he's got a great chart about the number of applications that they did over the first year. And he's got a good narration about some hiccups that they found. And I think what he said in that, when we were in the hallway there with him is that they, at the end of the year, they had like 130 or so applications. And, you know, some of them were easy, simple applications, and some of them were like the Paint Desk one and the Pro Tools one we talked with them about.

So there's a good mix of them. But, like, you know, you shouldn't expect to convert everything over in a year. So, one, you should have realistic notions about pacing yourself. And I think, especially if you look at the pace of change that you see on slide two or three of any presentation from a vendor, it can feel like you need to operate really quickly and do things fast. Which, sure, but, like, as a large organization, you have a lot more time. And also, to your point, I think it's a lot more important in an existing large organization that's looking to do a new thing to make sure they don't screw it up too bad. And you don't want to do everything at once and have big catastrophic failure. You want to prove out to the organization and yourself that it works. And so you want to be a little more slow and steady than doing everything at once or, you know, doing giant things at once. Yeah, I think it's underrated, too. I think you and I have both seen this. The underrated value of evangelism internally. Like, you can't take for granted that, hey, we're this funded group. Let's go do our thing. You have to hit the road internally. Like, to other teams and demonstrate, why did this make a difference? What did we have to change? Field some questions. You know, make this less othering of this group that might feel like they're just wearing weird T-shirts and doing their own DevOps Cloud push thing. Instead, you've got to sell this. And then, sure, going to conferences. Again, I love seeing people evangelize their own things. It helps them hire some of that new blood when some of their existing staff isn't going to adapt. And at the same time, it helps their colleagues see what they're up to. And so, I think the evangelism sometimes gets undersold as what you need to do because you just can't assume you have this charter. Now, just go do it. Look at 18F, even in the government. You know, making sure they're showing what it means to do digital and improve what you're doing in government processing. They talk about it. They blog. They share information. They share open source tools. You know, you're trying to change an entrenched org. I think we have to make sure you invest some actual time in that. Yeah, yeah. I was just typing that in my document as a thing to write up because I think there are some pretty specific tactics that are useful.

And more broadly, I think your point is useful as well is you're going to need to do some marketing. And I think there's maybe like white hat marketing and gray hat marketing. And the white hat marketing is just like we did this thing and like, you know, we got a newsletter and we're going to conferences and we're talking about it. And I think I'm always like slightly baffled is the wrong word because I understand why it happens. But I'm always slightly saddened. Let's let's use some Bridget terminology. Makes me sad that that like like I talk with companies and like, oh, yeah, but we don't we we don't do that. Like they don't let us go talk with people with people. And I try to go over the benefits like it's good for like, you know, hiring. It's it's it's altruistic to share with people. It's also good for retention. Right. Like there's probably a lot of high performing people who are like, I'm not being famous enough. So I'm going to go somewhere where I can. And like it's just all around good. And the risks are extremely minimal. Right. I mean, I guess, you know, if you're like some sort of like make multiple changes a day, like flash trader on Wall Street. You don't want to go like publish your algorithms or whatever, how you've how you've torqued your networking and flash stuff so that you can get like some millisecond, like whatever it is, all that mess is. But it's the mistaken belief that my technology stack is somehow proprietary to how my company succeeds, which is so dumb. Like if every customer in the world used Pivotal Cloud Foundry, guess what? There will still be competitive advantages. Exactly. It's all about how to use this stuff. I was at a conference recently just staffing the table and one of the competitors of one of our customers came up and I was talking with him. They're like, yeah, we love all the talking that we do because we can like understand how they fix how they're they're improving themselves and doing things. And like there was a bit of a wry smile with it. And, you know, in my mind, I was thinking like, well, you know, you should do that. Right. Like, like just because someone's going over how they're operating and doing best practices, like it's not really going to make your it's not going to make your competitors' lives easier because it's kind of the whole premise of this whole paper I'm working on.

It's sort of like we all know the right things we're supposed to be doing. What's difficult is in your organization how to apply it and how to how to get it to work. Well, all that matters is execution. You know, it's why the ideas people don't usually win out at the end. It's you got to take that idea and do something with it. So share your ideas. And as you say, make people want to work there or feel like their work is valued because they can share it and they can learn something from these conferences when someone says, yeah, we use that technology, too. What did you think about this? It's not just broadcasting. It's actually listening when you do these sort of marketing, quote unquote, exercises as an architect or an evangelist within your individual company and team. Yeah, I was reading the notes from, I think, our first ever Pivotal Cloud Foundry community talk up or something. And it was, yeah, I wish I wish I had, well, I don't necessarily like traveling so much, but I wish I had at least heard all that conversation because it was a lot of the bullet points are good. But it was it was awesome to hear to sort of read what what various people were talking about and how they'd been sorting things out. And so anyways, it's that spirit of like sharing the knowledge that you have that I think is that I think is helpful. And so before we wrap up, I'll I'll I'll give one more example of like the type. It's actually two, but they're kind of like, I don't know, they intersect with each other. Well, but it's a good example of like kind of like what I'm trying to find with this. And again, I mean, as always, I'm just trying to troll for content from I don't know. I hopefully that's I don't mean troll in the bad way. I mean, like fish. Naturally, you're trying to like you're driving around with that tiny little motor on the front of your boat, just sort of like eating your kippered snacks with a little funny hat on. And you got a line in the river or the pond or lake, whatever body of water you may be in. So, sure, you know, there's this pairing thing that Pivotal is always talking about. And there's pair programming, first of all. And then you got you got on on on the balance teams or the the all in unsiloed teams that we espouse. Right. So we espouse we say we call a DevOps team a balance team.

I don't I don't know what it's a nice term. And, you know, you've got developers and design people and a product owner and various other stuff. And the idea is that you have all the roles you need to to own the software end to end and deliver on a weekly cycle. And so that's a good example of the bromides that we throw out there. It's like everyone's like, yes, sounds great. Yes, I should eat more fruits and vegetables. Thank you. But one of the practices that people actually do and and more importantly, that they they don't do and they kind of fail at this is is is pairing. And it's not only pair programming, but you pair up with the product owner and the designer and pretty much everyone works in pairs. And this has some sort of obvious benefits. One of them is that there's a better diffusion of knowledge and therefore risk management of things happening, which people will. A lot of these phrases having come up with sort of like cynical people like myself, we have we they're all death based. So there's a bus number. Right. Like if how many how many of your your team could be hit and killed by a bus or severely incapacitated without halting your project. So if you're diffusing this knowledge, you have a lower I guess it would be a higher. I don't know. Your bus number is better. Higher tolerance. Have to go look higher partition tolerance. So there's that. But there's another there's there's a productivity. There's a few productivity enhancements. One of them is the obvious just upping the quality of code because it's constantly under review and you can do testing much faster. So there's that. But there's another one that I think is a little more subtle that gets to a larger one. That's important, which is the time that people spend syncing up and communicating with each other gets cut because they're constantly working with each other, especially if you're rotating people. And so you don't need to have that like three hour design review meeting where you're communicating what's happening and explaining it to people. And then because you have such a high throughput of people coming up and designing things together, it even starts to bump up against what we were talking about earlier, where you don't need to have enterprise architects coming up with things as much because you have two heads.

And then if you rotate them, multiple heads, like working on the overall architecture and design of the system. But it really is. I think that the key interesting benefit under the waterline is this this idea that you spend less time communicating and syncing up, which we know from the Church of Bezos. Anytime you have to communicate and have a meeting, you're on the path to failure. Yeah, it might be. I might have the same amount of communication. I think it's just you shrinking the cost of it because it's all fine. I'm still having all the same chats, but I'm going from like I'm interested in this to I'm seeing it in the app in an hour later because someone just took it and CI'd it. And that's right. So it's exciting. And so I guess I think about it as less communication because if you're already up to speed because you've been working on something, you spend a lot of time communicating about it. But you don't have to spend all that time bringing someone in a context and let's bring him to it. It's like it's like the Donnie pattern from the Big Lebowski, right? Like you don't want to be like a child that's walked into a conversation. You want to like be involved in it and have all the context. And that in fact roles like you point out starting to point out in your document that business analysts and others, it's a different role. And then you think about how that spreads to the rest of the organization is as you bring these people on the team and pair them up and have them work on the team, like you break down that communication overhead. And then if you don't need to have all these meetings, that starts to get towards the kind of ideal that Allstate has spoken about and also others. I think HCSC has gone over this is like people find that their developers or their product team people were actually not spending the majority of their time actually working. This goes back to my dad advice to my seven-year-old. They're just like doing all this other stuff, you know, syncing up and meeting and like whatever and transitioning. And so also once you pair people up and they have a more high throughput with syncing and communicating, they actually spend a lot of time working. And then you just get a productivity boost from that. And this also goes back to my principle that like you should never assume you're actually doing anything if you're worried about adopting a new process.

Like my suspicion based on my own experience is that like no matter what you're doing, if you try some new methodology and you actually do it, you're probably going to get a productivity boost because you're doing something. Whereas previously you were just kind of messing around and then, and then finally, so the other thing this connects to is, um, I was just talking with, uh, with a, with a large bank last week and the topic of offshoring came up. And so offshoring is one of those things where like, uh, I don't know, you're probably stuck with it. And it's also, it's also really easy to be a jerk about offshoring. Right. Um, and because I think the premise of offshoring, there's some premise of like follow the sun, but I have a pretty strong suspicion that the primary premise of offshoring is cheapness, right? Is basically like, it's cheaper to hire people, uh, in Eastern Europe or Asia or whatever. And they have the same skillset and they have the same ability than it is to hire people in, in, uh, onshore or whatever. And, and, and there's all sorts of issues that result from that. Uh, you know, we might only have like an hour or two of, of time to talk with them. And so it's hard to, to synchronize both ways, both from your offshore location. I mean, I guess offshore is a state of mind depending on where you live, but it's, it's hard for the two shores to talk with each other and synchronize. And so this issue comes up a lot when we talk about co-location and people working and based on, I was in a, uh, I, I went to, uh, some cloud native roadshows we had in Omaha and Kansas city. And we had the Garmin people who are a customer of ours come talk at it. And they had this issue and they were saying that there were a few things that they walked through. Um, and first I'll insert my, my first unhelpful one, which is, well, don't do that. Barring that as your solution, they, they were saying that what they had found is that, um, one, it's good to co-locate for some regular amount of time, maybe initially for like a month. And, you know, and, and they were saying that it's, it's kind of good, whoever the primary person is for them to go over there to base.

And then I'm reading, I'm interpolating a lot in here and I kind of had this conversation with them that like, uh, it's good to build up trust between the both organizations. And one of the ways to show trust is to like, you do you, the primary people do the drudgery work of traveling somewhere and, uh, giving up being there. So you want to build up that initial trust and, and, uh, um, also start pairing with them and try to do it remotely. But I think, I think the, and you do this pairing to sort of like build up trust with them both ways and to kind of transfer skills. And so you might do that for several months and go back and forth and do some pairing to kind of have this diffusion of knowledge. And then I think another pattern that, that you notice is if you're following a, a, uh, sort of like a, a highly microserviced approach for doing things. Well, if you remember the point of microservices, it's like not only all the benefits of SOA, but you're supposed to take advantage of organizational, um, problems. One of those being that when you have a silo or a group of people, they locally optimize whatever they're working on and don't pay attention to how it integrates and works with other things. So you should take advantage of that in an offshoring situation and basically fully partition off the code that you're working on to just that offshore location. And if they're following all the microservices practices, it's not like it's going to a hundred percent solve the problem, but it's at least going to get to them the point where they can be an independent team operating on whatever service they have. And you can minimize the, the sinking that you need to do with them. So, sure. So like a balanced team. Yeah, yeah, exactly. Or else. Right, right. And, and the analogy I was using with, with the, uh, with the bank last week was like, you know, it's not like Apple, Google, and Facebook and Twitter sit down and co-locate with each other to make sure that the whole internet works. Right. Like they just, they just sort of like set up a bunch of services and they talk with each other here and there. And it just kind of sort of all works together. And so like, similarly, like you need to set up your teams like that. If, if you're going to be truly like set up as, as, uh, as separate teams across the globe.

Yeah. Which we do at pivotal, right? We have 50, 60 teams all across different countries to build individual parts of cloud foundry and pivotal cloud foundry and spring and all those things. And somehow they all kind of stay in sync with some light touch, but they're also independent units. I think that maybe is a nice path forward for some of those teams. If you can give them some of the ownership of the whole life cycle and not just be order takers from architects somewhere else. Yeah, that's right. Well, so that's, uh, that's my little preview of what I'm doing there. We'll see, we'll see, uh, we'll see how quickly I can get little bits and pieces of it out, but I'll, uh, I'll, I'll make sure there's nothing embarrassing in it and, and, uh, and make it public for listeners here that they can go look at. And if they have any suggestions, that'd be great. Well, as always, this has been Pivotal Conversations. You can find the most recent up-to-date things, or what does that mean? Most recent up-to-date things you can, you can find, you can get episodes the quickest. If you go to, uh, if you subscribe to the feed, you can find us in iTunes and probably other places, definitely in overcast. Just search for Pivotal Conversations and subscribe to it. And then we also host everything over at SoundCloud. You can go to soundcloud.com slash Pivotal Conversations, all one word. You can go check out all the past episodes and things we've been doing there. I think I've finally released all of our, uh, our red monk episodes. So you can go see a little playlist of all of that. And then also, uh, as always, uh, we, uh, about every week we post the full show notes for an episode over at, uh, pivotal.io slash podcast. And you can find the, uh, things we've referenced and little summaries over there. And, uh, it's great if you want to go leave us a review or rating or just recommend us to people. Uh, that's, that's always handy so that, so that we're not just talking to ourself as delightful and therapeutic as that may be each week. And with that, we'll see everyone next time. Bye-bye.

Tanzu Talk · 🍕 Get more pizza.