Putting the Ops back into DevOps, SpringOne Tour Istanbul with Nate Schutta & Paul Czarkowski

Episode 127 · 40:15 · SRT · archive.org · all Tanzu Talk

0:00

Hello there. This week I've got a discussion that we had as part of what we kind of confusingly call our pivotal conversation part of the SpringOne Tour events, where in the afternoon we have kind of open sessions with people instead of the prepared talks. Anyhow, this one is with Paul and Nate and myself, and we talk about some of the, I don't know, theoretic meandering thinking I've been having recently about how DevOps is kind of turning into OpsOps, if you will, as platform engineering and SRE comes into play. There's also some discussion of how enterprise architecture is done or should be done in kind of like this decoupled microservice-y event-driven world. And there's a few other topics on the areas of culture change and remote work and things like that. We recorded this in a big open room, so the audio is not as ideal as it should be, but hopefully you enjoy the slight tinkling of silverware and things being picked up at the end. With that, enjoy the episode. Well, Paul, how do you notice people doing what DevOps is? No, I mean, definitely what I just heard described are very common patterns. Everyone kind has interpreted DevOps to mean their own things. Everyone does do it slightly differently. So you get everything from like a DevOps team that for all intents and purposes is just like a traditional ops team that maybe writes a bit of Ansible to where you have, it's a bit more embedded where you might have like a DevOps group, but then you also have like an ops person in each dev team or you have everyone someone who's kind of an ops person. And then you sometimes have it where like, and then sometimes you have like every single team is there a completely separate entity and goes off and does their own thing. Which I think that used to be pretty common in like startups. And then they hit a size point where they realize that there's just like this massive amount of sprawl and then they tend to move back to

2:01

some sort of centralized ops like mentality, which I think the really centralized ops is actually where you want to be in a way. Um, but you want to be, have that centralized ops team, like building a platform and treating that platform as a product with contracts and APIs that the dev teams work with. So in a way it's sort of, you're taking that like cloud native, like microservices way of writing software and deploying software. And you're saying, okay, let's align the ops team around that same, like I had to use the word, but around the same paradigm. Um, and you know, that way you're kind of, uh, hacking Conway's law to do like what you want. Right. You, you, you, you know, Conway's law, right? Give us the Conway's law. The architecture of your application matches the communication patterns of your organization. Right. Which generally is better said as like your architecture will look like the structure of your organization. Right. That's technically it'll mimic your orchard. Right. Yeah. Yeah. Yeah. And so if you, if you have, if you know that and you keep that in mind, you structure, you restructure your organization to match how you want your software to look. So if you want to go into a, a super deep microservice architecture, you need to build your organization to be like that all the way down to like your operations folks. Yeah. It's like the reverse Conway maneuver, I think. Right. Yeah. But you know, to your point, it sounds like something you don't want to look up on open dictionary. Probably shouldn't just in fairness, you know, you and I've talked about this. I think what you're really arguing for, and certainly the way I look at it from an architecture perspective is we want to pave these well-worn paths so that we've done this before. We know how to do it. We know how to support it and we can get you to production in a reliable, repeatable fashion. You know, the problem you're running into and everybody's doing their own thing is all of a sudden there are a thousand ways to do the same thing. There's no consistency. There's no reliability. That's a really bad place to be with software. You know, we've known this forever. You know, that whole notion of, well, it worked in dev, but not here. It worked in test, but not here.

4:02

That's misery. We don't want to be there. Our customers don't want us to be there. And the only way we get there is by having more consistency. And now where that gets challenging is sometimes people are like, but I don't want the restrictions that are applied to me when I have that. But you have to look at those things as guardrails and they're there to protect you, not to hinder you. Yeah. And, and more, more, you, you need to look at it not as restrictions, but as, um, like part of the contract that gets you something. It like, because you get like, if you follow my, my guardrails, you're going to get monitoring, you're going to get logging, you're going to get observability, you're going to get dashboards, you're going to get a common way to like operate your software. So you're not really being restricted. You're actually being given a lot more freedoms because you're just focused solely on writing your application and letting like the, the, the people running the platform manage all this extra stuff. And I think that's kind of important is, um, I, there's, there's a quote somewhere. I don't know the exact quote, but basically it was like all, all the good DevOps work happens before the software is deployed, right? It's not like DevOps isn't devs carrying a pager. DevOps is like all the stuff that happens from CICD through to running and production. That includes like hooking it up to monitoring, making sure the right alerting is in place, getting the right dashboards in place. That's, that's where all the DevOps work is. I, and yes, like there's, you know, infracode, config management, writing kube manifests and stuff is part of that, but that's still all happening before the code is being deployed. And hopefully that pager doesn't have to go off very often. And hopefully that pager doesn't have to go off very often. Now, the interesting thing though, is I completely agree with you that the platform should free you up to focus on what you really care about. And frankly, what your customers are paying for, you know, you've never had a customer come to you and say, you know, Hey, great job upgrading your servers from version X to X plus one. That's dial talent for that. Right. And they want features, they want bugs fixed. And so if we're wasting time reinventing

6:02

reinventing the wheel, reinventing all these things that should just be there, that's time we, by definition, can't spend fixing things, adding new features. Now, what I find fascinating though, is there's lots of people that still reinvent that wheel. And I think the rationale there goes back to a quote from a friend of mine, Neil Ford, who said, meta work is more interesting than work. And this is why in so many projects, like the first thing people do is like, all right, we have to sit down and build a framework for building frameworks, because that's interesting work, you know, putting a web front end on a relational database. Well, that's all plaza, who cares? I want to be the framework builder. And it's like, well, yeah, I mean, that may be what you want to put on your resume. That is not what really any of our companies have ever asked us to do. So what, one, one thing, and I'd be interested in y'all's reaction to this is, is, uh, so it sounds like what, what y'all to some extent have just described is not, is, is basically let's take DevOps and take the dev out of it and just have like better ops in, in the sense of like, you do all this work beforehand. And like, you're going to build out a platform, but you're not really like, I feel like you don't even have all the like blameless stuff and all the cultural stuff of DevOps as, as we sort of understand it. It's almost like, well, we had all that application stuff and all that, like how to actually focus on, on, uh, all the usual agile things that we have combined together into DevOps. But sort of what we mean now is like, you should do things, you should do things well. But really what we mean by DevOps is focusing on like building out this platform and being a centralized team. That's not part of the application development team that really automates and does all of this stuff, which, which is sort of, uh, maybe like three or four years ago in the DevOps world that was like explicitly said, not what it is that, that you would have a separate DevOps team. So like, I mean, that's, that's like a, uh, I don't know if it's a contradiction, but it's a little like intellectual thing I always think about is it seems like nowadays DevOps is basically just like tops. Yeah. I mean, like you look at the, the purest view of it and

8:06

DevOps is like a set of practices and it's a type of like organizational culture. Well, that's all great, but you're not going to be able to do that if you don't have tools in place to actually work. Right. Right. You went to use that to follow along your paradigm word. You need an enabler. Yeah, exactly. Yeah. So, you know, it's, it's great to talk about the cultural aspects of it, but you don't get a lot of actionable takeaways. Right. Right. Like I can't like as a, as a practitioner, I can't action culture. Right. That's something that needs to happen like at the executive level, because they're the ones that set the, um, uh, incentives and all that sort of stuff. Right. Um, but I can certainly like, you know, figure out how to automate some shit. Right. You know, use some tools to automate stuff and I can kind of tackle things from the, the, from the ground, but it's definitely not like culture focused. It's more like, in a lot of ways it's almost selfish. It's like, I just, I just want to stop being so miserable. So I'm going to like have the robots do the work that I don't enjoy doing. Yeah. So, so in y'all's experience, like when it comes to the DevOps you were describing, is it more just like the tools and techniques you use? Or do you have a lot of like, uh, cultural things where you learn to work together better and get along and focus on, I think basically what, what's the calm stuff? It's, you basically culture automation, learning measurement sharing. Yeah. Yeah. So basically, I guess what I'm proposing is it's all just the A and the M there's, there's, there's a lot less culture learning sharing. Did I get that right? Culture's hard though. I mean that, that's the hardest part of this whole game. And, and so, so, I mean, anyway, like does, do y'all when you're doing DevOps, is it just more about the, uh, the automation and that the monitoring and not that you don't do those other things. And you can make an argument that that's where this stuff has to start because, you know, to Paul's point, we rarely have the ability, especially in an existing

10:09

company to say, I want to change culture. You know, we're usually not empowered to do that. So as we start to nudge the ball on the direction, the culture starts to evolve along with that. Now, it takes a really long time and that's where this gets frustrating. You know, I think one of the, I think I made this comment earlier, but a lot of what, what I've done when I work with clients really boils down to essentially technology marriage counselor. You know what I mean? These things are actually a technology issue. It usually is, Hey, these two groups aren't talking to one another. You guys need to hug it out and start working together to solve these problems, you know, but the challenge it's, that's, you know, and you had a great way of phrasing this this morning is, you know, in an organization that culture gets cemented in so early and the people that rise up through that have essentially learned how to gain that culture. And so they have very little incentive to change it because they're by definition successful inside that culture. You know, so the challenge then becomes, well, how do you start moving that needle? And, and I've seen this, I was in an organization that went from very traditional waterfall, four releases a year, big bang integration, you know, and it's taken more than a decade to get them to the point where agile is like the default. You know, when we first tried to build project rooms, it was like pulling teeth. Nobody's like, what's a project room? We don't know how to do that. You're not supposed to have that. And then before I left, like the last big thing that it was like renovate, like all the IT floors to be all project rooms, you know, but that took years. And I took in many cases to phrase it, I guess, politely, a certain level of management had to kind of age out or move on and realize, Hey, it's time for me to go do something different. Who made the decision? Who changed it? Well, I believe it's, it's, it's, it's usually a bit of both though. You, you know, you need to have that, that ground level buy-in, but you also need to have some champion at the top who's able to kind of navigate some of that. Yeah. And, and, and what's interesting is to see like you need to find that person who's willing to basically keep pushing, keep pushing, keep pushing. And that's what happened for us. I mean, we, we had

12:12

one, one manager in particular, who's just like, yeah, no, we want a project room. Yeah, we want a project room. Okay. We're going to book this room for a year and dare you to stop us, you know? And, and so he just wouldn't take no for an answer, you know? And, and that's, that's what you need in a lot of these organizations is to find somebody who's willing to essentially ram their head against the wall for a while until cracks start to appear. I'll, I'll, I'll go a little bit even stronger is like the, uh, pretty much all the organizations I've looked at, they have to have, as we would say, executive sponsorship to change beyond like two or three teams, right? Which I guess sounds obvious if you're going to change a lot of teams, it's important. But I think, I think the reason to point that out is, uh, you need to find one of those people, right? Like I'm sure maybe y'all's work isn't like this, but we definitely talk with a lot of people who, uh, they want to change. And then you kind of go up two or three levels in the management chain and it's usually a vice president or a senior vice president. And what you hear is that like, they don't want to change. And so this, thanks for coming in and showing us how like, uh, events work and, uh, we'll see you later. Uh, but it is very much so, I mean, most of the, the organizations I talk with, they really have, you could call it a champion or whatever, but at a high level, there are executives that sponsor that. And there's a few tactics like in that book I was referencing that I kind of document where, uh, if you have one of those executives, they can start to build up a track record and slowly build up. Like I've done two projects. Well, now I've done five, now I've done 10, but again, you need that managerial executive thing to plan all that out and book a conference room for a year. So it's almost like the strangler patent for microservices, but for your culture. Right. That's right. Right. Yeah. Yeah. Yeah. And, and as, as another thing, you know, another thing I noticed a lot of organizations do is they set up a brand new organization. Yes. So there's, there's the old organization that does everything the old way and slowly. And again, this is why you need to have high level buy-in is they move one team over to doing things in a

14:14

new way, working on a new stack. And then they move two teams. And you can kind of see how the, the things start to equalize or not equalize, but they start to shift around. And that's oftentimes people call that a labs or whatever, but that happens very frequently. Uh, and it's one way of, um, not having to change the old organization, which you could spend a lot of time doing that. It's like, I guess if you had a, uh, a refrigerator and it keeps breaking down at some point, it's better just to buy a new refrigerator, right? But at some point you do need to change the rest of the organization. Otherwise you're like innovation lab or whatever you call it becomes like its own silo that everyone else hates. Cause you get to play with the cool shit and you don't share your toys. Right. Yeah. Yeah. Because you're not allowed to share your toys. Right. And so you have all these other problems. So, you know, you have to like, at some point you still need the organ, like the, the larger executive group to decide that we need to change, which usually requires some sort of external force. Yeah. Well, and, and I think the other challenge there and something you and I were talking about earlier today is that takes time, you know, and in a lot of organizations, you know, I mean, I've seen various statistics about like how long a vice president, a senior vice president usually lasts in an organization and it's generally not measured in decades, you know, it's measured in like months, you know? And so you need to find somebody who has enough political capital, knows how to spend it wisely and is empowered to stay in that position long enough to see the change happen. Because there's a Machiavelli quote about that, that, you know, when you try to make change, there's people are often just going to sit on the sidelines because you've got a group of people who are going to actively resist you because they're doing well in the old system. So they have zero incentive to change. But then most other people are like, well, I'm just going to hang back and see which way things fall because I don't want to be on the losing side. You know, it's sort of like game of thrones, you know, you, you either win or you die. Right. And so that's part of it, I think in a lot of organizations. So how do you find that where it's like, I have enough time to let the organization evolve and, and move along with it. And again,

16:14

in a lot of situations, at least from my perspective, it does take a change in that management. You know, the, the old guard is typically not going to be welcoming open arms for a brand new way of doing things. As companies look around and go, wow, we're having a really hard time attracting talent or people are coming in and leaving six months later. You know, you start to realize, hey, we might need to start changing things or we're not going to have, we have to, we don't have a choice, you know? And in fairness, sometimes that's what it takes is sort of that, that burning ship moment to say, all right, well, we, we don't have a choice. We can't go backwards. We have to go forwards, you know, and that's usually the impetus in some cases to, to cause that change to happen. So on a, on a slightly related, but also related, uh, topic since, since, uh, following up on some architectural stuff. So that's on the other end of the scale. So a lot of the stuff we've seen today and people talk about is like, you know, we should have these, uh, independent microservices talking with each other. And, uh, we're going to have these events that hook together and, uh, everyone's going to be like coordinating and awesome. So we can get rid of all the enterprise architects because they don't need to like mandate things anymore. And we're probably going to have, I don't know what any of this means, but it's what I always hear. We're going to have like contracts that specify how these things coordinate together. Um, and I don't know, that sounds like that wouldn't work. So like how, how, if you're going to have all these independent autonomous people working together on their little part of the functionality, how are people making sure that all fits together at some point? Cause you know, it's cool sometimes that I have to like reload Twitter or Instagram, but I assume like the bills, the cash needs to move from one account to another. Maybe I don't want to reload my bank. Yeah. Yeah. And I mean, it's always, sometimes the answer is like, well, yeah, you don't apply all this nonsense to that. However, like there must be some kind of coordination that needs to go on between all these independent things. And usually an EA would do that, but they don't, or they do. So how does that sort of pan out? How do you do enterprise governance without doing enterprise governance? So one of the ways that you're starting to see that be applied is with fitness functions.

18:19

And so this is a concept. I mean, I think the easiest way for me to describe it is it's essentially a unit test for your architecture. And so we've done this traditionally where, you know, you write your tests for your code, but we've really never tested our architecture. You know, we, we come up with these principles. We think, okay, like performance is a big thing. So we want to make sure our app performs well. But usually what that means in most orgs is, well, we did a performance test before we went to production, you know, and maybe we found some problems and maybe we fixed it, but that's like a one-off kind of thing. And then we never talk about it again until, oh, hey, we're starting to see some performance issues in production. The concept of fitness functions, which, which comes out of this evolutionary architecture book basically says, well, you should identify those core features of your architecture and then endeavor to come up with a set of tests that make sure you're still following those principles. You know, because one of the things that became really clear to me after I got into the architecture world is I can't be involved in every decision. It's just not possible. There's, we're spread so thinly that teams have to be, and they have to be empowered to make decisions quickly as well. But how do we make sure we're all making good ones? How could you test that? Yeah. Well, there's ways, right? And so like, we might decide, uh, what's a good one? Cyclomatic complexity. We might decide we don't want our cyclomatic complexity to be above a single digit number. You can easily write a test for that. You might say, we don't want to have any circular dependencies. You can write a test for that. You know, so there's things like that that we can do. And there's actually a project called, I think it's arc unit that, that exposes some of that kind of stuff. There's a dot net variant as well. You know, and so I think that's the way I would argue. We try to shift governance more towards these, these automated things. And that might be like a spring cloud contract kind of a thing where it's like, Hey, this is my contract. You know, if we need to change it, then we need to communicate. And I think that's the important part of a lot of these things is they're not, they're not a replacement for the conversation. They're like a placeholder for it. You know? So if we come up with a spring cloud contract, it says, this is what you expect. This is what I'm going to give you. If something needs to change and that test breaks, then we need to talk about that and actually work it out. You know,

20:24

I've seen too many technologies that like try to replace the people part of this business. And I just don't think that's going to happen anytime soon. Because a lot of times the most effective thing is, Hey, how about you and I just like chat about this? What do you need it to do? What do you have? You know, that's way more effective than, well, I created a service catalog. And, you know, it's like, that just is nonsense to me, but so, so how did, how does like a contract do that? Like, like, like, let's say, let's say I, uh, I have like a login service and I've got a contract for it and there's like 10 different people who use the login service. And I want to say like, here's, there's five ways you can log in. Stop doing this one way. Like, well, so again, this is where, you know, as Paul brought up, as we get to these microservices, there's a lot more complexity that comes along with it. And so we need the alerting and the observability and the monitoring. And so one of the things that we inevitably have to do is, Hey, this microservice is deprecated. You shouldn't be using it anymore. And there's various ways we can communicate that could be some tests could be, Hey, everybody don't use this anymore. And we have to talk to one another about that. We send out an email, we put something on the wiki, but then we have monitoring in place that says, Hey, wait, you're still using that deprecated microservice. Why are you still using it? What do you need from us to get you transitioned over? You know, and do we have to put kind of a proxy in place for a little bit? I mean, what do we need to do to get you there? You know, and then work through that, you know, so, so that's where it does get more complicated. And that's why, you know, I did that thing last year on like, should this even be a microservice, right? Because too many teams are going in that direction because we'll shiny new thing, but there's a lot of situations where you don't actually need microservices. That's not actually buying you anything. You've made your life much harder than it needed to be. You're paying this, this tax for this initial complexity and you're not getting any benefit for it. You know, and again, not that that's new for microservices. We've done that throughout the history of technology, but it's important that if we go down that path, we understand the consequences

22:26

of it. You know, there's positive consequences. I can scale them independently, deploy them independently, use different technologies, but there's negative consequences as well that now I have to get these things to communicate together in a large enough organization. You're going to have hundreds, if not thousands of these things floating around. And, and the reasoning about your system becomes much more difficult as well, that you might be willing to pay that price if you're getting the benefit along with it. And if you're not, well, reconsider that architectural choice. As you explained in seminars. Exactly. Yeah. So, you know, it also seems like the first half, maybe all of it's obvious, but like, it seems like the enterprise architecture governance one needs to use, like basically matches the infrastructure that you have. Yeah. And then, and then, so, I mean, I, and I guess this was basically what a leaky abstraction is, right? That lower levels of your stack cause you to behave in a certain way as you go up. But then the more positive, not positive, the, what to do about that thing is that if your infrastructure changes, it's probably highly likely that the way you do governance should change as well. Right? So that's almost like you can sort of predict that the more we change infrastructure around the, the enterprise architects need to figure out something new, right? Instead of kind of getting stuck on doing things in the old way. Well, I think a lot of the enterprise architecture should be done like at the platform layer. So like the decisions you're making from an architectural perspective, a good percentage of it can be implemented at the, at the platform level. Like if you decide that, you know, your primary means of communication is via Kafka streams, or you'd make damn well sure that your platform enables easy use of Kafka streams. Right. And then it becomes that like no brainer for developers. Like I don't need to run Kafka. I just need to connect this stream to here and my messages go to wherever they need to go by some crazy magic that I don't need to care about. And it just works. I mean, I think that's a great point, Paul, that, that we should be striving to make the right thing to do the easy thing to do, you know?

24:31

And what's been fascinating to me is to see how over, you know, the last probably, I don't know, eight or 10 years as, as these sort of shadow ops groups have floated up in companies, you look at the way we've traditionally tried to do this with the heavyweight governance and the whatever change review board type things you've got. And then we're shocked, shocked when our customers go out and spin up an instance on AWS, because they can get it done like that. You know, and, and that's part of what, what we're finally starting to respond to. I would argue in it that our business partners are tired of waiting for us. We can't be that anchor anymore. And if that's what we continue to do, they'll just find someone who can do it more quickly because we don't need to do it the way we used to. I mean, you think about the way we built servers 20 years ago, it was someone like us literally putting together a server, literally sliding it into a rack. We can't do it at that pace. Now that just doesn't work. You know, things are moving too quickly. Innovation is too rapid. I mean, we're all getting compressed by all these new entrants into these markets. And so if we continue to respond with, well, that'll take nine months, why would we be surprised that our business partners are saying, okay, well, forget you guys. I'm just going to spin up my own little it group over here and get it done now, I'm just going to spin up my own little bit. But I'm just going to spin up my own little bit. Well, I was reading a thing recently, I forget the name of the person, but I think it was a he, he was a physicist who somehow got involved in like doing organizational stuff for software. And, uh, of course he came up with the formula, which, you know, I don't understand how formulas work. You multiply a bunch of things together and then divide it by something. And that gives you a number, carry the two. Yeah. And you're like, the innovation potential is 0.7, whatever that means. Uh, anyways, but there was, it was an interesting, right. One of the things he was going over is,

26:32

so you have two types of organizations. You have an innovation based organization that values doing new things. And then you have, and I like the neutral scientific way he was putting things like the following. And then you have an organization that values career advancement, right? And you know, we can all agree that's great. Like we all would like, I think there's very few people who are like, no, no, I don't need more money. I totally feel like the balance of input that I give and the compensation I have is, is fine. Here's some more back. Uh, but so anyways, there was these two modes of operating and obviously, maybe not obviously, if you're focused on career advancement, you want to minimize risk, uh, and kind of show off as much as not show off, but basically you want to minimize risk. Cause I think that's, that's like a Nassim Talebian thing is like actually success is based on not screwing up, right? Like, and in those orgs, it's like people would rather fail conventionally. Yes, exactly. Yeah. And so one of the things he pointed out, and then I'll finally, I was trying to break this habit I have of asking a question and then talking by talking and then asking a question, which is going just as well. Uh, but like on the topic of how you incent people, like he had a, the, an interesting other tactic from career advancement was, so instead of rewarding individuals monetarily, like for, for being good and putting that in quotes, instead you should, uh, not even really like reward them, but just like sort of like talk about how this was a good innovation and, and also, and base it also on a different team of people. Like one interesting aspect was that your manager will reward your behavior, which kind of drives towards this career advancement versus if you have some external board who's saying this was a good innovation or this was a good thing, it kind of removes that whole career advancement angle, which when I read this, it made a lot more sense. But anyways, so that's one little interesting way of shifting people to be more innovation focused. And I wonder in y'all's experience, like if there's other things that you use to shift people to change and do things in a new way, like what are some tactics you've seen? I guess another good

28:33

one is like find a new job. Yeah. Like, although, you know, that there's kind of that, that saying that you, you know, change your company or change your company. Right, right. And like, you can't change your company, unless you change your company. So like, it should be like, be cool with the company you're at or leave the company you're at. Right. So like, if you know the culture of your, your, your company, you can figure out how to survive in that culture in a way that hopefully you can get some level of satisfaction from, even if it's not the innovative DevOps culture you want. Right. If you can find that, then you're cool. If you can't find that, then you fight, you fight, you end up burning out or whatever. Um, so I'm not quite answering the question you asked. No, no, no. I'm not quite asking the question I asked. Right. I feel like it's like, as, as a practitioner, like don't get hung up on the culture of the place you're working at, get hung up in figuring out how you can like thrive in that culture, both from, you know, people around you like you, but also from like, I go home not wanting to like slip my own wrists. Right. Kind of, you know, and so a lot, and a lot of that you can get from like, you know, DevOps is, you know, one of the primary drivers for DevOps is like reducing toil, reducing the, the, like the day-to-day like repetitive work that everyone hates doing. Well, you start like just doing that for yourself, even if you can't do it for your whole team, your whole company, at least you'll start feeling better yourself. Right. And then maybe you can start like, you know, sharing a little bit with the people around you and getting more people interested in doing that. Right. And I suppose from a managerial level, you could, to use the word again, enable your, your staff to do that as well. Right. Like you could sort of start, start to prioritize not doing like annoying toil type of work. Right. Yeah. So just, just giving people some trust. Right. It's probably an incentive to actually start working in a better, smarter way. Right. Right. And, and do you, what are some ways to like demonstrate you're giving them trust? Well, other than saying, I trust you. Well, but

30:33

one of the, one of the things that, that I know I've seen pop up multiple times is what tools do you want to use? Well, that one's, that one's dangerous in a lot of ways, but I mean, I had a good friend of mine once who his theory on this was if you really want to do attract better talent, one of the easiest ways to do it is you tell us what kind of laptop you want. We're not, we're not going to dictate to you, here's the laptop you're going to order. You know, what do you want? What do you need? You know, and I look at it in terms of tools. You know, so if, if you were a professional chef and you were in a kitchen somewhere, there isn't a block of knives and you're forced to use whatever block of knives the restaurant manager got on sale or, you know, got comped by the rep, you bring your own knives to the party. And if you like Wustaf, you bring Wustaf and it's your responsibility to keep those, those things sharp and clean, you know, but it's your choice. And I would argue that that, frankly, it should be no different for us when it comes to how we do our jobs. If you're super productive in, in IntelliJ, we should buy you an IntelliJ license. If you're super productive with Sublime and a bunch of plugins, knock yourself out. It isn't my job to dictate tooling to you necessarily. You know, again, that, that, that should be a developer level thing. And then now the platform is a different story, but you know, we should give you as much choice as possible. I, I think in a lot of ways that is a great first step to say, we trust you to do the right thing. We trust you to use the tools that you are most productive with, you know, go forth and prosper. I guess maybe, maybe to use the contract metaphor again, like for a long time, there's always been this, uh, it's probably a developer who came up with this where the developer would say like, give me a requirement and don't tell me how to implement it. Don't solution me is the business analyst wife refers to that. And it, and it seems like, so that's sort of like a, uh, working relationship between, let's call them a product manager or a business analyst and, and the development team. Uh, and it seems like that kind of thinking basically the contract is, and therefore the requirement giver,

32:34

uh, their responsibility is to say like, here's what I want to happen, right? Like as a result of your, whatever software you write executing, this thing should happen this. And, and so that's sort of a way of doing a contract. And, and it's a little bit of what you're saying is you take that, or at least what I'm hearing you say, you take that same approach and spread it to the rest of the system, right? Like I don't care how, what tool you use to write your software, but it needs to like be packaged this way and be this code and result in this. And like, I don't care how you do this reservation lookup, whatever is behind this, this contract, like API microservice thing, you can do whatever you want, but you have to satisfy this, this contract. And I don't know, I mean, it seems like a pushing out of this, like, don't tell me what to do or don't tell me how to do it methodology. Well, I think the other way you're starting to see more companies kind of extend this trust metaphor is to get beyond like the really strict confines of this is what our day is, you know, and we, you know, and I think more companies are finally starting to realize, hey, if you're more productive in the afternoon or in the evening or in the early morning, like, knock yourself out, you know, and you're seeing more and more companies finally figuring out that, hey, I might not have all the best talent within, you know, 60 clicks of my headquarters. I might actually have to enable remote work. I mean, that's the one that kills me is how many companies are like, oh, no, we can't, can't be remote, got to be on site. Really? Why? You know, I mean, now maybe I'm jaded on this because we're a completely remote team, but the notion that I need to physically be in the exact same places you are for us to be able to communicate effectively just doesn't hold anymore. We've got lots of great tools that allow us to communicate. And I can even see you. I mean, we've got pretty good screen sharing technology, pretty good. Every laptop sold in the last however many years has a pretty good camera on it. I mean, you can even do a lot of this on your phone. It's like, if you don't trust me to get my job done, maybe you should just get rid of me. Yeah, that's the expense policy we should have. Well, as some companies do. Yeah. Well, well, before, before we end up, before we run out of time, are there any questions that

34:38

y'all have or comments? For example, how often do you meet with each of you generally in your team? Oh, us? You said you work remotely? We do these events together. We usually overlap for at least six or eight hours. But also, to be clear, we don't actually write software together. No, no, we're pretty unique that way. We just talk about writing software. We don't actually do it. We're just a bunch of talking monkeys at this point. No, it's mostly at conferences, mostly at events like this. You know, we do, I guess we've got an offset coming up in October, something like that. Oh, yeah, yeah, yeah. You know, we do occasionally get together and meet space, but... But like, we're all in Slack, so you know, we're constantly talking about stuff. I guess sometimes seeing is also important. Yeah, absolutely. Face-to-face. Yeah, and I can add two things to that. So one, I guess this is what ironic means. Ironically, like, there's a part of Pivotal called labs, and they're very into a disciplined way that you do everything. Seemingly the opposite of what we were saying. Yes. But when it comes, they often like to co-locate, but they also have figured out how to do remote work a lot. And it's like all these things. It's just like, I don't know, I've used video conferencing. It works. So that's what they use. And then I think another thing that's more pulling, that's a generalized thing that definitely our team does, and you know, this is one of those things you hear about all the time, and it sounds like that was a great 20-minute talk with a bunch of pictures on a slide. But it really is effectively if you figure out what the principles are and the, I forget like the militaristic term for it, but like, what are we doing here that you're going with? And so on our team, we don't really report back to Central very much. And I'll speak for myself, but it's probably them too. We don't really ever watch the all-hands meetings or like those things. So what that means is that we do have to find out like what this quarter's or this year's sort of like deal is for the company. So we have to somehow proactively like sync up on the official

36:40

propaganda of the company so that we don't embarrass the company, right? As an example, you saw all of us emphasizing you need to build a platform, right? Which is a very pivotal principle thing. So I think that is a very real sort of, if you're doing distributed development, either the team has to take it on on their own, or the management, to use that term loosely, has to be very clear about like, here's the principles we're operating under, right? Here's the things that we need to accomplish. And then again, I'll trust you all to figure it out. But you got to, if you have dispersed teams, you really have to tell them like what you want them because they're not going to go to those. And you have to enable it, right? You have to make sure you have like Slack and Zoom and like whatever your equivalent tools are. Right. If you're not like, if you have all these crazy corporate policies against those things, then remote work's never going to be effective. Like you can probably get by, but you're not going to be as effective as in person. If you enable the right tools and the right people, I mean, you could be just as, or if not more in remote as you are in person. I do think even in these remote scenarios though, you should get together face to face periodically, whether that's once a year, three times a year, but you should try to get people together. If you've got multiple sites, you know, first quarter, three or four people go to that site. Second quarter, three or four people come to this site. There's something that does get missed from sitting down and having dinner together, going to grab a cup of coffee together, you know, and a lot of that sort of adjacent possible stuff doesn't happen as easily when you're remote. And it's a much different experience to be face to face than it is even on a screen, you know, which is still much better than just disembodied voice on a call. You know, I remember being on one remote project where, you know, we had one project room in one city, there's another project room in another city, and it was a bit of a cluster until we got video conferencing that was live all the time. And so each room had a TV and a camera, and we kind of referred to it as like the window, because like you'd be, you know, communicating to somebody over some proverbial slack type thing,

38:42

you know, like, hey, wait, we're not being very effective here. Can we just chat and we'd go to each other and then essentially face to face across the screen and a whole bunch of problems just disappear. But having that togetherness is where you start to develop some of that trust and you have some of those inside joke things and a little bit of ribbing, and that really congeals the team. If you don't have that at all, it's really hard to work together effectively, I would argue. And then, and then you can share the pain of who's going to have to work with the expense auditors on the $3,000 meal, which not that I've ever had to go through that. But like I said, that's a fun process. Well, well, great. Well, thanks for thanks. This, this turned out well, from my perspective, well, that's it for this episode. If you enjoyed that conversation and you're interested in things like that, or learning all about spring and other types of programming and a little bit of operational and devopsy stuff, you should try to come to one of our SpringOne tour events. They're in various cities globally. We've got a lot going on to the rest of the year and we should be having some next year as well. But if you go to SpringOne tour.io, you can find the dates for them and see how you can get to one. Maybe it's even right around the corner from you. And of course, in October, we've got our big conference coming up SpringOne platform at SpringOne platform.io. I've been picking through some of the talks and arranging some of the other parts are relevant to the tracks I'm helping with. It's going to be a really great conference in my hometown of Austin, Texas, which has all sorts of great stuff going on. So again, if you're interested in either of those, you can either go to SpringOne tour.io or SpringOne platform.io to check that out. And with that, we'll see everyone next time. Bye bye.

Tanzu Talk · 🍕 Get more pizza.