The kubernetes community for developers, with Jorge Castro

Episode 181 · 58:22 · SRT · archive.org · all Tanzu Talk

Hey everybody, this is another interview that I did first in our live streaming thing over there in the Tanzu TV land, which you can check out if you want to, if you go to twitch.tv slash VMware Tanzu. Now, I do a broadcast just about every weekday at 11 a.m. Amsterdam time if you want to see it live, or if that's not a good time for you, there's the full archives of them up on YouTube, which I list at Coté.io slash Tanzu Talk. So enjoy this one, and if you want to, I don't know, get sort of fresher things and see some mostly monologues from me, but every now and then I have guests in, just go over to Coté.io slash Tanzu Talk, and you can find the playlist over on YouTube to look at, or you can figure out how to watch it live, if that's more your thing. And with that, enjoy this interview. Well, this week, well, not this week, it's today. Since I do, I, can you believe this? I try to do these things daily because I want to make my life hard. Do you think that, you think I'm going to be able to sustain that? You really do these daily? Yeah, yeah. I mean, I mean, I've been doing it now. This is one of those things I've been doing it daily for, I think this is the fifth day. So now I don't, I don't do it on the weekend because we live, we work in a sophisticated organization that would not tolerate such nonsense. So I got that going on, but I forgot what I said I was talking about, but here we are. But do you publish daily or do you have like a little buffer so that when you go on holiday, I am creating a buffer, both in my midsection and in my media asset area. So I, you know, we do have, I live here in Amsterdam and we've got like a, there's a week off in October at some point. And so I was thinking like, even though we can't really go anywhere or do very much, I should take that time off. So I do need to buffer something up. I'm going to do some, some buffering for sure. Yeah. The internal hobbies have been a godsend. I think this, that's right. Yeah. Yeah. Anyhow, why don't you introduce yourself? Hey, so I'm George Castro. I work at VMware as a Kubernetes community manager.

So I'm part of the Tanzu organization. Yeah. And you are reminding me that we met a long time ago back in Austin somewhere. We weren't quite sure where, but we knew that we knew that, you know who Woody is. What was that Woody Allen movie where there's like Zelinsky or Zelig? I think, I think the Zelig of our world is a Barton George. You just look, all I remember is we were hanging out and I think I'd heard of you before, like from your blogs and Twitters from like a long past. And then Barton just introduced you as Coté. He was like, do you know Coté? And I was like, doesn't he have a first name? And I, for some reason I've always remembered that. Yes. Yeah. That is my strange affectation. Over here in Europe, it freaks people out or it doesn't freak them out. Europeans don't get freaked out about anything. It, it raises, they raise their eyebrows a little bit. They're curious. And then I believe we were trying to convince them to give us laptops and I don't think that worked. Oh yeah. It's, it's, it's, it's always hard to, to get some of those out of, uh, yeah, they're fine laptops. The XPS 13 developer edition. I think, I think there's, there's two or three other profiles now. I think that was, yeah, that's kind of, I should have him on here at some point to talk about that. Cause that's a, that is just got refreshed too. I literally started looking internally. I was like, how do I get one of these? Yeah. Yeah. Cause I'm due for that laptop refresh soon. Yeah. That would, well, anyways, so, so is this an accurate way of putting it that in the, the VMware Tanzu formerly Heptio world, you're, you do community stuff. Yeah. In the, in the Kubernetes community world. Yes. Yeah. Yeah. Yeah. So do you want me to explain like how that works? The position and. Yeah. Yeah. Yeah. Yeah. So usually. Yeah. Go ahead. You know, VMware, we, we obviously build our products on top of Kubernetes. And things like that. And similar to other companies that build products on Kubernetes, the health and maintenance of the community itself is important. So VMware has this position

that specifically to work on upstream stuff. So I'm a co-chair of SIG contributor experience. So Kubernetes is organized into these things called SIGs, which are working group style things where it's like a bunch of people and they work on a certain part of Kubernetes. So I have kind of parts at red hat, Google, et cetera, things like that. And so there's a diverse group of people who work in Kubernetes. Some don't work full-time on it at all. Some do. And then some work at maybe smaller companies and do it on, you know, a shorter amount of time, or some people just do it for fun on the weekend kind of things or on the spare time. So. But for us, because Kubernetes is so crucial for us, we have, we usually like invest in ensuring that things are looking good upstream. So I wear my upstream hat probably 80% of the time, I would say. Oh, upstream hat. Do you have that laying around somewhere? I do actually, but I gave it to someone at KubeCon. Paris made these ridiculous sailor hats. Because everything has to have a sailor nautical theme. So someone actually made those silly Navy looking hats with the symbols on them. Yeah. Yeah. I did not like. Yeah. I was keen to give that one away. You got, you got, you got the sartorial, uh, like diversity of Nantucket there, all sorts of crazy stuff going on. Just, yeah. It's, it's either nautical or Greek and then they just mix it up and then everything has to be named. It's like the, it's like the meme. So, so, so, all right. So you already laid out a whole lot of, a lot of, uh, things there. And this is, this is exactly what I wanted to, why I wanted to talk with you is, is I'll, I'll fully admit, I don't actually really know much about the, the Kubernetes community. And so far as like, um, even the words to you, but just like how it works and what the parts are and like, you know, nevermind the sentiment that could be a whole other second episode. We can be like, boy, check out those people. How about these ones and how, how they all fit together and enjoy each other. Uh, but more of just sort of like, if, if, if you look at any community as not only this,

but one way of looking at community is it's just like a system or a machine or a, um, it's a tool, right? Like, like a living organism of a tool. Uh, so I was curious just to like, from my own edification to kind of go over how it all works, how it's laid out. Sure. But then, but then also the thing I'm curious about is from the perspective of like a, let's say an application developer, right? Like what, like how you would kind of help them navigate through it, through, through what they're supposed to do. And, and just to end my, you know, as I'm fond of saying, I like to ask questions that end in periods. So let me finish my sentence here is like, I think in thinking about what we talk about in the framing, I was thinking like, you know, at least in my experience for any given community, there's kind of, let's say three to four parties involved. And you could even think of them as types of developers, um, or let's just say developers, there's parties that aren't developers, but basically there are, there's the community of the developers who work on the stuff, right? They're the ones who actually write the code. Right. And then you have, you've got this other set of developers that are interested in the community. And usually they come from the people, if the community is like this, that commercialize it, right. That, that take, that use the open source and do whatever, whatever, however, it might be commercialized. Uh, and that could include like, whether it's vendors like ourselves or system integrators or whatever, but there are people who are interested in this chunk of code because it's part of their business to sell it. And then the third one that you have, and I guess I would put service, you know, cloud people in that second bucket. I don't know. But then the third bucket is basically like what I'm calling application developers. And I'm going to come up with a fourth one now, I'm remembering, but the application developer is like, you know, I write, if you want to transfer money between two people, I write that application and all the stuff that you would need for that. And now, now that I'm, I'm talking to myself here, there's a fourth category, I think,

in Kubernetes, which is they're not application developers, but they're like platform developers. I don't know. I don't know what the Kubernetes world calls that, but anyways, yeah, yeah, yeah. That's my theory of like things that move around and now you are here and you can like, you can tell me how it actually works. Let's go over all four of them, but I want to do application developers last. Sounds good. So first I would, I would categorize as Kubernetes core developers, right? Like these are people who are working in the project. You know, they have, they're in owners files, which so in the source tree of Kubernetes, there's a little file called owners in every directory. And in there's a list of GitHub handles, right? And who's, if your name is in there, you're responsible for that code and, you know, some of the requests that you review. So we have on any given time, we have about a thousand and I'm kind of squiggly, a thousand core developers that have contributed significant contributions to the code base. Yeah. Out of those probably around 750 ish have been active in the past cycle or so. The reason I know this is we're in the middle of having an election and we had to figure out, you know, who's been active. So that's, that's kind of the, the, the figures there. So about, you know, I like to say about a thousand core developers and then it depends, right? Sometimes somebody might be working on something and they are in one of these other groups that you mentioned. So they might cycle out to do product or something for a while at wherever they work and then come back and do more upstream stuff. So, you know, depending on the organizations, that's just kind of how that goes. So that, you know, you, you look at it cause every time you go to a conference, you know, everyone's talking about these numbers and how great they are. However, I think what makes Kubernetes really cool is the second group, which is more of like outside the core developers. So we've had about, I think the number is up to like 42,000, what we call drive-by contributors. And these are people who

just wanted to fix something, you know, they did a pull request, but you know, maybe they don't work on it for full time or they're trying to like deploy it at work and they find a bug, et cetera, et cetera, that kind of thing. I've been doing Kubernetes for almost three years now. And the entire time I've been here, there's always been a thousand open pull requests to give you an idea of like the velocity of the amount of pull requests that are coming in. And our velocity has been increasing as far as, you know, trying to integrate as many of those as possible and get those reviewed. You know, obviously we're not going to catch everyone and I can give you tons of examples of where, you know, we kind of messed that up, but you know, generally just to give you the size of the scope of this thing, it's just every time I think we finally, it's like, okay, well maybe this cycle, you know, we'll level out. And then every, every single time I look, the numbers just keep continuing. So that's one of the things that's, I don't want to say keeps me up at night, but one of the things that we try to focus on is things like sustainability and, you know, how do we get maybe senior developers that started off in the project, writing lots of code? You know, how do you get people into kind of a ladder? So they're mentoring new contributors and things like that. So we can keep all of that going. Um, I guess you're right. Cause that is, that is one of those, uh, you know, one of my favorite, uh, business BSE kind of phrases is, uh, a good problem to have. Right. And, and, and, and so what you just described, I think is a good problem to have that you have a lot of, a lot of people who want to contribute and then you have to like, uh, deal with that. And it's complex because like Kubernetes is, you know, you've heard it described as like the operating system for like the data center or the cloud, whatever, whatever you believe. But either way, there's tons of components, right? We have something like 43 SIGs or it's, it's above 30 or four, you know, around depending on what's cycled up or down. And then there's working groups that we spin up and down, but there are lots of components

to Kubernetes. So it's not just, Hey, I want to do a thing on Kubernetes. It's like, are you into storage networking? Are you into what happens on the node itself? Right. Are you into etcd? Like there's all these kinds of distributed components. Um, and on top of that, you got the meta parts of the project that you need. So the SIG, I participate contributor experience, you know, our whole job is when someone does a pull request, are they getting, you know, is that pull request getting in the right eyeballs, you know, in a certain, in a reasonable amount of time? Yeah. Yeah. You've got to, you've got to route it to the right group. That's responsible for that part. Right. So we depend a lot on automation. Right. Yeah. And as a result of that, that can lead to some friction. Right. So if you wanted to do a pull request today, a bot, literally a bot is the first thing that talks to you, which is kind of, kind of awesome. Wow. Like I, you come from a background, right? Where smaller open source projects, you like the personal touch and, you know, it's nice to meet you. Thanks, you know, but like at this scale, there's just no, there's just no reasonable way to do this without automation. So, you know, we have a bot checks to see if you sign the CLA, you know, it'll, it'll say, Hey, you haven't assigned a SIG to this here, the commands you can assign, or if, if you don't know what to do and it sits there after a while, you know, the bot will try to put it on the radar of whatever SIG we think it might, you know, depending on where you did a pull request. And then it kind of goes through this like long, you know, process. And if you're new, that could be very intimidating. Right. Cause it's like, I did a pull request and a bot's already, you know, yeah, that's, it's, it's interesting. It'd be overwhelming. Yeah. I mean, I mean, you know, I, I always, uh, nowadays I always get kind of excited when there's an actual bot for customer service. Cause like the alternative is like talking to someone and, uh, it's, it's made me realize that like, I mean, I mean, I guess, I guess not talking to a human has a bad reputation because of like IVRs and stuff, but like it has a bad reputation when it doesn't work well.

And so like, if it works, it's anyways, that's an unrelated, uh, parenthetical there, but okay. Okay. So, so, so we have labels, you know, and, and we do like, when I mentioned the scale, there is a lot of, you know, like we can't just consume normal GitHub. Yeah. So there's like, you know, part of the things we do in our SIG and we have representatives that actually talk to GitHub directly, right. We're working with them when it comes to like, Hey, you know, you have a senior developer, uh, you know, who's getting pinged, you know, and the notifications workflow that they first started to land, like, you know, that might work for, you know, 90% of, of developers on GitHub. Right. But you know, at our scale, you know, how do you manage all these, how do you rattle that through? Right. Yeah. So that's like an awesome thing that we do where there's like a constant feedback loop where, you know, GitHubers will come to KubeCon and the, you know, and we'll meet with them regularly. And we're kind of always, Hey, you know, we want to automate doing this. How should we do it? You know, just the kind of day-to-day things to make that process easier. And that helps us kind of manage the, the drive-by contributor. Do you use that? Uh, what's, what's that, that actions thing they have in GitHub? I forget what they call it, but you can like chain together a bunch of things. Y'all use that? Oh, we, yeah, we, yeah. Actions. Is that what they're called? Without, yeah, without, we use actions for lots of stuff. I mean, we, we have a bot, we have a bot called Prow, um, that I think actually other projects have started to use as well, but that's the thing that, uh, you know, and it's all open source. It's all in our infra, uh, you know, everything we make is open source. Yeah. You got to watch out because then you're going to have a bunch of people who are building a platform for building a platform that are building a platform for building platforms that just on the kind of like, you're going to be four or five levels deep. Once, once you start getting there. When I first started, there was an awesome bug in the bot where it was actually talking to itself. And I, for, I should have bookmarked it because it's a bot talking to itself. And I remember Chris Nova responding, being like, what's happening here? And it was just like this, it was like this perfect storm of like when automation,

you know, goes wild. Yeah, exactly. So, okay. So you've mentioned a couple of times you have these SIGs. I assume that's the old, you haven't come up with a new meaning for that. It's a special interest group, special interest. And, and so, so what is a SIG in the, uh, the Kubernetes world? So a SIG is a working unit. Um, it's kind of our basic working unit where things get done. So SIGs own code and responsible for certain parts of the Kubernetes code base. Um, so let's say you were a networking person and you were like, Hey, I want to get into Kubernetes. And I would say, Hey, what are you into? And you say networking. I would tell you, you probably want to go to SIG network and each SIG has a public meeting, a YouTube playlist, um, meeting notes. Uh, they have co-chairs, they have tech leads, they have members, uh, they generate their own caps, which are Kubernetes enhancement proposals. So if you want to write a new feature, you work on a cap, those get checked into an enhancements repo. People discuss it, you know, everything happens in GitHub. Um, you know, and then people implement it. If there, if you file a bug on a networking component, you know, that ends up on SIG networks radar. So we have those SIGs kind of control, uh, the different areas of the code base. And, and then, and do SIGs have like sub SIGs or is it just like, there's more or less a one-to-one mapping of like, yeah, they have, they have sub projects. Yeah. Um, that own certain parts that you want to split up. Right. Right. Like, I mean, I imagine networking is quite large and as far as like things that it would do. So you would have many sub SIGs there. Yeah. Right. And then like SIG cluster lifecycle, which is responsible for the installation upgrades and stuff. They have tons of sub projects as well. So if you've used the tool kind, you know, there's a sub project for kind, there's one just for cube admin. There's one for cops. I think, I think cops is a sub project. I'm, I might need to look that up, but tools that you use, you know, to use Kubernetes, lots of those belong to SIGs and they may or may not, uh, be a

sub project, uh, within a certain SIG, depending on how they're organized. Okay. Um, so, so in other, in other open source communities, I mean, you have the similar construct, right? Like, like it, the, depending on the type of open source community, like the project might be small enough that it is its own thing. Right. Like it's like, right. It's just like, I don't know, whatever. And, uh, but then you might have much, much larger, uh, like I imagine, I don't know how, like, for example, well in the eclipse world, I mean, eclipse is a lot of different things. And so there's many different subgroups and subgroups of subgroups, but okay. So that, so the unit of the people who work on a concern or a component of Kubernetes is a SIG, which can be further divided into sub ones. And then there are also the ones who basically, that is the, the entity that can make suggestions. What would you call them? Kreps, like suggestions of new things, keps of things to do. They're not, they're not very thin pancakes. You got to be careful over here in Europe. Sometimes when they tell you there's a pancake, you get a crepe and it's just like that. I don't, you don't like them. Oh, that's like, no, no, no. I mean, they're good. But I mean, imagine, imagine if, imagine if you're like, I would like a big stack of American pancakes and then, and then you get like, like some delicious sugary doilies instead. Right. Like, I mean, maybe it would be delightful. It's just the expectations don't match up is, is the issue. So, uh, okay. Okay. So that point I do want to make before you go on, however, is in the early days, a lot of things were in core, right? In github.com slash Kubernetes slash Kubernetes. We've been making, remember that whole thing I talked about sustainability. Yeah. We've been making efforts to, if something isn't core or important, kind of move it out of the core repository and break it down into other repositories that may or may not be in core. So something like cloud providers, you know, used to live in core and it's taken us years to like remove those, cloud providers to work on things at their own pace without having like the kind of things,

you know, having to deal with getting reviews and core and all that kind of stuff. So there's been an effort to actually, you know, if this isn't a core thing, does it really need to live in the core repository? And a lot of people have been working to remove that to kind of help, you know, like I said, with the sustainability, if we're removing things that aren't important to core, core can remain tightly scoped. Yeah. That makes sense. You've got to continually make sure that you, you're not creating a giant SIG, I guess, which, which is, is unmanageable. Okay. Okay. So then, so then, then typically, I mean, like you said, if there's code to be owned, there's code, but I imagine there's other SIGs that aren't necessarily directly code related, right? Exactly. So this is one of the cool things for those of you who aren't coders in the audience is Kubernetes needs every skillset that you can possibly have, not just coders. So we have a few SIGs. First of all, we have a steering committee, which kind of, you know, sets the tone for the project. The steering committee doesn't lead the technical direction of the project. That is powered down to the SIGs themselves. Right. So there's no, like, there's no meeting where everyone sits down and says, you know, we're all going to do like, you know, the SIGs kind of, kind of lead the way there. And usually steering is there for like governance reasons. Sure. And to, you know, to help kind of delegate responsibility. And then you have SIG release, which is kind of a meta SIG that kind of coordinates with all the SIGs on what it takes to release all at once. Right. So they, you know, they select a release lead. There's a release team and it is very new user friendly as well. You can absolutely not know a lot about, uh, Kubernetes or the project and get involved in SIG release. They have a nice shadow program. They have, um, like little cookbooks. It's like, Hey, I volunteered to be the person that's going to write the blog posts. You know, I hear your instructions and past work and here's all the people that are going to help you. And you know, that, that kind of stuff. SIG contributor experience. We tend to also do a lot of, uh,

the meta work. So everything by running the YouTube channel, moderating the forums, uh, the lists helping run the elections. Um, let's see what else did I do? Uh, the GitHub management is an entire sub project of SIG contributor experience. Um, we run events normally. So if you go to a cube con and you're a contributor, we have a contributor summit with new content for people who were like, Hey, this is my first time contributing to, Hey, I've been around here for years running workshops, uh, that kind of things. Uh, we were on mentorship programs and, you know, all of this organizing, all that, uh, always needs different types of skills. So lately we've been getting a lot of people that are interested in program management, right? Like, you know, so, so, so, so yeah, no. So talk about that more in the sense of, well, in the sense of, because of like, it's easy to conceptualize. Here's like a, here's like a PR, right? I wrote some code and then, and then, and then I talked to your delightful bot and I, it, which was just a great experience. And, uh, and now it routes that around. But like, if I want to like, like, how, how do I contribute if I want to like work on the docs or, you know, like, like you can't like submit yourself as a pull request. Yeah. So SIG docs is actually the first SIG that I got involved with. Yeah. Uh, when, when I started, I needed to fix something for work and I was like, okay, I'm gonna, I'm gonna do that. So that is an entire state that you could just show up. And, uh, docs is always an area where people could contribute. We always need technical writers and we always need like copywriters. Um, and things like the Kubernetes blog also fall under docs. So that's also a great place. You know, I try to tell people, Hey, you saw something cool at work, write it up for the blog. It's just right down pull request. Yeah. Um, like it's, you know, sometimes people aren't aware, Hey, it's an open source project. I can submit up a blog post. Yeah, absolutely. You know, put it, put it up there. So docs is an excellent place to contribute because it's always an area. The project moves fast, right? We, you know, we release four times a

year. That's like twice as fast as like your normal Linux distros. So, and the docs are versioned. Uh, right. So there's always something new and, uh, that can, that you can work on. So six boxes is one of the areas that, you know, I tell people, even if you don't want to contribute to docs, just going to the meetings and like helping take notes or listening or seeing, you know, okay, here are the, here are the enhancement proposals that the SIG is talking about. Yeah. It can help spin you up on, on what's going on there. Um, but yeah, docs, docs is definitely a place where we could, we could use help. It's relatively low, uh, barrier to entry. It's just marked down in Hugo. Uh, you know, you just, so the avenue there, I mean, there's always the avenue of like, if you just send in a, a PR for something, right, that'll notify things, but then there's also, I imagine you could also like talk to someone like, is there, is there that like that, you know, and be like, I have a non-technical thing I want to do. Like, how do I, yeah. I tell people to show up to a meeting, um, all of our SIGs, we have, we have a little section that says, Hey, is there anybody new here? Anyone's first timer? Um, you know, and then it's not uncommon to see, Hey, everybody, you know, I'm George, I have these skills. I want to do this. Um, and it's like, well, what do you want to work on? And like, yeah, I submitted my first PR and I'm kind of lost, you know, and then, you know, in, in our meeting, in our SIG, we, okay. Uh, do we have a volunteer to help this person out, close out this loop? And then they kind of start that way. I also recommend people to like find a buddy, find a mentor. Uh, and that's something we try to do is pair people up where it's like, okay, you're both new here. Um, you know, we, uh, we tried to curate certain issues or, you know, in GitHub, that's a good first issue. This is something that always do a better job at. That's like, Hey, I'm new here. What do we need to do? And we, like in our SIG, we always have a list of, there's always a list of stuff that, um, needs to get done, but might not be, might not look

important to new people or outsiders coming in, but that need to get done. So what I like to do is I like to save things that look minor, but, um, are good for new people. Right? So if I meet a technical person, they say, I'm like, what are you good at? And they're like, I know, get, I know markdown. I've done requests before. Okay. Here's an easy one because we also do things the Kubernetes way with this bot and all that stuff. So here, you know, change the dates on this document to be more up to date, you know, for the next milestones or what, you know, whatever. Yeah. No, that's interesting. Easy, but it gives you a chance to kind of, okay, I've submitted my pull request. Uh, there's a reviewer. Someone does a LGTM, which means someone says, look good to me. And then a second person who is the approver kind of gives it the final thumbs up. So you get two reviews on everything. And then, you know, okay, cool. You got your first pull request is congratulate, you know, and then we all, we all give them a thumbs up and slack and then, you know, then they're ready to move on to next something more complicated and that in certain cases, there are people who are just, they just dive in, you know, or, or they, they, they might know someone or they're a senior person in a networking stack on some other component, you know, in some other open source project. And then they slot in, like, it's not uncommon to see lots of open stackers, um, you know, in Kubernetes that just kind of like ramped up very quickly because they know the problem space. Yeah. Yeah. Yeah. It's a similar gargantuan community with a similar management. Right. Exactly. Yeah. Yeah. No, that's interesting. That's, I mean, that, that, that's the idea I'll have to see if I remember to think about later is the idea of, uh, saving tasks to onboard people with, right. Like that, that would make, it's, it's an interesting form of like delegation of, of like, but it's, it's, it's not only like delegation to like, let you focus on spending your time more valuably, but also like make sure that as you were saying, you can like, uh,

train other people or, you know, get other people under it. Okay. All right. So let's see if I can get us back, back up here. So we have, uh, you're organized and into, uh, into SIGs of the various components. And these are the people like working on the actual Kubernetes code and, you know, and, and docs and, and other sort of the, the thing of Kubernetes. They are, they are the, the builders of it. So then, so then, uh, where do we expand to after that? What, what, uh, what else is going on? So I want to do app developers last. Of course. Uh, so the, the next step is people who are building products on, on, on top, right? You have core developers and then your outside kind of drive by developers. Um, there's a lot that the Venn diagram is like really meshed in when it comes to the second group. So these are people like, uh, you know, you might be a Googler who works on Kubernetes, but your day job is Anthos, right? Uh, you know, I work on Kubernetes, but my, you know, part of my day job is Tanzu and so on and so forth. EKS, AKS, uh, all the cloud providers, you know, have one, uh, and you have integrators. You, there's this nice intersection of, uh, smaller companies that fill niches within the ecosystem that are also deeply involved, um, in, in Kubernetes. You know, I'm thinking people like Weaveworks, uh, super giant, um, you know, and a lot of these companies have been around for a while. I've been doing a lot of Kubernetes. Kinvoke is another one. Um, I shouldn't have started a list now. Now you want to finish it. Anybody. Yeah. So, um, you know, you have these kinds of different, uh, areas of the ecosystem being served by different companies and things like that. So a lot of, a lot of that balance I think goes from, you know, uh, we, we ship product, uh, but there's the open source thing. Uh, and the analogies we were trying, I try to use the ecosystem of like a forest, you know, we put furniture. Um, but it's important that, you know, the trees are healthy, that they're being renewed properly. Uh, and in a way, a lot of us are forest Rangers, right? Like, sure. I sell furniture, but I also like

care about, you know, your supply chain basically. Yeah, exactly. Everything from like the composition of the soil, the chemistry, you know, all that kind of stuff that you think about, you know, to keep a ecosystem healthy, you know, software needs the same thing. Right. And that's not just the code itself, but that's like support in, in other ways. Um, whether that be, uh, you know, doing things at KubeCon or, um, you know, any of the myriad kind of things that happen to, to make this project kind of continue to move forward. Yeah. Um, so those, I mean, like I said, like, they're very similar to the first group. It's just, you know, there's like a Venn diagram. So, so what, like how, I mean, yeah. So what, what is that part of the community? What's, how do they, not how do they fit in? What am, what am I trying to ask? Yeah. Like, is there a programmatic way of dealing with them? Like, like, or, or do you just, is it just sort of like, you know, we're all humans. We'll figure it out. Coolest thing Kubernetes ever did. This is an open secret. Nobody ever figures it out. Like, this is like, this is like the one thing Sarah Novotny taught me. She was like one of the first Kubernetes community manager is Kubernetes explicitly when it was started, when it was open sourced, um, has a set of values. Right. And one of one is this community is always bigger than an individual company. So in a way, when we are working in the community, everyone is kind of forced to wear the upstream hat. Now in a lot of projects preceding it, you know, this could, this might have been a problem. It may have, might not been a problem. But one of the things that drew me to Kubernetes is that it, the project itself explicitly says, you know, community comes first, then your company. And then that kind of helps limit, I think a lot of the issues that might affect other things. So we have things like affiliation limits, right? On steering committee, you can't have more than, you know, X amount of people from a certain company. All right. It's not uncommon for, uh, you know, if you have two co-chairs of a

SIG, uh, you know, one works at one company, a one works at company B. If the person in company B ends up working at company A, it's not uncommon for that SIG to say, you know what, we need diversity in co-chairs. So let's swap someone out. And I think our leadership and our contributors are very cognizant of that because they've seen how that benefits, um, the community in general. And then in some cases, some people might have experienced when that doesn't work and, and have the expertise to be able to say, you know what, this is a really good idea and we should sustain it. So I've, it's, it's never been in my, in my experience, there's never really been a, um, you know, an issue where there's like an upstream thing where you have like two competitors fighting, you know, or, you know, like that just doesn't, doesn't really, you know, in my experience, you know, I'm, I think that's, uh, I mean, I think, I think that's what, what over the years you learned from, or one learns from watching open source communities is it's right. As you are going over, it's important to have like some, some norms, if not rules about, about the involvement of, of the vendor world. Right. I mean, because, you know, I mean, as, as you're alluding to, right. Like the thing you always want to avoid is, is in, in, in some previous communities and time, like whether it's true or not, like once you get the specter of like some money to interest, I think we call them a special interest in American politics, but some, some vendor, uh, or, or someone else is trying to like push the direction of something like it's, whether it's true or not, like it becomes, uh, annoying to deal with. So that, you know, And the leadership of the companies, they need to understand that. Yeah. Like it would be untenable, right. If you were to say, Hey, we need this feature in Tanzu, but we are depending on the community to ship it. Like you would never make that promise like internally to put something in a product depending on open source. Now what you do is right. Oh, this is important to us. We will invest in that feature or that area, but that

is a longterm process. It's not, Hey, we need this by this release, you know, so we're going to give you this engineers. Let's make it happen. You know, people have tried that in the past and other projects and things like that. It can never work that way. Right. Tanzu is very dependent on being installed in multiple clouds. Right. So we invest in SIG cluster lifecycle as VMware. That is a longterm investment that isn't, you know, we don't sit there and say, Oh boy, if we don't get this feature in kind, then the next six months, the entire product is going to fall apart. You know what I'm saying? Yeah. Yeah. Like you have to kind of use it, you know, the tree, the tree has to grow before you, you know, before you, uh, make furniture resource. Yeah. Before you can make the furniture. Right. But you got to think about planning and you know, all the, the life cycle of the entire thing. So yeah, yeah. The leadership definitely has to be invested and has to understand that. And we do. Um, and that's how you kind of make, make the whole thing work. So that's, that's, that's kind of how like the people who productize this and then ship it kind of. Right, right, right. No, that makes sense. That makes sense. Okay. So, so we got, we got that, the vendor part of the community there. We got two. What comes after this? Third is your platform builders, which I think this is one of those things where I wish I could go back in time and like try to re-explain people what, what Kubernetes is. Um, I think every time you've seen someone try to do Kubernetes stuff and then they fail and you look at them and then you ask, you know, there's times where I'm like, what are you trying to do here? You know, Kubernetes is, and Joe Bito, one of the co-founders has said this, you know, Kubernetes is a platform for making platforms. And I think a lot of people approach it in a different way where they, uh, you know, the meme where it's like, Hey, I have a small problem. Let me use Kubernetes. Right. And, you know, in a lot of cases, wrong tool for the wrong job. So like the way I try to explain it to people is look, you work in technology, you go, you work for a large company. They have a thing internally,

right? Everyone's always trying to build their own Heroku thing internally. They always have cool names too. You know what I'm saying? Um, so you work at a large retailer and then you have this huge system and it was built and it works. You go to another one, they have the same thing. You go to another thing, they have the same thing. All of those things are all snowflakes. You know, if I work for retailer a and I want to, you know, work at retailer B, it's a whole different technology stack and you know, all that kind of stuff. The one thing Kubernetes does well is it makes building systems like that commoditized. Right. Right. Right. So that you can build your platforms on that and just use the Kubernetes APIs. It's all about APIs. And then I tell people if Kubernetes is successful at one thing, it's ensuring that whatever platform people are working, building at your place will be building common components. And that's a shared talent pool of developers, people who know the technology lower costs for everybody. So it's kind of a tool set for people who make the tool sets. Yeah. I wish I could tell people, look, you read on Hacker News that you just throw Kubernetes at something and all your problems are going away. And now, you know, it's a year later and you're sitting, you're sitting there in your pile of fail, you know, wondering what happened. Yeah. Yeah. I've, I've always, I mean, I always have found the platform for building platforms framing confusing and, and, and like, right. And, and also like frustrating. I mean, just to allow myself a miniature rant with someone who's actually in the Kubernetes world, right? Like, like, I feel like there was this, and this doesn't really exist so much in the Kubernetes world anymore. But at some point early on, there was this strange, uh, I don't know the phrase for it, but there was this phenomenon where it was like developers were really interested in Kubernetes. And so everyone who's a Kubernetes advocate would talk to them about it and go over it. But then they would immediately say, this is not for you. Right. And so it was this weird, like, it's almost like this fuel that I feel, I feel like drove early Kubernetes interest was like

fueled by these people that somehow the thing, it was very weird and incongruous. And at some point, the whole notion of like, it's a platform for platforms finally like settled out that or, or, or I feel like the official Kubernetes, like thought line, the party line was like, yes, that's what it is. And we should kind of toe the line. And then so when a developer wants to come and do our stuff, we should be like, Hey man, this is maybe not for you. Right. Like we really appreciate your interest, but like, we're not going to sort of like enable your different view of what this is. Right. Yeah. I tell individual when they ask, they're like, what should I know about Kubernetes? And we'll move to the app developer in a second. I was like, look, man, for most, you know, for most people figure out your workflow to spit out a container. Right. Right. Right. Exactly. We'll be at an organization where a team will take that and a computer will deploy that thing somewhere. Yeah. Yeah. Yeah. And so, so is that, and so the thing, the thing I've been trying to like wrap my head around over the past, well, whenever this was, yeah, yeah. Since the beginning of this year or so. Right. I mean, you know, when, as, as you know, when, uh, when Pivotal and Heptio came into, to VMware, now we need to like know about each other's stuff more. So I've been trying to figure out like how it all fits together. And like, I think maybe from that developer, that application developer standpoint, right? Like what I've been trying to figure out is like, I think what I would tell them is that Kubernetes, as far as you're concerned, is like an architecture and not so much that it's even an architecture, but it specifies an architecture that you should use. Right. And so like, don't get obsessed, like, like, like, cause you know, developers, one of the things they're taught is like, do things in the Unix way. Right. Which is, you know, have a, I forget what the official phrasing is, have a bunch of small little things that do a really good job that know when to use a carrot versus a pipe and versus a double, a double greater than sign or whatever. Right. And so just coordinate a bunch of small things that pass stuff onto each other. And like, what that phrase is not saying is understand how

to build a Unix system. Right. Like, and instead, like just use the platform as Unix and do this development in this style, this way of architecting. It's distributed Unix, right? Exactly. People are like, you know, you know, the criticism of the Kubernetes, oh, it's too complicated. You know, you're, you're, you're using a huge hammer for a tiny nail. But when you look at the things people are trying to do, it is complicated for a reason. Right. I would love to tell you, you know, it's really simple and stuff. But then people are like, well, I need multi-regions. I need failover. I need self-healing. I need. Yeah. Yeah. Yeah. We, you know, the network should be transparent and magical. I should just, yeah. Yeah. And that should just work magically. Like, we, we, we had a, we had a joke. We used to have a joke, like way back in the, the, the cloud foundry world where like, you know, people would say, I want to build my own cloud. And then we would talk with them and they would say like, this is so complicated. Yeah. And you would be like, yeah, yeah, but you wanted to build your own cloud. Like it's not, there's no, it's not simple. Yeah. And, and that's, that's moving on the app developers. Yeah. That's what I'm trying to tell them. I wanted to save them for last because seriously, I want app developers to never care about Kubernetes at all. Right. Right. Like, like I started to go, I was very unfamiliar with like pivotal stuff. Right. So I have a local spring group here in Detroit. So I went to it and I was just, my eyes opened on what app developers, you know, look and stuff. And in my brain, I sat there. So, you know, if these people have to learn Kubernetes, I have failed them. I want, you know, do your spring, do your spring. And then all the other stuff, that integration, that's all, that's all on us to figure out for them. Right. Like, um, obviously we want them to be designing their applications with those patterns in mind. Right. But I feel like these general kind of distributed computing patterns are easier for them to grok that, like, I just see like a, you know, like a Java developer sitting there trying to figure out their deployment.yml or whatever. Maybe not. Right. There's certain things that we can do to make that experience easier for them. Absolutely.

That isn't, it's just right now because the excitement was there and it was solving problems for a lot of people. And I'm still surprised every day where it's like, oh yeah, we've been running this in production for like a really long time. And, uh, you know, the speed upgrades are still kind of, you know, for some people, some people still can't figure out things, but it's showing a value to them. So I, it still feels like early days. Right. So when people are like, I don't like Kubernetes, uh, you know, this YAML and all that kind of stuff is like, yeah, trust me, we hate it too. You know, it's just, it's still like kind of early days as far as like gelling all that stuff, the same way that when I got into Linux, you had to build your own kernel if you wanted your USB keyboard to work, you know? Yes. And then, you know, I, and I think we'll look back hopefully in a few years and see the progress on the kind of higher level abstractions that we're making. And I think it's up to not just us as Kubernetes, but those platform making people as well, right? If you're building this at work for someone, if you're deploying a Kubernetes style thing, I think everyone's looking for like the cool thing that just lets a developer on their laptop do one command, close it. Right. Yeah. Yeah. Yeah. Dashboard. Everything's green. I'm going to the pub. See ya. You know? Well, I think, I think that's a, that's a good launching point into our last little topic and then we'll wrap up, which is so, so you're making me think like, so is there a SIG group that's like for the application developers like to figure out, like, uh, we do have, we do have working for them? Yeah. Yeah. So we do, we have SIG apps. Um, however, this is a huge space. If you think about it, there's no one SIG that could cover this, right? Like, um, like for example, Helm used to be part of SIG apps, right? Then it got mature and things so much so that it just kind of split out from Kubernetes and now it's its own top level CNCF project. Yeah. Yeah. Yeah. That's just Helm. Yeah. That would, that would make sense that that would, I mean, cause that is mostly a kind of sort of ish for application developers, maybe sort of, right? Like,

yeah. Yeah. Yeah. Or things that application developers will consume, I guess is more. Yes. Yeah, exactly. Exactly. But, but even beyond that, even beyond developer tools, let's, let's call it that. But like, like, so I, I remember I had a conversation with one of the, the, a long time ago, it's one of the open stack board people. And I was at maybe red monk or four or five one at the time. So of course I was talking about developers, right? And I was basically like, so with application developers, like, what do you do for them? Right. Because like, how do they find out what to do beyond to get open stack up and running? And it seems like, you know, as we were just discussing, right? Like, like it is a distributed Unix application or a distributed application. So there's almost like, here's how to architect and write an application that will run well on Kubernetes, never minding actual commands that you use or like, never minding this, but this is just like this, doing things this way runs better. Like that, you know, with my extremely limited technical thing, right? Like the, the, the one, when I was kind of going down this path over the past year, like the one thing that I realized is like at a very minimum, you have to implement, if you're writing code, that's going to be running in a container, that's going to be in a node that's in a pod. Did I get that order, right? That's going to be in a cluster, whatever, right? Like you write some code and what you're responsible for as the developer is you have to implement, I don't even remember what they're called, but you've got to implement these five hooks or life cycle events, which is one boot up. Are you booting up? Are you okay? Uh, I'm about to kill you. You're dead. Right. And like, like, so these, you have to tell Kubernetes to call these, these methods that you somehow define, whether it's through, you know, rest or memory or, uh, or sticking a file, so whatever it is. And like you as the application developer, that must be implemented. Now you can go get a framework like spring to help you like do all that. But eventually you are responsible for telling this infrastructure you're running in,

if you can be killed or not. Right. And so like, like, like that's, that's what I mean as far as like, there's this whole like architecture and like series of patterns that the application developer does need to care about beyond like, you know, how do I get networking to work? Right. And then it actually gets worse. Right. Because when you first started, you're like, okay, I think I figured out this model. Right. And then someone says, how do I do this? 12 factor apps, have a good time. I'll see you later. Right. But then, you know, in the real world, it's totally different. Right. It's like, well, you know, my, I have an application that if you change its IP address, when it gets killed in a pod and come somewhere else, like it totally isn't set up. Right. Right. Right. Exactly. Because now you're doing stuff in the real world and it's totally different. Right. And it's like, hey, I know we said no, no state. All my apps use state, you know, and then it's like, well, oh, now what do we do? Right. And then it started from like, okay, you know, this is the model it's going to fit. But after a while, as certain patterns started to use and people started to use it in anger, all of a sudden, someone was like, you know, stateful sets might be a good idea. Right. When I first started, everyone's like, no, no, never databases, never databases. That's what, what are you doing? Yeah. Now, now you can hang out with like a Josh Berkus and he's like, hey, check out all these things I can do with databases that I get in Kubernetes. And this actually solves problems for a lot of people. And you're like, you know what? I think you convinced me. Yeah, I get it. So I think there's this like kind of meme where it's like, guess what? Everybody, we're rewriting the entire world of micros and go microservices or whatever. It's like, no, you know, but I think you've seen the project adapt to, to things like that. But I think we're, it's an area that's still very much like the situation is still developing. I could give you so many tools that are in this space that the questions aren't answered yet. And this is always my favorite time too. And like an ecosystem and open source project, you have tons of tools. Nobody really knows what the right answer is. People tend to gravitate towards big ones, you know, and then just kind of watching to see how the situation develops

is the exciting part right now for everyone, except the enterprise. He's like watching to see what happens. Right. Um, so I think, I think this kind of like, um, accelerated like evolution, you know, seeing which organisms will, will thrive in this specific ecosystem is interesting because I, if you would have asked me a few years ago, you know, Hey, I have a PHP monolith app at work. I was like, man, I don't know how we're going to, you know, what we're going to do about that. Right. I ain't got nothing for you. I mean, yeah, you know, like, but now you see people just kind of, oh, there's my son. Yeah. Like now you kind of see people saying, you know what? I took this pattern. I took this pattern from Kubernetes and I'm running a thing that no one, you know, and you kind of look at it and be like, oh, it's really interesting. And then this, these are always the best talks at KubeCon because it's always like, here's what we learned. And here's what we broke. And it's always a bunch of things that are unexpected. And that kind of helps just push the state of the art forward when it comes to like data center. Like we make data center operating systems. Yeah. Yeah. No, that's, that's what, that's what I'm, I'm, I'm like, uh, looking forward to over the next year is like how, how the Kubernetes world product manages application developers to put it in one way. Right. In the sense of like, like, there's all of these sort of like needs that that we're discovering and, and nice things that we can do for application developers and the people and the enterprises or organizations they're working on behalf of. And so, uh, so we should do that. Oh, uh, well, I think that's my signal that it's time to end this live broadcast. Hi. And, uh, these, these are my kids here. Uh, but, uh, yeah, thanks for being on that. That was, I feel, I feel like I've gotten pretty smart on this. And, uh, so if people wanted to, uh, uh, in a very non-creepy way, find you on the internet and see what's going on, what, what would you see? CasterJO on Twitter. Um, if you go to kates.dev, k-8s.dev, you can go in there and you'll see the

list of all the things that get started. We have contributor guides, we have documentation. Uh-huh. If you find yourself lost, you could just always ping some, ping me on Twitter, ping me on something. We can get you started. Uh, like I said, you don't have to be a hacker. You don't have to be a developer. In fact, those, those kinds of other skills that you have, program management, technical writing, all that stuff. We love it. We, we have, we have people whose contributions are like, you know what? I, I'm really good at like helping managing Slack channels, you know? We have 100,000 people on our Slack. So, so basically that's a, that's a real job. That's a need that the project needs. Right. And it's not, you know, it's not, it doesn't make it unimportant. You know what I mean? The machine needs to keep going. And if one part breaks down, the whole thing slows down. So, so everybody's important. There is no, there, there's nothing to be left on the floor as far as energy that people want to expend to help out. Exactly. So let's see, we, we, we've come up with, uh, we got four parts of the community, how, how the, how the, the, the, the sort of internally focused people like organize around how the, uh, a little bit of how, um, the Kubernetes community kind of like works beneficially with the, the vendor world. And then, and then we've got the, the kind of, as you were kind of saying the, the solidification, is that how you say it? Solidification? That sounds like one of those words I might make up. That sounds right. Yeah. The, the, the solidification of the, the platform builder operator role, which, which I think was almost kind of not as explicit as it, as it could have been in early years, but now it's very explicit and it makes a lot of sense. And then we got our application developers and, uh, and then more importantly, uh, there's all sorts of things to help out with. You just go to, uh, kates.dev and you can, uh, start to get involved. And it seems like the best way to figure out contributing and getting involved in the kind of the core of it is just like, just, uh, as they like to say, show up at meetings and, and, uh, people are pretty interested in, uh, getting help from that.

I can't tell you the amount of people who were just like, you know, I started fixing a little bug and then I get, and then I went to a meeting and the next thing you know, you get a KubeCon, you're making friends, you're having a good time. Uh, the community is very inclusive, is very friendly. We pride ourselves in like, you know, taking the time to, to, to mentor people and to bring them in, you know, and that's something, that's just something that's great. And I, I think that's like Kubernetes has brought software to the world and I think that's good, but kind of, you know, and Kubernetes wasn't the first to do this. Right. And it won't be the last, but the whole idea of, you know, forming a friendly, inclusive community that is trying to solve a problem, you know, in a, in a way that's fun and helpful and, uh, fulfilling to the people who contribute, right. Not just, Hey, we're making a product. Yay. Like, you know? Yeah. I mean, maybe as one last bonus thing, you know, in, in writing instructions, they're always telling you in your conclusion, you should add an unexpected delight to someone, which seems like, you know, I've always found the problem in like reading, uh, reading, like tips on writing is it's one of those things where like, once you tell people the tricks, they're no longer delightful. But anyways, like I'm stealing that. I'm totally stealing. Anyhow, uh, that, you know, I hadn't thought about it until you mentioned it, but there is a certain like funness to the Kubernetes community that has existed in other open source communities, but not that many. And, and it's, it's almost like, it's almost like what, what is the word? That's the, the positive version of flippant, like the tone of adventure time, right? The tone of like adventure time is not like cynical, right. But it's very like, there's people who've been around, you know, and then we're, and I've known them for years in other open source projects. And, you know, we, I've sat there and had conversations. I just having one with Clint Byram, you know, and him and I go way back, lots of open source projects. And I was like, if we could only go back and tell ourselves how to do it this way, think about how much less gray hair we would have. I would have some like, and, and that's the thing now it's like, we learned these lessons, people like Clint and I learned these lessons.

And now we see these new young people who are starting their careers. Right. Right. Right. Right. If I can start them here, you know what I'm saying? If we can save everybody, the famous, you know, getting flamed on a mailing list thing, where are we going to be in five, 10 years from now? And then like, you know, that ties into the sustainability thing. So I, you know, from a professional, you know, leave a place better than when I found it, you know, in my brain, I'm starting to think out there, you know, when I was young and getting started in this and listening to podcasts and figuring out how I was going to do stuff. You know, there's someone out there who's looking for something to do and has the skills or wants to learn the skills and open sources. We've had people who've grown up in the project. There's a, there's a kid, he started on the project when he was like 14. He had to bring his dad to the conferences because he wasn't old enough to travel by himself. Right. And then we got to know his, his dad's like part of the project now. Right. Like we, you know, he's 18 now and you know, I could tell you so many people that started earlier in their careers and things like that. And for those of us that have kind of been around for a while, it's just, I kind of feel that that gives us purpose, you know, to kind of, yeah, this is where we're going to start you off. So you don't have to go through all the things that we did and then, but you can learn the lessons. So like a lot of the stuff Kubernetes is just built on the shoulders of giants. And I think it's awesome that the early founders of the project and the early community people looked at all the things that worked in projects and kind of kind of wrapped it all together. And then you see this kind of, I used to call it the kind of Kubernetes way, but it's kind of the cloud native way, right? You can go to another cloud native project that isn't Kubernetes and related and meet the same kind of people. And they have the same kind of, you know, friendliness and openness. And when it, you know, if you're sitting in a group and a new person approaches you, you open the, you know, and you bring them in. Hey, how are you? Just these kind of little things that aren't even, they're not even, it doesn't even make the webpage. Yeah, yeah, yeah. No, no, no. I mean, all of that. Yeah, no, it is. It's, it's, it's like the,

the, that community is the most recent big community to try to evolve and improve the idea of how open source community works. Yeah. All right. Well, great. Well, well, this is, this is exactly what I was looking for now. Uh, if this is your, if you've actually watched this far, dear listener, watcher, and you don't know what's happening. This is, this is, uh, as I was explaining earlier, this is a multi-headed, uh, Tanzu talk, uh, thing. This may be a podcast. You might be seeing it right now as a live stream, or maybe you're seeing the replay on the recorded YouTube thing, but we should do either way is definitely, uh, if you're interested in like seeing more of this type of stuff, if you go to, uh, twitch.tv slash VMware Tanzu, we've got all sorts of shows going on. And, uh, you can also just subscribe to a podcast if you like Tanzu talk, but you don't get all the shows. And then, uh, as always, well, we'll see how long this goes on, but like more or less during the weekdays, I'm here in Amsterdam. So I do a daily, uh, sort of thing. Tanzu talk live, if you will, at 11 a.m. Central European time. I don't know if it's daylight savings or not. I lose track of what that even means, but, uh, you know, basically if you're in Europe and not in the Canary Islands or the UK, I think it's 11 a.m. as I recall. And so, uh, with that, thanks again, and we'll see everyone next time. Thanks for having me. Thank you.

Tanzu Talk · 🍕 Get more pizza.