Well, hey everyone, this is Coté. For this episode of Tanzu Talk, I've got an interview that Alex Williams of the Newstack and I did back at VMware Explore. It wasn't that long ago as I sit here recording this. Anyways, it's with Solomon Hikes about Dagger.io, the new company that he and some of his, I don't know, past co-workers are working on, basically to work on pipelines or builds or continuous integration, continuous deployment, or secure software supply chains, as he mentions later. Anyways, he had just given a demo of a new way to use Dagger, basically opening it up to other programming languages. And then he came over to talk with Alex and I, and we talked more broadly about the idea of what Dagger is doing, a programmable pipeline, platform engineering. And as I was eager to get to kind of an idea of how Dagger wants you to think in the directed acyclical graph, as I recall. I'm no mathematics or geometry person, so I don't know if I use those words correctly. But, you know, there's a lot going on in the pipeline, the supply chain, the build space. We have a lot going on in VMware Tanzu land with various open source things like Cartographer and other stuff. And Dagger is figuring things out and many other companies. I think it's an area that's going to be pretty exciting for the next couple of years as people try to get to that programmable pipeline that we end up talking with, with Solomon. Well, before we get to that, if you're interested in these kinds of things, whether it's application development, DevOps, you fancy yourself a platform engineer, or you don't really fancy it, you are one, anything that you're doing there, including up to the executive and enterprise architect layer. If you're in a setup where you're just trying to get better at software, you should check out the conference we have coming up SpringOne, December 6th and 8th. This is our annual conference from the pivotal days. Now the VMware Tanzu days, all about programming from Java to.net, other languages, operating cloud native environments from passes to Kubernetes, things like the Tanzu application platform and the Tanzu Kubernetes for operators. And of course,
2:05the Tanzu application service case studies and tips and things like that to agile software development. Anyways, you should check it out. Go to SpringOne.io. I think in a week or two, we're going to have the full talk schedule posted. And when you register, which I'm sure you will, because it'll be great. You can use the code Coté 200, that's C O T E 200 to get $200 off. So go to SpringOne.io, check out what we have to offer there. Maybe look at some past talks, but I think it'll be well worth your time to go there to San Francisco, December 6th and 8th. And of course, use the code Coté 200 to get $200 off. Now with that, we jump in to the interview when you know what, I kind of forget what I was asking Solomon. So I'm just going to leave that in. And you can hear the question that I go through. You know, what's fun about this interview is you can hear some kind of long pauses and how Alex and I come up with questions to ask, especially me, where I just kind of like, I have this technique when I listen to myself where I just kind of like make mouth noises and babble about stuff as I'm thinking of the question to ask, which I don't know, it's kind of annoying to listen to, I guess, but that's how it works. Enjoy the interview. Well, you know, when, when, when we've been talking about this, uh, we were talking last week about it and to, to, to modify like a discussion that we had a little bit, it was, it was, well, to some, to add some context, like, uh, I don't know, maybe, I don't know if it's fair to say, but like y'all are trying to make the pipeline more programmable, kind of apply, like, application development ideas to, to a pipeline. And I wonder being here at VMware Explorer, I should make sure to say it the right way. Like, how would you explain what y'all are up to, to like an operations person who's like more, maybe not so steeped in the idea of like, how would we make something programmable? But how do you, how do you kind of think it layers onto their mindset? Yeah, I think, is this, is this a good, is this, okay. Yeah, I, we're, that's definitely what we're doing. Um, I mean, I, I think, I mean, there's a, there's a big change going on and, and, you know,
4:07that the, the, the, the work of shipping software is getting so complicated that it needs to be automated. The, the bar is rising, you know, it needs to be automated in a more and more sophisticated way. Yeah. To the point where a script here and there is no longer enough. You need to think of your supply chain, the, the supply chain of your software project as a software project of its own. So, you know, it's the code that ships your code, it's the robot that builds the robot, that kind of stuff. So, I mean, if you're involved in the operations of that supply chain in any way, whether it's, you know, all the way from, I guess, the, the racking the servers part, I think everyone kind of is on board with, okay, that's automated. But, you know, the next layer on top of that, um, it's got to be fully automated from the, the code is ready to go. Next version of the code is ready to, okay, it's online, everything else, it needs to be automated as much as possible. So, a lot of, you know, there's a whole community, I, you know, let's call it the DevOps community, the people helping developers ship more productively, that community is transforming into a developer community of its own. Yeah, yeah, it's kind of, kind of like the, uh, I was just discussing this, but there's a, kind of a lack of a phrase for that at the moment. Like, I mean, the, the most recent thing that's kind of, you know, you hear with platform engineer a lot and people kind of using that term for it. Yeah, that's definitely part of it. Yeah, yeah. And so, so exactly who would own is always a weird word, but be in charge of that, that tool change is, uh, emerging as they say. Yeah. I mean, in our experience, I mean, we like, we are drawn to pain. That's a weird thing to say, but, like, you're basically, our, as a team, we just like to find painful problems and solve them. Right. Within our area of, uh. Have a useful solution. Right. And so, that involves talking to people a lot and listening to their problems. Otherwise, you, you, you're, you're just randomly building stuff. Right, right. And so, one thing we ask is about that, where, okay, who's trying to do this and where are they in the org?
6:12Like, how is it organized? And it's all over the place. Titles change. Titles are different. Org charts are different. Um, but there's always, I mean, it starts, there's a moment in the life cycle of a project, you know, that, where one person has to be dedicated full time to just helping everyone else ship. Right. And, you know, the title could be anything, really anything, you know, DevOps engineer, CICD, uh, sometimes it's just, you're the lead and that's what you do, but you do it full time. Um, so the label on the hat is, you know, varies, but the hat is, is, is always there as soon as you reach a point where your project is growing. Right. It's, it's like there's often the accidental build person. Like, they, they just happen to be the one that they started doing the build. You say yes one time and then you get pulled into it. Right. Uh, but eventually, you know, at the end of the road, I mean, you, you, you, you reach a point where you have this dilemma that we were talking about where we got scripts and that's what we need now, but we need a platform, you know, sometime next year. And there's a, there's a huge gap and we, you know, it's really hard to bridge that gap and it usually fails, you know, sometimes the failure gets hidden, you know, but, but it's usually, it's, it's really painful. So, yeah. You know, I was thinking about kind of the terms you use, like incremental revolution. And I also think about what I'm reminded of back from your earliest days at Docker, where, what I really loved about Docker in its beginnings was how it served as a way to almost embody your philosophy about programming the internet. Like there's this capability that we all have and there was a certain amount of, I think, minimalism in it too. And I think, I think of Kelsey out here who's here to talk, who's a real minimalist himself. And I'm curious how you're trying to embody that in what we're seeing today and what you're building. Yeah, it's, it's, it's a design, I think it's a design, it's, it's a design problem.
8:16You know, I mean, we like, we like to make things really simple to use and get to the essence of it, but it's really hard. Personally, I'm attracted to it because it seems so effortless once it, once it's done, but the process of getting there is just so painful. And each increment, yeah, each increment of simplicity, of extra simplicity, you know, it just gets, it's, you know, it's like the, the, the, in physics, you know, the closer you get to the, the speed of light, the more energy you have to spend to get just a little bit closer. It's the same thing, you know, like the closer you get to the, just the perfectly simple design where there's everything you need and nothing else, the, the, the harder it gets to advance. So that's, that's, we went through that process with Docker with some success, but with also, you know, some, some, we stumbled along the way. And so we're, we're trying to, we're like, we're starting over basically. So that's fun. Interesting. And so you're starting over in some respects. So what did you learn then that you're thinking about developing in this incremental manner? So I could see how their incremental revolution kind of concept comes by like trying to do things more and more simple, right? Trying to like, and there's a pain in that, right? Revolutions can be very painful no matter if it's, you know, happens, you know, at its peaks and valleys, wherever that revolution is. So, yeah, um, well, I think the one, one thing is that what did we learn, right? Uh, in that process? Yeah. Well, one thing that we, we kind of had our sights on with in the early days with Docker, but, um, I think didn't realize all the way. Uh, well, we ended up following another path, which is fine, but I think there's something really magical about platforms. Um, if you can build them right, and I think the, the, the term is really overused, obviously, when I think of a platform, I think of something that you write code for, like an operating system is a platform, you know what I mean?
10:21Yeah, and, and, and, and so I think to solve this particular problem, you need to sort of somehow come up with a new platform so that this community of people we were talking about that are, that are trying to automate the supply chain, you know, DevOps, that community, they actually have a platform that they can write software for. And it's actually a, a proper platform, you know, you can re, which means mostly you can reuse each other's work and trust that it's going to, trust that you can reuse each other's work safely. Um, and, and we don't really have that. I mean, we tried with Docker, you know, with Docker images, and then later with Docker compose files, and it's kind of in the uncanny valley where it's almost soft, it's almost code. And so the thing running it is almost a platform, but just not, it's not quite. Yeah, I, it, it, it, it piques my curiosity because here at the show, we're hearing a lot about platforms and platform engineering and my colleague here, Michael Coté, knows a lot about this topic on platform engineering. I know he spent some time on this, but a lot of it comes down to all those tools, right? And then that, and then that whole tool chain, but a lot of it is process, isn't it? And a lot of it is workflow. And, and so how do you, how do you like settle, you know, how do you settle process and workflow, you know, with, you know, with platforms? Because if you don't have that process and workflow, it's just a bag of tools. Yeah. I mean, the, I think all those things like workflows, run books, whatever it is that you have to do to, to ship, as long as it's not software that you can hand to someone and say, run this, then you're not done basically. And I think that's what's, what's happening right now, you know, like the, the whole software is eating the world. Like right now software is eating the supply chain. So those workflows, those policies, you know, they're going to be code and that code is going to have to run somewhere.
12:21And right now writing that code and getting it to run somewhere and everywhere it needs to run and collaborating on it is, it's just a mess. You know, it's like the early days of, you know, I don't know, the, the web platform or early days of personal computing. You know, if you're, if you're writing that software now, you're like a pioneer, you got to glue stuff together and you got to know 50 technologies inside out and you got to be an absolute expert. But, you know, hopefully five years, 10 years from now, let's say things move slower than I used to think they do. Hopefully that won't be the case. You won't need to be an expert in all these crazy tools and technologies to write, you know, to say, here's our, here's our supply chain, you know, here's our workflow for this part of our compliance problem or whatever it is, you know. So to pull together a few things there, I mean, we were talking about making your pipelines programmable. Yes. And, and kind of like one of the things you were saying is a platform is a place you put code and, and like what, what y'all are kind of putting together here is putting code into a pipeline. So that sort of makes a pipeline, a platform to, to some extent. And so to that end, like, I mean, anytime a platform emerges, there's kind of, I always think of, there's two things. There's one, like to anthropomorphize it, like the platform wants you to think some way or it's easier to use if you think some way, like, like the model on top of it. And then also, even though it's extremely early, like you kind of find some best practices and worst practices along the way. So I don't know. I mean, I'm curious, like what, how, how does this platform want you to think about a build and like, like what's the model in the head and the behavior? Yeah, yeah, that's, that's a really good point because I think, I mean, there's a cost to a new platform, you know, you, oh, okay, I got to learn this new API and I got to write code for it, you know. So I think there's a sweet spot where to be useful, like to add value, it's, you know, that it's got to bring some sort of new opinion on how you're supposed to do things.
14:23You know, oh, you got to structure, just structure it this way and you'll see, it'll be great and we'll help you. But at the same time, it's got to, it's got to be, there's got to be a path to doing that. And so, like if you're too prescriptive and then it restricts the flexibility that you need in a given area, then you're just going to be, you're going to be less adopted because it's harder to use it. So the sweet spot here is that if your software supply chain is itself becoming software and we're talking about, okay, what's the platform to run that? Well, everyone's software supply chain is different, but everyone's software supply chain kind of looks the same. It's a graph, you know, you got nodes and you got lines between them. Specifically, it's a DAG, you know, it's a graph where everything goes in a specific direction and there's no cycles. That's what a DAG is. That's what we're called DAG, by the way. So how are you managing that underlying graph? Well, so that's where we bake in an opinion. If you use the, you know, the Cloak API or the DAG API to encode your supply chain and software, you know, your pipelines are going to be nodes in a graph. They're all interconnected in a graph. We have that opinion. And we say, if you're encoding your supply chain software, organize it like a graph, and here is a platform for doing that. And so that's where, that's the trick for adding value. We can have an opinion about how your supply chain is going to be shaped because we know it's going to be a graph. Therefore, we can narrow down the focus of our platform on just that problem. So it doesn't, then what you're going to do is just swap out the nodes in your graph. You know, oh, I build with this tool. Well, my keys are in hash core vault. Well, I host on Kubernetes on this thing, whatever. You know, you just swap it out. You know, you're just reorganizing your graph, but it's still a graph. And if under the hood, the API lets you just reuse someone else's components into your graph and then, or take a whole area of your graph and wrap it in a box and say, take all of that.
16:27You know, that's one node in your mega graph. Then you have potentially an amazing level of collaboration and innovation, which is what's missing. So we just, what's missing is like a platform that's actually capable of doing that in code and it actually works. You know, it's, so it's a design, again, it's an engineering design problem. The way Lego is an engineering and design problem, you know, it boils down to the individual brick. You know, you got to engineer in a way that it's simple enough to snap together, but it actually like holds. You know, you can stack it up and you can break records in height with your super tower or something and it still somehow works. That's an amazing feat of engineering. That's, that's the hard part of what we're doing. You know, saying, okay, you know, we're obsessing over tiny little details so that you can just stack up, up and up and up and up and up. And one day run, you know, a giant supply chain on over it and it, and it just, it works and you can trust it. Right. So it seems like, I mean, there's at very, at least, I'm sure there's more like two ways of thinking that are going on here. One is that you should think of, I mean, whether you want to call it a build or a pipeline or a supply chain, it's always fun to come up with a phrase that the most fundamental thing you need to do is think of it as, as a graph full of nodes. And if, if your pipeline doesn't think that way, that's what it should do, right? Right. And then the second thing is basically like, and then you need to think about it as not so much configurable or scriptable, but programmable. Right. And, and you'll be coming in from, with a programmer's mindset. And then that brings you being able to reuse and share and swap things in and out, which is, again, much different than I think the majority of pipelines are thought of and going through. Yeah. What's interesting is just finding a way to ease into it. We talk a lot about scripts because that's where, that's what exists today. So being able to embed something more powerful inside an existing script and kind of give you a way to gradually adopt this programming model on your own terms, you, you need that.
18:33Otherwise, you know, it's really hard to adopt. Transition is difficult. Yeah. Yeah. That's why, I mean, that's why Docker files work so well, because they met you where, they didn't force you to, to become a better human being, to, to be worthy of using the tool. Just the tool, just, you know, it's, it, you do it this way. Okay, cool. Write this Docker file. It works. I just got the sign that we got to wrap up, but maybe in conclusion, you can talk to us about how you're looking to build out the community platform. I know you built it initially on Q, the programming language. What else are you doing now to help build it up? So basically this prototype we showed, you know, we, we haven't launched it yet. It's not ready, but you know, it's, we're, we're adding the support for doing this in any language. And Q is one of those languages, but it's going to be one of many, and we're going to open all of it. And hopefully you'll see people reusing each other's components through this common API across any language. So you get an element of cross-language collaboration and code reuse that's very exciting. We'll have to learn more about that another time. All right. Well, thanks. Yeah, it was fun talking to you. Thanks so much. Yeah. And thanks for the demo. You did a good job. Thanks for the motions. Thanks. Well, thanks for listening. If you're interested in the show notes for that, you can always go to tanzutalk.com and find them listed there. There's a video version of this interview. If you're interested, which also includes a demo of those new features and dagger. If you go to tanzutalk.com, you can find that video interview as well. And as a reminder, you should check out the conference. We have SpringOne coming up December 6th and 8th in San Francisco. It's all about software development, platform engineering, Kubernetes operations. And also, if you fancy this kind of thing, digital transformation, the sort of management and executive and business value and outcomes focus things. But if you go to SpringOne.io, you can use the code COTE200 to register and get $200 off. Because, you know, why not? And you can use that $200 to, I don't know, have a little party or something or just have a nice meal.
20:37If you end up going, I only know a few restaurants nowadays, but I'll recommend a place where you could spend much less than $200 and have a good meal. And with that, we'll see you next time. Bye-bye. Bye-bye.