1 00:00:00,000 --> 00:00:05,980 well uh this week we've got two people on as a guest the four person four person uh episode here 2 00:00:05,980 --> 00:00:10,120 why don't you introduce yourself to guests you can however you want to do if you want to battle 3 00:00:10,120 --> 00:00:15,780 it out do rock paper scissors whatever figure out which order you want to go in let's see what 4 00:00:15,780 --> 00:00:22,660 happens all right so i guess i'll go first uh this is james mod the product manager for concourse 5 00:00:22,660 --> 00:00:33,060 here in toronto i work with wow murphy who like can introduce himself um the product director for 6 00:00:33,060 --> 00:00:41,120 total r&d um one of my focuses is concourse here in toronto as well and and is that uh like like 7 00:00:41,120 --> 00:00:46,460 what's what's going on up in toronto at the moment what what season is it you know like you gotta i 8 00:00:46,460 --> 00:00:51,140 mean it's obviously you got halloween and things like that but like what uh what do people do up 9 00:00:51,140 --> 00:01:00,240 there now it's like a it's like fluctuating between spring and winter yeah honestly my favorite time 10 00:01:00,240 --> 00:01:07,780 of year it's uh 10 degree 10 to 15 degrees celsius it's jacket weather um but you know it's still 11 00:01:07,780 --> 00:01:13,040 nice it's sunny the grime of the november rains haven't come down yet so you know trying to enjoy 12 00:01:13,040 --> 00:01:18,460 the outdoors while you can before the slush sets in oh yeah it's it's like it's like the ideal of how 13 00:01:18,460 --> 00:01:24,000 life is supposed to be that i got from all those john hughes movies in the 80s like like that's that 14 00:01:24,000 --> 00:01:29,520 sounds nice there weren't a lot of movies filmed down here in texas it wasn't that that gritty so i 15 00:01:29,520 --> 00:01:35,120 forget do y'all have like do y'all have like those sky bridges and tunnels and stuff up there like we 16 00:01:35,120 --> 00:01:40,980 have an extensive underground network yeah i don't know if this is true but i heard the second largest 17 00:01:40,980 --> 00:01:49,380 um indoor network of like walkway paths i think it's second to some american city so i'm not too sure 18 00:01:49,380 --> 00:01:54,400 someone can call me out on that um but yeah in the winters you can basically made it for make it from 19 00:01:54,400 --> 00:02:02,200 the main trade station to our office with only a few minutes of uh braving the outdoors so now now 20 00:02:02,200 --> 00:02:06,580 what's like what's like the social culture of those tunnels is it is it sort of like you're not supposed 21 00:02:06,580 --> 00:02:10,940 to acknowledge other people exists or do you smile at people like how does how does it work out 22 00:02:10,940 --> 00:02:13,780 like lemmings and people say sorry yeah 23 00:02:13,780 --> 00:02:21,380 it's kind of like our own highway because like a lot there's like a huge economy underground economy 24 00:02:21,380 --> 00:02:26,940 of coffee shops and convenience stores so if you want to make an exit you gotta you gotta like do 25 00:02:26,940 --> 00:02:32,360 lane lane changes do a little shoulder check make sure uh you're not you're not stepping in front of 26 00:02:32,360 --> 00:02:38,700 someone if you try to make it to timmy's you use use the uh pair of pair of pathetic blinkers as it 27 00:02:38,700 --> 00:02:44,060 were that's that's good so now now last question before we get to whatever our ostensible topics are 28 00:02:44,060 --> 00:02:48,680 like is uh are the is the coffee down there good do you have the thing where it's like good coffee 29 00:02:48,680 --> 00:02:55,520 down there or is it just like you know coffee coffee oh no we we have we're i i as an office we 30 00:02:55,520 --> 00:03:01,520 we are very proud of our coffee selection chosen probably at least two of the top five 31 00:03:01,520 --> 00:03:09,700 coffee snobs in uh in toronto i see it's arguably better than the offerings you have 32 00:03:09,700 --> 00:03:15,240 in san francisco oh that's impressive i know shots fired uh blue bottle watch it 33 00:03:15,240 --> 00:03:22,020 yeah well it's not new york blue bottle has like i feel i feel like blue bottle has podcast ads now 34 00:03:22,020 --> 00:03:28,320 so they're definitely mainstream so maybe maybe maybe it all all success and coffee it it leaves room 35 00:03:28,320 --> 00:03:32,940 for new entrants to come and try new things so i said that was the last question but now now actually 36 00:03:32,940 --> 00:03:37,840 since you've professed your coffee snobbery or intelligence so what what exactly is a piccolo what 37 00:03:37,840 --> 00:03:49,200 is that the cortado that you get to leave with a cortado that okay that's fair a lot of uh most i would 38 00:03:49,200 --> 00:03:54,100 say a lot of coffee places worth their dime there's certain drinks that you're not able to put in a 39 00:03:54,100 --> 00:04:01,480 paper cup um they reduce the flavor nodes and everything else cortado is one of those um so a lot of 40 00:04:01,480 --> 00:04:07,820 places will not let you give a cortado to go but a piccolo is a clever way to ask for a cortado to go 41 00:04:07,820 --> 00:04:16,740 yeah that's an unexpected answer i like that that's good i was just reviewing uh someone's someone's 42 00:04:16,740 --> 00:04:22,580 notes for a SpringOne platform talk that was basically like um at the management layer here's 43 00:04:22,580 --> 00:04:29,920 all sorts of like um uh hopefully devious ways to get around management stuff and i and i feel like 44 00:04:29,920 --> 00:04:33,720 that's the answer you just gave there is like i would like to take this drink out of the coffee shop 45 00:04:33,720 --> 00:04:40,000 but i have to trick the whole ecosystem into this being an okay thing to do so i'll rename it which 46 00:04:40,000 --> 00:04:44,200 which i think there's probably a lot to learn there right for doing uh the old digital transformation 47 00:04:44,200 --> 00:04:49,740 well we have uh we have some news items to go through then we'll get back to our uh our guests of course 48 00:04:49,740 --> 00:04:54,920 if y'all have any uh input beyond uh you know ways to get a cortado out of a coffee shop 49 00:04:54,920 --> 00:05:00,960 about what we talk about feel free to chime in but i think the first item i i guess uh i guess 50 00:05:00,960 --> 00:05:06,000 i saw a lot of write-ups about this all all of the uh the public cloud stuff according to the numbers 51 00:05:06,000 --> 00:05:10,260 seems to be going uh rip-roaring good i was thinking i was reading through the earning stuff 52 00:05:10,260 --> 00:05:15,600 of last week and thinking like oh i should i should have gotten a time machine and invested in those 53 00:05:15,600 --> 00:05:21,340 companies that that seems like a good idea but uh yeah the results seem pretty good i think if i 54 00:05:21,340 --> 00:05:25,500 remember wasn't there some big uh benchmark that that google was approaching there was some article 55 00:05:25,500 --> 00:05:30,180 that they're uh they're about to reach some new run rate on on their their public cloud spend 56 00:05:30,180 --> 00:05:36,080 i think it was microsoft was hitting the 20 billion dollar yeah run rate and then uh google still 57 00:05:36,080 --> 00:05:40,660 doesn't break out their cloud revenue so you're still guessing as to like other buckets so they 58 00:05:40,660 --> 00:05:45,920 growth still seems good microsoft had really good azure growth it's funny they they still bunch it in 59 00:05:45,920 --> 00:05:50,180 with some other stuff as well in some areas so you wonder is azure kind of lifting up some other 60 00:05:50,180 --> 00:05:54,500 kind of flatter server revenues and things like that but microsoft did great amazon did great 61 00:05:54,500 --> 00:06:00,740 what bezos is the richest person on the planet now again so good for him but uh you know the growth 62 00:06:00,740 --> 00:06:04,460 rates are a little different i think microsoft's growing a little faster but i think amazon is 63 00:06:04,460 --> 00:06:10,200 still five times bigger than the next four or five cloud providers combined so all of this stuff is 64 00:06:10,200 --> 00:06:15,320 scale related but most importantly everybody's making uh or the big three are making lots of money in 65 00:06:15,320 --> 00:06:22,400 cloud yeah lots of usage that that seems to be uh you know when i go gallivanting about i i think i think pretty 66 00:06:22,400 --> 00:06:28,620 solidly those are the three that come up nowadays like i was i was in uh i was in dc last week and 67 00:06:28,620 --> 00:06:33,120 there there was there was a fair amount of consideration i mean it was nice hearing about 68 00:06:33,120 --> 00:06:38,740 various agencies who actually uh want to use public cloud and are intent on it which i think is a pretty 69 00:06:38,740 --> 00:06:44,440 big change since i don't know two or three years ago where you know uh no one in in government seemed 70 00:06:44,440 --> 00:06:49,780 to want to move to public cloud and in in mass if you will but uh yep those are the three that people 71 00:06:49,780 --> 00:06:56,180 tend to cycle through so then over in microsoft land they have uh i i guess we didn't cover this 72 00:06:56,180 --> 00:07:02,100 last time but they they uh they have a uh a kubernetes service as well so everyone's got it 73 00:07:02,100 --> 00:07:07,880 that's right hot off the press yeah they did this last week uh i mean and they did the k too so 74 00:07:07,880 --> 00:07:14,040 clearly you know it's it's the azure container service with a k so that's uh that's a thing now 75 00:07:14,040 --> 00:07:19,060 so yeah everybody's doing some sort of cluster somewhere which is interesting and hopefully the 76 00:07:19,060 --> 00:07:23,680 conversation continues to go down to though what is still the right way to use these things what is 77 00:07:23,680 --> 00:07:28,620 the smart things to put inside let's just not i don't know move on to the next shiny thing and 78 00:07:28,620 --> 00:07:32,280 necessarily create ourselves a bunch of technical debt we're all going to deal with in two more 79 00:07:32,280 --> 00:07:37,780 years but hey you know clearly uh popular stuff after you and i talked last i mean that that scenario 80 00:07:37,780 --> 00:07:42,700 you described i mean surely i mean that never happens right that's that that seems like a pattern 81 00:07:42,700 --> 00:07:48,740 that us in the tech industry could easily break out of absolutely we'll be fine this time it's 82 00:07:48,740 --> 00:07:54,520 different as the t-shirt yeah exactly and then i didn't look at the uh i i saw this uh github 83 00:07:54,520 --> 00:07:59,620 contributor thing floating around but i didn't look at it so now here we are what's what's what's in it 84 00:07:59,620 --> 00:08:05,300 yeah it was some analysis of kind of which companies are contributing the most to open 85 00:08:05,300 --> 00:08:09,760 source and you know it's just kind of surprising i guess to some who may not be as aware of pivotal 86 00:08:09,760 --> 00:08:14,560 but you know and then james governor red monk also did some analysis but you know like pivotal 87 00:08:14,560 --> 00:08:19,380 contributes as much to open source as ibm and significantly more than companies like amazon and 88 00:08:19,380 --> 00:08:26,260 others so great to see you know microsoft google others leading the pack there but you know and 89 00:08:26,260 --> 00:08:30,060 the methodology is not perfect you know people could be submitting using their their personal 90 00:08:30,060 --> 00:08:34,960 accounts and it's not representing the company or this or that so it's not going to be perfect but 91 00:08:34,960 --> 00:08:40,440 there's still a clear indicator that you know companies like pivotal are investing a lot in open 92 00:08:40,440 --> 00:08:45,540 source and other companies are doing some but maybe not as much given they're they're much larger size 93 00:08:45,540 --> 00:08:51,220 so let me ask all three of you this i'm i'm a little bit uh you know as i'm fond of saying i'm good at 94 00:08:51,220 --> 00:08:56,300 slides so i'm a little bit out of the at least i like to think i am uh i'm a little bit out of the 95 00:08:56,300 --> 00:09:02,120 coding thing but like what is what how would you describe the currency of github stars like like what is 96 00:09:02,120 --> 00:09:07,340 that like how does that what what how is it a good thing or a bad thing or like what do you do with 97 00:09:07,340 --> 00:09:13,840 that i mean is it let me is it is it like as bad as a linkedin endorsement thing or is it somehow 98 00:09:13,840 --> 00:09:22,320 uh no no it's not quite that bad i mean uh stars i think are useful as like a validation of like hey 99 00:09:22,320 --> 00:09:28,380 people want to keep track of my project so i mean is it worthwhile to track your rising stars over time 100 00:09:28,380 --> 00:09:31,260 probably not uh have we done that maybe 101 00:09:31,260 --> 00:09:36,940 right you know at the end of the day it's a number and like it's up to you whether or not you want to track 102 00:09:36,940 --> 00:09:42,620 that number and uh short of some drastic like unstarring event uh we don't really pay attention 103 00:09:42,620 --> 00:09:45,420 to uh stars on a track 104 00:09:45,420 --> 00:09:51,240 an unstarring event i like that that's that sounds like something from a uh some part of like a bruce 105 00:09:51,240 --> 00:09:55,680 sterling or william gibson book that they decided to delete there's a short story about an unstarring 106 00:09:55,680 --> 00:09:58,620 event or a really bad black mirror episode 107 00:09:58,620 --> 00:10:04,960 yeah um it is a good hint though if you see something with uh not a lot of stars to unpack it 108 00:10:04,960 --> 00:10:10,460 to see like wait a second like how old or how big is this project and like it's a good cue to actually 109 00:10:10,460 --> 00:10:15,320 see if you need to dive deeper um into the project if you want to if you actually want to truly commit 110 00:10:15,320 --> 00:10:21,760 to something yeah like ryan morgan uh who had the spring for us um from measuring perspective 111 00:10:21,760 --> 00:10:28,160 has like a theory around like it's an indirect correlation but maybe not a good one in that 112 00:10:28,160 --> 00:10:34,700 it matters to a point when you're really small to make sure that's growing but what you want to 113 00:10:34,700 --> 00:10:38,800 make sure over time is like your ratio of people who are actually contributing and whether it's 114 00:10:38,800 --> 00:10:42,640 writing code or just participating and like getting issue comments and everything else 115 00:10:42,640 --> 00:10:49,240 in theory as your stars go up hopefully that grows at least the or maintains the same percentage 116 00:10:49,240 --> 00:10:55,500 um but i think it's really easy to get i think to Coté's point it's really easy to get 117 00:10:55,500 --> 00:11:02,760 kind of um tied to like reddit upvotes or it indicates something happened a lot like 118 00:11:02,760 --> 00:11:07,220 there may have been a good blog post or something that generated people to go and start that repo 119 00:11:07,220 --> 00:11:14,300 but if you're not getting that actual follow-up um and commitment back to the community then it becomes 120 00:11:14,300 --> 00:11:21,320 almost like a smell oh smell you know when i first started at red monk a while ago i wrote this paper 121 00:11:21,320 --> 00:11:27,260 called the smells of agile but you know i know what a smell is that's uh sorry i got distracted there 122 00:11:27,260 --> 00:11:33,660 was a deer in the street which is really odd uh so uh yeah that's that's good input maybe i should go 123 00:11:33,660 --> 00:11:42,640 figure out how github works do do a pull request github stars is just as relevant to the success of your 124 00:11:42,640 --> 00:11:48,740 community as velocity is to predicting what's going to happen in your agile working team oh no that's 125 00:11:48,740 --> 00:11:54,740 good that's good it it can be a pot it's likely that it's a positive indicator but you shouldn't 126 00:11:54,740 --> 00:12:01,540 assume that it is you should check in on it yeah yeah huh all right well uh you know uh 127 00:12:01,540 --> 00:12:07,660 there's also some neo4j news what's that richard i was i was seeing that and i was thinking uh you know 128 00:12:07,660 --> 00:12:13,460 kenny on my team he used to work there i should i should see if he's excited about it did he yeah 129 00:12:13,460 --> 00:12:17,780 you know they've been around a while i just you know i look for kind of interesting product news that 130 00:12:17,780 --> 00:12:22,300 isn't just the top three cloud providers and you know neo4j has been around a while as a graph 131 00:12:22,300 --> 00:12:27,100 database graph databases are cool they've kind of gotten rolled into other things sometimes you'll 132 00:12:27,100 --> 00:12:31,740 see these multimodal databases that represent graph and key value and other stuff like that but 133 00:12:31,740 --> 00:12:37,700 neo4j is alive and kicking they kind of kicked up this newer kind of graphing platform and again i just 134 00:12:37,700 --> 00:12:42,300 like seeing some of these smaller database providers whether them or mongo or others not just uh 135 00:12:42,300 --> 00:12:46,620 kind of rolling over just because oracle or microsoft or others add these capabilities to 136 00:12:46,620 --> 00:12:50,900 their products they still think they can offer best of breed and i think that's a cool example so if 137 00:12:50,900 --> 00:12:55,700 we have customers interested in things like this they should let us know to see if we should get them 138 00:12:55,700 --> 00:13:00,820 into our marketplace that's a good point and nothing sounds cooler than a graph database except maybe 139 00:13:00,820 --> 00:13:05,100 like right machine learning that yeah you're gonna be a you're gonna be hot this year at 140 00:13:05,100 --> 00:13:10,280 thanksgiving aren't you just dropping like graph database and yeah github stars you're gonna be big 141 00:13:10,280 --> 00:13:16,580 yeah you know i've been thinking i've been in in well i don't know we'll see and in such a uh i feel 142 00:13:16,580 --> 00:13:21,120 positive steady state people keep i run into them they keep asking me what's new and i have to say 143 00:13:21,120 --> 00:13:26,440 like well it's pretty much the same as last time which uh i need to work on that for my thanksgiving 144 00:13:26,440 --> 00:13:31,080 game come up with some new uh exciting stuff i can tell them i ate grub worms in china that'll be a 145 00:13:31,080 --> 00:13:34,800 good that'll blow a good 20 minutes of conversation while we're waiting for the turkey 146 00:13:34,800 --> 00:13:40,260 yeah no you'll be saw a deer today that that's a highlight there's a lot of things happening for 147 00:13:40,260 --> 00:13:45,020 you right now it's another thrilling life well speaking of a lot of things happening 148 00:13:45,020 --> 00:13:51,220 two guests so uh we we had you on i've i've been i've been excited for this topic for quite some time 149 00:13:51,220 --> 00:13:56,720 to talk about uh concourse and and how it's going and what it does and how it sort of fits into the 150 00:13:56,720 --> 00:14:01,900 overall scheme of uh uh making my life more exciting so first of all thanks for being on what 151 00:14:01,900 --> 00:14:07,360 why don't you why don't uh what one of you give us like an introduction of what uh what concourse is 152 00:14:07,360 --> 00:14:16,540 like what it gets used for yeah um thanks for having us so uh concourse is we jokingly call it the 153 00:14:16,540 --> 00:14:23,860 continuous thing doer um it's very much a cicd that was born out of pivotal's cloud native journey and 154 00:14:23,860 --> 00:14:31,320 in building out a path that was worthy of like sitting on uh enterprises and in multi in a multi 155 00:14:31,320 --> 00:14:38,320 cloud environment and first and foremost um you know it it comes with a lot of unique innovations 156 00:14:38,320 --> 00:14:44,600 that make continuous integration delivery in an enterprise and in a cloud native environment 157 00:14:44,600 --> 00:14:50,120 that much better and that much easier and that much more powerful and it comes down to three kind 158 00:14:50,120 --> 00:14:56,860 of core tenants the declarative configuration so no faffing around with a ui um everything you need 159 00:14:56,860 --> 00:15:03,600 is defined in your pipeline um and that's codified in your github repository or whatever your code 160 00:15:03,600 --> 00:15:09,360 management system of choice and that's a lifesaver right so if your system goes down there's no oh i 161 00:15:09,360 --> 00:15:13,740 got to remember to install that plug-in where's the license file for this everything's in a single 162 00:15:13,740 --> 00:15:19,760 place where you put your code you set your pipeline and off you go the other big innovation with concourse 163 00:15:19,760 --> 00:15:27,200 has been um it's reliance on containerization for consistent execution so uh a lot of engineers can 164 00:15:27,200 --> 00:15:34,540 relate to you know messing around with configurations and flaky builds and uh trying to get their pipelines 165 00:15:34,540 --> 00:15:39,380 cleaned up after an execution you don't have to deal with a lot of that with concourse we built it from 166 00:15:39,380 --> 00:15:45,980 the ground up uh with that kind of uh stability and consistency in mind and we we take a lot of 167 00:15:45,980 --> 00:15:51,720 advantage of containerization to do that and finally uh because we don't rely so much on the ui to 168 00:15:51,720 --> 00:15:59,760 configure pipelines we say we make a lot more use of the ui to uh be an information radiator so every time 169 00:15:59,760 --> 00:16:05,980 you walk around a pivotal labs office every tv you see has that concourse pipeline emitting key 170 00:16:05,980 --> 00:16:11,580 information about the status of a project how it's performing and you know is there any fires that 171 00:16:11,580 --> 00:16:18,660 you need to stop drop and roll for so so i think in in that context and i don't really i'm kind of 172 00:16:18,660 --> 00:16:26,180 fishing for something here but like it feels like uh builds and and build pipelines are historically 173 00:16:26,180 --> 00:16:32,220 fraught with terribleness right like like you spend a lot of effort on them and and the results are always 174 00:16:32,220 --> 00:16:37,440 weird and things and they're i mean part of why because it's a extremely critical part of the whole 175 00:16:37,440 --> 00:16:43,660 uh process i mean it's it's it's good to have your software run by building it and testing it and so 176 00:16:43,660 --> 00:16:49,800 and you you kind of hit upon this but i mean are like what about containerizing things kind of like 177 00:16:49,800 --> 00:16:57,020 addresses some of that fraughtness like how does that how does that help out beyond like uh using vms or 178 00:16:57,020 --> 00:17:03,220 using really like you know strict puppet and chef things or whatever like how does a container improve 179 00:17:03,220 --> 00:17:09,620 the uh the difficulties yeah so containers are a pretty neat innovation because they're just so 180 00:17:09,620 --> 00:17:16,360 lightweight and easy to generate create and destroy so with containers what we do is say like hey this 181 00:17:16,360 --> 00:17:22,900 thing that is traditionally flaky it's flaky a lot of times because um you have to make sure the things 182 00:17:22,900 --> 00:17:27,720 that you need to run your tests are there and also you have to make sure that cleans up properly after 183 00:17:27,720 --> 00:17:35,460 itself by specifying containers we can say hey we've pre-baked a certain amount like the things that you 184 00:17:35,460 --> 00:17:42,800 require for a test so you can just grab a docker image that has go and the dependencies that you need 185 00:17:42,800 --> 00:17:48,060 pull that down from the cloud know for a fact that your test could run with the dependencies there 186 00:17:48,060 --> 00:17:54,120 and then um once you're done we just nuke it we just destroy it and then every time we want it we can 187 00:17:54,120 --> 00:18:01,460 create a new one so because of that um life cycle it becomes a lot cleaner a lot more consistent and a lot 188 00:18:01,460 --> 00:18:07,920 more reliable makes sense you get this benefit of a lot of these really powerful awesome tools that are 189 00:18:07,920 --> 00:18:15,500 in like the entire like devops um and app ops space is this idea of like things like you mentioned 190 00:18:15,500 --> 00:18:22,420 like ansible chef bosch um terraform which is a really cool one um you get all these really powerful 191 00:18:22,420 --> 00:18:27,540 tools that end up needing their own kind of like dead man switch type of thing which is like go up and 192 00:18:27,540 --> 00:18:32,900 kind of clean up after themselves and like they're independently they're all very very powerful tools 193 00:18:32,900 --> 00:18:38,480 to do one thing very well um and and unfortunately or i guess fortunately for us in concourse 194 00:18:38,480 --> 00:18:42,820 these things don't need to be used one after another in certain some sort of predictable order 195 00:18:42,820 --> 00:18:49,440 and we get that kind of that layer of obfuscation in which this can all happen now obviously you still 196 00:18:49,440 --> 00:18:55,940 have the problem of a dead man switch for concourse but uh one area of failure is considerably better 197 00:18:55,940 --> 00:19:03,620 than say six or seven right right so you have sort of like the lightweightness which is also so so here 198 00:19:03,620 --> 00:19:08,460 you have the lightweightness of containers to hear it seems like tell me if i'm wrong but a fair amount of 199 00:19:08,460 --> 00:19:15,540 it translates to if you can do everything really fast then um it's better in in the sense of bringing 200 00:19:15,540 --> 00:19:21,080 something up and bringing it down and versus like if it would take several minutes even to like bring 201 00:19:21,080 --> 00:19:27,040 up a part of like i don't know a 50 part system as as that as a failure happens there something screwy 202 00:19:27,040 --> 00:19:31,580 it just adds in a time whereas if you have a much faster cycle you might still have errors but you 203 00:19:31,580 --> 00:19:36,220 discover them really quickly and you can fix them and and all that stuff plus maybe it's just more 204 00:19:36,220 --> 00:19:42,120 reliable to build build up container things and then you add to that to like all the various things for 205 00:19:42,120 --> 00:19:50,280 like a cloud native e which is to say a distributed application uh running all the stuff that that you 206 00:19:50,280 --> 00:19:56,520 need to bring a cluster like that up and coordinate everything there's lots of good little uh best of 207 00:19:56,520 --> 00:20:02,240 breed applications in their own silos but having a way to orchestrate all that stuff not to mention 208 00:20:02,240 --> 00:20:06,200 having them clean up after themselves i mean it sounds like they're like my four-year-old daughter 209 00:20:06,200 --> 00:20:10,740 and son who can clean up after themselves but you gotta have to have something that's orchestrating 210 00:20:10,740 --> 00:20:17,140 uh all of those various steps in building out your your dev and your staging and then eventually your 211 00:20:17,140 --> 00:20:23,720 production cloud exactly and i think it's important to highlight the fact that at pivotal we believe in 212 00:20:23,720 --> 00:20:32,440 in tdd like wholeheartedly right we don't have a qa department we codify all of our experiences our 213 00:20:32,440 --> 00:20:39,640 testing and our acceptance into our suite of tests and now our pipelines and uh the containerization 214 00:20:39,640 --> 00:20:47,160 aspect allows us to do that because we run into a lot of issues where well given a piece of code how do i test that it runs on 215 00:20:47,160 --> 00:20:55,640 as well in azure gcp um aws vsphere and you know all these other combinations in a consistent way 216 00:20:55,640 --> 00:21:03,640 using a single vm it's just it's just very frustrating to do so tapping into containerization with uh cheap 217 00:21:03,640 --> 00:21:10,200 setups and teardowns makes it so much easier and so much more reliable i guess it can be like simply 218 00:21:10,200 --> 00:21:17,160 summarized like um adding consistently consistency to to things that inherently are inconsistent 219 00:21:19,240 --> 00:21:24,280 yeah that's a nice way of thinking about it so lyle you told me a while back a little bit of the uh 220 00:21:24,280 --> 00:21:28,920 origin story of some of this i mean you mentioned like james i mean there's a great ecosystem out 221 00:21:28,920 --> 00:21:35,320 there of devops the app ops tooling so why another one like why why did we build something that's now 222 00:21:35,320 --> 00:21:39,960 open source and gotten fairly popular so revisionist history what a great move but at the 223 00:21:39,960 --> 00:21:47,160 time like where did this come from why did we go down this path so james actually spends most of 224 00:21:47,160 --> 00:21:52,840 his days it's not all of his days with alex rachi who was one of the initial um it's like 225 00:21:54,280 --> 00:22:00,920 brainchild of uh of concourse so i think he can better answer that one yeah so um it didn't start 226 00:22:00,920 --> 00:22:07,160 off as this grand idea of hey we need to build a new ci cd system right it came out of us genuinely 227 00:22:07,160 --> 00:22:14,360 want to wanting to apply our td principles into this very massive and interesting problem of building 228 00:22:14,360 --> 00:22:20,280 this platform as a service known now known as uh pivotal cloud foundry and over time we ran into 229 00:22:20,280 --> 00:22:26,760 those issues that we talked about around how do we make um this kind of pipeline of tests and build 230 00:22:26,760 --> 00:22:34,360 instructions to run consistently automatically and against all these uh permutations and combinations and 231 00:22:34,360 --> 00:22:40,600 i think we tried for a very very long time to work with the tools that we had and we basically ran into 232 00:22:40,600 --> 00:22:45,560 a wall where you know it's like we we got to try something else we need we're seeing all these 233 00:22:45,560 --> 00:22:52,120 technologies come around containerization in fact we're using it and incorporating it into cloud foundry 234 00:22:52,120 --> 00:22:57,800 it's just silly that to not take advantage of those learnings and that knowledge to create a better 235 00:22:57,800 --> 00:23:06,440 system so alex and chris brown kind of worked on it on the side uh as a side project um over time and 236 00:23:06,440 --> 00:23:12,440 it wasn't again it's not like it's easy to think now like hey it's awesome it's it's super obvious to 237 00:23:12,440 --> 00:23:19,320 use and pivots love it um they they had the task of creating a system to win over uh pivots and that's 238 00:23:19,320 --> 00:23:25,240 not an easy task uh we really like our tools and we really like our workflows and so for them to 239 00:23:25,880 --> 00:23:32,440 build this tool and evolve it and to get um to convince more and more folks to adopt it and use it and 240 00:23:32,440 --> 00:23:39,800 understand it and um over time it just became such a powerful tool that it has become an official 241 00:23:39,800 --> 00:23:46,200 project within pivotal that um is basically used in every corner of our offices and i i don't know 242 00:23:46,200 --> 00:23:53,640 what we would do without it um we rely on it for automated testing pulling in security patches stem cell 243 00:23:53,640 --> 00:23:59,960 updates i mean you name it it's probably codified into a concourse pipeline somewhere that's pretty 244 00:23:59,960 --> 00:24:04,440 awesome i know i love even uh in seattle we've got concourse pipelines up on the walls from our 245 00:24:04,440 --> 00:24:10,440 our pivotal labs teams so you mentioned you know some of the different scenarios that concourse gets 246 00:24:10,440 --> 00:24:14,520 used stem cell updates this and that so i mean i think that again when i would chat with with you 247 00:24:14,520 --> 00:24:20,200 guys as we were getting down this path of productizing concourse a bit for our customers and 248 00:24:20,200 --> 00:24:24,760 it sounds like we learned some things that maybe they weren't using it for apps as much as we 249 00:24:24,760 --> 00:24:28,600 thought they were actually using it differently themselves so can you tell us a little bit about 250 00:24:28,600 --> 00:24:34,360 concourse for pcf and kind of this journey that made us think about ci cd not for just apps but for 251 00:24:34,360 --> 00:24:41,960 platforms like where in the world did that come from yeah so um it came out of um i guess the the big 252 00:24:41,960 --> 00:24:50,280 moment was in december and january of i guess 2016 2017 when the concourse top beta went out and we 253 00:24:50,280 --> 00:24:57,160 started getting this influx of issues from operators which really surprised us because at that point we had 254 00:24:57,160 --> 00:25:03,240 the context of like hey our app developers love doing this um we are sure that enterprise or our pcf customers 255 00:25:03,240 --> 00:25:08,280 would love using it too and that's still the case but it turns out that um there's a set of people who 256 00:25:08,280 --> 00:25:16,280 love it even more than our pcf uh app developers and that was the operators and they found it as an 257 00:25:16,280 --> 00:25:24,120 incredible tool for uh consistently and reliably setting up uh pcf foundations installing tiles and applying 258 00:25:24,120 --> 00:25:31,400 updates automatically from pivnet um because they too want to be uh up to date with security patches 259 00:25:31,400 --> 00:25:36,840 and all the good things that we consistently push out through our pivnet and other distribution channels 260 00:25:36,840 --> 00:25:44,600 and they uh found that concourse was one of the best ways to do that and uh shout out to customer zero who 261 00:25:44,600 --> 00:25:52,120 you know experimented and kind of distributed those first pipelines that let operators do that um it's because of 262 00:25:52,120 --> 00:25:58,600 them that it really surfaced this uh incredible use case from the devops side that we basically saw 263 00:25:58,600 --> 00:26:04,360 as this is something that's really going to take off and we really need to cater towards these folks 264 00:26:04,360 --> 00:26:10,120 because at the end of the day if the devops person signs off on concourse you can be sure that the 265 00:26:10,120 --> 00:26:15,880 application developer is going to have no problem getting authorization and encouragement to use that tool 266 00:26:15,880 --> 00:26:22,840 once they see the power from on high of that artifact promotion process yeah it's it's an interesting 267 00:26:22,840 --> 00:26:30,440 continuation of um i don't know whatever to to retcon it whatever the sre devopsy uh soup has been in 268 00:26:30,440 --> 00:26:36,520 recent years like the idea that your your developers and your operations people should probably have a huge 269 00:26:36,520 --> 00:26:43,800 amount of overlap in the tools that they use which uh certainly not been the case in the past but in in that as 270 00:26:43,800 --> 00:26:48,520 as as as i think you know you're saying your your application developers would would use this tool 271 00:26:48,520 --> 00:26:53,960 and then your operations people would use the tool as well for their own platform so to speak the 272 00:26:53,960 --> 00:26:59,480 production that they run it's it's uh it seems like it seems like it's useful to consolidate down to uh 273 00:26:59,480 --> 00:27:05,640 the same tool and the method across across that divide well it's this theme of consistency right 274 00:27:05,640 --> 00:27:11,240 that keeps coming up which is like in an area that inherently forever for many many decades has been 275 00:27:11,240 --> 00:27:15,160 inconsistent which is like managing kind of infrastructure and everything else like we're 276 00:27:15,160 --> 00:27:19,960 now in this world where infrastructure has code and everything else is now a a thing what everyone 277 00:27:19,960 --> 00:27:25,960 wants to do um like infrastructure as code isn't a new idea it was now it's an idea that everyone can 278 00:27:25,960 --> 00:27:36,040 actually do um and like the great thing about that is like the biggest areas that either operators have um 279 00:27:36,920 --> 00:27:43,080 in terms of managing what gets to happen on their internal platform and what app developers need to 280 00:27:43,080 --> 00:27:49,880 go through in terms like the governance of mainly like pushing your app to production um and and so much 281 00:27:49,880 --> 00:27:57,560 happened before someone or something gets to type cf push into a command line and the nice thing that 282 00:27:57,560 --> 00:28:03,080 we're noticing with concourse being used to kind of continuously consistently do anything for you 283 00:28:03,800 --> 00:28:11,560 is that it gives this awesome layer to like the cso individual and auditors and governance people within 284 00:28:12,120 --> 00:28:18,280 these enterprise companies to say if you're using these pipelines or these set of pipelines we know what's 285 00:28:18,280 --> 00:28:25,560 happening we know it's consistent and if it's not consistent it fails and that causes to fix something and it helps these 286 00:28:25,560 --> 00:28:32,600 customers get past this these kind of internal check boxes and approvals they've made for themselves 287 00:28:32,600 --> 00:28:39,720 because they had to over the past many many years um and we can start optimizing and automating that 288 00:28:39,720 --> 00:28:46,600 during production which faster feedback loops more else being pushed and it means our customers get to 289 00:28:46,600 --> 00:28:52,440 make better applications faster for their customers yeah but that ops change has to be fascinating right of 290 00:28:52,440 --> 00:28:57,320 actually using tooling like this to update the base platform so like i haven't seen too many places 291 00:28:57,320 --> 00:29:01,880 where you'd be updating you know the database platform or you know even some some of those 292 00:29:01,880 --> 00:29:06,200 components in this sort of rolling fashion so did you get much pushback when you first started talking 293 00:29:06,200 --> 00:29:11,320 about this or were ops folks eager for a way to say look you're updating os's you know maybe a dozen 294 00:29:11,320 --> 00:29:15,400 times a month i'd love to not patch this stuff manually especially without taking a 295 00:29:16,120 --> 00:29:20,760 maintenance window what was the kind of reaction when you first started approaching this i think that james's 296 00:29:20,760 --> 00:29:25,640 point and he can correct me if i'm wrong but so we have a team internal team customer zero which is a 297 00:29:26,200 --> 00:29:31,960 marriage between our platform services team so the ones who do the really hard work helping our 298 00:29:31,960 --> 00:29:38,200 customers get their platform stood up um from absolutely nothing um and to be honest i actually don't think 299 00:29:38,200 --> 00:29:46,520 r and d saw this uh to what jane was saying we actually initially were making a tile for concourse which 300 00:29:46,520 --> 00:29:53,480 is a thing like an app that's installed within your deployed foundation um the reason we chose not to 301 00:29:53,480 --> 00:30:01,000 do a tile was um the management plane that you provide yourself by installing a concourse on a specific 302 00:30:01,000 --> 00:30:07,240 ops manager you can no longer control the management plan your many many foundations and it was actually 303 00:30:07,240 --> 00:30:11,720 that team that just by working really close with our customers and seeing the problems they're trying 304 00:30:11,720 --> 00:30:17,240 to solve ended up doing this and we kind of just learned from them and adapted what that meant over 305 00:30:17,240 --> 00:30:26,440 time um which ultimately meant the first i guess quote unquote service thing from pivotal that was not a 306 00:30:26,440 --> 00:30:32,760 tile but supported as a bosh release yeah and i think it's a it was a combination of uh the operators 307 00:30:32,760 --> 00:30:37,880 screaming hey this is great we really love this possibility of automation and 308 00:30:37,880 --> 00:30:43,640 me being able to have weekends and not worry about the state of my platform but on the other hand 309 00:30:43,640 --> 00:30:48,920 it's looking at the realities of what we were making and saying like hey this doesn't really help me 310 00:30:48,920 --> 00:30:55,560 if it's a tile and a foundation how does that help me operate against my other foundations 311 00:30:55,560 --> 00:31:01,560 so um it was this kind of conversation that we started with customers and our field team and 312 00:31:01,560 --> 00:31:08,520 customer zero around okay what's the right thing to do here and uh the right thing that we saw was 313 00:31:08,520 --> 00:31:16,200 okay let's go back to basics let's focus on um meeting some of the operator needs for uh concourse 314 00:31:16,200 --> 00:31:22,280 as a product that controls foundations uh from a security and stability perspective really kind of hardening 315 00:31:22,280 --> 00:31:28,840 that up and then putting a version of concourse out there that we could kind of put our put our full 316 00:31:28,840 --> 00:31:34,680 support behind and say hey use this to um manage your foundations and we're really going to support you 317 00:31:34,680 --> 00:31:43,160 as you go on that journey towards being a fully automated self-updating platform thing lots of thingies 318 00:31:43,160 --> 00:31:49,560 involved as always continuously so so speaking of thingies could can you give us an example of 319 00:31:49,560 --> 00:31:55,720 like for an application like uh so if i had a thingy that i wanted to uh to to build and deploy 320 00:31:55,720 --> 00:32:00,040 and run through the whole pipeline like what is i mean obviously it's all verbal so you can't have 321 00:32:00,040 --> 00:32:06,200 any visuals but like uh like what do i do what does that look like i mean i'm assuming uh that there's 322 00:32:06,200 --> 00:32:12,120 some yaml involved right like i mean yeah so so what what are the artifacts and what do i do and and 323 00:32:12,120 --> 00:32:19,000 what's the process like and how's yaml just yaml all the way down um so it starts off with uh what we 324 00:32:19,000 --> 00:32:23,720 call like a three box pipeline right so it starts off with a single with a yaml file that you describe 325 00:32:23,720 --> 00:32:31,320 your your build your test and your deploy actions and and so even that is kind of very powerful in 326 00:32:31,320 --> 00:32:37,640 the concourse world where you know you have to hook up your pipeline in your yaml file to uh get 327 00:32:37,640 --> 00:32:44,360 repositories so hey instead of explicitly executing modes you say just watch this repository every time 328 00:32:44,360 --> 00:32:49,880 we do a change just test it for me because you know we might not remember to to execute those changes 329 00:32:49,880 --> 00:32:56,040 every time we commit so just watch this repository and feed it through these three stages so that 330 00:32:56,040 --> 00:33:03,400 that's like the most basic concourse 101 get up and running um you are now continuously testing and 331 00:33:03,400 --> 00:33:10,040 delivering things and then you can start adding in your complexity and automation to capture more use 332 00:33:10,040 --> 00:33:18,280 cases maybe your product um needs to go against a security checker that your team um your devops team 333 00:33:18,280 --> 00:33:25,160 forces you to go through just to be compliant so for example um we have osl tooling right internally 334 00:33:25,160 --> 00:33:29,960 and and that's to check for open source licenses that to make sure we're using them correctly so 335 00:33:29,960 --> 00:33:36,600 you can add another box that says hey um maybe you need to put your code in a certain location or signal 336 00:33:36,600 --> 00:33:44,040 another uh team to say pull in my code um we we need to do a check and once that passes then proceed 337 00:33:44,040 --> 00:33:52,200 to deploy and and so the the basic elements of concourse go way beyond just hey take this piece of code 338 00:33:52,200 --> 00:33:59,160 and test it it's actually um extended to what we call resources so everything's a resource we have um 339 00:33:59,160 --> 00:34:06,920 things like hey watch this um bit bucket or watch this s3 repository every time a file changes um do 340 00:34:06,920 --> 00:34:15,400 something with it so um there's a there's a very uh basic but powerful language behind concourse that 341 00:34:15,400 --> 00:34:23,480 allows you to really add complexity to to relatively basic uh pipelines and and then and then so so let 342 00:34:23,480 --> 00:34:29,400 me see if if i can rephrase it so you have uh you have some triggering events right like something 343 00:34:29,400 --> 00:34:36,600 changes and that means you need to like uh recheck everything essentially and and uh you have those 344 00:34:36,600 --> 00:34:43,400 resources of like it's you know whether it's just like a uh like a git repo or or maybe even just some 345 00:34:43,400 --> 00:34:47,640 storage somewhere and something's on there and you sort of pull that in and then i assume there's 346 00:34:47,640 --> 00:34:53,000 some sort of um like from my understanding like sort of so then you'll have like a container and i 347 00:34:53,000 --> 00:34:59,800 assume it gives it some configuration kind of like 12 factory style and and then it just like executes 348 00:34:59,800 --> 00:35:05,480 whatever that box is and then i guess i assume it has a good or a bad result right and then maybe it 349 00:35:05,480 --> 00:35:11,000 goes to the next part of the workflow to do something else uh is that is that right so far 350 00:35:11,800 --> 00:35:20,120 exactly it can be as simple as that just um and that's the that's the convenience factor of concourse 351 00:35:20,120 --> 00:35:25,400 where you just need to specify the right docker file right and docker has such a large community 352 00:35:25,400 --> 00:35:31,400 around it where if you need something it probably already exists and if you don't it's relatively easy 353 00:35:31,400 --> 00:35:34,760 to just roll your own even if you don't need something it probably yeah 354 00:35:37,000 --> 00:35:40,120 yeah i think a really good way to simplify it code you're going towards there which is like 355 00:35:41,160 --> 00:35:47,400 um if you have like a state machine like really what ends up happening is whether you're a developer 356 00:35:47,400 --> 00:35:54,040 um platform engineer or anything like normally those people know the inputs that they want to have 357 00:35:54,600 --> 00:35:58,680 um and what's going to trigger like that black box that state machine to happen 358 00:35:59,240 --> 00:36:04,200 um and all they really know is the ultimate output they want so more simply put if i'm an app developer 359 00:36:04,200 --> 00:36:11,080 i know i need to put my bits um onto something like a like a github or a bit bucket or whatever else 360 00:36:11,080 --> 00:36:15,560 and a bunch of things are going to happen that i may or may not care about depending on what it is 361 00:36:15,560 --> 00:36:20,840 but the eventual output is my app is in the hands of users um and like the strength of concourse and all 362 00:36:20,840 --> 00:36:25,240 its resources is really being able to do all that stuff in the middle um 363 00:36:26,440 --> 00:36:31,000 for however enterprise like grow and adopt and change uh one of the things you brought up 364 00:36:31,560 --> 00:36:39,160 for before was the question around um not just the easy solutions but where customers are going with 365 00:36:39,160 --> 00:36:46,840 the more difficult um and harder challenges and we're seeing as more customers adopt microservices um which 366 00:36:46,840 --> 00:36:53,400 is great with that becomes a complexity of maintaining the architecture of your arch container deployment 367 00:36:54,200 --> 00:37:00,280 and you have these apps of apps so you have the the final application that say we all use for say 368 00:37:00,280 --> 00:37:04,600 on amazon.com or else it's actually made up of hundreds of separate different applications 369 00:37:04,600 --> 00:37:08,920 um and we need to manage like which one of them are deployed in which order and how does it get 370 00:37:08,920 --> 00:37:14,280 there and if this one goes down what's the mean for this one um and the strength that we're 371 00:37:14,280 --> 00:37:21,000 starting to uncover the concourse is what other partners and people can we find in the space to help 372 00:37:21,000 --> 00:37:26,840 us all these problems together like as your as you build these very complex applications you get the 373 00:37:26,840 --> 00:37:32,600 need of release management which is like oh like maybe your entire pipeline takes 18 hours to finish 374 00:37:33,320 --> 00:37:39,080 so you have to figure out what kind of deployment topology do i want to use like rolling or canary 375 00:37:39,080 --> 00:37:44,760 or even like rollback roll forward type of thing um when you're trying to stage these apps of apps 376 00:37:44,760 --> 00:37:50,520 and you get to the challenge that james bring up in terms of like policy management so if my three vps 377 00:37:50,520 --> 00:37:55,560 need to approve this thing before see a push can finally happen how do we take that off the hands of 378 00:37:55,560 --> 00:37:59,800 the of the app developer and all they know is i don't know it goes in the black box and eventually 379 00:37:59,800 --> 00:38:05,560 get cf pushed um and find a way to actually make that happen consistently right right and that that 380 00:38:05,560 --> 00:38:12,680 brings back the aspect of like uh it's it's it's not it's not at all just like a development stage 381 00:38:12,680 --> 00:38:18,200 thing like it reaches all the intent is for it to reach all the way to production as as i mean i guess 382 00:38:18,200 --> 00:38:23,400 as you would want the most highfalutin definition of a pipeline to be right it's not right uh it's not 383 00:38:23,400 --> 00:38:28,440 the wagil fall stuff of like uh we can make our golden images as fast as you want but it takes three 384 00:38:28,440 --> 00:38:34,840 months to deploy them right and and so so i'm assuming uh you know what i should do is just 385 00:38:34,840 --> 00:38:40,120 go read the documentation but as i like to say here we are uh i'm assuming there's like a set of 386 00:38:40,120 --> 00:38:45,720 common services that that are available in the black box right like here is the uh request the 387 00:38:45,720 --> 00:38:51,400 configuration that you want and here's like here's how you pass some kind of state between each of the 388 00:38:51,400 --> 00:38:55,480 boxes in there and things like that i mean is that is that accurate that there's some like common 389 00:38:55,480 --> 00:39:03,640 support services in there so we we do bake in a lot of uh resources with concourse and that's mostly 390 00:39:04,760 --> 00:39:11,640 to make internally out of the box because a lot of folks don't have access they run their concourse 391 00:39:11,640 --> 00:39:17,800 in a firewall environment so they can't really happen to occur as freely as possible um so one of the 392 00:39:17,800 --> 00:39:23,880 things we had to do before we could become concourse for pcf was to give folks the ability to 393 00:39:25,080 --> 00:39:33,560 pre-bake uh docker images and have them referenced uh offline with access to docker hub um so so that's 394 00:39:33,560 --> 00:39:38,520 one aspect of it and another aspect that we're really trying to improve on is kind of the day one 395 00:39:38,520 --> 00:39:44,360 experience of hey like we hear a lot about these three box pipelines or we hear a lot about these 396 00:39:44,360 --> 00:39:52,120 integration points with um these typical services and stories like can you give me some uh reference 397 00:39:52,120 --> 00:39:58,840 points and some pointers and examples and now short of paying us on slack we don't really have a scalable 398 00:39:58,840 --> 00:40:05,400 consistent way of uh socializing uh that information so one of the first steps we've really taken is to 399 00:40:05,400 --> 00:40:12,520 commit to a concourse engineering blog that really tries to dive into some of the core mechanics of 400 00:40:12,520 --> 00:40:18,280 concourse and the typical patterns that we've ended up recommending and seeing uh both around the office and 401 00:40:18,280 --> 00:40:23,000 within the community all right now that makes sense it it reminds me like i always figured there's uh 402 00:40:23,000 --> 00:40:28,600 probably thousands of applications that are the uh the the the sort of descendants of the spring music 403 00:40:28,600 --> 00:40:33,400 and the the old j2e pet shop app right like you need some template to start with i don't know what 404 00:40:33,400 --> 00:40:38,120 they have in dot net world wasn't it what was the database name richard like wind river or something 405 00:40:38,120 --> 00:40:44,120 a good that's close north wind north wind there you go yeah yeah i mean would you uh so i mean with that 406 00:40:44,120 --> 00:40:48,040 what do you think of the spring cloud pipeline stuff i was reading the new docs again this 407 00:40:48,040 --> 00:40:52,360 morning and that seems like some nice canonical examples opinionated but of how you would take a 408 00:40:52,360 --> 00:40:57,080 concourse pipeline and take an app through the various stages and smoke tests and you know blue 409 00:40:57,080 --> 00:41:02,760 green sort of stuff has that helped you think humanize some of the more complex aspects of a deployment 410 00:41:02,760 --> 00:41:09,000 pipeline yeah absolutely and we've worked closely with the author and i feel bad i can't remember his name 411 00:41:09,000 --> 00:41:13,160 or not i think it's marston yeah marston but we can't say the last name good luck with that one yeah 412 00:41:13,160 --> 00:41:21,480 there's like no there's like four k's and a couple of b's but marston in the spring team yeah so we we've 413 00:41:21,480 --> 00:41:28,440 had conversations with him and we we uh well it first started as like a collaboration of hey uh congress 414 00:41:28,440 --> 00:41:33,560 team we've made these pipelines what do you think how can we improve um so it started off that way and 415 00:41:33,560 --> 00:41:39,400 then it ended up being something that i've relied on consistently to as a recommendation to a lot of 416 00:41:39,400 --> 00:41:44,440 our spring users to say hey if you're interested in concourse and really changing how you want to do 417 00:41:44,440 --> 00:41:49,240 continuous integration delivery that's one of the pipelines i sent to them and i think one of the 418 00:41:49,240 --> 00:41:55,000 great things that uh they did with the pipeline was to actually do a jenkins version in a concourse 419 00:41:55,000 --> 00:42:00,760 version so a lot of folks get to see side by side what the what the differences are and and kind of like 420 00:42:00,760 --> 00:42:06,040 understand at least the general concepts of continuous integration and delivery um and then 421 00:42:06,040 --> 00:42:14,120 let them kind of decide what they need to do um on their journey right to see you know are they ready 422 00:42:14,120 --> 00:42:21,320 for that level of automation and and really kind of understand what we think a full-blown pipeline 423 00:42:21,320 --> 00:42:26,760 might look like and then they can choose uh what to incorporate and that probably ties into the power 424 00:42:26,760 --> 00:42:31,800 of conquest right because it's just a bunch of code it's readable it's something that they can 425 00:42:32,680 --> 00:42:41,160 like add and remove from it freely modify yeah like we benefit by having um what might be one of the 426 00:42:41,160 --> 00:42:47,400 most complicated app teams that would be at like an enterprise uh customer site which is the spring team 427 00:42:48,040 --> 00:42:55,160 seeing all the problems they're tackling and having to face by trying to constantly package up um a very 428 00:42:55,960 --> 00:43:02,360 quickly growing constantly growing um like spring framework and spring applications um 429 00:43:03,560 --> 00:43:08,680 we've been working that really closely i know skipper which is a tool that was recently just uh 430 00:43:09,320 --> 00:43:16,200 like baited or launched by mark pollock in the spring team um it's helping spring solve their release 431 00:43:16,200 --> 00:43:23,160 management problems so in this in this challenge where i have this app it's made up of 12 other apps 432 00:43:23,160 --> 00:43:28,760 i need app one two three and four to be up running and healthy before you can do four five and six and 433 00:43:28,760 --> 00:43:34,360 then if like six rolls back how do i like recover and everything else so um i know something we're 434 00:43:34,360 --> 00:43:39,800 looking at there is a resource for concourse to make sure that you can use skipper um and as we 435 00:43:39,800 --> 00:43:46,440 continue to find these very complicated um application architectures making sure we're exploring uh different 436 00:43:46,440 --> 00:43:50,920 partnerships um there's a few we're exploring right now policy management with some like the larger 437 00:43:51,480 --> 00:43:55,560 business process management tools that our customers are using and these will just kind of 438 00:43:55,560 --> 00:44:00,760 strengthen the ecosystem of concourse and and everything else um and making sure that we're 439 00:44:00,760 --> 00:44:05,640 doing it in a way that we can also partner with others who and leverage like they're very powerful 440 00:44:05,640 --> 00:44:12,680 tools yeah now that's great so i've got one last question for you so as you look at this uh both of 441 00:44:12,680 --> 00:44:17,080 you kind of keep looking at this landscape and looking at how people and what our customers are trying 442 00:44:17,080 --> 00:44:22,760 to do continuously do you see any things emerging i mean specifically i'm thinking about those sort 443 00:44:22,760 --> 00:44:26,840 of things you mentioned skipper and we think about even spring cloud data flow and these more 444 00:44:26,840 --> 00:44:32,920 non-traditional assets it's not just an application code and a binary it's kind of a system of sorts 445 00:44:32,920 --> 00:44:37,080 so as we think about databases as you think about these other components do you see those really 446 00:44:37,080 --> 00:44:42,520 getting roped more into a cicd pipeline or tools going to have to adjust what are you thinking about 447 00:44:42,520 --> 00:44:49,320 the uh the next couple years of cicd on the first thing yeah so i see two aspects one is kind of the 448 00:44:49,320 --> 00:44:55,240 payload what you mentioned is what what are people really interested in continuously deploying and 449 00:44:56,520 --> 00:45:03,320 a lot of it is this idea of you know um folks are adopting this microservice strategy uh there are 450 00:45:03,320 --> 00:45:10,600 apps of apps how how do it make it easier for folks to track and redeploy and really create tools and 451 00:45:10,600 --> 00:45:15,880 techniques to help with that automation and the second thing that we're really excited about is how to 452 00:45:15,880 --> 00:45:25,000 incorporate um more um kind of information radiation around uh workflow efficiency so more analytics 453 00:45:25,000 --> 00:45:30,600 and data around your performance of the pipeline how well you've been doing how consistent is your 454 00:45:30,600 --> 00:45:36,680 execution um not not in flakes but more uh how are you trending over time are you doing better are you 455 00:45:36,680 --> 00:45:42,760 doing worse and i think those are kind of the tools that will really help people understand the value and 456 00:45:42,760 --> 00:45:49,960 reap the rewards of um of cicd to really understand now that i've all this automation in place maybe i can 457 00:45:49,960 --> 00:45:57,800 start uh figure out where to improve my processes and where to find bottlenecks remove them and uh improve 458 00:45:58,360 --> 00:46:04,440 the overall kind of experience yeah that would be awesome if you had some sort of uh i don't know 459 00:46:05,080 --> 00:46:10,760 hybrid like software meatware value stream map that was always uh updated i guess it'd be hard to 460 00:46:10,760 --> 00:46:17,560 bring in like the meat wary stuff at some point it would be nice to have have have that end-to-end view 461 00:46:17,560 --> 00:46:23,800 of everything and i mean going all the way back to our uh github stars thing right like have have a good uh a good 462 00:46:23,800 --> 00:46:30,680 heuristic if you will that was telling you if things were uh good or bad or somewhere where you might 463 00:46:30,680 --> 00:46:36,440 need to look and do investigate some more yeah that that'd be fun stuff all right well thanks for being 464 00:46:36,440 --> 00:46:43,240 on this episode this is good um you'll have to send us a um uh a link to the uh the the sort of like 465 00:46:43,240 --> 00:46:48,120 side by side of of a jenkins versus a concourse thing because i think that would be very interesting to 466 00:46:48,120 --> 00:46:54,040 see um but uh if people want to uh check out more concourse stuff where should they uh go look 467 00:46:55,720 --> 00:47:02,120 uh concourse.ci and our slack channel oh yeah and how about y'all self where do y'all scurry around 468 00:47:02,120 --> 00:47:11,480 on the internet um i'm at pie over pie on twitter i'm at what is it freak style on twitter i should 469 00:47:11,480 --> 00:47:17,720 probably redo that one f-r-e-k-s-t-i-l-e oh you know it's it's pretty good this so what what is 470 00:47:17,720 --> 00:47:24,680 pie over pie zero or is that some magic math thing uh it's a bit of a it's a bit of a joke uh from my 471 00:47:24,680 --> 00:47:29,880 university days where i was trying to answer some smart computer theory question and i answered with 472 00:47:29,880 --> 00:47:36,200 pie over pie but we all know that's just zero yeah or a delicious dessert that uh you know you can 473 00:47:36,200 --> 00:47:41,720 double up on yeah yeah you know what you maybe you put a pumpkin pie over a cobbler and see what you 474 00:47:41,720 --> 00:47:46,840 get with that that seems like a good good solution all right well well as always this has been pivotal 475 00:47:46,840 --> 00:47:52,280 conversations if you want to get the latest episodes and browse our back catalog if you're looking for a 476 00:47:52,280 --> 00:47:58,360 way to spend uh the winter time in the tunnels enjoying your takeaway cortada uh you can go to 477 00:47:58,360 --> 00:48:04,280 soundcloud.com slash Pivotal Conversations and you know obviously you should just subscribe to this 478 00:48:04,280 --> 00:48:08,200 whether you listen to it or not we don't really care it's just nice that you download it it would be 479 00:48:08,200 --> 00:48:13,960 good to listen to it i guess uh and if if uh you know if if you're interested in more of this stuff 480 00:48:13,960 --> 00:48:18,120 you can check out the show notes we post them more or less about every thursday if you go to 481 00:48:18,120 --> 00:48:24,360 pivotal.io slash podcast and with that we'll see everyone next time bye bye bye