Mick Pollard Explains How (and Why) Platform Teams Track Developer NPS

Episode 136 · 41:38 · SRT · archive.org · all Tanzu Talk

okay so one thing that i've noticed here with having mick on as our guest this week so i think this is at least our third australian on the podcast this year correct me if i'm wrong jeff but you know we had dr nick we had tammy nick and tammy those are the two that came to mine i'm trying to think if we've had anyone else has joe been on uh no no that's definitely there's some white space there we could fill in we've had a couple brits as well but you know i've got a special place in my heart for australians so it's you know i feel like this we're on a roll we should probably keep this going yeah the only one other one i can think of actually is another mick and he might be from new zealand i don't know and i know that could be mick friedman is he australian he's definitely that from that part of the world and i don't know if my ability to pick up accents is good enough to distinguish between the two but i think he's australian i think mick can now can this mick with a k can confirm right so i don't know mick friedman but i know i could possibly know his brother and if that's the same family then yes they are from sydney australia yeah well so that's what's great is that it's like you you're like oh i don't know him but i know his brother because it's us we're all australians and so just just like any time i've been in australia i was like oh do you know my cousin yeah i totally do right it's like she's my gp um i do have a cousin in australia who's a gp she's lovely cool so anyways we uh mick uh thank you for for joining like i have to say like mick with a k now to specify that not the other australian mick who was on but um mick tell us a little bit about your your role at pivotal um and you know obviously any of your favorite australian pastimes

that you want to throw in there i really loved your comments about your signature hats that you are always wearing um but either of those things i think are really relevant to the conversation today cool so just a little bit of background hi my name is mick i am known as aussie lunix online i moved to san francisco in january this year to join pivotal as a platform architect and platform architecture means we're a field engineer so i spend a bunch of time doing um pre-sales work but majority of my time is post-sales work helping customers be successful and my focus is on um platform engineering teams so more an ops focus than a developer focus and that comes from my background of always been working in ops so in the 90s i mean a friend we had a quote we wanted faster internet and someone else had to pay for it out of that came we built an isp so we got to surf the backbone 1997 i think it was we were a nice fast internet and someone else was paying for it until we worked out there's actually work to be done to make that thing happen so i discovered linux and became a system administrator so i started there i've stuck with the telco isp industry pretty much all my career um then naturally moved into the cloud side of things so 2015 i did a contract with the aussie government the australian government and we were a pivotal customer and that was to implement um we implemented back then it was just pcf which is people application service and the idea was to enable developers to live to deliver software at speed but with safety so that they could then showcase a more modern way of delivering software that was built for the people of australia not for governments okay so that kind of kind of put pivotal on your radar and then here we are 2019 and you actually joined pivotal while you were still in australia before you moved up here

i i had my offer my formal offer i did some contracting work with pivotal in australia whilst navigating the visa process to move over here there's um without a university degree it's a little bit of a challenge to to get your visa here it's not impossible it's just a lengthy process okay so still in that time yeah i did some contract work okay because i just remember meeting you at a pivotal kind of internal thing and your hat you know i mean you and your hat uh like gosh what was it like 20 it was 17 2018 maybe anyways yeah it was 2018 yeah so it was early 2018 so it's like oh yeah you've been you've been in with the crew for a while yeah okay that's interesting and i didn't realize you kind of came in through that um that uh that work that we had been doing with the the australian government yeah so so that time there that changed some of my perspectives of what your role is in systems administration as in ops engineering to more of i my role is no longer i'm building servers or technology i'm enabling developers and i'm doing what it takes to enable developers to write and deliver software that then enables the value for the end user that's really a pivotal moment can i use that sorry no pun intended um sorry not sorry no that's right so yeah it was it was a big change for me um in my career to to have a different focus and it's been a really good change yeah yeah no that's an interesting like you know you had this aha moment and it's sort of it's changed the course of make with a case trajectory through you know time and space literally and i'm just space-wise you know you coming to the us um and so now this kind of relates back to what you and i were talking about the other

week which i thought would be really interesting for our listeners to hear about which is how how you've used net promoter score to actually get other people to have some of the same aha moments so maybe you could kind of start that story at the beginning for folks um but i'm just going to put that teaser out there as like this is this is where the story of mick with a k and the rest of the podcast are going to come together but yeah tell us a little bit you the you don't have to name names but the client you're working with what what was going on like like i'm sure what many enterprises would recognize in their own you know environments sure so what so it took me a little bit of time to come to that realization that my role and when i say time we're talking you know maybe a decade of being engineer before it really dawned on me that hey i need to actually listen to what my developers need to so that they can be efficient at their jobs and safe and then it's not that easy though it's not something that's common in our industry around you go to conferences and tech conferences are mostly about the tech side of the role and i remember being at a container conference once and put a proposal in for a just a lightning talk and i was told it must be technology it can't be like philosophy or process or anything and i was i felt a little annoyed at that because there was a lot of talk around um it felt very resume at this conference it was like everything was about what's the latest technology and what no one was asking why like why would you want this thing in a business or a company and how does it help your developers it was just about how cool is this technology so to help so working with when i first started here in um san francisco with pivotal i was just you know i've got a few customers i work with

one of the signs i noticed was very much we're building you know for ourselves not for our developers and to take them on the journey of that mindset change i tried to think of you know um a few different things we could do and i noticed they were currently only being measured by velocity as in like agile scrum we did 40 points this week as a team next week they did 10 and then they got questioned what have you all done this week if you only delivered 10 and it didn't feel very good for the team that's like what's not visible in that was a we spent a bunch of time on you know um outages and you know reactive type work and we had a day where it was an all-hands staff meeting so therefore velocity's down so we as a team you know as a pivotal team working with that customer we said what metric could we measure them by that showed a more accurate um journey towards helping the company on their innovation journey and to help change their mindset so i want to pause you there though because you talked about okay so they're being measured on velocity velocity because they're working in these agile scrums and i mean isn't isn't agile good isn't that a good thing sure but um one size doesn't fit all if you're in an operations team agile itself it's a big word but so say scrum is very uh fixed time fixed scope and you get a lot of uh reactionary work during the week even early in your innovation journey so you'll have you know lots and lots of uh say defects or outages or you know problems support requests and and if you've got a lot of that work and it doesn't fit in it will affect that your velocity and if your management's only measuring your team by that

velocity number it's not a great indicative of a is the team delivering value and b the other thing is it can be gamed if it's the metric that's being measured all of a sudden every story is 10 points regardless whether it takes five minutes to implement or it's really hard and it takes two days it's 10 points and before you know it the team's delivering 100 points of velocity and they look awesome but the how much value have they delivered we haven't measured it we don't know so yes we started with you know we need to go on a journey to changing mindset and that was let's measure them on the let's start asking the developers hey are you happy with your you know the journey of getting your code into production and that was done so there's a legacy platform in place at the company i'm working with and the new pivotal application service platform the the net promoter score survey was it was crafted in a very simple way in that it asks a question and that's pretty much it it is are you happy with your current platform do you use pcf or legacy and why did you give would you recommend it to to another developer and why so at the end of that first survey they got a score that they promote it for those that don't know never promote a score it you're working out a score between zero i'm sorry minus and positive 100 you if you're above zero you're doing really well so a combined score that got was i think it was minus 27 which is and most of the reasons given were that there was zero communications from the platform team like we don't know who you are we don't know what you do other than there's a thing and there was a real wake-up call for them and and you know one of the comments was you know this is a bitter pill to swallow but we hear it loud and clear

that we actually need to do better and talk to our our users our customers as such so so we changed the mindset dramatically like overnight almost and and so it had the effect that we were after on its own though it's not going to be the only metric again a single metric on its own can be gained and i'm watching i'm watching some of that happen already it's like you know what are the things we can do just to get that score up yeah where's the value in for the actual company itself if you walk around you know oprah style you get this you get this and everyone's happy sorry the developers are happy does that mean value for the company that was being delivered yeah well that's interesting to see how like especially over time those things are going to get balanced but it's it's interesting to hear how this has been um a wake-up call for that team to sort of move from as you worded it like we're building this for ourselves right we're sort of the infrastructure software team but we're just you know this looks like cool tech this is what we're going to build to suddenly like oh we get it we need to build stuff for developers what what have you seen um on the positive side what have you seen happen as a result of you know taking that hard pill of okay wow that's a really negative score and how has that changed that team's you know everything right how they're they're organized what they're working on anything yeah so they've from there one of the key things that they've they're working on is that teams are starting to onboard and it's a small platform team and they've really starting to feel the pressure of having lots and lots of people trying to onboard at once and it's become very obvious that you can't

scale supporting that many people on your own with a small platform team so you need a community and the idea of net promoter score find your promoters and make use of those promoters so the idea is find the developers that like your system they understand it and they'll champion it so they become key players in your community to help scale the support so the basic you know one-on-one questions no longer come to the platform team the community can start answering some of these themselves so what there's what's happening now is there are brown bag sessions happening regularly and they're very well attended there's been a whole bunch of other community stuff happening they now have a product website that's and they've branded so they've branded their products it is a it's not often you see someone branding their infrastructure as a product it has a name name that has meaning and meaning to the area and they've got a website now that has a whole bunch of you know like faqs how do i on board where do i get extra help and there's still a lot more work to do in that space but already they've started that hey our developers didn't know who we were or what we do and we need to listen to them so now they've they've they've done a lot of work around that okay um and you know tell tell us a little bit more about just the the process of setting up and running that net promoter score because this is an area actually jeff i know has done some research in um you know the net promoter score process and it can be like you know it's it's it's interesting it's designed to be relatively easy a sort of simple high level metric that a lot of people can grok relatively easily except what goes into actually getting that number is a little bit more complex so i think folks would really benefit from hearing just how this

how this whole process of setting up um an internal net promoter score for uh an internal infrastructure you know platform team to their developers you know even going back to getting people on board with doing it at all like and then like the mechanics of how did you do it yeah okay so great topic the the net promoter score you'll we've all probably come across them in our time that we spend online it's you know at the end of a conversation or an end of a transaction you'll get a out of zero to ten how likely would you be to you know recommend this service to your friend and the idea is so zero all the way up to four is classes not likely at all you know the five and six you're it's a neutral and all that so zero to six is classes a detractor you would not promote the product seven and eight are classes passive nine and ten are classes promoters and they're extremely likely to recommend this service to others so the way we worked this out was um the questionnaire was sent out as a just a simple google form a really simple google form it was sent to all developers not just those using the pivot location service and the idea of that was to gauge a two things one we can work out the um developer happiness across the entire company so for the executives they can see that actually yeah our entire development teams are not exactly happy with the way things work or they are and that we're getting better at stuff the other one is the pivotal application uh pivotal platform team the team there can they can work out that they're improving and they're building the right things they um and that's probably the most important part so we take

these scores from each individual person then they're added up and you take i'm gonna get this right all your subtracting the percentage of detractors yeah from the percentage of promoters that's where you come up with your mps so it's going to be if it's below if it's above zero then you're doing reasonably well so don't be overwhelmed by oh my god it was minus 30 or it was only two okay and then the neutrals just you don't even you don't count those they're they're neutral they don't move the number okay yeah because the idea is they they're not that excited they're going to tell everyone the idea is net promoter score is to find out who your promoters are okay like who's going to help you scale your support and your community things like that and get adoption okay so let's say i mean just just to walk through some numbers because who doesn't love math especially when referred to verbally with absolutely no visual cues but um you know if you if you had a hundred respondents to a net promoter score then if you had uh if 30 were promoters and 10 were neutral and 60 were detractors which means they scored anything from zero all the way up to six that's how you would end up with a negative 30 net promoter score like as just an example right but you could see how it's like well 30 were nines and tens and ten percent worse sevens and eights like you know you kind of get a sense for how hard it is to really get that number close to and above zero like you have to really have an overwhelming number of people in the nines and tens category yes and it makes a difference in that if you've got a sample size of 10 000 versus 100 um yeah it's it's a little the numbers will fluctuate a little bit at with only

say 100 respondents right yeah if you get 10 people that are unhappy the next time and become detractors that reflects quite a large number of out of that 100 whereas if it's you know 10 people out of a thousand it's much smaller it doesn't really show up as much right but the idea is the nps score is an indication measured over a long period of time it's not a hey weekly thing that you're going to measure so it's probably worth talking about that so with the cadence with that the this team are using is quarterly um so once a quarter they'll send out the survey it goes out again i think in the next few weeks to get their second score so to see how they've gone in the last quarter um we'll see i'm confident they'll it will be a higher score that you know they've done a lot of work in building a community and and you know listening to developers there's still plenty more to do yeah but that's good yeah there's there's another thing you mentioned as you were describing okay so we we put together we used a simple google form and it seems like the math is actually relatively easy to just do on your own um but and you you walk through like the three questions that were asked what i thought was interesting is is clearly like one of these questions was geared towards getting an nps that was needs to be shared with executives right and it's almost like answering an ex a question that like higher level exec leader type people are having um and then other ones might be more immediate for that team so to me that's kind of interesting and very thoughtful to be like you know you've got limited real estate like the way you run these you don't want to you know ask like 15 questions if you can ask one that's great but if you can ask three like thinking about okay who are we going to be sharing these scores with in order to motivate change like who do we think needs to change behaviors we know

that our team might need to change some things so we want something that's going to be really immediate for us but we also know that we want you know the the appropriate say executive uh air cover on certain things and their buy-in on making changes so we kind of need to set up one of those questions to really be like get them motivated uh i don't know if that was how you planned it but to me like that seems like a really really good way to go in terms of like don't be greedy with your questions right and make them all like this are just what i personally want to know like you got to think about who's who are you going to be using these nps scores with is that fair yeah definitely so it wasn't something that i'd thought of up front that hey this is going to have a bigger effect it was really about the team itself so but by having it so that the questionnaire was sent to all developers not just those that were users of this new platform but to everyone and then but having it so that you could pull out the metric for either as well as a combined um had a really good effect so you can you can bubble up to yeah you know the executives that actually are the way we're working across the board is making everyone unhappy so things need to change and then there's a way of measuring that that change is increasing and that could be culture it could be all sorts of things but the other one is limiting the questions is really i think important if you get a questionnaire every three months it's got 15 questions in it it's likely to get you know marked a spam very quickly and your petition participation rate will be quite low so the idea is you want to make the barrier of getting the results really low so that your participation rate is high so having two or three questions at the most it's like which platform do you use are you happy with it and why did you give that score

yeah jeff is there anything from your kind of poking around on on mps um you know processes and whatnot that that you you've done in the past that comes to mind for you yeah well i mean this when i started doing some research on this as you know part of the marketing group i was thinking well how can we use this you know externally to market to you know potential customers you know hey here's here's a great nps score for a pivotal cloud foundry great let's attract some more customers uh but the thing that the some of the folks that i talked to when i was exploring this pointed out well it's really a great tool for your internal teams as well to get a sense of what's working what's not working uh and to help the product teams determine uh well just be well not determined but it's one input that they can use as they determine uh how they're going to develop the product new features that kind of thing and it helps them understand where they're where they're challenged and where they're doing well um but you know i totally echo your point it's it it's a simple score but there's so much that goes goes on behind it uh and and nick what you said about keeping it short that's definitely some more feedback i got from some of the folks i talked to uh you know keep it keep it short keep it concise uh and simple and you're much more likely to get get more people to to answer so um but you know i think it's you know it's it's it's deceptively simple but can be very powerful yeah and i think at the end of the day it's also the you know the the basic question of like how likely are you to recommend this to someone which is the you know you see that wording all over the place that there's a reason why that question has bubbled up in across many different industries and across many different products and services out there that people have measured as like a very telling indicator right it's just a signal but it's a it's a telling signal it's like would you actually go out and you know tell your friends

that they should go use this um it sets a really clear and high bar for you know not just like did you like it um and so it's interesting to see that starting to get applied internally because we usually think of that as you know like hey you know you just downloaded our app and you like bought some oranges on some online do whiz bang i don't know i'm i probably shouldn't call things whiz bang ever again um but yeah whatever you know it's like we think of that in a very consumer sense and thinking about it as just a way to like hey everything that you're working on is kind of like it's your product and your customers are you know not just the end customers of the company you work for but all these internal people that you're supporting and so how do you find out like how am i doing um you know we run into this all the time like internally like in product marketing it's like if we were to ever say you know we're going to stop doing this thing um you know people be like no no no we we want it we love it but then you try to look at the metrics of like oh are you using it like are you recommending it to other people and you know they're not and the other things like if you quietly take it away um and then you see who screams that's how you find out if people are really using something but that's probably like uh you know that's like a brinksmanship um um uh diplomacy style of uh finding this out where it's just like a really simple nps score might tell you like you know is this thing on track or not yeah and it helps i think it helps also that by having a measurement like this where you can get indicators that you're not keeping your developers happy still is much better than waiting until your developers are really dissatisfied and angry and don't want to come to work anymore so so you know you're starting to go oh actually we're not listening to our customers or we didn't

do the quite the right thing we need to improve it's fun it's it's much better to get that early yeah so what do you what would you recommend for folks if they you know are interested in going down this path which with whatever platform they're building internally right infrastructure folks who are you know whether they're they're pulling it together with their own mix of open source and what they've got or if they're using um you know pivotal cloud foundry or anything right i think anybody could benefit from this what would you recommend as as how they can go about it probably the first one is making sure that your developers know why you're doing things so don't just give them a surprise you know here's a survey in their inbox engage with them to say hey think we want to now listen we want to work with you and engage with you so we can help you be efficient at your job then send that survey to out to developers obviously get executive buy-in that you're doing this because you want you know everyone to understand it but yeah send it out to the send that small simple survey out and don't don't get tempted to put lots of questions in it it's very easy to be tempted by it you're like oh i could ask about this feature right and get great feedback but there's a that's for another time like this is really just are you happy with my platform or not it's it's and send that out and follow that up that's the key sending out a survey and having no response or no follow-up um to your developers is not actually it's probably worse than not sending at all oh yeah that's a good point like you know hey oh i got this great message that the team really cares about my opinion and they want to help and they're going to listen and then i got a survey and i took it and then radio silence like that is super awkward and like you're like wait a minute i thought i thought you were listening yeah followed up with action yeah so as in you know have a brown bag

session and say hey here's a new feature we're rolling out we've listened to you and we know that this is something that's been painful for you and things like that yeah show that like you know i mean do you even go so far as you'd say in this case the team had like a a negative score it was a wake-up call do they go back out with that broad-based communication of you know thank you for taking a survey this is sort of what we've immediately learned from it and some of the steps we're going to be taking uh you know in addition to kind of hey let's do a specific brown bag on that feature that we we know is something that you've asked for do you also do kind of the broad high level just like hey this was the score like do you do you share that with everybody yeah so the team did that they put together a report like a small slideshow presentation and they got up at i think it was a lunch and learn where you know it was a room full of people and they said you know they were very vulnerable they stood up and said you know we've listened here's the score here's what we need to do better and we're going to improve we'll work on this that's the first step hey we got a problem and we're going to work on it yeah and they've started on that so the next the next one will be really interesting to see the results and i look forward to the journey with them yeah like to make a really extreme parallel i've been watching that chernobyl series i haven't okay jeff have you watched this i have it's excellent it's really good it's like bizarrely beautiful um it's also like a uh feels like a sci-fi horror film except it actually happened um but there's it's really interesting seeing how you know these teams at least you know especially like the first episode is really visceral like how these teams are dealing with failure and um you know the idea of you you mentioned how you know

this team got up there and they were they allowed themselves to be vulnerable and kind of show their weakness and then you know they use that to kind of move past it and get better um yeah if you want a really visceral example of like the opposite of that and like people never wanting to show any vulnerability and and therefore making things worse uh watch the chernobyl miniseries on hbo um i'll be collecting my my promotion check from hbo for plugging that later i mean by later i mean you you know when the half-life of uh uranium 257 or whatever actually is done in 10 000 years um but anyways so okay like this is great you know uh make sure the developers know why like that makes a ton of sense don't just like surprise them like inbox hey you know like here comes an mps service like what is this fake fake mail get the exact buy-in don't get tempted to add questions follow up um with communication and action um anything else in your kind of recipe for success with this um there's a lot but staying with just like the mps and things like that then i'll say one of the key things i learned when my time with the australian government was there was a mantra that was used a lot and it was mindset then skill set then tool set so this was part of getting the mindset to being product and customer focused the idea is then you can work on skill sets next the tool sets actually last so so the idea is that because the tools come and go regularly you know we're in technology and the technology changes quite fast so it's not necessarily hey what tools can i use today if you don't have the right mindset you're going to be implementing the wrong thing right yeah so this was part of that sort of journey take them get the

mindset set we can work on the skills next and that's probably the next thing is now that they're working on you know with customers and understand they have customers the next two things i think are finding other metrics that will help tie the work back to actually delivering business value so that the mps score is an isolated score of just keeping people happy it's and the other one is then around um continual learning so becoming a more of a learning organization because the skill set the tool set sorry will change regularly and if you're not a learning organization you get left behind pretty quickly okay and that makes sense in the that that's a great like order of operations you know the mindset school set uh tool set that's uh it's catchy too so i always like that um yeah that kind of that's a clever order of operations that makes a lot of sense and then helps kind of prioritize you know what what needs to happen both first and also strategically because it's it's almost like those those plans sometimes you have that's like you know vision strategy execution uh where it's like your vision might be this kind of big picture over five years and then a strategy is like a you know two two year horizon and then the tactics are like what are we doing in the next six months yeah and that's execution but you know the execution is going to needs to be driven by like the vision otherwise you've got execution that's just for you know random acts of of execution and that applies to you know anything right marketing strategies and product strategies and it strategies so it sort of maps really well to that order of operations and thinking about you know what's the the big vision of where we need to be thinking um before we get into like the tactics of

what tools are we using um and then i the tying back to business value i think makes a lot of sense that's actually something that we're going to be talking about um a lot more on an upcoming webinar that we have with uh jeffrey hammond of forester and patrick feeney from hcsc so talking a lot about um creating metrics that matter and i know from doing prep work with them they're going to be getting into that so i'll include that in the show notes for folks are there any other resources you would recommend for folks we can always drop them in the show notes so that the links are handy so there's a really good book if you're starting out in this journey there's a great book called um project to product i think it's mick kirsten i think his name is another mick yeah i'll double check that and um so it it really helps go on that journey of hey we're doing project management you know we've got tpms everywhere and um to product it takes you on that journey so it's probably a really good book so if you're in a big project based company and you want to go on the journey then then that that is probably one of the better books that and when you get into the metrics that matter there's the staple and that's the uh the accelerate yes the science of of devops which is i think that if you're only going to read two books at the beginning of a of you know a transformation journey they're probably the two i'd start okay well that's that's a strong endorsement and i also like really short reading lists because i'm a slow reader so um wonderful where you mentioned that you're you're aussie linux on twitter where everyone can see you know your hat uh one of the two hats you mentioned yeah um and i also i always really appreciate you you know internally like on slack you have the emoji with the hat you know kind of it's like kind of a cowboy hat but it's kind of like the aussie

outback hat you can't really tell on emoji because it's emoji but you just have it embedded as part of your you know your handle so it's not you don't have to add it it's just always there it's so the hat i i so that's six seven years ago i bought 25 acres of land out of the city like it was pretty remote so yeah the nearest my nearest neighbor was a mile away my driveway was three quarters of a mile long you get some speed on that driveway no it was pretty rough there were two creek crossings like it wasn't exactly a racetrack um but part of that so that was an experiment around living off grid starting from it was a clean slate there was no building there was no electricity supply there was nothing and it was a really tough experience but a really great experience but when you live out in the land like that and it's hot it's you know 100 115 degrees um yeah you wear a hat yeah becomes your best friend pretty much so and in australia there's the cubra hat which is the one i'm wearing now it's made from a rabbit fur it's iconic aussie hat but my other hat is my winter hat it's made from case so it's a it's a um it's made from kangaroo leather a really soft but really tough leather and so it became part of an image though when i started wearing it to like meetups and conferences it just sort of stuck right now it's your personal brand and now it's recognizable i can go places and they go oh the guy with the hat we need to talk to him again i can't remember my name but yeah um it's especially when in like the 90s there's all the news about like the ozone layer like melting away and for some reason it was like the holes were over australia yeah and i think i feel like australia got really good at making sure that like all school children were wearing hats and sunscreen was like you know flowed out of taps or something like that because there isn't actually enough water in australia so

it's actually cheaper to just turn the tap on and sunscreen comes out i'm joking obviously but um you know there's just a family down there so i i have experienced the culture around sun protection partly because you know you were all going to get melted by the sun um and and it is actually quite hot in australia so i i get it um but it's wonderful thank you for sharing some of your your hat stories with us as well um it's good luck welcome i dig it all right we'll we'll have folks follow you um on the twitters and of course look out for the the man with the hat at various meetups and such thank you thank you it's been great being on

Tanzu Talk · 🍕 Get more pizza.