Welcome to a special episode of the Cloud Native in 15 Minutes podcast, where over the extended holiday season, we'll be sharing some of the highlights from the show's first six months. I'm Derek Harris from Pivotal, and if you follow the show on the Intersect site, that's pivotal.io slash intersect, you know I put together a blog post for every episode where I pull together some of the highlights in a text-based format. Well, this is like the audio-based format of that, except spanning several episodes. And while the first episode focused on highlights regarding cloud-native technologies and architectures, this episode focuses on some of the highlights regarding the business considerations of going cloud-native and modernizing your IT practices overall. We talk about SRE and AI and open source and enterprise startups with some of the smartest people in those spaces. And we're going to get to that in just a minute. But before that, a quick reminder that if you enjoy the podcast, please do share it with your friends or colleagues or on social media and rate it in the app or platform of your choice. Apple Podcasts is particularly helpful. So yes, here's the show. First, we have Andrew Wing, one of the godfathers of deep learning, a Stanford professor, serial entrepreneur, co-founder of Coursera, you name it, talking about the best way to get started with AI projects to wit by starting small and picking an easy win. I've seen a lot more companies fail by starting too big than by starting too small. So I recommend to most companies to start a few pilot projects, deliver a quick win in six months, and then use that to ratchet up to bigger and bigger projects. Even Google, when I was starting out the Google brain team, there was a lot of skepticism about deep learning at that time, both within Google as well as in the wider world, much as today, a lot of companies may still be uncertain how to use AI. So my first internal customer of the Google brain team was Google's speech recognition team.
2:00And speech recognition was not, maybe still is not Google's most important AI application. It's not web search or advertising. But by delivering a quick win there, it helped other teams within Google gain more fame than what my team and I could do. And then we got our second customer, which was to help Google Maps use AI to read house numbers, to more accurately geolocate houses and buildings on Google Maps. And only after delivering this second win to a second internal customer, did I then go on and start the more serious conversation with the advertising team. So I think the lesson to a lot of companies is start small and try to pick a project that can allow you to deliver a quick win because that often helps the organization learn how to do AI and opens the door to then doing bigger projects over time. And here's Andrew explaining why it's important for executives and other decision makers to have an understanding of what AI can and cannot do when they're trying to scope initial projects and figure out how it might be helpful to their organizations. Scoping the right AI projects is really hard. And there are a few gotchas on identifying the right thing to work on. One of them is just unrealistic expectations about what AI can and cannot do. For example, a few years ago, a lot of people thought chatbots could soon have fully general purpose conversations and talk about almost anything with anyone. That turned out not to be true. And companies that tried to build fully general purpose chatbots just did not succeed because the technology was not there. So I think it's important for executives to have enough of an understanding of what AI can do to scope the valuable and feasible projects. And then also to have the appropriate mechanisms for staffing them, tracking them, and helping set up a machine learning team for success. And finally, here is Andrew giving a quick and dirty explanation of deep learning, how it works and how it's useful. If you've been following AI over the past several years, you've no doubt heard this term. But if you think it's some sort of magic, Andrew is here to explain that it absolutely is not.
4:06Deep learning is maybe one of the hottest technologies in AI right now. And what we've found over the last several years is this is a wonderful technique for doing supervised learning. So, for example, if you're running a factory and you want an AI to look at the smartphones going down your manufacturing line and figure out if it is a scratch or not, you know, to do automatic defect inspection, then it turns out that deep learning is a very effective way to learn that input-output mapping, where the input would be, say, a picture of a smartphone, and the output will be, does the smartphone have a scratch on it or not? And a lot of recent rise of AI and supervised learning is driven by discoveries in deep learning, allowing us to make them go very accurate input-to-output mapping. And here we have Dave Rensen, a Senior Director of Engineering at Google and one of the company's original proponents and pundits of SRE, a.k.a. Site Reliability Engineering, explaining kind of the general principle behind Site Reliability Engineering, which is to reduce human toil. The way I like to think of it is this. You can live in one of two worlds. In the first world, a machine called a pager wakes you up at 3 in the morning because some other machine is having a hard time. In that world, you work for the computers. The world you want to live in is one where some system you're responsible for is having a problem. It sort of mitigates itself and then it writes a bunch of information out for you to debug the next morning after your morning coffee. That's a world where the machines work for you. SRE is about getting the machines to work for you. It's about performing all the operational things we have to do to keep our users happy, but getting the computers to perform as many of them as humanly possible. And that's the difference between sort of staring at a monitor and poking at a keyboard versus, you know, trying to write software or implement systems that fix themselves. SRE is often positioned as an alternative to DevOps, but here's Dave explaining why he doesn't think the two are mutually exclusive at all and in fact are about 99% the same thing.
6:09You know, this is just the silliest argument in the history. Well, maybe not in the history of tech, but certainly in modern tech. Okay. So here is the truth. SRE and DevOps develops mostly independently of one another, mostly at the same time in response to exactly the same set of problems. So unsurprisingly, they landed in really similar spaces. They share, you know, 99% of the same principles. So I don't, you know, I don't like to argue about like chronology and history and all that stuff. The way I like to think of it mentally is that SRE is a concrete opinionated, and it certainly is, implementation of DevOps principles. The thing I like about SRE is if you do SRE work at Google and then go work and then go to, I don't know, LinkedIn or Netflix or some other place with an SRE culture, you know, the, the activities will rhyme with one another. Like you will recognize the things. And here is Dave explaining one of these sometimes counterintuitive aspects of SRE, which is that if you're doing too well on your error budgets, that is, if you're, if you budget for X amount of downtime and significantly outperform that expectation, you may have over-engineered, which is not a good thing. There are actually two bad ways to blow an error budget. Obviously you can spend too much, meaning, you know, let's say, well, let's just use some math. Let's say that you have a system that you need where your SLO is 99.9% reliable, whatever reliable means in this context. It means you can have 43.2 bad minutes, or that's your error budget, every 30 days, right? So your system can be unreliable, however, we choose to measure that, for 43 minutes every 30 days and still meet its three nines target. Okay, what happens though when you blow that, when you're, you know, 50 minutes unreliable or 90 minutes or something in some 30-day period? Okay, that's an error budget policy, right? And there are two bad things that can happen. You can spend too much of the error budget.
8:10That's easy. You can also consistently spend too little. So if you determine that you can tolerate 43 minutes of error or bad minutes over 30 days, and you're consistently only spending, say, 10 minutes, that's not a victory. That means you've over-engineered reliability. You have more reliability than your users need, and that's time and resource and expense you could be applying to innovation or risk or some other thing. And so the question is, what do you do in that context? You know, how do you approach those things? And that's where you get to philosophical differences, you know, between companies and teams. You know, I, on my teams, I like to keep a pretty simple error budget policy. If we blow our error budget, I want a feature freeze, right? No new features. We are going to spend the time it takes to earn back our good minutes doing reliability-related things. But not every company has that or every team has that as a default policy. So that's the kind of a thing that might be a little different going from company to company. And finally, here is Dave Rensen explaining the importance of measuring the right things if you're going to undertake an SRE practice. That is specifically measuring the impact on users and the things that directly impact users, less so than measuring things such as CPU load or memory utilization. No one's users care about CPU load or memory pressure. Or disk fullness, right? You know, they don't care about that stuff. They care about how long did the thing I want to take? And did I get the correct answer? You know, or did the server throw an error or some other thing? You want to measure the things your users carry about. Carry, excuse me, care about. We like to say, you know, the important things are the symptoms, not the causes. I mean, the causes are important because you need the data to be able to debug and fix the thing. But your users care about the symptoms. And so those are the things not only that you want to measure, but you want to learn on. Steven O'Grady, the co-founder and principal analyst at RedMonk, writes a lot about open source and talks a lot about open source.
10:12And here he is explaining why, you know, open source is never really free these days. You're going to be paying for it one way or another, whether that's in paying for software or paying to develop that stuff in-house. Open source is not free as in beer, right? And that has been a sort of, much less so these days, but certainly years ago, people would sort of look at this and say, hey, I can sort of go get this off the shelf, you know, pick a category operating system database, whatever, and say, hey, I can use this for free. Well, at the end of the day, most businesses are not in the business of maintaining or developing software, and therefore they need help with that, right? And so at some point or another, you're going to be paying for it, even if you're not paying for it in the form of an upfront license, you know, sort of as you're used to. So, yeah, in other words, that's, you know, that's the most sort of important change. Now, as I mentioned before, you know, with open source, you are, you can theoretically enjoy lesser costs in the sense that in many cases, there are multiple suppliers versus something like a Windows, you know, for example, where you can only get that from Microsoft. And even in cases where, you know, there is, you know, I mentioned MySQL earlier, which was primarily developed by MySQL, other people can pick up and support that code, right? So if you don't like the support or service that you're getting from MySQL, theoretically, you could go to some third party, you know, supplier or Percona or somebody else. So, you know, that has a, you know, sort of has the ability to, you know, sort of somewhat decrease your costs. But, you know, at the end of the day, you know, like I said, the most important sort of takeaways is that this isn't, you know, this isn't free. You're going to have to pay for software to be developed and software to be supported and serviced over time, one way or another. One of the big considerations in IT today is whether to buy from a major cloud provider or to buy from maybe a startup or smaller company focused on a specific technology. And here is Stephen O'Grady again, this time talking about how to do the math on that equation, how to figure out when to buy from whom.
12:12Typically, what you find is, is that, you know, the company, you know, taking Elastic, you know, they are going to be in many, you know, they're the ones who primarily develop the software. They employ the vast majority of the engineers in the planet qualified to develop the software. So, for sure, there's going to be a, you know, sort of a technical distinction, right, in terms of sort of what they're able to offer. You know, so, you know, depending on what you need, you know, that may be sort of the right fit. In other cases, you know, it may be a case of, all right, well, the 20% or whatever the sort of actual technical delta is, you know, it's not super important to me. I just have some basic needs, in which case I'll go to a mega cloud provider, right? I'll go to one of the hyperscale folks. And, yes, this is not going to be bleeding edge, and I may lack features A, B, and C, but I don't care about those, right? So, again, it really comes down to what are you trying to do and how strategic, I guess, is the asset in question? And if the asset, you know, in question is strategic, then, you know, it's pretty reasonable at least to consider the people developing that asset as your first stop, you know, versus folks that are more in the business of, you know, essentially, hey, I run this and sort of then maintain this at scale. You know, those are meaningfully different things. And finally, we have Stephen O'Grady weighing in on the importance of contributing back to open source projects, both in terms of helping advance the projects, helping understand your understanding of the technology, and oftentimes to help your hiring and retention efforts. If you're an organization of sufficient size, you know, that you run some open source piece of infrastructure, and you have a patch, you have a fix, you have a, you know, sort of whatever, something, you know, to potentially contribute back to the project, you know, essentially, you have two options, right? Which is, you know, try to contribute that back upstream to the project, and have that become part of the core, or essentially, applying your unique patch every time a new release has come out. And in many cases, that'll void your support, your warranty, right? And that doesn't seem like a terribly complicated choice, you know, to most people. So that's why we're seeing sort of more, more thoughtful policies about contributing back to projects.
14:18And it can be difficult in some circumstances, because, you know, what you hear from, from enterprises is, I don't want to subsidize the development of, you know, you know, this vendor's product, and so on. But we're, we're sort of gradually getting people to the, to the point where it's like, look, this isn't, you're not paying to develop it, you're paying not to have to repatch this thing over and over, right? So, you know, progress, I think, is being made there. Launching their own open source projects, that is a more complicated question. It can be a very, very good thing. You know, Capital One has done this with, you know, things like Hygeia, and so on. It's harder, simply because, you know, basically, launching a project, you know, takes a fair amount of time and effort. And that may or may not be of, you know, sort of benefit, you know, to the parent organization. But in some cases, you know, if the, if the software itself is valuable, and, you know, sort of meaningfully, you know, received by the market, you can find, you know, homes, you know, in the form of foundations that will take on that code for you. And therefore, you get to offset some of the legal trademark IP validation, et cetera, overhead, you know, of maintaining these projects. So there are benefits. That's a, that's a more complicated question, though, than just sort of contributing fixes and patches back. And you can also recruit, you know, sort of, in many cases, better talent. Because, you know, if, you know, like, if I'm a talented engineer, and I can pick between two options, you know, one in which I write a whole bunch of code, nobody ever sees it. Or two, I write a whole bunch of code, you know, on behalf of an enterprise, and it makes its way into, you know, sort of these, these sort of public, open source projects, I'm more marketable, you know, sort of in the second example. So, you know, yeah, it certainly, you know, it helps with recruiting, it helps with, you know, sort of the talent that you can recruit, it helps for retention, in many cases. Here we have Jonathan Laird, general partner at venture capital firm Workbench, explaining how he used to go about, and his team used to go about during his tenure at Morgan Stanley, considering whether or not to look at a startup to solve a particular technology need. So, not to make this a total Morgan Stanley talk, but one of the ways that we viewed things internally was across really a couple quadrants, or four quadrants, rather, with these axes of, number one is the risk of the startup itself.
16:31And number two is the risk of the service internally. So, at the end of the day, most very early stage companies, Series A and B, would be in that high risk category. But the key is, can you map that to a service or a use case internally, where the hypothetical question of, if they disappear Monday, what happens, you're actually okay. Because you'd be surprised how many times, even in, let's use an example of like the database world. Back in the day, when we were looking at columnar databases and in-memory databases, when they were kind of a new thing, we had a whole team that was set up cross-functionally with different expertise to meet with some of the leading startups at the time. And this was a big undertaking. So, what we figured out was, what are use cases, whether it was like a fixed income team or another small pocket of the firm, where, look, we're not going to replace out IBM DB2 databases. We're not going to get rid of Oracle databases. They're not going away, and no one's going to touch them. But for these net new capabilities, could we actually generate revenue if we brought in a new analytic database that could compress the data, enable cost savings, and frankly, just give us faster calculations? So, part of it is really just, as a starting point, figuring out where can you actually explore in a safe, so to speak, way. And here is Jonathan again, this time with his co-founder and fellow general partner, Jessica Lin, discussing the importance of negotiation in software contracts, both from the enterprise side and the startup side, in terms of finding a middle ground. It has to start from the top, as we said, with these initiatives, but then there needs to be buying across legal, procurement, security, because the worst thing that we've seen is companies that are dying. And let's play off that chatbot example again, that say, hey, we have a use case. We want to share some data that's actually not sensitive. Maybe it's support tickets. And then they get stuck where they'll start engaging the startup community. They've gathered their requirements. And they'll meet with these companies and throw a 50-page kind of POC agreement in front of them. Or they'll put crazy, onerous security architecture reviews.
18:32Or things, as Jess is saying, like this code escrow. So one of the things that companies could do best is actually learn from some of these earlier adopters and more sophisticated buyers around what does procurement look like? Can we meet internally with our legal teams and come up with a way abridged version of an early test and ultimately support contract for this particular use case? And if you're buying from IBM, of course, that procurement cycle is made for them, right? But if you're an emerging startup, that's not going to be the best fit. And that's where, quite frankly, we spend all of our time at Workbench is coaching our startups to say, hey, what is actually non-negotiable in a procurement? But what are other things that, quite frankly, you need to give in on, whether it's an MSA or warranty of the product or indemnification, right? There's a lot of education. I think an opportunity for both the suit and the hoodie to meet in the middle in order to be able to accelerate the procurement process. One of the most important parts about engaging with a startup, especially if you're going to be using its technology for, any serious applications or workloads, is making sure that's a long-term viable company. And here's Jonathan Lehrer, again, explaining some of the questions he would ask at Morgan Stanley to vet startups and figure out whether they really are all that they claim to be. Part of my old job was to help with the financial viability assessments of the companies. And it was actually a lot of fun because what that entailed was getting on the phone with CEOs and CFOs of all of the startups that we did business with, which is actually a huge number. And reviewing their P&L, but also beyond just the pure numbers, because again, most of them were early, understanding things like who are your top 10 customers? What is the concentration risk associated with that revenue? Help us understand the background of your management team. Help us understand your net burn and then also your VCs. And does it look like they would fund you in the future? How much cash do you have left in the bank? And of course, a lot of startups are naturally reticent to share this information, but we actually did have a Chinese wall where it wasn't going to the procure of the house for the users.
20:35So they ultimately would share in an effort to try to win that Morgan Stanley contract. But the things that you would see in terms of some of these high-flying companies and candidly how small the revenue was, a lot of these companies that you see hyped in the news are pretty early. So the trick becomes who actually will get the benefit from that use case that they're pitching, so much so that they wouldn't use the AWS offering of it or the Microsoft Managed Service offering. And when a startup can actually solve a problem that they can bypass some of these big vendors, you know that you can be onto something huge. And finally, here's Jessica Lin one more time discussing the importance for startups as well as for enterprises of engaging with the right people inside the organization. Many times that's going to be a line of business user who has a budget to buy and has a need that needs to be filled ASAP. So there's a whole spectrum, right? And we've advised our companies, and John blogged about this. How do you find the ones that are much more sophisticated, much more early adopting? And the reality is that there's also, you know, people behind that curve. And the question is, with those folks, what tends to happen is they have these internal innovation teams. And the challenge is if they're not tied to actual business challenges or BU's, it can suck a lot of time out of startups. And they end up in these cycles with them, doing demos, showing them their product. And the end of the day doesn't end to actually anything meaningful, whether it's a POC or pilot. And for startups, you know, every day is burn. And so that can be really, really painful and actually harmful to the startups. And so for these large companies who are trying to, you know, make their KPIs, whether it's how many new startups they've met with, they don't realize that it can be really, really hurtful to startups. So we actually advise most companies to stay away from, quote unquote, innovation teams and as soon as possible, get into the line of business.
22:39And that's a wrap on this best of episode of Cloud Native in 15 minutes. If you enjoyed it and you learned something, please go ahead and rate us on the platform of your choice or wherever you're listening. And if you want more about digital transformation, you want to read some of the write-ups of this podcast, you want to just generally get smarter about cloud-native technologies, follow us on the web at pivotal.io slash intersect. Thank you so much for listening.