Welcome to another episode of Cloud and Culture. I'm Danielle Burrow from VMware. And I'm Derek Harris from VMware. If you're new, this podcast focuses on the process of application and IT modernization through the lens of Tanzu Labs, previously called Pivotal Labs. Over the years, Tanzu Labs experts have helped hundreds of companies and organizations, big and small, build better software and adopt modern development practices. The tradition carries on today as Tanzu Labs experts help clients get up to speed with cloud-native computing and the flexibility and resiliency it provides. Our guest for this episode is Kevin Murley, who leads the data transformation team in Tanzu Labs. Kevin explains the ins and outs of data transformation for modern applications. It's a wide-ranging topic that covers new architectural patterns, data engineering for machine learning workloads, designing for GDPR compliance, and a lot more. He also shares some anecdotes from his years watching data trends come and go and share some thoughts on what we can learn from them. So stick around to hear Kevin's insights, as well as how Tanzu Labs helps enterprises approach data modernization. Hi, Kevin. Thanks so much for joining us today. Could you just go ahead and give us a quick introduction of who you are and what you do at VMware? Hi, Danielle. Thanks for having me. Sure. So my name is Kevin Murley, and I am the director of data transformation services. Data transformation services is the newly combined services entity that we've put together, which combines a couple of different services teams focused on helping our customers be successful with data transformation and modernization. So the over-encompassing title of data transformation services includes data solution architects, data engineers, data scientists, and data engagement managers. And so we really help customers. You can think of it in really three different areas. We help customers enabling the success of Tanzu data products. So
2:01Greenplum, Gemfire, RabbitMQ, Postgres, MySQL, upcoming DMS products. So we help customers with implementation and enablement to achieve their business goals with the Tanzu data products. We'll do health checks, evaluate how efficiently they're using the data products. We'll do upgrade services, and of course, full implementations. Then a second area is you can think of it as data modernization, which is often in support of application modernization. So we'll work with the Tanzu Labs team. And we'll provide data modernization, such as helping customers do data domain-driven design, where they're moving to microservices-based app infrastructure. We'll look at how to optimize their data access. We'll look at breaking apart data monoliths to provide better access to provide better access to the data for these new app microservices. We'll build API-based layers so that there's nice, clean access. The example I always use is the companies that we deal with are generally very large companies who have done acquisitions. And we'll have, for lack of a better way to put it, they'll have multiple views of what a customer is, because they'll have incorporated multiple customer databases through their acquisitions, for example. And so what we can help them do is, as they're modernizing their applications and looking for a nice, clean API for a customer, we can take these disparate customer databases and create an operational data layer that gives the API or gives the microservices a nice, clean, single view of what customer looks like across all these disparate customer databases. So that would be an example of the data modernization activities. We also optimize legacy data access. So Oracle or DB2 access, we can help optimize that to improve the efficiency and the performance of these modern applications as they're interacting with legacy systems.
4:14Do those legacy systems go away ever? Or are there those things that tend to stick around and stick around and stick around? Yeah, good question. A little bit of both. So where it's appropriate to fully modernize a legacy system where we can break it apart, break apart this monolith into more distinct components, or change the actual data access mechanism from what might be what might be very SQL oriented access into something that is more event driven or event based access. In time, those systems can go away. But quite often, these systems are very tightly ingrained in the corporate infrastructure. So there's audit and compliance activities built around those systems. And so they will, they're going to be there for a while, they're off in the system of record, and they're not going to move in many cases. So we will find ways to interact with those systems, to make it more efficient for the these modern applications that we're helping customers. So the short answer is both. So then the third sort of pillar I was mentioning was really helping modernize these customers' data platforms. And so that could be, to your point earlier, is phasing out these data monoliths could be enabling the platform that they're using for analytics or data science. So we also help operationalize the idea of data science and machine learning and AI by transitioning these old legacy platforms into something that's more agile, that's a more modern data platform, and really helps operationalize these concepts of business value through learning algorithms and data science models. And then, of course, modernizing these old expensive platforms too. So taking advantage of NoSQL platforms, taking advantage of messaging, etc. So what are some of the things that customers need to consider about data modernization before they get started? I imagine there's a lot of security and compliance issues amongst other things.
6:20Yeah, a lot of the initial areas to consider are around ideas of, to get started, you want to find a nice discrete set of information that aligns with a business goal. So if you're in the process or if a customer is in the process of doing application modernization, then we want to be able to find discrete pieces of information or discrete data sets or even a discrete domain area that we can help modernize along with that. And so that might entail taking information from multiple legacy data sources and creating an operational data store that might be a Postgres database or it might be a Gemfire cache and being able to show the value and return business value by doing these activities. Whether it's performance or cloud, you know, being able to access information more efficiently from cloud based applications or moving these data systems to a Kubernetes based system there, you really want to be able to derive business value and not move data systems for the sake of the next shiny toy. Let's make sure that there's real business value and then we really try to help other customers do that analysis and we go all the way back to data domain driven design and development. So we look at the domains that you're working with and what domains make sense. Do we need to refactor those domains, these data domains to align with the application domains, for example? So you mentioned like operationalizing AI and ML and data science and that type of workload. Is that a common, what I guess I'm thinking about is, is that something customers need to think about? Like, because you, because you want the business value, you want to be able to do a certain thing, but I think the, maybe it's, maybe it's overlooked. Maybe it's not, but like you actually have to build the infrastructure, the data infrastructure to be able to do that, right?
8:21Like the algorithms are one thing, right? But I think the data engineering part of it is, it's kind of the foundation. Yeah, you're, you're spot on the, what we've, what we find is that we don't want to talk about selling data science. What we want to talk about is the business value of smart applications or smart models or solving a business problem through data science and AI and ML. And there's a, there's an old saying that, you know, PowerPoint and Excel is where data science goes to die because, you know, once you've built this great model and you've, you've shown the results, that's great. But if you can't operationalize that, then you're not going to get long lasting consistent value from the work that you've put in. And data engineering is a huge part of that, making sure that the data that goes into these models is able to be consistently, efficiently and performantly refreshed. So you're, you're feeding these models with new data and it's continuing to learn. And these models are being updated and operationalized so that the people who need this information are getting consistently updated and valuable information out of these data science models. How much would you say, you know, the desire to do more ML or AI is driving data modernization versus just needing to, you know, build applications differently, access data differently? I think it's, it's, it's a hard to, it's a hard question to answer, Danielle. It's the, it, a lot depends of course on the customer priorities. So the, if a customer has an impending business value need, you know, they know that they are losing money because they're losing customers or customers are not buying additional services or products from them. And they want to have, they want to have, they want to understand how that's, why that's happening and how to make that not happen or how to make customers buy more, you know, you're looking at efficiency.
10:30Those types of things lend themselves toward building a data science model to really understand what the data is, can tell you what the data can tell you where a lot of other customers just have a need to get off. Old legacy systems because they're paying too much for those systems or they're not able to, to be efficient with those systems or they can't make changes with the systems. They're, they're, they're dealing with a system that's an anchor to their ability to be agile. And so by modernizing these, the systems, or at least the data oriented around them, their systems allows them to make changes to the information faster, allows them to gather information faster and allows them to then think about what they can do with this data in the future. So it really depends on what I, what I call the data journey. It depends on where they're at in this data journey will really drive their decisions about what they want to do from a data transformation and modernization perspective. You mentioned, and you mentioned the, like, like cloud and Kubernetes as potential platforms for this. How do, I know this is probably a broad question and there may be varying answers, but like, I'm going to sort of lump them together for the sake of, for the sake of this, but like, but like, like how do, how does taking advantage of something cloud or Kubernetes? And when I say cloud, I guess, you know, that could be cloud services or it could be just, you know, the, the infrastructure or, or, you know, whatever the on-demand nature of it. But how do, how do, how do, how do those kind of shift the, the nature of how, how you're modernizing data systems or how people are, are managing them? Yeah. Yeah. The platform that a customer chooses to go to, as they're transforming and modernizing their data infrastructures or, or even create new ones, honestly, you know, building brand new data systems is often dependent on where their future is headed. So by moving to a Kubernetes based system, they get a lot of advantages in terms of operational and reliability, operational efficiencies and reliability.
12:33And so they can, they can start to leverage those advantages for those systems, moving to a cloud-based system, same kind of activity. People are moving to cloud-based systems because they want ubiquitous access. And so there's still the, the level of security that they need, but that they want to be able to have the, the general access from anywhere and not be bound to a data center. So we're seeing, you know, people who are moving to cloud-based data systems, whether it's our own Tanzu data products in the cloud or other, because they want to have, they want to get, reduce their dependency on their data center operations. And have more flexibility and agility in how they're dealing with their data. Yeah. And I'd love to get into some of the challenges that, that customers you're working with are facing when they're trying to, you know, be more agile and be able to access data with, you know, along with these applications that they're also trying to modernize and move. Just wondering like, what are some of the big, the big hurdles that you're seeing when folks are trying to do this? Sure. A lot of it is access-based, you know, is there. Can they just get at the data they need in a timely fashion? Is the data being brought into one system or multiple systems? And then do they have to do additional steps to, to make the data available for use? Whether that's reporting or business intelligence or data science or application access, you know, getting the data to a place or in a state that makes it available can be, can be time consuming and difficult. So customers are looking at, you know, these are some of the challenges that they're thinking about here is what's, what data is most important to me and how can I get it as fast as I need it? Yeah. So if it's a combination of multi-cloud or on-prem and cloud deployments, then, you know, access has a different sort of connotation because now you're looking at where does my data, where's my system of record?
14:37Where does my data live? Where does my data live and where is my data going to be used? And how efficiently can I move the data or access the data across these virtual and physical boundaries? And so that's another aspect of data transformation and modernization is, is making the access to that information efficient. If the system of record is on-prem because of audit and compliance reasons, but yet I want to have applications based in the cloud, do I have to do that? Do I have to go from cloud to on-prem for every single piece of data access, or should I create a cache or an operational data store more local to where my applications are going to be running? Or do I just need an event based system where I'm moving events in a, in a stream to get to those applications or to allow those applications to get at the data they need? So it's really, you know, we spend a lot of time making sure we understand our, our customers requirements so that we can guide them and help them move to the right direction and use the right technologies to achieve their goals. All right. Yeah. That's something we, in our last episode, we touched on data gravity a bit in that concept. And I think that's to some degree what you're talking about here is like the data that the data is in one place, either it's, it makes sense to build other stuff there because you want access to it. And that's, that's what the system of record is, but also, yeah. How do you start when you're talking about laws of physics and we start talking about how are you communicating or, you know, sending data across clouds and, and data centers and, and everything. So yeah, absolutely. And, and the idea of data gravity, as well as, you know, the idea of data data is going to sit where it lands and trying to reduce that gravity so that data can be used efficiently can be very difficult, especially when you're looking at the concept of IoT and we're bringing in tons and tons of information. And are we landing it somewhere to then operate on it? Or are we landing it somewhere to then move it somewhere else because we're augmenting that information?
16:44So the idea of data gravity has a lot of aspects to it. There is data is sitting in one place because it's always been there and, and it's been there for the last 20 years. So we're just going to keep putting more and more data there. There's that aspect of data gravity. There's the anchors to the data gravity, as I mentioned, like compliance and audit, for example, where you've got systems and processes and procedures set up for where that data lives today. And it's very difficult to move that. But then there's the aspect of, of does all that data that's sitting there really need to be there? We had a customer in the Midwest who had a, had a big system that was a SOX compliant database, the big Oracle system that was a SOX compliance. And it took them three months to make any changes to the data structure in this giant Oracle database because of all the procedures and policies and compliance they had to go through for SOX compliance. When we looked inside that database, there was something like 12 out of the nearly 400 schemas inside that database. Only 12 of them actually were, were any related to finance, any, anything even related to SOX compliance. So all this data was sitting in this Oracle database due to data gravity, because that's where it was just easy to put. That did not need to be there and could not have any kind of agile modifications to that data structure because of the compliance of such a tiny percentage of information that was actually in that system. It's weird. This is kind of off topic, but, but related, I mean, it has, do, do regulations like GDPR or other things start to start to pop up now, especially in terms of we could, we talk about like, does this really need to be there? How do we access it out? Cause I mean, that's kind of the gist, right? Is minimize what you're keeping, make it possible to delete and do whatever on a user by user basis. Like that's a data engineering problem. Very much so. Yeah. I was, I was literally on a call yesterday talking about GDPR because it's not only, you know, does that data need to be there, but can it legally be there and who can access it?
18:55And what's more interesting these days is how long does that data live? Because now with, with GDPR data can may only need to live or may only be allowed to live for a certain period of time, then you have to get rid of it. So now you have policies in place about who can access that data. Can the data actually leave a particular geographic area and how long should that data live from the time of its, its initial written, being written to a data store. So yeah, lots of, lots of questions and, and, you know, data governance and, and these lots of different things coming into play these days around data. Are there kind of a set of best practices emerging on how to deal with that, or is it really on a case by case basis because everyone's setup is so different and unique? For GDPR, there's a whole, you know, set of regulations and compliance and there's companies popping up every day to, to figure out how to best deal with GDPR. And so that's, that's not an area that we're as the data transformation team that we're going to own. We're going to partner with companies who, who do that and have spent the investment to really understand all the regulations. So there's, there's a couple of companies that we're talking to, to be able to partner with them and be able to then offer our customers some, some more fully thought out solutions than we could do on our own. You just put that, that warning on the website that says, do you accept our cookies? Nothing to worry about. Exactly. That covers it all. I think another question, Kevin is like the, the idea of, because I mean, this is something I think where, especially coming back from the old pivotal days, right, where there was so much talk about, you know, 12 factor apps, you know, this is just how, and, and, you know, there's new, new ways of architecting applications. So I'm curious, I guess, I'm curious, I guess, on how we talk about, or how, how, how it's maybe changed thinking about approaching like stateful versus stateless applications when you're, when you're modernizing and where the data layer works into that as well.
21:06Because I think, you know, like not all modern apps are stateless anymore. Right. Oh, very much so. Yeah. And, and that's always been, you know, a philosophical challenge because by its nature, data is stateful. No data has, it has a state and it has a value at that moment in time. And the idea of a stateless application accessing stateful data can cause some cognitive dissonance. And so we've, you know, the idea of how often is data refreshed and who gets to refresh it and making sure that we have the right design patterns around accessing data is really important. The, the idea that, you know, data is critical to the applications is pretty well understood these days, you know, the applications need data and finding out where that boundary is and how to write smart applications and design smart data infrastructures. So that you can scale out the applications as easy as possible and, and make them as stateless as possible while still having accurate and consistent data is, is part of what we're all about. And part of the expertise that we bring to these data and app joint modernization projects. Yeah, I feel like you almost just kind of answered my next question, which was kind of digging into that around how, like how much of the transformation is really going to be focused on the technology, you know, changing the type of database you're using versus kind of focusing more on the process or the architecture or focusing more on that domain driven design approach, et cetera. Yeah, if we're, if we're, if we're, if we're given the opportunity, we, we spend as much time as possible, making sure that we understand the requirements because that will help guide the technology to deliver the appropriate solution for the customer.
23:10And sometimes it's not as much about the technology as it is understanding the ultimate goals of what the customer is trying to do. And so by that, I mean, um, just saying they need better performance, you know, a customer saying we need better performance for our customers who are using our mobile app that in and of itself can be solved with technology to a degree, but then there's the aspect of. And there's, you can, you can only pull so much data through, you know, a certain size pipe at any one period of time. And is the data that you're pulling through that pipe, the most efficient set of data for those applications to be used the idea of looking at the actual domain design of the data underneath. And then potentially breaking apart that, that, that design that is potentially monolithic from 20 years of use. And then moving that in, and creating a, a smart data domain driven design in support of this mobile app can now make the APIs that are accessing that information much more efficient. So you're putting smaller information through the pipe, which is going to be much faster and more efficient for the application to use. Kind of a long answer to a short question, but that makes sense. Yeah. Cause I think it relates to another thing we, I think we want to touch on was like, how, how new trends in data systems affect. I know. Cause you mentioned earlier too, like, you don't just want to chase the shiniest new thing. Right. But for a while it was, everyone's moving to no SQL systems. Right. And then it was, you got to have a Hadoop data lake. And then it was, you got to know you got to spark. And then it was whatever the case was, but then like you started to see eventually companies go, Oh, we're moving back to my SQL. Postgres is great. And it was like, so I'm curious. Yeah. I think that's, it's like, yeah.
25:10Like, what does that balance or like, or have you seen, frankly, companies chasing these trends because it seems like the right thing to do, but like, you know, it's ultimately maybe it's not, or, you know, you get the point. Oh, absolutely. Absolutely. You know, it's, we certainly have seen that in the Hadoop world. You know, Hadoop was the shiny toy for years and everyone was talking about, Oh, I'm going to do ELT. I'm going to extract land and transform. I'm going to take all of the data that I have, whether it's corporate data systems or IOT data, or where I'm going to put it all in Hadoop. And then I'm going to transform it and it's going to be magic. And I'm going to have all of the answers to every business question I ever wanted to ask by, by querying Hadoop. And that concept was an interesting, you know, it was a good, good concept, but the reality was that was really hard to do. Hadoop's really hard to manage. Hadoop's a great place to land information. It's not necessarily the great place to access information from. And so customers, you know, and companies learned over time, what you could do with Hadoop, what was, what was efficient with Hadoop and, and then what wasn't so efficient. And sometimes they learn very painfully because they made a, they jumped, you know, into the pool without really looking first and, or they, they conceptually understood it. But practically there was issues with how fast they could access small bits of information, for example, or they would be doing big transformations that would slow down other aspects that they needed to be performant. Or it just wasn't flexible enough to give them the information the way they needed it. So customers have, have looked at Hadoop and some have moved and backed away from using Hadoop for everything to solve all of their data problems. No SQL was, it was another great example of, Hey, let's put everything into no SQL. Cause I'm going to get really fast access. But then they realized the ad hoc access to a no SQL database is maybe not as robust as they were.
27:13They were used to with their Postgres system or their, my SQL system, or, you know, the, something that has a more fully formed and, and mature SQL interface to connect to Tableau or Cognos or, or your reporting application of choice. And so there's a, you know, there's the, the old right tool for the right job concept. You know, let's make sure we understand your requirements and then we can talk about the technology to best fit what you're trying to accomplish for today, as well as try to help you be a little bit future proof for moving forward. Are you seeing any specific trends now, either in how customers are approaching their data or maybe specific technologies? Well, the, the shiny toy for the last few years is Kafka. You know, everybody, everybody wants to use Kafka and talk about Kafka. And, and, you know, I view that as it's a fantastic tool and let's make sure it is the right tool for the right choice. The, we have, obviously we have our own RabbitMQ messaging system, which is fantastic and has an incredible value itself. Kafka brings a different set of value. And so when our customers are talking about messaging and eventing, you know, we, we go back to what are your requirements? Not only for the messaging side of it, but your data storage side of it, your data query side of it, your performance side of it, your recovery, your operationalization. We, you know, we want to make sure we understand the customer's complete requirements before we go recommend, you know, Kafka over Rabbit or vice versa. And I mean, more generally, how have you seen, I'm curious how you've seen the, the, the, the space evolve over the past, you know, you can maybe, you could even start, maybe, maybe the point is like from when, you know, from the, from, you know, the last decade or so, right. Cause I think there's been a lot of shifts, especially during that time for, but I'm curious how, you know, like how data systems look, you know, if you're looking at a data system from 20, from 2005 or 2010 versus today, like what generally speaking is, is it, are there, are there, are there shifts that we, that we've seen happen?
29:21I think, you know, some of the big trends and shifts that we're seeing have certainly have to do with, you know, the, the types of technologies and the shiny toy aspect of things. But they also have to do with, you know, and skipping Hadoop and, and those SQL, which we've kind of already covered, but they also have to do with how data systems are being used and managed. So I think there is a huge shift away from big, expensive data centers where you put all of your data and your, all of your data is managed locally there in that data center. And you have lots of DBAs managing those data systems where people are more open to the idea of, I may use smaller purpose built data systems that don't necessarily have to live in the data center today. Like they might live in the cloud or they might live more locally to where my applications are being used. So there's the decentralization of data that is pushing the advent of now I need to message between my systems to make sure I move the data to, or I have access to the data across systems. So there's a, there's an aspect of decentralization, but there's also an aspect of the management of those systems. There's a big push to go toward lighter weight management of data systems. And that may be, you know, obviously something, some huge percentage of, of, of all of AWS services are data services and customers are going that direction because they don't want to have to deal with the hardware and the networking.
31:23And the management of those systems. So it's not just the infrastructure, but it's also the management of the data systems themselves. They want somebody else to do that for them. And with, you know, with our own Tanzu data products, we're looking at how we can manage our customers' data systems, whether it's, you know, essentially our Tanzu data, data services, how we can manage that better for them. So how they can rely on us to, to use, provide them a single system that will deploy, manage and monitor their data systems, whether it's on VMs or on Kubernetes, and whether ultimately that is on-prem or in a cloud-based system. And that is, that is, that is a huge trend that we're definitely seeing over and over again and that we're actively engaging in. All right. Well, that seems like a, a good place to, Dan did. Kevin, thank you so much. Really, really insightful discussion. Thank you very much for having me. I enjoyed it.