Grappling with Data and Application Modernization with Redis Labs’ Adi Foulger and Cassie Zimmerman

Episode 124 · 33:20 · SRT · archive.org · all Tanzu Talk

so the big question uh for our guests today um Adi Folger from Redis Labs and we've also got Cassie Zimmerman from Redis Labs on as well big question is about what's different about data as you modernize thinking kind of modernizing monoliths to microservices uh but what's what's different as you kind of get into the data layer uh obviously like all the the sort of the 12 factor principles don't really apply and so you know what do you do and what what are the challenges that are different what are the goals like the kind of why are you doing it at all yeah that's a great place to start drumming um I think really uh it's going to come down to a few different things um usually it's going to be some kind of response time driven or some kind of user experience driven initiative either uh you have some kind of competition that's pushing a faster website load time or just a better user experience like I I had one experience at my previous job where it was all about kind of the all-encompassing user experience and that was how we were losing it they were just they were just beating us out on the on that whole multi-dimensional thing um so ultimately you kind of have to get to a few different things right you need to evaluate how expensive the data accessing process is both in the context of time as well as in context of dollars um and then if you want to do some of the more personalization or some of the more advanced things that we're doing with data science today you need to have a more sophisticated data architecture um than what most companies have have historically had um and I think let's let's kind of just start out with with the the easier one which

is uh things need to load faster right so I worked for a CTO who was part of his uh his evaluation process was the website had to load in a second or less um and that can be a really big challenge for a lot of a lot of companies because um most people that are kind of getting into the industry today aren't really we haven't really had the experience with performance minded uh development it's been a lot more like web web development which is usually a lot more lenient when it comes to these sorts of things um and then this is where kind of the the reddest part of the story comes in right so we we we have a slide that we use all the time that shows like when you talk to the database it's going to cost you 100 milliseconds um the response time for the web layer is you know 50 milliseconds um and all of that time adds up um but using redis for caching or as the operational data store um is how we've had several really really large customers move beyond um this challenge right and then on the the personalization and the kind of data warehousing analytics side um being that front-end high high ball highly volatile buffer um has helped drive a lot of other uh a lot of other initiatives as well yeah i i want to add one thing on that audi just taking a step back and looking at you know where the enterprises come from cloud native development i know we were on a podcast with you guys or audi was on a podcast with you guys months ago or maybe a year ago at this point i'm losing my track of time but um we talked about hey we talked a lot about cloud native application development but we're not so much talking about what's happening with the database there and i think kind of from day one it was very much state from stateless kind of don't have your database running in with your application keep it external in the monolith and what we've experienced and audi can get

into the specific customer examples of this is people are realizing that you need to think about your database and what you're doing with your data from day one in order to have that instantaneous user experience um whether you're in the financial industry or the retail industry or you know insurance whatever it is but being able to say hey we're not just going to keep our database in the back end in the database or excuse me in the um in the basement somewhere but we need something more flexible that can react in real time and a lot of that comes from tools like redis that can be implemented as you know a cache for your microservices architecture um or you know other no sql more flexible databases like mongo db or um you know my sql and things of that sort so kind of taking you know application development and cloud native uh you know features and functionality or you know i guess that's the right word um but also thinking about what tools you need from the database perspective in order to get you there so that that your applications are running the way that they should so there's a couple couple questions so if we look at things like auto you mentioned solving for faster load times is something that's different about uh how you deal with data as you modernize so are you saying that folks have had that kind of problems solved before like as a in the in a monolith world they sort of had that solved but by modernizing they reintroduce it like why is this something that's different for modernizing as opposed to just hasn't making sure you have faster load times been a problem for as long as like internet commerce has been making yeah like that that's been actually i mean for your business well

i i think i think there's a there's two parts to that answer one is um certain really big websites the the facebooks the googles of the world have kind of set the standard for this performance um and modern broadband internet is still kind of relatively new if we think about it in the grand scheme of things like yes we've had we've had internet we've had fast internet for a while but um the whole notion of it has to be there now right now now now that's that's a little bit more kind of you know post 2010 um and i think why is because of a more customer user centric focus um that's kind of been coming in and increasing in the industry and this is of course you know what you guys the pivotal do all the time right is is how do we think about things from the customer mindset um and and the customers don't want to wait and so when you have competition or you have a standard out there of hey this is expected to load this quickly and i shouldn't have to wait for my data or for my experience whatever that might be um it it just it they they've they've moved the goal or they they've set the the bar higher for everyone else i i that i think that kind of gets at what what you're getting at right but i i would like to add i i second what you said audie and i think that user experience now in this digital world is like the most important interaction that your customers are going to have with your business so it's not just you know your static website that your customers are going to but it's your online banking that you're using from your mobile device and it's tracking your shipping codes from your mobile device to see where your package is wherever in the world and this needs to happen instantaneously and that's a total shift in what businesses had to do even 10 years ago right so

the more that we move to digital the the faster that we expect it to do and the more competitive it is with the the enterprise landscape to to get the customer experience there so are we modernizing because we need a better experience and better customer experience and then that has data implications or are we modernizing and then that has performance implications that we have to go fix i really kind of want you both to blurt out like an answer at the same time and it's not the first one in my opinion i know what were you going to say i'll let you go first chocolate no vanilla yeah no the i i don't think i don't think people really go about hey we need to modernize for the sake of modernization um it's it's really it's really usually driven by like i said competition or or customer demand um sure kathy what what do you what do you think i would say the same thing so i would say um the reason that performance has become so important isn't necessarily because um you know people want to just try new technology and and re-architects the way that they build applications but um the the internet giants today right uber airbnb twitter all of these you know very successful companies that were born in the cloud have set these expectations for users to be able to you know have real-time experiences use it from anywhere um have global access all of these different requirements that i think has gone into what cloud native means and that's just because of the changing user requirements out there in the world and that's why you see you know fortune 500 fortune 1000 companies having to say we need to increase the performance and improve the user experience for our customers in order to retain them um so you know i think part of it

is you want to make our business better every day and you do want to be at the forefront but in general as trends move you're going to need to re-architect the way that your database systems and the rest of your applications are running because of the demands from the market and i think too um there's another component that's more than just straight performance it's also uptime right so in the same vein you don't necessarily uh you you need the website to load fast but if it goes down it can't be down for more than five minutes right and i think that piece is is actually more intrinsically tied to the cloud native part of things um because that's not just redis or databases that's the entire application that's being able to do multi-cloud that's being able to do hybrid architectures right and this is this is of course what what tcf it helps enable people to do and so um the fact that we can be bundled in with that concept that we can be a part of that platform um i think it's helped these really big customers that we're talking about um be able to to get over that hurdle and not only solve the performance problem and have a better user experience but also be able to solve the ha problem and the scaling problem and you know all all of the other things that modern software has to be able to endure in order to be successful and that's not going to come with one tool or one product right you're going to need a platform you're going to need the infrastructure you're going to need databases that are supporting that um and a whole lot more so it's important to look at the full picture which we can talk about more um so you mentioned something also earlier uh i might be mangling this but i paraphrasing you mentioned something about you need to evaluate the essentially how you value your data and how you value access to the data can you expand expand on that a little bit what do you mean by that i heard some statistics recently that we're creating like uh you know some some ridiculously large amount of data

every day um and and more and more importantly especially if you look at kind of the the bigger conversation that we have going on in the us and the rest of the world about uh how how much our data uh is is valuable right so how we have all these big companies who are making a lot of money off of things that people are willing to give off for free so um let's let's kind of take that concept and look at it through an enterprise lens right you can you can generate all the data in the world but if you don't know how to use it if you don't know how to manage it if you don't know how to serve it um it's really just bits sitting on a hard drive somewhere right um and i think there's a there's a few different dimensions to that um one is how does it fit into the nature of your products right so um in in the the company that i that was acquired by mbcu that i worked for um a big part of that was the database of all the movie information that we had gathered and all of the the like low level tagging and um kind of the the level of granularity that we were able to provide to that um was a big part of the value um and that that was sitting in a sql database and we figured out how to how to leverage it um to ultimately drive things like seo and and and ultimately back to that user experience that is kind of the the key theme here um but we're kind of the next phase of that thought is um the larger translitical conversation which is what we were having you know when i was on y'all's podcast last year um around how do we get data out of the production system how do we use things like machine learning to provide insight um and then ultimately make that data more valuable and i don't think genuinely um that most most companies are really there yet i think they've kind of over the

course of the last few years kind of gotten a hint or a sniff that that's that's a thing that they need to do um i don't think most of them know how to do it um but but for example kind of the the tracking the the tracking example that cassie cited before that's a that's a place where it works really really well right so like if i can get a text message on a phone within seconds that my package is delivered that's improving my life right um and and that's understanding the value of that little piece of data to that user and that's just a very small example of what i think we should what we will see uh more and more of going forward as companies continue to modernize yeah i want to give one more example aside from you know the the shipping customer that we're talking about um but another way that we're kind of driving you know data driven applications is in the financial industry with pivotal so fraud detection we've seen come up a lot as a use case in the financial vertical and that's i mean think about it you're in europe you get a text message from your credit card provider that says hey we've identified fraudulent or abnormal activity on your credit card and sometimes that's really irritating to us if it's not fraudulent but if it is then you're going to be pretty upset if somebody's in frankfurt spending all of your money um so having that real-time response and being able to say hey cassie is not in germany right now she's in california um we need to immediately alert her that something is going wrong that's the type of application that needs to literally be able to process in sub millisecond response times so and to just go off of that to add a very awesome fine point um that's one of those performance driven situations as well if you have to wait a minute for your credit card to be processed it's not going to be a good user experience so that's why three out of the four credit card companies use us to do that getting back just for a minute to the valuing your data i mean do you think we need to take

this to the point where we're actually putting a dollar value on data you know looking at it like a like we would an accounting issue i mean it seems a little squishy like yeah i can see it would be valuable yeah have you know that insight to that user but i mean you have to balance that on how much it's going to cost to do that and resources and time and energy and all that kind of thing yeah also potentially a liability if you want to go down that that's exactly where my head headwinds are made yeah um i i think it's a very good question and a very good point um and this is kind of why i think it's been very difficult for companies to to to move forward in this case um and and you know it it has a certain amount of value right you can there's an easy way to frame it if all of that data that was released publicly tomorrow how how damaging could that be or or how much value would your company have lost had that happened um and if if you don't have at least a ballpark figure in your head if you think about it and that kind of most catastrophic scenario then you then you don't really understand the value of your data um but more broadly this really kind of falls into the product management um and positioning kind of perspective every product out there that's being really successful today that i see has some component of leveraging the data that they're that either they get from the user or that they're getting from you know third parties and providing to the user uh and if you can't if you don't have an answer for these questions and if you don't understand like what you can do with your data then and in the long run you're just going to be left behind okay so what are some of the you mentioned that like most enterprises just aren't very good at this um and you know the like the the credit card fraud detection i mean totally get like it's very valuable and it's it's very um performance sensitive but in many ways it's like it's it's an existing

use case right like credit card companies have always been in the business of trying to minimize their you know fraud exposure because they incur a lot of that yeah but so if we step away from that kind of example what are you seeing with the folks that you're working with in these modernization efforts where they're they're doing kind of the right things around how they then incorporate that data back into the products that they're building i the immediate thought comes to mind if we talk about kind of the newer stuff is going to be in the iot space um and and there's some really interesting things that we're doing um around providing tools for edge computing um and and ultimately kind of where i want to go with this and it's not specific to iot it's going to be um it's going to be the case for everyone um or at least any anyone who wants to do some of these more personalization or data driven sort of things um it it's going to be around ai ml and and being able to serve insight quickly back to where you're getting the data from so in the case of iot it's being able to serve that on the edge on a raspberry pi and this is something that we're we have tools that we're we have that are going to be released um here pretty soon um probably around the time this podcast actually comes out so that might be a cool thing uh but yeah anyway uh being able to serve these these ai models on the edge um being able to get the insights back to the user in real time i think that's the next phase of that um and and and we don't have a whole lot of people who really are kind of there yet um so being able to in the short term or bridging that gap is is just starting to understand what kinds of insights could you get out of your data right so for example um going back to kind

of that movie data uh that i was talking about being able to do recommendations with that being able to do something similar to like what you see on all of the video on demand services of hey uh you like this movie so therefore you might like this movie how how could you do something like that with your customer data right and and maybe that fits in your business world and maybe it doesn't um but that those are the kinds of questions and and the things that like data scientists are really kind of getting paid all those great dollars to be able to figure out yeah i think that's a great point audi and i to your point domain in the financial vertical fraud detection will always be a top use case right but as we're getting into these entirely new industries and working with these cases that we've never seen before the business values that they're going to get out of that are still kind of unknown right they don't have personal profiles for all of their users and um you know what cassie does on a day-to-day basis on her credit card and how to identify fraud with an iot company um you're going to be tracking millions and millions of devices globally um and and using that for a totally different outcome so i don't know if we're there yet and we've been able to see that in a real life scenario but we're at least having really really interesting conversations when it comes to the iot iot space um and really even other use cases in the financial vertical as well and retail when it comes to recommendations um well i guess just to like tie kind of to tie this all back to the modernization efforts right that was the the question at hand um and so you know cassie you had some really interesting points about you know geodistribution uh both from a governance and compliance perspective but this is also something that you know folks as they modernize are also taking into consideration is you know how are we we're now moving away from a monolith we're having much more distributed applications we can potentially

run this across multiple clouds we can run it in you know multiple geographies and serve users uh globally in a more um you know you didn't folks can get a more low latency experience if they're they're actually accessing a um instance of the application running much closer to them so um that kind of you know thinking about that the modernization work to go into that is that something that you're seeing a lot of folks as they modernize um they're starting to think about that that geographic distribution yeah i mean i would i would argue that those geodistributed architectures are coming up more and more in our customer conversations and not just on the whiteboard when we're at their offices but in production as well um and i'm sure that audi will have some great points to make on this but i think the work that little in that space is especially interesting because pcf is providing kind of that horizontal layer so that you can run hybrid cloud and you can have different foundations globally but aside from that you also need a database that's going to be able to cross replicate to be able to communicate uh across the world um in the same way that i guess cf kind of is and and redis enterprise and redis can provide that layer of data to run redis clusters across the world and be able to replicate in either you know an active passive or an active active type of replication model um and those requirements i think are coming up more and more with customers as they are expanding their global footprint from a data center perspective and understanding the value of hybrid cloud and diversifying what their infrastructure looks like and data you know obviously will fall into that point at some point when you're looking at how you need to how you need to globally distribute your your databases as well the concept of data governance across international borders is probably is the

hardest part of all of this right so we have we have customers who can only host data in germany or data in france and and there's a lot of requirements around that and certain data can lead the country and certain certain data can't um and as well um with with our technology as well as some of our other databases out there um the whole kind of standard model of active passive is is starting to go the way of the dodo right like active active when you have a platform when you have multiple cloud providers or your own multiple data centers is a lot cleaner of an architecture and if you have the tools to be able to build it um then you're just going to make your life easier in the long run yeah and i and on kind of snowball off of that when you're looking at the developer experience to build those types of applications you need the tools that are i would say easier to implement than than others and i i think since redis is already so well known in the developer community we've spent years engineering this really robust architecture to be able to run your redis clusters globally so that the interface from a developer perspective looks the same so when they're writing their redis applications and writing code they don't have to worry about the ins and outs of a database in the back end and how to globally distribute your redis clusters um that's where our software comes in and obviously pivotal tells that story you know very well the developers should be doing what they what they do best which is uh writing good software and writing good code um and i think what we've the work that we've been doing at redis labs falls really good into that story of fast time to market and ease of use from a development standpoint when we're talking about how do we get to these large-scale global applications so the other question that i had that cropped up was you know talking about the the kind of hybrid cloud and multi-cloud is um also just you know there's a lot of there's kind of this interesting

quandary i think sometimes folks are in comes up as they modernize it can come up with you know building net new you know applications as well but wanting to take advantage of native uh services on the cloud providers and you know there's a lot of interesting you know machine learning services as well as just kind of raw basic whether it's caches or sequel or what have you um and on the one hand like there's some great innovation happening um in those services on the other hand there's a tremendous amount of like stickiness to now having your data sitting with that particular cloud provider and if it's in like uh you know a a sort of non-standard version of uh whatever type of database or or cash or um data warehouse or what have you now you know that's a sort of uh a bit of a commitment to that cloud provider so how do you see folks navigating that in terms of well i want you know i'd love to have my data in the cloud i want it close to where i'm running my workloads which is i want those to run in the cloud um but i'm i'm a little bit concerned about really marrying this data to a native cloud service or like i want to be able to mix and match right like how do you see people navigating that so um once upon a time i was part of a cloud forklift project and it was i was we're on it for like two years or something like that and and the one thing that kept us from really like doing a real cloud forklift was uh our sql server uh we couldn't figure out how to get it up into the cloud and and we didn't even have the problem of like multi-cloud or anything like that um so so i think data as a whole has a lot of

gravity um and and that's not exclusive to to cloud services right like i think that it takes a lot of time and energy to to build out a database right um whether it's redis whether it's sql or or whomever else um and and once you get it you don't really want to touch it right you wanted to it took a lot of time and effort to get to that point in the first place let's just leave it as it is um but to to kind of get more to your core point there i think in the earlier days of cloud forklifting and and the migration of the cloud of the bigger companies um we were seeing a lot more of just kind of hey let's just use that vendor service and and it'll be easy um but nowadays especially the more sophisticated companies realize they can't do that and they can't really do vendor lock into an individual cloud um the nice thing about redis even if you are using one of the cloud provided versions and order or like our uh cloud managed service for example is that the protocol aspect of it is very portable um so we we've helped a lot of different companies migrate from one cloud to the other from on-prem into into cloud and and every every everywhere in between um so it it's not as big of a problem with say like ours but if you look at um you know for example like dynamo right dynamo is very easy to get started with um but then you kind of end up where you you're very locked into it and it ends up being very costly very quickly um and and now you're stuck in that one cloud right so i think if anything i don't have an answer for it right i think if anything i have a cautionary tale of be careful wherever you put your data that's probably where it's going to go and i and i'll do do my best to try to help you figure out what's the best way to distribute it and this is where um you know again

kind of the the platform story of pcf and that that horizontal dimension um is is really important um but yeah i i cannot recommend the opposite enough uh to people in all of my conversations with you know the various big companies the world i get to talk to yeah i i would echo that as well and when we're working with our customers who have already decided that they will take a hybrid approach i think a lot of them recognize that buying a particular data service or other services from one particular cloud provider just won't make sense for the architecture that they've already decided on because they understand the advantages of going hybrid cloud and multi-cloud um and luckily today customers have so many options when it comes to the ecosystem and cloud providers and platform players and you know services like redis uh have built up these ecosystems to work together and i think that's one of the really cool things that pivotal and redis labs does very well is making sure that wherever the customer wants to run we can enable them to do so and we're not going to lock them into running you know on x y or z cloud or platform it's it's what they want to do and you know people like audi and other solutions architects at our company can help guide them for what we think is the best option there um but having that conversation with the customer is key to understand what they're trying to get to what their requirements are and if if the outcome of that is going with you know a cloud service provider then that's fine but if your requirements really do require a data service that can cross multiple clouds and be globally distributed then you know you should probably have a different conversation um good i'm also having good healthy conversations with um you know with their teams about like what are we really trying to get out of this and so that we're not just taking the easy route uh

or something that you know was the we've already been doing this route uh so that they can start to start to identify like what they should be doing um so thanks so much for for joining and great to have you and lots of great interesting stuff continuing to happen in the data world um so talk to you next time thanks for having us

Tanzu Talk · 🍕 Get more pizza.