Bringing Agility to Enterprise Data Workflows, with Sina Sojoodi (Ep. 55)

Episode 44 · 47:53 · SRT · archive.org · all Tanzu Talk

Well, this week, we've gotten one or two episodes. I've lost track the way that I work on these. But why don't you introduce yourself, guest? Hi, Michael. This is Sina Sojudi. I am part of the product team for our cloud platform, particularly focusing on data-intensive microservices. Now, are you up there in Toronto, or do you just need to update Skype? No, I need to update Skype. I'm actually in San Diego. I've been here for the past two and a half years. That's like the opposite place. Yeah, that's not close at all. A huge improvement weather-wise. How would you compare the two cities? I mean, obviously, the weather's different. But, like, you know, I've only been to Toronto a few times. And my feeling is that they must be booing up the blue glass industry globally. Like, you drive into that city, and it's, like, full of these towers that are, like, just, like, total blue glass. It's a lot of condos and everything there. But I don't know what San Diego's like. How would you compare the two? Yeah, I mean, San Diego, the best way I can describe it is every time I return home, I feel like I'm going on vacation. And I feel like I've heard this from other people from San Diego. Like, literally, it feels like that. It is, you know, perfect weather, very scenic, and just a nice place to call home. Toronto's an amazing city, too. I have a lot of friends there. And, you know, other than the weather, I think it's one of the greatest cities I've ever lived in. Yeah, yeah, it is nice. Yeah, you know, I actually, I guess I have been to San Diego many times, now that I think of it. But we were there on a beach vacation. I forget which, you know, one of the, like, long strips of lands with, that actually had some civilization on it. It wasn't just, like, one big resort. But I remember I went into a Ralph's that they had there, and I was very excited because, you know, of course, in the Big Lebowski, Ralph's is the only ID card he has. So I was like, I'm going to get me a Ralph's card. And then they were so overly helpful that they were like, no, no, no, no, you don't need to get a Ralph's card. We'll just swipe the ones that we have at the register since you're out of town.

And it was a bit of a letdown. But other than that, the vacation was great. That's great. Yeah. Yeah, I went to college in San Diego. What? It's about the same as here in Seattle, where it's 50 and ranking. That would be an awesome town to go to college in. So nice. Yeah. Yeah. Yeah. And they got – my last San Diego memory is at the convention center. You know, it's always – they give the retired people some stuff to do to keep them off the streets at conference centers every now and then. And if I remember, their uniform is, like, this, like, Easter egg-colored pinky blazer that they wear. And it's just – it kind of fits the whole vibe. I guess it's there. Yeah, I'm wearing one right now. So, funny enough. Trying to liven up the Pacific Northwest. Just bring a – Stand out like a sore thumb. That's right. Not only should we eat salmon, but we should bedeck ourselves with that color. Just to – as long as it's water-retardant. So, we're going to talk about, like, data applications and data flows and data stuff. I – anyways, I won't spoil that topic too much. But before we get to that, let's go over, as always, like a little bit of the news. So, last week was Google Cloud Next, which I guess is not to be confused with Google I.O., which is fine. But you wrote – you actually conveniently wrote a summary over in your InfoQ sphere, Richard. So, why don't you give us a recap of what went on there? Yeah, I attended it as well, which was fun to actually be at the event. So, yeah, big event, 10,000 plus people. It was kind of – you know, Google mashes together their kind of cloud stuff that we think of, virtual machines and apps and data, but also with all their business-y stuff. So, it's weird to have keynotes where, at one hand, you're talking about highly distributed databases, and then you switch to Hangouts. Like, that's a weird thing for me to get my head around. But overall, tons of announcements. I summarized a handful of them. There's a lot of really cool machine learning stuff that's starting to make its way into products.

So, it's not just, hey, person, figure out how to train models and use them. Instead, it's like, hey, we have some models so that you can actually take a picture of a credit card, and we'll automatically redact all the – well, you know, we'll put a block over the number and the person's name because we can tell it's a credit card or a social security card. So, just a lot of really interesting machine learning services, a number of good app services, more data expansion, our data center expansion to three new spots, and then tons of enterprise-y stuff, as you might expect as Google's trying to win enterprise. So, interesting pricing schemes for kind of how do I do bulk discounts, how do I make sure that my support plan is somewhat predictable. So, a number of really good things for enterprises trying to use Google Cloud. They brought up some great customers, some of Pivotal's customers like HSBC and Home Depot and others who are doing some cool stuff in the platform. So, I guess it's one of these, you can't sleep on Google. They're doing some great stuff in cloud and conferred off a lot of good stuff. You know, I always find that phrase odd because the first thing I think of is like you were, if you were to, you know, anthropomorphize Google as a person or maybe like a, maybe like, how do you say it in Lovecraft's world? One of those shoggless, like a gigantic blob that's ever morphing with lots of arms, whatever creature it would be. My first thought is that like you just physically get on top of Google and sleep on them. But I guess the point is you're going to sleep while they're doing something, not that you're using them as a bed. But I, you know, phrase. I was thinking we were smothering them. Oh, smothering. The true smother. But no, sure, I think it's probably not sleeping while they go do great stuff. This is probably why I don't get as much done as I want a day is I spend like five minutes here and there thinking about idioms too much. Anyway, so now in the alphabet constellation of stuff, I guess this is like the Google Enterprise edition, or edition, division. And therefore, that's why you see like their Slack attack and figuring out like Hangouts and stuff better because it's all enterprise-y stuff over there

versus I guess Android on its own and search and robots. Yeah, so things with, you know, team drives. So you can actually have organizations that share a drive and when one person leaves, you don't lose all their documents or, you know, a number of things they put together to try to make even sure the document side, you know, they're not super exciting part to tech geeks, but how do I have document sharing and security policies and fun stuff like that on Hangouts and Drive and things like that? But then also, hey, we also doubled the number of cores you can have in a virtual machine. Like these are weird things to say back to back, but I guess it works. Now, since because of your background special guest, what do you think about all the machine learning data stuff in Google Land? And let me ask the question and then make a statement to contextualize it. Like I don't really get what the deal is with machine learning. Like I sort of understand what it is, but I'm always a little suspicious that it's one of those things where the press is like really overly excited to cover it. And so it gets a lot more coverage than maybe it deserves. But I don't know. Maybe it's just like, maybe I'm not seeing the, I don't know, forest for the machines. But what, like what, in the community of thinking about data and all that kind of stuff, like in analytics and stuff, like what, where does Google fit into that? Yeah, I mean, there's so much to talk on that topic. Google is obviously one of the pioneers in machine learning, artificial intelligence, in terms of putting it into their products. If you think of the, you know, the top sort of five giant internet technology companies, Google was one of the first ones. And in fact, you know, just the cloud, you know, they have actually invented and contributed as pioneers to the field. So what they do and what they say is actually fairly important because they have invested so much into it and they've succeeded with it. One thing I've seen from Google that's unique is that while a lot of the other technology companies are trying to offer machine learning

more as a sort of off the shelf component through its APIs and whatnot, that, you know, really do solve the problem when it comes to, you know, tech search or things that are easy to sort of incorporate into your products. But Google has actually gone one step further where they're allowing, or they've pioneered this, where they're allowing sort of custom developed machine learning algorithms that a sort of in-house data science team develops to be also get closer to production. And that's one of the first, you know, real projects that was open source and sort of open the road to others. TensorFlow is a good example of that where they're looking at this holistically. They think that, you know, you not only need to have hardware and compute for training models, they're also offering them, you know, out of the box for some of the sort of pre-made models that you can use in your product, but also helping their customers to also put their sort of machine learning trained models into production too. And they're offering software tools around that. And that's something that I think in the next, you know, two to five years is going to bloom quite a bit in, you know, the more leading enterprise customers that they're going to have. And they do, I mean, there's sort of two points you made in there that are interesting. Like one, all of their stuff, maybe there's some open source things here and there, but it's basically as a service that you have to use it, right? Or that they would offer it. It's not necessarily behind the firewall versions. They've got TensorFlow, which is the open source machine learning library. You can run anywhere. But sure, a lot of this, I think, is trying to productize it because most people are not data scientists. Yeah, yeah. And then it seems like the other one that you're going to oversee now is they have actual models, which I think in the world of analytics is probably, I don't know. I mean, what do you think? Do you think having the right kind of model is like more valuable than the actual technology that uses the model?

Like I would assume, not being sort of a data analytics person, that it's more difficult to come up with the right model to solve a problem. And then the actual like infrastructure, the actual sort of like code that does all the metaphoric crunching, I don't know if people still use that term, is just sort of like kind of dumb and brute forcey and like not that big of a deal compared to knowing what a model is for something. Yeah, the models that they have or the ones that they're productizing as a service, you know, solve very specific sort of, you know, problems, at least in that domain. They may be applicable to various industries, but, you know, this is the result of years of research that they've done in, you know, natural language processing, for example, or many other examples, the video sort of video image detection, an example where it's taken them probably quite a bit of effort to create. So, you know, for that to be offered as a service makes a lot of sense because not everybody needs to sit down and do the same thing. So that model actually has a lot of value. Yeah. But when you talk about sort of domain specific models, things like, you know, fraud detection in online transactions, those are things that, you know, the domain changes quite a bit. Fraud detection for the IRS is very different than fraud detection for, you know, a bank that's trying to look at live transactions. All right. Same concept, but, you know, the actual models and the features and the algorithms differ. And that's where, that's where the, you know, that's where it becomes interesting to kind of treat model creation and model scoring and productization of that, just like any software. You know, you have custom off-the-shelf software, like a content management system that will be useful to a lot of people. And then you have, you know, apps that are written in spring boot by a company that actually is competitive advantage to them. And they're trying to cover both sides, which is, or all sides, I should say, which is quite innovative. Yeah. I was just saying, that's such an interesting point because they had some sessions there where they were trying to exactly delineate that,

which was saying, look, there's some training sets where we have tons of voices and we have tons of pictures of cats. And so we can actually, with really great accuracy, tell you this is a cat or that you said this certain thing. But there's plenty of domain-specific training sets that you have to apply to these models because we don't know your business domain. We don't know how you ship products. So I think that sometimes does get lost that you can't do everything as a service because the training set doesn't exist. And so that model does have to be defined in your domain. I think it's easy to conflate those two things. It's cool that you called that out. Yeah, well, we'll have to return a little bit to that, like how you take all this stuff we're talking about and apply it to the big old fascinating enterprise space so that you can get your mortgages approved or whatever it is you may be doing in enterprise land other than figuring out if this photo is a dog or a polar bear or your child. Or all three. That's right. The cat approves your mortgage. Whizcat. Whizcat. Yeah, that's right. Whiztensor cat. Yeah, that's good. So, also last week I think, you know, we were, I think if, well, not I think we were named Google's, well, at least this part of Google, Google's Partner of the Year which is exciting and we do do a lot of work with them and we have some stuff around. I think actually I have scheduled after this one to talk with my buddy and boss, Andrew Schaefer and maybe we'll talk about whatever that CRE stuff is. But, you know, there's all sorts of interesting stuff we're doing but, that is we Pivotal, but also we had an announcement of some of our momentum namely that, and tell me if I get this wrong, it's always dangerous to talk about revenue stuff unless you're like a triple certified accountant person and whatever. But like we said that last year we had, for Pivotal Cloud Foundry, not all of Pivotal, but we had $270 million in bookings and then, I think we might have had a few new named customers but that was essentially,

and I think there might have been some growth numbers thrown in here and there but as Richard knows, in addition to analyzing idioms too much, I analyze, being a former analyst and M&A person, I analyze revenue stuff too much. So, it was exciting to get some more public numbers to put into my spreadsheet of other public numbers that have been put out. Yeah, I look forward to getting those Slack messages from you. Does this chart look right? Did I calculate this correctly? Right. I'm sure Richard's first thought is like, why does he think I would know? No, it was fun to finally share some of that stuff. It's hard to productize open source and that's probably a whole other chat for us to have one day that there's plenty of awesome open source that doesn't actually solve the product problem or the company that's trying to sell it can't justify a charge for it. So, we've taken Cloud Foundry which is sure publicly available and turned it into a remarkable growth business on top of our data services and Pivotal Labs and other parts of Pivotal. I think it's a really cool story. I think, you know, James Watters liked to say we're the fastest growing open source company in history. I don't know if that's true but that sounds right and that sounds awesome. Yeah. Yeah. No, it's good. It's, you know, people often ask me to be a little Pivotal boosty like what I like about working at Pivotal and other than, you know, working with the people and stuff like that. Like, it's nice that we have like actual real customers. Like, it's a little, it's a little seemingly snarky to say that but it's very enjoyable to have like so many regular organizations to work with as is evidenced by like our sort of like momentum that we put out. And I mention these two together because that's, it's, you know, as an analyst you're always, companies just like Pivotal and others don't like to be extremely open especially if they're private companies about all of their customer count and revenue so you're always trying to decode their actual momentum to kind of match the hypiness they talk about themselves and I think, I think putting out something like, like, like even, even your bookings revenue is just good evidence

of that. So it's, I don't know, it's just a proof point along that line. And then, and then I would also mention there's, there's some mentions of our customers and what they do but your, someone on your team, Richard, it was John, right? Not Jared. I need to stop mixing those two up but they, they had a first post in their, their series about, I don't know, what's the point? What is that, the arc of that series? I spend so much time looking at each, the content of them I've forgotten the whole project. It's sort of like documenting out five, five reasons Cloud Foundry is awesome, if, if I remember. Well, yeah, so this is John Allwright and so he wrote, he's second post in the series on kind of cloud native ops. There you go, there you go. And it's taking this idea of, you know, we spend so much time thinking about the app story but the operation story is almost where you get the biggest change in efficiency because now you're able to serve so many more people at scale through automation. So this kind of post looked at some core reasons for that. He's also going to be looking at the changes to an org structure and all these other sorts of where automation comes into play. So it's a fun series so far that actually has got some good play on social media yesterday. Thanks to an image from you that John put into the blog post. So well played, Coté. Yeah, an image that I stole from someone internally so it's exciting to see the chain of custody on such things. But anyways, the reason I bring that up is in that post that I embarrassingly exposed, I haven't memorized the abstract for the series but that's incredibly accurate. He actually does a good laundry list of several, I think maybe four or five customers of ours and the benefits that they've gotten that make them be part of that $270 million in booking. So it's a good roundup of the before and after effect to the point of that series, right? like it is I think a lot of the operation story is a little bit of a underneath the waterline of the iceberg if you will but it is like once you clear up all of that junk underneath the big fat part of the iceberg you can focus on the part where you can breathe the air and do things.

It's a terrible metaphor hole. I just went down there. But anyways, there's a tremendous amount of good operation stuff that's in there and so his series is nice plus there's, you know, again, what I like about all this momentum stuff is like more and more there's actual proof points and all sorts of metrics that this whole cloud native thing is actually panning out which is fun. Yeah, I'll point out I like your point there if I put on the pivotal booster hat for a second. You know, we did the I'm a pivot stuff I think it was this week launched as well and again, to have one of the biggest banks in the world in Citi willingly want to do videos with us about how this transformation's happening or we spent time with some of our customers last week you know, Coté, I spent time with our friends from Home Depot and we had Express Scripts and other companies at Google Next and they spent a ton of time at our pivotal office or they would openly talk about their work with us so it is nice to work at a place where your customers actually like you. Yeah, so then just minorly we've talked about this a little bit on my other main podcast Software Defined Talk. everyone should go listen to that at softwaredefinedtalk.com it's fun stuff and very, very, what do they say in tweet? Tweets are my own opinion not my employer's so, you know, don't think of that as some official thing. Anyways, there's a bunch of like IPOs planned for like Okta and MuleSoft and Cloudera and I think, as with all of this stuff, there's always some in the iBanker rush on either side there's either some planned or unplanned leaks that come out and so we'll see how the numbers actually plan out but hopefully it'll be an exciting year for people whose hobby is following computers as mine is because I'm thrilling at parties. it'll be an exciting year for seeing what's going on there and I put it that way because those are all in my mind infrastructure software companies and not, what was it you're saying? WizCat? They're not WizCat companies which are fine but that's just not like my,

that's not my bag of stuff. I'm not interested in the WizCats. So you're tracking this space more. Are you seeing a pickup here? It just feels like I'm noticing more M&A, IPO stuff in the last two months. Yeah, I think I would have to go back to my buddy Brennan at 451 and his team who tracks this stuff more but like it doesn't seem like over the past couple of years like the infrastructure software space has been too hot as it were. I don't really know why that is but yeah, there's a lot more IPOs around. Like I mean, I think last year there were maybe like three to five like major ones including like Talend and like some other ones like that but these are these are like larger companies. I mean, MuleSoft's been around since the mid-2000s that have been around and Okta kind of the same actually and Cloudera as well. I mean, these are companies that have been around for a while and therefore have had a lot of runway to figure out what their product is and build up their customer base and all of that and so I think it'll be interesting to like see what the financials are and usually what do they publish? maybe three to five years in the past because I'm always fascinated by like for an infrastructure software company like what what the speaking of models what the shape of their finances look like to fuel you know their growth and revenue and things like that and I think I think in particular I don't know about Okta I mean Okta's fine we use it it's not terrible it seems to work and I get to see those happy green pivotal people every time I have to log in because you can set your background but it's I'm particularly interested in like MuleSoft and Cloudera because they're they're I don't know they're basically middleware I mean Cloudera are probably like it's it's also because it's data and big data stuff and also has the name Clouder and it's got all sorts of interesting stuff going for it but it's interesting to see what a what the finances looks like for a middleware

infrastructure company like I remember I don't know why I'm going on about this so much but here I am I remember when when New Relic and New Relic and Splunk kind of came out it was interesting to see what since I used to program in the systems management space what a company like that looks like so that's a long answer to your question it should be fun and it does seem like there's more more more companies either trading hands like when AppDynamics was sold to Cisco for something before IPOing or just IPOing it just gives you more visibility into what's happening yeah and then all these technologies are in disruptible spaces themselves if I'm allowed to use that I mean you'll saw if the integration space is changing we're going to talk with Sina about that today Cloudera and Hadoop is kind of going through its own thing right now so what's interesting is it seems like these windows for platforms to be dominant keep shrinking so unless you keep evolving in some way sometimes you run the risk of IPOing too late or having already peaked so it'll be fascinating to watch these companies reinvent themselves yeah we'll see what happens well speaking of that let's go over to you Sina like what give us a longer I mean I'm sure I'm sure you're fascinating we could talk about introduction to yourself more and more but give us tell us like what it is you do around here like the types of problems that you work on and types of products and things like that yeah I'd be happy to so one of the things that you know we're starting to see as our customers are trying to go through this digital transformation however you want to sort of categorize that and define what that means becoming more software enabled becoming software organizations becoming more data enabled and running their businesses on data I'm starting to notice a pattern which is pretty interesting because we've seen this pattern in regular sort of physical goods you know marketplace and manufacturing and sort of distribution and that is to say

you know for a good while I'd say in the 2000s at least after 2000 mid 2000s where we sort of started to see enterprises really focus on their storefront you saw responsive web and all sorts of different capabilities mobile and sort of your omni-channel presence of an enterprise was the key sort of focus for a lot of our customers and generally in an enterprise as a whole and that I would sort of compare it to a storefront for a physical good company you think of PG&E or Gillette how you sort of shelf that component and present it was where they saw the biggest gains in terms of you know increasing revenue and sort of becoming a more digital friendly enterprise the next sort of momentum that we saw and this this I attribute the rise of Cloud Foundry to is where that sort of product got packaged and distributed how you applied your sort of flavors of things and made applied your business in a digital fashion to the products that you were selling whether that's an insurance company or a bank we're seeing a huge transformation in terms of how people are building their apps their business applications this is where you're effectively offering your solutions whether as APIs or as however you want to render that into your storefront and the third wave that has just begun in my opinion and this may have happened you know for the Googles and the Microsofts a long time ago but for a lot of enterprises is that sort of supply chain where the actual manufacturing of the data or the services are taking place and I'm seeing a trend where that supply chain which I call data supply chain is going through its own version of a transformation it's a little bit further from the user but that's where a lot of the both cutting waste and savings are coming from and also a lot of the new possibilities are being created so that's

where I am really focusing on these days working with insurance companies with financial services companies and the like where you know when you look too deeply at any enterprise today most are not really selling physical goods a lot of it is data and insights and decisions and actions that are driven from these type of data processing and data analysis and they're trying to see better ways of doing that so that they can deliver more value to their customers but at the same time you know mitigate more risk and cut down some of the waste that goes inside that data manufacturing processing workflows that they're doing so to get some more work out of you give us some examples of what those applications are like at the beginning you know beginning of your explanation you're going over how you can use some analytics to figure out like you know in the consumer goods industry like you know shelf placement all sorts of things that as you say like drive a decision of what to do and how to present things obviously to both retain and grow your revenue I would assume that's what companies like to do but then you mentioned a few things in the financial sector and insurance area and I think it's always helpful to kind of like figure out and hear examples of what those applications kind of are end to end because I find that if you're not working on those you know to your whizcat point like you don't really realize how much custom written software there is out there for companies just to like run their business so going back to my original prompting if you will can you give us like you know one or two examples of what these kinds of workflows so to speak are these businesses that are supported by the applications and how it's more than just like a really nice spreadsheet that you're working off of yeah so let's use insurance and this applies to you know car insurance health insurance it doesn't really matter

a lot of the data that the insurance companies actually sort of work based on is not necessarily what you go on a website and you enter and you sort of have a transactional system it actually comes sort of out of band from various sources either purchased or sort of collected you know insurance claims for example they are put in by folks that are not necessarily behind a web portal trying to see whether they're covered or not they're actually brought in by the doctors and various exchanges and that happened in various places these companies if you think about it you know as their evolution has gone and we have customers in that domain particularly that have done this you know a lot of it was just ease of use for their customers and you saw the various mobile applications and websites sort of recreations that came about and has helped a lot and then you know the sort of API economy sort of came up and building custom software that actually manages the way this information gets exposed to the consumers and managed by the enterprise the third step as I'm calling for insurance company in particular is actually the act of processing those claims and correlating that with you know various sorts of data you know in the home insurance space that data might be weather data you know you were trying to claim a you know a claim that your roof caved because there was a hail that came through and you know an insurance company or a company that supplies that information can actually cross correlate that data with the insurance claim that are coming in and also the weather pattern weather data that's coming in for that particular time to make sure that there's no fraudulent claim being done so that processing of the data the raw data into operational data into insights and running that in production and actually making decisions based on it whether you sell those decisions or you just sort of sell services on top of those

is something that's you know fairly key and critical to enterprise as we see it go through the next few years that's one example that I want to use fraud detection is another one you know banking we sort of see the front end of that the messaging that comes and says hey we notice you're in a different country you're trying to use this reply back that's the consumer aspect of it but there's a whole lot of data crunching that happens to try and detect whether that user was in that location the pattern of spending all sorts of different things that they can look at and investigate all that comes from being able to handle a lot of data processing a lot of it in real time but also in batch and being into reason about that and that process as I'm finding is hasn't necessarily gone through the same software evolution that the sort of more upstream closer to the consumer products have and that's something that we're seeing a lot of customers come to us to ask for help because it's being sort of debilitating yeah what's that look like today then technology wise is that mostly batch is it you know dealing with legacy flat files like how are people dealing with that today if it hasn't gotten modernized it's a combination of all I mean batch is the predominant sort of mode of operation but you know in terms of how data actually comes in there's a lot of FTP and R-Sync and things that you know you would pull your hair thinking that can't possibly be but as sort of more data has come in as more acquisitions have happened in a lot of these organizations have sort of lost a little bit of control over how that data supply chain goes through so there's a lot of work that just needs to be done in getting the house in order and the next step would be making actual use of that data in

wow did you say rsync isn't that like what you used to move your what was the old your usenet stuff around or something I guess that's something else but wow rsync that's good times I hear that a lot actually I use even rsync and people start laughing in my conversations saying no no don't say even that's a lot of the organizations that I talk to what's the cost of that nowadays is that speed unacceptable or obviously it's still tolerable because those companies make billions of dollars but what's the opportunity cost of not having that data accessible more in real time or having even mythical machine learning applied to it to make decisions is there an impact here well yeah the impact is actually much lower level I mean forget about machine learning and you know that's what gets me you from their data but a lot of the conversations just you know not even being able to you know inject new data I mean I've talked to customers who really say you know just bringing a new data set and this is not necessarily a new sort of exotic data set like sensors just you know the same data set but from a different source more less the same structure takes months and in the DevOps we talk about weeks to months of making a change some of these are facing months to years to never being able to actually change something because it's too complicated too risky too monolithic to change and they're losing customers I mean I have customers who are the monopoly on certain data and yet they're losing customers because they're not able to make changes to that data processing pipeline that they have and that actually is tens to hundreds of millions of dollars of revenue lost just on that forget about any intelligence that you

can build on top of that which everybody wants I think just even getting integration solved is going to be a huge opportunity for these companies to realize yeah when we were it must have been a month ago now when you were helping school me up on this for some people I was talking with about a month ago I think and you you've kind of gone over it but that's what started to get me even more fascinated in this kind of how data is running through and driving your business because the examples you went over and other ones are it's very core to we have a business process like insurance and paying out claims when something happens and that's one of the core parts of the business I assume another core part of insurance business is how you invest that money and make your percentage points on the carry of all that cash you have but whatever paying out claims and avoiding fraud there but then also generally paying them out in a way that makes people happy so they don't leave you and think you're scoundrelist if that's a word is extremely important and so it's an example of what we talk about all the time is that if you want to have like a programmable business you need to have all of the agility and speed and rich feedback loops that comes with a DevOps and a cloud native and an agile approach and so like you're saying if you're stuck with a bunch of rsync and on your ability to evolve the business essentially and I mean probably as with all things I come up with there's actually a quick work around for it but I would imagine if you're an insurance company and you want to make it so that a claimant or whoever the people that come inspect your car like if they could just take photos with their iPhone and submit

that to you that probably takes a modification to your workflow and you know having increasing that speed and accuracy of that information in your process probably is great for the business if they're not stuck with a monolithic system that they can't adjust too well so I mean anyways the way that all this seemingly boring stuff of workflows and data collection analysis like I mean the word workflow just makes most people fall asleep but it actually turns out that's the way a tremendous amount of businesses operate and I forget the three or four things you said but it's just like getting all your data in front of you to help basically drive a decision about doing something very tangible to your business and all of the workflow around that seems to be a lot of what we're talking about here so to that point it seems like for many businesses you can't get off the shelf stuff to do all of that you really do need to work on your own custom written not only the software the applications around it that are facilitating all of that right I think you nailed on the head and I don't want to dismiss the real problems not everybody has an rsync problem not figured out pretty well but one thing that's happened and this is sort of the if you look at history of any monolithic system why systems become monolithic is that A the tool that they used worked and then you start putting more and more faith in it and you start putting a lot of faith into it and you just bloated with a whole lot of stuff that now is so tightly coupled that you deal with and that could be a database that could be your ingestion pipelines the ETL tools that you use all of them are great this is not a ding on the tools but rather what has transpired

as that tool has become successful so an innocent tool like a stored procedure in a database that actually serves a very legitimate purpose is has been adopted by many of our customers as a way to do everything in data processing so what happens is at some point that poor database is not able to scale vertically anymore and you need to sort of you're not getting a scale out of it but also making any changes you know everybody is sort of holding their fingers cross making sure that you know integrity of that data is kept more efficient and being able to break that down into an architecture where you know you can have proper contracts language agnostic contracts between these components and have different teams work on those components without worrying about breaking changes down the stream and having that DevOps practices being brought into that data preparation data engineering practices helps in a tremendous way so before we wrap up I mean just briefly tell us like other than having consultative help from folks like yourself to kind of explain this and help people with the architecture stuff like what are the actual products and services that we bring into this area right how do we actually make life better in any way here to help out yeah so on the product side you know the there is generally a glue framework architecture is no different it's just a tenet of application architecture of software architecture it's no different you have to really look at what the outcomes are and what the context and the boundaries are and you have to define that architecture a tool that we recommend in our open source sort of chest is

spring and particularly spring cloud data flows ecosystem it allows you to build these data pipelines in a very agile composable fashion and you can apply the same sort of software engineering practices like test driven development contracts CICD and apply that to these big monolithic systems now on the services side we also realize that you can't make these the same concepts of strangling the monolith and making a migration towards a more cloud native way of building these tools and components apply here you probably won't be able to drop everything that you've done and say from today on all of my multi decades of work that's gone into these data manufacturing pipelines are going to be put aside and I'm going to recreate them in a cloud native way not possible so how do you go about changing that strangling that monolith and extracting the pieces that are high value prone to change and you can basically rewrite those in a more cloud native way and kind of work your way through so the products you know Spring Cloud Dataflow caching caching you know the pivotal cache and Gemfire have a huge influence on moving these pipelines into real time and helping downstream products connect to these data in a much more easier way and as you go up the chain you know in terms of bringing more value and insights we also have our BDS suite and analytics products that can help integrate that insights into your running applications and exposing that as either decisions or byproducts or the data that's sort of a byproduct of your raw data that you can basically bring into your consumers So you're seeing practically that companies do have

some of that legacy heritage and they're dealing with nothing but batch that they see a path forward with some of these more modern data pipeline tools that's not a bridge too far that's not too the tech but because they're thinking how do I morph a bunch of ETL or even ESB processes or stored procs into this sort of decoupled data pipeline that's you're seeing people jump on that yeah when there's a business need behind it it makes it a lot easier to make that jump if we go to an ETL team and say hey here's a new tool use this that doesn't go very far because people generally like the tools they use and they're familiar with but when you go to a business team that requires this change it becomes a lot easier because you focus the attention on the pieces of the problem that need to change and that doesn't need a complete overhaul of the system you can start small and you can start carving pieces out that you want to bring into more flexible more agile more cloud native way makes sense thanks so if people want to dig deeper into this do we have some blog posts or overviews or keynotes recorded presentations what do you think a good source is to dig more into all that yeah so we're working on a lot of this stuff this is not an adopted pattern so the information is a bit scarce but if you look at our you know a lot of the conversations around a lot of blog posts around spring cloud data flow they usually start with you know proving the problem and talking about how you can solve this so I would say all the blog posts in the spring community about spring cloud data flow sort of apply here also our team has done quite a bit of work on talking about you know microservices and you know this concept of data microservices or as I call it data intensive microservices we have a bunch of blog posts

about that that really try to describe this problem and ways of approaching it and how we can solve it with architectural patterns but also specifically with the tools that we recommend well I'll have to get some links for you to put in the show notes and I think I forget if it needs to be unconfidentialized if you have another experimental word here but I think you have a pretty good before and after visual of going from your efficient but archaic rsync architecture to a more cloud native architecture which should be interesting well thanks for being on I think just like when we were talking about a month ago I always like to find people who can explain how a space is evolving and I think we did a good job of that and I appreciate it so if people want to figure out what kind of cats you're playing with in Twitter do you have some where they can follow you there are you more of a LinkedIn person where should people go if they want to check you out yeah at cnosojudy is my Twitter handle I'm also on LinkedIn as well cnosojudy there's fortunately not that many of them out there so I'm pretty easy to find if you know my last name and first name yeah happy to engage with folks on either platform well perfect well I have one little mid-rolly ad thing to put in here so we've got the cloud foundry summit coming June 13th and 15th and you know we were delightfully be bloviating and everything about how awesome the cloud pivotal and the cloud foundry world is and the cloud foundry summit is great because there's everyone from the broader cloud foundry world comes there's always a lot of the schedule is going to be announced sometime later this week I think but there's always a lot of customers talking about how they're using it at the high business level and plenty of technical conversations as well going on and we and everyone else will be there from the big dirty machines to the slick whiz cats and everything in between your insurance claim processing things

and if you want to I have a 20% off registration code I guess it's cloud foundry silicon valley so hence this ornate code which is cfsv 17 Coté my name there but I'll put a link in the show notes and you get 20% off whatever we're actually leading the chair for our Java track for the summit tons of great submissions and over 400 conference submissions so it's brutal trying to pare this down it's going to be a great program yeah maybe past guests Kenny and Josh will have their book out at least I know you can get some samples of cloud native Java but I know it'll be an exciting relief for them to have it actually finished finally that'll be good stuff right I'm sure we'll have all the other great little rally books you can you know those are fun well as always this has been pivotal conversations if you want to get the most up to date podcast because we kind of lag posting the official show notes because we just because Richard's always hoisting so many of these on me to do just real task driver that guy anyways you can go to soundcloud.com slash pivotal conversations and there you can find the RSS feed to subscribe star rating or a review comment I think there might be one in there so you could be the glorious second person you know Apple didn't really invent the mobile phone but they're now number one because they were second so you could be that person except there's probably no revenue associated with it just making us happy and to find the show notes when we get around to posting them you can

Tanzu Talk · 🍕 Get more pizza.