Cloud-native Malay Food, Tracing in Java, and Databases

Episode 90 · 38:07 · SRT · archive.org · all Tanzu Talk

All right, well, here we are at, where are we, Josh? Huh, Singapore. Can you believe it? Singapore, yeah. How many times have you been here? Just here? Just in general. Just more than a handful? Oh, yeah. Or a bucket? So when you come here, what would you recommend as a good thing to eat? Laksa. What's that? That's not related to lox, is it? No, it's a soup with spice and prawns and shellfish. And prawns? I don't do... Now, how do you distinguish between a prawn and a shrimp? I think it's the same thing. It's a shrimp and fish of some sort and spicy noodles. My feeling is that a prawn is bigger than a shrimp. Like, have you ever seen a prawn that's the size of an American shrimp? I don't know. No, I don't... No. So I think prawns are always big. Is that why they call them shrimps? Because they're shrimp-y? Or it may be the other way around. Maybe they were called shrimp originally and someone... We use that as a slang for a diminutive person. I guess. I don't know. Yeah. Okay, so... What was it again? Laksa. L-A-K-S-A. So you got prawns? Yeah. Broth. Spice. Uh-huh. Now, how do you eat that? Just with a spoon? There's a spoon and there's a lot of slurping involved. It's not... So here's a challenge I have with a lot of soups. Sure. It's like, it's really hot. Yeah, and this is... So how do you deal with it? Not spicy hot. Right. You have that problem with this as well. And so is it expected that there's sort of like a five-minute literal cooling-off period? Yeah, yeah. That's fine. So you just kind of hang out? Totally. You have the soup in front of you? Yeah. I mean, it is going to be the longest five minutes you've ever had. It'll sit there salivating. The other thing is, here in Singapore in particular, they have chicken rice. Hainan chicken rice. So you are the third person to mention this now. What's the deal with chicken rice? Well, okay. So it's not actually a Singaporean thing. It's... Well, you can get in Hong Kong. You can get in Malaysia. It's everywhere but here. But they have here the... And I did this like four trips ago. They have the first Michelin star, Michelin guide, one star, hawker stall. So they have hawker stalls. Oh, yeah. Someone was telling me that's here. Right, right, right. They have hawker stalls. Hawker stalls are...

Where you eat hawk. Well, they're little sheds, basically. They're little booths. Right, right, right. And they're open. You can go to these stall centers. You know, they have like 50 of them. Right, right. All lined up in a queue. Street food is what we would call it back home. But you don't have to wander the streets. It's like a mall but it's outside. It's just a fanciful term. Yeah, it's an outside mall. And every vendor is a food vendor. So in this vendor, are they the chicken rice people? Yeah, Hainan chicken rice, specifically. And it's just rice, some like vinegar and soy and whatever. You know, there's like a few sauces and rice and then chicken strips. It's just chicken. Now, is the chicken flavored or is the rice flavored? What's the deal? I don't know. It just tastes good. You know, if you taste chicken rice... And how would you describe the flavor? It's not sweet. No, it's a little bit salty sometimes. It's nice. It's like soy sauce and vinegar. Do they include the skin with the chicken and the bones? And that's what makes it so, oh, the fat. It's just perfect. And you dip it in, you dip into the soup. You got the chicken, you got the rice, you got the sauces. Life is good. Dip into the soup. Okay. Well, maybe I'll get some of that. I've gone towards more of the, I don't know what people call it, but the dumpling things. Yes. I always like that. Sha Long Bao, that kind of thing? Sure. I don't know what that is, but it sounds good. So they have a, in China, they have this thing, in Shanghai, in that region, they have this thing called Sha Long Bao, which is hot soup dumplings. And it's the same thing. You better wait until, better wait a few minutes before you eat it because otherwise you'll... You'll burn your mouth. You'll scald your mouth. And it'll hurt so good because it does taste good. But they have a, Sha Long Bao, there's a restaurant that's pretty, I think it's from Taiwan. I could be wrong. I could be wrong on this part. There's a restaurant called Din Tai Fung, and they have that, it's international, right? So they have it in Taiwan, they have it in Singapore, they have it in Malaysia, they have it in all over China. Yeah. And in Hong Kong. And they have it here in Singapore. So you go to the Din Tai Fung, and it's famous. It's like these locations, you know, every celebrity in Asia has been to these places.

Is it like a steakhouse where they've got pictures of people on the wall who've been there? Or caricatures and stuff. Oh, even better. The one in... Small body, big head? Yeah, exactly. And signatures, too. They actually signed it, you know? So I lived in Shanghai for several months in 2012, and I just got fat on these Sha Long Bao, you know? So how do you eat those? How are they presented to you? You get a bucket, you get a wicker basket kind of with some paper on it. And there's a bunch of Sha Long Bao in there. Okay. And you use your top stick, and just carefully, when you bite into it, it's going to explode in soup, you know? And there's different flavors, like garlic and whatever, all sorts of different flavored Sha Long Bao soups. What's your favorite? Classic is great. Garlic is great. What's a classic? I don't know. It's just, it's a little bit of vinegary. It's a little sour. It's nice. It's just a soup. All right. All right. Well, now that we've got the important things covered. Yeah, yeah. The point is, Singapore is great, because you've got this nexus of all these different flavors. Yeah, yeah, yeah. Of course. Right, so you're right next to Malaysia, and they have their own, like... That's why I've been asking what people recommend, because there's so many things to choose from. And there's no real... There are some things that are Singaporean, but the best part about Singapore is the nexus of all these other things from Asia proper, like the rest of it, China, Thailand's north of here. You've got, you know... A veritable melage. Yeah, and then, oh, and the Indian food. Oh, God. And the Malay food. Just everything you can imagine. Now, what's the characteristic of Malay food? Malay food... What's its deal? Well, it's not too different, because they are just next door, right? Sure, sure. But when you say Malaysian, that encompasses the country... It's big. Well, yeah, but the country... And then there's different regions that have their own culture, like Penang. Right, right, right. It's own food, you know, thing. But Malaysia has, like, a 20% ethnic Chinese population. So you've got that going on. And then you've got the rest who are Malay. Right, right, right. Right, and so Malay doesn't describe everybody in the country. Malaysians describe everybody in the country. Sure, sure. So the Malay people predate, obviously, the Chinese, and they have a very different cuisine.

And what's their cuisine like? Oh, um, as far as I, you know, it's in some, for some parts of that cuisine, for some part of the population, it's informed by religion, which in this case is Muslim, right? Sure. So there are some things that are not in that cuisine. For sure. But otherwise, it's, I think it's fairly, I don't know, it's, I don't know how to describe it, actually. Well, let me model for you, right? I would say, and this is not complete, but... Sure. Uh, if I were to describe American food, right? We have a, we don't have a... Hold on, if I were to describe American food, I would say we have, we have a lot of meat that we cook, and we'll have sides of vegetables with it. That's kind of like a template for generic American food. Right. As opposed to the rest of the world where it's vegetables and some salad. You might fry the chicken, or you might have slices of beef, you might have a steak, but you have separate, separate sides. Now, we have another style, which I would call anything between two slices of bread. Right. And you could have a hamburger, you could have a sandwich, you could have a Reuben, whatever it is, we're going to put it between two slices of bread. Yeah. And then I think, it's debatable which one is bigger, but I would say we have the Chinese slash Mexican. Yeah. Right? And then that's a, that's another subset of food. Right. And the American Chinese food is its own distinct thing. It's from, from, right, right. Sugar with some flavors, yeah. Right, right. And, and, and us cosmopolitan worldly people be like, oh my God, it's terrible. But then, in fact, American Chinese food is kind of like chicken and dumplings, it's comfort food. Yeah. So, and then, but then we also have Mexican food, which is like tacos, burritos, occasionally enchiladas, chips. American Mexican, yeah. Right. So like, that whole collection, if someone were to ask me, how, what's the deal with American food, that's what I would start with. Well, okay, so it's the opposite for the rest of the world. The rest of the world does vegetables first and a dash of meat. Right, right, right. So if you go to China, if you go to, even here, soup is mostly broth and vegetables and, you know, mushrooms and whatever, and then maybe for flavoring, meat. Right, right. Sure. And, so now, so now back, let me take you back to Malaysia. So Malaysia actually has a little bit, I think in the, in the meat to vegetable ratio, I

think it's a little bit more even handed than, than it is, for example, China. Right. Where you have a lot of, a lot of vegetables. Oh, oh, and another characteristic of American food, not spicy. Oh, sorry. Malay food can be spicy and sometimes not. It can be a lot of fried food. There can be a lot of, you know, I don't know, it's just. Oh, now fried food, that's an important like trait of food. Like it's, it would be a slight misconception to think that American food is fried a lot. In fact, there are some special parts and not special. There's, there's some sections that there's a lot of fried food, but really you don't really see fried food as much as I think you do in other, other cultures in American food. So there's a lot of frying. Right. Well, not, yeah, at least it seems to me, but. A noticeable amount. Here's my, here's where I get into trouble is because I want, while I want to characterize certain things as being typically Malaysian, a lot, it's not, a lot of the food that's in Malaysia is actually also Indonesian. Right, right, right. So, and so you get into trouble. Well, I mean, the basis of American food is like, you know, Western Europeans and then, and then South Americans, right? South, yeah, yeah. So. Southwestern cuisine. Corn. Right, right. Maize. Yeah. Yeah. So our, our food isn't from us. Well, but I mean, yeah, exactly. So I don't, so one of my favorite dishes that I would, that I would eat all day for, I mean, I could eat it every meal for the rest of my life is Nazi goreng, which is a Malaysian, sorry, Indonesian dish. Right. But you can get it in Malaysia. So, so what is it? It's rice and meat and certain spices in the rice. And it is so good. It is, they have it actually in our hotel where you and I are staying at one of those Starwood properties here in Singapore, of which there are many. And that is not the real thing. No, no. That's like American Chinese food. You don't want to get that. It's, I mean, they do, normally our hotel does a pretty good job, but that's not, it's not the real thing. But yeah, yeah. Okay. All right. All right. Well, you've done a good job of dodging, describing Malaysian food. I'll have to go look it up in Wikipedia and I'll sort it out. Or just go try it. You can get it here in Singapore. They have it. You see, if you could identify where I would go to get it, then therefore it exists.

Or that's Malaysia. And then you could say what it was. Yeah, yeah. I mean, there's, yeah. You know who would know even better? Our buddy Adrian Cole. Oh, yeah, yeah. He probably wouldn't know. He lives in Malaysia. He probably wouldn't know. Well, all right. For the second half of our conversation. So, so, you know. You're just going to let that pass. We don't even want to explain who Adrian is. No, it's fine. People that have the internet, they can look it up. Oh, look, I'll be like you. Does anyone have the internet? Yeah, yeah. The internet. Internet. Internet. Well, Adrian. Do you have three and a half floppy disks? Anyone? Three and a half floppy disks. Exactly. Well, you should Google him. He's a, he works on the Spring Cloud team, does distributed tracing. Yeah, yeah. Open tracing. So, so, so, so you, you have this book that you wrote a while ago, and I heard the journey of the bird who flies through the clouds and everything. Yeah, from Indonesia, no less. Yeah. I like, I always like the way you, you open up your talks. Very nice. It's very important to be, make sure they understand from the outside that this is, this is not, this is not the smartest talk you could be in. Sure, sure. I don't, because they're going to arrive at that conclusion one way or another. I want to make sure that I'm the one who breaks the music. That's right. I prefer that approach as well. Yeah, yeah. The, please do not expect very much. Under promise, over deliver. Well, or at least just under promise. Yeah, there you go. That first book. So, so then, so you wrote this big book with Kenny, right? Yes. Another great person. Amazing. And what, what is, maybe there's more than one, but what's one part of the book that you find you like never talk about that you wish you would? Operations. And so, so what's in that part? It's all about, it's all, it's all, it's all. It is everything involved after you've gotten the REST API up and running or the message. So all the stuff like observability, how do you monitor it? How do you, what about remediation? How do you, what happens when the service falls down? Right. And we use Spring and Cloud Foundry in the book. The book is all about how to build apps that survive and thrive in the cloud using Spring Boot, Spring Cloud, and Cloud Foundry, right? And so one thing that we really harp on in the book is that if there's a problem you want to solve with infrastructure, you can program that infrastructure. So the way to solve problems with code is just to write more code, right? So, and that's the nice thing about Cloud Foundry is you can just, you can program it.

And there's a nice reactive Java client that people on the Spring team wrote. So it works nicely with that stack, you know? And we, so for example, if something goes wrong, you can detect, you can write sensors. You can write things that look at the output of logs using, you can use Java API to read things from LogoGator. You can start to analyze what's been going on. Yeah, and then use that to feed into event processing, basically. Say, okay, well, I've detected, or complex event processing, right? Mm-hmm. I've detected, I've detected the word error in the log two times in the last minute. Send in a pager or something, you know? Oh, yeah, yeah, yeah. And you can use, you can use Spring Cloud Data Flow to compose pipelines that respond to those errors. You can do, so, I mean, Cloud Foundry, this book predates ever so slightly some of the more complex options for load, for auto-scaling in Cloud Foundry. Oh, right, right. You put that in the updates. Yeah, well, definitely, the update is already, there's so much to do, but in the book, we demonstrate, okay, well, if there's an error condition detected by some sort of sensor, put that into a Spring Cloud Data Flow pipeline, have something on the other end, a sync, which is a component that takes inputs and then does something accordingly, have a sync that auto-scales based on whatever arbitrary condition. So you can take whatever input over here, turn that into a ratio between 1 and 10 or whatever, and if it exceeds that number, then you scale up. If it falls below that number, then you scale down, right? So where do you think the line, and just like defining Malaysian food, maybe it's very blurry, but like, where do you think the line is between the developers putting all that instrumentation and stuff in there and determining how it should behave, and then the people who are, like, operating the software? Like, who, how does the developer figure out, oh, I've done enough putting stuff in a sync. Right. Like, now I've done all I can, or all that I should, because I don't really know what I'm doing here, and someone else needs to, like, figure out what they would do with the stuff I'm throwing. So the nice thing about this is the platform has had, at the time we wrote this, three existing options for auto-scaling. And those are just knobs you could turn that were built into the platform.

And so all we're doing in this... Well, that's true. You've got limited options, so you can kind of, like... Like RAM, CPU... Or, I shouldn't say limited. You've got the perfect amount of options. Right. Well, you've got the ones that are calm, 80% cases. So by definition, you're already set for the 80% case. What we wanted to show was that if you wanted to, if there's something outside of that 20%, outside that 80% in the 20% case, that you had the toolkit there to build that, right? Right. And so operators, this would be an operator solution first, and only then does it turn into a developer solution. And if it's... Hopefully, at that point, it doesn't go back to the operator. Hopefully, at that point, we've solved it, you know? Yeah, yeah. But if you do need to, again, you can drive alerting based on this. You could... Again, I showed... I mentioned having the output of those sensors feed into something that does auto-scaling, but the sync that does auto-scaling could just as easily be something that sends an email by using SendGrid or something, right? Yeah, yeah, yeah. Or PagerDuty or whatever. Right. And then, so, like, we... You know, you mentioned Adrian earlier, right? Yeah. I assume he still does this, but, you know, he's worked on distributed tracing and things like that. Open tracing, yeah. And it seems like a large part of open tracing, as you say, or whatever, is having the developers properly instrument their code somewhere, right? Like, because, you know, it's entertaining to see the execution path of something, right? Like, I guess what I'm getting at is there's two models of doing tracing. One of them is from the outside, and you're kind of looking at the opaqueness. Right. You can't really get into the code. And another one is, like, it's actually... How would it go? Outside, no, inside out. Right. Right? But that implies that developers... Need to be aware. Someone has come around and, like, stuck things in the code. Now, maybe they do it, like... I don't know what the Java kids call it anymore, but maybe they do it after it was compiled and they reflectively put it in or whatever. But there's, like, actual getting in there and messing around with stuff. Right. And so is that accurate, first of all? And then so are there... Like, what's emerging is, like, if you're going to do this tracing, here are the things you should trace. Or does it... There's no common pattern. It's just, like, whatever.

How do you figure out what to put in there? Well, okay. So, yeah. So the platform will give you... The platform will give you the opportunity to... Like, for example, these days we have Istio, right? We can use Istio. Istio is an ideal fit for ingress coming into the application. But once you've passed that threshold from the cloud controller or whatever, then it's up to your code. So there's actually intra-process, not between processes, but inside one single process. There's actually stuff happening inside that opaque black box, right? So if you're using something like... If you're using a service mesh of some sort, or a route service in Cloud Foundry or something like that, right, you can easily add tracing in that layer. Right, on the outside. And that'll give you the network. That'll give you the, you know, one hop to another. But when you want to go a step further, you need something that you can use for component boundaries, if not network boundaries, right? Yep, yep. And that is where you need to have code in your application. So these days we have Spring Cloud. Spring Cloud has an SPI called Sleuth, S-L-E-U-T-H, which is one of the hard... I get so many questions about that. You wouldn't believe the proportion of the world for which that particular English word is a troublesome word. Oh, yeah. Just knowing what it means or saying it? Just pronouncing it and all of the above. Yeah. It's just not a word that is easy to... That is a weird word. I have to look up the etymology of that. That feels like a word that came from, like, the Normans. Not the Normans. That came from, like, Vikings or something. Right, yeah. Something that was designed to flummox people a thousand years later. Oh, flummox. See, that's in the same school there. Darn it again. Either way. Yeah, so we have this SPI called Sleuth, of which we have one implementation that supports Zipkin, the Twitter distributed tracing platform, on which, of course, Adrian works. And so Sleuth provides hooks. We implement the Sleuth API. We use it in Spring Cloud and in Spring. So everywhere you have ingress and egress, whether it's across the network or from one

service to another or from one component to another, that's useful to have that, we do it. Right. And then if you add the Zipkin binding for Sleuth, then it automatically publishes those bindings out of band over either HTTP, which is not the best choice because it's synchronous. So your request arrives and then something sends an HTTP request. Well, I don't want to sit there and wait for an acknowledgement. It's for all those curl programmers out there. Yeah, don't do that. Or you can use a broker. So you can publish it, fire and forget. And this is automatically done for you. So you make a REST call in your Spring code, for example, or if you invoke another method or whatever, you can tell Spring, I want all these things to be sampled or some percentage thereof. And you can prescribe a sampler. And Sleuth will publish that to whatever, you can say to publish it over Kafka or RabbitMQ. And then those messages eventually arrive at Zipkin itself, which then assimilates all the trace information into a graph. So we've got this wonderful framework. Right. And then equally wonderful is that you have a standardized way of doing it. Right. Like, you might even call that a, what are they used to, I never got a computer science degree, but a data structure. Yeah. Like, it's like, this is the way that you format stuff. This is the expected interaction pattern. So you don't invent that every single time. And actually, you don't even deal with it. So if you're using Cloud Foundry and you're doing something smart with the cloud controller and the route service or using a service mesh, or if you're using Spring Cloud Sleuth, there is no, you have the library, you have the dependency on the cost path, that's it. Right, right. So you're not actually. So then, like, one of your, if not the most common demo is a reservation system. Yes. Right? And if you remember the term business logic. Yes. Some places then there is some business logic. Yeah. Not much. Yeah, not much. So maybe this is a hard question and it automatically gets done. But for those little parts where there's business logic. Yeah. And let's assume you had to put in the tracing stuff.

Right. Like, what would you put in? Ah, so if you wanted to instrument particular paths yourself. Yeah, yeah. Like, we're really inside the black box. So you can inject. Not only the application, but the actual, like, business logic-y code. Well, okay. So you can do that. You can inject what's called a span accessor. I think that's what it's called. It may have changed, actually. But we just moved some of that API to somewhere else. No, it's a different. Slightly changed. So it used to be you can inject something called a span accessor. Uh-huh. And then you would, basically there's some way by which you can get access to what we call a span. Uh-huh. Yeah, yeah, yeah. That's right. Which is, like, a segment of a process that's running somewhere. It's a hop. Yeah, yeah, yeah. That's right. That's right. So you have the trace, which is the aggregate A to Z. Yeah. I forgot I read about that. Right. Yeah, yeah. That makes sense. And then E to hop in the journey is called a span. So you might have. I just impressed myself there. I've actually read up on this stuff. You're pretty awesome. You should. But, like, I mean, what I'm getting at is, like, what do you want to log in the business logic? Oh. Right? Like, and what do you put in there? If I have a five-line method call. Yeah. That's just writing data to the database, I might want to trace that in addition to what Springwater automatically do. So you might say, if I remember your reservation system, I want to look up a person. Sure. Right? And one of the things you might log is when I get this person's name, I want to say, here's the name that I got, and here's what I looked up, or something like that. Now, the issue is that a lot of that could automatically be done for you, but, like, I mean, so that's why the reservation system's hard. But at some point, you want to know that, like, someone searched for the Coté Reservation, and this happened or not happened. Like, here's something that I'm sure, I don't know if this happens to you, but this happens to me all the time, and it's infuriating, is, like, on my passport, my name is Michael Richard Coté. Okay. Right? And for whatever reason, in Concur, my name is Michael R. Coté. And so the computers get very confused. Yeah. Right? And it causes all sorts of problems. And so at some point, I feel like there should be some tracing that's like, Michael Richard Coté came in, but then we were looking for Michael R. Coté.

Awkward. So what, and, like, how do I log that or whatever? Well, okay, so tracing isn't a great fit for that. Right, right. Because, so you can include context about the journey of a particular message, but the goal isn't so much to record the context, it's about to record the message. So basically all I'm doing in tracing is, whatever span I'm operating in, whatever sub-segment of the overall transaction, I'm just kind of throwing extra stuff in there. Meditate it if you want. That might be useful at some point. But you shouldn't use this as a way of finding information about that. So you'd have a whole separate system that was tracing, like, this name. Well, because, and the reason is because sampler, the reason is because your distributed tracing system isn't going to keep by default if you're doing something, unless you're doing something terribly wrong, which is, by the way, what I do on stage, and I tell people don't do it. Right, you're like, I'm the number one contributor, the number one contributor of, the number one contributor of bugs. But no, but I tell people don't, because in your demo, in your exam, in my demo, I record every single trace. Right, right, right. Every single request response anywhere in the system I record. So that's why my question is going to worry. But you're not going to do that for most people. You're actually recording, you're tracking the execution of the program. Right, but most people don't, you would overwhelm your tracing system if you did that. So by default, Spring Cloud Sleuth would capture 10%. Right, right, and then you do, like, I don't know, numbers, but you do the statistical thing where it's a sampling. Yeah, exactly. And you can provide a custom sampler, which Spring Cloud will consult to determine whether it should trace this request or this span. So my point is, that information is incidental. It's nice to know it was Michael, but we're more interested in the timing and the latency. And as I found out by the airline systems in the world, no one cares if that error happens. No. Yeah, no. They just return some red text on the web page and they say, must check in at desk. All the time. Oh, that's so annoying. Yeah. Like, you would think we would use some of our Bayesian big data machine learning to figure out that, like, this is the same person. It can't be done. Huh. All right, so is there another section that you feel like doesn't, that was operations? Yeah, and that's a big one. And then we burrowed into, like, tracing and sleuths and Zipkinites.

No, that's covered in that chapter as well. Right, right. And also actuator, the observable. So sampling for traces is one thing, but also metrics. Oh, yes. And these are two things, these are two sides of the same corner. What does metrics mean in your world? Metrics are, I want to accumulate keys and values and calculate statistics based on those. Like, wait, what's a good example? So if you think about... Or a bad one. Well, okay, so imagine, so now, instead of asking about Richard or Josh's particular transaction, I want to know just how many transactions I've had, period. I want to count every single one of them. But I'm not going to store a heap of metadata about each one. I just want to increment a counter somewhere. Right. And then I want to be able to see a subset for that. I want to say, okay, these are all, for example, all the HTTP requests. I had one million of them, right? And then I could say, oh, well, how many of them were HTTP GET requests? Well, there's, you know, 80% of those. And then how many of them were HTTP GET that returned 200? Okay, so 99% of those. And so you keep different derivations of these numbers. But they're still just numbers. And you can keep those in your Java process, if you like. But that's pretty useless, because then you'd have to visit each node to see the stats about that process. Or you could publish them to a time series database or a monitoring tool. And so we have now this new framework in Spring Boot. Well, we have this framework that we Pivotal developed. We hired a guy named John Schneider from Netflix who, you know, they're not terrible at operations and monitoring, are they? So we thought he'd be a good fit. So he came in and he built this framework called Micrometer, which is you can use it independent of Spring. But why you would, I would, you know, it's not a thing. Well, developers like to do all sorts of crazy things. They're crazy, wild and crazy, yeah. So anyway, this Micrometer framework, it's a metrics facade. So you use one SPI, one SPI, and it'll publish based on whatever adapter you've contributed, you've configured. It'll publish those metrics to things like Prometheus, Graphite, OpenTSDB, Dynatrace, Atlas, you know, everything you can imagine.

Datadog, SignalFX, InflexDB. And so these tools will then give you the ability to drill down, to draw pretty graphs. Right. So you pump your, from their perspective, raw metrics into this. And then they know how to analyze it and make sense of it, or make a sense of it. I don't want to be mono-judgmental and say there's only one sense. But like, you know, they make sense of all this stuff and do something with it. And you can query into that now. So again, Spring Boot and the platform will both give you automatically certain metrics that are interesting, like CPU usage and number of HTTP requests. But you can also say, okay, all these HTTP requests, you know, I don't want to know just the HTTP requests. I also want to know how many people have checked out the shopping cart. How many people purchased this product? How many people visited this page? How many people signed up for new accounts today? These are key... Right, right, right. What is that? Performance indicators? Yes. KPIs. My brain's a little... There you go. So you can have business metrics as well as technical metrics. And this informs both operations and the business, right? Right. And developers. So you get that ever-important single pane of glass, you know? Right, right, right. I have two more questions. Okay, I've got them. Are you ready? Then we'll wrap up one. So what's the deal with time series databases? Why do you just call it a time series database and not just a database? Well, yeah. It's like we have document databases, we have key value databases, we have graph databases, and we have time series databases. So a time series database is like a dictionary, like Redis. It stores the keys and values. But it also stores every iteration of that value over time. Right, right, right. So you can look at the history of that value. You know, what number was it a minute ago, a week ago, an hour ago? And then because you have that availability, you have that visibility, you have that data, you can then feed that into a graph and then plot trends. Right, right, right. And you can summarize it if you wanted to. And I guess more of my question is like, so do you call it a time series database? Is it because it's built around doing only that? Yeah. And so most performant doing it? Or because you could use that data structure anywhere, right?

Like you could just store that wherever you wanted. But it may be a bad idea performance-wise. Right. But, you know, you could make a relational database that's stored time series stuff. But it may technically be a bad idea. Terribly, yeah. And so the reason you call it a time series database is because it's optimized for that way of storing things. And that's an important, this is actually a burgeoning community right now. Netflix Atlas is a fairly new technology that was open sourced in the last five years, I think. Prometheus is super new. That came out of, I think, SoundCloud.com. This is, and there's, there's a, the, you know, do you ever read the morning paper from Adrian Collier? Sometimes. Yeah. He's a, he did one that looked at a time series database a few months ago. And it was, it was something out of academia. And basically this was like, it was optimized for huge data sets, like, you know, trillions of records or whatever. So do you think there was a time when sticking everything into a relational database was a good idea? No. Because that seems to be what we had for like 20 years. It's like, I think the last thing before this whole no SQL, no one calls it, but whatever, was basically, remember object-oriented databases? Yeah, yeah. Like that existed. And then everyone was like, well, that's not cool. I don't know about that. And then we went back to RDBMSs forever. And so like, whereas now we have the opposite, like as we were just discussing, we're like, you want a database that fits the type, not even the type of stuff you're putting in there, but the way that you want to retrieve and conceptualize the stuff. Purpose built for certain spaces. Right, right. But what matters, what the database is doing is servicing pretty much you, the human, right? Right. You want this in a document format. You want it in a time series format. It's not you, the human. It's the representation of that data that you're serving. Oh, yeah. What I mean to say is that we humans are programming and so we have a preference for the way that we arrange things. Right. If you ask the computers if you could, they would probably be like, I don't know, make it numbers. Sure. Right. Like, zero and one. Like, whenever all these databases store something, it's probably not in a format that humans would find useful. Right. So. No. So, like, the whole point of all these different databases is, like, we want our data to be arranged in these different ways.

Now, it's not purely our desire. It might be because we're doing this stuff, it has to be in this format to optimize it. But we make those choices. Right. So, in contrast, we used to be like, I'll deal with relational databases. Right. Like, it didn't even occur to us that we could have an opinion about, like, what things would look like. I think, so I think a lot of, yeah, okay, so fair enough. But I think it's weird because any programmer with her or his salt would know that you need a data structure suited to the data. Right, right. No, no. And that's a, you would have your ORM tools or whatever that would take stuff from the database and get it in a format that works for you. They would, the thing is, we would, we all know to use a data structure, but then you get to the database section. You're like, oh, I give up. I guess I just don't know how to do any of this. Exactly. I've forgotten how to do data structures. That's kind of the weird shift that happened. It's like the, to put it in this term, the taboo of making demands on the database was somehow removed. Like, we lived under this error for a long time. Whereas you characterize it, it was just like, no, you can't tell the database what to do. It does the one, it's relational, and that's it. And developers would forget. They knew all the, like, in the code themselves, they'll sit there and haggle. To, to no end about what data structure we should use. That's right. But then it's like, oh, I need to persist this longer than the session of my current running application. I guess we only have one choice. There's only one option here. Maybe, and we'll skip my last, but closing out with food, right? Okay. For a long time, it was like, a taco is the following. You either have a crispy shell or a soft shell. Yeah. Or it's not a shell. And you're going to put some spicy meat into it. Mm-hmm. Or, if you're a vegetarian, we'll figure that out for you. But whatever. You have some ground meat or some chicken or some skirt steak that's been marinated and made into a fajita. And then you're going to put some cheese on top of it. And then you're going to put lettuce on top of it. And maybe some tomatoes, and if you're lucky, some guacamole, right? You're making me hungry. That was like 30 to maybe 200 years of tacos, right? And then at some point in the last 15 years, someone was like, you know, we could change the taco. Right? We could make demands on the taco. It's not a taco. It's not a taco. And we could put, like, bulgaki in there.

Oh. Or, like, we could put, like, what if I put, like, a sushi roll in the taco? And nowadays, you put whatever you want in a taco, right? Yeah, you're right. So, similarly, like, it used to be, like, you go to the relational database, and it was just, like, rows and columns, buddy, and inner and outer whatevers. Like, that's what you got. You got tacos a taco. But then someone decided, no, I'll do whatever I want. You know, I think, and I think they decided this because, well, there's two things. I think there's several factors that play into this. First of all, data got bigger. I mean, we all, like, physicists have been looking at this going, yeah, it's always been big. It's like Ant-Man. It's bigger. Yeah, well, now, yeah. So, and by the way, that new movie, Ant-Man and Wasp, looks amazing. Oh, yeah? Anyway, we had, data's gotten, okay, it's gotten bigger for Enterprise. Uh-huh. I should caveat that NoSQL has not been taboo, specialized data stores have not been taboo in, you know, science. Oh, sure, sure. Physicists generating billions of events. But they're not real programmers, are they? They're much more than real. They're, like, they're actually doing really hard stuff, right? There's no, nobody's ever bothered to invent Visual Basic for physicists, you know? No, no, they probably wouldn't use it. No, because it's just way too hard. Right, right, right. But, so that, they've always used specialized data stores. They've written their own databases, you know? And, and by the way, it's, it's for that reason that it's no small coincidence that Dr. Dave Pollack, sorry, Dr. Mug Pollack, on the Spring team, his previous life was in physicists. He was in physics. Right, right. And he was the first founder of Spring Data, which is our project for specialized data stores, you know, SQL, whatever. Right, right. Right, because that was, his background was doing that, you know? Anyway, so, first of all, it got more, it got bigger, right? And then we, we started needing to be able to query it. We needed to be able to query this data in particular ways. Right. And still be performing, right? We couldn't, we have, we move more and more things online. And so, if it's offline, then yeah, you can afford to take this database, which is in a representation. Get it at a warehouse and put it, put it in whatever format is handy for querying it. But, but yeah, but okay, there you go, that's a good point.

The warehouse model, that, that offline, you know, non-online transaction, that, that's the exact sort of thing that we did before. But we just assumed that you would only access that warehouse ever so infrequently. So even there, you had it in your SQL database, and then you'd turn it into something that would be optimized for reporting. Yep, yep. But now, that reporting has to be up to date almost instantly. And so, and so my theory, it would be that there was, there was a, point is the wrong word. There was, there was a time series, points of time, so to speak, where maybe it was not ideal, but the most efficient, optimized way to do stuff was to have a relational database, right? Like, it was like the tool that worked the best. I don't know about that. And again, and, and, and, and, and again, we can invalidate this theory at some point. But then, as you're saying, the needs for data changed, right? Right. And then also, the technologies changed such that we could actually make something better than an RDBMS. Well, we, because, okay, so I'm, I see what you're saying, but I think, I would say that we've always had specialized databases. Right. But the performance wasn't so, so provably bad that we were forced to care about it. So, so this, this, the, the amount of data that we have and the speed at which we want to do something with it. Yeah. Sort of like, what did it do? It exacerbated the problem. Whatever a nice way of saying broke the backup is. Right. That sounds very violent. But, but it, but it, it's, it's superseded the, the functionality of the relational database. Yeah. And, and, and I think we all, and, you know, solve some, in doing so, in changing that, we've, we've fixed some problems and spawned a whole industry of vendors who want to, who want to create more problems. Right, right, right. Well, every, every solution today is tomorrow's problem. And, oh, golf game. Golf game. There's always a golf game behind some terrible solution deployed in some, some, you know, vendorware. That's right. All right. Well, well, I've got to give a talk. Oh, yeah. And then maybe. What's it going to be about? Oh, it's just, it's just me trying to figure out what Kubernetes is. And then maybe, maybe I'll get some chicken rice.

Honey and chicken rice? Or something. I don't know. We should go together. Yeah. That would be great. All right. Well, as always, this has been Pivotal Conversations. Why don't you tell people where you are on the Twitter? Oh, find me at, at, at, at Starbucksman, S-T-A-R-B-U-X-M-A-N. And, uh, you know. And do you use email, Josh? Email? Yeah. Email? You can. It's, it's a little async, you know. Telephone. Josh at joshlong.com. You use voicemail. But, but also, um, uh, check out the, uh, the Gitter. You know, you know, Gitter? Do people use Gitter anymore? We have a G-I-T-T-E-R. Is that like Finger or something? No, the, uh, the chat room. It's like a, it's like a open source. It's not, it's, it's like an open Slack. So it's G-I-T-T-E-R dot I-M for slash. And then Spring Hyphen Projects. And you'll find chat rooms for all the different Spring Projects. Oh, I see. And what's the, what's the gender neutral word for manned? For manned? Like manned the ship. Staffed? Staffed. Thank you. Thank you. Staffed by the Spring Engineers. Ah. So if you have questions, comments, feedback, whatever, we are, as always, happy to engage. That sounds good. And also you can find all the old episodes at soundcloud.com slash pivotalconversations. Now do you think there's a space in that URL or an underscore? What do you think, Josh? Underscore? No, no underscore. Trick question. Just all, all one word, no spaces. That's the worst. Yeah. No, no percent 20s in there. See, my mind would want to forget that. I would actually try to forget that. I know. And that's why I like to tell people, Pivotal Conversations, no spaces, hyphens, other special characters. So, you know the one thing you would never think to do? Do that. That's right. That's right. Anti-Larry Wall. Isn't he the one who said software should, no, I made this mistake the other day. It was one of the pragmatic programmers who was like, software should do what you expect it to. Oh. Which, which I think that's like saying food should taste good. Right. So, you know, that's still good. It's true, but kind of useless. Yeah, yeah. So, we'll see everyone next time. Bye-bye. Bye.

Tanzu Talk · 🍕 Get more pizza.