1
00:00:00,000 --> 00:00:04,180
So why don't you introduce yourself yet, because I had a few questions right off the bat.

2
00:00:04,280 --> 00:00:05,740
Hey, yeah, I'm Ivan Novick.

3
00:00:05,860 --> 00:00:06,820
I'm at Pivotal.

4
00:00:06,940 --> 00:00:11,840
I'm a product manager who works on Pivotal Greenplum, which is a data analytics database

5
00:00:11,840 --> 00:00:17,240
used throughout the world for storing and analyzing big volumes of data.

6
00:00:17,380 --> 00:00:20,960
So I was watching, I think it was one of your videos.

7
00:00:20,960 --> 00:00:25,460
I was watching a video essentially about Greenplum, but then it turned out delightfully to be

8
00:00:25,460 --> 00:00:26,560
History of Databases.

9
00:00:27,240 --> 00:00:28,300
You did that one, right?

10
00:00:28,880 --> 00:00:29,340
Yeah, yeah.

11
00:00:29,440 --> 00:00:30,300
That was awesome.

12
00:00:30,740 --> 00:00:32,000
Yeah, I've done that a few times.

13
00:00:32,180 --> 00:00:35,600
So the question I had, I mean, just to boil it all down, like it was fun to see, like,

14
00:00:35,640 --> 00:00:37,660
you know, you got the, I'm going to mess it all up.

15
00:00:37,700 --> 00:00:42,580
But there's basically like, you know, the Apollo mission has a database and then there's IBM

16
00:00:42,580 --> 00:00:46,780
and then there's still IBM, more IBM, very IBM.

17
00:00:46,780 --> 00:00:53,240
It started with the census, the U.S. census back in, with punch cards, right?

18
00:00:53,320 --> 00:00:58,880
You had punch card computers and the first usage was to add up all the people

19
00:00:58,880 --> 00:01:02,980
in the country, in the U.S. and put them in some sort of a calculating device.

20
00:01:03,180 --> 00:01:03,600
That's right.

21
00:01:03,700 --> 00:01:04,040
That's right.

22
00:01:04,160 --> 00:01:10,080
And so to my question, which at some point you went over, but it's always curious to hear,

23
00:01:10,160 --> 00:01:14,740
is like, when did the jump over, well, let me phrase this another way.

24
00:01:14,740 --> 00:01:20,900
Why do you think it is that, like, the relational database became, like, the king of databases

25
00:01:20,900 --> 00:01:22,020
at some point?

26
00:01:22,360 --> 00:01:28,640
Like, what about it and the time that it happened was sort of like, and what I mean by king of

27
00:01:28,640 --> 00:01:36,420
databases is, like, I feel like maybe by revenue, by number of instances that were out there,

28
00:01:36,520 --> 00:01:42,980
by people using it, by simplistic ways of saying it was the king of databases, that it was sort

29
00:01:42,980 --> 00:01:48,300
of, it's for many years, it probably still is, was just like, that's what people do.

30
00:01:48,300 --> 00:01:53,220
In the same way that you would say, like, the combustion engine is the king of locomotion.

31
00:01:53,480 --> 00:01:59,260
Relational database is based on set theory, relational algebra and set theory.

32
00:01:59,480 --> 00:02:05,240
So, you know, at a high level, you've got a set of data and you want to reason on it in

33
00:02:05,240 --> 00:02:12,680
terms of overlaps, intersections, unions, you know, you think of your Venn diagram, right,

34
00:02:12,780 --> 00:02:18,460
in terms of, you know, I want to, you want to select and understand the data grouped by

35
00:02:18,460 --> 00:02:18,920
the data.

36
00:02:19,180 --> 00:02:26,480
And it's a, it's a natural way to, people want to think about and process the data.

37
00:02:27,920 --> 00:02:34,860
Is to, is, as opposed to, for example, programmatic processing of data, where you say, you know,

38
00:02:34,880 --> 00:02:39,180
I'm going to write a for loop and just write a program to process data.

39
00:02:39,720 --> 00:02:45,340
And you say, while something, I mean, that's a very kind of cumbersome way to, to deal with

40
00:02:45,340 --> 00:02:46,100
large set of data.

41
00:02:46,240 --> 00:02:54,980
So a more natural way is to say, okay, here's a table of, of orders and a table of products

42
00:02:54,980 --> 00:02:56,240
and a table of customers.

43
00:02:56,240 --> 00:03:01,340
And I want to understand which customers overlap, which orders, which overlap, which products.

44
00:03:02,220 --> 00:03:09,720
So fundamentally that was brought up by EF COD in the seventies or eighties.

45
00:03:10,300 --> 00:03:11,880
And it's stuck, right?

46
00:03:11,920 --> 00:03:16,620
It's stuck as a way to, to store and query the data.

47
00:03:16,620 --> 00:03:21,820
I mean, is it, is it sort of like, was it driven by like cost or performance or maybe

48
00:03:21,820 --> 00:03:24,000
even like cognitive knowability?

49
00:03:24,160 --> 00:03:27,320
And, and, and I asked on the third, cause like, I remember when I was learning relational

50
00:03:27,320 --> 00:03:30,720
databases and like, it made no sense to my feeble human mind.

51
00:03:31,500 --> 00:03:34,800
And more specifically, someone who didn't know set theory very well.

52
00:03:34,800 --> 00:03:38,620
Like it was just like this whole idea of like joining, putting stuff together always seemed

53
00:03:38,620 --> 00:03:39,480
very like weird.

54
00:03:40,120 --> 00:03:43,740
So it's like, and then, and then there's the whole like object relational mapping thing,

55
00:03:43,800 --> 00:03:49,820
which is this whole sub industry, which is like the translate between object orientation

56
00:03:49,820 --> 00:03:51,740
and like relational databases.

57
00:03:51,740 --> 00:03:59,180
So like cognitively, a relational database is like its own island of fun, at least from my

58
00:03:59,180 --> 00:03:59,600
perspective.

59
00:03:59,780 --> 00:04:04,500
And so like across those three characteristics or maybe a fourth of like cost performance and

60
00:04:04,500 --> 00:04:10,280
like a knowability, like, I'm always curious, like why that became the way of doing things

61
00:04:10,280 --> 00:04:16,100
instead of like poet or whatever object or in a database was out there, which seemed like

62
00:04:16,100 --> 00:04:16,480
cooler.

63
00:04:16,660 --> 00:04:23,100
Well, and, and poet came and these, these object databases came after I think relational was

64
00:04:23,100 --> 00:04:30,660
already established, but, um, and, and, and obviously never got, um, critical mass or traction

65
00:04:30,660 --> 00:04:31,860
because.

66
00:04:32,260 --> 00:04:32,440
Right.

67
00:04:32,500 --> 00:04:32,700
Right.

68
00:04:32,700 --> 00:04:37,020
And, and, and, and I guess, I guess I bring them up because whatever that initial power

69
00:04:37,020 --> 00:04:43,160
of the relational database with was, I guess, I guess something can clobber itself by being

70
00:04:43,160 --> 00:04:47,140
insufficient, but I feel like the relational database was just sort of like pushed all the

71
00:04:47,140 --> 00:04:50,880
O databases aside and was like, no, no, no, I'm still going to rule the nineties.

72
00:04:51,320 --> 00:04:51,660
Sure.

73
00:04:51,660 --> 00:04:57,140
So, I mean, so, so the sequence was, there was something called some sort of a hierarchical

74
00:04:57,140 --> 00:05:02,460
data, hierarchical database, um, that, that IBM was pushing.

75
00:05:02,700 --> 00:05:09,260
So, so, so basically the first thing was, um, the first fundamental thing was the electricity,

76
00:05:09,260 --> 00:05:09,780
right?

77
00:05:09,780 --> 00:05:10,600
The electric computer.

78
00:05:11,100 --> 00:05:16,880
So we went from punch cards to electric computers and the first databases on electric computers

79
00:05:16,880 --> 00:05:22,260
were, um, obviously from IBM because IBM was dominating computer industry.

80
00:05:22,260 --> 00:05:27,660
And so, um, they were, there was a system called, I think IMS was the name of it.

81
00:05:27,660 --> 00:05:32,620
And it was a hierarchical database, which to be honest, I don't know too much about because

82
00:05:32,620 --> 00:05:34,020
it's basically obsolete right now.

83
00:05:34,220 --> 00:05:41,040
Then, so the reason relational came in, uh, as I mentioned before was that it was, it had

84
00:05:41,040 --> 00:05:47,720
a kind of a mathematical, um, backing and it also was business practical.

85
00:05:48,180 --> 00:05:53,620
So from a, and I think the key thing on the price performance, all these things was, was

86
00:05:53,620 --> 00:05:59,400
to be honest, the usability and the, the, um, kind of from a business point of view, the

87
00:05:59,400 --> 00:06:08,940
ability to ask a question and really not write any software but just store the data and ask the

88
00:06:08,940 --> 00:06:13,880
questions that need to be asked to generate business reports so it mapped well to to business

89
00:06:13,880 --> 00:06:19,900
and when it maps well to business then that drives economics right because there's only so many

90
00:06:19,900 --> 00:06:26,800
people who want to buy a huge database not for business right yeah yeah and and then and then uh

91
00:06:26,800 --> 00:06:32,820
like like i mean maybe the guy i think the point you you make good is it's sort of like it's useful

92
00:06:32,820 --> 00:06:39,640
yeah right and it's it's it's easily useful the usability on a tie and then and then my last

93
00:06:39,640 --> 00:06:47,600
like uh you know walk down the uh the ivy encrusted memory lane path so how how would you describe

94
00:06:47,600 --> 00:06:54,360
the relationship between spreadsheets and relational databases right because it it seems like there's

95
00:06:54,360 --> 00:06:59,500
a uh i don't know if it's a carton horse and a chicken and egg type of thing but they seem like

96
00:06:59,500 --> 00:07:05,320
very closely tied together i think they are yeah they operate they are i think they are very closely

97
00:07:05,320 --> 00:07:13,380
tied and microsoft excel probably economically is equally successful as any database commercial

98
00:07:13,380 --> 00:07:24,220
um it doesn't it so the fundamental so so okay it brings to the one of the key points about the database business which is

99
00:07:24,220 --> 00:07:33,780
it's extremely hard to implement and there is no solution that has been created by any humans that is the ideal database yet

100
00:07:33,780 --> 00:07:43,360
right every major database product has clear known um compromises made whether it be around scalability

101
00:07:43,360 --> 00:07:51,160
or or mostly around scalability and how they do trade-offs so the trade-off excel makes one of the main trade-offs excel

102
00:07:51,160 --> 00:07:56,820
makes it's a single user system right i mean yeah sure you can share the file but you can't really have

103
00:07:56,820 --> 00:08:04,860
100 people edit the same spreadsheet at once well i guess with google spreadsheet that changed but not um

104
00:08:04,860 --> 00:08:09,060
not atomically right if you want to remove money from from one bank account and edit to another

105
00:08:09,060 --> 00:08:15,860
that's one of the principles of relational databases which is the acids the the that when you do a

106
00:08:15,860 --> 00:08:22,100
transaction it's guaranteed to either happen or not happen right so you can you can account you can have an

107
00:08:22,100 --> 00:08:27,820
accounting of the world you can say this is the current true state and with locking you can have the

108
00:08:27,820 --> 00:08:33,800
different people update the data and so if you wanted to just create a single user database that

109
00:08:33,800 --> 00:08:40,300
can pretty much be done in a college project right but once it becomes a billion dollar investment is when

110
00:08:40,300 --> 00:08:47,460
you want to have concurrency and scale so excel is super useful for a person who just wants their own

111
00:08:47,460 --> 00:08:56,700
to look one person to look at some data um it doesn't really have easy ability to do let's say um let's

112
00:08:56,700 --> 00:09:02,600
say you had one spreadsheet of orders and one spreadsheet of customers and then you say okay i want to see

113
00:09:02,600 --> 00:09:09,120
give me a report of the average order size per customer i mean it could take a while to to make that come

114
00:09:09,120 --> 00:09:15,240
out whereas it's one line of sql in a database right yeah that's true i i think i think i spent many

115
00:09:15,240 --> 00:09:21,480
late nights trying to understand the theory of pivot table just like i still don't know i still don't

116
00:09:21,480 --> 00:09:30,440
know how to use pivot table exactly anyhow uh we'll come back to uh to uh more database stuff talk about

117
00:09:30,440 --> 00:09:36,980
green funds before that uh there's just a few little news items to go over now you know there's there's

118
00:09:36,980 --> 00:09:44,560
uh around here we had uh we had a big relief pivotal cloud foundry 2.2 now uh i think it was jared who

119
00:09:44,560 --> 00:09:49,180
wrote this stuff right like i think i think i still i'm going to set aside four hours to read that blog

120
00:09:49,180 --> 00:09:56,140
post which which i mean more of which i mean more of a complimented way of like uh there's a lot of

121
00:09:56,140 --> 00:10:02,820
stuff on there and uh it's very well documented but there's a there's all sorts of uh under the hood

122
00:10:02,820 --> 00:10:08,700
i don't need like one of the i was trying to pick out one that's notable and i think uh having more

123
00:10:08,700 --> 00:10:15,040
detail and controls over auto scaling is always nice because that's uh one of the immediate useful

124
00:10:15,040 --> 00:10:22,220
things of of doing anything cloud related is having elasticity and automation uh something like scale

125
00:10:22,220 --> 00:10:28,120
so let's see that we have more improvement there but how about how about yourself richard if you were to

126
00:10:28,120 --> 00:10:34,000
pick out one or two things to highlight what what are your favorites in there yeah i'm glad you like

127
00:10:34,000 --> 00:10:39,560
jared patented a giant list of things blog post structure for announcements but he makes good use of

128
00:10:39,560 --> 00:10:45,500
headings and pictures that makes it very easy to read so that's a patent pending i was i was i always

129
00:10:45,500 --> 00:10:52,460
awaited the uh back in the early 2000s when i was a programmer i waited each release because they would

130
00:10:52,460 --> 00:10:59,480
have like 20 page release and it was sure i've been working to read the release note yeah so this is a

131
00:10:59,480 --> 00:11:04,780
good one as you mentioned i think the auto scaling stuff which uses this new log cache subsystem for

132
00:11:04,780 --> 00:11:09,220
for reading logs but i think what's pretty cool about the auto scaling is yes you can do custom metrics but

133
00:11:09,220 --> 00:11:15,120
there's even some smart things on comparisons like hey if the maximum is different than this you know

134
00:11:15,120 --> 00:11:19,800
other minimum and both may be custom metrics you know like let's say the you know the current is

135
00:11:19,800 --> 00:11:24,480
approaching the max maybe that's when you auto scale so it's not even just like hey this number hits some

136
00:11:24,480 --> 00:11:28,940
sort of threshold maybe it's actually based on two different values coming from the app or from the

137
00:11:28,940 --> 00:11:34,540
service so some really cool stuff coming available now in auto scale and that it's a in pivotal web

138
00:11:34,540 --> 00:11:38,980
services as well that's always where you can see these things before it hits the the software itself so

139
00:11:38,980 --> 00:11:44,900
that's pretty cool uh the other thing that jumped out to me i think was bosch and credhub

140
00:11:44,900 --> 00:11:50,200
support so bosch credhub is generally available the runtime credhub is available this has been

141
00:11:50,200 --> 00:11:56,140
a component a part of cloud foundry for a year or so now but it's kind of continually been beefing up

142
00:11:56,140 --> 00:12:01,440
it hasn't been just released at one time so now credhub has hit a really nice mature point

143
00:12:01,440 --> 00:12:06,780
storing credentials securely for your apps for the different credentials within cloud foundry itself

144
00:12:06,780 --> 00:12:11,340
so really neat to see that available and then just some some good infrastructure things supporting

145
00:12:11,340 --> 00:12:17,080
azure stack formally supporting things like deploying to multiple v centers as part of like

146
00:12:17,080 --> 00:12:22,080
an az and high availability architecture that has been there before but there's been some nice new

147
00:12:22,080 --> 00:12:27,720
support for supporting multiple data centers on v sphere so getting these multi-site deployments is

148
00:12:27,720 --> 00:12:31,720
tricky so we want to make sure that we make that simpler so some good things in there definitely read the

149
00:12:31,720 --> 00:12:36,400
giant release notes that we'll add in the show notes but good release we keep marching forward

150
00:12:36,400 --> 00:12:40,820
with something good every quarter and it and speaking of releases although in a good way i i haven't

151
00:12:40,820 --> 00:12:45,420
had time to look at it but what do you think about uh like at any given time there's like six percent

152
00:12:45,420 --> 00:12:52,680
problems happening in your life computation-wise oh i'm sorry i'm sorry 6.1 percent chance of breaking

153
00:12:52,680 --> 00:13:00,340
something in a library yeah it's about accuracy Coté 6.1 percent uh so i had uh it's funny sometimes i

154
00:13:00,340 --> 00:13:04,680
purposely think about you know hey if i tweet something at you know eight o'clock on a week

155
00:13:04,680 --> 00:13:11,320
night pacific time no one sees it but this was him uh from devops fame and writing the phoenix

156
00:13:11,320 --> 00:13:16,720
project devops handbook uh wrote just kind of shared this paper that said look this this group monitored

157
00:13:16,720 --> 00:13:23,140
400 open source libraries and in just like three months 116 days they saw 282 breaking changes so

158
00:13:23,140 --> 00:13:27,500
more or less each day assume there's this six percent chance you're gonna have some sort of breaking

159
00:13:27,500 --> 00:13:32,220
change in one of your open source libraries and the reason that stuff matters and i retweeted this was

160
00:13:32,220 --> 00:13:37,140
that this is why your apps need to be on pipelines because your apps are probably even if your code's

161
00:13:37,140 --> 00:13:41,700
not changing you know your functionality isn't changing every day something is changing in your

162
00:13:41,700 --> 00:13:46,360
dependencies whether it's a vulnerability whether it's a breaking change it's a patch it's whatever

163
00:13:46,360 --> 00:13:51,240
that your apps because we now build a lot of code with a lot of dependencies it's not just

164
00:13:51,240 --> 00:13:56,840
encapsulated and stuff i've written i'm pulling in libraries from all over the place that means i need to be

165
00:13:56,840 --> 00:14:00,900
able to constantly integrate them test them and deploy them because i could be sitting here with

166
00:14:00,900 --> 00:14:06,620
breaking changes vulnerability for weeks or months because i only ship software once a quarter so

167
00:14:06,620 --> 00:14:11,340
don't put all your apps on pipelines because you're shipping 30 times a day like some web startup put

168
00:14:11,340 --> 00:14:14,880
your applications on a pipeline because you need to constantly update them for things like this

169
00:14:14,880 --> 00:14:20,920
yeah you know i've been uh uh writing up what i think is maybe the hardest section of stuff to write up

170
00:14:20,920 --> 00:14:28,600
about uh you know how to run your iq better which is like a metric like i keep and that's that's difficult

171
00:14:28,600 --> 00:14:34,780
because uh it's one of the things where like you have to uh if you remember several episodes you know

172
00:14:34,780 --> 00:14:40,560
i tried to ask josh long what made his life he just wouldn't answer me so i had to like measure him

173
00:14:40,560 --> 00:14:45,260
a lot to draw it out from him and it's the same thing with metrics like you ask them who actually

174
00:14:45,260 --> 00:14:51,600
know it's important and they go like socrates socratic on you it's like oh metrics well what you know

175
00:14:51,600 --> 00:14:54,840
they'll never just tell you like you got to measure this you got to measure this you got to measure

176
00:14:54,840 --> 00:15:02,080
this and rightly so things are uh are relative but i think i think you know what uh what uh 0.1

177
00:15:02,080 --> 00:15:08,020
person is looking to is uh i think of like there's sort of like two a new sort of philosophic approach

178
00:15:08,020 --> 00:15:15,180
to what you could call monitoring or operations or whatever and originally you have this idea of like

179
00:15:15,180 --> 00:15:21,320
uh the computer systems must be available and perfect and there is no further discussion

180
00:15:21,320 --> 00:15:26,960
and so so your your job is to keep them up and running and when it goes red you're in trouble

181
00:15:26,960 --> 00:15:32,800
whereas like there's just a slight new way of thinking about things which what uh what you're

182
00:15:32,800 --> 00:15:37,880
talking about here as a representative is is uh everything is going to be breaking all the time

183
00:15:37,880 --> 00:15:44,160
and what we care about is how quickly we can fix it and it's almost like uh it's sort of like a point

184
00:15:44,160 --> 00:15:49,760
that like mess dying or you know in in a much more abstract way uh not see how it makes right

185
00:15:49,760 --> 00:15:55,560
i don't know ball demand or something but essentially like what's important is to uh not be fragile

186
00:15:55,560 --> 00:16:01,000
and have a way of covering things which is a lot of what like uh you know we i focus on a pipeline

187
00:16:01,000 --> 00:16:07,840
is able to uh deliver business value or useful software sort of back to the the database point

188
00:16:07,840 --> 00:16:13,760
but it's also valuable for uh being able to fix things which i think is uh even now like kind of

189
00:16:13,760 --> 00:16:21,600
an under appreciated metric if you will uh how how quickly our how well our system is at uh fixing so

190
00:16:21,600 --> 00:16:28,360
it is it is a good reminder uh or i should say proof point that basically everything is commonly

191
00:16:28,360 --> 00:16:34,520
decaying and we're not so much preventing entropy we're just making sure that total entropy doesn't

192
00:16:34,520 --> 00:16:42,340
happen uh you know before we die right yeah i think your word proof points the right thing right this is a good

193
00:16:42,340 --> 00:16:46,120
proof point that if you thought that app transformation was jamming stuff in containers

194
00:16:46,120 --> 00:16:50,720
throwing it in prod and calling it success that's not it's just no you have to also do things that

195
00:16:50,720 --> 00:16:55,340
can help improve your velocity which is adding test coverage adding other ways to monitor vulnerabilities

196
00:16:55,340 --> 00:17:00,840
or monitor dependencies that sometimes that looks like fool's gold of just hey we got to decommission

197
00:17:00,840 --> 00:17:04,880
some servers yeah i know but you have the same problems you had before and so i think these are

198
00:17:04,880 --> 00:17:08,020
nice reminders that these problems will not go away they're frankly going to increase

199
00:17:08,020 --> 00:17:13,940
because any modern development language now really promotes this idea of pulling in dependencies so

200
00:17:13,940 --> 00:17:18,080
again you don't have these sort of self-sufficient apps anymore and you're instead as you build a

201
00:17:18,080 --> 00:17:21,940
serverless app right you're building live pulling in libraries with thousands of lines of code

202
00:17:21,940 --> 00:17:26,700
so at this point we are all at the mercy of our dependencies and you should design accordingly

203
00:17:26,700 --> 00:17:34,020
at the mercy of our dependencies that's a good title for some of you we should suggest that to jared for

204
00:17:34,020 --> 00:17:43,420
his next uh tomic blog post no i think it's my autobiography title yeah exactly and then finally

205
00:17:43,420 --> 00:17:50,680
uh the uh like in azure there's uh they ship the i guess they ga the open service broker or they have

206
00:17:50,680 --> 00:17:55,440
they have that available there which is you know it's i think is emerging as a nice way to uh whether

207
00:17:55,440 --> 00:18:00,880
you want to call it services backing services or uh as as i i think annoyingly for the world so-called

208
00:18:00,880 --> 00:18:08,540
middleware you know i i like to be an old bogey but uh essentially a way to like look up uh what

209
00:18:08,540 --> 00:18:14,440
would you call it to use a new word other other things that you and your software rely on to get

210
00:18:14,440 --> 00:18:21,040
stuff done you're not necessarily responsible yeah i mean that fits on a t-shirt right there i think

211
00:18:21,040 --> 00:18:27,820
that's exactly perfect very fine print yeah i mean how do you consume right it's application and data

212
00:18:27,820 --> 00:18:32,960
services and so the one thing you know people i think it's funny people kind of forget already like

213
00:18:32,960 --> 00:18:36,740
wow this open service broker is cool i wonder when it comes to cloud foundry like it started in

214
00:18:36,740 --> 00:18:41,000
cloud foundry like that's where that's where it came from so what's neat is that you know we took

215
00:18:41,000 --> 00:18:45,000
the service broker that's been in cloud foundry for the from the beginning kind of made it this

216
00:18:45,000 --> 00:18:49,500
more open source it was already open source but made it a more open standard now you see

217
00:18:49,500 --> 00:18:54,340
microsoft and other companies building these sort of brokers on that model bringing them then back to

218
00:18:54,340 --> 00:18:58,800
those platforms so i just thought it was a good news point good validation that this brokerage model

219
00:18:58,800 --> 00:19:06,240
seems like a nice way to decouple if you look at 12 factor apps one of the factors is kind of treating

220
00:19:06,240 --> 00:19:11,200
these services as backing services not embedded in your app but instead kind of an attachable service

221
00:19:11,200 --> 00:19:15,360
and so the service broker does a great job of that where my same code could just continue to be

222
00:19:15,360 --> 00:19:19,680
deployed the dev test and prod with no changes i'm just attaching to different broker instances

223
00:19:19,680 --> 00:19:24,700
and therefore connecting to different databases so really nice decoupling mechanism it's cool to see

224
00:19:24,700 --> 00:19:29,420
it hit this this sort of mainstream yeah yeah and i and i think i mean just in general to be some

225
00:19:29,420 --> 00:19:35,540
sort of prognosticator i think it's uh on on the one hand it's it's i think it's notable in the

226
00:19:35,540 --> 00:19:40,740
overall cloud world that we don't spend tremendous amount of time talking about see i'll do it right

227
00:19:40,740 --> 00:19:47,480
here middleware like like at the moment we talk a lot about how to do containers run things yeah but

228
00:19:47,480 --> 00:19:54,040
obviously uh you need external services to write successful software unless you're writing everything

229
00:19:54,040 --> 00:20:00,860
yourself which would be delightful but ridiculous so it seems like hopefully over the next couple of

230
00:20:00,860 --> 00:20:05,900
years there'll be a lot more discussion of here's uh here's external services or backing services or

231
00:20:05,900 --> 00:20:11,100
middleware how they fit into those things and in fact like i think there's there's almost like a large uh

232
00:20:11,100 --> 00:20:15,920
what would you call it uh glossing over of like all these existing systems that you have how do you

233
00:20:15,920 --> 00:20:21,500
integrate them into uh your new stuff and in fact every now and then that's probably i don't know if it's

234
00:20:21,500 --> 00:20:27,840
a large part but a a used to say in corporate development a material uh amount of the discussion is

235
00:20:27,840 --> 00:20:33,920
about uh how do i interface with non-cloud native services make sure they don't slow me down which

236
00:20:33,920 --> 00:20:39,600
that'll be fun to talk about that uh more but speaking of extendable things we should talk about

237
00:20:39,600 --> 00:20:45,900
uh more related things now yeah yeah this is a uh a non-traditional guest ivan you should feel very

238
00:20:45,900 --> 00:20:50,980
special that we usually do a lot of kind of apps focus on the podcast but but you and i have gotten to

239
00:20:50,980 --> 00:20:56,520
be uh good buddies now and one thing we've talked about actually our last chat which is what's what

240
00:20:56,520 --> 00:21:01,460
kind of sparked this was you and i talked about data monoliths when we were talking and this idea of

241
00:21:01,460 --> 00:21:06,140
kind of data analytics monolith monoliths i should say one thing you've been helping me make sure i do

242
00:21:06,140 --> 00:21:10,120
is don't just say data because data could be a database data could be the literal data i mean

243
00:21:10,120 --> 00:21:15,560
it's very lazy sometimes just to refer to data there are companies who sell data they're right

244
00:21:15,560 --> 00:21:23,160
they're like uh what you call dun and brad street right and so one of the things you mentioned was data

245
00:21:23,160 --> 00:21:27,480
monoliths and kind of not having data monoliths much like we talk about with apps where i you know

246
00:21:27,480 --> 00:21:32,820
app monoliths can be beneficial in some cases but sometimes you're now coupled to a single set of

247
00:21:32,820 --> 00:21:36,940
gatekeepers to use them or you've got deployment challenges because it's all these big bulk

248
00:21:36,940 --> 00:21:43,560
deployments so what do you mean by this idea of kind of data analytic monoliths and how does some of

249
00:21:43,560 --> 00:21:48,300
what we're trying to do help break that up and you know thinking of the app people mindset how can

250
00:21:48,300 --> 00:21:55,680
they draw these parallels well i mean the the huge one of the huge trends in the in data analytics has

251
00:21:55,680 --> 00:22:03,740
been this idea of creating a corporate data lake and so probably that's permeated into the app

252
00:22:03,740 --> 00:22:08,760
conversation as well at least you probably have heard of people talking about data lakes and and we've had

253
00:22:08,760 --> 00:22:16,500
now it's now the data lake concept kind of started in around 2008 9 10 so we've had enough time to to

254
00:22:16,500 --> 00:22:23,560
kind of see it play out in a lot of cases and from observation what i see with the data lake idea is

255
00:22:23,560 --> 00:22:34,120
you generally have a centrally managed team inside a large corporation who is tasked to collect all of

256
00:22:34,120 --> 00:22:41,960
the company data and to make it available to all the business users for for business purposes um and it

257
00:22:41,960 --> 00:22:48,600
it essentially they're not driven by specific business outcomes but they're they're there to provide this

258
00:22:48,600 --> 00:22:57,960
kind of shared service and it it turns out to to look like the the ultimate monolith in the sense that

259
00:22:57,960 --> 00:23:05,940
you've got non-business people driving a data architecture and a collection of storage of

260
00:23:05,940 --> 00:23:12,720
information um that is meant to provide value to all the different business users but then

261
00:23:12,720 --> 00:23:21,060
generally tends to frustrate and and and not um actually satisfy the requirements of

262
00:23:21,060 --> 00:23:26,600
most of the business departments i also see parallels there i know you'll keep going into the monolith

263
00:23:26,600 --> 00:23:30,560
side but the sort of DevOps challenge in the first place is these ops teams that are somewhat

264
00:23:30,560 --> 00:23:35,280
disconnected from business outcomes running these sort of shared services and kind of having to go

265
00:23:35,280 --> 00:23:41,780
through that but i think you also have some other monolith comparisons as well no i mean it's it's a lot of

266
00:23:41,780 --> 00:23:48,520
it starts from whether who's driving the project right is it is so for example if we had a marketing

267
00:23:48,520 --> 00:23:57,200
team let's say we're working at uh a um at widget manufacturer incorporated and the marketing team

268
00:23:57,200 --> 00:24:03,720
says hey we could really improve our marketing if we had good data analytics and we're going to drive

269
00:24:03,720 --> 00:24:09,460
a project and here's our requirements that's one point of view and the other point of view is you've

270
00:24:09,460 --> 00:24:14,900
got a central team in widget incorporated that says we're going to build a data architecture so the data

271
00:24:14,900 --> 00:24:21,920
architecture they're thinking of how to optimize for reducing data copies and and the cost of the data

272
00:24:21,920 --> 00:24:27,660
and how to manage the data how to track the data but they're really not coming at it from how do we do

273
00:24:27,660 --> 00:24:33,500
marketing so from the from the first step you take the marketing people now have a gatekeeper

274
00:24:33,500 --> 00:24:40,340
which is data architects and so from the very first step now they're constantly trying to say okay what can the

275
00:24:40,340 --> 00:24:46,860
data architecture team give for me why not just start with the business and say hey let's build look what

276
00:24:46,860 --> 00:24:51,180
is the ultimate system for marketing yeah i mean that makes sense that's so when you're seeing the data

277
00:24:51,180 --> 00:24:57,020
lake concept fading in popularity as a whole are people trying to do things they're more departmental or

278
00:24:57,020 --> 00:25:03,260
more local where they're not dependent on these sort of shared data teams and data services so so i think

279
00:25:03,260 --> 00:25:10,060
the popularity has faded a bit but not going away but having looked at you know eight years of it and

280
00:25:10,060 --> 00:25:17,140
in a position where we get to see lots of different customers um there there's definitely companies that

281
00:25:17,140 --> 00:25:23,960
that don't build the hey we're a fortune 1000 company and here's our data lake and they build

282
00:25:23,960 --> 00:25:30,740
basically you know departmental specific data systems maybe using some shared infrastructure

283
00:25:30,740 --> 00:25:36,020
or some shared technologies but it's driven by by the business and there are other companies where

284
00:25:36,020 --> 00:25:43,980
i've seen build the corporate wide system and the the ones who are building the departmental systems

285
00:25:43,980 --> 00:25:50,680
generally tend to be from what i can see um happy and successful and the the ones building the data

286
00:25:50,680 --> 00:26:00,960
like either end up switching to to to spin out specific systems or um are always three to four

287
00:26:00,960 --> 00:26:09,480
years away in an agile project from um the ultimate delivery and i say agile is kind of as a joke right

288
00:26:09,480 --> 00:26:14,960
because you'll see you'll go in and you'll say we're doing this agile and in four years the data lake will

289
00:26:14,960 --> 00:26:20,620
be complete and it will provide this and it's always four years away i i mean this leads to the

290
00:26:20,620 --> 00:26:27,380
obvious question so uh how does green plum take all this what what does it do exactly and uh maybe

291
00:26:27,380 --> 00:26:35,200
how does it reduce that four years well so so and and really green plum per se is a technology it

292
00:26:35,200 --> 00:26:40,400
doesn't sell to your guys previous discussion about if you just take a new technology and use the same

293
00:26:40,400 --> 00:26:44,960
mindset you're not going to fix anything right that you're talking about in the previous conversation

294
00:26:44,960 --> 00:26:50,560
so in the same sense you know green plum by itself is a database it's not going to solve or

295
00:26:50,560 --> 00:26:58,400
organizational problems and and strategy problems but the great so let's start with what is green plum it's

296
00:26:58,400 --> 00:27:05,400
a it's a relational database that's that's optimized for storing large amounts of data and doing analytical

297
00:27:05,400 --> 00:27:12,900
type workloads as opposed to rapid updates to to state so you're going to be asking questions like

298
00:27:12,900 --> 00:27:22,300
given given every purchase at a retail store chain in the last 10 years you know what was the average

299
00:27:22,300 --> 00:27:31,000
purchase volume related to um consumer goods or related to alcohol or you know so it allows you to scan

300
00:27:31,000 --> 00:27:36,740
through billions of records and ask questions um it's that and and that comes back to the question about

301
00:27:36,740 --> 00:27:42,840
the idea that there's no perfect database the world hasn't created one so every every database designer

302
00:27:42,840 --> 00:27:48,000
creates some sort of compromise the compromise for green plum is it's really not good at transactional

303
00:27:48,000 --> 00:27:55,580
processing where you're doing um highly concurrent updates right it's it's optimized for

304
00:27:55,580 --> 00:28:04,640
large high speed reporting type workloads and so um the reason that this comes into play as far as the data

305
00:28:04,640 --> 00:28:13,920
lake discussion is that um the trend was so popular that it really overwhelmed the industry to say let's

306
00:28:13,920 --> 00:28:21,740
basically take hdfs and hadoop and build a data lake and put everything in it and um that that trend

307
00:28:21,740 --> 00:28:32,480
um resulted in people in a in a kind of a approach which was to to get a central it team driving

308
00:28:32,480 --> 00:28:39,480
a data project and um that doesn't mean that there aren't people who have not tried to create and

309
00:28:39,480 --> 00:28:46,360
and haven't successfully to some degree created corporate wide databases and enterprise data

310
00:28:46,360 --> 00:28:53,640
warehouses which can be done in green plum or another technology but the the technology of hadoop

311
00:28:53,640 --> 00:29:02,460
drove drove drove people to this approach and um they they kind of has shied away from thinking about it

312
00:29:02,460 --> 00:29:07,860
because it's too expensive to create a hadoop cluster for each department right and to say hey you know

313
00:29:07,860 --> 00:29:15,700
let's build a uh an analytical system for marketing and let's build one for finance and and let's build one

314
00:29:15,700 --> 00:29:23,800
for um r&d but with a database it's not too expensive you can just get 10 servers loaded up with a petabyte of

315
00:29:23,800 --> 00:29:31,880
data and start asking questions and it's so it's a lower overhead faster way to to take you know one

316
00:29:31,880 --> 00:29:37,200
petabyte of data and start storing it and asking questions and this is i mean it's a software of course

317
00:29:37,200 --> 00:29:42,000
so you can run this anywhere do we are most of our customers like to run this on premises on their own

318
00:29:42,000 --> 00:29:46,780
hardware does it require specialized hardware or do i just take anything off the shelf and run

319
00:29:46,780 --> 00:29:52,540
a green plum environment again it's postgres based it's about distributed mpp stuff so what's it look

320
00:29:52,540 --> 00:30:00,100
like yeah yeah so it's it is open source um postgres based database so it takes um postgres database and

321
00:30:00,100 --> 00:30:07,440
builds it um lots of them together to make one large database um as far as the the hardware the hardware

322
00:30:07,440 --> 00:30:12,900
really is it's what i like to tell people is that it's you need the hardware for the type of job you're

323
00:30:12,900 --> 00:30:18,940
doing so don't even think about green plum if if the type of your job you want to do is to take um

324
00:30:18,940 --> 00:30:23,300
hundreds of terabytes of data from two different tables and then correlate them you need enough

325
00:30:23,300 --> 00:30:27,880
hardware that can number one read hundreds of terabytes of data in a reasonable amount of time

326
00:30:27,880 --> 00:30:34,840
and do computation on it to do all the different comparisons and shuffling of the data over the

327
00:30:34,840 --> 00:30:41,120
network so regardless of green plum if you wanted to do that type of a report you're going to need

328
00:30:41,120 --> 00:30:47,260
it's going to run it can run on a docker container and in a macbook air but how long will it take to

329
00:30:47,260 --> 00:30:54,620
do that type of a processing will take a lot longer if you don't have enough hardware to turn through the

330
00:30:54,620 --> 00:31:04,320
data so you need both cpu um network and disk in balanced proportions to um to basically be able

331
00:31:04,320 --> 00:31:10,900
to feed that data into the cpus do the processing and enough network speed for the for whatever degree

332
00:31:10,900 --> 00:31:18,220
you're going to be doing shuffling so what it tends to be is we we say premium economy hardware so you

333
00:31:18,220 --> 00:31:23,680
don't want first class okay because it's it's a way it's a waste of money um but you don't want to be

334
00:31:23,680 --> 00:31:28,340
sitting and coach so premium economy means you go to your favorite hardware vendor and say what do you

335
00:31:28,340 --> 00:31:34,080
have that's like normal hardware but it's the hottest most expensive version of normal hardware so you

336
00:31:34,080 --> 00:31:40,980
don't want bespoke hardware which is you know some special purpose um proprietary hardware that's

337
00:31:40,980 --> 00:31:47,720
expensive and because the economics aren't as good you want commodity mass-produced hardware which is

338
00:31:47,720 --> 00:31:54,460
going to be um give you the the right economics but then you want as high performance as possible so

339
00:31:54,460 --> 00:32:01,520
like today they've come out with nv ram in non-volatile ram and 100 gig networks and one terabyte

340
00:32:01,520 --> 00:32:07,820
ram servers so so why wouldn't you want that if you can make your your data processing run a hundred times

341
00:32:07,820 --> 00:32:12,620
faster in your mind and i'm maybe in your mind because i'm implying it might be delusional but

342
00:32:12,620 --> 00:32:18,320
what what makes green palm kind of stand out right this is a mature space this is not like analytics is

343
00:32:18,320 --> 00:32:23,200
not a new problem we're solving here so when you look at that yeah what stands out to you but richard

344
00:32:23,200 --> 00:32:30,180
the point is is that it isn't a solved it's not mature in a sense in other words there is no perfect

345
00:32:30,180 --> 00:32:37,760
database today at all there's no there's no like there is no database in the world that can um can hand that

346
00:32:37,760 --> 00:32:48,140
can scale data end users and can scale um transactions and analytics both linearly do it with relational

347
00:32:48,140 --> 00:32:56,040
sql like so so it's it's a not it's non-solved problem by by humanity meaning so for example if you look

348
00:32:56,040 --> 00:33:02,080
at something like oracle it doesn't scale to once you get over 100 terabytes it starts slowing down

349
00:33:02,080 --> 00:33:09,980
um exponentially once you take green plum and start scaling it to high volumes of updates then it

350
00:33:09,980 --> 00:33:18,280
it doesn't it doesn't scale right and so every database has a choke point so there is no perfect

351
00:33:18,280 --> 00:33:23,560
database and what you want to do is look and see okay given the type of workload i'm going to run

352
00:33:23,560 --> 00:33:28,700
what what makes sense and then you also want to look at the platforms right so one of the huge

353
00:33:28,700 --> 00:33:37,420
value props of pivotal is that we sell software right um if you look at someone like amazon they're selling

354
00:33:37,420 --> 00:33:44,840
only services on amazon right so if if what you're looking for is an amazon service and you want to work

355
00:33:44,840 --> 00:33:50,240
exclusively with amazon and you want to you know be get get married to amazon for the long run

356
00:33:50,240 --> 00:33:54,760
then then then you can use amazon right but if you want a software product that

357
00:33:54,760 --> 00:34:02,760
is independent and can run where you want then then pivotal software generally is is a good approach

358
00:34:02,760 --> 00:34:08,540
right so the kind of open source can run on relative commodity infrastructure i think the

359
00:34:08,540 --> 00:34:13,720
database analytics was new to me to some extent when i joined pivotal as a i hadn't done much with

360
00:34:13,720 --> 00:34:18,600
that can you kind of describe what that means that what we offer there well it's a great um connection

361
00:34:18,600 --> 00:34:24,160
back to our excel conversation microsoft excel provides in spreadsheet analytics right you can

362
00:34:24,160 --> 00:34:31,300
run machine learning and or you call it artificial intelligence right because the words get used so

363
00:34:31,300 --> 00:34:37,200
so liberally right the artificial intelligence is a bit of a buzzword right now but ultimately

364
00:34:37,200 --> 00:34:43,300
artificial intelligence is another form of machine learning machine learning is another form of

365
00:34:43,300 --> 00:34:49,860
analytics so you've got stuff like regression algorithm linear regression algorithms these are all available in

366
00:34:49,860 --> 00:34:58,860
microsoft excel right and so the idea is these are there are important mathematical and statistical functions

367
00:34:58,860 --> 00:35:04,940
that you can run on your data in the database you have so if you have excel and you've got this big

368
00:35:04,940 --> 00:35:11,120
spreadsheet why would you want to export that data to run um you know a standard deviation or or linear

369
00:35:11,120 --> 00:35:16,820
regression you want to run it right in place in the spreadsheet right the same way if you've got data

370
00:35:16,820 --> 00:35:22,020
in oracle you want to run it directly there and if you have data in green plum you want to run it

371
00:35:22,020 --> 00:35:27,380
directly there so the more that you're doing analytical type processing the more likely you're going to want

372
00:35:27,380 --> 00:35:34,100
a full suite of the most popular algorithms that you can just call as functions directly in the platform you

373
00:35:34,100 --> 00:35:39,900
have and so green plum has a full suite of the most popular algorithms for kind of machine learning

374
00:35:39,900 --> 00:35:46,600
and geospatial analytics and graph analytics and text analytics and all different types of ways to

375
00:35:46,600 --> 00:35:53,460
slice and dice and do math and and um algorithms on your data directly where it lives

376
00:35:53,460 --> 00:36:00,240
yeah that's handy again avoiding extra duplication yeah i mean why export it and especially when you have

377
00:36:00,240 --> 00:36:07,340
a mpp platform so if you compare it for example excel excel runs on your laptop right you can't do a

378
00:36:07,340 --> 00:36:12,820
linear regression in excel and distribute the work to 100 cpus but because green plum is mpp

379
00:36:12,820 --> 00:36:21,460
meaning massively parallel and every query you run will get subdivided by all the different computers that

380
00:36:21,460 --> 00:36:30,640
you have assigned to that green plum system then it's convenient to be able to um to run statistical

381
00:36:30,640 --> 00:36:36,800
algorithms on hundreds of terabytes of data if you can take one calculation and subdivide it automatically

382
00:36:36,800 --> 00:36:44,980
to all the different um servers available going going back to one thing uh you were saying i mean it is uh

383
00:36:44,980 --> 00:36:51,400
it sounds like in the the database world you also suffer from i don't know what i think of as the

384
00:36:51,400 --> 00:36:56,680
uh the yesterday's ai problem that like once we figure out some ai thing it's everyone's used

385
00:36:56,680 --> 00:37:01,340
to it and it's totally like meaningless and then we're back to like ai is an impossible problem to

386
00:37:01,340 --> 00:37:06,660
solve it's just like at some point facial recognition was this mysterious thing and then we solve that

387
00:37:06,660 --> 00:37:15,180
problem right so facial recognition comes in a library called open cv right you could you could just get

388
00:37:15,180 --> 00:37:24,480
open cv on your laptop and and um take some snap photos from your webcam and then do facial recognition with

389
00:37:24,480 --> 00:37:34,040
open cv you can also take open cv and install it on your green plum cluster and have uh mpp processing of

390
00:37:34,040 --> 00:37:41,960
facial recognition but your point is that the word artificial intelligence gets could mean that you've got this

391
00:37:41,960 --> 00:37:49,540
like um intelligence that's in the in the like in some science fiction movie where it's been uploaded into

392
00:37:49,540 --> 00:37:57,180
the internet right in the ultimate exactly and it escaped the mainframe yeah but in in practice

393
00:37:57,180 --> 00:38:06,240
there's there's very specific um mathematical algorithms that have been invented that do specific jobs

394
00:38:06,240 --> 00:38:11,660
and those are all available and so when people say hey you know i go to let's say i go to

395
00:38:11,660 --> 00:38:17,660
a random customer in a random country and they say i want to do artificial intelligence they say great

396
00:38:17,660 --> 00:38:23,380
what do you want to do artificial intelligence like okay for what to to have artificial intelligence i'm

397
00:38:23,380 --> 00:38:31,220
like right to solve what once you get to do you want to have like a a cyborg computer running your

398
00:38:31,220 --> 00:38:36,860
business and they're like maybe once you get to a specific problem then you don't actually need

399
00:38:36,860 --> 00:38:43,560
some artificial intelligence you need an algorithm that can do let's say detect crop failure by looking

400
00:38:43,560 --> 00:38:50,700
at pictures of the farm right and that's just to your point a mundane algorithm that was invented 30 years

401
00:38:50,700 --> 00:38:57,700
ago and just run it on a large amount of data yeah yeah and that that sort of bumps up against the

402
00:38:57,700 --> 00:39:03,160
other the other question i had before we wrap up is like how like you know you're saying you could do

403
00:39:03,160 --> 00:39:09,260
you could run uh you know your facial recognition over here or you could run it like in in green plum or

404
00:39:09,260 --> 00:39:16,740
all these things like how how do you like advise people to figure out what to run in your database versus

405
00:39:16,740 --> 00:39:23,340
run on your own like is there is there sort of like obvious answers to that or does it get nuanced at some point

406
00:39:23,340 --> 00:39:30,860
it's fair i think it's fairly obvious i mean if if your use case doesn't have any kind of requirement

407
00:39:30,860 --> 00:39:37,500
to have a database in it then you wouldn't use a database like if you just said i have these files

408
00:39:37,500 --> 00:39:42,420
of data and i want to calculate something and then i want to take those results and put it somewhere else

409
00:39:42,420 --> 00:39:50,780
so there's no database requirement right but once you start wanting to have storing varieties of data

410
00:39:50,780 --> 00:39:55,800
and start interacting with it and asking different questions now it starts sounding more like a

411
00:39:55,800 --> 00:40:02,600
database so the question is do you you wouldn't force the use of a database when all you want to do is

412
00:40:02,600 --> 00:40:08,620
do a job right let's say a batch job or a or a streaming job a very tailored specific thing but if if you

413
00:40:08,620 --> 00:40:16,600
want to start asking interactive questions the the beauty of the database is that you put in any arbitrary

414
00:40:16,600 --> 00:40:24,220
sql there can be billions of possible sqls and then it can calculate the most efficient way to answer that

415
00:40:24,220 --> 00:40:31,600
question without you writing the procedural code like in go or in java or whatever to to get back that

416
00:40:31,600 --> 00:40:38,000
result you can say start writing complex queries and say i want to select this and this and this and then

417
00:40:38,000 --> 00:40:44,200
group it by this and filter it by that why would you want to do that by hand but if you don't have any use

418
00:40:44,200 --> 00:40:49,600
case to query a database then i wouldn't put it in a database yeah and and so i mean to grossly

419
00:40:49,600 --> 00:40:56,080
oversimplify it's basically like uh in your application layer you're sort of like displaying

420
00:40:56,080 --> 00:41:01,740
and inputting data maybe but then when it comes to like querying it and manipulating it it's probably

421
00:41:01,740 --> 00:41:07,200
a good default thing to like do it in your database layer rather than having your application figure all

422
00:41:07,200 --> 00:41:11,800
that like you know grab some big data set and sort it all for you and figure out how you want to do

423
00:41:11,800 --> 00:41:16,800
something it's highly likely the database will do that much better than you'll be able to code yeah

424
00:41:16,800 --> 00:41:21,500
i think i think exploring the data you're more likely to get benefit from a database but then once

425
00:41:21,500 --> 00:41:27,480
but you know a lot of times if you had a let's say an app um let's say you had a you know a taxi

426
00:41:27,480 --> 00:41:33,020
application right there's there's probably certain things that you want to productionize outside the

427
00:41:33,020 --> 00:41:40,260
database where where it's known known workflows right you said okay i want to have an api to display

428
00:41:40,260 --> 00:41:47,600
you know taxis within some region of me so probably that's that's a well-known enough problem that you

429
00:41:47,600 --> 00:41:55,720
don't need to ask arbitrary questions in a database you can maybe do it more efficiently um with some

430
00:41:55,720 --> 00:42:01,640
other sort of a platform where you where you create a pipeline to get that data process it push it out

431
00:42:01,640 --> 00:42:07,620
into some sort of in-memory cache and make it available through an api now that makes sense

432
00:42:07,620 --> 00:42:13,200
well then finally uh you know by nature of being open source there's a certain like visibility into

433
00:42:13,200 --> 00:42:18,260
the uh the team and the product management and things like that but like how like how how does

434
00:42:18,260 --> 00:42:24,600
how does how does the the green plum development organization work like like uh how do you go about

435
00:42:24,600 --> 00:42:30,040
doing doing doing the work are you on like big monolithic five-year release cycles so so it's

436
00:42:30,040 --> 00:42:38,900
interesting because we green plum started as a startup then was part of emc and then um became part of

437
00:42:38,900 --> 00:42:44,980
pivotal and when we when the green plum product became part of pivotal we we basically went through a full

438
00:42:44,980 --> 00:42:53,180
agile transformation ourselves i mean we had a team of 100 people in r&d and we essentially are a case study

439
00:42:53,180 --> 00:43:00,960
in in transforming a team from um from let's say non-agile development practice to agile development

440
00:43:00,960 --> 00:43:09,000
practice and so we we use concourse um we have um we use we have weekly sprints through using pivotal

441
00:43:09,000 --> 00:43:18,420
tracker we use all the pivotal labs methodologies we have um pair programming um and then releases we can

442
00:43:18,420 --> 00:43:27,420
release within 24 hours when whenever we want to do a release so we always keep the the code shippable

443
00:43:27,420 --> 00:43:35,480
and um but however we do want to time releases we found that with enterprise customers there is an art

444
00:43:35,480 --> 00:43:42,860
to timing so they don't want to get a database update every day and they there's also when do you create

445
00:43:42,860 --> 00:43:48,380
api changes right so if you if it's backwards compatible that's one thing but if you if they

446
00:43:48,380 --> 00:43:54,360
need to change something on their code there's only a very you know certain frequency they can do that so

447
00:43:54,360 --> 00:44:02,120
we're shippable at any time we generally ship patches every four weeks um to keep it digestible

448
00:44:02,120 --> 00:44:08,280
and then um but yeah we're i think we're a great case study and i personally learned a lot on how to

449
00:44:08,280 --> 00:44:14,100
transform an organization into agile development piece by piece while while the car is driving

450
00:44:14,100 --> 00:44:21,520
that's a pretty awesome story well uh thanks for being on for people like myself who are maybe even

451
00:44:21,520 --> 00:44:26,100
willfully ignorant of these things what would you point them to to uh read up more what's what's a good

452
00:44:26,100 --> 00:44:31,380
like starting path my my starting recommendation is youtube we put a lot of energy into youtube

453
00:44:31,380 --> 00:44:36,980
and so we've got over a hundred different videos talking about geospatial analytics and text

454
00:44:36,980 --> 00:44:42,360
analytics and containerization and i would go to youtube and check that out it's easy digestible

455
00:44:42,360 --> 00:44:47,460
place to get some quick info yeah that makes sense for sure well great as always this has been

456
00:44:47,460 --> 00:44:52,860
Pivotal Conversations if you want to find our back catalog you can go to soundcloud.com

457
00:44:52,860 --> 00:44:59,260
slash pivotal conversation now maybe you can load all the mp3s into a database and do some kind of

458
00:44:59,260 --> 00:45:05,880
analysis of like what's the ratio between helpful things richard says and a name gobbly book i say

459
00:45:05,880 --> 00:45:10,580
that would be a fun metric we should make a chart of that and i'll try to improve over time

460
00:45:10,580 --> 00:45:16,180
i just have to tell you we did a study with the talk show host and we found out that the audience

461
00:45:16,180 --> 00:45:22,540
started drifting when the talk show host started talking and the broadcast company ended up having

462
00:45:22,540 --> 00:45:27,740
to replace the host because of that analytics see that would be useful we should figure that out

463
00:45:27,740 --> 00:45:32,660
we can uh we can we can use some it'll it'll be like that that would be some so i guess it would

464
00:45:32,660 --> 00:45:37,740
be black mirror not twilight zone anymore yeah i i always think of us i was speaking of ai i always

465
00:45:37,740 --> 00:45:43,000
think of that one twilight zone episode where there's like a big computer and the ceo is very happy to

466
00:45:43,000 --> 00:45:48,980
replace all these workers and eventually the computer replaces ceo right and uh maybe maybe we should just

467
00:45:48,980 --> 00:45:55,360
have a really complicated query that replaces richard and i on this show right and it'll just spit up spit

468
00:45:55,360 --> 00:46:02,720
out some results well on on that on that optimistic note for humanity uh we'll see everyone next time

469
00:46:02,720 --> 00:46:03,480
bye bye