1
00:00:00,000 --> 00:00:01,760
Well, here we are at Vox Days in Singapore.

2
00:00:01,940 --> 00:00:02,540
It's very exciting.

3
00:00:02,780 --> 00:00:04,220
I just saw Josh Long talk.

4
00:00:04,820 --> 00:00:08,160
He gives a good, I was noticing he gives like a good 20-minute introduction,

5
00:00:08,500 --> 00:00:11,320
just talking about the bird on the book that he has.

6
00:00:11,840 --> 00:00:13,440
But it was fun nonetheless.

7
00:00:13,580 --> 00:00:14,460
Why don't you introduce yourself?

8
00:00:15,140 --> 00:00:15,420
Yeah.

9
00:00:15,800 --> 00:00:16,860
Hi, I'm Rahul.

10
00:00:17,340 --> 00:00:20,320
I'm a software engineer here at Pivotal Labs Singapore.

11
00:00:22,040 --> 00:00:23,640
Just a little bit about myself.

12
00:00:23,640 --> 00:00:31,160
I've been part of NEO from before Pivotal acquired us.

13
00:00:31,840 --> 00:00:34,640
And so all in all, I've been here for around four years.

14
00:00:34,880 --> 00:00:35,900
Oh, that's longer than me.

15
00:00:36,160 --> 00:00:36,680
Oh, wow.

16
00:00:37,120 --> 00:00:39,440
You know where all the secret bathrooms are.

17
00:00:41,500 --> 00:00:42,020
Yeah.

18
00:00:42,120 --> 00:00:49,540
So I've seen the office grow from a really small number to a much bigger space,

19
00:00:49,620 --> 00:00:50,820
grow into a bigger space.

20
00:00:51,100 --> 00:00:53,420
Also have been part of a lot of different projects,

21
00:00:53,560 --> 00:00:58,860
everything from like startups to like government agencies

22
00:00:58,860 --> 00:01:02,160
to multinational companies that include like banks

23
00:01:02,160 --> 00:01:07,120
and have worked in like lots of different domains.

24
00:01:07,440 --> 00:01:07,600
Yeah.

25
00:01:07,600 --> 00:01:09,500
It's been one of the great things of working at Pivotal Labs.

26
00:01:09,500 --> 00:01:11,780
That is an advantage you have like doing work like Pivotal

27
00:01:11,780 --> 00:01:13,600
is you get exposed to a whole lot of things.

28
00:01:13,800 --> 00:01:18,240
Well, so we were talking earlier about how some of the projects you've worked on,

29
00:01:18,240 --> 00:01:20,860
I mean, it's for larger organizations and, you know,

30
00:01:20,900 --> 00:01:24,020
a characteristic of large organizations is they tend to make a lot of money,

31
00:01:24,420 --> 00:01:24,900
be large.

32
00:01:25,140 --> 00:01:27,200
Otherwise, they would be non-existent organizations.

33
00:01:27,440 --> 00:01:27,800
That's true.

34
00:01:27,900 --> 00:01:31,880
And in the IT area, I always figure like that implies they have a lot of existing software

35
00:01:31,880 --> 00:01:33,380
that makes a lot of money.

36
00:01:33,880 --> 00:01:35,500
And we call that legacy.

37
00:01:37,080 --> 00:01:37,440
Right.

38
00:01:37,760 --> 00:01:39,000
And I don't know.

39
00:01:39,020 --> 00:01:43,140
I mean, the jokey thing I always think about legacy is like legacy is that software

40
00:01:43,140 --> 00:01:45,900
that like you have to change, but you're really afraid to change.

41
00:01:46,080 --> 00:01:47,620
Otherwise, you just call it software.

42
00:01:48,620 --> 00:01:51,240
But you've had the chance to work on some of those things.

43
00:01:51,240 --> 00:01:54,580
And I mean, how would you characterize these legacy systems, the software?

44
00:01:54,820 --> 00:01:59,140
I think even before I get into like what I've seen that could be labeled legacy,

45
00:01:59,540 --> 00:02:04,320
this very interesting thing that someone told me sometime back that really stuck with me

46
00:02:04,320 --> 00:02:08,820
was computer science is the only place where legacy is used in a negative connotation.

47
00:02:09,180 --> 00:02:13,240
You don't say Albert Einstein's legacy as left behind.

48
00:02:13,460 --> 00:02:14,080
Yeah, that's funny.

49
00:02:14,180 --> 00:02:14,340
Right?

50
00:02:14,540 --> 00:02:21,900
So it's a shame that many times when we are building software, we don't think about how

51
00:02:21,900 --> 00:02:22,460
it's going.

52
00:02:23,240 --> 00:02:27,940
We're not leaving something in a state that is to support the future engineers who are going

53
00:02:27,940 --> 00:02:28,500
to help it.

54
00:02:28,860 --> 00:02:37,000
And when care is not put into maintaining your software and maintaining a high code quality,

55
00:02:37,620 --> 00:02:41,200
it basically becomes really hard for anyone else who's going to be working on it in the

56
00:02:41,200 --> 00:02:41,500
future.

57
00:02:41,620 --> 00:02:41,720
Yeah.

58
00:02:41,720 --> 00:02:45,200
And that is why we call it like label it legacy software.

59
00:02:45,400 --> 00:02:47,420
And it's like something that you don't want to mess with.

60
00:02:47,520 --> 00:02:47,720
Yeah.

61
00:02:47,800 --> 00:02:48,000
Yeah.

62
00:02:48,120 --> 00:02:50,340
No one calls it like heritage software.

63
00:02:50,860 --> 00:02:51,720
Yeah, basically.

64
00:02:52,160 --> 00:02:52,440
Yeah.

65
00:02:52,700 --> 00:02:54,220
And so why do you think that is?

66
00:02:54,280 --> 00:03:01,860
Why do you think it hasn't become common practice to avoid it going legacy?

67
00:03:02,080 --> 00:03:07,680
Like, and as an example of that not happening, I mean, I'm sure we could always as an industry

68
00:03:07,680 --> 00:03:09,940
get better at unit testing and testing.

69
00:03:09,940 --> 00:03:16,640
But it seems like over the past 15 or so years, it's now part of a developer's mindset that

70
00:03:16,640 --> 00:03:18,060
like I do the tests, right?

71
00:03:18,060 --> 00:03:19,580
So we somehow fix that.

72
00:03:19,760 --> 00:03:24,980
But how has it come about that this creating legacy is still something?

73
00:03:25,160 --> 00:03:27,700
I mean, even, I mean, testing is definitely one part of it.

74
00:03:27,760 --> 00:03:33,420
So the fundamental thing to think about is when we are talking about testing, the primary

75
00:03:33,420 --> 00:03:35,480
goal of testing is to get high confidence.

76
00:03:35,940 --> 00:03:40,080
And the reason for high confidence is just to let you refactor more often.

77
00:03:40,620 --> 00:03:44,900
And that gives you the freedom to leave your code at a very high quality.

78
00:03:44,900 --> 00:03:49,640
So anytime you see something that could be optimized, you could go back in and fix it.

79
00:03:49,740 --> 00:03:54,220
So I wouldn't say that just adding tests for the sake of adding tests is going to solve

80
00:03:54,220 --> 00:03:54,680
that problem.

81
00:03:54,780 --> 00:03:58,640
You need to have the right kind of test to give you the confidence and the freedom to

82
00:03:58,640 --> 00:03:59,680
refactor often.

83
00:04:00,060 --> 00:04:00,580
Oh, right, right.

84
00:04:00,640 --> 00:04:07,320
And oftentimes, what I've seen is although you have tests, the way it is tested is really

85
00:04:07,320 --> 00:04:14,200
rigid, or you're not testing the right things, which are the risky parts of the system, all

86
00:04:14,200 --> 00:04:17,680
the tests are so flaky that you can't actually trust your test code base.

87
00:04:18,100 --> 00:04:29,420
So in those scenarios, developers tend to be reluctant to go back in and make changes to parts of

88
00:04:29,420 --> 00:04:31,640
the code that they don't understand intimately.

89
00:04:32,160 --> 00:04:36,240
And that is one of the things I've seen contribute to legacy code very often.

90
00:04:36,240 --> 00:04:42,040
Yeah, I guess there is a healthy fear of screwing things up that developers have.

91
00:04:42,100 --> 00:04:46,840
But then if you don't have the safety nets of tests in there, which I guess, like, I mean,

92
00:04:46,860 --> 00:04:50,880
one of the only books I've ever found about legacy code is, it's pretty old now, but it's

93
00:04:50,880 --> 00:04:51,960
that Michael Feathers book.

94
00:04:52,020 --> 00:04:53,780
Yes, working effectively with legacy code.

95
00:04:53,780 --> 00:04:57,560
Yeah, which starts off with one of my favorite, I don't know, Zen things.

96
00:04:57,760 --> 00:05:00,580
You probably remember it too, but it's like, what is it?

97
00:05:00,760 --> 00:05:02,740
To change legacy code, we need tests.

98
00:05:03,140 --> 00:05:05,180
But then legacy code is code without tests.

99
00:05:05,820 --> 00:05:07,840
So what do we do now?

100
00:05:09,940 --> 00:05:14,320
Yeah, I think that book has kind of held the test of time.

101
00:05:14,320 --> 00:05:17,240
And it's something that we still recommend to clients.

102
00:05:17,700 --> 00:05:22,680
Because oftentimes, when people see the practices that we follow, like test-driven development,

103
00:05:23,360 --> 00:05:28,300
it is, it seems very intuitive to apply it to something like a greenfield project where

104
00:05:28,300 --> 00:05:31,400
you're starting from scratch, when before you write your first line of code, you write

105
00:05:31,400 --> 00:05:31,780
your test.

106
00:05:32,180 --> 00:05:37,840
That, although it is a mindset shift, it's still something that's easier to palette.

107
00:05:38,380 --> 00:05:38,500
Right.

108
00:05:39,520 --> 00:05:44,920
But what's harder is, in most cases, like you mentioned, like these big companies that have

109
00:05:44,920 --> 00:05:48,700
been making a lot of money and have these systems that have been, they have been working,

110
00:05:49,320 --> 00:05:52,940
but they're not, they're not, they're very rigid and people are scared to go.

111
00:05:52,940 --> 00:05:53,660
They can't be evolved.

112
00:05:53,860 --> 00:05:54,480
They can't be evolved.

113
00:05:54,920 --> 00:05:55,040
Yeah.

114
00:05:55,380 --> 00:05:58,560
Every change that you need to make to that system takes a lot more time.

115
00:05:59,080 --> 00:06:05,480
So in these cases, companies want to know, companies and teams want to know how can they

116
00:06:05,480 --> 00:06:06,520
start doing TDD?

117
00:06:06,680 --> 00:06:10,100
Do we only test things that we build, like new things we build?

118
00:06:10,880 --> 00:06:15,780
Even then, since you're working with this legacy system, you need some kind of security.

119
00:06:15,960 --> 00:06:19,180
So there are some tactical stuff in that book that we still recommend.

120
00:06:19,500 --> 00:06:19,520
Yeah.

121
00:06:19,520 --> 00:06:23,820
So what are y'all, when you're working with people, what do you recommend for them?

122
00:06:24,040 --> 00:06:29,540
And then, you know, part of that is what works well and doesn't work well or is challenging?

123
00:06:29,800 --> 00:06:31,080
Like how does it play out?

124
00:06:31,720 --> 00:06:31,900
Yeah.

125
00:06:32,520 --> 00:06:38,660
So one of the things that I think the book covers really well is that you need to have

126
00:06:38,660 --> 00:06:45,920
few really high value automation tests that document the existing functionality so that

127
00:06:45,920 --> 00:06:49,980
whenever you are making new changes to the system, you're not breaking existing functionality

128
00:06:49,980 --> 00:06:51,640
that other systems are relying on.

129
00:06:51,640 --> 00:06:54,920
So that's just like adding like that basic net.

130
00:06:55,420 --> 00:07:01,240
But that aside, the main thing that we try to look at is most often these kind of legacy

131
00:07:01,240 --> 00:07:09,140
systems tend to be this like very opaque, monolithic applications.

132
00:07:09,320 --> 00:07:16,180
We try to go in and find these domain boundaries, trying to find where does it make sense to slice

133
00:07:16,180 --> 00:07:20,940
and dice the system so that we can maybe it could be like we rewrite parts of it.

134
00:07:20,940 --> 00:07:27,720
We may pull things out and make it separate services, but it's effectively just finding

135
00:07:27,720 --> 00:07:33,460
these boundaries that we can wrap more tests around and then slowly like do this iteratively

136
00:07:33,460 --> 00:07:38,280
till we are able to make them really well factored code.

137
00:07:38,520 --> 00:07:38,740
Right.

138
00:07:38,860 --> 00:07:43,840
And if I remember the feather stuff, one of the words he uses is like you find the seams.

139
00:07:44,580 --> 00:07:47,460
Like the, and nowadays you would say boundary.

140
00:07:47,780 --> 00:07:48,600
Bounded context.

141
00:07:48,900 --> 00:07:49,000
Yeah, yeah.

142
00:07:49,780 --> 00:07:51,180
And, and, and, and along those lines.

143
00:07:51,340 --> 00:07:54,640
So in this context, how do you, how do you figure out those boundaries?

144
00:07:54,640 --> 00:07:59,660
I mean, I mean, like, I don't know, you could pick a simple domain, like bill paying or reservations

145
00:07:59,660 --> 00:08:03,100
or whatever, but how would you, what's a good example of what a boundary is?

146
00:08:03,700 --> 00:08:06,260
Oh, this is, this is a really tough question.

147
00:08:06,260 --> 00:08:11,860
I mean, I don't really have a good answer for this, but the way generally when you talk

148
00:08:11,860 --> 00:08:17,720
to one, one thing I like to do is I go to these teams and ask them, okay, why don't you draw,

149
00:08:17,840 --> 00:08:19,080
draw me a component diagram?

150
00:08:19,240 --> 00:08:23,400
Like, tell me what are the boxes and arrows and just tell me what are the systems that

151
00:08:23,400 --> 00:08:24,240
interact with each other.

152
00:08:24,280 --> 00:08:28,900
And by systems, I don't mean separate deployments, but it's just like different components.

153
00:08:28,900 --> 00:08:30,080
Like what is your mental mark?

154
00:08:30,280 --> 00:08:30,440
Yeah.

155
00:08:30,440 --> 00:08:33,540
And generally they have a really good sense and you can really extract that out.

156
00:08:33,640 --> 00:08:34,360
That's, that's interesting.

157
00:08:34,580 --> 00:08:36,620
They sort of like already know the boundaries.

158
00:08:36,920 --> 00:08:38,240
They just may not think about it that way.

159
00:08:38,240 --> 00:08:38,520
Exactly.

160
00:08:38,720 --> 00:08:40,260
So that is already the case.

161
00:08:40,340 --> 00:08:45,360
Most often these tend to be really intertwined, but that's where we can help them try to extract

162
00:08:45,360 --> 00:08:45,580
it.

163
00:08:45,840 --> 00:08:46,400
That is one.

164
00:08:46,400 --> 00:08:51,920
If that doesn't help, the other approach is, I really believe in Pivotal's way of having

165
00:08:51,920 --> 00:08:58,840
the balance team and we take cues from the design and the product requirements to see what

166
00:08:58,840 --> 00:09:00,100
would be a bounded context.

167
00:09:00,780 --> 00:09:07,820
So we kind of try to find a ubiquitous language where we have this common terminologies across

168
00:09:07,820 --> 00:09:10,220
business, technology, and design.

169
00:09:10,880 --> 00:09:17,380
And those terminologies kind of help us, help guide our decision on like whether this makes

170
00:09:17,380 --> 00:09:18,420
sense as a bounded context.

171
00:09:18,420 --> 00:09:21,920
And so in that instance, is it, is it often the case?

172
00:09:22,140 --> 00:09:29,020
So you have, just to simplify it, you've got the, the business's conception of, of the

173
00:09:29,020 --> 00:09:33,520
boundaries and then the, the developer's conception of the boundaries.

174
00:09:33,740 --> 00:09:38,200
And are those things often the same or do they conflict or how does that line up?

175
00:09:39,540 --> 00:09:43,540
I've seen surprisingly, in most cases, they line up closely.

176
00:09:43,720 --> 00:09:44,220
Oh, that's great.

177
00:09:44,380 --> 00:09:49,340
That is good because generally they get the requirements from the business and they generally

178
00:09:49,340 --> 00:09:50,580
use the same terminologies.

179
00:09:50,800 --> 00:09:51,400
That is good.

180
00:09:51,620 --> 00:09:54,620
The other reason they end up lining up is Conway's law.

181
00:09:54,880 --> 00:09:59,220
So the people who are like high up decide, okay, these are the things that I'm, I've been

182
00:09:59,220 --> 00:09:59,740
talking about.

183
00:09:59,820 --> 00:10:01,400
These are the things I promised my CTO.

184
00:10:01,620 --> 00:10:03,060
So I'm going to make these teams.

185
00:10:03,300 --> 00:10:04,140
Oh, that makes a lot of sense.

186
00:10:04,140 --> 00:10:12,180
So these teams, so Conway's law is that the structure of a code base reflects the communication

187
00:10:12,180 --> 00:10:13,180
pathways of your team.

188
00:10:13,680 --> 00:10:18,300
So if you have two different teams and you have like a team that is like a booking team

189
00:10:18,300 --> 00:10:19,940
and the other one is a reservation team.

190
00:10:20,480 --> 00:10:25,240
So it's so handsome that there's a booking bounded context and the reservation bounded context.

191
00:10:27,160 --> 00:10:33,540
The level of separation between these contexts depend on the discipline of the developers in

192
00:10:33,540 --> 00:10:35,460
the team itself.

193
00:10:35,720 --> 00:10:40,240
But generally that's an easy way to see like, okay, that seems like a scene which you could

194
00:10:40,240 --> 00:10:40,600
pull out.

195
00:10:40,760 --> 00:10:46,460
Yeah, yeah, yeah, that's interesting because it's, it's very hopeful and, and it's almost

196
00:10:46,460 --> 00:10:50,260
like, again, like there's the, there are these latent boundaries that people don't realize

197
00:10:50,260 --> 00:10:51,900
they've, they've put that work in for.

198
00:10:51,960 --> 00:10:57,600
And then, and in the context of legacy, I imagine it's very helpful because as you're

199
00:10:57,600 --> 00:11:01,900
saying, it starts to make that opaque black, black monolith, so to speak, like a lot more

200
00:11:01,900 --> 00:11:02,440
understandable.

201
00:11:02,440 --> 00:11:07,100
So, so let's say you've broken up a couple of boundaries, then like, what's the next step

202
00:11:07,100 --> 00:11:10,740
for, for doing, and, and I always like to ask a question and then keep

203
00:11:10,740 --> 00:11:11,100
talking.

204
00:11:11,300 --> 00:11:16,920
So it's a pardon that tick, but, but I'm guessing what you do is, so our, our main

205
00:11:16,920 --> 00:11:20,940
problem with, with updating legacy stuff is we don't have good enough tests.

206
00:11:21,260 --> 00:11:25,160
And so we're trying to break it up so that then we can put tests in place and then we

207
00:11:25,160 --> 00:11:26,220
can start changing it.

208
00:11:26,240 --> 00:11:26,600
Right.

209
00:11:26,620 --> 00:11:30,900
And then, so, so then going back, we found the boundaries, you've got the monolith.

210
00:11:30,900 --> 00:11:31,760
So then what do you do?

211
00:11:32,520 --> 00:11:39,240
So, um, oftentimes what ends up happening is, uh, what we like to think about, think of

212
00:11:39,240 --> 00:11:40,240
as the app continuum.

213
00:11:40,900 --> 00:11:49,240
You, when, once you like one end of the app continuum is when you have like, um, uh, opaque

214
00:11:49,240 --> 00:11:49,440
monoliths.

215
00:11:49,440 --> 00:11:50,440
Monoliths.

216
00:11:50,440 --> 00:11:53,440
And the other end is having like lots of well-structured microservices.

217
00:11:53,440 --> 00:11:54,440
Right.

218
00:11:54,440 --> 00:11:58,680
Because you could have like a distributed monolith, that's a terrible thing, but like, if you

219
00:11:58,680 --> 00:12:07,680
think of like completely, um, decoupled services and a monolith as the scale, we are basically

220
00:12:07,680 --> 00:12:12,200
trying to shift it towards somewhere in the middle where you want a well-structured, um,

221
00:12:12,680 --> 00:12:18,600
well, um, uh, a monolith that has separation across functional boundaries.

222
00:12:19,240 --> 00:12:24,820
And, um, the benefit of doing that is you have testing at multiple levels.

223
00:12:24,820 --> 00:12:30,960
So you can have unit testing, you can have integration testing between closely coupled, uh, components,

224
00:12:30,960 --> 00:12:36,960
and then you can have like, uh, automation testing for the whole system or even like one

225
00:12:36,960 --> 00:12:42,160
extra level of testing for how this system works with other systems that, um, exist in

226
00:12:42,160 --> 00:12:43,980
the organization.

227
00:12:43,980 --> 00:12:49,200
So, so, I mean, is it, uh, to ask a, it's not obvious, but a simplistic question.

228
00:12:49,200 --> 00:12:53,600
Is it, do you find that it's extremely common that people only test outside the box?

229
00:12:53,720 --> 00:12:57,680
Like, like for example, if you followed a classic three-tier application thing, I mean, you've

230
00:12:57,680 --> 00:13:02,640
got pretty, pretty good boundaries and, and you would expect that people would have tests in

231
00:13:02,640 --> 00:13:05,240
between them, but maybe they don't.

232
00:13:05,660 --> 00:13:10,320
Actually, uh, that's an interesting one because there we are talking about whether we want to

233
00:13:10,320 --> 00:13:12,560
do, how do we find these boundaries?

234
00:13:12,920 --> 00:13:16,340
Oftentimes it becomes really easy to say, okay, database is one boundary.

235
00:13:16,560 --> 00:13:17,800
My service is one boundary.

236
00:13:17,900 --> 00:13:19,060
My controller is another boundary.

237
00:13:19,160 --> 00:13:20,280
So this is my three boundaries.

238
00:13:20,420 --> 00:13:21,100
I'm going to test between.

239
00:13:21,700 --> 00:13:26,000
What we, uh, you could definitely have unit tests for each individual one, but when we're

240
00:13:26,000 --> 00:13:31,220
thinking of having like integration tests, you want to split across functional vertical

241
00:13:31,220 --> 00:13:31,660
slices.

242
00:13:31,840 --> 00:13:32,900
So that's a good insight.

243
00:13:33,020 --> 00:13:33,180
Yeah.

244
00:13:33,180 --> 00:13:37,560
A little bit of the, uh, database layer, a little bit of the services, a little bit of

245
00:13:37,560 --> 00:13:39,160
the controller, but they make sense together.

246
00:13:39,160 --> 00:13:42,500
So they all work together to fulfill one user requirement.

247
00:13:42,500 --> 00:13:48,060
And I guess that's a characteristic of a, of a boundary in the BDD sense is that it's

248
00:13:48,060 --> 00:13:54,460
from the perspective of an end to end process instead of database or controlled here or UI.

249
00:13:54,620 --> 00:13:59,620
Because at the end of the day, your controller, the framework you use, the database choice,

250
00:13:59,760 --> 00:14:01,240
everything is an implementation detail.

251
00:14:01,240 --> 00:14:07,760
And the real crux of the value lies in the business, uh, business models and the domain itself.

252
00:14:07,980 --> 00:14:08,260
Okay.

253
00:14:08,260 --> 00:14:11,220
And that's, that's what you are customizing for your clients.

254
00:14:11,400 --> 00:14:15,620
So, and so I guess in that context, then it would make a lot more sense that people hadn't

255
00:14:15,620 --> 00:14:19,620
thought about these, let's call them, uh, I always get my vertical and horizontal, these

256
00:14:19,620 --> 00:14:22,340
vertical slices as, as the boundaries.

257
00:14:22,340 --> 00:14:26,920
And people probably haven't been writing tests on, on that boundary, but they might've been

258
00:14:26,920 --> 00:14:28,440
writing tests on talking to the database.

259
00:14:28,440 --> 00:14:28,840
Exactly.

260
00:14:29,020 --> 00:14:29,240
Yes.

261
00:14:29,560 --> 00:14:38,940
So, uh, sometimes we see that, um, so it changes from different, um, in different engagements.

262
00:14:38,940 --> 00:14:42,120
Like one of the common ones I see is there's just unit testing.

263
00:14:42,120 --> 00:14:46,780
There's no integration testing whatsoever because unit testing is easy to understand and easy

264
00:14:46,780 --> 00:14:48,360
for devs to start writing.

265
00:14:48,360 --> 00:14:52,680
So that's one, the other extreme is there's no unit testing.

266
00:14:52,860 --> 00:14:57,900
There's no like, uh, component level testing, but there's like, um, end to end integration

267
00:14:57,900 --> 00:14:59,860
testing of user flow.

268
00:15:00,000 --> 00:15:04,640
So they basically bring up the whole app and have automation testing where some of the,

269
00:15:04,640 --> 00:15:08,900
the code goes and clicks actually physically clicks buttons and type stuff.

270
00:15:09,020 --> 00:15:10,560
Just like classic APM stuff.

271
00:15:10,720 --> 00:15:11,000
Yeah.

272
00:15:11,200 --> 00:15:13,620
Like the classic style of automation testing.

273
00:15:13,760 --> 00:15:14,820
So what do they think about?

274
00:15:14,820 --> 00:15:21,040
So what often happens is in either one of these scenarios, what, uh, the problem we face is

275
00:15:21,040 --> 00:15:24,260
there's no, uh, there's not a really good testing pyramid.

276
00:15:24,560 --> 00:15:34,000
So we don't have like a good distribution of, uh, uh, good mix of, uh, having a lot of cheap

277
00:15:34,000 --> 00:15:39,100
tests and a few very costly tests, but highly valuable tests, because you want your code base

278
00:15:39,100 --> 00:15:45,540
to be, uh, um, your testing, uh, your tests to have high confidence.

279
00:15:45,680 --> 00:15:50,040
So you provide you high confidence at the same time, be very cheap because you want to be

280
00:15:50,040 --> 00:15:51,660
able to run them as often as possible.

281
00:15:51,660 --> 00:15:56,240
And they shouldn't take more than, I don't know, a couple of minutes to execute end to

282
00:15:56,240 --> 00:15:56,440
end.

283
00:15:56,540 --> 00:15:56,840
So, right.

284
00:15:56,940 --> 00:16:02,380
So you want to have as much of your testing done in the lower levels where the cost of

285
00:16:02,380 --> 00:16:04,040
each individual test is very small.

286
00:16:04,340 --> 00:16:04,760
That makes sense.

287
00:16:04,840 --> 00:16:10,080
So you want a lot of unit tests, but you also want to have these very high value, um,

288
00:16:10,460 --> 00:16:12,060
integration tests, but very few of them.

289
00:16:12,100 --> 00:16:13,160
So you need to be very selective.

290
00:16:13,420 --> 00:16:13,720
That makes sense.

291
00:16:13,720 --> 00:16:19,820
So having a good balance between these different levels of testing is what I find being, uh,

292
00:16:19,820 --> 00:16:22,200
one of the most common shortcomings of the testing.

293
00:16:22,200 --> 00:16:22,300
Yeah.

294
00:16:22,300 --> 00:16:22,560
Yeah.

295
00:16:22,560 --> 00:16:27,380
Cause I guess it's a kind of a fallacy to think that the, uh, with unit tests, that the sum

296
00:16:27,380 --> 00:16:30,820
is greater than the parts, but in fact, the sum is less than the parts.

297
00:16:31,400 --> 00:16:35,620
Or, I mean, so to speak, like, like you can't just, you can't just sum up all your unit tests

298
00:16:35,620 --> 00:16:36,820
and be done with the whole system.

299
00:16:36,820 --> 00:16:39,900
You need to actually write system-wide sort of.

300
00:16:39,900 --> 00:16:43,800
There are, there are different, uh, there are different things that you're testing at

301
00:16:43,800 --> 00:16:44,320
each level.

302
00:16:44,420 --> 00:16:44,820
Right, right.

303
00:16:44,840 --> 00:16:48,300
At the unit level, you're just testing each individual unit behaves the way it's expected.

304
00:16:48,300 --> 00:16:54,020
At each, um, each level above that, you're trying to see how they interact with the, uh,

305
00:16:54,020 --> 00:16:56,280
these units interact with each other and so on.

306
00:16:56,620 --> 00:17:01,840
So the, the type of confidence you get at each level of testing is very different and

307
00:17:01,840 --> 00:17:03,120
all of them are equally valid.

308
00:17:03,440 --> 00:17:07,020
So I, I, I mean, tell me more about this concept of confidence.

309
00:17:07,020 --> 00:17:10,380
You keep, uh, you keep mentioning it in, in, in the context of all of this, right?

310
00:17:10,380 --> 00:17:13,880
Like what, that sounds interesting, but what, what do you mean by that?

311
00:17:13,980 --> 00:17:14,160
Okay.

312
00:17:14,160 --> 00:17:18,320
So the, this is something that I've recently been thinking about a lot as well.

313
00:17:18,820 --> 00:17:24,900
Um, oftentimes when we start talking to clients about, you know, testing is important,

314
00:17:24,900 --> 00:17:27,780
you should make sure that your, your code is all very well tested.

315
00:17:28,440 --> 00:17:30,820
Oftentimes they're like, okay, how do I track this?

316
00:17:31,140 --> 00:17:36,580
And they jump onto this, uh, vanity metric, what I consider vanity metric, which is code

317
00:17:36,580 --> 00:17:37,400
test coverage.

318
00:17:37,860 --> 00:17:38,000
Right.

319
00:17:38,540 --> 00:17:44,140
So I want to really be careful about when I'm talking about good level of testing.

320
00:17:44,140 --> 00:17:46,040
I don't want, I don't mean test coverage.

321
00:17:46,180 --> 00:17:50,700
You just need the right amount of testing to give you high confidence because that's the

322
00:17:50,700 --> 00:17:51,800
goal at the end of the day.

323
00:17:52,100 --> 00:17:56,380
If you are aiming towards getting a test coverage as close to a hundred percent as possible,

324
00:17:56,380 --> 00:17:58,760
it's actually, there's like diminishing returns.

325
00:17:58,760 --> 00:18:03,640
You just, you can make do with like 80 percent or 90 percent, uh, in different projects.

326
00:18:03,640 --> 00:18:07,440
Like I've been on projects where the code coverage has, uh, the test coverage has been

327
00:18:07,440 --> 00:18:08,640
like 97 percent.

328
00:18:08,640 --> 00:18:12,240
And sometimes it's been down to like 80 or something.

329
00:18:12,780 --> 00:18:18,980
But at the end of the day, we, all we want to track is that, um, we are having the confidence

330
00:18:18,980 --> 00:18:21,880
to deploy code to production.

331
00:18:21,880 --> 00:18:26,460
As soon as I see a green CIP, I don't want to, if I see green, I should be able to deploy.

332
00:18:26,860 --> 00:18:27,180
Right, right.

333
00:18:27,240 --> 00:18:30,320
Yeah, that's that like abstracting that several layers up.

334
00:18:30,380 --> 00:18:36,080
It seems like there's a ongoing, uh, there's like an, in the area of thought technologies,

335
00:18:36,080 --> 00:18:41,120
so to speak, there's an ongoing like tension between, I don't know, absolute thinking versus

336
00:18:41,120 --> 00:18:42,180
relative thinking.

337
00:18:42,180 --> 00:18:46,480
That's the wrong framing, but it's sort of like, you could think about coverage that you

338
00:18:46,480 --> 00:18:46,780
have.

339
00:18:46,780 --> 00:18:48,240
You have a hundred percent coverage.

340
00:18:48,520 --> 00:18:49,440
So everything's green.

341
00:18:49,760 --> 00:18:49,840
Yeah.

342
00:18:50,520 --> 00:18:51,680
But do you care?

343
00:18:51,900 --> 00:18:52,040
Right.

344
00:18:52,120 --> 00:18:53,560
Like, like, just because you have a hundred.

345
00:18:53,580 --> 00:18:54,180
It doesn't matter to you.

346
00:18:54,180 --> 00:18:54,360
Yeah.

347
00:18:54,420 --> 00:18:55,240
It's just a vanity matter.

348
00:18:55,260 --> 00:18:55,440
Yeah.

349
00:18:55,440 --> 00:18:59,480
Just because you have a hundred percent coverage doesn't actually mean that it's good.

350
00:19:00,280 --> 00:19:04,300
So it's, I'm, it's, it's good to make sure you're covering everything, but it's also

351
00:19:04,300 --> 00:19:08,420
good to make sure that you're confident that that means your software is of high quality.

352
00:19:08,420 --> 00:19:12,380
And so how do, how do you, how do people like track that?

353
00:19:12,500 --> 00:19:16,420
Like, do you just ask them or do you do the thing where five people vote and you take

354
00:19:16,420 --> 00:19:17,200
the average of it?

355
00:19:17,220 --> 00:19:18,460
Or like, how do you figure out what that is?

356
00:19:18,460 --> 00:19:23,120
I mean, there are ways, like you can obviously do the voting thing that you talk to the team

357
00:19:23,120 --> 00:19:25,480
and ask like, what's our confidence in our test right now?

358
00:19:25,680 --> 00:19:25,900
Yeah.

359
00:19:26,620 --> 00:19:32,620
But yes, this is one of those difficult things because it's not easy to estimate confidence.

360
00:19:34,000 --> 00:19:41,740
Although you have some tells, like one of the easiest ones is if you, if you can ask

361
00:19:41,740 --> 00:19:43,560
the data, hey, I see a green CI board.

362
00:19:43,720 --> 00:19:46,060
Can I deploy your code to production?

363
00:19:46,060 --> 00:19:50,800
And if they're like, yes, that's, that's a sign that, yeah, there is enough confidence.

364
00:19:51,220 --> 00:19:53,120
And that's what we are aiming towards.

365
00:19:53,660 --> 00:19:59,220
The further away it is from that ideal state, the less reliable your test.

366
00:19:59,220 --> 00:20:07,020
That uses like a Nassim Talebian kind of trick, which is like, he would call it skin in the

367
00:20:07,020 --> 00:20:12,260
game, which is to say like, if you want to get good, I don't know math stuff, but if you

368
00:20:12,260 --> 00:20:17,200
want to get like good probability predictions from someone, you have to make it so that if

369
00:20:17,200 --> 00:20:19,220
they're wrong, there's a consequence.

370
00:20:19,960 --> 00:20:20,100
Right.

371
00:20:20,320 --> 00:20:23,620
And then, and then so they're still going to be wrong because it's probability and stats

372
00:20:23,620 --> 00:20:26,500
and stuff, but like, it at least makes them put in that extra effort.

373
00:20:26,560 --> 00:20:30,080
Just like you're saying, you would go to the developer and say like, all right, it's Friday.

374
00:20:30,620 --> 00:20:31,920
Can we deploy this code?

375
00:20:32,140 --> 00:20:34,960
If something goes wrong, you're going to have to come in on Saturday.

376
00:20:35,060 --> 00:20:38,560
Now, if you tell me we can't deploy the code, you can still leave for the weekend.

377
00:20:38,620 --> 00:20:39,520
We'll come back on Monday.

378
00:20:39,680 --> 00:20:40,520
It's totally fine.

379
00:20:40,600 --> 00:20:40,720
Yeah.

380
00:20:40,720 --> 00:20:43,420
But I just want to make sure that you're confident in this code.

381
00:20:43,640 --> 00:20:43,860
Yeah.

382
00:20:43,980 --> 00:20:48,860
I mean, obviously like we don't, I'm not encouraging anyone to come back on Saturday.

383
00:20:49,300 --> 00:20:51,460
But I mean, it's just a demonstration of how you have some consequences.

384
00:20:51,460 --> 00:20:55,640
It's one of those things because there are a few vanity metrics, like for example, like

385
00:20:55,640 --> 00:20:56,500
velocity, right?

386
00:20:56,540 --> 00:20:58,940
The actual number of points per week.

387
00:20:59,160 --> 00:21:01,920
What you're aiming for is not like crazy high points.

388
00:21:02,000 --> 00:21:06,440
You're not saying like, I want to aim for 50 points of velocity this week.

389
00:21:06,520 --> 00:21:09,600
What you want is consistency and you want to see it growing week after week.

390
00:21:09,680 --> 00:21:10,600
That's what you're aiming for.

391
00:21:10,960 --> 00:21:16,300
Similarly, in your test coverage, you want to see a relatively high number, but there is

392
00:21:16,300 --> 00:21:26,620
no hard and fast rule that if you are able to hit 95% test coverage, your testing is on

393
00:21:26,620 --> 00:21:29,020
point and you don't need to worry about anything else.

394
00:21:29,160 --> 00:21:29,760
No, that makes sense.

395
00:21:29,760 --> 00:21:37,400
Because I've been in organizations where the technical leadership decided, let's put a

396
00:21:37,400 --> 00:21:40,280
number, let's put like 80% test coverage.

397
00:21:40,380 --> 00:21:42,820
And if you don't hit 80% test coverage, you can't deploy your code.

398
00:21:43,080 --> 00:21:43,300
Yeah.

399
00:21:43,400 --> 00:21:47,760
It's really easy to get high test coverage because you can just have one test that exercises

400
00:21:47,760 --> 00:21:50,500
all your code and never actually test anything.

401
00:21:50,880 --> 00:21:51,640
Yeah, that's true.

402
00:21:51,780 --> 00:21:52,040
Exactly.

403
00:21:52,040 --> 00:21:53,860
You can reduce the raw number of tests.

404
00:21:53,860 --> 00:21:58,560
You can market, you can really get around these kinds of metrics.

405
00:21:59,340 --> 00:22:04,560
And as engineers, we are really smart and we find easy solutions to solve these problems.

406
00:22:04,560 --> 00:22:05,040
Right, right.

407
00:22:05,180 --> 00:22:11,140
But getting a high test coverage number, it shouldn't be the goal of the team.

408
00:22:11,180 --> 00:22:12,060
It's a happy side effect.

409
00:22:12,440 --> 00:22:12,820
Right, right.

410
00:22:12,900 --> 00:22:13,520
And so, all right.

411
00:22:13,580 --> 00:22:18,740
So, we found our seams or our boundaries, we've got some tests in, and we have confidence

412
00:22:18,740 --> 00:22:20,080
that we could do these changes.

413
00:22:20,380 --> 00:22:21,700
So, then what do you do?

414
00:22:21,840 --> 00:22:25,580
Like, how do you, I assume the next step is you actually make the changes?

415
00:22:26,700 --> 00:22:29,600
I mean, we are making changes all throughout this.

416
00:22:29,940 --> 00:22:30,300
Right, right.

417
00:22:30,300 --> 00:22:35,600
We don't just say, okay, let's take two months to sit and refactor all your code before we

418
00:22:35,600 --> 00:22:36,680
start making changes.

419
00:22:36,800 --> 00:22:37,200
Okay, okay.

420
00:22:37,200 --> 00:22:43,540
So, in general, what we like to do is we start with something, we start with one single user

421
00:22:43,540 --> 00:22:48,880
functionality that we have talked to users, we have validated it, something that's important.

422
00:22:49,060 --> 00:22:53,360
We have made sure that businesses invested in trying to solve this one problem for their

423
00:22:53,360 --> 00:23:01,360
users, and we start by implementing that new functionality, and along the way, slowly refactor

424
00:23:01,360 --> 00:23:06,200
the parts of the existing code that this new piece of code touches, and so on and so forth.

425
00:23:06,200 --> 00:23:12,160
And we slowly want to change as much as possible, but it starts with that one seed.

426
00:23:12,460 --> 00:23:15,220
We don't want to go boil the ocean, we want to start small.

427
00:23:15,220 --> 00:23:15,500
Right, right.

428
00:23:15,680 --> 00:23:20,700
So, you sort of attack, correct me if I'm wrong, but you attack the big picture of testing,

429
00:23:20,840 --> 00:23:20,980
right?

430
00:23:21,060 --> 00:23:24,600
Like, we have to test it, we have to be confident that we can test the whole system.

431
00:23:24,600 --> 00:23:27,280
We add some level of, because at the end, again, we need confidence.

432
00:23:27,360 --> 00:23:32,280
And then very early on, when you've identified some boundary, you actually go in and start making

433
00:23:32,280 --> 00:23:33,440
changes to the code.

434
00:23:33,560 --> 00:23:33,720
Yeah.

435
00:23:33,920 --> 00:23:37,160
Almost, maybe not exactly in parallel, but very early on.

436
00:23:37,280 --> 00:23:37,400
Yeah.

437
00:23:37,400 --> 00:23:41,960
And then you have overall confidence that the system can be checked, and you're really adding

438
00:23:41,960 --> 00:23:43,940
more tests to that little component that you have.

439
00:23:43,940 --> 00:23:50,160
And what do you find, like, I don't know, like, what's the oldest type of code that you found

440
00:23:50,160 --> 00:23:51,600
that you go in and change?

441
00:23:52,200 --> 00:23:53,340
Oldest type of code?

442
00:23:53,380 --> 00:23:57,860
Or not type, but like, do you go in and you're like, oh, this was written 10 years ago, and

443
00:23:57,860 --> 00:23:58,800
you have to change your...

444
00:23:58,800 --> 00:24:03,020
I've actually been part of a project where we were trying to replace something that was

445
00:24:03,020 --> 00:24:06,840
written 15 to 20 years ago?

446
00:24:06,840 --> 00:24:07,540
Mm-hmm.

447
00:24:07,540 --> 00:24:13,060
It's a really old piece of code, but we kind of, the...

448
00:24:13,060 --> 00:24:19,520
It was written in a mainframe system, and the decision was made that we will replace it

449
00:24:19,520 --> 00:24:22,600
rather than trying to update parts of it.

450
00:24:22,600 --> 00:24:22,880
Right.

451
00:24:22,880 --> 00:24:25,000
Because it no longer made sense to do that.

452
00:24:25,000 --> 00:24:27,560
So, it was part of, like, a slow replacement.

453
00:24:27,840 --> 00:24:33,800
So, because this was a backend system, we could go by API endpoint by API endpoint.

454
00:24:33,940 --> 00:24:34,340
Oh, right.

455
00:24:34,400 --> 00:24:38,040
So, you could just replace the actual implementation between the API...

456
00:24:38,040 --> 00:24:38,180
Yeah.

457
00:24:38,260 --> 00:24:41,800
So, we would just, like, route the new API endpoints to our system.

458
00:24:41,860 --> 00:24:42,440
Right, right, right.

459
00:24:42,460 --> 00:24:42,900
We could do that.

460
00:24:43,140 --> 00:24:43,380
Okay.

461
00:24:43,500 --> 00:24:47,780
So, that's sort of a way, if you have to make a decision, like, we're going to rewrite this

462
00:24:47,780 --> 00:24:50,400
code, or, you know, update the code, or we're just...

463
00:24:50,400 --> 00:24:51,440
Let me rephrase that.

464
00:24:51,440 --> 00:24:55,900
We're going to modify this code versus we're just going to rewrite it.

465
00:24:56,060 --> 00:24:57,280
We're going to, like, get rid of it totally.

466
00:24:57,280 --> 00:24:58,460
I mean, it's always a decision, yeah.

467
00:24:58,620 --> 00:24:58,760
Yeah.

468
00:25:00,100 --> 00:25:05,200
I am wary of just asking people to just rewrite whatever doesn't work.

469
00:25:05,240 --> 00:25:05,840
Yeah, it's expensive.

470
00:25:06,200 --> 00:25:06,980
It's very expensive.

471
00:25:07,500 --> 00:25:12,340
And, basically, although you would be getting a lot of velocity and everyone is happy, devs

472
00:25:12,340 --> 00:25:16,180
are happy, in reality, users are not getting any new value.

473
00:25:16,180 --> 00:25:21,560
You have to first reach, like, at least value parity to the existing system before you're

474
00:25:21,560 --> 00:25:22,960
able to provide new value.

475
00:25:23,360 --> 00:25:23,540
Yeah.

476
00:25:23,660 --> 00:25:23,820
Right?

477
00:25:23,900 --> 00:25:25,760
So, you would spend...

478
00:25:25,760 --> 00:25:31,440
You're expecting your clients and the dev teams to invest months or maybe even, like,

479
00:25:31,480 --> 00:25:37,020
years of effort trying to replace the system and doing the whole time the company is not

480
00:25:37,020 --> 00:25:38,040
releasing new functionality.

481
00:25:38,040 --> 00:25:40,480
Yeah, and then the software just acts exactly like it did before.

482
00:25:40,780 --> 00:25:41,020
This is scary.

483
00:25:41,320 --> 00:25:41,740
Yeah, exactly.

484
00:25:41,900 --> 00:25:42,880
So, you don't want that.

485
00:25:42,880 --> 00:25:48,220
So, there has to be a very clear reason, there has to be a very clear goal for why we are

486
00:25:48,220 --> 00:25:48,660
doing what.

487
00:25:48,860 --> 00:25:51,200
Yeah, that's always a part of...

488
00:25:51,200 --> 00:25:52,440
And we're not the only ones to do it.

489
00:25:52,480 --> 00:25:57,780
But part of the labs process and overall pivotal approach, I think, is good is, like, someone

490
00:25:57,780 --> 00:25:59,140
should care, right?

491
00:25:59,280 --> 00:25:59,520
Yeah.

492
00:25:59,580 --> 00:26:03,840
Like you were saying early on, like, you need to identify the business side of things and

493
00:26:03,840 --> 00:26:05,200
find out what they need to happen.

494
00:26:05,420 --> 00:26:07,460
And that's ultimately what you're servicing, right?

495
00:26:07,540 --> 00:26:11,820
Like, if you've got to do all this rigmarole that we talked about of updating legacy code,

496
00:26:11,820 --> 00:26:16,720
the end is still, like, we need to make the software better, not just update it or whatever,

497
00:26:16,960 --> 00:26:20,320
which I think that's really easy for people to track up.

498
00:26:20,660 --> 00:26:26,180
And then also, it's not only the nerd's fault, it's also, like, the business people, they need

499
00:26:26,180 --> 00:26:30,120
to know that they should be involved in that and be pushing for it, that they should expect

500
00:26:30,120 --> 00:26:30,320
it.

501
00:26:30,540 --> 00:26:30,680
Yeah.

502
00:26:30,800 --> 00:26:36,800
I mean, oftentimes, in our projects, the technical aspects are not the most challenging

503
00:26:36,800 --> 00:26:38,140
things to overcome.

504
00:26:38,500 --> 00:26:44,040
It's often just changing the mindset or trying to make sure that the different parts of the

505
00:26:44,040 --> 00:26:50,280
business and the different parts of business, IT, design, user support, all of them are working

506
00:26:50,280 --> 00:26:51,980
together on trying to solve something.

507
00:26:52,360 --> 00:26:58,780
So, the thing I like to remind devs is that, you know, anything can be built, but should

508
00:26:58,780 --> 00:26:59,900
it be built is the question.

509
00:27:00,560 --> 00:27:01,160
That's right.

510
00:27:01,380 --> 00:27:02,740
Yeah, you could do whatever you want.

511
00:27:03,020 --> 00:27:08,260
And so, in doing this work, are there sort of, like, common tools that you end up using

512
00:27:08,260 --> 00:27:09,260
a lot and that come up?

513
00:27:09,420 --> 00:27:13,860
Like, what, like, like, if you were to get your, I guess, in the world of software, it's

514
00:27:13,860 --> 00:27:15,900
metaphoric, but your metaphoric toolbox together.

515
00:27:16,260 --> 00:27:19,460
Like, like, what are the things that you try, you typically want to start with?

516
00:27:19,660 --> 00:27:26,340
Like, do you have, like, code scanners or, like, do you use a lot of, like, you know, auto

517
00:27:26,340 --> 00:27:29,820
refactoring things or you right click on a mouse button and it does stuff or like, what?

518
00:27:29,920 --> 00:27:30,700
I don't even know.

519
00:27:30,840 --> 00:27:32,560
So, what are the things that come up over and over again?

520
00:27:32,560 --> 00:27:34,560
There's a lot of different tools that we use.

521
00:27:35,060 --> 00:27:38,160
I mean, but we keep it really light for the most part.

522
00:27:38,160 --> 00:27:43,220
So, we do, I like IntelliJ and it comes with lots of-

523
00:27:43,220 --> 00:27:44,280
Everyone around here seems to like that.

524
00:27:44,280 --> 00:27:44,760
Yeah, exactly.

525
00:27:44,960 --> 00:27:47,700
So, I like IntelliJ and it's an easy way.

526
00:27:47,960 --> 00:27:56,020
It's good to, since just stepping back one step is one of the things that we try to focus

527
00:27:56,020 --> 00:28:01,580
on is pairing and that is our way of introducing new concepts and making sure that we are able

528
00:28:01,580 --> 00:28:07,020
to ramp up new devs or, like, our counterparts from the client side.

529
00:28:07,600 --> 00:28:13,080
And when we're doing pairing, it's often nice to come up with a shared understanding on the tools,

530
00:28:13,400 --> 00:28:18,480
languages, and frameworks that we're going to use for this duration so that everyone is comfortable.

531
00:28:18,640 --> 00:28:23,280
So, I can't be writing in Vim while my pair wants to work with Emacs.

532
00:28:23,600 --> 00:28:24,280
It's not going to work.

533
00:28:24,380 --> 00:28:29,100
So, we both have to, like, one person has to be like, okay, I'll work in your way.

534
00:28:29,480 --> 00:28:29,880
Yeah.

535
00:28:30,200 --> 00:28:35,720
So, oftentimes we come to that agreement on what tools we use and we try to standardize

536
00:28:35,720 --> 00:28:42,640
at least, like, some things around, like, code style or just, like, auto-refactoring.

537
00:28:43,180 --> 00:28:45,760
IntelliJ does a really good job because you have plugins that do that.

538
00:28:45,840 --> 00:28:46,000
Right.

539
00:28:46,240 --> 00:28:50,820
So, like, you can have the Google Java styling if you're using Java.

540
00:28:51,060 --> 00:28:56,380
Similarly, JavaScript has lots of different style existing available style guides that you

541
00:28:56,380 --> 00:28:56,780
can use.

542
00:28:57,120 --> 00:29:01,940
So, that just makes sure that the code looks similar from different pairs.

543
00:29:02,300 --> 00:29:07,240
And it's one of those low-hanging fruits that really helps the readability of your code

544
00:29:07,240 --> 00:29:07,460
base.

545
00:29:08,480 --> 00:29:17,260
Other than that, one of the other tools that we set up very early on in our system is having

546
00:29:17,260 --> 00:29:20,040
a repository and having a continuous integration system.

547
00:29:20,540 --> 00:29:24,880
So, one of the happy side effects of test-driven development is that you have a lot of tests

548
00:29:24,880 --> 00:29:31,380
and good coverage and you can rely on a machine to run your tests every time anyone makes changes

549
00:29:31,380 --> 00:29:32,020
to the code base.

550
00:29:32,180 --> 00:29:32,300
Right.

551
00:29:32,300 --> 00:29:39,540
And the CI will be our automated pipeline that promotes code between different environments.

552
00:29:40,400 --> 00:29:45,420
And each time it promotes the code and it gets closer to production, the level of confidence

553
00:29:45,420 --> 00:29:47,740
in that particular build is higher and higher.

554
00:29:47,960 --> 00:29:48,120
Yeah.

555
00:29:48,220 --> 00:29:53,240
And once you're happy and everybody is, like, signed off, you can just automatically just deploy

556
00:29:53,240 --> 00:29:53,920
it to product.

557
00:29:54,200 --> 00:29:55,300
So, it should...

558
00:29:55,920 --> 00:29:59,800
We try to make it as seamless as possible to get to production because getting to production

559
00:29:59,800 --> 00:30:02,740
is as easily as possible is very important.

560
00:30:03,080 --> 00:30:03,320
Yeah, yeah.

561
00:30:03,320 --> 00:30:06,860
Because it shouldn't be something that, like, okay, fine, let's all stop work.

562
00:30:07,120 --> 00:30:08,420
Let's freeze our code today.

563
00:30:08,580 --> 00:30:12,420
And then next one week, we are just going to do prod support.

564
00:30:12,700 --> 00:30:13,520
We don't want that.

565
00:30:13,600 --> 00:30:15,060
We don't want it to be a big ceremony.

566
00:30:15,240 --> 00:30:17,680
We just want it to be, like, a regular thing that we can do anytime.

567
00:30:18,020 --> 00:30:18,720
Yeah, that makes sense.

568
00:30:18,800 --> 00:30:19,220
That makes sense.

569
00:30:20,080 --> 00:30:26,060
So, as part of these processes, sometimes we do have some tools that can do, like, security

570
00:30:26,060 --> 00:30:27,120
vulnerability checks.

571
00:30:27,240 --> 00:30:31,360
It can do, like, test coverage, just one thing.

572
00:30:31,940 --> 00:30:38,000
We sometimes, especially when we're working with really old systems, which we don't fully

573
00:30:38,000 --> 00:30:42,640
understand, with lots of conditionals and things like that, we sometimes opt in for a

574
00:30:42,640 --> 00:30:43,660
mutation testing tool.

575
00:30:43,940 --> 00:30:44,820
Wait, what's that?

576
00:30:44,820 --> 00:30:52,060
This one, these are some tools that can, what it does, it goes through your actual code.

577
00:30:52,960 --> 00:30:56,480
And oftentimes, when you're doing test coverage, it's a static analysis.

578
00:30:56,660 --> 00:30:58,720
It looks at the lines of code that are being executed.

579
00:30:59,400 --> 00:31:05,960
But when you do look at mutation testing tools, it goes through all the possible code parts.

580
00:31:06,360 --> 00:31:06,920
Oh, okay.

581
00:31:06,920 --> 00:31:11,500
All the possible execution parts in your code, and then it flips some ifs here and there,

582
00:31:11,760 --> 00:31:14,080
and sees if any of your tests have broken.

583
00:31:14,440 --> 00:31:14,960
That's interesting.

584
00:31:14,960 --> 00:31:20,120
So, suppose you flip an if, like, a fork in your logic, and nothing breaks.

585
00:31:20,340 --> 00:31:23,480
That means there's one part of the logic that is not being tested.

586
00:31:23,940 --> 00:31:27,340
And it will give you a report that talks about it.

587
00:31:27,560 --> 00:31:29,180
So, it's a really interesting tool.

588
00:31:29,180 --> 00:31:33,960
I've used it only once, and, yeah, it was for a very specific module, which we didn't

589
00:31:33,960 --> 00:31:35,000
have visibility on.

590
00:31:35,580 --> 00:31:37,240
I'm not really sure how it scales.

591
00:31:37,620 --> 00:31:38,420
I've never tried it.

592
00:31:38,600 --> 00:31:39,320
Yeah, yeah, yeah.

593
00:31:39,320 --> 00:31:43,000
It seems like with certain types of code bases, it would run longer than we live.

594
00:31:44,540 --> 00:31:47,240
Yeah, and, yeah, that's it.

595
00:31:47,280 --> 00:31:48,540
We try to keep it really light.

596
00:31:49,060 --> 00:31:54,840
Most of the other things that we introduce, most of the other practices are some things

597
00:31:54,840 --> 00:31:56,720
that we introduce while working together.

598
00:31:56,720 --> 00:32:03,080
So, we have a lot of whiteboards, so we try to jump up there and draw lines and boxes

599
00:32:03,080 --> 00:32:05,580
to have these shared conversations.

600
00:32:06,260 --> 00:32:11,680
We have regular retros to understand what the team's feeling.

601
00:32:12,000 --> 00:32:12,240
Yeah.

602
00:32:12,440 --> 00:32:13,880
Lots of meatware, as it were.

603
00:32:14,340 --> 00:32:14,600
Exactly.

604
00:32:15,040 --> 00:32:20,340
And so, I mean, when you're working with older languages, do you need, like, tools to, like,

605
00:32:20,460 --> 00:32:21,940
comprehend COBOL or something?

606
00:32:22,120 --> 00:32:23,740
Or, like, does that ever come up?

607
00:32:23,820 --> 00:32:29,320
Luckily, I've never had to work with COBOL, but there have been some clients who have come

608
00:32:29,320 --> 00:32:33,260
to us asking about, like, working in these kind of older technologies.

609
00:32:34,540 --> 00:32:41,060
We would like to have some domain experts and maybe language experts embedded into the team

610
00:32:41,060 --> 00:32:46,060
so that we are able to, you know, help each other out in making sure that the clients are successful.

611
00:32:46,060 --> 00:32:46,280
Yeah.

612
00:32:46,520 --> 00:32:51,040
My understanding of COBOL is as long as you keep the cap slot key on, then you're an expert at it.

613
00:32:51,400 --> 00:32:52,980
You just need to do that.

614
00:32:53,080 --> 00:32:53,240
Yeah.

615
00:32:54,340 --> 00:32:57,420
And oftentimes, if you're working with COBOL, it's not just the COBOL.

616
00:32:57,560 --> 00:32:59,300
It also comes in, like, a mainframe.

617
00:32:59,320 --> 00:33:00,080
It's all the ancillary stuff.

618
00:33:00,340 --> 00:33:00,580
Yeah, yeah, yeah.

619
00:33:00,580 --> 00:33:04,980
And it's, like, deploying things to it is really difficult.

620
00:33:05,120 --> 00:33:08,160
And working, if you're not working on-site, it's very difficult.

621
00:33:08,280 --> 00:33:09,300
So there's lots of challenges.

622
00:33:09,620 --> 00:33:09,800
Right.

623
00:33:09,840 --> 00:33:13,260
And we try to resolve them case by case.

624
00:33:13,420 --> 00:33:15,460
So then finally, before we wrap up.

625
00:33:15,580 --> 00:33:15,740
Okay.

626
00:33:16,000 --> 00:33:20,240
So as you're going through this process, like, and you mentioned a little bit of this, but

627
00:33:20,240 --> 00:33:26,120
what are, like, what's your sort of, like, starting amount of, let's, I don't want to

628
00:33:26,120 --> 00:33:28,180
say just metrics, but the reporting that you do.

629
00:33:28,280 --> 00:33:31,940
Like, you know, you've got the test coverage and you might have reporting about the confidence

630
00:33:31,940 --> 00:33:35,920
that you have, but are there other things that come up when you're in the meeting with

631
00:33:35,920 --> 00:33:39,560
the managers or, you know, meeting with yourself is one thing in retros.

632
00:33:39,560 --> 00:33:44,660
But, like, when it's time for the monthly review of should you still get funding and how is

633
00:33:44,660 --> 00:33:45,080
this going?

634
00:33:45,240 --> 00:33:47,360
Like, what do you present to those people?

635
00:33:47,660 --> 00:33:51,800
So at the end of the day, what we present is the real product.

636
00:33:52,380 --> 00:33:58,780
So if you're deploying often enough, we always show the real product and be like, here it

637
00:33:58,780 --> 00:33:58,960
is.

638
00:33:59,460 --> 00:34:00,480
Feel free to play with it.

639
00:34:01,800 --> 00:34:08,400
We try to steer clear of reports because often they give you a false sense of predictability

640
00:34:09,280 --> 00:34:11,920
This is just like a unit test coverage stuff, right?

641
00:34:11,920 --> 00:34:13,920
Like, you get obsessed with the report.

642
00:34:14,000 --> 00:34:14,100
Yeah.

643
00:34:14,200 --> 00:34:21,120
But at the same time, we are working with organizations that have been doing these strict deadlines,

644
00:34:21,420 --> 00:34:23,700
burn down charts, burn up charts, whatever.

645
00:34:24,160 --> 00:34:27,340
So we can obviously generate these reports.

646
00:34:27,340 --> 00:34:34,800
We use tools like Tracker has a really detailed analytics tool that lets you see how you're progressing

647
00:34:34,800 --> 00:34:39,960
through the different stories and different features, which is something that we can use.

648
00:34:43,200 --> 00:34:45,920
But in general, we try to steer clear of those.

649
00:34:45,920 --> 00:34:50,060
Well, just reporting on the work being done.

650
00:34:50,560 --> 00:34:54,620
I mean, it's generally good enough and whichever direction you're burning, like that would make

651
00:34:54,620 --> 00:34:59,640
sense because then it seems like the conceptual thing you have to tell people is this is representative

652
00:34:59,640 --> 00:35:02,860
of us delivering features basically, right?

653
00:35:02,860 --> 00:35:06,820
Every time we complete one of these tasks, it gets us closer to business value or whatever.

654
00:35:07,400 --> 00:35:09,520
And that's what the task represents.

655
00:35:09,740 --> 00:35:11,100
Each one is user value.

656
00:35:11,420 --> 00:35:11,520
Right.

657
00:35:11,620 --> 00:35:17,280
And every point that you're delivering is one point of user value.

658
00:35:17,640 --> 00:35:18,000
Right, right.

659
00:35:18,000 --> 00:35:21,200
And it's definitely a mindset shift.

660
00:35:21,840 --> 00:35:29,200
It is oftentimes difficult conversations, but clients generally trust us because they are

661
00:35:29,200 --> 00:35:31,900
coming to Pivotal Labs for a reason.

662
00:35:32,480 --> 00:35:37,040
And oftentimes what we tell them is this is going to be very weird.

663
00:35:37,400 --> 00:35:38,420
This is going to be uncomfortable.

664
00:35:38,660 --> 00:35:40,400
You just have to trust in us.

665
00:35:40,400 --> 00:35:44,120
Try it for a couple months and you will really see the value in it.

666
00:35:44,280 --> 00:35:49,620
And that's what I tell all my client devs and my peers when we're working on a project.

667
00:35:50,000 --> 00:35:50,320
All right.

668
00:35:50,380 --> 00:35:51,700
Well, how long have you been living in Singapore?

669
00:35:52,260 --> 00:35:54,500
I've been in Singapore for now eight years.

670
00:35:54,640 --> 00:35:55,080
Oh, eight years.

671
00:35:55,080 --> 00:35:55,460
So pretty long.

672
00:35:55,660 --> 00:35:55,900
Wow.

673
00:35:56,320 --> 00:36:02,000
So when you came here and you've been here this whole time, what is one food item that

674
00:36:02,000 --> 00:36:03,020
they only have here?

675
00:36:03,740 --> 00:36:07,660
Or it could also be a food item that's elsewhere, but it's in a certain style here.

676
00:36:08,820 --> 00:36:10,040
Or is there not one?

677
00:36:10,840 --> 00:36:15,840
Singapore, the beautiful thing about Singapore is that it's like a cultural melting pot.

678
00:36:16,180 --> 00:36:19,220
You get cuisines from pretty much anywhere in the world.

679
00:36:19,560 --> 00:36:21,740
And sometimes it's like a slightly different take on everything.

680
00:36:22,120 --> 00:36:26,120
For example, we have Indian food in Singapore.

681
00:36:26,600 --> 00:36:30,040
It's like from, it's kind of Malaysian Indian food.

682
00:36:30,260 --> 00:36:30,700
Right, right.

683
00:36:30,740 --> 00:36:33,480
It's like Pratas and Roti John.

684
00:36:34,480 --> 00:36:37,800
It's like a very different thing from Indian food from India.

685
00:36:37,800 --> 00:36:39,880
Right, it's sort of like American Chinese food.

686
00:36:39,880 --> 00:36:41,060
Basically, that's one.

687
00:36:41,460 --> 00:36:46,380
You also have chicken rice, which people always tout as the one thing you have to try.

688
00:36:46,460 --> 00:36:48,500
So what's chicken rice?

689
00:36:48,680 --> 00:36:49,520
Oh, chicken rice.

690
00:36:49,720 --> 00:36:56,000
It's basically boiled chicken on top of fragrant rice.

691
00:36:56,560 --> 00:36:57,280
Oh, okay, okay.

692
00:36:57,300 --> 00:36:58,200
It's relatively healthy.

693
00:36:58,940 --> 00:37:01,640
And it's delicious as well.

694
00:37:01,640 --> 00:37:06,840
I bet it's healthier than those like hanging slabs of bacon that I see everywhere.

695
00:37:07,660 --> 00:37:09,380
Yeah, you get that as well.

696
00:37:09,580 --> 00:37:09,860
Right, right.

697
00:37:10,280 --> 00:37:15,260
One of my favorite combination in Singapore, just because of the diversity, is you get like

698
00:37:15,260 --> 00:37:22,820
really good craft beer with dim sum from like, you know, Chinese dim sum and malay satay.

699
00:37:23,020 --> 00:37:24,920
Put these together, it's a delicious meal.

700
00:37:25,080 --> 00:37:25,540
Oh, yeah.

701
00:37:26,540 --> 00:37:30,900
So then along those lines, the other thing I'm always interested in is like how people

702
00:37:30,900 --> 00:37:33,540
go up the ceremony of how they eat things, right?

703
00:37:34,320 --> 00:37:38,620
And, you know, like for example, well, let me ask you this one.

704
00:37:38,680 --> 00:37:39,620
You probably figured this out.

705
00:37:39,740 --> 00:37:42,400
It's popular here in Singapore and elsewhere.

706
00:37:43,040 --> 00:37:46,760
You get one of those big cast iron pots that's all like sizzling.

707
00:37:46,840 --> 00:37:50,220
There's even in the mall, there's like this little strip that's like, cook it your way or

708
00:37:50,220 --> 00:37:52,840
whatever, but like, so how do you operate that?

709
00:37:52,920 --> 00:37:59,520
So I get one of these meals with this pot, it's all sizzling and usually it's dry.

710
00:37:59,660 --> 00:38:00,580
Let's just take the dry one.

711
00:38:00,640 --> 00:38:04,560
There's no soup and each thing is kind of perfectly compartmentalized on its own.

712
00:38:04,620 --> 00:38:06,920
And I might have some little side condiments.

713
00:38:07,060 --> 00:38:08,140
Like what do I do with that?

714
00:38:09,040 --> 00:38:14,400
I think, I don't think I've received like official instruction on how to do this, but generally

715
00:38:14,400 --> 00:38:20,180
what I end up doing is I just let the meat cook with some of the condiments on it.

716
00:38:20,880 --> 00:38:22,080
Oh, that's you flavor the meat as it's cooking.

717
00:38:22,080 --> 00:38:23,660
Yeah, I flavor the meat as it's cooking.

718
00:38:23,680 --> 00:38:24,140
That's good.

719
00:38:24,280 --> 00:38:24,520
Yeah.

720
00:38:24,820 --> 00:38:25,620
It's delicious.

721
00:38:26,040 --> 00:38:26,160
Okay.

722
00:38:26,160 --> 00:38:28,200
And you cook it to the level you want.

723
00:38:28,320 --> 00:38:32,620
And once you're pretty much happy with how cooked it is, you can just mix it all together

724
00:38:32,620 --> 00:38:33,280
and then...

725
00:38:33,280 --> 00:38:33,760
I see.

726
00:38:34,040 --> 00:38:34,980
That's fascinating.

727
00:38:35,140 --> 00:38:37,540
So the whole point is the meat is still cooking.

728
00:38:37,640 --> 00:38:37,840
Yeah.

729
00:38:37,840 --> 00:38:42,400
I'm repeating back what you said and you season it with the condiments and then the act

730
00:38:42,400 --> 00:38:45,840
of mixing it with everything else cools it down and stops the cooking and then you can

731
00:38:45,840 --> 00:38:46,160
eat it.

732
00:38:46,240 --> 00:38:46,380
Yep.

733
00:38:46,820 --> 00:38:47,100
Wow.

734
00:38:47,160 --> 00:38:48,000
That's a good idea.

735
00:38:48,660 --> 00:38:52,100
That's kind of like the, you know, when you go to Korean barbecue and you get the uncooked

736
00:38:52,100 --> 00:38:54,200
stuff and you cook it exactly the way you want.

737
00:38:54,320 --> 00:38:54,720
It's similar.

738
00:38:54,980 --> 00:38:58,180
But I don't think it's like a Singaporean thing per se.

739
00:38:58,680 --> 00:39:00,480
I'm sure that's the way it works everywhere.

740
00:39:00,480 --> 00:39:00,660
Yeah.

741
00:39:00,760 --> 00:39:07,120
The hot plate that you're talking about, I think it's a Japanese thing.

742
00:39:07,440 --> 00:39:07,720
Yeah.

743
00:39:08,260 --> 00:39:09,980
It's at Korean restaurants a lot too.

744
00:39:10,080 --> 00:39:10,300
It's a Korean.

745
00:39:10,460 --> 00:39:10,640
Okay.

746
00:39:10,720 --> 00:39:10,880
Yeah.

747
00:39:10,900 --> 00:39:14,060
Well, I mean, it could be both, but that's where I always get it at a Korean restaurant.

748
00:39:14,120 --> 00:39:14,560
That's awesome.

749
00:39:14,720 --> 00:39:14,980
All right.

750
00:39:15,000 --> 00:39:15,440
Well, that's good.

751
00:39:15,520 --> 00:39:19,060
Well, if people wanted to, I don't know, catch up with you or see what you're doing,

752
00:39:19,120 --> 00:39:20,820
you got like a Twitter account or anything like that?

753
00:39:20,820 --> 00:39:28,460
I'm not really active on Twitter, but my Twitter handle is, it's Rahul Rajiv without any

754
00:39:28,460 --> 00:39:30,420
of the bubbles in it.

755
00:39:30,420 --> 00:39:32,680
So it's just R-H-L-R-J-V.

756
00:39:33,440 --> 00:39:36,060
And that's the same as my GitHub account.

757
00:39:36,280 --> 00:39:43,380
And if you are hearing this from inside Pivotal or even outside, my email is rrajiv at pivotal.io.

758
00:39:43,540 --> 00:39:44,020
Well, there you go.

759
00:39:44,060 --> 00:39:44,440
Well, thanks.

760
00:39:44,520 --> 00:39:45,660
Well, thanks for going over this.

761
00:39:45,720 --> 00:39:46,080
It was great.

762
00:39:46,300 --> 00:39:46,460
Sure thing.

763
00:39:47,100 --> 00:39:47,840
It's great talking to you.

764
00:39:48,000 --> 00:39:50,720
Well, as always, this has been Pivotal Conversations.

765
00:39:50,940 --> 00:39:56,320
If you want to get all the old episodes, you can go to soundcloud.com slash Pivotal Conversations.

766
00:39:56,980 --> 00:39:59,020
And we also post show notes somewhere.

767
00:39:59,440 --> 00:40:01,940
I know, at pivotal.io slash podcast.

768
00:40:02,160 --> 00:40:05,940
Usually every Thursday or as I do nowadays whenever I get around to it.

769
00:40:06,060 --> 00:40:07,300
So we'll see everyone next time.

770
00:40:07,540 --> 00:40:08,000
Bye-bye.