WEBVTT 1 00:00:13.720 --> 00:00:16.420 Hey, Ben. 2 00:00:16.420 --> 00:00:17.560 Hey Matt. 3 00:00:17.560 --> 00:00:23.200 We should really not be talking over the intro that I edited out. We should be more professional about this thing. 4 00:00:23.200 --> 00:00:24.060 I don't know. It can be a good source of outtakes. 5 00:00:24.060 --> 00:00:34.700 It's better than me singing, I suppose. It gives me the cold opens and cold closes, which is worthwhile. 6 00:00:34.700 --> 00:00:38.660 We'll do the cold open at the open. That'll be a new interesting way to do a cold open. 7 00:00:38.660 --> 00:01:05.020 A new interesting way to actually just launch into one and go, "Oh, I suppose we should press the button to make the theme music play now." I was actually thinking about arranging the theme music for clarinet, cello and drums so that me and my kids could play it together. But, you know, given that they think everything I do is the most terrible, boring thing to do, I don't imagine that will go through. 8 00:01:05.020 --> 00:01:08.600 But anyway, that's not what we're here to talk about today. 9 00:01:08.600 --> 00:01:18.110 We can talk about anything that we want. That's the amazing thing about this podcast. No one can stop us. 10 00:01:18.110 --> 00:01:40.660 That's very true. What power we have! You and I already discussed this when we went out with friends last night. On the walk to the bar, you suggested this topic and we had a little pre-game. So I think we should talk about it - and that topic is about hands. 11 00:01:40.660 --> 00:01:49.940 And you've spoken about this as one of your little aphorisms and sayings. 12 00:01:49.940 --> 00:02:01.900 So the one that we're going to talk about today is "dirty hands are right." This comes from an internet thing - this is before memes, I feel like. 13 00:02:01.900 --> 00:02:11.500 I mean, memes in their cultural, appropriated sense, as opposed to the scientific sort of Richard Dawkins-ish memes. 14 00:02:11.500 --> 00:02:42.340 The Cult of Done, yes. This was like a set of ideas invented by a person named Bre Pettis. They basically sat down one day and came up with approximately 10 rules for getting things done. It took them like 25 minutes. They're like, "I'm just gonna knock this out and these are gonna be my rules." I actually have the poster over my- 15 00:02:42.340 --> 00:02:52.440 I was going to say there's a- I can't quite see it because of the blur, but I know it's there from having been to your house. It's the Done Manifesto, right? 16 00:02:52.440 --> 00:03:14.360 Yes, the Done Manifesto. The Done Manifesto is the manifesto of the Cult of Done. This was introduced to me by a friend of mine back in, gosh, 2008 something like that. The thing that I love about it is it literally was created in like 20 minutes. 17 00:03:14.360 --> 00:03:35.620 It's not super well thought out, but that's kind of the point - it's about getting things done. It doesn't have to be super well thought out. It just needs to be done. Then you can move on from it, learn from it, improve it. But if you don't ever finish it, then you can't do those things. 18 00:03:35.620 --> 00:03:57.580 You can Google this, listener - the Cult of Done, the Done Manifesto. One of the things in the Cult of Done, I'm going to try to quote this directly from the manifesto: "People without dirty hands are wrong. Doing something makes you right." 19 00:03:57.580 --> 00:04:01.220 That's actually interesting because that's not how you've said it to me before. 20 00:04:01.220 --> 00:04:12.300 Yeah, I've said it as "dirty hands are right." That's the shorter version of it. 04:21.77 Matt Godbolt Right, but there is there's a subtle difference. 21 00:04:12.300 --> 00:04:59.040 I wanted to talk about this because I've repeated it many times since learning it, and my thinking has become more nuanced, perhaps like the original quote. It has shifted because I think if you want to apply this in an effective and useful way, you need to really understand what it means to have dirty hands. Being able to walk around saying "dirty hands are right" is a great way to make a lot of mistakes very quickly if you're not careful about wielding that power. 22 00:04:59.040 --> 00:05:31.960 Got it, so should we talk about that simplified version first? Forgive me for simplifying further, but what this essentially says is: if somebody's in the middle of doing something and they say "we should do it this way," and you aren't in the middle of doing it - trust them. They're probably the right person to be doing it. Don't throw stuff in from the peanut gallery just because you might have an opinion. Don't bikeshed. Unless you're doing it yourself, you have no horse in this race. Shut up and let them get on with it, right? 23 00:05:31.960 --> 00:05:32.420 Exactly right. 24 00:05:32.420 --> 00:05:57.780 And that's how I've taken it from all the things. From that point of view, I understand why it gets things done, because it means you don't have this constant interruption from other people saying, "Have you thought about doing this?" And you're like, "No, I haven't. And I'm not going to." I chose to do it this way, and I will therefore get this thing done, and then we will move on to something else. 25 00:05:57.780 --> 00:06:15.280 Obviously, everyone can think of reasons why this isn't always true, but it's not a bad first approximation for a startup environment like we've been in for a while. Let's just say: if you're interested in doing it and you're already doing it, you keep on trucking and we'll trust you to get on with it. 26 00:06:15.280 --> 00:06:52.240 Absolutely right. If your goal is to get things done, this is very effective. If your goal is to be SOX-compliant, this is maybe not the approach you want to use. But I think it's actually more interesting to talk about the exceptions. When are dirty hands wrong? 27 00:06:52.240 --> 00:06:53.900 When you're preparing food - that's right! It's a slightly silly example, but... 28 00:06:53.900 --> 00:07:14.060 When you're preparing food is a great example of when dirty hands are wrong. You're in the kitchen, you just took the trash out and you're like, "I'm gonna cut up this chicken anyway..." 29 00:07:14.060 --> 00:07:58.780 The SOX-compliance thing is definitely like, yeah, you can get things done and then you're not going to be compliant. But one of the more important nuances that I've added into this, especially in the last five years, is understanding the distinction between the hands that are currently dirty because they are building a thing, and the hands of the people who will be expected to support that thing. If those are not the same hands, then that is a situation in which dirty hands can be wrong. 30 00:07:58.780 --> 00:08:48.600 Interesting. This is like the typical support engineer versus productionization engineer scenario. At Google, you have the SREs who are supporting it. Way back in time, it was like engineers threw it over the fence, and the SREs just had to deal with the fires. They pushed back, and eventually, it was like: "No, before you can throw this at me, the SRE has to sign off on it," or "You also get to go on the pager duty rotation for that thing" so that everybody learns. Hopefully that distills a correctness out of the process. Just because "Hey, I'm the guy who wrote it, I'm right" - that doesn't follow when I'm not the person who's going to be woken up at 4:00 in the morning when the memory leak finally crashes the machine. 31 00:08:48.600 --> 00:09:06.120 Right. The problem with that transition of "I'm going to build something with dirty hands and then take my filthy dirty hands and use them to hand this thing to you" - just mixing metaphors here for fun... 32 00:09:06.120 --> 00:09:10.460 It's like "free as in goldfish." 33 00:09:10.460 --> 00:09:13.880 I've never heard that before, is that yours? 34 00:09:13.880 --> 00:09:16.620 No, I got that from somewhere. I forget where. 35 00:09:16.620 --> 00:09:24.980 Free as in goldfish, yeah - free as in puppy. Here you go, this is yours now, but it comes with a massive amount of responsibility. 36 00:09:24.980 --> 00:09:43.070 Right. I've given you this thing and really it's like a curse. Now you are responsible for this, and all of my mistakes have become your mistakes, which is great for me and terrible for you. 37 00:09:43.070 --> 00:09:45.620 Yeah. Absolutely awful for you. 38 00:09:45.620 --> 00:10:33.900 I think the main way to avoid this organizationally is just: don't do that. Don't let engineers absolve themselves of their own sins by passing unsupported software off onto other people. I'm going to couch this a little bit - I don't understand how to apply the software engineering techniques that I know and have been successful with to that kind of SRE model, where you have software engineers that build software and then hand it off to SREs to support them. 39 00:10:33.900 --> 00:10:54.240 You can certainly build software that way. I'm not saying you can't. I'm just saying I don't know how to take all of the useful, effective techniques that I have and fit them into that model. They all just completely fall over when you're like, "Yeah, I'm just going to build a thing and then hand it off to these people, and then I'm going to go onto my next project and never worry about it again." 40 00:10:54.240 --> 00:11:20.280 So, one way to avoid this failure of "dirty hands are right" and allow dirty hands to be right is to just don't let people do that. Or maybe more realistically, don't structure your organization to do that on purpose. 41 00:11:20.280 --> 00:11:52.780 It is, as I say - perhaps we should talk about the original phrasing in a second, but I can think of another example where dirty hands may not always be right. I'm definitely trying this on, thinking this through. Folks who are inexperienced, either at the particular task at hand or at software engineering in general... 42 00:11:52.780 --> 00:12:00.000 I mean, I can't help but feel like maybe I'm just a grumpy old man who wants to tell these young whippersnappers what to do. 43 00:12:00.000 --> 00:12:03.160 As previously stated, this is our podcast and we get to say whatever we want. 44 00:12:03.160 --> 00:12:31.940 I know, but it doesn't come without repercussions. I like to be thoughtful about these things, as you are as well. So with that in mind, it is the case that you can have a junior developer or somebody who isn't a software engineer develop something, and you look at it as a seasoned engineer or somebody with more domain experience and go, "Wow, you've really made that hard on yourself." 45 00:12:31.940 --> 00:12:59.100 And at that point, those dirty hands, they're like, "Well, we've just done it this way." This is where we are now. It's hard to argue that they're right. I mean, they're right as much as they've got something done, presumably. Maybe that's how you get out of this. Maybe it's the wiggle words - it's like, well, they're done with V1, and now we need to think about this again as a V2. 46 00:12:59.100 --> 00:13:49.740 I think that point is absolutely correct. Any time that you put someone in a situation where they're not capable of doing the task that you've given to them, you need to think about how you're gonna make it possible for them to succeed. If you want to operate an organization with this mantra of "dirty hands are right," then when you give someone a task they may not be capable of doing, you need to find a way to support them. That cannot be what we were just talking about where you've got some seasoned engineers saying, "Did you think about this? Have you considered this? Have you bikeshedded this?" 47 00:13:49.740 --> 00:14:03.940 Like either at the very end of a pull request when you've designed a system and got it all wrong, or as you say, fly-by - literally someone posts in Slack and says, "Hey, I finished my thing," and then everyone takes a pot shot at it. That's not a good feeling. 48 00:14:03.940 --> 00:14:08.020 So you have to get the seasoned engineers' hands equally dirty. Is that where you're going? 49 00:14:08.020 --> 00:14:31.660 Yes, and in the right way. It depends on what you're trying to do here. I think most software engineering organizations value developing the skills of their engineers. Not all do, but the better ones do. You can use this as an opportunity to teach those junior engineers the things that they don't know that make their dirty hands decisions bad. 50 00:14:31.660 --> 00:14:44.460 And you have to do it in a way where you're not just inflicting your help on them. You need to give them a goal, tell them that this is too much for them. 51 00:14:44.460 --> 00:14:46.880 This is this game on hard mode. 52 00:14:46.880 --> 00:14:58.280 And if I just left you alone, you wouldn't be able to do it well. You're smart, maybe you're going to figure out how to do it, but you're going to make a lot of mistakes if we just give you no help. 53 00:14:58.280 --> 00:15:01.080 So we're going to give you some help, and we're going to structure it in this way. 54 00:15:01.080 --> 00:15:22.640 The particular way that you structure it depends on the person, the organization, and a lot of other factors. The key element here is if you just overwhelm somebody with a task that is way too hard for them, they're going to compensate for that in ways that are bad for everybody. 55 00:15:22.640 --> 00:15:29.600 And one of those ways is to make a bunch of dirty hand decisions that the organization is going to be very upset that they made. 56 00:15:29.600 --> 00:15:38.840 This comes to things you've given presentations about at work - you've talked about the zone of proximal learning. Is that the right term? 57 00:15:38.840 --> 00:15:40.780 Zone of proximal development, yes. 58 00:15:40.780 --> 00:16:06.880 Right, which is like the Goldilocks zone of giving someone something they couldn't achieve on their own, but with a little bit of help, they can achieve. That's the best place to be learning - stretching yourself a little bit without being overwhelmed. 59 00:16:06.880 --> 00:16:24.360 You want to be whelmed. Not overwhelmed, not underwhelmed, you want to be whelmed. That's when you're learning. 60 00:16:24.360 --> 00:16:36.100 Obviously, it requires a certain amount of forethought to give them a task which is stretching, and that you have somebody available to be equally dirty handed in the correct way. 61 00:16:36.100 --> 00:17:23.940 That doesn't always happen though. I'm thinking now in more general terms where folks who are smart and wanted to just get a job done have gone off by themselves and gotten that job done. Then you're stuck with the decisions they made. Like, pragmatically, we needed a solution to store our files, so I made a directory on the network share, and I stored 250,000 files in one directory. And it works for me. 62 00:17:23.940 --> 00:17:36.080 That's exactly right. That's not a junior software engineer. It might be like a senior analyst who learned a little bit of Python just so they can automate some parts of their job and then built this thing. 63 00:17:36.080 --> 00:17:46.220 And you come along one day and you realize that the operation of the entire company depends on this untested 1000 line Python script where half the code is commented out. 64 00:17:46.220 --> 00:18:17.720 I remember the very first day I interviewed at the finance job that brought me to America. I was speaking with one of the people in their London office and I said, "Oh, what are you doing?" expecting to hear some kind of cool debugging story. They said, "Yeah, I'm debugging this spreadsheet." 65 00:18:17.720 --> 00:18:47.020 When I'm trying to explain to people that are not programmers why it's so important to reduce complexity in software, and why we spend so much time with techniques and tools trying to reduce the complexity of our software, they ask why this is so important. 66 00:18:47.020 --> 00:19:01.540 I tell them to imagine they had a spreadsheet that was responsible for their entire company, and there was a whole bunch of stuff in it that they didn't understand, but they were tasked to change it. 67 00:19:01.540 --> 00:19:14.600 Our code base is 150,000 lines. Imagine that spreadsheet was 150,000 lines and not 150,000 lines of data - 150,000 lines of functions. 68 00:19:14.600 --> 00:19:29.460 Formulas that refer to other formulas that refer to other formulas. That's what we're dealing with. And if we don't keep it clean so that people understand how to change it, we'll not be able to change it. 69 00:19:29.460 --> 00:19:35.360 So the next time you come to us and say "we need to add this thing," I'm going to be like "no, I can't." 70 00:19:35.360 --> 00:19:39.740 I literally can't because every time I change this cell, all these other cells break. 71 00:19:39.740 --> 00:19:47.900 And we don't understand it and it's too complicated. 72 00:19:47.900 --> 00:20:15.880 But to your point - you have that person that is not a junior engineer eager to learn, that wants to be involved. It is a person who is just trying to do their other job, which is not software engineering, and they have found a way to automate some things. They asked ChatGPT a question and they paste the answer into whatever.py. 73 00:20:15.880 --> 00:20:25.000 I think that is absolutely a case where dirty hands are not right. 74 00:20:25.000 --> 00:20:42.900 Interestingly, this is a facet of another situation in which dirty hands can be wrong. And the hands in question in this case can be someone like us, a very seasoned professional software engineer who's spent a lot of time doing this. And it's when the scenario is flipped. 75 00:20:42.900 --> 00:21:03.900 Imagine that you have the interest rate calculation code, and you don't really understand the calculation that's going on here. There's some analyst or an accountant who is an expert in pricing this thing. 76 00:21:03.900 --> 00:21:05.940 Maybe not an industry calculation, but like a pricing algorithm. 77 00:21:05.940 --> 00:21:15.240 About pricing options or something complicated. 78 00:21:15.240 --> 00:21:34.660 If you go off and try to build that solution without really involving that other person, you're almost certainly not going to do it right. Or you're going to design it in a way where maybe it fits the one particular use case that you thought about, but if you want to adapt it or use it for something else, it won't work. 79 00:21:34.660 --> 00:21:42.380 You're missing the context to be able to generalize it in a meaningful way. 80 00:21:42.380 --> 00:22:11.540 And that has other dimensions too. Like imagine a security aspect, or a privacy aspect where you may be an accomplished software engineer, but if you don't understand the legal ramifications of whether or not we obfuscate this field or that field, or the international law around cookie storage. 81 00:22:11.540 --> 00:22:17.620 Then you can be a very good software engineer and have dirty hands and be very wrong. 82 00:22:17.620 --> 00:22:57.320 So really, the dirtiness is a function of responsibility and skill. There's a threshold by which you have to have a certain amount of skill and responsibility to be able to claim the dirty hand mantle. All the things we talked about until you brought in the analyst or the security expert have been about programmers having dirty hands, and if they've got enough seniority or experience then they can prevent the bike shedding that could happen when you have four or five other smart people with reasonable ways of doing it. 83 00:22:57.320 --> 00:23:06.440 Your analyst isn't necessarily a good programmer and so their hands don't necessarily qualify for the dirtiness in this case. 84 00:23:06.440 --> 00:23:09.920 But equally, we're not saying this is unique to programmers. 85 00:23:09.920 --> 00:23:29.800 It's just like, if this isn't your main role, if you're not expected to be doing this, then maybe you don't count in the dirty hand analogy. 86 00:23:29.800 --> 00:23:42.000 Let's go back to reading the original thing, because I think we've got the inverse of the actual phrase. 87 00:23:42.000 --> 00:23:44.480 It says "people without dirty hands are wrong." 88 00:23:44.480 --> 00:23:47.160 Doing something makes you right. 89 00:23:47.160 --> 00:24:01.280 If we just take that first half, that sort of says, if you've got no skin in the game at all, then you just can't be right by definition. 90 00:24:01.280 --> 00:24:10.120 That stops the peanut gallery. That stops the pot shots from other people coming along saying, "Have you ever written a database before? No?" 91 00:24:10.120 --> 00:24:29.940 Then stop telling me how to manage files because I'm doing it right now. 92 00:24:29.940 --> 00:24:43.020 That is the condensed version. The second sentence is "dirty hands are right. Doing something makes you right." 93 00:24:43.020 --> 00:25:17.620 But focusing on that first part is probably the more universally true thing. If you're not involved in something, don't just offer your opinion because you feel like you want to offer your opinion. If you don't have any skin in the game, if you're not the person who's going to get rewarded if it succeeds or yelled at if it fails, then be careful. 94 00:25:17.620 --> 00:25:48.320 Unsolicited, because obviously, if somebody comes to you and says, "Look, you don't know what I'm doing here, but can I use you as my rubber duck to debug or to bounce ideas off of?" 95 00:25:48.320 --> 00:26:01.560 We're going to give you some dirty gloves that you can put on. 96 00:26:01.560 --> 00:26:16.900 So the original thesis of this discussion as we walked to Monks Pub last night was, "When are dirty hands wrong?" 97 00:26:16.900 --> 00:26:27.580 It's interesting because it's definitely something that we have trotted out over the last few years as being like, "Let's just stop bike shedding about it." 98 00:26:27.580 --> 00:26:41.440 I think it's a really good way to think about the importance of building software in teams. A lot of organizations like to build cross-functional teams. 99 00:26:41.440 --> 00:27:01.660 You'll have a team made up of two or three software engineers, maybe a trader, an analyst, and a risk expert. Or you'll have a product team with a UX expert, a couple of front-end developers, a couple of back-end developers, and a security expert. 100 00:27:01.660 --> 00:27:19.780 These cross-functional teams, if you structure and align them right, saying "you're going to build this thing together, you need to work together, you need to come to consensus" - that's when it works. 101 00:27:19.780 --> 00:27:35.040 The one senior engineer by themselves isn't going to know the pricing algorithm, but when you put that person on the same team as someone who has that knowledge and you have everyone working together toward a common goal, that's when it works. 102 00:27:35.040 --> 00:27:40.720 You collectively have dirty hands. You collectively have ownership of the outcome. 103 00:27:40.720 --> 00:27:49.680 If it's not going to work, then you're all going to suffer. And if it does work, then you all succeed. That's a much better model. 104 00:27:49.680 --> 00:28:06.760 For organizational staffing and resourcing reasons, that can be difficult to achieve. Sometimes those individuals might be very limited in their time or numbers, and you can't put them on every team because their goals get divided. 105 00:28:06.760 --> 00:28:22.660 You can't always do it. But when you can, it solves a lot of the problems with dirty hands. You get all the capabilities and skills needed to solve a problem. 106 00:28:22.660 --> 00:28:41.260 Together and pointed in the same direction. And as long as people aren't jerks to each other, generally magic will happen. 107 00:28:41.260 --> 00:28:44.880 Whereas if you have a structure that's more like that SRE structure... 108 00:28:44.880 --> 00:28:59.020 Where it's "you're not on the same team, you get to go home at night and you don't" - that kind of approach. Then this attitude can be very dysfunctional. 109 00:28:59.020 --> 00:29:02.840 If you think "I'm doing this and I'm right." 110 00:29:02.840 --> 00:29:10.400 I don't even know how you do something like that and not have it blow up in your face. 111 00:29:10.400 --> 00:29:25.140 I just don't understand how you could apply these ideas to that. Maybe someone out there has done it. Maybe they'll tell us. 112 00:29:25.140 --> 00:29:45.840 I think we've pretty much covered the dirtiness of hands. If you've got skin in the game and you have someone on your team that can help you, or you have the skills yourself, or maybe you are helping someone else, then dirty hands are the right approach. 113 00:29:45.840 --> 00:30:12.580 But it's not always the case that just because you're doing a task, you get a free pass on the Dirty Hands rule. If it's not your area of expertise or if you're out of your depth, that needs to be considered. And obviously, if you're preparing food, you should always have clean hands. 114 00:30:12.580 --> 00:30:29.760 And sing the alphabet song while you wash your hands, people. 115 00:30:29.760 --> 00:30:40.480 Given where we are now, I'm imagining the theme tune is playing over in the edit. So I guess on that note, we should call it quits and chat next time. 116 00:30:40.480 --> 00:30:42.480 Sounds good.