WEBVTT 1 00:00:22.000 --> 00:00:24.000 Good afternoon, Ben. 2 00:00:24.000 --> 00:00:27.380 Hello, Matt. How are you? 3 00:00:27.380 --> 00:00:33.660 Wow. Yeah, you've confused the heck out of half of our listeners now. Not only did I not say hi Ben, and I said good afternoon. 4 00:00:33.660 --> 00:00:47.930 You also asked me how I'm doing. I'm doing really great. Thank you very much. I am still recovering from a bout of COVID that I caught from seeing far too many people at a conference, which is unfortunately the occupational hazard of such a thing. 5 00:00:47.930 --> 00:00:48.550 Oh no. 6 00:00:48.550 --> 00:00:57.240 But I'm doing fine other than just a bit of a cough. How about you? 7 00:00:57.240 --> 00:00:58.300 I'm doing all right. I have no real complaints. I only have zeroth world complaints. 8 00:00:58.300 --> 00:01:01.800 Imaginary complaints that have no real component to it. 9 00:01:01.800 --> 00:01:04.700 Right. Yes. There's no real component to it. They're truly imaginary. 10 00:01:04.700 --> 00:01:07.400 Whereas if there's a mix, it's a complex complaint. 11 00:01:07.400 --> 00:01:11.050 They're 90 degrees out from any sort of real complaint. 12 00:01:11.050 --> 00:01:14.440 Is that right? 13 00:01:14.440 --> 00:01:22.390 So, yeah, you've completely derailed immediately, one minute in and already going, what was it we were going to talk about? 14 00:01:22.390 --> 00:01:22.420 Right. Right. 15 00:01:22.420 --> 00:01:27.100 And I'm going to blame the COVID for that because that's what you're allowed to do. 16 00:01:27.100 --> 00:01:29.360 Right. Something, something, brain fog. 17 00:01:29.360 --> 00:01:30.980 You've got fog brain. That's like a, yeah. 18 00:01:30.980 --> 00:01:35.490 Yeah, I feel like someone posted something on Twitter to the effect of something like, 19 00:01:35.490 --> 00:01:35.520 Uh-huh. Right. 20 00:01:35.520 --> 00:01:46.520 Cognitive hypochondria or something like that. And I'm like, maybe it is a bit of that. But, you know, even if it's psychosomatic, it seems really not. Anyway, that's not what we were going to talk about today. 21 00:01:46.520 --> 00:01:46.560 Right. 22 00:01:46.560 --> 00:01:56.040 We're going to talk about not my inability to think, but perhaps thinking or getting something else to think for us. 23 00:01:56.040 --> 00:01:57.720 Yes. Outsourcing your thinking to another thinky thing. 24 00:01:57.720 --> 00:02:03.700 Outsourcing it to a thing that can think or maybe can't think. 25 00:02:03.700 --> 00:02:04.220 Right. Yeah. 26 00:02:04.220 --> 00:02:09.280 Let's not get too caught up in the specifics, the semantics, but something which looks like it might think. 27 00:02:09.280 --> 00:02:09.420 Yeah. 28 00:02:09.420 --> 00:02:17.600 And that person may be an intern, right? As far as I can tell, they can think, but I've got no proof in my solipsistic worldview. 29 00:02:17.600 --> 00:02:19.000 Nope. 30 00:02:19.000 --> 00:02:23.790 Everyone else is a figment of my imagination. So that seems, no, no. 31 00:02:23.790 --> 00:02:23.900 Mm-hmm. 32 00:02:23.900 --> 00:02:29.420 But realistically, no. You had an idea before we pressed the record button. 33 00:02:29.420 --> 00:02:43.880 I did. So I think there's sort of an interesting phenomenon going on with, you know, the development of software engineers, not the development of software, but the engineers that make them. 34 00:02:43.880 --> 00:02:58.880 And AI and sort of learning things and the way that people think about, you know, costs. I don't know. There's a whole bunch of interrelated things that I think are going on right now. 35 00:02:58.880 --> 00:03:01.280 And it might make for a good podcast. 36 00:03:01.280 --> 00:03:08.240 Well, we can certainly sort of explore it, you and I without really any plan as per usual. 37 00:03:08.240 --> 00:03:08.540 Yeah, yeah. 38 00:03:08.540 --> 00:03:23.080 Yeah, as per usual. So I think the way... First, I'm going to back up a level and be like, what is programming? Just to set some really basic foundational layers for this conversation we're about to have. 39 00:03:23.080 --> 00:03:29.860 Okay, that seems reasonable, yeah. So are we talking about software development, like the whole process of it, or programming very specifically, or... 40 00:03:29.860 --> 00:03:31.480 Software development, really, because there's a dimension. 41 00:03:31.480 --> 00:03:32.920 Okay, okay. So yeah, tell me what you... 42 00:03:32.920 --> 00:03:57.310 There's an aspect of this that I think is important for this conversation, which is a lot of what software development is, in my opinion at least, is taking the sort of messy, inconsistent, poorly specified real world, not the imaginary world, the real world and mapping it into, 43 00:03:57.310 --> 00:03:59.020 Okay. 44 00:03:59.020 --> 00:04:10.080 the completely uncompromising world of computers where everything has to be specified exactly. There is no room for vagaries. 45 00:04:10.080 --> 00:04:16.380 And if you are unaware of the vagaries that you're creating, the computer will create them for you much to your dismay. 46 00:04:16.380 --> 00:04:18.340 Yes. 47 00:04:18.340 --> 00:04:21.960 Very, very apt. 48 00:04:21.960 --> 00:04:24.340 You know, let's talk about undefined behavior, right? 49 00:04:24.340 --> 00:04:26.700 Right. Exactly that. Yeah. 50 00:04:26.700 --> 00:04:47.600 So when a thing is engaging this activity of mapping the real world to the sort of digital world, inevitably, one of the things that you have to be able to do is report back to whatever it is that is telling you to, like, hey, map this thing into a computer for me, please. 51 00:04:47.600 --> 00:04:55.860 And be like, actually the thing that you asked me to do is physically impossible. I can't actually do that because it is impossible to do that. 52 00:04:55.860 --> 00:04:57.100 Right. Mm-hmm. 53 00:04:57.100 --> 00:05:18.640 Or logically impossible. And so that's sort of like a feedback loop that is essential to the act of programming. I think it's naive to think that anyone really, even an experienced software engineer, can just say, go build a thing, and then the thing appears 100% of the time, right? 54 00:05:18.640 --> 00:05:32.560 Because there is a very good chance that unless you have actually done the work of taking your ideas and mapping them into the digital world, there's a very good chance that they don't actually map and that compromises will need to be made. 55 00:05:32.560 --> 00:05:36.160 Questions will need to be answered. Things will need to be clarified. and 56 00:05:36.160 --> 00:05:42.640 Well, if I can, if I can interrupt, even with a well specified problem, 57 00:05:42.640 --> 00:05:42.640 you know, Mm-hmm. 58 00:05:42.640 --> 00:05:55.240 where you say solve X, I need something that will do X. There is an almost infinite number of valid solutions to that problem. 59 00:05:55.240 --> 00:06:09.120 And some of them will be more maintainable than the others. Some of them, which may or may not be a goal. Some of them may be more performant than others, which may or may not be a goal. So many dimensions feed into what is a good X, 60 00:06:09.120 --> 00:06:11.860 that aren't often specified up front. 61 00:06:11.860 --> 00:06:23.000 And you know if you work for a high-frequency trading company, then probably it's a given that when you're writing something to go in the trading system, it should be as fast as possible and do no extra work. 62 00:06:23.000 --> 00:06:23.000 Right. 63 00:06:23.000 --> 00:06:23.000 Yeah. 64 00:06:23.000 --> 00:06:34.380 But if you're writing a framework for wider consumption that's, you know, for anyone to come and use, then you probably want to be as generable as general... 65 00:06:34.380 --> 00:06:36.080 as general and a generally... 66 00:06:36.080 --> 00:06:36.080 Generable. 67 00:06:36.080 --> 00:06:46.520 as general as possible without tying yourself in knots. And so there's a huge amount of unspoken requirements that also come. 68 00:06:46.520 --> 00:06:46.710 Yeah. Yeah. 69 00:06:46.710 --> 00:06:46.800 Yeah. 70 00:06:46.800 --> 00:06:54.320 And as an experienced software developer, you like to think you can make an intelligent guess about it, but also you can be so wrong so often on that, and we are. 71 00:06:54.320 --> 00:06:54.460 Right. 72 00:06:54.460 --> 00:07:03.980 So that's not even taking into account the fact that what you are asked may not, as you say, *be* possible because it involves solving the halting problem or achieving something that is unattainable. 73 00:07:03.980 --> 00:07:04.280 Right. 74 00:07:04.280 --> 00:07:06.020 Right, right. Going faster than the speed of light or something like that. 75 00:07:06.020 --> 00:07:06.430 Things like this. 76 00:07:06.430 --> 00:07:06.580 Yeah, right. 77 00:07:06.580 --> 00:07:18.880 Right, yeah. So, yeah, there is a back and forth process where you say, maybe even internally, do I do X? Given two equally possible solutions, should I do X or Y? 78 00:07:18.880 --> 00:07:29.050 What are the trade-offs between them and all that stuff? And at that point, you may turn around and go back to your stakeholder and say, hey, do you want it fast or do you want it general or whatever? 79 00:07:29.050 --> 00:07:29.240 Right. 80 00:07:29.240 --> 00:07:33.340 Even that may not be the right axis, but something like that. 81 00:07:33.340 --> 00:07:35.970 Yeah, right. But there's a trade-off to be made and it's not clear what the right one is, right 82 00:07:35.970 --> 00:07:36.020 Yeah. 83 00:07:36.020 --> 00:07:39.600 And there are, well, there's, yeah, there are many, many, many trade-offs to be made. 84 00:07:39.600 --> 00:07:39.600 Yeah. Yeah. 85 00:07:39.600 --> 00:07:40.700 Yeah. 86 00:07:40.700 --> 00:07:56.900 Right. So the question is, who or, and it's my belief by the way. So this is a problem, this doing this mapping from the physical world into the digital world is a thing that people have been trying to solve for a very long time. 87 00:07:56.900 --> 00:08:01.770 When I was in school, the big thing was like UML. 88 00:08:01.770 --> 00:08:02.310 Are you going to say UML? 89 00:08:02.310 --> 00:08:02.400 Yeah. 90 00:08:02.400 --> 00:08:02.940 Yes, exactly. 91 00:08:02.940 --> 00:08:06.280 Right. You know, you know what I'm going to say, you know, the deal. 92 00:08:06.280 --> 00:08:09.640 Yeah, I'm a bit older than you, but it was still around after me. 93 00:08:09.640 --> 00:08:20.380 Yes. Right. UML, Rational Rose, all of these tools that were like, you know, we're going to have all these things that let you solve this problem. 94 00:08:20.380 --> 00:08:20.520 Oh my golly, there's a word I haven't heard. 95 00:08:20.520 --> 00:08:24.460 And they didn't work, of course. But um it is my current... 96 00:08:24.460 --> 00:08:29.170 And, know, things like 4GLs and things like that came along and said, hey, you know, you'll never need to write another line of code. 97 00:08:29.170 --> 00:08:29.300 Yes, yes. 98 00:08:29.300 --> 00:08:31.780 You just draw a picture and it'll all just work. 99 00:08:31.780 --> 00:08:52.460 Right, right. So I have come to believe over the sort of decades of this that the least specific definition of what um you need to create in order to actually solve a problem on a computer is the source code, right? 100 00:08:52.460 --> 00:09:06.240 Anything more abstract than that is not going to work because of this mapping problem, because you're going to have things that are unspecified, whether you know they're unspecified or not, you're going to have things that are unspecified that need to be specified in order for it to work. 101 00:09:06.240 --> 00:09:06.340 Yep. 102 00:09:06.340 --> 00:09:17.440 And so human beings have kind of invented programming languages, this incredible, fairly objective, like the most objective thing that we've maybe ever made beside maybe mathematics, 103 00:09:17.440 --> 00:09:38.300 of like how you actually solve a problem and that you cannot be any, you can't be any less abstract than that and actually solve the problem. Right? So fast forward to today and we have a new attempt, which is probably, which is turning out to be a much better attempt to solve this mapping problem, which is, 104 00:09:38.300 --> 00:09:40.080 Specifically LLM coding assistant type stuff that people get. 105 00:09:40.080 --> 00:09:45.400 Specifically, LLMs and maybe more specifically agentic coding, right? Although, you know, sort of depends. 106 00:09:45.400 --> 00:09:46.640 Yeah, fair. Yeah. There are other things. 107 00:09:46.640 --> 00:10:03.400 Yeah. Like it is conceivable, and I know that you have actually done this, to completely offload this mapping problem to an agent and have it solve a problem in a way where it creates source code, but you have not even read it, let alone understand it, right? 108 00:10:03.400 --> 00:10:05.320 Correct. 109 00:10:05.320 --> 00:10:15.240 And that is not, and I'm going to reference what you were kind of saying a little bit before the podcast here. That is not completely different from what we've been doing for decades where a manager says, hey, go build this thing. 110 00:10:15.240 --> 00:10:26.140 And the developer goes and writes some code and then comes back to the manager and the manager doesn't read the code at all. Sends pull requests, whatever, approve. And then they try it and they're like, yeah, that doesn't work. Do the thing again. 111 00:10:26.140 --> 00:10:32.800 and then it goes and tries again. And then we just, you know, it's like we're doing the exact same thing. You know, what what what was the phrase that you used for this? 112 00:10:32.800 --> 00:10:39.320 Oh, I need to find it out now. this is I'm going to have to say thank you to ah my friend Jeremy, our mutual friend Jeremy, in fact. 113 00:10:39.320 --> 00:10:39.600 Yeah. 114 00:10:39.600 --> 00:10:42.550 It says, managers have been vibe coding forever. 115 00:10:42.550 --> 00:10:43.780 Right. Yes, exactly. 116 00:10:43.780 --> 00:10:50.160 And it's the loop of like what the vibe coding looks like. And at the end, the manager says, good job, but be faster the next time. 117 00:10:50.160 --> 00:10:54.660 Or worse. 118 00:10:54.660 --> 00:10:55.040 Right. 119 00:10:55.040 --> 00:11:11.080 So I think there was a perception, I don't know, a year-ish ago, maybe a couple of years ago when these tools started coming out. And I know you and I maybe even had some conversations about this of like, you know, is software engineering as a profession going to go away because of this? 120 00:11:11.080 --> 00:11:22.760 Is this going to just be, you know, non-technical people or maybe just like junior people who are right out of school who are not managers yet using these tools to build software? 121 00:11:22.760 --> 00:11:23.060 Right. 122 00:11:23.060 --> 00:11:49.970 And if you especially if you look at some of like the job numbers in like the the, you know, numbers of openings and people were are looking for jobs. It definitely seems like what has happened in the last year or two is that it's the junior engineers who are not getting hired because what people are doing is they're just putting these tools in the hands of very senior software engineers who are now able to get way more leverage than they may be used to, or at least the same amount of leverage, but with many fewer people, right? 123 00:11:49.970 --> 00:11:50.060 Yeah. Yeah. 124 00:11:50.060 --> 00:11:52.640 Like you don't have the thing of like, okay, we're going to give you two 125 00:11:52.640 --> 00:11:56.190 or three you know less skilled or more junior engineer. 126 00:11:56.190 --> 00:11:56.380 Junior. 127 00:11:56.380 --> 00:11:56.770 Yeah. Whatever. 128 00:11:56.770 --> 00:11:56.960 Yeah. 129 00:11:56.960 --> 00:11:57.340 Yeah. 130 00:11:57.340 --> 00:12:04.620 um And then you're go you you have this team and you just do this thing of like, you sort of farm the work out to them and you do basically this like vibe manager coding thing, right? 131 00:12:04.620 --> 00:12:05.200 Yeah. Yeah. 132 00:12:05.200 --> 00:12:15.500 And it's like, okay, yeah, don't do that. Just have like two or three different Claude agents running on your computer at the same time. And there's your team essentially, and you just do that, right? 133 00:12:15.500 --> 00:12:16.500 Yeah. 134 00:12:16.500 --> 00:12:21.660 And so that makes sense. And that's like a very rational thing for businesses to do, right? 135 00:12:21.660 --> 00:12:37.230 But just so of like to go back to your point, I think you know the point we were making earlier on was that the senior developers would be the ones that would be being cut because they're expensive and their experience and skill level could be replaced by a sufficiently intelligent 136 00:12:37.230 --> 00:12:38.100 Right. Yeah. 137 00:12:38.100 --> 00:12:38.920 AI system. 138 00:12:38.920 --> 00:12:39.660 Yeah. Yeah. 139 00:12:39.660 --> 00:12:52.670 And then it just takes guidance from either a junior dev or no dev at all, because, you know, you just have the manager, the non-technical stakeholder just talking into a ah window saying, no, no, make it bluer. 140 00:12:52.670 --> 00:12:52.780 Right. 141 00:12:52.780 --> 00:12:57.680 No, no, no. Now make it do this when you click that kind of thing or more, more generally. 142 00:12:57.680 --> 00:12:58.560 Yeah. 143 00:12:58.560 --> 00:13:11.870 But the surprise here is at least fast forward a year, that does not seem to be the case. In fact, the opposite. It's more the senior folks like you and I who have gone, hey, this seems valuable and we can perhaps get some use out of it as a tool. 144 00:13:11.870 --> 00:13:12.220 Right. 145 00:13:12.220 --> 00:13:27.720 But the junior folks have not been ah able to get the same amount of leverage as we do. But their role is being potentially usurped by the fact that, yeah, you don't need a team of people anymore. You need one guy and and and the patience of Job to talk to. 146 00:13:27.720 --> 00:13:27.720 Yeah. 147 00:13:27.720 --> 00:13:37.000 So many forgetful, stupid, ah you know, ah ADHD robots. But nonetheless, that, and obviously I don't think we're there yet. 148 00:13:37.000 --> 00:13:37.040 Right. Right. 149 00:13:37.040 --> 00:13:37.080 Right. right 150 00:13:37.080 --> 00:13:43.510 You know, you and I both work in companies where there are still are junior folks around, but that was the interesting point. 151 00:13:43.510 --> 00:13:44.140 Right. 152 00:13:44.140 --> 00:13:51.040 That seems to be what is kind of happening as as as best that I can tell. And, you know, I'm not far from like an economic expert here. 153 00:13:51.040 --> 00:13:51.150 Yeah, we, 154 00:13:51.150 --> 00:13:51.560 i just see what I see. 155 00:13:51.560 --> 00:14:12.890 Well, and also we only really have yeah our own vantage point from where we are and we're in a particularly niche industry perhaps as well, um you know where technical um and so and and ah technical knowledge and a specific ah domain-specific knowledge is quite kind of important to what we do and maybe in a more general world where 156 00:14:12.890 --> 00:14:13.680 Right. Also very true. 157 00:14:13.680 --> 00:14:28.900 AI has been trained on more publicly available information. I mean, that may be they're easier to replace folks with. I mean, certainly in my limited experience of the last few months where I've been less employed, um fewer fewer employed, fewer employed, 158 00:14:28.900 --> 00:14:29.880 If you were employed. 159 00:14:29.880 --> 00:14:43.120 I've been using AI to help me on my open source projects, and that's all in TypeScript and JavaScript and web. And there's like thousands of websites that have good examples that presumably it's been trained on. 160 00:14:43.120 --> 00:14:43.600 Right. Yeah. 161 00:14:43.600 --> 00:14:54.930 When I go to conferences and talk to people, they have more equivocal results with more technical things like the minutia of C++, especially some of the new things that are coming down the pipeline because there exists no training data. 162 00:14:54.930 --> 00:14:55.460 Yeah. Yeah. 163 00:14:55.460 --> 00:14:58.970 And that's also true of me, right? You know, someone to come to me and say, can you write me a co-routine? 164 00:14:58.970 --> 00:14:59.140 yeah 165 00:14:59.140 --> 00:15:15.680 I'd be like, give me a few months to go and learn how to do that first, and then I'll be able to. But maybe our particular industry and the intersection of the technology we use and the special things we use makes it more likely that we will still hire and use people. 166 00:15:15.680 --> 00:15:19.860 I don't know. I'm just spitballing here, as indeed this entire podcast is. 167 00:15:19.860 --> 00:15:27.720 No. Yeah. I mean, what, that's the whole point of the podcast, but yeah, that's a very, very important point is that it's not like we're talking, it's not like I've analyzed a bunch of economic data to come to this conclusion, right? 168 00:15:27.720 --> 00:15:27.880 That is. 169 00:15:27.880 --> 00:15:37.520 Like this is all mostly from just my personal experience and the things that I've seen in the industry. So, you know, take this with however many grains of salt you believe are appropriate, listener. 170 00:15:37.520 --> 00:15:39.280 Right. 171 00:15:39.280 --> 00:16:02.300 But the kind of result of this now is you have, senior people. And I think the explanation is, is that sort of makes sense to me is that it's like LLM still require a decent amount of babysitting and you need to be able to know when they're hallucinating. You need to be able to know when they're going off in a weird direction to correct them. Otherwise they're like, it's the blind leading the blind. 172 00:16:02.300 --> 00:16:14.480 And that sort of explains why you get the most leverage from from having one experienced software engineer, you know, using all these various tools to build the things that you need, right? 173 00:16:14.480 --> 00:16:31.390 And yeah, I mean, I was going to say, i just interject there, I think that knowing about the kind of hallucinations, or I don't know, even if that's such a loaded term these days, but you know, like when, when mistakes are made and assumptions are made about, oh, there must be something that does x and then it says, "Hey, call x", and you know, that's never existed. 174 00:16:31.390 --> 00:16:31.480 Yeah. 175 00:16:31.480 --> 00:16:32.840 it would be great if it did. 176 00:16:32.840 --> 00:16:33.960 Right, yeah. 177 00:16:33.960 --> 00:16:49.540 But like identifying that and keeping it on track, a kind of slightly different techniques or different skills than we've had to use before, because perhaps as an experienced programmer who has led and mentored junior developers, 178 00:16:49.540 --> 00:17:07.290 You know, there is definitely a set of mistakes that I have seen folks fall foul to that sometimes I let them and, you know, you let them and then you guide them through the process of saying, why do you think that didn't work? And, you know, you use it as a teaching point, which is a valuable thing, I think, for everybody to learn from. 179 00:17:07.290 --> 00:17:07.660 Mm-hmm. Mm-hmm. 180 00:17:07.660 --> 00:17:28.660 But those are very human mistakes. And also the process of doing it that way allows them to grow. And LLMs will make those kinds of mistakes as well. And so I think that if you have led a team before that being able to control a suite of LLMs, some of those skills do overlap and you will go, yeah, yeah, please don't do this. 181 00:17:28.660 --> 00:17:37.890 Whatever you do, don't comment out the test to make everything green or don't check in without running things, which you see junior folks do sometimes, you know, like, hey, that's there for a reason. 182 00:17:37.890 --> 00:17:38.260 Yeah, right. Yeah. 183 00:17:38.260 --> 00:17:53.430 Don't just disable it because it didn't make sense to you. You know, always ask me, all that kind of stuff. But the thing that's frustrating is that the LLM will not learn in the same way as your junior devs. You know, the third or fourth time you said, please, look, if I have to do this again, we're going to have to have a talk, serious talk about this. 184 00:17:53.430 --> 00:17:53.740 Right. 185 00:17:53.740 --> 00:17:58.280 Then, you know, the LLM is like, nope, you've told me a hundred times before and I forget every single time. 186 00:17:58.280 --> 00:17:59.420 I mean, like a happy puppy, but... 187 00:17:59.420 --> 00:17:59.820 Right, right. 188 00:17:59.820 --> 00:18:00.020 Yep. 189 00:18:00.020 --> 00:18:01.860 So I think there are some new skills. 190 00:18:01.860 --> 00:18:02.720 I don't have that context in my window, right? Yeah, exactly. 191 00:18:02.720 --> 00:18:12.230 Yeah, right. Yeah. There are definitely some new skills for for leading LLMs. And I think it's like probably a 75% overlap with the kind of skills that help you with a team of real humans. 192 00:18:12.230 --> 00:18:12.560 Yeah. 193 00:18:12.560 --> 00:18:21.780 But there's definitely, there's a 25% that's new for LLMs. And there's also probably, I mean, I'm probably underselling it. There's a ton of much more important, you know, interpersonal skills. 194 00:18:21.780 --> 00:18:26.320 And didactic techniques that you can use when you're trying to help an individual learn. 195 00:18:26.320 --> 00:18:26.520 Yeah. 196 00:18:26.520 --> 00:18:26.720 Mm-hmm. 197 00:18:26.720 --> 00:18:32.000 And I've forgotten where I was going with this, but I interjected nonetheless. You were in the middle of something and I put my hand up and you... 198 00:18:32.000 --> 00:18:47.720 Oh, well, no, I mean, it it's just that, you know, the the phenomena now of having these sort of senior engineers leveraged using these tools is is great from an economic standpoint. Like it makes a lot of sense. 199 00:18:47.720 --> 00:18:49.220 Right. 200 00:18:49.220 --> 00:19:00.230 But the thing that I wonder about is if we are, you know, as an industry, setting ourselves up for future pain. 201 00:19:00.230 --> 00:19:12.840 Because if you do this for too long and the senior engineers are too successful. So the problem with companies in general is that if you have very talented people and you don't pay them enough, they'll leave. 202 00:19:12.840 --> 00:19:25.120 And if you pay them too much, they'll leave because they're rich and they don't need to work anymore. Right. So, you have to sort of, you know, strike this middle ground of not paying them too much so they quit because they don't need to work. 203 00:19:25.120 --> 00:19:29.500 So you're saying the only reason they don't pay me more money is because I might quit. 204 00:19:29.500 --> 00:19:35.860 Uh, well, let me ask you this. I mean, if you had, you know, hundreds of millions of dollars in the bank, would you continue to work? 205 00:19:35.860 --> 00:19:44.900 Probably, but not, yeah, not for... yeah, yeah, yeah, I think I'd never stop doing, I mean, we have the same, I'd never stop doing this stuff, but yeah, but no, yeah, all right, your point stands, Ben, your point stands. 206 00:19:44.900 --> 00:19:49.000 Maybe for yourself. Right. Right. Like I, yeah, I'm never going to stop programming because it's fun, right? But would I be? 207 00:19:49.000 --> 00:20:03.740 Yeah. So this is my point, is you got to pay people the right amount. So eventually, though, they are going to make enough money or they're just going to get to the point where they want to retire and they're going to retire. And the question is, I've just been teaching robots for the last 10 years. 208 00:20:03.740 --> 00:20:07.910 Who's going to take over my job? Who's going to start being the person commanding the robots? 209 00:20:07.910 --> 00:20:08.080 Right. Right. 210 00:20:08.080 --> 00:20:08.360 Right 211 00:20:08.360 --> 00:20:12.820 And this is a problem that capitalism is exceptionally bad at solving. 212 00:20:12.820 --> 00:20:40.240 Right. Of these sort of like long term systemic problems. And so I think what may happen is companies will start to discover that as the junior engineers who are currently looking for jobs sort of diffuse into other industries because they don't find any of the the opportunities that are there, that they're going to start to have a really hard time finding the people who can lead the robots. 213 00:20:40.240 --> 00:20:52.430 And what it what it might be is kind of like this pinch point of like, okay, well, we we got all these productivity gains from from structuring things this way. And it made complete sense for this quarter and the next quarter. 214 00:20:52.430 --> 00:21:06.090 But 10 years on, now we can't find anybody that understands any of this or how to make any of it work because they don't work in this industry anymore. Or those people maybe have just gotten very expensive. Right. 215 00:21:06.090 --> 00:21:10.450 Because there aren't that many of them that sort of survived the filter. Right. 216 00:21:10.450 --> 00:21:10.830 Right. 217 00:21:10.830 --> 00:21:11.200 Yeah. 218 00:21:11.200 --> 00:21:15.060 They will be the COBOL programmers of their of their day. 219 00:21:15.060 --> 00:21:32.260 Yeah. Yeah. Like it'll, just be a much harder, like an even harder thing than it was, you know, 2021 when it was like there were, ridiculous ah salaries being handed out because it's like you couldn't find enough programmers, right? 220 00:21:32.260 --> 00:21:52.270 um On the flip side of this, just thinking of it from like the the sort of trader perspective, it probably also means that in the next five to 10 years, there are going to be some really really, really good quote unquote junior programmers out there that could run circles around anyone else in your organization with 10 years of experience or 20 years of experience 221 00:21:52.270 --> 00:21:52.700 Yeah 222 00:21:52.700 --> 00:21:56.100 And if you can find them, they will be worth their weight in gold. 223 00:21:56.100 --> 00:21:59.760 That's an interesting observation to turn it that way around. Yeah, that's it. 224 00:21:59.760 --> 00:22:08.600 Yeah. No, it's interesting you say that this is a problem with capitalism. This is not a politics podcast, obviously, so i won't we won't go too much into the other choices out there. 225 00:22:08.600 --> 00:22:08.820 Yeah. 226 00:22:08.820 --> 00:22:09.040 Right. 227 00:22:09.040 --> 00:22:13.060 But no, I see it as almost like it's ah it's a local minima. 228 00:22:13.060 --> 00:22:28.080 We're heading towards a local minima, and nobody is going to do the sort of global optimization problem of like, well, now we just have to do the simulated annealing step of just throwing some randomness into it just to see, because we just need to get out of this this potentially local minima. 229 00:22:28.080 --> 00:22:28.630 Yeah. 230 00:22:28.630 --> 00:22:29.740 Right, right. 231 00:22:29.740 --> 00:22:48.600 But to sort of counter that, obviously, the the more... AI um ah proponent or a but I don't even know what the right term would be, but someone who is more confident in the role of AI will just say, well, AI will get better. And actually even the senior engineers, 232 00:22:48.600 --> 00:23:14.360 Won't have to be around anymore. We can just, any anyone can ask that question. And I think that obviously that that gets to the heart of the original thing that you brought up, which is that there is a ton of unknown unknowns that something or someone has to say, either make a call on explicitly and say, look, based on my years of experience, this is a database thing we should probably do use blah, versus this is a trading application, we should probably do something different. 233 00:23:14.360 --> 00:23:15.220 Yeah. 234 00:23:15.220 --> 00:23:24.640 Or you have to ask the the the person who's asking the question, should we optimize for so for throughput or latency? And at which point they're going to say, "the what now"? 235 00:23:24.640 --> 00:23:27.340 And then, so yeah, but where would that line be? And yeah, 236 00:23:27.340 --> 00:23:28.120 Yeah. 237 00:23:28.120 --> 00:23:41.220 Right. I mean, the the the premise here is that the although AI is great, it still hasn't actually solved the mapping problem, where the mapping problem is map the physical world into the digital world in a reliable, sustainable, continuous way. 238 00:23:41.220 --> 00:23:51.940 Obviously, you can have somebody sit down that knows very little about programming and interact with Claude and build like a simple app. Right? Like those things are absolutely possible. Right? 239 00:23:51.940 --> 00:23:52.740 Yeah. 240 00:23:52.740 --> 00:24:06.660 It remains to be seen whether you can build more complicated things that way. It remains to be seen whether you can change things that have been built that way and revise things that have been built that way and grow and scale things that have been built that way. 241 00:24:06.660 --> 00:24:24.020 It's possible that maybe one day, know, Claude version 75 will be so good at all of this that it really it really has like sort of fundamentally solved the mapping problem where you can have a completely non-technical person, somebody who does not really understand that computers work at all, sit down and build. 242 00:24:24.020 --> 00:24:37.480 A horizontally scalable web app or a trading system or you know a microcontroller for something that's going to go in the space shuttle. And all of that stuff will just work. 243 00:24:37.480 --> 00:24:44.740 It's entirely possible. But i I could also see a world where where that problem is so difficult that it doesn't actually get solved. 244 00:24:44.740 --> 00:24:44.860 It's possible, but yeah, it's hard to see. 245 00:24:44.860 --> 00:24:54.440 Well, I mean, that problem is so difficult that it often doesn't get solved even with experienced programmers and experienced, you know, this is a difficult thing to get to get right. 246 00:24:54.440 --> 00:24:54.520 Right. 247 00:24:54.520 --> 00:25:06.700 But there is, i mean, so you mentioned earlier about the fact that I had had some limited experience of like this whole closed loop vibe-codey type thing where I haven't even looked at the the code of something. 248 00:25:06.700 --> 00:25:18.580 And that is that is true. um But uniquely in the situation I was in, I had a very clear goal and a very clear output that I was looking for. 249 00:25:18.580 --> 00:25:33.120 And I didn't really care how we achieved that goal. It was unimportant to me. And that... was a surprise to me because normally I have strong opinions about how the software should be developed. 250 00:25:33.120 --> 00:25:35.940 And I actually care about that more than the outcome of the software. 251 00:25:35.940 --> 00:25:36.160 Yeah 252 00:25:36.160 --> 00:25:43.960 And I think we've talked about this before. I think there are some people for whom the, the goal and the result is the important aspect. 253 00:25:43.960 --> 00:26:01.130 And then really the difference between, you know, somebody who is, I don't know, a quant or a physicist or whatever, and a programmer is the programmer is like, yeah, I want to do, and a particle system, but it's going to be a really nice piece of code and I'm going to enjoy that process. And that's what floats my boat. 254 00:26:01.130 --> 00:26:01.860 Yeah. 255 00:26:01.860 --> 00:26:03.460 And so that's definitely 100% me. 256 00:26:03.460 --> 00:26:03.620 Right. 257 00:26:03.620 --> 00:26:05.790 But in this particular case, I just didn't care. 258 00:26:05.790 --> 00:26:05.860 Yeah. 259 00:26:05.860 --> 00:26:07.620 I'm like, do a thing, make it happen. 260 00:26:07.620 --> 00:26:19.850 you know and that was I've done it a couple of times. Another one was like, you know hey, I've got some YouTube videos. Can you just write a thing that tells me when there's a new... comment on them, please, because I don't own the video, so I can't subscribe to them myself and get notifications. 261 00:26:19.850 --> 00:26:19.960 Oh yeah. yeah 262 00:26:19.960 --> 00:26:20.000 Right. 263 00:26:20.000 --> 00:26:39.310 But if I run a Python script once a day, then you keep a .txt file, the ones you've seen before, and just tell me. And that was like, again, I don't care how that works, and it /seems/ to work. That use case covers probably a lot of companies that aren't technology-focused need for technology. 264 00:26:39.310 --> 00:26:39.860 Tight 265 00:26:39.860 --> 00:26:46.200 Your ah HR company that just says, I need a thing that goes through all the resumes and gives me a list of everybody's phone numbers or whatever. 266 00:26:46.200 --> 00:26:46.420 Yeah, yeah, yeah. 267 00:26:46.420 --> 00:26:47.500 Right. You know, those kinds of things. 268 00:26:47.500 --> 00:26:47.960 Right. 269 00:26:47.960 --> 00:26:55.680 And so those I can see, they don't care how maintainable it is, or they don't maybe not even care if it's reproducible at all. 270 00:26:55.680 --> 00:26:55.700 Right. Yes. 271 00:26:55.700 --> 00:27:01.160 It's just a one-off and they're done with it. Right. And that's, that's fine. 272 00:27:01.160 --> 00:27:01.720 Right. 273 00:27:01.720 --> 00:27:03.660 Write once read never software. Right. 274 00:27:03.660 --> 00:27:07.510 Right. And that's kind of, yeah, it's sort of anathema to everything that you and I stand for, and really. 275 00:27:07.510 --> 00:27:07.700 Yeah. 276 00:27:07.700 --> 00:27:09.660 But also it is pragmatic and it's useful and and it's... 277 00:27:09.660 --> 00:27:30.000 It's very pragmatic. And honestly, I've been doing some more of that myself lately. And I try to be very intentional about it. It's like, I am going to have Claude generate this Python script for me. And I am basically going to test it empirically. I'm going to run it. I'm going to see if it produces the result that I want. And if it does, I don't really care how it works. I might read it. I might not. 278 00:27:30.000 --> 00:27:30.490 Right. 279 00:27:30.490 --> 00:27:30.840 Yeah. 280 00:27:30.840 --> 00:27:41.920 And for certain tasks, that makes a ton of sense. It's incredibly fast. And I can get enough confidence that it does what I want it to do, =by running it. Right? 281 00:27:41.920 --> 00:27:50.810 But then there's a sort of trade-off where that one-off tool becomes more and more useful. You start relying on it, and then eventually you have to turn it into a like a proper tool. 282 00:27:50.810 --> 00:27:50.880 Yeah. 283 00:27:50.880 --> 00:27:57.300 And now you're like, oh, gosh, ah do I treat it as a prototype? Do I redo it myself? 284 00:27:57.300 --> 00:27:57.740 Yeah. 285 00:27:57.740 --> 00:28:03.460 It's interesting. But then, you know again, I think we've talked about this before, like I do a lot of open source software. 286 00:28:03.460 --> 00:28:04.000 Yeah. 287 00:28:04.000 --> 00:28:16.100 At least I have in the last you know few months. And to some extent, all of the code that I maintain is like that because it was written by other people that I don't have even hope of contacting anymore necessarily. 288 00:28:16.100 --> 00:28:30.860 So they might as well have been an agent that that fired up, built up its kind of idea, submitted a small change to the code and then disappeared never to be able to be contacted again because I have this weird language that somebody sent for Compiler Explorer and I haven't got a clue what it is. 289 00:28:30.860 --> 00:28:32.060 Yeah. Yeah. 290 00:28:32.060 --> 00:28:44.170 Now I have reviewed the code, but it's still a feeling of like, there is a sort of sense in which as team size grows, either open source or a closed source, you end up looking at code that you don't recognize before. 291 00:28:44.170 --> 00:28:44.240 Right. 292 00:28:44.240 --> 00:28:55.380 It's just whether or not it fits your style, whether you're you've got some overarching idea about how the the the whole should be put together, which we've we've maybe we talked about before. I think, we yeah, we have. 293 00:28:55.380 --> 00:29:06.300 I'm sure we've talked about the fact that like, Setting a project up well for success as a senior engineer with all the things that come with it can help everyone, including AIs, come in and onboard. 294 00:29:06.300 --> 00:29:06.620 Yeah, yeah. 295 00:29:06.620 --> 00:29:07.500 Yeah. 296 00:29:07.500 --> 00:29:25.500 Yeah. But yeah, I don't know. I just think that there's a thing that that is likely to happen that I think the very well could happen. And again, you know I'd kind of say you know capitalism is famously bad at solving these types of problems where it's like stuck in a local minima. 297 00:29:25.500 --> 00:29:32.000 It makes completely a ton of sense why you would be in that local minima, but there will come a day when you try to 298 00:29:32.000 --> 00:29:39.220 And, you know, like your shareholders will ask questions if you say, yeah, we just hired a bunch of people to do work that we can do with the computer. And why have you done that? 299 00:29:39.220 --> 00:29:42.210 Well, because we need to train them up to be the the next middle tier. 300 00:29:42.210 --> 00:29:42.540 Right, right. 301 00:29:42.540 --> 00:29:45.260 And that's a harder thing to to convince people to spend money on. 302 00:29:45.260 --> 00:29:45.500 Yes 303 00:29:45.500 --> 00:29:45.760 Right, right. 304 00:29:45.760 --> 00:29:46.180 Right. 305 00:29:46.180 --> 00:30:12.970 Like you might be able to have something in in much larger companies where you have a little bit, but certainly like medium and smallish companies, In particular, and really, like you say, you know, if you have that sort of like shareholder question of like, why are you why are you guys hiring all of these expensive, you know, MIT grads when you could just take your existing engineers and have them be just as productive, if not more by just you know giving them a hundred-dollar-a-month, you know, Claude license or whatever. 306 00:30:12.970 --> 00:30:13.640 Yeah. 307 00:30:13.640 --> 00:30:29.620 But then, I mean, again, taking the other side of that, again, you know, in in terms of R&D, companies have always, ah you know, forward thinking companies have always had like, hey, we've got a department of people who just sit around and tinker with things in the hope that we find something cool and new. 308 00:30:29.620 --> 00:30:29.800 Mm-hmm. Yeah. 309 00:30:29.800 --> 00:30:41.920 The trick with that, I suppose, now I think about it out loud, is that the results of that, whatever they are, positive or negative, are owned unambiguously by the company. Like, you're creating intellectual property for the company. 310 00:30:41.920 --> 00:30:46.170 Whereas when you're investing in the brain trust of new grads, 311 00:30:46.170 --> 00:30:46.430 Right. 312 00:30:46.430 --> 00:30:46.680 Mm-hmm. 313 00:30:46.680 --> 00:30:59.720 You are investing in that person, which is wonderful and thoughtful and helps the world, but does not specifically help you unless you can retain that person, which is not necessarily possible. 314 00:30:59.720 --> 00:31:00.220 Right, right. 315 00:31:00.220 --> 00:31:09.740 And so you are let you have kind of you're giving somebody a gift, as it were, that your competitor could could benefit from. 316 00:31:09.740 --> 00:31:09.740 Exactly. 317 00:31:09.740 --> 00:31:16.180 And then, yeah, back to your point about capitalism. 318 00:31:16.180 --> 00:31:22.820 Yeah, i I have had some discussions with ah people who may be in school in a computer science program, and I think we both know who they are. 319 00:31:22.820 --> 00:31:27.740 I, I've no idea. 320 00:31:27.740 --> 00:31:43.440 And one of the thoughts that I had in the course of those discussions is it is when I was graduating and certainly throughout my career, one of the most important things as a new grad, as ah as a junior person, 321 00:31:43.440 --> 00:31:51.190 Was that you demonstrate that you can learn quickly, right? Like that's always sort of the thing is like, we're not expecting you to be an expert in any of this. 322 00:31:51.190 --> 00:31:51.280 Yep. 323 00:31:51.280 --> 00:32:02.420 That would be silly. We just want to find people who can learn quickly. Right. And I think that's always going to be true to an extent. I mean, no one wants to hire people who learn slowly. 324 00:32:02.420 --> 00:32:19.020 Right? Like, so it's a little silly, but there may be a little bit of a shift in emphasis, I think, and that that is created by this this local minima where it's like, yeah, yeah, okay, you can learn fast. Everybody can learn fast. What can you /do/? 325 00:32:19.020 --> 00:32:19.800 Yeah. 326 00:32:19.800 --> 00:32:33.880 Because we're going to give you, ah you know, your cloud licenses and a development environment and we, cool if you can learn stuff, but what we want you to do is build this thing and it has to be right and it has to be good. 327 00:32:33.880 --> 00:32:55.800 And so ah i kind of wonder if the thing the thing to do is to say, you know if you are a person who's in school, if you're a person who's trying to get a job, and this was always kind of true, but I wonder if it's becoming more true, is the thing to do is demonstrate that you can get things done, that you can build stuff, right? 328 00:32:55.800 --> 00:33:00.420 The old Joel Spolsky test of, you know, "smart and gets things done", right? 329 00:33:00.420 --> 00:33:13.400 Yeah, that's. But yeah, and then but then there's still the the oh sort of sort of getting things done does not give you the opportunity to learn the skills that will allow you to get better necessarily. Maybe you just have to balance those things. 330 00:33:13.400 --> 00:33:13.620 Yeah. 331 00:33:13.620 --> 00:33:27.340 I mean, I certainly know that talking to some some junior folks that like some of them avoid using any kind of AI because they perceive that they won't learn from it, which is valid, absolutely valid. 332 00:33:27.340 --> 00:33:27.440 Yeah, yeah, right. 333 00:33:27.440 --> 00:33:40.000 I mean, there's nothing like doing things to learn things, but balancing that with the sort of the changing winds of our industry, that this is an enabling technology and finding the right balance. 334 00:33:40.000 --> 00:33:40.060 Yeah. 335 00:33:40.060 --> 00:33:40.120 Yeah. 336 00:33:40.120 --> 00:33:48.380 And I wish I had better advice for them when they come and say, i like, hey, no, I just won't use ChatGPT. I won't use Claude. And you're like, well, you know, everyone is. 337 00:33:48.380 --> 00:33:49.560 I think, yeah. 338 00:33:49.560 --> 00:33:56.060 I was, I was actually chatting with, um, a friend of mine who is a theoretical physicist at a university. 339 00:33:56.060 --> 00:33:56.720 i don't know, man. 340 00:33:56.720 --> 00:34:05.520 And he was, we were talking about AI usage in general. And he's like, I have to kind of essentially ban my students from using it, even though I use it myself daily. 341 00:34:05.520 --> 00:34:06.160 Yeah. Yeah. 342 00:34:06.160 --> 00:34:38.520 It's an incredibly invaluable tool for doing certain things. And, um, he and I actually mooted some ideas. so so I don't don't know if I've already said this before, and I know it rise, we're getting at time here. I have to be a bit thoughtful about not starting yet a new topic, despite the fact that we've just said, Hey, we should probably look at the end of each episode and see if there are ones we could continue. But no, um I wonder if there is a sweet spot for AI, which is a bit like NP-hard versus NP-complete or one way one of the things around those ways, I forget which one's which, right? [[EDITOR MATT HERE - NEITHER OF THEM ARE THIS, IT IS NOT LIKE THAT AT ALL!]] 343 00:34:38.520 --> 00:34:53.760 But my understanding, limited as it is about these things, is that there are some problems that are NP-hard, which means that there is not a polynomial way of solving them. And for a subset of those, there is a polynomial way of checking that the answer is right. 344 00:34:53.760 --> 00:35:07.580 So for example, if I said to you, find me a route through America, going through every single city that is less than 5,000 miles total, then it's NP, it you know, it's it's stupendously difficult to search that space. 345 00:35:07.580 --> 00:35:07.610 Yeah. 346 00:35:07.610 --> 00:35:07.790 Yeah, I think that's right. 347 00:35:07.790 --> 00:35:13.700 But once you found a solution, I can check it trivially just by following it along going, oh, it's less than 5,000. 348 00:35:13.700 --> 00:35:16.000 Right. Yeah. Prime factorization is another one of those, right? 349 00:35:16.000 --> 00:35:16.800 But if I then said, perfect example, 350 00:35:16.800 --> 00:35:19.840 It's like, yeah, you just multiply it all together and if you get the thing, then yeah that's right. Yeah. 351 00:35:19.840 --> 00:35:20.820 Exactly. Perfect. 352 00:35:20.820 --> 00:35:36.530 Exactly. But um there is another family of, I think this is the NP-complete, but again, I'm really - mathematicians who are listening, I'm so sorry if I'm wrong [[I MOST CERTAINLY AM]] - but one of them is not only is it NP in terms of the time complexity of solving it, but also to check the answer it is also. 353 00:35:36.530 --> 00:35:36.550 Mm-hmm. 354 00:35:36.550 --> 00:35:36.580 Mm-hmm. 355 00:35:36.580 --> 00:35:39.810 So like the classical traveling salesman problem, which is to find the shortest 356 00:35:39.810 --> 00:35:40.360 Mm-hmm. 357 00:35:40.360 --> 00:35:46.550 In order to show that it's the shortest, you have to essentially show that no other path is is long. So every other path is longer. 358 00:35:46.550 --> 00:35:47.200 Mm-hmm. Mm-hmm. 359 00:35:47.200 --> 00:35:57.700 And so where does the analogy break down with AI? AI, I think can be very useful for the kind of NP-hard things or whichever is the one that can be polynomial time checked. 360 00:35:57.700 --> 00:35:59.560 Yeah, easy to verify, right? 361 00:35:59.560 --> 00:36:00.640 That's where I'm going with this. 362 00:36:00.640 --> 00:36:12.140 Exactly right. Thank you for saying it much more succinctly than I. But yeah, so again, physicist friend is like, yeah, I can set it on certain things. And as long as it gives me the citations of papers, which I can click on afterwards and then read the ones that it's found. 363 00:36:12.140 --> 00:36:12.460 Yeah, yeah, yeah, yeah. 364 00:36:12.460 --> 00:36:12.540 Right. 365 00:36:12.540 --> 00:36:25.080 There's obviously an error of omission where it didn't read things and they might be important. But the ones it did find or the ones that it summarized in a particular way, I can read through myself and go, yes, this seems reasonable. And it was much quicker than me reading them all. 366 00:36:25.080 --> 00:36:40.220 But, but then there are the the other problems where you, you know, you just have to kind of go, I guess it's right. And I suppose to an extent, actually, if you're a non programmer and you asked AI to make you a Python program and it's generated 3000 lines of Python, you run it and it appears to work. 367 00:36:40.220 --> 00:36:40.640 Right 368 00:36:40.640 --> 00:36:47.720 I mean, mate you know, maybe that's, maybe that is the the first category because you can prove this right, but you still don't really know that it works on your system. 369 00:36:47.720 --> 00:37:05.680 i i would yeah i would I would argue that if you really generated 3000 line Python program, unless there are there's no conditional logic in it whatsoever, and it is just straight through 3000 lines executed the exact same way every time, there's no way that you can actually know that that's correct in all cases, right? 370 00:37:05.680 --> 00:37:05.980 Yeah. Yeah. 371 00:37:05.980 --> 00:37:14.260 But, you know, to your point earlier, you said, you know, you just ran it a few times empirically, it's okay. And and that's so kind of almost like the verification "seems okay to me". 372 00:37:14.260 --> 00:37:14.290 Yeah. 373 00:37:14.290 --> 00:37:14.340 Yeah. 374 00:37:14.340 --> 00:37:20.000 Yeah. And I mean, usually what I, be honest, what I'm usually doing there is looking through the conditionals being like, can I delete this? Is this, is this work? 375 00:37:20.000 --> 00:37:21.740 Oh, that's an interesting way of doing it. 376 00:37:21.740 --> 00:37:31.640 No, I mean, this is what I did with bash scripts forever. Like I would, I would have, I intentionally try to write bash scripts as I'm not, I never found a good unit testing framework for bash. I think we talked about this maybe in another episode. 377 00:37:31.640 --> 00:37:33.080 We have actually, yeah. That was one the early ones. 378 00:37:33.080 --> 00:37:52.740 Yeah. And so my de facto method for writing bash scripts up until the, you know, creation of Claude, which now I just use that, was take a shell script, make it executable, an empty file, i'll make it executable, put it in watch and have it run over and over and over again, and then try to solve the problem by writing as few conditionals as I possibly can, right? 379 00:37:52.740 --> 00:37:59.530 Just have it go straight through. There's only one code path through this and I can just tell empirically that it works because it prints out seven at the end, right? 380 00:37:59.530 --> 00:38:00.660 Right. And then keep editing. 381 00:38:00.660 --> 00:38:11.930 Yeah, and that was my method because it's like, okay, it's dumb, but it it like there's only one possible way to run this program and the one way works empirically. So that's good enough for now, right? 382 00:38:11.930 --> 00:38:12.240 Right. 383 00:38:12.240 --> 00:38:30.850 It's when you have like the, oh, it handles this case and handles that case and has a combination of these two cases. it is is that Like that's when you start introducing bugs without realizing it. And that's when you actually need to understand all of the possible paths through the code and, you know, hopefully structure them in a way where there's not like 2^n of them, right? 384 00:38:30.850 --> 00:38:31.940 Mm-hmm. Mm-hmm. 385 00:38:31.940 --> 00:38:54.100 Because otherwise you're just going to build unreliable software. Right? So I completely like, um I think that, you know, this, the sort of vibe model of coding, especially when you have sort non-technical people doing it, is really like a rising tide lifting all boats. It's making people much more productive. 386 00:38:54.100 --> 00:39:09.200 But if if we start to try to build complicated systems with this, I'm not really sure that that actually works. Because at the end of the day, the context window of the AI can only be so big and it can only remember so many things. 387 00:39:09.200 --> 00:39:10.100 Yes, and... 388 00:39:10.100 --> 00:39:13.820 And when somebody asks the question, why does this not work? There's not going to be anyone that can provide an answer. 389 00:39:13.820 --> 00:39:17.510 ..That is more to the point. But yeah, I think like the the context window thing is a whole other thing. 390 00:39:17.510 --> 00:39:17.660 Yeah. 391 00:39:17.660 --> 00:39:29.260 There are obviously technical reasons there, upping those and stuff. And then you know the fact that my own context window as a human is, getting as I'm getting older, much smaller. 392 00:39:29.260 --> 00:39:31.050 And you rely much on, right, yeah. 393 00:39:31.050 --> 00:39:33.500 COVID brain frog, right? Brain fog... 394 00:39:33.500 --> 00:39:36.440 Yeah, yeah. I, there could be a frog in there as well. That would explain a lot. 395 00:39:36.440 --> 00:39:39.800 Brain frog is a, that's a whole other symptom of COVID and trust me, you don't want that one. 396 00:39:39.800 --> 00:39:41.520 That would absolutely. 397 00:39:41.520 --> 00:39:50.420 Oh, man. Yeah. No, I think that that those those issues will start to be, that's where the innovations are happening, as far as I can tell, in the AI world now. 398 00:39:50.420 --> 00:39:50.960 Yeah. 399 00:39:50.960 --> 00:40:04.600 But um just one final thing as we're hitting towards the end of one of our longer episodes here. In, I think, 1980, Fred Brooks of Mythical Man Month fame wrote an essay called No Silver Bullet. 400 00:40:04.600 --> 00:40:05.840 Mm-hmm. Mm-hmm. Mm-hmm. 401 00:40:05.840 --> 00:40:18.280 And I've been ruminating that on a ah lot recently as I've gone through the arc of LLM sort of usage myself from, oh my gosh, this is mind blowing and I think it's going to change the universe to this is terrible. 402 00:40:18.280 --> 00:40:33.340 This is worse. I'm wasting my time on this, to: ah maybe, maybe not. To, now reasonably, it's another useful tool in my arsenal. Sometimes I can fire it on a couple of low-hanging bugs and I can then PR and I'm not sure if that's faster or not. I don't know yet. 403 00:40:33.340 --> 00:40:35.990 Sometimes it can help me by doing things that I'd never think of doing. 404 00:40:35.990 --> 00:40:36.280 Yeah. 405 00:40:36.280 --> 00:40:48.300 Other times it gets in my way. I don't know. It's a tool. It's like grep. It's like sed. It's a bit more involved. But to your point earlier, I think it's probably got the biggest chance of being a silver bullet out of all the things that we've seen so far. 406 00:40:48.300 --> 00:41:00.030 But you know, Fred's point was, everything has come and gone and nothing has ever fundamentally changed the problems of software engineering. That is, we need software engineers to do software engineering practices. 407 00:41:00.030 --> 00:41:00.380 Yeah 408 00:41:00.380 --> 00:41:07.080 And I think it's probably worth a re-read 409 00:41:07.080 --> 00:41:25.620 Yeah, there's always going to be somebody who is ultimately responsible for making sure the technology works. And that person is going to become a technologist whether they want to or not. Because they'll just there'll be situations where they're like, why doesn't this work? It's my job to figure it out. I guess I need to figure that out. 410 00:41:25.620 --> 00:41:40.060 Yeah. And whether they work with an AI to like kind of say, why is this not working? And have that sort of debug experience, pair programming experience, but they are still being brought along for the ride on the technology journey and there's no evading it. 411 00:41:40.060 --> 00:41:40.490 And that's just, there's no avoiding that. 412 00:41:40.490 --> 00:41:40.680 Sure. Yeah. 413 00:41:40.680 --> 00:41:42.760 And maybe that's hope for us all. 414 00:41:42.760 --> 00:41:42.880 Yeah. 415 00:41:42.880 --> 00:41:50.640 We'll see. All right. I think we have reached a good enough stopping point for now. 416 00:41:50.640 --> 00:41:50.960 Yeah, that works. 417 00:41:50.960 --> 00:41:54.660 And I will see you next time, my friend. 418 00:41:54.660 --> 00:41:57.660 Until next time.