Matt Godbolt: Good afternoon, Ben.
Ben Rady: Hello, Matt. How are you?
Matt Godbolt: Wow. Yeah, you've confused the heck out of half of our listeners now. Not only did I not say hi Ben, and I said good afternoon.
Matt Godbolt: You also asked me how I'm doing. I'm doing really great. Thank you very much. I am still recovering from a bout of COVID that I caught from seeing far too many people at a conference, which is unfortunately the occupational hazard of such a thing.
Ben Rady: Oh no.
Matt Godbolt: But I'm doing fine other than just a bit of a cough. How about you?
Ben Rady: I'm doing all right. I have no real complaints. I only have zeroth world complaints.
Matt Godbolt: Imaginary complaints that have no real component to it.
Ben Rady: Right. Yes. There's no real component to it. They're truly imaginary.
Matt Godbolt: Whereas if there's a mix, it's a complex complaint.
Ben Rady: They're 90 degrees out from any sort of real complaint.
Matt Godbolt: Is that right?
Matt Godbolt: So, yeah, you've completely derailed immediately, one minute in and already going, what was it we were going to talk about?
Ben Rady: Right. Right.
Matt Godbolt: And I'm going to blame the COVID for that because that's what you're allowed to do.
Matt Godbolt: Right. Something, something, brain fog.
Ben Rady: You've got fog brain. That's like a, yeah.
Matt Godbolt: Yeah, I feel like someone posted something on Twitter to the effect of something like,
Ben Rady: Uh-huh. Right.
Matt Godbolt: Cognitive hypochondria or something like that. And I'm like, maybe it is a bit of that. But, you know, even if it's psychosomatic, it seems really not. Anyway, that's not what we were going to talk about today.
Ben Rady: Right.
Matt Godbolt: We're going to talk about not my inability to think, but perhaps thinking or getting something else to think for us.
Ben Rady: Yes. Outsourcing your thinking to another thinky thing.
Matt Godbolt: Outsourcing it to a thing that can think or maybe can't think.
Ben Rady: Right. Yeah.
Matt Godbolt: Let's not get too caught up in the specifics, the semantics, but something which looks like it might think.
Ben Rady: Yeah.
Matt Godbolt: And that person may be an intern, right? As far as I can tell, they can think, but I've got no proof in my solipsistic worldview.
Ben Rady: Nope.
Matt Godbolt: Everyone else is a figment of my imagination. So that seems, no, no.
Ben Rady: Mm-hmm.
Matt Godbolt: But realistically, no. You had an idea before we pressed the record button.
Ben Rady: I did. So I think there's sort of an interesting phenomenon going on with, you know, the development of software engineers, not the development of software, but the engineers that make them.
Ben Rady: And AI and sort of learning things and the way that people think about, you know, costs. I don't know. There's a whole bunch of interrelated things that I think are going on right now.
Ben Rady: And it might make for a good podcast.
Matt Godbolt: Well, we can certainly sort of explore it, you and I without really any plan as per usual.
Ben Rady: Yeah, yeah.
Ben Rady: Yeah, as per usual. So I think the way... First, I'm going to back up a level and be like, what is programming? Just to set some really basic foundational layers for this conversation we're about to have.
Matt Godbolt: Okay, that seems reasonable, yeah. So are we talking about software development, like the whole process of it, or programming very specifically, or...
Ben Rady: Software development, really, because there's a dimension.
Matt Godbolt: Okay, okay. So yeah, tell me what you...
Ben Rady: There's an aspect of this that I think is important for this conversation, which is a lot of what software development is, in my opinion at least, is taking the sort of messy, inconsistent, poorly specified real world, not the imaginary world, the real world and mapping it into,
Matt Godbolt: Okay.
Ben Rady: the completely uncompromising world of computers where everything has to be specified exactly. There is no room for vagaries.
Ben Rady: And if you are unaware of the vagaries that you're creating, the computer will create them for you much to your dismay.
Matt Godbolt: Yes.
Matt Godbolt: Very, very apt.
Ben Rady: You know, let's talk about undefined behavior, right?
Matt Godbolt: Right. Exactly that. Yeah.
Ben Rady: So when a thing is engaging this activity of mapping the real world to the sort of digital world, inevitably, one of the things that you have to be able to do is report back to whatever it is that is telling you to, like, hey, map this thing into a computer for me, please.
Ben Rady: And be like, actually the thing that you asked me to do is physically impossible. I can't actually do that because it is impossible to do that.
Matt Godbolt: Right. Mm-hmm.
Ben Rady: Or logically impossible. And so that's sort of like a feedback loop that is essential to the act of programming. I think it's naive to think that anyone really, even an experienced software engineer, can just say, go build a thing, and then the thing appears 100% of the time, right?
Ben Rady: Because there is a very good chance that unless you have actually done the work of taking your ideas and mapping them into the digital world, there's a very good chance that they don't actually map and that compromises will need to be made.
Ben Rady: Questions will need to be answered. Things will need to be clarified. and
Matt Godbolt: Well, if I can, if I can interrupt, even with a well specified problem,
Ben Rady: you know, Mm-hmm.
Matt Godbolt: where you say solve X, I need something that will do X. There is an almost infinite number of valid solutions to that problem.
Matt Godbolt: And some of them will be more maintainable than the others. Some of them, which may or may not be a goal. Some of them may be more performant than others, which may or may not be a goal. So many dimensions feed into what is a good X,
Matt Godbolt: that aren't often specified up front.
Matt Godbolt: And you know if you work for a high-frequency trading company, then probably it's a given that when you're writing something to go in the trading system, it should be as fast as possible and do no extra work.
Ben Rady: Right.
Ben Rady: Yeah.
Matt Godbolt: But if you're writing a framework for wider consumption that's, you know, for anyone to come and use, then you probably want to be as generable as general...
Matt Godbolt: as general and a generally...
Ben Rady: Generable.
Matt Godbolt: as general as possible without tying yourself in knots. And so there's a huge amount of unspoken requirements that also come.
Ben Rady: Yeah. Yeah.
Ben Rady: Yeah.
Matt Godbolt: And as an experienced software developer, you like to think you can make an intelligent guess about it, but also you can be so wrong so often on that, and we are.
Ben Rady: Right.
Matt Godbolt: So that's not even taking into account the fact that what you are asked may not, as you say, *be* possible because it involves solving the halting problem or achieving something that is unattainable.
Ben Rady: Right.
Ben Rady: Right, right. Going faster than the speed of light or something like that.
Matt Godbolt: Things like this.
Ben Rady: Yeah, right.
Matt Godbolt: Right, yeah. So, yeah, there is a back and forth process where you say, maybe even internally, do I do X? Given two equally possible solutions, should I do X or Y?
Matt Godbolt: What are the trade-offs between them and all that stuff? And at that point, you may turn around and go back to your stakeholder and say, hey, do you want it fast or do you want it general or whatever?
Ben Rady: Right.
Matt Godbolt: Even that may not be the right axis, but something like that.
Ben Rady: Yeah, right. But there's a trade-off to be made and it's not clear what the right one is, right
Matt Godbolt: Yeah.
Matt Godbolt: And there are, well, there's, yeah, there are many, many, many trade-offs to be made.
Ben Rady: Yeah. Yeah.
Matt Godbolt: Yeah.
Ben Rady: Right. So the question is, who or, and it's my belief by the way. So this is a problem, this doing this mapping from the physical world into the digital world is a thing that people have been trying to solve for a very long time.
Ben Rady: When I was in school, the big thing was like UML.
Matt Godbolt: Are you going to say UML?
Matt Godbolt: Yeah.
Ben Rady: Yes, exactly.
Ben Rady: Right. You know, you know what I'm going to say, you know, the deal.
Matt Godbolt: Yeah, I'm a bit older than you, but it was still around after me.
Ben Rady: Yes. Right. UML, Rational Rose, all of these tools that were like, you know, we're going to have all these things that let you solve this problem.
Matt Godbolt: Oh my golly, there's a word I haven't heard.
Ben Rady: And they didn't work, of course. But um it is my current...
Matt Godbolt: And, know, things like 4GLs and things like that came along and said, hey, you know, you'll never need to write another line of code.
Ben Rady: Yes, yes.
Matt Godbolt: You just draw a picture and it'll all just work.
Ben Rady: Right, right. So I have come to believe over the sort of decades of this that the least specific definition of what um you need to create in order to actually solve a problem on a computer is the source code, right?
Ben Rady: Anything more abstract than that is not going to work because of this mapping problem, because you're going to have things that are unspecified, whether you know they're unspecified or not, you're going to have things that are unspecified that need to be specified in order for it to work.
Matt Godbolt: Yep.
Ben Rady: And so human beings have kind of invented programming languages, this incredible, fairly objective, like the most objective thing that we've maybe ever made beside maybe mathematics,
Ben Rady: of like how you actually solve a problem and that you cannot be any, you can't be any less abstract than that and actually solve the problem. Right? So fast forward to today and we have a new attempt, which is probably, which is turning out to be a much better attempt to solve this mapping problem, which is,
Matt Godbolt: Specifically LLM coding assistant type stuff that people get.
Ben Rady: Specifically, LLMs and maybe more specifically agentic coding, right? Although, you know, sort of depends.
Matt Godbolt: Yeah, fair. Yeah. There are other things.
Ben Rady: Yeah. Like it is conceivable, and I know that you have actually done this, to completely offload this mapping problem to an agent and have it solve a problem in a way where it creates source code, but you have not even read it, let alone understand it, right?
Matt Godbolt: Correct.
Ben Rady: And that is not, and I'm going to reference what you were kind of saying a little bit before the podcast here. That is not completely different from what we've been doing for decades where a manager says, hey, go build this thing.
Ben Rady: And the developer goes and writes some code and then comes back to the manager and the manager doesn't read the code at all. Sends pull requests, whatever, approve. And then they try it and they're like, yeah, that doesn't work. Do the thing again.
Ben Rady: and then it goes and tries again. And then we just, you know, it's like we're doing the exact same thing. You know, what what what was the phrase that you used for this?
Matt Godbolt: Oh, I need to find it out now. this is I'm going to have to say thank you to ah my friend Jeremy, our mutual friend Jeremy, in fact.
Ben Rady: Yeah.
Matt Godbolt: It says, managers have been vibe coding forever.
Ben Rady: Right. Yes, exactly.
Matt Godbolt: And it's the loop of like what the vibe coding looks like. And at the end, the manager says, good job, but be faster the next time.
Matt Godbolt: Or worse.
Ben Rady: Right.
Ben Rady: So I think there was a perception, I don't know, a year-ish ago, maybe a couple of years ago when these tools started coming out. And I know you and I maybe even had some conversations about this of like, you know, is software engineering as a profession going to go away because of this?
Ben Rady: Is this going to just be, you know, non-technical people or maybe just like junior people who are right out of school who are not managers yet using these tools to build software?
Matt Godbolt: Right.
Ben Rady: And if you especially if you look at some of like the job numbers in like the the, you know, numbers of openings and people were are looking for jobs. It definitely seems like what has happened in the last year or two is that it's the junior engineers who are not getting hired because what people are doing is they're just putting these tools in the hands of very senior software engineers who are now able to get way more leverage than they may be used to, or at least the same amount of leverage, but with many fewer people, right?
Matt Godbolt: Yeah. Yeah.
Ben Rady: Like you don't have the thing of like, okay, we're going to give you two
Ben Rady: or three you know less skilled or more junior engineer.
Matt Godbolt: Junior.
Matt Godbolt: Yeah. Whatever.
Ben Rady: Yeah.
Matt Godbolt: Yeah.
Ben Rady: um And then you're go you you have this team and you just do this thing of like, you sort of farm the work out to them and you do basically this like vibe manager coding thing, right?
Matt Godbolt: Yeah. Yeah.
Ben Rady: And it's like, okay, yeah, don't do that. Just have like two or three different Claude agents running on your computer at the same time. And there's your team essentially, and you just do that, right?
Matt Godbolt: Yeah.
Ben Rady: And so that makes sense. And that's like a very rational thing for businesses to do, right?
Matt Godbolt: But just so of like to go back to your point, I think you know the point we were making earlier on was that the senior developers would be the ones that would be being cut because they're expensive and their experience and skill level could be replaced by a sufficiently intelligent
Ben Rady: Right. Yeah.
Matt Godbolt: AI system.
Ben Rady: Yeah. Yeah.
Matt Godbolt: And then it just takes guidance from either a junior dev or no dev at all, because, you know, you just have the manager, the non-technical stakeholder just talking into a ah window saying, no, no, make it bluer.
Ben Rady: Right.
Matt Godbolt: No, no, no. Now make it do this when you click that kind of thing or more, more generally.
Ben Rady: Yeah.
Matt Godbolt: But the surprise here is at least fast forward a year, that does not seem to be the case. In fact, the opposite. It's more the senior folks like you and I who have gone, hey, this seems valuable and we can perhaps get some use out of it as a tool.
Ben Rady: Right.
Matt Godbolt: But the junior folks have not been ah able to get the same amount of leverage as we do. But their role is being potentially usurped by the fact that, yeah, you don't need a team of people anymore. You need one guy and and and the patience of Job to talk to.
Ben Rady: Yeah.
Matt Godbolt: So many forgetful, stupid, ah you know, ah ADHD robots. But nonetheless, that, and obviously I don't think we're there yet.
Ben Rady: Right. Right.
Ben Rady: Right. right
Matt Godbolt: You know, you and I both work in companies where there are still are junior folks around, but that was the interesting point.
Ben Rady: Right.
Ben Rady: That seems to be what is kind of happening as as as best that I can tell. And, you know, I'm not far from like an economic expert here.
Matt Godbolt: Yeah, we,
Ben Rady: i just see what I see.
Matt Godbolt: Well, and also we only really have yeah our own vantage point from where we are and we're in a particularly niche industry perhaps as well, um you know where technical um and so and and ah technical knowledge and a specific ah domain-specific knowledge is quite kind of important to what we do and maybe in a more general world where
Ben Rady: Right. Also very true.
Matt Godbolt: AI has been trained on more publicly available information. I mean, that may be they're easier to replace folks with. I mean, certainly in my limited experience of the last few months where I've been less employed, um fewer fewer employed, fewer employed,
Ben Rady: If you were employed.
Matt Godbolt: I've been using AI to help me on my open source projects, and that's all in TypeScript and JavaScript and web. And there's like thousands of websites that have good examples that presumably it's been trained on.
Ben Rady: Right. Yeah.
Matt Godbolt: When I go to conferences and talk to people, they have more equivocal results with more technical things like the minutia of C++, especially some of the new things that are coming down the pipeline because there exists no training data.
Ben Rady: Yeah. Yeah.
Matt Godbolt: And that's also true of me, right? You know, someone to come to me and say, can you write me a co-routine?
Ben Rady: yeah
Matt Godbolt: I'd be like, give me a few months to go and learn how to do that first, and then I'll be able to. But maybe our particular industry and the intersection of the technology we use and the special things we use makes it more likely that we will still hire and use people.
Matt Godbolt: I don't know. I'm just spitballing here, as indeed this entire podcast is.
Ben Rady: No. Yeah. I mean, what, that's the whole point of the podcast, but yeah, that's a very, very important point is that it's not like we're talking, it's not like I've analyzed a bunch of economic data to come to this conclusion, right?
Matt Godbolt: That is.
Ben Rady: Like this is all mostly from just my personal experience and the things that I've seen in the industry. So, you know, take this with however many grains of salt you believe are appropriate, listener.
Matt Godbolt: Right.
Ben Rady: But the kind of result of this now is you have, senior people. And I think the explanation is, is that sort of makes sense to me is that it's like LLM still require a decent amount of babysitting and you need to be able to know when they're hallucinating. You need to be able to know when they're going off in a weird direction to correct them. Otherwise they're like, it's the blind leading the blind.
Ben Rady: And that sort of explains why you get the most leverage from from having one experienced software engineer, you know, using all these various tools to build the things that you need, right?
Matt Godbolt: And yeah, I mean, I was going to say, i just interject there, I think that knowing about the kind of hallucinations, or I don't know, even if that's such a loaded term these days, but you know, like when, when mistakes are made and assumptions are made about, oh, there must be something that does x and then it says, "Hey, call x", and you know, that's never existed.
Ben Rady: Yeah.
Matt Godbolt: it would be great if it did.
Ben Rady: Right, yeah.
Matt Godbolt: But like identifying that and keeping it on track, a kind of slightly different techniques or different skills than we've had to use before, because perhaps as an experienced programmer who has led and mentored junior developers,
Matt Godbolt: You know, there is definitely a set of mistakes that I have seen folks fall foul to that sometimes I let them and, you know, you let them and then you guide them through the process of saying, why do you think that didn't work? And, you know, you use it as a teaching point, which is a valuable thing, I think, for everybody to learn from.
Ben Rady: Mm-hmm. Mm-hmm.
Matt Godbolt: But those are very human mistakes. And also the process of doing it that way allows them to grow. And LLMs will make those kinds of mistakes as well. And so I think that if you have led a team before that being able to control a suite of LLMs, some of those skills do overlap and you will go, yeah, yeah, please don't do this.
Matt Godbolt: Whatever you do, don't comment out the test to make everything green or don't check in without running things, which you see junior folks do sometimes, you know, like, hey, that's there for a reason.
Ben Rady: Yeah, right. Yeah.
Matt Godbolt: Don't just disable it because it didn't make sense to you. You know, always ask me, all that kind of stuff. But the thing that's frustrating is that the LLM will not learn in the same way as your junior devs. You know, the third or fourth time you said, please, look, if I have to do this again, we're going to have to have a talk, serious talk about this.
Ben Rady: Right.
Matt Godbolt: Then, you know, the LLM is like, nope, you've told me a hundred times before and I forget every single time.
Matt Godbolt: I mean, like a happy puppy, but...
Ben Rady: Right, right.
Ben Rady: Yep.
Matt Godbolt: So I think there are some new skills.
Ben Rady: I don't have that context in my window, right? Yeah, exactly.
Matt Godbolt: Yeah, right. Yeah. There are definitely some new skills for for leading LLMs. And I think it's like probably a 75% overlap with the kind of skills that help you with a team of real humans.
Ben Rady: Yeah.
Matt Godbolt: But there's definitely, there's a 25% that's new for LLMs. And there's also probably, I mean, I'm probably underselling it. There's a ton of much more important, you know, interpersonal skills.
Matt Godbolt: And didactic techniques that you can use when you're trying to help an individual learn.
Ben Rady: Yeah.
Ben Rady: Mm-hmm.
Matt Godbolt: And I've forgotten where I was going with this, but I interjected nonetheless. You were in the middle of something and I put my hand up and you...
Ben Rady: Oh, well, no, I mean, it it's just that, you know, the the phenomena now of having these sort of senior engineers leveraged using these tools is is great from an economic standpoint. Like it makes a lot of sense.
Matt Godbolt: Right.
Ben Rady: But the thing that I wonder about is if we are, you know, as an industry, setting ourselves up for future pain.
Ben Rady: Because if you do this for too long and the senior engineers are too successful. So the problem with companies in general is that if you have very talented people and you don't pay them enough, they'll leave.
Ben Rady: And if you pay them too much, they'll leave because they're rich and they don't need to work anymore. Right. So, you have to sort of, you know, strike this middle ground of not paying them too much so they quit because they don't need to work.
Matt Godbolt: So you're saying the only reason they don't pay me more money is because I might quit.
Ben Rady: Uh, well, let me ask you this. I mean, if you had, you know, hundreds of millions of dollars in the bank, would you continue to work?
Matt Godbolt: Probably, but not, yeah, not for... yeah, yeah, yeah, I think I'd never stop doing, I mean, we have the same, I'd never stop doing this stuff, but yeah, but no, yeah, all right, your point stands, Ben, your point stands.
Ben Rady: Maybe for yourself. Right. Right. Like I, yeah, I'm never going to stop programming because it's fun, right? But would I be?
Ben Rady: Yeah. So this is my point, is you got to pay people the right amount. So eventually, though, they are going to make enough money or they're just going to get to the point where they want to retire and they're going to retire. And the question is, I've just been teaching robots for the last 10 years.
Ben Rady: Who's going to take over my job? Who's going to start being the person commanding the robots?
Matt Godbolt: Right. Right.
Matt Godbolt: Right
Ben Rady: And this is a problem that capitalism is exceptionally bad at solving.
Ben Rady: Right. Of these sort of like long term systemic problems. And so I think what may happen is companies will start to discover that as the junior engineers who are currently looking for jobs sort of diffuse into other industries because they don't find any of the the opportunities that are there, that they're going to start to have a really hard time finding the people who can lead the robots.
Ben Rady: And what it what it might be is kind of like this pinch point of like, okay, well, we we got all these productivity gains from from structuring things this way. And it made complete sense for this quarter and the next quarter.
Ben Rady: But 10 years on, now we can't find anybody that understands any of this or how to make any of it work because they don't work in this industry anymore. Or those people maybe have just gotten very expensive. Right.
Ben Rady: Because there aren't that many of them that sort of survived the filter. Right.
Matt Godbolt: Right.
Ben Rady: Yeah.
Matt Godbolt: They will be the COBOL programmers of their of their day.
Ben Rady: Yeah. Yeah. Like it'll, just be a much harder, like an even harder thing than it was, you know, 2021 when it was like there were, ridiculous ah salaries being handed out because it's like you couldn't find enough programmers, right?
Ben Rady: um On the flip side of this, just thinking of it from like the the sort of trader perspective, it probably also means that in the next five to 10 years, there are going to be some really really, really good quote unquote junior programmers out there that could run circles around anyone else in your organization with 10 years of experience or 20 years of experience
Matt Godbolt: Yeah
Ben Rady: And if you can find them, they will be worth their weight in gold.
Matt Godbolt: That's an interesting observation to turn it that way around. Yeah, that's it.
Matt Godbolt: Yeah. No, it's interesting you say that this is a problem with capitalism. This is not a politics podcast, obviously, so i won't we won't go too much into the other choices out there.
Ben Rady: Yeah.
Ben Rady: Right.
Matt Godbolt: But no, I see it as almost like it's ah it's a local minima.
Matt Godbolt: We're heading towards a local minima, and nobody is going to do the sort of global optimization problem of like, well, now we just have to do the simulated annealing step of just throwing some randomness into it just to see, because we just need to get out of this this potentially local minima.
Ben Rady: Yeah.
Ben Rady: Right, right.
Matt Godbolt: But to sort of counter that, obviously, the the more... AI um ah proponent or a but I don't even know what the right term would be, but someone who is more confident in the role of AI will just say, well, AI will get better. And actually even the senior engineers,
Matt Godbolt: Won't have to be around anymore. We can just, any anyone can ask that question. And I think that obviously that that gets to the heart of the original thing that you brought up, which is that there is a ton of unknown unknowns that something or someone has to say, either make a call on explicitly and say, look, based on my years of experience, this is a database thing we should probably do use blah, versus this is a trading application, we should probably do something different.
Ben Rady: Yeah.
Matt Godbolt: Or you have to ask the the the person who's asking the question, should we optimize for so for throughput or latency? And at which point they're going to say, "the what now"?
Matt Godbolt: And then, so yeah, but where would that line be? And yeah,
Ben Rady: Yeah.
Ben Rady: Right. I mean, the the the premise here is that the although AI is great, it still hasn't actually solved the mapping problem, where the mapping problem is map the physical world into the digital world in a reliable, sustainable, continuous way.
Ben Rady: Obviously, you can have somebody sit down that knows very little about programming and interact with Claude and build like a simple app. Right? Like those things are absolutely possible. Right?
Matt Godbolt: Yeah.
Ben Rady: It remains to be seen whether you can build more complicated things that way. It remains to be seen whether you can change things that have been built that way and revise things that have been built that way and grow and scale things that have been built that way.
Ben Rady: It's possible that maybe one day, know, Claude version 75 will be so good at all of this that it really it really has like sort of fundamentally solved the mapping problem where you can have a completely non-technical person, somebody who does not really understand that computers work at all, sit down and build.
Ben Rady: A horizontally scalable web app or a trading system or you know a microcontroller for something that's going to go in the space shuttle. And all of that stuff will just work.
Ben Rady: It's entirely possible. But i I could also see a world where where that problem is so difficult that it doesn't actually get solved.
Matt Godbolt: It's possible, but yeah, it's hard to see.
Matt Godbolt: Well, I mean, that problem is so difficult that it often doesn't get solved even with experienced programmers and experienced, you know, this is a difficult thing to get to get right.
Ben Rady: Right.
Matt Godbolt: But there is, i mean, so you mentioned earlier about the fact that I had had some limited experience of like this whole closed loop vibe-codey type thing where I haven't even looked at the the code of something.
Matt Godbolt: And that is that is true. um But uniquely in the situation I was in, I had a very clear goal and a very clear output that I was looking for.
Matt Godbolt: And I didn't really care how we achieved that goal. It was unimportant to me. And that... was a surprise to me because normally I have strong opinions about how the software should be developed.
Matt Godbolt: And I actually care about that more than the outcome of the software.
Ben Rady: Yeah
Matt Godbolt: And I think we've talked about this before. I think there are some people for whom the, the goal and the result is the important aspect.
Matt Godbolt: And then really the difference between, you know, somebody who is, I don't know, a quant or a physicist or whatever, and a programmer is the programmer is like, yeah, I want to do, and a particle system, but it's going to be a really nice piece of code and I'm going to enjoy that process. And that's what floats my boat.
Ben Rady: Yeah.
Matt Godbolt: And so that's definitely 100% me.
Ben Rady: Right.
Matt Godbolt: But in this particular case, I just didn't care.
Ben Rady: Yeah.
Matt Godbolt: I'm like, do a thing, make it happen.
Matt Godbolt: you know and that was I've done it a couple of times. Another one was like, you know hey, I've got some YouTube videos. Can you just write a thing that tells me when there's a new... comment on them, please, because I don't own the video, so I can't subscribe to them myself and get notifications.
Ben Rady: Oh yeah. yeah
Ben Rady: Right.
Matt Godbolt: But if I run a Python script once a day, then you keep a .txt file, the ones you've seen before, and just tell me. And that was like, again, I don't care how that works, and it /seems/ to work. That use case covers probably a lot of companies that aren't technology-focused need for technology.
Ben Rady: Tight
Matt Godbolt: Your ah HR company that just says, I need a thing that goes through all the resumes and gives me a list of everybody's phone numbers or whatever.
Ben Rady: Yeah, yeah, yeah.
Matt Godbolt: Right. You know, those kinds of things.
Ben Rady: Right.
Matt Godbolt: And so those I can see, they don't care how maintainable it is, or they don't maybe not even care if it's reproducible at all.
Ben Rady: Right. Yes.
Matt Godbolt: It's just a one-off and they're done with it. Right. And that's, that's fine.
Ben Rady: Right.
Ben Rady: Write once read never software. Right.
Matt Godbolt: Right. And that's kind of, yeah, it's sort of anathema to everything that you and I stand for, and really.
Ben Rady: Yeah.
Matt Godbolt: But also it is pragmatic and it's useful and and it's...
Ben Rady: It's very pragmatic. And honestly, I've been doing some more of that myself lately. And I try to be very intentional about it. It's like, I am going to have Claude generate this Python script for me. And I am basically going to test it empirically. I'm going to run it. I'm going to see if it produces the result that I want. And if it does, I don't really care how it works. I might read it. I might not.
Ben Rady: Right.
Matt Godbolt: Yeah.
Ben Rady: And for certain tasks, that makes a ton of sense. It's incredibly fast. And I can get enough confidence that it does what I want it to do, =by running it. Right?
Matt Godbolt: But then there's a sort of trade-off where that one-off tool becomes more and more useful. You start relying on it, and then eventually you have to turn it into a like a proper tool.
Ben Rady: Yeah.
Matt Godbolt: And now you're like, oh, gosh, ah do I treat it as a prototype? Do I redo it myself?
Ben Rady: Yeah.
Matt Godbolt: It's interesting. But then, you know again, I think we've talked about this before, like I do a lot of open source software.
Ben Rady: Yeah.
Matt Godbolt: At least I have in the last you know few months. And to some extent, all of the code that I maintain is like that because it was written by other people that I don't have even hope of contacting anymore necessarily.
Matt Godbolt: So they might as well have been an agent that that fired up, built up its kind of idea, submitted a small change to the code and then disappeared never to be able to be contacted again because I have this weird language that somebody sent for Compiler Explorer and I haven't got a clue what it is.
Ben Rady: Yeah. Yeah.
Matt Godbolt: Now I have reviewed the code, but it's still a feeling of like, there is a sort of sense in which as team size grows, either open source or a closed source, you end up looking at code that you don't recognize before.
Ben Rady: Right.
Matt Godbolt: It's just whether or not it fits your style, whether you're you've got some overarching idea about how the the the whole should be put together, which we've we've maybe we talked about before. I think, we yeah, we have.
Matt Godbolt: I'm sure we've talked about the fact that like, Setting a project up well for success as a senior engineer with all the things that come with it can help everyone, including AIs, come in and onboard.
Ben Rady: Yeah, yeah.
Matt Godbolt: Yeah.
Ben Rady: Yeah. But yeah, I don't know. I just think that there's a thing that that is likely to happen that I think the very well could happen. And again, you know I'd kind of say you know capitalism is famously bad at solving these types of problems where it's like stuck in a local minima.
Ben Rady: It makes completely a ton of sense why you would be in that local minima, but there will come a day when you try to
Matt Godbolt: And, you know, like your shareholders will ask questions if you say, yeah, we just hired a bunch of people to do work that we can do with the computer. And why have you done that?
Matt Godbolt: Well, because we need to train them up to be the the next middle tier.
Ben Rady: Right, right.
Matt Godbolt: And that's a harder thing to to convince people to spend money on.
Ben Rady: Yes
Ben Rady: Right, right.
Matt Godbolt: Right.
Ben Rady: Like you might be able to have something in in much larger companies where you have a little bit, but certainly like medium and smallish companies, In particular, and really, like you say, you know, if you have that sort of like shareholder question of like, why are you why are you guys hiring all of these expensive, you know, MIT grads when you could just take your existing engineers and have them be just as productive, if not more by just you know giving them a hundred-dollar-a-month, you know, Claude license or whatever.
Ben Rady: Yeah.
Matt Godbolt: But then, I mean, again, taking the other side of that, again, you know, in in terms of R&D, companies have always, ah you know, forward thinking companies have always had like, hey, we've got a department of people who just sit around and tinker with things in the hope that we find something cool and new.
Ben Rady: Mm-hmm. Yeah.
Matt Godbolt: The trick with that, I suppose, now I think about it out loud, is that the results of that, whatever they are, positive or negative, are owned unambiguously by the company. Like, you're creating intellectual property for the company.
Matt Godbolt: Whereas when you're investing in the brain trust of new grads,
Ben Rady: Right.
Ben Rady: Mm-hmm.
Matt Godbolt: You are investing in that person, which is wonderful and thoughtful and helps the world, but does not specifically help you unless you can retain that person, which is not necessarily possible.
Ben Rady: Right, right.
Matt Godbolt: And so you are let you have kind of you're giving somebody a gift, as it were, that your competitor could could benefit from.
Ben Rady: Exactly.
Matt Godbolt: And then, yeah, back to your point about capitalism.
Ben Rady: Yeah, i I have had some discussions with ah people who may be in school in a computer science program, and I think we both know who they are.
Matt Godbolt: I, I've no idea.
Ben Rady: And one of the thoughts that I had in the course of those discussions is it is when I was graduating and certainly throughout my career, one of the most important things as a new grad, as ah as a junior person,
Ben Rady: Was that you demonstrate that you can learn quickly, right? Like that's always sort of the thing is like, we're not expecting you to be an expert in any of this.
Matt Godbolt: Yep.
Ben Rady: That would be silly. We just want to find people who can learn quickly. Right. And I think that's always going to be true to an extent. I mean, no one wants to hire people who learn slowly.
Ben Rady: Right? Like, so it's a little silly, but there may be a little bit of a shift in emphasis, I think, and that that is created by this this local minima where it's like, yeah, yeah, okay, you can learn fast. Everybody can learn fast. What can you /do/?
Matt Godbolt: Yeah.
Ben Rady: Because we're going to give you, ah you know, your cloud licenses and a development environment and we, cool if you can learn stuff, but what we want you to do is build this thing and it has to be right and it has to be good.
Ben Rady: And so ah i kind of wonder if the thing the thing to do is to say, you know if you are a person who's in school, if you're a person who's trying to get a job, and this was always kind of true, but I wonder if it's becoming more true, is the thing to do is demonstrate that you can get things done, that you can build stuff, right?
Ben Rady: The old Joel Spolsky test of, you know, "smart and gets things done", right?
Matt Godbolt: Yeah, that's. But yeah, and then but then there's still the the oh sort of sort of getting things done does not give you the opportunity to learn the skills that will allow you to get better necessarily. Maybe you just have to balance those things.
Ben Rady: Yeah.
Matt Godbolt: I mean, I certainly know that talking to some some junior folks that like some of them avoid using any kind of AI because they perceive that they won't learn from it, which is valid, absolutely valid.
Ben Rady: Yeah, yeah, right.
Matt Godbolt: I mean, there's nothing like doing things to learn things, but balancing that with the sort of the changing winds of our industry, that this is an enabling technology and finding the right balance.
Ben Rady: Yeah.
Matt Godbolt: And I wish I had better advice for them when they come and say, i like, hey, no, I just won't use ChatGPT. I won't use Claude. And you're like, well, you know, everyone is.
Ben Rady: I think, yeah.
Matt Godbolt: I was, I was actually chatting with, um, a friend of mine who is a theoretical physicist at a university.
Ben Rady: i don't know, man.
Matt Godbolt: And he was, we were talking about AI usage in general. And he's like, I have to kind of essentially ban my students from using it, even though I use it myself daily.
Ben Rady: Yeah. Yeah.
Matt Godbolt: It's an incredibly invaluable tool for doing certain things. And, um, he and I actually mooted some ideas. so so I don't don't know if I've already said this before, and I know it rise, we're getting at time here. I have to be a bit thoughtful about not starting yet a new topic, despite the fact that we've just said, Hey, we should probably look at the end of each episode and see if there are ones we could continue. But no, um I wonder if there is a sweet spot for AI, which is a bit like NP-hard versus NP-complete or one way one of the things around those ways, I forget which one's which, right? [[EDITOR MATT HERE - NEITHER OF THEM ARE THIS, IT IS NOT LIKE THAT AT ALL!]]
Matt Godbolt: But my understanding, limited as it is about these things, is that there are some problems that are NP-hard, which means that there is not a polynomial way of solving them. And for a subset of those, there is a polynomial way of checking that the answer is right.
Matt Godbolt: So for example, if I said to you, find me a route through America, going through every single city that is less than 5,000 miles total, then it's NP, it you know, it's it's stupendously difficult to search that space.
Ben Rady: Yeah.
Ben Rady: Yeah, I think that's right.
Matt Godbolt: But once you found a solution, I can check it trivially just by following it along going, oh, it's less than 5,000.
Ben Rady: Right. Yeah. Prime factorization is another one of those, right?
Matt Godbolt: But if I then said, perfect example,
Ben Rady: It's like, yeah, you just multiply it all together and if you get the thing, then yeah that's right. Yeah.
Matt Godbolt: Exactly. Perfect.
Matt Godbolt: Exactly. But um there is another family of, I think this is the NP-complete, but again, I'm really - mathematicians who are listening, I'm so sorry if I'm wrong [[I MOST CERTAINLY AM]] - but one of them is not only is it NP in terms of the time complexity of solving it, but also to check the answer it is also.
Ben Rady: Mm-hmm.
Matt Godbolt: So like the classical traveling salesman problem, which is to find the shortest
Ben Rady: Mm-hmm.
Matt Godbolt: In order to show that it's the shortest, you have to essentially show that no other path is is long. So every other path is longer.
Ben Rady: Mm-hmm. Mm-hmm.
Matt Godbolt: And so where does the analogy break down with AI? AI, I think can be very useful for the kind of NP-hard things or whichever is the one that can be polynomial time checked.
Ben Rady: Yeah, easy to verify, right?
Matt Godbolt: That's where I'm going with this.
Matt Godbolt: Exactly right. Thank you for saying it much more succinctly than I. But yeah, so again, physicist friend is like, yeah, I can set it on certain things. And as long as it gives me the citations of papers, which I can click on afterwards and then read the ones that it's found.
Ben Rady: Yeah, yeah, yeah, yeah.
Ben Rady: Right.
Matt Godbolt: There's obviously an error of omission where it didn't read things and they might be important. But the ones it did find or the ones that it summarized in a particular way, I can read through myself and go, yes, this seems reasonable. And it was much quicker than me reading them all.
Matt Godbolt: But, but then there are the the other problems where you, you know, you just have to kind of go, I guess it's right. And I suppose to an extent, actually, if you're a non programmer and you asked AI to make you a Python program and it's generated 3000 lines of Python, you run it and it appears to work.
Ben Rady: Right
Matt Godbolt: I mean, mate you know, maybe that's, maybe that is the the first category because you can prove this right, but you still don't really know that it works on your system.
Ben Rady: i i would yeah i would I would argue that if you really generated 3000 line Python program, unless there are there's no conditional logic in it whatsoever, and it is just straight through 3000 lines executed the exact same way every time, there's no way that you can actually know that that's correct in all cases, right?
Matt Godbolt: Yeah. Yeah.
Matt Godbolt: But, you know, to your point earlier, you said, you know, you just ran it a few times empirically, it's okay. And and that's so kind of almost like the verification "seems okay to me".
Ben Rady: Yeah.
Ben Rady: Yeah. And I mean, usually what I, be honest, what I'm usually doing there is looking through the conditionals being like, can I delete this? Is this, is this work?
Matt Godbolt: Oh, that's an interesting way of doing it.
Ben Rady: No, I mean, this is what I did with bash scripts forever. Like I would, I would have, I intentionally try to write bash scripts as I'm not, I never found a good unit testing framework for bash. I think we talked about this maybe in another episode.
Matt Godbolt: We have actually, yeah. That was one the early ones.
Ben Rady: Yeah. And so my de facto method for writing bash scripts up until the, you know, creation of Claude, which now I just use that, was take a shell script, make it executable, an empty file, i'll make it executable, put it in watch and have it run over and over and over again, and then try to solve the problem by writing as few conditionals as I possibly can, right?
Ben Rady: Just have it go straight through. There's only one code path through this and I can just tell empirically that it works because it prints out seven at the end, right?
Matt Godbolt: Right. And then keep editing.
Ben Rady: Yeah, and that was my method because it's like, okay, it's dumb, but it it like there's only one possible way to run this program and the one way works empirically. So that's good enough for now, right?
Matt Godbolt: Right.
Ben Rady: It's when you have like the, oh, it handles this case and handles that case and has a combination of these two cases. it is is that Like that's when you start introducing bugs without realizing it. And that's when you actually need to understand all of the possible paths through the code and, you know, hopefully structure them in a way where there's not like 2^n of them, right?
Matt Godbolt: Mm-hmm. Mm-hmm.
Ben Rady: Because otherwise you're just going to build unreliable software. Right? So I completely like, um I think that, you know, this, the sort of vibe model of coding, especially when you have sort non-technical people doing it, is really like a rising tide lifting all boats. It's making people much more productive.
Ben Rady: But if if we start to try to build complicated systems with this, I'm not really sure that that actually works. Because at the end of the day, the context window of the AI can only be so big and it can only remember so many things.
Matt Godbolt: Yes, and...
Ben Rady: And when somebody asks the question, why does this not work? There's not going to be anyone that can provide an answer.
Matt Godbolt: ..That is more to the point. But yeah, I think like the the context window thing is a whole other thing.
Ben Rady: Yeah.
Matt Godbolt: There are obviously technical reasons there, upping those and stuff. And then you know the fact that my own context window as a human is, getting as I'm getting older, much smaller.
Matt Godbolt: And you rely much on, right, yeah.
Ben Rady: COVID brain frog, right? Brain fog...
Matt Godbolt: Yeah, yeah. I, there could be a frog in there as well. That would explain a lot.
Ben Rady: Brain frog is a, that's a whole other symptom of COVID and trust me, you don't want that one.
Matt Godbolt: That would absolutely.
Matt Godbolt: Oh, man. Yeah. No, I think that that those those issues will start to be, that's where the innovations are happening, as far as I can tell, in the AI world now.
Ben Rady: Yeah.
Matt Godbolt: But um just one final thing as we're hitting towards the end of one of our longer episodes here. In, I think, 1980, Fred Brooks of Mythical Man Month fame wrote an essay called No Silver Bullet.
Ben Rady: Mm-hmm. Mm-hmm. Mm-hmm.
Matt Godbolt: And I've been ruminating that on a ah lot recently as I've gone through the arc of LLM sort of usage myself from, oh my gosh, this is mind blowing and I think it's going to change the universe to this is terrible.
Matt Godbolt: This is worse. I'm wasting my time on this, to: ah maybe, maybe not. To, now reasonably, it's another useful tool in my arsenal. Sometimes I can fire it on a couple of low-hanging bugs and I can then PR and I'm not sure if that's faster or not. I don't know yet.
Matt Godbolt: Sometimes it can help me by doing things that I'd never think of doing.
Ben Rady: Yeah.
Matt Godbolt: Other times it gets in my way. I don't know. It's a tool. It's like grep. It's like sed. It's a bit more involved. But to your point earlier, I think it's probably got the biggest chance of being a silver bullet out of all the things that we've seen so far.
Matt Godbolt: But you know, Fred's point was, everything has come and gone and nothing has ever fundamentally changed the problems of software engineering. That is, we need software engineers to do software engineering practices.
Ben Rady: Yeah
Matt Godbolt: And I think it's probably worth a re-read
Ben Rady: Yeah, there's always going to be somebody who is ultimately responsible for making sure the technology works. And that person is going to become a technologist whether they want to or not. Because they'll just there'll be situations where they're like, why doesn't this work? It's my job to figure it out. I guess I need to figure that out.
Matt Godbolt: Yeah. And whether they work with an AI to like kind of say, why is this not working? And have that sort of debug experience, pair programming experience, but they are still being brought along for the ride on the technology journey and there's no evading it.
Ben Rady: And that's just, there's no avoiding that.
Ben Rady: Sure. Yeah.
Matt Godbolt: And maybe that's hope for us all.
Ben Rady: Yeah.
Matt Godbolt: We'll see. All right. I think we have reached a good enough stopping point for now.
Ben Rady: Yeah, that works.
Matt Godbolt: And I will see you next time, my friend.
Ben Rady: Until next time.