Matt Godbolt: Hey Ben so this is the first time we've recorded this while we listen to our own jingle to start with and I can't help but grin it's so weird. So when I was expected to I'm so used to listening to this while I'm editing it.
Ben Rady: Hey Matt.
Ben Rady: Um, yeah, that's right.
Ben Rady: Ah, ah yeah.
Matt Godbolt: And then I had to remember? Oh no I have to speak now. So what are we talking about today.
Ben Rady: Yep yep, um I had I had a vague idea of something that we can talk about and as usual I have 12 pages of copious notes on this topic. So there's no risk that I'm going to run out of things to say in fairness.
Matt Godbolt: Awesome.
Ben Rady: I don't think there's a risk that I'm going to run on to things to say on any topic at any time but I have done 0 preparation for this. So there is a chance that this is our first minute map um but the the idea that I had is.
Matt Godbolt: True.
Matt Godbolt: Ah, we'll see how it goes it'd be fine.
Matt Godbolt: Oh.
Ben Rady: Avoiding Abandonware I think that we can talk about um a particular Phenomena that occurs a lot that I see a lot in the software industry that I read about a lot in software and technology and basically any company that uses software.
Ben Rady: I see this Phenomena and it's and it's regrettable and I is somewhat avoidable but also like many things is a product of sort of systemic forces and it may not be possible for you as an individual To. Excuse me, it. It may not be possible for you as an individual to avoid it. But it is certainly come possible for companies to avoid it. So I think I think we could maybe talk.
Matt Godbolt: Right.
Matt Godbolt: That's interesting I mean it immediately makes me think of that Xkcd with the big pyramid of of things that companies have built Ofton you know from the bottom up all these different pieces and then there's this one thing like at the bottom right? Hand corner a tiny little sliver of a stick and the little arrow is you know.
Ben Rady: Yeah, right.
Matt Godbolt: That one library that some guy in the middle of nowhere is is the only person left supporting and it's like well you're talking about 1 step beyond that where nobody is supporting that thing anymore.
Ben Rady: Yes.
Ben Rady: Ah, well yeah, that is a particular so that's like the load-bearing internet person who's like you know, like all of modern technology is propped up by this one guy in Sweden right? Um, so this is.
Matt Godbolt: Yes.
Matt Godbolt: Right.
Ben Rady: Not like I think some of the same forces are at play here. But what I'm talking about is is within the context of a company um a situation that I I think occurs a lot where people will build a piece of software and it will go through certain phases.
Matt Godbolt: M.
Ben Rady: And the phases are roughly like in the early days. There's a lot of excitement about it. There's a lot of change that's happening. There's a lot of new development and then the software starts to ah stabilize and then the software starts to ah, kind of.
Matt Godbolt: M.
Ben Rady: Freeze like be be stay be stable in a bad way. Yes, ossify That's a great way to describe that um and then because it is difficult to change the organization will start routing around it.
Matt Godbolt: Like ossify really more than yeah, like.
Ben Rady: Because they need change to happen because the world is change but they can't change the software for various reasons which we'll talk about and then other bad things happen when they start routing around it. So I kind of want to walk through that sort of whole process from beginning of end to end of like.
Matt Godbolt: Um.
Ben Rady: A typical project right? And how it and how it suffers this fate and what happens to the people involved in it and what happens to the company. So so starting off I think a lot of projects. You know the sort of Greenfield project right? And many of us have had opportunities work on green people projects and they're great.
Matt Godbolt: Yep.
Matt Godbolt: I'm intrigued. Go ahead.
Matt Godbolt: Um, yep, and no dependents. No customers nobody nagging you about it. No bugs.
Ben Rady: And the reason that they're great is because they have 0 complexity right? There is there is no no exactly you have no bugs. You have no dependencies. You have no code that you need to read? Ah. You have no slow test suite that you need to you know, run for 12 hours before you can do a deployment right? You have nothing and it's great. Um, and not surprisingly one of the reasons that programmers love those kinds of projects is because they are very productive at the start.
Matt Godbolt: I.
Ben Rady: Right? And I think part of the reason why is or they can be. You can also get tied down in a lot of like you know analysis paralysis and other things and yes, yes, so that can certainly happen and and does.
Matt Godbolt: Yeah, how should I do this? Yeah should what what technologies should we use in like well yeah, but but.
Ben Rady: But there are many situations in which you started in one of these projects and you have a very clear idea of what you want to build or at least you have a rough idea right? like maybe recalling our inner ah iterative versus incremental episode. Which yeah, um, you know, maybe you're like well I'm going to paint the Mona Lisa so I know roughly what she looks like.
Matt Godbolt: I Want hold that again. Ah yeah.
Ben Rady: And I'm going to pencil that in and that's going to be my first draft right? and you have a very clear vision in your head and you can be very productive and as a person who likes writing code there is maybe nothing more fun in the world than having that vision in your head and making it real right or making some draft of it real and so you're very productive.
Matt Godbolt: Yeah, yeah.
Matt Godbolt: Yeah.
Ben Rady: And then you know eventually if you're doing your job. Well you build some software that somebody wants and you know like the bjorn ah strasstra quote about you know there are 2 things in the World. You know the. The programminging languages that people use and and ah complain about and the ones that they don't use right? and I'm batching that quote terribly but I'm hopefully getting the idea right? The precise wording is much more Ah, pithy. Yes, He's very good at that. Um, but it's this where you know you build something that somebody wants.
Matt Godbolt: Um, yeah, yeah, yeah, yeah, yeah, but yeah.
Matt Godbolt: Pithy. Yeah yeah.
Matt Godbolt: Yeah, yeah, often with the word just in it. Can you just make it. Yeah, but.
Ben Rady: And they immediately start complaining about it and they're like oh this is great but can you add this and can you do this and can you change this and can you do this right? Yeah, Can you just do? can you just you know make it 10 times faster. Um, the the um and so what happens is is that you start. Having to adapt to change which you didn't really have to do with the beginning right? at the beginning you're in this phase where you're like I have this vision I'm going to build a thing I'm going to build it until somebody says ooh that's cool and they start using it and then you start getting some feedback and some people will tell you to try to get to that phase as soon as possible. That's maybe a whole other discussion.
Matt Godbolt: M.
Matt Godbolt: Yeah, ah.
Ben Rady: But eventually at some point in the process you're going to start if you're successful in building software that someone wants they're going to ask you to change it and they're either going to ask you to change it because the world changes or they're going to ask you to change it because of some new capability that they want or they're going to ask you to change it because they encountered a bug but whatever The reason they're going to ask you to change it. And then you're going to be in this process of changing it right? and you're going to change it and change it and change it and change it. Um now I think we've kind of talked about before we were in our technical data dead episode about complexity and how complexity is one of those things that I think is a more.
Matt Godbolt: U.
Matt Godbolt: Ah.
Ben Rady: Useful thing for programmers to talk about rather than the metaphor of technical debt. Um.
Matt Godbolt: Especially given how folks reasonably disagree on what debt is you know? Ward's the original idea is not how I use it and so on and so forth. And yeah, so so complexity is something that I think we all have a better understanding about what we mean when we say this is complex and there's necessary complexity.
Ben Rady: Right? exactly.
Matt Godbolt: And then there's the other kinds.
Ben Rady: Right? right? Yes and I mean very very ah ah, practically here. It's like you start out in the greenfield project. You have 0 lines of code right? You get to the point where you've deployed your first version and someone uses it and you have end lines of code. And then you get to the point where you're making change number 5263 and you have some multiplier on endlines of code and that is complexity right? Whether it's more complexity than you need or less. It doesn't really matter. It's complex and it's hard to change and it's hard to understand and it's hard to add more people to the team so that you can add more changes right? and all of this gets.
Matt Godbolt: Um.
Matt Godbolt: Yeah.
Ben Rady: Slower and more difficult and harder and eventually you reach a point where the people who are asking you for changes start to get frustrated because the changes aren't coming quickly enough and in dysfunctional organizations This can lead to situations where people are pressured to work long hours.
Matt Godbolt: M.
Ben Rady: To try to get changes in and then you know usually that has some more complexity because they're a bleary eye to the middle of the night trying to figure out how to do things and that just makes the problem worse and then it just gets slower and slower and slower and eventually that frustration reaches a point where people say okay.
Matt Godbolt: I.
Ben Rady: We shouldn't count on this system being able to change quickly right? That's not a good strategic plan because they have shown in the past that they can't change quickly and you know there are some really negative things that happen when a project reaches that stage you know people start. Seeing that the future of this thing isn't really going anywhere. They start to leave which means you know you have a loss of talent working on the project which again makes the rate of change even slower. Um you.
Matt Godbolt: Um.
Matt Godbolt: And you lose the institutional knowledge that they had when they leave. So yeah.
Ben Rady: Exactly right? exactly right? So all that complexity that was maybe loaded up into someone's head that that head is now somewhere else right now and you can't use it to figure out what the hell is going on in this code the organizational support might start to get pulled Away. You know in terms of funding or in terms of you know. Other resources that the organization can Provide. You know it starts to be thought of as this thing that is that is ah in a more positive light maybe stable and and useful but in a negative light something that is old and crusty and legacy and no one wants to deal with right.
Matt Godbolt: Ah, right? Yeah yeah.
Ben Rady: But the needs of the organization are still there and the reason that you built this software is probably still there and so what happens is people start to route around it and initially they may start to route around it by doing things on their own that the system also does or also could do but for the reasons previously stated that it's difficult to change. They. They choose not to where they feel like they can't right and that can be real or imagined it doesn't even really matter and so you almost get this like Scar tissue where it's like the organization starts to like like work around this thing that is part of the organization itself like this critical piece of software they but they start.
Matt Godbolt: Yes.
Ben Rady: Routing around it and what I see quite often and I think that's a pretty common pattern and what I see quite often is that it can easily reach sort of a tipping point where people start to say like maybe you you have like an a little ah, an inflection point almost where it's like 2 core developers leave the team or they leave the company because they're sick of working on this legacy thing that no one cares about anymore and then maybe ah, they hire a couple new people to come in and take it over and the new people look at it for two months and they're like oh my god this thing is so complex what we need.
Matt Godbolt: Right.
Matt Godbolt: Right? right.
Ben Rady: Is a rewrite and now the cycle begins a new you have a green field project where you get to do the whole thing over again. Yeah.
Matt Godbolt: Um, I mean there's there's well documented version. Two problems of things as well. Which is his own second system.
Ben Rady: Yes, second second system effect and all of this other stuff and and you know in dysfunctional organizations this this process of you know renewal and renewal is of way too positive of a term to put on this.
Matt Godbolt: Ah Phoenix from the ashes kind of thing. Yeah is poetic. Yeah.
Ben Rady: Yes, it's not nearly that good for anyone. Um just happens over and over and over again and and so you know um I think you know there are some ways that you can accidentally encourage this and there are some ways that you can help prevent it I think that at some point it it is.
Matt Godbolt: Um.
Ben Rady: Almost like the natural outcome of every project to sort of reach this point where it becomes difficult to change and that can be for a number of reasons it could be because of the complexity it can be because so many people depend on the existing behavior that there just isn't that much left to change right.
Matt Godbolt: Right? right? right? The the thing that immediately Springs to mind when you talk about this is lib curl and the curl command line right? which is used in every program ever and I know.
Ben Rady: Um, yeah.
Matt Godbolt: Daniel I think his name is I've subscribed to his ah Github Repo I can see the changes going through. It's like ah or zedlib is another one you know, um, that's another one I find it fascinating to look at these things which are essentially. Immutable and can't change ever again because you know like Gzip files have to keep working deflate as a spec and it isn't changing. Ah but they are maintained. Ah but and you'll see. Taking zedlib for it. As an example, you'll see like lots of people come in and say hey I'd love to make this thing far. Go faster I want to change this thing. Whatever and they're immediately closed down was like yeah, but unfortunately that exposes this issue and this bug you or the fifteenth person to try and make this change. It doesn't work really sorry. Go have to write an faq for like why you can't do this or something.
Ben Rady: You have.
Matt Godbolt: Rather than this and it's well-meaning. But it's like it can't change because it is like so fundamental to everything. Ah you know that's that and likewise curl I mean obviously both of these projects still go Curl is probably more actively maintaining as things are being added to it and whatnot but like it is.
Ben Rady: Yeah, yeah. Yeah, yeah, yeah.
Matt Godbolt: It's a core piece of infrastructure in the world and necessarily it has to be conservative in its change now even if change is required.
Ben Rady: Right? right? right? And the people who are there in the in the more functional forms of this The people who are there the custodians of these stabilized projects have probably seen it all at this point right? like if you come along with some bright idea like it's it's It's probably it's probably in the issue list like it's not new.
Matt Godbolt: M yeah.
Ben Rady: It reminds me of this sort of satirical thing that I saw once of like the ah you know back this is or is going to date me a whole bunch hereriff like the the forum like the bulletin board for the time travelers association and this the sticky post at the top of the forum is ah. Do not go back in time and kill hitler because if you do then? no world war 2 which means no darpa which means no internet which means no time machine which means I have to go fix this again. So do not go kill Hitler. That's the sticky post right? and it's sort of like.
Matt Godbolt: Ah, but.
Ben Rady: The obvious thing is like oh kit, you just can't you just do this and in those mature projects. It's like don't you think that we would have by now if we could yes.
Matt Godbolt: Yeah, hey, thanks.
Matt Godbolt: Yeah, the the poor gray beard tripping over their beard as they are. Um, yeah it is is like well let me tell you a story. We tried that in 1977 and it didn't work out then and.
Ben Rady: Um, right? Yeah yeah, right right.
Matt Godbolt: Many people have tried since you know we've talked about this with comments actually in our comment episode. There was the comment where it's like you know you put the explanatory comment that says please don't change the following line. It looks like it should be changed to this but whatever. However, if you want to try it and get it working and get all the unit test passing then go ahead.
Ben Rady: Um, yes.
Matt Godbolt: But if you don't come back to this comment and bump the number and the numbers like 27 27 other people have failed before you.
Ben Rady: Right? Yep Yep! So I think that's like the functional vert the because like I said I think every so piece of software will undergo this life cycle right.
Matt Godbolt: It tends. There's like the yeah the D the greenfield um ah sort of um explosion of activity. There's you know you can do anything you like you can completely change the api once a day once an hour even because you've got no users.
Ben Rady: And then.
Ben Rady: Okay.
Matt Godbolt: Even once you've got 1 or 2 users and and they're happy enough. You can sort of say really sorry we're just completely revamping everything you're going to have to migrate and we'll either help you or good luck you know, move over to it and then you reach the sort of middle- age part right? So that's just sort of like your your youth and then your teenage years when you've got you, you can be a bit spiky still because you know your users are kind of like well okay I guess so.
Ben Rady: Yeah.
Ben Rady: Um, business.
Matt Godbolt: And then you become a middle aged person or an adult and now you're like well I can't just go and change this thing because you know thousands of people will be affected by this and so you have to slow down and be a bit more sensible and boring. Um, and then you know.
Ben Rady: Um, yeah, and well I think I I mean I think I think this is like I think this is the you know do you have.
Matt Godbolt: But then so what's the midlife crisis version of this. What what is it where the project changes. Anyway, it goes Ha We're rewriting all in rust.
Ben Rady: Are you like a you know cheeseburgers 6 times a week ah no exercise sedentary. You have a heart attack in your fifty s project right? Yeah oh no, yeah, right? Um, or are you the like you know, super exercise.
Matt Godbolt: No politics. Please Oh I thought you were going to go somewhere else with this. Ah.
Ben Rady: Ah, Multivita you know the whole thing and you live into your ninety s right? and the way to like achieve that sort of like longer lifecycle is to preserve your ability to change for the things that are really important and not sort of fall into the trap of oh we're trying to cram everything in here.
Matt Godbolt: Right.
Ben Rady: But we can't so what this needs is a rewrite that can do all the things and so there's lots of factors to this one factor is being able to like manage change over time for a very long period and being able to say no to changes that don't make sense as we sort of have been talking about here. Um, another thing I think is is.
Matt Godbolt: I say.
Ben Rady: Ah, understanding that projects should only ever really get so big right? if you try to cram everything into your system. You're going to either have to throw a ton of resources at it to be able to.
Matt Godbolt: M.
Ben Rady: Manage that kind of change and it's still the cost of that change is still going to be incredibly high but a better way is okay, maybe we don't have to rewrite this what if we just made something else that could fulfill the needs right? like or split it in half right? That's another way to do it.
Matt Godbolt: Um, which is there's ah, there's a sort of version 2 I feeling to that because that's one of the ways of avoiding the trauma the the trouble and the problems of changing. Um, ah. An existing project for which you have downstream customers is to essentially make a new version and say well never the Twain shall meet. You know this is a new Api. It's a new version. You never need to touch it. It's essentially a new project. Um, and that is a way of managing the the change um, aspect of it.
Ben Rady: Miller.
Matt Godbolt: And what you're saying is you could split a project in 2 I guess it's a different dimension there Rody what you're saying with 1 is you're saying like this serves the need of something and you're saying that. The only reason you would rewrite it? Um, well no, not the only reason a reason to rewrite it is because the internals are so hard. You can't change it. You can't make those changes and I'm thinking. Well there's also this the periphery. There's the the external interface and.
Ben Rady: Yes.
Ben Rady: Ah.
Matt Godbolt: So it's very hard or maybe even impossible to have a good experience when you have to support like a v one Api and a v two api. It can be done then obviously you can think about it beforehand but um, um yeah I wonder.
Ben Rady: Um, yes, yes, right? Yeah and and like the v 1 v two api I'm I'm I would be less um suggestive of that and this is more about like as you're getting those requests.
Matt Godbolt: Who.
Ben Rady: Like can it also do this and can it Also do that really thinking about like should it do this and should it do that like I'm not saying that your request is invalid and that we shouldn't build that capability within our organization I'm just saying like maybe not in this codebase. Um, because because if you look for those opportunities.
Matt Godbolt: Right. But not necessarily in this.? Yeah yeah.
Ben Rady: Then you're reducing the complexity of the system and also creating opportunities to deliver on those requests quickly because you're you're potentially doing it in in something else If you're just throwing it into your codebase because that's where your build system is and you've got the deployment already set up and now because it actually belongs there. That's.
Matt Godbolt: Ah I don't know what you're talking about right.
Ben Rady: Going to hasten your demise as a software project right? That's like eating the cheeseburgers right? It's not good for you? Um, so ah so that's so that is sort of like 1 way in which um, ah I think you can kind of deal with this right.
Matt Godbolt: You can stave off the the the this this tendency to make to a project becoming ossified is to say well. Okay, let's let's not put lots of things into it which brings us possibly arguably unnecessary complexity.
Ben Rady: Yeah, um, yes.
Ben Rady: Right.
Matt Godbolt: Which then also tangles our feet um makes it harder to change down the line and then eventually we kind of abandon the project to move on to v 2 of the thing or something different entirely. Okay that that I could understand. Yeah.
Ben Rady: Yeah, yeah, yeah, yeah, and that is separate from like splitting it. So like 1 one approach for this is if you've identified parts of your system that have stopped needing to change right? like like you're not getting any like like you're you know message.
Matt Godbolt: Oh.
Ben Rady: Bust steely whatever it is I don't know whatever part, whatever part of the system. It's like yeah, we've got this code. It's in the codebase we run the tests on every build we haven't changed it in 3 years right maybe that's a library and it's could be a library. You don't share with a put'll just pull it out compile it 1 time run the test the 1 time.
Matt Godbolt: M.
Ben Rady: And if you ever need to change it. It's there but you don't have to carry the complexity costs of that forever right? And that's sort of a cheap way to split things.
Matt Godbolt: That's interesting. So we we've talked about things like ah the the anti-patn of commenting out code because it might be useful before now and this seems like an sort of similar feel to it where you're saying look.
Ben Rady: Man.
Matt Godbolt: Ah, it's not that I want to delete this code. It's valuable. Um I'm just going to extract it and put it somewhere else and it is slightly more live than in source control and gone in my current incarnation because it's just it's been published as a library separately I like that I like that the the usual argument against it.
Ben Rady: Man.
Matt Godbolt: Which is the one we did with the whole commented out code or code that you don't use that's in your project but is that refactoring tools and things like that can bring it along for the ride. But if you can find a nice seam to extract that library that's less of a problem Anyways, like yeah, well this is where the compression library goes. We're not using compression anymore.
Ben Rady: Right.
Matt Godbolt: But it's a really nice component I'll take it out. We'll put it in the in the the company drawer of like useful little components and then we'll get on with the life I don't have to keep it up to date or I don't have to do anything with it. But it's there if I need it. Oh right.
Ben Rady: Right? Well and I'm saying like even if it is being used even if it is actively being used but it's not being changed right? like if you go through and you look at your git history and you see like yeah this code is executed every day in our system we haven't changed it since 2014
Matt Godbolt: Oh That's fascinating. Well, That's a fascinating. Yeah yeah I actually have a perfect example of that in my current code base where we lazily so this is exhibiting all of the bad things actually that you said.
Ben Rady: Right.
Matt Godbolt: So it was a serialization library and ah in order to have like this the the tools that um like command line tools that allow you to decode and reencodes and dump and debug serialized forms of data. There's a number of applications that go alongside the library that you would otherwise link with and. As I was writing that out I it was one of the first times in the company that we had a bunch of native code and so I wrote a somewhat straightforward and and small um application like Cli app framework with command line parsing and all that good stuff and ah, it's so convenient that in my subsequent projects. Um, although I I started off needing the serialization library part I'd actually pull across the application framework as well through the library and be like oh this is cool. We'll use it in my rest of my project and now of course the serialization library has gone by the wayside but I'm still depending on it because it's got this application framework and my and and I'm like this is terrible I'm pulling this.
Ben Rady: Um, for rama.
Ben Rady: Ah, yeah.
Matt Godbolt: Huge amount of complexity of this serialization library to get like 15 lines of code that now if I needed to change I have to cut a new version. The serialization library and that is of smell and I should go and fix that sooner rather than later of myself. But um, yeah, it's easy to happen I see what you mean? yeah.
Ben Rady: I Want her. No.
Ben Rady: Um, yeah, yeah, and and you know with with all of these things your ability to do this is going to be a function of your ability to decompose and refactor which unfortunately is changing the system. So if you wait until the system is hard to change before you start thinking about these things then you will never come back right? and you will and you will wind up in that place where you are like you know the ah the bad Phoenix where you're you're trying to rewrite the whole thing. Um.
Matt Godbolt: Yeah, ah.
Matt Godbolt: Ah, there's a username bad Phoenix There has entered the chat room. Oh oh no.
Ben Rady: That Phoenix Yes, So so I I think you know again this I keep drawing parallels to you know Health and New Year the sort of middle age and everything but you got to be if you yeah right? Well there you go? um.
Matt Godbolt: Ah, so top of mind is for some reason. Ah.
Ben Rady: Ah, maybe the the tacos and beer that I had last night has has this on my mind. Um, is you know if you want longevity in these projects if you don't want them to ossify and then be completely routed around and then rewritten.
Matt Godbolt: Ah.
Ben Rady: Ah, then you have to have a long-term view on these things you have to be like I realize that we don't need to break this out now. But if we don't start doing things like this. It will be very difficult to do later and then we've just practically won't do it and that means that we won't be able to change it and then the organization will route around us right.
Matt Godbolt: Um.
Ben Rady: Problem is a lot of organizations and this gets into some of the structural causes of this Phenomena A lot of organizations take a sort of project oriented approach to building these kinds of systems right? They don't think of them as ah.
Matt Godbolt: Yep.
Ben Rady: You know the people and the software like when I think of software. It's like I think those 2 things are are inseparable like ah you know, handing someone a codebase with no team to go with it is like a curse right? It's like if you want to slow someone down like you know.
Matt Godbolt: Mmm. Ah.
Ben Rady: Take like ah like an unspecified chunk of code be like here's this this chunk of code from your most successful competitor boop you'll you'll you'll like ah you know, kill their performance over the next six months of as they try to poke through it and figure out what the hell's going on right? Um, and.
Matt Godbolt: Yeah, yeah.
Ben Rady: So I view that those things as as inseparable, right? Like if you have the people and you have the code and the code is loaded up into the people's heads and the people know how to change it and as soon as you pull those things apart and the people move on to something else. The code will solidify. It will ossify right.
Matt Godbolt: Oh.
Ben Rady: And nothing will accelerate that more than pulling people off of the team without giving them the ability to sort of organically replace the folks on the team. It's not that people should stay on projects forever. It's that you should bring people on bring them up to speed and then have other people move off right? have ah have a good transfer of knowledge. Not like the fake.
Matt Godbolt: Yeah.
Matt Godbolt: Yeah, yeah, yeah.
Ben Rady: Two weeks where you're like oh yeah, read the code and ask me if you have any questions right? like that's not going to do anything for anyone right? It has to be like months. Um, but if you if you have an organization that is project oriented if you see software systems as you assemble a group of people they do a project.
Matt Godbolt: That's not really it. Yeah.
Ben Rady: When the project is over they move on to the next project then you are encouraging. You are encouraging this behavior of ah not looking out for the long term and I'd say why would we extract a library I'm going to be working on something else in 2 years
Matt Godbolt: Um, yeah, yeah.
Ben Rady: My goal right now as told to me by my boss is add these new pieces of capability and I'm not going to stop to think if it's appropriate for this project that would be in not in my best benefit right? like that would not be in my self-interest if if the if my boss or some of their stakeholders says.
Matt Godbolt: I why.
Ben Rady: Add this capability to my project. Even if I know that it's not appropriate for the project. It's probably better for my bank account if I add it because that's what success looks like at this company right is adding capabilities to software right? So if that's the way that you think about it then.
Matt Godbolt: Yeah, right? Yeah yeah.
Ben Rady: Problem is is that this becomes a very sustaining and self-fulfilling prophecy where it's like project number 3 is rewriting project number 2 and project number 2 is rewriting project number one and that just repeats over and over and over with a new crop of software engineers every for you years and a new group of product owners and managers every few years and they're just rewriting the same solving the same problems over and over and over again and all of them getting promoted up and out as a result. Um and I think in very large organizations, especially this can go on and it's just a huge waste and it's like you know you wonder why? these.
Matt Godbolt: Yeah, yeah, yeah, yeah.
Ben Rady: Giant tech companies. Get so stagnant I think this can be 1 of the reasons why it happens is because they have this sort of project-oriented view.
Matt Godbolt: Yeah, yeah, that's that's a fascinating observation. Um, So what have we got so far I'm just looking at my list of how to how to how to not. Um, make abandoned where we actually didn't really sort of define it as abandoned where I don't know. Yeah, but like you know projects that have reached their their sort of light terminal. Ah, Ah, yeah, right, as abandoned where right.
Ben Rady: Yeah,, they've they've ossified and they've lost their support is how I would describe this support as in there are no longer engaged competent software engineers who are working on it. However, slowly in the you know later phases right? but working to change it like they have it all. They have all the complexity loaded up into their heads. They can be the ones to answer like oh yeah, no, you can't do that to zlib and here's why right? That's the the healthy outcome the healthy longterm outcome the abandon where is.
Matt Godbolt: Yeah.
Ben Rady: Ah, anyone who would work on this project is a sucker right? No one understands how it works No one wants to change it. There's no organizational benefit or value or recognition from changing it. It's just sort of seen like this old legacy thing. That's.
Matt Godbolt: Yeah.
Ben Rady: Abandon where and I think avoiding that is important.
Matt Godbolt: That's ban andware right? So that right? and so yes to to a point one is one way to avoid getting yourself into that situation is to be very careful about what you put into the project in the first place so that you don't add complexity that's unnecessary which might be beguiling.
Ben Rady: Man.
Matt Godbolt: In the short term. Um, because if your users need it. Maybe you should supply it. But then you know you end up growing the homer car right? If you're not careful and then you're like and oh this is too complicated to even work and then the the project comes to a natural ah end. Um, what else have we said.
Ben Rady: Right.
Ben Rady: Splitting splitting splitting software and you can split it into multiple systems. You can pull libraries out. You can just change the way your build works so you can just sort of shove a piece of code over in a corner and be like yeah I've even changed this in 10 years and we're probably never going to. There's lots of ways to split things.
Matt Godbolt: Splitting. So yes, it.
Ben Rady: But splitting right right.
Matt Godbolt: But splitting and then not paying the cost of having it there or minim. Ah absolutely minimizing it so scaling down to 0 the cost of carrying a bit of code be it because it's put in a repo and then you don't look at it again. But it's there. Yeah back down again. Yeah.
Ben Rady: Yeah, yeah, you're shrinking your complexity burden by doing that which makes it easier to change right.
Matt Godbolt: Yeah, yeah, and then obviously using techniques that make it easier to change full stop and be you know? So obviously the the word that you haven't said so far what work could that be I'm just trying to remember what word. Let's think is it ah arresting, no chesting no something.
Ben Rady: Wonder it's not coming. It's not coming to mind I don't know. Yeah, but yes, yes, right.
Matt Godbolt: All right? Okay, but yeah, if you've if you've written it so that it's tested and test a bowl and you've got good tests that help you and don't aren't brittle and don't make it so that you have to do things in a particular way else. You know it's hard to change all the things we've talked about once or twice on this podcast then.
Ben Rady: Man.
Matt Godbolt: Then it it makes it easier to change. You know a good test make it easier to make change and be confident that you've made um made the right changes and you haven't broken anything so that all those good things that obviously helps a project. It's longevity. Ah I mean ah maybe implied in what you've said is.
Ben Rady: Hit it.
Matt Godbolt: Organizational foresight and usage of appropriate usage of Conway's law in the right way to say you know you mentioned project-based development and like moving on from a project to the next project I need to have the last one cave in and then you kind of replace it as a sort of almost like a cycle which certain.
Ben Rady: Right? right. I Can right.
Matt Godbolt: Again, Conway's law I'm going to bring out here. You know you can see that happening maybe with with a consultant based development where you bring in a bunch of consultants and they build a thing and build a thing and build a thing and they're like okay we've done the thing and they walk away from it and then a couple of years or maybe less later. The folks who are left with the thing are like this doesn't and we don't know how to change it. We don't know really what it is.
Ben Rady: Um, yeah.
Ben Rady: Right? yep.
Matt Godbolt: And we hire some more consultants who say well it will cost us this much to take your existing one or we can write a new one for you and you're like well let's do with a new one that seems yeah and then you yeah I mean I've never been a consultant but you're nodding at me. So I'm assuming right.
Ben Rady: Yeah.
Ben Rady: No I have worked on projects that were very much that were almost literally that. ah yeah, ah yeah sidebar 2009 I did a project for a bank of America which had recently acquired countrywide mortgage in the mortgage collapse. Ah, and.
Matt Godbolt: Yikes.
Ben Rady: They had to revamp their ah mortgage default system because it had gone from processing like 1 thing a day to hundreds of things a day and couldn't keep up. Yeah.
Matt Godbolt: Oh but us what a that's actually the human cost sad sort of other side of that's kind of poignant actually. Wow Yeah I can imagine.
Ben Rady: It was ah it was a crazy time. It was a crazy crazy time for me and the world. Um, but yeah, that was very much that project is that like the right thing to do in theory on paper would have been to take that existing system and scale it up or change it in some way. Ah, but it wasn't.
Matt Godbolt: Ah.
Ben Rady: Changeable. Ah, for various reasons right? And so what they had to do was rewrite it from scratch.
Matt Godbolt: Right? right? and I mean I don't think it's never a good idea to write something rewrite something from scratch because sometimes there are some fundamental things that you wanted to change or you want to try a different way. But yeah, it shouldn't be your default.
Ben Rady: Um, and.
Matt Godbolt: Strategy for dealing with the with with a changing environment around you.
Ben Rady: Right? right? Yeah I mean I I think when I when I think about rewrites. It's like first of all I'll be really clear that you really actually do need to do it because it's almost certainly going to be so much harder than you think it's going to be but also. Please make sure this is the last 1 right? The cycle ends with me. Um, because and you need to sort of ask some questions like there are you know important organizational questions about like how did we get to the state in the first place like what forces were in play that got us to a point where this.
Matt Godbolt: Ah, right.
Matt Godbolt: Yeah.
Ben Rady: Code had been abandoned and needs to be rewritten because as an individual person you may not be able to avoid that future unless something has changed in the environment to to make it feasible right.
Matt Godbolt: Ah.
Ben Rady: Um, and maybe and so things do and and it's not say that you can't but that would be the first question that I ask with a rewrite you know, maybe the second question. The first question being like do we really really really need to do this and the second one is how are we going to make this the last version.
Matt Godbolt: The last time we do it? Yeah, that's a great thought. Yeah, um so organizational structure ah ease of change in the first place.
Ben Rady: Um, yeah, yeah.
Matt Godbolt: What am'm I missing I've already forgotten they see I also went to the pub last night a different pub to you but I'm I'm suffering a little bit here I should have written these things down um not taking on yeah, splitting the project if it gets you.
Ben Rady: Ah, yeah, yeah, yeah, yeah, now us. But what's your projects don't take on functionality capabilities that you can move out to other things that are reasonable to move out to other things just because you can add them to your software doesn't mean you should think about whether you can you can sort of.
Matt Godbolt: And.
Ben Rady: Presp you can think of it as pre-splitting right instead of putting it in here and then taking it out again. We're just going to put it somewhere else right? because it can't go there and it will work fine over there right? That's a way to keep your complexity down for me like I actually have pretty specific metrics in my brain for when I start getting worried.
Matt Godbolt: Um, yeah, yeah, yeah, yeah.
Ben Rady: Ah, a piece of software is reaching its middle age and needs to be. You need to start thinking about how many cheeseburgers a week you're eating um when the lines of code are in the you know hundreds of thousands when the test suite starts to take longer.
Matt Godbolt: Just okay.
Ben Rady: Then my attention span. You know we've talked about the rule of eights before right? you know, ah point 8 seconds yeah that's great 8 seconds okay that's fine 80 seconds I've lost attention right? And so when you start making that transition from 8
Matt Godbolt: Ah.
Ben Rady: Seconds and just 20 seconds and you should be thinking. How are we gonna split this. How are we gonna move parts out of this. How are we goingnna decommission things. How are we goingnna get a little bit more life out of this software because if it keeps going like this eventually. It's going to take 20 minutes to run the unit tests and 3 of them will fail randomly when you do and you just got to rerun them and. You know the build takes a day to actually work and deploy and any request that comes in to change the software will take a minimum of a week
Matt Godbolt: Um, right? Yeah, so you're you're looking like you've come to the end of your your thoughts.
Ben Rady: Um, yeah, yeah, I'm sure. Yeah I'm trying to think if there's anything else to say about this and I feel like that's that's probably about it I Mean. There's other tangents here that we could maybe go on but I I think the main the main takeaway is just you know? Um, if you if you're a person in a technology organization as a leadership role and you're looking at your organization and you see this pattern this dysfunctional pattern of of death and rebirth. And you're wondering how to fix it I think the place to start with is what is your relationship between the software and the people because if your model of this is people build things and then they build other things and then they build other things and you don't have an engineering culture of sort of.
Matt Godbolt: Yeah, yeah.
Ben Rady: Stodialism I Guess where you have you know well compensated Senior engineers who are responsible for software that is a key part of your organization and isn't expected to change all that much because it's in its latter stages of life then you're just going to get this. Constant Rebirth right? and maybe because of your Organization. You're incentivized to do that and that's what you're supposed to be doing there and that's how you're going to get your exit and move on to your next job. That's not a place where I want to work either. Yeah.
Matt Godbolt: Um.
Matt Godbolt: Right? But but that's not a place where I want to work and it's no well I think we've done a pretty good job of covering all the bases and given this is you know as you say your 12 pages of notes which dear listener do not exist. Ah, Ben is completely making us up on the cuff. Ah that was very interesting and I think I'm going to have to go back and do some refactoring in my my serialization library. Ah, which I've been meaning to do. But yeah now I feel ashamed. Ah.
Ben Rady: Yeah, yeah.
Ben Rady: That. No, you have you have an opportunity to extend your software's life and you should take advantage of it. Not many people have that opportunity. Yeah.
Matt Godbolt: Right? Ah I should that is true. All right. My friend? Well ah I guess we'll leave it here then all right?
Ben Rady: Yeah, sounds good.