Matt Godbolt: Hey, Ben.
Ben Rady: Hey, Matt.
Matt Godbolt: You've got a new microphone.
Ben Rady: I do. I have a lot of new things. I have so many new things today,
Matt Godbolt: And we're going to find out in the edit whether or not this microphone actually works or not. So we're going to talk for like 45 minutes, and hopefully there's just more than silence on your side when we come to look at the recording.
Ben Rady: Yes. I mean, I'd say one chance in three,
Matt Godbolt: I'll take those odds.
Ben Rady: Yeah. Yeah. I have a whole new setup in lots of ways. I have a new podcasting setup here with the microphone. I have a new work laptop that for the first time in, well, since 2010, my primary work computer is not a Linux computer, and this is a whole new world for me. I'm still deciding if I like it or not, but
Matt Godbolt: That is an interesting, I have to dig into that because, well, I also have switched to a Mac.
Ben Rady: Oh, wow.
Matt Godbolt: A few months back. But that was out of a practical consideration of I didn't like being one of the strange people at the company who didn't have a Mac. And so begrudgingly I accepted the corporate Mac and all the things that come with the manageability,
Ben Rady: The corporate machinations.
Matt Godbolt: Oh,
Ben Rady: I'm sorry. I'll leave now. Yeah,
Matt Godbolt: No, but all of the, I understand why if you're a company, particularly one like ours with the kind of intellectual property considerations and things, you want to have a machine that isn't just kind of a free for all. And so there's a certain amount of manageability and ensuring of encrypted hard disk nurse and all that kind of stuff, which Ubuntu lags behind, understandably, compared to a full operating system like Mac, sorry, not that it's not a full operating system, but a corporate friendly operating
Ben Rady: System. A corporate friendly operating system, which is weird because it didn't used to be like that, right? It used to be sort of, so the last time that I didn't have a Linux computer as my main computer was 2009 when I was consulting. And basically my choices back then if I wanted to as a consultant sort of go into a random company that I had no idea who they were going to be and have any sort of hope of connecting to their local network and being able to share a presentation. The choices were a Windows laptop, which absolutely not, and a Mac. And so that's really the last time that I have used a Mac as my primary laptop, my primary work laptop. And that was for consulting reasons.
Matt Godbolt: Got
Ben Rady: It. And fast forward 13 years, and now I'm kind of back to that. But yeah, I had this sort of, because we started at our current company very early on, and it's like, what's the policy on work laptops? It's like, well, you
Matt Godbolt: Should have one.
Ben Rady: You should have one. Do you own one already? Well maybe use that one for now and then we'll get you a real one later.
Matt Godbolt: That's right. And we both got managed to sort of wangle essentially a pick your own adventure laptop from, here's a budget, go get a computer. We're like, sure, fine, whatever. We both, no. Well, I ended up with one of the Lenovo Carbon X Series, which I've been very happy with. But I think you had something different, didn't you from that point of view? No,
Ben Rady: I had that one for a good long while, and then about I have some problem with it. And then about six months ago, nine months ago, I switched over to a Dell, which was fine. So perfectly reasonable laptop. I enjoyed using it. And then I was like, yeah, I'm kind of grandfathered in here and I don't want to spend a whole bunch of time reconfiguring everything, but I probably should switch at some point. And then someone on my team went and took the project that we're working on and built it on a Mac and ran the tests. And the tests ran in half the time, and I said, we're switching right now.
Matt Godbolt: So true to form, it was testing that got you to switch computers, whereas for me, I mean it was just,
Ben Rady: Yeah, 50% of the original twice as fast that's sold. Where do I sign up?
Ben Rady: Yes, yes. So this is different. And I have kind of used the sort of Mac as a dumb terminal in some situations in the last few years. I think it works okay. It works fine. It makes me a little sad that you've got this incredibly powerful machine that you're basically just using to shuffle pixels back and forth. But aside from that, it works great. So the things that we were seeing is with the tests that we were running, I'm going to have to explain a little bit of context here to really make this make sense, but I guess that's the point of this podcast.
Matt Godbolt: Wait, we have a point. I thought the point was just every Friday afternoon, you and I get to hang out and not feel too guilty about it.
Ben Rady: News to me. Wait, we're trying to do something here. So the thing with the tests in this project is that they are at a very particular phase in the project. So the way that I think about as software projects grow, you measure them by things like complexity and two ways, I think, and capabilities and two ways to sort of measure those things is how many lines of code do you have and how many tests do you have? And if you're building things, at least the way that I espouse 'em to be built, you wind up being able to run your tests, hundreds of tests per second. And so what that gets you are some sort of interesting metrics that you can look at and see where are we in the lifetime of this project? Because when you're in very early stages and you've got a hundred tests or 200 tests, they should be able to run basically instantaneously on a single thread. You should have to do no additional work, no special tricks to get your tests to run in a second or less.
Matt Godbolt: And this is because of all the things that you talk about in terms of making tests targeted. So one test in your instance might be something like take a list, add an element to the list, is that element in the list? Yes. Okay. That's one whole test. And then you've got a different test and another test and another test so that you, it's isolated. This isn't a test of the test is do a comprehensive try of every possible combination of operations on a list. That's not a test. That's
Ben Rady: Right. If you were running a test like that, you would be running it in a different environment, probably. You'd be running it in ci and honestly, and that's
Matt Godbolt: More of a sort of property based test type thing as well where you're like, but you're talking about, or even
Ben Rady: Test where you're like, yes, exactly. Even a test where you're spinning up a database or something and doing a bunch. All of those kinds of tests. I write them very infrequently, first of all, and even if I did, I wouldn't run them in this way.
Matt Godbolt: Got it. This is very much, yeah, with you keep going. Yeah.
Ben Rady: So you've got hundreds of these small focused fast tests, and then the software gets bigger and bigger and bigger, and then you eventually have tens of thousands of these tests. Maybe you have 10,000 of these tests. And at that point, even with very well-written tests that you're sort of crossing a boundary there. I think we've talked about on the show before, sort of the rule of eights. Yes.
Matt Godbolt: One of the earlier episodes, in fact.
Ben Rady: Yes, exactly. And when you're in that early phase and you've got a hundred or so tests, they maybe they'll run in like 800 milliseconds and might as well be instantaneous for the purposes of developer workflow. And then when you start to get up into the thousands of tests, even if they're well-written, they start getting closer to that eight second range, maybe even 10 seconds, 20 seconds, and you stop being able to work interactively. So as you make the transition from 0.8 to eight, you can still work interactively. As you get beyond that phase, as you get beyond the 10,000 test phase, you stop being able to work interactively. Now you can put that off a little bit longer by running your tests in parallel. At that point, the overhead of distributing the work of the tests is worth it, because right now, if I run my tests in parallel on this project, it gets slower because the overhead of distributing them
Matt Godbolt: Across multiple is actually greater than the fact that each test takes less than a couple of hundred microseconds even. Right?
Matt Godbolt: That is very interesting.
Ben Rady: On the MacBooks, they take five seconds
Matt Godbolt: That
Ben Rady: This is basically doubling the lifetime that we get to live in this not quite honeymoon phase, but this very interactive, simplistic world where all of our tests run in a single thread. You run all of them on every change. They're nice and fast, and you get to work in this iterative style. And so we've been working on this project for about 15 months now, and basically by doing this, I'm kind of buying myself another 15 months,
Matt Godbolt: This, I guess. So you've doubled the speed unless you increase the size of the team and the output or whatever. If anything, you would've maybe asymptotically slowed down or something.
Ben Rady: Yeah. But then there's also, and this is especially true with Java, there's sort of a fixed overhead for running sort of any non-trivial number of tests. You're probably going to spend someone in the order of a second or two just loading all the classes and starting up the jvm,
Matt Godbolt: All those. And I suppose that's definitely worth saying. Right. So that was the thing I was going to ask you about is how are you able to take the tests that you are going to be running in our big fat infrastructure? That's all X 86 E is all big server class machines. Run it, first of all on your laptop natively, and secondly, then switch your laptop out for a completely different type of computer. And you've given it away there, you've moved to Java or you, this is all in
Ben Rady: J. Well, this project has always been
Matt Godbolt: In Java a long, long time ago. We had a big old discussion on this very podcast about when you should pick C++, when you should pick Java, when you should have Python or whatever. And I think although something like Python, which a lot of the code base that our company has is written in, is processor agnostic. Every library that Python uses is not agnostic. So we would probably have difficulty running, even if it was Python, the kind of quanti mathy python code that our folks write wouldn't run on a laptop without, sorry, an ARM-based laptop like the apples are. But Java just does it
Ben Rady: Seems. Yeah, right. Exactly. Yeah. All the work that I've been doing, I basically spent this whole week, I'm going to give away our time here, but we're sort of at the end of the year, things are a little slow. We've finished everything that we needed to do for the year, and I'm like, all right, I'm going to take this week. I'm going to move things over. And none of the Java code had to change at all. Actually, that's not true. There was one thing thing that I needed to change, which is some extended socket options that I had on a TCP socket, which were not supported on the Mac
Ben Rady: I think the nature of the problem that we're solving with this means that it's very unlikely that we're going to run into something like that because this project runs entirely in AWS and Google Cloud, first of all. So the whole concept of any sort of customized hardware is just like, that's just not a practical solution for us, period. And in addition to that, one of the things that I've certainly seen is it's not Conway's Law, but there's some sort of weird law or something like that where it's the operating system that you develop on affects the architecture of your system.
Matt Godbolt: Yeah, no, for real. And
Ben Rady: So if the model that we have, which we're going to have, is one where we are developing primarily on Macs, occasionally on Linux. And my intention here is to support both for development purposes, but if we're developing on Macs and deploying on Linux, then we're going to be very reluctant to change the architecture in a way that's not compatible with that model. It's not to say that we won't, but there's going to be a sort of pressure to be like, can you just design this so that it also runs okay on a Mac? And that means we're going to be doing, so a lot of what I've been doing this week has basically been taking our very sort of Ubuntu and Linux specific scripts and tools, Bash scripts and things like that, all the places where it's like, we've got some path that's got Linux dash 64 in it and doing dollar uname
Matt Godbolt: Dash P, whatever.
Ben Rady: Exactly. Stuff like that. And so once we've made that switch, it's sort of, and combined with a fact again that we're running in the cloud anyway, I think it would be very unlikely that we would wind up in a situation where it's like, oh yeah, we've got this weird proprietary binary thing that we have to plug into the middle of this.
Ben Rady: Yeah, absolutely. And I mean, for me, that is a really important thing to do is as a team decide these are the tools we're going to support as a team, and these are the tools that unfortunately just, I know that you're a Vim expert, but we're not going to be writing this Java code in Vim. Right, right. And I think it's a really important aspect of team cohesion is to be able to come together as a group and decide that and really be okay with it and mean it and commit to it.
Matt Godbolt: That's a really interesting aspect of this is that whole cohesion about a team that you get around a set of principles and be those principles as simple. And I realize now we're going off in a completely different direction here, but those principles are, do we vaguely agree on how to name things? Do we vaguely agree on the code format? Do we enforce it perhaps? Do we have bits that are in some formats if bits that or another, is that okay? Or do we want to unified stuff? Do we use the same tools? Do we allow people to choose the tool that's right for them where it makes sense or do we just say, no, you just have to use visual source safe for your code, source code and that can make or break a team?
Ben Rady: Oh, absolutely.
Ben Rady: Right? No, and I mean, this one is particularly easy. We had someone join our team back in October, and they had come in and they had gotten their standard issue MacBook, and they had installed in Intelij on it and all these other tools and clearly had used a Mac before and really liked it. And it's like, yeah, so show me how to load up the code and edit and IntelliJ. I'm like, oh, yeah, you need to do this on Linux. And I could see her face fall and be like, oh, okay. And then I was like, oh,
Matt Godbolt: So presumably this is again to give away what time we're recording. If we're releasing this in what March or whatever, that's a nice Christmas present for her to have congratulations and you can now edit on the machine that you are running on and use all your keyboard shortcuts and whatnot.
Ben Rady: Right, right. Yeah. So it's not a hard sell for me. And any of the other folks on my team are definitely like, oh yeah, no, this is great. And me too. Of course I worked this way before. It's going to take some time for my fingers to adjust to the keyboard shortcuts, but it'll all work. But yeah, I mean I think that whole aspect of team cohesion, we've talked about intersubjectivity on this podcast before. It's really hard to have intersubjectivity if you don't have the same tools. Obviously the build tools and the testing tools and those things are important. But also just like, oh yeah, when I was refactoring this thing, I extracted this thing over here and then I clicked this button and I did that, and someone might be like, oh, yeah, no, I wouldn't do it like that. It's like, well, why not? It's like, well, that would be really hard with my editor.
Ben Rady: Yeah, I mean I think the more of say cohesion, yeah, I value that a lot, honestly. And it's sort of like I really want the people on my team to agree on the tools that they're going to use and the techniques that they're going to use, which is not to say we can't change it. We're doing that this week, right?
Matt Godbolt: Yeah, for real.
Ben Rady: But if we're going to change, we all change together and we change for a reason that we all agree with and understand we don't necessarily all love, although in this case I think everyone kind of does, but being able to commit to it and be like, this is maybe slightly less optimal for me, but it's more optimal for the team and it's more optimal for the company and therefore I'm committed to doing it. And I think that kind of consistency and sort of commitment is one of the things that defines what a team is because they're all committed to succeeding and failing together. It's much harder to make that commitment when it's like, okay, well yeah, I sort of see in an abstract way how this helps the company, but at the end of the year, they're going to be looking at what I did and they're going to be looking at what these other people did, and they're going to be comparing them in different buckets. And it makes it really hard for me to be like, all right, I guess I'll just use this thing that I'm not very productive in and take the time and energy that it takes to
Matt Godbolt: Learn how to, that is a fascinating observation.
Ben Rady: So you need to have to have people who succeed and fail together in order to be able to have this cohesion. Otherwise, I find it to be very sort of difficult to maintain. But if you have it, then it's like someone joins my team and it's like when you're working in Java, you're going to be using IntelliJ full stop. There's no option here. That's how all the tools are set up. That's how all the techniques are set up. That's how the whole environment is set up. That's what you're going to use. And if you don't know how to use it, that's fine. You'll learn. Right. It
Matt Godbolt: Is. Yeah. No, that's cool. The whole team cohesion and succeeding or failing together is very interesting. A lot deeper than I was thinking when you were just basically, you said to me before we started recording, I got my Mac, let's talk about why I got my Mac, and then here we are. And I'm like, wow, I've had such a moment of what feels very affecting an insight into what makes a team is like, yeah, we're in it together and that means we either succeed together or we fail together and we only succeed if we choose JetBrains products. Is that what you're saying? We're not sponsored. Not sponsored by Jet, not sponsored by Jet Brains. Although you will find my ugly mug on the CLion.
Ben Rady: That's right. CLion Jet Brains is sponsored by you. That's
Matt Godbolt: Right. Oh dear. Yeah, that is funny. So anyway, you've got a Mac. I've got a Mac. They're impressive pieces of engineering.
Ben Rady: Oh, they really are. I honestly wish I knew more about what went into it in the architecture of the chip and everything
Matt Godbolt: Else. I do too. We have a mutual friend who is very secretive and works for Apple and keeps hinting at the amazing things that he's doing. I don't think it's anything to do with this, but it just gives you the idea of what level of thing can go down in a big company like Apple, the amount of engineering effort that they can focus on. Something like essentially ground rebuilding a processor. As far as I know, they license the architecture for arm, but nothing else. Essentially it's like getting the VHS compatibility or the DVD sticker that says this is A-D-V-D-D-V-D compatible, but you're like, I don't buy any of the parts from JVC or Toshiba or whoever. I've built one from scratch and literally from the component up, it's compatible with arm chips, but it's not made by arm. And I don't use any of their blueprints. I didn't license their blueprints, I didn't buy any of their pieces or their intellectual property. I just made my own arm chip. It's like the most bonkers thing to do. And that's because they believe strongly that they could do it better than ARM could.
Ben Rady: And I was going to ask you about this. Of all the people that I know, I figured you would be understand it a little better than anyone else that I could talk. Well,
Matt Godbolt: Yeah, but I just mean to say, I'm sure our listener is screaming into their headphones now going headphones, right?
Ben Rady: It doesn't work
Matt Godbolt: Like that. No, exactly.
Ben Rady: So I sort of was confused because when I heard that Apple was building their own processor, I also heard that it was an arm. Wait a second. Isn't there a company that does that? And so what you're saying is is that they basically license the OP codes
Ben Rady: That's incredible.
Matt Godbolt: And again, I've probably butchered that story as well. Again, comments welcomes to correct me in the appropriate places, but yeah, so ARM have always been efficient.
Ben Rady: Wow. I mean, I was going to say, one of the great things I've noticed about my MacBook is that the battery life is ridiculous and I had this, I also have an iPad, which is an M1 chip in it, and I think I've basically never charged that iPad. I bought it a year ago. I don't think I've
Matt Godbolt: Ever plugged into, it's been slowly 1% every three or four weeks goes down you, yeah,
Ben Rady: It's just fine. Apparently it's absorbing energy from the surrounding atmosphere, atmosphere or something.
Matt Godbolt: Maybe it, maybe it's like one of those really high-end that self winding by the very motion itself.
Ben Rady: Yes, it's incredibly, incredibly efficient and that was actually one of my complaints with the Dell laptop that I had is that it would really chew through the battery. I actually had a few times where I took it over a weekend, didn't plug it in, and I would come into work Monday morning and the battery would be dead and it had been sleeping the whole time, but even the sleep power draw was so much that it couldn't last
Matt Godbolt: For more than 48 hours. Yeah. Hours. I've never been that sure about how good Linux is at putting it into really into sleep mode as well. There's this delicate dance between the bios and the operating system for negotiating which bits of RAM to keep powered up and which bits not to and things like that. And then when to wake it back up again, how to wake it up. I remember having to go into a bios on one of my Lenovos and sort of have to put it into legacy.
Ben Rady: I did that too
Matt Godbolt: So that it could talk to Linux rather than Windows, which of course is the default for these things, so who knows who's really to blame there. But no, it's definitely the case that with my Lenovo, which I use for my personal stuff, that I'll often have it on my lap in the other room and then I will find my lap is on fire because somewhere is taking up all the CPU and I don't know what it is.
Ben Rady: Probably Chrome.
Matt Godbolt: Yeah, that is a fair point. Chrome, which is not really Chromes fault either. It's also the 330 tabs I have open, one of them is possibly doing something stupid
Ben Rady: Still in the Google Meet from two hours ago. That's
Matt Godbolt: Right.
Ben Rady: Yeah. It's like showing the conference room video and the audio. It's like, oh, right, I should probably quit
Matt Godbolt: That. It is worth just taking a moment, and we should probably stop in a second getting to time here, but I'm currently looking at you on a screen on a 4K monitor. You are several miles away from me, and Magic is both compressing the video on my side and sending it to you, decompressing your video that's been sent to me, then compositing it into a window on a 4K monitor. I have another 4K monitor next to me that has a waveform of this recording going on, and I know that if I pick up my mouse and grab the window, I can move it around essentially at a hundred frames a second. There's not any lag between the movement of the mouse and me moving this picture of you around. You just have to think how much data is being moved around whimsically by me wiggling the mouse by holding down the like, oh my gosh. I remember rendering an MP three audio file to an uncompressed format so I could listen to it because my computer wasn't fast enough to decode MP three, and yet now I get annoyed when there's like 18 people in a room in a meeting room and sometimes it drops out a bit or the fan comes on and I'm like, oh really? Do we have to, I'm decoding like 12 different video streams and compositing them and I'm running Chrome in the background. Oh, we are very blessed. We live in a blessed time for this kind of stuff.
Ben Rady: You could also say spoiled. That works
Matt Godbolt: Too. That works too. Yeah. Yeah.
Ben Rady: It's sort of a weird variation on risk compensation psychology, which is another thing we've talked about in the podcast. It's sort of the whole thing of as you have more safety mechanisms, people just use those and the level of safety that they actually achieve remains constant. They just do more dangerous things because they have
Matt Godbolt: A seatbelt on and now they drive faster. Right,
Ben Rady: And it's the same thing with computing power is the more of it you give people, the more of it they're going to use. Well,
Matt Godbolt: I mean, yeah, we definitely don't want to start a new topic now, but we had a dinner table conversation with my family the other day about exactly that with the idea all apps nowadays are chrome repackaged as a binary, running a JavaScript program that's communicating with a dom that's being rendered and then drawn back to the screen rather than using the operating system widgets that would do it natively, but it's convenient. Why would you spend all this time to squeeze everything down into 512 K of Ram or whatever one used to have to be able to do something when you're like, ah, just make an array of big enough. How big is big enough? I dunno, four gig. You don't really think about it. It's like, that's fine.
Ben Rady: Yeah. Yeah.
Matt Godbolt: Well, I guess we should call it at that point. Otherwise we're going to have the longest podcast ever. I guess it's not
Ben Rady: Too bad. This isn't too bad. I I'm going to go back to converting my B scripts calling you name in all the various places actually. So one of the interesting things that I think is going to fall out of this is I'm actually going to shift more stuff into AWS that used to run locally. Oh, interesting. Because I sort of have this problem now of, okay, if these things are running on the laptop, then as you were saying, we normally have this stuff very, it's running in a data center. The workstations are actually in a data center, and so you have nice internet connectivity and access to all these things and it's like, well, if I'm not going to have that because running on a laptop, then I'm going to be doing a lot more remote execution in AWS in lieu of doing it
Matt Godbolt: Locally rather than
Ben Rady: To pull that is much more local than it used to be. Yeah,
Matt Godbolt: Yeah, that makes sense.
Ben Rady: So I'm going to go back to that and hopefully get it done today.
Matt Godbolt: Well,
Ben Rady: Yeah. Cool.
Matt Godbolt: Good luck with that, Ben. And I guess I'll chat to you. I was going to say next time as if we don't talk to each other in between these podcasts.
Ben Rady: Yes.
Matt Godbolt: Well, as we've alluded to several times, it's about to be Christmas. We're at the end of the year here, so happy holidays to you and I'll see you in 2024 when this podcast comes out.
Ben Rady: Right, exactly.