Matt Godbolt: Hey, Ben.
Ben Rady: Hey Matt.
Matt Godbolt: This, as always, is our very uninspiring introduction. ah We say hey to each other, which is, I don't think we've never knowingly done that in person. I know that like other people will take the mickey out of us by going, "hey, Matt"
Ben Rady: Yeah.
Matt Godbolt: So you know I know that obviously it has become our trademark, but it is ah not intentional.
Ben Rady: Yeah.
Matt Godbolt: it was just You have to have something to say at the beginning of a podcast, it turns out.
Ben Rady: This was the best that we could do in episode zero and therefore.
Matt Godbolt: And it's the best that we could do. and all of them from then onwards, exactly.
Ben Rady: Right. Right.
Matt Godbolt: So we were talking because before this we arranged, ah ah well, before we we started talking, we we kind of joined a ah meeting and you were still doing your day job.
Matt Godbolt: And then you started telling a story and I'm like, wait a second.
Ben Rady: Mm-hmm.
Matt Godbolt: This sounds like a podcast episode. Don't tell me now. Let's tell me with the recording going. So you finished up what you were doing.
Ben Rady: Yeah.
Matt Godbolt: And then you had a name. Do you want to even say the name thing? Yeah.
Ben Rady: Yeah.
Matt Godbolt: So what is the name of this?
Ben Rady: It's, it's the name of this episode needs to be, "Is Football Possible Anymore?"
Ben Rady: And
Matt Godbolt: And I said, what on earth? Where's that from? What you said, I've got a story. And then I said, we should record this.
Ben Rady: yeah, yeah, yeah.
Matt Godbolt: And here we are. So dear listener, tell me, Ben, what the heck is this all about?
Ben Rady: So in the in the heady days of year or so before the financial crisis in 2008, I was working at a startup.
Ben Rady: And this startup was, it had like ChatGPT in its eyes. it like it like It saw it. It's like, oh, we could do a thing. But we didn't, the startup was they had nothing. we had We had nothing. We had no,
Matt Godbolt: this was 2007 you were saying era.
Ben Rady: two thousand and seven yeah yeah
Matt Godbolt: So yeah, neural nets things that people played around with and then abandoned at this point.
Ben Rady: um and well
Ben Rady: Right, right. And so like, you know, the glory of the LLM and the transformer and everything. I'm sure there were people that knew about them, but we were not some of those people. But what we did have was a I'm just going to call it a parlor trick.
Ben Rady: Uh, we, we had, we had ah a trick and it was like a patented trick. So, you know, we had something, but, um, but it was basically this, we were building the purpose of the startup was to build a chat bot for a database.
Matt Godbolt: oh
Ben Rady: And the idea here was, is that you could ask natural language, English questions, just like you would with ChatGPT, honestly, these days, but specifically and only about the data in your database.
Ben Rady: And then the way this worked is it would take your natural language questions and it would map them into SQL. And then it would execute that SQL. And then it would give you a response that it thought was appropriate based on the question that you asked and the shape of the data that came back.
Ben Rady: Right?
Matt Godbolt: Got it.
Ben Rady: And so it might generate a report for you or it might just tell you an answer. It's like the number of, you know, sales of this unit is 37 or it might do something else. Right? um But the intention was to...
Matt Godbolt: And so questions that people would ask would be like, yeah, give me a summary of the last three months of sales, that kind of thing.
Ben Rady: Right. Yeah, exactly.
Matt Godbolt: Right. So you're very ChatGPT-ish, you know, the kind of thing I see people put into AI all the time.
Ben Rady: Yes.
Matt Godbolt: Yeah.
Ben Rady: Oh, yeah, yeah, yeah, yeah. It was just very narrow, right? It was just it was just like only about a you know a database. And the idea was we're going to build a tool. You could point it at any database. It would ingest the schema. It would cache some of the data.
Ben Rady: And it would sort of build up the semantic model, which it could then use to process your questions, right?
Matt Godbolt: Got it. So there was like a pre-step where you pointed it at a database and it did some investigations first and foremost. And then that's what you use to like build your parlour trick list of ah whatever snippets of SQL that you presumably behind the scenes were were part of this.
Ben Rady: Yeah. Yeah. I mean, and it was, you know, the, the, the basic idea behind it was that you'd have like this concept map, And so the concept map just related to databases, tables, and columns within the database, right?
Ben Rady: So it's like, if you're talking about sales, well, then you're talking about this table. And sometimes it was as simple as literally the name of the table is like YearlySales, but also sometimes there were some things where you had to teach it a little bit and, you know, you get a little, maybe a DBA involved and you say like, this is where we keep our customers.
Matt Godbolt: And by teach it, you mean hard code, presumably in a big if statement and regular expressions.
Ben Rady: Well, in the early days, it was that. Eventually, we got it to be smarter than that. But this was this was sort of the project. And I worked on this for about a year, I guess it was. don't even remember now. And so because it was so constrained,
Ben Rady: It had a tendency to hallucinate. I mean, we didn't think of it as hallucinating, but sort of retroactively now you'd be like, that's exactly what you would call it as hallucinating, where it would like reach for information that there's no way it could possibly even have and try to like solve for it and then give you something which was like nonsense, right?
Ben Rady: Right. And so as engineers working on this, you know, we were concerned with this problem because we didn't want it to like purport to be something that it wasn't to have information and it didn't have.
Ben Rady: And so one of our favorite pastimes was to, you know, grab a couple of beers and then ask it stupid questions. And ah The sort of canonical version of this, the the very first one that you would always ask in one of these sessions is, "Is football possible anymore?"
Ben Rady: ah Because that is not a question that even makes sense or is answerable by information in a database or it's like, what do you even mean? And so what we were hoping to see is some sort of informative error message saying like, I don't understand what you just said because it doesn't make any sense.
Matt Godbolt: um I'm really sorry that I'm going to be now typing that directly into Claude 4.0 Opus, the best possible thing. (typing) "Is football possible anymore?"
Matt Godbolt: Is that the question?
Ben Rady: That is it. Is football possible anymore?
Matt Godbolt: Okay. And for what I think your cell phone is on the desk. Every time it buzzes, it the microphone picks it up super loud.
Ben Rady: Oh.
Matt Godbolt: So...
Ben Rady: Oh, no. i would I naively and stupidly used my phone as a microphone rest, which is, that's a terrible idea.
Matt Godbolt: Oh, right. Hence the loudness of that.
Ben Rady: I don't know why anyone would do that. I'm just going to replace that with some DVDs, which are sitting on my desk.
Matt Godbolt: Oh, my gosh. Okay.
Ben Rady: And those are definitely not doing anything, so.
Matt Godbolt: So, Claude is is thinking a lot more. It's gone onto the internet now.
Ben Rady: Oh boy.
Matt Godbolt: And and ah it says, "Ah, you're asking if football is still possible. Bit of a loaded question, that. From what I'm seeing in the search results, football is very much alive and kicking in 2025, although it's facing some interesting challenges, depending on which side of the pond you're on."
Matt Godbolt: Okay. So, apparently... Claude can deliver some kind of question, answer to this question.
Ben Rady: Yes.
Matt Godbolt: but But if for example, I'd given it just only the ah the marketing results for last month's widget sales, I might reasonably expect it to say that's not a question I can answer to right now.
Ben Rady: Yes. Yes. um Yeah. And I mean, you know, the the the interesting thing about that that startup was, ah there's actually a couple of interesting things about that startup.
Ben Rady: One is, is that is maybe sort of like the most kind of like by the book,
Matt Godbolt: Mm-hmm.
Ben Rady: extreme programming team that I have ever been on. Right. Everyone on that team worked, had worked in consultancies.
Ben Rady: I think there were a lot of thought workers there. There were some folks from, ah oh man, what is the name of that company?
Matt Godbolt: The usual suspects, yeah.
Ben Rady: i don't know what it doesn't matter, but they, they all, they all kind of knew what they were doing when it came to things like test-driven development, continuous integration, bunch of other things.
Matt Godbolt: That kind of extreme programming, not the one where you climb a mountain and with a laptop.
Ben Rady: And, you know, again, this is like, yes, Right.
Ben Rady: Red Bull. There's a lot of Red Bull involved in that one. um And so like we had a group of people that were all very like-minded, that all kind of knew the same practices and knew them very well.
Ben Rady: And this yeah that was the core engineering team. That was like the only engineering team in the startup. And it grew to about eight people at the, maybe nine even, at the biggest...
Matt Godbolt: Gosh.
Ben Rady: and And that is like the biggest, that is my example in my my career of like, that's as big as I think you can make a team. If you have nine people that all are working in the same codebase, solving the same problem, and they all know these practices really, really well, and they all agree on what all the practices are, yes.
Matt Godbolt: Right. no Super aligned on the way to do things as well as what you're trying to do.
Ben Rady: Yes, yes. and And we managed to do it with nine people. And even at nine, it was like straining at the edges, right? um But that was, as that's I think, about as big as you can make a team. Before you start suffering, um you're just making it worse by adding more people.
Matt Godbolt: Right. I'm surprised that it got to nine. Again, you've you've qualified it with all of these extra like alignment and you know skilled people at the the top of the...
Ben Rady: Yeah.
Matt Godbolt: you know ah with the same kind of mindset for approaching problems. But just in terms of the the the number of pairs of people to talk between, ah you know that's what, nine times eight?
Ben Rady: Mm-hmm.
Ben Rady: Right.
Matt Godbolt: What are eight nines? I don't know. Over two. That. `nm - 1`... Well
Ben Rady: yeah.
Matt Godbolt: It's a lot of people. It's a lot of possible...
Ben Rady: It's a lot, a lot of interactions. Yeah.
Matt Godbolt: A lot of interactions, but but you're able to make it work.
Ben Rady: Uh-huh. Yeah.
Matt Godbolt: And um I mean, by it work, I mean the the team work.
Ben Rady: Yeah.
Matt Godbolt: But clearly, because you youre you are not um currently talking to me from your private yacht, the company perhaps did not make it.
Ben Rady: The team was functional. Yes.
Ben Rady: Right. The IPO'd,
Matt Godbolt: what What was the story there? are or you Yeah.
Ben Rady: Oh yeah, no, actually, so this is a somewhat personal story. I don't know how much of this I want to share on the podcast, but the punchline of this story was basically that um we were, ah my wife was pregnant with our youngest at the time.
Matt Godbolt: Sure. No, that's fair. Yeah.
Ben Rady: And on ah Friday, March 13th, of it would have been 2008. I know I'm going to screw up these dates and it's going to be recorded for all time on this podcast. ah Friday, March 13th, we were finding out whether we were going to get another round of funding.
Ben Rady: And my wife went into labor, turned out to be false labor. ah But after a long day at the hospital and a lot of other things, ah called my boss that evening and said, hey man, do we still have a company? And he said, nooooo...
Matt Godbolt: oh
Ben Rady: Because our our and our primary investor had decided to take all the money that he was going to put into the second round and the third round, or don't even remember what it was, and put it into real estate instead because real estate was falling apart and it was super cheap.
Matt Godbolt: And went to Vegas.
Ben Rady: And I'm like, you know what? I don't disagree with what you're doing right here.
Matt Godbolt: Yeah.
Ben Rady: I wish it would be different, but that's not how that turns out.
Matt Godbolt: No. Well, that is how these things go sometimes. You know, that's what the the risk of startups
Ben Rady: Yeah.
Matt Godbolt: ah
Ben Rady: Yeah. That is sometimes that goes.
Matt Godbolt: But you got a good story out of it.
Ben Rady: I did. i did.
Matt Godbolt: To what extent can you talk about how it actually worked, pre-LLM? I'm intrigued, you know, expert model, was it, you know, held together with regular expressions and and text snippets and and, "hey, we know our three customers are X, Y and Z, and so therefore they will talk about this, these will people will talk about this other thing", and then we just sort of, yeah, what was it?
Ben Rady: Right. ah
Ben Rady: Yeah. Well, the parlor trick essentially was if you take the sort of English concepts in a typical sentence, And you can you can map those concepts to structures in the database, which is like this, again, this whole concept mapping of it needed to like the information you needed to seed it with. And it could with did a decent job of finding this stuff out for itself, I think. But the information you needed to seed it with was what tables and columns and other things map to what concepts.
Ben Rady: And if you could if you did that, then it's actually like a pretty limited search space of like, take this English sentence and map it into this set of related concepts because, yeah.
Matt Godbolt: I see. So like you say, it's a trick. So you start from the database rather than starting from the human and say, what could they possibly mean? You go like from the database, what things could make sense to ask of this database?
Ben Rady: Right.
Matt Godbolt: And then you, yeah, like, like all these, as you say, parlour trick, but you know, it's right now.
Ben Rady: Right.
Matt Godbolt: That's a neat way of doing it. Um,
Ben Rady: Yeah. But yeah, these days it's just like, I mean, this is why i have i have zero fear that somebody is going to be like, you gave away our secrets. I'm like, dude, have you seen have you seen what Claude can do like, come on, man.
Matt Godbolt: Yep. Although I will say there's, you know, obviously there's a lot of, interesting discussion as we've had on this podcast. And I definitely don't want to ah revisit that just this minute, but you know, uh, the power consumption alone of your parlour trick, uh,
Ben Rady: Well, that's fair, maybe, perhaps.
Matt Godbolt: fits onto microprocessor on a cell phone. ah It doesn't require a datacentre's worth of um heavy ah compute equipment. um So there's something to be said for that, for sure.
Matt Godbolt: But no, that's fascinating.
Ben Rady: Mm-hmm.
Matt Godbolt: But like how can I ask, how did this question come into being? What was the forming for this? Because it's set for a minute, when you first, ah i going to be honest with you, when you first said it to me, I thought, oh gosh, is this a quote from like Ted Lasso?
Matt Godbolt: I haven't seen that far in the series yet. And have I just missed it? Because I obviously think of football as being the one where you use your foot and the ball, not "hand egg".
Ben Rady: Yes. Not hand egg. I mean, it works for both really.
Matt Godbolt: But I presume it was meant in the... I mean, it does. It does. and obviously, Claude picked up on that when we asked earlier. It's like, which side of the pond are you from?
Ben Rady: Yeah. Yeah.
Matt Godbolt: But um yeah, i do you remember how it came about?
Ben Rady: Uh, huh. How did this come into being? I remember this coming into being ah from one of the engineers on our team, ah Paul, who ah was ah an expert Scrabble player.
Ben Rady: and kind of like a, he was just really into words. Like he he just he just really, and like, and I think this startup for him was kind of like really cool. And then it was like, we're doing a bunch of natural language processing and like all this concept mapping.
Matt Godbolt: Yeah, yeah. yeah yeah
Ben Rady: And he he just really loved it. And he had way with words, I guess I would maybe say, not the way with words that you would normally associate with that phrase, but like, yeah.
Matt Godbolt: ha
Ben Rady: um And um my memory of this, which could be completely false, is that he he came up with that phrase as a way to kind of poke at this thing that we were building.
Ben Rady: And then we're like, oh my God, that's the most brilliant thing I've...
Matt Godbolt: Right. Drawing on his sort of like deeper understanding of the things that you were doing with the natural language and how it was processed, he was like, well, what would trip this up the worst?
Ben Rady: Yeah.
Matt Godbolt: What is the unit test for this thing that exercises the worst possible behavior?
Ben Rady: Yes. Yes.
Matt Godbolt: Because there's no concept in this sentence that maps to anything remotely like database.
Ben Rady: Right.
Matt Godbolt: so it should correctly say, yeah.
Ben Rady: Yeah. And, and I, I think it, I think it actually sort of works on two levels too, which is this like, it's sort of a nonsensical question regardless of your dataset.
Matt Godbolt: yeah
Ben Rady: Right. um Like, What??? And then if you add in the additional constraint of it's going to have to take this and map it to a schema, that's probably about like, you know, employees or sales records or some other sort of mundane thing that is probably not sports.
Ben Rady: Although I do remember baseball being one of the data sets like...
Matt Godbolt: I was going to say there's a danger that with the right kind of database, this might actually make sense.
Ben Rady: Yeah. i feel like we had some MLB database that we would test that thing with and you could ask it like
Matt Godbolt: You know, like you could look at sales, sales cost, ah sorry, incoming ticket revenue versus cost of running a game. And then is football possible?
Matt Godbolt: No, yeah we we lose money on every game.
Ben Rady: No, definitely not possible.
Matt Godbolt: Yeah. ah
Ben Rady: No, I actually, I hadn't thought about this until just now, but I seem to recall that we had this, it was like player statistics from the MLB. And it was like, you know, it's like the color commentator in a baseball game. It's like, oh, this...
Ben Rady: pitcher has has a ERA of 1.3. (I'm just making up numbers. i don't know anything about baseball) against, you know, right handed batters in ah the East Coast over the last four years.
Ben Rady: And it's like, how does anyone know these things?
Matt Godbolt: right it's
Ben Rady: And so we were like putting that guy out of a job by building this thing that could take, you know, the the MLB stats and give you like all kinds of crazy things based off of it.
Matt Godbolt: Right, like, did you know that this is the highest scoring game on a Thursday where it's raining since blah...?
Ben Rady: Right, right.
Matt Godbolt: And you're like, what? Yeah, when people come up with this nonsense, yeah, you think there there is definitely, there's ah there's got to be a market for it.
Ben Rady: Yeah.
Matt Godbolt: So, yeah, you've put but you well, you were trying to put that person out of a job. but Evidently, you perhaps...
Ben Rady: We failed. So they still have a job and it's everything is fine.
Matt Godbolt: job is safe for a little longer. Everything's fine.
Matt Godbolt: Well, now you've explained to me that football may or may not be possible anymore. and ah
Ben Rady: Mm-hmm.
Matt Godbolt: Yeah, I don't know quite where to take it from here now, other than maybe that's... that's Is that the end of what we got here? I think it might be.
Ben Rady: I think it might be. I mean, you know, if there's, if there's a takeaway from all of this, it's, uh, you're probably not going to get things right the first time.
Matt Godbolt: Yeah.
Ben Rady: That's sort of like the way that I think about it is back in 2007, we were building this thing we were like, this going to be really cool all that stuff. But we also knew that it was a startup and this was iteration one and it's like, you know okay, the the chances of this actually taking off are not high.
Ben Rady: We didn't really expect the ah Spanish Inquisition...
Matt Godbolt: Well, nobody does. I mean, obviously.
Ben Rady: We didn't really expect the financial crisis to come and smack us in the face. ah But even without that, it's sort of like when you're building new technology, you know, I think we've said on this podcast before, nothing new ever works.
Matt Godbolt: Yeah.
Ben Rady: The way to get something working is to get the new off of it as quickly as you possibly can, right?
Matt Godbolt: Yeah.
Ben Rady: um And so, you know, I think when you you could you could take away a lesson of like when you're building technology like this, it's speculative, it's probably not going to work, but you will learn things: And when you learn things, then version two can be a little bit better.
Ben Rady: And you repeat that process over and over and over again. And then eventually you get things like Claude answering like, well, depending on what side of the pond you're on it may or may not be possible.
Matt Godbolt: Gosh. Well, that seems like a decent conclusion to this. I was wondering where we were going to go with it, actually. you know like i mean Given our, you know again, comprehensive planning, um this seems like a good as place we need to leave it.
Ben Rady: Mm-hmm.
Matt Godbolt: Until next time.
Ben Rady: I would say so. Till next time.