Matt Godbolt: Hey, Ben.
Ben Rady: Hey, Matt.
Matt Godbolt: I can't believe we're laughing already. Ben, very, very, I don't know what the right word would be, but like, you've been auspiciously auspicious.
Ben Rady: Auspiciously?
Matt Godbolt: That's not even, though that's not the right word, but you took your microphone and kind of bent it out of the way so we could have a nice talk, and your headset microphone isn't in the way.
Ben Rady: Yeah, right.
Matt Godbolt: Oh gosh, it's too early in the week to be recording a podcast, and yet here we are.
Ben Rady: You mean Friday? It's...
Matt Godbolt: Yeah, I guess. I don't know, now everyone knows when we record these.
Ben Rady: There's not a lot of time left, my man.
Matt Godbolt: That's true. Somehow we're in May as well, which, this podcast will go out in May as well, because it has to.
Ben Rady: Yeah. Yeah, right, right.
Matt Godbolt: Otherwise, we're in trouble.
Matt Godbolt: We've not got any in the bank.
Ben Rady: Okay.
Matt Godbolt: Yeah, so what are we talking about today, Ben?
Ben Rady: I thought we could do at least one podcast, and maybe we'll do this again, on kind of like how computers actually work. And I think a way in which to cover the extremely broad topic of how computers actually work is to look at some of the maybe non-intuitive behaviors that they have.
Matt Godbolt: I love it.
Matt Godbolt: Oh.
Ben Rady: Things that, you know, people kind of take for granted as sort of like an abstraction that you rely on.
Ben Rady: And, you know, peel back the covers a little bit and explore how it works. And maybe even, if we get to it, some like non-obvious ways in which it breaks.
Ben Rady: Right. And like when it doesn't work, what is actually happening? Because I think that sort of understanding the failure modes of these things is a great way to understand what's actually going on.
Ben Rady: So that's my general idea.
Matt Godbolt: For certain, yeah. OK, so we've got like this sort of general idea of the theme of how computers really work, which obviously is very close to my heart as well. So this sounds brilliant.
Matt Godbolt: And then we were discussing some things before we started, and you know, shooting around ideas. And I sort of suggested the, there used to be like an old Google interview question, which was, what happens, you know, you type google.com into a browser, you hit enter, what happens?
Matt Godbolt: And you know, I always liked that because there's almost any number of ways that you can go, but how about we simplify that bit?
Matt Godbolt: What happens if I have curl rather than a whole web browser there, and I curl some simple URL, right?
Ben Rady: Right.
Matt Godbolt: Not, well, not JavaScript or anything. Well, I guess it doesn't matter, but yeah.
Ben Rady: I mean, could be JavaScript. A document.
Matt Godbolt: I guess so.
Ben Rady: You're fetching a document with curl.
Matt Godbolt: It doesn't even matter at that point, because it'll just print it out in front of you.
Ben Rady: That's right.
Matt Godbolt: Yeah.
Matt Godbolt: So, so yeah. When should we try something like that? What happens if we curled google.com? In fact, I did this only earlier today as I was playing around with sandboxes.
Ben Rady: Yeah.
Ben Rady: Oh, perfect.
Matt Godbolt: And I curled exactly HTTPS google.com. And I know exactly what it returns.
Ben Rady: Can I, can I offer a guess before you tell me the answer?
Matt Godbolt: You can offer away.
Ben Rady: Yes. Does it give you a redirect? Something in the HTTP 300 range would be my guess.
Matt Godbolt: It does. It tells me the document has moved, and it tells me the document has moved to www.google.com, which was like, oh, really?
Ben Rady: Yes, okay.
Matt Godbolt: Yeah, I guess because I was very lazy.
Ben Rady: Yeah, so how does it tell you that?
Matt Godbolt: Well, in this instance, the curl told me just a document. It printed out a document in front of me that said 300 or whatever it was, moved, document moved.
Matt Godbolt: And then it had a tiny little bit of HTML that came to me, because I didn't have, you know, dash D dash or whatever the thing is to print out all the headers and stuff like that. But no.
Ben Rady: Okay. Okay. So you didn't see, so this was going to be my question. My understanding of this is, whenever you request a document over HTTP or HTTPS, you're getting two things back: you're getting the headers and you're getting the document, the document body.
Matt Godbolt: Aha.
Matt Godbolt: Yeah.
Ben Rady: Right. And so when it was redirecting you, all you were seeing there was the document body, which was some HTML. Is that correct?
Matt Godbolt: Correct. A very small piece of HTML, which you would normally never see. You know, a browser would render that, really because the real meat of it was in the headers, right?
Ben Rady: Mm-hmm. Mm-hmm. It would just redirect you. Yeah.
Ben Rady: Yeah. Okay. So then when you do that, and I'm just kind of going through, I'm trying to like answer the interview question here a little bit.
Matt Godbolt: That's right, yeah.
Ben Rady: So when you, or maybe I'm interviewing you. I don't even know how this is supposed to work.
Matt Godbolt: I don't know which way around this is anymore, because I thought it was going to go the other way and now we seem to have flipped it. Yeah, so carry on.
Ben Rady: I don't know who's, who's, I mean, you know, it's a collaboration.
Matt Godbolt: What even job is this?
Ben Rady: We're collaborating here. No one knows.
Matt Godbolt: What are the benefits anymore?
Ben Rady: We don't have jobs anymore. They're all taken by robots.
Matt Godbolt: Oh, yeah, that's true.
Ben Rady: So when you make this request, you're going to get back headers and you're going to get back the body. Like, how is the request delivered and how exactly is the body returned? Like, what protocol is it using to do that?
Matt Godbolt: Right, right. Well, I mean, I know that the particular command I was using is absolutely HTTP slash 1.1.
Ben Rady: Mm-hmm.
Matt Godbolt: And I mean, this is one of those cool things. I know that maybe this is showing our age a little bit, or certainly my age. I think I've got a few years on you. But at university in the late 90s, in the mid 90s, darn, you know, learning that you could Telnet to a port on a machine and just type stuff in.
Matt Godbolt: And get them to do things. So like, you know, the most happy day was to discover how to send emails from other people. I don't recommend that folks try this anymore. I don't think it works anymore, but you used to be able to Telnet to the mail port and give it just raw commands.
Ben Rady: Yeah.
Matt Godbolt: And it would be like, very trustingly, say, sure.
Ben Rady: Right,
Matt Godbolt: that you're god@universe.com, fine, send your emails please, right? So you kind of got this idea that everything was a text-based thing. And I remember this early web thing, and this was HTTP one, so you would just literally Telnet to port 80 of some machine and type get space slash and hit enter, and then you'd get the document back. No headers, no nothing. It was just, that was the whole thing.
Ben Rady: Right. Right.
Matt Godbolt: But nowadays, HTTP 1.1, we have to tell it more information. So you are going to connect to a TCP port on the machine, usually port 80. Sorry, 443 nowadays.
Matt Godbolt: Everything's HTTPS. So for the purposes of talking about the protocol, at least this level of the protocol, HTTP, let's ignore SSL for now.
Ben Rady: Yes.
Ben Rady: For HTTPS it's 443.
Matt Godbolt: Let's see if we can get to it in time.
Ben Rady: Right.
Matt Godbolt: But I don't know.
Ben Rady: Right. Like Tom said in his most recent video.
Matt Godbolt: Correct. And I'm so sorry, this is my family coming home in the background, and we thought we'd beat them, but no. So my dog is announcing their arrival. Much as you announce your arrival at the HTTP server by asking it...
Ben Rady: Yeah.
Matt Godbolt: Whoa, segue there.
Ben Rady: Meta.
Matt Godbolt: So get, it's still a verb, like get or put or post. And most of these things would be like get. Then the URL, rather, the document locator on that machine itself, and I forget the exact terminology for it, but that's the bit after the host name,
Ben Rady: Yeah.
Matt Godbolt: but before the hash, if there's one in your URL. So like in Google, it will probably just be slash, forward slash, if it was your Google, because that's the only thing you've got.
Ben Rady: Mm-hmm.
Matt Godbolt: If you're going to google.com, then you put a space and then you tell it that it's HTTP slash 1.1, which tells it, hey, I'm talking to you in the new way, the new new way.
Ben Rady: Mm-hmm. Mm-hmm.
Matt Godbolt: I mean, there's QUIC and there's HTTP 2 and there's all other things, but like, for simplicity, let's stick with this. Then you're expected to send a bunch of headers. Each header is a bunch of ASCII bytes followed by a colon, followed by a space, I think. Maybe the space is optional. And then some other stuff and the new line.
Ben Rady: Yeah.
Matt Godbolt: So it's pretty human readable, as these things go. And the one thing that you will be most required, most required, I don't know if it can be more or less required, but...
Ben Rady: The utmost requirement.
Matt Godbolt: The highest requirement is to put host colon and say, actually, what host (again, my dog in the background, you'll have to apologize), but what host I'm actually requesting from. Because this was like the big unlocker, I think, in like the early aughts, whenever this thing came out, which allowed you to have more than one website per IP address.
Ben Rady: Right, right.
Matt Godbolt: Because if you just do get slash, you're like, well, which website did you want? I don't know, right? It's whatever.
Ben Rady: Mm-hmm. Mm-hmm.
Matt Godbolt: But that was the way to solve it. You know, maybe it would have made more sense to do get HTTP colon blah and just give it the whole URL and say, you work it out. But anyway.
Ben Rady: Maybe.
Matt Godbolt: But host colon google.com is what we would have said. And then, I mean, from my own personal experience of Telnetting to google.com to see how it works, I know that all you need to do to get it to work at all is to do get slash HTTP 1.1, host colon google.com return, and then a blank line that says, I'm done with the headers now.
Ben Rady: Mm-hmm.
Matt Godbolt: And then you get the document that says, no, you meant, surely you meant www.google.com. You're like, ah,
Ben Rady: Right, right.
Matt Godbolt: but nevertheless, connectivity is there. And then it will close the connection, I believe. So that's what I think would happen if I'm doing it manually myself with Netcat or Telnet or something of that nature.
Matt Godbolt: What am I missing? What's next?
Ben Rady: No, that's good. And I mean, I love doing that exercise when kind of explaining HTTP to people, because it takes the magic out of it. Now, you introduce a whole lot more magic when you add in like TLS and a bunch of WebSockets and other things like that.
Matt Godbolt: There's a fantastic quote by one of our friends. It was to do with optimization, as it happens, but, you mentioned magic and I wanted to sort of bend it to this, which is that he said, you know, like, HTTP is like magic.
Matt Godbolt: And you're like, well, it isn't. It's just like a bunch of ASCII being sent over it.
Ben Rady: It's just text.
Matt Godbolt: Yeah. And then...
Ben Rady: It's words.
Matt Godbolt: And that is like magic, because if you actually show people how it's done, it's kind of disappointing afterwards. You're like, oh, it's just, oh, I see. Yeah, there's just two of them. Like, oh, yeah.
Ben Rady: The rabbit was already in the hat.
Matt Godbolt: Yeah.
Ben Rady: It was there the whole time.
Matt Godbolt: That's right. It was there the whole time.
Ben Rady: It's not special.
Matt Godbolt: Oh.
Ben Rady: It's not special. It's not a special hat.
Matt Godbolt: It's not special.
Ben Rady: It's not a special rabbit. It's just, the rabbit was in there the whole time.
Matt Godbolt: It was there the whole time. Exactly. But I love that feeling of, like, magic is that. That's what magic means, really.
Ben Rady: Yeah, yeah, yeah.
Matt Godbolt: It's disappointing when you know how it's done, actually.
Ben Rady: Right, right.
Matt Godbolt: But anyway, not disappointing, because now you know how it's done. You can go, oh cool, this is something I can build on and I understand. I can debug. And so on.
Ben Rady: Mm-hmm.
Matt Godbolt: But this obviously is not curl.
Ben Rady: Mm-hmm.
Matt Godbolt: This is just me doing it manually. And you do it manually.
Ben Rady: Right. So when curl is doing this, in the base case, when it just works as expected, it is, yes,
Matt Godbolt: Bass. Not, not that kind of bass.
Ben Rady: It is opening a TCP connection to whatever IP address was resolved from the host name that you gave it.
Matt Godbolt: No treble.
Matt Godbolt: Yes.
Ben Rady: It is sending those, that initial request with the HTTP verb, as it's sometimes called, the get, the post, the put, whatever it is.
Ben Rady: It's sending all the headers that it might send. And curl sends a lot of headers on your behalf, right? You don't, you can tell it to send specific headers, but you can also not. I just want this document, please.
Ben Rady: And it will add in whatever headers it thinks are appropriate, including a user agent.
Matt Godbolt: For example, like the user agent, yeah, hey, I'm curl this version and stuff like that.
Ben Rady: Yep, yep.
Matt Godbolt: And what type of documents they'll accept, I think, is a list of things.
Ben Rady: Mm-hmm.
Matt Godbolt: And then potentially something to do with the connection, in terms of closing the connection or keeping the connection open for further requests, which, yeah.
Ben Rady: Mm-hmm. Okay. One, side note here, one of the interesting ways that you can detect which client is actually connecting to you... because the user agent is supposed to say, but obviously that's like kind of voluntary.
Matt Godbolt: You can change that, yeah.
Ben Rady: You can put whatever you want in there. But one of the interesting things that you can do is you can look at the order of the headers, because the order of the headers is not specified in the protocol.
Ben Rady: So it can be whatever you want it to be.
Matt Godbolt: Oh, but it's a sort of, it's a fingerprint for the client code that connected.
Ben Rady: Yes, certain clients tend to put the headers in certain orders, and depending on what client you're using, that can be an indication of what it actually is.
Matt Godbolt: Oh my, that's sneaky.
Ben Rady: Yes.
Matt Godbolt: Ben, you are a sneaky man.
Ben Rady: Right.
Matt Godbolt: What is, is the sort of like fingerprinting based on the fact that, yeah, well, you know, curl always sends them in this order, and Chrome's fetcher does this and Firefox does this?
Ben Rady: Right.
Matt Godbolt: So it doesn't matter how you configure it. It's, you know, to say like, lie, user agent liar, is like, yeah, but we know you're Chrome really.
Ben Rady: Yeah, yeah. Similarly with like, you know, as you said, like spaces in between the headers and the values, again, optional, and you can do it either way.
Matt Godbolt: Right.
Ben Rady: And that creates, also the casing on the values is another thing that can create sort of a fingerprint here. So curl is choosing to do it the way that curl does it. And even if you change the user agent to be like, oh, no, actually I'm Firefox,
Ben Rady: it's possible that somebody else might be able to figure that out. But the header order is not actually really, and I might be lying here, but I think it is specifically not important.
Ben Rady: Like, the order is completely unspecified.
Matt Godbolt: I think it is unspecified which sequence they're in. Yeah, I think so. But yeah, reader, dear reader, listener, sorry, not reader, gosh. Listen, but again, it's, it's been a week.
Ben Rady: Yeah.
Matt Godbolt: You know, check what we're saying here, but this is just two old farts comparing notes on what they remember about how this works.
Ben Rady: Right, right, right. And, well, and as you were saying before, oftentimes web servers are very tolerant of any kind of, you know, sort of garbage they may receive. And I think you said that you literally were just doing get slash on Google earlier today with no protocol.
Matt Godbolt: No, this one had a, yeah, no, this one did have a get slash.
Ben Rady: Okay.
Matt Godbolt: No, this was still using curl. I didn't actually Telnet today.
Ben Rady: Okay.
Matt Godbolt: But like, I think nowadays, if you do get space slash, it will tell you to go away, because that's HTTP 0.9 or whatever, the one that didn't even have the version in it, pre-versioning.
Ben Rady: Yeah.
Ben Rady: Yeah, the unversioned version. Yeah, right.
Matt Godbolt: Yeah, something like that.
Ben Rady: Exactly. So yeah, so those headers, you're sending those headers whether you know curl is doing it for you or not. You can obviously be explicit about it. But if you don't do anything, it'll do it for you. And then, depending on the type of verb you choose, the request itself may also have a body.
Matt Godbolt: Oh, yes, that's a good point.
Ben Rady: And this is particularly important for put and post and probably patch, but you can also do it with get.
Matt Godbolt: And patch, I think, is another one I'm thinking of.
Matt Godbolt: Oh.
Ben Rady: You can, I'm pretty sure that you can include a request body with get.
Matt Godbolt: I think one of the headers, one of the header fields, is content length. Is that right? Is that the thing that gates this?
Ben Rady: Ooh.
Matt Godbolt: And then you tell it how much you're going to be sending it or receiving. You know, certainly the response has a content length that says, this is how long it's going to be, or it can do.
Ben Rady: Yeah, yes.
Matt Godbolt: But if you're going to do anything that requires you uploading, you need to tell it where the end of your thing is, your data.
Ben Rady: I think you're right. I think you're right. Which is why, by the way, it is particularly difficult, if not impossible, to stream the body of a request.
Matt Godbolt: Uh-huh.
Ben Rady: Like, if you have some input stream that you're reading from and you're like, I don't know how many bytes are in this stream, I'm just, you know, reading it, by the time you're writing the body, you should have already written the header that says the length.
Matt Godbolt: Yes.
Ben Rady: So you have to consume the whole stream in order to figure out how many bytes you're going to send.
Matt Godbolt: Which is, I mean, it's fine if you're serving static content off of a disk, where you're like streaming it from disk and you can go, how long is this file?
Ben Rady: So you can put that in the header.
Ben Rady: Right.
Matt Godbolt: And it's immutable. It's not going to change on me. Okay. It's two terabytes. Cool. Content length, two terabytes. Here it comes. But yeah, if it's being streamed by a process for some reason, it's like outputting, you know, you're catting a file or whatever, something ungodly.
Matt Godbolt: Don't do this in the web.
Ben Rady: Or another socket that you're reading from, right? Like you're trying to act as a proxy perhaps.
Matt Godbolt: Right, then you don't know ahead of time how long the thing is going to be to write it into the header, because you've already sent it. And I mean, anyone who's written web server software has almost certainly had some kind of exception thrown when they've tried to do something silly in the body handler, where it's like, no, you can't do that because the header has already been sent.
Ben Rady: Mm-hmm.
Matt Godbolt: Like, you know, this is, forget, I've definitely hit it with, oh, crikey, I can't think of the name. But in some cases where you're trying to do something, it's like, no, look, I've already sent the header.
Matt Godbolt: I can't now mutate this. I can't like turn it into a streaming request. I can't turn it into a different size request because we've committed. We're done now. You just have to send text now.
Ben Rady: Mm-hmm. Mm-hmm.
Matt Godbolt: Sorry about that.
Ben Rady: Mm-hmm.
Matt Godbolt: Yeah, there is multi-part though, which is a pretty crazy hack.
Ben Rady: Right, right, multi-part uploads. If you've ever used like an object store that supports HTTP like Amazon S3 and other ones like this, you wind up sometimes doing multi-part uploads, because otherwise, if you have a lot of data to upload and something goes wrong on the 99th percentile byte of your five terabyte file, you've got to do it all over again, which is very inconvenient.
Ben Rady: I think there might also be, for S3, now that I say it out loud, there might also be another portion of the S3 protocol that allows you to do that.
Matt Godbolt: I was going to say, I think those are separable, because certainly S3 lets you configure, like, where do the slices of a multi-part upload go and how long do they live before I delete them away.
Ben Rady: Yeah.
Ben Rady: Yeah, yeah, yeah. How long they live. Right. Right.
Matt Godbolt: But then there's multi-part, which is more like, you say, it's almost like a streaming protocol where you say, hey, I'm going to keep sending you data. Here is the delimiter, and you come up with some string that you hope to God doesn't appear in your actual data.
Ben Rady: Right.
Matt Godbolt: And then you say, this is the thing that tells us that we've done this chunk, going to give you some more headers, I think.
Ben Rady: Right.
Matt Godbolt: I think that's how that works. Now, I'm very much in dodgy territory here.
Ben Rady: Yep.
Matt Godbolt: I've looked at it with WebSockets before. There's some kind of thing that you have to do to get it to, during the handshake, at least WebSockets as they were in the 2010s. So 15 years ago.
Ben Rady: Yeah, yeah. Okay, so exploring other areas where you might do this, and we've explained this simple model of the world, we're sending text, we're receiving text, everything's fine.
Matt Godbolt: Oh, God. Right.
Matt Godbolt: Right.
Ben Rady: And you sit down with Telnet and you're like, I'm gonna do what Ben and Matt said and gonna request the thing.
Matt Godbolt: Oh, we didn't really, we didn't see, so, yeah, the curl, for example, or if you Telnet directly, you will see the headers of the response, but curl was hiding them from me because I just did curl, the end.
Ben Rady: Yes.
Matt Godbolt: So typically you'll see something along the line of, first of all, something that says 200 OK, the response thing that says, like, your get verb and whatever, you get a response document that starts with the...
Ben Rady: Yep, yeah, there you go, yep.
Matt Godbolt: What was the code? And you know, we all have seen the codes. The two XX ones are like, everything's fine, and they have slightly different flavors, but usually 200 OK is the one that you'll see.
Ben Rady: Right.
Matt Godbolt: I think there's a, one of the two XX ones is like, OK but no data, is like, just, yes, this is fine, but there's nothing to come, which you see only very occasionally.
Ben Rady: Yeah. Yes.
Matt Godbolt: Three XX ones are like exceptional case type stuff. Is that right? What is the definition of three XXs?
Ben Rady: I think the...
Matt Godbolt: Cause they're like moved and stuff like that, isn't there?
Ben Rady: Yeah, like there's temporarily moved, permanently moved.
Matt Godbolt: Yeah, you know, like...
Ben Rady: I mean, it's most of the, I think all the redirects are in 300. There might be other things in 300.
Matt Godbolt: That's the kind of edge casey thing where there might be. And then certainly 400 XXs are errors of sorts, and like, the world famous 404.
Ben Rady: Yeah, the world famous 404, which you probably know even if you're not a programmer. But there's the one that Twitter proposed and I think actually got introduced to the standard, 420 Enhance Your Calm, for rate limiting responses, which is a reference to the movie Demolition Man.
Matt Godbolt: Yeah.
Matt Godbolt: Oh. Oh, no, I'd not seen that.
Matt Godbolt: Oh, that's cool. I know there's "I'm a teapot," which is one of the things, which is an error code that a coffee, a tea machine may return if it is asked to serve coffee or something like that.
Ben Rady: Yep.
Ben Rady: Yep, exactly.
Matt Godbolt: And then obviously if something breaks in the server itself and the server wants to let you know that something hideous went wrong internally, it'll give you a five-XX. What are the 1XX things?
Matt Godbolt: Are those like continuations, I think, for stuff like multi-part?
Ben Rady: I think so.
Matt Godbolt: Oh, no. See, I'm trying to do this without Googling because I have the loudest mechanical keyboard.
Ben Rady: I know.
Matt Godbolt: So everything you're hearing, dear listener, is coming off the top of our heads, and that, possibly, other parts of our anatomy, depending on how accurate they are.
Ben Rady: That's right.
Matt Godbolt: [laughing] So anyway, and then after that, you get to see a sequence, sorry, of headers that look like the ones that you sent up.
Ben Rady: Right. That's exactly right.
Matt Godbolt: So it's, you know, something colon something else. And then there'll be two new lines, and then off you go with the data, the response.
Ben Rady: Mm-hmm.
Matt Godbolt: So that's, it's pretty cool.
Ben Rady: Mm-hmm.
Matt Godbolt: It's, yeah.
Ben Rady: It is. It is. And so this is perfect, actually.
Matt Godbolt: Yeah.
Ben Rady: This is perfect. So coming back to what I was saying before about, like, okay, I'm going to do what Ben and Matt said, I'm going to sit down and I'm going to like type these things out in Telnet, and I'm going to get back some response. And when I do that, all I see is a bunch of gobbledygook.
Ben Rady: I don't see any text. I don't see any HTML.
Matt Godbolt: Yes.
Ben Rady: What is going on? One of the things that might be going on is that the server has compressed the response. And it will tell you that it has done that, and it will tell you which compression algorithm it has used to do that in the headers.
Ben Rady: So you will see a compression header, and then you will see...
Matt Godbolt: Yes. So the header, the text-based headers are still there, just to be clear, right?
Ben Rady: Yes. Yes.
Matt Godbolt: And then there'll be a new line, and then you'll see garbage, which, although a well-configured server ought not to do that unless you said I would accept a compressed thing.
Matt Godbolt: But nevertheless, this is exactly the kind of thing, to your point at the beginning of this, is like, when something goes wrong, you know, I'm getting an endpoint that just assumes every client can support a compression algorithm, which I believe you're about to go and talk about.
Ben Rady: That's true.
Ben Rady: Right.
Ben Rady: Right, right.
Matt Godbolt: So I don't want to keep, I don't know.
Ben Rady: Well, yeah, no, and that's another thing that curl is, again, you know, peeling back the abstraction here, that's another thing that curl is doing for you, right? It's reading that header, it's applying the correct compression. I think you might need to tell it.
Ben Rady: I can't remember if, like, if you just, but can you disable it? Can you tell it not to do that?
Matt Godbolt: I don't remember either. I know I've had problems before because, and this is going to lead to one of my favorite observations, or rather a mutual friend's observation, you know, whether it's transparently decompressing because it says it's deflated, and so curl is then going to say, well, I'll undeflate it then, because that's what it says that I should do.
Matt Godbolt: Or whether, you know, that just means the fact that the compression, whether the compression is there or not, is now opaque to you. You don't know if it's happening or not, which is the kind of dichotomy of transparency: it means that now it's impossible to see whether it's actually happening or not, you know?
Ben Rady: Right.
Ben Rady: Yes.
Matt Godbolt: So our friend coined the term, which I introduced to somebody at work yesterday, transpaque, which is like that kind of combination of, like, well, it's like you're meant to not know it's there,
Ben Rady: Yeah, yeah, yeah.
Ben Rady: Right.
Matt Godbolt: but if it is there, now you can't tell if it's there or not, and maybe it's broken, and now you're just being gaslit by your system.
Ben Rady: Right.
Matt Godbolt: You know, I think we had this with at least one of the things at work where a vendor was sending gzipped data, but whatever...
Matt Godbolt: accept header we were getting was then un-gzipping it for us, even though we'd asked, you know, like, get bob.gz or get, you know, this.gz. And then the library was like, oh, cool, I'll un-gzip that for you. And then it was storing it as, and we were writing it to disk as, you know, bob.gz.
Matt Godbolt: And you're like, why is everything so huge? Oh, wait, everything's called .gz because that's what we asked for, but something transparently, again, transpaquely...
Ben Rady: Transpaquely.
Matt Godbolt: un-gzipped it for us, being helpful, thinking it was being helpful, and it wasn't, right? Anyway, so, but yes, it could be deflate compressed, I think. Or what are the other ones that are out there? Zstandard, I think, is that standardized? I don't know, but deflate's the one that, you know, everyone uses, which is to say gzip.
Matt Godbolt: Pretty much, to a first approximation, deflate is the algorithm that powers gzip, and then there's a little tiny bit of a header associated with an actual gzip file, which I think is slightly different, and a CRC.
Ben Rady: Okay.
Matt Godbolt: So that's definitely something that can go wrong.
Ben Rady: Yeah. Yeah. Do we want to try to get into TLS and compression and HTTPS? Is that a bridge too far for this conversation?
Matt Godbolt: Well... that might be a bridge too far. Certainly, I don't know enough about it. So what we've been talking about so far, let's just graze the topic and see if we feel comfortable with this.
Ben Rady: Yeah, right.
Matt Godbolt: And then our listener can roll their eyes at how wrong we are. But what we've been talking about is like the HTTP protocol. And in HTTPS, there's another layer, another sort of wrapping of the protocol where the, a certain amount of, well, a certain amount, some,
Matt Godbolt: encryption is applied, and authentication and certificate exchange, all that kind of stuff. But once you've peeled that back, you're back to HTTP again. So like, effectively, from the point of view of the web server itself and from the web client, you can almost, almost ignore the fact that the tunnel that connects them is...
Ben Rady: Mm-hmm.
Matt Godbolt: an encrypted tunnel, as opposed to just a regular Telnet. So obviously you can't Telnet to port 443 and type anything and expect it to happen, which is unfortunate. What you can expect, what will happen is, if you're very lucky, you might see a lot of garbage bytes being sent back to you, and then the connection is closed. That's probably what you're gonna see if you do anything at all, if you Telnet to port 443.
Matt Godbolt: Something that surprised me the other day is, I naively assumed that you connect to port 443, some kind of negotiation happens with a certificate and key exchange and the whole Diffie-Hellman thing to, like, you know, get a unique key on both sides, and then, and then, and only then, you start saying, hi, I'd like to get this, you know,
Ben Rady: Right. I was going to say this exact same thing.
Matt Godbolt: But, yeah, yeah. But it turns out, for all that, for the same reason that we added the host colon to the sort of, like, the message body rather than just relying on the Telnet, you know, the IP address that you're connecting to being unique for that website.
Matt Godbolt: We lose that if we encrypt it, which means that, like, you know, the web server you're talking to, if it's, you know, google.com is co-located with yahoo.com, unlikely.
Matt Godbolt: But whose certificate should the server give you when you connect to it?
Ben Rady: Right.
Matt Godbolt: You're like, ah, good question. I don't know. Which means that you, the client, need to tell it first.
Ben Rady: Yes.
Matt Godbolt: But how do you encrypt that so that no one can tell which site you're actually browsing? And the surprising answer is, you don't.
Ben Rady: Yep.
Matt Godbolt: Which was like, mind blown.
Ben Rady: Yep. Yep.
Matt Godbolt: Oh.
Ben Rady: Not as secure as you thought. All that information is being exposed to the interwebs.
Matt Godbolt: So... does this mean that all these adverts that I keep fast-forwarding through about VPNs...
Matt Godbolt: And, dear listener, do not worry, this is not about to segue into an advert for one of those. This is, it's not a way of announcing that finally Surfshark or whatever is...
Ben Rady: We've taken sponsorship. Yeah.
Matt Godbolt: No, no sponsorship.
Ben Rady: Uh huh.
Matt Godbolt: This is just my fun.
Ben Rady: Right.
Matt Godbolt: We're not turning into anything else. But no, that is the one argument I can see for that, because I just assume, you know, like, fine, HTTPS everything, and it's, you know, no one need know what I'm doing.
Matt Godbolt: But it's, you know, you are actually, if you cracked out Wireshark and visited a website (which maybe we should do live sometime, we need to work out a way of screen recording this or something).
Ben Rady: Yeah. Yeah, yeah, yeah.
Matt Godbolt: Because, yeah, that's, I think the first time you show somebody Wireshark and then you're browsing or doing anything trivial on the network is the first time the scales fall.
Ben Rady: Mm-hmm.
Matt Godbolt: Certainly was for me. Like, oh my gosh, look how much stuff's going on on my network or on my computer.
Ben Rady: Mm-hmm.
Matt Godbolt: Oh, golly.
Ben Rady: Mm-hmm.
Matt Godbolt: But anyway, yeah, so, popping back, you were going to say the same thing about this. Surprising, was it SNI or something, was the name of it?
Ben Rady: Yeah, you should be very careful about assuming that the data that is in your request, the URL, including the query parameters, the headers that you are either adding yourself or being added on your behalf by the tools that you're using, like curl, are actually secure, because they're almost certainly not, in many, many cases.
Matt Godbolt: Hmm.
Ben Rady: So when you make that request,
Matt Godbolt: I was thinking it was only the host that was the thing that was leaked, just purely so the correct certificate could be served back to you. But I may be wrong.
Ben Rady: So I think that, so my understanding of this is that you're doing this sort of, like, escalation request, you're like, I'd like to escalate to this other protocol. And I would not be very confident in telling anybody that I knew for certain that, in all cases, it could only be these things, right?
Matt Godbolt: No, that's fair.
Ben Rady: It's, yeah, it's one of those things where it's like, if it's important to you that that information is encrypted, then you should check, probably using Wireshark or something else like that.
Matt Godbolt: I've just fired it up. I couldn't help myself. Oh, I need to sudo make me a sandwich, of course, because, why, I shall. Yeah, okay, given that I have to escalate, I'm not going to do that now, otherwise, you know, you are going to hear my long password rattling out. But no, that's definitely an exercise for the reader. And I'm sure, hopefully people will comment and tell us where we're going wrong here. But again, this is from the top of our heads. So, yeah, no. So.
Matt Godbolt: Okay, I think that's probably the most we should do on the encryption, because it's clear that we've reached the limit of our knowledge.
Ben Rady: Yeah, yeah.
Ben Rady: We've hit the layer of abstraction that we can recall sitting here.
Matt Godbolt: I mean, there is definitely some things to do with certificates and things, and, you know, the little lock that you see.
Ben Rady: Yeah.
Matt Godbolt: And a little bit about that.
Ben Rady: Not going to pretend to know nearly enough about that to explain it to the world.
Matt Godbolt: Well, I mean, there is like a chain of trust, isn't there? We can probably talk a little about that, where...
Ben Rady: Mm-hmm. Yeah, that's true. Certificate authorities.
Matt Godbolt: Yeah, so your browser and curl or whatever, and your operating system, may provide a list of certificate authorities, which are like trusted third parties that have said, we will sign other people's certificates to say it's definitely them. Trust me, bro. And then, as long as you trust the person who signed the certificate, you can trust.
Matt Godbolt: Google.com is in fact Google.com because it's signed by VeriSign or whoever the heck actually signs Google. Google probably are a CA, aren't they? Probably, and that's fine.
Ben Rady: Yeah, probably at this point.
Matt Godbolt: And that's kind of how you know that you are actually looking to the correct site, the authentication part of HTTPS. And "know" has got air quotes here, because obviously it really does depend how much you trust the people who signed these certificates to say it really is who they say that it is.
Ben Rady: Mm-hmm.
Matt Godbolt: And if you look at the number of CAs that there are inside your browser, if you go and look at the list, and there's like a bewildering number of them, including some from, to me, things that would raise an eyebrow, and I'm like, oh, apparently we trust this company.
Ben Rady: Mm-hmm.
Matt Godbolt: I don't know who they are, and it doesn't look very trustworthy to me.
Ben Rady: Right, right, right, right.
Matt Godbolt: Similarly, in a corporate environment, you might discover, reasonably, your own company's root CA is in that list, because then you can sign, first of all, you can sign all of your internal applications with your own certificate that doesn't have to go out to the outside world, doesn't have to be signed by VeriSign.
Matt Godbolt: It means that you can go to, you know, benscoolapp.internal.com, and you can get a certificate for that, and your browser won't go, wait a second, this is just self-signed.
Matt Godbolt: That's no good. You know, it can be signed by, you know, benscoolcompany.com, which is then in there.
Ben Rady: Mm-hmm.
Matt Godbolt: But it also allows potential for, like, security scanning software to decrypt, by man-in-the-middling you in your own, you know, on your own network, to be able to check that things aren't going outside, which is really important in, certainly in a large corporate environment.
Matt Godbolt: I don't know that it happens in any, like, the companies we're in, but.
Ben Rady: No, no, I mean, that's definitely a thing. I mean, again, like, you know, the theme of this being, like, you know, breaking people's expectations around abstractions and things like that.
Matt Godbolt: Right.
Ben Rady: Like, it's probably been told to you in indirect ways by people at your company if they're doing this, but maybe not in as direct ways as, we're reading your personal email if you check your personal email at work, right?
Matt Godbolt: Right.
Ben Rady: Like, the little security lock in the browser that you're running, right?
Matt Godbolt: Is not as strong as you think it is, because if you're using your company, I mean, obviously, if you're on your personal device on the guest Wi-Fi of your company or whatever, fine.
Ben Rady: Yeah.
Ben Rady: Sure. Right.
Matt Godbolt: And that's a very good reason to do that, right? But yes, if you are, most companies have at least a clause that says something like, there's no expectation of privacy, which is reasonable, right? You're at work, you're doing work things.
Ben Rady: Yeah.
Matt Godbolt: I mean, reasonable people could have disagreements about this.
Ben Rady: You're on a work computer doing work things.
Matt Godbolt: I get it. I get it. I get it. But like, that is the way in which they can potentially look at the encrypted data, is that they can say, well, any website you go and get, if you go to secure.com, I will say, yes, sure, I'm secure.com, and I can prove I am because I'm signed by this certificate, which, by the way, your company-given security system says I can sign anything I damn well like, and that's fine.
Ben Rady: Yeah.
Ben Rady: Yeah, right.
Matt Godbolt: And that's, yeah. So yeah, just knowing that that exists is all you need to do. And it's an exercise to the listener to go and see whether that's something that's happening to them or not. And I mean, it's a fairly standard thing.
Matt Godbolt: Anyway, popping the stack all the way back, right? So we said, what about curl, right? You're right, with curling this endpoint. What else? Is there anything we've missed? Let's just think about this.
Ben Rady: Well, I got, so I got a good one.
Matt Godbolt: OK.
Ben Rady: So let's say that, what you see, when you do this request, and maybe you're trying to drop down a level here and you're gonna use, like, the Telnet approach, is that you connect to the site, you type in your headers, you get back your response headers, and then you get some of the body, and then it stops.
Matt Godbolt: Ah, yeah,
Ben Rady: What happened? What is happening? Like, how do you even know that it's not done? What is the mechanism for that, if I recall correctly?
Matt Godbolt: Yeah.
Ben Rady: It's like, two new lines is the fancy technology that they use to tell you that the body of the document is complete. It's that we're gonna do a new line and then we're gonna do another new line, and that's how you know that it's done.
Matt Godbolt: I think that's the hardcore thing.
Ben Rady: But let's say that you don't see those two new lines and it just stops.
Matt Godbolt: Right.
Ben Rady: What is curl doing in that situation?
Matt Godbolt: That's an excellent question. I don't actually know the answer to that.
Matt Godbolt: Yeah, I don't know, because, like, my instinct, if this was an interview question and you, Ben Rady, were asking me what's going on here, I would be like, oh, this sounds like the remote end has decided to keep the connection open.
Matt Godbolt: But it has, in fact, finished the document. And I don't know if it puts something at the end of that. I don't think it does. Maybe that's one of those 100, 100 continue is like one of those things, as it puts it in. I don't know. But, totally separately, and just pause there, I did, in fact, just Telnet to google.com and do get space slash enter.
Ben Rady: I think you could get something like that, yeah.
Matt Godbolt: And it doesn't even tell me that I'm at the wrong address, because it doesn't know that I came to the wrong address. It doesn't try and redirect. It just gives me the whole...
Matt Godbolt: you know, minimized google.com thing pukes out. So it still works with even, you know, HTTP 0.9.
Ben Rady: Interesting.
Matt Godbolt: Okay. What happened then, Ben? Tell me the answer. What was going on with my hung connection, or what might be happening?
Ben Rady: So this could be, this could be a number of things, but one scenario in which you would see this is that there was, like, an error, like a programming error on the server side, where it like read a document,
Matt Godbolt: Oh.
Ben Rady: and encountered an error and for whatever reason didn't close the connection. Right, maybe it's waiting for, itself is waiting for data to show up that just isn't showing up.
Matt Godbolt: Right.
Matt Godbolt: It's not going to come, or it's deadlocked in itself trying to do something.
Ben Rady: Right. It's, like, in the midst, stuck on some database query or streaming some data from another source or whatever it might be, and it's just stopped.
Matt Godbolt: Yeah.
Ben Rady: curl has a number of different timeouts that you can configure. And part of the reason for those timeouts is situations like this, because you don't want some process to just be stuck forever because it made some request that is now stuck forever.
Ben Rady: It has a connection timeout, which is more of a TCP level thing. It is a TCP level thing. It's like, hey, I want to connect to this IP address and send it some text so I can get my documents.
Ben Rady: If it never establishes the TCP connection, or it might take some time to establish a TCP connection, there's a timeout that you can control for that. I believe there is also a response timeout, to say, like, if I start getting data and then I stop getting data, just whacked my microphone.
Matt Godbolt: That's fine, I'm sure that will be edited away.
Matt Godbolt: It won't be edited away!
Ben Rady: Yeah.
Ben Rady: And you stop getting data, and then you continue to not get any data, how long should I wait before I just give up? And it's important to know, like, with all of these protocols, we use it often enough, you almost think of them as, like...
Ben Rady: You almost think of it as being, like, this synchronous thing, right? Like, almost like an atomic thing, where it's like, I'm going to do the request thing and we get the response, and it all works.
Matt Godbolt: Right, right, right, right.
Ben Rady: And it is entirely possible that you get some, but not all, of the response. It is entirely possible that you get halfway through sending the request and then the connection just dies. It's entirely possible that you start sending the request and, depending on exactly what is happening on your network, it may not be clear how much of it was actually sent.
Ben Rady: Like, did you, I'm doing this, like, put request or delete request, right?
Matt Godbolt: Oh.
Ben Rady: Like, did you actually delete it on the other side?
Matt Godbolt: Right.
Ben Rady: I'm not sure.
Matt Godbolt: That is, yeah, the sort of, like, what's the name of it? The Byzantine generals problem or whatever, where you've got, like, the two, you're like, I sent a thing to you that says, did you delete it?
Ben Rady: Yeah.
Matt Godbolt: And you go, I got it, but I didn't hear your response. You're like, well, did you then? I don't know. I don't know, because I didn't, and like, there's any number of acks on both sides that you, and you can never be a hundred percent sure. And yeah.
Ben Rady: Exactly, exactly.
Matt Godbolt: Yeah.
Ben Rady: So that is definitely another place where this sort of abstraction can break down, where it's like, well, I did the delete request and it failed.
Matt Godbolt: I've...
Ben Rady: So clearly it's not deleted.
Matt Godbolt: There's at least five different timeouts.
Ben Rady: Well,
Matt Godbolt: I've just been looking through the man page of curl, five different termouts, termouts, timeouts for different conditions.
Ben Rady: Termouts.
Matt Godbolt: You know, like, how often, how long do you wait before connection? As you said, overall timeout for the entire request, the first response, and then, you know, subsequent chunks of data, that kind of thing.
Ben Rady: Mm-hmm.
Matt Godbolt: And then some other things to do with IPv4 and IPv6, if you're trying to do both at the same time and get the first one to respond, some really clever things like that. But yeah, a lot of those things are sort of, yeah, as you say, we think of these things as atomic.
Matt Godbolt: We think about, you know, like, everyone talks about RESTful APIs and you're like, okay, it's like a magic RPC that I just do, and it happens or it doesn't happen and I'm done. And you're like, you could see an error on your side that says the connection never made, you know, the request never went through, but it did in fact make it through.
Matt Godbolt: It was just the response didn't get to you and the TCP was torn down, but it did in fact take effect on the remote end, and now you're stuck.
Matt Godbolt: You didn't get back, you know, the response. So it's a tricky world out there, but yeah.
Ben Rady: Absolutely. Yeah.
Matt Godbolt: No, there's many timeouts. Oh my gosh, yeah.
Ben Rady: You said there's five timeouts?
Matt Godbolt: I think so. I've got connect timeout, expect 100 timeout.
Matt Godbolt: "Maximum time in seconds you'll wait for curl for a 100 continue." Oh, good. I wasn't completely making that up.
Ben Rady: That's good.
Matt Godbolt: See, you also, connect timeout. Happy eyeballs timeout. That's hilarious to me.
Ben Rady: What? Happy eyeballs timeout?
Matt Godbolt: Yeah, I'm going to read this to you because it's joyous. "Happy Eyeballs", with a capital H and a capital E, "is an algorithm that attempts to connect to both IPv4 and IPv6 addresses for dual-stack hosts, giving IPv6 a head start of the specified number of milliseconds." So it's like a way of, you know, preferring IPv6, but not necessarily...
Ben Rady: Oh man, that's good.
Matt Godbolt: So yeah, we've got, see also, a max timeout, retry max time. So there's an overall max time, and then there's a retry maximum time, which is like the number of times it will, including, yeah, retry time is reset before the first... It's complicated, but I mean...
Matt Godbolt: And then we're going to keep talking.
Ben Rady: Yeah, and we're back.
Matt Godbolt: What happened there was a terrible break in the abstraction layer, because my USB hub crashed and put us into a very bad state. And we've just about recovered it.
Matt Godbolt: And, can't remember what on earth we were talking
Ben Rady: No idea. No idea at all. It's like it was a different dimension.
Matt Godbolt: about. And if only, I mean, if only we had some kind of permanent record of what we'd said. But sadly, neither you nor, you said you could download the MP3s okay, and it was working. So at least we've got that on our side.
Ben Rady: I did do that, yes.
Matt Godbolt: Okay, right. So we've got the previous part. So, dear listener, we apologize. Whatever we were talking about before has slipped our minds. But we didn't want to leave you in the lurch, or, more importantly, we didn't want to leave editing Matt with a job of, like, how do you finish an episode where...
Ben Rady: Right.
Matt Godbolt: Clearly something went wrong. But I think there's another abstraction layer we can talk about another time. It's like, how on earth do our devices talk to each other in a sort of day-to-day environment?
Ben Rady: Yeah, USB actually would be kind of a cool one.
Matt Godbolt: USB, USB-C and DVI and digital video and all those kinds of magic.
Ben Rady: Maybe a bit ambitious, but I think I like that one.
Matt Godbolt: I mean, it's a miracle that it works as well as it does, frankly.
Ben Rady: Yeah. Yeah. Yeah.
Matt Godbolt: But yeah, we were looking at curl, and I think we were just about looking through all the timeouts. That's what I'd recall us talking about.
Ben Rady: Yeah. Oh, the happy eyeballs.
Matt Godbolt: Happy eyeballs for IPv6, IPv4, which is crazy. I saw someone put an IPv8 proposal out there, which I think is somewhat tongue-in-cheek, but also I read through it and I was like, this isn't as stupid as it looks.
Ben Rady: What?
Matt Godbolt: Maybe this is real. You know, I can't tell.
Ben Rady: And they skipped seven because we skipped five. That's how, yeah.
Matt Godbolt: I guess. I think the, it uses, like, a 64-bit IP address rather than the full crazy IPv6, I don't know, 256-bit, 128, whatever bit size that is, where it's like, hey, all this information about the host, you're like, I don't want to leak that outside my network.
Matt Godbolt: Thank you very much.
Ben Rady: Yeah, yeah, yeah.
Matt Godbolt: Right, so it was, and it's sort of backwards compatible with IPv4 because it's essentially, all existing IPv4s are 0.0.0.0.0.0, the rest. You're like, oh, the rest, you're like, oh, that makes sense.
Ben Rady: Interesting.
Matt Godbolt: Maybe that's got legs. Anyway, that's really off topic, and I have, that's about as much as I know about it. My friend, I don't know if there's anything that we can recover from where we were going.
Matt Godbolt: So I think we should probably just call it at this point here.
Ben Rady: No, yes.
Matt Godbolt: And having introduced the world to transpacity, I feel like, you know, we've at least achieved something.
Ben Rady: Transpacity?
Matt Godbolt: Yeah.
Ben Rady: Transpacity. Okay.
Matt Godbolt: I guess it would be, wouldn't it?
Ben Rady: Yes.
Matt Godbolt: Yeah.
Ben Rady: Yes, it is.
Matt Godbolt: Transpacity. Opacity, transpacity.
Ben Rady: That's exactly, transpacity.
Matt Godbolt: We'll have to ask our friend and see if maybe he'll join us on the podcast sometime to talk about it. But it's definitely a thing.
Ben Rady: Yeah, there you go.
Matt Godbolt: All right, mate. Well, it's been a journey today.
Ben Rady: Yep. Fun times.
Matt Godbolt: I will chat to you another time.
Ben Rady: Sounds good.