Matt Godbolt: Hey, Ben.
Ben Rady: Hey. Hey, Matt.
Matt Godbolt: Um, I was just looking at our, uh, twos compliment org, uh, thingamajig
Ben Rady: Yeah. Yeah. I, I was looking at that too.
Matt Godbolt: I always type twos compliment.org into my browser. And it seems like Chrome these days does the automatically, Hey, you probably meant www dot that. Um, but if you don't do that, like if I use Curl or whatever, right. Uh, I get like nothing. In fact, I think it hangs, uh, let me just try. Now I know I'm doing that https compliment.org.
Ben Rady: Yeah. I get a, I get a redirect if I use Curl.
Matt Godbolt: So I'm literally just using Curl https s
Ben Rady: Oh, that not doing OTPs. I'm just doing Curl Wells. Everyone does
Matt Godbolt: HBS these days. All the cool people. True. It's just hanging.
Ben Rady: Yes, true, true, true. But I'm saying like, if I just do curl twos compliments.org, I get a redirect.
Matt Godbolt: Oh, okay. But yeah,
Ben Rady: So I would expect, I think I would expect https
Matt Godbolt: I Yeah, yeah. I see you are being redirected in. I see. Yeah. If you, if I use http,
Ben Rady: Because there's nothing listening on port 443 for whatever that is. Right. Because I think it's
Matt Godbolt: I don't even know how this stuff's done. I know that I check stuff into the GitHub and something something
Ben Rady: Magic, magic happens. Yeah.
Matt Godbolt: So should we try and fix that? Is that a thing
Ben Rady: We could do? Um, well I think we should. I mean, there might be part of that. So like, like right now, the way this works is I think there's a cloud front distribution, and then when you check something in it like builds the podcast xml, the RSS feed, basically
Matt Godbolt: the RSS feed. Right. Right, right.
Ben Rady: And then, uh, you know, and the rest of the website, if it has any changes to load, it's all static. So there's nothing to build. The only thing that actually gets built is the RSS feed and then, you know, uploads that to an S3 bucket and then that, that cloud front distribution is backed by the S3 bucket.
Matt Godbolt: Got it.
Ben Rady: Uh, yeah, that cloud front distribution is backed by the S3 bucket and then you have to like invalidate the cloud front distribution to basically invalidate all the caches. Right.
Matt Godbolt: Right because we are not doing anything clever with like cash busting URLs and things. And we definitely like, essentially index.html To always be like, just come and get it again. Um, I mean, I guess we could monkey with headers and things. I've done that before, but like the, the simple straightforward thing to do for a tiny site like ours is to say it's just all invalid. Like literally no one's really cares about the caching of this.
Ben Rady: Exactly. Exactly. So I'm trying to like, okay, so if I do an NSlookup, like in a Yeah, I was gonna say,
Matt Godbolt: Where does the DNS actually point at? Where
Ben Rady: Does DNS DNS actually point at? Um, or actually I don't want to do a DNS lookup. I wanna use like dig. Right.
Matt Godbolt: That's what the cool people do. Yeah.
Ben Rady: Um,
Matt Godbolt: But I, I'm still old school and I end up using nslookup, but let's do, let's learn Dig.
Ben Rady: Yeah. So DIG is giving me an A record
Matt Godbolt: A.
Ben Rady: Yeah.
Matt Godbolt: For um, twos compliment.org. Yeah. I think there's something funky. I think you can't have cnames for the top level domain. Oh, as I recall, that's what
Ben Rady: It is. That's what it's, yeah,
Matt Godbolt: That's my, because there's, so there's always like a thing. I know my DNS provider had some hacky way of pointing the, A record at where you needed it to be always. Where are we? Where is the, who's our DNS provider ours? Yours.
Ben Rady: So it used to be Hover, um, and I am pretty sure that I moved it. Let me pull up the DNS or the AWS console here and confirm that my memory is not failing me because it very much, I think I moved to, to
Matt Godbolt: My memory is still been,
Ben Rady: And I have a memory of doing this because I was like, oh yeah, if we're ever gonna get this top level domain to work, like Amazon has some cool stuff that will make that work. And I actually reached out to the Hover support folks and they're like, yeah, we don't do that. Um,
Matt Godbolt: Right. I was using dnsimple, dnsimple whatever. And they have like this fake, um, they have a name for it. It's like a made up type of record that's just like, it's itself, DNS look up something else and then keeps changing the, A record to point at it which obviously is not great for all the reasons, but, I think Route 53 or whatever it is on Amazon kind of does this out of the box.
Ben Rady: Yeah. Now I'm starting to worry that I didn't actually move the domain.
Matt Godbolt: Oh, that's smelly
Ben Rady: Hold on a second. What is even? Come on browser.
Matt Godbolt: What even are a domain. You just don't know.
Ben Rady: Hover?
Ben Rady: dot com.
Matt Godbolt: What are birds?
Ben Rady: Uh, what uh, I mean are they real? I don't think they're real.
Matt Godbolt: I don't . Well, there's two. Yeah, that's true. Actually there's two memes a associated with the birds, isn't it? There's obviously the, uh, uh, our birds real thing Uhhuh where someone said they're all spy drones. And then obviously the thing that you and I shared a joke about is, uh, um, uh, look around you. What are birds?
Matt Godbolt: Uh, we just don't know. one might as well ask what are birds
Ben Rady: Status is redemption, what does that mean? Yeah. No, unfortunately two's compliment. So that is almost certainly what that A record is, is my, is hovers attempt to redirect you and that's why http works. But https does not, cuz hover doesn't have any concept of the SSL cert.
Matt Godbolt: So what the, the, the, the A record for twos compliment org, that is the un the prefixed twos compliment org. is pointing at hover's own magic to do this. Is that?
Ben Rady: Yes. Yeah.
Matt Godbolt: Okay. And obviously they don't have our SSL cert, so they couldn't be able to, to serve up a redirect on port 443.
Ben Rady: Exactly. Exactly. Yep.
Matt Godbolt: So could we at least set up the CNAME in Route 53, like now, and then test it with Dig that it would work, uh, and then, you know, then set up, kick off the, the process of like moving the, the whatever the Main record and then we can at least go through that so you can Yeah,
Ben Rady: Yeah. Yeah. How are
Matt Godbolt: You, do you, how do you do this in the console or do you terraform for this kind of stuff?
Ben Rady: Um, I mean, generally I terraform, but I don't actually remember what I did for two's compliments
Matt Godbolt: . Fair enough. I don't think
Ben Rady: I hate me. I don't think I did set up any char. I mean, it would be really cool. I would love to, um, add that Terraform into the CI build that does the deployment and everything. Because you
Matt Godbolt: Can do just like our day job,
Ben Rady: Right? Like you just have it do a Terraform apply, uh, and then do the deployment and then it's like, oh, I want to change the CNAME records or something. Like that's a commit along with anything else that just gets applied automatically. And
Matt Godbolt: Just like, you know, when we publish an episode, we just check in an updated thing to our like Python program that generates the XML . And that's it. So yeah. It would be cool if it if it did that. Although, you know, also in fairness right now we don't have very many things to worry about. We've got a cloud front distribution, we've got an S3 bucket and we've got, um, the, well, we would have the dns, I dunno if there's anything else in there. So I don't mind starting out by like just doing this manually and then importing it at a later date just to see if we can get it working. Unless you feel like we should spend a, you know, 10, 15 minutes no
Ben Rady: Hacking
Matt Godbolt: Up a Terraform thing. It's just there's a lot of credential nonsense to worry about and CI and all that.
Ben Rady: My only, I I agree with you and the thing that pains me is that I did this before once for a bunch of my domains and I have zero memory of how I did it, which means I'm gonna have to learn it all over again. Oh. Because I didn't encode it into anything that is automated. So
Matt Godbolt: It, it embarrasses me to say that, like I know that I have my, you know, hobby project, but I have enough other hobby projects that I have a God bolt Terraform, like personal Terraform set that we might want to crib from. So if you go to my GitHub, let's just have a quick look. Oh, is it public? Uh, it is, yes. And I'm after me carefully going, I'm so sure there isn't anything in here that, that that needs to be done. It doesn't auto apply because I do it manually because, you know, I know when I made a change. But github.com, Matt godbolt slash godbolt hyphen Terraform. Sorry for all the clicking noises. Yeah, there it is. And so in here is like owlet, that's a deploy. That's the deployment thing. I'm wondering if, do I have in here, I dunno if I do have a, uh, a do that ACM certificates jsbeeb,
Ben Rady: what's the name of this repo?
Matt Godbolt: Has resources and crap in it? It's got whatever, I don't think this has root dodad. Cuz it kind of like jsb lives on bbc.gobo.org and, and godbolt.org as long been adopted as Compiler Explorers real domain. So it lives there. Really? Ah, yeah. I don't think this says any Route 53 stuff in it or blog. Aha. Hang on a second. I have, so there's xania.org,
Ben Rady: Something like this, but I don't know if it's actually relevant to this.
Matt Godbolt: CloudFront distribution has got all the crap for xania.org, Which is my blog. I don't know that this has got the root, the, the DNS for it though, which surprises me a little bit and embarrasses me also, cuz I don't, there I am touting like, Hey, well I did it. You're like, no you didn't
Ben Rady: I've got something for this, for an old version. I, I have a recipe tracking website that I made called Tastypenny. Um, which
Matt Godbolt: Is again with the Hobby project side
Ben Rady: Project things. Right. And I, I had an, a more sophisticated version of Tastypenny that I was working on for a while, um, that had like an actual app server instead of just being a bunch of static content. And I still have the Terraform for that and that used Route 53.
Matt Godbolt: So do you wanna just copy, do some copy pasta and see if it looks near enough? Like we could just paste it as a subter into the twoscp. Oh,
Ben Rady: Cool. You know what? I think I actually have a better one. Let me read this through this one. So Earth is Dead. Oh. You know my board game.
Matt Godbolt: Oh. Oh, I thought you, it was a statement of fact.
Ben Rady: Yes. No, that's, I clicked on the wrong button. I, and now I've deleted the Earth.
Ben Rady: Well it was nice knowing you. Yeah. Um, yeah, no, my board game, so earth is dead.com. I have
Matt Godbolt: Oh,
Ben Rady: Something for that. I don't know if I have an SSL cert. Hey you.
Matt Godbolt: Hey that's cool. Rady Set Go
Ben Rady: Yeah, that was a friend of mine came up with that name. I was like that's a good name.
Matt Godbolt: It is. How did the other ra I mean actually the same thing could be asked of me, right? Like, how did the other Rady's feel about you? Like being a Rady that's kind of stolen the name for your own machination.
Ben Rady: You know my name, my last name is kind of made up anyway. My, my dad invented it.
Matt Godbolt: Oh. Because most of 'em aren't though, you know most of 'em, right. Clearly . Yes. Real .
Ben Rady: Some of them are
Matt Godbolt: So, I mean some of 'em are like, yeah, I think we talked about this before. Like Taylor and Yeah. Baker and uh, Fletcher and Smith and all that kind of stuff. Yeah. But you know, ultimately it's all made up. Yeah.
Ben Rady: Every, everything is, but
Matt Godbolt: Yours is very specifically made up by like, I it's very recent history
Ben Rady: Mine is very specifically made up.
Matt Godbolt: Was it your grand grandpa or? That was
Ben Rady: My dad. My dad. I mean, it, my dad, he just changed the spelling to be more Americanized. I say it's made up, it's like, you know, changed it from R A D I to R A D Y.
Matt Godbolt: Oh, that's cool.
Ben Rady: Yeah. But it's, it's funny because there's this like business school out in San Diego that apparently was paid for or founded or whatever by somebody with coincidentally the same last name. And we, uh, my brothers and I have got into the habit of collecting the swag from this, uh, business school. Cause it has, they have like stickers that say I love Rady. And it's like all, you know, backpacks and all those other crazy really bananas stuff.
Matt Godbolt: That's super cool. Alright. Anyway,
Ben Rady: Sorry
Matt Godbolt: I'm
Ben Rady: Distracting you this tangent, but I, so what I'm seeing right now is I don't necessarily have any, any terraform that we could drop in that would be exactly what we want. I definitely have a couple of things. And it sounds like you have a couple of things that would be like, close?
Matt Godbolt: That are like 80%.
Ben Rady: So maybe we, maybe we could do it this way. I think so backing up a second, I think the big thing is, is that I have never done the transfer over to Route 53. So if we feel like we can, if we feel like we can actually set up, cuz like, okay, so my, my memory is fuzzy on this and maybe you remember,
Matt Godbolt: I, I actually, I don't even know if we need to transfer it, right. If if who your current provider, they let you set the, the name servers, they don't have to be their own name servers. They can still be the registrar to start with. And all you need to do, all you quote need to do is edit their name server record to point at the Route 53 Route Route 53 ones. Sorry, this is a terrible word for us to have to you keep going back and forth on.
Ben Rady: Okay. So let me look at, okay. Yeah. So right now the name servers for NS one. Yeah. nsone.hover.com and nstwo.hover.com. And can I see that if I use a dig command?
Matt Godbolt: I think so. I think that's how you know, and, and it'll take a while to propagate, but we can actually make that change. But we need to go into the Route 53 and at least copy over what you currently have, which we can either do manually or we can terraform that up.
Ben Rady: Yeah, yeah. Okay. Okay. Yeah, let's, let's start with Terraform. I like that idea is the cool, I don't even remember is the twos compliment. Is the repo public? Or is it?
Matt Godbolt: Do we have any Terraform? I don't think it's public. I think it's private.
Ben Rady: Okay. But you have access to it, right?
Matt Godbolt: I do have access to it. And yeah, I've farted around making it all use poetry for the Python and all that stuff recently. Yes. And we have assets and tools and Okay. The public directory ironically, which of course is not actually public in this instance. We will be, that
Ben Rady: Is the public part. The public directory is the part of that that is actually like visible to the whole internet
Matt Godbolt: but it's not light open to the world because, you know, well it is
Ben Rady: Through the West website because we
Matt Godbolt: Put stuff in it is, but yeah, like importantly it means, you know, when we put our, the reason why I think we shouldn't make it public is cuz that's where we do put our like episodes before we finished uploading them and farting around with them, editing them and whatever. Right.
Ben Rady: Um, yeah, there's
Matt Godbolt: A lot. Although in fairness, that's the same true on my blog post, but I don't blog quite as often as I used to. Like once a year is now my cadence. Yeah. So why don't we make a Terraform directory there and just copy paste in the main TF and the terraform.tf or whatever
Ben Rady: Yeah, yeah.
Matt Godbolt: From some other projects, you know, in the way that we do
Ben Rady: And of course I have some makefile magic that I think I can steal to apply this stuff so we can make sure that the, that the Terraform that's checked in is always representative of the real world. Right? Right. Like, you never want those things to be able to get outta sync. So
Matt Godbolt: That is true. That is true. Um, do we have this pre-commit hooks in? I'm a big fan of pre-commit hooks now, and I, there's, so I I use dot pre-commit, like, uh, we, well we use it at work and it'll be cool to make it so that you can't even commit if it says, hey, you know? Oh, no, no, no, no. That would be different because we want the CI to apply the Terraform. Yeah.
Ben Rady: I mean, it's a nice sort of credential management thing where it's like you don't have to have the AWS credentials. All you have to have is access to the repository and the CI build is what holds the AWS credentials. So I'm not handing out like personal a, like credentials my personal AWS account to people in, I mean,
Matt Godbolt: That's for absolutely for certain. I think that's a, that's that's a very, very good, uh, uh, thing to consider. But pragmatically I have found that like, it's very hard to test the Terraform config without actually applying it and, or like, especially if you wanna adopt existing things. So anyway, yeah. Let's start from, let's start from just getting it working at all. Yeah. Um, even manually I would say. So let's make a Yeah. Make a directory or something.
Ben Rady: Yeah. Okay. All right. Uh, yeah. And let me, let me grab this. Hopefully I have some makefile magic I can throw into this thing.
Matt Godbolt: Do we need the makefile magic? I mean, it doesn't do much. I mean, Terraform apply what you want to type.
Ben Rady: Well yeah. But like, installing Terraform as a tool and knowing what version you've got.
Matt Godbolt: Oh, neat. Neat, neat, neat. Yeah, that's better.
Ben Rady: Um, which, and if I, I think I have this, and if I don't have this, I'm gonna give up on it, but I, I think I actually just have this handy
Matt Godbolt: Terraform, I was gonna say, yeah. I actually use ozy for this, which is the thing. Oh yeah. Like as in our op our company's open source version of ozy, which has Terraform as a, as a an example thing.
Ben Rady: Mm-hmm.
Matt Godbolt: mm-hmm. . So that is a possibility. But you've probably got Terraform on your path if we, and Terraform is a bit subtle cuz you need to have the same version
Ben Rady: And yeah. So I have these like Terraform and Terraform and init targets that Oh neat. Like install Terraform and initialize the deal
Matt Godbolt: Sounds Perfect.
Ben Rady: Especially
Matt Godbolt: We're both on Linux, so Yeah. It's not gonna cost me No problem. We don't have to worry about like, oh, but what operating system. Yeah. And it'll get the right version and all that crap. So Yeah.
Ben Rady: Uh, do you have a major allergy to just putting the Terraform file like a single Terraform file in the root of the repo? Because this will only really ever be like a couple of things. Or do you want to make a a Terraform directory and put
Matt Godbolt: I would make a Terraform directory cuz it, it just, cuz it's one extra command mkdir .
Ben Rady: Yeah that works for me. That works me. All right.
Matt Godbolt: And especially as Terraform is like, it sort of globs the directory it's running in Yeah. To get all the.tf files. like the, yeah. Anyway, let's do it now.
Ben Rady: It now. No, I'm, I'm down with that. Okay. Um, yeah, so I I you'll be able to see this in a minute when I commit it, but just to explain it to Yeah,
Matt Godbolt: Yeah. I was gonna say like, this is,
Ben Rady: Um, I'm making do this through
Matt Godbolt: The medium of
Ben Rady: through the medium of sound
Matt Godbolt: Speakings
Ben Rady: uhhuh talkie. Uh, so I have a Terraform variable, a make file variable that is
Matt Godbolt: Like dollar open Squigglies terraform shout caps. No doubt. Yeah.
Ben Rady: I
Matt Godbolt: Mean we could share screen, but it's more fun to do it this way.
Ben Rady: Yes. Uhhuh . Uh, okay. So I have that, and that depends on a tools home, uh, you know, dollar parens. Cool. Uh, CURDIR slash dot tools. So I have a little dot file directory that holds all the tool installation. That's a
Matt Godbolt: Nice subtle thing that actually CURDIR is the right thing to use in makefile because I've used wood so many times on Yep. Yep. Some other thing. And it's like it own, it works by coincidence most of the time, but it's a, a sharp edge ready to cut your knees off. When you do make dash C or whatever that I, I can't even remember what breaks it under, but like now I've just learned. Yeah.
Ben Rady: Just, just CURDIR is the right thing. Coo. Um, okay. And then I have a Terraform version variable just to make it specifically called out as like, yeah, this is the version we're using. Yep. And then I have yeah, this Terraform variable, which points to the executable and you know, because Hash Corp is Hash Corp, all of their tools are super easy to install with just
Matt Godbolt: Because they are just a static executable you curl and put somewhere and then chmod or Unzip or whatever it is
Ben Rady: It's, it's the best thing ever. I just wish everything in the world worked like that.
Matt Godbolt: This is, yeah. We've talked about this before as being like a killer feature of, of Go for example, which I think they're using. Yeah. And,
Ben Rady: Um,
Matt Godbolt: For our tool actually, so we, uh, did you see that we opensourced rpy
Ben Rady: Rpy? No, I think I missed that.
Matt Godbolt: Yeah, so we, you know, uh, it is a read the nearest quote, nearest pyproject.toml, And then there's a little stanza in there that says, no, this is the, the, the, the right python to use. Like this is the right thing to do beforehand. So like, for example, um, you can put a stanza that says, Hey, before you run Python, do make deps in this root directory where you found the py project. Um, and then, um, uh, effectively it just means like I can go into any project and it's, it's, uh, rpy splash space and then whatever. But anyway, the whole, it's, it's a, it, it's a simple cool tool, but we did it in Rust and it's relatively straightforward to make that static as well, relatively straightforward. So that was a nice find recently. So you can just curl rpy down. And also, uh, rizzy I think is the other thing we've open sourced, which is another rusting that's similarly, we just want that, just copy the binary and it works feel nice. And that's all hashicorp and Go goes sort of like pushing , pushing us in that direction. Mm-hmm. .
Ben Rady: Yeah. That's good stuff.
Matt Godbolt: That's my rambling covered enough while you're frantically tapping away there. Yeah, that's good. You don't have an offensive keyboard incidentally. I, I feel like when I, or maybe it's just Google Oh, filter. It's doing this amazing filtering of like non sound. It might be nonhuman sound. Yeah.
Ben Rady: When you come back and edit this podcast later, you're gonna be like, why is this just 90% typing? Um,
Matt Godbolt: Yeah. Right. So we are just agreed that instead of just , instead of us recording a podcast episode, we're, this is the episode now, ? Yes, I guess so. Hey everybody, this is our podcast. I guess we kind of started recording and Yeah. .
Ben Rady: Yeah. I don't know. I mean, we'll see if this ever sees the light of day
Matt Godbolt: . Right, right. We're boring the heck out of people as we like do our debugs in, but it's like listening in a pairing session, I guess. Yeah,
Ben Rady: Yeah.
Matt Godbolt: Of a sort.
Ben Rady: Um, okay, so I got,
Matt Godbolt: I'm gonna make you write tests for this Terraform . Oh,
Ben Rady: You know, I, that would actually be a really interesting exercise is trying to write tests for Terraform. Um, I've written tests for SQL before. That was a fun project.
Matt Godbolt: Oh my God.
Ben Rady: Um, I've written tests for bash, I mean, we should do an episode on, on, we should do,
Matt Godbolt: Yeah, we talk about that. Yeah, it's in the list isn't it? About testing Bash? Although it's, there's a num Yeah. Yeah. I'd be interested in the Terraform test type stuff. Yeah. I definitely use Terraform validate in some of my pre-commit hooks to make sure that like, it's at least synt tactically valid. And I think it's Terraform. Terraform format that sort of makes it canonical formatting. Anyway, ,
Ben Rady: Uh, okay. So I think, I think I gotta change something here because I'm running my target and it says nothing to be done. So I, oh. Usually what you have, oh, if
Matt Godbolt: You, if you, you haven't made it dot phony have you, or have you made it dot phony if you haven't made it dot phony, you have a Terraform directory and that's what it thinks is up to date. It's like, yeah, I got a Terraform directory. You did make Terraform. And it's like, yep. Got that.
Ben Rady: Yeah. Well, I don't think that's it, but I I I did not, that's called,
Matt Godbolt: That's before now where I had directory.
Ben Rady: Yes. I mean that's, I just like, I'm okay, I'm just gonna do this because there is no, the target I'm trying to run is called infra and there is no infra directory. Oh,
Matt Godbolt: Okay. Great. But
Ben Rady: I, I'm gonna make it phony just to, no, that's not it. Yeah.
Matt Godbolt: It's not gonna be that, but like it was worth checking out, but that Yeah. Yeah.
Ben Rady: Terraform init. I think the problem here is I have like an undefined target. So basically like terraform init. Mm-hmm. , that's totally what it is. So I had a, I had a target that I was referencing by a variable name, so it's dollar parens terraform init. And the intention of that is it's gonna be the directory that Terraform initializes itself into, but I hadn't declared that variable, so I think it was just blank
Matt Godbolt: Empty.
Ben Rady: So it's like, yes. Wow. So then make was like, I, there's, you know, I'm not gonna make an empty thing, so it does nothing, but I also need to do terra CURDIR slash terraform dot terraform because we want to put this in a Terraform sub directory.
Matt Godbolt: I think so. And then we can hide all squirrel away, all the dot yes dot files and drop them in there as well
Ben Rady: And I think what that also means is that I need to do a CD in my Terraform init target before I, yeah. Uh, and because it's make, you can't do that on two separate lines, right? ?
Matt Godbolt: No, let me just have a look. Does Terraform has a dash chdir equals Okay. So you can use that instead. So a bit like make dash capital C or Ninja Dash capital C where you're like, Hey, I'm running you, but don't run here, run there. It might make more sense to, to use that. It's nicer because it doesn't need to use sub commands.
Ben Rady: Do I need to make the directory first?
Matt Godbolt: Oh, sorry.
Ben Rady: Do I need to make the directory first?
Matt Godbolt: Oh, I think so. Yeah. Okay. It literally is like, Hey, you're gonna CD into that directory first. Okay. But it means you don't have to do the thing on one line. Yeah.
Ben Rady: Like,
Matt Godbolt: Which is essentially a sub shell. And, uh, and make doesn't know it, it just sort of executes it in the whatever the prevailing shell is and says, Hey, you do this.
Ben Rady: Okay. So Yeah. Which
Matt Godbolt: Is almost always sh or Bash, but for example, fish doesn't support that kind of nonsense. Yeah. Even though I, I love it dearly.
Ben Rady: Um, okay. Yes. So I missed a target no rule to make Target source two compliment.tools. So I, I defined that tools directory, but I never made a target to create it.
Matt Godbolt: To create it.
Ben Rady: Yes. So, so I gotta grab that target, which is very simple. What're
Matt Godbolt: Doing here is kind of working it backwards from like, I need this thing, so, and then mm-hmm. , but you haven't actually told it how to make that thing yet. Exactly. The glory of makefiles. Right,
Ben Rady: Exactly. Okay. So now I have a tools home target, which is just mkdir dash p dollar @. So it's like, you know, just make the thing that you wanted,
Matt Godbolt: Make the thing that you said I need needed to be, yeah.
Ben Rady: Yep. Uh, okay. So I ran that and it installed Terraform and then unpacked it to the tools directory and then it initialized Terraform in an empty directory. So now I would expect to see a Terraform directory, and I do, uh,
Matt Godbolt: fabulous.
Ben Rady: Inside of that Terraform directory, I would expect to see a dot Terraform directory. And I don't . Oh, because it says you have no Terraform files. Yeah.
Matt Godbolt: You need to make a main.tf Or just anything in there. Let's just create any old thing. Yes. Um, while we're thinking about it, I know there are various lock files and other nonsense, some of which need to be checked in and some of which don't need to be checked in. And I can never remember which ones are which. I
Ben Rady: Think this project that I have has a gitignore that will
Matt Godbolt: Has a exactly. Thank you. That's what I was asking basically is do you Absolutely. We can copy.
Ben Rady: Yeah, yeah, yeah. Um, all right, let me try this again. And we're gonna see how good this, this make setup is if it knows to do the right thing, because I've already created some of these directories, but no, it might do nothing. Um, check infra. All right. Terraform has been successfully initialized. So now if I go into Terraform and I have my main tf, I still do not have a Terraform directory.
Matt Godbolt: A dot terraform directory dot terraform that directory. What, what was the, what is the dot Terraform directory for?
Ben Rady: So that is gonna be Terraform init. So when I run Terraform init, I'm expecting it to create a dot Terraform directory. Is
Matt Godbolt: It created it in the root. Did he do the dot should ch d No.
Ben Rady: Yeah, I have it. Terraform in it is CURDIR slash Terraform slash terraform. Am I just not looking at this,
Matt Godbolt: Uh,
Ben Rady: Correctly? Maybe it's like, because there's nothing to do cuz there's nothing in the main do tf it just doesn't bother to create it.
Matt Godbolt: Yeah, almost certainly. It doesn't know what versions of anything to pull down because the main thing it puts in there is all the plugins that are gonna be like, here's the AWS provider and here's the whatever provider. So
Ben Rady: Let's Yeah. Do a very minimal thing in there then. Yeah.
Matt Godbolt: Uh, the most you, you're gonna need to have the Terraform thing and you need to have the state, um, management Yes. Part the thing that says this is where the state lives, which should probably be another Yeah. Another bucket somewhere. Like it always is in my world, dynamodb
Ben Rady: Dv. Yeah. But definitely the best way to do that is with you store it in a bucket. Um, I have sort of gotten away with not doing that in cases, but like yeah. That's just, that's not the best. All right. I'm
Matt Godbolt: Just, if you're gonna have to solve the credentials problem anyway for like me being to apply this, then
Ben Rady: Yeah. Yes. I mean at this point I'm just trying to get to something where we can test this interest subjectively where I'm gonna commit this and push it and, and then I'm
Matt Godbolt: Gonna take down and
Ben Rady: Then I'm
Matt Godbolt: Gonna put it down
Ben Rady: And make sure that I didn't screw anything up,
Matt Godbolt: Which has the credentials. Sort of caveat by, for what it's worth, I think I have AWS credentials. I'm gonna have to check, take a look.
Ben Rady: Well, if all I'm doing, I mean we're gonna have to solve that problem eventually, but if all I'm doing is just trying to initialize this directory,
Matt Godbolt: Then that will be intersubjective. You can
Ben Rady: Just hopefully just do
Matt Godbolt: That. Absolutely. Yeah.
Ben Rady: Okay. Installing the backend, installing provider plugins.
Matt Godbolt: Love it.
Ben Rady: Finding hash cor AWS versions, matching blah, installing AWS matching, blah. And now inside of the Terraform directory, do I have, I do! I have a dot terraform directory.
Matt Godbolt: drum roll
Ben Rady: And I have that lock file that you were talking about. So let me go take a look at this gitignore and see what I did, at least in here. So in here I am ignoring a bunch of things. I'm ignoring the dot tools directory, which makes sense. I want, you know, I don't wanna check in any of these tools. Um,
Matt Godbolt: The how ironic, ironically we could actually, because we both are linux, we could just check it. Terraform, you could, that would work, solve that whole thing. But you've done a nice thing for downloading it and changing it and upgrading it, all that stuff. So let's keep it.
Ben Rady: Um, all right. And then I want to say terraform.tf state, uh, I've got an s thing in here. I don't think I need that. Terraform. And then we're not doing anything with Terraform plan. So Yeah, so the three getting ignores, I'm adding are the dot tools directory terraform slash Do I need that one? I was gonna say Terraform slash TF State, but I don't actually think I need that yet, so I'm not gonna put that in. And then, okay. Terraform slash dot Terraform, which will be that Terraform initialized directory. And I think
Matt Godbolt: I'm just looking at my own final
Ben Rady: Do it. Oh, I got, yeah,
Matt Godbolt: That's the whole dot Terraform directory is all I've ignore in mine. And there is a dot terraform dot lock hcl, which I've accidentally been checking in, which probably I don't need to, we should, maybe I have to Google that.
Ben Rady: No, I actually think you are supposed to check in the lock file because it's like
Matt Godbolt: A lock file for when you do upgrade. Yeah, yeah, yeah, yeah, yeah. Okay then that's fine. The dot terraform dot lock hcl, I think you is like a stanza is when it's solved the Yeah. The constraints. You want everyone to be agreeing on what version Yes.
Ben Rady: Check I, yes. I have that file locked into or checked into my other project. So I think that's actually correct. Okay. Okay. So I think I've got to the point now where I can push this.
Matt Godbolt: All right
Ben Rady: And you can give it a try. So let me say to go up into skeleton Terraform config with make target with With infra make Target. And you know, we can rename this later, but this is what I got right now. Okay. So that's pushed.
Matt Godbolt: All right. Git pull, I see some things type make and there's nothing in the list because you didn't add the help because you're a bad person.
Ben Rady: For No, I didn't. It's not, it's not cool enough for that list yet.
Matt Godbolt: And I'm gonna do make what, what, what should you suggest? Make Terraform
Ben Rady: Make infra or
Matt Godbolt: Make make Infra.
Ben Rady: Yeah, that's in, like I said, we're gonna have to rename that later. And that's probably not even gonna be the, like we're gonna need like a make plan and a make apply like Terraform Plan, Terraform apply I think would be good targets, but they're gonna have like base things that sort of maybe did do something.
Matt Godbolt: It archived terraform.zip and Inflated. It made, made a, a Terraform sub directory, uh, which I guess we don't need to make it, you know, that maker is probably unnecessary cuz in you've made Oh
Ben Rady: Duh, I've got the main file in there. Yeah, thank you. Yeah, perfect.
Matt Godbolt: That's the whole point of that. But yeah, so we can get rid of that. Check that out. It says Initializing the backing initializing provider plugins Terraform has been successfully initialized everything. Quote just worked.
Ben Rady: Awesome.
Matt Godbolt: You may now begin working with Terraform. It tells me. So now we are inside that Terraform directory. I could do Terraform plan and it should do absolutely nothing. I know you wanna do makefile nonsense, but I like, I know old school, I'm just gonna go in there. No changes your infrastructure matches the configuration. That is the no infrastructure correctly matches the No configuration. Hooray
Ben Rady: . Cool. Uh, all right. I'm gonna do a very small commit to remove that unnecessary mkdir
Matt Godbolt: I'll allow it
Ben Rady: Uh, but that requires that I spell unnecessary correctly. So I'm just gonna say unneeded,
Matt Godbolt: oh, that's difficult. Yeah.
Ben Rady: ,
Matt Godbolt: That's a pragmatic solution to the problem.
Ben Rady: Okay. All right. I'll push
Matt Godbolt: That. Well, my laptop fan has just kicked on, so when I edit this, I'm gonna have a big old hiss in the background.
Ben Rady: Ah, sorry.
Matt Godbolt: Listener.
Ben Rady: Sorry listener. Okay. Okay.
Matt Godbolt: I'm sorry. Listener also that you maybe are expecting a podcast and instead you're hearing this debugging
Ben Rady: Session. We should just call this episode Yak shaving. That's what . Yeah, .
Matt Godbolt: Alright. So,
Ben Rady: Um, all right.
Matt Godbolt: The next thing that might be useful to do is either we um, we, we set up the, the, the, this Terraform Terraform State Storage mm-hmm.
Matt Godbolt: And then work out credentials, which is a pain because I'll ask them, God knows how hard it is to do AWS credentials, especially when we're kind of up against the clock. We've got about like 15 minutes on my calendar before I'm supposed to be doing something else, but I can probably ignore that. Um, or we just don't worry about the intersubjectivity of the next bit and get the Route 53 stuff done with your own credentials on your machine so that we know that in principle it's going to work and then see where we get out from there. Yeah.
Ben Rady: That maybe that's a place to, to start, because I guess if this is gonna be like really onerous, there's a chance we might just want to bail on automating this entirely. Right. And just be like, you know what, we're just gonna do this one in the console cause I'm not gonna spend 20 hours getting this right.
Matt Godbolt: I don't think it's gonna be that bad though. Like, I mean, let me, let me go. So if we go to Compiler Explorer, even, let me go to.dot compile devey compiler. No, infra Terraform, sorry. There's, there's a billion-ty files in here.
Matt Godbolt: Uhhhh, one of which is, is there a Route 53 tf? Yes. Haha. So I have, okay.
Ben Rady: Okay.
Matt Godbolt: A route 53 we can copy from. And let me just, I will send you the link. Oh, and I'll just hit my microphone.
Ben Rady: Sorry. Oh, wait a second. Don't we want to do like a Terraform import?
Matt Godbolt: You can, but it doesn't, it just says, now I manage that resource.
Ben Rady: Oh,
Matt Godbolt: It doesn't, but what I typically do is I manage, I import the resource, so I, you have to make an empty resource and then you import it and then you do Terraform plan. It says, oh, I would delete all of these things. You're like, great. Those are the things I'm gonna put into the stanza in the tf, so you can definitely do it.
Ben Rady: Okay. Yeah. Um,
Matt Godbolt: It's pretty straightforward though. Um, like,
Ben Rady: Yeah, no, no, I, I, so I hear what you're saying, like let's just get the Route 53 stuff in there cuz it's new Yes. And worry about everything else later.
Matt Godbolt: I think so we can definitely import the bucket stuff and we, that's a pretty straightforward thing to do, but we are gonna need to be able to store the Terraform state first and foremost. And then the Route 53 stuff we can just add and apply and then you should be able to refresh the console and go, this looks good. And then we should be able to dig at Route 53 or whatever it is. Yeah. See that they're there. And then maybe that's where we can, I mean
Ben Rady: We could just do local state for now and then I can move it into a bucket later.
Matt Godbolt: Yeah. Yeah. I dunno how easy moving it is, but I It's fine. I think
Ben Rady: It's, I think it's just, I feel like I've done this once already, but,
Matt Godbolt: But I mean it's, it should be straightforward. Like, uh, hang on, let me just see what my own crap is. Main dot TF backend, S3 Key bucket. That's what it is. It's just a backend and then a bucket name, which can be something you we we'll call like, you know, infra dot twos compliment.org. just a region and a key and then we're done. Mm-hmm. . Okay. And it's in the Terraform stanza.
Ben Rady: Yes.
Matt Godbolt: So I'm gonna Yes. Paste this into, oh my God, I don't even know where to paste it into, uh, uh, discord, .
Ben Rady: Yeah. That's probably the best place.
Matt Godbolt: So that is what I have in the com. I mean, obviously it's open sourcing, you gonna grab it. Um, so you, you already put a provider in there for AWS of you or have you not? I,
Ben Rady: Yeah, no, that's in there. Although the version that you're using is way newer than the version that I have so much. Yeah,
Matt Godbolt: Yeah. Yeah. I mean that's unimportant really. Um,
Ben Rady: Now is the time to do it. Not gonna get any easier than now
Matt Godbolt: The latest version, you know? Yeah. And then that is what I have for like the provider block above that. And that's kinda like my main.tf that is like the hey is all the versions, this is the whatever's and this is where I want you to store your state. And I think that's all it took, I think now I think there's something you can do with DynamoDB for locking and all that kind of crap, but I
Ben Rady: Haven't done that. No, I see what you're saying there. So do you have another piece of Terraform somewhere that's actually creating that bucket?
Matt Godbolt: No, no, no. The, the back end knows how to do that. That's like, you know, otherwise you've gotta chicken and egg situation. How do you create the bucket where this, where it's gonna store its state before you've stored the state to be able to create the bucket. So, okay. Okay. So I think, I think you can just type put anything in there. All right.
Ben Rady: Well
Matt Godbolt: I'm just, or in fact we already have the bucket actually as it happens. We can just use the, the existing bucket. Although the No, we don't wanna put the TF state there cause it actually goes to a website. So let's not do that. Yeah, yeah.
Ben Rady: , I can't do that. So let's, okay, so bucket
Matt Godbolt: State something two compliment.org or can be the name of the bucket. This is cool. If it works,
Ben Rady: If it works
Matt Godbolt: Retrospectively it'll be cool. But right now it might just be, as you say, another yak.
Ben Rady: Okay.
Matt Godbolt: We shaved this yak and then inside the yak was another yak
Ben Rady: . It's
Matt Godbolt: Like Russian yaks the stack and Russian and do Ys yak stack Uhhuh,
Ben Rady: , uh, alright. Right. Oh boy. And then I, I'm gonna do this and I'm gonna wantonly steal from this example that you have because it's I think
Matt Godbolt: Probably better than that my friend is what it's all about.
Ben Rady: Should I have typed enough times to be able to spell correctly? Okay. Um, okay. All right. So now if I run this, I would almost expect a credential, error. Cause I don't think I have any, I I tend to not just throw, have
Matt Godbolt: Them one in the global
Ben Rady: Credentials into like my Bash RC or anything like that. And I actually really kind of don't like the fact that the AWS client will discover credentials that you put in, um, your, you know, home directory. So I don't really like having a default profile
Matt Godbolt: There, there are some actually, like, there's some like aws pseudo as in S U D O, pseudo, not pseudo as in PS U E D O mm-hmm like scripts and things you can get, which kind of make it so you can run a shell on as a, this set of credentials, which is kind of a nice way of doing it effectively is just a canonified way of setting all those horrendous environment variables. But it also, some of 'em will support the things where you need to, um, log in as yourself and then effectively change role to like developer and this, that kind of stuff. But right now, um, yeah, whatever you wanna do, you can export some credential files. Um, I typically do use the horrible global variable pattern, which is nasty, but I have them, I have different credentials for different projects and I specified the credentials with the one environment variable that says no, these are the ones. I mean this time.
Ben Rady: Yeah, yeah, yeah. Um, okay. So unfortunately I don't think I've made this makefile quite mature enough to where it will detect changes to the TF and rebuild. Um, which I think I have in number. Well
Matt Godbolt: That's typically I make those things like if it's a, it's more, it is a phony target. So it's not that there are changes, it's because like, who knows? The thing is that Terraform, there are two sources of changes in Terraform. One is in the dot TFR that you edited and the other one is some clown changed it on the console and you wanna make sure that actually you apply that. Yeah,
Ben Rady: Yeah, yeah. Okay. So yeah. So I need to change that.
Matt Godbolt: So don't worry about that for now we can
Ben Rady: No, no I got this, I got this. So I'm gonna change that in for Target to actually, I'm just gonna leave it the way it is and then I'm gonna make the Terraform apply Target or probably Terraform plan to start, uh, target. That's gonna be phony. Yeah. This is what we want do. And then, um, this is just gonna depend. I am gonna get rid of the infra target and replace it with the real one. Alright. Um, but it's plan, that's what I want is Terraform plan and then I just wanna see the plan, right. Which should be nothing initially. Okay. So then we're gonna do that and then I'm going to use my variable name for the Terraform executable instead of trying to guess where it might be. Be
Matt Godbolt: Naked Terra, you're much, see I'd be in there just typing Terraform and then I'd do make fell afterwards. This is quite, I'm interested in how your process, how how, um, rigorous you are actually saying no, I never want to know.
Ben Rady: Yeah,
Matt Godbolt: Right.
Ben Rady: Or right. I don't know. I've just done this enough times where it's like I just really like the result from it because it's like there's a very high probability that when I get to the end of this it will actually just work mm-hmm. . Whereas if I, if I sort of go back and try to do this after the fact, like because you didn't build it up interactively and you never had a chance to verify each step, because the nature of make is this sort of like, you know, branching tree of dependent tasks. Mm-hmm. , if you haven't actually walked that tree one branch at a time and make, make sure that each one worked, then it, you tend to wind up in these situations where it like mostly works except this situation. Um, but anyway, I digress. So if I.
Matt Godbolt: No, no worries.
Ben Rady: Uh, make Terraform plan, what the hell does it do no configuration files, right? Because I didn't use the dash C
Matt Godbolt: dash chdir. Yeah. Yeah. You can bake that into the Terraform command if you're being really horrible.
Ben Rady: Oh yeah, good point.
Matt Godbolt: I don't as in the, the variable Terraform that's slightly, Hey, run the Terraform in the way that I need you to run Terraform rather than being the executable is, is Terraform space dash chk. Mm-hmm.
Ben Rady: ,
Matt Godbolt: Which is kind of naughty, but, but nice. Mm-hmm.
Ben Rady: . Uh, okay, so I'm getting backend initialization required. Please run Terraform init. So my guess is that because I've already initialized this once with the wrong backend, it's now complete. It
Matt Godbolt: Needs to be reinitialized. This is So
Ben Rady: I'm gonna do the thing.
Matt Godbolt: So honestly, I've had enough of these problems where I, this is one of the reasons why I haven't driven a lot of the Terraform by make, cuz this like knowing the subtleties of when you need to do the re init and is hard, so often you end up with makefiles that always re init all the time. And then that's painful. It means like the first 25 lines of every time you do a make thing, is it re initializing the same thing? Mm-hmm. .
Ben Rady: Yeah. I mean I've, I've definitely gotten this pretty solid on some of the other projects that I've done, so I'm not worried about that. Yeah, yeah. for this, I'm just gonna delete the directory and rebuild it. Um, sounds great because once you get that background right, it should be pretty stable. But now I'm getting a different error, which is unsupported dot terraform core version
Matt Godbolt: The dot Terraform directory you mean?
Ben Rady: Which is why, why do you this configuration
Matt Godbolt: What version of Terraform do you have?
Ben Rady: probably like an ancient one compared to what you were using? I have it set at 1.0.11.
Matt Godbolt: Let me have a look. I am on 1.3.2
Ben Rady: Well I will change the
Matt Godbolt: Latest version is 1.3.7 though. So why the help
Ben Rady: 1 37? Yeah. Love it. Okay, so now this is another great exercise. And
Matt Godbolt: Here's the funny thing, like when you check this in, if I try and test it, I can actually make a dot ozy dot yaml in that directory to point it at version 1.3.7 just for when I'm running in that, in that directory, which is just a me a saying. ozy pretty cool too. Mm-hmm. . Anyway. Um, we're rewriting ozy in Rust as well for, for anyone who's actually using it.
Ben Rady: I, which I I really like that idea by the way.
Matt Godbolt: Anyway, sorry. We're, we're getting all all up in the confusion.
Ben Rady: Uh, okay. So I ran Make Terraform plan installed the new version of Terraform automatically, which was cool. And now it's saying error configuring s3 backend no valid credential, which is the error that I expected to see. Awesome. Right. Okay. All right. So now I can go and
Matt Godbolt: I wonder, you know, if you can specify where the credentials are expected to, to be.
Ben Rady: Yes.
Matt Godbolt: And then you can say, well, and then we can gitignore them and then we can make it so that there's a make file target that says, you know, you need to put the, you need to go get the credentials from exogenous place. Mm-hmm. And put them here. Mm-hmm. and then it'll all be like nice. Mm-hmm. . Yeah. I can't believe this is a lot of effort for like Well we could just, you could just paste it into discord that I could type it in .
Ben Rady: Oh man. But that's
Matt Godbolt: How we go, that's how we roll.
Ben Rady: Yeah. Yeah. No, I think I'm gonna, I think I might do that. Um, cuz I, I actually have a, another example of doing that in a different project.
Matt Godbolt: Oh. Even better.
Ben Rady: Uh, where I say, uh, AWS shared credentials file, I think is the name of the environment variable.
Matt Godbolt: Nice.
Ben Rady: And so, and then I have a make target that is just that file and the, it just prints out like, Hey, you need to,
Matt Godbolt: Yeah, I can't make this file because Yes.
Ben Rady: , uhhuh, . Okay. So I'm gonna do that. Lets
Matt Godbolt: Do we have vault, we can make Avol and then no duh. Now
Ben Rady: We're right. Okay.
Matt Godbolt: Oh, Myre
Ben Rady: Done here. The, the, the yaks The yaks are stacking. The yaks are stacking. Yeah. Um, one pause. Yes. All right. So yeah, I'm gonna grab that. And then AWS shared credentials file is that, and then, um, this needs to be exported and that's why we had that AWS directory in the Ignore, because that is where it's storing the credentials. Um, so I will add that again because I absolutely, positively do not wanna check these credentials into uh, yes. git, under any circumstances.
Matt Godbolt: I think something will alert you pretty quickly to it, but it's not worth testing it. Yeah. ,
Ben Rady: Let's not find out, actually there
Matt Godbolt: Is, there's a pre-commit hook for, for this. So if we do set up pre-commit, oh, as in the Python pre-commit thing, one of the hooks is, you know, it has an obvious look for like mm-hmm. API looking keys and says, no, you probably didn't wanna do that. Mm-hmm. . Which is, which is nice.
Ben Rady: Uh, okay, so then I would assume I want my Terraform init to depend on this AWS shared credentials file, because otherwise Terraforms not gonna be able to do very much for you. Mm-hmm. . And then hopefully if I run this again, I should see something else. What the hell did it just do overriding recipe? Oh, do I, did I
Matt Godbolt: Done a typo or braino?
Ben Rady: What?
Matt Godbolt: You got two recipes with the same
Ben Rady: Name? Yeah. Yeah. I think that's what's going on here.
Matt Godbolt: I'm hungry.
Ben Rady: Do that Uh, oh. We have this already. This is already in the make file already. Oh. Get
Matt Godbolt: Out. We,
Ben Rady: We did this.
Matt Godbolt: I'm gonna, I've lost my mouse pointer. There it is.
Ben Rady: We totally did this.
Matt Godbolt: I'm sitting funny cuz I'm trying to keep my mouth there, the microphone. But it means that now I'm
Ben Rady: AWS shared credentials file is already in here. We did this already for something else. Okay. Awesome. What if I just Oh, use that? Yes. You must get credentials and save them here. Oh yeah.
Matt Godbolt: What? Because that's how we tested some of, I'm sure. Hang on.
Ben Rady: Yeah, yeah.
Matt Godbolt: Config file.
Ben Rady: It's like line 29 in the make file.
Matt Godbolt: I don't have that. Which is fine.
Ben Rady: How do you not have that?
Matt Godbolt: Credentials? You must get credentials from Google Drive and save them in, blah blah, blah, blah. Yeah, that, look at that, look at that. Ben Ben Rady is a clever person, so I'm gonna go to Google Drive and go get them right now
Ben Rady: About that. Did I put them there? Maybe I did put them there. I think I put them
Matt Godbolt: There. Anyway. You worry about that. I'm gonna get them. Yeah. And then we can do an intersubjective test at the end of this, this all. Yes. I like this. Pat ourselves on the back and say mm-hmm. . Hey, we we'll, clever than we thought. Yeah.
Ben Rady: Yeah. I would infer I have zero memory of anything that I've ever done, but I in would infer from this hint that I left my future self, that I made some, uh, two compliments, credentials.
Matt Godbolt: There's a credentials directory in, in the shared gdrive where we steal all this
Ben Rady: Stuff. Look at, look at me. There
Matt Godbolt: It is. Aws December the 14th, 2020 by Ben Rady.
Ben Rady: Wonderful.
Matt Godbolt: Esquire.
Ben Rady: Wonderful.
Matt Godbolt: I'm gonna download those credentials right now. Yes. And I'm gonna move them into place. mkdir, aws,
Ben Rady: Wunderbar,
Matt Godbolt: Wunderbar.
Ben Rady: I'm actually gonna do that too. I don't, don't even know that I have a copy of those credentials, locally.
Matt Godbolt: Oh, that's not, I just typed vi instead of mv. That is just how stupid I am. Okay. Make credentials, nothing to be done for credentials. Look at that. Okay. How would I test that they work?
Ben Rady: Uh, well, what are they used in?
Matt Godbolt: I shared credentials, file credentials. Credentials.
Ben Rady: I guess it's an environment variable, so it's gonna get used by like if you did a deploy, which Yeah. Maybe don't wanna do, but that would be one way to test. Um, I guess you could try to do like an s3 ls, uh, uh, on
Matt Godbolt: The, but I'd have to hack that in because it would need to set the things Yeah.
Ben Rady: Up.
Matt Godbolt: Yeah. No, no, no worries. Anyway, I, I'm gonna quickly, I I will locally hack that just so that I can do it just to see that we aren't gonna fail while you try and get the next step done, right?
Ben Rady: Oh
Matt Godbolt: Yeah. No, no. We have a a website info. Okay. Put bucket crate bucket. Oh, no, no. Yeah, I thought we had on ls.
Ben Rady: I mean, really like the test target list
Matt Godbolt: Bucket. There we are. Make list. You should put it in
Ben Rady: The test target,
Matt Godbolt: Make list bucket, and it works. So,
Ben Rady: Oh, list bucket. Yeah.
Matt Godbolt: Okay. We, we've done this already. Wonderful. I just, it's
Ben Rady: All coming back. Oh. Oh my god.
Matt Godbolt: Dear listener, we are not the smartest. It turns out,
Ben Rady: Uh, programming is a detective novel where you are both the detective and the murderer.
Matt Godbolt: So very true, my friend. All right. What we got going on now then? Uh, so,
Ben Rady: Okay.
Matt Godbolt: You are just about to check in. The thing that points the Terraform config at this preexisting configuration, having made it also our, uh, terraform init depend upon the credentials. Mm-hmm. in the makefile.
Ben Rady: Mm-hmm. . I'm still having trouble. W where is it's Twos compliment? Is that where the credentials are? Ben? Two compliment. I I haven't found the credentials yet. Oh.
Matt Godbolt: Oh. Um, it's in The Twos Compliment drive under credentials. Sorry. Under, yeah. Uh, credentials. Aws. Oh. I will paste a link to you right now in the chat, which says, chat with everyone. Yes,
Ben Rady: I found it. I found it.
Matt Godbolt: You found it.
Ben Rady: All right. Wonderful. Okay.
Matt Godbolt: I don't even know what that is, credentials, but you know, I'm gonna use it now to find all your secret projects. Mm-hmm. .
Ben Rady: Yeah. I, I had completely lost track of the fact that the Twos Compliment Drive was a drive you shared with me. I was looking at my own folders and I couldn't find it.
Matt Godbolt: Oh, yeah, yeah, yeah. I, that's, that's me. That is,
Ben Rady: Okay, cool. All right. So download it there and now. I
Matt Godbolt: See, so you were waiting to get your cred.
Ben Rady: Yeah. I just wanna like check to see if this actually works cause I haven't, you know, tried it. Um, mkdir AWS and then copy from downloads credentials to here. And I'm just gonna take a look at this thing real quick to make sure that it is the credentials that I expected to be,
Matt Godbolt: Let me have a look.
Ben Rady: Uh, nope, I don't. Cause I put in the wrong directory.
Matt Godbolt: Can you just read out the, uh, secret access key to
Ben Rady: Me? Uhhuh? ? Yep.
Matt Godbolt: To check ? Yep.
Ben Rady: It is, uh, no, I'm not gonna do that joke. All right. , you're
Matt Godbolt: Not even gonna do .
Ben Rady: All right, cool. Yes, that looks very reasonable. So now I should be able to do the same thing that you just did and say list bucket.
Matt Godbolt: make list bucket.
Ben Rady: And that worked. Awesome. Yay. So now if I do make Terraform plan, okay, so if I get Terraform,
Matt Godbolt: These credentials may not have the right permissions to do whatever, but
Ben Rady: So when I do this, it says S3 bucket does not exist.
Matt Godbolt: Okay. Maybe we do need to create this and then maybe permissions, things about that for the state.
Ben Rady: I mean, it would be very convenient if we could store these things in the same bucket. So one yak that, oh my God. The yaks, um, the yak that I would, that the, the programmer wants to shave right now is moving the public content into a sub key
Matt Godbolt: Yeah.
Ben Rady: So that we can store other stuff in this bucket safely. Um, but I don't want to do that right now.
Matt Godbolt: Buckets are free.
Ben Rady: Yeah.
Matt Godbolt: Buckets are absolutely free and it's much, much safer to have a top level bucket thing and just say like, this is, there's, it can't even be public. Yeah. So I have no problems with you going into the console for this one task. Yep.
Ben Rady: Yep. If
Matt Godbolt: That's what it takes. I'm sorry, I I did think it could create them, but maybe you could, I
Ben Rady: Mean, maybe there is a way I would, I would expect it to be able to do it, but maybe not
Matt Godbolt: So dear listener as well, if you're not sure what we're talking about here. So Terraform is a way, and I'm gonna pad this while be Ben fiddles around setting it up, right? Um, Terraform takes a textual description of the infrastructure and assets you would like to exist, and it makes sure that they are created, which is wonderful. It can make, uh, ec2 instances, it can make, you know, anything Google Cloud, any kind of cloud provider, it can make, um, all the resources that live in there, but it needs some external state because it needs to be able to map something that you locally have called this bucket to the identifier that Amazon calls that bucket, which will be some horrific ARN and some giant long name for it that uniquely identifies the resource in Amazon's world or Google's world or whatever.
Matt Godbolt: And so there is a state file which says, this is what the code says it is, and this is what it got created as on the remote end. So that I can, when you then make apply again, it doesn't try and create another bucket, it goes, no, I already have that bucket. And then it can query that bucket and say, does it have the parameters that I expect it to have and it can make a modification instead. So that's what this state file is, and it has to live outside of, well, it's better if it lives outside of source control because if you've got multiple people, um, changing it and applying, um, in different locations, then they, you have this kind of like race condition where until you've checked it in the state is not valid for the other person.
Ben Rady: Perfect.
Matt Godbolt: Have we, uh, have I sufficiently padded to
Ben Rady: Get to this? Yes. Yes. And I realized that I'm gonna have to use a slightly different approach, um, for Okay. Right. Of course. I have to add access to the bucket. I created the bucket. Um, the, the backend attempted to initialize and it got an, an access error, which makes sense because I haven't actually,
Matt Godbolt: We don't know quite what the credentials, I don't know what you did with these credentials in terms of like this The creation
Ben Rady: No, I just,
Matt Godbolt: The perennial problem of like, how do you. Grant the granter. Yes.
Ben Rady: Um, yeah. I, I think I have some trade craft on this for that I can pull into this, but I'm not gonna shave that yak right now. What I'm gonna do right now is I'm gonna go into this user
Matt Godbolt: And just say, Hey, you can do the thing.
Ben Rady: Yeah. Where the hell is this user?
Matt Godbolt: Oh, the IAM thing is a painter. I
Ben Rady: Only, oh duh. I clicked on user groups instead of users. I'm like, where are my users?
Matt Godbolt: That'll be why
Ben Rady: No, this is Yes. Uh oh. Interesting. You want to guess what the name of this user is?
Matt Godbolt: Uh oh,
Ben Rady: this is funny.
Matt Godbolt: Twos compliment would be my guess, but.
Ben Rady: Incorrect!
Matt Godbolt: Oh,
Ben Rady: What is the name of this user?
Matt Godbolt: Uh, is it, uh, oh, what is it? Something about argument that we had some other name
Ben Rady: Constant arguments, which was the constant
Matt Godbolt: Arguments, which was, so anyone who's made it this far into the episode gets a little bonus treat of learning what this podcast might have been called, which was constant arguments. How funny.
Ben Rady: Oh my god. So funny. Okay. Um, yeah. Attached to existing policies and we're gonna go, uh, s3, uh, so I'll come back later and I will create a custom policy that. Only but for now, yes.
Matt Godbolt: So what Ben is doing is granting the user for which these credentials allows us to log in as, or effectively access, uh, Amazon as. It's granting it the rights to rights to a separate bucket which won't be publicly available, unlike the website. So that Ben and I can store this state somewhere. Mm-hmm.
Ben Rady: , uh, okay. So next error, uh, failed to query available provider packages could not retrieve the list of available versions from provider hashicorp. Block provider does not match constraint. Oh.
Matt Godbolt: Oh, that sounds like another,
Ben Rady: So I have re initialized Terraform and I'm still getting this error.
Matt Godbolt: Should you do Terraform uh, dash init, dash update I think is the other thing which allows it to bring things forward.
Ben Rady: Okay. I think. Okay.
Matt Godbolt: Give it a go. This is now we're into hand waving Magical. Yeah.
Ben Rady: Voodoo. Yeah. Yeah. In it. I'm trying the same So Dash update?
Matt Godbolt: I think so. Uh oh. Upgrade dash, dash upgrade
Ben Rady: Upgrade.
Matt Godbolt: No, that's not right. Oh no, hang on init
Ben Rady: Because I could just blow away the lock file cause there's nothing useful in there yet.
Matt Godbolt: That's possible. Yeah. You wanna give that a go? Yeah. It is terraform init dash dash upgrade. We'll allow it to do that.
Ben Rady: Oh well if that is what it does, I'd rather use that. So let's try that. Okay.
Matt Godbolt: We'll give it a go. Does it make any difference.
Ben Rady: It's, it's working.
Matt Godbolt: Okay. Installing. So now we've made the lock files betterer.
Ben Rady: Yep, exactly.
Matt Godbolt: But the thing is about that dash, dash dash upgrade is you don't wanna check that in as being something cuz it will just keep pushing everything forward all the time.
Ben Rady: Mm, yeah. Okay. Good point.
Matt Godbolt: I think cuz it would've updated. Yeah. Yeah.
Ben Rady: Okay. Anyway, so we have a new lock file, which I'm assuming contains our information about the bucket. Uh, and I'm just gonna take a look at that real fast. Uh, I guess it's just the registry a s corporate registry. What actually contains the information about the bucket. Oh duh. It's the main, like I, yes, of course. That's what you put in the main, dot TF file. You put the name of the bucket. Awesome. All right. So I think that actually worked. So I think we're finally at the point now where I can commit this and you can give it a try,
Matt Godbolt: Right? I am just ha which version did you do? Uh, 1.3.7 I think of Terraform, right?
Ben Rady: One through seven. Yeah. Mm-hmm.
Matt Godbolt: . Okay. I am just adding an ozy dot yaml Okay. Into the two compliment things. So that I will Yeah, yeah.
Ben Rady: Terraform No, that's cool.
Matt Godbolt: Dash version. Lemme
Ben Rady: Make sure I'm not doing anything.
Matt Godbolt: Installing Terraform 13. Oh no, no, wait a second. Sorry. No, uh, it manages Terraform itself, doesn't it? Sorry. Oh yeah, I don't need to do this because you're doing it in the make file. That's the whole point of it. Right?
Ben Rady: I mean though
Matt Godbolt: That said, it's convenient to be able to type, I had
Ben Rady: Kind of assumed that you wanted to just be able to run arbitrary Terraform commands in addition to this, the intersubjective stuff that lives in the make file. Alright, well, so I have no heartburn at all about it. Putting a ozy file on there. Check as well.
Matt Godbolt: Alright. Right. Have you checked this in?
Ben Rady: I'm about to, um, adding, uh, configuration for, um,
Matt Godbolt: You're spending far too long on the, um, the thinking carefully about the, the checking comment for us. I know it's a good, it's best practice to have a decent checking comment and you very rarely
Ben Rady: Pushed.
Matt Godbolt: Okay.
Ben Rady: Give that a, give that a try.
Matt Godbolt: Pushed the ozy yaml as well. Okay, so I'm gonna do make terraform plan. It's inflating a new version of Terraform. It's initializing, the provider plugins says it's reusing previous versions of some things, but installing some newer things of other things and your infrastructure matches the configuration. Yay.
Ben Rady: Also, we accomplished nothing ,
Matt Godbolt: We've successfully done nothing, but it does mean that we should now be able to literally quote Yeah, just type a tiny bit, make a route 53 TF or something in that directory. Yep. I'm gonna try that right now. Okay.
Ben Rady: I I, so this is, I actually, I think I actually need to bounce here in a couple of minutes, but let's, I wanna see what you do, do your thing and then we'll Okay. And then we'll see what happens.
Matt Godbolt: I, okay. I'm literally gonna copy infra from Compiler Explorer, Terraform , route53.tf ah TF into the telephone directory. And then I'm gonna vi it and I'm gonna destroy everything that's in it other than,
Matt Godbolt: Module zone id. Ah, no, we have to get zone IDs and other things, plus the user doesn't
Ben Rady: Have permissions to do any of this stuff anyway, so we're
Matt Godbolt: Gonna Right then we are done for now. I think, well, we have successfully done nothing, we did not solve the original problem in any meaningful way. Um, we haven't even solved the, the simplified version of the problem where it's like we're not gonna get it to resolve. First of all, we're just gonna set up AWS so it's ready to resolve Uhhuh , but we have got a really shiny little place to put that change when we get it, so Yeah. Cool. It could be worse, so Yeah. All right. then. I think we should quit while we're not behind any further
Ben Rady: uhhuh Right. Before, before we create more work for us to do .
Matt Godbolt: Right. But just to summarize, we have a place to hang future infrastructural changes that should be straightforward. We've gotta sort out some permissioning problems with the user so we can create a Route 53 thing. We're gonna have to actually make a Route 53 zone, which should be pretty straightforward. I have got the, so the, the compiler explorer stuff is in a module because I have multiple domains and so I set them all up the same with a module, which is cool stuff, but unnecessarily complicated for what we need. Um, then we can, with a zone which will be the twos compliment org, we can add some records, one of which will be, this is a, an alias record I think is what they call it to Yes, that's right. The cloud front thing, which we can also bring in and then we can just actually point them to each other. Mm-hmm. and Terraform import it and then it is like, Hey, I mean that cloud front id, I don't even know, need to know what it is. It's just the one that's defined and declared over here. And then once that's all applied and affixed, we can change your DNS at your current provider mm-hmm. to say the name servers at Amazon's one and it will just work. Yes. And then finally I can curl HTTPS who cumber org or discover that the certificate doesn't work for that.
Ben Rady: All right, friend. Uh, well this was a fun adventure.
Matt Godbolt: This was an adventure and, uh, yeah, I'll, this will be a fun one to edit. I think this one will be the least edited ever. So, um, and it'll just be as it came out. So apologies to our, our listener for the lack of, uh, of us sounding more intelligent than we are, which is normally how I go try to in edit it. Perfect.
Ben Rady: I love it. Alright. Cool.
Matt Godbolt: All right. Until next time, my friend.
Ben Rady: Until Next time.