Matt Godbolt: Hey Ben.
Ben Rady: Hey Matt.
Matt Godbolt: So we were looking at the problem with our SSL certificate for uh, twoscompliment.org. In other words, you know, we wanted to be able to like host from just twoscompliment.org, not www.twoscompliment.org, which sounds straightforward. And through the miracle of podcasting, we recorded that many weeks ago. Our, our sort of, uh, attempts to fix it, but we never quite got there. And so I figure we should probably finish the job, try and get it so that our website's actually working, and, uh, everyone can laugh at how much we've forgotten between what may be back to back episodes as far as our listener is concerned. But what, for us, a month has passed.
Ben Rady: Mm-hmm. , the fact that you think we're gonna get this fixed today is very ambitious. And I like that. I like that attitude.
Matt Godbolt: Well, we've got half an hour or so. Let's give it a go.
Ben Rady: Let's see how far we can get at least.
Matt Godbolt: So we had a whole bunch of Terraform Me stuff was how we left
Ben Rady: It. Yeah. So I have, so right now, if I remember this correctly, our plan was to create an AWS Route 53 domain. And then change the domain to use, uh, like the wild card certificates?
Matt Godbolt: I think so, yeah. We could use a Wild Cert. Wild Card cert, or at least a cert that has multiple hosts listed, one of which could be a wild card, but it could just have dub dub dub and the no domain, which I believe is what Compiler Explorer does. But I can't remember, I think, I think actually Compiler Explorer has like empty and star dot godbolt org or whatever.
Ben Rady: So I have right now, so looking at this, so we had kind of terraformed some of this up before. Um, and right now there is a certificate that we have for www.twocomplement.org. And there's a little TODO here that should be, it says should be just twocompliment.org. And then I have another little TODO here that says Subject alternative names equals, and then square brackets quote star dot twoscompliment.org.
Matt Godbolt: Right.
Ben Rady: Um, and I have a little bit of four each magic in the Route 53 record that I think attempts to create a record for each of the things that it sees in the certificate. And I have this, I think because I have done this once before, and gotten this to work, and I copy pasta-ed some of that in here, but now I have zero memory of how it
Matt Godbolt: Will all fitted into together
Ben Rady: how we even got here, let alone how it works. So this is what happens when you put things down for a month and then,
Matt Godbolt: And then don't pick him up again. Right,
Ben Rady: Right. Uh, let me go check. I have a project on my GitHub that I think this maybe came from. So let me go see if that is even remotely true.
Matt Godbolt: And if so, we can crip from that.
Ben Rady: Yeah. And I guess I should try tastypenny.com. And that does work. And it is secure. So I have done this on that site. Yeah. My recipe tracking website called tastypenny. I have , I have done this
Matt Godbolt: Tasty Penny?
Ben Rady: Tasty Penny. Yeah. I don't even know where is that thing. Yeah. You know, it's like all recipe websites are terrible
Matt Godbolt: Because
Ben Rady: It's all like,
Matt Godbolt: They're not really recipes said websites. They're advertising websites that were trying to put as many adverts between the obvious thing you want, which is the damn recipe.
Ben Rady: Right. And try to, you know, get enough SEO from the text that they're putting on there about like, oh, I traveled to Paris three years ago and I had this wonderful, you know, whatever. Yeah. But yeah, I can't even find, am I like not logged in? Oh, I'm logged in as, yeah. Okay. That's what's going on there. I'm not,
Matt Godbolt: Oh, you on, I can't see
Ben Rady: My right now, repositories. Oh, no,
Matt Godbolt: I see you're logged to GitHub.
Ben Rady: Right. I'm trying to find where this stuff came from. So I'm going to GitHub, but I'm logged in as a different user, so I can't see my press.
Matt Godbolt: Whoa. You have more than one user.
Ben Rady: I do. I have an aquatic user for my official Aquatic things, and I
Matt Godbolt: Have Oh, I see. Oh, I,
Ben Rady: Me, which is less official.
Matt Godbolt: You're less official. I you're not the official Ben Rady.
Ben Rady: I am not the official Ben Rady. I am the, I'm just the casual Ben Rady.
Matt Godbolt: I see
Ben Rady: . Um, okay. So yes. Tastypenny. Uh, here's some Terraform. It says site. And Yes, I think that is exactly where that came from because that looks very similar except some commented out stuff.
Matt Godbolt: Uh, before we get too excited, if I go to Tastypenny, or if I could type Tastypenny. dot org? com?
Ben Rady: Com.
Matt Godbolt: Com. No dub dub dub. It is working. No ads, no junk, just Tasty apparently is the uh is the little, uh, byline for this and. Connection is secure, it says, and certificate is valid. And the co common name is tasty penny.com. And there you go. And looking at the, yeah, you just got a certificate and it only mentions tastypenny.com in this certificate. Now I'm gonna go to www dot tasty penny.com and connection is Secure Cookies, all the things. Maybe it redirected me then
Ben Rady: It might have.
Matt Godbolt: I see. But obviously in the interim, it, it was, it served up something which did not upset my browser. Yeah. In terms
Ben Rady: Of you could curl it if you wanted to know for sure.
Matt Godbolt: I certainly could, but yeah. That's awesome. Alright, so this is going to be a good thing to crib from because it works. That's what I'm checking is like, before we get all excited and changing it, let's just see that like we're heading the right way. Uhhuh and I will curl it actually.
Ben Rady: While we're going. Yeah. So yeah, when I do a curl dash v I see server certificate, subject, see on Tastypenny start date, expire date, which is uh, in May. Okay. We'll keep that in mind. . Uh, and then, uh,
Matt Godbolt: Picking up rocks
Ben Rady: Subject alt name host www tasty penny.com max. I see certs star tasty penny.com.
Matt Godbolt: Perfect. Okay. So that's a good
Ben Rady: Sure. Amazon
Matt Godbolt: Analog then of that. And we know what to look for when we, when we do it for twos compliment.
Ben Rady: Yeah. So, you know, the question with this is how do you fix the airplane while it's in the middle of the air? And, uh, I would hate to, uh, you know, apply some terraform change here that all of a sudden makes our podcast disappear for, you know, 24 hours or however long the DNS is poisoned or whatever it might be. You know,
Matt Godbolt: So how about this? Can we make a change straight away to change the TTL of the DNS down to like two days and just apply exactly as is, but with a really low ttl, which means that already, or like two hours, which means that we're starting to promote the idea that we're gonna screw this up and we wanna be able to undo it. Right. You know? Exactly. Yes. A good friend of mine once told me that if you can't test it properly, then at least make it cheap to roll back.
Ben Rady: Oh, yeah. Okay. That sounds like that guy was making **** up as he went along. . Uh, okay. Um, let's see here. So right now the Route 53 record is set to 60, which is in seconds, right? Oh,
Matt Godbolt: Right. I think so. Right. So we might be, uh, I mean DNS is it's own mysteries
Ben Rady: The question is, I don't think that currently AWS is the name server for twos compliment. I'm using the other provider.
Matt Godbolt: That means that we can make all the changes we like here and just use NS lookup with the server being, or host or dig or whatever the cool kids use these days, um, and test that it's doing the right thing.
Ben Rady: Right.
Matt Godbolt: So right now, I, how
Ben Rady: Would we, how would we confirm that that is true?
Matt Godbolt: I'm gonna do NS lookup and I'm gonna do set type equals any and I'm gonna do twos and sorry for my offensively loud keyboard twoscompliment.org. And it tells me non authoritative answer name server is ns2.hover.com and then,
Ben Rady: Yeah,
Matt Godbolt: That's blah, blah, blah, blah. And then address is 2 16, 40, 34, 41, whatever. Yeah. Okay. Yeah. Now, beautiful. If I were to set the server to be, do have you have, do you happen to have a a, a AWS DNS IP there?
Ben Rady: Uh, let me go see if I can do that. One moment please.
Matt Godbolt: And meanwhile, I'm looking at the Tasty Penny website going, this looks great. ,
Ben Rady: I wanna, I, I, I have some updates I wanna make to it. Uh, I want to make it a little bit more tablet friendly, cuz it's not right now, but it, you know, it has some good recipes on it. Uh, yeah, I wanna sign into the console. I don't know if you've got this thing recently where I've finally had to separate my amazon.com, you know, ordering
Matt Godbolt: Oh shopping,
Ben Rady: Shopping, shopping password and my AWS password. Yeah. Through a reminder of like, it used to be that Amazon was a bookstore.
Matt Godbolt: Yeah, that's right. Yeah, it is pretty bonkers. At one stage, actually, I had a problem where, um, I enabled two factor authentication on one or other of the two and it affected the other, even though they were supposedly independent. I think there's still some link between the two. They're different account names now. And I, that was the one and only time I ever spoke to an, an Amazon person on the phone while they were trying to reset it out. Oh, this is an interesting problem. I'm like, yes, yes it is. I can't log into either. And this is kind of panicking me right now. .
Ben Rady: Hmm. Uh, okay,
Matt Godbolt: Well this padding. Yeah. Has that allowed you to find the IP address of
Ben Rady: Yes, yes, yes. So I have the twos compliment name servers.
Matt Godbolt: Yep.
Ben Rady: There's four of them.
Matt Godbolt: Anyone will do,
Ben Rady: Let's go NS dash 60 eight.aws dns dash zero eight.com.
Matt Godbolt: Wow. That's a beautiful thing. Oh, the, so it has an IPv6 address. There you are. That's crazy. Okay, so now I've just said server that, and I'm typing twoscompliment.org again. And now it tells me, uh, the name servers are, now I can see the other DNS uh, servers at Amazon, which are like NS 1, 1 50 and one 1600, all this kind of crazy things. And apparently it has an address of 202.251.192.68 is what it's resolving to, which is that the alias to the load balancer?
Ben Rady: Yeah, it's a cloud front distribution.
Matt Godbolt: Yeah. Okay. So I'm gonna, I, what I'm gonna do is I'm gonna look up, no, yeah, it doesn't, oh wait, it's refusing me. I want me do this on another one
Matt Godbolt: 1, 2 0 5, uh,
Matt Godbolt: 2 51 1 2 2 68.
Ben Rady: Uh
Matt Godbolt: Oh. That says NS 68 aws D n I mean, it could be the same IP addresses for all I know. So yeah, we need to look up what the cloud front, um, distribution is set to, just to see if just, I mean obviously this is just us testing the water here, like this should, should all work out. But while you do that, um, what this means is, yeah. CloudFront is essentially a caching proxy in front of all of the, um, aws, uh, infrastructure. And, um, when one creates one, one tells it where to get the information from that it's gonna be proxying and, and edge caching and it gives you a sort of a unique, uh, distribution name and then that maps to an IP address or a DNS that you then use to point your web services at. And then whatever you land on knows how to serve up from CloudFront, your web server, your web traffic, your web, whatever.
Ben Rady: So I have arn, I have a distribution name, but I'm having a hard time finding,
Matt Godbolt: What is the distribution?
Ben Rady: An address? Oh, wait, no, maybe this is, maybe,
Matt Godbolt: I think it is and I I think it might actually be a full FQDN
Ben Rady: D n I mean the distribution name is a Okay. Yeah, I think I do. Yeah. Okay. Uh, this is, this is gonna be a little painful. You ready for this?
Matt Godbolt: Okay, then. Alright. Right. Well maybe, yeah, go on. Is it as something or other?
Ben Rady: It's, no, it's, it's a big long stream string of care characters.cloudfront.net.
Matt Godbolt: Okay. So why, if you've got that in your console, why don't you copy that and just do host space that and see if you get an IP address that looks like this one and then that will sort of con confirm while you do that. I'm another copy, www.twoscompliment.org
Ben Rady: Uh, I see 54 2 30 18 99. Okay. 54, 2 30 18 49, 54, 2 30 18, 82 54, 2 30, 18 69.
Matt Godbolt: Got it. None of those matched because I've just realized that NS lookup was giving me a bad answer. It tried to connect and it got time out. And then what it's doing is it's just telling me all about the domain. There is no, i, there is no, uh, a, uh, record associated with twocomplement.org. So that's what we need to fix.
Ben Rady: Okay.
Matt Godbolt: There's also no a, there's no a record associated with dub dub dub dot two complement do org. Right.
Ben Rady: Cuz isn't there not going to be, isn't it gonna be this like different kind? Oh, what are the, what is the name of that type of DNS record that's like, it's not specific to Amazon, but it's like,
Matt Godbolt: Well, it, it's called sort of alias around. So like the, the, the underlying problem here is that there is no such thing really, as much as people would love there to be, there is no such thing in DNS as an, uh, a c name, which is what we really want for. The, um, uh, the, the naked domain. Like, so what you might want mm-hmm. is do dub dub do whatever to be the address of a machine. And then if someone puts in the thing without the dub dub dub, you say, Hey, this is the same as dub dub dub dot. Which is a c name, a kind of a symbolic link if you like at the DNS level. But unfortunately you can't have a C name record for a naked domain itself. You have to have an A record. And the problem with that is that the, the C name actually needs to point at the cloud front distribution because Amazon wanna move it around. They wanna change it. Yeah.
Ben Rady: Yeah.
Matt Godbolt: And so what typically happens is that DNS providers will have a product where they track the DNS entry for the cloud front end that you've got and they'll just keep periodically changing your A record. But Amazon natively supports this, so we should just be able to configure it. So I think we're just missing the configuration in the Terraform and an app, an application should just make this work here without affecting the real twoscompliment.org Cause it is still being served up by hover.com.
Ben Rady: Right, right, right. So I'm actually looking at this now, and this, this makes sense to me, which is I've actually got some commented out stuff in this Terraform that does, I think exactly that. And that is what my Tastypenny Terraform does. And looking at my Tastypenny configuration in Route 53, I can see an a record there. Um, that is a very strange looking a record because the value of it is that big long list of characters cloudfront.net. Right? Well, not the same
Matt Godbolt: One. Oh, that's interesting. Yeah.
Ben Rady: Um, uh, and that is for, uh, the www one and for the sort of bare domain. Yeah,
Matt Godbolt: That sounds,
Ben Rady: So that to me seems like Amazon, you know, doing an a record, you know, trick.
Matt Godbolt: Trick behind the scenes. Let me, I'm gonna have a very quick look at how I did this for some other website that I'm, I'm involved in, uh, Route 53 tf, where the hell all this stuff? Uh oh. Yeah. I actually have modules for this because it's, uh, so awful that I have so many stupid things. Main tf uh, okay. I set a CName and the records are the, yeah. Something like Route 53 address a FQDN. So it's kind of looking up somewhere else. I'm trying, this is obviously makes for great radio, um, uh, zone the alias name. Yeah. Okay. It looks like it's an alias that I'm setting. So I do, for both the, the A record and the AAAA record, I have, um, an alias stanza inside of the Terraform itself. So it's not an address record, even though it could be. And it has a name, a zone ID and some other bits and pieces in it. And I dunno if that corresponds to the thing that you are looking at now.
Ben Rady: Yeah, I think that is, I think we're looking at the same thing here.
Matt Godbolt: So I've got, yeah, alias name equals, and then I've got a variable which holds the CloudFront distribution domain name, and then another thing that's CloudFront distribution dot hosted zone id. And that essentially configures the A and the AAAA for the top level name, which is, in my case, you know, godbolt.org or godbo.lt or compiler explorer.com for all the times it's instantiated, which is like the four each that you've got. But I think we only need one of these. So you could probably just write it out longhand right now.
Ben Rady: Yeah. Well I, I think this would actually just work if the certificate was the, if I switched, so kind of parsing through this now and having some vague memory of what we did here. Yeah. Um, I think the, this will all work if we can just,
Matt Godbolt: "just"
Ben Rady: change this certificate to be a wild card certificate. So if I were to change that in the Terraform and then try to run it. What would it just replace the existing certificate with a wildcard certificate?
Matt Godbolt: I think so. I think so. I, I have some magic to do that too if needs be. So why don't we try, try that.
Ben Rady: Let's give that a try. Okay.
Matt Godbolt: What could go wrong? We could, I mean, right.
Ben Rady: Well, in theory people could start getting certificate errors going to, to twoscompliment because I do think that this is the real certificate. This
Matt Godbolt: One will be the real certificate. Yes. Right. The DNS can do whatever it likes, but we're about to tell CloudFront to use a different certificate when it's pretending to be us.
Ben Rady: Yes. Which is probably why I stopped here. ,
Matt Godbolt: I mean, Yolo.
Ben Rady: Let's do it.
Matt Godbolt: Did you make a new certificate? Actually, you already made a certificate.
Ben Rady: Well, I, I was gonna, I mean, can Okay, wait a second. Stop. If I change this Terraform, it's not going to make a new certificate. I have to go and do it. Manuel-y?
Matt Godbolt: Uh, I don't remember if you, I mean you can absolutely have certificates created in Terraform two. I don't know if Did, did we do that last time?
Ben Rady: Okay, well let's do this, let's start by making the change in the Terraform and doing a terraform plan and seeing what Terraform says.
Matt Godbolt: What the heck it thinks.
Matt Godbolt: Yes. Always, always a good start. Where are we now? Is what am I, is what I have on my computer an accurate representation of what the cloud provider thinks I've got.
Ben Rady: Right. Right. Well, I mean, so I did this once and it said it was up to date, but I'm gonna change it. And now we're gonna do a plan again, and then we're gonna see what Terraform says about what it feels like it wants to change. And I'm gonna make this look very much like the existing one that I have for my recipe project. Right.
Matt Godbolt: Okay. And I found the certificate stanza that I have for my site so we can steal from if needs be.
Ben Rady: Okay. So I'm gonna do Terraform plan.
Matt Godbolt: Yep. What is it saying? It say refreshing.
Ben Rady: It says three to add, one to change, two to destroy. And so it is going to Yeah, say AWS acm certificate twos compliment.org must be replaced.
Matt Godbolt: Awesome. Okay. Cuz you've changed the subject alternative names in there, right? Uh, I
Ben Rady: Think it's, uh, I changed the subject alternative names and I also changed the domain name from www.twoscomplement.org to twoscomplement.org.
Matt Godbolt: Perfect. Perfect.
Ben Rady: Uh, and then it says AWS CloudFront distribution. S3 distribution will be updated in place. And then it says, uh, AWS route 53 twoscompliment.org bracket star twoscompliment.org will be created. And then another aw, uh, route 53 record for twos compliment.org will be
Matt Godbolt: Created. That sounds good to me. Let's do it.
Matt Godbolt: What could go wrong? Well, let's get a list.
Ben Rady: Many things,
Matt Godbolt: . All right.
Ben Rady: All right. So do I have an applied, do I have a, oh, I do have a Terraform apply. All right. Firing the rockets.
Matt Godbolt: Firing the rockets
Ben Rady: Rockets. Um, if we wanted to troll our audience, we should cut off the audio in the middle of this supply . It's like, wait a second. That, how does that that doesn't,
Matt Godbolt: Yeah.
Ben Rady: Doesn't work like that.
Matt Godbolt: Well, ironically, you're stitched then for me, so I'm like, oh, he's joking about the connection going down, and then I'm like, you froze on my screen.
Ben Rady: Oh, man. Which was
Matt Godbolt: Epic trolling in its own right.
Ben Rady: All right. It says destroying still destroying, still destroying. Still
Matt Godbolt: Destroying. It's destroying
Ben Rady: Everything. Oh. And we got an error.
Matt Godbolt: All right. Is there some crate before destroy thing that I've, I've got in mind?
Ben Rady: No, this is a, uh, what does it say? Access denied not authorized to perform ACM request certificate. So this is where we go into the IAM console and we give this service user that we're running as has a whole bunch of permissions that shouldn't probably have, and then we dial 'em back later.
Matt Godbolt: Far too clever. I just
Ben Rady: Have, cause I don't, I don't actually myself manage the, I, no, I don't know. I don't manage the IAM
Matt Godbolt: Here.
Ben Rady: Okay. So where is this user? Tastypenny. And, uh, yeah, we're gonna attach a permission and this is gonna be, um, what is the name of this service? Certificate? Something AWS certificate. I think it's this one. I don't even know. Who knows? AWS Certificate Manager Private.
Matt Godbolt: Yeah. You, this is outside of my purview of understanding. Well, that's not even the right word.
Ben Rady: We'll try this one. Yeah. And see what happens. And if this doesn't work, then we'll remove that. We'll take that out. You know, if it ain't fixed, don't break it.
Matt Godbolt: If it, yeah, if it don't, don't leave it broken. More broken.
Ben Rady: Yeah. That's
Matt Godbolt: Like's, yeah. The programming by coincidence thing. I think we've talked, have we talked about that before?
Ben Rady: Um, maybe, maybe not.
Matt Godbolt: Yeah, maybe not. Maybe
Ben Rady: We have not. Yeah, so that didn't do it. So I'm removing the policy because that did not fix the problem, so I don't want to create a whole other problem by putting something in there that wasn't in there before. Uh, but I AWS certificate, so this guy should have this already. ACM.
Matt Godbolt: Association of Computer Machinists. Yeah. No, not that. What,
Ben Rady: What? Oh, I guess I can go and look at this actually and see what it's, yeah. This, this user. I th I thought
Matt Godbolt: Are, are you the right user though? Is that,
Ben Rady: Oh, it's a different user. I'm an idiot. I'm looking at the Tastypenny user, which
Matt Godbolt: Clearly does work, which
Ben Rady: Already works. Yeah. Like this. I did this already. This is the one that works. I guess I should have thought of that before. It's like you have a user that does this. Go look at what they do. I'm a doofus. I think I was maybe thrown off by the, uh, fact that, uh, our, the user that I have for this has the original podcast name. Can we talk about the original podcast?
Matt Godbolt: I don't think we talked about, oh my golly. This is all these things. Yeah. Programming by. We, I should be taking notes.
Ben Rady: , uh, all right. Certificate.
Matt Godbolt: All right. Certificate. Give me all your certificates
Matt Godbolt: Are belong to us. Oh, I've just gotten
Ben Rady: Certificate manager.
Matt Godbolt: I dunno if this is, this is certainly completely off topic, but I've just been given the okay to push an update to compile Explorer, which I will do in the background of this. So the continued tapping noises will be me pushing a kind of cool thing to compiler Explorer.
Ben Rady: Okay. We're creating, we're creating a certificate. Alright.
Matt Godbolt: Oh, uh,
Ben Rady: That's, that's a good sign.
Matt Godbolt: I'm pushing Compiler Explorer 6 7 25 2 production from the staging environment, unrelated to this podcast. But, you know, we're all tap, we're both tapping away our keyboards if we've got a filter to the air with talking or some description.
Ben Rady: So, yeah. So it's interesting to talk about how we would do this if this were not just our hobby podcast. Right. So cuz right now we are literally testing this in production, right. . Um, which, which I've heard is a bad idea.
Matt Godbolt: I've seen, we've all seen the meme, the, the most interesting man in the world meme, you know, with him, with his little beer going, you know, I don't often do testing, but when I do, I do it in production.
Ben Rady: I do it, I do it in production. And that
Matt Godbolt: That's not our, that's not our, our our mo in our day job. So if anyone's thinking that this is the kind of cowboy activity that we would do, if it was anything other than you and me chatting . No. So what, how would we, Ben, how would we do this?
Ben Rady: How, how would we do this if it was, well, so obviously you want to have a separate environment for testing this out, but the trick with creating that separate environment is how do you know that your separate environment is a copy of the state of the environment that you want to change for real.
Matt Godbolt: Right. Right.
Ben Rady: Um, which has the additional problem of it is it's gonna take you some time to make these changes and in a large enough organization or in a large enough project, that means that the environment, the production environment may change while you are working on making the changes, right? Yes. So you might be able to make a copy of your production environment as it stands right now. And then make some changes to it, test those changes out. And while you're doing that work, someone else might be doing the same thing and making other infrastructure changes to the main environment. So when you finish that, you need a mechanism for basically reapplying the changes that you made on top. It's almost like a, like a fast forward in git right? Yeah. Where it's like, yeah, yeah, yeah. You need to reapply the changes that you made on top of the environment as it exists now, not as it existed when you started working on the, on the new thing that you wanted to add. Yeah, yeah, yeah. Right. Yeah. Um, so I feel like the only way to even have a hope of being able to do this is to just automate everything. Infrastructure is code style with Terraform. Like, I, I feel like,
Matt Godbolt: And have the only thing that pushes any of this stuff to be the main branch of your GitHub repo so that you've kind of post hoc, already merged everything in at the point of where things are applied. Um, you kinda get a merge commit queue at that point, right? The only thing that's really making changes to your production deployment is the, the, the, the head of the line where all of the, the intermediate branches have to definitionally have been merged in. Otherwise it goes, oh, I'm rejecting you because you know, you're not at the latest, you know, oh, I have to get it again or whatever. That kind of feel or are you Yeah,
Ben Rady: No. Yeah. I, I think it, I think it is that and, and then being able to sort of rebuild your test environments based on changes that are, have been actually deployed. So being able to either tear them down and build them again. And reapply the new things that you did or merge a change in, in a way that's realistic. Like, like, you know, it's, it's probably like the order of operations, uh, potentially can result in this in the same environment where it's like I had some environment and then I applied someone else's change and then I applied my own change. Uh, that is probably, that is representative of what is gonna happen in the main environment when you merge your change. Flipping them might not, right? Like if you apply yours first and then there's, like, you might get the same thing hopefully if Terraform works the way that it says on the tin. Uh, but you might not, right? Yeah. So you have to like, think about like how that's all gonna get applied. Uh, so speaking of Terraform, that doesn't work. Uh, it said error updating CloudFront distribution, right? The specified SSL certificate doesn't exist, isn't in the US East one region isn't valid or doesn't include a valid certificate chain.
Matt Godbolt: Okay. Well,
Ben Rady: So I don't know if we destroy our other certificate and made a new one or what just happened
Matt Godbolt: To here. I think you do up arrow return and see what it does the second time because some of these things have disgusting. Like, oh, it takes a while at the back end of,
Ben Rady: Of,
Matt Godbolt: Of, um, which is not ideal.
Ben Rady: Especially a certificate, right? Yeah. Um, all right. I'm gonna go look at the cloud front distribution.
Matt Godbolt: Yeah, that's a good idea. Um,
Ben Rady: And see what state it's in right now. Uh, it says it's enabled, uh, can you curl the site real fast and just see if it returns anything?
Matt Godbolt: I can certainly curl it
Ben Rady: If it gives you some sort of weird certificate error.
Matt Godbolt: Um,
Matt Godbolt: Oh, hang on a second. Dub dub, dub dub twoscomplement.org importantly. Cause that's exactly what we're trying to fix, ,
Ben Rady: Right?
Matt Godbolt: You see, this is why it's a problem for me. This is why we have to fix it. Cause I I'm too lazy to type dub dub dumb or even say it properly. Yeah, no, it's working fine still.
Ben Rady: Okay. Whatever it is. Yeah, it probably created the new certificate and was trying to flip the, uh,
Matt Godbolt: The, the, the cloud front to it
Ben Rady: Over. It was like, no,
Matt Godbolt: I, you've got the console open too, so you can actually have a look in the ACM certificate thingamajig and see if it's there or not, or
Ben Rady: Oh yeah. Good call.
Matt Godbolt: I know. So we've deliberately not shared screen so that I have to ask Ben what he's seeing so that, that you dear listener can actually sort of hopefully follow along. I dunno how much anyone will be able to follow on what we're
Ben Rady: Doing here. Yeah, I see. Okay. So, um, yes. So I see four certificates in here. Uh, two of them are twos compliment ones, one is the www one that is, uh, issued and in use and eligible for renewal. And another one is, uh, without the www its status is pending validation. Ah.
Matt Godbolt: So,
Ben Rady: So we may have to wait. Uh,
Matt Godbolt: There's usually a DNS validation. That's how these things. Did you have, what type validation did you have? Is it, I mean, this is, it could be that it's, you might have an email right now because it's like, Hey, are you really sure this is your certificate?
Ben Rady: Oh, interesting.
Matt Godbolt: Uh, mine set up for dns, which I think because Route 50, whatever monkey, uh, uh, is in cahoots with itself, it can basically set its own DNS records and reque. Right? Oh. But there's the problem. Now we've got two, now we have two problems. We can't, we won't be able to use DNS validation because you honestly haven't flipped the flag yet for the real DNS provider to be Amazon.
Ben Rady: Yeah. I could copy those things over into the other one though. Right?
Matt Godbolt: You certainly could. If it tells you what the, uh, challenge is that it's put in the dns, then you can put them
Ben Rady: In. I mean, I could go if it, if it added it automatically, I could go look. Right. Go look around 53 and be like, what did you add to this thing?
Matt Godbolt: Yeah, yeah, that's
Ben Rady: True. And just copy those over. Uh, but yes, I agree with your assessment of the situation here.
Matt Godbolt: Yeah. Which may have been, I, this rings a bell from the last time we did this and like, hey, yeah, this thing might take a while , which won't be good, won't be good radio of like, yeah. We just have to wait two days.
Ben Rady: Yeah, yeah, yeah. Um, so I don't see any new records.
Matt Godbolt: Maybe it's not set up to do it that way. So I mean, if you look at the acm, it's the certificate. Does it say why or how to au to do that thingamajig?
Ben Rady: Um, it says pending validation, renewal status, number of additional names.
Matt Godbolt: And you don't have an email or something. I can't remember how this works if it's not set that way.
Ben Rady: Not that I see.
Matt Godbolt: Just checking my email. Cause some of those addresses you put a little forwarder on. I don't know that it's
Ben Rady: Yeah. Nothing in my, uh
Matt Godbolt: Oh, right.
Ben Rady: Spam folder real fast, just to make sure.
Matt Godbolt: Yeah, I can't help. I'm gonna go and find my, my certificates
Ben Rady: No
Matt Godbolt: Certificate manager. Oh, of course. I need to log back in again.
Ben Rady: How did I do this for,
Matt Godbolt: For your magic penny.
Ben Rady: Yeah.
Matt Godbolt: This, I'm just looking at mine and I can see Yeah. In use CS renewal. Elig durability. Right. Okay. Um, so I can see, oh no, that's, yeah, I can see that if I, I've gone to one of my certificates and I can see that it has in the sort of more information inside the console itself, under the acm, uh, it's got a list of domains and it tells me status and renewal. It's just type and then it's got cname, name and cname value. And those are the two things that need to be put into the route rty thing. Oh. And there's even a button that says create records in Route 53, but you can click
Ben Rady: Oh,
Matt Godbolt: Well, but obviously you don't wanna do that because we don't necessarily
Ben Rady: Yeah, yeah. Right. Okay. Let's, I'm gonna go into my other registrar, dudad, right. And I'm gonna go to choose compliment.org. Why is there oh one ? I've got two domain names for twoscompliment.org. One of them is the misspelling.
Matt Godbolt: be very careful. Very careful.
Ben Rady: Yes. But I gotta make sure I, I mean, it's not gonna hurt anything if I do the wrong one, but like what
Matt Godbolt: Fewer things
Ben Rady: If that, uh, so dns, and then we're gonna add a record and it's gonna be a cname record,
Matt Godbolt: Which you can copy paste from, thankfully from the other thing.
Ben Rady: Yep. Yep. And then that's
Matt Godbolt: Gonna be how, honestly, how much of software engineering or administration is goes through the clipboard. I mean, it's just,
Ben Rady: Oh my God. So much. So very much. And I'm gonna set the TTL to five minutes.
Matt Godbolt: Wonderful.
Ben Rady: Gonna. Add this record, and then I'm gonna do this same thing again for the wild card. Yep.
Matt Godbolt: Yeah, you've got the two I, I can see for each of my domains, I've got two
Ben Rady: Mm-hmm.
Matt Godbolt: Thingies.
Ben Rady: And then,
Matt Godbolt: And then of course we have to hope that it notices this within.
Ben Rady: Yeah. All right. So yeah, I've got set two of them set here. Um, and it's, it's probably a good sign that I ha actually had another one for the www certificate. Yeah. That is in here. I can see it.
Matt Godbolt: Right. Okay. So,
Ben Rady: So now there's actually three, right? Right.
Matt Godbolt: But these are all like, interim. So like, just to sort of recap in case that we're, we're, we are trying to prove to Amazon that we own that domain name. And one of the many ways that we can prove that is to make a change to the dns records with some magical things that they've given us. These are the C name records that, that we've just been talking about. Mm-hmm. . Um, this allows Amazon to say, we believe you own that domain and therefore we will issue you a certificate for that domain that says you own the domain and we signed the traffic and all that kind of stuff. Now, this is an interim step because eventually Amazon themselves will be the people that are serving up the domain name, and therefore they can, it, it just, they just know we own it because we've transferred it to them in some capacity. Nut we're not there right now. So as an intermediate step where we wanna just be able to test it by getting ourselves a new certificate, we are going to issue the certificate, uh, use your existing DNS to prove that we own it, and then apply the certificate and then finally we can move things over if we're happy. The certificate looks good.
Ben Rady: Mm-hmm.
Matt Godbolt: .
Ben Rady: Okay. Exactly. Is there a way to poke the AWS certificate manager and say, Hey, can you,
Matt Godbolt: Can you take another look
Ben Rady: Now? Range to target one ping only, please. Yeah,
Matt Godbolt: Yeah, yeah. Come on. One ping only. That's a good,
Ben Rady: Uh,
Matt Godbolt: That's amazing.
Ben Rady: I don't know if there is, maybe, maybe I can do this here. I can delete it. I don't think I want to do that. Uh, request.
Matt Godbolt: Yeah. Does that maybe gonna make a
Ben Rady: New one? I can say that's probably gonna make it, well, this is where we manage x free events. Yeah. This might be we just wait, you know, 10 minutes
Matt Godbolt: For, so
Ben Rady: We just wait.
Matt Godbolt: Well, compiler explorer is 67% through doing an update very excitingly in another window. All
Ben Rady: Right.
Matt Godbolt: Mm-hmm. behind you.
Ben Rady: Mm-hmm. .
Matt Godbolt: So that's,
Ben Rady: Uh, oh man.
Matt Godbolt: So then we were talking, right, two things. We talked about one, obviously we just, we, we sort of briefly mentioned was the idea that in our day job, the way that we do this is that the CI build in main applies the production configuration. And so it's been through all the testing and there's not like the two people fighting over two independent things, uh, changes along the way because you always are seeing the union of whatever has been merged into trunk
Ben Rady: Mm-hmm. , correct. Yeah.
Matt Godbolt: Then how do you test it? How do you test a separate like thing? How would we, um, so in Compiler Explorer, I have some very hard coded staging and beta, or beta just to con uh, de confuse people. Honestly, I've had this conversation so many times with Americans, they're like, what beta? And they're like, thinking like egg beaters or like uhhuh, , like pub, uh, boxes. No, not that kind of beta, no. Um, beta. And so, um, those are very special case for me. And they share kind of a lot of infrastructure as well, because at one stage I was trying to save money, uh, . Nowadays, I think actually I probably, this is a false economy, but there are better ways of doing it, or at least there are different ways of doing it. Um, rather than having just some very special hardcoded things that we've sort of pushed things through, uh. And I know you've been involved in a lot of those recently. So do you wanna talk a little bit about, like, some ideas that you've had about how it should be done?
Ben Rady: Yeah. I, I don't remember if we've talked about this on the podcast or not, but we lately have been doing a thing, uh, with a, a data warehouse project that I'm working on where the branch in GitHub represents an environment. So we don't have a production environment. We have a main environment because we have a main branch,
Matt Godbolt: Because the main branch is that Right? It's not special case in any way.
Ben Rady: It's not special case. It's, there's like a couple of additional protections for deleting things.
Matt Godbolt: Got it.
Ben Rady: And that's it.
Matt Godbolt: But other than that
Ben Rady: That, you have to, other than that, it's identical to every other branch and identical to every other environment. And so when you create a new branch, it, you know, says, oh, this environment doesn't exist. I guess I need to apply this Terraform, I apply the Terraform every time. So Terraform just has more, more work to do this time. Uh, and it, you know, spins up all of the infrastructure that this project requires, and it's doing that obviously from a fork of the Terraform file that was just in the main branch.
Matt Godbolt: Got it.
Ben Rady: And is therefore a copy of the infrastructure that is running in the main and environment. So you wind up making an exact copy of whatever the environment was at that time. Right? Right. Um, and so that will all get created. It will then automatically deploy, uh, to that environment. And now you have a completely separate running copy of that system.
Ben Rady: There's a different URL that you can go to that's got your branch name in it. And you can Yeah. Uh, and you can play around with it. You can test things out, uh, and then as you push changes to that branch, it goes through the exact same process. It applies any terraform changes. If you have them, it deploys the new version of the software that you built. Uh, and then you can sort of iterate and continue on working in that. And then when you have something that you're confident is correct, you know, all the tests are passing and maybe you've done some exploratory testing, um, I think this is especially important with the sort of cloud-based services that you use on some of these projects because it's very difficult to test them, obviously, like from your, you know, your workstation, your laptop. So the only real way that you have to test them, uh, in any sort of exploratory sense is, um, by using them for real
Matt Godbolt: Exactly. As we, or at least been doing right now, except that because we don't have this set up, we are experimenting directly in prod. Right?
Ben Rady: Exactly right. Exactly right. And so once you are confident that your changes work and that all your software works with any other infrastructure changes that you have made, you can at automatically merge those things back into the main branch. So your infrastructure changes and your software changes that may be interdependent on each other, all get merged into the main environment at the same time, uh, the same sort of Terraform application process that you used in your branch then gets applied to the main branch, your new software version gets deployed, and if everything goes according to plan, uh, now you've updated your environment while, while doing so in a way that gave you high confidence that the changes that you were making were actually going to work before you tried to do them for real.
Matt Godbolt: Right. Right. And presumably, like we've discussed before, if it, if it doesn't at that moment in time, the hope is you could just revert that commit to main and it goes back to everything before. As long as Terraform does it, its job. And as you know, if anyone from HashiCorp is listening, never, I don't distrust it in any way. Uh, it, it's pretty reliable. So you can almost bet the farm on, on it doing the right thing most of the time.
Ben Rady: Yeah. Yeah. There are gonna be some situations in which you can't figure out some path to go from wherever you were, wherever you are. But really, I would say 99 times out of a hundred, uh, it does exactly what you, it would expect it to do. Right. So if you revert that change in the, in the main environment, it's gonna then have a different Terraform configuration and then Terraforms gonna try to change that configuration. Um, you obviously have to be careful of things, and this is why we have a few individual protections in place. If you were to say add a, um, an S3 bucket or add a data store, add some other thing, roll that into production, write some data to that data store, and then realize that you had another unrelated problem, if you were to roll that back, it might, it's going to delete your data store. Right. By default. Yeah. And so you want some additional protections in there to say like, Hey, if you ever try to do this, just don't . Right. Right. Um,
Matt Godbolt: Sounds awesome. Uh, from my own personal experience, the trickiest part of this is when you start doing refactoring in Terraform and you wanna like say, well, I do have 10 running e c two instances, but they've got terrible names in the Terraform. And I wanna rename them in Terraform, which means I have to do this unfortunate two-stage thing where I changed the name and I don't wanna delete them and recreate them. I want them to be this. And there's ways and means inside terraform of like using state to actually say, okay, I'm renaming this thing in the actual, uh, state. And if you go full automated, you, you don't have the little breathing room to do that. Where I'm like, I have to kind of literally call around people and say, okay, I'm doing some like surgery on Terraform, I'm gonna rename this thing, which means I have to rename it in the backing store, which is a Terraform command, and then I'm gonna change the text file, and then I'm gonna do Terraform plan. Then it should say no changes needed. I'm like, good, because I didn't really change anything. Right. So I dunno if you've had any, uh, experiences with that stuff yet, or do you just say, I
Ben Rady: Haven't had to go through that process yet. Um, right. Part of it is because, and, and I think this also is sort of related to another, uh, potential trade off with this approach that I'm talking about, is that your branches can get very expensive. Yeah. Right? Like if you have lots of infrastructure that has like a per hour cost to it,
Matt Godbolt: Right. Load balancers, for example. Exactly.
Ben Rady: Uh, then, you know, running a branch can be, can be very expensive. Right. And so one sort of side effect that, that I have kind of seen or felt working on this project is that it leans, it, it, it, it leans me toward using more like serverless things and things that can basically scale from zero.
Matt Godbolt: Scale to zero. Right. So Yeah. Yeah. If you have like auto scaling groups, you say, well, they start out zero and the first request that comes in, unfortunately it's gonna be delayed, but that's fine for this.
Ben Rady: Right. Yeah, exactly.
Matt Godbolt: Or, or as you say, lambda type things or Yeah,
Ben Rady: Yeah. Lambda type thing. I mean, there's lots of them out there. Right. But it sort of has, has me using those things more because I know that, you know, we're gonna be creating a lot of these branches and we wanna be able to iterate and it's like, yeah, if you use them, you want to scale up to be able to test them. But you know, if you're not using some particular functionality in a branch because you're testing something else, you don't wanna pay for it. Yeah. Um, so, you know, for better or worse, it's, it's sort of like the architectural direction of this project has headed in, in that way. Um, just for cost reasons.
Matt Godbolt: That's really interesting. As I say, like, well on the extreme end of like what compiler explorer does, I'm like deliberately sharing a whole bunch of things so that I don't pay the cost for the low balancers and the storage or whatever. And the other thing that we deliberately don't bifurcate is the storage of a whole bunch of stuff because we have, you know, three terabytes of crap and you know, there's no way I'm gonna keep deploying that to a new environment every time one gets spun up. And similarly, I wanna be able to create a short link in one domain and test that it still works on the old version or the new version and stuff like that. And that's sharing that tables behind the scenes. So there's some sort of edge cases with that. But I would also like to be able to say, no, I just want a whole new copy of the whole thing somewhere else so I can make a wholesale test.
Ben Rady: Mm-hmm. mm-hmm. and I have the advantage on this project of, it's a data warehouse, so one of the things that it is supposed to be really good at is copying data around.
Matt Godbolt: I suppose so.
Ben Rady: And so what I want to copy data from one environment into another. Turns out we have a lot of great tools for that. Um, you already,
Matt Godbolt: That's part of your mo Yeah,
Ben Rady: Yeah. Yeah. So we sort of lucked into that, but otherwise it would be kind of painful. Like you'd either have to have a thing where you have, you know, like maybe read permissions into the main environment for many of the branch environments so that you can sort of test things out.
Matt Godbolt: Right. And every time you do it, you're sort slightly eroding the nice guarantees that you had before about like the isolation of things and whatever. Right.
Ben Rady: Exactly.
Matt Godbolt: Sometimes you just, this is what I mean, this is what makes it engineering and not science or art. Right. It's like Right. There are trade offs all the way through this.
Ben Rady: Right, right, right. And we have had one situation thus far on this project. It's been going for about six-ish months now, something like that. We've had a one situation where a change in a branch environment leaked over into the main environment. Oh. And this was because of this thing. We had some data in the main environment, uh, that was being reused for testing in the branch environment. And additionally we had a permission that was set incorrectly. Right. And what had happened was basically the, uh, system running in the test environment saw this main environment data and said, oh, I need to go disable this object, this thing, this resource. But it was the main resource. Right. Um, and it went in and it disabled it in the middle of the day. Right. Um, and so it shouldn't have had the permissions to do that, but, you know, permissions in, in AWS and in Terraform can be a little tricky to get Correct
Matt Godbolt: As, as discussed today, you know. Exactly. It's not necessarily the easiest thing to get. Right.
Ben Rady: Yeah. Yeah. It's not like you can write tests for those kinds of things, so you just have to sort of like, I
Matt Godbolt: Don't know if they mean I know they,
Ben Rady: Cause if there's a testing framework for AWS permissions? That would be kinda amazing.
Matt Godbolt: AWS has a built in, um, permissions thing where you can run what if scenarios, but it's a very much as a service. It would be cool if there was a standalone thing. Oh yeah. I've
Ben Rady: Seen that,
Matt Godbolt: That allowed you to sort of write these things where assert like given this environment and this mm-hmm. User assert that they would not succeed in deleting this file. That would be pretty Right. Pretty cool. Right. Maybe something exists.
Ben Rady: I guess you could, you could maybe do a thing where you like decorate parts of the AWS SDK and you say, run as if I had this policy. Right. And then you could like, try to do operations against a, basically like an, a non-existent environment and say like, you know, you don't have to give me the result, but just tell me what I would've been would I have been permissioned to perform this action
Matt Godbolt: Order? The the real trick though is that it's so incredibly complicated. It's not like there is a policy.
Ben Rady: Well, that's true.
Matt Godbolt: You know, the user, the IAM role has a policy, the user has a policy, the machine you're running on has a policy. The then on the receiving end, like, oh, the, the bucket has a policy that grants anyone with like a name who's, you know, ends in a queue, they're fine. They can write to me. You can do literally anything Right. As well as the other way around. So, I mean, who knows?
Ben Rady: Yeah, yeah.
Matt Godbolt: Yeah. How is our certificate doing?
Ben Rady: Uh, let's give it
Matt Godbolt: One more check. Let's give it a go. Cause we're running out time here and we might, this might be a a, a ramly third part coming where we actually get it to work for reals.
Ben Rady: Yep. Yep, yep, yep. Okay. Drum roll, uh, certificates. It says it's issued. Oh, let's try running the Terraform then. Terraform
Matt Godbolt: That would this what a wonderful way to end if we actually, well I say end, we still got more work to do, right? Because we always have more work to do
Ben Rady: Uhhuh Yeah. Issued in use. No. So hopefully that will turn to Yes here in a second when we modify our cloud front distribution, which says it's modifying.
Matt Godbolt: Okay.
Ben Rady: So that's cool.
Matt Godbolt: And then the, has it made the DNS change? Cause I, that's something I've still got open in the terminal is I'm still got DNS looking up to complement org to sort of see if there're Oh, I guess once the cloud point. Yeah. It has to be after the cloud front, um,
Ben Rady: Stuff. Cause yeah, so I would expect the CloudFront distribution would use the new certificate, but I don't know. I'm trying to remember.
Matt Godbolt: But you haven't put the alias in into the, the dns.
Ben Rady: Yeah. And even if you, yeah, let's see here. Actually no, I think it might. I think it might. Okay. Let me go take a look here.
Matt Godbolt: So I'm still not getting it on that and I'm talking directly to it, to the DNS. That should be reserving up these requests. Yeah. There's no caching going on, so I dub up.
Ben Rady: I don't, I don't see it in the console. I'm hopeful that when this Terraform applies that it will actually
Matt Godbolt: Got it. So at the moment it's modifying the cloud front a uh, um, thing. And presumably yes, because in the, um, in the new DNS records, you use a VAR that comes from the the CloudFront domain that is its unique name. It's probably depends upon it. So it's waiting for that to be applied before it does it, even though we know that it would be kind of okay. So, all right. Well we CloudFront takes a while.
Ben Rady: Oh crap. No, that's still commented out. Okay, well
Matt Godbolt: That's quick to apply though. So we can probably,
Ben Rady: Yeah, I'm gonna have to, I'm gonna have to add that in there.
Matt Godbolt: Yeah, that's all good. And compiler explorer is rolled out, which uh, is other good news. Um,
Ben Rady: Okay. And then yeah, so that's applying.
Matt Godbolt: Yeah. The cloudfront takes a while as it has to kind of get permit, uh, the Okay. From all of its geographically diverse, uh, regions before it says Alis gut.
Ben Rady: My guess is that I'm also gonna have to add in a couple of these guys
Matt Godbolt: Here. We're so close. We're so close. I'm actually gonna say to the people who I'm supposed to be now meeting that I'm not going to be there. Most of whom have said they can't make it anyway, so this is fine. , Apparently we have a day job as well.
Ben Rady: There is that
Matt Godbolt: Should probably check on the other computer that I'm not being hassled or harangued. Long silence will be cut from the podcast during the edit.
Ben Rady: Yeah, we can as we wait, we can do that.
Matt Godbolt: We can, the magic will.
Ben Rady: Okay. So that applied. So I'm gonna do one more plan for these other Route 53 changes and then I think,
Matt Godbolt: I think we're at it and then
Ben Rady: Getting close. No, I have an undeclared resource probably cuz I spelled twoscompliment wrong would be my guess. ,
Matt Godbolt: I do that all the time.
Ben Rady: Yeah. Uh, well, I put a.com instead of a.org. That'll do it. That'll do it.
Matt Godbolt: That's not really a misspelling, is it? I mean, strictly speaking,
Ben Rady: . No, it's just wrong. . All right, let's do this plan.
Matt Godbolt: Okay. What does the plan say? So good. The story so far that the CloudFront domain is using the new certificate, and now we are about to apply the DNS changes that will be still not used by the internet at large, but will be used by my console that is set up to use Amazon directly.
Ben Rady: So yeah, we're, we're creating two, we're removing a, uh, a Route 53 entry and adding two more. Oh. Um, oh, because one of them, God, love it.
Matt Godbolt: Oh, What happened?
Ben Rady: I think I did a.com somewhere.
Matt Godbolt: Oh, really?
Ben Rady: Maybe it doesn't, hopefully after this runs, I just, I just named something.com. It wasn't actually like a domain name.
Matt Godbolt: Oh, okay. Right. It was just resource name com. Like a variable name. Effectively, in, in, yeah. All right. Yes.
Ben Rady: Okay. Well, so still running, but
Matt Godbolt: Look really cool in the, in the edit because it'll just work first time. Mm-hmm. Every time. Mm-hmm.
Ben Rady: . Yep. 60% of the time
Matt Godbolt: works a hundred percent of the time.
Ben Rady: Uhhuh.
Matt Godbolt: All right. So we're ways in an application, uh,
Ben Rady: Try to create record. Set a record, but it already exists.
Matt Godbolt: Oh, did you manually make one before or, or have you duplicated it accidentally in the Terraform and Terraform hasn't noticed this mistake, which is
Ben Rady: I think that is exactly what I did. Yeah,
Matt Godbolt: That's, that's my Emma. Because like Terraform will go, this looks valid to me and it'll do the plan, and it said, this is what I'm gonna do. And then Amazon turns around and says, no, no, those are the same thing. You fool, you already got one of those.
Ben Rady: Mm-hmm. .
Matt Godbolt: I told them we already got one
Ben Rady: Two that I was already declared main tf. Oh, yeah. Is that what this is? Oh yeah. Okay. So we, I do have. In this thing. The, yes, the verification records.
Matt Godbolt: Oh, I just went to just the naked twoscompliment org, and that has applied. I can see that it has lots of a records for all of the various different service. So that is excitesightful. We just need dub dub dub to be the same, which are you are working on, presumably.
Ben Rady: Yes.
Matt Godbolt: And see, this is one of those examples, incidentally, if you've already created one of those, um, things in the console, and you've got it in Terraform as well, which, then that's one of those things where you adopt an existing, you know, Terraform import, and that's harder to do in an automated environment, unfortunately. Mm-hmm. . But then, you know,
Ben Rady: Yeah. I really feel like you gotta be all Terraform or No, Terraform, you know what I mean? Yeah. Like living in the middle ground is just
Matt Godbolt: That's true. But like, you know, you have legacy projects, for example, course, where you need a lot of adopting of what's there. And my, my usual trick is to, um, write a skeleton of my best guess as to what I think a, a resource that I already have looks like, and then import it. Mm-hmm. and then go plan, and then basically copy the inverse of everything it says it's gonna do back into the terra form. It's like, oh, I'm gonna delete that. Oh yeah. Add that. All right. What does it say? What does our survey say?
Ben Rady: Uh, it's saying try to create resource set twoscomplement.org a record, but it already exists.
Matt Godbolt: That's because it does already exist. Now I can see dub dub dub twos com org is also those addresses. So that's good. Good from a It's working. Yeah. But not necessarily good from, it's, uh, it's gonna work each time we apply cuz it thinks there's something there that is,
Ben Rady: This is almost certainly something that I'm just like copying wrong here. What in the great googly moogly .
Matt Godbolt: Sometimes when we say stupid things like this, it makes me worry. Uh, not worry. It makes me feel sad for you when you have to do the transcription of these, because the automatic stuff has got no hope with a lot of these words. .
Ben Rady: Yeah. True facts. Uh, so those two records look right,
Matt Godbolt: And yet it thinks,
Ben Rady: And yet
Matt Godbolt: Did you switch from having one that was managed by, oh, it should be deleted then. No, I was gonna say, um, did you move from a four each or to a four each from not a four each or stuff like that? Is that potentially.
Ben Rady: No.
Matt Godbolt: The problem, I don't, no. I've had that before where it's tried to like, create something before it destroyed the old version and they happened to have the same name, and it didn't realize that they were gonna stomp over each other. But that doesn't sound like this
Ben Rady: Dvo records source name, value and type. Uh, you know, the other thing I'm gonna do is I'm gonna open this up in, uh, some Jet Brains tools so that I can get the Terraform plugin to tell me if I've done anything consciously stupid.
Matt Godbolt: But the thing is, Terraform would tell you itself, right. You know, Terraform validate and Terraform itself plan will do at least its side. Usually the problems come when it tries, when the rubber hits the road and it doesn't know it, it doesn't properly model what the provider is going to do when it actually applies these things. Mm-hmm. . So it does, has no idea that those things already exist.
Ben Rady: All right. Here's what I'm gonna do. Yep. I'm gonna comment out the verification. No, I, the two ones that we need are the, the, the top level domain and the dub dub dub. So I'm gonna comment out the record for the verification, cuz we did that manually once already. Yeah.
Matt Godbolt: And we can always just blast those all the way in both the console and here or whatever. Yeah. Let's apply this. Let's try and get the closure of knowing that it applies cleanly and then I think we're pretty much done here.
Ben Rady: Yeah, yeah, yeah. Yeah.
Matt Godbolt: Did that apply cleanly now? Is that, will that you It's gone. Ah it's going.
Ben Rady: I mean, we'll see. We'll see. But it's, it's trying to, it's trying to do it 20 seconds lapse, stay on Target,
Matt Godbolt: .
Ben Rady: Oh, console's looking good though. I got two records both point to the cloudformation
Matt Godbolt: That's what we wanted to see. I mean, I'm seeing that on my side as well here. So I think we're, we're
Ben Rady: There. Yep. Yep. And yes, Terraform apply complete.
Matt Godbolt: Complete. Okay. Then I think Awesome. Think we can declare almost complete victory at this point. We don't fully understand why those, those other records were either they're not there or whatever. Maybe Amazon's putting them in automatically as well as you trying to put them in manually or something like that. That would be my guess now because it's, it's managed by them already. Um, so you just, just leave them out and then Terraform never needs to know they exist. Right. And it'll just work. So final work for this then is to, uh, double check the certificate Looks good. Which I think it probably must do. And then point the top level domain registrar at aws. Change the domain, the DNS records, um DNS servers. Sorry to put be Amazon's ones and or move the whole thing. It's up to you how you own them.
Ben Rady: Yep. No, and then that should,
Matt Godbolt: And then finally we should bump the TTL back up to something kind to, uh, everybody. That's the other last thing that No, everyone forgets myself included, is that like, well if you don't need it to be 60 seconds, then it might as, I mean who knows who say anyone pays attention to these TTLs properly anyway.
Ben Rady: . Right, right. Cool. Cool,
Matt Godbolt: Cool. Well, there we go. We got success. We did it.
Ben Rady: We got success.
Matt Godbolt: Hopefully by the time this airs, people will actually be able to go to https to compliment org and it will just work.
Ben Rady: It'll just work. Fabulous. Awesome.
Matt Godbolt: Okay, my friend. Until next time.
Ben Rady: Until next time.