WEBVTT 1 00:00:14.960 --> 00:00:19.620 Hey, Ben 2 00:00:19.620 --> 00:00:20.420 Hey Matt 3 00:00:20.420 --> 00:00:21.560 How you doing? 4 00:00:21.560 --> 00:00:22.500 I'm doing great. 5 00:00:22.500 --> 00:00:31.640 One of these days, we're gonna have to come up with a better way of starting these podcasts than that, but we just seem to have fallen into it and it works. So I guess why not stick with it? 6 00:00:31.640 --> 00:00:34.680 Yeah, that's what we did on the very first one. It was like, all right. That's 7 00:00:34.680 --> 00:00:40.620 Absolutely. We just shot from the hip. And it was like, oh, that, I guess that's our intro then, but this is no good. I mean, with the exception of 8 00:00:40.620 --> 00:00:44.460 Classic programmer move of like, why do you do it this way? Well that's the way we've always done it. 9 00:00:44.460 --> 00:00:52.710 Exactly. Yeah. Right. We've. Why would we change that? We've always used three spaces and a tab as indent 10 00:00:52.710 --> 00:00:53.480 11 00:00:53.480 --> 00:00:54.420 Okay. 12 00:00:54.420 --> 00:00:55.140 Three spaces and a tab? 13 00:00:55.140 --> 00:00:56.900 I think I have probably triggered everybody 14 00:00:56.900 --> 00:00:58.630 Now. Oh my God. I, 15 00:00:58.630 --> 00:01:00.020 You have no words for that. I know. 16 00:01:00.020 --> 00:01:05.740 I'm just going through, in my mind, all the terrible things that would do. 17 00:01:05.740 --> 00:01:12.560 I think I met someone some once who mentioned that they had some, some kind of compromise space tab thing. It was a bit like that. 18 00:01:12.560 --> 00:01:12.960 compromise? 19 00:01:12.960 --> 00:01:24.120 So like everyone was equally offended by that set up. Well, we've I can't, this is probably the quickest we've ever gone off any kind of script that we had. 20 00:01:24.120 --> 00:01:30.580 Oh, my it's worth it. So worth it. Okay. Compromising on three spaces and a tab. It 21 00:01:30.580 --> 00:01:43.240 Wasn't exactly that, but it was some something to do with that. And I mean, yeah, well, we've, we've, we've never really talked about spaces and tabs here and I don't propose that we do now because you know, there's some, we should leave some things to the reader's imagination. 22 00:01:43.240 --> 00:01:46.140 Yeah. We'd have to do a two-parter on that one. 23 00:01:46.140 --> 00:01:49.920 . So instead let's talk about VI or Emacs 24 00:01:49.920 --> 00:01:50.820 No, right? 25 00:01:50.820 --> 00:02:33.620 Yeah. No enough enough. Yeah. So what we were gonna talk about, in fact, what we are going to talk about today was our own particular unusual experiences with web development. Particularly in the industry that we, we have found ourselves both in and how we've learned web development, and then how we've taken that on into other aspects. I mean, I certainly have, in terms of things like, um, you know, compiler Explorer has definitely, um, learned, um, stuff, but it comes essentially from the trading industry and actually indirectly through the same person who taught us both yes. His particular flavor and brand of how to do web development. So should we talk a bit about, let's talk about Joe. 26 00:02:33.620 --> 00:02:36.680 Let's talk about Joe. He's a person we're talking about. 27 00:02:36.680 --> 00:02:53.820 He is a person worth talking about. So I, I first started working with Joe many years ago when we worked together at Google. And then when he left, he's the guy who brought me into finance effectively. But is, did you meet him first when you were at the trading company? We've we first worked together 28 00:02:53.820 --> 00:02:54.420 At Dr. W 29 00:02:54.420 --> 00:03:29.430 Dr. W yeah. It was in jokes. No, in jokes. Yeah. Um, yeah. And Joe is one of those rare individuals who just get stuff done. He's an ideas person and has an amazing ability to just crank out stuff so quickly in a really really, uh, um, seemingly I impossibly short amount of time and, that's awesome. And part of that comes down to the techniques that he uses. And one of those is his, how he hacks together and I'm gonna say hack, and I, hopefully he won't be offended if he ever listens to this, which I'm sure he won't, um, hacks together websites. 30 00:03:29.430 --> 00:03:31.100 Mm-hmm . 31 00:03:31.100 --> 00:03:37.860 So what are your experiences with the, the Joe approach or what do you, what, what is your definition of the Joe approach? Cuz we haven't really thought about what it is 32 00:03:37.860 --> 00:03:38.740 Yeah, yeah, yeah. 33 00:03:38.740 --> 00:03:41.100 or quantified it and why is it important. 34 00:03:41.100 --> 00:04:38.300 I think rather than trying to define it in sort of any like specific terms, I think at least for me, it makes more sense to sort of talk about how I learned it and sort of the evolution, you know, we talk about like learning from other people a lot on this podcast and sort of the way that you, you do things. And I have this memory of Joe giving a presentation, sort of like a geek lunch style presentation at this trading company, uh, talking about how he was using, how he was building, you know, trading apps and other tools in Chrome. And the approach that he was using was very different from anything that I'd seen before. Every, all the web development that I'd seen before had been extremely framework, heavy so you would go and you would get some web development framework and you would try to, you know, build your application in that framework and do whatever the framework did. 35 00:04:38.300 --> 00:05:12.600 And you know, oftentimes the framework would make promises about like, oh, well this will now work across different browsers. So you won't have different problems in different browsers, which always turned out to be false. Uh, you had other problems in different browsers. It was just harder to figure out what they were, cuz you had this framework there that was, you know, trying to do things for you and was maybe sometimes doing some things for you, but you know, wasn't always doing everything for you. Um, and so he sort of presented this thing and it was like, you know, a very simple approach where it's like, actually, and this was like in 2011, 36 00:05:12.600 --> 00:05:14.460 I was gonna say it's about a decade ago, isn't it? 37 00:05:14.460 --> 00:05:23.520 Yeah. Um, he's like actually, if you just use Chrome and you don't worry about other browsers, um, which 38 00:05:23.520 --> 00:05:26.040 Uniquely we were a place to be able to do for what it's worth. 39 00:05:26.040 --> 00:05:26.360 Right. Exactly. 40 00:05:26.360 --> 00:05:29.620 Which is a blessing that most people don't have, but, simplifies 41 00:05:29.620 --> 00:06:43.900 Right. I mean this whole approach is very, I think a little idiosyncratic to the industry that we work in, the fact that you can get away with this. Um, but he's like, we're just gonna use Chrome. We're gonna tell everybody like, Hey, if you want to use these apps, you should use them in Chrome. Maybe you can put a little check in the app and say like, Hey, you're not using Chrome. I'm not gonna do anything for you until you install Chrome. Um, but if you just use Chrome and you just use the APIs that Chrome gives you, which are all web standard APIs, because at that point and certainly today, but at that point, the, the, you know, the web standards had kind of grown up, they sort of gave you a lot of things that you needed. You didn't really need a bunch of extra stuff, sort of like it's all there in the browser. Um, so if you just use that, uh, and you, you know, use some like very simple techniques for like HTML templating and a few other things, like you can build web applications without anything else and they work great and they're extremely simple. Right. Um, and the, and advantage that, that has at least in the trading industry is that, you know, if you're building like a, especially like a click trading application, I mean, this is also true, true for tools and other things, but especially in click trading, you know, hundreds of milliseconds matters a lot. 42 00:06:43.900 --> 00:07:07.290 And just, just for those who don't know, uh, click trading is for like a manual trade where, uh, human is deciding to make trades. Obviously you and I have worked in industry parts of the industry where everything's completely automated, but a click trader is like the UI for somebody to place orders. And they're gonna be human quick, which is not high frequency quick, but it's still, if you click and it takes two seconds to respond because of whatever, 43 00:07:07.290 --> 00:07:08.100 Right. 44 00:07:08.100 --> 00:07:17.580 Then you're gonna get a very awkward question asked of you, come the, the miss, uh, of, of not actually yeah. Not, not getting that trade done. 45 00:07:17.580 --> 00:07:45.220 Yeah. Yeah. Why do we miss this? Because this button didn't do anything. Um, and then the other place where I think this matters a lot is in control systems, which I definitely did a lot as web apps, which again was kind of novel at the time. Like the fact that Joe was sort of presenting this and saying like, Hey, you can do this. Um, you know, to say like, you know, oh, the button to stop, the trading system is here. And like, if you click that button and it doesn't work, then really, really, really bad things are gonna happen. 46 00:07:45.220 --> 00:07:45.860 Yes. 47 00:07:45.860 --> 00:08:01.560 So, um, you need to make sure that you understand, you know, basically the whole stack of like, okay, I clicked this button, this event fires, this bit of JavaScript runs, it sends a message up to this websocket to this other thing, which then turns off the trading system. Right? 48 00:08:01.560 --> 00:08:39.860 Um, so like if you can't, you know, sort of lay your hands on every little bit of that and make sure that you understand it all, it's harder and maybe not even wise to, um, use, uh, web apps to build these things. But that was the brilliance of Joe, is that he sort of saw that you could do this and showed people how to do it. And I learned how to do it and a bunch of people that I worked with how to do it. And to this day, I, I still know a lot of people that, uh, are there and build apps this way and, and use them this way. Um, but it is a, it, it is a little bit of an idiosyncratic method because you're just, you're not using, you know, you, you might use some sort of light libraries like jQuery. 49 00:08:39.860 --> 00:09:24.860 Yeah. Well, I mean, even, even jQuery I think is mostly out my, my canon as well, and I've mostly dropped out now. I don't need it. There are a couple of things I occasionally go, oh, I wish I had jQuery for this, but I just can't be bothered to bring it in. And you know, when you are, when your webpages, uh, uh, an HTML file, which is all of like a dozen lines and then a single JavaScript file that has no external dependencies, that's kind of a nice place to be for a lot of like little systems that you are running, you know? And, and you can do quite a lot with that, you know, tables gonna be, be made to, to work buttons, uh, update status bars, things like that. And effectively you are using the way that I think of it. It is, is I'm using, um, the, the, the web browser as purely a simplified canvas for my application to sort of render to it. 50 00:09:24.860 --> 00:10:10.000 And most of the time it's, it's, we're not, not in like an actual canvas sense, right. But like, there's about half dozen buttons that need to be updated the text on. And for that, I don't need React and I don't need these other things. Now, occasionally you do find yourself going down the path where you, you know, you're like, well, if I edit the, the, the value in this text box and hit enter and meanwhile, someone else is editing the same text box box. How do we deal with this kind of like state thing, if there's a server, all those kind of issues. And you know, that frameworks do sometimes have solutions for you along those way, but you still have to think about them. So I dunno if it is that much of a, a saving grace there, and maybe I haven't been exposed to the right framework to be able to have that opinion, which is, but I think, you know, again, uh, this is now going a bit wider than what we're talking about. 51 00:10:10.000 --> 00:10:59.580 Like frameworks in general are kind of can be an issue cuz if they do what you want, that's amazing. But if they don't, you're in trouble and you're kind of stuck. And I, I'm not fond of that inversion of control thing, you know, where. Framework calls you, you know, in Soviet Russia yeah. Right. So, um, but so anyway, that aside, but um, yeah, the, the, for me sitting down with Joe and Joe back in 10 years ago, we were just jQuery would be the thing that we would do just to unify and make a whole bunch of stuff relatively straightforward. And, you know, especially CSS selector type stuff is really, really straightforward in, in jQuery, but then going, how, how am I gonna write, uh, how am I gonna make this table populate with all these rows and columns and things because I really don't wanna be doing it in code. 52 00:10:59.580 --> 00:12:06.860 And you mentioned HTML templating, which I think is another amazing, like trick, well, we'll have a single CSS tag that says template, and then we'll have the single piece of CSS that says anything that's template is display:none. And now I can find that template and I can clone it and add it to its parent, which means now I get another row in my table and then I can remove the template CSS tag from it. And now suddenly I have used, uh, the, the, the thing that I've written out in HTML with just the class template as an, a template. And I've got a new row of my table and now I can just fill it in. I can find all of the things and find the, you know, the second, uh, thing that's, um, marked in a particular way. And that that's where I'm gonna change the text or whatever. And that was an eyeopener for me is like, oh, how simple and elegant it is to be able to in almost one continuous line of JavaScript, just with continuation, like dot this, dot, this dot remove class dot, whatever dot, find this thing and dot text, new new line. And you're like, oh, it in instantly reacts. And, um, and I have a web app that is that's, that's responding to me straight away. That's great. I love it. 53 00:12:06.860 --> 00:12:35.180 Right. Yeah. And one of my favorite parts about that is that that means that you're editing HTML in an HTML editor, right. You don't have this weird thing where you're like got HTML, interleaved inside of JavaScript. And like, you gotta have a very smart editor that knows it's like, oh no, this string is actually like an HTML snippet. And so when you, when you open one, you know, uh, element, that's gonna automatically close on the end, right? Like, it's like, oh, this is just HTML. You can load it in pretty much any editor. 54 00:12:35.180 --> 00:13:18.080 And if you just delete that template tag of it as well, you can actually load it up and without the JavaScript and see what a single row would look like. So you have got a little bit of the, like, this is what the placeholders would look like. Uh, so that works pretty well too. So yeah, that was an, an eyeopener for me. And then, um, the web sockets, you mentioned there, which is, I dunno, how many folks have actually done anything with web sockets, but like the, the other sort of take home for me was, um, just using web sockets for everything. Like you go to a webpage, the web servers, um, responsibilities are to give you the HTML to give you the JavaScript and then, and the CSS maybe, and then sit there. And as soon as you connect back through web sockets, it gets out the way. 55 00:13:18.080 --> 00:14:16.280 And now you're talking pretty much directly to your server and forgetting the fact that there's a web, there's a web browser there at all, and now you're just exchanging messages with the webpage directly. And that was again, another there's all this restful stuff I'd done before, you know, in query parameters. You know, do I put it in the, in the hash string or do I put it as a query pram? Do I post or get, or whatever, all that goes out the window. Cause once you've got your app, the app is loaded and now you're just doing client server stuff. Like it was a TCP connection and that's amazing. And one of the tricks that we used to do as well, I mean, this is, again, is something you should never, ever, ever, ever do, unless you are 100% in control of everything. Like we are when we're saying about this. But, um, what, um, what, uh, I've seen done before and with, with Joe was like to have a with block that says, uh, with, uh, some, uh, I can't even remember how it works now in, in JavaScript. It's been so long since I done this, but essentially we just eval'ed whatever the text was that came down the web socket. 56 00:14:16.280 --> 00:14:17.860 Yeah. Right. 57 00:14:17.860 --> 00:15:01.680 So that meant that the server could do literally anything it wanted to. And if you wanted every sort of command in your RPC was just like literally calling a JavaScript function that you had previously arranged. Um, but you could also send down things that were like alert and anything, anything at all, which of course is terribly dangerous. You know, you should never let your server execute directly code, but it meant that you didn't actually have to write a protocol document. That was it either. Right. So that's eyeopening as well. And it's really, really, really powerful to, to, to be able to do those, that kind of stuff. And in fact, I was so enamored of this approach that when we started, uh, when I moved on to C++, um, trading systems, uh, I ended up writing there weren't many ex uh, things that were around for, for web sockets at that time for, for C++. 58 00:15:01.680 --> 00:15:50.640 And so I just wrote a web server that had web sockets thing, uh, support and, um, embedded it into all of our trading systems. And then very quickly it was like, no more, are we kind of SSHing onto boxes and poking around with processes. Or kill, you know, kill minus HUP to sort of shut down the trading system or having text files, whatever it's like, no, you just go to the webpage and there's the big red button. Exactly as you described. And then very, very quickly you're like, well, wouldn't it be cool if we could visualize exactly what the internal state of the trading system was. And then quickly it becomes this, this beautiful webpage with all of these beautiful, like CSS things that are going on and gifs that are, uh, uh, lights that go red and, and, and yeah, it just felt like, oh, this is a natural way of extending my application. And, um, uh, yeah, it, it, it just all came out of like a web socket connection. There's no, there's nothing really clever behind the back of it. 59 00:15:50.640 --> 00:15:51.780 Yeah. That's Seasocks, right? Isn't that that library. 60 00:15:51.780 --> 00:15:54.260 That's Seasocks. Yeah. Which is a terrible name. Yeah. but 61 00:15:54.260 --> 00:15:56.720 Yeah, no, that's a, that's a very clever name. I like that. 62 00:15:56.720 --> 00:16:35.560 It. I don't know. It's, it's not as clever as I thought. Uh, but yeah, Seasocks is, is, uh, pretty good. I mean, there are plenty of other ones out there now that are, that are better, that are more C++ 17 and 20 or whatever, but this one was made to work and it's about, it's been running for about 12 years now. And in fact, somebody managed to port it to work on Windows the other day, which were, I was surprised. Cuz it's just a Linux thing that I was wanted to just have work. So say relatively low overhead. Um, yeah. Good fun. Yeah. And yeah, the world's your oyster once you've, you've got, uh, a web server and JavaScript general purpose programming language on both sides or on one side. So yeah. What, What about you? What about your experiences with this kind of stuff? 63 00:16:35.560 --> 00:16:53.470 I mean, very the, I did basically the same thing. The one thing that I sort of realized as I was learning this stuff from Joe, that again was a huge departure from anything that I'd seen. Cause I've done some web development stuff before starting, you know, before working with Joe, but all of it was just super painful and I 64 00:16:53.470 --> 00:16:54.700 cgi-bin 65 00:16:54.700 --> 00:17:03.360 Don't, I don't like it. I don't like anything about this. Right. And, and you know yeah. All of the arguing about, you know, which HTTP verb is the right one here and all this nonsense 66 00:17:03.360 --> 00:17:05.960 Is this a PUT or POST? 67 00:17:05.960 --> 00:17:20.080 Yeah. Um, um, but the, the, the two things that I, the two additional things that I kind of got out of that were, um, A) If you build web applications this way, they are extremely testable. Oh, like extremely testable. 68 00:17:20.080 --> 00:17:20.260 Well. 69 00:17:20.260 --> 00:17:38.560 Like, and I still have carried this on to this day where it's like, when I'm building web applications, like my standard for unit testing is maybe even a little higher than it is in server side languages. Like, you know, Java or Python or Ruby or Clojure or stuff like that. 70 00:17:38.560 --> 00:17:45.860 So, um, how does that work? How do you, how do you, what makes this testable I'm genuinely intrigued now? Because I don't, I dunno, 71 00:17:45.860 --> 00:18:08.210 it mostly, it mostly gets back to that whole templating approach. Right. And you can, you can expand on this and you can add lots of cool things to it. But the fundamental of it is when I wanna show some content on a page, the way I'm gonna do that is I'm gonna have a function. And that function takes some amount of input and it returns an HTML element. 72 00:18:08.210 --> 00:18:08.600 Oh. 73 00:18:08.600 --> 00:18:47.840 And that HTML element is almost certainly a template that has been cloned and then populated with stuff. And that stuff could actually be data or it could also be click handlers. But none of that matters because if all I've got is a function that returns an element, then I can pass stuff into the function, get the element back, say like, Hey, is the value of this field, the string foo? Cool. What happens if I trigger a click event on this element over here? Does it change to bar? Yes. And now fundamentally all you're doing is testing a tree. It's just a tree data structure. That's all it is right. 74 00:18:47.840 --> 00:19:15.420 With some handles associated with it, but not, but, but yeah, it's just a, yeah. So you actually used the DOM objects themselves as sort of like the, the conduit by which you can interrogate and say, did you do the right thing and not, did you do the right thing because of the state of the program change anything, but like, but like, did the tree contain the right information and does it have the right behavior when I, and treat it effectively as the testable object? That's amazing. Yeah. I'd never thought of doing that. Yeah, that's so cool. 75 00:19:15.420 --> 00:19:52.940 And, and, and, and really, it was like, you know, Joe, like did some stuff with this. Joe was never as, I mean, a few people are as bananas about testing as I am, but, you know, he was never like his main thing, but I sort of saw what he was doing. And I, I was like, oh, I could write tests for this. This would be great. And this was going back all the way to like, you know, 2011, 2012, we started working on the, the, you know, the Monocle project and other things like that at, at DRW, we started writing tests this way, the team I was working started writing test this way. And I've kind of done it that way ever since. Um, and it's just, it creates this, this really great seam, which I had done a lot of work when I was right outta school with Java and Swing. 76 00:19:52.940 --> 00:20:39.380 Right. Uh, I was actually hired to be a QT developer and like a month into it, they were like, oh, you're not gonna do any C++ or QT. You're gonna do Java Swing instead. I'm like, ohkay, I dunno. Um, and I spent probably two or three years. Oh, a little more than that. Actually building 2d graphics tools in Swing. So this was a lot of like 2d canvas stuff, but also just user interfaces, you know, buttons and forms and things like that. And all of that stuff. I mean, even back then, I was like super test infected and, and even, and, and I always, always had a really hard time testing any of that stuff. Um, and part of the problem was this sort of like fundamental issue that you have with testing UIs, which is, there's no such thing as assertPretty(), right? 77 00:20:39.380 --> 00:20:39.480 Yeah. 78 00:20:39.480 --> 00:21:12.220 You have to look at it as a human being. You have to look at it and be like, is this intuitive? Does this make sense? Does this communicate the ability of the software to a person who would use it, a normal human being of which I am a reasonable approximation? Um, yeah. Close enough. Right. if you've, anytime you see an open source, uh, tool with a user interface in it, I guarantee you that user interface like nine times outta 10, maybe eight times outta 10, it's just gonna be terrible. It's gonna be absolutely terrible. Cuz it's designed by programmers. 79 00:21:12.220 --> 00:21:14.410 Have you, have you seen my website? 80 00:21:14.410 --> 00:21:15.600 81 00:21:15.600 --> 00:21:21.960 Just saying, I mean, it's not the best from that point of view. It's utilitarian. That's what I tell myself. Yes. 82 00:21:21.960 --> 00:21:29.940 User experience designed by people who at best are able to, uh, synthetically replicate human emotion. Right. Like it's 83 00:21:29.940 --> 00:21:31.840 I barely passed the turing test. 84 00:21:31.840 --> 00:22:26.880 Yes. Right, exactly. Um, but it's there's no, the, the problem with testing is like, you always have this thing of like, I have to look at it. I have to look at it to know that's right. And one of the great things that sort of came out of that experience, you know, building these web UIs and being able to build things this way is I found a, a seam, I found a line between that was very clear that I could explain to people, uh, between the stuff that you could reasonably unit test and the stuff that you just had to verify as a human being. Right. And that is the stuff that's just the tree, the DOM all completely testable, all the events that you can trigger on there, all the, everything, all the classes that are on there, everything is completely testable. It's just a tree data structure. That's it. You can test it. It's super simple. The place that you can't is the CSS it's the styling have, have the 85 00:22:26.880 --> 00:22:31.440 Tree and how it reacts in terms of like the, the, the browsers representation of it. 86 00:22:31.440 --> 00:22:33.040 Right, right, right, right. 87 00:22:33.040 --> 00:23:08.260 But the semantical semantical, the semantic, um, thing that says the input box, it's an input. And I know it is, it has two fields and one of 'em has this tag. And one of them has this other tag and the content should be this and that. And then it's like the, it's up to the styling to make them look how it is presentable to a user, but I'm sure that I put the right information there and now I can kind of, my, the responsibilities of the program have finished and the responsibilities of the styling takeover at the point where you hand over and just agree on class names or whatever. 88 00:23:08.260 --> 00:23:32.760 Mm-hmm , mm-hmm . Yeah. And, and it creates this sort of like very clear boundary between, uh I'm I'm writing code and I'm writing tests to manipulate the DOM versus I'm basically acting as a UX designer now, and I am styling that DOM and I am choosing colors and I'm choosing fonts. And I'm at, I maybe, you 89 00:23:32.760 --> 00:23:48.340 Know, doing assert if you're right. Yeah. I suppose the thing here is this is like asserting, you know, if I get an error, if I send an error to this, this thing, does it look red? And if you're writing your test like that, you're doing it wrong because it shouldn't be that it looks red it's that it has the style class associated with it. 90 00:23:48.340 --> 00:23:54.400 Yes. It has the error style. It has the error class on it. And that's the only thing I'm asserting in the test is does it have the error class? 91 00:23:54.400 --> 00:23:58.030 Because you dunno whether or not yeah. It's a bit like we're doing localization. actually, now I'm thinking about it. 92 00:23:58.030 --> 00:23:58.680 Yeah, yeah, it is. 93 00:23:58.680 --> 00:24:07.640 Again, you wanna put the, the semantics of what are important. And the thing that comes afterwards is the presentation layer, which is, you know, what does it actually come out of as, yeah, yeah. Right, No, that's really interesting. 94 00:24:07.640 --> 00:24:34.200 Right. That's a really good. That's a great point. Is it is it's a little bit like localization. It's a lot like localization actually, but that separation being able to have that separation, um, was, was a, was a game changer for me. And we, and I got, I've gotten more sophisticated with it over the years, you know, started using different, like you'll have classes on the DOMds that are strictly for styling. Like, you know, you'll see this a lot now with a lot of CSS frameworks where it's like, you get these like, um, like 95 00:24:34.200 --> 00:24:34.940 Prefixes 96 00:24:34.940 --> 00:24:38.520 Classes that are like mt-5, right. Margin, top five, right? 97 00:24:38.520 --> 00:24:39.680 Oh yeah, yeah yeah. 98 00:24:39.680 --> 00:25:00.020 And, uh, you know, stuff like that. And I like that stuff a lot, but like that is purely for styling. And I'm putting that in the templates. I'm almost certainly not writing unit tests to assert that it's there, unless I've got something where it's like, oh yeah. When this event happens, you need to handle that event by adding some margin to this element. 99 00:25:00.020 --> 00:25:00.220 I see. 100 00:25:00.220 --> 00:25:09.480 In which case I might write a test that says like, Hey, uh, after the, you know, blur event happens, does this element now have the mt-5 class. Cool. Great. 101 00:25:09.480 --> 00:25:31.640 But you should also never do that, please. like, if, if you, if you are changing the styling, when, when somebody loses focus, then I'm gonna come around with a club. Yeah. I think I showed you a website the other day that where, where it was like wontonly on mouse move would like make a whole bunch of stuff appear on the screen that moved everything away to the point where like, it was impossible to click one of yeah. Bug bears over 102 00:25:31.640 --> 00:25:33.680 The input just like runs away from your cursor. 103 00:25:33.680 --> 00:25:37.660 Yeah. Exactly. Is like one of those things like come here. 104 00:25:37.660 --> 00:26:10.460 Oh my God. Um, but yeah, so, and, and that was, that was sort of a, a natural evolution out of that. And it tied into the whole thing of like, you know, and a lot of things that I did, um, at aforementioned company was, was building like these click trading apps. Like we built them for a couple of different trading desks there. And, you know, they are, some of them are like, actually, it's weird to say latency sensitive, but it's like, that's what it is. Like speed really matters. And, and obviously correctness also matters a ton cuz you know, you're, you're acting in, in, 105 00:26:10.460 --> 00:26:14.740 You're sending millions of dollars worth of of, of, of trades into the market. Yeah. You wouldn't don't wanna do 106 00:26:14.740 --> 00:26:49.140 So the combination of those two things like really led me to this place of building applications this way, testing applications this way. And having this like super clear separation between here's the styling. And honestly we're probably not gonna change it that much. We're gonna write the CSS. We're gonna put a couple of classes in there for the things that we need and we're gonna mostly leave it alone. And then primarily what we're doing is just, you know, getting messages from web sockets, sending messages to web sockets, calling functions, creating elements and putting them in the DOM. And that's it. 107 00:26:49.140 --> 00:27:43.940 No, that's cool. How do you, so one of the things that I always found difficult with the limited amount compared to what you're describing of testing I was doing in the front end and anyone who's used compiler explore knows that there's not much in the way of front end testing. Um, uh, despite some of my, my friends helping very much on that, but like how, how does one in general do testing when the only thing that can really give you a DOM that acts the way that, well maybe, maybe you were telling me that this isn't, but my, my instinct is the only thing that will provide a DOM interface that is a true to life is a web browser, which is not something that I can script or debug in the same way. It's a very heavyweight thing, uh, compared to, um, you know, like what I'm used to with like, yeah, here's an assertion, it's a framework that I just literally call assert on it. And it's like, you know, call square root, square root of 144 is 12. End of story. Yeah. How do you, how do you, how do you test it? 108 00:27:43.940 --> 00:27:54.360 So lately what I've been doing? I mean, I can go back and give you the whole history of how I got here, but lately what I've been doing is I've been using just headless browsers, usually headless Chrome. 109 00:27:54.360 --> 00:27:54.600 Got it. 110 00:27:54.600 --> 00:28:09.200 And I run all of my tests in CI and on the command line, just with the headless Chrome, um, there's a particular tool that I've, I've been using more in the last couple of years, which I actually like a lot called Karma. And what Karma will do... 111 00:28:09.200 --> 00:28:11.320 Okay. I think that might be what we're using. 112 00:28:11.320 --> 00:28:50.620 I like karma it's it's good. I mean, it lets you run, you can run multiple headless browsers at once. So it's like, and it has the whole, you know, continuous testing model of like, whenever you change your code, it'll automatically rerun your tests and it can rerun in multiple browsers if you want. Um, I'm still, you know, if I'm building internal apps, I'm still doing the thing of like, Hey, use Chrome uh, but uh, you know, if you have multiple browsers, you need to test against, uh, cuz yeah. I'm with you. Like I've never been satisfied. I've tried, this was more 5, 6, 7 years ago. I've tried all like, you know, the Node.JS simulated DOM stuff and libraries and all that. 113 00:28:50.620 --> 00:29:11.000 Or even having the web browser open and like just having to manually eyeball the green on that and like reload the page and it's actually running in the browser, like as a thing and it sets up. And I've never felt satisfied with that because it's like, unless unless some Unix process returns a non-zero exit value. Like I don't think that my test of, of are testing anything. 114 00:29:11.000 --> 00:29:29.060 Yeah. Yeah. And I mean, in fairness, what I used to do is I would set up some kind of live reload set up. So this live reload is actually a library, but it's also kind of a technique of like every time you change, you know, a bit of markup or JavaScript or whatever, whatever browsers you have open that are connected to that app automatically reload. 115 00:29:29.060 --> 00:29:31.060 Reload. Right. There's a hot reloading thing that, yeah, 116 00:29:31.060 --> 00:30:04.290 That's sort of hot reloading thing. And I've done that for a long, long time. And when I would just run my tests in the browser, that was fine. Cause I just bring the browser window up and I'd see the test results there. But in the last few years, I've I've the tools have gotten good enough to where I, I feel like you can just do it entirely on the command line right now and it gives you what you want and if something goes wrong, there's still a web server running there. So the headless browsers are connecting to it. You can just connect a regular web browser to it and you can debug it. Right. So if something weird happens, you just open up a browser, go to local host 9876. 117 00:30:04.290 --> 00:30:05.000 I see. 118 00:30:05.000 --> 00:30:11.140 And there's your test runner and you can open up the console and you can see what's wrong and you can step through the code and you can do all the things that you normally do. 119 00:30:11.140 --> 00:30:26.200 Daft question, what exactly does the headless Chrome, do you know, what is it like you run headless Chrome and give it a web address. And then it returns when the webpage calls exit or how, how does that work? Where, where, you know, where does the output of it go and how, how 120 00:30:26.200 --> 00:31:11.800 Is it? Yeah. Yeah. So the, the, the Karma tool that I've been using recently, the way it works is it launches the headless browser for you. Right. And it communicates with it, right? So it is serving up this like generated HTML file that has all of your tests and the, you know, testing framework and a bunch of other stuff in it to communicate. Um, and then it just opens the headless browser and points it at that thing. Uh, and then, you know, executes your tests and then gathers the results. And then if you're running it in continuous mode, it just prints the results out to a terminal. And every time you change it, it reruns the test. If you're running it in batch mode, it just returns you that, you know, non zero error code or zero error code that you want prints out the results of the test as it's doing it. 121 00:31:11.800 --> 00:31:18.900 And you can run it in your, you know, I, you know, I have a couple of projects right now where I'm running it in GitHub actions and it works great. Right. 122 00:31:18.900 --> 00:31:19.780 Nice. 123 00:31:19.780 --> 00:31:35.660 Um, so I'm, I'm a, I'm a huge fan of that. And it's definitely a, it's been a, a long evolution cuz you know, back in 2012, when we were doing this, it was, it was literally like, well we're gonna write these tests and we're gonna just put them in the browser and we're gonna just reload the page every time we make a change. Right. Right. 124 00:31:35.660 --> 00:32:35.020 I remember we deployed, like test.html that you could go to. So we we would deploy it and then go to test.HTML to make sure that the actual deployed system, uh, pass all the tests before we tell the traders. Yeah. Okay. You can use it, which is unsatisfying , you know, but necessary. And I mean obviously, um, there are things like, and now I'm blanking web, um, web driver selenium, I think. And the two or they, they maybe the same thing. Now I know Simon Stewart who also knows Joe incidentally through this strange sort of like set of people that we we know, um, are involved with that. And, and that was very much more, as I am understand, it would like run a full web browser and then write tests in terms of interrogating the DOM kind of exogenously to say like, Hey, there should be a thing somewhere on this entire page that says hello world. And if there isn't, then we didn't load the webpage properly or things like that. Those kinds of assertions are what you could run. And I'm sure I remember having conversations about like snapshotting and doing like pixel level 125 00:32:35.020 --> 00:32:36.680 Yeah. 126 00:32:36.680 --> 00:32:57.380 Comparisons about like, well it ought to look something like this, you know, give or take 20% of error, which immediately is like, well, that's a very brittle test that you've just designed for yourself, but I also it also it is a way of locking in, uh, a very specific, uh, behavior if you require it, which seems a little bit too much for, for, for me. And obviously it's not what we've been talking about now, but yeah. 127 00:32:57.380 --> 00:33:13.880 Yeah. I think if you, if you have a lot of money to spend on testing and um, you have a, a system that's fairly mature and not changing that much, uh, then those kinds of approaches can, can be effective. They can be better than not doing them. 128 00:33:13.880 --> 00:33:15.260 Yeah. 129 00:33:15.260 --> 00:33:21.600 Um, and, and, you know, especially if you, if you had no front end test, you're trying to retrofit them. Um, I think that's a situation where that can make sense. 130 00:33:21.600 --> 00:33:32.020 These are, I mean, if you cast your mind back, um, however many years it is that we've been going now when we had, uh, Claire on the, on the program and the golden test, not golden tests. Um, oh gosh, that's what I 131 00:33:32.020 --> 00:33:33.980 Always call them. Golden image. 132 00:33:33.980 --> 00:33:35.260 Uh, sorry, what did you say? 133 00:33:35.260 --> 00:33:36.220 I said golden image, but 134 00:33:36.220 --> 00:33:44.460 Yeah. Golden image. That's right. Yeah. Yeah. And now I can't um, oh, darn it. I can't remember the name of the the term that, that, that, that she uses and 135 00:33:44.460 --> 00:33:47.060 Oh yeah, Yeah, yeah. 136 00:33:47.060 --> 00:33:53.100 Uh, oh gosh. Yeah. You've I can see you. The color of Ben's face has changed as he clearly alt-tabbed away 137 00:33:53.100 --> 00:33:53.700 Two's Compliment dot org 138 00:33:53.700 --> 00:33:59.000 Acceptance test. I remember it without even looking, you just have to like, sorry, sorry. Claire 139 00:33:59.000 --> 00:34:00.580 Approval, testing approval testing, 140 00:34:00.580 --> 00:34:48.390 Approvals approval. There you go. Approval even better approval. Sorry. I got it wrong there. So approvals tests, which are a great way of retrofitting, um, tests into a system that has none at all, when you can just go to the far end and say, well, okay, if I feed this text file into my compiler, I should get this executable out the end. And I'm just gonna do a bit wise comparison. It's a great starting point for doing no harm. Similarly, presumably this kind of test where you say, okay, simulate an 800 by 600 image, render the screen and then kind of go, it ought also look like this. That's, that's a equivalent of an approval test, but probably just two, two. Um, uh, what's the word too brittle. Yeah. Um, I wanna talk back a bit about, um, performance because you know, if you like testing, um, then it's in character for me to talk about performance 141 00:34:48.390 --> 00:34:49.400 . 142 00:34:49.400 --> 00:35:32.200 I remember, um, at the time that we were first looking at, uh, integrating some of these, um, Chrome based trading apps, they were displacing some C#, uh, applications like thick clients that were, were going. And the concern was that with, you know, a, a couple of hundred, if not a couple of thousand, uh, stock tickers and all of the information about them, that there would be no way that the UI could keep up with the torrent of information that's coming down. The, the, the line, if you're a trader and you're saying, what's the trading volume in your little graphs, you can imagine little spark line type graphs, and then a huge giant monitor full of these things, all being updated in real time. You might imagine that it, it wouldn't be fast enough to keep up. And, um, 143 00:35:32.200 --> 00:36:17.660 Amazingly it was, I think that was the first surprise is that Joe was like, actually the Chrome people are pretty smart. And this was again, a decade ago, they've done a really, really good job of, of only updating the parts of the screen that actually changed. And if you're not daft in the way you write your JavaScript, if you're not always looking things up with like complicated CSS queries, you know, you, you know, whatever internal representation you have is, you know, you've got a, your array or your map of stock ticker to some object and that some object contains the actual Dom element references of like where the hell things are. Then it's really quite quick to sort of update the text of them or to, to manipulate, uh, a little canvas tag associated with them. And that was super, super surprising to us at the time. 144 00:36:17.660 --> 00:37:21.800 Uh, we did hit some level of, uh, limiting factors, uh, when we were doing, uh, live views of a particular, uh, stock tickers market. So hundreds of thousands of orders that were being added and removed from the market and a graphical representation of them. And one of the limitations was, was, uh, was bypassed by using canvases directly, which was one thing rather than having lots of DOM elements that you were manipulating in this particular case, where there were very, very specialist, uh, objects. So it made sense to take the time to actually draw them and undraw them ourself. Um, but, um, another thing was not using, uh, JSON com coms between, uh, the server and the, and the client like literally having like, well, there's a one character, which says if either, is this a new order? Is it the change to an existing order? Or is it a deletion of an order? It's like one character an ASCII followed by a space followed by a couple of other space, separated things and like hard coding our parser to do that was quite a win back then, but maybe nowadays evaling a JSON object is faster. I don't know. 145 00:37:21.800 --> 00:37:58.470 I actually did a test on this just, just about a year ago. And I found, yeah, I was, I was, cause I was thinking about, I was thinking about this exact scenario where it was sort of like, you know, we were communicating, uh, using these custom protocols instead of JSON I'm like, is this still worthwhile? And so I had like a, a five field message. And so I was like, I'm just gonna take this five field message and I'm gonna delimit it with some character. And I don't even remember what character I picked, probably colon or something like that. And I'm gonna have an equivalent JSON message with, with five key value pairs and I'm gonna see what the performance difference is. Just splitting the string on colons. 146 00:37:58.470 --> 00:37:58.890 Yep. 147 00:37:58.890 --> 00:38:10.740 versus parsing the whole message. And it was more than 10 X. It was more than 10 X faster, uh, in Chrome. Uh, and I was honestly surprised by that. Part of that might have been. 148 00:38:10.740 --> 00:38:15.760 I'm still surprised now. I mean, despite bringing it up, I thought it may have, you know, ed come together by that. 149 00:38:15.760 --> 00:38:37.880 But yeah. Part of this may have been the way that I had these messages structured is, um, the, just the nature. I forget exactly what I was doing, but there was something where it was like four, like reasonably small messages or fields in the message. And then one big field at the end that was like a, you know, like a one K field, you know? 150 00:38:37.880 --> 00:38:39.000 Right. 151 00:38:39.000 --> 00:38:54.840 Um, and I, and I did take advantage of a little bit of a trick to say, I know there's five fields here. I'm always gonna put the big field at the end and I'm gonna only split up to the first four, uh, colons. So that after you found that last one, you can, can just assume the whole rest of it 152 00:38:54.840 --> 00:38:58.820 So you don't even have to escape colons or anything that were in that blob at the end or whatever. Yeah, 153 00:38:58.820 --> 00:39:07.540 Yeah, yeah. It's just, it's just all in there. And so that was a little bit of a hack, but, but with that, it was like significantly faster to not use JSON, which was surprising to me. 154 00:39:07.540 --> 00:39:43.510 Yeah, I guess so. I mean, obviously JSON is the right or something like it is a very good choice almost all of the time because it's, self-describing at some level. It's human readable for some level of some definition of, uh, of human readable and, um, and certainly if you have to, if you're in the unenviable task of having to bust out wire shark and look at some kind of communications between a client and sever to work out what the heck happened, you've got more of a hope of understanding it, but for things where you have a very specific, um, client and server that just need to be very performant. It's still, I I'm surprised, but 155 00:39:43.510 --> 00:39:44.160 Yeah. 156 00:39:44.160 --> 00:39:44.880 Yeah. Maybe I shouldn't have been. 157 00:39:44.880 --> 00:40:08.680 And the dev tools just treat JSON so much nicer too, too, you know, you can go and you can look at those websocket messages going back and forth and it'll like parse them for you and do all the nice things for you. Um, the other performance thing that I, that I got from from Joe, and I don't know if this is still true. Um, but it certainly was true back then is, is just avoiding DOM layout and just using fixed positioning for everything. 158 00:40:08.680 --> 00:40:09.860 Oh, interesting. 159 00:40:09.860 --> 00:40:24.460 Right. Like if you and I, I remember him talking about this and I think it was with, with a ladder that he was building where he is just like, yeah, I just, I just absolutely position these things on the screen, in the places where I know they are and I, you know, can verify in the profiler that it's never triggering a layout when I change things. 160 00:40:24.460 --> 00:40:25.720 Got it. Yeah, that's an interesting one. 161 00:40:25.720 --> 00:40:44.960 Cause it's absolutely positions and it never has to re-layout. And that was a huge thing. I don't know. I, I don't know if that's still true, but like, it was certainly true back then. And that was one of the, sort of the, the little tricks that we use to, to make it, you know, as you say, you've got this giant monitor with all these different, you know, graphs and charts and, you know, position tables and things, and you just gotta make sure that it can keep up. 162 00:40:44.960 --> 00:41:23.100 Yeah. And, uh, we also experimented, although never came to any firm conclusions, uh, at the time, um, with using, uh, GL style things, you know, you've got WebGL that lets you do hardware accelerated things. So if you've got, uh, and obviously that's, you might naturally think of that as being a 3d thing. And indeed, you know, that's a lot of fun making a huge model spin around and be lit and everything, but you can use it for 2d operations as well. And it, that might fit the bill for certain things. We really have gone off of topic here. I dunno. I can't even remember what the topic was. It was just like web development our strange way, I suppose. So that kind of fits into this category. 163 00:41:23.100 --> 00:41:27.920 Yeah, it's sort of our, our strange model of web development that we, that we both, I think learned from Joe. 164 00:41:27.920 --> 00:41:49.000 So let's, let's sort of do a let's recap where we are with this. Yeah. And that is like, we both come from a luxurious position of being able to dictate which web browser people can use. Right. Right. We don't advocate the needless dependency on third party libraries. jQuery might get in there, but probably not anymore, maybe in 165 00:41:49.000 --> 00:41:50.560 Not even these days days, no. 166 00:41:50.560 --> 00:42:14.820 Underscore? Maybe may come in there. There certainly that's very few and far between that I would add libraries and that that's more to do with, um, being able to explain what the system is doing. Obviously, once it hits Chrome, you can't explain what's going on in there, but then there are Chrome dev tools and things that do help you there. But if there's any amount of eventing frameworky things in between and you know, maybe that's FUD-ish on our part, but certainly we have. 167 00:42:14.820 --> 00:42:25.560 Yeah, it is. I mean, it's, it's a little superstitious, it's a little FUD-ish, but at the same time, it's sort of like if I don't add those things, there's this whole category of problems that I can guarantee I won't have. 168 00:42:25.560 --> 00:42:25.800 Right. 169 00:42:25.800 --> 00:42:29.360 Now, would I have had those problems if I had added it in, I don't know. 170 00:42:29.360 --> 00:43:17.600 Have you booked in different problems by having to write reinvent wheels yourself maybe, but, so anyway, so minimizing the, your dependencies, I think in the sort of pre-conversation before this, you said, unless it brings a 10x benefit, then you don't bring it in that's that seems like a not unreasonable thing to say. Similarly with that performance kind of thing. Know if it's 10x faster than maybe it's worth doing it, a different way if you need the performance, rather than just sucking it up and using JSON, whatever, um, testing the DOM, that was another thing we brought up, uh, using the DOM as the kind of unit of, uh, what functions return and mutate and that kind of stuff. Uh, and yeah, that, that sort of a, a weird, uh, a weird way of developing web tools, but it works for us. 171 00:43:17.600 --> 00:44:17.460 Uh, yeah. So, yeah, I'd be interested in what other people think of, of this and whether they're horrified the fact that we're not using React.JS or dollar framework here, or jQuery. Yeah. Actually I, I'm gonna, I'm gonna sort of take a, walk this back a little bit. I've had some very good success with knockout as a framework, which is extremely, extremely thin framework. It's not really a framework at all. Yeah. Uh, but, um, that has, that's like the, the least framework-y framework that I've had good success with and certainly for slightly more complicated interactions and DOM, uh, manipulations where things you've got sliders that need to update numbers and uh, like in another field or vice versa where there's sort of bidirectional communications between like widgets, then something like Knockout. It can be a good thing. Uh, but, but yeah, I haven't used it for a long while. And compiler Explorer is mostly handwritten for what it's worth similar deal, right. Or everything inside of it, we do use a third party component called golden layout, which is like the windowing system. 172 00:44:17.460 --> 00:44:18.400 Oh Yeah. Uhhuh. 173 00:44:18.400 --> 00:44:38.040 And it, coincidentally was written by a bunch of folks who were, uh, at a trading company, and open source. So, uh, you know, if you need a big old window full of like sub windows that can be moved and repositioned and, uh, scaled and whatnot, then you know, it turns out that, uh, the trading industry apparently needs that more than most other industries. 174 00:44:38.040 --> 00:44:47.540 Yeah. Um, yeah. You talk about Knockout had me thinking, I, I have used a, I dunno if it's a library or framework. Depends on what word you want to use there called uh, backbone. 175 00:44:47.540 --> 00:44:48.960 Oh yeah. 176 00:44:48.960 --> 00:45:30.260 Um, for and specifically we used it for data binding. So this is this problem that we were just talking about, like, okay, you've got your template, right. You've just cloned your template. You're inside the function. You got your data as an argument to the function. You've got your template now what? Um, and that goes two ways. One is you gotta put the data into the template and then you might also want to have what some people call two data binding where it's like, okay, I put the data into the template. I put it on the screen. Now someone's changed it. I gotta pull it back out again. Right. And you can do a lot of stuff yourself. You can do a lot of stuff with some very simple conventions, right? Yes. I'm gonna name the field this, and I'm gonna use the same thing on the class. So I know, and I'm gonna select it. 177 00:45:30.260 --> 00:45:35.920 It. And I have my generic on, on change thing that just connects the two together or whatever. Yeah. 178 00:45:35.920 --> 00:45:47.670 Yep. Yep. So you can do it like that. Um, but there are also libraries and tools and things that'll do that for you. And, and Backbone was one that I remember using and remember not hating. So , I dunno how much of an endorsement that was 179 00:45:47.670 --> 00:45:49.000 That's uh, very weak. 180 00:45:49.000 --> 00:46:11.420 We definitely . Yeah. Right. But we definitely like recognize one of the, you know, one of the downsides to this, you were just saying like solving problems yourself that you maybe don't have to data binding is one of those problems where it's like, I I'm gonna put data into these DOM elements. I'm probably gonna have to pull it back out again, in other instances. Are you gonna hand craft all of that? Are you gonna have some simple mechanism for that? Are you gonna use a library for that? 181 00:46:11.420 --> 00:46:26.560 Right. And especially when there, there are some stupidities in the DOM where sometimes it's a dot value on something and sometimes it's the dot text and you're like, which one is it? I don't know. Oh, no. The number of times that I've, I've mucked that up and ended up with like an empty string or a none or something. 182 00:46:26.560 --> 00:46:36.480 And like escaping and dealing with, um, I mean, certainly jQuery's capable of handling this stuff, but it's sort of like escaping things so that you don't get like JavaScript injection attacks. 183 00:46:36.480 --> 00:46:40.200 Aw. Do you see? 184 00:46:40.200 --> 00:46:50.840 Yeah. You know, it it, uh, on purpose or otherwise, right. Oh, definitely have some things where it's like, where is the text of this thing? It's like, oh, there's some square braces or there's some angle brackets in there that's, that's not gonna show up that thinks that's a tag. 185 00:46:50.840 --> 00:47:29.180 So one of the, one of the internal tools that, uh, Joe was responsible for at DRW that we, we won't talk about what it is specifically, but it did have a way of naming, uh, some of the little tabs that you could put up. And, uh, rather, unfortunately, because Joe rolled it all himself, uh, it was totally, uh, exploitable. And, uh, and so I used to embed my JavaScript, emulator running, Sonic the hedgehog in, in a little pane. And then I would send, you could bookmark links that would like reproduce them. And then I would send it out to the traders and say, have you seen this so don't ever take that out. That's great. 186 00:47:29.180 --> 00:47:31.100 Yeah. Cool. 187 00:47:31.100 --> 00:47:37.420 In summary, then don't use a framework unless you have to use a framework but you sometimes use a framework but, 188 00:47:37.420 --> 00:48:30.880 Well, I would say this, I, again, I, I, based on my experience, and again, we have these, this sort of weird background and this weird experience doing this, but I don't feel particularly wrong in saying you should learn the web standards.They're there. They're always there. They're, they're gonna be there for a really long time. Uh, and it never hurts. Like if you think of yourself as a React developer, you think of yourself as somebody that maybe knows a lot of different frameworks. And as, as a web developer, um, there's a lot of knowledge there that you have, and that's great. And you will definitely not diminish that at all, by learning how the web standards work and learning how browsers, uh, work under like one layer below you. Um, because there's just so much gold there. There's so much great stuff there and you basically get it for free. So you should know about it. You should at least be aware that it exists. 189 00:48:30.880 --> 00:48:44.970 I, I, I'm absolutely loving this. You see me grinning like a fool at you while you've been telling this, because this is the argument I use. But as you substitute framework users with C++, or compiled language people with, and, and web standards with the assembly code. 190 00:48:44.970 --> 00:48:45.720 Yep. 191 00:48:45.720 --> 00:49:04.860 And we are making the same argument. It's like, you don't have to use it, but you should know it's there and have a working understanding of it so that you know, how the tools that you use are built on the layers and layers and layers above. And I mean, it goes all the way down as well. You can keep going down. It's great. That's what makes this job so interesting and working in the field 192 00:49:04.860 --> 00:49:06.500 There's always another layer? 193 00:49:06.500 --> 00:49:10.940 Every time you think you've hit the bottom. There's another layer. Yeah, absolutely. 194 00:49:10.940 --> 00:49:11.620 Yeah. 195 00:49:11.620 --> 00:49:14.600 All right. My friend, well, until next time 196 00:49:14.600 --> 00:49:17.600 Until next time.