diff --git a/transcripts/513-stories-from-python-history.txt b/transcripts/513-stories-from-python-history.txt index 57b7d4f..8291d4c 100644 --- a/transcripts/513-stories-from-python-history.txt +++ b/transcripts/513-stories-from-python-history.txt @@ -10,7 +10,7 @@ 00:00:34 This is Talk Python To Me, episode 513, recorded June 6th, 2025. -00:00:53 to unite. We started in pyramid cruising. +00:00:53 to unite. We started in Pyramid cruising. 00:00:56 Welcome to Talk Python To Me, a weekly podcast on Python. diff --git a/transcripts/513-stories-from-python-history.vtt b/transcripts/513-stories-from-python-history.vtt index fdd8459..7ba9b3f 100644 --- a/transcripts/513-stories-from-python-history.vtt +++ b/transcripts/513-stories-from-python-history.vtt @@ -19,7 +19,7 @@ You'll hear how Import This came to be and how the first PyCon had around 30 att This is Talk Python To Me, episode 513, recorded June 6th, 2025. 00:00:53.520 --> 00:00:55.820 -to unite. We started in pyramid cruising. +to unite. We started in Pyramid cruising. 00:00:56.120 --> 00:00:58.920 Welcome to Talk Python To Me, a weekly podcast on Python. diff --git a/transcripts/514-python-language-summit-2025.txt b/transcripts/514-python-language-summit-2025.txt index fed2300..8ff0a85 100644 --- a/transcripts/514-python-language-summit-2025.txt +++ b/transcripts/514-python-language-summit-2025.txt @@ -14,9 +14,9 @@ 00:00:31 This is Talk Python To Me, episode 514, recorded June 17th, 2025. -00:00:51 to unite. We started in pyramid cruising. Welcome to Talk Python To Me, a weekly podcast on Python. +00:00:51 to unite. We started in Pyramid cruising. Welcome to Talk Python To Me, a weekly podcast on Python. -00:00:54 to unite. We started in pyramid cruising. Welcome to Talk Python To Me, a weekly podcast on Python. +00:00:54 to unite. We started in Pyramid cruising. Welcome to Talk Python To Me, a weekly podcast on Python. 00:00:57 This is your host, Michael Kennedy. Follow me on Mastodon where I'm @mkennedy and follow the diff --git a/transcripts/514-python-language-summit-2025.vtt b/transcripts/514-python-language-summit-2025.vtt index 02daee9..edd5e2b 100644 --- a/transcripts/514-python-language-summit-2025.vtt +++ b/transcripts/514-python-language-summit-2025.vtt @@ -25,10 +25,10 @@ giving their attention to. This is Talk Python To Me, episode 514, recorded June 17th, 2025. 00:00:51.460 --> 00:00:53.760 -to unite. We started in pyramid cruising. Welcome to Talk Python To Me, a weekly podcast on Python. +to unite. We started in Pyramid cruising. Welcome to Talk Python To Me, a weekly podcast on Python. 00:00:54.060 --> 00:00:56.860 -to unite. We started in pyramid cruising. Welcome to Talk Python To Me, a weekly podcast on Python. +to unite. We started in Pyramid cruising. Welcome to Talk Python To Me, a weekly podcast on Python. 00:00:57.460 --> 00:01:02.700 This is your host, Michael Kennedy. Follow me on Mastodon where I'm @mkennedy and follow the diff --git a/transcripts/518-django-birthday.txt b/transcripts/518-django-birthday.txt index 8e70256..2176567 100644 --- a/transcripts/518-django-birthday.txt +++ b/transcripts/518-django-birthday.txt @@ -12,7 +12,7 @@ 00:01:02 This is Talk Python To Me, episode 518, recorded August 18th, 2025. -00:01:19 It's time to unite. We started in pyramid cruising. Welcome to Talk Python To Me, a weekly podcast +00:01:19 It's time to unite. We started in Pyramid cruising. Welcome to Talk Python To Me, a weekly podcast 00:01:25 on Python. This is your host, Michael Kennedy. Follow me on Mastodon where I'm @mkennedy and follow the podcast using @talkpython, both accounts over at fosstodon.org and keep up with the show and listen to over nine years of episodes at talkpython.fm. If you want to be part of our live episodes, you can find the live streams over on YouTube. Subscribe to our YouTube channel over at talkpython.fm/youtube and get notified about upcoming shows. This episode is brought to you entirely by Sentry. It's a bit of an episode takeover, if you will. Sentry has two excellent and exciting services to tell you about. Sear, your agentic AI debugging assistant, which takes all the data already gathered by Sentry to help discover the problems and even propose fixes as GitHub PRs. And the other is AI agent monitoring, which adds deep observability to your AI agents in your app. If you're adding AI and LLM features to your Python apps, you'll want to know about AI agent monitoring. I'll tell you more about both of these later in the episode. And remember, however you happen to sign up for Sentry, if you do, use our code TALKPYTHON, one word, all caps. diff --git a/transcripts/518-django-birthday.vtt b/transcripts/518-django-birthday.vtt index 06575a0..f5992f7 100644 --- a/transcripts/518-django-birthday.vtt +++ b/transcripts/518-django-birthday.vtt @@ -22,7 +22,7 @@ Finally, we look ahead at the next decade of speed, security, and sustainability This is Talk Python To Me, episode 518, recorded August 18th, 2025. 00:01:19.960 --> 00:01:25.480 -It's time to unite. We started in pyramid cruising. Welcome to Talk Python To Me, a weekly podcast +It's time to unite. We started in Pyramid cruising. Welcome to Talk Python To Me, a weekly podcast 00:01:25.860 --> 00:02:36.740 on Python. This is your host, Michael Kennedy. Follow me on Mastodon where I'm @mkennedy and follow the podcast using @talkpython, both accounts over at fosstodon.org and keep up with the show and listen to over nine years of episodes at talkpython.fm. If you want to be part of our live episodes, you can find the live streams over on YouTube. Subscribe to our YouTube channel over at talkpython.fm/youtube and get notified about upcoming shows. This episode is brought to you entirely by Sentry. It's a bit of an episode takeover, if you will. Sentry has two excellent and exciting services to tell you about. Sear, your agentic AI debugging assistant, which takes all the data already gathered by Sentry to help discover the problems and even propose fixes as GitHub PRs. And the other is AI agent monitoring, which adds deep observability to your AI agents in your app. If you're adding AI and LLM features to your Python apps, you'll want to know about AI agent monitoring. I'll tell you more about both of these later in the episode. And remember, however you happen to sign up for Sentry, if you do, use our code TALKPYTHON, one word, all caps. diff --git a/transcripts/536-fly-inside-fastapi-cloud.txt b/transcripts/536-fly-inside-fastapi-cloud.txt new file mode 100644 index 0000000..6701191 --- /dev/null +++ b/transcripts/536-fly-inside-fastapi-cloud.txt @@ -0,0 +1,2294 @@ +00:00:00 You've built your FastAPI app. It's running great locally. Now you want to share it with the world. + +00:00:05 But then reality hits. Containers, load balancers, HTTPS certificates, cloud consoles with 200 + +00:00:12 options. What if deploying was just one command? That's exactly what Sebastian Ramirez and the + +00:00:18 FastAPI cloud team are building. On this episode, we sit down with Sebastian, Patrick Arminio, + +00:00:24 Savannah Ostrowski, and Jonathan Ewald to go inside FastAPI cloud, explore what it means + +00:00:30 to build a Pythonic cloud and dig into how this commercial venture is actually making FastAPI, + +00:00:35 the open source project, stronger than ever. This is Talk Python To Me, episode 536, recorded January + +00:00:42 13th, 2026. Talk Python To Me, yeah, we ready to roll. Upgrading the code, no fear of getting old. + +00:00:51 Async in the air, new frameworks in sight, geeky rap on deck. Quark crew, it's time to unite. We + +00:01:01 Welcome to Talk Python To Me, the number one Python podcast for developers and data scientists. + +00:01:06 This is your host, Michael Kennedy. I'm a PSF fellow who's been coding for over 25 years. + +00:01:12 Let's connect on social media. You'll find me and Talk Python on Mastodon, BlueSky, and X. + +00:01:17 The social links are all in your show notes. You can find over 10 years of past episodes at + +00:01:23 Talk Python.fm. And if you want to be part of the show, you can join our recording live streams. + +00:01:27 That's right, we live stream the raw uncut version of each episode on YouTube. + +00:01:32 Just visit talkpython.fm/youtube to see the schedule of upcoming events. + +00:01:36 Be sure to subscribe there and press the bell so you'll get notified anytime we're recording. + +00:01:41 This episode is brought to you by CommandBook, a native macOS app that I built that gives + +00:01:46 long-running terminal commands a permanent home. + +00:01:49 No more juggling six terminal tabs every morning. + +00:01:51 Carefully craft a command once, run it forever with auto-restart, URL detection, and a full CLI. + +00:01:56 Download it for free at talkpython.fm/command book app. + +00:02:00 And it's brought to you by the Talk Python in Production Book, + +00:02:03 an inside look at 10 years of the real-world DevOps behind the Talk Python sites and apps. + +00:02:08 Check it out at talkpython.fm/DevOps book. + +00:02:12 Before we dive in, I have something excellent to announce. + +00:02:14 A few episodes back, I told you about our new AI integrations, + +00:02:18 the Talk Python MCP servers, and our llms.txt file, so that AI tools can tap into our over 550 episodes and 7.5 million words of info around the Python community and history. + +00:02:32 Well, I'm building on that now. + +00:02:34 Talk Python now has an open source CLI tool. + +00:02:37 You can search episodes, transcripts, guests, and even our courses right from your terminal. + +00:02:42 No browser required. + +00:02:44 It's fast too, backed by our optimized APIs, taking about five milliseconds of response time, + +00:02:49 plus, you know, whatever the internet ping time is. + +00:02:51 It supports rich text for humans, JSON for programs, and markdown output for AIs. + +00:02:57 You just install it with uvtoolinstall talk-python-cli, and check it out on GitHub. + +00:03:04 The link is in the podcast player show notes. + +00:03:07 I also wrote up a blog post on the hows and whys of it. + +00:03:09 Check that out over on the Talk Python blog at, well, talkpython.fm/blog. + +00:03:14 The full link to the exact article is in the show notes. + +00:03:17 All right, let's jump in. + +00:03:20 Hello, everyone. + +00:03:21 Sebastian, Patrick, Savannah, and Jonathan. + +00:03:24 Awesome to have you all here. + +00:03:26 Excited to talk about FastAPI Cloud. + +00:03:28 Welcome. + +00:03:28 Yes. + +00:03:28 Thanks for having me. + +00:03:29 Thank you. + +00:03:30 We're also ahead, Mike. + +00:03:31 What a project. + +00:03:32 It's been going on for a while. + +00:03:34 I've heard stuff from Sebastian that maybe something was brewing and all these things, + +00:03:40 but not too long ago, you all announced it. + +00:03:43 And I heard that FastAPI, some people have been using it recently. + +00:03:47 Some of the surveys show that some people use it for websites. + +00:03:51 I'm not sure. + +00:03:51 Rumors. + +00:03:52 Yeah, yeah. + +00:03:53 Rumors. + +00:03:54 Rumors. + +00:03:55 Oh, my gosh. + +00:03:56 I mean, congratulations on that. + +00:03:58 But before we dive into FastAPI and FastAPI Cloud, let's just do a quick introduction. + +00:04:03 Who are you? + +00:04:04 We'll just go around the Brady Bunch squares of our live stream here and start with Sebastian. + +00:04:08 You've been on the show a few times. + +00:04:09 In fact, you've been on the show just recently for a really fun episode, Sebastian. + +00:04:14 Who are you? + +00:04:15 In real, that was super fun. + +00:04:17 So, hello, everyone. + +00:04:18 I'm Sebastian Ramirez, or Tiangulo. + +00:04:20 I created FastAPI. + +00:04:22 That is this Python framework for building web APIs and backend. + +00:04:26 In case you've been living in a hole and haven't done any Python for 10 years. + +00:04:30 You also are famous for really pointing out the ridiculousness of modern tech recruiting. + +00:04:36 You know what I'm talking about? + +00:04:38 Yeah, you know, like it's fun. + +00:04:40 This is probably the thing that I am known for. + +00:04:42 It's for writing a tweet saying, yeah, what was it? + +00:04:47 that I saw a job asking for five years of experience in FastAPI, + +00:04:52 and I only had 2.5 since I created the thing. + +00:04:56 So you didn't qualify for the FastAPI job? + +00:04:59 I didn't qualify for that, yeah. + +00:05:00 And then the funny thing is, you know, like people sometimes, + +00:05:03 even people in Python itself, and tell me like, oh, wait, like you're, and I say like, oh, yeah, + +00:05:08 I created this thing called FastAPI. + +00:05:09 Oh, wait, okay, so what is FastAPI? + +00:05:11 Oh, wait, you are the guy from the meme, the meme about FastAPI. + +00:05:16 Are you serious? + +00:05:17 Yeah, you know, like suddenly that is super important that I am the guy for the meme about FastAPI. + +00:05:23 Not the guy from FastAPI, the guy from the meme. + +00:05:26 Oh my gosh, I saw you on TikTok. It was amazing. + +00:05:29 It was a live achievement. I wrote a viral tweet. So yeah, nice to meet you all. + +00:05:34 You know what? Sometimes your moment in the sun is not the one you expected. + +00:05:37 No, congratulations on how good FastAPI is. + +00:05:40 On the tweet. + +00:05:41 Exactly. You really nailed it. Patrick, welcome to the show. + +00:05:45 Nice to be here. Yeah, I'm Patrick. + +00:05:47 I guess the main thing I'm kind of known for in the community is like this library called + +00:05:50 Storary, which is similar to FastAPI, but instead of REST is for GraphQL. + +00:05:55 Other than that, I help organize PyCon Italy and I used to also do EuroPython as well, + +00:06:00 but I stopped because of way too many things. + +00:06:03 Yeah, that's pretty much me. + +00:06:05 How do you see GraphQL these days? + +00:06:07 Is it still popular? + +00:06:08 I think it's mostly popular in the enterprises, unfortunately. + +00:06:12 I'm a bit, to be honest, I'm a bit annoyed about the companies that do tooling around + +00:06:16 GraphQL because I don't know, I feel like they're not really pushing it forward. They're just, + +00:06:20 I don't know, trying to work with enterprises and that's it. Or maybe people think to AI. + +00:06:25 Yeah. It feels a little bit like the soap, soap, wisdom, XML, modern version. Savannah. + +00:06:33 Yeah. + +00:06:34 You like tapping out of being an organizer for EuroPython is like, you know, the classic + +00:06:40 open source oversubscribed doing all the things very relatable. + +00:06:44 Yeah. + +00:06:45 Yeah. + +00:06:45 But yeah, I'm Savannah. + +00:06:46 What can I say? + +00:06:47 I am on the Python Steering Council for 2096, which is very exciting. + +00:06:52 Congratulations. + +00:06:53 Oh, thank you. + +00:06:53 I am also the release manager for the upcoming version of Python, Python 3.16. + +00:06:59 And so that'll kick off later this year, which is really cool and very exciting. + +00:07:04 I work on CPython stuff, the JIT, arg parse, basically whatever needs help. + +00:07:09 It's kind of where you'll find me. + +00:07:09 Awesome. + +00:07:10 Congratulations on the Steering Council. + +00:07:12 And yeah, that's a lot of cool stuff. + +00:07:15 Hopefully we don't get a Python 4.0 right after 3.16 because then your job will never end, is what I've learned. + +00:07:23 Yeah, yeah. + +00:07:23 Benjamin Peterson, Python 2.7 forever kind of situation. + +00:07:28 Yeah, yeah. + +00:07:29 I mean, release management is still, I mean, it's still quite a commitment. + +00:07:33 It's like seven-ish years when you think about all the staggered releases + +00:07:36 because you're a release manager 2.0 and then you have the five-year maintenance cycle. + +00:07:41 So yeah, it's Python forever is what I only said. + +00:07:45 Yeah, it's probably not a fad. + +00:07:47 It's probably going to stick around, this Python thing. + +00:07:50 No, that's awesome. + +00:07:51 Congratulations. + +00:07:51 Also, cool with arg parse. + +00:07:54 I feel like that's making a strong comeback now that we have these AI things that can just put stuff together for us instead of like, oh, I need to depend on this library and that library. + +00:08:04 Like, I just need to take a few arguments and have a little help text. + +00:08:08 And it's like, well, you've already got this built-in thing. + +00:08:10 Oh, who knew? + +00:08:11 You know, people are like, oh, I didn't even know. + +00:08:12 I thought I used typer or click or something, right? + +00:08:15 There's, you know, the typers and clicks of the world. + +00:08:18 But sometimes you just want the simplest thing. + +00:08:20 And ArgParse is pretty great at that. + +00:08:22 Although it has many quirks that are probably and most definitely unfixable at this point. + +00:08:28 Because bugs are features when you have things that have been around as long as Python. + +00:08:32 But yeah, no, I mean, AI loves to write Python. + +00:08:35 I think it's like the language used the most, AI-generated code. + +00:08:39 I'll just say we live in weird times, very weird times. + +00:08:42 I would love a precedented time at some time. + +00:08:45 Exactly. + +00:08:46 Yeah. + +00:08:46 Can we just get the boring times? + +00:08:48 No, nothing interesting, please. + +00:08:49 What I said about GraphQL may sound like a bit of a smash, but I didn't mean it in a negative, + +00:08:56 super negative way anyway. + +00:08:57 Like it used to be all the enterprises were all about soap and wisdom and like subscribing + +00:09:02 your tooling. + +00:09:03 Please don't write me. + +00:09:04 I'm not trying to bash on your technology. + +00:09:07 All right. + +00:09:08 Jonathan, also welcome. + +00:09:10 Hi. + +00:09:10 Yeah, I'm not nearly as famous as everyone else in this call. + +00:09:14 I'm more infamous internally at FastAPI Cloud, I would say, for a bunch of things. + +00:09:21 I've heard of emojis or something along the lines of line. + +00:09:23 One Mima way. + +00:09:24 You're just one Mima way. + +00:09:25 Just one Mima way. + +00:09:27 Yeah, that's true. + +00:09:28 We keep piling them up internally. + +00:09:30 But yeah, I used to work with Patrick together for years, also on Craftia. + +00:09:34 Same library as him. + +00:09:35 That's how I know him. + +00:09:36 And that's why I, well, made a weird sound when you said soap. + +00:09:41 I've been with FastAPI Cloud since EuroPattern, actually, the last one. + +00:09:45 I promised Sebastian I would implement server-send events in FastAPI, + +00:09:48 and I haven't started with it yet at all, but somehow I'm still here. + +00:09:54 So that's great. + +00:09:55 Well, actually, yeah, and it was actually like a sneak peek, I guess. + +00:10:00 We already started having a bunch of chats and discussing what we do. + +00:10:04 Should we do it here? Should we do it there? + +00:10:06 What should we do? + +00:10:07 So like, yeah, it's something that is coming to FastAPI probably soon-ish. + +00:10:11 Like there was a lot of things that needed to happen before that. + +00:10:14 Like Patrick is slightly smiley, like, oh no, this is pressure. + +00:10:19 There were some things that needed to happen in FastAPI, as you know, dropping support for + +00:10:23 Pydantic version one or things like that, that just made the internal code so complex. + +00:10:28 And now that it's over, we can actually work more on improving performance, adding features + +00:10:33 and things like that. + +00:10:34 I definitely want to dive into how FastAPI Cloud has sort of influenced the whole FastAPI side of things. + +00:10:43 But I made aware that there is, in fact, an entire website, an entire website dedicated to the meme. + +00:10:51 Yeah. And out in the audience, we get, hey, everyone, is that the guy from the meme? + +00:10:54 And meme is greater than Nobel Prize. + +00:10:57 So, you know what? + +00:10:59 It may be true. + +00:11:00 It may be true. + +00:11:01 I recognize the person saying, this is the guy from the Ming. + +00:11:04 He might be my husband. + +00:11:09 Incredible. + +00:11:11 Incredible. + +00:11:12 All right, well, let's start with FastAPI Cloud, and then we'll bring it back around to the FastAPI. + +00:11:19 Let's talk origin story. + +00:11:21 So what is this FastAPI Cloud? + +00:11:23 Nice. + +00:11:24 So if you were looking at the FastAPI Labs website, + +00:11:27 that doesn't really show that much. + +00:11:29 If you click on the join the waiting list that takes you to the website for FastAPI Cloud, + +00:11:34 there we can see this is what we are building. + +00:11:36 This is the thing that we are doing. + +00:11:39 It's actually super simple. + +00:11:40 The funny thing is that the pitch, the explanation of the product is so short. + +00:11:44 So it's one command. + +00:11:45 It's FastAPI Deploy. + +00:11:46 You have a FastAPI app, you just hit FastAPI Deploy, and then it's on the cloud. + +00:11:51 We take care of everything. + +00:11:52 We build a thing, deploy it, handle HTTPS, auto-scaling, all this stuff. + +00:11:57 and then you can just like focus on building the application, + +00:12:01 building apps. + +00:12:01 The funny thing is that it's super short to explain, + +00:12:03 but then building it is so cold, Blake. + +00:12:06 I'm glad it's so short. + +00:12:07 So thanks for being here. + +00:12:08 That was a great show, y'all. + +00:12:09 Well, no, just kidding. + +00:12:11 I think it's a little bit like Jupyter Notebooks in that sense that like you all are taking one for the team + +00:12:18 so that other people can have a simple experience. + +00:12:22 Whereas, you know, it's like those Jupyter folks, they write tons of TypeScript and do all sorts of things + +00:12:27 that nobody wants to necessarily do in the data science space + +00:12:31 so that you can just drag your widgets around. + +00:12:33 You know what I mean? + +00:12:33 I think that is a great analogy. + +00:12:35 I feel like the deployment space, it's a bit of a mixed bag. + +00:12:39 And I've been really frustrated to the point such that I wrote a book about it, + +00:12:45 that I think about an alternative, that I think over the last five plus years, + +00:12:51 it's just trended towards a little more complex, a little more complex. + +00:12:56 could we just add one of these things? + +00:12:57 And oh, now we got these three. + +00:12:58 We need one more thing to like make sure those things are doing, you know what I mean? + +00:13:02 And it's just like, wow, why are there 200 choices in my console to use this? + +00:13:07 Which is like kind of funny, right? + +00:13:08 Because I feel like a lot of these companies started with this, like, + +00:13:12 I don't want to understand all the ins and outs of all the infrastructure + +00:13:15 that comes with the cloud service provider. + +00:13:17 And that's really complicated to understand because I'm an app dev + +00:13:19 and I don't know anything about, you know, whatever, right? + +00:13:22 Now we're like, I don't know, kind of slowly accumulating. + +00:13:25 complexity but i think one of the cool things about what we're building and i like i've worked + +00:13:30 on cloud tooling before is like this is like just spoke for python developers and i think that's like + +00:13:35 quite like unique in that like we are really trying to like bring the bleeding edge and like + +00:13:41 all the new tooling that people are using and making sure that we play well with like uv and + +00:13:45 like i think like the there's a lot of thought and care put into that by the team that's a super good + +00:13:52 point. I mean, I remember Azure came out with like, here's your platform as a service. You just + +00:13:57 upload your web app and we'll just take it and go. And now that thing is so complicated along with + +00:14:01 many, many others, right? It's not just them. It's you've got AWS, you've got Vercel. There's + +00:14:06 lots of things we could point at for, wow, there's a lot of options here, you know? + +00:14:09 And then there are a lot of tools and like, you know, like many tools and many companies are also + +00:14:14 like doing a great job at many of the things that they are doing. But in many cases, it's just so + +00:14:18 complex, it's so complicated. + +00:14:19 You know, like I was, I have always been so, what, so adamant, I think is the word, to + +00:14:25 just teaching people how to use the tools. + +00:14:28 I think I have the most documentation about how to deploy things on your own than any other + +00:14:33 framework. + +00:14:34 I have so much information. + +00:14:35 MARK MANDEL: I hear that all the time from people. + +00:14:37 They say one of the reasons they chose FastAPIs is because how clear the documentation + +00:14:41 was, you know? + +00:14:41 FRANCESC CAMPOY: And then the thing is, you know, like just learning all those concepts + +00:14:45 and learning all the stuff that needs to be learned + +00:14:47 just to deploy something, and then you barely have the minimum. + +00:14:51 It's like, this is just too much. + +00:14:53 It's too much complacency. + +00:14:54 I think for me, I guess personally, my analogy is that FastAPI Cloud + +00:15:00 is the equivalent of what FastAPI is to building web APIs and backend. + +00:15:06 You could do the same with any other framework. + +00:15:08 You could validate data. + +00:15:09 You could generate open API. + +00:15:11 You could have automatic docs, But you will probably have to do a lot of the wiring yourself and making sure that it's actually correct and that it doesn't explode, all that stuff. + +00:15:19 That is, you know, like we are trying to do a lot of that work for the final users. + +00:15:25 Yeah, and I think it's great. + +00:15:27 I think it's really nice to just provide this on-ramp because, as you said at the opening, when I asked, you know, what the origin story is just FastAPI deploy. + +00:15:37 That solves so many stories. + +00:15:38 But I'm sure behind the scenes, what happens is just about as simple as that. + +00:15:44 Oh my gosh. + +00:15:45 About that. + +00:15:48 Some of us don't even get to write Python anymore to make all of this happen. + +00:15:52 So speaking about taking one for the team. + +00:15:56 Yeah, that is taking one for a team, right? + +00:15:57 It is. + +00:16:00 This portion of Talk Python To Me is brought to you by us. + +00:16:02 I'm thrilled to announce a brand new app built for developers created by yours truly. + +00:16:07 It's called Command Book. + +00:16:09 You know that thing you do every morning? + +00:16:11 Open up six terminal tabs, CD into this directory, activate that virtual environment, + +00:16:16 run the server with --reload. + +00:16:18 Now, CD somewhere else, start the background worker, another tab for Docker, another one to tail production logs. + +00:16:24 Every tab just says Python, Python, Python, Docker tail. + +00:16:28 And you're clicking through them going, which Python was that again? + +00:16:31 Where my app is running? + +00:16:32 Then sometime later, your dev server silently dies because it tried to reload + +00:16:36 while you're in the middle of a code edit, unmatched brace, a half-written import, or something. + +00:16:42 Now you're hunting through tabs to figure out which process crashed and how to restart it. + +00:16:46 My app, CommandBook, gives all of these long-running commands a permanent home. + +00:16:51 You save a command once, the working directory, the environment, + +00:16:54 three commands like git pull, and from then on, you just click run. + +00:16:58 You can even group commands together to start and stop everything for a project + +00:17:01 with a single click. + +00:17:02 It also has what I call honey badger mode, auto restart on crash. + +00:17:07 so when your dev server goes down mid-reload, Command Book just brings it right back up + +00:17:12 and does so over and over until the code is fixed. + +00:17:15 It also detects URLs from your output so you're never scrolling through thousands of lines of logs + +00:17:19 just to figure out how to reopen your web app. + +00:17:22 And it shows you uptime, memory usage, and all sorts of cool things about your process. + +00:17:26 The whole thing is a native macOS app. + +00:17:28 No Electron, no Chromium, just 21 megs. + +00:17:31 And it comes with a full CLI so anything you've configured in the UI, + +00:17:35 you can fire off from your terminal with just a single command. + +00:17:38 Right now, it's macOS only, but if there's enough interest, + +00:17:41 I'll build a Windows version too, so let me know. + +00:17:44 Please check it out at talkpython.fm/commandbook app. + +00:17:49 Download it for free, level up your developer workflow. + +00:17:52 The link is in your podcast player's show notes. + +00:17:54 That's talkpython.fm/commandbook. + +00:17:56 I really hope you enjoy this new app that I built. + +00:18:00 Let's save the internals for a little bit later. + +00:18:02 Maybe what we could do right now, Maybe we could do a bit of a walkthrough of just kind of what it's like to set up an app from scratch, right? + +00:18:12 Nice. + +00:18:12 I see that uv is here, which is, I've been certainly an advocate for uv in all sorts of deployment, + +00:18:20 but especially when you have like repeated build type of scenarios for like Docker, + +00:18:26 Docker Compose or Kubernetes or whatever, uv makes that stuff so much faster and so on. + +00:18:31 So who would like to be my guide that just kind of talks us through what it means to set up a new project here? + +00:18:37 I mean, there is like this really nice command that Savannah built, just FastAPI new, + +00:18:41 which I think is something like, I don't know, like super helpful. + +00:18:45 What does FastAPI new do? + +00:18:46 Like, is that kind of a cookie cutter-esque experience or what is it? + +00:18:51 Yes, exactly. + +00:18:52 At the moment, Onesco holds a super basic FastAPI application using uv. + +00:18:57 It also installs dependencies, creates a folder, everything that you need. + +00:19:00 In future, I think we're going to plan support for templates + +00:19:03 so you can build multiple kind of things as well. + +00:19:06 But for now, it's just basically just uv FastAPI new, + +00:19:09 sorry, uvx FastAPI new, and then that scaffolds the project for you. + +00:19:12 I don't know if you want to try it live or... + +00:19:14 No, go ahead. + +00:19:15 Just, I would think it might disrupt you. + +00:19:18 Just let's talk us through it. + +00:19:19 It could work. + +00:19:20 I'm just going to put that out there. + +00:19:21 I'll tell you the most insane, like let's do that live on the podcast experience. + +00:19:25 I'm pretty sure, yeah, this is definitely the most insane. + +00:19:28 I had Matthew Rocklin on from Coiled, and those guys are all about like, hey, we're going to scale up like a bunch of available servers for you, right? + +00:19:36 So that you can do your data science. + +00:19:38 Like I want to do some ML thing, and it needs 500 servers. + +00:19:41 So during the podcast, he's, oh, let me just spin up 2,000 EC2 instances. + +00:19:45 Hold on. + +00:19:47 And then we ran some code on it during the show. + +00:19:49 And he's like, oh, let's try that on ARM. + +00:19:50 And then spin up another 2,000 on ARM Linux machines. + +00:19:53 I'm like, okay, that's nuts. + +00:19:55 But let's just. + +00:19:56 That's a lot of power. + +00:19:57 So I was impressed, but Patrick, sorry, I kind of did realty there. + +00:20:02 Let's talk through it. + +00:20:03 Yeah, so you do uvx FastAPI app, FastAPI new, then you specify the name of the application. + +00:20:08 And that's almost there. + +00:20:10 You just need one more command to deploy, which is FastAPI deploy. + +00:20:13 The first time it's going to ask you to log in or join the waiting list if you haven't been invited yet. + +00:20:17 It's still in beta. + +00:20:19 And then you follow the steps. + +00:20:21 So like FastAPI deploy, log in, decide the team. + +00:20:25 If you have multiple teams, design the application name, + +00:20:28 and then you wait a few seconds and the application is going to be live. + +00:20:31 And just to be clear, FastAPI new is not required if you already have a FastAPI app. + +00:20:36 If you've already written your own code and you have your application, + +00:20:40 you can just go right into logging in and deploying. + +00:20:43 This is just so that if you're starting something new, + +00:20:46 you don't have to do any thinking about all the right things that need to be there. + +00:20:50 So this is more of a greenfield application. + +00:20:52 I'm bootstrapping a project. + +00:20:54 Right, right, because you want to have the best structure. + +00:20:57 Now, it uses uv. + +00:20:59 It is nodes required. + +00:21:00 Yeah, I was going to say, do I have to use the uv project management type of thing? + +00:21:04 Do I have to use the uv.lock files and uv add uv sync? + +00:21:08 Can I do requirements.txt? + +00:21:09 What's the story there? + +00:21:10 Yes, so we support uv with uv lock. + +00:21:13 We also support the, forget the name, the other, the PyLock file. + +00:21:17 And we also support plain requirements.txt. + +00:21:19 And maybe something else, I don't know, Jonathan, can you? + +00:21:22 PyLock's pretty new, right? + +00:21:23 I think Brett Cannon just got that out pretty recently, right? + +00:21:26 Brett was pretty excited. + +00:21:27 I know. + +00:21:29 Implemented that. + +00:21:30 Oh, was he? + +00:21:30 Okay, I'm sure he was. + +00:21:31 That's awesome. + +00:21:32 He put years of work into that. + +00:21:33 And he also said that one of the motivations was also like, you know, like cloud providers. + +00:21:38 So it's like, yes. + +00:21:40 The other thing is like, you know, if you use other different package managers, if they use the standard PyProject Autonomous format, that will also be supported. + +00:21:49 That means that, you know, like if you use PDM or if you use poetry with one of the recent versions, like that will work. + +00:21:56 If you use a very old version of poetry or like you use some other strange package manager or something, that will probably be problematic. + +00:22:03 But for like most of the use cases that use the standard package formats, it will just work. + +00:22:08 And if you use uv, then like you're going to have the best experience because we are fans of uv and Astro. + +00:22:14 They've definitely put a dent in the way that sort of Python gets started and making that a lot easier. + +00:22:19 So it totally makes sense. + +00:22:21 And also, I noticed, speaking of uv, that there's, at least in the recommended way, or the way in the docs, let's say, + +00:22:29 it doesn't say, here's how you install FastAPI. + +00:22:32 You just, here's how you run FastAPI-new, leveraging uv, which then will silently install and manage. + +00:22:42 All right, that's pretty neat. + +00:22:43 That helps you guys tell a simpler story, right? + +00:22:47 Instead of, here's how you create the virtual environment to install our thing and so on, you know? + +00:22:51 The idea is to make it like, as I was saying, just super simple for people just to start from scratch. + +00:22:56 Like no idea how to create an app, how to start, how to create an environment. + +00:23:00 It's just you run this command and you're off to go. + +00:23:03 Off to the races, I'm missing sands. + +00:23:06 Anyway, that's what Colombians do. + +00:23:09 But then if you already have an app, you have like, you know, like anything with FastAPI standard installed, then like that also just works. + +00:23:18 And Savannah, you pointed out that it doesn't have to be a new project. + +00:23:21 If you want to start from an existing one, that's totally fine. + +00:23:24 But what do I got to do if I'm starting from, if I'm migrating an existing one? + +00:23:29 Like how easy or hard is this? + +00:23:30 I have some like legacy project demo apps I've built at other companies I've worked with that have used FastAPI. + +00:23:37 And I literally just ran like FastAPI login and then FastAPI deploy and it just worked, which felt really magical. + +00:23:44 Right. Like I think that's like, I don't know, like having worked on cloud products for quite a while, like I think one of the biggest gaps is like the just I don't know, like the disparity between like my local dev environment and what is actually like lives up in the cloud somewhere. + +00:23:59 And so being able to just run one command and having the project as it exists on my machine go and work somewhere without having to think about like the infrastructure. + +00:24:08 And of course, like, you know, we want to be like amenable to folks who do want a little bit, you know, like higher touch. + +00:24:15 But we also want to work for people who are like learning FastAPI and Python, right? + +00:24:19 Like educators and people that are teaching Python. + +00:24:22 I think this is like something that you've had some interest in as well from those folks. + +00:24:26 Yeah, I was just listening to the Teaching Python podcast books just the other day and thinking, you know, like this, when I look at this, I know this is not necessarily your focus, but certainly people who are trying to teach a class, be it college class or high school class or whatever. + +00:24:44 And if you build anything on the web, the next question is, this is cool. + +00:24:49 How do I share it with people? + +00:24:50 And then they're like, oh, no. + +00:24:52 Oh, no. + +00:24:53 Hold on. + +00:24:54 Like coding boot camps, right? + +00:24:55 Like if you're teaching someone how to write Python or how to build an API with FastAPI, + +00:25:02 like actually setting up the environment for them to deploy is not part of it, right? + +00:25:06 Like that's not actually part of the curriculum. + +00:25:08 It's like this peripheral thing that ends up eating up a bunch of the educator's time or + +00:25:12 the student's time trying to understand both like how to write code and then also understand + +00:25:16 cloud stuff. + +00:25:17 And that's like a lot to ask people when they're fresh up the gate. + +00:25:20 I feel the same way about like tutorials and stuff at conferences. + +00:25:24 Yeah, totally. + +00:25:25 Yeah. Or training sessions. If you're doing like corporate training or like, they're all like, + +00:25:29 Oh, well, let's get everybody's machine working. There goes an hour and whatever. But yeah, + +00:25:36 if you can just say, look, I think when you're either, when you're trying to learn something, + +00:25:40 you'll be it through school or on your own or through these like more structured ways, + +00:25:46 like bootcamps and training and so on. I think if it's not the main purpose, + +00:25:51 I feel so often there's like, we're going to do 20 steps for four hours before you get any sort of + +00:25:56 reward of what you've done. And if you can go, okay, do you have it running? Okay, now you run + +00:26:01 this command. Look, now it's on the internet. Like, oh, wait, awesome. I got an app on the internet. + +00:26:05 Everybody look at me. You know what I mean? And I think shortening that cycle to where people can + +00:26:10 have that aha moment. And then later they can dive into like, well, how is it really working? And what + +00:26:14 do we really need to understand? But that quick iteration cycle, especially in the early parts of + +00:26:21 in new tech. It's really important. But also, you know, like down the line as well, I think, + +00:26:25 like, I don't know, there are so many things that I have been wanting to build and I don't, + +00:26:29 but I didn't because it was just so complex to deploy stuff. You know, like knowing, + +00:26:34 knowing how to do the whole thing, how to set up the clusters, the machines, install the Linux + +00:26:40 systems, deploy the cluster, whatever, like all that stuff, deploy the things, + +00:26:44 handling load balancers and HTTPS. I'm like, you know, like I know how to do that. I built one of + +00:26:49 the most popular websites teaching how to use Docker to ARM, which was like the contender + +00:26:54 before Kubernetes won everything. + +00:26:56 I like it. + +00:26:57 But still, it's just so complicated, like doing all those steps that are like, yeah, no, I'll + +00:27:02 just not do it. + +00:27:04 Like some other day. + +00:27:05 Now I can just like play around and do random stuff and just like deploy when it just works. + +00:27:09 It is, I really like that. + +00:27:11 I guess like coming back to that, like taking one for the team point earlier, like I feel + +00:27:16 like building Python tooling. It's kind of like taking one for the team sometimes because you have + +00:27:20 these folks that are like, you know, brand new to Python. Like Python is an extremely approachable + +00:27:26 language for people who are new to writing code. But then, you know, we also want to make FastAPI + +00:27:30 cloud work for someone that's building like an enterprise grade application, right? And so like, + +00:27:35 like pretty wide spectrum of folks with like a million different use cases and different types + +00:27:41 of applications they want to deploy with different constraints and like security stuff. + +00:27:46 And like, so yeah, I think, I don't know, maybe that's just like Python tooling. + +00:27:50 It's a lot of work, I guess, to like build something that works for the masses. + +00:27:54 Yeah, well, it's certainly tough to make something that feels simple, + +00:27:57 but it's not overly simplistic, you know, that can actually solve the problems. + +00:28:01 Has the right knobs for the right users too, right? + +00:28:04 I would argue we're not only trying to do it simple and easy. + +00:28:08 I feel like we're choosing a particular flavor of simple, which is... + +00:28:13 We have this discussion a few times. + +00:28:15 It's like, if you make a cloud, how do we make it feel Pythonic? + +00:28:18 What does that mean in a cloud setting? + +00:28:20 We talk about Pythonic libraries, Pythonic coding style in the community a lot. + +00:28:25 And now we kind of try to transfer that flavor, that feeling to the cloud + +00:28:29 and make everything around that feel just like we want our libraries to feel. + +00:28:34 So you feel at home as a Python developer and it just feels right. + +00:28:37 So that's extra step on top of making it simple. + +00:28:40 And we discuss that a lot. + +00:28:41 That's how I feel about it. + +00:28:42 I love it. + +00:28:43 I think it's one of the coolest things about this team. + +00:28:47 Like, you know, like people are being able to hear a few of us. + +00:28:50 There's like, there are like a bunch of others, but like that each one of us is so passionate + +00:28:55 about the things that we are working on. + +00:28:57 So like, you know, like each one of us is trying to make the best out of the things that + +00:29:01 we are building. + +00:29:02 And then like, we are so passionate about the thing that we care about and that we are building. + +00:29:07 that I think that ends up being an amazing result. + +00:29:11 For example, the CLI. + +00:29:12 We wanted to have some specific, you know, like behavior, + +00:29:17 some look and feel. + +00:29:18 And like we wanted to be able to have like the best kind of CLIs. + +00:29:22 So Patrick went ahead and built this bunch of tooling + +00:29:25 that we needed to be able to have it and like made it open source and everything. + +00:29:29 So we could have this great experience when working with CLIs. + +00:29:33 Jonathan recently was doing so much stuff about the something that caches and handling security, + +00:29:38 making sure that everything was super secure, super fast, super snappy. + +00:29:42 You know, like Alejandra is super careful about all the UI. + +00:29:46 Martin is super careful about all the infra. + +00:29:49 You know, it's like this passionate mess, which is a word I just made up. + +00:29:57 This, Alejandra goes and says, like, this thing doesn't have the proper margins. + +00:30:01 We need to increase this a little bit. + +00:30:02 I don't like it. + +00:30:04 She just goes and fixes it. + +00:30:05 The same with Marvin. + +00:30:06 He says, like, we need to have, like, this sort of thing in infrastructure. + +00:30:09 And, like, just comes and tells me, hey, we are doing this. + +00:30:12 And he's like, yes, sir. + +00:30:13 Like this with the OpenVPN, like, Unix, for example, + +00:30:16 that is mainly focused on the open source, is constantly looking at all the discussions, PRs, conversations, + +00:30:22 making sure that everything that we do, that doesn't. + +00:30:24 So why, you know, like, there have been, like, recently way more releases of FastAPI friends of the open source projects + +00:30:31 and very fast book fixes, very fast responses to handle everything for the community. + +00:30:37 Now we actually have people that is paying attention constantly + +00:30:40 to what is happening, what are the things that we have to do, + +00:30:43 and that really care about that part as well. + +00:30:46 So I think this extreme care about what we are doing. + +00:30:50 You know, like Savannah is making Python. + +00:30:53 This detail that each one of us cares so, so much about each one of the things that we build, + +00:30:59 making sure that the product is actually amazing. + +00:31:01 It's as good as it can be, and we can all feel at home when... + +00:31:06 I get so excited for talking about it because I really enjoy the end result of the product + +00:31:12 and of being able to use it. + +00:31:13 I would use it in the end. + +00:31:15 I would use to work with it. + +00:31:16 It's super simple. + +00:31:17 Yeah, that's awesome. + +00:31:18 Hey, let me adjust your mic real quick. + +00:31:20 I think it was like ducking, ducking out a little bit. + +00:31:23 We just went through a lot, a lot of content and a lot of sweating + +00:31:26 because your microphone went through like six different stages of weirdness. + +00:31:31 I think that really leads to like something I wanted to talk about is just what impact has this had on FastAPI? + +00:31:39 And before you jump in and answer that question, everyone, there's especially I think with Astral, + +00:31:44 but with because they've had so much success, there's been an undercurrent of concern of like, + +00:31:49 oh, my gosh, commercialism is getting into our open source. + +00:31:53 And what if it pollutes it and causes these negative aspects? + +00:31:57 But just hearing all of the energy around FastAPI with so many people, + +00:32:03 because of FastAPI Cloud, that's super neat. + +00:32:05 So I wanted to throw out to you all, how has this building FastAPI Cloud and the existence of FastAPI Cloud + +00:32:12 been giving back to FastAPI, I guess? + +00:32:14 I'm waiting to see if someone will speak FastAPI. + +00:32:18 I'm always the one that is speaking the most. + +00:32:22 I mean, it might be your project. + +00:32:24 Like, you may have started the project. + +00:32:25 Yeah, maybe so. + +00:32:27 Like, last year, I had, like, a few keynotes in some picons in different places. + +00:32:31 And, like, one of the key points that I wanted to bring was this idea that I'm trying to show that, in many cases, people worry about the boss factor. + +00:32:41 And the boss factor is just this idea. + +00:32:43 Yes, yes, I've heard this, yes. + +00:32:45 Yeah, you know, like, the boss factor is the idea that, oh, what happens if, like, there's one person doing this work? + +00:32:51 What happens if a boss runs over this person? + +00:32:54 And there's so much worry about this boss factor. + +00:32:58 It's sort of a morbid analogy, but I understand, right? + +00:33:02 Like, what will happen to the open source project if the maintainer vanishes for some reason, right? + +00:33:07 Exactly. + +00:33:08 But, you know, like, it also applies to projects and to many other different things. + +00:33:11 But what I think is that it's a disproportionate amount of attention to this detail of the boss factor. + +00:33:19 And I think every time people talk about the boss factor, you know, like one of my points in what I was trying to say in these talks was I would like people to think about the boss ticket factor. + +00:33:30 Who is paying for those tickets? + +00:33:32 It doesn't matter how big is the team. + +00:33:34 You know, like you have seen Google, Amazon, Meta, all the big ones. + +00:33:38 They don't have a small boost factor. + +00:33:40 They have a lot of people in their payroll and still they finish products. + +00:33:45 They just cancel them. + +00:33:46 Open source or private or whatever. + +00:33:49 is not the main factor defining the success of a project, + +00:33:55 being it commercial or open source of any type, is not really how many people are behind it. + +00:34:00 It's more of what is the value that whoever is putting the effort to keep it alive + +00:34:05 is getting from putting all that effort. + +00:34:07 It could be just satisfaction. + +00:34:09 It could be like open source, like, oh, I feel so good that I'm contributing to society. + +00:34:13 And that is valid. + +00:34:14 It doesn't pay the rent, but it's still valid. + +00:34:17 It might last for a while. + +00:34:18 But then also like, you know, like when you see like there are so many Python projects, so many Python, so many open source projects that can do well or can do bad. + +00:34:26 And it doesn't really depend on how many people they have. + +00:34:28 And when you are using a project, when you're using an open source project or when you are using a product of any type, I will encourage you to think about what is the best ticket factor of this project? + +00:34:41 What are the things that whoever is building this is receiving in exchange for giving it away? + +00:34:47 So like, you know, like what are they expecting to sell you at some point? + +00:34:52 Or what are they receiving in exchange? + +00:34:55 You know, for example, Bun, the JavaScript Brompton LiDAR. + +00:34:58 Like it was like, we don't know what they're going to sell. + +00:35:01 But now, you know, Cloud and Entropic really want to have like this thing keep working because they are using it internally. + +00:35:07 So you can say like, OK, I'm going to use it. + +00:35:09 I'm going to use it for free. + +00:35:10 I know that what they receive for me using is just like that they just really want it. + +00:35:15 So I can just like, whenever you are using bond, you are getting, now you are getting free services + +00:35:20 from Antropoc, that's it. + +00:35:21 But you know, like every time you are using a project, + +00:35:24 you can think about why are people receiving an exchange + +00:35:27 for giving this away for me? + +00:35:28 This is like the thing that I would like people to think about, you know, like also like + +00:35:34 how can they give back? + +00:35:35 Maybe they can actually contribute to that community + +00:35:38 or to that project. + +00:35:38 There are many ways and in many cases, the thing that is needed the most is just like help + +00:35:43 and work, just answering questions and issues. + +00:35:47 this portion of talk python enemy is brought to you by us i'm excited to talk about my first solo + +00:35:52 book talk python in production it's an inside look at how we host all the talk python sites + +00:35:58 apis mobile apps and way more here's the thing i believe most hosting stories sold to developers + +00:36:03 and data scientists are way over complicated and overpriced you've heard me say you're not google + +00:36:09 you're not netflix so you shouldn't run your infrastructure the way they do but if not that + +00:36:13 then what? This book is both a blueprint for what I chose for Talk Python and a story arc of 10 years + +00:36:20 of running my own infrastructure from a complete newbie, apprehensive to Linux, to some pretty + +00:36:25 neat infrastructures code DevOps. It covers Docker, Nginx, Let's Encrypt, self-hosted analytics and + +00:36:32 monitoring, CDN setup, framework migrations, and a whole philosophy that I've termed stack native, + +00:36:38 keeping things streamlined, powerful, and free of cloud lock-in. And it's more than just your + +00:36:43 standard tech book. It comes with code and figure galleries on GitHub, a discussion forum, and + +00:36:48 something unique, over an hour of audio readers briefs, short conversations that bookend each + +00:36:54 chapter to prime your focus or broaden your takeaways. Oh, and 0% of this book was written + +00:37:00 by AI. Every word is mine, written over the course in high months, for better or worse. + +00:37:04 I've made the first third of the book available for free online. After that, you can grab the DRM + +00:37:09 free EPUB and Kindle editions. And I'm working on a paperback edition as well. Please check it out + +00:37:15 at talkpython.fm/DevOps, or just click book in the nav bar on the website. It's a great way to + +00:37:20 support the podcast. And I hope it changes a bit how you think about running your apps in production. + +00:37:26 Kind of related to what you're saying, I think one of the angles that I really appreciate about + +00:37:30 the way we think about FastAPI and FastAPI Cloud is like where like a lot of our team was involved + +00:37:36 in open source before coming to work at FastAPI Cloud on various projects around the Python + +00:37:41 ecosystem, outside of Python. + +00:37:42 And I think all of us have deep appreciation and understanding of the value of open source + +00:37:48 and really, really try and build in a way that is like, I mean, Sebastian, you've talked + +00:37:53 about this a lot, but solving a real problem for folks, right? + +00:37:56 And so FastAPI Cloud is sort of this extension of this open source ecosystem people would + +00:38:01 be using. + +00:38:03 FastAPI Cloud may be an option. + +00:38:05 Maybe someone picks some other cloud for some reason. + +00:38:07 I don't think like, I think we're all very mindful of that. + +00:38:10 But like the angle that's very cool, I think, is that like, because we all work at FastAPI Cloud, + +00:38:15 like I know that I personally have time, more time for my open source work + +00:38:19 and my employer understands the value of my open source work, + +00:38:23 which isn't that positive for the open source community. + +00:38:25 Like I get to work on CPython sometimes and I have, you know, the bandwidth + +00:38:29 to go and do my steering council work or upcoming release management work. + +00:38:33 I understand like this sort of like, tempering, like open source, commercial, bad, all bad. + +00:38:39 It's not all bad. + +00:38:40 It's actually like really good in a lot of cases for folks to build business. + +00:38:43 Look at uv for an example to hold up, right? + +00:38:46 Astral, yeah, yeah, totally. + +00:38:47 Yeah, yeah. + +00:38:48 I think there are some really good examples of this. + +00:38:50 So I think like that's another angle that, I mean, I really, I get a lot of energy out of our team + +00:38:55 because we all, I don't have to, I don't have to fight the open source battle + +00:38:59 at FastAPI Cloud. + +00:39:01 I think that's really cool. + +00:39:02 I do think that's super cool as well. + +00:39:03 Let me put out two examples for you. + +00:39:05 here that I think everyone will be aware of as sort of to add to what Sebastian was saying is + +00:39:12 look how much Apple freaked out when Steve Jobs died and how many people work at Apple, right? + +00:39:17 Like that was still like, oh my gosh. But, you know, I think there's, they're hanging in there. + +00:39:23 They're going to be probably making it. They are not in our business. I tell you what, + +00:39:27 they got some of my money. That's for sure. But also, you know, look at Flask, right? Armin + +00:39:35 drifted away, which is totally fine. And David Lord and Pallets picked it up and kept running, + +00:39:41 right? Like it's still one of the most popular frameworks out there, right? So it's, I think + +00:39:46 the bus factor is over, overblown a bit, but also looking at the team of folks here, I think it's, + +00:39:51 it's even more obvious that there's a bunch of people who are on the inside, you know? + +00:39:54 For example, Flask, you know, like I learned so many things from Flask and like, the thing is, + +00:39:59 I feel like sometimes, sometimes people go and complain about the tool and say like, oh, + +00:40:04 this is not working for this or for that. And in many cases, it's in this insensitive way towards + +00:40:09 the people that are working on that. And it's like, you know, like in the end, realize that + +00:40:13 there's actually people behind the scenes doing the work. And like, in many cases, it's just like + +00:40:17 one or two people doing a lot of work in many cases, just for free. And, you know, like, I think + +00:40:22 it's worth calling that out. Like all the work that David Lord does for Flask is just like so + +00:40:27 much work. And yeah, deserves a lot of respect. I totally agree. The other thing that I forgot to + +00:40:32 mention is that there are so many ideas of potential products that I could build over the years, and I + +00:40:37 never did, and I never started a company because I didn't have clarity of what will be a good thing + +00:40:43 to actually sell and will have a good alignment. The cloud product has such a good alignment with + +00:40:49 the open source side because as more successful FastAPI is, the more successful FastAPI cloud + +00:40:57 has a potential to be. + +00:40:59 The more people using Python effectively, the more people might end up checking out FastAPI + +00:41:05 and the more people might end up checking out the product. + +00:41:08 So if FastAPI does well, if the open source does well, + +00:41:11 if Python does well, that's better for the company. + +00:41:13 So it doesn't really depend on my personal principles + +00:41:17 and values or something like that. + +00:41:19 It's aligned with, it's financially aligned with the company. + +00:41:24 So it's just going to be beneficial in the end It doesn't depend on good intentions. + +00:41:30 And FastAPI is open source. + +00:41:31 It has like 7,000 forks or something. + +00:41:34 So if a boost runs over me, there are 7,000 forks. + +00:41:38 It's not going away. + +00:41:39 I definitely agree with you on that. + +00:41:40 I feel like I should maybe give a little bit of a, I'll tell a little bit of the story + +00:41:45 of what's going on with, where did I put it? + +00:41:47 I don't think I pasted it over here, is what's going on with Tailwind right now. + +00:41:51 And I think Tailwind is having a tough time, Tailwind CSS. + +00:41:55 Traffic to Tailwind is up six times year over year on npm downloads. + +00:42:02 But the revenue of Tailwind is down five times. + +00:42:07 You know, I mean, these are completely out of whack things because instead of people going + +00:42:11 to docs to learn about it, it's just like, well, when you go to the docs, you learn they + +00:42:15 also have premium offerings, right? + +00:42:17 And I think you guys are different because it's not just, oh, here's a little bit nicer + +00:42:22 of a thing, right? + +00:42:23 I feel like it would be a little bit as if you were selling cookie cutter templates for FastAPI apps, you know, it's like, well, the AI can make the shape of the thing that comes out of the cookie cutter, to be honest. + +00:42:34 But you're offering something that has ongoing value that it costs more and is more complex in other places. + +00:42:42 And so I think maybe just thinking about the how this just keeps the team going for FastAPI is really awesome. + +00:42:50 And I think it's got a nice flywheel effect there. + +00:42:53 is I'll link to this, I guess, audio track. + +00:42:56 I don't know what I call it. + +00:42:57 It's a blog post that has one sentence, but a 30-minute audio you can check out + +00:43:01 from the guy, Adam, who's one of the founders of Tailwind, + +00:43:04 talking about going into this. + +00:43:06 It's kind of rough. + +00:43:07 I think I don't necessarily want to go into a deep AI, + +00:43:10 what it means for the industry. + +00:43:11 Like, let's stay focused on what you guys are doing. + +00:43:13 But I think it's going to be its own series. + +00:43:17 I mean, Stack Overflow had as many questions asked this month as they did in the first month of their existence, right? + +00:43:25 Three or 4,000, whereas at their peak, they were 200,000 questions a month. + +00:43:29 There's like real turmoil that's coming from some of these things, which is tricky. + +00:43:35 But I'm really excited to see you all doing this because I'm a big fan of FastAPI. + +00:43:40 And I think this is just sustaining and more for FastAPI, right? + +00:43:45 Like, what do you all think? + +00:43:46 That's what we hope that is going on. + +00:43:49 I thought about Taiwan for a second, right? + +00:43:52 It's not like we're immune to what happened to them. + +00:43:54 Like we also have a lot of documentation online. + +00:43:56 AI could train on that. + +00:43:58 And if it's good enough, it could maintain your infrastructure and stuff. + +00:44:01 It's just too hard at the moment. + +00:44:03 And there's an additional thing we're kind of selling, which is like, I guess, responsibility. + +00:44:08 Like you're shifting the risk from like letting your AI or your infantry team maintain your infrastructure to us. + +00:44:15 So we're staying up at night and worry about it. + +00:44:18 that has a lot of value as well. + +00:44:20 And that's probably not going to get removed by AI. + +00:44:24 Here's a very common cloud code, cursor, whatever conversation. + +00:44:29 Hey, build me something with Python and needs an API. + +00:44:33 Okay, we built it with FastAPI. + +00:44:34 How do I host it? + +00:44:36 Right, that doesn't just, it will build a cloud for you, right? + +00:44:39 It's going to recommend something out there. + +00:44:41 And a real natural way to how do I host FastAPI is FastAPI cloud, right? + +00:44:46 Like if it suggests, oh, you're just going to like spread it across Lambda by breaking. + +00:44:51 Like, whoa, no, I want something simple. + +00:44:52 Okay, give me FastAPI cloud, right? + +00:44:54 I think that that's a really good thing. + +00:44:55 And then on the enterprise side, enterprise folks are notoriously not good at supporting open source + +00:45:03 in that they're not like paying for it. + +00:45:05 I know some companies are big supporters of the PSF and Python and open source. + +00:45:11 But in general, it's like, yeah, we have this project with 5,000 people working on it. + +00:45:15 It's all Python. + +00:45:16 And are we sponsoring this? + +00:45:19 Nope. + +00:45:20 We're just enjoying the money, right? + +00:45:22 And we're a bank. + +00:45:23 So we got the money. + +00:45:24 We got all the money. + +00:45:25 So they're just not good at paying for like a really great framework that they use a lot. + +00:45:29 But they got plenty of hosting, plenty of internal apps that they just need to make run and stuff. + +00:45:34 So I think both on like the low end and the high end, there's a lot of synergy between these things. + +00:45:40 That is not just, you know, slightly advanced, not to diminish it, but slightly advanced UI widgets that you could ask your AI to build or something or like cookie cutter templates for project starters. + +00:45:52 I think we are in a somewhat fortunate position of like, you know, like FastAPI. FastAPI has grown so much. + +00:45:59 Like, you know, like when you check the statistics about downloads or GitHub stars or entries in developer surveys, + +00:46:06 like it's at the top in like in each category. + +00:46:08 It's like, you know, like the backend framework with the most GitHub stars across languages, + +00:46:14 even like, you know, like Java, Go, Ruby, JS, like whatever. + +00:46:17 It's like the top one, at least in GitHub stars. + +00:46:20 So like, you know, like FastAPI is like people are liking it, fortunately. + +00:46:25 And there's probably going to be people deploying things to FastAPI Cloud. + +00:46:29 So that's probably going to be like, we are probably going to be fine. + +00:46:33 I think, you know, like the, I guess it will be like a good point to ask people to go and + +00:46:38 check where the open source project is that they are using and check where is the bus ticket + +00:46:42 factor for those open source projects. + +00:46:45 You know, like if you are using Tailwind CSS, it would have been very cool if at some point you check if the premium things were useful for you and for your company or your project or something like that, you know? + +00:46:56 Yeah, because what is the thing that keeps that project going? + +00:46:59 Exactly. + +00:47:00 And I really personally admire if a project or something offers like more value, not just, hey, buy me a coffee, but here's a thing that you get way more of, you know? + +00:47:12 And in that regard, I think Tailwind was doing that, right? + +00:47:14 They were offering this suite of pre-built things. + +00:47:17 And I think that that's great. + +00:47:19 But yeah, I do think you've got more of these crazy AI things + +00:47:24 are going to maybe recommend FastAPI Cloud more than they're just going to undercut it. + +00:47:28 So I think that's really great. + +00:47:29 And by the way, I was just looking for the GitHub Stars graph. + +00:47:33 Like there's a whole, I can't remember what the domain of that site is. + +00:47:36 And I ran across, by the way, I just want to give a quick shout out. + +00:47:39 Like your cult repo documentary on FastAPI was awesome. + +00:47:44 Right? + +00:47:44 That was so fun. + +00:47:45 They made me look good. + +00:47:46 I didn't see that coming. + +00:47:47 Yeah, it came right on the heels of the Python official + +00:47:50 documentary, the one hour one. + +00:47:51 This is the same group, and the production quality is really nice. + +00:47:54 So like-- + +00:47:55 When they released the trailer for the Python documentary, + +00:47:58 before releasing the documentary, when they released the trailer, + +00:48:01 they contacted me and said, hey, we're doing these mini documentaries about different frameworks, + +00:48:06 different tools, and we want to include FastAPI there. + +00:48:08 I was like, oh, nice. + +00:48:10 But then I was just trying to stay excited, but super excited. + +00:48:13 Oh, that's so cool. Yeah, I watched it as soon as it came out. So I'll link to that. People should + +00:48:17 definitely, it's only like 10 minutes or something, but it's worth it. We're checking out. So it's not + +00:48:22 a huge investment time. People can watch it, I suppose. It's not TikTok. I mean, it's not like, + +00:48:27 oh, I saw the documentary, but it doesn't take you on huge about it. + +00:48:34 You have to listen for 10 minutes, overly excited Colombian. + +00:48:37 I don't understand what's happened to the attention span of society. I'm really, + +00:48:41 honestly a little concerned. I used to, when I would create my courses, people would say, + +00:48:45 you know, like a four hour course and there'd be like a 10, 15 minute sort of, Hey, here's how you + +00:48:49 set up your computer. And here's all the introduction and people, Oh, that's so awesome. + +00:48:52 I loved how you kind of set the stage. I'm really motivated to take the course. + +00:48:55 Nowadays, I just get messages like, why are you still talking? This is five minutes long. Do you + +00:48:59 understand? I'm like, this is your job. You can't spend five minutes. Oh my gosh. Anyway, that's, + +00:49:06 that's sort of the origin of my comment there. It's all right. So we're kind of getting so much + +00:49:11 time, I think I want to talk about a couple of things. Let's talk a little bit about internals. + +00:49:17 Like what, I don't know who wants to take this one, but let's talk about just how, when I say + +00:49:23 FastAPI deploy, then what? It's just a uv pip install and it just goes and it's magic and it's + +00:49:30 easy, right? We have a nickname for Jonathan. Can we say it or no? I don't know. It's so funny. + +00:49:34 This happened because I told my friends, I'm so concerned about being at the podcast because + +00:49:38 Because everyone here is a visionary, and then I'm the back-end guy. + +00:49:42 I think the things I could contribute to this conversation, I should probably keep to myself. + +00:49:47 But you're just leaking your turnouts, right? + +00:49:50 There are some things that are not really secret. + +00:49:53 Like, as Sebastian said earlier, Kubernetes 1 in the infrastructure and deployment field, to some extent. + +00:50:01 So that's somewhere in there, right? + +00:50:03 But it's all the way deep down, so no one has to worry about it. + +00:50:07 But it's still a foundation. + +00:50:08 which is a good foundation. + +00:50:09 I think one thing that's, you might have guessed it, + +00:50:12 but FastAPI Cloud is built on FastAPI, which kind of makes sense, right? + +00:50:16 And that also has an effect on like recent patches, updates and stuff. + +00:50:20 Because if we find something internally which we're not happy with, + +00:50:24 then we just fix it. + +00:50:25 And that's how some releases came out faster than months before. + +00:50:30 Power of dogfooding. + +00:50:31 Yeah, that's awesome. + +00:50:32 Dogfooding a lot. + +00:50:33 Also all the related libraries like SQL model and, well, others. + +00:50:38 they experience the same thing. + +00:50:40 New library is coming out. + +00:50:42 Patrick will announce at some point soon. + +00:50:43 It's not just FastJPay and friends. + +00:50:45 We're like really open. + +00:50:47 Like recently, Patrick just open-sourced everything we use for authentication authorization, + +00:50:52 for example. + +00:50:52 Is it open-source yet? + +00:50:53 Did they just leak something? + +00:50:55 It will be announced soon at some point. + +00:50:57 We build stuff internally in the moment really. + +00:51:00 Like we build it in a way, like in a separate package, + +00:51:03 just like an open-source library. + +00:51:04 And if we feel like the time is ripe, it's just getting open-sourced + +00:51:07 because a lot of things are reusable. + +00:51:09 And that's in all departments. + +00:51:10 That happens a lot. + +00:51:12 When I started there, I already realized that. + +00:51:14 Everyone was building open source, but now I joined in myself as well. + +00:51:17 I open source the library for compressing and decompressing archives in Python + +00:51:24 because the internal top high thing is just slow and we needed it to be faster + +00:51:28 because we're staring at the deployment process and we're like, hey, we could probably shave off a few seconds here. + +00:51:33 Then that's just open source for everyone to use. + +00:51:35 So we're contributing to the old Python ecosystem as well. + +00:51:38 You have to say the name. + +00:51:40 It's so good. + +00:51:41 Is it good? + +00:51:42 No, it's just, it's faster because it's, you know, faster than just tar. + +00:51:47 Fast tar? + +00:51:48 I love it. + +00:51:48 Fast tar, yes. + +00:51:49 And you can say with that very, very German accent, fast tar. + +00:51:53 I'll go star it. + +00:51:54 We'll get you some stars. + +00:51:56 This is going to happen. + +00:51:57 That's the irony about it. + +00:51:58 Like, it literally has no stars. + +00:52:00 But if you scroll down, you see the downloads. + +00:52:01 That's going to prove we're actually using it. + +00:52:04 Yeah, I like it. + +00:52:05 It's a little context manager. + +00:52:07 It's almost working the same as the TAL file in the standard library. + +00:52:13 Like the same, like almost similar API to that. + +00:52:16 It's basically a drop-in replacement, more or less. + +00:52:19 But they know they need everything to happen in Rust. + +00:52:21 Because Rust. + +00:52:22 Because Rust, yeah. + +00:52:23 Well, as soon as it becomes infrastructure and you've got to run it a million times, + +00:52:27 that starts to make sense, right? + +00:52:29 Yeah. + +00:52:29 Python is one of the fastest programming languages in the world. + +00:52:33 when you think about human time to build the things, right? + +00:52:37 Like that's one of its real superpowers is like, I mean, there's the whole story of Google Video and YouTube, right? + +00:52:44 And Google Video was written in C++ with 100 engineers + +00:52:47 and YouTube was a small team in Python and they just couldn't keep up with the features. + +00:52:51 So they bought this little old thing, YouTube, and see if we're going to make something with it. + +00:52:55 And last I checked, it was still in Python. + +00:52:57 I'm sure some of it isn't, but a few years ago it was, which is wild. + +00:53:00 Anyway, there's different ways of fast, But when it's down to like little utilities, yeah. + +00:53:04 I know some people that are trying to make Python fast. + +00:53:06 I know a couple. + +00:53:07 Yeah. + +00:53:08 And honestly, massive success in the last five years, right? + +00:53:11 Like since 3.11, since the specializing adaptive interpreter, there's been pretty big improvements. + +00:53:17 3.9 and 3.11 did a lot of like foundational work. + +00:53:20 And then 3.9 onward really just uncorked a lot of innovation there. + +00:53:24 Yeah, that's pretty awesome. + +00:53:26 All right. + +00:53:26 It sounds like, Sebastian, you've talked a lot about Kubernetes. + +00:53:29 So I imagine Kubernetes is happening. + +00:53:31 Do we get to pick what data centers it runs on? + +00:53:34 Do we get to pick what clouds it runs on? + +00:53:37 You're going to get to pick some of these things. + +00:53:41 Not yet. + +00:53:42 It's not released yet. + +00:53:43 But, you know, like it's top, of course. + +00:53:45 Like we have like regular cloud providers on the MIT + +00:53:48 and there's a bunch of Kubernetes. + +00:53:49 Then there's a bunch of additional stuff that needs to run on top. + +00:53:53 Then there's like custom Kubernetes controllers and things that Jonathan was saying + +00:53:58 that he's having to write in Go so that people in Python can be happy to be able to, you know, + +00:54:04 like manage all the Kubernetes shenanigans that need to happen + +00:54:07 because there's so much complexity that needs to be handled. + +00:54:10 There's a lot of that. + +00:54:11 We do a lot of advanced tricks also. + +00:54:15 Jonathan was recently doing a bunch of advanced tricks + +00:54:17 to handle the caches for the builds. + +00:54:20 So the way that we handle caches, and we also like tap into uv and how things work + +00:54:25 so that builds can be super, super fast because it's like something is, + +00:54:30 we are, you know, like we are very much targeted at FastAPI and Python in general. + +00:54:36 So we can take advantage of knowing how things run internally, + +00:54:40 how things are installed, how to optimize everything. + +00:54:43 So everything is just like super fast, super fast to install, to run, to like do everything. + +00:54:47 I imagine you all have base Docker images that are like just one layer away + +00:54:54 from whoever's code is running. + +00:54:56 You know, like you've got it all optimized, already pre-built with FastAPI and whatever settings of Python you want. + +00:55:02 A bunch of things and tricks. + +00:55:04 But there are also different things, like the different ways that we do to actually build the things + +00:55:08 and install things and put them inside of the actual build application. + +00:55:14 There's a lot of sourcing that we do there, and Jonathan has been working on a lot of that. + +00:55:21 And there's also all the logic and all the stuff. + +00:55:23 We have a bunch of stuff on top of that to handle out of scaling, + +00:55:27 which is something that is actually not that easy to find in different providers. + +00:55:31 We have auto-scaling based on requests, including scaling down to zero, which saves costs. + +00:55:37 But this is not Lambdas. + +00:55:40 It's not AWS Lambdas. + +00:55:42 It's like the full deployed application, the full container or whatever it is, + +00:55:47 which is the full thing with all the dependencies. + +00:55:50 It's running for whenever it has to run, but we can scale based on requests. + +00:55:54 So I guess it's like the type of thing that you will have if you have this giant cluster for a huge enterprise with a bunch of infra people making sure everything just works perfectly. + +00:56:08 But you just pay us to do that for you. + +00:56:10 This is also a good time for us to probably say lots of stuff is coming and we're in private beta. + +00:56:15 And so you should sign up for the wait list so that you can get admitted and try out these very cool things we've been talking about. + +00:56:22 Absolutely. + +00:56:23 And I think I'll let Tech Insider out in the audience sort of lean into it. + +00:56:29 Public release one. + +00:56:30 Sebastian, when? + +00:56:31 Public release one. + +00:56:32 My final topic, which is just what's the roadmap? + +00:56:35 When is this stuff? + +00:56:36 Like, how do we get into it here? + +00:56:38 Why would they like Litestar? + +00:56:40 We have the, right now we have the waiting list and we are onboarding people. + +00:56:44 We already have like a bunch of people in the private beta. + +00:56:46 We're going to keep onboarding people from the waiting list and like, you know, like ramp that up. + +00:56:51 But it will be like, you know, like through the waiting list is the main place where we are onboarding. + +00:56:56 People will want to make sure that everything is super fine-tuned. + +00:56:58 And we're going to keep it that way for a while. + +00:57:00 So like people that are on the waiting list are going to be like the ones that are going to be able to start using it the soonest. + +00:57:06 At some point, we'll probably have ways for people to invite others and things like that. + +00:57:10 About the things that we are building, we want to, you know, like we are super focused on FastAPI and then Python in general. + +00:57:18 at some point will probably support different tools, + +00:57:21 different ways to run, also like Python code in general, + +00:57:24 probably different frameworks. + +00:57:25 It will also depend a lot on what the users are asking for, + +00:57:30 whether like the tools, the frameworks, the use cases, + +00:57:33 the things that they need to build. + +00:57:34 And like, we're going to evolve the platform and the system + +00:57:38 based on what people need out of it. + +00:57:40 We have like a GitHub repo where we have issues, but we also have like a Slack that once people are admitted, + +00:57:45 they can talk directly to us and that feedback is really, really valuable for shaping the roadmap + +00:57:50 and figuring out all the fun things you want us to support. + +00:57:53 Awesome. Of course, you're going to charge money for it. It runs on servers and you guys are not + +00:57:59 a charity, but can you give any sense of what you're thinking about that kind of stuff or + +00:58:05 join the waitlist and see? + +00:58:06 Well, first join the waitlist and see, but we don't have like that predefined yet, + +00:58:12 but they will be on the ballpark of what you could get from different cloud providers. + +00:58:16 So different similar-ish providers will be on the ballpark of what you will get. + +00:58:24 But it's not written in stone yet. + +00:58:27 It's still a little bit different because we can auto-scale based on requests. + +00:58:32 So we can increase the amount of replicas of your application automatically, + +00:58:36 and then we can decrease them automatically, and we can scale down to zero. + +00:58:40 So you can probably handle all the load that you need and in the end spend a lot less because you don't have to have a bunch of instances constantly running or things like that, you know. + +00:58:50 So it will probably work a little bit different than what it will be for other providers. + +00:58:56 But in the end, it should be roughly similar. + +00:58:58 Okay. And given the fact that you all handle so much of it as a platform as a service type of thing, you don't have to have a cloud expert on hand or a DevOps expert necessarily, right? As soon as a company hires somebody to be an AWS cloud architect or something like that, it's no longer just what is your AWS bill. + +00:59:18 It's also a little bit of pain that we are swallowing so you don't have to take it. + +00:59:22 Exactly. It's part of taking one for the team, right? + +00:59:24 Yes. + +00:59:25 Yes. + +00:59:27 Indeed. + +00:59:27 All right, so I had one or two things specifically that I was seeking. + +00:59:32 It's like custom domains. + +00:59:34 How far off are custom domains? + +00:59:35 I was like, oh, I could put some cool things on there. + +00:59:39 I could tell Jonathan is psyched about this. + +00:59:41 It'd be really fun to put one of my really small FastAPI projects over there, + +00:59:46 something I set up for some of my courses or something, + +00:59:48 and then I can point people to go, look, it's running on FastAPI Cloud. + +00:59:51 How neat, you guys can check that out over there. + +00:59:54 And I'm like, but it's on its own domain, that domain is baked into the course videos, you know what I mean? And it's written in stone. + +01:00:00 It's marketing. + +01:00:01 Yeah, exactly. So I can't really move it because it has, you know, some subdomain of Talk Python, + +01:00:08 right? + +01:00:08 I was working on it. And then I got the notification by Google Calendar that I should + +01:00:12 join a certain podcast. So... + +01:00:14 Are you telling me we don't have custom domains? Because I'm here asking you about custom domains. + +01:00:19 How meta is that? + +01:00:19 You got it. It could be here already. But no, you have to wait a bit more. + +01:00:23 Okay. But soon? + +01:00:24 Yeah. As soon as broad enough, but I'm actively working on it. Let's put it like that. + +01:00:29 Okay. That sounds great. And then, I mean, just, it's never simple. You know, I just, + +01:00:34 I set up some stuff and it's like, you get the pop-up. Oh, you got to put this, you know, + +01:00:39 this TXT record or this CNAME or whatever record into your DNS and then we're checking it. Oh, + +01:00:45 it might take three days for your DNS to propagate. So hang in there and just, I can imagine like + +01:00:50 you're having fun yeah i guess you're kidding me that's like wow i i thought i'm almost off work + +01:00:56 but no you're bringing it all back but yeah that's a that's a thing i'm sure the company + +01:01:00 could support therapy to like work work through the issues and the trauma that you've suffered + +01:01:04 from the dns it's always dns that's right i mean you got that's an yes it's always dns yes + +01:01:10 i guess one of our goals with custom domains also to make it super simple for you to set up them + +01:01:15 Like, for example, if you're using one of the providers that support OAuth, we can also just do one click and then it's going to be automatically. + +01:01:23 Oh, that's cool. Yeah, that's really nice. + +01:01:25 But unfortunately, it depends on the platform you're using. + +01:01:27 All of them support this. + +01:01:29 This is said by the person in charge of most of the integrations. + +01:01:32 So Patrick has built, we have integrations for a bunch of database providers and things like that. + +01:01:38 I think now Patrick knows by memory, open ID specification. + +01:01:42 I don't know. + +01:01:44 Yeah, the other thing I wanted to talk a bit about was just integrations, like what kind of stuff you guys have coming in. + +01:01:48 I saw that Hugging Face is going to be integrated soon. + +01:01:52 You've got Supabase, which is kind of Postgres as a service. + +01:01:56 There's a lot of those things out there that theoretically could be added. + +01:02:00 Someone also asked for MongoDB. + +01:02:01 Maybe that's one that we're going to take a look into. + +01:02:04 It really depends on the provider. + +01:02:06 So at the moment, we don't want to ask databases for you because that's also another kind of rabbit hole. + +01:02:11 Jonathan is probably not ready for that. + +01:02:14 But yeah, definitely database. + +01:02:16 But I guess we can say that we're also talking with the people from Pydantic + +01:02:21 so we can integrate maybe Logfire automatically, that kind of stuff. + +01:02:25 Yeah. + +01:02:25 And also things like Redis, which is also another kind of database. + +01:02:29 That's also coming soon. + +01:02:30 Yeah, there's a couple of database as a service type things + +01:02:33 that don't require too much other than just connecting API keys + +01:02:37 and something like that, right? + +01:02:38 Those seem like low-hanging fruit. + +01:02:40 Like the kind of goal with the integration is not just done. + +01:02:44 Like, yeah, right now it's just setting up an environment variable. + +01:02:47 But the idea is also to more, I don't know, like the proper integration, I would say. + +01:02:52 Like, for example, for things like Superbase, if, yeah, I think that's support branching. + +01:02:56 Like, for example, once we support ProQuest Previews for GitHub, like we can also create + +01:03:00 a branch automatically for you if you have the Superbase integration enabled. + +01:03:04 And we can do this kind of stuff as well. + +01:03:06 Or even we could show like some information about database. + +01:03:10 I don't know, like load or like memory usage, things are directly from our dashboard. + +01:03:14 So you don't have to go there. + +01:03:15 That's the main reason why we're building this infrastructure + +01:03:18 for the integration. + +01:03:19 Well, people can sign up to the waiting list and hopefully get on the private beta. + +01:03:24 We actually check the waiting list. + +01:03:26 We actually check the use cases, team sizes, like what are people building with it? + +01:03:31 Like we actually go and check it and we bring in people from the waiting list. + +01:03:37 Nice. You know, I didn't join the waiting list directly. + +01:03:39 I was added by some guy I know who was very kind to help me get some behind-the-scene look. + +01:03:45 So I don't know what the process is. + +01:03:46 Do you actually say what you want to do with it? + +01:03:48 And you evaluate that a little bit as well based on, like, + +01:03:51 hey, this would be a cool use case for us to support? + +01:03:53 There are many types of applications and many types of different team sizes, + +01:03:57 many types of things that people might want to build. + +01:04:00 And we try to see, like, okay, where is a case where we could be a good fit and we can provide a great service? + +01:04:06 And where are the things that people are trying to build? + +01:04:09 Also, it also helps us see, like, you know, like, what are people trying to do with Fasted + +01:04:14 Vehicle so that we know what we have to provide? + +01:04:17 But we actually go and check those, you know, like those submissions on, like, sexually thousands + +01:04:24 of people in the waiting list, but we still go and check and approve kind of manually still + +01:04:30 to bring a bunch of people on board in the different ways that we have been bringing people. + +01:04:35 So if people go and join the waiting list and actually tell us what they are, what is their + +01:04:39 use case, their team, what are they planning on doing? + +01:04:42 There's a much higher chance that we are going to go + +01:04:45 on to bring them up. + +01:04:46 MARK MANDEL: Awesome. + +01:04:47 So everyone, go join the waitlist. + +01:04:49 If you're doing FastAPI, I'll link to it in the show notes, + +01:04:53 of course. + +01:04:53 Thank you all for being here and sharing the story. + +01:04:57 And I, for one, am very excited to see FastAPI Cloud exist + +01:05:01 and just one more way to make FastAPI stronger and more resilient and so on. + +01:05:06 FRANCESC CAMPOY: Thank you very much. + +01:05:07 Thank you for having us. + +01:05:08 Yeah, it's super fun. + +01:05:09 Thanks for having us. + +01:05:10 Yeah, you bet. + +01:05:11 Bye, everyone. + +01:05:11 Bye, folks. + +01:05:12 Bye-bye. + +01:05:12 Bye. + +01:05:14 This has been another episode of Talk Python To Me. + +01:05:16 Thank you to our sponsors. + +01:05:17 Be sure to check out what they're offering. + +01:05:19 It really helps support the show. + +01:05:21 This episode is brought to you by CommandBook, a native macOS app that I built + +01:05:26 that gives long-running terminal commands a permanent home. + +01:05:29 No more juggling six terminal tabs every morning. + +01:05:31 Carefully craft a command once, run it forever with auto-restart, + +01:05:34 Ural detection, and a full CLI. + +01:05:36 Download it for free at talkpython.fm/command book app. + +01:05:41 And it's brought to you by the Talk Python in Production Book, + +01:05:44 an inside look at 10 years of the real-world DevOps behind the Talk Python sites and apps. + +01:05:49 Check it out at talkpython.fm/DevOps book. + +01:05:52 If you or your team needs to learn Python, we have over 270 hours of beginner and advanced courses on topics ranging from + +01:05:59 complete beginners to async code, Flask, Django, HTML, and even LLMs. + +01:06:05 Best of all, there's no subscription in sight. + +01:06:08 Browse the catalog at talkpython.fm. + +01:06:10 And if you're not already subscribed to the show on your favorite podcast player, + +01:06:14 what are you waiting for? + +01:06:15 Just search for Python in your podcast player. + +01:06:17 We should be right at the top. + +01:06:18 If you enjoy that geeky rap song, you can download the full track. + +01:06:21 The link is actually in your podcast blur show notes. + +01:06:24 This is your host, Michael Kennedy. + +01:06:26 Thank you so much for listening. + +01:06:27 I really appreciate it. + +01:06:28 I'll see you next time. + +01:06:41 I'm out. + diff --git a/transcripts/536-fly-inside-fastapi-cloud.vtt b/transcripts/536-fly-inside-fastapi-cloud.vtt new file mode 100644 index 0000000..1367c48 --- /dev/null +++ b/transcripts/536-fly-inside-fastapi-cloud.vtt @@ -0,0 +1,3842 @@ +WEBVTT + +00:00:00.020 --> 00:00:04.600 +You've built your FastAPI app. It's running great locally. Now you want to share it with the world. + +00:00:05.360 --> 00:00:12.480 +But then reality hits. Containers, load balancers, HTTPS certificates, cloud consoles with 200 + +00:00:12.680 --> 00:00:18.560 +options. What if deploying was just one command? That's exactly what Sebastian Ramirez and the + +00:00:18.660 --> 00:00:24.460 +FastAPI cloud team are building. On this episode, we sit down with Sebastian, Patrick Arminio, + +00:00:24.720 --> 00:00:29.980 +Savannah Ostrowski, and Jonathan Ewald to go inside FastAPI cloud, explore what it means + +00:00:30.000 --> 00:00:35.340 +to build a Pythonic cloud and dig into how this commercial venture is actually making FastAPI, + +00:00:35.380 --> 00:00:42.060 +the open source project, stronger than ever. This is Talk Python To Me, episode 536, recorded January + +00:00:42.440 --> 00:00:51.440 +13th, 2026. Talk Python To Me, yeah, we ready to roll. Upgrading the code, no fear of getting old. + +00:00:51.580 --> 00:00:59.960 +Async in the air, new frameworks in sight, geeky rap on deck. Quark crew, it's time to unite. We + +00:01:01.820 --> 00:01:06.460 +Welcome to Talk Python To Me, the number one Python podcast for developers and data scientists. + +00:01:06.980 --> 00:01:12.300 +This is your host, Michael Kennedy. I'm a PSF fellow who's been coding for over 25 years. + +00:01:12.940 --> 00:01:17.460 +Let's connect on social media. You'll find me and Talk Python on Mastodon, BlueSky, and X. + +00:01:17.710 --> 00:01:23.000 +The social links are all in your show notes. You can find over 10 years of past episodes at + +00:01:23.120 --> 00:01:27.400 +Talk Python.fm. And if you want to be part of the show, you can join our recording live streams. + +00:01:27.680 --> 00:01:31.580 +That's right, we live stream the raw uncut version of each episode on YouTube. + +00:01:32.180 --> 00:01:36.600 +Just visit talkpython.fm/youtube to see the schedule of upcoming events. + +00:01:36.860 --> 00:01:40.480 +Be sure to subscribe there and press the bell so you'll get notified anytime we're recording. + +00:01:41.480 --> 00:01:46.440 +This episode is brought to you by CommandBook, a native macOS app that I built that gives + +00:01:46.640 --> 00:01:48.680 +long-running terminal commands a permanent home. + +00:01:49.080 --> 00:01:51.080 +No more juggling six terminal tabs every morning. + +00:01:51.700 --> 00:01:56.340 +Carefully craft a command once, run it forever with auto-restart, URL detection, and a full CLI. + +00:01:56.780 --> 00:01:59.840 +Download it for free at talkpython.fm/command book app. + +00:02:00.520 --> 00:02:03.140 +And it's brought to you by the Talk Python in Production Book, + +00:02:03.700 --> 00:02:08.360 +an inside look at 10 years of the real-world DevOps behind the Talk Python sites and apps. + +00:02:08.640 --> 00:02:11.340 +Check it out at talkpython.fm/DevOps book. + +00:02:12.100 --> 00:02:14.220 +Before we dive in, I have something excellent to announce. + +00:02:14.440 --> 00:02:17.820 +A few episodes back, I told you about our new AI integrations, + +00:02:18.120 --> 00:02:21.880 +the Talk Python MCP servers, and our llms.txt file, + +00:02:22.300 --> 00:02:31.420 +so that AI tools can tap into our over 550 episodes and 7.5 million words of info around the Python community and history. + +00:02:32.080 --> 00:02:33.580 +Well, I'm building on that now. + +00:02:34.240 --> 00:02:37.120 +Talk Python now has an open source CLI tool. + +00:02:37.490 --> 00:02:41.960 +You can search episodes, transcripts, guests, and even our courses right from your terminal. + +00:02:42.260 --> 00:02:43.240 +No browser required. + +00:02:44.240 --> 00:02:49.040 +It's fast too, backed by our optimized APIs, taking about five milliseconds of response time, + +00:02:49.300 --> 00:02:51.140 +plus, you know, whatever the internet ping time is. + +00:02:51.680 --> 00:02:57.120 +It supports rich text for humans, JSON for programs, and markdown output for AIs. + +00:02:57.900 --> 00:03:04.540 +You just install it with uvtoolinstall talk-python-cli, and check it out on GitHub. + +00:03:04.920 --> 00:03:06.460 +The link is in the podcast player show notes. + +00:03:07.160 --> 00:03:09.560 +I also wrote up a blog post on the hows and whys of it. + +00:03:09.880 --> 00:03:14.100 +Check that out over on the Talk Python blog at, well, talkpython.fm/blog. + +00:03:14.780 --> 00:03:17.120 +The full link to the exact article is in the show notes. + +00:03:17.860 --> 00:03:18.680 +All right, let's jump in. + +00:03:20.100 --> 00:03:20.700 +Hello, everyone. + +00:03:21.220 --> 00:03:23.940 +Sebastian, Patrick, Savannah, and Jonathan. + +00:03:24.460 --> 00:03:25.580 +Awesome to have you all here. + +00:03:26.160 --> 00:03:27.460 +Excited to talk about FastAPI Cloud. + +00:03:28.120 --> 00:03:28.300 +Welcome. + +00:03:28.310 --> 00:03:28.460 +Yes. + +00:03:28.980 --> 00:03:29.620 +Thanks for having me. + +00:03:29.860 --> 00:03:30.100 +Thank you. + +00:03:30.240 --> 00:03:31.080 +We're also ahead, Mike. + +00:03:31.660 --> 00:03:32.640 +What a project. + +00:03:32.920 --> 00:03:34.420 +It's been going on for a while. + +00:03:34.880 --> 00:03:40.100 +I've heard stuff from Sebastian that maybe something was brewing and all these things, + +00:03:40.320 --> 00:03:42.780 +but not too long ago, you all announced it. + +00:03:43.020 --> 00:03:47.100 +And I heard that FastAPI, some people have been using it recently. + +00:03:47.500 --> 00:03:50.820 +Some of the surveys show that some people use it for websites. + +00:03:51.140 --> 00:03:51.560 +I'm not sure. + +00:03:51.860 --> 00:03:52.320 +Rumors. + +00:03:52.900 --> 00:03:53.180 +Yeah, yeah. + +00:03:53.380 --> 00:03:53.800 +Rumors. + +00:03:54.680 --> 00:03:55.080 +Rumors. + +00:03:55.440 --> 00:03:56.020 +Oh, my gosh. + +00:03:56.100 --> 00:03:58.060 +I mean, congratulations on that. + +00:03:58.100 --> 00:04:03.120 +But before we dive into FastAPI and FastAPI Cloud, let's just do a quick introduction. + +00:04:03.720 --> 00:04:04.320 +Who are you? + +00:04:04.380 --> 00:04:08.220 +We'll just go around the Brady Bunch squares of our live stream here and start with Sebastian. + +00:04:08.620 --> 00:04:09.680 +You've been on the show a few times. + +00:04:09.800 --> 00:04:14.380 +In fact, you've been on the show just recently for a really fun episode, Sebastian. + +00:04:14.760 --> 00:04:15.060 +Who are you? + +00:04:15.260 --> 00:04:16.519 +In real, that was super fun. + +00:04:17.019 --> 00:04:18.100 +So, hello, everyone. + +00:04:18.280 --> 00:04:20.299 +I'm Sebastian Ramirez, or Tiangulo. + +00:04:20.799 --> 00:04:22.300 +I created FastAPI. + +00:04:22.400 --> 00:04:26.200 +That is this Python framework for building web APIs and backend. + +00:04:26.940 --> 00:04:29.980 +In case you've been living in a hole and haven't done any Python for 10 years. + +00:04:30.500 --> 00:04:36.480 +You also are famous for really pointing out the ridiculousness of modern tech recruiting. + +00:04:36.800 --> 00:04:38.220 +You know what I'm talking about? + +00:04:38.580 --> 00:04:40.140 +Yeah, you know, like it's fun. + +00:04:40.280 --> 00:04:42.700 +This is probably the thing that I am known for. + +00:04:42.780 --> 00:04:47.700 +It's for writing a tweet saying, yeah, what was it? + +00:04:47.980 --> 00:04:52.460 +that I saw a job asking for five years of experience in FastAPI, + +00:04:52.920 --> 00:04:56.040 +and I only had 2.5 since I created the thing. + +00:04:56.140 --> 00:04:58.920 +So you didn't qualify for the FastAPI job? + +00:04:59.000 --> 00:05:00.460 +I didn't qualify for that, yeah. + +00:05:00.800 --> 00:05:03.260 +And then the funny thing is, you know, like people sometimes, + +00:05:03.940 --> 00:05:06.240 +even people in Python itself, and tell me like, + +00:05:06.400 --> 00:05:08.220 +oh, wait, like you're, and I say like, oh, yeah, + +00:05:08.270 --> 00:05:09.580 +I created this thing called FastAPI. + +00:05:09.700 --> 00:05:11.340 +Oh, wait, okay, so what is FastAPI? + +00:05:11.480 --> 00:05:14.440 +Oh, wait, you are the guy from the meme, + +00:05:14.900 --> 00:05:16.160 +the meme about FastAPI. + +00:05:16.160 --> 00:05:16.780 +Are you serious? + +00:05:17.440 --> 00:05:22.960 +Yeah, you know, like suddenly that is super important that I am the guy for the meme about FastAPI. + +00:05:23.260 --> 00:05:26.020 +Not the guy from FastAPI, the guy from the meme. + +00:05:26.740 --> 00:05:29.080 +Oh my gosh, I saw you on TikTok. It was amazing. + +00:05:29.840 --> 00:05:34.240 +It was a live achievement. I wrote a viral tweet. So yeah, nice to meet you all. + +00:05:34.360 --> 00:05:37.680 +You know what? Sometimes your moment in the sun is not the one you expected. + +00:05:37.920 --> 00:05:40.340 +No, congratulations on how good FastAPI is. + +00:05:40.340 --> 00:05:40.620 +On the tweet. + +00:05:41.820 --> 00:05:44.980 +Exactly. You really nailed it. Patrick, welcome to the show. + +00:05:45.160 --> 00:05:46.580 +Nice to be here. Yeah, I'm Patrick. + +00:05:47.040 --> 00:05:50.780 +I guess the main thing I'm kind of known for in the community is like this library called + +00:05:50.940 --> 00:05:54.820 +Storary, which is similar to FastAPI, but instead of REST is for GraphQL. + +00:05:55.130 --> 00:06:00.740 +Other than that, I help organize PyCon Italy and I used to also do EuroPython as well, + +00:06:00.940 --> 00:06:03.160 +but I stopped because of way too many things. + +00:06:03.550 --> 00:06:05.120 +Yeah, that's pretty much me. + +00:06:05.200 --> 00:06:06.780 +How do you see GraphQL these days? + +00:06:07.290 --> 00:06:08.220 +Is it still popular? + +00:06:08.500 --> 00:06:12.160 +I think it's mostly popular in the enterprises, unfortunately. + +00:06:12.800 --> 00:06:16.039 +I'm a bit, to be honest, I'm a bit annoyed about the companies that do tooling around + +00:06:16.060 --> 00:06:20.200 +GraphQL because I don't know, I feel like they're not really pushing it forward. They're just, + +00:06:20.380 --> 00:06:24.900 +I don't know, trying to work with enterprises and that's it. Or maybe people think to AI. + +00:06:25.140 --> 00:06:31.180 +Yeah. It feels a little bit like the soap, soap, wisdom, XML, modern version. Savannah. + +00:06:33.040 --> 00:06:33.180 +Yeah. + +00:06:34.740 --> 00:06:39.840 +You like tapping out of being an organizer for EuroPython is like, you know, the classic + +00:06:40.060 --> 00:06:44.180 +open source oversubscribed doing all the things very relatable. + +00:06:44.720 --> 00:06:44.880 +Yeah. + +00:06:45.030 --> 00:06:45.180 +Yeah. + +00:06:45.660 --> 00:06:46.280 +But yeah, I'm Savannah. + +00:06:46.760 --> 00:06:47.660 +What can I say? + +00:06:47.780 --> 00:06:52.360 +I am on the Python Steering Council for 2096, which is very exciting. + +00:06:52.640 --> 00:06:52.920 +Congratulations. + +00:06:53.340 --> 00:06:53.780 +Oh, thank you. + +00:06:53.980 --> 00:06:59.280 +I am also the release manager for the upcoming version of Python, Python 3.16. + +00:06:59.910 --> 00:07:03.800 +And so that'll kick off later this year, which is really cool and very exciting. + +00:07:04.320 --> 00:07:08.800 +I work on CPython stuff, the JIT, arg parse, basically whatever needs help. + +00:07:09.060 --> 00:07:09.880 +It's kind of where you'll find me. + +00:07:09.960 --> 00:07:10.160 +Awesome. + +00:07:10.760 --> 00:07:12.300 +Congratulations on the Steering Council. + +00:07:12.880 --> 00:07:15.000 +And yeah, that's a lot of cool stuff. + +00:07:15.540 --> 00:07:19.740 +Hopefully we don't get a Python 4.0 right after 3.16 + +00:07:20.540 --> 00:07:22.960 +because then your job will never end, is what I've learned. + +00:07:23.240 --> 00:07:23.560 +Yeah, yeah. + +00:07:23.920 --> 00:07:27.920 +Benjamin Peterson, Python 2.7 forever kind of situation. + +00:07:28.460 --> 00:07:28.860 +Yeah, yeah. + +00:07:29.070 --> 00:07:32.920 +I mean, release management is still, I mean, it's still quite a commitment. + +00:07:33.140 --> 00:07:36.900 +It's like seven-ish years when you think about all the staggered releases + +00:07:36.990 --> 00:07:38.620 +because you're a release manager 2.0 + +00:07:38.620 --> 00:07:40.700 +and then you have the five-year maintenance cycle. + +00:07:41.980 --> 00:07:45.800 +So yeah, it's Python forever is what I only said. + +00:07:45.960 --> 00:07:47.040 +Yeah, it's probably not a fad. + +00:07:47.120 --> 00:07:48.720 +It's probably going to stick around, this Python thing. + +00:07:50.400 --> 00:07:51.080 +No, that's awesome. + +00:07:51.360 --> 00:07:51.580 +Congratulations. + +00:07:51.940 --> 00:07:53.780 +Also, cool with arg parse. + +00:07:54.080 --> 00:08:04.680 +I feel like that's making a strong comeback now that we have these AI things that can just put stuff together for us instead of like, oh, I need to depend on this library and that library. + +00:08:04.700 --> 00:08:08.060 +Like, I just need to take a few arguments and have a little help text. + +00:08:08.120 --> 00:08:10.000 +And it's like, well, you've already got this built-in thing. + +00:08:10.120 --> 00:08:11.240 +Oh, who knew? + +00:08:11.420 --> 00:08:12.820 +You know, people are like, oh, I didn't even know. + +00:08:12.870 --> 00:08:15.800 +I thought I used typer or click or something, right? + +00:08:15.940 --> 00:08:18.320 +There's, you know, the typers and clicks of the world. + +00:08:18.430 --> 00:08:20.720 +But sometimes you just want the simplest thing. + +00:08:20.880 --> 00:08:22.740 +And ArgParse is pretty great at that. + +00:08:22.950 --> 00:08:27.900 +Although it has many quirks that are probably and most definitely unfixable at this point. + +00:08:28.220 --> 00:08:32.080 +Because bugs are features when you have things that have been around as long as Python. + +00:08:32.490 --> 00:08:35.000 +But yeah, no, I mean, AI loves to write Python. + +00:08:35.320 --> 00:08:39.240 +I think it's like the language used the most, AI-generated code. + +00:08:39.680 --> 00:08:42.039 +I'll just say we live in weird times, very weird times. + +00:08:42.539 --> 00:08:44.940 +I would love a precedented time at some time. + +00:08:45.440 --> 00:08:45.840 +Exactly. + +00:08:46.160 --> 00:08:46.220 +Yeah. + +00:08:46.320 --> 00:08:47.660 +Can we just get the boring times? + +00:08:48.000 --> 00:08:49.640 +No, nothing interesting, please. + +00:08:49.940 --> 00:08:55.460 +What I said about GraphQL may sound like a bit of a smash, but I didn't mean it in a negative, + +00:08:56.020 --> 00:08:56.940 +super negative way anyway. + +00:08:57.220 --> 00:09:02.500 +Like it used to be all the enterprises were all about soap and wisdom and like subscribing + +00:09:02.580 --> 00:09:03.060 +your tooling. + +00:09:03.460 --> 00:09:04.160 +Please don't write me. + +00:09:04.160 --> 00:09:05.780 +I'm not trying to bash on your technology. + +00:09:07.780 --> 00:09:08.140 +All right. + +00:09:08.800 --> 00:09:10.080 +Jonathan, also welcome. + +00:09:10.320 --> 00:09:10.440 +Hi. + +00:09:10.580 --> 00:09:14.080 +Yeah, I'm not nearly as famous as everyone else in this call. + +00:09:14.980 --> 00:09:20.420 +I'm more infamous internally at FastAPI Cloud, I would say, for a bunch of things. + +00:09:21.000 --> 00:09:23.860 +I've heard of emojis or something along the lines of line. + +00:09:23.920 --> 00:09:24.160 +One Mima way. + +00:09:24.300 --> 00:09:25.580 +You're just one Mima way. + +00:09:25.680 --> 00:09:26.820 +Just one Mima way. + +00:09:27.020 --> 00:09:27.800 +Yeah, that's true. + +00:09:28.400 --> 00:09:30.000 +We keep piling them up internally. + +00:09:30.560 --> 00:09:34.060 +But yeah, I used to work with Patrick together for years, also on Craftia. + +00:09:34.360 --> 00:09:35.280 +Same library as him. + +00:09:35.540 --> 00:09:36.340 +That's how I know him. + +00:09:36.620 --> 00:09:40.300 +And that's why I, well, made a weird sound when you said soap. + +00:09:41.320 --> 00:09:44.940 +I've been with FastAPI Cloud since EuroPattern, actually, the last one. + +00:09:45.220 --> 00:09:48.620 +I promised Sebastian I would implement server-send events in FastAPI, + +00:09:48.720 --> 00:09:54.220 +and I haven't started with it yet at all, but somehow I'm still here. + +00:09:54.290 --> 00:09:54.840 +So that's great. + +00:09:55.960 --> 00:10:00.420 +Well, actually, yeah, and it was actually like a sneak peek, I guess. + +00:10:00.540 --> 00:10:04.280 +We already started having a bunch of chats and discussing what we do. + +00:10:04.430 --> 00:10:05.980 +Should we do it here? Should we do it there? + +00:10:06.100 --> 00:10:06.660 +What should we do? + +00:10:07.140 --> 00:10:11.640 +So like, yeah, it's something that is coming to FastAPI probably soon-ish. + +00:10:11.800 --> 00:10:14.520 +Like there was a lot of things that needed to happen before that. + +00:10:14.700 --> 00:10:18.360 +Like Patrick is slightly smiley, like, oh no, this is pressure. + +00:10:19.740 --> 00:10:23.160 +There were some things that needed to happen in FastAPI, as you know, dropping support for + +00:10:23.420 --> 00:10:27.480 +Pydantic version one or things like that, that just made the internal code so complex. + +00:10:28.020 --> 00:10:32.820 +And now that it's over, we can actually work more on improving performance, adding features + +00:10:33.040 --> 00:10:33.760 +and things like that. + +00:10:34.220 --> 00:10:42.820 +I definitely want to dive into how FastAPI Cloud has sort of influenced the whole FastAPI side of things. + +00:10:43.480 --> 00:10:50.780 +But I made aware that there is, in fact, an entire website, an entire website dedicated to the meme. + +00:10:51.180 --> 00:10:54.680 +Yeah. And out in the audience, we get, hey, everyone, is that the guy from the meme? + +00:10:54.960 --> 00:10:56.980 +And meme is greater than Nobel Prize. + +00:10:57.480 --> 00:10:59.040 +So, you know what? + +00:10:59.460 --> 00:11:00.280 +It may be true. + +00:11:00.660 --> 00:11:01.220 +It may be true. + +00:11:01.240 --> 00:11:04.540 +I recognize the person saying, this is the guy from the Ming. + +00:11:04.720 --> 00:11:05.740 +He might be my husband. + +00:11:09.140 --> 00:11:09.580 +Incredible. + +00:11:11.500 --> 00:11:11.840 +Incredible. + +00:11:12.600 --> 00:11:16.100 +All right, well, let's start with FastAPI Cloud, + +00:11:16.920 --> 00:11:19.260 +and then we'll bring it back around to the FastAPI. + +00:11:19.380 --> 00:11:21.300 +Let's talk origin story. + +00:11:21.540 --> 00:11:23.700 +So what is this FastAPI Cloud? + +00:11:23.900 --> 00:11:24.240 +Nice. + +00:11:24.660 --> 00:11:27.600 +So if you were looking at the FastAPI Labs website, + +00:11:27.660 --> 00:11:29.600 +that doesn't really show that much. + +00:11:29.660 --> 00:11:33.680 +If you click on the join the waiting list that takes you to the website for FastAPI Cloud, + +00:11:34.200 --> 00:11:36.720 +there we can see this is what we are building. + +00:11:36.920 --> 00:11:38.120 +This is the thing that we are doing. + +00:11:39.120 --> 00:11:40.220 +It's actually super simple. + +00:11:40.400 --> 00:11:44.540 +The funny thing is that the pitch, the explanation of the product is so short. + +00:11:44.760 --> 00:11:45.520 +So it's one command. + +00:11:45.590 --> 00:11:46.600 +It's FastAPI Deploy. + +00:11:46.960 --> 00:11:51.400 +You have a FastAPI app, you just hit FastAPI Deploy, and then it's on the cloud. + +00:11:51.410 --> 00:11:52.520 +We take care of everything. + +00:11:52.590 --> 00:11:57.580 +We build a thing, deploy it, handle HTTPS, auto-scaling, all this stuff. + +00:11:57.920 --> 00:12:00.760 +and then you can just like focus on building the application, + +00:12:01.230 --> 00:12:01.580 +building apps. + +00:12:01.920 --> 00:12:03.760 +The funny thing is that it's super short to explain, + +00:12:03.880 --> 00:12:06.100 +but then building it is so cold, Blake. + +00:12:06.220 --> 00:12:07.020 +I'm glad it's so short. + +00:12:07.220 --> 00:12:08.000 +So thanks for being here. + +00:12:08.080 --> 00:12:08.960 +That was a great show, y'all. + +00:12:09.260 --> 00:12:10.040 +Well, no, just kidding. + +00:12:11.120 --> 00:12:14.960 +I think it's a little bit like Jupyter Notebooks + +00:12:15.050 --> 00:12:17.960 +in that sense that like you all are taking one for the team + +00:12:18.620 --> 00:12:21.600 +so that other people can have a simple experience. + +00:12:22.500 --> 00:12:25.100 +Whereas, you know, it's like those Jupyter folks, + +00:12:25.130 --> 00:12:27.620 +they write tons of TypeScript and do all sorts of things + +00:12:27.640 --> 00:12:30.380 +that nobody wants to necessarily do in the data science space + +00:12:31.040 --> 00:12:32.760 +so that you can just drag your widgets around. + +00:12:33.000 --> 00:12:33.360 +You know what I mean? + +00:12:33.500 --> 00:12:35.060 +I think that is a great analogy. + +00:12:35.360 --> 00:12:39.000 +I feel like the deployment space, it's a bit of a mixed bag. + +00:12:39.500 --> 00:12:44.540 +And I've been really frustrated to the point such that I wrote a book about it, + +00:12:45.020 --> 00:12:47.340 +that I think about an alternative, + +00:12:48.000 --> 00:12:51.240 +that I think over the last five plus years, + +00:12:51.580 --> 00:12:55.080 +it's just trended towards a little more complex, a little more complex. + +00:12:56.000 --> 00:12:57.460 +could we just add one of these things? + +00:12:57.680 --> 00:12:58.760 +And oh, now we got these three. + +00:12:58.860 --> 00:13:01.920 +We need one more thing to like make sure those things are doing, you know what I mean? + +00:13:02.500 --> 00:13:07.180 +And it's just like, wow, why are there 200 choices in my console to use this? + +00:13:07.440 --> 00:13:08.780 +Which is like kind of funny, right? + +00:13:08.920 --> 00:13:12.100 +Because I feel like a lot of these companies started with this, like, + +00:13:12.280 --> 00:13:15.300 +I don't want to understand all the ins and outs of all the infrastructure + +00:13:15.680 --> 00:13:17.400 +that comes with the cloud service provider. + +00:13:17.580 --> 00:13:19.840 +And that's really complicated to understand because I'm an app dev + +00:13:19.960 --> 00:13:22.380 +and I don't know anything about, you know, whatever, right? + +00:13:22.520 --> 00:13:25.400 +Now we're like, I don't know, kind of slowly accumulating. + +00:13:25.460 --> 00:13:30.480 +complexity but i think one of the cool things about what we're building and i like i've worked + +00:13:30.480 --> 00:13:35.800 +on cloud tooling before is like this is like just spoke for python developers and i think that's like + +00:13:35.940 --> 00:13:41.220 +quite like unique in that like we are really trying to like bring the bleeding edge and like + +00:13:41.240 --> 00:13:45.420 +all the new tooling that people are using and making sure that we play well with like uv and + +00:13:45.520 --> 00:13:52.180 +like i think like the there's a lot of thought and care put into that by the team that's a super good + +00:13:52.200 --> 00:13:56.880 +point. I mean, I remember Azure came out with like, here's your platform as a service. You just + +00:13:57.280 --> 00:14:01.640 +upload your web app and we'll just take it and go. And now that thing is so complicated along with + +00:14:01.860 --> 00:14:06.160 +many, many others, right? It's not just them. It's you've got AWS, you've got Vercel. There's + +00:14:06.240 --> 00:14:09.900 +lots of things we could point at for, wow, there's a lot of options here, you know? + +00:14:09.960 --> 00:14:13.920 +And then there are a lot of tools and like, you know, like many tools and many companies are also + +00:14:14.040 --> 00:14:18.379 +like doing a great job at many of the things that they are doing. But in many cases, it's just so + +00:14:18.400 --> 00:14:19.500 +complex, it's so complicated. + +00:14:19.740 --> 00:14:25.340 +You know, like I was, I have always been so, what, so adamant, I think is the word, to + +00:14:25.700 --> 00:14:27.580 +just teaching people how to use the tools. + +00:14:28.000 --> 00:14:33.540 +I think I have the most documentation about how to deploy things on your own than any other + +00:14:33.760 --> 00:14:33.920 +framework. + +00:14:34.020 --> 00:14:35.360 +I have so much information. + +00:14:35.460 --> 00:14:36.940 +MARK MANDEL: I hear that all the time from people. + +00:14:37.140 --> 00:14:41.100 +They say one of the reasons they chose FastAPIs is because how clear the documentation + +00:14:41.440 --> 00:14:41.760 +was, you know? + +00:14:41.760 --> 00:14:44.979 +FRANCESC CAMPOY: And then the thing is, you know, like just learning all those concepts + +00:14:45.000 --> 00:14:47.520 +and learning all the stuff that needs to be learned + +00:14:47.750 --> 00:14:48.740 +just to deploy something, + +00:14:48.810 --> 00:14:50.900 +and then you barely have the minimum. + +00:14:51.460 --> 00:14:53.480 +It's like, this is just too much. + +00:14:53.640 --> 00:14:54.740 +It's too much complacency. + +00:14:54.960 --> 00:14:58.340 +I think for me, I guess personally, + +00:14:58.520 --> 00:15:00.400 +my analogy is that FastAPI Cloud + +00:15:00.560 --> 00:15:03.280 +is the equivalent of what FastAPI is + +00:15:03.350 --> 00:15:05.800 +to building web APIs and backend. + +00:15:06.140 --> 00:15:08.340 +You could do the same with any other framework. + +00:15:08.390 --> 00:15:09.580 +You could validate data. + +00:15:09.650 --> 00:15:10.940 +You could generate open API. + +00:15:11.070 --> 00:15:12.280 +You could have automatic docs, + +00:15:12.640 --> 00:15:19.340 +But you will probably have to do a lot of the wiring yourself and making sure that it's actually correct and that it doesn't explode, all that stuff. + +00:15:19.710 --> 00:15:25.600 +That is, you know, like we are trying to do a lot of that work for the final users. + +00:15:25.800 --> 00:15:26.840 +Yeah, and I think it's great. + +00:15:27.260 --> 00:15:36.500 +I think it's really nice to just provide this on-ramp because, as you said at the opening, when I asked, you know, what the origin story is just FastAPI deploy. + +00:15:37.050 --> 00:15:38.300 +That solves so many stories. + +00:15:38.580 --> 00:15:42.920 +But I'm sure behind the scenes, what happens is just about as simple as that. + +00:15:44.280 --> 00:15:44.940 +Oh my gosh. + +00:15:45.060 --> 00:15:45.720 +About that. + +00:15:48.460 --> 00:15:51.820 +Some of us don't even get to write Python anymore to make all of this happen. + +00:15:52.300 --> 00:15:54.480 +So speaking about taking one for the team. + +00:15:56.040 --> 00:15:57.620 +Yeah, that is taking one for a team, right? + +00:15:57.740 --> 00:15:58.200 +It is. + +00:16:00.220 --> 00:16:02.660 +This portion of Talk Python To Me is brought to you by us. + +00:16:02.980 --> 00:16:07.520 +I'm thrilled to announce a brand new app built for developers created by yours truly. + +00:16:07.980 --> 00:16:09.140 +It's called Command Book. + +00:16:09.930 --> 00:16:11.200 +You know that thing you do every morning? + +00:16:11.880 --> 00:16:14.460 +Open up six terminal tabs, CD into this directory, + +00:16:14.980 --> 00:16:16.020 +activate that virtual environment, + +00:16:16.550 --> 00:16:17.960 +run the server with --reload. + +00:16:18.260 --> 00:16:20.960 +Now, CD somewhere else, start the background worker, + +00:16:21.360 --> 00:16:23.940 +another tab for Docker, another one to tail production logs. + +00:16:24.400 --> 00:16:27.120 +Every tab just says Python, Python, Python, Docker tail. + +00:16:28.220 --> 00:16:29.160 +And you're clicking through them going, + +00:16:29.620 --> 00:16:30.840 +which Python was that again? + +00:16:31.260 --> 00:16:32.040 +Where my app is running? + +00:16:32.620 --> 00:16:35.560 +Then sometime later, your dev server silently dies + +00:16:35.750 --> 00:16:36.879 +because it tried to reload + +00:16:36.900 --> 00:16:38.420 +while you're in the middle of a code edit, + +00:16:39.020 --> 00:16:41.580 +unmatched brace, a half-written import, or something. + +00:16:42.320 --> 00:16:43.340 +Now you're hunting through tabs + +00:16:43.360 --> 00:16:45.800 +to figure out which process crashed and how to restart it. + +00:16:46.280 --> 00:16:47.260 +My app, CommandBook, + +00:16:47.500 --> 00:16:50.820 +gives all of these long-running commands a permanent home. + +00:16:51.380 --> 00:16:52.560 +You save a command once, + +00:16:53.000 --> 00:16:54.180 +the working directory, the environment, + +00:16:54.500 --> 00:16:55.640 +three commands like git pull, + +00:16:55.980 --> 00:16:57.640 +and from then on, you just click run. + +00:16:58.180 --> 00:16:59.540 +You can even group commands together + +00:16:59.840 --> 00:17:01.640 +to start and stop everything for a project + +00:17:01.880 --> 00:17:02.520 +with a single click. + +00:17:02.960 --> 00:17:05.040 +It also has what I call honey badger mode, + +00:17:05.260 --> 00:17:06.439 +auto restart on crash. + +00:17:07.220 --> 00:17:09.640 +so when your dev server goes down mid-reload, + +00:17:10.199 --> 00:17:12.079 +Command Book just brings it right back up + +00:17:12.360 --> 00:17:14.400 +and does so over and over until the code is fixed. + +00:17:15.020 --> 00:17:16.819 +It also detects URLs from your output + +00:17:17.079 --> 00:17:19.400 +so you're never scrolling through thousands of lines of logs + +00:17:19.600 --> 00:17:21.560 +just to figure out how to reopen your web app. + +00:17:22.079 --> 00:17:23.780 +And it shows you uptime, memory usage, + +00:17:24.120 --> 00:17:25.959 +and all sorts of cool things about your process. + +00:17:26.660 --> 00:17:28.640 +The whole thing is a native macOS app. + +00:17:28.780 --> 00:17:31.200 +No Electron, no Chromium, just 21 megs. + +00:17:31.640 --> 00:17:32.960 +And it comes with a full CLI + +00:17:33.180 --> 00:17:35.040 +so anything you've configured in the UI, + +00:17:35.420 --> 00:17:37.900 +you can fire off from your terminal with just a single command. + +00:17:38.440 --> 00:17:41.580 +Right now, it's macOS only, but if there's enough interest, + +00:17:41.840 --> 00:17:43.840 +I'll build a Windows version too, so let me know. + +00:17:44.660 --> 00:17:48.580 +Please check it out at talkpython.fm/commandbook app. + +00:17:49.140 --> 00:17:51.640 +Download it for free, level up your developer workflow. + +00:17:52.080 --> 00:17:53.720 +The link is in your podcast player's show notes. + +00:17:54.320 --> 00:17:56.380 +That's talkpython.fm/commandbook. + +00:17:56.580 --> 00:17:58.500 +I really hope you enjoy this new app that I built. + +00:18:00.320 --> 00:18:02.540 +Let's save the internals for a little bit later. + +00:18:02.940 --> 00:18:04.440 +Maybe what we could do right now, + +00:18:04.540 --> 00:18:12.300 +Maybe we could do a bit of a walkthrough of just kind of what it's like to set up an app from scratch, right? + +00:18:12.480 --> 00:18:12.700 +Nice. + +00:18:12.900 --> 00:18:20.220 +I see that uv is here, which is, I've been certainly an advocate for uv in all sorts of deployment, + +00:18:20.260 --> 00:18:25.380 +but especially when you have like repeated build type of scenarios for like Docker, + +00:18:26.060 --> 00:18:31.680 +Docker Compose or Kubernetes or whatever, uv makes that stuff so much faster and so on. + +00:18:31.780 --> 00:18:37.140 +So who would like to be my guide that just kind of talks us through what it means to set up a new project here? + +00:18:37.300 --> 00:18:41.500 +I mean, there is like this really nice command that Savannah built, just FastAPI new, + +00:18:41.920 --> 00:18:44.960 +which I think is something like, I don't know, like super helpful. + +00:18:45.160 --> 00:18:46.660 +What does FastAPI new do? + +00:18:46.880 --> 00:18:51.540 +Like, is that kind of a cookie cutter-esque experience or what is it? + +00:18:51.600 --> 00:18:52.080 +Yes, exactly. + +00:18:52.160 --> 00:18:56.960 +At the moment, Onesco holds a super basic FastAPI application using uv. + +00:18:57.360 --> 00:19:00.080 +It also installs dependencies, creates a folder, everything that you need. + +00:19:00.440 --> 00:19:03.240 +In future, I think we're going to plan support for templates + +00:19:03.500 --> 00:19:05.940 +so you can build multiple kind of things as well. + +00:19:06.210 --> 00:19:08.840 +But for now, it's just basically just uv FastAPI new, + +00:19:09.220 --> 00:19:10.640 +sorry, uvx FastAPI new, + +00:19:10.910 --> 00:19:12.540 +and then that scaffolds the project for you. + +00:19:12.900 --> 00:19:14.680 +I don't know if you want to try it live or... + +00:19:14.700 --> 00:19:15.440 +No, go ahead. + +00:19:15.580 --> 00:19:17.980 +Just, I would think it might disrupt you. + +00:19:18.030 --> 00:19:19.080 +Just let's talk us through it. + +00:19:19.240 --> 00:19:19.920 +It could work. + +00:19:20.240 --> 00:19:21.740 +I'm just going to put that out there. + +00:19:21.880 --> 00:19:23.000 +I'll tell you the most insane, + +00:19:23.330 --> 00:19:25.320 +like let's do that live on the podcast experience. + +00:19:25.520 --> 00:19:28.160 +I'm pretty sure, yeah, this is definitely the most insane. + +00:19:28.620 --> 00:19:36.320 +I had Matthew Rocklin on from Coiled, and those guys are all about like, hey, we're going to scale up like a bunch of available servers for you, right? + +00:19:36.700 --> 00:19:37.940 +So that you can do your data science. + +00:19:38.020 --> 00:19:41.000 +Like I want to do some ML thing, and it needs 500 servers. + +00:19:41.600 --> 00:19:45.520 +So during the podcast, he's, oh, let me just spin up 2,000 EC2 instances. + +00:19:45.800 --> 00:19:46.060 +Hold on. + +00:19:47.020 --> 00:19:49.040 +And then we ran some code on it during the show. + +00:19:49.100 --> 00:19:50.200 +And he's like, oh, let's try that on ARM. + +00:19:50.380 --> 00:19:53.040 +And then spin up another 2,000 on ARM Linux machines. + +00:19:53.120 --> 00:19:54.180 +I'm like, okay, that's nuts. + +00:19:55.440 --> 00:19:55.840 +But let's just. + +00:19:56.420 --> 00:19:57.180 +That's a lot of power. + +00:19:57.260 --> 00:20:01.960 +So I was impressed, but Patrick, sorry, I kind of did realty there. + +00:20:02.500 --> 00:20:02.980 +Let's talk through it. + +00:20:03.020 --> 00:20:08.260 +Yeah, so you do uvx FastAPI app, FastAPI new, then you specify the name of the application. + +00:20:08.740 --> 00:20:10.080 +And that's almost there. + +00:20:10.080 --> 00:20:13.240 +You just need one more command to deploy, which is FastAPI deploy. + +00:20:13.640 --> 00:20:17.660 +The first time it's going to ask you to log in or join the waiting list if you haven't been invited yet. + +00:20:17.940 --> 00:20:18.940 +It's still in beta. + +00:20:19.500 --> 00:20:20.900 +And then you follow the steps. + +00:20:21.200 --> 00:20:24.640 +So like FastAPI deploy, log in, decide the team. + +00:20:25.040 --> 00:20:27.820 +If you have multiple teams, design the application name, + +00:20:28.220 --> 00:20:31.020 +and then you wait a few seconds and the application is going to be live. + +00:20:31.100 --> 00:20:36.220 +And just to be clear, FastAPI new is not required if you already have a FastAPI app. + +00:20:36.520 --> 00:20:39.780 +If you've already written your own code and you have your application, + +00:20:40.120 --> 00:20:42.940 +you can just go right into logging in and deploying. + +00:20:43.440 --> 00:20:46.120 +This is just so that if you're starting something new, + +00:20:46.500 --> 00:20:49.900 +you don't have to do any thinking about all the right things that need to be there. + +00:20:50.180 --> 00:20:52.160 +So this is more of a greenfield application. + +00:20:52.880 --> 00:20:53.960 +I'm bootstrapping a project. + +00:20:54.240 --> 00:20:56.660 +Right, right, because you want to have the best structure. + +00:20:57.560 --> 00:20:58.980 +Now, it uses uv. + +00:20:59.420 --> 00:21:00.240 +It is nodes required. + +00:21:00.420 --> 00:21:03.900 +Yeah, I was going to say, do I have to use the uv project management type of thing? + +00:21:04.190 --> 00:21:08.100 +Do I have to use the uv.lock files and uv add uv sync? + +00:21:08.360 --> 00:21:09.480 +Can I do requirements.txt? + +00:21:09.640 --> 00:21:10.380 +What's the story there? + +00:21:10.500 --> 00:21:13.320 +Yes, so we support uv with uv lock. + +00:21:13.390 --> 00:21:16.860 +We also support the, forget the name, the other, the PyLock file. + +00:21:17.200 --> 00:21:19.580 +And we also support plain requirements.txt. + +00:21:19.980 --> 00:21:22.200 +And maybe something else, I don't know, Jonathan, can you? + +00:21:22.700 --> 00:21:23.720 +PyLock's pretty new, right? + +00:21:23.900 --> 00:21:26.520 +I think Brett Cannon just got that out pretty recently, right? + +00:21:26.820 --> 00:21:27.680 +Brett was pretty excited. + +00:21:27.980 --> 00:21:28.360 +I know. + +00:21:29.240 --> 00:21:29.960 +Implemented that. + +00:21:30.240 --> 00:21:30.660 +Oh, was he? + +00:21:30.850 --> 00:21:31.620 +Okay, I'm sure he was. + +00:21:31.800 --> 00:21:32.160 +That's awesome. + +00:21:32.260 --> 00:21:33.740 +He put years of work into that. + +00:21:33.860 --> 00:21:38.100 +And he also said that one of the motivations was also like, you know, like cloud providers. + +00:21:38.560 --> 00:21:39.200 +So it's like, yes. + +00:21:40.600 --> 00:21:49.220 +The other thing is like, you know, if you use other different package managers, if they use the standard PyProject Autonomous format, that will also be supported. + +00:21:49.500 --> 00:21:56.300 +That means that, you know, like if you use PDM or if you use poetry with one of the recent versions, like that will work. + +00:21:56.600 --> 00:22:03.100 +If you use a very old version of poetry or like you use some other strange package manager or something, that will probably be problematic. + +00:22:03.500 --> 00:22:08.440 +But for like most of the use cases that use the standard package formats, it will just work. + +00:22:08.640 --> 00:22:14.080 +And if you use uv, then like you're going to have the best experience because we are fans of uv and Astro. + +00:22:14.220 --> 00:22:19.380 +They've definitely put a dent in the way that sort of Python gets started and making that a lot easier. + +00:22:19.490 --> 00:22:20.500 +So it totally makes sense. + +00:22:21.310 --> 00:22:28.920 +And also, I noticed, speaking of uv, that there's, at least in the recommended way, or the way in the docs, let's say, + +00:22:29.580 --> 00:22:32.240 +it doesn't say, here's how you install FastAPI. + +00:22:32.650 --> 00:22:41.660 +You just, here's how you run FastAPI-new, leveraging uv, which then will silently install and manage. + +00:22:42.120 --> 00:22:43.260 +All right, that's pretty neat. + +00:22:43.420 --> 00:22:46.960 +That helps you guys tell a simpler story, right? + +00:22:47.350 --> 00:22:51.380 +Instead of, here's how you create the virtual environment to install our thing and so on, you know? + +00:22:51.380 --> 00:22:56.340 +The idea is to make it like, as I was saying, just super simple for people just to start from scratch. + +00:22:56.680 --> 00:23:00.760 +Like no idea how to create an app, how to start, how to create an environment. + +00:23:00.890 --> 00:23:03.340 +It's just you run this command and you're off to go. + +00:23:03.760 --> 00:23:06.200 +Off to the races, I'm missing sands. + +00:23:06.820 --> 00:23:09.460 +Anyway, that's what Colombians do. + +00:23:09.920 --> 00:23:17.880 +But then if you already have an app, you have like, you know, like anything with FastAPI standard installed, then like that also just works. + +00:23:18.010 --> 00:23:21.000 +And Savannah, you pointed out that it doesn't have to be a new project. + +00:23:21.620 --> 00:23:23.860 +If you want to start from an existing one, that's totally fine. + +00:23:24.300 --> 00:23:28.820 +But what do I got to do if I'm starting from, if I'm migrating an existing one? + +00:23:29.040 --> 00:23:30.320 +Like how easy or hard is this? + +00:23:30.580 --> 00:23:36.860 +I have some like legacy project demo apps I've built at other companies I've worked with that have used FastAPI. + +00:23:37.120 --> 00:23:44.460 +And I literally just ran like FastAPI login and then FastAPI deploy and it just worked, which felt really magical. + +00:23:44.740 --> 00:23:58.860 +Right. Like I think that's like, I don't know, like having worked on cloud products for quite a while, like I think one of the biggest gaps is like the just I don't know, like the disparity between like my local dev environment and what is actually like lives up in the cloud somewhere. + +00:23:59.360 --> 00:24:08.160 +And so being able to just run one command and having the project as it exists on my machine go and work somewhere without having to think about like the infrastructure. + +00:24:08.340 --> 00:24:14.920 +And of course, like, you know, we want to be like amenable to folks who do want a little bit, you know, like higher touch. + +00:24:15.480 --> 00:24:19.280 +But we also want to work for people who are like learning FastAPI and Python, right? + +00:24:19.400 --> 00:24:22.240 +Like educators and people that are teaching Python. + +00:24:22.420 --> 00:24:26.600 +I think this is like something that you've had some interest in as well from those folks. + +00:24:26.720 --> 00:24:44.200 +Yeah, I was just listening to the Teaching Python podcast books just the other day and thinking, you know, like this, when I look at this, I know this is not necessarily your focus, but certainly people who are trying to teach a class, be it college class or high school class or whatever. + +00:24:44.840 --> 00:24:49.020 +And if you build anything on the web, the next question is, this is cool. + +00:24:49.340 --> 00:24:50.420 +How do I share it with people? + +00:24:50.620 --> 00:24:51.800 +And then they're like, oh, no. + +00:24:52.420 --> 00:24:53.500 +Oh, no. + +00:24:53.720 --> 00:24:54.080 +Hold on. + +00:24:54.280 --> 00:24:55.560 +Like coding boot camps, right? + +00:24:55.640 --> 00:25:01.780 +Like if you're teaching someone how to write Python or how to build an API with FastAPI, + +00:25:02.110 --> 00:25:06.480 +like actually setting up the environment for them to deploy is not part of it, right? + +00:25:06.720 --> 00:25:08.660 +Like that's not actually part of the curriculum. + +00:25:08.840 --> 00:25:12.880 +It's like this peripheral thing that ends up eating up a bunch of the educator's time or + +00:25:12.890 --> 00:25:16.680 +the student's time trying to understand both like how to write code and then also understand + +00:25:16.840 --> 00:25:17.240 +cloud stuff. + +00:25:17.520 --> 00:25:20.380 +And that's like a lot to ask people when they're fresh up the gate. + +00:25:20.520 --> 00:25:23.720 +I feel the same way about like tutorials and stuff at conferences. + +00:25:24.200 --> 00:25:25.240 +Yeah, totally. + +00:25:25.780 --> 00:25:29.820 +Yeah. Or training sessions. If you're doing like corporate training or like, they're all like, + +00:25:29.940 --> 00:25:36.120 +Oh, well, let's get everybody's machine working. There goes an hour and whatever. But yeah, + +00:25:36.140 --> 00:25:40.520 +if you can just say, look, I think when you're either, when you're trying to learn something, + +00:25:40.960 --> 00:25:45.860 +you'll be it through school or on your own or through these like more structured ways, + +00:25:46.180 --> 00:25:50.760 +like bootcamps and training and so on. I think if it's not the main purpose, + +00:25:51.380 --> 00:25:56.200 +I feel so often there's like, we're going to do 20 steps for four hours before you get any sort of + +00:25:56.520 --> 00:26:01.080 +reward of what you've done. And if you can go, okay, do you have it running? Okay, now you run + +00:26:01.180 --> 00:26:04.880 +this command. Look, now it's on the internet. Like, oh, wait, awesome. I got an app on the internet. + +00:26:05.140 --> 00:26:09.780 +Everybody look at me. You know what I mean? And I think shortening that cycle to where people can + +00:26:10.000 --> 00:26:14.780 +have that aha moment. And then later they can dive into like, well, how is it really working? And what + +00:26:14.780 --> 00:26:21.340 +do we really need to understand? But that quick iteration cycle, especially in the early parts of + +00:26:21.360 --> 00:26:24.980 +in new tech. It's really important. But also, you know, like down the line as well, I think, + +00:26:25.280 --> 00:26:29.400 +like, I don't know, there are so many things that I have been wanting to build and I don't, + +00:26:29.780 --> 00:26:34.320 +but I didn't because it was just so complex to deploy stuff. You know, like knowing, + +00:26:34.680 --> 00:26:39.560 +knowing how to do the whole thing, how to set up the clusters, the machines, install the Linux + +00:26:40.299 --> 00:26:44.060 +systems, deploy the cluster, whatever, like all that stuff, deploy the things, + +00:26:44.360 --> 00:26:49.940 +handling load balancers and HTTPS. I'm like, you know, like I know how to do that. I built one of + +00:26:49.960 --> 00:26:54.400 +the most popular websites teaching how to use Docker to ARM, which was like the contender + +00:26:54.620 --> 00:26:56.160 +before Kubernetes won everything. + +00:26:56.580 --> 00:26:56.960 +I like it. + +00:26:57.380 --> 00:27:02.520 +But still, it's just so complicated, like doing all those steps that are like, yeah, no, I'll + +00:27:02.660 --> 00:27:03.380 +just not do it. + +00:27:04.000 --> 00:27:04.780 +Like some other day. + +00:27:05.360 --> 00:27:09.500 +Now I can just like play around and do random stuff and just like deploy when it just works. + +00:27:09.860 --> 00:27:11.300 +It is, I really like that. + +00:27:11.300 --> 00:27:16.220 +I guess like coming back to that, like taking one for the team point earlier, like I feel + +00:27:16.240 --> 00:27:20.860 +like building Python tooling. It's kind of like taking one for the team sometimes because you have + +00:27:20.860 --> 00:27:25.920 +these folks that are like, you know, brand new to Python. Like Python is an extremely approachable + +00:27:26.000 --> 00:27:30.160 +language for people who are new to writing code. But then, you know, we also want to make FastAPI + +00:27:30.420 --> 00:27:34.760 +cloud work for someone that's building like an enterprise grade application, right? And so like, + +00:27:35.240 --> 00:27:41.159 +like pretty wide spectrum of folks with like a million different use cases and different types + +00:27:41.180 --> 00:27:45.760 +of applications they want to deploy with different constraints and like security stuff. + +00:27:46.180 --> 00:27:49.540 +And like, so yeah, I think, I don't know, maybe that's just like Python tooling. + +00:27:50.060 --> 00:27:53.600 +It's a lot of work, I guess, to like build something that works for the masses. + +00:27:54.000 --> 00:27:57.000 +Yeah, well, it's certainly tough to make something that feels simple, + +00:27:57.520 --> 00:28:01.260 +but it's not overly simplistic, you know, that can actually solve the problems. + +00:28:01.540 --> 00:28:04.200 +Has the right knobs for the right users too, right? + +00:28:04.300 --> 00:28:07.920 +I would argue we're not only trying to do it simple and easy. + +00:28:08.220 --> 00:28:12.440 +I feel like we're choosing a particular flavor of simple, which is... + +00:28:13.400 --> 00:28:14.880 +We have this discussion a few times. + +00:28:15.080 --> 00:28:18.620 +It's like, if you make a cloud, how do we make it feel Pythonic? + +00:28:18.760 --> 00:28:20.480 +What does that mean in a cloud setting? + +00:28:20.900 --> 00:28:25.340 +We talk about Pythonic libraries, Pythonic coding style in the community a lot. + +00:28:25.700 --> 00:28:29.620 +And now we kind of try to transfer that flavor, that feeling to the cloud + +00:28:29.700 --> 00:28:33.720 +and make everything around that feel just like we want our libraries to feel. + +00:28:34.020 --> 00:28:36.960 +So you feel at home as a Python developer and it just feels right. + +00:28:37.280 --> 00:28:40.160 +So that's extra step on top of making it simple. + +00:28:40.560 --> 00:28:41.560 +And we discuss that a lot. + +00:28:41.820 --> 00:28:42.740 +That's how I feel about it. + +00:28:42.770 --> 00:28:43.160 +I love it. + +00:28:43.170 --> 00:28:46.860 +I think it's one of the coolest things about this team. + +00:28:47.440 --> 00:28:50.280 +Like, you know, like people are being able to hear a few of us. + +00:28:50.360 --> 00:28:55.040 +There's like, there are like a bunch of others, but like that each one of us is so passionate + +00:28:55.230 --> 00:28:57.320 +about the things that we are working on. + +00:28:57.760 --> 00:29:01.780 +So like, you know, like each one of us is trying to make the best out of the things that + +00:29:01.960 --> 00:29:02.540 +we are building. + +00:29:02.860 --> 00:29:07.040 +And then like, we are so passionate about the thing that we care about and that we are building. + +00:29:07.160 --> 00:29:10.740 +that I think that ends up being an amazing result. + +00:29:11.000 --> 00:29:12.360 +For example, the CLI. + +00:29:12.660 --> 00:29:16.960 +We wanted to have some specific, you know, like behavior, + +00:29:17.140 --> 00:29:18.140 +some look and feel. + +00:29:18.220 --> 00:29:22.200 +And like we wanted to be able to have like the best kind of CLIs. + +00:29:22.320 --> 00:29:25.700 +So Patrick went ahead and built this bunch of tooling + +00:29:25.840 --> 00:29:27.380 +that we needed to be able to have it + +00:29:27.460 --> 00:29:28.720 +and like made it open source and everything. + +00:29:29.160 --> 00:29:32.780 +So we could have this great experience when working with CLIs. + +00:29:33.140 --> 00:29:35.859 +Jonathan recently was doing so much stuff + +00:29:35.880 --> 00:29:38.580 +about the something that caches and handling security, + +00:29:38.940 --> 00:29:40.680 +making sure that everything was super secure, + +00:29:41.060 --> 00:29:42.420 +super fast, super snappy. + +00:29:42.800 --> 00:29:46.220 +You know, like Alejandra is super careful about all the UI. + +00:29:46.740 --> 00:29:48.820 +Martin is super careful about all the infra. + +00:29:49.200 --> 00:29:52.940 +You know, it's like this passionate mess, + +00:29:53.320 --> 00:29:54.420 +which is a word I just made up. + +00:29:57.340 --> 00:29:58.600 +This, Alejandra goes and says, + +00:29:58.640 --> 00:30:01.220 +like, this thing doesn't have the proper margins. + +00:30:01.320 --> 00:30:02.920 +We need to increase this a little bit. + +00:30:02.980 --> 00:30:04.340 +I don't like it. + +00:30:04.360 --> 00:30:05.360 +She just goes and fixes it. + +00:30:05.600 --> 00:30:06.100 +The same with Marvin. + +00:30:06.190 --> 00:30:09.340 +He says, like, we need to have, like, this sort of thing in infrastructure. + +00:30:09.550 --> 00:30:12.040 +And, like, just comes and tells me, hey, we are doing this. + +00:30:12.140 --> 00:30:12.800 +And he's like, yes, sir. + +00:30:13.340 --> 00:30:16.260 +Like this with the OpenVPN, like, Unix, for example, + +00:30:16.350 --> 00:30:18.100 +that is mainly focused on the open source, + +00:30:18.320 --> 00:30:21.960 +is constantly looking at all the discussions, PRs, conversations, + +00:30:22.340 --> 00:30:24.840 +making sure that everything that we do, that doesn't. + +00:30:24.840 --> 00:30:26.300 +So why, you know, like, there have been, like, + +00:30:26.810 --> 00:30:31.159 +recently way more releases of FastAPI friends of the open source projects + +00:30:31.620 --> 00:30:33.520 +and very fast book fixes, + +00:30:34.150 --> 00:30:37.060 +very fast responses to handle everything for the community. + +00:30:37.600 --> 00:30:40.360 +Now we actually have people that is paying attention constantly + +00:30:40.720 --> 00:30:41.460 +to what is happening, + +00:30:42.320 --> 00:30:43.360 +what are the things that we have to do, + +00:30:43.700 --> 00:30:46.620 +and that really care about that part as well. + +00:30:46.650 --> 00:30:50.220 +So I think this extreme care about what we are doing. + +00:30:50.440 --> 00:30:52.180 +You know, like Savannah is making Python. + +00:30:53.060 --> 00:30:56.860 +This detail that each one of us cares so, so much + +00:30:57.100 --> 00:30:58.500 +about each one of the things that we build, + +00:30:59.540 --> 00:31:01.620 +making sure that the product is actually amazing. + +00:31:01.840 --> 00:31:03.140 +It's as good as it can be, + +00:31:03.280 --> 00:31:05.500 +and we can all feel at home when... + +00:31:06.940 --> 00:31:08.920 +I get so excited for talking about it + +00:31:08.980 --> 00:31:12.140 +because I really enjoy the end result of the product + +00:31:12.220 --> 00:31:13.280 +and of being able to use it. + +00:31:13.820 --> 00:31:15.020 +I would use it in the end. + +00:31:15.140 --> 00:31:16.420 +I would use to work with it. + +00:31:16.800 --> 00:31:17.560 +It's super simple. + +00:31:17.660 --> 00:31:18.300 +Yeah, that's awesome. + +00:31:18.480 --> 00:31:19.680 +Hey, let me adjust your mic real quick. + +00:31:20.140 --> 00:31:21.840 +I think it was like ducking, + +00:31:22.300 --> 00:31:23.080 +ducking out a little bit. + +00:31:23.080 --> 00:31:24.000 +We just went through a lot, + +00:31:24.140 --> 00:31:26.399 +a lot of content and a lot of sweating + +00:31:26.420 --> 00:31:31.380 +because your microphone went through like six different stages of weirdness. + +00:31:31.720 --> 00:31:39.180 +I think that really leads to like something I wanted to talk about is just what impact has this had on FastAPI? + +00:31:39.180 --> 00:31:44.640 +And before you jump in and answer that question, everyone, there's especially I think with Astral, + +00:31:44.820 --> 00:31:49.920 +but with because they've had so much success, there's been an undercurrent of concern of like, + +00:31:49.920 --> 00:31:53.200 +oh, my gosh, commercialism is getting into our open source. + +00:31:53.480 --> 00:31:56.440 +And what if it pollutes it and causes these negative aspects? + +00:31:57.560 --> 00:32:02.500 +But just hearing all of the energy around FastAPI with so many people, + +00:32:03.140 --> 00:32:05.580 +because of FastAPI Cloud, that's super neat. + +00:32:05.770 --> 00:32:07.260 +So I wanted to throw out to you all, + +00:32:07.660 --> 00:32:12.100 +how has this building FastAPI Cloud and the existence of FastAPI Cloud + +00:32:12.300 --> 00:32:14.840 +been giving back to FastAPI, I guess? + +00:32:14.980 --> 00:32:17.560 +I'm waiting to see if someone will speak FastAPI. + +00:32:18.760 --> 00:32:20.580 +I'm always the one that is speaking the most. + +00:32:22.400 --> 00:32:23.920 +I mean, it might be your project. + +00:32:24.220 --> 00:32:25.680 +Like, you may have started the project. + +00:32:25.900 --> 00:32:26.600 +Yeah, maybe so. + +00:32:27.020 --> 00:32:31.220 +Like, last year, I had, like, a few keynotes in some picons in different places. + +00:32:31.320 --> 00:32:41.660 +And, like, one of the key points that I wanted to bring was this idea that I'm trying to show that, in many cases, people worry about the boss factor. + +00:32:41.960 --> 00:32:43.580 +And the boss factor is just this idea. + +00:32:43.840 --> 00:32:45.180 +Yes, yes, I've heard this, yes. + +00:32:45.320 --> 00:32:51.320 +Yeah, you know, like, the boss factor is the idea that, oh, what happens if, like, there's one person doing this work? + +00:32:51.420 --> 00:32:54.180 +What happens if a boss runs over this person? + +00:32:54.860 --> 00:32:58.260 +And there's so much worry about this boss factor. + +00:32:58.420 --> 00:33:01.960 +It's sort of a morbid analogy, but I understand, right? + +00:33:02.040 --> 00:33:06.980 +Like, what will happen to the open source project if the maintainer vanishes for some reason, right? + +00:33:07.140 --> 00:33:07.460 +Exactly. + +00:33:08.080 --> 00:33:11.160 +But, you know, like, it also applies to projects and to many other different things. + +00:33:11.600 --> 00:33:18.760 +But what I think is that it's a disproportionate amount of attention to this detail of the boss factor. + +00:33:19.180 --> 00:33:30.300 +And I think every time people talk about the boss factor, you know, like one of my points in what I was trying to say in these talks was I would like people to think about the boss ticket factor. + +00:33:30.720 --> 00:33:32.140 +Who is paying for those tickets? + +00:33:32.400 --> 00:33:34.140 +It doesn't matter how big is the team. + +00:33:34.440 --> 00:33:37.960 +You know, like you have seen Google, Amazon, Meta, all the big ones. + +00:33:38.240 --> 00:33:39.960 +They don't have a small boost factor. + +00:33:40.020 --> 00:33:45.100 +They have a lot of people in their payroll and still they finish products. + +00:33:45.340 --> 00:33:46.340 +They just cancel them. + +00:33:46.660 --> 00:33:48.940 +Open source or private or whatever. + +00:33:49.620 --> 00:33:51.820 +is not the main factor + +00:33:52.680 --> 00:33:54.640 +defining the success of a project, + +00:33:55.080 --> 00:33:57.240 +being it commercial or open source of any type, + +00:33:57.700 --> 00:34:00.280 +is not really how many people are behind it. + +00:34:00.640 --> 00:34:02.960 +It's more of what is the value + +00:34:03.140 --> 00:34:05.380 +that whoever is putting the effort to keep it alive + +00:34:05.840 --> 00:34:07.660 +is getting from putting all that effort. + +00:34:07.980 --> 00:34:09.020 +It could be just satisfaction. + +00:34:09.419 --> 00:34:10.600 +It could be like open source, + +00:34:10.679 --> 00:34:13.240 +like, oh, I feel so good that I'm contributing to society. + +00:34:13.360 --> 00:34:13.940 +And that is valid. + +00:34:14.600 --> 00:34:15.440 +It doesn't pay the rent, + +00:34:15.899 --> 00:34:16.899 +but it's still valid. + +00:34:17.080 --> 00:34:18.100 +It might last for a while. + +00:34:18.440 --> 00:34:25.820 +But then also like, you know, like when you see like there are so many Python projects, so many Python, so many open source projects that can do well or can do bad. + +00:34:26.000 --> 00:34:28.300 +And it doesn't really depend on how many people they have. + +00:34:28.820 --> 00:34:40.780 +And when you are using a project, when you're using an open source project or when you are using a product of any type, I will encourage you to think about what is the best ticket factor of this project? + +00:34:41.260 --> 00:34:47.500 +What are the things that whoever is building this is receiving in exchange for giving it away? + +00:34:47.940 --> 00:34:51.840 +So like, you know, like what are they expecting to sell you at some point? + +00:34:52.300 --> 00:34:54.500 +Or what are they receiving in exchange? + +00:34:55.080 --> 00:34:58.520 +You know, for example, Bun, the JavaScript Brompton LiDAR. + +00:34:58.820 --> 00:35:00.880 +Like it was like, we don't know what they're going to sell. + +00:35:01.040 --> 00:35:06.940 +But now, you know, Cloud and Entropic really want to have like this thing keep working because they are using it internally. + +00:35:07.000 --> 00:35:09.360 +So you can say like, OK, I'm going to use it. + +00:35:09.420 --> 00:35:10.340 +I'm going to use it for free. + +00:35:10.500 --> 00:35:15.840 +I know that what they receive for me using is just like that they just really want it. + +00:35:15.980 --> 00:35:17.740 +So I can just like, whenever you are using bond, + +00:35:17.860 --> 00:35:20.380 +you are getting, now you are getting free services + +00:35:20.600 --> 00:35:21.460 +from Antropoc, that's it. + +00:35:21.860 --> 00:35:23.940 +But you know, like every time you are using a project, + +00:35:24.040 --> 00:35:26.680 +you can think about why are people receiving an exchange + +00:35:27.080 --> 00:35:28.700 +for giving this away for me? + +00:35:28.960 --> 00:35:31.100 +This is like the thing that I would like people + +00:35:31.340 --> 00:35:33.800 +to think about, you know, like also like + +00:35:34.020 --> 00:35:35.120 +how can they give back? + +00:35:35.280 --> 00:35:37.860 +Maybe they can actually contribute to that community + +00:35:38.080 --> 00:35:38.660 +or to that project. + +00:35:38.800 --> 00:35:40.440 +There are many ways and in many cases, + +00:35:41.140 --> 00:35:43.260 +the thing that is needed the most is just like help + +00:35:43.620 --> 00:35:45.420 +and work, just answering questions and issues. + +00:35:47.260 --> 00:35:52.320 +this portion of talk python enemy is brought to you by us i'm excited to talk about my first solo + +00:35:52.520 --> 00:35:57.780 +book talk python in production it's an inside look at how we host all the talk python sites + +00:35:58.120 --> 00:36:03.440 +apis mobile apps and way more here's the thing i believe most hosting stories sold to developers + +00:36:03.520 --> 00:36:08.720 +and data scientists are way over complicated and overpriced you've heard me say you're not google + +00:36:09.000 --> 00:36:13.859 +you're not netflix so you shouldn't run your infrastructure the way they do but if not that + +00:36:13.960 --> 00:36:20.140 +then what? This book is both a blueprint for what I chose for Talk Python and a story arc of 10 years + +00:36:20.140 --> 00:36:25.680 +of running my own infrastructure from a complete newbie, apprehensive to Linux, to some pretty + +00:36:25.860 --> 00:36:31.940 +neat infrastructures code DevOps. It covers Docker, Nginx, Let's Encrypt, self-hosted analytics and + +00:36:32.140 --> 00:36:37.880 +monitoring, CDN setup, framework migrations, and a whole philosophy that I've termed stack native, + +00:36:38.460 --> 00:36:42.980 +keeping things streamlined, powerful, and free of cloud lock-in. And it's more than just your + +00:36:43.000 --> 00:36:48.120 +standard tech book. It comes with code and figure galleries on GitHub, a discussion forum, and + +00:36:48.500 --> 00:36:54.020 +something unique, over an hour of audio readers briefs, short conversations that bookend each + +00:36:54.460 --> 00:36:59.920 +chapter to prime your focus or broaden your takeaways. Oh, and 0% of this book was written + +00:37:00.040 --> 00:37:03.960 +by AI. Every word is mine, written over the course in high months, for better or worse. + +00:37:04.500 --> 00:37:09.820 +I've made the first third of the book available for free online. After that, you can grab the DRM + +00:37:09.840 --> 00:37:15.380 +free EPUB and Kindle editions. And I'm working on a paperback edition as well. Please check it out + +00:37:15.460 --> 00:37:20.740 +at talkpython.fm/DevOps, or just click book in the nav bar on the website. It's a great way to + +00:37:20.820 --> 00:37:25.040 +support the podcast. And I hope it changes a bit how you think about running your apps in production. + +00:37:26.040 --> 00:37:30.480 +Kind of related to what you're saying, I think one of the angles that I really appreciate about + +00:37:30.560 --> 00:37:36.160 +the way we think about FastAPI and FastAPI Cloud is like where like a lot of our team was involved + +00:37:36.180 --> 00:37:41.020 +in open source before coming to work at FastAPI Cloud on various projects around the Python + +00:37:41.260 --> 00:37:42.240 +ecosystem, outside of Python. + +00:37:42.740 --> 00:37:48.140 +And I think all of us have deep appreciation and understanding of the value of open source + +00:37:48.780 --> 00:37:53.460 +and really, really try and build in a way that is like, I mean, Sebastian, you've talked + +00:37:53.560 --> 00:37:56.540 +about this a lot, but solving a real problem for folks, right? + +00:37:56.840 --> 00:38:01.920 +And so FastAPI Cloud is sort of this extension of this open source ecosystem people would + +00:38:01.980 --> 00:38:02.380 +be using. + +00:38:03.680 --> 00:38:05.120 +FastAPI Cloud may be an option. + +00:38:05.500 --> 00:38:07.580 +Maybe someone picks some other cloud for some reason. + +00:38:07.970 --> 00:38:10.580 +I don't think like, I think we're all very mindful of that. + +00:38:10.790 --> 00:38:12.520 +But like the angle that's very cool, I think, + +00:38:12.600 --> 00:38:15.600 +is that like, because we all work at FastAPI Cloud, + +00:38:15.880 --> 00:38:17.580 +like I know that I personally have time, + +00:38:17.860 --> 00:38:19.220 +more time for my open source work + +00:38:19.380 --> 00:38:21.420 +and my employer understands the value + +00:38:21.560 --> 00:38:22.540 +of my open source work, + +00:38:23.020 --> 00:38:25.160 +which isn't that positive for the open source community. + +00:38:25.290 --> 00:38:27.300 +Like I get to work on CPython sometimes + +00:38:27.820 --> 00:38:29.780 +and I have, you know, the bandwidth + +00:38:29.950 --> 00:38:31.480 +to go and do my steering council work + +00:38:31.540 --> 00:38:32.880 +or upcoming release management work. + +00:38:33.300 --> 00:38:34.880 +I understand like this sort of like, + +00:38:35.420 --> 00:38:39.200 +tempering, like open source, commercial, bad, all bad. + +00:38:39.400 --> 00:38:39.940 +It's not all bad. + +00:38:40.060 --> 00:38:41.920 +It's actually like really good in a lot of cases + +00:38:42.280 --> 00:38:43.740 +for folks to build business. + +00:38:43.920 --> 00:38:46.320 +Look at uv for an example to hold up, right? + +00:38:46.420 --> 00:38:47.700 +Astral, yeah, yeah, totally. + +00:38:47.860 --> 00:38:48.160 +Yeah, yeah. + +00:38:48.340 --> 00:38:50.180 +I think there are some really good examples of this. + +00:38:50.370 --> 00:38:52.900 +So I think like that's another angle that, + +00:38:52.980 --> 00:38:55.480 +I mean, I really, I get a lot of energy out of our team + +00:38:55.890 --> 00:38:57.380 +because we all, I don't have to, + +00:38:57.630 --> 00:38:59.220 +I don't have to fight the open source battle + +00:38:59.580 --> 00:39:00.860 +at FastAPI Cloud. + +00:39:01.160 --> 00:39:02.080 +I think that's really cool. + +00:39:02.260 --> 00:39:03.620 +I do think that's super cool as well. + +00:39:03.660 --> 00:39:05.260 +Let me put out two examples for you. + +00:39:05.340 --> 00:39:11.840 +here that I think everyone will be aware of as sort of to add to what Sebastian was saying is + +00:39:12.140 --> 00:39:17.500 +look how much Apple freaked out when Steve Jobs died and how many people work at Apple, right? + +00:39:17.940 --> 00:39:23.040 +Like that was still like, oh my gosh. But, you know, I think there's, they're hanging in there. + +00:39:23.140 --> 00:39:27.920 +They're going to be probably making it. They are not in our business. I tell you what, + +00:39:27.950 --> 00:39:35.300 +they got some of my money. That's for sure. But also, you know, look at Flask, right? Armin + +00:39:35.500 --> 00:39:41.000 +drifted away, which is totally fine. And David Lord and Pallets picked it up and kept running, + +00:39:41.090 --> 00:39:46.020 +right? Like it's still one of the most popular frameworks out there, right? So it's, I think + +00:39:46.030 --> 00:39:51.200 +the bus factor is over, overblown a bit, but also looking at the team of folks here, I think it's, + +00:39:51.340 --> 00:39:54.880 +it's even more obvious that there's a bunch of people who are on the inside, you know? + +00:39:54.990 --> 00:39:59.560 +For example, Flask, you know, like I learned so many things from Flask and like, the thing is, + +00:39:59.750 --> 00:40:04.020 +I feel like sometimes, sometimes people go and complain about the tool and say like, oh, + +00:40:04.140 --> 00:40:08.980 +this is not working for this or for that. And in many cases, it's in this insensitive way towards + +00:40:09.260 --> 00:40:13.360 +the people that are working on that. And it's like, you know, like in the end, realize that + +00:40:13.480 --> 00:40:17.440 +there's actually people behind the scenes doing the work. And like, in many cases, it's just like + +00:40:17.620 --> 00:40:22.480 +one or two people doing a lot of work in many cases, just for free. And, you know, like, I think + +00:40:22.820 --> 00:40:27.480 +it's worth calling that out. Like all the work that David Lord does for Flask is just like so + +00:40:27.600 --> 00:40:32.780 +much work. And yeah, deserves a lot of respect. I totally agree. The other thing that I forgot to + +00:40:32.800 --> 00:40:37.740 +mention is that there are so many ideas of potential products that I could build over the years, and I + +00:40:37.820 --> 00:40:42.840 +never did, and I never started a company because I didn't have clarity of what will be a good thing + +00:40:43.140 --> 00:40:49.860 +to actually sell and will have a good alignment. The cloud product has such a good alignment with + +00:40:49.860 --> 00:40:57.640 +the open source side because as more successful FastAPI is, the more successful FastAPI cloud + +00:40:57.920 --> 00:40:59.140 +has a potential to be. + +00:40:59.840 --> 00:41:02.640 +The more people using Python effectively, + +00:41:03.040 --> 00:41:05.800 +the more people might end up checking out FastAPI + +00:41:05.820 --> 00:41:07.800 +and the more people might end up checking out the product. + +00:41:08.240 --> 00:41:10.980 +So if FastAPI does well, if the open source does well, + +00:41:11.060 --> 00:41:13.140 +if Python does well, that's better for the company. + +00:41:13.340 --> 00:41:17.420 +So it doesn't really depend on my personal principles + +00:41:17.580 --> 00:41:19.020 +and values or something like that. + +00:41:19.340 --> 00:41:23.680 +It's aligned with, it's financially aligned with the company. + +00:41:24.040 --> 00:41:27.620 +So it's just going to be beneficial in the end + +00:41:27.640 --> 00:41:29.680 +It doesn't depend on good intentions. + +00:41:30.519 --> 00:41:31.760 +And FastAPI is open source. + +00:41:31.980 --> 00:41:33.800 +It has like 7,000 forks or something. + +00:41:34.020 --> 00:41:36.320 +So if a boost runs over me, + +00:41:37.020 --> 00:41:38.100 +there are 7,000 forks. + +00:41:38.220 --> 00:41:39.000 +It's not going away. + +00:41:39.160 --> 00:41:40.420 +I definitely agree with you on that. + +00:41:40.470 --> 00:41:43.300 +I feel like I should maybe give a little bit of a, + +00:41:43.860 --> 00:41:44.880 +I'll tell a little bit of the story + +00:41:45.130 --> 00:41:46.120 +of what's going on with, + +00:41:46.910 --> 00:41:47.600 +where did I put it? + +00:41:47.840 --> 00:41:48.780 +I don't think I pasted it over here, + +00:41:49.200 --> 00:41:51.160 +is what's going on with Tailwind right now. + +00:41:51.550 --> 00:41:54.180 +And I think Tailwind is having a tough time, + +00:41:54.280 --> 00:41:54.700 +Tailwind CSS. + +00:41:55.480 --> 00:42:01.500 +Traffic to Tailwind is up six times year over year on npm downloads. + +00:42:02.280 --> 00:42:07.060 +But the revenue of Tailwind is down five times. + +00:42:07.540 --> 00:42:11.580 +You know, I mean, these are completely out of whack things because instead of people going + +00:42:11.580 --> 00:42:15.560 +to docs to learn about it, it's just like, well, when you go to the docs, you learn they + +00:42:15.640 --> 00:42:16.960 +also have premium offerings, right? + +00:42:17.440 --> 00:42:22.240 +And I think you guys are different because it's not just, oh, here's a little bit nicer + +00:42:22.420 --> 00:42:23.520 +of a thing, right? + +00:42:23.680 --> 00:42:34.040 +I feel like it would be a little bit as if you were selling cookie cutter templates for FastAPI apps, you know, it's like, well, the AI can make the shape of the thing that comes out of the cookie cutter, to be honest. + +00:42:34.260 --> 00:42:41.380 +But you're offering something that has ongoing value that it costs more and is more complex in other places. + +00:42:42.020 --> 00:42:50.720 +And so I think maybe just thinking about the how this just keeps the team going for FastAPI is really awesome. + +00:42:50.880 --> 00:42:53.260 +And I think it's got a nice flywheel effect there. + +00:42:53.340 --> 00:42:56.540 +is I'll link to this, I guess, audio track. + +00:42:56.730 --> 00:42:57.540 +I don't know what I call it. + +00:42:57.580 --> 00:42:59.000 +It's a blog post that has one sentence, + +00:42:59.260 --> 00:43:00.960 +but a 30-minute audio you can check out + +00:43:01.120 --> 00:43:02.960 +from the guy, Adam, + +00:43:03.420 --> 00:43:04.600 +who's one of the founders of Tailwind, + +00:43:04.780 --> 00:43:06.060 +talking about going into this. + +00:43:06.740 --> 00:43:07.360 +It's kind of rough. + +00:43:07.450 --> 00:43:09.900 +I think I don't necessarily want to go into a deep AI, + +00:43:10.080 --> 00:43:11.000 +what it means for the industry. + +00:43:11.180 --> 00:43:13.120 +Like, let's stay focused on what you guys are doing. + +00:43:13.150 --> 00:43:16.940 +But I think it's going to be its own series. + +00:43:17.280 --> 00:43:21.240 +I mean, Stack Overflow had as many questions asked + +00:43:21.260 --> 00:43:25.240 +this month as they did in the first month of their existence, right? + +00:43:25.530 --> 00:43:29.480 +Three or 4,000, whereas at their peak, they were 200,000 questions a month. + +00:43:29.900 --> 00:43:34.920 +There's like real turmoil that's coming from some of these things, which is tricky. + +00:43:35.560 --> 00:43:40.000 +But I'm really excited to see you all doing this because I'm a big fan of FastAPI. + +00:43:40.500 --> 00:43:45.840 +And I think this is just sustaining and more for FastAPI, right? + +00:43:45.920 --> 00:43:46.520 +Like, what do you all think? + +00:43:46.640 --> 00:43:48.260 +That's what we hope that is going on. + +00:43:49.920 --> 00:43:51.840 +I thought about Taiwan for a second, right? + +00:43:52.060 --> 00:43:54.280 +It's not like we're immune to what happened to them. + +00:43:54.420 --> 00:43:56.500 +Like we also have a lot of documentation online. + +00:43:56.880 --> 00:43:58.040 +AI could train on that. + +00:43:58.240 --> 00:43:59.400 +And if it's good enough, + +00:43:59.420 --> 00:44:01.300 +it could maintain your infrastructure and stuff. + +00:44:01.380 --> 00:44:02.960 +It's just too hard at the moment. + +00:44:03.080 --> 00:44:05.460 +And there's an additional thing we're kind of selling, + +00:44:05.760 --> 00:44:07.620 +which is like, I guess, responsibility. + +00:44:08.180 --> 00:44:11.480 +Like you're shifting the risk from like letting your AI + +00:44:11.800 --> 00:44:15.540 +or your infantry team maintain your infrastructure to us. + +00:44:15.920 --> 00:44:18.660 +So we're staying up at night and worry about it. + +00:44:18.980 --> 00:44:20.400 +that has a lot of value as well. + +00:44:20.780 --> 00:44:24.300 +And that's probably not going to get removed by AI. + +00:44:24.980 --> 00:44:27.040 +Here's a very common cloud code, + +00:44:27.540 --> 00:44:28.760 +cursor, whatever conversation. + +00:44:29.520 --> 00:44:33.140 +Hey, build me something with Python and needs an API. + +00:44:33.240 --> 00:44:34.600 +Okay, we built it with FastAPI. + +00:44:34.900 --> 00:44:35.920 +How do I host it? + +00:44:36.230 --> 00:44:37.160 +Right, that doesn't just, + +00:44:37.380 --> 00:44:39.060 +it will build a cloud for you, right? + +00:44:39.140 --> 00:44:41.400 +It's going to recommend something out there. + +00:44:41.840 --> 00:44:45.120 +And a real natural way to how do I host FastAPI + +00:44:45.280 --> 00:44:46.740 +is FastAPI cloud, right? + +00:44:46.880 --> 00:44:48.060 +Like if it suggests, + +00:44:48.200 --> 00:44:50.940 +oh, you're just going to like spread it across Lambda by breaking. + +00:44:51.080 --> 00:44:52.460 +Like, whoa, no, I want something simple. + +00:44:52.640 --> 00:44:54.180 +Okay, give me FastAPI cloud, right? + +00:44:54.320 --> 00:44:55.860 +I think that that's a really good thing. + +00:44:55.860 --> 00:44:57.800 +And then on the enterprise side, + +00:44:58.340 --> 00:45:03.220 +enterprise folks are notoriously not good at supporting open source + +00:45:03.440 --> 00:45:05.720 +in that they're not like paying for it. + +00:45:05.720 --> 00:45:11.000 +I know some companies are big supporters of the PSF and Python and open source. + +00:45:11.440 --> 00:45:15.240 +But in general, it's like, yeah, we have this project with 5,000 people working on it. + +00:45:15.240 --> 00:45:15.680 +It's all Python. + +00:45:16.340 --> 00:45:19.240 +And are we sponsoring this? + +00:45:19.640 --> 00:45:19.780 +Nope. + +00:45:20.140 --> 00:45:22.060 +We're just enjoying the money, right? + +00:45:22.180 --> 00:45:22.740 +And we're a bank. + +00:45:23.360 --> 00:45:24.140 +So we got the money. + +00:45:24.260 --> 00:45:24.860 +We got all the money. + +00:45:25.780 --> 00:45:29.520 +So they're just not good at paying for like a really great framework that they use a lot. + +00:45:29.860 --> 00:45:34.460 +But they got plenty of hosting, plenty of internal apps that they just need to make run and stuff. + +00:45:34.680 --> 00:45:40.220 +So I think both on like the low end and the high end, there's a lot of synergy between these things. + +00:45:40.340 --> 00:45:52.540 +That is not just, you know, slightly advanced, not to diminish it, but slightly advanced UI widgets that you could ask your AI to build or something or like cookie cutter templates for project starters. + +00:45:52.840 --> 00:45:59.120 +I think we are in a somewhat fortunate position of like, you know, like FastAPI. FastAPI has grown so much. + +00:45:59.560 --> 00:46:06.100 +Like, you know, like when you check the statistics about downloads or GitHub stars or entries in developer surveys, + +00:46:06.220 --> 00:46:08.680 +like it's at the top in like in each category. + +00:46:08.860 --> 00:46:13.560 +It's like, you know, like the backend framework with the most GitHub stars across languages, + +00:46:14.040 --> 00:46:17.180 +even like, you know, like Java, Go, Ruby, JS, like whatever. + +00:46:17.280 --> 00:46:19.840 +It's like the top one, at least in GitHub stars. + +00:46:20.260 --> 00:46:24.620 +So like, you know, like FastAPI is like people are liking it, fortunately. + +00:46:25.480 --> 00:46:29.080 +And there's probably going to be people deploying things to FastAPI Cloud. + +00:46:29.100 --> 00:46:32.600 +So that's probably going to be like, we are probably going to be fine. + +00:46:33.180 --> 00:46:38.760 +I think, you know, like the, I guess it will be like a good point to ask people to go and + +00:46:38.880 --> 00:46:42.580 +check where the open source project is that they are using and check where is the bus ticket + +00:46:42.800 --> 00:46:44.920 +factor for those open source projects. + +00:46:45.040 --> 00:46:55.800 +You know, like if you are using Tailwind CSS, it would have been very cool if at some point you check if the premium things were useful for you and for your company or your project or something like that, you know? + +00:46:56.100 --> 00:46:59.400 +Yeah, because what is the thing that keeps that project going? + +00:46:59.560 --> 00:46:59.880 +Exactly. + +00:47:00.260 --> 00:47:12.820 +And I really personally admire if a project or something offers like more value, not just, hey, buy me a coffee, but here's a thing that you get way more of, you know? + +00:47:12.820 --> 00:47:14.840 +And in that regard, I think Tailwind was doing that, right? + +00:47:14.960 --> 00:47:17.240 +They were offering this suite of pre-built things. + +00:47:17.570 --> 00:47:18.940 +And I think that that's great. + +00:47:19.280 --> 00:47:24.440 +But yeah, I do think you've got more of these crazy AI things + +00:47:24.470 --> 00:47:26.680 +are going to maybe recommend FastAPI Cloud more + +00:47:26.960 --> 00:47:28.340 +than they're just going to undercut it. + +00:47:28.350 --> 00:47:29.200 +So I think that's really great. + +00:47:29.460 --> 00:47:33.180 +And by the way, I was just looking for the GitHub Stars graph. + +00:47:33.530 --> 00:47:36.200 +Like there's a whole, I can't remember what the domain of that site is. + +00:47:36.840 --> 00:47:38.980 +And I ran across, by the way, I just want to give a quick shout out. + +00:47:39.130 --> 00:47:43.420 +Like your cult repo documentary on FastAPI was awesome. + +00:47:44.000 --> 00:47:44.080 +Right? + +00:47:44.420 --> 00:47:45.080 +That was so fun. + +00:47:45.440 --> 00:47:46.280 +They made me look good. + +00:47:46.280 --> 00:47:47.040 +I didn't see that coming. + +00:47:47.660 --> 00:47:50.060 +Yeah, it came right on the heels of the Python official + +00:47:50.700 --> 00:47:51.800 +documentary, the one hour one. + +00:47:51.900 --> 00:47:54.200 +This is the same group, and the production quality + +00:47:54.260 --> 00:47:54.660 +is really nice. + +00:47:54.750 --> 00:47:55.320 +So like-- + +00:47:55.440 --> 00:47:58.580 +When they released the trailer for the Python documentary, + +00:47:58.800 --> 00:48:00.940 +before releasing the documentary, when they released the trailer, + +00:48:01.180 --> 00:48:02.980 +they contacted me and said, hey, we're + +00:48:03.670 --> 00:48:06.080 +doing these mini documentaries about different frameworks, + +00:48:06.230 --> 00:48:08.360 +different tools, and we want to include FastAPI there. + +00:48:08.410 --> 00:48:09.260 +I was like, oh, nice. + +00:48:10.040 --> 00:48:12.220 +But then I was just trying to stay excited, + +00:48:12.440 --> 00:48:13.200 +but super excited. + +00:48:13.760 --> 00:48:17.740 +Oh, that's so cool. Yeah, I watched it as soon as it came out. So I'll link to that. People should + +00:48:17.880 --> 00:48:22.500 +definitely, it's only like 10 minutes or something, but it's worth it. We're checking out. So it's not + +00:48:22.500 --> 00:48:26.900 +a huge investment time. People can watch it, I suppose. It's not TikTok. I mean, it's not like, + +00:48:27.440 --> 00:48:34.900 +oh, I saw the documentary, but it doesn't take you on huge about it. + +00:48:34.960 --> 00:48:37.840 +You have to listen for 10 minutes, overly excited Colombian. + +00:48:37.940 --> 00:48:41.320 +I don't understand what's happened to the attention span of society. I'm really, + +00:48:41.560 --> 00:48:45.600 +honestly a little concerned. I used to, when I would create my courses, people would say, + +00:48:45.760 --> 00:48:49.460 +you know, like a four hour course and there'd be like a 10, 15 minute sort of, Hey, here's how you + +00:48:49.580 --> 00:48:52.280 +set up your computer. And here's all the introduction and people, Oh, that's so awesome. + +00:48:52.400 --> 00:48:55.200 +I loved how you kind of set the stage. I'm really motivated to take the course. + +00:48:55.840 --> 00:48:59.800 +Nowadays, I just get messages like, why are you still talking? This is five minutes long. Do you + +00:48:59.960 --> 00:49:06.500 +understand? I'm like, this is your job. You can't spend five minutes. Oh my gosh. Anyway, that's, + +00:49:06.620 --> 00:49:11.520 +that's sort of the origin of my comment there. It's all right. So we're kind of getting so much + +00:49:11.540 --> 00:49:17.380 +time, I think I want to talk about a couple of things. Let's talk a little bit about internals. + +00:49:17.460 --> 00:49:22.820 +Like what, I don't know who wants to take this one, but let's talk about just how, when I say + +00:49:23.080 --> 00:49:30.140 +FastAPI deploy, then what? It's just a uv pip install and it just goes and it's magic and it's + +00:49:30.240 --> 00:49:34.320 +easy, right? We have a nickname for Jonathan. Can we say it or no? I don't know. It's so funny. + +00:49:34.380 --> 00:49:38.660 +This happened because I told my friends, I'm so concerned about being at the podcast because + +00:49:38.680 --> 00:49:42.000 +Because everyone here is a visionary, and then I'm the back-end guy. + +00:49:42.300 --> 00:49:46.300 +I think the things I could contribute to this conversation, I should probably keep to myself. + +00:49:47.540 --> 00:49:50.020 +But you're just leaking your turnouts, right? + +00:49:50.180 --> 00:49:52.940 +There are some things that are not really secret. + +00:49:53.340 --> 00:50:01.300 +Like, as Sebastian said earlier, Kubernetes 1 in the infrastructure and deployment field, to some extent. + +00:50:01.500 --> 00:50:03.600 +So that's somewhere in there, right? + +00:50:03.880 --> 00:50:07.260 +But it's all the way deep down, so no one has to worry about it. + +00:50:07.420 --> 00:50:08.500 +But it's still a foundation. + +00:50:08.580 --> 00:50:09.580 +which is a good foundation. + +00:50:09.950 --> 00:50:11.300 +I think one thing that's, + +00:50:11.610 --> 00:50:12.500 +you might have guessed it, + +00:50:12.800 --> 00:50:15.240 +but FastAPI Cloud is built on FastAPI, + +00:50:15.480 --> 00:50:16.440 +which kind of makes sense, right? + +00:50:16.760 --> 00:50:17.940 +And that also has an effect + +00:50:18.200 --> 00:50:20.500 +on like recent patches, updates and stuff. + +00:50:20.720 --> 00:50:22.560 +Because if we find something internally + +00:50:23.000 --> 00:50:23.860 +which we're not happy with, + +00:50:24.040 --> 00:50:25.240 +then we just fix it. + +00:50:25.440 --> 00:50:27.080 +And that's how some releases + +00:50:27.600 --> 00:50:29.460 +came out faster than months before. + +00:50:30.220 --> 00:50:31.420 +Power of dogfooding. + +00:50:31.820 --> 00:50:32.480 +Yeah, that's awesome. + +00:50:32.620 --> 00:50:33.080 +Dogfooding a lot. + +00:50:33.280 --> 00:50:35.100 +Also all the related libraries + +00:50:35.540 --> 00:50:38.360 +like SQL model and, well, others. + +00:50:38.840 --> 00:50:39.940 +they experience the same thing. + +00:50:40.160 --> 00:50:41.500 +New library is coming out. + +00:50:42.020 --> 00:50:43.680 +Patrick will announce at some point soon. + +00:50:43.860 --> 00:50:45.680 +It's not just FastJPay and friends. + +00:50:45.980 --> 00:50:46.980 +We're like really open. + +00:50:47.260 --> 00:50:49.160 +Like recently, Patrick just open-sourced + +00:50:49.540 --> 00:50:51.680 +everything we use for authentication authorization, + +00:50:52.160 --> 00:50:52.520 +for example. + +00:50:52.840 --> 00:50:53.700 +Is it open-source yet? + +00:50:53.960 --> 00:50:54.840 +Did they just leak something? + +00:50:55.000 --> 00:50:56.940 +It will be announced soon at some point. + +00:50:57.920 --> 00:50:59.720 +We build stuff internally in the moment really. + +00:51:00.520 --> 00:51:01.820 +Like we build it in a way, + +00:51:01.960 --> 00:51:02.800 +like in a separate package, + +00:51:03.260 --> 00:51:04.420 +just like an open-source library. + +00:51:04.540 --> 00:51:05.740 +And if we feel like the time is ripe, + +00:51:05.860 --> 00:51:07.260 +it's just getting open-sourced + +00:51:07.280 --> 00:51:08.860 +because a lot of things are reusable. + +00:51:09.420 --> 00:51:10.460 +And that's in all departments. + +00:51:10.900 --> 00:51:11.700 +That happens a lot. + +00:51:12.320 --> 00:51:14.660 +When I started there, I already realized that. + +00:51:14.760 --> 00:51:15.880 +Everyone was building open source, + +00:51:16.030 --> 00:51:17.600 +but now I joined in myself as well. + +00:51:17.720 --> 00:51:22.020 +I open source the library for compressing + +00:51:22.120 --> 00:51:24.240 +and decompressing archives in Python + +00:51:24.860 --> 00:51:27.440 +because the internal top high thing is just slow + +00:51:27.520 --> 00:51:28.660 +and we needed it to be faster + +00:51:28.840 --> 00:51:30.620 +because we're staring at the deployment process + +00:51:30.690 --> 00:51:33.160 +and we're like, hey, we could probably shave off a few seconds here. + +00:51:33.760 --> 00:51:35.740 +Then that's just open source for everyone to use. + +00:51:35.800 --> 00:51:38.900 +So we're contributing to the old Python ecosystem as well. + +00:51:38.900 --> 00:51:39.940 +You have to say the name. + +00:51:40.140 --> 00:51:40.820 +It's so good. + +00:51:41.500 --> 00:51:42.080 +Is it good? + +00:51:42.240 --> 00:51:46.280 +No, it's just, it's faster because it's, you know, faster than just tar. + +00:51:47.260 --> 00:51:47.880 +Fast tar? + +00:51:48.160 --> 00:51:48.500 +I love it. + +00:51:48.510 --> 00:51:49.140 +Fast tar, yes. + +00:51:49.760 --> 00:51:53.300 +And you can say with that very, very German accent, fast tar. + +00:51:53.420 --> 00:51:54.140 +I'll go star it. + +00:51:54.200 --> 00:51:55.820 +We'll get you some stars. + +00:51:56.520 --> 00:51:57.060 +This is going to happen. + +00:51:57.200 --> 00:51:58.300 +That's the irony about it. + +00:51:58.440 --> 00:51:59.960 +Like, it literally has no stars. + +00:52:00.090 --> 00:52:01.660 +But if you scroll down, you see the downloads. + +00:52:01.880 --> 00:52:04.100 +That's going to prove we're actually using it. + +00:52:04.180 --> 00:52:04.740 +Yeah, I like it. + +00:52:05.060 --> 00:52:06.100 +It's a little context manager. + +00:52:07.660 --> 00:52:12.700 +It's almost working the same as the TAL file in the standard library. + +00:52:13.000 --> 00:52:16.060 +Like the same, like almost similar API to that. + +00:52:16.360 --> 00:52:18.720 +It's basically a drop-in replacement, more or less. + +00:52:19.440 --> 00:52:21.400 +But they know they need everything to happen in Rust. + +00:52:21.680 --> 00:52:22.080 +Because Rust. + +00:52:22.420 --> 00:52:23.340 +Because Rust, yeah. + +00:52:23.560 --> 00:52:27.180 +Well, as soon as it becomes infrastructure and you've got to run it a million times, + +00:52:27.740 --> 00:52:29.100 +that starts to make sense, right? + +00:52:29.220 --> 00:52:29.280 +Yeah. + +00:52:29.420 --> 00:52:33.180 +Python is one of the fastest programming languages in the world. + +00:52:33.680 --> 00:52:36.960 +when you think about human time to build the things, right? + +00:52:37.140 --> 00:52:39.480 +Like that's one of its real superpowers is like, + +00:52:39.800 --> 00:52:44.060 +I mean, there's the whole story of Google Video and YouTube, right? + +00:52:44.160 --> 00:52:46.920 +And Google Video was written in C++ with 100 engineers + +00:52:47.160 --> 00:52:49.220 +and YouTube was a small team in Python + +00:52:49.460 --> 00:52:51.100 +and they just couldn't keep up with the features. + +00:52:51.200 --> 00:52:53.000 +So they bought this little old thing, YouTube, + +00:52:53.560 --> 00:52:55.000 +and see if we're going to make something with it. + +00:52:55.280 --> 00:52:57.220 +And last I checked, it was still in Python. + +00:52:57.420 --> 00:53:00.260 +I'm sure some of it isn't, but a few years ago it was, which is wild. + +00:53:00.760 --> 00:53:02.560 +Anyway, there's different ways of fast, + +00:53:02.680 --> 00:53:04.520 +But when it's down to like little utilities, yeah. + +00:53:04.620 --> 00:53:06.660 +I know some people that are trying to make Python fast. + +00:53:06.710 --> 00:53:07.220 +I know a couple. + +00:53:07.440 --> 00:53:07.620 +Yeah. + +00:53:08.200 --> 00:53:11.540 +And honestly, massive success in the last five years, right? + +00:53:11.740 --> 00:53:17.400 +Like since 3.11, since the specializing adaptive interpreter, there's been pretty big improvements. + +00:53:17.720 --> 00:53:20.360 +3.9 and 3.11 did a lot of like foundational work. + +00:53:20.410 --> 00:53:24.020 +And then 3.9 onward really just uncorked a lot of innovation there. + +00:53:24.340 --> 00:53:25.260 +Yeah, that's pretty awesome. + +00:53:26.000 --> 00:53:26.260 +All right. + +00:53:26.560 --> 00:53:29.320 +It sounds like, Sebastian, you've talked a lot about Kubernetes. + +00:53:29.740 --> 00:53:31.360 +So I imagine Kubernetes is happening. + +00:53:31.960 --> 00:53:34.820 +Do we get to pick what data centers it runs on? + +00:53:34.840 --> 00:53:37.720 +Do we get to pick what clouds it runs on? + +00:53:37.920 --> 00:53:40.560 +You're going to get to pick some of these things. + +00:53:41.340 --> 00:53:41.800 +Not yet. + +00:53:42.100 --> 00:53:43.000 +It's not released yet. + +00:53:43.600 --> 00:53:45.320 +But, you know, like it's top, of course. + +00:53:45.620 --> 00:53:48.680 +Like we have like regular cloud providers on the MIT + +00:53:48.720 --> 00:53:49.720 +and there's a bunch of Kubernetes. + +00:53:49.920 --> 00:53:52.140 +Then there's a bunch of additional stuff + +00:53:52.360 --> 00:53:53.560 +that needs to run on top. + +00:53:53.760 --> 00:53:55.599 +Then there's like custom Kubernetes controllers + +00:53:56.320 --> 00:53:58.560 +and things that Jonathan was saying + +00:53:58.580 --> 00:54:00.399 +that he's having to write in Go + +00:54:00.420 --> 00:54:04.180 +so that people in Python can be happy to be able to, you know, + +00:54:04.200 --> 00:54:07.040 +like manage all the Kubernetes shenanigans that need to happen + +00:54:07.160 --> 00:54:09.700 +because there's so much complexity that needs to be handled. + +00:54:10.340 --> 00:54:11.440 +There's a lot of that. + +00:54:11.760 --> 00:54:14.860 +We do a lot of advanced tricks also. + +00:54:15.380 --> 00:54:17.440 +Jonathan was recently doing a bunch of advanced tricks + +00:54:17.580 --> 00:54:20.160 +to handle the caches for the builds. + +00:54:20.520 --> 00:54:22.120 +So the way that we handle caches, + +00:54:22.260 --> 00:54:25.220 +and we also like tap into uv and how things work + +00:54:25.340 --> 00:54:28.399 +so that builds can be super, super fast + +00:54:28.420 --> 00:54:30.080 +because it's like something is, + +00:54:30.540 --> 00:54:33.280 +we are, you know, like we are very much targeted + +00:54:33.560 --> 00:54:35.940 +at FastAPI and Python in general. + +00:54:36.100 --> 00:54:38.440 +So we can take advantage of knowing + +00:54:39.060 --> 00:54:40.260 +how things run internally, + +00:54:40.720 --> 00:54:41.700 +how things are installed, + +00:54:42.220 --> 00:54:43.280 +how to optimize everything. + +00:54:43.380 --> 00:54:45.000 +So everything is just like super fast, + +00:54:45.140 --> 00:54:47.640 +super fast to install, to run, to like do everything. + +00:54:47.700 --> 00:54:50.400 +I imagine you all have base Docker images + +00:54:50.960 --> 00:54:53.700 +that are like just one layer away + +00:54:54.080 --> 00:54:55.880 +from whoever's code is running. + +00:54:56.440 --> 00:54:58.060 +You know, like you've got it all optimized, + +00:54:58.120 --> 00:55:02.020 +already pre-built with FastAPI and whatever settings of Python you want. + +00:55:02.140 --> 00:55:03.460 +A bunch of things and tricks. + +00:55:04.100 --> 00:55:05.340 +But there are also different things, + +00:55:05.560 --> 00:55:08.440 +like the different ways that we do to actually build the things + +00:55:08.540 --> 00:55:13.860 +and install things and put them inside of the actual build application. + +00:55:14.220 --> 00:55:18.060 +There's a lot of sourcing that we do there, + +00:55:18.120 --> 00:55:20.840 +and Jonathan has been working on a lot of that. + +00:55:21.340 --> 00:55:23.380 +And there's also all the logic and all the stuff. + +00:55:23.500 --> 00:55:26.840 +We have a bunch of stuff on top of that to handle out of scaling, + +00:55:27.040 --> 00:55:31.160 +which is something that is actually not that easy to find in different providers. + +00:55:31.820 --> 00:55:37.220 +We have auto-scaling based on requests, including scaling down to zero, which saves costs. + +00:55:37.840 --> 00:55:40.080 +But this is not Lambdas. + +00:55:40.280 --> 00:55:41.340 +It's not AWS Lambdas. + +00:55:42.119 --> 00:55:46.860 +It's like the full deployed application, the full container or whatever it is, + +00:55:47.300 --> 00:55:49.900 +which is the full thing with all the dependencies. + +00:55:50.100 --> 00:55:54.060 +It's running for whenever it has to run, but we can scale based on requests. + +00:55:54.740 --> 00:56:07.280 +So I guess it's like the type of thing that you will have if you have this giant cluster for a huge enterprise with a bunch of infra people making sure everything just works perfectly. + +00:56:08.000 --> 00:56:10.320 +But you just pay us to do that for you. + +00:56:10.420 --> 00:56:15.360 +This is also a good time for us to probably say lots of stuff is coming and we're in private beta. + +00:56:15.540 --> 00:56:22.660 +And so you should sign up for the wait list so that you can get admitted and try out these very cool things we've been talking about. + +00:56:22.660 --> 00:56:22.840 +Absolutely. + +00:56:23.240 --> 00:56:29.020 +And I think I'll let Tech Insider out in the audience sort of lean into it. + +00:56:29.020 --> 00:56:29.520 +Public release one. + +00:56:30.019 --> 00:56:30.900 +Sebastian, when? + +00:56:31.180 --> 00:56:32.140 +Public release one. + +00:56:32.440 --> 00:56:34.960 +My final topic, which is just what's the roadmap? + +00:56:35.560 --> 00:56:36.320 +When is this stuff? + +00:56:36.660 --> 00:56:38.540 +Like, how do we get into it here? + +00:56:38.820 --> 00:56:40.180 +Why would they like Litestar? + +00:56:40.400 --> 00:56:43.940 +We have the, right now we have the waiting list and we are onboarding people. + +00:56:44.080 --> 00:56:46.460 +We already have like a bunch of people in the private beta. + +00:56:46.920 --> 00:56:51.720 +We're going to keep onboarding people from the waiting list and like, you know, like ramp that up. + +00:56:51.780 --> 00:56:56.060 +But it will be like, you know, like through the waiting list is the main place where we are onboarding. + +00:56:56.140 --> 00:56:58.440 +People will want to make sure that everything is super fine-tuned. + +00:56:58.820 --> 00:57:00.300 +And we're going to keep it that way for a while. + +00:57:00.540 --> 00:57:06.100 +So like people that are on the waiting list are going to be like the ones that are going to be able to start using it the soonest. + +00:57:06.420 --> 00:57:10.160 +At some point, we'll probably have ways for people to invite others and things like that. + +00:57:10.500 --> 00:57:17.920 +About the things that we are building, we want to, you know, like we are super focused on FastAPI and then Python in general. + +00:57:18.060 --> 00:57:20.880 +at some point will probably support different tools, + +00:57:21.340 --> 00:57:24.060 +different ways to run, also like Python code in general, + +00:57:24.480 --> 00:57:25.360 +probably different frameworks. + +00:57:25.900 --> 00:57:30.240 +It will also depend a lot on what the users are asking for, + +00:57:30.360 --> 00:57:33.300 +whether like the tools, the frameworks, the use cases, + +00:57:33.460 --> 00:57:34.700 +the things that they need to build. + +00:57:34.740 --> 00:57:37.860 +And like, we're going to evolve the platform and the system + +00:57:38.500 --> 00:57:40.780 +based on what people need out of it. + +00:57:40.800 --> 00:57:42.900 +We have like a GitHub repo where we have issues, + +00:57:43.060 --> 00:57:45.780 +but we also have like a Slack that once people are admitted, + +00:57:45.820 --> 00:57:50.760 +they can talk directly to us and that feedback is really, really valuable for shaping the roadmap + +00:57:50.970 --> 00:57:53.500 +and figuring out all the fun things you want us to support. + +00:57:53.560 --> 00:57:58.620 +Awesome. Of course, you're going to charge money for it. It runs on servers and you guys are not + +00:57:59.170 --> 00:58:04.640 +a charity, but can you give any sense of what you're thinking about that kind of stuff or + +00:58:05.060 --> 00:58:06.080 +join the waitlist and see? + +00:58:06.240 --> 00:58:11.920 +Well, first join the waitlist and see, but we don't have like that predefined yet, + +00:58:12.220 --> 00:58:16.080 +but they will be on the ballpark of what you could get from different cloud providers. + +00:58:16.280 --> 00:58:23.920 +So different similar-ish providers will be on the ballpark of what you will get. + +00:58:24.300 --> 00:58:26.980 +But it's not written in stone yet. + +00:58:27.320 --> 00:58:32.060 +It's still a little bit different because we can auto-scale based on requests. + +00:58:32.340 --> 00:58:36.580 +So we can increase the amount of replicas of your application automatically, + +00:58:36.960 --> 00:58:40.000 +and then we can decrease them automatically, and we can scale down to zero. + +00:58:40.060 --> 00:58:49.940 +So you can probably handle all the load that you need and in the end spend a lot less because you don't have to have a bunch of instances constantly running or things like that, you know. + +00:58:50.420 --> 00:58:55.840 +So it will probably work a little bit different than what it will be for other providers. + +00:58:56.360 --> 00:58:58.380 +But in the end, it should be roughly similar. + +00:58:58.560 --> 00:59:18.360 +Okay. And given the fact that you all handle so much of it as a platform as a service type of thing, you don't have to have a cloud expert on hand or a DevOps expert necessarily, right? As soon as a company hires somebody to be an AWS cloud architect or something like that, it's no longer just what is your AWS bill. + +00:59:18.580 --> 00:59:22.380 +It's also a little bit of pain that we are swallowing so you don't have to take it. + +00:59:22.740 --> 00:59:24.580 +Exactly. It's part of taking one for the team, right? + +00:59:24.740 --> 00:59:24.920 +Yes. + +00:59:25.140 --> 00:59:25.200 +Yes. + +00:59:27.160 --> 00:59:27.340 +Indeed. + +00:59:27.960 --> 00:59:30.960 +All right, so I had one or two things specifically + +00:59:31.610 --> 00:59:32.360 +that I was seeking. + +00:59:32.550 --> 00:59:33.880 +It's like custom domains. + +00:59:34.410 --> 00:59:35.580 +How far off are custom domains? + +00:59:35.700 --> 00:59:38.100 +I was like, oh, I could put some cool things on there. + +00:59:39.420 --> 00:59:41.140 +I could tell Jonathan is psyched about this. + +00:59:41.280 --> 00:59:44.320 +It'd be really fun to put one of my really small + +00:59:44.799 --> 00:59:46.100 +FastAPI projects over there, + +00:59:46.740 --> 00:59:48.360 +something I set up for some of my courses or something, + +00:59:48.520 --> 00:59:50.100 +and then I can point people to go, + +00:59:50.160 --> 00:59:51.400 +look, it's running on FastAPI Cloud. + +00:59:51.400 --> 00:59:53.620 +How neat, you guys can check that out over there. + +00:59:54.060 --> 00:59:55.360 +And I'm like, but it's on its own domain, + +00:59:55.580 --> 01:00:00.380 +that domain is baked into the course videos, you know what I mean? And it's written in stone. + +01:00:00.580 --> 01:00:01.140 +It's marketing. + +01:00:01.520 --> 01:00:07.880 +Yeah, exactly. So I can't really move it because it has, you know, some subdomain of Talk Python, + +01:00:08.060 --> 01:00:08.120 +right? + +01:00:08.240 --> 01:00:12.120 +I was working on it. And then I got the notification by Google Calendar that I should + +01:00:12.440 --> 01:00:14.140 +join a certain podcast. So... + +01:00:14.140 --> 01:00:18.920 +Are you telling me we don't have custom domains? Because I'm here asking you about custom domains. + +01:00:19.100 --> 01:00:19.700 +How meta is that? + +01:00:19.720 --> 01:00:23.540 +You got it. It could be here already. But no, you have to wait a bit more. + +01:00:23.640 --> 01:00:24.400 +Okay. But soon? + +01:00:24.520 --> 01:00:29.660 +Yeah. As soon as broad enough, but I'm actively working on it. Let's put it like that. + +01:00:29.740 --> 01:00:34.320 +Okay. That sounds great. And then, I mean, just, it's never simple. You know, I just, + +01:00:34.780 --> 01:00:39.140 +I set up some stuff and it's like, you get the pop-up. Oh, you got to put this, you know, + +01:00:39.280 --> 01:00:45.320 +this TXT record or this CNAME or whatever record into your DNS and then we're checking it. Oh, + +01:00:45.780 --> 01:00:50.239 +it might take three days for your DNS to propagate. So hang in there and just, I can imagine like + +01:00:50.560 --> 01:00:56.560 +you're having fun yeah i guess you're kidding me that's like wow i i thought i'm almost off work + +01:00:56.780 --> 01:01:00.260 +but no you're bringing it all back but yeah that's a that's a thing i'm sure the company + +01:01:00.760 --> 01:01:04.540 +could support therapy to like work work through the issues and the trauma that you've suffered + +01:01:04.760 --> 01:01:10.340 +from the dns it's always dns that's right i mean you got that's an yes it's always dns yes + +01:01:10.460 --> 01:01:15.699 +i guess one of our goals with custom domains also to make it super simple for you to set up them + +01:01:15.940 --> 01:01:23.040 +Like, for example, if you're using one of the providers that support OAuth, we can also just do one click and then it's going to be automatically. + +01:01:23.160 --> 01:01:24.960 +Oh, that's cool. Yeah, that's really nice. + +01:01:25.120 --> 01:01:27.300 +But unfortunately, it depends on the platform you're using. + +01:01:27.460 --> 01:01:29.300 +All of them support this. + +01:01:29.480 --> 01:01:32.720 +This is said by the person in charge of most of the integrations. + +01:01:32.860 --> 01:01:37.340 +So Patrick has built, we have integrations for a bunch of database providers and things like that. + +01:01:38.060 --> 01:01:41.900 +I think now Patrick knows by memory, open ID specification. + +01:01:42.260 --> 01:01:42.720 +I don't know. + +01:01:44.300 --> 01:01:48.600 +Yeah, the other thing I wanted to talk a bit about was just integrations, like what kind of stuff you guys have coming in. + +01:01:48.600 --> 01:01:51.940 +I saw that Hugging Face is going to be integrated soon. + +01:01:52.120 --> 01:01:55.320 +You've got Supabase, which is kind of Postgres as a service. + +01:01:56.040 --> 01:01:59.860 +There's a lot of those things out there that theoretically could be added. + +01:02:00.100 --> 01:02:01.440 +Someone also asked for MongoDB. + +01:02:01.660 --> 01:02:03.640 +Maybe that's one that we're going to take a look into. + +01:02:04.070 --> 01:02:06.320 +It really depends on the provider. + +01:02:06.370 --> 01:02:11.360 +So at the moment, we don't want to ask databases for you because that's also another kind of rabbit hole. + +01:02:11.780 --> 01:02:13.200 +Jonathan is probably not ready for that. + +01:02:14.180 --> 01:02:15.880 +But yeah, definitely database. + +01:02:16.040 --> 01:02:21.260 +But I guess we can say that we're also talking with the people from Pydantic + +01:02:21.360 --> 01:02:24.720 +so we can integrate maybe Logfire automatically, that kind of stuff. + +01:02:25.120 --> 01:02:25.220 +Yeah. + +01:02:25.440 --> 01:02:28.600 +And also things like Redis, which is also another kind of database. + +01:02:29.300 --> 01:02:30.200 +That's also coming soon. + +01:02:30.360 --> 01:02:33.580 +Yeah, there's a couple of database as a service type things + +01:02:33.740 --> 01:02:37.700 +that don't require too much other than just connecting API keys + +01:02:37.920 --> 01:02:38.740 +and something like that, right? + +01:02:38.940 --> 01:02:40.180 +Those seem like low-hanging fruit. + +01:02:40.300 --> 01:02:43.940 +Like the kind of goal with the integration is not just done. + +01:02:44.000 --> 01:02:46.620 +Like, yeah, right now it's just setting up an environment variable. + +01:02:47.080 --> 01:02:51.240 +But the idea is also to more, I don't know, like the proper integration, I would say. + +01:02:52.320 --> 01:02:56.040 +Like, for example, for things like Superbase, if, yeah, I think that's support branching. + +01:02:56.080 --> 01:03:00.500 +Like, for example, once we support ProQuest Previews for GitHub, like we can also create + +01:03:00.500 --> 01:03:03.680 +a branch automatically for you if you have the Superbase integration enabled. + +01:03:04.180 --> 01:03:05.920 +And we can do this kind of stuff as well. + +01:03:06.020 --> 01:03:09.620 +Or even we could show like some information about database. + +01:03:10.000 --> 01:03:12.200 +I don't know, like load or like memory usage, + +01:03:12.320 --> 01:03:13.920 +things are directly from our dashboard. + +01:03:14.080 --> 01:03:14.860 +So you don't have to go there. + +01:03:15.080 --> 01:03:17.920 +That's the main reason why we're building this infrastructure + +01:03:18.180 --> 01:03:19.160 +for the integration. + +01:03:19.480 --> 01:03:21.620 +Well, people can sign up to the waiting list + +01:03:21.820 --> 01:03:24.420 +and hopefully get on the private beta. + +01:03:24.880 --> 01:03:26.180 +We actually check the waiting list. + +01:03:26.220 --> 01:03:29.080 +We actually check the use cases, team sizes, + +01:03:29.440 --> 01:03:31.440 +like what are people building with it? + +01:03:31.540 --> 01:03:33.400 +Like we actually go and check it + +01:03:33.480 --> 01:03:36.580 +and we bring in people from the waiting list. + +01:03:37.020 --> 01:03:39.220 +Nice. You know, I didn't join the waiting list directly. + +01:03:39.460 --> 01:03:42.820 +I was added by some guy I know who was very kind + +01:03:42.850 --> 01:03:44.560 +to help me get some behind-the-scene look. + +01:03:45.200 --> 01:03:46.360 +So I don't know what the process is. + +01:03:46.460 --> 01:03:48.560 +Do you actually say what you want to do with it? + +01:03:48.590 --> 01:03:51.360 +And you evaluate that a little bit as well based on, like, + +01:03:51.360 --> 01:03:53.340 +hey, this would be a cool use case for us to support? + +01:03:53.430 --> 01:03:54.960 +There are many types of applications + +01:03:55.150 --> 01:03:57.320 +and many types of different team sizes, + +01:03:57.600 --> 01:04:00.520 +many types of things that people might want to build. + +01:04:00.720 --> 01:04:03.020 +And we try to see, like, okay, where is a case + +01:04:03.160 --> 01:04:06.360 +where we could be a good fit and we can provide a great service? + +01:04:06.980 --> 01:04:08.880 +And where are the things that people are trying to build? + +01:04:09.220 --> 01:04:14.860 +Also, it also helps us see, like, you know, like, what are people trying to do with Fasted + +01:04:14.980 --> 01:04:17.080 +Vehicle so that we know what we have to provide? + +01:04:17.560 --> 01:04:23.800 +But we actually go and check those, you know, like those submissions on, like, sexually thousands + +01:04:24.010 --> 01:04:30.480 +of people in the waiting list, but we still go and check and approve kind of manually still + +01:04:30.780 --> 01:04:34.840 +to bring a bunch of people on board in the different ways that we have been bringing people. + +01:04:35.240 --> 01:04:39.180 +So if people go and join the waiting list and actually tell us what they are, what is their + +01:04:39.200 --> 01:04:41.900 +use case, their team, what are they planning on doing? + +01:04:42.440 --> 01:04:45.160 +There's a much higher chance that we are going to go + +01:04:45.340 --> 01:04:46.280 +on to bring them up. + +01:04:46.280 --> 01:04:46.500 +MARK MANDEL: Awesome. + +01:04:47.100 --> 01:04:49.280 +So everyone, go join the waitlist. + +01:04:49.360 --> 01:04:52.980 +If you're doing FastAPI, I'll link to it in the show notes, + +01:04:53.080 --> 01:04:53.420 +of course. + +01:04:53.860 --> 01:04:57.040 +Thank you all for being here and sharing the story. + +01:04:57.340 --> 01:05:00.900 +And I, for one, am very excited to see FastAPI Cloud exist + +01:05:01.000 --> 01:05:04.180 +and just one more way to make FastAPI stronger + +01:05:04.420 --> 01:05:06.400 +and more resilient and so on. + +01:05:06.400 --> 01:05:07.340 +FRANCESC CAMPOY: Thank you very much. + +01:05:07.580 --> 01:05:08.400 +Thank you for having us. + +01:05:08.580 --> 01:05:09.480 +Yeah, it's super fun. + +01:05:09.480 --> 01:05:10.260 +Thanks for having us. + +01:05:10.460 --> 01:05:10.840 +Yeah, you bet. + +01:05:11.220 --> 01:05:11.500 +Bye, everyone. + +01:05:11.640 --> 01:05:12.080 +Bye, folks. + +01:05:12.380 --> 01:05:12.680 +Bye-bye. + +01:05:12.940 --> 01:05:13.060 +Bye. + +01:05:14.300 --> 01:05:16.540 +This has been another episode of Talk Python To Me. + +01:05:16.820 --> 01:05:17.620 +Thank you to our sponsors. + +01:05:17.860 --> 01:05:19.120 +Be sure to check out what they're offering. + +01:05:19.340 --> 01:05:20.700 +It really helps support the show. + +01:05:21.560 --> 01:05:23.720 +This episode is brought to you by CommandBook, + +01:05:23.980 --> 01:05:26.080 +a native macOS app that I built + +01:05:26.240 --> 01:05:28.780 +that gives long-running terminal commands a permanent home. + +01:05:29.180 --> 01:05:31.180 +No more juggling six terminal tabs every morning. + +01:05:31.620 --> 01:05:33.040 +Carefully craft a command once, + +01:05:33.300 --> 01:05:34.700 +run it forever with auto-restart, + +01:05:34.900 --> 01:05:36.420 +Ural detection, and a full CLI. + +01:05:36.880 --> 01:05:39.940 +Download it for free at talkpython.fm/command book app. + +01:05:41.040 --> 01:05:43.660 +And it's brought to you by the Talk Python in Production Book, + +01:05:44.190 --> 01:05:48.820 +an inside look at 10 years of the real-world DevOps behind the Talk Python sites and apps. + +01:05:49.160 --> 01:05:51.860 +Check it out at talkpython.fm/DevOps book. + +01:05:52.720 --> 01:05:54.580 +If you or your team needs to learn Python, + +01:05:54.790 --> 01:05:59.620 +we have over 270 hours of beginner and advanced courses on topics ranging from + +01:05:59.960 --> 01:06:04.860 +complete beginners to async code, Flask, Django, HTML, and even LLMs. + +01:06:05.140 --> 01:06:07.360 +Best of all, there's no subscription in sight. + +01:06:08.020 --> 01:06:09.680 +Browse the catalog at talkpython.fm. + +01:06:10.420 --> 01:06:12.360 +And if you're not already subscribed to the show + +01:06:12.600 --> 01:06:13.740 +on your favorite podcast player, + +01:06:14.400 --> 01:06:15.040 +what are you waiting for? + +01:06:15.720 --> 01:06:17.480 +Just search for Python in your podcast player. + +01:06:17.640 --> 01:06:18.460 +We should be right at the top. + +01:06:18.880 --> 01:06:20.420 +If you enjoy that geeky rap song, + +01:06:20.500 --> 01:06:21.720 +you can download the full track. + +01:06:21.860 --> 01:06:23.760 +The link is actually in your podcast blur show notes. + +01:06:24.600 --> 01:06:25.920 +This is your host, Michael Kennedy. + +01:06:26.320 --> 01:06:27.380 +Thank you so much for listening. + +01:06:27.600 --> 01:06:28.400 +I really appreciate it. + +01:06:28.820 --> 01:06:29.540 +I'll see you next time. + +01:06:41.180 --> 01:06:43.980 +I'm out. + diff --git a/transcripts/537-datastar.txt b/transcripts/537-datastar.txt new file mode 100644 index 0000000..5f49e54 --- /dev/null +++ b/transcripts/537-datastar.txt @@ -0,0 +1,2588 @@ +00:00:00 You love building web apps with Python, and HTMX got you excited about the hypermedia approach. + +00:00:05 Let the server drive the HTML, skip the JavaScript build step, keep things simple, right? + +00:00:11 But then you hit that last 10%. You need AlpineJS for interactivity, or your state gets out of sync, + +00:00:16 and suddenly you're juggling two unrelated libraries that weren't really designed to work + +00:00:20 together. What if there was a single 11-kilobyte framework that gave you everything HTMX and + +00:00:26 AlpineJS did, and more with real-time updates, multiplayer collaboration out of the box, + +00:00:31 and performance so fast, you're actually bottlenecked by your monitor's refresh rate. + +00:00:37 That's Datastar. + +00:00:38 On this episode, I sit down with its creator, Delany Galan, core maintainer, Ben Crocker, + +00:00:43 and Datastar convert, Chris May, to help explore how this backend-driven, service-sent event-first + +00:00:50 framework is changing the way full-stack developers think about the modern web. + +00:00:55 This is Talk Python To Me, episode 537, recorded January 15th, 2026. + +00:01:19 Welcome to Talk Python To Me, the number one Python podcast for developers and data scientists. + +00:01:24 This is your host, Michael Kennedy. + +00:01:26 I'm a PSF fellow who's been coding for over 25 years. + +00:01:30 Let's connect on social media. + +00:01:32 You'll find me and Talk Python on Mastodon, BlueSky, and X. + +00:01:35 The social links are all in your show notes. + +00:01:38 You can find over 10 years of past episodes at talkpython.fm. + +00:01:42 And if you want to be part of the show, you can join our recording live streams. + +00:01:45 That's right. + +00:01:46 We live stream the raw uncut version of each episode on YouTube. + +00:01:50 Just visit talkpython.fm/youtube to see the schedule of upcoming events. + +00:01:54 Be sure to subscribe there and press the bell so you'll get notified anytime we're recording. + +00:01:59 This episode is brought to you by Sentry. + +00:02:01 Don't let those errors go unnoticed. + +00:02:02 Use Sentry like we do here at Talk Python. + +00:02:04 Sign up at talkpython.fm/sentry. + +00:02:08 And it's brought to you by CommandBook, a native macOS app that I built that gives long-running + +00:02:13 terminal commands a permanent home. + +00:02:15 No more juggling six terminal tabs every morning. + +00:02:18 Carefully craft a command once, run it forever with auto restart, URL detection, and a full CLI. + +00:02:23 Download it for free at talkpython.fm/command book app. + +00:02:27 Ben, Delaney, Chris, welcome to you all. + +00:02:30 Thanks for being here on Talk Python To Me. + +00:02:31 Thanks for having us. + +00:02:32 Hey, how are you doing? + +00:02:33 Doing well, doing well. + +00:02:34 Very excited to talk about Datastar and some cool web frameworks for Python people and beyond, of course. + +00:02:42 But, you know, most people listening doing Python web frameworks. + +00:02:45 So talk about how that all integrates. + +00:02:47 And if you like the HTMX vibe, which we've talked a lot about on the show, I think there's + +00:02:52 going to be a lot to like here as well. + +00:02:54 And maybe more. + +00:02:55 We'll see. + +00:02:55 A case to be made. + +00:02:57 But, you know, before we get into all of that, though, let's just talk about a quick introduction + +00:03:02 for everyone here and like go around the squares of Ben, I'll let you go first. + +00:03:07 Who are you, Ben? + +00:03:08 Based in Costa Rica at the moment. + +00:03:10 I'm based in Europe most of the year, but half of the year my wife and I spend here. + +00:03:14 In terms of background, I've been primarily working with PHP for well over 20 years and got involved with Delaney and Datastar, been a core maintainer on that project ever since. + +00:03:26 And I looked at my commit history for last year, and it turns out now I write more Go code than PHP, so I don't want to call myself a PHP developer anymore. + +00:03:37 I'm just a web developer, a backend web developer, primarily that also writes TypeScript and maintains a front end. + +00:03:44 framework. There's a lot of stuff going on and ways in which you can write code for the web these days. + +00:03:50 Well, thanks. Awesome to have you here. Delaney, hello. + +00:03:53 Hi, how you doing? Yeah, I have kind of a weird checkered background into web development. I was + +00:03:58 originally in the circus, then I became a 3D artist, then I became an engineer. I've worked in + +00:04:04 games, video games, slot machines, military applications, all kinds of crazy things. + +00:04:11 I tend to work on really highly optimized, fast things. + +00:04:14 I love the ideas of the web, but I got really tired of how you actually implement things in that. + +00:04:20 And I was doing very large applications with millions of updates a second. + +00:04:24 And the tools that were out there just weren't good enough. + +00:04:27 So I ended up going down many, many rabbit holes and finally found something to make it better for everybody else. + +00:04:33 So yeah, that's really cool. + +00:04:34 And wow, what a really interesting issue. + +00:04:36 I know you got some crazy stories. + +00:04:38 Yes, I do. + +00:04:39 I always have a funny, weird outcome of something. + +00:04:42 Ironically, people talk about things being a circus, + +00:04:45 but like circuses are very well run logistic machines + +00:04:48 compared to most developer situations. + +00:04:50 So it's kind of funny. + +00:04:50 Yeah, it's an insult to circuses. + +00:04:52 Yes, it is. + +00:04:52 It really is. + +00:04:54 Amazing. + +00:04:55 Okay. + +00:04:55 And what we're going to talk about, Datastar has this amazing ability to update many things + +00:05:02 really quickly in real time, which we'll get into, but yeah, sort of foreshadowing there. + +00:05:07 And Chris May, welcome to the show. + +00:05:10 I've known you for a long time and I'm really happy to have you here. + +00:05:12 Great to be here. Thank you so much. + +00:05:14 Yeah. + +00:05:14 Yeah. So about me, I started writing websites back in 1995 and then picked up Python about 10 or so years later and just have really enjoyed the ride since then. + +00:05:25 Picked along the way, became technical coach and just loved making single page applications. + +00:05:30 I loved, I just love the web. + +00:05:32 You know, I love that we can publish something from our computer and anybody around the world can see it. + +00:05:36 And then what, maybe a little over a year ago, I, oh no, it was more than that. + +00:05:40 I remember I was on a trip and I was listening to a podcast of HXPod, the HTMX podcast, + +00:05:46 and heard about this crazy, cool tool, Datastar. + +00:05:50 And I was like, I even put in my DjangoCon presentation, like you should, everybody else + +00:05:54 should try it out. + +00:05:55 And finally I did and I'm converted. + +00:05:56 I love it. + +00:05:57 So I'm excited that the three of us get to talk about it. + +00:06:00 The reason that we're having this podcast is because I read your article about switching + +00:06:05 to Datastar. And I'm like, okay, this is interesting. You made the case very well. Of + +00:06:10 course, I'll link to the article. And so I thought, hey, I need to have Chris here as my Tony Romo to + +00:06:18 my Al Michaels or Nico Rosberg to my Crofty or whatever, right? So I'm happy to have you here. + +00:06:24 Exactly. Awesome to have you here. So let's just start with what is Datastar, right? I mean, + +00:06:30 we've hinted that it has some similarities to htmx but also not so ben and delaney give us the + +00:06:37 overview what is datastar so i can give a little bit of history and then ben's probably better at + +00:06:42 saying what it is now i have a background in like low-level stuff um even though i was a 3d artist + +00:06:47 first i'm much more comfortable in like shader development and that kind of thing like so glsl + +00:06:52 web thing like i'm a c guy that knows some other things but the thing is that i was working on + +00:06:57 some military applications where I needed really fast updates of a browser. And the reason why you + +00:07:03 in this military situation is that getting things approved is really hard, like executables to go + +00:07:09 into deployment. But having a browser means that you have this nice little sandbox that things can + +00:07:13 go in. So it's actually more of a deployment platform in my background than, you know, just + +00:07:17 the regular web. But I was doing things that were pushing the browser really, really far. I was using + +00:07:21 Vue and Spa. And I basically was like, well, these are the smartest people out here, but it's not + +00:07:26 fast enough. So I was using crazy WebSocket stuff, all this binary stuff. And then I tried doing, + +00:07:31 you had someone on last week talking about LiveView and like they have a Python version of that. I went + +00:07:37 hard in making a binary version of that, like going down to the protocol level, changing, + +00:07:41 optimizing that 10 different ways. I had an entire framework for doing this. And basically, + +00:07:45 in my opinion, that's a complete dead end. It is untenable. We can go into the reasons why, + +00:07:49 but the thing is, long story short, I ended up seeing what was happening in HTMLX in the hyper + +00:07:53 media space. And I completely discounted all of that because I said, like, I'm doing low level + +00:07:58 binary stuff. There's no way this other approach can be faster. And then my thing is always check + +00:08:02 the metrics, always don't take your assumptions and do the work. And the thing is, there's things + +00:08:07 that are wrong in the implementation, but there's things that are 100% right in the overall ideas of + +00:08:11 how to use that. So I went and I took a year and a half work and just threw it in the trash + +00:08:16 and said, okay, I'm starting over and like ended up doing some basic things would probably get into + +00:08:22 and ended up with this thing that is a backend diagnostic backend framework that has a 10 + +00:08:27 kilobyte shim that is the fastest, smallest thing out there by orders of magnitude. So it's not just + +00:08:33 a slightly different thing. It is literally a different paradigm shift. It's a crazy shift. + +00:08:37 So the difference between reacts to something like HTMX is different from HTMX to the data + +00:08:42 start way. So I'll let Ben actually explain what that is. But the thing is from a low level C + +00:08:46 guys point of view, it is one of the fastest things in your stack now, which is crazy to think + +00:08:51 Yeah, like a 10K shim can do that. + +00:08:53 That's incredible. + +00:08:54 And also, it sounds like your advice comes from somebody who's done a lot of profiling. + +00:08:59 Very much so. + +00:08:59 Like, that's the only thing. + +00:09:00 You got to measure, not guess. + +00:09:03 Yeah. + +00:09:03 In fact, there's funny things that we've had things on Twitter fighting with people and + +00:09:06 they're like, oh, this one situation was really slow. + +00:09:09 We actually looked at their flangraphs and it was a bug in the Safari GPU stuff. + +00:09:14 Because we were actually at the level where the JavaScript doesn't even show up. + +00:09:17 It's actually a GPU issue of it rendering fast stuff. + +00:09:20 in the browser, nothing to do with the JavaScript. + +00:09:23 Because the fastest JavaScript you can write is no JavaScript. + +00:09:25 So we really lean into what the browser can already do. + +00:09:28 And we're just making it so that that's easy to do so that the average person with the average website + +00:09:32 doesn't have to write any JavaScript at all. + +00:09:34 And they get to be a full stack developer in whatever language they choose. + +00:09:37 And I'll let everyone else talk from there. + +00:09:40 Awesome. Ben? + +00:09:41 Yeah, my version is going to be quite different to Delaney's + +00:09:43 because we care about different things. + +00:09:45 Fortunately, we do care about some of the same things. + +00:09:49 We work well together because I think we complement each other. + +00:09:52 But coming from a PHP background, I want the backend to be driving the front end. + +00:09:57 And it naturally does, right? + +00:09:58 Because even your HTML is being produced by your backend. + +00:10:02 And that's what's being served to the front end. + +00:10:05 I describe Datastar as a hypermedia framework. + +00:10:09 And some people get tripped up on what hypermedia is, but it's essentially hypertext with other media like images and CSS and that kind of thing. + +00:10:16 And everybody should know what hypertext is because it's the H in HTTP and HTML. + +00:10:22 There is an expectation for people coming into Datastar that you have a basic understanding of the web and web browsers and the web browser API because we lean as heavily as possible on the browser API. + +00:10:34 We get a lot of people coming into the Discord asking us, you know, how should I do this the Datastar way? + +00:10:39 And it got to the point where I'd heard that question so often I decided, OK, I'm going to write a page in the Datastar docs. + +00:10:45 We call it the tau of datastar. + +00:10:47 So it's kind of like the way of datastar. + +00:10:49 And if there's one thing to take from that, it's use as little datastar as possible. + +00:10:54 Like leverage the browser, because the browser is an incredible thing, right? + +00:10:58 Like it's basically an operating system, our operating system as web developers. + +00:11:02 So, and everything happens at the C level, super optimized. + +00:11:06 We're not going to be able to build something faster. + +00:11:08 So leverage the browser as much as possible on the browser APIs. + +00:11:12 And where HTML kind of lacks or where there are some gaps, that's essentially what Datastar is trying to fill. + +00:11:19 So I did a lot of work. + +00:11:21 So just to relate this, I guess, to something that other people might be familiar with, which is HTMLX. + +00:11:27 I was an early contributor to HTMLX, actually, and I was sold on the idea of hypermedia from the very beginning. + +00:11:33 So HTML is the language of the web. + +00:11:36 Why are we trying to replace it with JavaScript? + +00:11:39 And the problem that I ran into after several years of thinking HTMX is all I need is that last 10%, right? + +00:11:46 Because it'll get you 90% of what you're trying to do. + +00:11:50 But that last 10%, which we all know is the hardest piece that takes the most work, just isn't covered. + +00:11:56 So with HTMX, for example, you will very often reach for another library like AlpineJS, + +00:12:02 or you'll start writing vanilla JS perhaps to fill in those gaps to interactivity to the page, + +00:12:09 because HTMX is really just going to the back end, replacing the DOM. + +00:12:13 But now you have two dependencies. + +00:12:15 Now you have HTMX and Alpine, for example, and you're trying to make those play well together. + +00:12:20 And because I think that might be a little bit of the missing sauce from HTMX. + +00:12:24 I've had Carson Gross on and I really admire HTMX. + +00:12:28 But as I've worked with it over a couple of years, I feel like it's really good as salt or seasoning, + +00:12:35 something you sprinkle on to really make a website better. + +00:12:38 But if you try to make a meal out of salt, you're not going to want to eat it. + +00:12:41 And what I mean is, you have three different disjointed parts of the page, + +00:12:47 and you're like, this is so amazing to update this with HTML and partials, and so is that. + +00:12:51 But then you start talking about AlpineJS and connecting different things, + +00:12:56 and then the JavaScript gets out of sync with this server response. + +00:12:59 And it just, you start to feel constrained by it. + +00:13:03 And I think you all have a really nice solution. + +00:13:05 It's something a little bit like how you, we're going to talk about it, + +00:13:08 but sort of how you specify the HTML to be updated by the server, + +00:13:13 but then also connecting different parts of the pages. + +00:13:16 Chris put it in his article that like the problem is AlpineJS and HTMX + +00:13:20 are just two unrelated different things that happen to go together a lot. + +00:13:24 And so they're not cohesive in a sense, right? + +00:13:26 Well, and that's one thing that's definitely an issue. + +00:13:28 Like, for example, this was my thing because I actually tried to fix HTMX back in the day. + +00:13:33 And like the things that I wanted to fix were the problem that I see at least + +00:13:37 is that you have HTMX, you can add, it has extensions, so you can add stuff to it. + +00:13:41 But it fundamentally was built to be like, here's our way of doing it. + +00:13:45 And then you can do your own stuff on top of it. + +00:13:47 The problem is, is that I thought that's broken. + +00:13:51 I've done enough game development to know that you need to be agile. + +00:13:53 I need to be able to like be able to move quickly. + +00:13:55 So I wanted it so that nothing was basically like the core of data star is like 300 lines + +00:14:01 long. + +00:14:01 And it is basically setting up data dash star elements, hooking up plugins, and then everything + +00:14:06 else is a plugin. + +00:14:07 So if you don't agree with us, or if someone's better than I am, great, that's wonderful. + +00:14:12 We will be able to just pop that part out, put the new part in. + +00:14:15 But plugins can now depend on each other. + +00:14:17 They can understand. + +00:14:17 It's an ecosystem. + +00:14:18 Ironically, that's what happens under the hood. + +00:14:21 But the ideas of that make it so much more powerful. + +00:14:23 And the irony is that if you build it in that kind of plugin style way, in the more game developer style way, we are smaller than HTMLX and Alpine alone, let alone combined, let alone Hyperscript and all these other things. + +00:14:34 So it's just a different way of thinking about the problem. + +00:14:36 When I first encountered Datastar and looked at the source code, it looked very foreign to me because Delaney coming from game development, he built Datastar like a game engine. + +00:14:47 So you have this very thin core and then everything else pretty much is a plugin. + +00:14:52 And all Datastar core is a way for registering plugins and having Datastar attributes. + +00:14:59 And that's pretty much it. + +00:15:00 Everything else is an add-on that you, is a plugin that you can take away. + +00:15:03 So we even have a bundler on the site that allows you to just, well, you can just download + +00:15:08 Datastar core or you can just select what plugins you want. + +00:15:12 Now, that in and of itself is not that interesting because we're, at the end of the day, we're + +00:15:16 talking about a 10 kilobyte JavaScript file with all of the plugins. + +00:15:19 But it is open source, which we didn't mention. + +00:15:21 And so anybody can go just kind of look at it if you're interested. + +00:15:25 But that approach means that everything is modular and everything is there for a reason. + +00:15:30 And we'll get into this later, I guess. + +00:15:32 But like deciding what plugins go in and what stay out is one of the challenges. + +00:15:36 And we just try to keep it as lean as possible. + +00:15:39 My way of thinking about it is that Datastar gives you everything you need and nothing you don't. + +00:15:44 And that's how we try to kind of keep it lean and fast. + +00:15:48 This portion of Talk Python Maze brought to you by Sentry. + +00:15:51 I've been using Sentry personally on almost every application and API that I've built for + +00:15:56 Talk Python and beyond over the last few years. + +00:15:59 They're a core building block for keeping my infrastructure solid. + +00:16:03 They should be for yours as well. + +00:16:04 Here's why. + +00:16:05 Sentry doesn't just catch errors. + +00:16:07 It catches all the stuff that makes your app feel broken. + +00:16:10 The random slowdown, the freeze you can't reproduce, that bug that only shows up once + +00:16:14 real users hit it. + +00:16:15 And when something goes wrong, Sentry gives you the whole chain of events in one place. + +00:16:19 errors, traces, replays, logs, dots connected. + +00:16:22 You can see what's led to the issue without digging through five different dashboards. + +00:16:27 Seer, Sentry's AI debugging agent, builds on this data, taking the full context, + +00:16:32 explaining why the issue happened, pointing to the code responsible, drafts a fix, + +00:16:37 and even flags if your PR is about to introduce a new problem. + +00:16:41 The workflow stays simple. + +00:16:43 Something breaks, Sentry alerts you, the dashboard shows you the full context. + +00:16:47 Seer helps you fix it and catch new issues before they ship. It's totally reasonable to go from an error occurred to fixed in production in + +00:16:55 just 10 minutes. I truly appreciate the support that Sentry has given me to help solve my bugs + +00:17:01 and issues in my apps, especially those tricky ones that only appear in production. I know you will + +00:17:06 too if you try them out. So get started today with Sentry. Just visit talkpython.fm/sentry + +00:17:12 and get $100 in Sentry credits. Please use that link. It's in your podcast player show notes. If + +00:17:19 our code talkpython26, all one word talkpython26 to get $100 in credits. + +00:17:26 Thank you to Sentry for supporting the show. + +00:17:29 Cool. + +00:17:29 That's a super interesting philosophy to say you should be able to take, even take + +00:17:33 stuff out of what we're giving you by default, right? + +00:17:35 Now, before we move on from sort of introducing Datastar, I do want to point out at data-star.dev, + +00:17:42 which of course I'll link this notes, there's some cool examples on here. + +00:17:45 You've got a really nice space 2001 sort of theme with Hal and all that, which is great. + +00:17:53 I like the aesthetic here, which is very fun. + +00:17:56 It's got a little bit of a retro gaming feel, which is nice. + +00:18:00 But what I want to point out is I want to encourage people to go watch your little video. + +00:18:03 Your video is fun. + +00:18:05 It's really fun. + +00:18:06 This video is all about how Datastar fits in the world of SPAs. + +00:18:11 And one thing we didn't really mention is that Datastar is a full-fledged SPA replacement. + +00:18:17 So again, like that last 10%, often people will think, oh, well, I need to go to React or Vue.js or some single page application framework. + +00:18:26 Whereas we're saying that, no, no, no, Datastar will not only, it's not like a subset or like SPAs are not a superset. + +00:18:34 it's on the contrary. I think Datastark, we think Datastark can do more than SBAs because we are + +00:18:42 driven by the backend and we are focused on hypermedia, which is the language of the web. + +00:18:46 So this, yeah, so this video is kind of throwing, yeah, anyway, everybody should watch it. + +00:18:51 I'd also like to, if you can scroll back up to the top of the page, the Starfield animation was + +00:18:57 one of the things like when Delaney and when everybody who worked on this published it, + +00:19:02 Like I didn't realize how amazing this was because if you like right click and inspect that thing, + +00:19:07 it's a web component. + +00:19:09 And so all the JavaScript that's required for making all the stars go faster and slower + +00:19:13 and tracking your mouse where, you know, wherever you do it, + +00:19:16 it's all within that web component. + +00:19:18 And data star is essentially subscribing to like, where's the mouse pointer + +00:19:22 and passing it into the web component. + +00:19:24 Yeah, in fact, if you go to more examples, you will see that there's, + +00:19:29 and then go scroll down to, or use the hamburger thing. + +00:19:32 Yeah, go down to the rocket. + +00:19:35 There's the actual star field. + +00:19:38 So you can see the entire, the star field, the entire component is there. + +00:19:41 So if you scroll down from there, you'll see how it actually gets hooked up + +00:19:45 and the entire component, that's the whole thing, it's right there. + +00:19:48 - That's incredible. + +00:19:49 - And the thing is if you start moving around, like if you scroll up just a little bit more, + +00:19:52 so you can see the sliders, you'll see that they're live, everything's, + +00:19:56 if you move it around, like you move your mouse around the canvas, + +00:19:59 you'll see everything's live editing, everything's thing. + +00:20:02 It's the irony of Datastar. + +00:20:03 And this is the part that I don't think people quite get. + +00:20:06 And it's not that you're trying to like, we love what Carson has done with HCMS. + +00:20:10 We love that all the things they've done, but it does not do everything. + +00:20:13 It doesn't do enough. + +00:20:14 It is a library, not a framework. + +00:20:15 And the thing is, the irony is that Datastar actually has + +00:20:18 the fastest reactive signal, like reactive signals. + +00:20:22 We are the fastest thing out there. + +00:20:23 So it's not just like we did something that's kind of like VDOM, + +00:20:26 or we are like, we can compete with React. + +00:20:28 We demolish them with actual numbers. + +00:20:30 So we have the fastest morphing strategy and we also have the fastest signals, which means doing these kinds of things. + +00:20:35 It's just a non-issue. + +00:20:36 Like this star field thing is 1K. + +00:20:38 Like it's just these are the kinds of things that are just a non-issue in this if you do things our way. + +00:20:42 And you're leaning into the web ecosystem by leveraging web components instead of having to like build, have a build time pipeline to, you know, do all the custom JavaScript. + +00:20:53 Like once I realized like you can do these things, it just made, it just clicked. + +00:20:57 And I just make it's I feel like it's so much more fun now to work on the web now that I understand these things. + +00:21:04 Let's talk through some of the core examples. + +00:21:06 I feel like there's some similarities to the example section of the HTMX place. + +00:21:11 But, you know, HTMX doesn't have a star field, certainly. + +00:21:14 Best place to start is on the homepage. + +00:21:17 Before we get into those examples, just just to kind of take a step back and say, OK, we've mentioned HTMX a few times and we don't we don't even like to compare ourselves to HTMX. + +00:21:26 but it is a good maybe starting point for some people. + +00:21:29 We have a hello world example there, if you could find that. + +00:21:32 Yeah, let's scroll down just a little bit more. + +00:21:33 Yeah, you got it. + +00:21:34 One of the maybe differences between HTMX and Datastar is that Datastar can receive HTML responses, + +00:21:41 but it also by default, or the recommendation is to use server sent events. + +00:21:46 So if you hit start there, you're going to see kind of the network response tab, + +00:21:50 and those are server sent events. + +00:21:51 And SSE server sent events are an old technology that work just over HTTP. + +00:21:57 And essentially what happens is that the server holds a connection open to the browser + +00:22:02 and it's unidirectional. + +00:22:03 So you send a request to the server and then the server can stream events back down, + +00:22:08 which is what you're seeing here. + +00:22:10 Now, this is obviously a trivial example, right? + +00:22:12 We're sending one, or we're updating the message one character at a time. + +00:22:16 But when you see how simple this is, then you can perhaps see potential for this, right? + +00:22:22 And SSE or service end events have had kind of a renaissance in recent years with all of the LLMs, right? + +00:22:29 All the chatbots are streaming the responses back to you. + +00:22:33 So this type of technology, while it's not old, sorry, it's not new, it's actually been around a long time, has kind of been underused. + +00:22:43 And Delaney kind of tapped into that and said, well, because I also always thought, well, if I want pure reactivity or true reactivity, + +00:22:50 I need two-way communication. + +00:22:53 So I need web sockets. + +00:22:53 You need web sockets. + +00:22:54 You need binary and all that kind of stuff. + +00:22:56 Yeah. + +00:22:57 Yeah. + +00:22:57 There's problems with those, which we can get into. + +00:23:00 SSE is much simpler. + +00:23:01 It works over HTTP 1, 2, and 3. + +00:23:05 And as you can see, it's just plain text. + +00:23:06 There is no complicated handshake. + +00:23:08 If you change the interval to zero and hit start, you're going to see a different type of response, which is, + +00:23:16 and I don't know if you saw the content type change, but content type now is text HTML. + +00:23:20 Oh, intro. + +00:23:20 Oh, interesting. + +00:23:21 Yeah. + +00:23:22 So this is what HTMLX would do by default. + +00:23:24 You send back HTML responses, whereas here the content type + +00:23:28 is text event stream. + +00:23:30 And this allows you to hold that connection open for as long as you want. + +00:23:34 It can be open and closed, or it can stay open until the words + +00:23:39 hello world have been spelled out. + +00:23:41 Or you can keep it open indefinitely. + +00:23:44 So we're going to see some more advanced examples where the SSE connection is held open for longer. + +00:23:50 So I think wrapping your head around this example taps you into the potential of Datastar. + +00:23:57 MARK MANDEL: Yeah. + +00:23:58 And one of the things that-- + +00:24:00 well, when I looked at Datastar, I'm like, OK, there's some interesting aspects here. + +00:24:05 And we'll get into them, how you can set up-- + +00:24:08 when I click the Start button, it might replace a piece of the page-- hey, that sounds familiar-- + +00:24:13 with HTML, not through JavaScript, right? + +00:24:15 but it didn't specify anywhere what part of the page to replace or not. + +00:24:21 Like, how does it know? + +00:24:22 And so with Datastar, you lean more on the server for many things, + +00:24:28 including deciding what part of the page that the server created in the first place to update. + +00:24:32 I really like that. + +00:24:33 I think that that's super neat. + +00:24:35 It lets you not just have sort of closer to one source of truth, + +00:24:39 but also just you can pass down multiple things. + +00:24:43 is like, we need to update this pane on the right, this text, and this element all in one response. + +00:24:50 There's a lot of interesting aspects to what you're talking about here. + +00:24:54 JOHN MCWHORTER: Anyone who's familiar with out-of-band swaps + +00:24:56 in HTMX, well, guess what? + +00:24:59 Datastar is out-of-band by default. + +00:25:01 So it's matching currently based on the ID. + +00:25:04 So you see h3 id equals message. + +00:25:07 And every event that's coming back has an ID of message. + +00:25:10 But guess what? + +00:25:11 you can use any ID you want, right? + +00:25:13 So you can use actually any CSS selector you want. + +00:25:16 But yes, we put the onus more on the backend because that is where we believe state should live + +00:25:23 or that's the source of truth for state. + +00:25:27 And you send and you work with state on the front end only when and where it makes sense to, + +00:25:33 which is more the web component aspect. + +00:25:35 And I'll caveat what Ben said there in that like state mostly lives in the backend. + +00:25:40 And that's the problem is that like state lives where it lives. + +00:25:43 Like if the user is actively able to move their mouse cursor, + +00:25:46 they own that state of the mouse cursor. + +00:25:48 You don't own that, but most of the state from your database should be in the backend. + +00:25:52 The one thing that's interesting about the SSE compared to how most people + +00:25:56 think this stuff, I will say I fell into this trap too, right? + +00:25:59 Cause I did the live view crazy stuff is that your job as a web developer is to + +00:26:03 get strings to the browser as efficiently, as fast as possible. + +00:26:06 Cause like the browser is going to deal with that, that into html and all that there's nothing faster than giving it html right so the thing that i i + +00:26:14 know i lost for a long time is that sse i thought oh it's this big string thing how is that better + +00:26:20 than binary but the irony is that because it's so regular because there's already things like + +00:26:25 compression built into the browser there's streaming things there's things that are so + +00:26:28 much easier to do here in an efficient way that the irony is if you if you you don't have to care + +00:26:33 about all these things but if you just follow our way of doing it your python app will be faster + +00:26:38 than most people's like compiled, you know, like low level language thing + +00:26:42 because you're getting orders of magnitude in the algorithms + +00:26:44 and how we're doing stuff from the hood. + +00:26:46 So I don't know if you're interested in like the deep down stuff + +00:26:49 or just like how you use it as a Python developer. + +00:26:51 But the irony is that you now have tapped into this. + +00:26:53 It seems so simple. + +00:26:54 You're like, oh, this is just a different text response. + +00:26:56 How can this be orders of magnitude faster? + +00:26:58 Like, again, I don't know how much you want to get into the weeds of that + +00:27:01 compared to just it's fun to use, right? + +00:27:03 Yeah, I really like the philosophy of having so much of it controlled by the server. + +00:27:08 It just felt disheartening. + +00:27:10 It's like, okay, so what you're going to do is you're just going to create some JSON responses + +00:27:14 on your server, and then everything is some crazy build series of steps + +00:27:19 to end up with, I don't know, Vue or React or something on the front end. + +00:27:23 And there's just so much power and flexibility to write really cool server code. + +00:27:29 But, you know, like a lot of the trends have been, yeah, that's kind of just there to support the rest of it, + +00:27:34 you know, and so I don't know, this really appeals to me. + +00:27:37 question that comes up often is like, OK, well, how do I format this? Because it has its own syntax. + +00:27:44 Very simple to read, obviously, right? An event name and then these data lines. And you can just + +00:27:48 have as many data lines as you want. And that's your HTML. If you scroll up, though, we do have... + +00:27:54 So you do need to format this, but we essentially have all of these SDKs, including Python, you'll + +00:28:01 see there. And the Python SDK is actually, I would say, one of the most intricate ones we have. + +00:28:07 Spatuel King, he's a member of the community, or Chase, I believe is his first name, his real first + +00:28:13 name, and many other contributors did an amazing job on that. So lots and lots of Python frameworks + +00:28:18 are supported. You can maybe speak more to this, Chris. And really, the SDKs are very simple, + +00:28:24 because all they do is they take a function, a patch elements or patch signals function, + +00:28:30 and you just dump in the HTML that you want swapped into the DOM or the signals you want + +00:28:34 output on the page and it just does the formatting for you so so it's really just there's three + +00:28:39 functions i think in total that every sdk has to implement and it's such a time saver you know um + +00:28:45 i doved into service and events a lot with htmx and when you get the syntax wrong it is so painful + +00:28:52 to debug because pretty much can't it just doesn't work you know or whatever it's harder to debug and + +00:28:58 so to have the helper syntax it's just a dream well and also just so people are aware i like + +00:29:04 because I was originally going to try, the irony is I was trying to get server sent events + +00:29:08 like their plugin up to snuff like years ago. + +00:29:11 Like I would highly recommend not using SSE with HTMLX + +00:29:14 because the problem is that the entire model of how you build things is very poll based + +00:29:18 and it's built out of band. + +00:29:19 It's like a weird concept, like the idea of updating, + +00:29:22 like it is not built with that in mind. + +00:29:24 So I know that they're trying to move towards that in the future, + +00:29:26 but the whole way that you interact with it is based on polling. + +00:29:30 And the thing about our way is that not only are you doing push events, + +00:29:33 But the thing is that really does change the semantics of the language. + +00:29:36 So first of all, you get like 40X compression by doing our way. + +00:29:40 But also you only send data when you need to instead of polling. + +00:29:43 So now you're using less resources. + +00:29:45 You're using less network. + +00:29:46 It changes the whole dynamic in a deeper way that you can literally save 5,000X in your network bandwidth. + +00:29:54 It sounds crazy, but it's just a reality. + +00:29:57 Right. + +00:29:57 Another thing, Delaney, that's really nice about that is the latency. + +00:30:00 That's something that drives me crazy about polling. + +00:30:03 general is just like okay well we don't want to hammer the server too hard so let's make this you + +00:30:09 know one second two second but then it's like well i click this button and then it updates and you're + +00:30:13 like ah if if something happens on the server it's sent right if it wants to one of the things that + +00:30:18 ironically because i do a lot of like go or low level language stuff is that i tend to put a debounce + +00:30:23 in my server to like five milliseconds so that i get i'm not updating more than you know 200 times + +00:30:30 a second even on a monitor because the browser actually break after 500 fps so like the + +00:30:36 interesting thing is not that it's basically data starts no longer the issue in your thing if you + +00:30:40 are on a low low powered battery device like a mobile on a 3g this is it will just work like it's + +00:30:47 stuff that you just don't have to worry about so it does change the semantics of how you build things + +00:30:52 just so that you're aware because even things like for example built into the htmx they don't do + +00:30:57 automatic exponential back off. + +00:30:59 It doesn't have all the verbs. + +00:31:01 There's caveats there that I would recommend not doing it, honestly, if you're going to do it. + +00:31:05 It's crazy that you're talking about going below the monitor refresh rate. + +00:31:09 You're not going to see it. This is only 120 hertz. + +00:31:13 120 times a second. + +00:31:15 So why would you pull faster than that? That's wild. + +00:31:20 This portion of Talk Python To Me is brought to you by us. + +00:31:23 I'm thrilled to announce a brand new app built for developers + +00:31:26 created by yours truly. + +00:31:28 It's called Command Book. + +00:31:30 You know that thing you do every morning? + +00:31:32 Open up six terminal tabs, CD into this directory, + +00:31:35 activate that virtual environment, run the server with --reload. + +00:31:38 Now, CD somewhere else, start the background worker, + +00:31:41 another tab for Docker, another one to tail production logs. + +00:31:44 Every tab just says Python, Python, Python, Docker tail. + +00:31:48 And you're clicking through them going, which Python was that again? + +00:31:51 Where my app is running? + +00:31:52 Then sometime later, your dev server silently dies + +00:31:55 because it tried to reload while you're in the middle of a code edit, + +00:31:59 unmatched brace, a half-written import, or something. + +00:32:02 Now you're hunting through tabs to figure out which process crashed + +00:32:05 and how to restart it. + +00:32:06 My app, CommandBook, gives all of these long-running commands a permanent home. + +00:32:11 You save a command once, the working directory, the environment, + +00:32:14 free commands like git pull, and from then on, you just click run. + +00:32:18 You can even group commands together to start and stop everything + +00:32:21 for a project with a single click. + +00:32:23 It also has what I call Honey Badger Mode, auto restart on crash. + +00:32:27 So when your dev server goes down mid-reload, Command Book just brings it right back up and does so over and over until the code is fixed. + +00:32:35 It also detects URLs from your output, so you're never scrolling through thousands of lines of logs + +00:32:39 just to figure out how to reopen your web app. + +00:32:42 And it shows you uptime, memory usage, and all sorts of cool things about your process. + +00:32:46 The whole thing is a native macOS app. + +00:32:49 No electron, no Chromium, just 21 megs. + +00:32:51 And it comes with a full CLI. + +00:32:53 So anything you've configured in the UI, you can fire off from your terminal with just a single command. + +00:32:58 Right now it's macOS only, but if there's enough interest, + +00:33:02 I'll build a Windows version too. + +00:33:03 So let me know. + +00:33:05 Please check it out at talkpython.fm/command book app, + +00:33:09 download it for free, level up your developer workflow. + +00:33:12 The link is in your podcast player show notes. + +00:33:14 That's talkpython.fm/command book. + +00:33:16 I really hope you enjoy this new app that I built. + +00:33:20 Yeah, on the topic of latency and all that, if you go to the examples, there's some we could look at that I think really demonstrate this. + +00:33:28 Well, maybe start with bad Apple just because we're talking about refresh rates. + +00:33:33 OK. + +00:33:33 What's happening is that the back end is streaming down just a bunch of symbols, but it creates this animation. + +00:33:39 And if you were to open the network tab, you would see it actually would be interesting to see. + +00:33:44 You probably have to refresh the page just to. + +00:33:46 Yeah. + +00:33:47 And you're going to see updates. + +00:33:49 That one there. + +00:33:49 Yeah, if you click that-- + +00:33:51 This one? + +00:33:52 Yeah, that's the one. + +00:33:53 You click Event Stream. + +00:33:54 There's an Event Stream tab for Event Stream responses. + +00:33:57 You're going to see these streaming. + +00:33:58 I don't know what frames per second we have this set to, but you see it streaming past, right? + +00:34:03 Right. + +00:34:03 The first time many people see this, this is a surprise that the browser is capable of this. + +00:34:09 But the browser can stream video, so why can't it stream a bunch of text? + +00:34:12 I mean, it's not that big of a leap of faith. + +00:34:15 But you can see, it looks like it's about every 10, 20 milliseconds. + +00:34:19 I think we're doing like 30 frames a second, but again, we can do like, we're doing this on a, + +00:34:24 basically a free tier server. So like, this is just a non-issue and it's doing all the compression + +00:34:28 stuff. So if you notice that your update, even though we're doing like full ASCII development at, + +00:34:32 you know, thousands of characters, your updates are actually not updating that. Like you see how + +00:34:37 it's transferring, but it's not transferring that much compared to how much it's actually coming out. + +00:34:41 I can see we got 1.9 megs for the whole page. Yeah. But do you see next to it? What, what was + +00:34:46 actually like the resources so you see the compression well yeah we're probably not seeing + +00:34:51 it there but in the bottom you'll see two two megabytes have been transferred but 10 megabytes + +00:34:55 of resources and so that's oh yeah yeah so it's a 5x compression yeah it's going to be much more + +00:35:01 on the stream i think because it's streaming uh normally you can hover over the size + +00:35:07 and you'll see the uncompressed but i guess it's changing too fast that's pretty wild and you know + +00:35:13 in practical usage, like I have a status screen that I have from my production app at work. And + +00:35:19 it's just amazing to just constantly be seeing these things update. And I'm doing that by having + +00:35:24 the database tell my Python code, hey, refresh. I actually ask it to get all the entries from the + +00:35:31 database and send it down the pipe. And so it's not like I'm doing the optimized thing. I'm doing + +00:35:35 the simple thing and I get all these cool things just updating all the time. And it's just such a + +00:35:40 useful thing, especially for status screens, dashboards, stuff like that. + +00:35:44 Speaking of that, go to the DB Mon example. This is one of my favorites because when React + +00:35:50 first had their first conference, they said, look at what we're doing. We're able to update at a rate + +00:35:54 that no one else can compete with us in how fast they could update the browser, right? If we + +00:35:59 actually, yeah, you're still there. So the thing is, if you actually set the FPS to something like + +00:36:03 80, whatever. So that is how fast it's coming from the backend to you. So go ahead. Yeah, + +00:36:10 because we just don't want people blasting the server. + +00:36:12 Yeah, you don't want to walk away. + +00:36:14 Yeah, but the point is that this is coming. + +00:36:16 See, we're doing stuff in microseconds on a potato. + +00:36:19 Yeah, let me just describe this a little bit for people listening. + +00:36:22 So it's like a database monitoring table that shows you how many transactions are the database overloads. + +00:36:29 So it's updating a grid of maybe 10 or 12 databases with five or six elements, + +00:36:36 and it's doing that in microseconds, 80 times a second. + +00:36:38 A lot of people see these examples and they think, well, I'm not building this kind of stuff. + +00:36:43 And me included. + +00:36:44 I build crud apps most of the time. + +00:36:47 And there are plenty of examples here that are just cruddy things. + +00:36:51 They're kind of the more boring examples. + +00:36:53 But one example that might be worth looking at is the to-do MVC. + +00:36:58 And if you can figure out how to open that in split screen. + +00:37:01 Okay. + +00:37:02 What part do you want me to open up in split? + +00:37:04 Oh, just this, the example. + +00:37:05 Yeah, so I can do these two and then I can tile them. + +00:37:09 How's that? + +00:37:10 So this is a CRUD app, but what Datastar gives you is the ability to do multiplayer out of the box. + +00:37:15 And that is like real-time collaborative apps are not easy to do and not easy to scale as well. + +00:37:22 But as you'll see here, when you have like two sessions open, it's going to be near instant. + +00:37:27 You're going to basically be observing the latency on your network connection, + +00:37:31 which is going to be 50 milliseconds to 100, but barely perceivable. + +00:37:35 So just to describe to people, we've got this 2D MVC, which allows you to, well, it's like a + +00:37:40 to-do example, which is required to be a legitimate JavaScript framework. But I've opened it in two + +00:37:46 tabs and I've used Vivaldi's tile. So these are legitimately two browsers. They just appear to be + +00:37:51 kind of in the same window. And when I enter stuff into it, it literally looks like they update in + +00:37:57 parallel, which is crazy. If you check a few of them, you'll see, + +00:38:00 You can barely tell which one's updating which. + +00:38:03 It happens almost instantly. + +00:38:04 Yeah, if I look at the other one and I click on one, it feels like that's responding to my click. + +00:38:10 I need to correct myself that it is happening instantly because when you click, when you check one of those, + +00:38:15 it's not, and this is an interesting thing we can get into. + +00:38:19 We're not doing optimistic updates. + +00:38:21 It's actually sending a request to the server and the server is simultaneously updating + +00:38:27 both of your tabs at the same time. + +00:38:29 Even if I had just one open, it's still going round trip to the server. + +00:38:33 That's why it looks like it's simultaneous because it's effectively. + +00:38:35 This is a thing that you can, we can talk for like three hours + +00:38:38 and I will yell at most spa developers because there's this weird thing + +00:38:43 that because it's easy, people will actively lie to users in the spa world + +00:38:47 and they'll do optimistic updates, which means I'm going to make it + +00:38:50 so that I'm making this change. + +00:38:52 And then if there's a problem, then fix it. + +00:38:54 Whereas we say you should do indicator saying, I'm trying to make a change to this + +00:38:58 and then fix it. + +00:38:59 Because you don't want, like when you're playing a video game, you can do what's called dead + +00:39:02 reckoning and you can do some stuff to net rollback code. + +00:39:05 You can do some clever things to hide latency, but you don't want to hide latency when it + +00:39:10 comes to like a bank transfer or did I buy that thing or did I get that theater ticket + +00:39:14 or any of that stuff. + +00:39:15 Like people just have the wrong mental model of how the web should work. + +00:39:19 I'm actually going to send you another thing that this might blow your mind even more because + +00:39:23 the three of us basically can play. + +00:39:25 This is an example where all of us could be playing live with each other right now in an active shared state that's been at the top of the Hacker News and again, runs on a potato. + +00:39:35 I don't know if you just put that in your... + +00:39:37 Yeah, let me drop it over. + +00:39:38 Hold on. + +00:39:38 I'm going to put it in the other tab. + +00:39:39 So right now, don't touch anything. + +00:39:42 I'm going to actively start. + +00:39:43 I am purple. + +00:39:43 I am literally starting to click right now. + +00:39:45 All right. + +00:39:46 So we're looking at a multiplayer game of life here. + +00:39:48 I'm seeing that live here. + +00:39:50 And if you open up that in the other tab, you would actively see the exact same state. + +00:39:54 So everyone in the world, like if you open that up in the other tab, + +00:39:58 you cannot get out of sync. + +00:40:00 It's not faking it in the front end. + +00:40:01 This is literally sending in. + +00:40:02 What's even crazier about this, here's the crazy part. + +00:40:04 It's actually a rendering demo. + +00:40:06 The guy who wrote it is writing in a scripting language, Clojure, + +00:40:09 and he's sending down 2,500 divs per frame styled, inline styled. + +00:40:14 Now go to your network tab now and look at what's actually, + +00:40:17 like look at your network tab and you'll see how little data we're actually sending over. + +00:40:21 even though he's updating 2,500 divs per frame, like if you go to wherever it's updated, + +00:40:26 yeah, whichever one's the one that's updated, there you go, yeah. + +00:40:29 So if you look here and look at how much is being sent + +00:40:32 versus how many is actually, like try to, like, this is just a different paradigm + +00:40:36 for how you build. + +00:40:37 And the thing is, again, not everybody has to care about these low level things, + +00:40:40 but the thing is, is that once you do this, the idea of CRUD kind of goes away + +00:40:44 because in our opinion, you go to a multi, you make a multi-page app like you would normally do + +00:40:49 in HTMX or anything else, but you keep an open stream + +00:40:52 and you just update whatever's happening in your backend + +00:40:54 as it's happening. + +00:40:55 And it simplifies the world. + +00:40:57 And what's also interesting is because of how we do compression and all that, + +00:41:00 you just send your entire page. + +00:41:01 You don't need like out of band. + +00:41:03 It doesn't even really make sense because we're so fast that you can just, + +00:41:06 you as a Python developer, you just give us your entire page + +00:41:09 and let us deal with it. + +00:41:10 And we will come up with the fast stuff. + +00:41:12 So Chris should probably talk a lot more to that because the fat morpher stuff, + +00:41:15 it's a fundamental change in how you build web apps, I think. + +00:41:18 Yeah, yeah. + +00:41:19 especially the kind of the mental shift of like, because I kept thinking, okay, I need to like send one row at a time. + +00:41:25 And I actually have one status screen that does that because we use a Firestore, Google's Firestore as our backend for this app. + +00:41:32 But for some reason, sometimes it just doesn't send every update. + +00:41:36 And so on another status screen, I actually, you know, query the whole database table or collection and send it down to Pipe. + +00:41:42 And because sometimes it doesn't send from Firestore, I get the entire latest state of all the things that are in flight and updated on my screen. + +00:41:51 And it just makes things easier. + +00:41:52 Yeah, it's amazing. + +00:41:53 Sounds like a good opportunity to subscribe to database query changes. + +00:41:57 I know some databases you can say, if this query updates, you know, trigger this event and then keep it flowing, + +00:42:03 like straight from events on the database, straight to your front end. + +00:42:06 Pretty cool. + +00:42:07 I do want to go back and just put a little bit of commentary, Delaney. + +00:42:12 Well, you said optimistic updates. + +00:42:15 So one of the things that's really common in JavaScript is I click this thing, it changes. + +00:42:20 I want to mark it as changed. + +00:42:21 And then I'm going to tell the server, hey, we made this change. + +00:42:25 It's very possible the server died, that you're not allowed to make that change or whatever. + +00:42:29 And then you've got to come back and go actually undo that. + +00:42:30 That really, you know, there's like a weird. + +00:42:33 So what you're saying is you don't have to worry about that kind of stuff. + +00:42:35 We're a framework, not just a library. + +00:42:37 The idea is that you have these indicators that not only basically your indicators drive a signal. + +00:42:41 Like, again, the details don't really matter. + +00:42:43 But the idea is that you have instantaneous, like within the same frame updates of, + +00:42:48 hey, I'm going off to do something. + +00:42:50 Like usually you make a spinner or you say, I'm going to do this to gray out the field + +00:42:53 or I'm going to do, like there's all kinds of things you can drive. + +00:42:55 Because again, the state of what the local stuff is while the change is there, that lives in the client. + +00:43:02 Like that is part of Datastar. + +00:43:04 It has all the right tools to make it so that you can disable it or gray it out + +00:43:08 or say, I'm going to put a spinner next to it. + +00:43:10 Like you can do all those things. + +00:43:11 But the thing is you're not lying to your user. + +00:43:13 That's my whole thing. + +00:43:14 And people say, well, that's not really a lie. + +00:43:15 It's like, yes, it is. + +00:43:16 You're literally lying to people. + +00:43:18 Like, please stop. + +00:43:18 It's a DX issue. + +00:43:20 The reason why people do it is because it's convenient, not because it's correct. + +00:43:24 And again, like you can do optimistic updates. + +00:43:26 You can do SBA-like things using Datastar. + +00:43:29 We don't recommend it because Datastar is more than just this tech. + +00:43:34 It's also like a way of doing things. + +00:43:37 What I wanted to point out here is that you might imagine that, you know, this is something that when you click edit, + +00:43:43 it turns it into a form. + +00:43:44 So you might like load the form into the page, hide the form, + +00:43:47 and then just do a show hide approach. + +00:43:49 But the hypermedia approach is kind of like the REST approach + +00:43:53 where you can only take the next action at any given time. + +00:43:57 So if you open the network tab, I just want to kind of walk you through this briefly. + +00:44:00 If you cancel that, when you hit edit, you will see a network request to the server. + +00:44:04 And what comes back is that form. + +00:44:07 So it's real time as in like what you're seeing now is the actual state reflected on the backend. + +00:44:13 And when you save, you're also going to see the same thing. + +00:44:16 You're going to see a network request to the server and it gets the true current state + +00:44:22 as it has been saved is now all comes back down. + +00:44:24 So it's like you don't even need optimistic updates most of the time. + +00:44:28 And when you do use it, it's because you're trying to cover up poor performance. + +00:44:33 You're favoring perceived performance over true performance. + +00:44:37 One of the things I hear a lot is people saying, but it's so much slower. + +00:44:41 But I think people are used to or think it's much slower than it is + +00:44:45 because the web, the spa life that we see around us feels so slow. + +00:44:51 But anytime I've seen people try to lean into just using the network, + +00:44:55 it's so much faster than you expect. + +00:44:57 Well, and also you have so much less our way. + +00:45:00 Your usage of network can be easily 100x less, which means you have less contention, + +00:45:04 which means when you do send something, it's there immediately. + +00:45:07 And also because you're not doing polling with polling, you have to send to the server and the server sends back. If you just send from server + +00:45:13 when something updates, now you've just halved your RTT, right? Your round trip has just halved. + +00:45:19 So you half it and you're doing like a thousand less of something. All of a sudden, things opened + +00:45:23 up for you in weird ways, right? It's a fundamentally different way of thinking about the problem. + +00:45:29 Another example that we like to bring up a lot is there was a while back, someone did a million + +00:45:33 checkbox demo and they had a whole write-up on it, right? And they basically had to take it down + +00:45:38 because it was just too expensive to run. + +00:45:39 We have a version that's not just checkboxes, but color checkboxes. + +00:45:43 So you can actually make ASCII art and stuff like that. + +00:45:45 And it's a billion. + +00:45:46 And it runs on the same server that was running that Game of Life demo. + +00:45:49 It's on the same server. + +00:45:50 It's actively, and it's been on top of Hacker News. + +00:45:52 It's a $5 VPS as far as I know. + +00:45:54 Yeah, it's a $5 one. + +00:45:56 It runs all these demos all the time, active, top of Hacker News, + +00:45:59 and it's never gone down. + +00:46:01 What's really interesting about that demo is that it becomes a backend optimization challenge, right? + +00:46:07 You're no longer trying to optimize the front end. + +00:46:09 You rely on the browser and the browser API to take care of that for you. + +00:46:14 And now you're doing, I don't know, you're optimizing your database. + +00:46:17 You're optimizing your queries. + +00:46:20 I actually threw the link to that in there. + +00:46:22 Because it's a nice demo to look at when you realize there are a billion of these being stored in a SQLite database somewhere. + +00:46:31 So you can scroll anywhere on the board. + +00:46:33 And it's like a 30,000 by 30,000 something grid because the square root of a billion + +00:46:39 is some weird number, as it turns out. + +00:46:42 And obviously this is, or not obviously, but this is multiplayer. + +00:46:45 So if I was to view this or you were to open a different browser tab, + +00:46:49 then you would see the exact same thing. + +00:46:51 The board is the same everywhere. + +00:46:52 That is crazy. + +00:46:54 The one thing that I will say that's hard, I don't know, Chris can really probably speak to this more + +00:46:58 and it sounds like a weird thing, Like, Datastar ends up in reality being like five or six things on your page, and it just + +00:47:05 gets out of the way. + +00:47:06 All of a sudden, like, most Datastar, you're going to get to a point where you try it, + +00:47:09 and you're like, that's it? + +00:47:10 Like, you will feel weird about every other approach once you really try it. + +00:47:14 Like, just try it, and you will see every other approach is wrong. + +00:47:18 Like, it's not because I made it. + +00:47:20 Like, I wish someone would have made this because it just, it's so simple. + +00:47:23 It feels like cheating in a weird way. + +00:47:26 That's hard to explain. + +00:47:27 it really it's a weird like i don't know what we all were doing i was part of the problem like right + +00:47:31 like i was like oh well google and everyone facebook and all the other guys have this figured + +00:47:36 out like this has to be the best approach so that's the weird thing is it's so simple i don't know what + +00:47:41 like crystal probably it sounds like i'm selling it but it's just i don't know it's weird it's so + +00:47:45 exciting it's so amazing and yet it's it's using all these boring technologies and like yeah like + +00:47:50 i remember i showed my wife this my status board and she's like oh yeah that looks really cool and + +00:47:55 I'm like, oh yeah, because you don't understand what everything is going on behind it, you know? + +00:47:59 Yeah, exactly. + +00:48:00 It's like, it used to be so complicated. + +00:48:03 So let's do, we got a little bit of time left. + +00:48:05 Let's do this. + +00:48:06 I think it might be fun to talk through kind of some of the attributes and what it looks like, + +00:48:13 kind of program with this a little bit and then what it looks like on the server. + +00:48:15 How's that sound? + +00:48:16 Would it make sense? + +00:48:17 Like there's a good example of a kind of a meta framework for Python called Stario, + +00:48:21 which they just got their V2. + +00:48:23 Okay. + +00:48:23 Just launched. + +00:48:24 I don't know if that is a more Python-esque way of doing it. + +00:48:27 It depends on how you want to. + +00:48:28 Let's start with some of the Datastar attributes, and then we could talk about that. + +00:48:32 How's that sound? + +00:48:33 Like, just, you know, what does it look like to say to, + +00:48:36 you know, I want to connect a button to Datastar actions + +00:48:41 on the back end or wired up and so on? + +00:48:43 We've talked about a lot about, you know, the back end driving the front end through patching elements, + +00:48:48 which is kind of the lower half of what you're looking at. + +00:48:51 To access that, you need to have a click listener or some sort of event listener to trigger that. + +00:48:58 And datastar, as the name suggests, uses data-star or asterisk attributes. + +00:49:05 So these are part of the HTML spec data set. + +00:49:09 And we just leverage that. + +00:49:10 And we have a small grammar that you'd find on the reference page with all of the data-attributes. + +00:49:19 And data on is just registering an event listener on the current element. + +00:49:23 So data on colon click is just obviously registering a click event handler on the button. + +00:49:30 And what's happening is that then Datasar also gives you actions. + +00:49:34 So that at get is an action to send a get request to the server. + +00:49:38 You pass in the path, which is slash endpoint there. + +00:49:42 And then the server takes care of the rest. + +00:49:44 So what you're seeing is a div underneath with an ID. + +00:49:48 IDs are obviously unique in HTML, so they're ideal. + +00:49:51 And Datastar just uses that fact. + +00:49:55 And what Datastar is going to do from the backend is it's going to, yeah, just send back down that div with some text content inside of it. + +00:50:03 And then what Datastar does is it mutates the incoming DOM into the existing DOM. + +00:50:10 I'm sorry, it morphs. + +00:50:11 So it uses a morphing strategy. + +00:50:14 So rather than doing a straight swap, which is what HTMLX does, it will actually morph the incoming HTML into what's currently on the page. + +00:50:23 That's kind of what opens up the door to these kind of broad, like where you send the entire document down, but only what changes get swapped in. + +00:50:30 But in this case, it's more of a fine grained thing. + +00:50:34 So only that div is going to get swapped out. + +00:50:36 And the reason why that morph matters is because you, since you aren't replacing it, things like + +00:50:41 focus and like where your input is and all that stays the same. So when you, even though you update + +00:50:46 the whole page, you're actually not actually changing the state and that's really important. + +00:50:51 So you do declarative development. You just say, I want it to look this way and it just does the + +00:50:56 right things to do it. It's from a mental ball. It's almost like having the BDOM in the backend. + +00:51:00 You just say, here's what I want this page to look like. And it does all the work, but we don't do + +00:51:03 BDOM. We don't do any of that stuff. We do the fast thing. So in terms of what your backend would + +00:51:08 send, if you can just scroll back up, it's that text that you were looking at. Let's look at the + +00:51:14 raw version because, yeah, so that's the HTML. If you scroll down to the next text, it's a code + +00:51:19 field. Yeah. There's a section that has like event, data star, patch elements, and then what the + +00:51:24 elements are and so on, right? This is like the SSE stream. Yeah. And that would be the raw events + +00:51:29 that you would send down. + +00:51:31 But if you look at the next one where we have a Python example, + +00:51:35 you would see like, well, how do you do that in Python + +00:51:38 without actually writing, you know, the raw format out? + +00:51:41 And that's how you would do it there using the Python SDK. + +00:51:44 Let's dive in a little bit to the SDK itself. + +00:51:48 So I got so many things open. + +00:51:51 Hold on. + +00:51:52 We got another link for you. + +00:51:53 No, I'm kidding. + +00:51:54 You know what? + +00:51:55 I'm just going to go. + +00:51:55 I'm going from the homepage. + +00:51:56 There we go. + +00:51:57 There you go. + +00:51:57 Chris, maybe you could talk us through this. + +00:51:59 I think before I throw it to you, though, yeah, there's a lot of framework support here. + +00:52:03 So if you're a Django person, a FastAPI person, even fast HTML, it's interesting, + +00:52:08 LightSar, Quartz, SanEck, or Starlet. + +00:52:10 There's a bunch of different ones here, but maybe just talk us through this, if you'll, Chris. + +00:52:16 I'm trying to remember. + +00:52:16 I'm not as familiar with the example, but as you can see, this one method is, I think, where the magic happens. + +00:52:22 I'm trying to remember which tool. + +00:52:24 This is a Quartz. + +00:52:25 Yeah, the Quartz is the examples in Quartz. + +00:52:27 So, you know, they first define a route, a home route slash, and it returns HTML and it's just in the string there. + +00:52:34 Right. And then that. + +00:52:35 This could be a Ginget or Chameleon or whatever template. + +00:52:38 Like it's just whatever. It doesn't matter. + +00:52:39 But somehow they get it. Yeah. + +00:52:40 Makes the example easier to see in one go. + +00:52:42 And obviously you see that it's pulling Datastar from the CDN and then it on the load, it gets it sends a request to the slash updates endpoint. + +00:52:54 See what comes from that. + +00:52:56 And so down below that, you have the slash updates endpoint, which has a decorator called data, data star underscore response. + +00:53:03 And that just does a couple of nice things like sets the HTTP headers and whatnot to be the service and event protocol. + +00:53:10 And then what I like to the first line says signals equals await read signals. + +00:53:16 And so that's another helper that essentially says when I have a request coming in, + +00:53:21 data star has a specific way of sending the state of the front end to the back end. + +00:53:25 So the back end can do whatever it needs. + +00:53:26 Right. We haven't even talked about signals yet. They're like kind of a data binding set of JavaScript data that loads, you know, reactive data loads on the front end, right? + +00:53:35 In some ways, the Alpine JS kind of, I don't want to say equivalent, + +00:53:39 but it covers similar functionality. + +00:53:41 And so if you have data on the front end that the backend would like to know, + +00:53:45 that's an easy way to get it. + +00:53:47 And then essentially what happens is we get into this loop, this while true loop, + +00:53:51 and Datastar will just start sending down server sent events in text + +00:53:56 by using the sse.patchElements function, or I guess it's a method technically. + +00:54:01 And all it's doing is sending a string that has the current date time dot now in ISO format down. + +00:54:07 And then we wait, we sleep for a second, or is it a second? + +00:54:11 I guess it's a microsecond. + +00:54:12 I keep forgetting which one. + +00:54:13 Yeah, that's a millisecond. + +00:54:14 No, no, it's second. + +00:54:15 And sleep is seconds. + +00:54:16 Sleep is seconds. + +00:54:17 It takes a float. + +00:54:18 So once it sleeps, it sends another server sign event. + +00:54:21 With this time, it's instead of sending the HTML down, we're sending a signal. + +00:54:27 So essentially changing, say, you can say, like the JavaScript value or script data + +00:54:31 onto the front of the page. + +00:54:32 Right, right. + +00:54:33 So it's showing that you can send the HTML and let Datastar patch it, or you can basically + +00:54:38 from the server set one of these signal things that will be reactive on the front end, right? + +00:54:43 Yeah. + +00:54:43 You said it much better than I did. + +00:54:45 Thanks. + +00:54:45 It's a long way of saying it's a clock, right? + +00:54:48 Yeah. + +00:54:48 Also the thing, just for people that aren't used to thinking about this way, especially + +00:54:51 if you're doing Python, like all a signal is, is instead of saying that here's, I'm + +00:54:56 setting a variable, you're saying I'm setting a relationship that says like, kind of like + +00:55:00 in an Excel document when you set a formula for a cell, + +00:55:04 it's the same idea. + +00:55:04 You're setting up a relationship saying, when this thing and this thing changes, update this. + +00:55:08 And it does smart things to do that efficiently. + +00:55:10 But the idea is it's a relationship. + +00:55:12 It's declarative. + +00:55:13 So kind of like with SQL, you think of SQL as a declarative language, right? + +00:55:17 You don't care how it creates an index. + +00:55:19 You just say, create index. + +00:55:20 Same thing happens here. + +00:55:21 You just say, hey, I want when this thing changes, this other thing to change. + +00:55:25 And the problem is that declarativeness is not built into JavaScript. + +00:55:28 It's not built into the browser, but we just made the web a little bit more declarative. + +00:55:32 That's all we did, basically. + +00:55:33 Right. + +00:55:34 Declarative is generally pretty good. + +00:55:36 It's a good way to work. + +00:55:37 It keeps things simple and lets the underlying system have at it. + +00:55:40 So a couple of things, well, we still got a little bit of time, + +00:55:44 but to wrap things up a little bit. + +00:55:46 Editors. + +00:55:47 I think having good editor support is really important for adoption. + +00:55:52 You know, drives me crazy when I go and try to work with JavaScript, CSS, + +00:55:56 attributes or whatever, and I'm like, they're not here. + +00:55:59 No help. + +00:56:00 So you all have nice extensions and plugins for common editors Python people might use, right? + +00:56:06 Yeah, we have VS Code, which you're seeing here, and PHP Storm. + +00:56:12 Or sorry, I use PHP Storm, but all JetBrains editors. + +00:56:16 PHP Storm, PyToram, WebStorm, all of them things. + +00:56:19 So it's in the JetBrains marketplace, so it'll work for, I believe, all JetBrains IDEs. + +00:56:25 I believe so. + +00:56:25 You also have the AI editors covered. + +00:56:28 Do we? + +00:56:29 In the OpenVSX registry, all the ones that have been kicked out from VS Code, this is where they all have to go to get their installs, right? + +00:56:37 That explains why people requested this from me. + +00:56:41 Yeah, if you're doing cursor, anti-gravity, windsurf, like all those things, they were all kicked out of the VS Code registry. + +00:56:50 That's not a complaint. + +00:56:51 I mean, it's a Microsoft product. + +00:56:53 They built it. + +00:56:54 They don't have to build all the other ones. + +00:56:56 But that's why they're here, right? + +00:56:57 We keep those up to date. + +00:56:58 We do those ourselves, the SDKs. + +00:57:01 I mean, Delaney wrote the Go one, I wrote the PHP one, and the rest are just community contributions. + +00:57:07 We've had contributions to these too, to the IDE extensions. + +00:57:11 We maintain these primarily. + +00:57:13 Yeah, and these are great. + +00:57:14 These just, you know, save on typing, but more importantly, save on making typos. + +00:57:20 You know, they show you all of the available data attributes. + +00:57:23 Maybe Chris can speak more to is that the irony is, though, + +00:57:26 you won't need that many tags to actually do your work. + +00:57:29 So it's not like a tailwind thing where you're like, + +00:57:31 oh, I rely on it to autocomplete. + +00:57:33 It's just... + +00:57:33 Yeah, absolutely. + +00:57:34 In fact, it's one of those things where I discover more things I can do with Datastar + +00:57:38 because as I'm typing data dash and I'm like, oh, I didn't actually remember + +00:57:42 that there's a attribute to do whatever it is. + +00:57:45 I don't remember. + +00:57:46 Like, I can't remember at this point. + +00:57:48 And then I went to the documentation like, oh, check this out. + +00:57:50 This is so much more I can do. + +00:57:52 But yeah, I find I love the plugin, but I find I don't use it too much + +00:57:55 just because I'm not writing as much HTML with it. + +00:57:58 While I'm sitting here on this open VSX registry, do you all have advice for making Datastar work well + +00:58:05 with Identic AI and Claude Code, Cursor, et cetera? + +00:58:08 There's some active research going on like in Oslo at a college + +00:58:13 that's doing, ironically, using Datastar to do some stuff around like how LLMs work with code bases. + +00:58:22 And the reason why is because the entire code base fits in basically every context, even the nano ones, like the entire code base fits there. + +00:58:28 And what they found, we've gone back and forth a bit, is that almost all of them are completely + +00:58:33 overfitted. So if you just want to make a website with agentic stuff, go do React, + +00:58:38 because that's what it's built for. And it's overfitted to such a degree that if you try to + +00:58:43 use the spec correctly and to say, here's all the sorts of data star, go use it to build websites, + +00:58:49 it will fall over almost in every regard. + +00:58:51 So it's one of those things where you don't need that much, + +00:58:54 but it will ironically show you how bad things like Clawed and Codex and stuff + +00:58:59 are at just using the current context to solve things. + +00:59:03 Hopefully that gets better, but we have something around, + +00:59:05 like we have a slash docs page that you can feed into your LLM, + +00:59:08 but I'll say that we do not focus on that at all because you're basically fighting against what the training already happened. + +00:59:15 So you're better off, like if you want to use, if you want to make better size, + +00:59:19 you want to be fast and efficient and all that stuff, + +00:59:21 we're 100% the right thing to do. + +00:59:22 If you just want to like one shot something, go use React and stay in that world. + +00:59:26 You want to vibe code it? + +00:59:28 Hey, I've got something. + +00:59:29 I feel like this might resonate with you, Delaney, especially the way you just described it. + +00:59:34 Have you all seen the Kai Lintit Senior Engineer Tries Vibe Coding? + +00:59:39 This is an amazing video. + +00:59:41 And like half of the video is like, no, no, no, not in being installed. + +00:59:46 What are you doing? + +00:59:47 It reminds me very much of like, it's just like, nope, that's not what I told you to do. + +00:59:52 I know that's what you think the most common thing is, please stop. + +00:59:54 Yeah. + +00:59:55 And the thing is that it's not that I actually like a lot of the stuff, but I treat it as + +00:59:58 an autocomplete or like it can write code faster than I can when it comes to like, hey, change + +01:00:03 this in 27 different places. + +01:00:05 And I forget which files I did it like. + +01:00:06 There's value to it, but people are trying to use it to learn. + +01:00:10 It's a complete, it actively is working against you. + +01:00:13 Ben has done an amazing job with the guide. + +01:00:15 Like, please, like, it's fine to use the LLMs to help, like, guide your process and to, like, + +01:00:20 knock stuff out quickly once you have a baseline. + +01:00:22 But you have to know when to say no. + +01:00:24 And he has done it. + +01:00:25 The guide takes half an hour, less than a half, like, probably 15, 20 minutes to read + +01:00:29 and then, like, an hour to actually work through. + +01:00:32 Please try it first before you try to throw it at the LLMs. + +01:00:35 It's not because I hate them. + +01:00:37 It's more that they are just overfit to the, like, the sea of badly written SPA code. + +01:00:42 That's, unfortunately, that's the situation we're in. + +01:00:45 Yeah, especially with JavaScript, the agentic AI is very trained. + +01:00:49 It wants what it wants. + +01:00:51 All right. + +01:00:52 Let's talk, speaking of being near the guy, if I go over here to more, there's a pro section. + +01:00:59 I'll let you all give a shout out to pro. + +01:01:02 I know you have a really strong sales pitch here. + +01:01:04 You were talking about earlier. + +01:01:06 Now, what is this data, Datastar Pro? + +01:01:08 It's been about a year since we released the beta one of Datastar. + +01:01:14 We are taking our sweet ass time for a very good reason, which is that we want version one to be the last version or like the last major version. + +01:01:23 We don't really want to force people through breaking changes and major updates because that's really just a pain. + +01:01:30 And I think like Python has done a great job with that and Go as well. + +01:01:36 And like there are some ecosystems where you just don't make breaking changes. + +01:01:39 That's the norm. And that's what we want to be. + +01:01:41 And the JavaScript ecosystem is, you know, the antithesis to that. + +01:01:45 They're like, here, hold my beer. + +01:01:47 I'll show you breaking changes. + +01:01:48 Yeah. + +01:01:48 Have you heard of LeftPad? + +01:01:49 To give you an idea of how far we take that, we don't have npm. + +01:01:53 Like we don't actually even submit to npm. + +01:01:55 We have no package.json in our JavaScript framework. + +01:01:59 We actually, like, there's none of that stuff. + +01:02:02 It does not exist in our ecosystem. + +01:02:03 So we take it very seriously when we say it's funny to have a JavaScript framework that + +01:02:07 actively hates the JavaScript ecosystem. + +01:02:09 And you guys, I think also it's worth pointing out that you don't have a strong build step, + +01:02:14 tree shaking, web packing story, right? + +01:02:17 You just dropped- + +01:02:17 No, we do. + +01:02:18 The thing is that, like for example, beat is kind of the well-known way to do this stuff. + +01:02:25 But guess what? + +01:02:25 Under the hood, it uses ES build. + +01:02:27 And ES build is a go thing. + +01:02:29 We build a lot of our stuff in go. + +01:02:30 So it's literally embedded in our, like we just use the ES build directly. + +01:02:33 We don't need 20,000 things from npm. + +01:02:36 We just use the go tools inside of our binary because that's the fast thing to do. + +01:02:40 So we don't need all of that. + +01:02:41 So we have no dependencies, nothing, and we don't even use npm or any of that at all. + +01:02:46 Yeah, so the reason I mentioned the beta is during the beta phase, which lasted about six months, + +01:02:51 we, Datastar gained a lot of traction, a lot of interest, and people had a lot of requests. + +01:02:56 And we were like, yeah, we see, and because it's plugin-based, + +01:03:00 you can always just add another plugin. + +01:03:01 You can add it yourself, or we can build a plugin and add it to Datastar. + +01:03:05 But we were very adamant about keeping the open source Datastar framework as tight as possible. + +01:03:15 Like I said, it should do everything you need, but nothing you don't. + +01:03:18 So how do we do that while adding plugins? + +01:03:21 So during that beta phase, we started thinking about, well, do we have multiple versions of Datastar? + +01:03:26 Do we have a marketplace of plugins? + +01:03:28 Or how do we manage that? + +01:03:30 And at the same time, we were also asking ourselves, because Delaney and I both, we have full-time things that we're doing. + +01:03:35 And this is a side project, but we're almost doing it full time alongside our other full time things. So how do we make this project sustainable? Because it doesn't stop at Datastar. You probably see Rocket and Stellar CSS on that page in the navigation sidebar. Those are like projects that build on top of Datastar. So Datastar is just the foundation. And Rocket kind of takes it to web components and Stellar CSS is a CSS framework that builds on top of these concepts. + +01:04:05 So we're trying to fix not only JavaScript web components, but also CSS. + +01:04:10 So we have a long-term vision. + +01:04:12 How do we make that sustainable when we're both busy people anyway, and this just takes + +01:04:18 so much of our time and the project appears to be growing? + +01:04:22 So at that point, we decided, well, how do we want to even run this? + +01:04:25 So we decided we don't want to found some company and do VC like we're, if anything, + +01:04:31 anti-VC funding. + +01:04:32 So we founded a nonprofit organization in the U.S. called Star Federation, and that's what backs this project, including Rocket and Stellar CSS. + +01:04:44 To help fund that organization, we decided let's have Datastar be the open source framework, but then something called Datastar Pro, which is like all those plugins that we think are good ideas, but that most people don't need. + +01:04:58 We'll put those into Datastar Pro, and that can kind of grow over time. + +01:05:01 It's a collection of plugins that you might want if you're using Datastar in a professional setting. + +01:05:07 But, you know, if you're just using Datastar, you don't actually need it. + +01:05:11 And so that's what we tell people. + +01:05:12 Most people don't need it. + +01:05:13 It's a collection of plugins and it's a Datastar inspector, which sits on your page. + +01:05:19 You get access to the bundler and now you get access to Rocket and StadrCSS, which is a work in progress. + +01:05:26 Yeah, that was, I think, a good decision. + +01:05:28 Like there was definitely some uproar initially that, you know, some plugins were taken away, but those plugins were never taken away. + +01:05:35 They still exist in the repo if anybody needs them. + +01:05:38 What the result is, is that we have like some money coming into a bank account, which is not even used to pay maintainers. + +01:05:46 We use that for running costs and like for, you know, podcasting software. + +01:05:53 And if we need to travel to conferences, which we've yet to do. + +01:05:56 But essentially, it's like a way of having some money into the bank so that we can justify all of the work that we do in maintaining Datastar and pushing that forward. + +01:06:06 But the V1 thing. + +01:06:07 100% free sounds great until that means it becomes abandoned where, you know, and like people can't work on it anymore. + +01:06:14 And I think it's fair. + +01:06:16 There's one thing that's kind of interesting about the model, because especially with the tailwind stuff that's been going on lately. + +01:06:21 One of the things that we talked about, and people get very angry about this, but for example, + +01:06:26 Rocket is a web component layer that you basically just write Datastar in a declarative way, + +01:06:31 and it dynamically generates web components for you on the fly. + +01:06:34 And it's a great way to build web components. + +01:06:36 It'll save you tons of hours. + +01:06:38 And people won't pay for features. + +01:06:39 They pay for convenience. + +01:06:40 So the thing is, people said, well, I want you to generate out the content and make that + +01:06:46 open and available. + +01:06:46 And we said no, because basically the way we look at it is that almost like Pico 8, + +01:06:51 or any kind of game engine, you pay for the game and then all the mods are free. + +01:06:55 So all the rocket components and all this stuff is gonna be free, but the core engine is not free. + +01:07:01 It's a paid thing. + +01:07:02 And the reason why is if it becomes successful, if we do our job and we make it so it's easy for everybody, + +01:07:07 the Star Federation will do better over time. + +01:07:10 Whereas Tailwind's model of they're competing against every other person in that space, + +01:07:15 whereas it just does not work. + +01:07:16 So our thing is if we do get successful and we do get more people, + +01:07:20 then it's self-sustaining as in you paid for this little engine + +01:07:24 and now you get all the ecosystem around it of open source. + +01:07:27 So you can do open source in a way, but you have to find a core engine that is not open source. + +01:07:32 Otherwise it will fail in the modern world. + +01:07:34 Let's close this thing out with two super quick things + +01:07:37 because I know we're over time. + +01:07:38 Roadmap. + +01:07:39 Ben, you talked about taking your sweet time to 1.0. + +01:07:42 Is there a forward-looking roadmap? + +01:07:44 Are you guys done or what are things? + +01:07:46 The release counted RC1, I think it was about six months ago. + +01:07:51 and like the release has just been slowing down, slowing down. + +01:07:54 So that stagnation in like just releases with fixes is a good sign to me that we're very, very close. + +01:08:00 At this point, like the switch from release candidate to stable + +01:08:05 is just literally like just, you know, dropping the RC. + +01:08:09 There's no like features that are going into it. + +01:08:11 There's no big changes. + +01:08:13 We're taking our time because like I said, it's easy to put something up slightly prematurely + +01:08:18 and get some defaults wrong. + +01:08:20 I mean, that's what happened with HMX 2 was just like fixing some defaults that they decided they got wrong in version one. + +01:08:27 So we're trying to avoid a situation like that. + +01:08:30 And the only way to do that is to just let it simmer, let people use it, let people dog food it. + +01:08:35 And us, too, we're actively using for many projects using data ourselves and discovering every now and then, oh, this default is probably we're trying to avoid foot guns. + +01:08:46 So we're trying to make it so that the defaults give you the best possible experience that you need zero configuration, ideally, but you can configure as needed. + +01:08:55 But getting those defaults right is really the only thing stopping us from, not right, but locked down is the only thing stopping us from a V1 stable. + +01:09:05 I don't like to give timelines. + +01:09:07 In fact, it's one of our things that I tell Delaney, never promise a timeline. + +01:09:12 But I could see us in the first half of this year, just flipping the switch. + +01:09:18 But it sounds like you might be able to use the RC and you might more or less be safe, yeah. + +01:09:22 In fact, we recommend people rename the RC and they change the name to React-Foo + +01:09:28 so that they just drop it in their React projects because the entire framework is smaller than most components. + +01:09:34 Just start hiding it places. + +01:09:36 Yeah, don't even name it Datastar. + +01:09:38 A stealth takeover of the spa world. + +01:09:41 Awesome. I love it. + +01:09:42 All right, let's wrap up the show with a final call to action + +01:09:46 for people who want to use Datastar, learn more, get started. + +01:09:50 Chris, I'll let you go first so Ben and Delaney can have the final word. + +01:09:54 The first thing I was thinking of is because I get asked so much about + +01:09:59 how long it takes to connect to the server and things like that, + +01:10:03 there is a portion in the DjangoCon talk I gave in, I think it was 2023, + +01:10:08 where I showed a video of five phones, five Android phones, trying to do the same thing, + +01:10:14 shopping for eggs, I believe it was. And essentially one of them is an HTML driven + +01:10:20 multi-page app and it smokes the single page applications, the native apps and everything. + +01:10:25 And so I put a link in our chat. Maybe you'll be a part of the show notes. It's a deep link to go + +01:10:30 straight to that portion of the talk because it is like that video reminded me like, this is what + +01:10:36 I want to build. I want to build websites that are fun for people to use. And, you know, Datastar + +01:10:43 enables me to use real-time interactions with way less complexity than I ever thought + +01:10:50 possible. So I guess the two things I would say is, one, check out the deep link if you're at all + +01:10:55 interested. And number two, definitely try out something, you know, just even clone the Python + +01:11:00 repo and just try some of the examples and see what it's like. Yeah, I'll definitely link to that. + +01:11:05 Cool. Thanks, Ben. + +01:11:06 I also gave a conference talk last year. + +01:11:08 There's a recording, so I'll send you the link to that, which really walks through my journey of Datastar + +01:11:13 and how Datastar has truly opened my eyes to what's possible. + +01:11:18 I feel like I talk a lot about how Datastar is a journey of unlearning + +01:11:23 old and bad patterns, deeply rooted ones in what I think web development is. + +01:11:29 And these days, as I mentioned, I never would have thought that I'd be developing in Go, + +01:11:35 but I see like all like the, like I think even Python + +01:11:39 is getting better concurrency support, right? + +01:11:41 So I think you talked about that recently, Michael, here. + +01:11:44 So now I'm seeing with Datastar, I can do so much more on the backend. + +01:11:49 I can be so much more creative on the backend and that's what interests me. + +01:11:53 So it's just fun. + +01:11:55 What can I say? + +01:11:56 It's fun. + +01:11:57 I'm still really jealous of that presentation too. + +01:11:59 Well done with it. + +01:12:00 Yeah, certainly send me the link. + +01:12:01 I'll put it in the show notes and well done. + +01:12:03 Delaney. + +01:12:04 The irony is that like, I don't consider myself a web dev at all. + +01:12:07 It just happens to be something I do a little bit of. + +01:12:09 The thing that is the, when I first started making this public, I was like, Hey, I think + +01:12:14 I'm onto something like someone proved me wrong. + +01:12:16 I was, I'm a little bit more like kind of, I always say like in the jujitsu world type + +01:12:21 stuff, like you want someone to roll with you, not because you're trying to up them. + +01:12:24 It's that like, they're trying to help you find weaknesses in your game. + +01:12:27 Right. + +01:12:27 So I want there to be an active, like someone proved me wrong. + +01:12:31 And I'm at the point now where I feel so confident. + +01:12:34 I will put money on it. + +01:12:35 I've tried going out to people out in the dev Twitter and all that. + +01:12:38 I guarantee you, and I'm happy to put money up on this, + +01:12:41 if you could do it the Datastar way, whether you're using React or HTMLX or any other approach, + +01:12:46 it will be less code. + +01:12:47 It'll be faster. + +01:12:48 It'll be cheaper. + +01:12:49 And it'll be simpler to understand. + +01:12:52 I will take up anybody anywhere on that thing. + +01:12:55 Basically, it's not a boast. + +01:12:56 It's just the facts on the table. + +01:12:59 And it's a paradigm shift that I want the world to know about + +01:13:02 just so that people understand, hey, there's going to be someone + +01:13:05 that comes up with something better than I did, right? + +01:13:06 Like I'm standing, the reason why we have the fastest signal library in the world + +01:13:10 is because we listen to the people that are really good at that. + +01:13:12 We use alien signals. + +01:13:13 The reason why we have the fastest morphing library is that we listen to people and said, + +01:13:16 hey, there's people that care about this stuff and are working towards it. + +01:13:20 It's not that there's anything special here. + +01:13:22 It's that it's trying to build an ecosystem of like people that care about performance + +01:13:25 and people care about the details. + +01:13:27 And if you do that, then everything gets better. + +01:13:30 So it's not just where are we at now, but if anyone thinks they can do better, + +01:13:34 please join us. + +01:13:34 We want to hear it. + +01:13:35 But like, I'm done having the vibe code, or not the vibe code, + +01:13:39 but like the vibes around like, well, this doesn't feel like a spa + +01:13:42 or like a spa has its place. + +01:13:44 A couple of episodes ago, there was a Cody from the Litestar stuff said, + +01:13:48 there's a time and place for HTMX or Datastar. + +01:13:50 And he's just, that's just not true. + +01:13:52 It's just like hypermedia is the way to build things + +01:13:55 for a hypermedia client, which is the browser. + +01:13:57 So I will, anyone that can show that it's wrong, please let us know. + +01:14:01 Like we're here to help. + +01:14:02 I would love to see this paired with some Electron JS apps + +01:14:06 to make your desktop apps a little better too. + +01:14:08 So anyway. + +01:14:09 Seriously, oh my God. + +01:14:10 That's a different episode. + +01:14:11 So I just want to say thank you, Delaney, Ben and Chris. + +01:14:15 Thank you all for being here. + +01:14:16 And congrats on Datastar. + +01:14:17 It looks like a super project. + +01:14:18 I'm looking at some projects or an app running right over there + +01:14:23 on my left that I kind of wish I'd built Datastar. + +01:14:25 Well, and the thing is to make sure to not think that it's just used, like, yes, we talk + +01:14:29 about the real-time stuff, but it's better for even just normal crud stuff. + +01:14:32 And that's kind of hard to, like, it's not as sexy to talk about, but it's better at + +01:14:36 that too. + +01:14:36 Well, it's also 80% of what gets built. + +01:14:38 So it's important to like point it out, right? + +01:14:40 Yeah. + +01:14:41 All right. + +01:14:41 Bye everyone. + +01:14:42 Thanks for being here. + +01:14:42 Thank you. + +01:14:44 This has been another episode of Talk Python To Me. + +01:14:46 Thank you to our sponsors. + +01:14:47 Be sure to check out what they're offering. + +01:14:49 It really helps support the show. + +01:14:50 Take some stress out of your life. + +01:14:52 Get notified immediately about errors and performance issues in your web or mobile applications with Sentry. + +01:14:58 Just visit talkpython.fm/sentry and get started for free. + +01:15:03 Be sure to use our code talkpython26. + +01:15:06 That's Talk Python, the numbers two, six, all one word. + +01:15:11 This episode is brought to you by CommandBook, a native macOS app that I built that gives long-running terminal commands a permanent home. + +01:15:18 No more juggling six terminal tabs every morning. + +01:15:21 Carefully craft a command once, run it forever with auto restart, + +01:15:24 URL detection, and a full CLI. + +01:15:26 Download it for free at talkpython.fm/command book app. + +01:15:30 If you or your team needs to learn Python, we have over 270 hours of beginner and advanced courses + +01:15:35 on topics ranging from complete beginners to async code, + +01:15:39 Flask, Django, HTMX, and even LLMs. + +01:15:42 Best of all, there's no subscription in sight. + +01:15:45 Browse the catalog at talkpython.fm. + +01:15:47 And if you're not already subscribed to the show on your favorite podcast player, + +01:15:51 what are you waiting for? + +01:15:53 Just search for Python in your podcast player. + +01:15:55 We should be right at the top. + +01:15:56 If you enjoyed that geeky rap song, you can download the full track. + +01:15:59 The link is actually in your podcast blur show notes. + +01:16:02 This is your host, Michael Kennedy. + +01:16:03 Thank you so much for listening. + +01:16:05 I really appreciate it. + +01:16:06 I'll see you next time. + +01:16:18 I'm out. + diff --git a/transcripts/537-datastar.vtt b/transcripts/537-datastar.vtt new file mode 100644 index 0000000..c21f5d7 --- /dev/null +++ b/transcripts/537-datastar.vtt @@ -0,0 +1,4586 @@ +WEBVTT + +00:00:00.020 --> 00:00:05.060 +You love building web apps with Python, and HTMX got you excited about the hypermedia approach. + +00:00:05.720 --> 00:00:10.220 +Let the server drive the HTML, skip the JavaScript build step, keep things simple, right? + +00:00:11.020 --> 00:00:16.780 +But then you hit that last 10%. You need AlpineJS for interactivity, or your state gets out of sync, + +00:00:16.810 --> 00:00:20.780 +and suddenly you're juggling two unrelated libraries that weren't really designed to work + +00:00:20.960 --> 00:00:26.660 +together. What if there was a single 11-kilobyte framework that gave you everything HTMX and + +00:00:26.680 --> 00:00:31.760 +AlpineJS did, and more with real-time updates, multiplayer collaboration out of the box, + +00:00:31.760 --> 00:00:36.480 +and performance so fast, you're actually bottlenecked by your monitor's refresh rate. + +00:00:37.120 --> 00:00:37.840 +That's Datastar. + +00:00:38.420 --> 00:00:43.240 +On this episode, I sit down with its creator, Delany Galan, core maintainer, Ben Crocker, + +00:00:43.780 --> 00:00:50.320 +and Datastar convert, Chris May, to help explore how this backend-driven, service-sent event-first + +00:00:50.720 --> 00:00:55.160 +framework is changing the way full-stack developers think about the modern web. + +00:00:55.820 --> 00:01:01.900 +This is Talk Python To Me, episode 537, recorded January 15th, 2026. + +00:01:19.600 --> 00:01:24.360 +Welcome to Talk Python To Me, the number one Python podcast for developers and data scientists. + +00:01:24.880 --> 00:01:26.280 +This is your host, Michael Kennedy. + +00:01:26.720 --> 00:01:30.200 +I'm a PSF fellow who's been coding for over 25 years. + +00:01:30.860 --> 00:01:31.960 +Let's connect on social media. + +00:01:32.280 --> 00:01:35.440 +You'll find me and Talk Python on Mastodon, BlueSky, and X. + +00:01:35.610 --> 00:01:37.560 +The social links are all in your show notes. + +00:01:38.320 --> 00:01:41.840 +You can find over 10 years of past episodes at talkpython.fm. + +00:01:42.020 --> 00:01:45.260 +And if you want to be part of the show, you can join our recording live streams. + +00:01:45.560 --> 00:01:46.040 +That's right. + +00:01:46.250 --> 00:01:49.500 +We live stream the raw uncut version of each episode on YouTube. + +00:01:50.060 --> 00:01:54.500 +Just visit talkpython.fm/youtube to see the schedule of upcoming events. + +00:01:54.730 --> 00:01:58.380 +Be sure to subscribe there and press the bell so you'll get notified anytime we're recording. + +00:01:59.380 --> 00:02:01.000 +This episode is brought to you by Sentry. + +00:02:01.380 --> 00:02:02.620 +Don't let those errors go unnoticed. + +00:02:02.900 --> 00:02:04.420 +Use Sentry like we do here at Talk Python. + +00:02:04.920 --> 00:02:07.780 +Sign up at talkpython.fm/sentry. + +00:02:08.300 --> 00:02:13.600 +And it's brought to you by CommandBook, a native macOS app that I built that gives long-running + +00:02:13.800 --> 00:02:15.240 +terminal commands a permanent home. + +00:02:15.640 --> 00:02:17.640 +No more juggling six terminal tabs every morning. + +00:02:18.020 --> 00:02:22.900 +Carefully craft a command once, run it forever with auto restart, URL detection, and a full CLI. + +00:02:23.260 --> 00:02:26.380 +Download it for free at talkpython.fm/command book app. + +00:02:27.180 --> 00:02:29.900 +Ben, Delaney, Chris, welcome to you all. + +00:02:30.380 --> 00:02:31.560 +Thanks for being here on Talk Python To Me. + +00:02:31.680 --> 00:02:32.120 +Thanks for having us. + +00:02:32.320 --> 00:02:33.020 +Hey, how are you doing? + +00:02:33.060 --> 00:02:34.060 +Doing well, doing well. + +00:02:34.280 --> 00:02:42.040 +Very excited to talk about Datastar and some cool web frameworks for Python people and beyond, of course. + +00:02:42.220 --> 00:02:44.940 +But, you know, most people listening doing Python web frameworks. + +00:02:45.140 --> 00:02:46.980 +So talk about how that all integrates. + +00:02:47.160 --> 00:02:52.400 +And if you like the HTMX vibe, which we've talked a lot about on the show, I think there's + +00:02:52.500 --> 00:02:54.140 +going to be a lot to like here as well. + +00:02:54.190 --> 00:02:54.960 +And maybe more. + +00:02:55.140 --> 00:02:55.420 +We'll see. + +00:02:55.530 --> 00:02:56.860 +A case to be made. + +00:02:57.380 --> 00:03:02.280 +But, you know, before we get into all of that, though, let's just talk about a quick introduction + +00:03:02.660 --> 00:03:06.840 +for everyone here and like go around the squares of Ben, I'll let you go first. + +00:03:07.300 --> 00:03:07.760 +Who are you, Ben? + +00:03:08.320 --> 00:03:09.920 +Based in Costa Rica at the moment. + +00:03:10.260 --> 00:03:14.140 +I'm based in Europe most of the year, but half of the year my wife and I spend here. + +00:03:14.640 --> 00:03:26.280 +In terms of background, I've been primarily working with PHP for well over 20 years and got involved with Delaney and Datastar, been a core maintainer on that project ever since. + +00:03:26.840 --> 00:03:37.320 +And I looked at my commit history for last year, and it turns out now I write more Go code than PHP, so I don't want to call myself a PHP developer anymore. + +00:03:37.320 --> 00:03:44.220 +I'm just a web developer, a backend web developer, primarily that also writes TypeScript and maintains a front end. + +00:03:44.560 --> 00:03:49.120 +framework. There's a lot of stuff going on and ways in which you can write code for the web these days. + +00:03:50.420 --> 00:03:52.900 +Well, thanks. Awesome to have you here. Delaney, hello. + +00:03:53.060 --> 00:03:58.180 +Hi, how you doing? Yeah, I have kind of a weird checkered background into web development. I was + +00:03:58.380 --> 00:04:04.280 +originally in the circus, then I became a 3D artist, then I became an engineer. I've worked in + +00:04:04.780 --> 00:04:10.720 +games, video games, slot machines, military applications, all kinds of crazy things. + +00:04:11.140 --> 00:04:13.880 +I tend to work on really highly optimized, fast things. + +00:04:14.480 --> 00:04:16.459 +I love the ideas of the web, + +00:04:16.820 --> 00:04:20.600 +but I got really tired of how you actually implement things in that. + +00:04:20.600 --> 00:04:24.420 +And I was doing very large applications with millions of updates a second. + +00:04:24.900 --> 00:04:27.120 +And the tools that were out there just weren't good enough. + +00:04:27.320 --> 00:04:29.580 +So I ended up going down many, many rabbit holes + +00:04:30.060 --> 00:04:32.920 +and finally found something to make it better for everybody else. + +00:04:33.470 --> 00:04:34.400 +So yeah, that's really cool. + +00:04:34.480 --> 00:04:36.200 +And wow, what a really interesting issue. + +00:04:36.300 --> 00:04:38.120 +I know you got some crazy stories. + +00:04:38.420 --> 00:04:38.920 +Yes, I do. + +00:04:39.080 --> 00:04:42.400 +I always have a funny, weird outcome of something. + +00:04:42.820 --> 00:04:44.860 +Ironically, people talk about things being a circus, + +00:04:45.140 --> 00:04:47.500 +but like circuses are very well run logistic machines + +00:04:48.100 --> 00:04:50.020 +compared to most developer situations. + +00:04:50.090 --> 00:04:50.700 +So it's kind of funny. + +00:04:50.800 --> 00:04:52.060 +Yeah, it's an insult to circuses. + +00:04:52.140 --> 00:04:52.640 +Yes, it is. + +00:04:52.640 --> 00:04:53.260 +It really is. + +00:04:54.220 --> 00:04:54.560 +Amazing. + +00:04:55.040 --> 00:04:55.200 +Okay. + +00:04:55.540 --> 00:04:57.980 +And what we're going to talk about, + +00:04:58.100 --> 00:05:02.220 +Datastar has this amazing ability to update many things + +00:05:02.570 --> 00:05:05.020 +really quickly in real time, which we'll get into, + +00:05:05.260 --> 00:05:06.980 +but yeah, sort of foreshadowing there. + +00:05:07.160 --> 00:05:09.820 +And Chris May, welcome to the show. + +00:05:10.380 --> 00:05:12.600 +I've known you for a long time and I'm really happy to have you here. + +00:05:12.740 --> 00:05:13.980 +Great to be here. Thank you so much. + +00:05:14.180 --> 00:05:14.380 +Yeah. + +00:05:14.520 --> 00:05:25.120 +Yeah. So about me, I started writing websites back in 1995 and then picked up Python about 10 or so years later and just have really enjoyed the ride since then. + +00:05:25.700 --> 00:05:30.220 +Picked along the way, became technical coach and just loved making single page applications. + +00:05:30.560 --> 00:05:32.220 +I loved, I just love the web. + +00:05:32.400 --> 00:05:36.340 +You know, I love that we can publish something from our computer and anybody around the world can see it. + +00:05:36.620 --> 00:05:40.800 +And then what, maybe a little over a year ago, I, oh no, it was more than that. + +00:05:40.860 --> 00:05:46.300 +I remember I was on a trip and I was listening to a podcast of HXPod, the HTMX podcast, + +00:05:46.820 --> 00:05:50.220 +and heard about this crazy, cool tool, Datastar. + +00:05:50.220 --> 00:05:54.520 +And I was like, I even put in my DjangoCon presentation, like you should, everybody else + +00:05:54.640 --> 00:05:55.120 +should try it out. + +00:05:55.150 --> 00:05:56.820 +And finally I did and I'm converted. + +00:05:56.910 --> 00:05:57.560 +I love it. + +00:05:57.770 --> 00:06:00.200 +So I'm excited that the three of us get to talk about it. + +00:06:00.240 --> 00:06:05.919 +The reason that we're having this podcast is because I read your article about switching + +00:06:05.940 --> 00:06:10.760 +to Datastar. And I'm like, okay, this is interesting. You made the case very well. Of + +00:06:10.760 --> 00:06:18.000 +course, I'll link to the article. And so I thought, hey, I need to have Chris here as my Tony Romo to + +00:06:18.120 --> 00:06:24.520 +my Al Michaels or Nico Rosberg to my Crofty or whatever, right? So I'm happy to have you here. + +00:06:24.760 --> 00:06:30.380 +Exactly. Awesome to have you here. So let's just start with what is Datastar, right? I mean, + +00:06:30.420 --> 00:06:37.220 +we've hinted that it has some similarities to htmx but also not so ben and delaney give us the + +00:06:37.430 --> 00:06:41.920 +overview what is datastar so i can give a little bit of history and then ben's probably better at + +00:06:42.140 --> 00:06:47.720 +saying what it is now i have a background in like low-level stuff um even though i was a 3d artist + +00:06:47.900 --> 00:06:52.660 +first i'm much more comfortable in like shader development and that kind of thing like so glsl + +00:06:52.770 --> 00:06:57.939 +web thing like i'm a c guy that knows some other things but the thing is that i was working on + +00:06:57.960 --> 00:07:03.140 +some military applications where I needed really fast updates of a browser. And the reason why you + +00:07:03.300 --> 00:07:09.240 +in this military situation is that getting things approved is really hard, like executables to go + +00:07:09.400 --> 00:07:13.060 +into deployment. But having a browser means that you have this nice little sandbox that things can + +00:07:13.070 --> 00:07:17.160 +go in. So it's actually more of a deployment platform in my background than, you know, just + +00:07:17.160 --> 00:07:21.700 +the regular web. But I was doing things that were pushing the browser really, really far. I was using + +00:07:21.840 --> 00:07:26.300 +Vue and Spa. And I basically was like, well, these are the smartest people out here, but it's not + +00:07:26.320 --> 00:07:31.160 +fast enough. So I was using crazy WebSocket stuff, all this binary stuff. And then I tried doing, + +00:07:31.620 --> 00:07:36.440 +you had someone on last week talking about LiveView and like they have a Python version of that. I went + +00:07:37.180 --> 00:07:40.820 +hard in making a binary version of that, like going down to the protocol level, changing, + +00:07:41.180 --> 00:07:44.600 +optimizing that 10 different ways. I had an entire framework for doing this. And basically, + +00:07:45.030 --> 00:07:48.840 +in my opinion, that's a complete dead end. It is untenable. We can go into the reasons why, + +00:07:49.080 --> 00:07:53.319 +but the thing is, long story short, I ended up seeing what was happening in HTMLX in the hyper + +00:07:53.340 --> 00:07:58.060 +media space. And I completely discounted all of that because I said, like, I'm doing low level + +00:07:58.200 --> 00:08:02.420 +binary stuff. There's no way this other approach can be faster. And then my thing is always check + +00:08:02.430 --> 00:08:07.000 +the metrics, always don't take your assumptions and do the work. And the thing is, there's things + +00:08:07.070 --> 00:08:11.080 +that are wrong in the implementation, but there's things that are 100% right in the overall ideas of + +00:08:11.360 --> 00:08:15.520 +how to use that. So I went and I took a year and a half work and just threw it in the trash + +00:08:16.100 --> 00:08:21.839 +and said, okay, I'm starting over and like ended up doing some basic things would probably get into + +00:08:22.160 --> 00:08:27.120 +and ended up with this thing that is a backend diagnostic backend framework that has a 10 + +00:08:27.280 --> 00:08:33.140 +kilobyte shim that is the fastest, smallest thing out there by orders of magnitude. So it's not just + +00:08:33.190 --> 00:08:37.039 +a slightly different thing. It is literally a different paradigm shift. It's a crazy shift. + +00:08:37.070 --> 00:08:41.940 +So the difference between reacts to something like HTMX is different from HTMX to the data + +00:08:42.080 --> 00:08:46.620 +start way. So I'll let Ben actually explain what that is. But the thing is from a low level C + +00:08:46.680 --> 00:08:51.120 +guys point of view, it is one of the fastest things in your stack now, which is crazy to think + +00:08:51.120 --> 00:08:52.940 +Yeah, like a 10K shim can do that. + +00:08:53.000 --> 00:08:53.600 +That's incredible. + +00:08:54.180 --> 00:08:58.340 +And also, it sounds like your advice comes from somebody who's done a lot of profiling. + +00:08:59.120 --> 00:08:59.760 +Very much so. + +00:08:59.880 --> 00:09:00.660 +Like, that's the only thing. + +00:09:00.760 --> 00:09:02.900 +You got to measure, not guess. + +00:09:03.160 --> 00:09:03.260 +Yeah. + +00:09:03.370 --> 00:09:06.940 +In fact, there's funny things that we've had things on Twitter fighting with people and + +00:09:06.940 --> 00:09:08.880 +they're like, oh, this one situation was really slow. + +00:09:09.350 --> 00:09:14.420 +We actually looked at their flangraphs and it was a bug in the Safari GPU stuff. + +00:09:14.770 --> 00:09:17.500 +Because we were actually at the level where the JavaScript doesn't even show up. + +00:09:17.580 --> 00:09:20.580 +It's actually a GPU issue of it rendering fast stuff. + +00:09:20.940 --> 00:09:22.640 +in the browser, nothing to do with the JavaScript. + +00:09:23.030 --> 00:09:25.320 +Because the fastest JavaScript you can write is no JavaScript. + +00:09:25.860 --> 00:09:28.140 +So we really lean into what the browser can already do. + +00:09:28.540 --> 00:09:30.220 +And we're just making it so that that's easy to do + +00:09:30.270 --> 00:09:32.380 +so that the average person with the average website + +00:09:32.960 --> 00:09:34.540 +doesn't have to write any JavaScript at all. + +00:09:34.810 --> 00:09:36.320 +And they get to be a full stack developer + +00:09:36.370 --> 00:09:37.340 +in whatever language they choose. + +00:09:37.780 --> 00:09:39.980 +And I'll let everyone else talk from there. + +00:09:40.040 --> 00:09:40.760 +Awesome. Ben? + +00:09:41.020 --> 00:09:43.560 +Yeah, my version is going to be quite different to Delaney's + +00:09:43.800 --> 00:09:45.800 +because we care about different things. + +00:09:45.980 --> 00:09:48.260 +Fortunately, we do care about some of the same things. + +00:09:49.000 --> 00:09:51.620 +We work well together because I think we complement each other. + +00:09:52.270 --> 00:09:57.280 +But coming from a PHP background, I want the backend to be driving the front end. + +00:09:57.290 --> 00:09:58.640 +And it naturally does, right? + +00:09:58.840 --> 00:10:02.580 +Because even your HTML is being produced by your backend. + +00:10:02.610 --> 00:10:04.280 +And that's what's being served to the front end. + +00:10:05.100 --> 00:10:08.420 +I describe Datastar as a hypermedia framework. + +00:10:09.240 --> 00:10:16.660 +And some people get tripped up on what hypermedia is, but it's essentially hypertext with other media like images and CSS and that kind of thing. + +00:10:16.740 --> 00:10:21.440 +And everybody should know what hypertext is because it's the H in HTTP and HTML. + +00:10:22.060 --> 00:10:33.840 +There is an expectation for people coming into Datastar that you have a basic understanding of the web and web browsers and the web browser API because we lean as heavily as possible on the browser API. + +00:10:34.340 --> 00:10:39.060 +We get a lot of people coming into the Discord asking us, you know, how should I do this the Datastar way? + +00:10:39.080 --> 00:10:44.860 +And it got to the point where I'd heard that question so often I decided, OK, I'm going to write a page in the Datastar docs. + +00:10:45.280 --> 00:10:46.940 +We call it the tau of datastar. + +00:10:47.110 --> 00:10:49.160 +So it's kind of like the way of datastar. + +00:10:49.520 --> 00:10:54.300 +And if there's one thing to take from that, it's use as little datastar as possible. + +00:10:54.720 --> 00:10:58.100 +Like leverage the browser, because the browser is an incredible thing, right? + +00:10:58.100 --> 00:11:01.920 +Like it's basically an operating system, our operating system as web developers. + +00:11:02.580 --> 00:11:05.500 +So, and everything happens at the C level, super optimized. + +00:11:06.100 --> 00:11:08.120 +We're not going to be able to build something faster. + +00:11:08.400 --> 00:11:11.480 +So leverage the browser as much as possible on the browser APIs. + +00:11:12.280 --> 00:11:19.200 +And where HTML kind of lacks or where there are some gaps, that's essentially what Datastar is trying to fill. + +00:11:19.680 --> 00:11:21.380 +So I did a lot of work. + +00:11:21.580 --> 00:11:26.760 +So just to relate this, I guess, to something that other people might be familiar with, which is HTMLX. + +00:11:27.380 --> 00:11:33.440 +I was an early contributor to HTMLX, actually, and I was sold on the idea of hypermedia from the very beginning. + +00:11:33.800 --> 00:11:35.740 +So HTML is the language of the web. + +00:11:36.100 --> 00:11:38.340 +Why are we trying to replace it with JavaScript? + +00:11:39.000 --> 00:11:46.740 +And the problem that I ran into after several years of thinking HTMX is all I need is that last 10%, right? + +00:11:46.910 --> 00:11:49.980 +Because it'll get you 90% of what you're trying to do. + +00:11:50.290 --> 00:11:56.280 +But that last 10%, which we all know is the hardest piece that takes the most work, just isn't covered. + +00:11:56.680 --> 00:12:02.059 +So with HTMX, for example, you will very often reach for another library like AlpineJS, + +00:12:02.760 --> 00:12:09.300 +or you'll start writing vanilla JS perhaps to fill in those gaps to interactivity to the page, + +00:12:09.390 --> 00:12:12.860 +because HTMX is really just going to the back end, replacing the DOM. + +00:12:13.270 --> 00:12:15.240 +But now you have two dependencies. + +00:12:15.820 --> 00:12:19.920 +Now you have HTMX and Alpine, for example, and you're trying to make those play well together. + +00:12:20.420 --> 00:12:24.880 +And because I think that might be a little bit of the missing sauce from HTMX. + +00:12:24.980 --> 00:12:28.200 +I've had Carson Gross on and I really admire HTMX. + +00:12:28.700 --> 00:12:34.920 +But as I've worked with it over a couple of years, I feel like it's really good as salt or seasoning, + +00:12:35.020 --> 00:12:38.200 +something you sprinkle on to really make a website better. + +00:12:38.660 --> 00:12:41.300 +But if you try to make a meal out of salt, you're not going to want to eat it. + +00:12:41.480 --> 00:12:46.940 +And what I mean is, you have three different disjointed parts of the page, + +00:12:47.030 --> 00:12:51.280 +and you're like, this is so amazing to update this with HTML and partials, and so is that. + +00:12:51.980 --> 00:12:55.940 +But then you start talking about AlpineJS and connecting different things, + +00:12:56.140 --> 00:12:59.340 +and then the JavaScript gets out of sync with this server response. + +00:12:59.590 --> 00:13:02.880 +And it just, you start to feel constrained by it. + +00:13:03.260 --> 00:13:05.400 +And I think you all have a really nice solution. + +00:13:05.560 --> 00:13:08.560 +It's something a little bit like how you, we're going to talk about it, + +00:13:08.560 --> 00:13:13.040 +but sort of how you specify the HTML to be updated by the server, + +00:13:13.580 --> 00:13:16.380 +but then also connecting different parts of the pages. + +00:13:16.940 --> 00:13:20.900 +Chris put it in his article that like the problem is AlpineJS and HTMX + +00:13:20.930 --> 00:13:24.220 +are just two unrelated different things that happen to go together a lot. + +00:13:24.280 --> 00:13:26.600 +And so they're not cohesive in a sense, right? + +00:13:26.760 --> 00:13:28.540 +Well, and that's one thing that's definitely an issue. + +00:13:28.880 --> 00:13:30.520 +Like, for example, this was my thing + +00:13:30.680 --> 00:13:33.500 +because I actually tried to fix HTMX back in the day. + +00:13:33.760 --> 00:13:35.040 +And like the things that I wanted to fix + +00:13:35.440 --> 00:13:37.280 +were the problem that I see at least + +00:13:37.400 --> 00:13:39.060 +is that you have HTMX, you can add, + +00:13:39.200 --> 00:13:41.340 +it has extensions, so you can add stuff to it. + +00:13:41.700 --> 00:13:43.900 +But it fundamentally was built to be like, + +00:13:44.200 --> 00:13:45.660 +here's our way of doing it. + +00:13:45.920 --> 00:13:47.540 +And then you can do your own stuff on top of it. + +00:13:47.740 --> 00:13:51.040 +The problem is, is that I thought that's broken. + +00:13:51.260 --> 00:13:52.180 +I've done enough game development + +00:13:52.240 --> 00:13:53.200 +to know that you need to be agile. + +00:13:53.300 --> 00:13:55.100 +I need to be able to like be able to move quickly. + +00:13:55.560 --> 00:14:01.180 +So I wanted it so that nothing was basically like the core of data star is like 300 lines + +00:14:01.270 --> 00:14:01.380 +long. + +00:14:01.670 --> 00:14:06.580 +And it is basically setting up data dash star elements, hooking up plugins, and then everything + +00:14:06.690 --> 00:14:07.220 +else is a plugin. + +00:14:07.530 --> 00:14:12.160 +So if you don't agree with us, or if someone's better than I am, great, that's wonderful. + +00:14:12.470 --> 00:14:15.120 +We will be able to just pop that part out, put the new part in. + +00:14:15.500 --> 00:14:17.060 +But plugins can now depend on each other. + +00:14:17.110 --> 00:14:17.520 +They can understand. + +00:14:17.800 --> 00:14:18.500 +It's an ecosystem. + +00:14:18.960 --> 00:14:20.660 +Ironically, that's what happens under the hood. + +00:14:21.080 --> 00:14:23.140 +But the ideas of that make it so much more powerful. + +00:14:23.200 --> 00:14:34.120 +And the irony is that if you build it in that kind of plugin style way, in the more game developer style way, we are smaller than HTMLX and Alpine alone, let alone combined, let alone Hyperscript and all these other things. + +00:14:34.320 --> 00:14:36.700 +So it's just a different way of thinking about the problem. + +00:14:36.840 --> 00:14:46.720 +When I first encountered Datastar and looked at the source code, it looked very foreign to me because Delaney coming from game development, he built Datastar like a game engine. + +00:14:47.160 --> 00:14:52.420 +So you have this very thin core and then everything else pretty much is a plugin. + +00:14:52.480 --> 00:14:58.680 +And all Datastar core is a way for registering plugins and having Datastar attributes. + +00:14:59.130 --> 00:15:00.380 +And that's pretty much it. + +00:15:00.600 --> 00:15:03.580 +Everything else is an add-on that you, is a plugin that you can take away. + +00:15:03.710 --> 00:15:08.020 +So we even have a bundler on the site that allows you to just, well, you can just download + +00:15:08.560 --> 00:15:11.660 +Datastar core or you can just select what plugins you want. + +00:15:12.060 --> 00:15:16.160 +Now, that in and of itself is not that interesting because we're, at the end of the day, we're + +00:15:16.200 --> 00:15:19.300 +talking about a 10 kilobyte JavaScript file with all of the plugins. + +00:15:19.840 --> 00:15:21.720 +But it is open source, which we didn't mention. + +00:15:21.760 --> 00:15:24.580 +And so anybody can go just kind of look at it if you're interested. + +00:15:25.080 --> 00:15:29.700 +But that approach means that everything is modular and everything is there for a reason. + +00:15:30.320 --> 00:15:32.180 +And we'll get into this later, I guess. + +00:15:32.230 --> 00:15:36.760 +But like deciding what plugins go in and what stay out is one of the challenges. + +00:15:36.890 --> 00:15:38.940 +And we just try to keep it as lean as possible. + +00:15:39.400 --> 00:15:43.860 +My way of thinking about it is that Datastar gives you everything you need and nothing you don't. + +00:15:44.240 --> 00:15:47.400 +And that's how we try to kind of keep it lean and fast. + +00:15:48.680 --> 00:15:51.060 +This portion of Talk Python Maze brought to you by Sentry. + +00:15:51.400 --> 00:15:56.500 +I've been using Sentry personally on almost every application and API that I've built for + +00:15:56.660 --> 00:15:59.120 +Talk Python and beyond over the last few years. + +00:15:59.480 --> 00:16:02.400 +They're a core building block for keeping my infrastructure solid. + +00:16:03.000 --> 00:16:04.300 +They should be for yours as well. + +00:16:04.580 --> 00:16:04.960 +Here's why. + +00:16:05.620 --> 00:16:07.060 +Sentry doesn't just catch errors. + +00:16:07.140 --> 00:16:09.840 +It catches all the stuff that makes your app feel broken. + +00:16:10.160 --> 00:16:14.580 +The random slowdown, the freeze you can't reproduce, that bug that only shows up once + +00:16:14.880 --> 00:16:15.660 +real users hit it. + +00:16:15.940 --> 00:16:19.440 +And when something goes wrong, Sentry gives you the whole chain of events in one place. + +00:16:19.680 --> 00:16:22.640 +errors, traces, replays, logs, dots connected. + +00:16:22.980 --> 00:16:24.860 +You can see what's led to the issue + +00:16:25.100 --> 00:16:26.820 +without digging through five different dashboards. + +00:16:27.640 --> 00:16:29.660 +Seer, Sentry's AI debugging agent, + +00:16:30.180 --> 00:16:32.120 +builds on this data, taking the full context, + +00:16:32.800 --> 00:16:34.840 +explaining why the issue happened, + +00:16:35.360 --> 00:16:37.720 +pointing to the code responsible, drafts a fix, + +00:16:37.800 --> 00:16:40.780 +and even flags if your PR is about to introduce a new problem. + +00:16:41.620 --> 00:16:42.660 +The workflow stays simple. + +00:16:43.100 --> 00:16:44.860 +Something breaks, Sentry alerts you, + +00:16:45.020 --> 00:16:46.820 +the dashboard shows you the full context. + +00:16:47.160 --> 00:16:49.640 +Seer helps you fix it and catch new issues + +00:16:49.660 --> 00:16:55.400 +before they ship. It's totally reasonable to go from an error occurred to fixed in production in + +00:16:55.600 --> 00:17:01.160 +just 10 minutes. I truly appreciate the support that Sentry has given me to help solve my bugs + +00:17:01.320 --> 00:17:06.620 +and issues in my apps, especially those tricky ones that only appear in production. I know you will + +00:17:06.640 --> 00:17:12.020 +too if you try them out. So get started today with Sentry. Just visit talkpython.fm/sentry + +00:17:12.540 --> 00:17:19.620 +and get $100 in Sentry credits. Please use that link. It's in your podcast player show notes. If + +00:17:19.640 --> 00:17:25.839 +our code talkpython26, all one word talkpython26 to get $100 in credits. + +00:17:26.740 --> 00:17:28.300 +Thank you to Sentry for supporting the show. + +00:17:29.600 --> 00:17:29.760 +Cool. + +00:17:29.800 --> 00:17:33.240 +That's a super interesting philosophy to say you should be able to take, even take + +00:17:33.310 --> 00:17:35.580 +stuff out of what we're giving you by default, right? + +00:17:35.850 --> 00:17:42.100 +Now, before we move on from sort of introducing Datastar, I do want to point out at data-star.dev, + +00:17:42.220 --> 00:17:45.420 +which of course I'll link this notes, there's some cool examples on here. + +00:17:45.720 --> 00:17:53.260 +You've got a really nice space 2001 sort of theme with Hal and all that, which is great. + +00:17:53.920 --> 00:17:56.260 +I like the aesthetic here, which is very fun. + +00:17:56.800 --> 00:18:00.620 +It's got a little bit of a retro gaming feel, which is nice. + +00:18:00.880 --> 00:18:03.400 +But what I want to point out is I want to encourage people to go watch your little video. + +00:18:03.960 --> 00:18:04.800 +Your video is fun. + +00:18:05.520 --> 00:18:06.360 +It's really fun. + +00:18:06.480 --> 00:18:11.160 +This video is all about how Datastar fits in the world of SPAs. + +00:18:11.920 --> 00:18:16.480 +And one thing we didn't really mention is that Datastar is a full-fledged SPA replacement. + +00:18:17.020 --> 00:18:26.140 +So again, like that last 10%, often people will think, oh, well, I need to go to React or Vue.js or some single page application framework. + +00:18:26.340 --> 00:18:34.640 +Whereas we're saying that, no, no, no, Datastar will not only, it's not like a subset or like SPAs are not a superset. + +00:18:34.680 --> 00:18:41.620 +it's on the contrary. I think Datastark, we think Datastark can do more than SBAs because we are + +00:18:42.180 --> 00:18:46.300 +driven by the backend and we are focused on hypermedia, which is the language of the web. + +00:18:46.640 --> 00:18:51.480 +So this, yeah, so this video is kind of throwing, yeah, anyway, everybody should watch it. + +00:18:51.800 --> 00:18:56.840 +I'd also like to, if you can scroll back up to the top of the page, the Starfield animation was + +00:18:57.040 --> 00:19:02.460 +one of the things like when Delaney and when everybody who worked on this published it, + +00:19:02.780 --> 00:19:04.700 +Like I didn't realize how amazing this was + +00:19:04.810 --> 00:19:07.460 +because if you like right click and inspect that thing, + +00:19:07.960 --> 00:19:08.740 +it's a web component. + +00:19:09.230 --> 00:19:11.100 +And so all the JavaScript that's required + +00:19:11.300 --> 00:19:13.540 +for making all the stars go faster and slower + +00:19:13.760 --> 00:19:16.180 +and tracking your mouse where, you know, wherever you do it, + +00:19:16.600 --> 00:19:18.280 +it's all within that web component. + +00:19:18.520 --> 00:19:21.180 +And data star is essentially subscribing to like, + +00:19:21.260 --> 00:19:22.020 +where's the mouse pointer + +00:19:22.440 --> 00:19:24.120 +and passing it into the web component. + +00:19:24.380 --> 00:19:26.520 +Yeah, in fact, if you go to more examples, + +00:19:27.220 --> 00:19:29.260 +you will see that there's, + +00:19:29.360 --> 00:19:31.240 +and then go scroll down to, + +00:19:31.720 --> 00:19:32.780 +or use the hamburger thing. + +00:19:32.850 --> 00:19:35.200 +Yeah, go down to the rocket. + +00:19:35.980 --> 00:19:37.740 +There's the actual star field. + +00:19:38.110 --> 00:19:39.940 +So you can see the entire, the star field, + +00:19:40.010 --> 00:19:41.700 +the entire component is there. + +00:19:41.790 --> 00:19:43.020 +So if you scroll down from there, + +00:19:43.320 --> 00:19:45.100 +you'll see how it actually gets hooked up + +00:19:45.460 --> 00:19:46.680 +and the entire component, + +00:19:46.900 --> 00:19:48.200 +that's the whole thing, it's right there. + +00:19:48.340 --> 00:19:48.840 +- That's incredible. + +00:19:49.200 --> 00:19:50.580 +- And the thing is if you start moving around, + +00:19:51.060 --> 00:19:52.820 +like if you scroll up just a little bit more, + +00:19:52.870 --> 00:19:54.280 +so you can see the sliders, + +00:19:54.690 --> 00:19:56.160 +you'll see that they're live, everything's, + +00:19:56.360 --> 00:19:57.080 +if you move it around, + +00:19:57.540 --> 00:19:59.240 +like you move your mouse around the canvas, + +00:19:59.460 --> 00:20:00.860 +you'll see everything's live editing, + +00:20:01.040 --> 00:20:01.820 +everything's thing. + +00:20:02.040 --> 00:20:03.800 +It's the irony of Datastar. + +00:20:03.800 --> 00:20:05.840 +And this is the part that I don't think people quite get. + +00:20:06.260 --> 00:20:08.100 +And it's not that you're trying to like, + +00:20:08.500 --> 00:20:10.160 +we love what Carson has done with HCMS. + +00:20:10.320 --> 00:20:11.680 +We love that all the things they've done, + +00:20:11.800 --> 00:20:13.120 +but it does not do everything. + +00:20:13.460 --> 00:20:13.980 +It doesn't do enough. + +00:20:14.080 --> 00:20:15.140 +It is a library, not a framework. + +00:20:15.560 --> 00:20:16.120 +And the thing is, + +00:20:16.360 --> 00:20:18.180 +the irony is that Datastar actually has + +00:20:18.520 --> 00:20:19.880 +the fastest reactive signal, + +00:20:21.060 --> 00:20:21.860 +like reactive signals. + +00:20:22.200 --> 00:20:23.700 +We are the fastest thing out there. + +00:20:23.980 --> 00:20:25.420 +So it's not just like we did something + +00:20:25.520 --> 00:20:26.300 +that's kind of like VDOM, + +00:20:26.360 --> 00:20:26.980 +or we are like, + +00:20:27.180 --> 00:20:28.160 +we can compete with React. + +00:20:28.220 --> 00:20:30.220 +We demolish them with actual numbers. + +00:20:30.860 --> 00:20:35.160 +So we have the fastest morphing strategy and we also have the fastest signals, which means doing these kinds of things. + +00:20:35.420 --> 00:20:35.980 +It's just a non-issue. + +00:20:36.010 --> 00:20:37.960 +Like this star field thing is 1K. + +00:20:38.460 --> 00:20:42.740 +Like it's just these are the kinds of things that are just a non-issue in this if you do things our way. + +00:20:42.860 --> 00:20:52.320 +And you're leaning into the web ecosystem by leveraging web components instead of having to like build, have a build time pipeline to, you know, do all the custom JavaScript. + +00:20:53.000 --> 00:20:57.380 +Like once I realized like you can do these things, it just made, it just clicked. + +00:20:57.540 --> 00:21:03.900 +And I just make it's I feel like it's so much more fun now to work on the web now that I understand these things. + +00:21:04.180 --> 00:21:06.200 +Let's talk through some of the core examples. + +00:21:06.450 --> 00:21:11.600 +I feel like there's some similarities to the example section of the HTMX place. + +00:21:11.700 --> 00:21:14.520 +But, you know, HTMX doesn't have a star field, certainly. + +00:21:14.880 --> 00:21:16.840 +Best place to start is on the homepage. + +00:21:17.470 --> 00:21:26.720 +Before we get into those examples, just just to kind of take a step back and say, OK, we've mentioned HTMX a few times and we don't we don't even like to compare ourselves to HTMX. + +00:21:26.880 --> 00:21:29.100 +but it is a good maybe starting point for some people. + +00:21:29.370 --> 00:21:31.220 +We have a hello world example there, + +00:21:31.640 --> 00:21:32.300 +if you could find that. + +00:21:32.360 --> 00:21:33.680 +Yeah, let's scroll down just a little bit more. + +00:21:33.840 --> 00:21:34.520 +Yeah, you got it. + +00:21:34.820 --> 00:21:38.200 +One of the maybe differences between HTMX and Datastar + +00:21:38.200 --> 00:21:40.900 +is that Datastar can receive HTML responses, + +00:21:41.050 --> 00:21:42.400 +but it also by default, + +00:21:42.720 --> 00:21:45.520 +or the recommendation is to use server sent events. + +00:21:46.020 --> 00:21:47.260 +So if you hit start there, + +00:21:47.660 --> 00:21:50.240 +you're going to see kind of the network response tab, + +00:21:50.310 --> 00:21:51.580 +and those are server sent events. + +00:21:51.840 --> 00:21:55.179 +And SSE server sent events are an old technology + +00:21:55.200 --> 00:21:56.740 +that work just over HTTP. + +00:21:57.480 --> 00:22:00.120 +And essentially what happens is that the server + +00:22:00.360 --> 00:22:01.960 +holds a connection open to the browser + +00:22:02.140 --> 00:22:03.520 +and it's unidirectional. + +00:22:03.820 --> 00:22:05.720 +So you send a request to the server + +00:22:05.960 --> 00:22:08.820 +and then the server can stream events back down, + +00:22:08.980 --> 00:22:09.980 +which is what you're seeing here. + +00:22:10.300 --> 00:22:12.160 +Now, this is obviously a trivial example, right? + +00:22:12.240 --> 00:22:12.900 +We're sending one, + +00:22:13.360 --> 00:22:16.040 +or we're updating the message one character at a time. + +00:22:16.600 --> 00:22:19.300 +But when you see how simple this is, + +00:22:19.320 --> 00:22:22.600 +then you can perhaps see potential for this, right? + +00:22:22.660 --> 00:22:29.720 +And SSE or service end events have had kind of a renaissance in recent years with all of the LLMs, right? + +00:22:29.750 --> 00:22:33.740 +All the chatbots are streaming the responses back to you. + +00:22:33.970 --> 00:22:42.680 +So this type of technology, while it's not old, sorry, it's not new, it's actually been around a long time, has kind of been underused. + +00:22:43.660 --> 00:22:50.940 +And Delaney kind of tapped into that and said, well, because I also always thought, well, if I want pure reactivity or true reactivity, + +00:22:50.960 --> 00:22:52.840 +I need two-way communication. + +00:22:53.020 --> 00:22:53.720 +So I need web sockets. + +00:22:53.720 --> 00:22:54.560 +You need web sockets. + +00:22:54.650 --> 00:22:56.660 +You need binary and all that kind of stuff. + +00:22:56.820 --> 00:22:56.940 +Yeah. + +00:22:57.120 --> 00:22:57.220 +Yeah. + +00:22:57.460 --> 00:22:59.440 +There's problems with those, which we can get into. + +00:23:00.200 --> 00:23:01.240 +SSE is much simpler. + +00:23:01.470 --> 00:23:04.300 +It works over HTTP 1, 2, and 3. + +00:23:05.120 --> 00:23:06.620 +And as you can see, it's just plain text. + +00:23:06.780 --> 00:23:08.280 +There is no complicated handshake. + +00:23:08.700 --> 00:23:12.480 +If you change the interval to zero and hit start, + +00:23:13.260 --> 00:23:15.740 +you're going to see a different type of response, which is, + +00:23:16.060 --> 00:23:17.860 +and I don't know if you saw the content type change, + +00:23:18.160 --> 00:23:19.820 +but content type now is text HTML. + +00:23:20.210 --> 00:23:20.580 +Oh, intro. + +00:23:20.700 --> 00:23:21.320 +Oh, interesting. + +00:23:21.520 --> 00:23:21.620 +Yeah. + +00:23:22.070 --> 00:23:24.300 +So this is what HTMLX would do by default. + +00:23:24.410 --> 00:23:28.140 +You send back HTML responses, whereas here the content type + +00:23:28.320 --> 00:23:29.620 +is text event stream. + +00:23:30.250 --> 00:23:32.900 +And this allows you to hold that connection open + +00:23:33.370 --> 00:23:34.300 +for as long as you want. + +00:23:34.300 --> 00:23:39.400 +It can be open and closed, or it can stay open until the words + +00:23:39.540 --> 00:23:41.420 +hello world have been spelled out. + +00:23:41.940 --> 00:23:43.660 +Or you can keep it open indefinitely. + +00:23:44.100 --> 00:23:46.720 +So we're going to see some more advanced examples + +00:23:46.740 --> 00:23:50.400 +where the SSE connection is held open for longer. + +00:23:50.680 --> 00:23:53.880 +So I think wrapping your head around this example + +00:23:55.260 --> 00:23:57.600 +taps you into the potential of Datastar. + +00:23:57.620 --> 00:23:57.980 +MARK MANDEL: Yeah. + +00:23:58.280 --> 00:23:59.880 +And one of the things that-- + +00:24:00.020 --> 00:24:03.040 +well, when I looked at Datastar, I'm like, OK, + +00:24:03.740 --> 00:24:05.320 +there's some interesting aspects here. + +00:24:05.320 --> 00:24:08.220 +And we'll get into them, how you can set up-- + +00:24:08.660 --> 00:24:10.860 +when I click the Start button, it might replace + +00:24:10.960 --> 00:24:12.900 +a piece of the page-- hey, that sounds familiar-- + +00:24:13.280 --> 00:24:15.460 +with HTML, not through JavaScript, right? + +00:24:15.900 --> 00:24:21.140 +but it didn't specify anywhere what part of the page to replace or not. + +00:24:21.180 --> 00:24:22.480 +Like, how does it know? + +00:24:22.940 --> 00:24:26.960 +And so with Datastar, you lean more on the server for many things, + +00:24:28.000 --> 00:24:32.220 +including deciding what part of the page that the server created in the first place to update. + +00:24:32.300 --> 00:24:33.420 +I really like that. + +00:24:33.480 --> 00:24:35.220 +I think that that's super neat. + +00:24:35.580 --> 00:24:39.440 +It lets you not just have sort of closer to one source of truth, + +00:24:39.660 --> 00:24:43.400 +but also just you can pass down multiple things. + +00:24:43.480 --> 00:24:45.720 +is like, we need to update this pane on the right, + +00:24:46.280 --> 00:24:50.100 +this text, and this element all in one response. + +00:24:50.640 --> 00:24:52.240 +There's a lot of interesting aspects + +00:24:52.480 --> 00:24:54.520 +to what you're talking about here. + +00:24:54.560 --> 00:24:56.860 +JOHN MCWHORTER: Anyone who's familiar with out-of-band swaps + +00:24:56.920 --> 00:24:59.000 +in HTMX, well, guess what? + +00:24:59.160 --> 00:25:00.640 +Datastar is out-of-band by default. + +00:25:01.480 --> 00:25:03.880 +So it's matching currently based on the ID. + +00:25:04.600 --> 00:25:06.920 +So you see h3 id equals message. + +00:25:07.280 --> 00:25:10.300 +And every event that's coming back has an ID of message. + +00:25:10.940 --> 00:25:11.820 +But guess what? + +00:25:11.960 --> 00:25:13.680 +you can use any ID you want, right? + +00:25:13.790 --> 00:25:16.760 +So you can use actually any CSS selector you want. + +00:25:16.790 --> 00:25:19.700 +But yes, we put the onus more on the backend + +00:25:20.120 --> 00:25:23.160 +because that is where we believe state should live + +00:25:23.500 --> 00:25:26.720 +or that's the source of truth for state. + +00:25:27.300 --> 00:25:30.320 +And you send and you work with state on the front end + +00:25:30.660 --> 00:25:32.680 +only when and where it makes sense to, + +00:25:33.040 --> 00:25:34.840 +which is more the web component aspect. + +00:25:35.080 --> 00:25:36.920 +And I'll caveat what Ben said there + +00:25:36.970 --> 00:25:39.960 +in that like state mostly lives in the backend. + +00:25:40.460 --> 00:25:43.380 +And that's the problem is that like state lives where it lives. + +00:25:43.500 --> 00:25:46.280 +Like if the user is actively able to move their mouse cursor, + +00:25:46.540 --> 00:25:48.420 +they own that state of the mouse cursor. + +00:25:48.560 --> 00:25:49.340 +You don't own that, + +00:25:49.620 --> 00:25:52.240 +but most of the state from your database should be in the backend. + +00:25:52.780 --> 00:25:56.660 +The one thing that's interesting about the SSE compared to how most people + +00:25:56.700 --> 00:25:59.060 +think this stuff, I will say I fell into this trap too, right? + +00:25:59.060 --> 00:26:03.500 +Cause I did the live view crazy stuff is that your job as a web developer is to + +00:26:03.560 --> 00:26:06.700 +get strings to the browser as efficiently, as fast as possible. + +00:26:06.840 --> 00:26:09.160 +Cause like the browser is going to deal with that, + +00:26:09.340 --> 00:26:14.640 +that into html and all that there's nothing faster than giving it html right so the thing that i i + +00:26:14.690 --> 00:26:20.440 +know i lost for a long time is that sse i thought oh it's this big string thing how is that better + +00:26:20.580 --> 00:26:24.900 +than binary but the irony is that because it's so regular because there's already things like + +00:26:25.000 --> 00:26:28.560 +compression built into the browser there's streaming things there's things that are so + +00:26:28.660 --> 00:26:33.720 +much easier to do here in an efficient way that the irony is if you if you you don't have to care + +00:26:33.800 --> 00:26:38.520 +about all these things but if you just follow our way of doing it your python app will be faster + +00:26:38.540 --> 00:26:40.200 +than most people's like compiled, + +00:26:40.660 --> 00:26:42.080 +you know, like low level language thing + +00:26:42.200 --> 00:26:43.800 +because you're getting orders of magnitude + +00:26:44.030 --> 00:26:44.640 +in the algorithms + +00:26:44.770 --> 00:26:46.280 +and how we're doing stuff from the hood. + +00:26:46.420 --> 00:26:47.620 +So I don't know if you're interested + +00:26:47.710 --> 00:26:48.980 +in like the deep down stuff + +00:26:49.180 --> 00:26:51.000 +or just like how you use it as a Python developer. + +00:26:51.030 --> 00:26:53.540 +But the irony is that you now have tapped into this. + +00:26:53.670 --> 00:26:54.420 +It seems so simple. + +00:26:54.580 --> 00:26:56.260 +You're like, oh, this is just a different text response. + +00:26:56.880 --> 00:26:58.480 +How can this be orders of magnitude faster? + +00:26:58.860 --> 00:26:59.960 +Like, again, I don't know how much + +00:26:59.970 --> 00:27:00.920 +you want to get into the weeds of that + +00:27:01.280 --> 00:27:03.180 +compared to just it's fun to use, right? + +00:27:03.340 --> 00:27:05.440 +Yeah, I really like the philosophy + +00:27:05.580 --> 00:27:07.780 +of having so much of it controlled by the server. + +00:27:08.280 --> 00:27:09.920 +It just felt disheartening. + +00:27:10.160 --> 00:27:11.480 +It's like, okay, so what you're going to do + +00:27:11.480 --> 00:27:13.780 +is you're just going to create some JSON responses + +00:27:14.780 --> 00:27:15.620 +on your server, + +00:27:15.730 --> 00:27:18.880 +and then everything is some crazy build series of steps + +00:27:19.420 --> 00:27:21.120 +to end up with, I don't know, + +00:27:21.400 --> 00:27:23.160 +Vue or React or something on the front end. + +00:27:23.250 --> 00:27:26.420 +And there's just so much power and flexibility + +00:27:26.900 --> 00:27:29.060 +to write really cool server code. + +00:27:29.260 --> 00:27:31.140 +But, you know, like a lot of the trends have been, + +00:27:31.680 --> 00:27:34.680 +yeah, that's kind of just there to support the rest of it, + +00:27:34.750 --> 00:27:37.120 +you know, and so I don't know, this really appeals to me. + +00:27:37.500 --> 00:27:43.600 +question that comes up often is like, OK, well, how do I format this? Because it has its own syntax. + +00:27:44.040 --> 00:27:48.620 +Very simple to read, obviously, right? An event name and then these data lines. And you can just + +00:27:48.670 --> 00:27:54.460 +have as many data lines as you want. And that's your HTML. If you scroll up, though, we do have... + +00:27:54.730 --> 00:28:01.240 +So you do need to format this, but we essentially have all of these SDKs, including Python, you'll + +00:28:01.320 --> 00:28:06.660 +see there. And the Python SDK is actually, I would say, one of the most intricate ones we have. + +00:28:07.240 --> 00:28:12.900 +Spatuel King, he's a member of the community, or Chase, I believe is his first name, his real first + +00:28:13.020 --> 00:28:18.340 +name, and many other contributors did an amazing job on that. So lots and lots of Python frameworks + +00:28:18.520 --> 00:28:24.120 +are supported. You can maybe speak more to this, Chris. And really, the SDKs are very simple, + +00:28:24.360 --> 00:28:29.780 +because all they do is they take a function, a patch elements or patch signals function, + +00:28:30.160 --> 00:28:34.840 +and you just dump in the HTML that you want swapped into the DOM or the signals you want + +00:28:34.820 --> 00:28:39.680 +output on the page and it just does the formatting for you so so it's really just there's three + +00:28:39.900 --> 00:28:45.120 +functions i think in total that every sdk has to implement and it's such a time saver you know um + +00:28:45.620 --> 00:28:52.260 +i doved into service and events a lot with htmx and when you get the syntax wrong it is so painful + +00:28:52.680 --> 00:28:58.840 +to debug because pretty much can't it just doesn't work you know or whatever it's harder to debug and + +00:28:58.920 --> 00:29:04.780 +so to have the helper syntax it's just a dream well and also just so people are aware i like + +00:29:04.800 --> 00:29:06.060 +because I was originally going to try, + +00:29:06.270 --> 00:29:07.980 +the irony is I was trying to get server sent events + +00:29:08.320 --> 00:29:10.760 +like their plugin up to snuff like years ago. + +00:29:11.240 --> 00:29:14.580 +Like I would highly recommend not using SSE with HTMLX + +00:29:14.860 --> 00:29:16.800 +because the problem is that the entire model + +00:29:16.830 --> 00:29:18.260 +of how you build things is very poll based + +00:29:18.720 --> 00:29:19.940 +and it's built out of band. + +00:29:19.970 --> 00:29:20.900 +It's like a weird concept, + +00:29:21.010 --> 00:29:21.940 +like the idea of updating, + +00:29:22.050 --> 00:29:23.800 +like it is not built with that in mind. + +00:29:24.130 --> 00:29:25.680 +So I know that they're trying to move towards that + +00:29:25.770 --> 00:29:26.180 +in the future, + +00:29:26.440 --> 00:29:28.800 +but the whole way that you interact with it + +00:29:28.900 --> 00:29:29.700 +is based on polling. + +00:29:30.180 --> 00:29:31.780 +And the thing about our way + +00:29:31.900 --> 00:29:33.300 +is that not only are you doing push events, + +00:29:33.540 --> 00:29:36.700 +But the thing is that really does change the semantics of the language. + +00:29:36.900 --> 00:29:40.020 +So first of all, you get like 40X compression by doing our way. + +00:29:40.230 --> 00:29:43.040 +But also you only send data when you need to instead of polling. + +00:29:43.330 --> 00:29:44.600 +So now you're using less resources. + +00:29:45.060 --> 00:29:45.960 +You're using less network. + +00:29:46.520 --> 00:29:54.620 +It changes the whole dynamic in a deeper way that you can literally save 5,000X in your network bandwidth. + +00:29:54.900 --> 00:29:57.300 +It sounds crazy, but it's just a reality. + +00:29:57.540 --> 00:29:57.640 +Right. + +00:29:57.900 --> 00:30:00.200 +Another thing, Delaney, that's really nice about that is the latency. + +00:30:00.900 --> 00:30:03.060 +That's something that drives me crazy about polling. + +00:30:03.400 --> 00:30:09.060 +general is just like okay well we don't want to hammer the server too hard so let's make this you + +00:30:09.060 --> 00:30:13.420 +know one second two second but then it's like well i click this button and then it updates and you're + +00:30:13.460 --> 00:30:18.000 +like ah if if something happens on the server it's sent right if it wants to one of the things that + +00:30:18.120 --> 00:30:23.040 +ironically because i do a lot of like go or low level language stuff is that i tend to put a debounce + +00:30:23.380 --> 00:30:29.780 +in my server to like five milliseconds so that i get i'm not updating more than you know 200 times + +00:30:30.100 --> 00:30:36.500 +a second even on a monitor because the browser actually break after 500 fps so like the + +00:30:36.700 --> 00:30:40.380 +interesting thing is not that it's basically data starts no longer the issue in your thing if you + +00:30:40.520 --> 00:30:47.460 +are on a low low powered battery device like a mobile on a 3g this is it will just work like it's + +00:30:47.700 --> 00:30:52.560 +stuff that you just don't have to worry about so it does change the semantics of how you build things + +00:30:52.980 --> 00:30:57.140 +just so that you're aware because even things like for example built into the htmx they don't do + +00:30:57.220 --> 00:30:59.300 +automatic exponential back off. + +00:30:59.500 --> 00:31:00.920 +It doesn't have all the verbs. + +00:31:01.320 --> 00:31:03.180 +There's caveats there that I would recommend + +00:31:03.250 --> 00:31:04.960 +not doing it, honestly, if you're going to do it. + +00:31:05.000 --> 00:31:07.260 +It's crazy that you're talking about going below + +00:31:07.330 --> 00:31:08.720 +the monitor refresh rate. + +00:31:09.020 --> 00:31:11.020 +You're not going to see it. This is only + +00:31:11.710 --> 00:31:12.160 +120 hertz. + +00:31:13.740 --> 00:31:14.800 +120 times a second. + +00:31:15.150 --> 00:31:16.700 +So why would you pull + +00:31:17.020 --> 00:31:18.700 +faster than that? That's wild. + +00:31:20.480 --> 00:31:20.960 +This portion + +00:31:21.010 --> 00:31:22.980 +of Talk Python To Me is brought to you by us. + +00:31:23.320 --> 00:31:24.920 +I'm thrilled to announce a brand + +00:31:25.020 --> 00:31:26.560 +new app built for developers + +00:31:26.580 --> 00:31:27.840 +created by yours truly. + +00:31:28.260 --> 00:31:29.460 +It's called Command Book. + +00:31:30.280 --> 00:31:31.520 +You know that thing you do every morning? + +00:31:32.020 --> 00:31:33.460 +Open up six terminal tabs, + +00:31:33.860 --> 00:31:34.700 +CD into this directory, + +00:31:35.300 --> 00:31:36.340 +activate that virtual environment, + +00:31:36.820 --> 00:31:38.280 +run the server with --reload. + +00:31:38.580 --> 00:31:39.620 +Now, CD somewhere else, + +00:31:40.280 --> 00:31:41.280 +start the background worker, + +00:31:41.820 --> 00:31:42.540 +another tab for Docker, + +00:31:42.900 --> 00:31:44.260 +another one to tail production logs. + +00:31:44.720 --> 00:31:46.740 +Every tab just says Python, Python, Python, + +00:31:47.020 --> 00:31:47.440 +Docker tail. + +00:31:48.600 --> 00:31:49.480 +And you're clicking through them going, + +00:31:49.940 --> 00:31:51.100 +which Python was that again? + +00:31:51.660 --> 00:31:52.360 +Where my app is running? + +00:31:52.920 --> 00:31:53.860 +Then sometime later, + +00:31:54.140 --> 00:31:55.960 +your dev server silently dies + +00:31:55.980 --> 00:31:58.660 +because it tried to reload while you're in the middle of a code edit, + +00:31:59.320 --> 00:32:01.880 +unmatched brace, a half-written import, or something. + +00:32:02.640 --> 00:32:05.180 +Now you're hunting through tabs to figure out which process crashed + +00:32:05.270 --> 00:32:06.120 +and how to restart it. + +00:32:06.600 --> 00:32:11.140 +My app, CommandBook, gives all of these long-running commands a permanent home. + +00:32:11.700 --> 00:32:14.500 +You save a command once, the working directory, the environment, + +00:32:14.860 --> 00:32:17.940 +free commands like git pull, and from then on, you just click run. + +00:32:18.500 --> 00:32:21.260 +You can even group commands together to start and stop everything + +00:32:21.370 --> 00:32:22.840 +for a project with a single click. + +00:32:23.360 --> 00:32:26.960 +It also has what I call Honey Badger Mode, auto restart on crash. + +00:32:27.540 --> 00:32:30.060 +So when your dev server goes down mid-reload, + +00:32:30.520 --> 00:32:34.720 +Command Book just brings it right back up and does so over and over until the code is fixed. + +00:32:35.340 --> 00:32:37.140 +It also detects URLs from your output, + +00:32:37.400 --> 00:32:39.720 +so you're never scrolling through thousands of lines of logs + +00:32:39.810 --> 00:32:41.860 +just to figure out how to reopen your web app. + +00:32:42.400 --> 00:32:46.280 +And it shows you uptime, memory usage, and all sorts of cool things about your process. + +00:32:46.980 --> 00:32:48.960 +The whole thing is a native macOS app. + +00:32:49.160 --> 00:32:51.520 +No electron, no Chromium, just 21 megs. + +00:32:51.900 --> 00:32:53.280 +And it comes with a full CLI. + +00:32:53.520 --> 00:32:55.360 +So anything you've configured in the UI, + +00:32:55.680 --> 00:32:58.220 +you can fire off from your terminal with just a single command. + +00:32:58.760 --> 00:33:01.900 +Right now it's macOS only, but if there's enough interest, + +00:33:02.140 --> 00:33:03.200 +I'll build a Windows version too. + +00:33:03.460 --> 00:33:04.160 +So let me know. + +00:33:05.000 --> 00:33:08.980 +Please check it out at talkpython.fm/command book app, + +00:33:09.460 --> 00:33:11.980 +download it for free, level up your developer workflow. + +00:33:12.440 --> 00:33:14.040 +The link is in your podcast player show notes. + +00:33:14.640 --> 00:33:16.620 +That's talkpython.fm/command book. + +00:33:16.980 --> 00:33:18.820 +I really hope you enjoy this new app that I built. + +00:33:20.180 --> 00:33:28.140 +Yeah, on the topic of latency and all that, if you go to the examples, there's some we could look at that I think really demonstrate this. + +00:33:28.180 --> 00:33:32.740 +Well, maybe start with bad Apple just because we're talking about refresh rates. + +00:33:33.120 --> 00:33:33.180 +OK. + +00:33:33.600 --> 00:33:39.140 +What's happening is that the back end is streaming down just a bunch of symbols, but it creates this animation. + +00:33:39.520 --> 00:33:44.020 +And if you were to open the network tab, you would see it actually would be interesting to see. + +00:33:44.220 --> 00:33:46.100 +You probably have to refresh the page just to. + +00:33:46.880 --> 00:33:47.000 +Yeah. + +00:33:47.660 --> 00:33:48.660 +And you're going to see updates. + +00:33:49.160 --> 00:33:49.680 +That one there. + +00:33:49.980 --> 00:33:51.060 +Yeah, if you click that-- + +00:33:51.100 --> 00:33:51.600 +This one? + +00:33:52.160 --> 00:33:52.880 +Yeah, that's the one. + +00:33:53.100 --> 00:33:53.960 +You click Event Stream. + +00:33:54.500 --> 00:33:56.780 +There's an Event Stream tab for Event Stream responses. + +00:33:57.540 --> 00:33:58.500 +You're going to see these streaming. + +00:33:58.610 --> 00:34:01.800 +I don't know what frames per second we have this set to, + +00:34:01.920 --> 00:34:03.160 +but you see it streaming past, right? + +00:34:03.340 --> 00:34:03.460 +Right. + +00:34:03.640 --> 00:34:05.920 +The first time many people see this, + +00:34:05.980 --> 00:34:09.360 +this is a surprise that the browser is capable of this. + +00:34:09.460 --> 00:34:11.179 +But the browser can stream video, so why + +00:34:11.320 --> 00:34:12.659 +can't it stream a bunch of text? + +00:34:12.860 --> 00:34:14.679 +I mean, it's not that big of a leap of faith. + +00:34:15.060 --> 00:34:18.280 +But you can see, it looks like it's about every 10, 20 + +00:34:18.500 --> 00:34:18.740 +milliseconds. + +00:34:19.100 --> 00:34:23.720 +I think we're doing like 30 frames a second, but again, we can do like, we're doing this on a, + +00:34:24.040 --> 00:34:28.419 +basically a free tier server. So like, this is just a non-issue and it's doing all the compression + +00:34:28.550 --> 00:34:32.620 +stuff. So if you notice that your update, even though we're doing like full ASCII development at, + +00:34:32.820 --> 00:34:36.820 +you know, thousands of characters, your updates are actually not updating that. Like you see how + +00:34:37.040 --> 00:34:41.200 +it's transferring, but it's not transferring that much compared to how much it's actually coming out. + +00:34:41.460 --> 00:34:46.420 +I can see we got 1.9 megs for the whole page. Yeah. But do you see next to it? What, what was + +00:34:46.440 --> 00:34:51.580 +actually like the resources so you see the compression well yeah we're probably not seeing + +00:34:51.580 --> 00:34:55.780 +it there but in the bottom you'll see two two megabytes have been transferred but 10 megabytes + +00:34:55.980 --> 00:35:01.080 +of resources and so that's oh yeah yeah so it's a 5x compression yeah it's going to be much more + +00:35:01.320 --> 00:35:06.760 +on the stream i think because it's streaming uh normally you can hover over the size + +00:35:07.520 --> 00:35:13.380 +and you'll see the uncompressed but i guess it's changing too fast that's pretty wild and you know + +00:35:13.240 --> 00:35:19.020 +in practical usage, like I have a status screen that I have from my production app at work. And + +00:35:19.100 --> 00:35:24.660 +it's just amazing to just constantly be seeing these things update. And I'm doing that by having + +00:35:24.860 --> 00:35:31.140 +the database tell my Python code, hey, refresh. I actually ask it to get all the entries from the + +00:35:31.220 --> 00:35:35.520 +database and send it down the pipe. And so it's not like I'm doing the optimized thing. I'm doing + +00:35:35.640 --> 00:35:40.660 +the simple thing and I get all these cool things just updating all the time. And it's just such a + +00:35:40.720 --> 00:35:44.060 +useful thing, especially for status screens, dashboards, stuff like that. + +00:35:44.140 --> 00:35:49.880 +Speaking of that, go to the DB Mon example. This is one of my favorites because when React + +00:35:50.340 --> 00:35:54.680 +first had their first conference, they said, look at what we're doing. We're able to update at a rate + +00:35:54.760 --> 00:35:59.640 +that no one else can compete with us in how fast they could update the browser, right? If we + +00:35:59.760 --> 00:36:03.460 +actually, yeah, you're still there. So the thing is, if you actually set the FPS to something like + +00:36:03.840 --> 00:36:10.360 +80, whatever. So that is how fast it's coming from the backend to you. So go ahead. Yeah, + +00:36:10.520 --> 00:36:12.620 +because we just don't want people blasting the server. + +00:36:12.740 --> 00:36:13.760 +Yeah, you don't want to walk away. + +00:36:14.240 --> 00:36:16.400 +Yeah, but the point is that this is coming. + +00:36:16.820 --> 00:36:19.520 +See, we're doing stuff in microseconds on a potato. + +00:36:19.780 --> 00:36:21.800 +Yeah, let me just describe this a little bit for people listening. + +00:36:22.000 --> 00:36:25.180 +So it's like a database monitoring table + +00:36:25.360 --> 00:36:29.200 +that shows you how many transactions are the database overloads. + +00:36:29.580 --> 00:36:33.600 +So it's updating a grid of maybe 10 or 12 databases + +00:36:34.180 --> 00:36:35.880 +with five or six elements, + +00:36:36.280 --> 00:36:38.860 +and it's doing that in microseconds, 80 times a second. + +00:36:38.960 --> 00:36:43.480 +A lot of people see these examples and they think, well, I'm not building this kind of stuff. + +00:36:43.880 --> 00:36:44.800 +And me included. + +00:36:44.960 --> 00:36:46.560 +I build crud apps most of the time. + +00:36:47.400 --> 00:36:51.500 +And there are plenty of examples here that are just cruddy things. + +00:36:51.940 --> 00:36:53.480 +They're kind of the more boring examples. + +00:36:53.940 --> 00:36:57.640 +But one example that might be worth looking at is the to-do MVC. + +00:36:58.000 --> 00:37:01.520 +And if you can figure out how to open that in split screen. + +00:37:01.960 --> 00:37:02.120 +Okay. + +00:37:02.560 --> 00:37:04.560 +What part do you want me to open up in split? + +00:37:04.660 --> 00:37:05.680 +Oh, just this, the example. + +00:37:05.880 --> 00:37:09.560 +Yeah, so I can do these two and then I can tile them. + +00:37:09.820 --> 00:37:10.140 +How's that? + +00:37:10.320 --> 00:37:15.440 +So this is a CRUD app, but what Datastar gives you is the ability to do multiplayer out of the box. + +00:37:15.710 --> 00:37:22.320 +And that is like real-time collaborative apps are not easy to do and not easy to scale as well. + +00:37:22.570 --> 00:37:27.620 +But as you'll see here, when you have like two sessions open, it's going to be near instant. + +00:37:27.820 --> 00:37:31.340 +You're going to basically be observing the latency on your network connection, + +00:37:31.620 --> 00:37:35.680 +which is going to be 50 milliseconds to 100, but barely perceivable. + +00:37:35.860 --> 00:37:40.360 +So just to describe to people, we've got this 2D MVC, which allows you to, well, it's like a + +00:37:40.440 --> 00:37:46.200 +to-do example, which is required to be a legitimate JavaScript framework. But I've opened it in two + +00:37:46.360 --> 00:37:51.000 +tabs and I've used Vivaldi's tile. So these are legitimately two browsers. They just appear to be + +00:37:51.140 --> 00:37:57.060 +kind of in the same window. And when I enter stuff into it, it literally looks like they update in + +00:37:57.280 --> 00:38:00.320 +parallel, which is crazy. If you check a few of them, you'll see, + +00:38:00.420 --> 00:38:02.920 +You can barely tell which one's updating which. + +00:38:03.110 --> 00:38:04.520 +It happens almost instantly. + +00:38:04.920 --> 00:38:07.380 +Yeah, if I look at the other one and I click on one, + +00:38:07.600 --> 00:38:09.980 +it feels like that's responding to my click. + +00:38:10.420 --> 00:38:13.000 +I need to correct myself that it is happening instantly + +00:38:13.230 --> 00:38:15.500 +because when you click, when you check one of those, + +00:38:15.700 --> 00:38:19.420 +it's not, and this is an interesting thing we can get into. + +00:38:19.620 --> 00:38:21.260 +We're not doing optimistic updates. + +00:38:21.700 --> 00:38:23.680 +It's actually sending a request to the server + +00:38:24.160 --> 00:38:26.560 +and the server is simultaneously updating + +00:38:27.200 --> 00:38:29.400 +both of your tabs at the same time. + +00:38:29.480 --> 00:38:31.000 +Even if I had just one open, + +00:38:31.500 --> 00:38:33.140 +it's still going round trip to the server. + +00:38:33.480 --> 00:38:34.780 +That's why it looks like it's simultaneous + +00:38:34.990 --> 00:38:35.820 +because it's effectively. + +00:38:35.980 --> 00:38:36.940 +This is a thing that you can, + +00:38:37.360 --> 00:38:38.540 +we can talk for like three hours + +00:38:38.840 --> 00:38:40.640 +and I will yell at most spa developers + +00:38:41.439 --> 00:38:43.020 +because there's this weird thing + +00:38:43.030 --> 00:38:43.980 +that because it's easy, + +00:38:44.460 --> 00:38:47.340 +people will actively lie to users in the spa world + +00:38:47.590 --> 00:38:48.920 +and they'll do optimistic updates, + +00:38:49.030 --> 00:38:50.640 +which means I'm going to make it + +00:38:50.650 --> 00:38:52.020 +so that I'm making this change. + +00:38:52.200 --> 00:38:54.300 +And then if there's a problem, then fix it. + +00:38:54.660 --> 00:38:56.700 +Whereas we say you should do indicator saying, + +00:38:56.880 --> 00:38:58.400 +I'm trying to make a change to this + +00:38:58.840 --> 00:38:59.280 +and then fix it. + +00:38:59.360 --> 00:39:02.600 +Because you don't want, like when you're playing a video game, you can do what's called dead + +00:39:02.780 --> 00:39:05.400 +reckoning and you can do some stuff to net rollback code. + +00:39:05.620 --> 00:39:10.540 +You can do some clever things to hide latency, but you don't want to hide latency when it + +00:39:10.540 --> 00:39:14.060 +comes to like a bank transfer or did I buy that thing or did I get that theater ticket + +00:39:14.280 --> 00:39:15.220 +or any of that stuff. + +00:39:15.620 --> 00:39:18.680 +Like people just have the wrong mental model of how the web should work. + +00:39:19.080 --> 00:39:23.360 +I'm actually going to send you another thing that this might blow your mind even more because + +00:39:23.640 --> 00:39:25.240 +the three of us basically can play. + +00:39:25.480 --> 00:39:35.060 +This is an example where all of us could be playing live with each other right now in an active shared state that's been at the top of the Hacker News and again, runs on a potato. + +00:39:35.170 --> 00:39:37.160 +I don't know if you just put that in your... + +00:39:37.180 --> 00:39:38.080 +Yeah, let me drop it over. + +00:39:38.190 --> 00:39:38.440 +Hold on. + +00:39:38.720 --> 00:39:39.620 +I'm going to put it in the other tab. + +00:39:39.860 --> 00:39:41.800 +So right now, don't touch anything. + +00:39:42.240 --> 00:39:43.180 +I'm going to actively start. + +00:39:43.240 --> 00:39:43.680 +I am purple. + +00:39:43.950 --> 00:39:45.740 +I am literally starting to click right now. + +00:39:45.840 --> 00:39:45.980 +All right. + +00:39:46.030 --> 00:39:48.740 +So we're looking at a multiplayer game of life here. + +00:39:48.940 --> 00:39:50.100 +I'm seeing that live here. + +00:39:50.340 --> 00:39:54.500 +And if you open up that in the other tab, you would actively see the exact same state. + +00:39:54.880 --> 00:39:57.960 +So everyone in the world, like if you open that up in the other tab, + +00:39:58.360 --> 00:39:59.860 +you cannot get out of sync. + +00:40:00.020 --> 00:40:01.280 +It's not faking it in the front end. + +00:40:01.320 --> 00:40:02.300 +This is literally sending in. + +00:40:02.520 --> 00:40:04.700 +What's even crazier about this, here's the crazy part. + +00:40:04.760 --> 00:40:05.960 +It's actually a rendering demo. + +00:40:06.360 --> 00:40:09.000 +The guy who wrote it is writing in a scripting language, Clojure, + +00:40:09.400 --> 00:40:14.500 +and he's sending down 2,500 divs per frame styled, inline styled. + +00:40:14.840 --> 00:40:17.480 +Now go to your network tab now and look at what's actually, + +00:40:17.580 --> 00:40:21.720 +like look at your network tab and you'll see how little data we're actually sending over. + +00:40:21.860 --> 00:40:24.420 +even though he's updating 2,500 divs per frame, + +00:40:24.880 --> 00:40:26.380 +like if you go to wherever it's updated, + +00:40:26.600 --> 00:40:28.400 +yeah, whichever one's the one that's updated, + +00:40:28.450 --> 00:40:29.180 +there you go, yeah. + +00:40:29.540 --> 00:40:31.940 +So if you look here and look at how much is being sent + +00:40:32.070 --> 00:40:33.260 +versus how many is actually, + +00:40:33.870 --> 00:40:36.380 +like try to, like, this is just a different paradigm + +00:40:36.700 --> 00:40:37.380 +for how you build. + +00:40:37.440 --> 00:40:38.040 +And the thing is, again, + +00:40:38.600 --> 00:40:40.380 +not everybody has to care about these low level things, + +00:40:40.430 --> 00:40:42.240 +but the thing is, is that once you do this, + +00:40:42.450 --> 00:40:43.980 +the idea of CRUD kind of goes away + +00:40:44.110 --> 00:40:46.520 +because in our opinion, you go to a multi, + +00:40:46.600 --> 00:40:49.240 +you make a multi-page app like you would normally do + +00:40:49.360 --> 00:40:50.220 +in HTMX or anything else, + +00:40:50.440 --> 00:40:51.620 +but you keep an open stream + +00:40:52.180 --> 00:40:54.360 +and you just update whatever's happening in your backend + +00:40:54.840 --> 00:40:55.400 +as it's happening. + +00:40:55.940 --> 00:40:56.980 +And it simplifies the world. + +00:40:57.080 --> 00:40:58.020 +And what's also interesting + +00:40:58.520 --> 00:40:59.880 +is because of how we do compression and all that, + +00:41:00.160 --> 00:41:01.480 +you just send your entire page. + +00:41:01.820 --> 00:41:03.100 +You don't need like out of band. + +00:41:03.360 --> 00:41:04.420 +It doesn't even really make sense + +00:41:04.680 --> 00:41:06.220 +because we're so fast that you can just, + +00:41:06.560 --> 00:41:07.680 +you as a Python developer, + +00:41:08.140 --> 00:41:09.300 +you just give us your entire page + +00:41:09.480 --> 00:41:10.420 +and let us deal with it. + +00:41:10.600 --> 00:41:11.900 +And we will come up with the fast stuff. + +00:41:12.320 --> 00:41:14.540 +So Chris should probably talk a lot more to that + +00:41:14.620 --> 00:41:15.620 +because the fat morpher stuff, + +00:41:15.720 --> 00:41:18.520 +it's a fundamental change in how you build web apps, I think. + +00:41:18.640 --> 00:41:18.980 +Yeah, yeah. + +00:41:19.180 --> 00:41:21.960 +especially the kind of the mental shift of like, + +00:41:22.270 --> 00:41:25.360 +because I kept thinking, okay, I need to like send one row at a time. + +00:41:25.840 --> 00:41:27.680 +And I actually have one status screen that does that + +00:41:27.810 --> 00:41:32.180 +because we use a Firestore, Google's Firestore as our backend for this app. + +00:41:32.700 --> 00:41:35.600 +But for some reason, sometimes it just doesn't send every update. + +00:41:36.040 --> 00:41:38.040 +And so on another status screen, I actually, you know, + +00:41:38.320 --> 00:41:41.960 +query the whole database table or collection and send it down to Pipe. + +00:41:42.180 --> 00:41:45.640 +And because sometimes it doesn't send from Firestore, + +00:41:45.840 --> 00:41:50.760 +I get the entire latest state of all the things that are in flight and updated on my screen. + +00:41:51.200 --> 00:41:52.400 +And it just makes things easier. + +00:41:52.580 --> 00:41:53.220 +Yeah, it's amazing. + +00:41:53.680 --> 00:41:57.540 +Sounds like a good opportunity to subscribe to database query changes. + +00:41:57.780 --> 00:42:03.280 +I know some databases you can say, if this query updates, you know, trigger this event and then keep it flowing, + +00:42:03.600 --> 00:42:06.640 +like straight from events on the database, straight to your front end. + +00:42:06.980 --> 00:42:07.220 +Pretty cool. + +00:42:07.500 --> 00:42:12.060 +I do want to go back and just put a little bit of commentary, Delaney. + +00:42:12.240 --> 00:42:14.160 +Well, you said optimistic updates. + +00:42:15.040 --> 00:42:19.940 +So one of the things that's really common in JavaScript is I click this thing, it changes. + +00:42:20.310 --> 00:42:21.860 +I want to mark it as changed. + +00:42:21.990 --> 00:42:24.500 +And then I'm going to tell the server, hey, we made this change. + +00:42:25.060 --> 00:42:28.860 +It's very possible the server died, that you're not allowed to make that change or whatever. + +00:42:29.050 --> 00:42:30.800 +And then you've got to come back and go actually undo that. + +00:42:30.880 --> 00:42:32.760 +That really, you know, there's like a weird. + +00:42:33.240 --> 00:42:35.400 +So what you're saying is you don't have to worry about that kind of stuff. + +00:42:35.540 --> 00:42:36.920 +We're a framework, not just a library. + +00:42:37.340 --> 00:42:41.700 +The idea is that you have these indicators that not only basically your indicators drive a signal. + +00:42:41.850 --> 00:42:43.660 +Like, again, the details don't really matter. + +00:42:43.760 --> 00:42:45.720 +But the idea is that you have instantaneous, + +00:42:46.320 --> 00:42:48.460 +like within the same frame updates of, + +00:42:48.760 --> 00:42:49.860 +hey, I'm going off to do something. + +00:42:50.020 --> 00:42:52.040 +Like usually you make a spinner or you say, + +00:42:52.360 --> 00:42:53.680 +I'm going to do this to gray out the field + +00:42:53.960 --> 00:42:54.420 +or I'm going to do, + +00:42:54.620 --> 00:42:55.800 +like there's all kinds of things you can drive. + +00:42:55.980 --> 00:42:59.240 +Because again, the state of what the local stuff is + +00:42:59.600 --> 00:43:02.060 +while the change is there, that lives in the client. + +00:43:02.360 --> 00:43:03.960 +Like that is part of Datastar. + +00:43:04.360 --> 00:43:06.180 +It has all the right tools to make it + +00:43:06.180 --> 00:43:08.080 +so that you can disable it or gray it out + +00:43:08.280 --> 00:43:10.200 +or say, I'm going to put a spinner next to it. + +00:43:10.280 --> 00:43:11.400 +Like you can do all those things. + +00:43:11.900 --> 00:43:13.320 +But the thing is you're not lying to your user. + +00:43:13.700 --> 00:43:14.340 +That's my whole thing. + +00:43:14.380 --> 00:43:15.780 +And people say, well, that's not really a lie. + +00:43:15.860 --> 00:43:16.440 +It's like, yes, it is. + +00:43:16.560 --> 00:43:17.560 +You're literally lying to people. + +00:43:18.060 --> 00:43:18.740 +Like, please stop. + +00:43:18.920 --> 00:43:20.240 +It's a DX issue. + +00:43:20.720 --> 00:43:23.780 +The reason why people do it is because it's convenient, not because it's correct. + +00:43:24.020 --> 00:43:26.020 +And again, like you can do optimistic updates. + +00:43:26.200 --> 00:43:29.120 +You can do SBA-like things using Datastar. + +00:43:29.220 --> 00:43:34.500 +We don't recommend it because Datastar is more than just this tech. + +00:43:34.760 --> 00:43:36.300 +It's also like a way of doing things. + +00:43:37.060 --> 00:43:42.860 +What I wanted to point out here is that you might imagine that, you know, this is something that when you click edit, + +00:43:43.120 --> 00:43:44.520 +it turns it into a form. + +00:43:44.520 --> 00:43:47.240 +So you might like load the form into the page, hide the form, + +00:43:47.460 --> 00:43:48.880 +and then just do a show hide approach. + +00:43:49.920 --> 00:43:53.060 +But the hypermedia approach is kind of like the REST approach + +00:43:53.400 --> 00:43:56.380 +where you can only take the next action at any given time. + +00:43:57.060 --> 00:43:58.300 +So if you open the network tab, + +00:43:58.500 --> 00:44:00.480 +I just want to kind of walk you through this briefly. + +00:44:00.480 --> 00:44:02.280 +If you cancel that, when you hit edit, + +00:44:02.820 --> 00:44:04.680 +you will see a network request to the server. + +00:44:04.840 --> 00:44:06.900 +And what comes back is that form. + +00:44:07.220 --> 00:44:10.380 +So it's real time as in like what you're seeing now + +00:44:10.400 --> 00:44:12.960 +is the actual state reflected on the backend. + +00:44:13.570 --> 00:44:16.120 +And when you save, you're also going to see the same thing. + +00:44:16.480 --> 00:44:18.340 +You're going to see a network request to the server + +00:44:18.730 --> 00:44:21.440 +and it gets the true current state + +00:44:22.010 --> 00:44:24.340 +as it has been saved is now all comes back down. + +00:44:24.800 --> 00:44:27.620 +So it's like you don't even need optimistic updates + +00:44:28.060 --> 00:44:28.600 +most of the time. + +00:44:28.880 --> 00:44:30.820 +And when you do use it, + +00:44:30.880 --> 00:44:33.000 +it's because you're trying to cover up poor performance. + +00:44:33.620 --> 00:44:35.520 +You're favoring perceived performance + +00:44:36.070 --> 00:44:36.980 +over true performance. + +00:44:37.280 --> 00:44:39.480 +One of the things I hear a lot is people saying, + +00:44:39.760 --> 00:44:40.820 +but it's so much slower. + +00:44:41.350 --> 00:44:43.160 +But I think people are used to + +00:44:43.250 --> 00:44:45.160 +or think it's much slower than it is + +00:44:45.420 --> 00:44:47.680 +because the web, the spa life + +00:44:47.880 --> 00:44:50.440 +that we see around us feels so slow. + +00:44:51.020 --> 00:44:53.640 +But anytime I've seen people try to lean + +00:44:53.900 --> 00:44:55.160 +into just using the network, + +00:44:55.660 --> 00:44:57.120 +it's so much faster than you expect. + +00:44:57.340 --> 00:44:59.520 +Well, and also you have so much less our way. + +00:45:00.000 --> 00:45:02.860 +Your usage of network can be easily 100x less, + +00:45:03.360 --> 00:45:04.460 +which means you have less contention, + +00:45:04.900 --> 00:45:06.160 +which means when you do send something, + +00:45:06.600 --> 00:45:07.460 +it's there immediately. + +00:45:07.490 --> 00:45:09.140 +And also because you're not doing polling + +00:45:09.420 --> 00:45:13.280 +with polling, you have to send to the server and the server sends back. If you just send from server + +00:45:13.600 --> 00:45:18.880 +when something updates, now you've just halved your RTT, right? Your round trip has just halved. + +00:45:19.060 --> 00:45:23.640 +So you half it and you're doing like a thousand less of something. All of a sudden, things opened + +00:45:23.760 --> 00:45:28.780 +up for you in weird ways, right? It's a fundamentally different way of thinking about the problem. + +00:45:29.259 --> 00:45:33.660 +Another example that we like to bring up a lot is there was a while back, someone did a million + +00:45:33.820 --> 00:45:37.980 +checkbox demo and they had a whole write-up on it, right? And they basically had to take it down + +00:45:38.000 --> 00:45:39.220 +because it was just too expensive to run. + +00:45:39.720 --> 00:45:41.720 +We have a version that's not just checkboxes, + +00:45:41.840 --> 00:45:43.060 +but color checkboxes. + +00:45:43.060 --> 00:45:44.660 +So you can actually make ASCII art and stuff like that. + +00:45:45.020 --> 00:45:45.680 +And it's a billion. + +00:45:46.160 --> 00:45:47.220 +And it runs on the same server + +00:45:47.600 --> 00:45:48.940 +that was running that Game of Life demo. + +00:45:49.280 --> 00:45:49.860 +It's on the same server. + +00:45:50.300 --> 00:45:52.380 +It's actively, and it's been on top of Hacker News. + +00:45:52.460 --> 00:45:54.640 +It's a $5 VPS as far as I know. + +00:45:54.700 --> 00:45:55.700 +Yeah, it's a $5 one. + +00:45:56.020 --> 00:45:57.760 +It runs all these demos all the time, + +00:45:58.020 --> 00:45:59.220 +active, top of Hacker News, + +00:45:59.680 --> 00:46:01.180 +and it's never gone down. + +00:46:01.300 --> 00:46:02.780 +What's really interesting about that demo + +00:46:03.020 --> 00:46:06.740 +is that it becomes a backend optimization challenge, right? + +00:46:07.040 --> 00:46:09.060 +You're no longer trying to optimize the front end. + +00:46:09.220 --> 00:46:14.100 +You rely on the browser and the browser API to take care of that for you. + +00:46:14.340 --> 00:46:17.460 +And now you're doing, I don't know, you're optimizing your database. + +00:46:17.740 --> 00:46:19.020 +You're optimizing your queries. + +00:46:20.420 --> 00:46:22.360 +I actually threw the link to that in there. + +00:46:22.740 --> 00:46:30.380 +Because it's a nice demo to look at when you realize there are a billion of these being stored in a SQLite database somewhere. + +00:46:31.060 --> 00:46:32.700 +So you can scroll anywhere on the board. + +00:46:33.080 --> 00:46:37.180 +And it's like a 30,000 by 30,000 something grid + +00:46:37.500 --> 00:46:39.040 +because the square root of a billion + +00:46:39.580 --> 00:46:41.580 +is some weird number, as it turns out. + +00:46:42.240 --> 00:46:44.200 +And obviously this is, or not obviously, + +00:46:44.340 --> 00:46:45.220 +but this is multiplayer. + +00:46:45.540 --> 00:46:47.060 +So if I was to view this + +00:46:47.160 --> 00:46:49.020 +or you were to open a different browser tab, + +00:46:49.320 --> 00:46:50.980 +then you would see the exact same thing. + +00:46:51.060 --> 00:46:52.500 +The board is the same everywhere. + +00:46:52.820 --> 00:46:53.780 +That is crazy. + +00:46:54.120 --> 00:46:55.660 +The one thing that I will say that's hard, + +00:46:56.280 --> 00:46:58.720 +I don't know, Chris can really probably speak to this more + +00:46:58.820 --> 00:47:00.060 +and it sounds like a weird thing, + +00:47:00.280 --> 00:47:05.560 +Like, Datastar ends up in reality being like five or six things on your page, and it just + +00:47:05.760 --> 00:47:06.220 +gets out of the way. + +00:47:06.290 --> 00:47:09.720 +All of a sudden, like, most Datastar, you're going to get to a point where you try it, + +00:47:09.780 --> 00:47:10.600 +and you're like, that's it? + +00:47:10.910 --> 00:47:14.880 +Like, you will feel weird about every other approach once you really try it. + +00:47:14.890 --> 00:47:18.000 +Like, just try it, and you will see every other approach is wrong. + +00:47:18.290 --> 00:47:19.920 +Like, it's not because I made it. + +00:47:20.050 --> 00:47:23.280 +Like, I wish someone would have made this because it just, it's so simple. + +00:47:23.880 --> 00:47:26.060 +It feels like cheating in a weird way. + +00:47:26.540 --> 00:47:27.320 +That's hard to explain. + +00:47:27.400 --> 00:47:31.680 +it really it's a weird like i don't know what we all were doing i was part of the problem like right + +00:47:31.860 --> 00:47:35.940 +like i was like oh well google and everyone facebook and all the other guys have this figured + +00:47:36.120 --> 00:47:41.280 +out like this has to be the best approach so that's the weird thing is it's so simple i don't know what + +00:47:41.460 --> 00:47:45.000 +like crystal probably it sounds like i'm selling it but it's just i don't know it's weird it's so + +00:47:45.180 --> 00:47:50.500 +exciting it's so amazing and yet it's it's using all these boring technologies and like yeah like + +00:47:50.700 --> 00:47:55.619 +i remember i showed my wife this my status board and she's like oh yeah that looks really cool and + +00:47:55.640 --> 00:47:59.220 +I'm like, oh yeah, because you don't understand what everything is going on behind it, you know? + +00:47:59.460 --> 00:48:00.180 +Yeah, exactly. + +00:48:00.660 --> 00:48:02.900 +It's like, it used to be so complicated. + +00:48:03.620 --> 00:48:05.780 +So let's do, we got a little bit of time left. + +00:48:05.960 --> 00:48:06.280 +Let's do this. + +00:48:06.330 --> 00:48:12.580 +I think it might be fun to talk through kind of some of the attributes and what it looks like, + +00:48:13.080 --> 00:48:15.520 +kind of program with this a little bit and then what it looks like on the server. + +00:48:15.940 --> 00:48:16.420 +How's that sound? + +00:48:16.680 --> 00:48:17.220 +Would it make sense? + +00:48:17.380 --> 00:48:21.240 +Like there's a good example of a kind of a meta framework for Python called Stario, + +00:48:21.740 --> 00:48:23.240 +which they just got their V2. + +00:48:23.480 --> 00:48:23.620 +Okay. + +00:48:23.860 --> 00:48:24.440 +Just launched. + +00:48:24.600 --> 00:48:27.320 +I don't know if that is a more Python-esque way of doing it. + +00:48:27.560 --> 00:48:28.680 +It depends on how you want to. + +00:48:28.820 --> 00:48:31.260 +Let's start with some of the Datastar attributes, + +00:48:31.530 --> 00:48:32.680 +and then we could talk about that. + +00:48:32.800 --> 00:48:33.160 +How's that sound? + +00:48:33.580 --> 00:48:36.360 +Like, just, you know, what does it look like to say to, + +00:48:36.870 --> 00:48:40.960 +you know, I want to connect a button to Datastar actions + +00:48:41.130 --> 00:48:43.380 +on the back end or wired up and so on? + +00:48:43.460 --> 00:48:45.380 +We've talked about a lot about, you know, + +00:48:45.430 --> 00:48:48.340 +the back end driving the front end through patching elements, + +00:48:48.570 --> 00:48:51.080 +which is kind of the lower half of what you're looking at. + +00:48:51.720 --> 00:48:57.380 +To access that, you need to have a click listener or some sort of event listener to trigger that. + +00:48:58.080 --> 00:49:05.180 +And datastar, as the name suggests, uses data-star or asterisk attributes. + +00:49:05.820 --> 00:49:09.120 +So these are part of the HTML spec data set. + +00:49:09.680 --> 00:49:10.800 +And we just leverage that. + +00:49:10.940 --> 00:49:18.920 +And we have a small grammar that you'd find on the reference page with all of the data-attributes. + +00:49:19.160 --> 00:49:23.400 +And data on is just registering an event listener on the current element. + +00:49:23.820 --> 00:49:29.900 +So data on colon click is just obviously registering a click event handler on the button. + +00:49:30.520 --> 00:49:34.220 +And what's happening is that then Datasar also gives you actions. + +00:49:34.440 --> 00:49:38.340 +So that at get is an action to send a get request to the server. + +00:49:38.680 --> 00:49:41.540 +You pass in the path, which is slash endpoint there. + +00:49:42.320 --> 00:49:44.620 +And then the server takes care of the rest. + +00:49:44.640 --> 00:49:47.520 +So what you're seeing is a div underneath with an ID. + +00:49:48.100 --> 00:49:50.940 +IDs are obviously unique in HTML, so they're ideal. + +00:49:51.470 --> 00:49:54.320 +And Datastar just uses that fact. + +00:49:55.020 --> 00:50:03.200 +And what Datastar is going to do from the backend is it's going to, yeah, just send back down that div with some text content inside of it. + +00:50:03.640 --> 00:50:09.860 +And then what Datastar does is it mutates the incoming DOM into the existing DOM. + +00:50:10.220 --> 00:50:11.300 +I'm sorry, it morphs. + +00:50:11.820 --> 00:50:13.740 +So it uses a morphing strategy. + +00:50:14.440 --> 00:50:22.580 +So rather than doing a straight swap, which is what HTMLX does, it will actually morph the incoming HTML into what's currently on the page. + +00:50:23.160 --> 00:50:30.840 +That's kind of what opens up the door to these kind of broad, like where you send the entire document down, but only what changes get swapped in. + +00:50:30.860 --> 00:50:33.060 +But in this case, it's more of a fine grained thing. + +00:50:34.000 --> 00:50:35.960 +So only that div is going to get swapped out. + +00:50:36.120 --> 00:50:40.920 +And the reason why that morph matters is because you, since you aren't replacing it, things like + +00:50:41.110 --> 00:50:46.260 +focus and like where your input is and all that stays the same. So when you, even though you update + +00:50:46.370 --> 00:50:51.260 +the whole page, you're actually not actually changing the state and that's really important. + +00:50:51.370 --> 00:50:56.120 +So you do declarative development. You just say, I want it to look this way and it just does the + +00:50:56.130 --> 00:50:59.840 +right things to do it. It's from a mental ball. It's almost like having the BDOM in the backend. + +00:51:00.010 --> 00:51:03.940 +You just say, here's what I want this page to look like. And it does all the work, but we don't do + +00:51:03.920 --> 00:51:08.720 +BDOM. We don't do any of that stuff. We do the fast thing. So in terms of what your backend would + +00:51:08.920 --> 00:51:13.980 +send, if you can just scroll back up, it's that text that you were looking at. Let's look at the + +00:51:14.040 --> 00:51:19.780 +raw version because, yeah, so that's the HTML. If you scroll down to the next text, it's a code + +00:51:19.880 --> 00:51:24.260 +field. Yeah. There's a section that has like event, data star, patch elements, and then what the + +00:51:24.500 --> 00:51:29.880 +elements are and so on, right? This is like the SSE stream. Yeah. And that would be the raw events + +00:51:29.900 --> 00:51:30.720 +that you would send down. + +00:51:31.140 --> 00:51:32.860 +But if you look at the next one + +00:51:33.040 --> 00:51:34.980 +where we have a Python example, + +00:51:35.300 --> 00:51:35.940 +you would see like, + +00:51:36.340 --> 00:51:37.420 +well, how do you do that in Python + +00:51:38.260 --> 00:51:39.240 +without actually writing, + +00:51:39.720 --> 00:51:40.980 +you know, the raw format out? + +00:51:41.560 --> 00:51:43.000 +And that's how you would do it there + +00:51:43.200 --> 00:51:44.520 +using the Python SDK. + +00:51:44.660 --> 00:51:45.640 +Let's dive in a little bit + +00:51:45.760 --> 00:51:47.860 +to the SDK itself. + +00:51:48.260 --> 00:51:49.980 +So I got so many things open. + +00:51:51.820 --> 00:51:52.260 +Hold on. + +00:51:52.440 --> 00:51:53.240 +We got another link for you. + +00:51:53.360 --> 00:51:53.700 +No, I'm kidding. + +00:51:54.740 --> 00:51:54.980 +You know what? + +00:51:55.020 --> 00:51:55.640 +I'm just going to go. + +00:51:55.800 --> 00:51:56.500 +I'm going from the homepage. + +00:51:56.940 --> 00:51:57.260 +There we go. + +00:51:57.400 --> 00:51:57.640 +There you go. + +00:51:57.820 --> 00:51:59.240 +Chris, maybe you could talk us through this. + +00:51:59.360 --> 00:52:03.480 +I think before I throw it to you, though, yeah, there's a lot of framework support here. + +00:52:03.540 --> 00:52:08.020 +So if you're a Django person, a FastAPI person, even fast HTML, it's interesting, + +00:52:08.360 --> 00:52:10.240 +LightSar, Quartz, SanEck, or Starlet. + +00:52:10.800 --> 00:52:15.520 +There's a bunch of different ones here, but maybe just talk us through this, if you'll, Chris. + +00:52:16.000 --> 00:52:16.540 +I'm trying to remember. + +00:52:16.680 --> 00:52:20.180 +I'm not as familiar with the example, but as you can see, + +00:52:20.480 --> 00:52:22.560 +this one method is, I think, where the magic happens. + +00:52:22.940 --> 00:52:24.600 +I'm trying to remember which tool. + +00:52:24.780 --> 00:52:25.680 +This is a Quartz. + +00:52:25.900 --> 00:52:27.720 +Yeah, the Quartz is the examples in Quartz. + +00:52:27.900 --> 00:52:34.860 +So, you know, they first define a route, a home route slash, and it returns HTML and it's just in the string there. + +00:52:34.970 --> 00:52:35.740 +Right. And then that. + +00:52:35.890 --> 00:52:37.940 +This could be a Ginget or Chameleon or whatever template. + +00:52:38.010 --> 00:52:39.220 +Like it's just whatever. It doesn't matter. + +00:52:39.480 --> 00:52:40.380 +But somehow they get it. Yeah. + +00:52:40.580 --> 00:52:42.340 +Makes the example easier to see in one go. + +00:52:42.820 --> 00:52:54.280 +And obviously you see that it's pulling Datastar from the CDN and then it on the load, it gets it sends a request to the slash updates endpoint. + +00:52:54.840 --> 00:52:56.360 +See what comes from that. + +00:52:56.980 --> 00:53:02.840 +And so down below that, you have the slash updates endpoint, which has a decorator called data, data star underscore response. + +00:53:03.420 --> 00:53:10.260 +And that just does a couple of nice things like sets the HTTP headers and whatnot to be the service and event protocol. + +00:53:10.720 --> 00:53:15.880 +And then what I like to the first line says signals equals await read signals. + +00:53:16.440 --> 00:53:21.140 +And so that's another helper that essentially says when I have a request coming in, + +00:53:21.580 --> 00:53:25.260 +data star has a specific way of sending the state of the front end to the back end. + +00:53:25.280 --> 00:53:26.780 +So the back end can do whatever it needs. + +00:53:26.940 --> 00:53:35.460 +Right. We haven't even talked about signals yet. They're like kind of a data binding set of JavaScript data that loads, you know, reactive data loads on the front end, right? + +00:53:35.500 --> 00:53:37.680 +In some ways, the Alpine JS kind of, + +00:53:38.470 --> 00:53:39.160 +I don't want to say equivalent, + +00:53:39.390 --> 00:53:41.580 +but it covers similar functionality. + +00:53:41.990 --> 00:53:43.660 +And so if you have data on the front end + +00:53:44.100 --> 00:53:45.480 +that the backend would like to know, + +00:53:45.520 --> 00:53:47.260 +that's an easy way to get it. + +00:53:47.530 --> 00:53:48.940 +And then essentially what happens + +00:53:49.010 --> 00:53:50.940 +is we get into this loop, this while true loop, + +00:53:51.160 --> 00:53:53.300 +and Datastar will just start sending down + +00:53:53.410 --> 00:53:55.800 +server sent events in text + +00:53:56.080 --> 00:53:59.460 +by using the sse.patchElements function, + +00:53:59.680 --> 00:54:00.840 +or I guess it's a method technically. + +00:54:01.580 --> 00:54:03.880 +And all it's doing is sending a string + +00:54:03.880 --> 00:54:07.320 +that has the current date time dot now in ISO format down. + +00:54:07.730 --> 00:54:10.280 +And then we wait, we sleep for a second, + +00:54:10.460 --> 00:54:11.160 +or is it a second? + +00:54:11.470 --> 00:54:12.380 +I guess it's a microsecond. + +00:54:12.450 --> 00:54:13.100 +I keep forgetting which one. + +00:54:13.100 --> 00:54:14.400 +Yeah, that's a millisecond. + +00:54:14.540 --> 00:54:15.300 +No, no, it's second. + +00:54:15.440 --> 00:54:16.180 +And sleep is seconds. + +00:54:16.340 --> 00:54:16.880 +Sleep is seconds. + +00:54:17.280 --> 00:54:18.180 +It takes a float. + +00:54:18.320 --> 00:54:20.880 +So once it sleeps, it sends another server sign event. + +00:54:21.640 --> 00:54:24.600 +With this time, it's instead of sending the HTML down, + +00:54:24.980 --> 00:54:26.680 +we're sending a signal. + +00:54:27.030 --> 00:54:28.520 +So essentially changing, say, you can say, + +00:54:28.660 --> 00:54:31.320 +like the JavaScript value or script data + +00:54:31.830 --> 00:54:32.620 +onto the front of the page. + +00:54:32.780 --> 00:54:33.080 +Right, right. + +00:54:33.200 --> 00:54:37.800 +So it's showing that you can send the HTML and let Datastar patch it, or you can basically + +00:54:38.240 --> 00:54:43.200 +from the server set one of these signal things that will be reactive on the front end, right? + +00:54:43.390 --> 00:54:43.520 +Yeah. + +00:54:43.520 --> 00:54:44.480 +You said it much better than I did. + +00:54:45.760 --> 00:54:45.900 +Thanks. + +00:54:45.900 --> 00:54:47.940 +It's a long way of saying it's a clock, right? + +00:54:48.120 --> 00:54:48.220 +Yeah. + +00:54:48.440 --> 00:54:51.720 +Also the thing, just for people that aren't used to thinking about this way, especially + +00:54:51.900 --> 00:54:56.100 +if you're doing Python, like all a signal is, is instead of saying that here's, I'm + +00:54:56.210 --> 00:55:00.099 +setting a variable, you're saying I'm setting a relationship that says like, kind of like + +00:55:00.120 --> 00:55:03.660 +in an Excel document when you set a formula for a cell, + +00:55:04.000 --> 00:55:04.540 +it's the same idea. + +00:55:04.590 --> 00:55:05.700 +You're setting up a relationship saying, + +00:55:05.770 --> 00:55:08.080 +when this thing and this thing changes, update this. + +00:55:08.170 --> 00:55:10.520 +And it does smart things to do that efficiently. + +00:55:10.890 --> 00:55:12.280 +But the idea is it's a relationship. + +00:55:12.580 --> 00:55:13.220 +It's declarative. + +00:55:13.670 --> 00:55:14.700 +So kind of like with SQL, + +00:55:15.280 --> 00:55:17.600 +you think of SQL as a declarative language, right? + +00:55:17.680 --> 00:55:18.980 +You don't care how it creates an index. + +00:55:19.070 --> 00:55:20.120 +You just say, create index. + +00:55:20.620 --> 00:55:21.500 +Same thing happens here. + +00:55:21.570 --> 00:55:23.880 +You just say, hey, I want when this thing changes, + +00:55:24.110 --> 00:55:24.820 +this other thing to change. + +00:55:25.250 --> 00:55:26.880 +And the problem is that declarativeness + +00:55:27.040 --> 00:55:28.100 +is not built into JavaScript. + +00:55:28.400 --> 00:55:29.380 +It's not built into the browser, + +00:55:29.440 --> 00:55:32.380 +but we just made the web a little bit more declarative. + +00:55:32.800 --> 00:55:33.580 +That's all we did, basically. + +00:55:33.700 --> 00:55:33.780 +Right. + +00:55:34.320 --> 00:55:36.020 +Declarative is generally pretty good. + +00:55:36.260 --> 00:55:37.160 +It's a good way to work. + +00:55:37.300 --> 00:55:38.040 +It keeps things simple + +00:55:38.240 --> 00:55:40.280 +and lets the underlying system have at it. + +00:55:40.760 --> 00:55:42.140 +So a couple of things, + +00:55:42.880 --> 00:55:44.260 +well, we still got a little bit of time, + +00:55:44.440 --> 00:55:46.260 +but to wrap things up a little bit. + +00:55:46.560 --> 00:55:46.800 +Editors. + +00:55:47.200 --> 00:55:49.080 +I think having good editor support + +00:55:49.840 --> 00:55:51.840 +is really important for adoption. + +00:55:52.100 --> 00:55:53.220 +You know, drives me crazy + +00:55:53.720 --> 00:55:56.220 +when I go and try to work with JavaScript, CSS, + +00:55:56.800 --> 00:55:57.520 +attributes or whatever, + +00:55:57.720 --> 00:55:59.120 +and I'm like, they're not here. + +00:55:59.200 --> 00:55:59.720 +No help. + +00:56:00.260 --> 00:56:06.760 +So you all have nice extensions and plugins for common editors Python people might use, right? + +00:56:06.860 --> 00:56:12.040 +Yeah, we have VS Code, which you're seeing here, and PHP Storm. + +00:56:12.560 --> 00:56:16.140 +Or sorry, I use PHP Storm, but all JetBrains editors. + +00:56:16.580 --> 00:56:19.580 +PHP Storm, PyToram, WebStorm, all of them things. + +00:56:19.740 --> 00:56:25.180 +So it's in the JetBrains marketplace, so it'll work for, I believe, all JetBrains IDEs. + +00:56:25.360 --> 00:56:25.740 +I believe so. + +00:56:25.850 --> 00:56:28.440 +You also have the AI editors covered. + +00:56:28.770 --> 00:56:29.000 +Do we? + +00:56:29.180 --> 00:56:37.620 +In the OpenVSX registry, all the ones that have been kicked out from VS Code, this is where they all have to go to get their installs, right? + +00:56:37.740 --> 00:56:40.060 +That explains why people requested this from me. + +00:56:41.519 --> 00:56:49.060 +Yeah, if you're doing cursor, anti-gravity, windsurf, like all those things, they were all kicked out of the VS Code registry. + +00:56:50.640 --> 00:56:51.480 +That's not a complaint. + +00:56:51.570 --> 00:56:53.060 +I mean, it's a Microsoft product. + +00:56:53.070 --> 00:56:53.560 +They built it. + +00:56:54.050 --> 00:56:55.880 +They don't have to build all the other ones. + +00:56:56.020 --> 00:56:57.460 +But that's why they're here, right? + +00:56:57.560 --> 00:56:58.580 +We keep those up to date. + +00:56:58.980 --> 00:57:01.320 +We do those ourselves, the SDKs. + +00:57:01.330 --> 00:57:04.080 +I mean, Delaney wrote the Go one, I wrote the PHP one, + +00:57:04.110 --> 00:57:06.660 +and the rest are just community contributions. + +00:57:07.280 --> 00:57:11.040 +We've had contributions to these too, to the IDE extensions. + +00:57:11.680 --> 00:57:13.340 +We maintain these primarily. + +00:57:13.760 --> 00:57:14.360 +Yeah, and these are great. + +00:57:14.590 --> 00:57:19.480 +These just, you know, save on typing, but more importantly, save on making typos. + +00:57:20.100 --> 00:57:22.760 +You know, they show you all of the available data attributes. + +00:57:23.120 --> 00:57:26.120 +Maybe Chris can speak more to is that the irony is, though, + +00:57:26.300 --> 00:57:29.220 +you won't need that many tags to actually do your work. + +00:57:29.330 --> 00:57:30.960 +So it's not like a tailwind thing where you're like, + +00:57:31.140 --> 00:57:33.320 +oh, I rely on it to autocomplete. + +00:57:33.520 --> 00:57:33.860 +It's just... + +00:57:33.860 --> 00:57:34.220 +Yeah, absolutely. + +00:57:34.540 --> 00:57:35.800 +In fact, it's one of those things + +00:57:35.890 --> 00:57:38.140 +where I discover more things I can do with Datastar + +00:57:38.280 --> 00:57:40.500 +because as I'm typing data dash and I'm like, + +00:57:40.710 --> 00:57:42.600 +oh, I didn't actually remember + +00:57:42.780 --> 00:57:45.800 +that there's a attribute to do whatever it is. + +00:57:45.800 --> 00:57:46.240 +I don't remember. + +00:57:46.440 --> 00:57:47.620 +Like, I can't remember at this point. + +00:57:48.000 --> 00:57:49.860 +And then I went to the documentation like, + +00:57:49.960 --> 00:57:50.640 +oh, check this out. + +00:57:50.760 --> 00:57:51.740 +This is so much more I can do. + +00:57:52.100 --> 00:57:54.080 +But yeah, I find I love the plugin, + +00:57:54.280 --> 00:57:55.460 +but I find I don't use it too much + +00:57:55.480 --> 00:57:58.160 +just because I'm not writing as much HTML with it. + +00:57:58.280 --> 00:58:01.680 +While I'm sitting here on this open VSX registry, + +00:58:02.000 --> 00:58:05.420 +do you all have advice for making Datastar work well + +00:58:05.580 --> 00:58:08.500 +with Identic AI and Claude Code, Cursor, et cetera? + +00:58:08.740 --> 00:58:10.280 +There's some active research going on + +00:58:10.440 --> 00:58:13.300 +like in Oslo at a college + +00:58:13.840 --> 00:58:15.900 +that's doing, ironically, using Datastar + +00:58:16.720 --> 00:58:21.940 +to do some stuff around like how LLMs work with code bases. + +00:58:22.120 --> 00:58:24.440 +And the reason why is because the entire code base + +00:58:24.380 --> 00:58:28.320 +fits in basically every context, even the nano ones, like the entire code base fits there. + +00:58:28.620 --> 00:58:33.340 +And what they found, we've gone back and forth a bit, is that almost all of them are completely + +00:58:33.540 --> 00:58:37.840 +overfitted. So if you just want to make a website with agentic stuff, go do React, + +00:58:38.260 --> 00:58:43.020 +because that's what it's built for. And it's overfitted to such a degree that if you try to + +00:58:43.420 --> 00:58:49.160 +use the spec correctly and to say, here's all the sorts of data star, go use it to build websites, + +00:58:49.380 --> 00:58:51.500 +it will fall over almost in every regard. + +00:58:51.840 --> 00:58:54.760 +So it's one of those things where you don't need that much, + +00:58:54.940 --> 00:58:59.420 +but it will ironically show you how bad things like Clawed and Codex and stuff + +00:58:59.600 --> 00:59:02.680 +are at just using the current context to solve things. + +00:59:03.080 --> 00:59:05.480 +Hopefully that gets better, but we have something around, + +00:59:05.840 --> 00:59:08.440 +like we have a slash docs page that you can feed into your LLM, + +00:59:08.680 --> 00:59:11.500 +but I'll say that we do not focus on that at all + +00:59:11.740 --> 00:59:14.900 +because you're basically fighting against what the training already happened. + +00:59:15.160 --> 00:59:17.820 +So you're better off, like if you want to use, + +00:59:18.140 --> 00:59:19.060 +if you want to make better size, + +00:59:19.100 --> 00:59:20.080 +you want to be fast and efficient + +00:59:20.140 --> 00:59:20.720 +and all that stuff, + +00:59:21.120 --> 00:59:22.200 +we're 100% the right thing to do. + +00:59:22.380 --> 00:59:24.360 +If you just want to like one shot something, + +00:59:24.780 --> 00:59:26.880 +go use React and stay in that world. + +00:59:26.940 --> 00:59:27.820 +You want to vibe code it? + +00:59:28.340 --> 00:59:29.120 +Hey, I've got something. + +00:59:29.220 --> 00:59:32.200 +I feel like this might resonate with you, Delaney, + +00:59:32.420 --> 00:59:33.640 +especially the way you just described it. + +00:59:34.040 --> 00:59:37.740 +Have you all seen the Kai Lintit Senior Engineer + +00:59:37.980 --> 00:59:38.660 +Tries Vibe Coding? + +00:59:39.080 --> 00:59:41.280 +This is an amazing video. + +00:59:41.660 --> 00:59:43.060 +And like half of the video is like, + +00:59:43.480 --> 00:59:45.400 +no, no, no, not in being installed. + +00:59:46.100 --> 00:59:46.520 +What are you doing? + +00:59:47.180 --> 00:59:52.600 +It reminds me very much of like, it's just like, nope, that's not what I told you to do. + +00:59:52.660 --> 00:59:54.740 +I know that's what you think the most common thing is, please stop. + +00:59:54.860 --> 00:59:54.940 +Yeah. + +00:59:55.080 --> 00:59:58.880 +And the thing is that it's not that I actually like a lot of the stuff, but I treat it as + +00:59:58.900 --> 01:00:03.500 +an autocomplete or like it can write code faster than I can when it comes to like, hey, change + +01:00:03.500 --> 01:00:04.920 +this in 27 different places. + +01:00:05.400 --> 01:00:06.580 +And I forget which files I did it like. + +01:00:06.960 --> 01:00:10.080 +There's value to it, but people are trying to use it to learn. + +01:00:10.620 --> 01:00:12.800 +It's a complete, it actively is working against you. + +01:00:13.180 --> 01:00:15.300 +Ben has done an amazing job with the guide. + +01:00:15.780 --> 01:00:20.180 +Like, please, like, it's fine to use the LLMs to help, like, guide your process and to, like, + +01:00:20.660 --> 01:00:22.120 +knock stuff out quickly once you have a baseline. + +01:00:22.640 --> 01:00:24.160 +But you have to know when to say no. + +01:00:24.720 --> 01:00:25.580 +And he has done it. + +01:00:25.770 --> 01:00:29.760 +The guide takes half an hour, less than a half, like, probably 15, 20 minutes to read + +01:00:29.940 --> 01:00:31.840 +and then, like, an hour to actually work through. + +01:00:32.320 --> 01:00:35.580 +Please try it first before you try to throw it at the LLMs. + +01:00:35.680 --> 01:00:36.960 +It's not because I hate them. + +01:00:37.080 --> 01:00:42.300 +It's more that they are just overfit to the, like, the sea of badly written SPA code. + +01:00:42.700 --> 01:00:44.820 +That's, unfortunately, that's the situation we're in. + +01:00:45.020 --> 01:00:49.260 +Yeah, especially with JavaScript, the agentic AI is very trained. + +01:00:49.700 --> 01:00:50.600 +It wants what it wants. + +01:00:51.280 --> 01:00:51.560 +All right. + +01:00:52.220 --> 01:00:55.760 +Let's talk, speaking of being near the guy, + +01:00:55.760 --> 01:00:59.460 +if I go over here to more, there's a pro section. + +01:00:59.980 --> 01:01:01.740 +I'll let you all give a shout out to pro. + +01:01:02.300 --> 01:01:04.440 +I know you have a really strong sales pitch here. + +01:01:04.980 --> 01:01:06.440 +You were talking about earlier. + +01:01:06.940 --> 01:01:08.380 +Now, what is this data, Datastar Pro? + +01:01:08.840 --> 01:01:13.800 +It's been about a year since we released the beta one of Datastar. + +01:01:14.420 --> 01:01:23.180 +We are taking our sweet ass time for a very good reason, which is that we want version one to be the last version or like the last major version. + +01:01:23.820 --> 01:01:30.460 +We don't really want to force people through breaking changes and major updates because that's really just a pain. + +01:01:30.630 --> 01:01:35.820 +And I think like Python has done a great job with that and Go as well. + +01:01:36.060 --> 01:01:39.240 +And like there are some ecosystems where you just don't make breaking changes. + +01:01:39.560 --> 01:01:41.320 +That's the norm. And that's what we want to be. + +01:01:41.440 --> 01:01:44.840 +And the JavaScript ecosystem is, you know, the antithesis to that. + +01:01:45.800 --> 01:01:46.980 +They're like, here, hold my beer. + +01:01:47.100 --> 01:01:48.020 +I'll show you breaking changes. + +01:01:48.300 --> 01:01:48.520 +Yeah. + +01:01:48.660 --> 01:01:49.560 +Have you heard of LeftPad? + +01:01:49.700 --> 01:01:53.240 +To give you an idea of how far we take that, we don't have npm. + +01:01:53.740 --> 01:01:55.680 +Like we don't actually even submit to npm. + +01:01:55.810 --> 01:01:59.380 +We have no package.json in our JavaScript framework. + +01:01:59.840 --> 01:02:01.900 +We actually, like, there's none of that stuff. + +01:02:02.090 --> 01:02:03.480 +It does not exist in our ecosystem. + +01:02:03.480 --> 01:02:07.460 +So we take it very seriously when we say it's funny to have a JavaScript framework that + +01:02:07.800 --> 01:02:09.560 +actively hates the JavaScript ecosystem. + +01:02:09.940 --> 01:02:11.620 +And you guys, I think also it's worth pointing out + +01:02:11.720 --> 01:02:13.540 +that you don't have a strong build step, + +01:02:14.120 --> 01:02:17.080 +tree shaking, web packing story, right? + +01:02:17.080 --> 01:02:17.840 +You just dropped- + +01:02:17.860 --> 01:02:18.940 +No, we do. + +01:02:18.940 --> 01:02:21.740 +The thing is that, like for example, + +01:02:21.740 --> 01:02:25.280 +beat is kind of the well-known way to do this stuff. + +01:02:25.280 --> 01:02:25.820 +But guess what? + +01:02:25.820 --> 01:02:27.400 +Under the hood, it uses ES build. + +01:02:27.400 --> 01:02:29.280 +And ES build is a go thing. + +01:02:29.280 --> 01:02:30.400 +We build a lot of our stuff in go. + +01:02:30.400 --> 01:02:32.100 +So it's literally embedded in our, + +01:02:32.100 --> 01:02:33.960 +like we just use the ES build directly. + +01:02:33.960 --> 01:02:36.620 +We don't need 20,000 things from npm. + +01:02:36.620 --> 01:02:38.559 +We just use the go tools inside of our binary + +01:02:38.580 --> 01:02:40.020 +because that's the fast thing to do. + +01:02:40.440 --> 01:02:41.860 +So we don't need all of that. + +01:02:41.870 --> 01:02:43.420 +So we have no dependencies, nothing, + +01:02:43.690 --> 01:02:46.360 +and we don't even use npm or any of that at all. + +01:02:46.720 --> 01:02:48.300 +Yeah, so the reason I mentioned the beta + +01:02:48.560 --> 01:02:51.200 +is during the beta phase, which lasted about six months, + +01:02:51.640 --> 01:02:53.920 +we, Datastar gained a lot of traction, + +01:02:54.250 --> 01:02:56.500 +a lot of interest, and people had a lot of requests. + +01:02:56.800 --> 01:02:58.520 +And we were like, yeah, we see, + +01:02:58.960 --> 01:02:59.960 +and because it's plugin-based, + +01:03:00.170 --> 01:03:01.660 +you can always just add another plugin. + +01:03:01.730 --> 01:03:02.600 +You can add it yourself, + +01:03:03.080 --> 01:03:05.520 +or we can build a plugin and add it to Datastar. + +01:03:05.960 --> 01:03:15.260 +But we were very adamant about keeping the open source Datastar framework as tight as possible. + +01:03:15.560 --> 01:03:18.280 +Like I said, it should do everything you need, but nothing you don't. + +01:03:18.420 --> 01:03:20.860 +So how do we do that while adding plugins? + +01:03:21.540 --> 01:03:26.720 +So during that beta phase, we started thinking about, well, do we have multiple versions of Datastar? + +01:03:26.780 --> 01:03:28.260 +Do we have a marketplace of plugins? + +01:03:28.470 --> 01:03:29.480 +Or how do we manage that? + +01:03:30.080 --> 01:03:35.880 +And at the same time, we were also asking ourselves, because Delaney and I both, we have full-time things that we're doing. + +01:03:35.920 --> 01:04:05.680 +And this is a side project, but we're almost doing it full time alongside our other full time things. So how do we make this project sustainable? Because it doesn't stop at Datastar. You probably see Rocket and Stellar CSS on that page in the navigation sidebar. Those are like projects that build on top of Datastar. So Datastar is just the foundation. And Rocket kind of takes it to web components and Stellar CSS is a CSS framework that builds on top of these concepts. + +01:04:05.820 --> 01:04:09.800 +So we're trying to fix not only JavaScript web components, but also CSS. + +01:04:10.560 --> 01:04:12.160 +So we have a long-term vision. + +01:04:12.820 --> 01:04:18.380 +How do we make that sustainable when we're both busy people anyway, and this just takes + +01:04:18.920 --> 01:04:21.900 +so much of our time and the project appears to be growing? + +01:04:22.600 --> 01:04:25.720 +So at that point, we decided, well, how do we want to even run this? + +01:04:25.880 --> 01:04:31.360 +So we decided we don't want to found some company and do VC like we're, if anything, + +01:04:31.830 --> 01:04:32.840 +anti-VC funding. + +01:04:32.920 --> 01:04:43.700 +So we founded a nonprofit organization in the U.S. called Star Federation, and that's what backs this project, including Rocket and Stellar CSS. + +01:04:44.780 --> 01:04:57.680 +To help fund that organization, we decided let's have Datastar be the open source framework, but then something called Datastar Pro, which is like all those plugins that we think are good ideas, but that most people don't need. + +01:04:58.010 --> 01:05:01.460 +We'll put those into Datastar Pro, and that can kind of grow over time. + +01:05:01.740 --> 01:05:07.120 +It's a collection of plugins that you might want if you're using Datastar in a professional setting. + +01:05:07.520 --> 01:05:11.280 +But, you know, if you're just using Datastar, you don't actually need it. + +01:05:11.580 --> 01:05:12.520 +And so that's what we tell people. + +01:05:12.680 --> 01:05:13.700 +Most people don't need it. + +01:05:13.860 --> 01:05:19.380 +It's a collection of plugins and it's a Datastar inspector, which sits on your page. + +01:05:19.900 --> 01:05:25.840 +You get access to the bundler and now you get access to Rocket and StadrCSS, which is a work in progress. + +01:05:26.720 --> 01:05:28.220 +Yeah, that was, I think, a good decision. + +01:05:28.520 --> 01:05:35.400 +Like there was definitely some uproar initially that, you know, some plugins were taken away, but those plugins were never taken away. + +01:05:35.480 --> 01:05:38.080 +They still exist in the repo if anybody needs them. + +01:05:38.540 --> 01:05:46.040 +What the result is, is that we have like some money coming into a bank account, which is not even used to pay maintainers. + +01:05:46.760 --> 01:05:52.500 +We use that for running costs and like for, you know, podcasting software. + +01:05:53.000 --> 01:05:55.940 +And if we need to travel to conferences, which we've yet to do. + +01:05:56.120 --> 01:06:05.400 +But essentially, it's like a way of having some money into the bank so that we can justify all of the work that we do in maintaining Datastar and pushing that forward. + +01:06:06.300 --> 01:06:07.300 +But the V1 thing. + +01:06:07.600 --> 01:06:14.440 +100% free sounds great until that means it becomes abandoned where, you know, and like people can't work on it anymore. + +01:06:14.740 --> 01:06:16.660 +And I think it's fair. + +01:06:16.860 --> 01:06:21.380 +There's one thing that's kind of interesting about the model, because especially with the tailwind stuff that's been going on lately. + +01:06:21.840 --> 01:06:26.260 +One of the things that we talked about, and people get very angry about this, but for example, + +01:06:26.660 --> 01:06:30.820 +Rocket is a web component layer that you basically just write Datastar in a declarative way, + +01:06:31.320 --> 01:06:34.300 +and it dynamically generates web components for you on the fly. + +01:06:34.370 --> 01:06:36.460 +And it's a great way to build web components. + +01:06:36.620 --> 01:06:38.080 +It'll save you tons of hours. + +01:06:38.540 --> 01:06:39.560 +And people won't pay for features. + +01:06:39.630 --> 01:06:40.320 +They pay for convenience. + +01:06:40.920 --> 01:06:45.860 +So the thing is, people said, well, I want you to generate out the content and make that + +01:06:46.060 --> 01:06:46.620 +open and available. + +01:06:46.960 --> 01:06:51.580 +And we said no, because basically the way we look at it is that almost like Pico 8, + +01:06:51.780 --> 01:06:54.000 +or any kind of game engine, you pay for the game + +01:06:54.140 --> 01:06:55.140 +and then all the mods are free. + +01:06:55.180 --> 01:06:57.940 +So all the rocket components and all this stuff + +01:06:58.000 --> 01:07:01.060 +is gonna be free, but the core engine is not free. + +01:07:01.520 --> 01:07:02.340 +It's a paid thing. + +01:07:02.380 --> 01:07:04.340 +And the reason why is if it becomes successful, + +01:07:04.580 --> 01:07:06.820 +if we do our job and we make it so it's easy for everybody, + +01:07:07.300 --> 01:07:09.980 +the Star Federation will do better over time. + +01:07:10.100 --> 01:07:12.340 +Whereas Tailwind's model of they're competing + +01:07:12.480 --> 01:07:14.580 +against every other person in that space, + +01:07:15.240 --> 01:07:16.680 +whereas it just does not work. + +01:07:16.800 --> 01:07:18.480 +So our thing is if we do get successful + +01:07:18.980 --> 01:07:20.340 +and we do get more people, + +01:07:20.740 --> 01:07:23.600 +then it's self-sustaining as in you paid for this little engine + +01:07:24.100 --> 01:07:27.400 +and now you get all the ecosystem around it of open source. + +01:07:27.710 --> 01:07:28.940 +So you can do open source in a way, + +01:07:29.040 --> 01:07:32.320 +but you have to find a core engine that is not open source. + +01:07:32.520 --> 01:07:34.420 +Otherwise it will fail in the modern world. + +01:07:34.620 --> 01:07:37.500 +Let's close this thing out with two super quick things + +01:07:37.640 --> 01:07:38.360 +because I know we're over time. + +01:07:38.800 --> 01:07:39.140 +Roadmap. + +01:07:39.700 --> 01:07:42.400 +Ben, you talked about taking your sweet time to 1.0. + +01:07:42.880 --> 01:07:44.400 +Is there a forward-looking roadmap? + +01:07:44.740 --> 01:07:46.400 +Are you guys done or what are things? + +01:07:46.740 --> 01:07:50.380 +The release counted RC1, I think it was about six months ago. + +01:07:51.200 --> 01:07:54.240 +and like the release has just been slowing down, slowing down. + +01:07:54.240 --> 01:07:57.360 +So that stagnation in like just releases with fixes + +01:07:57.960 --> 01:08:00.380 +is a good sign to me that we're very, very close. + +01:08:00.800 --> 01:08:04.760 +At this point, like the switch from release candidate to stable + +01:08:05.000 --> 01:08:08.360 +is just literally like just, you know, dropping the RC. + +01:08:09.040 --> 01:08:11.520 +There's no like features that are going into it. + +01:08:11.640 --> 01:08:12.640 +There's no big changes. + +01:08:13.240 --> 01:08:15.060 +We're taking our time because like I said, + +01:08:15.320 --> 01:08:18.580 +it's easy to put something up slightly prematurely + +01:08:18.720 --> 01:08:19.839 +and get some defaults wrong. + +01:08:20.160 --> 01:08:27.160 +I mean, that's what happened with HMX 2 was just like fixing some defaults that they decided they got wrong in version one. + +01:08:27.359 --> 01:08:29.980 +So we're trying to avoid a situation like that. + +01:08:30.380 --> 01:08:35.500 +And the only way to do that is to just let it simmer, let people use it, let people dog food it. + +01:08:35.740 --> 01:08:46.359 +And us, too, we're actively using for many projects using data ourselves and discovering every now and then, oh, this default is probably we're trying to avoid foot guns. + +01:08:46.660 --> 01:08:55.480 +So we're trying to make it so that the defaults give you the best possible experience that you need zero configuration, ideally, but you can configure as needed. + +01:08:55.910 --> 01:09:05.440 +But getting those defaults right is really the only thing stopping us from, not right, but locked down is the only thing stopping us from a V1 stable. + +01:09:05.960 --> 01:09:07.020 +I don't like to give timelines. + +01:09:07.380 --> 01:09:12.160 +In fact, it's one of our things that I tell Delaney, never promise a timeline. + +01:09:12.759 --> 01:09:16.160 +But I could see us in the first half of this year, + +01:09:16.540 --> 01:09:17.859 +just flipping the switch. + +01:09:18.040 --> 01:09:20.060 +But it sounds like you might be able to use the RC + +01:09:20.279 --> 01:09:22.299 +and you might more or less be safe, yeah. + +01:09:22.380 --> 01:09:25.200 +In fact, we recommend people rename the RC + +01:09:25.440 --> 01:09:28.480 +and they change the name to React-Foo + +01:09:28.880 --> 01:09:30.560 +so that they just drop it in their React projects + +01:09:30.920 --> 01:09:34.060 +because the entire framework is smaller than most components. + +01:09:34.580 --> 01:09:36.160 +Just start hiding it places. + +01:09:36.700 --> 01:09:37.900 +Yeah, don't even name it Datastar. + +01:09:38.180 --> 01:09:40.339 +A stealth takeover of the spa world. + +01:09:41.000 --> 01:09:42.020 +Awesome. I love it. + +01:09:42.380 --> 01:09:45.980 +All right, let's wrap up the show with a final call to action + +01:09:46.860 --> 01:09:50.259 +for people who want to use Datastar, learn more, get started. + +01:09:50.799 --> 01:09:54.460 +Chris, I'll let you go first so Ben and Delaney can have the final word. + +01:09:54.500 --> 01:09:58.460 +The first thing I was thinking of is because I get asked so much about + +01:09:59.420 --> 01:10:02.840 +how long it takes to connect to the server and things like that, + +01:10:03.240 --> 01:10:07.920 +there is a portion in the DjangoCon talk I gave in, I think it was 2023, + +01:10:08.480 --> 01:10:14.180 +where I showed a video of five phones, five Android phones, trying to do the same thing, + +01:10:14.380 --> 01:10:20.060 +shopping for eggs, I believe it was. And essentially one of them is an HTML driven + +01:10:20.740 --> 01:10:25.540 +multi-page app and it smokes the single page applications, the native apps and everything. + +01:10:25.920 --> 01:10:30.400 +And so I put a link in our chat. Maybe you'll be a part of the show notes. It's a deep link to go + +01:10:30.560 --> 01:10:36.639 +straight to that portion of the talk because it is like that video reminded me like, this is what + +01:10:36.660 --> 01:10:43.520 +I want to build. I want to build websites that are fun for people to use. And, you know, Datastar + +01:10:43.980 --> 01:10:50.260 +enables me to use real-time interactions with way less complexity than I ever thought + +01:10:50.680 --> 01:10:55.060 +possible. So I guess the two things I would say is, one, check out the deep link if you're at all + +01:10:55.220 --> 01:11:00.600 +interested. And number two, definitely try out something, you know, just even clone the Python + +01:11:00.920 --> 01:11:05.060 +repo and just try some of the examples and see what it's like. Yeah, I'll definitely link to that. + +01:11:05.240 --> 01:11:05.980 +Cool. Thanks, Ben. + +01:11:06.080 --> 01:11:08.120 +I also gave a conference talk last year. + +01:11:08.380 --> 01:11:10.160 +There's a recording, so I'll send you the link to that, + +01:11:10.340 --> 01:11:13.760 +which really walks through my journey of Datastar + +01:11:13.770 --> 01:11:18.440 +and how Datastar has truly opened my eyes to what's possible. + +01:11:18.740 --> 01:11:23.120 +I feel like I talk a lot about how Datastar is a journey of unlearning + +01:11:23.800 --> 01:11:29.060 +old and bad patterns, deeply rooted ones in what I think web development is. + +01:11:29.300 --> 01:11:31.780 +And these days, as I mentioned, + +01:11:32.060 --> 01:11:34.880 +I never would have thought that I'd be developing in Go, + +01:11:35.160 --> 01:11:37.120 +but I see like all like the, + +01:11:37.580 --> 01:11:39.060 +like I think even Python + +01:11:39.240 --> 01:11:41.220 +is getting better concurrency support, right? + +01:11:41.360 --> 01:11:44.400 +So I think you talked about that recently, Michael, here. + +01:11:44.620 --> 01:11:47.220 +So now I'm seeing with Datastar, + +01:11:47.360 --> 01:11:49.280 +I can do so much more on the backend. + +01:11:49.420 --> 01:11:51.720 +I can be so much more creative on the backend + +01:11:51.840 --> 01:11:53.200 +and that's what interests me. + +01:11:53.960 --> 01:11:55.120 +So it's just fun. + +01:11:55.420 --> 01:11:56.180 +What can I say? + +01:11:56.500 --> 01:11:57.080 +It's fun. + +01:11:57.220 --> 01:11:59.160 +I'm still really jealous of that presentation too. + +01:11:59.420 --> 01:12:00.000 +Well done with it. + +01:12:00.080 --> 01:12:01.100 +Yeah, certainly send me the link. + +01:12:01.100 --> 01:12:03.120 +I'll put it in the show notes and well done. + +01:12:03.600 --> 01:12:03.900 +Delaney. + +01:12:04.140 --> 01:12:07.080 +The irony is that like, I don't consider myself a web dev at all. + +01:12:07.410 --> 01:12:09.380 +It just happens to be something I do a little bit of. + +01:12:09.650 --> 01:12:13.940 +The thing that is the, when I first started making this public, I was like, Hey, I think + +01:12:14.000 --> 01:12:16.180 +I'm onto something like someone proved me wrong. + +01:12:16.670 --> 01:12:21.380 +I was, I'm a little bit more like kind of, I always say like in the jujitsu world type + +01:12:21.490 --> 01:12:24.880 +stuff, like you want someone to roll with you, not because you're trying to up them. + +01:12:24.980 --> 01:12:27.240 +It's that like, they're trying to help you find weaknesses in your game. + +01:12:27.520 --> 01:12:27.580 +Right. + +01:12:27.780 --> 01:12:30.780 +So I want there to be an active, like someone proved me wrong. + +01:12:31.080 --> 01:12:33.840 +And I'm at the point now where I feel so confident. + +01:12:34.240 --> 01:12:35.420 +I will put money on it. + +01:12:35.500 --> 01:12:38.440 +I've tried going out to people out in the dev Twitter and all that. + +01:12:38.760 --> 01:12:41.680 +I guarantee you, and I'm happy to put money up on this, + +01:12:41.980 --> 01:12:43.140 +if you could do it the Datastar way, + +01:12:43.500 --> 01:12:45.800 +whether you're using React or HTMLX or any other approach, + +01:12:46.140 --> 01:12:47.020 +it will be less code. + +01:12:47.300 --> 01:12:47.940 +It'll be faster. + +01:12:48.360 --> 01:12:48.940 +It'll be cheaper. + +01:12:49.460 --> 01:12:51.620 +And it'll be simpler to understand. + +01:12:52.060 --> 01:12:54.640 +I will take up anybody anywhere on that thing. + +01:12:55.100 --> 01:12:56.580 +Basically, it's not a boast. + +01:12:56.840 --> 01:12:58.860 +It's just the facts on the table. + +01:12:59.220 --> 01:13:02.480 +And it's a paradigm shift that I want the world to know about + +01:13:02.900 --> 01:13:03.940 +just so that people understand, + +01:13:04.280 --> 01:13:05.220 +hey, there's going to be someone + +01:13:05.240 --> 01:13:06.760 +that comes up with something better than I did, right? + +01:13:06.880 --> 01:13:07.520 +Like I'm standing, + +01:13:07.800 --> 01:13:10.280 +the reason why we have the fastest signal library in the world + +01:13:10.420 --> 01:13:11.620 +is because we listen to the people + +01:13:11.640 --> 01:13:12.360 +that are really good at that. + +01:13:12.460 --> 01:13:13.400 +We use alien signals. + +01:13:13.800 --> 01:13:15.440 +The reason why we have the fastest morphing library + +01:13:15.640 --> 01:13:16.900 +is that we listen to people and said, + +01:13:16.960 --> 01:13:18.900 +hey, there's people that care about this stuff + +01:13:18.920 --> 01:13:19.880 +and are working towards it. + +01:13:20.140 --> 01:13:21.700 +It's not that there's anything special here. + +01:13:22.080 --> 01:13:23.900 +It's that it's trying to build an ecosystem + +01:13:24.180 --> 01:13:25.500 +of like people that care about performance + +01:13:25.560 --> 01:13:27.160 +and people care about the details. + +01:13:27.600 --> 01:13:28.240 +And if you do that, + +01:13:28.560 --> 01:13:29.840 +then everything gets better. + +01:13:30.200 --> 01:13:32.340 +So it's not just where are we at now, + +01:13:32.440 --> 01:13:33.900 +but if anyone thinks they can do better, + +01:13:34.320 --> 01:13:34.900 +please join us. + +01:13:34.910 --> 01:13:35.660 +We want to hear it. + +01:13:35.960 --> 01:13:38.500 +But like, I'm done having the vibe code, + +01:13:38.660 --> 01:13:39.500 +or not the vibe code, + +01:13:39.560 --> 01:13:41.160 +but like the vibes around like, + +01:13:41.580 --> 01:13:42.620 +well, this doesn't feel like a spa + +01:13:42.710 --> 01:13:43.780 +or like a spa has its place. + +01:13:44.220 --> 01:13:45.300 +A couple of episodes ago, + +01:13:45.880 --> 01:13:47.880 +there was a Cody from the Litestar stuff said, + +01:13:48.160 --> 01:13:50.020 +there's a time and place for HTMX or Datastar. + +01:13:50.350 --> 01:13:52.000 +And he's just, that's just not true. + +01:13:52.400 --> 01:13:55.200 +It's just like hypermedia is the way to build things + +01:13:55.360 --> 01:13:57.160 +for a hypermedia client, which is the browser. + +01:13:57.600 --> 01:14:00.700 +So I will, anyone that can show that it's wrong, + +01:14:01.020 --> 01:14:01.640 +please let us know. + +01:14:01.980 --> 01:14:02.760 +Like we're here to help. + +01:14:02.860 --> 01:14:06.400 +I would love to see this paired with some Electron JS apps + +01:14:06.440 --> 01:14:08.120 +to make your desktop apps a little better too. + +01:14:08.280 --> 01:14:08.920 +So anyway. + +01:14:09.220 --> 01:14:10.320 +Seriously, oh my God. + +01:14:10.500 --> 01:14:11.500 +That's a different episode. + +01:14:11.740 --> 01:14:15.000 +So I just want to say thank you, Delaney, Ben and Chris. + +01:14:15.200 --> 01:14:15.940 +Thank you all for being here. + +01:14:16.480 --> 01:14:17.500 +And congrats on Datastar. + +01:14:17.500 --> 01:14:18.680 +It looks like a super project. + +01:14:18.940 --> 01:14:22.700 +I'm looking at some projects or an app running right over there + +01:14:23.160 --> 01:14:25.500 +on my left that I kind of wish I'd built Datastar. + +01:14:25.800 --> 01:14:29.220 +Well, and the thing is to make sure to not think that it's just used, like, yes, we talk + +01:14:29.260 --> 01:14:32.180 +about the real-time stuff, but it's better for even just normal crud stuff. + +01:14:32.260 --> 01:14:35.920 +And that's kind of hard to, like, it's not as sexy to talk about, but it's better at + +01:14:36.020 --> 01:14:36.280 +that too. + +01:14:36.400 --> 01:14:38.200 +Well, it's also 80% of what gets built. + +01:14:38.320 --> 01:14:40.180 +So it's important to like point it out, right? + +01:14:40.560 --> 01:14:40.640 +Yeah. + +01:14:41.080 --> 01:14:41.220 +All right. + +01:14:41.640 --> 01:14:41.920 +Bye everyone. + +01:14:42.260 --> 01:14:42.580 +Thanks for being here. + +01:14:42.740 --> 01:14:42.980 +Thank you. + +01:14:44.180 --> 01:14:46.320 +This has been another episode of Talk Python To Me. + +01:14:46.700 --> 01:14:47.420 +Thank you to our sponsors. + +01:14:47.640 --> 01:14:48.960 +Be sure to check out what they're offering. + +01:14:49.140 --> 01:14:50.480 +It really helps support the show. + +01:14:50.940 --> 01:14:52.320 +Take some stress out of your life. + +01:14:52.540 --> 01:14:58.120 +Get notified immediately about errors and performance issues in your web or mobile applications with Sentry. + +01:14:58.620 --> 01:15:03.060 +Just visit talkpython.fm/sentry and get started for free. + +01:15:03.560 --> 01:15:06.020 +Be sure to use our code talkpython26. + +01:15:06.720 --> 01:15:10.340 +That's Talk Python, the numbers two, six, all one word. + +01:15:11.020 --> 01:15:18.240 +This episode is brought to you by CommandBook, a native macOS app that I built that gives long-running terminal commands a permanent home. + +01:15:18.640 --> 01:15:20.640 +No more juggling six terminal tabs every morning. + +01:15:21.240 --> 01:15:24.040 +Carefully craft a command once, run it forever with auto restart, + +01:15:24.360 --> 01:15:25.900 +URL detection, and a full CLI. + +01:15:26.400 --> 01:15:29.380 +Download it for free at talkpython.fm/command book app. + +01:15:30.140 --> 01:15:32.020 +If you or your team needs to learn Python, + +01:15:32.240 --> 01:15:35.640 +we have over 270 hours of beginner and advanced courses + +01:15:35.960 --> 01:15:39.380 +on topics ranging from complete beginners to async code, + +01:15:39.520 --> 01:15:42.300 +Flask, Django, HTMX, and even LLMs. + +01:15:42.520 --> 01:15:44.800 +Best of all, there's no subscription in sight. + +01:15:45.340 --> 01:15:47.120 +Browse the catalog at talkpython.fm. + +01:15:47.860 --> 01:15:49.839 +And if you're not already subscribed to the show + +01:15:49.900 --> 01:15:51.200 +on your favorite podcast player, + +01:15:51.840 --> 01:15:52.480 +what are you waiting for? + +01:15:53.160 --> 01:15:54.900 +Just search for Python in your podcast player. + +01:15:55.080 --> 01:15:55.900 +We should be right at the top. + +01:15:56.340 --> 01:15:57.860 +If you enjoyed that geeky rap song, + +01:15:57.970 --> 01:15:59.140 +you can download the full track. + +01:15:59.310 --> 01:16:01.200 +The link is actually in your podcast blur show notes. + +01:16:02.040 --> 01:16:03.360 +This is your host, Michael Kennedy. + +01:16:03.760 --> 01:16:04.820 +Thank you so much for listening. + +01:16:05.050 --> 01:16:05.820 +I really appreciate it. + +01:16:06.260 --> 01:16:06.960 +I'll see you next time. + +01:16:18.840 --> 01:16:21.640 +I'm out. + diff --git a/transcripts/538-python-in-digital-humanities.txt b/transcripts/538-python-in-digital-humanities.txt new file mode 100644 index 0000000..67af577 --- /dev/null +++ b/transcripts/538-python-in-digital-humanities.txt @@ -0,0 +1,2282 @@ +00:00:00 Digital humanities sounds niche until you realize that it can mean a searchable archive of U.S. + +00:00:05 amendment proposals, Irish folklore, or pigment science in ancient art. Today I'm talking with + +00:00:11 David Flood from Harvard's DARTH team about an unglamorous problem. What happens when the grant + +00:00:18 ends? But the website can't. His answer? Static sites, client-side search, and sneaky Python. + +00:00:24 Let's dive in. This is Talk Python To Me, episode 538, recorded January 22nd, 2026. + +00:00:48 Welcome to Talk Python To Me, the number one Python podcast for developers and data scientists. + +00:00:53 This is your host, Michael Kennedy. + +00:00:55 I'm a PSF fellow who's been coding for over 25 years. + +00:00:59 Let's connect on social media. + +00:01:00 You'll find me and Talk Python on Mastodon, BlueSky, and X. + +00:01:04 The social links are all in your show notes. + +00:01:06 You can find over 10 years of past episodes at talkpython.fm. + +00:01:10 And if you want to be part of the show, you can join our recording live streams. + +00:01:14 That's right. + +00:01:14 We live stream the raw uncut version of each episode on YouTube. + +00:01:18 Just visit talkpython.fm/youtube to see the schedule of upcoming events. + +00:01:23 Be sure to subscribe there and press the bell so you'll get notified anytime we're recording. + +00:01:27 This episode is brought to you by Sentry. + +00:01:29 Don't let those errors go unnoticed. + +00:01:31 Use Sentry like we do here at Talk Python. + +00:01:33 Sign up at talkpython.fm/sentry. + +00:01:37 And it's brought to you by CommandBook, a native macOS app that I built that gives long-running + +00:01:42 terminal commands a permanent home. + +00:01:44 No more juggling six terminal tabs every morning. + +00:01:46 Carefully craft a command once, run it forever with auto-restart, URL detection, and a full + +00:01:51 CLI. + +00:01:51 Download it for free at talkpython.fm/command book app. + +00:01:56 Hello, David. Welcome to Talk Python To Me. Amazing to have you here. + +00:01:59 I'm glad to be here. Talk Python has been part of my story up to this point. + +00:02:03 Has it? Okay. Well, you are about to write the next chapter in the story. So that's pretty excellent. + +00:02:10 I have a sense of what's coming. We planned out what we're going to talk about and that sort of thing. + +00:02:15 And I'm really excited about this topic. So it's going to be a good one. + +00:02:21 Honestly, I think one of the real powers of the Python community and the reason the language has such staying power is there's such a diversity of use cases, technology, like technology standpoints, right? + +00:02:34 Like I build software for this group or I build these types of apps and it's not just, you know, like Ruby on Rails, which, you know, it's been very popular, but it's, it's for websites, right? + +00:02:44 You know what I mean? + +00:02:45 Yeah, absolutely. + +00:02:47 I mean, web development has dominated my use of it, but my entry into it, which I suppose I'll mention in a moment, was through all those little tools. + +00:02:57 Let's hear it. + +00:02:58 Who are you, David Flood? + +00:03:00 Tell us, introduce yourself real quick and tell us about how you got into it. + +00:03:04 So my background is in music and the humanities. + +00:03:09 I mean, in 2019, I didn't know what Python was or the name of any programming language. + +00:03:16 and I've been doing textual criticism, which is, you know, there's lots of criticisms in the academy. + +00:03:22 This is the one where if you have lots and lots of versions of the same text, + +00:03:27 you are comparing them to work out what the initial text was and like how it changed over time. + +00:03:33 Okay, give us an example. + +00:03:35 Okay, so one of the famous examples, hope I can remember it off the top of my head, + +00:03:40 is from Shakespeare. + +00:03:42 We're all familiar with the line to be or not to be. + +00:03:45 is the question. That is the question. Well, there's a variant of it. One of the early copies + +00:03:53 written by Shakespeare himself has... Somebody's going to be able to type into the chat exactly + +00:03:58 what it is. They'll know this anecdote. But it's something more like, "To be or not to be, I." + +00:04:04 That's the question. And so, which one is the original one? Why did he change it? That's kind + +00:04:09 of one example i work mainly in the in the new testament which is especially complicated because + +00:04:15 no other corpus from ancient history has as many copies of the same text as that corpus does so it's + +00:04:23 quite um quite quite complicated and our techniques have have grown grown because of that and perhaps + +00:04:29 become more advanced than now i mean that many variations over that huge span of time over + +00:04:37 different groups with different, maybe not intentions, but certainly colored by different + +00:04:43 worldviews and philosophies and so on. And yeah, I see the trouble. + +00:04:47 No, yeah. And they were people of the book. So copying it is something that happened a lot. And + +00:04:54 they copied the monks, like the medieval monks copied everything. They copied our Greek classics. + +00:05:01 So that's what I was interested in. And because of the wealth of data that we have, + +00:05:07 Computer tools are more and more important in that field. + +00:05:11 So when I started my PhD in 2019, I knew that I wanted to use some of these cutting-edge tools. + +00:05:17 Some of them may be surprising. + +00:05:19 For example, we've been using phylogenetic software. + +00:05:24 This is software that evolutionary biologists are using or computational biologists are using to track, for example, how COVID strains mutate over time. + +00:05:35 Oh, interesting. + +00:05:36 What they're comparing are the DNA letters. + +00:05:40 And so you have the sequence of letters and you're comparing how those change over time. + +00:05:44 Well, you can swap in textual variants for DNA letters. + +00:05:48 And now we can track how texts change over time and group them into families, things like that. + +00:05:56 It's like a time series, but of words or letters or something. + +00:05:59 Yeah, I mean, yeah, there's lots of important algorithms for comparing + +00:06:06 sequences of things. And so if we can just swap in Greek words and Greek text instead, + +00:06:12 then we can maybe apply it to textual criticism. So I was pretty interested in those things. That + +00:06:16 wasn't actually the method that brought me into it, but something like that, kind of computer + +00:06:21 intensive tools. What I learned is that these tools weren't actually available to me. They + +00:06:27 weren't desktop applications. And for the most part, they weren't public web applications. They + +00:06:36 PyPI or something like that, right? + +00:06:38 Yeah, exactly. + +00:06:39 Exactly. + +00:06:39 Or Java. + +00:06:41 And I needed to glue them together. + +00:06:43 So the long story short on that is during the first year of my PhD, I was picking up Python, + +00:06:50 watching YouTube videos while I was doing the dishes. + +00:06:52 And then the pandemic hit while I was living in Edinburgh in Scotland, probably not far + +00:06:57 from Will McCoogan. + +00:06:59 And so the pandemic gave me the excuse to spend even a few more hours each day picking up these + +00:07:06 new, these new technical skills. And so I did it, I was able to use these advanced tools in my in my + +00:07:13 work. But what was really important to me was sharing, like making that available to my colleagues, + +00:07:18 is I had to I had to move from writing these like bad top to bottom Python scripts into things that + +00:07:23 could be reused by other people. And that led me into the web, because the web is where that's how + +00:07:29 I can share with anybody. It's really wild how much the web is kind of the last bastion of + +00:07:36 app freedom. It's so bizarre because, you know, I've many times told the stories of the insane + +00:07:42 battles of just getting our apps that just playback video of content that's already on the web + +00:07:48 into the app store. I mean, weeks of fighting about the weirdest, most nonsensical things with + +00:07:54 both Google and Apple. But we also now have the Mac platform and the Windows platform very + +00:08:01 aggressively looking for digital code certificates and all sorts of signing and other kinds of proof + +00:08:07 like it you can't even just send somebody an executable anymore it won't run it's it's crazy + +00:08:13 it's it's down to like okay put it on the web i guess that's right i i i played the game of + +00:08:19 distributing desktop apps that's how i did it that's why i initially distributed things um + +00:08:25 and at this point i just require people to install python and then install my desktop app from pypi + +00:08:30 because it's too hard otherwise for me. + +00:08:33 I mean, I could pay for the code signing from Apple and do all of that, + +00:08:37 but it's just, it's too much work for the time that I have. + +00:08:40 Yeah, I'm about to do another round of it. + +00:08:42 I'm working on an app and my developer account is still active. + +00:08:45 So we might have a fresh round of fun. + +00:08:47 Hopefully it goes through this time. + +00:08:50 Anyway, I do think it's such a challenge. + +00:08:52 And are you leveraging? + +00:08:53 I don't know if the timing was right. + +00:08:55 Like maybe this was too early, but these days, are you leveraging things like uvx + +00:09:00 to run, or are you just pip install this thing and then run it? + +00:09:04 Yeah, I haven't updated the readme in a while, so I think it just asks for pip. + +00:09:08 But certainly, if somebody asked me today, I would say, yeah, just install this with uv. + +00:09:14 Because then they don't even need Python. + +00:09:16 Exactly. + +00:09:17 And that's brilliant. + +00:09:18 And that's a really, it is another barrier reduced in distributing these applications, + +00:09:23 right? + +00:09:23 Like, if you can get uv installed on a machine, then you don't even have to say install, just + +00:09:28 The way you run it is uvx my thing and it's all transparent to you, right? + +00:09:33 Which is beautiful. + +00:09:33 So what was it like? + +00:09:35 Yeah. + +00:09:35 So what was it like coming from what sounds like a not super screen focus, super + +00:09:43 techie aspect and having to dive into this world and someday you're probably + +00:09:47 like, how is it that I'm publishing stuff to PyPI? + +00:09:49 What has happened to me? + +00:09:51 Yeah. + +00:09:51 well, yeah, I remember when I, when I first signed up for GitHub, because + +00:09:56 you know, whatever YouTube tutorial I was working through at the time, you know, said that I needed + +00:10:02 to do that. You know, I think it all started making a lot of sense. I didn't have any technical + +00:10:08 background, but the world kind of open source software, it just kind of made sense. It felt + +00:10:17 like it fit really well into my academic, you know, circle. I think a lot of the attitudes are + +00:10:23 similar. I agree. I think they are actually. And I think that's, I think that's a pretty neat thing. + +00:10:28 Yeah. Very cool. All right. Well, let's talk about what you're doing with digital humanities. + +00:10:34 You're actually at a really interesting project or organization, I guess, that does many projects, + +00:10:40 right? Yeah. Yeah. So fast, fast forwarding, I did, I finished my PhD in the humanities. + +00:10:45 Sorry. I had so much fun. No, that's fine. That's fine. I had so much fun writing like these tools + +00:10:50 and then just solving the distribution problem to share them with other scholars. + +00:10:56 That was so fun that I was open to this kind of opportunity + +00:11:00 where now I'm doing this full time. + +00:11:02 And so, yes, so I'm on the, we call it affectionately Darth, + +00:11:07 which is digital arts and humanities at Harvard. + +00:11:11 There has to be a lot of Star Wars memes and references, + +00:11:14 I'm sure. + +00:11:14 If you can pull up a 404, I think there will be a Darth Vader reference. + +00:11:19 Seriously, I'm here for it. + +00:11:22 Yes, page not found. I find your lack of nav disturbing. + +00:11:27 You know what? I think that is beautiful. And I really, I really think that people should embrace + +00:11:33 the 404, the fun 404 page, you know, more, right? There should really be something going on that + +00:11:40 like makes it, you know, something hasn't worked out, but you can just, you can make people laugh. + +00:11:46 Yeah. I appreciate that. + +00:11:48 I've heard people push back against it. + +00:11:50 Like if you're on a, if you're on like your medical website and you're maybe about to get bad news and then you get like a picture of a kitten. + +00:12:00 Dr. Kitten doesn't know where your results went. + +00:12:02 Like I get that. + +00:12:02 That's not funny. + +00:12:04 But I mean, most things are not that serious. + +00:12:06 Yeah. + +00:12:07 Mostly. + +00:12:08 Okay. + +00:12:09 So what kind of things does Darth do? + +00:12:12 You've described this as kind of a web or tech agency within Harvard. + +00:12:17 Yeah, it is very much. + +00:12:18 So, you know, Harvard has a gigantic IT group. + +00:12:21 I don't know how many hundreds of people work, but more than 500 people in IT. + +00:12:28 We are a small team and we operate very much like a small agency. + +00:12:33 So usually what happens is a faculty member has a funded research project that's going to last for an amount of time. + +00:12:42 And then we consult with them to build it. + +00:12:44 And most of the time, I kind of think of these as I kind of have these different categories of these kinds of projects that I think of. + +00:12:54 I lost in my notes what I call them. + +00:12:56 But they are there. + +00:12:57 You have like a one is like a virtual research environment. + +00:13:01 So the focus is this is this is a platform that we're building for the research to be done on. + +00:13:07 Like the reason the research should be done in like a web app would be because you have access to visualization, to Postgres, to Pandas. + +00:13:17 So we can kind of build up this platform to do the actual research on and some of the data entry. + +00:13:23 So like a full on research application. + +00:13:26 Yeah, exactly. + +00:13:27 I guess you can also kind of see your work through the different stages of research projects and academic research and so on. + +00:13:36 And we'll get to maybe end of life in a sense further down in the conversation. + +00:13:42 But so this would be we have a grant or we just work here and we're going to work on some form of research. + +00:13:49 What do you give them? + +00:13:50 Right. And I think that's a super interesting challenge because one of the real common answers would be Jupyter, Jupyter Lab, Marimo, whatever. + +00:13:59 But that's still pretty code heavy for people who are possibly philosophers or something, you know. + +00:14:05 Oh, exactly. That's why in digital humanities, I won't even, maybe I won't even attempt to define + +00:14:13 it in any narrow sense, because I'll get in trouble with somebody. But you have two groups + +00:14:20 that are interfacing with each other. And one is digital humanities as a field, like as a subfield, + +00:14:26 all of its own. And these are people who have humanities domain, like knowledge, + +00:14:31 and technical skills, and they're bringing them together. And in a lot of cases, the audience for + +00:14:36 that kind of work is other people working in the digital humanities. But far more common, + +00:14:42 and this is what we work with, is people who have humanities domain expertise, and they want to + +00:14:49 publish or do research or share with other people who have that same humanities domain expertise, + +00:14:55 and they are now interested in adding a technical component to it. + +00:14:59 How can we supercharge what they have? + +00:15:03 This portion of Talk Python is brought to you by Sentry. + +00:15:06 I've been using Sentry personally on almost every application + +00:15:10 and API that I've built for Talk Python and beyond over the last few years. + +00:15:14 They're a core building block for keeping my infrastructure solid. + +00:15:18 They should be for yours as well. + +00:15:19 Here's why. + +00:15:20 Sentry doesn't just catch errors. + +00:15:22 It catches all the stuff that makes your app feel broken, + +00:15:25 the random slowdown, the freeze you can't reproduce, that bug that only shows up once real users hit it. + +00:15:30 And when something goes wrong, Sentry gives you the whole chain of events in one place. + +00:15:34 Errors, traces, replays, logs, dots connected. + +00:15:38 You can see what's led to the issue without digging through five different dashboards. + +00:15:42 SEER, Sentry's AI debugging agent, builds on this data, taking the full context, + +00:15:47 explaining why the issue happened, pointing to the code responsible, drafts a fix, + +00:15:52 and even flags if your PR is about to introduce a new problem. + +00:15:56 The workflow stays simple. + +00:15:58 Something breaks, Sentry alerts you, the dashboard shows you the full context, + +00:16:02 Seer helps you fix it and catch new issues before they ship. + +00:16:06 It's totally reasonable to go from an error occurred to fixed in production in just 10 minutes. + +00:16:12 I truly appreciate the support that Sentry has given me + +00:16:15 to help solve my bugs and issues in my apps, especially those tricky ones that only appear in production. + +00:16:21 I know you will too if you try them out. + +00:16:22 So get started today with Sentry. + +00:16:24 Just visit talkpython.fm/sentry and get $100 in Sentry credits. + +00:16:30 Please use that link. + +00:16:31 It's in your podcast player show notes. + +00:16:32 If you're signing up some other way, you can use our code talkpython26, all one word, + +00:16:38 talkpython26, to get $100 in credits. + +00:16:41 Thank you to Sentry for supporting the show. + +00:16:44 Maybe just take a moment and speak to, maybe, I don't know if this venue will actually speak + +00:16:49 directly to anybody who I was imagining here, but people who work with folks, what would you tell + +00:16:54 somebody who works with a group who have some technical skill, who could create some of these + +00:16:58 things that we're going to talk about, but the people who they've created for don't necessarily + +00:17:02 think they need it or know that they need it. I've gone often on rants about how programming is a + +00:17:09 superpower, not a replacement for your job, right? Yeah. That's a problem for a lot of people, + +00:17:15 especially because you might use some new computer tools to supercharge your research. + +00:17:20 But the article that you publish or the research output of that, the audience, they may not + +00:17:25 be interested in hearing about that at all. + +00:17:28 And so for most people who are working in this space, the tools, you have to use them + +00:17:33 in such a way that you can talk about the research output without talking about the + +00:17:37 tool. + +00:17:38 And we have other venues to talk about the tools themselves, like the Journal for Open + +00:17:43 source software and you can kind of get some of it out there. But that is a, that's the significant + +00:17:48 challenge is convincing people that it, that it could be useful and then convincing the audience + +00:17:53 that they should be interested in kind of the methods behind how some of the new research comes + +00:17:57 up. Also, I think I'm a big believer that presenting stuff in the right order is really, + +00:18:03 really important. If you present your research and it's beautiful and powerful and oh, look, + +00:18:07 we've also, by the way, covered a hundred times more data than any prior research. Surprise, + +00:18:12 I wonder how I did that. + +00:18:14 And then people are like, this is amazing. + +00:18:16 Then after you kind of hook them with the inspiration and what's possible, + +00:18:19 then you're like, let me tell you about the tool. + +00:18:21 And all of a sudden you're like, that's a cool tool, right? + +00:18:22 This is not just like geekery, like programmer, you know, + +00:18:26 Charlie Brown speak, wah, wah, wah, wah, wah. + +00:18:28 You know, it's like, no, I'm listening. + +00:18:29 Tell me now. + +00:18:30 Yeah, exactly. + +00:18:31 I mean, one of the things I think that really opens people's eyes + +00:18:35 is a really powerful search interface. + +00:18:38 You have all of this research data. + +00:18:40 just put it behind Elasticsearch with some really good filtering on it. And all of a sudden you have + +00:18:45 fast, rapid access to the data in a way you never had before. Like you were never scrolling through + +00:18:51 the Excel spreadsheets and finding exactly what you wanted, like you were with this new search + +00:18:55 interface. And that by itself is like so simple. We're so used to that in web development that + +00:19:00 like everything needs to have a fantastic search now. But so many people have their data locked + +00:19:05 behind, you know, a terrible search interface. + +00:19:07 Yeah, just a few things to sort of expose that. + +00:19:10 So this, give us a sense of what these data exploration web apps might look like. + +00:19:14 These are probably kind of mostly stuck to the inside, kind of internal to the research + +00:19:20 lab research team groups and so on. + +00:19:22 These are probably not that public facing, right? + +00:19:24 Almost everything we work on does end up having a public facing component. + +00:19:28 So maybe the research itself is done, locked behind a user login. + +00:19:34 That's just for the researchers. + +00:19:36 But then they expose that research to the public, usually with a good search interface + +00:19:41 and different pages for exploring their data and visualizations and things like that. + +00:19:47 So yeah, everything we do ends up becoming a production public web app in the end. + +00:19:52 And then another one of your categories, you put it was virtual research environments + +00:19:57 like data entry, publishing, authoring, collaboration. + +00:20:00 Tell us about that. + +00:20:01 Yeah, so a good example of this maybe is one of the projects that... Well, actually, the best example of it is the project I worked on + +00:20:08 during my PhD. It's called Apatosaurus. The short story behind the name is that it sounds like + +00:20:16 apparatus. In textual criticism, when you are displaying and visualizing variant readings to + +00:20:24 a base text, that form of visualizing it is a critical apparatus. A critical apparatus is a + +00:20:32 a pretty boring website name, but Apatosaurus dinosaurs might make textual criticism sound fun. + +00:20:37 Yeah, I do love dinosaurs. No, that's really cool. So this, this comes out as a web app. And I know + +00:20:43 you also have some, you talked about some desktop apps as well. + +00:20:46 Yep. Yep. That's right. So, yeah. So, so there's this people, people upload their, + +00:20:50 their collation to this and then they can visualize it. And like there, there's a public + +00:20:55 component of this as well, but really the backend is editing, editing a collation, + +00:21:00 and adding notes to all of the different readings and stuff. + +00:21:03 So I could show what the backend looks like, but we can also move on. + +00:21:08 - Let's move on just because most people will not totally hear, but just give us a sense of like, + +00:21:14 like what do people, what do you create for people so that they're like, yeah, I can use this app, right? + +00:21:21 Like give us a sense of some of the features, I guess is what I'm getting to. + +00:21:25 - Yeah, so another good example is we have a project at Harvard called Mapping Color in History. + +00:21:33 And this is a collaboration with a lab. + +00:21:37 This lab brings in pieces of artwork and they do spectral analysis on the pigments + +00:21:42 so they can identify what was used to make a particular color of this red + +00:21:48 or what was made to make this color of blue. + +00:21:51 And then the idea is tracking how did people make those pigments over time, + +00:21:57 over time and specifically in Asian art. + +00:22:02 Is this the Dharmra, Puna, Puna? + +00:22:05 No, this is mapping color in history. + +00:22:08 I don't think it's up here. + +00:22:09 Sorry about that. + +00:22:10 Somewhere. + +00:22:10 That's all right. + +00:22:11 I'll find it. + +00:22:12 Keep talking. + +00:22:13 Okay. + +00:22:14 So the front end is great. + +00:22:16 You know, like the public end, this is people can explore by pigments + +00:22:21 and then see the images that contain those pigments. + +00:22:24 Now in the back end, what the researchers will be able to do is correlate exactly which + +00:22:30 point of a painting the analysis was done on. + +00:22:34 So they have this deep zoom image viewer where they'll zoom in and they'll select the point + +00:22:39 where that was taken from. + +00:22:41 So how else would you do that other than a digital interface to indicate on an image of + +00:22:47 a painting where that spectral analysis was performed? + +00:22:52 Sounds almost like astronomy in a weird way. + +00:22:55 Oh, yeah. + +00:22:55 We zoomed into here and we took a different spectrum of the painting and we realized that it's actually identical to this, you know, something crazy like that, right? + +00:23:04 Yeah, yeah, yeah, that's right. + +00:23:06 Yeah, so it's essentially a pigments, like a pigments database. + +00:23:10 So the third category of these digital humanities projects that you put down was like data extraction, transformation. + +00:23:19 In data science, they often say, you know, 80% of the work is the data wrangling, which is like cleaning, organization, just getting it so you could possibly start asking questions about it. + +00:23:29 I'm sure you all do a lot of that. + +00:23:31 Absolutely. + +00:23:32 So often, the very beginning of a project might be an Excel sheet or several spreadsheets. + +00:23:41 And the first task is to ingest these into, you know, a proper database. + +00:23:46 Not so much MongoDB for us. + +00:23:48 It's going into Postgres. + +00:23:50 We're Django Shop. + +00:23:51 We're Django Shop. + +00:23:52 So it's going into Postgres. + +00:23:55 And yeah, no, that is probably the number one challenge of the early stage is figuring out what the right data model is, what the right relationships are to model the data. + +00:24:07 Doing that work is advantageous to everybody because, you know, it helps both the researchers who brought the data to think about it in a more organized way. + +00:24:17 I mean, they've been trying to do that. + +00:24:18 And they have the spreadsheets. + +00:24:20 But now we're modeling out the data so that we can add it to database tables and then to use later. + +00:24:27 So that works out well for everybody. + +00:24:30 And yeah, absolutely. + +00:24:31 Cleaning the data, getting dates, working with fuzzy dates, being able to parse July of 2020 or summer of 2020 and handling kind of all of those cases so that we do get dates in the end. + +00:24:45 One of the crazy stories from data parsing history is one of the, I can't remember exactly what it was, you talked about biology tools or genetics tools earlier. + +00:24:56 One of the groups that names genes had to change the name of a gene because it kept getting parsed by Excel into a date. + +00:25:04 Yeah, I remember that. + +00:25:05 I remember that. + +00:25:06 That's right. + +00:25:07 Yes. + +00:25:08 So these are the weird edge cases I'm sure you run into. + +00:25:11 Like it's not even supposed to be a date. + +00:25:13 Why is this a date? + +00:25:13 I don't know. + +00:25:14 Why is it helping out here? + +00:25:16 The code keeps crashing. + +00:25:18 Like pandas parsed it as a date and it's not or whatever. + +00:25:21 Absolutely. + +00:25:21 Yeah. + +00:25:22 Yeah. + +00:25:22 So yeah, usually lots of test suites around that ingest process until we've got it. + +00:25:27 Now, once we've got it in, usually the research is ongoing and then we're able to provide + +00:25:32 them now a new cleaned interface to do the additional data entry as the project is going. + +00:25:38 And that's usually a win-win for everybody. + +00:25:40 Sure. + +00:25:40 And so this sort of ETL ingestion side of everything is it's like, don't worry, + +00:25:46 Darth has got it for you. + +00:25:47 And then we'll provide you like a database connection to start working. + +00:25:51 Or do you give them the tools and then they kind of iterate on them? + +00:25:54 And how much is this you and how much is this you providing like CLI tools and stuff + +00:26:00 or notebooks over to people? + +00:26:03 I'd say most of the people that we're working with are aware of the technical tools, + +00:26:08 but they don't want a database connection. + +00:26:10 So we are giving them, we're doing the ingest and then building a platform where they can begin interacting with their data. + +00:26:17 Yeah, I'm sure they don't want one. + +00:26:20 Maybe you give them an app though, right? + +00:26:22 With like Elasticsearch and other things that they can. + +00:26:25 No, absolutely. + +00:26:25 Yeah, that's what we do. + +00:26:26 Yeah, okay. + +00:26:27 Yeah, we give them a web platform to begin exploring, to begin publishing. + +00:26:34 So I was thinking that you said you're a Django shop, which is cool. + +00:26:38 It sounds, though, to me like describing what you're doing, just imagining how this is. + +00:26:43 You're probably creating these projects often. + +00:26:46 How often does one of these projects actually last? + +00:26:49 Or how many of them do you iterate? + +00:26:53 I'm trying to get a sense. + +00:26:53 Do you work on stuff for a year or is it like every two weeks we're on a new project? + +00:26:58 It's why I think of us as like an agency. + +00:27:00 Because we get to work on greenfield projects fairly often, like you're imagining. + +00:27:04 Which would not be the case normally at a big university IT department. + +00:27:09 So, you know, maybe two or three projects a year, two or three big ones a year. + +00:27:15 And then we have to put to bed a few a year as well. + +00:27:18 Because these things, they're funded with grant money. + +00:27:21 And then the grant money runs out and it's time. + +00:27:24 And then we have to figure out what do we do with it now? + +00:27:26 We don't want to lose the data and this way of presenting it. + +00:27:31 But we can't keep paying for Elasticsearch. + +00:27:33 Yeah, of course. + +00:27:34 I'm certainly, we're going to dive into that because that is, but let's save that for the + +00:27:37 end. + +00:27:37 It seems like that's the arc of the story of these things. + +00:27:40 But I certainly think it's something that you don't think about that much, right? + +00:27:44 Like you said, it was only a hundred dollars a month for this. + +00:27:47 And we got a big grant. + +00:27:48 There's a bunch of, no big deal. + +00:27:49 But like when the grant's out, who's on the hook for a hundred dollars a month and making + +00:27:53 sure it survives upgrades and all that kind of business. + +00:27:56 No, that's right. + +00:27:57 Yeah. + +00:27:57 So my original question when I started on this path was thinking like, do you, how do you + +00:28:02 get started on these? + +00:28:03 Do you have like a big framework or a cookie cutter sort of thing or something like this + +00:28:07 is how we do it because it plugs into all this other automation and tools we built for + +00:28:11 the last 10 projects. + +00:28:13 You know, that's kind of a unique position. + +00:28:14 A lot of companies build one website for themselves and that's their app or they're + +00:28:19 an agency that goes across so many, so much variation. + +00:28:21 They can't do that kind of stuff. + +00:28:22 Right. + +00:28:23 That's right. + +00:28:24 That's right. + +00:28:25 That's a good question. + +00:28:26 We have things that we reuse. + +00:28:29 Some of them are open source, different search components and things that we maintain that + +00:28:36 we'll use across projects. + +00:28:37 And we have tried to do the cookie cutter Django project. + +00:28:41 The truth is, each project is different enough that really we like to evaluate it from first + +00:28:47 principles as we're evaluating it and thinking, what is the best technology to use? + +00:28:55 Yeah. + +00:28:55 Yeah. + +00:28:56 So yeah, we don't have a cookie cutter. + +00:28:59 We don't have a kind of a meta framework for bootstrapping them because they're sufficiently + +00:29:04 different from each other that we... + +00:29:05 I find that too. + +00:29:07 I find that too. + +00:29:08 The idea of how we could just grab this cookie cutter or copier. + +00:29:12 Are you familiar with copier? + +00:29:14 People out there might be familiar with that. + +00:29:15 It's a little bit like cookie cutter with the bonus that you can update it later if you + +00:29:21 change your mind about something, like actually change this project to use Postgres rather + +00:29:24 than SQLite or something, which is pretty cool. But every time that I do, every time I try to work + +00:29:30 with one of those projects, even ones that I've created for myself, I'm not, I hate not anyone. + +00:29:34 I'm like, oh, it's like 75% awesome and 25%. I just got to take this stuff out. You know, + +00:29:39 I'll just, I'll just do it from scratch. It's not, how hard is this? I'll just create a few folders + +00:29:43 and put a few things in there and I'll copy the one, like the pyproject.tom or like the one thing + +00:29:48 that's like, how do I do this again? I'll just copy that and we're good to go. Yeah. I mean, + +00:29:52 That's what I find. + +00:29:53 That's what I find. + +00:29:53 I find it, it seems like a really brilliant idea, but in practice, it hasn't saved us time yet. + +00:30:00 No, I mean, maybe it's a case study. + +00:30:02 Like, okay, let's see what they're doing for this one. + +00:30:04 Oh, that is interesting how they're integrating this other thing maybe, + +00:30:07 but as a true foundation, I find it in theory awesome. + +00:30:11 In practice, I just end up not doing it for various reasons. + +00:30:14 Don't know why. + +00:30:14 I'm gonna save this for later. + +00:30:16 Because the question I'm about to ask you is gonna send us just down a rat hole. + +00:30:21 So instead, before we go down the rat hole, maybe we could, not that one, maybe we could + +00:30:27 talk about, I mean, you talked about some, but let's maybe just feature some of the projects + +00:30:32 that are maybe more well-known that you guys have done. + +00:30:35 Sure. + +00:30:35 Yeah, good. + +00:30:36 So yeah, one of them is called the Amendments Project. + +00:30:40 And this is, I didn't know this until I started working on this project, that there are, there + +00:30:46 There have been thousands of, I think it's 22, at least 22,000 proposed amendments to + +00:30:52 the United States Constitution that never went anywhere. + +00:30:56 And so kind of the goal of this project is to show that there have been lots of attempts + +00:31:02 to amend the Constitution, but actually the Constitution is frozen. + +00:31:06 I mean, it's not actually amendable anymore, at least not in the politics of any time recently. + +00:31:12 So this is a database. + +00:31:14 I cannot imagine a situation where the U.S. Constitution gets amended. + +00:31:19 It has to be unanimous across all the states, right? + +00:31:21 Is that right? + +00:31:22 I can't remember. + +00:31:23 I don't know. + +00:31:23 I remember off the top of my head if it has to be unanimous, + +00:31:25 but it certainly has to be across party lines. + +00:31:28 Yeah, it's got to be pretty darn close if it's not at all. + +00:31:32 It's like time travel or travel to speed of light. + +00:31:36 Could be theoretically possible. + +00:31:38 Probably not going to happen. + +00:31:40 No, it's hard to see. + +00:31:41 It's hard to see. + +00:31:42 Yeah. + +00:31:42 So this is from a historian at Harvard. + +00:31:46 And so it's a database of all and the full text from all of these amendments. + +00:31:53 And, you know, it's from the public's point of view, it's a Postgres full text vector search interface for finding and filtering through on all of the different amendments that have been proposed. + +00:32:08 I love it. + +00:32:08 Yeah, this is a nice looking site. + +00:32:10 We work with a designer. + +00:32:12 she's very good yeah of course like an agency would right yep yep nice so we'll + +00:32:17 get a really pretty rich search interface and then off you go I have no idea even + +00:32:22 what I would search for but yeah well you can always search for something + +00:32:25 religious something abortion related there's gonna be lots of things there I + +00:32:29 thought all those also like guns but like I don't want to go down I'm not sure I + +00:32:32 even want to go down there right awesome though this looks super useful maybe + +00:32:37 someday we'll have a functional government again we'll see let's let's + +00:32:41 change it or maybe we'll go down and it's folklore like look at you so all right so yeah so another + +00:32:45 really great uh project at least from a content point of view uh that's interesting um the research + +00:32:51 that it's doing um is the fin folklore database um which so in in in celtic storytelling you know + +00:33:01 um moms have been telling and telling stories to daughters and and and and people have been + +00:33:08 telling stories for a very long time hundreds or a thousand years about um finn mcummel who is a + +00:33:14 hero a hero from irish mythology some of it some of it based in you know historical events but it + +00:33:21 goes back it goes back so far um so there are there's many hundreds or thousands of of of these + +00:33:29 stories that have been spread and versions of these stories that have that have been told and + +00:33:33 And so some of them are audio recordings where somebody like some researcher has gone out to an island off the coast of Scotland and recorded somebody telling their version of the hero of Finn and his band of heroes. + +00:33:47 You know, they defend Scotland and Ireland from invaders and attackers. + +00:33:53 Very exciting stories and stuff and a team of characters. + +00:33:59 So there's audio recordings and then there's documents, like written documents that contain + +00:34:05 these. + +00:34:05 And so this is a database of kind of all of those all in one place with, on the public + +00:34:11 side, a nice search interface for discovering them, you know, either using the map view or + +00:34:18 searching. + +00:34:18 Yeah, that's cool. + +00:34:19 I got my map view for some random thing I searched about here. + +00:34:22 Amazing. + +00:34:23 But this is pretty interesting, all these different tellings and stuff. + +00:34:26 Oh, and yeah, one of the big challenges with this project is that it's fully internationalized. + +00:34:33 So it's available in English. + +00:34:35 Everything is available in English, Scottish Gaelic, and Irish Gaelic, but that extends + +00:34:40 into the database. + +00:34:41 So usually people have multiple names recorded for them. + +00:34:45 And so, yeah, you may have one person with any number of names in different languages, + +00:34:51 sometimes more than one Scottish name, that kind of thing. + +00:34:54 And so the data model on this one is quite messy, but sensible. + +00:35:00 But yeah, it's quite a lot of different kinds of data to wrangle. + +00:35:03 And then with all of the translations for each thing. + +00:35:05 Yeah, that's wild. + +00:35:06 It's not just, we need the user interface of this thing to translate about. + +00:35:12 That's way more, right? + +00:35:13 Yeah, yeah, it is that. + +00:35:14 It is that. + +00:35:15 And then it is also, yes, all the items in the database have a translation or can. + +00:35:22 This portion of Talk Python To Me is brought to you by us. + +00:35:25 I'm thrilled to announce a brand new app built for developers created by yours truly. + +00:35:30 It's called Command Book. + +00:35:32 You know that thing you do every morning? + +00:35:34 Open up six terminal tabs, CD into this directory, activate that virtual environment, + +00:35:39 run the server with --reload. + +00:35:40 Now, CD somewhere else, start the background worker, another tab for Docker, + +00:35:45 another one to tail production logs. + +00:35:46 Every tab just says Python, Python, Python, Docker tail. + +00:35:50 and you're clicking through them going, which Python was that again? + +00:35:53 Where my app is running? + +00:35:55 Then sometime later, your dev server silently dies because it tried to reload + +00:35:59 while you're in the middle of a code edit, unmatched brace, a half-written import or something. + +00:36:04 Now you're hunting through tabs to figure out which process crashed + +00:36:07 and how to restart it. + +00:36:08 My app, CommandBook, gives all of these long-running commands a permanent home. + +00:36:13 You save a command once, the working directory, the environment, + +00:36:17 pre-commands like git pull, and from then on, you just click run. + +00:36:20 You can even group commands together to start and stop everything for a project + +00:36:24 with a single click. + +00:36:25 It also has what I call honey badger mode, auto restart on crash. + +00:36:29 So when your dev server goes down mid-reload, command book just brings it right back up + +00:36:34 and does so over and over until the code is fixed. + +00:36:37 It also detects URLs from your output so you're never scrolling through thousands of lines of logs + +00:36:42 just to figure out how to reopen your web app. + +00:36:44 And it shows you uptime, memory usage, and all sorts of cool things about your process. + +00:36:49 The whole thing is a native macOS app. + +00:36:51 No Electron, no Chromium, just 21 megs. + +00:36:54 And it comes with a full CLI. + +00:36:55 So anything you've configured in the UI, you can fire off from your terminal + +00:36:59 with just a single command. + +00:37:00 Right now it's macOS only, but if there's enough interest, + +00:37:04 I'll build a Windows version too. + +00:37:05 So let me know. + +00:37:07 Please check it out at talkpython.fm slash command book app. + +00:37:11 Download it for free, level up your developer workflow. + +00:37:14 The link is in your podcast player show notes. + +00:37:16 That's talkpython.fm/command book. + +00:37:19 I really hope you enjoy this new app that I built. + +00:37:22 You want to work in the native language of the people who did that part of the folklore + +00:37:26 or whatever, right? + +00:37:27 Yeah, well, and people are still speaking those languages. + +00:37:30 So people who would use this to, you know, like somebody may have heard a story from + +00:37:34 their mom or dad and are now would like to find other versions of that story. + +00:37:38 And they live in a part of Scotland where they speak Scottish Gaelic as their first language. + +00:37:42 They can still access the site. + +00:37:43 And then that mapping color history one, that's another one of the public ones that you said is pretty major. + +00:37:49 Yeah, that's right. + +00:37:50 Yeah. + +00:37:50 So, yeah, that's a pigments database. + +00:37:53 You can search by either English color names like blue and find all of these Asian paintings that have blue or a particular kind of pigment of how they made the blue. + +00:38:04 Yeah, nice. + +00:38:05 So what's the open source story? + +00:38:08 You're creating all these apps, maybe some of these frameworks. + +00:38:11 There's got to be some tools. + +00:38:12 Is there a big desire or already an effort to have a lot of these things open source or is it too niche or is it just like this is the advantage of Harvard has is other universities don't get this? + +00:38:27 No, it's something we talk about quite a bit. + +00:38:30 Usually these things start, usually they start closed source during development. + +00:38:35 And then we work with the faculty and we talk about how we can take, you know, like the repo for the web app, how we can take that public. + +00:38:45 And so we've done that for a number of projects. + +00:38:48 Not all of them are. + +00:38:50 But the ideal is that they all make their way into the open, and especially when they become archived. + +00:38:56 Sure. + +00:38:56 Yeah, that's a good way to help them live on. + +00:38:58 And they might even go into GitHub's Arctic Vault, which is crazy. + +00:39:03 I don't know if people know about that out there, but GitHub has, quite a while ago, started taking copies of all of the repos and backing them up and storing them in the Arctic vault. + +00:39:14 It's kind of cool. + +00:39:15 I really, really, really hope we never need that, but it's kind of neat. + +00:39:18 Yeah, me too. + +00:39:20 Usually universities have their own archival system, so any important research data is usually part of that system as well. + +00:39:30 I see. + +00:39:30 Okay. + +00:39:31 Yeah. + +00:39:32 Obviously, right? + +00:39:32 Like I'm just, I can't remember where it was. + +00:39:34 It was somewhere, I think it was South Korea or Taiwan where like seven years of government + +00:39:40 data got lost or something like that. + +00:39:41 It was really, really bad recently. + +00:39:43 There was a fire and I think they had backups, but maybe just into the building, you know, + +00:39:47 like we'll put that out. + +00:39:48 We'll back it up to the hard drive over here. + +00:39:50 Not good. + +00:39:51 No, not good. + +00:39:52 You definitely want this stuff to survive. + +00:39:54 I mean, academia has this history of like tomes that have survived the past and really, + +00:40:00 really long lived information. + +00:40:02 Right. + +00:40:02 besides the Library of Alexandria or something like that, maybe. + +00:40:05 That's what we want. + +00:40:06 That's what we want. + +00:40:07 We want it to, yeah, we want it to last. + +00:40:09 Absolutely. + +00:40:10 So maybe that's a good time to sort of talk about the trailing end. + +00:40:14 I think there's a lot of interesting things going on here. + +00:40:18 Just like you've run out of money, not because you actually run out of money. + +00:40:23 The grant is done and you've either spent or given back or whatever + +00:40:26 with the remaining little bits of money. + +00:40:28 It's always a weird balance with research. + +00:40:30 It's like, oh, we got $3,000 left on this research grant. + +00:40:33 What are we going to do with it? + +00:40:34 It's not like, oh, we're going to give it back. + +00:40:35 We just didn't need it. + +00:40:36 It's like, we're going to find a way to like fund a student to do a little more work or + +00:40:41 whatever. + +00:40:41 But eventually the grant is over. + +00:40:43 That's right. + +00:40:44 You've got some expensive app access to a big database because it needs a big search or + +00:40:49 a lot of compute or something. + +00:40:50 That's right. + +00:40:52 Everything during, like, I mean, anything, anything that's a, that's a Django app. + +00:40:56 We deploy to AWS using containers, which isn't the cheapest way to host anything. + +00:41:05 But that's for the most part the Harvard way. + +00:41:10 And it is robust and is reliable. + +00:41:12 And we don't have a DevOps person on call on the weekend to rescue one of these apps. + +00:41:22 So having them reliable is good. + +00:41:25 Okay, so it's on AWS and paying for the containers, paying for that Elasticsearch cluster, + +00:41:33 the RDS Postgres database. + +00:41:36 Okay, well, even if somebody wants to start paying for that out-of-pocket, + +00:41:40 all of those little services, they add up to enough that we need to do something + +00:41:44 when the project hits end of life. + +00:41:46 And so our gold standard that we've developed so far is asking, can this become a static website? + +00:41:55 Can we bake this out into all HTML files and acknowledge that there will be some trade-offs? + +00:42:01 We will trade off some searching. + +00:42:04 You know, it's not gonna have Elasticsearch. + +00:42:06 Doesn't mean that it won't have any search though. + +00:42:08 So we'll trade out Elasticsearch and it'll be very difficult to add new data, + +00:42:13 but that's okay because it's being archived. + +00:42:15 So can we get it into a static site? + +00:42:18 And that's challenging depending on how you've set it up. + +00:42:20 So we now have projects where we set them up from the beginning to be archivable like this. + +00:42:26 And one of them is called Water Stories. + +00:42:29 And it was a companion to an art installation at the Radcliffe Institute on the Harvard campus. + +00:42:36 And so this was this live site during the duration of the art installation where people could come in and add stories that they had about water onto an iPad. + +00:42:46 And then those went up to our database. + +00:42:49 we built that with something called Django bakery which if you opt in and you use all of their + +00:42:54 class-based views the way that they're meant to be used then you can bake this out into static files + +00:43:00 when you're done very low effort that was perfect that is such a cool idea and mad props to them for + +00:43:05 ASCII art logos come on now I feel like that should be in the view source if it's not but + +00:43:11 this is such a cool idea because you can you can just take a working site you guys are a Django + +00:43:17 shop. So you have a lot of your sites are written in Django and you just go make it static, right? + +00:43:22 Essentially. Yes. And, and what's, what's, what's really great about it is if they wanted to make + +00:43:27 a change and they have, they have asked since we, since we made it static, they've asked for a + +00:43:31 couple of changes. So locally, I just Docker compose up this whole application, make the change + +00:43:37 in the Django admin and rebake the site. And so it's, it can still be updated. Something, + +00:43:42 if you've never tried this, like something like, Hey, can we just add one more menu item? + +00:43:47 And you're like, no, no, no, we're not adding the menu item because you want that. + +00:43:50 That means we're changing 7,300 pages because they all bake in the whole HTML. + +00:43:56 Right? + +00:43:56 Exactly. + +00:43:57 Yeah, exactly. + +00:43:58 But if that's in my, in my Django database and my SQLite file, then no problem at + +00:44:02 all because then I just rebake it. + +00:44:04 Yeah, yeah, exactly. + +00:44:05 Absolutely. + +00:44:06 So I think this is super neat. + +00:44:09 There's also frozen, frozen flask. + +00:44:13 If I could get rid of all the ads, I do not need a Yeti thing, whatever that is. + +00:44:17 the glass, not the mythical thing, but frozen flask, which does a similar thing for flask + +00:44:25 apps. If you're a flask person probably would work with court. Don't know for sure, but probably. + +00:44:30 So that's a pretty interesting idea as well. throw that in there. but also what else? + +00:44:37 Also you talked about search, right? That can be, can be such a problem. And I'm a huge fan of your + +00:44:45 recommendation here with a page find. Tell us about page find. So this has been, I think it's been a + +00:44:50 bit of a game changer in how functional one of these archived sites can remain. So we're actually + +00:44:56 in the process of that amendments website that searches across 22,000 full texts of amendments. + +00:45:04 We are in the process of sunsetting that, and that will become a static site. And for that search, + +00:45:09 we already have an internal demo that proves that we can replace that Postgres full search + +00:45:16 with PageFind. You lose vector search. Yeah. You've kind of got to get really + +00:45:22 true keyword matching. Yeah. Yeah, that's right. But you still get filtering. I mean, + +00:45:27 and really faceting and filtering is when it comes to discovery of things, I mean, I find + +00:45:34 that's really what's useful. So filtering these amendments by state or by the Congress that was + +00:45:40 active at the time or by the person who co-wrote it. All of those are totally great in PageFind. + +00:45:50 And the keyword search is just fine in PageFind. One of the things I really like about it is that + +00:45:55 it takes your index and it chops it up into lots of little files that can just fly across the + +00:46:00 network. So it's a very fast search. It's not a huge network load, even if your index is + +00:46:07 initially very large. And it essentially cuts it up somewhat alphabetically. So if your search + +00:46:14 starts with T, or I should say a better word for audio, if it starts with W, then it will load up + +00:46:20 the index for words that start with W and fly that over the network instead of the whole thing. + +00:46:26 So it's pretty slick and it has a great Python API. + +00:46:29 So to do the proof of concept for the amendments search, I just took a database dump and then manually indexed with a Python script into PageFind. + +00:46:40 Wait, there's a Python API for PageFind? + +00:46:43 Yeah. So the way PageFind works, I should have said that, is the way most people will use it + +00:46:48 is by normally PageFind consumes HTML. So you give it access to your dist folder. + +00:46:56 Oh, okay. + +00:46:57 And then it crawls through all of your HTML files. + +00:47:00 And you can do great things like adding little HTML tags that are just for PageFind, + +00:47:05 that give it the filtering ability, or that you want to sort by something. + +00:47:09 And so that's great. + +00:47:11 Or you can just call PageFind from Python or from TypeScript and just build that index manually. + +00:47:18 Well, thanks a lot, David. + +00:47:19 I have another thing I've got to go research. + +00:47:21 This is awesome. + +00:47:22 I'm a huge fan of PageFind, as I said. + +00:47:24 on my personal website, mkennedy.codes, is just a pure stat. + +00:47:29 It starts in Markdown and ends up in HTML. + +00:47:31 But if you add page find in, you get a super rich, if you want to just know, you want to talk about, + +00:47:36 like what was about Docker, it shows you really nice results, + +00:47:40 pulling out the different parts of the page and sections that talk about it, + +00:47:43 like the headers and then what is said. + +00:47:45 And it even does like sub, sub word, you know, like you just type doc, + +00:47:50 it finds all the words that match that. + +00:47:52 And what I really like about it is a couple of things + +00:47:54 it's instant. It basically is like nearly instant. If you type a few things, it gets way faster + +00:47:59 because it's pulling down. And if you go and look in the network console here and you type + +00:48:05 something, you can see that it's actually pulling in these little tiny fragments, which this one's + +00:48:10 coming off disk cache in three milliseconds, right? But it breaks your index into a bunch of very small + +00:48:16 page find fragments that I think it's like, it starts with anything that starts with the word + +00:48:21 DO. These are all the prebuilt results and stuff like that. Right. + +00:48:25 That's right. That's right. + +00:48:26 Yeah. That's super cool. + +00:48:27 Yeah. One of our open source projects that, that we maintain is a view of a + +00:48:34 view JS component library for page find so that we can style it and reuse it + +00:48:39 across different projects. + +00:48:41 Oh, that's awesome. I love it. + +00:48:42 Yeah. I think this really unlocks it. + +00:48:44 And I mean, you go to so many, so many sites, like their documentation or just + +00:48:48 their web app in the search is so bad. + +00:48:51 You type something and it's like thinking, spinning, spinning, spinning, spinning. + +00:48:57 And then like five seconds later, it gives you kind of janky results. + +00:49:00 And if you just like throw a page find in there, it's, you can't type fast enough to + +00:49:05 outrun the results. + +00:49:05 You know what I mean? + +00:49:06 No, that's right. + +00:49:07 Yeah. + +00:49:07 Too many static site search solutions, they use like a, like a JSON blob that you, that + +00:49:12 you have to pull down and, and then iterate through. + +00:49:15 You know, what's worse. + +00:49:16 and I see this a lot, would be if you go to google.com + +00:49:21 and then you would say effectively site colon whatever + +00:49:24 and then you search Docker, right? + +00:49:26 They basically pull that. + +00:49:29 You know, they just say search this and you just get Google results for your site. + +00:49:33 And obviously it's, I mean, Google's fine, but it's just. + +00:49:36 No, I find that unusable, really. + +00:49:38 I do too. + +00:49:38 It really, you're like, ah, geez. + +00:49:41 But now I'm super excited to realize I can do that from my dynamic content as well. + +00:49:46 So with the Python integration. + +00:49:48 OK, nice. + +00:49:51 What about something truly static? + +00:49:53 Have you looked at Hugo and some of the other type of things? + +00:49:56 Sure. + +00:49:57 So when I see you've even got the tab up for the SUMEB project, + +00:50:02 which is-- that's essentially a database of many, many specimens + +00:50:09 taken from the SUMEB mine. + +00:50:11 So in the-- + +00:50:12 Oh, it is. + +00:50:13 Yeah, yeah, it is. + +00:50:13 So if you click on Minerals database, you open up that search interface and that's powered by PageFind. + +00:50:19 Oh, this is? + +00:50:21 Yes. + +00:50:22 I forget what I was... + +00:50:23 I see. + +00:50:24 You guys even hooked into... + +00:50:26 I was thinking just like pure static, like Hugo, like... + +00:50:30 Oh, yes. Yes. Yes. + +00:50:31 So this is an Astro site. + +00:50:33 So for this website, we have this as an Astro site so that we have a little... + +00:50:37 Because with Astro, they make it so easy to pull in like view components. + +00:50:42 So like our page find is a custom view JS component library with Astro. + +00:50:47 You can use React components, you can use the view components, but what it does is it's just + +00:50:52 a static site generator. Fantastic. So a little bit more designable + +00:50:57 than like Hugo or something. Here's your markdown file. Good luck with that. + +00:51:00 Yeah. I love Hugo though. Yeah. I use Hugo for different personal sites here and there, + +00:51:05 and it's just so fast and easy to get up and running. But yeah, it's great. + +00:51:08 - Great, great when it's a good friend. + +00:51:09 - That's what my website's written in, it's in Hugo. + +00:51:12 But if I'm integrating with anything else, I used to kind of like split it up, + +00:51:15 like this part's Hugo and this part's like a Python app. + +00:51:17 And it's pretty easy to get something that'll take a bunch of markdown files + +00:51:21 and just turn them into HTML and just put a page template around that. + +00:51:25 So I've kind of stepped away from mixing and matching that + +00:51:29 as much as I used to. + +00:51:30 So now if I got a static section of a dynamic site, but that doesn't address, + +00:51:34 has nothing to do with the archival side of things, right? + +00:51:38 Because the idea is that the thing that I'm describing is gone on purpose. + +00:51:42 That's right. + +00:51:42 So you've got some, we've got Django Bakery. + +00:51:46 I threw out Frozen Flask, and I'm sure there's a ton more that neither of us are aware of at the moment. + +00:51:52 So Django Bakery was really good for that purpose. + +00:51:56 And we're keeping our eyes open for projects that it's a good fit for. + +00:52:01 But that was a pretty simple website. + +00:52:03 It needed a dynamic backend, but it was quite straightforward. + +00:52:06 And for Django Bakery, you have to opt into inheriting from their class-based views. + +00:52:11 I see. + +00:52:12 So if you're doing, for example-- + +00:52:13 You've got to dig ahead of it, yeah. + +00:52:15 Yeah, yeah, yeah, absolutely. + +00:52:17 Yeah, hard to add retroactively. + +00:52:18 Probably impossible. + +00:52:20 Now, our other websites, like the fin example and the mapping color example, those are APIs. + +00:52:27 That's a Django API, Django REST framework for one, GraphQL for the other. + +00:52:32 One has a view front end, one has a React front end. + +00:52:34 OK, well, Django Bakery just isn't isn't going to work very well for like serializing JSON. + +00:52:39 Yeah, it's like awesome. + +00:52:40 Here's your unrendered JavaScript front end code and it's just going to look empty or something. + +00:52:45 Yeah. + +00:52:46 So it is a good reason to consider using like vanilla Django templates when possible, + +00:52:52 like for that reason. + +00:52:53 But those were, those were inherited from the vendors, those two sites. + +00:52:59 And we've made a lot of progress on those. + +00:53:01 So, you know, what, what to do in that, like in that situation, Django Bakery isn't an option. And those projects are not end of life + +00:53:10 yet. So we have some time, but we're, we're, we're, so what we're doing is strategizing, okay, + +00:53:15 how will we rescue them? How will we keep them alive once, once somebody needs to stop paying + +00:53:20 for hosting? And we have, we have ideas. We have, I think there's, there's clever, interesting + +00:53:26 things out there. We'll have to keep looking into it. There are some pretty interesting ideas. And + +00:53:34 that ran in a container, you could just have WebAssembly, but still have it go, right? + +00:53:41 Sort of a local loopback type of thing. + +00:53:43 Yeah, I'm really interested in this one because it enables essentially the full functionality + +00:53:51 of the live site to exist as what is just a static site. + +00:53:55 So because of Pyodide and projects like PyScript, we can run Python in the browser and we can + +00:54:03 run SQLite in the browser. And now we can even run Postgres in the browser with PG Lite. So if + +00:54:09 we can run all those things in the browser, then couldn't we have Django hosted right in the browser? + +00:54:15 And you can. So there's a proof of concept that proves it's possible called Django WebAssembly. + +00:54:23 And if you load this up, it'll let you log in to the Django admin. And you're not logging into + +00:54:29 anybody's backend, you're logging into your own browser where this is running in a service worker. + +00:54:36 Awesome. Look at that. Oh, hold on. I told me what the password was. Very secure. + +00:54:40 Matt, password. + +00:54:42 Well, it can be entirely insecure because, yeah, you're just, it's running right in your own browser. + +00:54:47 Yeah, that's awesome. And here we are, Django admin. Incredible. + +00:54:50 Yeah, so I'm pretty interested in this. You've got to convert an RDS Postgres database + +00:54:55 into either SQLite or something like PGLite, but I think that's all doable. + +00:54:59 So I think it's an exciting possibility. + +00:55:02 Yeah, for sure. + +00:55:03 I do think, so maybe you have a rich query system that you're powering by your database + +00:55:08 that's really heavy. + +00:55:09 Exactly. + +00:55:10 And it's got a bunch of data that's like, here's all of our working data + +00:55:13 that you might ask questions about. + +00:55:15 Maybe you just convert that to page find to help you find the pieces + +00:55:18 and then just keep the operational data and maybe like even a SQLite with like the Django RRM, + +00:55:23 you can just switch the connection, keep talking to it. + +00:55:25 I mean, there's possibilities to just get something not too terrible + +00:55:28 Well, it's not the same, but not that far off. + +00:55:31 Yeah, exactly. + +00:55:32 And then it goes on GitHub pages and it can live hopefully forever. + +00:55:35 I mean, it feels like GitHub will last forever, but it'll last longer than funding will anyways. + +00:55:41 It's definitely going to last longer than just something that we can't pay for anymore, right? + +00:55:48 I don't know how long GitHub's going to be around for, I think a while, but you never know, right? + +00:55:53 It seems like stuff's going to last forever, then it gets changed. + +00:55:57 We had subversion. + +00:55:59 Now it's completely gone, right? + +00:56:00 Just 20 years, 15 years later, but still, I think 100% there. + +00:56:05 Yeah. + +00:56:05 But if somebody can, if something ever happened, somebody just needs to copy that, + +00:56:09 that folder of HTML, CSS and JavaScript files and dump it into an S3 bucket or somewhere else. + +00:56:15 And then it can continue living there. + +00:56:17 So it's a good option. + +00:56:19 It's a great option. + +00:56:20 It's a really, really good option. + +00:56:21 I mean, I guess one of the long-term concerns might be what if the WebAssembly standard changes so much that it's not supported anymore? + +00:56:31 But you could probably bite-wise convert it if you had to, you know, like somebody would probably be able to create one. + +00:56:37 Yeah, that would be unfortunate. + +00:56:39 So I suppose if that happens, I mean, if that happens, yeah, we're booting up one of these projects is like booting up an emulator for some old DOS game. + +00:56:49 Right, right. + +00:56:49 Well, I mean, I guess let's think about this for a second. + +00:56:52 Somebody got, oh gosh, what was the chain? + +00:56:55 This is the whole, JavaScript, the PyCon talk where got like Firefox + +00:57:04 compiled into, not WASM, into, ASM JS or something like that. + +00:57:10 So it was run like Chrome was running Firefox, which was running, I think + +00:57:14 doom, which was also ASM JS. + +00:57:17 If we can do that, we could get something that would run, that would read old Web + +00:57:22 Assembly into new WebAssembly if it really mattered to the world. + +00:57:24 Absolutely. + +00:57:25 Yeah. + +00:57:26 Especially if it's in a public repo that people who care about the data can, + +00:57:30 can rescue it somehow. + +00:57:31 Yeah. + +00:57:32 What about like a virtual machine? + +00:57:34 You know, I agree. + +00:57:35 Yeah, absolutely. + +00:57:36 Could have saved me some, take a snapshot of Ubuntu LTS, some version, + +00:57:42 and just what are we going to do? + +00:57:44 Everything we do is Dockerized. + +00:57:46 Everything is in a container. + +00:57:47 So in the worst case scenario, we could give somebody the image, and they could run it if + +00:57:51 they have Docker. + +00:57:53 I think that's a nice peace of mind to know that no matter what, something will be able + +00:57:57 to run this container. + +00:57:59 And even in, I don't know if you've used GitHub, what is it called, Codespaces. + +00:58:05 I archived one project. + +00:58:07 It was kind of dramatic and sudden that it needed to be archived, so without much time + +00:58:12 to do anything. + +00:58:13 And it was a Ruby on Rails project. + +00:58:15 And I'm not a Rails developer, but I was able to get it archived in a way + +00:58:19 that anybody could, with one command, go to the repo on GitHub and boot it up in Codespaces + +00:58:27 and then have it live running from their Codespace. + +00:58:30 And so that works too. + +00:58:32 Very cool. + +00:58:32 I think as WebAssembly grows, there'll be more possibilities for these types of things. + +00:58:38 Yeah, amazing. + +00:58:39 I'm pretty excited about PageFind having a Python API. + +00:58:42 didn't realize that. So I'm going to be doing something with that for sure. So what else? + +00:58:46 Let me ask you one more thing before I kind of let you wrap up with some final thoughts here. + +00:58:51 What about AI? Oh, that's a good question. So AI, I mean, there's like, in my story, + +00:58:58 there's like one interesting part of AI, which is that I got started and self-learned everything I + +00:59:04 needed to about software development to begin doing this right before ChatGPT really came on + +00:59:10 was able to do real programming yeah you're like four years of legit programming before right so i + +00:59:17 think i mean so i was thinking i was thinking when i was thinking about how i got into it i thought + +00:59:21 what if i was four years later starting my phd and wanting to do these tools um i would have been + +00:59:28 able to accomplish what i needed to for my research without acquiring the technical skills and that + +00:59:34 would have been that's a good thing i'm not sure if that's good about it it could be both i would + +00:59:37 would have thought it was a good thing. I would have thought it's a good thing. But in my hands + +00:59:43 now, like a software engineer, AI is more powerful in my hands now than it would have been then. + +00:59:52 So I can make it work for me. Yeah, I can make it work for me in a way that I couldn't have been + +00:59:57 able to then. So I'm thankful for that, but it's something I think of. I don't want to say it's + +01:00:02 necessarily a bad thing, but it definitely marks a difference, a difference in time between other + +01:00:07 people who are maybe wanting to get into digital humanities, they're humanities researchers. They + +01:00:13 want to add some digital tools. You know, I think this will kind of, this will probably knock people + +01:00:18 off of the more technical path because it's not needed. I think it will too. And I think that that + +01:00:22 might be a negative. When you were telling me your story originally, I was thinking kind of like, + +01:00:27 how neat is it that you didn't sign up for, and the people you're working with probably didn't + +01:00:32 intend to sign you up for learning true software development. + +01:00:36 But look at this cool and interesting job that you now have that you never + +01:00:41 would have imagined. + +01:00:42 I'm sure when you signed up for your PhD, you're like, you know what I'm + +01:00:44 going to do when I get my PhD, I'm going to go X, Y, like, I'm going to + +01:00:47 join the Darth program. + +01:00:48 Like, no, probably not. + +01:00:49 Right. + +01:00:50 But here you are. + +01:00:51 And I think that's actually a really interesting knock on effect for a lot + +01:00:54 of researchers and people in grad schools, they're kind of put into this + +01:00:59 programming adjacent type of thing. + +01:01:01 You know, and a lot of folks sort of are like, actually, that's pretty interesting. + +01:01:04 I'm going to kind of lean into that. + +01:01:06 And I think AI might knock, like you said, knock people off that path to some degree. + +01:01:11 Yeah, yeah, definitely. + +01:01:12 So that's just like one part of the AI story. + +01:01:15 The other one is that, like how we use it. + +01:01:18 It's great for data extraction, pulling data out of different, you know, to make these + +01:01:25 search interfaces more powerful, to extract different data from them. + +01:01:30 That's just one example where it's been handy. + +01:01:33 We're looking for ways that it can really empower faculty. + +01:01:39 We're still very much in the exploration phase of how we can use it and provide it to faculty as a digital humanities tool. + +01:01:48 Sure. I was thinking pretty much when I asked the question of it, it's just like two parts. + +01:01:52 One, how is it? Are you guys using it to help take projects? + +01:01:56 Well, that would have been a month. No, actually, it's three days. + +01:01:58 You know what I mean? + +01:02:00 that. And then if people are asking, you know, a professor comes along and says, and we want our + +01:02:05 own custom AI thing, or we're using Harvard's internal one that we're allowed to use, but we + +01:02:13 won't be able to use it once the grant runs out. You know what I mean? Yeah. Yeah. I think one, + +01:02:17 one good example of this type of thing is that what we're starting to get is faculty who are + +01:02:23 vibe coding and now, and we are going to teach them. We're going to teach them how to do it. + +01:02:28 You know, instead of having them. + +01:02:31 Yeah, it's absolutely a skill. + +01:02:32 Yeah, no, it is. + +01:02:33 It is. + +01:02:34 Instead of copy and pasting from ChatGPT into VS Code, having them learn Copilot, maybe even having them download Cursor. + +01:02:43 Download some real dedicated tools to get this done to make them more productive. + +01:02:48 So, yeah, educating about how to do it is one thing. + +01:02:53 You asked if we're using it. + +01:02:54 We have access to Copilot. + +01:02:58 and that's great. I can't say that we've shipped anything in three days instead of a month yet, + +01:03:04 but one anecdote is that right now I'm doing some really interesting processing of music audio files, + +01:03:13 and somebody asked to have a beatboxer if I could chop that file up so that all of the individual + +01:03:19 sounds that the beatboxer makes are identified in a file. And so I'm using some music libraries, + +01:03:26 Python library called Librosa. There's some complicated math in there. It's a little bit + +01:03:32 too much for me. It's no problem for Claude. Claude knows how to do that math. And then, + +01:03:36 and I use my expertise to string it together to get a good output. + +01:03:39 Yeah. Awesome. You got time for one more quick question before we'll clap things up. + +01:03:44 For sure. + +01:03:45 Raymond out there, Raymond Yees asks, it says, it'd be good to hear how Harvard uses containers on AWS + +01:03:51 and its reliability. It's reliable, not cheapest way to host things. Are you thinking about moving + +01:03:56 moving that or is it not that much? Okay, I'll tell you about a failed experiment. + +01:04:03 We were using ECS and we're still using ECS. So that's AWS's main, you know, it's not Kubernetes, + +01:04:11 but it's one step down with their horizontal scaling container clusters. And I wanted to move + +01:04:17 us onto a single EC2 instance because our projects are popular, but they're not so popular that we + +01:04:23 actually have to worry about horizontal scaling. + +01:04:25 Right. + +01:04:26 It's not like it's front page in New York Times. + +01:04:30 I guess it probably could be. + +01:04:31 But even so, for the static sites, they probably still can take it. + +01:04:35 Yeah. + +01:04:35 So I priced it out and I got an example deployed, an example project deployed, and was able + +01:04:42 to confirm that it would indeed be much cheaper. + +01:04:45 And it was deployed in a similar way using AWS CDK. + +01:04:49 So it's all infrastructure is code all the way down. + +01:04:52 But it turns out there's all kinds of compliance. + +01:04:54 When you are in charge of the VM at like a big university, + +01:04:58 or I'm sure any corporate setting, if you are in charge of the VM and the OS on it, + +01:05:04 then you have to know that you have the latest patches in. + +01:05:07 You have to know that you have latest Ubuntu. + +01:05:09 And then there's other things, different observability things + +01:05:13 that you have to have in place that are not usually required + +01:05:17 if you're running in a container cluster like ECS. + +01:05:21 So it ends up being a lot less work and much easier to achieve compliance if we run containers + +01:05:28 or some other serverless thing. + +01:05:31 If I run all my personal projects, they all run in a single virtual machine, but we're + +01:05:37 running in containers. + +01:05:38 Yeah. + +01:05:38 Yeah. + +01:05:39 And you've got all the SOC 2 stuff and all those different things, right? + +01:05:42 Like there's layers. + +01:05:43 Yeah, that's right. + +01:05:44 Yeah. + +01:05:44 I mean, I'll mention that, but what I didn't say is that in that 2019, when I started learning + +01:05:50 Python. I discovered Talk Python almost immediately. And one of the first episodes that I listened to + +01:05:55 was the other digital humanities. Cornelius Van Litt. He was an awesome guest. + +01:06:01 That's right. Yeah. And I thought that was great. And that was also a bit about manuscripts, + +01:06:06 a little bit more on the image side than the text side. And I didn't understand everything + +01:06:11 that everybody was saying, but I just, I kept tuning in. And I think because of that, + +01:06:16 Because Talk Python was like this, you know, I've been remote working for most of my time. + +01:06:22 And Talk Python has been kind of like that conversation with the open source community + +01:06:27 that's been always in my ear. + +01:06:28 And I think that made, you know, a difference, making me feel like I understood the software + +01:06:34 landscape and like the developer culture and what was going on. + +01:06:37 And then the different Python libraries and what was possible. + +01:06:41 So to people who are interested in taking things in a more technical direction, I think + +01:06:47 it's helpful just to find a few things like that, that give you an insight into that world. + +01:06:53 And the more you listen to it, the more you start to hear the same acronyms and the same + +01:06:59 things said enough that you start to feel like, okay, now you're part of the club. + +01:07:03 I really appreciate that. + +01:07:05 That's cool. + +01:07:06 I've certainly had people reach out to me and say things that at first didn't make any + +01:07:09 sense to me. + +01:07:10 Like I've been listening for six weeks now and it's starting to make sense what you're talking about. + +01:07:14 Like, why have you been listening for six months when it made no sense? + +01:07:16 That's insane. + +01:07:17 But a lot of people use listening to the podcast, is it mine and others, as language immersion, right? + +01:07:24 Like I could get Duolingo and I could learn Portuguese + +01:07:28 or I could move to Brazil for a month. + +01:07:30 You know what I mean? + +01:07:31 And then I would really learn. + +01:07:32 - Yeah, exactly. + +01:07:33 - Right. + +01:07:34 - Exactly. + +01:07:34 No, I think there's truth to that. + +01:07:36 And some of the things I did was, you know, search through, like search the word deployment, because I'm trying to get my head around how to + +01:07:43 deploy for the first time. And I just want to hear people talk about it. Like I could read about it. + +01:07:47 I could read the tutorial, but I just want to hear people talk about deployment to get a sense of what + +01:07:52 actual deployment sounds like. There's something really different when you're learning or trying, + +01:07:57 even you're maybe an experienced programmer, but not in this particular area to hear a human + +01:08:01 side of it, not just the docs, not a sterile. These are the four steps, but like, I love it. + +01:08:08 I mean, it's probably why I created the show. + +01:08:10 It's because I didn't hear those stories. + +01:08:11 We got to tell those stories. + +01:08:13 Awesome. + +01:08:13 I appreciate that. + +01:08:14 So super cool. + +01:08:15 All right. + +01:08:16 So if other people are listening, maybe one of your pieces of advice is keep listening. + +01:08:21 You'll get there. + +01:08:22 Yeah. + +01:08:22 And if anybody is in the humanities and somehow found their way onto this episode with no technical experience, + +01:08:30 I just would give the caution of, like, you know, the anecdote that if AI coding had been + +01:08:37 around the way it is now when I was learning, I wouldn't be doing digital humanities at + +01:08:43 Harvard. + +01:08:43 I wouldn't have been able to get into this field. + +01:08:46 I wouldn't have known about it. + +01:08:47 So I guess just think about that when you're learning and applying new tools. + +01:08:52 I don't really know what the right fix for that is. + +01:08:55 That's a very challenging problem. + +01:08:56 I mean, you can say I'm just literally not going to fire it up. + +01:08:59 But I mean, we used to hunt through Stack Overflow and the web and over and over. + +01:09:03 And if you're really stuck or you really don't understand, like they're good at explaining + +01:09:06 stuff too. + +01:09:07 You just got to really stay in a learner's mindset, not just press the easy button and + +01:09:12 make this thing and move on. + +01:09:13 Easier said than done. + +01:09:14 Easier said than done. + +01:09:15 So yeah, I want to leave this with kind of a thought about how much things like Python + +01:09:22 and these tools and technology can really empower stuff that you wouldn't think is even + +01:09:27 related, like understanding old manuscripts and how painting is connected or changed over time and + +01:09:34 stuff, right? Those sound very much disjointed from tech and software, but they really are + +01:09:40 superpowers that you can bring to your work, whatever your industry is. I know our field of + +01:09:45 study, I know there's some sociologists out in the audience and I'm sure others as well. + +01:09:50 All right. Final thoughts, David, close it out. You said it great. I mean, you know, + +01:09:55 Just applying these technical tools to old questions, that is the core of digital humanities. + +01:10:02 When I first started hearing about this, I thought, I really don't know how this ties + +01:10:05 together. + +01:10:05 And after seeing it a few times, I definitely see the power of it. + +01:10:08 And I thank you for your time coming on. + +01:10:11 Thank you for sharing your look and the look inside of your team and inside of a small piece + +01:10:16 of Harvard. + +01:10:17 I really like these kinds of episodes because it's hard to see this from the outside, right? + +01:10:23 like you just see the results, but you don't see like the inner workings of the team + +01:10:27 and the motivation and stuff. + +01:10:28 So thank you so much for being here. + +01:10:31 And yeah, bye everyone. + +01:10:33 This has been another episode of Talk Python To Me. + +01:10:36 Thank you to our sponsors. + +01:10:37 Be sure to check out what they're offering. + +01:10:38 It really helps support the show. + +01:10:40 Take some stress out of your life. + +01:10:42 Get notified immediately about errors and performance issues in your web + +01:10:46 or mobile applications with Sentry. + +01:10:48 Just visit talkpython.fm/sentry and get started for free. + +01:10:53 Be sure to use our code, talkpython26. + +01:10:56 That's Talk Python, the numbers two, six, all one word. + +01:11:00 This episode is brought to you by CommandBook, a native macOS app that I built + +01:11:05 that gives long-running terminal commands a permanent home. + +01:11:08 No more juggling six terminal tabs every morning. + +01:11:10 Carefully craft a command once, run it forever with auto-restart, + +01:11:14 URL detection, and a full CLI. + +01:11:16 Download it for free at talkpython.fm/commandbook app. + +01:11:19 If you or your team needs to learn Python, We have over 270 hours of beginner and advanced courses on topics ranging from complete beginners to async code, Flask, Django, HTML, and even LLMs. + +01:11:32 Best of all, there's no subscription in sight. + +01:11:35 Browse the catalog at talkpython.fm. + +01:11:37 And if you're not already subscribed to the show on your favorite podcast player, what are you waiting for? + +01:11:42 Just search for Python in your podcast player. + +01:11:44 We should be right at the top. + +01:11:46 If you enjoy that geeky rap song, you can download the full track. + +01:11:49 The link is actually in your podcast blur show notes. + +01:11:51 This is your host, Michael Kennedy. + +01:11:53 Thank you so much for listening. + +01:11:54 I really appreciate it. + +01:11:56 I'll see you next time. + +01:12:08 I'm out. + diff --git a/transcripts/538-python-in-digital-humanities.vtt b/transcripts/538-python-in-digital-humanities.vtt new file mode 100644 index 0000000..a3be1c2 --- /dev/null +++ b/transcripts/538-python-in-digital-humanities.vtt @@ -0,0 +1,3713 @@ +WEBVTT + +00:00:00.020 --> 00:00:05.200 +Digital humanities sounds niche until you realize that it can mean a searchable archive of U.S. + +00:00:05.420 --> 00:00:11.580 +amendment proposals, Irish folklore, or pigment science in ancient art. Today I'm talking with + +00:00:11.720 --> 00:00:17.780 +David Flood from Harvard's DARTH team about an unglamorous problem. What happens when the grant + +00:00:18.060 --> 00:00:24.180 +ends? But the website can't. His answer? Static sites, client-side search, and sneaky Python. + +00:00:24.600 --> 00:00:30.680 +Let's dive in. This is Talk Python To Me, episode 538, recorded January 22nd, 2026. + +00:00:48.540 --> 00:00:52.880 +Welcome to Talk Python To Me, the number one Python podcast for developers and data scientists. + +00:00:53.180 --> 00:00:54.780 +This is your host, Michael Kennedy. + +00:00:55.130 --> 00:00:58.720 +I'm a PSF fellow who's been coding for over 25 years. + +00:00:59.360 --> 00:01:00.420 +Let's connect on social media. + +00:01:00.800 --> 00:01:03.880 +You'll find me and Talk Python on Mastodon, BlueSky, and X. + +00:01:04.239 --> 00:01:06.060 +The social links are all in your show notes. + +00:01:06.820 --> 00:01:10.340 +You can find over 10 years of past episodes at talkpython.fm. + +00:01:10.520 --> 00:01:13.720 +And if you want to be part of the show, you can join our recording live streams. + +00:01:14.080 --> 00:01:14.540 +That's right. + +00:01:14.730 --> 00:01:18.000 +We live stream the raw uncut version of each episode on YouTube. + +00:01:18.580 --> 00:01:23.020 +Just visit talkpython.fm/youtube to see the schedule of upcoming events. + +00:01:23.180 --> 00:01:26.880 +Be sure to subscribe there and press the bell so you'll get notified anytime we're recording. + +00:01:27.820 --> 00:01:29.480 +This episode is brought to you by Sentry. + +00:01:29.800 --> 00:01:31.040 +Don't let those errors go unnoticed. + +00:01:31.230 --> 00:01:32.840 +Use Sentry like we do here at Talk Python. + +00:01:33.340 --> 00:01:36.200 +Sign up at talkpython.fm/sentry. + +00:01:37.040 --> 00:01:42.320 +And it's brought to you by CommandBook, a native macOS app that I built that gives long-running + +00:01:42.520 --> 00:01:43.980 +terminal commands a permanent home. + +00:01:44.360 --> 00:01:46.360 +No more juggling six terminal tabs every morning. + +00:01:46.820 --> 00:01:51.180 +Carefully craft a command once, run it forever with auto-restart, URL detection, and a full + +00:01:51.300 --> 00:01:51.620 +CLI. + +00:01:51.960 --> 00:01:55.100 +Download it for free at talkpython.fm/command book app. + +00:01:56.040 --> 00:01:59.200 +Hello, David. Welcome to Talk Python To Me. Amazing to have you here. + +00:01:59.760 --> 00:02:03.500 +I'm glad to be here. Talk Python has been part of my story up to this point. + +00:02:03.760 --> 00:02:09.220 +Has it? Okay. Well, you are about to write the next chapter in the story. So that's pretty excellent. + +00:02:10.020 --> 00:02:14.800 +I have a sense of what's coming. We planned out what we're going to talk about and that sort of thing. + +00:02:15.200 --> 00:02:20.420 +And I'm really excited about this topic. So it's going to be a good one. + +00:02:21.060 --> 00:02:34.860 +Honestly, I think one of the real powers of the Python community and the reason the language has such staying power is there's such a diversity of use cases, technology, like technology standpoints, right? + +00:02:34.870 --> 00:02:44.740 +Like I build software for this group or I build these types of apps and it's not just, you know, like Ruby on Rails, which, you know, it's been very popular, but it's, it's for websites, right? + +00:02:44.770 --> 00:02:45.280 +You know what I mean? + +00:02:45.920 --> 00:02:46.780 +Yeah, absolutely. + +00:02:47.280 --> 00:02:57.260 +I mean, web development has dominated my use of it, but my entry into it, which I suppose I'll mention in a moment, was through all those little tools. + +00:02:57.660 --> 00:02:58.320 +Let's hear it. + +00:02:58.930 --> 00:03:00.440 +Who are you, David Flood? + +00:03:00.490 --> 00:03:03.760 +Tell us, introduce yourself real quick and tell us about how you got into it. + +00:03:04.480 --> 00:03:09.140 +So my background is in music and the humanities. + +00:03:09.630 --> 00:03:14.900 +I mean, in 2019, I didn't know what Python was or the name of any programming language. + +00:03:16.180 --> 00:03:22.280 +and I've been doing textual criticism, which is, you know, there's lots of criticisms in the academy. + +00:03:22.860 --> 00:03:26.520 +This is the one where if you have lots and lots of versions of the same text, + +00:03:27.100 --> 00:03:33.280 +you are comparing them to work out what the initial text was and like how it changed over time. + +00:03:33.760 --> 00:03:35.160 +Okay, give us an example. + +00:03:35.700 --> 00:03:39.800 +Okay, so one of the famous examples, hope I can remember it off the top of my head, + +00:03:40.220 --> 00:03:41.600 +is from Shakespeare. + +00:03:42.460 --> 00:03:44.500 +We're all familiar with the line to be or not to be. + +00:03:45.020 --> 00:03:52.520 +is the question. That is the question. Well, there's a variant of it. One of the early copies + +00:03:53.820 --> 00:03:58.900 +written by Shakespeare himself has... Somebody's going to be able to type into the chat exactly + +00:03:58.960 --> 00:04:03.780 +what it is. They'll know this anecdote. But it's something more like, "To be or not to be, I." + +00:04:04.080 --> 00:04:09.940 +That's the question. And so, which one is the original one? Why did he change it? That's kind + +00:04:09.900 --> 00:04:14.900 +of one example i work mainly in the in the new testament which is especially complicated because + +00:04:15.360 --> 00:04:23.300 +no other corpus from ancient history has as many copies of the same text as that corpus does so it's + +00:04:23.340 --> 00:04:29.320 +quite um quite quite complicated and our techniques have have grown grown because of that and perhaps + +00:04:29.560 --> 00:04:37.639 +become more advanced than now i mean that many variations over that huge span of time over + +00:04:37.660 --> 00:04:42.740 +different groups with different, maybe not intentions, but certainly colored by different + +00:04:43.060 --> 00:04:47.620 +worldviews and philosophies and so on. And yeah, I see the trouble. + +00:04:47.920 --> 00:04:53.940 +No, yeah. And they were people of the book. So copying it is something that happened a lot. And + +00:04:54.160 --> 00:05:01.260 +they copied the monks, like the medieval monks copied everything. They copied our Greek classics. + +00:05:01.900 --> 00:05:06.840 +So that's what I was interested in. And because of the wealth of data that we have, + +00:05:07.200 --> 00:05:10.700 +Computer tools are more and more important in that field. + +00:05:11.020 --> 00:05:17.200 +So when I started my PhD in 2019, I knew that I wanted to use some of these cutting-edge tools. + +00:05:17.660 --> 00:05:19.260 +Some of them may be surprising. + +00:05:19.860 --> 00:05:24.100 +For example, we've been using phylogenetic software. + +00:05:24.480 --> 00:05:35.440 +This is software that evolutionary biologists are using or computational biologists are using to track, for example, how COVID strains mutate over time. + +00:05:35.680 --> 00:05:36.420 +Oh, interesting. + +00:05:36.440 --> 00:05:39.800 +What they're comparing are the DNA letters. + +00:05:40.320 --> 00:05:43.740 +And so you have the sequence of letters and you're comparing how those change over time. + +00:05:44.000 --> 00:05:48.160 +Well, you can swap in textual variants for DNA letters. + +00:05:48.600 --> 00:05:55.380 +And now we can track how texts change over time and group them into families, things like that. + +00:05:56.120 --> 00:05:59.580 +It's like a time series, but of words or letters or something. + +00:05:59.720 --> 00:06:06.400 +Yeah, I mean, yeah, there's lots of important algorithms for comparing + +00:06:06.400 --> 00:06:11.720 +sequences of things. And so if we can just swap in Greek words and Greek text instead, + +00:06:12.480 --> 00:06:16.420 +then we can maybe apply it to textual criticism. So I was pretty interested in those things. That + +00:06:16.600 --> 00:06:21.380 +wasn't actually the method that brought me into it, but something like that, kind of computer + +00:06:21.620 --> 00:06:27.760 +intensive tools. What I learned is that these tools weren't actually available to me. They + +00:06:27.940 --> 00:06:36.380 +weren't desktop applications. And for the most part, they weren't public web applications. They + +00:06:36.400 --> 00:06:38.000 +PyPI or something like that, right? + +00:06:38.420 --> 00:06:39.060 +Yeah, exactly. + +00:06:39.360 --> 00:06:39.460 +Exactly. + +00:06:39.540 --> 00:06:40.040 +Or Java. + +00:06:41.180 --> 00:06:43.380 +And I needed to glue them together. + +00:06:43.780 --> 00:06:49.660 +So the long story short on that is during the first year of my PhD, I was picking up Python, + +00:06:50.000 --> 00:06:51.880 +watching YouTube videos while I was doing the dishes. + +00:06:52.800 --> 00:06:57.220 +And then the pandemic hit while I was living in Edinburgh in Scotland, probably not far + +00:06:57.460 --> 00:06:58.160 +from Will McCoogan. + +00:06:59.220 --> 00:07:06.360 +And so the pandemic gave me the excuse to spend even a few more hours each day picking up these + +00:07:06.380 --> 00:07:12.900 +new, these new technical skills. And so I did it, I was able to use these advanced tools in my in my + +00:07:13.100 --> 00:07:17.440 +work. But what was really important to me was sharing, like making that available to my colleagues, + +00:07:18.120 --> 00:07:23.860 +is I had to I had to move from writing these like bad top to bottom Python scripts into things that + +00:07:23.860 --> 00:07:29.500 +could be reused by other people. And that led me into the web, because the web is where that's how + +00:07:29.500 --> 00:07:35.740 +I can share with anybody. It's really wild how much the web is kind of the last bastion of + +00:07:36.640 --> 00:07:42.480 +app freedom. It's so bizarre because, you know, I've many times told the stories of the insane + +00:07:42.900 --> 00:07:48.500 +battles of just getting our apps that just playback video of content that's already on the web + +00:07:48.860 --> 00:07:54.740 +into the app store. I mean, weeks of fighting about the weirdest, most nonsensical things with + +00:07:54.860 --> 00:08:01.819 +both Google and Apple. But we also now have the Mac platform and the Windows platform very + +00:08:01.840 --> 00:08:07.780 +aggressively looking for digital code certificates and all sorts of signing and other kinds of proof + +00:08:07.920 --> 00:08:12.840 +like it you can't even just send somebody an executable anymore it won't run it's it's crazy + +00:08:13.120 --> 00:08:18.940 +it's it's down to like okay put it on the web i guess that's right i i i played the game of + +00:08:19.080 --> 00:08:24.540 +distributing desktop apps that's how i did it that's why i initially distributed things um + +00:08:25.140 --> 00:08:30.860 +and at this point i just require people to install python and then install my desktop app from pypi + +00:08:30.880 --> 00:08:33.400 +because it's too hard otherwise for me. + +00:08:33.820 --> 00:08:36.479 +I mean, I could pay for the code signing from Apple + +00:08:36.890 --> 00:08:37.599 +and do all of that, + +00:08:37.740 --> 00:08:40.320 +but it's just, it's too much work for the time that I have. + +00:08:40.500 --> 00:08:42.140 +Yeah, I'm about to do another round of it. + +00:08:42.200 --> 00:08:42.979 +I'm working on an app + +00:08:44.060 --> 00:08:45.680 +and my developer account is still active. + +00:08:45.880 --> 00:08:47.680 +So we might have a fresh round of fun. + +00:08:47.820 --> 00:08:49.260 +Hopefully it goes through this time. + +00:08:50.320 --> 00:08:52.160 +Anyway, I do think it's such a challenge. + +00:08:52.380 --> 00:08:53.520 +And are you leveraging? + +00:08:53.940 --> 00:08:55.180 +I don't know if the timing was right. + +00:08:55.300 --> 00:08:56.300 +Like maybe this was too early, + +00:08:56.780 --> 00:08:59.740 +but these days, are you leveraging things like uvx + +00:09:00.060 --> 00:09:03.640 +to run, or are you just pip install this thing and then run it? + +00:09:04.100 --> 00:09:08.200 +Yeah, I haven't updated the readme in a while, so I think it just asks for pip. + +00:09:08.740 --> 00:09:14.220 +But certainly, if somebody asked me today, I would say, yeah, just install this with uv. + +00:09:14.920 --> 00:09:16.260 +Because then they don't even need Python. + +00:09:16.700 --> 00:09:17.100 +Exactly. + +00:09:17.420 --> 00:09:17.860 +And that's brilliant. + +00:09:18.440 --> 00:09:22.900 +And that's a really, it is another barrier reduced in distributing these applications, + +00:09:23.160 --> 00:09:23.220 +right? + +00:09:23.300 --> 00:09:28.600 +Like, if you can get uv installed on a machine, then you don't even have to say install, just + +00:09:28.560 --> 00:09:32.960 +The way you run it is uvx my thing and it's all transparent to you, right? + +00:09:33.020 --> 00:09:33.520 +Which is beautiful. + +00:09:33.900 --> 00:09:34.880 +So what was it like? + +00:09:35.100 --> 00:09:35.320 +Yeah. + +00:09:35.790 --> 00:09:42.340 +So what was it like coming from what sounds like a not super screen focus, super + +00:09:43.020 --> 00:09:47.300 +techie aspect and having to dive into this world and someday you're probably + +00:09:47.420 --> 00:09:49.900 +like, how is it that I'm publishing stuff to PyPI? + +00:09:49.990 --> 00:09:50.740 +What has happened to me? + +00:09:51.300 --> 00:09:51.720 +Yeah. + +00:09:51.970 --> 00:09:56.259 +well, yeah, I remember when I, when I first signed up for GitHub, because + +00:09:56.320 --> 00:10:02.160 +you know, whatever YouTube tutorial I was working through at the time, you know, said that I needed + +00:10:02.160 --> 00:10:08.720 +to do that. You know, I think it all started making a lot of sense. I didn't have any technical + +00:10:08.980 --> 00:10:16.880 +background, but the world kind of open source software, it just kind of made sense. It felt + +00:10:17.020 --> 00:10:23.820 +like it fit really well into my academic, you know, circle. I think a lot of the attitudes are + +00:10:23.840 --> 00:10:27.980 +similar. I agree. I think they are actually. And I think that's, I think that's a pretty neat thing. + +00:10:28.480 --> 00:10:34.240 +Yeah. Very cool. All right. Well, let's talk about what you're doing with digital humanities. + +00:10:34.760 --> 00:10:40.400 +You're actually at a really interesting project or organization, I guess, that does many projects, + +00:10:40.600 --> 00:10:45.020 +right? Yeah. Yeah. So fast, fast forwarding, I did, I finished my PhD in the humanities. + +00:10:45.200 --> 00:10:50.780 +Sorry. I had so much fun. No, that's fine. That's fine. I had so much fun writing like these tools + +00:10:50.860 --> 00:10:54.120 +and then just solving the distribution problem + +00:10:54.450 --> 00:10:55.760 +to share them with other scholars. + +00:10:56.740 --> 00:11:00.120 +That was so fun that I was open to this kind of opportunity + +00:11:00.770 --> 00:11:01.980 +where now I'm doing this full time. + +00:11:02.570 --> 00:11:04.200 +And so, yes, so I'm on the, + +00:11:04.500 --> 00:11:06.720 +we call it affectionately Darth, + +00:11:07.440 --> 00:11:10.240 +which is digital arts and humanities at Harvard. + +00:11:11.200 --> 00:11:14.160 +There has to be a lot of Star Wars memes and references, + +00:11:14.310 --> 00:11:14.680 +I'm sure. + +00:11:14.980 --> 00:11:16.400 +If you can pull up a 404, + +00:11:16.770 --> 00:11:19.060 +I think there will be a Darth Vader reference. + +00:11:19.470 --> 00:11:20.660 +Seriously, I'm here for it. + +00:11:22.360 --> 00:11:25.680 +Yes, page not found. I find your lack of nav disturbing. + +00:11:27.660 --> 00:11:33.020 +You know what? I think that is beautiful. And I really, I really think that people should embrace + +00:11:33.580 --> 00:11:40.860 +the 404, the fun 404 page, you know, more, right? There should really be something going on that + +00:11:40.900 --> 00:11:45.560 +like makes it, you know, something hasn't worked out, but you can just, you can make people laugh. + +00:11:46.280 --> 00:11:47.340 +Yeah. I appreciate that. + +00:11:48.560 --> 00:11:50.360 +I've heard people push back against it. + +00:11:50.490 --> 00:11:57.900 +Like if you're on a, if you're on like your medical website and you're maybe about to get bad news and then you get like a picture of a kitten. + +00:12:00.160 --> 00:12:01.880 +Dr. Kitten doesn't know where your results went. + +00:12:02.020 --> 00:12:02.760 +Like I get that. + +00:12:02.800 --> 00:12:03.300 +That's not funny. + +00:12:04.060 --> 00:12:05.780 +But I mean, most things are not that serious. + +00:12:06.560 --> 00:12:06.780 +Yeah. + +00:12:07.540 --> 00:12:07.920 +Mostly. + +00:12:08.780 --> 00:12:08.920 +Okay. + +00:12:09.180 --> 00:12:11.740 +So what kind of things does Darth do? + +00:12:12.080 --> 00:12:16.680 +You've described this as kind of a web or tech agency within Harvard. + +00:12:17.280 --> 00:12:18.200 +Yeah, it is very much. + +00:12:18.420 --> 00:12:21.740 +So, you know, Harvard has a gigantic IT group. + +00:12:21.890 --> 00:12:28.080 +I don't know how many hundreds of people work, but more than 500 people in IT. + +00:12:28.840 --> 00:12:33.000 +We are a small team and we operate very much like a small agency. + +00:12:33.540 --> 00:12:41.500 +So usually what happens is a faculty member has a funded research project that's going to last for an amount of time. + +00:12:42.210 --> 00:12:44.640 +And then we consult with them to build it. + +00:12:44.880 --> 00:12:53.160 +And most of the time, I kind of think of these as I kind of have these different categories of these kinds of projects that I think of. + +00:12:54.070 --> 00:12:56.060 +I lost in my notes what I call them. + +00:12:56.170 --> 00:12:57.240 +But they are there. + +00:12:57.390 --> 00:13:01.160 +You have like a one is like a virtual research environment. + +00:13:01.450 --> 00:13:07.400 +So the focus is this is this is a platform that we're building for the research to be done on. + +00:13:07.720 --> 00:13:17.000 +Like the reason the research should be done in like a web app would be because you have access to visualization, to Postgres, to Pandas. + +00:13:17.170 --> 00:13:23.420 +So we can kind of build up this platform to do the actual research on and some of the data entry. + +00:13:23.700 --> 00:13:26.060 +So like a full on research application. + +00:13:26.660 --> 00:13:27.040 +Yeah, exactly. + +00:13:27.580 --> 00:13:36.040 +I guess you can also kind of see your work through the different stages of research projects and academic research and so on. + +00:13:36.220 --> 00:13:41.900 +And we'll get to maybe end of life in a sense further down in the conversation. + +00:13:42.470 --> 00:13:48.680 +But so this would be we have a grant or we just work here and we're going to work on some form of research. + +00:13:49.210 --> 00:13:49.960 +What do you give them? + +00:13:50.480 --> 00:13:58.540 +Right. And I think that's a super interesting challenge because one of the real common answers would be Jupyter, Jupyter Lab, Marimo, whatever. + +00:13:59.130 --> 00:14:05.380 +But that's still pretty code heavy for people who are possibly philosophers or something, you know. + +00:14:05.800 --> 00:14:12.820 +Oh, exactly. That's why in digital humanities, I won't even, maybe I won't even attempt to define + +00:14:13.710 --> 00:14:19.820 +it in any narrow sense, because I'll get in trouble with somebody. But you have two groups + +00:14:20.370 --> 00:14:26.580 +that are interfacing with each other. And one is digital humanities as a field, like as a subfield, + +00:14:26.800 --> 00:14:31.320 +all of its own. And these are people who have humanities domain, like knowledge, + +00:14:31.860 --> 00:14:36.680 +and technical skills, and they're bringing them together. And in a lot of cases, the audience for + +00:14:36.840 --> 00:14:42.480 +that kind of work is other people working in the digital humanities. But far more common, + +00:14:42.780 --> 00:14:49.220 +and this is what we work with, is people who have humanities domain expertise, and they want to + +00:14:49.600 --> 00:14:55.560 +publish or do research or share with other people who have that same humanities domain expertise, + +00:14:55.640 --> 00:14:59.640 +and they are now interested in adding a technical component to it. + +00:14:59.960 --> 00:15:02.000 +How can we supercharge what they have? + +00:15:03.500 --> 00:15:06.120 +This portion of Talk Python is brought to you by Sentry. + +00:15:06.580 --> 00:15:09.680 +I've been using Sentry personally on almost every application + +00:15:10.060 --> 00:15:12.480 +and API that I've built for Talk Python and beyond + +00:15:13.260 --> 00:15:14.260 +over the last few years. + +00:15:14.580 --> 00:15:17.460 +They're a core building block for keeping my infrastructure solid. + +00:15:18.060 --> 00:15:19.360 +They should be for yours as well. + +00:15:19.640 --> 00:15:20.020 +Here's why. + +00:15:20.680 --> 00:15:22.100 +Sentry doesn't just catch errors. + +00:15:22.200 --> 00:15:24.900 +It catches all the stuff that makes your app feel broken, + +00:15:25.280 --> 00:15:27.560 +the random slowdown, the freeze you can't reproduce, + +00:15:28.260 --> 00:15:30.620 +that bug that only shows up once real users hit it. + +00:15:30.960 --> 00:15:31.820 +And when something goes wrong, + +00:15:32.180 --> 00:15:34.500 +Sentry gives you the whole chain of events in one place. + +00:15:34.720 --> 00:15:37.700 +Errors, traces, replays, logs, dots connected. + +00:15:38.080 --> 00:15:39.900 +You can see what's led to the issue + +00:15:40.040 --> 00:15:41.880 +without digging through five different dashboards. + +00:15:42.700 --> 00:15:44.720 +SEER, Sentry's AI debugging agent, + +00:15:45.200 --> 00:15:47.180 +builds on this data, taking the full context, + +00:15:47.840 --> 00:15:49.820 +explaining why the issue happened, + +00:15:50.400 --> 00:15:52.780 +pointing to the code responsible, drafts a fix, + +00:15:52.880 --> 00:15:55.840 +and even flags if your PR is about to introduce a new problem. + +00:15:56.680 --> 00:15:57.720 +The workflow stays simple. + +00:15:58.160 --> 00:15:59.900 +Something breaks, Sentry alerts you, + +00:16:00.080 --> 00:16:01.880 +the dashboard shows you the full context, + +00:16:02.220 --> 00:16:05.360 +Seer helps you fix it and catch new issues before they ship. + +00:16:06.080 --> 00:16:08.920 +It's totally reasonable to go from an error occurred + +00:16:09.080 --> 00:16:11.140 +to fixed in production in just 10 minutes. + +00:16:12.200 --> 00:16:14.960 +I truly appreciate the support that Sentry has given me + +00:16:15.060 --> 00:16:17.520 +to help solve my bugs and issues in my apps, + +00:16:18.160 --> 00:16:20.740 +especially those tricky ones that only appear in production. + +00:16:21.100 --> 00:16:22.580 +I know you will too if you try them out. + +00:16:22.880 --> 00:16:24.520 +So get started today with Sentry. + +00:16:24.700 --> 00:16:29.720 +Just visit talkpython.fm/sentry and get $100 in Sentry credits. + +00:16:30.240 --> 00:16:30.960 +Please use that link. + +00:16:31.060 --> 00:16:32.320 +It's in your podcast player show notes. + +00:16:32.420 --> 00:16:37.760 +If you're signing up some other way, you can use our code talkpython26, all one word, + +00:16:38.340 --> 00:16:40.900 +talkpython26, to get $100 in credits. + +00:16:41.680 --> 00:16:43.360 +Thank you to Sentry for supporting the show. + +00:16:44.500 --> 00:16:49.319 +Maybe just take a moment and speak to, maybe, I don't know if this venue will actually speak + +00:16:49.340 --> 00:16:54.200 +directly to anybody who I was imagining here, but people who work with folks, what would you tell + +00:16:54.340 --> 00:16:58.720 +somebody who works with a group who have some technical skill, who could create some of these + +00:16:58.880 --> 00:17:02.280 +things that we're going to talk about, but the people who they've created for don't necessarily + +00:17:02.540 --> 00:17:09.420 +think they need it or know that they need it. I've gone often on rants about how programming is a + +00:17:09.680 --> 00:17:15.260 +superpower, not a replacement for your job, right? Yeah. That's a problem for a lot of people, + +00:17:15.360 --> 00:17:20.500 +especially because you might use some new computer tools to supercharge your research. + +00:17:20.980 --> 00:17:25.740 +But the article that you publish or the research output of that, the audience, they may not + +00:17:25.860 --> 00:17:27.660 +be interested in hearing about that at all. + +00:17:28.040 --> 00:17:32.760 +And so for most people who are working in this space, the tools, you have to use them + +00:17:33.000 --> 00:17:37.720 +in such a way that you can talk about the research output without talking about the + +00:17:37.860 --> 00:17:38.000 +tool. + +00:17:38.260 --> 00:17:42.979 +And we have other venues to talk about the tools themselves, like the Journal for Open + +00:17:43.000 --> 00:17:48.660 +source software and you can kind of get some of it out there. But that is a, that's the significant + +00:17:48.880 --> 00:17:53.020 +challenge is convincing people that it, that it could be useful and then convincing the audience + +00:17:53.230 --> 00:17:57.800 +that they should be interested in kind of the methods behind how some of the new research comes + +00:17:57.800 --> 00:18:02.940 +up. Also, I think I'm a big believer that presenting stuff in the right order is really, + +00:18:03.130 --> 00:18:07.620 +really important. If you present your research and it's beautiful and powerful and oh, look, + +00:18:07.760 --> 00:18:12.500 +we've also, by the way, covered a hundred times more data than any prior research. Surprise, + +00:18:12.760 --> 00:18:13.520 +I wonder how I did that. + +00:18:14.160 --> 00:18:15.400 +And then people are like, this is amazing. + +00:18:16.580 --> 00:18:19.580 +Then after you kind of hook them with the inspiration and what's possible, + +00:18:19.680 --> 00:18:21.480 +then you're like, let me tell you about the tool. + +00:18:21.600 --> 00:18:22.860 +And all of a sudden you're like, that's a cool tool, right? + +00:18:22.920 --> 00:18:26.100 +This is not just like geekery, like programmer, you know, + +00:18:26.440 --> 00:18:28.120 +Charlie Brown speak, wah, wah, wah, wah, wah. + +00:18:28.300 --> 00:18:29.660 +You know, it's like, no, I'm listening. + +00:18:29.880 --> 00:18:30.460 +Tell me now. + +00:18:30.820 --> 00:18:31.260 +Yeah, exactly. + +00:18:31.620 --> 00:18:34.960 +I mean, one of the things I think that really opens people's eyes + +00:18:35.300 --> 00:18:37.720 +is a really powerful search interface. + +00:18:38.260 --> 00:18:39.740 +You have all of this research data. + +00:18:40.120 --> 00:18:45.020 +just put it behind Elasticsearch with some really good filtering on it. And all of a sudden you have + +00:18:45.180 --> 00:18:50.740 +fast, rapid access to the data in a way you never had before. Like you were never scrolling through + +00:18:51.140 --> 00:18:55.160 +the Excel spreadsheets and finding exactly what you wanted, like you were with this new search + +00:18:55.400 --> 00:19:00.360 +interface. And that by itself is like so simple. We're so used to that in web development that + +00:19:00.480 --> 00:19:05.099 +like everything needs to have a fantastic search now. But so many people have their data locked + +00:19:05.120 --> 00:19:07.500 +behind, you know, a terrible search interface. + +00:19:07.960 --> 00:19:10.400 +Yeah, just a few things to sort of expose that. + +00:19:10.500 --> 00:19:14.880 +So this, give us a sense of what these data exploration web apps might look like. + +00:19:14.940 --> 00:19:20.060 +These are probably kind of mostly stuck to the inside, kind of internal to the research + +00:19:20.540 --> 00:19:22.820 +lab research team groups and so on. + +00:19:22.960 --> 00:19:24.720 +These are probably not that public facing, right? + +00:19:24.980 --> 00:19:28.580 +Almost everything we work on does end up having a public facing component. + +00:19:28.940 --> 00:19:33.960 +So maybe the research itself is done, locked behind a user login. + +00:19:34.300 --> 00:19:35.440 +That's just for the researchers. + +00:19:36.290 --> 00:19:38.880 +But then they expose that research to the public, + +00:19:39.520 --> 00:19:41.080 +usually with a good search interface + +00:19:41.640 --> 00:19:44.840 +and different pages for exploring their data + +00:19:45.020 --> 00:19:47.200 +and visualizations and things like that. + +00:19:47.380 --> 00:19:49.360 +So yeah, everything we do ends up becoming + +00:19:49.850 --> 00:19:52.560 +a production public web app in the end. + +00:19:52.760 --> 00:19:54.740 +And then another one of your categories, + +00:19:54.830 --> 00:19:57.000 +you put it was virtual research environments + +00:19:57.260 --> 00:19:59.740 +like data entry, publishing, authoring, collaboration. + +00:20:00.050 --> 00:20:00.540 +Tell us about that. + +00:20:01.280 --> 00:20:03.139 +Yeah, so a good example of this maybe + +00:20:03.160 --> 00:20:08.820 +is one of the projects that... Well, actually, the best example of it is the project I worked on + +00:20:08.930 --> 00:20:16.380 +during my PhD. It's called Apatosaurus. The short story behind the name is that it sounds like + +00:20:16.540 --> 00:20:24.280 +apparatus. In textual criticism, when you are displaying and visualizing variant readings to + +00:20:24.710 --> 00:20:31.959 +a base text, that form of visualizing it is a critical apparatus. A critical apparatus is a + +00:20:32.040 --> 00:20:37.500 +a pretty boring website name, but Apatosaurus dinosaurs might make textual criticism sound fun. + +00:20:37.720 --> 00:20:43.180 +Yeah, I do love dinosaurs. No, that's really cool. So this, this comes out as a web app. And I know + +00:20:43.180 --> 00:20:46.160 +you also have some, you talked about some desktop apps as well. + +00:20:46.640 --> 00:20:50.460 +Yep. Yep. That's right. So, yeah. So, so there's this people, people upload their, + +00:20:50.550 --> 00:20:54.960 +their collation to this and then they can visualize it. And like there, there's a public + +00:20:55.440 --> 00:21:00.320 +component of this as well, but really the backend is editing, editing a collation, + +00:21:00.500 --> 00:21:03.160 +and adding notes to all of the different readings and stuff. + +00:21:03.560 --> 00:21:07.060 +So I could show what the backend looks like, + +00:21:07.280 --> 00:21:08.200 +but we can also move on. + +00:21:08.440 --> 00:21:11.420 +- Let's move on just because most people + +00:21:11.620 --> 00:21:14.540 +will not totally hear, but just give us a sense of like, + +00:21:14.880 --> 00:21:18.740 +like what do people, what do you create for people + +00:21:18.900 --> 00:21:21.660 +so that they're like, yeah, I can use this app, right? + +00:21:21.760 --> 00:21:23.440 +Like give us a sense of some of the features, + +00:21:23.660 --> 00:21:24.960 +I guess is what I'm getting to. + +00:21:25.260 --> 00:21:29.319 +- Yeah, so another good example is we have a project + +00:21:29.360 --> 00:21:32.380 +at Harvard called Mapping Color in History. + +00:21:33.150 --> 00:21:36.980 +And this is a collaboration with a lab. + +00:21:37.150 --> 00:21:38.900 +This lab brings in pieces of artwork + +00:21:39.480 --> 00:21:42.560 +and they do spectral analysis on the pigments + +00:21:42.590 --> 00:21:45.160 +so they can identify what was used + +00:21:45.160 --> 00:21:48.360 +to make a particular color of this red + +00:21:48.550 --> 00:21:50.860 +or what was made to make this color of blue. + +00:21:51.420 --> 00:21:53.640 +And then the idea is tracking + +00:21:54.000 --> 00:21:56.880 +how did people make those pigments over time, + +00:21:57.280 --> 00:22:01.720 +over time and specifically in Asian art. + +00:22:02.260 --> 00:22:04.380 +Is this the Dharmra, Puna, Puna? + +00:22:05.440 --> 00:22:07.960 +No, this is mapping color in history. + +00:22:08.030 --> 00:22:09.640 +I don't think it's up here. + +00:22:09.770 --> 00:22:10.280 +Sorry about that. + +00:22:10.420 --> 00:22:10.680 +Somewhere. + +00:22:10.940 --> 00:22:11.300 +That's all right. + +00:22:11.330 --> 00:22:11.840 +I'll find it. + +00:22:12.030 --> 00:22:12.440 +Keep talking. + +00:22:13.680 --> 00:22:13.840 +Okay. + +00:22:14.050 --> 00:22:16.300 +So the front end is great. + +00:22:16.510 --> 00:22:18.040 +You know, like the public end, + +00:22:18.180 --> 00:22:21.000 +this is people can explore by pigments + +00:22:21.280 --> 00:22:24.440 +and then see the images that contain those pigments. + +00:22:24.560 --> 00:22:30.680 +Now in the back end, what the researchers will be able to do is correlate exactly which + +00:22:30.960 --> 00:22:34.260 +point of a painting the analysis was done on. + +00:22:34.490 --> 00:22:38.640 +So they have this deep zoom image viewer where they'll zoom in and they'll select the point + +00:22:39.390 --> 00:22:40.280 +where that was taken from. + +00:22:41.090 --> 00:22:47.640 +So how else would you do that other than a digital interface to indicate on an image of + +00:22:47.950 --> 00:22:52.060 +a painting where that spectral analysis was performed? + +00:22:52.380 --> 00:22:55.020 +Sounds almost like astronomy in a weird way. + +00:22:55.050 --> 00:22:55.300 +Oh, yeah. + +00:22:55.840 --> 00:23:04.580 +We zoomed into here and we took a different spectrum of the painting and we realized that it's actually identical to this, you know, something crazy like that, right? + +00:23:04.900 --> 00:23:06.140 +Yeah, yeah, yeah, that's right. + +00:23:06.200 --> 00:23:08.560 +Yeah, so it's essentially a pigments, like a pigments database. + +00:23:10.100 --> 00:23:17.940 +So the third category of these digital humanities projects that you put down was like data extraction, transformation. + +00:23:19.260 --> 00:23:29.540 +In data science, they often say, you know, 80% of the work is the data wrangling, which is like cleaning, organization, just getting it so you could possibly start asking questions about it. + +00:23:29.820 --> 00:23:30.960 +I'm sure you all do a lot of that. + +00:23:31.180 --> 00:23:31.440 +Absolutely. + +00:23:32.560 --> 00:23:40.880 +So often, the very beginning of a project might be an Excel sheet or several spreadsheets. + +00:23:41.800 --> 00:23:46.020 +And the first task is to ingest these into, you know, a proper database. + +00:23:46.640 --> 00:23:48.600 +Not so much MongoDB for us. + +00:23:48.760 --> 00:23:49.840 +It's going into Postgres. + +00:23:50.340 --> 00:23:51.420 +We're Django Shop. + +00:23:51.680 --> 00:23:52.480 +We're Django Shop. + +00:23:52.630 --> 00:23:53.760 +So it's going into Postgres. + +00:23:55.090 --> 00:24:06.560 +And yeah, no, that is probably the number one challenge of the early stage is figuring out what the right data model is, what the right relationships are to model the data. + +00:24:07.300 --> 00:24:17.020 +Doing that work is advantageous to everybody because, you know, it helps both the researchers who brought the data to think about it in a more organized way. + +00:24:17.410 --> 00:24:18.540 +I mean, they've been trying to do that. + +00:24:18.720 --> 00:24:19.680 +And they have the spreadsheets. + +00:24:20.160 --> 00:24:27.800 +But now we're modeling out the data so that we can add it to database tables and then to use later. + +00:24:27.880 --> 00:24:29.340 +So that works out well for everybody. + +00:24:30.000 --> 00:24:30.720 +And yeah, absolutely. + +00:24:31.100 --> 00:24:45.760 +Cleaning the data, getting dates, working with fuzzy dates, being able to parse July of 2020 or summer of 2020 and handling kind of all of those cases so that we do get dates in the end. + +00:24:45.780 --> 00:24:55.980 +One of the crazy stories from data parsing history is one of the, I can't remember exactly what it was, you talked about biology tools or genetics tools earlier. + +00:24:56.100 --> 00:25:03.780 +One of the groups that names genes had to change the name of a gene because it kept getting parsed by Excel into a date. + +00:25:04.880 --> 00:25:05.520 +Yeah, I remember that. + +00:25:05.590 --> 00:25:06.200 +I remember that. + +00:25:06.320 --> 00:25:06.600 +That's right. + +00:25:07.260 --> 00:25:07.580 +Yes. + +00:25:08.100 --> 00:25:10.580 +So these are the weird edge cases I'm sure you run into. + +00:25:11.940 --> 00:25:13.120 +Like it's not even supposed to be a date. + +00:25:13.220 --> 00:25:13.940 +Why is this a date? + +00:25:13.990 --> 00:25:14.780 +I don't know. + +00:25:14.940 --> 00:25:16.240 +Why is it helping out here? + +00:25:16.920 --> 00:25:17.820 +The code keeps crashing. + +00:25:18.000 --> 00:25:20.480 +Like pandas parsed it as a date and it's not or whatever. + +00:25:21.220 --> 00:25:21.540 +Absolutely. + +00:25:21.980 --> 00:25:22.060 +Yeah. + +00:25:22.140 --> 00:25:22.300 +Yeah. + +00:25:22.340 --> 00:25:27.320 +So yeah, usually lots of test suites around that ingest process until we've got it. + +00:25:27.640 --> 00:25:32.220 +Now, once we've got it in, usually the research is ongoing and then we're able to provide + +00:25:32.420 --> 00:25:38.140 +them now a new cleaned interface to do the additional data entry as the project is going. + +00:25:38.420 --> 00:25:39.780 +And that's usually a win-win for everybody. + +00:25:40.180 --> 00:25:40.340 +Sure. + +00:25:40.620 --> 00:25:45.780 +And so this sort of ETL ingestion side of everything is it's like, don't worry, + +00:25:46.420 --> 00:25:47.460 +Darth has got it for you. + +00:25:47.760 --> 00:25:51.180 +And then we'll provide you like a database connection to start working. + +00:25:51.480 --> 00:25:54.700 +Or do you give them the tools and then they kind of iterate on them? + +00:25:54.940 --> 00:26:00.220 +And how much is this you and how much is this you providing like CLI tools and stuff + +00:26:00.460 --> 00:26:01.540 +or notebooks over to people? + +00:26:03.560 --> 00:26:08.520 +I'd say most of the people that we're working with are aware of the technical tools, + +00:26:08.640 --> 00:26:10.380 +but they don't want a database connection. + +00:26:10.800 --> 00:26:16.520 +So we are giving them, we're doing the ingest and then building a platform where they can begin interacting with their data. + +00:26:17.240 --> 00:26:18.720 +Yeah, I'm sure they don't want one. + +00:26:20.140 --> 00:26:22.600 +Maybe you give them an app though, right? + +00:26:22.820 --> 00:26:24.940 +With like Elasticsearch and other things that they can. + +00:26:25.120 --> 00:26:25.400 +No, absolutely. + +00:26:25.680 --> 00:26:26.400 +Yeah, that's what we do. + +00:26:26.720 --> 00:26:27.040 +Yeah, okay. + +00:26:27.140 --> 00:26:32.520 +Yeah, we give them a web platform to begin exploring, to begin publishing. + +00:26:34.320 --> 00:26:38.760 +So I was thinking that you said you're a Django shop, which is cool. + +00:26:38.840 --> 00:26:43.280 +It sounds, though, to me like describing what you're doing, just imagining how this is. + +00:26:43.640 --> 00:26:46.000 +You're probably creating these projects often. + +00:26:46.520 --> 00:26:49.440 +How often does one of these projects actually last? + +00:26:49.980 --> 00:26:51.900 +Or how many of them do you iterate? + +00:26:53.180 --> 00:26:53.700 +I'm trying to get a sense. + +00:26:53.920 --> 00:26:56.960 +Do you work on stuff for a year or is it like every two weeks we're on a new project? + +00:26:58.180 --> 00:26:59.980 +It's why I think of us as like an agency. + +00:27:00.900 --> 00:27:04.240 +Because we get to work on greenfield projects fairly often, like you're imagining. + +00:27:04.700 --> 00:27:08.880 +Which would not be the case normally at a big university IT department. + +00:27:09.960 --> 00:27:15.040 +So, you know, maybe two or three projects a year, two or three big ones a year. + +00:27:15.500 --> 00:27:18.640 +And then we have to put to bed a few a year as well. + +00:27:18.740 --> 00:27:21.260 +Because these things, they're funded with grant money. + +00:27:21.600 --> 00:27:24.060 +And then the grant money runs out and it's time. + +00:27:24.200 --> 00:27:26.320 +And then we have to figure out what do we do with it now? + +00:27:26.380 --> 00:27:31.060 +We don't want to lose the data and this way of presenting it. + +00:27:31.100 --> 00:27:33.140 +But we can't keep paying for Elasticsearch. + +00:27:33.520 --> 00:27:34.120 +Yeah, of course. + +00:27:34.380 --> 00:27:37.620 +I'm certainly, we're going to dive into that because that is, but let's save that for the + +00:27:37.740 --> 00:27:37.780 +end. + +00:27:37.800 --> 00:27:40.920 +It seems like that's the arc of the story of these things. + +00:27:40.960 --> 00:27:44.700 +But I certainly think it's something that you don't think about that much, right? + +00:27:44.980 --> 00:27:47.360 +Like you said, it was only a hundred dollars a month for this. + +00:27:47.440 --> 00:27:48.280 +And we got a big grant. + +00:27:48.400 --> 00:27:49.200 +There's a bunch of, no big deal. + +00:27:49.280 --> 00:27:52.880 +But like when the grant's out, who's on the hook for a hundred dollars a month and making + +00:27:53.040 --> 00:27:55.920 +sure it survives upgrades and all that kind of business. + +00:27:56.400 --> 00:27:56.780 +No, that's right. + +00:27:57.080 --> 00:27:57.240 +Yeah. + +00:27:57.360 --> 00:28:02.720 +So my original question when I started on this path was thinking like, do you, how do you + +00:28:02.780 --> 00:28:03.400 +get started on these? + +00:28:03.460 --> 00:28:07.640 +Do you have like a big framework or a cookie cutter sort of thing or something like this + +00:28:07.760 --> 00:28:11.740 +is how we do it because it plugs into all this other automation and tools we built for + +00:28:11.800 --> 00:28:12.680 +the last 10 projects. + +00:28:13.220 --> 00:28:14.400 +You know, that's kind of a unique position. + +00:28:14.920 --> 00:28:19.020 +A lot of companies build one website for themselves and that's their app or they're + +00:28:19.020 --> 00:28:21.480 +an agency that goes across so many, so much variation. + +00:28:21.660 --> 00:28:22.600 +They can't do that kind of stuff. + +00:28:22.680 --> 00:28:22.820 +Right. + +00:28:23.220 --> 00:28:23.820 +That's right. + +00:28:24.080 --> 00:28:24.360 +That's right. + +00:28:25.300 --> 00:28:25.920 +That's a good question. + +00:28:26.320 --> 00:28:28.840 +We have things that we reuse. + +00:28:29.000 --> 00:28:35.400 +Some of them are open source, different search components and things that we maintain that + +00:28:36.110 --> 00:28:37.420 +we'll use across projects. + +00:28:37.930 --> 00:28:41.100 +And we have tried to do the cookie cutter Django project. + +00:28:41.640 --> 00:28:47.240 +The truth is, each project is different enough that really we like to evaluate it from first + +00:28:47.520 --> 00:28:54.120 +principles as we're evaluating it and thinking, what is the best technology to use? + +00:28:55.370 --> 00:28:55.520 +Yeah. + +00:28:55.750 --> 00:28:55.920 +Yeah. + +00:28:56.020 --> 00:28:59.000 +So yeah, we don't have a cookie cutter. + +00:28:59.050 --> 00:29:04.200 +We don't have a kind of a meta framework for bootstrapping them because they're sufficiently + +00:29:04.450 --> 00:29:05.780 +different from each other that we... + +00:29:05.960 --> 00:29:06.680 +I find that too. + +00:29:07.030 --> 00:29:07.640 +I find that too. + +00:29:08.120 --> 00:29:12.520 +The idea of how we could just grab this cookie cutter or copier. + +00:29:12.610 --> 00:29:13.600 +Are you familiar with copier? + +00:29:14.080 --> 00:29:15.480 +People out there might be familiar with that. + +00:29:15.600 --> 00:29:20.740 +It's a little bit like cookie cutter with the bonus that you can update it later if you + +00:29:21.020 --> 00:29:24.499 +change your mind about something, like actually change this project to use Postgres rather + +00:29:24.520 --> 00:29:29.960 +than SQLite or something, which is pretty cool. But every time that I do, every time I try to work + +00:29:30.040 --> 00:29:33.680 +with one of those projects, even ones that I've created for myself, I'm not, I hate not anyone. + +00:29:34.160 --> 00:29:39.400 +I'm like, oh, it's like 75% awesome and 25%. I just got to take this stuff out. You know, + +00:29:39.940 --> 00:29:43.700 +I'll just, I'll just do it from scratch. It's not, how hard is this? I'll just create a few folders + +00:29:43.860 --> 00:29:48.200 +and put a few things in there and I'll copy the one, like the pyproject.tom or like the one thing + +00:29:48.300 --> 00:29:52.260 +that's like, how do I do this again? I'll just copy that and we're good to go. Yeah. I mean, + +00:29:52.540 --> 00:29:53.160 +That's what I find. + +00:29:53.460 --> 00:29:53.920 +That's what I find. + +00:29:53.930 --> 00:29:56.660 +I find it, it seems like a really brilliant idea, + +00:29:56.920 --> 00:30:00.280 +but in practice, it hasn't saved us time yet. + +00:30:00.880 --> 00:30:02.320 +No, I mean, maybe it's a case study. + +00:30:02.460 --> 00:30:04.320 +Like, okay, let's see what they're doing for this one. + +00:30:04.320 --> 00:30:05.160 +Oh, that is interesting + +00:30:05.270 --> 00:30:07.460 +how they're integrating this other thing maybe, + +00:30:07.640 --> 00:30:10.820 +but as a true foundation, I find it in theory awesome. + +00:30:11.280 --> 00:30:13.840 +In practice, I just end up not doing it for various reasons. + +00:30:14.200 --> 00:30:14.560 +Don't know why. + +00:30:14.840 --> 00:30:15.840 +I'm gonna save this for later. + +00:30:16.700 --> 00:30:17.880 +Because the question I'm about to ask you + +00:30:17.900 --> 00:30:20.860 +is gonna send us just down a rat hole. + +00:30:21.260 --> 00:30:26.960 +So instead, before we go down the rat hole, maybe we could, not that one, maybe we could + +00:30:27.020 --> 00:30:32.300 +talk about, I mean, you talked about some, but let's maybe just feature some of the projects + +00:30:32.300 --> 00:30:34.580 +that are maybe more well-known that you guys have done. + +00:30:35.059 --> 00:30:35.460 +Sure. + +00:30:35.780 --> 00:30:36.120 +Yeah, good. + +00:30:36.580 --> 00:30:40.220 +So yeah, one of them is called the Amendments Project. + +00:30:40.900 --> 00:30:46.119 +And this is, I didn't know this until I started working on this project, that there are, there + +00:30:46.140 --> 00:30:52.740 +There have been thousands of, I think it's 22, at least 22,000 proposed amendments to + +00:30:52.740 --> 00:30:55.800 +the United States Constitution that never went anywhere. + +00:30:56.230 --> 00:31:01.500 +And so kind of the goal of this project is to show that there have been lots of attempts + +00:31:02.140 --> 00:31:06.400 +to amend the Constitution, but actually the Constitution is frozen. + +00:31:06.690 --> 00:31:11.480 +I mean, it's not actually amendable anymore, at least not in the politics of any time recently. + +00:31:12.440 --> 00:31:13.580 +So this is a database. + +00:31:14.180 --> 00:31:19.040 +I cannot imagine a situation where the U.S. Constitution gets amended. + +00:31:19.310 --> 00:31:21.440 +It has to be unanimous across all the states, right? + +00:31:21.820 --> 00:31:22.140 +Is that right? + +00:31:22.250 --> 00:31:22.740 +I can't remember. + +00:31:23.440 --> 00:31:23.840 +I don't know. + +00:31:23.940 --> 00:31:25.720 +I remember off the top of my head if it has to be unanimous, + +00:31:25.730 --> 00:31:27.900 +but it certainly has to be across party lines. + +00:31:28.520 --> 00:31:30.940 +Yeah, it's got to be pretty darn close if it's not at all. + +00:31:32.020 --> 00:31:36.200 +It's like time travel or travel to speed of light. + +00:31:36.590 --> 00:31:37.640 +Could be theoretically possible. + +00:31:38.180 --> 00:31:38.920 +Probably not going to happen. + +00:31:40.560 --> 00:31:41.340 +No, it's hard to see. + +00:31:41.600 --> 00:31:42.140 +It's hard to see. + +00:31:42.310 --> 00:31:42.400 +Yeah. + +00:31:42.620 --> 00:31:46.160 +So this is from a historian at Harvard. + +00:31:46.680 --> 00:31:53.120 +And so it's a database of all and the full text from all of these amendments. + +00:31:53.340 --> 00:32:07.600 +And, you know, it's from the public's point of view, it's a Postgres full text vector search interface for finding and filtering through on all of the different amendments that have been proposed. + +00:32:08.180 --> 00:32:08.660 +I love it. + +00:32:08.980 --> 00:32:10.020 +Yeah, this is a nice looking site. + +00:32:10.520 --> 00:32:11.560 +We work with a designer. + +00:32:12.080 --> 00:32:16.920 +she's very good yeah of course like an agency would right yep yep nice so we'll + +00:32:17.020 --> 00:32:22.060 +get a really pretty rich search interface and then off you go I have no idea even + +00:32:22.100 --> 00:32:24.900 +what I would search for but yeah well you can always search for something + +00:32:25.180 --> 00:32:28.720 +religious something abortion related there's gonna be lots of things there I + +00:32:29.080 --> 00:32:31.920 +thought all those also like guns but like I don't want to go down I'm not sure I + +00:32:32.020 --> 00:32:37.200 +even want to go down there right awesome though this looks super useful maybe + +00:32:37.380 --> 00:32:41.039 +someday we'll have a functional government again we'll see let's let's + +00:32:41.060 --> 00:32:45.820 +change it or maybe we'll go down and it's folklore like look at you so all right so yeah so another + +00:32:45.980 --> 00:32:51.680 +really great uh project at least from a content point of view uh that's interesting um the research + +00:32:51.840 --> 00:33:00.680 +that it's doing um is the fin folklore database um which so in in in celtic storytelling you know + +00:33:01.820 --> 00:33:07.919 +um moms have been telling and telling stories to daughters and and and and people have been + +00:33:08.000 --> 00:33:14.660 +telling stories for a very long time hundreds or a thousand years about um finn mcummel who is a + +00:33:14.910 --> 00:33:21.040 +hero a hero from irish mythology some of it some of it based in you know historical events but it + +00:33:21.120 --> 00:33:28.940 +goes back it goes back so far um so there are there's many hundreds or thousands of of of these + +00:33:29.300 --> 00:33:33.300 +stories that have been spread and versions of these stories that have that have been told and + +00:33:33.160 --> 00:33:47.440 +And so some of them are audio recordings where somebody like some researcher has gone out to an island off the coast of Scotland and recorded somebody telling their version of the hero of Finn and his band of heroes. + +00:33:47.640 --> 00:33:53.080 +You know, they defend Scotland and Ireland from invaders and attackers. + +00:33:53.880 --> 00:33:57.960 +Very exciting stories and stuff and a team of characters. + +00:33:59.020 --> 00:34:04.880 +So there's audio recordings and then there's documents, like written documents that contain + +00:34:05.120 --> 00:34:05.180 +these. + +00:34:05.180 --> 00:34:11.139 +And so this is a database of kind of all of those all in one place with, on the public + +00:34:11.340 --> 00:34:17.820 +side, a nice search interface for discovering them, you know, either using the map view or + +00:34:18.040 --> 00:34:18.139 +searching. + +00:34:18.480 --> 00:34:19.040 +Yeah, that's cool. + +00:34:19.350 --> 00:34:22.159 +I got my map view for some random thing I searched about here. + +00:34:22.600 --> 00:34:22.919 +Amazing. + +00:34:23.260 --> 00:34:26.300 +But this is pretty interesting, all these different tellings and stuff. + +00:34:26.720 --> 00:34:33.200 +Oh, and yeah, one of the big challenges with this project is that it's fully internationalized. + +00:34:33.450 --> 00:34:35.120 +So it's available in English. + +00:34:35.379 --> 00:34:40.560 +Everything is available in English, Scottish Gaelic, and Irish Gaelic, but that extends + +00:34:40.730 --> 00:34:41.419 +into the database. + +00:34:41.790 --> 00:34:45.360 +So usually people have multiple names recorded for them. + +00:34:45.850 --> 00:34:50.919 +And so, yeah, you may have one person with any number of names in different languages, + +00:34:51.040 --> 00:34:53.879 +sometimes more than one Scottish name, that kind of thing. + +00:34:54.020 --> 00:34:59.540 +And so the data model on this one is quite messy, but sensible. + +00:35:00.300 --> 00:35:03.020 +But yeah, it's quite a lot of different kinds of data to wrangle. + +00:35:03.240 --> 00:35:05.180 +And then with all of the translations for each thing. + +00:35:05.320 --> 00:35:05.960 +Yeah, that's wild. + +00:35:06.100 --> 00:35:11.380 +It's not just, we need the user interface of this thing to translate about. + +00:35:12.640 --> 00:35:13.420 +That's way more, right? + +00:35:13.720 --> 00:35:14.820 +Yeah, yeah, it is that. + +00:35:14.960 --> 00:35:15.540 +It is that. + +00:35:15.740 --> 00:35:20.360 +And then it is also, yes, all the items in the database have a translation or can. + +00:35:22.420 --> 00:35:25.160 +This portion of Talk Python To Me is brought to you by us. + +00:35:25.500 --> 00:35:30.020 +I'm thrilled to announce a brand new app built for developers created by yours truly. + +00:35:30.480 --> 00:35:31.640 +It's called Command Book. + +00:35:32.440 --> 00:35:33.700 +You know that thing you do every morning? + +00:35:34.200 --> 00:35:38.520 +Open up six terminal tabs, CD into this directory, activate that virtual environment, + +00:35:39.000 --> 00:35:40.460 +run the server with --reload. + +00:35:40.760 --> 00:35:44.720 +Now, CD somewhere else, start the background worker, another tab for Docker, + +00:35:45.080 --> 00:35:46.440 +another one to tail production logs. + +00:35:46.900 --> 00:35:49.620 +Every tab just says Python, Python, Python, Docker tail. + +00:35:50.380 --> 00:35:51.660 +and you're clicking through them going, + +00:35:52.120 --> 00:35:53.320 +which Python was that again? + +00:35:53.840 --> 00:35:54.540 +Where my app is running? + +00:35:55.200 --> 00:35:58.060 +Then sometime later, your dev server silently dies + +00:35:58.260 --> 00:35:59.260 +because it tried to reload + +00:35:59.480 --> 00:36:00.840 +while you're in the middle of a code edit, + +00:36:01.500 --> 00:36:04.080 +unmatched brace, a half-written import or something. + +00:36:04.820 --> 00:36:05.840 +Now you're hunting through tabs + +00:36:05.880 --> 00:36:07.380 +to figure out which process crashed + +00:36:07.440 --> 00:36:08.340 +and how to restart it. + +00:36:08.800 --> 00:36:09.680 +My app, CommandBook, + +00:36:10.000 --> 00:36:13.320 +gives all of these long-running commands a permanent home. + +00:36:13.880 --> 00:36:15.120 +You save a command once, + +00:36:15.500 --> 00:36:16.680 +the working directory, the environment, + +00:36:17.040 --> 00:36:18.140 +pre-commands like git pull, + +00:36:18.480 --> 00:36:20.120 +and from then on, you just click run. + +00:36:20.680 --> 00:36:22.040 +You can even group commands together + +00:36:22.340 --> 00:36:24.140 +to start and stop everything for a project + +00:36:24.460 --> 00:36:25.140 +with a single click. + +00:36:25.560 --> 00:36:27.540 +It also has what I call honey badger mode, + +00:36:27.750 --> 00:36:29.140 +auto restart on crash. + +00:36:29.720 --> 00:36:32.140 +So when your dev server goes down mid-reload, + +00:36:32.700 --> 00:36:34.580 +command book just brings it right back up + +00:36:34.840 --> 00:36:36.900 +and does so over and over until the code is fixed. + +00:36:37.520 --> 00:36:39.320 +It also detects URLs from your output + +00:36:39.470 --> 00:36:41.900 +so you're never scrolling through thousands of lines of logs + +00:36:42.080 --> 00:36:44.040 +just to figure out how to reopen your web app. + +00:36:44.580 --> 00:36:46.280 +And it shows you uptime, memory usage, + +00:36:46.480 --> 00:36:48.460 +and all sorts of cool things about your process. + +00:36:49.160 --> 00:36:51.140 +The whole thing is a native macOS app. + +00:36:51.360 --> 00:36:53.680 +No Electron, no Chromium, just 21 megs. + +00:36:54.160 --> 00:36:55.460 +And it comes with a full CLI. + +00:36:55.700 --> 00:36:57.540 +So anything you've configured in the UI, + +00:36:57.960 --> 00:36:59.220 +you can fire off from your terminal + +00:36:59.380 --> 00:37:00.400 +with just a single command. + +00:37:00.860 --> 00:37:02.700 +Right now it's macOS only, + +00:37:03.220 --> 00:37:04.080 +but if there's enough interest, + +00:37:04.320 --> 00:37:05.380 +I'll build a Windows version too. + +00:37:05.620 --> 00:37:06.340 +So let me know. + +00:37:07.180 --> 00:37:09.300 +Please check it out at talkpython.fm + +00:37:09.560 --> 00:37:11.160 +slash command book app. + +00:37:11.640 --> 00:37:12.400 +Download it for free, + +00:37:12.980 --> 00:37:14.140 +level up your developer workflow. + +00:37:14.600 --> 00:37:16.220 +The link is in your podcast player show notes. + +00:37:16.840 --> 00:37:18.780 +That's talkpython.fm/command book. + +00:37:19.210 --> 00:37:21.000 +I really hope you enjoy this new app that I built. + +00:37:22.640 --> 00:37:26.660 +You want to work in the native language of the people who did that part of the folklore + +00:37:26.930 --> 00:37:27.340 +or whatever, right? + +00:37:27.680 --> 00:37:30.280 +Yeah, well, and people are still speaking those languages. + +00:37:30.520 --> 00:37:34.580 +So people who would use this to, you know, like somebody may have heard a story from + +00:37:34.630 --> 00:37:37.960 +their mom or dad and are now would like to find other versions of that story. + +00:37:38.350 --> 00:37:41.940 +And they live in a part of Scotland where they speak Scottish Gaelic as their first language. + +00:37:42.520 --> 00:37:43.680 +They can still access the site. + +00:37:43.880 --> 00:37:48.700 +And then that mapping color history one, that's another one of the public ones that you said is pretty major. + +00:37:49.520 --> 00:37:50.080 +Yeah, that's right. + +00:37:50.280 --> 00:37:50.420 +Yeah. + +00:37:50.720 --> 00:37:53.140 +So, yeah, that's a pigments database. + +00:37:53.360 --> 00:38:04.160 +You can search by either English color names like blue and find all of these Asian paintings that have blue or a particular kind of pigment of how they made the blue. + +00:38:04.440 --> 00:38:04.960 +Yeah, nice. + +00:38:05.360 --> 00:38:07.700 +So what's the open source story? + +00:38:08.300 --> 00:38:11.240 +You're creating all these apps, maybe some of these frameworks. + +00:38:11.420 --> 00:38:12.280 +There's got to be some tools. + +00:38:12.680 --> 00:38:25.900 +Is there a big desire or already an effort to have a lot of these things open source or is it too niche or is it just like this is the advantage of Harvard has is other universities don't get this? + +00:38:27.300 --> 00:38:29.680 +No, it's something we talk about quite a bit. + +00:38:30.820 --> 00:38:35.140 +Usually these things start, usually they start closed source during development. + +00:38:35.380 --> 00:38:44.920 +And then we work with the faculty and we talk about how we can take, you know, like the repo for the web app, how we can take that public. + +00:38:45.450 --> 00:38:48.100 +And so we've done that for a number of projects. + +00:38:48.320 --> 00:38:49.100 +Not all of them are. + +00:38:50.000 --> 00:38:55.880 +But the ideal is that they all make their way into the open, and especially when they become archived. + +00:38:56.160 --> 00:38:56.440 +Sure. + +00:38:56.770 --> 00:38:58.840 +Yeah, that's a good way to help them live on. + +00:38:58.960 --> 00:39:03.900 +And they might even go into GitHub's Arctic Vault, which is crazy. + +00:39:03.980 --> 00:39:14.620 +I don't know if people know about that out there, but GitHub has, quite a while ago, started taking copies of all of the repos and backing them up and storing them in the Arctic vault. + +00:39:14.810 --> 00:39:15.240 +It's kind of cool. + +00:39:15.810 --> 00:39:18.660 +I really, really, really hope we never need that, but it's kind of neat. + +00:39:18.820 --> 00:39:19.340 +Yeah, me too. + +00:39:20.319 --> 00:39:30.300 +Usually universities have their own archival system, so any important research data is usually part of that system as well. + +00:39:30.560 --> 00:39:30.840 +I see. + +00:39:30.990 --> 00:39:31.080 +Okay. + +00:39:31.440 --> 00:39:31.520 +Yeah. + +00:39:32.160 --> 00:39:32.780 +Obviously, right? + +00:39:32.840 --> 00:39:34.940 +Like I'm just, I can't remember where it was. + +00:39:34.940 --> 00:39:39.760 +It was somewhere, I think it was South Korea or Taiwan where like seven years of government + +00:39:40.060 --> 00:39:41.820 +data got lost or something like that. + +00:39:41.820 --> 00:39:43.200 +It was really, really bad recently. + +00:39:43.500 --> 00:39:46.980 +There was a fire and I think they had backups, but maybe just into the building, you know, + +00:39:47.040 --> 00:39:48.000 +like we'll put that out. + +00:39:48.340 --> 00:39:49.800 +We'll back it up to the hard drive over here. + +00:39:50.180 --> 00:39:50.480 +Not good. + +00:39:51.000 --> 00:39:52.920 +No, not good. + +00:39:52.920 --> 00:39:54.340 +You definitely want this stuff to survive. + +00:39:54.360 --> 00:40:00.080 +I mean, academia has this history of like tomes that have survived the past and really, + +00:40:00.260 --> 00:40:02.340 +really long lived information. + +00:40:02.640 --> 00:40:02.700 +Right. + +00:40:02.760 --> 00:40:05.400 +besides the Library of Alexandria or something like that, maybe. + +00:40:05.680 --> 00:40:06.380 +That's what we want. + +00:40:06.620 --> 00:40:07.080 +That's what we want. + +00:40:07.080 --> 00:40:08.860 +We want it to, yeah, we want it to last. + +00:40:09.560 --> 00:40:09.860 +Absolutely. + +00:40:10.180 --> 00:40:14.540 +So maybe that's a good time to sort of talk about the trailing end. + +00:40:14.540 --> 00:40:17.020 +I think there's a lot of interesting things going on here. + +00:40:18.360 --> 00:40:22.740 +Just like you've run out of money, not because you actually run out of money. + +00:40:23.260 --> 00:40:26.520 +The grant is done and you've either spent or given back or whatever + +00:40:26.820 --> 00:40:28.460 +with the remaining little bits of money. + +00:40:28.780 --> 00:40:30.100 +It's always a weird balance with research. + +00:40:30.600 --> 00:40:33.740 +It's like, oh, we got $3,000 left on this research grant. + +00:40:33.790 --> 00:40:34.700 +What are we going to do with it? + +00:40:34.780 --> 00:40:35.820 +It's not like, oh, we're going to give it back. + +00:40:35.930 --> 00:40:36.580 +We just didn't need it. + +00:40:36.940 --> 00:40:41.300 +It's like, we're going to find a way to like fund a student to do a little more work or + +00:40:41.460 --> 00:40:41.520 +whatever. + +00:40:41.720 --> 00:40:43.400 +But eventually the grant is over. + +00:40:43.940 --> 00:40:44.240 +That's right. + +00:40:44.660 --> 00:40:48.560 +You've got some expensive app access to a big database because it needs a big search or + +00:40:49.260 --> 00:40:50.200 +a lot of compute or something. + +00:40:50.780 --> 00:40:51.040 +That's right. + +00:40:52.130 --> 00:40:56.000 +Everything during, like, I mean, anything, anything that's a, that's a Django app. + +00:40:56.640 --> 00:41:04.820 +We deploy to AWS using containers, which isn't the cheapest way to host anything. + +00:41:05.760 --> 00:41:09.100 +But that's for the most part the Harvard way. + +00:41:10.200 --> 00:41:12.240 +And it is robust and is reliable. + +00:41:12.800 --> 00:41:22.320 +And we don't have a DevOps person on call on the weekend to rescue one of these apps. + +00:41:22.480 --> 00:41:24.980 +So having them reliable is good. + +00:41:25.400 --> 00:41:29.940 +Okay, so it's on AWS and paying for the containers, + +00:41:30.180 --> 00:41:32.500 +paying for that Elasticsearch cluster, + +00:41:33.180 --> 00:41:36.280 +the RDS Postgres database. + +00:41:36.890 --> 00:41:38.940 +Okay, well, even if somebody wants to start paying + +00:41:38.940 --> 00:41:39.860 +for that out-of-pocket, + +00:41:40.100 --> 00:41:40.980 +all of those little services, + +00:41:41.090 --> 00:41:44.080 +they add up to enough that we need to do something + +00:41:44.530 --> 00:41:46.340 +when the project hits end of life. + +00:41:46.650 --> 00:41:50.359 +And so our gold standard that we've developed so far + +00:41:50.380 --> 00:41:55.340 +is asking, can this become a static website? + +00:41:55.860 --> 00:41:58.600 +Can we bake this out into all HTML files + +00:41:59.200 --> 00:42:01.800 +and acknowledge that there will be some trade-offs? + +00:42:01.960 --> 00:42:04.440 +We will trade off some searching. + +00:42:04.880 --> 00:42:06.540 +You know, it's not gonna have Elasticsearch. + +00:42:06.840 --> 00:42:08.320 +Doesn't mean that it won't have any search though. + +00:42:08.620 --> 00:42:10.160 +So we'll trade out Elasticsearch + +00:42:10.560 --> 00:42:12.840 +and it'll be very difficult to add new data, + +00:42:13.340 --> 00:42:15.020 +but that's okay because it's being archived. + +00:42:15.340 --> 00:42:16.940 +So can we get it into a static site? + +00:42:18.040 --> 00:42:20.940 +And that's challenging depending on how you've set it up. + +00:42:20.980 --> 00:42:26.460 +So we now have projects where we set them up from the beginning to be archivable like this. + +00:42:26.460 --> 00:42:28.960 +And one of them is called Water Stories. + +00:42:29.520 --> 00:42:35.960 +And it was a companion to an art installation at the Radcliffe Institute on the Harvard campus. + +00:42:36.700 --> 00:42:45.380 +And so this was this live site during the duration of the art installation where people could come in and add stories that they had about water onto an iPad. + +00:42:46.380 --> 00:42:47.660 +And then those went up to our database. + +00:42:49.000 --> 00:42:54.340 +we built that with something called Django bakery which if you opt in and you use all of their + +00:42:54.440 --> 00:43:00.480 +class-based views the way that they're meant to be used then you can bake this out into static files + +00:43:00.480 --> 00:43:05.660 +when you're done very low effort that was perfect that is such a cool idea and mad props to them for + +00:43:05.760 --> 00:43:11.440 +ASCII art logos come on now I feel like that should be in the view source if it's not but + +00:43:11.800 --> 00:43:17.260 +this is such a cool idea because you can you can just take a working site you guys are a Django + +00:43:17.260 --> 00:43:22.360 +shop. So you have a lot of your sites are written in Django and you just go make it static, right? + +00:43:22.860 --> 00:43:27.220 +Essentially. Yes. And, and what's, what's, what's really great about it is if they wanted to make + +00:43:27.240 --> 00:43:31.400 +a change and they have, they have asked since we, since we made it static, they've asked for a + +00:43:31.480 --> 00:43:37.020 +couple of changes. So locally, I just Docker compose up this whole application, make the change + +00:43:37.120 --> 00:43:42.420 +in the Django admin and rebake the site. And so it's, it can still be updated. Something, + +00:43:42.600 --> 00:43:46.800 +if you've never tried this, like something like, Hey, can we just add one more menu item? + +00:43:47.140 --> 00:43:50.100 +And you're like, no, no, no, we're not adding the menu item because you want that. + +00:43:50.140 --> 00:43:55.980 +That means we're changing 7,300 pages because they all bake in the whole HTML. + +00:43:56.400 --> 00:43:56.460 +Right? + +00:43:56.700 --> 00:43:57.020 +Exactly. + +00:43:57.560 --> 00:43:57.940 +Yeah, exactly. + +00:43:58.190 --> 00:44:02.740 +But if that's in my, in my Django database and my SQLite file, then no problem at + +00:44:02.840 --> 00:44:04.340 +all because then I just rebake it. + +00:44:04.620 --> 00:44:05.400 +Yeah, yeah, exactly. + +00:44:05.600 --> 00:44:06.100 +Absolutely. + +00:44:06.859 --> 00:44:09.480 +So I think this is super neat. + +00:44:09.560 --> 00:44:12.920 +There's also frozen, frozen flask. + +00:44:13.520 --> 00:44:16.740 +If I could get rid of all the ads, I do not need a Yeti thing, whatever that is. + +00:44:17.200 --> 00:44:24.900 +the glass, not the mythical thing, but frozen flask, which does a similar thing for flask + +00:44:25.300 --> 00:44:30.180 +apps. If you're a flask person probably would work with court. Don't know for sure, but probably. + +00:44:30.520 --> 00:44:36.680 +So that's a pretty interesting idea as well. throw that in there. but also what else? + +00:44:37.460 --> 00:44:45.380 +Also you talked about search, right? That can be, can be such a problem. And I'm a huge fan of your + +00:44:45.320 --> 00:44:50.960 +recommendation here with a page find. Tell us about page find. So this has been, I think it's been a + +00:44:50.960 --> 00:44:56.920 +bit of a game changer in how functional one of these archived sites can remain. So we're actually + +00:44:56.920 --> 00:45:03.360 +in the process of that amendments website that searches across 22,000 full texts of amendments. + +00:45:04.080 --> 00:45:09.580 +We are in the process of sunsetting that, and that will become a static site. And for that search, + +00:45:09.740 --> 00:45:16.020 +we already have an internal demo that proves that we can replace that Postgres full search + +00:45:16.760 --> 00:45:22.280 +with PageFind. You lose vector search. Yeah. You've kind of got to get really + +00:45:22.960 --> 00:45:27.340 +true keyword matching. Yeah. Yeah, that's right. But you still get filtering. I mean, + +00:45:27.360 --> 00:45:34.019 +and really faceting and filtering is when it comes to discovery of things, I mean, I find + +00:45:34.040 --> 00:45:40.360 +that's really what's useful. So filtering these amendments by state or by the Congress that was + +00:45:40.500 --> 00:45:50.160 +active at the time or by the person who co-wrote it. All of those are totally great in PageFind. + +00:45:50.380 --> 00:45:55.240 +And the keyword search is just fine in PageFind. One of the things I really like about it is that + +00:45:55.540 --> 00:46:00.939 +it takes your index and it chops it up into lots of little files that can just fly across the + +00:46:00.960 --> 00:46:06.640 +network. So it's a very fast search. It's not a huge network load, even if your index is + +00:46:07.260 --> 00:46:13.360 +initially very large. And it essentially cuts it up somewhat alphabetically. So if your search + +00:46:14.070 --> 00:46:20.800 +starts with T, or I should say a better word for audio, if it starts with W, then it will load up + +00:46:20.810 --> 00:46:26.000 +the index for words that start with W and fly that over the network instead of the whole thing. + +00:46:26.120 --> 00:46:29.220 +So it's pretty slick and it has a great Python API. + +00:46:29.760 --> 00:46:33.320 +So to do the proof of concept for the amendments search, + +00:46:33.950 --> 00:46:40.000 +I just took a database dump and then manually indexed with a Python script into PageFind. + +00:46:40.180 --> 00:46:42.980 +Wait, there's a Python API for PageFind? + +00:46:43.360 --> 00:46:47.800 +Yeah. So the way PageFind works, I should have said that, is the way most people will use it + +00:46:48.140 --> 00:46:55.160 +is by normally PageFind consumes HTML. So you give it access to your dist folder. + +00:46:56.040 --> 00:46:56.660 +Oh, okay. + +00:46:57.700 --> 00:47:00.360 +And then it crawls through all of your HTML files. + +00:47:00.580 --> 00:47:05.840 +And you can do great things like adding little HTML tags that are just for PageFind, + +00:47:05.940 --> 00:47:09.340 +that give it the filtering ability, or that you want to sort by something. + +00:47:09.620 --> 00:47:10.780 +And so that's great. + +00:47:11.380 --> 00:47:18.280 +Or you can just call PageFind from Python or from TypeScript and just build that index manually. + +00:47:18.660 --> 00:47:19.660 +Well, thanks a lot, David. + +00:47:19.800 --> 00:47:21.160 +I have another thing I've got to go research. + +00:47:21.360 --> 00:47:21.840 +This is awesome. + +00:47:22.560 --> 00:47:24.520 +I'm a huge fan of PageFind, as I said. + +00:47:24.540 --> 00:47:27.040 +on my personal website, mkennedy.codes, + +00:47:27.440 --> 00:47:29.040 +is just a pure stat. + +00:47:29.090 --> 00:47:31.520 +It starts in Markdown and ends up in HTML. + +00:47:31.940 --> 00:47:34.600 +But if you add page find in, you get a super rich, + +00:47:34.780 --> 00:47:36.240 +if you want to just know, you want to talk about, + +00:47:36.360 --> 00:47:37.120 +like what was about Docker, + +00:47:37.590 --> 00:47:39.960 +it shows you really nice results, + +00:47:40.500 --> 00:47:42.060 +pulling out the different parts of the page + +00:47:42.300 --> 00:47:43.480 +and sections that talk about it, + +00:47:43.560 --> 00:47:45.540 +like the headers and then what is said. + +00:47:45.670 --> 00:47:48.520 +And it even does like sub, sub word, + +00:47:48.920 --> 00:47:50.280 +you know, like you just type doc, + +00:47:50.620 --> 00:47:51.980 +it finds all the words that match that. + +00:47:52.160 --> 00:47:54.500 +And what I really like about it is a couple of things + +00:47:54.520 --> 00:47:59.660 +it's instant. It basically is like nearly instant. If you type a few things, it gets way faster + +00:47:59.720 --> 00:48:04.600 +because it's pulling down. And if you go and look in the network console here and you type + +00:48:05.220 --> 00:48:10.540 +something, you can see that it's actually pulling in these little tiny fragments, which this one's + +00:48:10.640 --> 00:48:16.480 +coming off disk cache in three milliseconds, right? But it breaks your index into a bunch of very small + +00:48:16.980 --> 00:48:22.000 +page find fragments that I think it's like, it starts with anything that starts with the word + +00:48:21.980 --> 00:48:24.860 +DO. These are all the prebuilt results and stuff like that. Right. + +00:48:25.080 --> 00:48:26.220 +That's right. That's right. + +00:48:26.440 --> 00:48:27.440 +Yeah. That's super cool. + +00:48:27.940 --> 00:48:34.440 +Yeah. One of our open source projects that, that we maintain is a view of a + +00:48:34.440 --> 00:48:39.780 +view JS component library for page find so that we can style it and reuse it + +00:48:39.810 --> 00:48:40.680 +across different projects. + +00:48:41.040 --> 00:48:42.460 +Oh, that's awesome. I love it. + +00:48:42.780 --> 00:48:44.180 +Yeah. I think this really unlocks it. + +00:48:44.180 --> 00:48:48.800 +And I mean, you go to so many, so many sites, like their documentation or just + +00:48:48.850 --> 00:48:51.440 +their web app in the search is so bad. + +00:48:51.640 --> 00:48:56.680 +You type something and it's like thinking, spinning, spinning, spinning, spinning. + +00:48:57.040 --> 00:49:00.280 +And then like five seconds later, it gives you kind of janky results. + +00:49:00.700 --> 00:49:04.680 +And if you just like throw a page find in there, it's, you can't type fast enough to + +00:49:05.100 --> 00:49:05.760 +outrun the results. + +00:49:05.820 --> 00:49:06.220 +You know what I mean? + +00:49:06.520 --> 00:49:07.100 +No, that's right. + +00:49:07.180 --> 00:49:07.260 +Yeah. + +00:49:07.880 --> 00:49:12.980 +Too many static site search solutions, they use like a, like a JSON blob that you, that + +00:49:12.980 --> 00:49:15.280 +you have to pull down and, and then iterate through. + +00:49:15.940 --> 00:49:16.580 +You know, what's worse. + +00:49:16.680 --> 00:49:21.240 +and I see this a lot, would be if you go to google.com + +00:49:21.820 --> 00:49:24.540 +and then you would say effectively site colon whatever + +00:49:24.790 --> 00:49:26.260 +and then you search Docker, right? + +00:49:26.350 --> 00:49:28.120 +They basically pull that. + +00:49:29.000 --> 00:49:30.520 +You know, they just say search this + +00:49:30.760 --> 00:49:33.560 +and you just get Google results for your site. + +00:49:33.650 --> 00:49:36.460 +And obviously it's, I mean, Google's fine, but it's just. + +00:49:36.600 --> 00:49:38.360 +No, I find that unusable, really. + +00:49:38.460 --> 00:49:38.820 +I do too. + +00:49:38.950 --> 00:49:40.260 +It really, you're like, ah, geez. + +00:49:41.140 --> 00:49:43.220 +But now I'm super excited to realize + +00:49:43.370 --> 00:49:46.220 +I can do that from my dynamic content as well. + +00:49:46.640 --> 00:49:48.460 +So with the Python integration. + +00:49:48.880 --> 00:49:49.760 +OK, nice. + +00:49:51.360 --> 00:49:53.480 +What about something truly static? + +00:49:53.600 --> 00:49:56.440 +Have you looked at Hugo and some of the other type of things? + +00:49:56.880 --> 00:49:57.160 +Sure. + +00:49:57.390 --> 00:50:01.960 +So when I see you've even got the tab up for the SUMEB project, + +00:50:02.300 --> 00:50:08.680 +which is-- that's essentially a database of many, many specimens + +00:50:09.200 --> 00:50:10.440 +taken from the SUMEB mine. + +00:50:11.460 --> 00:50:12.320 +So in the-- + +00:50:12.530 --> 00:50:13.040 +Oh, it is. + +00:50:13.040 --> 00:50:13.740 +Yeah, yeah, it is. + +00:50:13.900 --> 00:50:15.640 +So if you click on Minerals database, + +00:50:16.180 --> 00:50:19.620 +you open up that search interface and that's powered by PageFind. + +00:50:19.760 --> 00:50:20.660 +Oh, this is? + +00:50:21.200 --> 00:50:21.300 +Yes. + +00:50:22.520 --> 00:50:23.740 +I forget what I was... + +00:50:23.930 --> 00:50:24.260 +I see. + +00:50:24.530 --> 00:50:26.640 +You guys even hooked into... + +00:50:26.700 --> 00:50:29.880 +I was thinking just like pure static, like Hugo, like... + +00:50:30.080 --> 00:50:31.540 +Oh, yes. Yes. Yes. + +00:50:31.840 --> 00:50:32.980 +So this is an Astro site. + +00:50:33.360 --> 00:50:37.540 +So for this website, we have this as an Astro site so that we have a little... + +00:50:37.600 --> 00:50:41.520 +Because with Astro, they make it so easy to pull in like view components. + +00:50:42.100 --> 00:50:47.720 +So like our page find is a custom view JS component library with Astro. + +00:50:47.730 --> 00:50:52.620 +You can use React components, you can use the view components, but what it does is it's just + +00:50:52.620 --> 00:50:56.980 +a static site generator. Fantastic. So a little bit more designable + +00:50:57.460 --> 00:51:00.120 +than like Hugo or something. Here's your markdown file. Good luck with that. + +00:51:00.220 --> 00:51:05.020 +Yeah. I love Hugo though. Yeah. I use Hugo for different personal sites here and there, + +00:51:05.070 --> 00:51:08.420 +and it's just so fast and easy to get up and running. But yeah, it's great. + +00:51:08.440 --> 00:51:09.400 +- Great, great when it's a good friend. + +00:51:09.400 --> 00:51:10.740 +- That's what my website's written in, it's in Hugo. + +00:51:12.239 --> 00:51:14.280 +But if I'm integrating with anything else, + +00:51:14.400 --> 00:51:15.740 +I used to kind of like split it up, + +00:51:15.790 --> 00:51:17.920 +like this part's Hugo and this part's like a Python app. + +00:51:17.920 --> 00:51:20.000 +And it's pretty easy to get something + +00:51:20.140 --> 00:51:21.620 +that'll take a bunch of markdown files + +00:51:21.820 --> 00:51:23.200 +and just turn them into HTML + +00:51:23.700 --> 00:51:25.400 +and just put a page template around that. + +00:51:25.580 --> 00:51:29.000 +So I've kind of stepped away from mixing and matching that + +00:51:29.140 --> 00:51:29.960 +as much as I used to. + +00:51:30.230 --> 00:51:32.940 +So now if I got a static section of a dynamic site, + +00:51:33.400 --> 00:51:34.000 +but that doesn't address, + +00:51:34.140 --> 00:51:37.780 +has nothing to do with the archival side of things, right? + +00:51:38.440 --> 00:51:41.840 +Because the idea is that the thing that I'm describing is gone on purpose. + +00:51:42.180 --> 00:51:42.600 +That's right. + +00:51:42.840 --> 00:51:45.980 +So you've got some, we've got Django Bakery. + +00:51:46.440 --> 00:51:52.580 +I threw out Frozen Flask, and I'm sure there's a ton more that neither of us are aware of at the moment. + +00:51:52.800 --> 00:51:56.380 +So Django Bakery was really good for that purpose. + +00:51:56.640 --> 00:52:00.600 +And we're keeping our eyes open for projects that it's a good fit for. + +00:52:01.560 --> 00:52:03.420 +But that was a pretty simple website. + +00:52:03.620 --> 00:52:06.260 +It needed a dynamic backend, but it was quite straightforward. + +00:52:06.960 --> 00:52:09.860 +And for Django Bakery, you have to opt into inheriting + +00:52:10.080 --> 00:52:11.520 +from their class-based views. + +00:52:11.580 --> 00:52:11.840 +I see. + +00:52:12.700 --> 00:52:13.800 +So if you're doing, for example-- + +00:52:13.800 --> 00:52:14.880 +You've got to dig ahead of it, yeah. + +00:52:15.260 --> 00:52:16.680 +Yeah, yeah, yeah, absolutely. + +00:52:17.000 --> 00:52:18.640 +Yeah, hard to add retroactively. + +00:52:18.780 --> 00:52:19.380 +Probably impossible. + +00:52:20.340 --> 00:52:23.120 +Now, our other websites, like the fin example + +00:52:23.380 --> 00:52:27.060 +and the mapping color example, those are APIs. + +00:52:27.500 --> 00:52:29.800 +That's a Django API, Django REST framework for one, + +00:52:30.700 --> 00:52:31.920 +GraphQL for the other. + +00:52:32.540 --> 00:52:34.560 +One has a view front end, one has a React front end. + +00:52:34.900 --> 00:52:36.920 +OK, well, Django Bakery just isn't + +00:52:36.940 --> 00:52:39.580 +isn't going to work very well for like serializing JSON. + +00:52:39.760 --> 00:52:40.680 +Yeah, it's like awesome. + +00:52:40.940 --> 00:52:44.080 +Here's your unrendered JavaScript front end code + +00:52:44.180 --> 00:52:45.560 +and it's just going to look empty or something. + +00:52:45.980 --> 00:52:46.060 +Yeah. + +00:52:46.400 --> 00:52:48.800 +So it is a good reason to consider using + +00:52:49.680 --> 00:52:51.460 +like vanilla Django templates when possible, + +00:52:52.440 --> 00:52:53.220 +like for that reason. + +00:52:53.440 --> 00:52:57.880 +But those were, those were inherited from the vendors, + +00:52:58.880 --> 00:52:59.420 +those two sites. + +00:52:59.440 --> 00:53:00.960 +And we've made a lot of progress on those. + +00:53:01.520 --> 00:53:04.740 +So, you know, what, what to do in that, + +00:53:05.000 --> 00:53:10.360 +like in that situation, Django Bakery isn't an option. And those projects are not end of life + +00:53:10.600 --> 00:53:14.960 +yet. So we have some time, but we're, we're, we're, so what we're doing is strategizing, okay, + +00:53:15.280 --> 00:53:20.720 +how will we rescue them? How will we keep them alive once, once somebody needs to stop paying + +00:53:20.880 --> 00:53:25.620 +for hosting? And we have, we have ideas. We have, I think there's, there's clever, interesting + +00:53:26.060 --> 00:53:34.900 +things out there. We'll have to keep looking into it. There are some pretty interesting ideas. And + +00:53:34.920 --> 00:53:41.020 +that ran in a container, you could just have WebAssembly, but still have it go, right? + +00:53:41.140 --> 00:53:42.780 +Sort of a local loopback type of thing. + +00:53:43.000 --> 00:53:50.640 +Yeah, I'm really interested in this one because it enables essentially the full functionality + +00:53:51.140 --> 00:53:54.960 +of the live site to exist as what is just a static site. + +00:53:55.640 --> 00:54:03.160 +So because of Pyodide and projects like PyScript, we can run Python in the browser and we can + +00:54:03.120 --> 00:54:09.220 +run SQLite in the browser. And now we can even run Postgres in the browser with PG Lite. So if + +00:54:09.300 --> 00:54:15.320 +we can run all those things in the browser, then couldn't we have Django hosted right in the browser? + +00:54:15.880 --> 00:54:22.320 +And you can. So there's a proof of concept that proves it's possible called Django WebAssembly. + +00:54:23.360 --> 00:54:29.940 +And if you load this up, it'll let you log in to the Django admin. And you're not logging into + +00:54:29.960 --> 00:54:36.380 +anybody's backend, you're logging into your own browser where this is running in a service worker. + +00:54:36.680 --> 00:54:40.280 +Awesome. Look at that. Oh, hold on. I told me what the password was. Very secure. + +00:54:40.860 --> 00:54:41.940 +Matt, password. + +00:54:42.220 --> 00:54:47.000 +Well, it can be entirely insecure because, yeah, you're just, it's running right in your own browser. + +00:54:47.300 --> 00:54:50.080 +Yeah, that's awesome. And here we are, Django admin. Incredible. + +00:54:50.480 --> 00:54:55.020 +Yeah, so I'm pretty interested in this. You've got to convert an RDS Postgres database + +00:54:55.640 --> 00:54:59.640 +into either SQLite or something like PGLite, but I think that's all doable. + +00:54:59.980 --> 00:55:01.920 +So I think it's an exciting possibility. + +00:55:02.340 --> 00:55:02.940 +Yeah, for sure. + +00:55:03.010 --> 00:55:06.860 +I do think, so maybe you have a rich query system + +00:55:07.030 --> 00:55:08.140 +that you're powering by your database + +00:55:08.480 --> 00:55:09.040 +that's really heavy. + +00:55:09.480 --> 00:55:09.840 +Exactly. + +00:55:10.120 --> 00:55:11.680 +And it's got a bunch of data that's like, + +00:55:11.720 --> 00:55:13.500 +here's all of our working data + +00:55:13.620 --> 00:55:14.740 +that you might ask questions about. + +00:55:15.060 --> 00:55:16.920 +Maybe you just convert that to page find + +00:55:17.580 --> 00:55:18.540 +to help you find the pieces + +00:55:18.960 --> 00:55:20.500 +and then just keep the operational data + +00:55:20.720 --> 00:55:23.300 +and maybe like even a SQLite with like the Django RRM, + +00:55:23.300 --> 00:55:25.600 +you can just switch the connection, keep talking to it. + +00:55:25.750 --> 00:55:26.900 +I mean, there's possibilities + +00:55:27.050 --> 00:55:28.900 +to just get something not too terrible + +00:55:28.920 --> 00:55:30.740 +Well, it's not the same, but not that far off. + +00:55:31.080 --> 00:55:31.680 +Yeah, exactly. + +00:55:32.190 --> 00:55:35.420 +And then it goes on GitHub pages and it can live hopefully forever. + +00:55:35.700 --> 00:55:40.300 +I mean, it feels like GitHub will last forever, but it'll last longer than funding will anyways. + +00:55:41.120 --> 00:55:48.380 +It's definitely going to last longer than just something that we can't pay for anymore, right? + +00:55:48.520 --> 00:55:53.900 +I don't know how long GitHub's going to be around for, I think a while, but you never know, right? + +00:55:53.960 --> 00:55:57.400 +It seems like stuff's going to last forever, then it gets changed. + +00:55:57.520 --> 00:55:58.180 +We had subversion. + +00:55:59.000 --> 00:56:00.480 +Now it's completely gone, right? + +00:56:00.800 --> 00:56:04.780 +Just 20 years, 15 years later, but still, I think 100% there. + +00:56:05.020 --> 00:56:05.260 +Yeah. + +00:56:05.580 --> 00:56:09.520 +But if somebody can, if something ever happened, somebody just needs to copy that, + +00:56:09.750 --> 00:56:15.800 +that folder of HTML, CSS and JavaScript files and dump it into an S3 bucket or somewhere else. + +00:56:15.950 --> 00:56:17.360 +And then it can continue living there. + +00:56:17.860 --> 00:56:18.800 +So it's a good option. + +00:56:19.440 --> 00:56:20.020 +It's a great option. + +00:56:20.320 --> 00:56:21.400 +It's a really, really good option. + +00:56:21.660 --> 00:56:30.940 +I mean, I guess one of the long-term concerns might be what if the WebAssembly standard changes so much that it's not supported anymore? + +00:56:31.520 --> 00:56:36.860 +But you could probably bite-wise convert it if you had to, you know, like somebody would probably be able to create one. + +00:56:37.240 --> 00:56:38.560 +Yeah, that would be unfortunate. + +00:56:39.060 --> 00:56:48.860 +So I suppose if that happens, I mean, if that happens, yeah, we're booting up one of these projects is like booting up an emulator for some old DOS game. + +00:56:49.060 --> 00:56:49.540 +Right, right. + +00:56:49.720 --> 00:56:52.320 +Well, I mean, I guess let's think about this for a second. + +00:56:52.840 --> 00:56:55.460 +Somebody got, oh gosh, what was the chain? + +00:56:55.510 --> 00:57:03.180 +This is the whole, JavaScript, the PyCon talk where got like Firefox + +00:57:04.280 --> 00:57:10.080 +compiled into, not WASM, into, ASM JS or something like that. + +00:57:10.250 --> 00:57:14.300 +So it was run like Chrome was running Firefox, which was running, I think + +00:57:14.620 --> 00:57:17.060 +doom, which was also ASM JS. + +00:57:17.940 --> 00:57:21.800 +If we can do that, we could get something that would run, that would read old Web + +00:57:22.000 --> 00:57:24.540 +Assembly into new WebAssembly if it really mattered to the world. + +00:57:24.860 --> 00:57:25.180 +Absolutely. + +00:57:25.800 --> 00:57:25.980 +Yeah. + +00:57:26.240 --> 00:57:30.380 +Especially if it's in a public repo that people who care about the data can, + +00:57:30.680 --> 00:57:31.560 +can rescue it somehow. + +00:57:31.980 --> 00:57:32.080 +Yeah. + +00:57:32.420 --> 00:57:34.040 +What about like a virtual machine? + +00:57:34.500 --> 00:57:35.140 +You know, I agree. + +00:57:35.220 --> 00:57:35.640 +Yeah, absolutely. + +00:57:36.440 --> 00:57:42.220 +Could have saved me some, take a snapshot of Ubuntu LTS, some version, + +00:57:42.420 --> 00:57:43.600 +and just what are we going to do? + +00:57:44.200 --> 00:57:46.000 +Everything we do is Dockerized. + +00:57:46.400 --> 00:57:47.320 +Everything is in a container. + +00:57:47.780 --> 00:57:51.900 +So in the worst case scenario, we could give somebody the image, and they could run it if + +00:57:51.910 --> 00:57:52.420 +they have Docker. + +00:57:53.310 --> 00:57:57.780 +I think that's a nice peace of mind to know that no matter what, something will be able + +00:57:57.790 --> 00:57:59.040 +to run this container. + +00:57:59.440 --> 00:58:03.000 +And even in, I don't know if you've used GitHub, what is it called, Codespaces. + +00:58:05.319 --> 00:58:06.680 +I archived one project. + +00:58:07.570 --> 00:58:12.740 +It was kind of dramatic and sudden that it needed to be archived, so without much time + +00:58:12.850 --> 00:58:13.320 +to do anything. + +00:58:13.500 --> 00:58:15.460 +And it was a Ruby on Rails project. + +00:58:15.680 --> 00:58:18.220 +And I'm not a Rails developer, but I + +00:58:18.260 --> 00:58:19.600 +was able to get it archived in a way + +00:58:19.780 --> 00:58:22.620 +that anybody could, with one command, + +00:58:23.300 --> 00:58:27.040 +go to the repo on GitHub and boot it up in Codespaces + +00:58:27.440 --> 00:58:30.540 +and then have it live running from their Codespace. + +00:58:30.540 --> 00:58:31.800 +And so that works too. + +00:58:32.040 --> 00:58:32.380 +Very cool. + +00:58:32.600 --> 00:58:35.120 +I think as WebAssembly grows, there'll + +00:58:35.120 --> 00:58:38.200 +be more possibilities for these types of things. + +00:58:38.600 --> 00:58:39.300 +Yeah, amazing. + +00:58:39.660 --> 00:58:42.640 +I'm pretty excited about PageFind having a Python API. + +00:58:42.900 --> 00:58:46.440 +didn't realize that. So I'm going to be doing something with that for sure. So what else? + +00:58:46.960 --> 00:58:51.180 +Let me ask you one more thing before I kind of let you wrap up with some final thoughts here. + +00:58:51.620 --> 00:58:58.300 +What about AI? Oh, that's a good question. So AI, I mean, there's like, in my story, + +00:58:58.660 --> 00:59:04.580 +there's like one interesting part of AI, which is that I got started and self-learned everything I + +00:59:04.660 --> 00:59:10.840 +needed to about software development to begin doing this right before ChatGPT really came on + +00:59:10.860 --> 00:59:17.240 +was able to do real programming yeah you're like four years of legit programming before right so i + +00:59:17.380 --> 00:59:21.320 +think i mean so i was thinking i was thinking when i was thinking about how i got into it i thought + +00:59:21.660 --> 00:59:28.500 +what if i was four years later starting my phd and wanting to do these tools um i would have been + +00:59:28.570 --> 00:59:34.580 +able to accomplish what i needed to for my research without acquiring the technical skills and that + +00:59:34.610 --> 00:59:38.140 +would have been that's a good thing i'm not sure if that's good about it it could be both i would + +00:59:37.980 --> 00:59:43.220 +would have thought it was a good thing. I would have thought it's a good thing. But in my hands + +00:59:43.740 --> 00:59:52.220 +now, like a software engineer, AI is more powerful in my hands now than it would have been then. + +00:59:52.610 --> 00:59:57.560 +So I can make it work for me. Yeah, I can make it work for me in a way that I couldn't have been + +00:59:57.610 --> 01:00:01.980 +able to then. So I'm thankful for that, but it's something I think of. I don't want to say it's + +01:00:02.800 --> 01:00:07.940 +necessarily a bad thing, but it definitely marks a difference, a difference in time between other + +01:00:07.960 --> 01:00:13.120 +people who are maybe wanting to get into digital humanities, they're humanities researchers. They + +01:00:13.140 --> 01:00:17.740 +want to add some digital tools. You know, I think this will kind of, this will probably knock people + +01:00:18.040 --> 01:00:22.280 +off of the more technical path because it's not needed. I think it will too. And I think that that + +01:00:22.460 --> 01:00:27.640 +might be a negative. When you were telling me your story originally, I was thinking kind of like, + +01:00:27.760 --> 01:00:32.740 +how neat is it that you didn't sign up for, and the people you're working with probably didn't + +01:00:32.760 --> 01:00:36.300 +intend to sign you up for learning true software development. + +01:00:36.820 --> 01:00:41.000 +But look at this cool and interesting job that you now have that you never + +01:00:41.160 --> 01:00:41.880 +would have imagined. + +01:00:42.000 --> 01:00:44.400 +I'm sure when you signed up for your PhD, you're like, you know what I'm + +01:00:44.400 --> 01:00:47.320 +going to do when I get my PhD, I'm going to go X, Y, like, I'm going to + +01:00:47.400 --> 01:00:48.020 +join the Darth program. + +01:00:48.120 --> 01:00:49.780 +Like, no, probably not. + +01:00:49.900 --> 01:00:50.000 +Right. + +01:00:50.120 --> 01:00:50.760 +But here you are. + +01:00:51.380 --> 01:00:54.880 +And I think that's actually a really interesting knock on effect for a lot + +01:00:54.960 --> 01:00:59.040 +of researchers and people in grad schools, they're kind of put into this + +01:00:59.660 --> 01:01:01.020 +programming adjacent type of thing. + +01:01:01.400 --> 01:01:04.740 +You know, and a lot of folks sort of are like, actually, that's pretty interesting. + +01:01:04.940 --> 01:01:06.160 +I'm going to kind of lean into that. + +01:01:06.490 --> 01:01:10.300 +And I think AI might knock, like you said, knock people off that path to some degree. + +01:01:11.100 --> 01:01:11.720 +Yeah, yeah, definitely. + +01:01:12.210 --> 01:01:14.700 +So that's just like one part of the AI story. + +01:01:15.050 --> 01:01:17.900 +The other one is that, like how we use it. + +01:01:18.840 --> 01:01:25.540 +It's great for data extraction, pulling data out of different, you know, to make these + +01:01:25.890 --> 01:01:30.100 +search interfaces more powerful, to extract different data from them. + +01:01:30.540 --> 01:01:33.000 +That's just one example where it's been handy. + +01:01:33.800 --> 01:01:38.180 +We're looking for ways that it can really empower faculty. + +01:01:39.160 --> 01:01:47.460 +We're still very much in the exploration phase of how we can use it and provide it to faculty as a digital humanities tool. + +01:01:48.220 --> 01:01:52.240 +Sure. I was thinking pretty much when I asked the question of it, it's just like two parts. + +01:01:52.400 --> 01:01:56.300 +One, how is it? Are you guys using it to help take projects? + +01:01:56.440 --> 01:01:58.320 +Well, that would have been a month. No, actually, it's three days. + +01:01:58.820 --> 01:01:59.260 +You know what I mean? + +01:02:00.300 --> 01:02:05.840 +that. And then if people are asking, you know, a professor comes along and says, and we want our + +01:02:05.930 --> 01:02:12.880 +own custom AI thing, or we're using Harvard's internal one that we're allowed to use, but we + +01:02:13.040 --> 01:02:17.600 +won't be able to use it once the grant runs out. You know what I mean? Yeah. Yeah. I think one, + +01:02:17.820 --> 01:02:23.280 +one good example of this type of thing is that what we're starting to get is faculty who are + +01:02:23.780 --> 01:02:28.180 +vibe coding and now, and we are going to teach them. We're going to teach them how to do it. + +01:02:28.540 --> 01:02:30.780 +You know, instead of having them. + +01:02:31.200 --> 01:02:32.500 +Yeah, it's absolutely a skill. + +01:02:32.900 --> 01:02:33.500 +Yeah, no, it is. + +01:02:33.720 --> 01:02:34.040 +It is. + +01:02:34.800 --> 01:02:43.200 +Instead of copy and pasting from ChatGPT into VS Code, having them learn Copilot, maybe even having them download Cursor. + +01:02:43.600 --> 01:02:48.320 +Download some real dedicated tools to get this done to make them more productive. + +01:02:48.780 --> 01:02:52.860 +So, yeah, educating about how to do it is one thing. + +01:02:53.200 --> 01:02:54.240 +You asked if we're using it. + +01:02:54.900 --> 01:02:58.000 +We have access to Copilot. + +01:02:58.980 --> 01:03:04.140 +and that's great. I can't say that we've shipped anything in three days instead of a month yet, + +01:03:04.780 --> 01:03:13.440 +but one anecdote is that right now I'm doing some really interesting processing of music audio files, + +01:03:13.940 --> 01:03:19.500 +and somebody asked to have a beatboxer if I could chop that file up so that all of the individual + +01:03:19.820 --> 01:03:26.440 +sounds that the beatboxer makes are identified in a file. And so I'm using some music libraries, + +01:03:26.840 --> 01:03:32.000 +Python library called Librosa. There's some complicated math in there. It's a little bit + +01:03:32.040 --> 01:03:36.160 +too much for me. It's no problem for Claude. Claude knows how to do that math. And then, + +01:03:36.720 --> 01:03:39.580 +and I use my expertise to string it together to get a good output. + +01:03:39.940 --> 01:03:44.500 +Yeah. Awesome. You got time for one more quick question before we'll clap things up. + +01:03:44.500 --> 01:03:44.660 +For sure. + +01:03:45.300 --> 01:03:51.160 +Raymond out there, Raymond Yees asks, it says, it'd be good to hear how Harvard uses containers on AWS + +01:03:51.840 --> 01:03:56.060 +and its reliability. It's reliable, not cheapest way to host things. Are you thinking about moving + +01:03:56.380 --> 01:04:02.480 +moving that or is it not that much? Okay, I'll tell you about a failed experiment. + +01:04:03.520 --> 01:04:11.180 +We were using ECS and we're still using ECS. So that's AWS's main, you know, it's not Kubernetes, + +01:04:11.560 --> 01:04:17.840 +but it's one step down with their horizontal scaling container clusters. And I wanted to move + +01:04:17.840 --> 01:04:23.580 +us onto a single EC2 instance because our projects are popular, but they're not so popular that we + +01:04:23.500 --> 01:04:25.580 +actually have to worry about horizontal scaling. + +01:04:25.860 --> 01:04:26.120 +Right. + +01:04:26.220 --> 01:04:29.760 +It's not like it's front page in New York Times. + +01:04:30.280 --> 01:04:31.300 +I guess it probably could be. + +01:04:31.460 --> 01:04:34.300 +But even so, for the static sites, they probably still can take it. + +01:04:35.300 --> 01:04:35.380 +Yeah. + +01:04:35.640 --> 01:04:42.180 +So I priced it out and I got an example deployed, an example project deployed, and was able + +01:04:42.180 --> 01:04:44.860 +to confirm that it would indeed be much cheaper. + +01:04:45.940 --> 01:04:48.780 +And it was deployed in a similar way using AWS CDK. + +01:04:49.020 --> 01:04:51.540 +So it's all infrastructure is code all the way down. + +01:04:52.080 --> 01:04:54.680 +But it turns out there's all kinds of compliance. + +01:04:54.970 --> 01:04:58.300 +When you are in charge of the VM at like a big university, + +01:04:58.630 --> 01:05:00.580 +or I'm sure any corporate setting, + +01:05:00.980 --> 01:05:03.920 +if you are in charge of the VM and the OS on it, + +01:05:04.220 --> 01:05:07.260 +then you have to know that you have the latest patches in. + +01:05:07.460 --> 01:05:08.920 +You have to know that you have latest Ubuntu. + +01:05:09.490 --> 01:05:10.960 +And then there's other things, + +01:05:12.460 --> 01:05:13.860 +different observability things + +01:05:13.860 --> 01:05:14.740 +that you have to have in place + +01:05:15.900 --> 01:05:17.600 +that are not usually required + +01:05:17.880 --> 01:05:20.700 +if you're running in a container cluster like ECS. + +01:05:21.480 --> 01:05:27.700 +So it ends up being a lot less work and much easier to achieve compliance if we run containers + +01:05:28.120 --> 01:05:31.120 +or some other serverless thing. + +01:05:31.440 --> 01:05:37.160 +If I run all my personal projects, they all run in a single virtual machine, but we're + +01:05:37.280 --> 01:05:37.800 +running in containers. + +01:05:38.340 --> 01:05:38.560 +Yeah. + +01:05:38.560 --> 01:05:38.660 +Yeah. + +01:05:39.300 --> 01:05:42.260 +And you've got all the SOC 2 stuff and all those different things, right? + +01:05:42.320 --> 01:05:43.380 +Like there's layers. + +01:05:43.940 --> 01:05:44.440 +Yeah, that's right. + +01:05:44.740 --> 01:05:44.800 +Yeah. + +01:05:44.920 --> 01:05:50.300 +I mean, I'll mention that, but what I didn't say is that in that 2019, when I started learning + +01:05:50.520 --> 01:05:55.520 +Python. I discovered Talk Python almost immediately. And one of the first episodes that I listened to + +01:05:55.520 --> 01:06:01.060 +was the other digital humanities. Cornelius Van Litt. He was an awesome guest. + +01:06:01.260 --> 01:06:06.220 +That's right. Yeah. And I thought that was great. And that was also a bit about manuscripts, + +01:06:06.820 --> 01:06:11.760 +a little bit more on the image side than the text side. And I didn't understand everything + +01:06:11.790 --> 01:06:15.880 +that everybody was saying, but I just, I kept tuning in. And I think because of that, + +01:06:16.120 --> 01:06:21.660 +Because Talk Python was like this, you know, I've been remote working for most of my time. + +01:06:22.400 --> 01:06:27.000 +And Talk Python has been kind of like that conversation with the open source community + +01:06:27.700 --> 01:06:28.920 +that's been always in my ear. + +01:06:28.920 --> 01:06:33.340 +And I think that made, you know, a difference, making me feel like I understood the software + +01:06:34.060 --> 01:06:37.420 +landscape and like the developer culture and what was going on. + +01:06:37.640 --> 01:06:40.900 +And then the different Python libraries and what was possible. + +01:06:41.640 --> 01:06:47.280 +So to people who are interested in taking things in a more technical direction, I think + +01:06:47.280 --> 01:06:52.560 +it's helpful just to find a few things like that, that give you an insight into that world. + +01:06:53.020 --> 01:06:59.060 +And the more you listen to it, the more you start to hear the same acronyms and the same + +01:06:59.360 --> 01:07:02.640 +things said enough that you start to feel like, okay, now you're part of the club. + +01:07:03.000 --> 01:07:04.360 +I really appreciate that. + +01:07:05.180 --> 01:07:05.580 +That's cool. + +01:07:06.080 --> 01:07:09.780 +I've certainly had people reach out to me and say things that at first didn't make any + +01:07:09.940 --> 01:07:10.240 +sense to me. + +01:07:10.360 --> 01:07:12.200 +Like I've been listening for six weeks now + +01:07:12.400 --> 01:07:14.540 +and it's starting to make sense what you're talking about. + +01:07:14.540 --> 01:07:15.980 +Like, why have you been listening for six months + +01:07:16.030 --> 01:07:16.800 +when it made no sense? + +01:07:16.940 --> 01:07:17.420 +That's insane. + +01:07:17.680 --> 01:07:20.880 +But a lot of people use listening to the podcast, + +01:07:21.070 --> 01:07:24.500 +is it mine and others, as language immersion, right? + +01:07:24.640 --> 01:07:28.380 +Like I could get Duolingo and I could learn Portuguese + +01:07:28.720 --> 01:07:30.580 +or I could move to Brazil for a month. + +01:07:30.830 --> 01:07:31.380 +You know what I mean? + +01:07:31.580 --> 01:07:32.200 +And then I would really learn. + +01:07:32.200 --> 01:07:32.480 +- Yeah, exactly. + +01:07:33.160 --> 01:07:33.460 +- Right. + +01:07:34.000 --> 01:07:34.140 +- Exactly. + +01:07:34.270 --> 01:07:36.040 +No, I think there's truth to that. + +01:07:36.260 --> 01:07:38.660 +And some of the things I did was, you know, + +01:07:38.820 --> 01:07:42.920 +search through, like search the word deployment, because I'm trying to get my head around how to + +01:07:43.020 --> 01:07:47.000 +deploy for the first time. And I just want to hear people talk about it. Like I could read about it. + +01:07:47.000 --> 01:07:52.120 +I could read the tutorial, but I just want to hear people talk about deployment to get a sense of what + +01:07:52.300 --> 01:07:56.480 +actual deployment sounds like. There's something really different when you're learning or trying, + +01:07:57.240 --> 01:08:01.380 +even you're maybe an experienced programmer, but not in this particular area to hear a human + +01:08:01.840 --> 01:08:08.500 +side of it, not just the docs, not a sterile. These are the four steps, but like, I love it. + +01:08:08.700 --> 01:08:10.080 +I mean, it's probably why I created the show. + +01:08:10.280 --> 01:08:11.680 +It's because I didn't hear those stories. + +01:08:11.780 --> 01:08:12.940 +We got to tell those stories. + +01:08:13.440 --> 01:08:13.540 +Awesome. + +01:08:13.860 --> 01:08:14.660 +I appreciate that. + +01:08:14.860 --> 01:08:15.620 +So super cool. + +01:08:15.840 --> 01:08:16.020 +All right. + +01:08:16.359 --> 01:08:21.080 +So if other people are listening, maybe one of your pieces of advice is keep listening. + +01:08:21.580 --> 01:08:22.299 +You'll get there. + +01:08:22.480 --> 01:08:22.859 +Yeah. + +01:08:22.960 --> 01:08:30.060 +And if anybody is in the humanities and somehow found their way onto this episode with no technical experience, + +01:08:30.819 --> 01:08:37.060 +I just would give the caution of, like, you know, the anecdote that if AI coding had been + +01:08:37.259 --> 01:08:42.940 +around the way it is now when I was learning, I wouldn't be doing digital humanities at + +01:08:43.060 --> 01:08:43.200 +Harvard. + +01:08:43.540 --> 01:08:45.600 +I wouldn't have been able to get into this field. + +01:08:46.420 --> 01:08:47.420 +I wouldn't have known about it. + +01:08:47.799 --> 01:08:52.380 +So I guess just think about that when you're learning and applying new tools. + +01:08:52.720 --> 01:08:54.980 +I don't really know what the right fix for that is. + +01:08:55.060 --> 01:08:56.299 +That's a very challenging problem. + +01:08:56.500 --> 01:08:59.560 +I mean, you can say I'm just literally not going to fire it up. + +01:08:59.720 --> 01:09:03.279 +But I mean, we used to hunt through Stack Overflow and the web and over and over. + +01:09:03.460 --> 01:09:06.859 +And if you're really stuck or you really don't understand, like they're good at explaining + +01:09:06.960 --> 01:09:07.319 +stuff too. + +01:09:07.359 --> 01:09:12.200 +You just got to really stay in a learner's mindset, not just press the easy button and + +01:09:12.319 --> 01:09:13.259 +make this thing and move on. + +01:09:13.700 --> 01:09:14.380 +Easier said than done. + +01:09:14.680 --> 01:09:15.359 +Easier said than done. + +01:09:15.620 --> 01:09:22.000 +So yeah, I want to leave this with kind of a thought about how much things like Python + +01:09:22.220 --> 01:09:27.260 +and these tools and technology can really empower stuff that you wouldn't think is even + +01:09:27.279 --> 01:09:34.620 +related, like understanding old manuscripts and how painting is connected or changed over time and + +01:09:34.799 --> 01:09:39.720 +stuff, right? Those sound very much disjointed from tech and software, but they really are + +01:09:40.080 --> 01:09:45.319 +superpowers that you can bring to your work, whatever your industry is. I know our field of + +01:09:45.460 --> 01:09:49.600 +study, I know there's some sociologists out in the audience and I'm sure others as well. + +01:09:50.279 --> 01:09:54.700 +All right. Final thoughts, David, close it out. You said it great. I mean, you know, + +01:09:55.340 --> 01:10:01.840 +Just applying these technical tools to old questions, that is the core of digital humanities. + +01:10:02.220 --> 01:10:04.900 +When I first started hearing about this, I thought, I really don't know how this ties + +01:10:05.060 --> 01:10:05.160 +together. + +01:10:05.400 --> 01:10:08.780 +And after seeing it a few times, I definitely see the power of it. + +01:10:08.780 --> 01:10:11.000 +And I thank you for your time coming on. + +01:10:11.260 --> 01:10:16.760 +Thank you for sharing your look and the look inside of your team and inside of a small piece + +01:10:16.940 --> 01:10:17.260 +of Harvard. + +01:10:17.780 --> 01:10:22.960 +I really like these kinds of episodes because it's hard to see this from the outside, right? + +01:10:23.060 --> 01:10:24.880 +like you just see the results, + +01:10:24.950 --> 01:10:27.180 +but you don't see like the inner workings of the team + +01:10:27.320 --> 01:10:28.140 +and the motivation and stuff. + +01:10:28.360 --> 01:10:30.640 +So thank you so much for being here. + +01:10:31.150 --> 01:10:32.480 +And yeah, bye everyone. + +01:10:33.980 --> 01:10:36.100 +This has been another episode of Talk Python To Me. + +01:10:36.370 --> 01:10:37.200 +Thank you to our sponsors. + +01:10:37.390 --> 01:10:38.700 +Be sure to check out what they're offering. + +01:10:38.940 --> 01:10:40.260 +It really helps support the show. + +01:10:40.720 --> 01:10:42.100 +Take some stress out of your life. + +01:10:42.480 --> 01:10:44.280 +Get notified immediately about errors + +01:10:44.640 --> 01:10:46.440 +and performance issues in your web + +01:10:46.450 --> 01:10:47.920 +or mobile applications with Sentry. + +01:10:48.440 --> 01:10:51.300 +Just visit talkpython.fm/sentry + +01:10:51.800 --> 01:10:52.860 +and get started for free. + +01:10:53.280 --> 01:10:55.800 +Be sure to use our code, talkpython26. + +01:10:56.760 --> 01:11:00.140 +That's Talk Python, the numbers two, six, all one word. + +01:11:00.820 --> 01:11:02.920 +This episode is brought to you by CommandBook, + +01:11:03.240 --> 01:11:05.320 +a native macOS app that I built + +01:11:05.480 --> 01:11:08.040 +that gives long-running terminal commands a permanent home. + +01:11:08.440 --> 01:11:10.440 +No more juggling six terminal tabs every morning. + +01:11:10.880 --> 01:11:12.280 +Carefully craft a command once, + +01:11:12.440 --> 01:11:14.020 +run it forever with auto-restart, + +01:11:14.160 --> 01:11:15.700 +URL detection, and a full CLI. + +01:11:16.060 --> 01:11:19.180 +Download it for free at talkpython.fm/commandbook app. + +01:11:19.920 --> 01:11:21.800 +If you or your team needs to learn Python, + +01:11:22.040 --> 01:11:32.080 +We have over 270 hours of beginner and advanced courses on topics ranging from complete beginners to async code, Flask, Django, HTML, and even LLMs. + +01:11:32.400 --> 01:11:34.580 +Best of all, there's no subscription in sight. + +01:11:35.240 --> 01:11:36.900 +Browse the catalog at talkpython.fm. + +01:11:37.600 --> 01:11:42.260 +And if you're not already subscribed to the show on your favorite podcast player, what are you waiting for? + +01:11:42.900 --> 01:11:44.700 +Just search for Python in your podcast player. + +01:11:44.790 --> 01:11:45.680 +We should be right at the top. + +01:11:46.100 --> 01:11:48.940 +If you enjoy that geeky rap song, you can download the full track. + +01:11:49.070 --> 01:11:50.980 +The link is actually in your podcast blur show notes. + +01:11:51.760 --> 01:11:53.140 +This is your host, Michael Kennedy. + +01:11:53.560 --> 01:11:54.600 +Thank you so much for listening. + +01:11:54.830 --> 01:11:55.620 +I really appreciate it. + +01:11:56.040 --> 01:11:56.760 +I'll see you next time. + +01:12:08.400 --> 01:12:11.200 +I'm out. + diff --git a/transcripts/539-catching-up-with-the-python-typing-council.txt b/transcripts/539-catching-up-with-the-python-typing-council.txt new file mode 100644 index 0000000..1714687 --- /dev/null +++ b/transcripts/539-catching-up-with-the-python-typing-council.txt @@ -0,0 +1,2360 @@ +00:00:00 You're adding type-ins to your Python code. + +00:00:02 Your editor is happy, autocomplete is working great, but then you switch tools + +00:00:06 and suddenly there are red squigglies everywhere. + +00:00:09 Who decides what a float annotation actually means or whether passing none where an int is expected + +00:00:15 should be an error? + +00:00:17 It turns out there's a five-person council dedicated to exactly these questions + +00:00:21 and two brand new Rust-based type checkers are raising the bar as well. + +00:00:27 On this episode, I sit down with three of the members of the Python Typing Council, + +00:00:31 Yela Zylstra, Rebecca Chen, and Carl Meyer to learn about how the type system is governed, + +00:00:37 where the spec and type checkers agree and disagree, and I get the council's official advice + +00:00:42 on how much typing is just enough. + +00:00:45 This is Talk Python To Me, episode 539, recorded January 27th, 2026. + +00:00:53 Talk Python To Me, yeah, we ready to roll. + +00:00:56 Upgrading the code, no fear of getting old. + +00:00:59 Async in the air, new frameworks in sight, geeky rap on deck. + +00:01:03 Quart crew, it's time to unite. + +00:01:05 We started in Pyramid, cruising old school lanes, had that stable base, yeah, sir. + +00:01:09 Welcome to Talk Python To Me, the number one Python podcast for developers + +00:01:13 and data scientists. + +00:01:14 This is your host, Michael Kennedy. + +00:01:16 I'm a PSF fellow who's been coding for over 25 years. + +00:01:20 Let's connect on social media. + +00:01:22 You'll find me and Talk Python on Mastodon, Bluesky, and X. + +00:01:25 The social links are all in your show notes. + +00:01:27 You can find over 10 years of past episodes at talkpython.fm. + +00:01:31 And if you want to be part of the show, you can join our recording live streams. + +00:01:35 That's right, we live stream the raw, uncut version of each episode on YouTube. + +00:01:39 Just visit talkpython.fm/youtube to see the schedule of upcoming events. + +00:01:44 Be sure to subscribe there and press the bell so you'll get notified anytime we're recording. + +00:01:48 This episode is brought to you by Sentry. + +00:01:50 Don't let those errors go unnoticed. + +00:01:52 Use Sentry like we do here at Talk Python. + +00:01:53 Sign up at talkpython.fm/sentry. + +00:01:57 And it's brought to you by our Agentic AI programming for Python course. + +00:02:02 Learn to work with AI that actually understands your code base and build real features. + +00:02:07 Visit talkpython.fm/agentic-ai. + +00:02:11 Della, Rebecca, and Carl, welcome to all of you type-loving Pythonistas. + +00:02:18 Awesome to have you here on the show. + +00:02:20 Thanks for being here. + +00:02:20 We're going to talk Python typing, especially from the perspective of the Python Typing Council, + +00:02:27 which honestly, I am a huge fan of Python typing. + +00:02:30 It's still something I learned about not too long ago. + +00:02:33 So I'm going to be learning along with everyone else, what it is you all do and so on. + +00:02:38 So I'm really excited to be diving into this. + +00:02:41 I think since types came to Python, I think it's made it a little bit more rigorous, + +00:02:46 you know, for all those people out there like, oh, it's not a real language without any form of static typing. + +00:02:51 We can't use it on real projects. + +00:02:53 I don't know how true that was, but certainly it's less true now. + +00:02:56 You know, you can pick per project. + +00:02:58 So it's super cool. + +00:02:59 Before we get into all that, though, let's just go around for a quick introductions. + +00:03:03 Jala, welcome to the show. + +00:03:05 Awesome to have you here. + +00:03:06 Who are you? + +00:03:06 Hi, yeah. + +00:03:07 Jala, I've been on the Python Typing Council since the beginning. + +00:03:10 I helped set it up a couple of years ago. + +00:03:12 Outside of the typing work, I currently work at OpenAI, where I work on developer productivity, + +00:03:17 which means things like running CI for people and helping, generally helping people be productive. + +00:03:23 I've been working with Python for more than a decade. + +00:03:25 Started out because my previous job was mostly in Python and then got more and more involved with the language. + +00:03:32 So let me get this right. + +00:03:33 At OpenAI, you're basically helping developers there have better developer tooling + +00:03:37 and common packages and workflows and stuff like that. + +00:03:41 Is that kind of the story? + +00:03:42 That's right. + +00:03:43 Mostly around things that happen in CI, like running tests efficiently, figuring out the right tests to run, + +00:03:49 getting the right CI workers out. + +00:03:50 That sounds very exciting. + +00:03:51 Right in the epicenter of all the big tech stuff these days. + +00:03:56 Super cool. + +00:03:57 Rebecca, hello. + +00:03:58 Welcome. + +00:03:58 Hey, thanks for having me. + +00:04:00 I'm Rebecca. + +00:04:01 I've been on the Typing Council also for about three years, I think, since the, less than three, + +00:04:07 since the beginning. + +00:04:08 But my day job, I work at Meta on Python typing, gone Pyrefly, which is a new type checker + +00:04:16 and language server written in Rust, still in beta. + +00:04:20 Prior to that, I was at Google for eight years, also on the Python team. + +00:04:24 I just, I really like Python. + +00:04:26 Yeah, super neat. + +00:04:27 I'm a big fan of both Pyrefly and ty, which will both have representatives here, I know. + +00:04:33 And I think it's just a super exciting time for Python types. + +00:04:37 And certainly that's one of the reasons. + +00:04:38 So very cool. + +00:04:39 Carl, welcome back. + +00:04:40 Thank you. + +00:04:41 Great to be here. + +00:04:42 Yeah, Carl Meyer. + +00:04:43 I currently work at Astral, where I work on ty, which is a Python type checker and language server + +00:04:50 written in Rust, also in beta. + +00:04:53 And yeah, I guess, how did I get into typing? + +00:04:55 Or I've been on the Typing Council, not since the beginning. + +00:04:59 I think it's been a year and a half. + +00:05:01 And yeah, I got into Python typing at the time in 2016, 2017. + +00:05:07 I was working at Instagram. + +00:05:08 And that was in the very early days of Python typing. + +00:05:12 The PEP44, PEP43, the early Python typing PEPs had recently come out within the last couple of years. + +00:05:19 And one of the co-authors of some of those PEPs, Lukash Lange, was actually sitting at a desk + +00:05:24 right next to me at the time. + +00:05:25 And at some point, we started to think that we should try this Python typing stuff + +00:05:28 on the Instagram server monolith. + +00:05:31 And so I took that on as a side project. + +00:05:33 And then it eventually became the main project. + +00:05:35 And then it took like three years. + +00:05:37 So a lot of Python typing experience there. + +00:05:40 There absolutely is. + +00:05:40 You know, I think a couple of things I'd like to touch on there. + +00:05:43 First of all, Instagram, is it maybe the biggest Django deployment in the world? + +00:05:48 It's certainly one of the bigger ones, right? + +00:05:50 And I think a lot of people don't necessarily know that a core chunk of Instagram is actually Python, right? + +00:05:55 I mean, I don't know if we have any way to know how big the Django deployments in the wild might be. + +00:06:00 But it's certainly a big one. + +00:06:01 Yeah, it's definitely a big one. + +00:06:02 There were some talks about dismissing the garbage collector from the Instagram folks. + +00:06:08 That wasn't you giving the talk, but at PyCon. + +00:06:11 So that was pretty interesting. + +00:06:12 But I think actually that work that you're talking about, especially with Lukash, really kind of opened + +00:06:19 a lot of people's eyes about Python typing, right? + +00:06:22 He gave a couple of PyCon talks, showed, you know, real metrics of how much of the code base is typed, + +00:06:28 how much it's changed, like error detection, that kind of stuff. + +00:06:33 So let me ask you, do you feel like it would be different? + +00:06:36 Would it have gone different now if tools like TY and Pyrefly existed back then? + +00:06:42 Is Python typing different now than it was then? + +00:06:44 Certainly, yes. + +00:06:45 I mean, there's been, the type system has gotten more complex over time. + +00:06:49 So it is both more expressive and more complex. + +00:06:52 And yeah, we have more type checkers available now. + +00:06:56 I do agree that it's more complicated, and I don't know how to feel about that. + +00:06:59 It is more expressive, but I feel like it's starting to get, I mean, we're not at C++ ATL, + +00:07:06 like templates of templates of templates, but still, it's getting more serious. + +00:07:12 But I guess one of the really nice parts is that you can just take as much as you want + +00:07:16 of the complexity, and you can just leave the rest, right? + +00:07:19 That's part of the magic of Python typing, is that it's a gradual typing system. + +00:07:23 That's a choice people get to make. + +00:07:25 It can be none, it can be quite a bit, and anywhere in between. + +00:07:30 So I guess that's probably one of the decisions. + +00:07:33 Let's talk about the typing council. + +00:07:34 So when did the typing council come along, and did the typing council exist to create + +00:07:40 all of these PEPs and make this happen, or was it afterwards? + +00:07:43 Like, what's the history of the typing council and its purpose, folks? + +00:07:47 We'll run it. + +00:07:47 Yeah, it postdates most of the PEPs. + +00:07:49 So initially, the text system was created just through the regular PEP process. + +00:07:52 It means that something gets submitted, first still to Guido as the BDFL, + +00:07:57 later to the steering council. + +00:07:59 Meant that it's very hard to make changes to, like, this specification. + +00:08:03 Like, anytime you want to make change something about how the type systems would work, + +00:08:06 we had to go through this PEP procedure, talk to the steering council, who are very busy people, + +00:08:11 who deal with a lot of other aspects of the language other than typing. + +00:08:14 So Shantanu and I came up with this idea of creating a separate council to specifically in charge of typing, + +00:08:21 that would be in a specification where we can make small changes ourselves + +00:08:25 without having to go through this whole PEP process. + +00:08:27 And this way, when all the type checkers agreed that something needs to go a certain way + +00:08:31 and it's not exactly what's in the PEPs, we can change it and have a place to record that + +00:08:36 and people can refer to it and new type checkers can also try to follow those decisions. + +00:08:41 Very interesting. + +00:08:41 I didn't realize that it was sort of, was there to allow for small changes + +00:08:46 to be made to make that much easier. + +00:08:48 But of course that makes sense because the PEP process is, it's pretty serious and drawn out. + +00:08:52 And we've seen even small language changes have quite passionate folks, I guess we should say. + +00:09:00 So yeah, yeah, very nice. + +00:09:02 Do you have any examples of the types of changes that y'all have, that have happened over the years + +00:09:06 that maybe were typing council only? + +00:09:09 One was the specification where how overloads work, which is perhaps not really a small change, + +00:09:14 but one of the most complicated features in the type system really is the overloads, + +00:09:17 where you can give multiple signatures for a function and type checkers sort of select + +00:09:22 which one to use based on the arguments when the function is called. + +00:09:26 And when it was initially created, from what I recall, there just wasn't really a specification. + +00:09:31 It's just like you use the signatures in a way that makes sense. + +00:09:35 And Eric Trout, who's currently on the council, came up with a pretty specific procedure + +00:09:40 for exactly how overload should work to make it so that type checkers have, + +00:09:45 well, sort of users can understand how it works and sort of type checkers can have something + +00:09:47 to work towards to make sure that they work infinite overloads in the same way. + +00:09:51 Maybe a smaller example that is an example of something that would have been too small for a pep + +00:09:56 and hard to accomplish before the typing council existed. + +00:10:00 And this is actually a change that I pushed through before the, before I was on the typing council, + +00:10:05 but the typing council approved it, was a clarification around the interpretation of data class fields. + +00:10:11 If a final annotation is applied to a data class field, does that mean, so if you apply a final annotation + +00:10:18 to a regular class attribute, since it can't be changed, that implies that it's a class variable. + +00:10:24 And there was a question of if that should be the interpretation with the data class or not. + +00:10:28 So we discussed that and made a clarification to the spec. + +00:10:31 I've never really thought about final being applied to a class field, but I've always used them + +00:10:36 sort of just for constants. + +00:10:38 But, you know, maybe people out there don't know, like typing dot final bracket type, + +00:10:43 right? + +00:10:44 That's kind of the way you can do constants in Python, right? + +00:10:47 Constants for the type checker. + +00:10:48 Nothing in the runtime will stop you from editing it. + +00:10:51 That's... + +00:10:51 Not there. + +00:10:52 Not there. + +00:10:52 I have some examples coming up and I'm interested to hear your thoughts on it, + +00:10:56 but for sure it's, there is this tension, right? + +00:11:00 I mean, I think that's probably worth touching on as well is this is a tension for Python + +00:11:04 in general is you can write all the types you want and then when you run your code, + +00:11:09 it just doesn't care. + +00:11:11 There's a few instances, Pydantic, FastAPI, a few others, but generally speaking, + +00:11:15 it's there for the editors and the type checkers and the linters and not for runtime, right? + +00:11:20 Yeah, that's right. + +00:11:21 There's many exceptions to that. + +00:11:23 There's a product like mypyC, which comes with mypy that's used those types + +00:11:27 to compile your code into more efficient machine codes. + +00:11:30 Maybe there's going to be more products like that in the future. + +00:11:32 I don't know. + +00:11:33 But yes, in general, it's separate from the runtime. + +00:11:36 Sort of a similar model to TypeScript where TypeScript gets compiled into JavaScript + +00:11:40 and types just go away. + +00:11:42 Here, we don't do a compilation step, but still the same idea of the types + +00:11:45 just not influencing the runtime. + +00:11:47 Although we do make them available for introspection via done annotations attributes, + +00:11:51 which is what has enabled the projects like Pydantic and other sort of runtime checkers + +00:11:56 to make use of type annotations at runtime also. + +00:11:59 Yeah, I don't know if the typing council was around for this, but there was proposed, + +00:12:03 I don't remember the exact details, but something to the effect of for type checking, + +00:12:07 not actually doing some of the full imports or something along those lines, + +00:12:13 right, where the runtime behavior would have made it hard for tools like Pydantic + +00:12:17 and others to get that. + +00:12:19 And there was some kind of compromise, right? + +00:12:21 I don't remember the details here. + +00:12:22 Anyone does? + +00:12:23 Yeah, what happened was that there was going to be a change. + +00:12:25 That's what the from future import annotations import does, that changes all annotations + +00:12:29 into raw strings. + +00:12:30 So the default behavior before recently was that annotations that are regular codes. + +00:12:36 If you write devf return to ints and you import the module, it just looks up + +00:12:40 the name ints and puts that in an annotations dictionary, which makes introspection easy, + +00:12:44 but it made a test on costs on performance because memory usage sometimes was high + +00:12:50 and also made things harder to use sometimes because if you use a name that's not defined yet + +00:12:55 at runtime, you get an error. + +00:12:57 That often comes up if you have like a class that has a reference in an adaptation + +00:13:01 to the class itself or circular dependency classes. + +00:13:05 Right. + +00:13:06 The circular imports because you want to say this class is created by that thing + +00:13:11 and it returns one, but you know, somehow you've got to import the other one + +00:13:15 and that's such a hassle. + +00:13:17 Yeah, it's, yeah, even out in the audience we have, Tom says, circular imports. + +00:13:21 Oh, yeah, for sure. + +00:13:22 What about lazy imports? + +00:13:24 Like that just recently got accepted and will be in 3.15. + +00:13:27 Which I'm super excited about because I think it'll make app startup a lot faster + +00:13:32 for many use cases. + +00:13:34 But does that have knock-on effects for typing? + +00:13:36 Not that directly because I think for a type checker lazy imports mostly just look + +00:13:41 like regular imports. + +00:13:42 I guess I should maybe leave that for the people who are actually working on type checkers + +00:13:46 and being written right now. + +00:13:47 Yeah, Rebecca, do you see this making any difference for you? + +00:13:50 Lazy imports? + +00:13:51 To be honest, it's not something we've looked at too carefully yet. + +00:13:55 3.15 seems a little more in the future, but I don't think it's likely to make a huge difference. + +00:14:03 Carl? + +00:14:04 I've thought about it briefly and I think that it, I think the type checkers + +00:14:07 really won't need to care. + +00:14:08 Maybe there will be some edge cases that will come up that I haven't thought of, + +00:14:11 but it shouldn't be a big deal. + +00:14:12 Yeah, that's what I thought as well. + +00:14:14 The one variation that I can certainly see is if you have a, if you have something + +00:14:20 specified in a type, like say for a field of a class or a Pydantic model + +00:14:25 or something that would otherwise not trigger the lazy import to become imported, + +00:14:30 would potentially having types specify cause more importing to happen sooner + +00:14:36 in the runtime? + +00:14:36 Yeah, there's actually an issue related to this that I think we may need + +00:14:39 to resolve before 3.15, but I don't know how yet. + +00:14:43 If you use a type in a data class annotation that's lazy imported, actually creating + +00:14:47 a data class will delay by the import. + +00:14:49 It will try to resolve the import and actually make it not lazy. + +00:14:55 This is because data classes doesn't really need to look at all of the annotations + +00:14:58 in your class, but it looks at them enough to trigger reification of the import. + +00:15:04 I shared this with some of the people on the lazy imports team, but we haven't + +00:15:07 yet come up with a good way around it. + +00:15:09 I think this might end up being a bit of a food gun, so I feel like we should ideally + +00:15:12 find a workaround, but I don't know what it would be yet. + +00:15:15 I don't know that it's wrong that it converts it to an eager import, which it needs + +00:15:20 to know what it is potentially, right? + +00:15:22 It actually doesn't. + +00:15:24 Data classes just need to know whether it is classed for or not. + +00:15:27 I think that's pretty much all. + +00:15:28 I guess there's an init for also, but it doesn't really need to know anything else. + +00:15:32 So in theory, it should be possible to just say, hey, it is not classed for, so don't bother + +00:15:37 importing it. + +00:15:38 Okay, so that's for data classes, but say if I specify a parameter type on a function. + +00:15:43 Yeah, then it should be fine. + +00:15:45 I guess, again, unless something is, if it does annotate, so if you have + +00:15:49 something like a decorator that looks at annotations in your function, it might reify + +00:15:53 those imports. + +00:15:54 There is one other potentially interesting thing for type checkers. + +00:15:57 It's already difficult for type checkers to figure out when like a submodule + +00:16:02 should be considered to be an attribute of the parent module because the way + +00:16:06 this happens in Python is that any import of a submodule anywhere will attach + +00:16:10 that submodule as an attribute on the parent module, but that at runtime, + +00:16:15 that could literally happen anywhere. + +00:16:17 It could happen in totally unrelated code outside of the module and a type checker + +00:16:20 probably won't be able to see that. + +00:16:22 So type checkers already have sort of complex sets of rules around where they look + +00:16:26 for these submodule imports and when they consider a submodule import to be reliably + +00:16:30 happening enough that it should, that the type checker should consider this submodule + +00:16:35 to exist as an attribute. + +00:16:38 And lazy imports may make that even, we'll add one more wrinkle to those + +00:16:43 sets of heuristics in that we'll have to decide if you have a lazy import + +00:16:46 of a submodule and you're done to init.py, it's lazy. + +00:16:50 So should the type checker consider that submodule to be imported or not be imported? + +00:16:55 It'll be another case where there's no clear right answer and we'll just have + +00:16:58 to make a decision one way or the other. + +00:17:02 This portion of Talk Python is brought to you by Sentry. + +00:17:05 I've been using Sentry personally on almost every application and API that I've built + +00:17:10 for Talk Python and beyond over the last few years. + +00:17:13 They're a core building block for keeping my infrastructure solid. + +00:17:16 They should be for yours as well. + +00:17:18 Here's why. + +00:17:19 Sentry doesn't just catch errors. + +00:17:21 It catches all the stuff that makes your app feel broken. + +00:17:23 The random slowdown, the freeze you can't reproduce, that bug that only shows up + +00:17:28 once real users hit it. + +00:17:29 And when something goes wrong, Sentry gives you the whole chain of events + +00:17:32 in one place. + +00:17:33 Errors, traces, replays, logs, dots connected. + +00:17:36 You can see what's led to the issue without digging through five different dashboards. + +00:17:41 Sear, Sentry's AI debugging agents builds on this data, taking the full context, + +00:17:46 explaining why the issue happened, pointing to the code responsible, drafts a fix, + +00:17:51 and even flags if your PR is about to introduce a new problem. + +00:17:55 The workflow stays simple. + +00:17:56 Something breaks, Sentry alerts you, the dashboard shows you the full context, + +00:18:01 Sear helps you fix it and catch new issues before they ship. + +00:18:04 It's totally reasonable to go from an error occurred to fixed in production + +00:18:08 in just 10 minutes. + +00:18:10 I truly appreciate the support that Sentry has given me to help solve my + +00:18:14 bugs and issues in my apps, especially those tricky ones that only appear in + +00:18:19 production. + +00:18:19 I know you will too if you try them out. + +00:18:21 So get started today with Sentry. + +00:18:23 Just visit talkpython.fm slash Sentry and get $100 in Sentry credits. + +00:18:28 Please use that link. + +00:18:29 It's in your podcast player show notes. + +00:18:31 If you're signing up some other way, you can use our code talkpython26, all one word, + +00:18:36 talkpython26, to get $100 in credits. + +00:18:40 Thank you to Sentry for supporting the show. + +00:18:43 Yeah, there's some variations across type checkers, which we'll get to later. + +00:18:47 I think, though, before we move off this, there's actually off introducing the + +00:18:52 typing council. + +00:18:53 I think we should point out that there's two other folks who couldn't be here + +00:18:56 who are also on the typing council, Eric Trout and Yuka Letts, the docilo? + +00:19:03 Sorry, Yuka. + +00:19:05 But I want to make sure that we point out there's actually five people, not just the + +00:19:08 three of you, right? + +00:19:09 How do you get on the council? + +00:19:11 Is there an election? + +00:19:13 Do you just apply? + +00:19:14 I think these are filled by the members themselves. + +00:19:16 So when somebody declares the intention to leave the council, we basically ask for + +00:19:20 people who are interested and then make a selection. + +00:19:23 Generally, we try to get people who have experience in the type system. + +00:19:27 We try to get a good cross representation of people working on different + +00:19:29 type checkers. + +00:19:30 We have Carl and Rebecca here who work on two type checkers, ty and Pyrefly. + +00:19:36 Yuka works on Pyrites, which are two of the most lightly used type checkers. + +00:19:41 So we try to get representation of people working on those parts of the ecosystem. + +00:19:46 That's really cool that it's got a bias towards finding people actually doing the work. + +00:19:50 So let's talk about the specification project at typing.python.org. + +00:19:55 What is this here? + +00:19:56 I'll talk a bit about it. + +00:19:58 I guess it's a specification for how the type system used to work. + +00:20:03 The way it started was that, Yela, you basically took all the typing peps + +00:20:07 and like stapled them together, right, to make like one long doc. + +00:20:12 And since then, we've been iterating on it, filling in parts that were missing + +00:20:16 like overload evaluation and making other changes as well. + +00:20:21 Yeah, it's tricky, right? + +00:20:22 Because traditionally, the typing system is kind of defined across a series of + +00:20:28 peps. + +00:20:28 And so what is the document that tells you how it works, right? + +00:20:31 Yeah, that made it hard because often there's peps built on top of each other. + +00:20:35 So then in the extreme, you might see like one thing in one pep and then there's + +00:20:39 another pep that adds an aspect of it, another one that adds another aspect. + +00:20:43 And overall it makes it very hard to follow. + +00:20:44 One of the things I did recently was rewrite the typed dict another. + +00:20:55 Ended up rewriting the whole thing to basically put all those features together + +00:20:58 in a coherent whole rather than just having them all copy-pasted one after the + +00:21:04 other. + +00:21:04 Okay, so if somebody really wants a good understanding of the Python typing system, + +00:21:09 they go to typing.python.org. + +00:21:11 You know, one thing I think maybe is worth touching on, it's just kind of out of the + +00:21:15 blue a bit, but I think it's a really interesting aspect of the Python typing system + +00:21:20 is the, what is it called, the numerical tower or the number tower, where + +00:21:24 it's like, if I have a number, I could specify it as an int, or I could specify + +00:21:29 it as a float, and those kinds of things, but do you really need to say it's an + +00:21:34 int pipe float, or a union of int and float, if it could be either, right? + +00:21:39 And the, what is it called? + +00:21:40 It's the numerical tower, right? + +00:21:42 Yeah, there are different towers too. + +00:21:43 In Python, there's also this thing called a numbers module that you have there, that's + +00:21:48 just basically ignored by the type system. + +00:21:50 It's been useful for some people, I feel like in general that module just + +00:21:52 hasn't worked out very well as being very useful. + +00:21:55 I think the interesting aspect is that you know, that you can say it's a + +00:21:59 float, and that's basically equivalent to union of integer and float, and so + +00:22:04 on, right? + +00:22:05 I think the typing numbers in Python is pretty interesting. + +00:22:08 I think every type checker has a different interpretation of what a float + +00:22:13 annotation actually means. + +00:22:16 It's an area of some lack of clarity in the spec. + +00:22:19 Yeah, a lot of contentiousness. + +00:22:20 If we could go back in time, I would, like, knowing what I know now, I probably advocate + +00:22:25 for things being done differently because, like, beginning, you know, like, there + +00:22:30 were multiple things, like, with similar flavor. + +00:22:33 Like, there was also one where you could give a parameter a non-none annotation and + +00:22:39 default it to none for convenience, and we've largely, like, moved away from stuff like + +00:22:44 that in favor of explicitness. + +00:22:46 Yeah, what the current spec says is that basically if you have a function that takes + +00:22:49 a float, you're also allowed to pass an int. + +00:22:52 That's not really enough. + +00:22:54 It doesn't tell you how these things work in all cases, and we've had some + +00:22:59 attempts to try to come up with a way to specify that special case in a way that + +00:23:04 makes more sense, at least makes more sense to me. + +00:23:07 It's been very contentious. + +00:23:08 People have very strong opinions about this. + +00:23:10 I guess non-obvious is what I'd like to say, really, honestly. + +00:23:13 So I'd like to get the official counsel's thoughts on this. + +00:23:17 When is typing too much typing? + +00:23:20 I made the joke about C++, ATL, if you've ever worked with that, it's like + +00:23:24 a template class where templated classes are part of the concrete type of the + +00:23:30 template. + +00:23:30 It's just off the hook. + +00:23:32 There's certainly places where typing can be too much, and a lot of the purity of Python + +00:23:38 or the readability of Python is the fact that it's got so few symbols. + +00:23:44 And so adding types adds context, but it also makes it a little harder to read. + +00:23:49 When is too much typing? + +00:23:50 When do you recommend typing? + +00:23:53 Rebecca, I'll let you go first, but what are your thoughts on how much typing should + +00:23:57 I use in Python? + +00:23:59 I'll give you what is my official stance, which is that if you want your + +00:24:04 type checker to work well, you should type annotate your API boundaries. + +00:24:09 So parameters and turns in public functions, public class attributes, things + +00:24:14 like that, and even things that seem true trivial, like, oh, this function returns + +00:24:18 none, better to annotate it because, you know, someone else might be depending on + +00:24:24 your library and consuming that type of information. + +00:24:27 I will say personally, what I tend to do is I annotate things that I think are + +00:24:32 non-trivial because I want to see that as documentation. + +00:24:37 And if something, you know, a function that does return none, to be honest, I will + +00:24:42 probably forget to annotate it half the time because I'll be like, I honestly don't + +00:24:46 need to see it. + +00:24:47 One of the interesting features of the Pyrefly VS Code extension, that's the only + +00:24:53 one I can speak of at the moment, and Carl, you've got to tell me if the + +00:24:56 ty one does this as well, is it will sort of overlay its belief of what types + +00:25:03 are. + +00:25:03 Like, if there's, you say x equals a function return value and it knows what the + +00:25:07 function returns, it'll have a gray, like, colon int, if it returned an int or something. + +00:25:11 So you can kind of read the code and see what the types are without actually putting + +00:25:17 it into the text of the code. + +00:25:19 It's only within the editor. + +00:25:20 Does ty do something like that, Carl? + +00:25:22 Yes, we also have inlay type ins. + +00:25:23 Yeah, inlay type ins, that's what it's called. + +00:25:25 So, yeah, I don't know, that also brings an interesting challenge, not a challenge, + +00:25:28 like a wrinkle to the recommendation of should I put types on, like, the return value + +00:25:35 because I want to know that's a list of user, not a list of user IDs or whatever, for + +00:25:39 example, like a list of UUID. + +00:25:41 But if, it's going to show up anyway in the editor, maybe I don't have to write + +00:25:45 that, right? + +00:25:45 And so that becomes sort of somewhere where you could debate again, I think. + +00:25:50 However, I do 100% agree with you, Rebecca, that put it on your API boundaries. + +00:25:54 If, like, this is the place that people get into some part of your code and they + +00:25:58 don't know or want to know about the inside of it, having types there is really helpful + +00:26:03 both for editors, for type checkers, and just for reading code, and even for AI, which + +00:26:08 is a crazy world. + +00:26:09 Yeah. + +00:26:09 Carl? + +00:26:10 What are your thoughts here? + +00:26:11 How much typing is too much typing? + +00:26:13 What's the guidelines here? + +00:26:14 I think I agree with Rebecca's answer. + +00:26:16 I mean, that one place you definitely want to have explicit type annotations is that + +00:26:21 API boundaries, the public API of a library, etc. + +00:26:24 In terms of what's too much typing, I mean, there are certainly patterns + +00:26:28 that have historically been used in Python that we still can't express well + +00:26:34 in the type system, or that require extremely complex type annotations to + +00:26:39 express well, and I think there it becomes a judgment call. + +00:26:43 If it's like a core, widely used API, you may get a lot of benefit from some + +00:26:48 very complex and verbose annotations, and so then it's worth sort of going + +00:26:53 through that pain and the pain of adding them and of reading them in order to get that + +00:26:57 additional typing coverage everywhere you use that API. + +00:27:01 If it's much less frequently used code that's highly dynamic, maybe it's not worth + +00:27:05 it in that case. + +00:27:06 I think there's a lot of judgment calls here. + +00:27:08 What about like one-off scripts? + +00:27:10 You know, I'm going to write this thing to just move this data from here to there, + +00:27:13 and once it's moved, I don't need it again. + +00:27:15 It's done with that old system, we're going to the new one. + +00:27:17 Maybe less typing. + +00:27:18 Yeah, I think that's what's useful for you. + +00:27:20 Often I feel like one-off scripts are not really one-off like maybe you want to move some + +00:27:24 similar data later, and then it's useful if you can understand your code again, + +00:27:27 if you want to read what you did. + +00:27:29 You thought you didn't need it again, and all of a sudden at six months old, + +00:27:32 you don't understand it, and the types that help a lot, right? + +00:27:35 Yeah. + +00:27:36 Yellow, what's your advice? + +00:27:38 What Cara and Rebecca said makes sense to me too. + +00:27:40 I think types have advantages in terms of documenting humor readers, what + +00:27:45 is going on, and in terms of catching mistakes that otherwise would not be + +00:27:49 caught until runtime perhaps. + +00:27:51 They have costs in maybe making your code harder to read if there's too much + +00:27:54 going on. + +00:27:54 So add types as long as those benefits outweigh the costs. + +00:27:57 Yeah. + +00:27:58 I mean, do you recommend to anyone that they just 100% go full like C++, + +00:28:04 C# on it, and just type it every single thing? + +00:28:08 Is there an advantage like for static type checkers, you know, like mypy type stuff + +00:28:12 you can run across and get that? + +00:28:14 I mean, you could do that with Powerfly or TY and the CLI as well, but you know, + +00:28:18 thinking more mypy is like kind of being real strict on some of that stuff. + +00:28:21 Personally, I do tend to annotate almost all like function parameters and if class + +00:28:26 attributes, if I make a class, sometimes it's not as necessary, like you don't + +00:28:30 really need to annotate your tests perhaps, or you don't need to annotate + +00:28:33 internal functions as much, but for my own coding, I usually find it helpful + +00:28:36 to do that. + +00:28:37 But sometimes I see people annotating even local variables where it's very + +00:28:41 obvious to type check if the type is and they can just infer it reliably, and then + +00:28:46 it really just adds noise and you shouldn't do it. + +00:28:48 Yeah, exactly. + +00:28:48 If you've got a function that's annotated with a return value and you say x equals + +00:28:53 the function call, then the type checkers can infer that and you're just causing + +00:28:57 extra noise, I guess. + +00:29:00 So suppose you all want to change something. + +00:29:03 What's the process of actually going through and making some changes? + +00:29:08 Mostly sort of two levels of this. + +00:29:10 Well, maybe there's even three levels. + +00:29:11 The first one is if it's something that's so small that's just like a wording + +00:29:15 clarification or something, we just make a PR to the repo and a few of us + +00:29:19 look at it and we change it. + +00:29:21 The second level is when it's sort of a smaller change that doesn't really + +00:29:25 introduce a new feature and then we make a PR to the typing spec repo and + +00:29:30 we formally have all of us sign off on it. + +00:29:32 That's what happens like what Carl mentioned earlier of the final change in data + +00:29:37 classes. + +00:29:37 It's had to merge this one, yeah, add Carl. + +00:29:41 I love it. + +00:29:42 This repo itself doesn't have anything. + +00:29:44 It's the Python typing repo where the decisions are made. + +00:29:48 The typing council just has like some documentation. + +00:29:52 Yeah. + +00:29:52 And then the third level is peps, like really big new changes. + +00:29:55 You can still write a PEP and then we make a recommendation and the steering + +00:29:59 council makes a decision eventually. + +00:30:00 So if I wanted to suggest something, I could come up here and I could open up + +00:30:05 an issue, maybe start a conversation on typing, Python slash typing. + +00:30:10 And you can make a pull request to change the spec. + +00:30:12 Okay. + +00:30:12 And so the pull request would not be to change the code, like how Python + +00:30:17 maybe interprets code that has this new thing, but to suggest that the spec + +00:30:22 has it, which then would start a process that ultimately might make CPython + +00:30:26 understand it, right? + +00:30:27 Well, CPython itself probably doesn't do anything with it. + +00:30:30 I guess most of the things that go directly here are changes to how to interpret + +00:30:33 things that are already in CPython. + +00:30:37 This portion of Talk Python To Me is brought to you by us. + +00:30:40 I want to tell you about a course I put together that I'm really proud of, Agentic + +00:30:45 AI Programming for Python Developers. + +00:30:48 I know a lot of you have tried AI coding tools and come away thinking, well, this is + +00:30:52 more hassle than it's worth. + +00:30:54 And honestly, all the vibe coding hype isn't helping. + +00:30:58 It's a smokescreen that hides what these tools can actually do. + +00:31:01 This course is about agentic engineering, applying real software engineering + +00:31:06 practices with AI that understands your entire code database, runs your tests, and + +00:31:11 builds complete features under your direction. + +00:31:14 I've used these techniques to ship real production code across Talk Python, + +00:31:18 Python Bytes, and completely new projects. + +00:31:21 I migrated an entire CSS framework on a production site with thousands of lines of HTML + +00:31:25 in a few hours, twice. + +00:31:28 I shipped a new search feature with caching and async in under an hour. + +00:31:32 I built a complete CLI tool for Talk Python from scratch, tested, documented, and + +00:31:38 published to PyPI in an afternoon. + +00:31:41 Real projects, real production code, both Greenfield and legacy. + +00:31:46 No toy demos, no fluff. + +00:31:48 I'll show you the guardrails, the planning techniques, and the workflows that + +00:31:51 turn AI into a genuine engineering partner. + +00:31:54 Check it out at talkpython.fm slash agentic dash engineering. + +00:31:58 That's talkpython.fm slash agentic dash engineering. + +00:32:01 The link is in your podcast player's show notes. + +00:32:05 If it's adding something new, it will usually need to go through a path, + +00:32:09 except if it's something very small. + +00:32:10 Let's talk about that for a minute. + +00:32:11 We got two representatives here of the newer breed of tools. + +00:32:17 What's the story for inconsistencies across interpretations of the spec? + +00:32:23 I know that there's slight variations. + +00:32:25 I've also, you know, not putting either of you on the spotlight, but like using, say, + +00:32:31 PyCharm and like writing code. + +00:32:33 So it's type checkers happy and then using something like Pyright. + +00:32:37 And so it has a real different interpretation of what you should let slide and what you + +00:32:42 shouldn't. + +00:32:43 I feel like Pyright is more, much more focused on like enforcing the nullability and or + +00:32:49 the lack thereof. + +00:32:50 And it warns of inconsistencies there where PyCharm doesn't seem to care as much. + +00:32:54 I don't know which one I like better, but I know they're different. + +00:32:57 And if I write code in one that I open the other, I'm like, huh, why is it + +00:33:00 upset? + +00:33:00 It seemed like it was fine. + +00:33:02 How do you all navigate this? + +00:33:03 Yeah. + +00:33:03 One thing useful to say about the spec there is that the spec covers a lot of + +00:33:07 things. + +00:33:08 In particular, it tends to cover sort of the details of more advanced type + +00:33:11 system features. + +00:33:12 But there's a lot of very fundamental stuff about how a type checker works + +00:33:17 in terms of how it does inference and how it does type narrowing. + +00:33:21 And even in some cases, like you mentioned, you know, what it chooses to + +00:33:24 emit errors on that isn't really covered by the spec, partly maybe because we + +00:33:29 haven't gotten to it and also partly intentionally in that there may be room in + +00:33:35 some of those cases for different type checkers to work differently if they're + +00:33:38 serving different needs. + +00:33:39 Like if PyCharm is primarily concerned about being a useful kind of IDE and providing go-to + +00:33:45 definition and that sort of thing, maybe emitting lots of warnings or errors + +00:33:50 and all kinds of things where your code might be doing something wrong isn't + +00:33:54 as high a priority. + +00:33:54 And another type checker might have a different priority. + +00:33:57 One thing I do want to mention is that it may not seem like it, but things are already + +00:34:04 much better than they used to be. + +00:34:07 Like previously, I worked on a different type checker called PyType. + +00:34:11 And at that time, it was, you know, sort of the Wild West. + +00:34:14 Like we want to know how other type checkers like do something. + +00:34:18 Well, you know, like open up the mypy playground, open up the PyRite playground, + +00:34:22 see what tells you. + +00:34:25 Now we at least have spec and conformance tests. + +00:34:28 Yeah, that's really cool. + +00:34:29 How much would you say that your two type checkers maybe bring in mypy as well? + +00:34:36 Like how much do they agree versus disagree? + +00:34:38 You know, like you only see the differences. + +00:34:40 You don't see in which ways that they are the same as a consumer of them so much, + +00:34:44 right? + +00:34:44 You're like, why is this one squiggly when it wasn't squiggly before? + +00:34:47 But how similar or different are they? + +00:34:49 I don't know how we would quantify that. + +00:34:51 I think there's a lot that is the same just because it's based on how Python actually + +00:34:55 works. + +00:34:56 We're both trying to model the same language and then there's certainly also + +00:34:59 plenty of differences or things that we handle differently. + +00:35:02 So Rebecca, do you have a better way to quantify that? + +00:35:04 Yeah, I agree. + +00:35:05 it's hard to quantify, I suppose, talk a bit abstractly about various type + +00:35:11 checkers philosophies. + +00:35:13 With Pyrefly, we really try to do a lot of type inference. + +00:35:17 So that's a way in which we intentionally diverge a bit from mypy. + +00:35:21 But other than that deliberate decision, if we see ways in which we are accidentally + +00:35:25 different, we do try to fix that because otherwise people would have a hard time running + +00:35:31 multiple type checkers or migrating. + +00:35:33 Yeah, differences obviously cause pain for users who are using multiple type + +00:35:37 checkers or writing libraries that need to support multiple type checkers. + +00:35:40 So like Rebecca said, it's like if we are different from other type checkers, we want + +00:35:45 to be sure that there's a good reason for that difference. + +00:35:47 The difference should be because of philosophical choice, not just you happen to have + +00:35:52 chosen slightly differently, right? + +00:35:54 Yeah, and it's not just people who run different type checkers. + +00:35:58 Like you pointed out, Carl, a lot of times it is if I have a library and + +00:36:02 then different people want to consume that library, then their type checker + +00:36:06 may or may not warn them about how my library declares its types and so on. + +00:36:11 I'll give you a real quick example. + +00:36:14 I have a, I can't remember which one it was, I have three or four different open + +00:36:19 source libraries that I've created that somehow work with creating, basically passing + +00:36:24 data to templates in web apps, right? + +00:36:27 So one is like I want to use the Chameleon web template framework, but with fast + +00:36:31 API or with Flask or there's some other variations like partials and so on. + +00:36:35 I can't remember which one, but it doesn't really matter. + +00:36:37 One of them decorated a Flask. + +00:36:40 I think it was a Flask, especially makes it irrelevant. + +00:36:43 A Flask endpoint and PyRite was really upset. + +00:36:47 Like the error message filled the entire page of how it was inconsistent with + +00:36:53 what it expected for the definition of the Flask view method. + +00:36:56 I'm like, no one is going to call this. + +00:36:58 Like what does it even matter what this type is? + +00:37:01 It still runs fine. + +00:37:02 The runtime is fine. + +00:37:03 You know, it's no problem with this decorator. + +00:37:05 It worked fine, but something about the way that the Flask at get returned + +00:37:11 the type versus what my thing returned varied in like a really slight way. + +00:37:16 I didn't care, but somebody was using some editor that used PyRite and they're like, you + +00:37:21 have to help fix this. + +00:37:22 I can't take all these warnings. + +00:37:23 They're huge and they're everywhere. + +00:37:25 Like, okay, I'll go fix it. + +00:37:27 Right. + +00:37:27 And I went and I put way more effort than was justified into a function type + +00:37:32 that no one ever calls just to make the errors on some type checker I didn't use go + +00:37:38 away. + +00:37:38 Right. + +00:37:39 And that's the kind of thing where it becomes just a headache. + +00:37:41 I don't know. + +00:37:42 I wish I remember. + +00:37:42 I probably got that written down in an issue somebody filed, but it was, it was + +00:37:46 a gnarly error. + +00:37:47 And, or if you're working on an open source project, you know, you can't make + +00:37:50 everybody use the same editor that wants to contribute on a big project. + +00:37:55 And so you might run into this variation as well. + +00:37:57 So there's a lot of cases. + +00:37:58 Yeah. + +00:37:58 It can be really difficult to make these decisions about what kind of, what + +00:38:03 sorts of errors people want their type checker to catch or what's too pedantic. + +00:38:08 You want your type checker to catch non-obvious errors, not just the obvious + +00:38:12 ones that you probably would have seen by looking at the code yourself. + +00:38:15 But then there'll be cases where somebody says, well, I don't care. + +00:38:18 That's too pedantic. + +00:38:19 And it is difficult to make everyone happy. + +00:38:21 Who decides what the right signature of a flask view and point should be like if you + +00:38:27 the framework can call it. + +00:38:28 It should be okay. + +00:38:29 There's not. + +00:38:30 Just because it had a decorator before, that doesn't mean that's the official structure. + +00:38:33 But anyway, I do think one of the bigger philosophical differences has to do around this + +00:38:39 concept of nullability. + +00:38:41 Do you guys call it nullability or none ability? + +00:38:44 Like nullability comes from the other languages. + +00:38:46 And by that, I mean, I can specify that I have an integer. + +00:38:49 And in the Python type system, it cannot be set to none, even though in the runtime it can. + +00:38:55 It has to be a concrete int type unless you make it a optional int or an + +00:39:00 int pipe none or one of those type things, right? + +00:39:03 And how strong that gets enforced seems to be one of the biggest difference + +00:39:07 of opinions that I've seen around. + +00:39:10 Like, how do you all think about that? + +00:39:11 That's interesting to me that that's your experience because my experience has been that + +00:39:16 that's actually an area where everyone seems to agree as far as I can tell that these are is + +00:39:21 an important source of bugs and it's better to catch them. + +00:39:23 So I think all of the type checkers, maybe you said PyCharm doesn't. + +00:39:26 I don't think PyCharm does that. + +00:39:28 I'm pretty sure it doesn't because I agree that it's an important thing to check, but it's + +00:39:34 also a point of a lot of friction. + +00:39:37 And by that, I mean, let's suppose I'm going to have a class that I need to create an + +00:39:42 instance of and then put values into. + +00:39:45 And I know once I put the values into it, let's say it has a user ID, I know for certain that + +00:39:50 that's going to be an integer, right? + +00:39:53 So I'd like to say user ID colon int because everywhere I use that object later, if it's a + +00:39:58 function that takes an int and I specify it as optional int, I will get a type check warning + +00:40:03 every single call site when I try to pass that. + +00:40:07 But I know from the semantics of the behavior that it's going to always be an int + +00:40:12 unless it's not initialized, right? + +00:40:14 And like in this short period where I want to create it. + +00:40:17 So I can't set the type to int. + +00:40:19 I have to set the optional int until I've loaded it. + +00:40:21 And, but there's like this, I don't know, that's, that's the part where I see a lot of it + +00:40:25 show up is inconsistencies and then warnings all over the place. + +00:40:29 So I'm like, well, but that function is actually checking if it's none and it'll return null, + +00:40:35 you know, none or something like that. + +00:40:36 So I totally agree with you. + +00:40:38 It's just somewhere I've seen the most inconsistencies across maybe PyCharm versus + +00:40:43 others. + +00:40:44 mypy also has a legacy mode for not checking none things called non-strict optional. + +00:40:50 We're trying to get rid of that from mypy because yeah, strict optional, like being + +00:40:54 strict about it is the more sensible thing to do. + +00:40:56 But it's possible that you've seen that too. + +00:40:59 Yeah, I agree. + +00:40:59 So what you mentioned is maybe sort of a special case of the case where you pass + +00:41:03 something to a class and there's initialization that changes the types. + +00:41:06 Doesn't necessarily have to deal with none. + +00:41:08 It could also just be like the attribute doesn't exist at all beforehand + +00:41:10 or something. + +00:41:11 Yeah, we don't have a good solution for that. + +00:41:13 Maybe there's room for something to support that use case better. + +00:41:17 I don't know what it would look like. + +00:41:18 In some cases, there's ways you can, these things can sometimes nudge you towards a + +00:41:22 different design that is actually safer and will avoid errors. + +00:41:26 Like in the kind of case you're talking about, you know, is it actually necessary + +00:41:30 that an uninitialized object and an initialized one are represented by the + +00:41:34 same type? + +00:41:34 Or is there a way to adjust the API so that those are actually different types than you + +00:41:38 solve the problem and your code is safer or so? + +00:41:41 I'm thinking like you submit a web form and before you parse it, you've got to create the instance + +00:41:46 to set the values. + +00:41:47 And I don't know. + +00:41:48 It's not worth diving into, but I do find this differentiation between like + +00:41:52 the strict enforcement of none versus not none. + +00:41:55 I think it's powerful and I do think you all are right that it does catch a lot of + +00:41:58 errors. + +00:41:58 It's just, it's just a difference and it's just an interesting, interesting + +00:42:02 choice. + +00:42:02 But I didn't get a concrete answer from the official counsel. + +00:42:06 Nullable or noneable? + +00:42:08 What is it? + +00:42:09 I feel like you just don't really even talk about it as a term mostly. + +00:42:12 It's, yeah, none is special in the type system in like how you represent + +00:42:16 it, but it's not really special in other ways. + +00:42:19 You don't have a term for int pipe none? + +00:42:22 Int or none. + +00:42:22 Historically, the term was optional, although I think that term has problems and + +00:42:26 we're sort of moving away from it because specifically one problem is that optional can mean + +00:42:33 you don't have to pass it in, like I say, as a function parameter. + +00:42:37 Let's talk a little bit about TypeShed. + +00:42:40 I think TypeShed is pretty interesting. + +00:42:41 Maybe people don't know too much about it. + +00:42:44 So I'm sure you all are familiar with this project that you can basically add + +00:42:48 type information that the libraries didn't bother to include for you, right? + +00:42:53 What are thoughts on TypeShed? + +00:42:54 How much do you all lean on this to sort of round out missing types? + +00:42:58 There are two parts to TypeShed, right? + +00:43:00 There's the standard library type stubs, which I think are invaluable. + +00:43:06 Like all the type checkers use those. + +00:43:08 And I mean, will the standard library itself ever have inline types? + +00:43:12 Who knows? + +00:43:12 This might be around forever. + +00:43:14 And then there are also the third party stubs. + +00:43:18 And I think that's what you're describing. + +00:43:20 They're libraries that for whatever reason don't ship with stubs themselves. + +00:43:24 Those are in TypeShed. + +00:43:26 And I think it's been like for a while, there's sort of been a question of like what + +00:43:31 we want to do with like TypeShed's third party stubs, right? + +00:43:35 Because like ideally like libraries would ship with their own types, but there + +00:43:39 are various obstacles to that. + +00:43:41 The obstacles that I know of used to be like, we want this to run on Python 2 and Python 3. + +00:43:47 Or we want it to run on Python 3.3 still. + +00:43:50 But it's been a long time since any non-type supporting version of Python was a real, + +00:43:57 you know, a supported type of thing, right? + +00:43:59 I mean, even 3.9 became deprecated. + +00:44:02 So on one hand, I feel like they could be merged in, but there's also a lot of + +00:44:07 other areas that are maybe we don't, they're not common, right? + +00:44:12 Like other libraries, like pick some, let's say Pyramid. + +00:44:16 I don't think the Pyramid web framework really ever got types added to it. + +00:44:20 Somebody could go and create a typeshed stub or a types underscore pyramid you could + +00:44:26 pip install and then we'll add the types, right? + +00:44:28 I certainly see it being really valuable for third party things that are just + +00:44:31 not going to get the type attention they need. + +00:44:33 Yeah, I think typeshed is great. + +00:44:35 I've spent a lot of time on improving it. + +00:44:37 As Rebecca said, especially with a standard library, it's irreplaceable. + +00:44:41 For third party libraries, I think it's become less needed over time. + +00:44:45 It used to be that very few third party libraries had any types. + +00:44:49 Now that's obviously changed. + +00:44:51 A lot of libraries ship their own types, but still there are quite a few types + +00:44:56 of libraries left where there aren't inline types and typeshed can provide + +00:45:00 useful types. + +00:45:01 I think typeshed also provides a service because it has a really great framework for testing + +00:45:05 these types. + +00:45:06 We have tools like step tests and various type checkers that help to make sure these types are + +00:45:12 good and meet a high standard. + +00:45:15 So yeah, I think they're still useful for many libraries. + +00:45:17 Yeah, I was just looking at the types dash flask and I guess it must be, must + +00:45:23 be gone because now type flask must have it internally. + +00:45:26 So it's kind of an interim sort of thing. + +00:45:28 That's pretty cool. + +00:45:29 In general, typeshed has the policy that we remove the snaps from typeshed if + +00:45:32 they are in the library itself. + +00:45:34 I find these super valuable because if there's a library I want to work with and it just doesn't + +00:45:39 have types for whatever reason, you can install stuff from here and all of a sudden your editor's + +00:45:43 way happier. + +00:45:44 I mean, I know we, you all agreed on like the API boundaries and I did as well. + +00:45:50 It's like that's one of the really cool things. + +00:45:51 The other thing that really makes me excited about types is if I hit dot in my editor, I get + +00:45:57 a meaningful list of real information about what I'm working on. + +00:46:00 And so adding, adding these types of things are pretty interesting. + +00:46:04 I want to ask you all about sort of these rogue, rogue tools that do stuff + +00:46:10 with Python typing that maybe y'all didn't intend. + +00:46:12 Like we all mentioned Pydantic, we've got Typer and FastAPI, but even a little farther out + +00:46:19 there is a bear type. + +00:46:21 Are you familiar with bear type? + +00:46:22 Yeah. + +00:46:22 Bear type's interesting. + +00:46:24 You can import, they have fun. + +00:46:27 They have fun with their, their import names and stuff. + +00:46:31 But basically you can put a, either a decorator onto some sort of call site or + +00:46:36 something, or you can just do it to an entire package or entire modules rather. + +00:46:41 So just run bear type dot claw import bear type this. + +00:46:45 And then it actually turns into runtime type checks. + +00:46:49 Good idea, bad idea. + +00:46:51 Interesting. + +00:46:52 What do you all think? + +00:46:53 So un-Pythonic, you won't even open the webpage. + +00:46:56 People should feel free to write whatever code helps them make like better + +00:47:00 software. + +00:47:01 I haven't really used bear type much myself, but it's clearly useful for + +00:47:04 some people. + +00:47:05 And I think generally in designing a type system, we should try to accommodate all users who + +00:47:09 do useful things to the type system. + +00:47:10 And that includes things like Pydentic or bear type. + +00:47:13 It's pretty fast. + +00:47:14 It's not as big of a hit as you would, you would imagine. + +00:47:18 They, let me see, what are they, somewhere they had a really fun, fun saying in here, but here + +00:47:23 we go. + +00:47:24 Bear type brings Rust C++ inspired zero cost abstractions into the lawless world of dynamically + +00:47:29 typed Python by enforcing type safety at the granular level of functions and + +00:47:33 methods against type hints standardized by the Python community. + +00:47:37 order one, non-amortized worst case time with negligible constant factors. + +00:47:41 Like, how about that? + +00:47:43 No, it's a pretty neat library and it's pretty fast. + +00:47:45 I honestly, I've never used it in production. + +00:47:47 Having type hints and squigglies in the editors or in the linters has always + +00:47:53 been enough for me, but I can see using this if it's really critical and you're + +00:47:57 having issues, maybe you want to catch some runtime errors. + +00:48:00 I don't know. + +00:48:00 It's not quite an endorsement, but it sure is like a, huh, that's different. + +00:48:04 I definitely think that the extent to which type checkers may have a different understanding of + +00:48:11 your code from what happens at runtime and there isn't anything built in to catch + +00:48:15 that is sometimes a pain point. + +00:48:18 And so the desire to have your type annotations, to find out at runtime if your + +00:48:22 type annotations are telling you a lie, it makes a lot of sense why people would like + +00:48:26 that. + +00:48:27 I mean, it's something used to from other languages where the type checker is built + +00:48:30 into the compiler. + +00:48:30 Right. + +00:48:31 You get like a runtime type cast, like cannot. + +00:48:33 We kind of get that if you try to parse a thing, you know, like put the int + +00:48:38 param around a string and it's not really a parsable as an int. + +00:48:42 But for like real type information, I think personally I would use this as like I might apply types, + +00:48:48 type checking to a module for debugging and development for a minute and just see + +00:48:53 what happens and then turn it back off. + +00:48:55 You know, I don't know that I'd just ship production code that way. + +00:48:58 But anyway, I got a couple more questions. + +00:49:00 We're getting shorter on time here. + +00:49:02 What was one of the harder questions that you all, harder decisions you all had to address on the + +00:49:09 council? + +00:49:09 I think the most contentious one was PEP 724, if I remember the number correctly. + +00:49:15 It was around a feature called type guards, which is around user-defined type + +00:49:20 narrowing functions. + +00:49:22 Initially, they find that in a way that later was found to be somewhat problematic + +00:49:25 and we basically came up with a better set of proposed semantics that maybe we should have done + +00:49:30 the first time around. + +00:49:33 And what this PEP proposed, and as you can see, I sponsored it, is that we + +00:49:37 basically changed the meaning of the existing type guards under certain conditions. + +00:49:41 What is a type guard? + +00:49:42 A type guard is a function, like there's a good example there, the isiterable. + +00:49:46 It's a function that tells you how to narrow something. + +00:49:50 So in this example, there's an isiterable type guard, which narrows an object to + +00:49:55 an iterable of anything. + +00:49:56 And then inside the func there, you can see if isiterable file, it knows + +00:50:01 that it's an iterable. + +00:50:04 And in this case, yeah, I guess it just narrows exactly to iterable any. + +00:50:08 That's one of the ways that type guards works. + +00:50:10 I see. + +00:50:11 And the type that returns kind of communicates to the type system, like that this + +00:50:15 function ensures that this, the thing that came in as an arbitrary object, in fact, + +00:50:21 is one of these. + +00:50:22 Okay. + +00:50:22 Interesting. + +00:50:23 Yeah. + +00:50:23 So that was a tricky one, huh? + +00:50:25 Any other standout, Rebecca or Carl? + +00:50:27 Well, the current discussion around what is the meaning of a float annotation, still + +00:50:32 unresolved, contentious topic. + +00:50:34 Okay. + +00:50:34 Gotcha. + +00:50:35 I mean, this on PEP724 is also what came to my mind immediately as well, because + +00:50:42 this was challenging discussion because, you know, like there were very conflicting considerations at + +00:50:48 play. + +00:50:49 It's like, what semantics did we want in the long term? + +00:50:52 And what did we want the type system to look like, you know, say like 10 years + +00:50:56 from now versus backwards compatibility and what the migration story would look + +00:51:00 like? + +00:51:01 It was quite tricky. + +00:51:02 I guess that's something you will always have to be cognizant of is like every + +00:51:07 change, even if it's an improvement, has to justify the fact that now you have + +00:51:12 challenges with the version history over time. + +00:51:16 I'm thinking like dict of string comma int with a capital or lowercase d. + +00:51:22 I've got people, I did a YouTube video showing something with the lowercase + +00:51:26 version because I was using something super modern like Python 3.11. + +00:51:30 And I got a message like, hey, Michael, you don't know how to write Python. + +00:51:34 Your code is broken. + +00:51:35 This code that you wrote just doesn't even run. + +00:51:38 I don't know how this is. + +00:51:39 I'm like, what version of Python is in? + +00:51:40 3.8. + +00:51:41 Nope. + +00:51:41 You can't use 3.8 for that. + +00:51:43 You're going to need to get a newer one. + +00:51:44 You know what I mean? + +00:51:44 But like those are complexities that get added to Python because of that. + +00:51:49 Now you've got two ways to specify what a dict is. + +00:51:52 There's a preferred new way, but there's still the old way and it just, it sort of piles + +00:51:57 up. + +00:51:58 And it's very hard to ever actually get rid of the old way, even if there's no good + +00:52:00 reason to use it anymore. + +00:52:01 Exactly. + +00:52:02 Once it's there, it's written in ink pretty much, right? + +00:52:05 Like we have five or six different ways to format strings. + +00:52:08 Maybe with t-strings at six now. + +00:52:10 They're all going to still be there, right? + +00:52:12 So every change, every decision you make is not just a matter of, is it the + +00:52:16 right decision, right? + +00:52:17 It's the, is it worth it? + +00:52:19 I'm sure. + +00:52:20 Yeah. + +00:52:21 I don't know. + +00:52:21 How do you all balance that? + +00:52:22 Like that's tricky. + +00:52:23 With things like the dict chains, at least we sort of know we're moving towards + +00:52:27 better states and there's two things, but they mean exactly the same thing. + +00:52:31 So the confusion is not as bad. + +00:52:34 The problem with type cards is that we're going to change how some existing thing + +00:52:38 works, like what it meant. + +00:52:39 And I think there are good reasons that maybe that's the right thing to do, but the, + +00:52:44 it would also have been pretty confusing for people if their existing types suddenly started + +00:52:47 meaning something completely different. + +00:52:49 Absolutely. + +00:52:50 Hence float. + +00:52:51 Okay. + +00:52:51 What's coming next? + +00:52:53 Like 3.15, 3.16, do you all have things that are in the works that you think are going to come + +00:52:58 or debates that are brewing? + +00:53:01 For 3.15, the, there's a type dict feature coming, extra items. + +00:53:05 you can already use it in tapping extensions if you want to use it, but it will be in + +00:53:10 CPath as of 2.15. + +00:53:12 It's likely we'll have a small thing I added called disjoint basis, which is very technical, + +00:53:17 but helps type narrowing in some cases. + +00:53:19 Yeah. + +00:53:19 I think those are the things that are likely to make it. + +00:53:22 There's, we can only speculate about what else people can propose. + +00:53:25 We're sort of bound by what people actually write up as peps. + +00:53:28 We have to wait for Google to write the peps before we can approve them. + +00:53:30 I think there's PEP 747 for type form, which I think is not, I think we recommended + +00:53:35 its acceptance, but I don't think the steering council accepted it yet or it hasn't + +00:53:39 been accepted formally. + +00:53:40 I think that's on their plate. + +00:53:41 Yeah. + +00:53:42 Yeah. + +00:53:42 So that's also pretty likely to make it into 3.15. + +00:53:45 This is one example of a case that will be pretty useful to people working + +00:53:49 with type annotations at runtime because it'll allow you to, it's sort of a meta + +00:53:53 thing where you can annotate, have a type annotation that describes another type + +00:53:58 annotation. + +00:53:59 So that's useful if you're, if you're writing code that works with type annotations. + +00:54:03 Make the peidantics of the world very happy. + +00:54:05 I am actually pretty excited about type form because, you know, I feel like there's + +00:54:10 a gap and we can express in the type system and we're good. + +00:54:14 And there are cases in the existing type system, like for instance, the cast + +00:54:18 function and some other cases where something takes any type expression as an + +00:54:24 argument. + +00:54:24 We actually don't have a good way to annotate that today and this will provide a nice + +00:54:28 way to express that. + +00:54:28 Let me pull up one thing really quick. + +00:54:31 Quick shout out to Will McGuggan here. + +00:54:32 He just released his Toad project, which is the new, takes textual and rich + +00:54:38 and all that kind of stuff and applies it to like, what if we had a better cloud code + +00:54:42 type of experience, which is pretty interesting. + +00:54:44 So the reason I'm bringing this up is, you know, final question. + +00:54:47 What about, do you all even worry about the role of like how types interact with AI + +00:54:54 and agentic coding tools? + +00:54:56 I know that if you have some code that has types on it and you give it to an AI, it's + +00:55:02 got a better chance of understanding what's happening than if you give it purely + +00:55:05 untyped code and say, tell me about this, right? + +00:55:08 It doesn't even know necessarily what's being passed to it. + +00:55:10 But is that anything you'll think about or what are your thoughts on this? + +00:55:14 Certainly think about it some. + +00:55:15 I mean, I think overall my feeling is that these coding agents seem to do better than + +00:55:21 more kind of the tighter feedback loops you can give them to work with. + +00:55:26 And so typing is another useful source of feedback where you can say, add type + +00:55:29 annotations and make sure the type checker passes and seems so it still seems pretty useful + +00:55:34 in that world. + +00:55:35 Yeah, you can easily write rules that say when you are done on anything I've asked + +00:55:39 you to do, always run ty or always run Pyrefly and make sure that there's no more, no + +00:55:45 new errors or at least or ideally zero errors, right? + +00:55:48 But nothing has been introduced. + +00:55:50 Yeah, pretty interesting. + +00:55:51 You other folks, Rebecca, Yela? + +00:55:53 Yeah, I guess in general, I think typing will remain useful for AI. + +00:55:57 We are probably rapidly moving to a world where a large proportion of all code + +00:56:00 is written by AI. + +00:56:02 Not everybody likes that opinion, Yael. + +00:56:03 Not everybody likes that. + +00:56:04 I guess I, maybe my current line of work makes me think that's more likely to happen. + +00:56:08 You don't have to like the fact it's going to be night soon, but it's going to be + +00:56:11 night. + +00:56:12 You know what I mean? + +00:56:12 Like there's the, I just think there's so much momentum on this, at least in + +00:56:16 the next five years or something, that it's going to be really, it's, it's a + +00:56:19 truth of how many people are writing code regardless of whether individuals want to + +00:56:23 write code that way. + +00:56:24 You know what I mean? + +00:56:25 So I think it's a consideration. + +00:56:26 Yeah. + +00:56:26 Yeah. + +00:56:26 I forgot that you worked at OpenAI. + +00:56:28 So of course, I should pull up a codex example or something, shouldn't I? + +00:56:32 Yeah. + +00:56:33 Codex is great. + +00:56:33 Use it. + +00:56:34 No, but I mean, do you have any further insight into like the role of types + +00:56:38 and coding agents? + +00:56:40 I know that's not exactly what you work on, right? + +00:56:41 You're more at the lower. + +00:56:42 As Carl said, types can also be helpful for AI to understand code better and to + +00:56:47 get a better feedback loop. + +00:56:48 I feel like the very big AI, the board is like humans. + +00:56:51 And if AI makes, sorry, if typing makes humans better at writing understanding + +00:56:55 this code, they're probably also big AI better at it. + +00:56:58 It's the locality of information. + +00:57:00 You can read the function and know everything you need to know about what's going + +00:57:03 into it without bouncing around and trying to understand blocks of code and like what might've been + +00:57:08 created that's getting impacted. + +00:57:09 It's good for humans and also good for AI. + +00:57:12 Right. + +00:57:12 Rebecca. + +00:57:12 Because I don't have much need to, and I'll say I am maybe a little more skeptical + +00:57:18 than most of my coworkers about the quality of AI generated code. + +00:57:23 But that means I think I am particularly gung-ho about, you know, like get AI to use + +00:57:29 types, type checkers, keep the guardrails there. + +00:57:33 I think that'll be very important. + +00:57:34 Yeah, if it's going to make a mistake, don't let it at least like make the type + +00:57:38 system become disconnected and not working. + +00:57:40 Like it has to keep the types hanging together as a minimum bar, right? + +00:57:44 And you can easily set that up as an automation. + +00:57:46 Yeah. + +00:57:47 Interesting to think of it as guardrails rather than an accelerant. + +00:57:50 But yeah, 100% it is. + +00:57:52 All right, folks. + +00:57:52 I think that's it for all the time that we have. + +00:57:55 Thank you. + +00:57:56 Thank you for being here. + +00:57:57 Final thoughts before we go. + +00:57:59 Carl, I'll let you go first. + +00:58:00 Final thoughts for people out there interested in Python typing. + +00:58:02 Yeah. + +00:58:02 Well, first of all, thanks for having us on the podcast. + +00:58:05 Really appreciate it. + +00:58:06 And thoughts for people out there. + +00:58:08 I guess if you have ideas of how Python typing could be improved, discuss.python. + +00:58:14 Python.org is a good place to bring up ideas and discuss them with the typing community and see + +00:58:19 what positive changes we can make. + +00:58:22 Rebecca. + +00:58:22 First, thank you, Michael. + +00:58:24 This is a lot of fun. + +00:58:26 Last thoughts? + +00:58:28 Hey, so, you know, like we'll look at the typing council and sometimes think, + +00:58:32 oh, you know, like the PEP has like governance in its name, but I wouldn't say we're really + +00:58:37 a governing body or anything. + +00:58:39 It's like people who are using the type system, like users, they're the ones who come up with, + +00:58:45 you know, like all the best ideas, propose them, discuss them. + +00:58:48 And we're just here to sort of be like, hey, you know, like we have some background and like + +00:58:55 how type checkers work and maybe some of the history and we can provide input. + +00:58:58 But I just encourage people, if there's a change you want to see in the type system, you know, + +00:59:03 like propose it yourself. + +00:59:05 It's very friendly and open community. + +00:59:07 Yeah. + +00:59:08 Now people who have listened know a little bit more about how to do so. + +00:59:11 Awesome. + +00:59:12 Thanks. + +00:59:12 Jale, final word. + +00:59:13 Yeah. + +00:59:13 Also, again, thank you for having me here. + +00:59:16 It's been great talking to all of you. + +00:59:17 I guess what I want to say is similar to what Karin Rebecca just said. + +00:59:20 If you want to have something changed to the type system, I'd really encourage you to sign up + +00:59:24 for discuss.python.org, make a proposal, go through the process. + +00:59:27 It can be somewhat daunting, perhaps, especially if you have to create a PEP, but it is + +00:59:32 doable. + +00:59:32 There are several recent typing PEPs have just been community members who saw something they + +00:59:37 wanted to improve, proposed a PEP, and saw it to completion. + +00:59:41 If there's something you want to see in the type system, then you can do + +00:59:44 it too. + +00:59:44 Thank you all for keeping Python typing going strong. + +00:59:48 Really appreciate your time on the show. + +00:59:49 See you all later. + +00:59:50 Bye. + +00:59:51 Bye. + +00:59:51 This has been another episode of Talk Python To Me. + +00:59:55 Thank you to our sponsors. + +00:59:56 Be sure to check out what they're offering. + +00:59:57 It really helps support the show. + +00:59:59 Take some stress out of your life. + +01:00:01 Get notified immediately about errors and performance issues in your web or mobile + +01:00:06 applications with Sentry. + +01:00:07 Just visit talkpython.fm slash Sentry and get started for free. + +01:00:12 Be sure to use our code talkpython26. + +01:00:15 That's talkpython, the numbers two, six, all one word. + +01:00:19 And it's brought to you by our Agentic AI programming for Python course. + +01:00:24 Learn to work with AI that actually understands your code base and build real + +01:00:28 features. + +01:00:29 Visit talkpython.fm slash agentic dash AI. + +01:00:33 If you or your team needs to learn Python, we have over 270 hours of beginner + +01:00:38 and advanced courses on topics ranging from complete beginners to async code, Flask, Django, + +01:00:44 HTMX, and even LLMs. + +01:00:46 Best of all, there's no subscription in sight. + +01:00:48 Browse the catalog at talkpython.fm. + +01:00:51 And if you're not already subscribed to the show on your favorite podcast + +01:00:54 player, what are you waiting for? + +01:00:56 Just search for Python in your podcast player. + +01:00:58 We should be right at the top. + +01:00:59 If you enjoy that geeky rap song, you can download the full track. + +01:01:03 The link is actually in your podcast blur show notes. + +01:01:05 This is your host, Michael Kennedy. + +01:01:07 Thank you so much for listening. + +01:01:08 I really appreciate it. + +01:01:09 I'll see you next time. + +01:01:18 Voyager. + +01:01:19 Voyager. + +01:01:21 And we ready to roll Upgrading the code No fear of getting whole We tapped into that modern vibe + +01:01:33 Overcame each storm Talk Python To Me IceSync is the norm + diff --git a/transcripts/539-catching-up-with-the-python-typing-council.vtt b/transcripts/539-catching-up-with-the-python-typing-council.vtt new file mode 100644 index 0000000..ccae9b1 --- /dev/null +++ b/transcripts/539-catching-up-with-the-python-typing-council.vtt @@ -0,0 +1,9868 @@ +WEBVTT + +00:00:00.000 --> 00:00:02.000 +You're adding type-ins to your Python code. + +00:00:02.240 --> 00:00:04.560 +Your editor is happy, autocomplete is working great, + +00:00:05.000 --> 00:00:06.260 +but then you switch tools + +00:00:06.260 --> 00:00:08.540 +and suddenly there are red squigglies everywhere. + +00:00:09.220 --> 00:00:12.160 +Who decides what a float annotation actually means + +00:00:12.160 --> 00:00:15.680 +or whether passing none where an int is expected + +00:00:15.680 --> 00:00:16.700 +should be an error? + +00:00:17.180 --> 00:00:19.260 +It turns out there's a five-person council + +00:00:19.260 --> 00:00:21.260 +dedicated to exactly these questions + +00:00:21.260 --> 00:00:25.100 +and two brand new Rust-based type checkers + +00:00:25.100 --> 00:00:26.680 +are raising the bar as well. + +00:00:27.200 --> 00:00:29.620 +On this episode, I sit down with three of the members + +00:00:29.620 --> 00:00:31.300 +of the Python Typing Council, + +00:00:31.620 --> 00:00:34.380 +Yela Zylstra, Rebecca Chen, and Carl Meyer + +00:00:34.380 --> 00:00:37.040 +to learn about how the type system is governed, + +00:00:37.520 --> 00:00:40.240 +where the spec and type checkers agree and disagree, + +00:00:40.720 --> 00:00:42.880 +and I get the council's official advice + +00:00:42.880 --> 00:00:45.240 +on how much typing is just enough. + +00:00:45.800 --> 00:00:48.640 +This is Talk Python To Me, episode 539, + +00:00:49.260 --> 00:00:51.860 +recorded January 27th, 2026. + +00:00:53.460 --> 00:00:56.300 +Talk Python To Me, yeah, we ready to roll. + +00:00:56.540 --> 00:00:59.160 +Upgrading the code, no fear of getting old. + +00:00:59.160 --> 00:01:01.780 +Async in the air, new frameworks in sight, + +00:01:01.920 --> 00:01:02.960 +geeky rap on deck. + +00:01:03.260 --> 00:01:04.960 +Quart crew, it's time to unite. + +00:01:05.060 --> 00:01:07.980 +We started in Pyramid, cruising old school lanes, + +00:01:08.260 --> 00:01:09.760 +had that stable base, yeah, sir. + +00:01:09.760 --> 00:01:10.840 +Welcome to Talk Python To Me, + +00:01:10.940 --> 00:01:13.180 +the number one Python podcast for developers + +00:01:13.180 --> 00:01:14.220 +and data scientists. + +00:01:14.640 --> 00:01:16.080 +This is your host, Michael Kennedy. + +00:01:16.420 --> 00:01:20.060 +I'm a PSF fellow who's been coding for over 25 years. + +00:01:20.600 --> 00:01:21.740 +Let's connect on social media. + +00:01:22.060 --> 00:01:23.940 +You'll find me and Talk Python on Mastodon, + +00:01:24.040 --> 00:01:25.220 +Bluesky, and X. + +00:01:25.400 --> 00:01:27.380 +The social links are all in your show notes. + +00:01:27.380 --> 00:01:30.420 +You can find over 10 years of past episodes + +00:01:30.420 --> 00:01:31.620 +at talkpython.fm. + +00:01:31.720 --> 00:01:33.380 +And if you want to be part of the show, + +00:01:33.500 --> 00:01:35.140 +you can join our recording live streams. + +00:01:35.300 --> 00:01:37.260 +That's right, we live stream the raw, + +00:01:37.380 --> 00:01:39.360 +uncut version of each episode on YouTube. + +00:01:39.680 --> 00:01:42.400 +Just visit talkpython.fm/youtube + +00:01:42.400 --> 00:01:44.360 +to see the schedule of upcoming events. + +00:01:44.520 --> 00:01:46.160 +Be sure to subscribe there and press the bell + +00:01:46.160 --> 00:01:48.180 +so you'll get notified anytime we're recording. + +00:01:48.720 --> 00:01:50.600 +This episode is brought to you by Sentry. + +00:01:50.720 --> 00:01:52.240 +Don't let those errors go unnoticed. + +00:01:52.400 --> 00:01:53.980 +Use Sentry like we do here at Talk Python. + +00:01:53.980 --> 00:01:57.360 +Sign up at talkpython.fm/sentry. + +00:01:57.720 --> 00:01:58.880 +And it's brought to you by + +00:01:58.880 --> 00:02:01.920 +our Agentic AI programming for Python course. + +00:02:02.340 --> 00:02:05.360 +Learn to work with AI that actually understands your code base + +00:02:05.360 --> 00:02:06.900 +and build real features. + +00:02:07.400 --> 00:02:10.800 +Visit talkpython.fm/agentic-ai. + +00:02:11.940 --> 00:02:14.320 +Della, Rebecca, and Carl, + +00:02:14.780 --> 00:02:17.940 +welcome to all of you type-loving Pythonistas. + +00:02:18.480 --> 00:02:19.780 +Awesome to have you here on the show. + +00:02:20.100 --> 00:02:20.860 +Thanks for being here. + +00:02:20.860 --> 00:02:22.700 +We're going to talk Python typing, + +00:02:23.100 --> 00:02:26.920 +especially from the perspective of the Python Typing Council, + +00:02:27.380 --> 00:02:30.700 +which honestly, I am a huge fan of Python typing. + +00:02:30.840 --> 00:02:33.160 +It's still something I learned about not too long ago. + +00:02:33.300 --> 00:02:35.900 +So I'm going to be learning along with everyone else, + +00:02:35.980 --> 00:02:38.020 +what it is you all do and so on. + +00:02:38.100 --> 00:02:41.540 +So I'm really excited to be diving into this. + +00:02:41.700 --> 00:02:44.420 +I think since types came to Python, + +00:02:44.540 --> 00:02:46.620 +I think it's made it a little bit more rigorous, + +00:02:46.920 --> 00:02:48.520 +you know, for all those people out there like, + +00:02:48.520 --> 00:02:51.820 +oh, it's not a real language without any form of static typing. + +00:02:51.900 --> 00:02:52.960 +We can't use it on real projects. + +00:02:53.060 --> 00:02:53.920 +I don't know how true that was, + +00:02:54.020 --> 00:02:56.680 +but certainly it's less true now. + +00:02:56.880 --> 00:02:58.380 +You know, you can pick per project. + +00:02:58.480 --> 00:02:59.220 +So it's super cool. + +00:02:59.480 --> 00:03:00.700 +Before we get into all that, though, + +00:03:00.920 --> 00:03:03.000 +let's just go around for a quick introductions. + +00:03:03.880 --> 00:03:04.860 +Jala, welcome to the show. + +00:03:05.160 --> 00:03:05.920 +Awesome to have you here. + +00:03:06.180 --> 00:03:06.580 +Who are you? + +00:03:06.700 --> 00:03:07.200 +Hi, yeah. + +00:03:07.540 --> 00:03:09.260 +Jala, I've been on the Python Typing Council + +00:03:09.260 --> 00:03:09.980 +since the beginning. + +00:03:10.240 --> 00:03:12.040 +I helped set it up a couple of years ago. + +00:03:12.380 --> 00:03:13.860 +Outside of the typing work, + +00:03:13.860 --> 00:03:15.060 +I currently work at OpenAI, + +00:03:15.320 --> 00:03:17.120 +where I work on developer productivity, + +00:03:17.480 --> 00:03:19.480 +which means things like running CI for people + +00:03:19.480 --> 00:03:22.760 +and helping, generally helping people be productive. + +00:03:23.240 --> 00:03:25.440 +I've been working with Python for more than a decade. + +00:03:25.940 --> 00:03:28.880 +Started out because my previous job was mostly in Python + +00:03:28.880 --> 00:03:31.440 +and then got more and more involved with the language. + +00:03:32.240 --> 00:03:33.500 +So let me get this right. + +00:03:33.620 --> 00:03:36.440 +At OpenAI, you're basically helping developers there + +00:03:36.440 --> 00:03:37.700 +have better developer tooling + +00:03:37.700 --> 00:03:41.360 +and common packages and workflows and stuff like that. + +00:03:41.520 --> 00:03:42.380 +Is that kind of the story? + +00:03:42.380 --> 00:03:42.940 +That's right. + +00:03:43.380 --> 00:03:45.060 +Mostly around things that happen in CI, + +00:03:45.600 --> 00:03:46.940 +like running tests efficiently, + +00:03:47.520 --> 00:03:48.800 +figuring out the right tests to run, + +00:03:49.140 --> 00:03:50.540 +getting the right CI workers out. + +00:03:50.620 --> 00:03:51.460 +That sounds very exciting. + +00:03:51.720 --> 00:03:56.180 +Right in the epicenter of all the big tech stuff these days. + +00:03:56.260 --> 00:03:56.700 +Super cool. + +00:03:57.140 --> 00:03:58.080 +Rebecca, hello. + +00:03:58.200 --> 00:03:58.400 +Welcome. + +00:03:58.640 --> 00:04:00.000 +Hey, thanks for having me. + +00:04:00.100 --> 00:04:00.540 +I'm Rebecca. + +00:04:01.040 --> 00:04:05.340 +I've been on the Typing Council also for about three years, + +00:04:05.700 --> 00:04:07.140 +I think, since the, less than three, + +00:04:07.440 --> 00:04:08.120 +since the beginning. + +00:04:08.120 --> 00:04:13.400 +But my day job, I work at Meta on Python typing, + +00:04:13.760 --> 00:04:16.820 +gone Pyrefly, which is a new type checker + +00:04:16.820 --> 00:04:19.580 +and language server written in Rust, still in beta. + +00:04:20.080 --> 00:04:23.360 +Prior to that, I was at Google for eight years, + +00:04:23.460 --> 00:04:24.800 +also on the Python team. + +00:04:24.920 --> 00:04:26.280 +I just, I really like Python. + +00:04:26.600 --> 00:04:27.460 +Yeah, super neat. + +00:04:27.460 --> 00:04:29.680 +I'm a big fan of both Pyrefly and ty, + +00:04:30.180 --> 00:04:33.040 +which will both have representatives here, I know. + +00:04:33.720 --> 00:04:36.840 +And I think it's just a super exciting time for Python types. + +00:04:37.000 --> 00:04:38.620 +And certainly that's one of the reasons. + +00:04:38.940 --> 00:04:39.620 +So very cool. + +00:04:39.940 --> 00:04:40.700 +Carl, welcome back. + +00:04:40.980 --> 00:04:41.300 +Thank you. + +00:04:41.380 --> 00:04:41.900 +Great to be here. + +00:04:42.260 --> 00:04:43.800 +Yeah, Carl Meyer. + +00:04:43.800 --> 00:04:47.800 +I currently work at Astral, where I work on ty, + +00:04:48.080 --> 00:04:50.760 +which is a Python type checker and language server + +00:04:50.760 --> 00:04:52.780 +written in Rust, also in beta. + +00:04:53.320 --> 00:04:55.600 +And yeah, I guess, how did I get into typing? + +00:04:55.900 --> 00:04:58.680 +Or I've been on the Typing Council, not since the beginning. + +00:04:59.120 --> 00:05:01.100 +I think it's been a year and a half. + +00:05:01.720 --> 00:05:07.280 +And yeah, I got into Python typing at the time in 2016, 2017. + +00:05:07.280 --> 00:05:08.880 +I was working at Instagram. + +00:05:08.880 --> 00:05:12.520 +And that was in the very early days of Python typing. + +00:05:12.880 --> 00:05:17.020 +The PEP44, PEP43, the early Python typing PEPs + +00:05:17.020 --> 00:05:19.160 +had recently come out within the last couple of years. + +00:05:19.500 --> 00:05:21.440 +And one of the co-authors of some of those PEPs, + +00:05:21.580 --> 00:05:24.180 +Lukash Lange, was actually sitting at a desk + +00:05:24.180 --> 00:05:25.360 +right next to me at the time. + +00:05:25.860 --> 00:05:27.060 +And at some point, we started to think + +00:05:27.060 --> 00:05:28.840 +that we should try this Python typing stuff + +00:05:28.840 --> 00:05:30.960 +on the Instagram server monolith. + +00:05:31.400 --> 00:05:33.880 +And so I took that on as a side project. + +00:05:33.880 --> 00:05:35.680 +And then it eventually became the main project. + +00:05:35.680 --> 00:05:37.400 +And then it took like three years. + +00:05:37.400 --> 00:05:39.880 +So a lot of Python typing experience there. + +00:05:40.060 --> 00:05:40.780 +There absolutely is. + +00:05:40.880 --> 00:05:42.600 +You know, I think a couple of things + +00:05:42.600 --> 00:05:43.600 +I'd like to touch on there. + +00:05:43.840 --> 00:05:47.520 +First of all, Instagram, is it maybe the biggest Django + +00:05:47.520 --> 00:05:48.420 +deployment in the world? + +00:05:48.520 --> 00:05:50.020 +It's certainly one of the bigger ones, right? + +00:05:50.080 --> 00:05:51.640 +And I think a lot of people don't necessarily + +00:05:51.640 --> 00:05:55.320 +know that a core chunk of Instagram is actually Python, right? + +00:05:55.440 --> 00:05:57.380 +I mean, I don't know if we have any way to know + +00:05:57.380 --> 00:06:00.400 +how big the Django deployments in the wild might be. + +00:06:00.460 --> 00:06:01.480 +But it's certainly a big one. + +00:06:01.560 --> 00:06:02.500 +Yeah, it's definitely a big one. + +00:06:02.500 --> 00:06:06.680 +There were some talks about dismissing the garbage collector + +00:06:06.680 --> 00:06:08.500 +from the Instagram folks. + +00:06:08.660 --> 00:06:11.040 +That wasn't you giving the talk, but at PyCon. + +00:06:11.160 --> 00:06:12.120 +So that was pretty interesting. + +00:06:12.320 --> 00:06:15.980 +But I think actually that work that you're talking about, + +00:06:16.040 --> 00:06:19.480 +especially with Lukash, really kind of opened + +00:06:19.480 --> 00:06:22.520 +a lot of people's eyes about Python typing, right? + +00:06:22.560 --> 00:06:25.600 +He gave a couple of PyCon talks, showed, you know, + +00:06:25.640 --> 00:06:28.440 +real metrics of how much of the code base is typed, + +00:06:28.640 --> 00:06:31.780 +how much it's changed, like error detection, + +00:06:32.400 --> 00:06:33.180 +that kind of stuff. + +00:06:33.180 --> 00:06:36.760 +So let me ask you, do you feel like it would be different? + +00:06:36.860 --> 00:06:39.500 +Would it have gone different now if tools like TY + +00:06:39.500 --> 00:06:41.840 +and Pyrefly existed back then? + +00:06:42.180 --> 00:06:44.440 +Is Python typing different now than it was then? + +00:06:44.560 --> 00:06:45.320 +Certainly, yes. + +00:06:45.420 --> 00:06:47.260 +I mean, there's been, the type system has gotten + +00:06:47.260 --> 00:06:49.320 +more complex over time. + +00:06:49.320 --> 00:06:52.240 +So it is both more expressive and more complex. + +00:06:52.740 --> 00:06:55.880 +And yeah, we have more type checkers available now. + +00:06:56.460 --> 00:06:58.080 +I do agree that it's more complicated, + +00:06:58.080 --> 00:06:59.540 +and I don't know how to feel about that. + +00:06:59.540 --> 00:07:03.320 +It is more expressive, but I feel like it's starting to get, + +00:07:03.860 --> 00:07:06.880 +I mean, we're not at C++ ATL, + +00:07:06.960 --> 00:07:08.860 +like templates of templates of templates, + +00:07:09.060 --> 00:07:12.200 +but still, it's getting more serious. + +00:07:12.340 --> 00:07:14.080 +But I guess one of the really nice parts + +00:07:14.080 --> 00:07:16.820 +is that you can just take as much as you want + +00:07:16.820 --> 00:07:18.940 +of the complexity, and you can just leave the rest, right? + +00:07:19.220 --> 00:07:21.060 +That's part of the magic of Python typing, + +00:07:21.300 --> 00:07:23.460 +is that it's a gradual typing system. + +00:07:23.820 --> 00:07:25.780 +That's a choice people get to make. + +00:07:25.780 --> 00:07:28.400 +It can be none, it can be quite a bit, + +00:07:28.800 --> 00:07:30.220 +and anywhere in between. + +00:07:30.560 --> 00:07:32.780 +So I guess that's probably one of the decisions. + +00:07:33.220 --> 00:07:34.480 +Let's talk about the typing council. + +00:07:34.700 --> 00:07:37.380 +So when did the typing council come along, + +00:07:37.800 --> 00:07:40.640 +and did the typing council exist to create + +00:07:40.640 --> 00:07:42.320 +all of these PEPs and make this happen, + +00:07:42.400 --> 00:07:43.460 +or was it afterwards? + +00:07:43.720 --> 00:07:45.380 +Like, what's the history of the typing council + +00:07:45.380 --> 00:07:46.780 +and its purpose, folks? + +00:07:47.160 --> 00:07:47.460 +We'll run it. + +00:07:47.520 --> 00:07:49.340 +Yeah, it postdates most of the PEPs. + +00:07:49.480 --> 00:07:51.260 +So initially, the text system was created + +00:07:51.260 --> 00:07:52.740 +just through the regular PEP process. + +00:07:52.740 --> 00:07:54.220 +It means that something gets submitted, + +00:07:54.800 --> 00:07:57.440 +first still to Guido as the BDFL, + +00:07:57.860 --> 00:07:58.900 +later to the steering council. + +00:07:59.360 --> 00:08:01.520 +Meant that it's very hard to make changes + +00:08:01.520 --> 00:08:02.740 +to, like, this specification. + +00:08:03.020 --> 00:08:04.860 +Like, anytime you want to make change something + +00:08:04.860 --> 00:08:06.580 +about how the type systems would work, + +00:08:06.900 --> 00:08:08.520 +we had to go through this PEP procedure, + +00:08:09.040 --> 00:08:09.940 +talk to the steering council, + +00:08:09.940 --> 00:08:11.020 +who are very busy people, + +00:08:11.160 --> 00:08:12.980 +who deal with a lot of other aspects + +00:08:12.980 --> 00:08:14.100 +of the language other than typing. + +00:08:14.640 --> 00:08:17.360 +So Shantanu and I came up with this idea + +00:08:17.360 --> 00:08:19.000 +of creating a separate council + +00:08:19.000 --> 00:08:21.040 +to specifically in charge of typing, + +00:08:21.040 --> 00:08:22.840 +that would be in a specification + +00:08:22.840 --> 00:08:25.080 +where we can make small changes ourselves + +00:08:25.080 --> 00:08:26.000 +without having to go through + +00:08:26.000 --> 00:08:27.100 +this whole PEP process. + +00:08:27.520 --> 00:08:29.340 +And this way, when all the type checkers + +00:08:29.340 --> 00:08:31.360 +agreed that something needs to go a certain way + +00:08:31.360 --> 00:08:32.940 +and it's not exactly what's in the PEPs, + +00:08:33.420 --> 00:08:36.520 +we can change it and have a place to record that + +00:08:36.520 --> 00:08:37.680 +and people can refer to it + +00:08:37.680 --> 00:08:39.720 +and new type checkers can also try + +00:08:39.720 --> 00:08:40.740 +to follow those decisions. + +00:08:41.060 --> 00:08:41.480 +Very interesting. + +00:08:41.680 --> 00:08:43.540 +I didn't realize that it was sort of, + +00:08:43.540 --> 00:08:46.360 +was there to allow for small changes + +00:08:46.360 --> 00:08:47.920 +to be made to make that much easier. + +00:08:48.040 --> 00:08:49.140 +But of course that makes sense + +00:08:49.140 --> 00:08:50.360 +because the PEP process is, + +00:08:50.540 --> 00:08:52.360 +it's pretty serious and drawn out. + +00:08:52.420 --> 00:08:55.560 +And we've seen even small language changes + +00:08:55.560 --> 00:08:58.080 +have quite passionate folks, + +00:08:58.180 --> 00:08:59.800 +I guess we should say. + +00:09:00.140 --> 00:09:01.620 +So yeah, yeah, very nice. + +00:09:02.020 --> 00:09:04.180 +Do you have any examples of the types of changes + +00:09:04.180 --> 00:09:05.080 +that y'all have, + +00:09:05.420 --> 00:09:06.600 +that have happened over the years + +00:09:06.600 --> 00:09:09.020 +that maybe were typing council only? + +00:09:09.140 --> 00:09:09.980 +One was the specification + +00:09:09.980 --> 00:09:11.660 +where how overloads work, + +00:09:11.780 --> 00:09:13.940 +which is perhaps not really a small change, + +00:09:14.040 --> 00:09:16.040 +but one of the most complicated features + +00:09:16.040 --> 00:09:17.940 +in the type system really is the overloads, + +00:09:17.940 --> 00:09:19.680 +where you can give multiple signatures + +00:09:19.680 --> 00:09:20.540 +for a function + +00:09:20.540 --> 00:09:22.820 +and type checkers sort of select + +00:09:22.820 --> 00:09:25.080 +which one to use based on the arguments + +00:09:25.080 --> 00:09:26.280 +when the function is called. + +00:09:26.820 --> 00:09:28.080 +And when it was initially created, + +00:09:28.460 --> 00:09:29.400 +from what I recall, + +00:09:29.640 --> 00:09:31.120 +there just wasn't really a specification. + +00:09:31.540 --> 00:09:33.480 +It's just like you use the signatures + +00:09:33.480 --> 00:09:34.960 +in a way that makes sense. + +00:09:35.420 --> 00:09:36.200 +And Eric Trout, + +00:09:36.280 --> 00:09:37.400 +who's currently on the council, + +00:09:37.620 --> 00:09:40.080 +came up with a pretty specific procedure + +00:09:40.080 --> 00:09:42.000 +for exactly how overload should work + +00:09:42.000 --> 00:09:44.720 +to make it so that type checkers have, + +00:09:45.160 --> 00:09:46.620 +well, sort of users can understand how it works + +00:09:46.620 --> 00:09:47.820 +and sort of type checkers can have something + +00:09:47.820 --> 00:09:49.000 +to work towards to make sure + +00:09:49.000 --> 00:09:51.220 +that they work infinite overloads in the same way. + +00:09:51.220 --> 00:09:52.480 +Maybe a smaller example + +00:09:52.480 --> 00:09:54.100 +that is an example of something + +00:09:54.100 --> 00:09:56.140 +that would have been too small for a pep + +00:09:56.140 --> 00:09:57.660 +and hard to accomplish + +00:09:57.660 --> 00:09:59.780 +before the typing council existed. + +00:10:00.420 --> 00:10:01.600 +And this is actually a change + +00:10:01.600 --> 00:10:03.680 +that I pushed through before the, + +00:10:04.220 --> 00:10:05.440 +before I was on the typing council, + +00:10:05.560 --> 00:10:06.980 +but the typing council approved it, + +00:10:07.240 --> 00:10:08.780 +was a clarification around + +00:10:08.780 --> 00:10:11.620 +the interpretation of data class fields. + +00:10:11.920 --> 00:10:13.860 +If a final annotation is applied + +00:10:13.860 --> 00:10:15.080 +to a data class field, + +00:10:15.680 --> 00:10:16.500 +does that mean, + +00:10:16.980 --> 00:10:18.800 +so if you apply a final annotation + +00:10:18.800 --> 00:10:20.340 +to a regular class attribute, + +00:10:20.880 --> 00:10:22.120 +since it can't be changed, + +00:10:22.260 --> 00:10:24.260 +that implies that it's a class variable. + +00:10:24.620 --> 00:10:25.720 +And there was a question of + +00:10:25.720 --> 00:10:26.900 +if that should be the interpretation + +00:10:26.900 --> 00:10:28.060 +with the data class or not. + +00:10:28.360 --> 00:10:29.360 +So we discussed that + +00:10:29.360 --> 00:10:31.320 +and made a clarification to the spec. + +00:10:31.420 --> 00:10:32.740 +I've never really thought about final + +00:10:32.740 --> 00:10:35.700 +being applied to a class field, + +00:10:35.840 --> 00:10:36.960 +but I've always used them + +00:10:36.960 --> 00:10:38.220 +sort of just for constants. + +00:10:38.520 --> 00:10:38.840 +But, you know, + +00:10:38.860 --> 00:10:40.320 +maybe people out there don't know, + +00:10:40.320 --> 00:10:43.320 +like typing dot final bracket type, + +00:10:43.540 --> 00:10:43.800 +right? + +00:10:44.260 --> 00:10:45.360 +That's kind of the way + +00:10:45.360 --> 00:10:47.160 +you can do constants in Python, right? + +00:10:47.360 --> 00:10:48.440 +Constants for the type checker. + +00:10:48.860 --> 00:10:49.560 +Nothing in the runtime + +00:10:49.560 --> 00:10:50.660 +will stop you from editing it. + +00:10:51.080 --> 00:10:51.400 +That's... + +00:10:51.400 --> 00:10:51.980 +Not there. + +00:10:52.200 --> 00:10:52.820 +Not there. + +00:10:52.920 --> 00:10:54.460 +I have some examples coming up + +00:10:54.460 --> 00:10:55.660 +and I'm interested + +00:10:55.660 --> 00:10:56.620 +to hear your thoughts on it, + +00:10:56.640 --> 00:10:58.160 +but for sure it's, + +00:10:58.520 --> 00:10:59.860 +there is this tension, right? + +00:11:00.000 --> 00:11:01.340 +I mean, I think that's probably + +00:11:01.340 --> 00:11:02.500 +worth touching on as well + +00:11:02.500 --> 00:11:04.140 +is this is a tension for Python + +00:11:04.140 --> 00:11:05.280 +in general is + +00:11:05.280 --> 00:11:07.380 +you can write all the types you want + +00:11:07.380 --> 00:11:09.160 +and then when you run your code, + +00:11:09.280 --> 00:11:10.900 +it just doesn't care. + +00:11:11.000 --> 00:11:11.940 +There's a few instances, + +00:11:12.360 --> 00:11:14.020 +Pydantic, FastAPI, + +00:11:14.160 --> 00:11:14.600 +a few others, + +00:11:14.720 --> 00:11:15.740 +but generally speaking, + +00:11:15.960 --> 00:11:17.200 +it's there for the editors + +00:11:17.200 --> 00:11:18.180 +and the type checkers + +00:11:18.180 --> 00:11:18.760 +and the linters + +00:11:18.760 --> 00:11:20.460 +and not for runtime, right? + +00:11:20.660 --> 00:11:21.280 +Yeah, that's right. + +00:11:21.580 --> 00:11:23.440 +There's many exceptions to that. + +00:11:23.660 --> 00:11:25.160 +There's a product like mypyC, + +00:11:25.380 --> 00:11:26.460 +which comes with mypy + +00:11:26.460 --> 00:11:27.580 +that's used those types + +00:11:27.580 --> 00:11:28.660 +to compile your code + +00:11:28.660 --> 00:11:30.600 +into more efficient machine codes. + +00:11:30.920 --> 00:11:31.520 +Maybe there's going to be + +00:11:31.520 --> 00:11:32.240 +more products like that + +00:11:32.240 --> 00:11:32.660 +in the future. + +00:11:32.760 --> 00:11:33.220 +I don't know. + +00:11:33.540 --> 00:11:34.220 +But yes, in general, + +00:11:34.400 --> 00:11:35.980 +it's separate from the runtime. + +00:11:36.520 --> 00:11:37.060 +Sort of a similar + +00:11:37.060 --> 00:11:38.380 +model to TypeScript + +00:11:38.380 --> 00:11:39.820 +where TypeScript + +00:11:39.820 --> 00:11:40.940 +gets compiled into JavaScript + +00:11:40.940 --> 00:11:42.020 +and types just go away. + +00:11:42.620 --> 00:11:43.260 +Here, we don't do + +00:11:43.260 --> 00:11:44.060 +a compilation step, + +00:11:44.180 --> 00:11:45.120 +but still the same idea + +00:11:45.120 --> 00:11:45.720 +of the types + +00:11:45.720 --> 00:11:47.180 +just not influencing the runtime. + +00:11:47.420 --> 00:11:48.300 +Although we do make them + +00:11:48.300 --> 00:11:49.700 +available for introspection + +00:11:49.700 --> 00:11:51.500 +via done annotations attributes, + +00:11:51.500 --> 00:11:52.740 +which is what has enabled + +00:11:52.740 --> 00:11:54.420 +the projects like Pydantic + +00:11:54.420 --> 00:11:55.940 +and other sort of + +00:11:55.940 --> 00:11:56.840 +runtime checkers + +00:11:56.840 --> 00:11:58.660 +to make use of type annotations + +00:11:58.660 --> 00:11:59.400 +at runtime also. + +00:11:59.540 --> 00:12:00.000 +Yeah, I don't know + +00:12:00.000 --> 00:12:01.060 +if the typing council + +00:12:01.060 --> 00:12:02.020 +was around for this, + +00:12:02.120 --> 00:12:03.340 +but there was proposed, + +00:12:03.480 --> 00:12:03.780 +I don't remember + +00:12:03.780 --> 00:12:04.520 +the exact details, + +00:12:04.620 --> 00:12:05.660 +but something to the effect + +00:12:05.660 --> 00:12:07.420 +of for type checking, + +00:12:07.640 --> 00:12:08.760 +not actually doing + +00:12:08.760 --> 00:12:11.040 +some of the full imports + +00:12:11.040 --> 00:12:12.840 +or something along those lines, + +00:12:13.200 --> 00:12:13.420 +right, + +00:12:13.460 --> 00:12:15.080 +where the runtime behavior + +00:12:15.080 --> 00:12:15.960 +would have made it hard + +00:12:15.960 --> 00:12:17.420 +for tools like Pydantic + +00:12:17.420 --> 00:12:18.940 +and others to get that. + +00:12:19.300 --> 00:12:20.000 +And there was + +00:12:20.000 --> 00:12:21.040 +some kind of compromise, + +00:12:21.180 --> 00:12:21.280 +right? + +00:12:21.280 --> 00:12:22.600 +I don't remember the details here. + +00:12:22.720 --> 00:12:23.140 +Anyone does? + +00:12:23.260 --> 00:12:23.960 +Yeah, what happened was + +00:12:23.960 --> 00:12:24.540 +that there was going + +00:12:24.540 --> 00:12:25.180 +to be a change. + +00:12:25.600 --> 00:12:26.240 +That's what the + +00:12:26.240 --> 00:12:26.980 +from future import + +00:12:26.980 --> 00:12:28.040 +annotations import does, + +00:12:28.100 --> 00:12:29.580 +that changes all annotations + +00:12:29.580 --> 00:12:30.600 +into raw strings. + +00:12:30.600 --> 00:12:32.540 +So the default behavior + +00:12:32.540 --> 00:12:34.260 +before recently + +00:12:34.260 --> 00:12:35.400 +was that annotations + +00:12:35.400 --> 00:12:36.500 +that are regular codes. + +00:12:36.660 --> 00:12:37.760 +If you write devf + +00:12:37.760 --> 00:12:38.700 +return to ints + +00:12:38.700 --> 00:12:39.800 +and you import the module, + +00:12:39.900 --> 00:12:40.720 +it just looks up + +00:12:40.720 --> 00:12:41.400 +the name ints + +00:12:41.400 --> 00:12:41.920 +and puts that + +00:12:41.920 --> 00:12:43.000 +in an annotations dictionary, + +00:12:43.420 --> 00:12:44.780 +which makes introspection easy, + +00:12:44.940 --> 00:12:45.680 +but it made + +00:12:45.680 --> 00:12:47.100 +a test on costs + +00:12:47.100 --> 00:12:48.400 +on performance + +00:12:48.400 --> 00:12:49.520 +because memory usage + +00:12:49.520 --> 00:12:50.260 +sometimes was high + +00:12:50.260 --> 00:12:51.580 +and also made things + +00:12:51.580 --> 00:12:52.720 +harder to use sometimes + +00:12:52.720 --> 00:12:54.340 +because if you use a name + +00:12:54.340 --> 00:12:55.420 +that's not defined yet + +00:12:55.420 --> 00:12:56.020 +at runtime, + +00:12:56.540 --> 00:12:57.120 +you get an error. + +00:12:57.120 --> 00:12:58.160 +That often comes up + +00:12:58.160 --> 00:12:59.700 +if you have like a class + +00:12:59.700 --> 00:13:00.940 +that has a reference + +00:13:00.940 --> 00:13:01.760 +in an adaptation + +00:13:01.760 --> 00:13:02.820 +to the class itself + +00:13:02.820 --> 00:13:03.840 +or circular + +00:13:03.840 --> 00:13:05.420 +dependency classes. + +00:13:05.880 --> 00:13:05.980 +Right. + +00:13:06.080 --> 00:13:07.120 +The circular imports + +00:13:07.120 --> 00:13:08.640 +because you want to say + +00:13:08.640 --> 00:13:09.720 +this class + +00:13:09.720 --> 00:13:11.520 +is created by that thing + +00:13:11.520 --> 00:13:12.760 +and it returns one, + +00:13:12.840 --> 00:13:13.360 +but you know, + +00:13:13.420 --> 00:13:15.020 +somehow you've got + +00:13:15.020 --> 00:13:15.740 +to import the other one + +00:13:15.740 --> 00:13:17.040 +and that's such a hassle. + +00:13:17.420 --> 00:13:17.860 +Yeah, it's, + +00:13:18.220 --> 00:13:18.780 +yeah, even out + +00:13:18.780 --> 00:13:19.900 +in the audience we have, + +00:13:20.280 --> 00:13:20.740 +Tom says, + +00:13:20.820 --> 00:13:21.500 +circular imports. + +00:13:21.620 --> 00:13:22.480 +Oh, yeah, for sure. + +00:13:22.800 --> 00:13:24.260 +What about lazy imports? + +00:13:24.260 --> 00:13:25.420 +Like that just recently + +00:13:25.420 --> 00:13:26.040 +got accepted + +00:13:26.040 --> 00:13:27.000 +and will be in 3.15. + +00:13:27.620 --> 00:13:29.320 +Which I'm super excited about + +00:13:29.320 --> 00:13:29.980 +because I think + +00:13:29.980 --> 00:13:31.140 +it'll make app startup + +00:13:31.140 --> 00:13:32.100 +a lot faster + +00:13:32.100 --> 00:13:33.980 +for many use cases. + +00:13:34.400 --> 00:13:35.280 +But does that have + +00:13:35.280 --> 00:13:35.980 +knock-on effects + +00:13:35.980 --> 00:13:36.480 +for typing? + +00:13:36.800 --> 00:13:37.860 +Not that directly + +00:13:37.860 --> 00:13:39.040 +because I think + +00:13:39.040 --> 00:13:39.740 +for a type checker + +00:13:39.740 --> 00:13:40.380 +lazy imports + +00:13:40.380 --> 00:13:41.160 +mostly just look + +00:13:41.160 --> 00:13:42.320 +like regular imports. + +00:13:42.680 --> 00:13:43.180 +I guess I should + +00:13:43.180 --> 00:13:43.920 +maybe leave that + +00:13:43.920 --> 00:13:44.320 +for the people + +00:13:44.320 --> 00:13:44.740 +who are actually + +00:13:44.740 --> 00:13:46.040 +working on type checkers + +00:13:46.040 --> 00:13:47.060 +and being written right now. + +00:13:47.160 --> 00:13:47.660 +Yeah, Rebecca, + +00:13:47.880 --> 00:13:48.560 +do you see this + +00:13:48.560 --> 00:13:49.600 +making any difference + +00:13:49.600 --> 00:13:50.040 +for you? + +00:13:50.540 --> 00:13:51.060 +Lazy imports? + +00:13:51.060 --> 00:13:51.920 +To be honest, + +00:13:52.080 --> 00:13:53.480 +it's not something + +00:13:53.480 --> 00:13:54.860 +we've looked at + +00:13:54.860 --> 00:13:55.760 +too carefully yet. + +00:13:55.760 --> 00:13:56.760 +3.15 + +00:13:56.760 --> 00:13:57.740 +seems a little + +00:13:57.740 --> 00:13:59.080 +more in the future, + +00:13:59.380 --> 00:14:01.460 +but I don't think + +00:14:01.460 --> 00:14:02.220 +it's likely + +00:14:02.220 --> 00:14:03.680 +to make a huge difference. + +00:14:03.960 --> 00:14:04.180 +Carl? + +00:14:04.340 --> 00:14:05.300 +I've thought about it briefly + +00:14:05.300 --> 00:14:06.260 +and I think that it, + +00:14:06.560 --> 00:14:07.400 +I think the type checkers + +00:14:07.400 --> 00:14:08.540 +really won't need to care. + +00:14:08.920 --> 00:14:09.520 +Maybe there will be + +00:14:09.520 --> 00:14:10.140 +some edge cases + +00:14:10.140 --> 00:14:10.660 +that will come up + +00:14:10.660 --> 00:14:11.420 +that I haven't thought of, + +00:14:11.500 --> 00:14:12.500 +but it shouldn't be a big deal. + +00:14:12.620 --> 00:14:13.380 +Yeah, that's what I thought + +00:14:13.380 --> 00:14:13.920 +as well. + +00:14:14.100 --> 00:14:15.660 +The one variation + +00:14:15.660 --> 00:14:17.080 +that I can certainly see + +00:14:17.080 --> 00:14:18.620 +is if you have a, + +00:14:19.040 --> 00:14:20.260 +if you have something + +00:14:20.260 --> 00:14:21.600 +specified in a type, + +00:14:21.920 --> 00:14:23.580 +like say for a field + +00:14:23.580 --> 00:14:24.260 +of a class + +00:14:24.260 --> 00:14:25.260 +or a Pydantic model + +00:14:25.260 --> 00:14:25.640 +or something + +00:14:25.640 --> 00:14:26.800 +that would otherwise + +00:14:26.800 --> 00:14:27.840 +not trigger + +00:14:27.840 --> 00:14:28.780 +the lazy import + +00:14:28.780 --> 00:14:30.100 +to become imported, + +00:14:30.460 --> 00:14:31.180 +would potentially + +00:14:31.180 --> 00:14:33.220 +having types specify + +00:14:33.220 --> 00:14:34.720 +cause more importing + +00:14:34.720 --> 00:14:36.000 +to happen sooner + +00:14:36.000 --> 00:14:36.640 +in the runtime? + +00:14:36.920 --> 00:14:37.520 +Yeah, there's actually + +00:14:37.520 --> 00:14:38.460 +an issue related to this + +00:14:38.460 --> 00:14:39.980 +that I think we may need + +00:14:39.980 --> 00:14:41.540 +to resolve before 3.15, + +00:14:41.940 --> 00:14:42.840 +but I don't know how yet. + +00:14:43.200 --> 00:14:44.120 +If you use a type + +00:14:44.120 --> 00:14:45.120 +in a data class annotation + +00:14:45.120 --> 00:14:46.260 +that's lazy imported, + +00:14:46.580 --> 00:14:47.220 +actually creating + +00:14:47.220 --> 00:14:47.920 +a data class + +00:14:47.920 --> 00:14:48.820 +will delay + +00:14:48.820 --> 00:14:49.520 +by the import. + +00:14:49.520 --> 00:14:50.660 +It will try to + +00:14:50.660 --> 00:14:52.760 +resolve the import + +00:14:52.760 --> 00:14:54.140 +and actually make it + +00:14:54.140 --> 00:14:54.520 +not lazy. + +00:14:55.000 --> 00:14:55.700 +This is because + +00:14:55.700 --> 00:14:56.640 +data classes + +00:14:56.640 --> 00:14:57.760 +doesn't really need + +00:14:57.760 --> 00:14:58.280 +to look at all + +00:14:58.280 --> 00:14:58.960 +of the annotations + +00:14:58.960 --> 00:14:59.620 +in your class, + +00:14:59.700 --> 00:15:00.300 +but it looks at them + +00:15:00.300 --> 00:15:01.840 +enough to trigger + +00:15:01.840 --> 00:15:03.860 +reification of the import. + +00:15:04.120 --> 00:15:04.740 +I shared this + +00:15:04.740 --> 00:15:05.220 +with some of the people + +00:15:05.220 --> 00:15:06.160 +on the lazy imports team, + +00:15:06.260 --> 00:15:07.460 +but we haven't + +00:15:07.460 --> 00:15:08.200 +yet come up + +00:15:08.200 --> 00:15:09.120 +with a good way around it. + +00:15:09.240 --> 00:15:09.980 +I think this might + +00:15:09.980 --> 00:15:10.980 +end up being + +00:15:10.980 --> 00:15:11.740 +a bit of a food gun, + +00:15:11.820 --> 00:15:12.260 +so I feel like + +00:15:12.260 --> 00:15:12.800 +we should ideally + +00:15:12.800 --> 00:15:13.880 +find a workaround, + +00:15:14.060 --> 00:15:14.720 +but I don't know + +00:15:14.720 --> 00:15:15.420 +what it would be yet. + +00:15:15.580 --> 00:15:15.920 +I don't know + +00:15:15.920 --> 00:15:16.680 +that it's wrong + +00:15:16.680 --> 00:15:17.900 +that it converts it + +00:15:17.900 --> 00:15:18.820 +to an eager import, + +00:15:18.820 --> 00:15:20.000 +which it needs + +00:15:20.000 --> 00:15:20.740 +to know what it is + +00:15:20.740 --> 00:15:21.200 +potentially, + +00:15:21.500 --> 00:15:21.820 +right? + +00:15:22.560 --> 00:15:23.660 +It actually doesn't. + +00:15:24.000 --> 00:15:24.420 +Data classes + +00:15:24.420 --> 00:15:25.240 +just need to know + +00:15:25.240 --> 00:15:26.080 +whether it is classed + +00:15:26.080 --> 00:15:26.600 +for or not. + +00:15:27.400 --> 00:15:27.960 +I think that's + +00:15:27.960 --> 00:15:28.520 +pretty much all. + +00:15:28.660 --> 00:15:28.960 +I guess there's + +00:15:28.960 --> 00:15:29.640 +an init for also, + +00:15:29.780 --> 00:15:30.760 +but it doesn't really + +00:15:30.760 --> 00:15:31.080 +need to know + +00:15:31.080 --> 00:15:31.800 +anything else. + +00:15:32.320 --> 00:15:33.220 +So in theory, + +00:15:33.280 --> 00:15:33.900 +it should be possible + +00:15:33.900 --> 00:15:34.640 +to just say, + +00:15:34.800 --> 00:15:35.700 +hey, it is not classed + +00:15:35.700 --> 00:15:37.240 +for, so don't bother + +00:15:37.240 --> 00:15:37.860 +importing it. + +00:15:38.100 --> 00:15:39.000 +Okay, so that's + +00:15:39.000 --> 00:15:39.660 +for data classes, + +00:15:39.820 --> 00:15:41.320 +but say if I specify + +00:15:41.320 --> 00:15:42.400 +a parameter type + +00:15:42.400 --> 00:15:43.180 +on a function. + +00:15:43.420 --> 00:15:43.880 +Yeah, then it + +00:15:43.880 --> 00:15:44.420 +should be fine. + +00:15:45.100 --> 00:15:46.280 +I guess, again, + +00:15:46.380 --> 00:15:47.540 +unless something is, + +00:15:47.960 --> 00:15:48.800 +if it does annotate, + +00:15:48.820 --> 00:15:49.580 +so if you have + +00:15:49.580 --> 00:15:49.920 +something like + +00:15:49.920 --> 00:15:50.480 +a decorator + +00:15:50.480 --> 00:15:50.960 +that looks at + +00:15:50.960 --> 00:15:51.880 +annotations in your + +00:15:51.880 --> 00:15:52.320 +function, + +00:15:52.660 --> 00:15:53.660 +it might reify + +00:15:53.660 --> 00:15:54.220 +those imports. + +00:15:54.540 --> 00:15:55.120 +There is one other + +00:15:55.120 --> 00:15:56.160 +potentially interesting + +00:15:56.160 --> 00:15:57.200 +thing for type checkers. + +00:15:57.440 --> 00:15:58.040 +It's already + +00:15:58.040 --> 00:15:59.880 +difficult for type checkers + +00:15:59.880 --> 00:16:00.620 +to figure out + +00:16:00.620 --> 00:16:02.480 +when like a submodule + +00:16:02.480 --> 00:16:03.380 +should be considered + +00:16:03.380 --> 00:16:04.380 +to be an attribute + +00:16:04.380 --> 00:16:05.340 +of the parent module + +00:16:05.340 --> 00:16:06.380 +because the way + +00:16:06.380 --> 00:16:07.260 +this happens in Python + +00:16:07.260 --> 00:16:08.440 +is that any import + +00:16:08.440 --> 00:16:09.440 +of a submodule + +00:16:09.440 --> 00:16:10.420 +anywhere will attach + +00:16:10.420 --> 00:16:11.360 +that submodule + +00:16:11.360 --> 00:16:12.280 +as an attribute + +00:16:12.280 --> 00:16:13.220 +on the parent module, + +00:16:13.540 --> 00:16:15.100 +but that at runtime, + +00:16:15.560 --> 00:16:16.560 +that could literally + +00:16:16.560 --> 00:16:17.220 +happen anywhere. + +00:16:17.220 --> 00:16:17.780 +It could happen + +00:16:17.780 --> 00:16:18.540 +in totally unrelated + +00:16:18.540 --> 00:16:19.860 +code outside of the module + +00:16:19.860 --> 00:16:20.680 +and a type checker + +00:16:20.680 --> 00:16:21.380 +probably won't be able + +00:16:21.380 --> 00:16:21.920 +to see that. + +00:16:22.220 --> 00:16:22.820 +So type checkers + +00:16:22.820 --> 00:16:23.740 +already have sort of + +00:16:23.740 --> 00:16:24.800 +complex sets of rules + +00:16:24.800 --> 00:16:26.080 +around where they look + +00:16:26.080 --> 00:16:27.020 +for these submodule + +00:16:27.020 --> 00:16:27.920 +imports and when they + +00:16:27.920 --> 00:16:28.980 +consider a submodule + +00:16:28.980 --> 00:16:30.600 +import to be reliably + +00:16:30.600 --> 00:16:31.740 +happening enough + +00:16:31.740 --> 00:16:32.360 +that it should, + +00:16:32.480 --> 00:16:33.160 +that the type checker + +00:16:33.160 --> 00:16:34.800 +should consider + +00:16:34.800 --> 00:16:35.620 +this submodule + +00:16:35.620 --> 00:16:36.900 +to exist as an attribute. + +00:16:38.080 --> 00:16:38.740 +And lazy imports + +00:16:38.740 --> 00:16:40.200 +may make that even, + +00:16:40.660 --> 00:16:41.500 +we'll add one more + +00:16:41.500 --> 00:16:41.820 +wrinkle + +00:16:41.820 --> 00:16:43.140 +to those + +00:16:43.140 --> 00:16:44.080 +sets of heuristics + +00:16:44.080 --> 00:16:44.820 +in that we'll have + +00:16:44.820 --> 00:16:45.300 +to decide + +00:16:45.300 --> 00:16:46.060 +if you have + +00:16:46.060 --> 00:16:46.760 +a lazy import + +00:16:46.760 --> 00:16:47.540 +of a submodule + +00:16:47.540 --> 00:16:47.940 +and you're done + +00:16:47.940 --> 00:16:48.720 +to init.py, + +00:16:48.860 --> 00:16:49.900 +it's lazy. + +00:16:50.420 --> 00:16:51.480 +So should the type checker + +00:16:51.480 --> 00:16:53.180 +consider that submodule + +00:16:53.180 --> 00:16:54.520 +to be imported + +00:16:54.520 --> 00:16:55.400 +or not be imported? + +00:16:55.860 --> 00:16:57.000 +It'll be another case + +00:16:57.000 --> 00:16:57.500 +where there's no + +00:16:57.500 --> 00:16:58.320 +clear right answer + +00:16:58.320 --> 00:16:58.920 +and we'll just have + +00:16:58.920 --> 00:17:00.020 +to make a decision + +00:17:00.020 --> 00:17:00.700 +one way or the other. + +00:17:02.700 --> 00:17:03.200 +This portion of + +00:17:03.200 --> 00:17:04.060 +Talk Python is brought + +00:17:04.060 --> 00:17:04.840 +to you by Sentry. + +00:17:05.060 --> 00:17:05.980 +I've been using + +00:17:05.980 --> 00:17:06.920 +Sentry personally + +00:17:06.920 --> 00:17:07.920 +on almost every + +00:17:07.920 --> 00:17:09.180 +application and API + +00:17:09.180 --> 00:17:10.000 +that I've built + +00:17:10.000 --> 00:17:10.860 +for Talk Python + +00:17:10.860 --> 00:17:11.420 +and beyond + +00:17:11.420 --> 00:17:13.040 +over the last few years. + +00:17:13.040 --> 00:17:13.900 +They're a core + +00:17:13.900 --> 00:17:14.540 +building block + +00:17:14.540 --> 00:17:14.980 +for keeping + +00:17:14.980 --> 00:17:15.720 +my infrastructure + +00:17:15.720 --> 00:17:16.260 +solid. + +00:17:16.780 --> 00:17:17.300 +They should be + +00:17:17.300 --> 00:17:18.140 +for yours as well. + +00:17:18.300 --> 00:17:18.780 +Here's why. + +00:17:19.420 --> 00:17:20.020 +Sentry doesn't + +00:17:20.020 --> 00:17:20.880 +just catch errors. + +00:17:21.020 --> 00:17:21.900 +It catches all + +00:17:21.900 --> 00:17:22.300 +the stuff + +00:17:22.300 --> 00:17:22.820 +that makes your + +00:17:22.820 --> 00:17:23.700 +app feel broken. + +00:17:23.940 --> 00:17:24.860 +The random slowdown, + +00:17:25.080 --> 00:17:25.680 +the freeze you + +00:17:25.680 --> 00:17:26.320 +can't reproduce, + +00:17:26.840 --> 00:17:27.500 +that bug that + +00:17:27.500 --> 00:17:28.100 +only shows up + +00:17:28.100 --> 00:17:29.080 +once real users + +00:17:29.080 --> 00:17:29.440 +hit it. + +00:17:29.700 --> 00:17:30.220 +And when something + +00:17:30.220 --> 00:17:30.700 +goes wrong, + +00:17:30.900 --> 00:17:31.480 +Sentry gives you + +00:17:31.480 --> 00:17:32.140 +the whole chain + +00:17:32.140 --> 00:17:32.600 +of events + +00:17:32.600 --> 00:17:33.260 +in one place. + +00:17:33.420 --> 00:17:33.740 +Errors, + +00:17:33.920 --> 00:17:34.200 +traces, + +00:17:34.460 --> 00:17:34.940 +replays, + +00:17:35.000 --> 00:17:35.320 +logs, + +00:17:35.740 --> 00:17:36.460 +dots connected. + +00:17:36.800 --> 00:17:37.360 +You can see + +00:17:37.360 --> 00:17:38.320 +what's led to + +00:17:38.320 --> 00:17:38.720 +the issue + +00:17:38.720 --> 00:17:39.320 +without digging + +00:17:39.320 --> 00:17:39.880 +through five + +00:17:39.880 --> 00:17:40.660 +different dashboards. + +00:17:41.400 --> 00:17:41.660 +Sear, + +00:17:42.000 --> 00:17:42.740 +Sentry's AI + +00:17:42.740 --> 00:17:43.520 +debugging agents + +00:17:43.520 --> 00:17:44.480 +builds on this + +00:17:44.480 --> 00:17:44.700 +data, + +00:17:44.820 --> 00:17:45.420 +taking the full + +00:17:45.420 --> 00:17:46.020 +context, + +00:17:46.640 --> 00:17:47.580 +explaining why + +00:17:47.580 --> 00:17:48.120 +the issue + +00:17:48.120 --> 00:17:48.600 +happened, + +00:17:49.140 --> 00:17:49.740 +pointing to + +00:17:49.740 --> 00:17:50.180 +the code + +00:17:50.180 --> 00:17:50.720 +responsible, + +00:17:50.940 --> 00:17:51.560 +drafts a fix, + +00:17:51.640 --> 00:17:52.320 +and even flags + +00:17:52.320 --> 00:17:52.960 +if your PR + +00:17:52.960 --> 00:17:53.620 +is about to + +00:17:53.620 --> 00:17:54.340 +introduce a new + +00:17:54.340 --> 00:17:54.640 +problem. + +00:17:55.020 --> 00:17:55.880 +The workflow + +00:17:55.880 --> 00:17:56.520 +stays simple. + +00:17:56.820 --> 00:17:57.460 +Something breaks, + +00:17:57.920 --> 00:17:58.680 +Sentry alerts you, + +00:17:58.840 --> 00:17:59.360 +the dashboard + +00:17:59.360 --> 00:17:59.860 +shows you + +00:17:59.860 --> 00:18:00.680 +the full context, + +00:18:01.020 --> 00:18:01.680 +Sear helps you + +00:18:01.680 --> 00:18:02.400 +fix it and + +00:18:02.400 --> 00:18:03.380 +catch new issues + +00:18:03.380 --> 00:18:04.100 +before they ship. + +00:18:04.760 --> 00:18:05.400 +It's totally + +00:18:05.400 --> 00:18:06.140 +reasonable to go + +00:18:06.140 --> 00:18:07.160 +from an error + +00:18:07.160 --> 00:18:07.980 +occurred to + +00:18:07.980 --> 00:18:08.880 +fixed in production + +00:18:08.880 --> 00:18:10.000 +in just 10 minutes. + +00:18:10.000 --> 00:18:11.360 +I truly + +00:18:11.360 --> 00:18:11.980 +appreciate the + +00:18:11.980 --> 00:18:12.460 +support that + +00:18:12.460 --> 00:18:12.920 +Sentry has + +00:18:12.920 --> 00:18:13.980 +given me to + +00:18:13.980 --> 00:18:14.720 +help solve my + +00:18:14.720 --> 00:18:15.560 +bugs and issues + +00:18:15.560 --> 00:18:16.340 +in my apps, + +00:18:16.880 --> 00:18:17.620 +especially those + +00:18:17.620 --> 00:18:18.460 +tricky ones that + +00:18:18.460 --> 00:18:19.180 +only appear in + +00:18:19.180 --> 00:18:19.500 +production. + +00:18:19.840 --> 00:18:20.420 +I know you will + +00:18:20.420 --> 00:18:21.020 +too if you try + +00:18:21.020 --> 00:18:21.360 +them out. + +00:18:21.640 --> 00:18:22.520 +So get started + +00:18:22.520 --> 00:18:23.020 +today with + +00:18:23.020 --> 00:18:23.280 +Sentry. + +00:18:23.460 --> 00:18:23.940 +Just visit + +00:18:23.940 --> 00:18:24.940 +talkpython.fm + +00:18:24.940 --> 00:18:25.860 +slash Sentry + +00:18:25.860 --> 00:18:27.080 +and get $100 + +00:18:27.080 --> 00:18:28.560 +in Sentry credits. + +00:18:28.960 --> 00:18:29.520 +Please use that + +00:18:29.520 --> 00:18:29.720 +link. + +00:18:29.780 --> 00:18:30.100 +It's in your + +00:18:30.100 --> 00:18:30.640 +podcast player + +00:18:30.640 --> 00:18:31.080 +show notes. + +00:18:31.280 --> 00:18:31.500 +If you're + +00:18:31.500 --> 00:18:31.920 +signing up + +00:18:31.920 --> 00:18:32.360 +some other + +00:18:32.360 --> 00:18:32.620 +way, + +00:18:32.740 --> 00:18:33.420 +you can use + +00:18:33.420 --> 00:18:33.880 +our code + +00:18:33.880 --> 00:18:35.220 +talkpython26, + +00:18:35.780 --> 00:18:36.520 +all one word, + +00:18:36.660 --> 00:18:37.620 +talkpython26, + +00:18:37.620 --> 00:18:38.880 +to get $100 + +00:18:38.880 --> 00:18:39.720 +in credits. + +00:18:40.380 --> 00:18:40.900 +Thank you to + +00:18:40.900 --> 00:18:41.460 +Sentry for + +00:18:41.460 --> 00:18:41.940 +supporting the + +00:18:41.940 --> 00:18:42.120 +show. + +00:18:43.160 --> 00:18:43.920 +Yeah, there's + +00:18:43.920 --> 00:18:44.540 +some variations + +00:18:44.540 --> 00:18:45.280 +across type + +00:18:45.280 --> 00:18:45.840 +checkers, which + +00:18:45.840 --> 00:18:46.540 +we'll get to + +00:18:46.540 --> 00:18:47.200 +later. + +00:18:47.600 --> 00:18:48.020 +I think, + +00:18:48.140 --> 00:18:48.700 +though, before + +00:18:48.700 --> 00:18:49.440 +we move off + +00:18:49.440 --> 00:18:50.420 +this, there's + +00:18:50.420 --> 00:18:51.960 +actually off + +00:18:51.960 --> 00:18:52.740 +introducing the + +00:18:52.740 --> 00:18:53.400 +typing council. + +00:18:53.560 --> 00:18:54.220 +I think we + +00:18:54.220 --> 00:18:54.680 +should point out + +00:18:54.680 --> 00:18:55.720 +that there's two + +00:18:55.720 --> 00:18:56.380 +other folks who + +00:18:56.380 --> 00:18:56.980 +couldn't be here + +00:18:56.980 --> 00:18:58.240 +who are also on + +00:18:58.240 --> 00:18:58.760 +the typing + +00:18:58.760 --> 00:18:59.560 +council, Eric + +00:18:59.560 --> 00:19:00.280 +Trout and + +00:19:00.280 --> 00:19:02.000 +Yuka Letts, + +00:19:02.380 --> 00:19:03.180 +the docilo? + +00:19:03.480 --> 00:19:03.940 +Sorry, + +00:19:04.100 --> 00:19:04.340 +Yuka. + +00:19:05.480 --> 00:19:06.280 +But I want to + +00:19:06.280 --> 00:19:06.660 +make sure that + +00:19:06.660 --> 00:19:07.440 +we point out + +00:19:07.440 --> 00:19:07.780 +there's actually + +00:19:07.780 --> 00:19:08.320 +five people, + +00:19:08.420 --> 00:19:08.880 +not just the + +00:19:08.880 --> 00:19:09.300 +three of you, + +00:19:09.320 --> 00:19:09.460 +right? + +00:19:09.840 --> 00:19:10.180 +How do you + +00:19:10.180 --> 00:19:10.860 +get on the + +00:19:10.860 --> 00:19:11.440 +council? + +00:19:11.820 --> 00:19:12.220 +Is there an + +00:19:12.220 --> 00:19:12.600 +election? + +00:19:13.020 --> 00:19:13.620 +Do you just + +00:19:13.620 --> 00:19:14.240 +apply? + +00:19:14.980 --> 00:19:15.300 +I think these + +00:19:15.300 --> 00:19:15.760 +are filled by + +00:19:15.760 --> 00:19:16.100 +the members + +00:19:16.100 --> 00:19:16.460 +themselves. + +00:19:16.740 --> 00:19:17.080 +So when + +00:19:17.080 --> 00:19:17.960 +somebody declares + +00:19:17.960 --> 00:19:18.400 +the intention + +00:19:18.400 --> 00:19:19.120 +to leave the + +00:19:19.120 --> 00:19:20.000 +council, we + +00:19:20.000 --> 00:19:20.880 +basically ask for + +00:19:20.880 --> 00:19:21.260 +people who are + +00:19:21.260 --> 00:19:22.080 +interested and + +00:19:22.080 --> 00:19:22.640 +then make a + +00:19:22.640 --> 00:19:23.000 +selection. + +00:19:23.440 --> 00:19:23.860 +Generally, we + +00:19:23.860 --> 00:19:24.640 +try to get + +00:19:24.640 --> 00:19:25.420 +people who have + +00:19:25.420 --> 00:19:26.120 +experience in the + +00:19:26.120 --> 00:19:26.680 +type system. + +00:19:27.140 --> 00:19:27.700 +We try to get a + +00:19:27.700 --> 00:19:28.120 +good cross + +00:19:28.120 --> 00:19:28.780 +representation of + +00:19:28.780 --> 00:19:29.200 +people working + +00:19:29.200 --> 00:19:29.720 +on different + +00:19:29.720 --> 00:19:30.420 +type checkers. + +00:19:30.700 --> 00:19:32.020 +We have Carl + +00:19:32.020 --> 00:19:32.620 +and Rebecca + +00:19:32.620 --> 00:19:33.320 +here who work + +00:19:33.320 --> 00:19:34.280 +on two type + +00:19:34.280 --> 00:19:34.620 +checkers, + +00:19:34.760 --> 00:19:35.160 +TY and + +00:19:35.160 --> 00:19:35.720 +Pyrefly. + +00:19:36.160 --> 00:19:36.820 +Yuka works + +00:19:36.820 --> 00:19:38.800 +on Pyrites, + +00:19:39.040 --> 00:19:39.560 +which are two + +00:19:39.560 --> 00:19:39.980 +of the most + +00:19:39.980 --> 00:19:40.420 +lightly used + +00:19:40.420 --> 00:19:41.060 +type checkers. + +00:19:41.480 --> 00:19:42.400 +So we try to + +00:19:42.400 --> 00:19:43.240 +get representation + +00:19:43.240 --> 00:19:44.340 +of people working + +00:19:44.340 --> 00:19:45.260 +on those parts + +00:19:45.260 --> 00:19:45.840 +of the ecosystem. + +00:19:46.100 --> 00:19:46.360 +That's really + +00:19:46.360 --> 00:19:47.620 +cool that it's + +00:19:47.620 --> 00:19:48.120 +got a bias + +00:19:48.120 --> 00:19:48.680 +towards finding + +00:19:48.680 --> 00:19:49.340 +people actually + +00:19:49.340 --> 00:19:50.300 +doing the work. + +00:19:50.700 --> 00:19:51.320 +So let's talk + +00:19:51.320 --> 00:19:52.020 +about the + +00:19:52.020 --> 00:19:53.380 +specification project + +00:19:53.380 --> 00:19:55.320 +at typing.python.org. + +00:19:55.880 --> 00:19:56.440 +What is this + +00:19:56.440 --> 00:19:56.720 +here? + +00:19:56.720 --> 00:19:57.840 +I'll talk a bit + +00:19:57.840 --> 00:19:58.320 +about it. + +00:19:58.320 --> 00:19:59.560 +I guess it's + +00:19:59.560 --> 00:20:00.880 +a specification + +00:20:00.880 --> 00:20:01.900 +for how the + +00:20:01.900 --> 00:20:02.680 +type system + +00:20:02.680 --> 00:20:03.620 +used to work. + +00:20:03.760 --> 00:20:04.460 +The way it + +00:20:04.460 --> 00:20:05.120 +started was + +00:20:05.120 --> 00:20:05.660 +that, Yela, + +00:20:05.740 --> 00:20:06.320 +you basically + +00:20:06.320 --> 00:20:07.220 +took all the + +00:20:07.220 --> 00:20:07.920 +typing peps + +00:20:07.920 --> 00:20:08.320 +and like + +00:20:08.320 --> 00:20:09.160 +stapled them + +00:20:09.160 --> 00:20:09.860 +together, + +00:20:09.860 --> 00:20:10.620 +right, to + +00:20:10.620 --> 00:20:11.180 +make like + +00:20:11.180 --> 00:20:11.920 +one long + +00:20:11.920 --> 00:20:12.320 +doc. + +00:20:12.460 --> 00:20:13.320 +And since + +00:20:13.320 --> 00:20:13.780 +then, we've + +00:20:13.780 --> 00:20:14.480 +been iterating + +00:20:14.480 --> 00:20:15.080 +on it, + +00:20:15.360 --> 00:20:15.800 +filling in + +00:20:15.800 --> 00:20:16.280 +parts that + +00:20:16.280 --> 00:20:16.800 +were missing + +00:20:16.800 --> 00:20:18.160 +like overload + +00:20:18.160 --> 00:20:19.660 +evaluation and + +00:20:19.660 --> 00:20:20.720 +making other + +00:20:20.720 --> 00:20:21.400 +changes as + +00:20:21.400 --> 00:20:21.560 +well. + +00:20:21.660 --> 00:20:22.200 +Yeah, it's + +00:20:22.200 --> 00:20:22.780 +tricky, right? + +00:20:22.860 --> 00:20:23.340 +Because + +00:20:23.340 --> 00:20:24.800 +traditionally, the + +00:20:24.800 --> 00:20:25.960 +typing system + +00:20:25.960 --> 00:20:26.420 +is kind of + +00:20:26.420 --> 00:20:27.320 +defined across + +00:20:27.320 --> 00:20:28.020 +a series of + +00:20:28.020 --> 00:20:28.360 +peps. + +00:20:28.780 --> 00:20:29.440 +And so what + +00:20:29.440 --> 00:20:30.000 +is the document + +00:20:30.000 --> 00:20:30.480 +that tells you + +00:20:30.480 --> 00:20:31.000 +how it works, + +00:20:31.060 --> 00:20:31.220 +right? + +00:20:31.340 --> 00:20:31.680 +Yeah, that + +00:20:31.680 --> 00:20:32.100 +made it hard + +00:20:32.100 --> 00:20:32.800 +because often + +00:20:32.800 --> 00:20:33.900 +there's peps + +00:20:33.900 --> 00:20:34.500 +built on top + +00:20:34.500 --> 00:20:35.000 +of each other. + +00:20:35.320 --> 00:20:36.440 +So then in + +00:20:36.440 --> 00:20:36.820 +the extreme, + +00:20:37.020 --> 00:20:37.580 +you might see + +00:20:37.580 --> 00:20:38.220 +like one thing + +00:20:38.220 --> 00:20:39.140 +in one pep + +00:20:39.140 --> 00:20:39.640 +and then there's + +00:20:39.640 --> 00:20:40.080 +another pep + +00:20:40.080 --> 00:20:40.480 +that adds + +00:20:40.480 --> 00:20:41.120 +an aspect of + +00:20:41.120 --> 00:20:41.480 +it, another + +00:20:41.480 --> 00:20:41.960 +one that adds + +00:20:41.960 --> 00:20:42.700 +another aspect. + +00:20:43.060 --> 00:20:43.400 +And overall + +00:20:43.400 --> 00:20:43.760 +it makes it + +00:20:43.760 --> 00:20:44.280 +very hard to + +00:20:44.280 --> 00:20:44.540 +follow. + +00:20:44.920 --> 00:20:45.300 +One of the + +00:20:45.300 --> 00:20:45.720 +things I did + +00:20:45.720 --> 00:20:46.260 +recently was + +00:20:46.260 --> 00:20:47.060 +rewrite the + +00:20:47.060 --> 00:20:47.740 +typed dict + +00:20:47.740 --> 00:20:47.740 + + +00:20:54.800 --> 00:20:55.000 +another. + +00:20:55.280 --> 00:20:55.680 +Ended up + +00:20:55.680 --> 00:20:56.240 +rewriting the + +00:20:56.240 --> 00:20:56.720 +whole thing + +00:20:56.720 --> 00:20:57.680 +to basically + +00:20:57.680 --> 00:20:58.100 +put all those + +00:20:58.100 --> 00:20:58.700 +features together + +00:20:58.700 --> 00:20:59.960 +in a coherent + +00:20:59.960 --> 00:21:00.720 +whole rather than + +00:21:00.720 --> 00:21:02.240 +just having them + +00:21:02.240 --> 00:21:03.260 +all copy-pasted + +00:21:03.260 --> 00:21:04.080 +one after the + +00:21:04.080 --> 00:21:04.260 +other. + +00:21:04.400 --> 00:21:05.080 +Okay, so if + +00:21:05.080 --> 00:21:05.580 +somebody really + +00:21:05.580 --> 00:21:06.520 +wants a good + +00:21:06.520 --> 00:21:07.960 +understanding of + +00:21:07.960 --> 00:21:08.480 +the Python + +00:21:08.480 --> 00:21:09.240 +typing system, + +00:21:09.560 --> 00:21:09.960 +they go to + +00:21:09.960 --> 00:21:11.340 +typing.python.org. + +00:21:11.540 --> 00:21:12.020 +You know, one + +00:21:12.020 --> 00:21:12.700 +thing I think + +00:21:12.700 --> 00:21:13.320 +maybe is worth + +00:21:13.320 --> 00:21:13.840 +touching on, + +00:21:13.920 --> 00:21:14.500 +it's just kind + +00:21:14.500 --> 00:21:15.320 +of out of the + +00:21:15.320 --> 00:21:15.960 +blue a bit, + +00:21:16.060 --> 00:21:16.540 +but I think + +00:21:16.540 --> 00:21:17.000 +it's a really + +00:21:17.000 --> 00:21:17.960 +interesting aspect + +00:21:17.960 --> 00:21:19.400 +of the Python + +00:21:19.400 --> 00:21:20.240 +typing system + +00:21:20.240 --> 00:21:21.000 +is the, + +00:21:21.320 --> 00:21:21.560 +what is it + +00:21:21.560 --> 00:21:21.840 +called, the + +00:21:21.840 --> 00:21:22.720 +numerical tower + +00:21:22.720 --> 00:21:23.280 +or the number + +00:21:23.280 --> 00:21:24.340 +tower, where + +00:21:24.340 --> 00:21:25.280 +it's like, if + +00:21:25.280 --> 00:21:25.620 +I have a + +00:21:25.620 --> 00:21:26.400 +number, I + +00:21:26.400 --> 00:21:27.080 +could specify + +00:21:27.080 --> 00:21:27.880 +it as an + +00:21:27.880 --> 00:21:28.800 +int, or I + +00:21:28.800 --> 00:21:29.340 +could specify + +00:21:29.340 --> 00:21:29.920 +it as a + +00:21:29.920 --> 00:21:30.940 +float, and + +00:21:30.940 --> 00:21:31.400 +those kinds + +00:21:31.400 --> 00:21:32.520 +of things, but + +00:21:32.520 --> 00:21:33.420 +do you really + +00:21:33.420 --> 00:21:34.020 +need to say + +00:21:34.020 --> 00:21:34.580 +it's an + +00:21:34.580 --> 00:21:35.860 +int pipe + +00:21:35.860 --> 00:21:36.620 +float, or a + +00:21:36.620 --> 00:21:37.140 +union of + +00:21:37.140 --> 00:21:37.520 +int and + +00:21:37.520 --> 00:21:38.160 +float, if it + +00:21:38.160 --> 00:21:38.500 +could be + +00:21:38.500 --> 00:21:39.080 +either, right? + +00:21:39.200 --> 00:21:40.080 +And the, what + +00:21:40.080 --> 00:21:40.480 +is it called? + +00:21:40.600 --> 00:21:40.980 +It's the + +00:21:40.980 --> 00:21:41.680 +numerical tower, + +00:21:41.780 --> 00:21:42.020 +right? + +00:21:42.240 --> 00:21:42.560 +Yeah, there + +00:21:42.560 --> 00:21:42.920 +are different + +00:21:42.920 --> 00:21:43.560 +towers too. + +00:21:43.760 --> 00:21:44.540 +In Python, there's + +00:21:44.540 --> 00:21:45.100 +also this thing + +00:21:45.100 --> 00:21:45.680 +called a numbers + +00:21:45.680 --> 00:21:46.920 +module that you + +00:21:46.920 --> 00:21:48.080 +have there, that's + +00:21:48.080 --> 00:21:48.920 +just basically + +00:21:48.920 --> 00:21:49.420 +ignored by the + +00:21:49.420 --> 00:21:49.920 +type system. + +00:21:50.140 --> 00:21:50.520 +It's been + +00:21:50.520 --> 00:21:51.100 +useful for some + +00:21:51.100 --> 00:21:51.660 +people, I feel + +00:21:51.660 --> 00:21:52.060 +like in general + +00:21:52.060 --> 00:21:52.980 +that module just + +00:21:52.980 --> 00:21:53.780 +hasn't worked out + +00:21:53.780 --> 00:21:54.320 +very well as + +00:21:54.320 --> 00:21:54.740 +being very + +00:21:54.740 --> 00:21:55.060 +useful. + +00:21:55.260 --> 00:21:55.840 +I think the + +00:21:55.840 --> 00:21:56.700 +interesting aspect + +00:21:56.700 --> 00:21:58.080 +is that you + +00:21:58.080 --> 00:21:58.560 +know, that you + +00:21:58.560 --> 00:21:59.840 +can say it's a + +00:21:59.840 --> 00:22:00.580 +float, and that's + +00:22:00.580 --> 00:22:01.480 +basically equivalent + +00:22:01.480 --> 00:22:02.860 +to union of + +00:22:02.860 --> 00:22:03.540 +integer and + +00:22:03.540 --> 00:22:04.800 +float, and so + +00:22:04.800 --> 00:22:05.280 +on, right? + +00:22:05.340 --> 00:22:05.880 +I think the + +00:22:05.880 --> 00:22:07.520 +typing numbers in + +00:22:07.520 --> 00:22:08.180 +Python is pretty + +00:22:08.180 --> 00:22:08.540 +interesting. + +00:22:08.760 --> 00:22:09.300 +I think every + +00:22:09.300 --> 00:22:10.740 +type checker has + +00:22:10.740 --> 00:22:11.160 +a different + +00:22:11.160 --> 00:22:12.640 +interpretation of + +00:22:12.640 --> 00:22:13.680 +what a float + +00:22:13.680 --> 00:22:14.600 +annotation actually + +00:22:14.600 --> 00:22:14.960 +means. + +00:22:16.020 --> 00:22:17.100 +It's an area of + +00:22:17.100 --> 00:22:18.160 +some lack of + +00:22:18.160 --> 00:22:18.780 +clarity in the + +00:22:18.780 --> 00:22:19.020 +spec. + +00:22:19.140 --> 00:22:19.660 +Yeah, a lot of + +00:22:19.660 --> 00:22:20.340 +contentiousness. + +00:22:20.480 --> 00:22:21.740 +If we could go + +00:22:21.740 --> 00:22:22.960 +back in time, I + +00:22:22.960 --> 00:22:23.760 +would, like, + +00:22:23.780 --> 00:22:24.380 +knowing what I + +00:22:24.380 --> 00:22:24.940 +know now, I + +00:22:24.940 --> 00:22:25.660 +probably advocate + +00:22:25.660 --> 00:22:26.420 +for things being + +00:22:26.420 --> 00:22:27.520 +done differently + +00:22:27.520 --> 00:22:28.540 +because, like, + +00:22:28.700 --> 00:22:29.820 +beginning, you + +00:22:29.820 --> 00:22:30.440 +know, like, there + +00:22:30.440 --> 00:22:31.440 +were multiple + +00:22:31.440 --> 00:22:32.700 +things, like, with + +00:22:32.700 --> 00:22:33.460 +similar flavor. + +00:22:33.660 --> 00:22:34.180 +Like, there was + +00:22:34.180 --> 00:22:35.420 +also one where + +00:22:35.420 --> 00:22:36.480 +you could give + +00:22:36.480 --> 00:22:37.940 +a parameter a + +00:22:37.940 --> 00:22:38.480 +non-none + +00:22:38.480 --> 00:22:39.320 +annotation and + +00:22:39.320 --> 00:22:40.340 +default it to + +00:22:40.340 --> 00:22:40.800 +none for + +00:22:40.800 --> 00:22:41.700 +convenience, and + +00:22:41.700 --> 00:22:42.220 +we've largely, + +00:22:42.360 --> 00:22:43.420 +like, moved away + +00:22:43.420 --> 00:22:44.120 +from stuff like + +00:22:44.120 --> 00:22:44.740 +that in favor of + +00:22:44.740 --> 00:22:45.680 +explicitness. + +00:22:46.060 --> 00:22:46.540 +Yeah, what the + +00:22:46.540 --> 00:22:47.360 +current spec says + +00:22:47.360 --> 00:22:48.300 +is that basically + +00:22:48.300 --> 00:22:48.960 +if you have a + +00:22:48.960 --> 00:22:49.540 +function that takes + +00:22:49.540 --> 00:22:50.640 +a float, you're + +00:22:50.640 --> 00:22:51.320 +also allowed to + +00:22:51.320 --> 00:22:51.940 +pass an int. + +00:22:52.440 --> 00:22:53.480 +That's not + +00:22:53.480 --> 00:22:54.160 +really enough. + +00:22:54.260 --> 00:22:54.820 +It doesn't tell + +00:22:54.820 --> 00:22:55.620 +you how these + +00:22:55.620 --> 00:22:56.320 +things work in + +00:22:56.320 --> 00:22:57.800 +all cases, and + +00:22:57.800 --> 00:22:59.040 +we've had some + +00:22:59.040 --> 00:23:00.140 +attempts to try to + +00:23:00.140 --> 00:23:01.280 +come up with a + +00:23:01.280 --> 00:23:02.220 +way to specify + +00:23:02.220 --> 00:23:03.440 +that special case + +00:23:03.440 --> 00:23:04.160 +in a way that + +00:23:04.160 --> 00:23:05.360 +makes more sense, + +00:23:05.560 --> 00:23:06.020 +at least makes + +00:23:06.020 --> 00:23:06.700 +more sense to me. + +00:23:07.080 --> 00:23:07.740 +It's been very + +00:23:07.740 --> 00:23:08.260 +contentious. + +00:23:08.460 --> 00:23:09.040 +People have very + +00:23:09.040 --> 00:23:09.600 +strong opinions + +00:23:09.600 --> 00:23:10.220 +about this. + +00:23:10.400 --> 00:23:11.340 +I guess non-obvious + +00:23:11.340 --> 00:23:12.180 +is what I'd like to + +00:23:12.180 --> 00:23:12.760 +say, really, + +00:23:12.980 --> 00:23:13.320 +honestly. + +00:23:13.760 --> 00:23:15.000 +So I'd like to get + +00:23:15.000 --> 00:23:16.060 +the official + +00:23:16.060 --> 00:23:17.120 +counsel's thoughts + +00:23:17.120 --> 00:23:17.560 +on this. + +00:23:17.760 --> 00:23:18.620 +When is typing + +00:23:18.620 --> 00:23:19.960 +too much typing? + +00:23:20.440 --> 00:23:21.380 +I made the joke + +00:23:21.380 --> 00:23:22.300 +about C++, + +00:23:22.600 --> 00:23:23.700 +ATL, if you've + +00:23:23.700 --> 00:23:24.200 +ever worked with + +00:23:24.200 --> 00:23:24.820 +that, it's like + +00:23:24.820 --> 00:23:27.100 +a template class + +00:23:27.100 --> 00:23:28.020 +where templated + +00:23:28.020 --> 00:23:28.940 +classes are part + +00:23:28.940 --> 00:23:29.540 +of the concrete + +00:23:29.540 --> 00:23:30.220 +type of the + +00:23:30.220 --> 00:23:30.440 +template. + +00:23:30.560 --> 00:23:31.600 +It's just off + +00:23:31.600 --> 00:23:32.020 +the hook. + +00:23:32.400 --> 00:23:32.900 +There's certainly + +00:23:32.900 --> 00:23:33.740 +places where + +00:23:33.740 --> 00:23:34.880 +typing can be + +00:23:34.880 --> 00:23:36.180 +too much, and + +00:23:36.180 --> 00:23:37.220 +a lot of the + +00:23:37.220 --> 00:23:38.640 +purity of Python + +00:23:38.640 --> 00:23:40.240 +or the readability + +00:23:40.240 --> 00:23:41.320 +of Python is the + +00:23:41.320 --> 00:23:42.460 +fact that it's + +00:23:42.460 --> 00:23:43.320 +got so few + +00:23:43.320 --> 00:23:43.740 +symbols. + +00:23:44.260 --> 00:23:44.700 +And so adding + +00:23:44.700 --> 00:23:45.700 +types adds + +00:23:45.700 --> 00:23:46.460 +context, but it + +00:23:46.460 --> 00:23:47.920 +also makes it + +00:23:47.920 --> 00:23:48.260 +a little harder + +00:23:48.260 --> 00:23:48.700 +to read. + +00:23:49.060 --> 00:23:49.960 +When is too + +00:23:49.960 --> 00:23:50.520 +much typing? + +00:23:50.960 --> 00:23:51.460 +When do you + +00:23:51.460 --> 00:23:52.320 +recommend typing? + +00:23:53.320 --> 00:23:54.120 +Rebecca, I'll + +00:23:54.120 --> 00:23:54.820 +let you go first, + +00:23:54.940 --> 00:23:55.840 +but what are your + +00:23:55.840 --> 00:23:57.060 +thoughts on how + +00:23:57.060 --> 00:23:57.980 +much typing should + +00:23:57.980 --> 00:23:58.760 +I use in Python? + +00:23:59.040 --> 00:24:00.380 +I'll give you + +00:24:00.380 --> 00:24:01.420 +what is my + +00:24:01.420 --> 00:24:03.160 +official stance, + +00:24:03.260 --> 00:24:03.940 +which is that if + +00:24:03.940 --> 00:24:04.460 +you want your + +00:24:04.460 --> 00:24:05.240 +type checker to + +00:24:05.240 --> 00:24:06.500 +work well, you + +00:24:06.500 --> 00:24:07.460 +should type + +00:24:07.460 --> 00:24:08.160 +annotate your + +00:24:08.160 --> 00:24:09.320 +API boundaries. + +00:24:09.880 --> 00:24:10.500 +So parameters + +00:24:10.500 --> 00:24:11.260 +and turns in + +00:24:11.260 --> 00:24:12.060 +public functions, + +00:24:12.340 --> 00:24:12.840 +public class + +00:24:12.840 --> 00:24:14.000 +attributes, things + +00:24:14.000 --> 00:24:14.900 +like that, and + +00:24:14.900 --> 00:24:15.980 +even things that + +00:24:15.980 --> 00:24:16.440 +seem true + +00:24:16.460 --> 00:24:17.200 +trivial, like, + +00:24:17.300 --> 00:24:17.560 +oh, this + +00:24:17.560 --> 00:24:18.900 +function returns + +00:24:18.900 --> 00:24:20.160 +none, better + +00:24:20.160 --> 00:24:21.400 +to annotate + +00:24:21.400 --> 00:24:22.040 +it because, you + +00:24:22.040 --> 00:24:22.540 +know, someone + +00:24:22.540 --> 00:24:23.080 +else might be + +00:24:23.080 --> 00:24:24.120 +depending on + +00:24:24.120 --> 00:24:25.080 +your library and + +00:24:25.080 --> 00:24:25.700 +consuming that + +00:24:25.700 --> 00:24:26.080 +type of + +00:24:26.080 --> 00:24:26.600 +information. + +00:24:27.120 --> 00:24:27.780 +I will say + +00:24:27.780 --> 00:24:29.040 +personally, what + +00:24:29.040 --> 00:24:29.900 +I tend to do + +00:24:29.900 --> 00:24:31.760 +is I annotate + +00:24:31.760 --> 00:24:32.420 +things that I + +00:24:32.420 --> 00:24:32.900 +think are + +00:24:32.900 --> 00:24:33.680 +non-trivial + +00:24:33.680 --> 00:24:34.480 +because I want + +00:24:34.480 --> 00:24:35.160 +to see that + +00:24:35.160 --> 00:24:36.440 +as documentation. + +00:24:37.400 --> 00:24:38.020 +And if + +00:24:38.020 --> 00:24:39.080 +something, you + +00:24:39.080 --> 00:24:39.540 +know, a + +00:24:39.540 --> 00:24:39.900 +function that + +00:24:39.900 --> 00:24:40.560 +does return + +00:24:40.560 --> 00:24:41.540 +none, to be + +00:24:41.540 --> 00:24:42.040 +honest, I will + +00:24:42.040 --> 00:24:42.600 +probably forget + +00:24:42.600 --> 00:24:43.460 +to annotate it + +00:24:43.460 --> 00:24:44.340 +half the time + +00:24:44.340 --> 00:24:45.260 +because I'll + +00:24:45.260 --> 00:24:46.060 +be like, I + +00:24:46.060 --> 00:24:46.700 +honestly don't + +00:24:46.700 --> 00:24:47.220 +need to see + +00:24:47.220 --> 00:24:47.460 +it. + +00:24:47.660 --> 00:24:48.500 +One of the + +00:24:48.500 --> 00:24:49.940 +interesting features + +00:24:49.940 --> 00:24:50.760 +of the + +00:24:50.760 --> 00:24:52.440 +Pyrefly VS + +00:24:52.440 --> 00:24:53.240 +Code extension, + +00:24:53.380 --> 00:24:53.840 +that's the only + +00:24:53.840 --> 00:24:54.540 +one I can speak + +00:24:54.540 --> 00:24:54.920 +of at the + +00:24:54.920 --> 00:24:55.300 +moment, and + +00:24:55.300 --> 00:24:55.740 +Carl, you've + +00:24:55.740 --> 00:24:56.080 +got to tell + +00:24:56.080 --> 00:24:56.860 +me if the + +00:24:56.860 --> 00:24:58.000 +TY one does + +00:24:58.000 --> 00:24:58.600 +this as well, + +00:24:59.040 --> 00:24:59.580 +is it will + +00:24:59.580 --> 00:25:00.440 +sort of overlay + +00:25:00.440 --> 00:25:02.440 +its belief of + +00:25:02.440 --> 00:25:03.240 +what types + +00:25:03.240 --> 00:25:03.580 +are. + +00:25:03.960 --> 00:25:04.280 +Like, if + +00:25:04.280 --> 00:25:05.060 +there's, you + +00:25:05.060 --> 00:25:06.080 +say x equals a + +00:25:06.080 --> 00:25:06.560 +function return + +00:25:06.560 --> 00:25:07.180 +value and it + +00:25:07.180 --> 00:25:07.700 +knows what the + +00:25:07.700 --> 00:25:08.340 +function returns, + +00:25:08.440 --> 00:25:08.900 +it'll have a + +00:25:08.900 --> 00:25:09.520 +gray, like, + +00:25:09.680 --> 00:25:10.580 +colon int, if + +00:25:10.580 --> 00:25:11.200 +it returned an + +00:25:11.200 --> 00:25:11.820 +int or something. + +00:25:11.820 --> 00:25:12.620 +So you can + +00:25:12.620 --> 00:25:14.120 +kind of read + +00:25:14.120 --> 00:25:15.140 +the code and + +00:25:15.140 --> 00:25:15.600 +see what the + +00:25:15.600 --> 00:25:16.200 +types are without + +00:25:16.200 --> 00:25:17.060 +actually putting + +00:25:17.060 --> 00:25:18.040 +it into the + +00:25:18.040 --> 00:25:18.720 +text of the + +00:25:18.720 --> 00:25:18.980 +code. + +00:25:19.080 --> 00:25:19.580 +It's only within + +00:25:19.580 --> 00:25:20.120 +the editor. + +00:25:20.520 --> 00:25:21.080 +Does ty do + +00:25:21.080 --> 00:25:21.460 +something like + +00:25:21.460 --> 00:25:21.840 +that, Carl? + +00:25:22.040 --> 00:25:22.860 +Yes, we also + +00:25:22.860 --> 00:25:23.560 +have inlay type + +00:25:23.560 --> 00:25:23.800 +ins. + +00:25:23.900 --> 00:25:24.400 +Yeah, inlay type + +00:25:24.400 --> 00:25:24.800 +ins, that's + +00:25:24.800 --> 00:25:25.200 +what it's called. + +00:25:25.500 --> 00:25:26.060 +So, yeah, I + +00:25:26.060 --> 00:25:26.400 +don't know, that + +00:25:26.400 --> 00:25:27.420 +also brings an + +00:25:27.420 --> 00:25:28.300 +interesting challenge, + +00:25:28.400 --> 00:25:28.880 +not a challenge, + +00:25:28.960 --> 00:25:30.160 +like a wrinkle to + +00:25:30.160 --> 00:25:31.460 +the recommendation + +00:25:31.460 --> 00:25:33.640 +of should I put + +00:25:33.640 --> 00:25:34.360 +types on, like, + +00:25:34.400 --> 00:25:35.180 +the return value + +00:25:35.180 --> 00:25:35.780 +because I want to + +00:25:35.780 --> 00:25:36.280 +know that's a + +00:25:36.280 --> 00:25:36.920 +list of user, + +00:25:37.040 --> 00:25:37.880 +not a list of + +00:25:37.880 --> 00:25:38.880 +user IDs or + +00:25:38.880 --> 00:25:39.320 +whatever, for + +00:25:39.320 --> 00:25:39.940 +example, like a + +00:25:39.940 --> 00:25:40.780 +list of UUID. + +00:25:41.220 --> 00:25:41.580 +But if, + +00:25:41.820 --> 00:25:42.340 +it's going to + +00:25:42.340 --> 00:25:43.000 +show up anyway + +00:25:43.000 --> 00:25:43.660 +in the editor, + +00:25:44.160 --> 00:25:44.740 +maybe I don't + +00:25:44.740 --> 00:25:45.280 +have to write + +00:25:45.280 --> 00:25:45.780 +that, right? + +00:25:45.840 --> 00:25:46.220 +And so that + +00:25:46.220 --> 00:25:47.260 +becomes sort of + +00:25:47.260 --> 00:25:48.260 +somewhere where + +00:25:48.260 --> 00:25:49.000 +you could debate + +00:25:49.000 --> 00:25:49.680 +again, I think. + +00:25:50.120 --> 00:25:51.160 +However, I do + +00:25:51.160 --> 00:25:52.280 +100% agree with + +00:25:52.280 --> 00:25:53.040 +you, Rebecca, that + +00:25:53.040 --> 00:25:53.760 +put it on your + +00:25:53.760 --> 00:25:54.560 +API boundaries. + +00:25:54.740 --> 00:25:55.560 +If, like, this + +00:25:55.560 --> 00:25:56.600 +is the place that + +00:25:56.600 --> 00:25:57.500 +people get into + +00:25:57.500 --> 00:25:58.340 +some part of your + +00:25:58.340 --> 00:25:58.820 +code and they + +00:25:58.820 --> 00:25:59.700 +don't know or + +00:25:59.700 --> 00:26:00.380 +want to know about + +00:26:00.380 --> 00:26:01.120 +the inside of it, + +00:26:01.500 --> 00:26:02.460 +having types there + +00:26:02.460 --> 00:26:03.360 +is really helpful + +00:26:03.360 --> 00:26:04.440 +both for editors, + +00:26:04.960 --> 00:26:05.800 +for type checkers, + +00:26:05.840 --> 00:26:06.480 +and just for + +00:26:06.480 --> 00:26:07.300 +reading code, and + +00:26:07.300 --> 00:26:08.360 +even for AI, which + +00:26:08.360 --> 00:26:09.240 +is a crazy world. + +00:26:09.580 --> 00:26:09.680 +Yeah. + +00:26:09.900 --> 00:26:10.220 +Carl? + +00:26:10.220 --> 00:26:11.000 +What are your + +00:26:11.000 --> 00:26:11.400 +thoughts here? + +00:26:11.460 --> 00:26:12.180 +How much typing is + +00:26:12.180 --> 00:26:12.760 +too much typing? + +00:26:13.200 --> 00:26:13.780 +What's the + +00:26:13.780 --> 00:26:14.580 +guidelines here? + +00:26:14.680 --> 00:26:15.240 +I think I agree + +00:26:15.240 --> 00:26:15.800 +with Rebecca's + +00:26:15.800 --> 00:26:16.100 +answer. + +00:26:16.280 --> 00:26:17.120 +I mean, that one + +00:26:17.120 --> 00:26:17.740 +place you definitely + +00:26:17.740 --> 00:26:18.740 +want to have + +00:26:18.740 --> 00:26:19.840 +explicit type + +00:26:19.840 --> 00:26:21.100 +annotations is that + +00:26:21.100 --> 00:26:22.140 +API boundaries, + +00:26:22.420 --> 00:26:23.300 +the public API of + +00:26:23.300 --> 00:26:24.120 +a library, etc. + +00:26:24.120 --> 00:26:25.680 +In terms of what's + +00:26:25.680 --> 00:26:26.660 +too much typing, I + +00:26:26.660 --> 00:26:27.220 +mean, there are + +00:26:27.220 --> 00:26:28.320 +certainly patterns + +00:26:28.320 --> 00:26:28.900 +that have + +00:26:28.900 --> 00:26:29.740 +historically been + +00:26:29.740 --> 00:26:31.280 +used in Python + +00:26:31.280 --> 00:26:32.380 +that we still + +00:26:32.380 --> 00:26:34.060 +can't express well + +00:26:34.060 --> 00:26:34.540 +in the type + +00:26:34.540 --> 00:26:35.580 +system, or that + +00:26:35.580 --> 00:26:36.480 +require extremely + +00:26:36.480 --> 00:26:37.680 +complex type + +00:26:37.680 --> 00:26:39.240 +annotations to + +00:26:39.240 --> 00:26:40.020 +express well, and + +00:26:40.020 --> 00:26:40.640 +I think there it + +00:26:40.640 --> 00:26:41.720 +becomes a judgment + +00:26:41.720 --> 00:26:42.140 +call. + +00:26:43.020 --> 00:26:44.420 +If it's like a + +00:26:44.420 --> 00:26:45.220 +core, widely + +00:26:45.220 --> 00:26:46.540 +used API, you + +00:26:46.540 --> 00:26:47.620 +may get a lot of + +00:26:47.620 --> 00:26:48.940 +benefit from some + +00:26:48.940 --> 00:26:50.020 +very complex and + +00:26:50.020 --> 00:26:51.020 +verbose annotations, + +00:26:51.440 --> 00:26:52.100 +and so then it's + +00:26:52.100 --> 00:26:53.140 +worth sort of going + +00:26:53.140 --> 00:26:53.940 +through that pain + +00:26:53.940 --> 00:26:55.020 +and the pain of + +00:26:55.020 --> 00:26:55.660 +adding them and + +00:26:55.660 --> 00:26:56.960 +of reading them in + +00:26:56.960 --> 00:26:57.700 +order to get that + +00:26:57.700 --> 00:26:59.000 +additional typing + +00:26:59.000 --> 00:27:00.020 +coverage everywhere you + +00:27:00.020 --> 00:27:00.820 +use that API. + +00:27:01.120 --> 00:27:02.340 +If it's much less + +00:27:02.340 --> 00:27:03.400 +frequently used code + +00:27:03.400 --> 00:27:04.280 +that's highly dynamic, + +00:27:04.720 --> 00:27:05.520 +maybe it's not worth + +00:27:05.520 --> 00:27:06.280 +it in that case. + +00:27:06.620 --> 00:27:07.140 +I think there's a lot + +00:27:07.140 --> 00:27:07.780 +of judgment calls + +00:27:07.780 --> 00:27:07.960 +here. + +00:27:08.140 --> 00:27:08.960 +What about like + +00:27:08.960 --> 00:27:10.200 +one-off scripts? + +00:27:10.680 --> 00:27:11.140 +You know, I'm going + +00:27:11.140 --> 00:27:11.940 +to write this thing to + +00:27:11.940 --> 00:27:12.940 +just move this data + +00:27:12.940 --> 00:27:13.580 +from here to there, + +00:27:13.620 --> 00:27:14.220 +and once it's moved, + +00:27:14.280 --> 00:27:15.020 +I don't need it again. + +00:27:15.120 --> 00:27:16.160 +It's done with that + +00:27:16.160 --> 00:27:16.700 +old system, we're + +00:27:16.700 --> 00:27:17.260 +going to the new + +00:27:17.260 --> 00:27:17.460 +one. + +00:27:17.800 --> 00:27:18.660 +Maybe less typing. + +00:27:18.820 --> 00:27:19.080 +Yeah, I think + +00:27:19.080 --> 00:27:20.040 +that's what's useful + +00:27:20.040 --> 00:27:20.600 +for you. + +00:27:20.980 --> 00:27:21.780 +Often I feel like + +00:27:21.780 --> 00:27:22.420 +one-off scripts are + +00:27:22.420 --> 00:27:23.120 +not really one-off + +00:27:23.140 --> 00:27:23.880 +like maybe you + +00:27:23.880 --> 00:27:24.500 +want to move some + +00:27:24.500 --> 00:27:25.460 +similar data later, + +00:27:25.620 --> 00:27:26.260 +and then it's useful + +00:27:26.260 --> 00:27:27.160 +if you can understand + +00:27:27.160 --> 00:27:27.800 +your code again, + +00:27:27.860 --> 00:27:29.040 +if you want to + +00:27:29.040 --> 00:27:29.600 +read what you did. + +00:27:29.720 --> 00:27:30.140 +You thought you + +00:27:30.140 --> 00:27:30.880 +didn't need it again, + +00:27:30.940 --> 00:27:31.760 +and all of a sudden + +00:27:31.760 --> 00:27:32.500 +at six months old, + +00:27:32.540 --> 00:27:33.220 +you don't understand + +00:27:33.220 --> 00:27:34.540 +it, and the types + +00:27:34.540 --> 00:27:35.060 +that help a lot, + +00:27:35.100 --> 00:27:35.260 +right? + +00:27:35.560 --> 00:27:35.720 +Yeah. + +00:27:36.160 --> 00:27:36.780 +Yellow, what's + +00:27:36.780 --> 00:27:38.280 +your advice? + +00:27:38.540 --> 00:27:39.060 +What Cara and + +00:27:39.060 --> 00:27:39.480 +Rebecca said + +00:27:39.480 --> 00:27:40.240 +makes sense to me + +00:27:40.240 --> 00:27:40.440 +too. + +00:27:40.840 --> 00:27:41.480 +I think types + +00:27:41.480 --> 00:27:42.460 +have advantages + +00:27:42.460 --> 00:27:43.340 +in terms of + +00:27:43.340 --> 00:27:44.240 +documenting humor + +00:27:44.240 --> 00:27:45.120 +readers, what + +00:27:45.120 --> 00:27:46.180 +is going on, + +00:27:46.400 --> 00:27:47.040 +and in terms of + +00:27:47.040 --> 00:27:47.940 +catching mistakes + +00:27:47.940 --> 00:27:48.500 +that otherwise + +00:27:48.500 --> 00:27:49.160 +would not be + +00:27:49.160 --> 00:27:49.720 +caught until + +00:27:49.720 --> 00:27:50.740 +runtime perhaps. + +00:27:51.160 --> 00:27:51.680 +They have costs + +00:27:51.680 --> 00:27:52.340 +in maybe making + +00:27:52.340 --> 00:27:52.880 +your code harder + +00:27:52.880 --> 00:27:53.500 +to read if + +00:27:53.500 --> 00:27:54.020 +there's too much + +00:27:54.020 --> 00:27:54.580 +going on. + +00:27:54.940 --> 00:27:55.740 +So add types + +00:27:55.740 --> 00:27:56.140 +as long as + +00:27:56.140 --> 00:27:56.720 +those benefits + +00:27:56.720 --> 00:27:57.240 +outweigh the + +00:27:57.240 --> 00:27:57.560 +costs. + +00:27:57.760 --> 00:27:57.940 +Yeah. + +00:27:58.420 --> 00:27:59.100 +I mean, do you + +00:27:59.100 --> 00:27:59.780 +recommend to + +00:27:59.780 --> 00:28:01.060 +anyone that they + +00:28:01.060 --> 00:28:03.620 +just 100% go full + +00:28:03.620 --> 00:28:04.600 +like C++, + +00:28:04.840 --> 00:28:05.600 +C# on it, + +00:28:05.660 --> 00:28:06.620 +and just type it + +00:28:06.620 --> 00:28:07.740 +every single thing? + +00:28:08.080 --> 00:28:08.740 +Is there an + +00:28:08.740 --> 00:28:10.060 +advantage like for + +00:28:10.060 --> 00:28:11.100 +static type checkers, + +00:28:11.240 --> 00:28:11.540 +you know, like + +00:28:11.540 --> 00:28:12.400 +mypy type stuff + +00:28:12.400 --> 00:28:13.320 +you can run across + +00:28:13.320 --> 00:28:14.680 +and get that? + +00:28:14.780 --> 00:28:15.040 +I mean, you could + +00:28:15.040 --> 00:28:15.420 +do that with + +00:28:15.420 --> 00:28:16.160 +Powerfly or TY + +00:28:16.160 --> 00:28:17.260 +and the CLI as + +00:28:17.260 --> 00:28:18.140 +well, but you know, + +00:28:18.400 --> 00:28:19.340 +thinking more mypy + +00:28:19.340 --> 00:28:19.980 +is like kind of + +00:28:19.980 --> 00:28:20.640 +being real strict + +00:28:20.640 --> 00:28:21.200 +on some of that + +00:28:21.200 --> 00:28:21.500 +stuff. + +00:28:21.580 --> 00:28:22.140 +Personally, I do + +00:28:22.140 --> 00:28:23.140 +tend to annotate + +00:28:23.140 --> 00:28:23.880 +almost all like + +00:28:23.880 --> 00:28:24.700 +function parameters + +00:28:24.700 --> 00:28:26.240 +and if class + +00:28:26.240 --> 00:28:27.060 +attributes, if I + +00:28:27.060 --> 00:28:27.760 +make a class, + +00:28:28.100 --> 00:28:28.680 +sometimes it's not + +00:28:28.680 --> 00:28:29.560 +as necessary, + +00:28:29.740 --> 00:28:30.640 +like you don't + +00:28:30.640 --> 00:28:30.980 +really need to + +00:28:30.980 --> 00:28:31.400 +annotate your + +00:28:31.400 --> 00:28:32.060 +tests perhaps, + +00:28:32.220 --> 00:28:32.560 +or you don't + +00:28:32.560 --> 00:28:33.080 +need to annotate + +00:28:33.080 --> 00:28:33.800 +internal functions + +00:28:33.800 --> 00:28:34.840 +as much, but + +00:28:34.840 --> 00:28:35.680 +for my own + +00:28:35.680 --> 00:28:36.460 +coding, I usually + +00:28:36.460 --> 00:28:36.960 +find it helpful + +00:28:36.960 --> 00:28:37.540 +to do that. + +00:28:37.900 --> 00:28:39.200 +But sometimes I + +00:28:39.200 --> 00:28:39.540 +see people + +00:28:39.540 --> 00:28:40.220 +annotating even + +00:28:40.220 --> 00:28:41.000 +local variables + +00:28:41.000 --> 00:28:41.760 +where it's very + +00:28:41.760 --> 00:28:42.460 +obvious to type + +00:28:42.460 --> 00:28:43.100 +check if the type + +00:28:43.100 --> 00:28:44.040 +is and they can + +00:28:44.040 --> 00:28:44.600 +just infer it + +00:28:44.600 --> 00:28:46.000 +reliably, and then + +00:28:46.000 --> 00:28:46.860 +it really just + +00:28:46.860 --> 00:28:47.580 +adds noise and + +00:28:47.580 --> 00:28:48.140 +you shouldn't do + +00:28:48.140 --> 00:28:48.340 +it. + +00:28:48.500 --> 00:28:48.960 +Yeah, exactly. + +00:28:48.960 --> 00:28:49.620 +If you've got a + +00:28:49.620 --> 00:28:50.160 +function that's + +00:28:50.160 --> 00:28:50.860 +annotated with a + +00:28:50.860 --> 00:28:51.660 +return value and + +00:28:51.660 --> 00:28:53.020 +you say x equals + +00:28:53.020 --> 00:28:53.900 +the function call, + +00:28:54.320 --> 00:28:55.020 +then the type + +00:28:55.020 --> 00:28:55.740 +checkers can infer + +00:28:55.740 --> 00:28:56.560 +that and you're + +00:28:56.560 --> 00:28:57.560 +just causing + +00:28:57.560 --> 00:28:59.560 +extra noise, I + +00:28:59.560 --> 00:28:59.820 +guess. + +00:29:00.240 --> 00:29:01.400 +So suppose you + +00:29:01.400 --> 00:29:02.200 +all want to + +00:29:02.200 --> 00:29:03.640 +change something. + +00:29:03.960 --> 00:29:04.880 +What's the process + +00:29:04.880 --> 00:29:05.640 +of actually going + +00:29:05.640 --> 00:29:06.700 +through and making + +00:29:06.700 --> 00:29:07.420 +some changes? + +00:29:08.280 --> 00:29:09.000 +Mostly sort of + +00:29:09.000 --> 00:29:09.800 +two levels of + +00:29:09.800 --> 00:29:10.100 +this. + +00:29:10.400 --> 00:29:10.860 +Well, maybe + +00:29:10.860 --> 00:29:11.120 +there's even + +00:29:11.120 --> 00:29:11.660 +three levels. + +00:29:11.780 --> 00:29:12.320 +The first one is + +00:29:12.320 --> 00:29:12.780 +if it's something + +00:29:12.780 --> 00:29:13.580 +that's so small + +00:29:13.580 --> 00:29:14.500 +that's just like + +00:29:14.500 --> 00:29:15.100 +a wording + +00:29:15.100 --> 00:29:15.860 +clarification or + +00:29:15.860 --> 00:29:16.540 +something, we + +00:29:16.540 --> 00:29:17.120 +just make a PR + +00:29:17.120 --> 00:29:18.420 +to the repo and + +00:29:18.420 --> 00:29:19.220 +a few of us + +00:29:19.220 --> 00:29:20.040 +look at it and + +00:29:20.040 --> 00:29:20.740 +we change it. + +00:29:21.000 --> 00:29:21.640 +The second level + +00:29:21.640 --> 00:29:22.420 +is when it's sort + +00:29:22.420 --> 00:29:22.940 +of a smaller + +00:29:22.940 --> 00:29:24.180 +change that + +00:29:24.180 --> 00:29:25.200 +doesn't really + +00:29:25.200 --> 00:29:25.860 +introduce a new + +00:29:25.860 --> 00:29:26.680 +feature and then + +00:29:26.680 --> 00:29:28.220 +we make a PR + +00:29:28.220 --> 00:29:28.800 +to the typing + +00:29:28.800 --> 00:29:30.200 +spec repo and + +00:29:30.200 --> 00:29:31.140 +we formally have + +00:29:31.140 --> 00:29:31.900 +all of us sign + +00:29:31.900 --> 00:29:32.380 +off on it. + +00:29:32.740 --> 00:29:33.360 +That's what + +00:29:33.360 --> 00:29:34.040 +happens like + +00:29:34.040 --> 00:29:34.560 +what Carl + +00:29:34.560 --> 00:29:35.120 +mentioned earlier + +00:29:35.120 --> 00:29:35.800 +of the final + +00:29:35.800 --> 00:29:37.140 +change in data + +00:29:37.140 --> 00:29:37.680 +classes. + +00:29:37.860 --> 00:29:38.440 +It's had to + +00:29:38.440 --> 00:29:39.480 +merge this one, + +00:29:39.620 --> 00:29:40.300 +yeah, add + +00:29:40.300 --> 00:29:40.580 +Carl. + +00:29:41.900 --> 00:29:42.740 +I love it. + +00:29:42.840 --> 00:29:43.580 +This repo itself + +00:29:43.580 --> 00:29:44.040 +doesn't have + +00:29:44.040 --> 00:29:44.520 +anything. + +00:29:44.980 --> 00:29:45.460 +It's the + +00:29:45.460 --> 00:29:46.160 +Python typing + +00:29:46.160 --> 00:29:47.400 +repo where the + +00:29:47.400 --> 00:29:48.160 +decisions are + +00:29:48.160 --> 00:29:48.420 +made. + +00:29:48.840 --> 00:29:49.160 +The typing + +00:29:49.160 --> 00:29:49.800 +council just + +00:29:49.800 --> 00:29:50.800 +has like some + +00:29:50.800 --> 00:29:51.560 +documentation. + +00:29:52.420 --> 00:29:52.560 +Yeah. + +00:29:52.840 --> 00:29:53.240 +And then the + +00:29:53.240 --> 00:29:53.780 +third level is + +00:29:53.780 --> 00:29:54.580 +peps, like really + +00:29:54.580 --> 00:29:55.480 +big new changes. + +00:29:55.620 --> 00:29:56.000 +You can still + +00:29:56.000 --> 00:29:56.920 +write a PEP and + +00:29:56.920 --> 00:29:57.880 +then we make a + +00:29:57.880 --> 00:29:58.680 +recommendation and + +00:29:58.680 --> 00:29:59.000 +the steering + +00:29:59.000 --> 00:29:59.720 +council makes a + +00:29:59.720 --> 00:30:00.480 +decision eventually. + +00:30:00.740 --> 00:30:01.380 +So if I wanted + +00:30:01.380 --> 00:30:01.920 +to suggest + +00:30:01.920 --> 00:30:02.940 +something, I + +00:30:02.940 --> 00:30:03.500 +could come up + +00:30:03.500 --> 00:30:04.100 +here and I + +00:30:04.100 --> 00:30:05.580 +could open up + +00:30:05.580 --> 00:30:06.500 +an issue, maybe + +00:30:06.500 --> 00:30:07.380 +start a conversation + +00:30:07.380 --> 00:30:08.540 +on typing, + +00:30:09.000 --> 00:30:09.840 +Python slash + +00:30:09.840 --> 00:30:10.120 +typing. + +00:30:10.280 --> 00:30:10.760 +And you can make + +00:30:10.760 --> 00:30:11.360 +a pull request + +00:30:11.360 --> 00:30:11.960 +to change the + +00:30:11.960 --> 00:30:12.240 +spec. + +00:30:12.240 --> 00:30:12.680 +Okay. + +00:30:12.900 --> 00:30:13.420 +And so the + +00:30:13.420 --> 00:30:13.940 +pull request + +00:30:13.940 --> 00:30:14.480 +would not be + +00:30:14.480 --> 00:30:15.640 +to change the + +00:30:15.640 --> 00:30:16.720 +code, like + +00:30:16.720 --> 00:30:17.340 +how Python + +00:30:17.340 --> 00:30:18.920 +maybe interprets + +00:30:18.920 --> 00:30:19.920 +code that has + +00:30:19.920 --> 00:30:20.600 +this new thing, + +00:30:20.700 --> 00:30:21.620 +but to suggest + +00:30:21.620 --> 00:30:22.240 +that the spec + +00:30:22.240 --> 00:30:22.760 +has it, which + +00:30:22.760 --> 00:30:23.680 +then would start + +00:30:23.680 --> 00:30:24.660 +a process that + +00:30:24.660 --> 00:30:25.420 +ultimately might + +00:30:25.420 --> 00:30:26.240 +make CPython + +00:30:26.240 --> 00:30:26.880 +understand it, + +00:30:26.920 --> 00:30:27.060 +right? + +00:30:27.120 --> 00:30:27.600 +Well, CPython + +00:30:27.600 --> 00:30:28.380 +itself probably + +00:30:28.380 --> 00:30:29.080 +doesn't do + +00:30:29.080 --> 00:30:29.720 +anything with it. + +00:30:30.220 --> 00:30:30.800 +I guess most of + +00:30:30.800 --> 00:30:31.120 +the things that + +00:30:31.120 --> 00:30:31.880 +go directly here + +00:30:31.880 --> 00:30:32.800 +are changes to + +00:30:32.800 --> 00:30:33.420 +how to interpret + +00:30:33.420 --> 00:30:33.960 +things that are + +00:30:33.960 --> 00:30:35.380 +already in CPython. + +00:30:37.140 --> 00:30:38.060 +This portion of + +00:30:38.060 --> 00:30:38.700 +Talk Python To Me + +00:30:38.700 --> 00:30:39.200 +is brought to you + +00:30:39.200 --> 00:30:39.860 +by us. + +00:30:40.380 --> 00:30:41.000 +I want to tell + +00:30:41.000 --> 00:30:42.020 +you about a + +00:30:42.020 --> 00:30:42.600 +course I put + +00:30:42.600 --> 00:30:43.160 +together that + +00:30:43.160 --> 00:30:44.160 +I'm really proud + +00:30:44.160 --> 00:30:45.360 +of, Agentic + +00:30:45.360 --> 00:30:46.340 +AI Programming + +00:30:46.340 --> 00:30:46.980 +for Python + +00:30:46.980 --> 00:30:47.620 +Developers. + +00:30:48.280 --> 00:30:49.160 +I know a lot + +00:30:49.160 --> 00:30:49.540 +of you have + +00:30:49.540 --> 00:30:50.560 +tried AI coding + +00:30:50.560 --> 00:30:51.280 +tools and come + +00:30:51.280 --> 00:30:51.900 +away thinking, + +00:30:52.260 --> 00:30:52.980 +well, this is + +00:30:52.980 --> 00:30:53.560 +more hassle + +00:30:53.560 --> 00:30:54.160 +than it's worth. + +00:30:54.520 --> 00:30:55.120 +And honestly, + +00:30:55.360 --> 00:30:56.200 +all the vibe + +00:30:56.200 --> 00:30:57.020 +coding hype + +00:30:57.020 --> 00:30:57.820 +isn't helping. + +00:30:58.080 --> 00:30:59.080 +It's a smokescreen + +00:30:59.080 --> 00:30:59.800 +that hides what + +00:30:59.800 --> 00:31:00.480 +these tools can + +00:31:00.480 --> 00:31:01.280 +actually do. + +00:31:01.640 --> 00:31:02.560 +This course is + +00:31:02.560 --> 00:31:03.320 +about agentic + +00:31:03.320 --> 00:31:04.080 +engineering, + +00:31:04.660 --> 00:31:05.540 +applying real + +00:31:05.540 --> 00:31:06.320 +software engineering + +00:31:06.320 --> 00:31:07.520 +practices with AI + +00:31:07.520 --> 00:31:08.260 +that understands + +00:31:08.260 --> 00:31:09.160 +your entire code + +00:31:09.160 --> 00:31:10.160 +database, runs + +00:31:10.160 --> 00:31:11.200 +your tests, and + +00:31:11.200 --> 00:31:12.080 +builds complete + +00:31:12.080 --> 00:31:13.100 +features under + +00:31:13.100 --> 00:31:13.720 +your direction. + +00:31:14.380 --> 00:31:14.960 +I've used these + +00:31:14.960 --> 00:31:15.760 +techniques to ship + +00:31:15.760 --> 00:31:17.120 +real production code + +00:31:17.120 --> 00:31:18.140 +across Talk Python, + +00:31:18.600 --> 00:31:19.520 +Python Bytes, and + +00:31:19.520 --> 00:31:20.180 +completely new + +00:31:20.180 --> 00:31:20.580 +projects. + +00:31:21.000 --> 00:31:21.840 +I migrated an + +00:31:21.840 --> 00:31:22.960 +entire CSS framework + +00:31:22.960 --> 00:31:23.660 +on a production + +00:31:23.660 --> 00:31:24.720 +site with thousands + +00:31:24.720 --> 00:31:25.900 +of lines of HTML + +00:31:25.900 --> 00:31:27.160 +in a few hours, + +00:31:27.580 --> 00:31:27.900 +twice. + +00:31:28.240 --> 00:31:29.360 +I shipped a new + +00:31:29.360 --> 00:31:30.280 +search feature with + +00:31:30.280 --> 00:31:31.580 +caching and async + +00:31:31.580 --> 00:31:32.500 +in under an hour. + +00:31:32.500 --> 00:31:33.820 +I built a complete + +00:31:33.820 --> 00:31:35.200 +CLI tool for + +00:31:35.200 --> 00:31:36.640 +Talk Python from + +00:31:36.640 --> 00:31:37.700 +scratch, tested, + +00:31:38.020 --> 00:31:38.780 +documented, and + +00:31:38.780 --> 00:31:40.400 +published to PyPI in + +00:31:40.400 --> 00:31:41.060 +an afternoon. + +00:31:41.580 --> 00:31:42.780 +Real projects, real + +00:31:42.780 --> 00:31:43.960 +production code, both + +00:31:43.960 --> 00:31:45.060 +Greenfield and + +00:31:45.060 --> 00:31:45.540 +legacy. + +00:31:46.020 --> 00:31:47.360 +No toy demos, no + +00:31:47.360 --> 00:31:47.660 +fluff. + +00:31:48.220 --> 00:31:48.980 +I'll show you the + +00:31:48.980 --> 00:31:49.940 +guardrails, the + +00:31:49.940 --> 00:31:50.880 +planning techniques, and + +00:31:50.880 --> 00:31:51.880 +the workflows that + +00:31:51.880 --> 00:31:53.000 +turn AI into a + +00:31:53.000 --> 00:31:53.980 +genuine engineering + +00:31:53.980 --> 00:31:54.360 +partner. + +00:31:54.960 --> 00:31:55.680 +Check it out at + +00:31:55.680 --> 00:31:56.620 +talkpython.fm + +00:31:56.620 --> 00:31:57.520 +slash agentic + +00:31:57.520 --> 00:31:58.420 +dash engineering. + +00:31:58.640 --> 00:32:00.280 +That's talkpython.fm + +00:32:00.280 --> 00:32:01.340 +slash agentic dash + +00:32:01.340 --> 00:32:01.780 +engineering. + +00:32:01.780 --> 00:32:02.880 +The link is in your + +00:32:02.880 --> 00:32:03.680 +podcast player's + +00:32:03.680 --> 00:32:04.120 +show notes. + +00:32:05.320 --> 00:32:06.180 +If it's adding + +00:32:06.180 --> 00:32:07.240 +something new, it + +00:32:07.240 --> 00:32:08.200 +will usually need to + +00:32:08.200 --> 00:32:08.940 +go through a path, + +00:32:09.040 --> 00:32:09.600 +except if it's + +00:32:09.600 --> 00:32:10.600 +something very small. + +00:32:10.680 --> 00:32:11.240 +Let's talk about + +00:32:11.240 --> 00:32:11.900 +that for a minute. + +00:32:11.980 --> 00:32:13.280 +We got two + +00:32:13.280 --> 00:32:14.720 +representatives here + +00:32:14.720 --> 00:32:15.740 +of the newer + +00:32:15.740 --> 00:32:17.000 +breed of tools. + +00:32:17.700 --> 00:32:18.780 +What's the story + +00:32:18.780 --> 00:32:21.100 +for inconsistencies + +00:32:21.100 --> 00:32:22.760 +across interpretations + +00:32:22.760 --> 00:32:23.540 +of the spec? + +00:32:23.620 --> 00:32:24.340 +I know that there's + +00:32:24.340 --> 00:32:25.080 +slight variations. + +00:32:25.680 --> 00:32:26.940 +I've also, you + +00:32:26.940 --> 00:32:27.940 +know, not putting + +00:32:27.940 --> 00:32:28.640 +either of you on + +00:32:28.640 --> 00:32:29.440 +the spotlight, but + +00:32:29.440 --> 00:32:31.040 +like using, say, + +00:32:31.040 --> 00:32:32.320 +PyCharm and + +00:32:32.320 --> 00:32:33.380 +like writing code. + +00:32:33.500 --> 00:32:34.140 +So it's type + +00:32:34.140 --> 00:32:35.760 +checkers happy and + +00:32:35.760 --> 00:32:36.520 +then using something + +00:32:36.520 --> 00:32:37.780 +like Pyright. + +00:32:37.980 --> 00:32:39.320 +And so it has a + +00:32:39.320 --> 00:32:39.940 +real different + +00:32:39.940 --> 00:32:41.100 +interpretation of + +00:32:41.100 --> 00:32:42.160 +what you should let + +00:32:42.160 --> 00:32:42.860 +slide and what you + +00:32:42.860 --> 00:32:43.140 +shouldn't. + +00:32:43.200 --> 00:32:43.820 +I feel like Pyright + +00:32:43.820 --> 00:32:45.400 +is more, much more + +00:32:45.400 --> 00:32:47.300 +focused on like + +00:32:47.300 --> 00:32:48.200 +enforcing the + +00:32:48.200 --> 00:32:49.880 +nullability and or + +00:32:49.880 --> 00:32:50.660 +the lack thereof. + +00:32:50.700 --> 00:32:51.860 +And it warns of + +00:32:51.860 --> 00:32:52.960 +inconsistencies there + +00:32:52.960 --> 00:32:53.800 +where PyCharm doesn't + +00:32:53.800 --> 00:32:54.680 +seem to care as much. + +00:32:54.980 --> 00:32:55.500 +I don't know which + +00:32:55.500 --> 00:32:56.180 +one I like better, + +00:32:56.260 --> 00:32:56.780 +but I know they're + +00:32:56.780 --> 00:32:57.120 +different. + +00:32:57.120 --> 00:32:57.980 +And if I write code + +00:32:57.980 --> 00:32:58.480 +in one that I + +00:32:58.480 --> 00:32:59.220 +open the other, I'm + +00:32:59.220 --> 00:33:00.300 +like, huh, why is it + +00:33:00.300 --> 00:33:00.580 +upset? + +00:33:00.720 --> 00:33:01.480 +It seemed like it + +00:33:01.480 --> 00:33:01.900 +was fine. + +00:33:02.180 --> 00:33:02.540 +How do you all + +00:33:02.540 --> 00:33:03.100 +navigate this? + +00:33:03.260 --> 00:33:03.380 +Yeah. + +00:33:03.560 --> 00:33:04.580 +One thing useful to + +00:33:04.580 --> 00:33:05.480 +say about the spec + +00:33:05.480 --> 00:33:06.360 +there is that the + +00:33:06.360 --> 00:33:07.780 +spec covers a lot of + +00:33:07.780 --> 00:33:08.060 +things. + +00:33:08.160 --> 00:33:08.900 +In particular, it + +00:33:08.900 --> 00:33:09.740 +tends to cover sort + +00:33:09.740 --> 00:33:10.860 +of the details of + +00:33:10.860 --> 00:33:11.740 +more advanced type + +00:33:11.740 --> 00:33:12.560 +system features. + +00:33:12.940 --> 00:33:13.580 +But there's a lot of + +00:33:13.580 --> 00:33:14.980 +very fundamental + +00:33:14.980 --> 00:33:16.560 +stuff about how a + +00:33:16.560 --> 00:33:17.460 +type checker works + +00:33:17.460 --> 00:33:18.900 +in terms of how it + +00:33:18.900 --> 00:33:19.960 +does inference and + +00:33:19.960 --> 00:33:20.700 +how it does type + +00:33:20.700 --> 00:33:21.360 +narrowing. + +00:33:21.440 --> 00:33:22.460 +And even in some + +00:33:22.460 --> 00:33:23.100 +cases, like you + +00:33:23.100 --> 00:33:23.660 +mentioned, you know, + +00:33:23.660 --> 00:33:24.560 +what it chooses to + +00:33:24.560 --> 00:33:25.740 +emit errors on that + +00:33:25.740 --> 00:33:27.080 +isn't really covered + +00:33:27.080 --> 00:33:28.560 +by the spec, partly + +00:33:28.560 --> 00:33:29.760 +maybe because we + +00:33:29.760 --> 00:33:30.400 +haven't gotten to + +00:33:30.400 --> 00:33:31.260 +it and also partly + +00:33:31.260 --> 00:33:33.500 +intentionally in that + +00:33:33.500 --> 00:33:35.160 +there may be room in + +00:33:35.160 --> 00:33:35.880 +some of those cases + +00:33:35.880 --> 00:33:36.600 +for different type + +00:33:36.600 --> 00:33:37.400 +checkers to work + +00:33:37.400 --> 00:33:38.100 +differently if they're + +00:33:38.100 --> 00:33:38.680 +serving different + +00:33:38.680 --> 00:33:39.040 +needs. + +00:33:39.040 --> 00:33:40.440 +Like if PyCharm is + +00:33:40.440 --> 00:33:41.420 +primarily concerned + +00:33:41.420 --> 00:33:42.200 +about being a + +00:33:42.200 --> 00:33:44.020 +useful kind of IDE + +00:33:44.020 --> 00:33:45.240 +and providing go-to + +00:33:45.240 --> 00:33:46.100 +definition and that + +00:33:46.100 --> 00:33:47.340 +sort of thing, maybe + +00:33:47.340 --> 00:33:48.900 +emitting lots of + +00:33:48.900 --> 00:33:50.000 +warnings or errors + +00:33:50.000 --> 00:33:51.160 +and all kinds of + +00:33:51.160 --> 00:33:52.060 +things where your + +00:33:52.060 --> 00:33:52.760 +code might be doing + +00:33:52.760 --> 00:33:54.040 +something wrong isn't + +00:33:54.040 --> 00:33:54.920 +as high a priority. + +00:33:54.920 --> 00:33:56.200 +And another type + +00:33:56.200 --> 00:33:56.780 +checker might have + +00:33:56.780 --> 00:33:57.640 +a different priority. + +00:33:57.860 --> 00:33:58.760 +One thing I do + +00:33:58.760 --> 00:33:59.820 +want to mention is + +00:33:59.820 --> 00:34:01.320 +that it may not + +00:34:01.320 --> 00:34:02.540 +seem like it, but + +00:34:02.540 --> 00:34:04.080 +things are already + +00:34:04.080 --> 00:34:05.900 +much better than + +00:34:05.900 --> 00:34:06.700 +they used to be. + +00:34:07.120 --> 00:34:08.160 +Like previously, I + +00:34:08.160 --> 00:34:09.200 +worked on a + +00:34:09.200 --> 00:34:09.700 +different type + +00:34:09.700 --> 00:34:10.520 +checker called + +00:34:10.520 --> 00:34:11.420 +PyType. + +00:34:11.580 --> 00:34:12.380 +And at that time, + +00:34:12.420 --> 00:34:13.140 +it was, you know, + +00:34:13.480 --> 00:34:14.280 +sort of the Wild + +00:34:14.280 --> 00:34:14.640 +West. + +00:34:14.760 --> 00:34:15.440 +Like we want to + +00:34:15.440 --> 00:34:16.680 +know how other + +00:34:16.680 --> 00:34:17.700 +type checkers like + +00:34:17.700 --> 00:34:18.400 +do something. + +00:34:18.520 --> 00:34:18.860 +Well, you know, + +00:34:18.880 --> 00:34:19.500 +like open up the + +00:34:19.500 --> 00:34:20.260 +mypy playground, + +00:34:20.520 --> 00:34:21.040 +open up the + +00:34:21.040 --> 00:34:22.380 +PyRite playground, + +00:34:22.800 --> 00:34:24.660 +see what tells + +00:34:24.660 --> 00:34:24.880 +you. + +00:34:25.020 --> 00:34:25.920 +Now we at least + +00:34:25.920 --> 00:34:26.740 +have spec and + +00:34:26.740 --> 00:34:27.760 +conformance tests. + +00:34:28.000 --> 00:34:28.460 +Yeah, that's + +00:34:28.460 --> 00:34:28.880 +really cool. + +00:34:29.080 --> 00:34:29.640 +How much would + +00:34:29.640 --> 00:34:31.260 +you say that + +00:34:31.260 --> 00:34:32.940 +your two type + +00:34:32.940 --> 00:34:34.140 +checkers maybe + +00:34:34.140 --> 00:34:35.420 +bring in mypy + +00:34:35.420 --> 00:34:35.900 +as well? + +00:34:36.020 --> 00:34:36.460 +Like how much + +00:34:36.460 --> 00:34:37.200 +do they agree + +00:34:37.200 --> 00:34:38.060 +versus disagree? + +00:34:38.320 --> 00:34:38.880 +You know, like + +00:34:38.880 --> 00:34:40.060 +you only see the + +00:34:40.060 --> 00:34:40.380 +differences. + +00:34:40.540 --> 00:34:41.140 +You don't see in + +00:34:41.140 --> 00:34:41.900 +which ways that + +00:34:41.900 --> 00:34:42.640 +they are the same + +00:34:42.640 --> 00:34:43.840 +as a consumer of + +00:34:43.840 --> 00:34:44.380 +them so much, + +00:34:44.440 --> 00:34:44.580 +right? + +00:34:44.580 --> 00:34:45.040 +You're like, why + +00:34:45.040 --> 00:34:45.420 +is this one + +00:34:45.420 --> 00:34:46.060 +squiggly when it + +00:34:46.060 --> 00:34:46.680 +wasn't squiggly + +00:34:46.680 --> 00:34:47.120 +before? + +00:34:47.600 --> 00:34:48.540 +But how similar + +00:34:48.540 --> 00:34:49.020 +or different are + +00:34:49.020 --> 00:34:49.140 +they? + +00:34:49.240 --> 00:34:49.640 +I don't know how + +00:34:49.640 --> 00:34:50.260 +we would quantify + +00:34:50.260 --> 00:34:50.680 +that. + +00:34:51.000 --> 00:34:51.900 +I think there's a + +00:34:51.900 --> 00:34:52.640 +lot that is the + +00:34:52.640 --> 00:34:53.580 +same just because + +00:34:53.580 --> 00:34:54.760 +it's based on how + +00:34:54.760 --> 00:34:55.480 +Python actually + +00:34:55.480 --> 00:34:55.860 +works. + +00:34:56.200 --> 00:34:56.960 +We're both trying + +00:34:56.960 --> 00:34:57.420 +to model the + +00:34:57.420 --> 00:34:58.280 +same language and + +00:34:58.280 --> 00:34:58.540 +then there's + +00:34:58.540 --> 00:34:59.100 +certainly also + +00:34:59.100 --> 00:34:59.680 +plenty of + +00:34:59.680 --> 00:35:00.660 +differences or + +00:35:00.660 --> 00:35:01.120 +things that we + +00:35:01.120 --> 00:35:01.700 +handle differently. + +00:35:02.060 --> 00:35:02.960 +So Rebecca, do you + +00:35:02.960 --> 00:35:03.360 +have a better way + +00:35:03.360 --> 00:35:04.020 +to quantify that? + +00:35:04.400 --> 00:35:05.360 +Yeah, I agree. + +00:35:05.360 --> 00:35:06.840 +it's hard to + +00:35:06.840 --> 00:35:08.180 +quantify, I suppose, + +00:35:08.300 --> 00:35:08.940 +talk a bit + +00:35:08.940 --> 00:35:10.220 +abstractly about + +00:35:10.220 --> 00:35:11.040 +various type + +00:35:11.040 --> 00:35:12.640 +checkers philosophies. + +00:35:13.180 --> 00:35:14.520 +With Pyrefly, we + +00:35:14.520 --> 00:35:15.380 +really try to do + +00:35:15.380 --> 00:35:16.360 +a lot of type + +00:35:16.360 --> 00:35:16.960 +inference. + +00:35:17.220 --> 00:35:17.940 +So that's a way + +00:35:17.940 --> 00:35:18.840 +in which we + +00:35:18.840 --> 00:35:19.760 +intentionally diverge + +00:35:19.760 --> 00:35:21.140 +a bit from mypy. + +00:35:21.320 --> 00:35:22.560 +But other than + +00:35:22.560 --> 00:35:23.300 +that deliberate + +00:35:23.300 --> 00:35:24.260 +decision, if we + +00:35:24.260 --> 00:35:24.880 +see ways in which + +00:35:24.880 --> 00:35:25.940 +we are accidentally + +00:35:25.940 --> 00:35:27.120 +different, we do + +00:35:27.120 --> 00:35:28.800 +try to fix that + +00:35:28.800 --> 00:35:29.460 +because otherwise + +00:35:29.460 --> 00:35:30.540 +people would have + +00:35:30.540 --> 00:35:31.580 +a hard time running + +00:35:31.580 --> 00:35:32.160 +multiple type + +00:35:32.160 --> 00:35:32.760 +checkers or + +00:35:32.760 --> 00:35:33.220 +migrating. + +00:35:33.460 --> 00:35:33.960 +Yeah, differences + +00:35:33.960 --> 00:35:34.980 +obviously cause + +00:35:35.620 --> 00:35:36.560 +pain for users + +00:35:36.560 --> 00:35:37.220 +who are using + +00:35:37.220 --> 00:35:37.760 +multiple type + +00:35:37.760 --> 00:35:38.340 +checkers or + +00:35:38.340 --> 00:35:39.120 +writing libraries + +00:35:39.120 --> 00:35:39.580 +that need to + +00:35:39.580 --> 00:35:40.160 +support multiple + +00:35:40.160 --> 00:35:40.820 +type checkers. + +00:35:40.960 --> 00:35:41.960 +So like Rebecca + +00:35:41.960 --> 00:35:43.000 +said, it's like if + +00:35:43.000 --> 00:35:43.720 +we are different + +00:35:43.720 --> 00:35:44.440 +from other type + +00:35:44.440 --> 00:35:45.040 +checkers, we want + +00:35:45.040 --> 00:35:45.540 +to be sure that + +00:35:45.540 --> 00:35:45.940 +there's a good + +00:35:45.940 --> 00:35:46.920 +reason for that + +00:35:46.920 --> 00:35:47.340 +difference. + +00:35:47.600 --> 00:35:47.960 +The difference + +00:35:47.960 --> 00:35:49.040 +should be because + +00:35:49.040 --> 00:35:49.780 +of philosophical + +00:35:49.780 --> 00:35:51.580 +choice, not just + +00:35:51.580 --> 00:35:52.760 +you happen to have + +00:35:52.760 --> 00:35:53.480 +chosen slightly + +00:35:53.480 --> 00:35:54.140 +differently, right? + +00:35:54.420 --> 00:35:54.840 +Yeah, and it's + +00:35:54.840 --> 00:35:56.420 +not just people who + +00:35:56.420 --> 00:35:57.740 +run different type + +00:35:57.740 --> 00:35:58.240 +checkers. + +00:35:58.800 --> 00:35:59.600 +Like you pointed + +00:35:59.600 --> 00:36:00.260 +out, Carl, a lot + +00:36:00.260 --> 00:36:01.140 +of times it is if I + +00:36:01.140 --> 00:36:02.500 +have a library and + +00:36:02.500 --> 00:36:03.740 +then different people + +00:36:03.740 --> 00:36:04.420 +want to consume + +00:36:04.420 --> 00:36:05.360 +that library, then + +00:36:05.360 --> 00:36:06.600 +their type checker + +00:36:06.600 --> 00:36:07.300 +may or may not + +00:36:07.300 --> 00:36:08.200 +warn them about how + +00:36:08.200 --> 00:36:09.220 +my library declares + +00:36:09.220 --> 00:36:10.660 +its types and so + +00:36:10.660 --> 00:36:10.960 +on. + +00:36:11.700 --> 00:36:12.660 +I'll give you a + +00:36:12.660 --> 00:36:13.760 +real quick example. + +00:36:14.380 --> 00:36:16.540 +I have a, I can't + +00:36:16.540 --> 00:36:17.100 +remember which one + +00:36:17.100 --> 00:36:17.420 +it was, I have + +00:36:17.420 --> 00:36:17.880 +three or four + +00:36:17.880 --> 00:36:19.000 +different open + +00:36:19.000 --> 00:36:19.620 +source libraries + +00:36:19.620 --> 00:36:20.140 +that I've created + +00:36:20.140 --> 00:36:21.700 +that somehow work + +00:36:21.700 --> 00:36:22.960 +with creating, + +00:36:23.540 --> 00:36:24.440 +basically passing + +00:36:24.440 --> 00:36:25.600 +data to templates + +00:36:25.600 --> 00:36:26.980 +in web apps, + +00:36:27.340 --> 00:36:27.460 +right? + +00:36:27.500 --> 00:36:28.700 +So one is like I + +00:36:28.700 --> 00:36:29.260 +want to use the + +00:36:29.260 --> 00:36:30.020 +Chameleon web + +00:36:30.020 --> 00:36:30.740 +template framework, + +00:36:30.740 --> 00:36:31.480 +but with fast + +00:36:31.480 --> 00:36:32.680 +API or with + +00:36:32.680 --> 00:36:33.480 +Flask or there's + +00:36:33.480 --> 00:36:33.920 +some other + +00:36:33.920 --> 00:36:34.580 +variations like + +00:36:34.580 --> 00:36:35.360 +partials and so + +00:36:35.360 --> 00:36:35.500 +on. + +00:36:35.700 --> 00:36:36.360 +I can't remember + +00:36:36.360 --> 00:36:37.180 +which one, but it + +00:36:37.180 --> 00:36:37.500 +doesn't really + +00:36:37.500 --> 00:36:37.720 +matter. + +00:36:37.840 --> 00:36:38.280 +One of them + +00:36:38.280 --> 00:36:39.760 +decorated a + +00:36:39.760 --> 00:36:40.680 +Flask. + +00:36:40.820 --> 00:36:41.600 +I think it was a + +00:36:41.600 --> 00:36:42.420 +Flask, especially + +00:36:42.420 --> 00:36:43.480 +makes it irrelevant. + +00:36:43.900 --> 00:36:44.980 +A Flask endpoint + +00:36:44.980 --> 00:36:46.760 +and PyRite was + +00:36:46.760 --> 00:36:47.660 +really upset. + +00:36:47.860 --> 00:36:48.480 +Like the error + +00:36:48.480 --> 00:36:49.460 +message filled the + +00:36:49.460 --> 00:36:50.620 +entire page of + +00:36:50.620 --> 00:36:51.240 +how it was + +00:36:51.240 --> 00:36:53.000 +inconsistent with + +00:36:53.000 --> 00:36:54.020 +what it expected + +00:36:54.020 --> 00:36:54.920 +for the definition + +00:36:54.920 --> 00:36:55.980 +of the Flask + +00:36:55.980 --> 00:36:56.500 +view method. + +00:36:56.600 --> 00:36:57.720 +I'm like, no one + +00:36:57.720 --> 00:36:58.320 +is going to call + +00:36:58.320 --> 00:36:58.540 +this. + +00:36:58.540 --> 00:36:58.980 +Like what does + +00:36:58.980 --> 00:36:59.640 +it even matter + +00:36:59.640 --> 00:37:00.840 +what this type + +00:37:00.840 --> 00:37:01.080 +is? + +00:37:01.180 --> 00:37:01.960 +It still runs + +00:37:01.960 --> 00:37:02.260 +fine. + +00:37:02.320 --> 00:37:02.920 +The runtime is + +00:37:02.920 --> 00:37:03.260 +fine. + +00:37:03.540 --> 00:37:04.780 +You know, it's no + +00:37:04.780 --> 00:37:05.480 +problem with this + +00:37:05.480 --> 00:37:05.860 +decorator. + +00:37:05.980 --> 00:37:07.260 +It worked fine, but + +00:37:07.260 --> 00:37:08.800 +something about the + +00:37:08.800 --> 00:37:10.040 +way that the Flask + +00:37:10.040 --> 00:37:11.980 +at get returned + +00:37:11.980 --> 00:37:12.820 +the type versus + +00:37:12.820 --> 00:37:13.560 +what my thing + +00:37:13.560 --> 00:37:14.500 +returned varied in + +00:37:14.500 --> 00:37:15.060 +like a really + +00:37:15.060 --> 00:37:16.320 +slight way. + +00:37:16.460 --> 00:37:17.600 +I didn't care, but + +00:37:17.600 --> 00:37:18.600 +somebody was using + +00:37:18.600 --> 00:37:19.780 +some editor that + +00:37:19.780 --> 00:37:20.700 +used PyRite and + +00:37:20.700 --> 00:37:21.140 +they're like, you + +00:37:21.140 --> 00:37:22.380 +have to help fix + +00:37:22.380 --> 00:37:22.560 +this. + +00:37:22.640 --> 00:37:23.300 +I can't take all + +00:37:23.300 --> 00:37:23.760 +these warnings. + +00:37:23.880 --> 00:37:24.500 +They're huge and + +00:37:24.500 --> 00:37:25.080 +they're everywhere. + +00:37:25.420 --> 00:37:26.780 +Like, okay, I'll + +00:37:26.780 --> 00:37:27.360 +go fix it. + +00:37:27.360 --> 00:37:27.720 +Right. + +00:37:27.720 --> 00:37:29.060 +And I went and I + +00:37:29.060 --> 00:37:30.740 +put way more effort + +00:37:30.740 --> 00:37:31.440 +than was justified + +00:37:31.440 --> 00:37:32.840 +into a function type + +00:37:32.840 --> 00:37:33.340 +that no one ever + +00:37:33.340 --> 00:37:34.580 +calls just to make + +00:37:34.580 --> 00:37:36.080 +the errors on some + +00:37:36.080 --> 00:37:37.140 +type checker I + +00:37:37.140 --> 00:37:38.420 +didn't use go + +00:37:38.420 --> 00:37:38.800 +away. + +00:37:38.920 --> 00:37:39.220 +Right. + +00:37:39.260 --> 00:37:39.800 +And that's the + +00:37:39.800 --> 00:37:40.540 +kind of thing where + +00:37:40.540 --> 00:37:41.300 +it becomes just a + +00:37:41.300 --> 00:37:41.580 +headache. + +00:37:41.760 --> 00:37:42.260 +I don't know. + +00:37:42.280 --> 00:37:42.820 +I wish I remember. + +00:37:42.880 --> 00:37:43.540 +I probably got that + +00:37:43.540 --> 00:37:44.800 +written down in an + +00:37:44.800 --> 00:37:45.900 +issue somebody filed, + +00:37:46.000 --> 00:37:46.840 +but it was, it was + +00:37:46.840 --> 00:37:47.520 +a gnarly error. + +00:37:47.660 --> 00:37:48.520 +And, or if you're + +00:37:48.520 --> 00:37:49.120 +working on an open + +00:37:49.120 --> 00:37:50.240 +source project, you + +00:37:50.240 --> 00:37:50.900 +know, you can't make + +00:37:50.900 --> 00:37:51.700 +everybody use the + +00:37:51.700 --> 00:37:53.100 +same editor that + +00:37:53.100 --> 00:37:53.940 +wants to contribute + +00:37:53.940 --> 00:37:54.800 +on a big project. + +00:37:55.160 --> 00:37:55.980 +And so you might run + +00:37:55.980 --> 00:37:56.880 +into this variation as + +00:37:56.880 --> 00:37:57.020 +well. + +00:37:57.020 --> 00:37:57.580 +So there's a lot + +00:37:57.580 --> 00:37:58.000 +of cases. + +00:37:58.240 --> 00:37:58.320 +Yeah. + +00:37:58.340 --> 00:37:58.940 +It can be really + +00:37:58.940 --> 00:38:00.220 +difficult to make + +00:38:00.220 --> 00:38:01.240 +these decisions about + +00:38:01.240 --> 00:38:03.020 +what kind of, what + +00:38:03.020 --> 00:38:04.460 +sorts of errors people + +00:38:04.460 --> 00:38:06.220 +want their type checker + +00:38:06.220 --> 00:38:07.560 +to catch or what's + +00:38:07.560 --> 00:38:08.120 +too pedantic. + +00:38:08.460 --> 00:38:09.440 +You want your type + +00:38:09.440 --> 00:38:10.200 +checker to catch + +00:38:10.200 --> 00:38:11.700 +non-obvious errors, + +00:38:11.700 --> 00:38:12.740 +not just the obvious + +00:38:12.740 --> 00:38:13.440 +ones that you probably + +00:38:13.440 --> 00:38:14.020 +would have seen by + +00:38:14.020 --> 00:38:14.660 +looking at the code + +00:38:14.660 --> 00:38:15.040 +yourself. + +00:38:15.300 --> 00:38:16.020 +But then there'll be + +00:38:16.020 --> 00:38:17.440 +cases where somebody + +00:38:17.440 --> 00:38:18.040 +says, well, I don't + +00:38:18.040 --> 00:38:18.240 +care. + +00:38:18.300 --> 00:38:18.940 +That's too pedantic. + +00:38:19.160 --> 00:38:20.220 +And it is difficult to + +00:38:20.220 --> 00:38:20.960 +make everyone happy. + +00:38:21.640 --> 00:38:22.560 +Who decides what the + +00:38:22.560 --> 00:38:24.360 +right signature of a + +00:38:24.360 --> 00:38:25.860 +flask view and point + +00:38:25.860 --> 00:38:26.940 +should be like if you + +00:38:27.020 --> 00:38:27.900 +the framework can call + +00:38:27.900 --> 00:38:28.080 +it. + +00:38:28.080 --> 00:38:29.080 +It should be okay. + +00:38:29.080 --> 00:38:30.080 +There's not. + +00:38:30.200 --> 00:38:30.800 +Just because it had a + +00:38:30.800 --> 00:38:31.600 +decorator before, that + +00:38:31.600 --> 00:38:32.300 +doesn't mean that's the + +00:38:32.300 --> 00:38:33.360 +official structure. + +00:38:33.480 --> 00:38:34.700 +But anyway, I do think + +00:38:34.700 --> 00:38:35.440 +one of the bigger + +00:38:35.440 --> 00:38:37.000 +philosophical differences + +00:38:37.000 --> 00:38:39.120 +has to do around this + +00:38:39.120 --> 00:38:41.100 +concept of nullability. + +00:38:41.340 --> 00:38:42.080 +Do you guys call it + +00:38:42.080 --> 00:38:43.360 +nullability or none + +00:38:43.360 --> 00:38:43.920 +ability? + +00:38:44.280 --> 00:38:45.600 +Like nullability comes + +00:38:45.600 --> 00:38:46.040 +from the other + +00:38:46.040 --> 00:38:46.420 +languages. + +00:38:46.620 --> 00:38:48.080 +And by that, I mean, I + +00:38:48.080 --> 00:38:49.220 +can specify that I have + +00:38:49.220 --> 00:38:49.760 +an integer. + +00:38:49.760 --> 00:38:51.100 +And in the Python type + +00:38:51.100 --> 00:38:53.180 +system, it cannot be set + +00:38:53.180 --> 00:38:54.220 +to none, even though in + +00:38:54.220 --> 00:38:54.940 +the runtime it can. + +00:38:55.440 --> 00:38:56.740 +It has to be a concrete + +00:38:56.740 --> 00:38:58.820 +int type unless you make + +00:38:58.820 --> 00:39:00.940 +it a optional int or an + +00:39:00.940 --> 00:39:02.460 +int pipe none or one of + +00:39:02.460 --> 00:39:03.480 +those type things, right? + +00:39:03.580 --> 00:39:05.340 +And how strong that gets + +00:39:05.340 --> 00:39:06.480 +enforced seems to be one + +00:39:06.480 --> 00:39:07.980 +of the biggest difference + +00:39:07.980 --> 00:39:09.460 +of opinions that I've + +00:39:09.460 --> 00:39:10.040 +seen around. + +00:39:10.240 --> 00:39:10.860 +Like, how do you all + +00:39:10.860 --> 00:39:11.460 +think about that? + +00:39:11.600 --> 00:39:12.580 +That's interesting to me + +00:39:12.580 --> 00:39:13.540 +that that's your + +00:39:13.540 --> 00:39:15.100 +experience because my + +00:39:15.100 --> 00:39:16.000 +experience has been that + +00:39:16.000 --> 00:39:16.860 +that's actually an area + +00:39:16.860 --> 00:39:18.220 +where everyone seems to + +00:39:18.220 --> 00:39:19.120 +agree as far as I can + +00:39:19.120 --> 00:39:21.300 +tell that these are is + +00:39:21.300 --> 00:39:22.100 +an important source of + +00:39:22.100 --> 00:39:22.980 +bugs and it's better to + +00:39:22.980 --> 00:39:23.400 +catch them. + +00:39:23.480 --> 00:39:24.120 +So I think all of the + +00:39:24.120 --> 00:39:25.700 +type checkers, maybe you + +00:39:25.700 --> 00:39:26.720 +said PyCharm doesn't. + +00:39:26.900 --> 00:39:27.720 +I don't think PyCharm + +00:39:27.720 --> 00:39:28.220 +does that. + +00:39:28.340 --> 00:39:29.680 +I'm pretty sure it + +00:39:29.680 --> 00:39:31.440 +doesn't because I agree + +00:39:31.440 --> 00:39:32.200 +that it's an important + +00:39:32.200 --> 00:39:34.020 +thing to check, but it's + +00:39:34.020 --> 00:39:35.920 +also a point of a lot of + +00:39:35.920 --> 00:39:36.480 +friction. + +00:39:37.100 --> 00:39:37.920 +And by that, I mean, + +00:39:38.180 --> 00:39:39.480 +let's suppose I'm going + +00:39:39.480 --> 00:39:41.700 +to have a class that I + +00:39:41.700 --> 00:39:42.340 +need to create an + +00:39:42.340 --> 00:39:43.920 +instance of and then put + +00:39:43.920 --> 00:39:44.900 +values into. + +00:39:45.320 --> 00:39:46.580 +And I know once I put + +00:39:46.580 --> 00:39:47.680 +the values into it, let's + +00:39:47.680 --> 00:39:49.480 +say it has a user ID, I + +00:39:49.480 --> 00:39:50.960 +know for certain that + +00:39:50.960 --> 00:39:51.960 +that's going to be an + +00:39:51.960 --> 00:39:53.020 +integer, right? + +00:39:53.060 --> 00:39:54.260 +So I'd like to say user + +00:39:54.260 --> 00:39:55.520 +ID colon int because + +00:39:55.520 --> 00:39:57.000 +everywhere I use that + +00:39:57.000 --> 00:39:58.840 +object later, if it's a + +00:39:58.840 --> 00:39:59.820 +function that takes an + +00:39:59.820 --> 00:40:01.540 +int and I specify it as + +00:40:01.540 --> 00:40:03.000 +optional int, I will get + +00:40:03.000 --> 00:40:03.840 +a type check warning + +00:40:03.840 --> 00:40:05.400 +every single call site + +00:40:05.400 --> 00:40:06.960 +when I try to pass that. + +00:40:07.240 --> 00:40:08.100 +But I know from the + +00:40:08.100 --> 00:40:09.000 +semantics of the + +00:40:09.000 --> 00:40:10.780 +behavior that it's going + +00:40:10.780 --> 00:40:12.480 +to always be an int + +00:40:12.480 --> 00:40:13.440 +unless it's not + +00:40:13.440 --> 00:40:14.820 +initialized, right? + +00:40:14.820 --> 00:40:16.080 +And like in this short + +00:40:16.080 --> 00:40:17.060 +period where I want to + +00:40:17.060 --> 00:40:17.480 +create it. + +00:40:17.560 --> 00:40:18.700 +So I can't set the type + +00:40:18.700 --> 00:40:19.060 +to int. + +00:40:19.120 --> 00:40:19.660 +I have to set the + +00:40:19.660 --> 00:40:20.760 +optional int until I've + +00:40:20.760 --> 00:40:21.300 +loaded it. + +00:40:21.920 --> 00:40:22.680 +And, but there's like + +00:40:22.680 --> 00:40:23.660 +this, I don't know, + +00:40:23.720 --> 00:40:24.380 +that's, that's the part + +00:40:24.380 --> 00:40:25.480 +where I see a lot of it + +00:40:25.480 --> 00:40:27.720 +show up is inconsistencies + +00:40:27.720 --> 00:40:29.400 +and then warnings all + +00:40:29.400 --> 00:40:29.980 +over the place. + +00:40:29.980 --> 00:40:31.140 +So I'm like, well, but + +00:40:31.140 --> 00:40:32.300 +that function is actually + +00:40:32.300 --> 00:40:33.240 +checking if it's none + +00:40:33.240 --> 00:40:34.880 +and it'll return null, + +00:40:35.080 --> 00:40:35.900 +you know, none or + +00:40:35.900 --> 00:40:36.500 +something like that. + +00:40:36.500 --> 00:40:38.200 +So I totally agree with + +00:40:38.200 --> 00:40:38.380 +you. + +00:40:38.520 --> 00:40:39.420 +It's just somewhere I've + +00:40:39.420 --> 00:40:40.320 +seen the most + +00:40:40.320 --> 00:40:41.860 +inconsistencies across + +00:40:41.860 --> 00:40:43.660 +maybe PyCharm versus + +00:40:43.660 --> 00:40:44.280 +others. + +00:40:44.480 --> 00:40:45.580 +mypy also has a legacy + +00:40:45.580 --> 00:40:46.900 +mode for not checking + +00:40:46.900 --> 00:40:48.340 +none things called + +00:40:48.340 --> 00:40:49.760 +non-strict optional. + +00:40:50.200 --> 00:40:51.240 +We're trying to get + +00:40:51.240 --> 00:40:52.300 +rid of that from mypy + +00:40:52.300 --> 00:40:53.480 +because yeah, strict + +00:40:53.480 --> 00:40:54.360 +optional, like being + +00:40:54.360 --> 00:40:55.200 +strict about it is the + +00:40:55.200 --> 00:40:55.940 +more sensible thing to + +00:40:55.940 --> 00:40:56.160 +do. + +00:40:56.480 --> 00:40:57.580 +But it's possible that + +00:40:57.580 --> 00:40:58.540 +you've seen that too. + +00:40:59.020 --> 00:40:59.460 +Yeah, I agree. + +00:40:59.720 --> 00:41:00.320 +So what you mentioned + +00:41:00.320 --> 00:41:01.720 +is maybe sort of a + +00:41:01.720 --> 00:41:02.540 +special case of the + +00:41:02.540 --> 00:41:03.540 +case where you pass + +00:41:03.540 --> 00:41:04.360 +something to a class + +00:41:04.360 --> 00:41:05.380 +and there's initialization + +00:41:05.380 --> 00:41:06.440 +that changes the types. + +00:41:06.820 --> 00:41:07.660 +Doesn't necessarily have + +00:41:07.660 --> 00:41:08.220 +to deal with none. + +00:41:08.300 --> 00:41:08.960 +It could also just be + +00:41:08.960 --> 00:41:09.780 +like the attribute doesn't + +00:41:09.780 --> 00:41:10.740 +exist at all beforehand + +00:41:10.740 --> 00:41:11.240 +or something. + +00:41:11.240 --> 00:41:12.940 +Yeah, we don't have a + +00:41:12.940 --> 00:41:13.780 +good solution for that. + +00:41:13.900 --> 00:41:14.840 +Maybe there's room for + +00:41:14.840 --> 00:41:16.120 +something to support that + +00:41:16.120 --> 00:41:16.720 +use case better. + +00:41:17.020 --> 00:41:17.660 +I don't know what it + +00:41:17.660 --> 00:41:18.120 +would look like. + +00:41:18.260 --> 00:41:19.180 +In some cases, there's + +00:41:19.180 --> 00:41:20.720 +ways you can, these + +00:41:20.720 --> 00:41:21.900 +things can sometimes + +00:41:21.900 --> 00:41:22.760 +nudge you towards a + +00:41:22.760 --> 00:41:23.760 +different design that is + +00:41:23.760 --> 00:41:24.920 +actually safer and will + +00:41:24.920 --> 00:41:26.020 +avoid errors. + +00:41:26.020 --> 00:41:27.040 +Like in the kind of + +00:41:27.040 --> 00:41:27.640 +case you're talking + +00:41:27.640 --> 00:41:29.000 +about, you know, is + +00:41:29.000 --> 00:41:30.580 +it actually necessary + +00:41:30.580 --> 00:41:31.820 +that an uninitialized + +00:41:31.820 --> 00:41:32.600 +object and an + +00:41:32.600 --> 00:41:33.360 +initialized one are + +00:41:33.360 --> 00:41:34.120 +represented by the + +00:41:34.120 --> 00:41:34.580 +same type? + +00:41:34.780 --> 00:41:35.840 +Or is there a way to + +00:41:35.840 --> 00:41:37.260 +adjust the API so that + +00:41:37.260 --> 00:41:37.840 +those are actually + +00:41:37.840 --> 00:41:38.960 +different types than you + +00:41:38.960 --> 00:41:40.080 +solve the problem and + +00:41:40.080 --> 00:41:41.220 +your code is safer + +00:41:41.220 --> 00:41:41.580 +or so? + +00:41:41.580 --> 00:41:42.500 +I'm thinking like you + +00:41:42.500 --> 00:41:44.500 +submit a web form and + +00:41:44.500 --> 00:41:45.680 +before you parse it, you've + +00:41:45.680 --> 00:41:46.440 +got to create the instance + +00:41:46.440 --> 00:41:47.360 +to set the values. + +00:41:47.940 --> 00:41:48.300 +And I don't know. + +00:41:48.420 --> 00:41:49.320 +It's not worth diving into, + +00:41:49.380 --> 00:41:50.340 +but I do find this + +00:41:50.340 --> 00:41:52.260 +differentiation between like + +00:41:52.260 --> 00:41:53.880 +the strict enforcement of + +00:41:53.880 --> 00:41:55.200 +none versus not none. + +00:41:55.340 --> 00:41:56.500 +I think it's powerful and I + +00:41:56.500 --> 00:41:57.440 +do think you all are right + +00:41:57.440 --> 00:41:58.600 +that it does catch a lot of + +00:41:58.600 --> 00:41:58.840 +errors. + +00:41:58.940 --> 00:42:00.000 +It's just, it's just a + +00:42:00.000 --> 00:42:00.880 +difference and it's just an + +00:42:00.880 --> 00:42:02.280 +interesting, interesting + +00:42:02.280 --> 00:42:02.560 +choice. + +00:42:02.680 --> 00:42:03.620 +But I didn't get a + +00:42:03.620 --> 00:42:05.280 +concrete answer from the + +00:42:05.280 --> 00:42:06.140 +official counsel. + +00:42:06.720 --> 00:42:07.640 +Nullable or + +00:42:07.640 --> 00:42:08.200 +noneable? + +00:42:08.640 --> 00:42:09.140 +What is it? + +00:42:09.140 --> 00:42:09.700 +I feel like you just + +00:42:09.700 --> 00:42:10.600 +don't really even talk + +00:42:10.600 --> 00:42:11.700 +about it as a term mostly. + +00:42:12.340 --> 00:42:13.900 +It's, yeah, none is + +00:42:13.900 --> 00:42:15.420 +special in the type system + +00:42:15.420 --> 00:42:16.720 +in like how you represent + +00:42:16.720 --> 00:42:17.820 +it, but it's not really + +00:42:17.820 --> 00:42:19.660 +special in other ways. + +00:42:19.920 --> 00:42:20.940 +You don't have a term for + +00:42:20.940 --> 00:42:21.860 +int pipe none? + +00:42:22.140 --> 00:42:22.620 +Int or none. + +00:42:22.860 --> 00:42:23.780 +Historically, the term was + +00:42:23.780 --> 00:42:24.820 +optional, although I think + +00:42:24.820 --> 00:42:26.560 +that term has problems and + +00:42:26.560 --> 00:42:28.220 +we're sort of moving away + +00:42:28.220 --> 00:42:29.080 +from it because + +00:42:29.080 --> 00:42:31.680 +specifically one problem is + +00:42:31.680 --> 00:42:33.360 +that optional can mean + +00:42:33.360 --> 00:42:34.920 +you don't have to pass it + +00:42:34.920 --> 00:42:36.200 +in, like I say, as a + +00:42:36.200 --> 00:42:36.900 +function parameter. + +00:42:37.080 --> 00:42:38.200 +Let's talk a little bit + +00:42:38.200 --> 00:42:39.600 +about TypeShed. + +00:42:40.060 --> 00:42:41.440 +I think TypeShed is pretty + +00:42:41.440 --> 00:42:41.800 +interesting. + +00:42:41.940 --> 00:42:42.860 +Maybe people don't know + +00:42:42.860 --> 00:42:44.160 +too much about it. + +00:42:44.640 --> 00:42:45.840 +So I'm sure you all are + +00:42:45.840 --> 00:42:47.000 +familiar with this project + +00:42:47.000 --> 00:42:48.500 +that you can basically add + +00:42:48.500 --> 00:42:50.220 +type information that the + +00:42:50.220 --> 00:42:51.880 +libraries didn't bother to + +00:42:51.880 --> 00:42:52.740 +include for you, right? + +00:42:53.160 --> 00:42:54.460 +What are thoughts on TypeShed? + +00:42:54.620 --> 00:42:55.640 +How much do you all lean on + +00:42:55.640 --> 00:42:57.060 +this to sort of round out + +00:42:57.060 --> 00:42:58.020 +missing types? + +00:42:58.240 --> 00:42:59.160 +There are two parts to + +00:42:59.160 --> 00:43:00.040 +TypeShed, right? + +00:43:00.040 --> 00:43:02.440 +There's the standard library + +00:43:02.440 --> 00:43:04.480 +type stubs, which I think + +00:43:04.480 --> 00:43:05.600 +are invaluable. + +00:43:06.000 --> 00:43:07.260 +Like all the type checkers + +00:43:07.260 --> 00:43:07.980 +use those. + +00:43:08.480 --> 00:43:09.620 +And I mean, will the standard + +00:43:09.620 --> 00:43:10.660 +library itself ever have + +00:43:10.660 --> 00:43:11.880 +inline types? + +00:43:12.060 --> 00:43:12.640 +Who knows? + +00:43:12.700 --> 00:43:13.900 +This might be around forever. + +00:43:14.520 --> 00:43:15.900 +And then there are also + +00:43:15.900 --> 00:43:18.640 +the third party stubs. + +00:43:18.960 --> 00:43:20.320 +And I think that's what you're + +00:43:20.320 --> 00:43:20.780 +describing. + +00:43:20.940 --> 00:43:21.980 +They're libraries that for + +00:43:21.980 --> 00:43:23.160 +whatever reason don't ship + +00:43:23.160 --> 00:43:24.540 +with stubs themselves. + +00:43:24.920 --> 00:43:26.640 +Those are in TypeShed. + +00:43:26.640 --> 00:43:29.080 +And I think it's been like + +00:43:29.080 --> 00:43:31.040 +for a while, there's sort of + +00:43:31.040 --> 00:43:31.940 +been a question of like what + +00:43:31.940 --> 00:43:33.560 +we want to do with like + +00:43:33.560 --> 00:43:35.440 +TypeShed's third party stubs, + +00:43:35.540 --> 00:43:35.740 +right? + +00:43:35.860 --> 00:43:37.420 +Because like ideally like + +00:43:37.420 --> 00:43:38.480 +libraries would ship with + +00:43:38.480 --> 00:43:39.780 +their own types, but there + +00:43:39.780 --> 00:43:41.220 +are various obstacles to that. + +00:43:41.360 --> 00:43:43.120 +The obstacles that I know of + +00:43:43.120 --> 00:43:45.000 +used to be like, we want this + +00:43:45.000 --> 00:43:47.080 +to run on Python 2 and Python 3. + +00:43:47.520 --> 00:43:48.920 +Or we want it to run on + +00:43:48.920 --> 00:43:50.480 +Python 3.3 still. + +00:43:50.740 --> 00:43:52.480 +But it's been a long time + +00:43:52.480 --> 00:43:55.160 +since any non-type supporting + +00:43:55.160 --> 00:43:57.700 +version of Python was a real, + +00:43:57.840 --> 00:43:58.880 +you know, a supported type + +00:43:58.880 --> 00:43:59.500 +of thing, right? + +00:43:59.740 --> 00:44:01.300 +I mean, even 3.9 became + +00:44:01.300 --> 00:44:02.080 +deprecated. + +00:44:02.560 --> 00:44:04.440 +So on one hand, I feel like + +00:44:04.440 --> 00:44:05.880 +they could be merged in, + +00:44:05.920 --> 00:44:07.100 +but there's also a lot of + +00:44:07.100 --> 00:44:09.700 +other areas that are maybe + +00:44:09.700 --> 00:44:11.720 +we don't, they're not common, + +00:44:12.120 --> 00:44:12.300 +right? + +00:44:12.360 --> 00:44:13.900 +Like other libraries, like + +00:44:13.900 --> 00:44:15.960 +pick some, let's say Pyramid. + +00:44:16.120 --> 00:44:17.400 +I don't think the Pyramid web + +00:44:17.400 --> 00:44:19.220 +framework really ever got + +00:44:19.220 --> 00:44:20.280 +types added to it. + +00:44:20.280 --> 00:44:21.620 +Somebody could go and create + +00:44:21.620 --> 00:44:24.800 +a typeshed stub or a types + +00:44:24.800 --> 00:44:26.160 +underscore pyramid you could + +00:44:26.160 --> 00:44:27.180 +pip install and then we'll + +00:44:27.180 --> 00:44:28.040 +add the types, right? + +00:44:28.420 --> 00:44:29.680 +I certainly see it being + +00:44:29.680 --> 00:44:30.780 +really valuable for third + +00:44:30.780 --> 00:44:31.680 +party things that are just + +00:44:31.680 --> 00:44:33.000 +not going to get the type + +00:44:33.000 --> 00:44:33.840 +attention they need. + +00:44:33.960 --> 00:44:34.780 +Yeah, I think typeshed is + +00:44:34.780 --> 00:44:35.040 +great. + +00:44:35.180 --> 00:44:36.880 +I've spent a lot of time on + +00:44:36.880 --> 00:44:37.600 +improving it. + +00:44:37.800 --> 00:44:39.020 +As Rebecca said, especially + +00:44:39.020 --> 00:44:39.880 +with a standard library, + +00:44:39.980 --> 00:44:41.080 +it's irreplaceable. + +00:44:41.440 --> 00:44:42.520 +For third party libraries, + +00:44:42.780 --> 00:44:44.360 +I think it's become less + +00:44:44.360 --> 00:44:45.200 +needed over time. + +00:44:45.980 --> 00:44:47.480 +It used to be that very few + +00:44:47.480 --> 00:44:49.060 +third party libraries had + +00:44:49.060 --> 00:44:49.760 +any types. + +00:44:49.760 --> 00:44:51.480 +Now that's obviously changed. + +00:44:51.600 --> 00:44:53.080 +A lot of libraries ship + +00:44:53.080 --> 00:44:54.700 +their own types, but still + +00:44:54.700 --> 00:44:56.180 +there are quite a few types + +00:44:56.180 --> 00:44:57.680 +of libraries left where + +00:44:57.680 --> 00:44:59.460 +there aren't inline types + +00:44:59.460 --> 00:45:00.580 +and typeshed can provide + +00:45:00.580 --> 00:45:01.300 +useful types. + +00:45:01.800 --> 00:45:02.660 +I think typeshed also + +00:45:02.660 --> 00:45:03.620 +provides a service because + +00:45:03.620 --> 00:45:04.880 +it has a really great + +00:45:04.880 --> 00:45:05.980 +framework for testing + +00:45:05.980 --> 00:45:06.660 +these types. + +00:45:06.940 --> 00:45:07.820 +We have tools like step + +00:45:07.820 --> 00:45:09.260 +tests and various type + +00:45:09.260 --> 00:45:11.260 +checkers that help to + +00:45:11.260 --> 00:45:12.480 +make sure these types are + +00:45:12.480 --> 00:45:14.280 +good and meet a high + +00:45:14.280 --> 00:45:14.620 +standard. + +00:45:15.200 --> 00:45:16.100 +So yeah, I think they're + +00:45:16.100 --> 00:45:17.000 +still useful for many + +00:45:17.000 --> 00:45:17.460 +libraries. + +00:45:17.800 --> 00:45:18.640 +Yeah, I was just looking at + +00:45:18.640 --> 00:45:21.640 +the types dash flask and + +00:45:21.640 --> 00:45:23.320 +I guess it must be, must + +00:45:23.320 --> 00:45:24.440 +be gone because now type + +00:45:24.440 --> 00:45:25.520 +flask must have it + +00:45:25.520 --> 00:45:25.940 +internally. + +00:45:26.260 --> 00:45:27.720 +So it's kind of an interim + +00:45:27.720 --> 00:45:28.520 +sort of thing. + +00:45:28.580 --> 00:45:29.080 +That's pretty cool. + +00:45:29.240 --> 00:45:29.940 +In general, typeshed has + +00:45:29.940 --> 00:45:31.320 +the policy that we remove + +00:45:31.320 --> 00:45:32.460 +the snaps from typeshed if + +00:45:32.460 --> 00:45:34.120 +they are in the library + +00:45:34.120 --> 00:45:34.520 +itself. + +00:45:34.600 --> 00:45:35.340 +I find these super + +00:45:35.340 --> 00:45:37.160 +valuable because if there's + +00:45:37.160 --> 00:45:38.120 +a library I want to work + +00:45:38.120 --> 00:45:39.440 +with and it just doesn't + +00:45:39.440 --> 00:45:40.240 +have types for whatever + +00:45:40.240 --> 00:45:41.760 +reason, you can install + +00:45:41.760 --> 00:45:43.000 +stuff from here and all + +00:45:43.000 --> 00:45:43.920 +of a sudden your editor's + +00:45:43.920 --> 00:45:44.740 +way happier. + +00:45:44.740 --> 00:45:47.120 +I mean, I know we, you + +00:45:47.120 --> 00:45:48.340 +all agreed on like the + +00:45:48.340 --> 00:45:49.960 +API boundaries and I did + +00:45:49.960 --> 00:45:50.400 +as well. + +00:45:50.480 --> 00:45:51.040 +It's like that's one of + +00:45:51.040 --> 00:45:51.880 +the really cool things. + +00:45:51.980 --> 00:45:52.780 +The other thing that + +00:45:52.780 --> 00:45:53.620 +really makes me excited + +00:45:53.620 --> 00:45:54.960 +about types is if I hit + +00:45:54.960 --> 00:45:57.080 +dot in my editor, I get + +00:45:57.080 --> 00:45:58.960 +a meaningful list of real + +00:45:58.960 --> 00:46:00.420 +information about what I'm + +00:46:00.420 --> 00:46:00.900 +working on. + +00:46:00.960 --> 00:46:02.040 +And so adding, adding + +00:46:02.040 --> 00:46:03.320 +these types of things are + +00:46:03.320 --> 00:46:04.000 +pretty interesting. + +00:46:04.260 --> 00:46:05.440 +I want to ask you all + +00:46:05.440 --> 00:46:07.560 +about sort of these rogue, + +00:46:07.760 --> 00:46:10.300 +rogue tools that do stuff + +00:46:10.300 --> 00:46:11.460 +with Python typing that + +00:46:11.460 --> 00:46:12.900 +maybe y'all didn't intend. + +00:46:12.900 --> 00:46:14.060 +Like we all mentioned + +00:46:14.060 --> 00:46:15.840 +Pydantic, we've got + +00:46:15.840 --> 00:46:18.240 +Typer and FastAPI, but + +00:46:18.240 --> 00:46:19.640 +even a little farther out + +00:46:19.640 --> 00:46:20.960 +there is a bear type. + +00:46:21.020 --> 00:46:21.620 +Are you familiar with + +00:46:21.620 --> 00:46:22.160 +bear type? + +00:46:22.600 --> 00:46:22.740 +Yeah. + +00:46:22.800 --> 00:46:23.640 +Bear type's interesting. + +00:46:24.280 --> 00:46:26.840 +You can import, they have + +00:46:26.840 --> 00:46:27.180 +fun. + +00:46:27.840 --> 00:46:29.080 +They have fun with their, + +00:46:29.180 --> 00:46:30.920 +their import names and + +00:46:30.920 --> 00:46:31.240 +stuff. + +00:46:31.480 --> 00:46:33.120 +But basically you can put + +00:46:33.120 --> 00:46:35.080 +a, either a decorator onto + +00:46:35.080 --> 00:46:36.940 +some sort of call site or + +00:46:36.940 --> 00:46:37.860 +something, or you can just + +00:46:37.860 --> 00:46:40.480 +do it to an entire package + +00:46:40.480 --> 00:46:41.700 +or entire modules rather. + +00:46:41.700 --> 00:46:43.740 +So just run bear type + +00:46:43.740 --> 00:46:45.200 +dot claw import bear type + +00:46:45.200 --> 00:46:45.440 +this. + +00:46:45.540 --> 00:46:47.380 +And then it actually turns + +00:46:47.380 --> 00:46:49.020 +into runtime type checks. + +00:46:49.640 --> 00:46:51.240 +Good idea, bad idea. + +00:46:51.440 --> 00:46:51.800 +Interesting. + +00:46:52.380 --> 00:46:53.060 +What do you all think? + +00:46:53.340 --> 00:46:54.860 +So un-Pythonic, you won't + +00:46:54.860 --> 00:46:55.920 +even open the webpage. + +00:46:56.060 --> 00:46:57.320 +People should feel free to + +00:46:57.320 --> 00:46:58.800 +write whatever code helps + +00:46:58.800 --> 00:47:00.220 +them make like better + +00:47:00.220 --> 00:47:00.600 +software. + +00:47:01.080 --> 00:47:01.940 +I haven't really used + +00:47:01.940 --> 00:47:02.840 +bear type much myself, + +00:47:03.040 --> 00:47:04.240 +but it's clearly useful for + +00:47:04.240 --> 00:47:04.700 +some people. + +00:47:05.020 --> 00:47:06.200 +And I think generally in + +00:47:06.200 --> 00:47:07.300 +designing a type system, + +00:47:07.400 --> 00:47:08.180 +we should try to + +00:47:08.180 --> 00:47:09.580 +accommodate all users who + +00:47:09.580 --> 00:47:10.460 +do useful things to the + +00:47:10.460 --> 00:47:10.980 +type system. + +00:47:10.980 --> 00:47:12.000 +And that includes things + +00:47:12.000 --> 00:47:13.020 +like Pydentic or bear + +00:47:13.020 --> 00:47:13.280 +type. + +00:47:13.400 --> 00:47:14.240 +It's pretty fast. + +00:47:14.360 --> 00:47:15.900 +It's not as big of a hit + +00:47:15.900 --> 00:47:17.540 +as you would, you would + +00:47:17.540 --> 00:47:17.920 +imagine. + +00:47:18.580 --> 00:47:19.360 +They, let me see, what + +00:47:19.360 --> 00:47:20.560 +are they, somewhere they + +00:47:20.560 --> 00:47:21.960 +had a really fun, fun + +00:47:21.960 --> 00:47:23.240 +saying in here, but here + +00:47:23.240 --> 00:47:23.560 +we go. + +00:47:24.100 --> 00:47:25.640 +Bear type brings Rust C++ + +00:47:25.640 --> 00:47:26.780 +inspired zero cost + +00:47:26.780 --> 00:47:27.860 +abstractions into the + +00:47:27.860 --> 00:47:29.400 +lawless world of dynamically + +00:47:29.400 --> 00:47:31.020 +typed Python by enforcing + +00:47:31.020 --> 00:47:32.360 +type safety at the granular + +00:47:32.360 --> 00:47:33.660 +level of functions and + +00:47:33.660 --> 00:47:35.540 +methods against type hints + +00:47:35.540 --> 00:47:36.800 +standardized by the Python + +00:47:36.800 --> 00:47:37.260 +community. + +00:47:37.260 --> 00:47:39.260 +order one, non-amortized + +00:47:39.260 --> 00:47:40.200 +worst case time with + +00:47:40.200 --> 00:47:41.420 +negligible constant factors. + +00:47:41.520 --> 00:47:42.240 +Like, how about that? + +00:47:43.180 --> 00:47:43.940 +No, it's a pretty neat + +00:47:43.940 --> 00:47:44.820 +library and it's pretty + +00:47:44.820 --> 00:47:45.120 +fast. + +00:47:45.240 --> 00:47:46.660 +I honestly, I've never used + +00:47:46.660 --> 00:47:47.200 +it in production. + +00:47:47.720 --> 00:47:49.680 +Having type hints and + +00:47:49.680 --> 00:47:50.840 +squigglies in the editors + +00:47:50.840 --> 00:47:53.540 +or in the linters has always + +00:47:53.540 --> 00:47:54.880 +been enough for me, but I + +00:47:54.880 --> 00:47:56.120 +can see using this if it's + +00:47:56.120 --> 00:47:57.520 +really critical and you're + +00:47:57.520 --> 00:47:58.460 +having issues, maybe you + +00:47:58.460 --> 00:47:59.600 +want to catch some runtime + +00:47:59.600 --> 00:48:00.000 +errors. + +00:48:00.340 --> 00:48:00.720 +I don't know. + +00:48:00.960 --> 00:48:02.000 +It's not quite an endorsement, + +00:48:02.120 --> 00:48:03.300 +but it sure is like a, huh, + +00:48:03.400 --> 00:48:04.340 +that's different. + +00:48:04.340 --> 00:48:06.120 +I definitely think that the + +00:48:06.120 --> 00:48:08.240 +extent to which type + +00:48:08.240 --> 00:48:10.820 +checkers may have a + +00:48:10.820 --> 00:48:11.760 +different understanding of + +00:48:11.760 --> 00:48:12.780 +your code from what happens + +00:48:12.780 --> 00:48:14.040 +at runtime and there isn't + +00:48:14.040 --> 00:48:15.660 +anything built in to catch + +00:48:15.660 --> 00:48:17.420 +that is sometimes a pain + +00:48:17.420 --> 00:48:17.740 +point. + +00:48:18.160 --> 00:48:19.300 +And so the desire to have + +00:48:19.300 --> 00:48:20.860 +your type annotations, to + +00:48:20.860 --> 00:48:22.360 +find out at runtime if your + +00:48:22.360 --> 00:48:23.420 +type annotations are telling + +00:48:23.420 --> 00:48:24.880 +you a lie, it makes a lot of + +00:48:24.880 --> 00:48:26.560 +sense why people would like + +00:48:26.560 --> 00:48:26.860 +that. + +00:48:27.140 --> 00:48:27.500 +I mean, it's something + +00:48:27.500 --> 00:48:28.980 +used to from other languages + +00:48:28.980 --> 00:48:30.080 +where the type checker is built + +00:48:30.080 --> 00:48:30.760 +into the compiler. + +00:48:30.920 --> 00:48:31.100 +Right. + +00:48:31.160 --> 00:48:32.480 +You get like a runtime type + +00:48:32.480 --> 00:48:33.400 +cast, like cannot. + +00:48:33.400 --> 00:48:35.280 +We kind of get that if you + +00:48:35.280 --> 00:48:37.480 +try to parse a thing, you + +00:48:37.480 --> 00:48:38.860 +know, like put the int + +00:48:38.860 --> 00:48:40.280 +param around a string and + +00:48:40.280 --> 00:48:41.840 +it's not really a parsable + +00:48:41.840 --> 00:48:42.280 +as an int. + +00:48:42.460 --> 00:48:43.740 +But for like real type + +00:48:43.740 --> 00:48:44.720 +information, I think + +00:48:44.720 --> 00:48:45.860 +personally I would use this + +00:48:45.860 --> 00:48:48.280 +as like I might apply types, + +00:48:48.700 --> 00:48:50.480 +type checking to a module + +00:48:50.480 --> 00:48:52.780 +for debugging and development + +00:48:52.780 --> 00:48:53.700 +for a minute and just see + +00:48:53.700 --> 00:48:54.500 +what happens and then turn + +00:48:54.500 --> 00:48:55.000 +it back off. + +00:48:55.280 --> 00:48:55.880 +You know, I don't know that + +00:48:55.880 --> 00:48:57.420 +I'd just ship production code + +00:48:57.420 --> 00:48:57.820 +that way. + +00:48:58.140 --> 00:48:59.760 +But anyway, I got a couple + +00:48:59.760 --> 00:49:00.760 +more questions. + +00:49:00.880 --> 00:49:01.680 +We're getting shorter on + +00:49:01.680 --> 00:49:02.280 +time here. + +00:49:02.620 --> 00:49:03.520 +What was one of the + +00:49:03.520 --> 00:49:05.400 +harder questions that you + +00:49:05.400 --> 00:49:07.240 +all, harder decisions you + +00:49:07.240 --> 00:49:09.420 +all had to address on the + +00:49:09.420 --> 00:49:09.700 +council? + +00:49:09.700 --> 00:49:10.440 +I think the most + +00:49:10.440 --> 00:49:11.620 +contentious one was + +00:49:11.620 --> 00:49:14.680 +PEP 724, if I remember + +00:49:14.680 --> 00:49:15.520 +the number correctly. + +00:49:15.940 --> 00:49:17.460 +It was around a feature + +00:49:17.460 --> 00:49:19.020 +called type guards, which + +00:49:19.020 --> 00:49:20.660 +is around user-defined type + +00:49:20.660 --> 00:49:21.540 +narrowing functions. + +00:49:22.040 --> 00:49:23.100 +Initially, they find that in + +00:49:23.100 --> 00:49:24.740 +a way that later was found + +00:49:24.740 --> 00:49:25.820 +to be somewhat problematic + +00:49:25.820 --> 00:49:27.600 +and we basically came up + +00:49:27.600 --> 00:49:28.420 +with a better set of + +00:49:28.420 --> 00:49:30.000 +proposed semantics that + +00:49:30.000 --> 00:49:30.900 +maybe we should have done + +00:49:30.900 --> 00:49:32.980 +the first time around. + +00:49:33.520 --> 00:49:35.060 +And what this PEP proposed, + +00:49:35.300 --> 00:49:36.080 +and as you can see, I + +00:49:36.080 --> 00:49:37.840 +sponsored it, is that we + +00:49:37.840 --> 00:49:38.660 +basically changed the + +00:49:38.660 --> 00:49:39.420 +meaning of the existing + +00:49:39.420 --> 00:49:40.940 +type guards under certain + +00:49:40.940 --> 00:49:41.500 +conditions. + +00:49:41.800 --> 00:49:42.620 +What is a type guard? + +00:49:42.820 --> 00:49:43.780 +A type guard is a function, + +00:49:43.940 --> 00:49:44.840 +like there's a good example + +00:49:44.840 --> 00:49:45.780 +there, the isiterable. + +00:49:46.220 --> 00:49:47.980 +It's a function that tells + +00:49:47.980 --> 00:49:49.680 +you how to narrow something. + +00:49:50.240 --> 00:49:52.040 +So in this example, there's + +00:49:52.040 --> 00:49:53.320 +an isiterable type guard, + +00:49:53.480 --> 00:49:55.240 +which narrows an object to + +00:49:55.240 --> 00:49:56.280 +an iterable of anything. + +00:49:56.680 --> 00:49:58.400 +And then inside the func + +00:49:58.400 --> 00:49:59.840 +there, you can see if + +00:49:59.840 --> 00:50:01.860 +isiterable file, it knows + +00:50:01.860 --> 00:50:03.740 +that it's an iterable. + +00:50:04.320 --> 00:50:06.200 +And in this case, yeah, I + +00:50:06.200 --> 00:50:07.000 +guess it just narrows + +00:50:07.000 --> 00:50:08.200 +exactly to iterable any. + +00:50:08.600 --> 00:50:09.520 +That's one of the ways that + +00:50:09.520 --> 00:50:10.500 +type guards works. + +00:50:10.760 --> 00:50:10.940 +I see. + +00:50:11.040 --> 00:50:12.320 +And the type that returns + +00:50:12.320 --> 00:50:14.020 +kind of communicates to the + +00:50:14.020 --> 00:50:15.680 +type system, like that this + +00:50:15.680 --> 00:50:18.280 +function ensures that this, + +00:50:18.560 --> 00:50:19.760 +the thing that came in as an + +00:50:19.760 --> 00:50:21.160 +arbitrary object, in fact, + +00:50:21.220 --> 00:50:22.040 +is one of these. + +00:50:22.380 --> 00:50:22.660 +Okay. + +00:50:22.800 --> 00:50:23.180 +Interesting. + +00:50:23.600 --> 00:50:23.740 +Yeah. + +00:50:23.840 --> 00:50:25.200 +So that was a tricky one, + +00:50:25.260 --> 00:50:25.400 +huh? + +00:50:25.740 --> 00:50:27.400 +Any other standout, Rebecca or + +00:50:27.400 --> 00:50:27.620 +Carl? + +00:50:27.620 --> 00:50:28.740 +Well, the current discussion + +00:50:28.740 --> 00:50:29.800 +around what is the meaning + +00:50:29.800 --> 00:50:32.020 +of a float annotation, still + +00:50:32.020 --> 00:50:33.960 +unresolved, contentious topic. + +00:50:34.360 --> 00:50:34.760 +Okay. + +00:50:34.920 --> 00:50:35.180 +Gotcha. + +00:50:35.420 --> 00:50:38.720 +I mean, this on PEP724 is + +00:50:38.720 --> 00:50:41.180 +also what came to my mind + +00:50:41.180 --> 00:50:42.900 +immediately as well, because + +00:50:42.900 --> 00:50:44.280 +this was challenging + +00:50:44.280 --> 00:50:45.560 +discussion because, you + +00:50:45.560 --> 00:50:47.140 +know, like there were very + +00:50:47.140 --> 00:50:48.840 +conflicting considerations at + +00:50:48.840 --> 00:50:49.140 +play. + +00:50:49.300 --> 00:50:50.660 +It's like, what semantics + +00:50:50.660 --> 00:50:52.000 +did we want in the long + +00:50:52.000 --> 00:50:52.420 +term? + +00:50:52.620 --> 00:50:53.580 +And what did we want the + +00:50:53.580 --> 00:50:55.220 +type system to look like, + +00:50:55.280 --> 00:50:56.280 +you know, say like 10 years + +00:50:56.280 --> 00:50:57.620 +from now versus backwards + +00:50:57.620 --> 00:50:59.680 +compatibility and what the + +00:50:59.680 --> 00:51:00.780 +migration story would look + +00:51:00.780 --> 00:51:01.060 +like? + +00:51:01.060 --> 00:51:02.120 +It was quite tricky. + +00:51:02.300 --> 00:51:03.340 +I guess that's something you + +00:51:03.340 --> 00:51:04.880 +will always have to be + +00:51:04.880 --> 00:51:07.160 +cognizant of is like every + +00:51:07.160 --> 00:51:08.400 +change, even if it's an + +00:51:08.400 --> 00:51:10.400 +improvement, has to justify + +00:51:10.400 --> 00:51:12.600 +the fact that now you have + +00:51:12.600 --> 00:51:14.480 +challenges with the version + +00:51:14.480 --> 00:51:16.440 +history over time. + +00:51:16.440 --> 00:51:19.880 +I'm thinking like dict of string + +00:51:19.880 --> 00:51:21.800 +comma int with a capital or + +00:51:21.800 --> 00:51:22.540 +lowercase d. + +00:51:22.920 --> 00:51:24.640 +I've got people, I did a + +00:51:24.640 --> 00:51:25.500 +YouTube video showing + +00:51:25.500 --> 00:51:26.640 +something with the lowercase + +00:51:26.640 --> 00:51:27.840 +version because I was using + +00:51:27.840 --> 00:51:29.280 +something super modern like + +00:51:29.280 --> 00:51:30.260 +Python 3.11. + +00:51:30.680 --> 00:51:32.260 +And I got a message like, + +00:51:32.340 --> 00:51:33.600 +hey, Michael, you don't know + +00:51:33.600 --> 00:51:34.400 +how to write Python. + +00:51:34.520 --> 00:51:35.400 +Your code is broken. + +00:51:35.860 --> 00:51:37.220 +This code that you wrote just + +00:51:37.220 --> 00:51:38.160 +doesn't even run. + +00:51:38.240 --> 00:51:39.140 +I don't know how this is. + +00:51:39.280 --> 00:51:40.180 +I'm like, what version of + +00:51:40.180 --> 00:51:40.740 +Python is in? + +00:51:40.860 --> 00:51:41.340 +3.8. + +00:51:41.580 --> 00:51:41.820 +Nope. + +00:51:41.880 --> 00:51:42.940 +You can't use 3.8 for that. + +00:51:43.000 --> 00:51:43.620 +You're going to need to get a + +00:51:43.620 --> 00:51:44.040 +newer one. + +00:51:44.140 --> 00:51:44.580 +You know what I mean? + +00:51:44.880 --> 00:51:47.240 +But like those are complexities + +00:51:47.240 --> 00:51:48.540 +that get added to Python + +00:51:48.540 --> 00:51:49.820 +because of that. + +00:51:49.900 --> 00:51:51.000 +Now you've got two ways to + +00:51:51.000 --> 00:51:52.680 +specify what a dict is. + +00:51:52.840 --> 00:51:54.000 +There's a preferred new way, + +00:51:54.080 --> 00:51:55.080 +but there's still the old way + +00:51:55.080 --> 00:51:57.440 +and it just, it sort of piles + +00:51:57.440 --> 00:51:57.780 +up. + +00:51:58.000 --> 00:51:58.880 +And it's very hard to ever + +00:51:58.880 --> 00:51:59.840 +actually get rid of the old + +00:51:59.840 --> 00:52:00.940 +way, even if there's no good + +00:52:00.940 --> 00:52:01.820 +reason to use it anymore. + +00:52:01.960 --> 00:52:02.280 +Exactly. + +00:52:02.440 --> 00:52:04.020 +Once it's there, it's written + +00:52:04.020 --> 00:52:05.540 +in ink pretty much, right? + +00:52:05.580 --> 00:52:06.880 +Like we have five or six + +00:52:06.880 --> 00:52:07.680 +different ways to format + +00:52:07.680 --> 00:52:08.120 +strings. + +00:52:08.220 --> 00:52:09.920 +Maybe with t-strings at six + +00:52:09.920 --> 00:52:10.200 +now. + +00:52:10.560 --> 00:52:11.380 +They're all going to still be + +00:52:11.380 --> 00:52:12.160 +there, right? + +00:52:12.160 --> 00:52:13.520 +So every change, every + +00:52:13.520 --> 00:52:14.560 +decision you make is not + +00:52:14.560 --> 00:52:16.420 +just a matter of, is it the + +00:52:16.420 --> 00:52:17.420 +right decision, right? + +00:52:17.660 --> 00:52:19.580 +It's the, is it worth it? + +00:52:19.720 --> 00:52:20.140 +I'm sure. + +00:52:20.620 --> 00:52:20.840 +Yeah. + +00:52:21.200 --> 00:52:21.460 +I don't know. + +00:52:21.460 --> 00:52:22.300 +How do you all balance that? + +00:52:22.400 --> 00:52:23.660 +Like that's tricky. + +00:52:23.820 --> 00:52:24.540 +With things like the dict + +00:52:24.540 --> 00:52:25.860 +chains, at least we sort of + +00:52:25.860 --> 00:52:27.060 +know we're moving towards + +00:52:27.060 --> 00:52:29.720 +better states and there's two + +00:52:29.720 --> 00:52:30.860 +things, but they mean exactly + +00:52:30.860 --> 00:52:31.580 +the same thing. + +00:52:31.720 --> 00:52:33.700 +So the confusion is not as + +00:52:33.700 --> 00:52:34.040 +bad. + +00:52:34.460 --> 00:52:35.600 +The problem with type cards + +00:52:35.600 --> 00:52:36.700 +is that we're going to change + +00:52:36.700 --> 00:52:38.200 +how some existing thing + +00:52:38.200 --> 00:52:39.720 +works, like what it meant. + +00:52:39.720 --> 00:52:41.360 +And I think there are good + +00:52:41.360 --> 00:52:42.260 +reasons that maybe that's the + +00:52:42.260 --> 00:52:43.640 +right thing to do, but the, + +00:52:44.120 --> 00:52:45.340 +it would also have been pretty + +00:52:45.340 --> 00:52:46.400 +confusing for people if their + +00:52:46.400 --> 00:52:47.820 +existing types suddenly started + +00:52:47.820 --> 00:52:48.640 +meaning something completely + +00:52:48.640 --> 00:52:48.980 +different. + +00:52:49.320 --> 00:52:49.680 +Absolutely. + +00:52:50.180 --> 00:52:50.760 +Hence float. + +00:52:51.120 --> 00:52:51.460 +Okay. + +00:52:51.940 --> 00:52:53.000 +What's coming next? + +00:52:53.120 --> 00:52:55.920 +Like 3.15, 3.16, do you all + +00:52:55.920 --> 00:52:57.800 +have things that are in the works + +00:52:57.800 --> 00:52:58.880 +that you think are going to come + +00:52:58.880 --> 00:53:00.980 +or debates that are brewing? + +00:53:01.180 --> 00:53:03.320 +For 3.15, the, there's a type + +00:53:03.320 --> 00:53:04.940 +dict feature coming, extra + +00:53:04.940 --> 00:53:05.400 +items. + +00:53:05.400 --> 00:53:07.040 +you can already use it in + +00:53:07.040 --> 00:53:08.740 +tapping extensions if you want + +00:53:08.740 --> 00:53:10.560 +to use it, but it will be in + +00:53:10.560 --> 00:53:11.940 +CPath as of 2.15. + +00:53:12.100 --> 00:53:13.720 +It's likely we'll have a small + +00:53:13.720 --> 00:53:14.980 +thing I added called disjoint + +00:53:14.980 --> 00:53:16.900 +basis, which is very technical, + +00:53:17.140 --> 00:53:18.620 +but helps type narrowing in some + +00:53:18.620 --> 00:53:19.040 +cases. + +00:53:19.440 --> 00:53:19.540 +Yeah. + +00:53:19.540 --> 00:53:20.240 +I think those are the things + +00:53:20.240 --> 00:53:21.960 +that are likely to make it. + +00:53:22.440 --> 00:53:23.800 +There's, we can only speculate + +00:53:23.800 --> 00:53:24.900 +about what else people can + +00:53:24.900 --> 00:53:25.280 +propose. + +00:53:25.380 --> 00:53:26.380 +We're sort of bound by what + +00:53:26.380 --> 00:53:27.440 +people actually write up as + +00:53:27.440 --> 00:53:27.760 +peps. + +00:53:28.060 --> 00:53:28.860 +We have to wait for Google to + +00:53:28.860 --> 00:53:29.760 +write the peps before we can + +00:53:29.760 --> 00:53:30.200 +approve them. + +00:53:30.200 --> 00:53:32.240 +I think there's PEP 747 for + +00:53:32.240 --> 00:53:34.100 +type form, which I think is + +00:53:34.100 --> 00:53:35.820 +not, I think we recommended + +00:53:35.820 --> 00:53:37.260 +its acceptance, but I don't + +00:53:37.260 --> 00:53:38.140 +think the steering council + +00:53:38.140 --> 00:53:39.340 +accepted it yet or it hasn't + +00:53:39.340 --> 00:53:40.240 +been accepted formally. + +00:53:40.400 --> 00:53:41.040 +I think that's on their + +00:53:41.040 --> 00:53:41.500 +plate. + +00:53:41.660 --> 00:53:41.820 +Yeah. + +00:53:42.180 --> 00:53:42.380 +Yeah. + +00:53:42.400 --> 00:53:43.740 +So that's also pretty likely + +00:53:43.740 --> 00:53:45.180 +to make it into 3.15. + +00:53:45.320 --> 00:53:46.720 +This is one example of a + +00:53:46.720 --> 00:53:47.780 +case that will be pretty + +00:53:47.780 --> 00:53:49.180 +useful to people working + +00:53:49.180 --> 00:53:50.880 +with type annotations at + +00:53:50.880 --> 00:53:52.300 +runtime because it'll allow + +00:53:52.300 --> 00:53:53.940 +you to, it's sort of a meta + +00:53:53.940 --> 00:53:55.960 +thing where you can annotate, + +00:53:56.760 --> 00:53:57.900 +have a type annotation that + +00:53:57.900 --> 00:53:58.960 +describes another type + +00:53:58.960 --> 00:53:59.400 +annotation. + +00:53:59.940 --> 00:54:00.940 +So that's useful if you're, + +00:54:01.000 --> 00:54:02.040 +if you're writing code that + +00:54:02.040 --> 00:54:03.240 +works with type annotations. + +00:54:03.240 --> 00:54:04.620 +Make the peidantics of the + +00:54:04.620 --> 00:54:05.400 +world very happy. + +00:54:05.540 --> 00:54:07.620 +I am actually pretty excited + +00:54:07.620 --> 00:54:09.780 +about type form because, + +00:54:10.060 --> 00:54:10.760 +you know, I feel like there's + +00:54:10.760 --> 00:54:12.100 +a gap and we can express in + +00:54:12.100 --> 00:54:13.460 +the type system and we're + +00:54:13.460 --> 00:54:14.380 +good. + +00:54:14.680 --> 00:54:15.760 +And there are cases in the + +00:54:15.760 --> 00:54:17.780 +existing type system, like + +00:54:17.780 --> 00:54:18.740 +for instance, the cast + +00:54:18.740 --> 00:54:20.360 +function and some other + +00:54:20.360 --> 00:54:22.320 +cases where something takes + +00:54:22.320 --> 00:54:24.120 +any type expression as an + +00:54:24.120 --> 00:54:24.400 +argument. + +00:54:24.520 --> 00:54:25.220 +We actually don't have a good + +00:54:25.220 --> 00:54:27.000 +way to annotate that today + +00:54:27.000 --> 00:54:28.040 +and this will provide a nice + +00:54:28.040 --> 00:54:28.880 +way to express that. + +00:54:28.960 --> 00:54:30.000 +Let me pull up one thing + +00:54:30.000 --> 00:54:30.680 +really quick. + +00:54:31.020 --> 00:54:31.840 +Quick shout out to Will + +00:54:31.840 --> 00:54:32.500 +McGuggan here. + +00:54:32.500 --> 00:54:33.800 +He just released his + +00:54:33.800 --> 00:54:35.760 +Toad project, which is the + +00:54:35.760 --> 00:54:38.400 +new, takes textual and rich + +00:54:38.400 --> 00:54:39.580 +and all that kind of stuff + +00:54:39.580 --> 00:54:40.980 +and applies it to like, what + +00:54:40.980 --> 00:54:42.300 +if we had a better cloud code + +00:54:42.300 --> 00:54:43.460 +type of experience, which is + +00:54:43.460 --> 00:54:43.980 +pretty interesting. + +00:54:44.560 --> 00:54:45.440 +So the reason I'm bringing + +00:54:45.440 --> 00:54:47.200 +this up is, you know, final + +00:54:47.200 --> 00:54:47.620 +question. + +00:54:47.840 --> 00:54:49.560 +What about, do you all even + +00:54:49.560 --> 00:54:52.680 +worry about the role of like + +00:54:52.680 --> 00:54:54.480 +how types interact with AI + +00:54:54.480 --> 00:54:56.120 +and agentic coding tools? + +00:54:56.120 --> 00:54:58.980 +I know that if you have some + +00:54:58.980 --> 00:55:00.660 +code that has types on it + +00:55:00.660 --> 00:55:02.300 +and you give it to an AI, it's + +00:55:02.300 --> 00:55:03.340 +got a better chance of + +00:55:03.340 --> 00:55:04.280 +understanding what's happening + +00:55:04.280 --> 00:55:05.880 +than if you give it purely + +00:55:05.880 --> 00:55:07.560 +untyped code and say, tell me + +00:55:07.560 --> 00:55:08.300 +about this, right? + +00:55:08.340 --> 00:55:09.540 +It doesn't even know necessarily + +00:55:09.540 --> 00:55:10.580 +what's being passed to it. + +00:55:10.720 --> 00:55:12.160 +But is that anything you'll + +00:55:12.160 --> 00:55:13.320 +think about or what are your + +00:55:13.320 --> 00:55:14.500 +thoughts on this? + +00:55:14.760 --> 00:55:15.660 +Certainly think about it some. + +00:55:15.860 --> 00:55:17.460 +I mean, I think overall my + +00:55:17.460 --> 00:55:19.500 +feeling is that these coding + +00:55:19.500 --> 00:55:21.200 +agents seem to do better than + +00:55:21.200 --> 00:55:23.700 +more kind of the tighter + +00:55:23.700 --> 00:55:25.120 +feedback loops you can give them + +00:55:25.120 --> 00:55:25.760 +to work with. + +00:55:26.020 --> 00:55:27.080 +And so typing is another + +00:55:27.080 --> 00:55:28.400 +useful source of feedback + +00:55:28.400 --> 00:55:29.840 +where you can say, add type + +00:55:29.840 --> 00:55:30.880 +annotations and make sure the + +00:55:30.880 --> 00:55:32.920 +type checker passes and seems + +00:55:32.920 --> 00:55:34.600 +so it still seems pretty useful + +00:55:34.600 --> 00:55:35.140 +in that world. + +00:55:35.240 --> 00:55:36.340 +Yeah, you can easily write + +00:55:36.340 --> 00:55:38.120 +rules that say when you are + +00:55:38.120 --> 00:55:39.540 +done on anything I've asked + +00:55:39.540 --> 00:55:42.340 +you to do, always run ty or + +00:55:42.340 --> 00:55:43.760 +always run Pyrefly and make + +00:55:43.760 --> 00:55:45.180 +sure that there's no more, no + +00:55:45.180 --> 00:55:46.720 +new errors or at least or + +00:55:46.720 --> 00:55:48.240 +ideally zero errors, right? + +00:55:48.320 --> 00:55:49.340 +But nothing has been + +00:55:49.340 --> 00:55:49.840 +introduced. + +00:55:50.220 --> 00:55:50.800 +Yeah, pretty interesting. + +00:55:51.360 --> 00:55:52.820 +You other folks, Rebecca, + +00:55:53.260 --> 00:55:53.440 +Yela? + +00:55:53.700 --> 00:55:54.560 +Yeah, I guess in general, I + +00:55:54.560 --> 00:55:55.460 +think typing will remain + +00:55:55.460 --> 00:55:57.040 +useful for AI. + +00:55:57.400 --> 00:55:58.180 +We are probably rapidly + +00:55:58.180 --> 00:55:59.780 +moving to a world where a + +00:55:59.780 --> 00:56:00.960 +large proportion of all code + +00:56:00.960 --> 00:56:01.940 +is written by AI. + +00:56:02.120 --> 00:56:03.020 +Not everybody likes that + +00:56:03.020 --> 00:56:03.620 +opinion, Yael. + +00:56:03.640 --> 00:56:04.440 +Not everybody likes that. + +00:56:04.660 --> 00:56:06.760 +I guess I, maybe my current + +00:56:06.760 --> 00:56:07.780 +line of work makes me think + +00:56:07.780 --> 00:56:08.820 +that's more likely to happen. + +00:56:08.940 --> 00:56:09.900 +You don't have to like the + +00:56:09.900 --> 00:56:10.800 +fact it's going to be night + +00:56:10.800 --> 00:56:11.840 +soon, but it's going to be + +00:56:11.840 --> 00:56:11.980 +night. + +00:56:12.040 --> 00:56:12.380 +You know what I mean? + +00:56:12.420 --> 00:56:14.080 +Like there's the, I just + +00:56:14.080 --> 00:56:14.900 +think there's so much + +00:56:14.900 --> 00:56:16.220 +momentum on this, at least in + +00:56:16.220 --> 00:56:17.460 +the next five years or + +00:56:17.460 --> 00:56:18.260 +something, that it's going to + +00:56:18.260 --> 00:56:19.400 +be really, it's, it's a + +00:56:19.400 --> 00:56:21.320 +truth of how many people are + +00:56:21.320 --> 00:56:22.360 +writing code regardless of + +00:56:22.360 --> 00:56:23.920 +whether individuals want to + +00:56:23.920 --> 00:56:24.640 +write code that way. + +00:56:24.840 --> 00:56:25.180 +You know what I mean? + +00:56:25.200 --> 00:56:25.780 +So I think it's a + +00:56:25.780 --> 00:56:26.200 +consideration. + +00:56:26.380 --> 00:56:26.460 +Yeah. + +00:56:26.460 --> 00:56:26.640 +Yeah. + +00:56:26.700 --> 00:56:27.740 +I forgot that you worked + +00:56:27.740 --> 00:56:28.160 +at OpenAI. + +00:56:28.300 --> 00:56:30.920 +So of course, I should pull + +00:56:30.920 --> 00:56:32.060 +up a codex example or + +00:56:32.060 --> 00:56:32.840 +something, shouldn't I? + +00:56:32.960 --> 00:56:33.100 +Yeah. + +00:56:33.180 --> 00:56:33.820 +Codex is great. + +00:56:33.900 --> 00:56:34.220 +Use it. + +00:56:34.360 --> 00:56:35.160 +No, but I mean, do you + +00:56:35.160 --> 00:56:36.780 +have any further insight into + +00:56:36.780 --> 00:56:38.840 +like the role of types + +00:56:38.840 --> 00:56:39.660 +and coding agents? + +00:56:40.040 --> 00:56:40.900 +I know that's not exactly + +00:56:40.900 --> 00:56:41.700 +what you work on, right? + +00:56:41.740 --> 00:56:42.600 +You're more at the lower. + +00:56:42.740 --> 00:56:43.900 +As Carl said, types can + +00:56:43.900 --> 00:56:45.720 +also be helpful for AI to + +00:56:45.720 --> 00:56:47.060 +understand code better and to + +00:56:47.060 --> 00:56:48.020 +get a better feedback loop. + +00:56:48.320 --> 00:56:49.700 +I feel like the very big AI, + +00:56:49.700 --> 00:56:50.740 +the board is like humans. + +00:56:51.060 --> 00:56:53.140 +And if AI makes, sorry, if + +00:56:53.140 --> 00:56:54.800 +typing makes humans better + +00:56:54.800 --> 00:56:55.940 +at writing understanding + +00:56:55.940 --> 00:56:56.780 +this code, they're probably + +00:56:56.780 --> 00:56:58.560 +also big AI better at it. + +00:56:58.620 --> 00:56:59.540 +It's the locality of + +00:56:59.540 --> 00:56:59.940 +information. + +00:57:00.080 --> 00:57:01.120 +You can read the function + +00:57:01.120 --> 00:57:03.100 +and know everything you need + +00:57:03.100 --> 00:57:03.900 +to know about what's going + +00:57:03.900 --> 00:57:05.660 +into it without bouncing + +00:57:05.660 --> 00:57:06.720 +around and trying to + +00:57:06.720 --> 00:57:07.780 +understand blocks of code + +00:57:07.780 --> 00:57:08.660 +and like what might've been + +00:57:08.660 --> 00:57:09.320 +created that's getting + +00:57:09.320 --> 00:57:09.580 +impacted. + +00:57:09.740 --> 00:57:10.980 +It's good for humans and + +00:57:10.980 --> 00:57:11.980 +also good for AI. + +00:57:12.160 --> 00:57:12.300 +Right. + +00:57:12.600 --> 00:57:12.880 +Rebecca. + +00:57:12.980 --> 00:57:13.700 +Because I don't have much + +00:57:14.240 --> 00:57:15.920 +need to, and I'll say I am + +00:57:15.920 --> 00:57:18.320 +maybe a little more skeptical + +00:57:18.320 --> 00:57:20.680 +than most of my coworkers + +00:57:20.680 --> 00:57:22.040 +about the quality of AI + +00:57:22.040 --> 00:57:23.360 +generated code. + +00:57:23.500 --> 00:57:25.500 +But that means I think I am + +00:57:25.500 --> 00:57:27.700 +particularly gung-ho about, + +00:57:27.800 --> 00:57:29.860 +you know, like get AI to use + +00:57:29.860 --> 00:57:32.200 +types, type checkers, keep + +00:57:32.200 --> 00:57:33.400 +the guardrails there. + +00:57:33.600 --> 00:57:34.300 +I think that'll be very + +00:57:34.300 --> 00:57:34.540 +important. + +00:57:34.540 --> 00:57:35.160 +Yeah, if it's going to make + +00:57:35.160 --> 00:57:36.440 +a mistake, don't let it at + +00:57:36.440 --> 00:57:38.180 +least like make the type + +00:57:38.180 --> 00:57:39.720 +system become disconnected + +00:57:39.720 --> 00:57:40.820 +and not working. + +00:57:40.980 --> 00:57:42.600 +Like it has to keep the types + +00:57:42.600 --> 00:57:43.560 +hanging together as a + +00:57:43.560 --> 00:57:44.360 +minimum bar, right? + +00:57:44.360 --> 00:57:45.760 +And you can easily set that + +00:57:45.760 --> 00:57:46.500 +up as an automation. + +00:57:46.980 --> 00:57:47.140 +Yeah. + +00:57:47.220 --> 00:57:48.220 +Interesting to think of it as + +00:57:48.220 --> 00:57:49.740 +guardrails rather than an + +00:57:49.740 --> 00:57:50.280 +accelerant. + +00:57:50.520 --> 00:57:51.860 +But yeah, 100% it is. + +00:57:52.140 --> 00:57:52.760 +All right, folks. + +00:57:52.900 --> 00:57:54.120 +I think that's it for all + +00:57:54.120 --> 00:57:55.240 +the time that we have. + +00:57:55.640 --> 00:57:55.960 +Thank you. + +00:57:56.000 --> 00:57:56.800 +Thank you for being here. + +00:57:57.240 --> 00:57:59.040 +Final thoughts before we go. + +00:57:59.340 --> 00:58:00.280 +Carl, I'll let you go first. + +00:58:00.600 --> 00:58:01.260 +Final thoughts for people + +00:58:01.260 --> 00:58:02.020 +out there interested in + +00:58:02.020 --> 00:58:02.520 +Python typing. + +00:58:02.740 --> 00:58:02.900 +Yeah. + +00:58:02.960 --> 00:58:03.740 +Well, first of all, thanks + +00:58:03.740 --> 00:58:05.540 +for having us on the podcast. + +00:58:05.640 --> 00:58:06.520 +Really appreciate it. + +00:58:06.860 --> 00:58:07.840 +And thoughts for people + +00:58:07.840 --> 00:58:08.340 +out there. + +00:58:08.680 --> 00:58:10.660 +I guess if you have ideas + +00:58:10.660 --> 00:58:12.400 +of how Python typing could + +00:58:12.400 --> 00:58:14.340 +be improved, discuss.python. + +00:58:14.360 --> 00:58:15.300 +Python.org is a good + +00:58:15.300 --> 00:58:16.760 +place to bring up ideas + +00:58:16.760 --> 00:58:18.180 +and discuss them with the + +00:58:18.180 --> 00:58:19.900 +typing community and see + +00:58:19.900 --> 00:58:21.160 +what positive changes we + +00:58:21.160 --> 00:58:21.420 +can make. + +00:58:22.080 --> 00:58:22.400 +Rebecca. + +00:58:22.580 --> 00:58:24.040 +First, thank you, Michael. + +00:58:24.120 --> 00:58:25.760 +This is a lot of fun. + +00:58:26.760 --> 00:58:27.680 +Last thoughts? + +00:58:28.240 --> 00:58:30.180 +Hey, so, you know, like + +00:58:30.180 --> 00:58:31.260 +we'll look at the typing + +00:58:31.260 --> 00:58:32.500 +council and sometimes think, + +00:58:32.780 --> 00:58:33.480 +oh, you know, like the + +00:58:33.480 --> 00:58:34.280 +PEP has like governance + +00:58:34.280 --> 00:58:35.640 +in its name, but I + +00:58:35.640 --> 00:58:37.200 +wouldn't say we're really + +00:58:37.200 --> 00:58:39.300 +a governing body or + +00:58:39.300 --> 00:58:39.720 +anything. + +00:58:39.720 --> 00:58:42.120 +It's like people who are + +00:58:42.120 --> 00:58:43.300 +using the type system, + +00:58:43.480 --> 00:58:44.780 +like users, they're the + +00:58:44.780 --> 00:58:45.520 +ones who come up with, + +00:58:45.580 --> 00:58:46.140 +you know, like all the + +00:58:46.140 --> 00:58:47.840 +best ideas, propose them, + +00:58:47.940 --> 00:58:48.640 +discuss them. + +00:58:48.640 --> 00:58:50.500 +And we're just here to + +00:58:50.500 --> 00:58:52.940 +sort of be like, hey, + +00:58:53.040 --> 00:58:53.840 +you know, like we have + +00:58:53.840 --> 00:58:55.140 +some background and like + +00:58:55.140 --> 00:58:56.560 +how type checkers work and + +00:58:56.560 --> 00:58:57.420 +maybe some of the history + +00:58:57.420 --> 00:58:58.940 +and we can provide input. + +00:58:58.940 --> 00:59:00.040 +But I just encourage + +00:59:00.040 --> 00:59:00.740 +people, if there's a + +00:59:00.740 --> 00:59:02.120 +change you want to see in + +00:59:02.120 --> 00:59:03.740 +the type system, you know, + +00:59:03.740 --> 00:59:05.400 +like propose it yourself. + +00:59:05.400 --> 00:59:06.980 +It's very friendly and + +00:59:06.980 --> 00:59:07.680 +open community. + +00:59:07.880 --> 00:59:07.940 +Yeah. + +00:59:08.020 --> 00:59:09.080 +Now people who have + +00:59:09.080 --> 00:59:10.560 +listened know a little bit + +00:59:10.560 --> 00:59:11.460 +more about how to do so. + +00:59:11.680 --> 00:59:11.920 +Awesome. + +00:59:12.100 --> 00:59:12.320 +Thanks. + +00:59:12.640 --> 00:59:13.440 +Jale, final word. + +00:59:13.620 --> 00:59:13.760 +Yeah. + +00:59:13.840 --> 00:59:14.920 +Also, again, thank you for + +00:59:14.920 --> 00:59:15.600 +having me here. + +00:59:16.000 --> 00:59:17.040 +It's been great talking to + +00:59:17.040 --> 00:59:17.480 +all of you. + +00:59:17.700 --> 00:59:18.360 +I guess what I want to say + +00:59:18.360 --> 00:59:18.940 +is similar to what + +00:59:18.940 --> 00:59:19.920 +Karin Rebecca just said. + +00:59:20.300 --> 00:59:21.220 +If you want to have + +00:59:21.220 --> 00:59:22.220 +something changed to the + +00:59:22.220 --> 00:59:23.140 +type system, I'd really + +00:59:23.140 --> 00:59:24.540 +encourage you to sign up + +00:59:24.540 --> 00:59:25.680 +for discuss.python.org, + +00:59:25.800 --> 00:59:26.920 +make a proposal, go + +00:59:26.920 --> 00:59:27.680 +through the process. + +00:59:27.920 --> 00:59:28.620 +It can be somewhat + +00:59:28.620 --> 00:59:29.500 +daunting, perhaps, + +00:59:29.620 --> 00:59:30.700 +especially if you have to + +00:59:30.700 --> 00:59:32.160 +create a PEP, but it is + +00:59:32.160 --> 00:59:32.500 +doable. + +00:59:32.960 --> 00:59:34.060 +There are several recent + +00:59:34.060 --> 00:59:34.980 +typing PEPs have just + +00:59:34.980 --> 00:59:36.280 +been community members + +00:59:36.280 --> 00:59:37.880 +who saw something they + +00:59:37.880 --> 00:59:38.580 +wanted to improve, + +00:59:38.720 --> 00:59:40.200 +proposed a PEP, and saw + +00:59:40.200 --> 00:59:40.800 +it to completion. + +00:59:41.180 --> 00:59:41.900 +If there's something you + +00:59:41.900 --> 00:59:42.480 +want to see in the type + +00:59:42.480 --> 00:59:44.240 +system, then you can do + +00:59:44.240 --> 00:59:44.480 +it too. + +00:59:44.580 --> 00:59:46.180 +Thank you all for keeping + +00:59:46.180 --> 00:59:47.120 +Python typing going + +00:59:47.120 --> 00:59:47.600 +strong. + +00:59:48.040 --> 00:59:48.880 +Really appreciate your + +00:59:48.880 --> 00:59:49.560 +time on the show. + +00:59:49.980 --> 00:59:50.540 +See you all later. + +00:59:50.640 --> 00:59:50.940 +Bye. + +00:59:51.080 --> 00:59:51.260 +Bye. + +00:59:51.260 --> 00:59:54.180 +This has been another episode + +00:59:54.180 --> 00:59:55.140 +of Talk Python To Me. + +00:59:55.280 --> 00:59:56.260 +Thank you to our sponsors. + +00:59:56.440 --> 00:59:57.280 +Be sure to check out what + +00:59:57.280 --> 00:59:57.720 +they're offering. + +00:59:57.920 --> 00:59:59.100 +It really helps support the + +00:59:59.100 --> 00:59:59.280 +show. + +00:59:59.980 --> 01:00:01.000 +Take some stress out of + +01:00:01.000 --> 01:00:01.420 +your life. + +01:00:01.760 --> 01:00:02.920 +Get notified immediately + +01:00:02.920 --> 01:00:04.600 +about errors and performance + +01:00:04.600 --> 01:00:06.140 +issues in your web or mobile + +01:00:06.140 --> 01:00:07.200 +applications with Sentry. + +01:00:07.680 --> 01:00:09.680 +Just visit talkpython.fm + +01:00:09.680 --> 01:00:11.420 +slash Sentry and get + +01:00:11.420 --> 01:00:12.180 +started for free. + +01:00:12.660 --> 01:00:13.900 +Be sure to use our code + +01:00:13.900 --> 01:00:15.140 +talkpython26. + +01:00:15.740 --> 01:00:17.360 +That's talkpython, the + +01:00:17.360 --> 01:00:19.220 +numbers two, six, all one + +01:00:19.220 --> 01:00:19.440 +word. + +01:00:19.820 --> 01:00:20.960 +And it's brought to you by + +01:00:20.960 --> 01:00:22.900 +our Agentic AI programming + +01:00:22.900 --> 01:00:24.000 +for Python course. + +01:00:24.420 --> 01:00:25.600 +Learn to work with AI that + +01:00:25.600 --> 01:00:26.920 +actually understands your + +01:00:26.920 --> 01:00:28.500 +code base and build real + +01:00:28.500 --> 01:00:28.980 +features. + +01:00:29.480 --> 01:00:30.940 +Visit talkpython.fm + +01:00:30.940 --> 01:00:33.000 +slash agentic dash AI. + +01:00:33.560 --> 01:00:35.220 +If you or your team needs + +01:00:35.220 --> 01:00:36.240 +to learn Python, we have + +01:00:36.240 --> 01:00:38.220 +over 270 hours of beginner + +01:00:38.220 --> 01:00:39.740 +and advanced courses on + +01:00:39.740 --> 01:00:40.820 +topics ranging from + +01:00:40.820 --> 01:00:42.280 +complete beginners to + +01:00:42.280 --> 01:00:44.020 +async code, Flask, Django, + +01:00:44.220 --> 01:00:46.020 +HTMX, and even LLMs. + +01:00:46.260 --> 01:00:47.340 +Best of all, there's no + +01:00:47.340 --> 01:00:48.680 +subscription in sight. + +01:00:48.680 --> 01:00:49.920 +Browse the catalog at + +01:00:49.920 --> 01:00:50.880 +talkpython.fm. + +01:00:51.520 --> 01:00:52.360 +And if you're not already + +01:00:52.360 --> 01:00:53.820 +subscribed to the show on + +01:00:53.820 --> 01:00:54.660 +your favorite podcast + +01:00:54.660 --> 01:00:55.800 +player, what are you + +01:00:55.800 --> 01:00:56.200 +waiting for? + +01:00:56.660 --> 01:00:57.960 +Just search for Python in + +01:00:57.960 --> 01:00:58.680 +your podcast player. + +01:00:58.780 --> 01:00:59.420 +We should be right at the + +01:00:59.420 --> 01:00:59.660 +top. + +01:00:59.980 --> 01:01:01.080 +If you enjoy that geeky + +01:01:01.080 --> 01:01:01.960 +rap song, you can + +01:01:01.960 --> 01:01:02.960 +download the full track. + +01:01:03.060 --> 01:01:03.860 +The link is actually in + +01:01:03.860 --> 01:01:04.760 +your podcast blur show + +01:01:04.760 --> 01:01:04.980 +notes. + +01:01:05.540 --> 01:01:06.800 +This is your host, Michael + +01:01:06.800 --> 01:01:07.100 +Kennedy. + +01:01:07.300 --> 01:01:08.320 +Thank you so much for + +01:01:08.320 --> 01:01:08.600 +listening. + +01:01:08.780 --> 01:01:09.560 +I really appreciate it. + +01:01:09.960 --> 01:01:10.720 +I'll see you next time. + +01:01:18.680 --> 01:01:19.680 +Voyager. + +01:01:19.920 --> 01:01:21.980 +Voyager. + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:21.980 + + +01:01:21.980 --> 01:01:24.760 +And we ready to roll + +01:01:24.760 --> 01:01:27.540 +Upgrading the code + +01:01:27.540 --> 01:01:29.960 +No fear of getting whole + +01:01:29.960 --> 01:01:33.560 +We tapped into that modern vibe + +01:01:33.560 --> 01:01:34.940 +Overcame each storm + +01:01:34.940 --> 01:01:36.940 +Talk Python To Me + +01:01:36.940 --> 01:01:38.400 +IceSync is the norm diff --git a/youtube_transcripts/536-fly-inside-fastapi-cloud-youtube.vtt b/youtube_transcripts/536-fly-inside-fastapi-cloud-youtube.vtt new file mode 100644 index 0000000..d4580d4 --- /dev/null +++ b/youtube_transcripts/536-fly-inside-fastapi-cloud-youtube.vtt @@ -0,0 +1,3377 @@ +WEBVTT + +00:00:01.659 --> 00:00:06.040 +Hello everyone, Sebastian, Patrick, Savannah, and Jonathan. + +00:00:06.830 --> 00:00:07.860 +Awesome to have you all here. + +00:00:08.680 --> 00:00:10.200 +Excited to talk about FastAPI Cloud. + +00:00:10.920 --> 00:00:11.140 +Welcome. + +00:00:11.260 --> 00:00:11.660 +Yes. + +00:00:12.240 --> 00:00:12.460 +Sorry. + +00:00:16.619 --> 00:00:17.720 +What a project. + +00:00:17.930 --> 00:00:19.540 +It's been going on for a while. + +00:00:20.060 --> 00:00:26.580 +I've heard stuff from Sebastian that maybe something was brewing and all these things, + +00:00:26.820 --> 00:00:29.180 +but not too long ago you all announced it. + +00:00:30.340 --> 00:00:34.780 +And I heard that FastAPI, some people have been using it recently. + +00:00:35.080 --> 00:00:38.580 +You know, some of the surveys show that some people use it for websites. + +00:00:38.760 --> 00:00:39.280 +I'm not sure. + +00:00:42.180 --> 00:00:42.660 +Rumors. + +00:00:43.660 --> 00:00:44.140 +Rumors. + +00:00:44.940 --> 00:00:45.420 +Rumors. + +00:00:45.820 --> 00:00:46.220 +Oh, my gosh. + +00:00:46.420 --> 00:00:48.400 +I mean, congratulations on that. + +00:00:48.420 --> 00:00:53.880 +But before we dive into FastAPI and FastAPI Cloud, let's just do a quick introduction. + +00:00:54.140 --> 00:00:55.260 +Who are you? + +00:00:55.360 --> 00:00:59.560 +We'll just go around the Brady Bunch squares of our live stream here and start with Sebastian. + +00:01:00.860 --> 00:01:01.820 +You've been on the show a few times. + +00:01:01.980 --> 00:01:06.580 +In fact, you've been on the show just recently for a really fun episode, Sebastian. + +00:01:07.230 --> 00:01:07.580 +Who are you? + +00:01:07.910 --> 00:01:08.740 +That was super fun. + +00:01:09.010 --> 00:01:10.200 +So hello, everyone. + +00:01:10.270 --> 00:01:12.380 +I'm Sebastian Ramirez or Tiangulo. + +00:01:12.720 --> 00:01:14.380 +I created FastAPI. + +00:01:14.620 --> 00:01:20.660 +That is this Python framework for building web APIs and backend in case you know. + +00:01:20.940 --> 00:01:24.100 +In case you've been living in a hole and haven't done any Python for 10 years. + +00:01:24.260 --> 00:01:32.640 +You also are famous for really pointing out the ridiculousness of modern tech recruiting. + +00:01:33.960 --> 00:01:35.140 +You know what I'm talking about? + +00:01:35.920 --> 00:01:37.500 +Yeah, you know, like it's fun. + +00:01:37.780 --> 00:01:44.800 +This is probably the thing that I am known for is for writing a tweet saying, yeah, that + +00:01:44.800 --> 00:01:53.659 +I saw a job post asking for five years of experience with API and I only had 2.5 since + +00:01:53.680 --> 00:01:54.520 +I created the thing. + +00:01:55.260 --> 00:01:58.040 +So you didn't qualify for the FastAPI job. + +00:01:58.040 --> 00:01:59.280 +I didn't qualify for it, yeah. + +00:01:59.720 --> 00:02:06.220 +And then the funny thing is, you know, like people sometimes, even people in Python itself + +00:02:06.660 --> 00:02:09.820 +and tell me like, oh, wait, like you're, and I said like, oh, yeah, I created this thing + +00:02:09.940 --> 00:02:10.399 +called FastAPI. + +00:02:10.440 --> 00:02:12.100 +Oh, wait, okay, so what is FastAPI? + +00:02:12.420 --> 00:02:15.540 +Oh, wait, you are the guy from the meme. + +00:02:16.799 --> 00:02:17.800 +Are you serious? + +00:02:18.800 --> 00:02:23.640 +Yeah, you know, like, suddenly that is super important that I am the guy for the meme of + +00:02:23.680 --> 00:02:27.220 +about FastAPI. Not the guy from Far of the Club, the guy from the. + +00:02:28.200 --> 00:02:30.720 +Israel, oh my gosh, I saw you on TikTok. It was amazing. + +00:02:31.640 --> 00:02:34.480 +It was my achievement. I wrote a viral tweet. + +00:02:35.280 --> 00:02:39.160 +You know what? Sometimes your moment in the sun is not the one you expected. No. + +00:02:39.580 --> 00:02:41.620 +Congratulations on how good FastAPI was. + +00:02:41.620 --> 00:02:41.900 +On the tweet. + +00:02:43.000 --> 00:02:46.140 +Exactly. You really nailed it. Patrick, welcome to the show. + +00:02:47.200 --> 00:02:50.180 +Hello. Thank you. It's nice to be here. Yeah, I'm Patrick. + +00:02:50.680 --> 00:02:55.220 +I guess the main thing I'm kind of known for in the community is like this library called + +00:02:55.380 --> 00:02:59.400 +Storary, which is pretty similar to FastAPI, but instead of REST is for GraphQL. + +00:03:00.540 --> 00:03:06.240 +Other than that, I help organize PyCon Italy and I used to also do EuroPython as well, + +00:03:06.360 --> 00:03:08.860 +but I stopped because of way too many things. + +00:03:10.360 --> 00:03:11.980 +Yeah, that's pretty much me. + +00:03:12.200 --> 00:03:12.560 +Okay. + +00:03:13.120 --> 00:03:13.740 +Yeah, that's awesome. + +00:03:15.260 --> 00:03:16.920 +How do you see GraphQL these days? + +00:03:17.600 --> 00:03:18.540 +Is it still popular? + +00:03:20.140 --> 00:03:24.340 +I think it's mostly popular in the enterprises, unfortunately. + +00:03:25.910 --> 00:03:29.700 +To be honest, I'm a bit annoyed about the companies that do tooling around GraphQL because + +00:03:30.879 --> 00:03:32.960 +they're not really pushing it forward. + +00:03:33.180 --> 00:03:36.520 +They're just trying to work with enterprises and that's it. + +00:03:36.760 --> 00:03:38.720 +Or maybe you're putting to AI. + +00:03:41.480 --> 00:03:43.600 +Yeah, it feels a little bit like the soap. + +00:03:44.270 --> 00:03:48.120 +The soap with the XML modern version. + +00:03:49.160 --> 00:03:49.240 +Savannah. + +00:03:50.980 --> 00:03:51.380 +Yeah. + +00:03:52.060 --> 00:03:58.000 +I was just going to say that I think you like tapping out of being an organizer for EuroPython + +00:03:58.240 --> 00:04:03.460 +is like, you know, the classic open source oversubscribed doing all the things. + +00:04:04.000 --> 00:04:04.540 +Very relatable. + +00:04:07.580 --> 00:04:07.780 +Yeah. + +00:04:08.500 --> 00:04:08.600 +Yeah. + +00:04:09.420 --> 00:04:10.100 +But yeah, I'm Savannah. + +00:04:12.060 --> 00:04:13.060 +What can I say? + +00:04:13.960 --> 00:04:18.560 +I am on the Python Steering Council for 2026, which is very exciting. + +00:04:19.120 --> 00:04:26.980 +I am also the release manager for the upcoming version of Python, Python 3.16. + +00:04:28.410 --> 00:04:32.320 +And so that'll kick off later this year, which is really cool and very exciting. + +00:04:33.250 --> 00:04:40.380 +I work on CPython stuff, the JIT, arg parse, basically whatever needs help is kind of where + +00:04:40.380 --> 00:04:40.880 +you'll find me. + +00:04:41.680 --> 00:04:42.280 +Awesome. + +00:04:43.240 --> 00:04:44.420 +Congratulations on the steering council. + +00:04:44.680 --> 00:04:47.900 +And yeah, that's a lot of cool stuff. + +00:04:49.600 --> 00:04:53.980 +Hopefully we don't get a Python 4.0 right after 3.16 + +00:04:54.810 --> 00:04:57.260 +because then your job will never end is what I've learned. + +00:04:57.800 --> 00:04:58.260 +Yeah, yeah. + +00:04:58.760 --> 00:05:02.580 +Benjamin Peterson, Python 2.7 forever kind of situation. + +00:05:03.200 --> 00:05:03.520 +Yeah, yeah. + +00:05:03.860 --> 00:05:07.620 +I mean, release management is still, I mean, it's still quite a commitment. + +00:05:07.860 --> 00:05:11.799 +It's like seven-ish years when you think about all the staggered releases + +00:05:11.820 --> 00:05:12.940 +to your release management too. + +00:05:13.340 --> 00:05:15.520 +And then you have the five year maintenance cycle. + +00:05:15.920 --> 00:05:20.740 +So yeah, it's Python forever is what I can really say. + +00:05:21.440 --> 00:05:22.840 +- Yeah, it's probably not a fad. + +00:05:22.840 --> 00:05:24.520 +It's probably gonna stick around this Python thing. + +00:05:27.380 --> 00:05:28.620 +No, that's awesome. Congratulations. + +00:05:29.020 --> 00:05:31.160 +Also cool with argpars. + +00:05:31.360 --> 00:05:34.100 +I feel like that's making a strong comeback now + +00:05:34.340 --> 00:05:39.299 +that we have these AI things that can just + +00:05:40.460 --> 00:05:42.560 +put stuff together for us instead of like, + +00:05:42.570 --> 00:05:44.400 +oh, I need to depend on this library and that library. + +00:05:44.560 --> 00:05:47.140 +I just need to take a few arguments + +00:05:47.370 --> 00:05:48.380 +and have a little help text. + +00:05:48.450 --> 00:05:50.300 +And it's like, well, you've already got this built-in thing. + +00:05:50.420 --> 00:05:51.880 +Oh, who knew? + +00:05:52.240 --> 00:05:53.480 +People were like, oh, I didn't even know. + +00:05:53.530 --> 00:05:56.380 +I thought I used typer or click or something, right? + +00:05:57.140 --> 00:05:59.480 +- Yeah, I mean, I was gonna say, + +00:06:00.320 --> 00:06:02.440 +there's the typers and clicks of the world, + +00:06:02.640 --> 00:06:04.820 +but sometimes you just want the simplest thing + +00:06:04.980 --> 00:06:06.800 +and ArgParse is pretty great at that, + +00:06:07.080 --> 00:06:09.580 +although it has many quirks that are probably + +00:06:09.980 --> 00:06:12.160 +and most definitely unfixable at this point + +00:06:12.460 --> 00:06:14.980 +because bugs are features when you have things + +00:06:15.140 --> 00:06:16.320 +that have been around as long as Python. + +00:06:18.200 --> 00:06:20.720 +But yeah, no, I mean, AI loves to write Python. + +00:06:22.000 --> 00:06:24.340 +I think it's like the language used the most + +00:06:24.940 --> 00:06:26.240 +in AI generated code, so. + +00:06:26.460 --> 00:06:29.180 +- Yeah, I'll just say we live in weird times. + +00:06:29.480 --> 00:06:30.320 +We live in very weird times. + +00:06:30.820 --> 00:06:33.160 +- I would love a precedented time at some time. + +00:06:33.220 --> 00:06:33.960 +- Exactly, yeah. + +00:06:34.130 --> 00:06:35.460 +Can we just get the boring times? + +00:06:35.860 --> 00:06:37.440 +Nothing interesting, please. + +00:06:38.180 --> 00:06:38.400 +Yeah. + +00:06:38.980 --> 00:06:39.280 +All right. + +00:06:39.950 --> 00:06:45.680 +Oh, also, I did what I said about GraphQL may sound like a bit of a smash, + +00:06:45.860 --> 00:06:49.760 +but I didn't mean it in a negative, super negative way anyway. + +00:06:50.050 --> 00:06:54.240 +Like, it used to be all the enterprises were all about SOAP and Wisdle + +00:06:54.460 --> 00:06:55.760 +and, like, subscribing your tooling. + +00:06:57.240 --> 00:06:57.980 +Please don't write me. + +00:06:58.030 --> 00:06:59.520 +I'm not trying to bash on your technology. + +00:07:01.860 --> 00:07:02.220 +All right. + +00:07:02.980 --> 00:07:04.160 +Jonathan, also, welcome. + +00:07:04.410 --> 00:07:04.520 +Hi. + +00:07:05.380 --> 00:07:05.600 +Hi. + +00:07:06.560 --> 00:07:10.060 +Yeah, I'm not nearly as famous as everyone else in this call. + +00:07:10.940 --> 00:07:17.040 +I'm more infamous internally at FastAPI Cloud, I would say, for a bunch of things. + +00:07:17.580 --> 00:07:20.300 +I've heard of emojis or something along the lines. + +00:07:20.300 --> 00:07:20.660 +One meme away. + +00:07:20.900 --> 00:07:21.980 +You're just one meme away. + +00:07:22.300 --> 00:07:23.380 +Just one meme away. + +00:07:23.680 --> 00:07:24.420 +Yeah, that's true. + +00:07:25.040 --> 00:07:26.500 +We keep piling them up internally. + +00:07:27.840 --> 00:07:32.440 +But yeah, I used to work with Patrick together for years, also on CraftGuard. + +00:07:33.140 --> 00:07:33.720 +Same like we assume. + +00:07:34.260 --> 00:07:35.540 +That's how I know him. + +00:07:35.820 --> 00:07:40.060 +And that's why I, well, made a weird sound when you said so. + +00:07:42.320 --> 00:07:47.780 +Yeah, and I've been with FastAPI Cloud since EuropePython, actually, the last one. + +00:07:48.440 --> 00:07:52.100 +I promised Sebastian I would implement some Ascent events in FastAPI, + +00:07:52.100 --> 00:07:58.180 +and I haven't started with it yet at all, but somehow I'm still here. + +00:07:58.240 --> 00:07:58.760 +So that's great. + +00:08:00.900 --> 00:08:05.300 +Well, actually, and that's actually like a sneak peek, I guess. + +00:08:05.340 --> 00:08:10.160 +we already started like having a bunch of chats and like discussing what we do should we do it + +00:08:10.320 --> 00:08:14.460 +here should we do it there what should we do so like yeah it's something that is coming to fast + +00:08:14.520 --> 00:08:20.480 +api probably soonish like there was a lot of things that needed to happen before that like + +00:08:20.900 --> 00:08:27.300 +the patrick is slightly smiley like oh no this is pressure there were some things that needed to + +00:08:27.380 --> 00:08:31.480 +happen in fast api like you know dropping support for findante version one or things like that that + +00:08:31.500 --> 00:08:33.960 +that just made the code, the internal code, so complex. + +00:08:34.500 --> 00:08:37.740 +And now that it's over, we can actually work more + +00:08:37.940 --> 00:08:39.800 +on improving performance, adding features, + +00:08:39.969 --> 00:08:40.680 +and things like that. + +00:08:40.880 --> 00:08:41.200 +So yeah. + +00:08:43.000 --> 00:08:44.080 +Yeah, very exciting. + +00:08:44.240 --> 00:08:51.060 +I definitely want to dive into how FastAPI Cloud + +00:08:51.320 --> 00:08:56.660 +has sort of influenced the whole FastAPI side of things. + +00:08:57.360 --> 00:09:03.000 +But I'm made aware that there is, in fact, an entire website. + +00:09:05.980 --> 00:09:08.120 +An entire website dedicated to the meme. + +00:09:08.940 --> 00:09:09.480 +All right. + +00:09:10.080 --> 00:09:13.560 +Yeah, and out of the audience we get, hey, everyone, is that the guy from the meme? + +00:09:13.840 --> 00:09:16.580 +And the meme is greater than Nobel Prize. + +00:09:17.080 --> 00:09:19.020 +So, you know what? + +00:09:19.480 --> 00:09:20.280 +It may be true. + +00:09:20.620 --> 00:09:21.160 +It may be true. + +00:09:21.220 --> 00:09:24.660 +I recognize the person saying, this is the guy from the meme. + +00:09:24.840 --> 00:09:25.980 +He might be my husband. + +00:09:31.300 --> 00:09:38.760 +incredible incredible all right well let's start with fast api cloud and then we'll bring it back + +00:09:38.980 --> 00:09:45.540 +around to the fast api let's let's talk org origin story so what is this fast api cloud + +00:09:47.380 --> 00:09:51.899 +nice so uh here we're looking at the fast api labs website that doesn't really show + +00:09:52.100 --> 00:09:56.700 +that much. If you click on the join the waiting list that takes you to the website for FastAPI + +00:09:56.840 --> 00:10:01.780 +Cloud, there we can see like this is what we are building. This is the thing that we are doing. + +00:10:02.620 --> 00:10:07.280 +It's actually super simple. The funny thing is that the pitch, the explanation of the product + +00:10:07.540 --> 00:10:13.920 +is so short. So it's one command. It's FastAPI Deploy. And you have a FastAPI app, you just hit + +00:10:13.980 --> 00:10:18.380 +FastAPI Deploy, and then it's on the cloud. We take care of everything. We build a thing, + +00:10:19.420 --> 00:10:23.180 +deploy it, handle HTTPS, auto-scaling, all this stuff. + +00:10:23.700 --> 00:10:27.460 +And then you can just focus on building the application, building apps. + +00:10:28.420 --> 00:10:30.500 +The funny thing is that it's super short to explain, + +00:10:30.500 --> 00:10:32.560 +but then building it is so complex. + +00:10:35.200 --> 00:10:38.500 +I feel, well, first of all, I'm glad it's so short. + +00:10:38.580 --> 00:10:39.460 +So thanks for being here. + +00:10:39.480 --> 00:10:40.440 +That was a great show, y'all. + +00:10:40.720 --> 00:10:41.540 +Well, I'm just kidding. + +00:10:44.360 --> 00:10:49.240 +No, I think it's a little bit like Jupyter Notebooks. + +00:10:49.320 --> 00:10:56.000 +in that sense that like you all are taking one for the team so that other people can have a simple + +00:10:56.340 --> 00:11:01.580 +experience whereas you know it's like those those jupyter folks they write tons of typescript and + +00:11:01.580 --> 00:11:06.140 +do all sorts of things that nobody wants to necessarily do in the data science space so that + +00:11:06.160 --> 00:11:14.319 +you can just drag your widgets around you know what i mean um exactly yeah yeah yeah thanks i feel + +00:11:14.340 --> 00:11:22.500 +like no i was just i feel like the deployment space it's a bit of a mixed bag and it's my i've + +00:11:22.580 --> 00:11:28.220 +been really frustrated to the point such that i wrote a book about it that i think about an + +00:11:28.380 --> 00:11:36.180 +alternative that i think over the last five plus years it's just trended towards a little more + +00:11:36.480 --> 00:11:41.219 +complex a little more complex oh could we just add one of these things and oh now we got these + +00:11:41.240 --> 00:11:45.160 +through we need one more thing to like make sure those things are doing you know what i mean and + +00:11:45.220 --> 00:11:52.080 +it's just like wow why are there 200 choices in my console to use this which is like kind of funny + +00:11:52.260 --> 00:11:57.000 +right because i feel like a lot of these companies started with this like i don't want to understand + +00:11:57.260 --> 00:12:01.380 +all the ins and outs of all the infrastructure that comes with the cloud service provider and + +00:12:01.480 --> 00:12:04.759 +that's really complicated to understand because i'm an app dev and i don't know anything about + +00:12:05.020 --> 00:12:11.060 +never, right? And now we're like, I don't know, kind of slowly accumulating complexity. But I + +00:12:11.070 --> 00:12:15.620 +think one of the cool things about what we're building, and I like I've worked on cloud tooling + +00:12:15.790 --> 00:12:20.540 +before is like, this is like, just spoke for Python developers. And I think that's like, + +00:12:20.880 --> 00:12:26.540 +quite, like unique in that, like, we are really trying to like, bring the bleeding edge and like + +00:12:26.620 --> 00:12:34.740 +all the new tooling that people are using and making sure that we play well with like uv. And + +00:12:34.740 --> 00:12:37.480 +care put into that by the team. + +00:12:37.660 --> 00:12:39.240 +Yeah. That's a super good point. + +00:12:39.300 --> 00:12:43.400 +I mean, I remember Azure came out with like, + +00:12:43.480 --> 00:12:44.640 +here's your platform as a service. + +00:12:44.860 --> 00:12:47.440 +You just upload your web app and we'll just take it and go. + +00:12:47.660 --> 00:12:50.960 +And now that thing is so complicated along with many, many others. + +00:12:51.180 --> 00:12:51.940 +It's not just them. + +00:12:52.040 --> 00:12:55.120 +It's you've got AWS, you've got Vercel. + +00:12:55.220 --> 00:12:57.440 +There's lots of things we could point at for, + +00:12:58.540 --> 00:12:59.380 +there's a lot of options here. + +00:13:00.820 --> 00:13:02.620 +And then there are a lot of tools and like, + +00:13:02.800 --> 00:13:07.500 +You know, like many tools and many companies are also doing a great job at many of the things that they are doing. + +00:13:08.100 --> 00:13:09.840 +But in many cases, it's just so complex. + +00:13:10.000 --> 00:13:10.700 +It's so complicated. + +00:13:10.920 --> 00:13:19.340 +You know, like I was I have always been so adamant, I think is the word, to just teaching people how to use the tools. + +00:13:19.920 --> 00:13:25.980 +I think I have the most documentation about how to deploy things on your own than any other framework. + +00:13:26.160 --> 00:13:27.440 +I have so much information. + +00:13:27.960 --> 00:13:29.520 +I hear that all the time from people. + +00:13:29.800 --> 00:13:32.060 +they say one of the reasons they chose FastAPI + +00:13:32.080 --> 00:13:34.360 +is because how clear the documentation was, you know? + +00:13:35.319 --> 00:13:37.840 +- Yeah, and then the thing is, you know, + +00:13:37.960 --> 00:13:39.780 +like just learning all those concepts + +00:13:40.000 --> 00:13:42.520 +and like learning all this stuff that needs to be learned + +00:13:42.820 --> 00:13:43.700 +just to deploy something + +00:13:43.820 --> 00:13:45.860 +and then just you barely have like the minimum. + +00:13:46.500 --> 00:13:48.660 +It's like, this is, you know, like it's just too much. + +00:13:48.780 --> 00:13:49.800 +It's too much complexity. + +00:13:50.480 --> 00:13:54.560 +I think for me, like I guess like personally, + +00:13:54.880 --> 00:13:58.719 +my analogy is that FastAPI Cloud is the equivalent + +00:13:58.740 --> 00:14:02.600 +of what FastAPI is to building web APIs and backend. + +00:14:03.000 --> 00:14:04.340 +You know, like you could do the same + +00:14:04.600 --> 00:14:05.340 +with any other framework. + +00:14:05.400 --> 00:14:07.920 +You could validate data, you could generate open API, + +00:14:08.080 --> 00:14:09.240 +you could have automatic docs, + +00:14:09.840 --> 00:14:12.560 +but you will probably have to do a lot of the wiring yourself + +00:14:13.300 --> 00:14:14.640 +and making sure that it's actually correct + +00:14:14.880 --> 00:14:16.640 +and that it doesn't explode, all the stuff. + +00:14:17.360 --> 00:14:22.540 +That is, you know, like we are trying to do a lot of that work + +00:14:23.019 --> 00:14:24.280 +for the final users. + +00:14:26.339 --> 00:14:27.640 +- Yeah, and I think it's great. + +00:14:28.280 --> 00:14:33.220 +I think it's really nice to just provide this on-ramp + +00:14:33.420 --> 00:14:36.220 +because as you said at the opening, + +00:14:36.360 --> 00:14:38.120 +when I asked the origin story, + +00:14:38.280 --> 00:14:42.660 +it's just FastAPI deploy, right? + +00:14:43.300 --> 00:14:45.000 +That solves so many stories. + +00:14:45.120 --> 00:14:46.360 +And I'm sure behind the scenes, + +00:14:46.460 --> 00:14:49.740 +what happens is just about as simple as that. + +00:14:51.520 --> 00:14:52.180 +Oh, my gosh. + +00:14:52.840 --> 00:14:53.520 +About that. + +00:14:56.800 --> 00:15:00.400 +Some of us don't even get to write Python anymore to make all of this happen. + +00:15:01.920 --> 00:15:03.560 +I'm taking one for the team. + +00:15:05.540 --> 00:15:07.100 +Yeah, that is taking one for our team, right? + +00:15:07.920 --> 00:15:08.440 +It is. + +00:15:09.240 --> 00:15:09.320 +Yeah. + +00:15:11.560 --> 00:15:14.000 +Let's save the internals for a little bit later. + +00:15:14.360 --> 00:15:19.000 +Maybe what we could do right now, maybe we could do a bit of a walkthrough + +00:15:19.940 --> 00:15:26.660 +of just kind of what it's like to set up an app from scratch, right? + +00:15:27.700 --> 00:15:28.020 +Nice. + +00:15:28.120 --> 00:15:31.280 +I see that uv is here, which is, + +00:15:32.080 --> 00:15:39.000 +I've been certainly an advocate for uv in all sorts of deployment, + +00:15:39.020 --> 00:15:42.860 +but especially when you have like repeated build type of scenarios + +00:15:43.100 --> 00:15:46.900 +for like Docker, Docker Compose or Kubernetes or whatever. + +00:15:47.400 --> 00:15:50.620 +UV makes that stuff so much faster and so on. + +00:15:50.740 --> 00:15:54.460 +So who would like to be my guide that just kind of talks us through what it + +00:15:54.580 --> 00:15:56.100 +means to set up a new project here? + +00:15:58.320 --> 00:15:58.680 +Patrick. + +00:15:59.480 --> 00:15:59.620 +Yeah. + +00:16:00.180 --> 00:16:01.460 +I feel like it should be Patrick. + +00:16:02.440 --> 00:16:04.520 +That was great to suggest, Lavana, but I can. + +00:16:05.620 --> 00:16:06.400 +I've talked enough. + +00:16:06.560 --> 00:16:07.060 +I feel like. + +00:16:07.760 --> 00:16:08.880 +Patrick has the best mic. + +00:16:10.460 --> 00:16:10.800 +Oh, yeah. + +00:16:11.200 --> 00:16:11.580 +That's the reason. + +00:16:11.740 --> 00:16:12.180 +That could be. + +00:16:13.420 --> 00:16:13.580 +Yeah. + +00:16:14.380 --> 00:16:15.600 +I mean, do you want to go from scratch? + +00:16:15.760 --> 00:16:19.680 +I mean, there is this really nice command that Savannah built, + +00:16:19.920 --> 00:16:23.200 +which is FastAPI-new, which I think is something, I don't know, + +00:16:25.010 --> 00:16:25.740 +super helpful. + +00:16:27.740 --> 00:16:30.020 +Yeah, so what does FastAPI-new do? + +00:16:31.000 --> 00:16:35.020 +Is that kind of a cookie-cutter-esque experience, or what is it? + +00:16:35.800 --> 00:16:36.360 +Yes, exactly. + +00:16:36.580 --> 00:16:41.320 +At the moment, Onesco holds a super basic FastAPI application using uv. + +00:16:41.660 --> 00:16:44.400 +It also installs dependencies, creates a folder, everything that you need. + +00:16:46.220 --> 00:16:49.040 +In future, I think we're going to plan support for templates + +00:16:49.300 --> 00:16:51.880 +so you can build multiple kind of things as well. + +00:16:52.530 --> 00:16:55.180 +But for now, it's basically just uv FastAPI new, + +00:16:55.560 --> 00:16:56.980 +sorry, uvx FastAPI new, + +00:16:57.420 --> 00:16:59.040 +and then that scaffolds the project for you. + +00:17:00.560 --> 00:17:03.360 +I don't know if you want to try it live or... + +00:17:04.040 --> 00:17:04.800 +No, go ahead. + +00:17:05.380 --> 00:17:06.260 +Let's just... + +00:17:06.579 --> 00:17:08.240 +I think it might have disrupted you. + +00:17:08.240 --> 00:17:09.280 +Just let's talk us through it. + +00:17:12.140 --> 00:17:13.180 +It could work. + +00:17:13.240 --> 00:17:15.000 +I'm just going to put that out there. + +00:17:15.180 --> 00:17:15.760 +I'm sure. + +00:17:16.439 --> 00:17:16.819 +I'm sure. + +00:17:17.160 --> 00:17:18.699 +I've had a few times recently. + +00:17:19.939 --> 00:17:21.079 +I started to do... + +00:17:22.160 --> 00:17:23.620 +I'll tell you the most insane, + +00:17:23.959 --> 00:17:25.939 +like let's do that live on the podcast experience. + +00:17:26.140 --> 00:17:26.860 +I'm pretty sure, + +00:17:27.600 --> 00:17:29.160 +yeah, this is definitely the most insane. + +00:17:29.740 --> 00:17:31.760 +I had Matthew Rocklin on from Coiled + +00:17:32.320 --> 00:17:34.100 +and those guys are all about like, + +00:17:34.100 --> 00:17:35.080 +hey, we're going to scale up + +00:17:35.140 --> 00:17:37.660 +like a bunch of available servers for you, right? + +00:17:38.160 --> 00:17:39.440 +So that you can do your data science. + +00:17:39.500 --> 00:17:41.140 +Like I want to do some ML thing + +00:17:41.140 --> 00:17:42.520 +and it needs 500 servers. + +00:17:43.160 --> 00:17:47.020 +So during the podcast, he said, oh, let me just spin up 2,000 EC2 instances. + +00:17:47.300 --> 00:17:47.500 +Hold on. + +00:17:48.480 --> 00:17:50.520 +And then we ran some code on it during the show. + +00:17:50.600 --> 00:17:51.720 +And he's like, oh, let's try that on ARM. + +00:17:51.800 --> 00:17:54.560 +And then spin up another 2,000 on ARM Linux machines. + +00:17:54.700 --> 00:17:55.700 +I'm like, okay, that's nuts. + +00:17:57.180 --> 00:17:57.940 +But let's just... + +00:17:58.320 --> 00:17:58.980 +That's a lot of time. + +00:18:00.820 --> 00:18:01.280 +I was impressed. + +00:18:01.620 --> 00:18:03.860 +But Patrick, sorry. + +00:18:04.200 --> 00:18:04.960 +I had a debrief there. + +00:18:05.540 --> 00:18:06.020 +Let's talk through it. + +00:18:07.300 --> 00:18:10.160 +Yeah, so you do uvx FastAPI new. + +00:18:10.600 --> 00:18:12.580 +Then you specify the name of the application. + +00:18:13.160 --> 00:18:14.480 +and that's almost there. + +00:18:14.480 --> 00:18:16.580 +You just need one more command to deploy, + +00:18:16.940 --> 00:18:17.700 +which is FastAPI deploy. + +00:18:18.780 --> 00:18:20.220 +The first time it's going to ask you to log in + +00:18:20.660 --> 00:18:22.900 +or join the waiting list if you haven't been invited yet. + +00:18:23.320 --> 00:18:24.320 +So you're still in beta. + +00:18:25.980 --> 00:18:27.400 +And then you follow the steps. + +00:18:27.690 --> 00:18:29.760 +So like FastAPI deploy, log in, + +00:18:31.000 --> 00:18:33.580 +decide the team, if you have multiple teams, + +00:18:34.740 --> 00:18:35.860 +decide the application name, + +00:18:36.780 --> 00:18:38.200 +and then you wait a few seconds + +00:18:38.460 --> 00:18:39.580 +and the application is going to be live. + +00:18:40.740 --> 00:18:42.100 +And just to be clear, + +00:18:42.220 --> 00:18:46.320 +FastAPI new is not required if you already have a FastAPI app. + +00:18:46.570 --> 00:18:49.640 +Like if you've already written your own code and you have your application, + +00:18:50.280 --> 00:18:53.020 +you can just go right into like logging in and deploying. + +00:18:53.540 --> 00:18:56.160 +This is just so that if you're starting something new, + +00:18:56.600 --> 00:18:59.980 +you don't have to do any thinking about all the right things that need to be there. + +00:19:00.150 --> 00:19:02.060 +So this is more of a green field application. + +00:19:02.500 --> 00:19:04.140 +I'm bootstrapping a project. + +00:19:04.640 --> 00:19:07.800 +Right, right, because you want to have the best structure. + +00:19:08.620 --> 00:19:10.120 +Now it uses uv. + +00:19:11.340 --> 00:19:13.420 +So do I have to use-- + +00:19:13.420 --> 00:19:14.020 +You can use required. + +00:19:14.720 --> 00:19:17.220 +Yeah, I was going to say, do I have to use the uv project + +00:19:17.540 --> 00:19:18.420 +management type of thing? + +00:19:18.700 --> 00:19:23.540 +Do I have to use the uv.lock files and uv add uv sync? + +00:19:23.800 --> 00:19:24.920 +Can I do requirements.txt? + +00:19:25.060 --> 00:19:25.820 +What's the story there? + +00:19:26.559 --> 00:19:29.740 +Yeah, so we support uv with uv lock. + +00:19:29.880 --> 00:19:31.300 +We also support the-- + +00:19:31.380 --> 00:19:33.300 +forget the name-- the PyLock file. + +00:19:33.700 --> 00:19:36.140 +And we also support plain requirements.txt. + +00:19:36.740 --> 00:19:38.980 +And maybe something else I don't know Jonathan can use. + +00:19:39.060 --> 00:19:40.020 +PyLock's pretty new, right? + +00:19:40.200 --> 00:19:42.820 +I think Brett Cannon just got that out pretty recently, right? + +00:19:43.400 --> 00:19:43.620 +- Yeah. + +00:19:44.730 --> 00:19:46.220 +- Brett was pretty excited, I think. + +00:19:46.420 --> 00:19:46.720 +- I know. + +00:19:47.080 --> 00:19:48.460 +- He implemented that. + +00:19:48.780 --> 00:19:49.180 +- Oh, was he? + +00:19:49.370 --> 00:19:50.600 +Okay, I'm sure he was, that's awesome. + +00:19:50.780 --> 00:19:52.300 +He put years of work into that. + +00:19:53.020 --> 00:19:53.500 +- Sure did. + +00:19:53.550 --> 00:19:53.640 +- Yeah. + +00:19:55.100 --> 00:19:57.100 +I can also say that one of the motivations + +00:19:57.400 --> 00:19:59.660 +was also like, you know, like cloud providers. + +00:20:00.100 --> 00:20:02.440 +So it's like, yes, we did it. + +00:20:03.240 --> 00:20:03.520 +- Okay. + +00:20:03.580 --> 00:20:04.760 +- The other thing is like, you know, + +00:20:04.900 --> 00:20:07.000 +if you use other different package managers, + +00:20:08.000 --> 00:20:10.980 +if they use the standard PI project, the Tomo format, + +00:20:11.360 --> 00:20:12.240 +that will also be supported. + +00:20:13.760 --> 00:20:16.400 +That means that, you know, like if you use PDM + +00:20:16.520 --> 00:20:19.660 +or if you use poetry with one of the recent versions, + +00:20:20.080 --> 00:20:20.820 +like that will work. + +00:20:21.120 --> 00:20:23.040 +If you use a very old version of poetry + +00:20:23.360 --> 00:20:25.660 +or like you use some other strange package manager + +00:20:25.720 --> 00:20:27.660 +or something that will probably be problematic. + +00:20:28.220 --> 00:20:30.820 +But for like most of the use cases + +00:20:30.960 --> 00:20:33.840 +that use the standard package formats, it will just work. + +00:20:34.640 --> 00:20:35.460 +And if you use uv, + +00:20:35.820 --> 00:20:43.820 +like you're gonna have the best experience because we are fans of uv and astra yeah they've definitely + +00:20:44.240 --> 00:20:49.800 +put a dent in the way that sort of python gets started and making that a lot easier so it totally + +00:20:49.940 --> 00:20:56.880 +makes sense and also i noticed speaking of uv that there's at least in the recommended way + +00:20:57.780 --> 00:21:04.160 +or over the way in the docs let's say it doesn't say here's how you install fast api you just + +00:21:04.880 --> 00:21:13.140 +here's how you run fast api dash new leveraging uv which then will you know silently install + +00:21:13.660 --> 00:21:19.320 +and manage all right that's pretty neat that that helps you guys tell a simpler story + +00:21:20.500 --> 00:21:25.220 +right yeah here's how you create the virtual environment to install our thing and so exactly + +00:21:25.980 --> 00:21:30.759 +yeah the idea is to make it like as someone i was saying just super simple for people just to + +00:21:30.820 --> 00:21:36.080 +start from scratch, like no idea how to create an app, how to start, how to create an environment. + +00:21:36.200 --> 00:21:41.020 +It's just you run this command and you're off to go. After the races, I'm missing. + +00:21:42.520 --> 00:21:49.780 +Anyway, that's what Colombians do. But then if you already have an app, + +00:21:50.260 --> 00:21:55.620 +you have anything with FastAPI standard installed, then that also just works. + +00:21:56.700 --> 00:22:03.100 +Yeah. Okay. And Savannah, you pointed out that it doesn't have to be a new project. If you want to + +00:22:03.130 --> 00:22:09.240 +start from an existing one, that's totally fine. But what do I got to do if I'm starting from, + +00:22:09.620 --> 00:22:14.060 +if I'm migrating an existing one? Like how easy or hard is this? + +00:22:15.000 --> 00:22:21.799 +Yeah. I mean, honestly, like I have some like legacy project demo apps I've built at other + +00:22:21.820 --> 00:22:28.180 +companies I've worked with that have used FastAPI and I literally just ran like FastAPI login and + +00:22:28.260 --> 00:22:34.600 +then FastAPI deploy and it just worked which felt really magical right like I think that's like I + +00:22:34.600 --> 00:22:39.040 +don't know like having worked on cloud products for quite a while like I think one of the biggest + +00:22:39.320 --> 00:22:44.160 +gaps is like the just I don't know like the disparity between like my local dev environment + +00:22:44.380 --> 00:22:50.519 +and what is actually like lives up in the cloud somewhere and so being able to just run one command + +00:22:50.540 --> 00:22:55.440 +and then having the project as it exists on my machine go and work somewhere without having to + +00:22:55.440 --> 00:23:00.320 +think about like the infrastructure. And of course, like, you know, we want to be like amenable to + +00:23:00.480 --> 00:23:05.940 +folks who do want a little bit, you know, like higher touch. But we also want to work for people + +00:23:05.950 --> 00:23:11.760 +who are like learning FastAPI and Python, right? Like educators and people that are teaching Python. + +00:23:11.970 --> 00:23:16.540 +I think this is like something that we've had some interest in as well from most folks. So, + +00:23:18.260 --> 00:23:37.760 +Yeah, I was just listening to the Teaching Python podcast folks just the other day and thinking, you know, like this, when I look at this, I know this is not necessarily your focus, but certainly people who are trying to teach a class, be it college class or high school class or whatever. + +00:23:39.100 --> 00:23:44.440 +And if you build anything on the web, the next question is, this is cool. + +00:23:44.660 --> 00:23:45.780 +How do I share it with people? + +00:23:46.100 --> 00:23:49.460 +And then like, oh, no, no, no, hold on. + +00:23:49.920 --> 00:23:53.400 +Are you like in like coding boot camps, right? + +00:23:53.540 --> 00:23:58.820 +Like if you're teaching someone how to write Python or how to write, build an API with + +00:23:58.960 --> 00:24:04.020 +FastAPI, like actually setting up the environment for them to deploy is not part of it. + +00:24:04.640 --> 00:24:04.720 +Right. + +00:24:04.960 --> 00:24:06.900 +Like that's not actually part of the curriculum. + +00:24:07.060 --> 00:24:11.240 +It's like this peripheral thing that ends up eating up a bunch of the educators time or + +00:24:11.240 --> 00:24:15.000 +the students time trying to understand both like how to write code and then also understand + +00:24:15.040 --> 00:24:15.620 +cloud stuff. + +00:24:15.920 --> 00:24:18.700 +And that's like a lot to ask people when they're fresh out the gate. + +00:24:19.500 --> 00:24:23.560 +Yeah. I mean, I feel the same way about like tutorials and stuff at conferences. + +00:24:24.360 --> 00:24:27.540 +Yeah. Or, yeah. Or training sessions. + +00:24:27.950 --> 00:24:31.000 +If you're doing like corporate training or like, they're all like, Oh, well, + +00:24:31.180 --> 00:24:35.720 +let's get everybody's machine work. There goes an hour, whatever. + +00:24:36.040 --> 00:24:39.040 +But yeah, if you can just say, look, I think when you're, + +00:24:40.320 --> 00:24:41.900 +either when you're trying to learn something, + +00:24:42.360 --> 00:24:48.900 +be it through school or on your own or through these like more structured ways like boot camps and + +00:24:49.740 --> 00:24:56.360 +training and so on i think if it's not the main purpose i feel so often there's like we're going + +00:24:56.360 --> 00:25:03.080 +to do 20 steps for four hours before you get any sort of reward of what you've done and if you can + +00:25:03.140 --> 00:25:07.180 +go okay do you have it running okay now you run this command look now it's on the internet like + +00:25:07.210 --> 00:25:11.699 +oh wait awesome i got an app on the internet everybody look at me you know what i mean and + +00:25:11.820 --> 00:25:16.920 +I think shortening that cycle to where people can have that aha moment. + +00:25:17.070 --> 00:25:19.820 +And then later they can dive into like, well, how is it really working? + +00:25:20.160 --> 00:25:21.380 +And what do we really need to understand? + +00:25:21.980 --> 00:25:28.900 +But that quick iteration cycle, especially in the early parts of learning new tech, it's really important. + +00:25:30.060 --> 00:25:35.480 +But also, you know, like down the line as well, I think, like, I don't know, there are so many things that I have been wanting to build. + +00:25:36.140 --> 00:25:41.540 +And I don't, but I didn't because it was just so complex to deploy stuff. + +00:25:41.630 --> 00:25:44.000 +You know, like knowing, knowing how to do the whole thing, + +00:25:44.220 --> 00:25:48.440 +how to set up the clusters, the machines, install the Linux systems, + +00:25:49.100 --> 00:25:51.160 +deploy the cluster, whatever, like all the stuff, + +00:25:51.680 --> 00:25:54.420 +deploy the things, handling load balancers and HTTPS. + +00:25:54.420 --> 00:25:57.120 +And like, you know, like I know how to do that. + +00:25:57.280 --> 00:26:01.800 +I built one of the most popular websites teaching how to use Docker Swarm, + +00:26:02.000 --> 00:26:04.940 +which was like the contender before Kubernetes won everything. + +00:26:06.680 --> 00:26:09.460 +But still, it's just so complicated doing all those steps. + +00:26:09.580 --> 00:26:12.660 +I'll just not do it. + +00:26:15.100 --> 00:26:18.500 +Now I can just play around and do random stuff and just deploy, + +00:26:18.660 --> 00:26:19.400 +and it just works. + +00:26:20.620 --> 00:26:21.260 +I really like that. + +00:26:21.940 --> 00:26:26.140 +I guess coming back to that taking one for the team point + +00:26:26.300 --> 00:26:28.520 +earlier, I feel like building Python tooling + +00:26:28.820 --> 00:26:30.560 +is kind of like taking one for the team sometimes, + +00:26:31.420 --> 00:26:34.760 +because you have these folks that are brand new to Python. + +00:26:35.440 --> 00:26:38.700 +Python is an extremely approachable language for people who are new to writing code. + +00:26:38.780 --> 00:26:44.540 +But then, you know, we also want to make FastAPI cloud work for someone that's building like an enterprise grade application. + +00:26:44.900 --> 00:26:45.040 +Right. + +00:26:45.540 --> 00:26:57.220 +And so like pretty wide spectrum of folks with like a million different use cases and different types of applications they want to deploy with different constraints and like security stuff. + +00:26:57.620 --> 00:27:01.240 +And like, so, yeah, I think, I don't know, maybe that's just like Python tooling. + +00:27:02.240 --> 00:27:05.900 +It's a lot of work, I guess, to build something that works for the masses. + +00:27:06.920 --> 00:27:10.260 +Yeah, well, it's certainly tough to make something that feels simple, + +00:27:11.000 --> 00:27:14.780 +but it's not overly simplistic, you know, that can actually solve the problems. + +00:27:15.140 --> 00:27:17.860 +Has the right knobs for the right users too, right? + +00:27:18.920 --> 00:27:22.900 +I would argue we're not only trying to do it simple and easy. + +00:27:23.180 --> 00:27:27.620 +I feel like we're choosing a particular flavor of simple, which is... + +00:27:28.220 --> 00:27:30.080 +We have this discussion a few times. + +00:27:30.340 --> 00:27:36.320 +like if you make a cloud how do we make it feel pythonic like what does that mean in a cloud + +00:27:36.540 --> 00:27:41.620 +setting like we talk about pythonic libraries by them a coding style in the community a lot + +00:27:42.160 --> 00:27:46.700 +and now we kind of try to transfer that like that flavor that feeling to the cloud and make + +00:27:47.100 --> 00:27:52.120 +everything around that feel just like we want our libraries to feel so you feel at home as a + +00:27:52.220 --> 00:27:57.599 +pipeline developer and it just feels right so that's an extra step on top of making it simple + +00:27:57.640 --> 00:28:00.760 +and we discuss that a lot. At least that's how I feel about it. + +00:28:02.360 --> 00:28:04.280 +Yeah. I love it. + +00:28:05.200 --> 00:28:07.540 +I think it's one of the... Sorry, go ahead. + +00:28:07.780 --> 00:28:08.080 +No, go ahead. + +00:28:08.580 --> 00:28:13.380 +I was going to say that I think it's one of the coolest things about this thing. + +00:28:15.500 --> 00:28:22.600 +People are being able to hear a few of us. There are a bunch of others. But each one of us is so + +00:28:22.580 --> 00:28:28.480 +passionate about the things that we are working on. So like, you know, like, each one of us is + +00:28:28.580 --> 00:28:33.340 +trying to make the best out of the things that we are building. And then like, we are so passionate + +00:28:33.500 --> 00:28:39.340 +about the thing that we care about, and we are building that I think that that ends up in an + +00:28:39.400 --> 00:28:47.080 +amazing result. For example, the CLI, we wanted to have some specific, you know, like, behavior, + +00:28:47.300 --> 00:28:53.020 +some look and feel and like we wanted to be able to have like the best kind of the clis so patrick + +00:28:53.160 --> 00:28:58.280 +went ahead and built this way bunch of tooling that we needed to be able to have it and like + +00:28:58.400 --> 00:29:04.280 +maybe open source and everything so we could have this great experience when working with clis uh + +00:29:04.880 --> 00:29:10.700 +like uh jonathan recently recently was doing so much stuff about the handling the caches and + +00:29:10.760 --> 00:29:16.640 +handling security making sure that everything was super secure super fast super snappy uh you know + +00:29:16.500 --> 00:29:22.920 +like a Alejandra is super careful about all the UI. Martin is super careful about all the infra. + +00:29:23.280 --> 00:29:28.880 +You know, it's like this, this, this passionate nest, which is a word I just made up. + +00:29:31.700 --> 00:29:36.600 +This, Alejandra goes and says like, this thing doesn't have the proper margins, we need to + +00:29:36.760 --> 00:29:41.100 +increase this a little bit. I didn't, I don't like it. She just goes and fix it. The same with Martin. + +00:29:41.140 --> 00:29:44.880 +He says like, we need to have like this, this other thing in infrastructure. And it's like, + +00:29:45.320 --> 00:29:51.640 +just comes and tells me hey we are doing this like yes sir you know it's like this with the + +00:29:51.740 --> 00:29:56.800 +work of the team like uh yuri for example that is mainly focused on the open sources constantly + +00:29:57.140 --> 00:30:03.080 +looking at all the discussions prs conversations making sure that everything that we do uh that + +00:30:03.180 --> 00:30:09.039 +that's also why you know like there have been like recently way more releases of fast api and friends + +00:30:09.100 --> 00:30:15.700 +of the open source projects and very fast book fixes very fast responses to handle everything + +00:30:15.880 --> 00:30:20.620 +for the community and now we actually like have people that is paying attention constantly to + +00:30:20.680 --> 00:30:25.040 +what is happening what what is what are the things that we have to do and that really care about + +00:30:25.700 --> 00:30:31.760 +a that part as well so i think this extreme care about what what we are you know like savannah is + +00:30:31.780 --> 00:30:42.320 +making python and hello i don't know i think this detail that each one of us cares so so much about + +00:30:42.540 --> 00:30:47.240 +each one of the things that we build hey that that helps a lot making sure that the product is + +00:30:47.400 --> 00:30:54.620 +actually amazing it's as good as it can be and we can all feel at home when doing it i don't know + +00:30:54.920 --> 00:31:01.740 +i get i get so excited because i really enjoy the end result of the product and being able to use + +00:31:01.760 --> 00:31:04.920 +and how it works in the end, how it needs to work with it, + +00:31:05.280 --> 00:31:06.200 +it's super simple. + +00:31:07.940 --> 00:31:08.600 +Yeah, that's awesome. + +00:31:08.760 --> 00:31:09.980 +Here, let me adjust your mic real quick. + +00:31:10.380 --> 00:31:13.800 +I think it was like ducking, ducking out a little bit. + +00:31:13.800 --> 00:31:17.220 +We just went through a lot of content and a lot of sweating + +00:31:17.560 --> 00:31:21.660 +because your microphone went through like six different stages. + +00:31:24.100 --> 00:31:24.940 +Yeah, that's, I think, + +00:31:27.440 --> 00:31:36.300 +So I think that really leads to something I wanted to talk about is just what impact has this had on FastAPI? + +00:31:36.300 --> 00:31:53.820 +And before you jump in and answer that question, everyone, there's, especially I think with Astral, because they've had so much success, there's been an undercurrent of concern of like, oh my gosh, commercialism is getting into our open source. + +00:31:54.240 --> 00:31:57.740 +What if it pollutes it and causes these negative aspects? + +00:31:59.299 --> 00:32:03.640 +But just hearing all of the energy around FastAPI + +00:32:03.670 --> 00:32:07.220 +with so many people because of FastAPI Cloud, + +00:32:07.550 --> 00:32:09.120 +I think that's super neat. + +00:32:09.400 --> 00:32:10.720 +So I wanted to throw out to you all, + +00:32:12.720 --> 00:32:17.180 +how is this building FastAPI Cloud and the existence of FastAPI + +00:32:17.380 --> 00:32:21.140 +Cloud been giving back to FastAPI, I guess? + +00:32:28.860 --> 00:32:31.380 +I'm always the one that's speaking the most. + +00:32:33.040 --> 00:32:34.680 +I mean, it might be your project. + +00:32:35.060 --> 00:32:36.440 +You might have started the project. + +00:32:37.679 --> 00:32:38.780 +Yeah, maybe so. + +00:32:39.660 --> 00:32:47.100 +No, but you know, like last year, I had a few keynotes in some PyCons in different places. + +00:32:47.320 --> 00:32:57.940 +And like one of the key points that I wanted to bring was this idea that I'm trying to show that in many cases, people worry about the boss factor. + +00:32:58.340 --> 00:33:00.100 +Yes, yes, I've heard this. Yes. + +00:33:01.060 --> 00:33:07.000 +Yeah, you know, like the boss factor is the idea that, oh, what happens if like there's one person doing this work? + +00:33:07.060 --> 00:33:09.960 +What happens if a boss runs over this person? + +00:33:10.480 --> 00:33:14.200 +And there's so much worry about this boss factor. + +00:33:14.860 --> 00:33:18.620 +It's sort of a morbid analogy, but I understand, right? + +00:33:18.700 --> 00:33:24.000 +It's like, what will happen to the open source project if the maintainer vanishes for some reason, right? + +00:33:24.240 --> 00:33:24.560 +Exactly. + +00:33:25.380 --> 00:33:28.400 +But, you know, like it also applies to products and to many other different things. + +00:33:29.320 --> 00:33:36.480 +But what I think is that is a disproportionate amount of attention to this detail of the boss factor. + +00:33:37.320 --> 00:33:44.760 +And I think every time people talk about the boss factor, you know, like one of my points in what I was trying to say + +00:33:44.780 --> 00:33:51.320 +these talks was I would like people to think about the bus ticket factor who is paying for those + +00:33:51.540 --> 00:33:57.340 +tickets it doesn't matter how big is the team you know like you have seen google amazon meta all the + +00:33:57.400 --> 00:34:03.420 +big ones they don't have a small boost factor they have a lot of people in their payroll and still + +00:34:03.900 --> 00:34:11.899 +they they finish products they just cancel them open source or private or whatever it's not + +00:34:11.919 --> 00:34:18.340 +that the main factor defining the success of a project being it commercial or open source of + +00:34:18.419 --> 00:34:26.060 +any type is not really how many people are behind it it's more of what is the value that whoever + +00:34:26.060 --> 00:34:31.399 +is putting the effort to keep it alive is getting from putting all that effort it could be just + +00:34:31.639 --> 00:34:36.139 +satisfaction you know it could be like open source like i feel i feel so good that i'm contributing + +00:34:36.159 --> 00:34:41.600 +into society and that is valid. It doesn't pay the rent but it is still valid. It might + +00:34:41.690 --> 00:34:46.260 +last for a while. But then also like when you see there are so many Python projects, + +00:34:46.340 --> 00:34:52.540 +so many open source projects that can do well or can do bad and it doesn't really depend + +00:34:52.700 --> 00:34:57.740 +on how many people they have. And when you are using a project, when you are using an + +00:34:57.820 --> 00:35:03.240 +open source project or when you are using a product of any type, I will encourage you + +00:35:03.260 --> 00:35:08.780 +to think about what is the bus ticket factor of this project what are the + +00:35:08.900 --> 00:35:14.540 +things that whoever is building this is receiving in exchange for giving it away + +00:35:15.460 --> 00:35:20.440 +so like you know like what are they expecting to sell you at some point or + +00:35:20.560 --> 00:35:26.380 +what are they receiving in exchange you know for example Bunn the JavaScript + +00:35:26.560 --> 00:35:29.820 +Brompton library like it was like we don't know what they are going to sell + +00:35:29.840 --> 00:35:36.840 +but now you know cloud and entropic really want to have like this thing keep working because they + +00:35:36.860 --> 00:35:41.520 +are using it internally so you can say like okay i'm gonna use it i'm gonna use it for free i know + +00:35:41.640 --> 00:35:47.360 +that what they receive for me using is just like that they just really want it so i can just like + +00:35:47.480 --> 00:35:52.060 +whenever you are using bond you are getting now you are getting free services from entropic that's + +00:35:52.140 --> 00:35:58.460 +it but you know like every time you are using a project you can think about why are people receiving + +00:35:58.480 --> 00:36:03.740 +in this change for giving this away for me. I think this is... sorry, go ahead, Sonia. + +00:36:04.000 --> 00:36:05.060 +No, no, no, you finish it. + +00:36:06.460 --> 00:36:11.560 +No, no, that I think I... this is like the thing that I would like people to think about, you know, + +00:36:11.640 --> 00:36:17.560 +like, also like how can they give back? Maybe they can also actually contribute to that community or + +00:36:17.660 --> 00:36:22.400 +to the project. There are many ways and in many cases, the thing that is needed the most is just + +00:36:22.400 --> 00:36:24.940 +like help and work just answering questions and issues. + +00:36:26.020 --> 00:36:26.400 +So yeah. + +00:36:27.420 --> 00:36:30.000 +Yeah, I was just going to say that, like kind of related to what you're saying, + +00:36:30.080 --> 00:36:34.520 +I think one of the angles that I really appreciate about the way we think about + +00:36:35.000 --> 00:36:39.380 +FastAPI and FastAPI Cloud is like we're like a lot of our team was involved in open + +00:36:39.520 --> 00:36:44.960 +source before coming to work at FastAPI Cloud on various projects around the Python ecosystem + +00:36:45.160 --> 00:36:45.680 +outside of Python. + +00:36:46.440 --> 00:36:51.640 +And I think like all of us have a deep appreciation and understanding of the value + +00:36:51.660 --> 00:36:58.460 +of open source and really, really try and build in a way that is like, I mean, Sebastian, + +00:36:58.560 --> 00:37:03.100 +you've talked about this a lot, but like solving a real problem for folks, right? And so like + +00:37:03.260 --> 00:37:07.700 +FastAPI cloud is sort of this like extension of this open source ecosystem people would + +00:37:07.700 --> 00:37:13.200 +be using. We, you know, FastAPI cloud may be an option, maybe someone picks some other + +00:37:13.320 --> 00:37:21.620 +cloud for some reason. I don't think like, I think we're all very mindful of that. But + +00:37:21.940 --> 00:37:26.560 +all work at FastAPI Cloud, like I know that I personally have time, more time for my open source + +00:37:26.760 --> 00:37:32.700 +work and my employer understands the value of my open source work, which is net positive for the + +00:37:32.740 --> 00:37:38.700 +open source community. Like I get to work on CPython sometimes and I have, you know, the bandwidth to + +00:37:38.720 --> 00:37:44.480 +go and do my steering council work or upcoming release management work. So I think like, I + +00:37:44.660 --> 00:37:50.360 +understand like this sort of like, I don't know, like tempering, like open source commercial bad, + +00:37:50.860 --> 00:37:54.440 +all bad. It's not all bad. It's actually like really good in a lot of cases for + +00:37:55.040 --> 00:37:59.900 +look at uv for an example of the holdup, right? Yeah. Astral. Yeah, yeah, totally. Yeah. Yeah. + +00:38:00.620 --> 00:38:06.220 +I think there are some really good examples of this. So I think like, that's another angle that + +00:38:06.220 --> 00:38:10.680 +I mean, I really, I get a lot of energy out of our team because we all, I don't have to, + +00:38:10.960 --> 00:38:15.660 +I don't have to fight the open source battle at FastAPI and Cloud. I think that's really cool. + +00:38:17.160 --> 00:38:18.660 +I do think that's super cool as well. + +00:38:18.660 --> 00:38:23.100 +Let me put out two examples for you here that I think everyone will be aware of. + +00:38:24.580 --> 00:38:27.920 +As sort of to add to what Sebastian was saying is, + +00:38:28.960 --> 00:38:33.600 +look how much Apple freaked out when Steve Jobs died and how many people work at Apple. + +00:38:34.280 --> 00:38:34.360 +Right. + +00:38:34.940 --> 00:38:37.360 +Like that was still like, oh, my gosh. + +00:38:37.960 --> 00:38:40.820 +But, you know, I think there's they're hanging in there. + +00:38:40.940 --> 00:38:42.080 +They're going to be probably making it. + +00:38:44.540 --> 00:38:44.820 +And then. + +00:38:44.940 --> 00:38:46.260 +They are not enough people. + +00:38:47.200 --> 00:38:48.800 +I tell you what, they got some of my money. + +00:38:49.340 --> 00:38:49.800 +That's for sure. + +00:38:51.920 --> 00:38:54.460 +But also, you know, look at Flask, right? + +00:38:54.660 --> 00:38:56.740 +Armin Roeneker worked on it for a long time, + +00:38:57.580 --> 00:38:59.360 +drifted away, which is totally fine. + +00:39:00.040 --> 00:39:03.860 +And David Lord and Pallets picked it up and kept running, right? + +00:39:04.040 --> 00:39:09.140 +Like, it's still one of the most popular frameworks out there, right? + +00:39:09.340 --> 00:39:10.880 +So it's... + +00:39:12.000 --> 00:39:21.620 +I think the bus factor is over, overblown a bit, but also looking at the team of folks here, I think it's, it's even more obvious that there's a bunch of people on the inside. + +00:39:22.280 --> 00:39:37.060 +But, but you know, like, for example, for example, Flask, you know, like I learned so many things from Flask and like, the thing is, I feel like sometimes, sometimes people go and complain about the tool and say like, oh, this is not working for this or for that. + +00:39:37.280 --> 00:39:42.260 +and in many cases is in this insensitive way towards the people that are working on that. + +00:39:42.900 --> 00:39:47.320 +And it's like, you know, like in the end, realize that there's actually people behind the scenes doing the work. + +00:39:47.600 --> 00:39:51.480 +And like, in many cases, it's just like one or two people doing a lot of work. + +00:39:51.480 --> 00:39:52.860 +In many cases, it's just for free. + +00:39:53.200 --> 00:39:57.720 +And, you know, like I think it's worth calling that out. + +00:39:57.740 --> 00:40:01.600 +Like all the work that David Lord does for Flask, it's just like so much work. + +00:40:01.860 --> 00:40:05.780 +And yeah, I think deserves a lot of respect. + +00:40:06.820 --> 00:40:06.980 +Yeah. + +00:40:07.060 --> 00:40:09.400 +Yeah, for sure. I totally agree. + +00:40:11.000 --> 00:40:19.600 +Sorry, the other thing that I forgot to mention is that there were so many ideas of potential products that I could build over the years, + +00:40:19.680 --> 00:40:27.800 +and I never did, and I never started a company, because I didn't have clarity of what will be a good thing to actually sell and will have a good alignment. + +00:40:28.300 --> 00:40:33.620 +the cloud product has such a good alignment with the open source side because + +00:40:34.400 --> 00:40:34.960 +the + +00:40:36.020 --> 00:40:37.820 +you know like as + +00:40:38.460 --> 00:40:42.120 +More successful FastAPI is the more successful + +00:40:42.700 --> 00:40:47.860 +FastAPI cloud has a potential to be the more people using Python's + +00:40:48.500 --> 00:40:54.060 +Effectively the more people might end up checking out FastAPI and the more people might end up checking out the product + +00:40:54.360 --> 00:40:59.300 +So if FastAPI does well, if the open source does well, if Python does well, that's better for the company. + +00:40:59.390 --> 00:41:05.420 +So, you know, like, it doesn't really depend on my personal principles and values or something like that. + +00:41:05.760 --> 00:41:10.100 +It's aligned with, it's financially aligned with the company. + +00:41:10.370 --> 00:41:16.740 +So, you know, like, it's just going to be beneficial in the end because it doesn't depend on good intentions. + +00:41:17.640 --> 00:41:19.020 +And FastAPI is open source. + +00:41:19.190 --> 00:41:21.120 +It has, like, 7,000 forks or something. + +00:41:21.200 --> 00:41:23.380 +So if I was running, so right. + +00:41:24.240 --> 00:41:26.300 +There are 7,000 points is not going away. + +00:41:27.360 --> 00:41:27.760 +Yeah. + +00:41:27.760 --> 00:41:27.820 +Yeah. + +00:41:28.860 --> 00:41:29.260 +Yeah. + +00:41:29.560 --> 00:41:31.640 +I, I definitely agree with you on that. + +00:41:31.680 --> 00:41:36.160 +I think, I feel like I should maybe give a little bit of a, + +00:41:36.820 --> 00:41:41.060 +I tell a little bit of the story of what's going on with, where did I put it? + +00:41:42.560 --> 00:41:46.020 +I don't think I paste it over here is what's going on with tail end right now. + +00:41:46.500 --> 00:41:50.380 +And I think tail end is having a tough time to one CSS. + +00:41:51.839 --> 00:42:03.540 +traffic to tailwind is up six times year over year on npm downloads but the revenue of tailwind + +00:42:04.010 --> 00:42:11.960 +is down five times you know i mean these are completely out of whack things because instead + +00:42:11.960 --> 00:42:16.700 +of people going to docs to learn about it it's just like well when you go to the docs you learn + +00:42:16.640 --> 00:42:20.480 +they also have premium offerings right and I think you guys are different + +00:42:20.780 --> 00:42:26.060 +because it's not just oh here's a little bit nicer of a thing right I feel like + +00:42:26.060 --> 00:42:29.160 +it would be a little bit as if you were selling cookie cutter templates for + +00:42:29.300 --> 00:42:34.960 +FastAPI apps you know it's like well the AI can make the shape of the thing + +00:42:35.060 --> 00:42:38.440 +that comes out of the cookie cutter to be honest you know I mean but but you're + +00:42:38.580 --> 00:42:45.300 +offering something that has ongoing value that it costs more and is more + +00:42:45.320 --> 00:42:57.640 +complex in other places. And so I think maybe just thinking about the how this just keeps the team + +00:42:57.900 --> 00:43:02.740 +going for FastAPI is really awesome. And I think it's got a nice flywheel effect there. + +00:43:03.920 --> 00:43:10.100 +I'll link to this, I guess, audio track. I don't know what I call it. I think blog post that has + +00:43:10.100 --> 00:43:15.180 +one sentence, but a 30 minute audio you can check out from the guy, Adam, who's one of + +00:43:15.180 --> 00:43:17.700 +the founders of Tailwind talking about going into this. + +00:43:18.540 --> 00:43:19.400 +It's kind of rough. + +00:43:20.960 --> 00:43:24.520 +I think I don't necessarily want to go into a deep AI, what it means for the industry. + +00:43:24.660 --> 00:43:26.660 +Like, let's stay focused on what you guys are doing. + +00:43:26.660 --> 00:43:30.540 +But I think it's going to be, it's going to be, I think it's going to be its own series. + +00:43:30.700 --> 00:43:37.060 +I mean, Stack Overflow had as many questions asked this month as they did in the first + +00:43:37.260 --> 00:43:38.880 +month of their existence. + +00:43:39.820 --> 00:43:39.940 +Right. + +00:43:40.040 --> 00:43:44.620 +three or four thousand, whereas at their peak, they were 200,000 questions a month. + +00:43:45.080 --> 00:43:50.620 +There's like real turmoil that's coming from some of these things, which is tricky. + +00:43:51.460 --> 00:43:57.400 +But yeah, I'm really excited to see you all doing this because I'm a big fan of FastAPI. + +00:43:57.790 --> 00:44:04.800 +And I think this is just sustaining and more for FastAPI, right? + +00:44:04.830 --> 00:44:05.480 +Like, what do you all think? + +00:44:07.440 --> 00:44:08.980 +I'm thinking about it. + +00:44:09.680 --> 00:44:10.440 +That's what we hope. + +00:44:14.860 --> 00:44:17.420 +I thought about Taiwan for a second, right? + +00:44:17.580 --> 00:44:19.780 +It's not like we're immune to what happened to them. + +00:44:19.920 --> 00:44:22.000 +Like, we also have a lot of documentation online. + +00:44:22.400 --> 00:44:23.560 +AI could train on that. + +00:44:23.760 --> 00:44:26.800 +And if it's good enough, it could maintain your infrastructure and stuff. + +00:44:26.940 --> 00:44:28.480 +It's just too hard at the moment. + +00:44:28.600 --> 00:44:33.660 +And there's an additional thing we're kind of selling, which is like, I guess, responsibility. + +00:44:33.820 --> 00:44:42.020 +like and you're shifting the risk from like letting your ai or your infantry the team um maintain your + +00:44:42.240 --> 00:44:48.540 +infrastructure to us so we're staying up at night and worry about it that's that that has a lot of + +00:44:48.620 --> 00:44:54.920 +value as well and that's probably not going to get removed yeah and i also think here here here's a + +00:44:54.940 --> 00:45:04.240 +very common cloud code, cursor, whatever conversation. Hey, build me something with Python and + +00:45:04.300 --> 00:45:09.700 +needs an API. Okay, we built it with FastAPI. How do I host it? Right? That doesn't just, + +00:45:10.220 --> 00:45:15.840 +it will build a cloud for you, right? It's going to recommend something out there. And a real + +00:45:16.040 --> 00:45:22.140 +natural way to how do I host FastAPI is FastAPI cloud, right? Like, if it suggests, oh, + +00:45:22.220 --> 00:45:26.140 +you're just going to like spread it across lamb but lambda by breaking like whoa no I want something + +00:45:26.280 --> 00:45:30.360 +simple okay give me FastAPI cloud right I think that that's a really good thing and then on the + +00:45:31.000 --> 00:45:40.200 +the enterprise side you know enterprise folks are notoriously not good at supporting open source and + +00:45:40.240 --> 00:45:47.260 +that they're not like paying for it I know some companies are big supporters of the PSF and Python + +00:45:47.280 --> 00:45:52.620 +and open source but in general it's like yeah we have this project with 5 000 people working on it's + +00:45:52.660 --> 00:45:59.840 +all python and how are we sponsoring this nope just we're just enjoying the money right and we're + +00:45:59.840 --> 00:46:05.960 +a bank so we got the money we got all the money um so they they're just not good at paying for like + +00:46:05.960 --> 00:46:11.820 +a really great framework that they use a lot but they got plenty of hosting plenty of internal apps + +00:46:11.860 --> 00:46:16.460 +that they just need to make run and stuff so i think both on like the low end and the high end + +00:46:17.200 --> 00:46:24.480 +there's a lot of synergy between these things that is not just, you know, slightly advanced, not to + +00:46:26.420 --> 00:46:31.860 +diminish it, but slightly advanced UI widgets that you could ask your AI to build or, + +00:46:32.180 --> 00:46:34.740 +or something or like cookie cutter templates for project starters. + +00:46:36.060 --> 00:46:43.940 +Yeah. Yeah, yeah. That's true. Like, I think we are in a somewhat fortunate position of like, + +00:46:43.960 --> 00:46:49.260 +you know like fast api has grown so much in like you know like when you check the statistics about + +00:46:49.660 --> 00:46:55.840 +downloads or github stars or entries in developer surveys like it's at the top in like in each + +00:46:55.960 --> 00:47:00.760 +category it's like you know like the backend framework with the most github stars across + +00:47:00.920 --> 00:47:07.140 +languages even like you know like java go ruby js like whatever it's like the top one at least + +00:47:07.200 --> 00:47:14.060 +in github stars so like you know like fast api at least is like people are liking it fortunately + +00:47:14.500 --> 00:47:19.480 +and there's probably gonna be people deploying things to fast api cloud so that's probably gonna + +00:47:19.620 --> 00:47:27.020 +be like we are probably gonna be fine i think did you know like the i guess it will be like a good + +00:47:27.060 --> 00:47:32.560 +point to to ask people to go and check where the open source projects that they are using and check + +00:47:32.440 --> 00:47:37.300 +what is the boss ticket factor for those open source projects? You know, like if you are using + +00:47:37.740 --> 00:47:43.100 +Tailwind CSS, it would have been very cool if at some point you check if the premium things were + +00:47:43.180 --> 00:47:46.640 +useful for you and for your company or your project or something like that, you know? + +00:47:49.039 --> 00:47:52.100 +Because what is the thing that keeps that project going? + +00:47:53.980 --> 00:48:02.400 +Right, exactly. And I really personally admire if a project or something + +00:48:02.620 --> 00:48:06.200 +offers like more value, not just, hey, buy me a coffee, + +00:48:07.000 --> 00:48:09.920 +but here's a thing that you get way more of, you know, + +00:48:10.000 --> 00:48:11.980 +and in that regard, I think Tailwind was doing that, right? + +00:48:11.980 --> 00:48:14.480 +They were offering this suite of pre-built things, + +00:48:14.810 --> 00:48:16.160 +and I think that that's great. + +00:48:17.320 --> 00:48:20.880 +But, yeah, I do think you've got more of a, + +00:48:22.380 --> 00:48:24.400 +these crazy AI things are going to maybe recommend + +00:48:24.779 --> 00:48:27.280 +FastAPI Cloud more than they're just going to undercut it. + +00:48:27.290 --> 00:48:28.140 +So I think that's really great. + +00:48:28.460 --> 00:48:32.380 +And by the way, I was just looking for the GitHub Stars graph + +00:48:32.400 --> 00:48:36.540 +Like there's a whole, I can't remember what the domain of that site is, but, and I + +00:48:36.680 --> 00:48:38.500 +ran across by the, I just want to give a quick shout out. + +00:48:38.650 --> 00:48:43.380 +Like your cult repo documentary on FastAPI was awesome. + +00:48:44.500 --> 00:48:44.860 +Right. + +00:48:45.000 --> 00:48:45.680 +That was so fun. + +00:48:46.400 --> 00:48:47.140 +I didn't see that coming. + +00:48:47.920 --> 00:48:48.060 +Yeah. + +00:48:48.270 --> 00:48:52.460 +It came right on the heels of the Python official documentary, the one hour one. + +00:48:52.470 --> 00:48:55.600 +This is the same group and it, the production quality was really nice. + +00:48:55.760 --> 00:48:57.520 +So like, oh, yeah, it was super fun. + +00:48:58.040 --> 00:49:01.880 +When they released the trailer for the Python documentary, + +00:49:02.160 --> 00:49:04.220 +before releasing the documentary, when they released the trailer, + +00:49:04.460 --> 00:49:08.360 +they contacted me and said, "Hey, we're doing these mini documentaries + +00:49:08.780 --> 00:49:10.220 +about different frameworks, different tools, + +00:49:10.300 --> 00:49:11.920 +and we want to include FastAPI." + +00:49:11.920 --> 00:49:13.120 +They were like, "Oh, nice." + +00:49:14.320 --> 00:49:17.260 +I was just trying to stay silent, but super excited. + +00:49:17.840 --> 00:49:19.840 +I'm sure. That's so cool. + +00:49:20.060 --> 00:49:21.520 +Yeah, I watched it as soon as it came out. + +00:49:22.320 --> 00:49:23.100 +I'll link to that. + +00:49:23.220 --> 00:49:25.280 +People should definitely-- it's only like 10 minutes or something, + +00:49:25.480 --> 00:49:30.540 +but it's worth it. We're checking out. So it's not a huge investment time. People can watch it, + +00:49:30.540 --> 00:49:35.520 +I suppose. It's not tick tock. I mean, it's not like, Oh, I saw the documentary. + +00:49:40.080 --> 00:49:42.940 +It doesn't take you on huge about it. Yeah, people. + +00:49:44.540 --> 00:49:49.840 +It is such a weird time. You guys, the I don't understand what's happened to the + +00:49:50.200 --> 00:49:55.440 +attention span of society. I'm really honestly a little concerned. I used when I create my + +00:49:55.480 --> 00:49:59.260 +is people would say, you know, like a four hour course, and there'd be like a 10, 15 minute sort + +00:49:59.260 --> 00:50:02.300 +of, hey, here's how you set up your computer. And here's all the introduction. And people, oh, + +00:50:02.440 --> 00:50:05.840 +that's so awesome. I loved how you kind of set the stage. I'm really motivated to take the course. + +00:50:06.420 --> 00:50:10.360 +Nowadays, I just get messages like, why are you still talking? This is five minutes long. Do you + +00:50:10.560 --> 00:50:17.080 +understand? I'm like, this is your job. You can't spend five minutes. Oh my gosh. Anyway, that's + +00:50:17.180 --> 00:50:21.700 +that's sort of the origin of my comment there. It's all right. So we're kind of getting somewhat + +00:50:21.720 --> 00:50:30.420 +short on time so i think i think i want to talk about a couple of things let's let's talk a little + +00:50:30.540 --> 00:50:36.020 +bit about internals like what i don't know who wants to take this one but let's talk about just + +00:50:36.240 --> 00:50:41.800 +how when i say fast api deploy then what + +00:50:46.040 --> 00:50:52.000 +It's just a UB pip install and it just goes and it's magic and it's easy, right? + +00:50:52.550 --> 00:50:55.200 +We haven't had anything before, Jonathan, can we save or no? + +00:50:55.480 --> 00:50:58.960 +Yeah, I was going to say, I was going to say, Kronathan, you should take this one. + +00:51:00.760 --> 00:51:02.820 +Oh my God, it's so funny, this happened. + +00:51:03.260 --> 00:51:06.180 +Because I told my friends, I'm so concerned about being at the podcast + +00:51:06.490 --> 00:51:10.060 +because everyone here is a visionary and then I'm the backend guy. + +00:51:10.780 --> 00:51:14.020 +And I think the things I could contribute to this conversation, + +00:51:14.090 --> 00:51:15.220 +I should probably keep to myself. + +00:51:16.480 --> 00:51:24.600 +but it's just leaking internals right um there are some things that are like not really a secret + +00:51:25.000 --> 00:51:33.300 +like as Sebastian said earlier like kubernetes one and in the infrastructure and deployment field + +00:51:33.540 --> 00:51:40.000 +to some extent so that's somewhere in there right uh but it's it's all the way deep down so no one + +00:51:40.000 --> 00:51:45.400 +has to worry about it but it's still a foundation which is a good foundation um I think one thing + +00:51:45.420 --> 00:51:50.860 +that's you you might have guessed it but fast api cloud is built on fast api which kind of makes + +00:51:50.980 --> 00:51:57.760 +sense right um and that also has an effect on like recent patches updates and stuff because if we find + +00:51:57.860 --> 00:52:03.920 +something internally uh which we're not happy with then we just fix it and that's how some releases + +00:52:04.250 --> 00:52:11.960 +uh came out faster than once before dog fooding yeah that's awesome putting a lot yeah also all + +00:52:11.980 --> 00:52:16.420 +all the related libraries like SQL model and others, + +00:52:17.400 --> 00:52:18.500 +they experience the same thing. + +00:52:18.560 --> 00:52:19.820 +- We're actually using everything now. + +00:52:20.580 --> 00:52:21.780 +- Oh yeah, okay, that's cool. + +00:52:22.360 --> 00:52:24.100 +- New library is coming out. + +00:52:25.440 --> 00:52:26.240 +- That's also a thing. + +00:52:26.680 --> 00:52:29.560 +Yeah, it's not just fast.jpi and friends. + +00:52:30.460 --> 00:52:31.360 +We're like really open. + +00:52:32.360 --> 00:52:34.940 +Recently, Patrick just open-sourced everything we use + +00:52:35.040 --> 00:52:37.380 +for authentication authorization, for example. + +00:52:37.720 --> 00:52:38.500 +Is it open-source yet? + +00:52:39.100 --> 00:52:40.000 +Did they just fix something? + +00:52:40.960 --> 00:52:42.380 +No, it is. Yeah, but no, no. + +00:52:44.079 --> 00:52:44.820 +It will be. + +00:52:45.120 --> 00:52:45.860 +Yeah, it doesn't make better. + +00:52:46.100 --> 00:52:49.560 +But yeah, we build stuff internally in the moment really. + +00:52:49.670 --> 00:52:52.880 +I like we build it in a way, like in a separate package, + +00:52:53.320 --> 00:52:54.460 +just like an open source library. + +00:52:54.560 --> 00:52:55.760 +And if we feel like the time is right, + +00:52:55.860 --> 00:52:58.900 +it's just getting open sourced because a lot of things are reusable. + +00:53:00.300 --> 00:53:01.460 +And that's in all departments. + +00:53:01.840 --> 00:53:02.720 +Like that happens a lot. + +00:53:02.840 --> 00:53:06.700 +Like when I started there, I already realized that, + +00:53:06.710 --> 00:53:07.920 +like everyone was building open source, + +00:53:08.100 --> 00:53:09.560 +but now I joined in myself as well. + +00:53:09.680 --> 00:53:13.000 +I open source the library for compression, + +00:53:13.540 --> 00:53:18.760 +like compressing and decompressing archives in Python. + +00:53:19.860 --> 00:53:22.480 +Because the internal TAR5 thingy is just slow, + +00:53:22.540 --> 00:53:24.240 +and we needed it to be faster because we're + +00:53:24.440 --> 00:53:25.620 +stirring up the deployment process. + +00:53:25.820 --> 00:53:26.740 +We're like, hey, we could probably + +00:53:27.060 --> 00:53:28.180 +shape off a few seconds here. + +00:53:29.220 --> 00:53:31.840 +And then that's just open source for everyone to use. + +00:53:32.060 --> 00:53:35.060 +So we're contributing to the whole Python ecosystem as well. + +00:53:35.100 --> 00:53:36.460 +You have to say the name. + +00:53:36.680 --> 00:53:37.320 +It's so good. + +00:53:38.420 --> 00:53:43.460 +Is it good? It's just it's faster because it's faster than just tar. + +00:53:44.680 --> 00:53:46.200 +Fast tar. I love it. + +00:53:46.400 --> 00:53:51.220 +Fast tar. You can say with that very German accent. + +00:53:51.540 --> 00:53:51.760 +With the. + +00:53:55.960 --> 00:53:57.680 +I could have, but you did it better. + +00:53:58.830 --> 00:53:58.980 +Yeah. + +00:53:59.860 --> 00:54:00.700 +I'm afraid to see. + +00:54:01.420 --> 00:54:03.740 +I'm scared of leaking stuff from our deployments. + +00:54:04.780 --> 00:54:06.180 +That's why I didn't say much. + +00:54:06.220 --> 00:54:08.380 +Is that the FASTA or is this different? + +00:54:08.680 --> 00:54:12.280 +No, it's literally the same name, but it's so fresh that, yeah, + +00:54:12.500 --> 00:54:14.200 +I guess Google showed you this one instead. + +00:54:14.820 --> 00:54:14.960 +Yeah. + +00:54:15.500 --> 00:54:16.100 +Let's see. + +00:54:16.400 --> 00:54:17.800 +Dr. John FASTA. + +00:54:18.700 --> 00:54:18.860 +Yeah. + +00:54:19.040 --> 00:54:19.340 +Okay. + +00:54:20.820 --> 00:54:20.960 +Yeah. + +00:54:21.460 --> 00:54:23.300 +But it's also like, you know, like we were- + +00:54:23.440 --> 00:54:25.540 +Doctor, full name. + +00:54:25.900 --> 00:54:26.240 +Full story. + +00:54:26.360 --> 00:54:27.400 +Yeah, yeah, yeah. + +00:54:29.700 --> 00:54:30.660 +There you go. + +00:54:31.160 --> 00:54:31.520 +One word. + +00:54:31.520 --> 00:54:31.860 +Wow. + +00:54:31.860 --> 00:54:33.420 +That's a lot of zeros. + +00:54:34.220 --> 00:54:37.100 +You need to like SEO-ify this package, man. + +00:54:37.260 --> 00:54:39.020 +I'm searching. I know. I'm searching on GitHub. + +00:54:40.120 --> 00:54:41.680 +Why is that pull request now? + +00:54:43.440 --> 00:54:44.520 +Am I that hard to find? + +00:54:45.020 --> 00:54:45.640 +Oh my gosh. + +00:54:46.120 --> 00:54:47.380 +Hold on, hold on, hold on. + +00:54:48.200 --> 00:54:48.860 +It's in the Chelsea. + +00:54:52.720 --> 00:54:53.840 +Yes, there we go. + +00:54:54.140 --> 00:54:54.660 +Okay, okay. + +00:54:55.840 --> 00:54:57.740 +The detective is at work. Here we go faster. + +00:54:58.000 --> 00:54:59.240 +Yeah, it's already gone. + +00:54:59.380 --> 00:55:01.200 +Well, I'm not logged in on this machine. + +00:55:01.320 --> 00:55:02.820 +This is just my streaming, not my dev machine. + +00:55:03.240 --> 00:55:05.900 +I'll go star it. We'll get you some stars. + +00:55:08.200 --> 00:55:09.620 +That's the irony about it. + +00:55:10.160 --> 00:55:11.280 +It literally has no stars, + +00:55:11.500 --> 00:55:12.860 +but if you scroll down and you see the downloads, + +00:55:13.200 --> 00:55:15.440 +that's going to prove we're actually using it. + +00:55:16.220 --> 00:55:18.380 +Yeah, awesome. Super cool. + +00:55:20.440 --> 00:55:24.340 +This is pretty neat. Yeah, I like it. + +00:55:25.180 --> 00:55:26.220 +It's a little context manager. + +00:55:29.360 --> 00:55:34.880 +It's almost working the same as the TAR file in the standard library, + +00:55:35.180 --> 00:55:38.760 +like the same, like almost similar API to that. + +00:55:39.340 --> 00:55:41.620 +It's basically a drop-in replacement, more or less. + +00:55:42.820 --> 00:55:42.900 +Cool. + +00:55:43.120 --> 00:55:44.960 +But then everything happens in Rust. + +00:55:45.360 --> 00:55:45.760 +Because Rust. + +00:55:46.360 --> 00:55:47.300 +Because it's fast. + +00:55:47.580 --> 00:55:48.480 +Because Rust, yeah. + +00:55:48.640 --> 00:55:50.240 +Well, as soon as it becomes infrastructure + +00:55:51.260 --> 00:55:52.640 +and you've got to run it a million times, + +00:55:53.200 --> 00:55:54.580 +that starts to make sense, right? + +00:55:55.220 --> 00:55:55.280 +Yeah. + +00:55:55.460 --> 00:55:58.760 +I mean, Python is one of the fastest programming languages + +00:55:59.480 --> 00:56:04.120 +in the world when you think about human time + +00:56:04.250 --> 00:56:05.400 +to build the things, right? + +00:56:05.540 --> 00:56:07.360 +Like that's one of its real superpowers. + +00:56:07.490 --> 00:56:09.700 +It's like, I mean, there's the whole story + +00:56:11.850 --> 00:56:14.600 +of Google Video and YouTube, right? + +00:56:14.690 --> 00:56:17.460 +And Google Video is written in C++ with 100 engineers, + +00:56:17.690 --> 00:56:19.780 +and YouTube was a small team in Python. + +00:56:19.990 --> 00:56:21.680 +And they just couldn't keep up with the features. + +00:56:21.840 --> 00:56:23.900 +So they bought this little thing YouTube + +00:56:24.040 --> 00:56:28.020 +and see if we're going to make something with it. And last I checked, it was still in Python. I'm + +00:56:28.080 --> 00:56:34.000 +sure some of it isn't but a few years ago it was, which is wild. Anyway, there's different ways of + +00:56:34.140 --> 00:56:38.920 +fast but when it's down to like little utilities. Yeah. I know. They're trying to make Python fast. + +00:56:39.120 --> 00:56:47.720 +Yeah, I honestly massive success in the last five years, right? Yeah, yeah. Like since 311, + +00:56:48.220 --> 00:56:53.880 +since the specializing adaptive interpreter has been pretty big. Yeah, 3.9 and 3.11 did a lot of + +00:56:53.900 --> 00:56:58.320 +foundational work and then three nine onward really just uncorked a lot of innovation there + +00:56:58.740 --> 00:57:04.960 +yeah yeah that's pretty awesome all right um it sounds like sebastian you've talked a lot about + +00:57:05.140 --> 00:57:12.340 +kubernetes so i imagine kubernetes is happening uh do we get to pick what data centers it runs on do + +00:57:12.380 --> 00:57:20.680 +we get to pick what clouds it runs on like you're going to get to pick some of these things not yet + +00:57:20.760 --> 00:57:27.760 +is not released yet. But you know, like it's, so of course, like we have like regular 12 providers + +00:57:27.920 --> 00:57:32.360 +in the middle and there's a bunch of Kubernetes, then there's a bunch of additional stuff that needs + +00:57:32.500 --> 00:57:36.760 +to run on top. Then there's like custom Kubernetes controllers and things that + +00:57:37.600 --> 00:57:43.260 +Jonathan was saying that he's having to write in Go so people in Python can be happy to be able to + +00:57:43.840 --> 00:57:47.160 +you know, like manage all the Kubernetes shenanigans that need to happen because + +00:57:47.180 --> 00:57:53.460 +there's so much complexity that needs to be handled and there's a lot of that we do a lot of + +00:57:54.000 --> 00:57:58.800 +advanced tricks also jonathan was recently doing a bunch of advanced tricks to handle the caches + +00:57:59.660 --> 00:58:05.720 +for the builds so the way that we handle caches and we also like tap into uv and how things work + +00:58:05.900 --> 00:58:12.440 +so that builds can be super super fast because it's like something it's we are you know like we + +00:58:12.460 --> 00:58:20.140 +are very much targeted at FastAPI and Python in general. So we can take advantage of knowing how + +00:58:20.380 --> 00:58:25.160 +things run internally, how things are installed, how to optimize everything. So everything is just + +00:58:25.240 --> 00:58:31.920 +like super fast, super fast. Yeah. I imagine you all have base Docker images that are like just + +00:58:32.860 --> 00:58:39.020 +one one layer away from whoever's code is running. You know, like you've got it all + +00:58:39.020 --> 00:58:43.360 +already pre-built with FastAPI and whatever settings of Python you want. + +00:58:43.900 --> 00:58:50.280 +Yeah. Like a bunch of things and tricks. Yeah. But also like different things, + +00:58:50.500 --> 00:58:55.260 +like the different ways that we do to actually build the things and install things and put them + +00:58:55.560 --> 00:59:01.980 +inside of the actual, you know, like build application. There's like a lot of sourcing + +00:59:02.720 --> 00:59:08.100 +that we do there and like Jonathan has been like working on a lot of that. And there's also all the + +00:59:07.900 --> 00:59:12.760 +logic and all the stuff we have a bunch of stuff on top of that to handle auto scaling which is + +00:59:12.860 --> 00:59:18.700 +something that is actually not that easy to find in different providers we have auto scaling based + +00:59:18.840 --> 00:59:25.180 +on requests including scaling down to zero which saves costs but it's you know like this is not + +00:59:25.300 --> 00:59:30.980 +lambdas it's not aws lambdas it's we are like it's like the full uh deployed application the full + +00:59:31.220 --> 00:59:36.680 +container or whatever it is uh which is you know like the full thing with all the dependencies is + +00:59:36.700 --> 00:59:42.240 +running for whenever it has to run, but we can scale based on requests. So it's like, I guess + +00:59:42.360 --> 00:59:49.700 +it's like the type of thing that you will have if you have this giant cluster for a huge enterprise + +00:59:50.220 --> 00:59:57.720 +with a bunch of infra people making sure everything just works perfectly, but you just pay us to do + +00:59:57.840 --> 01:00:05.780 +that for you. Yeah. But I think this is also a good time for us to probably say lots of stuff is + +01:00:05.740 --> 01:00:07.260 +is coming and we're in private beta. + +01:00:07.460 --> 01:00:09.300 +And so you should sign up for the wait list + +01:00:10.760 --> 01:00:12.140 +so that you can get admitted + +01:00:12.220 --> 01:00:14.460 +and try out these very cool things we've been talking about. + +01:00:14.460 --> 01:00:15.260 +Yeah, absolutely. + +01:00:15.560 --> 01:00:19.900 +And I think I'll let Tech Insider out in the audience + +01:00:21.280 --> 01:00:22.120 +sort of lead to this. + +01:00:22.120 --> 01:00:22.760 +Public release when? + +01:00:23.440 --> 01:00:27.260 +Yeah, lead us into my final topic, + +01:00:27.460 --> 01:00:28.640 +which is just what's the roadmap? + +01:00:29.220 --> 01:00:29.940 +When is this stuff? + +01:00:30.440 --> 01:00:32.820 +How do we get into it here? + +01:00:34.260 --> 01:00:45.820 +So right now we have the waiting list and we are onboarding people. + +01:00:45.860 --> 01:00:48.400 +We already have like a bunch of people in the private beta. + +01:00:48.820 --> 01:00:53.960 +We are going to keep onboarding people from the waiting list and like, you know, like wrap that up. + +01:00:54.060 --> 01:00:58.520 +But it will be like, you know, like through the waiting list is the main place where we are onboarding people. + +01:00:58.640 --> 01:01:02.600 +We want to make sure that everything is super fine tuned and we're going to keep it that way for a while. + +01:01:02.660 --> 01:01:09.160 +So people that are on the waiting list are going to be the ones that are going to be able to start using it the soonest. + +01:01:09.490 --> 01:01:13.180 +At some point, we'll probably have ways for people to invite others and things like that. + +01:01:15.340 --> 01:01:23.780 +About the things that we are building, we are super focused on FastAPI and then Python in general. + +01:01:23.920 --> 01:01:27.320 +at some point will probably support different tools, + +01:01:27.810 --> 01:01:31.320 +different ways to run, like Python code in general, + +01:01:31.560 --> 01:01:32.600 +probably different frameworks. + +01:01:33.120 --> 01:01:37.980 +It will also depend a lot on what the users are asking for, + +01:01:38.080 --> 01:01:41.020 +where are the tools, the frameworks, the use cases, + +01:01:41.190 --> 01:01:42.420 +the things that they need to build. + +01:01:42.580 --> 01:01:45.620 +And we're gonna evolve the platform and the system + +01:01:46.200 --> 01:01:48.840 +based on what people need out of it. + +01:01:49.420 --> 01:01:53.040 +- Yeah, and we have a GitHub repo where we have issues, + +01:01:53.240 --> 01:01:57.540 +But we also have like a Slack that once people are admitted, they can talk directly to us. + +01:01:57.660 --> 01:02:04.080 +And that feedback is really, really valuable for shaping the roadmap and figuring out all the fun things you want us to support. + +01:02:04.680 --> 01:02:05.120 +Yeah. + +01:02:05.790 --> 01:02:05.900 +Awesome. + +01:02:07.700 --> 01:02:09.080 +Of course, you're going to charge money for it. + +01:02:09.630 --> 01:02:13.640 +It runs on the servers and you guys are not a charity. + +01:02:16.420 --> 01:02:17.500 +But I don't know. + +01:02:17.950 --> 01:02:22.320 +Can you give any sense of what you're thinking about that kind of stuff? + +01:02:22.520 --> 01:02:23.960 +or join the waitlist and see? + +01:02:25.220 --> 01:02:30.580 +Well, first join the waitlist and see, but we don't have like that predefined yet, + +01:02:31.440 --> 01:02:34.980 +but it will be on the ballpark of what you could get from different cloud providers. + +01:02:35.080 --> 01:02:44.100 +So like, you know, like different similar-ish providers will be on the ballpark of what you will get. + +01:02:45.200 --> 01:02:48.240 +But it's not written in stone yet. + +01:02:49.000 --> 01:02:53.780 +It's still a little bit different because we can auto scale based on requests. + +01:02:54.080 --> 01:02:58.140 +So we can increase the amount of replicas of your application automatically. + +01:02:59.100 --> 01:03:02.240 +And then we can decrease them automatically and we can scale down to zero. + +01:03:02.860 --> 01:03:08.840 +So you can probably handle all the load that you need and in the end spend a lot less because + +01:03:08.860 --> 01:03:12.420 +you don't have to have a bunch of instances constantly running or things like that. + +01:03:13.220 --> 01:03:19.340 +So it will probably work a little bit different than what it will be for other providers. + +01:03:19.880 --> 01:03:22.140 +But in the end, it should be roughly similar. + +01:03:22.840 --> 01:03:23.220 +Okay. + +01:03:23.900 --> 01:03:29.840 +And given the fact that you all handle so much of it as a platform as a service type of thing, + +01:03:30.860 --> 01:03:36.280 +you don't have to have a cloud expert on hand or a DevOps expert necessarily, right? + +01:03:36.500 --> 01:03:43.800 +it's soon as a company has to hire somebody to be an AWS cloud architect or something like that. + +01:03:44.580 --> 01:03:51.500 +It's no longer just what is your AWS bill, right? Yeah. Yeah. And it's, yeah, it's also, + +01:03:52.060 --> 01:03:55.480 +it's also a lot of the pain that we are swallowing. So you don't have to. + +01:03:56.330 --> 01:03:58.880 +Exactly. It's part of taking one for the team, right? Yes. + +01:04:00.740 --> 01:04:08.120 +Indeed. All right. So I had one or two things specifically that I was seeing is like + +01:04:09.620 --> 01:04:13.880 +custom domains. How far off are custom domains? I was like, oh, I could put some cool things + +01:04:13.960 --> 01:04:21.600 +on there. I could tell Jonathan is psyched about this. I was like, I would be really + +01:04:21.800 --> 01:04:28.380 +fun to put one of my really small FastAPI projects over there. You know, something I + +01:04:28.340 --> 01:04:31.440 +set up for some of my courses or something that I can like point people to go + +01:04:31.520 --> 01:04:34.880 +look, it's running FastAPI cloud on it, you guys can check that out over there. + +01:04:35.660 --> 01:04:36.960 +And I like, but it's on its own domain. + +01:04:37.100 --> 01:04:41.500 +And those that domain is baked into the course videos, you know what I mean? + +01:04:41.580 --> 01:04:42.820 +And it's, it's written in stone. + +01:04:42.980 --> 01:04:43.360 +It's marketing. + +01:04:44.260 --> 01:04:45.380 +Yeah, exactly. + +01:04:45.740 --> 01:04:52.440 +So, so I can't really move it because it has, you know, some sub domain + +01:04:52.560 --> 01:04:53.220 +of talk Python, right? + +01:04:55.380 --> 01:04:55.640 +Jonathan. + +01:04:57.880 --> 01:04:59.160 +I was going to say that work. + +01:04:59.900 --> 01:05:03.100 +See, I was working on it and then I got the notification by Google + +01:05:03.300 --> 01:05:05.540 +calendars that I should join a certain podcast. + +01:05:06.080 --> 01:05:08.560 +Are you telling me we don't have custom domains? + +01:05:08.960 --> 01:05:10.980 +Because I'm here asking you about custom domains. + +01:05:11.030 --> 01:05:11.780 +How meta is that? + +01:05:11.790 --> 01:05:12.180 +You got it. + +01:05:12.610 --> 01:05:12.720 +Yeah. + +01:05:14.460 --> 01:05:18.420 +It could be here already, but no, you have to wait a bit more. + +01:05:19.160 --> 01:05:20.160 +OK, but soon? + +01:05:21.020 --> 01:05:21.460 +Yes. + +01:05:21.710 --> 01:05:21.840 +Yeah. + +01:05:22.200 --> 01:05:22.360 +Yeah. + +01:05:23.220 --> 01:05:26.640 +Soon is broad enough, but I'm actively working on it. + +01:05:27.120 --> 01:05:28.300 +Let's put it like that. + +01:05:28.800 --> 01:05:28.900 +Awesome. + +01:05:30.420 --> 01:05:31.180 +Okay, that sounds great. + +01:05:31.420 --> 01:05:35.500 +And then, I mean, I just, it's never simple. + +01:05:35.840 --> 01:05:39.240 +You know, I just, I set up some stuff and it's like, you get the pop-up. + +01:05:39.560 --> 01:05:47.220 +Oh, you got to put this, you know, this TXT record, this CNAME or whatever record into your DNS and then we're checking it. + +01:05:47.260 --> 01:05:50.060 +Oh, it might take three days for your DNS to propagate. + +01:05:50.280 --> 01:05:53.200 +So hang in there and just, I can imagine like you're having fun. + +01:05:54.040 --> 01:05:54.780 +Yeah, I guess. + +01:05:55.180 --> 01:05:55.580 +Yeah. + +01:05:56.320 --> 01:05:58.700 +Yeah, that's like, wow. + +01:05:58.920 --> 01:06:02.240 +I thought I'm almost off work, but no, you're bringing it all back. + +01:06:02.420 --> 01:06:03.060 +I'm sorry. + +01:06:04.560 --> 01:06:09.180 +I'm sure the company could support therapy to work through the issues + +01:06:09.260 --> 01:06:11.080 +and the trauma that you've suffered from the DNS. + +01:06:12.080 --> 01:06:12.880 +It's always DNS. + +01:06:13.160 --> 01:06:13.340 +That's right. + +01:06:13.420 --> 01:06:14.320 +It's always DNS. + +01:06:14.860 --> 01:06:15.700 +Yes, it's always DNS. + +01:06:16.420 --> 01:06:20.460 +One of our goals with custom domains is also to make it super simple + +01:06:20.500 --> 01:06:21.600 +for you to set up them. + +01:06:22.260 --> 01:06:27.340 +Like, for example, if you're using one of the providers that support OAuth, + +01:06:27.370 --> 01:06:30.120 +we can also just do one click, and then it's going to be automatically. + +01:06:30.190 --> 01:06:31.840 +Oh, that's cool. Yeah, yeah, that's really nice. + +01:06:32.440 --> 01:06:34.580 +But unfortunately, it depends on the platform you're using, + +01:06:34.660 --> 01:06:36.600 +because all of them support this. + +01:06:39.000 --> 01:06:40.700 +Yeah, just a shout-out for people out there. + +01:06:40.920 --> 01:06:43.000 +This is made by the person in charge of most of the integrations. + +01:06:43.110 --> 01:06:47.500 +So Patrick has built, we have integrations for a bunch of database providers + +01:06:47.670 --> 01:06:48.260 +and things like that. + +01:06:48.520 --> 01:06:53.360 +Patrick, I think now Patrick knows by memory the open ID specification. + +01:06:53.720 --> 01:06:54.120 +I don't know. + +01:06:57.180 --> 01:07:00.020 +Yeah, the other thing I wanted to talk a bit about was just integrations, + +01:07:00.180 --> 01:07:01.660 +like what kind of stuff you guys have come in. + +01:07:01.700 --> 01:07:05.200 +I saw that Hugging Face is going to be integrated soon. + +01:07:05.400 --> 01:07:09.320 +You've got Supabase, which is kind of Postgres as a service. + +01:07:10.240 --> 01:07:15.480 +There's a lot of those things out there that theoretically could be added. + +01:07:16.620 --> 01:07:19.900 +Yeah, I mean, someone also asked for MongoDB. + +01:07:20.000 --> 01:07:22.100 +Maybe that's one that we're going to take a look into. + +01:07:22.300 --> 01:07:24.780 +It really depends on the provider. + +01:07:24.920 --> 01:07:27.640 +So at the moment, we don't want to ask databases for you + +01:07:27.800 --> 01:07:29.800 +because that's also another kind of rabbit hole. + +01:07:30.920 --> 01:07:32.300 +Jonathan is probably not ready for that. + +01:07:34.180 --> 01:07:36.180 +But yeah, definitely like databases. + +01:07:36.320 --> 01:07:38.220 +But I guess we can say that we also + +01:07:38.640 --> 01:07:41.640 +talking with the people from Pydantic + +01:07:41.740 --> 01:07:44.080 +so we can integrate maybe Logfire automatically, + +01:07:44.500 --> 01:07:45.040 +that kind of stuff. + +01:07:46.980 --> 01:07:47.200 +Yeah. + +01:07:47.420 --> 01:07:50.880 +And also like Redis, which is also another kind of database + +01:07:51.820 --> 01:07:52.740 +that's also coming soon. + +01:07:54.060 --> 01:07:56.740 +- Yeah, there's a couple of database as a service + +01:07:58.160 --> 01:08:00.020 +type things that don't require too much + +01:08:00.080 --> 01:08:04.160 +other than just connecting API keys + +01:08:04.260 --> 01:08:05.200 +and something like that, right? + +01:08:05.440 --> 01:08:06.640 +Those seem like low hanging fruit. + +01:08:07.400 --> 01:08:11.680 +- Yeah, so like the kind of goal with the integration + +01:08:11.940 --> 01:08:13.780 +is not just like, yeah, right now, + +01:08:13.860 --> 01:08:15.360 +It's just setting up an environment variable. + +01:08:16.279 --> 01:08:19.100 +But the idea is also to do more-- + +01:08:21.150 --> 01:08:23.500 +I don't know-- the proper integration, I would say. + +01:08:24.460 --> 01:08:26.500 +For example, for things like Superbase, + +01:08:27.470 --> 01:08:28.640 +yeah, I think that support branching. + +01:08:29.020 --> 01:08:32.359 +For example, once we support Procrast previews for GitHub, + +01:08:32.700 --> 01:08:34.580 +we can also create a branch automatically for you + +01:08:34.670 --> 01:08:36.620 +if you have the Superbase integration enabled. + +01:08:37.020 --> 01:08:38.799 +And we can do this kind of stuff as well. + +01:08:38.980 --> 01:08:42.680 +Or even we could show some information about database, + +01:08:42.960 --> 01:08:45.560 +I don't know, like load or like memory usage, + +01:08:45.660 --> 01:08:47.279 +things are directly from our dashboard. + +01:08:47.339 --> 01:08:48.220 +So you don't have to go there. + +01:08:48.920 --> 01:08:50.920 +That's the main reason why we're building this + +01:08:52.140 --> 01:08:53.880 +infrastructure for the integration. + +01:08:54.900 --> 01:08:55.400 +- All right. + +01:08:56.960 --> 01:08:59.120 +Well, people can sign up to the waiting list + +01:08:59.319 --> 01:09:02.339 +and hopefully get on the private beta and... + +01:09:03.660 --> 01:09:05.140 +- We actually check the waiting list. + +01:09:05.140 --> 01:09:08.040 +We actually check the use cases, theme sizes, + +01:09:08.299 --> 01:09:10.380 +like what are people building with it. + +01:09:10.400 --> 01:09:12.419 +Like we actually go and check it + +01:09:12.440 --> 01:09:16.520 +and we bring in people from the waiting list. + +01:09:16.859 --> 01:09:19.120 +Nice. I didn't join the waiting list directly. + +01:09:19.359 --> 01:09:25.319 +I was added by some guy I know who was very kind to help me get some behind the scene look. + +01:09:26.140 --> 01:09:27.220 +I don't know what the process is. + +01:09:27.290 --> 01:09:32.140 +Do you actually say what you want to do with it and you evaluate that a little bit as well based on like, + +01:09:32.150 --> 01:09:34.200 +hey, this would be a cool use case for us to support? + +01:09:35.440 --> 01:09:36.359 +Yes, yes, of course. + +01:09:36.450 --> 01:09:41.199 +Because there are many types of applications and + +01:09:41.220 --> 01:09:46.560 +many types of different team sizes, many types of things that people might want to build. + +01:09:46.700 --> 01:09:52.400 +And we try to see, like, okay, where is the case where we could be, like, a good fit and we can provide a great service? + +01:09:53.000 --> 01:09:54.880 +And what are the things that people are trying to build? + +01:09:55.440 --> 01:10:03.420 +Also, it helps us see, like, you know, like, what are people trying to do with FastAPI Cloud so that we know what we have to provide? + +01:10:04.440 --> 01:10:11.020 +But we actually go and check those submissions. + +01:10:12.000 --> 01:10:20.760 +It's actually thousands of people in the waiting list, but we still go and check and approve kind of manually. + +01:10:21.940 --> 01:10:22.060 +Yeah. + +01:10:22.260 --> 01:10:26.600 +To bring a bunch of people on board in the different ways that we have been bringing people. + +01:10:26.960 --> 01:10:32.220 +So if people go and join the waiting list and actually tell us what is their use case, their team, + +01:10:32.360 --> 01:10:37.700 +like what are they planning on doing, there's a much higher chance that we are going to + +01:10:38.020 --> 01:10:39.400 +go and just like bring them up. + +01:10:39.960 --> 01:10:40.400 +Okay. + +01:10:41.310 --> 01:10:41.420 +Awesome. + +01:10:42.040 --> 01:10:45.880 +So, everyone, go join the waitlist if you're doing FastAPI. + +01:10:46.560 --> 01:10:48.880 +I'll link to it in the show notes, of course. + +01:10:51.040 --> 01:10:54.680 +Thank you all for being here and sharing the story. + +01:10:55.000 --> 01:10:58.960 +And I, for one, am very excited to see FastAPI Cloud exist. + +01:11:00.380 --> 01:11:05.380 +just one more way to make FastAPI stronger and more resilient and so on. + +01:11:06.780 --> 01:11:08.700 +Thank you very much. Thank you for having us. + +01:11:09.120 --> 01:11:10.060 +Yeah, thanks for having us. + +01:11:11.020 --> 01:11:12.520 +Yeah, you bet. Bye, everyone. + +01:11:13.200 --> 01:11:13.720 +Bye, folks. + +01:11:13.960 --> 01:11:14.040 +Bye. + diff --git a/youtube_transcripts/538-digital-humanities-original.vtt b/youtube_transcripts/538-digital-humanities-original.vtt new file mode 100644 index 0000000..e1c6915 --- /dev/null +++ b/youtube_transcripts/538-digital-humanities-original.vtt @@ -0,0 +1,3299 @@ +WEBVTT + +00:00:01.540 --> 00:00:03.560 +Hello, David. Welcome to Talk Python To Me. + +00:00:04.080 --> 00:00:04.700 +Amazing to have you here. + +00:00:06.200 --> 00:00:06.800 +I'm glad to be here. + +00:00:07.320 --> 00:00:10.060 +Talk Python has been part of my story up to this point. + +00:00:10.360 --> 00:00:11.760 +Has it? Okay. Well, + +00:00:12.700 --> 00:00:15.180 +you are about to write the next chapter in the story. + +00:00:15.350 --> 00:00:16.500 +So that's pretty excellent. + +00:00:17.840 --> 00:00:19.000 +I have a sense of what's coming. + +00:00:19.530 --> 00:00:22.640 +We planned out what we're going to talk about and that sort of thing. + +00:00:23.500 --> 00:00:26.280 +And I'm really excited about this topic. + +00:00:26.700 --> 00:00:29.640 +So it's going to be a good one. + +00:00:30.580 --> 00:00:34.600 +Honestly, I think one of the real powers of the Python community + +00:00:34.640 --> 00:00:37.320 +and the reason the language has such staying power + +00:00:37.320 --> 00:00:44.900 +is there's such a diversity of use cases, technology standpoints, right? + +00:00:44.960 --> 00:00:48.640 +Like I build software for this group or I build these types of apps + +00:00:48.640 --> 00:00:50.680 +and it's not just, you know, like Ruby on Rails, + +00:00:51.040 --> 00:00:53.200 +which, you know, it's been very popular and stuff, + +00:00:53.220 --> 00:00:54.840 +but it's for websites, right? + +00:00:54.960 --> 00:00:55.260 +You know what I mean? + +00:00:58.280 --> 00:01:07.880 +Yeah, absolutely. I mean, web development has dominated my use of it, but my entry into it, + +00:01:08.030 --> 00:01:12.180 +which I suppose I'll mention in a moment, was through all those little tools. + +00:01:13.260 --> 00:01:21.540 +Was it? Okay. Yeah. Well, let's hear it. Who are you, David Flood? Introduce yourself real quick + +00:01:21.560 --> 00:01:23.300 +and tell us about how you got into it. + +00:01:24.140 --> 00:01:24.200 +Yep. + +00:01:25.500 --> 00:01:30.500 +So my background is in music and the humanities. + +00:01:31.280 --> 00:01:34.820 +I mean, in 2019, I didn't know what Python was + +00:01:35.120 --> 00:01:36.560 +or the name of any programming language. + +00:01:38.800 --> 00:01:41.060 +And I've been doing textual criticism, + +00:01:41.780 --> 00:01:46.060 +which is, you know, there's lots of criticisms in the academy. + +00:01:46.640 --> 00:01:49.320 +This is the one where if you have lots and lots of versions + +00:01:49.340 --> 00:01:56.620 +of the same text you are comparing them to to work out what the initial text was and like how + +00:01:56.670 --> 00:02:03.980 +it changed over time okay give it give us an example okay so uh one of the famous examples + +00:02:04.090 --> 00:02:09.880 +hope i can remember it off the top of my head um is from shakespeare we're all familiar with the + +00:02:10.020 --> 00:02:16.280 +line to be or not to be that is the question that is the question uh well there's a there's a variant + +00:02:16.300 --> 00:02:23.840 +of it. There's a variant of it. One of the early copies written by Shakespeare himself has... + +00:02:25.720 --> 00:02:29.420 +Somebody's going to be able to type into the chat exactly what it is. They'll know this anecdote, + +00:02:29.800 --> 00:02:38.180 +but it's something more like to be or not to be. I, that's the question. And so which one is the + +00:02:38.340 --> 00:02:42.860 +original one? Why did he change it? That's kind of one example. I work mainly in the New Testament, + +00:02:43.360 --> 00:02:53.600 +which is especially complicated because no other corpus from ancient history has as many copies of the same text as that corpus does. + +00:02:53.790 --> 00:02:56.740 +So it's quite complicated. + +00:02:57.050 --> 00:03:03.140 +And our techniques have grown because of that and perhaps become more advanced than other fields. + +00:03:03.760 --> 00:03:21.400 +I mean, that many variations over that huge span of time over different groups with different, maybe not intentions, but certainly colored by different worldviews and philosophies and so on. + +00:03:21.410 --> 00:03:23.420 +And yeah, I see the travel. + +00:03:24.600 --> 00:03:24.980 +No, yeah. + +00:03:25.200 --> 00:03:26.980 +And they were people of the book. + +00:03:27.660 --> 00:03:30.420 +So copying it is something that happened a lot. + +00:03:30.500 --> 00:03:34.680 +And they copied the monks, like the medieval monks copied everything. + +00:03:35.110 --> 00:03:38.640 +They copied, you know, our Greek classics. + +00:03:39.420 --> 00:03:41.900 +So that's what I was interested in. + +00:03:42.430 --> 00:03:48.980 +And because of the wealth of data that we have, computer tools are more and more important in that field. + +00:03:49.540 --> 00:03:55.780 +So when I started my PhD in 2019, I knew that I wanted to use some of these cutting edge tools. + +00:03:56.320 --> 00:03:57.940 +Some of them may be surprising. + +00:03:58.400 --> 00:04:03.640 +For example, we've been using phylogenetic software. + +00:04:04.300 --> 00:04:15.460 +This is software that evolutionary biologists are using or computational biologists are using to track, for example, how COVID strains like mutate over time. + +00:04:16.030 --> 00:04:16.680 +Oh, interesting. + +00:04:17.519 --> 00:04:20.380 +What they're comparing are the DNA letters. + +00:04:20.890 --> 00:04:24.320 +And so you have the sequence of letters and you're comparing how those change over time. + +00:04:24.860 --> 00:04:37.120 +Well, you can swap in textual variants for DNA letters, and now we can track how texts change over time and group them into families, things like that. + +00:04:38.500 --> 00:04:41.980 +It's like a time series, but of words or letters or something. + +00:04:42.140 --> 00:04:42.720 +Yeah, how interesting. + +00:04:42.980 --> 00:04:43.540 +Yeah, I mean, yeah. + +00:04:44.100 --> 00:04:50.720 +There's lots of important algorithms for comparing sequences of things. + +00:04:51.540 --> 00:04:58.360 +And so if we can just swap in Greek, you know, Greek words and Greek text instead, then we can maybe apply it to textual criticism. + +00:04:58.870 --> 00:05:00.380 +So I was pretty interested in those things. + +00:05:00.430 --> 00:05:07.660 +That wasn't actually the method that brought me into it, but something like that kind of computer intensive tools. + +00:05:09.240 --> 00:05:13.700 +What I learned is that these tools were not like they weren't actually available to me. + +00:05:14.200 --> 00:05:15.840 +They weren't desktop applications. + +00:05:16.050 --> 00:05:18.780 +And for the most part, they weren't public web applications. + +00:05:19.050 --> 00:05:20.660 +They were software libraries. + +00:05:22.139 --> 00:05:25.780 +I see. So something on PyPI or something like that, right? + +00:05:26.280 --> 00:05:33.600 +Yeah, exactly. Exactly. Or Java. And I needed to glue them together. So the long story short on + +00:05:33.720 --> 00:05:39.300 +that is during the first year of my PhD, I was picking up Python, watching YouTube videos while + +00:05:39.300 --> 00:05:45.360 +I was doing the dishes. And then the pandemic hit while I was living in Edinburgh in Scotland, + +00:05:45.420 --> 00:05:53.760 +probably not far from will mccougan um and uh so the pandemic gave me the excuse to spend even a + +00:05:53.880 --> 00:06:00.500 +few more hours um you know each day um picking up these new these new technical skills uh but so i + +00:06:00.620 --> 00:06:06.780 +did it i was able to use these advanced tools in my in my work but what was really important to me + +00:06:06.880 --> 00:06:12.519 +was sharing like making that available to my colleagues as i had to i had to move from writing + +00:06:12.580 --> 00:06:17.800 +these like bad top to bottom Python scripts into things that could be reused by other people. + +00:06:18.470 --> 00:06:23.140 +And that led me into the web because the web is where that's how I can share with anybody. + +00:06:24.000 --> 00:06:33.020 +It's the easiest way. And it's really wild how much the web is kind of the last bastion of + +00:06:34.220 --> 00:06:40.639 +app freedom. It's so bizarre because, you know, I've many times told the stories of the insane + +00:06:40.660 --> 00:06:46.820 +battles of just getting our apps that just playback video of content that's already on the web + +00:06:47.420 --> 00:06:53.560 +into the app store. I mean, weeks of fighting about the weirdest, most nonsensical things with + +00:06:53.700 --> 00:07:01.280 +both Google and Apple. But we also now have the Mac platform and the Windows platform very + +00:07:01.580 --> 00:07:07.380 +aggressively looking for digital code certificates and all sorts of signing and other kinds of, + +00:07:07.460 --> 00:07:10.540 +You can't even just send somebody an executable anymore. + +00:07:11.150 --> 00:07:12.640 +It won't run. It's crazy. + +00:07:12.900 --> 00:07:15.840 +It's down to like, okay, put it on the web, I guess. + +00:07:17.080 --> 00:07:21.200 +- That's right. I played the game of distributing desktop apps. + +00:07:21.340 --> 00:07:21.880 +That's how I did it. + +00:07:22.980 --> 00:07:24.420 +That's how I initially distributed things. + +00:07:25.760 --> 00:07:28.840 +And at this point, I just require people to install Python + +00:07:29.200 --> 00:07:31.560 +and then install my desktop app from PyPI + +00:07:31.780 --> 00:07:34.660 +because it's too hard otherwise for me. + +00:07:34.980 --> 00:07:38.960 +I mean, I could pay for the code signing from Apple and do all of that, + +00:07:39.020 --> 00:07:42.200 +but it's just too much work for the time that I have. + +00:07:42.920 --> 00:07:44.500 +Yeah, I'm about to do another round of it. + +00:07:44.500 --> 00:07:48.060 +I'm working on an app, and my developer account is still active, + +00:07:48.260 --> 00:07:50.060 +so we might have a fresh round of fun. + +00:07:50.320 --> 00:07:51.660 +Hopefully it goes through this time. + +00:07:52.640 --> 00:07:54.540 +Anyway, I do think it's such a challenge. + +00:07:54.740 --> 00:07:55.920 +And are you leveraging? + +00:07:56.320 --> 00:07:57.560 +I don't know if the timing was right. + +00:07:57.700 --> 00:07:58.700 +Maybe this was too early. + +00:07:59.620 --> 00:08:03.980 +But these days, are you leveraging things like uvx to run, + +00:08:04.340 --> 00:08:07.840 +or are you just pip install this thing and then run it? + +00:08:08.899 --> 00:08:11.120 +Yeah, I haven't updated the readme in a while, + +00:08:11.180 --> 00:08:13.180 +so I think it just asks for pip. + +00:08:13.860 --> 00:08:17.340 +But certainly, if somebody asked me today, + +00:08:17.980 --> 00:08:20.100 +I would say, "Yeah, just install this with uv." + +00:08:20.640 --> 00:08:20.940 +Yeah, yeah. + +00:08:21.100 --> 00:08:22.380 +Because then they don't even need Python. + +00:08:23.300 --> 00:08:23.560 +Exactly. + +00:08:23.900 --> 00:08:24.700 +If they get it, + +00:08:25.020 --> 00:08:25.620 +and that's a really, + +00:08:25.820 --> 00:08:28.340 +it is another barrier reduced + +00:08:28.480 --> 00:08:29.420 +in distributing these applications. + +00:08:30.700 --> 00:08:33.719 +If you can get uv installed on a machine, + +00:08:34.080 --> 00:08:37.700 +then you don't even have to say install just the way you run it is uvx my thing + +00:08:38.419 --> 00:08:44.980 +and it's all transparent to you right which is beautiful so what was it like yeah so what was + +00:08:44.980 --> 00:08:55.920 +it like coming from what sounds like a not super screen focus super techie aspect and having to + +00:08:56.100 --> 00:09:01.019 +dive into this world and someday you're probably like how is it that i'm publishing stuff to pypi + +00:09:01.040 --> 00:09:08.160 +what has happened to me yeah uh well yeah i i remember when i when i first signed up for github + +00:09:08.500 --> 00:09:12.140 +because you know whatever youtube tutorial i was working through at the time + +00:09:13.200 --> 00:09:21.700 +um you know said that i need that that i needed to do that um you know i i think it all started + +00:09:21.860 --> 00:09:28.379 +making a lot of sense um i didn't have any technical background but the uh the world + +00:09:29.920 --> 00:09:36.560 +the world kind of open source software it it just it kind of made sense it felt like it fit really + +00:09:36.740 --> 00:09:43.740 +well into my into my academic um you know circle yeah i think a lot of the attitudes are similar + +00:09:44.540 --> 00:09:48.820 +i agree i think they are actually and i think that's i think that's a pretty neat thing + +00:09:51.040 --> 00:09:58.379 +yeah very cool all right well let's talk about what you're doing with digital humanities you're + +00:09:58.400 --> 00:10:03.960 +actually at a really interesting project or organization, I guess that does many projects, + +00:10:04.240 --> 00:10:09.700 +right? Yeah, yeah. So fast, fast forwarding, I did, I finished my PhD in the humanities. + +00:10:09.880 --> 00:10:15.500 +Yeah, sorry, I cut you off there, didn't I? That's fine. I had so much fun writing like these tools + +00:10:15.680 --> 00:10:20.640 +and then just get it solving the distribution problem to share them with other scholars. + +00:10:21.960 --> 00:10:27.120 +That was so fun that I was open to this kind of opportunity where now I'm doing this full time. + +00:10:28.080 --> 00:10:36.940 +And so, yes, I so I'm on the we call it affectionately Darth, which is digital arts and humanities at Harvard. + +00:10:37.840 --> 00:10:42.380 +Awesome. There has to be a lot of Star Wars memes and references, I'm sure. + +00:10:43.060 --> 00:10:46.360 +If you can pull up a 404, I think there will be a Darth Vader. + +00:10:47.240 --> 00:10:48.860 +Oh, seriously, I'm here for it. + +00:10:54.000 --> 00:10:57.020 +Yes, page not found. I find your lack of nav disturbing. + +00:10:59.660 --> 00:11:05.020 +you know what i think that is beautiful and i really i really think that people should embrace + +00:11:05.540 --> 00:11:14.300 +the 404 the fun 404 page you know more right there should really be something going on that + +00:11:14.440 --> 00:11:19.600 +like makes it you know something hasn't worked out but you can just you can make people laugh + +00:11:20.420 --> 00:11:21.500 +but I appreciate that + +00:11:23.200 --> 00:11:25.120 +I've heard people push back against it + +00:11:25.240 --> 00:11:26.020 +like if you're on + +00:11:27.560 --> 00:11:29.080 +your medical website + +00:11:29.660 --> 00:11:31.320 +and you're maybe about to get bad news + +00:11:31.460 --> 00:11:33.120 +and then you get like a picture of a kitten + +00:11:35.780 --> 00:11:37.720 +Dr. Kitten doesn't know where your results went + +00:11:37.820 --> 00:11:39.140 +I get that that's not funny + +00:11:40.740 --> 00:11:42.840 +but I mean most things are not that serious + +00:11:44.280 --> 00:11:44.760 +mostly + +00:11:45.840 --> 00:11:47.560 +okay so what kind of things + +00:11:47.960 --> 00:11:48.899 +does Darth do + +00:11:49.120 --> 00:11:54.160 +You've described this as kind of a web or tech agency within Harvard. + +00:11:55.020 --> 00:11:56.260 +Maybe more within the digital art group. + +00:11:56.420 --> 00:11:57.060 +Yeah, it is very much. + +00:11:57.060 --> 00:11:59.520 +So, you know, Harvard has a gigantic IT group. + +00:12:00.040 --> 00:12:06.800 +I don't know how many hundreds of people work, but more than 500 people in IT. + +00:12:08.480 --> 00:12:12.720 +We are a small team, and we operate very much like a small agency. + +00:12:13.540 --> 00:12:22.160 +So usually what happens is a faculty member has a funded research project that's going to last for an amount of time. + +00:12:23.410 --> 00:12:25.820 +And then we consult with them to build it. + +00:12:27.240 --> 00:12:35.760 +And most of the time, I kind of think of these as I kind of have these different categories of these kinds of projects that I think of. + +00:12:36.500 --> 00:12:36.900 +Mm-hmm. + +00:12:43.520 --> 00:12:46.640 +I lost in my notes what I called them, but they are there. + +00:12:46.780 --> 00:12:50.480 +You have like a one is like a virtual research environment. + +00:12:51.460 --> 00:12:57.560 +So the focus is this is this is a platform that we're building for the research to be done. + +00:12:58.080 --> 00:13:05.720 +Like the reason the research should be done in like a web app would be because you have access to visualization, to Postgres, + +00:13:06.740 --> 00:13:12.620 +to pandas so you we can we can kind of build up this platform to do the actual research on + +00:13:13.040 --> 00:13:19.220 +and some of the data entry all right so like a full-on research application yeah and i guess + +00:13:19.240 --> 00:13:27.920 +you can also kind of see your work through the different stages of research projects and academic + +00:13:28.100 --> 00:13:34.479 +research and so on and we'll get to maybe end of life in a sense uh further down down in the + +00:13:34.500 --> 00:13:41.500 +conversation. But so this would be, we have a grant or we just work here and we're going to work on + +00:13:41.920 --> 00:13:47.260 +some form of research. What do you give them? Right? Like, I think that's a super interesting + +00:13:47.560 --> 00:13:55.140 +challenge because one of the real common answers would be Jupyter, JupyterLab, Marimo, whatever. + +00:13:56.420 --> 00:14:03.940 +But that's still pretty code heavy for people who are possibly philosophers or something, you know, + +00:14:04.420 --> 00:14:16.220 +Oh, exactly. That's why in digital humanities, I won't even, maybe I won't even attempt to define it in any narrow sense, because I'll get in trouble with somebody. + +00:14:17.640 --> 00:14:33.700 +But you have two groups that are interfacing with each other. And one is digital humanities as a field, like as a subfield, all of its own. And these are people who have humanities domain, like knowledge, and technical skills. + +00:14:34.140 --> 00:14:35.140 +and they're bringing them together. + +00:14:35.720 --> 00:14:38.200 +And in a lot of cases, the audience for that kind of work + +00:14:38.540 --> 00:14:41.060 +is other people working in the digital humanities. + +00:14:42.320 --> 00:14:45.060 +But far more common, and this is what we work with, + +00:14:45.240 --> 00:14:49.280 +is people who have humanities domain expertise + +00:14:49.820 --> 00:14:52.840 +and they want to publish or do research or share + +00:14:53.040 --> 00:14:57.060 +with other people who have that same humanities domain expertise + +00:14:57.560 --> 00:15:01.400 +and they are now interested in adding a technical component to it. + +00:15:01.640 --> 00:15:03.920 +How can we supercharge what they have? + +00:15:06.360 --> 00:15:08.280 +Maybe just take a moment and speak to, + +00:15:10.140 --> 00:15:14.640 +maybe I don't know if this venue will actually speak directly to anybody who I was imagining here, + +00:15:14.760 --> 00:15:16.520 +but people who work with folks, + +00:15:17.380 --> 00:15:23.560 +what would you tell somebody who works with a group who have some technical skill, + +00:15:23.700 --> 00:15:25.920 +who could create some of these things that we're going to talk about, + +00:15:26.600 --> 00:15:30.960 +but the people who they agree for don't necessarily think they need it or know that they need it. + +00:15:31.000 --> 00:15:38.460 +Like, you know, I've gone often on rants about how programming is a superpower, not a replacement for your job, right? + +00:15:41.880 --> 00:15:51.240 +Yeah, that's a problem for a lot of people, especially because you might use some new computer tools to supercharge your research. + +00:15:51.980 --> 00:15:59.200 +But the article that you publish or the research output of that, the audience, they may not be interested in hearing about that at all. + +00:15:59.820 --> 00:16:10.380 +And so for most people who are working in this space, the tools, you have to use them in such a way that you can talk about the research output without talking about the tool. + +00:16:11.360 --> 00:16:16.700 +And we have other venues to talk about the tools themselves, like the Journal for Open Source Software. + +00:16:17.180 --> 00:16:19.100 +And you can kind of get some of it out there. + +00:16:20.200 --> 00:16:26.500 +But that's the significant challenge is convincing people that it could be useful and then convincing + +00:16:26.560 --> 00:16:31.620 +the audience that they should be interested in the methods behind how some of the new research + +00:16:31.700 --> 00:16:32.080 +comes out. + +00:16:32.980 --> 00:16:38.780 +Also, I think I'm a big believer that presenting stuff in the right order is really, really + +00:16:39.180 --> 00:16:39.320 +important. + +00:16:40.100 --> 00:16:44.240 +If you present your research and it's beautiful and powerful, and oh, look, we've also, by + +00:16:44.320 --> 00:16:47.600 +the way, covered 100 times more data than any prior research. + +00:16:48.160 --> 00:16:48.400 +Surprise. + +00:16:48.520 --> 00:16:49.420 +I wonder how I did that. + +00:16:50.120 --> 00:16:51.300 +And then people are like, this is amazing. + +00:16:52.640 --> 00:16:55.640 +Then after you kind of hook them with the inspiration and what's possible, + +00:16:55.740 --> 00:16:57.680 +then you're like, let me tell you about the tool. + +00:16:57.800 --> 00:16:59.000 +Don't say that's a cool tool, right? + +00:16:59.160 --> 00:17:03.080 +This is not just like geekery, like programmer, you know, + +00:17:03.320 --> 00:17:05.079 +a Charlie Brown speak, wah, wah, wah, wah, wah. + +00:17:05.319 --> 00:17:06.620 +You know, it's like, no, I'm listening. + +00:17:06.839 --> 00:17:07.420 +Tell me now. + +00:17:08.819 --> 00:17:09.280 +Yeah, exactly. + +00:17:09.680 --> 00:17:12.839 +I mean, one of the things I think that really opens people's eyes + +00:17:13.140 --> 00:17:15.579 +is a really powerful search interface. + +00:17:16.380 --> 00:17:17.860 +You have all of this research data. + +00:17:18.480 --> 00:17:22.180 +Just put it behind Elasticsearch with some really good filtering on it. + +00:17:22.560 --> 00:17:27.880 +And all of a sudden you have fast, rapid access to the data in a way you never had before. + +00:17:28.110 --> 00:17:34.440 +Like you were never scrolling through the Excel spreadsheets and finding exactly what you wanted, like you were with this new search interface. + +00:17:35.420 --> 00:17:37.640 +And that by itself is like so simple. + +00:17:37.710 --> 00:17:42.320 +We're so used to that in web development that like everything needs to have a fantastic search now. + +00:17:43.320 --> 00:17:47.900 +But so many people have their data locked behind, you know, a terrible search interface. + +00:17:48.540 --> 00:17:57.140 +Yeah. Yeah. Just a few things to sort of expose that. So this, give us a sense of what these data + +00:17:57.300 --> 00:18:02.840 +exploration web apps might look like. These are probably kind of mostly stuck to the inside, + +00:18:03.140 --> 00:18:09.160 +kind of internal to the research lab research team groups and so on. These are probably not + +00:18:09.460 --> 00:18:15.499 +that public facing, right? Almost everything we work on does end up having a public facing + +00:18:15.520 --> 00:18:24.000 +component. So maybe the research itself is done locked behind a user login that's just for the + +00:18:24.240 --> 00:18:30.500 +researchers. But then they expose that research to the public, usually with a good search interface + +00:18:31.040 --> 00:18:37.360 +and different pages for exploring their data and visualizations and things like that. So + +00:18:38.420 --> 00:18:43.200 +yeah, everything we do ends up becoming a production public web app in the end. + +00:18:44.340 --> 00:18:45.000 +- Yeah, awesome. + +00:18:47.020 --> 00:18:50.100 +And then another one of your categories you put in + +00:18:50.180 --> 00:18:52.360 +was virtual research environments, + +00:18:52.500 --> 00:18:55.180 +like data entry, publishing, authoring, collaboration. + +00:18:55.540 --> 00:18:56.020 +Tell us about that. + +00:18:57.800 --> 00:18:59.920 +- Yeah, so a good example of this maybe + +00:19:00.100 --> 00:19:01.420 +is one of the projects that, + +00:19:04.140 --> 00:19:05.320 +well, actually the best example of it + +00:19:05.380 --> 00:19:09.460 +is the project I worked on during my PhD. + +00:19:10.340 --> 00:19:12.400 +It's called Apatosaurus. + +00:19:13.740 --> 00:19:18.060 +short story behind the name is that it sounds like apparatus in textual + +00:19:18.360 --> 00:19:24.200 +criticism when you are when you are displaying and visualizing variant + +00:19:24.520 --> 00:19:29.460 +readings to a to a base text that like that that form of visualizing it is a + +00:19:29.680 --> 00:19:35.420 +critical apparatus a critical apparatus is a pretty boring website name but a + +00:19:35.540 --> 00:19:41.600 +patasaurus dinosaurs might make textual criticism sound fun yeah I do love + +00:19:41.540 --> 00:19:47.040 +dinosaurs. No, that's really cool. So this comes out as a web app. And I know you also have some, + +00:19:47.300 --> 00:19:53.440 +you talked about some desktop apps as well. Yeah, yeah, that's right. So yeah, so there's + +00:19:53.560 --> 00:19:57.460 +people upload their collation to this, and then they can visualize it. And + +00:20:01.020 --> 00:20:07.280 +like there's a public component of this as well. But really the back end is editing a collation, + +00:20:07.720 --> 00:20:10.040 +adding notes to all of the different readings and stuff. + +00:20:10.740 --> 00:20:15.320 +So I could show what the back end looks like, + +00:20:15.520 --> 00:20:16.740 +but we can also move on. + +00:20:19.840 --> 00:20:24.480 +Let's move on, just because most people will not tell you. + +00:20:24.530 --> 00:20:31.120 +But just give us a sense of what do you create for people + +00:20:31.300 --> 00:20:34.040 +so that they're like, yeah, I can use this app. + +00:20:34.420 --> 00:20:36.500 +Give us a sense of some of the features, I guess, + +00:20:36.680 --> 00:20:43.740 +what I'm getting to yeah so another another good example is we have what we + +00:20:43.740 --> 00:20:50.460 +have project at Darth at Harvard called mapping color in history and this is a + +00:20:52.719 --> 00:20:59.040 +collaboration with a lab this lab brings in pieces of artwork and they do like + +00:20:59.120 --> 00:21:04.400 +spectral analysis on the pigments so they can identify like what was used to + +00:21:04.360 --> 00:21:10.920 +make a particular color of this red or what was made to make this color of blue. + +00:21:11.910 --> 00:21:17.160 +And then the idea is tracking how did people make those pigments over time, + +00:21:17.960 --> 00:21:23.500 +over time, and specifically in Asian art. + +00:21:24.180 --> 00:21:25.780 +Is this the Dharmra of Puna? + +00:21:27.720 --> 00:21:31.400 +No, this is mapping color in history. + +00:21:31.430 --> 00:21:33.140 +I don't think it's up here. + +00:21:33.230 --> 00:21:33.700 +Sorry about that. + +00:21:34.040 --> 00:21:40.620 +somewhere. That's all right. I'll find it. Keep talking. Okay. So, so the front end is great, + +00:21:40.900 --> 00:21:46.220 +you know, the, like the public end, this is, people can, can explore by pigments and then + +00:21:46.500 --> 00:21:51.940 +see the images that, that contain those pigments. Now in the back end, what the research, + +00:21:52.480 --> 00:22:00.699 +researchers will be able to do is correlate exactly which point of a painting the analysis was done + +00:22:00.720 --> 00:22:04.420 +that. So they have like this deep zoom image viewer where they'll zoom in and + +00:22:04.480 --> 00:22:10.320 +they'll select the point where that was taken from. And so, you know, how else + +00:22:10.440 --> 00:22:15.760 +would you do that other than a digital interface to indicate on an image of a + +00:22:15.820 --> 00:22:20.120 +painting where that spectral analysis was performed? + +00:22:21.580 --> 00:22:24.700 +Sounds almost like astronomy in a weird way. + +00:22:24.720 --> 00:22:24.960 +Oh yeah. + +00:22:27.180 --> 00:22:32.740 +We zoomed into here and we took a different spectrum of the painting and + +00:22:32.780 --> 00:22:36.280 +we realized that it's actually identical to this, something crazy like that, right? + +00:22:36.650 --> 00:22:37.320 +Yeah, yeah. + +00:22:39.840 --> 00:22:40.080 +Nice. + +00:22:40.220 --> 00:22:42.820 +That's right, yeah, so it's essentially a pigments database. + +00:22:45.320 --> 00:22:46.360 +Yeah, wild. + +00:22:47.960 --> 00:22:54.039 +So the third category of these digital humanities projects that you put down was + +00:22:54.060 --> 00:22:56.060 +like data extraction, transformation. + +00:22:58.740 --> 00:23:00.480 +In data science, they often say, + +00:23:00.920 --> 00:23:03.080 +80% of the work is the data wrangling, + +00:23:03.280 --> 00:23:05.220 +which is like cleaning, organization, + +00:23:05.960 --> 00:23:07.320 +just getting it so you could possibly + +00:23:07.500 --> 00:23:09.040 +start asking questions about it. + +00:23:09.460 --> 00:23:10.600 +I'm sure you all do a lot of that. + +00:23:11.240 --> 00:23:11.800 +- Absolutely. + +00:23:13.500 --> 00:23:17.600 +So often the very beginning of a project + +00:23:18.020 --> 00:23:23.120 +might be an Excel sheet or several spreadsheets. + +00:23:24.380 --> 00:23:28.900 +And the first task is to ingest these into a proper database. + +00:23:29.580 --> 00:23:31.460 +Not so much MongoDB for us. + +00:23:31.820 --> 00:23:34.420 +It's going into Postgres or Django Shop. + +00:23:35.960 --> 00:23:37.120 +So it's going into Postgres. + +00:23:38.640 --> 00:23:44.280 +And yeah, no, that is probably the number one challenge of the early stage + +00:23:45.040 --> 00:23:47.800 +is figuring out what the right data model is, + +00:23:48.260 --> 00:23:51.700 +what the right relationships are to model the data. + +00:23:52.320 --> 00:24:03.360 +Doing that work is advantageous to everybody because, you know, it helps both the researchers who brought the data to think about it in a more organized way. + +00:24:03.810 --> 00:24:06.100 +I mean, they've been trying to do that and they have the spreadsheets. + +00:24:06.980 --> 00:24:14.860 +But now we're modeling out the data so that we can add it to database tables and then to use later. + +00:24:14.970 --> 00:24:16.400 +So that works out well for everybody. + +00:24:17.120 --> 00:24:21.200 +And yeah, absolutely cleaning the data, getting dates, + +00:24:21.460 --> 00:24:26.340 +working with fuzzy dates, being able to parse July of 2020 + +00:24:27.700 --> 00:24:31.680 +or summer of 2020 and handling kind of all of those cases + +00:24:31.750 --> 00:24:33.840 +so that we do get dates in the end. + +00:24:35.240 --> 00:24:38.800 +One of the crazy stories from data parsing history + +00:24:39.060 --> 00:24:40.980 +is one of the-- + +00:24:41.540 --> 00:24:43.040 +I can't remember exactly what it was. + +00:24:44.720 --> 00:24:47.800 +We talked about biology tools or genetics tools earlier. + +00:24:47.880 --> 00:24:50.520 +One of the groups that names genes + +00:24:50.940 --> 00:24:52.940 +had to change the name of a gene + +00:24:53.140 --> 00:24:55.560 +because it kept getting parsed by Excel into a date. + +00:24:56.980 --> 00:24:57.760 +- I remember that. + +00:24:57.840 --> 00:24:58.940 +I remember that. - So crazy. + +00:24:59.640 --> 00:25:00.040 +- Yes. + +00:25:00.660 --> 00:25:02.440 +- So like these are the weird edge cases + +00:25:02.640 --> 00:25:03.460 +I'm sure you run into. + +00:25:05.200 --> 00:25:06.440 +Like it's not even supposed to be a date. + +00:25:06.580 --> 00:25:07.240 +Why is this a date? + +00:25:07.420 --> 00:25:08.640 +I don't know why is it a date? + +00:25:08.720 --> 00:25:09.540 +Help me out here. + +00:25:10.380 --> 00:25:11.160 +The code keeps crashing. + +00:25:11.420 --> 00:25:13.820 +Like pandas parsed it as a date and it's not or whatever. + +00:25:15.600 --> 00:25:16.360 +Yeah, absolutely. + +00:25:16.680 --> 00:25:16.820 +Yeah. + +00:25:16.940 --> 00:25:17.120 +Yeah. + +00:25:17.260 --> 00:25:22.200 +So yeah, usually lots of test suites around that ingest process until we've got it. + +00:25:22.420 --> 00:25:28.100 +Now, once we've got it in, usually the research is ongoing and then we're able to provide them + +00:25:28.220 --> 00:25:33.800 +now a new cleaned interface to, to do the additional data entry as the project is going. + +00:25:34.280 --> 00:25:35.660 +And that's usually a win-win for everybody. + +00:25:36.200 --> 00:25:36.520 +Sure. + +00:25:36.760 --> 00:25:43.380 +And so this sort of ETL ingestion side of everything is it's like, don't worry. + +00:25:43.760 --> 00:25:47.300 +Darth has got it for you. And then we'll provide you like a + +00:25:47.960 --> 00:25:51.620 +database connection to start working or do you give them the + +00:25:51.780 --> 00:25:55.160 +tools and then they kind of iterate on them? And how much? + +00:25:55.790 --> 00:25:58.340 +How much is this you and how much is this you providing like + +00:25:58.400 --> 00:26:00.680 +CLI tools and stuff or notebooks over to people? + +00:26:02.580 --> 00:26:06.660 +Our the I'd say most of the people that we're working with + +00:26:06.820 --> 00:26:10.620 +are aware of the technical tools, but they don't want a + +00:26:10.480 --> 00:26:15.640 +database connection. So we are giving them, we're doing the ingest and then building a platform + +00:26:15.990 --> 00:26:21.980 +where they can begin interacting with their data. Yeah, I'm sure they don't want one. + +00:26:23.470 --> 00:26:28.420 +Maybe you give them an app though, right, with like Elasticsearch and other things that they can... + +00:26:28.470 --> 00:26:31.020 +No, absolutely. Yeah, that's what we do. Yeah, okay. + +00:26:31.370 --> 00:26:36.860 +Yeah, we give them a web platform to begin exploring, to begin publishing. + +00:26:38.960 --> 00:26:46.420 +So I was thinking that you said you're a Django shop, which is cool. + +00:26:47.820 --> 00:26:51.760 +It sounds, though, to me like describing what you're doing, just imagining how this is. + +00:26:52.400 --> 00:26:55.300 +You're probably creating these projects often. + +00:26:56.720 --> 00:27:00.460 +How often does one of these projects actually last? + +00:27:02.240 --> 00:27:04.240 +Or how many of them do you iterate? + +00:27:05.740 --> 00:27:06.440 +I'm trying to get a sense. + +00:27:06.680 --> 00:27:09.680 +Do you work on stuff for a year or is it like every two weeks we're on a new project? + +00:27:11.100 --> 00:27:21.920 +It's why I think of us as like an agency because we get to work on greenfield projects fairly often, like you're imagining, which would not be the case normally at a big university IT department. + +00:27:23.420 --> 00:27:28.860 +So, you know, maybe two or three projects a year, two or three big ones a year. + +00:27:29.510 --> 00:27:35.960 +And then we have to put to bed a few a year as well because these things, they're funded with grant money. + +00:27:36.440 --> 00:27:38.940 +and then the grant money runs out, and it's time. + +00:27:39.180 --> 00:27:41.280 +And then we have to figure out, what do we do with it now? + +00:27:41.340 --> 00:27:46.620 +We don't want to lose the data and this way of presenting it. + +00:27:46.680 --> 00:27:48.700 +But we can't keep paying for Elasticsearch. + +00:27:49.560 --> 00:27:50.200 +Yeah, of course. + +00:27:50.800 --> 00:27:51.680 +Certainly, we're going to dive into that. + +00:27:53.020 --> 00:27:54.360 +But let's save that for the end. + +00:27:54.400 --> 00:27:57.500 +It seems like that's the arc of the story of these things. + +00:27:57.600 --> 00:28:02.400 +But I certainly think it's something that you don't think about that much, right? + +00:28:02.680 --> 00:28:06.240 +Like you said, it was only $100 a month for this, and we got a big grant. + +00:28:06.380 --> 00:28:07.180 +There's a bunch of, no big deal. + +00:28:07.210 --> 00:28:08.480 +But like when the grant's out, + +00:28:08.680 --> 00:28:10.160 +who's on the hook for a hundred dollars a month + +00:28:11.240 --> 00:28:13.260 +and making sure it survives upgrades + +00:28:13.700 --> 00:28:15.260 +and all that kind of business. + +00:28:16.660 --> 00:28:17.080 +No, that's right. + +00:28:17.700 --> 00:28:17.780 +Yeah. + +00:28:17.970 --> 00:28:21.240 +So my original question when I started on this path + +00:28:21.400 --> 00:28:22.660 +was thinking like, do you, + +00:28:25.380 --> 00:28:26.560 +how do you get started on these? + +00:28:26.560 --> 00:28:27.900 +Do you have like a big framework + +00:28:28.760 --> 00:28:30.440 +or a cookie cutter sort of thing or something? + +00:28:30.530 --> 00:28:31.700 +Like this is how we do it + +00:28:31.810 --> 00:28:33.520 +because it plugs into all this other automation + +00:28:33.750 --> 00:28:35.920 +and tools we built for the last 10 projects. + +00:28:36.400 --> 00:28:37.700 +You know, that's kind of a unique position. + +00:28:39.400 --> 00:28:47.420 +A lot of companies build one website for themselves and that's their app or they're an agency that goes across so much variation they can't do that kind of stuff, right? + +00:28:48.680 --> 00:28:49.060 +That's right. + +00:28:49.620 --> 00:28:50.000 +That's right. + +00:28:51.640 --> 00:28:52.240 +That's a good question. + +00:28:52.840 --> 00:28:56.300 +We have things that we reuse. + +00:28:56.540 --> 00:29:06.080 +Some of them are open source, different, you know, like search components and things that we maintain that we'll use across projects. + +00:29:06.570 --> 00:29:09.900 +And we have tried to do the cookie cutter Django project. + +00:29:10.720 --> 00:29:13.200 +The truth is each project is different enough. + +00:29:13.640 --> 00:29:13.980 +Yeah. + +00:29:14.440 --> 00:29:24.120 +Really, we like to evaluate it from first principles as we're evaluating it and thinking, what is the best technology to use? + +00:29:26.920 --> 00:29:32.280 +Yeah, so we don't have a cookie cutter. + +00:29:32.320 --> 00:29:38.520 +We don't have a kind of a meta framework for bootstrapping them because they're sufficiently different from each other. + +00:29:40.639 --> 00:29:41.680 +I find that too. + +00:29:42.820 --> 00:29:47.780 +The idea of how we could just grab this cookie cutter or copier. + +00:29:47.880 --> 00:29:48.880 +Are you familiar with copier? + +00:29:49.320 --> 00:29:50.740 +People out there might be familiar with that. + +00:29:50.840 --> 00:29:56.360 +It's a little bit like cookie cutter with the bonus that you can update it later + +00:29:56.800 --> 00:29:58.220 +if you change your mind about something, + +00:29:58.300 --> 00:30:01.840 +like actually change this project to use Postgres rather than SQLite or something, + +00:30:03.800 --> 00:30:04.480 +which is pretty cool. + +00:30:04.640 --> 00:30:08.759 +But every time that I do, every time I try to work with one of those projects, + +00:30:09.600 --> 00:30:11.900 +even ones that I've created for myself, I'm not hating on anyone. + +00:30:12.380 --> 00:30:16.660 +I'm like, oh, it's like 75% awesome and 25% I just got to take this stuff out. + +00:30:17.720 --> 00:30:19.600 +You know, I'll just do it from scratch. + +00:30:19.700 --> 00:30:20.700 +It's not how hard is this? + +00:30:20.780 --> 00:30:23.240 +I'll just create a few folders and put a few things in there, + +00:30:23.740 --> 00:30:27.040 +and I'll copy the PyProject.tom, like the one thing that's like, + +00:30:27.120 --> 00:30:27.900 +how do I do this again? + +00:30:27.940 --> 00:30:29.620 +I'll just copy that, and we're good to go. + +00:30:33.880 --> 00:30:35.460 +Yeah, I mean, that's what I find. + +00:30:35.660 --> 00:30:36.220 +That's what I find. + +00:30:36.440 --> 00:30:40.080 +I find it seems like a really brilliant idea, but in practice, + +00:30:41.380 --> 00:30:42.920 +it hasn't saved us time yet. + +00:30:44.080 --> 00:30:45.480 +No, I mean, maybe it's a case study. + +00:30:45.620 --> 00:30:47.480 +Like, okay, let's see what they're doing for this one. + +00:30:47.520 --> 00:30:49.360 +Oh, that is interesting how they're integrating + +00:30:49.780 --> 00:30:52.580 +this other thing maybe, but as a true foundation, + +00:30:52.720 --> 00:30:55.060 +I find it in theory awesome. + +00:30:55.480 --> 00:30:58.080 +In practice, I just end up not doing it for various reasons. + +00:30:58.440 --> 00:30:58.780 +Don't know why. + +00:31:04.720 --> 00:31:05.680 +I'm going to save this for later. + +00:31:06.460 --> 00:31:07.820 +Because the question I'm about to ask you + +00:31:07.780 --> 00:31:10.760 +is going to send us just down a rat hole. + +00:31:11.460 --> 00:31:17.300 +So instead, before we go down the rat hole, maybe we could, not that one, maybe we could + +00:31:17.380 --> 00:31:23.340 +talk about, I mean, you talked about something, but let's maybe just feature some of the projects + +00:31:23.520 --> 00:31:25.520 +that are maybe more well-known that you guys have done. + +00:31:26.380 --> 00:31:26.480 +Sure. + +00:31:27.060 --> 00:31:27.420 +Yeah, good. + +00:31:28.020 --> 00:31:31.600 +So yeah, one of them is called the amendments project. + +00:31:33.519 --> 00:31:37.380 +And this is, I didn't know this until I started working on this project. + +00:31:37.600 --> 00:31:45.260 +that there have been thousands of, I think it's at least 22,000 proposed amendments + +00:31:45.930 --> 00:31:49.600 +to the United States Constitution that never went anywhere. + +00:31:50.530 --> 00:31:55.840 +And so kind of the goal of this project is to show that there have been lots of attempts + +00:31:56.660 --> 00:32:01.140 +to amend the Constitution, but actually the Constitution is frozen. + +00:32:01.990 --> 00:32:06.940 +I mean, it's not actually amendable anymore, at least not in the politics of any time recently. + +00:32:07.740 --> 00:32:16.080 +yeah yeah it's so this is a database i cannot imagine a situation where the u.s constitution + +00:32:16.400 --> 00:32:22.000 +gets admitted it has to be unanimous across all the states right is that right if i can't remember + +00:32:22.160 --> 00:32:25.400 +but it's been a long time i remember off the top of my head it has to be unanimous but it certainly + +00:32:25.580 --> 00:32:30.519 +has to be it's got to be lines yeah it's got to be pretty darn close if it's not it all + +00:32:30.540 --> 00:32:36.480 +Alstair in. Yeah, it's like, you know, time travel, or the speed of your travels be light, + +00:32:36.960 --> 00:32:39.200 +could be theoretically possible, probably not going to happen. + +00:32:40.840 --> 00:32:44.640 +No, it's hard to see. It's hard to see. Yeah. So this is from a historian, + +00:32:45.700 --> 00:32:51.560 +historian at Harvard. Interesting. Okay. So, so it's a database of all of and the full text + +00:32:52.740 --> 00:33:00.180 +from all of these amendments. And, you know, it's, it's from the public's point of view, + +00:33:00.480 --> 00:33:12.340 +It's a Postgres full text vector search interface for finding and filtering through on all of the different amendments that have been proposed. + +00:33:13.980 --> 00:33:14.200 +Awesome. + +00:33:14.480 --> 00:33:14.660 +Okay. + +00:33:18.080 --> 00:33:18.640 +Let's see. + +00:33:20.660 --> 00:33:21.220 +I love it. + +00:33:23.600 --> 00:33:24.700 +Yeah, this is a nice looking site. + +00:33:26.000 --> 00:33:27.240 +We work with a designer. + +00:33:27.860 --> 00:33:28.280 +Okay. + +00:33:29.060 --> 00:33:29.240 +Okay. + +00:33:29.240 --> 00:33:29.320 +Yeah. + +00:33:29.400 --> 00:33:31.800 +Yeah. Of course, like an agency would, right? + +00:33:32.580 --> 00:33:33.040 +Yep. Yep. + +00:33:36.840 --> 00:33:42.380 +Nice. So we'll get a really pretty rich search interface and then off you go. + +00:33:42.660 --> 00:33:44.280 +I have no idea even what I would search for. + +00:33:45.720 --> 00:33:47.460 +Yeah. Well, you can always search for something religious, + +00:33:47.900 --> 00:33:50.600 +something abortion related. There's going to be lots of things there. + +00:33:50.860 --> 00:33:53.520 +I thought all those also like guns, but like, I don't want to go down. + +00:33:53.640 --> 00:33:56.020 +I'm not sure I even want to go down that route. + +00:33:57.020 --> 00:33:58.320 +Awesome. This looks super useful. + +00:33:59.440 --> 00:34:02.940 +Maybe someday we'll have a functional government again. + +00:34:03.200 --> 00:34:03.460 +We'll see. + +00:34:05.400 --> 00:34:06.120 +Let's change it. + +00:34:06.520 --> 00:34:07.780 +Or maybe we'll go down in its folklore. + +00:34:08.610 --> 00:34:09.300 +Like, look, it used to work. + +00:34:09.300 --> 00:34:09.460 +All right. + +00:34:10.190 --> 00:34:10.520 +So, yeah. + +00:34:10.530 --> 00:34:21.440 +So another really great project, at least from a content point of view, that's interesting, the research that it's doing, is the Finn Folklore Database. + +00:34:23.419 --> 00:34:39.940 +So in Celtic storytelling, you know, moms have been telling stories to daughters and people have been telling stories for a very long time, hundreds or a thousand years. + +00:34:42.280 --> 00:34:49.540 +Finn McCommel who is a hero a hero from Irish mythology some of it some of it + +00:34:49.700 --> 00:34:56.820 +based in you know historical events but it goes back it goes back so far so there + +00:34:56.919 --> 00:35:03.020 +are there's many hundreds or thousands of of these stories that have been spread + +00:35:03.160 --> 00:35:06.640 +and versions of these stories that have that have been told and so some of them + +00:35:06.540 --> 00:35:12.220 +are audio recordings where somebody like some research researcher has has gone out to an island + +00:35:12.250 --> 00:35:18.180 +off the coast of scotland and recorded somebody telling you know their version of the hero of finn + +00:35:18.360 --> 00:35:25.000 +and his band of you know his band of heroes you know they defend scotland and ireland from from + +00:35:25.480 --> 00:35:31.959 +invaders and attackers uh very exciting uh stories and stuff and a team of a team of characters + +00:35:33.580 --> 00:35:37.440 +So there's audio recordings and then there's documents, + +00:35:37.830 --> 00:35:39.920 +like written documents that contain these. + +00:35:39.920 --> 00:35:44.280 +And so this is a database of kind of all of those all in one place with, + +00:35:45.550 --> 00:35:51.980 +on the public side, a nice search interface for discovering them, + +00:35:52.330 --> 00:35:54.700 +you know, either using the map view or searching. + +00:35:55.340 --> 00:35:56.140 +Yeah, that's cool. + +00:35:56.540 --> 00:35:59.320 +I got my map view for some random thing I searched about here. + +00:36:00.380 --> 00:36:00.820 +Amazing. + +00:36:02.980 --> 00:36:06.720 +Yeah, but this is pretty interesting, all these different tellings and stuff. + +00:36:07.610 --> 00:36:14.020 +Oh, and yeah, one of the big challenges with this project is that it's fully internationalized. + +00:36:14.330 --> 00:36:16.000 +So it's available in English. + +00:36:16.220 --> 00:36:21.040 +Everything is available in English, Scottish Gaelic, and Irish Gaelic. + +00:36:21.370 --> 00:36:22.960 +But that extends into the database. + +00:36:23.330 --> 00:36:26.900 +So usually people have multiple names recorded for them. + +00:36:28.180 --> 00:36:33.100 +And so, yeah, you may have one person with any number of names in different languages, + +00:36:33.260 --> 00:36:36.260 +sometimes more than one Scottish name, that kind of thing. + +00:36:36.280 --> 00:36:43.180 +And so the data model on this one is quite messy, but sensible. + +00:36:44.400 --> 00:36:47.100 +But yeah, it's quite a lot of different kinds of data to wrangle. + +00:36:47.320 --> 00:36:49.220 +And then with all of the translations for each thing. + +00:36:49.960 --> 00:36:50.460 +Yeah, that's wild. + +00:36:50.540 --> 00:36:56.520 +It's not just, we need the user interface of this thing to translate about. + +00:36:57.920 --> 00:36:58.720 +That's way more, right? + +00:36:59.420 --> 00:37:04.180 +Right. Yeah. Yeah. It is that. It is that. And then it is also, yes, all the items in the database + +00:37:04.820 --> 00:37:06.380 +have a translation or can. + +00:37:06.450 --> 00:37:11.940 +Okay. Yeah. I can see how that makes a lot of sense. You want to work in the native language + +00:37:12.460 --> 00:37:15.180 +of the people who did that part of the folklore or whatever, right? + +00:37:17.520 --> 00:37:21.280 +Yeah. Well, and people are still speaking those languages. So people who would use this + +00:37:21.700 --> 00:37:25.499 +to, you know, like somebody may have heard a story from their mom or dad + +00:37:25.680 --> 00:37:28.000 +and now would like to find other versions of that story. + +00:37:28.430 --> 00:37:32.200 +And they live in a part of Scotland where they speak Scottish Gaelic as their first language. + +00:37:32.760 --> 00:37:33.700 +Right, right. Very cool. + +00:37:33.700 --> 00:37:34.540 +They can still access the site. + +00:37:35.230 --> 00:37:39.960 +Yeah. And then that mapping color history one, that's another one of the public ones that you + +00:37:40.740 --> 00:37:41.940 +said is pretty major. + +00:37:43.620 --> 00:37:51.339 +Yeah, that's right. Yeah. So yeah, that's a pigments database. You can search by either + +00:37:51.360 --> 00:37:57.520 +English color names like blue and find all of these Asian paintings that have blue or a particular kind + +00:37:57.520 --> 00:38:05.440 +of pigment of how they made the blue. Yeah, nice. So what's the open source story? You're creating + +00:38:05.440 --> 00:38:11.740 +all these apps, maybe some of these frameworks, there's got to be some tools. Is there a big + +00:38:12.540 --> 00:38:19.579 +desire or already an effort to have a lot of these things open source or is it too niche or is it just + +00:38:19.600 --> 00:38:26.560 +like this is the advantage of harvard has is other universities don't get this uh no it's it's + +00:38:26.820 --> 00:38:32.720 +something we talk about quite a bit um usually these things start usually they start closed source + +00:38:33.100 --> 00:38:38.460 +um during development and then we and then we work with the faculty and we talk about um + +00:38:39.200 --> 00:38:45.099 +how we can take you know like the repo for the for the web app how we can take that um public + +00:38:46.000 --> 00:38:48.640 +And so we've done that for a number of projects. + +00:38:48.760 --> 00:38:49.620 +Not all of them are. + +00:38:50.720 --> 00:38:54.420 +But the ideal is that they all make their way into the open, + +00:38:55.440 --> 00:38:57.220 +and especially when they become archived. + +00:38:58.580 --> 00:38:58.740 +Sure. + +00:38:59.220 --> 00:39:01.500 +Yeah, that's a good way to help them live on. + +00:39:02.320 --> 00:39:08.800 +And they might even go into GitHub's Arctic Vault, which is crazy. + +00:39:08.840 --> 00:39:10.180 +I don't know if people know about that out there, + +00:39:10.360 --> 00:39:17.240 +but GitHub has quite a while ago started taking copies of all of the repos and putting them, + +00:39:18.200 --> 00:39:20.780 +backing them up and storing them in the Arctic vault. + +00:39:21.700 --> 00:39:22.140 +It's kind of cool. + +00:39:23.150 --> 00:39:26.280 +I really, really, really hope we never need that, but it's kind of neat. + +00:39:26.920 --> 00:39:27.380 +Yeah, me too. + +00:39:28.380 --> 00:39:31.140 +Usually universities have their own archival system, + +00:39:31.440 --> 00:39:38.680 +so any important research data is usually part of that system as well. + +00:39:39.080 --> 00:39:39.460 +I see. + +00:39:39.590 --> 00:39:39.700 +Okay. + +00:39:39.930 --> 00:39:40.060 +Yeah. + +00:39:40.840 --> 00:39:48.960 +obviously right like i'm i can't remember where it was it was somewhere i think it was south korea + +00:39:49.040 --> 00:39:55.180 +or taiwan where like seven years of government data got lost or something like that it was really + +00:39:55.340 --> 00:39:59.280 +really bad recently there was a fire and i think they had backups but maybe just into the building + +00:39:59.750 --> 00:40:02.979 +you know like we'll put that out we'll back it up to the hard drive over here + +00:40:05.920 --> 00:40:06.280 +Not good. + +00:40:06.880 --> 00:40:07.280 +No. + +00:40:09.180 --> 00:40:09.620 +Not good. + +00:40:09.640 --> 00:40:11.060 +You definitely want this stuff to survive. + +00:40:11.060 --> 00:40:14.460 +I mean, academia has this history of, like, + +00:40:15.520 --> 00:40:17.040 +tomes that have survived the past + +00:40:17.680 --> 00:40:21.500 +and really, really long-lived information, right? + +00:40:23.400 --> 00:40:26.160 +Besides the Library of Alexandria or something like that, maybe. + +00:40:26.680 --> 00:40:27.460 +That's what we want. + +00:40:27.700 --> 00:40:28.160 +That's what we want. + +00:40:28.220 --> 00:40:30.400 +We want it to, yeah, we want it to last. + +00:40:32.020 --> 00:40:32.300 +Absolutely. + +00:40:32.680 --> 00:40:37.820 +So maybe that's a good time to sort of talk about the trailing end. + +00:40:37.820 --> 00:40:40.300 +I think there's a lot of interesting things going on here. + +00:40:43.180 --> 00:40:49.840 +Just like you've run out of money, not because you actually run out of money. + +00:40:50.380 --> 00:40:53.620 +The grant is done, and you've either spent or given back or whatever + +00:40:53.810 --> 00:40:55.580 +with the remaining little bits of money. + +00:40:56.800 --> 00:40:58.260 +It's always a weird balance with research. + +00:40:58.620 --> 00:41:01.800 +It's like, oh, we got $3,000 left on this research grant. + +00:41:01.900 --> 00:41:02.780 +What are we going to do with it? + +00:41:02.840 --> 00:41:03.900 +It's not like, well, we're going to give it back. + +00:41:04.040 --> 00:41:04.720 +We just didn't need it. + +00:41:04.960 --> 00:41:09.380 +It's like, we're going to find a way to like fund a student to do a little more work or + +00:41:09.460 --> 00:41:09.540 +whatever. + +00:41:09.760 --> 00:41:11.880 +But eventually the grant is over. + +00:41:14.440 --> 00:41:18.380 +Then you've got some expensive app access to a big database because it needs a big search + +00:41:18.640 --> 00:41:20.740 +or a lot of compute or something. + +00:41:21.760 --> 00:41:22.040 +That's right. + +00:41:22.460 --> 00:41:27.720 +Everything during like, I mean, anything, anything that's a, that's a Django app. + +00:41:28.520 --> 00:41:38.600 +We deploy to AWS using containers, which isn't the cheapest way to host anything. + +00:41:40.699 --> 00:41:43.700 +But that's for the most part the Harvard way. + +00:41:45.680 --> 00:41:47.780 +And it is robust and is reliable. + +00:41:48.540 --> 00:41:58.460 +And we don't have a DevOps person on call on the weekend to rescue one of these people. + +00:41:58.480 --> 00:42:04.780 +apps so having having them reliable is good okay so it's so it's on AWS and + +00:42:05.120 --> 00:42:09.320 +paying you know paying for the containers paying for that elastic search cluster + +00:42:09.800 --> 00:42:16.640 +the art RDS Postgres database okay well even if somebody wants to start paying + +00:42:16.740 --> 00:42:19.740 +for that out-of-pocket all of those little services they add up they add up + +00:42:19.780 --> 00:42:26.359 +to enough that we need to do something when the project hits end of life and so + +00:42:26.420 --> 00:42:53.720 +So our gold standard that we've developed so far is asking, can this become a static website? Can we bake this out into all HTML files and acknowledge that there will be some trade-offs? We will trade off some searching. It's not going to have Elasticsearch. It doesn't mean that it won't have any search, though. So we'll trade out Elasticsearch, and it'll be very difficult to add new data. + +00:42:54.360 --> 00:43:20.360 +But that's okay because it's being archived. So can we get it into a static site? And that's challenging depending on how you've set it up. So we now have projects where we set them up from the beginning to be archivable like this. And one of them is called Water Stories. And it was a companion to an art installation at the Radcliffe Institute on the Harvard campus. + +00:43:21.440 --> 00:43:30.420 +And so this was this live site during the duration of the art installation where people could come in and add stories that they had about water onto an iPad. + +00:43:31.960 --> 00:43:33.200 +And then those went up to our database. + +00:43:35.500 --> 00:43:43.980 +We built that with something called Django Bakery, which if you opt in and you use all of their class-based views the way that they're meant to be used, + +00:43:45.680 --> 00:43:48.520 +then you can bake this out into static files when you're done. + +00:43:49.000 --> 00:43:49.700 +Very low effort. + +00:43:50.310 --> 00:43:50.880 +That was perfect. + +00:43:51.320 --> 00:44:15.740 +That is such a cool idea. And mad props to them for ASCII art logos. Come on now. I feel like that should be in the view source if it's not. But this is such a cool idea because you can just take a working site. You guys are at Django Shop, so a lot of your sites are written in Django and you just go, make it static, right? More or less? + +00:44:15.800 --> 00:44:20.900 +Yes. And what's really great about it is if they wanted to make a change, and they have, + +00:44:21.040 --> 00:44:24.620 +they have asked since we, since we made it static, they've asked for a couple of changes. + +00:44:25.300 --> 00:44:31.220 +So locally, I just Docker compose up this whole application, make the change in the Django admin + +00:44:31.840 --> 00:44:32.740 +and rebake the site. + +00:44:34.020 --> 00:44:34.620 +Yeah, that's cool. + +00:44:34.980 --> 00:44:35.820 +It can still be updated. + +00:44:36.540 --> 00:44:40.660 +Something, if you've never tried this, like something like, hey, can we just add one more + +00:44:40.840 --> 00:44:44.720 +menu item? And you're like, no, no, no, we're not adding the menu item because you know what that + +00:44:44.700 --> 00:44:51.180 +That means we're changing 7,300 pages because they all bake in the whole HTML, right? + +00:44:51.680 --> 00:44:52.060 +Exactly. + +00:44:52.530 --> 00:44:53.140 +Yeah, exactly. + +00:44:53.230 --> 00:44:58.060 +But if that's in my Django database, in my SQLite file, then no problem at all. + +00:44:58.570 --> 00:44:59.840 +Because then I just rebake it. + +00:45:00.270 --> 00:45:00.440 +Yeah. + +00:45:00.780 --> 00:45:02.240 +Yeah, absolutely. + +00:45:03.960 --> 00:45:07.480 +So I think this is super neat. + +00:45:07.540 --> 00:45:14.720 +There's also Frozen Flask, if I could get rid of all the ads. + +00:45:14.750 --> 00:45:16.540 +I do not need a Yeti thing, whatever that is. + +00:45:17.700 --> 00:45:20.320 +The glass, not the mythical thing. + +00:45:21.260 --> 00:45:27.680 +But Frozen Flask, which does a similar thing for Flask apps, + +00:45:27.840 --> 00:45:30.740 +if you're a Flask person, probably would work with Quart. + +00:45:31.040 --> 00:45:32.460 +Don't know for sure, but probably. + +00:45:33.600 --> 00:45:35.720 +So that's a pretty interesting idea as well. + +00:45:36.490 --> 00:45:37.060 +Throw that in there. + +00:45:38.200 --> 00:45:42.140 +But also, what else? + +00:45:42.900 --> 00:45:46.420 +Also, you talked about search, right? + +00:45:46.660 --> 00:45:49.920 +That can be such a problem. + +00:45:50.560 --> 00:45:56.040 +And I'm a huge fan of your recommendation here with PageFind. + +00:45:58.620 --> 00:45:59.760 +Tell us about PageFind. + +00:46:00.140 --> 00:46:04.299 +So this has been, I think it's been a bit of a game changer in how + +00:46:04.320 --> 00:46:10.640 +functional one of these archived sites can remain. So we're actually in the process of that amendments + +00:46:11.200 --> 00:46:18.640 +website that searches across 22,000 full texts of amendments. We are in the process of sunsetting + +00:46:18.800 --> 00:46:24.620 +that and that will become a static site. And that's for that search. We already have an internal demo + +00:46:24.840 --> 00:46:33.020 +that that proves that we can replace that Postgres full search with PageFind. You lose vector search. + +00:46:33.980 --> 00:46:38.200 +Yeah. You kind of got to get really true keyword matching. + +00:46:39.670 --> 00:46:44.820 +Yeah. Yeah, that's right. But you still get filtering. I mean, and really faceting and filtering + +00:46:45.520 --> 00:46:52.460 +is when it comes to discovery of things, I mean, I find that's really what's useful. So filtering + +00:46:52.800 --> 00:46:59.500 +these amendments by state or by the Congress that was active at the time or by the person who + +00:47:02.280 --> 00:47:09.660 +who co-wrote it um all of those are totally great in page find and the uh the keyword search is just + +00:47:09.780 --> 00:47:14.240 +fine in page find uh one of the things i really like about it is that it it takes your index + +00:47:15.300 --> 00:47:20.520 +and it chops it up into lots of little files that can just fly across the network so it's a very fast + +00:47:20.740 --> 00:47:28.140 +search um it's it's not a huge network load even if your index is initially very large and it + +00:47:28.160 --> 00:47:30.580 +It essentially cuts it up somewhat alphabetically. + +00:47:31.110 --> 00:47:36.800 +So if your search starts with T, or I should say a better word for audio, + +00:47:36.920 --> 00:47:43.400 +if it starts with W, then it will load up the index for words that start with W + +00:47:43.970 --> 00:47:46.180 +and fly that over the network instead of the whole thing. + +00:47:46.270 --> 00:47:49.400 +So it's pretty slick, and it has a great Python API. + +00:47:50.280 --> 00:47:53.960 +So to do the proof of concept for the amendments search, + +00:47:54.200 --> 00:48:00.860 +I just took a database dump and then manually indexed with a Python script into PageFind. + +00:48:01.060 --> 00:48:03.880 +Wait, there's a Python API for PageFind? + +00:48:04.820 --> 00:48:04.900 +Yeah. + +00:48:05.080 --> 00:48:13.100 +So the way PageFind works, I should have said that, is the way most people will use it is by normally PageFind consumes HTML. + +00:48:13.780 --> 00:48:16.880 +So you give it access to your dist folder. + +00:48:18.280 --> 00:48:18.880 +Oh, okay. + +00:48:20.040 --> 00:48:22.700 +And then it crawls through all of your HTML files. + +00:48:22.840 --> 00:48:31.800 +And you can do great things like adding little HTML tags that are just for PageFind that give it the filtering ability or that you want to sort by something. + +00:48:32.260 --> 00:48:33.300 +And so that's great. + +00:48:34.100 --> 00:48:41.600 +Or you can just call PageFind from Python or from TypeScript and just build that index manually. + +00:48:42.400 --> 00:48:43.200 +Well, thanks a lot, David. + +00:48:43.320 --> 00:48:44.680 +I have another thing I've got to go research. + +00:48:44.900 --> 00:48:45.380 +This is awesome. + +00:48:46.300 --> 00:48:48.060 +I'm a huge fan of PageFind, as I said. + +00:48:48.080 --> 00:48:53.540 +on my personal website, mkennedy.codes, is just a pure stat. + +00:48:53.570 --> 00:48:56.020 +It starts in Markdown and ends up in HTML. + +00:48:56.440 --> 00:48:59.100 +But if you add page find in, you get a super rich, + +00:48:59.320 --> 00:49:00.740 +if you want to just know, you want to talk about, + +00:49:00.860 --> 00:49:06.480 +like what was about Docker, it shows you really nice results, + +00:49:07.500 --> 00:49:10.620 +pulling out the different parts of the page and sections that talk about it, + +00:49:10.720 --> 00:49:12.720 +like the headers and then what is said. + +00:49:12.760 --> 00:49:17.400 +and it even does like sub word, you know, + +00:49:17.520 --> 00:49:20.300 +like if you just type doc, it finds all the words that match that. + +00:49:20.560 --> 00:49:23.520 +And what I really like about it is a couple of things is it's instant. + +00:49:23.800 --> 00:49:26.580 +It basically is like nearly instant. + +00:49:26.760 --> 00:49:29.800 +If you type a few things, it gets way faster because it's pulling down. + +00:49:29.910 --> 00:49:33.560 +And if you go and look in the network console here + +00:49:33.740 --> 00:49:35.000 +and you type something, + +00:49:35.210 --> 00:49:39.520 +you can see that it's actually pulling in these little tiny fragments, + +00:49:40.440 --> 00:49:43.120 +which this one's coming off disk cache in three milliseconds, right? + +00:49:43.240 --> 00:49:48.940 +But it breaks your index into a bunch of very small page find fragments + +00:49:50.600 --> 00:49:54.000 +that I think it starts with anything that starts with the word DO. + +00:49:54.260 --> 00:49:56.680 +These are all the prebuilt results and stuff like that, right? + +00:49:57.420 --> 00:49:58.360 +That's right. That's right. + +00:49:58.840 --> 00:49:59.820 +Yeah, that's super cool. + +00:50:00.620 --> 00:50:00.700 +Yeah. + +00:50:01.320 --> 00:50:09.740 +One of our open source projects that we maintain is a Vue.js component library for page find. + +00:50:10.120 --> 00:50:13.780 +so that we can style it and reuse it across different projects. + +00:50:15.500 --> 00:50:16.460 +Oh, that's awesome. + +00:50:16.850 --> 00:50:17.280 +I love it. + +00:50:18.300 --> 00:50:19.760 +Yeah, I think this really unlocks it. + +00:50:19.760 --> 00:50:24.980 +And when you go to so many sites, like their documentation + +00:50:25.620 --> 00:50:28.760 +or just their web app, and the search is so bad. + +00:50:29.000 --> 00:50:32.500 +You type something, and it's like thinking, spinning, spinning, + +00:50:33.020 --> 00:50:34.040 +spinning, spinning. + +00:50:34.420 --> 00:50:37.800 +And then five seconds later, it gives you kind of janky results. + +00:50:38.240 --> 00:50:39.960 +And if you just throw a page find in there, + +00:50:41.740 --> 00:50:44.520 +you can't type fast enough to outrun the results. + +00:50:44.540 --> 00:50:44.980 +You know what I mean? + +00:50:45.280 --> 00:50:45.860 +No, that's right. + +00:50:45.860 --> 00:50:46.000 +Yeah. + +00:50:46.900 --> 00:50:48.880 +Too many static site search solutions, + +00:50:49.040 --> 00:50:53.060 +they use like a JSON blob that you have to pull down + +00:50:53.700 --> 00:50:54.520 +and then iterate through. + +00:50:55.100 --> 00:50:57.060 +You know what's worse, and I see this a lot, + +00:50:57.200 --> 00:51:03.780 +would be if you go to google.com, and then you would say, + +00:51:04.400 --> 00:51:05.780 +effectively, site, colon, whatever, + +00:51:05.960 --> 00:51:06.880 +and then you search Docker. + +00:51:07.780 --> 00:51:10.040 +They basically pull that. + +00:51:11.080 --> 00:51:13.040 +They just say, search this, and you just + +00:51:13.180 --> 00:51:15.840 +get Google results for your site. + +00:51:15.930 --> 00:51:18.840 +And obviously, Google's fine, but it's just-- + +00:51:18.840 --> 00:51:20.580 +No, I find that unusable, really. + +00:51:20.720 --> 00:51:21.100 +I do, too. + +00:51:21.210 --> 00:51:22.720 +It really-- you're like, ah, jeez. + +00:51:23.500 --> 00:51:25.500 +But now I'm super excited to realize + +00:51:25.650 --> 00:51:28.500 +I can do that from my dynamic content as well. + +00:51:29.740 --> 00:51:31.580 +So with the Python integration. + +00:51:32.070 --> 00:51:32.180 +OK. + +00:51:34.000 --> 00:51:34.100 +Nice. + +00:51:36.880 --> 00:51:38.660 +What about something truly static? + +00:51:38.780 --> 00:51:41.620 +Have you looked at Hugo and some of the other type of things? + +00:51:42.100 --> 00:51:42.340 +Sure. + +00:51:42.570 --> 00:51:47.220 +So when I see you've even got the tab up for the SUMEB project, + +00:51:47.560 --> 00:51:49.040 +which is-- + +00:51:50.260 --> 00:51:57.400 +that's essentially a database of many, many specimens taken + +00:51:57.620 --> 00:51:58.540 +from the SUMEB mine. + +00:52:00.140 --> 00:52:00.580 +OK. + +00:52:01.440 --> 00:52:02.080 +Looks beautiful. + +00:52:02.920 --> 00:52:03.540 +Oh, it is. + +00:52:03.550 --> 00:52:03.660 +Yeah. + +00:52:03.770 --> 00:52:04.240 +Yeah, it is. + +00:52:04.700 --> 00:52:06.240 +So if you click on Minerals database, + +00:52:06.740 --> 00:52:07.920 +you open up that search interface + +00:52:08.100 --> 00:52:10.700 +and that's powered by PageFind. + +00:52:10.880 --> 00:52:11.840 +- Oh, this is? + +00:52:12.320 --> 00:52:12.620 +- Yes. + +00:52:16.160 --> 00:52:18.260 +I forget what I was... + +00:52:19.220 --> 00:52:19.560 +- I see. + +00:52:19.940 --> 00:52:21.620 +You've even like hooked into, + +00:52:21.980 --> 00:52:25.240 +I was thinking just like pure static, like Hugo, like... + +00:52:25.800 --> 00:52:27.200 +- Oh yes, yes, yes. + +00:52:27.460 --> 00:52:28.620 +So this is an astro site. + +00:52:29.420 --> 00:52:32.680 +So for this website, we have this as an astro site + +00:52:32.700 --> 00:52:34.080 +so that we have a little bit, + +00:52:34.260 --> 00:52:37.960 +Because with Astro, they make it so easy to pull in Vue components. + +00:52:38.720 --> 00:52:39.280 +I see. + +00:52:39.420 --> 00:52:44.180 +So our page find is a custom Vue.js component library. + +00:52:44.920 --> 00:52:46.900 +With Astro, you can use React components. + +00:52:47.000 --> 00:52:48.620 +You can use Vue components. + +00:52:48.820 --> 00:52:51.640 +But what it does is it's just a static site generator. + +00:52:51.740 --> 00:52:51.880 +Yeah. + +00:52:52.460 --> 00:52:52.560 +Yeah. + +00:52:53.780 --> 00:52:54.180 +Fantastic. + +00:52:54.820 --> 00:52:58.780 +So a little bit more designable than Hugo or something. + +00:52:59.040 --> 00:52:59.800 +Here's your markdown file. + +00:52:59.880 --> 00:53:00.420 +Good luck with that. + +00:53:00.480 --> 00:53:01.320 +Yeah, I love Hugo, though. + +00:53:01.560 --> 00:53:05.240 +Yeah, I use Hugo for different personal sites here and there. + +00:53:05.560 --> 00:53:07.780 +And it's just so fast and easy to get up and running. + +00:53:08.000 --> 00:53:08.940 +But it's great. + +00:53:08.940 --> 00:53:09.140 +I do too. + +00:53:09.620 --> 00:53:11.320 +That's what my website's written in, is in Hugo. + +00:53:13.540 --> 00:53:15.660 +But if I'm integrating with anything else, + +00:53:15.780 --> 00:53:17.120 +I used to kind of split it up. + +00:53:17.160 --> 00:53:19.300 +Like this part's Hugo and this part's a Python app. + +00:53:20.300 --> 00:53:22.300 +And it's pretty easy to get something + +00:53:22.440 --> 00:53:23.920 +that will take a bunch of Markdown files + +00:53:24.060 --> 00:53:27.160 +and just turn them into HTML and just put a page template + +00:53:27.300 --> 00:53:27.720 +around that. + +00:53:27.900 --> 00:53:31.740 +So I've kind of stepped away from mixing and matching that + +00:53:31.840 --> 00:53:32.700 +as much as I used to. + +00:53:32.940 --> 00:53:35.960 +So now if I've got a static section of a dynamic site. + +00:53:36.340 --> 00:53:40.500 +But that has nothing to do with the archival side of things, + +00:53:40.700 --> 00:53:40.860 +right? + +00:53:41.480 --> 00:53:43.780 +Because the idea is that the thing that I'm describing + +00:53:43.980 --> 00:53:44.900 +is gone on purpose. + +00:53:45.220 --> 00:53:45.620 +That's right. + +00:53:47.800 --> 00:53:48.280 +OK. + +00:53:51.560 --> 00:53:53.100 +So you've got some-- + +00:53:54.540 --> 00:53:55.760 +we've got Django Bakery. + +00:53:56.340 --> 00:54:02.780 +I threw out Frozen Flask, and I'm sure there's a ton more that neither of us are aware of at the moment. + +00:54:03.260 --> 00:54:10.800 +So Django Bakery was really good for that purpose, and we're keeping our eyes open for projects that it's a good fit for. + +00:54:11.730 --> 00:54:13.740 +But that was a pretty simple website. + +00:54:13.950 --> 00:54:16.580 +It needed a dynamic backend, but it was quite straightforward. + +00:54:17.420 --> 00:54:22.080 +And for Django Bakery, you have to opt into inheriting from their class-based views. + +00:54:22.190 --> 00:54:22.440 +I see. + +00:54:23.060 --> 00:54:25.120 +So you've got to think ahead of it. + +00:54:25.920 --> 00:54:27.260 +Yeah, absolutely. + +00:54:27.920 --> 00:54:30.080 +Hard to add retroactively, probably impossible. + +00:54:31.680 --> 00:54:34.500 +Now our other websites, like the fin example + +00:54:34.740 --> 00:54:40.080 +and the mapping color example, those are APIs. + +00:54:40.640 --> 00:54:42.940 +That's a Django API, Django REST framework for one, + +00:54:43.960 --> 00:54:45.360 +GraphQL for the other. + +00:54:46.160 --> 00:54:48.460 +One has a view front end, one has a React front end. + +00:54:48.940 --> 00:54:52.480 +Okay, well Django Bankery just isn't gonna work very well + +00:54:52.700 --> 00:54:54.080 +for like serializing JSON. + +00:54:54.700 --> 00:54:57.980 +It's like, awesome, here's your unrendered JavaScript + +00:54:58.220 --> 00:55:00.220 +front-end code and it's just gonna look empty or something. + +00:55:01.340 --> 00:55:04.360 +- Yeah, so it is a good reason to consider using + +00:55:05.230 --> 00:55:07.140 +like vanilla Django templates when possible, + +00:55:08.110 --> 00:55:11.300 +like for that reason, but those were inherited + +00:55:12.700 --> 00:55:15.580 +from the vendors, those two sites, + +00:55:15.610 --> 00:55:17.080 +and we've made a lot of progress on those. + +00:55:17.700 --> 00:55:21.020 +So, you know, what to do in that, + +00:55:21.580 --> 00:55:24.380 +Like in that situation, Django Bakery isn't an option. + +00:55:26.130 --> 00:55:28.120 +And those projects are not end of life yet. + +00:55:28.310 --> 00:55:32.080 +So we have some time, but we're, so what we're doing is strategizing. + +00:55:32.280 --> 00:55:33.740 +Okay, how will we rescue them? + +00:55:33.790 --> 00:55:39.160 +How will we keep them alive once somebody needs to stop paying for hosting? + +00:55:40.220 --> 00:55:41.600 +And we have ideas. + +00:55:41.990 --> 00:55:45.160 +We have, I think there's clever, interesting things out there. + +00:55:46.160 --> 00:55:47.400 +We'll have to keep looking into it. + +00:55:51.320 --> 00:55:55.760 +There are some pretty interesting ideas and I know that you had been thinking about them. + +00:55:56.640 --> 00:55:59.720 +What if instead of having a back-end that ran in a container, + +00:56:00.440 --> 00:56:03.440 +you could just have WebAssembly, + +00:56:04.070 --> 00:56:05.520 +but still have it go, + +00:56:05.730 --> 00:56:07.460 +just a local loopback type of thing? + +00:56:08.000 --> 00:56:14.159 +Yeah, I'm really interested in this one because it enables essentially + +00:56:14.180 --> 00:56:21.260 +the full functionality of the live site to exist as what is just a static site. + +00:56:22.440 --> 00:56:30.300 +So because of Pyodide and projects like PyScript, we can run Python in the browser. + +00:56:31.760 --> 00:56:33.620 +And we can run SQLite in the browser. + +00:56:34.240 --> 00:56:37.960 +And now we can even run Postgres in the browser with PG Lite. + +00:56:39.540 --> 00:56:44.280 +So we can run all those things in the browser, then couldn't we run, couldn't we have Django + +00:56:44.900 --> 00:56:46.620 +hosted right in the browser? + +00:56:47.500 --> 00:56:47.880 +And you can. + +00:56:48.730 --> 00:56:49.300 +You can. + +00:56:49.310 --> 00:56:54.860 +So there's a proof of concept that proves it's possible called Django WebAssembly. + +00:56:56.660 --> 00:57:02.860 +And if you load up this, if you load this up, it'll let you log in to the Django admin + +00:57:03.100 --> 00:57:04.900 +and you're not logging into anybody's backend. + +00:57:05.240 --> 00:57:11.120 +logging into your own browser where this is running in a service worker. + +00:57:11.960 --> 00:57:12.400 +Awesome. + +00:57:12.980 --> 00:57:13.380 +Look at that. + +00:57:14.200 --> 00:57:14.700 +Oh, hold on. + +00:57:14.760 --> 00:57:15.640 +I told you where the password was. + +00:57:17.280 --> 00:57:17.820 +Very secure. + +00:57:18.420 --> 00:57:18.800 +Matt? + +00:57:19.960 --> 00:57:20.400 +Password. + +00:57:20.500 --> 00:57:25.600 +Well, it can be entirely insecure because, yeah, it's running right in your own browser. + +00:57:25.920 --> 00:57:26.500 +Yeah, that's awesome. + +00:57:26.570 --> 00:57:27.980 +And here we are, Django admin. + +00:57:28.520 --> 00:57:28.940 +Incredible. + +00:57:30.220 --> 00:57:31.360 +Yeah, so I'm pretty interested in this. + +00:57:32.140 --> 00:57:38.160 +You've got to convert an RDS Postgres database into either SQLite or something like PGLite. + +00:57:38.440 --> 00:57:39.400 +But I think that's all doable. + +00:57:39.820 --> 00:57:41.800 +So I think it's an exciting possibility. + +00:57:42.160 --> 00:57:42.760 +Yeah, for sure. + +00:57:42.820 --> 00:57:49.680 +I do think, so maybe you have a rich query system that you're powering by your database + +00:57:50.020 --> 00:57:50.580 +that's really heavy. + +00:57:51.560 --> 00:57:51.940 +Exactly. + +00:57:52.240 --> 00:57:56.420 +And it's got a bunch of data that's like, here's all of our working data that you might ask + +00:57:56.580 --> 00:57:57.080 +questions about. + +00:57:57.760 --> 00:58:01.240 +Maybe you just convert that to PageFind to help you find the pieces. + +00:58:01.360 --> 00:58:03.200 +and just keep the operational data + +00:58:03.330 --> 00:58:06.000 +and maybe like even a SQLite with like the Django ORM, + +00:58:06.000 --> 00:58:09.000 +you can just switch the connection, keep talking to it. + +00:58:09.110 --> 00:58:12.780 +I mean, there's possibilities to just get something not too terrible + +00:58:12.920 --> 00:58:14.560 +that's not the same, but not that far off. + +00:58:15.560 --> 00:58:15.960 +Yeah, exactly. + +00:58:16.680 --> 00:58:19.900 +And then it goes on GitHub pages and it can live hopefully forever. + +00:58:20.050 --> 00:58:22.520 +I mean, it feels like GitHub will last forever, + +00:58:23.160 --> 00:58:25.020 +but it'll last longer than funding will anyways. + +00:58:26.400 --> 00:58:34.920 +It's definitely going to last longer than just something that we can't pay for anymore. + +00:58:36.320 --> 00:58:41.900 +I don't know how long GitHub is going to be around for, I think a while, but you never know. + +00:58:42.430 --> 00:58:43.740 +It seems like stuff is going to last forever. + +00:58:43.830 --> 00:58:46.920 +Then it gets changed. + +00:58:47.050 --> 00:58:47.740 +We had subversion. + +00:58:49.380 --> 00:58:50.880 +Now it's completely gone, right? + +00:58:51.430 --> 00:58:53.100 +Just 20 years, 15 years later. + +00:58:53.320 --> 00:58:55.440 +But still, I think 100% there. + +00:58:56.340 --> 00:58:59.120 +- Yeah, but if something ever happened, + +00:58:59.540 --> 00:59:02.360 +somebody just needs to copy that folder of HTML, + +00:59:03.200 --> 00:59:06.740 +CSS and JavaScript files and dump it into an S3 bucket + +00:59:07.400 --> 00:59:10.140 +or somewhere else and then it can continue living there. + +00:59:10.840 --> 00:59:11.900 +So it's a good option. + +00:59:12.640 --> 00:59:13.420 +- It's a great option. + +00:59:13.740 --> 00:59:14.820 +It's a really, really good option. + +00:59:14.950 --> 00:59:21.040 +I mean, I guess one of the long-term concerns might be + +00:59:22.100 --> 00:59:25.440 +what if the WebAssembly standard changes so much + +00:59:25.460 --> 00:59:32.020 +it's not supported anymore but you could probably bite wise convert it if you had to you know like + +00:59:32.140 --> 00:59:37.300 +somebody would probably be able to create one yeah that would that would be unfortunate so + +00:59:37.720 --> 00:59:43.820 +i suppose if that happens i mean if that happens yeah we're um booting up one of these projects + +00:59:44.360 --> 00:59:50.680 +projects is like booting up an emulator for some old dos game right right well i mean i guess + +00:59:51.940 --> 00:59:59.180 +let's think about this for a second somebody got oh gosh what was the chain this is the whole um + +01:00:02.360 --> 01:00:09.820 +yavascript the pycon talk where got like firefox compiled into + +01:00:11.060 --> 01:00:18.960 +wet not wasm into asm js or something like that so it was run like chrome was running firefox + +01:00:19.060 --> 01:00:25.820 +which was running I think Doom which was also Asm.js. If we can do that we could get something + +01:00:25.960 --> 01:00:30.800 +that would run that would read old WebAssembly into new WebAssembly if it really mattered to the world. + +01:00:33.440 --> 01:00:39.780 +Absolutely. Yeah. Especially if it's in a public repo that people who care about the data can + +01:00:40.260 --> 01:00:47.000 +can rescue it somehow. Yeah. What about like a virtual machine? You know, I agree. Yeah, + +01:00:47.020 --> 01:00:53.960 +I'm gonna save me some, take a snapshot of Ubuntu LTS, + +01:00:54.100 --> 01:00:55.880 +some version and just, what are we gonna do? + +01:00:56.680 --> 01:00:58.560 +- Everything we do is Dockerized. + +01:00:59.100 --> 01:00:59.980 +Everything is in a container. + +01:01:00.450 --> 01:01:01.560 +So in the worst case scenario, + +01:01:02.010 --> 01:01:03.240 +we could give somebody the image + +01:01:03.970 --> 01:01:05.300 +and they could run it if they have Docker. + +01:01:06.500 --> 01:01:10.060 +I think that's a nice peace of mind to know that + +01:01:10.170 --> 01:01:13.360 +no matter what, something will be able to run this container. + +01:01:13.620 --> 01:01:15.660 +And even in, I don't know if you've used GitHub, + +01:01:16.340 --> 01:01:17.440 +What is it called? Codespaces. + +01:01:18.740 --> 01:01:18.940 +- Yeah. + +01:01:19.760 --> 01:01:22.080 +- I've, I've, I archived one project. + +01:01:23.480 --> 01:01:25.220 +It was kind of dramatic and all, + +01:01:25.440 --> 01:01:27.320 +and sudden that it needed to be archived. + +01:01:27.500 --> 01:01:29.260 +So without much time to do anything. + +01:01:29.320 --> 01:01:31.660 +And it was a Ruby on Rails project + +01:01:31.980 --> 01:01:34.200 +and I'm not a Rails developer, + +01:01:34.380 --> 01:01:37.880 +but I was able to get it archived in a way that anybody could + +01:01:38.480 --> 01:01:41.320 +with one command, go to the repo on GitHub + +01:01:41.640 --> 01:01:45.220 +and boot it up in Codespaces and then have it live. + +01:01:46.120 --> 01:01:47.980 +running from their code space. + +01:01:48.040 --> 01:01:49.340 +And so that works too. + +01:01:51.240 --> 01:01:54.040 +Very cool. I think as WebAssembly grows, + +01:01:54.520 --> 01:01:57.700 +there'll be more possibilities for these types of things. + +01:01:59.780 --> 01:02:03.280 +Yeah, amazing. I'm pretty excited + +01:02:03.460 --> 01:02:05.520 +about PageFind having a Python API. + +01:02:05.620 --> 01:02:06.400 +I didn't realize that. + +01:02:06.540 --> 01:02:09.020 +So I'm going to be doing something with that for sure. + +01:02:12.300 --> 01:02:13.700 +Let me ask you one more thing + +01:02:13.700 --> 01:02:19.180 +forward, I kind of let you wrap up with some final thoughts here. What about AI? + +01:02:20.700 --> 01:02:27.640 +Oh, that's a good question. So AI, I mean, there's like, in my story, there's like one + +01:02:27.840 --> 01:02:33.900 +interesting part of AI, which is that I got started and self-learned everything I needed + +01:02:34.000 --> 01:02:41.319 +to about software development to begin doing this right before ChatGPT really came on and was able + +01:02:41.340 --> 01:02:48.420 +to do real programming yeah you're like four years of legit programming before right so i think i + +01:02:48.520 --> 01:02:52.340 +mean so i was thinking i was thinking and i was thinking about how i got into it i thought what + +01:02:52.500 --> 01:03:02.120 +if i was four years later starting my phd and wanting to do these tools um i would have been + +01:03:02.160 --> 01:03:08.760 +able to accomplish what i needed to for my research without acquiring the technical skills i know and + +01:03:08.780 --> 01:03:12.020 +I don't know that's a good thing. I'm not sure if that's a good or bad thing. It could be both. + +01:03:12.170 --> 01:03:29.260 +I would have thought it was a good thing. I would have thought it's a good thing. But in my hands now, like a software engineer, AI is more powerful in my hands now than it would have been then. + +01:03:30.240 --> 01:03:31.720 +Even the same model, same everything. + +01:03:31.840 --> 01:03:36.380 +for me. Yeah, I can make it work for me in a way that I couldn't have been able to then. So I'm + +01:03:36.560 --> 01:03:41.860 +thankful for that. But it's something I think of, you know, I don't want to say it's a necessarily + +01:03:41.990 --> 01:03:47.580 +a bad thing. But it definitely marks a difference, a difference in time between other people who are + +01:03:48.220 --> 01:03:53.140 +maybe wanting to get into digital humanities, their humanities researchers, they want to add + +01:03:53.150 --> 01:03:58.660 +some digital tools. You know, I think this will kind of this will probably knock people off of + +01:03:58.680 --> 01:03:59.980 +the more technical path. + +01:04:00.680 --> 01:04:01.700 +I think it will too. + +01:04:01.850 --> 01:04:03.800 +And I think that that might be a negative. + +01:04:04.080 --> 01:04:07.000 +When you were telling me your story originally, + +01:04:07.630 --> 01:04:12.520 +I was thinking kind of like how neat is it that you didn't sign up for + +01:04:12.570 --> 01:04:15.900 +and the people you were working with probably didn't intend to sign you up + +01:04:16.020 --> 01:04:18.960 +for learning true software development. + +01:04:19.610 --> 01:04:22.940 +But look at this cool and interesting job that you now have + +01:04:23.560 --> 01:04:24.900 +that you never would have imagined. + +01:04:25.180 --> 01:04:26.780 +I'm sure when you signed up for your PhD, + +01:04:26.810 --> 01:04:28.440 +you're like, you know what I'm going to do when I get my PhD? + +01:04:28.600 --> 01:04:31.260 +I'm going to go X, Y, like, I'm going to join the Darth program. + +01:04:31.380 --> 01:04:34.020 +Like, no, probably not. Right. But here you are. + +01:04:34.740 --> 01:04:38.260 +And I think that's actually a really interesting knock on effect for a lot of + +01:04:38.560 --> 01:04:41.300 +researchers and people in grad schools. + +01:04:41.660 --> 01:04:44.960 +They're kind of put into this programming adjacent type of thing. + +01:04:46.100 --> 01:04:49.760 +You know, and a lot of folks sort of, actually, that's pretty interesting. + +01:04:49.960 --> 01:04:53.160 +I'm going to kind of lean into that. And I think AI might knock, + +01:04:54.580 --> 01:04:56.560 +not like you said, knock people off that path to some degree. + +01:04:57.800 --> 01:05:22.000 +Yeah, definitely. So that's just one part of the AI story. The other one is how we use it. It's great for data extraction, pulling data out of different, to make these search interfaces more powerful, to extract different data from them. That's just one example where it's been handy. + +01:05:22.780 --> 01:05:27.700 +we're looking for ways that it can really empower faculty. + +01:05:28.380 --> 01:05:30.840 +You know, we're still very much in the exploration phase + +01:05:31.040 --> 01:05:34.220 +of like how we can use it and provide it to faculty + +01:05:34.620 --> 01:05:37.520 +as a like digital humanities tool. + +01:05:38.380 --> 01:05:40.140 +- Sure, I was thinking pretty much + +01:05:40.780 --> 01:05:41.800 +when I asked the question of it, + +01:05:41.820 --> 01:05:44.800 +it's just like two parts, like one, how does it, + +01:05:45.020 --> 01:05:48.080 +are you guys using it to help take project? + +01:05:48.260 --> 01:05:49.100 +Well, that would have been a month. + +01:05:49.180 --> 01:05:50.140 +No, actually it's three days. + +01:05:50.540 --> 01:05:51.060 +You know what I mean? + +01:05:52.200 --> 01:05:57.880 +that and then if people are asking you know a professor comes along and says and we want our + +01:05:57.930 --> 01:06:05.280 +own custom ai thing or we're using harvard's internal one that we're allowed to use + +01:06:06.880 --> 01:06:09.400 +but we won't be able to use it once the grant runs out you know what i mean + +01:06:11.120 --> 01:06:16.340 +yeah yeah i think um one one good example of this type of thing is that um what we're starting to + +01:06:16.280 --> 01:06:20.540 +to get is faculty who are vibe coding. + +01:06:21.160 --> 01:06:23.300 +And now we're going to teach them. + +01:06:23.780 --> 01:06:25.020 +We're going to teach them how to do it. + +01:06:25.860 --> 01:06:25.960 +Yeah. + +01:06:26.600 --> 01:06:27.120 +Instead of-- + +01:06:27.120 --> 01:06:27.780 +It is a skill. + +01:06:28.520 --> 01:06:28.860 +Yeah. + +01:06:28.940 --> 01:06:30.120 +It's absolutely a skill. + +01:06:30.940 --> 01:06:31.140 +Yeah. + +01:06:31.220 --> 01:06:31.780 +No, it is. + +01:06:31.900 --> 01:06:32.260 +It is. + +01:06:33.020 --> 01:06:37.980 +Instead of copy and pasting from ChatGPT into VS Code, + +01:06:39.400 --> 01:06:42.060 +having them learn Copilot, maybe even having them download + +01:06:42.300 --> 01:06:42.460 +cursor. + +01:06:43.100 --> 01:06:45.660 +Download some real dedicated tools + +01:06:45.680 --> 01:06:47.880 +to get this done to make them more productive. + +01:06:48.320 --> 01:06:52.820 +So yeah, educating about how to do it is one thing. + +01:06:53.740 --> 01:06:54.760 +You asked if we're using it. + +01:06:55.920 --> 01:07:02.640 +We have access to a co-pilot and that's great. + +01:07:04.820 --> 01:07:06.540 +I can't say that we've shipped anything in three days + +01:07:06.940 --> 01:07:10.500 +instead of a month yet, but one anecdote is that + +01:07:11.880 --> 01:07:13.620 +right now I'm doing some really interesting + +01:07:15.400 --> 01:07:22.420 +Processing of music of music audio files and somebody asked to have a beatboxer if I could chop that file up + +01:07:22.860 --> 01:07:29.100 +So that all of the individual sounds that the beatboxer makes are identified in a file and so okay + +01:07:30.120 --> 01:07:34.400 +So I'm using some music library. So a Python library library called Librosa + +01:07:35.000 --> 01:07:38.400 +There's some complicated math in there. It's a little bit too much for me + +01:07:39.080 --> 01:07:45.360 +It's no problem for Claude Claude knows how to do that math and then and I use my expertise to string it together to get + +01:07:45.600 --> 01:07:53.420 +good output. Yeah, awesome. You got time for one more quick question before we clap things up. + +01:07:53.420 --> 01:07:58.320 +For sure. Raymond out there, Raymond Yees asks, it says, it'd be good to hear how Harvard uses + +01:07:59.240 --> 01:08:04.160 +containers on AWS and its reliability. It's reliable, not the cheapest way to host things. + +01:08:05.280 --> 01:08:11.420 +Are you thinking about moving that or is it not that much, you know, not that big of an expense? + +01:08:11.440 --> 01:08:21.839 +about a failed experiment. We were using ECS and we're still using ECS. So that's AWS's main, + +01:08:22.759 --> 01:08:28.859 +it's not Kubernetes, but it's one step down with their horizontal scaling container clusters. + +01:08:30.859 --> 01:08:36.100 +And I wanted to move us onto a single EC2 instance because our projects are popular, + +01:08:36.319 --> 01:08:39.900 +but they're not so popular that we actually have to worry about horizontal scaling. + +01:08:40.080 --> 01:08:46.600 +Right. It's not like it's front page in New York Times. I guess it probably could be. But even so, + +01:08:46.630 --> 01:08:48.759 +for the static sites, they probably still can take it. + +01:08:50.049 --> 01:08:55.640 +Yeah. So I priced it out and I got an example deployed, an example project deployed, + +01:08:56.420 --> 01:09:03.199 +and was able to confirm that it would indeed be much cheaper. And it was deployed in a similar way + +01:09:03.220 --> 01:09:10.319 +using AWS CDK. So it's all infrastructure is code all the way down. But it turns out there's all + +01:09:10.500 --> 01:09:16.180 +kinds of compliance when you are in charge of the VM at like a big university, or I'm sure any + +01:09:16.620 --> 01:09:23.620 +any corporate setting, if you are in charge of the VM and the OS on it, then you have to know that you + +01:09:23.700 --> 01:09:28.720 +have the latest patches in. You have to know that you have latest Ubuntu. And then there's other + +01:09:28.740 --> 01:09:35.759 +things of different observability things that you have to have in place that you that are not usually + +01:09:35.980 --> 01:09:41.859 +required if you're running in a container cluster like ecs i see so it ends up being a lot less + +01:09:42.540 --> 01:09:48.020 +a lot less work and much easier to achieve compliance if we run containers or some other + +01:09:48.180 --> 01:09:54.960 +serverless some other serverless thing if i run my all my personal projects they all run in a single + +01:09:55.020 --> 01:10:01.500 +a single virtual machine but um sure we're running in containers yeah yeah and you've got all the + +01:10:01.640 --> 01:10:08.520 +sock 2 stuff and all those different things right like there's layers yeah that's right awesome okay + +01:10:09.000 --> 01:10:16.360 +yeah very cool final thoughts you want to talk to people who may be doing similar all right go ahead + +01:10:16.660 --> 01:10:20.760 +you've got something more to add for you yeah i mean i'll mention that um but what i didn't say is + +01:10:20.660 --> 01:10:28.060 +that in that 2019 when i started learning python i discovered talk python almost immediately and + +01:10:28.180 --> 01:10:33.780 +one of the first episodes that i listened to was was the other digital humanities um the one with + +01:10:34.260 --> 01:10:39.140 +cornelius van lit he was an awesome guest yeah that's right yeah and i thought i thought that + +01:10:39.140 --> 01:10:44.620 +was great and that was also a bit about manuscripts a little bit more in the image side than the text + +01:10:44.800 --> 01:10:50.620 +side and um i didn't understand everything that everybody was saying but i just i kept tuning + +01:10:50.640 --> 01:10:56.180 +in. And I think because of that, because Talk Python was like this, you know, I've been remote + +01:10:56.590 --> 01:11:03.480 +working for most of my time. And Talk Python has been kind of like that conversation with + +01:11:03.610 --> 01:11:08.340 +the open source community that's been always in my ear. And I think that made, you know, + +01:11:08.510 --> 01:11:14.820 +a difference making me feel like I understood the software landscape and like the developer + +01:11:14.940 --> 01:11:19.660 +culture and what was going on. And then the different Python libraries and what was possible. + +01:11:20.400 --> 01:11:32.520 +So to people who are interested in taking things in a more technical direction, I think it's helpful just to find a few things like that, that give you an insight into that world. + +01:11:32.980 --> 01:11:43.320 +And the more you listen to it, the more you start to hear the same acronyms and the same things said enough that you start to feel like, okay, now you're part of the club. + +01:11:43.820 --> 01:11:45.060 +I really appreciate that. + +01:11:45.940 --> 01:11:46.300 +That's cool. + +01:11:46.880 --> 01:11:52.140 +I've certainly had people reach out to me and say things that at first didn't make any sense to me. + +01:11:52.260 --> 01:11:56.440 +Like, I've been listening for six weeks now, and it's starting to make sense what you're talking about. + +01:11:56.500 --> 01:11:58.720 +Like, why have you been listening for six months when it made no sense? + +01:11:58.860 --> 01:11:59.200 +That's insane. + +01:11:59.820 --> 01:12:06.640 +But a lot of people use listening to the podcast, mine and others, as language immersion, right? + +01:12:06.880 --> 01:12:13.000 +Like, I could get Duolingo, and I could learn Portuguese, or I could move to Brazil for a month. + +01:12:13.340 --> 01:12:13.800 +You know what I mean? + +01:12:14.100 --> 01:12:15.060 +Then I would really learn it. + +01:12:15.120 --> 01:12:15.220 +Yeah, exactly. + +01:12:15.580 --> 01:12:15.840 +Right. + +01:12:16.600 --> 01:12:23.820 +Exactly. I think there's truth to that. Some of the things I did was search the word deployment + +01:12:24.440 --> 01:12:28.340 +because I'm trying to get my head around how to deploy for the first time. I just want to hear + +01:12:28.480 --> 01:12:32.280 +people talk about it. I could read about it. I could read the tutorial, but I just want to hear + +01:12:32.400 --> 01:12:37.560 +people talk about deployment to get a sense of what actual deployment sounds like. + +01:12:37.860 --> 01:12:42.940 +There's something really different when you're learning or trying, even you're maybe an experienced + +01:12:42.960 --> 01:12:48.820 +programmer but not in this particular area to hear a human side of it not just the docs not a sterile + +01:12:49.260 --> 01:12:55.160 +these are the four steps but like a human i love it i mean it's probably why i create the show + +01:12:55.420 --> 01:13:00.840 +because like i didn't hear those stories we got to tell those stories awesome i appreciate that so + +01:13:02.100 --> 01:13:06.980 +super cool all right so if other people are listening maybe one of your pieces of advice is + +01:13:08.000 --> 01:13:15.480 +keep listening you'll get there yeah and and if anybody's in is like in the humanities and somehow + +01:13:15.620 --> 01:13:22.120 +found their way onto this episode with no technical uh experience i just would would give the caution + +01:13:22.280 --> 01:13:29.840 +of of like you know the anecdote that if um ai coding had been around the way it is now when i + +01:13:29.940 --> 01:13:36.620 +was learning i wouldn't be doing digital humanities at harvard i wouldn't have been able to get into + +01:13:36.640 --> 01:13:39.240 +this field, I wouldn't have known about it. + +01:13:39.980 --> 01:13:45.280 +So I guess just think about that when you're learning and applying new tools. + +01:13:46.960 --> 01:13:49.740 +I don't really know what the right fix for that is. + +01:13:49.840 --> 01:13:51.080 +That's a very challenging problem. + +01:13:51.300 --> 01:13:54.780 +I mean, you can say I'm just literally not going to fire it up. + +01:13:54.940 --> 01:13:59.440 +But I mean, we used to hunt through Stack Overflow and the web and over and over. + +01:13:59.560 --> 01:14:03.680 +And if you're really stuck or you really don't understand, like they're good at explaining + +01:14:03.780 --> 01:14:04.260 +stuff to you. + +01:14:04.800 --> 01:14:10.220 +really stay in a learner's mindset, not just press the easy button and make this thing and move on. + +01:14:13.440 --> 01:14:18.940 +Easier said than done. Easier and said than done. So yeah, I just I want to leave this with kind of a + +01:14:19.800 --> 01:14:29.239 +thought about how much things like Python and these, these tools and technology can really empower + +01:14:29.940 --> 01:14:31.460 +stuff that you wouldn't think is even related, + +01:14:31.980 --> 01:14:33.400 +like understanding old manuscripts + +01:14:33.800 --> 01:14:40.480 +and how painting is connected or changed over time and stuff, right? + +01:14:40.680 --> 01:14:43.720 +Those sound very much disjointed from tech and software, + +01:14:44.200 --> 01:14:47.800 +but they really are superpowers that you can bring to your work, + +01:14:48.460 --> 01:14:50.000 +whatever your industry is. + +01:14:50.020 --> 01:14:51.540 +I know our field of study, + +01:14:51.560 --> 01:14:54.540 +I know there's some sociologists out in the audience + +01:14:54.720 --> 01:14:55.700 +and I'm sure others as well. + +01:14:58.960 --> 01:15:01.500 +All right. Final thoughts, David, close it out. + +01:15:09.800 --> 01:15:10.400 +That's it. + +01:15:10.820 --> 01:15:11.400 +That's it. All right. + +01:15:12.280 --> 01:15:12.800 +You said it great. + +01:15:12.990 --> 01:15:18.600 +I mean, just applying these technical tools to old questions, + +01:15:19.170 --> 01:15:21.080 +that is the core of digital humanities. + +01:15:21.940 --> 01:15:22.520 +Awesome. Yeah. + +01:15:24.080 --> 01:15:25.120 +When I first started hearing about this, + +01:15:25.120 --> 01:15:26.900 +I thought I really don't know how this ties together. + +01:15:27.120 --> 01:15:30.760 +and after seeing it a few times, I definitely see the power of it. + +01:15:31.680 --> 01:15:32.980 +Thank you for your time coming on. + +01:15:33.220 --> 01:15:37.460 +Thank you for sharing your look and the look inside of your team + +01:15:37.800 --> 01:15:40.200 +and inside of a small piece of Harvard. + +01:15:41.240 --> 01:15:43.020 +I really like these kinds of episodes + +01:15:43.320 --> 01:15:46.000 +because it's hard to see this from the outside. + +01:15:47.260 --> 01:15:48.720 +You just see the results, + +01:15:48.830 --> 01:15:51.000 +but you don't see the inner workings of the team + +01:15:51.070 --> 01:15:51.980 +and the motivation and stuff. + +01:15:52.840 --> 01:15:54.460 +Thank you so much for being here. + +01:15:55.760 --> 01:15:56.420 +Bye, everyone. + +01:15:56.860 --> 01:15:57.000 +Yep. + diff --git a/youtube_transcripts/539-catching-up-with-the-python-typing-council-original.vtt b/youtube_transcripts/539-catching-up-with-the-python-typing-council-original.vtt new file mode 100644 index 0000000..6993e95 --- /dev/null +++ b/youtube_transcripts/539-catching-up-with-the-python-typing-council-original.vtt @@ -0,0 +1,3380 @@ +WEBVTT + +00:00:02.040 --> 00:00:04.000 +Hello, Rebecca and Carl. + +00:00:04.780 --> 00:00:07.700 +Welcome to all of you type loving Pythonistas. + +00:00:08.480 --> 00:00:09.640 +Awesome to have you here on the show. + +00:00:11.639 --> 00:00:12.680 +Thanks for being here. + +00:00:14.040 --> 00:00:15.340 +We're going to talk Python typing, + +00:00:16.200 --> 00:00:19.880 +especially from the perspective of the Python typing council, + +00:00:20.660 --> 00:00:24.360 +which honestly, I am a huge fan of Python typing. + +00:00:24.620 --> 00:00:26.840 +It's still something I learned about not too long ago. + +00:00:26.980 --> 00:00:33.040 +So I'm going to be learning along with everyone else what it is you all do and so on. + +00:00:33.220 --> 00:00:36.560 +So I'm really excited to be diving into this. + +00:00:36.950 --> 00:00:42.520 +I think since types came to Python, I think it's made it a little bit more rigorous. + +00:00:43.100 --> 00:00:47.880 +You know, for all those people out there like, oh, it's not a real language without any form of static typing. + +00:00:47.930 --> 00:00:49.040 +We can't use it on real projects. + +00:00:49.090 --> 00:00:52.820 +I don't know how true that was, but certainly it's less true now. + +00:00:53.120 --> 00:00:54.560 +You know, you can pick per project. + +00:00:54.630 --> 00:00:55.380 +So it's super cool. + +00:00:56.600 --> 00:01:00.040 +Before we get into all that, though, let's just go around for a quick introductions. + +00:01:01.420 --> 00:01:03.440 +Jala, welcome to the show. Awesome to have you here. + +00:01:04.480 --> 00:01:08.900 +Hi, yeah. I'm Jala. I've been on the Python typing council since the beginning. + +00:01:09.360 --> 00:01:15.440 +I helped set it up a couple years ago. Outside of my typing work, I currently work at OpenAI, + +00:01:15.740 --> 00:01:19.859 +where I work on developer productivity, which means things like running CI for people + +00:01:21.319 --> 00:01:24.080 +and generally helping people be productive. + +00:01:25.920 --> 00:01:28.140 +I've been working with Python for more than a decade. + +00:01:28.740 --> 00:01:31.460 +I started out because my previous job was mostly in Python + +00:01:31.920 --> 00:01:34.140 +and then got more and more involved with the language. + +00:01:35.700 --> 00:01:35.900 +Awesome. + +00:01:36.590 --> 00:01:37.860 +So let me get this right. + +00:01:38.110 --> 00:01:41.440 +At OpenAI, you're basically helping developers there + +00:01:41.620 --> 00:01:45.040 +have better developer tooling and common packages + +00:01:45.500 --> 00:01:46.720 +and workflows and stuff like that. + +00:01:47.000 --> 00:01:47.580 +That's right. + +00:01:48.450 --> 00:01:48.600 +Yeah. + +00:01:48.680 --> 00:01:54.480 +Yeah, mostly around things that happen in CI, like running tests efficiently, figuring + +00:01:54.550 --> 00:01:58.820 +out the right tests to run, getting the right CI workers up. + +00:01:59.800 --> 00:02:01.320 +Yeah, that sounds very exciting. + +00:02:01.920 --> 00:02:06.300 +Right in the epicenter of all the big tech stuff these days. + +00:02:06.440 --> 00:02:06.820 +Super cool. + +00:02:07.700 --> 00:02:09.240 +Rebecca, hello, welcome. + +00:02:10.500 --> 00:02:12.160 +Hey, thanks for having me. + +00:02:13.120 --> 00:02:13.540 +I'm Rebecca. + +00:02:14.280 --> 00:02:19.360 +I've been on the typing council also for about three years, I think, + +00:02:19.360 --> 00:02:21.940 +since the, less than three, since the beginning. + +00:02:22.480 --> 00:02:27.260 +But my day job, I work at Meta on Python typing, + +00:02:27.840 --> 00:02:31.980 +gone Pyrefly, which is a new type checker and language server + +00:02:32.060 --> 00:02:33.680 +written in Rust, still in beta. + +00:02:34.500 --> 00:02:37.940 +Prior to that, I was at Google for eight years, + +00:02:38.280 --> 00:02:39.480 +also on the Python team. + +00:02:39.520 --> 00:02:40.920 +I just, I really like Python. + +00:02:42.240 --> 00:02:42.560 +Awesome. + +00:02:43.580 --> 00:02:50.040 +Yeah, super neat. I'm a big fan of both Pyrefly and ty, which will both have representatives here, + +00:02:50.150 --> 00:02:55.660 +I know. And I think it's just a super exciting time for Python types. And certainly that's one + +00:02:55.740 --> 00:03:03.580 +of the reasons. So very cool. Carl, welcome back. Thank you. Great to be here. Yeah, Carl Meyer. + +00:03:04.540 --> 00:03:10.919 +I currently work at Astral, where I work on ty, which is a Python type checker and language server + +00:03:12.180 --> 00:03:18.600 +written in Rust, also in beta. And yeah, I guess how did I get into typing? Or I've been on the + +00:03:18.680 --> 00:03:23.860 +typing council not since the beginning. I think it's been a year and a half. + +00:03:28.000 --> 00:03:36.199 +And yeah, I got into Python typing at the time in 2016, 2017. I was working at Instagram and + +00:03:37.160 --> 00:03:39.320 +That was in the very early days of Python typing. + +00:03:41.360 --> 00:03:46.780 +The PEP484, PEP483, the early Python typing peps had recently come out within the last couple of years. + +00:03:47.610 --> 00:03:53.680 +And one of the co-authors of some of those peps, Lukash Lange, was actually sitting at a desk right next to me at the time. + +00:03:56.459 --> 00:04:02.420 +And at some point we started to think that we should try this Python typing stuff on the Instagram server monolith. + +00:04:03.420 --> 00:04:10.140 +And so I took that on as a side project, and then it eventually became the main project, and then it took like three years. + +00:04:10.640 --> 00:04:13.160 +So a lot of Python typing experience there. + +00:04:13.900 --> 00:04:15.000 +Wow, there absolutely is. + +00:04:15.220 --> 00:04:17.859 +You know, I think a couple of things I'd like to touch on there. + +00:04:20.239 --> 00:04:25.800 +First of all, Instagram, is it maybe the biggest Django deployment in the world? + +00:04:25.960 --> 00:04:27.360 +It's certainly one of the bigger ones, right? + +00:04:27.420 --> 00:04:32.860 +And I think a lot of people don't necessarily know that a core chunk of Instagram is actually Python, right? + +00:04:36.980 --> 00:04:42.600 +i don't i mean i don't know if we have any way to know how big uh the django to plant in the + +00:04:42.760 --> 00:04:47.340 +wild might be but it's certainly a big one yeah it's definitely a big one there was some talks + +00:04:47.660 --> 00:04:54.300 +about um dismissing the garbage collector from the instagram folks uh that wasn't you giving the talk + +00:04:54.440 --> 00:05:00.699 +but at pycon so that was pretty interesting but i think actually that work that you're talking + +00:05:00.720 --> 00:05:07.140 +about, especially with Lukash, really kind of opened a lot of people's eyes about Python typing. + +00:05:08.260 --> 00:05:14.780 +He gave a couple of PyCon talks, showed real metrics of how much of the code base is typed, + +00:05:16.400 --> 00:05:23.520 +how much it's changed, like error detection, that kind of stuff. So let me ask you, + +00:05:23.960 --> 00:05:29.679 +do you feel like it would be different? Would it have gone different now if tools like ty and + +00:05:29.700 --> 00:05:36.000 +Pyrefly exists existed back then is Python typing different now than it was then + +00:05:41.420 --> 00:05:44.240 +uh certainly yes I mean there's been the type system has gotten + +00:05:45.220 --> 00:05:49.780 +more complex over time so it is both more expressive and more complex + +00:05:52.380 --> 00:05:56.159 +and yeah we have more more type checkers available now now + +00:06:08.060 --> 00:06:08.400 +Hold on. + +00:06:09.700 --> 00:06:11.780 +I think for some reason my audio just cut out. + +00:06:11.820 --> 00:06:12.580 +Can you all still hear me? + +00:06:13.680 --> 00:06:14.340 +I can hear you. + +00:06:15.500 --> 00:06:16.260 +Okay, yeah, it came back. + +00:06:16.640 --> 00:06:17.000 +I don't know. + +00:06:19.420 --> 00:06:19.800 +These AirPods. + +00:06:20.100 --> 00:06:21.240 +You know how AirPods switch? + +00:06:23.940 --> 00:06:28.160 +I got a text which made a noise on my iPad and I think my sound switched to my iPad. + +00:06:28.280 --> 00:06:28.900 +Anyway, sorry about that. + +00:06:29.670 --> 00:06:32.600 +I do agree that it's more complicated and I don't know how to feel about that. + +00:06:33.400 --> 00:06:40.440 +It is more expressive, but I feel like it's starting to get, I mean, we're not at C++ + +00:06:41.360 --> 00:06:49.400 +ATL like templates of templates of templates, but still, it's getting more serious. + +00:06:49.560 --> 00:06:57.220 +But I guess one of the really nice parts is that you can just take as much as you want of the complexity and you can just leave the rest, right? + +00:06:57.780 --> 00:07:02.860 +That's part of the magic of Python typing is that it's a gradual typing system, right? + +00:07:03.300 --> 00:07:07.720 +That's a choice people get to make. + +00:07:07.720 --> 00:07:08.380 +It can be none. + +00:07:08.900 --> 00:07:12.920 +It can be quite a bit and anywhere in between. + +00:07:14.440 --> 00:07:16.660 +So I guess that's probably one of the decisions. + +00:07:17.760 --> 00:07:19.160 +Let's talk about the typing council. + +00:07:19.240 --> 00:07:23.780 +So when did the typing council come along? + +00:07:25.940 --> 00:07:30.520 +And did the typing council exist to create all these peps and make this happen? + +00:07:30.610 --> 00:07:31.680 +Or was it afterwards? + +00:07:32.600 --> 00:07:35.420 +What's the history of the typing council and its purpose, folks? + +00:07:36.040 --> 00:07:37.860 +Yeah, it postdates most of the peps. + +00:07:38.370 --> 00:07:42.400 +So initially, the type system was created just through the regular PEP process, + +00:07:42.610 --> 00:07:43.960 +which means that something gets submitted. + +00:07:45.140 --> 00:07:47.820 +At first, still to Guido as the BDFL. + +00:07:48.500 --> 00:07:49.580 +later the steering council. + +00:07:50.960 --> 00:07:54.720 +But that meant that it's very hard to make changes to like this specification. + +00:07:54.990 --> 00:07:58.520 +Like anytime we want to make change something about how the type systems would work, + +00:07:59.400 --> 00:08:00.960 +we had to go through this PEP procedure, + +00:08:01.800 --> 00:08:03.640 +talk to the steering council who are very busy people + +00:08:03.900 --> 00:08:06.900 +who deal with a lot of other aspects of the language other than typing. + +00:08:07.980 --> 00:08:13.000 +So, Shantanu and I came up with this idea of creating a separate council + +00:08:13.780 --> 00:08:15.279 +to specifically in charge of typing + +00:08:16.160 --> 00:08:17.360 +that would be in a specification + +00:08:17.680 --> 00:08:19.440 +where we can make small changes ourselves + +00:08:19.740 --> 00:08:21.440 +without having to go through this whole PEP process. + +00:08:23.800 --> 00:08:26.060 +And this way, when all the type checkers agree + +00:08:26.200 --> 00:08:27.680 +that something needs to go a certain way + +00:08:27.820 --> 00:08:29.240 +and it's not exactly what's in the PEPs, + +00:08:29.740 --> 00:08:32.780 +we can change it and have a place to record that + +00:08:32.940 --> 00:08:34.020 +and people can refer to it + +00:08:34.280 --> 00:08:36.940 +and new type checkers can also try to follow those decisions. + +00:08:39.419 --> 00:08:39.900 +Very interesting. + +00:08:40.120 --> 00:08:41.520 +I didn't realize that it was sort of, + +00:08:42.340 --> 00:08:46.660 +was there to allow for small changes to be made to make that much easier. + +00:08:46.770 --> 00:08:49.120 +But of course that makes sense because the PEP process is, + +00:08:50.060 --> 00:08:51.560 +it's pretty serious and drawn out. + +00:08:51.570 --> 00:08:58.880 +And we've seen even small language changes have quite passionate folks, + +00:08:58.910 --> 00:09:00.640 +I guess we should say. + +00:09:05.180 --> 00:09:06.620 +So yeah, yeah, very nice. + +00:09:09.940 --> 00:09:18.560 +Do you have any examples of the types of changes that you all have that have happened over the years that maybe were typing council only? + +00:09:23.660 --> 00:09:28.580 +One was the specification for how overloads work, which is perhaps not really a small change. + +00:09:28.700 --> 00:09:35.920 +But one of the most complicated features in the type system really is the overloads, where you can give multiple signatures for a function. + +00:09:37.000 --> 00:09:39.800 +and type checkers sort of select which one to use + +00:09:40.040 --> 00:09:42.140 +based on the arguments when the function is called. + +00:09:43.560 --> 00:09:44.840 +And when it was initially created, + +00:09:45.840 --> 00:09:46.460 +I, from what I recall, + +00:09:46.860 --> 00:09:48.280 +there just wasn't really a specification. + +00:09:48.740 --> 00:09:50.860 +It's just like you use the signatures + +00:09:51.180 --> 00:09:52.260 +in a way that makes sense. + +00:09:53.820 --> 00:09:55.820 +And Eric Trout, who's currently on the council + +00:09:56.180 --> 00:09:58.800 +came up with a pretty specific procedure + +00:09:58.920 --> 00:10:00.680 +for exactly how overload should work + +00:10:01.580 --> 00:10:03.840 +to make it so that type checkers have, + +00:10:04.840 --> 00:10:06.419 +both sort of users can understand how it works + +00:10:06.440 --> 00:10:10.300 +type checkers can have something to work towards to make sure that they will implement overloads + +00:10:10.300 --> 00:10:16.780 +in the same way. Yeah. Maybe a smaller example that is an example of something that would have been + +00:10:17.160 --> 00:10:23.040 +too small for a PEP and hard to accomplish before the typing council existed. And this + +00:10:23.040 --> 00:10:28.860 +is actually a change that I pushed through before I was on the typing council, but the typing + +00:10:29.060 --> 00:10:36.400 +council approved it, was a clarification around the interpretation of data class fields. If a final + +00:10:36.440 --> 00:10:43.740 +annotation is applied to a data class field, does that mean, so if you apply a final annotation to + +00:10:43.740 --> 00:10:48.840 +a regular class attribute, since it can't be changed, that implies that it's a class variable. + +00:10:49.620 --> 00:10:53.040 +And there was a question of if that should be the interpretation with the data class or not. + +00:10:53.640 --> 00:11:00.100 +So we discussed that and made a clarification to the spec. Okay, interesting. I've never really + +00:11:00.100 --> 00:11:04.820 +thought about final being applied to a class field, + +00:11:05.060 --> 00:11:07.180 +but I've always used them sort of just for constants. + +00:11:08.220 --> 00:11:09.920 +But maybe people out there don't know, + +00:11:10.280 --> 00:11:13.960 +like typing dot final bracket type, right? + +00:11:15.460 --> 00:11:18.280 +That's kind of the way you can do constants in Python, right? + +00:11:20.860 --> 00:11:22.200 +Constant for the type checker. + +00:11:22.720 --> 00:11:24.340 +Nothing in the runtime will stop you from editing it. + +00:11:25.220 --> 00:11:25.700 +But-- + +00:11:25.700 --> 00:11:26.500 +Not there. + +00:11:26.880 --> 00:11:27.380 +Not there. + +00:11:27.580 --> 00:11:29.320 +I have some examples coming up, and I'm + +00:11:30.240 --> 00:11:31.380 +interested to hear your thoughts on them. + +00:11:31.480 --> 00:11:35.400 +But for sure, there is this tension, right? + +00:11:35.860 --> 00:11:38.120 +I think that's probably worth touching on as well + +00:11:38.320 --> 00:11:40.600 +is this is a tension for Python in general + +00:11:40.860 --> 00:11:43.340 +is you can write all the types you want. + +00:11:43.920 --> 00:11:48.060 +And then when you run your code, it just doesn't care. + +00:11:48.180 --> 00:11:52.120 +There's a few instances, Pydantic, FastAPI, a few others. + +00:11:52.260 --> 00:11:55.120 +But generally speaking, it's there + +00:11:55.260 --> 00:11:57.300 +for the editors and the type checkers and the linters. + +00:11:57.520 --> 00:12:00.980 +and not for runtime, right? + +00:12:04.330 --> 00:12:04.960 +- Yeah, that's right. + +00:12:05.380 --> 00:12:07.680 +There's many exceptions to that. + +00:12:07.940 --> 00:12:10.320 +There's a product like mypyC, + +00:12:11.000 --> 00:12:13.060 +which comes with mypy that's used those types + +00:12:13.260 --> 00:12:16.340 +to compile your code into more efficient machine codes. + +00:12:17.760 --> 00:12:19.100 +Maybe there's gonna be more products like that + +00:12:19.100 --> 00:12:20.040 +in the future, I don't know. + +00:12:20.880 --> 00:12:23.260 +But yes, in general, it's separate from the runtime. + +00:12:24.140 --> 00:12:26.599 +Sort of a similar model to TypeScript + +00:12:26.620 --> 00:12:28.740 +where TypeScript gets compiled into JavaScript + +00:12:28.980 --> 00:12:29.940 +and types just go away. + +00:12:30.580 --> 00:12:31.980 +Here we don't do a compilation step, + +00:12:32.140 --> 00:12:33.620 +but still the same idea of the types, + +00:12:33.840 --> 00:12:35.100 +just not influencing the runtime. + +00:12:36.400 --> 00:12:38.960 +- Although we do make them available for introspection + +00:12:39.340 --> 00:12:40.780 +via the annotations attributes, + +00:12:41.180 --> 00:12:44.140 +which is what has enabled the projects like Pydantic + +00:12:44.420 --> 00:12:46.540 +and other sort of runtime checkers + +00:12:46.740 --> 00:12:49.280 +to make use of type annotations at runtime also. + +00:12:49.860 --> 00:12:51.520 +- Yeah, I don't know if the typing council + +00:12:51.720 --> 00:12:52.460 +was around for this, + +00:12:52.560 --> 00:12:55.460 +but there was a proposal, + +00:12:55.620 --> 00:13:01.500 +I don't remember the exact details, but something to the effect of for type checking, not actually + +00:13:01.780 --> 00:13:06.880 +doing some of the full imports or something along those lines, right? + +00:13:06.980 --> 00:13:11.720 +Where the runtime behavior would have made it hard for tools like Pydantic and others + +00:13:11.820 --> 00:13:12.420 +to get that. + +00:13:12.920 --> 00:13:14.720 +And there was some kind of compromise, right? + +00:13:14.760 --> 00:13:15.720 +I don't remember the details. + +00:13:16.220 --> 00:13:16.600 +Anyone does? + +00:13:17.240 --> 00:13:18.120 +Future import? + +00:13:18.960 --> 00:13:19.740 +Yes, I think so. + +00:13:20.000 --> 00:13:23.180 +Yeah, it's like 563 and 649 or something. + +00:13:24.120 --> 00:13:24.840 +And 749. + +00:13:25.520 --> 00:13:31.600 +Yeah, what happened was that there was going to be a change. That's what the from future + +00:13:31.740 --> 00:13:37.200 +import annotations import does. It changes all annotations into raw strings. So the default + +00:13:37.360 --> 00:13:43.440 +behavior before recently was that annotations are regular codes. If you write devf return + +00:13:43.530 --> 00:13:48.259 +to ints and you import the module, it just looks up the name ints and puts that in annotations + +00:13:48.260 --> 00:13:55.480 +dictionary, which makes introspection easy, but it's made, it has some costs on performance + +00:13:55.880 --> 00:14:00.160 +because memory usage sometimes is high and also made things harder to use sometimes because + +00:14:01.180 --> 00:14:06.060 +if you use a name that's not defined yet at runtime, you get an error. That often comes + +00:14:06.080 --> 00:14:14.379 +up if you have like a class that has a reference in an annotation to the class itself or circular + +00:14:14.400 --> 00:14:21.280 +dependency classes right the circular imports because you want to say this + +00:14:21.340 --> 00:14:27.240 +glass is created by that thing and it returns one but you know somehow you've + +00:14:27.300 --> 00:14:33.180 +got to import the other one and that's such a hassle yeah it's yeah even out in + +00:14:33.180 --> 00:14:40.359 +the audience we have circular imports oh yeah for sure what about lazy imports + +00:14:40.380 --> 00:14:43.040 +That just recently got accepted and will be in 3.15, + +00:14:44.080 --> 00:14:45.700 +which I'm super excited about + +00:14:45.870 --> 00:14:48.480 +because I think it'll make app startup a lot faster + +00:14:49.060 --> 00:14:50.680 +for many use cases. + +00:14:51.560 --> 00:14:53.600 +But does that have knock-on effects for typing? + +00:14:57.480 --> 00:14:58.560 +Not that directly, + +00:14:58.980 --> 00:15:00.860 +because I think for a type checker, + +00:15:01.000 --> 00:15:03.380 +lazy imports mostly just look like regular imports. + +00:15:03.800 --> 00:15:05.560 +I guess I should maybe leave that to the people + +00:15:05.560 --> 00:15:07.300 +who are actually working on type checkers + +00:15:07.360 --> 00:15:08.300 +that are being written right now. + +00:15:10.320 --> 00:15:17.460 +yeah yeah rebecca do you see this making any difference for you lazy imports yeah uh + +00:15:18.680 --> 00:15:25.520 +to be honest it's not something we've uh looked at too carefully yet 315 it seems a little + +00:15:26.400 --> 00:15:32.500 +fine in the future but i don't think it's likely to make a huge difference + +00:15:36.880 --> 00:15:37.040 +Karl? + +00:15:38.860 --> 00:15:43.600 +Yeah, I've thought about it briefly, and I think that it, I think the type checkers really won't need to care. + +00:15:44.560 --> 00:15:48.080 +Maybe there will be some edge cases that will come up that I haven't thought of, but it shouldn't be a big deal. + +00:15:48.940 --> 00:15:50.260 +Yeah, that's what I thought as well. + +00:15:50.440 --> 00:15:58.720 +the one variation that I can certainly see is if you have a type if you have something specified + +00:15:58.940 --> 00:16:05.880 +in a type like say for a field of a class or a pydantic model or something that would otherwise + +00:16:06.310 --> 00:16:13.320 +not trigger the lazy import to become imported would potentially having types specify + +00:16:14.410 --> 00:16:20.280 +cause more importing to happen sooner in the runtime yeah there's actually an issue related + +00:16:20.300 --> 00:16:25.460 +But I think we may need to resolve before 3.15, but I don't know how yet. + +00:16:26.800 --> 00:16:32.220 +In data classes, if you have, if you use a type in a data class annotation that's lazily + +00:16:32.380 --> 00:16:35.780 +imported, actually creating a data class will delay the import. + +00:16:36.140 --> 00:16:41.720 +It will try to resolve the import and actually make it not lazy. + +00:16:42.460 --> 00:16:46.620 +This is because data classes doesn't really need to look at all of the annotations in + +00:16:46.580 --> 00:16:51.720 +year class but it looks at them enough to trigger reification of the imports. + +00:16:53.690 --> 00:16:57.560 +I shared this with some of the people on the lazy imports team but we haven't yet come up + +00:16:57.560 --> 00:16:58.520 +with a good way around it. + +00:16:58.650 --> 00:17:02.640 +I think this might end up being a bit of a food gun so I feel like we should ideally + +00:17:02.880 --> 00:17:05.339 +find a workaround but I don't know what it would be yet. + +00:17:05.820 --> 00:17:11.920 +Yeah, I don't know that it's wrong that it converts it to an eager import which it needs + +00:17:12.069 --> 00:17:13.839 +to know what it is potentially, right? + +00:17:14.079 --> 00:17:17.980 +It actually doesn't. Data classes just need to know whether it's a classifier or not. + +00:17:19.680 --> 00:17:23.560 +I think that's pretty much all. I guess there's an initifier also, but it doesn't really need to + +00:17:23.600 --> 00:17:29.080 +know anything else. In theory, it should be possible to just say, hey, this is not classifier, + +00:17:29.300 --> 00:17:33.780 +so don't bother importing it. Interesting. That's for data classes, + +00:17:34.040 --> 00:17:40.600 +but say if I specify a parameter type on a function. Yeah, then it should be fine. I guess, + +00:17:40.840 --> 00:17:46.180 +again unless something is uh if it does annotations so if you have something like a decorator that + +00:17:46.340 --> 00:17:54.000 +looks at annotations in your function it might reify those imports okay interesting there is one + +00:17:54.070 --> 00:18:00.020 +other potentially interesting thing for type checkers it's already uh difficult for type + +00:18:00.200 --> 00:18:10.780 +checkers to figure out when uh like a sub module should be considered to be an attribute of the + +00:18:10.800 --> 00:18:17.260 +anywhere will attach that submodule as an attribute on the parent module. But at runtime, + +00:18:18.180 --> 00:18:21.680 +that could literally happen anywhere. It could happen in totally unrelated code outside of the + +00:18:21.830 --> 00:18:26.860 +module, and a type checker probably won't be able to see that. So type checkers already have sort + +00:18:26.870 --> 00:18:31.920 +of complex sets of rules around where they look for these submodule imports and when they consider + +00:18:31.930 --> 00:18:39.520 +a submodule import to be reliably happening enough that the type checker should consider + +00:18:39.540 --> 00:18:47.280 +this submodule to exist as an attribute. And lazy imports may make that even... We'll add one more + +00:18:47.460 --> 00:18:52.760 +wrinkle to those bits of heuristics in that we'll have to decide if you have a lazy import of a + +00:18:52.940 --> 00:18:59.420 +submodule and you're under init.py, it's lazy. So should the type checker consider that submodule to + +00:19:00.160 --> 00:19:06.160 +be imported or not be imported? I think it'll be another case where there's no clear right answer + +00:19:06.180 --> 00:19:09.440 +and we'll just have to make a decision one way or the other. + +00:19:10.180 --> 00:19:10.300 +Right. + +00:19:11.160 --> 00:19:13.180 +Yeah, there's some variations across type checkers, + +00:19:13.270 --> 00:19:15.240 +which we'll get to later. + +00:19:16.020 --> 00:19:18.080 +I think, though, before we move off this, + +00:19:18.680 --> 00:19:21.740 +there's actually off introducing the typing council. + +00:19:21.900 --> 00:19:24.560 +I think we should point out that there's two other folks + +00:19:24.590 --> 00:19:27.460 +who couldn't be here who are also on the typing council, + +00:19:27.730 --> 00:19:32.360 +Eric Trout and Yuka Letosilo. + +00:19:33.020 --> 00:19:34.360 +Sorry, Yuka. + +00:19:35.540 --> 00:19:39.120 +But I want to make sure that we point out there's actually five people, not just the three of you, right? + +00:19:41.420 --> 00:19:42.940 +How do you get on the council? + +00:19:43.800 --> 00:19:44.540 +Is there an election? + +00:19:44.940 --> 00:19:46.200 +Do you just apply? + +00:19:51.880 --> 00:19:53.400 +Vacancies are filled by the members themselves. + +00:19:53.680 --> 00:20:00.460 +So when somebody declares the intention to leave the council, we basically ask for people who are interested and then make a selection. + +00:20:01.840 --> 00:20:05.040 +Generally, we try to get people who have experience in the type system. + +00:20:05.840 --> 00:20:09.060 +We try to get a good cross-representation of people working on different type checkers. + +00:20:11.200 --> 00:20:17.040 +So we have Carl and Rebecca here who work on two type checkers, ty and Pyrefly. + +00:20:18.020 --> 00:20:19.380 +Yuka works on mypy. + +00:20:19.530 --> 00:20:22.920 +Eric Chart works on Pyrites, which are two of the most widely used type checkers. + +00:20:25.580 --> 00:20:30.000 +Yeah, so we try to get wider representation of people working on those parts of the ecosystem. + +00:20:30.180 --> 00:20:31.400 +Yeah, that's really cool about it. + +00:20:31.740 --> 00:20:34.760 +you know, it's got a bias towards finding people actually doing the work. + +00:20:36.400 --> 00:20:41.020 +So let's talk about the specification project at typing.python.org. + +00:20:42.300 --> 00:20:43.880 +What is, what is this here? + +00:20:50.320 --> 00:20:51.520 +I mean, I can. + +00:20:51.980 --> 00:20:52.500 +Yeah, go for it. + +00:20:53.060 --> 00:20:58.879 +Talk a bit about, I guess it's a specification for how the type system + +00:20:59.320 --> 00:21:04.500 +used to work. The way it started was that, Yala, you basically took all the typing peps and like + +00:21:04.900 --> 00:21:12.240 +stapled them together, right, to make like one long doc. And since then, we've been iterating on it, + +00:21:12.620 --> 00:21:19.180 +filling in parts that were missing, like overload evaluation and making other changes as well. + +00:21:20.260 --> 00:21:28.219 +Yeah, it's tricky, right? Because traditionally, the typing system is kind of defined across a + +00:21:28.220 --> 00:21:29.480 +series of peps, right? + +00:21:30.940 --> 00:21:33.280 +And so what is the document that tells you how it works, right? + +00:21:34.800 --> 00:21:38.760 +Yeah, that is hard because often those peps build on top of each other. + +00:21:39.900 --> 00:21:45.320 +So then in the extreme, you might see like one thing in one PEP and then another PEP that + +00:21:45.420 --> 00:21:47.640 +adds an aspect of it, another one that adds another aspect. + +00:21:49.080 --> 00:21:50.500 +And overall it makes it very hard to follow. + +00:21:51.210 --> 00:21:54.340 +One of the things I did recently was rewrite the typed dict spec. + +00:21:55.180 --> 00:22:01.860 +dict is a feature of the python type system that has been added to a lot from one PEP to another + +00:22:02.720 --> 00:22:07.280 +and i ended up rewriting the whole thing to basically put all those features together in + +00:22:07.740 --> 00:22:12.800 +a coherent whole rather than just having them all copy pasted one after the other + +00:22:14.220 --> 00:22:20.519 +very nice okay so if somebody really wants a good understanding of the python type system + +00:22:21.360 --> 00:22:23.160 +They go to typing.python.org. + +00:22:24.300 --> 00:22:26.440 +You know, one thing I think maybe is worth touching on, + +00:22:26.660 --> 00:22:30.100 +it's just kind of out of the blue a bit, + +00:22:30.260 --> 00:22:32.080 +but I think it's a really interesting aspect + +00:22:32.480 --> 00:22:35.260 +of the Python typing system is the, + +00:22:35.630 --> 00:22:37.940 +what is it called, the numerical tower or the number tower, + +00:22:39.080 --> 00:22:42.020 +where it's like, if I have a number, + +00:22:42.490 --> 00:22:44.360 +I could specify it as an int, + +00:22:44.840 --> 00:22:46.440 +or I could specify it as a float, + +00:22:47.320 --> 00:22:48.280 +and those kinds of things. + +00:22:49.760 --> 00:22:55.460 +But do you really need to say it's a int pipe float + +00:22:55.640 --> 00:22:58.340 +or a union of int and float if it could be either, right? + +00:22:58.460 --> 00:22:59.700 +And what is it called? + +00:22:59.880 --> 00:23:01.220 +It's the numerical tower, right? + +00:23:04.240 --> 00:23:05.660 +There are many different-- yeah, there + +00:23:05.660 --> 00:23:06.640 +are different towers too. + +00:23:06.800 --> 00:23:09.080 +In Python, there's also this thing called a numbers module + +00:23:09.360 --> 00:23:11.280 +that you have there. + +00:23:11.920 --> 00:23:13.820 +That's just basically ignored by the type system. + +00:23:15.160 --> 00:23:16.640 +I think it's-- + +00:23:17.539 --> 00:23:18.160 +I don't know. + +00:23:19.000 --> 00:23:20.160 +It's been useful for some people. + +00:23:20.220 --> 00:23:23.860 +I feel like in general that module just hasn't worked out very well as being very useful. + +00:23:25.420 --> 00:23:29.860 +Yeah, I think the item, for some reason I'm failing to find this in real time. + +00:23:29.880 --> 00:23:35.400 +But I think the interesting aspect is that you can say it's a float, + +00:23:35.500 --> 00:23:41.000 +and that's basically equivalent to union of integer and float and so on, right? + +00:23:41.200 --> 00:23:44.320 +I think the typing numbers in Python is pretty interesting. + +00:23:46.160 --> 00:23:52.640 +I think every type checker has a different interpretation of what a float annotation actually means. + +00:23:53.520 --> 00:23:54.100 +Oh, really? + +00:23:54.360 --> 00:23:57.660 +It's an area of some lack of clarity in the spec. + +00:23:59.300 --> 00:24:00.520 +Yeah, and a lot of contentiousness. + +00:24:01.600 --> 00:24:05.760 +If we could go back in time, I would... + +00:24:05.760 --> 00:24:09.460 +Knowing what I know now, I'd probably advocate for things being done differently. + +00:24:09.720 --> 00:24:16.120 +because like beginning, you know, like there were multiple things like with similar flavor, + +00:24:16.360 --> 00:24:22.020 +like there was also one where you could put, give some, give a parameter a non-none annotation + +00:24:22.270 --> 00:24:24.420 +and default it to none for convenience. + +00:24:24.730 --> 00:24:28.820 +And we've largely like moved away from stuff like that in favor of explicitness. + +00:24:29.600 --> 00:24:33.460 +Yeah, what the current spec says is that basically if you have a function that takes a float, + +00:24:34.040 --> 00:24:35.540 +you're also allowed to pass an int. + +00:24:36.560 --> 00:24:39.160 +But that's not really enough. + +00:24:39.400 --> 00:24:41.940 +It doesn't tell you how these things work in all cases. + +00:24:43.640 --> 00:24:47.560 +And we've had some attempts to try to come up with a way + +00:24:47.780 --> 00:24:52.540 +to specify that special case in a way that makes more sense, + +00:24:52.880 --> 00:24:53.840 +at least makes more sense to me. + +00:24:54.900 --> 00:24:55.820 +It's been very contentious. + +00:24:56.060 --> 00:24:57.700 +People have very strong opinions about this. + +00:24:59.200 --> 00:24:59.640 +Yeah. + +00:25:02.300 --> 00:25:03.760 +It is a little weird, isn't it? + +00:25:04.420 --> 00:25:05.020 +It's quite weird. + +00:25:05.320 --> 00:25:07.860 +I guess non-obvious is what I'd like to say, really, + +00:25:08.200 --> 00:25:19.360 +honestly. So I'd like to get the official counsel's thoughts on this. When, when is too much, + +00:25:19.600 --> 00:25:25.960 +when is typing too much typing, right? I made the joke about C++ ATL. If you've ever worked with + +00:25:26.080 --> 00:25:31.520 +that, it's like a parameter, you know, a template class where templated classes are part of the + +00:25:31.700 --> 00:25:36.319 +concrete type of the template. Like it's just off the hook. There's certainly places where + +00:25:36.800 --> 00:25:45.800 +typing can be too much. And a lot of the purity of Python or the readability of Python is the fact + +00:25:45.940 --> 00:25:52.380 +that it's got so few symbols. And so adding types adds context, but it also takes away, + +00:25:53.080 --> 00:25:58.560 +you know, makes it a little harder to read. When is too much typing? When do you recommend typing? + +00:25:58.860 --> 00:26:03.640 +You know, maybe Rebecca, I'll let you go first, but like, what are your thoughts in the sort of, + +00:26:03.860 --> 00:26:06.120 +How much typing should I use in Python? + +00:26:10.200 --> 00:26:14.600 +I'll give you what is sort of my official stance, + +00:26:14.720 --> 00:26:17.240 +which is that if you want your type checker to work well, + +00:26:17.900 --> 00:26:20.700 +you should type annotate your API boundaries. + +00:26:21.320 --> 00:26:23.520 +So like parameters and returns in public functions, + +00:26:24.040 --> 00:26:26.460 +public class attributes, things like that. + +00:26:26.720 --> 00:26:28.680 +And like even things that seem trivial, + +00:26:28.900 --> 00:26:31.500 +like, oh, this function returns none, + +00:26:32.180 --> 00:26:37.500 +better to annotate it because you know someone else might be depending on your library and + +00:26:37.780 --> 00:26:46.400 +consuming that type of information. I will say personally what I tend to do is like I annotate + +00:26:46.460 --> 00:26:53.980 +things that I think are non-trivial because I want to see that as documentation and if something + +00:26:54.640 --> 00:26:59.119 +like you know a function that does return none to be honest I will probably forget to annotate it + +00:26:59.180 --> 00:27:03.240 +half the time because i'll be like i honestly don't need to see it so + +00:27:06.580 --> 00:27:14.300 +yeah you know one of the interesting features of the pyrefly vs code extension that's the only one + +00:27:14.340 --> 00:27:18.860 +i can speak of at the moment and carl you've got to tell me if the ty one does this as well + +00:27:19.480 --> 00:27:27.780 +is it will sort of overlay its belief of what types are like if there's you say x equals a + +00:27:27.780 --> 00:27:30.060 +a function return value and it knows what the function returns, + +00:27:30.260 --> 00:27:33.540 +it'll have a gray like colon int if it returned an int or something. + +00:27:33.940 --> 00:27:37.980 +So you can read the code and see what the types are + +00:27:38.160 --> 00:27:40.980 +without actually putting it into the text of the code. + +00:27:41.160 --> 00:27:42.100 +It's only within the editor. + +00:27:43.400 --> 00:27:44.580 +Does ty do something like that, Carl? + +00:27:45.440 --> 00:27:47.360 +Yes, we also have inlay type ints. + +00:27:47.660 --> 00:27:48.500 +Yeah, inlay type ints. + +00:27:48.560 --> 00:27:49.000 +That's what it's called. + +00:27:49.240 --> 00:27:51.040 +So yeah, I don't know. + +00:27:51.140 --> 00:27:53.620 +That also brings an interesting challenge-- not a challenge, + +00:27:53.780 --> 00:27:56.980 +like a wrinkle to the recommendation of, + +00:27:59.470 --> 00:28:02.220 +should I put types on like the return value + +00:28:02.350 --> 00:28:03.960 +because I want to know that's a list of user, + +00:28:04.180 --> 00:28:06.900 +not a list of user IDs or whatever, for example, + +00:28:07.080 --> 00:28:08.100 +like a list of UUID. + +00:28:09.920 --> 00:28:12.680 +But if it's going to show up anyway in the editor, + +00:28:13.460 --> 00:28:15.020 +maybe I don't have to write that, right? + +00:28:15.070 --> 00:28:18.560 +And so that becomes sort of somewhere + +00:28:18.610 --> 00:28:20.200 +where you could debate again, I think. + +00:28:20.640 --> 00:28:23.360 +However, I do 100% agree with you, Rebecca, + +00:28:23.420 --> 00:28:25.260 +that put it on your API boundaries. + +00:28:25.540 --> 00:28:30.260 +If this is the place that people get into some part of your code + +00:28:30.420 --> 00:28:33.020 +and they don't know or want to know about the inside of it, + +00:28:33.820 --> 00:28:36.580 +having types there is really helpful both for editors, + +00:28:37.660 --> 00:28:39.760 +for type checkers, and just for reading code, + +00:28:39.940 --> 00:28:42.120 +and even for AI, which is a crazy world. + +00:28:43.100 --> 00:28:43.180 +Yeah. + +00:28:43.630 --> 00:28:45.240 +Carl, what are your thoughts here? + +00:28:45.320 --> 00:28:46.540 +How much typing is too much typing? + +00:28:47.160 --> 00:28:48.460 +What's the guidelines here? + +00:28:49.600 --> 00:28:52.840 +I think I agree with Rebecca's answer. + +00:28:52.980 --> 00:29:02.260 +I mean, that one place you definitely want to have explicit type annotations is that API boundaries, public API of a library, et cetera. + +00:29:03.260 --> 00:29:22.460 +In terms of what's too much typing, I mean, there's certainly patterns that have historically been used in Python that we still can't express well in the type system or that require extremely complex type annotations to express well. + +00:29:22.540 --> 00:29:25.760 +And I think there it becomes a judgment call. + +00:29:27.380 --> 00:29:31.300 +If it's like a core widely used API, + +00:29:31.790 --> 00:29:33.340 +you may get a lot of benefit + +00:29:33.960 --> 00:29:36.360 +from some very complex and verbose annotations. + +00:29:37.280 --> 00:29:40.320 +And so then it's worth sort of going through that pain + +00:29:40.480 --> 00:29:43.400 +and the pain of adding them and of reading them + +00:29:43.610 --> 00:29:46.320 +in order to get that additional typing coverage + +00:29:46.600 --> 00:29:47.680 +everywhere you use that API. + +00:29:48.860 --> 00:29:51.579 +If it's much less frequently used code + +00:29:51.600 --> 00:29:54.660 +that's highly dynamic, maybe it's not worth it in that case. + +00:29:55.290 --> 00:29:56.740 +I think there's a lot of judgment calls here. + +00:29:57.180 --> 00:29:59.600 +What about one-off scripts? + +00:30:01.260 --> 00:30:04.080 +I'm going to write this thing to just move this data from here to there. + +00:30:04.220 --> 00:30:05.520 +Once it's moved, I don't need it again. + +00:30:06.920 --> 00:30:07.900 +We're done with that old system. + +00:30:07.990 --> 00:30:08.860 +We're going to the new one. + +00:30:09.900 --> 00:30:10.580 +Maybe less typing. + +00:30:12.680 --> 00:30:14.440 +Yeah, I think that's what's useful for you. + +00:30:15.440 --> 00:30:17.640 +Often I feel like one-off scripts are not really one-off. + +00:30:17.780 --> 00:30:19.680 +Maybe you want to move some similar data later, + +00:30:20.000 --> 00:30:22.080 +And then it's useful if you can understand your code again, + +00:30:22.230 --> 00:30:23.880 +if you want to read what you did. + +00:30:24.600 --> 00:30:26.020 +You thought you didn't need it again. + +00:30:26.180 --> 00:30:27.660 +And all of a sudden, at six months old, + +00:30:27.710 --> 00:30:28.620 +you don't understand it. + +00:30:29.310 --> 00:30:30.560 +And the types would help a lot, right? + +00:30:31.420 --> 00:30:31.480 +Yeah. + +00:30:32.180 --> 00:30:32.360 +Yeah. + +00:30:33.120 --> 00:30:35.020 +Jelal, what's your advice? + +00:30:35.640 --> 00:30:38.140 +Yeah, I think what Kar and Rebecca said + +00:30:38.340 --> 00:30:38.980 +makes sense to me too. + +00:30:39.930 --> 00:30:44.380 +I think types have advantages in terms of documenting human + +00:30:44.560 --> 00:30:48.279 +readers, what is going on, and in terms of catching mistakes + +00:30:48.300 --> 00:30:50.700 +that otherwise would not be called until runtime, perhaps. + +00:30:51.660 --> 00:30:53.840 +They have costs in maybe making your code harder to reach + +00:30:53.890 --> 00:30:55.020 +if there's too much going on. + +00:30:56.020 --> 00:30:58.760 +So add types as long as those benefits outweigh the costs. + +00:31:01.080 --> 00:31:01.600 +Yeah. + +00:31:02.210 --> 00:31:07.980 +I mean, do you recommend to anyone that they just 100% go full + +00:31:08.370 --> 00:31:12.540 +like C++, C# on it and just type it every single thing? + +00:31:13.720 --> 00:31:16.580 +Is there an advantage for static type checkers, + +00:31:17.140 --> 00:31:20.740 +like mypy type stuff you can run across and get that. + +00:31:20.740 --> 00:31:23.200 +I mean, you could do that with PowerFly or ty and the CLI + +00:31:23.200 --> 00:31:23.540 +as well. + +00:31:23.720 --> 00:31:27.360 +But thinking more mypy is like kind of being real strict + +00:31:27.520 --> 00:31:28.200 +on some of that stuff. + +00:31:29.220 --> 00:31:32.000 +Personally, I do tend to annotate almost all function + +00:31:32.260 --> 00:31:35.440 +parameters and if class attributes, if I make a class. + +00:31:37.900 --> 00:31:39.340 +Sometimes it's not as necessary. + +00:31:39.640 --> 00:31:41.900 +Like you don't need to annotate your tests, perhaps, + +00:31:42.200 --> 00:31:44.240 +or you don't need to annotate internal functions as much. + +00:31:45.520 --> 00:31:48.460 +But for my own coding, I usually find it helpful to do that. + +00:31:50.680 --> 00:31:53.520 +But sometimes I see people annotating even local variables + +00:31:53.860 --> 00:31:55.920 +where it's very obvious to type check what the type is, + +00:31:56.220 --> 00:31:57.420 +and they can just infer it reliably. + +00:31:58.190 --> 00:31:59.960 +And then it really just adds noise, + +00:32:00.180 --> 00:32:01.000 +and you shouldn't do it. + +00:32:02.140 --> 00:32:02.840 +Yeah, exactly. + +00:32:02.980 --> 00:32:05.380 +If you've got a function that's annotated with a return value, + +00:32:05.430 --> 00:32:07.760 +and you say x equals the function call, + +00:32:08.380 --> 00:32:09.900 +then the type checkers can infer that. + +00:32:10.100 --> 00:32:14.880 +And you're just causing extra noise, I guess. + +00:32:17.920 --> 00:32:21.420 +So suppose you all want to change something. + +00:32:22.820 --> 00:32:25.860 +What's the process of actually going through and making some changes? + +00:32:30.020 --> 00:32:32.500 +Yeah, there's mostly sort of two levels of this. + +00:32:34.460 --> 00:32:35.600 +Well, maybe there's even three levels. + +00:32:35.740 --> 00:32:40.160 +The first one is if it's something that's so small that's just like a wording clarification or something, + +00:32:40.160 --> 00:32:45.140 +We just make a PR to the repo and a few of us look at it and we change it. + +00:32:45.920 --> 00:32:51.680 +The second level is when it's sort of a smaller change that doesn't really introduce a new + +00:32:51.880 --> 00:32:58.600 +feature and then we make a PR to the typing spec repo and we formally have all of us sign + +00:32:58.600 --> 00:32:59.020 +off on it. + +00:32:59.900 --> 00:33:04.480 +That's what happens like with what Carl mentioned earlier of the final change in data classes. + +00:33:06.660 --> 00:33:07.100 +Nice. + +00:33:10.720 --> 00:33:12.520 +I just had to merge this one. + +00:33:14.980 --> 00:33:15.780 +I love it. + +00:33:16.260 --> 00:33:19.140 +Yeah, I guess this repo itself doesn't have anything. + +00:33:20.740 --> 00:33:24.080 +It's the Python typing repo where the decisions are made. + +00:33:24.840 --> 00:33:28.780 +The typing council just has some documentation. + +00:33:29.880 --> 00:33:30.340 +Maybe this one. + +00:33:30.350 --> 00:33:30.480 +Yeah. + +00:33:30.990 --> 00:33:32.200 +And then the third level is peps. + +00:33:32.270 --> 00:33:33.540 +Like really big new changes. + +00:33:33.650 --> 00:33:34.700 +You can still write a pep. + +00:33:35.420 --> 00:33:38.780 +And then we make a recommendation in the steering code so it makes a decision eventually. + +00:33:39.800 --> 00:33:40.140 +Okay. + +00:33:42.840 --> 00:33:50.240 +So if I wanted to suggest something, I could come up here and I could open up an issue, + +00:33:50.500 --> 00:33:55.080 +maybe start a conversation on typing, Python/typing. + +00:33:56.120 --> 00:33:58.300 +And you can make a pull request to change the spec. + +00:33:59.560 --> 00:33:59.660 +Okay. + +00:33:59.920 --> 00:34:07.600 +and so the pull request would not be to change the code like how python maybe interprets code that + +00:34:07.740 --> 00:34:12.899 +has this new thing but to suggest that the spec has it which then would start a process that + +00:34:13.260 --> 00:34:18.480 +ultimately might make c python understand it right well c python itself probably doesn't do anything + +00:34:18.620 --> 00:34:25.020 +with it um i guess most of the things that go directly here are changes to how to interpret + +00:34:25.040 --> 00:34:26.780 +the things that are already in CPython. + +00:34:27.659 --> 00:34:29.659 +If it's adding something new, it will usually + +00:34:29.899 --> 00:34:32.440 +need to go through a path, except if it's something very small. + +00:34:34.380 --> 00:34:37.200 +Yeah, I guess maybe let's talk about that for a minute. + +00:34:37.300 --> 00:34:42.340 +We got two representatives here of the newer breed of tools. + +00:34:44.020 --> 00:34:48.780 +What's the story for inconsistencies + +00:34:50.100 --> 00:34:51.760 +across interpretations of the spec? + +00:34:51.919 --> 00:34:53.280 +I know that there's slight variations. + +00:34:54.520 --> 00:34:58.480 +I've also, you know, not putting either of you on the spotlight, + +00:34:58.760 --> 00:35:04.320 +but like using, say, PyCharm and like writing code. + +00:35:04.320 --> 00:35:09.120 +So it's type checkers happy and then using something like Pyright. + +00:35:09.560 --> 00:35:14.140 +And so it has a real different interpretation of what you should let slide + +00:35:14.140 --> 00:35:14.760 +and what you shouldn't. + +00:35:14.780 --> 00:35:23.040 +I feel like Pyright is much more focused on like enforcing the nullability + +00:35:23.040 --> 00:35:27.720 +or the lack thereof, and it warns of inconsistencies there + +00:35:27.900 --> 00:35:29.460 +where PyCharm doesn't seem to care as much. + +00:35:30.060 --> 00:35:32.120 +I don't know which one I like better, but I know they're different. + +00:35:32.400 --> 00:35:34.160 +And if I write code in one, then I open the other. + +00:35:34.230 --> 00:35:35.680 +I'm like, huh, why is it upset? + +00:35:35.920 --> 00:35:37.040 +It seemed like it was fine. + +00:35:40.020 --> 00:35:40.960 +How do you all navigate this? + +00:35:41.240 --> 00:35:41.300 +Yeah. + +00:35:41.740 --> 00:35:45.260 +I think one thing useful to say about the spec there + +00:35:45.290 --> 00:35:48.000 +is that the spec covers a lot of things. + +00:35:48.250 --> 00:35:50.440 +In particular, it tends to cover sort of the details + +00:35:50.760 --> 00:35:52.500 +of more advanced type system features. + +00:35:53.420 --> 00:35:58.940 +but there's a lot of very fundamental stuff about how a type checker works in terms of + +00:35:59.620 --> 00:36:04.840 +how it does inference and how it does type narrowing. And even in some cases, like you + +00:36:04.960 --> 00:36:10.240 +mentioned, what it chooses to emit errors on that isn't really covered by the spec, + +00:36:11.320 --> 00:36:18.540 +partly maybe because we haven't gotten to it and also partly intentionally in that there may be + +00:36:18.860 --> 00:36:22.640 +room in some of those cases for different type checkers to work differently if they're serving + +00:36:22.660 --> 00:36:29.400 +needs. Like if PyCharm is primarily concerned about being a useful kind of IDE and providing + +00:36:29.580 --> 00:36:36.160 +go-to definition and that sort of thing, maybe emitting lots of warnings or errors and all kinds + +00:36:36.160 --> 00:36:41.260 +of things where your code might be doing something wrong isn't as high a priority and another type + +00:36:41.420 --> 00:36:50.640 +checker might have a different priority. One thing I want to mention is that it may not + +00:36:50.660 --> 00:36:58.280 +seem like it, but things are already much better than they used to be. Like previously I worked + +00:36:58.660 --> 00:37:05.000 +on different type checker called PyType. And at that time it was, you know, sort of the wild west. + +00:37:05.130 --> 00:37:10.480 +Like we want to know how other type checkers like do something. Well, you know, like open up the mypy + +00:37:10.560 --> 00:37:17.539 +playground, open up the PyRite playground, see what tells you. Now we at least have spec and + +00:37:17.560 --> 00:37:27.740 +conformance tests. Yeah, that's really cool. How much would you say that your two type checkers + +00:37:28.380 --> 00:37:34.980 +maybe bring in mypy as well? How much do they agree versus disagree? You only see the differences. + +00:37:35.100 --> 00:37:39.400 +You don't see in which ways that they are the same as a consumer of them so much, right? You're + +00:37:39.480 --> 00:37:45.920 +like, why is this one squiggly when it wasn't squiggly before? But how similar or different are + +00:37:48.280 --> 00:37:53.060 +I don't know how we would quantify that there. I think there's a lot that is the same just because + +00:37:53.820 --> 00:37:59.380 +it's based on how Python actually works. We're both trying to model the same language. And then + +00:37:59.440 --> 00:38:03.020 +there's certainly also plenty of differences or things that we handle differently. So + +00:38:04.880 --> 00:38:15.900 +quantify that. Yeah. I agree. It's hard to quantify, suppose, talk a bit abstractly about + +00:38:15.920 --> 00:38:20.320 +various like type checkers philosophies like with Pyrefly, + +00:38:20.380 --> 00:38:23.100 +you know, like we really try to do a lot of type inference. + +00:38:23.400 --> 00:38:24.840 +So in that way in which, you know, + +00:38:24.940 --> 00:38:27.400 +like we intentionally diverge a bit from myPy, + +00:38:27.640 --> 00:38:30.200 +but like other than like that deliberate decision, + +00:38:30.440 --> 00:38:32.660 +if we see ways in which we are like accidentally different, + +00:38:32.940 --> 00:38:37.000 +you know, we do try to fix that because otherwise people + +00:38:37.060 --> 00:38:39.440 +would have a hard time like running multiple type checkers + +00:38:39.520 --> 00:38:40.000 +or migrating. + +00:38:41.180 --> 00:38:45.200 +- Yeah, differences obviously cause pain for users + +00:38:45.220 --> 00:38:46.520 +who are using multiple type checkers + +00:38:46.680 --> 00:38:48.240 +or writing libraries that need to support + +00:38:48.480 --> 00:38:49.260 +multiple type checkers. + +00:38:49.400 --> 00:38:51.020 +So like Rebecca said, it's like, + +00:38:51.560 --> 00:38:53.780 +if we are different from other type checkers, + +00:38:53.780 --> 00:38:55.160 +we wanna be sure that there's a good reason + +00:38:55.800 --> 00:38:56.280 +for that difference. + +00:39:02.400 --> 00:39:03.640 +Oh, lost your audio, Michael. + +00:39:05.120 --> 00:39:05.760 +- Right, sorry. + +00:39:06.620 --> 00:39:11.100 +The difference should be because of philosophical choice, + +00:39:11.420 --> 00:39:14.800 +not just you happen to have chosen slightly differently, right? + +00:39:18.520 --> 00:39:22.560 +Yeah, and it's not just people who run different type checkers. + +00:39:23.640 --> 00:39:26.340 +Like you pointed out, Carl, a lot of times it is if I have a library + +00:39:27.660 --> 00:39:30.500 +and then different people want to consume that library, + +00:39:30.620 --> 00:39:33.160 +then their type checker may or may not warn them + +00:39:33.400 --> 00:39:36.640 +about how my library declares its types and so on. + +00:39:37.560 --> 00:39:39.520 +I'll give you a real quick example. + +00:39:41.080 --> 00:39:43.800 +I have a, I can't remember which one it was, + +00:39:43.880 --> 00:39:50.960 +I have three or four different open source libraries that I've created that somehow work with creating, + +00:39:51.980 --> 00:39:55.780 +basically passing data to templates in web apps. + +00:39:56.420 --> 00:40:02.220 +So one is like, I want to use the Chameleon web template framework, but with FastAPI or with Flask. + +00:40:02.380 --> 00:40:04.680 +And there's some other variations like partials and so on. + +00:40:04.920 --> 00:40:06.900 +I can't remember which one, but it doesn't really matter. + +00:40:07.000 --> 00:40:10.000 +One of them decorated a Flask. + +00:40:10.520 --> 00:40:12.900 +I think it was a Flask, especially makes it irrelevant. + +00:40:13.040 --> 00:40:21.220 +a flask endpoint and pyrite was really upset like the error message filled the entire page of how it + +00:40:21.280 --> 00:40:27.200 +was inconsistent with what it expected for the definition of the flask view method i'm like + +00:40:27.620 --> 00:40:33.500 +no one is going to call this like what does it even matter what this type is like it still runs + +00:40:33.740 --> 00:40:40.020 +fine the runtime is fine you know it's no problem with this decorator it worked fine but something + +00:40:40.040 --> 00:40:47.360 +about the way that the flask at get returned the type versus what my thing returned varied in like + +00:40:47.360 --> 00:40:54.560 +a really slight way. I didn't care. But somebody was using some editor that used pyrite and they're + +00:40:54.620 --> 00:40:58.880 +like, you have to help fix this. I can't take all these warnings that are huge and they're everywhere. + +00:40:59.120 --> 00:41:06.560 +Like, okay, I'll go fix it. Right. And I went and I put a way more effort than was justified into a + +00:41:06.560 --> 00:41:12.300 +function type that no one ever calls just to make the errors on some type checker I didn't use + +00:41:12.900 --> 00:41:16.220 +go away, right? And that's the kind of thing where it becomes just a headache. + +00:41:19.580 --> 00:41:24.000 +I don't know. I wish I remember. I probably got that written down in an issue somebody filed, + +00:41:24.180 --> 00:41:28.880 +but it was a gnarly error. Or if you're working on an open source project, you know, + +00:41:28.930 --> 00:41:33.840 +you can't make everybody use the same editor that wants to contribute on a big project. + +00:41:34.380 --> 00:41:36.200 +And so you might run into this variation as well. + +00:41:36.260 --> 00:41:37.060 +So there's a lot of cases. + +00:41:40.220 --> 00:41:43.100 +Yeah, it can be really difficult to make these decisions + +00:41:43.440 --> 00:41:49.660 +about what sorts of errors people want their type checker + +00:41:49.660 --> 00:41:51.620 +to catch or what's too pedantic. + +00:41:53.380 --> 00:41:55.080 +Because you want your type checker + +00:41:55.080 --> 00:41:59.120 +to catch non-obvious errors, not just the obvious ones + +00:41:59.180 --> 00:42:00.800 +that you probably would have seen by looking at the code + +00:42:00.900 --> 00:42:01.020 +yourself. + +00:42:03.780 --> 00:42:06.120 +But then there'll be cases where somebody says, + +00:42:06.120 --> 00:42:07.340 +well, I don't care, that's too pedantic. + +00:42:07.560 --> 00:42:10.120 +And it is difficult to make everyone happy. + +00:42:11.060 --> 00:42:11.700 +- Yeah, exactly. + +00:42:12.760 --> 00:42:13.960 +When I was seeing, I'm like, well, + +00:42:15.520 --> 00:42:18.480 +who decides what the right signature + +00:42:18.720 --> 00:42:21.440 +of a flask view and point should be like? + +00:42:21.620 --> 00:42:24.500 +If the framework can call it, it should be okay. + +00:42:24.920 --> 00:42:26.900 +There's not, just 'cause it had a decorator before, + +00:42:26.960 --> 00:42:28.820 +that doesn't mean that's the official structure. + +00:42:28.980 --> 00:42:29.300 +But anyway, + +00:42:32.120 --> 00:42:34.800 +I do think one of the bigger philosophical differences + +00:42:35.160 --> 00:42:39.560 +has to do around this concept of nullability. + +00:42:39.910 --> 00:42:42.320 +Do you guys call it nullability or noneability? + +00:42:42.600 --> 00:42:44.820 +Like, nullability comes from the other languages. + +00:42:44.950 --> 00:42:48.340 +And by that, I mean I can specify that I have an integer. + +00:42:48.770 --> 00:42:52.280 +And in the Python type system, it cannot be set to none, + +00:42:52.470 --> 00:42:53.580 +even though in the runtime it can. + +00:42:54.220 --> 00:42:57.560 +It has to be a concrete int type, unless you + +00:42:57.680 --> 00:43:01.240 +make it a optional int or an int pipe none, + +00:43:01.560 --> 00:43:02.880 +or one of those type things, right? + +00:43:03.120 --> 00:43:05.460 +And how strong that gets enforced + +00:43:05.760 --> 00:43:09.000 +seems to be one of the biggest difference of opinions + +00:43:09.700 --> 00:43:10.520 +that I've seen around. + +00:43:10.900 --> 00:43:11.980 +How do you all think about that? + +00:43:15.200 --> 00:43:17.620 +- That's interesting to me that that's your experience + +00:43:17.900 --> 00:43:20.640 +because my experience has been that that's actually an area + +00:43:20.710 --> 00:43:24.060 +where everyone seems to agree as far as I can tell + +00:43:24.660 --> 00:43:26.080 +that these are- - Interesting. + +00:43:26.080 --> 00:43:27.420 +- Is an important source of bugs + +00:43:27.420 --> 00:43:28.500 +and it's better to catch them. + +00:43:28.500 --> 00:43:29.460 +So I think all of the type checkers, + +00:43:30.080 --> 00:43:31.500 +Maybe you said PyCharm doesn't? + +00:43:31.640 --> 00:43:33.640 +I don't think PyCharm does that. + +00:43:33.840 --> 00:43:38.740 +I'm pretty sure it doesn't because I + +00:43:38.980 --> 00:43:40.840 +agree that it's an important thing to check, + +00:43:41.940 --> 00:43:44.540 +but it's also a point of a lot of friction. + +00:43:45.580 --> 00:43:48.400 +And by that, I mean, let's suppose + +00:43:48.700 --> 00:43:52.740 +I'm going to have a class that I need to create an instance of + +00:43:53.060 --> 00:43:54.600 +and then put values into. + +00:43:55.380 --> 00:43:57.540 +And I know once I put the values into it, + +00:43:57.560 --> 00:43:58.860 +let's say it has a user ID. + +00:43:59.900 --> 00:44:02.840 +I know for certain that that's going to be an integer. + +00:44:03.560 --> 00:44:08.400 +So I'd like to say userid colon int because everywhere I use that object later, + +00:44:09.440 --> 00:44:13.740 +if it's a function that takes an int and I specify it as optional int, + +00:44:15.060 --> 00:44:19.540 +I will get a type check warning every single call site when I try to pass that. + +00:44:20.060 --> 00:44:26.460 +But I know from the semantics of the behavior that it's going to always be an int + +00:44:27.320 --> 00:44:28.760 +unless it's not initialized. + +00:44:29.500 --> 00:44:32.220 +like in this short period where I want to create it. + +00:44:32.270 --> 00:44:33.800 +So I can't set the type to int. + +00:44:33.810 --> 00:44:35.960 +I have to set to optional int until I've loaded it. + +00:44:36.870 --> 00:44:38.500 +But there's like this, I don't know, + +00:44:38.620 --> 00:44:41.380 +that's the part where I see a lot of it show up + +00:44:42.319 --> 00:44:46.320 +is inconsistencies and then warnings all over the place. + +00:44:46.720 --> 00:44:49.060 +So I'm like, well, but that function is actually checking + +00:44:49.130 --> 00:44:52.240 +if it's none and it'll return null, you know, none + +00:44:52.410 --> 00:44:53.120 +or something like that. + +00:44:53.280 --> 00:44:54.460 +So I don't know. + +00:44:54.920 --> 00:44:56.200 +I totally agree with you. + +00:44:56.480 --> 00:45:02.540 +It's just somewhere I've seen the most inconsistencies across maybe PyCharm versus others. + +00:45:03.940 --> 00:45:09.640 +mypy also has a legacy mode for not checking none things called non-strict optional. + +00:45:10.860 --> 00:45:15.700 +We are trying to get rid of that from mypy because strict optional, + +00:45:16.000 --> 00:45:18.000 +like being strict about it is the more sensible thing to do. + +00:45:18.860 --> 00:45:19.000 +Sure. + +00:45:19.180 --> 00:45:21.120 +But it's possible that you've seen that too. + +00:45:21.580 --> 00:45:22.060 +Yeah, I agree. + +00:45:22.320 --> 00:45:29.200 +So what you mentioned is maybe sort of a special case of the case where you pass something to a class and there's initialization that changes the types. + +00:45:30.120 --> 00:45:31.540 +Doesn't necessarily have to deal with none. + +00:45:31.570 --> 00:45:34.480 +It could also just be like the attribute doesn't exist at the little beforehand or something. + +00:45:35.220 --> 00:45:35.360 +Yeah. + +00:45:35.500 --> 00:45:35.720 +Yeah. + +00:45:36.040 --> 00:45:37.660 +We don't have a good solution for that. + +00:45:37.820 --> 00:45:41.400 +Maybe there's room for something to support that use case better. + +00:45:41.720 --> 00:45:42.880 +I don't know what it would look like. + +00:45:44.200 --> 00:45:44.980 +Yeah, I don't know. + +00:45:45.240 --> 00:45:54.320 +In some cases, there's ways you can, these things can sometimes nudge you towards a different design that is actually safer and will avoid errors. + +00:45:54.540 --> 00:46:03.640 +Like in the kind of case you're talking about, you know, is it actually necessary that an uninitialized object and an initialized one are represented by the same type? + +00:46:04.400 --> 00:46:11.660 +Or is there a way to adjust the API so that those are actually different types than you solve the problem and your code is safer? + +00:46:11.760 --> 00:46:16.620 +So yeah, no, I totally hear you. + +00:46:16.730 --> 00:46:21.560 +You know, I'm thinking like you submit a web form and before you, + +00:46:22.000 --> 00:46:22.820 +before you parse it, + +00:46:22.860 --> 00:46:24.480 +you've got to create the instance to set the values. + +00:46:26.300 --> 00:46:28.800 +And I don't know, it's, it's not worth diving into, + +00:46:28.880 --> 00:46:33.780 +but I do find this differentiation between like the strict enforcement of none + +00:46:34.030 --> 00:46:36.000 +versus not none. I think it's powerful. + +00:46:36.050 --> 00:46:39.020 +And I do think you all are right that it does catch a lot of errors. It's just, + +00:46:39.120 --> 00:46:42.700 +It's just a difference, and it's just an interesting choice. + +00:46:43.100 --> 00:46:46.240 +But I didn't get a concrete answer from the official counsel. + +00:46:47.160 --> 00:46:48.440 +Nullable or noneable? + +00:46:49.180 --> 00:46:49.540 +What is it? + +00:46:51.140 --> 00:46:53.200 +I feel like you just don't really even talk about it + +00:46:53.220 --> 00:46:53.980 +as a term, mostly. + +00:46:55.860 --> 00:46:59.260 +None is special in the type system in how you represent it, + +00:46:59.440 --> 00:47:02.520 +but it's not really special in other ways. + +00:47:04.180 --> 00:47:06.720 +So you don't have a term for int type none? + +00:47:08.120 --> 00:47:13.780 +to our none. Historically, the term was optional, although I think that term has problems and + +00:47:14.540 --> 00:47:20.620 +we're sort of moving away from it because specifically one problem is that optional can + +00:47:20.840 --> 00:47:26.960 +mean you don't have to pass it in, like I say, as a function parameter. Right. And that's not + +00:47:27.400 --> 00:47:32.240 +the intention. Yeah. It has two total different meanings there. Interesting. + +00:47:34.580 --> 00:47:37.680 +Let's talk a little bit about TypeShed. + +00:47:38.110 --> 00:47:39.840 +I think TypeShed is pretty interesting. + +00:47:40.020 --> 00:47:42.980 +Maybe people don't know too much about it. + +00:47:43.520 --> 00:47:46.240 +So I'm sure you all are familiar with this project, + +00:47:46.350 --> 00:47:50.220 +that you can basically add type information + +00:47:50.500 --> 00:47:53.180 +that the libraries didn't bother to include for you, right? + +00:47:57.240 --> 00:47:58.600 +What are thoughts on TypeShed? + +00:47:58.800 --> 00:48:03.900 +How much do you all lean on this to round out + +00:48:03.920 --> 00:48:04.580 +missing types? + +00:48:07.660 --> 00:48:11.720 +I mean, there are two parts of types shed, right? + +00:48:11.980 --> 00:48:18.380 +There's the standard library type stubs, which I think are invaluable, like all the type + +00:48:19.300 --> 00:48:20.100 +checkers use those. + +00:48:20.540 --> 00:48:24.960 +And I mean, will the standard library itself ever have inline types? + +00:48:25.860 --> 00:48:27.480 +Who knows, this might be around forever. + +00:48:28.300 --> 00:48:32.720 +Then there are also the third party stubs. + +00:48:33.080 --> 00:48:34.860 +And I think that's what you're describing. + +00:48:35.040 --> 00:48:36.700 +Their libraries that for whatever reason + +00:48:36.940 --> 00:48:38.640 +don't ship with stubs themselves. + +00:48:39.200 --> 00:48:41.260 +Those are in TypeShed. + +00:48:42.880 --> 00:48:46.260 +And it's been like, for a while, + +00:48:46.420 --> 00:48:48.260 +this has been a question of like what we want to do + +00:48:49.060 --> 00:48:51.380 +with like TypeShed third party stubs, right? + +00:48:51.680 --> 00:48:54.560 +'Cause like, ideally like libraries would ship + +00:48:54.590 --> 00:48:56.220 +with their own types, + +00:48:56.480 --> 00:48:58.040 +but there are various obstacles to that. + +00:49:00.020 --> 00:49:04.260 +- You know, the obstacles that I know of used to be like, + +00:49:04.920 --> 00:49:07.500 +we want this to run on Python 2 and Python 3, + +00:49:08.100 --> 00:49:10.940 +or we want it to run on Python 3.3 still. + +00:49:12.600 --> 00:49:14.020 +But it's been a long time + +00:49:14.740 --> 00:49:17.680 +since any non-type supporting version of Python + +00:49:20.920 --> 00:49:24.080 +was a real, you know, a supported type of thing, right? + +00:49:25.500 --> 00:49:27.800 +I mean, even 3.9 became deprecated. + +00:49:30.180 --> 00:49:33.880 +So on one hand, I feel like they could be merged in, + +00:49:33.880 --> 00:49:40.180 +but there's also a lot of other areas that are maybe we don't-- + +00:49:40.580 --> 00:49:42.360 +they're not common, right? + +00:49:42.540 --> 00:49:49.120 +Like other libraries like-- + +00:49:49.700 --> 00:49:49.880 +I don't know. + +00:49:49.920 --> 00:49:51.860 +Just pick some-- let's say Pyramid. + +00:49:52.200 --> 00:49:56.180 +I don't think the Pyramid Web Framework really ever got types added to it. + +00:49:56.980 --> 00:50:00.080 +Somebody could go and create a type shed stub or + +00:50:01.100 --> 00:50:05.180 +a types_pyramid you could pip install and then we'll add the types, right? + +00:50:06.430 --> 00:50:09.920 +I certainly see it being really valuable for third party things that are just not + +00:50:10.020 --> 00:50:12.380 +going to get the type attention they need. + +00:50:20.020 --> 00:50:20.680 +Yellow, what do you think? + +00:50:21.680 --> 00:50:22.620 +- Yeah, I think TypeShed is great. + +00:50:22.880 --> 00:50:25.360 +I've spent a lot of time on improving it. + +00:50:26.660 --> 00:50:28.460 +As Rebecca said, especially with a standard library, + +00:50:28.800 --> 00:50:29.760 +it's irreplaceable. + +00:50:30.880 --> 00:50:31.800 +For third party libraries, + +00:50:32.170 --> 00:50:34.960 +I think it's become less needed over time. + +00:50:36.640 --> 00:50:38.900 +It used to be that very few third party libraries + +00:50:39.200 --> 00:50:39.960 +had any types. + +00:50:40.680 --> 00:50:41.940 +Now that's obviously changed. + +00:50:41.950 --> 00:50:44.680 +A lot of libraries ship their own types. + +00:50:46.580 --> 00:50:49.900 +But still, there are quite a few types of libraries left + +00:50:49.920 --> 00:50:54.520 +where there aren't inline types and TypeShit can provide useful types. + +00:50:55.040 --> 00:50:59.780 +I think TypeShit also provides a service because it has a really great framework for testing these types. + +00:51:00.560 --> 00:51:11.540 +We have tools like step test and various type checkers that help to make sure these types are good and meet a high standards. + +00:51:12.760 --> 00:51:14.820 +So yeah, I think they're still useful for many libraries. + +00:51:15.980 --> 00:51:19.820 +Yeah, I was just looking at the types-flask, + +00:51:20.180 --> 00:51:22.940 +and I guess it must be gone + +00:51:23.090 --> 00:51:25.180 +because now typeflask must have it internally. + +00:51:25.580 --> 00:51:28.120 +So it's kind of an interim sort of thing. + +00:51:28.220 --> 00:51:28.640 +That's pretty cool. + +00:51:28.670 --> 00:51:30.740 +Yeah, in general, types has a policy + +00:51:30.810 --> 00:51:32.380 +that we remove the steps from typeset + +00:51:32.410 --> 00:51:34.460 +if they are in the library itself. + +00:51:35.359 --> 00:51:37.280 +Yeah, yeah, very cool. + +00:51:37.610 --> 00:51:39.600 +Okay, I find these super valuable + +00:51:39.780 --> 00:51:42.540 +because if there's a library I want to work with + +00:51:42.540 --> 00:51:44.720 +and it just doesn't have types for whatever reason, + +00:51:45.100 --> 00:51:48.620 +You can install stuff from here, and all of a sudden, + +00:51:48.630 --> 00:51:50.480 +your editor's way happier. + +00:51:50.800 --> 00:51:55.940 +I mean, I know you all agreed on the API boundaries, + +00:51:56.230 --> 00:51:57.340 +and I did as well. + +00:51:57.350 --> 00:51:58.780 +It's like that's one of the really cool things. + +00:51:58.870 --> 00:52:01.100 +The other thing that really makes me excited about types + +00:52:01.130 --> 00:52:04.660 +is if I hit dot in my editor, I get + +00:52:04.760 --> 00:52:08.260 +a meaningful list of real information + +00:52:08.540 --> 00:52:09.380 +about what I'm working on. + +00:52:09.380 --> 00:52:11.700 +And so adding these types of things + +00:52:11.920 --> 00:52:12.840 +are pretty interesting. + +00:52:13.400 --> 00:52:18.980 +I want to ask you all about these rogue tools + +00:52:19.340 --> 00:52:23.380 +that do stuff with Python typing that maybe you all didn't intend. + +00:52:23.580 --> 00:52:25.380 +Like we all mentioned Pydantic. + +00:52:26.540 --> 00:52:28.260 +We've got Typer and FastAPI. + +00:52:30.380 --> 00:52:33.180 +But even a little farther out there is BearType. + +00:52:33.280 --> 00:52:34.440 +Are you familiar with BearType? + +00:52:36.560 --> 00:52:37.000 +Yeah. + +00:52:37.440 --> 00:52:38.800 +Yeah, BearType's interesting. + +00:52:40.340 --> 00:52:44.440 +You can import-- they have fun. + +00:52:45.540 --> 00:52:50.560 +They have fun with their import names and stuff. + +00:52:53.720 --> 00:52:57.140 +But basically, you can put either a decorator + +00:52:57.300 --> 00:53:00.360 +onto some sort of call site or something, + +00:53:00.460 --> 00:53:03.820 +or you can just do it to an entire package-- + +00:53:04.020 --> 00:53:04.880 +or entire modules, rather. + +00:53:05.800 --> 00:53:09.720 +So just from bear type dot claw, import bear type this. + +00:53:09.840 --> 00:53:13.180 +and then it actually turns into runtime type checks. + +00:53:15.060 --> 00:53:17.600 +Good idea, bad idea, interesting. + +00:53:18.260 --> 00:53:18.860 +What do you all think? + +00:53:20.600 --> 00:53:23.380 +So un-Pythonic, you won't even open the web page? + +00:53:24.100 --> 00:53:27.460 +I think people should feel free to write whatever code helps + +00:53:27.860 --> 00:53:29.380 +them make better software. + +00:53:30.410 --> 00:53:32.140 +I haven't really used bar type much myself, + +00:53:32.500 --> 00:53:34.040 +but clearly useful for some people. + +00:53:35.560 --> 00:53:37.700 +And I think in general, in designing a type system, + +00:53:37.900 --> 00:53:40.660 +We should try to accommodate all users who do useful things + +00:53:40.760 --> 00:53:41.340 +for the type system. + +00:53:42.900 --> 00:53:44.960 +And that includes things like Pydantic or BearType. + +00:53:48.020 --> 00:53:49.500 +Yeah, it's pretty fast. + +00:53:49.720 --> 00:53:53.920 +It's not as big of a hit as you would imagine. + +00:53:54.940 --> 00:53:55.320 +Let me see. + +00:53:55.380 --> 00:54:00.900 +What are they-- somewhere they had a really fun saying in here. + +00:54:01.060 --> 00:54:02.340 +But here we go. + +00:54:03.000 --> 00:54:06.380 +BearType brings Rust C++-inspired zero-cost abstractions + +00:54:06.400 --> 00:54:09.100 +into the lawless world of dynamically typed Python + +00:54:09.240 --> 00:54:12.160 +by enforcing type safety at the granular level of functions + +00:54:12.420 --> 00:54:15.200 +and methods against type hints standardized + +00:54:15.540 --> 00:54:16.480 +by the Python community. + +00:54:16.660 --> 00:54:19.520 +And order one, non-amortize worst case time + +00:54:19.660 --> 00:54:20.860 +with negligible constant factors. + +00:54:21.080 --> 00:54:21.660 +Like how about that? + +00:54:24.880 --> 00:54:26.740 +No, it's a pretty neat library and it's pretty fast. + +00:54:27.120 --> 00:54:28.780 +Honestly, I've never used it in production. + +00:54:30.260 --> 00:54:33.880 +Having type hints and squigglies in the editors + +00:54:33.880 --> 00:54:37.000 +or in the linters has always been enough for me. + +00:54:37.140 --> 00:54:40.480 +But I can see using this if it's really critical + +00:54:40.740 --> 00:54:41.800 +and you're having issues, maybe you + +00:54:41.880 --> 00:54:43.300 +want to catch some runtime errors. + +00:54:43.840 --> 00:54:44.160 +I don't know. + +00:54:45.900 --> 00:54:48.240 +It's not quite an endorsement, but it sure is like, huh, + +00:54:48.540 --> 00:54:50.700 +that's different type of thing. + +00:54:51.020 --> 00:54:56.940 +I definitely think that the extent to which type checkers + +00:54:59.260 --> 00:55:01.580 +may have a different understanding of your code + +00:55:01.590 --> 00:55:02.640 +from what happens at runtime. + +00:55:03.100 --> 00:55:06.100 +and there isn't anything built in to catch that + +00:55:06.260 --> 00:55:07.800 +is sometimes a pain point. + +00:55:08.230 --> 00:55:10.380 +And so the desire to have your type annotations, + +00:55:10.910 --> 00:55:13.180 +to find out at runtime if your type annotations + +00:55:13.230 --> 00:55:14.140 +are telling you a lie, + +00:55:14.880 --> 00:55:17.940 +it makes a lot of sense why people would like that. + +00:55:18.660 --> 00:55:20.780 +I mean, it's something you get used to from other languages + +00:55:20.870 --> 00:55:22.580 +where the type checker is built into the compiler. + +00:55:22.660 --> 00:55:23.020 +- Right, right. + +00:55:23.160 --> 00:55:24.800 +You get like a runtime typecast. + +00:55:24.980 --> 00:55:27.600 +Like cannot, you know, we kind of get that + +00:55:27.700 --> 00:55:30.860 +if you try to parse a thing, you know, + +00:55:31.000 --> 00:55:33.440 +like put the int param around a string, + +00:55:33.550 --> 00:55:35.920 +and it's not really a parsable as an int. + +00:55:36.660 --> 00:55:39.100 +But for real type information, I think personally, + +00:55:39.250 --> 00:55:42.560 +I would use this as like I might apply types, + +00:55:43.510 --> 00:55:49.460 +type checking to a module for debugging and development + +00:55:49.550 --> 00:55:50.680 +for a minute and just see what happens, + +00:55:50.810 --> 00:55:51.680 +and then turn it back off. + +00:55:52.000 --> 00:55:54.700 +I don't know that I'd just ship production code that way. + +00:55:55.940 --> 00:55:58.740 +But anyway, I got a couple more questions. + +00:55:58.940 --> 00:56:00.240 +We're getting shorter on time here. + +00:56:02.240 --> 00:56:03.800 +What was one of the harder questions + +00:56:04.700 --> 00:56:07.120 +that you all-- harder decisions you all + +00:56:07.120 --> 00:56:10.340 +had to address on the council? + +00:56:12.420 --> 00:56:16.880 +I think the most contentious one was PEP724, + +00:56:17.520 --> 00:56:18.740 +if I remember the number correctly. + +00:56:19.820 --> 00:56:22.440 +We've just around a feature called type guards, + +00:56:22.640 --> 00:56:25.200 +which is around user-defined type narrowing functions. + +00:56:27.340 --> 00:56:31.720 +We initially defined that in a way that later was found to be somewhat problematic, + +00:56:31.980 --> 00:56:35.980 +and we basically came up with a better set of proposed semantics + +00:56:37.440 --> 00:56:40.700 +that maybe we should have done the first time around. + +00:56:41.860 --> 00:56:44.940 +And what this PEP proposed, and as you can see, I sponsored it, + +00:56:45.420 --> 00:56:48.720 +is that we basically changed the meaning of the existing type cards + +00:56:49.040 --> 00:56:52.900 +under certain conditions, which, yeah. + +00:56:53.920 --> 00:56:55.220 +What is a type card? + +00:56:55.720 --> 00:56:58.720 +A type card is a function, like there's a good example there, the isiterable. + +00:56:59.280 --> 00:57:02.860 +It's a function that tells you how to narrow something. + +00:57:03.840 --> 00:57:06.960 +So in this example, there's an isiterable type card, + +00:57:07.340 --> 00:57:09.960 +which narrows an object to an iterable of anything. + +00:57:11.080 --> 00:57:15.500 +And then inside defunc there, you can see if isiterable file, + +00:57:16.180 --> 00:57:18.540 +it knows that it's an iterable. + +00:57:19.140 --> 00:57:23.620 +And in this case, yeah, I guess it just narrows exactly to iterable any. + +00:57:24.140 --> 00:57:26.000 +That's one of the ways that type cards works. + +00:57:26.780 --> 00:57:27.400 +I see. + +00:57:27.540 --> 00:57:30.960 +And the type that returns kind of communicates to the type system + +00:57:31.230 --> 00:57:36.380 +like that this function ensures that the thing that came in + +00:57:36.430 --> 00:57:38.960 +as an arbitrary object, in fact, is one of these. + +00:57:39.420 --> 00:57:39.640 +Yeah. + +00:57:40.300 --> 00:57:41.040 +Okay, interesting. + +00:57:42.800 --> 00:57:44.520 +Yeah, so that was a tricky one, huh? + +00:57:45.760 --> 00:57:45.980 +Yeah. + +00:57:48.000 --> 00:57:49.800 +Any other standout, Rebecca or Carl? + +00:57:50.960 --> 00:57:52.259 +Well, the current discussion around + +00:57:52.280 --> 00:57:53.880 +what is the meaning of a float annotation? + +00:57:54.960 --> 00:57:57.060 +Still unresolved contentious topic. + +00:57:57.540 --> 00:57:58.520 +Okay, gotcha. + +00:58:00.520 --> 00:58:07.980 +I mean, this on PEP724 is also what came to my mind immediately as well + +00:58:08.140 --> 00:58:10.440 +because this was challenging discussion + +00:58:10.880 --> 00:58:15.660 +because there were very conflicting considerations at play. + +00:58:15.840 --> 00:58:20.079 +It's like what semantics did we want in the long term + +00:58:20.100 --> 00:58:22.740 +and what did we want the type system to look like, + +00:58:22.900 --> 00:58:24.280 +you know, say like 10 years from now + +00:58:24.580 --> 00:58:25.820 +versus backwards compatibility + +00:58:26.880 --> 00:58:28.700 +and what the migration story would look like. + +00:58:28.740 --> 00:58:31.120 +It was quite tricky. + +00:58:31.360 --> 00:58:32.900 +Yeah, I guess that's something + +00:58:32.980 --> 00:58:35.920 +you will always have to be cognizant of + +00:58:36.000 --> 00:58:38.640 +is like every change, + +00:58:38.940 --> 00:58:39.980 +even if it's an improvement, + +00:58:40.380 --> 00:58:45.700 +has to justify the fact that now you have challenges + +00:58:46.120 --> 00:58:48.680 +with the version history over time. + +00:58:49.320 --> 00:58:55.100 +I'm thinking like dict of string comma int with a capital or lowercase d. + +00:58:56.480 --> 00:59:00.440 +I've got people, I did a YouTube video showing something with the lowercase version + +00:59:00.530 --> 00:59:03.780 +because I was using something super modern like Python 3.11. + +00:59:05.160 --> 00:59:09.600 +And I got a message like, hey, Michael, you don't know how to write Python. + +00:59:09.710 --> 00:59:10.600 +Your code is broken. + +00:59:11.480 --> 00:59:13.640 +This code that you wrote just doesn't even run. + +00:59:13.710 --> 00:59:14.700 +I don't know how this is. + +00:59:14.840 --> 00:59:17.600 +I'm like, what version of Python is in? + +00:59:17.750 --> 00:59:18.160 +3.8. + +00:59:18.620 --> 00:59:19.920 +Nope, you can't use 3A for that. + +00:59:20.020 --> 00:59:21.020 +You're going to need to get a newer one. + +00:59:21.050 --> 00:59:21.580 +You know what I mean? + +00:59:21.990 --> 00:59:26.120 +But those are complexities that get added to Python + +00:59:26.470 --> 00:59:27.440 +because of that. + +00:59:27.520 --> 00:59:30.480 +Now you've got two ways to specify what a dict is. + +00:59:30.820 --> 00:59:32.860 +There's a preferred new way, but there's still the old way. + +00:59:32.890 --> 00:59:35.740 +And it just sort of piles up. + +00:59:36.299 --> 00:59:38.720 +And it's very hard to ever actually get rid of the old way, + +00:59:38.870 --> 00:59:40.420 +even if there's no good reason to use it anymore. + +00:59:40.960 --> 00:59:41.320 +Exactly. + +00:59:41.620 --> 00:59:44.600 +Once it's there, it's written in ink pretty much, right? + +00:59:44.780 --> 00:59:47.520 +Like we have five or six different ways to format strings. + +00:59:47.760 --> 00:59:49.620 +maybe with t-strings at six now. + +00:59:51.900 --> 00:59:53.480 +They're all gonna still be there, right? + +00:59:53.500 --> 00:59:55.520 +So every change, every decision you make + +00:59:55.540 --> 00:59:58.780 +is not just a matter of, is it the right decision, right? + +00:59:58.900 --> 01:00:01.980 +It's the, is it worth it, I'm sure. + +01:00:06.760 --> 01:00:08.920 +Yeah, I don't know, how do you all balance that? + +01:00:09.060 --> 01:00:10.560 +Like, that's tricky. + +01:00:16.040 --> 01:00:20.880 +With things like the big change, at least we know we're moving towards better states. + +01:00:21.200 --> 01:00:24.860 +And there's two things, but they mean exactly the same thing. + +01:00:25.160 --> 01:00:27.660 +So the confusion is not as bad. + +01:00:28.700 --> 01:00:33.720 +The problem with type card is that we're going to change how some existing thing works, like + +01:00:33.840 --> 01:00:34.320 +what it meant. + +01:00:35.700 --> 01:00:38.400 +And I think there are good reasons that maybe that was the right thing to do. + +01:00:38.740 --> 01:00:43.699 +But it would also have been pretty confusing for people if their existing types suddenly + +01:00:43.720 --> 01:00:52.740 +started meaning something completely different yeah absolutely hence float okay all right let's + +01:00:52.880 --> 01:01:02.180 +let's uh do what's coming next like 315 316 do you all have things that are in the works that + +01:01:02.180 --> 01:01:10.220 +you think are going to come or debates that are brewing i think for 315 the there's a type dict + +01:01:10.200 --> 01:01:16.260 +feature coming, extra items. You can already use it in typing extensions if you want to use it, + +01:01:16.600 --> 01:01:23.360 +but it will be in CPython as of 2.15. It's likely we'll have a small thing I added called + +01:01:23.560 --> 01:01:28.120 +disjoint basis, which is very technical, but helps type narrowing in some cases. + +01:01:30.670 --> 01:01:36.140 +Yeah, I think those are the things that are likely to make it. We can only speculate about + +01:01:36.120 --> 01:01:37.280 +but what else people can propose, + +01:01:37.450 --> 01:01:39.840 +we're sort of bound by what people actually write up as peps. + +01:01:41.680 --> 01:01:42.940 +We have to wait for people to write the peps + +01:01:43.030 --> 01:01:43.840 +before we can approve them. + +01:01:45.780 --> 01:01:48.360 +I think there's PEP 747 for type form, + +01:01:48.660 --> 01:01:52.080 +which I think we recommended its acceptance, + +01:01:52.320 --> 01:01:54.320 +but I don't think the steering council accepted it yet, + +01:01:54.480 --> 01:01:55.640 +or it hasn't been accepted formally. + +01:01:56.220 --> 01:01:57.160 +I think that's right. + +01:01:57.730 --> 01:01:59.160 +I think that's on their plate. + +01:01:59.270 --> 01:02:02.680 +Yeah, so that's also pretty likely to make it into 3.15. + +01:02:04.640 --> 01:02:04.800 +Okay. + +01:02:05.460 --> 01:02:10.920 +So this is one example of a case that will be pretty useful to people working with type + +01:02:11.140 --> 01:02:17.820 +annotations at runtime because it'll allow you to... It's sort of a meta thing where you can annotate, + +01:02:18.700 --> 01:02:21.400 +have a type annotation that describes another type annotation. + +01:02:22.670 --> 01:02:25.940 +So it's useful if you're writing code that works with type annotations. + +01:02:26.920 --> 01:02:31.560 +Sure. Okay. Make the peidantics of the world very happy. + +01:02:33.600 --> 01:02:37.580 +I am actually pretty excited about type form + +01:02:38.000 --> 01:02:39.640 +because I feel like there's a gap + +01:02:39.680 --> 01:02:41.260 +and we can express in the type system. + +01:02:41.860 --> 01:02:42.080 +Yeah. + +01:02:42.380 --> 01:02:43.280 +And we good. + +01:02:43.800 --> 01:02:46.840 +And there are cases in the existing type system, + +01:02:47.040 --> 01:02:48.520 +like for instance, the cast function + +01:02:49.140 --> 01:02:53.100 +and some other cases where something takes any type + +01:02:53.300 --> 01:02:54.520 +expression as an argument. + +01:02:54.560 --> 01:02:57.100 +We actually don't have a good way to annotate that today. + +01:02:57.140 --> 01:02:59.040 +And this will provide a nice way to express that. + +01:03:00.400 --> 01:03:00.880 +Yeah. + +01:03:01.480 --> 01:03:02.020 +OK, cool. + +01:03:03.240 --> 01:03:05.860 +Let me pull up one thing really quick. + +01:03:10.460 --> 01:03:11.940 +Quick shout out to Will McGuggan here. + +01:03:11.990 --> 01:03:15.800 +He just released his Toad project, which is the new-- + +01:03:16.840 --> 01:03:19.420 +takes textual and rich and all that kind of stuff + +01:03:19.620 --> 01:03:21.460 +and applies it to like, what if we had a better + +01:03:21.760 --> 01:03:23.880 +cloud code type of experience, which is pretty interesting. + +01:03:26.020 --> 01:03:27.960 +So the reason I bring this up is-- + +01:03:29.160 --> 01:03:29.820 +final question. + +01:03:30.220 --> 01:03:39.400 +What about, do you all even worry about the role of like how types interact with AI and agentic coding tools? + +01:03:40.940 --> 01:03:46.640 +I know that if you have some code that has types on it and you give it to an AI, + +01:03:47.100 --> 01:03:52.120 +it's got a better chance of understanding what's happening than if you give it purely untyped code and say, + +01:03:53.180 --> 01:03:54.140 +tell me about this, right? + +01:03:54.250 --> 01:03:56.380 +It doesn't even know necessarily what's being passed to it. + +01:03:57.360 --> 01:03:59.540 +But is that anything you'll think about + +01:03:59.610 --> 01:04:01.800 +or what are your thoughts on this? + +01:04:05.220 --> 01:04:06.460 +- Certainly think about it some. + +01:04:06.570 --> 01:04:09.620 +I mean, I think overall my feeling is that + +01:04:10.400 --> 01:04:12.280 +these coding agents seem to do better + +01:04:12.390 --> 01:04:15.800 +the more kind of the tighter feedback loops + +01:04:15.830 --> 01:04:17.140 +you can give them to work with. + +01:04:17.380 --> 01:04:19.740 +And so typing is another useful source of feedback + +01:04:20.000 --> 01:04:23.480 +where you can say add type annotations + +01:04:23.510 --> 01:04:25.920 +and make sure the type checker passes and seems. + +01:04:26.260 --> 01:04:28.520 +So it still seems pretty useful in that world. + +01:04:29.000 --> 01:04:31.420 +Yeah, you can easily write rules that say, + +01:04:31.860 --> 01:04:34.140 +when you are done on anything I've asked you to do, + +01:04:34.820 --> 01:04:37.580 +always run ty or always run Pyrefly + +01:04:37.680 --> 01:04:40.900 +and make sure that there's no new errors or at least-- + +01:04:41.280 --> 01:04:42.360 +ideally, zero errors, right? + +01:04:43.100 --> 01:04:44.340 +But nothing has been introduced. + +01:04:45.440 --> 01:04:45.940 +Yeah, pretty interesting. + +01:04:47.360 --> 01:04:48.320 +Other folks? + +01:04:48.500 --> 01:04:49.400 +Rebecca, Yella? + +01:04:53.780 --> 01:04:55.239 +Yeah, I guess in general, I think + +01:04:55.260 --> 01:04:57.740 +Typing will remain useful for AI. + +01:04:58.260 --> 01:04:59.840 +We are probably rapidly moving to a world + +01:05:00.000 --> 01:05:03.080 +where a large proportion of all code is written by AI. + +01:05:05.440 --> 01:05:05.840 +And-- + +01:05:06.600 --> 01:05:08.120 +Not everybody likes that opinion. + +01:05:08.260 --> 01:05:09.060 +Not everybody likes that. + +01:05:09.460 --> 01:05:12.140 +I guess maybe my current line of work makes me + +01:05:12.320 --> 01:05:13.380 +think that's more likely to happen. + +01:05:14.160 --> 01:05:16.400 +Though, I mean, it's also-- + +01:05:17.860 --> 01:05:20.300 +you don't have to like the fact it's going to be night soon. + +01:05:20.500 --> 01:05:21.200 +But it's going to be night-- + +01:05:21.200 --> 01:05:21.600 +you know what I mean? + +01:05:23.220 --> 01:05:25.640 +I just think there's so much momentum on this, + +01:05:25.760 --> 01:05:27.600 +at least in the next five years or something, + +01:05:27.760 --> 01:05:28.680 +that it's going to be really-- + +01:05:28.740 --> 01:05:31.700 +it's a truth of how many people are writing code + +01:05:31.920 --> 01:05:34.660 +regardless of whether individuals want to write code that way. + +01:05:34.860 --> 01:05:35.240 +You know what I mean? + +01:05:35.320 --> 01:05:36.280 +So I think it's a consideration. + +01:05:37.000 --> 01:05:37.180 +Yeah. + +01:05:38.220 --> 01:05:40.960 +Yeah, I forgot that you worked at OpenAI, so of course-- + +01:05:42.180 --> 01:05:44.880 +I should pull up a codex example or something, shouldn't I? + +01:05:45.940 --> 01:05:47.060 +Yeah, codex is great. + +01:05:47.260 --> 01:05:47.520 +Use it. + +01:05:50.640 --> 01:05:55.720 +No, but I mean, do you have any further insight into the role of types and coding agents? + +01:05:56.400 --> 01:05:58.120 +I know that's not exactly what you work on, right? + +01:05:58.340 --> 01:06:00.200 +Yeah, it's not too much, really. + +01:06:00.340 --> 01:06:06.560 +I think, as Carl said, types can also be helpful for AI to understand code better and to get a better feedback loop. + +01:06:08.920 --> 01:06:11.480 +I feel like the better we make AI, the more it is like humans. + +01:06:11.940 --> 01:06:19.460 +And if typing makes humans better at writing and understanding this code, they'll probably also make AI better at it. + +01:06:20.080 --> 01:06:21.740 +Yeah, it's the locality of information. + +01:06:21.850 --> 01:06:26.440 +You can read the function and know everything you need to know about what's going into it + +01:06:26.620 --> 01:06:30.940 +without bouncing around and trying to understand blocks of code and like what might have been + +01:06:31.110 --> 01:06:31.880 +created that's getting passed. + +01:06:32.980 --> 01:06:35.580 +It's good for humans and also good for AI, right? + +01:06:36.300 --> 01:06:36.560 +Rebecca? + +01:06:38.500 --> 01:06:46.479 +I mean, I guess I don't have much need to, I'll say I am maybe a little more skeptical + +01:06:46.500 --> 01:06:51.960 +than most of my coworkers about the quality of AI generated code. + +01:06:52.220 --> 01:06:56.200 +But that means I think I am particularly gung-ho about, + +01:06:56.440 --> 01:07:00.460 +you know, like get AI to use types, type checkers, + +01:07:00.960 --> 01:07:02.960 +keep the guardrails there. + +01:07:03.240 --> 01:07:04.020 +I think that'd be very important. + +01:07:04.020 --> 01:07:05.100 +- Yeah, if it's gonna make a mistake, + +01:07:05.440 --> 01:07:08.380 +don't let it at least like make the type system become + +01:07:08.840 --> 01:07:10.320 +disconnected and not working. + +01:07:10.560 --> 01:07:12.620 +Like it has to keep the types hanging together + +01:07:12.820 --> 01:07:13.920 +as a minimum bar, right? + +01:07:13.920 --> 01:07:15.880 +And you can easily set that up as an automation. + +01:07:18.340 --> 01:07:21.700 +Yeah, interesting to think of it as guardrails rather than an accelerant. + +01:07:21.880 --> 01:07:23.080 +But yeah, 100% it is. + +01:07:26.000 --> 01:07:26.580 +All right, folks. + +01:07:26.940 --> 01:07:29.080 +I think that's it for all the time that we have. + +01:07:29.640 --> 01:07:29.860 +Thank you. + +01:07:29.960 --> 01:07:30.680 +Thank you for being here. + +01:07:31.900 --> 01:07:33.680 +Final thoughts before we go. + +01:07:35.380 --> 01:07:36.180 +Carl, I'll let you go first. + +01:07:36.620 --> 01:07:38.560 +Final thoughts for people out there interested in Python typing. + +01:07:39.580 --> 01:07:42.200 +Yeah, well, first of all, thanks for having us on the podcast. + +01:07:42.460 --> 01:07:43.200 +Really appreciate it. + +01:07:44.580 --> 01:07:46.000 +And thoughts for people out there. + +01:07:46.960 --> 01:08:00.900 +I guess if you have ideas of how Python typing could be improved, discuss.python.org is a good place to bring up ideas and discuss them with the typing community and see what positive changes we can make. + +01:08:03.079 --> 01:08:04.280 +Awesome. Rebecca? + +01:08:06.400 --> 01:08:10.260 +Yeah, first, thank you, Michael. This is a lot of fun. + +01:08:11.980 --> 01:08:25.940 +Last thoughts. So, you know, like, look at the typing council and sometimes think, oh, you know, like the PEP has like governance in its name, but I wouldn't say we're really a governing body or anything. + +01:08:26.299 --> 01:08:32.040 +It's like people who are using the type system, like users, they're the ones who come up with, + +01:08:32.080 --> 01:08:35.200 +you know, like all the best ideas, propose them, discuss them. + +01:08:35.400 --> 01:08:43.380 +And we're just here to sort of be like, hey, you know, like we have some background and + +01:08:43.460 --> 01:08:47.279 +like how type checkers work and maybe some of the history and we can provide input. + +01:08:47.680 --> 01:08:51.240 +But I just encourage people, if there's a change you want to see in the type system, + +01:08:52.040 --> 01:08:54.060 +you know, like propose it yourself. + +01:08:54.240 --> 01:08:56.339 +It's very friendly and open community. + +01:08:56.720 --> 01:09:00.640 +Yeah, now people who have listened know a little bit more about how to do so. + +01:09:00.970 --> 01:09:01.060 +Awesome. + +01:09:01.400 --> 01:09:01.460 +Thanks. + +01:09:02.080 --> 01:09:02.819 +Jale, final word. + +01:09:03.520 --> 01:09:05.380 +Also, again, thank you for having me here. + +01:09:06.060 --> 01:09:07.240 +It's been great talking to all of you. + +01:09:08.370 --> 01:09:10.640 +I guess what I want to say is similar to what Karo and Rebecca just said. + +01:09:11.759 --> 01:09:16.339 +If you want to say something changes to the type system, I'd really encourage you to sign + +01:09:17.190 --> 01:09:20.259 +up for discuss.python.org, make a proposal, go through the process. + +01:09:20.799 --> 01:09:23.420 +can be somewhat daunting perhaps, especially if you have to + +01:09:24.140 --> 01:09:27.380 +read a pep, but it is doable. There are several recent typing + +01:09:27.580 --> 01:09:31.720 +peps have just been community members who saw something they + +01:09:31.839 --> 01:09:35.960 +wanted to improve, proposed a PEP and saw it to completion. If + +01:09:36.060 --> 01:09:38.560 +there's something you want to see in the type system, then you + +01:09:38.560 --> 01:09:43.940 +can do it too. Okay, awesome. Well, thank you all for keeping + +01:09:44.100 --> 01:09:47.020 +Python typing going strong. Really appreciate your time on + +01:09:47.020 --> 01:09:48.380 +the show. See you later. + +01:09:51.960 --> 01:09:52.080 +Bye. + +01:09:52.580 --> 01:09:53.060 +Bye. + diff --git a/youtube_transcripts/540-modern-python-monorepo-timeline-original.vtt b/youtube_transcripts/540-modern-python-monorepo-timeline-original.vtt new file mode 100644 index 0000000..3b053fd --- /dev/null +++ b/youtube_transcripts/540-modern-python-monorepo-timeline-original.vtt @@ -0,0 +1,2978 @@ +WEBVTT + +00:00:00.639 --> 00:00:06.800 +Hello, hello, Jarek, Amogh. Welcome to Talk Python To Me. Awesome to have Amogh, you here, + +00:00:07.260 --> 00:00:14.900 +and Jarek, you back. Yep. Welcome. Very nice to be again at Talk Python To Me. That's one of my + +00:00:15.170 --> 00:00:21.480 +favorite podcasts I listen to all the time. Thank you. Thank you. It's my first, but yeah, + +00:00:22.260 --> 00:00:27.080 +thanks for having me here, Michael. Yeah. Happy to have you here. You all have built, + +00:00:27.720 --> 00:00:35.760 +You and a team of people, given the scale of this project, have built an amazing product with Apache Airflow. + +00:00:37.240 --> 00:00:38.960 +It's going to be really fun to dive into it. + +00:00:38.980 --> 00:00:46.480 +And specifically, we're going to focus on not building workflows exactly, although I'm sure we'll talk about that somewhat. + +00:00:47.140 --> 00:00:53.400 +The real goal, the thing that we're going to focus on is how do you manage such a big project + +00:00:53.640 --> 00:00:57.600 +with so many different little inner internal packages + +00:00:57.940 --> 00:00:59.720 +that all depend upon each other and so on. + +00:01:00.500 --> 00:01:02.300 +And monorepos and that. + +00:01:02.520 --> 00:01:06.280 +I've touched on monorepos before, but two things. + +00:01:06.340 --> 00:01:08.300 +I think this makes a really interesting discussion + +00:01:08.520 --> 00:01:09.640 +for listeners out there. + +00:01:09.960 --> 00:01:14.140 +One, this is going to be very concrete with exact steps. + +00:01:14.700 --> 00:01:15.420 +And it's even open source. + +00:01:15.560 --> 00:01:17.280 +You can go check it out and play with it. + +00:01:17.800 --> 00:01:21.740 +And two, the tooling and the standards + +00:01:21.880 --> 00:01:23.160 +have changed significantly. + +00:01:23.180 --> 00:01:25.060 +since I talked about this three or four years ago, + +00:01:25.300 --> 00:01:27.360 +making much of what we're going to talk about possible, right? + +00:01:28.840 --> 00:01:29.640 +Yes, absolutely. + +00:01:30.500 --> 00:01:30.580 +Yeah. + +00:01:31.580 --> 00:01:33.300 +Now, before we dive into that, of course, + +00:01:34.260 --> 00:01:35.820 +let's do quick introductions. + +00:01:36.740 --> 00:01:38.220 +Jarek, it's been a while since you've been on the show. + +00:01:38.220 --> 00:01:38.640 +Who are you? + +00:01:38.920 --> 00:01:39.640 +Tell people who you are. + +00:01:40.920 --> 00:01:43.340 +I'm an Apache Airflow maintainer, + +00:01:43.380 --> 00:01:45.420 +one of the PMC members as well, + +00:01:45.600 --> 00:01:48.960 +and also one of the Apache Software Foundation members. + +00:01:49.120 --> 00:01:51.920 +I've got this nice pin, new logo of Apache Software Foundation + +00:01:51.940 --> 00:02:00.940 +that we got at FOSDEM. I'm also an Apache Airflow security committee member, which is an important + +00:02:01.190 --> 00:02:06.140 +aspect for what we are discussing today because of supply chain and dependencies and + +00:02:08.460 --> 00:02:15.100 +lots of security potential potential security issues these dependencies bring. Yeah, I'm one of + +00:02:15.100 --> 00:02:20.800 +the few lucky people to get to contribute to open source full time and get paid for it, which is like + +00:02:20.880 --> 00:02:26.500 +amazing. Maybe another podcast one day about that because I think that's also an interesting one. + +00:02:26.800 --> 00:02:30.700 +Yeah, I have something like that, a topic somewhat like that brewing. So yeah, potentially, + +00:02:31.239 --> 00:02:34.920 +I'll have to have you back for that. Amok? + +00:02:35.850 --> 00:02:41.860 +Yep. So, hey, I'm Amok Desai. Again, similar to Jarek, I'm a PMC member and + +00:02:42.420 --> 00:02:51.200 +computer at Apache airflow. And I'm also a part of, I'm one of the top 10 contributors to the project, + +00:02:51.760 --> 00:02:54.400 +top 10 all time contributors to the project, Jarek being number one. + +00:02:57.100 --> 00:03:03.040 +So I work at Astronomer as a senior software engineer, where I get to live in both worlds. + +00:03:03.500 --> 00:03:10.080 +One is contributing to airflow score development, and also supporting the companies that are trying + +00:03:10.020 --> 00:03:17.540 +to run air flow at scale right awesome what is astronomer tell people about that yeah it's a + +00:03:17.760 --> 00:03:23.480 +it's a company where most of our uh we're a company which is almost one of the leading + +00:03:23.920 --> 00:03:29.760 +contributors to apache airflow and also the leading consumer of it we supply uh and we provide a + +00:03:29.860 --> 00:03:35.680 +managed distribution of uh corporate managed distribution of apache airflow inside astro + +00:03:36.680 --> 00:03:44.380 +and yeah i think we have a data platform as well to try and make your life easier to use airflow at + +00:03:44.460 --> 00:03:50.980 +scale yeah incredible let me add to it two comments so as airflow has a number of stakeholders and the + +00:03:51.760 --> 00:03:57.040 +commercial stakeholders who are hosting airflow as a service as well and you know like using airflow + +00:03:57.420 --> 00:04:01.939 +and we have contributed contributions from all over the place astronomer by far like the biggest + +00:04:01.860 --> 00:04:08.020 +number of contributions and fantastic open source stakeholder. We were very much focused on making + +00:04:08.060 --> 00:04:15.340 +Apache Airflow truly vendor-neutral Apache project. I'm always amazed how well this works. + +00:04:16.560 --> 00:04:22.740 +And the second thing, the number one, I'm cheating a bit. I do a lot of small PRs. This is how you get + +00:04:22.740 --> 00:04:31.820 +the number one. I guess it depends how you measure it. You could always just do one ginormous AI + +00:04:32.380 --> 00:04:36.600 +PR that's like 100,000 lines of code in your PR and people would love you for it. + +00:04:36.640 --> 00:04:37.940 +You'd be a mega contributor. + +00:04:38.560 --> 00:04:39.680 +Oh, yeah. + +00:04:40.180 --> 00:04:40.500 +Well, no. + +00:04:40.920 --> 00:04:40.980 +No. + +00:04:40.980 --> 00:04:41.460 +He does both. + +00:04:42.100 --> 00:04:42.460 +Well, no. + +00:04:43.340 --> 00:04:44.820 +The funny part is Jarek does both. + +00:04:45.140 --> 00:04:48.460 +It's his velocity amazes me or, I don't know, shocks me sometimes. + +00:04:49.360 --> 00:04:53.420 +He does massive PRs and also like a lot of tiny ones. + +00:04:53.520 --> 00:04:55.900 +And by the time I'm looking, there are like three more out of it. + +00:04:55.940 --> 00:04:56.780 +I don't know how he does it. + +00:04:57.800 --> 00:05:05.040 +Yeah, we're going to get to a bit of how much traffic there is on Airflow in terms of like open source activity. + +00:05:06.320 --> 00:05:07.520 +It's some, it's a little bit. + +00:05:09.940 --> 00:05:14.240 +Before we move on though, Jarek, what is the Apache Software Foundation? + +00:05:14.340 --> 00:05:19.680 +What is this Apache thing that you're just talking about and why is Airflow part of it? + +00:05:20.220 --> 00:05:24.660 +Okay, so very quickly, it's a foundation, one of the oldest foundations, open source foundation in the world. + +00:05:25.560 --> 00:05:28.040 +25, 26, seven years now, I think. + +00:05:29.400 --> 00:05:31.760 +The main thing about Apache Software Foundation + +00:05:31.760 --> 00:05:33.120 +is that it's individual driven. + +00:05:33.960 --> 00:05:36.200 +So every member is an individual, not a corporate, + +00:05:36.720 --> 00:05:38.160 +as opposed to like Linux Software Foundation + +00:05:38.360 --> 00:05:39.760 +where members are corporates. + +00:05:40.560 --> 00:05:43.920 +And people make decisions in both foundation and projects + +00:05:44.960 --> 00:05:48.200 +or in PMCs so-called or project management committees. + +00:05:49.120 --> 00:05:50.960 +And Airflow is one of the PMCs. + +00:05:51.300 --> 00:05:53.720 +So one of the project management committees + +00:05:53.740 --> 00:05:55.420 +which has PMC members. + +00:05:56.260 --> 00:05:59.940 +We are both PMC members and we have like 50 other individuals + +00:06:00.400 --> 00:06:02.240 +or 60, I can't remember, like the number changes, + +00:06:02.420 --> 00:06:04.160 +like we are inviting new ones all the time. + +00:06:05.460 --> 00:06:08.540 +And we make decisions as humans, as individuals, + +00:06:08.960 --> 00:06:11.820 +not the corporates who are employing us, for example, + +00:06:12.000 --> 00:06:15.320 +because it's a meritocracy based system + +00:06:16.060 --> 00:06:20.160 +where people have merit and the merit doesn't expire + +00:06:20.280 --> 00:06:23.040 +and the merit doesn't belong to individuals, + +00:06:23.320 --> 00:06:25.580 +not to the corporates. + +00:06:26.960 --> 00:06:29.880 +And yeah, so that's one of the big, + +00:06:31.700 --> 00:06:34.380 +like pretty much all the open source software out there + +00:06:34.520 --> 00:06:38.420 +like has some Apache foundation or Apache component in it. + +00:06:38.420 --> 00:06:42.220 +It started with Apache server 28, 30 years ago almost, + +00:06:42.860 --> 00:06:46.060 +but now we have more than 300, 200 PMCs. + +00:06:47.020 --> 00:06:51.940 +We just passed 10,000 committers mark two months ago, I think. + +00:06:52.620 --> 00:06:56.420 +So like lots of individuals, lots of people contributing to the foundation. + +00:06:56.660 --> 00:06:59.760 +And the main thing about foundation is community over code. + +00:06:59.800 --> 00:07:02.980 +So we value building communities more than actually producing code. + +00:07:03.060 --> 00:07:07.800 +We believe producing code is just byproduct of great communities working together. + +00:07:10.100 --> 00:07:15.200 +And ASF is a charity, is a public good charity in US registered in Delaware. + +00:07:15.920 --> 00:07:17.340 +So we actually cannot be sold. + +00:07:17.400 --> 00:07:18.540 +We cannot change our license. + +00:07:18.840 --> 00:07:21.340 +Nothing like that can happen because of the status of the foundation. + +00:07:22.040 --> 00:07:25.880 +How interesting. And a really positive force for open source, right? + +00:07:26.440 --> 00:07:37.560 +Oh, absolutely. Absolutely. And it's when I met when I first got into like learning how ISF works, I said like, that it has no chance to work like there is no way it works. + +00:07:38.240 --> 00:07:39.680 +It's too idealistic. There's no way. + +00:07:39.680 --> 00:07:44.260 +It's too. Yeah, absolutely. And nobody in the foundation who makes decision gets any money. + +00:07:44.500 --> 00:07:47.120 +Like everyone is a volunteer, all the PMC members, + +00:07:47.220 --> 00:07:50.160 +all the commuters, all the board members, + +00:07:50.400 --> 00:07:52.800 +all president, all the VVPs, + +00:07:53.120 --> 00:07:55.100 +those are all volunteer driven roles. + +00:07:55.440 --> 00:07:56.880 +And those are the people who make decisions. + +00:07:57.400 --> 00:08:00.000 +We just pay a few people in infrastructure and security. + +00:08:00.280 --> 00:08:00.840 +That's basically it. + +00:08:01.140 --> 00:08:01.380 +- Right. + +00:08:02.500 --> 00:08:02.980 +Awesome. + +00:08:03.340 --> 00:08:03.880 +Well, very cool. + +00:08:04.520 --> 00:08:06.400 +So let's dive into it. + +00:08:06.420 --> 00:08:09.680 +What is, let's start by just talking about + +00:08:10.300 --> 00:08:11.620 +high level abstract. + +00:08:12.780 --> 00:08:13.860 +What is a monorepo? + +00:08:14.700 --> 00:08:19.120 +I think it's so easy to make that sound + +00:08:19.320 --> 00:08:20.900 +like the same thing as a monolith. + +00:08:21.020 --> 00:08:23.140 +You're like, oh yeah, monorepo, monolith, same thing, right? + +00:08:23.760 --> 00:08:25.280 +And yet you're shaking your head. + +00:08:25.680 --> 00:08:26.800 +- No, not even close. + +00:08:27.380 --> 00:08:29.900 +Yeah, the first time I met personally monorepo, + +00:08:30.040 --> 00:08:31.460 +maybe I can continue with that, + +00:08:31.480 --> 00:08:32.719 +but that was like at Google, + +00:08:33.300 --> 00:08:34.880 +where I worked at Google years ago, + +00:08:35.099 --> 00:08:37.960 +and I was surprised coming to Google + +00:08:38.140 --> 00:08:41.159 +that all the code there is in a single monorepo, + +00:08:41.719 --> 00:08:43.580 +even though we have hundreds of products + +00:08:43.760 --> 00:08:45.520 +and all the stuff you see. + +00:08:45.840 --> 00:08:47.420 +- It's gotta be a lot of code, right? + +00:08:47.500 --> 00:08:49.740 +Like a giant, giant repo. + +00:08:50.170 --> 00:08:52.460 +- It's like now they have like maybe four. + +00:08:52.840 --> 00:08:54.620 +I don't know, I've heard some stories. + +00:08:55.790 --> 00:08:57.220 +I don't work there for a long time now. + +00:08:57.780 --> 00:08:59.980 +But for me, that was a sign that + +00:09:00.780 --> 00:09:03.120 +you don't really have to split and dice and slice + +00:09:03.340 --> 00:09:06.620 +your repositories into many, many small ones, + +00:09:06.790 --> 00:09:09.580 +even if you have like non-monolithical product, + +00:09:10.080 --> 00:09:14.420 +that it all can be kept in a single repository, + +00:09:15.280 --> 00:09:17.060 +separate source trees maybe, separate like, + +00:09:18.040 --> 00:09:19.740 +we'll talk about how we do it in Airflow, + +00:09:20.190 --> 00:09:24.120 +but it's a way how you can bind it together + +00:09:24.480 --> 00:09:28.000 +and have it tested together and have it developed together, + +00:09:28.640 --> 00:09:30.880 +even though each piece is pretty much separate + +00:09:31.140 --> 00:09:32.700 +and you can work on them separately. + +00:09:33.080 --> 00:09:35.160 +So that's the monorepo, as opposed to multirepo, + +00:09:35.280 --> 00:09:38.679 +which is like when you have multiple repositories + +00:09:38.700 --> 00:09:41.360 +consisting of whatever is comes up as a product. + +00:09:43.720 --> 00:09:48.360 +Yep. I agree with everything that Jarek said, plus just a small addition, + +00:09:48.560 --> 00:09:54.440 +which is each of the component or the tiny bit of a monorepo + +00:09:55.000 --> 00:09:58.000 +can have its own build artifacts, its dependencies. + +00:09:58.900 --> 00:10:03.480 +And also it can also have its own release cycle or a release vehicle. + +00:10:05.320 --> 00:10:06.820 +That's the only addition, + +00:10:06.980 --> 00:10:11.220 +but everything is put together as a big puzzle just to keep the right it could have it if it's + +00:10:11.400 --> 00:10:16.380 +in python terms you know not every mono repo is python but in python terms it could have its own + +00:10:16.560 --> 00:10:22.540 +pyproject.toml potentially its own virtual environment yes right yes exactly yeah yeah + +00:10:22.720 --> 00:10:31.919 +i think one of the nomenclature ironies of this is often the mono repo i think makes more sense + +00:10:31.940 --> 00:10:37.140 +when you are working with lots of small parts right where the monolith maybe it has a couple + +00:10:37.140 --> 00:10:41.800 +of things but it doesn't depend real deeply the more interconnections you have and the harder + +00:10:41.900 --> 00:10:48.120 +it is to manage those versions the more something like this makes sense right absolutely so so the + +00:10:48.140 --> 00:10:54.200 +main thing is like people really make a connection between isolated work on part of the system + +00:10:54.960 --> 00:11:00.260 +into having to have separate repository for that which is completely not the case like you can + +00:11:00.260 --> 00:11:05.080 +actually have an isolated sub part of the repository even if it's git git doesn't have + +00:11:05.080 --> 00:11:09.700 +like gift have some modules and sub repos and all the stuff but even like in a single git + +00:11:09.820 --> 00:11:15.500 +repository you can easily have like start working and focusing on a small part of of the whole + +00:11:15.760 --> 00:11:21.040 +monorepo and only care about that and that's what the monorepo is + +00:11:23.680 --> 00:11:27.920 +yeah you know i'm not a i'm gonna go ahead and put it out there i'm not a big fan of + +00:11:27.940 --> 00:11:37.260 +of microservice architectures, I kind of find it's trading code complexity for DevOps and + +00:11:37.380 --> 00:11:41.160 +deployment complexity. And I think we have better tools to manage code complexity than DevOps + +00:11:41.540 --> 00:11:49.860 +complexity. But something like this does help you manage those kinds of deployments as well, + +00:11:50.340 --> 00:11:50.800 +better, right? + +00:11:51.440 --> 00:11:57.900 +Yes, yes. I use the term mini services, not microservices. Microservices is just + +00:11:57.920 --> 00:12:02.360 +much. But then you can have a lot of a number of mini services, but not micro. + +00:12:02.780 --> 00:12:05.480 +It's like micro was just too much of a mixed feed. + +00:12:05.520 --> 00:12:07.440 +Yeah, I can get on board with that. Amak, what do you think? + +00:12:08.320 --> 00:12:12.040 +Yeah, I like that idea as well. Mini services. Maybe you should coin that. + +00:12:12.880 --> 00:12:16.540 +Yeah, it's like it's the microservices are too small. It feels to me like the + +00:12:16.840 --> 00:12:22.620 +equivalent of when you're trying to write unit tests and you're like, oh, what if I + +00:12:23.020 --> 00:12:26.580 +get a customer and I set their first name and then I check that their first name is + +00:12:26.600 --> 00:12:29.600 +set, like, what are you doing? You don't need to check that assignment works. This is too, + +00:12:29.960 --> 00:12:31.840 +you're just too much in the weeds. You know what I mean? + +00:12:31.840 --> 00:12:34.320 +This is what AI agents do now all the time. + +00:12:38.280 --> 00:12:41.240 +Yeah, think of the code coverage. Just think of the code coverage. Come on. + +00:12:43.880 --> 00:12:47.700 +You've got some goals to hit. You said 80% code coverage. It's on top of it. + +00:12:49.100 --> 00:12:53.880 +All right. So that's, that sets the stage. Let's talk a little bit about + +00:12:54.700 --> 00:12:56.940 +specifically how Apache Airflow + +00:12:58.600 --> 00:13:01.120 +has come to need this basically. + +00:13:03.300 --> 00:13:07.160 +You shared with me the GitHub pulse for Apache Airflow. + +00:13:08.240 --> 00:13:13.020 +And I think it's kind of worth looking at just how much + +00:13:14.560 --> 00:13:17.120 +open source interest and traffic there is. + +00:13:17.330 --> 00:13:20.800 +Who wants to kind of summarize this weekly pulse here? + +00:13:21.020 --> 00:13:21.380 +I'm okay. + +00:13:23.280 --> 00:13:24.100 +No, Mark, do it. + +00:13:25.260 --> 00:13:33.700 +So, yep. This is not the best week in terms of the number of commits. We have had even more read, + +00:13:33.920 --> 00:13:43.160 +but this is just one of those weeks. Yeah, one of the usual weeks. So between Feb 3 and Feb 10, + +00:13:43.220 --> 00:13:53.440 +we've had about 310 active pull requests so you can imagine that's uh about 40 plus pull requests + +00:13:53.600 --> 00:14:00.620 +a day a lot of a lot of them are being assisted by the uh you know the ai revolution going on but + +00:14:00.720 --> 00:14:07.120 +yeah again that's a lot of pull requests and we have merged about 200 of them and + +00:14:08.400 --> 00:14:14.280 +about 100 are open and similarly with issues right uh 35 new issues uh five issues per day + +00:14:14.920 --> 00:14:21.580 +so that's a lot of traffic so you can imagine the amount of review pressure each of + +00:14:21.800 --> 00:14:29.080 +uh each of the maintenance has here there's 300 pull requests spread across i don't know one + +00:14:29.660 --> 00:14:37.519 +120 130 maybe 140 distributions and each of each of the distributions having like a swim lane owner + +00:14:38.660 --> 00:14:42.060 +who is actively trying to take a look at these pull requests. + +00:14:42.220 --> 00:14:45.780 +So it's just another week, to be very honest. + +00:14:46.860 --> 00:14:50.040 +And it's more than 25 PRs a day, including weekends. + +00:14:51.860 --> 00:14:52.340 +Incredible. + +00:14:52.620 --> 00:14:57.580 +And how many of these PRs are high value? + +00:14:58.380 --> 00:14:59.800 +I guess I'm trying to get the sense of, + +00:14:59.920 --> 00:15:01.140 +how much does this get accepted? + +00:15:01.280 --> 00:15:03.420 +Are these just people throwing stuff out there + +00:15:03.440 --> 00:15:05.320 +that doesn't make sense for the direction of airflow? + +00:15:05.440 --> 00:15:13.060 +So well, those merged all make sense because they are reviewed and merged by Airflow maintainers. + +00:15:13.240 --> 00:15:14.520 +And we are very serious about that. + +00:15:14.670 --> 00:15:18.300 +So like we don't merge anything that doesn't pass our bar, which is like very high, + +00:15:18.780 --> 00:15:19.580 +like extremely high. + +00:15:19.670 --> 00:15:28.720 +Like we have 170 prac hooks which are checking if the code is doing what it was supposed to be doing + +00:15:28.830 --> 00:15:30.320 +and if it's architected properly. + +00:15:30.600 --> 00:15:38.200 +on top of that we have uh individuals people like among myself and then maybe 50 other uh pmc + +00:15:38.320 --> 00:15:43.880 +members and committees uh who are reviewing it and making their comments and know the system uh + +00:15:44.280 --> 00:15:50.040 +enough to direct people so they may make sense we do have recently and that was a recurring them at + +00:15:50.040 --> 00:15:55.860 +the fosden uh conference last week when i was there about like the ai uh generated contributions + +00:15:55.920 --> 00:16:01.760 +and many of the ai generated contributions are not the best quality it's not like ai is bad + +00:16:01.960 --> 00:16:07.440 +quality is that like many of those are easier to produce and they might have bad quality so we are + +00:16:07.540 --> 00:16:14.460 +now learning how to filter them out and how to make the to handle them quickly but but those are the + +00:16:14.470 --> 00:16:23.940 +the actual high value prs that we merged nice in in terms of numbers if you if i may the it would be + +00:16:23.960 --> 00:16:28.640 +maybe a third of the open pull requests that are nice, general trend. + +00:16:29.200 --> 00:16:30.320 +Yeah, that's pretty good, honestly. + +00:16:31.770 --> 00:16:32.380 +Yep. That's pretty good. + +00:16:32.390 --> 00:16:36.920 +We have some guidelines published very recently and due to that we have seen a dip in + +00:16:37.960 --> 00:16:45.840 +such quality of PRs. We published some guidelines in our contribution guides about what will be the + +00:16:45.860 --> 00:16:54.220 +action taken if uh you know bad quality prs are raised or uh non or ps erased where the author + +00:16:54.340 --> 00:17:01.340 +does not know the context but the ai does yeah sure yeah yeah i don't want to go down this rat + +00:17:01.340 --> 00:17:07.260 +hole people hear this enough lately but i just it's been in the news lately that um open source + +00:17:07.500 --> 00:17:14.800 +projects have been kind of getting a barrage of ai submissions and i think that comes in a couple of + +00:17:14.819 --> 00:17:22.079 +flavors one people who just want to get their name listed as a contributor maybe it helps them with + +00:17:22.079 --> 00:17:27.120 +their job or whatever so there's like a small incentive there but it's been really bad for bug + +00:17:27.180 --> 00:17:32.480 +bounties like curl closed its bug bounty program because people were trying to make the 50 or 250 + +00:17:32.780 --> 00:17:38.740 +dollars by finding some issue with ai is that a problem for you all just taking the pulse of a big + +00:17:38.740 --> 00:17:44.120 +project like that it it it is i actually had a talk about that at the global vulnerability + +00:17:44.560 --> 00:17:49.920 +intelligence platform summit just before fosden so that was exactly like i i even quoted daniel + +00:17:50.060 --> 00:17:56.640 +stanberg and i met him there at fosden like they were really cool uh that uh but there are some + +00:17:56.860 --> 00:18:02.280 +different motivations of people who are submitting those those ai issues and we should fight with the + +00:18:02.960 --> 00:18:07.920 +in different ways with different uh approaches or like you know respond to those motivations + +00:18:07.940 --> 00:18:13.760 +somehow we have some ideas we have an open discussion uh in github maintainers uh list + +00:18:13.980 --> 00:18:19.340 +right now and github is trying to up to uh to address it by like just discussing what they can + +00:18:19.370 --> 00:18:24.200 +do right now and that's the highest priority for them also we have a discussion with ossf + +00:18:24.810 --> 00:18:31.760 +for security uh kind of guidelines or policies for open source maintainers how to deal with those + +00:18:31.800 --> 00:18:38.160 +issues and i'm sure we will work out some ways and toolings and most of all uh processes + +00:18:38.380 --> 00:18:42.840 +and like being assertive is one thing like just saying no when the report doesn't meet all the bars + +00:18:43.740 --> 00:18:48.480 +immediately and you know directing people to the description is good enough of a of a you know + +00:18:48.740 --> 00:18:55.760 +barrier for uh you know getting kind of completely broken prs because we have to just make it more + +00:18:55.760 --> 00:19:01.860 +expensive for the reporters than for the maintainers to diagnose the issues or yeah decide if the + +00:19:01.920 --> 00:19:06.840 +issues are bad or good yeah and i'm not necessarily saying that there's something inherently bad + +00:19:07.100 --> 00:19:11.940 +because ai wrote some of the code than a person ai can write really good code better than a lot of + +00:19:12.040 --> 00:19:19.080 +people i've seen but it it has this sort of shotgun effect often of just like i'm going to change all + +00:19:19.180 --> 00:19:24.780 +these files and it's not as focused and clear a lot of times it just it doesn't it doesn't get the + +00:19:24.660 --> 00:19:25.560 +the Zen of it, you know? + +00:19:26.100 --> 00:19:26.800 +Amogh, what do you think? + +00:19:27.420 --> 00:19:28.420 +- Yeah, I agree with that. + +00:19:29.540 --> 00:19:32.080 +It'll generate code, which it thinks is good, + +00:19:32.240 --> 00:19:34.380 +but we don't really know the ripple effects. + +00:19:35.060 --> 00:19:37.700 +And we want to avoid such things. + +00:19:38.260 --> 00:19:43.760 +- Yeah, yeah, you've such a long living app + +00:19:43.820 --> 00:19:45.440 +with lots of complexity, right? + +00:19:46.020 --> 00:19:48.440 +- We all are using AI for generating the code, + +00:19:48.900 --> 00:19:49.220 +to be honest. + +00:19:49.560 --> 00:19:51.020 +So like, most of my-- + +00:19:51.020 --> 00:19:52.520 +- And you should, you should, yeah. + +00:19:53.100 --> 00:19:53.300 +- Yeah. + +00:19:55.100 --> 00:19:55.560 +it's incredible. + +00:19:56.880 --> 00:20:00.200 +I pulled up this graphic here, and I'll link to it in the show notes. + +00:20:00.980 --> 00:20:05.160 +Just giving people a sense, I got this little utility that I released this week + +00:20:05.230 --> 00:20:09.420 +called Tallyman, which analyzes code and gives you more of a breakdown + +00:20:09.610 --> 00:20:11.640 +than just this many lines or whatever. + +00:20:12.630 --> 00:20:16.460 +So I want to just highlight, maybe you all can riff on this a little bit + +00:20:16.460 --> 00:20:17.160 +to give a sense. + +00:20:17.320 --> 00:20:23.860 +So 1.2 million lines of Python, 918,000 excluding comments. + +00:20:24.320 --> 00:20:30.140 +Maybe a little overcounting the way this thing works, but still 200,000 restructured texts. + +00:20:31.250 --> 00:20:38.420 +The one that really stood out to me, 81,000 lines of YAML and 16,000 lines of TAML, you guys. + +00:20:41.280 --> 00:20:41.940 +That's impressive. + +00:20:42.230 --> 00:20:43.180 +And you know what? + +00:20:43.680 --> 00:20:49.940 +Hat tip to just a sprinkle, just a hint of Java at 42 lines of Java. + +00:20:51.020 --> 00:20:54.520 +But you know, a hundred, almost a million, + +00:20:54.980 --> 00:20:57.080 +just over a million lines of code without comments. + +00:20:57.960 --> 00:20:59.440 +That's a big project. + +00:21:00.400 --> 00:21:01.340 +What do you think? + +00:21:03.640 --> 00:21:05.440 +- Amok, what happened when you joined? + +00:21:07.400 --> 00:21:08.220 +- I don't know. + +00:21:08.500 --> 00:21:10.920 +I think it was much lesser, but yeah. + +00:21:11.420 --> 00:21:12.880 +- Yes, you did contribute a lot. + +00:21:13.600 --> 00:21:14.680 +- It's a lot of code. + +00:21:15.200 --> 00:21:18.720 +And you can imagine so because of the number of packages + +00:21:18.980 --> 00:21:20.980 +we have in the monorepo discussion + +00:21:21.000 --> 00:21:26.440 +We have a lot of packages and the YAML might surprise you at first, but + +00:21:26.940 --> 00:21:32.000 +if you actually go and see why the YAML, it's mostly for our providers. + +00:21:32.790 --> 00:21:35.380 +So integrations with other systems is something we call as providers. + +00:21:36.240 --> 00:21:38.680 +And the spec of the providers is written in YAML. + +00:21:39.500 --> 00:21:42.400 +And Toml, I'm sure we'll come to it very, very soon. + +00:21:42.420 --> 00:21:50.960 +Yeah, yes, that's kind of why I pulled this up actually, is the Toml aspect is + +00:21:50.980 --> 00:21:52.460 +of us with that number as we move on. + +00:21:53.280 --> 00:21:54.340 +16,000 lines of TAML. + +00:21:54.440 --> 00:21:57.720 +That's a lot of pyproject.taml going on right there, folks. + +00:21:58.340 --> 00:21:59.000 +Oh, yes. + +00:21:59.040 --> 00:21:59.640 +Oh, yes. + +00:22:00.320 --> 00:22:01.740 +And lots of it is generated, actually. + +00:22:01.900 --> 00:22:05.580 +So because we actually generate quite a lot of the YAML + +00:22:05.600 --> 00:22:07.760 +and TAML that we have, and we keep it in the repo, + +00:22:08.120 --> 00:22:10.280 +because we don't want to regenerate every time. + +00:22:10.560 --> 00:22:14.960 +So it's a lot-- we don't write YAML by hand, no. + +00:22:16.019 --> 00:22:17.100 +Yeah, of course. + +00:22:18.140 --> 00:22:18.480 +Very cool. + +00:22:18.660 --> 00:22:21.300 +- Okay, so now, you know, let's, + +00:22:22.320 --> 00:22:24.460 +maybe we can start by introducing this + +00:22:24.460 --> 00:22:27.940 +by just giving a shout out to this series + +00:22:28.100 --> 00:22:31.940 +that you wrote over here on Medium, Erik. + +00:22:32.960 --> 00:22:33.140 +- Yeah. + +00:22:33.340 --> 00:22:35.760 +- Modern Python repo for Apache Airflow, + +00:22:36.140 --> 00:22:37.300 +parts one through four. + +00:22:38.100 --> 00:22:38.460 +- Yes. + +00:22:39.620 --> 00:22:42.920 +Yes, I initially started discussing this blog post idea + +00:22:43.260 --> 00:22:44.020 +with a few people. + +00:22:45.280 --> 00:22:51.000 +And people are busy and I couldn't get people to write it. + +00:22:51.220 --> 00:22:52.560 +So I decided to write it myself. + +00:22:52.900 --> 00:22:55.800 +Well, with a lot of AI help, of course. + +00:22:56.060 --> 00:22:58.500 +It's not that everything is written by hand. + +00:22:59.900 --> 00:23:03.180 +And when I wrote it, I realized it's too big. + +00:23:03.580 --> 00:23:05.320 +And I had to split it into four. + +00:23:06.500 --> 00:23:08.460 +But the idea was to document what we've done. + +00:23:09.180 --> 00:23:10.260 +Because I think that a lot of people + +00:23:10.260 --> 00:23:13.500 +are struggling with monorepo versus multirepo + +00:23:13.520 --> 00:23:15.720 +or like how they should do their repository + +00:23:16.140 --> 00:23:18.380 +in when they are a project growth. + +00:23:19.500 --> 00:23:22.600 +And there were lots of discussions in the past, + +00:23:22.760 --> 00:23:26.260 +including one of the podcasts of yours + +00:23:26.460 --> 00:23:28.100 +were Monorepo versus MultiRepo. + +00:23:28.100 --> 00:23:29.580 +And I can't remember who that was, + +00:23:29.580 --> 00:23:31.940 +but there was discussion about like going back and forth + +00:23:32.160 --> 00:23:34.420 +and like finding that people sometimes go back + +00:23:34.600 --> 00:23:37.640 +and then go forth and like in different directions + +00:23:37.900 --> 00:23:41.520 +because there are different problems with both approaches. + +00:23:42.180 --> 00:23:44.780 +So I just wanted to document the reasoning why we are doing it, + +00:23:45.880 --> 00:23:49.680 +like why it's possible now because of the packaging ecosystem + +00:23:50.700 --> 00:23:56.140 +maturing for Python and uv and other tools coming into the space. + +00:23:57.080 --> 00:24:01.160 +And then the last part was like really the kind of a little bit innovative + +00:24:01.330 --> 00:24:05.000 +approach that we do where the tooling is still not catching up with what we need + +00:24:05.100 --> 00:24:06.900 +and what we did. + +00:24:07.120 --> 00:24:11.460 +So yeah, so those are the kind of history + +00:24:11.840 --> 00:24:14.840 +why we are doing it, the packaging, + +00:24:16.779 --> 00:24:19.300 +the automated verification with Prack, + +00:24:19.470 --> 00:24:20.360 +that was the third part. + +00:24:20.370 --> 00:24:23.000 +And the fourth part was about like this shared + +00:24:24.600 --> 00:24:28.040 +libraries, innovative concept that we added for Earthful. + +00:24:28.520 --> 00:24:28.960 +- Nice. + +00:24:29.090 --> 00:24:31.980 +Yeah, I'll link to the series as well as to a talk + +00:24:31.980 --> 00:24:34.580 +that you gave at Vostum that just got published, right? + +00:24:35.600 --> 00:24:36.740 +Yes, a few days ago, yes. + +00:24:37.240 --> 00:24:38.320 +Yeah, that's a good talk. + +00:24:38.400 --> 00:24:41.880 +They have an amazing system of recording and publishing stuff. + +00:24:42.180 --> 00:24:45.460 +Like for the volunteer-driven conference, 1,000 speakers. + +00:24:46.080 --> 00:24:47.240 +Oh, that's amazing. + +00:24:47.440 --> 00:24:48.480 +That works like brilliant. + +00:24:50.160 --> 00:24:51.740 +Probably some automation going on there. + +00:24:52.340 --> 00:24:53.480 +Oh, a lot. + +00:24:55.980 --> 00:25:00.760 +Yeah, so let's talk a little bit about, I guess, the problems that you ran into + +00:25:02.600 --> 00:25:06.820 +because initially there were some challenges with the standards and tooling + +00:25:06.940 --> 00:25:11.080 +not be there and you actually one of the takeaways if people read the series or + +00:25:11.120 --> 00:25:16.600 +watch the talk is you actually had to work with some of the tool providers to + +00:25:16.760 --> 00:25:21.020 +make this possible so not only is it like well the tools have changed what we + +00:25:21.180 --> 00:25:26.020 +could do this it's you all have changed the tools a little bit through you know + +00:25:25.860 --> 00:25:33.860 +working closely like hey we've got this 1 million line project with 100 sub modules or more help + +00:25:34.500 --> 00:25:39.600 +like it's your tools to support this help me make this work right what were some of the problems + +00:25:41.740 --> 00:25:46.140 +okay so let me start with this cooperation and maybe you know amok can also explain like what + +00:25:46.180 --> 00:25:51.480 +was before and after because like he experienced that firsthand as a as a user kind of this kind of + +00:25:51.300 --> 00:25:57.620 +repository structure but for me the idea was like i i was working on it for years like uh when + +00:25:57.740 --> 00:26:03.920 +we went to airflow two five years ago uh we or four years ago i can't remember that's a long time + +00:26:04.420 --> 00:26:09.000 +we didn't have all the tooling and we had to do pretty much everything that we do now with the + +00:26:09.140 --> 00:26:16.880 +with monorepo and uv uh by hand uh by bash scripts by that time by crazy so like if you run it three + +00:26:16.900 --> 00:26:24.520 +years ago your code you would see more than 10 000 lines of bash code which i wrote oh wow we since + +00:26:24.620 --> 00:26:30.120 +that is not joyful that doesn't spark joy no no no that's why we removed it with some outreach + +00:26:30.360 --> 00:26:36.200 +internship actually and shout out to edit and borna who were our outreach mentors who helped us to + +00:26:36.879 --> 00:26:44.480 +convert it to python which was really helpful uh so that that's how it started no tooling uh a need + +00:26:44.540 --> 00:26:50.500 +need because we grew we wanted to have more providers more integrations and it already was + +00:26:50.980 --> 00:26:55.760 +quite difficult to manage if they all were part of single distribution so we have to split into + +00:26:56.240 --> 00:27:03.880 +many distributions 60 i think at the beginning now we have more than 100 now now uh and uh when we + +00:27:03.890 --> 00:27:09.760 +did that i we had to do all manually and like working with that was like really cumbersome + +00:27:09.820 --> 00:27:14.940 +and maybe you know like i can i can switch to to amok so he can say like the past experience a new + +00:27:15.060 --> 00:27:21.680 +experience because like he experienced the change himself yeah the past experience was + +00:27:21.780 --> 00:27:28.960 +scary to be to speak the least uh whenever i uh switch branches or have to rebase for whatever + +00:27:29.140 --> 00:27:35.680 +reason it i had a nightmare a very bad time trying to you know package things together and try to + +00:27:35.700 --> 00:27:36.300 +to run something. + +00:27:36.440 --> 00:27:38.200 +And I think Jarek found me often, + +00:27:39.220 --> 00:27:40.680 +ranting on the Slack channels that, + +00:27:40.700 --> 00:27:42.380 +hey, this doesn't work, hey, that doesn't work, + +00:27:42.580 --> 00:27:42.940 +what do we do? + +00:27:44.740 --> 00:27:46.300 +Now it's very easy. + +00:27:48.059 --> 00:27:50.320 +It's effortless, almost effortless compared to + +00:27:51.060 --> 00:27:53.040 +what we had years, maybe like five years ago, + +00:27:53.140 --> 00:27:53.640 +four years ago. + +00:27:54.340 --> 00:27:54.960 +- Yeah, amazing. + +00:27:55.580 --> 00:27:56.680 +How does GitHub deal? + +00:27:57.780 --> 00:27:58.840 +- Just to add to that, + +00:27:59.060 --> 00:27:59.240 +- Go ahead. + +00:27:59.360 --> 00:28:00.320 +- Before we go on. + +00:28:00.700 --> 00:28:02.900 +So like the part of it was also that the, + +00:28:05.240 --> 00:28:09.500 +been listening so like i was the only one who actually managed the whole thing for years and + +00:28:09.550 --> 00:28:14.960 +i was like overwhelmed as well when people have problems of course so then the change that we've + +00:28:15.020 --> 00:28:20.020 +done was not only with the tooling and as you mentioned we were actually cooperating with charlie + +00:28:20.500 --> 00:28:27.000 +from astral charlie marsh and with joe from feck uh because we had this need we had it implemented + +00:28:27.220 --> 00:28:31.700 +ourselves and then they could look at how we've done that and they could implement it properly in + +00:28:31.720 --> 00:28:37.280 +their tooling and we've been like exchanging the you know like charlie was even interviewing me at + +00:28:37.360 --> 00:28:43.340 +some point of time how we how what what are our needs uh so so i have for a long time i have this + +00:28:43.960 --> 00:28:50.960 +this motto that uh the best way to foresee future is to shape it and like so we did shape the future + +00:28:51.160 --> 00:28:55.760 +by you know talking to those tool providers so that they can or builders so that they could build + +00:28:55.770 --> 00:29:00.400 +it for us and work with us and we help them to test and everything like that but also it was like + +00:29:00.360 --> 00:29:07.120 +listening to amok and other contributors like all the problems they had or like and then when i + +00:29:07.180 --> 00:29:12.240 +solved it i would i wouldn't also only solve it with the new tooling but we also engaged all the + +00:29:12.540 --> 00:29:17.800 +more people from the from the team like amog and few other active contributors and they were + +00:29:17.980 --> 00:29:22.800 +actually part of the whole process of conversion and they are now part of the team and now we can + +00:29:22.840 --> 00:29:27.539 +have this podcast while things are being broken in airflow right now and somebody is probably + +00:29:27.560 --> 00:29:28.800 +we're fixing it as we speak. + +00:29:28.980 --> 00:29:30.360 +So not me anymore. + +00:29:30.540 --> 00:29:33.600 +So those old things are really great. + +00:29:34.860 --> 00:29:36.060 +- That's really, really good. + +00:29:37.260 --> 00:29:37.740 +Yeah, incredible. + +00:29:38.660 --> 00:29:41.600 +How does GitHub deal with so many files + +00:29:41.860 --> 00:29:42.700 +and such a big project? + +00:29:42.940 --> 00:29:45.760 +Is it fine or is it a challenge? + +00:29:46.160 --> 00:29:49.140 +- Except yesterday where half of the time- + +00:29:49.140 --> 00:29:50.780 +- Yeah, yeah, except yesterday. + +00:29:50.780 --> 00:29:52.380 +Yeah, for people who don't know, + +00:29:52.460 --> 00:29:55.100 +yesterday morning, at least morning US time, + +00:29:55.100 --> 00:29:55.480 +GitHub was having a moment. + +00:29:55.720 --> 00:29:58.380 +Like it was, I couldn't clone stuff. + +00:29:58.840 --> 00:30:04.180 +I pulled up the random page on GitHub and got the 503 Unicorn. + +00:30:04.480 --> 00:30:05.520 +It was not good, right? + +00:30:05.720 --> 00:30:07.480 +Besides that, excluding that time. + +00:30:08.040 --> 00:30:11.900 +So the Unicorn is actually a little bit like looking kind of angry at you. + +00:30:12.360 --> 00:30:14.340 +That's one of the observations I had from yesterday. + +00:30:14.870 --> 00:30:18.260 +I saw it so many times that it doesn't look nice. + +00:30:18.840 --> 00:30:19.520 +I agree. + +00:30:19.580 --> 00:30:20.940 +That's not a great error page. + +00:30:21.080 --> 00:30:23.420 +Like some error pages are amazing where it's like, + +00:30:23.940 --> 00:30:25.780 +you know, the coyote fell off of a cliff. + +00:30:26.000 --> 00:30:26.140 +Woo! + +00:30:26.760 --> 00:30:28.680 +That one just looks like it's angry back at you. + +00:30:29.380 --> 00:30:30.140 +- Exactly, exactly. + +00:30:30.460 --> 00:30:32.100 +So no, besides that, it's perfect. + +00:30:32.360 --> 00:30:33.720 +It's like, it works like seamlessly, + +00:30:33.960 --> 00:30:36.100 +no problems whatsoever with the size, with the numbers. + +00:30:36.340 --> 00:30:38.000 +Like we are very, very happy in general. + +00:30:38.660 --> 00:30:40.300 +And of course, like things like that happen. + +00:30:40.420 --> 00:30:41.260 +There is nothing wrong. + +00:30:41.660 --> 00:30:42.900 +Like there is something wrong, + +00:30:42.980 --> 00:30:45.020 +but like it's not like that it happens all the time. + +00:30:45.200 --> 00:30:45.440 +Not really. + +00:30:45.660 --> 00:30:47.740 +- No, it's actually, it's super rare. + +00:30:47.900 --> 00:30:49.100 +GitHub is an incredible service. + +00:30:49.400 --> 00:30:56.800 +I mean, I know there's been some grief about the GitHub actions, but that's a different conversation, right? + +00:30:57.060 --> 00:30:57.200 +Yeah. + +00:30:59.340 --> 00:31:00.040 +All right. + +00:31:00.300 --> 00:31:09.120 +Let's talk next about how the package standards have changed and how basically some of those things have made it possible. + +00:31:09.500 --> 00:31:12.580 +So in your talk, you pulled up a bunch of different peps, + +00:31:13.080 --> 00:31:14.180 +nine of them or something like that, + +00:31:15.480 --> 00:31:17.400 +that were about packaging, + +00:31:18.780 --> 00:31:21.160 +recently packaging standards and different things like that, + +00:31:21.300 --> 00:31:25.060 +that have made basically the structure that you're working with + +00:31:25.120 --> 00:31:26.440 +and the tools that do it possible. + +00:31:27.500 --> 00:31:29.060 +Do you want to maybe highlight either of you, + +00:31:29.100 --> 00:31:31.300 +some of these things that stand out as like, + +00:31:31.740 --> 00:31:33.460 +actually this one is really important. + +00:31:37.400 --> 00:31:41.760 +Well, for me, the one which is maybe not super related to Monorepo, + +00:31:41.760 --> 00:31:44.460 +but it actually helped us a lot, like the PEP 7.2.3, + +00:31:45.180 --> 00:31:49.180 +the last one but last inline script metadata, + +00:31:50.060 --> 00:31:52.200 +which is like one of the biggest successes + +00:31:52.500 --> 00:31:57.160 +and the biggest kind of usages I see from a PEP implemented. + +00:31:57.380 --> 00:31:59.460 +It caught up very, very quickly. + +00:32:00.080 --> 00:32:04.560 +It allows to embed inline script metadata into the Python scripts, + +00:32:04.760 --> 00:32:07.460 +which is like something that we've been dreaming of for years, + +00:32:08.280 --> 00:32:11.200 +especially for this kind of tooling, the FCI environment, etc. + +00:32:11.520 --> 00:32:12.860 +This is really, really helpful. + +00:32:12.940 --> 00:32:14.760 +So that's the one that I would like to highlight. + +00:32:15.520 --> 00:32:18.480 +But I read all of them many times, all the tabs, + +00:32:18.580 --> 00:32:20.980 +and they are difficult things to read and understand. + +00:32:23.020 --> 00:32:26.560 +But they were like, we actually did all that we could + +00:32:26.800 --> 00:32:28.780 +to be fully compliant, + +00:32:29.580 --> 00:32:31.540 +not only with the specification of those tabs, + +00:32:31.900 --> 00:32:34.400 +but also with the kind of spirit of the specification, + +00:32:34.740 --> 00:32:39.020 +sometimes things are not very precisely described and there are some interpretations and stuff + +00:32:39.700 --> 00:32:45.700 +so we just we just made sure and this is our our goal as well like we just make sure that all the + +00:32:45.800 --> 00:32:51.640 + PEP standards that are being published are actually very very meticulously followed and we just try + +00:32:51.640 --> 00:32:56.460 +to adapt to any changes that are coming in the environment so we know how difficult it is if + +00:32:56.580 --> 00:33:00.879 +people are sticking to the old ways and like that's that makes difficult for python maintainers + +00:33:05.080 --> 00:33:06.380 +- Mag, any other thoughts? + +00:33:07.480 --> 00:33:13.960 +- Yeah, this one is a particularly very important one for us also because it simplifies our + +00:33:13.990 --> 00:33:20.960 +pre-commit configurations where earlier we had to specify the dependencies as requires + +00:33:21.210 --> 00:33:25.960 +or whatever the particular version was, but now it's all in the script. + +00:33:27.540 --> 00:33:34.000 +the pre-commit remains as clean as it could just with the hook name and you know the rejects for + +00:33:34.100 --> 00:33:42.180 +the file filter and minimal configurations for it to work well and dependency group is also the other + +00:33:42.380 --> 00:33:48.280 + PEP i don't recall the name but i recall the number i think it's six oh i can't remember all the + +00:33:48.400 --> 00:33:56.340 +numbers but one of that would be 735 folks 735 yeah so that's also particularly nice for us we can + +00:33:57.380 --> 00:34:05.400 +define the dependency groups in our by projects and it's it's nice to uh how it's how it's really + +00:34:05.410 --> 00:34:12.280 +nice how it works with uv so we are very happy with this particular uh dependency group PEP as + +00:34:12.280 --> 00:34:18.080 +well as the uh inline scripts i think right the inline scripts are cool i you know especially + +00:34:18.260 --> 00:34:26.799 +with uv these days it really makes running some kind of python code so much easier it's it's almost + +00:34:26.820 --> 00:34:32.980 +as if everything is standard library you know i can give somebody a file i can say the way you run + +00:34:33.100 --> 00:34:38.320 +it no no no don't i know it looks like you say python but don't say that you say uv run this + +00:34:38.560 --> 00:34:43.540 +and then and that's it like they didn't have to have python they might need 10 dependencies and + +00:34:43.600 --> 00:34:49.340 +so on it but it doesn't matter right yeah yeah and being standard it makes it also you know like + +00:34:49.419 --> 00:34:54.879 +other tools are doing the same or hatch run that's the same that's like yeah and then there are there + +00:34:54.899 --> 00:34:57.060 +There is even support for inline script metadata + +00:34:57.460 --> 00:35:00.160 +just released in latest pip 26. + +00:35:00.960 --> 00:35:04.280 +So it's all good because of the standards + +00:35:04.380 --> 00:35:06.140 +and not because a single particular tool + +00:35:06.200 --> 00:35:07.620 +does it in an opinionated way. + +00:35:07.740 --> 00:35:09.680 +So this is really, really, really cool. + +00:35:09.860 --> 00:35:13.780 +And there is one big benefit of those kinds of apps, + +00:35:14.100 --> 00:35:16.060 +and particularly inline script metadata, + +00:35:16.660 --> 00:35:18.880 +is we have less YAML because of that. + +00:35:19.180 --> 00:35:19.480 +- Yeah. + +00:35:21.600 --> 00:35:22.760 +- You already have a lot of YAML. + +00:35:22.860 --> 00:35:23.680 +- Less is better. + +00:35:24.840 --> 00:35:26.740 +We have a lot still, we can't complain about that. + +00:35:28.380 --> 00:35:29.480 +- It's better than it was. + +00:35:30.020 --> 00:35:32.020 +Yeah, and so the dependency groups are like, + +00:35:32.740 --> 00:35:37.060 +you know, for dev or for test or something like that, right? + +00:35:37.060 --> 00:35:44.640 +So you can say like uv sync or uv pip install + +00:35:44.820 --> 00:35:47.160 +and you can say like thing bracket dev + +00:35:47.200 --> 00:35:48.060 +or something like that, right? + +00:35:48.540 --> 00:35:51.380 +- Or actually the nice thing about uv sync + +00:35:51.520 --> 00:35:53.920 +is that it syncs the dev dependencies automatically + +00:35:53.940 --> 00:35:55.220 +without you even specifying that, + +00:35:55.790 --> 00:35:57.860 +which is like the best thing for development + +00:35:58.080 --> 00:35:59.460 +because you actually always want to have + +00:35:59.580 --> 00:36:02.120 +development tools with you. + +00:36:02.600 --> 00:36:03.520 +- That's a good point, yeah. + +00:36:04.540 --> 00:36:04.940 +Totally agree. + +00:36:05.170 --> 00:36:06.200 +Okay, fantastic. + +00:36:06.200 --> 00:36:07.600 +- That's really cool, that's really cool. + +00:36:10.300 --> 00:36:13.540 +- So that was the changes to Python itself + +00:36:13.690 --> 00:36:15.960 +through the PEPS, but there's also tools + +00:36:16.240 --> 00:36:18.820 +and you've already mentioned some of them, both of them, + +00:36:18.960 --> 00:36:20.460 +but tools that make this possible, + +00:36:21.140 --> 00:36:24.420 +which I mean, I think uv has to be number one + +00:36:24.510 --> 00:36:25.620 +that goes on this list, right? + +00:36:25.720 --> 00:36:29.380 +Like uv has really done some powerful stuff here, right? + +00:36:29.950 --> 00:36:30.900 +- Yeah, absolutely. + +00:36:31.820 --> 00:36:33.820 +Yeah, so maybe again, Amok can say like, + +00:36:34.580 --> 00:36:37.360 +I introduced it, but Amok was the one to switch + +00:36:37.410 --> 00:36:38.920 +to use uv at some point of time. + +00:36:40.560 --> 00:36:42.720 +- Yup, uv has been a game changer. + +00:36:42.790 --> 00:36:45.100 +I think we were using poetry before this or? + +00:36:45.860 --> 00:36:47.280 +- No, no, no, not even that, just pip. + +00:36:48.210 --> 00:36:48.620 +Just pip, just pip. + +00:36:48.840 --> 00:36:49.360 +- Just pip, right? + +00:36:50.280 --> 00:36:53.220 +It's so good. I don't even remember the last thing. + +00:36:53.900 --> 00:36:55.440 +Yeah. So, yep. + +00:36:55.580 --> 00:36:58.220 +I think the main, you know, + +00:36:58.220 --> 00:37:02.060 +game changing aspect that uv brought in was this notion of workspaces. + +00:37:04.259 --> 00:37:08.680 +It's something very, you can compare it very similar to, you know, + +00:37:09.160 --> 00:37:13.240 +a coworking space or something similar where it's a unified environment where + +00:37:14.920 --> 00:37:19.020 +multiple interconnected pieces coexist and they're very easy to manage. + +00:37:19.960 --> 00:37:22.360 +And that's something that eventually led us + +00:37:22.650 --> 00:37:26.880 +to splitting the whole repository across our distributions. + +00:37:26.990 --> 00:37:29.080 +And that's the reason you see so many TOML files. + +00:37:29.800 --> 00:37:32.240 +So everything has a PI project TOML, + +00:37:32.440 --> 00:37:35.680 +everything defines the dependency groups it needs + +00:37:36.400 --> 00:37:39.700 +and development of a particular package + +00:37:40.420 --> 00:37:42.020 +is restricted only to its dependencies. + +00:37:43.360 --> 00:37:46.040 +So you develop it, you run uv sync, + +00:37:46.840 --> 00:37:50.940 +You can run your pytest using uv, + +00:37:51.260 --> 00:37:53.360 +and everything that is supposed to run with it + +00:37:53.560 --> 00:37:57.240 +is running with it, and any bad or cross imports + +00:37:57.320 --> 00:37:58.520 +are caught really easily. + +00:37:58.780 --> 00:38:02.780 +So I think the workspace feature, at least, + +00:38:02.940 --> 00:38:05.020 +was the most important one for me, + +00:38:05.880 --> 00:38:07.640 +and obviously the speed that it brings with it. + +00:38:07.820 --> 00:38:10.380 +So that's very impressive as well. + +00:38:11.140 --> 00:38:12.020 +It is. + +00:38:12.080 --> 00:38:15.920 +And I think this workspace concept-- + +00:38:15.940 --> 00:38:16.560 +It's new to me. + +00:38:16.640 --> 00:38:17.560 +I'll say it's new to me. + +00:38:17.580 --> 00:38:20.180 +I don't know how new it is to other folks. + +00:38:22.080 --> 00:38:26.120 +So you've got this giant monorepo. + +00:38:27.020 --> 00:38:29.720 +And how many different, conceptually, + +00:38:29.920 --> 00:38:34.540 +different packages or projects are in there right now? + +00:38:35.160 --> 00:38:35.960 +120 plus. + +00:38:36.700 --> 00:38:39.440 +It changes by day because Amok is doing a lot + +00:38:39.580 --> 00:38:41.360 +to increase the number very, very quickly + +00:38:41.880 --> 00:38:44.560 +because we are just now in the middle of finishing + +00:38:44.580 --> 00:38:46.960 +some isolation kind of restructuring. + +00:38:47.540 --> 00:38:48.360 +And Amog is the one that, + +00:38:48.520 --> 00:38:50.900 +that's why he's here also to lead the introduction + +00:38:51.120 --> 00:38:52.840 +of new packages that we, + +00:38:52.900 --> 00:38:54.580 +or new distributions that we have, + +00:38:54.740 --> 00:38:57.400 +like a shared libraries that we will talk about later. + +00:38:58.080 --> 00:39:00.000 +So we have a lot of those, yes. + +00:39:00.500 --> 00:39:01.180 +Yeah, amazing. + +00:39:01.880 --> 00:39:04.060 +So I think this is super important to dive into + +00:39:04.360 --> 00:39:05.840 +and how uv makes this possible. + +00:39:05.960 --> 00:39:08.200 +And I think you said also Hatch, + +00:39:08.420 --> 00:39:09.460 +you talked with Ofec, + +00:39:10.020 --> 00:39:11.860 +who runs Hatch as well about this, right? + +00:39:12.320 --> 00:39:12.760 +Yes, yes. + +00:39:12.980 --> 00:39:14.260 +Hatch is also supporting WorkSpaces, + +00:39:14.340 --> 00:39:20.920 +which are modeled mainly about what like after the what uv has done has done we haven't tried it yet + +00:39:21.140 --> 00:39:25.660 +but i've heard it's very very similar or even like you can use it as a one-to-one replacement + +00:39:26.100 --> 00:39:32.020 +in some cases or maybe even in all uh but generally i would love this eventually to become some kind + +00:39:32.020 --> 00:39:37.360 +of standard so that multiple tools are supporting this but but yes the there are a few other tools + +00:39:37.480 --> 00:39:43.060 +that we were considering before but uv is by far the kind of like yeah well we worked together we + +00:39:42.980 --> 00:39:47.540 +shaped it together with the U18. So it definitely works well. + +00:39:47.600 --> 00:39:53.620 +Amazing. Yeah, amazing. So let me describe this a little bit. And then you all can, can actually + +00:39:53.860 --> 00:39:59.700 +introduce it. So the idea is we've got this mono repo with a bunch of different folders for the + +00:40:00.660 --> 00:40:07.460 +sections, right, like airflow dash CLI or CTL, and airflow dash core and so on. And you'd like to be + +00:40:07.440 --> 00:40:13.120 +able to kind of just jump into one section and treat it as a top level project, right? It's got + +00:40:13.120 --> 00:40:18.180 +a pyproject.toml, it's got a source file, tests, and so on. But the challenge is you can't just + +00:40:18.180 --> 00:40:24.240 +have a bunch of disconnected pieces, like maybe Airflow core depends on five other parts of it + +00:40:24.340 --> 00:40:31.240 +that are also themselves have their own pyproject.toml and different things. And you've got to set up, + +00:40:31.520 --> 00:40:34.780 +you know, set up. If you jump into the Airflow core, you got to set up the environment just + +00:40:34.820 --> 00:40:40.240 +right to be working on those other parts right it sounds um it sounds pretty tricky so how does + +00:40:40.690 --> 00:40:48.300 +how does that work who wants to make sense of this for us yeah okay so like it works perfectly + +00:40:48.730 --> 00:40:52.880 +like it's super it's super simple actually you know like the whole thing about the uv is + +00:40:53.080 --> 00:40:58.380 +like its simplicity of the of of the of the of the not of the concept the implementation is actually + +00:40:58.560 --> 00:41:02.900 +quite tricky but the way how you use it is very simple you just go to the directory and run + +00:41:02.880 --> 00:41:08.840 +uv sync that's basically it so like this is uh this is to the directory you want to work on and + +00:41:08.840 --> 00:41:14.440 +it it does exactly what you would expect it to do which means that it syncs it actually updates the or + +00:41:15.220 --> 00:41:19.260 +recreates basically the virtual environment that you're using with all the dependencies that this + +00:41:19.500 --> 00:41:26.000 +particular uh distribution needs and anything that it needs uh as a transitive dependency as well + +00:41:26.370 --> 00:41:30.780 +so if it refers to another project project inside the workspace it will also use it from there + +00:41:30.860 --> 00:41:35.120 +not from like installed by pi pi so we can immediately start working on this because + +00:41:35.760 --> 00:41:39.400 +everything after uv sync everything is exactly as you expect for this particular + +00:41:40.040 --> 00:41:44.560 +subset of the repository that you work on and that's basically it this is this is all + +00:41:44.710 --> 00:41:49.760 +like there is nothing more basically it's like that's it it works and and you can when + +00:41:49.860 --> 00:41:55.580 +you're done you run uv sync py test run it will do exactly what you want so in this folder because + +00:41:55.600 --> 00:41:58.200 +because it will just, oh, sorry, uv run pytest. + +00:41:58.280 --> 00:41:58.680 +- uv run pytest. + +00:41:58.680 --> 00:41:59.760 +- You can do exactly what you want, + +00:41:59.940 --> 00:42:02.060 +because even uv run will automatically sync + +00:42:03.180 --> 00:42:05.660 +the virtual env very, very quickly + +00:42:06.160 --> 00:42:08.060 +to the one that your project needs, + +00:42:08.420 --> 00:42:10.260 +and then it will just run pytest + +00:42:10.420 --> 00:42:11.340 +in this virtual environment, + +00:42:11.620 --> 00:42:13.120 +and it will run all the tests in your project, + +00:42:13.220 --> 00:42:14.560 +and that's basically it. + +00:42:15.020 --> 00:42:17.180 +So it's like, conceptually for the users, + +00:42:17.380 --> 00:42:19.040 +it's like, you don't have to do much, + +00:42:19.180 --> 00:42:20.640 +just uv sync, and that's it. + +00:42:24.520 --> 00:42:30.000 +yeah how how interesting i i think one of the big challenges here is you know how does how do + +00:42:30.170 --> 00:42:37.460 +different parts of the project know about each other right yeah yeah you said that it uh it sim + +00:42:37.780 --> 00:42:44.200 +links the different elements in well okay so like the basic the basic the basic kind of workspace + +00:42:44.460 --> 00:42:48.580 +and implementation is just a workspace definition so you have to have the finishing of workspace + +00:42:48.920 --> 00:42:54.400 +in the top level byproduct tom so there you have all of them listed you have links to it you have + +00:42:54.420 --> 00:43:00.780 +describe where they are and uv will read the PI project on from the top level and will + +00:43:01.260 --> 00:43:03.520 +know where to look for particular distributions. + +00:43:04.190 --> 00:43:10.680 +So that's the simple discovery and the way how we know that we are using it from the sources + +00:43:10.900 --> 00:43:13.780 +and not from the PI PI. + +00:43:14.920 --> 00:43:19.840 +But then like the shared libraries is like something that we added on top of it and the + +00:43:19.780 --> 00:43:25.400 +sim links are on the top of it and this is kind of extra innovative thing that we are doing for + +00:43:25.580 --> 00:43:29.680 +something else that we need but you know we can we can talk about that now or like i'm not can + +00:43:29.780 --> 00:43:37.640 +talk about that sure okay yeah yeah good yeah we can talk about it when we're talking about it + +00:43:39.700 --> 00:43:41.480 +okay sounds good so + +00:43:43.860 --> 00:43:51.180 +this is really cool one of the things that happens here is these different these different slices or + +00:43:51.500 --> 00:43:56.640 +subsections of the monorepo have a pyproject.toml that pyproject.toml + +00:43:56.860 --> 00:44:03.540 +defines its true dependencies and its dev dependencies and so on so when you go and jump + +00:44:03.740 --> 00:44:12.320 +into a section it will uv will basically realign the virtual environment with whatever + +00:44:13.440 --> 00:44:17.080 +dependencies are supposed to be there from those things right so that means installing stuff + +00:44:17.400 --> 00:44:22.160 +obviously but actually what surprised me a little bit not a lot but oh yeah i guess it does do that + +00:44:22.280 --> 00:44:28.820 +that's cool is it actually uninstalled stuff that's not explicitly put there which i can imagine + +00:44:29.300 --> 00:44:35.900 +before that you could be like well this one part way down here depends on this weird library + +00:44:37.060 --> 00:44:41.640 +and somehow i used to be over there then i went back to the this other piece then i came back and + +00:44:41.560 --> 00:44:43.640 +and I forgot where that even came from. + +00:44:43.720 --> 00:44:45.260 +Like, why is that in my virtual environment? + +00:44:45.500 --> 00:44:47.080 +And like, how do I specify that? + +00:44:47.360 --> 00:44:49.140 +Probably juggling that was a big problem, right? + +00:44:49.260 --> 00:44:52.700 +This like loading and unloading dependencies + +00:44:52.920 --> 00:44:55.020 +based on what part of the monorepo you're in. + +00:44:55.020 --> 00:44:58.880 +And I think that actually makes it really much easier + +00:44:58.940 --> 00:45:01.000 +to deal with like this type of code structure. + +00:45:01.740 --> 00:45:02.260 +Yeah, absolutely. + +00:45:02.540 --> 00:45:04.300 +And let me add to that one more thing + +00:45:04.400 --> 00:45:06.340 +because it's also not only the dependencies + +00:45:06.520 --> 00:45:07.960 +that you might have from somewhere else, + +00:45:08.700 --> 00:45:14.940 +but also it's cross dependencies between different distributions inside so for example if our flow + +00:45:15.880 --> 00:45:21.040 +ctl does not use airflow core if you go there and run you think you will not be able to import and + +00:45:21.140 --> 00:45:26.160 +use any of the source code which is in airflow core because it's not a dependency of our flow ctl + +00:45:26.660 --> 00:45:30.540 +so uv sync will not only uninstall the dependencies that you have but also i mean install the + +00:45:30.680 --> 00:45:35.700 +the source code that you have from other parts of europeo which is a fantastic thing for us and + +00:45:35.580 --> 00:45:37.220 +And that was exactly what was missing before, + +00:45:37.440 --> 00:45:38.860 +kind of isolation between those. + +00:45:39.490 --> 00:45:41.660 +You only actually can, from your source, + +00:45:41.730 --> 00:45:45.000 +you only can refer to the source code + +00:45:45.090 --> 00:45:46.980 +of those distribution that you depend on + +00:45:47.490 --> 00:45:48.960 +and nothing else from the monorepo. + +00:45:49.150 --> 00:45:51.220 +So this means that it's like, + +00:45:51.720 --> 00:45:54.560 +you can slice and dice your repository as you want. + +00:45:55.120 --> 00:45:58.020 +So depending on in which directory you are + +00:45:58.020 --> 00:46:00.240 +and when you run uv sync, you will have like subset, + +00:46:00.720 --> 00:46:03.800 +like the actual useful and the used subset + +00:46:03.930 --> 00:46:04.660 +from your repository. + +00:46:04.940 --> 00:46:07.560 +And it can be completely different if you go to another directory. + +00:46:07.920 --> 00:46:09.000 +Some of that can be overlapping. + +00:46:09.280 --> 00:46:11.040 +Some of that can be completely different. + +00:46:11.520 --> 00:46:13.740 +Depends like which dependencies are defined. + +00:46:13.850 --> 00:46:16.300 +And this is like, this all magically happens + +00:46:16.960 --> 00:46:20.340 +like by just defining the dependency in PyProject.com, nothing else. + +00:46:20.480 --> 00:46:22.780 +I mean, this thing will handle it for you in the workspace. + +00:46:23.460 --> 00:46:27.420 +This is like exactly the reason why it's so useful for development. + +00:46:29.280 --> 00:46:32.520 +- One thing, one more thing to add to that is, yeah, exactly. + +00:46:32.780 --> 00:46:37.580 +as I said, it helped us in our vision to actually, you know, + +00:46:37.740 --> 00:46:42.800 +decompose the project into multiple parts and avoid the classic + +00:46:43.300 --> 00:46:44.700 +problem of coupling, + +00:46:45.910 --> 00:46:49.320 +which every Mono repo faces at some point in their life cycle, + +00:46:49.740 --> 00:46:52.740 +because everything is out there. Why don't we just, you know, + +00:46:53.260 --> 00:46:57.500 +have code leaks all over the place. So this helps us prevent that. + +00:46:57.720 --> 00:47:01.580 +And I cannot imagine a time how we did it earlier before uv. + +00:47:02.420 --> 00:47:04.500 +I don't know if we did it, but if we did it, + +00:47:04.500 --> 00:47:06.500 +it would have been a really tough thing to do. + +00:47:07.240 --> 00:47:07.420 +Yeah. + +00:47:08.000 --> 00:47:09.820 +There's a bunch of tools that you can, + +00:47:10.700 --> 00:47:12.440 +like linters and code analysis, + +00:47:12.580 --> 00:47:15.060 +things you can run on your code that breaks down. + +00:47:15.540 --> 00:47:17.400 +For these different modules and these layers, + +00:47:17.740 --> 00:47:20.380 +here's like a directed graph of how this thing, + +00:47:21.020 --> 00:47:22.440 +and you can set up rules to say, + +00:47:22.540 --> 00:47:24.280 +this should never cross that boundary, + +00:47:24.820 --> 00:47:29.680 +but these are just very vague things. + +00:47:29.940 --> 00:47:35.580 +And this setup actually makes it so it's not accessible to your code if you didn't say it + +00:47:35.680 --> 00:47:35.860 +should be. + +00:47:36.360 --> 00:47:36.480 +Yes. + +00:47:36.940 --> 00:47:37.040 +Yes. + +00:47:37.110 --> 00:47:42.200 +So it's just built in exactly the definition of your distribution, which you anyhow have + +00:47:42.340 --> 00:47:45.420 +to do because you have to define what the dependencies are. + +00:47:45.980 --> 00:47:47.540 +And yes, we did something like that before. + +00:47:47.670 --> 00:47:49.960 +So we got a number of rough rules or whatever. + +00:47:50.520 --> 00:47:51.840 +Don't import here. + +00:47:53.000 --> 00:47:56.960 +We still have them for shared libraries, which we can talk about now, because I think this + +00:47:56.900 --> 00:48:02.980 +is an important modification of the of the concept so we do do have some automated check for uh + +00:48:03.260 --> 00:48:10.340 +quality and for imports uh with prague our precomit hook implementation but but before that it was + +00:48:10.460 --> 00:48:16.900 +just completely completely uh like handwritten and unmaintainable people will not we're not actually + +00:48:17.240 --> 00:48:23.359 +updating it with all the distributions you couldn't really you know follow uh when things change with + +00:48:23.380 --> 00:48:28.960 +with my project tom being the for each distribution being the single source of truth you don't have to + +00:48:29.020 --> 00:48:33.880 +do anything because the dependency is declared there and this is like the best part of of + +00:48:34.520 --> 00:48:40.000 +uv understanding that and and doing everything that is like reasonable in this case yeah + +00:48:42.160 --> 00:48:51.680 +yeah yeah so the other major tool involved here was prick which it's a pre-commit framework for + +00:48:51.700 --> 00:48:58.120 +running hooks many languages but especially python relevant here written in rust so it pairs well + +00:48:58.250 --> 00:49:04.680 +with uv i suppose oh yeah it was inspired by uv as well and joe was mentioning mentions that + +00:49:06.180 --> 00:49:14.960 +he was actually contributing to uv yeah cool all right so where's preck how's preck show up here + +00:49:15.450 --> 00:49:18.800 +uh among i feel like this is leading towards what you were hinting at earlier + +00:49:19.740 --> 00:49:29.960 +yeah i'm okay yeah yep so i think it started with uh it was called pre-flight earlier i believe + +00:49:30.420 --> 00:49:35.360 +that's the name preferably git that was the original name oh yeah it was like terrible + +00:49:37.440 --> 00:49:43.200 +it's a new name uh preq so yep this allows us to do things which uh + +00:49:44.520 --> 00:49:52.740 +pre-commit did not do or you know did not accept as suggestions so one certain thing that preck + +00:49:52.840 --> 00:50:00.880 +offers is obviously it's written in rust so speed is the obvious uh advantage that we get but apart + +00:50:00.940 --> 00:50:05.980 +from that we also get this notion of it pairing well with uv in terms of modularized hooks + +00:50:06.800 --> 00:50:13.200 +earlier we had all the hooks in one place in that in the top level pre-commit yaml right and + +00:50:14.060 --> 00:50:22.840 +it was it was a big fight it was you can imagine so yeah so this break allowed us to + +00:50:23.670 --> 00:50:30.240 +break again you know it it consumed the concept of workspaces here i would say so it allowed you to + +00:50:30.680 --> 00:50:38.480 +define pre-commit hooks or preq hooks within a module itself and this paired well with uv in + +00:50:38.500 --> 00:50:41.660 +in the sense that when you have to run hooks + +00:50:41.980 --> 00:50:43.820 +that are bound to a certain distribution, + +00:50:44.400 --> 00:50:48.640 +all you have to do is check in into the, you know, + +00:50:48.670 --> 00:50:50.400 +the sub module and just do a preck run. + +00:50:50.450 --> 00:50:54.060 +It will run the relevant hooks for that particular module. + +00:50:54.700 --> 00:50:58.540 +And the other thing that I really love about preck + +00:50:58.620 --> 00:51:03.680 +is auto-completion, which is not something pre-commit had. + +00:51:03.750 --> 00:51:06.800 +So you can imagine that something fails in the CI, + +00:51:06.980 --> 00:51:13.860 +have to copy that and copy the ID and try to kind of backtrack it in your repo as to which one is + +00:51:13.900 --> 00:51:20.200 +failing. So it used to be a nightmare, but now with the dApp completion, it's amazing. + +00:51:20.700 --> 00:51:24.680 +Nice. Are you talking about like shell autocomplete integration? + +00:51:24.680 --> 00:51:24.800 +Yes. + +00:51:25.540 --> 00:51:28.260 +Yeah. Yeah. So, okay. Nice. I've seen... + +00:51:28.260 --> 00:51:31.160 +So I have some story about that very short one. So like, + +00:51:31.560 --> 00:51:36.900 +we actually tried to get autocompletion for hook names with pre-commit, which was the predecessor. + +00:51:36.900 --> 00:51:43.680 +of Prac, like Prac was largely based on Pracomit, but somehow the author of it didn't accept + +00:51:44.020 --> 00:51:50.420 +even idea of us contributing it or actually had some very, very excessive expectations for that. + +00:51:51.120 --> 00:51:55.200 +And we, you know, discussed and like, there were like other people were also trying to + +00:51:55.640 --> 00:52:01.460 +convince the author to do that, but they refused, he refused basically, and refused to accept + +00:52:01.540 --> 00:52:07.200 +contributions even uh so when when we spoke to joe that was like completely different stories like + +00:52:07.480 --> 00:52:13.060 +we need that and next day it was there like it's like completely different approach so so this + +00:52:13.140 --> 00:52:17.800 +is uh and then we said like we need workspaces and like a few weeks later because it took a little + +00:52:17.940 --> 00:52:22.220 +bit of time it was there and we worked together and we tested that and like i raised i don't know + +00:52:22.220 --> 00:52:28.740 +how many issues in the initial kind of pre-release version when when we wanted to use it so so the + +00:52:28.760 --> 00:52:33.800 +i think the collaboration and being you know working together listening to your users and be + +00:52:33.920 --> 00:52:39.040 +responding to them and and actually working as an open source maintainers together this this + +00:52:39.220 --> 00:52:43.700 +actually worked perfectly well here both both in uv and and in practice and this is why we love + +00:52:43.920 --> 00:52:49.240 +prague actually because because we know we can rely if something is not working that it's gonna be + +00:52:49.320 --> 00:52:56.560 +like we can discuss and either submit a fix or or you know joe will do this or even like lots of + +00:52:56.360 --> 00:53:01.440 +other people can do it because i think the there was uh there was a few features that we wanted + +00:53:01.620 --> 00:53:06.820 +and somebody else implemented it and that wasn't joe they contributed to crack because of this + +00:53:06.880 --> 00:53:13.780 +openness and you know being able to accept the needs of the users so that was yeah that was very + +00:53:13.900 --> 00:53:18.980 +very important part like why we moved to crack yeah i think airflow was also one of the initial + +00:53:19.300 --> 00:53:25.620 +case studies for break it's a project of that scale and if you kind of satisfy that project's + +00:53:25.720 --> 00:53:33.520 +needs you are you're pretty good with most use cases i think that's exactly both like and uv as + +00:53:34.500 --> 00:53:39.420 +yeah right there at the top of the prec repo it says although prec is pretty new it's already + +00:53:39.480 --> 00:53:45.460 +powering real projects you know little things like c python apache airflow and fast api so yeah you + +00:53:45.460 --> 00:53:51.540 +know i i know who gov and kamerate from uh release manager of python uh so we met at for them as + +00:53:51.480 --> 00:53:56.000 +well. And like he was actually listening to our preg discussion and he converted, you know, + +00:53:56.540 --> 00:54:00.980 +cpython to use preg because of the of the needs they had. So like, it was all about, you know, + +00:54:01.100 --> 00:54:04.500 +people talking to each other, word of mouth and things like that. + +00:54:05.240 --> 00:54:12.780 +Yeah. You know, there's a feature listed here that just makes me jealous. One of the features + +00:54:12.960 --> 00:54:17.540 +of preg is a single binary with no dependencies that doesn't require Python or any other runtime + +00:54:17.560 --> 00:54:24.680 +to be installed. Like, how incredible would it be with Python if we had a, you know, a Python, dash, + +00:54:24.870 --> 00:54:30.300 +dash, build app or something, you know what I mean, you can put it at your thing and you get something + +00:54:30.600 --> 00:54:35.940 +you could distribute. I know uv solves a lot, but you still got to have uv installed. And then you + +00:54:35.970 --> 00:54:41.520 +know, like this, that is a huge advantage of things like Rust and go and some other languages. + +00:54:42.500 --> 00:54:45.000 +It's both good and bad in some cases. + +00:54:45.140 --> 00:54:46.900 +So it's like there are always trade-offs. + +00:54:47.140 --> 00:54:49.160 +It's different choice made by Python here. + +00:54:49.600 --> 00:54:52.860 +I don't think it's like the best choice for Python. + +00:54:53.100 --> 00:54:54.440 +I think Python being script language, + +00:54:55.360 --> 00:54:57.480 +it's okay to have, you know, like dependencies + +00:54:57.940 --> 00:54:59.920 +and especially like inline script metadata + +00:55:00.600 --> 00:55:02.960 +almost did it because you just, you know, + +00:55:03.420 --> 00:55:04.900 +can install stuff and uv also, + +00:55:05.280 --> 00:55:07.660 +and the kind of tooling is also doing all the stuff + +00:55:07.820 --> 00:55:09.640 +like uv install or uv tool install, + +00:55:11.080 --> 00:55:13.780 +whatever, and it would not only install the project, + +00:55:14.320 --> 00:55:15.780 +its dependencies, but also install Python + +00:55:16.000 --> 00:55:17.320 +that is needed to run it. + +00:55:17.440 --> 00:55:20.100 +So like all this is really a matter of two weeks + +00:55:20.330 --> 00:55:22.340 +and it has improved dramatically over the last few years. + +00:55:22.340 --> 00:55:23.220 +- It definitely has. + +00:55:24.060 --> 00:55:27.000 +Yeah, I was pining for an option, + +00:55:27.240 --> 00:55:29.900 +not a only binary thing. + +00:55:30.600 --> 00:55:32.120 +All right, so one thing I actually wanna talk about, + +00:55:32.240 --> 00:55:34.720 +going back to this workspaces thing real quick is, + +00:55:36.660 --> 00:55:43.800 +what does it look like from a ide or editor experience to work on this all right like + +00:55:44.520 --> 00:55:51.200 +you've got pyjarm projects you've got maybe vs code workspaces where you can pull in different + +00:55:51.420 --> 00:56:00.800 +pieces how do you all manage that i'm working yeah so i cannot talk for vs code i'm a pyjarm user + +00:56:00.820 --> 00:56:09.940 +here, but we had to do a little bit of hacking, I would say, or more like a helper script for the + +00:56:10.180 --> 00:56:18.140 +IDs, right? Because so we have a ID helper script right in the repo and we recommend the users to + +00:56:18.140 --> 00:56:26.320 +run it so that the ID knows what is where in terms of maintaining things, right? Because in normal + +00:56:26.340 --> 00:56:33.420 +projects, there's usually just one source, one test at the top level, but as 120 plus and + +00:56:34.520 --> 00:56:39.140 +the helper script is, it does a pretty simple thing. It just auto discovers all the packages + +00:56:39.210 --> 00:56:48.600 +in the monorepo and adds, there's a, so IntelliJ and PyCharm both have a dot idea within each, + +00:56:49.790 --> 00:56:54.340 +a hidden folder within each of the projects that it opens and it has a, + +00:56:55.060 --> 00:57:01.160 +and it supports XML like format for IML where you can define certain things. So this essentially + +00:57:01.500 --> 00:57:07.820 +does a very simple thing. It just for each package, it adds the module slash source as the source + +00:57:07.980 --> 00:57:14.940 +root and the module slash tests are the test. So it's as if you went through all 120 things + +00:57:14.970 --> 00:57:18.360 +and right clicked and said mark as sources root or something like that. Yeah. You wouldn't want + +00:57:18.180 --> 00:57:21.960 +to do that. We did that actually. We did that for some time. + +00:57:22.520 --> 00:57:22.960 +- Initially. + +00:57:25.780 --> 00:57:30.300 +Yeah, we had this PyCharm script and then we have the same approach for VS Code. So we have another + +00:57:30.500 --> 00:57:35.240 +script for VS Code as well, which was contributed by someone who uses VS Code because either me or + +00:57:35.720 --> 00:57:44.060 +Amog are VS Code users. But communities also, and somebody said, "Okay, I'll do it." And there it was. + +00:57:44.100 --> 00:57:48.840 +and they tested it and you know like that's that was super cool actually so so yeah it works well + +00:57:49.240 --> 00:57:54.720 +also the you know a little bit of words uh i would probably we don't talk we won't talk too much + +00:57:55.360 --> 00:58:00.560 +about like the we don't have too much time but the shared libraries concept a little bit might maybe + +00:58:01.100 --> 00:58:06.500 +it's the right time to introduce the yeah yeah because we like one thing that amov mentioned + +00:58:06.740 --> 00:58:12.160 +is like the uh we had we solved this coupling problem but also we wanted to solve the dry + +00:58:12.180 --> 00:58:17.540 +problem. And those two are always kind of mixture, like you get dry and then you get more dry and + +00:58:17.660 --> 00:58:22.540 +less coupling and like, like more dry and more coupling and like, all these things are complex + +00:58:22.720 --> 00:58:27.960 +when you have lots of code. Dry being the architectural philosophy of do not repeat + +00:58:28.140 --> 00:58:32.800 +yourself. But if you're not repeating yourself, every, everything where if it exists somewhere, + +00:58:33.000 --> 00:58:37.100 +everything's got to depend on that somewhere. And it starts to become more linked together, right? + +00:58:37.540 --> 00:58:44.060 +precisely so so it's a little bit of like uh uh you know it can have it too like the the + +00:58:44.300 --> 00:58:49.000 +we want to have dry code and not to repeat it for like common utilities like logging + +00:58:49.600 --> 00:58:53.480 +configuration whatever all the things that that are kind of common between all the different + +00:58:53.640 --> 00:58:59.240 +distributions but also we didn't want to depend on a single version of those because if we do + +00:58:59.820 --> 00:59:04.000 +then it means that we have to make sure that the backwards compatibility is maintained because like + +00:59:04.020 --> 00:59:08.120 +when we install different version of different distributions coming from different time + +00:59:08.940 --> 00:59:14.540 +of repository they might use different version of those search libraries and like how to make sure + +00:59:14.620 --> 00:59:18.580 +that they don't do not have breaking changes and stuff like so this is all the whole level + +00:59:18.590 --> 00:59:23.740 +of complexity between like how to manage the dependencies there and manage versions especially + +00:59:24.020 --> 00:59:29.600 +manage the backwards compatibility so we figure out that with some very simple approach we tried + +00:59:29.620 --> 00:59:35.160 +few different approaches but like one of the approaches was using the vendor ink library from + +00:59:35.320 --> 00:59:42.640 +pip and from byton uh not from people from pip and the second one and that's the one we came up we + +00:59:42.690 --> 00:59:47.200 +we finally implemented with like using sim links to share the code between different distributions + +00:59:47.490 --> 00:59:52.880 +and that's a very innovative approach that i hope uh will make it into some kind of standard + +00:59:53.160 --> 00:59:58.000 +eventually so like we came with this approach where we actually have cake and eat it too like + +00:59:58.020 --> 01:00:02.740 +which is like pretty amazing if you fought with like for years with this kind of + +01:00:03.360 --> 01:00:08.720 +common dependency issues that and and and backwards compatibility so in our case like the symlink + +01:00:08.880 --> 01:00:14.880 +approach we have and it needs some preq kind of pre-processing of by project tom so some parts of + +01:00:14.940 --> 01:00:19.720 +the by project tom are generated to make it actually work but this is all automated with + +01:00:19.920 --> 01:00:24.240 +preq which is like you don't have to think about that even and once we do that and once we create + +01:00:24.260 --> 01:00:30.260 +some similar sim links between different parts of code like one library one distribution is + +01:00:30.360 --> 01:00:35.340 +sim linking code from the shared distribution the end result is that this code gets automatically + +01:00:35.780 --> 01:00:41.300 +vendored in during the building of the package which means that we actually have the same library + +01:00:41.520 --> 01:00:46.920 +in different package in different version in different distributions so distribution released + +01:00:47.520 --> 01:00:53.600 +week ago will have a shared configuration from a week ago but this another distribution will have + +01:00:53.620 --> 01:00:59.280 +the same shared configuration code from today if it's released today and we can install them + +01:00:59.520 --> 01:01:06.200 +together and all of them have effectively uh like if they had a different version of the + +01:01:06.360 --> 01:01:13.400 +same library installed i see it's as if the airflow dash ctl said it had a dependency on + +01:01:13.600 --> 01:01:18.840 +core and it pinned that version to something but a different part of the repo pinned it to a + +01:01:18.940 --> 01:01:22.980 +different version right they can both kind of coexist that's but it's actually all within the + +01:01:22.900 --> 01:01:28.800 +same code file. That's insane. Okay. And this is like largely like, it's nothing new. It's largely + +01:01:30.380 --> 01:01:35.940 +inspired by how the, you know, libraries work in C and like traditional kind of building code. Like + +01:01:35.940 --> 01:01:41.880 +you have dynamic libraries and static libraries. So this is like essentially equivalent of static + +01:01:42.000 --> 01:01:47.280 +static libraries where you take the code of the version that you compile the stuff in and put it + +01:01:47.220 --> 01:01:52.960 +inside the final binary and then it results like in rust the kind of single binary thing so + +01:01:53.200 --> 01:01:59.120 +it's a little bit like so this we have a little bit of this single binary by doing that in like + +01:01:59.240 --> 01:02:03.600 +in the sense that we automatically vendor in all the you know shared dependencies that we have + +01:02:04.220 --> 01:02:10.820 +in the same distribution so so it's it's kind of hybrid but it's always like it's also like so rust + +01:02:10.880 --> 01:02:14.900 +is a little bit too far because everything is single binary in our case we have a bit of both + +01:02:14.900 --> 01:02:20.360 +like we can use libraries dynamically but we can also embed libraries as shared inside the + +01:02:21.140 --> 01:02:33.180 +single distribution. That's very cool. Amogh? Sounds like you were instrumental in this part. + +01:02:34.450 --> 01:02:44.860 +Yeah, so that's the nice thing about the approach that was chosen, right? We all came together as a + +01:02:44.980 --> 01:02:50.360 +we had one email uh devilish discussion one fine day that hey we want to achieve something like this + +01:02:51.080 --> 01:02:57.420 +which more or less was a uh was something everyone agreed upon so people started chiming in and we + +01:02:57.480 --> 01:03:01.920 +started trying different things out the first one obviously using the rendering tool from + +01:03:02.650 --> 01:03:08.980 +uh somebody did a poc on that but it felt like it's going to be difficult to achieve that over + +01:03:08.880 --> 01:03:13.920 +long term and and also it would it could be brittle so Jarek came up with this particular + +01:03:14.940 --> 01:03:20.640 +option with sim links which again was discussed within the community few of us a few of us picked + +01:03:20.640 --> 01:03:26.980 +it the picked this pr up passed it locally played around and gave the feedback so i don't think + +01:03:27.050 --> 01:03:34.060 +this would be possible with ai in in the sense that this has never been done before yeah so some + +01:03:34.120 --> 01:03:40.140 +something like this where a community comes together and solves a rather difficult problem + +01:03:40.440 --> 01:03:47.680 +is something that makes me really happy and also something that all of us are working towards a + +01:03:47.720 --> 01:03:55.860 +common goal while while also bound by our corporate hats right is something that is again really nice + +01:03:55.920 --> 01:04:04.660 +to see we have about how 11 i think at this point we have about 11 to 12 shared libraries where + +01:04:05.420 --> 01:04:12.700 +the main notion here is to reimagine airflow as a independent server and more like a control plane + +01:04:12.700 --> 01:04:19.240 +and execution plane what we did with airflow three and this shared libraries is helping us achieve + +01:04:20.560 --> 01:04:27.020 +that model. And we have about 11 to 12 of them. And I think a few more coming very soon. But yeah, + +01:04:28.860 --> 01:04:32.720 +it's been nice working on the shared libraries. It's been great. + +01:04:33.300 --> 01:04:38.200 +That's cool. Is this something that people can take and adopt into their monorepo if they want + +01:04:38.200 --> 01:04:44.540 +to live that life? Absolutely. It's really like one or two kind of + +01:04:44.400 --> 01:04:46.840 +of preq hooks, which are maintaining the consistency. + +01:04:47.590 --> 01:04:51.080 +And like, so that you don't forget to add this sim link here + +01:04:51.240 --> 01:04:54.480 +and that kind of PyProjectTOM definition here + +01:04:54.700 --> 01:04:57.560 +and the, or that hatch definition for the hatch link + +01:04:57.820 --> 01:05:01.320 +to actually embed your sim linked code + +01:05:01.490 --> 01:05:02.660 +into the final distribution. + +01:05:03.000 --> 01:05:04.380 +So like there are like a few pieces + +01:05:04.580 --> 01:05:07.420 +that have to be put together from existing libraries. + +01:05:08.320 --> 01:05:09.180 +So that's basically it. + +01:05:09.210 --> 01:05:11.340 +And once you do it, it's just, those are, + +01:05:11.730 --> 01:05:13.840 +the funny thing is like those shared libraries + +01:05:13.860 --> 01:05:19.760 +just standalone distributions you can actually build them separately as a library as well we could + +01:05:19.920 --> 01:05:24.560 +we could potentially even you know like just use them as library as well no problem whatsoever + +01:05:25.000 --> 01:05:29.880 +because they are just standard plain distributions or any other we just happen to take the source code + +01:05:29.980 --> 01:05:36.180 +of it and then embed it in into into the target distribution that wants to use it rather than you + +01:05:36.200 --> 01:05:41.120 +know link to it by dependency so that's basically other than that it's it's it's a kind of + +01:05:41.140 --> 01:05:46.320 +completely standard library and uh or standard distribution and one one more thing that is really + +01:05:46.400 --> 01:05:52.440 +important to add here is like this also has a side effect but i think a very nice one and ammo can + +01:05:52.560 --> 01:05:57.380 +confirm that because he has been doing a lot of that is like we actually come up with like way + +01:05:57.560 --> 01:06:03.020 +better internal architecture because of that because a lot of those that the shared libraries + +01:06:03.180 --> 01:06:08.000 +they depended on each other sometimes in a circular fashion sometimes it really depended + +01:06:08.020 --> 01:06:10.060 +and like which import you did first, like what happened, + +01:06:10.260 --> 01:06:11.480 +like what was initialized? + +01:06:11.960 --> 01:06:13.920 +And it was like complete spaghetti of dependencies + +01:06:14.360 --> 01:06:17.500 +between generally independent pieces of functionality. + +01:06:18.240 --> 01:06:20.140 +Right now by having shared libraries, + +01:06:20.860 --> 01:06:24.380 +we are actually forcing ourselves to make them isolated + +01:06:24.800 --> 01:06:28.380 +and we are changing the way how we initialize them. + +01:06:28.420 --> 01:06:30.540 +For example, we are injecting all the configuration + +01:06:30.940 --> 01:06:32.900 +rather than using them from inside the library + +01:06:33.380 --> 01:06:35.360 +because like configuration library is another library, + +01:06:35.400 --> 01:06:37.320 +so you don't want to depend on the other library. + +01:06:37.560 --> 01:06:39.660 +So it's really nice. + +01:06:39.820 --> 01:06:46.040 +I think it comes, the result is that really the architecture of Airflow internally is + +01:06:46.160 --> 01:06:47.460 +so much better because of that. + +01:06:47.680 --> 01:06:53.660 +So less surprises and explicit initialization is like something that we'll have to do rather + +01:06:53.680 --> 01:07:00.200 +than implicit initialization during imports, which has always been plaguing as a big issue. + +01:07:02.320 --> 01:07:21.180 +Certainly. It also allows you to imagine each component having an entry point, per se, where you have an initial starting point and it initializes everything it needs by injecting and calling certain factories, which makes a very clean for anyone visiting the project also. + +01:07:21.560 --> 01:07:23.660 +they look at something and they know the entry point very clearly. + +01:07:24.180 --> 01:07:26.640 +But here, this is how it starts, this is what it initializes. + +01:07:27.660 --> 01:07:34.200 +Reminds me of Golang or Java projects where they have a nice main where in Python, + +01:07:35.200 --> 01:07:38.720 +Python it's not really the same way, but yeah. + +01:07:39.480 --> 01:07:40.500 +Yeah, very cool. + +01:07:42.180 --> 01:07:45.280 +All right, well, I think that's about it for all the time we have. + +01:07:46.580 --> 01:07:50.600 +I guess let's close it out with one final thought here. + +01:07:50.610 --> 01:07:54.080 +It's just people who are maybe inspired by your design, + +01:07:54.660 --> 01:07:56.860 +by the way you put together Airflow + +01:07:56.930 --> 01:08:00.460 +and this monorepo concept, especially Python people, + +01:08:00.780 --> 01:08:01.900 +what do you say to them? + +01:08:02.690 --> 01:08:03.280 +Final thoughts here. + +01:08:04.660 --> 01:08:06.780 +- Final thoughts, just do it. + +01:08:06.790 --> 01:08:10.840 +I mean, like I initially, there was always discussion. + +01:08:10.950 --> 01:08:12.720 +Like we had lots of discussions internally, + +01:08:12.890 --> 01:08:14.760 +even some of the PM's members in Airflow, + +01:08:14.920 --> 01:08:19.620 +they let's split the repository into smaller one like let's make more of them because it's going + +01:08:19.620 --> 01:08:26.100 +to make things easier i was always the monorepo fan and and i made a lot of work to make it possible + +01:08:26.400 --> 01:08:30.900 +but that was a very very difficult thing it's changed so like the reasons why you would + +01:08:31.020 --> 01:08:36.700 +like to have multiple repos are gone now if you're using the right tooling and only the benefits or + +01:08:36.940 --> 01:08:41.060 +mostly the benefits from having it in one place where you can test everything together and work + +01:08:41.100 --> 01:08:48.060 +on it together remain all the rest is basically gone so for me uh like uh the discussion + +01:08:48.080 --> 01:09:00.580 +on the repo versus multi-repo is already solved nice yeah just do it uh we it's it's not even uh + +01:09:01.060 --> 01:09:07.779 +so personally i've been using uh using the readme that we have present in the + +01:09:08.680 --> 01:09:15.359 +shared libraries as a context for my uh id so it's it's turning out to be very nice uh for the + +01:09:15.880 --> 01:09:21.220 +shared library split for example all all i have to do is just provide it the context and tell + +01:09:21.240 --> 01:09:26.920 +it hey just just construct the structure for me and i can do everything else so it's that easy + +01:09:27.839 --> 01:09:33.700 +we have all the things in place we are in the right era to do it so just just do it okay + +01:09:34.859 --> 01:09:44.480 +very inspiring thank you for being here awesome for this look inside and it's apache airflow it's + +01:09:44.480 --> 01:09:50.680 +on github people can go look and see it's not just a talking vaguely about some internal project right + +01:09:50.839 --> 01:09:56.740 +so people can go check it out yeah all right see you later thanks thank you thanks bye +