This is Hacker Public Radio episode 3,656 from Monday the 8th of August 2022.
Today's show is entitled, Importance of Small Toy Projects.
It is hosted by Norris, and is about 19 minutes long.
It carries a clean flag, the summary is, toy projects are a great way to
learn a new language, and a project I did just for fun.
So, I wanted to do a quick episode on Toy Projects, or maybe a programming exercise or
project, or something you want to do that isn't necessarily important, or maybe not, even
usable by something else, but it's something you want to do, and a lot of times, you
know, call them Toy Projects, because they're small, but a lot of times they're fun.
So it's something you want to do and something you would enjoy doing just for the sake
of working on the project, not necessarily, you know, a lot of times Toy Projects do solve
a problem, but a lot of times they're more about the process and less about the problem
they're actually solving.
So, there's a, ever a couple podcasts that got me thinking about Toy Projects and sort
of why they're important, and I'm going to tell you what they are, but I'm going to tell
myself a little bit, I've been thinking about doing this episode for a while, so that some
of the, specifically this first one, the first podcast, is from the top Python, it's an
episode of the top Python podcast, and it's going to be about the time this comes out,
it's going to be over a year old, so that's just sort of how long I've had, I've been
making notes and thinking about this episode, but anyway, if you haven't listened to the
top Python podcast, it's a really good podcast, I always have a lot of good guests, and
it's obviously about Python, but it's, it's a really easy listen, and you know, I'm not
a Python developer, I'm just sort of a Python hobbyist, but I can always follow along
what they're saying, and it's a lot of fun, but anyway, in episode 327, they had a panel
of Python users, and it was, you know, from Python professionals to other professionals who
use Python on the side, and they were just talking about how some small things they did
in Python just to sort of make their life easier, so if you want to podcast, recommendation,
talk Python, specifically episode 327, and hacker public radio episode 3, 5, 8, the host talks
about learning Haskell and says something like, I'll get the quote exactly right, but
it's something like, finishing a small project is better than starting a big project
and not finishing it, and also that you can't learn to code by watching videos or reading
you need to practice, my sort of recommendation for if you want to learn to program, or
if you want to pick up a new language, if you want to pick up a new language, you probably
already know this, but one of the best ways to start is with a toy project, so you start
about, you start thinking about a small problem you have that you want to solve, and then
just start, it's not going to be right at first or good at first, but if you just start working
on it and banging on it, eventually you'll get something that works, and then you can
redo it, and you can do it over and over again, every time you do it, you can think, well,
I could have done that a little better so you can go back and refactor bits and pieces
here and there and make it a little better, and then if you decide to take a project you've
done and redo it, you can, you know, you're practicing, you're learning to code, you're
right in better code, but it's also an opportunity to learn some new technologies, so
if there's a Python library or a different language or something you want to use, that's
a good opportunity to incorporate these new pieces of tech in a sort of a rewrite of
a project that you've already done.
So I'm going to tell you about a toy project of mine, it's definitely something I've
done and re-done a few different times, so I listened to a lot of podcasts, just like you
probably listened to a lot of podcasts, and I've been listening to podcasts for what
I think is a long-term primarily Linux and tech-related podcast, and I was sort of,
I would think about sometimes I would be reminded of a podcast episode and I would go back
and that the podcast was gone or the website was gone or the podcast, you know, they
stopped making episodes or on and on, but I'm sure you're familiar with the term
podvated and it's just where a podcast just ends, sometimes it's announced, typically
when you say a podcast podvated that's usually, that was an, an, an out sort of unintentional
ending of a podcast, but I thought it would be fun to look at the podcast that had podvated
and just sort of see if I could track the release cadence of a bunch of different podcast,
you know, the podcast that I was interested in, and then rate how if they were podvated,
so in, in my head, I can, I can decide, okay, if it's, you know, if they're podcast
that usually releases weekly and they've been missed one or two weeks, you know, they're
sort of maybe they're getting in a little danger or something like that, that's the sort
of things that we're talking about, and one of my original ideas was to be able to use
them to research API to, um, find the podcast or the podcast, um, podcast that I would
be interested in. So you use Google, using Google APIs or something like that, search the
web for bar access fees. So my plan to search the web didn't really pay now. It was harder
to do than I thought it would. So then my second idea was to, um, look at specific networks
of podcast. Um, so you know, there will be a group of podcasts, we'll sort of join together
into a network and I'm, I'm sure you get your all thinking of examples. Um, so my, my
set sort of second idea was, okay, use, uh, Python to scrape the web pages of these networks,
and then, um, load all those, um, RSS feeds into the database and then I can use, um, use
the feeds that I found by scraping these network pages to, um, determine if the show
had, had podpated or not. Um, so I was using, uh, beautiful soup, uh, which is a Python library
for specifically for parsing, uh, HTML documents. And it makes it really easy. You can sort
of, uh, treat an HTML document. I'm kind of like a database and just kind of step through, uh,
the different tags and stuff they're in there. I'm from, when I, once I got the data, using, uh,
beautiful soup, uh, once I gathered all that data, um, I built a web page, just sort of
listing out all the different podcasts, and then, you know, Bob, my own determination if they
had podpated or not. And then, um, would use, um, the ginger templating engine and, uh, bootstraps
CSS, and then based on some logic that I'd written in Python, you know, if it, if I said it,
if I determined that it hadn't podfated and everything was fun, it would be green. And,
you know, I had different rules for, um, you know, it feels a weekly podcast or a person's
if it was a monthly podcast or how long it had been, is it getting close? Uh, it could be,
maybe I would color it as yellow. And then finally, if it hadn't, at least an episode
in six months or if it was a weekly podcast and it missed like five or six episodes or something
like that, then I could say it had podfated and then I would use the bootstraps colors to, um,
display that podcast as red. So there were, uh, there were some problems with that method.
One was, since I was using, um, beautiful soup and I'll scrape in each individual network.
Um, every network had, uh, different page layouts. Oh, I had to use, I had to write a custom script
for every podcast network. Uh, and that, that works fine until it doesn't. So, you know,
um, I can't control what some podcast network does with their webpage. So, any time it's a, you
know, move something around or edit an advertisement or tweak their menus a little bit,
whatever, it would break the script. And sometimes it was easy to fix. Sometimes it was hard to
fix. Sometimes I couldn't get it fixed. And sometimes I'd have a podcast network that I was
watching for changes, uh, and they would change your website so much that I would just give up and
and stop monitoring them. Um, another problem was it took a really long time to run. And then
like I said, it was too much, it was way too much trouble just trying to keep up with, you know,
a few different podcasts network all, um, they're, what page modifications. That, that just became
so I decided I wanted to redo, um, project, um, and really basically I was just going to have to start
over. And since I was starting over, I thought to myself, well, you know, now it would be a,
a great time to, you know, simplify it, um, and also maybe learn some new tech. So, one of the things,
one of the processes I wanted to learn is test driven development. And I'm, I'm not, I'm not necessarily
advocating for our district, but I at least wanted to work a project using test driven development,
uh, and just a real quick overview of what that means is, um, you know, whenever you write code,
you can have a test to make sure that test that code is working correctly. So in test driven
development, before you and write the code, you write the test, um, and then, you know, you write your
test, you write your code, um, and you work on it until the test passes, and then you write another
test, and then you write the code for that, and you work on it until the test passes. And it,
um, it kind of forces you to, you have to write your code a little differently, and you have to think
about your code a little differently, um, and it's, um, you know, having code that's well tested,
you know, one, it probably makes you write better code, but it also, um, one of the primary reasons
I like doing it is I'm, I don't consider myself a very good developer. So when I, if I have a project
and I need to make some changes to it, I'm always scared to do it. I'm a little nervous about doing it,
but if I have a, because if I, you know, if I have something that's working, um, that's always
better than, uh, something that isn't working, you know, I mean, so having test, having a well-developed
test suite, sort of giving the confidence to make it much of changes, um, because I can always,
can make a small change, run the test, make another small change, and run the test. And as long
as my tests were working, then I knew I was going to be okay. So another big change I wanted to
make to the project is I just wanted to one script, right? I wanted to simple find things that
it won't, you know, one script for this network and one script for this network, trying to do it
to simple as possible. Um, then I also wanted to, something I hadn't done before is the embedded
audio players. I want to be sure that, you know, whenever I finally built the HTML that there
would be, um, you know, the ability to play, uh, podcasts from the webpage. So, uh, one thing I,
I decided to do was instead of, you know, scraping a specific podcast network, or trying to
search for the search for the internet for RSS feeds. I just said, hey, there's curated list out there
that already have where people have put together a list of, um, their Linux podcast or, uh, or whatever.
So, um, I said, I'm going to start with these lists. I'm just going to, um, instead of making a
custom scraper for that page, I'm just going to use, I'm still going to use a beautiful suit,
but all I'm going to do is find all the links on there. And then, um, I'm going to take
all just, every link I can find and test it is an RSS feed or not. Uh, then if it is, uh,
throw that into the database to use later. So, instead of having a complex, uh, web scraper,
I just throw everything I can at a RSS feed barzer. If it could parse it, um, great. And if it
can't just throw it away, it's garbage. So, the, um, three places that I'm currently getting podcast from
are, um, the Ubuntu Wicking. It's got a podcast page, Ubuntu Wicking has a podcast page, um,
the Linux link.net is got a page, just a podcast, and then sort of a friend of HDR, uh, free,
freeculturepodcast.org. Um, so those are three places for now. I'm getting podcast. Now, I'm
I'm open to ideas about, um, where to get more podcast, but again, I'm trying to avoid, um,
like automated lists. So, you know, I'm specifically avoiding things like, um, RSS, uh,
podcast search sites or, um, Apple podcast search or something like that. Um, but, uh,
on the other hand, if you know of a web page that lists is just a list of, or contains a list of
Linux or tech related or hack-related podcast, I would definitely be interested in, um,
finding that and adding it to, um, adding it to this project. You know, uh, on the side note, I know the,
HPR show notes is probably a really good place to look. So it's sort of in the back of my mind as
a to-do item is to figure out a way to look through all the show notes, because there's show notes
out there that are that contain links to, uh, listeners and contributors, uh, favorite podcast.
So I think that's a really good idea. Just to something I haven't got.
So after we find all the RSS feeds, we test them, um, and to see, to see if they're actual,
actually an RSS feed and not just the link to something, uh, random, um, then I've got a script
called feed info, uh, that loops through all the feeds, um, it uses the feed parser, Python library,
um, and the feed parser Python library just kind of, it's kind of like beautiful suit, but specifically
for RSS feeds where you can sort of, uh, programmically, um, look through an RSS feed and pull things
out like titles and enclosures and things like that. So I'll use that library after, after I find
the RSS feeds, uh, I'll use the feed parser library to pull out the title and so I'll take
all that information and I'll build a web page and I'll still do a little bit of rating, um, whether
or not, I think the podcast has podfated and I'll give it a color, uh, regular green, um, the logic
isn't as complicated as it is. It was the first time, um, but there's a, uh, a do keep a copy of the
web page up on the public internet, uh, I'll have a link to it in the show notes. It's uh, podfated.norst.xlanzi,
um, it's currently running on a free tier of one of the big cloud providers. So as um, um,
I can't guarantee it's going to be there forever, but um, as long as the free tier is for you,
it will definitely be there. So I'm definitely welcome to feedback. Um, I'll have a link to the
GitLab page where I host a code for this. I'll have a, uh, that link in the show notes. I've got
a couple of bugs. Um, I'll be working on where sometimes, sometimes I will, um, unfortunately,
I'll have to exclude some podcast links, uh, because there's, there's links that look like
valid RSS feeds that it turns out they're just ads. Um, but sometimes those excluded podcast will
still show up on the, on the page. And there's other little bugs here and there in the HTML.
And like I said earlier, um, I'm, I'm looking for more places to pull podcasts from. You know,
I don't necessarily want a podcast search site and um, um, you know, I'm, I'll definitely look
into using um, our, um, HBR show notes. But if you don't have any other, uh, pages that just list out,
podcasts, uh, especially if they're Linux or hacker related, um, but that's something I'd like to
include if you can know of any. Uh, that HBR submissions page says it doesn't allow in my
HTML. I thought it would be nice if I could, um, since it's just HTML if I could put a little
code snippet of HTML into the show notes and you can see what it's like. But instead, I'll just grab
a screen shot. Uh, I think I got to work just as well. Well, that's it short, uh, episode on why I think
toy projects are important. And an example of a toy project I worked on, uh, I think it's a good
idea for um, if you're, if you want to contribute an episode, but you're not sure what, um, think about
a project like this that you can maybe what kind of walk us through, uh, what it is, how it works.
Um, yeah, I'm glad to hear about it. Um, I know, do you guys next time?
Our guide and our synch.net. On the side of the stages, they show us released on our creative
comments, attribution, for going to an international license.