skip to main content - dyslexic font - mobile - text - print

Hacker Public Radio

Your ideas, projects, opinions - podcasted.

New episodes Monday through Friday.

Correspondent

Dave Morriss


Host ID: 225
website:
email: dave.morriss.nospam@nospam.gmail.com
episodes: 9

1473 - FOSDEM Discussion | 2014-03-26

I decided to attend FOSDEM 2014 this year. I had thought about going to last year's conference but didn't get organised enough to make it. When I mentioned my plans to my friend Tom, he thought he'd attend too, and we agreed to meet up there.

When we got back from the conference I wanted to record a conversation with Tom about our impressions of the event. We tried to do this four times before we finally managed it. We struggled through one recorder battery failure and two Mumble failures before we achieved success. This is the result of our conversation.

Apologies for the phone interference in the background, I hadn't realised the recorder (a Tascam DR-07) would pick them up.

Links:


1461 - FOSDEM Keysigning Event | 2014-03-10

I attended FOSDEM 2014 in Brussels, Belgium. During the conference there was a key signing event which I attended. These are my impressions of the process and the follow-up.

Detailed notes:

http://www.hackerpublicradio.org/eps/hpr1461/FOSDEM_Keysigning_Event.html


1362 - Fixing a bad RSS feed | 2013-10-22

There have been problems with the podcast feed for "mintCast", apparently as a result of a bug in Wordpress. The feed contains multiple "enclosure" tags containing the same audio over and over again. While the mintCast hosts are looking for a fix I would like to find a local work-around.

I have also encountered a problem with the "Pod Delusion Extra" feed which contains multiple enclosures in some episodes. Unlike the "mintCast" example I don't want to lose these enclosures but want to find a way of repackaging them into individual episodes.

These problems affect some podcatchers, the modified Bashpodder I use being amongst them. To counteract this problem I have written two short Perl scripts to copy and clean each feed before submitting it to my podcatcher.

Detailed notes: http://www.hackerpublicradio.org/eps/hpr1362/Dealing_with_bad_RSS_feeds.html


1303 - A Music Pairing Under Unlikely Circumstances | 2013-07-31

Today Dave interviews Tim, his son, and Tim's friend John, who is visiting from the USA.

Tim and John met on the Internet in 2006 as collaborating composers of electronic music. They have become good friends over the years; Tim has visited John in the States, in 2011 where they met for the first time in real life, for John's wedding. This also marks the first time that John and his wife Caitlin have travelled overseas, which they did to visit Tim in the UK.

In the podcast we discuss how they met, how their different world views affected each other, and how their relationship quickly transcended music.

Here's a picture of Tim and John visiting Edinburgh Castle in July 2013:

Tim and John visiting Edinburgh Castle in July

Contrary to what was said in the podcast, Tim prepared a mix of the various compositions he and John have made. Links to some of the full tracks are available below.

Here are Tim's notes for the music mix:

http://www.hackerpublicradio.org/eps/hpr1303/Music_Notes.html

Links:

Some of Tim and John's work - https://soundcloud.com/heartshapedboxband/
Their latest collaboration - https://soundcloud.com/heartshapedboxband/accord


1291 - Parsing an ISO8601 formatted duration field with Perl | 2013-07-15

Ken recently asked Dave for help with a Perl regular expression for parsing ISO8601 time durations. As a consequence a Perl script was written, which is available at https://gitorious.org/hprmisc/hprmisc/blobs/master/parse_8601_duration.

In this show Ken and Dave discuss this script at some (considerable) length. Keen listeners might want to view the script as they listen. Detailed show notes describing how to put together a Perl regular expression are also available at URL.

Unfortunately some of the line numbers in the script referred to in the show are now incorrect since Dave could not stop himself updating it.

For detailed show notes on how Dave created the script see:


1286 - iCalendar Hacking | 2013-07-08

Having failed to make repeating reminders in his calendar for the HPR Community News shows on the Saturday before the first Monday of every month, Dave resorts to writing raw iCalendar rules. This also proves to be quite difficult and a Perl script is resorted to, also with mixed success.


1204 - My Magnatune Downloader | 2013-03-14

The Problem

I'm a fan of Magnatune (http://magnatune.com/) and have been buying music from them for 7 or 8 years. The Magnatune website itself is good for exploring and downloading, and interfaces for browsing and purchasing are available in a number of players on Linux. I have direct experience of:

  • Amarok: allows you to browse, purchase, examine artist information and album details.
  • Rhythmbox: the plugin, which used to allow browsing and purchasing, is currently unavailable, but is apparently due to return soon.
  • Gnome Music Player Client: (a front-end to the Music Player Daemon, mpd) offers a Magnatune browser plugin
  • Magnatune Web 2.0 player: a web-based tool which will browse, play and download Magnatune music.
  • Magnatune Android player: a fairly basic browser and player for Android 2.0 and up.

The Magnatune Web 2.0 player is the best of the bunch as far as I am concerned, particularly since it allows me to explore the music collection whilst listening to streamed music at the same time. However, none of these interfaces provide me with exactly what I want in terms of the download process, so I decided to write my own.

The Plan

I currently host my music on my HP Proliant microserver, share it across the home network, and play it with the Music Player Daemon (http://sourceforge.net/projects/musicpd/) on my desktop system. I normally keep the album cover image, artwork and related material in the same directory as the album itself, and I want to be able to save all files in their appropriate places automatically.

Magnatune provides an API which is documented at http://download.magnatune.com/info/api, though this information is only available to members. Data is available in several formats: XML, SQlite and MySQL.

Design

I didn't want to launch into building a full-blown application, especially since I only needed a downloader, so I decided to create a collection of scripts.

I decided to use the XML data organised by album. This is updated on about a weekly or two weekly basis, and there is a signalling mechanism through a downloadable file containing a checksum. When this changes the large data file has changed and can be downloaded. At the time of writing I simply run this by hand when I receive an email alert from Magnatune.

Magnatune uses an unique key made from the artist and album names which it refers to as the SKU (Stock Keeping Unit) or albumsku. They use this as an URL component and in XML tags. I use it to identify the stuff I download and to keep a simple inventory.

I decided to write some basic scripts:

  • To download the catalogue

  • To extract information from the catalogue

  • To download an album

  • To unpack the downloaded items into the target directory

I wanted to learn more about manipulating XML data, so I decided to use XSL, the Extensible Stylesheet Language. This lets you define stylsheets for XML data, including ways of identifying XML components with XPath and of transforming XML with XSLT.

I have included a number of links to the resources I used in the shownotes.

Repository

I have placed all of the scripts, their associated files, and HTML and PDF README files (extended shownotes) in a gitorious repository. This can be browsed at https://gitorious.org/magnatune-downloader or, if a copy is required it can be obtained with the command:

  git clone https://git.gitorious.org/magnatune-downloader/magnatune-downloader.git

This makes a local git repository containing a copy of all of the files in the current directory.

Scripts

  • update_albums: a Bash script to download a new version of the album catalogue, as a bzipped XML file, if it is different from the current version. It generates a summary of the catalogue for simple searching using XSLT.

  • report_albumsku: a Bash script to take a SKU code and look up the album details in the XML file.

  • get_album: a Bash script to download an album, cover images and artwork. It takes the SKU as an argument and uses it to make an URL for an XML file which points at all of the components, and this is downloaded (with authentication). The script then parses this file to get the necessary URLs for downloading. I only use the OGG format but it could easily collect any or all formats available from Magnatune. The script records the fact that this particular SKU code has been downloaded so that it isn't collected again in error. All downloaded files are given names beginning with the SKU code and are stored for the installation phase.

  • install_download: a Perl script which unpacks the downloaded zip file to its final destination then adds the cover images and artwork to the same place. I used Perl because it allowed me to query the zip file to determine the name of the directory that was going to be created.

Further Developments

I have added further scripts to this system since I created it. I have one that synchronises the music files from my workstation to the server, and two that give me a simple wish-list or queue functionality.

Since I have a 200GB download limit per month on my broadband contract I try not to download music too often and avoid contention with the rest of the family. My queueing system is used to keep a list of stuff I'd like to buy from Magnatune, and I simply feed the top element from the queue into my download script every week or so.

In the future I expect to be refining all of these scripts and making them less vulnerable to errors. For example, I have found a few cases where Magnatune's XML is not valid and this causes the xsltproc tool to fail. I'd like to be able to recover from such errors more elegantly than I'm doing now.

At some point I may well be tempted to consolidate all of the current functions into a single Perl script.

Disclaimer

I have no connections to Magnatune other than being a contented customer.

Links


1148 - Development Discussion | 2012-12-26

I am trying to write a script which will implement the scheduling rules for HPR. I spoke to Ken Fallon about this, and where it would fit in the overall design of the HPR system, when we met up at OggCamp 2012 in August, but we didn't manage to resolve very much. So, recently Ken and I began a discussion over Mumble to try and make progress. A few minutes in we decided to record our discussion for posterity, and this is the result.

The notes which I had sent Ken before our Mumble session are available in PDF format.

http://hackerpublicradio.org/eps/hpr1148_Design_Notes_20121108.html
http://hackerpublicradio.org/eps/hpr1148_Design_Notes_20121108.pdf


1091 - Useful Vim Plugins | 2012-10-07

Useful Vim Plugins

Dave Morriss
Last Updated: 2012-10-06

I started learning vi a long time ago when I first encountered Unix systems. In those days you could usually count on a system having vi (or at a pinch, when the system would only boot into single user mode, ed). Learning Emacs did't seem like a good choice since it wasn't usually available on the systems I was administering.

I dont remember when I changed to Vim, but for many years I have used it (actually gVim) as an IDE (Integrated Development Environment), particularly for writing Bash and Perl scripts.

In these notes I have listed some of the plugins I use to enhance Vim and gVim's functionality. If you have never made enhancements to this editor, here's a site with a very good tutorial on how to install plugins http://www.installationwiki.com/Installing_Vim_Scripts.


NERDTree

This plugin provides a file browser within Vim/gVim. A particular directory is opened by typing :NERDTree somedir. A tree is displayed in a window which can be scrolled and traversed to find files to edit.

In case you're interested, the colour theme I use in gVim is "Murphy".

Type :h NERD_tree for the large and comprehensive help file.

I prefer to use NERDTree in gVim and not in Vim. To achieve this I have the following in my ~/.vimrc

  " NERDTree settings (only in GUI mode)
  if has("gui_running")
      let NERDTreeRoot = '~'
      let NERDTreeIgnore = ['\~$', '\.swp$']
      let NERDTreeShowHidden = 1
      let NERDTreeShowBookmarks = 1
      let NERDTreeChDirMode = 2
  else
      let loaded_nerd_tree = 1
  endif

Bash-support

This one provides a Bash scripting IDE for Vim and gVim. I find it most usable in gVim where it offers a comprehensive set of menus which you can see in the example.

From the Help text:

  It is written to considerably speed up writing code in a consistent style.
  This is done by inserting complete statements, comments, idioms, and code
  snippets. Syntax checking, running a script, starting a debugger can be done
  with a keystroke. There are many additional hints and options which can
  improve speed and comfort when writing shell scripts.

The plugin can be used from the menu or by typing short-cut sequences such as \ct which inserts the current date and time at the cursor position. It's an amazingly detailed package that's well worth dedicating the time to learn if you often write Bash scripts.


Perl-support

This plugin provides a Perl scripting IDE for Vim and gVim. As with the Bash plugin from the same author I find it most useful from gVim where the menus can be used to perform a wide range of actions.

Again, you can use it to insert statements, boilerplate text and comments, in a similar way to the Bash plugin. You can also get regular expression help, check, run and debug the script

Two of the functions I find particularly useful are perltidy and perlcritic. Perltidy will reformat your Perl script, and Perlcritic will critique it according to the rules based on Damian Conway's book Perl Best Practices.

Note the lower window showing the various problems detected by perlcritic.

If you are a C or C++ programmer Fritz Mehner also provides a plugin for these languages here.


manpageview

This is a man page viewer for use within Vim. Use :Man topic or press K on a keyword. Can view perl, php and python help.

This is a great way to view man pages, with the ability to search and cut and paste the contents.


Surround

The plugin provides mappings to easily delete, change and add "surroundings" such as quotes, braces and parentheses.

The following extract from the Help text should make this clearer:

  Consider the following examples.  An asterisk (*) is used to denote the cursor
  position.
  
    Old text                  Command     New text
    "Hello *world!"           ds"         Hello world!
    [123+4*56]/2              cs])        (123+456)/2
    "Look ma, I'm *HTML!"     cs"<q>      <q>Look ma, I'm HTML!</q>
    if *x>3 {                 ysW(        if ( x>3 ) {
    my $str = *whee!;         vlllls'     my $str = 'whee!';

Fugitive

A very powerful interface to git.

The image shows the result of the :Gstatus command which is equivalent to typing git status on the command line.


Taglist

This plugin provides source code browsing by listing tags (functions, classes, structures, variables, etc.). The list is in a window in Vim or in a menu in gVim.

The plugin makes us of the Exuberant Ctags utility to index tags in a source file. I believe that most Linux distributions ship with this utility, but if not it seems to be easily available in repositories.

The image shows gVim with the tag list in a window. I find the menu in gVim more useful myself.


Supertab

Provides insert completion functionality using the Tab key.

The example shows a Perl script where the choice being made from the list offered by Supertab is uptodate. Note also that manpageview is being used to view the manpage for the CPAN module.

It has taken me a while to learn not to use the Tab key to enter TAB characters but to use CTRL-Tab instead. However, I have decided that the advantages of redefining the Tab key in this way outweigh the disadvantages. You might not agree!


Become a Correspondent