<H3>The Problem</H3>

<P>
I'm a fan of <I>Magnatune</I> (<A HREF="https://magnatune.com/">https://magnatune.com/</A>) and have been buying music
from them for 7 or 8 years. The Magnatune website itself is good for exploring
and downloading, and interfaces for browsing and purchasing are available in
a number of players on Linux. I have direct experience of:
</P>

<UL>
<LI>Amarok: allows you to browse, purchase, examine artist information and album
  details.
<LI>Rhythmbox: the plugin, which used to allow browsing and purchasing, is
  currently unavailable, but is apparently due to return soon.
<LI>Gnome Music Player Client: (a front-end to the Music Player Daemon, mpd)
  offers a Magnatune browser plugin
<LI>Magnatune Web 2.0 player: a web-based tool which will browse, play and
  download Magnatune music.
<LI>Magnatune Android player: a fairly basic browser and player for Android 2.0
  and up.
</UL>

<P>
The Magnatune Web 2.0 player is the best of the bunch as far as I am
concerned, particularly since it allows me to explore the music collection
whilst listening to streamed music at the same time. However, none of these
interfaces provide me with exactly what I want in terms of the download
process, so I decided to write my own.
</P>

<H3>The Plan</H3>

<P>
I currently host my music on my HP Proliant microserver, share it across the
home network, and play it with the Music Player Daemon
(<A HREF="https://sourceforge.net/projects/musicpd/">https://sourceforge.net/projects/musicpd/</A>) on my desktop system. I normally
keep the album cover image, artwork and related material in the same directory
as the album itself, and I want to be able to save all files in their
appropriate places automatically.
</P>
<P>
Magnatune provides an API which is documented at
<A HREF="https://download.magnatune.com/info/api">https://download.magnatune.com/info/api</A>, though this information is only available
to members. Data is available in several formats: XML, SQlite and MySQL.
</P>

<H3>Design</H3>

<P>
I didn't want to launch into building a full-blown application, especially
since I only needed a downloader, so I decided to create a collection of
Bash and Perl scripts.
</P>
<P>
I decided to use the XML data organised by album. This is updated on about
a weekly or two weekly basis, and there is a signalling mechanism through
a downloadable file containing a checksum. When this changes the large data
file has changed and can be downloaded. At the time of writing I simply run
this by hand when I receive an email alert from Magnatune.
</P>
<P>
Magnatune uses an unique key made from the artist and album names which it
refers to as the <I>SKU</I> (Stock Keeping Unit) or <I>albumsku</I>. They use this
as an URL component and in XML tags. I use it to identify the stuff I download
and to keep a simple inventory.
</P>
<P>
I decided to write some basic scripts:
</P>

<UL>
<LI>To download the catalogue
<LI>To extract information from the catalogue
<LI>To download an album
<LI>To unpack the downloaded items into the target directory
</UL>

<P>
I wanted to learn more about manipulating XML data, so I decided to use
<I>XSL</I>, the E<B>x</B>tensible <B>S</B>tylesheet <B>L</B>anguage. This lets you define
stylsheets for XML data, including ways of identifying XML components with
<I>XPath</I> and of transforming XML with <I>XSLT</I>.
</P>
<P>
I have included a number of links to the resources I used in the shownotes.
</P>

<H3>Repository</H3>

<P>
I have placed all of the scripts, their associated files, and HTML and PDF
<I>README</I> files (extended shownotes) in a GitLab repository. This can be
browsed at <A HREF="https://gitlab.com/davmo/magnatune-downloader">https://gitlab.com/davmo/magnatune-downloader</A>
or, if a copy is required it can be obtained with the command:
</P>

<PRE>
  git clone https://gitlab.com/davmo/magnatune-downloader.git
</PRE>

<P>
This makes a local <I>git</I> repository containing a copy of all of the files in
the current directory.
</P>

<p><b>Note:</b> The code was originally hosted on Gitorious
(https://gitorious.org/magnatune-downloader), but with the demise of this
service it was moved to GitHub and the details above updated. Then since the
Microsoft takeover of GitHub, it has been moved to GitLab and the details
updated as needed.</p>

<H3>Scripts</H3>

<UL>
<LI><I>update_albums</I>: a Bash script to download a new version of the album
  catalogue, as a bzipped XML file, if it is different from the current
  version.  It generates a summary of the catalogue for simple searching using
  XSLT.
<LI><I>report_albumsku</I>: a Bash script to take a <I>SKU</I> code and look up the
  album details in the XML file.
<LI><I>get_album</I>: a Bash script to download an album, cover images and artwork.
  It takes the <I>SKU</I> as an argument and uses it to make an URL for an XML
  file which points at all of the components, and this is downloaded (with
  authentication). The script then parses this file to get the necessary URLs
  for downloading. I only use the OGG format but it could easily collect any
  or all formats available from Magnatune. The script records the fact that
  this particular <I>SKU</I> code has been downloaded so that it isn't
  collected again in error. All downloaded files are given names beginning
  with the <I>SKU</I> code and are stored for the installation phase.
<LI><I>install_download</I>: a Perl script which unpacks the downloaded zip file to
  its final destination then adds the cover images and artwork to the same
  place. I used Perl because it allowed me to query the zip file to determine
  the name of the directory that was going to be created.
</UL>

<H3>Further Developments</H3>

<P>
I have added further scripts to this system since I created it. I have one that
synchronises the music files from my workstation to the server, and two that
give me a simple wish-list or queue functionality.
</P>
<P>
Since I have a 200GB download limit per month on my broadband contract I try
not to download music too often and avoid contention with the rest of the
family. My queueing system is used to keep a list of stuff I'd like to buy
from Magnatune, and I simply feed the top element from the queue into my
download script every week or so.
</P>
<P>
In the future I expect to be refining all of these scripts and making them
less vulnerable to errors. For example, I have found a few cases where
Magnatune's XML is not valid and this causes the <I>xsltproc</I> tool to fail.
I'd like to be able to recover from such errors more elegantly than I'm doing
now.
</P>
<P>
At some point I may well be tempted to consolidate all of the current
functions into a single Perl script.
</P>

<H3>Disclaimer</H3>

<P>
I have no connections to Magnatune other than being a contented customer.
</P>

<H3>Links</H3>

<UL>
<LI>Magnatune: <A HREF="https://magnatune.com/">https://magnatune.com/</A>
<LI>Magnatune API: <A HREF="https://magnatune.com/info/api">https://magnatune.com/info/api</A>
<LI>AsciiDoc: <A HREF="https://www.methods.co.nz/asciidoc/index.html">https://www.methods.co.nz/asciidoc/index.html</A>
<LI>XSLT information:
  <UL>
  <LI>XSLT Tutorial: <A HREF="https://linux.dd.com.au/wiki/XSLT_Tutorial">https://linux.dd.com.au/wiki/XSLT_Tutorial</A>
  <LI>XSL Transformations: <A HREF="https://www.cafeconleche.org/books/bible3/chapters/ch15.html">https://www.cafeconleche.org/books/bible3/chapters/ch15.html</A>
  <LI>XSLT tutorial: <A HREF="https://www.w3schools.com/xsl/default.asp">https://www.w3schools.com/xsl/default.asp</A>
  </UL>
</UL>
