Automate Raspberry Pi Downloads: A Step-by-Step Guide

Automating Your Raspberry Pi Download Server

Previously, we demonstrated the process of configuring a Raspberry Pi as a consistently-on, energy-efficient downloading system. This article builds upon that foundation, detailing how to implement automation for a nearly fully autonomous operation.

Leveraging Automation Tools

The key to a hands-off download server lies in utilizing appropriate automation tools. These tools minimize the need for manual intervention, ensuring continuous operation.

Several options are available, each offering varying levels of complexity and functionality. We will focus on methods that are both effective and relatively straightforward to implement.

Scheduling Downloads with Cron

Cron is a time-based job scheduler in Unix-like operating systems, including the Raspberry Pi’s operating system. It allows you to schedule commands or scripts to run automatically at specific times or intervals.

For example, you can schedule a script to initiate a download every day at midnight. This ensures that your Raspberry Pi consistently checks for and retrieves new content.

Automating File Management

Beyond initiating downloads, automation can also handle file management tasks. This includes organizing downloaded files, deleting old content, and transferring data to other storage locations.

Scripts can be written to automatically move files to designated folders based on their type or source. Regular cleanup routines can be scheduled to prevent storage from becoming full.

Utilizing Download Managers

Employing a robust download manager can significantly enhance automation. These tools often provide features like pausing and resuming downloads, scheduling, and bandwidth control.

Popular options include wget and aria2, both of which are command-line based and well-suited for automated operation on a Raspberry Pi.

Monitoring System Health

To ensure long-term reliability, it’s crucial to monitor the health of your automated system. This includes checking disk space, CPU usage, and network connectivity.

Automated alerts can be configured to notify you of any issues, allowing for prompt intervention if necessary. Regular system logs should also be reviewed for potential problems.

Benefits of a Fully Automated System

Implementing these automation techniques offers several advantages. A fully automated Raspberry Pi download server requires minimal user interaction, freeing up your time and resources.

It also ensures consistent operation, even when you are unavailable. This is particularly valuable for tasks like backing up important data or downloading large files overnight.

The Benefits of Automation

Assuming you’ve been following our Raspberry Pi project focused on downloading content, you now possess a fully functional BitTorrent and Usenet system. This is a significant achievement, however, it isn’t yet operating with complete autonomy.

Regular interaction is still required to direct the system towards new files and select desired episodes of television programs.

Upon completion of this guide on automation – the concluding part of our Raspberry Pi download box tutorial – you will have a device capable of fully automated operation.

Prerequisites

The system will be able to locate files online based on your specifications, eliminating the need for constant manual input.

Should you be joining us mid-project, we strongly recommend reviewing the preceding articles: How to Turn a Raspberry Pi into an Always-On BitTorrent Box and How to Turn a Raspberry Pi into an Always-On Usenet Machine.

What is Required for Automation?

As with the preceding guides, this tutorial presumes you possess a Raspberry Pi device running Raspbian. You should be capable of accessing the device either directly through a connected monitor and keyboard or remotely utilizing SSH and VNC. Furthermore, a USB drive (or multiple drives) must be connected to the Pi.

Prerequisites

It is also assumed that you have already completed at least one, if not both, of our tutorials detailing the transformation of a Raspberry Pi into a BitTorrent and/or Usenet download system. Should you require foundational knowledge in these areas, we highly recommend reviewing the following guides in the sequence presented:

• The HTG Guide to Getting Started with Raspberry Pi
• How to Configure Your Raspberry Pi for Remote Shell, Desktop, and File Transfer
• How to Turn a Raspberry Pi into a Low-Power Network Storage Device
• How to Turn a Raspberry Pi into an Always-On BitTorrent Box
• How to Turn a Raspberry Pi into an Always-On Usenet Machine

The subsequent sections of this tutorial, along with the tools discussed, will be ineffective if you haven't first established access to BitTorrent/Usenet downloading capabilities on your Raspberry Pi.

Having reached that stage, you can proceed to automate your entire setup for completely hands-off downloading.

Tutorial Structure

Each segment of this tutorial operates independently. If your intention is to automate the acquisition of television shows, but not movies, you are welcome to navigate directly to the pertinent section and disregard the remainder.

A note regarding the installation of all automation tools detailed in this guide: we discovered that installing them all on a Raspberry Pi does not unduly strain the system. However, each automation system can be resource-intensive during initial setup.

Specifically, the initial installation often involves significant processing, such as locating and downloading past episodes of desired television programs. Therefore, we advise allowing each installation sufficient processing time to utilize system resources, including SABnzbd and the connected USB HDD, without concurrent demands.

Once the initial catch-up process is complete, performance should stabilize. Simultaneously launching all three automation applications at full capacity during the initial phase can be challenging given the Raspberry Pi’s limited resources.

Setting Up Essential Software Components

To effectively utilize the applications detailed in this guide – namely SickBeard, CouchPotato, and Headphones – a preliminary step involves installing several necessary dependencies. This process excludes components already present within the standard Rasbian operating system.

Initially, it’s crucial to update and upgrade the apt-get package installer. Users who recently completed our prior Raspberry Pi tutorials can bypass this step, as a recent upgrade was likely performed.

Execute the following commands within the terminal:

sudo apt-get update

sudo apt-get upgrade

Following the update and upgrade, the installation of specific dependencies can commence. All three applications rely on Git, a distributed version control system. As Git isn't included by default in Rasbian, its installation is required.

sudo apt-get install git-core

Confirm the installation by typing "git --version" at the command prompt. A version number should be displayed, verifying successful installation. If no version is shown, re-run the installation command.

Beyond Git, SickBeard specifically requires Cheetah, a Python templating engine. If you previously installed SABnzbd following our guide on creating a Usenet machine, Cheetah is already present on your system. However, if you only followed the BitTorrent tutorial, installation is necessary.

sudo apt-get install python-cheetah

With Git and Python-Cheetah (for SickBeard users) installed, the setup process is ready to proceed.

Installing and Configuring SickBeard on Raspbian

SickBeard functions as a television management solution, facilitating downloads via Usenet and BitTorrent. While its Usenet capabilities are more developed, both methods are operational. Essentially, you specify the TV shows you desire, and SickBeard operates like a network-connected TiVo, acquiring those programs for you – either completing a backlog or obtaining new episodes as they become available.

Installing SickBeard

To begin the installation process, access the terminal and execute the following command:

git clone git://github.com/midgetspy/Sick-Beard.git

Upon completion of the installation, navigate into the SickBeard directory and initiate the program for the first time:

cd Sick-Beard

python SickBeard.py

The script may appear to pause momentarily before displaying a stream of text, eventually settling into an idle state. At this point, open a web browser and direct it to the following address to access SickBeard:

http://[Your PI's IP Address]:8081/home/

Configuring SickBeard

Unlike some tools explored in prior tutorials, SickBeard lacks a configuration wizard. We will guide you through the essential setup steps, organized by section within SickBeard’s Configuration interface.

In the Config -> General section:

• Uncheck Launch Browser: As you'll be accessing the system remotely, this option is unnecessary and consumes resources.
• Under Web Interface: You have the option to establish a username and password for SickBeard, and/or modify the port number.

In the Config -> Search Settings section:

This is where you configure SickBeard to interact with your Usenet and/or BitTorrent client. SickBeard performs optimally with Usenet, and that approach is recommended.

Adjust Usenet Retention: The default is 500 days. If your Usenet provider offers a longer retention period – typically 1100+ days for premium accounts – verify with your provider and update the setting accordingly.

Under NZB Search, ensure "Search NZBs" is enabled and SABnzbd is selected. Input the URL, username, and password (if configured), and the API key from your SABnzbd installation. Confirm successful communication by clicking "Test SABnzbd."

Note: If you prefer BitTorrent, check "Search Torrents" and specify the watch folder for your Torrent client.

In the Config -> Search Providers section:

Here, you select the indexes to search and enter API keys for private or paid search indexes.

Under Provider Priorities, enable providers by checking their boxes and then drag and drop them to establish a priority order. Womble's Index is enabled by default. You can also enable Sick Beard Index and nzbX, but the remaining options require accounts and API keys – obtained through the respective websites’ signup processes.

Note: If torrent searching was enabled previously, you will also find TV torrent trackers listed here.

In the Config -> Post Processing section:

In the initial "Post-Processing" section, leave the "TV Download Dir" field blank. A helper script for SABnzbd will manage this function.

Uncheck "Keep Original Files"; storage space on the Raspberry Pi is limited, making duplicate files impractical.

Check "Move Associated Files" and "Rename Episodes" to ensure SickBeard transfers any accompanying files and renames episodes according to your chosen convention in the "Naming" section.

Finally, configure SickBeard to download metadata in a format compatible with your media center. Since we utilize XBMC, we selected XBMC for "Metadata Type" and enabled the desired metadata file types (such as folder images and fanart).

In the Config -> Notifications section:

This section is optional. You can configure notifications for media clients like XBMC and notification services like Growl. XBMC notifications can be beneficial for a fully automated system, allowing SickBeard to trigger library updates after downloads.

Linking SABnzbd and SickBeard

A properly configured SickBeard and SABnzbd operate synergistically. SickBeard locates shows, directs NZB files to SABnzbd, and SABnzbd then utilizes a SickBeard helper script to post-process and organize the downloads.

To establish this link, copy the helper script and modify its accompanying configuration file. Enter the following command in the terminal:

cd /home/pi/Sick-Beard/autoProcessTV

Within this directory, you'll find the two necessary files: autoProcessTV.py (the script) and autoProcessTV.cfg.sample (the sample configuration). Open the configuration file for editing:

nano autoProcessTV.cfg.sample

In the nano editor, verify that the configuration matches your installation. The default configuration points to localhost on port 8081, without a username or password. Adjust the file if you’ve altered these settings. Press CTRL+X to exit when finished, and rename the file to autoProcessTV.cfg (removing the ".sample" extension).

There are two methods for granting SABnzbd access to the scripts. The simpler approach is to change the post-processing script directory in SABnzbd’s Config -> Folder section to /home/pi/Sick-Beard/autoProcessTV. This works if SickBeard scripts are your only post-processing tools.

Alternatively, to centralize all SABnzbd scripts, use the following command to copy them to your SABnzbd scripts folder (established in the Usenet guide):

cp autoProcessTV.cfg autoProcessTV.py sabToSickBeard.py /media/USBHDD1/shares/SABnzbd/scripts

Regardless of the method, tell SABnzbd to use the scripts. In SABnzbd, navigate to Config -> Categories and create a new category named "tv". In the script section, select sabToSickBeard.py as the post-processing script. Optionally, define a folder for temporary TV downloads (e.g., SickBeardTV). Save your changes by clicking Add.

Adding Shows to SickBeard

With SickBeard configured, the next step is to add the television shows you want to automate.

First, create a directory for SickBeard to store the shows. (Skip this if you already have a TV show directory on your external hard drive):

mkdir /media/USBHDD1/shares/TV/

From the SickBeard interface, click "Add Shows." You can either create a new entry for a new show or import existing downloaded shows. Even without an existing collection, start with "Add Existing Shows" to point SickBeard to your TV directory.

Click "Add Existing Shows," then click "New" under the "Manage Directories" tab. Select your TV folder in the file explorer, click "OK," then "Set as Default," and finally "Submit."

With the default TV directory set, navigate to "Add Shows" -> "Add New Show." Search for the show, select it, and proceed to verify the folder location and specify whether missing episodes should be marked as "Wanted" or "Skipped," along with the desired video quality. Click "Add" to complete the process. SickBeard will create the folder, download metadata, and begin searching for episodes.

Running SickBeard on Startup

To ensure SickBeard automatically launches upon boot, add a startup script.

Enter the following commands at the terminal:

sudo wget -O /etc/init.d/sickbeard/ https://www.howtogeek.com/wp-content/uploads/gg/up/sshot5161b529c109d.txt

sudo chmod 755 /etc/init.d/sickbeard

sudo update-rc.d sickbeard defaults

This completes the SickBeard configuration. You can now add shows, both current and those you’ve missed, and SickBeard will automatically download them for you.

Installing and Configuring CouchPotato on Raspbian

If you found the previous configuration process demanding, be encouraged. SickBeard represents the most complex tool in this series to set up. Let’s proceed with CouchPotato – the film counterpart to SickBeard’s automated television show downloading capabilities.

Installing CouchPotato

Given that all necessary dependencies have already been installed, we can directly proceed to obtain the source code from Git. Enter the following command in your terminal:

cd /home/pigit clone http://github.com/RuudBurger/CouchPotatoServer.gitpython CouchPotatoServer/CouchPotato.py

Upon executing the Python script, the cursor may simply remain idle. However, if a crash dump occurs, a minor modification is required to enable the web daemon and startup wizard. Apply this edit only if the script encounters an error. Use the following command in the terminal:

nano /home/pi/CouchPotatoServer/couchpotato/runner.py

Within the script, navigate to the specified section and incorporate the line highlighted in bold:

config = {'use_reloader': reloader,'port': tryInt(Env.setting('port', default = 5000)),'host': Env.setting('host', default = ''),'ssl_cert': Env.setting('ssl_cert', default = None),'ssl_key': Env.setting('ssl_key', default = None),<strong>'host': Env.setting('host', default = "0.0.0.0"),</strong>}

Save the changes and rerun the script. Once the script completes successfully, access the following address in your web browser to begin:

http://[Your PI's IP Address]:5050/wizard/

Similar to SickBeard, a guided setup process will be provided.

Configuring CouchPotato

The Welcome section can be skipped. There is no existing CouchPotato database to import, unless you are migrating an older CouchPotato installation to your new Raspberry Pi download box; in that case, follow the provided instructions and direct the wizard to your old data.db file.

In the General section, you can define a username and password, and optionally adjust the port number. Disable the "Launch Browser" option.