How to install minidlna on a Raspberry Pi

In this posting we are going to turn a Raspberry Pi into a media streaming device, using minidlna.

Preparation of the Raspberry Pi

  • Follow the instructions at How to install the Raspbian OS on a Raspberry Pi.
  • Change the hostname of your Raspberry Pi to something more meaningful by logging on to the device as user “pi” and issuing the command:

    sudo raspi-config

  • Select “Advanced Options” and set the following:
    • Hostname = pi-media (or whatever you want to call it).
  • Select Finish
  • Reboot the Raspberry Pi

Install minidlna

  • First, make sure that the package indexes are up to date with:

    sudo apt-get update

  • Installing the minidlna package is as easy as entering the commands:


    sudo apt-get install minidlna
    sudo chown minidlna:minidlna /var/lib/minidlna

  • On completion, create a directory in which the media you wish to stream will be stored. For the moment we will save some media to subdirectories under /home/pi on the Raspberry Pi’s SD card, but once we are happy it is working, we will mount an external hard disk and serve the media up from that. For now, type:


    mkdir -p "/home/pi/media/videos"
    mkdir -p "/home/pi/media/pictures"
    mkdir -p "/home/pi/media/music"

  • To ensure that the minidlna user has the privilege to read from the directories under the pi user, add minidlna to the pi user group with:

    sudo usermod -a -G pi minidlna

  • Now we will create the directories which will be used for storing the minidlna database and the log files:


    sudo mkdir /var/{cache,log}/minidlna
    sudo chown minidlna:minidlna /var/{cache,log}/minidlna

  • The next step is to make some changes to the minidlna configuration file. Open the file for editing with:


    sudo vi /etc/minidlna.conf

    …and make the following changes (changing the IP address to match your Raspberry Pi. If you’re not sure of the IP address, use the ifconfig command):
    media_dir=A,/home/pi/media/music
    media_dir=P,/home/pi/media/pictures
    media_dir=V,/home/pi/media/videos
    db_dir=/var/cache/minidlna
    log_dir=/var/log/minidlna
    listening_ip=192.168.XXX.XXX
    inotify=yes
    notify_interval=300

  • Open the sysctl.conf file for editing with:


    sudo vi /etc/sysctl.conf

    …and add the following to the bottom of the file:
    # minidlna server tweaks
    fs.inotify.max_user_watches = 100000
    # end of minidlna server tweaks

  • Rebuild the database and restart minidlna with:


    sudo service minidlna stop
    sudo service minidlna force-reload
    sudo service minidlna start

You can now add music, pictures and videos to the respective directories under /home/pi/media.

Advertisements

How to install Piwigo on a Raspberry Pi

In this posting, we will be installing the Piwigo (photo gallery), along with Nginx (lightweight web server), PHP and MySql (database package). At a later stage, we will move the Piwigo directory structure to an external hard disk and also install and configure Samba to allow the directory containing photographs to be shared with a Windows PC.

Preparation of the Raspberry Pi

  • Follow the instructions at How to install the Raspbian OS on a Raspberry Pi.
  • Change the hostname of your Raspberry Pi to something more meaningful by logging on to the device as user “pi” and issuing the command:

    sudo raspi-config

  • Select “Advanced Options” and set the following:
    • Hostname = pi-piwigo (or whatever you want to call it).
  • Select Finish
  • Reboot the Raspberry Pi

Install PHP, MySql, the Nginx web server, and other support packages

  • Create a user for the Nginx web server with the command:

    sudo groupadd www-data ; sudo usermod -a -G www-data www-data

  • Install the necessary packages with the command:


    sudo apt-get install nginx php5-fpm php5 php5-mysql php5-pgsql php5-imap php-pear php5-sqlite php5-ldap php5-gd php5-imagick php5-curl php-apc openssl ssl-cert php5-mcrypt php5-pspell php5-xmlrpc php5-xsl php5-cgi php-auth php-auth-sasl php-net-smtp mysql-server

  • Create your SSL certificates (which will last for 2 years) with the commands:


    sudo openssl req $@ -new -x509 -days 730 -nodes -out /etc/nginx/cert.pem -keyout /etc/nginx/cert.key
    sudo chmod 600 /etc/nginx/cert.pem ; sudo chmod 600 /etc/nginx/cert.key

Configure the Nginx web server

  • Open the configuration file for editing:

    sudo vi /etc/nginx/nginx.conf

  • Because the Raspberyy Pi only has 2 cores, change


    worker_processes 4;
    …to:
    worker_processes 2;

  • Change the top of the http section to:


    http {

    ##
    # Basic Settings
    ##

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    # keepalive_timeout 65;
    types_hash_max_size 2048;
    server_tokens off;

    client_header_timeout 10;
    client_body_timeout 10;
    keepalive_timeout 10 10;
    send_timeout 10;

  • Change the Gzip section from:


    ##
    # Gzip Settings
    ##

    gzip on;
    gzip_disable "msie6";

    # gzip_vary on;
    # gzip_proxied any;
    # gzip_comp_level 6;
    # gzip_buffers 16 8k;
    # gzip_http_version 1.1;
    # gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
    …to:
    ##
    # Gzip Settings
    ##

    gzip on;
    gzip_disable "msie6";

    gzip_min_length 1100;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_buffers 16 8k;
    gzip_http_version 1.1;
    gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

  • Now do the following:


    sudo vi /etc/php5/fpm/pool.d/www.conf
    …and uncomment the following two lines::
    listen.owner = www-data
    listen.group = www-data

  • Enter the following command:

    sudo vi /etc/nginx/sites-available/default

  • …and replace the entire content of the file with the code below (modified from http://doc.owncloud.org/server/5.0/admin_manual/installation/installation_others.html#nginx-configuration). Note: You’ll have to replace two occurrences of 192.168.XXX.XXX with the local IP address of your Raspberry Pi.


    server {
    listen 80;
    server_name 192.168.XXX.XXX;
    return 301 https://$server_name$request_uri; # enforce https
    }

    server {
    listen 443 ssl;
    server_name 192.168.XXX.XXX;

    ssl_certificate /etc/nginx/cert.pem;
    ssl_certificate_key /etc/nginx/cert.key;

    # Path to the root of your installation
    root /var/www/piwigo;

    client_max_body_size 1000M; # set max upload size
    fastcgi_buffers 64 4K;

    index index.php;
    error_page 403 /core/templates/403.php;
    error_page 404 /core/templates/404.php;

    location = /robots.txt {
    allow all;
    log_not_found off;
    access_log off;
    }

    location ~ ^/(data|config|\.ht|db_structure\.xml|README) {
    deny all;
    }

    location / {
    # The following 2 rules are only needed with webfinger
    rewrite ^/.well-known/host-meta /public.php?service=host-meta last;
    rewrite ^/.well-known/host-meta.json /public.php?service=host-meta-json last;

    rewrite ^(/core/doc/[^\/]+/)$ $1/index.html;

    try_files $uri $uri/ index.php;
    }

    location ~ ^(.+?\.php)(/.*)?$ {
    try_files $1 = 404;

    include fastcgi_params;
    fastcgi_param SCRIPT_FILENAME $document_root$1;
    fastcgi_param PATH_INFO $2;
    fastcgi_param HTTPS on;
    fastcgi_pass 127.0.0.1:9000;
    # Or use unix-socket with 'fastcgi_pass unix:/var/run/php5-fpm.sock;'
    }

    # Optional: set long EXPIRES header on static assets
    location ~* ^.+\.(jpg|jpeg|gif|bmp|ico|png|css|js|swf)$ {
    expires 30d;
    # Optional: Don't log access to assets
    access_log off;
    }

    }

  • Enter the following command (if you don’t do this, you will get a 502 bad Gateway error when you later try to connect with your browser):


    sudo vi /etc/php5/fpm/pool.d/www.conf

    Change the following line from:


    listen = /var/run/php5-fpm.sock

    …to:


    listen = 127.0.0.1:9000

Test the web server

  • Create a directory for Piwigo with\;


    sudo mkdir -p /var/www/piwigo
    sudo chown -R www-data:www-data /var/www

  • Create a test file with:


    sudo vi /var/www/piwigo/test.php

    …and paste in the following:


    <?php
    phpinfo();
    ?>

    …then save and exit

  • Restart the web server and Php with the commands:


    sudo /etc/init.d/php5-fpm restart
    sudo /etc/init.d/nginx restart

  • Point your browser at the https://192.168.XXX.XXX/test.php page and check that it works.
  • If it does, you can delete the test page now:

    sudo rm -f /var/www/piwigo/test.php

Prepare for Piwigo installation

Create a database and user

  • Create a database and user with (obviously you need to change PASSWORD with the appropriate password):


    sudo mysql -u root -p
    create database piwigo; grant all on piwigo.* to 'gallery'@'localhost' identified by 'PASSWORD'; flush privileges; \q;

  • Improve the Mysql security by running the command:


    sudo mysql_secure_installation

    …and responding appropriately to the prompts.

Get the netinstall-file for Piwigo

Configuration tweaks

  • Enter the following command to edit the file:


    sudo vi /etc/php5/fpm/php.ini

    …and set:


    upload_max_filesize = 1000M
    post_max_size = 1000M
    max_file_uploads = 20

    At the end of the file, add  the following:


    upload_tmp_dir = /tmp
    extension = apc.so
    apc.enabled = 1
    apc.include_once_override = 0
    apc.shm_size = 256

  • Enter the following command to edit the file:


    sudo vi /etc/nginx/nginx.conf

    …and set:


    client_max_body_size 20M;
    client_body_buffer_size 128k;

  • Restart the web server and Php with the commands:


    sudo /etc/init.d/php5-fpm restart
    sudo /etc/init.d/nginx restart

Install Piwigo

Optional steps

Enabling IPTC support in Piwigo, and copying XMP tags into IPTC

Before transferring my photographs across to Piwigo, I like to add tags/keywords to them using Windows Live Photo Gallery on my Windows PC. The problem with this, is that Live Photo Gallery saves the tags in XMP fields, and Piwigo can only handle the tags in the IPTC fields. Therefore, it’s necessary to copy the XMP tags into the IPTC fields for each photograph. For this, I use exiv2 on pi-piwigo.

  • Update the packages:

    sudo apt-get update

  • Install exiv2 on your Raspberry Pi with:

    sudo apt-get install exiv2

  • You now need to configure Piwigo to be able to use IPTC tags. To do this, sign in to http://192.168.XXX.XXX as your Admin user and install and activate the “LocalFiles Editor” plugin. Then go to:


    Administration > Plugins > LocalFiles Editor > Local config

    …and paste in the following:


    <?php
    $conf['use_iptc'] = true;

    $conf['use_iptc_mapping'] = array(
    'keywords' => '2#025',
    );
    ?>

Once this has been done, you can run the commands that will copy the XMP fields into the IPTC fields of your photographs. You can write a script to do this, but in the commands below we modify the photographs directory by directory.

  • Change to the directory containing the photographs:

    cd /var/www/piwigo/galleries/Digital/1999/199911

  • Make sure that the temporary script we create for each run doesn’t already exist:

    rm /tmp/updateTags.sh

  • Create a temporary script which contains all the information we need to modify the photographs in the current directory:


    for f in *.@([Jj][Pp][Gg]); do echo "exiv2 -M\"set Iptc.Application2.Keywords String " `exiv2 -PXt -gXmp.dc.subject $f 2>/dev/null | sed s/", "/";"/g` "\" $f" >> /tmp/updateTags.sh; done ; sudo chmod 777 /tmp/updateTags.sh

  • If you like, you can inspect the script to make sure everything looks correct before we run it:

    less /tmp/updateTags.sh

  • If you are happy with it, run the script:

    sudo /tmp/updateTags.sh

  • Make sure that all the files still have the correct ownership:

    sudo chown www-data:www-data *

  • Delete our temporary script:

    rm /tmp/updateTags.sh

To force Piwigo to read these tags, perform a metadata synchronization.

Sharing the photographs directory with a PC using samba

As well as being able to view the photographs on Piwigo, I like to be able to view them in Windows Live Photo Gallery too. For this reason, in the next set of instructions, we are going to install samba, which will allow us to share the relevant directory on pi-piwigo with other computers on the network. This directory will be configured as read only for Windows user “smith”, but user www-data will be able to write to it.

  • Update the packages:

    sudo apt-get update

  • Install samba on the Raspberry Pi:

    sudo apt-get install samba samba-common-bin

  • Change into the directory containg the samba configuration file:

    cd /etc/samba/

  • Backup the original configuration file:

    sudo mv smb.conf backsmb.conf

  • Edit the samba configuration file with:


    sudo vi smb.conf

    …and paste the following, changing the workgroup (XXX) as appropriate\;


    #
    [global]
    workgroup = XXX
    server string = My Samba Share %v
    security = user
    map to guest = bad user
    dns proxy = no
    hosts deny = ALL
    hosts allow = 192.168.0.
    #
    [SharePiwigo]
    comment = This is a folder containing photos
    path = /var/www/piwigo/galleries
    browsable = yes
    guest ok = no
    read only = yes
    writeable = no
    valid users = smith www-data

  • Restart samba with:

    sudo service samba restart

  • Create a user on the Raspberry Pi, with the same name as the Windows user you intend to connect with:

    sudo adduser smith
    You will be prompted for a password, which can be anything you like.

  • Create a user on the Raspberry Pi which will be used by samba. The name should be the same as the one you created in the last step, and match the name of the Windows user:

    sudo smbpasswd -a smith
    You will be prompted for a password, which should be the same as the one the user uses on Windows. This will prevent them having to sign in when they access the share.

  • Create another samba user, this time for www-data. If you connect to samba as this user, you will be able to write files to the share, because the filesystem below the mount point is owned by www-data:

    sudo smbpasswd -a www-data
    You will be prompted for a password.

  • Restart samba with:

    sudo service samba restart

You should now be able to browse to this share using your Windows PC.

Transferring the /var/www/piwigo filesystem to an external USB hard disk

Storing the photographs on the Raspberry Pi’s SD card is not an ideal choice, as we want to reduce writes to the SD card as much as possible (to prolong its life), and it is an expensive option in terms of cost per megabyte. It is relatively easy to add a USB hard disk to the Raspberry Pi. More detailed information can be found at…, but here is a summary of the steps I performed.

  • Connect the drive (which should be mains powered as the Raspberry Pi cannot provide it with enough power via USB) to the Raspberry Pi using a USB cable. To determine which device it is recognized as, enter:


    sudo fdisk -l

    In my case, it was recognized as /dev/sda1

  • Create a temporary mount point for the drive:

    sudo mkdir /mnt/USB

  • Change the ownership of the mount point to www-data:

    sudo chown -R www-data:www-data /mnt/USB

  • Completely clear out the disk and create a new ext4 filesystem on it. You will lose any existing data on the drive, and you will no longer be able to attach it to a Windows PC (unless you reformat it):


    sudo mkfs.ext4 /dev/sda1 -L pi-piwigo_USB

    Note that we have given it a label of “pi-piwigo-USB”

  • We can now mount the disk to our temporary mount point with:

  • To test that it is recognized, type in the following to check the disk usage on each drive:

    df -h

  • If all is well, we can copy the content of the /var/www/piwigo directory to the USB drive, and then delete it from the original location:


    cd /var/www/piwigo
    sudo cp -pR * /mnt/USB/
    sudo rm -Rf *

  • We sould configure the Raspberry Pi to automount the USB drive whenever it boots up. To avoid any risk of multiple USB drives being confused and switched over, we should identify the unique ID assigned to the drive and mount that. To get the ID, enter:


    sudo ls -laF /dev/disk/by-uuid/

    In my case, the drive ID is 76a9e7e1-1218-4ba2-9c9d-3bda2e78d610

  • To get the drive to automount, we need to edit the fstab file:


    sudo vi /etc/fstab

    ...and add the following (obviously changing the ID as appropriate):


    UUID=76a9e7e1-1218-4ba2-9c9d-3bda2e78d610 /mnt/USB ext4 rw,defaults 0 0

    Save and exit the file.

  • Reboot the Raspberry Pi with:

    sudo reboot

  • On rebooting, check that /var/www/piwigo contains the files, and that the ownership of the files is www-data:

    ls -l /var/www/piwigo

  • If all is well, delete the temporary mount point:

    sudo rmdir /mnt/USB

How to install Samba on a Raspberry Pi

In this posting, we will be installing samba file sharing on a Raspberry Pi.

Preparation of the Raspberry Pi

  • Follow the instructions at How to install the Raspbian OS on a Raspberry Pi.
  • Change the hostname of your Raspberry Pi to something more meaningful by logging on to the device as user “pi” and issuing the command:

    sudo raspi-config

  • Select “Advanced Options” and set the following:
    • Hostname = pi-samba (or whatever you want to call it).
  • Select Finish
  • Reboot the Raspberry Pi

Installing samba

In the next set of instructions, we are going to install samba, which will allow us to share a directory on pi-samba with other computers on the network. This directory will be configured as writable for Windows user “smith”.

  • Update the packages:

    sudo apt-get update

  • Install samba on the Raspberry Pi:

    sudo apt-get install samba samba-common-bin

  • Change into the directory containg the samba configuration file:

    cd /etc/samba/

  • Backup the original configuration file:

    sudo mv smb.conf backsmb.conf

  • Edit the samba configuration file with:


    sudo vi /etc/samba/smb.conf

    …and replace the content with the following, obviously changing the workgroup (XXX) as appropriate:


    #
    [global]
    workgroup = XXX
    server string = My Samba Share %v
    security = user
    map to guest = bad user
    dns proxy = no
    hosts deny = ALL
    hosts allow = 192.168.0.
    #
    [Share1]
    comment = This is a shared folder
    path = /home/smith/share1
    browsable = yes
    guest ok = no
    read only = no
    writeable = yes
    valid users = smith
    force user = smith
    force group = smith
    create mask = 0644

  • Create a user on the Raspberry Pi, with the same name as the Windows user you intend to connect with:

    sudo adduser smith
    You will be prompted for a password, which can be anything you like.

  • Create a user on the Raspberry Pi which will be used by samba. The name should be the same as the one you created in the last step, and match the name of the Windows user:

    sudo smbpasswd -a smith
    You will be prompted for a password, which should be the same as the one the user uses on Windows. This will prevent them having to sign in when they access the share.

  • Create the appropriate directory with:

    sudo mkdir -p /home/smith/share1

  • Change the ownership of the shared directory so that it matches our new user

    sudo chown -R smith:smith /home/smith/share1

  • Restart samba with:

    sudo service samba restart

You should now be able to browse to this share using your Windows PC.

Further steps

Storing the files on the Raspberry Pi’s SD card is not an ideal choice, as we want to reduce writes to the SD card as much as possible (to prolong its life), and it is an expensive option in terms of cost per megabyte. It is relatively easy to add a USB hard disk to the Raspberry Pi. More detailed information can be found at

Using rsnapshot with a Raspberry Pi to create a backup device

In this posting we are going to turn a Raspberry Pi into a backup device, using the excellent rsnapshot utility to save full and incremental backups to an external USB hard drive. More information about rsnapshot can be found at http://www.rsnapshot.org/.

Preparation of the Raspberry Pi

  • Follow the instructions at How to install the Raspbian OS on a Raspberry Pi.
  • Change the hostname of your Raspberry Pi to something more meaningful by logging on to the device as user “pi” and issuing the command:

    sudo raspi-config

  • Select “Advanced Options” and set the following:
    • Hostname = pi-rsync (or whatever you want to call it).
  • Select Finish
  • Reboot the Raspberry Pi

Install rsnapshot

  • Installing the rsnapshot package is as easy as entering the command:

    sudo apt-get install rsnapshot

  • On completion, create a directory in which the backups will be stored. For the moment we will back up to the /tmp/ directory on the Raspberry Pi’s SD card, but once we are happy it is working, we will mount an external hard disk and back up to that. For now, type:


    mkdir /tmp/backup

    This directory will be lost whenever the Raspberry Pi is rebooted, so it should not be used as a permanent storage location. Also, we want to avoid writing to the SD card as much as possible, as this will shorten it’s working life.

Configure the backup

  • Create an rsnapshot configuration file by entering the command:


    sudo vi /etc/rsnapshot.conf.pi-rsync

    …and paste the following into the file (note that tab characters are used between the parameters, NOT spaces. Spaces will cause rsnapshot to fail when you test it):

    #################################################
    # rsnapshot.conf - rsnapshot configuration file #
    #################################################
    # #
    # PLEASE BE AWARE OF THE FOLLOWING RULES: #
    # #
    # This file requires tabs between elements #
    # #
    # Directories require a trailing slash: #
    # right: /home/ #
    # wrong: /home #
    # #
    #################################################

    #######################
    # CONFIG FILE VERSION #
    #######################

    config_version 1.2
    ###########################
    # SNAPSHOT ROOT DIRECTORY #
    ###########################

    # All snapshots will be stored under this root directory.
    snapshot_root /tmp/backup/
    # If no_create_root is enabled, rsnapshot will not automatically create the
    # snapshot_root directory. This is particularly useful if you are backing
    # up to removable media, such as a FireWire or USB drive.
    no_create_root 1
    cmd_cp /bin/cp
    cmd_rm /bin/rm
    cmd_rsync /usr/bin/rsync
    cmd_ssh /usr/bin/ssh
    cmd_logger /usr/bin/logger
    cmd_du /usr/bin/du
    #interval hourly 6
    interval daily 7
    interval weekly 4
    interval monthly 12
    interval yearly 2
    verbose 2
    loglevel 3
    logfile /var/log/rsnapshot
    lockfile /var/run/rsnapshot.pid
    ssh_args -o BatchMode=yes
    #include ???
    #include ???
    #exclude ???
    #exclude ???
    #include_file /path/to/include/file
    #exclude_file /path/to/exclude/file
    backup /home/pi/ pi-rsync/

  • Test that the backup works with the command:


    sudo /usr/bin/rsnapshot -c /etc/rsnapshot.conf.pi-rsync daily

    The last line of the configuration file told rsnapshot to copy the /home/pi/ directory to the directory pi-rsync/ under /tmp/backup/ (which is the “snapshot_root” directory).

  • If the test succeeded, you will find the backup by typing the command:


    sudo ls -l /tmp/backup/pi-rsync

    Below this you will find the directory daily.0 which contains a complete replica of the /home/pi/ directory. If you run the test again, daily.0 will be moved to daily.1, and a new daily.0 will be created with the same files. However, because the files are linked, and they haven’t changed, the backup will be no larger than before. Clever, eh?

Change the backup destination to an external USB drive

Once you have confirmed that rsnapshot is working correctly, an external USB hard disk can be mounted on the Pi to allow for far greater storage capacity for backups. Ideally, the USB hard drive that you choose should be mains powered, as the Pi cannot provide much power to external devices. Alternatively, a powered USB hub could be used as an intermediary between the Pi and the USB drive.

  • Follow the instructions at How to mount and automount a USB hard drive on the Raspberry Pi.
  • Ensure that the USB drive is mounted to /mnt/USB/
  • Change the rsnapshot configuration file by entering the command:


    sudo vi /etc/rsnapshot.conf.pi-rsync

    …and change the line:

    snapshot_root /tmp/backup/

    …to:

    snapshot_root /mnt/USB/

  • Test that the backup still works with the command:


    sudo /usr/bin/rsnapshot -c /etc/rsnapshot.conf.pi-rsync daily

    Because we changed “snapshot_root” directory, you should now fine the backup under /mnt/USB/pi-rsync/daily.0

Add pi-owncloud to the backup

In the next set of instructions, we will configure backing up the ownCloud server we created in How to install ownCloud on a Raspberry Pi. Here, we will backup the /var/www/owncloud/ directory owned by the user “www-data” and an export of the database, which is saved to the /home/pi/ directory and owned by the user "pi".

On the pi-rsync device:

  • Open a command shell as the root user, create a directory in which to store our key pairs, and then change to that directory:


    sudo bash
    mkdir -p ~/.ssh
    cd ~/.ssh

  • Create a key pair which will be used for authorizing access from pi-rsync to pi-owncloud:


    ssh-keygen -t dsa -b 1024 -f ./pi-owncloud-rsnapshot-key

    (Press "Enter" when asked for a passphrase, leaving the passphrase empty).

  • Now type the command:


    vi ~/.ssh/config

    ...and paste the following to the end of the file:

    Host remotehost-rsnapshot-pi-owncloud
    Hostname 192.168.0.129
    IdentityFile /root/.ssh/pi-owncloud-rsnapshot-key

    Save it, and exit vi.

  • Change the permissions, to increase security:


    chmod 700 ~/.ssh
    chmod 600 ~/.ssh/*-rsnapshot-key

  • Exit the root user:

    exit

On the pi-owncloud device:

  • Schedule a backup of the owncloud SQLLite database to occur every day at 10:50, by opening the crontab file:


    sudo crontab -e

    ...and adding the following:


    50 10 * * * sqlite3 /var/www/owncloud/data/owncloud.db .dump > /home/pi/owncloud-sqlbkp.bak

    Save it, and exit vi. Note that this backup will be overwritten every day on the Raspberry Pi, but that's fine because it will be backed up daily to pi-rsync.

  • Add the content of the file /root/.ssh/pi-owncloud-rsnapshot-key.pub on pi-rsync to the file /home/pi/.ssh/authorized_keys on pi-owncloud
  • Add the content of the file /root/.ssh/pi-owncloud-rsnapshot-key.pub on pi-rsync to the file /var/www/.ssh/authorized_keys on pi-owncloud
  • Ensure that user "www-data" can connect to the pi-owncloud device via ssh. Issue the following command:


    sudo vi /etc/ssh/sshd_config

    ...and change the "AllowUsers" line to include "www-data", for example:

    AllowUsers pi www-data

  • Restart ssh by running the command:

    sudo service ssh restart

On the pi-rsync device:

  • Test that you can connect to pi-owncloud as user "pi" from pi-rsync with the command:


    sudo ssh pi@remotehost-rsnapshot-pi-owncloud

    ...and test that you can also connect to pi-owncloud as user "www-data" from pi-rsync with the command:

    sudo ssh www-data@remotehost-rsnapshot-pi-owncloud

    ...and when you have confirmed that you have opened a shell on the remote server, quit the remote shell with the command:

    exit

Restrict access to pi-owncloud by IP address

  • Tighten up the security a little, so that only the IP address of pi-rsync can connect to pi-owncloud using this key. You need to know the IP address of pi-rsync, which you can get by entering the command:


    ifconfig

  • Edit the authorized_keys file for user "pi" on the pi-owncloud device by entering the command:


    sudo vi /home/pi/.ssh/authorized_keys

    ...and insert the following at the very start of the key you just pasted in the previous section. Note that there is a trailing space character after the last double-quote, and that you need to replace the IP address with that of your own pi-rsync:

    from="192.168.0.105"

  • Test the connection again from pi-rsync with the command:


    sudo ssh pi@remotehost-rsnapshot-pi-owncloud

    ...and when you have confirmed that you have opened a shell on the remote server, quit the remote shell with:

    exit

  • Now do the same for the "www-data" user. Edit the authorized_keys file for user "pi" on the pi-owncloud device by entering the command:


    sudo vi /var/www/.ssh/authorized_keys

    ...and insert the following at the very start of the key you just pasted in the previous section. Note that there is a trailing space character after the last double-quote, and that you need to replace the IP address with that of your own pi-rsync:

    from="192.168.0.105"

  • Test the connection again from pi-rsync with the command:


    sudo ssh www-data@remotehost-rsnapshot-pi-owncloud

    ...and when you have confirmed that you have opened a shell on the remote server, quit the remote shell with:

    exit

Restrict the commands that can be run by pi-rsync on pi-owncloud

  • To tighten up security further, so that only rsync commands can be run on pi-owncloud using this key, edit the authorized_keys file on pi-owncloud with:


    sudo vi /home/pi/.ssh/authorized_keys

    ...and change the command at the start of the key with the following. Note that there is a trailing space character after the last double-quote, and that you need to replace the IP address with that of your own pi-rsync:

    from="192.168.0.105",command="/usr/local/sbin/validate-rsync"

  • Now do the same for the key in www-data.edit the authorized_keys file on pi-owncloud with:


    sudo vi /var/www/.ssh/authorized_keys

    , and change the command at the start of the key with the following. Note that there is a trailing space character after the last double-quote, and that you need to replace the IP address with that of your own pi-rsync:

    from="192.168.0.105",command="/usr/local/sbin/validate-rsync"
    .

  • Now run the following on pi-owncloud:


    sudo vi /usr/local/sbin/validate-rsync

    ...and paste the following into the file:

    #!/bin/sh

    case "$SSH_ORIGINAL_COMMAND" in
    *\&*)
    echo "Rejected"
    ;;
    *\(*)
    echo "Rejected"
    ;;
    *\{*)
    echo "Rejected"
    ;;
    *\;*)
    echo "Rejected"
    ;;
    *\<*)
    echo "Rejected"
    ;;
    *\`*)
    echo "Rejected"
    ;;
    *\|*)
    echo "Rejected"
    ;;
    rsync\ --server*)
    $SSH_ORIGINAL_COMMAND
    ;;
    *)
    echo "Rejected"
    ;;
    esac

  • Change the permission with: sudo chmod +x /usr/local/sbin/validate-rsync

Configure rsnapshot to run

  • Add the following 3 lines to the bottom of /etc/rsnapshot.conf.pi-rsync on pi-rsync (the spaces between the fields must actually be tabs)


    # pi-owcloud = ownCloud server on pi
    backup www-data@remotehost-rsnapshot-pi-owncloud:/var/www/owncloud pi-owncloud/
    backup pi@remotehost-rsnapshot-pi-owncloud:/home/pi/ pi-owncloud/

  • Test that the backup works with:


    sudo /usr/bin/rsnapshot -c /etc/rsnapshot.conf.pi-rsync daily

Configure cron

  • open the crontab by typing the command:


    sudo crontab -e

    ...and paste the following into the file:


    45 22 * * * /usr/bin/rsnapshot -c /etc/rsnapshot.conf.pi-rsync daily
    30 22 * * 0 /usr/bin/rsnapshot -c /etc/rsnapshot.conf.pi-rsync weekly
    15 22 1 * * /usr/bin/rsnapshot -c /etc/rsnapshot.conf.pi-rsync monthly
    0 22 1 12 * /usr/bin/rsnapshot -c /etc/rsnapshot.conf.pi-rsync yearly

    Save and exit the file

How to install ownCloud on a Raspberry Pi

In this posting, we are going to install a personal Cloud package, called “ownCloud”, on a Raspberry Pi.

Preparation of the Raspberry Pi

  • Follow the instructions at How to install the Raspbian OS on a Raspberry Pi.
  • Change the hostname of your Raspberry Pi to something more meaningful by logging on to the device as user “pi” and issuing the command:

    sudo raspi-config

  • Select “Advanced Options” and set the following:
    • Hostname = pi-owncloud (or whatever you want to call it).
  • Select Finish
  • Reboot the Raspberry Pi

Install PHP, SqlLite, the Nginx web server, and other support packages

These instructions have been modified from thos originally found at http://www.techjawab.com/2013/09/how-to-setup-your-own-cloud-on.html.

  • Create a user for the Nginx web server with the command:

    sudo groupadd www-data ; sudo usermod -a -G www-data www-data

  • Install the necessary packages with the command:

    sudo apt-get install nginx openssl ssl-cert php5-cli php5-sqlite php5-gd php5-curl php5-common php5-cgi sqlite3 php-pear php-apc curl libapr1 libtool curl libcurl4-openssl-dev php-xml-parser php5 php5-dev php5-gd php5-fpm memcached php5-memcache varnish

  • Create your SSL certificates (which will last for 2 years) with the commands:


    sudo openssl req $@ -new -x509 -days 730 -nodes -out /etc/nginx/cert.pem -keyout /etc/nginx/cert.key
    sudo chmod 600 /etc/nginx/cert.pem ; sudo chmod 600 /etc/nginx/cert.key

Configure the Nginx web server

  • Open the configuration file for editing:

    sudo vi /etc/nginx/nginx.conf

  • Because the Raspberyy Pi only has 2 cores, change


    worker_processes 4;
    …to:
    worker_processes 2;

  • Change the top of the http section to:


    http {

    ##
    # Basic Settings
    ##

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    # keepalive_timeout 65;
    types_hash_max_size 2048;
    server_tokens off;

    client_header_timeout 10;
    client_body_timeout 10;
    keepalive_timeout 10 10;
    send_timeout 10;

  • Change the Gzip section from:


    ##
    # Gzip Settings
    ##

    gzip on;
    gzip_disable "msie6";

    # gzip_vary on;
    # gzip_proxied any;
    # gzip_comp_level 6;
    # gzip_buffers 16 8k;
    # gzip_http_version 1.1;
    # gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
    …to:
    ##
    # Gzip Settings
    ##

    gzip on;
    gzip_disable "msie6";

    gzip_min_length 1100;
    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_buffers 16 8k;
    gzip_http_version 1.1;
    gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

  • Now do the following:


    sudo vi /etc/php5/fpm/pool.d/www.conf
    …and uncomment the following two lines::
    listen.owner = www-data
    listen.group = www-data

  • Enter the following command:

    sudo vi /etc/nginx/sites-available/default

  • …and replace the entire content of the file with the code below (taken from http://doc.owncloud.org/server/5.0/admin_manual/installation/installation_others.html#nginx-configuration). Note: You’ll have to replace two occurrences of 192.168.XXX.XXX with the local IP address of your Raspberry Pi.


    server {
    listen 80;
    server_name 192.168.XXX.XXX;
    return 301 https://$server_name$request_uri; # enforce https
    }

    server {
    listen 443 ssl;
    server_name 192.168.XXX.XXX;

    ssl_certificate /etc/nginx/cert.pem;
    ssl_certificate_key /etc/nginx/cert.key;

    # Path to the root of your installation
    root /var/www/owncloud;

    client_max_body_size 1000M; # set max upload size
    fastcgi_buffers 64 4K;

    rewrite ^/caldav(.*)$ /remote.php/caldav$1 redirect;
    rewrite ^/carddav(.*)$ /remote.php/carddav$1 redirect;
    rewrite ^/webdav(.*)$ /remote.php/webdav$1 redirect;

    index index.php;
    error_page 403 /core/templates/403.php;
    error_page 404 /core/templates/404.php;

    location = /robots.txt {
    allow all;
    log_not_found off;
    access_log off;
    }

    location ~ ^/(data|config|\.ht|db_structure\.xml|README) {
    deny all;
    }

    location / {
    # The following 2 rules are only needed with webfinger
    rewrite ^/.well-known/host-meta /public.php?service=host-meta last;
    rewrite ^/.well-known/host-meta.json /public.php?service=host-meta-json last;

    rewrite ^/.well-known/carddav /remote.php/carddav/ redirect;
    rewrite ^/.well-known/caldav /remote.php/caldav/ redirect;

    rewrite ^(/core/doc/[^\/]+/)$ $1/index.html;

    try_files $uri $uri/ index.php;
    }

    location ~ ^(.+?\.php)(/.*)?$ {
    try_files $1 = 404;

    include fastcgi_params;
    fastcgi_param SCRIPT_FILENAME $document_root$1;
    fastcgi_param PATH_INFO $2;
    fastcgi_param HTTPS on;
    fastcgi_pass 127.0.0.1:9000;
    # Or use unix-socket with 'fastcgi_pass unix:/var/run/php5-fpm.sock;'
    }

    # Optional: set long EXPIRES header on static assets
    location ~* ^.+\.(jpg|jpeg|gif|bmp|ico|png|css|js|swf)$ {
    expires 30d;
    # Optional: Don't log access to assets
    access_log off;
    }

    }

Configure the maximum upload limit for php

  • Enter the following command to edit the file:

    sudo vi /etc/php5/fpm/php.ini

  • …and set:


    upload_max_filesize = 1000M
    post_max_size = 1000M

  • At the end of the file, add ┬áthe following:


    upload_tmp_dir = /srv/http/owncloud/data
    extension = apc.so
    apc.enabled = 1
    apc.include_once_override = 0
    apc.shm_size = 256

Configure PHP

  • Enter the following command:

    sudo vi /etc/php5/fpm/pool.d/www.conf

  • Change the following line from:


    listen = /var/run/php5-fpm.sock

    …to


    listen = 127.0.0.1:9000

Increase the size of the swap file

  • Enter the following command:


    sudo vi /etc/dphys-swapfile

    …and change the following line from:


    CONF_SWAPSIZE=100

    …to :


    CONF_SWAPSIZE=512

  • Restart the web server and Php with the commands:


    sudo /etc/init.d/php5-fpm restart
    sudo /etc/init.d/nginx restart

Install ownCloud

Enter the following commands to install ownCloud on your Raspberry Pi (by the time you read, this there may be a newer version of ownCloud available).

  • To create a folder for ownCloud with the relevant rights, enter the following commands:


    sudo mkdir -p /var/www/owncloud
    sudo wget http://download.owncloud.org/community/owncloud-6.0.2.tar.bz2
    sudo tar xvf owncloud-6.0.2.tar.bz2
    sudo mv owncloud/ /var/www/
    sudo chown -R www-data:www-data /var/www
    rm -rf owncloud owncloud-6.0.2.tar.bz2

Finally:

  • Go to https://192.168.XXX.XXX/ (replace the address with your own IP address) and create an Admin account.
  • You should now have a working copy of ownCloud on your Raspberry Pi, and you can follow the manual at http://owncloud.org/ to figure out how it works.

How to prepare and automount an external USB hard disk on the Raspberry Pi

When choosing the USB external hard drive you will be attaching to the PI, bear in mind that ideally it should be mains powered. This is because the Raspberry Pi cannot provide much power to external devices. Alternatively, a powered USB hub could be used as an intermediary between the Raspberry Pi and the USB drive.

In the instructions below, we will be mounting a brand new, straight out of the box, Western Digital Elements 3Tb desktop drive to our Raspberry Pi, and formatting it with the EXT4 filesystem. I don’t intend to use this disk for any purpose other than as a backup drive, attached to the Pi, so this type of filesystem is perfect for my needs.

Getting information about our drive

  • With your Raspberry Pi powered up, connect the hard disk to it using a USB cable.
  • Switch on the hard disk (if it is mains powered and not already switched on)
  • Wait a few seconds for the drive to be recognized and then enter the following command to check if the Raspberry Pi acknowledges its existence:


    sudo fdisk -l

    You should see something like the following:


    Disk /dev/mmcblk0: 7888 MB, 7888437248 bytes
    4 heads, 16 sectors/track, 240736 cylinders, total 15407104 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0x000981cb

    Device Boot Start End Blocks Id System
    /dev/mmcblk0p1 8192 122879 57344 c W95 FAT32 (LBA)
    /dev/mmcblk0p2 122880 15407103 7642112 83 Linux
    Note: sector size is 4096 (not 512)

    Disk /dev/sda: 3000.6 GB, 3000590401536 bytes
    255 heads, 63 sectors/track, 45600 cylinders, total 732566016 sectors
    Units = sectors of 1 * 4096 = 4096 bytes
    Sector size (logical/physical): 4096 bytes / 4096 bytes
    I/O size (minimum/optimal): 4096 bytes / 4096 bytes
    Disk identifier: 0x0009b8b3

    Device Boot Start End Blocks Id System
    /dev/sda1 256 732566015 2930263040 7 HPFS/NTFS/exFAT

    The entry we are interested in is the bottom one, and we can see that the USB drive has been assigned as device /dev/sda1.

Reformat the drive with EXT4

Note that once we do this, you will no longer be able to use the drive on a Windows PC (unless you reformat it again and lose all the data on it). If you want to use the disk on a Windows PC as well, NTFS would be a better choice of filesystem.

  • Ensure that the drive is not mounted:

    sudo umount /dev/sda1

  • Now we can recreate the filesystem on it (note that we are giving the drive a label “backup0”):


    sudo mkfs.ext4 /dev/sda1 -L backup0

    …and you should see something like:


    mke2fs 1.42.5 (29-Jul-2012)
    Filesystem label=backup0
    OS type: Linux
    Block size=4096 (log=2)
    Fragment size=4096 (log=2)
    Stride=0 blocks, Stripe width=0 blocks
    183148544 inodes, 732565760 blocks
    36628288 blocks (5.00%) reserved for the super user
    First data block=0
    Maximum filesystem blocks=0
    22357 block groups
    32768 blocks per group, 32768 fragments per group
    8192 inodes per group
    Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
    4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
    102400000, 214990848, 512000000, 550731776, 644972544

    Allocating group tables: done
    Writing inode tables: done
    Creating journal (32768 blocks): done
    Writing superblocks and filesystem accounting information: done

  • On completion, you can mount the drive using the instructions in the next section.

Mounting our drive

  • To mount the disk so that it can be used, you first need to create a mount point, which in our case will be /mnt/USB. Do this with the command:

    sudo mkdir /mnt/USB

  • using the information that we gleaned earlier about the drive, we can now mount the drive to this mount point with the command:

    sudo mount -t ext4 /dev/sda1 /mnt/USB

  • To check that the drive is, indeed, mounted, enter:


    df -h

    …and you should see something like:


    Filesystem Size Used Avail Use% Mounted on
    rootfs 7.2G 2.1G 4.8G 30% /
    /dev/root 7.2G 2.1G 4.8G 30% /
    devtmpfs 235M 0 235M 0% /dev
    tmpfs 49M 220K 49M 1% /run
    tmpfs 5.0M 0 5.0M 0% /run/lock
    tmpfs 98M 0 98M 0% /run/shm
    /dev/mmcblk0p1 56M 19M 38M 34% /boot
    /dev/sda1 2.7T 73M 2.6T 1% /mnt/USB

Automount the drive on bootup

We don’t want to manually mount the drive every time the Raspberry Pi reboots, so these instructions will automate the procedure.

  • Instead of using the device name when mounting the drive automatically, we should use the device’s UUID, which is a unique identifier assigned to the drive hardware itself. This ensures that drives do not get mounted to incorrect mount points when multiple USB drives are in use. Be careful, though, the UUID will change if you create a new filesystem on it in the future. To determine the drive’s current UUID, enter the following command while it is mounted:


    sudo ls -laF /dev/disk/by-uuid/

    You should see something like:


    total 0
    drwxr-xr-x 2 root root 100 Mar 28 12:44 ./
    drwxr-xr-x 6 root root 120 Mar 28 12:38 ../
    lrwxrwxrwx 1 root root 15 Jan 1 1970 993B-8922 -> ../../mmcblk0p1
    lrwxrwxrwx 1 root root 10 Mar 28 12:44 be2f6f63-99ac-4202-a540-70f59d64aa01 -> ../../sda1
    lrwxrwxrwx 1 root root 15 Mar 26 12:28 fc254b57-8fff-4f96-9609-ea202d871acf -> ../../mmcblk0p2
    Our drive is therefore be2f6f63-99ac-4202-a540-70f59d64aa01

  • To mount the drive automatically on boot, you need to edit /etc/fstab by entering the following command:


    sudo vi /etc/fstab
    …and then add the following line at the end:


    UUID=be2f6f63-99ac-4202-a540-70f59d64aa01 /mnt/USB ext4 rw,defaults 0 0

  • Test that the drive auto-mounts by rebooting the Raspberry Pi with:

    sudo reboot

  • When the Raspberry Pi is booted up, check that the drive is mounted by typing:

    df -h

Checking which filesystem is in use on a hard disk

If you have an existing hard disk, and want to check which filesystem is on it, use the following instructions.

  • To check what type of file system is in use, enter the command:


    sudo parted -l

    You should see something like:


    Model: Seagate External Drive (scsi)
    Disk /dev/sda: 320GB
    Sector size (logical/physical): 512B/512B
    Partition Table: msdos

    Number Start End Size Type File system Flags
    1 32.3kB 320GB 320GB primary ext4

    Model: SD 00000 (sd/mmc)
    Disk /dev/mmcblk0: 7888MB
    Sector size (logical/physical): 512B/512B
    Partition Table: msdos

    Number Start End Size Type File system Flags
    1 4194kB 62.9MB 58.7MB primary fat16 lba
    2 62.9MB 7888MB 7826MB primary ext4

    We can see that drive /dev/sda (the top one in the example above) uses the EXT4 filesystem.

How to set up iptables on a Raspberry Pi

iptables is a form of firewall included in many Linux packages, it can also be used for network address translation. Here, we configure it on a Raspberry Pi to allow communication on port 80, and requests from other devices on the 192.168.0.* IP range.

Set up iptables

  • We will configure our iptables rules in a file, and then load that file into iptables. Open the rules file for editing with:

    sudo vi /etc/iptables.firewall.rules

  • …and replace the content with the following:


    *filter
    :INPUT DROP [23:2584]
    :FORWARD ACCEPT [0:0]
    :OUTPUT ACCEPT [1161:105847]
    -A INPUT -i lo -j ACCEPT
    -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
    -A INPUT -s 192.168.0.0/24 -j ACCEPT
    -A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT
    -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
    COMMIT

  • Where:
    *filter
    :INPUT DROP [23:2584]For all packets destined for the host computer, the default policy is to drop the packet.
    :FORWARD ACCEPT [0:0]For all packets passing through (or being routed by) the host computer, the default policy is to accept the packet.
    :OUTPUT ACCEPT [1161:105847]For all packets originating from the host computer, the default policy is to accept the packet.
    -A INPUT -i lo -j ACCEPTAccept all incoming packets that are destined for the localhost (lo, 127.0.0.1) interface.
    -A INPUT -p tcp -m tcp --dport 80 -j ACCEPTAccept all incoming packets destined for tcp port 80 (i.e. http).
    -A INPUT -s 192.168.0.0/24 -j ACCEPTAccept all incoming packets, on all network interfaces, originating from devices on our local network IP address range.
    -A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPTAccept any ping requests to the host.
    -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPTIf the state of an incoming packet is ESTABLISHED or RELATED (i.e. not a NEW connection that wasn’t initiated by the host), the packet will be accepted.
    COMMITCommit the rules above.
  • Activate the new iptables rules by loading the file with:

    sudo iptables-restore < /etc/iptables.firewall.rules

  • Check to see what rules you configured above with:

    sudo iptables -L

  • To make sure the iptables rules are loaded on a reboot we’ll create a new file. Issue the following command:


    sudo vi /etc/network/if-pre-up.d/firewall

    …and paste the content below:


    #!/bin/sh
    /sbin/iptables-restore < /etc/iptables.firewall.rules

  • Make the file executable:

    sudo chmod +x /etc/network/if-pre-up.d/firewall