How I Created My Own Personal Cloud Using BitTorrent Sync, Owncloud, and Raspberry Pi

I’ve had a Raspberry Pi lying around the house for a few months now. Every once in a while I’ll take it out and tinker around with it. When BitTorrent released the BitTorrent Sync Alpha a few weeks back, I decided to see what kind of options I had for using the program with my Raspberry Pi.

BitTorrent Sync is awesome. If you don’t know about it, you can read all about it here. In a nutshell, it allows you to sync folders between machines using P2P. No cloud necessary. I’ve been using it for syncing several gigabytes of RAW photos and video across my various machines. There is the occasional scenario however, where I’ve wanted to grab a few files, but my other machine is turned off. To solve this problem, I’m using a Raspberry Pi as a low power, always-on device with Sync installed. Just for kicks, I’m also using Owncloud (open source) to provide me with a web interface for accessing my files from any computer, including my mobile phone. Sound interesting?

If you have a DIY attitude and aren’t afraid of a few terminal commands, then read on to learn about how I did it.

First, you’ll need the following:

  • A Raspberry Pi
  • USB Keyboard for setup
  • USB Mouse for setup
  • SD card (faster, the better)
  • A thumb drive (I like using this for its low profile). You can also use a self powered external hard drive, or just the space available on the SD card.
  • Micro USB charger
  • WiFi adaptor (optional. I use this one)
  • Some type of display for setup. I just used HDMI out to my TV.

Step 1: Setting up Raspbian

Fire up your favorite torrent client and download the Raspbian “wheezy” image. Click here for the link.

I won’t go in to full detail on how to write the images to the SD card. There are plenty of resources available for that. If you are using Windows, check out this link. If you are on a Mac, check out this link.

raspi-config_main

When that is done, plug the SD card into your RasPi and fire it up. For the initial configuration screen that appears I recommend setting your username and password, expanding the root partition, setting your boot_behavior to start desktop on boot and enabling SSH. Exit the configuration screen and you’ll eventually be presented with the Wheezy desktop.

Screen Shot 2013-05-23 at 3.51.38 PM

Step 2: Installing BitTorrent Sync

Make sure you have an ethernet cable plugged in to your RasPi, and fire up lxterminal (there is a shortcut on the desktop). We’ll need to download the binary for Sync. To do this, type in:

wget "http://btsync.s3-website-us-east-1.amazonaws.com/btsync_arm.tar.gz"

Then change the download directory:

cd /home/pi

extract the tar.gz:

tar -zxvf btsync_arm.tar.gz

And finally, run the program:

./btsync

Now, we want to make sure Sync will run at startup. This requires an extra step. In terminal, change your directory to the config folder:

cd ~/.config/

Create a new folder called “autostart”

mkdir autostart

Now create a new text document. You can use the leafpad program that comes pre-installed. The contents of the text document need to be as follows:

[Desktop Entry]

Type=Application

Exec=/home/pi/btsync

Save this file in the newly created ~/.config/autostart as: syncstart.desktop

Sync is now up and running on your tiny Raspberry Pi. You can access the web GUI by opening a browser and typing in [YOUR IP ADDRESS]:8888/gui. For example: 192.168.1.2:8888/gui. You can find the IP address for your RasPi by opening terminal and typing in: ifconfig

sync-SS

At this point, you can stop right here. If your SD card is big enough, you can set up Sync and have your always-on Sync device….but why stop there? Lets add a thumb drive so that we can have more space.

Step 3: Adding a thumb drive

As mentioned, you have plenty of options for adding hard drive space to your Raspberry Pi. If you want to, you can plug in an external hard drive and go crazy. For this blog post however, I’m just using a micro-sd thumb drive. I love how small it is.

Plug the thumb drive in to the Raspberry Pi. (if you are short on usb ports, I would suggest setting up remote desktop). Open up terminal and type in:

df -h

Find your thumb drive. Mine was at the bottom and was called /dev/sda1. Take note of the name. You’ll need it. Format your hard drive by doing the following (replace sda1 with the name of your drive):

sudo umount /dev/sda1
sudo mkfs.ext4 /dev/sda1 -L SYNC

If you aren’t interested in Owncloud, then you can stop here and simply set up Sync to point to /media/SYNC. If you are interested in Owncloud, then keep reading.

step 4: Setting up Owncloud

Setting up Owncloud on linux is a mess of various terminal commands. Thankfully, Github user Petrockblog has made it very simple. He created a shell script for installing and updating Owncloud. Simply do the following:

Install Github:

sudo apt-get update
sudo apt-get install -y git dialog

Download the latest setup script:

cd
git clone git://github.com/petrockblog/OwncloudPie.git

And execute the script:

cd OwncloudPie
chmod +x owncloudpie_setup.sh
sudo ./owncloudpie_setup.sh

…and wait. This will take a while. Grab a coffee. Watch this video. When it is done, you can navigate to [Your IP Address]/owncloud to set everything up. It is very important that you click on “advanced” and set your data folder to /media/SYNC before finishing setup.

owncloud

Step 5: Combining BitTorrent Sync and Owncloud

First we need to clear up a few file permissions. Add sync to the www-data group:

sudo usermod -a -G www-data sync

Allow sync to write to the thumb drive:

sudo chmod -R 770 /media/SYNC 

Now open up the BitTorrent Sync WebUI (YourIpAddress:8888/gui) and add a folder. You’ll want to add: /media/SYNC/USERNAME/files where “username” is the name you selected when setting up Owncloud. Generate a key and you are done. You can now use that key to set up a folder on any computer you want. Any files added to that folder will be synced to your RasPi and be available through the Owncloud web interface. While this is pretty cool, there’s just one more little step to make this much more useful.

Step 6: Set up Owncloud to be accessed from anywhere (including mobile phones)

To access our files from anywhere, we’ll need to first set up a static IP address. If you used the wifi dongle I recommended, you can simply plug it in and it will work. Open terminal and type in:

sudo nano /etc/network/interfaces

You’ll need to make a few changes in order to get a static IP address. Below is what I ended up using after a bit of trial and error. Your address, gateway, net mask, SSID, and PSK will need to be changed according to your network. The settings may be different if you used a different wireless adaptor, or if you are only using an ethernet cable.

auto lo
iface lo inet loopback

iface eth0 inet static
        address 192.168.1.20
        gateway 192.168.1.1
        netmask 255.255.255.0

auto wlan0
#allow-hotplug wlan0
iface wlan0 inet static
        address 192.168.1.22
        gateway 192.168.1.1
        netmask 255.255.255.0
        wpa-ssid "YOUR SSID"
        wpa-psk "YOUR PSK"
#wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp

Finally, you’ll need to forward a port so that owncloud can be accessed from outside networks. I just forwarded port 80 for the IP address of my RasPi. In this example, my IP address when using wifi is 192.168.1.22. The procedure for forwarding ports is different for every router. If I grab my real IP address from a site such as this one, I can now access Owncloud and my Sync files from anywhere by going to [MyRealIpAddress]/owncloud (where the IP address is what you find from the website I linked to). You can also set up Owncloud with an Android or iOS client for mobile access.

owncloud2

IMG_4222

Phew. If you made it this far, congrats. I had fun creating my own personal BitTorrent-powered cloud and I hope you found this walkthrough helpful. There are so many uses for something like BitTorrent Sync, and I think we’ve only begun to scratch the surface.

If you have found any creative uses for BitTorrent Sync, then share with us on our forums. We’d love to hear about it.


IMG_4226
Written by: Dan

Dan is the Digital Creative Manager for BitTorrent Inc. He can usually be found writing blog posts, updating BitTorrent Labs, or browsing Reddit when he thinks no one is looking. He likes working at BitTorrent because they let him keep his Star Trek bobble heads on his desk.

 Related Posts:
  • Colin Poly

    First off this is awesome, great guide start to finish, unfortunately though Im having trouble at the very end and I cannot figure out what to do to make it work. Ill try and lay out my steps in case anyone can help.

    After the following finished:
    cd OwncloudPie
    chmod +x owncloudpie_setup.sh
    sudo ./owncloudpie_setup.sh

    It gives you some options which may have been the cause of my mistake but not sure. The options were something like this:
    1. Enter webserver or IP address
    2. Install yada (Nginx)
    3. Install yada (Apache)
    4. Dont remember
    5. Dont remember

    I wasnt sure what to do so I chose (1) and entered 192.168.1.131 (IP of my rasPi)…Then I chose option (2) and installed.

    Then when I went to set up owncloud for the first time I think I got an error for something maybe permissions but then the browser got stuck and when I refreshed it it was telling me permission problems writing to /media/SYNC.

    It also suggested I use the following command:
    (Specifically it had something along the lines of /path/to/your/installed/files in the place of /media/SYNC)
    chown -R www-data:www-data /media/SYNC

    Okay this is where my memory is fuzzy because I pretty much rebooted and tried the following 3 commands in various orders:

    sudo chmod -R 770 /media/SYNC
    sudo usermod -a -G www-data sync
    sudo chown -R www-data:www-data /media/SYNC

    until finally 192.168.1.131/owncloud started working and stayed working

    So now to the problem….When I go to 192.168.1.131:8888/gui everything seems okay but when I try and add the /media/SYNC/admin/files folder it says:
    Dont have permissions to write to selected folder.

    Note I am doing all of this from ssh on my mac if that matters (I lent out my mouse lol), and when I try to cd to /media/SYNC I can only get to media without permission being denied. Ive tried going through the steps again and no matter what I cannot get that folder added successfully with Secret in my browser. I am so close but to be honest all this command line stuff is a bit out of my league. Let me know if there is anything I can do asides from starting the process over. I would greatly appreciate any insight. Thank you in advance.

    ~Colin

    • Garras

      I had the same issue,
      first i tried to run btsync from sudo, but it seemed that the files created from bitorrentsync where unreadable by owncloud, then i tried to add my user (pi) to www-data group and run btsync from the normal user pi and now it seemes to work.
      btw the command is:

      sudo adduser pi www-data

      I don’t know how much can be secure, though

    • Garras

      try:
      sudo adduser pi www-data

      (pi is your default user, the one who runs bysync)

    • Daniel R.

      I had the exact problem as you.
      I don’t know if I fixed it correctly(did all of this in SSH too :P), but I did

      “sudo chown -R pi /media/SYNC”
      and
      “sudo chmod -R 770 /media/SYNC”
      to give myself write permissions. And I was able to create the folder in the BitTorrent Sync :).

      I’m a Linux noob and I just googled the problem, so I’m not 100% I fixed the problem the correct way.

    • Ram

      Hi Colin,
      I’m stuck exactly at the same place. Did you manage to find a way around?

    • Clive

      I am having the same issue – anyone got any ideas?

    • Will

      I’m figuring this out as I go along but try running bitsync as root:

      (if it’s already running: sudo killall btsync)

      sudo ./btsync

  • Ron

    Me novice in all things. It works fine, no issue. Super easy spot on. Should caution be advised regarding it’s use as a primary store? Might a fault among synched files might be seen as an update lowering the aggregate reliability? The onetime keys could extend the utility if there was an API to add and remove files and directories. And where does the revenue coming from?

  • Darkguy

    Thanks! At http://forum.bittorrent.com/topic/18974-debian-and-ubuntu-server-packages-for-bittorrent-sync/ there is now also the info on how to add a repository and install BTSync via apt-get. This way, there is also a daemon that auto-starts for headless operations.

  • Tom

    Whats the point of using bittorent sync if you’re already using owncloud since owncloud already syncs across multiple devices?

    • Bjorn

      If he considers his LAN safe and uses ownCloud “at home”, but uses bittorent sync to sync beyond the LAN he saves the hassle to setup the SSL for his owncloud AND the issue if the IP changes his TCP/IP address.

  • Michal

    Hello,

    thanks for the post, nice to see what is possible on the little Pi.
    Could you comment on speed, esp. when accessing the Owncloud? And what would you expect to eventually become a bottleneck, with many files in the cloud: the CPU, or the USB (including the ethernet, which is also using USB, as far as I remember)?

    Thanks.

    Michal

  • http://www.kbve.com/c/ h0ly

    This was an amazing guide! This was the first time I had heard of OwnCloud, I am really excited to drop my dropbox :P and get going with an open source cloud. However, can you setup multiple “clouds” and create our own personal network? *Allowing friends to access certain folders or to share certain folder?*

    • Joe

      There is a way within the Owncloud WebUI to set up users and give them permissions although I am a bit fuzzy on exactly how to do so.

  • gabor

    So when there will be a btsync android app than I won’t need owncloud?

  • barberik

    @Colin Poly – I believe the problem is that btsync is running and writing the files as the root user but owncloud needs them to be owned by the www-admin user. I’m assuming that this is what the usermod command is for but it doesn’t seem to be working for me.

    @DAN – Do we need to run btsync as www-admin? How would one go about that?

  • Keamas

    Hi what exactly is the advantage of combining OwnCloud with BitTorrent Sync?
    Is it only to access the data by mobile phone and Web GUI?

    Isn’t it a problem to combine this two sync programs ? Sync conflicts and so on?
    If you sync big files with BitTorrent Sync like a 4 GB file can OwnCloud handle this?
    Owncloud is for smaller files and BitTorrent Sync for bigger files right?

  • Ben

    This destroyed my nicely working raspbian install … :(
    The owncloud install script requires root acces. Once granted, it will install and update many things without caring much for dependencies.

    At the end, my “pulseaudio” package is completely broken (what has pulseaudio to do with owncloud, really?), I cannot reinstall it, I cannot remove it. I used it to play music with MPD.
    Moreover, apache still doesn’t want to start. So no owncloud.

    I’ll suggest to dig more into what the script is actually doing before granting it root access.

  • http://www.biginjapan.co Dave

    Sorry if this is sounding dumb but I just want to be clear as I’m about to setup OwnCloud myself. So the BitTorrent Sync is used for synchronization between machines themselves, and you’ve set up OwnCloud to offer self-run cloud based services to yourself and anyone else you would share files with.

    Am I along the right lines with that?

  • http://linlunson.co Linlunson

    Hi, this is a very awesome guide! I have installed the owncloud and everything works!

    But when I finished install the owncloud and start to set the data folder, there was an error message “Data directory (/media/SYNC) not writable by ownCloud. Permissions can usually be fixed by giving the webserver write access to the ownCloud directory”. I searched the internet and found the solution: sudo chown -R www-data:www-data /media/SYNC/

    By the way, is that possible to translate this great guide to Chinese? I want to translate and share this guide with some famous linux forums in China. If I get the permission and finish the translate, I will add a link to this post :D

  • http://about.me/hongster Leong Hean Hong

    Thank you for writing this post, i think it is a very useful project.

    # ownCloud Synchronization Feature
    ownCloud comes with file synchronization feature, and it has Window/Mac/Linux/Android/iOS clients. Wouldn’t BitTorrent Sync be redundant? If the purpose of this project is to synchronize files across machines, then having ownCloud is enough, and ownCloud provides more features then BitTorrent Sync. Correct me if I am wrong ;)

    # Permission
    BitTorrent Syn is executed by the user, and ownCloud is ran by www-data. I foresee ownCloud may not be able to overwrite files created by BitTorrent Sync due to permission issue. This might be what @Colin experienced.

  • Kostas

    Very nice guide..

    I want to ask what is the advantage of using BitTorrent Sync as (if I am not missing something) .. the folder synchronisation can be handled by OwnCloud.

    so is it nessesary to use BitTorrent Sync for this project and what is it’s advantage /additional value

    • Brian

      I have not done this install but I can explain the various methods of shares & sync.
      Firstly, I use Linux on all my computers so I can’t comment on Windows/Mac/BSD.
      It is possible to do Samba shares on the server. This means that directories can be mounted on a desktop computer which are actually Samba shares but look like they are just ordinary directories on the desktop computer. So, anything saved to the Samba shares are actually stored on the server. For example, when I save a download to my Download directory it looks like it is on my desktop, but it is actually stored on my server. Shares can be made read only also. You may wish to do this with your photo collection to guard against accidental deletion.
      BitTorrentSync(BTS): This can be set to read/write or read only. It syncs all devices that are connected to the sync. It is very different to a share as described above because it copies files from/to devices. Whereas shares are not copied. Any computers using this use the same BTS program on each computer. There are also Apps for Android & iOS. I find that BTS works on both WiFi and the mobile network without any special setup. I don’t leave the BTS application on all the time, on my phone, because it is a battery eater.
      I have not looked to see what kind of sync OpenCloud uses but I find BTS works very well. It is ideal for syncing Tomboy notes from my Linux server to my Android phone using Tomdroid.
      I hope that this answers some questions.

  • AK

    Is it possible to use an external hard drive that is not self powered ?

    • Ace Smith

      yes as long long as you know the directory of the drive. it may be different.

  • http://ho.race.hk/ Horace Ho

    The BT Sync take ~90% of CPU … =_=!

    2044 root 20 0 72660 9.9m 3052 S 88.4 2.3 11:53.70 btsync
    2292 root 20 0 0 0 0 S 1.6 0.0 0:15.32 cifsd
    2319 pi 20 0 4668 1388 1024 R 1.0 0.3 0:03.42 top

  • http://ho.race.hk/ Horace Ho

    The maximum speed of my Pi can get is about 1.8 MB/s down. On the same network, my MacBook Air is BT Sync at 9 to 10 MB/s down.

  • http://vile.devlix.dk Jimmy Selgen Nielsen

    I took your guide a step further, and added some paranoia :-)

    I linked to your guide for setting up BTSync and OwnCloud.

    http://vile.devlix.dk/blog/2013/07/15/raspberry-pi-setup-pcextreme-dot-nl/

  • Adam

    Your OwnCloud section should read “Install Git:” not “Install Github:”
    Nice job on the guide.

  • Joao Medrado

    Great post!
    I just didn’t get one thing: if owncloud provide sync clients why use btsync + owncloud? Wouldn’t just Owncloud do?

  • http://jmoey.zxq.net Joshun

    Mostly a great tutorial, though I would strongly advice against leaving xorg running all the time (the raspberry pi’s resources are stretched enough). You would be better off creating an initscript for bittorrent sync or setting it to start automatically in /etc/rc.local.

  • Nick

    I’m confused, isn’t there are a redundancy in setting up two remote services for file access? I realise they are aimed at performing different tasks (owncloud provides more integrated “cloud” services, bittorent sync convenient and secure – encrypted – file synchronization), doesn’t running owncloud (with a static IP) remove any benefits that sync provices?

  • http://www.pictureandword.co.uk Nathan

    Another solution might be to get your router to serve up a static IP address instead of the (probably default DHCP) random assigning of IPs.

  • Errol

    After I set up a cloud this way can I bring down the files to any device by just the ip address or do I need the bit sync app or any other program?

  • olo

    I’m trying to make this work on raspbmc but after installing Owncloud I get 403 Forbidden error when trying to access https://raspbmc/owncloud. In error.log I can find – directory index of “/var/www/owncloud/” is forbidden, client: 192.168.*.*, server: raspbmc, request: “GET /owncloud/ HTTP/1.1″, host: “raspbmc “.

    Does anyone know how to fix it? I know nothing about nginx and linux.

  • Fdasho

    Dan
    What is the read and write speed using btsync between a pi and a laptop (omitting OwnCloud).
    Also, what is the average CPU utilisation on the pi during both the read and write.
    I would like to see if I should remove Samba and install btsync instead.

  • Alex J.

    The reason for setting up both BTSync and owncloud is to give yourself the pretty web interface+outside service integration capabilities of owncloud. BTSync is meant to not need a central server like owncloud (where if the central server goes down you can’t sync anymore) but you don’t get the plethora of web features that owncloud brings. So this is cool. I wonder if you could do something similar with AjaXplorer which has a very beautiful interface. Or maybe just go DIY the web interface (maybe django?).

  • Roger B

    ok super duper newbie here, how do I navigate to ~/.config/autostart to save the text file I created in leaf? thanks

  • Alex J.

    Roger,
    Use the command cp. So you would use cp source_path destination_path.
    Ex. If I saved my file as ~/Desktop/file.txt, I would use the following command:
    cp ~/Desktop/file.txt ~/.config/autostart/another_name.txt

    And that copies the file to ~/.config/autostart with the another_name.txt

  • JDUZZ

    Great guide! I had the problem with btsync not being able to gain proper permissions as well. Theres plenty of information from ubuntu forums.

    http://askubuntu.com/questions/74806/how-can-i-change-permissions-on-external-drives

    Read through this post gives you great information about fixing them

  • Greg

    how do I get my pi to have a static IP address? I can’t figure out what I’m supposed to do for that last step?

  • Tobias

    Hi,

    Thanks for a great guide.

    I open up the owncloud gui to set up the service (first time) and change the sync location to my path.

    After all this I get: “Failed to connect to database”.

    Being a nowbie, please advice ;)

    /Tobias

  • Drew

    You can simplify the process somewhat by using TeraTerm, or your favorite remote terminal app on whatever OS your main machine uses. Make an SSH connection with your pi and you’ll get a console window on your main box that lets you do all of the above without needing to bother connecting anything to the pi other than the network.

  • Brian

    Hello, thanks for this tutorial, I just set up a personal cloud with it, excited to start using it!. My issue is that it seems Owncloud does not have permission to delete, version, or rename any file/folder that it did not upload. For example, I uploaded a file within owncloud itself, which has versioning, as well as options to rename, download, share, and delete. A file uploaded by bitsync, however, within the owncloud web UI, I can only download and share. I cannot delete, and most importantly, it does not keep track of versions. Is this because of incorrect permissions, since bitsync is not in owncloud’s usergroup? Does anyone know a way to go about fixing this? I want to enable versioning on files that were uploaded with bitsync pretty much. Thank you.

  • http://marcandreuf.blogspot.co.uk Marc

    Hello all,

    Great post, many thanks Dan, really good to follow those steps. However I had some issues with file permissions and I have seen that I am not the only one. For that I wrote a solution on my blog to provide a full description with all details.

    See here http://marcandreuf.blogspot.co.uk/2014/01/sorting-out-permissions-for-raspberrypi.html

    I hope it helps, best regards.

  • Paul

    Hi,

    thanks fot the great post!

    Does anyone have an idea
    on how to update bittorrent sync on the Raspberry? On my Ubuntu computer
    I have a higher version than on my Raspberry. It says it is up to date,
    but is it true? Can I simply donwload the newer file and replace the
    old one?
    Or maybe add a repository, so this will happen automaticly in the future?

    Thanks,
    Paul

  • Joshua Bamford

    Fantastic guide, thank you!

  • Torbjørn Strøm Andersen

    When running “cd ~/.config/” I get the message: “-bash: cd: /home/pi/.config/: No such file or directory” Can anyone point me in the direction of where that .config-file is located? I have a fresh install of the OS with apt-get update and upgrade. Last step was to run the program, si i guess that is all set: “BitTorrent Sync is already running with pid 2163″

  • shoban

    Thank you for helping me build this :)

    • Karthikeyan

      Shoban, you’re more awesome than i thought. Great to see you here.

  • marcovs

    Sorry i Can’t understand: why do i need to use bit torrent sync? Is not owncloud enough? : Thanks

  • Meap

    could i be able to start btsync as a service so i could run the pi headless?

  • Andrew Hansen

    I hope someone can help me, I followed this step by step but when I try to login to owncloud I get Data directory (/media/SYNC) not writable by ownCloud

    Permissions can usually be fixed by giving the webserver write access to the root directory. How can I fix this, what have I done wrong