How to backup and sync your rooted Android phone automatically

1 03 2012

I have for a while now been working on a custom automatic sync solution for my HTC Hero running Froyo (Froydvillain ROM). As I am a Linux junkie and love scripts and hacks I wanted to do it all via cunning hacks and I’ve finally got it nailed.

This solution uses Scripting Layer 4 Android (SL4A) and Tasker alongside a custom ROM with rsync (any ROM should do so long as it has rsync). For those that don’t know, rsync is an awesome application that allows for remote backup and sync across machines. It turns out you don’t even need a ROM with rsync built in, as you can install an app that provides rsync, the app is called rsync backup for android and can be found here: https://market.android.com/details?id=eu.kowalczuk.rsync4android&hl=en

The only issue is you can’t call rsync from the command line using simply “rsync” since it isn’t in your systems path. However, if you use the following string instead,replacing calls to “rsync” with the following, the scripts still work: /data/data/eu.kowalczuk.rsync4android/files/rsync

SL4A is used to set out what to do via a script. You can write scripts in various languages in SL4A but I am using Bash as I am familiar with it. Rsync is used to actually handle the sync / backup and Tasker is used to launch the scripts when certain conditions are met.

I have created two scripts in SL4A, one backups my photos folder to my main photo folder on my server. The server runs the rsync daemon which rsync on the phone connects to. The other script does the reverse and copies a remote folder in my server that contains a bunch of music to my phone.

Tasker is set up with a profile that activates when my phone is plugged in and it’s between midnight and 7.00 am. This then connects to my WiFi network and then runs the two scripts via the SL4A plugin. Since I charge my phone each night this is effectively automatic.

The key here is getting permissions correct with rsync during the file transfer, as the memory card uses fat32 it hasn’t got any permissions. The rsync daemon doesn’t like this and errors out, hence the need for various settings. The second key here is exporting your password as an environmental variable. This is inherently insecure but since my server has multiple redundant backups and is only locally accessible I don’t care much. I could use trusted keys but I’m too lazy.

Here are the two scripts. First the music script that syncs from server to phone:

#rsync sync
export RSYNC_PASSWORD=password
DATE=$(date)
LOG=/mnt/sdcard/rsyncmusic.txt
echo rsync started $DATE > $LOG
TRY=1
rsync_com ()
{
DATE=$(date)
if [ $TRY = 15 ]; then
echo rsync failed, quitting on $DATE >> $LOG
exit
fi
sleep 10
echo rsync attempt $TRY started $DATE >> $LOG
rsync --progress -vHrltD --chmod=Du+rwx,go-rwx,Fu+rw,go-rw --no-perms --stats --password-file=/mnt/sdcard/scrt prupert@prupert::amusic /mnt/sdcard/amusic >> $LOG 2>&1
EXIT=$?
TRY=`expr $TRY + 1`
echo exit code is $EXIT >> $LOG
echo "********************" >> $LOG
}
rsync_com
while [ $EXIT != 0 ]; do
rsync_com
done
echo rsync finished $DATE >> $LOG
exit

The second script syncs the phones photos folder to my server:

#rsync sync photo
export RSYNC_PASSWORD=password
DATE=$(date)
LOG=/mnt/sdcard/rsyncphoto.txt
echo rsyncphoto started $DATE > $LOG
TRY=1
rsync_com ()
{
DATE=$(date)
if [ $TRY = 15 ]; then
echo rsync failed, quitting on $DATE >> $LOG
exit
fi
sleep 10
echo rsync attempt $TRY started $DATE >> $LOG
rsync -vHrltD --chmod=Du+rwx,go-rwx,Fu+rw,go-rw --no-perms --stats --password-file /mnt/sdcard/scrt /mnt/sdcard/DCIM prupert@prupert::apics >> $LOG 2>&1
EXIT=$?
TRY=`expr $TRY + 1`
echo exit code is $EXIT >> $LOG
echo "********************" >> $LOG
}
rsync_com
while [ $EXIT != 0 ]; do
rsync_com
done
echo rsync finished $DATE >> $LOG
exit

I have put some logging in to check progress and also some retry code that retries the sync if it timesout. It seems my HTC Hero’s WiFi connection claps out after a while so the script retries up to 15 times to run successfully based on the rsync exit code.





get_iplayer startup script

19 02 2012

For funsies I am looking to create a web pvr, using get_iplayer, sickbeard and sabnzbd+ (though I am planning to stay on the legal side and only download what I can watch for free anyway).

Since there is no startup script for the get_iplayer web pvr program, I created one. You can find it here:

https://github.com/pruperting/code_snippets/blob/master/get_iplayer





Not so long and thanks for all the clicks

4 02 2012

Edit, I’ve transferred over to wordpress.com, so the site stays, it’ll just stagnate for a while….

So, clearly, I have let this blog fall by the wayside. I haven’t written a new post since April 2011. Bad bad me.

As it’s time to renew my hosting, I have decided not to renew and to let this blog expire, rather than just let it attrite, as so many sites do. I use to have a lot more time, but now as I have a reasonably secure job (that I really enjoy), a wife, a lovely 2 year old and another on the way, I have no time to wax lyrical on tech any more.

You will be able to find me here:
http://www.rupert-plumridge.co.uk/
and my tech writings / scripts will move over to here: http://github.com/pruperting

So, it leaves to me say thanks to everyone who read and commented, it’s been fun.





A Better FFmpeg Progress Script

7 04 2011

So, a while ago I wrote an FFmpeg progress script. It worked, but it wasn’t brilliant and it fell apart on occassion.

After getting so annoyed by trying to get it work, I gave up. However, having a need to encode some stuff with FFmpeg again, I decided to return to the script. In the interveening period, my post on the Ubuntu forums, apart from getting one of the best response ever (see the matrix comment) had also encouraged others to give their solutions.

I thus used a combination of these ideas to create my updated version, which is below. It seems to be slightly more robust and still give the same useful info. I will be using this version for now, so I  hope it is of use for others ;)

Creative Commons License
YAFF by Rupert Plumridge is licensed under a Creative Commons Attribution-Non-Commercial-Share Alike 2.0 UK: England & Wales License





XBMC Ubuntu Minimal Maverick Install Script

1 01 2011

As a little New Year’s present, I have updated a script that has been knocking around to allow you to install XBMC on a minimal version of Ubuntu Maverick.

If you simply install Ubuntu from the mini.iso containing the basic Ubuntu Maverick install, then on reboot, download my script using:

wget www.prupert.co.uk/scripts/xbmcmaverickrevo.sh

make it executable, using:

chmod a+x xbmcmaverickrevo.sh

then finally run it using:

sudo ./xbmcmaverickrevo.sh

I find it is best to run it again, once the script reboots the machine, as this fixes a few issues.

It is still a work in progress, as you can’t yet shutdown / restart etc via XBMC since it seems the method to add permissions in Ubuntu 10.10 have changed and they conflict with the setup here, so I am still looking into that. To shutdown / reboot at the moment, I log in via SSH and issue:

sudo shutdown -hP now

Also, it is designed to work just with the Acer Aspire Revo 3610, but there is no reason why it wont work with any other device (apart from maybe the audio configurations).

Enjoy.

Download It Here





Install the Latest Version of LCD4LINUX on Ubuntu

12 08 2010

Update: It seems that I need to add some additional steps when installing on a completely vanilla version of Ubuntu. So I have added all the steps that should now be required.

Despite the most recent version (0.11.0) of lcd4linux being released in November 2009, Ubuntu Lucid comes with an older version and thus misses out on some new features and drivers, in particular drivers for the cool and cheap Pertelian X2040.

It is however, possible to build the latest version of lcd4linux from source, but you need to do a few things in order to get it to work.

This little guide assumes you have built stuff before on Ubuntu, so have all the necessary build tools.

First, navigate to the folder you want to download and build the source in.

Then, download the latest version via svn:

svn co https://ssl.bulix.org/svn/lcd4linux/trunk lcd4linux

Next, navigate into the newly created source folder using cd lcd4linux and install some dependencies:

sudo apt-get install automake autoconf m4 perl libtool gettext

Then, run ./configure, if you want to include a specific driver, for instance for Pertelian, include it here, like this:
./configure --with-drivers=Pertelian
Now, if you try to make, it wont work, I am not sure why, but there are some issues. To fix these, issue the two following commands:
mkdir m4
sudo ln -sf /usr/share/libtool/config/ltmain.sh .
Now, you can run:
make
sudo make install
And you now have a working and up-to-date lcd4linux. I’d recommend using checkinstall rather than make install, because you can uninstall the package at a later date if you want.

Doing all this allows me to run my Pertelan display on my little thin client that I use as a baby monitor. It now tells me the PIDs of FFmpeg and motion (so that I know that they are working and running), that the Internet connection is UP and the system load, alongside the date and time!





Backup All your Files Using Google Docs

10 08 2010

Google Docs recently upgraded to allow you to upload any file to Google Docs. This very handy feature means, if you purchase a little extra storage from Google, you can use Google Docs as your personal online backup service. Google offer a variety of storage plans:

20 GB ($5.00 USD per year)
80 GB ($20.00 USD per year)
200 GB ($50.00 USD per year)
400 GB ($100.00 USD per year)
1 TB ($256.00 USD per year)
2 TB ($512.00 USD per year)
4 TB ($1,024.00 USD per year)
8 TB ($2,048.00 USD per year)
16 TB ($4,096.00 USD per year)

From what I remember when I looked in to it, these prices are pretty good compared to dedicated online backup services (though anyone who needs and can afford the 16TB option needs their head examined!). The only issue is how to get all your files into Google Docs. Well, if your Command Line-Fu is strong, you could use Google CL, but it doesn’t work with a batch of files, unless you write some cunning script. A much easier alternative is to use software from Gladinet, in particular their Gladinet Cloud Desktop tool. This allows you to define various backup tasks, allowing you to sync folders and their subfolders to a folder in Google Docs (or Picasa or a number of other online sources). All your folders and subfolders are backed up to Google Docs, with the folder structure being replicated in Google Docs. You can set these tasks to run every day and it should only update new and changed files. I have been using it for the past month or so and it seems to work fairly well. The only real issue is due to slow upload speeds, but I am not sure whether this is an issue with my broadband provider or a limitation of the Google Docs servers. Either way, the results are that all my files are backed up to Google Docs automatically.








Follow

Get every new post delivered to your Inbox.