Unix Administration: 2011

Friday, June 3, 2011

Rsync To Transfer File under Linux

Transferring Files on Linux Platform (Tested with 1.5 TB of Data).
Admin Required to take Incremental Backups.
Want to keep Files Synced.

Tested on Fedora/Centos Linux Platforms (But Surely it works on all Distros)
Before Starting you should have xinetd working on your Linux Distro

Or get the Xinetd for your distro from
RPM.pbone
RPM.pbone1

How do I install Rsync?

Use one of the following commands to install Rsync.

If you are using Debian Based like Ubuntu Linux, Execute the following command

# apt-get install rsync
OR
$ sudo apt-get install rsync

If you are using Red Hat Enterprise Linux (RHEL) type the following command

# up2date rsync

If you are using CentOS/Fedora Core Linux, type the following command

# yum install rsync

Always use Rsync over ssh

Since Rsync does not provide any security while Transferring data it is recommended that you use rsync over ssh . This allows a secure remote connection.

You Can Find Some Good Example Here
Rsync Examples

Setting Up Rsync Server

You Need to Create or Edit Three File which would make you start Rsync server

/etc/rsyncd.conf
/etc/rsyncd.secrets
/etc/xinet.d/rsync

Lets Get in Details of all this 3 files
1. Rsyncd.conf


max connections = 5
log file = /var/log/rsync.log
timeout = 300

[backup]
comment = Anything You would like to Notify
path = /your/real/path/to_the_directory
read only = yes
list = yes
uid = root
gid = root
use chroot = no
auth users = pub
secrets file = /etc/rsyncd.secrets

The above Files say maximum connection to the server will be 2, log file to write for debugging and information about the process, timeout is set to be 300
The Next [backup] tells what should be the canonical name to the directory we have shared in the above case it is given by “path = /your/real/path/to_the_directory” (It is Similar to Samba if you have Used)
read only = yes Permission to the Directory Shared is Given Read
auth users = pub Only this user have the permission to view the directory.
secrets file = /etc/rsyncd.secrets contains the usernames and passwords

2. Rsyncd.secrets
Contains the Usernames and Passwords colon ” : ” seperated
pub:pub

Now the Super daemon Xinetd need to handle the Rsync daemon so


# default: off
# description: The rsync server is a good addition to an ftp server, as it \
# allows crc checksumming etc.
service rsync
{
disable = no
socket_type = stream
wait = no
user = root
server = /usr/bin/rsync
server_args = --daemon
log_on_failure += USERID
}

Just Copy the Above code where disable should be no.
Restart Xinetd
/etc/init.d/xinetd restart

Hoooraaayyyy We have setup the server now Check it from any remote machine having rsync.

rsync rsync://pub@ipaddr

where pub = username in the rsyncd.conf with the password specified in rsyncd.secrets

Note : rsyncd.secrets file should have only read permission from root.

Get percentage of Command executed

Do you want to know which command on you Linux box has executed the most and also would you like to get the Percentage. This is very Simple and can be achieved useing awk, grep. It can be acheived in many other possible ways too.

root# history | awk '{CMD[$2]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a;}' | grep -v "./" | column -c3 -s " " -t | sort -nr | nl | head -n10

OutPut:


1 133 13.3% ls
2 127 12.7% cd
3 78 7.8% vi
4 35 3.5% netstat
5 32 3.2% postfix
6 31 3.1% tail
7 30 3% mv
8 29 2.9% telnet
9 29 2.9% ll
10 23 2.3% postqueue