From Federico Cargnelutti

Introduction

FileSyncTask is a Phing extension for Unix systems which synchronizes files and directories from one location to another while minimizing data transfer. FileSyncTask can copy or display directory contents and copy files, optionally using compression and recursion.

Rather than using FTP or some other form of file transfer, FileSyncTask uses rsync to copy only the diffs of files that have actually changed. Only actual changed pieces of files are transferred, rather than the whole file, which results in transferring only a small amount of data and are very fast. FTP, for example, would transfer the entire file, even if only one byte changed. The tiny pieces of diffs are then compressed on the fly, further saving you file transfer time and reducing the load on the network.

FileSyncTask can be used to synchronize Website trees from staging to production servers and to backup key areas of the filesystems.

Features

Here are some other key features of FileSyncTask:

• Support for copying links, devices, owners, groups and permissions
• Exclude patterns from file
• Does not require root privileges
• Pipelining of file transfers to minimize latency costs

Attributes

Directory Structure

example.com
|-- build
|   |-- build.properties
|   |-- build.xml
|   |-- sync.exclude
|   `-- sync.properties
`-- public
    `-- index.php

Sync Properties File

The SSH client called by FileSyncTask uses connection settings from the sync.properties file:

#
# Base directory
#
sync.project.dir=/home/example.com

#
# Source directory
#
sync.source.projectdir=${sync.project.dir}/public

#
# Destination directory
#
sync.destination.projectdir=${sync.project.dir}

#
# Remote connection details 
#
sync.remote.host=remote-server.com
sync.remote.user=example

#
# Backup directory: Destination files are copied to a backup directory as 
# each file is transferred or deleted.
#
sync.destination.backupdir=${sync.destination.projectdir}_backup 

#
# Exclude patterns listed in a file: The filter rules allow for flexible selection of 
# which files to to exclude. As the list of files/directories to transfer is built, 
# rsync checks each name to be transferred against the list of exclude patterns in turn, 
# if it is an exclude pattern, then that file is skipped.
#
sync.exclude.file=/home/example.com/build/sync.exclude

Usage

There are 4 different ways of using FileSyncTask:

1. For copying local files.
2. For copying from the local machine to a remote machine using a remote shell program as the transport (ssh).
3. For copying from a remote machine to the local machine using a remote shell program.
4. For listing files on a remote machine.

Listing files

The "listonly" option will cause the modified files to be listed instead of transferred. You must specify a source and a destination, one of which may be remote.

<taskdef name="sync" classname="phing.tasks.ext.FileSyncTask" />
<sync
    sourcedir="${sync.source.projectdir}"
    destinationdir="${sync.destination.projectdir}"
    listonly="true" 
    verbose="true" />

Excluding irrelevant files

To exclude files from synchronizations, open and edit the sync.exclude file under the build/ directory. Each line can contain a file, a directory, or a pattern:

*~
_*
.svn
.htaccess
public/index.development.php
public/index.staging.php
public/images/uploads/*
public/favicon.ico
build/*
log/*
tmp/*

The "excludefile" option specifies the path to the sync.exclude file:

<taskdef name="sync" classname="phing.tasks.ext.FileSyncTask" />
<sync
    sourcedir="${sync.source.projectdir}"
    destinationdir="${sync.destination.projectdir}"
    listonly="true" 
    excludefile="${sync.exclude.file}" 
    verbose="true" />

Copying local files

The following task definition will transfer files from a local source to a local destination directory, excluding irrelevant files and providing a visual feedback on what's happening:

<taskdef name="sync" classname="phing.tasks.ext.FileSyncTask" />
<sync
    sourcedir="${sync.source.projectdir}"
    destinationdir="${sync.destination.projectdir}"
    excludefile="${sync.exclude.file}" 
    verbose="true" />

Copying files to a remote machine

The following task definition will transfer files from a local source to a remote destination:

<taskdef name="sync" classname="phing.tasks.ext.FileSyncTask" />
<sync
    sourcedir="${sync.source.projectdir}"
    destinationdir="${sync.remote.user}@${sync.remote.host}:${sync.destination.projectdir}"
    excludefile="${sync.exclude.file}" 
    verbose="true" />

Copying files from a remote machine

The following task definition will transfer files from a remote source to a local destination:

<taskdef name="sync" classname="phing.tasks.ext.FileSyncTask" />
<sync
    sourcedir="${sync.remote.user}@${sync.remote.host}:${sync.source.projectdir}"
    destinationdir="${sync.destination.projectdir}"
    excludefile="${sync.exclude.file}" 
    verbose="true" />

Backup files

The "backupdir" option allows you to copy destination files to a backup directory as each file is transferred or deleted:

<taskdef name="sync" classname="phing.tasks.ext.FileSyncTask" />
<sync
    sourcedir="${sync.remote.user}@${sync.remote.host}:${sync.source.projectdir}"
    destinationdir="${sync.destination.projectdir}"
    backupdir="${sync.destination.backupdir}" 
    excludefile="${sync.exclude.file}" 
    verbose="true}" />

Example

• Using FileSyncTask to synchronize files and directories

Source Code

SVN

$ svn checkout http://svn.fedecarg.com/repo/Phing/tasks/ext/

Browse

• http://svn.fedecarg.com/browser/Phing/tasks/ext