Shell Scripts November 23, 2015 3

Shell script: Copy files from source to target directory

Simple shell script to search and copy files from one folder to another.

Pretty useful if there is a need to copy/move files from one folder to another (e.g., Dropbox) in case that running program cannot be configured to write output files to multiple locations. The existing files in the destination folder are skipped. However bear in mind that this script doesn’t cover currently opened files, or it will not transfer the files again if the size of the source file has changed – for such scenario it would be useful to run rsync utility.

#!/bin/sh
#This script will search for files defined by SUFFIX_ARG in SPATH
#directory and copy them to DPATH directory
#The files that already exist in destination will be skipped

#Script Variables
SPATH="/home/user/source"
DPATH="/home/user/target"
SUFFIX_ARG="*.jpeg"
ACTION_TIME=`date +"[%d-%m-%y %T]:"`
COPY_LOG=COPY-`date +"%d-%m-%y_%T"`.log

echo "${ACTION_TIME} Starting copy task" >> ${COPY_LOG}
find ${SPATH} -type f -name "${SUFFIX_ARG}" -print | while read path
do
   FN="${path##*/}" #extract the file name from the path
   ACTION_TIME=`date +"[%d-%m-%y %T]:"`
  
   if [ -e "${DPATH}/${FN}" ]   #if the destination file exists, skip it
   then
      echo "${ACTION_TIME} Skipped: File ${DPATH}/${FN} already exist" >> ${COPY_LOG}
   else
       echo "${ACTION_TIME} Trying to copy ${FN}" >> ${COPY_LOG}
	   cp "$path" "${DPATH}/${FN}" >> ${COPY_LOG}
	   ACTION_TIME=`date +"[%d-%m-%y %T]:"`
	   echo "${ACTION_TIME} Copy ${FN} to ${DPATH} Complete" >> ${COPY_LOG}
   fi
done

As an alternative we could use rsync utility in order to synchronize multiple folders in the system. This is useful when we run rsync and there are still opened files. Once the particular file has been written, next rsync run will also synchronize this file, regardless whether it exists in the target folder or not.

rsync -q -axr --delete --exclude "DIR1" --exclude "DIR2" --include "*/" --include="*.jpeg" --exclude "*" /home/user/source /home/user/target

What we do in rsync is the following:

-q, --quiet		suppress non-error messages
-a, --archive		archive mode
-x, --one-file-system	don't cross filesystem boundaries
-r, --recursive		recurse into directories
--delete		delete extraneous files from destination
--exclude=PATTERN	exclude files matching PATTERN
--include=PATTERN	don't exclude files matching PATTERN

Cheers!!