Linux - Sum total of files in different directories

Question

How do I calculate the sum total size of multiple files located in different directories? I have a text file containing the full path and name of the files. I figure a simple script using while read line and du -h might do the trick...

Example of text file (new2.txt) containing list of files to sum:

/mount/st4000/media/A/amediafile.ext
/mount/st4000/media/B/amediafile.ext
/mount/st4000/media/C/amediafile.ext
/mount/st4000/media/D/amediafile.ext
/mount/st4000/media/E/amediafile.ext
/mount/st4000/media/F/amediafile.ext
/mount/st4000/media/G/amediafile.ext
/mount/st4000/media/H/amediafile.ext
/mount/st4000/media/I/amediafile.ext
/mount/st4000/media/J/amediafile.ext
/mount/st4000/media/K/amediafile.ext

Note: the folder structure is not necessarily consecutive as in A..K

Based on the suggestion from AndreaT, adapting it slightly, I tried

while read mediafile;do du -b "$mediafile"|cut -f -1>>subtotals.txt;done<new2.txt

subtotals.txt looks like

733402685
944869798
730564608
213768
13332480
366983168
6122559750
539944960
735039488
1755005744
733478912

To add all the subtotals

sum=0; while read num; do ((sum += num)); done < subtotals.txt; echo $sum

Answer 1

To sum one directory, you will have to do a while, and export the result to the parent shell. I used an echo an the subsequent eval :

    eval ' let sum=0$(
       ls -l | tail -n +2 |\
         while read perms link user uid size date day hour name ; do 
          echo -n "+$size" ; 
         done
     )'

It produces a line, directly evaluated, which looks like

let sum=0+205+1201+1201+1530+128+99

You just have to reproduce twice this command on both folders.

Answer 2

Assuming that file input is like this

/home/administrator/filesum/cliprdr.c 
/home/administrator/filesum/cliprdr.h 
/home/administrator/filesum/event.c 
/home/administrator/filesum/event.h 
/home/administrator/filesum/main.c 
/home/administrator/filesum/main.h 
/home/administrator/filesum/utils.c 
/home/administrator/filesum/utils.h

and the result of command ls -l is

-rw-r--r-- 1 administrator administrator 13452 Oct  4 17:56 cliprdr.c
-rw-r--r-- 1 administrator administrator  1240 Oct  4 17:56 cliprdr.h
-rw-r--r-- 1 administrator administrator  8141 Oct  4 17:56 event.c
-rw-r--r-- 1 administrator administrator  2164 Oct  4 17:56 event.h
-rw-r--r-- 1 administrator administrator 32403 Oct  4 17:56 main.c
-rw-r--r-- 1 administrator administrator  1074 Oct  4 17:56 main.h
-rw-r--r-- 1 administrator administrator  5452 Oct  4 17:56 utils.c
-rw-r--r-- 1 administrator administrator  1017 Oct  4 17:56 utils.h

the simplest command to run is:

cat filelist.txt | du -cb | tail -1 | cut -f -1 

with following output (in bytes)

69370

Keep in mind that du prints actual disk usage rounded up to a multiple of (usually) 4kb instead of logical file size. For small files this approximation may not be acceptable.

Answer 3

The du command doesn't have a -b option on the unix systems I have available. And there are other ways to get file size.

Assuming you like the idea of a while loop in bash, the following might work:

#!/bin/bash

case "$(uname -s)" in
  Linux)        stat_opt=(-c '%s') ;;
  *BSD|Darwin)  stat_opt=(-f '%z') ;;
  *) printf 'ERROR: I don'\''t know how to run on %s\n' "$(uname -s)" ;;
esac

declare -i total=0
declare -i count=0
declare filename

while read filename; do
  [[ -f "$filename" ]] || continue
  (( total+=$(stat "${stat_opt[@]}" "$filename") ))
  (( count++ ))
done

printf 'Total: %d bytes in %d files.\n' "$total" "$count"

This would take your list of files as stdin. You can run it in BSD unix or in Linux -- the options to the stat command (which is not internal to bash) are the bit that are platform specific.