如何使用 Bash 脚本将文件随机分布在 3 个文件夹中？

Question

I have many subdirectories and files in the folder mydata/files . I want to take files and copy them randomly into 3 folders:

train
test
dev

For example, mydata/files/ss/file1.wav could be copied into train folder:

train
  file1.wav

And so on and so forth, until all files from mydata/files are copied.

How can I do it using Bash script?

Answer 1

Steps to solve this:

1. Need to gather all the files in the directory
2. Assign directories to a map
3. Generate random number for each file
4. Move the file to the corresponding directory

The script:

#!/bin/bash

original_dir=test/

## define 3 directories to copy into
# define an associative array (like a map)
declare -A target_dirs

target_dirs[0]="/path/to/train/"
target_dirs[1]="/path/to/test/"
target_dirs[2]="/path/to/dev/"

# recursively find all the files, and loop through them
find $original_dir -type f | while read -r file ; do
        # find a random number 0 - (size of target_dirs - 1)
        num=$(($RANDOM % ${#target_dirs[@]}))
        # get that index in the associative array
        target_dir=${target_dirs[$num]}
        # copy the file to that directory
        echo "Copying $file to $target_dir"
        cp $file $target_dir
done

Things you'll need to change:

1. Change the destination of the directories to match the path in your system
2. Add executable priviledges to the file so that you can run it.

chmod 744 copy_script_name
./copy_script_name

Notes:

This script should easily be extendable to any number of directories if needed (just add the new directories, and the script will adjust the random numbers.

If you need to only get the files in the current directory (not recursively), you can add -maxdepth 1 (see How to list only files and not directories of a directory Bash? ).

Was able to leverage previous bash experience plus looking at bash documentation (it's generally pretty good). If you end up writing any scripts, be very careful about spaces

Answer 2

You can create a temp file, echo your destination folder to it, then use the shuf command.

dest=$(mktemp)
echo -e "test\ndev\ntrain" >> $dest
while IFS= read -r file; do
  mv "$file" "$(shuf -n1 < $dest)/."
done < <(find mydata/files -type f 2>/dev/null)
rm -f "$dest"