[英]Split a folder into multiple subfolders in terminal/bash script
I have several folders, each with between 15,000 and 40,000 photos.我有几个文件夹,每个文件夹包含 15,000 到 40,000 张照片。 I want each of these to be split into sub folders - each with 2,000 files in them.我希望将其中的每一个都拆分为子文件夹 - 每个文件夹中有 2,000 个文件。
What is a quick way to do this that will create each folder I need on the go and move all the files?有什么快速的方法可以在 go 上创建我需要的每个文件夹并移动所有文件?
Currently I can only find how to move the first x items in a folder into a pre-existing directory.目前我只能找到如何将文件夹中的前 x 个项目移动到预先存在的目录中。 In order to use this on a folder with 20,000 items... I would need to create 10 folders manually, and run the command 10 times.为了在包含 20,000 个项目的文件夹上使用它...我需要手动创建 10 个文件夹,并运行该命令 10 次。
ls -1 | sort -n | head -2000| xargs -i mv "{}" /folder/
I tried putting it in a for-loop, but am having trouble getting it to make folders properly with mkdir.我尝试将其放入 for 循环中,但无法使用 mkdir 正确创建文件夹。 Even after I get around that, I need the program to only create folders for every 20th file (start of a new group).即使解决了这个问题,我也需要该程序只为每 20 个文件创建文件夹(新组的开始)。 It wants to make a new folder for each file.它想为每个文件创建一个新文件夹。
So... how can I easily move a large number of files into folders of an arbitrary number of files in each one?那么......我怎样才能轻松地将大量文件移动到每个文件夹中任意数量的文件中?
Any help would be very... well... helpful!任何帮助都会非常……嗯……有帮助!
Try something like this: 尝试这样的事情:
for i in `seq 1 20`; do mkdir -p "folder$i"; find . -type f -maxdepth 1 | head -n 2000 | xargs -i mv "{}" "folder$i"; done
Full script version: 完整脚本版本:
#!/bin/bash
dir_size=2000
dir_name="folder"
n=$((`find . -maxdepth 1 -type f | wc -l`/$dir_size+1))
for i in `seq 1 $n`;
do
mkdir -p "$dir_name$i";
find . -maxdepth 1 -type f | head -n $dir_size | xargs -i mv "{}" "$dir_name$i"
done
vim split_files.sh
创建一个新文件: vim split_files.sh
dir_size
and dir_name
values to match your desires 更新dir_size
和dir_name
值,以匹配你的欲望
dir_name
will have a number appended 请注意, dir_name
将附加一个数字 cd my_folder
导航到所需的文件夹: cd my_folder
sh ../split_files.sh
运行脚本: sh ../split_files.sh
The code below assumes that the filenames do not contain linefeeds, spaces, tabs, single quotes, double quotes, or backslashes, and that filenames do not start with a dash. 下面的代码假定文件名不包含换行符,空格,制表符,单引号,双引号或反斜杠,并且文件名不以短划线开头。 It also assumes that IFS
has not been changed, because it uses while read
instead of while IFS= read
, and because variables are not quoted. 它还假设IFS
没有被更改,因为它使用while read
不是while IFS= read
,并且因为变量没有被引用。 Add setopt shwordsplit
in Zsh. 在Zsh中添加setopt shwordsplit
。
i=1;while read l;do mkdir $i;mv $l $((i++));done< <(ls|xargs -n2000)
The code below assumes that filenames do not contain linefeeds and that they do not start with a dash. 下面的代码假定文件名不包含换行符,并且它们不以短划线开头。 -n2000
takes 2000 arguments at a time and {#}
is the sequence number of the job. -n2000
获取2000个参数, {#}
是作业的序列号。 Replace {#}
with '{=$_=sprintf("%04d",$job->seq())=}'
to pad numbers to four digits. 将{#}
替换为'{=$_=sprintf("%04d",$job->seq())=}'
将数字填充为四位数。
ls|parallel -n2000 mkdir {#}\;mv {} {#}
The command below assumes that filenames do not contain linefeeds. 以下命令假定文件名不包含换行符。 It uses the implementation of rename
by Aristotle Pagaltzis which is the rename
formula in Homebrew, where -p
is needed to create directories, where --stdin
is needed to get paths from STDIN, and where $N
is the number of the file. 它使用Aristotle Pagaltzis rename
的实现,这是Homebrew中的rename
公式,其中需要-p
来创建目录,其中--stdin
需要从STDIN获取路径,其中$N
是文件的编号。 In other implementations you can use $.
在其他实现中,您可以使用$.
or ++$::i
instead of $N
. 或++$::i
而不是$N
ls|rename --stdin -p 's,^,1+int(($N-1)/2000)."/",e'
I would go with something like this: 我会用这样的东西:
#!/bin/bash
# outnum generates the name of the output directory
outnum=1
# n is the number of files we have moved
n=0
# Go through all JPG files in the current directory
for f in *.jpg; do
# Create new output directory if first of new batch of 2000
if [ $n -eq 0 ]; then
outdir=folder$outnum
mkdir $outdir
((outnum++))
fi
# Move the file to the new subdirectory
mv "$f" "$outdir"
# Count how many we have moved to there
((n++))
# Start a new output directory if we have sent 2000
[ $n -eq 2000 ] && n=0
done
This solution can handle names with whitespace and wildcards and can be easily extended to support less straightforward tree structures. 此解决方案可以处理带有空格和通配符的名称,并且可以轻松扩展以支持不太直接的树结构。 It will look for files in all direct subdirectories of the working directory and sort them into new subdirectories of those. 它将在工作目录的所有直接子目录中查找文件,并将它们分类到这些子目录的新子目录中。 New directories will be named 0
, 1
, etc.: 新目录将被命名为0
, 1
,等:
#!/bin/bash
maxfilesperdir=20
# loop through all top level directories:
while IFS= read -r -d $'\0' topleveldir
do
# enter top level subdirectory:
cd "$topleveldir"
declare -i filecount=0 # number of moved files per dir
declare -i dircount=0 # number of subdirs created per top level dir
# loop through all files in that directory and below
while IFS= read -r -d $'\0' filename
do
# whenever file counter is 0, make a new dir:
if [ "$filecount" -eq 0 ]
then
mkdir "$dircount"
fi
# move the file into the current dir:
mv "$filename" "${dircount}/"
filecount+=1
# whenever our file counter reaches its maximum, reset it, and
# increase dir counter:
if [ "$filecount" -ge "$maxfilesperdir" ]
then
dircount+=1
filecount=0
fi
done < <(find -type f -print0)
# go back to top level:
cd ..
done < <(find -mindepth 1 -maxdepth 1 -type d -print0)
The find -print0
/ read
combination with process substitution has been stolen from another question . find -print0
/ read
与进程替换的组合已从另一个问题中被盗。
It should be noted that simple globbing can handle all kinds of strange directory and file names as well. 应该注意的是,简单的globbing也可以处理各种奇怪的目录和文件名。 It is however not easily extensible for multiple levels of directories. 但是,对于多级目录而言,它不易扩展。
This solution worked for me on MacOS: 这个解决方案适用于MacOS:
i=0; for f in *; do d=dir_$(printf %03d $((i/100+1))); mkdir -p $d; mv "$f" $d; let i++; done
It creates subfolders of 100 elements each. 它创建每个100个元素的子文件夹。
You'll certainly have to write a script for that. 你肯定要写一个脚本。 Hints of things to include in your script: 脚本中包含的事项提示:
First count the number of files within your source directory 首先计算源目录中的文件数
NBFiles=$(find . -type f -name *.jpg | wc -l)
Divide this count by 2000 and add 1, to determine number of directories to create 将此计数除以2000并加1,以确定要创建的目录数
NBDIR=$(( $NBFILES / 2000 + 1 ))
Finally loop through your files and move them accross the subdirs. 最后遍历您的文件并将它们移动到子目录中。 You'll have to use two imbricated loops : one to pick and create the destination directory, the other to move 2000 files in this subdir, then create next subdir and move the next 2000 files to the new one, etc... 你将不得不使用两个叠加的循环:一个用于选择和创建目标目录,另一个用于在此子目录中移动2000个文件,然后创建下一个子目录并将下一个2000个文件移动到新的等等...
The answer above is very useful, but there is a very import point in Mac(10.13.6) terminal. 上面的答案非常有用,但在Mac(10.13.6)终端中有一个非常重要的点。 Because xargs "-i" argument is not available, I have change the command from above to below. 因为xargs“-i”参数不可用,所以我从上到下更改了命令。
ls -1 | sort -n | head -2000| xargs -I '{}' mv {} /folder/
Then, I use the below shell script(reference tmp's answer) 然后,我使用下面的shell脚本(参考tmp的答案)
#!/bin/bash
dir_size=500
dir_name="folder"
n=$((`find . -maxdepth 1 -type f | wc -l`/$dir_size+1))
for i in `seq 1 $n`;
do
mkdir -p "$dir_name$i";
find . -maxdepth 1 -type f | head -n $dir_size | xargs -I '{}' mv {} "$dir_name$i"
done
This is a tweak of Mark Setchell's 这是Mark Setchell的调整
Usage: bash splitfiles.bash $PWD/directoryoffiles splitsize 用法:bash splitfiles.bash $ PWD / directoryoffiles splitsize
It doesn't require the script to be located in the same dir as the files for splitting, it will operate on all files, not just the .jpg and allows you to specify the split size as an argument. 它不需要脚本与分割文件位于同一目录中,它将对所有文件进行操作,而不仅仅是.jpg,并允许您将分割大小指定为参数。
#!/bin/bash
# outnum generates the name of the output directory
outnum=1
# n is the number of files we have moved
n=0
if [ "$#" -ne 2 ]; then
echo Wrong number of args
echo Usage: bash splitfiles.bash $PWD/directoryoffiles splitsize
exit 1
fi
# Go through all files in the specified directory
for f in $1/*; do
# Create new output directory if first of new batch
if [ $n -eq 0 ]; then
outdir=$1/$outnum
mkdir $outdir
((outnum++))
fi
# Move the file to the new subdirectory
mv "$f" "$outdir"
# Count how many we have moved to there
((n++))
# Start a new output directory if current new dir is full
[ $n -eq $2 ] && n=0
done
Can be directly run in the terminal可以直接在终端运行
i=0;
for f in *;
do
d=picture_$(printf %03d $((i/2000+1)));
mkdir -p $d;
mv "$f" $d;
let i++;
done
This script will move all files within the current directory into picture_001, picture_002... and so on.该脚本会将当前目录中的所有文件移动到picture_001、picture_002...等等。 Each newly created folder will contain 2000 files每个新创建的文件夹将包含 2000 个文件
picture_
is the folder prefix picture_
是文件夹前缀
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.