在 Google Cloud Vertex AI 中为具有单标签图像的数据集创建导入文件

Question

I have a bucket in GCS with the following hierarchy:

dataset/class1/image1.png 
               image2.png 
 ..
dataset/class2/image1.png 
               image2.png 
  ..
dataset/class3/image1.png 
               image2.png 
  ..

So, all the examples for the same class are in the same folder.

I would like to create an import file that for each images creates a new line with the URI and the class. It would look like this:

gs://dataset/class1/image1.png, class1 
gs://dataset/class1/image2.png, class1
 ..
gs://dataset/class2/image1.png, class2
gs://dataset/class2/image2.png, class2
  ..
gs://dataset/class3/image1.png, class3
gs://dataset/class3/image2.png, class3
  ..

I am trying this but it doesn't work

export BUCKET=<bucket name>
export IMPORT_DATA=<import file>

gsutil ls -r gs://$BUCKET/** > $IMPORT_DATA
sed -i '1d' $IMPORT_DATA
sed -e 's/$/$(basename $)/' -i filename

Answer 1

I might have found one way to do it.

export BUCKET=<bucket name>
export IMPORT_DATA=<import file>

gsutil ls -r gs://$BUCKET/** > tmp.csv
sed -i '1d' tmp.csv  # the first line is not a file
cat tmp.csv | while read line ; do echo $line ',' $(basename $(dirname $line)) ; done > $IMPORT_DATA

wc -l $IMPORT_DATA
rm tmp.csv