简体   繁体   中英

Download files with differents extensions with bash and wget

I'm trying to download images with differents extensions with bash and wget . I have to target the files directly because i have a 403 error on the parents folders, so i can't use the -A option by targeting the parent folder.

For now i have the following code which works fine for the .jgp but if the next file is a .png for example my loop breaks and continue to the next folder so the .png in the current folder are not downloaded.

for i in {1..945}; do
    for j in {01..100}; do
        j=`printf '%02d' $j`
        ret=$(wget -O ch${i}-${j}.jpg https://www.domain.co/uploads/chapters/chapitre-${i}/${j}.jpg 2>&1)
        if [[ "$ret" =~ 404\ Not\ Found ]]; then
            break
        fi
    done
done

I'm having trouble finding a way to download the file regardless of its extension.

EDIT:

So here's my final snippet for those who are interested.
What it does is:

  • recursively crawl a folder and its content
  • check if the file is a .jpg or a .png and then rename it accordingly
  • goes back at the previous loop (parent folder) if 404
  • delete empty files created with 404


  • for i in {245..945}; do
        for j in {01..60}; do
            j=`printf '%02d' $j`
            ret=$(wget -O ch${i}-${j}.jpg https://www.yourdomain.co/uploads/parentfoldernumber-${i}/image-${j}.jpg 2>&1)
            if [[ $? -ne 0 ]]; then 
                ret=$(wget -O ch${i}-${j}.png https://www.yourdomain.co/uploads/parentfoldernumber-${i}/image-${j}.png 2>&1) 
            fi
            if [[ "$ret" =~ 404\ Not\ Found ]]; then
                break
            fi
        done
        find . -type f -empty -delete
    done
    

    I tried something, Maybe this will help you,

    $? is command of bash, which returns the output of last command ran.

    If *.png is not found, the output won't be 0. You could case it with that,

    if [[ $? -ne 0 ]]; then ret=$(wget -O ch${i}-${j}.png http://localhost/l4urstyle/wp-content/uploads/2018/${i}/${j}.png 2>&1) fi

        if [[ "$ret" =~ 404\ Not\ Found ]]; then
           echo "breaking code"
            break
        fi
    

    Hope this helps you. If this worked. Please accept the answer. :D And about file being downloaded of 0 octet size. I will think of something and let you know how i can help you with that.

    Happy Coding :D

    The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

     
    粤ICP备18138465号  © 2020-2024 STACKOOM.COM