簡體   English   中英

帶有多個“.”的 Python requests.get() url

[英]Python requests.get() url with multiple "."

我正在編寫一個腳本來抓取網站並下載當前的 mp3 文件,它使用帶有Python 3.6.3 BeautifulSoup將以下所有鏈接抓取到各個藝術家頁面中,但一些抓取的 URL 包含多個“.”。 當我開火時

request.get(url, header=<random header using fake_useragent>)

它不會下載文件,我該如何更正?

例如:

URL > abc.com/mp3/artist/songs/song.com.mp3 (not downloading)
URL > abc.com/mp3/artist/songs/song.mp3 (downloading) 

代碼 :

def download_mp3(url_list_file, download_dir):                                                                                                                                                                   
    with open(url_list_file, 'r', encoding='utf-8') as urls:                                                                                                                                                     

        for url in urls:                                                                                                                                                                                         
            ua = fake_useragent.UserAgent(verify_ssl=False)                                                                                                                                                      
            #header = {'User-Agent': str(ua.chrome.random)}                                                                                                                                                      
            artist_dir = url.split('/')[4]                                                                                                                                                                       
            song_name = url.split('/')[6].replace('\n', '')                                                                                                                                                      
            #corrected_url = ('path:url')                                                                                                                                                                        
            #print(corrected_url)                                                                                                                                                                                
            download_to = os.path.join(download_dir, artist_dir, song_name)                                                                                                                                      
            save_path = os.path.join(download_dir, artist_dir)                                                                                                                                                   
            #print(download_to)                                                                                                                                                                                  
            #print(save_path)                                                                                                                                                                                    
            print(url)                                                                                                                                                                                           
            if(os.path.isdir(save_path) == True):                                                                                                                                                                
                #print ('True')                                                                                                                                                                                  
                header = {'User-Agent': str(ua.random)}                                                                                                                                                          
                lower_download_to = str.lower(song_name)
                #To get correct file name 
                mp3_file_name = os.path.join(download_dir, artist_dir, lower_download_to.replace("www.", "").replace("[", "").replace("]", ""))  
                mp3_file = open(mp3_file_name, "wb")                                                                                                                                                             
                temp_file = requests.get(url, headers=header)                                                                                                                                          
                mp3_file.write(temp_file.content)                                                                                                                                                                
                time.sleep(random.randint(5,10))                                                                                                                                                                 
            else:                                                                                                                                                                                                
                #print('False')  
                #To create Artist folder names 
                os.mkdir(save_path)                                                                                                                                                                              
                header = {'User-Agent': str(ua.random)}                                                                                                                                                          
                lower_download_to = str.lower(download_to)
                #To get correct file name               
                mp3_file_name = os.path.join(download_dir, artist_dir, lower_download_to.replace("www.", "").replace("[", "").replace("]", ""))  
                mp3_file = open(mp3_file_name, "wb")                                                                                                                                                             
                temp_file = requests.get(url, headers=header)                                                                                                                                                    
                mp3_file.write(temp_file.content)                                                                                                                                                                
                time.sleep(random.randint(5,10))                                                                                                                                                                 
    urls.closed 

得到它更正替換所有“。” 在 URL 中,除了最后一個使用百分比編碼的“.mp3”

string.replace('.', "%2E", (string.count('.')-1))

暫無
暫無

聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM