簡體   English   中英

在 Python 中調用內部 function

[英]Calling an inner function in Python

我有這個最終的 main.py,它結合了我單獨編寫的每個 function,但我不能讓它工作,它實際上在最后返回了 Success,但它實際上什么也沒做,也不在我的本地文件夾或 MongoDB 中。 function 是這個:

def gw2_etl(url):

    def log_scrape(url):
        HEADERS = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246'}

        response = requests.get(url=url, headers=HEADERS)
        soup = BeautifulSoup(response.content, 'html.parser')

        data = soup.find_all('script')[8]
        dataString = data.text.rstrip()

        logData = re.findall(r'{.*}', dataString)

        try:
            urlLines = url.split('/')
            if len(urlLines) < 5:
                bossName = urlLines[3]
            elif len(urlLines) == 5:
                bossName = urlLines[4]
        except Exception as e:
            return 'Error' + str(e)
        
        tag = bossName.split('_')
        bossTag = tag[1]

        try:
            # Wing_1
            if bossTag == 'vg':
                pathName = 'ETL\EXTRACT_00\Web Scraping\Boss_data\Wing_1\Valley_Guardian'
            
        with open(f'{pathName}\{bossName}.json', 'w') as f:
            for line in logData:
                jsonFile = f.write(line)
                return jsonFile

        return log_scrape()
    
    def store_data(jsonFile):

        with open(jsonFile) as f:
            data = json.load(f)
        
        sp = jsonFile.split('\\')
        posSp = sp[-1]

        bossTag = posSp.split('_')
        nameTag = bossTag[1]


        if len(bossTag) > 2:
            nameTag = bossTag[1]
        elif len(bossTag) == 2:
            tagSplit = nameTag.split('.')
            nameTag = tagSplit[0]
        
        # Players Data:
        player_group = []
        player_acc = []
        player_names = []
        player_classes = []

        for player in data['players']:
            player_group.append(player['group'])
            player_acc.append(player['acc'])
            player_names.append(player['name'])
            player_classes.append(player['profession'])
        
        try:
            # Wing-1
            if nameTag == 'vg':
                # Create lists:
                player_dps1 = []
                player_dps2 = []
                player_dps3 = []

                # Phase_1
                phase1 = data['phases'][1]['dpsStats']

                phase1_time_raw = data['phases'][1]['duration']
                phase1_time = round(phase1_time_raw/1000,1)

                for dps in phase1:
                    dps1_raw = dps[0]
                    player_dps1.append(round(dps1_raw/phase1_time,2))

                # Phase_2
                phase2 = data['phases'][6]['dpsStats']

                phase2_time_raw = data['phases'][6]['duration']
                phase2_time = round(phase2_time_raw/1000,1)

                for dps in phase2:
                    dps2_raw = dps[0]
                    player_dps2.append(round(dps2_raw/phase2_time,2))

                # Phase_3
                phase3 = data['phases'][12]['dpsStats']

                phase3_time_raw = data['phases'][12]['duration']
                phase3_time = round(phase3_time_raw/1000,1)

                for dps in phase3:
                    dps3_raw = dps[0]
                    player_dps3.append(round(dps3_raw/phase3_time,2))

                stats_dict = {
                    'players':{
                        'group': player_group,
                        'account': player_acc,
                        'names': player_names,
                        'profession': player_classes,
                        'phase_1_dps': player_dps1,
                        'phase_2_dps': player_dps2,
                        'phase_3_dps': player_dps3
                    }
                }

                df = pd.DataFrame(stats_dict['players'], columns=['group','account','names','profession','phase_1_dps','phase_2_dps','phase_3_dps'])
        
                return stats_dict

        except Exception as e:
            print('Error' + str(e))
            sys.exit()
            
        # JSON generator (MongoDB)
        pathName = 'ETL\TRANSFORM_01\Players_info'

        jsonString = json.dumps(stats_dict)
        with open(f"{pathName}\{nameTag}_player_stats.json", 'w') as f:
            f.write(jsonString)
        
        # CSV generator (MySQL, PostgreSQL)
        
        df.to_csv(f"{pathName}\{nameTag}_player_stats.csv",index=True)

        return store_data()
    
    def mongo_connect(stats_dict):
        try:
            client = pymongo.MongoClient('mongodb://localhost:27017/')
        except Exception as e:
            print('Connection could not be done' + str(e))
            sys.exit()

        db = client['GW2_SRS']
        collection = db['players_info']

        mongo_insert = collection.insert_one(stats_dict)
        return mongo_connect()

    return 'Success!'
pass

我的目標是,當我調用 gw2_etl() 時,它會運行內部的每個進程(log_scrape、store_data 和 mongo_connect)並在最后返回成功消息。 我可能做錯了,因為它既不運行任何東西也不發送錯誤消息。

對於mongo連接,我需要返回stats_dict,因為它是我要上傳到那里的JSON文件,csv文件僅用於本地存儲。

我實際上得到了一些老板,因為代碼實際上很長。

如果您對我如何使這項工作有任何提示或線索,我將不勝感激。

在從 function 返回之前,您仍然需要在gw2_etl()中單獨調用所有這些函數。 在另一個內部定義函數只是意味着您無法在外部 function 之外訪問它們。 所以在return語句之前添加

log_scraper(url)
store_data(json_file)
mongo_connect(stats_dict)

並從那里繼續。 您會注意到您需要傳遞一些變量來調用具有正確 arguments 的函數,但我將這部分留給您自己弄清楚。

暫無
暫無

聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM