如何通過適當的異步/等待/承諾來提高此代碼的速度

Question

使用Node.js，我有一個任務來改進我創建的代碼。 此代碼執行60個HTTP請求，並為此使用庫。

完成所有HTTP請求並將每個請求保存到文件需要30秒！

據說可以在3秒內通過以下方式發出這些請求：

1.正確管理異步諾言

2.更智能的緩存

3.不使用集群

4.僅添加一次開銷

恐怕我不確定從哪里開始了解我能做些什么。

因此，以下代碼獲得了一個包含60個項目的數組，其中每個項目都是一個HTTP請求：

 const exchanges = ccxt.exchanges 

這些代碼進入：worker =異步函數，並在代碼末尾：等待Promise.all（workers）等待它們完成。

我不確定從哪里開始實際可以降到3秒。 如何提高此代碼的速度？

 'use strict'; const ccxt = require ('ccxt') , log = require ('ololog').noLocate // npm install ololog , fs = require ('fs') // the numWorkers constant defines the number of concurrent workers // those aren't really threads in terms of the async environment // set this to the number of cores in your CPU * 2 // or play with this number to find a setting that works best for you , numWorkers = 8 ;(async () => { // make an array of all exchanges const exchanges = ccxt.exchanges .filter (id => ![ 'cap1', 'cap2' ].includes (id)) // instantiate each exchange and save it to the exchanges list .map (id => new ccxt[id] ({ 'enableRateLimit': true, })) // the worker function for each "async thread" const worker = async function () { // while the array of all exchanges is not empty while (exchanges.length > 0) { // pop one exchange from the array const exchange = exchanges.pop() // check if it has the necessary method implemented if (exchange.has['fetchTickers']) { // try to do "the work" and handle errors if any try { // fetch the response for all tickers from the exchange const tickers = await exchange.fetchTickers() // make a filename from exchange id const filename = '/myproject/tickers/' + exchange.id + 'Tickers.json' // save the response to a file fs.writeFileSync(filename, JSON.stringify({ tickers })); } catch (e) { } //Error } } } // create numWorkers "threads" (they aren't really threads) const workers = [ ... Array (numWorkers) ].map (_ => worker ()) // wait for all of them to execute or fail await Promise.all (workers) }) () 

Answer 1

我認為您正在使事情變得復雜得多。 您可以在map回調中完成所有工作，然后使用Promise.all(promises)等待所有操作完成。 此過程確實比預期的“ 3秒”（在我的情況下為15秒）要長，並且會產生很多錯誤（例如缺少apiToken或未實現fetchTickers），但這可能是我的環境存在的問題（我之前從未使用過ccxt ，而且我沒有任何apiTokens）。

這是我想出的實現，希望它可以幫助您滿足您的需求：

const ccxt = require('ccxt');
const fs = require('fs');
const path = require('path');

(async () => {
    const start = Date.now();

    const dumps = ccxt.exchanges
        .filter((id) => !['coinmarketcap', 'theocean'].includes(id))
        .map(async (id) => {
            const Exchange = ccxt[id];
            const exchange = new Exchange({enableRateLimit: true});
            if (exchange.has['fetchTickers']) {
                try {
                    const tickers = await exchange.fetchTickers();
                    const dumpFile = path.join(__dirname, 'exchanges', `${id}-Tickers.json`);
                    await fs.promises.writeFile(dumpFile, JSON.stringify(tickers));
                } catch (e) {
                    console.error(e);
                }
            }
        });

    await Promise.all(dumps);

    const end = Date.now();
    console.log(`Done in ${(end - start) / 1000} seconds`);
})();

Answer 2

我嘗試看看是否有可能更快地做到這一點。 我嘗試緩存所需的所有可能的內存。在實際執行.fetchTickers（）請求之前。

我設法從15秒減少到了9秒。 但是下面的代碼甚至更進一步，但是我確實收到編譯錯誤，並且不確定我在做什么錯。

錯誤是：

ReferenceError：未定義ID

難道id不會傳遞到被推入'exchangesArray'的'exchange'對象中嗎？

我試圖將交換對象首先花費所有時間放入數組中：

 var exchangesArray = []; 

然后使用此“ exchangesArray”，嘗試執行執行fetchTickers的功能：

 'use strict'; const ccxt = require('ccxt'); const fs = require('fs'); const path = require('path'); //Cache some memories first var exchangesArray = []; (async () => { const allexchanges = ccxt.exchanges.filter((id) => !['coinmarketcap', 'theocean'].includes(id)) .map(async (id) => { const Exchange = ccxt[id]; const exchange = new Exchange({ enableRateLimit: true }); if (exchange.has['fetchTickers']) { exchangesArray.push(exchange); } }); await Promise.all(allexchanges); })(); //Use cached memories to do the "fetchTickers()" as fast as possible (async () => { const start = Date.now(); const exchanges = exchangesArray; while (exchanges.length > 0) { // pop one exchange from the array const exchange = exchanges.pop() try { const tickers = await exchange.fetchTickers(); const dumpFile = path.join(__dirname, 'exchanges', `${id}-Tickers.json`); await fs.promises.writeFile(dumpFile, JSON.stringify(tickers)); } catch (e) { console.error(e); } } await Promise.all(exchanges); const end = Date.now(); console.log(`Done in ${(end - start) / 1000} seconds`); })();