Downloading file on ESP32 using WiFiClient.client.read() fails -- "Task watchdog" error

Question

I'm testing code downloading large files (approx 1mb OTA binary file) from a server

The error happens midway through the download:

-> E (15787) task_wdt: Task watchdog got triggered. The following tasks did not reset the watchdog in time:
-> E (15787) task_wdt:  - async_tcp (CPU 0/1)
-> E (15787) task_wdt: Tasks currently running:
-> E (15787) task_wdt: CPU 0: IDLE0
-> E (15787) task_wdt: CPU 1: IDLE1
-> E (15787) task_wdt: Aborting.
-> abort() was called at PC 0x400e16af on core 0

My current understanding based on this ESP32 github link , is that the download process is blocking the ESP from performing necessary background functions.

The failure happens (in the code below) during a while() loop that runs a client.read() to fetch the file from a server.

I tried testing delay() and vTaskDelay() to see if they might help. Wasn't sure if that frees things up or just further adds to any task blocking. Neither helped. (and I think they're the same function anyway, correct?)

I'm not 100% sure blocking is even the issue. Monitoring the download using a Serial.println(thisClient.available()) showed the bytes remaining going from about 7k to around 5k before jumping backup to 7k -- and doing that repeatedly. That suggests a possible a server issue. BUT that same server, downloading the same file to a JS-coded ajax request works just fine.

The test code below is adapted from an Espressif OTA Example . I'm still new to C++, so forgive the use of String over char arrays. Having trouble working with those chars.

#include <WiFi.h>
#include <Update.h>
#include "AsyncJson.h"
#include "ArduinoJson.h"

AsyncWebServer EspServer(80);

void setup(){
    Serial.begin(115200);

    const char* ClientSsid = "***";
    const char* ClientPwd = "***";
    Serial.print("Connecting to LAN ");
    WiFi.begin(ClientSsid, ClientPwd);
    int CreepConnect;
    CreepConnect=0;
    while (WiFi.status()!=WL_CONNECTED && CreepConnect<30){
        delay(250);
        Serial.print(".");
        CreepConnect++;
    }
    Serial.println(" on SSID " + WiFi.SSID() + " at " + String(WiFi.localIP()));

    EspServer.on("*", HTTP_POST, [](AsyncWebServerRequest * Req){
        AsyncWebParameter* keyVal=Req->getParam(0);
        String key=keyVal->name();
        String val=keyVal->value();
        if(key=="req" && val=="execOTA"){
            Serial.println("Updating...");
            Req->send(200, "text/plain", "Updating...");
            //EspServer.end(); //Tested disabling server
            execOTA();
        }else{
            Serial.println("Request ignored");
            Req->send(200, "text/plain", "Request ignored");
        }
    });
    EspServer.begin();
    //execOTA();  This does work.  It only fails under the callback above.
}

String execOTA(){
    WiFiClient thisClient;
    IPAddress thisHost(192, 168, 1, 10);
    char thisPath[]="/testBigFile.ino.esp32.bin"; //Big = about 800k OTA bin file
    String thisPart, theseHeaders, thisBody;
    if(!thisClient.connect(thisHost, 8465)){
        Serial.println("Connection Failed");
        return "{\"Error\":\"Connection Failed\"}";
    }
    Serial.println(thisPath);
    Serial.println(String(thisPath));
    Serial.println("Connection succeeded");
    Serial.print("thisClient.available(): ");
    Serial.println(thisClient.available());
    String thisReq=String("GET ") + String(thisPath) + " HTTP/1.1\r\n" +
        "Host: 192.168.1.10:8465\r\n" +
        "Cache-Control: no-cache\r\n" +
        "Connection: close\r\n\r\n";
    Serial.println("thisReq: " + thisReq);
    thisClient.print(thisReq);
    unsigned long timeout = millis();
    while(thisClient.available()==0) {
        if(millis()-timeout > 5000){
            Serial.println("Client timed out");
            thisClient.stop();
            Serial.println("Client timed out");
            return "{\"Error\":\"Client timed out\"}";

        }
    }
    Serial.println("Headers Begin");
    thisPart="Header";
    while(thisClient.available()){
        Serial.println(thisClient.available());
        if(thisPart=="Header"){
            String thisLine=thisClient.readStringUntil('\n');
            theseHeaders.concat(thisLine);
            thisLine.trim();
            if(!thisLine.length()){
                Serial.println("Headers Complete:\n" + theseHeaders + "\n------\n");
                thisPart="Body";
            }
        }else{ //*** Task Watchdog Error happens in this block, after about 50 successful character reads with delay ***
            char thisChar=thisClient.read();
            thisBody.concat(thisChar);
            //delay(10); //Tested at various durations to see if it adds to or frees up blocking.  It seems to add further blocking?
            //vTaskDelay(15); //Also tested, at various durations.
        }
    }
    Serial.println("Body Complete");
    thisClient.stop();
    return "{\"Headers\":\"" + theseHeaders + "\",\"Body\":\"" + thisBody + "\"}";
}

String getHeaderValue(String header, String headerName){
    return header.substring(strlen(headerName.c_str()));
}

Answer 1

You're doing too much in the HTTP callback. While the callback is running, the watchdog timer can't be reset. If that happens for too long you'll get the error you're seeing - Task watchdog got triggered . The big clue is that it happens in the async_tcp task.

Try rewriting your code so that the HTTP_POST handler sets a global variable to indicate that execOTA() needs to be called, rather than calling it itself. Then have loop() do the heavy lifting.

Something like this:

boolean exec_ota_flag = false;

void setup() {

...

    EspServer.on("*", HTTP_POST, [](AsyncWebServerRequest * Req){
        AsyncWebParameter* keyVal=Req->getParam(0);
        String key=keyVal->name();
        String val=keyVal->value();
        if(key=="req" && val=="execOTA"){
            exec_ota_flag = true;
            Req->send(200, "text/plain", "Updating...");
        }else{

...

void loop() {
  if(exec_ota_flag) {
      exec_ota_flag = false;
      execOTA();
  }
}

Also the while loop in execOTA needs a delay() call. Try something like this:

    while(thisClient.available()==0) {
        delay(1);
        if(millis()-timeout > 5000){

When you call delay() you give other tasks a chance to run, which will let the watchdog timer be reset.

Answer 2

From your question, it is not clear if you are aware what the watchdog or IDLE tasks are, let alone what is triggering them. Thus I hope this helps: FreeRTOS & IDLE Tasks

Basically, No code should ever block the CPU 100% the time. But if necessary, you can, as long as you create a separate task for it, and assign the tskIDLE_PRIORITY to it. See the link for details.