Reading sensors data from the sysfs interface (hwmon) occasionally results in a blocking call (longer execution time for the function than expected)

Question

I have an ARM machine that runs Linux (BusyBox). I need to frequently read the data in this file /sys/class/hwmon/hwmon0/device/in7_input which contains voltage. It's located in the virtual file system sysfs under the /sys/class/hwmon/ directory.

The file contains data that looks like this (SSHed to the device).

root:~# cat /sys/class/hwmon/hwmon0/device/in7_input
10345
root:~# cat /sys/class/hwmon/hwmon0/device/in7_input
10250

Usually reading the data from that file takes about 1 ms but there are occasions where it takes about 1 sec . Unfortunately for me, I'm not able to replicate this issue on my bench since it doesn't happen very frequently on the field.

I have checked the CPU utilization and it's usually less than 60% when this issue occurs. I'm not sure why occasionally reading from that file results in what appears to be a blocking call that results in a longer execution time for the function.

I'm not sure if I'm dealing with a bigger problem that's occurring on the virtual file system sysfs or if the code I have in readVoltage() isn't completely non-blocking.

Here is a snippet of that code that I had to tweak

/******************************************************************************

                              Online C++ Compiler.
               Code, Compile, Run and Debug C++ program online.
Write your code in this editor and press "Run" button to compile and execute it.

*******************************************************************************/

#include <iostream>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>

using namespace std;

uint64_t GetClockCount(void);
float calculateLoopTime(uint32_t tUsec1, uint32_t tUsec2);

void readVoltage()
{

    static const char VoltageFile[] = "/sys/class/hwmon/hwmon0/device/in7_input";
    static int VoltageFD = -1;

    if (-1 == VoltageFD)
    {
        VoltageFD = open(VoltageFile, O_RDONLY | O_NONBLOCK);
    }

        if (-1 == VoltageFD)
        {
            std::cout << "couldn't open FD for " << VoltageFile << std::endl;
        }
        else
        {
            static const size_t bufSize = 15;
            char buffer[bufSize];

            fd_set input;
            FD_ZERO(&input);
            FD_SET(VoltageFD, &input);

            struct timeval to;
            to.tv_sec = 0;
            to.tv_usec = 0;
            int n = 0;
            n = select(VoltageFD + 1, &input, NULL, NULL, &to);

            if (n > 0)
            {
                ssize_t bytes_read = pread(VoltageFD, buffer, bufSize, 0);

                if (bytes_read > 0)
                {
                    float voltage = (atof(buffer) / 1000.0f);

                    std::cout << "voltage= " << voltage << std::endl;
                }
            }
        }
    }


int main()
{
    uint32_t start_time = GetClockCount();
    
    readVoltage();
    
    uint32_t end_time = GetClockCount();
    
    float time_diff = calculateLoopTime(start_time, end_time);
    
    std::cout << "function took " << time_diff << " ms to execute" << std::endl;
    
    
    return 0;
}

uint64_t GetClockCount(void)
{
    struct timespec now;
    if (clock_gettime(CLOCK_MONOTONIC, &now))
        return 0;
    return static_cast<uint64_t>(now.tv_sec) * 1000000 + now.tv_nsec / 1000;
}


float calculateLoopTime(uint32_t tUsec1, uint32_t tUsec2)
{
    float time_diff = 0;

    if (tUsec1 != tUsec2)
    {
        uint32_t time_diff_temp = 0;

        if (tUsec2 > tUsec1)
        {
            time_diff_temp = (tUsec2 - tUsec1);
        }
    
        // Scale from microseconds to milliseconds
        time_diff = static_cast<float>(time_diff_temp) / 1000;
    }

    return time_diff;
}

You can run the code here but obviously you won't get the the voltage since this file /sys/class/hwmon/hwmon0/device/in7_input doesn't exist on that online IDE.

https://onlinegdb.com/d0wcJ0tzT

Answer 1

The code you wrote in the readVoltage() is non-blocking. You can use the fact, that read() on non-blocking descriptor would return E_AGAIN / E_WOULDBLOCK errno if reading would block and therefore remove lines with select() and setting FD_SET (less code).

You can't be sure what is happening between GetClockCount() calls. If your system is under load (like you say less 60%) it can be doing something else. Hardware thread can be doing some other operation even if it's not doing it 100% time it can introduce some jitter when switching back to your sysfs reading code.

To be more accurate try to measure only pread() invocation time but therefore still be possibility for running some other process' code, especially when talking about system under load.

You can also try to use perf to check out what's going on in your program. Look here to check out that perf can show how many context-switches occured during running your program.

Try to also measure calling plain sysfs reading by using:

root:~# cat /sys/class/hwmon/hwmon0/device/in7_input

All the results should show that something other is done by the system and pread() call is not blocking itself as it's double checked - once using non-blocking flag and secondary when checking if select() returned ready descriptor.