perf线程绑定机制

Question

我对 perf_event 计数器绑定到目标分析过程的子线程的机制感到困惑。

我正在使用perf_event_open对目标程序的 CPU CYCLES 进行采样。 该程序将创建，比如说 99 个线程。 perf_event_open()系统调用只会使用目标程序的 pid（父程序的 pid）调用一次。

假设所有这 100 个线程（1 个父线程 + 99 个子线程）都在 100 个不同的 CPU 上运行，并且采样频率设置为 99Hz。 问题是这 99 个子线程是否会绑定到特定的 PMU 并像父进程一样自动被采样。 1 秒的采样后，我们会得到多少个样本，为什么？

A. 只有 99 个样本来自父进程。

B. 来自所有运行线程的 9900 个样本。

C。 其他

Answer 1

我用下面的代码做了一个小实验，我认为可以从中回答这个问题。 这个实验的过程是

1. 设置信号动作、TID 和计数器读数将在处理程序中打印
2. 将 perf_event_open 与main()的 PID 绑定
3. pthread_create另外 9 个线程在其中执行一些计算
4. 检查 output

#include <iostream>
#include <linux/perf_event.h>    /* Definition of PERF_* constants */
#include <linux/hw_breakpoint.h> /* Definition of HW_* constants */
#include <sys/syscall.h>         /* Definition of SYS_* constants */
#include <unistd.h>
#include <string.h>
#include <string>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <signal.h>
#include <vector>
#include <stdlib.h>
#include <time.h>
#include <pthread.h>

#define N 1024
#define RANDMAX 1000
#define PRIOR 9
#define SUBSEQUENT 9
#define MAXLEN 2000

struct timespec start, end;
typedef unsigned long long int u64;
std::string counters[MAXLEN];
int idx = 0;

// every SIGPROF would be handled 
void sigHandler(int signum, siginfo_t *siginfo, void *ucontext)
{
    u64 counter;
    read(siginfo->si_fd, &counter, sizeof(counter));

    pid_t tid = gettid();
    //concatenate tid and counter 
    int curr_idx = idx;    
    for(;;){
        if(__sync_bool_compare_and_swap(&idx, curr_idx, (curr_idx+1))){
            std::string sample_info = std::to_string(tid) + "::" + std::to_string(counter);
            counters[curr_idx] = sample_info;
            break;
        }
    }    
}

void *multiplyMatrix(void *nothing)
{
    auto A = new double[N][N];
    auto B = new double[N][N];
    auto C = new double[N][N];

    for (int i = 0; i < N; i++)
    {
        for (int j = 0; j < N; j++)
        {
            A[i][j] = (double)rand() / (double)RANDMAX;
            B[i][j] = (double)rand() / (double)RANDMAX;
        }
    }

    for (int i = 0; i < N; i++)
    {
        for (int j = 0; j < N; j++)
        {
            for (int k = 0; k < N; k++)
            {
                C[i][j] += A[i][k] * B[k][j];
            }
        }
    }
    return NULL;
}

int main()
{
    struct perf_event_attr pe;
    pid_t pid;
    pid = gettid();
    int fd;

    //set signal action
    struct sigaction sa, old_sa;
    sigemptyset(&sa.sa_mask);
    sa.sa_sigaction = &sigHandler;
    sa.sa_flags = SA_SIGINFO | SA_RESTART;
    sigaction(SIGPROF, &sa, &old_sa);

    //config perf_event attr
    memset(&pe, 0, sizeof(pe));
    pe.type = PERF_TYPE_HARDWARE;
    pe.config = PERF_COUNT_HW_CPU_CYCLES;
    pe.size = sizeof(pe);
    pe.disabled = 1;
    pe.sample_freq = 10; // sample mode with 10Hz
    pe.freq = 1;
    pe.wakeup_events = 1;
    pe.sample_type = PERF_SAMPLE_CALLCHAIN;

    //open perf_event binding with main() pid
    if ((fd = syscall(__NR_perf_event_open, &pe, pid, -1, -1, 0)) < 0)
    {
    };

    //fd control and io control
    struct f_owner_ex ex = {
        .type = F_OWNER_PID,
        .pid = pid,
    };
    if (fcntl(fd, F_SETSIG, SIGPROF) < 0 || fcntl(fd, F_SETOWN_EX, &ex) < 0 || fcntl(fd, F_SETFL, fcntl(fd, F_GETFL) | FASYNC | O_ASYNC) < 0)
    {
        std::cout << "fcntl error" << std::endl;
    }
    else if (ioctl(fd, PERF_EVENT_IOC_RESET, 0) < 0 || ioctl(fd, PERF_EVENT_IOC_ENABLE, 0) < 0)
    {
        std::cout << "ioctl error" << std::endl;
    }
    std::cout << "pid = "<< pid <<std::endl;

    // 9 pthreads would be created
    pthread_t sub_threads[SUBSEQUENT];
    int ret = 0;
    void *retval = NULL;

    std::cout << "start" << std::endl;
    clock_gettime(CLOCK_MONOTONIC, &start);

    for (int i = 0; i < SUBSEQUENT; i++)
    {
        ret = pthread_create(&sub_threads[i], NULL, &multiplyMatrix, NULL);
        if (ret != 0)
        {
            std::cout << "Error happens on pthread creating" << std::endl;
        }
    }

    for (int i = 0; i < SUBSEQUENT; i++)
    {
        ret = pthread_join(sub_threads[i], &retval);
        if (ret != 0)
        {
            std::cout << "Error happens on pthread joining" << std::endl;
        }
    }

    clock_gettime(CLOCK_MONOTONIC, &end);
    std::cout << "end" << std::endl;

    double tdiff = (end.tv_sec - start.tv_sec) + 1e-9 * (end.tv_nsec - start.tv_nsec);
    
    std::cout << "time used: " << std::to_string(tdiff) << std::endl;
    std::cout << "Start to print counter" << std::endl;

    //output the pid info and corresponding counter
    // std::cout << std::to_string(counters.size())<< std::endl;
    // for(std::vector<std::string>::iterator it = counters.begin(); it != counters.end(); it++){
    //     std::cout<<*it<<std::endl;
    // }
    // std::cout << std::to_string(counters.size())<< std::endl;
    for(int i = 0; i < idx; i++){
        std::cout<<counters[i]<<std::endl;
    }
    ioctl(fd, PERF_EVENT_IOC_DISABLE, 0);

    return 0;
}

这个实验已经进行了几十次，所有的结果都是一样的。下面是一次运行的 output。

pid = 2832444
start
end
time used: 11.872525
Start to print counter
2832444::22222
2832444::191758
2832444::200708
2832444::220478
2832444::236749
2832444::321106

看起来唯一会被 perf 采样并触发 SIGPROF 的线程是 main() 的 PID。

但我不确定这个结论。 如果有人有正确的答案，请与我分享。