SDL2如何在调整窗口大小后缩放鼠标坐标？

Question

I am using OpenCV ans SDL2 libraries for a C++ application.

Opencv is used to catch the frame from the webcam and the frame is then converted to be shown on a SDL window. SDL is used to catch inputs from the mouse.

At every click pn the SDL window, a circle is drawn in the position of the click over the webcam frame.

The problem is the following: when resizing the window manually when the application is running (from the OS not from the code), the points that were already drawn will be plotted correctly but if I left-click to draw another point in the resized window, the point will be plotted completely out of bounds of the window.

This is the main function:

#ifndef __OPENCV__
#define __OPENCV__
#include "opencv2/opencv.hpp"
#endif
#include <iostream>
#include <unistd.h>
#include <vector>
#include <SDL.h>
#include <SDL_events.h>

using namespace cv;
using namespace std;


int mouseCallback(SDL_MouseButtonEvent ev, Mat frame);


int main(int argc, char* argv[])
{
    /* Initialise SDL */
    if( SDL_Init( SDL_INIT_VIDEO ) < 0)
    {
        fprintf( stderr, "Could not initialise SDL: %s\n", SDL_GetError() );
        exit( -1 );
    }
    const String sourceReference = argv[1];
    int camNum;
    string sensorName;
    try
    {
        camNum = stoi(sourceReference); // throws std::length_error
    }
    catch (const std::exception& e)// reference to the base of a polymorphic object
    {
        std::cout<<"Exception: " << e.what()<<endl; // information from length_error printed
        return -1;
    }
    cout<<"camera initializing\n";
    VideoSettings cam(camNum + CAP_V4L);
    cout<<"camera initialized\n";
    /* or
    VideoCapture captUndTst;
    captUndTst.open(sourceCompareWith);*/
    cout<<"Ch3ck c4m3ra is 0p3n3d\n";
    if ( !cam.isOpened())
    {
        cout << "Could not open reference " << sourceReference << endl;
        return -1;
    }
    Mat frame;
    cam>>frame;
    SDL_Window* win = SDL_CreateWindow("Camera", 100, 100, frame.cols, frame.rows,
        SDL_WINDOW_SHOWN | SDL_WINDOW_RESIZABLE);

    SDL_SetWindowTitle(win, "Camera");
    SDL_Renderer * renderer = SDL_CreateRenderer(win, -1, 0);
    SDL_Event genericEvent;

    SDL_Surface* frameSurface;
    SDL_Texture* frameTexture;
    while(cam.isOpened())
    {
        while( SDL_PollEvent(&genericEvent) )
        {
            switch( genericEvent.type )
            {
                case SDL_MOUSEBUTTONDOWN:
                    mouseCallback(genericEvent.button, frame);
                    break;
                /* SDL_QUIT event (window close) */
                case SDL_QUIT:
                    return 0;
                    break;
                default:
                    break;
            }
        }
        cam>>frame;
        draw(frame_out);

        //Convert to SDL_Surface
        frameSurface = SDL_CreateRGBSurfaceFrom((void*)frame_out.data,
            frame_out.size().width, frame_out.size().height,
            24, frame_out.cols *3,
            0xff0000, 0x00ff00, 0x0000ff, 0);

        if(frameSurface == NULL)
        {
            SDL_Log("Couldn't convert Mat to Surface.");
            return -2;
        }

        //Convert to SDL_Texture
        frameTexture = SDL_CreateTextureFromSurface(renderer, frameSurface);
        if(frameTexture == NULL)
        {
            SDL_Log("Couldn't convert Mat(converted to surface) to Texture."); //<- ERROR!!
            return -1;
        }
        //imshow("Camera", frame_out);
        SDL_RenderCopy(renderer, frameTexture, NULL, NULL);
        SDL_RenderPresent(renderer);
        /* A delay is needed to show (it actually wait for an input)*/
        if(waitKey(delay)>delay){;}

    }
    SDL_DestroyTexture(frameTexture);
    SDL_FreeSurface(frameSurface);
    SDL_DestroyRenderer(renderer);
    SDL_DestroyWindow(win);
    return 0;
}


int mouseCallback(SDL_MouseButtonEvent ev, Mat frame)
{

    if(ev.button == SDL_BUTTON_LEFT)
    {
        mouseLeft( ev.x, ev.y);
    }
}

When clicking the left button of the mouse the function called is mouseCallback that is calling a function mouseLeft defined as:

void mouseLeft(int x, int y)
{
  Point pt;// = *((Point*)param);
  pt.x = x;
  pt.y = y;
  pts.push_back(pt);
  int ii=0;
  int size = pts.size()-1;
  while(ii<size && size>0)
  {
      if (euclideanDist(pt, pts.at(ii))<minDistance)
      {
          //cout<<pts->size()<<endl;
          pts.erase (pts.begin()+ii);

          // std::vector::end returns an iterator to the element following
          //the last element of the container, not to the last element.
          pts.erase(pts.end()-1);
          size-=2;
      }
      else
      {
          ;
      }
      ii++;

  }
}

pts is a vector of Points (it is originally a member of a class but can be declared globally for this example). When mouseLeft is called if the point of click is close enough to any point in pts then it is removed from the vector, otherwise a new point is pushed back.

The function draw is drawing the points stored in pts plus lines connecting the points:

void draw(Mat &frame)
{
  vector<Point> new_pts = pts; //avoid modifying the elements we are drawing
  int sizeVector = pts.size();
  if (sizeVector==1)
  {
      try
      {
        circle( frame, pts.at(0), radius, Scalar( 0, 0, 0), FILLED, 8,0);
      }
        catch (const out_of_range& e)
        {
            cout<<"Exception with command "<<endl;
            cout<<"circle( frame, pts.at(0), radius, Scalar( 0, 0, 0), FILLED, 8,0)"<<endl;
        }
  }
  else
  {
    int ii=0;
    while(ii<(sizeVector-1) && sizeVector>0)
    {
        try
        {
          circle( frame, pts.at(ii),   radius, Scalar( 0, 0, 0), FILLED, 8,0);
          circle( frame, pts.at(ii+1), radius, Scalar( 0, 0, 0), FILLED, 8,0);
          line(frame, pts.at(ii), pts.at(ii+1), Scalar( 0, 0, 0 ), 4, 8 );
        }
        catch (const out_of_range& e)
        {
          cout<<"Exception with commands: "<<endl;
          cout<<"\tcircle( frame, pts.at(ii),   radius, Scalar( 0, 0, 0), FILLED, 8,0)"<<endl;
          cout<<"\tcircle( frame, pts.at(ii+1), radius, Scalar( 0, 0, 0), FILLED, 8,0);"<<endl;
          cout<<"\tline(frame, pts.at(ii), pts.at(ii+1), Scalar( 0, 0, 0 ), 4, 8 );"<<endl;
        }

        if(pts.size()<=0) break;
        ii++;

        sizeVector= pts.size();
    }
  }
}

The same problem happens if instead of creating the window with SDL_Window* win = SDL_CreateWindow("Camera", 100, 100, frame.cols, frame.rows, SDL_WINDOW_SHOWN | SDL_WINDOW_RESIZABLE); I double its dimesions such as: SDL_Window* win = SDL_CreateWindow("Camera", 100, 100, 2*frame.cols, 2*frame.rows, SDL_WINDOW_SHOWN | SDL_WINDOW_RESIZABLE);

Note that the code posted is a readaptation of the original one to keep it simple and understandable. If anything is missing or not clear please tell me so that I can edit the question properly.

Answer 1

The problem was the following: basically the mouse coordinates catched from SDL are correct: when I make the window bigger the coordinates can even have large value.

The problem is that those coordinates were plotted on the frame taken from the cam by openCV which has always the same dimensions (basically the resolution of the cam).

Before drawing the point on the frame I have to rescale the coordinates of the mouse-click within the dimensions of the image taken from the cam.

I found a solution by basically storing the initial dimension of the window in two variables using after the window is defined:

SDL_Window* win = SDL_CreateWindow("Camera", 100, 100, frame.cols, frame.rows,
    SDL_WINDOW_SHOWN | SDL_WINDOW_RESIZABLE);
int oldWidth = frame.cols, oldHeight= frame.rows;
int width, height;
float scaleX=1, scaleY=1;

then adding the following event in the already existing switch-case:

            case SDL_WINDOWEVENT:
            {
                if (genericEvent.window.event==SDL_WINDOWEVENT_RESIZED)
                {
                    //oldWidth = width;
                    //oldHeight = height;
                    SDL_GetWindowSize(win, &width, &height);
                    scaleX =  (float)(width)/ float(oldWidth);
                    scaleY = (float)(height) / (float)(oldHeight);
                }
                break;
            }

and the modifying the `mouseCallback as

int mouseCallback(SDL_MouseButtonEvent ev, float scaleX, float scaleY, Mat frame)
{
    if(ev.button == SDL_BUTTON_LEFT)
    {
        cout<<scaleX<<" "<<scaleY<<endl;
        int scaled_x =  static_cast<int> ((float)(ev.x)/scaleX);
        int scaled_y = static_cast<int> ((float)(ev.y)/ scaleY);
        std::cout<<"scaled x: "<<scaled_x<<", scaled y: "<<scaled_y<<endl;
        mouseLeft( scaled_x,scaled_y);
    }
}