How can I find out what's changing the return address of a function in c++

Question

I have a program that behaves weirdly and probably has undefined behaviour. Sometimes, the return address of a function seems to be changed, and I don't know what's causing it.

The return address is always changed to the same address, an assertion inside a function the control shouldn't be able to reach. I've been able to stop the program with a debugger to see that when it's supposed to execute a return statement, it jumps straight to the line with the assertion instead.

This code approximates how my function works.

int foo(Vector t)
   double sum = 0;
   for(unsgined int i=0; i<t.size();++i){
        sum += t[i]; 
   }
   double limit = bar(); // bar returns a value between 0 and 1
   double a=0;
   for(double i=0; i<10; i++){
       a += f(i)/sum; // f(1)/sum + ... + f(10)/sum = 1.0f
       if(a>3)return a;
   }
   //shoudn'get here
   assert(false); // ... then this line is executed
}

This is what I've tried so far:

• Switching all std::vector [] operators with .at to prevent accidentily writing into memory
• Made sure all return-by-value values are const.
• Switched on -Wall and -Werror and -pedantic-errors in gcc
• Ran the program with valgrind

I get a couple of invalid read of size 8 , but they seem to originate from qt, so I'm not sure what to make of it. Could this be the problem?

The error happens only occasionally when I have run the program for a while and give it certain input values, and more often in a release build than in a debug build.

EDIT: So I managed to reproduce the problem in a console application (no qt loaded) I then manages to simulate events that caused the problem.

Like some of you suggested, it turns out I misjudged what was actually causing it to reach the assertion, probably due to my lack of experience with qt's debugger. The actual problem was a floating point error in the double i used as a loop condition.

I was implementing softmax, but exp(x) got rounded to zero with particular inputs.

Now, as I have solved the problem, I might rephrase it. Is there a method for checking problems like rounding errors automatically. Ie breaking on 0/0 for instance?

Answer 1

The short answer is:

The most portable way of determining if a floating-point exceptional condition has occurred is to use the floating-point exception facilities provided by C in fenv.h.

although, unfortunately, this is far from being perfect.

I suggest you to read both https://www.securecoding.cert.org/confluence/display/seccode/FLP04-C.+Check+floating-point+inputs+for+exceptional+values and https://www.securecoding.cert.org/confluence/display/seccode/FLP03-C.+Detect+and+handle+floating-point+errors which concisely address the exact question you are posing:

Is there a method for checking problems like rounding errors automatically.