使用 For 循环时，“堆栈不平衡”警告对我的 Output 有何影响？

Question

I made a function to calculate Population Weighted Average Densities (PWADs) of randomly generated (x,y) points on a graph in R. The function isn't perfect... I assign points to grid squares based not on the grid square they're in but instead based on which grid centre they're closest to, I then assume the area a centre pulls from is the size of the grid (either 1 or 0.25). The purpose of the function is to test sensitivity to grid location and size under varying "total populations". This is the function (the comments were written for a future version of me):

pwad.grid <- function(xval, yval, totalpop) {
  values <- data.frame(xval, yval)
  
  # I'll use the four grids that I've already got
  # the original size grids
  
  # I'm not cleaning up this code, i.e. it's as it was originally developed for a 
  # non-function use when I only wanted to do this once
  # again, I'm using minimum Euclidean distance of (x, y) point to grid centre to 
  # fudge assignation to grids
  
  lowbounds3 <- 0.25
  highbounds3 <- 10.25
  
  centres1 <- data.frame(x=seq(0.05, 1, .1) * 10, y=10 *  
                          as.vector(matrix(rep(seq(0.05, 1, .1), each=10), 
                                           nrow=10)))
  centres2 <- data.frame(x=seq(0, 10, 1), y=10 *  
                          as.vector(matrix(rep(seq(0, 1, .1), each=11),
                                           nrow=11)))
  centres3 <- data.frame(x=seq(lowbounds3, highbounds3, 1), y=
                          as.vector(matrix(rep(seq(lowbounds3, highbounds3, 1),
                                               each=11), nrow=11)))
  
  #the quarter size integer aligned grid
  
  lowbounds4 <- 0.25
  highbounds4 <- 9.75
  centres4 <- data.frame(x=seq(lowbounds4, highbounds4, .5), y=
                          as.vector(matrix(rep(seq(lowbounds4, highbounds4, .5), 
                                               each=20), nrow=20)))
  
  # and now the stores, which I alter somewhat to allow for varying populations
  # reminder: these are the calculations for the Euclidean distances
  # the code inside the matrix calculates the distances on a repeated entry basis
  # i.e. when there are 100 centres, each (x, y) is repeated 100 times, once for 
  # each centre #the matrix then arranges the results so that each (x, y) occupies 
  # only one row once again
  
  stores1 <- matrix(sqrt(rowSums((values[rep(1:totalpop, each=100), ] - 
                                    centres1[rep(1:100, totalpop), ])^2)), 
                    ncol=100, byrow=TRUE)
  
  stores2 <- matrix(sqrt(rowSums((values[rep(1:totalpop, each=121), ] - 
                                    centres2[rep(1:121, totalpop), ])^2)), 
                    ncol=121, byrow=TRUE)
  
  stores3 <- matrix(sqrt(rowSums((values[rep(1:totalpop, each=121), ] - 
                                    centres3[rep(1:121, totalpop), ])^2)), 
                    ncol=121, byrow=TRUE)
  
  stores4 <- matrix(sqrt(rowSums((values[rep(1:totalpop, each=400), ] - 
                                    centres4[rep(1:400, totalpop), ])^2)), 
                    ncol=400, byrow=TRUE)
  
  # assigning points to groups based on the minimum Euclidean Distance
  groups1 <- max.col(-stores1)
  groups2 <- max.col(-stores2)
  groups3 <- max.col(-stores3)
  groups4 <- max.col(-stores4)
  
  # calculating the PWADs
  pwad1 <- sum(table(groups1) * table(groups1)/totalpop)
  pwad2 <- sum(table(groups2) * table(groups2)/totalpop)
  pwad3 <- sum(table(groups3) * table(groups3)/totalpop)
  mill <- table(groups4) / 0.25
  pwad4 <- sum(mill * table(groups4)/totalpop)
  
  # outputs grouped together
  data.frame(pwad1, pwad2, pwad3, pwad4)
}

In order to look at the effects of varying population size, I have been using for loops within R. Each loop is 1000 iterations and generates four groups of 1000 PWADs (one for each grid type). For a population bigger than 100, the loop takes more than a minute to complete on my machine. For a population of 1000 it takes about 12-13 minutes. Based on the various populations I've already done, I expected a population of 5000 to take about 66 minutes. That's an age, but I was going out so why not run it?

This is the loop and the antecedent code I ran for the population of 5000:

# I created sims earlier when I ran my very first population.
sims <- data.frame(baseline=1:1000, ptfive=1:1000, pt75=1:1000, qtrsize=1:1000)
# I did not run it again when I ran the below:

xvalues <- matrix(runif(5000 * 1000) * 10, ncol=1000)
yvalues <- matrix(runif(5000 * 1000) * 10, ncol=1000)

dim(xvalues)

start_time <- Sys.time()
for (i in 1:1000) {
  xval <- xvalues[, i]
  yval <- yvalues[, i]
  
  sims[i, ] <- pwad.grid(xval, yval, 5000)
  #commented out just in case I forget and run all chunks
}
end_time <- Sys.time()
#started at 5:51, expect to finish approx 6:51-

#write.csv(sims, "5000sim.csv")

end_time - start_time

And this is the console output running that (sims aside) generated:

xvalues <- matrix(runif(5000 * 1000) * 10, ncol=1000)
yvalues <- matrix(runif(5000 * 1000) * 10, ncol=1000)
dim(xvalues)
# [1] 5000 1000
start_time <- Sys.time()
for (i in 1:1000) {
  xval <- xvalues[, i]
  yval <- yvalues[, i]
  
  sims[i, ] <- pwad.grid(xval, yval, 5000)
  #commented out just in case I forget and run all chunks
}
# Warning: stack imbalance in 'for', 2 then -1
end_time <- Sys.time()
end_time - start_time
# Time difference of 1.251416 hours

As you can see, I got a warning (not an error.), Unfortunately. because I've been saving the outputs as,csv files. I haven't used set.seed() so it was the specific set of numbers I used that caused the warning...

My questions are these:

• What is " Warning: stack imbalance in 'for', 2 then -1 "?
• Are my results for the population of 5000 compromised?
• Why did the warning happen?
• How might I avoid it if I ran the code for 10000 while watching a movie/overnight?

In searching Google, I see mostly descriptions of "stack imbalance" in the context of Rcpp or different languages. As you can see, I have used only base R functions to build my function, and a for loop, which is also from base R.

In case it's a memory thing:

But that's post-loop. I don't know what it was at before or during running it.

Not sure about tags, let me know if more details are needed. Many thanks!

Completion predictions from SLR:

cloudnumber <- c(10, 100, 250, 500, 500, 750, 900, 1000, 1250)
yseconds <- c(9.173949, 55.87789, 2.186122 * 60, 4.707054 * 60, 4.606928 * 60, 
              7.831578 * 60, 9.376838 * 60,  12.30255 * 60, 15.15093 * 60)
runtime <- lm(yseconds ~ cloudnumber)
predict(runtime, data.frame(cloudnumber=newdata, type="response")) / 60

Adding in the length for the 5000 population adjusts the predicted 10000 population from 120.62128 minutes to 150.58184 minutes.

Answer 1

The "Warning: stack imbalance in 'for', 2 then -1" message says that something in C/C++ level code is not programmed correctly. When C level code creates an R variable, it needs to call PROTECT on it, so that the garbage collector doesn't release it. At the end of the call it is supposed to make a matching UNPROTECT call so that the object can be freed.

R checks on these at the beginning and end of external calls, and warns if the results don't balance.

Now if the code you showed us is all that was running, this is a sign of an internal bug in R. It's unfortunate that your example is not reproducible due to not calling set.seed , and that it takes so long to run: it would be very difficult for anyone else to reproduce it.

You ask whether this compromises your results. I would say that it could, but of course I don't know that it did.

For your next run, you should definitely use set.seed(n) to fix the RNG value to known n at the start. If the warning happens again at least then you can try the identical run and see if it is reproducible. Hopefully it will be, and then you can try to debug it: does it happen with a shorter for loop? If you run options(warn=2) to turn the warning into an error, you might be able to narrow it down to exactly which step caused the problem. Let us (or the R developers) know if you get something reproducible, and maybe the bug can be fixed.