如何计算每分钟数据集的 15 分钟标准偏差？

Question

I have one array, raytimes, which is a fraction of an hour, eg, [0, 0.1, 0.2... 0.9, 1.0]. I have another list of floats, which is my velocity vr. Each data time corresponds to a list of velocities as a function of height.

I'm trying to compute a 15-min standard deviation of velocity from this velocity dataset, and keep it so that the std is performed at each height level (so I should have an array of the standard deviations, one for each height).

raytimes is time [0, 0.1, 0.2... 0.9, 1.0]. vr is a 108 list of 2500 float64 numbers. The 2500 numbers correspond to the velocity measured at each height (on a fixed height grid). I don't know how to separate the chunks of data such that I can compute std on just the first, second, third, and fourth 15-min intervals. THEN I need to compute the std at each specific height level.

for i in raytimes:
    if raytimes[i] < 0.25:
        w1 = w1.append(vr)
    if raytimes[i] > 0.25 & raytimes < 0.5:
        w2 = w2.append(vr)
    if raytimes[i] > 0.5 & raytimes < 0.75:
        w3 = w3.append(vr)
    if raytimes[i] < 1:
        w4 = w4.append(vr)
sigma_w1 = std(w1)
sigma_w2 = std(w2)
etc...

Problem is in my above code I am appending the entire vr matrix. How do I only append the list of vrs that correspond to times within the 15min chunk? And then how to I compute the std maintaining the height grid, so the std is computed along each height? I should end up with the same array size of 2500.

Answer 1

Here's the start of an answer which I can refine based on your feedback. Note this is not really a great way to actually transform the data, I'm just trying to demonstrate how to move your code toward something which gives the answer you want. Here I've assumed that you want an SD grouped by height and 15-minute period; so that's 10000 results. If you actually wanted the SD over height or some other grouping function let me know in the comments. I've also assumed from what you said above that vr is a list of lists. Specifically a list of length 108 of lists of length 2500. If that's not right, leave a comment.

EDIT - I realised there's a fundamental mistake in how you are using the for loop which I had inadvertently copied. You are using i as the index, but i is the actual value of the item. If you want the position of the item, you need to use enumerate. See my example below, I've made i the index and t the value for raytimes.

EDIT 2 - approach remains the same but I've actually run this code so I've corrected all the various mistakes that you and I had made in the previous iteration. Can you try this with your data and confirm that the output is correct, then we can look at how you need the output presented.

EDIT 3 - added four results lists to hold the output as requested

from statistics import pstdev
#remove these lines, these are just test data
raytimes=[0,0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1]
heights=[1,2]
vr=[[4,3],[5,3],[3,5],[4,1],[8,2],[2,3],[1,4],[9,5],[7,3],[6,7],[3,8]]
#initialise empty arrays
w1 = []
w2 = []
w3 = []
w4 = []
r1 = []
r2 = []
r3 = []
r4 = []


for j, h in enumerate(heights):
    for i, t in enumerate(raytimes):
        if raytimes[i] < 0.25:
            w1.append(vr[i][j])
        elif 0.25 < raytimes[i] < 0.5:
            w2.append(vr[i][j])
        elif 0.5 < raytimes[i] < 0.75:
            w3.append(vr[i][j])
        else:
            w4.append(vr[i][j])
    print(w1,w2,w3,w4)
    print("First Period - Height: ", str(h), " SD: ", str(pstdev(w1)))
    r1.append(pstdev(w1))
    print("Second Period - Height: ", str(h), " SD: ", str(pstdev(w2)))
    r2.append(pstdev(w2))
    print("Third Period - Height: ", str(h), " SD: ", str(pstdev(w3)))
    r3.append(pstdev(w3))
    print("Fourth Period - Height: ", str(h), " SD: ", str(pstdev(w4)))
    r4.append(pstdev(w4))
    w1 = []
    w2 = []
    w3 = []
    w4 = []

Answer 2

Ok, we can do that. So your expected output is a 2500-long list of 4-long lists right? 10000 values in total? I think the issue you are having is that you're trying to assign values outside the range of the list, you can't grow a list that way.

Edit - oops this wasn't supposed to be an answer. Mistook it for the comment box on mobile. Nevermind