Creating a list of lists of lists to sort data from a text file

Question

I'm trying to read and analyse data back from a molecular dynamics simulation, which looks like this, but has approximately 50000 lines :

40 443.217134221125 -1167.16960983145 -930.540717277902 -945.149746592058 14.6090293141563 -76510.1177229871 4955.17798368798 17.0485096390963 17.0485096390963 17.0485096390963
80 659.39103652059 -923.638916369481 -963.088128935875 -984.822539088925 21.7344101530497 14390.2520385682 4392.18167603894 16.3767140226773 16.3767140226773 16.3767140226773
120 410.282687399253 -979.413482414461 -978.270613122515 -991.794079036891 13.5234659143754 -416.30808174241 4398.37322990079 16.3844056974088 16.3844056974088 16.3844056974088

The second column represents temperature. I want to have the entire contents of the file inside a list, containing lists dividing every line depending on their temperature. So for example, the first list in the main list would have every line where the temperature is 50+/-25K, the second list in the main list would have every line where the temperature is 100+/-25K, the third for 150+/-25K, etc.
Here's the code I have so far :

for nbligne in tqdm(range(0,len(LogFullText),1), unit=" lignes", disable=False):
    string = LogFullText[nbligne]
    line = string.replace('\n','')
    Values = line.split(' ')

    divider = float(Values[1])
    number = int(round(divider/ecart,0))
    if number>0 and number < (nbpts+1):
        numericValues = []
        for nbresultat in range(0,len(Values)-1,1):
            numericValues = numericValues + [float(Values[nbresultat+1])]

        TotalResultats[number-1].append(numericValues)

The entire document with data is stored in the list LogFullText, in which I remove the \\n at the end and split the data, using line.split(' '), I then know in which "section" of the main list, TotalResultats, the line of data has to be stored with the variable number, ecart has in my example a value of 50.

From my testing in idle, this should work, but in reality what happens in that the list numericValues is appended to every section of TotalResultats, which makes the entire "sorting" process pointless, as I simply end up with nbpts times the same list.

EDIT : A desired output would be for example to have TotalResultats[0] contain only these lines :

440 49.9911561170447 -1002.727121613 -1002.72088094757 -1004.36865629012 1.64777534254374 -2.30045369926927 4346.38067015602 16.319590369315 16.319590369315 16.319590369315
480 42.0678318129411 -1002.69068695093 -1003.09270361295 -1004.47931559314 1.38661198019398 148.219667654185 4345.58826561836 16.3185985476593 16.3185985476593 16.3185985476593
520 43.0855216044083 -1003.4761833678 -1003.33820025832 -1004.75835665467 1.42015639634654 -50.877194096845 4345.23364199522 16.3181546401367 16.3181546401367 16.3181546401367

Whereas TotalResults[1] would contain these :

29480 109.504432929553 -980.560226069922 -998.958927113452 -1002.5683396275 3.6094125140473 6797.60091557441 4336.52501942717 16.3072458525354 16.3072458525354 16.3072458525354
29520 106.663291994583 -987.853629557979 -998.63436605413 -1002.15013076443 3.51576471029626 3975.43407740646 4344.84444478408 16.3176674266037 16.3176674266037 16.3176674266037
29560 112.712019757891 -1020.65735849343 -998.342638324154 -1002.05777718853 3.71513886437272 -8172.25412368794 4374.81748831773 16.3551041162317 16.3551041162317 16.3551041162317

And TotalResults[2] would be :

52480 142.86322849701 -983.254970494784 -995.977110177167 -1000.68607319299 4.70896301582636 4687.60299340191 4348.30194824999 16.321994657312 16.321994657312 16.321994657312
52520 159.953459288754 -984.221801201968 -995.711657311665 -1000.9839371836 5.27227987193358 4233.04866428826 4348.82254074761 16.3226460049712 16.3226460049712 16.3226460049712
52560 161.624843851124 -1011.76969126636 -995.320907086768 -1000.64827802848 5.32737094170867 -6023.57133443538 4375.12133631739 16.3554827492176 16.3554827492176 16.3554827492176

In the first case,

TotalResultats[0][0] = [49.9911561170447, -1002.727121613, -1002.72088094757, -1004.36865629012, 1.64777534254374, -2.30045369926927, 4346.38067015602, 16.319590369315, 16.319590369315, 16.319590369315]

If it can help, I'm coding this in Visual Studio, using python 3.6.8

Thanks a whole lot!

Answer 1

I recommend to use pandas. It's a very powerfull tool to treat tabular data in python. It's like excel or sql inside python. Suppose 1.csv contains the data you have provided in the question. Then you can easily load data, filter it, and save results:

import pandas as pd

# load data from file into pandas dataframe
df = pd.read_csv('1.csv', header=None, delimiter=' ')

# filter by temperature, column named 0 since there is no header in the file
df2 = df[df[0].between(450, 550)]

# save filtered rows in the same format
df2.to_csv('2.csv', header=None, index=False, sep=' ')

Pandas may be harder to learn than plain python syntax but it is well worth it.