Creating lists from data file

Question

I have a pre-defined list that gives data in the form of (min, max, increment). for example:

[[0.0 1.0 0.1 #mass 
  1.0 5.0 1.0 #velocity
  45.0 47.0 1.0 #angle in degrees
  0.05 0.07 0.1 #drag coeff.
  0.0 0.0 0.0 #x-position
  0.0 0.0 0.0]] #y-postion

and this goes on a for a few more variables. Ideally I want to take each one in as an individual variable declaration and create a finite list of each value in the given range.

For example, mass would be:

m = [0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]

this way I can utilize itertools.combinations((m, x, b,...), r) to create all possible combinations given the various possibilities of each variable.

Any suggestions?

Answer 1

You wrote the list as a flat list, with all numbers on the same level

[[0.0 1.0 0.1 1.0 5.0 1.0 45.0 47.0 1.0 ...]]

but it's possible you meant to write it as a nested list

[[0.0, 1.0, 0.1], [1.0, 5.0, 1.0], [45.0, 47.0, 1.0], ...]

so I'll show both solutions. Please let me know how your data/list is actually structured.

Python's range function doesn't support floats, but you can use NumPy's arange .

The try ... except part is for your unchanging values like 0.0 0.0 0.0 #x-position .

Flat list solution:

flat_list = [0.0, 1.0, 0.1,
             1.0, 5.0, 1.0,
             45.0, 47.0, 1.0,
             0.05, 0.07, 0.1,
             0.0, 0.0, 0.0,
             0.0, 0.0, 0.0]
import numpy as np
incremented_lists = []
for i in range(0, len(flat_list), 3):  # Step in threes
    minimum, maximum, increment = flat_list[i:i+3]
    try:
        incremented_list = list(np.arange(minimum, maximum + increment, increment))
    except ZeroDivisionError:
        incremented_list = [minimum]
    incremented_lists.append(incremented_list)

Nested list solution:

nested_list = [[0.0, 1.0, 0.1],
               [1.0, 5.0, 1.0],
               [45.0, 47.0, 1.0],
               [0.05, 0.07, 0.1],
               [0.0, 0.0, 0.0],
               [0.0, 0.0, 0.0]]
import numpy as np
incremented_lists = []
for sub_list in nested_list:
    minimum, maximum, increment = sub_list
    try:
        incremented_list = list(np.arange(minimum, maximum + increment, increment))
    except ZeroDivisionError:
        incremented_list = [minimum]
    incremented_lists.append(incremented_list)

Running either of these with Python 2.7 or Python 3.3 gets this:

incremented_lists: [[0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0],
                    [1.0, 2.0, 3.0, 4.0, 5.0],
                    [45.0, 46.0, 47.0],
                    [0.05, 0.15],
                    [0.0],
                    [0.0]]

The [0.05, 0.15] is probably undesirable, but I think your huge 0.1 increment for the drag coefficient is more likely a typo than something I should make the code handle. Please let me know if you would like the code to handle unnatural increments and avoid overshooting the maximum. One way to handle that would be to add incremented_list = [x for x in incremented_list if x <= maximum] right before incremented_lists.append(incremented_list) , though I'm sure there's a cleaner way to do it.

Answer 2

Not sure about you list structure, if you do need to take slices you can use itertools.islice and store all lists in a dict:

from itertools import islice

l = iter([0.0, 1.0, 0.1, #mass
  1.0, 5.0, 1.0,#velocity
  45.0 ,47.0, 1.0, #angle in degrees
  0.05, 0.07, 0.1, #drag coeff.
  0.0, 0.0 ,0.0 ,#x-position
  0.0 ,0.0, 0.0])#y-postion

d = {}

import numpy as np

for v in ("m","v","and","drg","x-p","y-p"): # put all "variable" names in order
    start, stop , step = islice(l, None, 3)
    # or use next()
    # start, stop , step = next(l), next(l), next(l)
    if stop > start: # make sure we have a step to take
        # create key/value pairing 
        d[v] = np.arange(start, stop + 1,step)
    else: 
         # add empty list for zero values
         d[v] = []

 print(d)
 {'x-p': [], 'drg': array([ 0.05,  0.15,  0.25,  0.35,  0.45,  0.55,  0.65,  0.75,  0.85,
    0.95,  1.05]), 'and': array([ 45.,  46.,  47.]), 'v': array([ 1.,  2.,  3.,  4.,  5.]), 'y-p': [], 'm': array([ 0. ,  0.1,  0.2,  0.3,  0.4,  0.5,  0.6,  0.7,  0.8,  0.9,  1. ,
    1.1,  1.2,  1.3,  1.4,  1.5,  1.6,  1.7,  1.8,  1.9])}

You can also create your own range that will take a float as a step:

def float_range(start=0, stop=None, step=1):
    while start <= stop:
        yield start
        start += step

Then call it with list(start, stop,step) , but you need to be careful when dealing with floats because of Floating Point Arithmetic: Issues and Limitations

Answer 3

I can't think of any existing format supporting your desired input -- with spaces as separator, newlines breaking sub-lists, and comments actually meaningful as you appear to desire the to define the sub-lists' names. So, I think you'll have to code your own parser, eg:

import re, numpy as np

res_dict = {}
with open('thefile.txt') as f:
    for line in f:
        mo = re.match(r'[?[(\S+)\s*(\S+)\s*(\S+)\s*#(\w)', line)
        keybase = mo.group(4)
        keyadd = 0
        key = keybase
        while key in res_dict:
            key = '{}{}'.format(keybase, keyadd)
            keyadd += 1
        res_dict[key] = np.arange(
            float(mo.group(1)),
            float(mo.group(2)),
            float(mo.group(3)),
        )

This won't give you a top-level variable m as you mention -- but rather a better-structured, more robust res_dict['m'] instead. If you insist on making your code brittle and fragile, you can globals().update(res_dict) to make it so:-)...