在没有循环的模式之前解析所有子字符串？

Question

I have a long string that consists of many numbers separated by spaces (and sometimes there's even a new line thrown in there). I'd like to go through the string and append all the numbers to a new list that come before the start of 0.000000000000000000e+00 numbers. So here's a sample of my string:

my_string = '1.249132165057832031e+13 1.638194600635518555e+13 2.127995187558799219e+13 2.744617593148214062e+13 -2.558800658636701519e+28 5.918883595148564680e+30 3.603563681248702509e+31 4.325917213186498068e+31 4.911908042151239481e+31 4.463331378152286632e+31 3.684371076399113503e+31 2.500614504012405068e+31 9.997365425073173512e+30 -7.046725649106466938e+30 -2.192076417151744811e+31 -2.531287564917444482e+31 -6.962936418905874724e+30 3.281685507310205847e+31 9.241630178064907840e+31 1.730544785932614751e+32 2.619210949875333106e+32 2.984440142196566918e+32 8.964375812060072923e+31 -8.515727465135046667e+32 -3.425309034394939997e+33 -8.145884847188906515e+33 -9.922370830834364410e+33 -2.119464668318252366e+28 -1.689726703118075140e+27 1.440101653069986610e+26 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 6.186324149659251562e+13 8.113154959294240625e+13 1.053889122977165625e+14 1.359271226298647969e+14 -2.097046363337115528e+28 4.850777756495711585e+30 2.953274256558218597e+31 3.545273642763729060e+31 4.025456872055449111e+31 3.657581460085835446e+31 3.018816679659856350e+31 2.048223110003727437e+31 8.176806147340775115e+30 -5.796250740354887641e+30 -1.798839398031696094e+31 -2.076444435341100150e+31 -5.711669151245612857e+30 2.691583747083509247e+31 7.579958708961477309e+31 1.419395486743453834e+32 2.148287875274468622e+32 2.447859658750551118e+32 7.352862842410293685e+31 -6.984595303325589259e+32 -2.809449882735912952e+33 -6.681296633318354125e+33 -8.138406580426555140e+33 -1.740744048703962454e+28 -1.411749034480591280e+27 8.079362883576220633e+25 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00'

and from this string, all I want in the end would be:

new_list = ['1.440101653069986610e+26', '8.079362883576220633e+25']

I was thinking I'd use regex, but this seems a little tricky since I there are a bunch of 0.000000000000000000e+00 occurrences grouped together and I only want the nonzero number right before the first zero occurrence. I also cannot assume that there's always an equal number of zeros grouped together.

I also thought of splitting on the spaces and iterating through, but my full string is actually far too long to do this efficiently. How can I do this?

Answer 1

You can use negative lookbehind assertion:

In [55]: re.findall(r'(\S+)(?<!0\.000000000000000000e\+00)\s+0\.000000000000000000e\+00', my_string)
Out[55]: ['1.440101653069986610e+26', '8.079362883576220633e+25']

Using negative lookahead assertion, the regex could be refined to improve performance, as mentioned in comment by @revo:

([-+]?\d\.(?!0+e\+0+)\S+)\s+(?:0\.0+e\+00\s*)+

Live demo

Answer 2

I also cannot assume that there's always an equal number of zeros grouped together.

How can we differentiate, say, 2 consecutive zero values from "a group of zeros" ?

Well, given you're looking for at least 5 0.000 patterns, you could use a non-capturing group on this multiple 0 pattern (to avoid matching it), following a non-blank pattern (for the number)

re.findall("(\S+)\s+(?:0\.0+e\+00\s+){5,}",my_string)

If there cannot be any zeroes except for the pattern itself, it can be generalized to:

re.findall("(\S+)\s+(?:0\.0+e\+00\s+)+",my_string)

(you need the + at the end of the non-capturing group to capture and discard all the zeroes)

result (in both cases):

['1.440101653069986610e+26', '8.079362883576220633e+25']

this also takes care of newlines, and is tolerant to variable number of zeroes in the decimal part

Answer 3

List comprehension and zip

This is about 10-70x times faster than the other solutions.

my_values = my_string.split()
output = [x for x,y in zip(my_values,my_values[1:]) 
           if (y == '0.000000000000000000e+00' and x != '0.000000000000000000e+00')]
print(output)

Or, with islice to save memory as kindly suggested by @Jean-François Fabre:

import itertools
my_values = my_string.split()
output = [x for x,y in zip(my_values,itertools.islice(myvalues,1,None)) 
               if (y == '0.000000000000000000e+00' and x != '0.000000000000000000e+00')]
print(output)

This works by grouping the elements in pairs (x,y). x should be different than 0.00.. while y should be equal to it. By doing the y check first this will evaluate fast to False in most cases and continue iterating. Returns:

['1.440101653069986610e+26', '8.079362883576220633e+25']

---

Pandas and numpy

However, another idea (which I would consider as smartest here) would be to use pandas and pd.to_numeric() . When you work with numbers you most likely want to use a library like numpy or pandas. This would be safer as you could also handle errors smoothly. Also note that I in both cases convert the numbers back to string (which you could skip).

import pandas as pd

data = pd.Series(pd.to_numeric(my_string.split()))
output = data[(data != 0) & (data.shift(-1) == 0)].astype(str).tolist()
print(output)

#['1.440101653069986610e+26', '8.079362883576220633e+25']

And numpy:

import numpy as np

data = np.loadtxt(my_string.split())
output = list(map(str,data[(data != 0) & (np.roll(ar, -1) == 0)]))
print(output)

#['1.440101653069986610e+26', '8.079362883576220633e+25']

---

Time comparison

fastest --> slowest

100000 loops, best of 3: 9.28 µs per loop  <-- Anton vBR list comprehension
10000 loops, best of 3: 98.4 µs per loop   <-- Revos Regex
1000 loops, best of 3: 256 µs per loop     <-- Anton vBR numpy
1000 loops, best of 3: 425 µs per loop     <-- Tzot Regex
1000 loops, best of 3: 513 µs per loop     <-- Jean-François Fabre Regex 
1000 loops, best of 3: 782 µs per loop     <-- liliscent 
1000 loops, best of 3: 794 µs per loop     <-- Anton vBR pandas

Answer 4

If you want the float values and not their string representations:

import re

list(
    filter(
        None,
        map(
            float,
            re.findall(r"\S+(?=\s0\.0+e)", my_string)
)))

• re.findall(r"\\S+(?=\\s0\\.0+e)", my_string) :
  finds all occurences of non-white-space character sequences before a white-space and 0.00000…e
• map(float, ^ ) :
  assume all of the above matches can be converted to float
• filter(None, ^ ) :
  filter out all zero floats
• list( ^ ) :
  make the above into a list (a no-op in Python 2, conversion of the generator into a list in Python 3)

Result:

>>> list(filter(None, map(float, re.findall(r"\S+(?=\s0\.0+e)", my_string))))
[1.4401016530699866e+26, 8.07936288357622e+25]

However, if you still want the string values themselves, let me know; in that case, the map & filter subexpressions need to be modified.