numpy 字符串数组比 python 字符串快吗

Question

I am creating a string that is about 30 million words long. As you can imagine, this takes absolutely forever to create with a for-loop increasing by about 100 words at a time. Is there a way to represent the string in a more memory-friendly way, like a numpy array? I have very little experience with numpy.

bigStr = ''
for tweet in df['text']:
  bigStr = bigStr + ' ' + tweet
len(bigStr)

Answer 1

I can see you're trying to get the length of all data. For that you don't need to append all strings. (And I see you add a white space for each element)

Just get the length of tweet and add it to an integer variable (+1 for each white space):

number_of_texts = 0
for tweet in df['text']:
  number_of_texts += 1 + len(tweet)

print(number_of_texts)

Answer 2

如果要构建字符串，请使用' '.join ，它将在 O(n) 时间内创建最终字符串，而不是一次构建一个字符串，这需要 O(n^2) 时间。

bigStr = ' '.join([tweet for tweet in df['text']])