Spark：如何迭代 TransformedDStream、DataFrame、RDD、DStream？

Question

I am getting streams of data from a TCP port which includes date timestamp info1 info2. The data is in one line and I want to split it.

I tried to iterate through the DStream. I am getting the following error:

TypeError: 'DStream' object is not iterable

Then I tried to split and create the dataframe. I am getting the following error:

TypeError: 'TransformedDStream' object is not iterable

I also tried to store the stream of data directly in the dataframe but I am getting the error:

'DataFrame' object is not 'iterable'

The end result I want is to get one column for date, one column for timestamp, one column for info1, and one column for info2 and perform SQL queries on the data.

I would really appreciate the help.

Answer 1

I am getting streams of data from a TCP port which includes date timestamp info1 info2. The data is in one line and I want to split it.

I tried to iterate through the DStream. I am getting the following error:

TypeError: 'DStream' object is not iterable

Then I tried to split and create the dataframe. I am getting the following error:

TypeError: 'TransformedDStream' object is not iterable

I also tried to store the stream of data directly in the dataframe but I am getting the error:

'DataFrame' object is not 'iterable'

The end result I want is to get one column for date, one column for timestamp, one column for info1, and one column for info2 and perform SQL queries on the data.

I would really appreciate the help.