如何使用文本文件中的列名在数据块中创建增量表的模式

Question

I have a text file which stores all the column names for ex: text file contains following data

table1=['column1','2_column2','3_column3']
table2=['column4','5_column5','6_column6']

I need to fetch all the values and create a schema for tables mentioned in the text file. and also some of the the columns starts with number as well as mentioned above. output needed:

table1 = StructType([
        StructField("column1", StringType(), True),
        StructField("2_column2", StringType(), True),
        StructField("3_column3", StringType(), True)
    ]

table2 = StructType([
        StructField("column4", StringType(), True),
        StructField("5_column5", StringType(), True),
        StructField("6_column6", StringType(), True)
    ]

all the columns will be string type .

how to achieve this using python/pyspark?

Answer 1

table1 = ['column1', '2_column2', '3_column3']
data = []
for i in table1:
    data.append(f'StructField({i}, StringType(), True)')
table1 = f'StructType({data})'
print(table1)
>>> StructType(['StructField(column1, StringType(), True)', 'StructField(2_column2, StringType(), True)', 'StructField(3_column3, StringType(), True)'])