create a large dataset using Python

Question

I want to create a large dataset (that conforms to a given schema) using Python. Is there a nice way to specify the schema (datatype & length of each of the fields), and let Python create about 100,000 observations for me? Any nice tools already there?

I am familiar with Python...hence would like to stick with it. if there is one using Bash or any other way, please let me know as well.

Thanks! PD.

Answer 1

您可能应该检查一下假工厂包装。

Answer 2

Please have a look at this:-

https://github.com/sanju51/Generate-large-Dataset-dynamically-in-Python

SPEED:- 100000 records in 5 seconds(10 columns) USAGE:- python generate_dataset.py -i Metadata.csv -f sample.csv -nrec 100000 -d ',' -hdr Y