I have a DataFrame that loads from a huge json file and gets the schema from it. The schema is basically around 1000 columns. I want the same output of printSchema to be saved in a file instead of the console.
Any ideas?
You can do the following if you are working in a local environment :
val filePath = "/path/to/file/schema_file"
new PrintWriter(filePath) { write(df.schema.treeString); close }
If you are on HDFS, you'll need to provide a URI.
This is the body of printSchema():
/**
* Prints the schema to the console in a nice tree format.
* @group basic
* @since 1.3.0
*/
// scalastyle:off println
def printSchema(): Unit = println(schema.treeString)
// scalastyle:on println
So you can't do much, but I have a work around that can work in your case. Set the out stream to a file Stream so that it gets printed to your File.
Something like this
val out = new PrintStream(new FileOutputStream("output.txt"));
System.setOut(out);
I hope I solved your query !
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.