The Question is:
Complete the writeToBronze
function to perform the following tasks:
gamingEventDF
-- the stream defined above -- to a bronze Delta table in path defined by outputPathBronze
.client_event_time
to a date format and rename the column to eventDate
eventDate
columnCode:
def writeToBronze(sourceDataframe, bronzePath, streamName):
(sourceDataframe
.withColumn("eventDate",
to_date(col("eventParams.client_event_time"), "yyyy-MM-dd"))
.filter(col("eventDate").isNotNull())
.writeStream
.format("delta")
.option("checkpointLocation", f"{bronzePath}_checkpoint")
.queryName(streamName)
.outputMode("append")
.start(outputPathBronze)
)
writeToBronze(gamingEventDF, outputPathBronze, "bronze_stream")
I was stuck on this too but have just figured it out In the provided template for this capstone cell they have included the line: .start(outputPathBronze) This is referring to a parameter that is outside of the function, not the path passed in as an argument. The reality check passes a different path in that variable. If you change this line to use the variable bronzePath you should be able to get past that step (now I have to get my count to match their expected count...)
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.