Is it possible to get lineage metadata from the pipeline in my Data Fusion Action plugin?

Question

I'm trying to get data lineage metadata like data source/schema and data target/schema in a custom Action plugin which gets executed after the successful run of the other steps in the pipeline.

I have a basic Action plugin that executes but I'm having trouble finding a way to get the metadata I'm after.

The use case I'm working on is pushing data lineage into a third party data governance tool.

I would very much appreciate if someone could point me in the right direction!

Answer 1

As was suggested in my comment, you might consider to use CDAP system metadata inventory to extract the particular property for the desired entity via CDAP existed RESTfull API methods by sending appropriate HTTP request as explained in CDAP Metadata Microservices documentation. Said this entity properties can also depict lineage of dataset fields returning the result in JSON format.

However, adjusting appropriate HTTP method mostly depends on the particular use case, therefore feel free to further contribute and share your further discovering.