Azure Data Factory V2 exploring pipeline dependencies

Question

I am working with quite a lot of pipelines, and with that involves a lot of dependencies between pipelines.

This isn't ideal for a couple reasons:

• It gets harder to know if you change something in one pipeline, what other pipelines could be affected
• Being able to document the overall data factory structure

Ideally I should be able to "select" a random pipeline and be able to know what pipelines dependencies it has for both before and after execution.

I was thinking about using the Data Factory SDK's to try and build the dependency structure of all my pipelines. But thought I would chuck this out there to see if anyone has discovered any solutions for this, or have any ideas before going down a rabbit hole.

I appreciate any advice.

Cheers, Brendan

Answer 1

Brendan, our ADF is connected to git and so when I need to know what will be affected if I change the pipeline with say name somePipelineName , I goto git bash and type out

grep --color=always -4 "somePipelineName" * 

on the pipelines folder

This helps me find all places from where the pipeline may be called.

Update: 2020-09-17

I noticed today that we now have the related pipelines listing