Technology

Working with Azure Data Factory Pipelines

Posted on

I am often tasked with automating tasks related to managing Azure Data Factory (ADF) pipelines. I am documenting below the PowerShell commands I frequently use to achieve the results I expect. Retrieve pipelines from a given factory

If you have multiple factories, you can append the results of another call in the same result variable

Similarly, there are commands to retrieve datasets and linked services

The hierarchy of ADF pipelines goes as follow (there may be variations to this pattern that I am not aware): Factory → Pipelines → Activities → DataSets → LinkedServices Looping on the hierarchy:

Much of the information retrieved by these commands are in complex types. Most of them contains a Type property […]