Technology

Working with Azure Data Factory Pipelines

Posted on

I am often tasked with automating tasks related to managing Azure Data Factory (ADF) pipelines. I am documenting below the PowerShell commands I frequently use to achieve the results I expect. Retrieve pipelines from a given factory

If you have multiple factories, you can append the results of another call in the same result variable

Similarly, there are commands to retrieve datasets and linked services

The hierarchy of ADF pipelines goes as follow (there may be variations to this pattern that I am not aware): Factory → Pipelines → Activities → DataSets → LinkedServices Looping on the hierarchy:

Much of the information retrieved by these commands are in complex types. Most of them contains a Type property […]

Technology

Service Health Check

Posted on

I spent some time back in Feb 2017 to implement a service health check for our WCF services. The idea was to attach a new endpoint for every real service endpoint, where the new endpoint would give a standard health check information. So, if you knew the service endpoint, the health check endpoint would be easily discovered from that. Microsoft has just released the Azure Monitor that provides monitoring telemetry for all cloud resources. Another tool for the toolbox! https://azure.microsoft.com/en-us/blog/announcing-the-general-availability-of-azure-monitor/