简体   繁体   English

使用 powershell 如何通过运行 ID 在 Azure 数据工厂中重新运行失败的管道?

[英]Using powershell how do you rerun failed pipeline in Azure Data Factory by run ID?

In Azure Data factory I need to re-run around 4000+ failed pipeline.在 Azure 数据工厂中,我需要重新运行大约 4000 多个失败的管道。 It is possible to do in the Azure portal UI but I am try to automate the run-run process in powershell.可以在 Azure 门户 UI 中执行此操作,但我尝试在 powershell 中自动执行运行-运行过程。

I am not able to find the command/steps in powershell to ru-run a failed pipeline by the run ID.我无法在 powershell 中找到命令/步骤来通过运行 ID 运行失败的管道。

I found this question while searching for a solution to the same problem, so here's an answer with what I've found.我在寻找相同问题的解决方案时发现了这个问题,所以这是我找到的答案。

If you want to re-run the entire pipeline and you don't care about it being technically considered a re-run by Data Factory, you could use the Get-AzDataFactoryV2PipelineRun cmdlet to get a list of runs, filter to the failed runs, then use the same parameters in a call to Invoke-AzureRmDataFactoryV2Pipeline如果您想重新运行整个管道并且您不关心它在技术上被数据工厂视为重新运行,您可以使用Get-AzDataFactoryV2PipelineRun cmdlet 获取运行列表,过滤到失败的运行,然后在调用Invoke-AzureRmDataFactoryV2Pipeline使用相同的参数

It looks like fairly soon that cmdlet will be updated to allow a true re-run (based on the response to this issue someone raised on the Visually monitor Azure Data Factory doc from Microsoft).似乎很快就会更新 cmdlet 以允许真正重新运行(基于有人在 Microsoft 的Visually monitor Azure 数据工厂文档中提出的对这个问题的回应)。

If you're in a hurry to be able to do a true re-run, the functionality is already included in the REST API by calling createRun with some optional parameters.如果您急于能够真正重新运行,那么通过使用一些可选参数调用createRun ,该功能已包含在REST API 中

Edit: This was added to the Azure PowerShell module v4.8.0 released in October 2020 ( docs ).编辑:这已添加到 2020 年 10 月发布的 Azure PowerShell 模块 v4.8.0 ( docs )。 You can now pass a pipeline run ID to the Invoke-AzDataFactoryV2Pipeline cmdlet using -ReferencePipelineRunId to have it use the parameters from that run.现在,您可以通过管道运行ID的Invoke-AzDataFactoryV2Pipeline使用cmdlet的-ReferencePipelineRunId有它使用的参数从运行。 You can also use the -StartFromFailure switch to have it only rerun the failed activities.您还可以使用-StartFromFailure开关让它只重新运行失败的活动。

I just find a tutorial Azure Data Factory: Detecting and Re-Running failed ADF Slices , it provides you the Powershell script to automate failed pipeline.我刚刚找到了一个教程Azure 数据工厂:检测和重新运行失败的 ADF 切片,它为您提供了 Powershell 脚本来自动化失败的管道。

Powershell script: Powershell 脚本:

Login-AzureRmAccount
$slices= @()
$tableName=@()
$failedSlices= @()
$failedSlicesCount= @()
$tableNames=@()

$Subscription="Provide Subscription ID"  

  Select-AzureRMSubscription -SubscriptionId  $Subscription    
$DataFactoryName="Provide Data Factory Name"
$resourceGroupName ="Porvide Resource Group Name for Data Factory"

$startDateTime ="2015-05-01" #Start Date for Slices
$endDateTime="2015-08-01" # End Date for Slices


#Get Dataset names in Data Factory - you can exlicitly give a table name using $tableName variable if you like to run only for an individual tablename
$tableNames = Get-AzureRMDataFactoryDataset -DataFactoryName $DataFactoryName -ResourceGroupName $resourceGroupName | ForEach {$_.DatasetName}

$tableNames #lists tablenames

foreach ($tableName in $tableNames)
{
    $slices += Get-AzureRMDataFactorySlice -DataFactoryName $DataFactoryName -DatasetName $tableName -StartDateTime $startDateTime -EndDateTime $endDateTime -ResourceGroupName $resourceGroupName -ErrorAction Stop
}


$failedSlices = $slices | Where {$_.Status -eq 'Failed'}

$failedSlicesCount = @($failedSlices).Count

if ( $failedSlicesCount -gt 0 ) 
{

    write-host "Total number of slices Failed:$failedSlicesCount"
    $Prompt = Read-host "Do you want to Rerun these failed slices? (Y | N)" 
    if ( $Prompt -eq "Y" -Or $Prompt -eq "y" )
    {

        foreach ($failed in $failedSlices)
        {
            write-host "Rerunning slice of Dataset "$($failed.DatasetName)" with StartDateTime "$($failed.Start)" and EndDateTime "$($failed.End)"" 
            Set-AzureRMDataFactorySliceStatus -UpdateType UpstreamInPipeline -Status Waiting -DataFactoryName $($failed.DataFactoryName) -DatasetName $($failed.DatasetName) -ResourceGroupName $resourceGroupName -StartDateTime "$($failed.Start)" -EndDateTime "$($failed.End)" 


        }
    }

}
else
{
    write-host "There are no Failed slices in the given time period."
}

Hope this helps.希望这可以帮助。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 如何使用PowerShell(Azure Automation)触发Azure Data Factory管道? - How to trigger Azure Data Factory pipeline using PowerShell (Azure Automation)? 如何在 Azure 数据工厂中下载失败的管道运行? - How do I download failed pipeline runs in Azure Data Factory? 如何在不使用触发器的情况下重新运行 Azure 数据工厂? - How to rerun an Azure Data Factory without using triggers? 如何使用 Azure 数据工厂 Web 活动运行 powershell 命令 - How to run a powershell command using Azure Data Factory web Activity 使用 powershell 从 azure 数据工厂中的管道中删除触发器引用 - Remove trigger reference from pipeline in azure data factory using powershell 如何在管道执行之间将运行时数据存储在 Azure 数据工厂中? - How do I store run-time data in Azure Data Factory between pipeline executions? 使用PowerShell进行Azure数据工厂管道部署 - Azure Data Factory Pipeline deployment with PowerShell 无法使用Visual Studio 2015运行Azure数据工厂管道 - Not able to run Azure Data Factory Pipeline using Visual Studio 2015 失败的数据工厂管道在重新运行时是否保留其原始输入? - Do failed Data Factory pipelines keep their original input when rerun? 如何在 Azure 数据工厂中的触发器运行期间将参数传递给管道? - How to pass parameters to pipeline during trigger run in Azure Data Factory?
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM