Get zipped TFS 2015 (vNext) build output logs through powershell (just like the download link after the build)

Question

I'm wondering if anyone has a PowerShell script to either download all the current build logs for this build id (up to the current step) through the Rest API for TFS 2015 (vNext), create separate text files for each logged build step, and zip all the text files?

Or, if there is already a way to get a download URL to do what the "Download all logs as zip" link already does (after a build), how can I get it in a PowerShell script?

I can probably do this myself given a little time, but I thought I'd ask, because someone must have this functionality already, and if you do, please share it.

Answer 1

Answer to my own post:

Ok, so I went ahead and spent the time to write a powershell script that does exactly what I wanted it to do, which essentially duplicates functionality of the "Download all logs as zip" link that TFS 2015 (vNext) does.

You can either put it into a step in a running build, or modify it to run as a post build step in a build definition after the running build.

The difference being that in the running build, it will only give you logs up to the step before this powershell script step. A post build definition with this powershell script allows you to get all of the logs.

I chose not to use a post build definition since this step was one of the last steps in my running build definition, and the steps that followed were inconsequential to me to have log information for (eg My last 2 steps are CreateZippedLogs.ps1 and Copy the Zipped Logs to a file server).

FYI, I'm using on-premise TFS 2015 (Update 3) as of this writing. Replace the -UseDefaultCredentials in the Invoke-RestMethod lines with your appropriate authorization credentials, suitable for your situation.

Here is CreateZippedLogs.ps1, modify it to your liking, and enjoy:

[CmdletBinding()]
param()

# Create base URI for the REST API
$baseURI = $Env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI + $Env:SYSTEM_TEAMPROJECT + "/_apis/"

# Get Build ID from the environment variable
$buildID = $Env:BUILD_BUILDID

# Build Timeline URI for $Env:BUILD_BUILDID
$buildTimelineURI = "$baseURI/build/builds/$buildID/timeline`?api-version=2.0"

# Output intent to create Zipped Log content for this build id
Write-Output "Attempting to create Zipped Log content for Build ID $buildID from URL: ""$buildTimelineURI"""

# Get sorted build timeline (ascending by default)
$buildTimeline = (Invoke-RestMethod -Uri $buildTimelineURI -UseDefaultCredentials).records | Sort-Object { $_.log.id }

# Get log count
$logsCount = $buildTimeline.Count  # the number of timeline records (i.e. logs in the array)

# Remove sub-staging folder recursively for the logs
# Note: $Env:My_BuildLabel is defined as "$Env:BUILD_DEFINITIONNAME`_$Env:BUILD_BUILDNUMBER" for my builds; use whatever name is appropriate for your builds
$singleLogFileFolder = "$Env:BUILD_ARTIFACTSTAGINGDIRECTORY\$Env:My_BuildLabel`_Logs"
Remove-Item "$singleLogFileFolder" -recurse -ErrorAction SilentlyContinue -Verbose

# Remove any previously created Zip file with the same filename that we are about to create
Remove-Item "$singleLogFileFolder.zip" -ErrorAction SilentlyContinue -Verbose

# If number of logs in the array is > 0, create sub-staging folders for the logs
if ($logsCount -gt 0)
{
    # First, a top-level folder to hold all logs in a single .txt file
    New-Item -ItemType directory -Path "$singleLogFileFolder" -ErrorAction Stop -Verbose
    $stepLogsSingleFile = "$singleLogFileFolder\1_Build.txt"

    # Second, a subfolder to hold individual build step log files
    $stepLogFilesFolder = "$singleLogFileFolder\Build"
    New-Item -ItemType directory -Path "$stepLogFilesFolder" -ErrorAction Stop -Verbose
}
else
{
    # Output that there were no logs found for this Build ID
    Write-Output "No logs found for Build ID $buildID"
    Exit 0  # Exit successfully
}

# Get list of invalid filename characters (in a regex string)
$invalidFilenameCharactersRegexString = "[{0}]+" -f [regex]::Escape(([System.IO.Path]::GetInvalidFileNameChars() -join ""))

# Output progress
Write-Output "Getting Log content and saving to files:"

# Iterate through each record in the build timeline array
foreach ($timelineRecord in $buildTimeline)
{
    # Extract url for each log
    $logURL = $timelineRecord.log.url

    # Don't try to get empty log URL's
    if ([string]::IsNullOrWhiteSpace($logURL))
    {
        continue
    }

    # Get log content
    $logContent = (Invoke-RestMethod -Uri $logURL -UseDefaultCredentials).value

    # Append step log output to the single file (not -Verbose because it duplicates information)
    $logContent | Out-File "$stepLogsSingleFile" -Append

    # Get log id
    $logID = $timelineRecord.log.id

    # Remove any invalid filename characters from the step name
    $stepName = $timelineRecord.name -replace "$invalidFilenameCharactersRegexString", ""

    # Create a step log output filename appropriate to the step number and name
    $stepLogFile = "$stepLogFilesFolder\$logID`_$stepName`.txt"

    # Save the step log content
    $logContent | Out-File "$stepLogFile" -Verbose
}

Write-Output "Compressing Log files:"
if ($PSVersionTable.PSVersion.Major -lt 5)
{
    Add-Type -assembly "system.io.compression.filesystem"
    [io.compression.zipfile]::CreateFromDirectory($singleLogFileFolder, $singleLogFileFolder + ".zip")
}
else
{
    Compress-Archive "$singleLogFileFolder\*" -DestinationPath "$singleLogFileFolder.zip" -Verbose
}

# ----------------------------------------------------------

# Remove temporary sub-staging folder recursively for the logs (don't leave it hanging around)
Write-Output "Removing temporary Log folders and files:"
Remove-Item "$singleLogFileFolder" -recurse -ErrorAction SilentlyContinue -Verbose

Answer 2

You can use TFS REST API in your script to get Build Logs directly:

GET https://{instance}/DefaultCollection/{project}/_apis/build/builds/{buildId}/logs

instance: VS Team Services account ({account}.visualstudio.com) or TFS server ({server:port}).

Answer 3

There isn't any way to download the zipped logs directly, you need to get them via Rest API and save it to text files and then zip the files. Attach my code for your reference which get the entire logs and save it to one text file:

$buildID = "12345"
$project = "ProjectName"
$projecturi = "https://xxxxxxxxxx/"

# Setting up basic authentication
$username="username"
$password="password"

$basicAuth= ("{0}:{1}"-f $username,$password)
$basicAuth=[System.Text.Encoding]::UTF8.GetBytes($basicAuth)
$basicAuth=[System.Convert]::ToBase64String($basicAuth)
$headers= @{Authorization=("Basic {0}"-f $basicAuth)}

$url= $projecturi + $project + "/_apis/build/builds/" + $buildID + "/logs?api-version=2.0"

$responseBuild = Invoke-RestMethod -Uri $url -headers $headers -Method Get | select value

$logid = $responseBuild.value.Count - 1;

$logurl = $projecturi + $project + "/_apis/build/builds/" + $buildID + "/logs/" + $logid;

$getlog = Invoke-RestMethod -Uri $logurl -headers $headers -Method Get 

$getlog | Out-File "D:\Temp\$buildID.txt"