How to upload an asset from Azure Release pipeline to Github Enterprise using Bash

Question

Since the company I work for will not allow any non-Microsoft released packages from the Azure Marketplace, I need to create a bash script to be able to upload my built/published asset to the Github Enterprise repo. For some reason I always get redirected when I try to get the info of a release tag, be it a specific tag or latest, does not matter.

The following is my script:

set -e
xargs=$(which gxargs || which xargs)

# Validate settings.
[ "$TRACE" ] && set -x

CONFIG=$@

for line in $CONFIG; do
  eval "$line"
done

# Define variables.
GH_API="https://git.[company].com/api/v3"
GH_REPO="$GH_API/repos/[owner]/$(Build.Repository.Name)"
GH_TAGS="$GH_REPO/releases/tags/$(Build.SourceBranchName)"
AUTH="Authorization: token $github_api_token"
WGET_ARGS="--content-disposition --auth-no-challenge --no-cookie"
CURL_ARGS="-LJO#"
tag="$(Build.SourceBranchName)"
filename="BaseRepoName_$(Build.SourceBranchName)_$(Build.BuildId).zip"
echo "tag is: $tag"
echo $AUTH
echo "Repo: $GH_REPO"

if [[ "$tag" == 'LATEST' ]]; then
  GH_TAGS="$GH_REPO/releases/latest"
fi
echo "Tags url:  $GH_TAGS"

echo "Validate token ..."
# Validate token.
curl -o /dev/null -sH "$AUTH" $GH_REPO || { echo "Error: Invalid repo, token or network issue!";          exit 1; }

echo "Get api endpoints"
apiresponse=$(curl -sH "$AUTH" "$GH_API")
echo "API:  $apiresponse"

echo "Read asset tags: curl -sH "$AUTH" $GH_TAGS"
# Read asset tags.
response=$(curl -sH "$AUTH" $GH_TAGS)
echo "Response: $response"

## In case of success, we upload a file
upload=$(echo $succ | grep upload_url)
if [[ $? -eq 0 ]]; then
    echo Release created.
else
    echo Error creating release!
    return
fi

echo "Get the upload url for the given tag"
upload=$(echo $upload | cut -d "\"" -f4 | cut -d "{" -f1)
upload="$upload?name=$filename"

# Upload asset
echo "Uploading asset... "
succ=$(curl -H "Authorization: token $perstok" \
     -H "Content-Type: $(file -b --mime-type $filename)" \
     --data-binary @$filename $upload)

The logs of the Bash task:

2020-03-13T05:53:35.4274045Z ##[section]Starting: Bash Script
2020-03-13T05:53:35.4654068Z ==============================================================================
2020-03-13T05:53:35.4654773Z Task         : Bash
2020-03-13T05:53:35.4655254Z Description  : Run a Bash script on macOS, Linux, or Windows
2020-03-13T05:53:35.4655535Z Version      : 3.163.1
2020-03-13T05:53:35.4656320Z Author       : Microsoft Corporation
2020-03-13T05:53:35.4656864Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/bash
2020-03-13T05:53:35.4657312Z ==============================================================================
2020-03-13T05:53:36.4699984Z Generating script.
2020-03-13T05:53:36.4703019Z [command]"C:\Program Files\Git\bin\bash.exe" --noprofile --norc -c pwd
2020-03-13T05:53:36.8704279Z /d/a/_temp
2020-03-13T05:53:36.8755504Z 
2020-03-13T05:53:36.8794400Z ========================== Starting Command Output ===========================
2020-03-13T05:53:36.8801355Z [command]"C:\Program Files\Git\bin\bash.exe" --noprofile --norc /d/a/_temp/c7a40af4-c1f2-4127-a2cf-4aa11ac19e48.sh
2020-03-13T05:53:36.9965374Z which: no gxargs in (/mingw64/bin:/usr/bin:/c/Users/VssAdministrator/bin:/c/hostedtoolcache/windows/Python/3.6.8/x64:/c/hostedtoolcache/windows/Python/3.6.8/x64/Scripts:/c/Program Files/Mercurial:/c/ProgramData/kind:/c/vcpkg:/c/cf-cli:/c/Program Files (x86)/NSIS:/c/Program Files/Mercurial:/c/hostedtoolcache/windows/Boost/1.69.0:/c/Program Files/dotnet:/c/mysql-5.7.21-winx64/bin:/c/Program Files/Java/zulu-8-azure-jdk_8.40.0.25-8.0.222-win_x64/bin:/c/SeleniumWebDrivers/GeckoDriver:/c/Program Files (x86)/sbt/bin:/c/Rust/.cargo/bin:/c/hostedtoolcache/windows/Ruby/2.5.7/x64/bin:/c/Go1.12.7/bin:/bin:/c/hostedtoolcache/windows/Python/3.6.8/x64/Scripts:/c/hostedtoolcache/windows/Python/3.6.8/x64:/c/npm/prefix:/c/Program Files (x86)/Microsoft SDKs/Azure/CLI2/wbin:/c/Program Files/Microsoft MPI/Bin:/c/windows/system32:/c/windows:/c/windows/System32/Wbem:/c/windows/System32/WindowsPowerShell/v1.0:/c/ProgramData/Chocolatey/bin:/c/Program Files/Docker:/c/Program Files/PowerShell/7:/c/Program Files/dotnet:/c/Program Files/Microsoft SQL Server/130/Tools/Binn:/c/Program Files (x86)/Microsoft SQL Server/110/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/120/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/130/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/140/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/150/DTS/Binn:/c/Program Files (x86)/Windows Kits/10/Windows Performance Toolkit:/c/Program Files/Microsoft Service Fabric/bin/Fabric/Fabric.Code:/c/Program Files/Microsoft SDKs/Service Fabric/Tools/ServiceFabricLocalClusterManager:/c/Program Files/nodejs:/c/Strawberry/c/bin:/c/Strawberry/perl/site/bin:/c/Strawberry/perl/bin:/cmd:/mingw64/bin:/usr/bin:/c/tools/php:/c/Program Files (x86)/sbt/bin:/c/Program Files (x86)/Subversion/bin:/c/SeleniumWebDrivers/ChromeDriver:/c/SeleniumWebDrivers/EdgeDriver:/c/ProgramData/chocolatey/lib/maven/apache-maven-3.6.3/bin:/c/Program Files/CMake/bin:/c/Program Files/OpenSSL/bin:/c/Users/VssAdministrator/.dotnet/tools:/c/Program Files (x86)/Microsoft SQ)
2020-03-13T05:53:37.0042653Z tag is: MBP_TestTag6
2020-03-13T05:53:37.0043260Z Authorization: token [Edited: secret]
2020-03-13T05:53:37.0043852Z Repo: https://git.[company].com/api/v3/repos/[owner]/[MyTestRepo]
2020-03-13T05:53:37.0044723Z Tags url:  https://git.[company].com/api/v3/repos/[owner]/[MyTestRepo]/releases/tags/MBP_TestTag6
2020-03-13T05:53:37.0045218Z Validate token ...
2020-03-13T05:53:38.5703717Z Get api endpoints
2020-03-13T05:53:38.8038551Z API:  {
2020-03-13T05:53:38.8050142Z   "current_user_url": "https://git.[company].com/api/v3/user",
2020-03-13T05:53:38.8053206Z   "current_user_authorizations_html_url": "https://git.[company].com/settings/connections/applications{/client_id}",
2020-03-13T05:53:38.8060689Z   "authorizations_url": "https://git.[company].com/api/v3/authorizations",
2020-03-13T05:53:38.8064874Z   "code_search_url": "https://git.[company].com/api/v3/search/code?q={query}{&page,per_page,sort,order}",
2020-03-13T05:53:38.8071647Z   "commit_search_url": "https://git.[company].com/api/v3/search/commits?q={query}{&page,per_page,sort,order}",
2020-03-13T05:53:38.8073059Z   "emails_url": "https://git.[company].com/api/v3/user/emails",
2020-03-13T05:53:38.8074445Z   "emojis_url": "https://git.[company].com/api/v3/emojis",
2020-03-13T05:53:38.8075767Z   "events_url": "https://git.[company].com/api/v3/events",
2020-03-13T05:53:38.8077032Z   "feeds_url": "https://git.[company].com/api/v3/feeds",
2020-03-13T05:53:38.8078149Z   "followers_url": "https://git.[company].com/api/v3/user/followers",
2020-03-13T05:53:38.8079345Z   "following_url": "https://git.[company].com/api/v3/user/following{/target}",
2020-03-13T05:53:38.8080572Z   "gists_url": "https://git.[company].com/api/v3/gists{/gist_id}",
2020-03-13T05:53:38.8081548Z   "hub_url": "https://git.[company].com/api/v3/hub",
2020-03-13T05:53:38.8082418Z   "issue_search_url": "https://git.[company].com/api/v3/search/issues?q={query}{&page,per_page,sort,order}",
2020-03-13T05:53:38.8083431Z   "issues_url": "https://git.[company].com/api/v3/issues",
2020-03-13T05:53:38.8084430Z   "keys_url": "https://git.[company].com/api/v3/user/keys",
2020-03-13T05:53:38.8086272Z   "label_search_url": "https://git.[company].com/api/v3/search/labels?q={query}&repository_id={repository_id}{&page,per_page}",
2020-03-13T05:53:38.8091275Z   "notifications_url": "https://git.[company].com/api/v3/notifications",
2020-03-13T05:53:38.8092166Z   "organization_repositories_url": "https://git.[company].com/api/v3/orgs/{org}/repos{?type,page,per_page,sort}",
2020-03-13T05:53:38.8093095Z   "organization_url": "https://git.[company].com/api/v3/orgs/{org}",
2020-03-13T05:53:38.8096487Z   "public_gists_url": "https://git.[company].com/api/v3/gists/public",
2020-03-13T05:53:38.8097620Z   "rate_limit_url": "https://git.[company].com/api/v3/rate_limit",
2020-03-13T05:53:38.8098584Z   "repository_url": "https://git.[company].com/api/v3/repos/{owner}/{repo}",
2020-03-13T05:53:38.8100945Z   "repository_search_url": "https://git.[company].com/api/v3/search/repositories?q={query}{&page,per_page,sort,order}",
2020-03-13T05:53:38.8102239Z   "current_user_repositories_url": "https://git.[company].com/api/v3/user/repos{?type,page,per_page,sort}",
2020-03-13T05:53:38.8104230Z   "starred_url": "https://git.[company].com/api/v3/user/starred{/owner}{/repo}",
2020-03-13T05:53:38.8104831Z   "starred_gists_url": "https://git.[company].com/api/v3/gists/starred",
2020-03-13T05:53:38.8105328Z   "team_url": "https://git.[company].com/api/v3/teams",
2020-03-13T05:53:38.8105772Z   "user_url": "https://git.[company].com/api/v3/users/{user}",
2020-03-13T05:53:38.8106279Z   "user_organizations_url": "https://git.[company].com/api/v3/user/orgs",
2020-03-13T05:53:38.8106871Z   "user_repositories_url": "https://git.[company].com/api/v3/users/{user}/repos{?type,page,per_page,sort}",
2020-03-13T05:53:38.8107491Z   "user_search_url": "https://git.[company].com/api/v3/search/users?q={query}{&page,per_page,sort,order}"
2020-03-13T05:53:38.8108556Z }
2020-03-13T05:53:38.8109344Z Read asset tags: curl -sH Authorization: token [Edited: Secret] https://git.[company].com/api/v3/repos/[owner]/[MyTestRepo]/releases/tags/MBP_TestTag6
2020-03-13T05:53:39.0378180Z Response: {
2020-03-13T05:53:39.0379040Z   "message": "Not Found",
2020-03-13T05:53:39.0379599Z   "documentation_url": "https://developer.github.com/enterprise/2.20/v3/repos/releases/#get-a-release-by-tag-name"
2020-03-13T05:53:39.0380100Z }
2020-03-13T05:53:39.2992296Z 
2020-03-13T05:53:39.3060923Z ##[error]Bash exited with code '1'.
2020-03-13T05:53:39.3120104Z ##[section]Finishing: Bash Script

As you can see from the logs it always fails when calling the api to get the info of the supplied tag (or even latest tag).

Any idea why the system is trying to redirect me?

Answer 1

You should make changes on this line:

GH_API="https://git.[company].com/api/v3"

To

GH_API="https://api.github.com"

Answer 2

The problem that I experienced has to do with the way Git handles releases. In this case there was a Pre-Release (Release name/Tag: v0.0.1), which is what I got back when querying for all releases. As soon as I deleted this pre-release I got back nothing, but then realized that none of the other 'releases' were ever published.

After publishing a couple of releases I then finally got back a list of published releases. All other commands then worked as expected as well.

So, no sinister problems or bugs. Just some minor mistakes from a DevOps pipeline newbie.