Azure DevOps REST API top parameter in PowerShell script is not working

Question

I have below script to generate releases report from Azure DevOps project.

$token="**************************************************************"

$url="https://dev.azure.com/{orgnization}/_apis/projects?api-version=6.0"

$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))



$response = Invoke-RestMethod -Uri $url -Headers @{Authorization = "Basic $token"} -Method Get -ContentType application/json

Foreach($projectName in $response.value.name)
{
  
  $url1="https://vsrm.dev.azure.com/{orgnization}/$($projectname)/_apis/release/releases?api-version=6.0"
 
  $response = Invoke-RestMethod -Uri $url1 -Headers @{Authorization = "Basic $token"} -Method Get -ContentType application/json-patch

  
   echo $response | ConvertTo-Json -Depth 99 |  Out-File "D:\\file.json" -Append

}

In this script the first API is returning just 100 record. When I tried to add top parameter to return more record like below, it is not returing anything. Am I doing something wrong here?

https://dev.azure.com/uniperteamservices/_apis/projects?api-version=6.0&$top=500

Can you suggest how can I add top parameter in REST API url which can run in my above PowerShell script?

Answer 1

Be sure to escape the $ in the query string, otherwise PowerShell will try to inject the value of whatever variable called top holds:

$urlBase="...?api-version=6.0&`$top=5"
                              ^^^^^

$top= has been tricky at times. It may or may not give you the requested number of items, it seems to depend on how busy the backend is. But it does promise you this: All of the REST API's will return a x-ms-continuation-token header next to the payload and you can use this to fetch the next batch of items:

You can fetch the next batch of projects by requesting the exact same REST query and adding the &continuationtoken=${ms-continuation-token} query parameter to the call. Repeat this until the server stops sending the x-ms-continuation-token header, which signifies you've gotten all the values you were after.

You can grab the headers by passing in a second variable:

$urlBase="https://dev.azure.com/jessehouwing/_apis/projects?api-version=6.0&`$top=5"

$url = $urlBase
$results = @();

do 
{
    write-host "Calling API"
    $response = Invoke-RestMethod -Uri $url -Headers @{Authorization = "Basic $token"} -Method Get -ContentType application/json -ResponseHeadersVariable headers
    $results += $response.value

    if ($headers["x-ms-continuationtoken"])
    {
        $continuation = $headers["x-ms-continuationtoken"]
        write-host "Token: $continuation"
        $url = $urlBase + "&continuationtoken=" + $continuation
    }
} while ($headers["x-ms-continuationtoken"])

$results

Will show:

Calling API
Token: 5
Calling API
Token: 10
Calling API

id             : 06cb494c-b535-4f16-9323-bd0f63c38163
name           : CMMI
url            : https://dev.azure.com/jessehouwing/_apis/projects/06cb494c-b535-4f16-9323-bd0f63c38163
state          : wellFormed
revision       : 414360096
visibility     : private
lastUpdateTime : 08/07/2019 20:19:09

id             : 88a7ca79-b5c8-41d2-99e7-a5578a1df424
name           : BattleJSip
description    : Battleship case in JS/TS
url            : https://dev.azure.com/jessehouwing/_apis/projects/88a7ca79-b5c8-41d2-99e7-a5578a1df424
state          : wellFormed
revision       : 414360059
visibility     : private
lastUpdateTime : 15/04/2018 13:43:44

... for 12 projects

In case of the /projects endpoint the continuation token takes on a predictable value of the number of projects to skip. Other API's may return a GUID or a base64 encoded string or the ID of the last returned item. Make no assumptions about the contents of the token, always copy it from the header and put it into the next request verbatim.

eg

https://dev.azure.com/p/_apis/projects?api-version=6.0&$top=5

x-ms-continuation-token:5

{"count":5,"value":[{"id":"06cb494c-b535-4f16-9323-bd0f63c38163","name":"CMMI","url":"https://dev.azure.com/jessehouwing/_apis/projects/06cb494c-b535-4f16-9323-bd0f63c38163","state":"wellFormed","revision":414360096,"visibility":"private","lastUpdateTime":"2019-07-08T20:19:09.333Z"},{"id":"88a7ca79-b5c8-41d2-99e7-a5578a1df424","name":"BattleJSip","description":"Battleship case in JS/TS","url":"https://dev.azure.com/jessehouwing/_apis/projects/88a7ca79-b5c8-41d2-99e7-a5578a1df424","state":"wellFormed","revision":414360059,"visibility":"private","lastUpdateTime":"2018-04-15T13:43:44Z"},{"id":"f5ffbb7d-11bc-4e2f-93e0-e9ee8151b428","name":"CodeToCloud-Workshop","url":"https://dev.azure.com/jessehouwing/_apis/projects/f5ffbb7d-11bc-4e2f-93e0-e9ee8151b428","state":"wellFormed","revision":414360127,"visibility":"private","lastUpdateTime":"2020-10-01T18:21:03.913Z"},{"id":"6d4b20e3-4afc-4fb0-9dc9-f4b1d3ff150d","name":"Agile2017","url":"https://dev.azure.com/jessehouwing/_apis/projects/6d4b20e3-4afc-4fb0-9dc9-f4b1d3ff150d","state":"wellFormed","revision":414360110,"visibility":"private","lastUpdateTime":"2019-10-09T11:52:58.767Z"},{"id":"2031f1a3-c549-46f1-8c55-74390077a606","name":"jessehouwing.net","url":"https://dev.azure.com/jessehouwing/_apis/projects/2031f1a3-c549-46f1-8c55-74390077a606","state":"wellFormed","revision":414360067,"visibility":"private","lastUpdateTime":"2018-07-02T11:08:39.76Z"}]}

And then the next call

https://dev.azure.com/p/_apis/projects?api-version=6.0&$top=5&continuationToken=5

x-ms-continuation-token:10

{"count":5,"value":[{"id":"a88536a2-a889-45a3-a955-ddf1af8aeba1","name":"azure-devops-extensions","description":"This projects hosts the pipelines for all my Azure DevOps marketplace extensions.","url":"https://dev.azure.com/jessehouwing/_apis/projects/a88536a2-a889-45a3-a955-ddf1af8aeba1","state":"wellFormed","revision":414360082,"visibility":"public","lastUpdateTime":"2019-06-28T09:48:16.943Z"},{"id":"a1627c96-8627-41c7-9c29-498d523517e0","name":"Agile","url":"https://dev.azure.com/jessehouwing/_apis/projects/a1627c96-8627-41c7-9c29-498d523517e0","state":"wellFormed","revision":414360119,"visibility":"private","lastUpdateTime":"2019-12-02T12:11:23.863Z"},{"id":"a57ce751-1dcc-4089-b850-359624e92977","name":"Torpydo","description":"Battleship case in Python","url":"https://dev.azure.com/jessehouwing/_apis/projects/a57ce751-1dcc-4089-b850-359624e92977","state":"wellFormed","revision":414360038,"visibility":"private","lastUpdateTime":"2017-11-19T19:06:20.23Z"},{"id":"73711003-5adb-4e06-a7bb-f1d12b29db42","name":"Actual Scrum","url":"https://dev.azure.com/jessehouwing/_apis/projects/73711003-5adb-4e06-a7bb-f1d12b29db42","state":"wellFormed","revision":414360069,"visibility":"private","lastUpdateTime":"2019-04-09T19:26:38.877Z"},{"id":"9c643486-32f0-44f5-a49b-900b72f8219b","name":"Test","url":"https://dev.azure.com/jessehouwing/_apis/projects/9c643486-32f0-44f5-a49b-900b72f8219b","state":"wellFormed","revision":414360078,"visibility":"private","lastUpdateTime":"2019-05-27T09:58:21.247Z"}]}

Will fetch projects 6 to 10 and sends a continuation-token back with the value 10 for the next call.

Until the x-ms-continuation-token header is no longer returned

https://dev.azure.com/p/_apis/projects?api-version=6.0&$top=5&continuationToken=10

{"count":2,"value":[{"id":"6484ebc3-af16-4af9-aa66-6b3398db7214","name":"demo","description":"This team is  meant to be used for all kinds of great demos","url":"https://dev.azure.com/jessehouwing/_apis/projects/6484ebc3-af16-4af9-aa66-6b3398db7214","state":"wellFormed","revision":414360040,"visibility":"private","lastUpdateTime":"2018-02-08T04:57:49.15Z"},{"id":"c3eb1420-aa31-4610-9bb1-30b9e3496967","name":"OhhShitGit","url":"https://dev.azure.com/jessehouwing/_apis/projects/c3eb1420-aa31-4610-9bb1-30b9e3496967","state":"wellFormed","revision":414360050,"visibility":"private","lastUpdateTime":"2018-03-13T10:41:22.567Z"}]}

Blogged: Accessing Azure DevOps APIs with large volumes of data