GitlabCI. Get merge request ID during master branch build

Question

Each time a new pull request is merged, the master branch is being built. During this stage those steps are proceeded:

1. A new Git tag is created (version is incremented automatically).
2. The library is published to the registry.
3. A new email is send to all the users that are tracking new versions.

I want to add the direct link to the merge request that was merged and triggered this release in the mail. Though I can get the information (the MR ID) during the merge request build itself, I don't understand how can I retrieve it once MR is merged.

Is there any approach to overcome this problem?

Answer 1

My other answer works, but I'd like to add a more reliable one, in the sense that it doesn't rely on the merge commits message template (see doc ).

This answer is an improvement on the idea "Get the latest merged pipeline" (accepted answer), which is not failproof if several MRs are merged quickly.

Solution

Find the MR whose merge_commit_sha matches the CI_COMMIT_SHA .

Implementation details

1. Get the commit sha of the current build: just take the predefined variable CI_COMMIT_SHA (I think git rev-parse HEAD should also work).
2. Get the list of all merged MRs:

API_MR_URL="${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/merge_requests"
curl --header "PRIVATE-TOKEN: ${YOUR_API_TOKEN}" "${API_MR_URL}?scope=all&state=merged" > mrs.json

3. Find the MR whose merge_commit_sha matches CI_COMMIT_SHA :

yq '.[] | select(.merge_commit_sha == strenv(CI_COMMIT_SHA)) | .iid' mrs.json

That's it! Note that here I used the yq tool instead of the more standard jq, but that's just because I'm more familiar with it, I'm sure it's easy to do the same with jq.

Pagination caveat

Annoyingly, the Gitlab API only returns a limited number of results per "page" (20 by default, max 100). If you are unlucky, the MR you are looking for might not be on the first page. For a completely failproof script, we thus need to iterate through all the pages. Here we go:

total_pages="$( curl --include --header "PRIVATE-TOKEN: ${YOUR_API_TOKEN}" "${API_MR_URL}?scope=all&state=merged&per_page=100" | egrep '^X-Total-Pages' | egrep -o '[0-9]+' )"
page=1
while [[ "${page}" -le "${total_pages}" ]] ; do
  curl --header "PRIVATE-TOKEN: ${CTE_DBT_API_TOKEN}" "${API_MR_URL}?scope=all&state=merged&per_page=100&page=${page}" > mrs.json
  mr_iid="$(yq '.[] | select(.merge_commit_sha == strenv(CI_COMMIT_SHA)) | .iid' mrs.json)"
  if [[ "${mr_iid}" -gt 0 ]] ; then
    echo "Found matching MR: ${mr_iid}"
    break
  fi
  page=$((page + 1))
done

Answer 2

The only I approached I managed to discover is to request the merge requests via REST API . In this case, I can filter them by status and creation date. So the latest merged one is the MR I need.

Here is a hypothetical query:

/api/v4/projects/:projectId/merge_requests?state=merged&created_after=:day_before_now

The first merge request in the received array is the target one.

PS The created_after=:day_before_now attribute just reduces the amount of data received by HTTP. You can omit it and won't change the behaviour.

Answer 3

I just now solved my problem similar to this, everyone has suggested API and I was using API for my pipeline.

My pipeline has two stages: Stage1 Stage2

Stage1 is triggered when MR is created (MR status: open)
Stage2 is triggered when MR (MR status: open) is merged (MR status: merged)

Now the problem is these two stages are triggered in different pipelines. Different pipelines mean all your variable world of GitLab is limited to the pipeline itself. Nothing goes out.

Then I used gitlab api . Stage1 whould fecth the first latest open merge request id in the api list which obviously is the MR which triggered Stage1. As Stage1 pipeline starts execution as soon as the status of MR goes to "Open", so that is always top in the list of open MR.

Same I did for Stage2 by fetching first latest "merged" merge request id. Everything works fine but the problem with using api is timing:(

It has a window where problem can occur when gitlab api gets updated with new latest merge id and your Stage1 has not yet fetched the merge id from gitlab api and whenever it does it will get new value for merge id. This problem was for both stages using gitlab api for merge id.

As these both stages were in different pipeline they were spinng seperate docker container for them and this time window till stage will execute and fecth merge id from gitlab api was the problem.

So gitlab api data was not reliable in this case. Then I turned towards variable world of gitlab.

I got merge id in Stage1 from variable CI_MERGE_REQUEST_IID

Stage1 was good.

Now, Stage2 runs in a new pipeline, and that pipeline shows CI_MERGE_REQUEST_IID empty. After checking the value of as many CI variables I could find I ended on using export command in GitLab.

I added export to.gitlab-ci.yml:

  script:
    - export

export handed out everything!!

The export command gives the list of all the variables and its value, available to be consumed by stages in that current pipeline.

I compared the list of variables from Satge1 and Satge2. Then I found

CI_COMMIT_MESSAGE 

though its value differ in between two stages, I found merge id in it for Stage2.

So for my problem this is the only variable available in the new pipeline which has some reference to merge id and is available in the variable world of this new pipeline for Stage2.

Though pipeline lines are different they are connected to the SAME merge request and are triggered on its different events like

merge_request_status = "open"

merge_request_status = "merged"

By default CI_COMMIT_MESSAGE has value:

Merge branch '%{source_branch}' into '%{target_branch}'

%{title}

%{issues}

See merge request %{reference}

This message is coming from the default template ( more info ).

%{reference} variable in default template has value which is path of the merge id. So I sliced merged id from CI_COMMIT_MESSAGE

merge_id=$(echo $CI_COMMIT_MESSAGE|awk -F! '{print $2}')

So now my both the pipelines were provding merge id in variable for both the stages to consume. So now pipelines are no more dependend on first piece of informaton from gitlab api. Instead, now they have merge id already and from that they can browse everything under it like pipelines, commits, jobs etc, using gitlab api easily and the data is reliable.

merge id is kind of parent variable which is connected to everything pipelines, commits, jobs etc while using gitlab api for data and that variable is so hard to find in merged event of MR is sad. This is important variable and should be available in all types of events which triggers pipeline.

Hope this helps.

Answer 4

Getting the latest merged pipeline with the API, like the accepted answer suggests, is a good idea but it is not failproof: if several MRs are merged quickly, the one you are looking for might not be the latest.

Fortunately I think there is an easier solution: look at the latest commit message. Just make sure that, in the settings of your project, you have the default template for merge commits: see Gitlab doc .

Your latest commit will then look like this:

commit 20780d95bd026e04d409d5c47c045915c6883333
Merge: e4368ba 4hfa518
Author: My Name <my-email>
Date:   Some-Timestamp

    Merge branch 'my-branch' into 'main'
    
    My commit message
    
    See merge request my-group/my-project!52

Here the MR iid is 52; you can then easily grab it automatically.

Tentative implementation: I'm not a regex/glob wizard, but I think this works:

matched_line="$(git log -1 --pretty=%B | grep 'See merge request')"
mr_iid=${matched_line##*\!}