Push files from Azure function triggered by blobstorage trigger to Github using octokit.net

Question

I use the following code on an Azure function to push files on a github repository when a new file is uploaded to a blobstorage, that trigger the function. But it doesn't work if multiple file are uploaded to blobstorage in a short time interval: only one random file is pushed to github and then the function throw an exception; in the log: Description: The process was terminated due to an unhandled exception. Exception Info: Octokit.ApiValidationException: Reference cannot be updated {"message":"Reference cannot be updated","documentation_url":"https://docs.github.com/rest/reference/git..."} This is the code:

  public static async void PushToGithub(string fileName, Stream myBlob)
    {
        // github variables
        var owner = GITHUB_USER;
        var repo = GITHUB_REPO;
        var token = GITHUB_TOKEN;

        //Create our Client
        var github = new GitHubClient(new ProductHeaderValue("GithubCommit"));
        var tokenAuth = new Credentials(token);

        github.Credentials = tokenAuth;

        var headMasterRef = "heads/master";

        // Get reference of master branch
        var masterReference = await github.Git.Reference.Get(owner, repo, headMasterRef);
        // Get the laster commit of this branch
        var latestCommit = await github.Git.Commit.Get(owner, repo, masterReference.Object.Sha);

        // For image, get image content and convert it to base64
        byte[] bytes;
        using (var memoryStream = new MemoryStream())
        {
            myBlob.Position = 0;
            myBlob.CopyTo(memoryStream);
            bytes = memoryStream.ToArray();
        }

        var pdfBase64 = Convert.ToBase64String(bytes);
        // Create blob
        var pdfBlob = new NewBlob { Encoding = EncodingType.Base64, Content = (pdfBase64) };
        var pdfBlobRef = await github.Git.Blob.Create(owner, repo, pdfBlob);

        // Create new Tree
        var nt = new NewTree { BaseTree = latestCommit.Tree.Sha };
        // Add items based on blobs
        nt.Tree.Add(new NewTreeItem { Path = fileName, Mode = "100644", Type = TreeType.Blob, Sha = pdfBlobRef.Sha });

        var newTree = await github.Git.Tree.Create(owner, repo, nt);

        // Create Commit
        var newCommit = new NewCommit("File update " + DateTime.UtcNow, newTree.Sha, masterReference.Object.Sha);
        var commit = await github.Git.Commit.Create(owner, repo, newCommit);

        // Update HEAD with the commit
        await github.Git.Reference.Update(owner, repo, headMasterRef, new ReferenceUpdate(commit.Sha, true));
    }

How can I solve so it pushes correctly to github all the files that are uploaded on the blobstorage? Thanks in advance, Marco

Answer 1

Have a look of this official doc:

https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-blob-trigger?tabs=csharp

In addition, storage logs are created on a "best effort" basis. There's no guarantee that all events are captured. Under some conditions, logs may be missed.

If you require faster or more reliable blob processing, consider creating a queue message when you create the blob. Then use a queue trigger instead of a blob trigger to process the blob. Another option is to use Event Grid; see the tutorial Automate resizing uploaded images using Event Grid.

If you focus on processing blob and don't care about the loose event, then you can use queue trigger to make sure the blob be precessed, if you care about the loose event, please use event grid.