Two entries made in Azure blob table when adding a single row, using Durable Functions with C#

Question

I'm working on a proof of concept, but I'm new to both Azure and C#. The following code works, sort of - it successfully stores incoming JSON data into a blob table. The problem is that it creates 2 different rows each time, each with its own unique RowKey (GUID I create), but, as you can see, I only add to the table once with await GrooveData.AddAsync (data); .

What am I missing? The two rows have slightly different Timestamps.

using System;
using System.Collections.Generic;
using System.IO;
using System.Net.Http;
using System.Threading.Tasks;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.DurableTask;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Extensions.Logging;

namespace Company.Function
{
    public static class GrooveOrchestrationTest
    {
        [FunctionName("GrooveOrchestrationTest")]
        public static async Task<List<string>> RunOrchestrator(
            [OrchestrationTrigger] IDurableOrchestrationContext context,
            [Table ("GrooveData")] IAsyncCollector<GrooveItem> GrooveData)
        {
            var outputs = new List<string>();
            GrooveItem data = context.GetInput<GrooveItem>();      

                // Indicate we got the data:
                outputs.Add(await context.CallActivityAsync<string>("NowProcessingMessage", data));
                
                // RowKey and PartitionKey are required by Azure, but we don't get them from Groove and must provide them:
                data.RowKey = Guid.NewGuid ().ToString ();
                data.PartitionKey = data.grooveevent;

                // and this stores the data:
                await GrooveData.AddAsync (data);

                //If we get this far, it probably worked :D
                outputs.Add(await context.CallActivityAsync<string>("GrooveDataTest", data));

            return outputs;
        }

        [FunctionName("NowProcessingMessage")]
        public static string ProcessingMessage([ActivityTrigger] GrooveItem cust, ILogger log)
        {
            log.LogInformation($"Now processing transaction for {cust.firstname} for {cust.amount}");
            return $"Trying to store transaction for {cust.firstname} for {cust.amount}";
        }

        //This works.  To use it, call it above in CallActivityAsync:
        [FunctionName("GrooveDataTest")]
        public static string ProcessGrooveData([ActivityTrigger] GrooveItem cust, ILogger log)
        {
            log.LogInformation($"Received transaction for {cust.firstname} {cust.lastname}: {cust.email}");
            return $"{cust.firstname} had type {cust.grooveevent} for {cust.product_name} for ${cust.amount}";
        
        }

        [FunctionName("GrooveDataHandlerTest_HttpStart")]
        public static async Task<HttpResponseMessage> HttpStart(
            [HttpTrigger(AuthorizationLevel.Anonymous, "get", "post")] HttpRequestMessage req,
            [DurableClient] IDurableOrchestrationClient starter,
            ILogger log)
        {
                        
            var data = await req.Content.ReadAsAsync<GrooveItem>();
            
            // Function input comes from the request content.
            string instanceId = await starter.StartNewAsync("GrooveOrchestrationTest", data);

            log.LogInformation($"Started orchestration with ID = '{instanceId}'.");

            return starter.CreateCheckStatusResponse(req, instanceId);
        }
    }
}

I am testing using Postman, sending JSON data: {"event":"sales","buyer_first_name":"Testy","buyer_last_name":"2","buyer_email":"testy@bugmenot.com","product_name":"Great Product","amount":"99.99"}

Here is my activity log when running in Visual Studio Code, which, as near as I can tell, shows that it's only called once:

[8/10/2020 2:19:08 PM] Executing 'GrooveOrchestrationTest' (Reason='', Id=064dd243-fccc-4d29-9c1e-b803d0aaf87f)
[8/10/2020 2:19:08 PM] 6ea660aef5924d0ebfe70d6dbfd52742: Function 'GrooveOrchestrationTest (Orchestrator)' started. IsReplay: False. Input: (2920 bytes). State: Started. HubName: TestHubName. AppName: . SlotName: . ExtensionVersion: 2.1.1. SequenceNumber: 3.
[8/10/2020 2:19:08 PM] 6ea660aef5924d0ebfe70d6dbfd52742: Function 'NowProcessingMessage (Activity)' scheduled. Reason: GrooveOrchestrationTest. IsReplay: False. State: Scheduled. HubName: TestHubName. AppName: . SlotName: . ExtensionVersion: 2.1.1. SequenceNumber: 4.
[8/10/2020 2:19:08 PM] Executed 'GrooveOrchestrationTest' (Succeeded, Id=064dd243-fccc-4d29-9c1e-b803d0aaf87f)
[8/10/2020 2:19:08 PM] 6ea660aef5924d0ebfe70d6dbfd52742: Function 'GrooveOrchestrationTest (Orchestrator)' awaited. IsReplay: False. State: Awaited. HubName: TestHubName. AppName: . SlotName: . ExtensionVersion: 2.1.1. SequenceNumber: 5.
[8/10/2020 2:19:08 PM] 6ea660aef5924d0ebfe70d6dbfd52742: Function 'NowProcessingMessage (Activity)' started. IsReplay: False. Input: (3088 bytes). State: Started. HubName: TestHubName. AppName: . SlotName: . ExtensionVersion: 2.1.1. SequenceNumber: 6.
[8/10/2020 2:19:08 PM] Executing 'NowProcessingMessage' (Reason='', Id=057e5131-f212-4234-a3ae-e2f504814146)
[8/10/2020 2:19:08 PM] Now processing transaction for Testy for 99.99
[8/10/2020 2:19:08 PM] Executed 'NowProcessingMessage' (Succeeded, Id=057e5131-f212-4234-a3ae-e2f504814146)
[8/10/2020 2:19:08 PM] 6ea660aef5924d0ebfe70d6dbfd52742: Function 'NowProcessingMessage (Activity)' completed. ContinuedAsNew: False. IsReplay: False. Output: (196 bytes). State: Completed. HubName: TestHubName. AppName: . SlotName: . ExtensionVersion: 2.1.1. SequenceNumber: 7.
[8/10/2020 2:19:08 PM] Executing 'GrooveOrchestrationTest' (Reason='', Id=9c7ae8df-8f48-451c-a021-7f54167f5be9)
[8/10/2020 2:19:08 PM] 6ea660aef5924d0ebfe70d6dbfd52742: Function 'GrooveDataTest (Activity)' scheduled. Reason: GrooveOrchestrationTest. IsReplay: False. State: Scheduled. HubName: TestHubName. AppName: . SlotName: . ExtensionVersion: 2.1.1. SequenceNumber: 8.
[8/10/2020 2:19:08 PM] Executed 'GrooveOrchestrationTest' (Succeeded, Id=9c7ae8df-8f48-451c-a021-7f54167f5be9)
[8/10/2020 2:19:09 PM] 6ea660aef5924d0ebfe70d6dbfd52742: Function 'GrooveOrchestrationTest (Orchestrator)' awaited. IsReplay: False. State: Awaited. HubName: TestHubName. AppName: . SlotName: . ExtensionVersion: 2.1.1. SequenceNumber: 9.
[8/10/2020 2:19:09 PM] 6ea660aef5924d0ebfe70d6dbfd52742: Function 'GrooveDataTest (Activity)' started. IsReplay: False. Input: (3236 bytes). State: Started. HubName: TestHubName. AppName: . SlotName: . ExtensionVersion: 2.1.1. SequenceNumber: 10.
[8/10/2020 2:19:09 PM] Executing 'GrooveDataTest' (Reason='', Id=8f70954c-64ac-48f5-83ce-ca3b89d11bcf)
[8/10/2020 2:19:09 PM] Received transaction for Testy 1: testy@bugmenot.com
[8/10/2020 2:19:09 PM] Executed 'GrooveDataTest' (Succeeded, Id=8f70954c-64ac-48f5-83ce-ca3b89d11bcf)
[8/10/2020 2:19:09 PM] 6ea660aef5924d0ebfe70d6dbfd52742: Function 'GrooveDataTest (Activity)' completed. ContinuedAsNew: False. IsReplay: False. Output: (204 bytes). State: Completed. HubName: TestHubName. AppName: . SlotName: . ExtensionVersion: 2.1.1. SequenceNumber: 11.
[8/10/2020 2:19:09 PM] Executing 'GrooveOrchestrationTest' (Reason='', Id=8656fff8-6b49-4fdd-93ef-38959e7af01f)
[8/10/2020 2:19:09 PM] 6ea660aef5924d0ebfe70d6dbfd52742: Function 'GrooveOrchestrationTest (Orchestrator)' completed. ContinuedAsNew: False. IsReplay: False. Output: (412 bytes). State: Completed. HubName: TestHubName. AppName: . SlotName: . ExtensionVersion: 2.1.1. SequenceNumber: 12.
[8/10/2020 2:19:09 PM] Executed 'GrooveOrchestrationTest' (Succeeded, Id=8656fff8-6b49-4fdd-93ef-38959e7af01f)

UPDATE I just read that the Orchestrator should not use any randomization such as calling the GUID function, which I am doing to create a ROWKEY value. Perhaps this is related? I tried moving it into the HTTPSTART, but that generated an actual error that it already exists. Perhaps there is another way I can get that into the item without running afoul of this limitation?

SOLUTION The answer provided by user1672994 is right, but Method #1 as originally posted had to be adjusted slightly so it would work. This is the fixed new code:

        [FunctionName("AddGrooveDataAsync2")]
        public static async Task<String> Run([ActivityTrigger] GrooveItem trans, [Table ("GrooveData")] IAsyncCollector<GrooveItem> GrooveData, ILogger log)
        {
        // RowKey and PartitionKey are required by Azure, but we don't get them from Groove and must provide them:
            trans.RowKey = Guid.NewGuid ().ToString ();
            trans.PartitionKey = trans.grooveevent;

            // and this stores the data:
            await GrooveData.AddAsync (trans);
            return $"Added {trans.firstname} {trans.lastname}: {trans.email}";
        }

Answer 1

The problem here is that you are adding the data into table directly by Orchestrator function which will be replayed multiple times until finishes. You can fix this issue by following approach. I would recommend you to for approach 1.

Approach 1: Convert the logic to activity function which won't be called again if message is replaying.

    [FunctionName("AddGrooveDataAsync")]
    public static async Task<string> AddGrooveDataAsync([ActivityTrigger] GrooveItem cust, [Table ("GrooveData")] IAsyncCollector<GrooveItem> GrooveData, ILogger log)
    {
        // RowKey and PartitionKey are required by Azure, but we don't get them from Groove and must provide them:
            data.RowKey = Guid.NewGuid ().ToString ();
            data.PartitionKey = data.grooveevent;

            // and this stores the data:
            await GrooveData.AddAsync (data);
    }

Approach 2: Add the context.IsReplaying check.

   [FunctionName("GrooveOrchestrationTest")]
    public static async Task<List<string>> RunOrchestrator(
        [OrchestrationTrigger] IDurableOrchestrationContext context,
        [Table ("GrooveData")] IAsyncCollector<GrooveItem> GrooveData)
    {
        var outputs = new List<string>();
        GrooveItem data = context.GetInput<GrooveItem>();      

            // Indicate we got the data:
            outputs.Add(await context.CallActivityAsync<string>("NowProcessingMessage", data));
             
            if (!context.IsReplaying)
            {
                  // RowKey and PartitionKey are required by Azure, but we don't get them from Groove and must provide them:
                  data.RowKey = Guid.NewGuid ().ToString ();
                  data.PartitionKey = data.grooveevent;
            }

            // and this stores the data:
            await GrooveData.AddAsync (data);

            //If we get this far, it probably worked :D
            outputs.Add(await context.CallActivityAsync<string>("GrooveDataTest", data));

        return outputs;
    }