Trying to create a data pipeline with Rx .Net

Question

I've been trying to pipeline some data using Rx .Net but I'm really struggling.

I've got a sequence of unsaved Foos which have some data. The Foos create a collection of Bars. After the bars have been saved, if the Bar has a BarId, the data from the Foo and the BarId from the Bar are used to create a Baz. Once all the Bars and Bazs for the Foo have been created, the Foo's data is cleared and the Foo is saved. Here are the classes:

public class Base
{
    public Guid Id { get; set; } = Guid.Empty;

    public override string ToString()
    {
        return $"{GetType().Name}::Id:{Id}";
    }

    public Task<Unit> Save()
    {
        Id = Guid.NewGuid();

        Console.WriteLine($"{ToString()} (save)");

        return Task.FromResult(Unit.Default);
    }
}

public sealed class Foo : Base
{
    public long FooId { get; }

    public string Data { get; private set; }

    public override string ToString()
    {
        return $"{base.ToString()},FooId:{FooId},Data:{Data ?? "NULL"}";
    }

    public Task<Unit> ClearData()
    {
        Data = null;

        return Task.FromResult(Unit.Default);
    }

    public Foo(long index)
    {
        FooId = index;
        Data = $"Foo({index})";

        Console.WriteLine($"{ToString()} (ctor)");
    }
}

public class Map : Base
{
    public int Index { get; }

    public string Mapping { get; }

    public Guid BarId { get; protected set; } = Guid.Empty;

    public sealed override string ToString()
    {
        return $"{base.ToString()},Index:{Index},BarId:{BarId},Mapping:{Mapping}";
    }

    protected Map(int index, string data, Func<string, string> mapper)
    {
        Mapping = mapper(data);
        Index = index;
    }
}

public sealed class Bar : Map
{
    public Bar(int index, string data, Func<string, string> mapper) : base(index, data, mapper)
    {
        if (new Random().Next(3) > 0)
        {
            BarId = Guid.NewGuid();

            Console.WriteLine($"{ToString()} (ctor)");
        }
    }
}

public sealed class Baz : Map
{
    public Baz(int index, Guid barId, string data, Func<string, string> mapper) : base(index, data, mapper)
    {
        BarId = barId;

        Console.WriteLine($"{ToString()} (ctor)");
    }
}

I tried a number of things, but this one is the closest I got to what I'm looking for:

var foos = Observable.Interval(TimeSpan.FromSeconds(1)).Take(2).Select(i => new Foo(i));

var units = from foo in foos
            let idxs = Observable.Range(1, 3)
            from idx in idxs
            let bar = new Bar(idx, foo.Data, s => $"{s}Bar")
            from barSaved in bar.Save()
            where bar.BarId != Guid.Empty
            let baz = new Baz(idx, bar.BarId, foo.Data, s => $"{s}Baz")
            from bazSaved in baz.Save()
            from fooChanged in foo.ClearData()
            from fooSaved in foo.Save()
            select Unit.Default;

units.Subscribe();

It creates the right amount of Bars and Bazs but it saves the Foo every time a saved Baz is created or never if there are no Bazs.

I'm hoping someone can help me out and maybe I'll also learn something new about Reactive Programming that will help me get a better understanding of the paradigm.

Edit:

I tried moving the Random number provider to a static instance but it didn't work. Here's some output that shows what occurs:

Foo::Id:00000000-0000-0000-0000-000000000000,FooId:0,Data:Foo(0) (ctor)
Bar::Id:00000000-0000-0000-0000-000000000000,Index:1,BarId:00000000-0000-0000-0000-000000000000,Mapping:Foo(0)Bar (ctor)
Bar::Id:28789b8b-03d4-4160-97f2-b2cbafd80c73,Index:1,BarId:00000000-0000-0000-0000-000000000000,Mapping:Foo(0)Bar (save)
Bar::Id:00000000-0000-0000-0000-000000000000,Index:2,BarId:81e79b81-6692-406f-a025-448cd203cb73,Mapping:Foo(0)Bar (ctor)
Bar::Id:d43d2e43-b812-4657-9e4f-a5b875a595fb,Index:2,BarId:81e79b81-6692-406f-a025-448cd203cb73,Mapping:Foo(0)Bar (save)
Baz::Id:00000000-0000-0000-0000-000000000000,Index:2,BarId:81e79b81-6692-406f-a025-448cd203cb73,Mapping:Foo(0)Baz (ctor)
Baz::Id:fdf464d1-9240-49e4-89cd-dbab758159fc,Index:2,BarId:81e79b81-6692-406f-a025-448cd203cb73,Mapping:Foo(0)Baz (save)
Foo::Id:04d9d819-26bc-41dc-8f1b-a0e509acd2e5,FooId:0,Data:NULL (save)
Bar::Id:00000000-0000-0000-0000-000000000000,Index:3,BarId:6894c1f0-d776-496f-9da4-4f272a338f90,Mapping:Bar (ctor)
Bar::Id:6b82fc20-80ff-41bd-848d-a115f58392c4,Index:3,BarId:6894c1f0-d776-496f-9da4-4f272a338f90,Mapping:Bar (save)
Baz::Id:00000000-0000-0000-0000-000000000000,Index:3,BarId:6894c1f0-d776-496f-9da4-4f272a338f90,Mapping:Baz (ctor)
Baz::Id:fa361c43-ee07-44cc-8628-4835fce6da9d,Index:3,BarId:6894c1f0-d776-496f-9da4-4f272a338f90,Mapping:Baz (save)
Foo::Id:6f423cf5-8260-4d98-a0bc-4762893b5fe4,FooId:0,Data:NULL (save)
Foo::Id:00000000-0000-0000-0000-000000000000,FooId:1,Data:Foo(1) (ctor)
Bar::Id:00000000-0000-0000-0000-000000000000,Index:1,BarId:5a79b126-27d8-43a1-9bb6-4d28394d5710,Mapping:Foo(1)Bar (ctor)
Bar::Id:94f5842f-eb1c-4248-a70a-dac5eb843cc1,Index:1,BarId:5a79b126-27d8-43a1-9bb6-4d28394d5710,Mapping:Foo(1)Bar (save)
Baz::Id:00000000-0000-0000-0000-000000000000,Index:1,BarId:5a79b126-27d8-43a1-9bb6-4d28394d5710,Mapping:Foo(1)Baz (ctor)
Baz::Id:d46b1d7e-eda2-4c63-8810-830e3416e975,Index:1,BarId:5a79b126-27d8-43a1-9bb6-4d28394d5710,Mapping:Foo(1)Baz (save)
Foo::Id:e198e636-bfd7-49c5-a260-346524ec4019,FooId:1,Data:NULL (save)
Bar::Id:00000000-0000-0000-0000-000000000000,Index:2,BarId:00000000-0000-0000-0000-000000000000,Mapping:Bar (ctor)
Bar::Id:206a8796-32da-493f-9582-7c551781e2d5,Index:2,BarId:00000000-0000-0000-0000-000000000000,Mapping:Bar (save)
Bar::Id:00000000-0000-0000-0000-000000000000,Index:3,BarId:154481ad-33c3-49d7-af2b-3f7738f1f692,Mapping:Bar (ctor)
Bar::Id:ffda6907-93b0-411f-aa40-3fee790b52cb,Index:3,BarId:154481ad-33c3-49d7-af2b-3f7738f1f692,Mapping:Bar (save)
Baz::Id:00000000-0000-0000-0000-000000000000,Index:3,BarId:154481ad-33c3-49d7-af2b-3f7738f1f692,Mapping:Baz (ctor)
Baz::Id:2071927f-27d6-4df6-a74f-1c0bdfbba8d5,Index:3,BarId:154481ad-33c3-49d7-af2b-3f7738f1f692,Mapping:Baz (save)
Foo::Id:07460e6e-d3db-4ed8-a441-e16f06cd908a,FooId:1,Data:NULL (save)

As you can see, when a Baz is saved, the Foo clears its data and saves, making all the subsequent Bars and Bazs lose their data - see their Mapping has no Foo(index).

Edit:

What I'm trying to achieve is that once all the Bars and Bazs for a Foo have been created and saved, only then is the Foo's data cleared and a save done.

Edited source code to include tracing.

Answer 1

I think your problem is the if (new Random().Next(3) > 0) . The Random object seeds from the system clock so if this code is called in rapid succession you get the same value. Try it this way:

public sealed class Bar : Map
{
    public static Random _rand = new Random();
    public Bar(int index, string data, Func<string, string> mapper) : base(index, data, mapper)
    {
        if (_rand.Next(3) > 0)
        {
            BarId = Guid.NewGuid();
        }
    }
}

It appears to give the right kind of result then.

---

I would keep the Random instance outside the method and then try this:

var units =
    from foo in foos
    from inner in
    (
        from idx in Observable.Range(1, 3)
        let bar = new Bar(idx, foo.Data, s => $"{s}Bar")
        from barSaved in bar.Save()
        where bar.BarId != Guid.Empty
        let baz = new Baz(idx, bar.BarId, foo.Data, s => $"{s}Baz")
        from bazSaved in baz.Save()
        select Unit.Default
    ).ToArray()
    from fooChanged in foo.ClearData()
    from fooSaved in foo.Save()
    select Unit.Default;