How to load csvs with different structures using CSVProvider?

Question

Any Ideas? My current code is below.

let rawdata csvfile 
        FinCsv.Load(data).Rows
        |> Seq.filter (fun row -> row.Id <> "---")
        |> Seq.filter (fun row -> row.Country <> "Honduras")
        |> Seq.filter (fun row -> row.Tax <> 0)
        |> Seq.groupBy (fun row -> row.Country)
        |> Seq.averageBy (fun row -> row.Tax)
        |> List.ofSeq

Answer 1

The problem with statically-resolved type parameters is that the syntax is cumbersome. Here's an example:

let inline snippet< ^T when ^T : (member Id : string)
                        and ^T : (member Country : string)
                        and ^T : (member Tax : float) >
                        xs =
    xs
    |> Seq.filter (fun row -> ( ^T : (member Id : string) row) <> "---")
    |> Seq.filter (fun row -> ( ^T : (member Country : string) row) <> "South America")
    |> Seq.filter (fun row -> ( ^T : (member Tax : float) row) <> 0.0)
    |> Seq.groupBy (fun row -> ( ^T : (member Country : string) row))
    |> Seq.map (fun (country, rows) -> country, (rows |> Seq.averageBy (fun row -> ( ^T : (member Tax : float) row))))
    |> List.ofSeq

(I've modified the averageBy call so it makes more sense than the one in the question.)

You can actually let the compiler infer the constraints rather than declaring them explicitly:

let inline snippet xs =
    xs
    |> Seq.filter (fun row -> ( ^T : (member Id : string) row) <> "---")
    |> Seq.filter (fun row -> ( ^T : (member Country : string) row) <> "South America")
    |> Seq.filter (fun row -> ( ^T : (member Tax : float) row) <> 0.0)
    |> Seq.groupBy (fun row -> ( ^T : (member Country : string) row))
    |> Seq.map (fun (country, rows) -> country, (rows |> Seq.averageBy (fun row -> ( ^T : (member Tax : float) row))))
    |> List.ofSeq

As an alternative, you could require the caller to pass value-getter functions:

let snippet2 getId getCountry getTax xs =
    xs
    |> Seq.filter (fun row -> getId row <> "---")
    |> Seq.filter (fun row -> getCountry row <> "South America")
    |> Seq.filter (fun row -> getTax row <> 0.0)
    |> Seq.groupBy (fun row -> getCountry row)
    |> Seq.map (fun (country, rows) -> country, (rows |> Seq.averageBy (fun row -> getTax row))))
    |> List.ofSeq

This lets you simplify the syntax somewhat:

let snippet2 getId getCountry getTax xs =
    xs
    |> Seq.filter (getId >> (<>) "---")
    |> Seq.filter (getCountry >> (<>) "South America")
    |> Seq.filter (getTax >> (<>) 0.0)
    |> Seq.groupBy getCountry
    |> Seq.map (fun (country, rows) -> country, (rows |> Seq.averageBy getTax))
    |> List.ofSeq

You can declare type-specific instances of the function with partial application:

let snippetForCsvFile1 (xs : CsvFile1 seq) = xs |> snippet2 (fun r -> r.Id) (fun r -> r.Country) (fun r -> r.Tax)

Then your switch could look like this:

let rawdata csvfile =
    match csvfile with
    | 1 -> snippetForCsvFile1 CsvFile1
    | 2 -> snippetForCsvFile2 CsvFile2
    | 3 -> snippetForCsvFile3 CsvFile3
    | 4 -> snippetForCsvFile4 CsvFile4
    | _ -> failwith "Not a File"

This then ought to work with your last expression:

let raw =
    [ 1 .. 4]
    |> List.collect rawdata

The switch in your question has a problem that you didn't mention in your question:

let rawdata csvfile =
    let data = 
        match csvfile with
        | 1 -> CsvFile1
        | 2 -> CsvFile2
        | 3 -> CsvFile3
        | 4 -> CsvFile4
        | _ -> failwith "Not a File"

The problem here is that the binding of data doesn't have a single type. You could achieve that with an object-oriented approach using an interface (or a base class), but that would not be particularly idiomatic for F#. A more idiomatic approach would be to declare a discriminated union, but given the example you've posted it doesn't seem necessary.