Fetching and mapping data with fp-ts and io-ts

Question

I'm struggling with "massaging" my fetched data into the shapes I want, using fp-ts for functional transformation and io-ts for data validation.

What I'm looking for

I want getSchools() to return either an Error describing what went wrong, or an array of validated School s. The code I have somewhat works, but the problem is that if one of the schools in the fetched array of schools fails validation, everything fails. I would like to just filter out the ones that failed, and return the rest.

The code I have so far

/**
 * API route for all Schools
 */
export default async (_: NextApiRequest, res: NextApiResponse<unknown>) => {
  return new Promise(
    pipe(
      getSchools(),
      fold(
        (e) => of(res.status(400).end(e.message)),
        (v) => of(res.status(200).json(v))
      )
    )
  );
};

/**
 * Handler for fetching Schools
 */
export function getSchools(): TaskEither<Error, Array<School>> {
  return pipe(
    fetch(schoolQuery(schoolQueryBody)),
    chain(mapToschools),
    chain(decode(t.array(School)))
  );
}

function mapToschools(
  inputs: Array<any>
): TaskEither<Error, Array<School>> {
  try {
    return right(inputs.map(mapToschool));
  } catch (e) {
    return left(new Error("Could not map input to school"));
  }
}

export function mapToschool(input: any): School // Can throw Error

export const schoolQueryBody = `...`;

function fetch(query: string): TaskEither<Error, unknown>

export function decodeError(e: t.Errors): Error {
  const missingKeys = e.map((e) => e.context.map(({ key }) => key).join("."));
  return new Error(`Missing keys: ${missingKeys}`);
}

export const decode = <I, A>(Type: t.Decoder<I, A>) => (
  res: I
): TaskEither<Error, A> => {
  return pipe(fromEither(Type.decode(res)), mapLeft(decodeError));
};

Answer 1

I think that you have some options here for what you want to return, and I think that the default behavior of fp-ts / io-ts isn't fully lining up with what you want.

What's going on

When you parse a t.array , you will get a failure whenever one of the values fails to be decoded. It sounds to me like you want to individually attempt to decode each of the values instead of using t.array .

Suggestions

I think I would instead say something like:

import { pipe } from 'fp-ts/lib/function';
import * as ArrayFP from 'fp-ts/lib/Array';

const undecodedSchools: unknown[] = [/* ... school response */];
const schools: School[] = pipe(
  undecodedSchools,
  ArrayFP.map(schoolCodec.decode), // Array<Either<t.Errors, School>>
  ArrayFP.rights, // Takes an Array<Either<E, A>> -> Array<A>
);

That completely gets rid of the Either which may not be what you want. If you want to see the errors, it might be worth it to instead say:

const schools: {
  left: Error[],
  right: School[],
} = pipe(
  undecodedSchools,
  ArrayFP.map(schoolCodec.decode), // Array<Either<t.Errors, School>>
  ArrayFP.separate, // Takes an Array<Either<A, B>> -> Separated<A[], B[]>
);

Which pulls apart the two types of things from the array of eithers.