Scala: How to define generic flatMap and map methods

Question

Question: how do you define a generic flatMap method on a trait or abstract class that can then be implemented on an object extending that trait/class?

I've been using Scala for a while now and have been doing some reading on functional programming, and as an exercise I've been trying to implement a number of basic types found in Haskell as Scala traits. I'm currently trying to make a Monad trait that defines a flatMap method, which can then be implemented by another class -- eg, a custom version of List -- and I've run into a swarm of type problems.

At first I tried the following:

trait Monad[ T[ _ ], +A ] {

  this : T[ A ] =>  // Won't compile unless the object extending this is of type 
                    // corresponding to the above type parameters
    def flatMap[ B ]( fn : A => T[ B ] ) : T[ B ]
    def map[ B ]( fn : A => B ) : T[ B ]
}

abstract class FpList[ +T ] extends Monad[ FpList, T ] {
  def tail : FpList[ T ]
  def head : T

  override def flatMap[ B ]( fn : T => FpList[ B ] ) : FpList[ B ] = this match {
    case FpNil => FpNil
    case FpList( FpNil, v ) => fn( v )
    case FpList( next, v ) => flatMap( next ) ++ fn( v )
  }

  override def map[ B ]( fn : T => B ) : FpList[ B ] = flatMap( v => FpNil :: fn( v ) )
}

The above worked nicely for my FpList class, but does not work for any type that does not take the form F[X]. I discovered this when I tried to implement a Writer monad:

case class FpWriter[ T, C <: Monoid[ _ ] ]( value : T, context : C ) extends Monad[ FpWriter, T ] {
 ...
}

The problem here is that since FpWriter has two type parameters, it cannot be passed as a type parameter to Monad[...] . If I change the definition of Monad to trait Monad[ T[ _, _ ], +A ] it no longer accepts FpList as a type parameter...

To come up with a more general solution, I arrived at the following:

trait Monad[ T, +U <: T, +A ] {
  this : U =>
    def flatMap[ V <: T ]( fn : A => V ) : V
    def map[ B, C <: T ]( fn : A => B ) : C
}

abstract class FpList[ +T ] extends Monad[ FpList[ Any ], FpList[ T ], T ] {
  def tail : FpList[ T ]
  def head : T

  override def flatMap[ V <: FpList[ Any ] ]( fn : A => V ) : V = this match {
        case FpNil => FpNil.asInstanceOf[ V ]
        case FpList( FpNil, v : A ) => fn( v ).asInstanceOf[ V ]
        case FpList( next : FpList[ A ], v : A ) => 
          (flatMap( next )( fn ).asInstanceOf[ V ] ++ fn( v ).asInstanceOf[V]).asInstanceOf[ V ]
    }

  // Runtime error
  override def map[ B, C <: T ]( fn : A => B ) : C = flatMap( v => FpNil :: fn( v ) )
}

This solved the problem for flatMap. But now map is broken because the return type C is unknown to the compiler, In the case of flatMap, the signature of the function fn passed to flatMap defines the return type V. In the case of map, none of the types identified by the parameter fn define what the return type C will be. Is there some way to clarify the relation between B and C to the compiler?

Some other related questions/answers have pointed me in the direction of using an implicit parameter that defines the relation between B and C, but I do not understand how to do so. Any thoughts?

UPDATE

So I tried out SimY4's suggestion in the following way:

Monad:

trait MonadStatic [ T[ _ ] ] extends ApplicativeStatic[ T ] {

    def flatMap[ A, B ]( a : T[ A ] )( fn : A => T[ B ] ) : T[ B ]

    override def map[ A, B ]( ele : T[ A ] )( fn : A => B ) : T[ B ] = flatMap( ele )( a => unit( fn( a ) ) )

    def sub[ X, Y ]( a : T[ X ], b : T[ Y ] ) : T[ Y ] = flatMap( a )( ( _ : X ) => b )

}

trait Monad [ T[ _ ], +A ] extends MonadStatic[ T ] with Applicative[ T, A ] {
    this : T[ A ] =>
        def flatMap[ A, B ]( fn : A => T[ B ] ) : T[ B ] = flatMap( this )( fn )
}

Monoid:

trait MonoidStatic [ T[ _ ] ] {
    def empty : T[ Nothing ]
    def emptyM : Monoid[ T, Nothing ] = empty.asInstanceOf[ Monoid[ T, Nothing ] ]
    def combine[ B  ]( a : T[ B ], b : T[ B ] ) : T[ B ]
    def combine[ B ]( a : Monoid[ T, B ], b : Monoid[ T, B ] ) : Monoid[ T, B ] = combine( a.native, b.native ).asInstanceOf[ Monoid[ T, B ] ]

    def concat[ B ]( eles : FpList[ T[ B ] ] ) : T[ B ] = eles match {
        case FpNil => empty.asInstanceOf[ T[ B ] ]
        case FpList( FpNil, head : T[ B ] ) => head
        case FpList( fpList : FpList[ T[ B ] ], head : T[ B ] ) => combine( concat( fpList ), head )
    }
}

trait Monoid[ T[ _ ], +A ] extends MonoidStatic[ T ] {

    this : T[ A ] =>
        def combine[ B >: A ]( a : T[ B ] ) : T[ B ] = combine( this.asInstanceOf[ T[ B ] ], a )
        def combine[ B >: A ]( a : Monoid[ T, B ] ) : T[ B ] = combine( a.native )
        def combineM[ B >: A ]( a : Monoid[ T, B ] ) : Monoid[ T,  B ] = combine( this.monoid, a )
        def native[ B >: A ] : T[ B ] = this.asInstanceOf[ T[ B ] ]
        def monoid[ B >: A ] : Monoid[ T, B ] = this.asInstanceOf[ Monoid[ T, B ] ]

}

and now FpWriter:

case class FpWriter[ T, A[ _ ], B ]( value : T, context : Monoid[ A, B ] ) extends Monad[ ({ type U[ X ] = FpWriter[ X, A, B ] })#U, T ] {
    override def flatMap[ R, S ]( a : FpWriter[ R, A, B ] )
                                ( fn : R => FpWriter[ S, A, B ] ) : FpWriter[ S, A, B ] = a match {
        case FpWriter( v : R, c : Monoid[ A, B ] ) =>
            val FpWriter( newV, newC : Monoid[ A, B ] ) = fn( v )
            FpWriter( newV, c.combineM( newC ) )
    }

    override def unit[ C ]( ele : C ) : FpWriter[ C, A, B ] = FpWriter( ele, context.emptyM )

    override def get[ C ]( ele : FpWriter[ C, A, B ] ) : Option[ C ] = Some( ele.value )
}

This is resulting in a compile error: "Illegal cyclic reference involving FpWriter"

The problem seems to be that I can't use FpWriter in a lambda in the signature of FpWriter itself. However, it looks like other people have tried some version of this and it has worked. Any idea of what the problem is?

UPDATE

@SimY4 pointed out that in a comment I had left out Monad from the extend clause of the FpWriter signature. When I rewrote the function along the lines they suggested, it had no problem compiling. This is the best solution I've been able to come up with that don't require implicit conversions.

Answer 1

You need to explain to scala compiler how to convert your T[_, _] into T[_] . In scala one way to do this is using a type lambda:

case class FpWriter[T, C <: Monoid[_]]( value : T, context : C ) extends Monad[({ type L[A] = FpWriter[A, C] })#L, T] {
  ...
}