IO Monad and Monad Transformers for Rookies


Fall 2019: updated to cats 2.x

This article is aimed at rookie Scala developers who want to take some first initial steps into a more monadic way of programming. You probably come from Java or C# and are confident with using monads such as Option and even Either. You feel more and more eager to try a more functional approach. Hey heck, you've got rid of most of the var declarations already. You are aware that there is a a lot more and most likely have already heard of the "IO Monad".

If this is you then join us below in a little journey. We would recommend you to follow along. The initial application can be found here: here .

Eventually you will learn about

  • The IO monad
  • Monad Transformers (EitherT)

We will be using cats-effect. The application already uses better-files for some simple file I/O.

There are lots of things missing from this article (async, using a threadpool for blocking IO, using MonadError for the errors using combinators) and some things could be done more idiomatic, but we wanted this to be for you, a rookie scala developer. There's a second part to this article that looks into that (see the link at the end of the post)

Initial application

Image you wrote the following rather straightforward program (available so you can follow along later on)

object FileConverter {

    def convert(in: File, out: File): Unit = {
        val str = read(in)
        write(out, str.toUpperCase())

    private[this] def write(file: File, s: String): Unit = {
        val bytes = s.getBytes.iterator

    private[this] def read(file: File): String = {
        val bytes = file.loadBytes
        new String(bytes)

As you can see the only public method convert takes two files and reads the contents of the first (using the private read method) and then writes the same text converted to upper case to the out file (using the private write). You used better-files for the actual I/O.

A small application using the above class might look something like this:

object DemoApp extends App {

    import FileReaderWriter._

    convert(File.temp / "demofile", File.temp / "result")

In order to test, create a file:

$ echo "hello friend" >> /tmp/demofile

And then run the program and check the contents of the resulting file (no pun intended with the name of the command below ;)

$ cat /tmp/result

You go to sleep and some cats in the street wake you up and now you are thinking about the code you wrote today. You would like to make it more "monadic" but have so far been afraid to unravel that world. You have so far been proud of using Option, once or twice Either and even used for-comprehensions.

IO Monad

The next day you decide to improve your code and make the side-effects your code have more explicit by using an IO Monad! As it was cats that woke you up, you will use the IO Monad from cats-effect (as you saw it as a sign)

We urge you to follow along, so make sure you downloaded the initial project.

You decide just to wrap the side-effects into IO and change the return types accordingly:

private def write(file: File, s: String): IO[Unit] = IO {
    val bytes = s.getBytes.iterator

private def read(file: File): IO[String] = IO {
    val bytes = file.loadBytes
    new String(bytes)

Compiling no longer works as your convert function needs to change as well. As you are using Monads with their map and flatMap you decide to use a for-comprehension:

def convert(in: File, out: File): IO[Unit] = {
    for {
        str <- read(in)
        _ <- write(out, str.toUpperCase())
    } yield ()

The read method will now returns an IO[String] and you use that for the write method, who on its turn returns now an IO[Unit] (iow a Unit inside the for-comprehension, hence the underscore). The whole for-comprehension now returns an IO[Unit] (we are yielding a Unit via () ).

With IO Monad it is turtles all the way down (as is with most of them) and therefore your convert method now also returns an IO[unit]. So this means we now need to turn our focus to the DemoApp itself (which is the object at the end of the universe if you like) and deal with the returned IO monad instance which has been pushed from the back.

So, the call to convert now returns IO[Unit], so lets assign it to a value:

val program: IO[Unit] = convert(File.temp / "demofile", File.temp / "result")

Now run the application again, but make sure you delete the previous out-file before your run

$ rm /tmp/result

Now when you run the application you'll notice nothing happens! No file is read and no file has been written (check, there is no /tmp/result)

This is the whole idea of the IO monad (well sort of). The program is not evaluated. We need to explicitly tell it to evaluate. This can be done in various ways, we will run it using unsafeRunSync:


This call will evaluate the IO monad and return "its" value. In our case the value is Unit (recall our convert method returns IO[Unit])

Run the program again and notice how it now produces a file!

After showing you code off to your co-workers with your precious IO Monad, you decide to call it a day. You feel a lot hipper than you did this morning (time to order that flannel shirt with that bucket hat you always wanted)


The next day one of your colleagues asks condescending if you thought about error handling and you realise you did not pay attention to that. However you've played with Either and Try before, so with a lot of confidence you tell your colleague: "No problem, won't take me more that 30 min to hack that in"

So you decide to wrap the calls to loadBytes and writeBytes into a Try(…). But because you don't like exceptions anymore, you decide to convert it to an Either and convert the possible Throwable to a custom Error ADT (see src/main/scala/demo/Error.scala)

import cats.implicits._

// for write
val bytes = s.getBytes.iterator
        .leftMap(t => IOError(t.getMessage))

// for read
        .map(bytes => new String(bytes))
        .leftMap(t => IOError(t.getMessage))

After lunch ;) You notice it no longer compiles as the return types no longer match:

Error:(31, 26) type mismatch;
    found   : scala.util.Either[IOError,String]
    required: String

Sure, you'll need to change the read method's return type from IO[String] to IO[Either[Error,String]] and similar the write needs to return IO[Either[Error,Unit]] now. While you're at it, you do the same for your convert method:

def convert(in: File, out: File): IO[Either[Error, Unit]] = …

private def write(file: File, s: String): IO[Either[Error,Unit]] = …

private def read(file: File): IO[Either[Error,String]] = …

You feel like you should be ordering those Doc Martens also soon.

Compile again, shoot a problem in the convert method:

Error:(15, 33) value toUpperCase is not a member of Either[demo.Error,String]

Now what? "I can't use a for-comprehension because i have a monad (Either) inside another monad (IO)", this does not feel good.

You need to rush for a dinner appointment for which you are already late, so this has to wait until tomorrow.

Monad Transformers

"So how is the error-handling coming along", your colleague asks the next day with her smirky smile.

You turn back at the convert method and keep trying different things, but nothing feels "good" and "monadic", which you were after to begin with.

Sure, within the for comprehension you are flatMapping the IO, which returns the Either. You in fact want to combine two monads. You want the effect of the IO monad together with the error effect ot the Either monad. This is what a monad transformer does.

Now (drum roll) to the rescue come Monad Transformers. The problem is that different monads don't compose. In this particular case we need an EitherT monad transformer which allows "the effect of an arbitrary type constructor F to be combined with the fail-fast effect of Either", "an EitherT[F, A, B] wraps a value of type F[Either[A, B]]" (both from api-doc)

So sounds easy, let's wrap our IO into EitherT using its constructor and change the type signatures accordingly:

def convert(in: File, out: File): IO[Either[Error, Unit]] = 
  for {
    str <- EitherT(read(in))
    _ <- EitherT(write(out, str.toUpperCase()))
  } yield ()

If you try to compile, you'll be confronted with yet another error. But before we dive into that one, take a moment to feel awesome about what's happening! The str.toUpperCase() is no longer a problem, because str is now of type String due to the monad Transformer you are using!

While it is getting dark outside you look at the error message presented to you this time:

Error:(14, 9) type mismatch;
 found   :[cats.effect.IO,demo.IOError,Unit]
 required: cats.effect.IO[Either[demo.Error,Unit]]
    str <- EitherT(read(in))

Your convert method return type is a "simple" IO[Either[Error, Unit]], but the result of the for-comprehension is a EitherT[IO,Error,Unit]. That makes sense, but we won't propagate the monad transformer to the application. So we need to get the value from the EitherT:

def convert(in: File, out: File): IO[Either[Error, Unit]] = {
    val r = for {
        str <- read(in)
        _ <- write(out, str.toUpperCase())
    } yield ()
    r.value // this gets the IO[Either[Error, Unit]]

You can now change the DemoApp to check if there was an error using pattern matching on its resulting Either[Error,Unit]

val program: IO[Either[Error, Unit]] =
        convert(File.temp / "demofile", File.temp / "result")

program.unsafeRunSync() match {
    case Left(error) => println(s"error: $error")
    case Right(_) =>  println("successfully written")

It works as expected!

But wait a minute... we are now performing side-effects after calling unsafeRunSync. That does not feel right.

It is close to midnight, but you now got the hang of it, plus this feels much simpler. Just wrap the side-effects inside a IO Monad as you've done before:

def handle(e: Either[Error, Unit]) = e match {
    case Left(error) => IO { println(s"error: $error") }
    case Right(value) => IO { println("successfully written") }

And then just chain this IO monad with the previous one. Let's not use a for-comprehension here, but just a flatMap

val program: IO[Either[Error, Unit]] =
        convert(File.temp / "demofile", File.temp / "result")
program.flatMap(e => handle(e)).unsafeRunSync()

You run the program and it all works.

You head home and find a box from Zalando with your ordered goods:

  • A green flannel shirt
  • bright suspenders
  • A black bucket hat
  • a pair of oversized glasses
  • a pair of Doc Martens

You say goodnight to the cats that woke you earlier this week and turn in to have a good night sleep, finally.

You can find the solution in the solution branch of the git project here

There's a second part that makes your solution more idiomatic. It is available here