Improved build time and other benefits by using sbt-crossproject

2018-06-01

In 2014 we created a new website (this one) and being being Scala fanatics here at edc4it, we decided to use scalajs together with angular (through angulate). The latter was probably not the best decision but it seemed good at the time (at the moment we are considering moving to outwatch, Binding.scala or monadic-html but are actually also discussing not using any framework). However the decision for scalajs has proven to be awesome.

At the back-end we are running Akka HTTP (migrated from Spray) and we are sharing the json model classes between the front- and back-end.

Previous approach (additional source/resource directories)

At the time the recommended approach was to customise the paths of the modules needing access to these shared classes. This is how these modules would add additional source/resource directories:

val SharedSrcDir = "shared"

lazy val frontEnd = project
                      .settings(sharedDirectorySettings: _*)
                      …

lazy val backEnd = project
                      .settings(sharedDirectorySettings: _*)
                      …
                      
lazy val shared = (project in file(SharedSrcDir))                       

lazy val sharedDirectorySettings = Seq(
    unmanagedSourceDirectories in Compile += new File((file(".") / SharedSrcDir / "src" / "main" / "scala").getCanonicalPath),
    unmanagedSourceDirectories in Test += new File((file(".") / SharedSrcDir / "src" / "test" / "scala").getCanonicalPath),
    unmanagedResourceDirectories in Compile += file(".") / SharedSrcDir / "src" / "main" / "resources",
    unmanagedResourceDirectories in Test += file(".") / SharedSrcDir / "src" / "test" / "resources"
)

This approached worked well, but always felt a but dirty and smelly. First of all it makes it hard to manage the dependency of the "shared" module. You need to add these libraries to each of to the modules depending on the shared code, In our case we have many front-end modules, so that adds up.

Another problem is build time. The shared classes get compiled one time per module depending on it. Again in our case that was quite a few times.

A problem for a few of the developers with this approach is the fact that IDEs such as IntelliJ don't support the above configuration. This means the classes in the shared "module" are not available as far as IntelliJ is concerned. You have to manually add the module dependency in the "module settings" of the project. This means that every time the sbt project needs to be reimport you loose that configuration.

The real issue with this last problem is the lack of a formal project dependency. In fact this is the fundamental problem with this approach.

sbt-crossproject to the rescue

All the of the problems are solved by using sbt-crossproject. This project allows "Cross-Compiling Scala.js, JVM and Native". We have written a second blog article explaining step by step how to setup a project using ScalaJS front-end with a Akka-HTTP backend with shared classes using sbt-crossproject. Here we will just show the result of that.

Setting up sbt-crossproject is very simple in fact:

Add the required plugins (in this case we need cross-compiling for JS and the JVM)

addSbtPlugin("org.portable-scala" % "sbt-scalajs-crossproject" % "0.4.0")
addSbtPlugin("org.scala-js" % "sbt-scalajs" % "0.6.23")

The shared classes are then placed in a `CrossProject:

lazy val shared = crossProject(JSPlatform, JVMPlatform)
        .crossType(CrossType.Pure) 
        .settings(
           // shared settings
        )
        .jvmSettings(
            // Add JVM-specific settings here
        )
        .jsSettings(
            // Add JS-specific settings here
        )

I guess most is self-explanatory, with the possible exception of crossType. This setting determines the source tree layout of the cross project. We are using Pure which means there are no directories for platform-specific code and all code in the src folder is considered "shared". Another setting is Pure which creates source directories such as js, jvm and shared. We have no platform-specific code, so only need a single src folder.

Next you define platform specific sbt Projects and add those as proper dependencies

lazy val sharedJVM = shared.jvm
lazy val sharedJS = shared.js

lazy val frontEnd = project
                      .dependsOn(sharedJS)
                      …

lazy val backEnd = project
                      .dependsOn(sharedJVM)
                      …

That's it! as simple as that.

The only small problem we had is that we had our JS modules on Scala 2.11 and JVM on Scala 2.12. This however did not work for us and we had to update some ScalaJS dependencies to work with Scala 2.12

We are very happy with the new build times and the fact that we can now properly manage library dependencies. (Plus our IntelliJ users are delighted as it now works directly due to proper module dependencies)

We know SBT gets a lot of bad press, but we here ar edc4it for one love SBT and we love it now even better.

Note we have updated our ScalaJS course and it now teaches/uses sbt-crossproject as well.