kam

one of those blogs

As I found out through trying.. It may not be an easy task to explain Scala’s CanBuildFrom.

Before I dive into a quick gist, I think it’d be helpful to mention the best explanation of what happens behind the CanBuildFrom’s scenes that can be found on Stack Overflow in this answer.

The gist is, Scala has multiple layers of collections extending different capabilities. Let’s look at one such capability: TraversableLike, that most of the collections implement. Since let’s be honest, a collection is not very useful if it cannot be traversed. One of the most “famous” methods from TraversableLike is “map”:

def map[B, That](f: A => B)(implicit bf: CanBuildFrom[Repr, B, That]): That = {
  val b = bf(repr)
  b.sizeHint(this) 
  for (x <- this) b += f(x)
  b.result
}

Despite of the fact that it is “Scala looking”, it is actually quite simple => takes each element of a collection that it is called on, applies a provided function “f” to each element of the collection, and returns another collection “That” as the result.

The interesting bit here is:

... (implicit bf: CanBuildFrom[Repr, B, That]) ...

which seems a bit awkward (aren’t all Scala implicits..). “implicit” just means that a Scala compiler will search for this type “CanBuildFrom[Repr, B, That]” anywhere in the “scope”. In this case it’ll first look whether there is an “implicit CanBuildFrom[Repr, B, That]..” defined on the collection that the “map” is invoked on, then it’ll look in its super type/class, etc.. until it finds it.

Once it finds it, it’ll use that as a “builder” for “That” resulting collection. The way it looks for it though is not just “let me look if “CanBuildFrom” is there”, but also “let me look if “CanBuildFrom” is there that is parametrized with a given ‘Repr’ (e.g. collection) and ‘B’ (element type)”.

Here is a quick example. Let’s say we have a BitSet:

scala> import scala.collection.immutable.BitSet
import scala.collection.immutable.BitSet
 
scala> val bits = BitSet( 42, 84, 126 )
bits: scala.collection.immutable.BitSet = BitSet(42, 84, 126)

Once we map over this BitSet with a function (“/ 2L”) that produces something different than “Int”s as elements, a BitSet can no longer handle the result (BitSet can only have Ints as its elements) hence a Scala compiler jumps to a super class of a BitSet, which is a Set, and uses its “CanBuildFrom”, since it is a bit more generic:

implicit def canBuildFrom[A]: CanBuildFrom[Coll, A, Set[A]] = setCanBuildFrom[A]

Here “A” matches a “Long” that is now (after a map was applied) a type of resulting elements:

scala> val aintBits = bits.map( _ / 2L )
aintBits: scala.collection.immutable.Set[Long] = Set(21, 42, 63)

But we want our BitSet back.. Give me my BitSet back I say:

scala> val bitsAgain = aintBits.map( _.toInt )
val bitsAgain = aintBits.map( _.toInt )
bitsAgain: scala.collection.immutable.Set[Int] = Set(21, 42, 63)

But no, it does not.. And how would it know I need a BitSet. Hmm.. Give me my BitSet I urge you:

scala> val bitsAgain = aintBits.map( _.toInt ).asInstanceOf[BitSet]
val bitsAgain = aintBits.map( _.toInt ).asInstanceOf[BitSet]
java.lang.ClassCastException: scala.collection.immutable.Set$Set3 cannot be cast to scala.collection.immutable.BitSet
                              ... ...

But no, it does not..

Logically if “CanBuildFrom” is what got us a Set from a BitSet in the first place, can it be used to get a BitSet back?

Well, let’s see. We know that we have a Set of Longs (Set[Long]), where each element after applying a map function “toInt” is of type “Int”, and we need a BitSet back. Let’s create our own “CanBuildFrom” that does just that:

scala> import scala.collection.generic.CanBuildFrom
import scala.collection.generic.CanBuildFrom
 
scala> val setToBitSetBuilder = new CanBuildFrom[Set[Long], Int, BitSet] { def apply(from: Set[Long]) = this.apply(); def apply() = BitSet.newBuilder }
setToBitSetBuilder: java.lang.Object with scala.collection.generic.CanBuildFrom[Set[Long],Int,scala.collection.immutable.BitSet] = $anon$1@60bc1caa

Now let’s use it:

scala> val bitsAgain = aintBits.map( _.toInt )( setToBitSetBuilder )
val bitsAgain = aintBits.map( _.toInt )( setToBitSetBuilder )
bitsAgain: scala.collection.immutable.BitSet = BitSet(21, 42, 63)

And woo hoo, the “bitsAgain” is truly a BitSet again. What really happened, a Scala compiler was looking for an implicit “CanBuildFrom” for a collection “Set[Long]” and the (resulting) element type “Int”. And we just handed such a thing (“setToBitSetBuilder”) to it. “setToBitSetBuilder” just returns a “builder” that is used to build a resulting collection. In this case we use Scala’s own “BitSet.newBuilder”.

To make it more readable, a pimp my library pattern can be later used => aintBits.to[BitSet].

This is rather a quick overview of what “CanBuildFrom” is, and it does not really discuss a function currying which is used by “map(A)(B):C”, skims over implicits, etc.. But it gives a little insight to where and how “CanBuildFrom” can be used.

Akka has a little scheduler written using actors. This can be convenient if you want to schedule some periodic task for maintenance or similar. It allows you to register a message that you want to be sent to a specific actor at a periodic interval.

Behind the scenes, AKKA scheduler relies on “ScheduledExecutorService” from the “java.util.concurrent” package. Hence when AKKA Scheduler needs to schedule “a message sent to an actor, given a certain initial delay and interval”, it just wraps the task of sending a message in a “java.lang.Runnable”, and uses a “ScheduledExecutorService” to schedule it:

service.scheduleAtFixedRate( createSendRunnable( receiver, message, true ), 
                             initialDelay, 
                             delay, 
                             timeUnit).asInstanceOf[ScheduledFuture[AnyRef]]

Let’s look at the example of scheduling a message that should be sent to an Actor’s “self” as the Actor on start. Why? Because it is a cool use case : )

“Heartbeat” would be an ideal example of such use case => “When a ‘Hearbeat Actor’ starts, it should start sending heartbeats with a given interval (e.g. every 2 seconds)”

First we need to create a message that will be scheduled to be sent every so often. We’ll call it a “SendHeartbeat” message:

sealed trait HeartbeatMessage
case object SendHeartbeat extends HeartbeatMessage

Since sending a heartbeat needs to be scheduled as the Actor starts, the scheduling logic should be placed in the AKKA “preStart()” hook, which is called right before the Actor is started:

override def preStart() {
 
  logger.debug( "scheduling a heartbeat to go out every " + interval + " seconds" )
 
  // scheduling the task (with the 'self') should be the last statement in preStart()
  scheduledTask = Scheduler.schedule( self, SendHeartbeat, 0, interval, TimeUnit.SECONDS )
}

Another thing to note, all the other non scheduling on start logic, if any, should go before the call to the scheduler, otherwise the task will not be scheduled.

Heartbeat should also be stoppable. We could have called “Scheduler.shutdown()” in Actor’s “postStop()”, but first, this would stop all the other tasks that were potentially scheduled by others, and second, it will result in a very dark AKKA magic behavior.

Instead, the heartbeat task itself should be cancelled => which is lot cleaner than calling for the dark magic for no good reason:

override def postStop() {
  scheduledTask.cancel( true )
}

Having the above two in mind here is the Hollywood Heartbeat himself:

class Heartbeat ( val interval: Long ) extends Actor {
 
  private val logger = LoggerFactory.getLogger( this.getClass )
  private var scheduledTask: ScheduledFuture[AnyRef] = null
 
  override def preStart() {
 
    logger.debug( "scheduling a heartbeat to go out every " + interval + " seconds" )
 
    // scheduling the task (with the 'self') should be the last statement in preStart()
    scheduledTask = Scheduler.schedule( self, SendHeartbeat, 0, interval, TimeUnit.SECONDS )
  }
 
  def receive = {
 
    case SendHeartbeat =>
 
      logger.debug( "Sending a hearbeat" )
      // sending a heartbeat here.. socket.write( heartbeatBytes )
      true
 
    case unknown =>
 
      throw new RuntimeException( "ERROR: Received unknown message [" + unknown + "], can't handle it" )
  }
 
  override def postStop() {
    scheduledTask.cancel( true )
  }
}

We’ll use ScalaTest to run the beast. This is more of a runner than a real test, since it does not really test for anything, but it proves the point:

class HeartbeatTest extends WordSpec  {
 
  val heartBeatInterval = 2
 
  "it" should {
 
    "send heartbeats every" + heartBeatInterval + " seconds" in {
       val heartbeat = actorOf ( new Heartbeat( heartBeatInterval ) ).start()
       Thread.sleep( heartBeatInterval * 1000 + 3000 )
       heartbeat.stop()
     }
  }
}

And.. We are “In the Money“:

17:02:54,118 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
 
Sending a hearbeat
Sending a hearbeat
Sending a hearbeat
 
Process finished with exit code 0

Scalate: Scala Template Engine: like JSP without the crap but with added Scala coolness

Here is a one minute way to start playing with Scalate: 30 seconds to finish reading this + 15 seconds to calm inner excitement + 15 seconds to actually do it. Get ready.. it is that simple.

( Step 1 is OS X based, check these install instructions in case your OS is different ):

Step 1. Install Scalate ( makes sense to install Scala as well, if you don’t have it )

sudo brew install scala
sudo brew install scalate

Step 2. Create a Scalate project

scalate create guice org.dotkam scalatez

where “org.dotkam” is a user defined groupId and “scalatez” is a user defined artifactId [ hellllo maven ]

NOTE: "scalate create jersey org.dotkam scalatez" may give you "Invalid syntax: No such archetype 'jersey' possible values are (empty, guice)" depending on the current Scalate version. Don't be distressed: use 'guice' as shown above

Step 3. Run it

cd scalatez/
mvn jetty:run

Once you see

2011-01-07 14:28:42.597:INFO::Started SelectChannelConnector@0.0.0.0:8080
[INFO] Started Jetty Server

Fire up the browser and go to “http://localhost:8080”

DONE.

At this point you are good to start working with the Scalate project.
Let’s modify the current home page that you see in your browser:

vi src/main/webapp/WEB-INF/org/dotkam/scalatez/resources/HomeResource.index.scaml

once you save it, no need to restart: Jetty would pick you changes immediately ( e.g. “scanIntervalSeconds” parameter ), so just refresh that browser: