"; */ ?>


Dec 15

Functional Programming for Humans

A couple of years ago, maybe even three, we had our Chariot Day. It’s a conference within our small company where we, developers, talk to us developers.

Not everyone at that point did functional programming, and it was fun to go over the ultimate FP power and some of the “why”s.

So Mr. Dan and I sat down and created a talk to lure “non functional” people in. We talked about assignment, concurrency, equality, thinking in sequences, took over the enterprise by writing Yelp and Trading Forecast in both: imperative and functional style, took on some design patterns, etc..

We had great discussions during the talk, but years went by and it remained to be internal. This is now fixed:

Oct 13

Scala: Where Ingenuity Lies

Love(d) It!

From my two years of “making money” with Scala, I went from loving an every bit of it to disliking majority of it. Not all of it though, since I still feel that “guilty pleasure” when I get to do Scala: it is a more concise Java, all the stream APIs built in, e.g. map/filter/groupBy/reduce/.. (Java 8 calls them Streams), pattern matching, nice way to construct things, “suggested” immutability, etc..

AKKA and Visual Basic

I am lucky I learned to be dangerous in Erlang before I picked up Scala. Hence as I was going deeper and deeper into AKKA, I could not help but notice how AKKA is missing the point of Erlang simplicity. It was 1.2, and of course 2.X fixed a lot of it, but again, it still has this feeling of leaking “unnecessary cleverness”.

And of course Erlang process isolation is what makes it the platform for distributed systems, where AKKA’s biggest limitation is, well.. JVM really. It had become apparent to me that, while I still liked Scala, whenever I need to create/work with distributed systems, Erlang is a lot simpler and does a much better job. With enough time put into Visual Basic, it can also “embrace” OTP, but .. why?

Ain’t Exactly Hammock Driven

Doing Scala I constantly lived in the world of the category theory. Not that it is necessary to know it to write decent Scala code, but anywhere you go, everybody “talks” it. It was ok, but felt somewhat like an unnecessary mental overload. Not the category theory itself (it was very nice), but the burden of constantly staying on your monad toes.

It’s About Time: Scala State Succeeded by Clojure

Finally, learning and appreciating Clojure, made it quite obvious that Scala is just.. overcooked. At that point it seemed to be an ML’s ugly step sister. Something that I could do in 10 lines of Clojure, I could also do in 15 lines of Scala (more often than not), but what an ugly, and not at all intuitive, 15 lines that was in comparison.

Clojure also taught me that it is a lot easier to reason about time when the whole codebase is just a series of state successions: (f””’… (f” (f’ (f 42)))), which makes a “time increment” be just a single increment from (f) to (f’). This, by the way, is also the reason why the resulting codebase in Clojure is tiny compared to Scala. Not because it is dynamically typed, but because Clojure is opinionated as a language (immutability, composition, state succession, …), and Scala is opinionated as a community, while the language lags behind.

It is Not Because Of Type

Typed Clojure is great and makes Clojure optionally typed.

Interesting thing though.. I have had problems with “would have better checked it at compile time” in Groovy, but rarely, if ever, in Clojure.

I suspect the reason behind this is immutability and just plain simple seq API (a.k.a. collections): [], (), {}, #{}, where everything has pretty much the same semantics.

Scala however, is a soup of mutable and immutable collections, with another soup of functions (which are explicitly objects => may also carry state) and “stateful” classes. Hence when you program in Scala, it is imperative to have a strong type system, otherwise there is no way to know whether A is contravariant to B and covariant to C and the method M can take it. In Clojure it is mostly a function that takes a seq or a map => not much to sweat about.

Where Ingenuity Lies

Simplicity and elegancy take a lot of work and dedication. It is a lot easier to write yet another ML, but make it several times more complex. It is hard however to absorb and channel all that complexity through a very simple set of APIs with minimal syntax, as Rich did with Clojure.


May 12

Scala: Fun with CanBuildFrom

As I found out through trying.. It may not be an easy task to explain Scala’s CanBuildFrom.

Before I dive into a quick gist, I think it’d be helpful to mention the best explanation of what happens behind the CanBuildFrom’s scenes that can be found on Stack Overflow in this answer.

The gist is, Scala has multiple layers of collections extending different capabilities. Let’s look at one such capability: TraversableLike, that most of the collections implement. Since let’s be honest, a collection is not very useful if it cannot be traversed. One of the most “famous” methods from TraversableLike is “map”:

def map[B, That](f: A => B)(implicit bf: CanBuildFrom[Repr, B, That]): That = {
  val b = bf(repr)
  for (x <- this) b += f(x)

Despite of the fact that it is “Scala looking”, it is actually quite simple => takes each element of a collection that it is called on, applies a provided function “f” to each element of the collection, and returns another collection “That” as the result.

The interesting bit here is:

... (implicit bf: CanBuildFrom[Repr, B, That]) ...

which seems a bit awkward (aren’t all Scala implicits..). “implicit” just means that a Scala compiler will search for this type “CanBuildFrom[Repr, B, That]” anywhere in the “scope”. In this case it’ll first look whether there is an “implicit CanBuildFrom[Repr, B, That]..” defined on the collection that the “map” is invoked on, then it’ll look in its super type/class, etc.. until it finds it.

Once it finds it, it’ll use that as a “builder” for “That” resulting collection. The way it looks for it though is not just “let me look if “CanBuildFrom” is there”, but also “let me look if “CanBuildFrom” is there that is parametrized with a given ‘Repr’ (e.g. collection) and ‘B’ (element type)”.

Here is a quick example. Let’s say we have a BitSet:

scala> import scala.collection.immutable.BitSet
import scala.collection.immutable.BitSet
scala> val bits = BitSet( 42, 84, 126 )
bits: scala.collection.immutable.BitSet = BitSet(42, 84, 126)

Once we map over this BitSet with a function (“/ 2L”) that produces something different than “Int”s as elements, a BitSet can no longer handle the result (BitSet can only have Ints as its elements) hence a Scala compiler jumps to a super class of a BitSet, which is a Set, and uses its “CanBuildFrom”, since it is a bit more generic:

implicit def canBuildFrom[A]: CanBuildFrom[Coll, A, Set[A]] = setCanBuildFrom[A]

Here “A” matches a “Long” that is now (after a map was applied) a type of resulting elements:

scala> val aintBits = bits.map( _ / 2L )
aintBits: scala.collection.immutable.Set[Long] = Set(21, 42, 63)

But we want our BitSet back.. Give me my BitSet back I say:

scala> val bitsAgain = aintBits.map( _.toInt )
val bitsAgain = aintBits.map( _.toInt )
bitsAgain: scala.collection.immutable.Set[Int] = Set(21, 42, 63)

But no, it does not.. And how would it know I need a BitSet. Hmm.. Give me my BitSet I urge you:

scala> val bitsAgain = aintBits.map( _.toInt ).asInstanceOf[BitSet]
val bitsAgain = aintBits.map( _.toInt ).asInstanceOf[BitSet]
java.lang.ClassCastException: scala.collection.immutable.Set$Set3 cannot be cast to scala.collection.immutable.BitSet
                              ... ...

But no, it does not..

Logically if “CanBuildFrom” is what got us a Set from a BitSet in the first place, can it be used to get a BitSet back?

Well, let’s see. We know that we have a Set of Longs (Set[Long]), where each element after applying a map function “toInt” is of type “Int”, and we need a BitSet back. Let’s create our own “CanBuildFrom” that does just that:

scala> import scala.collection.generic.CanBuildFrom
import scala.collection.generic.CanBuildFrom
scala> val setToBitSetBuilder = new CanBuildFrom[Set[Long], Int, BitSet] { def apply(from: Set[Long]) = this.apply(); def apply() = BitSet.newBuilder }
setToBitSetBuilder: java.lang.Object with scala.collection.generic.CanBuildFrom[Set[Long],Int,scala.collection.immutable.BitSet] = $anon$1@60bc1caa

Now let’s use it:

scala> val bitsAgain = aintBits.map( _.toInt )( setToBitSetBuilder )
val bitsAgain = aintBits.map( _.toInt )( setToBitSetBuilder )
bitsAgain: scala.collection.immutable.BitSet = BitSet(21, 42, 63)

And woo hoo, the “bitsAgain” is truly a BitSet again. What really happened, a Scala compiler was looking for an implicit “CanBuildFrom” for a collection “Set[Long]” and the (resulting) element type “Int”. And we just handed such a thing (“setToBitSetBuilder”) to it. “setToBitSetBuilder” just returns a “builder” that is used to build a resulting collection. In this case we use Scala’s own “BitSet.newBuilder”.

To make it more readable, a pimp my library pattern can be later used => aintBits.to[BitSet].

This is rather a quick overview of what “CanBuildFrom” is, and it does not really discuss a function currying which is used by “map(A)(B):C”, skims over implicits, etc.. But it gives a little insight to where and how “CanBuildFrom” can be used.

Oct 11

AKKA Scheduler: Sending Message to Actor’s Self on Start

Akka has a little scheduler written using actors. This can be convenient if you want to schedule some periodic task for maintenance or similar. It allows you to register a message that you want to be sent to a specific actor at a periodic interval.

How Does AKKA Schedule Things?

Behind the scenes, AKKA scheduler relies on “ScheduledExecutorService” from the “java.util.concurrent” package. Hence when AKKA Scheduler needs to schedule “a message sent to an actor, given a certain initial delay and interval”, it just wraps the task of sending a message in a “java.lang.Runnable”, and uses a “ScheduledExecutorService” to schedule it:

service.scheduleAtFixedRate( createSendRunnable( receiver, message, true ), 

“Heartbeat” Actor

Let’s look at the example of scheduling a message that should be sent to an Actor’s “self” as the Actor on start. Why? Because it is a cool use case : )

“Heartbeat” would be an ideal example of such use case => “When a ‘Hearbeat Actor’ starts, it should start sending heartbeats with a given interval (e.g. every 2 seconds)”

Creating a Message

First we need to create a message that will be scheduled to be sent every so often. We’ll call it a “SendHeartbeat” message:

sealed trait HeartbeatMessage
case object SendHeartbeat extends HeartbeatMessage
Heartbeat of the Hollywood

Since sending a heartbeat needs to be scheduled as the Actor starts, the scheduling logic should be placed in the AKKA “preStart()” hook, which is called right before the Actor is started:

override def preStart() {
  logger.debug( "scheduling a heartbeat to go out every " + interval + " seconds" )
  // scheduling the task (with the 'self') should be the last statement in preStart()
  scheduledTask = Scheduler.schedule( self, SendHeartbeat, 0, interval, TimeUnit.SECONDS )

Another thing to note, all the other non scheduling on start logic, if any, should go before the call to the scheduler, otherwise the task will not be scheduled.

Heartbeat should also be stoppable. We could have called “Scheduler.shutdown()” in Actor’s “postStop()”, but first, this would stop all the other tasks that were potentially scheduled by others, and second, it will result in a very dark AKKA magic behavior.

Instead, the heartbeat task itself should be cancelled => which is lot cleaner than calling for the dark magic for no good reason:

override def postStop() {
  scheduledTask.cancel( true )

Having the above two in mind here is the Hollywood Heartbeat himself:

class Heartbeat ( val interval: Long ) extends Actor {
  private val logger = LoggerFactory.getLogger( this.getClass )
  private var scheduledTask: ScheduledFuture[AnyRef] = null
  override def preStart() {
    logger.debug( "scheduling a heartbeat to go out every " + interval + " seconds" )
    // scheduling the task (with the 'self') should be the last statement in preStart()
    scheduledTask = Scheduler.schedule( self, SendHeartbeat, 0, interval, TimeUnit.SECONDS )
  def receive = {
    case SendHeartbeat =>
      logger.debug( "Sending a hearbeat" )
      // sending a heartbeat here.. socket.write( heartbeatBytes )
    case unknown =>
      throw new RuntimeException( "ERROR: Received unknown message [" + unknown + "], can't handle it" )
  override def postStop() {
    scheduledTask.cancel( true )
Making Sure the Heart is Beating

We’ll use ScalaTest to run the beast. This is more of a runner than a real test, since it does not really test for anything, but it proves the point:

class HeartbeatTest extends WordSpec  {
  val heartBeatInterval = 2
  "it" should {
    "send heartbeats every" + heartBeatInterval + " seconds" in {
       val heartbeat = actorOf ( new Heartbeat( heartBeatInterval ) ).start()
       Thread.sleep( heartBeatInterval * 1000 + 3000 )

And.. We are “In the Money“:

17:02:54,118 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [STDOUT] to Logger[ROOT]
Sending a hearbeat
Sending a hearbeat
Sending a hearbeat
Process finished with exit code 0

Jan 11

Starting with Scalate

scalateScalate: Scala Template Engine: like JSP without the crap but with added Scala coolness

Here is a one minute way to start playing with Scalate: 30 seconds to finish reading this + 15 seconds to calm inner excitement + 15 seconds to actually do it. Get ready.. it is that simple.

( Step 1 is OS X based, check these install instructions in case your OS is different ):

Step 1. Install Scalate ( makes sense to install Scala as well, if you don’t have it )

sudo brew install scala
sudo brew install scalate

Step 2. Create a Scalate project

scalate create guice org.dotkam scalatez

where “org.dotkam” is a user defined groupId and “scalatez” is a user defined artifactId [ hellllo maven ]

NOTE: "scalate create jersey org.dotkam scalatez" may give you "Invalid syntax: No such archetype 'jersey' possible values are (empty, guice)" depending on the current Scalate version. Don't be distressed: use 'guice' as shown above

Step 3. Run it

cd scalatez/
mvn jetty:run

Once you see

2011-01-07 14:28:42.597:INFO::Started SelectChannelConnector@
[INFO] Started Jetty Server

Fire up the browser and go to “http://localhost:8080

Scalate Default Home Resource


At this point you are good to start working with the Scalate project.
Let’s modify the current home page that you see in your browser:

vi src/main/webapp/WEB-INF/org/dotkam/scalatez/resources/HomeResource.index.scaml

once you save it, no need to restart: Jetty would pick you changes immediately ( e.g. “scanIntervalSeconds” parameter ), so just refresh that browser:

Scalate Custom Home Resource