06
Apr 11

Spock It Like You Mean It!

So here we go.. Yesterday night we hacked our way into The Ancient Database where besides the data about ancients themselves, we found incredible stats about many different living species of all the planets ancients traveled to.

So what do we do now? Well, we need a way to query/read this data. So we asked Sam to develop a reader ( ancients called it a ‘DAO’ ) to find a number of living species on each planet since the “beginning of time”.

So she did, and called this “species reader” a ‘SpeciesDao’.

We, of course, trust Sam. But in ancient technology, trust is always based on challenging the assumption by testing all the possible permutations. So we are calling Mr. Spock to help us out…

Sam of course used a well known ancient technique ( code name: Spring Framework ) to create a “species reader”, so Mr. Spock will use all the goodies of this technique to inject all the dependencies ( e.g. data source, a reader itself, etc.. ) for a test.

Since Mr. Spock is a peaceful inhabitant from a planet Simplicity, it speaks a simple language called Groovy. This language is unique in a way that living beings of all the galaxies can pick it up and understand it within minutes, which by the way makes it a perfect choice for various intergalactic conferences.

To start off, we’ll tell Mr. Spock where to look for all the dependencies:

@ContextConfiguration( locations = [ "classpath:/conf/test-context.xml", "classpath:/conf/persistence-context.xml" ] )

The data we are going to test against will be in a form of DbUnit datasets, which by the way can be XML based, Excel based or even YAML based. In this example we are going to focus on planets of two galaxies that peak our curiosity the most: Milky Way and Pegasus.

  // Species of the Milky Way Galaxy by Planet
  static final P3X_888_SPECIES_DATASET = "/dataset/stage/p3x-888-species.xml"
  static final DAKARA_SPECIES_DATASET = "/dataset/stage/dakara-species.xml"
  ....
 
  // Species of the Pegasus Galaxy by Planet
  static final ASURAS_SPECIES_DATASET = "/dataset/stage/asuras-species.xml"
  static final WRAITH_HOMEWORLD_SPECIES_DATASET = "/dataset/stage/wraith-homeworld-species.xml"
  ....

Since it is a test, we already know the exact number of living spices since the ‘beginning of time’ for each dataset. And we’d need to call the reader Sam wrote ( ‘SpeciesDao’ ) for each dataset and compare it with an expected number. So you see, lots of repetitive actions. But no worries humans! Spock has an elegant way to deal with all these repetitions, by using its ‘WHERE’ power:

    where:
 
             planet                        |          numberOfSpeciesFound
 
      // Milky Way Galaxy
      P3X_888_SPECIES_DATASET              |                  888
      DAKARA_SPECIES_DATASET               |                 123804
      HELIOPOLIS_SPECIES_DATASET           |                   7
      .....

That’s neat. The only problem is if the number does not match ( yes, even Sam has bugs ) for example for planet Dakara, Mr. Spock will tell us that [ “something did not match up..” ], but will forget to mention that “it” did not match up specifically for a Dakara planet. And if we have thousands of such planets, it’ll be extremely difficult to find the culprit. But again, humans, this is easily solvable by using a Mr. Spock’s secret power: The power of ‘@Unroll’!

  @Unroll("#planet should have #numberOfSpeciesFound species found")
  def "only living species since begining of time should be found"() { ... }

By annotating a test method with @Unroll, in case where a number of living species found did not match a number that we expected, Mr. Spock will say just that. For example for Dakara, it’ll now say: “Dakara should have 123804 species found”, while also telling us the actual number that was found to compare. Pretty handy!

One last touch before we can fully trust Mr. Spock.. The way the ancient technology ( Spring ) was written, it won’t allow to inject collaborators ( e.g. data source ) statically before all the specifications / permutations. It can be tweaked to do that, but cmmon, who are we to tweak the ancients.. Instead we’ll tell Mr. Spock to do all the setup it needs only the first time a data source is injected:

    // setupSpec() cannot access Spring beans ( e.g. dataSource ), hence need to check it every time
    if ( ! databaseTester ) {
      databaseTester = new DataSourceDatabaseTester( dataSource )
    }

Now we are ready to roll. Let’s put all the pieces together and make sure Sam’s creation does what it’s supposed to:

 
@ContextConfiguration( locations = [ "classpath:/conf/test-context.xml", "classpath:/conf/persistence-context.xml" ] )
class FindNumberOfSpeciesTest extends Specification {
 
  // Species of the Milky Way Galaxy by Planet
  static final P3X_888_SPECIES_DATASET = "/dataset/stage/p3x-888-species.xml"
  static final DAKARA_SPECIES_DATASET = "/dataset/stage/dakara-species.xml"
  static final HELIOPOLIS_SPECIES_DATASET = "/dataset/stage/heliopolis-species.xml"
  static final ASCHEN_PRIME_SPECIES_DATASET = "/dataset/stage/aschen-prime-species.xml"
  static final P4X_650_SPECIES_DATASET = "/dataset/stage/p4x-650-species.xml"
  static final VIS_UBAN_SPECIES_DATASET = "/dataset/stage/vis-uban-species.xml"
  static final PROCLARUSH_SPECIES_DATASET = "/dataset/stage/proclarush-species.xml"
  static final DAKARA_SPECIES_DATASET = "/dataset/stage/dakara-species.xml"
  static final HEBRIDAN_SPECIES_DATASET = "/dataset/stage/hebridan-species.xml"
 
  // Species of the Pegasus Galaxy by Planet
  static final ASURAS_SPECIES_DATASET = "/dataset/stage/asuras-species.xml"
  static final WRAITH_HOMEWORLD_SPECIES_DATASET = "/dataset/stage/wraith-homeworld-species.xml"
  static final SATEDA_SPECIES_DATASET = "/dataset/stage/sateda-species.xml"
  static final DAGAN_SPECIES_DATASET = "/dataset/stage/dagan-species.xml"
  static final LORD_PROTECTORS_SPECIES_DATASET = "/dataset/stage/lord-protectors-species.xml"
  static final M7G_677_SPECIES_DATASET = "/dataset/stage/m7g-677-species.xml"
  static final ATHOS_SPECIES_DATASET = "/dataset/stage/athos-677-species.xml"
 
  static final THE_BEGINNING_OF_TIME = Date.parse( "yyyy-M-d", "1979-01-01" )
 
  @Autowired
  SpeciesDao speciesDao
  @Autowired
  DataSource dataSource
 
  @Shared IDatabaseTester databaseTester
 
  @Unroll("#planet should have #numberOfSpeciesFound species found")
  def "only living species since the beginning of time should be found"() {
 
    when: stageTestData planet
 
    then: speciesDao.findNumberOfSpeciesLivingSince( THE_BEGINNING_OF_TIME ) == numberOfSpeciesFound
 
    where:
 
             planet                        |          numberOfSpeciesFound
 
      // Milky Way Galaxy
      P3X_888_SPECIES_DATASET              |                  888
      DAKARA_SPECIES_DATASET               |                 123804
      HELIOPOLIS_SPECIES_DATASET           |                   7
      ASCHEN_PRIME_SPECIES_DATASET         |                 2423984
      P4X_650_SPECIES_DATASET              |                  2600
      VIS_UBAN_SPECIES_DATASET             |                   0
      PROCLARUSH_SPECIES_DATASET           |                 8869346
      DAKARA_SPECIES_DATASET               |                  5672
      HEBRIDAN_SPECIES_DATASET             |                   67
 
      // Pagasus Galaxy
      ASURAS_SPECIES_DATASET               |                  823
      WRAITH_HOMEWORLD_SPECIES_DATASET     |                 62634
      SATEDA_SPECIES_DATASET               |                  327
      SATEDA_SPECIES_DATASET               |                   0
      DAGAN_SPECIES_DATASET                |                  777
      LORD_PROTECTORS_SPECIES_DATASET      |                  8786
      M7G_677_SPECIES_DATASET              |                  4739
      ATHOS_SPECIES_DATASET                |                3767822
 
  }
 
  def stageTestData = { dataSetLocation ->
 
    dataSource != null
    speciesDao != null
 
    // setupSpec() cannot access Spring beans ( e.g. dataSource ), hence need to check it every time
    if ( ! databaseTester ) {
      databaseTester = new DataSourceDatabaseTester( dataSource )
    }
 
    databaseTester.setDataSet( new FlatXmlDataFileLoader().load( dataSetLocation ) )
    databaseTester.onSetup()
  }
}

Spock away Humans!


05
Apr 11

Mobile Devices Visiting Dotkam

Looking at mobile stats I can conclude that Android users are most interested in my content ( number of users vs average 21 seconds on the page ), where iPhone users are most curious ( max number of visits ), and of course thank you the lonely “Sony” user who spent 4 and a half minutes here :)

Moible Devices Visiting Dotkam

But overall most of my daily visits ( currently about 1300 ) are coming from other places. Need to work on that..


15
Mar 11

Spring Batch: CommandLineJobRunner to Run Multiple Jobs

Sometimes this requirement may jump at you: “I still want to have them as several jobs, but I want no operational overhead, and need to run them in sequence via a single launch…”.

So if you were using CommandLineJobRunner to launch these jobs, it would only be natural to aggregate those calls under your own e.g. “CommandLineMultipleJobRunner” that just delegates its calls to “CommandLineJobRunner” for each job in a list..

The only caveat is the default SystemExiter which is set to JvmSystemExiter, which makes it imposible to make more than one launch using a “CommandLineJobRunner”, since “JvmSystemExiter” does:

public void exit(int status) {
    System.exit(status);
}

Fortunately, there is a static accessor to override this behavior:

// disabling a default "JvmSystemExiter" to be able to run several times
CommandLineJobRunner.presetSystemExiter( new SystemExiter() { public void exit( int status ) {} } );

Now, call “CommandLineJobRunner.main( String[] args )” as many times as you’d like.

If you have a choice, think about creating a single multi step job instead, but if you don’t have a luxury, “CommandLineMultipleJobRunner” should now be not all that hard to implement.

Batch Away! :)


10
Mar 11

Having Fun with Groovy Date Parsing

How do you convert a String to Date in Groovy? Well it’s simple:

   Date.parse( "yyyy-M-d", "2011-01-15" )

Now, let’s say I would like to shorten this: yes, it looks to long, remember the actual data I am interested in is “2011-01-15″, everything else means nothing really.. datawise.

Ok, so I can

  Date.metaClass.'static'.fromString = { str ->
    Date.parse( "yyyy-M-d", str )
  }

which gives me a shorter representation ( less fluff around the actual data ):

  Date.fromString( "2011-01-15" )

That’s not bad, but I would like to take it further :) I am staging data ( domain objects ) in my Spock tests, and need as less fluff as possible => “only data matters”. So here it is, the geeky solution:

Create a DataParser:

  class DateParser {
    def static te = { str -> Date.parse( "yyyy-M-d", str ) }
  }

Wherever you need parse dates, import it as ‘d':

  import org.dotkam.util.date.DateParser as d

Now create your d.tes ( I mean dates :) ) as:

  d.te( "2011-01-05" )

Awesome!
//TODO: it can of course be extended with multiple formats


10
Jan 11

Android Development: Best Practices

Cyanogen LogoI got introduced to an Android application development during Philly ETE conference by listening to “A guided tour of the Android ETE mobile application” talk, where Andrew Oswald ( a Java Architect from Chariot Solutions ) talked about creating an Android app for the conference, which was a cool introduction to “what it takes” to write one of those apps from scratch. I am looking forward to more talks around mobile development this year at Philly ETE conference as well.

Meanwhile I rely mostly on Android’s Developer Guide whenever I seek answers to my questions or/and best practices. Here are my notes on such practices I picked up listening to “A Beginner’s Guide to Android” Google I/O 2010 talk:

1. Avoid creating objects unless you really need to, try reusing Android’s API instead. Creating an object in a “desktop” world is relatively cheap, however in such a resource constraint environment as a mobile phone, it will drastically impact the performance.. not in a good way.

2. In order to avoid friendly “Force Close” popups from your applications, use Android’s “AsyncTask” which will allow to execute a certain activity in background:

 private class DownloadFilesTask extends AsyncTask<URL, Integer, Long> {
     protected Long doInBackground(URL... urls) {
         int count = urls.length;
         long totalSize = 0;
         for (int i = 0; i < count; i++) {
             totalSize += Downloader.downloadFile(urls[i]);
             publishProgress((int) ((i / (float) count) * 100));
         }
         return totalSize;
     }
 
     protected void onProgressUpdate(Integer... progress) {
         setProgressPercent(progress[0]);
     }
 
     protected void onPostExecute(Long result) {
         showDialog("Downloaded " + result + " bytes");
     }
 }

read more about AsyncTask in Android Developer Guide

3. Think about what an absolute minimum amount of updates / syncs you can do, and stick to this minimum. This will greatly improve battery life as well resource usage by the application.

4. Only use a WakeLock when you need one with as minimum level as possible: PARTIAL_WAKE_LOCK, SCREEN_DIM_WAKE_LOCK, FULL_WAKE_LOCK. Here is more about PowerManager

5. Respect a “Back” button: make sure it actually brings user back to a previous state rather than to another state of the application’s flow.

6. Always check whether “data transfer” is enabled on a device, before attempting to transfer data:

ConnectivityManager cm= ( ConnectivityManager ) getSystemService( Context.CONNECTIVITY_SERVICE );
boolean backgroundEnabled = cm.getBackgroundDataSetings();

this is especially important while roaming, when that “twitter update” can cost user a lot. So do respect user settings.

7. Don’t use undocumented ( not officially supported ) APIs => Next Android release your app is going to break.

8. Respect and use an application lifecycle:

void onCreate(Bundle savedInstanceState)
void onStart()
void onRestart()
void onResume()
void onPause()
void onStop()
void onDestroy()

read more about Android Application Lifecycle

9. Externalize resources: localization / optimized layouts / strings / array of strings / etc.. Android compiles them into a list of internal resources by assigning an integer ID to each of them, hence making it “cheaper” at runtime, and easier to change => since they are defined in a single location. Here is an example of auto generated ( random layout ) resources:

    public static final class layout {
        public static final int about_details=0x7f030000;
        public static final int all_upcoming_filtered_buttons=0x7f030001;
        public static final int details_text=0x7f030002;
        public static final int details_webview=0x7f030003;
        public static final int faq_details=0x7f030004;
        public static final int faq_details_row=0x7f030005;
        public static final int main_tabs=0x7f030006;
        public static final int map_details=0x7f030007;
 
        //.... the above is auto generated

10. Think of hiring or delegating UI to people who are designers. Beauty is important

11. Be resolution independent. Check out “layoutopt” and “hierarchyviewer” that come with Android SDK under “tools”. They help analyzing and optimizing layouts.

12. Consider using a “non sticky” services when appropriate:

@Override
public int onStartCommand( Intent intent, int flags, int startId ) {
 
     handleCommand( intent );
 
    // If this Service gets killed, don't restart it
    return START_NOT_STICKY;
}

this is useful for services that are going to be executed on a regular basis. So when you are pulling for updates every 10, 15 minutes, it is ok if one of such updates is missed in favor to a healthy resource management

13. Do not use foreground services unless you absolutely need to.

And if you do use foreground services, use an ongoing notification ( which, starting from Android 2.0, used automatically, if a service is started as a foreground one, but just to keep something in mind to be used for older OS versions )

14. Kill your own services via stopSelf():

@Override
prtoceted void onPostExecute( Void result ) {
    stopSelf();
}

15. Use Intents and Intent Filters to Leverage Other Apps

16. Prefer Alarms and Intent Receivers to Services

Now, with this in mind, go and Rock That Android Space!


10
Jan 11

Android: Prefer Alarms and Intent Receivers to Services

Cyanogen LogoContinue learning from “A Beginner’s Guide to Android” Google I/O 2010 talk, here is an example on how to use Intent Filter + Intent Receiver + Alarm to implement “schedule to execute every so often” functionality.

In Android, Alarms let you set an event to happen either in the future or on an ongoing basis in the future.

Let’s say we have a listener ( extends BroadcastReceiver ) that would execute a certain action ( MyService ):

public class MyReceiver extends BroadcastReceiver {
 
    @Override
    public void onReceive( Context context, Intent intent ) {
 
        Intent myIntent = new Intent( context, MyService.class );
        context.startService( myIntent );
    }
}

Now let’s connect/map this listener/receiver to a “REFRESH_THIS” Intent by creating an intent filter in a manifest file:

<receiver android:name="MyReceiver">
    <intent-filter>
        <action android:name="REFRESH_THIS"/>
    </intent-filter>
</receiver>

So whenever a system broadcasts a “REFRESH_THIS” intent, MyReceiver is going to spawn up and a “context.startService( myIntent )” is going to be executed.

Now in order to schedule “REFRESH_THIS” intent to be broadcasted, we would use an AlarmManager:

String alarm = Context.ALARM_SERVICE;
AlarmManager am = ( AlarmManager ) getSystemService( alarm );
 
Intent intent = new Intent( "REFRESH_THIS" );
PendingIntent pi = PendingIntent.getBroadcast( this, 0, intent, 0 );
 
int type = AlarmManager.ELAPSED_REALTIME_WAKEUP;
long interval = AlarmManager.INTERVAL_FIFTEEN_MINUTES;
long triggerTime = SystemClock.elapsedRealtime() + interval;
 
am.setRepeating( type, triggerTime, interval, pi );

The above Alarm will wake up a device every 15 minutes and execute MyReceiver’s onReceive() method. The cool thing is that even if your application is killed, this alarm will continue to run without your application running on the background consuming resources.

One thing to note..

Prefer Inexact Alarms …so OS can optimize when the alarm goes off

Why!?.. Let’s say there are 15 applications that set 15 alarms, which take a minute each to execute, and are all scheduled to be executed with a 15 minute interval => Potentially ( depending on the time they’ve been scheduled at ) they can end up executing every minute ( one after another ) resulting in Android device to be constantly on which is a dramatic impact at the resource usage.

“Inexact Alarms” would let OS “phase shift” these alarms to execute at the same time, rather than being arbitrary distributed depending on the time they were scheduled at. This allows OS to optimize and allocate resources in more intelligent fashion. So if you have something that needs to happen regularly, but does not need to happen at exact time, use “Inexact Alarms”.

In the above “alarm example”, in order to use Inexact Alarm change this line:

am.setRepeating( type, triggerTime, interval, pi );

to

am.setInexactRepeating( type, triggerTime, interval, pi );

Now the alarm will rely on the OS to optimize its execution time.


10
Jan 11

Android: Using Intents and Intent Filters to Leverage Other Apps

Cyanogen LogoUsing “Intent Filters” is a very powerful way to connect different applications together which allows greater reuse and makes a user experience transparent to the fact that more than one application is used to achieve a certain task.

Here is an example that is discussed in “A Beginner’s Guide to Android” Google I/O 2010 talk.

Let’s say there is an application that finds hotels and would like to use another application to book it. For that it creates an implicit “Intent” where it says: “hey android, I intent to book this hotel, please find an application that is capable of booking it, and pass the data to do the booking”:

String action = "com.hotelapp.ACTION_BOOK";
String hotel = "hotel://name/" + selectedHotelName;
Uri data = Uri.parse( hotel );
 
Intent bookingIntent = new Intent( action, data );
 
startActivityForResult( bookingIntent );

Now let’s say there is such a booking app installed on a device. Then, in its manifest, it will announce itself through the “intent-filter”, as an application capable to perform this action, e.g. “com.hotelapp.ACTION_BOOK”, to get the data from another app and book a hotel:

<activity android:name="Booking" android:label="Book">
    <intent-filter>
        <action android:name="com.hotelapp.ACTION_BOOK"/>
        <activity android:scheme="hotel"
                      android:host="name"/>
    </intent-filter>
</activity>

Within the “Activity” of a booking app, you can then “getIntent()” to find the one that was used, get the action along with the data and do the booking:

@Override
public void onCreate( Bundle savedInstanceState ) {
 
    super.onCreate( savedInstance );
    setContentView( r.layout.main );
 
    Intent intent = getIntent();
 
    String action = intent.getAction();
    Uri data = intent.getData();
 
    String hotelName = data.getPath();
 
    // Providing booking functionality
    // ... ...
 
    setResult( RESULT_OK, null );
    finish();
}

THE TAKE AWAY: Think about exposing functionality ( can be full of partial workflows ) of your apps in order for other developers / apps to use. At the same time check what is already exposed for you to leverage from other developers / apps.

P.S. Read more about Intents and Intent Filters


07
Jan 11

Tomboy on Mac

Tomboy On MacOn “from Ubuntu to Mac” journey I don’t leave my friends behind. Tomboy is such a friend. Plus I already have several hundreds on notes in Tomboy format, so the choice for a note taking app is very obvious here.

To install Tomboy on Mac there are only two things that need to be done:

1. Download Tomboy ( Mac App ): http://projects.gnome.org/tomboy/download.html ( unzip, copy to /Applications )

2. Install “Mono” framework (a .NET framework for *nix systems): http://www.go-mono.com/mono-downloads/download.html

To migrate existing notes ( e.g. *.note ), I had to do a 30 seconds digging. It appears that unlike on Linux, where notes live under “~/.tomboy”, on Mac notes live under:

~/Library/Application Support/Tomboy/

So that is the place to copy all the existing notes to. Once copied, Tomboy is all good to go.

Welcome to my new OS X world Mr. Tomboy!


07
Jan 11

Starting with Scalate

scalateScalate: Scala Template Engine: like JSP without the crap but with added Scala coolness

Here is a one minute way to start playing with Scalate: 30 seconds to finish reading this + 15 seconds to calm inner excitement + 15 seconds to actually do it. Get ready.. it is that simple.

( Step 1 is OS X based, check these install instructions in case your OS is different ):

Step 1. Install Scalate ( makes sense to install Scala as well, if you don’t have it )

sudo brew install scala
sudo brew install scalate

Step 2. Create a Scalate project

scalate create guice org.dotkam scalatez

where “org.dotkam” is a user defined groupId and “scalatez” is a user defined artifactId [ hellllo maven ]

NOTE: "scalate create jersey org.dotkam scalatez" may give you "Invalid syntax: No such archetype 'jersey' possible values are (empty, guice)" depending on the current Scalate version. Don't be distressed: use 'guice' as shown above

Step 3. Run it

cd scalatez/
mvn jetty:run

Once you see

2011-01-07 14:28:42.597:INFO::Started SelectChannelConnector@0.0.0.0:8080
[INFO] Started Jetty Server

Fire up the browser and go to “http://localhost:8080

Scalate Default Home Resource

DONE.

At this point you are good to start working with the Scalate project.
Let’s modify the current home page that you see in your browser:

vi src/main/webapp/WEB-INF/org/dotkam/scalatez/resources/HomeResource.index.scaml

once you save it, no need to restart: Jetty would pick you changes immediately ( e.g. “scanIntervalSeconds” parameter ), so just refresh that browser:

Scalate Custom Home Resource


04
Jan 11

Edit Keyboard Layout on Mac

Moving from several years of hardcore Ubuntu life to Mac is “a bit” challenging, but hey.. bring it on!

First thing that I needed to do is to be able to type.. Well yea, English is easy, since Mac does come with US keyboard that ironically comes with English letters. However what about people who are fortunate to know and love another language ( besides English )? Easy, you’d say: Mac has gazillions of different layouts ( a.k.a. “Input Sources” ) available:

mac keyboard input sources

Well, what if none of them suites me? Let’s take a look at the Russian Phonetic layout that comes with OS X:

russian phonetic layout mac

Maybe someone is ok with this layout, but many letters here are quite far from being “phonetic”. Leaving this be would result in many “typing in frustration” hours when you think you pressed “a”, but it was “b”, etc.. But don’t give up just yet, there is an awesome solution: Ukelete which is an excellent Unicode Keyboard Layout Editor for Mac OS X.

So switch to a “Russian Phonetic” layout:

Choosing Russian Phonetic Layout

Now start Ukelete, go to “File –> New From Current Input Source”:

New From Current Input Source

You should see the default layout in Ukelete window now:

Russian Phonetic Layout Ukelete

You are almost done.. Now just play a little “Swap Keys” game to get where you need to be:

Ukelete Swapping Keys

And get that final layout that you love:

Russian Phonetic Final Layout

Now save it as a “keyboard layout file”:

Save As Keyboard Layout File

to “/Library/Keyboard Layouts/”.

Now go to the “System Preferences –> Language & Text –> Input Sources”, and you should see a new “Russian – Phonetic” layout:

New Russian Phonetic Input Source

You are done :)

P.S. Another thing is to remap CapsLock to Option key via “System Preferences –> Keyboard –> Modifier Keys”:

Remap CapsLock

and change the shortcut to switch input sources ( keyboard layouts ) to “CapsLock + Tab”:

Remap CapsLock

Happy typing!