"; */ ?>


24
Oct 08

Getting Return Code from ANT in SHELL

This short tutorial shows how to get an exit code from Ant and act upon it. This can be useful when building a massive application that requires several Ant scripts/targets and a collection of shell scripts.

The rule of thumb when building the system is “when it should fail -> it should fail”. Sounds weird, but it is really true.

If the whole build of the application takes 15-20 minutes, and one of the libraries failed to build (for example), the whole build should fail. Otherwise, if the build master inspects the logs, and sees something went wrong (in case it is logged), s/he would have to spend another 20 minutes to rebuild it. Or what is even worse, the application could end up in an inconsistent state, without anybody raising a flag, and that would be very dangerous, especially if the current release goes beyond integration builds, for example to quality assurance or even worse – to production.

So, above are the reasons, below is an ultra simple example with a solution:

Let’s say we have an Ant script (build.xml) that checks for duplicate jars in a directory. The target name that does that is “check-duplicate-jars”:

build.xml:

    <target name="check-duplicate-jars" depends="some-other-task">
        <!-- does its magic here -->
    </target>

Let’s say we also have a simple shell script that calls “ant “. If this shell script is in the same directory as the “build.xml”, we can call it like this “./callant.sh check-duplicate-jars”, and it should check if there are any duplicate jars. Let’s run the shell script:

[user@host]$ ./callant.sh check-duplicate-jars
ANT: Running check-duplicate-jars task...
Buildfile: build.xml
 
check-duplicate-jars:
     [echo] Checking for duplicate jars......
     [exec] ***********    NO DUPLICATE JARS FOUND IN LIB!    ***********
 
BUILD SUCCESSFUL
Total time: 3 seconds
ANT: Return code is: "0"
GREAT SUCCESS: Niiice - I liiike!

This is an expected behavior. However what would be nice is to have this shell script “fail hard”, in case the Ant script (build.xml) fails.

Let’s try to misspell the target name “check-duplicate-jar” (should be plural: “check-duplicate-jars”) and run it:

[user@host]$ ./call-ant.sh check-duplicate-jar
ANT: Running check-duplicate-jar task...
Buildfile: build.xml
 
BUILD FAILED
Target `check-duplicate-jar' does not exist in this project.
 
Total time: 1 second
ANT: Return code is: "1"
BUILD ERROR: I am failing hard...

Wonderful – it failed! That is an expected behavior. And here is a sneak peek to the magic box – the “callant.sh” shell script:

#!/bin/bash
 
tname=$1
 
echo "ANT: Running $tname task..."
ant $tname
 
antReturnCode=$?
 
echo "ANT: Return code is: \""$antReturnCode"\""
 
if [ $antReturnCode -ne 0 ];then
 
    echo "BUILD ERROR: I am failing hard..."
    exit 1;
else
 
    echo "GREAT SUCCESS: Niiice - I liiike!"
    exit 0;
fi

Hence the whole magic is here:

antReturnCode=$?

In general, “$?” will return an exist code of any executable run within the shell script.
Well, now you ready for the massive build process. Get to work, and may the build force be with you! :)


22
Oct 08

Use HSQLDB Functions with Hibernate Mapping Files

Let’s say you need to do a “local integration” test, where you code does not depend on external systems (queues/external servers/DB/etc..). Creating unit tests, mocking/stubbing everything out is all good, but sometimes you need to be able to run tests that are as close as possible to the “real world” deal, while you are in “local mode” – e.g. plane, subway, basically somewhere without access to the real external systems.

One of ways you can approach it with databases is to load schemas you need in memory, and work (test) against those schemas. This is relatively easy to do with HSQLDB, and there are many “googlable” guides on how to do it. However in this little howto, I want to show you how you can define your own HSQLDB functions, and use them as either “stubs” (or even “real deal”).

Here is an example on why you may need it. Consider this Hibernate mapping file that is used in your application:

 
<?xml version="1.0"?>
 
<!DOCTYPE hibernate-mapping PUBLIC
    "-//Hibernate/Hibernate Mapping DTD 3.0//EN"
    "http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd">
 
<hibernate-mapping
	package="org.project.customer.dto.search">
	<class name="CustomerCreditCardSearchResultDto" table="CR_CREDIT_CARD">
 
		<id name="id" column="CR_CREDIT_CARD_ID">
			<generator class="native">
                                 <param name="sequence">SQ_CR_CREDIT_CARD</param>
			</generator>
		</id>
 
                <property name="accountNumber" formula="some_pkg.decode( ACCOUNT_NUMBER )" />
                <property name="secretNumber" formula="some_pkg.decode( SECRET_NUMBER )" />
                <property name="expirationDate" column="EXPIRATION_DATE" type="date" />
                <property name="zipCode" column="ZIP_CODE" />
 
	</class>
</hibernate-mapping>

Let’s say you created a CR_CREDIT_CARD table in in-memory DB, populated it, started your test. Now when you try to read a “CustomerCreditCardSearchResultDto” object somewhere in your test, Hibernate will construct an SQL query from the mapping file above, and execute it on the in-memory DB.

However there is a problem – it is going to fail with a similar Exception:

Caused by: java.sql.SQLException: Unexpected token: DECODE in statement [select <query here> .... ]
	at org.hsqldb.jdbc.Util.throwError(Unknown Source)
	at org.hsqldb.jdbc.jdbcPreparedStatement.<init>(Unknown Source)
	at org.hsqldb.jdbc.jdbcConnection.prepareStatement(Unknown Source)
	at org.apache.commons.dbcp.DelegatingConnection.prepareStatement(DelegatingConnection.java:248)
	at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.prepareStatement(PoolingDataSource.java:302)
	at org.hibernate.jdbc.AbstractBatcher.getPreparedStatement(AbstractBatcher.java:442)
	at org.hibernate.jdbc.AbstractBatcher.getPreparedStatement(AbstractBatcher.java:368)
	at org.hibernate.jdbc.AbstractBatcher.prepareQueryStatement(AbstractBatcher.java:105)
	at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1561)
	at org.hibernate.loader.Loader.doQuery(Loader.java:661)
	at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:224)
	at org.hibernate.loader.Loader.loadEntity(Loader.java:1785)
	... 60 more

If there is an Exception – there is a Reason for it :) Notice these two mappings in the Hibernate mapping file above:

<property name="accountNumber" formula="some_pkg.decode( ACCOUNT_NUMBER )" />
<property name="secretNumber" formula="some_pkg.decode( SECRET_NUMBER )" />

That says Hibernate to construct SQL that uses this “some_pkg.decode()” custom function/store procedure after (in case of SELECT) reading these values from a database. However, in you local testing environment, you do not have this function defined. It is probably defined in the DB (like Oracle/DB2/etc) itself.

But for most, if not for all, technical problems there is a solution, so don’t worry – you can define this function(s) yourself. Since it is a test, the most logical thing, since you want to abstract the code out from the external systems (and their functions) would be to define stubs to these functions.

Let’s create a utility “HsqlFunctions” Java class with static methods to be used by HSQLDB (it can only use static Java methods in its SQL, btw):

package org.project.test.util;
 
/**
 * HSQL functions to be aliased.
 *
 *    DDL example:
 *
 *    CREATE ALIAS DECODE FOR "org.project.test.util.HsqlFunctions.decodeString"
 *
 *
 * @author blog.dotkam.com
 *
 */
 
public final class HsqlFunctions
{
   private HsqlFunctions()
   {
     // static utility class - does not need to be constructed.
   }
 
    /**
     * Stub for the decode function.
     * Usually used to "please" Hibernate Mapping Files.
     *
     * @param value - String value for the column
     * @return - returns the same String value
     */
    public static String decodeString( String value )
    {
        return value;
    }
}

Now, when creating a test schema, you can ALIAS this static method as HSQLDB function like this:

     CREATE ALIAS DECODE FOR "org.project.test.util.HsqlFunctions.decodeString"

This will tell HSQLDB to call “decodeString” static Java method on the column value, every time it sees “decode( COLUMN )” in SQL.

One thing to notice, though – make sure the type that the Java methods take are exactly the same as defined by DDL (Database Schema). For example, if you pass in column value as an Object:

    public static String decodeString( Object value )
    {
        return String.valueOf (value );
    }

And the column is defined as VARCHAR in schema, HSQLDB will try to pack that String into an Object, and it will fail with a similar exception:

Caused by: java.sql.SQLException: Wrong data type: hexadecimal string with odd number of characters in statement [select ... <query here>]
	at org.hsqldb.jdbc.Util.throwError(Unknown Source)
	at org.hsqldb.jdbc.jdbcPreparedStatement.executeQuery(Unknown Source)
	at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:93)
	at org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:139)
	at org.hibernate.loader.Loader.getResultSet(Loader.java:1669)
	at org.hibernate.loader.Loader.doQuery(Loader.java:662)
	at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:224)
	at org.hibernate.loader.Loader.loadEntity(Loader.java:1785)
	... 60 more

Below is just an excerpt on how you would set up the schema to use it with HSQLDB in your test:

    ....
 
    private static final String CREATE_ALIASES =
        "CREATE ALIAS DECODE " +
        "FOR \"org.project.test.util.HsqlFunctions.decodeString\"";
 
    private static final String CREATE_SEQUENCES =
        "DROP SEQUENCE SQ_CR_CREDIT_CARD IF EXISTS;" +
 
        "CREATE SEQUENCE SQ_CR_CREDIT_CARD " +
        "START WITH 1 INCREMENT BY 1";
 
    private static final String CREATE_TABLES =
        "DROP TABLE cr_credit_card IF EXISTS;" +
 
        "CREATE TABLE cr_credit_card (" +
            "cr_credit_id NUMERIC(15), " +
            "account_number VARCHAR(48)," +
             ....
             ....
   ....
 
        //  Creating the schema
 
        m_jdbcTemplate.execute( CREATE_ALIASES );
        m_jdbcTemplate.execute( CREATE_SEQUENCES );
        m_jdbcTemplate.execute( CREATE_TABLES );
 
        //  HSQLDB is ready to be populated with data at this point.
   ....

You can use DBUnit to create the schema above and populate it with the data. The above is just a straight forward hardcoded example.
Notice how it also creates a sequence “SQ_CR_CREDIT_CARD” to please the Hibernate, and others who might use it in the application.

Happy “local integration” testing!

Feel free to post questions/comments/suggestions, I’ll try to respond when have a free second or two :)


02
Oct 08

Configure Multiple SSIDs with One Router

One Router Multiple SSIDsA standard “home setup” for the wireless router now days consists of just hosting one Service Set IDentifier – or SSID. And if only a year ago that could seem as a router (firmware) limitation, right now using multiple SSIDs with a single wireless router is just a matter of re-configuration.

Below is a simple guide on how to configure a single affordable (from $25 and up) router, that is available to anybody, to host multiple (up to 16 – at the moment of writing) SSIDs using DD-WRT firmware. In order to install the firmware on the router, follow the official DD-WRT Installation Guide.

Below step-by-step howto is good for any routers that could be found on DD-WRT supported hardware list. Which includes pretty much any “home” router that is out there. Given that DD-WRT is installed, let’s move on to configuring it to host multiple SSIDs.

First let’s look at what we would like to archive as our “end goal”:

Multiple SSIDs by several wireless VLANs served by one router

So what we would like to have is:

  • One router with DD-WRT firmware installed, that hosts 2 (in this example) SSIDs: “Home” and “Guest”

  • Two VLANs where one network (VLAN1) can “see” another (VLAN2), but not other way around.
    What that means is that all “Home” clients (computers that are connected to “Home” SSID) can see (ping/connect to/etc.) “Guest” computers, however none of “Guest” clients can see “Home” computers. (this is a matter of configuration, and can be configured differently, depending on what you need)

Step 1. Setup DHCP server for the “Home” (main) network.

As you can see from the “end goal” diagram above, the “Home” SSID (or VLAN1) has a 10.2.1.0 subnet, and its DHCP client addresses start from 10.2.1.100. Hence that is what needs to be configured on DD-WRT’s “Setup -> Basic Setup” screen (the “Router Local IP” should be set to 10.2.1.1 which would dictate the subnet to be 10.2.1.0):

DD-WRT - DHCP settings

Note that “10.2.1.0” is just an example – you would want to use something that is appropriate for your network – e.g. “192.168.1.0”

Step 2. Setup wireless networks (SSIDs).

Now we need to create two wireless networks – one main network (e.g. “Home”), and one virtual network (e.g. “Guest”). For that go to DD-WRT “Wireless -> Basic Settings” screen:

configure wireless networks with dd-wrt

Enter a desired name for “Physical Interface’s -> Wireless Network Name(SSID)” (this is going to be the main network). You can also stick to the “end goal” diagram above, and enter “Home”.

After that is done, click “Add” to add a “Virtual Interface” and enter its SSID name as well (e.g. you can enter “Guest”). Make sure that the “Network Configuration” is set to “Bridged” as shown on the screen in this step.

Step 3. Configure wireless network security.

In the previous step we configured two wireless networks, now let’s secure them. We will use 128 bit WEP algorithm for both of them due to the reason described in “Step 5”. To accomplish this go to DD-WRT “Wireless -> Wireless Security” screen:

configure wireless security with dd-wrt

Choose “WEP” for “Security Mode”, 128 bits for “Encryption”, enter “Passphrase” and click “Generate” button.

Do it for both networks (Physical and Virtual Interfaces)

Step 4. Setup a virtual interface, and its DHCP settings.

Now we will set the bridge for the virtual network – “Guest” (or VLAN2) from the “end goal” diagram above. For that go to DD-WRT “Services -> Services” screen:

Configuring DNSMasq and DHCP for virtual network

Find “DNSMasq” section, enable “DNSMasq”, and in “Additional DNSMasq Options” enter:

interface=br1
dhcp-range=br1,192.168.2.100,192.168.2.149,255.255.255.0,1440m

This would create a DHCP server for the virtual (“Guest”, VLAN2) network.
“192.168.2.100” is again – just an example, you can use any subnet that suits your needs.

Step 5. Setup firewall rules and a startup script.

This is the most complex step, that makes many network administrators confused, and regular people to give up on DD-WRT multiple SSID configuration. But don’t worry :) – below is a “copy/paste”, working deal.

Go to the DD-WRT “Administration -> Commands” screen:

Setting up firewall rules and a startup script for multiple SSIDs - DD-WRT

Enter the following firewall rules to the “Firewall” section:

##BRI1
iptables -I INPUT -i br1 -m state --state NEW -j logaccept
iptables -I FORWARD -i br1 -o vlan1 -m state --state NEW -j ACCEPT
 
#below keeps the two networks from talking
iptables -I FORWARD -i br0 -o br1 -j logdrop

Enter the following commands to the “Startup” (it is a startup script that executes when the router starts up):

##MOVES VIRTUAL WIRELESS TO OWN BRIDGE
brctl addbr br1
brctl delif br0 wl0.1
brctl addif br1 wl0.1
ifconfig br1 192.168.2.1 netmask 255.255.255.0
ifconfig vlan2 up
brctl addif br1 vlan2
ifconfig br1 up
 
##FIX NAS. Here NAS is disabled, cause it is NOT used for WEP, and these wifi networks will use WEP (for now)
killall nas
nas -P /tmp/nas.wl0lan.pid -H 34954 -l br0 -i eth1
nas -P /tmp/nas.wl0.1lan.pid -H 34954 -l br1 -i wl0.1

Here is where it gets interesting… Remember in “Step 3”, when configuring wireless security, we chose WEP? That was done because the current DD-WRT firmware “v24-sp1 (07/27/08) micro”, that is used at the moment of writing, has a bug in starting NAS, which is a proprietary binary tool that sets up dynamic encryption (WEP/WPA) on wireless devices.

UPDATE (12/22/2008):
           Tried "v24-sp2" (09/26/08 std - build 10431M VINT Eko) for WRTG54GL v1.1 router -
           WPA worked with multiple (tried 2) SSIDs.

In a startup script above, we start NAS in “vanilla” mode for “eth1” (the main network) and for “wl0.1” (guest, virtual nework), and therefore we are using WEP for both networks.

The only line from above startup script that you might want to change is:

ifconfig br1 192.168.2.1 netmask 255.255.255.0

Here “192.168.2.1” is, again, an example, so if you chose a different subnet for the virtual network (br1), you should enter it instead.

DONE!
Now you can save all the changes and restart the router. You should be good to go!
If you have any questions or comments, you are welcome to address them below in the “comments” section.


30
Sep 08

CNN: Bankruptcy, Not Bailout, is The Right Answer

Bankruptcy, not bailout, is the right answerI usually keep my blog technical and life oriented, however today, after the “$700 billion bailout deal” did not pass, and White House is cooking up a new deal, I think it is ultra important to convey the message of top US Economists from many US Universities that “This bailout is NOT the answer to the crisis!”.

Here is the official letter that was sent to Congress on Wed Sept 24 2008 regarding the Treasury plan as outlined on that date:

To the Speaker of the House of Representatives and the President pro tempore of the Senate:

“As economists, we want to express to Congress our great concern for the plan proposed by Treasury Secretary Paulson to deal with the financial crisis. We are well aware of the difficulty of the current financial situation and we agree with the need for bold action to ensure that the financial system continues to function. We see three fatal pitfalls in the currently proposed plan:

1) Its fairness. The plan is a subsidy to investors at taxpayers’ expense. Investors who took risks to earn profits must also bear the losses. Not every business failure carries systemic risk. The government can ensure a well-functioning financial industry, able to make new loans to creditworthy borrowers, without bailing out particular investors and institutions whose choices proved unwise.

2) Its ambiguity. Neither the mission of the new agency nor its oversight are clear. If taxpayers are to buy illiquid and opaque assets from troubled sellers, the terms, occasions, and methods of such purchases must be crystal clear ahead of time and carefully monitored afterwards.

3) Its long-term effects. If the plan is enacted, its effects will be with us for a generation. For all their recent troubles, America’s dynamic and innovative private capital markets have brought the nation unparalleled prosperity. Fundamentally weakening those markets in order to calm short-run disruptions is desperately short-sighted.

For these reasons we ask Congress not to rush, to hold appropriate hearings, and to carefully consider the right course of action, and to wisely determine the future of the financial industry and the U.S. economy for years to come. ”

Signed (updated at 9/27/2008 6:00PM CT) by 231(!) Top US Economists – go here to see all the signatures.


Although my personal opinion is that US Government instead of just give could possibly lend money to “unfortunate” corporation with a very high interest rate, CNN published a great article by Jeffrey A. Miron, a senior lecturer in economics at Harvard University, that suggests that US should not use US taxpayers money to bailout these “unfortunate” corporations at all.

some of the quotes from the article:

“Bankruptcy does not mean the company disappears; it is just owned by someone new (as has occurred with several airlines). Bankruptcy punishes those who took excessive risks while preserving those aspects of a businesses that remain profitable.

In contrast, a bailout transfers enormous wealth from taxpayers to those who knowingly engaged in risky subprime lending. Thus, the bailout encourages companies to take large, imprudent risks and count on getting bailed out by government. This “moral hazard” generates enormous distortions in an economy’s allocation of its financial resources.”

“Further, the current credit freeze is likely due to Wall Street’s hope of a bailout; bankers will not sell their lousy assets for 20 cents on the dollar if the government might pay 30, 50, or 80 cents.”

“The costs of the bailout, moreover, are almost certainly being understated. The administration’s claim is that many mortgage assets are merely illiquid, not truly worthless, implying taxpayers will recoup much of their $700 billion.”

“The right view of the financial mess is that an enormous fraction of subprime lending should never have occurred in the first place. Someone has to pay for that. That someone should not be, and does not need to be, the U.S. taxpayer.”


29
Sep 08

Sure Way to Restart a Wireless Network

linux penguin is watching windows fly

While wireless signal is good, the network is dead – why is that? Many reasons, of course. You can spend time to figure out the reason and then try to fix, which is a good approach, but requires some time. Or you can restart the network to see if it resolves the issue, and if it does – forget that the problem ever existed.

However the way to restart a wireless network is not always “black and white”. Sometimes it is possible using GUI, and sometimes by typing something that means “network service restart”. And yes these ways are “clean”, but have a drawback – they rely on operating system to do what it suppose to do and restart the network.

However, OS does not always behave (yes, Linux does not always behave, along with Mac, and Windows, and “any” OS.. ). But here is a sure way to restart it – you would need to get down to the driver level though – to be less OS (or distribution) specific. But I’ll guide you through, don’t worry..

So, the signal is full/good/strong:

wireless signal is good
but there is no network:

$ ping -c 4 google.com
ping: unknown host google.com

First thing to do is to see what wireless card you are using:

$ lspci | grep -i network
08:00.0 Network controller: Intel Corporation PRO/Wireless 3945ABG Network Connection (rev 02)

In my case it is Intel 3945ABG. Next, check what driver is used for this card. I did a simple google search, and saw that the driver is “ipw”something.

Let’s see what ipw-like modules/drivers are currently running/loaded:

$ modprobe -l | grep ipw
/lib/modules/2.6.20-17-generic/kernel/ubuntu/wireless/ipw3945/ipw3945.ko
/lib/modules/2.6.20-17-generic/kernel/drivers/usb/serial/ipw.ko
/lib/modules/2.6.20-17-generic/kernel/drivers/net/wireless/ipw2200.ko
/lib/modules/2.6.20-17-generic/kernel/drivers/net/wireless/ipw2100.ko

Here it is “ipw3945”. Let’s kill it (-r stands for “remove”):

$ sudo modprobe -r ipw3945

Let’s start it back up:

$ sudo modprobe ipw3945

Checking connectivity:

$ ping -c 4 google.com
 
PING google.com (64.233.187.99) 56(84) bytes of data.
64 bytes from jc-in-f99.google.com (64.233.187.99): icmp_seq=1 ttl=238 time=43.3 ms
64 bytes from jc-in-f99.google.com (64.233.187.99): icmp_seq=2 ttl=238 time=28.9 ms
64 bytes from jc-in-f99.google.com (64.233.187.99): icmp_seq=3 ttl=238 time=27.7 ms
64 bytes from jc-in-f99.google.com (64.233.187.99): icmp_seq=4 ttl=238 time=34.7 ms
 
--- google.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 27.742/33.685/43.323/6.165 ms

Perfect!