Thread.sleep: mark of the noob

…the post title is a bit harsh – but true in most cases.

Too often Thread.sleep is used to make the main application thread pause when it needs to wait for resources to be initialized on a secondary thread, like so:

while (resourceNotInitialized){
Thread.sleep (someArbitraryNumberOfMilliSeconds);
}

While this solution has the advantage of being easy to understand, it also has one drawback: it’s wrong.

Or more precisely the number of milliseconds the main thread must sleep for is most likely wrong:
- either it’s too small and the main thread will repeatedly awake too early, hogging CPU resources in the process (buzy wait)
- or it’s too large and the main thread will wake up long after all resources have been initialized, resulting in an application with sluggish behaviour (and irate users).

The proper way to handle this scenario is to use the wait and notify methods from the Object class.

- Decide on an instance (or class) on which both the main thread and secondary thread can synchronize
- Main thread starts the secondary thread and acquires the monitor (enters synchronized block)
- Main thread then waits, effectively releasing the monitor
- When the secondary thread is done it acquires in turn the monitor and notifies the main thread.
- Main thread wakes up and resumes it course

A bit more convoluted than Thread.sleep, granted, but much nicer and more importantly, bug-proof and latency-resistant

Top 10 Java interview questions

The 10 most frequently-asked Java interview questions, in my experience.
Mostly used in phone interviews to “weed out” the weakest candidates.

These questions are fairly simple so getting them wrong will raise a major red flag.
For the same reason I wont bother printing the answers, unless somebody specifically asks for it in the comments.

In no particular order:

- How to prevent concurrent access to a method

- Meaning of the volatile keyword

- Difference String and StringBuffer

- Difference between ArrayList and Vector

- Relationship between equals and hashcode

- Difference between checked and unchecked exceptions

- Meaning of the final keyword

- Explain garbage collection, can it be forced

- Difference between an interface and a abstract class

- what’s a deadlock

Log4j code snippets

A few log4j-related code snippets that I tend to re-use from time to time… All pretty self-explanatory.

There is a faq at http://logging.apache.org/log4j/ which already touches on the subjects below,
but in a fairly light manner, and without the support of any code.

1- how to change the log level dynamically

        Level debugLogLevel = Level.toLevel("DEBUG");
        Logger.getLogger(MyClass.class).setLevel(debugLogLevel);

2- add an appender at runtime

ConsoleAppender appender = new ConsoleAppender( new PatternLayout("%-5p [%t]: %m%n"));
Logger.getRootLogger().addAppender(appender);

3- how to reload the log4j configuration file at runtime

String configDir ="/path/to/config/directory";
LogManager.resetConfiguration();
String log4jConfigFile = configDir + java.io.File.separator +"log4j.xml";
DOMConfigurator.configure(log4jConfigFile);
logger.info("log4j initialized from " + log4jConfigFile);

4- how to direct the log output to different files.

In the configuration below the log statements originating from the “com.firstpackage” package
will be directed to FirstFile.log, and the also to the console.
Log statements from “com.secondpackage” will go to SecondFile.log, and to the console.

 <appender name="FIRST_FILE" class="org.apache.log4j.DailyRollingFileAppender">
       <param name="File" value="FirstFile.log"/>
       <layout class="org.apache.log4j.PatternLayout">
       <param name="ConversionPattern" value="%d{HH:mm:ss,SSS} [%t] %-5p %c{1}: %m%n"/>
       </layout>
    </appender>

 <appender name="SECOND_FILE" class="org.apache.log4j.DailyRollingFileAppender">
       <param name="File" value="SecondFile.log"/>
       <layout class="org.apache.log4j.PatternLayout">
       <param name="ConversionPattern" value="%d{HH:mm:ss,SSS} [%t] %-5p %c{1}: %m%n"/>
       </layout>
    </appender>

 <appender name="CONSOLE_APPENDER" class="org.apache.log4j.ConsoleAppender">
        <layout class="org.apache.log4j.PatternLayout">
            <param name="ConversionPattern" value="%d{HH:mm:ss,SSS} [%t] %-5p %c{1}: %m%n"/>
        </layout>
    </appender>

<appender name="ASYNC_APPENDER_1" class="org.apache.log4j.AsyncAppender">
        <appender-ref ref="CONSOLE_APPENDER"/>
        <appender-ref ref="FIRST_FILE"/>
    </appender>
   
    <appender name="ASYNC_APPENDER_2" class="org.apache.log4j.AsyncAppender">
        <appender-ref ref="CONSOLE_APPENDER"/>
        <appender-ref ref="SECOND_FILE"/>
    </appender>

 <logger name="com.firstpackage" additivity="false">
        <level value="DEBUG" />
        <appender-ref ref="ASYNC_APPENDER_1" />       
    </logger>
   
    <logger name="com.secondpackage" additivity="false">
        <level value="DEBUG" />
        <appender-ref ref="ASYNC_APPENDER_2" />       
    </logger>

First steps with Awk

Awk is a Unix programming language specifically dedicated to the processing of text files.

While it’s been around for ages (it’s almost as old as Unix) it remains fairly unknown and/or unused compared to other utilities such as grep, vi, find… Strange really as it’s powerful and quite easy to use.

Example case: looking up and summarizing data in a log file.

Imagine the “usual” application log file of the form: [Date] [Thread #] [Log level] [log message]

Say some of the lines logged in there account for the time spent on one given algorithm:
13:42:07,019 [Thread-1] DEBUG Calculation: time spent on algo #12 is 831 ms

It would be interesting to parse all the lines containing the word algo to extract 1) the total time spent on algo calculation 2) the average time spent across all calculations.

In Java,  coding such a parser from scratch would take about one full day for most developers (if not more)
- locate file, open and read , manage io exceptions
- parse lines (manage parsing exceptions)
- calculate results and print
- create a build script

By comparison the equivalent script can be written with Awk in minutes.

proceeding step by step

Step 1.
to print to screen all lines containing the term algo in the file myfile.log

awk ‘/algo/ {;print $0}’ myfile.log

Note: Awk divides each line in columns, where a column is a block of text separated by whitespaces.
Eg. in the format above [Date] [Thread] [Debug]… $0 will print the whole line, $1 will print the date, $2 the thread number ..etc…

Step2.
To sum the number of lines with the term algo:

awk ‘/algo/ {nb++} END {printf “%d”,nb}’ myfile.log

[here nb is a local variable declared on the fly and used as a line counter]

Step3.
To keep a running total of the time spent on algo calculations:

awk ‘/algo/ {total+=$9} END {printf “%d”,total}’ myfile.log

[assuming the time spent on a calculation is printed on the 9th column of the line]

Step4.
putting it all together – print the total time spent + average on each calculation:

awk ‘/algo/ {nb++;total+= $9} END {printf “%d %d”, total, total/nb}’ myfile.log

The best way to learn more about Awk is to try it….

- Awk should come as a standard with all linux distributions.

- On Windows it’s available via Cygwin. An alternative implementation, gawk, (for Gnu Awk) can also be found here.

How to copy a file in Java

Even the most basic of task – like copying a file – can be done in several different ways in Java…
a testament to the richness of the platform (or its complexity !).

Four possible ways to copy a file below, all error handling left aside.

1- the complicated way, manipulating io streams.

   import java.io.FileInputStream;
   import java.io.FileOutputStream;
   import java.io.InputStream;
   import java.io.OutputStream;
   ....

   String orig ="file.xml";
   String dest = "file.xml.bak";
   InputStream in = new FileInputStream(orig);
   OutputStream out = new FileOutputStream(dest);
   byte[] buf = new byte[1024];
   int len;
   while ((len = in.read(buf)) > 0) {
      out.write(buf, 0, len);
   }
   in.close();
   out.close(); 

2- The ugly , non-portable way, using the operating system environment:

   Runtime.getRuntime().exec("OS dependent file copy command here"); 

3- The opensource way, using the Apache Commons library.

   import java.file.io;
   import org.apache.commons.io.FileUtils;
   ....
   String orig ="file.xml";
   String dest = "file.xml.bak";
   File fOrig = new File(orig);
   File fDest = new File(dest);
   FileUtils.copyFile(fOrig, fDest);

4- The novel way- using Java 7 and it’s revamped I/O api.

   
    import java.io.File;
    import java.nio.file.Path;
    ...
    String orig ="file.xml";
    String dest = "file.xml.bak";
    File f = new File (orig);
    Path p = f.toPath();
    p.copyTo(new File (dest).toPath(), REPLACE_EXISTING, COPY_ATTRIBUTES);

Dependency graph in Netbeans 6.8

It seems that I discover something new in Netbeans every day… The “Show dependency graph” associated with every Maven projects in Netbeans 6.8 had escaped me so far.

This menu option (accessible by right-clicking on a Maven project in the “projects” window) generates a graph of all the projects’ dependencies  (libraries declared in the pom and their transitive dependencies).

Example below of the dependency graph  of the libraries required by a project called, rather imaginatively, mavenproject1.

The graph will be rather hard to read for a large number of dependencies, but it’s nice to have nevertheless.

Debugging classpath issues

There’s nothing more frustrating than wasting time figuring out why some resources (e.g. configuration files for log4j, hibernate…) are not loaded correctly from the classpath.

The few lines of code below help narrow down these kind of issues. Knowing what’s the classpath at runtime and being able to test if it covers a specific file is half the battle won already.

1- Print out name of all files on the classpath

String classpath = java.lang.System.getProperty( "java.class.path" );
for (String path : classpath.split(System.getProperty("path.separator"))){
   File f = new File (path);
   String resource = (f.isDirectory()?Arrays.asList( f.list()).toString():f.toString());
   System.out.println (resource);
}

2- Check wether a specific file is on the classpath

String myResource = .... ;
InputStream is = getClass().getResourceAsStream(myResource);
System.out.println (myResource + " is " +  (is==null?"not":"") + " on the classpath");

Migrating Eclipse plugins

Developers can sometimes hold back from upgrading their Eclipse installation due to the perceived difficulty of migrating already installed plugins to the new setup.

This process is actually fairly easy, as described below:
[CAVEAT - being able to migrate a plugin from one Eclipse version to another doesnt necessarily mean this plugin will still work with the newer version]

1- Download the latest and greatest version of Eclipse, unpack and install.

2. Create a file called “.eclipseextension” at the root of the old Eclipse installation.
On Windows the file explorer might not let you create a file with this name – the preferred way is then to open up a command prompt,
navigate to the root of the previous install and type “copy con .eclipseexension” to create the file, followed by “CTRL-F6″ to save.

3. Edit this file to the content below:
id=org.eclipse.platform
name=Eclipse Platform
version=3.5.0 — version of the newest Eclipse installation

4. Open the new version of Eclipse, Select Help menu -> Install new software.

5. Click the “Add” button on the right of the “work with” box, and pick the location of
your previous Eclipse install (i.e directory where the .eclipseextension file is located)

6. Unselect “group items by category” on the bottom panel
to see the plugins installed with the previous Eclipse.

7. Select which plugins to re-install.

8. Restart Eclipse.

9. Check that the plugins work correctly(!) in their respective contextual menus, views and perspectives.
If something goes wrong: select Help menu -> Install new software -> work with all available sites,
pick “what’s already installed” on the bottom panel and uninstall the faulty plugins.

(almost) painless profiling with TPTP 4.6.1 and Eclipse

The TPTP plugin for Eclipse in its versions 4.3, 4.4 and 4.5 was fairly bug prone – to the point of being barely usable. In particular it had a frustrating tendency to lock up the entire IDE…The latest version v.4.6.1  is more stable, even though setting it up still requires quite a lot of work… I only put up with it (just) because I prefer using Eclipse over Netbeans as my main development tool.

Summary of the steps involved below (tested with Eclipse 3.5, TPTP 4.6.1, running on Windows XP).

1.) Download TPTP  4.6.1 from within the comfort of your Eclipse IDE.
Replace 4.6.0 in the page linked above by 4.6.1… the documentation is not quite up to date :/
Once the install is over and Eclipse is restarted a new “Profiling and logging” perspective should be available in the main menu.

2.) Download the agent controller separately
In theory this step is not really needed, as TPTP now contains its own  Integrated Agent Controller (IAC).
In practice the standalone agent is more stable than the IAC.

3.) Unzip the agent controller on your local drive.

4.) Add the profiler DLLs to your path. From the command line:

Set TPTP_AC_HOME=<path to your local agent controller installation>
set JAVA_PROFILER_HOME=%TPTP_AC_HOME%\plugins\org.eclipse.tptp.javaprofiler
Set PATH=%JAVA_PROFILER_HOME%;%PATH%;%TPTP_AC_HOME%\bin

[more details on setting up the path here , check out section 3.3]

5.) Create a file called filters.txt (for example) where you’ll specify the classes which needs to be profiled.

Content of the file=
com.myclasses* * INCLUDE
* * EXCLUDE

This will profile all the methods of all classes in the com.myclasses packages.
Note that filtering is essential if you dont want to end up with hideously large profiling files.

6.) Run the application to be profiled
Add the following to the Java command line used to run the app so that all execution details are collected:
-agentlib:JPIBootLoader=JPIAgent:server=standalone,filters=filters.txt;CGProf:execdetails=true;

If all goes well a file name called trace.trcxml (by default) will start collecting the profiling info from the current directory.

7.) Open the profiling view in Eclipse and import the trace file generated
(a popup menu will appear where you can select additional filters and specific statistics to be run on the trace file)
Be prepared to wait if you didnt specify a broad enough set of filters in step 5)

8.) Once import is finished right click on the profiling file and open it with the appropriate editor
eg. use ExecutionStatistics and ExecutionFlow if profiling run with execdetails=true

Caveat: Profiling done that way doesnt give a realtime feedback on the behaviour of the app being profiled
(the trace.trcxml file generated needs to be fed into Eclipse repeatedly for up to date results).

On the plus side this method works for all kind of processes, remote or local, libraries or main programs.

If all else fails there’s always the Netbeans’s profiler, which is  very good, and free, or Yourkit (www.yourkit.com),which is excellent(but not free…)  Another alternative is VisualVm , which offers both profiling and sampling capabilities.

All these tools work pretty much out of the box, in stark contract with TPTP.

Google Collections’ computing map

“On demand computing”, as provided by Google Collections, turns a map into a computing map, which can be used as a basic cache for rare (but expensive) lookups.

Although the javadoc for this functionality is a good start, it can be opaque at times, especially for developers not entirely accustomed with the functional style of programming prevalent in the library.

The code below puts a computing map into context.

1. Computing map declaration.

//the 'usual' ConcurrentMap... 
ConcurrentMap<Key, Value> computingMap =   
// ... enhanced to support soft/weak keys/values, timed expiration and...
        .new MapMaker()  
//...on-demand computation...
        .makeComputingMap(   
//...passing into parameter an anonymous instance implementing the Function interface... 
        new Function<Key, Value>() { 
//...where the function transforming a key into a value is defined so that....   
        public Value apply(Key key) {  
//...it delegates to the a specialized, computationaly expensive function.
         return createExpensiveValue(key);  
       } 

2. Computationaly expensive function used to generate a value from a key.

  Value createExpensiveValue(Key key){ 
               Value computedValue = null; 
                /*transform the input key into the computedValue here 
                ... 
                */ 
             return computedValue; 
     } 

3. Retrieving values from the map

void retrieveValuesFromComputingMap() { 
        Key k = new Key(...); 
        //the map generates a value from the key and stores it.
        Value v =computingMap.get(k);   
        //subsequent calls to retrieve the value associated with the key will fetch it directly from the map, 
       // skipping any other computation
}