A lot of new ideas finally have been materialized in spf4j.

Along with measurements (tsdb), stack sample data can not be serialized to a file (dumped to file at a interval (ssdump)).

I have developed a simple UI to visualize the data.

latest release can be downloaded at http://code.google.com/p/spf4j/

Swallowing exception is a common mistake done by junior developers (some wrongfully think they are senior). 

Please do not write or use code like this (apache commons):

  132        * Unconditionally close an <code>Reader</code>.
  133        * <p>
  134        * Equivalent to {@link Reader#close()}, except any exceptions will be ignored.
  135        * This is typically used in finally blocks.
  136        *
  137        * @param input  the Reader to close, may be null or already closed
  138        */
  139       public static void closeQuietly(Reader input) {
  140           try {
  141               if (input != null) {
  142                   input.close();
  143               }
  144           } catch (IOException ioe) {
  145               // ignore
  146           }
  147       }

In Guava (my favorite open source library) the authors did the same mistake as the apache folk above. But I am happy they realized the problem recently and deprecated the bad code:

closeQuietly(Closeable closeable)

Deprecated. 

This method has few valid use cases and encourages misuse by making it easy to do the wrong thing. Among other things, it may swallow exceptions that really should be thrown, such as exceptions thrown when closing an output stream: this often involves flushing buffered data to the final output destination and as such it is just as important to throw the exception thrown when closing as it is to throw an exception thrown by a call to a write method. This method is scheduled to be removed in Guava 16.0.

One thing I would like to add to this theme, In guava they recommend the following pattern for not loosing the original exception:

public
void useStreamNicely() throws IOException {
SomeStream
stream = new SomeStream("foo");
boolean
threw = true;
try
{
//
... code which does something with the stream ...
threw
= false;
}
finally {
//
If an exception occurs, rethrow it only if threw==false:
Closeables.close(stream,
threw);
}
}

Which is not a bad recommendation, but I prefer using the exception chaining technique in which case both exceptions info will be thrown and no need for logging at this level needs to be done:

public void useStreamNicely() throws IOException {
SomeStream stream = new SomeStream("foo");
try {
// ... code which does something with the stream ...
} catch (Exception e)
  try { 
   stream.close()
  } catch (IOException ex) {
   throw Exceptions.chain(ex, e); 
  }
  if (e instanceof IOException)
   throw e;
  else 
   throw new RuntimeException(e);
}
stream.close(); 
} 

As you can see I am not handling Error. This is on purpose since when a Error happens your process is hosed and you should terminate it and not try to do things that will most likely fail... Exceptions.chain is available in spf4j on code.google.com. (http://code.google.com/p/spf4j/)

The first printer I owned was a Epson STAR LC-20, was a decent printer at the time, since then I have started using HP printers, and for a while I was happy. The last printer I owned was a HP Office Jet 6480, it was a disappointing experience. Unreliable Driver, hinges broke, and finally died... When researching for a new printer online to my surprise the HP printers had quite bad reviews, while the Epson printers had excellent reviews...

So I decided to go with a Epson WF 3520, and I am very happy with my decision!

I have finally upgraded my old TV (Sony Vega CRT). I stayed loyal to the Sony brand and don't regret it. (yet)

I wasn't able to get to my media from my TV via my existing interfaces (smb,nfs,http) (the embedded Sony web browser  is a bit of a disappointment).

So I decided to add yet another interface to my server: dlna.

After looking around I decided to try a java solution: http://www.serviio.org/, after a bit of fiddling around with it I was able to make it work (had to provide the ffmpeg path via a system var).

So far everything works perfectly, it seems to be able to trans-code all my media into the right format, and it can even stream online content. (european news sources in my case)

How many of you had bugs in your code where you try to dereference a null pointer?  (If you never had a bug like this you probably did not write enough code :-)

The typical good mannered reaction to fixing a bug, is write a unit test that reproduces the bug + fix the code. 

In this case if you use java, I disagree with this approach, and I would recommend the following: annotate your variable/parameter that causes your NPE with javax.annotation.Nullable or javax.annotation.Nonnull based on the behaviour you want, and if you enforce find bugs in your project your build will fail. All you will need to do is fix the bug now, and find bugs will do the checking from now on.

I like this approach because it has 2 advantages: no extra unit test code to maintain + you document your interfaces/code. 

I am a big fan of catching as many issues as possible during build time so I am enforcing coding style, complexity rules and find bugs in my projects. 

There is several other annotations that you can use to document your code and assist findbugs in its analisys:

http://jsr-305.googlecode.com/svn/trunk/javadoc/javax/annotation/package-tree.html

If you use scala you have: http://www.scala-lang.org/api/current/scala/Option.html  a nice abstraction that allows you to better represent optional variables.

I always been fascinated by programming languages, and one of my first languages that I implemented was a simple variant of pascal. I published it on codeguru, one of the main coding sites of the time:

http://www.codeguru.com/cpp/cpp/cpp_mfc/parsing/article.php/c831/ 

This project is the precursor of my second attempt to implement a language:  ZEL. 

One thing I don't like about ZEL and I will probably change when I will have some time, is that it is NOT statically typed. I designed ZEL to mainly be able to write mathematical formulas without worrying about overflow in my java application and to use BigDecimal or custom floating point objects using operators...

Dynamically typed languages are just NOTas good as statically typed languages for complex projects. That is one reason why I am not a big fan of javascript. I know that with closure you can mitigate the disadvantages of dynamically typing, but you are still far away in elegance when compared with a language like scala.

Scala is the only one of the new languages that gets me excited (sorry ruby, javascript fans), and I see a bright future for it. (and for Typesafe)

 UPDATE: you can find the small perf lib I put together at: http://code.google.com/p/spf4j/

After reading a while ago an article about the latest dtrace improvements (log linear quantizations), and thinking how cool it would be to have something similar for my java apps, I went ahead  implemented a small lib to monitor the performance of key components in all my java processes. 

annotation + aspectj + function => l l quantized store in a thread local

+ 

a periodic aggregation and store task, to aggregate and persist the thread local stores to a RRD database (rrd4j)

+

rrd charting capabilities or even better Graphite (like here) and voila, you have a decent performance monitoring solution, with superior capabilities compared with widely used jamon. (better scalability and performance + better detail) 

2 weeks ago I stumbled upon flame charts. Now this is something really simple and useful to implement so here is the initial implementation. This approach can be quite useful if you want to look at the performance of your java processes in production where you need minimal impact. (impact is controlled by the sample rate). 

you can easily get a report for your app by starting it:

java -cp <your classpath> com.zoltran.stackmonitor.Monitor -s 100 -w 2000 -f /tmp/perfChart.html -main dom.your-package.YourMain -- your app args

You will get the report in HTML format, and it will look like: 

CPU stats

WAIT stats

Until now, if I had a multi terabyte database, my implementation choice would have been Oracle. I am in no way a Oracle fan, Oracle has its flaws like for example its crappy installer application, but compared with the other chices like Sybase and DB2 for Linux, it is light years ahead...  

The only strategy of dealing with large amount of data is Divide and Conquer.

With Oracle you use the features that will help you achieve that are  Partitioning and Compression. Depending on your use case, RAC and Exadata might provide you some decent choices. Map reduce is a popular algorithm that can be used to analyze your data and you can do it with Oracle as well:  http://blogs.oracle.com/datawarehousing/entry/in-database_map-reduce 

OK, enough about classic databases, let look into the "new" technologies available out there that I could use instead.

Let's look into Cassandra + Hadoop.

Installation: after downloading, and setting up 4 solaris containers with cassandra I have a running test cluster:

user@host:/zones/common/apache-cassandra-1.0.5/bin$ ./nodetool -host 192.168.1.50 ring

Address         DC          Rack        Status State   Load            Owns    Token                                       

                                                                               111092484197642270557254836842258840364     

192.168.1.51    datacenter1 rack1       Up     Normal  20.45 KB        42.95%  14027083099678615413015982312665843724      

192.168.1.50    datacenter1 rack1       Up     Normal  22.09 KB        1.64%   16817568390080366247986870584687685022      

192.168.1.52    datacenter1 rack1       Up     Normal  11.1 KB         12.46%  38016701835136693969806387655967731276      

192.168.1.53    datacenter1 rack1       Up     Normal  15.47 KB        42.95%  111092484197642270557254836842258840364  

OK, installation was really easy, let's download the code (no maven based build, disapointing), and inspect code quality a bit...

Bad code case 1, comparing with equals different types (QueryProcessor.java):

where the equals compares a ByteBuffer with a RowPosition that will always be false, this one could be a potentially funny bug, there are 2 instances of like this in the codebase. 

Bad code case 2, resource leak:

What if options.keystore does not exist  and a ex is thrown? there 12 instances like this in the codebase.

Bad code case 3 (dead code):

easy to setup timemachine with details from:

http://marcoschuh.de/wp/?p=494 

Finally had some free time to solve the build issues caused by the native part of the build on MACOS (thread locals are to supported as nicely on macos by GCC).

the GMP support on macos works, all you need is to have it installed (recomend using mac ports). You will need to also have Xcode installed.

you can download the latest source at: http://kenai.com/projects/zel

cheers!  

Some bad weather allowed me to tinker a bit with Iphone development. So I dived in into Objective C, not a bad concept, and I like the dynamic aspect of it, however this adds yet another syntax for me to deal with (in addition to my own language, c, c++, java, xsh, sql ...).

So in my pursuit to optimize the utilization of my own memory resources I go ahead and search the internet for "java programming for the Iphone". And to my surprise I find xmlvm (http://www.xmlvm.org). I went ahead, downloaded  the latest source code, and 30 minutes later I had my first Java example application running on my Iphone. All I can say is: "Good Job xmlvm team!"

I believe this project will be a very successful project because it will allow developers to use their favorite language and tools combination (eclipse, netbeans)  to develop applications to develop applications with common code base for Iphone and Android devices. This will be of great advantage for developing enterprise phone applications. I am basing my projection on my observation of how GWT permeated the enterprise web development and replaced the standard web dev libraries in a multitude of projects.

Thou should not underestimate the power of the hordes of Java Developers...

I have introduced experimental parallel execution features in ZEL. This make use of the "future" concept. 

The way this works is that all function calls where the implementation exceeds a cost threshold, are executed in parallel(the vm has a thread pool). When the VM encounters a complex function invocation, it schedules the function for execution in parallel, puts a future in the stack and continues executing the next instruction:

expensiveCompute = lambda (pv,rate,n){ 
sleep(10); return pv * exp(rate * n) * 1.33 *1.33 * 1.33 *1.33 *1.33 * sqrt(10000) ;}; 
val1 = expensiveCompute(100000, 0.045, 10);
val2 = expensiveCompute(100000, 0.05, 10);
val3 = val1 + val2;
The execution engine will compute val1 and val2 in parallel, 
and val3 will be computed when the results for val1 and val2 are available.
There are some issues with recursion that I will further need to look into... (fixed) 
here is how to execute a a expression in parallel:
Program compiledProgram = Program.compile("Your Program");
Object result2 = compiledProgram.execute(new HashMap<Object, Object>(), null, null, null, Executors.newCachedThreadPool()));
Note that all deterministic function results are cached, so fibonacci implementation will look like:  
fib= lambda deterministic (x) { fib( x-1) + fib( x-2); };
fib(0)=0;
 fib(1)=1;
and will perform in O(n) time complexity. 
latest code is available at: http://kenai.com/projects/zel/sources/zel-svn/show
enjoy

As expected work is in progress to replace Linux as the OS building block in Oracle's Database Machines:

http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6965606

Optimizations are being done for a "solexadata_demo"

Nice to see Linux being replaced by the best OS :-)

With the new generation of developers I increasingly see less desire of writing efficient code. It also makes me quite angry when I hear arguments like "disk is cheap", "memory is cheap", ... "premature optimization is evil".

Writing code is a difficult task, it is a continuous balancing between implementation choices,  but writing a efficient implementation is not always hard and time consuming.

I will give a small, recent example:

I recently spent 10 minutes of my time to optimize a data cache component that was taking up 250 Mb of RAM, and reduced the memory usage by 150Mb. Since this optimization was on a component that was reused in several processes my 10 minutes spent became even more valuable. All I did I internalized the string attributes that were highly redundant.

By reducing the amount of memory my application uses, I also increased the performance:

Less GC overhead, higher CPU cache hit ratios, less interconnect utilization (NUMA), more memory available to the disk cache, less process swapping.

One of the metrics you can monitor is:  "cycles per instruction"  (how to measure: http://blogs.sun.com/brendan/entry/amd64_pics). This metric can improve significantly by reducing your memory footprint.

Conclusion: Using String.intern is a simple and efficient way of optimizing your implementation.

My "favorite" part of the brochure: "the ideal server to refresh inefficient and outdated HP Itanium and IBM Power servers." 

They also forgot to mention that the servers are unbreakable :-)

When embedding ZEL the implementation of the Floating point numbers can be chosen as:

ExtendedMathContext.DECIMAL64_MATHCONTEXT

ExtendedMathContext.DECIMAL128_MATHCONTEXT 

ExtendedMathContext.DOUBLE64_MATHCONTEXT

ExtendedMathContext.NATIVE.GNUFLOAT128_MATHCONTEXT 

ExtendedMathContext.NATIVE.GNUFLOAT256_MATHCONTEXT 

ExtendedMathContext.NATIVE.GNUFLOAT512_MATHCONTEXT  

A few days of bad wether and the ZEL language gets some significant enhancements:

Deterministic functions:

Q: What is a deterministic function?

A: A funtion that always returns the same value, for the same parameters.

How do we define a deterministic function:

fib= lambda deterministic (x) { fib( x-1) + fib( x-2); };

fib(0)=0; fib(1)=1;

You can find more info here:

http://blogs.sun.com/bonwick/

Congrats to jeff & co.

One more reason to use opensolaris for your file serving needs. If you want to build your own here is a good example:

http://blogs.sun.com/mebius/entry/diy_home_nas_box_with2

Today's latest trend is going Green, everybody is talking about reducing it's carbon footprint, we have Green cars, Green Buildings, Green Data Centers,.....

The major hardware vendors battle each other on who makes the most energy efficient hardware, and some electricity utility companies even offer rebates for using energy efficient hardware (ex: Sun's UltraSparc T1). 

One day, I wanted to get my MS Project tasks into MS Outlook, here is how I did it:

http://geekswithblogs.net/nestor/archive/2006/12/29/102260.aspx

There is no secret that I am a big Solaris fan. For me as a developer ZFS and DTrace changed my life, improoving my productivity significantly. With ZFS and its snapshoting/cloning capabilities, I have no fear in doing experiments and I can always go back in time and recover in case something goes wrong. On the other hand DTrace with D-Light allows you to look inside your running application.

If you want to learn more there is an excellent tutorial that describes how to use DLight to profile your applications:

http://developers.sun.com/sunstudio/documentation/tutorials/d_light_tutorial/index.html

My server, as every member of my family recently moved to Jersey City, so I i finally  was forced to configure vnc on it to be able to do my unix work.

The Xwin redirect does not work as well as VNC even on my local network...

found this useful blog that assisted me:

http://blogs.sun.com/scf/entry/vnc_setup_on_open_solaris

I have added support for lambda expressions in ZEL, also added a FOR statement that can iterate all variables that match a wildcard and apply a lambda function to it.

abc=1;

afe=2;

bfa =3;

x=0;

result=for("`a*`",lambda(x=x+arg(0);));

print x; // x will be 3

Functions are objects, so you can do, and apply using the () operator:

x = lambda(return arg(1)+arg(0);); z= x(1,2); print z;  //will return 3

In my projects I am required to switch back and forth from C++ and Java.

The typical reason for using C++ is performance and tight integration with OS/Hardware (ex: process management).

The typical reason for using Java is coding speed and portability.

One of the major barriers  that I encounter in my development work  is how software is organized in C++ vs Java, at this chapter java is arguably better than the classic  C++ way. (the hpp/cpp separation is evil :-) )

I found an interesting article about how C++ code can be organized in a more Java like way:

http://strlen.com/rants/javaclassesincpp.html

Now I think this might work if you create "modules" (groups of classes) ....