Troubleshooting version mismatch issue of map reduce job

When coding map reduce job and code to access Hadoop file system, you have to include hadoop jar files to make your java code to pass compiler. If the version of jar files are not compatible with the jar files of Hadoop system, the 500 error will be gotten when running your java code.

The 500 error code does not really tell anything about what is the problem.  This makes troubleshooting little bit hard.  To identify this kind of problem, we have to go to cluster to check logs.

For example, looking into hadoop-hdfs-namenode-*.log and try to find following error:

2014-08-14 14:21:42,386 WARN org.apache.hadoop.ipc.Server: Incorrect header or version mismatch from 192.168.72.1:15444 got version 7 expected version 4

Basically, this is telling us that cluster expects version 4, but client is using version 7.

Having this kind of information, lower the version of hadoop jar files.  Now my hadoop code works perfect fine with cluster.