Friday, December 18, 2015

A simple way for reducer to get how many records are processed in mapreduce job's mapper

›
JobClient client = new JobClient(jobConf); RunningJob mapreduceJob = client.getJob(JobID.forName( jobConf.get("mapred.job.id") ...
Wednesday, November 4, 2015

A simple way to calculate PageRank in R

›
library(igraph) source("http://michael.hahsler.net/SMU/ScientificCompR/code/map.R") # 1->2; 1->3, 2->1, 3->4, 4-...
Tuesday, October 27, 2015

Good articles for shingle, minhash and LSH

›
http://matthewcasperson.blogspot.com/2013/11/minhash-for-dummies.html http://infolab.stanford.edu/~ullman/mmds/bookL.pdf   (Chapter 3, rou...
Friday, September 25, 2015

How to override cluster jar files

›
If you need to use your own version of a jar file in your map reduce job, you can do so through following steps: 1. Bundle the jar file in...
Saturday, July 25, 2015

Dynamically add jar files to classpath

›
    /**      * Adds jar files to classpath.      * @param file the folder for jar files.      * @param classLoader the URLClassLoader   ...
Wednesday, June 10, 2015

Skip Hadoop mapper when condition is satisfied

›
If conf is an instance of JobConf: conf.setMapRunnerClass(MyMapRunner.class); A sample of your MyMapRunner,  public class FindAn...
Tuesday, February 3, 2015

Awesome solutions for some super good problems in Cryptograph

›
http://webstersprodigy.net/tag/aes/ Just some quotations from above website: One of the biggest reasons I think the class was so good wa...