Tech Notes

Your project can be on or off. Your project's priority can be changed. Your job can be changed. But the technology is always heading north!

Thursday, February 16, 2017

My co-worker Shawn's blogs for Hadoop machine setup and installation

http://wp.huangshiyang.com/env-set-up-centos

Solving protobuf version conflicting in your Java applications

Due to the bad design of protobuf, we faced the version conflicting from time to time. Finally, we figured out a way to solve this issue usi...

Tuesday, February 14, 2017

Hadoop Proxy user - Superusers Acting On Behalf Of Other Users

https://hadoop.apache.org/docs/r2.7.2/hadoop-project-dist/hadoop-common/Superusers.html

Wednesday, February 8, 2017

High performance regex open source packages

Java library to check for multiple regexp with a single deterministic automaton. Just a wrapper around dk.brics.automaton: https://github...

Monday, February 6, 2017

My first scala from Spark system: word count

[root@john1 john]# cd /var/lib/hadoop-hdfs [root@john1 hadoop-hdfs]# sudo -u hdfs spark-shell --master yarn --deploy-mode client Setting d...

Thursday, February 2, 2017

Solve KMS authentication issue under CDH5 HA environment

2017-01-31 21:53:41,538 WARN org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter: Authentication exception:...

Wednesday, January 18, 2017

Dynamically load jar file in Java

File customJarHome = new File(customJarHomePath); URLClassLoader classLoader = (URLClassLoader) Thread.currentThread().getContextClassL...

View web version

Powered by Blogger.