Your project can be on or off. Your project's priority can be changed. Your job can be changed. But the technology is always heading north!
Wednesday, May 10, 2017
Search anything from a bugzilla bug given a range of bug numbers
[root@centos my-data-set]# for ((i=13420;i<13500;i++)); do echo $i >> /tmp/search_result.txt; curl http://192.168.5.105/bugzilla/show_bug.cgi?id=$i | grep "passed to DE constructor may be null" >> /tmp/search_result.txt; done;
Tuesday, May 2, 2017
A simple way to clone 10,000 files to cluster
# Generate 10,000 files from one seed and put them into 100 subdirectories
[hdfs@cdh1 tmp]$ for((i=1;i<=100;i++));do mkdir -p 10000files/F$i; for((j=1;j<=100;j++));do echo $i-$j; cp 1000line.csv 10000files/F$i/$i-$j.csv;done;done;
# Move them to cluster. One sub-directory at one time.
[hdfs@cdh1 tmp]$ for((i=1;i<=100;i++));do echo $i; hadoop fs -mkdir /JohnZ/10000files/F$i; hadoop fs -copyFromLocal 10000files/F$i/* /JohnZ/10000files/F$i/.;done;
Subscribe to:
Posts (Atom)