same as compile hbase ,it 's necessary for u to compile your own dist of hadoop to get best performance specified to your hardwares.
bellow are some steps needed by this theme:
1.install some tools
a.cmake 2.6+
download and set the path variable
this is used only if u want to make with native option,same like:
mvn package -Pdist,native -DskipTests=true -Dmaven.javadoc.skip=true
b.protobuf 2.5+
download and set variable
c.mavn 3.0+
same as above
2.comile hadoop by issuing
mvn package -Pdist,native -DskipTests=true -Dmaven.javadoc.skip=true -Dtar
the results will like below:
main: [exec] $ tar cf hadoop-2.5.2.tar hadoop-2.5.2 [exec] $ gzip -f hadoop-2.5.2.tar [exec] [exec] Hadoop dist tar available at: /home/xxx/build-hadoop/hadoop-2.5.2-src/hadoop-dist/target/hadoop-2.5.2.tar.gz [exec] [INFO] Executed tasks [INFO] [INFO] --- maven-javadoc-plugin:2.8.1:jar (module-javadocs) @ hadoop-dist --- [INFO] Building jar: /home/xxx/build-hadoop/hadoop-2.5.2-src/hadoop-dist/target/hadoop-dist-2.5.2-javadoc.jar [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Apache Hadoop Main ................................ SUCCESS [3:01.134s] [INFO] Apache Hadoop Project POM ......................... SUCCESS [1:05.262s] [INFO] Apache Hadoop Annotations ......................... SUCCESS [36.726s] [INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.232s] [INFO] Apache Hadoop Project Dist POM .................... SUCCESS [40.162s] [INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [42.839s] [INFO] Apache Hadoop MiniKDC ............................. SUCCESS [2:45.883s] [INFO] Apache Hadoop Auth ................................ SUCCESS [36.327s] [INFO] Apache Hadoop Auth Examples ....................... SUCCESS [13.556s] [INFO] Apache Hadoop Common .............................. SUCCESS [3:46.794s] [INFO] Apache Hadoop NFS ................................. SUCCESS [4.543s] [INFO] Apache Hadoop Common Project ...................... SUCCESS [0.037s] [INFO] Apache Hadoop HDFS ................................ SUCCESS [3:32.422s] [INFO] Apache Hadoop HttpFS .............................. SUCCESS [28.563s] [INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [31.706s] [INFO] Apache Hadoop HDFS-NFS ............................ SUCCESS [3.193s] [INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.042s] [INFO] hadoop-yarn ....................................... SUCCESS [0.032s] [INFO] hadoop-yarn-api ................................... SUCCESS [1:15.932s] [INFO] hadoop-yarn-common ................................ SUCCESS [41.828s] [INFO] hadoop-yarn-server ................................ SUCCESS [0.032s] [INFO] hadoop-yarn-server-common ......................... SUCCESS [19.335s] [INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [52.061s] [INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [2.190s] [INFO] hadoop-yarn-server-applicationhistoryservice ...... SUCCESS [4.083s] [INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [10.974s] [INFO] hadoop-yarn-server-tests .......................... SUCCESS [0.469s] [INFO] hadoop-yarn-client ................................ SUCCESS [4.002s] [INFO] hadoop-yarn-applications .......................... SUCCESS [0.032s] [INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [2.106s] [INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS [1.710s] [INFO] hadoop-yarn-site .................................. SUCCESS [0.028s] [INFO] hadoop-yarn-project ............................... SUCCESS [4.258s] [INFO] hadoop-mapreduce-client ........................... SUCCESS [0.067s] [INFO] hadoop-mapreduce-client-core ...................... SUCCESS [14.625s] [INFO] hadoop-mapreduce-client-common .................... SUCCESS [15.159s] [INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [3.097s] [INFO] hadoop-mapreduce-client-app ....................... SUCCESS [7.088s] [INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [6.639s] [INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [13.638s] [INFO] hadoop-mapreduce-client-hs-plugins ................ SUCCESS [1.573s] [INFO] Apache Hadoop MapReduce Examples .................. SUCCESS [4.095s] [INFO] hadoop-mapreduce .................................. SUCCESS [3.397s] [INFO] Apache Hadoop MapReduce Streaming ................. SUCCESS [8.712s] [INFO] Apache Hadoop Distributed Copy .................... SUCCESS [25.432s] [INFO] Apache Hadoop Archives ............................ SUCCESS [1.784s] [INFO] Apache Hadoop Rumen ............................... SUCCESS [4.306s] [INFO] Apache Hadoop Gridmix ............................. SUCCESS [3.500s] [INFO] Apache Hadoop Data Join ........................... SUCCESS [2.054s] [INFO] Apache Hadoop Extras .............................. SUCCESS [2.298s] [INFO] Apache Hadoop Pipes ............................... SUCCESS [7.284s] [INFO] Apache Hadoop OpenStack support ................... SUCCESS [3.687s] [INFO] Apache Hadoop Client .............................. SUCCESS [8.373s] [INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [0.121s] [INFO] Apache Hadoop Scheduler Load Simulator ............ SUCCESS [10.919s] [INFO] Apache Hadoop Tools Dist .......................... SUCCESS [6.671s] [INFO] Apache Hadoop Tools ............................... SUCCESS [0.033s] [INFO] Apache Hadoop Distribution ........................ SUCCESS [32.206s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 26:07.575s [INFO] Finished at: Fri Dec 12 16:12:56 CST 2014 [INFO] Final Memory: 123M/940M [INFO] ------------------------------------------------------------------------
3.problems
a.Exception in thread "main" java.lang.AssertionError: Missing tools.jar at: /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/Classes/classes.jar. Expression: file.exists()
this is as the compiler needs a file named tools.jar but localted at classes.jar under dir Classes but found nothing.so u can do workaround:
cd `java_home`/ mkdir Classes cd Classes ln -s ../lib/tools.jar classes.jar
but u should know,in macbook,the command 'ln' is ONLY effect when u locate in the target dir,otherwise u will see:
cd `java_home` mkdir Classes ln -s lib/tools.jar Classes/classes.jar # or:ln -s lib/tools.jar /absolute/path/to/Classes/classes.jar cp Classes/classes.jar /tmp Error:no such file or dir found
2.failed to compile target:hadoop mavn doc
this is not important for this theme,so u can add a param to ignore it:
-Dmaven.javadoc.skip=true
Ref:
相关推荐
elasticsearch-head-compile-after.tar.gz+node-v8.1.2-linux-x64.ta
ant compile-native tar 将生成的 build/hadoop-lzo-0.4.15.jar cp 到 /usr/local/hadoop-1.0.2/lib 测试解压程序 bin/hadoop jar /usr/local/hadoop-1.0.2/lib/hadoop-lzo-0.4.15.jar ...
hadoop-compile-depend.zip
compile-node-sass,编译sass/scss为css。
opengl工具glfw以及完整的使用说明文档,可以应用于多种操作系统 compile.ami config.log lib readme.html compile.bat docs images license.txt support compile.sh examples include Makefile
编译后的 elasticsearch-head 源码包,建议配合安装教程《如何安装 elasticsearch-head 插件?》一同使用
Before you compile the table, you must install requirements packages, to do: #yum -y install ibus-table ibus-libs ibus-table-additional ibus-devel Now you can compile it: #./configure --prefix=/usr...
Python-2.7.13-xcompile.patch
./configure -confirm-license -opensource -debug-and-release -static -prefix ../build -no-opengl -nomake tests -no-compile-examples -nomake examples -qt-xcb ———————————————— 原文链接:...
2.如果想支持x264和openssl请先运行compile-x264.sh,compile-openssl.sh 3.运行compile-ffmpeg.sh 4.运行compile-ijk.sh 5.最后生成ffmpeg_pic,ffmpeg_pie, libijkplayer.so, libijksdl.so, libijkffmpeg.so.
python库。 资源全名:req_compile-0.10.15-py2.py3-none-any.whl
So after years of pain, I jumped on oDesk and contracted Ilya Kitaev, to both compile with Microsoft Visual Studio, and also prepare automated tools for easy compiling in the future. Update: MSVC isn...
otlin-compiler-embeddable-1.6-M1 下载慢 otlin-compiler-embeddable 下载慢 直接复制到 C:\Users\xxx\.gradle\caches\modules-2\files-2.1\org.jetbrains.kotlin\kotlin-compiler-embeddable\1.6.0-M1
资源来自pypi官网。 资源全名:pip_compile_multi-1.5.0-py2.py3-none-any.whl
// text file, the second is unknown and is written to a file on // the server Iterator iter = fileItems.iterator(); // 正则匹配,过滤路径取文件名 String regExp = ".+\\\\(.+)$"; // 过滤掉的文件...
下载之后解压文件,使用make && make install 解压安装。 这个是使用方法 http://www.360doc.com/content/19/0403/19/99071_826230532.shtml
Extensions to RxJava This library allow simple implementation for some tasks in android Usage Add library to project dependencies. repositories { ... compile 'com.github.jacek-marchwicki.rx-java-