Hadoop Could not load native gpl library异常解决
Hadoop Could not load native gpl library异常解决
完整异常:
- 11/11/07 10:15:02 ERROR lzo.GPLNativeCodeLoader: Could not load native gpl library
- java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path
- at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1738)
- at java.lang.Runtime.loadLibrary0(Runtime.java:823)
- at java.lang.System.loadLibrary(System.java:1028)
- at com.hadoop.compression.lzo.GPLNativeCodeLoader.<clinit>(GPLNativeCodeLoader.java:32)
- at com.hadoop.compression.lzo.LzoCodec.<clinit>(LzoCodec.java:71)
- at java.lang.Class.forName0(Native Method)
- at java.lang.Class.forName(Class.java:247)
- at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
- at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:89)
- at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:134)
- at com.twitter.elephantbird.mapreduce.input.LzoRecordReader.initialize(LzoRecordReader.java:61)
- at com.twitter.elephantbird.mapreduce.input.LzoBinaryB64LineRecordReader.initialize(LzoBinaryB64LineRecordReader.java:79)
- at org.apache.hadoop.mapreduce.lib.input.DelegatingRecordReader.initialize(DelegatingRecordReader.java:80)
- at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:450)
- at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:645)
- at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
- at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
- 11/11/07 10:15:02 ERROR lzo.LzoCodec: Cannot load native-lzo without native-hadoop
- 11/11/07 10:15:02 WARN mapred.LocalJobRunner: job_local_0001
- java.lang.RuntimeException: native-lzo library not available
- at com.hadoop.compression.lzo.LzopCodec.createDecompressor(LzopCodec.java:91)
- at com.hadoop.compression.lzo.LzopCodec.createInputStream(LzopCodec.java:76)
- at com.twitter.elephantbird.mapreduce.input.LzoRecordReader.initialize(LzoRecordReader.java:71)
- at com.twitter.elephantbird.mapreduce.input.LzoBinaryB64LineRecordReader.initialize(LzoBinaryB64LineRecordReader.java:79)
- at org.apache.hadoop.mapreduce.lib.input.DelegatingRecordReader.initialize(DelegatingRecordReader.java:80)
- at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:450)
- at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:645)
- at org.apache.hadoop.mapred.MapTask.run(MapTask.java:322)
- at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)
得到的结果发现已经是64位,结果如下:
java version "1.6.0_26"
- Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
- Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
后来想下自己压根就没有装什么lzo,同时找到了这篇:http://comments.gmane.org/gmane.comp.java.hadoop.hbase.user/15776.验证了我的想法,根据里面的提示http://code.google.com/a/apache-extras.org/p/hadoop-gpl-compression/wiki/FAQ?redir=1一步步安装.命令如下: