对象反序列化readFields时报Error: java.lang.NullPointerException
场景:在使用自定义对象序列化时,报错:
<2015-09-29 14:59:15,578> <INFO > <org.apache.hadoop.mapreduce.Job> - Task Id : attempt_1441852661551_0672_r_000002_2, Status : FAILED(Job.java:printTaskEvents:1462)
Error: java.lang.NullPointerException
at com.wbkit.cobub.models.UserActionDay.readFields(UserActionDay.java:86)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:71)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)
at org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKeyValue(ReduceContextImpl.java:146)
at org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKey(ReduceContextImpl.java:121)
at org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.nextKey(WrappedReducer.java:307)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:170)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
自定义对象序列化应用分析:
所有的Writable实现都必须有一个默认的构造函数,以便MapReduce框架可以对它们进行实例化,然后再调用readFields()函数查看(填充)各个字段的值。
解决办法:
在自定义序列化对象中,即使在默认构造中也对对象的成员变量进行初始化。
参考:
详细请参见 Hadoop权威指南(第三版) 4.3.3小节;
相关推荐
spring-data-redis RedisTemplate 操作redis时发现存储在redis中的key不是设置的string值,前面还多出了许多类似\xac\xed\x00\x05t\x00;