码迷,mamicode.com
首页 > 其他好文 > 详细

解决sqoop报错Invalid number; item = ITEM_UNICODE

时间:2017-07-04 23:27:07      阅读:491      评论:0      收藏:0      [点我收藏+]

标签:read   exti   adf   字符   exception   nat   control   access   类型   

报错栈:

java.sql.SQLException: Invalid number; item = ITEM_UNICODE
    at com.intersys.jdbc.SysList.getInt(SysList.java:1735)
    at com.intersys.jdbc.CacheResultSet.getInt(CacheResultSet.java:247)
    at org.apache.sqoop.lib.JdbcWritableBridge.readInteger(JdbcWritableBridge.java:52)
    at com.cloudera.sqoop.lib.JdbcWritableBridge.readInteger(JdbcWritableBridge.java:53)
    at QueryResult.readFields(QueryResult.java:246)
    at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:244)
    at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:556)
    at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
    at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
    at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

问题的根本原因是字段值类型异常,本来应该是数字型,里面却存了unicode字符。使用报错信息查不到相关资料,另外增加参数也不能解决:-D mapreduce.map.failures.maxpercent=1, query里面增加: and Specimen is not null 也不能解决。(Specimen是报错字段)。

靠谱的解决办法是找出这行异常数据,在数据库中纠正或删除。如果想临时解决,可以把这个字段在导入时转换类型为字符:--map-column-java Specimen=String。

解决sqoop报错Invalid number; item = ITEM_UNICODE

标签:read   exti   adf   字符   exception   nat   control   access   类型   

原文地址:http://www.cnblogs.com/aprilrain/p/7118848.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!