码迷,mamicode.com
首页 > 编程语言 > 详细

排查Hive报错:org.apache.hadoop.hive.serde2.SerDeException: java.io.IOException: Start of Array expected

时间:2017-05-28 19:33:30      阅读:3602      评论:0      收藏:0      [点我收藏+]

标签:cal   create   exception   ges   pac   creat   oca   排查   路径   

CREATE TABLE json_nested_test (
    count string,
    usage string,
    pkg map<string,string>,
    languages array<string>,
    store map<string,array<map<string,string>>>)
ROW FORMAT SERDE org.apache.hive.hcatalog.data.JsonSerDe
STORED AS TEXTFILE;

以上述sql创建表json_nested_test后,查询时发现报错:Failed with exception java.io.IOException:org.apache.hadoop.hive.serde2.SerDeException: java.io.IOException: Start of Array expected

于是进行如下测试,发现是org.apache.hive.hcatalog.data.JsonSerDe 对复杂类型支持不足造成,例如map<string,array<string>>,这个例子中就是不支持array作为map的value.
CREATE TABLE json_nested_test_openx (
    count string,
    usage string,
    pkg map<string,string>,
    languages array<string>,
    store map<string,array<map<string,string>>>)
ROW FORMAT SERDE org.openx.data.jsonserde.JsonSerDe
STORED AS TEXTFILE;

CREATE TABLE s1 (
    count string,
    usage string
)
ROW FORMAT SERDE org.apache.hive.hcatalog.data.JsonSerDe
STORED AS TEXTFILE;
load data local inpath /home/work/s1.txt overwrite into table s1;
select * from s1;
CREATE TABLE s2 (
    count string,
    usage string,
    pkg map<string,string>
)
ROW FORMAT SERDE org.apache.hive.hcatalog.data.JsonSerDe
STORED AS TEXTFILE;
load data local inpath /home/work/s2.txt overwrite into table s2;
select * from s2;

CREATE TABLE s3 (
    count string,
    usage string,
    pkg map<string,string>,
    languages array<string>
)
ROW FORMAT SERDE org.apache.hive.hcatalog.data.JsonSerDe
STORED AS TEXTFILE;
load data local inpath /home/work/s3.txt overwrite into table s3;
select * from s3;
CREATE TABLE s4 (
    count string,
    usage string,
    pkg map<string,string>,
    languages array<string>,
    store map<string,array<map<string,string>>>
)
ROW FORMAT SERDE org.apache.hive.hcatalog.data.JsonSerDe
STORED AS TEXTFILE;
load data local inpath /home/work/s4.txt overwrite into table s4;
select * from s4;

CREATE TABLE s5 (
    store map<string,array<map<string,string>>>
)
ROW FORMAT SERDE org.apache.hive.hcatalog.data.JsonSerDe
STORED AS TEXTFILE;
load data local inpath /home/work/s5.txt overwrite into table s5;
select * from s5;

CREATE TABLE s6 (
    store map<string,array<string>>
)
ROW FORMAT SERDE org.apache.hive.hcatalog.data.JsonSerDe
STORED AS TEXTFILE;
load data local inpath /home/work/s6.txt overwrite into table s6;
select * from s6;

这个Serde hive自带,路径为$HIVE_HOME/hcatalog/share/hcatalog/hive-hcatalog-core-1.2.1.jar,它也存在其它问题:不支持数据文件中的空行

排查Hive报错:org.apache.hadoop.hive.serde2.SerDeException: java.io.IOException: Start of Array expected

标签:cal   create   exception   ges   pac   creat   oca   排查   路径   

原文地址:http://www.cnblogs.com/aprilrain/p/6916359.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!