码迷,mamicode.com
首页 > 编程语言 > 详细

idea连接spark集群报错解析:Caused by: java.lang.ClassCastException

时间:2019-11-07 17:41:34      阅读:113      评论:0      收藏:0      [点我收藏+]

标签:lan   就会   没有   RKE   reg   sql   tee   exception   参数   

cannot assign instance of scala.collection.immutable.List$SerializationProxy to field org.apache.spark.sql.execution.aggregate.SortAggregateExec.aggregateExpressions of type scala.collection.Seq in instance of org.apache.spark.sql.execution.aggregate.SortAggregateExec
Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 8 in stage 4.0 failed 4 times, most recent failure: Lost task 8.3 in stage 4.0 (TID 332, 172.16.43.200, executor 1): java.lang.ClassCastException: cannot assign instance of scala.collection.immutable.List$SerializationProxy to field org.apache.spark.sql.execution.aggregate.SortAggregateExec.aggregateExpressions of type scala.collection.Seq in instance of org.apache.spark.sql.execution.aggregate.SortAggregateExec

  当只指定了spark集群的地址,没有设定setJars这个参数,那么就会报以上这种错误,解决方法就是设置setjars这个参数,如下:

System.setProperty("hadoop.home.dir", "E:\\winutils")
    val conf = new SparkConf().setAppName("DetailRatio")
      .setMaster("spark://172.xx.xx.xx:7077")
   // .setMaster("local")
     .setJars(List("E:\\vense_work\\venseData\\out\\artifacts\\DetailRatio_jar\\venseData.jar"))
    //.set("spark.submit.deployMode", "client")

  

idea连接spark集群报错解析:Caused by: java.lang.ClassCastException

标签:lan   就会   没有   RKE   reg   sql   tee   exception   参数   

原文地址:https://www.cnblogs.com/zhnagqi-dream/p/11813578.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!