码迷,mamicode.com
首页 > 其他好文 > 详细

Flink升级到1.4版本遇到的坑

时间:2017-12-18 18:37:12      阅读:907      评论:0      收藏:0      [点我收藏+]

标签:png   scala   res   time   使用   1.3   sem   环境   hang   

  Flink 1.4没出来以前,一直使用Flink 1.3.2,感觉还算稳定,最近将运行环境升级到1.4,遇到了一些坑:

 

1.需要将可运行程序,基于1.4.0重新编译一次

2.对比了一下flink-conf.yml中的配置,发现一处不同点:

# The port under which the web-based runtime monitor listens.
# A value of -1 deactivates the web server.

1.3.2:jobmanager.web.port: 8081

1.4.0:web.port: 8081

1.4.0 少了前面的“jobmanager”,但是查看官网1.4.0的文档,配置项却是:jobmanager.web.port(https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/config.html)

查看源代码,原来是两个都行,不过后面应该建议都用web开头的,改了还不少:

技术分享图片

3.在Configuration的Common Options部分,1.4多了如下的东西:

classloader.resolve-order: Whether Flink should use a child-first ClassLoader when loading 
user-code classes or a parent-first ClassLoader. Can be one of parent-first or child-first. (default: child-first) classloader.parent-first-patterns: A (semicolon-separated) list of patterns that specifies
which classes should always be resolved through the parent ClassLoader first.
A pattern is a simple prefix that is checked against the fully qualified class name.
By default, this is set to java.;org.apache.flink.;javax.annotation;org.slf4j;org.apache.log4j;org.apache.logging.log4j;ch.qos.logback.
If you want to change this setting you have to make sure to also include the default patterns in your list of patterns if you want to keep that default behaviour.

我的程序,原来在1.3.2上跑着没问题,换到1.4后,总是报:

java.lang.LinkageError: loader constraint violation:
loader (instance of org/apache/flink/runtime/execution/librarycache/FlinkUserCodeClassLoaders$ChildFirstClassLoader)
previously initiated loading for a different type with name "scala/collection/Iterable"

在flink-conf.yml中配置了

classloader.resolve-order: parent-first

后问题解决

老外貌似也遇到这个问题了:

https://issues.apache.org/jira/browse/FLINK-7669

http://mail-archives.apache.org/mod_mbox/flink-commits/201710.mbox/%3Cbf12238e06074089b1a590178e335063@git.apache.org%3E

Flink升级到1.4版本遇到的坑

标签:png   scala   res   time   使用   1.3   sem   环境   hang   

原文地址:http://www.cnblogs.com/liugh/p/8057656.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!