码迷,mamicode.com
首页 > Windows程序 > 详细

Window中调试HBase问题小结

时间:2014-08-05 00:37:28      阅读:849      评论:0      收藏:0      [点我收藏+]

标签:des   style   blog   http   color   java   使用   os   

  1.好久没用log4j了,转到logback好多年了,hbase程序运行时,报缺少log4j配置,那么,就转去logback吧(以下的XXX表示版本号)。

  原先lib包里面有log4j-XXX.jar、slf4j-api-XXX.jar、slf4j-log4j12-XXX.jar,干掉log4j-XXX.jar和slf4j-log4j12-XXX.jar,加入jcl-over-slf4j-XXX.jar,log4j-over-slf4j-XXX.jar、logback-classic-XXX.jar、logback-core-XXX.jar,至于原先的slf4j-api这个包,可以顺手替换成新的,然后,加入一个logback.xml到classpath下

<?xml version="1.0" encoding="UTF-8"?>

<!-- For assistance related to logback-translator or configuration  -->
<!-- files in general, please contact the logback user mailing list -->
<!-- at http://www.qos.ch/mailman/listinfo/logback-user             -->
<!--                                                                -->
<!-- For professional support please see                            -->
<!--    http://www.qos.ch/shop/products/professionalSupport         -->
<!--                                                                -->
<configuration scan="true" scanPeriod="120 seconds" debug="false">
  <appender name="FileApp" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <!--See also http://logback.qos.ch/manual/appenders.html#RollingFileAppender-->
    <filter class="ch.qos.logback.classic.filter.ThresholdFilter"> 
      <level>WARN</level> 
    </filter>
    <File>log/HBaseClient.log</File>
    <encoder>
      <pattern>%d [%t] %-5p %c - %m%n</pattern>
    </encoder>
    <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
      <maxIndex>10</maxIndex>
      <FileNamePattern>log/HBaseClient.log.%i</FileNamePattern>
    </rollingPolicy>
    <triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
      <MaxFileSize>1024KB</MaxFileSize>
    </triggeringPolicy>
  </appender>
  <appender name="ConApp" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <pattern>%d [%t] %-5p %c - %m%n</pattern>
    </encoder>
  </appender> 
  <root level="ERROR">
      <appender-ref ref="ConApp"/>
    <appender-ref ref="FileApp"/>
  </root>
</configuration>

  这样,log4j和系统的日志系统就被logback接管了,至于为啥不用log4j,用logback,这个。。。自己去看吧,有空我可能会写点啥,logback有点小bug,就是有的linux系统下会找不到配置文件,以前修复过,不知道新版本有处理掉没。

  2.Could not locate executable null\bin\winutils.exe in the Hadoop binaries

  老掉牙的问题了,系统变量设置HADOOP_HOME,我还是不想去设置环境变量,还是一行代码来得快

System.setProperty("hadoop.home.dir", "G:/hadoop/hadoop-2.4.1");

  3.程序运行正常,但是当log级别改为Info时,突然发现了这么一条

2014-08-04 16:19:22,942 [main-SendThread(Master:2222)] INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server Master/192.168.117.128:2222. Will not attempt to authenticate using SASL (java.lang.SecurityException: 无法定位登录配置)

  看到个Exception,什么情况?觉得哪里不对,但是程序又是一切正常的。

  搜索了一通,发现好多一样的问题,但是貌似和我的没啥关系,别人是程序跑不通,我是程序正常得很。

  有说是zookeeper版本不一致的,我看了下hadoop和hbase中的确实有区别,一个zookeeper-3.4.5.jar,一个zookeeper-3.4.6.jar,但是我总觉得不对,新旧怎么可能会不兼容啊。那就直接统一了试试,连客户端程序的都统一了,结果那个info信息还是存在,就和不换是一样的,果然不是这里的问题!

  有说是配置和hosts的,我这里都没这问题的,都配置过了的。加参数?我在配置中都写好的,直接打印一看就知道有了。

  这只是一个info信息,很奇怪为啥是个info,照说info级别的都是不影响啥的信息,程序确实是很正常的跑,但是看到这个信息就是不舒服。

  自己查代码去算了,根据日志,是在ClientCnxn.java中出现的信息,好吧,查看源码

private void logStartConnect(InetSocketAddress addr) {
            String msg = "Opening socket connection to server " + addr;
            if (zooKeeperSaslClient != null) {
              msg += ". " + zooKeeperSaslClient.getConfigStatus();
            }
            LOG.info(msg);
        }

  汗,还得换去查ZooKeeperSaslClient.java,继续吧

public ZooKeeperSaslClient(final String serverPrincipal)
            throws LoginException {
        /**
         * ZOOKEEPER-1373: allow system property to specify the JAAS
         * configuration section that the zookeeper client should use.
         * Default to "Client".
         */
        String clientSection = System.getProperty(ZooKeeperSaslClient.LOGIN_CONTEXT_NAME_KEY, "Client");
        // Note that ‘Configuration‘ here refers to javax.security.auth.login.Configuration.
        AppConfigurationEntry entries[] = null;
        RuntimeException runtimeException = null;
        try {
            entries = Configuration.getConfiguration().getAppConfigurationEntry(clientSection);
        } catch (SecurityException e) {
            // handle below: might be harmless if the user doesn‘t intend to use JAAS authentication.
            runtimeException = e;
        } catch (IllegalArgumentException e) {
            // third party customized getAppConfigurationEntry could throw IllegalArgumentException when JAAS
            // configuration isn‘t set. We can reevaluate whether to catch RuntimeException instead when more 
            // different types of RuntimeException found
            runtimeException = e;
        }
        if (entries != null) {
            this.configStatus = "Will attempt to SASL-authenticate using Login Context section ‘" + clientSection + "‘";
            this.saslClient = createSaslClient(serverPrincipal, clientSection);
        } else {
            // Handle situation of clientSection‘s being null: it might simply because the client does not intend to 
            // use SASL, so not necessarily an error.
            saslState = SaslState.FAILED;
            String explicitClientSection = System.getProperty(ZooKeeperSaslClient.LOGIN_CONTEXT_NAME_KEY);
            if (explicitClientSection != null) {
                // If the user explicitly overrides the default Login Context, they probably expected SASL to
                // succeed. But if we got here, SASL failed.
                if (runtimeException != null) {
                    throw new LoginException("Zookeeper client cannot authenticate using the " + explicitClientSection +
                            " section of the supplied JAAS configuration: ‘" +
                            System.getProperty(Environment.JAAS_CONF_KEY) + "‘ because of a " +
                            "RuntimeException: " + runtimeException);
                } else {
                    throw new LoginException("Client cannot SASL-authenticate because the specified JAAS configuration " +
                            "section ‘" + explicitClientSection + "‘ could not be found.");
                }
            } else {
                // The user did not override the default context. It might be that they just don‘t intend to use SASL,
                // so log at INFO, not WARN, since they don‘t expect any SASL-related information.
                String msg = "Will not attempt to authenticate using SASL ";
                if (runtimeException != null) {
                    msg += "(" + runtimeException + ")";
                } else {
                    msg += "(unknown error)";
                }
                this.configStatus = msg;
                this.isSASLConfigured = false;
            }
            if (System.getProperty(Environment.JAAS_CONF_KEY)  != null) {
                // Again, the user explicitly set something SASL-related, so they probably expected SASL to succeed.
                if (runtimeException != null) {
                    throw new LoginException("Zookeeper client cannot authenticate using the ‘" +
                            System.getProperty(ZooKeeperSaslClient.LOGIN_CONTEXT_NAME_KEY, "Client") +
                            "‘ section of the supplied JAAS configuration: ‘" +
                            System.getProperty(Environment.JAAS_CONF_KEY) + "‘ because of a " +
                            "RuntimeException: " + runtimeException);
                } else {
                    throw new LoginException("No JAAS configuration section named ‘" +
                            System.getProperty(ZooKeeperSaslClient.LOGIN_CONTEXT_NAME_KEY, "Client") +
                            "‘ was found in specified JAAS configuration file: ‘" +
                            System.getProperty(Environment.JAAS_CONF_KEY) + "‘.");
                }
            }
        }
    }

  根据上面的代码,可知问题在于红色部分,再注意一下上面的注释,我靠,果然只是一个普通的说明文字,但是,尼玛能不能不在info信息里面搞个Exception提示出来???

  好吧,那确实是一个info信息,是无关紧要的info信息,其实只是说明了。。。。。由于客户端没用SASL,所以不使用SASL进行验证。。。

  能不能换个好点的说明文字?我这些操作都是多余的了!人家就那样显示的,唉。看得不爽也没办法了,难道为了这点东西还去改源码不成?够汗的。  

  解决办法,客户端添加SASL信息,然后服务端也改掉,添加验证,这个官方文档http://hbase.apache.org/book.html#security有说明了,直接看去就是了,这里不说明了。

  唉,处女座追求完美的心,果然是不变的。。。加上,看不到烦心的文字,舒坦了。。

 

  

 

Window中调试HBase问题小结,布布扣,bubuko.com

Window中调试HBase问题小结

标签:des   style   blog   http   color   java   使用   os   

原文地址:http://www.cnblogs.com/zxub/p/3890641.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!