标签：ora.listener.lsnr spawned by agent lsnragentcheck
一：版本信息
操作系统版本：AIX 61009
数据库版本：11.2.0.3.11(RAC)
二：错误描述
1) crsctl stat res -t命令查看crs资源状态的时候，发现"ora.LISTENER.lsnr"资源状态异常：
ora.LISTENER.lsnr
ONLINE OFFLINE ****1 ##实例1
ONLINE OFFLINE ****2 ##实例2
2)检查监听状态正常
lsnrctl LSNRCTL for IBM/AIX RISC System/6000: Version 11.2.0.3.0 - Production on 11-NOV-2014 10:53:26 Copyright (c) 1991, 2011, Oracle. All rights reserved. Welcome to LSNRCTL, type "help" for information. LSNRCTL> set password Password: The command completed successfully LSNRCTL> status Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER))) STATUS of the LISTENER ------------------------ Alias LISTENER Version TNSLSNR for IBM/AIX RISC System/6000: Version 11.2.0.3.0 - Production Start Date 06-NOV-2014 17:22:05 Uptime 4 days 17 hr. 31 min. 48 sec Trace Level off Security ON: Password SNMP OFF Listener Parameter File /oracle/app/11.2.0/grid/network/admin/listener.ora Listener Log File /oracle/app/11.2.0/grid/log/diag/tnslsnr/bidb4/listener/alert/log.xml Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=***.***.**.**)(PORT=1521))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=***.***.**.***)(PORT=1521))) Services Summary... Service "+ASM" has 1 instance(s). Instance "+ASM2", status READY, has 1 handler(s) for this service... Service "bidb" has 1 instance(s). Instance "bidb2", status READY, has 1 handler(s) for this service... The command completed successfully
3)检查监听日志，发现目前监听还能正常接受连接
su - grid
cd $ORACLE_HOME/log/diag/tnslsnr/主机名/listener/alert
</msg> <msg time='2014-11-11T10:46:02.602+08:00' org_id='oracle' comp_id='tnslsnr' type='UNKNOWN' level='16' host_id='bidb4'##数据库主机名 host_addr='***.***.**.**>##数据库主机ip地址 <txt>11-NOV-2014 10:46:02 * (CONNECT_DATA=(SERVICE_NAME=bidb)(CID=(PROGRAM=osh)(HOST=EGSNS-49-24.nd.cmgame.com)(USER=dsadm))(SERVER=dedicated)(INSTANCE_NAME=bidb2)) * (ADDRESS=(PROTOCOL=tcp)(HOST=***.***.**.**)(PORT=61378)) * establish * bidb * 0 </txt> ##连接发起的IP地址
4)检查 CRS alert日志发现有如下报错：
/oracle/app/11.2.0/grid/bin/oraagent.bin(7274512)]CRS-5016:Process "/oracle/app/11.2.0/grid/bin/lsnrctl" spawned by agent "/oracle/app/11.2.0/grid/bin/oraagent.bin" f or action "start" failed: details at "(:CLSN00010:)" in "/oracle/app/11.2.0/grid/log/bidb4/agent/crsd/oraagent_grid/oraagent_grid.log" 2014-11-07 15:03:21.139: [/oracle/app/11.2.0/grid/bin/oraagent.bin(7274512)]CRS-5016:Process "/oracle/app/11.2.0/grid/bin/lsnrctl" spawned by agent "/oracle/app/11.2.0/grid/bin/oraagent.bin" f or action "start" failed: details at "(:CLSN00010:)" in "/oracle/app/11.2.0/grid/log/bidb4/agent/crsd/oraagent_grid/oraagent_grid.log" 2014-11-07 15:03:21.267: <span style="color:#ff0000;">[/oracle/app/11.2.0/grid/bin/oraagent.bin(7274512)]CRS-5016:Process "/oracle/app/11.2.0/grid/bin/lsnrctl" spawned by agent "/oracle/app/11.2.0/grid/bin/oraagent.bin" f or action "check" failed: details at "(:CLSN00010:)" in "/oracle/app/11.2.0/grid/log/bidb4/agent/crsd/oraagent_grid/oraagent_grid.log" 2014-11-07 15:08:01.278:</span>
##标红的部分报，监听检查报错，这个就跟我的问题联系上了。根据提示 查看oracle/app/11.2.0/grid/log/bidb4/agent/crsd/oraagent_grid/下的oraagent_grid日志(注意可能你要找的信息已经被刷出当前日志了，根据时间点查看相应日志。例如我需要的相关信息就在oraagent_grid.l02日志文件中)
5)查看oraagent_grid.l02日志
根据时间点，查找与"[ora.LISTENER.lsnr]"相关的日志，如下：
2014-11-07 15:03:21.143: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] <span style="color:#ff0000;">LsnrAgent::check { ##listener检查
</span>2014-11-07 15:03:21.143: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] lsnrctl status LISTENER
2014-11-07 15:03:21.143: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] getOracleHomeAttrib: oracle_home = /oracle/app/11.2.0/grid
2014-11-07 15:03:21.143: [ AGFW][2314] {2:2661:31941} Agent sending reply for: RESOURCE_START[ora.LISTENER.lsnr bidb4 1] ID 4098:431717
2014-11-07 15:03:21.144: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] getOracleHomeAttrib: oracle_home = /oracle/app/11.2.0/grid
2014-11-07 15:03:21.145: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] Utils::getCrsHome crsHome /oracle/app/11.2.0/grid
2014-11-07 15:03:21.145: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] Utils::execCmd 1 USR_ORA_ENV:ORACLE_BASE=/oracle/app/grid oracleHome:/oracle/app/11.2.0/grid
CrsHome:/oracle/app/11.2.0/grid
2014-11-07 15:03:21.145: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] Utils::getCrsHome crsHome /oracle/app/11.2.0/grid
2014-11-07 15:03:21.145: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] Utils::getCrsHome crsHome /oracle/app/11.2.0/grid
2014-11-07 15:03:21.145: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] Adding Environment Variables ORACLE_HOME=/oracle/app/11.2.0/grid
2014-11-07 15:03:21.145: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] Adding Environment Variables TNS_ADMIN=/oracle/app/11.2.0/grid/network/admin/
2014-11-07 15:03:21.146: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] Adding Environment variable from USR_ORA_ENV ORACLE_BASE=/oracle/app/grid
2014-11-07 15:03:21.146: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] Utils:execCmd action = 3 flags = 38 ohome = (null) cmdname = lsnrctl.
2014-11-07 15:03:21.146: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] getOracleHomeAttrib: oracle_home = /oracle/app/11.2.0/grid
2014-11-07 15:03:21.265: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] (:CLSN00010:)
2014-11-07 15:03:21.266: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] (:CLSN00010:)LSNRCTL for IBM/AIX RISC System/6000: Version 11.2.0.3.0 - Production on 07-NOV-
2014 15:03:21
2014-11-07 15:03:21.266: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] (:CLSN00010:)
2014-11-07 15:03:21.266: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] (:CLSN00010:)Copyright (c) 1991, 2011, Oracle. All rights reserved.
2014-11-07 15:03:21.266: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] (:CLSN00010:)
2014-11-07 15:03:21.266: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] (:CLSN00010:)Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
2014-11-07 15:03:21.266: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] (:CLSN00010:)<span style="color:#ff0000;">TNS-01169: The listener has not recognized the password ##检查报错(如果用过监听密码的人，因该对<span style="color:#ff0000;">TNS-01169</span>不会陌生，如果对监听器设置了密码，对监听进行操作时，需要输入密码)
</span>2014-11-07 15:03:21.266: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] (:CLSN00010:)
2014-11-07 15:03:21.267: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] (:CLSN00010:)Utils:execCmd scls_process_join() uret 1
2014-11-07 15:03:21.267: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] execCmd ret = 1
2014-11-07 15:03:21.267: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] (:CLSN00010:)
2014-11-07 15:03:21.267: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] (:CLSN00010:)LSNRCTL for IBM/AIX RISC System/6000: Version 11.2.0.3.0 - Production on 07-NOV-
2014 15:03:21
2014-11-07 15:03:21.267: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] (:CLSN00010:)
2014-11-07 15:03:21.267: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] (:CLSN00010:)Copyright (c) 1991, 2011, Oracle. All rights reserved.
2014-11-07 15:03:21.267: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] (:CLSN00010:)
2014-11-07 15:03:21.268: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] (:CLSN00010:)Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
2014-11-07 15:03:21.268: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] (:CLSN00010:)<span style="color:#ff0000;">TNS-01169: The listener has not recognized the password ##检查报错
</span>2014-11-07 15:03:21.268: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] (:CLSN00010:)
2014-11-07 15:03:21.268: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] return val (clsagfwret) = 1
2014-11-07 15:03:21.268: [ora.LISTENER.lsnr][4888] {2:2661:31941} [check] LsnrAgent::check }
2014-11-07 15:03:21.269: [ AGFW][2314] {2:2661:31941} <span style="color:#ff0000;">ora.LISTENER.lsnr bidb4 1 state changed from: STARTING to: OFFLINE ##检查失败后将"<span style="color:#ff0000;">ora.LISTENER.lsnr</span>"资源状态改成<span style="color:#ff0000;">OFFLINE </span></span>
三：问题原因
通过上面的分析，可以发现是因为为监听设置了密码，然后导致CRS在检查监听状态的时候报错(TNS-01169)，然后CRS将ora.LISTENER.lsnr资源状态修改为OFFLINE
四：解决方案
目前还没有找到解决方案，找到方案后会更新
##注：虽然我们看到crs资源状态为offline，但是其实监听运行正常，不会影响业务。
oracle crs中监听资源状态异常(ora.LISTENER.lsnr)
标签：ora.listener.lsnr spawned by agent lsnragentcheck
原文地址：http://blog.csdn.net/shaochenshuo/article/details/41007969
