"crsctl check crs" command hangs at EVMD check

时间：2014-07-04 08:16:00 阅读：335 评论：0 收藏：0 [点我收藏+]

Pre-11gR2: "crsctl check crs" command hangs at EVMD check (文档 ID 1578875.1)

APPLIES TO:

Oracle Database - Enterprise Edition - Version 10.2.0.3 to 11.1.0.7 [Release 10.2 to 11.1]
Information in this document applies to any platform.
SYMPTOMS

In a 2 node RAC environment, with 11.1.0.7 CRS, execution of the command "crsctl check crs" hangs at EVMD check only in Node 1

[oracle@srv03401 bin]$ ./crsctl check crs
Cluster Synchronization Services appears healthy
Cluster Ready Services appears healthy

From Node1, below is the output of strace for the command "crsctl check crs"

# strace -f -t -o /tmp/crschk.trc crsctl check crs

Content of the generated output file :/tmp/crschk.trc is as follows:

28268 11:47:03 execve("./crsctl", ["./crsctl", "check", "crs"], [/* 23 vars */]) = 0
28268 11:47:03 brk(0)                = 0x193d2000
28268 11:47:03 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b35b9436000
28268 11:47:03 uname({sys="Linux", node="srv03401.metra.com", ...}) = 0
28268 11:47:03 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
28268 11:47:03 open("/etc/ld.so.cache", O_RDONLY) = 3
28268 11:47:03 fstat(3, {st_mode=S_IFREG|0644, st_size=92563, ...}) = 0
28268 11:47:03 mmap(NULL, 92563, PROT_READ, MAP_PRIVATE, 3, 0) = 0x2b35b9437000
28268 11:47:03 close(3)                = 0
28268 11:47:03 open("/lib64/libtermcap.so.2", O_RDONLY) = 3
28268 11:47:03 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0`\20\300z2\0\0\0"..., 832) = 832
28268 11:47:03 fstat(3, {st_mode=S_IFREG|0755, st_size=15840, ...}) = 0
28268 11:47:03 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2b35b944e000
28268 11:47:03 mmap(0x327ac00000, 2108944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x327ac00000
28268 11:47:03 mprotect(0x327ac03000, 2093056, PROT_NONE) = 0
28268 11:47:03 mmap(0x327ae02000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x327ae02000
28268 11:47:03 close(3)                = 0
28268 11:47:03 open("/lib64/libdl.so.2", O_RDONLY) = 3
..
..
28268 11:47:03 close(3)                = 0
28268 11:47:03 write(1, "Cluster Ready Services appears h"..., 39) = 39
28268 11:47:03 socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = 3
28268 11:47:03 setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
28268 11:47:03 bind(3, {sa_family=AF_INET6, sin6_port=htons(0), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, 28) = 0
28268 11:47:03 getsockname(3, {sa_family=AF_INET6, sin6_port=htons(42027), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_flowinfo=0, sin6_scope_id=0}, [140733193388060]) = 0
28268 11:47:03 getpeername(3, 0x7fff5f19e1e0, [140733193388060]) = -1 ENOTCONN (Transport endpoint is not connected)
28268 11:47:03 getsockopt(3, SOL_SOCKET, SO_SNDBUF, [5536382933839118336], [4]) = 0
28268 11:47:03 getsockopt(3, SOL_SOCKET, SO_RCVBUF, [5536382933843050496], [4]) = 0
28268 11:47:03 fcntl(3, F_SETFD, FD_CLOEXEC) = 0
28268 11:47:03 fcntl(3, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
28268 11:47:03 geteuid()             = 700
28268 11:47:03 times({tms_utime=1, tms_stime=2, tms_cutime=0, tms_cstime=0}) = 7422615891
28268 11:47:03 socket(PF_FILE, SOCK_STREAM, 0) = 4
28268 11:47:03 access("/var/tmp/.oracle/sSYSTEM.evm.acceptor.auth", F_OK) = 0
28268 11:47:03 connect(4, {sa_family=AF_FILE, path="/var/tmp/.oracle/sSYSTEM.evm.acceptor.auth"...}, 110


CAUSE

Analysing  the strace output, looks like it was trying to write to a socket.

========
28268 11:47:03 socket(PF_FILE, SOCK_STREAM, 0) = 4
28268 11:47:03 access("/var/tmp/.oracle/sSYSTEM.evm.acceptor.auth", F_OK) = 0
28268 11:47:03 connect(4, {sa_family=AF_FILE, path="/var/tmp/.oracle/sSYSTEM.evm.acceptor.auth"...}, 110 <<<<<<<
========
This, indicates a problem with the network socket file.

SOLUTION

Get the PID of evmd.bin process and kill it

$ ps -ef | grep ‘d.bin‘

oracle 21046 21045  0  2012 ?       00:07:46 /u01/app/ract/crs/bin/evmd.bin
root    21054 15845  0  2012 ?       11:34:47 /u01/app/ract/crs/bin/crsd.bin reboot
oracle 22072 21453  0  2012 ?       05:44:50 /u01/app/ract/crs/bin/ocssd.bin
root    22135    1  0  2012 ?       00:00:00 /u01/app/ract/crs/bin/oclskd.bin
oracle 22410    1  0  2012 ?       00:00:00 /u01/app/ract/crs/bin/oclskd.bin
oracle 29834 27854  0 13:22 pts/8 00:00:00 egrep d.bin

$ kill -9 21046

After killing evmd.bin process, the command "crsctl check crs" returns the complete output without any hangs.

[oracle@srv03401 bin]$ ./crsctl check crs

CSS appears healthy
CRS appears healthy
EVM appears healthy

"crsctl check crs" command hangs at EVMD check,布布扣,bubuko.com

标签：style os linux for io div

原文地址：http://blog.csdn.net/lixora/article/details/36721083

踩

(0)

评论一句话评论（0）

分享档案

更多>

2021年07月29日 (22)
2021年07月28日 (40)
2021年07月27日 (32)
2021年07月26日 (79)
2021年07月23日 (29)
2021年07月22日 (30)
2021年07月21日 (42)
2021年07月20日 (16)
2021年07月19日 (90)
2021年07月16日 (35)

周排行