在知识库中专注阅读，并随时返回相关工具与课程
今天有一套测试 Oracle 11GR2 RAC 数据库挂了，本文记录分析过程以及解决方案。 检查数据库 alert 日志： bash ORA-63999: data file suffered media failure ORA-01114: IO error writing block to file 3 block 7667 ORA-01110: data file 3: '+DATA/
系统学习 Oracle DBA
开通会员 ¥128/月起
查看完整课程本章目标：掌握本章核心知识点 前置要求：完成前序章节学习 预计时长：60 分钟
今天有一套测试 Oracle 11GR2 RAC 数据库挂了，本文记录分析过程以及解决方案。
检查数据库 alert 日志：
1ORA-63999: data file suffered media failure2ORA-01114: IO error writing block to file 3 (block # 7667)3ORA-01110: data file 3: '+DATA/btdbt/datafile/undotbs1.258.1101901997'4ORA-15055: unable to connect to ASM instance5ORA-15055: unable to connect to ASM instance6ORA-15055: unable to connect to ASM instance7ORA-00600: internal error code, arguments: [ORA_NPI_ERROR], [603], [ORA-00603: ORACLE server session terminated by fatal error8ORA-27504: IPC error creating OSD context9ORA-27300: OS system dependent operation:sendmsg failed with status: 10510ORA-27301: OS failure message: No buffer space available11ORA-27302: failure occurred at: sskgxpsnd212], [], [], [], [], [], [], [], [], []13DBW0 (ospid: 10620): terminating the instance due to error 63999这个错误已经很眼熟了，遇到过很多次：
需要调整 MTU 和 vm.min_free_kbytes：
1ifconfig lo mtu 164362echo "MTU=16436" >> /etc/sysconfig/network-scripts/ifcfg-lo3service network restart45# min_free_kbytes = os_memory_total * 0.0046os_memory_total=$(awk '/MemTotal/{print $2}' /proc/meminfo)7((min_free_kbytes = os_memory_total / 250))8echo "vm.min_free_kbytes=$min_free_kbytes " >> /etc/sysctl.conf9sysctl -p因为在数据库日志中看到有连接不上 ASM 实例的提示，需要再看下 asm 日志：
1KSXP IPC protocol is incompatible with instance 22Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_lmon_18227.trc:3ORA-27300: OS system dependent operation:config_check failed with status: 04ORA-27301: OS failure message: Error 05ORA-27302: failure occurred at: skgxpvalpid6ORA-27303: additional information: Remote port MTU does not match local MTU. [local: 9000, remote: 1500] (169.254.213.94)检查两个节点的心跳网卡 MTU 值：
1## 节点一2eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc pfifo_fast state UP group default qlen 100034## 节点二5eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000修改节点二的 MTU 值为 9000：
1ifconfig eth1 mtu 90002echo "MTU=9000" >> /etc/sysconfig/network-scripts/ifcfg-eth1重启 crs 生效 MTU：
1crsctl stop crs2crsctl start crs问题解决。
参考 MOS 文档：