19C RAC跨网段修改IP
前几天客户有套19C的RAC需求调整，需要将原先的地址跨网段修改IP地址，包括public_ip，vip，scan_ip等。之前在11g版本也多次实施过网络调整，所以一开始也就完全按照11g的过程来调整IP地址，经过实战后发现还是和11g的版本在个别命令上有所区别。那么今天我就在虚拟机环境上，把19C的网络地址修改实验过程做一个完整的记录，供大家学习参考??
实验环境：
[root@db1:/root]$ more /etc/redhat-release Red Hat Enterprise Linux
Server release 7.6 (Maipo)
sys@ORCL 14:52:59> select BANNER from v$version;
BANNER
————————————————————————————
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
| 节点名 | public | vip | private |
|---|---|---|---|
| db1 | 192.168.56.11 | 192.168.56.13 | 10.10.10.1 |
| db2 | 192.168.56.12 | 192.168.56.14 | 10.10.10.2 |
注：由于是虚拟机环境，为了修改网段时能正常连接上虚拟机，故在db2上添加了一个单独网卡192.168.105.12方便连接操作
集群状态检查
[grid@db2:/home/grid]$ crsctl status res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
ONLINE ONLINE db1 STABLE
ONLINE ONLINE db2 STABLE
ora.chad
ONLINE ONLINE db1 STABLE
ONLINE ONLINE db2 STABLE
ora.net1.network
ONLINE ONLINE db1 STABLE
ONLINE ONLINE db2 STABLE
ora.ons
ONLINE ONLINE db1 STABLE
ONLINE ONLINE db2 STABLE
ora.proxy_advm
OFFLINE OFFLINE db1 STABLE
OFFLINE OFFLINE db2 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
1 ONLINE ONLINE db1 STABLE
2 ONLINE ONLINE db2 STABLE
ora.DATA.dg(ora.asmgroup)
1 ONLINE ONLINE db1 STABLE
2 ONLINE ONLINE db2 STABLE
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE db1 STABLE
ora.OCR.dg(ora.asmgroup)
1 ONLINE ONLINE db1 STABLE
2 ONLINE ONLINE db2 STABLE
ora.asm(ora.asmgroup)
1 ONLINE ONLINE db1 Started,STABLE
2 ONLINE ONLINE db2 Started,STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
1 ONLINE ONLINE db1 STABLE
2 ONLINE ONLINE db2 STABLE
ora.cvu
1 ONLINE ONLINE db1 STABLE
ora.db1.vip
1 ONLINE ONLINE db1 STABLE
ora.db2.vip
1 ONLINE ONLINE db2 STABLE
ora.orcl.db
1 ONLINE ONLINE db1 Open,HOME=/u01/app/o
racle/product/19.3.0
/db,STABLE
2 ONLINE ONLINE db2 Open,HOME=/u01/app/o
racle/product/19.3.0
/db,STABLE
ora.qosmserver
1 ONLINE ONLINE db1 STABLE
ora.scan1.vip
1 ONLINE ONLINE db1 STABLE
--------------------------------------------------------------------------------
禁止所有节点自动启动并关闭两边节点数据库、监听
[grid@db2:/home/grid]$ srvctl disable listener
[grid@db2:/home/grid]$ srvctl stop listener
[oracle@db2:/home/oracle]$ srvctl disable database -d orcl
[oracle@db2:/home/oracle]$ srvctl stop database -d orcl -o immediate
禁止所有节点自动启动并停止VIP
[root@db2:/root]$ /u01/app/19.3.0/grid/bin/srvctl disable vip -i db1-vip
[root@db2:/root]$ /u01/app/19.3.0/grid/bin/srvctl disable vip -i db2-vip
[grid@db2:/home/grid]$ srvctl stop vip -n db1
[grid@db2:/home/grid]$ srvctl stop vip -n db2
禁止所有节点的SCAN和SCAN_LISTENER的启动，停止所有节点的SCAN和SCAN_LISTENER
[grid@db2:/home/grid]$ srvctl disable scan_listener
[grid@db2:/home/grid]$ srvctl stop scan_listener
[root@db2:/root]$ /u01/app/19.3.0/grid/bin/srvctl disable scan
[root@db2:/root]$ /u01/app/19.3.0/grid/bin/srvctl stop scan
停止crs
[root@db2:/root]$ /u01/app/19.3.0/grid/bin/crsctl stop crs
[root@db1:/root]$ /u01/app/19.3.0/grid/bin/crsctl stop crs
修改两边的/etc/hosts文件
[root@db1:/root]$ more /etc/hosts
127.0.0.1 localhost localhost.localdomain
##OracleBegin
##Public IP
192.168.156.11 db1
192.168.156.12 db2
##Private IP
10.10.10.1 db1-priv
10.10.10.2 db2-priv
##Virtual IP
192.168.156.13 db1-vip
192.168.156.14 db2-vip
##Scan IP
192.168.156.15 db-scan
[root@db2:/root]$ more /etc/hosts
127.0.0.1 localhost localhost.localdomain
##OracleBegin
##Public IP
192.168.156.11 db1
192.168.156.12 db2
##Private IP
10.10.10.1 db1-priv
10.10.10.2 db2-priv
##Virtual IP
192.168.156.13 db1-vip
192.168.156.14 db2-vip
##Scan IP
192.168.156.15 db-scan
修改enp0s3网卡配置
[root@db1:/root]$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:9c:01:ce brd ff:ff:ff:ff:ff:ff
inet 192.168.156.11/24 brd 192.168.156.255 scope global noprefixroute enp0s3
valid_lft forever preferred_lft forever
inet6 fe80::4374:73e0:bd3:a5b6/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:8d:49:d6 brd ff:ff:ff:ff:ff:ff
inet 10.10.10.1/24 brd 10.10.10.255 scope global noprefixroute enp0s8
valid_lft forever preferred_lft forever
inet6 fe80::edac:5de7:4a37:d0a1/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@db2:/root]$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:a1:65:f6 brd ff:ff:ff:ff:ff:ff
inet 192.168.156.12/24 brd 192.168.156.255 scope global noprefixroute enp0s3
valid_lft forever preferred_lft forever
inet6 fe80::a547:6355:2009:2f24/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:3c:1c:4b brd ff:ff:ff:ff:ff:ff
inet 10.10.10.2/24 brd 10.10.10.255 scope global noprefixroute enp0s8
valid_lft forever preferred_lft forever
inet6 fe80::5cd6:7a9e:1f08:ef39/64 scope link noprefixroute
valid_lft forever preferred_lft forever
4: enp0s9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:46:0b:2b brd ff:ff:ff:ff:ff:ff
inet 192.168.105.12/24 brd 192.168.105.255 scope global noprefixroute enp0s9
valid_lft forever preferred_lft forever
inet6 fe80::a00:27ff:fe46:b2b/64 scope link
valid_lft forever preferred_lft forever
重启网络服务
[root@db1:/root]$ systemctl restart network
[root@db2:/root]$ systemctl restart network
启动crs
[root@db1:/root]$ /u01/app/19.3.0/grid/bin/crsctl start crs
[root@db2:/root]$ /u01/app/19.3.0/grid/bin/crsctl start crs
集群状态
[grid@db2:/home/grid]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
OFFLINE OFFLINE db1 STABLE
OFFLINE OFFLINE db2 STABLE
ora.chad
ONLINE ONLINE db1 STABLE
ONLINE ONLINE db2 STABLE
ora.net1.network
ONLINE OFFLINE db1 STABLE
ONLINE OFFLINE db2 STABLE
ora.ons
ONLINE OFFLINE db1 STABLE
ONLINE OFFLINE db2 STABLE
ora.proxy_advm
OFFLINE OFFLINE db1 STABLE
OFFLINE OFFLINE db2 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
1 ONLINE ONLINE db1 STABLE
2 ONLINE ONLINE db2 STABLE
ora.DATA.dg(ora.asmgroup)
1 ONLINE ONLINE db1 STABLE
2 ONLINE ONLINE db2 STABLE
ora.LISTENER_SCAN1.lsnr
1 OFFLINE OFFLINE STABLE
ora.OCR.dg(ora.asmgroup)
1 ONLINE ONLINE db1 STABLE
2 ONLINE ONLINE db2 STABLE
ora.asm(ora.asmgroup)
1 ONLINE ONLINE db1 Started,STABLE
2 ONLINE ONLINE db2 Started,STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
1 ONLINE ONLINE db1 STABLE
2 ONLINE ONLINE db2 STABLE
ora.cvu
1 ONLINE OFFLINE STABLE
ora.db1.vip
1 OFFLINE OFFLINE STABLE
ora.db2.vip
1 OFFLINE OFFLINE STABLE
ora.orcl.db
1 OFFLINE OFFLINE STABLE
2 OFFLINE OFFLINE STABLE
ora.qosmserver
1 ONLINE OFFLINE STABLE
ora.scan1.vip
1 OFFLINE OFFLINE STABLE
--------------------------------------------------------------------------------
#这里可以看到，监听和vip之类的服务都未启动
修改集群的public IP
[root@db1:/u01/app/19.3.0/grid/bin]$ ./oifcfg getif // 注意命令路径
enp0s3 192.168.56.0 global public
enp0s8 10.10.10.0 global cluster_interconnect,asm
[root@db2:/u01/app/19.3.0/grid/bin]$ ./oifcfg getif
enp0s3 192.168.56.0 global public
enp0s8 10.10.10.0 global cluster_interconnect,asm
#现在显示集群中的IP还是192.168.56.0段的地址
[root@db1:/u01/app/19.3.0/grid/bin]$ ./oifcfg delif -global enp0s3
[root@db1:/u01/app/19.3.0/grid/bin]$ ./oifcfg setif -global enp0s3/192.168.156.0:public
[root@db1:/u01/app/19.3.0/grid/bin]$ ./oifcfg getif
enp0s8 10.10.10.0 global cluster_interconnect,asm
enp0s3 192.168.156.0 global public
[root@db2:/u01/app/19.3.0/grid/bin]$ ./oifcfg getif
enp0s8 10.10.10.0 global cluster_interconnect,asm
enp0s3 192.168.156.0 global public
#现在显示集群中的IP已经修改成192.168.156.0段的地址
修改集群的VIP和SCAN_IP
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl config vip -n db1
VIP exists: network number 1, hosting node db1
VIP Name: db1-vip
VIP IPv4 Address: 192.168.156.13
VIP IPv6 Address:
VIP is disabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl config vip -n db2
VIP exists: network number 1, hosting node db2
VIP Name: db2-vip
VIP IPv4 Address: 192.168.156.14
VIP IPv6 Address:
VIP is disabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
#此时还没正式修改VIP，但是查看vip配置时集群显示已经是192.168.156.0段的地址
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl config nodeapps -a
Network 1 exists
Subnet IPv4: 192.168.56.0/255.255.255.0/enp0s3, static
Subnet IPv6:
Ping Targets:
Network is enabled
Network is individually enabled on nodes:
Network is individually disabled on nodes:
VIP exists: network number 1, hosting node db1
VIP Name: db1-vip
VIP IPv4 Address: 192.168.156.13
VIP IPv6 Address:
VIP is disabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
VIP exists: network number 1, hosting node db2
VIP Name: db2-vip
VIP IPv4 Address: 192.168.156.14
VIP IPv6 Address:
VIP is disabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
#查看nodeapps服务内容，你会发现虽然vip地址已经自动修改了，但是vip的地址段还是192.168.56.0。
❗️ 如果只检查了vip配置，可能会误认为此时不需要modify VIP。我们先将错就错，看看后面会出什么问题。现在开始修改SCANIP?
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl config scan
SCAN name: db-scan, Network: 1
Subnet IPv4: 192.168.56.0/255.255.255.0/enp0s3, static
Subnet IPv6:
SCAN 1 IPv4 VIP: 192.168.56.15
SCAN VIP is disabled.
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl modify scan -n 192.168.156.15
PRCS-1034 : Failed to modify Single Client Access Name 192.168.156.15
PRCS-1076 : Invalid Single Client Access Name 192.168.156.15. The subnet 192.168.156.0 of SCAN VIP 192.168.156.15 differs from the subnet 192.168.56.0 of network 1
#根据报错信息可以很明显的知道需要求改的scanip与nodeapps中的网段信息不一致。所以我们还是需要手工修改nodeapps中的网段信息
修改VIP
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl modify nodeapps -n db1 -A 192.168.156.13/255.255.255.0/enp0s3
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl modify nodeapps -n db2 -A 192.168.156.14/255.255.255.0/enp0s3
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl config nodeapps -a
Network 1 exists
Subnet IPv4: 192.168.156.0/255.255.255.0/enp0s3, static
Subnet IPv6:
Ping Targets:
Network is enabled
Network is individually enabled on nodes:
Network is individually disabled on nodes:
VIP exists: network number 1, hosting node db1
VIP Name: db1-vip
VIP IPv4 Address: 192.168.156.13
VIP IPv6 Address:
VIP is disabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
VIP exists: network number 1, hosting node db2
VIP Name: db2-vip
VIP IPv4 Address: 192.168.156.14
VIP IPv6 Address:
VIP is disabled.
VIP is individually enabled on nodes:
VIP is individually disabled on nodes:
#可以看到此时nodeapps的网段信息已经修改成功，下面修改scanIP
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl modify scan -n db-scan //这里可以使用水scan_name也可以直接使用IP地址
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl config scan
SCAN name: db-scan, Network: 1
Subnet IPv4: 192.168.156.0/255.255.255.0/enp0s3, static
Subnet IPv6:
SCAN 1 IPv4 VIP: 192.168.156.15
SCAN VIP is disabled.
启动监听和各服务
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl enable listener
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl enable vip -i db1-vip
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl enable vip -i db2-vip
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl enable scan_listener
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl enable scan
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl enable database -d orcl
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl start listener
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl start vip -n db1
PRKO-2420 : VIP db1-vip is already started on nodes: db1
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl start scan
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl start scan_listener
[root@db1:/u01/app/19.3.0/grid/bin]$ ./srvctl start database -d orcl
#集群状态
[grid@db2:/home/grid]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
ONLINE ONLINE db1 STABLE
ONLINE ONLINE db2 STABLE
ora.chad
ONLINE ONLINE db1 STABLE
ONLINE ONLINE db2 STABLE
ora.net1.network
ONLINE ONLINE db1 STABLE
ONLINE ONLINE db2 STABLE
ora.ons
ONLINE ONLINE db1 STABLE
ONLINE ONLINE db2 STABLE
ora.proxy_advm
OFFLINE OFFLINE db1 STABLE
OFFLINE OFFLINE db2 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
1 ONLINE ONLINE db1 STABLE
2 ONLINE ONLINE db2 STABLE
ora.DATA.dg(ora.asmgroup)
1 ONLINE ONLINE db1 STABLE
2 ONLINE ONLINE db2 STABLE
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE db2 STABLE
ora.OCR.dg(ora.asmgroup)
1 ONLINE ONLINE db1 STABLE
2 ONLINE ONLINE db2 STABLE
ora.asm(ora.asmgroup)
1 ONLINE ONLINE db1 Started,STABLE
2 ONLINE ONLINE db2 Started,STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
1 ONLINE ONLINE db1 STABLE
2 ONLINE ONLINE db2 STABLE
ora.cvu
1 ONLINE ONLINE db1 STABLE
ora.db1.vip
1 ONLINE ONLINE db1 STABLE
ora.db2.vip
1 ONLINE ONLINE db2 STABLE
ora.orcl.db
1 ONLINE ONLINE db1 Open,HOME=/u01/app/o
racle/product/19.3.0
/db,STABLE
2 ONLINE ONLINE db2 Open,HOME=/u01/app/o
racle/product/19.3.0
/db,STABLE
ora.qosmserver
1 ONLINE ONLINE db1 STABLE
ora.scan1.vip
1 ONLINE ONLINE db2 STABLE
--------------------------------------------------------------------------------
至此所有IP修改完毕，测试连接数据库scan_ip
本次实验中是修改public vip scanip，所以停止了所有的监听和IP服务。如果在你的实际环境中只是单独修改public，或者是vip，亦或是SCANIP，可以只停止相应的监听和服务，然后完成后续的配置修改☑️