Sudipta Bhaskar
May 2, 202123 min
Updated: May 3, 2021
I Searched over the internet and couldn't find many post on RAC Grid Upgrade in silent mode. That's why I thought of writing this post.
Right now I have a 12.2 RAC Setup and I will upgrade it to 19.8.
Following are the network configuration.
# Public
192.168.126.11 OEL7N1.localdomain OEL7N1
192.168.126.21 OEL7N2.localdomain OEL7N2
# Private
192.168.100.11 OEL7N1-priv.localdomain OEL7N1-priv
192.168.100.21 OEL7N2-priv.localdomain OEL7N2-priv
# Virtual
192.168.126.31 OEL7N1-vip.localdomain OEL7N1-vip
192.168.126.41 OEL7N2-vip.localdomain OEL7N2-vip
# SCAN
#192.168.126.50 scan.localdomain scan
#192.168.126.60 scan.localdomain scan
#192.168.126.70 scan.localdomain scan
I have created directory structure /grid/app/oracle/19.3/grid in both nodes. This will serve as the Grid Home for 19c.
Downloaded 19.3 Grid Binary from oracle website.
Let' s copy LINUX.X64_193000_grid_home.zip to the mentioned location in node 1 only.
sudip@DESKTOP-RDIUEUO MINGW64 /d/Softwares
$ scp LINUX.X64_193000_grid_home.zip oracle@192.168.126.11:/grid/app/oracle/19.3/grid
oracle@192.168.126.11's password:
LINUX.X64_193000_grid_home.zip 100% 2755MB 28.6MB/s 01:36
Let's unzip the binary.
[oracle@OEL7N1 grid]$ pwd
/grid/app/oracle/19.3/grid
[oracle@OEL7N1 grid]$ ls -ltr
total 2821472
-rw-r--r-- 1 oracle oinstall 2889184573 May 2 13:25 LINUX.X64_193000_grid_home.zip
[oracle@OEL7N1 grid]$ unzip LINUX.X64_193000_grid_home.zip
This is 19.3 binary. We need to take it to 19.8.0.0.200714. We will patch this binary with the PSU.
For 19.8.0.0.200714, we need patch number 31305339. I have downloaded and copied it to /dboracle/software. Unzipped it after copying. Let's apply it.
Before that installed latest OPatch.
[oracle@OEL7N1 OPatch]$ unzip -d /grid/app/oracle/19.3/grid /dboracle/software/p6880880_190000_Linux-x86-64.zip
Archive: /dboracle/software/p6880880_190000_Linux-x86-64.zip
replace /grid/app/oracle/19.3/grid/OPatch/emdpatch.pl? [y]es, [n]o, [A]ll, [N]one, [r]ename: A
inflating: /grid/app/oracle/19.3/grid/OPatch/emdpatch.pl
[oracle@OEL7N1 OPatch]$ ./opatch version
OPatch Version: 12.2.0.1.24
OPatch succeeded.
[oracle@OEL7N1 grid]$ ./gridSetup.sh -ignorePrereq -waitforcompletion -silent -applyRU /dboracle/software/31305339
Preparing the home to patch...
Applying the patch /dboracle/software/31305339...
Successfully applied the patch.
The log can be found at: /u01/app/oraInventory/logs/GridSetupActions2021-05-02_02-18-58PM/installerPatchActions_2021-05-02_02-18-58PM.log
Launching Oracle Grid Infrastructure Setup Wizard...
[FATAL] [INS-40426] Grid installation option has not been specified.
ACTION: Specify the valid installation option.
Note : This error is because it was trying to install grid 19c but our intention here is only to apply the latest RU.
/grid/app/oracle/19.3/grid
[oracle@OEL7N1 grid]$
[oracle@OEL7N1 grid]$ ./gridSetup.sh -ignorePrereq -waitforcompletion -silent -applyRU /dboracle/software/31305339
Preparing the home to patch...
Applying the patch /dboracle/software/31305339...
Successfully applied the patch.
The log can be found at: /u01/app/oraInventory/logs/GridSetupActions2021-05-02_02-54-04PM/installerPatchActions_2021-05-02_02-54-04PM.log
Launching Oracle Grid Infrastructure Setup Wizard...
^C[oracle@OEL7N1 grid]$
Let's run the pre checks. Run it from 19c home.
[oracle@OEL7N1 grid]$
[oracle@OEL7N1 grid]$ /grid/app/oracle/19.3/grid/runcluvfy.sh stage -pre crsinst -upgrade -rolling -src_crshome /grid/app/oracle/12.2 -dest_crshome /grid/app/oracle/19.3/grid -dest_version 19.8.0.0.200714 -fixupnoexec -verbose
Verifying Physical Memory ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 4.826GB (5060456.0KB) 8GB (8388608.0KB) failed
oel7n1 4.826GB (5060472.0KB) 8GB (8388608.0KB) failed
Verifying Physical Memory ...FAILED (PRVF-7530)
Verifying Available Physical Memory ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 2.4753GB (2595580.0KB) 50MB (51200.0KB) passed
oel7n1 3.1558GB (3309116.0KB) 50MB (51200.0KB) passed
Verifying Available Physical Memory ...PASSED
Verifying Swap Size ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 10GB (1.0485756E7KB) 4.826GB (5060456.0KB) passed
oel7n1 10GB (1.0485756E7KB) 4.826GB (5060472.0KB) passed
Verifying Swap Size ...PASSED
Verifying Free Space: oel7n2:/usr,oel7n2:/var,oel7n2:/etc,oel7n2:/sbin,oel7n2:/tmp ...
Path Node Name Mount point Available Required Status
---------------- ------------ ------------ ------------ ------------ ------------
/usr oel7n2 / 15.7295GB 25MB passed
/var oel7n2 / 15.7295GB 5MB passed
/etc oel7n2 / 15.7295GB 25MB passed
/sbin oel7n2 / 15.7295GB 10MB passed
/tmp oel7n2 / 15.7295GB 1GB passed
Verifying Free Space: oel7n2:/usr,oel7n2:/var,oel7n2:/etc,oel7n2:/sbin,oel7n2:/tmp ...PASSED
Verifying Free Space: oel7n1:/usr,oel7n1:/var,oel7n1:/etc,oel7n1:/sbin,oel7n1:/tmp ...
Path Node Name Mount point Available Required Status
---------------- ------------ ------------ ------------ ------------ ------------
/usr oel7n1 / 15.8174GB 25MB passed
/var oel7n1 / 15.8174GB 5MB passed
/etc oel7n1 / 15.8174GB 25MB passed
/sbin oel7n1 / 15.8174GB 10MB passed
/tmp oel7n1 / 15.8174GB 1GB passed
Verifying Free Space: oel7n1:/usr,oel7n1:/var,oel7n1:/etc,oel7n1:/sbin,oel7n1:/tmp ...PASSED
Verifying User Existence: oracle ...
Node Name Status Comment
------------ ------------------------ ------------------------
oel7n2 passed exists(54321)
oel7n1 passed exists(54321)
Verifying Users With Same UID: 54321 ...PASSED
Verifying User Existence: oracle ...PASSED
Verifying Group Existence: asmadmin ...
Node Name Status Comment
------------ ------------------------ ------------------------
oel7n2 passed exists
oel7n1 passed exists
Verifying Group Existence: asmadmin ...PASSED
Verifying Group Existence: asmdba ...
Node Name Status Comment
------------ ------------------------ ------------------------
oel7n2 passed exists
oel7n1 passed exists
Verifying Group Existence: asmdba ...PASSED
Verifying Group Existence: oinstall ...
Node Name Status Comment
------------ ------------------------ ------------------------
oel7n2 passed exists
oel7n1 passed exists
Verifying Group Existence: oinstall ...PASSED
Verifying Group Membership: asmadmin ...
Node Name User Exists Group Exists User in Group Status
---------------- ------------ ------------ ------------ ----------------
oel7n2 yes yes yes passed
oel7n1 yes yes yes passed
Verifying Group Membership: asmadmin ...PASSED
Verifying Group Membership: asmdba ...
Node Name User Exists Group Exists User in Group Status
---------------- ------------ ------------ ------------ ----------------
oel7n2 yes yes yes passed
oel7n1 yes yes yes passed
Verifying Group Membership: asmdba ...PASSED
Verifying Group Membership: oinstall(Primary) ...
Node Name User Exists Group Exists User in Group Primary Status
---------------- ------------ ------------ ------------ ------------ ------------
oel7n2 yes yes yes yes passed
oel7n1 yes yes yes yes passed
Verifying Group Membership: oinstall(Primary) ...PASSED
Verifying Run Level ...
Node Name run level Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 3 3,5 passed
oel7n1 3 3,5 passed
Verifying Run Level ...PASSED
Verifying Hard Limit: maximum open file descriptors ...
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
oel7n2 hard 65536 65536 passed
oel7n1 hard 65536 65536 passed
Verifying Hard Limit: maximum open file descriptors ...PASSED
Verifying Soft Limit: maximum open file descriptors ...
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
oel7n2 soft 1024 1024 passed
oel7n1 soft 1024 1024 passed
Verifying Soft Limit: maximum open file descriptors ...PASSED
Verifying Hard Limit: maximum user processes ...
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
oel7n2 hard 16384 16384 passed
oel7n1 hard 16384 16384 passed
Verifying Hard Limit: maximum user processes ...PASSED
Verifying Soft Limit: maximum user processes ...
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
oel7n2 soft 16384 2047 passed
oel7n1 soft 16384 2047 passed
Verifying Soft Limit: maximum user processes ...PASSED
Verifying Soft Limit: maximum stack size ...
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
oel7n2 soft 10240 10240 passed
oel7n1 soft 10240 10240 passed
Verifying Soft Limit: maximum stack size ...PASSED
Verifying Oracle patch:28553832 ...PASSED
Verifying Oracle patch:27006180 ...
Node Name Applied Required Comment
------------ ------------------------ ------------------------ ----------
oel7n1 27006180 27006180 passed
oel7n2 27006180 27006180 passed
Verifying Oracle patch:27006180 ...PASSED
Verifying This test checks that the source home "/grid/app/oracle/12.2" is suitable for upgrading to version "19.8.0.0.200714". ...PASSED
Verifying Architecture ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 x86_64 x86_64 passed
oel7n1 x86_64 x86_64 passed
Verifying Architecture ...PASSED
Verifying OS Kernel Version ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 4.14.35-2047.502.4.1.el7uek.x86_64 4.1.12 passed
oel7n1 4.14.35-2047.502.4.1.el7uek.x86_64 4.1.12 passed
Verifying OS Kernel Version ...PASSED
Verifying OS Kernel Parameter: semmsl ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
oel7n1 250 250 250 passed
oel7n2 250 250 250 passed
Verifying OS Kernel Parameter: semmsl ...PASSED
Verifying OS Kernel Parameter: semmns ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
oel7n1 32000 32000 32000 passed
oel7n2 32000 32000 32000 passed
Verifying OS Kernel Parameter: semmns ...PASSED
Verifying OS Kernel Parameter: semopm ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
oel7n1 100 100 100 passed
oel7n2 100 100 100 passed
Verifying OS Kernel Parameter: semopm ...PASSED
Verifying OS Kernel Parameter: semmni ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
oel7n1 128 128 128 passed
oel7n2 128 128 128 passed
Verifying OS Kernel Parameter: semmni ...PASSED
Verifying OS Kernel Parameter: shmmax ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
oel7n1 4398046511104 4398046511104 2590961664 passed
oel7n2 4398046511104 4398046511104 2590953472 passed
Verifying OS Kernel Parameter: shmmax ...PASSED
Verifying OS Kernel Parameter: shmmni ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
oel7n1 4096 4096 4096 passed
oel7n2 4096 4096 4096 passed
Verifying OS Kernel Parameter: shmmni ...PASSED
Verifying OS Kernel Parameter: shmall ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
oel7n1 1073741824 1073741824 1073741824 passed
oel7n2 1073741824 1073741824 1073741824 passed
Verifying OS Kernel Parameter: shmall ...PASSED
Verifying OS Kernel Parameter: file-max ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
oel7n1 6815744 6815744 6815744 passed
oel7n2 6815744 6815744 6815744 passed
Verifying OS Kernel Parameter: file-max ...PASSED
Verifying OS Kernel Parameter: ip_local_port_range ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
oel7n1 between 9000 & 65500 between 9000 & 65500 between 9000 & 65535 passed
oel7n2 between 9000 & 65500 between 9000 & 65500 between 9000 & 65535 passed
Verifying OS Kernel Parameter: ip_local_port_range ...PASSED
Verifying OS Kernel Parameter: rmem_default ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
oel7n1 262144 262144 262144 passed
oel7n2 262144 262144 262144 passed
Verifying OS Kernel Parameter: rmem_default ...PASSED
Verifying OS Kernel Parameter: rmem_max ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
oel7n1 4194304 4194304 4194304 passed
oel7n2 4194304 4194304 4194304 passed
Verifying OS Kernel Parameter: rmem_max ...PASSED
Verifying OS Kernel Parameter: wmem_default ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
oel7n1 262144 262144 262144 passed
oel7n2 262144 262144 262144 passed
Verifying OS Kernel Parameter: wmem_default ...PASSED
Verifying OS Kernel Parameter: wmem_max ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
oel7n1 1048576 1048576 1048576 passed
oel7n2 1048576 1048576 1048576 passed
Verifying OS Kernel Parameter: wmem_max ...PASSED
Verifying OS Kernel Parameter: aio-max-nr ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
oel7n1 1048576 1048576 1048576 passed
oel7n2 1048576 1048576 1048576 passed
Verifying OS Kernel Parameter: aio-max-nr ...PASSED
Verifying OS Kernel Parameter: panic_on_oops ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
oel7n1 1 1 1 passed
oel7n2 1 1 1 passed
Verifying OS Kernel Parameter: panic_on_oops ...PASSED
Verifying Package: kmod-20-21 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 kmod(x86_64)-20-28.0.3.el7 kmod(x86_64)-20-21 passed
oel7n1 kmod(x86_64)-20-28.0.3.el7 kmod(x86_64)-20-21 passed
Verifying Package: kmod-20-21 (x86_64) ...PASSED
Verifying Package: kmod-libs-20-21 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 kmod-libs(x86_64)-20-28.0.3.el7 kmod-libs(x86_64)-20-21 passed
oel7n1 kmod-libs(x86_64)-20-28.0.3.el7 kmod-libs(x86_64)-20-21 passed
Verifying Package: kmod-libs-20-21 (x86_64) ...PASSED
Verifying Package: binutils-2.23.52.0.1 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 binutils-2.27-44.base.0.3.el7 binutils-2.23.52.0.1 passed
oel7n1 binutils-2.27-44.base.0.3.el7 binutils-2.23.52.0.1 passed
Verifying Package: binutils-2.23.52.0.1 ...PASSED
Verifying Package: compat-libcap1-1.10 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 compat-libcap1-1.10-7.el7 compat-libcap1-1.10 passed
oel7n1 compat-libcap1-1.10-7.el7 compat-libcap1-1.10 passed
Verifying Package: compat-libcap1-1.10 ...PASSED
Verifying Package: libgcc-4.8.2 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 libgcc(x86_64)-4.8.5-44.0.3.el7 libgcc(x86_64)-4.8.2 passed
oel7n1 libgcc(x86_64)-4.8.5-44.0.3.el7 libgcc(x86_64)-4.8.2 passed
Verifying Package: libgcc-4.8.2 (x86_64) ...PASSED
Verifying Package: libstdc++-4.8.2 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 libstdc++(x86_64)-4.8.5-44.0.3.el7 libstdc++(x86_64)-4.8.2 passed
oel7n1 libstdc++(x86_64)-4.8.5-44.0.3.el7 libstdc++(x86_64)-4.8.2 passed
Verifying Package: libstdc++-4.8.2 (x86_64) ...PASSED
Verifying Package: libstdc++-devel-4.8.2 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 libstdc++-devel(x86_64)-4.8.5-44.0.3.el7 libstdc++-devel(x86_64)-4.8.2 passed
oel7n1 libstdc++-devel(x86_64)-4.8.5-44.0.3.el7 libstdc++-devel(x86_64)-4.8.2 passed
Verifying Package: libstdc++-devel-4.8.2 (x86_64) ...PASSED
Verifying Package: sysstat-10.1.5 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 sysstat-10.1.5-19.el7 sysstat-10.1.5 passed
oel7n1 sysstat-10.1.5-19.el7 sysstat-10.1.5 passed
Verifying Package: sysstat-10.1.5 ...PASSED
Verifying Package: ksh ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 ksh ksh passed
oel7n1 ksh ksh passed
Verifying Package: ksh ...PASSED
Verifying Package: make-3.82 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 make-3.82-24.el7 make-3.82 passed
oel7n1 make-3.82-24.el7 make-3.82 passed
Verifying Package: make-3.82 ...PASSED
Verifying Package: glibc-2.17 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 glibc(x86_64)-2.17-323.0.1.el7_9 glibc(x86_64)-2.17 passed
oel7n1 glibc(x86_64)-2.17-323.0.1.el7_9 glibc(x86_64)-2.17 passed
Verifying Package: glibc-2.17 (x86_64) ...PASSED
Verifying Package: glibc-devel-2.17 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 glibc-devel(x86_64)-2.17-323.0.1.el7_9 glibc-devel(x86_64)-2.17 passed
oel7n1 glibc-devel(x86_64)-2.17-323.0.1.el7_9 glibc-devel(x86_64)-2.17 passed
Verifying Package: glibc-devel-2.17 (x86_64) ...PASSED
Verifying Package: libaio-0.3.109 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 libaio(x86_64)-0.3.109-13.el7 libaio(x86_64)-0.3.109 passed
oel7n1 libaio(x86_64)-0.3.109-13.el7 libaio(x86_64)-0.3.109 passed
Verifying Package: libaio-0.3.109 (x86_64) ...PASSED
Verifying Package: libaio-devel-0.3.109 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 libaio-devel(x86_64)-0.3.109-13.el7 libaio-devel(x86_64)-0.3.109 passed
oel7n1 libaio-devel(x86_64)-0.3.109-13.el7 libaio-devel(x86_64)-0.3.109 passed
Verifying Package: libaio-devel-0.3.109 (x86_64) ...PASSED
Verifying Package: libvirt-devel-4.5.0-10.0 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 libvirt-devel-4.5.0-36.el7_9.3 libvirt-devel-4.5.0-10.0 passed
oel7n1 libvirt-devel-4.5.0-36.el7_9.3 libvirt-devel-4.5.0-10.0 passed
Verifying Package: libvirt-devel-4.5.0-10.0 ...PASSED
Verifying Package: libvirt-libs-4.5.0-10.0 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 libvirt-libs-4.5.0-36.el7_9.3 libvirt-libs-4.5.0-10.0 passed
oel7n1 libvirt-libs-4.5.0-36.el7_9.3 libvirt-libs-4.5.0-10.0 passed
Verifying Package: libvirt-libs-4.5.0-10.0 ...PASSED
Verifying Package: nfs-utils-1.2.3-15 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 nfs-utils-1.3.0-0.68.0.1.el7 nfs-utils-1.2.3-15 passed
oel7n1 nfs-utils-1.3.0-0.68.0.1.el7 nfs-utils-1.2.3-15 passed
Verifying Package: nfs-utils-1.2.3-15 ...PASSED
Verifying Package: smartmontools-6.2-4 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 smartmontools-7.0-2.el7 smartmontools-6.2-4 passed
oel7n1 smartmontools-7.0-2.el7 smartmontools-6.2-4 passed
Verifying Package: smartmontools-6.2-4 ...PASSED
Verifying Package: net-tools-2.0-0.17 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 net-tools-2.0-0.25.20131004git.el7 net-tools-2.0-0.17 passed
oel7n1 net-tools-2.0-0.25.20131004git.el7 net-tools-2.0-0.17 passed
Verifying Package: net-tools-2.0-0.17 ...PASSED
Verifying Users With Same UID: 0 ...PASSED
Verifying Current Group ID ...PASSED
Verifying Root user consistency ...
Node Name Status
------------------------------------ ------------------------
oel7n2 passed
oel7n1 passed
Verifying Root user consistency ...PASSED
Verifying correctness of ASM disk group files ownership ...PASSED
Verifying selectivity of ASM discovery string ...PASSED
Verifying ASM spare parameters ...PASSED
Verifying Disk group ASM compatibility setting ...PASSED
Verifying Package: cvuqdisk-1.0.10-1 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
oel7n2 cvuqdisk-1.0.10-1 cvuqdisk-1.0.10-1 passed
oel7n1 cvuqdisk-1.0.10-1 cvuqdisk-1.0.10-1 passed
Verifying Package: cvuqdisk-1.0.10-1 ...PASSED
Verifying Host name ...PASSED
Verifying Node Connectivity ...
Verifying Hosts File ...
Node Name Status
------------------------------------ ------------------------
oel7n1 passed
oel7n2 passed
Verifying Hosts File ...PASSED
Interface information for node "oel7n2"
Name IP Address Subnet Gateway Def. Gateway HW Address MTU
------ --------------- --------------- --------------- --------------- ----------------- ------
enp0s8 192.168.126.21 192.168.126.0 0.0.0.0 192.168.100.1 08:00:27:E7:37:1D 1500
enp0s8 192.168.126.41 192.168.126.0 0.0.0.0 192.168.100.1 08:00:27:E7:37:1D 1500
enp0s8 192.168.126.50 192.168.126.0 0.0.0.0 192.168.100.1 08:00:27:E7:37:1D 1500
enp0s9 192.168.100.21 192.168.100.0 0.0.0.0 192.168.100.1 08:00:27:BE:71:77 1500
Interface information for node "oel7n1"
Name IP Address Subnet Gateway Def. Gateway HW Address MTU
------ --------------- --------------- --------------- --------------- ----------------- ------
enp0s8 192.168.126.11 192.168.126.0 0.0.0.0 192.168.100.1 08:00:27:8C:A2:5F 1500
enp0s8 192.168.126.31 192.168.126.0 0.0.0.0 192.168.100.1 08:00:27:8C:A2:5F 1500
enp0s8 192.168.126.60 192.168.126.0 0.0.0.0 192.168.100.1 08:00:27:8C:A2:5F 1500
enp0s8 192.168.126.70 192.168.126.0 0.0.0.0 192.168.100.1 08:00:27:8C:A2:5F 1500
enp0s9 192.168.100.11 192.168.100.0 0.0.0.0 192.168.100.1 08:00:27:E6:53:62 1500
Check: MTU consistency on the private interfaces of subnet "192.168.100.0"
Node Name IP Address Subnet MTU
---------------- ------------ ------------ ------------ ----------------
oel7n2 enp0s9 192.168.100.21 192.168.100.0 1500
oel7n1 enp0s9 192.168.100.11 192.168.100.0 1500
Check: MTU consistency of the subnet "192.168.126.0".
Node Name IP Address Subnet MTU
---------------- ------------ ------------ ------------ ----------------
oel7n2 enp0s8 192.168.126.21 192.168.126.0 1500
oel7n2 enp0s8 192.168.126.41 192.168.126.0 1500
oel7n2 enp0s8 192.168.126.50 192.168.126.0 1500
oel7n1 enp0s8 192.168.126.11 192.168.126.0 1500
oel7n1 enp0s8 192.168.126.31 192.168.126.0 1500
oel7n1 enp0s8 192.168.126.60 192.168.126.0 1500
oel7n1 enp0s8 192.168.126.70 192.168.126.0 1500
Source Destination Connected?
------------------------------ ------------------------------ ----------------
oel7n1[enp0s8:192.168.126.11] oel7n2[enp0s8:192.168.126.41] yes
oel7n1[enp0s8:192.168.126.11] oel7n2[enp0s8:192.168.126.50] yes
oel7n1[enp0s8:192.168.126.11] oel7n2[enp0s8:192.168.126.21] yes
oel7n1[enp0s8:192.168.126.11] oel7n1[enp0s8:192.168.126.31] yes
oel7n1[enp0s8:192.168.126.11] oel7n1[enp0s8:192.168.126.60] yes
oel7n1[enp0s8:192.168.126.11] oel7n1[enp0s8:192.168.126.70] yes
oel7n2[enp0s8:192.168.126.41] oel7n2[enp0s8:192.168.126.50] yes
oel7n2[enp0s8:192.168.126.41] oel7n2[enp0s8:192.168.126.21] yes
oel7n2[enp0s8:192.168.126.41] oel7n1[enp0s8:192.168.126.31] yes
oel7n2[enp0s8:192.168.126.41] oel7n1[enp0s8:192.168.126.60] yes
oel7n2[enp0s8:192.168.126.41] oel7n1[enp0s8:192.168.126.70] yes
oel7n2[enp0s8:192.168.126.50] oel7n2[enp0s8:192.168.126.21] yes
oel7n2[enp0s8:192.168.126.50] oel7n1[enp0s8:192.168.126.31] yes
oel7n2[enp0s8:192.168.126.50] oel7n1[enp0s8:192.168.126.60] yes
oel7n2[enp0s8:192.168.126.50] oel7n1[enp0s8:192.168.126.70] yes
oel7n2[enp0s8:192.168.126.21] oel7n1[enp0s8:192.168.126.31] yes
oel7n2[enp0s8:192.168.126.21] oel7n1[enp0s8:192.168.126.60] yes
oel7n2[enp0s8:192.168.126.21] oel7n1[enp0s8:192.168.126.70] yes
oel7n1[enp0s8:192.168.126.31] oel7n1[enp0s8:192.168.126.60] yes
oel7n1[enp0s8:192.168.126.31] oel7n1[enp0s8:192.168.126.70] yes
oel7n1[enp0s8:192.168.126.60] oel7n1[enp0s8:192.168.126.70] yes
Source Destination Connected?
------------------------------ ------------------------------ ----------------
oel7n1[enp0s9:192.168.100.11] oel7n2[enp0s9:192.168.100.21] yes
Verifying Check that maximum (MTU) size packet goes through subnet ...PASSED
Verifying subnet mask consistency for subnet "192.168.126.0" ...PASSED
Verifying subnet mask consistency for subnet "192.168.100.0" ...PASSED
Verifying Node Connectivity ...PASSED
Verifying Multicast or broadcast check ...
Checking subnet "192.168.100.0" for multicast communication with multicast group "224.0.0.251"
Verifying Multicast or broadcast check ...PASSED
Verifying ASMLib installation and configuration verification. ...
Verifying '/etc/init.d/oracleasm' ...PASSED
Verifying '/dev/oracleasm' ...PASSED
Verifying '/etc/sysconfig/oracleasm' ...PASSED
Node Name Status
------------------------------------ ------------------------
oel7n1 passed
oel7n2 passed
Verifying ASMLib installation and configuration verification. ...PASSED
Verifying ACFS Driver Checks ...PASSED
Verifying OCR Integrity ...PASSED
Verifying Network Time Protocol (NTP) ...
Verifying '/etc/ntp.conf' ...
Node Name File exists?
------------------------------------ ------------------------
oel7n2 yes
oel7n1 yes
Verifying '/etc/ntp.conf' ...PASSED
Verifying '/etc/chrony.conf' ...
Node Name File exists?
------------------------------------ ------------------------
oel7n2 yes
oel7n1 yes
Verifying '/etc/chrony.conf' ...PASSED
Verifying Daemon 'chronyd' ...
Node Name Running?
------------------------------------ ------------------------
oel7n2 yes
oel7n1 yes
Verifying Daemon 'chronyd' ...PASSED
Verifying NTP daemon or service using UDP port 123 ...
Node Name Port Open?
------------------------------------ ------------------------
oel7n2 yes
oel7n1 yes
Verifying NTP daemon or service using UDP port 123 ...PASSED
Verifying chrony daemon is synchronized with at least one external time source ...FAILED (PRVG-13606)
Verifying Network Time Protocol (NTP) ...FAILED (PRVG-13606)
Verifying Same core file name pattern ...PASSED
Verifying User Mask ...
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
oel7n2 0022 0022 passed
oel7n1 0022 0022 passed
Verifying User Mask ...PASSED
Verifying User Not In Group "root": oracle ...
Node Name Status Comment
------------ ------------------------ ------------------------
oel7n2 passed does not exist
oel7n1 passed does not exist
Verifying User Not In Group "root": oracle ...PASSED
Verifying Time zone consistency ...PASSED
Verifying Time offset between nodes ...PASSED
Verifying VIP Subnet configuration check ...PASSED
Verifying Voting Disk ...PASSED
Verifying resolv.conf Integrity ...
Node Name Status
------------------------------------ ------------------------
oel7n1 passed
oel7n2 passed
checking response for name "oel7n2" from each of the name servers specified in
"/etc/resolv.conf"
Node Name Source Comment Status
------------ ------------------------ ------------------------ ----------
oel7n2 192.168.126.102 IPv4 passed
checking response for name "oel7n1" from each of the name servers specified in
"/etc/resolv.conf"
Node Name Source Comment Status
------------ ------------------------ ------------------------ ----------
oel7n1 192.168.126.102 IPv4 passed
Verifying resolv.conf Integrity ...PASSED
Verifying Single Client Access Name (SCAN) ...PASSED
Verifying DNS/NIS name service ...PASSED
Verifying Daemon "avahi-daemon" not configured and running ...
Node Name Configured Status
------------ ------------------------ ------------------------
oel7n2 no passed
oel7n1 no passed
Node Name Running? Status
------------ ------------------------ ------------------------
oel7n2 no passed
oel7n1 no passed
Verifying Daemon "avahi-daemon" not configured and running ...PASSED
Verifying Daemon "proxyt" not configured and running ...
Node Name Configured Status
------------ ------------------------ ------------------------
oel7n2 no passed
oel7n1 no passed
Node Name Running? Status
------------ ------------------------ ------------------------
oel7n2 no passed
oel7n1 no passed
Verifying Daemon "proxyt" not configured and running ...PASSED
Verifying loopback network interface address ...PASSED
Verifying Privileged group consistency for upgrade ...PASSED
Verifying Cluster Manager Integrity ...
Node Name Status
------------------------------------ ------------------------
oel7n1 running
oel7n2 running
Verifying Cluster Manager Integrity ...PASSED
Verifying Cluster Integrity ...
Node Name
------------------------------------
oel7n1
oel7n2
Verifying Cluster Integrity ...PASSED
Verifying CRS Integrity ...
Verifying Clusterware Version Consistency ...
Verifying cluster upgrade state ...PASSED
Verifying Clusterware Version Consistency ...PASSED
Verifying CRS Integrity ...PASSED
Verifying CRS user Consistency for upgrade ...PASSED
Verifying Clusterware Version Consistency ...
Verifying cluster upgrade state ...PASSED
Verifying Clusterware Version Consistency ...PASSED
Verifying Node Application Existence ...
Checking existence of VIP node application (required)
Node Name Required Running? Comment
------------ ------------------------ ------------------------ ----------
oel7n2 yes yes passed
oel7n1 yes yes passed
Checking existence of NETWORK node application (required)
Node Name Required Running? Comment
------------ ------------------------ ------------------------ ----------
oel7n2 yes yes passed
oel7n1 yes yes passed
Checking existence of ONS node application (optional)
Node Name Required Running? Comment
------------ ------------------------ ------------------------ ----------
oel7n2 no yes passed
oel7n1 no yes passed
Verifying Node Application Existence ...PASSED
Verifying Check incorrectly sized ASM Disks ...PASSED
Verifying ASM disk group free space ...PASSED
Verifying Network configuration consistency checks ...PASSED
Verifying /boot mount ...PASSED
Verifying OLR Integrity ...PASSED
Verifying Verify that the ASM instance was configured using an existing ASM parameter file. ...PASSED
Verifying User Equivalence ...PASSED
Verifying RPM Package Manager database ...INFORMATION (PRVG-11250)
Verifying Network interface bonding status of private interconnect network interfaces ...PASSED
Verifying /dev/shm mounted as temporary file system ...PASSED
Verifying File system mount options for path /var ...PASSED
Verifying DefaultTasksMax parameter ...PASSED
Verifying zeroconf check ...PASSED
Verifying ASM Filter Driver configuration ...PASSED
Verifying Systemd login manager IPC parameter ...PASSED
Pre-check for cluster services setup was unsuccessful.
Checks did not pass for the following nodes:
oel7n2,oel7n1
Failures were encountered during execution of CVU verification request "stage -pre crsinst".
Verifying Physical Memory ...FAILED
oel7n2: PRVF-7530 : Sufficient physical memory is not available on node
"oel7n2" [Required physical memory = 8GB (8388608.0KB)]
oel7n1: PRVF-7530 : Sufficient physical memory is not available on node
"oel7n1" [Required physical memory = 8GB (8388608.0KB)]
Verifying Network Time Protocol (NTP) ...FAILED
Verifying chrony daemon is synchronized with at least one external time
source ...FAILED
oel7n2: PRVG-13606 : chrony daemon is not synchronized with any external time
source on node "oel7n2".
oel7n1: PRVG-13606 : chrony daemon is not synchronized with any external time
source on node "oel7n1".
Verifying RPM Package Manager database ...INFORMATION
PRVG-11250 : The check "RPM Package Manager database" was not performed because
it needs 'root' user privileges.
CVU operation performed: stage -pre crsinst
Date: May 2, 2021 2:57:54 PM
CVU home: /grid/app/oracle/19.3/grid/
User: oracle
[oracle@OEL7N1 grid]$
We can see that almost everything have passed. While running the upgrade, these will be ignored.
Let's run the dryRun before starting actual upgrade.
[oracle@OEL7N1 grid]$ id
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54323(oper),54324(backupdba),54325(dgdba),54326(kmdba),54330(racdba),54331(asmdba),54332(asmadmin)
[oracle@OEL7N1 grid]$ pwd
/grid/app/oracle/19.3/grid
[oracle@OEL7N1 grid]$ ./gridSetup.sh -silent -dryRunForUpgrade -responseFile /home/oracle/gridupgrade.rsp
Launching Oracle Grid Infrastructure Setup Wizard...
[FATAL] [INS-13013] Target environment does not meet some mandatory requirements.
CAUSE: Some of the mandatory prerequisites are not met. See logs for details. /u01/app/oraInventory/logs/GridSetupActions2021-05-02_03-47-07PM/gridSetupActions2021-05-02_03-47-07PM.log
ACTION: Identify the list of failed prerequisite checks from the log: /u01/app/oraInventory/logs/GridSetupActions2021-05-02_03-47-07PM/gridSetupActions2021-05-02_03-47-07PM.log. Then either from the log file or from installation manual find the appropriate configuration to meet the prerequisites and fix it manually.
Reason : Probably this is because I forgot to use -ignorePrereqFailure
Let's run dryRun again with -ignorePrereqFailure argument
[oracle@OEL7N1 grid]$ ./gridSetup.sh -ignorePrereqFailure -silent -dryRunForUpgrade -responseFile /home/oracle/gridupgrade.rsp
Launching Oracle Grid Infrastructure Setup Wizard...
[WARNING] [INS-13013] Target environment does not meet some mandatory requirements.
CAUSE: Some of the mandatory prerequisites are not met. See logs for details. /u01/app/oraInventory/logs/GridSetupActions2021-05-02_03-55-57PM/gridSetupActions2021-05-02_03-55-57PM.log
ACTION: Identify the list of failed prerequisite checks from the log: /u01/app/oraInventory/logs/GridSetupActions2021-05-02_03-55-57PM/gridSetupActions2021-05-02_03-55-57PM.log. Then either from the log file or from installation manual find the appropriate configuration to meet the prerequisites and fix it manually.
The response file for this session can be found at:
/grid/app/oracle/19.3/grid/install/response/grid_2021-05-02_03-55-57PM.rsp
You can find the log of this install session at:
/u01/app/oraInventory/logs/GridSetupActions2021-05-02_03-55-57PM/gridSetupActions2021-05-02_03-55-57PM.log
As a root user, execute the following script(s):
1. /grid/app/oracle/19.3/grid/rootupgrade.sh
Execute /grid/app/oracle/19.3/grid/rootupgrade.sh on the following nodes:
[oel7n1]
Run the script on the local node.
Successfully Setup Software with warning(s).
This time it is succssful.
Now we need to run "/grid/app/oracle/19.3/grid/rootupgrade.sh" and review log. Only from node1.
[root@OEL7N1 ~]# /grid/app/oracle/19.3/grid/rootupgrade.sh
Check /grid/app/oracle/19.3/grid/install/root_OEL7N1.localdomain_2021-05-02_16-19-23-303067379.log for the output of root script
[root@OEL7N1 ~]#
[oracle@OEL7N1 ~]$ cat /grid/app/oracle/19.3/grid/install/root_OEL7N1.localdomain_2021-05-02_16-19-23-303067379.log
Performing root user operation.
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /grid/app/oracle/19.3/grid
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Performing Dry run of the Grid Infrastructure upgrade.
Using configuration parameter file: /grid/app/oracle/19.3/grid/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/grid/crsdata/oel7n1/crsconfig/rootcrs_oel7n1_2021-05-02_04-20-04PM.log
2021/05/02 16:22:25 CLSRSC-464: Starting retrieval of the cluster configuration data
2021/05/02 16:27:25 CLSRSC-729: Checking whether CRS entities are ready for upgrade, cluster upgrade will not be attempted now. This operation may take a few minutes.
2021/05/02 16:32:23 CLSRSC-693: CRS entities validation completed successfully.
Log is too long. Just pasting few lines which shows that dry run is successful
2021-05-02 16:32:23: crsctl sandbox clean output =
2021-05-02 16:32:23: Executing cmd: /grid/app/oracle/19.3/grid/bin/clsecho -p has -f clsrsc -m 693
2021-05-02 16:32:23: Executing cmd: /grid/app/oracle/19.3/grid/bin/clsecho -p has -f clsrsc -m 693
2021-05-02 16:32:23: Command output:
> CLSRSC-693: CRS entities validation completed successfully.
>End Command output
2021-05-02 16:32:23: CLSRSC-693: CRS entities validation completed successfully.
2021-05-02 16:32:23: Exiting upgrade dry run
Let's check the CRS active version from both the nodes.
[root@OEL7N1 ~]# crsctl query crs activeversion -f
Oracle Clusterware active version on the cluster is [12.2.0.1.0]. The cluster upgrade state is [NORMAL]. The cluster active patch level is [1831702305].
[root@OEL7N2 ~]# crsctl query crs activeversion -f
Oracle Clusterware active version on the cluster is [12.2.0.1.0]. The cluster upgrade state is [NORMAL]. The cluster active patch level is [1831702305].
Time for actual upgrade.
We will be running upgrade in silent mode but it will require a response file. I have generated the response file by running the upgrade in GUI mode and at the last step there will be an option to save response file. Let's start.
[oracle@OEL7N1 grid]$ pwd
/grid/app/oracle/19.3/grid
[oracle@OEL7N1 grid]$ id
uid=54321(oracle) gid=54321(oinstall) groups=54321(oinstall),54322(dba),54323(oper),54324(backupdba),54325(dgdba),54326(kmdba),54330(racdba),54331(asmdba),54332(asmadmin)
[oracle@OEL7N1 grid]$
[oracle@OEL7N1 grid]$ ./gridSetup.sh -silent -ignorePrereqFailure -waitForCompletion -responseFile /home/oracle/gridupgrade.rsp
Launching Oracle Grid Infrastructure Setup Wizard...
[WARNING] [INS-13013] Target environment does not meet some mandatory requirements.
CAUSE: Some of the mandatory prerequisites are not met. See logs for details. /u01/app/oraInventory/logs/GridSetupActions2021-05-03_05-41-37AM/gridSetupActions2021-05-03_05-41-37AM.log
ACTION: Identify the list of failed prerequisite checks from the log: /u01/app/oraInventory/logs/GridSetupActions2021-05-03_05-41-37AM/gridSetupActions2021-05-03_05-41-37AM.log. Then either from the log file or from installation manual find the appropriate configuration to meet the prerequisites and fix it manually.
The response file for this session can be found at:
/grid/app/oracle/19.3/grid/install/response/grid_2021-05-03_05-41-37AM.rsp
As a root user, execute the following script(s):
1. /grid/app/oracle/19.3/grid/rootupgrade.sh
Execute /grid/app/oracle/19.3/grid/rootupgrade.sh on the following nodes:
[oel7n2, oel7n1]
Run the script on the local node first. After successful completion, you can start the script in parallel on all other nodes, except a node you designate as the last node. When all the nodes except the last node are done successfully, run the script on the last node.
Successfully Setup Software with warning(s).
As install user, execute the following command to complete the configuration.
/grid/app/oracle/19.3/grid/gridSetup.sh -executeConfigTools -responseFile /home/oracle/gridupgrade.rsp [-silent]
Let's run /grid/app/oracle/19.3/grid/rootupgrade.sh starting with local node.
Last login: Mon May 3 05:22:14 2021
[root@OEL7N1 ~]# /grid/app/oracle/19.3/grid/rootupgrade.sh
Check /grid/app/oracle/19.3/grid/install/root_OEL7N1.localdomain_2021-05-03_05-52-00-082794319.log for the output of root script
[root@OEL7N1 ~]#
[root@OEL7N1 ~]# tail -f /grid/app/oracle/19.3/grid/install/root_OEL7N1.localdomain_2021-05-03_05-52-00-082794319.log
ORACLE_HOME= /grid/app/oracle/19.3/grid
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /grid/app/oracle/19.3/grid/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/grid/crsdata/oel7n1/crsconfig/rootcrs_oel7n1_2021-05-03_05-53-19AM.log
2021/05/03 05:53:31 CLSRSC-595: Executing upgrade step 1 of 18: 'UpgradeTFA'.
2021/05/03 05:53:31 CLSRSC-4015: Performing install or upgrade action for Oracle Trace File Analyzer (TFA) Collector.
2021/05/03 05:53:31 CLSRSC-595: Executing upgrade step 2 of 18: 'ValidateEnv'.
2021/05/03 05:53:31 CLSRSC-4003: Successfully patched Oracle Trace File Analyzer (TFA) Collector.
2021/05/03 05:53:33 CLSRSC-595: Executing upgrade step 3 of 18: 'GetOldConfig'.
2021/05/03 05:53:33 CLSRSC-464: Starting retrieval of the cluster configuration data
2021/05/03 05:58:26 CLSRSC-692: Checking whether CRS entities are ready for upgrade. This operation may take a few minutes.
2021/05/03 06:09:51 CLSRSC-693: CRS entities validation completed successfully.
2021/05/03 06:10:21 CLSRSC-515: Starting OCR manual backup.
2021/05/03 06:10:45 CLSRSC-516: OCR manual backup successful.
2021/05/03 06:11:09 CLSRSC-486:
At this stage of upgrade, the OCR has changed.
Any attempt to downgrade the cluster after this point will require a complete cluster outage to restore the OCR.
2021/05/03 06:11:09 CLSRSC-541:
To downgrade the cluster:
1. All nodes that have been upgraded must be downgraded.
2021/05/03 06:11:09 CLSRSC-542:
2. Before downgrading the last node, the Grid Infrastructure stack on all other cluster nodes must be down.
2021/05/03 06:11:29 CLSRSC-465: Retrieval of the cluster configuration data has successfully completed.
2021/05/03 06:11:29 CLSRSC-595: Executing upgrade step 4 of 18: 'GenSiteGUIDs'.
2021/05/03 06:11:32 CLSRSC-595: Executing upgrade step 5 of 18: 'UpgPrechecks'.
2021/05/03 06:11:46 CLSRSC-363: User ignored prerequisites during installation
2021/05/03 06:12:14 CLSRSC-595: Executing upgrade step 6 of 18: 'SetupOSD'.
2021/05/03 06:12:15 CLSRSC-595: Executing upgrade step 7 of 18: 'PreUpgrade'.
2021/05/03 06:17:17 CLSRSC-468: Setting Oracle Clusterware and ASM to rolling migration mode
2021/05/03 06:17:18 CLSRSC-482: Running command: '/grid/app/oracle/12.2/bin/crsctl start rollingupgrade 19.0.0.0.0'
CRS-1131: The cluster was successfully set to rolling upgrade mode.
2021/05/03 06:17:24 CLSRSC-482: Running command: '/grid/app/oracle/19.3/grid/bin/asmca -silent -upgradeNodeASM -nonRolling false -oldCRSHome /grid/app/oracle/12.2 -oldCRSVersion 12.2.0.1.0 -firstNode true -startRolling false '
ASM configuration upgraded in local node successfully.
2021/05/03 06:17:38 CLSRSC-469: Successfully set Oracle Clusterware and ASM to rolling migration mode
2021/05/03 06:17:51 CLSRSC-466: Starting shutdown of the current Oracle Grid Infrastructure stack
2021/05/03 06:18:48 CLSRSC-467: Shutdown of the current Oracle Grid Infrastructure stack has successfully completed.
2021/05/03 06:18:52 CLSRSC-595: Executing upgrade step 8 of 18: 'CheckCRSConfig'.
2021/05/03 06:18:55 CLSRSC-595: Executing upgrade step 9 of 18: 'UpgradeOLR'.
2021/05/03 06:19:30 CLSRSC-595: Executing upgrade step 10 of 18: 'ConfigCHMOS'.
2021/05/03 06:20:06 CLSRSC-595: Executing upgrade step 11 of 18: 'UpgradeAFD'.
2021/05/03 06:20:32 CLSRSC-595: Executing upgrade step 12 of 18: 'createOHASD'.
2021/05/03 06:20:54 CLSRSC-595: Executing upgrade step 13 of 18: 'ConfigOHASD'.
2021/05/03 06:20:54 CLSRSC-329: Replacing Clusterware entries in file 'oracle-ohasd.service'
2021/05/03 06:22:11 CLSRSC-595: Executing upgrade step 14 of 18: 'InstallACFS'.
2021/05/03 06:25:57 CLSRSC-595: Executing upgrade step 15 of 18: 'InstallKA'.
2021/05/03 06:26:16 CLSRSC-595: Executing upgrade step 16 of 18: 'UpgradeCluster'.
2021/05/03 06:28:55 CLSRSC-343: Successfully started Oracle Clusterware stack
clscfg: EXISTING configuration version 5 detected.
Successfully taken the backup of node specific configuration in OCR.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
2021/05/03 06:30:03 CLSRSC-595: Executing upgrade step 17 of 18: 'UpgradeNode'.
2021/05/03 06:30:14 CLSRSC-474: Initiating upgrade of resource types
2021/05/03 06:33:45 CLSRSC-475: Upgrade of resource types successfully initiated.
2021/05/03 06:34:19 CLSRSC-595: Executing upgrade step 18 of 18: 'PostUpgrade'.
2021/05/03 06:34:44 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
Let's run it in the second node.
[root@OEL7N2 ~]# /grid/app/oracle/19.3/grid/rootupgrade.sh
Check /grid/app/oracle/19.3/grid/install/root_OEL7N2.localdomain_2021-05-03_06-50-59-532517072.log for the output of root script
[root@OEL7N2 ~]#
[root@OEL7N2 ~]# tail -100f /grid/app/oracle/19.3/grid/install/root_OEL7N2.localdomain_2021-05-03_06-50-59-532517072.log
Performing root user operation.
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /grid/app/oracle/19.3/grid
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /grid/app/oracle/19.3/grid/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/grid/crsdata/oel7n2/crsconfig/rootcrs_oel7n2_2021-05-03_06-52-34AM.log
2021/05/03 06:53:06 CLSRSC-595: Executing upgrade step 1 of 18: 'UpgradeTFA'.
2021/05/03 06:53:06 CLSRSC-4015: Performing install or upgrade action for Oracle Trace File Analyzer (TFA) Collector.
2021/05/03 06:53:07 CLSRSC-595: Executing upgrade step 2 of 18: 'ValidateEnv'.
2021/05/03 06:53:07 CLSRSC-4003: Successfully patched Oracle Trace File Analyzer (TFA) Collector.
2021/05/03 06:53:10 CLSRSC-595: Executing upgrade step 3 of 18: 'GetOldConfig'.
2021/05/03 06:53:11 CLSRSC-464: Starting retrieval of the cluster configuration data
2021/05/03 06:57:24 CLSRSC-465: Retrieval of the cluster configuration data has successfully completed.
2021/05/03 06:57:24 CLSRSC-595: Executing upgrade step 4 of 18: 'GenSiteGUIDs'.
2021/05/03 06:57:24 CLSRSC-595: Executing upgrade step 5 of 18: 'UpgPrechecks'.
2021/05/03 06:57:27 CLSRSC-363: User ignored prerequisites during installation
2021/05/03 06:57:33 CLSRSC-595: Executing upgrade step 6 of 18: 'SetupOSD'.
2021/05/03 06:57:33 CLSRSC-595: Executing upgrade step 7 of 18: 'PreUpgrade'.
ASM configuration upgraded in local node successfully.
2021/05/03 06:58:00 CLSRSC-466: Starting shutdown of the current Oracle Grid Infrastructure stack
2021/05/03 06:58:44 CLSRSC-467: Shutdown of the current Oracle Grid Infrastructure stack has successfully completed.
2021/05/03 07:02:32 CLSRSC-595: Executing upgrade step 8 of 18: 'CheckCRSConfig'.
2021/05/03 07:02:34 CLSRSC-595: Executing upgrade step 9 of 18: 'UpgradeOLR'.
2021/05/03 07:02:57 CLSRSC-595: Executing upgrade step 10 of 18: 'ConfigCHMOS'.
2021/05/03 07:02:57 CLSRSC-595: Executing upgrade step 11 of 18: 'UpgradeAFD'.
2021/05/03 07:03:04 CLSRSC-595: Executing upgrade step 12 of 18: 'createOHASD'.
2021/05/03 07:03:09 CLSRSC-595: Executing upgrade step 13 of 18: 'ConfigOHASD'.
2021/05/03 07:03:10 CLSRSC-329: Replacing Clusterware entries in file 'oracle-ohasd.service'
2021/05/03 07:04:06 CLSRSC-595: Executing upgrade step 14 of 18: 'InstallACFS'.
2021/05/03 07:07:12 CLSRSC-595: Executing upgrade step 15 of 18: 'InstallKA'.
2021/05/03 07:07:17 CLSRSC-595: Executing upgrade step 16 of 18: 'UpgradeCluster'.
2021/05/03 07:10:33 CLSRSC-343: Successfully started Oracle Clusterware stack
clscfg: EXISTING configuration version 19 detected.
Successfully taken the backup of node specific configuration in OCR.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
2021/05/03 07:11:24 CLSRSC-595: Executing upgrade step 17 of 18: 'UpgradeNode'.
Start upgrade invoked..
2021/05/03 07:11:39 CLSRSC-478: Setting Oracle Clusterware active version on the last node to be upgraded
2021/05/03 07:11:39 CLSRSC-482: Running command: '/grid/app/oracle/19.3/grid/bin/crsctl set crs activeversion'
Started to upgrade the active version of Oracle Clusterware. This operation may take a few minutes.
Started to upgrade CSS.
CSS was successfully upgraded.
Started to upgrade Oracle ASM.
Oracle ASM was successfully upgraded.
Started to upgrade CRS.
CRS was successfully upgraded.
Started to upgrade Oracle ACFS.
Oracle ACFS was successfully upgraded.
Successfully upgraded the active version of Oracle Clusterware.
Oracle Clusterware active version was successfully set to 19.0.0.0.0.
2021/05/03 07:12:51 CLSRSC-479: Successfully set Oracle Clusterware active version
2021/05/03 07:12:52 CLSRSC-476: Finishing upgrade of resource types
2021/05/03 07:13:20 CLSRSC-477: Successfully completed upgrade of resource types
2021/05/03 07:16:25 CLSRSC-595: Executing upgrade step 18 of 18: 'PostUpgrade'.
Successfully updated XAG resources.
2021/05/03 07:17:23 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@OEL7N1 ~]# /grid/app/oracle/19.3/grid/bin/crsctl query crs activeversion -f
Oracle Clusterware active version on the cluster is [19.0.0.0.0]. The cluster upgrade state is [UPGRADE FINAL]. The cluster active patch level is [441346801].
[root@OEL7N2 ~]# /grid/app/oracle/19.3/grid/bin/crsctl query crs activeversion -f
Oracle Clusterware active version on the cluster is [19.0.0.0.0]. The cluster upgrade state is [UPGRADE FINAL]. The cluster active patch level is [441346801].
Let's run the next script as install user(oracle)
[oracle@OEL7N1 ~]$ /grid/app/oracle/19.3/grid/gridSetup.sh -executeConfigTools -silent -responseFile /home/oracle/gridupgrade.rsp
Launching Oracle Grid Infrastructure Setup Wizard...
You can find the logs of this session at:
/u01/app/oraInventory/logs/GridSetupActions2021-05-03_07-25-18AM
You can find the log of this install session at:
/u01/app/oraInventory/logs/UpdateNodeList2021-05-03_07-25-18AM.log
You can find the log of this install session at:
/u01/app/oraInventory/logs/UpdateNodeList2021-05-03_07-25-18AM.log
Configuration failed.
[WARNING] [INS-43080] Some of the configuration assistants failed, were cancelled or skipped.
ACTION: Refer to the logs or contact Oracle Support Services.
[oracle@OEL7N1 ~]$
Verification : Successful
[oracle@OEL7N1 ~]$ crsctl query crs activeversion -f
Oracle Clusterware active version on the cluster is [19.0.0.0.0]. The cluster upgrade state is [NORMAL]. The cluster active patch level is [441346801].
[root@OEL7N2 ~]# crsctl query crs activeversion -f
Oracle Clusterware active version on the cluster is [19.0.0.0.0]. The cluster upgrade state is [NORMAL]. The cluster active patch level is [441346801].
I can see everything is up and running
[oracle@OEL7N1 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.GHCHKPT.advm
OFFLINE OFFLINE oel7n1 STABLE
OFFLINE OFFLINE oel7n2 STABLE
ora.LISTENER.lsnr
ONLINE ONLINE oel7n1 STABLE
ONLINE ONLINE oel7n2 STABLE
ora.chad
ONLINE ONLINE oel7n1 STABLE
ONLINE ONLINE oel7n2 STABLE
ora.data.ghchkpt.acfs
OFFLINE OFFLINE oel7n1 volume /opt/oracle/r
hp_images/chkbase is
unmounted,STABLE
OFFLINE OFFLINE oel7n2 volume /opt/oracle/r
hp_images/chkbase is
unmounted,STABLE
ora.helper
OFFLINE OFFLINE oel7n1 STABLE
OFFLINE OFFLINE oel7n2 STABLE
ora.net1.network
ONLINE ONLINE oel7n1 STABLE
ONLINE ONLINE oel7n2 STABLE
ora.ons
ONLINE ONLINE oel7n1 STABLE
ONLINE ONLINE oel7n2 STABLE
ora.proxy_advm
OFFLINE OFFLINE oel7n1 STABLE
OFFLINE OFFLINE oel7n2 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ARCH.dg(ora.asmgroup)
1 ONLINE ONLINE oel7n1 STABLE
2 ONLINE ONLINE oel7n2 STABLE
3 OFFLINE OFFLINE STABLE
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
1 ONLINE ONLINE oel7n1 STABLE
2 ONLINE ONLINE oel7n2 STABLE
3 OFFLINE OFFLINE STABLE
ora.DATA.dg(ora.asmgroup)
1 ONLINE ONLINE oel7n1 STABLE
2 ONLINE ONLINE oel7n2 STABLE
3 OFFLINE OFFLINE STABLE
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE oel7n2 STABLE
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE oel7n1 STABLE
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE oel7n1 STABLE
ora.MGMTLSNR
1 ONLINE ONLINE oel7n1 169.254.4.1 192.168.
100.11,STABLE
ora.VOTE.dg(ora.asmgroup)
1 ONLINE ONLINE oel7n1 STABLE
2 ONLINE ONLINE oel7n2 STABLE
3 OFFLINE OFFLINE STABLE
ora.asm(ora.asmgroup)
1 ONLINE ONLINE oel7n1 Started,STABLE
2 ONLINE ONLINE oel7n2 Started,STABLE
3 OFFLINE OFFLINE STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
1 ONLINE ONLINE oel7n1 STABLE
2 ONLINE ONLINE oel7n2 STABLE
3 ONLINE OFFLINE STABLE
ora.cvu
1 ONLINE ONLINE oel7n2 STABLE
ora.mgmtdb
1 ONLINE ONLINE oel7n1 Open,STABLE
ora.oel7n1.vip
1 ONLINE ONLINE oel7n1 STABLE
ora.oel7n2.vip
1 ONLINE ONLINE oel7n2 STABLE
ora.orcl.db
1 ONLINE ONLINE oel7n1 Open,HOME=/dboracle/
app/oracle/product/1
2.2.0/dbhome_1,STABL
E
2 ONLINE ONLINE oel7n2 Open,HOME=/dboracle/
app/oracle/product/1
2.2.0/dbhome_1,STABL
E
ora.orcl.pdborcl_srv.svc
1 ONLINE ONLINE oel7n1 STABLE
2 ONLINE ONLINE oel7n2 STABLE
ora.qosmserver
1 ONLINE ONLINE oel7n2 STABLE
ora.rhpserver
1 OFFLINE OFFLINE STABLE
ora.scan1.vip
1 ONLINE ONLINE oel7n2 STABLE
ora.scan2.vip
1 ONLINE ONLINE oel7n1 STABLE
ora.scan3.vip
1 ONLINE ONLINE oel7n1 STABLE
--------------------------------------------------------------------------------
[oracle@OEL7N1 ~]$
[oracle@OEL7N1 ~]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[oracle@OEL7N1 ~]$ crsctl stat res -t -init
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
1 ONLINE ONLINE oel7n1 Started,STABLE
ora.cluster_interconnect.haip
1 ONLINE ONLINE oel7n1 STABLE
ora.crf
1 ONLINE ONLINE oel7n1 STABLE
ora.crsd
1 ONLINE ONLINE oel7n1 STABLE
ora.cssd
1 ONLINE ONLINE oel7n1 STABLE
ora.cssdmonitor
1 ONLINE ONLINE oel7n1 STABLE
ora.ctssd
1 ONLINE ONLINE oel7n1 OBSERVER,STABLE
ora.diskmon
1 OFFLINE OFFLINE STABLE
ora.drivers.acfs
1 ONLINE ONLINE oel7n1 STABLE
ora.evmd
1 ONLINE ONLINE oel7n1 STABLE
ora.gipcd
1 ONLINE ONLINE oel7n1 STABLE
ora.gpnpd
1 ONLINE ONLINE oel7n1 STABLE
ora.mdnsd
1 ONLINE ONLINE oel7n1 STABLE
ora.storage
1 ONLINE ONLINE oel7n1 STABLE
--------------------------------------------------------------------------------
[oracle@OEL7N1 ~]$
[root@OEL7N2 ~]# crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[root@OEL7N2 ~]# crsctl stat res -t -init
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
1 ONLINE ONLINE oel7n2 STABLE
ora.cluster_interconnect.haip
1 ONLINE ONLINE oel7n2 STABLE
ora.crf
1 ONLINE ONLINE oel7n2 STABLE
ora.crsd
1 ONLINE ONLINE oel7n2 STABLE
ora.cssd
1 ONLINE ONLINE oel7n2 STABLE
ora.cssdmonitor
1 ONLINE ONLINE oel7n2 STABLE
ora.ctssd
1 ONLINE ONLINE oel7n2 OBSERVER,STABLE
ora.diskmon
1 OFFLINE OFFLINE STABLE
ora.drivers.acfs
1 ONLINE ONLINE oel7n2 STABLE
ora.evmd
1 ONLINE ONLINE oel7n2 STABLE
ora.gipcd
1 ONLINE ONLINE oel7n2 STABLE
ora.gpnpd
1 ONLINE ONLINE oel7n2 STABLE
ora.mdnsd
1 ONLINE ONLINE oel7n2 STABLE
ora.storage
1 ONLINE ONLINE oel7n2 STABLE
--------------------------------------------------------------------------------
[root@OEL7N2 ~]#
Grid has been upgraded successfully from 12.2 to 19.8 in silent mode.