Oracle 11g RAC Implementation Guide

Oracle 11gR1 RAC Documentation

Configuration Details Config uration Summ ary

Orac le Rea l Ap plic atio n Clusters (RAC) (RAC) on Red hat Enterprise Linux 5 Upda te 2 b y using A SM

Serv er Pla tfo rm

HP Prolia roli a nt BL680C G5

Storag e Mod el

HP Storag ew orks MSA2012 MSA2012fc fc Storag e Array

Orac le Softwa re

Orac le Datab ase 11g Relea se 1 (11.1.0. (11.1.0.6) 6) for Linux x86

Linux Distribut Distribut ion

Red ha t Ente rprise Linux 5 Upda te 2 x86

Linux Distribution Details OS

RHEL HEL 5 Upd a te 2 x86

Kernel kernel-2.6.18-

Add itional Pac Pac ka ges Need ed From Distri Distribution bution binutils-2.17.50.0.6-5.el5 compat-libstdc++-33-3.2.3-61 compat-libstdc++-33-3.2.3-61 elfutils-libelf-devel-0.125-3.el5 gdb-6.5-25.el5 glibc -2.5-1 -2.5-18 8 glibc -2.5-1 -2.5-18 8 glibc-common-2.5-18 glibc-devel-2.5-18 glibc-devel-2.5-18 libXp-1.0.0-8.1.el5 libXtst-1.0.1-3.1 libaio-0.3.106-3.2 libaio-devel-0.3.106-3.2 libstd libstd c ++-4.1.2-14 ++-4.1.2-14.el5 .el5 libstd libstd c ++-4.1.2-14 ++-4.1.2-14.el5 .el5 libstdc libstdc ++-dev el-4.1.2 el-4.1.2-14 -14.el5 .el5 make-3.81-1.1 syssta yssta t-7.0.0-3.e l5 unixODBC-2.2.11-7.1 unixODBC-devel-2.2.11-7.1 util-linux-2.13-0.44.el5 xorg-x11-de p rec a te d -lib -lib s-6.8.2-1.E -6.8.2-1.EL.33.0.1 .33.0.1

1


Other Pac kag es HPDMmultipa th-4.3 th-4.3.0 .0 RH RHEL5.2 5.2 rpm (This (This c an be d ow nload ed from from http :// h2000 h20000. 0.ww ww w2.hp.c om ) de vice -map p er-multipa er-multipa th-0.4 th-0.4.7 .7-1 -12. 2.el5 el5 (This p ac kag e is ob ta ined from from OS distri distribut but ion)

Filesystem ilesystem

Mo unt Op tions

Details

ASM ASM

No ne

Using Aut om a tic Storag Storag e M an ag em ent Library Library Driver Driver (asmlib) (asmlib) for datafiles

-

-

oc r and voting disk disk loc loc ate d d irec tly tly on bloc k device , for c onfiguration w ith da ta files files on A SM storag storag e.

2


Background The illus illustration tration b elow show s the m ajor c om po nents of an Orac le RAC RAC 11g Relea se 1 co nfigurat nfigurat ion. Node s in the c luster luster a re typ ica lly lly sep sep a rate serve rs (hosts). (hosts).

Hardware At the ha rdwa re leve leve l, ea c h nod e in a RAC RAC c luster luster sha sha res three things: things: 1.

Ac c ess ess to sha red d isk isk storag storag e

2.

Conne c tion tion to a private private netw ork

3.

Ac c ess ess to a pub lic netwo rk.

4.

Shared Disk Storage Ora c le RAC RAC relies relies on sha sha red d isk isk arc hitec ture. The d a ta b a se files, files, online online red o log s, a nd c on trol files files for the d ata b ase must must b e a c c ess essible to e ac h nod e in the c luster. luster. The shared shared d isks isks also also store store the Orac le Cluster Cluster Reg istry istry and Vot ing Disk Disk (disc (disc uss ussed late r). There a re a va riety riety o f wa ys to c on figure sha sha red storag e including direct a ttac hed disks disks (typic ally SCS SCSII over c op pe r or fibe fibe r), r), Storag Storag e A rea Ne two rks rks (SAN), (SAN), and Netw ork Atta c hed Storag e (NAS). (NAS).

5.

Private Network Eac h cluster cluster nod e is c onne c ted to a ll othe r nod es via a p riva riva te high-sp high-sp ee d ne two rk, rk, also also know n as the c luster luster interc interc onne c t or high-spe high-spe ed interc interc onne c t (HSI). Orac Orac le Ca c he Fus Fusion ion allows d ata stored in the c ac he o f one Orac le ins instanc tanc e to be ac c ess essed by a ny other ins instanc tanc e b y trans transfer ferrring ing it ac ross oss the p rivate netwo rk. It a lso lso p reserves eserves da ta integri integrity ty and c ac he c oherency by transmitti transmitting ng loc king king and other synchronization information across cluster nodes.

6.

Public Network To ma intain high a va ilab ilab ility, ility, ea c h c luster luster nod e is ass assigned a virtual virtual IP ad dress dress (VIP (VIP). In In the eve nt o f nod e failure, failure, the fa iled iled nod e's IP ad dress dress c an be reass reassigned to a surviving urviving nod e to allow a pp lic lic a tions to continue accessing the database through the same IP address.

3


Pa rt I: Insta lling Linux Insta ll an d Co nfigure Linux. We nee d three IP a dd resses for ea c h server: one fo r the p riva te n etw ork, one for the pub lic netw ork, and one for the virtua l IP ad dress. Use the op erating system 's netw ork co nfiguration to ols to assign the p rivate and pub lic netw ork add resses. Server Mo de l 2 X HP BL680C G5 Server series Processors 2 X Intel(R) Xeon(TM) CPU 3.0 GHz Memory 24GB RAM OnBoardStorage 140 GB Ultra 320 SCSI 15k rpm Network/ Interco nnect Dual p ort Broad c om NetXtrem e II BCM5708 Giga bit Ethernet HBA 2 X QLogic QMH 2462 4 Gbps Mezzanine for HP Blade Servers Multipath Device ma pp er multipa th version 0.4.7 Storage Model MSA2012fc Stora g e Array Storage Details 12 X 300GB 15K rpm FC Drives (RAID 10) Configure Na me Resolution

DNS is c onfigured to resolve the follow ing. How eve r, loc al / etc / hosts is also c onfigured on a ll nod es. # vi /e tc/ hosts 127.0.0.1 192.168.11.5 192.168.11.6 192.168.100.5 192.168.100.6 192.168.10.5 192.168.10.6

loc alhost.loc ald om ain loca lhost rac1.asl.net rac1 rac 2.asc l.ne t rac 2 rac 1-priv.a sl.net rac 1-priv rac 2-priv.a sl.net rac 2.priv rac 1-vip.a sl.net rac 1-vip rac 2-vip.a sl.net rac 2-vip

Sav e a nd e xit # service netw ork resta rt Configure NTP on a ll node s so the ir time is sam e up to the sec ond . Con figure NTP on RAC1 nod e: # vi /etc/ ntp.conf # # # # # for server use this and on c lients co mme nt this and use server serverIP # # # # # # # # # # # # # # # # # # serve r 127.127.1.0 # loc a l cloc k Sav e a nd e xit # servic e ntp d resta rt # c hkco nfig ntpd on Con figure NTP on RAC2 nod e: # vi /etc/ ntp.conf # Use p ub lic servers from the p oo l.ntp .org p rojec t. # Please c onsider joining the po ol (http:// www .pool.ntp.org/join.html). server 192.168.11.5 ## # ad d this line on sec ond server # # # Sav e a nd e xit # servic e ntp d resta rt # c hkco nfig ntpd on # ntp d a te –u 192.168.11.5

4


Shut do wn un- nec essary services on a ll node s: # # # #

c hkco nfig --level 35 send ma il off c hkconfig --level 35 pcm cia o ff c hkco nfig --level 35 cup s off c hkconfig --level 35 hpoj off

Part II: Configure Linux for O rac le Create the Orac le Groups and User Ac c ount: Crea te the Linux group s and user ac c ount tha t will be used to insta ll an d m ainta in the Orac le 11 g Release 1 softw are. The user ac c ount w ill be c a lled 'orac le' and the g roups will be 'oinsta ll' and 'db a.' Exec ute the following c omm and s as root on o ne c luster node only: The User ID and Grou p IDs must be t he same on a ll c luster nod es. Using the informa tion from t he id o rac le c omm and , create the Orac le Groups and User Acc ount on the remaining c luster nodes: # groupa dd -g 501 oinstall # g roupa dd -g 502 db a # userad d -m -u 501 -g o install -G d ba orac le # id o rac le uid=501(oracle) gid=501(oinstall) groups=501(oinstall),502(dba) Set the pa ssword on the orac le ac c ount: # pa sswd orac le Cha nging p a sswo rd for user orac le. New password: Retype new password: pa sswd : all auth entic ation to kens upd a ted suc c essfully. Configure Kernel Parameters: Log in as root and c onfigure the Linux kernel parame ters on e ac h nod e. File Na me / etc / sysct l.c onf

Co nf File Setting s ke rne l.msgm ni=2878 kernel.shmmax=4185235456

Com me nts # Set to a v alue ha lf the size o f physica l memory

net.ipv4.ip_local_port_range=1024 65000 net.core.wmem_max=262144 net.core.wmem_default=262144 net.core.rmem_max=4194304

net.core.rmem_default=262144

# rmem_max ca n be tuned b ased o n workload to ba lanc e p erformanc e vs lowmem usag e # rmem_default ca n be tuned b ased o n workload to ba lanc e p erformanc e vs lowmem usag e

kerne l.sysrq=1 kernel.shmall=3279547 kernel.shmmni=4096 kerne l.sem =250 32000 100 142 kernel.msg mn b = 65536 fs.file-max=327679 fs.aio-max-nr=3145728 kernel.msgmax=8192 / etc / sysco nfig / orac lea sm ORACLEA SM_SCA NO RDER=d m / etc / sec urity/ limits.co nf

o rac le soft no file 131072

# de pe nding on size of d b, these ma y need to b e larger

oracle hard nproc 131072 orac le ha rd c ore unlimited orac le soft np roc 131072

5


orac le ha rd nofile 131072 orac le hard me mloc k 50000000 orac le soft m emloc k 50000000

# set mem lock greater than or equal to the sga size to allow orac le to use hugep ag es if c onfigured

/ etc / multip a th .c o nf

orac le soft c ore unlimited d evic e {ve nd or "HP" pro d uc t "MSA2[02] *"} defaults {polling_interval 10 no_path_retry 12}

/ etc / mo d p rob e .c o nf

# The se ha ng c hec k time r sett ings is for op tions hang c hec k_timer ha ngc hec k_tick=1 the d efa ult css missc ount v alue a nd hangcheck_margin=10 hangcheck_reboot=1 need s to be tune d b ased o n the css missc ount va lue & orac le RAC version

# vi /etc / sysc tl.c onf kernel.shmall=3279547 kernel.shmmax=4185235456 kernel.shmmni = 4096 kerne l.sem =250 32000 100 142 fs.file-max=327679 net .ipv 4.ip_loc a l_p ort_ra ng e = 1024 65000 net.c ore.rme m_defa ult = 262144 net.c ore.wme m_defa ult = 262144 net.core.rmem_max = 1048536 net.c ore.wme m_max = 1048536 save and exit # sysc tl -p Setting Shell Limits for the oracle User: Orac le rec om me nd s setting the limits to the nu mb er of proc esses an d num be r of op en files ea c h Linux ac c ount ma y use. To m ake these c ha nge s as roo t. # vi /etc/security/limits.conf orac le soft np roc 131072 oracle hard nproc 131072 ora c le soft n ofile 131072 orac le hard nofile 131072 orac le hard c ore unlimited orac le hard me mloc k 50000000 orac le soft me mloc k 50000000 save and exit # vi /etc/ pam.d/login session req uired / lib / sec urity/ p a m_limits.so session req uired p a m_limits.so save and exit Configure the Hang c hec k Timer: All RHEL relea ses: # vi /e tc/ rc.d/ rc.loc al mod probe ha ngc hec k-timer hang c hec k_tick=30 hang c hec k_ma rgin=180 save and exit

6


Setup user eq uivalenc e o n all RAC no de s: It is highly imp ortant for the o rac le user to b e a ble to login to the sam e no de from whic h ssh session is sta rted a s we ll as logo n to a ll othe r node s of the RAC , without be ing a sked for pa sswo rd. Two no de setup : Log on a s user ORACLE on a ll RAC nod es. Do not give pa ss ph rase to any of the me thod s be low. On Node rac1: # su – orac le $ c hmo d 700 ~/ .ssh $ ssh-keygen -t dsa Generating p ublic / private dsa key pa ir. Enter file in whic h to sav e the key (/ hom e/ orac le/ .ssh/ id_dsa): Created direc tory '/ home/ orac le/ .ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/oracle/.ssh/id_dsa. Your pub lic key has been save d in /hom e/ orac le/ .ssh/ id_dsa.pub . The key f ing erp rint is: 6f:d5:81:eb :25:26:51:c d :53:45:2b :af :01:51:e3:b 1 [email protected] $ ssh-key g en -t rsa Generating p ublic / private rsa key pa ir. Enter file in whic h to sav e the key (/ hom e/ orac le/ .ssh/ id_rsa ): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your id entific ation ha s be en sav ed in / hom e/ orac le/ .ssh/ id_rsa . Your public key has been saved in /home/oracle/.ssh/id_rsa.pub. The key f ing erp rint is: d 6:86:09:f5:2e:ec :dc :35:c8:d 8:7c :22:f9:80:3d:c 3 [email protected] $ c d ~/ .ssh/ $ c at id_rsa.p ub >> autho rized _keys $ ca t id_dsa .pub >> autho rized _keys $ ca t id_rsa.p ub >> / tmp / au thorized _keys.tmp $ cat id_dsa.pub >> /tmp/authorized_keys.tmp On Node rac2: # su – orac le $ c hmo d 700 ~/ .ssh $ ssh-keygen -t dsa Generating p ublic / private dsa key pa ir. Enter file in whic h to sav e the key (/ hom e/ orac le/ .ssh/ id_dsa): Created direc tory '/ home/ orac le/ .ssh'. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/oracle/.ssh/id_dsa.

7


Your pub lic key has been save d in /hom e/ orac le/ .ssh/ id_dsa.pub . The key f ing erp rint is: 6f:d5:81:eb :25:26:51:c d :53:45:2b :af :01:51:e3:b 1 [email protected] $ ssh-key g en -t rsa Generating p ublic / private rsa key pa ir. Enter file in whic h to sav e the key (/ hom e/ orac le/ .ssh/ id_rsa ): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your id entific ation ha s be en sav ed in / hom e/ orac le/ .ssh/ id_rsa . Your public key has been saved in /home/oracle/.ssh/id_rsa.pub. The key f ing erp rint is: d 6:86:09:f5:2e:ec :dc :35:c8:d 8:7c :22:f9:80:3d:c 3 [email protected] $ c d ~/ .ssh/ $ c at id_rsa.p ub >> autho rized _keys $ ca t id_dsa .pub >> autho rized _keys $ ca t id_rsa.p ub >> / tmp / au thorized _keys.tmp $ cat id_dsa.pub >> /tmp/authorized_keys.tmp On Node rac1 $ scp rac2:/tmp/authorized_keys.tmp . a uth o rized _keys.tm p 100% 852 0.8KB/ s 00:00 Then: $ c at au thorized _keys.tmp >> auth orized _keys $ chm od 644 autho rized _keys On Node rac2 $ scp rac1:/tmp/authorized_keys.tmp . a uth o rized _keys.tm p 100% 852 0.8KB/ s 00:00 Then: $ c at au thorized _keys.tmp >> auth orized _keys $ chm od 644 autho rized _keys Now ge t fingerprints of a ll po ssible interfac es / no de s of this RAC setup using ssh. You nee d to exit after ea c h suc c essful log -on to a void c onfusion. On Node rac1: $ ssh ra c 1.asl.ne t $ ssh ra c 1-priv.a sl.net $ ssh ra c 2.asl.ne t $ ssh ra c 2-priv.a sl.net $ ssh rac1-vip.asl.net $ ssh rac 2.vip.a sl.net $ ssh ra c 1 $ ssh ra c 2 $ ssh ra c 1-priv $ ssh ra c 2-priv $ ssh ra c 1-vip $ ssh ra c 2-vip

8


On Node rac2: $ ssh ra c 1.asl.ne t $ ssh ra c 1-priv.a sl.net $ ssh ra c 2.asl.ne t $ ssh ra c 2-priv.a sl.net $ ssh rac 1-vip.a sl.net $ ssh rac 2.vip.a sl.net $ ssh ra c 1 $ ssh ra c 2 $ ssh ra c 1-priv $ ssh ra c 2-priv $ ssh ra c 1-vip $ ssh ra c 2-vip Multipath: multipa th is used to d etec t multiple p aths to d evices for fail-over or performanc e a nd red unda ncy. Installing HPDM Multipa th Too ls on bo th no de s: 1. Log in as roo t to the ho st system . 2. Cop y the installation tar pa c kage to a temp orary direc tory (for exam ple, /tmp / HPDMmultipath). 3. To unbund le the pac kage , enter the following c omm and s: # cd / tmp/ HPDMmultipa th # ta r -xvzf HPDMm ultipa th-4.3.0.ta r.g z # c d HPDMmultipath-4.3.0 4. Ve rify t ha t t he d irec to ry c o nta ins the INSTALL.sh shell sc ript, t he SRPMS , READM E.txt, a nd thedocs directories. 5. To insta ll or upg rad e HPDM Multipa th too ls softw are o n the server, ente r the follow ing c om ma nd : # ./ INSTALL.sh 6. Follow the on-sc reen instructions to c om plete the installation. Configuring QLogic HBA Parameters: To c onfigure the QLogic HBA p a ra me ter, c om plete t he following step s: 1. For QLog ic 2xxx fa mily of HBAs, edit the / etc / mo d p rob e.c on f file in RHEL5 hosts with the follow ing va lues: # vi /etc/mo dprobe.conf options qla2xxx qlport_down_retry=10 ql2xfailover=0 save and exit 2. Reb uild the initrd by e xec uting the follow ing sc ript: # / op t/ hp/ src / hp_qla2x00src / ma ke_initrd 3. Reb oo t the host. Part III: Prepare the Shared Disks Both O rac le Clusterwa re and Orac le RAC req uires ac c ess to d isks tha t are shared b y ea c h nod e in the c luster. HP Storage is configured with RAID 1+0 and assigned 2 volumes for ocr, 3 volumes for voting disk and 3 volumes for ASM are ma pp ed to b oth no de s i.e., rac 1 and rac 2, a fter run the sc si_id c om ma nd q ueries a SCSI d evic e a nd also g enerate a n id with option -g a nd -s. Log in as root on b oth nod es On node rac1: # hp _resca n - a # # ## # # c omma nd is used to find and a dd LUN # sc si_id -g -s / bloc k/sd a 3600a0b80001327510000009b4362163e

On node rac2: # hp_resc an -a # # ## # # # c omma nd is used to find and add LUN # sc si_id -g -s / bloc k/sd a 3600a0b80001327510000009b4362163e

Or You c an user the com ma nd

9


# multipa th -ll # ## # ## instead of scsi_id# # # ## You w ill the outp ut of b elow like sc reen create: 3600a0b80001327d80000006d43621677 [size=12 GB][features="0"][hwhandler="0"] \\_ round-robin 0 \\_ 2:0:0:0 sdb 8:16 \\_ 3:0:0:0 sdf 8:80 create: 3600a0b80001327510000009a436215ec [size=12 GB][features="0"][hwhandler="0"] \\_ round-robin 0 \\_ 2:0:0:1 sdc 8:32 \\_ 3:0:0:1 sdg 8:96 create: 3600a0b80001327d800000070436216b3 [size=12 GB][features="0"][hwhandler="0"] \\_ round-robin 0 \\_ 2:0:0:2 sdd 8:48 \\_ 3:0:0:2 sdh 8:112 create: 3600a0b80001327510000009b4362163e [size=12 GB][features="0"][hwhandler="0"] \\_ round-robin 0 \\_ 2:0:0:3 sde 8:64 \\_ 3:0:0:3 sdi 8:128

After getting output o f ww id nos are to b e b ind w ith alias in the c onfiguration file /e tc/ multipa th.c onf On Node rac1: # vi /e tc/ multipa th.co nf multipath s { multipa th { ww id 3600a 00b 80001327510000009b4362163e # # # co py the a bo ve sc si_id outp ut id here# # # # # alias asm1 } multipa th { wwid 3600a00b80001327510000009b4362153e alias asm2 } multipa th { wwid 3600a00b80001327510000009b4362133e alias asm3 } multipa th { wwid 3600a00b80001327510000009b4362143e alias ocr } multipa th { wwid 3600a00b80001327510000009b4362163e alias ocrmirror } multipa th { wwid 3600a00b80001327510000009b4362163e alias voting } multipa th { wwid 3600a00b80001327510000009b4362163e alias votingmirror

10


} multipa th { wwid 3600a00b80001327510000009b4362163e alias votingmirror2 } } Sav e a nd e xit # vi /etc/rc.loca l # ## # # ## # C hange the ownership of oc r, voting disk and ASM## # # ## chow n root:oinstall /dev/ map per/oc r* chow n orac le:oinstall /d ev/m ap per/voting* chown oracle:oinstall /dev/mapper/asm*

# ## # # ## # Change the permission of oc r, voting and ASM## # # ## # chmod 640 /dev/ mapp er/oc r* chmod 660 /dev/mapper/voting* chmo d 640 /d ev/ map per/a sm*

sav e a nd e xit # scp /e tc/ multipa th.co nf rac 2:/e tc # scp /e tc/ rc .loc al rac 2:/e tc # service m ultipa thd restart # ll /dev/ma pper/ Displays the a lias nam e w hic h a re b ind informa tion c onta ined in the file / etc / multipa th.c onf On Node rac2: # service m ultipa thd restart # ll /dev/ma pper/ Displays the a lias nam e w hic h a re b ind informa tion c onta ined in the file / etc / multipa th.c onf Partition the Disks: On Node rac1: Run the fd isk co mm a nd to c reate the p artition for ASM only # fdisk /de v/ma pp er/a sm1 # fdisk /de v/ma pp er/a sm2 # fdisk /de v/ma pp er/a sm3 # pa rtprobe Run "kpa rtx -a" after FDISK is c om p leted to a dd all partition m ap ping s on the new ly-c reate d m ultipa th d evic e # # # # # # # #

kpa rtx -a /d ev/m ap per/a sm1 kpa rtx -a /d ev/m ap per/a sm2 kpa rtx -a /d ev/m ap per/a sm3 kpartx -l /dev/ map per/a sm1 kpartx -l /dev/ map per/a sm2 kpartx -l /dev/ map per/a sm3 ls /d ev/m ap per/ /e tc/ rc.loc al # # # ## Run this co mma nd for ownership and permission## ## #

On Node rac2: # pa rtprobe

11


Run "kpa rtx -a" after FDISK is c om p leted to a dd all partition m ap ping s on the new ly-c reate d m ultipa th d evic e # kpa rtx -a /d ev/m ap per/a sm1 # kpa rtx -a /d ev/m ap per/a sm2 # kpa rtx -a /d ev/m ap per/a sm3 # kpartx -l /dev/ map per/a sm1 # kpartx -l /dev/ map per/a sm2 # kpartx -l /dev/ map per/a sm3 # ls /d ev/m ap per/ # /e tc/ rc.loc al # ## # # Run this co mma nd for ownership and permission## ## # Installation of ASM on b oth nod es ASMLib 2.0 is d elivered a s a set o f three Linux pa c kag es: ora c lea smlib-2.0 - the A SM libraries • ora c lea sm-sup p ort-2.0 - utilities nee d ed to a d minister ASMLib • orac leasm - a kernel mod ule for the ASM library • First, dete rmine w hich kernel you are using by log ging in as root a nd running the fo llow ing c om ma nd : # uname –rm Download the kernel verion related oracleasm from the link http:// www .orac le.com/ technology/tec h/linux/a smlib/ index.html # rpm -ivh ora c lea sm-2.6.18-53.1.14.el5-2.0.4-1.el5 ora c lea sm-sup p ort-2.0.4-1.el5 ora c lea smlib-2.0.31.el5.x86_64.rpm Con figuring ASMLib on both nodes # /e tc/ init.d/ orac leasm co nfigure Default user to own the driver interface []: oracle Default group to own the driver interface []: oinstall Start Oracle ASM library driver on boot (y/n) [n]: y Fix permissions of Oracle ASM disks on boot (y/n) [y]: y Writing Oracle ASM library driver configuration: Creating /dev/oracleasm mount point: Loading module "oracleasm": Mounting ASMlib driver filesystem: Scanning system for ASM disks:

[ [ [ [ [

OK OK OK OK OK

] ] ] ] ]

Note: Mark disks for use by ASMLib by running the following c om ma nd a s root only on Nod e rac 1 Tip : Ent er th e DISK_NA ME in UPPERCASE lett ers. # /etc/init.d/oracleasm createdisk VOL1 Marking disk "/dev/mapper/asm1p1" as an # /etc/init.d/oracleasm createdisk VOL2 Marking disk "/dev/mapper/asm2p1" as an # /etc/init.d/oracleasm createdisk VOL3 Marking disk "/dev/mapper/asm3p1" as an

/dev/mapper/asm1p1 ASM disk: /dev/mapper/asm2p1 ASM disk: /dev/mapper/asm3p1 ASM disk:

[

OK

]

[

OK

]

[

OK

]

# /etc/init.d/oracleasm listdisks VOL1 VOL2 VOL3 On node rac2: Run the follow ing c om ma nd as roo t to sc a n for co nfigured A SMLib d isks: # / etc / init.d/ orac leasm sc and isks

12


Install the Clusterware softwa re Cop y and Unzip the C lusterware a t loca tion / stag e1 and c hec k the p rerequisites have be en m et using the “ runcluvfy.sh” utility in the Clusterwa re. On Node rac1: # c d /stage1 # c p -r /m ed ia/ c lusterwa re linux.x64_11gR1_c lusterwa re.zip / sta ge 1 # un zip c luste rwa re linux.x64_11gR1_c luste rwa re.zip # xhost + # su – orac le $ cd / home/oracle $ vi . c rs.en v e xpo rt ORAC LE_BASE=/ no d e1 export ORACLE_HOME=/node1/crs sav e a nd e xit $ . c rs.env $ cd / stag e1/c lusterware $ ./runc luvfy.sh sta ge -pre c rsinst –n ra c 1,ra c 2 –ve rbo se $ ./ runInsta ller

13


14


15


16


On Node rac1: Open a new terminal window Run t he sc rip t # c d / home/ orac le/ oraInventory/ # ls -la # ./o rainstRoo t.sh On Node rac2: Open a new terminal window Run t he sc rip t # c d / home/ orac le/ oraInventory/ # ls -la # ./o rainstRoo t.sh Then On Node rac1: Open a new terminal window Run t he sc rip t # c d / n o d e 1/ c rs # ls -la # ./root .sh On Node rac2: Open a new terminal window Run t he sc rip t # c d / n o d e 1/ c rs # ls -la # ./root .sh On Node Rac 1: Press OK

17


Wait for the c onfiguration a ssistan ts to c om plete .

The n selec t e xit.

18


Then run the env ironm ent va riab les is loc ate d in file /h om e/ orac le/ c rs.env $ . c rs.en v $ c rs_stat -t # # # # # # To c hec k the status of Clusterware # # # # # # # # Name Type R/RA F/FT Target State Host ---------------------------------------------------------------------ora.rac1.gsd application 0/5 0/0 ONLINE ONLINE rac1 ora.rac1.ons application 0/3 0/0 ONLINE ONLINE rac1 ora.rac1.vip application 0/0 0/0 ONLINE ONLINE rac1 ora.rac2.gsd application 0/5 0/0 ONLINE ONLINE rac2 ora.rac2.ons application 0/3 0/0 ONLINE ONLINE rac2 ora.rac2.vip application 0/0 0/0 ONLINE ONLINE rac2

$ exit Orac le Datab ase software Installation: Cop y and unzip the Orac le 11g da tab ase at loca tion / stag e1. On Node rac1: # c d /stage1 # c p -r / med ia/ linux.x64_11gR1_d a ta b a se.zip / sta g e1 # u nzip linux.x64_11gR1_da ta b a se.zip # xhost + # su – orac le $ cd / home/orace $ vi . a sm.e nv e xpo rt ORAC LE_BASE=/ no d e1 export ORACLE_HOME=/node1/asm save and exit $ . asm.en v $ cd /stage 1/da taba se $ ./ runInsta ller

19


Selec t the "Enterprise Edition" option a nd c lic k the "Next" button .

Choo se / node1 for Orac le ba se, lea ve Name a nd Path in Software Location at /nod e1/asm.

20


Selec t Cluster Installa tion and enab le rac 1(alread y marked) a nd rac 2 nodes.

Installer should verify your environment. In your configuration you probably do not have enough swap space, but this hasn't caused any problems, so you can safely “user-verify” this. Also you should ignore the kernel rmem_de fault pa rame ter notice (it's also O K).

21


At the Privileged Operating Systems Groups promp t ensure you have selec ted the “ db a” group e verywhere. (In serious dep loyment s these group s should be c a refully sep arat ed for sec urity rea sons.)

Selec t Install da tab ase software o nly as you want to create the d atab ase later.

22


Double-chec k everything and then clic k Install.

Installation p roce ss will oc c ur. Taking into a c c ount tha t iSCSI and storag e o p timizations have n't be en d one yet, it ca n take up to one hour dep ending on your hardware.

After installation you will be asked to run post-installation scripts on both nodes.

23


Execute the mentioned script on all the nodes.

24


Clic k Exit DBCA: Creation of Datab ase a nd ASM instanc e Creating a n ASM Instanc e a nd a Disk G roup w ith DBCA To c reate an ASM insta nc e a nd a disk group with DBCA, p erform the follow ing step s: DBCA sta rts its GUI inte rfac e . # xhost + # su – orac le $ . db .env $ c d /nod e1/asm/ product/ 11.1.0/db _1/bin/ $ ./d bc a

Selec t Configure Automatic Storag e Ma nage ment and c lick Next.

25


26


27


Click "Finish" to exit out from d bc a. Verify tha t LISTENER an d ASM instanc es are up an d running and are pro pe rly reg istered with CRS. CRS STACK STATUS AFTER THE INSTALLATIO N A ND CONFIG URATION O F ASM

28


================================================================ $ c rs_sta t -t Nam e Typ e Targ et State Host -----------------------------------------------------------ora....SM1.asm ap plic atio n ONLINE O NLINE no de 1-p ub ora ....UB.lsnr a p plic atio n ONLINE ONLINE no de 1-p ub ora....pub .gsd ap plic atio n ONLINE ONLINE nod e1- pu b ora....pub .ons ap plic atio n ONLINE ONLINE nod e1- pu b ora....pub .vip ap plic atio n ONLINE ONLINE nod e1-p ub ora....SM2.asm ap plic atio n ONLINE O NLINE no de 2-p ub ora ....UB.lsnr a p plic atio n ONLINE ONLINE no de 2-p ub ora....pub .gsd ap plic atio n ONLINE ONLINE nod e2- pu b ora....pub .ons ap plic atio n ONLINE ONLINE nod e2- pu b ora....pub .vip ap plic atio n ONLINE ONLINE nod e2-p ub A Database Configuration Assistant: Warning windo w informs you o f your ne xt step s.

Select Create a Database.

29


Ensure that b oth no d es a re selec ted .

30


Choose Custom Data ba se to have better flexibility in the database creation process.

Nam e yo ur da tab ase. For the p urpose o f this guid e c all it as “e rac .world ” . The SID prefix should a utom a tica lly be set to “ erac ” . Individua l SID are going to b e “ rac 1” .. “ rac 2”.

31


Dec ide w hethe r you w a nt to insta ll Orac le Enterprise Ma na ge r. If yes, c onfigure it ap p ropriate ly.

For testing p urposes you c a n selec t som e e asy pa sswo rd for all imp ortant O rac le ac c ounts.

32


As you w an t to b uild a n Extende d RAC c onfiguration, you must ch oo se ASM a s your storag e o ption.

You will use a no n-sha red PFILE for e ve ry ASM insta nc e. You c ould c ho ose to use a sha red SPFILE to ha ve c entralized ASM c onfiguration, bu t the n a prob lem a rises: At w hich storag e a rray should you store this c ritica l file?

33


At the Create Diskg roup Window c lick on Chang e Disk Disc ove ry Path; a new window should p op up.

At Initialization Parameters you c an c onfigure how muc h me mo ry (tota l) will be used by this insta nc e. This c an b e later alter by changing the memory_target parameter. Customize other database parameters to meet yours needs.

34


There a re ma ny new sec urity op tions in Orac le Data b ase 11g (which are on b y de fault). Ac ce pt them all.

Ensure you ha ve selec ted Enable a utomatic m aintenance tasks.

35


At Databa se Storage you c an tune pa rame ters relate d to REDO log s, co ntrolfiles, and so o n. Defaults are ap prop riate for initial te sting.

36


Selec t only Create Databa se; if you'd like you ca n gene rate da tab ase sc ripts to spe ed up c rea tion of a new DB after wiping a n old o ne (e.g. for new e xperiments).

You will be presente d w ith summ ary show ing whic h op tions are g oing to b e insta lled . The d ata ba se c reation proc ess c an b e som eho w long d ep end ing on the o pt ions be ing insta lled and the storag e used . Finally, you w ill get a quick summary about the c reated da taba se.

37

Oracle 11g RAC Implementation Guide

Recommend Documents