Pro clu_create
1. Virtual cluster interconnect device (isc0) 10.0.0.1
Phisical cluster interconnect device (tu0) 10.1.0.1
Zachem dublirovat' phisical virtual'nym i zachem dva adresa dlya CI?
Esli MC, to IP budet tol'ko odin.
Esli LAN - to virtual'nyj adres - dlya emulyacii MC-interfejsa
2. Particii pod root,var,usr nuzhno razmechat' zaranee vrukopashnuyu?
Otvet: da
3. Diski dlya quorum, boot1 clu_create rasapartishiruet samostoyatel'no?
Otvet: da
4. Gde fizicheski zhivet /cluster/members/{memb}
Otvet: sootvetstvuyushchij boot-disk root2_domain#root at /dev/disk/dsk2a
Naprimer: /vmunix --> /cluster/members/{memb}/boot_partition/vmunix
5. Kak sgenerit' yadro dlya tol'ko chto vstavlennogo v mashinu fiber
chanel-kontrollera
>>> boot -file genvmunix -fl s dka300
# doconfig
# cp /sys/MOS209/vmunix /
Esli dravera net, to postavit' NHD (New Hardware Delivery)
mount /cdrom
/cdrom/nhd_install nu a on zapuskaet setld
stavit v t.ch. draver. Esli stavilsya draver - to budet perezagruzka i
peresborka yadra
6. Kakoj vneshnij adres budet u kazhdogo chlena klastera?
odinakovyj - sovpadayushchij s IP klastera? Kak ih razlichat' snaruzhi?
Ili ya budu hodit' s uzla na uzel ispol'zuya vnutrennie CI-shnye adresa?
Phisical ili virtual'nyj?
U uzla budet neskol'ko IP-adresov. Svoj individual'nyj, obshcheklasternyj,
naznachennyj na IC-fizicheskij, na IC-virtual'nyj
Po umolchaniyu OUT-IP - adres uzla. Esli v /etc/cluster_services portu dat'
atribut outmulti, to naruzhu na etot port pojdem ot imeni klastera
7. DRD kasaetsya tol'ko diskov, ili drugie ustrojstva tozhe?
(Esli da, to kakie - com,lpt,ethernet...)?
DSK, TAPE
8. O kvorume. Dostatochnyj kvorum - kontanta, ili proporcionalen kolichestvu
dostizhimyh na dannyj moment uzlov? T.e. v 8-mashinnom klastere mozhno li
dobrat'sya vychitaniem po odnomu do 2-uzlovoj konfiguracii?
Minimal'nyj dostatochnyj kvorum - konstanta. No ee mozhno umen'shat' v
/etc/sysconfigtab No delat' eto nado zaranee, esli ostanetsya uzlov men'she,
chem etot minimum, klaster povisnet v io-suspend
9. Po analogii s simlinkami. Gde otrabatyvayuysya cdsl-linki pri NFS montazhe?
I kak obstoit delo s absolyutnymi i otnositel'nymi linkami?
Otvet:
On otdaetsya kak simlink, as is vmeste s {memb}. Esli ya hochu ego prochitat',
mne pridetsya samomu byt' trueunix'om. V klastere NFS-server yavlyaetsya imenno
klasternym, i otdaet tol'ko obshchuyu fajlovuyu sistemu.
10. ssh IP-klastera budet kidat' na odnu i tu zhe nodu vsegda? Ili round-robin
Esli port zaregistrirovan v /etc/cluster_services kak
in_single (default) - to vse prihody forvardyatsya na odnu i tu zhe nodu
(vybrannuyu sluchajno. Esli noda zakryla port, nachinaetsya probros na druguyu)
in_multi - to prihody forvardyatsya po nodam v sootvetstvii s atributom
selection-weight (po umolchaniyu 3 na odnu, zatem 3 na
druguyu, i tak po krugu)
---------------------------------------
eth eth
CLUSTER CLUSTER
ci ====== ci
MEMBER1 MEMBER2
scsi/fc fs
\______________|______________________________
RAID SCSI
massiv disk
ci - Cluster interconnect - libo dedicated LAN
libo MC - Memory Chanel
scsi/fs - obshchaya SCSI shina ili Fiber chanel dlya podklyucheniya
sistem hraneniya vidimyh so vvseh chlenov klastera
O zabege SCSI na servere DS15
Linejnoe 1*r-chtenie 66M/sec
Linejnoe 2*r-chtenie 2*66M/sec
Linejnoe 3*r-chtenie 2*66M/sec
Linejnoe 4*r-chtenie 2*66M/sec
Linejnoe 1*b-chtenie 18M/sec
Linejnoe 2*b-chtenie 2*18M/sec
Linejnoe 3*b-chtenie 2*18M/sec
Linejnoe 4*b-chtenie 2*18M/sec
UKAZATELX | ADVFS-draver | diskovaya metka | particiya a, b, c
Izgotovlenie zagruzchika
dd if=zagruzochnyjdisk of=novyjdisk
nhd
# tol'ko na staryh mashinah chtob rabotal wwidmgr
set mod diag
# tol'ko na staryh mashinah chtob rabotala shared bus
set bus_probe_algorithm new
set eib0_mode
set eib0 FastFD # ee0 = eia0, ee1 = eib0
help wwidmgr # poluchaem help
wwidmgr -show wwid # Poluchaem spisok yunitov
Naznachit' disk s udid 131 v peremennuyu wwid0, 132 v
wwidmgr -clear all
wwidmgr -quickset -item 1 -udid 131
wwidmgr -quickset -item 2 -udid 132
show wwid* # spisok raspoznanyh fc-yunitov
show n* # spisok vidimyh portov na raid-massive
eto pozvolit uvidet' ih v show dev
disklabel -r dsk8 # Posmotret'
disklabel -re dsk8 # Otredaktirovat' s ispol'zovaniem tekstovogo redaktora
diskconfig # Otredaktirovat' cherez GUI
Particii
/dev/rdisk/dsk[0-9][a-h] # syrye dsk7c "c" particiya na ves' dis'k
/dev/disk/dsk[0-9][a-h] # blochnye a,b,d,e,f,g,h - mogut perekryvat'sya
CFS - Cluster File system.
Vse chleny klastera vidyat edinuyu (odinakovuyu) fajlovuyu sistemu.
/ /etc /usr i t.d. - odna na vseh.
Dlya razmeshcheniya _razlichnyh_ (individual'nyh) fajlov ispol'zuetsya
/cluster/members/member1/...
/cluster/members/member2/...
/cluster/members/{memb}/ - katalog individual'nyh fajlov chlena klastera
/cluster/members/member --> /cluster/members/{memb}
Hotya /etc/ - obshchij, v nem hochetsya nekotorye fajly imet' individual'nymi.
|to delaetsya cherez Context Dependend Symbolic Link (CDFS) ukazyvayushchih vovnutr'
individual'nogo dereva
/etc/passwd --> ../cluster/members/{memb}/etc/passwd
CFS zhivet poverh DRD - obrashcheniya iz fajlovoj sistemy k diskam prohodyat
skvoz' sloj Device Request Dispatcher - esli disk lokal'nyj (t.e. viden s
mashiny napryamuyu - zapros obrabatyvaetsya napryamuyu, esli disk podklyuchen k
drugomu chlenu klastera - zapros forvarditsya na nego cherez Cluster Intercon
nect
kazhdaya fajlovaya sistema rabotayushchaya pod CFS hostitsya konkretnym uzlom,
ostal'nye redirektyat svoi zaprosy na nee cherez DRD. V sluchae gibeli uzla,
avtomaticheski proishodit perebros obsluzhivaniya fs na drugoj uzel. Failover
avtomaticheskij, avtomaticheskij load-balancing otsutstvuet.
cfsmgr /filesystem - kakoj uzel yavlyaetsya serverom dlya etoj fs
cfsmgr -a server=system2 / - perebrosit' obsluzhivanie "/" na system2
CAA - Cluster Aplication Availability
Sistema, obespechivayushchaya failover dlya prilozhenij - perebros na drugoj uzel v
sluchae nepriyatnostej na osnovnom uzle.
Single-instance - odna kopiya na klaster. CAA obespechivaet ej failover
Multi-instance - na kazhdom uzle po kopii. Ispol'zuyut locking, chtoby
ne konfliktovat' po obshchim resursam
Distributed - na kazhdom uzle po kopii. Ispol'zuyut klasternye sredstva
opoveshcheniya i blokirovki (Distributed Lock Manager)
chtoby ne konfliktovat' po resursam
/var/cluster/caa/profile/*.cap - Profajly vseh CAA i network-resursov
/var/cluster/caa/script/*.scr - Skripty vseh CAA
################# Network ###############
TYPE = network
NAME = net155
DESCRIPTION = Localnetwork of our office
SUBNET = 155.208.157.0
FAILURE_TRESHHOLD = 0
FAILURE_INTERVAL = 0
################# Tape, Mediachanger ###############
TYPE = tape # ili changer
NAME = TapeRMT0
DESCRIPTION = Tape drive 1
DEVICE_NAME = /dev/tape/tape1
FAILURE_TRESHHOLD = 0
FAILURE_INTERVAL = 0
################# Application ###############
TYPE = application
NAME = appl
DESCRIPTION = Application Description
HOSTING_MEMBERS
PLACEMENT = balanced # kuda uhodit'
favored (po poryadku spiska, a esli net, gde pridetsya),
restricted (po poryadku spiska, i tol'ko na nem)
REQUIRED_RESOURCES
OPTIONAL_RESOURCES
CHECK_INTERVAL = 60
FAILURE_TRESHHOLD = 0
FAILURE_INTERVAL = 0
RESTART_ATTEMPTS = 1
FAILOVER_DELAY =
AUTO_START = 1 # startovat' li posle reboot
ACTION_SCRIPT = appl.scr # starting/stoping,check-running
lezhit v kataloge /var/cluster/caa/script
Argumenty: start/stop/check, kod uspeha 0
SCRIPT_TIMEOUT = 60 # esli skript ne otrabotal za 60 sek, to fail
###############################################
caa_profile sozdanie caa-profajla
caa_register registraciya caa-prilozheniya ili caa-resursa(netw,tape)
caa_start zapusk caa-prilozheniya ili caa-resursa(netw,tape)
caa_stop ostanov caa-prilozheniya (no ne resursa)
caa_stat posmotret' status
caa_relocate perebrosit' na drugoj uzel
Klaster imeet hostname i IP klastera, dopolnitel'no
Kazhdyj chlen klastera imeet svoj sobstvennyj hostname i IP
Cluster interconnect konfiguritsya kak IP-dostup, s adresami uzlov 10.0.0.1,
10.0.0.2,...
Dublirovanie ethernet osushchestvlyaet vstroennyj v operacionku mehanizm NetRain
Interfejsy ee0,ee1,ee2 soedinyayutsya v virtual'nyj interfejs nr0, na kotoryj i
veshaetsya IP (v lyuboj moment rabotaet tol'ko odin interfejs, ostal'nye - v
zapase na goryachej podmene)
Na NetRain saditsya ClusterInterconnect. Esli potrebuetsya rezervirovanie
interfejsov v klientskuyu set' - eto delaetsya prosto standartnymi sredstvami
operacinki, klaster v etom ne uchastvuet.
Konfigurirovanie i installyaciya klastera
Sperva konfiguriruem operacionku na odnom servere so vsem
neobhodimym softom. Zatem sozdaem klaster. Pri etom
sozdaetsya otdel'nye (novye) diski dlya obshchih klasternyh fajlovyh sistem:
/ > 125M (luchshe 250M)
/usr/ > 680M (luchshe 1000M)
/var/
i t.d. I oni _ne sovpadayut_ s "zatravochnym" diskom ka kotorom zhila
iznachal'naya operacionnaya sistema. Dlya kazhdogo chlena klastera sozdaetsya svoj
sobstvennyj zagruzochnyj disk, na kotorom zhivet ego zagruzochnoe yadro (i
bol'she nichego).
Itogo, klasteru potrebuyutsya sleduyushchie diski (LUN'y):
tru64 (3Gb) - "Zatravochnyj disk" - na nego stavitsya operacionka i soft.
Nuzhen tol'ko pri sozdanii klastera, posle setapa klastera ne ispol'zuetsya.
root (particiya 250M) - obshchij dlya vseh chlenov klastera
/usr (particiya 1000M) - obshchij dlya vseh chlenov klastera
/var (particiya 1000M) - obshchij dlya vseh chlenov klastera
/dannye1,2 (po vkusu) - obshchie dlya vseh chlenov klastera
boo1 (disk 500M) - zagruzochnyj disk dlya member1
/dev/disk/dsk2a -- root2_domain#root on /cluster/members/{memb}/boot_partition
/dev/disk/dsk2b -- swap dlya member1
/dev/disk/dsk2h -- cnx particiya 1M, v samom konce diska -
dlya sinhronizacionnyh zamorochek klastera
boo2 (disk 500M) - zagruzochnyj disk dlya member2
quorum (disk 1M) - dlya golosovanij (kto ostanetsya zhit')posle raspada klastera
clu_create - sozdaet klaster - vse obshchie diski i boot1 dlya member1
zatem gruzimsya s boo1, i uzhe s sozdannogo member1 generim vse ostal'nye uzly
clu_add_client - pri etom budut sozdavat'sya boot2, boot3,...
mount /dev/disk/cdrom0c /cdrom
setld -l /cdrom/TruCluster/kit # s diska Additional kit 2
umount /cdrom
mount /dev/disk/cdrom0c /cdrom # NHD (New Hardware Detect) 1 of 1
cd /cdrom ; ./nhd_install
umount /cdrom
lmfsetup # stavim licenziyu na TruCluster TCS-UA
clu_create
zagruzka na novom uzle, na nem:
clu_add_member
Neklasternye (individual'nye) fajly v kazhdoj iz fajlovoj sistem lezhat v
/fsdir-name/cluster/members/member[N]/...
naprimer
/var/adm/abc.log - obshcheklasternyj fajl
mkcdsl -c /var/adm/abc.log peremeshchaet ego v member-specific i
sozdaet na nego CDSL
/var/adm/abc.log --> /var/cluster/members/{memb}/adm/abc.log
mkcdsl -a /var/adm/abcde - sozdaet lokal'nyj fajl i cdsl na nego na kazhdom
uzle klastera
dsfmgr - Device Special Files manager
hwmgr - Hardware manager
drdmgr - Device Request Dispatcher manager
hwmgr -scan scsi
hwmgr -view devices
hwmgr -view devices -cluster
hwmgr -view hierarchy -member membername
hwmgr -show scsi
SCSI DEVICE DEVICE DRIVER NUM DEVICE FIRST
HWID: DEVICEID HOSTNAME TYPE SUBTYPE OWNER PATH FILE VALID PATH
-------------------------------------------------------------------------
68: 0 mos214 disk none 0 1 dsk0 [0/0/0]
69: 1 mos214 disk none 0 1 dsk1 [0/1/0]
70: 2 mos214 disk none 2 1 dsk2 [0/3/0]
71: 3 mos214 cdrom none 0 1 cdrom0 [2/0/0]
75: 5 mos214 disk none 0 1 dsk4 [0/2/0]
114: 4 mos214 raid none 0 4 scp1 [6/0/0]
120: 6 mos214 disk none 0 1 dsk5 [6/3/11]
121: 7 mos214 disk none 0 1 dsk6 [6/3/12]
122: 8 mos214 disk none 2 1 dsk7 [6/3/13]
123: 9 mos214 disk none 0 1 dsk8 [6/3/14]
124: 10 mos214 disk none 0 1 dsk9 [6/3/15]
125: 11 mos214 disk none 0 1 dsk10 [6/3/16]
hwmgr -delete -id 125 # udalenie devajsa iz tablicy
dsfmgr -v - verify korrektnost' naznacheniya draverov
dsfmgr -vF - verify & fix korrektnost' naznacheniya draverov
drdmgr -a server dsk3 Kto server u diska dsk3
drdmgr -h system1 -a accessnode=system2 dsk3
Naznachit' sisteme system1 dostup k disku dsk3 cherez system2
Vypolnit' na vseh uzlah: hwmgr -scan comp -cat scsi_bus
Najti i naznachit' novye diski v klastere
sysman -station
--> cluster specific
clu_get_info status klastera i uzlov
clu_check_config podrobnaya diagnostika
Udalenie uzla navsegda
member2# shutdown -sh
member1# clu_delete_member -m 2
member1# clu_quorum dlya pereopredeleniya uslovij kvoruma
Udalenie uzla vremenno
member2: otorvat' kabeli
member1# clu_quorum dlya pereopredeleniya uslovij kvoruma
Pereimenovanie klastera:
podpravit' v /etc/sysconfigtab
clubase:cluster_name=newname
naprimer sipolnit' na kazhdom uzle
cat > tmp-file <Kvorum
naznachit' kvorum-disk
clu_quorum -f -d add dsk8 1
clu_quorum -f -d adjust dsk8 1
clu_quorum -f -d remove
Vydat' uzlu memb1 kvorumnyj golos
clu_quorum -f -m memb1 1
Skol'ko golosov maksimal'no mozhet byt' nabrano
clu_quorum -f -e 3
Dostatochnym dlya kvoruma budet (N+2)/2
Dublirovanie zagruzochnogo diska
disklabel -r /dev/rdisk/dsk$1c > lbl.txt
disklabel -z /dev/rdisk/dsk$2c
disklabel -Rr -t advfs /dev/rdisk/dsk$2c lbl.txt /mdec/rzboot.advfs /mdec/bootrz.advfs
dd if=/dev/disk/dsk$1a of=/dev/disk/dsk$2a bs=8192k
dd if=/dev/disk/dsk$1b of=/dev/disk/dsk$2b bs=8192k
dd if=/dev/disk/dsk$1g of=/dev/disk/dsk$2g bs=8192k
dd if=/dev/disk/dsk$1h of=/dev/disk/dsk$2h bs=8192k
Last-modified: Thu, 12 Aug 2004 17:23:33 GMT