execute from: for: dispatch LinuxFiles dispatch SunFiles dispatch XTSadmFiles (ssh, krb, ...) piboot CrayFiles (piman/pingo) ogboot CrayFiles (ogman/ognip) mn1sm LinuxFiles (midnight) node a node with access to *Files.long can validate itself
|
The chk_sanity.ksh script is in the sysmon crontab on appropriate management server hosts and can be invoked manually to validate one or more nodes on demand. The [Config|Linux|Sun]Files are conceptually identical but platform specific. The *Files directories map managed files for all platform nodes identified in the related machines.* file.
Related files (in /usr/local/adm/etc or /var/local/*Files unless noted):
Related commands and tools (in /usr/local/adm/[s]bin/):
file name: alternate names: purpose: etc/machines.* .list|.linux|.sun|.cray|... managed systems ConfigFiles/List.txt [Linux|Sun|Cray]Files managed files ConfigFiles/Long.txt [Linux|Sun|Cray]Files file mode,ownership,sum /var/local/ConfigFiles/ [Linux|Sun|Cray]Files configuration files
Reference ARSC Configuration Push Process for further information on these files and tools.
tool name: purpose: push [...] config [...] propagate ConfigFiles push [...] one [...] propagate ConfigFiles push -s [...] rcp -rq [...] back-copy ConfigFiles get_ConfigFiles.ksh identify *Files for push get_Machines.ksh parse entries from machines.list upd_ConfigFiles.ksh update ConfigFiles/List.txt,Long.txt upd_LinuxFiles.ksh update LinuxFiles/List.txt,Long.txt upd_SunFiles.ksh update SunFiles/List.txt,Long.txt upd_CrayFiles.ksh update CrayFiles/List.txt,Long.txt chk_sanity_Files generate file mode|ownership|sum on a node cfsanity validate node file lists against *Files.long cmp_sanity.ksh back-copy and compare *Files edconfig edit *Files making backout mkbko make backout copy of file ckbko check backout copy of file cksumnode compare files between systems
dispatch: /usr/local/adm/bin/chk_sanity.ksh -?
Usage: chk_sanity.ksh [-options] [hostname [...]]
Options:
-h sun| linux| cray
|| -sun| -linux|-cray
-u # run upd_*Files.ksh -u, current:
-F # force upd_*Files update, current:
-l # run on local node only, current:
-x # use 'set -x', current:
-q # non-quiet (verbose), current: tty
+q # quiet, current:
-node|-w|-m node1[,node2...] # same as 'hostname [...]'
-type type(hardware) #
-status status #
-group|-usage usage(group) #
-version|-os version[.sub] #
-other|-frame|-rack other(frame|rack) #
See: /usr/local/adm/etc/machines.*
/var/local/*Files/List.txt
Sample Report:
Mis-matched mode, ownership, sum, or type:
----------- (ConfigFiles) (actual)
*1 ochre :0644 root root :0600 . . :/etc/X11/xorg.conf
*2 kappa :0640 root 95 : . . 3 :/etc/cups/printers.conf
*3 kappa !: sum .template : . . . :/bin/ksh
*4 lemon !: sum .lemon :0644 . 0 :/etc/sysctl.conf
*5 kappa #: c=none . : t=link :/boot/grub/menu.lst
*6 neladm $: c=none . : t=directory :/etc/sysconfig/network
*7 kappa -: c=link .kappa :0600 root root :/boot/grub/grub.conf
*8a kappa -: c=none . :0644 root root :/etc/hosts
*8b puppychow-: c=nofile .template :0640 root linuxman:/etc/log.d/conf/services/named.conf
*9 puppychow?: c=file .template : t=. (missing) :/etc/profile.d/zARSC.sh
' ' node mode|ownership and ConfigFiles do not match '!' node sum -r and ConfigFiles do not match '#' node is a symlink and ConfigFiles entry is not or does not match '$' node is a directory and ConfigFiles entry is not '-' node is a file and ConfigFiles entry is not '?' node has no file and ConfigFiles entry exists
To facilitate resolving the errors, The report is sorted by:
Correcting errors will require knowledge of the systems. Discrepancies should be treated seriously, there is a possibility that checksum or permissions changes may indicate a file or system has been compromised. Most discrepancies are due to software updates, procedural errors, or temporary changes for testing. It may be necessary to work with the platform ISSO and administrator or identify who was working on a particular file or product.
For the sample report above, explanations of the errors, and possible corrections follow:
Correct this problem by issuing a chmod on ochre (as shown above) or correcting the LinuxFiles entry.*1 ochre :0644 root root :0600 . . :/etc/X11/xorg.conf dispatch: push -linux -m ochre cmd -eq uals --fields m /etc/X11/xorg.conf ochre:0600 /etc/X11/xorg.conf dispatch: cd /var/local/LinuxFiles/etc/X11/xorg.conf; uals --fields m xorg.conf.* 0644 xorg.conf.1920x1200 0700 xorg.conf.ochre -> xorg.conf.1920x1200 ... dispatch: push -s -linux -m ochre cmd -eq chmod 644 /etc/X11/xorg.conf
Correct this problem by issuing a chgrp on kappa (as shown above) or correcting the LinuxFiles entry.*2 kappa :0640 root 95 : . . 3 :/etc/cups/printers.conf dispatch: push -linux -m kappa cmd -eq uals --fields mg --both /etc/cups/printers.conf kappa:0640 3 sys /etc/cups/printers.conf dispatch: cd /var/local/LinuxFiles/etc/cups/printers.conf dispatch: uals -y mg --both printers.conf* 0640 95 linuxman printers.conf.template ... dispatch: push -s -linux -m kappa cmd -eq chgrp 95 /etc/cups/printers.conf
Above it was determined that two Ultra20 systems have different ksh binaries for the same rpm. It was corrected above by back-copying kappa to LinuxFiles, but that is likely an incorrect fix. The problem could be:*3 kappa !: sum .template : . . . :/bin/ksh dispatch: grep 'Ultra20' /usr/local/adm/etc/machines.linux | head -2 iota Ultra20 Linux any 2.6 15 0618FK300P W008-014 nis . kappa Ultra20 Linux any 2.6 15 0618FK300E W008-022 nis . dispatch: push -linux -m kappa,iota cmd -eq \ "uals /bin/ksh; sum -r /bin/ksh; rpm -qa | grep ksh" iota:- 0755 root root 2m 050302.0323 /bin/ksh iota:06893 1038 iota:ksh-20050202-1 kappa:- 0755 root root 2m 050302.0323 /bin/ksh kappa:44557 1043 kappa:ksh-20050202-1 dispatch: cd /var/local/LinuxFiles/bin/ksh; sum -r ksh.template 06893 2076 ksh.template dispatch: push -linux -m kappa,iota cmd -eq "ksh -c 'echo test; uname -n'" iota:test iota:iota.arsc.edu kappa:test kappa:kappa.arsc.edu dispatch: push -s -linux -m kappa rcp -rq /bin/ksh ksh; uals ksh.kappa push_rcp: from:/bin/ksh to:/var/local/LinuxFiles/bin/ksh/ksh kappa:/usr/software/bin/scp -q -p kappa:/bin/ksh /var/local/... - 0755 root root 2m 050302.0323 ksh.kappa
Correct this problem by removing the incorrect sysctl.conf.lemon. It appears lemon was regressed back to the template intentionally after some testing.*4 lemon !: sum .lemon :0644 . 0 :/etc/sysctl.conf dispatch: cd /var/local/LinuxFiles/etc/sysctl.conf; uals - 0650 root linuxman 842 060714.1421 sysctl.conf.lemon - 0644 root root 670 050525.0904 sysctl.conf.template ... dispatch: push -s -linux -m lemon rcp -rq /etc/sysctl.conf x push_rcp: from:/etc/sysctl.conf to:/var/local/LinuxFiles/etc/sysctl.conf/x lemon:/usr/software/bin/scp -q -p lemon:/etc/sysctl.conf /var/local/... dispatch: sudo sdiff -s s*lemon x.lemon # < # 2006-07-14 jsd Added a much larger than default shared memor < < #Set the max shared memory < kernel.shmall = 1073741824 < kernel.shmmax = 1073741824 < < dispatch: sudo sdiff -s x.lemon s*template dispatch: mkbko -s sysctl.conf.lemon; sudo rm x.lemon sysctl.conf.lemon
Correct this particular problem by creating menu.lst.template and removing the unnecessary entries. In this case, the majority of systems link the file to ./grub.conf so a template is appropriate.*5 kappa #: c=none . : t=link :/boot/grub/menu.lst dispatch: push -linux -m kappa cmd -eq uals /boot/grub/menu.lst kappa:l 0777 root root 11 051007.1453 /boot/grub/menu.lst -> ./grub.conf dispatch: grep ^kappa /usr/local/adm/etc/machines.linux kappa Ultra20 Linux any 2.6 15 0618FK300E W008-022 nis . dispatch: cd /var/local/LinuxFiles/boot/grub/menu.lst dispatch: uals *.kappa *.Ultra20 *.2.6* *.template uals: *.kappa: No such file or directory uals: *.Ultra20: No such file or directory uals: *.2.6*: No such file or directory uals: *.template: No such file or directory dispatch: uals --type l menu.lst.* | grep '\./grub.conf' | wc -l 51 dispatch: uals --type l --fields f menu.lst.* | while read F X; do sudo rm $F; done dispatch: sudo ln -s ./grub.conf menu.lst.template; uals menu.lst.* - 0600 root root 770 050517.0058 menu.lst.neladm - 0600 root root 641 051208.1454 menu.lst.nelchina
Correct this by creating a link to the DIRECTORY tag to indicate network is a directory on neladm. Mode and ownership of directory is not validated. The DIRECTORY tag exists to cope with different Linux implementations which use a directory of files vs. a file for some features.*6 neladm $: c=none . : t=directory :/etc/sysconfig/network dispatch: push -linux -n neladm cmd -eq uals -d /etc/sysconfig/network neladm:d 0755 root root 4096 050825.1247 /etc/sysconfig/network dispatch: cd /var/local/LinuxFiles/etc/sysconfig/network; uals *.nel* l 0700 root root 9 060918.1326 network.nelchina -> DIRECTORY dispatch: sudo ln -s DIRECTORY network.neladm
Correct this problem by removing the incorrect grub.conf.kappa.*7 kappa -: c=link .kappa :0600 root root :/boot/grub/grub.conf dispatch: push -linux -m kappa cmd -eq uals /boot/grub/grub.conf kappa:- 0600 root root 683 061023.1228 /boot/grub/grub.conf dispatch: cd /var/local/LinuxFiles/boot/grub/grub.conf; uals l 0700 root root 20 061026.1514 grub.conf.kappa -> ./grub.conf.template l 0700 root root 13 060517.1031 grub.conf.slewfoot -> grub.conf.smp - 0600 root root 692 060629.1030 grub.conf.smp - 0600 root root 683 060629.1030 grub.conf.template ... dispatch: push -s -linux -m kappa cmd -eq sum -r /boot/grub/grub.conf kappa:40064 1 dispatch: sudo sum -r grub.conf.template 40064 2 grub.conf.template dispatch: sudo rm grub.conf.kappa
Correct this problem by back-copying hosts.kappa. It is not possible to create a hosts.template, each linux workstation is different.*8a kappa -: c=none . :0644 root root :/etc/hosts dispatch: cd /var/local/LinuxFiles/etc/hosts dispatch: uals *.kappa *.Ultra20 *.2.6* *.template uals: *.kappa: No such file or directory uals: *.Ultra20: No such file or directory uals: *.2.6*: No such file or directory uals: *.template: No such file or directory dispatch: push -s -linux -m kappa rcp -rq /etc/hosts hosts push_rcp: from:/etc/hosts to:/var/local/LinuxFiles/etc/hosts/hosts kappa:/usr/software/bin/scp -q -p kappa:/etc/hosts /var/local/LinuxFiles/... dispatch: uals hosts.kappa - 0644 root root 180 051114.1316 hosts.kappa
Correct this problem by creating an entry for puppychow (and the entire dogteam). While most Linux systems should not have this file, there are exceptions.*8b puppychow-: c=nofile .template :0640 root linuxman:/etc/log.d/conf/services/named.conf dispatch: cd /var/local/LinuxFiles/etc/log.d/conf/services/named.conf dispatch: uals named* - 0640 root linuxman 1112 050822.1716 named.conf.cerebro l 0700 root root 8 060504.1109 named.conf.template -> ./NOFILE dispatch: M=puppychow,lindog1,lindog2,lindog3,lindog4,lindog5,lindog6 dispatch: push -s -linux -m $M rcp -rq /etc/log.d/conf/services/named.conf x ... dispatch: sudo sum -r x.* 33408 3 x.lindog1 33408 3 x.lindog2 33408 3 x.lindog3 33408 3 x.lindog4 33408 3 x.lindog5 33408 3 x.lindog6 33408 3 x.puppychow dispatch: sudo mv x.puppychow named.conf.dogteam dispatch: sudo ln -s named.conf.dogteam named.conf.puppychow dispatch: sudo rm x.*; mkbko -s * ... dispatch: sudo chgrp linuxman named.conf.dogteam
Correct this problem by pushing zARSC.sh to nodes that require it. In this case, double check with puppychow administrator that this does what they expect for that system.*9 puppychow?: c=file .template : t=. (missing) :/etc/profile.d/zARSC.sh dispatch: M=puppychow,lindog1,lindog2,lindog3,lindog4,lindog5,lindog6 dispatch: push -linux -m $M cmd -eqf \ "cd /etc/profile.d; if [ -r zARSC.sh ]; then uals zARSC.sh; fi" lindog3:- 0644 root root 1175 060515.1804 zARSC.sh lindog6:- 0644 root root 1175 060515.1804 zARSC.sh dispatch: push -linux -g dogteam config -eq zARSC.sh ... dispatch: push -linux -n puppychow config -eq zARSC.sh ...