/ *********************************************************************** *****/ /* Document : UNIX command examples, mainly based on Solaris, AIX, HP */ /* and ofcourse, also Linux. */ /* Doc. Version : 115 */ /* File : unix.txt */ /* Purpose : some examples for the Oracle, DB2, SQLServer DBA */ /* Date : 07-07-2009 */ /* Compiled by : Albert van der Sel */ /* Best use : Use find/search in your editor to find a string, command, */ /* or any identifier */ / *********************************************************************** *****/
############################################ SECTION 1. COMMANDS TO RETREIVE SYSTEM INFO: ############################################ ========================== 1. HOW TO GET SYSTEM INFO: ========================== 1.1 Short version: ================== See section 1.2 for more detailed commands and options. Memory: ------AIX:
Linux:
bootinfo -r lsattr -E -l mem0 lsattr -E -l sys0 -a realmem svmon -G vmstat -v vmo -L lparstat -i or use a tool as "topas" or "nmon" (these are utilities) cat /proc/meminfo dmesg | grep "Physical" free (the free command)
HP:
getmem grep MemTotal /proc/meminfo dmesg | grep -i phys wc -c /dev/mem or us a tool as "glance", like entering "glance -m" from prompt (is a utility) Solaris: prtconf | grep "Memory size" # total memory prtmem memps -m Tru64: vmstat -P | grep "Total Physical Memory" uerf | grep memory Swap: ----AIX:
lsps -a lsps -s pstat -s
HP: Solaris:
swapinfo -a swap -l prtswap -l swapon -s cat /proc/swaps cat /proc/meminfo
Linux:
cpu: ---HP:
ioscan -kfnC processor getconf CPU_VERSION getconf CPU_CHIP_TYPE model
AIX:
lparstat (-i) prtconf | grep proc pmcycles -m lsattr -El procx (x is 0,2, etc..) lscfg | grep proc pstat -S mpstat
Linux:
cat /proc/cpuinfo
Solaris:
psrinfo -v prtconf psrset -p prtdiag
OS version: -----------
HP:
uname -a
Linux:
cat /proc/version
Solaris: uname -a cat /etc/release (or other way to view that file, like "more /etc/release") Tru64: /usr/sbin/sizer -v AIX:
oslevel -r (only high-level version) oslevel -s (shows Version, SP, TL level) oslevel -qs (shows complete history) lslpp -h bos.rte
AIX Example: # oslevel -s 5300-08-03-0831 # oslevel -qs Known Service Packs ------------------5300-08-03-0831 5300-08-02-0822 5300-08-01-0819 5300-08-00-0000 5300-07-05-0831 5300-07-04-0818 5300-07-03-0811 5300-07-02-0806 5300-07-01-0748 5300-06-08-0831 5300-06-07-0818 5300-06-06-0811 5300-06-05-0806 5300-06-04-0748 5300-06-03-0732 5300-06-02-0727 5300-06-01-0722 5300-05-CSP-0000 5300-05-06-0000 5300-05-05-0000 5300-05-04-0000 5300-05-03-0000 5300-05-02-0000 5300-05-01-0000 5300-04-CSP-0000 5300-04-03-0000 5300-04-02-0000 5300-04-01-0000 5300-03-CSP-0000
AIX firmware:
lsmcode -c processor lsmcode -r -d scraid0 RAID adapter scraid0 lsmcode -A devices prtconf serial# etc..
display the system firmware level and service display the adapter microcode levels for a display the microcode level for all supported shows many setting including memory, firmware,
Notes about Power 4 or 5 lpars: ------------------------------For AIX: The uname -L command identifies a partition on a system with multiple LPARS. The LPAR id can be useful for writing shell scripts that customize system settings such as IP address or hostname. The output of the command looks like: # uname -L 1 lpar01 The output of uname -L varies by maintenance level. For consistent output across maintenance levels, add a -s flag. For illustrate, the following command assigns the partition number to the variable "lpar_number" and partiton name to "lpar_name". For HP-UX: Use commands like "parstatus" or "getconf PARTITION_IDENT" to get npar information.
patches: -------AIX:
Is a certain fix (APAR) installed? instfix -ik APAR_number instfix -a -ivk APAR_number
To determine your platform firmware level, at the command prompt, type: lscfg -vp | grep -p Platform The last six digits of the ROM level represent the platform firmware date in the format, YYMMDD. HP:
/usr/sbin/swlist -l patch swlist | grep patch Linux: rpm -qa Solaris: showrev -p
pkginfo -i package_name /usr/sbin/dupatch -track -type kit
Tru64:
Netcards: --------AIX:
lsdev -Cc adapter lsdev -Cc adapter | grep ent lsdev -Cc if lsattr -E -l ent1 ifconfig -a Solaris: prtconf -D / prtconf -pv prtdiag | grep "card" svcs -x ifconfig -a (up plumb)
/
prtconf | grep "card"
Quickly find out who is using most memory: -----------------------------------------See section marked &&& (use find/search on &&&)
Network sniffing: ----------------Here are a few short descriptions, and examples, of usefull network trace / dump commands. -- Solaris: snoop command examples: For example, if we want to observe traffic between systems alpha and beta we can use the following command: # snoop alpha,beta To enable data captures from the snoop output without losing packets while writing to the screen, send the snoop output to a file. For example: # snoop -o /tmp/snooper -V 128.50.1.250 To snoop a specific port: # snoop -o port xxx -- AIX: tcpdump command examples: # tcpdump port 23 # tcpdump -i en0
A good way to use tcpdump is to save the network trace to a file with the -w flag and then analyze the trace by using different filtering options together with the -r flag. The following example show how to run a basic tcpdump network trace, saving the output in a file with the -w flag (on a Ethernet network interface): # tcpdump -w /tmp/tcpdump.en0 -i en0 To limit the number of traced packets, use the -c flag and specify the number, such as in the following example that traces the first 128 packets (on a token-ring network interface): # tcpdump -c 128 -w /tmp/tcpdump.tr0 -i tr0 iptrace command examples: To start the iptrace daemon with the System Resource Controller (SRC), # startsrc -s iptrace -a "/tmp/nettrace" To stop the iptrace daemon with SRC enter the following: # stopsrc -s iptrace To record packets coming in and going out to any host on every interface, enter the command in the following format: # iptrace /tmp/nettrace The recorded packets are received on and sent from the local host. All packet flow between the local host and all other hosts on any interface is recorded. The trace information is placed into the /tmp/nettrace file. To record packets received on an interface from a specific remote host, enter the command in the following format: # iptrace - i en0 -p telnet -s airmail /tmp/telnet.trace The packets to be recorded are received on the en0 interface, from remote hostairmail, over the telnet port. The trace information is placed into the /tmp/telnet.trace file. To record packets coming in and going out from a specific remote host, enter the command in the following format: # iptrace -i en0 -s airmail -b /tmp/telnet.trace The packets to be recorded are received on the en0 interface, from remote host airmail. The trace information is placed into the /tmp/telnet.trace file. -- HPUX: nettl command: Initialize the tracing/logging facility: # nettl -start
Logging is enabled for all subsystems as determined by the /etc/nettlgen.conf file. Log messages are sent to a log file whose name is determined by adding the suffix .LOG000 to the log file name specified in the /etc/nettlgen.conf configuration file. To stop the tracing facility: # nettl -stop Turn on inbound and outbound PDU tracing for the transport and session (OTS/9000) subsystems and send binary trace messages to file /var/adm/trace.TRC000. # nettl -traceon pduin pduout -entity transport session \ -file /var/adm/trace Session using nettl and the formatter netfmt: 1. Capture packets nettl -tn all -e ns_ls_ip -tm 99999 -size 1024 -f some-raw-capture-file 2. Reproduce problem. 3. Turn off trace: nettl -tf -e all 4. Create formatter filter file. Example: filter tcp_sport 6699 filter tcp_dport 6699 5. Filter the packets: 5.1 "Long" display netfmt -Nlnc filter-file -f some-raw.capture > formatted.out 5.2 "One-liner" display netfmt -Nln1Tc filter-file -f some-raw.capture > one-liner.out -- Restart inetd, nfs: -- ------------------Starting and stopping NFS: -------------------------On all unixes, a number of daemons should be running in order for NFS to be functional, like for example the rpc.* processes, biod, nfsd and others. Once nfs is running, and in order to actually "share" or "export" your filesystem on your server, so remote clients are able to mount the nfs mount, in most cases you should edit the "/etc/exports" file. -- AIX: The following subsystems are part of the nfs group: nfsd, biod, rpc.lockd, rpc.statd, and rpc.mountd. The nfs subsystem (group) is under control of the "resource controller", so starting and stopping nfs is actually easy # startsrc -g nfs
# stopsrc -g nfs Or use smitty. -- Redhat Linux: # /sbin/service nfs restart # /sbin/service nfs start # /sbin/service nfs stop -- On some other Linux distros # /etc/init.d/nfs start # /etc/init.d/nfs stop # /etc/init.d/nfs restart -- Solaris: If the nfs daemons aren't running, then you will need to run: # /etc/init.d/nfs.server start -- HP-UX: Issue the following command on the NFS server to start all the necessary NFS processes (HP): # /sbin/init.d/nfs.server start Or if your machine is only a client: # cd /sbin/init.d # ./nfs.client start Restart or refresh inetd after you have edited "inetd.conf": -----------------------------------------------------------After you have edited "/etc/inetd.conf", for example, to enable or disable some service, you need to restart, or refresh inetd, to read the new configuration information. To let inetd to reread the configfile: -- AIX: # refresh -s inetd -- HPUX: # /usr/sbin/inetd -c -- Solaris: # /etc/init.d/inetd stop # /etc/init.d/inetd start # pkill -HUP inetd reread the configuration. -- RedHat / Linux # service xinetd restart or # /etc/init.d/inetd restart
1.2 More Detail:
# The command will restart the inetd and
================ 1.2.1 Show memory in Solaris: ============================= prtconf: -------Use this command to obtain detailed system information about your Sun Solaris installation # /usr/sbin/prtconf # prtconf -v Displays the size of the system memory and reports information about peripheral devices Use this command to see the amount of memory: # /usr/sbin/prtconf | grep "Mem" sysdef -i reports on several system resource limits. Other parameters can be checked on a running system using adb -k : # adb -k /dev/ksyms /dev/mem parameter-name/D ^D (to exit) Other commands: --------------# prtmem # memps -m
1.2.2 Show memory in AIX: ========================= >> Show Total memory: --------=====-------# bootinfo -r # lsattr -El sys0 -a realmem # prtconf (you can grep it on memory) >> Show Details of memory: -------------------------You can have a more detailed and comprehensive look at AIX memory by using "vmstat -v" and "vmo -L" or "vmo -a": For example: # vmstat -v 524288 memory pages 493252 lruable pages 67384 free pages
7 131820 80.0 20.0 80.0 25.4 125727 0.0 0 25.4 80.0 125575 0 14557 6526890 18631 0 49038 fsbuf faults
memory pools pinned pages maxpin percentage minperm percentage maxperm percentage numperm percentage file pages compressed percentage compressed pages numclient percentage maxclient percentage client pages remote pageouts scheduled pending disk I/Os blocked with no pbuf paging space I/Os blocked with no psbuf filesystem I/Os blocked with no fsbuf client filesystem I/Os blocked with no fsbuf external pager filesystem I/Os blocked with no
0 Virtualized Partition Memory Page Faults 0.00 Time resolving virtualized partition memory page
The vmo command really gives lots of output. In the following example only a small fraction of the output is shown: # vmo -L .. lrubucket 128K 128K 128K 64K 4KB pages D ------------------------------------------------------------------------------maxclient% 80 80 80 1 100 % memory D maxperm% minperm% ------------------------------------------------------------------------------maxfree 1088 1088 1088 8 200K 4KB pages D minfree memory_frames ------------------------------------------------------------------------------maxperm 394596 394596 S ------------------------------------------------------------------------------maxperm% 80 80 80 1 100 % memory D minperm% maxclient% -------------------------------------------------------------------------------
maxpin S .. ..
424179
424179
>> To further look at your virtual memory and its causes, you can use a combination of: -------------------------------------------------------------------------------------# # # # # # #
ipcs -bm lsps -a vmo -a or vmo -L svmon -G svmon -U svmon -P vmstat -v
(shared memory) (paging) (virtual memory options) (basic memory allocations) (virtual memory usage by user)
To print out the memory usage statistics for the users root and steve
taking into account only working segments, type: svmon -U root steve -w To print out the top 10 users of the paging space, type: svmon -U -g -t 10
To print out the memory usage statistics for the user steve, including the list of the process identifiers, type: svmon -U steve -l svmon -U emcdm -l # vmo -o npswarn=value # schedo -o pacefork=15 Note: sysdumpdev -e Although the sysdumpdev command is used to show or alter the dumpdevice for a system dump, you can also use it to show how much real memory is used. The command # sysdumpdev -e provides an estimated dump size taking into account the current memory (not pagingspace) currently in use by the system. Note: the rmss command: The rmss (Reduced-Memory System Simulator) command is used to ascertain the effects of reducing the amount of available memory on a system without the need to physically remove memory from the system. It is useful
for system sizing, as you can install more memory than is required and then use rmss to reduce it. Using other performance tools, the effects of the reduced memory can be monitored. The rmss command has the ability to run a command multiple times using different simulated memory sizes and produce statistics for all of those memory sizes. The rmss command resides in /usr/bin and is part of the bos.perf.tools fileset, which is installable from the AIX base installation media. Syntax rmss -p -c
-r Options -p Print the current value -c MB Change to M size (in Mbytes) -r Restore all memory to use -p Print the current value Example: find out how much memory you have online rmss -p Example: Change available memory to 256 Mbytes rmss -c 256 Example: Undo the above rmss -r Warning: rmss can damage performance very seriously Don't go below 25% of the machines memory Never forget to finish with rmss -r The pstat command: -----------------The pstat command, which displays many system tables such as a process table, inode table, or processor status table, The pstat command interprets the contents of the various system tables and writes it to standard output. Use the pstat command from the AIX 5.2 command prompt. See the command reference for details and examples, or use the syntax summary in the table below. Flags -a -A -f -i addresses -p -P -s usage -S -t
Displays Displays Displays Displays
entries in the process table all entries in the kernel thread table the file table the i-node table and the i-node data block
Displays the process table Displays runnable kernel thread table entries only Displays information about the swap or paging space Displays the status of the processors Displays the tty structures
-u ProcSlot Displays the user structure of the process in the designated slot of the process table. An error message is generated if you attempt to display a swapped out process. -T Displays the system variables. These variables are briefly described in var.h -U ThreadSlot Displays the user structure of the kernel thread in the designated slot of the kernel thread table. An error message is generated if you attempt to display a swapped out kernel thread.
&&& -------------------------------------------------------------------------------Note 1: How to get a "reaonable" view on memory consumption of a process in UNIX: -------------------------------------------------------------------------------With using just the command line, or some free utils. In general not so easy to answer, because of the "sub components" you might distinguish in memory occupation. For example, do you mean RSS, real, shared, virtual, paging, including all libraries loaded, etc..? -- Some people like to use the ps command with some special flags, like ps -vg ps auxw # or ps auxw | sort -r +3 |head -10 (top users) But those commands seems not so very satisfactory, and not "complete" in their output. -- There are some great common utilities like topas, nmon, top etc.., or tools specific to a certain Unix, like SMC for Solaris. No bad word on those tools, because they are great. But some people think that they are not satisfactory on the subject of memory consumption of a process (although they show a lot of other interesting information). -- Some other ways might be: # procmap pid # pmap -x pid
(in e.g. AIX) (in e.g. Solaris)
Those tools also show a "total" memory usage, which is a good indicator. For example: # pmap -x $$ 492328: -ksh Address Kbytes 00010000 192 00040000 8 00042000 40
RSS 192 8 40
Anon 8 8
Locked -
Mode r-x-rwx-rwx--
Mapped File ksh ksh [ heap ]
FF180000 680 680 FF23A000 24 24 FF240000 8 8 8 FF280000 576 576 FF310000 40 40 FF31A000 24 16 FF350000 16 16 FF364000 8 8 FF380000 40 40 FF39A000 8 8 FF3A0000 8 8 FF3B0000 8 8 8 FF3C0000 152 152 FF3F6000 8 8 8 FFBFC000 16 16 8 -------- ------- ------- ------- ------total Kb 1856 1848 48 -
r-x-rwx-rwx-r-x-rwx-rwx-r-x-rwx-r-x-rwx-r-x-rwx-r-x-rwx-rw---
libc.so.1 libc.so.1 libc.so.1 libnsl.so.1 libnsl.so.1 libnsl.so.1 libmp.so.2 libmp.so.2 libsocket.so.1 libsocket.so.1 libdl.so.1 [ anon ] ld.so.1 ld.so.1 [ stack ]
This gives you a reasonable idea on memory consumption of a pid. You can also try: # # # #
svmon svmon svmon svmon
-G -U -P -t 10 -U steve -l
(top 10 users) (memory stats for user steve)
But svmon is not available on all unixes. The following might also be helpfull (not on all unixes): # ls -l /proc/{pid}/as # prstat -a -s rss And ps can give some info as well # ps -ef | egrep -v "STIME|$LOGNAME" | sort +3 -r | head -n 15 # ps au
1.2.3 Show memory in Linux: =========================== # /usr/sbin/dmesg | grep "Physical:" # cat /proc/meminfo # free -m The ipcs, vmstat, iostat and that type of commands, are ofcourse more or less the same in Linux as they are in Solaris or AIX.
1.2.4 Show aioservers in AIX:
============================= # lsattr -El aio0 autoconfig available fastpath enable kprocprio 39 maxreqs 4096 maxservers 10 minservers 1
STATE to be configured at system restart State of fast path Server PRIORITY Maximum number of REQUESTS MAXIMUM number of servers per cpu MINIMUM number of servers
True True True True True True
# pstat -a | grep -c aios 20 # ps -k | grep aioserver 331962 - 0:15 aioserver 352478 - 0:14 aioserver 450644 - 0:12 aioserver 454908 - 0:10 aioserver 565292 - 0:11 aioserver 569378 - 0:10 aioserver 581660 - 0:11 aioserver 585758 - 0:17 aioserver 589856 - 0:12 aioserver 593954 - 0:15 aioserver 598052 - 0:17 aioserver 602150 - 0:12 aioserver 606248 - 0:13 aioserver 827642 - 0:14 aioserver 991288 - 0:14 aioserver 995388 - 0:11 aioserver 1007616 - 0:12 aioserver 1011766 - 0:13 aioserver 1028096 - 0:13 aioserver 1032212 - 0:13 aioserver What are aioservers in AIX5?: With IO on filesystems, for example if a database is involved, you may try to tune the number of aioservers (asynchronous IO) AIX 5L supports asynchronous I/O (AIO) for database files created both on file system partitions and on raw devices. AIO on raw devices is implemented fully into the AIX kernel, and does not require database processes to service the AIO requests. When using AIO on file systems, the kernel database processes (aioserver) control each request from the time a request is taken off the queue until it completes. The kernel database processes are also used with I/O with virtual shared disks (VSDs) and HSDs with FastPath disabled. By default, FastPath is enabled. The number of aioserver servers determines the number of AIO requests that can be executed in the system concurrently, so it is important to tune the number of aioserver processes when using file systems to store Oracle Database data files.
- Use one of the following commands to set the number of servers. This applies only when using asynchronous I/O on file systems rather than raw devices: # smit aio # chdev -P -l aio0 -a maxservers='128' -a minservers='20' - To set asynchronous IO to `Available': # chdev -l aio0 -P -a autoconfig=available You need to restart the Server: # shutdown -Fr 1.2.5 aio on Linux distro's: ============================ On some Linux distro's, Oracle 9i/10g supports asynchronous I/O but it is disabled by default because some Linux distributions do not have libaio by default. For Solaris, the following configuration is not required - skip down to the section on enabling asynchronous I/O. On Linux, the Oracle binary needs to be relinked to enable asynchronous I/O. The first thing to do is shutdown the Oracle server. After Oracle has shutdown, do the following steps to relink the binary: su - oracle cd $ORACLE_HOME/rdbms/lib make -f ins_rdbms.mk async_on make -f ins_rdbms.mk ioracle
1.2.6 The ipcs and ipcrm commands: ================================== The "ipcs" command is really a "listing" command. But if you need to intervene in memory structures, like for example if you need to "clear" or remove a shared memory segment, because a faulty or crashed application left semaphores, memory identifiers, or queues in place, you can use to "ipcrm" command to remove those structures. Example ipcrm command usage: ---------------------------Suppose an application crashed, but it cannot be started again. The following might help, if you happened to know which IPC identifier it used. Suppose the app used 47500 as the IPC key. Calcultate this decimal number to hex which is, in this example, B98C.
No do the following: # ipcs -bm | grep B89C This might give you, for example, the shared memory identifier "50855977". Now clear the segment: # ipcrm -m 50855977 It might also be, that still a semaphore and/or queue is still "left over". In that case you might also try commands like the following example: ipcs -q ipcs -s # ipcrm -s 2228248 # ipcrm -q 5111883
(remove semaphore) (remove queue)
Note: in some cases the "slibclean" command can be used to clear unused modules in kernel and library memory. Just give as root the command: # slibclean Other Example: -------------If you run the following command to remove a shared memory segment and you get this error: # ipcrm -m 65537 ipcrm: 0515-020 shmid(65537) was not found. However, if you run the ipcs command, you still see the segment there: # ipcs | grep 65537 m 65537 0x00000000 DCrw------- root system If you look carefully, you will notice the "D" in the forth column. The "D" means: D If the associated shared memory segment has been removed. It disappears when the last process attached to the segment detaches it. So, to clear the shared memory segment, find the process which is still associated with the segment: # ps -ef | grep process_owner where process_owner is the name of the owner using the shared segment Now kill the process found from the ps command above
# kill -9 pid Running another ipcs command will show the shared memory segment no longer exists: # ipcs | grep 65537 Example ipcrm -m 65537
1.2.7 Show patches, version, systeminfo: ======================================== Solaris: ======== showrev: -------#showrev Displays system summary information. #showrev -p Reports which patches are installed sysdef and dmesg: ----------------The follwing commands also displays configuration information # sysdef # dmesg versions: --------==> To check your Solaris version: # uname -a or uname -m # cat /etc/release # isainfo -v ==> To check your AIX version: # oslevel # oslevel -r
tells you which maintenance level you have.
>> To find the known recommended maintenance levels: # oslevel -rq >> To find all filesets lower than a certain maintenance level: # oslevel -rl 5200-06 >> To find all filesets higher than a certain maintenance level: # oslevel -rg 5200-05
>> To list all known recommended maintenance and technology levels on the system, type: # oslevel -q -s Known Service Packs ------------------5300-05-04 5300-05-03 5300-05-02 5300-05-01 5300-05-00 5300-04-CSP 5300-04-03 5300-04-02 5300-04-01 5300-03-CSP >> Example: 5300-02 is TL 02 5300-02-04 is TL 02 and SP 04 5300-02-CSP is TL 02 and CSP for TL 02 (and there won't be anymore SPs because when you see a CSP it is because the next TL has been released. In this case it would be TL 03). >> How can I determine which fileset updates are missing from a particular AIX level? To determine which fileset updates are missing from 5300-04, for example, run the following command: # oslevel -rl 5300-04 >> What SP (Service Pack) is installed on my system? To see which SP is currently installed on the system, run the oslevel -s command. Sample output for an AIX 5L Version 5.3 system, with TL4, and SP2 installed would be: # oslevel -s 5300-04-02 >> Is a CSP (Concluding Service Pack) installed on my system? To see if a CSP is currently installed on the system, run the oslevel -s command. Sample output for an AIX 5L Version 5.3 system, with TL3, and CSP installed would be: # oslevel -s 5300-03-CSP
==> To check your HP machine: # model 9000/800/rp7410
: machine info on AIX How do I find out the Chip type, System name, Node name, Model Number etc.? The uname command provides details about your system. uname -p Displays the chip type of the system. For example, powerpc. uname -r uname -s uname -n uname -a uname -M uname -v uname -m system. uname -u
Displays Displays Displays Displays Displays Displays Displays
the the the the the the the
release number of the operating system. system name. For example, AIX. name of the node. system name, nodename,Version, Machine id. system model name. For example, IBM, 7046-B50. operating system version machine ID number of the hardware running the
Displays the system ID number.
Architecture: ------------To see if you have a CHRP machine, log into the machine as the root user, and run the following command: # lscfg | grep Architecture # lscfg -pl sysplanar0 | more
or use:
The bootinfo -p command also shows the architecture of the pSeries, RS/6000 # bootinfo -p chrp 1.2.8 Check whether you have a 32 bit or 64 bit version: ======================================================== - Solaris: # iasinfo -vk If /usr/bin/isainfo cannot be found, then the OS only supports 32-bit process address spaces. (Solaris 7 was the first version that could run 64-bit binaries on certain SPARC-based systems.) So a ksh-based test might look something like if [ -x /usr/bin/isainfo ]; then bits=`/usr/bin/isainfo -b` else bits=32 fi - AIX:
Command: /bin/lslpp -l bos.64bit installed & committed. -or/bin/locale64 32bit machine such as:
...to see if bos.64bit is ...error message if on Could not load
program /bin/locale64:
Cannot run a 64-bit
program on a 32-bit machine. Or use: # bootinfo -K "64" # bootinfo -y # bootinfo -p running the machine is
displays the current kernel wordsize of "32" or tells if hardware is 64-bit capable If it returns the string 32 it is only capable of 32-bit kernel. If it returns the string chrp the capable of running the 64-bit kernel or the 32-
bit kernel. Or use: # /usr/bin/getconf HARDWARE_BITMODE This command should return the following output: 64
Note: ----HOW TO CHANGE KERNEL MODE OF IBM AIX 5L (5.1) --------------------------------------------The AIX 5L has pre-configured kernels. These are listed below for Power processors: /usr/lib/boot/unix_up /usr/lib/boot/unix_mp /usr/lib/boot/unix_64
32 bit uni-processor 32 bit multi-processor kernel 64 bit multi-processor kernel
Switching between kernel modes means using different kernels. This is simply done by pointing the location that is referenced by the system to these kernels. Use symbolic links for this purpose. During boot AIX system runs the kernel in the following locations: /unix /usr/lib/boot/unix
The base operating system 64-bit runtime fileset is bos.64bit. Installing bos.64bit also installs the /etc/methods/cfg64 file. The /etc/methods/cfg64 file provides the option of enabling or disabling the 64-bit environment via SMIT, which updates the /etc/inittab file with the load64bit line. (Simply adding the load64bit line does not enable the 64-bit environment). The command lslpp -l bos.64bit reveals if this fileset is installed. The bos.64bit fileset is on the AIX media; however, installing the bos.64bit fileset does not ensure that you will be able to run 64-bit software. If the bos.64bit fileset is installed on 32bit hardware, you should be able to compile 64-bit software, but you cannot run 64-bit programs on 32bit hardware. The syscalls64 extension must be loaded in order to run a 64-bit executable. This is done from the load64bit entry in the inittab file. You must load the syscalls64 extension even when running a 64-bit kernel on 64-bit hardware. To determine if the 64-bit kernel extension is loaded, at the command line, enter genkex |grep 64. Information similar to the following displays: 149bf58 a3ec /usr/lib/drivers/syscalls64.ext To change the kernel mode follow steps below: 1. Create location of the 2. Create 3. Reboot
symbolic link from /unix and /usr/lib/boot/unix to the desired kernel. boot image. AIX.
Below lists the detailed actions to change kernel mode: To change to 32 bit uni-processor mode: # # # #
ln -sf /usr/lib/boot/unix_up ln -sf /usr/lib/boot/unix_up bosboot -ad /dev/ipldevice shutdown -r
/unix /usr/lib/boot/unix
To change to 32 bit multi-processor mode: # # # #
ln -sf /usr/lib/boot/unix_mp ln -sf /usr/lib/boot/unix_mp bosboot -ad /dev/ipldevice shutdown -r
/unix /usr/lib/boot/unix
To change to 64 bit multi-processor mode: # ln -sf /usr/lib/boot/unix_64
/unix
# ln -sf /usr/lib/boot/unix_64 # bosboot -ad /dev/ipldevice # shutdown -r
/usr/lib/boot/unix
IMPORTANT NOTE: If you are changing the kernel mode to 32-bit and you will run 9.2 on this server, the following line should be included in /etc/inittab: load64bit:2:wait:/etc/methods/cfg64 >/dev/console 2>&1 # Enable 64-bit execs This allows 64-bit applications to run on the 32-bit kernel. Note that this line is also mandatory if you are using the 64-bit kernel. In AIX 5.2, the 32-bit kernel is installed by default. The 64-bit kernel, along with JFS2 (enhanced journaled file system), can be enabled at installation time. Checking if other unixes are in 32 or 64 mode: ---------------------------------------------- Digital UNIX/Tru64:
This OS is only available in 64bit form.
- HP-UX(Available in 64bit starting with HP-UX 11.0): Command: /bin/getconf KERNEL_BITS ...returns either 32 or 64 - SGI:
This OS is only available in 64bit form.
- The remaining supported UNIX platforms are only available in 32bit form. scinstall: ---------# scinstall -pv Displays Sun Cluster software release and package version information 1.2.9 Info about CPUs: ====================== Solaris: -------# psrinfo -v Shows the number of processors and their status. # psrinfo -v|grep "Status of processor"|wc -l Shows number of cpu's Linux: ------
# cat /proc/cpuinfo # cat /proc/cpuinfo | grep processor|wc -l Especially with Linux, the /proc directory contains special "files" that either extract information from or send information to the kernel HP-UX: -----# # # #
ioscan -kfnC processor /usr/sbin/ioscan -kf | grep processor grep processor /var/adm/syslog/syslog.log /usr/contrib/bin/machinfo (Itanium)
Several ways as, 1. 2. 3. 4. 5. 6.
sam -> performance monitor -> processor print_manifest (if ignite-ux installed) machinfo (11.23 HP versions) ioscan -fnC processor echo "processor_count/D" | adb /stand/vmunix /dev/kmem top command to get cpu count
The "getconf" command can give you a lot of interesting info. The parameters are: ARG_MAX BS_SCALE_MAX CHARCLASS_NAME_MAX CHAR_BIT CHILD_MAX COLL_WEIGHTS_MAX CPU_CHIP_TYPE CS_PARTITION_IDENT CS_PATH HW_CPU_SUPP_BITS INT_MIN LONG_BIT MACHINE_IDENT MACHINE_SERIAL MB_LEN_MAX NL_LANGMAX NL_SETMAX OPEN_MAX _POSIX_ARG_MAX _POSIX_NGROUPS_MAX _POSIX_OPEN_MAX _POSIX_SSIZE_MAX _POSIX_STREAM_MAX _POSIX_VERSION POSIX_ARG_MAX POSIX_JOB_CONTROL POSIX_LINK_MAX POSIX_MAX_INPUT
_BC_BASE_MAX BC_STRING_MAX
BC_DIM_MAX
CHAR_MAX CLK_TCK
CHAR_MIN
CS_MACHINE_IDENT CS_MACHINE_SERIAL HW_32_64_CAPABLE KERNEL_BITS LONG_MAX MACHINE_MODEL
EXPR_NEST_MAX INT_MAX LINE_MAX LONG_MIN
NGROUPS_MAX NL_MSGMAX NL_TEXTMAX PARTITION_IDENT _POSIX_JOB_CONTROL
NL_ARGMAX NL_NMAX NZERO PATH
_POSIX_SAVED_IDS _POSIX_TZNAME_MAX POSIX_CHILD_MAX POSIX_MAX_CANON
POSIX_NAME_MAX POSIX_OPEN_MAX POSIX_PATH_MAX POSIX_SAVED_IDS POSIX_SSIZE_MAX POSIX_TZNAME_MAX POSIX_VERSION POSIX2_BC_DIM_MAX POSIX2_BC_SCALE_MAX POSIX2_C_DEV POSIX2_CHAR_TERM POSIX_CHILD_MAX POSIX2_EXPR_NEST_MAX POSIX2_FORT_DEV POSIX2_LINE_MAX POSIX2_LOCALEDEF POSIX2_UPE SC_XOPEN_VERSION SHRT_MAX
POSIX_NGROUPS_MAX POSIX_PIPE_BUF POSIX_STREAM_MAX POSIX2_BC_BASE_MAX POSIX2_BC_STRING_MAX POSIX2_C_VERSION POSIX2_COLL_WEIGHTS_MAX POSIX2_FORT_RUN POSIX2_RE_DUP_MAX POSIX2_VERSION SCHAR_MAX SHRT_MIN
Example: # getconf CPU_VERSION sample function in shell script: get_cpu_version() { case `getconf CPU_VERSION` in # ???) echo "Itanium[TM] 2" ;; 768) echo "Itanium[TM] 1" ;; 532) echo "PA-RISC 2.0" ;; 529) echo "PA-RISC 1.2" ;; 528) echo "PA-RISC 1.1" ;; 523) echo "PA-RISC 1.0" ;; *) return 1 ;; esac return 0
AIX: ---# pmcycles Cpu 0 runs Cpu 1 runs Cpu 2 runs Cpu 3 runs
-m at at at at
1656 1656 1656 1656
MHz MHz MHz MHz
# lscfg | grep proc More cpu information on AIX:
POSIX2_C_BIND
POSIX2_SW_DEV SC_PASS_MAX SCHAR_MIN SSIZE_MAX
# lsattr -El procx (where x is the number of the cpu) type powerPC_POWER5 Processor type False frequency 165600000 Processor speed False .. .. where False means that the value cannot be changed through an AIX command. # lparstat # lparstat -i
(only for latest AIX versions)
To view CPU scheduler tunable parameters, use the schedo command: # schedo -a In AIX 5L on Power5, you can switch from Simultaneous Multithreading SMT, or Single Threading ST, as follows (smtcl) # smtctl -m off will set SMT mode to disabled # smtctl -m on will set SMT mode to enabled # smtctl -W boot makes SMT effective on next boot # smtctl -W now effects SMT now, but will not persist across reboots When you want to keep the setting across reboots, you must use the bosboot command in order to create a new boot image. 1.2.10 Other stuff: =================== runlevel: --------To show the init runlevel: # who -r Top users: ---------To get a quick impression about the top 10 users in the system at this time: ps auxw | sort -r +3 |head -10 ps auxw | sort -r +2 |head -10
-Shows top 10 memory usage by process -Shows top 10 CPU usage by process
More accuracy in memory usage with the ps command: ps -vg ps -vg: ------Using "ps vg" gives a per process tally of memory usage for each running process. Several fields give memory usage
in different units, but these numbers do not tell the whole story on where all the memory goes. First of all, the man page for ps does not give an accurate description of the memory related fields. Here is a better description: RSS - This tells for the text and for a particular be a multiple of
how much RAM resident memory is currently being used data segments process in units of kilobytes. (this value will always 4 since memory is allocated in 4 KB pages).
%MEM - This is the fraction of RSS divided by the total size of RAM for a particular process. Since RSS is some subset of the total resident memory usage for a process, the %MEM value will also be lower than actual. TRS - This tells how much RAM resident memory is currently being used for the text segment for a particular process in units of kilobytes. This will always be less than or equal to RSS. SIZE - This tells how much paging space is allocated for this process for the text and data segments in units of kilobytes. If the executable file is on a local filesystem, the page space usage for text is zero. If the executable is on an NFS filesystem, the page space usage will be nonzero. This number may be greater than RSS, or it may not, depending on how much of the process is paged in. The reason RSS can be larger is that RSS counts text whereas SIZE does not. TSIZ - This field is absolutely bogus because it is not a multiple of 4 and does not correlate to any of the other fields. These fields only report on a process text and data segments. Segment size which cannot be interrogated at this time are: Text portion of shared libraries (segment 13) Files that are in use. Open files are cached in memory as individual segments. Shared data segments created with shmat. Kernel segments such as kernel segment 0, kernel extension segments, and virtual memory management segments. In summary, ps is not a very good tool to measure system memory usage. It can give you some idea where some of the memory goes, but it leaves too many questions unanswered about the total usage.
shared memory: --------------
To check shared memory segment, semaphore array, and message queue limits, issue the ipcs -l command. # ipcs The following tools are available for monitoring the performance of your UNIX-based system. pfiles: ------/usr/proc/bin/pfiles This shows the open files for this process, which helps you diagnose whether you are having problems caused by files not getting closed. lsof: ----This utility lists open files for running UNIX processes, like pfiles. However, lsof gives more useful information than pfiles. You can find lsof at ftp://vic.cc.purdue.edu/pub/tools/unix/lsof/. Example of lsof usage: You can see CIO (concurrent IO) in the FILE-FLAG column if you run lsof +fg, e.g.: tarunx01:/home/abielewi:# /p570build/LSOF/lsof-4.76/usr/local/bin/lsof +fg /baanprd/oradat COMMAND PID USER FD TYPE FILE-FLAG DEVICE SIZE/OFF NODE NAME oracle 434222 oracle 16u VREG R,W,CIO,DSYN,LG;CX 39,1 6701056 866 /baanprd/oradat (/dev/bprdoradat) oracle 434222 oracle 17u VREG R,W,CIO,DSYN,LG;CX 39,1 6701056 867 /baanprd/oradat (/dev/bprdoradat) oracle 442384 oracle 15u VREG R,W,CIO,DSYN,LG;CX 39,1 1174413312 875 /baanprd/oradat (/dev/bprdoradat) oracle 442384 oracle 16u VREG R,W,CIO,DSYN,LG;CX 39,1 734011392 877 /baanprd/oradat (/dev/bprdoradat) oracle 450814 oracle 15u VREG R,W,CIO,DSYN,LG;CX 39,1 1174413312 875 /baanprd/oradat (/dev/bprdoradat) oracle 450814 oracle 16u VREG R,W,CIO,DSYN,LG;CX 39,1 1814044672 876 /baanprd/oradat (/dev/bprdoradat) oracle 487666 oracle 15u VREG R,W,CIO,DSYN,LG;CX 39,1 1174413312 875 /baanprd/oradat (/dev/bprdoradat You should also see O_CIO in your file open calls if you run truss, e.g.: open("/opt/oracle/rcat/oradat/redo01.log", O_RDWR|O_CIO|O_DSYNC|O_LARGEFILE) = 18
VMSTAT SOLARIS:
--------------# vmstat This command is ideal for monitoring paging rate, which can be found under the page in (pi) and page out (po) columns. Other important columns are the amount of allocated virtual storage (avm) and free virtual storage (fre). This command is useful for determining if something is suspended or just taking a long time. Example: kthr memory page disk faults cpu r b w swap free re mf pi po fr de sr m0 m1 m3 m4 in sy cs us sy id 0 0 0 2163152 1716720 157 141 1179 1 1 0 0 0 0 0 0 680 1737 855 10 3 87 0 0 0 2119080 1729352 0 1 0 0 0 0 0 0 0 1 0 345 658 346 1 1 98 0 0 0 2118960 1729232 0 167 0 0 0 0 0 0 0 0 0 402 1710 812 4 2 94 0 0 0 2112992 1723264 0 1261 0 0 0 0 0 0 0 0 0 1026 5253 1848 10 5 85 0 0 0 2112088 1722352 0 248 0 0 0 0 0 0 0 0 0 505 2822 1177 5 2 92 0 0 0 2116288 1726544 4 80 0 0 0 0 0 0 0 0 0 817 4015 1530 6 4 90 0 0 0 2117744 1727960 4 2 30 0 0 0 0 0 0 0 0 473 1421 640 2 2 97 procs/r: Run queue length. procs/b: Processes blocked while waiting for I/O. procs/w: Idle processes which have been swapped. memory/swap: Free, unreserved swap space (Kb). memory/free: Free memory (Kb). (Note that this will grow until it reaches lotsfree, at which point the page scanner is started. See "Paging" for more details.) page/re: Pages reclaimed from the free list. (If a page on the free list still contains data needed for a new request, it can be remapped.) page/mf: Minor faults (page in memory, but not mapped). (If the page is still in memory, a minor fault remaps the page. It is comparable to the vflts value reported by sar -p.) page/pi: Paged in from swap (Kb/s). (When a page is brought back from the swap device, the process will stop execution and wait. This may affect performance.) page/po: Paged out to swap (Kb/s). (The page has been written and freed. This can be the result of activity by the pageout scanner, a file close, or fsflush.) page/fr: Freed or destroyed (Kb/s). (This column reports the activity of the page scanner.) page/de: Freed after writes (Kb/s). (These pages have been freed due to a pageout.)
page/sr: Scan rate (pages). Note that this number is not reported as a "rate," but as a total number of pages scanned. disk/s#: Disk activity for disk # (I/O's per second). faults/in: Interrupts (per second). faults/sy: System calls (per second). faults/cs: Context switches (per second). cpu/us: User CPU time (%). cpu/sy: Kernel CPU time (%). cpu/id: Idle + I/O wait CPU time (%). When analyzing vmstat output, there are several metrics to which you should pay attention. For example, keep an eye on the CPU run queue column. The run queue should never exceed the number of CPUs on the server. If you do notice the run queue exceeding the amount of CPUs, it's a good indication that your server has a CPU bottleneck. To get an idea of the RAM usage on your server, watch the page in (pi) and page out (po) columns of vmstat's output. By tracking common virtual memory operations such as page outs, you can infer the times that the Oracle database is performing a lot of work. Even though UNIX page ins must correlate with the vmstat's refresh rate to accurately predict RAM swapping, plotting page ins can tell you when the server is having spikes of RAM usage. Once captured, it's very easy to take the information about server performance directly from the Oracle tables and plot them in a trend graph. Rather than using an expensive statistical package such as SAS, you can use Microsoft Excel. Copy and paste the data from the tables into Excel. After that, you can use the Chart Wizard to create a line chart that will help you view server usage information and discover trends. # VMSTAT AIX: ------------This is virtually equal to the usage of vmstat under solaris. vmstat can be used to give multiple statistics on the system. For CPUspecific work, try the following command: # vmstat -t 1 3 This will take 3 samples, 1 second apart, with timestamps (-t). You can, of course, change the parameters as you like. The output is shown below. time
kthr
memory
page
faults
----- ----------- ------------------------ ---------------------- --------
cpu
r b avm wa hr mi se 0 0 45483 0 15:10:22 0 0 45483 0 15:10:23 2 0 45483 1 15:10:24
fre
re
pi
po
fr
sr
cy
in
sy
cs us sy id
221
0
0
0
0
1
0 224
220
0
0
0
0
0
0 159
83
53
1
1 98
220
0
0
0
0
0
0 145
115
46
0
9 90
326 362 24
7 69
In this output some of the things to watch for are: "avm", which is Active Virtual Memory. Ideally, under normal conditions, the largest avm value should in general be smaller than the amount of RAM. If avm is smaller than RAM, and still exessive paging occurs, that could be due to RAM being filled with file pages. avm x 4K = number of bytes Columns r (run queue) and b (blocked) start going up, especially above 10. This usually is an indication that you have too many processes competing for CPU. If cs (contact switches) go very high compared to the number of processes, then you may need to tune the system with vmtune. In the cpu section, us (user time) indicates the time is being spent in programs. Assuming Java is at the top of the list in tprof, then you need to tune the Java application). In the cpu section, if sys (system time) is higher than expected, and you still have id (idle) time left, this may indicate lock contention. Check the tprof for lock related calls in the kernel time. You may want to try multiple instances of the JVM. It may also be possible to find deadlocks in a javacore file. In the cpu section, if wa (I/O wait) is high, this may indicate a disk bottleneck, and you should use iostat and other tools to look at the disk usage. Values in the pi, po (page in/out) columns that you are paging and need more memory. It may be possible that you have the stack of your JVM instances. It could also mean that you have allocated amount of memory on the system. Of course, you may also have other applications using may be taking up too much of the memory Other example:
are non-zero may indicate size set too high for some a heap larger than the memory, or that file pages
-------------# vmstat 1 System configuration: lcpu=2 mem=3920MB kthr ----r b 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
memory ----------avm fre 229367 332745 229367 332745 229367 332745 229367 332745 229367 332745 229367 332745 229367 332745 229367 332745 229367 332745 229367 332745 229367 332745
page faults cpu ------------------------ ------------ ----------re pi po fr sr cy in sy cs us sy id wa 0 0 0 0 0 0 3 198 69 0 0 99 0 0 0 0 0 0 0 3 33 66 0 0 99 0 0 0 0 0 0 0 2 33 68 0 0 99 0 0 0 0 0 0 0 80 306 100 0 1 97 1 0 0 0 0 0 0 1 20 68 0 0 99 0 0 0 0 0 0 0 2 36 64 0 0 99 0 0 0 0 0 0 0 2 33 66 0 0 99 0 0 0 0 0 0 0 2 21 66 0 0 99 0 0 0 0 0 0 0 1 237 64 0 0 99 0 0 0 0 0 0 0 2 19 66 0 0 99 0 0 0 0 0 0 0 6 37 76 0 0 99 0
The most important fields to look at here are: r -- The average number of runnable kernel threads over whatever sampling interval you have chosen. b -- The average number of kernel threads that are in the virtual memory waiting queue over your sampling interval. r should always be higher than b; if it is not, it usually means you have a CPU bottleneck. fre -- The size of your memory free list. Do not worry so much if the amount is really small. More importantly, determine if there is any paging going on if this amount is small. pi -- Pages paged in from paging space. po -- Pages paged out to paging space. CPU section: us sy id wa Let's look at the last section, which also comes up in most other CPU monitoring tools, albeit with different headings: us sy id wa
-----
user time system time idle time waiting on I/O
# IOSTAT: --------This command is useful for monitoring I/O activities. You can use the read and write rate to estimate the
amount of time required for certain SQL operations (if they are the only activity on the system). This command is also useful for determining if something is suspended or just taking a long time. Basic synctax is iostat
interval
count
option - let you specify the device for which information is needed like disk , cpu or terminal. (-d , -c , -t or -tdc ) . x options gives the extended statistics . interval - is time period in seconds between two samples . iostat will give data at each 4 seconds interval. count - is the number of times the data is needed . give data at 4 seconds interval 5 times.
4
iostat 4 5 will
Example: $ iostat -xtc 5 2 id 0
disk r/s
extended disk statistics w/s Kr/s Kw/s wait actv svc_t %w %b
sd0
2.6 3.0 20.7 22.7 0.1
0.2
59.2
6
19
sd1 sd2 sd3
4.2 1.0 33.5 8.0 0.0 0.0 0.0 0.0 0.0 0.0 10.2 1.6 51.4 12.8 0.1
0.2 0.0 0.3
47.2 0.0 31.2
2 0 3
23 0 31
tty cpu tin tout us sy wt 0
84
3
85 11
disk r/s w/s Kr/s Kw/s wait actv
name of the disk reads per second writes per second kilobytes read per second kilobytes written per second average number of transactions waiting for service (Q length) average number of transactions actively being serviced (removed from the queue but not yet completed) %w percent of time there are transactions waiting for service (queue non-empty) %b percent of time the disk is busy (transactions in progress) The values to look from the iostat output
are:
Reads/writes per second (r/s , w/s) Percentage busy (%b) Service time (svc_t) If a disk shows consistently high reads/writes along with , the percentage busy (%b) of the disks is greater than 5 percent, and the average service time (svc_t) is greater than 30 milliseconds, then action needs to be taken. # netstat This command lets you know the network traffic on each node, and the number of error packets encountered.
It is useful for isolating network problems. Example: To find out all listening services, you can use the command # netstat -a -f inet
1.2.11 Some other utilities for Solaris: ======================================== # top For example: load averages: 0.66, 0.54, 0.56 11:14:48 187 processes: 185 sleeping, 2 on cpu CPU states: % idle, % user, % kernel, % iowait, swap Memory: 4096M real, 1984M free, 1902M swap in use, 2038M swap free PID 2795 2294 13907 14138 2797 2787 2799 2743 2011 2007 2009 2804 2013 2035 114
USERNAME THR PRI NICE SIZE RES STATE oraclown 1 59 0 265M 226M sleep root 11 59 0 8616K 7672K sleep oraclown 11 59 0 271M 218M cpu2 oraclown 12 59 0 270M 230M sleep oraclown 1 59 0 189M 151M sleep oraclown 11 59 0 191M 153M sleep oraclown 1 59 0 190M 151M sleep oraclown 11 59 0 191M 155M sleep oraclown 11 59 0 191M 149M sleep oraclown 11 59 0 191M 149M sleep oraclown 11 59 0 191M 149M sleep oraclown 1 51 0 1760K 1296K cpu2 oraclown 11 59 0 191M 148M sleep oraclown 11 59 0 191M 149M sleep root 10 59 0 5016K 4176K sleep
TIME 0:13 10:54 4:02 9:03 0:01 0:06 0:02 0:25 2:50 2:22 1:54 0:00 0:36 2:44 23:34
CPU 4.38% 3.94% 2.23% 1.76% 0.96% 0.69% 0.45% 0.35% 0.27% 0.26% 0.20% 0.19% 0.14% 0.13% 0.05%
%
COMMAND oracle bpbkar oracle oracle oracle oracle oracle oracle oracle oracle oracle top oracle oracle picld
Process ID This column shows the process ID (pid) of each process. The process ID is a positive number, usually less than 65536. It is used for identification during the life of the process. Once a process has exited or been killed, the process ID can be reused. Username This column shows the name of the user who owns the process. The kernel stores this information as a uid, and top uses an appropriate table (/etc/passwd, NIS, or NIS+) to translate this uid in to a name. Threads This column displays the number of threads for the current process. This column is present only in the Solaris 2 port of top.
For Solaris, this number is actually the number of lightweight processes (lwps) created by the threads package to handle the threads. Depending on current resource utilization, there may not be one lwp for every thread. Thus this number is actually less than or equal to the total number of threads created by the process. Nice This column reflects the "nice" setting of each process. A process's nice is inhereted from its parent. Most user processes run at a nice of 0, indicating normal priority. Users have the option of starting a process with a positive nice value to allow the system to reduce the priority given to that process. This is normally done for long-running cpu-bound jobs to keep them from interfering with interactive processes. The Unix command "nice" controls setting this value. Only root can set a nice value lower than the current value. Nice values can be negative. On most systems they range from -20 to 20. The nice value influences the priority value calculated by the Unix scheduler. Size This column shows the total amount of memory allocated by each process. This is virtual memory and is the sum total of the process's text area (program space), data area, and dynamically allocated area (or "break"). When a process allocates additional memory with the system call "brk", this value will increase. This is done indirectly by the C library function "malloc". The number in this column does not reflect the amount of physical memory currently in use by the process. Resident Memory This column reflects the amount of physical memory currently allocated to each process. This is also known as the "resident set size" or RSS. A process can have a large amount of virtual memory allocated (as indicated by the SIZE column) but still be using very little physical memory. Process State This column reflects the last observed state of each process. State names vary from system to system. These states are analagous to those that appear in the process states line: the second line of the display. The more common state names are listed below. cpu - Assigned to a CPU and currently running run - Currently able to run sleep - Awaiting an external event, such as input from a device stop - Stopped by a signal, as with control Z swap - Virtual address space swapped out to disk zomb - Exited, but parent has not called "wait" to receive the exit status
CPU Time This column displayes the accumulated CPU time for each process. This is the amount of time that any cpu in the system has spent actually running this process. The standard format shows two digits indicating minutes, a colon, then two digits indicating seconds. For example, the display "15:32" indicates fifteen minutes and thirtytwo seconds. When a time value is greater than or equal to 1000 minutes, it is displayed as hours with the suffix H. For example, the display "127.4H" indicates 127 hours plus four tenths of an hour (24 minutes). When the number of hours exceeds 999.9, the "H" suffix is dropped so that the display continues to fit in the column. CPU Percentage This column shows the percentage of the cpu that each process is currently consuming. By default, top will sort this column of the output. Some versions of Unix will track cpu percentages in the kernel, as the figure is used in the calculation of a process's priority. On those versions, top will use the figure as calculated by the kernel. Other versions of Unix do not perform this calculation, and top must determine the percentage explicity by monitoring the changes in cpu time. On most multiprocessor machines, the number displayed in this column is a percentage of the total available cpu capacity. Therefore, a single threaded process running on a four processor system will never use more than 25% of the available cpu cycles. Command This column displays the name of the executable image that each process is running. In most cases this is the base name of the file that was invoked with the most recent kernel "exec" call. On most systems, this name is maintained separately from the zeroth argument. A program that changes its zeroth argument will not affect the output of this column.
# modinfo The modinfo command provides information about the modules currently loaded by the kernel. The /etc/system file: Available for Solaris Operating Environment, the /etc/system file contains definitions for kernel configuration limits such as the maximum number of users allowed on the system at a time, the maximum number of processes per user, and the inter-process communication (IPC) limits on size and number of resources. These limits are important because
they affect DB2 performance on a Solaris Operating Environment machine. See the Quick Beginnings information for further details. # more /etc/path_to_inst To see the mapping between the kernel abbreviated instance name for physical device names, view the /etc/path_to_inst file. # uptime uptime - show how long the system has been up /export/home/oraclown>uptime 11:32am up 4:19, 1 user,
load average: 0.40, 1.17, 0.90
1.2.12 proc toos for Solaris: ============================= The proc tools are called that way, because the retreive information fromn the /proc virtual filesystem They are: /usr/proc/bin/pflags /usr/proc/bin/pcred /usr/proc/bin/pmap /usr/proc/bin/pldd /usr/proc/bin/psig /usr/proc/bin/pstack /usr/proc/bin/pfiles /usr/proc/bin/pwdx /usr/proc/bin/pstop /usr/proc/bin/prun /usr/proc/bin/pwait /usr/proc/bin/ptree /usr/proc/bin/ptime /usr/proc/bin/pattr /usr/proc/bin/pclear /usr/proc/bin/plabel /usr/proc/bin/ppriv
[-r] pid... pid... [-rxlF] pid... [-F] pid... pid... [-F] pid... [-F] pid... [-F] pid... pid... pid... [-v] pid... [-a] [[pid| user]...] command [arg...] [-x ] [pid...] [pid...] [pid...] [-a] [pid...]
-- pfiles: reports all the files which are opened by a given pid -- pldd lists all the dynamic libraries linked to the process -- pwdx gives the directory from which the process is running -- ptree The ptree utility prints the process trees containing the specified pids or users, with child processes indented from their respective parent processes. An argument of all digits is taken to be a process-ID,
otherwise it is assumed to be a user login name. The default is all processes. Use it like # ptree Or use it with params, which enables you to produce different listings The following example prints the process tree (including children of process 0) for processes which match the command name ssh: $ ptree -a `pgrep ssh` 1 /sbin/init 100909 /usr/lib/ssh/sshd 569150 /usr/lib/ssh/sshd 569157 /usr/lib/ssh/sshd 569159 -ksh 569171 bash 569173 /bin/ksh 569193 bash ---------------------------------------------------------------------Remark: many Linux distros adopted the ptree command, as the "pstree" command. As in ubuntu$ pstree -pl init(1)---NetworkManager(5427) +-NetworkManagerD(5441) +-acpid(5210) +-apache2(6966)---apache2(2890) Ý +-apache2(2893) Ý +-apache2(7163) Ý +-apache2(7165) Ý +-apache2(7166) Ý +-apache2(7167) Ý +-apache2(7168) +-atd(6369) +-avahi-daemon(5658)---avahi-daemon(5659) +-bonobo-activati(7816)---{bonobo-activati}(7817) etc.. .. ----------------------------------------------------------------------Back to Solaris again: Suppose you did a pfiles on an Apache process: # pfiles 13789
13789: /apps11i/erpdev/10GAS/Apache/Apache/bin/httpd -d /apps11i/erpdev/10G Current rlimit: 1024 file descriptors 0: S_IFIFO mode:0000 dev:350,0 ino:114723 uid:65060 gid:54032 size:301 O_RDWR 1: S_IFREG mode:0640 dev:307,28001 ino:612208 uid:65060 gid:54032 size:386 O_WRONLY|O_APPEND|O_CREAT /apps11i/erpdev/10GAS/opmn/logs/HTTP_Server~1 2: S_IFIFO mode:0000 dev:350,0 ino:143956 uid:65060 gid:54032 size:0 O_RDWR 3: S_IFREG mode:0600 dev:307,28001 ino:606387 uid:65060 gid:54032 size:1056768 O_RDWR|O_CREAT /apps11i/erpdev/10GAS/Apache/Apache/logs/mm.19389.mem 4: S_IFREG mode:0600 dev:307,28001 ino:606383 uid:65060 gid:54032 size:0 O_RDWR|O_CREAT 5: S_IFREG mode:0600 dev:307,28001 ino:621827 uid:65060 gid:54032 size:1056768 O_RDWR|O_CREAT 6: S_IFDOOR mode:0444 dev:351,0 ino:58 uid:0 gid:0 size:0 O_RDONLY|O_LARGEFILE FD_CLOEXEC door to nscd[421] /var/run/name_service_door 7: S_IFIFO mode:0000 dev:350,0 ino:143956 uid:65060 gid:54032 size:0 O_RDWR 8: S_IFCHR mode:0666 dev:342,0 ino:47185924 uid:0 gid:3 rdev:90,0 O_RDONLY /devices/pseudo/kstat@0:kstat etc.. .. .. O_RDWR|O_CREAT /apps11i/erpdev/10GAS/Apache/Apache/logs/dms_metrics.19389.shm.sem 21: S_IFREG mode:0600 dev:307,28001 ino:603445 uid:65060 gid:54032 size:17408 O_RDONLY FD_CLOEXEC /apps11i/erpdev/10GAS/rdbms/mesg/ocius.msb 23: S_IFSOCK mode:0666 dev:348,0 ino:60339 uid:0 gid:0 size:0 O_RDWR SOCK_STREAM SO_SNDBUF(49152),SO_RCVBUF(49152),IP_NEXTHOP(0.0.192.0) sockname: AF_INET 3.56.189.4 port: 45395 peername: AF_INET 3.56.189.4 port: 12501 256: S_IFREG mode:0444 dev:85,0 ino:234504 uid:0 gid:3 size:1616 O_RDONLY|O_LARGEFILE /etc/inet/hosts Suppose you tried pldd on the same process gave this result: # pldd 13789 13789: /apps11i/erp dev/10GAS/Apache/Apache/bin/httpd -d /apps11i/erpdev/10G /apps11i/erpdev/10GAS/lib32/libdms2.so /lib/libpthread.so.1 /lib/libsocket.so.1
/lib/libnsl.so.1 /lib/libdl.so.1 /lib/libc.so.1 /platform/sun4u-us3/lib/libc_psr.so.1 /lib/libmd5.so.1 /platform/sun4u/lib/libmd5_psr.so.1 /lib/libscf.so.1 /lib/libdoor.so.1 /lib/libuutil.so.1 /lib/libgen.so.1 /lib/libmp.so.2 /lib/libm.so.2 /lib/libresolv.so.2 /apps11i/erpdev/10GAS/Apache/Apache/libexec/mod_onsint.so /lib/librt.so.1 /apps11i/erpdev/10GAS/lib32/libons.so /lib/libkstat.so.1 /lib/libaio.so.1 /apps11i/erpdev/10GAS/Apache/Apache/libexec/mod_mmap_static.so /apps11i/erpdev/10GAS/Apache/Apache/libexec/mod_vhost_alias.so /apps11i/erpdev/10GAS/Apache/Apache/libexec/mod_env.so .. .. etc /usr/lib/libsched.so.1 /apps11i/erpdev/10GAS/lib32/libclntsh.so.10.1 /apps11i/erpdev/10GAS/lib32/libnnz10.so /apps11i/erpdev/10GAS/Apache/Apache/libexec/mod_wchandshake.so /apps11i/erpdev/10GAS/Apache/Apache/libexec/mod_oc4j.so /apps11i/erpdev/10GAS/Apache/Apache/libexec/mod_dms.so /apps11i/erpdev/10GAS/Apache/Apache/libexec/mod_rewrite.so /apps11i/erpdev/10GAS/Apache/oradav/lib/mod_oradav.so /apps11i/erpdev/10GAS/Apache/modplsql/bin/modplsql.so # pmap -x $$ 492328: -ksh Address Kbytes 00010000 192 00040000 8 00042000 40 FF180000 680 FF23A000 24 FF240000 8 FF280000 576 FF310000 40 FF31A000 24 FF350000 16 FF364000 8 FF380000 40 FF39A000 8 FF3A0000 8 FF3B0000 8 FF3C0000 152 FF3F6000 8
RSS 192 8 40 680 24 8 576 40 16 16 8 40 8 8 8 152 8
Anon 8 8 8 8 8
Locked -
Mode r-x-rwx-rwx-r-x-rwx-rwx-r-x-rwx-rwx-r-x-rwx-r-x-rwx-r-x-rwx-r-x-rwx--
Mapped File ksh ksh [ heap ] libc.so.1 libc.so.1 libc.so.1 libnsl.so.1 libnsl.so.1 libnsl.so.1 libmp.so.2 libmp.so.2 libsocket.so.1 libsocket.so.1 libdl.so.1 [ anon ] ld.so.1 ld.so.1
FFBFC000 16 16 8 - rw---------- ------- ------- ------- ------total Kb 1856 1848 48 -
[ stack ]
1.2.13 Wellknown tools for AIX: =============================== 1. commands: -----------CPU Memory Subsystem I/O Subsystem Network Subsystem -------------------------------------------------------------------------------vmstat vmstat iostat netstat iostat lsps vmstat ifconfig ps svmon lsps tcpdump sar filemon filemon tprof ipcs lvmstat nmon and topas can be used to monitor those subsystems in general. 2. topas: --------topas is a useful graphical interface that will give you immediate results of what is going on in the system. When you run it without any command-line arguments, the screen looks like this: Topas Monitor for host: FILE/TTY Mon Apr 16 16:16:50 2001 4864 Writech Kernel 0 User 0 Wait 0 Idle 4
aix4prt Interval:
EVENTS/QUEUES 2
Cswitch
5984
Syscall
15776
Readch
34280 63.1 |##################
|
Reads
36.8
|##########
|
Writes
0.0
|
|
Forks
0
Igets
0.0
|
|
Execs
0
Namei
0 Network KBPS lo0 213.9 tr0 34.7 Real,MB 1023 Disk Busy% 27.0
I-Pack 2154.2 16.9 KBPS
O-Pack 2153.7 34.4
KB-In 107.0 0.9
KB-Out 106.9 33.8
TPS KB-Read KB-Writ
8 2469
Runqueue
11.5
Waitqueue
0.0
PAGING Faults
3862
Steals
1580
Rawin Ttyout
Dirblk
MEMORY % Comp
hdisk0 Noncomp
0.0 73.9
Client Name java SPACE java Size,MB lrud 1.2 aixterm 98.7 topas ksh Press: gil for help
0.5
0.0
0.0
0.0
0.0
PgspIn
0
%
PgspOut
0
%
PID CPU% PgSp Owner 16684 83.6 35.1 root
PageIn PageOut
0 0
PAGING
12192 12.7 86.2 root 512 1032 2.7 0.0 root
Sios
0
19502
0.5
0.7 root
NFS (calls/sec)
6908 18148
0.5 0.0
0.8 root 0.7 root
ServerV2 ClientV2
0 0
1806
0.0
0.0 root
ServerV3
0
% Used % Free
"h"
The information on the bottom left side shows the most active processes; here, java is consuming 83.6% of CPU. The middle right area shows the total physical memory (1 GB in this case) and Paging space (512 MB), as well as the amount being used. So you get an excellent overview of what the system is doing in a single screen, and then you can select the areas to concentrate based on the information being shown here. Note: about waits: -----------------Don't get caught up in this whole wait i/o thing. a single cpu system with 1 i/o outstanding and no other runable threads (i.e. idle) will have 100% wait i/o. There was a big discussion a couple of years ago on removing the kernel tick as it has confused many many many techs. So, if you have only 1 or few cpu, then you are going to have high wait i.o figures, it does not neccessarily mean your disk subsystem is slow.
3. trace: --------trace captures a sequential flow of time-stamped system events. The trace is a valuable tool for observing system and application execution. While many of the other tools provide high level statistics such as CPU and I/O utilization, the trace facility helps expand the information as to where the events happened, which process is responsible, when the events took place, and how they are affecting the system. Two post processing tools that can extract information from the trace are utld (in AIX 4) and curt
(in AIX 5). These provide statistics on CPU utilization and process/thread activity. The third post processing tool is splat which stands for Simple Performance Lock Analysis Tool. This tool is used to analyze lock activity in the AIX kernel and kernel extension for simple locks. 4. nmon: -------nmon is a free software tool that gives much of the same information as topas, but saves the information to a file in Lotus 123 and Excel format. The download site is http://www.ibm.com/developerworks/eserver/articles/analyze_aix/. The information that is collected included CPU, disk, network, adapter statistics, kernel counters, memory and the "top" process information. 5. tprof: --------tprof is one of the AIX legacy tools that provides a detailed profile of CPU usage for every AIX process ID and name. It has been completely rewritten for AIX 5.2, and the example below uses the AIX 5.1 syntax. You should refer to AIX 5.2 Performance Tools update: Part 3 for the new syntax. The simplest way to invoke this command is to use: # tprof -kse -x "sleep 10" # tprof -ske -x "sleep 30" At the end of ten seconds, or 30 seconds, a new file __prof.all, or sleep.prof, is generated that contains information about what commands are using CPU on the system. Searching for FREQ, the information looks something like the example below:
...
Process ======= oracle java wait
FREQ === 244 247 16
Total Kernel ===== ====== 10635 3515 3970 617 1515 1515
User Shared ==== ====== 6897 223 0 2062 0 0
Other ===== 0 1291 0
======= Total
=== 1060
===== ====== 19577 7947
==== ====== 7252 3087
===== 1291
This example shows that over half the CPU time is associated with the oracle application and that Java is using about 3970/19577 or 1/5 of the CPU. The wait usually means idle time, but can also include the I/O wait portion of the CPU usage.
svmon: -----The svmon command captures a snapshot of the current state om memory. use it with the -G switch to get global statistics for the whole system. svmon is the most useful tool at your disposal when monitoring a Java process, especially native heap. The article "When segments collide" gives examples of how to use svmon -P -m to monitor the native heap of a Java process on AIX. But there is another variation, svmon -P -m -r, that is very effective in identifying native heap fragmentation. The -r switch prints the address range in use, so it gives a more accurate view of how much of each segment is in use. As an example, look at the partially edited output below: Pid Command LPage 10556 java N Vsid Pgsp Virtual 22ac4 21047 126a2 7908c b2ad6 b1475 282 65536 30fe5 285 65536 91072 54 65536 6bced 261 65536 b1035 0 45054 e0f9f 3 48284 19100 463 47210 c965a 281 46953 7910c 0 37070 0
e801d 9220
Inuse 681613
Pin 2316
Pgsp
Esid Type Description
2461
Virtual 64-bit Mthrd 501080
N
Y
LPage
Inuse
Pin
9 mmap mapped to sid b1475
-
0
0
8 mmap mapped to sid 30fe5
-
0
0
a mmap mapped to sid 91072
-
0
0
7 mmap mapped to sid 6bced
-
0
0
b mmap mapped to sid b1035
-
0
0
- work
-
65536
0
- work
-
65536
0
- work
-
65536
0
- work
-
65536
0
- work
-
45054
0
Addr Range: 0..45055 5 work shmat/mmap
-
48284
0
3 work shmat/mmap
-
46997
0
4 work shmat/mmap
-
46835
0
6 work shmat/mmap
-
37070
0
Addr Range: 0..50453 d work shared library text
-
9172
0
1
a0fb7 106
1
21127 51
-
a8535 -
Addr Range: 0..30861 f work shared library data
-
105
0
Addr Range: 0..2521 2 work process private
-
50
2
Addr Range: 65300..65535 1 pers code,/dev/q109waslv:81938
-
11
0
Addr Range: 0..11
Other example: # svmon -G -i 2 5 memory size inuse inuse 16384 16250 12674 16384 16250 12674 16384 16250 12674 16384 16250 12674 16384 16250 12674
# sample five times at two second intervals
free pin
in use work pers
clnt
work
pin pers
clnt
pg space size
134
2006
10675 2939
2636
2006
0
0
40960
134
2006
10675 2939
2636
2006
0
0
40960
134
2006
10675 2939
2636
2006
0
0
40960
134
2006
10675 2939
2636
2006
0
0
40960
134
2006
10675 2939
2636
2006
0
0
40960
In this example, there are 16384 pages of total size of memory. Multuply this number by 4096 to see the total real memory size. In this case the total memory is 64 MB.
filemon: -------filemon can be used to identify the files that are being used most actively. This tool gives a very comprehensive view of file access, and can be useful for drilling down once vmstat/iostat confirm disk to be a bottleneck. Example: # filemon -o /tmp/filemon.log; sleep 60; trcstop The generated log file is quite large. Some sections that may be useful are: Most Active Files ----------------------------------------------------------------------#MBs #opns #rds #wrs file volume:inode
----------------------------------------------------------------------25.7 83 6589 0 unix /dev/hd2:147514 16.3 1 4175 0 vxe102 /dev/mailv1:581 16.3 1 0 4173 .vxe102.pop /dev/poboxv:62 15.8 1 1 4044 tst1 /dev/mailt1:904 8.3 2117 2327 0 passwd /dev/hd4:8205 3.2 182 810 1 services /dev/hd4:8652 ... ----------------------------------------------------------------------Detailed File Stats ----------------------------------------------------------------------FILE: /var/spool/mail/v/vxe102 volume: /dev/mailv1 (/var/spool2/mail/v) inode: 581 opens: 1 total bytes xfrd: 17100800 reads: 4175 (0 errs) read sizes (bytes): avg 4096.0 min 4096 max 0.0 read times (msec): avg 0.543 min 0.011 max 2.753 ...
4096 sdev 78.060 sdev
curt: ----curt Command Purpose The CPU Utilization Reporting Tool (curt) command converts an AIX trace file into a number of statistics related to CPU utilization and either process, thread or pthread activity. These statistics ease the tracking of specific application activity. curt works with both uniprocessor and multiprocessor AIX Version 4 and AIX Version 5 traces. Syntax curt -i inputfile [-o outputfile] [-n gennamesfile] [-m trcnmfile] [-a pidnamefile] [-f timestamp] [-l timestamp] [-ehpstP] Description The curt command takes an AIX trace file as input and produces a number of statistics related to processor (CPU) utilization and process/thread/pthread activity. It will work with both uniprocessor and multiprocessor AIX traces if the processor clocks are properly synchronized. genkld: -------
genkld Command Purpose The genkld command extracts the list of shared objects currently loaded onto the system and displays the address, size, and path name for each object on the list. Syntax genkld Description For shared objects loaded onto the system, the kernel maintains a linked list consisting of data structures called loader entries. A loader entry contains the name of the object, its starting address, and its size. This information is gathered and reported by the genkld command. Implementation Specifics This command is valid only on the POWER-based platform. Examples To obtain a list of loaded shared objects, enter: # genkld ..
d0791c00 d0194500 d019d0f8 d0237100 d01d5100 d02109e0 d01f6c60 d01b0000 d010a000 d0142000 d017f100 d016c100 d014c100 d0158100 d01410f8
18ab27 7e07 3d39 1eac0 1fff9 262b2 190dc 24cfd 367ad 3cee 1172a 128b2 b12d 13b41 846
/usr/lib/librtl.a[shr.o] /usr/lib/libbsd.a[shr.o] /usr/lib/libbind.a[shr.o] /usr/lib/libwlm.a[shr.o] /usr/lib/libC.a[shr.o] /usr/lib/libC.a[shrcore.o] /usr/lib/libC.a[ansicore_32.o] /usr/lib/boot/bin/libcfg_chrp /usr/lib/libpthreads.a[shr_xpg5.o] /usr/lib/libpthreads.a[shr_comm.o] /usr/lib/libcfg.a[shr.o] /usr/lib/libodm.a[shr.o] /usr/lib/libi18n.a[shr.o] /usr/lib/libiconv.a[shr4.o] /usr/lib/libcrypt.a[shr.o]
.. etc..
1.2.14 Not so well known tools for AIX: the proc tools: ======================================================= --proctree Displays the process tree containing the specified process IDs or users. To display the ancestors
and all the children of process 12312, enter: # proctree 21166 11238 /usr/sbin/srcmstr 21166 /usr/sbin/rsct/bin/IBM.AuditRMd To display the ancestors and children of process 21166, including children of process 0, enter: #proctree -a 21166 1 /etc/init 11238 /usr/sbin/srcmstr 21166 /usr/sbin/rsct/bin/IBM.AuditRMd
-- procstack Displays the hexadecimal addresses and symbolic names for each of the stack frames of the current thread in processes. To display the current stack of process 15052, enter: # procstack 15052 15052 : /usr/sbin/snmpd d025ab80 select (?, ?, ?, ?, ?) + 90 100015f4 main (?, ?, ?) + 1814 10000128 __start () + 8c Currently, procstack displays garbage or wrong information for the top stack frame, and possibly for the second top stack frame. Sometimes it will erroneously display "No frames found on the stack," and sometimes it will display: deadbeef ???????? (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ...) The fix for this problem had not been released at the writing of this article. When the fix becomes available, you need to download the APAR IY48543 for 5.2. For AIX 5.3 it all should work OK. -- procmap Displays a process address map. To display the address space of process 13204, enter: # procmap 13204 13204 : /usr/sbin/biod 10000000 3K 20000910 0K d0083100 79K 20013bf0 41K d007a100 34K 20011378 4K d0074000 11K d0077130 8K d00730f8 2K f03c7508 0K d01d4e20 1997K f0337e90 570K
6 read/exec read/write read/exec read/write read/exec read/write read/exec read/write read/exec read/write read/exec read/write
biod biod /usr/lib/libiconv.a /usr/lib/libiconv.a /usr/lib/libi18n.a /usr/lib/libi18n.a /usr/lib/nls/loc/en_US /usr/lib/nls/loc/en_US /usr/lib/libcrypt.a /usr/lib/libcrypt.a /usr/lib/libc.a /usr/lib/libc.a
-- procldd Displays a list of libraries loaded by a process. To display the list of dynamic libraries loaded by process 11928, enter # procldd 11928. T 11928 : -sh /usr/lib/nls/loc/en_US /usr/lib/libcrypt.a /usr/lib/libc.a -- procflags Displays a process tracing flags, and the pending and holding signals. To display the tracing flags of process 28138, enter: # procflags 28138 28138 : /usr/sbin/rsct/bin/IBM.HostRMd data model = _ILP32 flags = PR_FORK /64763: flags = PR_ASLEEP | PR_NOREGS /66315: flags = PR_ASLEEP | PR_NOREGS /60641: flags = PR_ASLEEP | PR_NOREGS /66827: flags = PR_ASLEEP | PR_NOREGS /7515: flags = PR_ASLEEP | PR_NOREGS /70439: flags = PR_ASLEEP | PR_NOREGS /66061: flags = PR_ASLEEP | PR_NOREGS /69149: flags = PR_ASLEEP | PR_NOREGS -- procsig Lists the signal actions for a process. To list all the signal actions defined for process 30552, enter: # procsig 30552 30552 : -ksh HUP caught INT caught QUIT caught ILL caught TRAP caught ABRT caught EMT caught FPE caught KILL default RESTART BUS caught -- proccred Prints a process' credentials. To display the credentials of process 25632, enter: # proccred 25632 25632: e/r/suid=0 -- procfiles
e/r/sgid=0
Prints a list of open file descriptors. To display status and control information on the file descriptors opened by process 20138, enter: # procfiles -n 20138 20138 : /usr/sbin/rsct/bin/IBM.CSMAgentRMd Current rlimit: 2147483647 file descriptors 0: S_IFCHR mode:00 dev:10,4 ino:4178 uid:0 gid:0 rdev:2,2 O_RDWR name:/dev/null 2: S_IFREG mode:0311 dev:10,6 ino:250 uid:0 gid:0 rdev:0,0 O_RDWR size:0 name:/var/ct/IBM.CSMAgentRM.stderr 4: S_IFREG mode:0200 dev:10,6 ino:255 uid:0 gid:0 rdev:0,0 -- procwdx Prints the current working directory for a process. To display the current working directory of process 11928, enter: # procwdx 11928 11928 : /home/guest -- procstop Stops a process. To stop process 7500 on the PR_REQUESTED event, enter: # procstop 7500 . -- procrun Restart a process. To restart process 30192 that was stopped on the PR_REQUESTED event, enter: # procrun 30192 . -- procwait Waits for all of the specified processes to terminate. To wait for process 12942 to exit and display the status, enter # procwait -v 12942 . 12942 : terminated, exit status 0
1.2.15 Other monitoring: ======================== Nagios: open source Monitoring for most unix systems: ----------------------------------------------------Nagios is an open source host, service and network monitoring program. Latest versions: 2.5 (stable) Overview
Nagios is a host and service monitor designed to inform you of network problems before your clients, end-users or managers do. It has been designed to run under the Linux operating system, but works fine under most *NIX variants as well. The monitoring daemon runs intermittent checks on hosts and services you specify using external "plugins" which return status information to Nagios. When problems are encountered, the daemon can send notifications out to administrative contacts in a variety of different ways (email, instant message, SMS, etc.). Current status information, historical logs, and reports can all be accessed via a web browser. System Requirements The only requirement of running Nagios is a machine running Linux (or UNIX variant) and a C compiler. You will probably also want to have TCP/IP configured, as most service checks will be performed over the network. You are not required to use the CGIs included with Nagios. However, if you do decide to use them, you will need to have the following software installed... - A web server (preferrably Apache) - Thomas Boutell's gd library version 1.6.3 or higher (required by the statusmap and trends CGIs)
rstat: Monitoring Machine Utilization with rstat: ------------------------------------------------rstat stands for Remote System Statistics service Ports exist for most unixes, like Linux, Solaris, AIX etc.. -- rstat on Linux, Solaris: rstat is an RPC client program to get and print statistics from any machine running the rpc.rstatd daemon, its server-side counterpart. The rpc.rstad daemon has been used for many years by tools such as Sun's perfmeter and the rup command. The rstat program is simply a new client for an old daemon. The fact that the rpc.rstatd daemon is already installed and running on most Solaris and Linux machines is a huge advantage over other tools that require the installation of custom agents. The rstat client compiles and runs on Solaris and Linux as well and can get statistics from any machine running a current rpc.rstatd daemon, such as Solaris, Linux, AIX, and OpenBSD. The rpc.rstatd daemon is started from /etc/inetd.conf on Solaris. It is similar to vmstat, but has some advantages over vmstat:
You can get statistics without logging in to the remote machine, including over the Internet. It includes a timestamp. The output can be plotted directly by gnuplot. The fact that it runs remotely means that you can use a single central machine to monitor the performance of many remote machines. It also has a disadvantage in that it does not give the useful scan rate measurement of memory shortage, the sr column in vmstat. rstat will not work across most firewalls because it relies on port 111, the RPC port, which is usually blocked by firewalls. To use rstat, simply give it the name or IP address of the machine you wish to monitor. Remember that rpc.rstatd must be running on that machine. The rup command is extremely useful here because with no arguments, it simply prints out a list of all machines on the local network that are running the rstatd demon. If a machine is not listed, you may have to start rstatd manually. To start rpc.rstatd under Red Hat Linux, run # /etc/rc.d/init.d/rstatd start
as root.
On Solaris, first try running the rstat client because inetd is often already configured to automatically start rpc.rstatd on request. If it the client fails with the error "RPC: Program not registered," make sure you have this line in your /etc/inet/inetd.conf and kill -HUP your inetd process to get it to re-read inetd.conf, as follows: rstatd/2-4 tli rpc/datagram_v wait root /usr/lib/netsvc/rstat/rpc.rstatd rpc.rstatd Then you can monitor that machine like this: % rstat enkidu 2001 07 10 10 36 08 12 0.1
0
0
0 100
0
27
54
1
0
0
This command will give you a one-second average and then it will exit. If you want to continuously monitor, give an interval in seconds on the command line. Here's an example of one line of output every two seconds: % rstat 2001 07 61 0.0 2001 07 15 0.0 2001 07 15 0.0
enkidu 2 10 10 36 28
0
0
1
98
0
0
7
2
0
0
10 10 36 30
0
0
0 100
0
0
0
2
0
0
10 10 36 32
0
0
0 100
0
0
0
2
0
0
2001 07 10 10 36 34 19 0.0 2001 07 10 10 36 36 108 0.0 ^C
0
0
0 100
0
5
10
2
0
0
0
0
0 100
0
0
46
2
0
0
To get a usage message, the output format, the version number, and where to go for updates, just type rstat with no parameters: % rstat usage: rstat machine [interval] output: yyyy mm dd hh mm ss usr wio sys idl pgin pgout intr ipkts opkts coll cs load docs and src at http://patrick.net/software/rstat/rstat.html Notice that the column headings line up with the output data. -- AIX: In order to get rstat working on AIX, you may need to configure rstatd. As root 1. Edit /etc/inetd.conf Uncomment or add entry for rstatd Eg rstatd sunrpc_udp udp wait root /usr/sbin/rpc.rstatd rstatd 100001 1-3 2. Edit /etc/services Uncomment or add entry for rstatd Eg rstatd 100001/udp 3. Refresh services refresh -s inetd 4. Start rstatd /usr/sbin/rpc.rstatd
1.2.16 UNIX ERROR CODES: ======================== It's always "handy" to have a list of errcodes from the errno.h headerfile. It should be reasonable the same accross the unix versions. Actually, this is only a very small list of errors and code. It is ONLY associated with the interaction of a process with the system. For example, the errors can be seen at boottime of a system, or what an
error logging daemon might write in a logfile, is a very different story. from the errno.h file: >>> Errcodes Linux (generic): #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define */ #define #define #define #define #define #define #define */ #define #define #define #define #define
EPERM ENOENT ESRCH EINTR EIO ENXIO E2BIG ENOEXEC EBADF ECHILD EAGAIN ENOMEM EACCES EFAULT ENOTBLK EBUSY EEXIST EXDEV ENODEV ENOTDIR EISDIR EINVAL ENFILE EMFILE ENOTTY ETXTBSY EFBIG ENOSPC ESPIPE EROFS EMLINK EPIPE EDOM
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
/* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /*
Operation not permitted */ No such file or directory */ No such process */ Interrupted system call */ I/O error */ No such device or address */ Arg list too long */ Exec format error */ Bad file number */ No child processes */ Try again */ Out of memory */ Permission denied */ Bad address */ Block device required */ Device or resource busy */ File exists */ Cross-device link */ No such device */ Not a directory */ Is a directory */ Invalid argument */ File table overflow */ Too many open files */ Not a typewriter */ Text file busy */ File too large */ No space left on device */ Illegal seek */ Read-only file system */ Too many links */ Broken pipe */ Math argument out of domain of func
ERANGE EDEADLK ENAMETOOLONG ENOLCK ENOSYS ENOTEMPTY ELOOP
34 35 36 37 38 39 40
/* /* /* /* /* /* /*
Math result not representable */ Resource deadlock would occur */ File name too long */ No record locks available */ Function not implemented */ Directory not empty */ Too many symbolic links encountered
EWOULDBLOCK ENOMSG EIDRM ECHRNG EL2NSYNC
EAGAIN 42 43 44 45
/* /* /* /* /*
Operation would block */ No message of desired type */ Identifier removed */ Channel number out of range */ Level 2 not synchronized */
#define EL3HLT #define EL3RST #define ELNRNG #define EUNATCH #define ENOCSI #define EL2HLT #define EBADE #define EBADR #define EXFULL #define ENOANO #define EBADRQC #define EBADSLT #define EDEADLOCK #define EBFONT #define ENOSTR #define ENODATA #define ETIME #define ENOSR #define ENONET #define ENOPKG #define EREMOTE #define ENOLINK #define EADV #define ESRMNT #define ECOMM #define EPROTO #define EMULTIHOP #define EDOTDOT #define EBADMSG #define EOVERFLOW type */ #define ENOTUNIQ #define EBADFD #define EREMCHG #define ELIBACC library */ #define ELIBBAD */ #define ELIBSCN #define ELIBMAX shared libraries */ #define ELIBEXEC directly */ #define EILSEQ #define ERESTART restarted */ #define ESTRPIPE #define EUSERS #define ENOTSOCK #define EDESTADDRREQ #define EMSGSIZE #define EPROTOTYPE #define ENOPROTOOPT #define EPROTONOSUPPORT #define ESOCKTNOSUPPORT #define EOPNOTSUPP endpoint */
46 47 48 49 50 51 52 53 54 55 56 57 EDEADLK 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75
/* /* /* /* /* /* /* /* /* /* /* /*
Level 3 halted */ Level 3 reset */ Link number out of range */ Protocol driver not attached */ No CSI structure available */ Level 2 halted */ Invalid exchange */ Invalid request descriptor */ Exchange full */ No anode */ Invalid request code */ Invalid slot */
/* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /*
Bad font file format */ Device not a stream */ No data available */ Timer expired */ Out of streams resources */ Machine is not on the network */ Package not installed */ Object is remote */ Link has been severed */ Advertise error */ Srmount error */ Communication error on send */ Protocol error */ Multihop attempted */ RFS specific error */ Not a data message */ Value too large for defined data
76 77 78 79
/* /* /* /*
Name not unique on network */ File descriptor in bad state */ Remote address changed */ Can not access a needed shared
80
/* Accessing a corrupted shared library
81 82
/* .lib section in a.out corrupted */ /* Attempting to link in too many
83
/* Cannot exec a shared library
84 85
/* Illegal byte sequence */ /* Interrupted system call should be
86 87 88 89 90 91 92 93 94 95
/* /* /* /* /* /* /* /* /* /*
Streams pipe error */ Too many users */ Socket operation on non-socket */ Destination address required */ Message too long */ Protocol wrong type for socket */ Protocol not available */ Protocol not supported */ Socket type not supported */ Operation not supported on transport
#define EPFNOSUPPORT #define EAFNOSUPPORT protocol */ #define EADDRINUSE #define EADDRNOTAVAIL #define ENETDOWN #define ENETUNREACH #define ENETRESET of reset */ #define ECONNABORTED #define ECONNRESET #define ENOBUFS #define EISCONN connected */ #define ENOTCONN */ #define ESHUTDOWN shutdown */ #define ETOOMANYREFS #define ETIMEDOUT #define ECONNREFUSED #define EHOSTDOWN #define EHOSTUNREACH #define EALREADY #define EINPROGRESS #define ESTALE #define EUCLEAN #define ENOTNAM #define ENAVAIL #define EISNAM #define EREMOTEIO #define EDQUOT #define ENOMEDIUM #define EMEDIUMTYPE
96 97
/* Protocol family not supported */ /* Address family not supported by
98 99 100 101 102
/* /* /* /* /*
Address already in use */ Cannot assign requested address */ Network is down */ Network is unreachable */ Network dropped connection because
103 104 105 106
/* /* /* /*
Software caused connection abort */ Connection reset by peer */ No buffer space available */ Transport endpoint is already
107
/* Transport endpoint is not connected
108
/* Cannot send after transport endpoint
109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124
/* /* /* /* /* /* /* /* /* /* /* /* /* /* /* /*
Too many references: cannot splice */ Connection timed out */ Connection refused */ Host is down */ No route to host */ Operation already in progress */ Operation now in progress */ Stale NFS file handle */ Structure needs cleaning */ Not a XENIX named type file */ No XENIX semaphores available */ Is a named type file */ Remote I/O error */ Quota exceeded */ No medium found */ Wrong medium type */
The list above should actually be enough, but we shall list the same for AIX: >>> errcodes AIX: #define #define #define #define #define #define #define #define #define #define #define #define #define #define
EPERM ENOENT ESRCH EINTR EIO ENXIO E2BIG ENOEXEC EBADF ECHILD EAGAIN ENOMEM EACCES EFAULT
1 2 3 4 5 6 7 8 9 10 11 12 13 14
/* /* /* /* /* /* /* /* /* /* /* /* /* /*
Operation not permitted No such file or directory No such process interrupted system call I/O error No such device or address Arg list too long Exec format error Bad file descriptor No child processes Resource temporarily unavailable Not enough space Permission denied Bad address
*/ */ */ */ */ */ */ */ */ */ */ */ */ */
#define ENOTBLK 15 #define EBUSY 16 #define EEXIST 17 #define EXDEV 18 #define ENODEV 19 #define ENOTDIR 20 #define EISDIR 21 #define EINVAL 22 #define ENFILE 23 #define EMFILE 24 #define ENOTTY 25 #define ETXTBSY 26 #define EFBIG 27 #define ENOSPC 28 #define ESPIPE 29 #define EROFS 30 #define EMLINK 31 #define EPIPE 32 #define EDOM 33 #define ERANGE 34 #define ENOMSG 35 #define EIDRM 36 #define ECHRNG 37 #define EL2NSYNC 38 #define EL3HLT 39 #define EL3RST 40 #define ELNRNG 41 #define EUNATCH 42 #define ENOCSI 43 #define EL2HLT 44 #define EDEADLK 45 #define ENOTREADY #define EWRPROTECT #define EFORMAT #define ENOLCK #define ENOCONNECT #define ESTALE #define EDIST #define EINPROGRESS #define EALREADY #define ENOTSOCK #define EDESTADDRREQ #define EDESTADDREQ #define EMSGSIZE #define EPROTOTYPE #define ENOPROTOOPT #define EPROTONOSUPPORT #define ESOCKTNOSUPPORT #define EOPNOTSUPP #define EPFNOSUPPORT #define EAFNOSUPPORT protocol family */ #define EADDRINUSE #define EADDRNOTAVAIL #define ENETDOWN #define ENETUNREACH
/* Block device required */ /* Resource busy */ /* File exists */ /* Improper link */ /* No such device */ /* Not a directory */ /* Is a directory */ /* Invalid argument */ /* Too many open files in system */ /* Too many open files */ /* Inappropriate I/O control operation */ /* Text file busy */ /* File too large */ /* No space left on device */ /* Invalid seek */ /* Read only file system */ /* Too many links */ /* Broken pipe */ /* Domain error within math function */ /* Result too large */ /* No message of desired type */ /* Identifier removed */ /* Channel number out of range */ /* Level 2 not synchronized */ /* Level 3 halted */ /* Level 3 reset */ /* Link number out of range */ /* Protocol driver not attached */ /* No CSI structure available */ /* Level 2 halted */ /* Resource deadlock avoided */ 46 /* Device not ready */ 47 /* Write-protected media */ 48 /* Unformatted media */ 49 /* No locks available */ 50 /* no connection */ 52 /* no filesystem */ 53 /* old, currently unused AIX errno*/ 55 /* Operation now in progress */ 56 /* Operation already in progress */ 57 /* Socket operation on non-socket */ 58 /* Destination address required */ EDESTADDRREQ /* Destination address required */ 59 /* Message too long */ 60 /* Protocol wrong type for socket */ 61 /* Protocol not available */ 62 /* Protocol not supported */ 63 /* Socket type not supported */ 64 /* Operation not supported on socket */ 65 /* Protocol family not supported */ 66 /* Address family not supported by 67 68 69 70
/* /* /* /*
Address already in use */ Can't assign requested address */ Network is down */ Network is unreachable */
#define ENETRESET */ #define ECONNABORTED #define ECONNRESET #define ENOBUFS #define EISCONN #define ENOTCONN #define ESHUTDOWN #define ETIMEDOUT #define ECONNREFUSED #define EHOSTDOWN #define EHOSTUNREACH #define ERESTART #define EPROCLIM #define EUSERS #define ELOOP */ #define ENAMETOOLONG */ #define EDQUOT #define ECORRUPT #define EREMOTE #define ENOSYS #define EMEDIA #define ESOFT */ #define ENOATTR #define ESAD #define ENOTRUST #define ETOOMANYREFS #define EILSEQ #define ECANCELED #define ENOSR #define ETIME #define EBADMSG #define EPROTO #define ENODATA #define ENOSTR #define ECLONEME stream ... */ #define ENOTSUP #define EMULTIHOP #define ENOLINK #define EOVERFLOW type */
71
/* Network dropped connection on reset
72 73 74 75 76 77 78 79 80 81 82 83 84 85
/* /* /* /* /* /* /* /* /* /* /* /* /* /*
86
/* File name too long
88 89 93 109 110 111
/* /* /* /* /* /*
Software caused connection abort */ Connection reset by peer */ No buffer space available */ Socket is already connected */ Socket is not connected */ Can't send after socket shutdown */ Connection timed out */ Connection refused */ Host is down */ No route to host */ restart the system call */ Too many processes */ Too many users */ Too many levels of symbolic links
Disc quota exceeded */ Invalid file system control data */ Item is not local to host */ Function not implemented POSIX */ media surface error */ I/O completed, but needs relocation
112 /* no attribute found */ 113 /* security authentication denied */ 114 /* not a trusted program */ 115 /* Too many references: can't splice */ 116 /* Invalid wide character */ 117 /* asynchronous i/o cancelled */ 118 /* temp out of streams resources */ 119 /* I_STR ioctl timed out */ 120 /* wrong message type at stream head */ 121 /* STREAMS protocol error */ 122 /* no message ready at stream head */ 123 /* fd is not a stream */ ERESTART /* this is the way we clone a 124 125 126 127
/* /* /* /*
================================== 2. NFS and Mount command examples: ==================================
POSIX threads unsupported value */ multihop is not allowed */ the link has been severed */ value too large to be stored in data
Let's start with something that might be of interrest right now: Examples of mounting a DVD or CDROM: =================================== AIX: ---# mount -r -v cdrfs /dev/cd0 /cdrom Solaris: -------# mount -r -F hsfs /dev/dsk/c0t6d0s2 /cdrom HPUX: ----mount -F cdfs -o rr /dev/dsk/c1t2d0 /cdrom SuSE Linux: ----------# mount -t iso9660 /dev/cdrom /cdrom # mount -t iso9660 /dev/cdrom /media/cdrom Redhat Linux: ------------# mount -t iso9660 /dev/cdrom /media/cdrom Other commands on Linux: -----------------------Sometimes on some Linux, and some scsi CDROM devices, you might try # mount /dev/sr0 /mount_point # mount -t iso9660 /dev/sr0 /mount_point Now we return to a discussion of "mounting" and NFS. 2.1 NFS: ======== We will discuss the most important feaures of NFS, by showing how its implemented on Solaris, Redhat and SuSE Linux. Most of this applies to HP-UX and AIX as well. 2.1.1 NFS and Redhat Linux: ---------------------------
Linux uses a combination of kernel-level support and continuously running daemon processes to provide NFS file sharing, however, NFS support must be enabled in the Linux kernel to function. NFS uses Remote Procedure Calls (RPC) to route requests between clients and servers, meaning that the portmap service must be enabled and active at the proper runlevels for NFS communication to occur. Working with portmap, various other processes ensure that a particular NFS connection is allowed and may proceed without error: rpc.mountd - The running process that receives the mount request from an NFS client and checks to see if it matches with a currently exported file system. rpc.nfsd - The process that implements the user-level part of the NFS service. It works with the Linux kernel to meet the dynamic demands of NFS clients, such as providing additional server threads for NFS clients to uses. rpc.lockd - A daemon that is not necessary with modern kernels. NFS file locking is now done by the kernel. It is included with the nfs-utils package for users of older kernels that do not include this functionality by default. rpc.statd - Implements the Network Status Monitor (NSM) RPC protocol. This provides reboot notification when an NFS server is restarted without being gracefully brought down. rpc.rquotad - An RPC server that provides user quota information for remote users. Not all of these programs are required for NFS service. The only services that must be enabled are rpc.mountd, rpc.nfsd, and portmap. The other daemons provide additional functionality and should only be used if your server environment requires them. NFS version 2 uses the User Datagram Protocol (UDP) to provide a stateless network connection between the client and server. NFS version 3 can use UDP or TCP running over an IP. The stateless UDP connection minimizes network traffic, as the NFS server sends the client a cookie after the client is authorized to access the shared volume. This cookie is a random value stored on the server's side and is passed with along with RPC requests from the client. The NFS server can be restarted without affecting the clients and the cookie will remain intact. NFS only performs authentication when a client system attempts to mount a remote file system. To limit access, the NFS server first employs TCP wrappers. TCP wrappers reads the /etc/hosts.allow and /etc/hosts.deny files to determine if a particular client should be permitted or prevented access to the NFS server.
After the client is allowed past TCP wrappers, the NFS server refers to its configuration file, "/etc/exports", to determine whether the client has enough privileges to mount any of the exported file systems. After granting access, any file and directory operations are sent to the server using remote procedure calls. Warning NFS mount privileges are granted specifically to a client, not a user. If you grant a client machine access to an exported file system, any users of that machine will have access to the data. When configuring the /etc/exports file, be extremely careful about granting read-write permissions (rw) to a remote host. -- NFS and portmap NFS relies upon remote procedure calls (RPC) to function. portmap is required to map RPC requests to the correct services. RPC processes notify portmap when they start, revealing the port number they are monitoring and the RPC program numbers they expect to serve. The client system then contacts portmap on the server with a particular RPC program number. portmap then redirects the client to the proper port number to communicate with its intended service. Because RPC-based services rely on portmap to make all connections with incoming client requests, portmap must be available before any of these services start. If, for some reason, the portmap service unexpectedly quits, restart portmap and any services running when it was started. The portmap service can be used with the host access files (/etc/hosts.allow and /etc/hosts.deny) to control which remote systems are permitted to use RPC-based services on your machine. Access control rules for portmap will affect all RPC-based services. Alternatively, you can specify each of the NFS RPC daemons to be affected by a particular access control rule. The man pages for rpc.mountd and rpc.statd contain information regarding the precise syntax of these rules. -- portmap Status As portmap provides the coordination between RPC services and the port numbers used to communicate with them, it is useful to be able to get a picture of the current RPC services using portmap when troubleshooting. The rpcinfo command shows each RPC-based service with its port number, RPC program number, version, and IP protocol type (TCP or UDP). To make sure the proper NFS RPC-based services are enabled for portmap, rpcinfo -p can be useful: # rpcinfo -p
program vers proto 100000 2 tcp 100000 2 udp 100024 1 udp 100024 1 tcp 100011 1 udp 100011 2 udp 100005 1 udp 100005 1 tcp 100005 2 udp 100005 2 tcp 100005 3 udp 100005 3 tcp 100003 2 udp 100003 3 udp 100021 1 udp 100021 3 udp 100021 4 udp
port 111 111 1024 1024 819 819 1027 1106 1027 1106 1027 1106 2049 2049 1028 1028 1028
portmapper portmapper status status rquotad rquotad mountd mountd mountd mountd mountd mountd nfs nfs nlockmgr nlockmgr nlockmgr
The -p option probes the portmapper on the specified host or defaults to localhost if no specific host is listed. Other options are available from the rpcinfo man page. From the output above, various NFS services can be seen running. If one of the NFS services does not start up correctly, portmap will be unable to map RPC requests from clients for that service to the correct port. In many cases, restarting NFS as root (/sbin/service nfs restart) will cause those service to correctly register with portmap and begin working. # /sbin/service nfs restart -- NFS Server Configuration Files Configuring a system to share files and directories using NFS is straightforward. Every file system being exported to remote users via NFS, as well as the access rights relating to those file systems, is located in the /etc/exports file. This file is read by the exportfs command to give rpc.mountd and rpc.nfsd the information necessary to allow the remote mounting of a file system by an authorized host. The exportfs command allows you to selectively export or unexport directories without restarting the various NFS services. When exportfs is passed the proper options, the file systems to be exported are written to /var/lib/nfs/xtab. Since rpc.mountd refers to the xtab file when deciding access privileges to a file system, changes to the list of exported file systems take effect immediately. Various options are available when using exportfs: -r - Causes all directories listed in /etc/exports to be exported by constructing a new export list in
/etc/lib/nfs/xtab. This option effectively refreshes the export list with any changes that have been made to /etc/exports. -a - Causes all directories to be exported or unexported, depending on the other options passed to exportfs. -o options - Allows the user to specify directories to be exported that are not listed in /etc/exports. These additional file system shares must be written in the same way they are specified in /etc/exports. This option is used to test an exported file system before adding it permanently to the list of file systems to be exported. -i - Tells exportfs to ignore /etc/exports; only options given from the command line are used to define exported file systems. -u - Unexports directories from being mounted by remote users. The command exportfs -ua effectively suspends NFS file sharing while keeping the various NFS daemons up. To allow NFS sharing to continue, type exportfs -r. -v - Verbose operation, where the file systems being exported or unexported are displayed in greater detail when the exportfs command is executed. If no options are passed to the exportfs command, it displays a list of currently exported file systems. Changes to /etc/exports can also be read by reloading the NFS service with the service nfs reload command. This keeps the NFS daemons running while re-exporting the /etc/exports file. -- /etc/exports The /etc/exports file is the standard for controlling which file systems are exported to which hosts, as well as specifying particular options that control everything. Blank lines are ignored, comments can be made using #, and long lines can be wrapped with a backslash (\). Each exported file system should be on its own line. Lists of authorized hosts placed after an exported file system must be separated by space characters. Options for each of the hosts must be placed in parentheses directly after the host identifier, without any spaces separating the host and the first parenthesis. In its simplest form, /etc/exports only needs to know the directory to be exported and the hosts permitted to use it: /some/directory bob.domain.com /another/exported/directory 192.168.0.3 n5111sviob
After re-exporting /etc/exports with the "/sbin/service nfs reload" command, the bob.domain.com host will be able to mount /some/directory and 192.168.0.3 can mount /another/exported/directory. Because no options are specified in this example, several default NFS preferences take effect. In order to override these defaults, you must specify an option that takes its place. For example, if you do not specify rw, then that export will only be shared read-only. Each default for every exported file system must be explicitly overridden. Additionally, other options are available where no default value is in place. These include the ability to disable sub-tree checking, allow access from insecure ports, and allow insecure file locks (necessary for certain early NFS client implementations). See the exports man page for details on these lesser used options. When specifying hostnames, you can use the following methods: single host - Where one particular host is specified with a fully qualified domain name, hostname, or IP address. wildcards - Where a * or ? character is used to take into account a grouping of fully qualified domain names that match a particular string of letters. Wildcards are not to be used with IP addresses; however, they may accidently work if reverse DNS lookups fail. However, be careful when using wildcards with fully qualified domain names, as they tend to be more exact than you would expect. For example, the use of *.domain.com as wildcard will allow sales.domain.com to access the exported file system, but not bob.sales.domain.com. To match both possibilities, as well as sam.corp.domain.com, you would have to provide *.domain.com *.*.domain.com. IP networks - Allows the matching of hosts based on their IP addresses within a larger network. For example, 192.168.0.0/28 will allow the first 16 IP addresses, from 192.168.0.0 to 192.168.0.15, to access the exported file system but not 192.168.0.16 and higher. netgroups - Permits an NIS netgroup name, written as @, to be used. This effectively puts the NIS server in charge of access control for this exported file system, where users can be added and removed from an NIS group without affecting /etc/exports. Warning
The way in which the /etc/exports file is formatted is very important, particularly concerning the use of space characters. Remember to always separate exported file systems from hosts and hosts from one another with a space character. However, there should be no other space characters in the file unless they are used in comment lines. For example, the following two lines do not mean the same thing: /home bob.domain.com(rw) /home bob.domain.com (rw) The first line allows only users from bob.domain.com read-write access to the /home directory. The second line allows users from bob.domain.com to mount the directory read-only (the default), but the rest of the world can mount it read-write. Be careful where space characters are used in /etc/exports. -- NFS Client Configuration Files - What to do on a client? Any NFS share made available by a server can be mounted using various methods. Of course, the share can be manually mounted, using the mount command, to acquire the exported file system at a particular mount point. However, this requires that the root user type the mount command every time the system restarts. In addition, the root user must remember to unmount the file system when shutting down the machine. Two methods of configuring NFS mounts include modifying the /etc/fstab or using the autofs service. > /etc/fstab Placing a properly formatted line in the /etc/fstab file has the same effect as manually mounting the exported file system. The /etc/fstab file is read by the /etc/rc.d/init.d/netfs script at system startup. The proper file system mounts, including NFS, are put into place. A sample /etc/fstab line to mount an NFS export looks like the following: : nfs 0 0 The relates to the hostname, IP address, or fully qualified domain name of the server exporting the file system. The tells the server what export to mount. The specifies where on the local file system to mount the exported directory. This mount point must exist before /etc/fstab is read or the mount will fail. The nfs option specifies the type of file system being mounted.
The area specifies how the file system is to be mounted. For example, if the options area states rw,suid on a particular mount, the exported file system will be mounted read-write and the user and group ID set by the server will be used. Note, parentheses are not to be used here.
2.1.2 NFS and SuSE Linux: -------------------------- Importing File Systems with YaST Any user authorized to do so can mount NFS directories from an NFS server into his own file tree. This can be achieved most easily using the YaST module `NFS Client'. Just enter the host name of the NFS server, the directory to import, and the mount point at which to mount this directory locally. All this is done after clicking `Add' in the first dialog. -- Importing File Systems Manually File systems can easily be imported manually from an NFS server. The only prerequisite is a running RPC port mapper, which can be started by entering the command # rcportmap start as root. Once this prerequisite is met, remote file systems exported on the respective machines can be mounted in the file system just like local hard disks using the command mount with the following syntax: # mount host:remote-path local-path If user directories from the machine sun, for example, should be imported, the following command can be used: # mount sun:/home /home -- Exporting File Systems with YaST With YaST, turn a host in your network into an NFS server - a server that exports directories and files to all hosts granted access to it. This could be done to provide applications to all coworkers of a group without installing them locally on each and every host. To install such a server, start YaST and select `Network Services' -> `NFS Server' Next, activate `Start NFS Server' and click `Next'. In the upper text field, enter the directories to export. Below, enter the hosts that should have access to them.
There are four options that can be set for each host: single host, netgroups, wildcards, and IP networks. A more thorough explanation of these options is provided by man exports. `Exit' completes the configuration. -- Exporting File Systems Manually If you do not want to use YaST, make sure the following systems run on the NFS server: RPC portmapper (portmap) RPC mount daemon (rpc.mountd) RPC NFS daemon (rpc.nfsd) For these services to be started by the scripts "/etc/init.d/portmap" and "/etc/init.d/nfsserver" when the system is booted, enter the commands # insserv /etc/init.d/nfsserver # insserv /etc/init.d/portmap.
and
Also define which file systems should be exported to which host in the configuration file "/etc/exports". For each directory to export, one line is needed to set which machines may access that directory with what permissions. All subdirectories of this directory are automatically exported as well. Authorized machines are usually specified with their full names (including domain name), but it is possible to use wild cards like * or ? (which expand the same way as in the Bash shell). If no machine is specified here, any machine is allowed to import this file system with the given permissions. Set permissions for the file system to export in brackets after the machine name. The most important options are: ro File system is exported with read-only permission (default). rw File system is exported with read-write permission. root_squash This makes sure the user root of the given machine does not have root permissions on this file system. This is achieved by assigning user ID 65534 to users with user ID 0 (root). This user ID should be set to nobody (which is the default). no_root_squash Does not assign user ID 0 to user ID 65534, keeping the root permissions valid. link_relative Converts absolute links (those beginning with /) to a sequence of ../. This is only useful if the entire file system of a machine is mounted (default). link_absolute Symbolic links remain untouched. map_identity User IDs are exactly the same on both client and server (default).
map_daemon Client and server do not have matching user IDs. This tells nfsd to create a conversion table for user IDs. The ugidd daemon is required for this to work. /etc/exports is read by mountd and nfsd. If you change anything in this file, restart mountd and nfsd for your changes to take effect. This can easily be done with "rcnfsserver restart". Example SuSE /etc/exports # # /etc/exports # /home /usr/X11 /usr/lib/texmf / /home/ftp # End of exports
sun(rw) venus(rw) sun(ro) venus(ro) sun(ro) venus(rw) earth(ro,root_squash) (ro)
2.2 Mount command: ================== The standard form of the mount command, is mount -F typefs device mountdir (solaris, HP-UX) mount -t typefs device mountdir (many other unix's) This tells the kernel to attach the file system found on "device" (which is of type type) at the directory "dir". The previous contents (if any) and owner and mode of dir become invisible, and as long as this file system remains mounted, the pathname dir refers to the root of the file system on device. The syntax is: mount [options] [type] [device] [mountpoint] -- mounting a remote filesystem: syntax: mount -F nfs <-o specific options> -O : # mount -F nfs hpsrv:/data /data # mount -F nfs -o hard,intr thor:/data
/data
- standard mounts are determined by files like or /etc/filesystems (AIX) or /etc/vfstab etc..
/etc/fstab (HP-UX)
2.2.1 Where are the standard mounts defined? ============================================ In Solaris: =========== - standard mounts are determined by /etc/vfstab etc.. - NFS mounts are determined by the file /etc/dfs/dfstab. Here you will find share commands. - currently mounted filesystems are listed in /etc/mnttab In Linux: ========= - standard mounts are determined by most Linux distros by "/etc/fstab". In AIX: ======= - standard mounts and properties are determined by the file "/etc/filesystems". In HP-UX: ========= There is a /etc/fstab which contains all of the filesystems are mounted at boot time. The filesystems that are OS related are / , /var, /opt , /tmp, /usr , /stand The filesystem that is special is /stand, this is where your kernel is built and resides. Notice that the filesystem type is "hfs". HPUX kernels MUST reside on an hfs filesystem
An example of /etc/vfstab: -------------------------starboss:/etc $ #device mount #to mount options # fd /proc /dev/md/dsk/d1 /dev/md/dsk/d0 /dev/md/dsk/d4 /dev/md/dsk/d3 /dev/md/dsk/d7 /dev/md/dsk/d5 logging
more vfstab device
mount
FS
fsck
mount
to fsck
point
type
pass
at boot
/dev/fd fd /proc proc /dev/md/rdsk/d0 /dev/md/rdsk/d4 /dev/md/rdsk/d3 /dev/md/rdsk/d7 /dev/md/rdsk/d5
no no swap / ufs /usr ufs /var ufs /export ufs /usr/local
no 1 1 1 2 ufs
no no no yes 2
logging logging logging logging yes
/dev/dsk/c2t0d0s0 /dev/rdsk/c2t0d0s0 yes logging swap - /tmp tmpfs - yes size=512m
/export2
ufs
2
mount adds an entry, umount deletes an entry. mounting applies to local filesystemes, or remote filesystems via NFS Local mount example: mount -F ufs -o logging /dev/dsk/c0t0d0s3 /mnt At Remote server: share, shareall, or add entry in /etc/dfs/dfstab # share -F nfs /var/mail Unmount a mounted FS First check who is using it # fuser -c mountpoint # umount mointpoint
2.2.2 Mounting a NFS filesystem in HP-UX: ========================================= Mounting Remote File Systems You can use either SAM or the mount command to mount file systems located on a remote system. Before you can mount file systems located on a remote system, NFS software must be installed and configured on both local and remote systems. Refer to Installing and Administering NFS for information. For information on mounting NFS file systems using SAM, see SAM's online help. To mount a remote file system using HP-UX commands, You must know the name of the host machine and the file system's directory on the remote machine. Establish communication over a network between the local system (that is, the "client") and the remote system. (The local system must be able to reach the remote system via whatever hosts database is in use.) (See named(1M) and hosts(4).) If necessary, test the connection with /usr/sbin/ping; see ping(1M). Make sure the file /etc/exports on the remote system lists the file systems that you wish to make available to clients (that is, to "export") and the local systems that you wish to mount the file systems.
For example, to allow machines called rolf and egbert to remotely mount the /usr file system, edit the file /etc/exports on the remote machine and include the line: /usr rolf egbert Execute /usr/sbin/exportfs -a on the remote system to export all directories in /etc/exports to clients. For more information, see exportfs(1M). NOTE: If you wish to invoke exportfs -a at boot time, make sure the NFS configuration file /etc/rc.config.d/nfsconf on the remote system contains the following settings: NFS_SERVER=1 and START_MOUNTD=1. The client's /etc/rc.config.d/nfsconf file must contain NFS_CLIENT=1. Then issue the following command to run the script: /sbin/init.d/nfs.server start Mount the file system on the local system, as in: # mount -F nfs remotehost:/remote_dir /local_dir
Just a bunch of mount command examples: --------------------------------------# # # # # # # # # # #
mount mount -a mountall -l mount -t type device dir mount -F pcfs /dev/dsk/c0t0d0p0:c /pcfs/c mount /dev/md/dsk/d7 /u01 mount sun:/home /home mount -t nfs 137.82.51.1:/share/sunos/local /usr/local mount /dev/fd0 /mnt/floppy mount -o ro /dev/dsk/c0t6d0s1 /mnt/cdrom mount -V cdrfs -o ro /dev/cd0 /cdrom
2.2.3 Solaris mount command: ============================ The unix mount command is used to mount a filesystem, and it attaches disks, and directories logically rather than physically. It takes a minimum of two arguments: 1) the name of the special device which contains the filesystem 2) the name of an existing directory on which to mount the file system Once the file system is mounted, the directory becomes the mount point. All the file systems will now be usable
as if they were subdirectories of the file system they were mounted on. The table of currently mounted file systems can be found by examining the mounted file system information file. This is provided by a file system that is usually mounted on /etc/mnttab. Mounting a file system causes three actions to occur: 1. The superblock for the mounted file system is read into memory 2. An entry is made in the /etc/mnttab file 3. An entry is made in the inode for the directory on which the file system is mounted which marks the directory as a mount point The /etc/mountall command mounts all filesystems as described in the /etc/vfstab file. Note that /etc/mount and /etc/mountall commands can only be executed by the superuser. OPTIONS -F FSType Used to specify the FSType on which to operate. The FSType must be specified or must be determinable from /etc/vfstab, or by consulting /etc/default/fs or /etc/dfs/fstypes. -a [ mount_points. . . ] Perform mount or umount operations in parallel, when possible. If mount points are not specified, mount will mount all file systems whose /etc/vfstab "mount at boot" field is "yes". If mount points are specified, then /etc/vfstab "mount at boot" field will be ignored. If mount points are specified, umount will only umount those mount points. If none is specified, then umount will attempt to unmount all file systems in /etc/mnttab, with the exception of certain system required file systems: /, /usr, /var, /var/adm, /var/run, /proc, /dev/fd and /tmp. -f Forcibly unmount a file system. Without this option, umount does not allow a file system to be unmounted if a file on the file system is busy. Using this option can cause data loss for open files; programs which access files after the file system has been unmounted will get an error (EIO). -p Print the list of mounted file systems in the /etc/vfstab format. Must be the only option specified. -v Print the list of mounted file systems in verbose format. Must be the only option specified. -V Echo the complete command line, but do not execute the command. umount generates a command line by using the
options and arguments provided by the user and adding to them information derived from /etc/mnttab. This option should be used to verify and validate the command line. generic_options Options that are commonly supported by most FSType-specific command modules. The following options are available: -m Mount the file system without making an entry in /etc/mnttab. -g Globally mount the file system. On a clustered system, this globally mounts the file system on all nodes of the cluster. On a non-clustered system this has no effect. -o Specify FSType-specific options in a comma separated (without spaces) list of suboptions and keyword-attribute pairs for interpretation by the FSTypespecific module of the command. (See mount_ufs(1M)) -O Overlay mount. Allow the file system to be mounted over an existing mount point, making the underlying file system inaccessible. If a mount is attempted on a pre-existing mount point without setting this flag, the mount will fail, producing the error "device busy". -r Mount the file system read-only. Example mount: mount -F ufs -o logging /dev/dsk/c0t0d0s3 /mnt Example mountpoints and disks: -----------------------------Mountpunt / /usr /var /home /opt /u01 /u02 /u03 /u04 /u05 /u06 /u07
Device Omvang /dev/md/dsk/d1 100 /dev/md/dsk/d3 1200 /dev/md/dsk/d4 200 /dev/md/dsk/d5 200 /dev/md/dsk/d6 4700 /dev/md/dsk/d7 8700 /dev/md/dsk/d8 8700 /dev/md/dsk/d9 8700 /dev/md/dsk/d10 8700 /dev/md/dsk/d110 /dev/md/dsk/d120 /dev/md/dsk/d123
Doel Unix Root-filesysteem Unix usr-filesysteem Unix var-filesysteem Unix opt-filesysteem Oracle_Home Oracle datafiles Oracle datafiles Oracle datafiles Oracle datafiles 8700 Oracle datafiles 8700 Oracle datafiles 8650 Oracle datafiles
Suppose you have only 1 disk of about 72GB, 2GB RAM: Entire disk= Slice 2
/ swap /export /var /opt /usr /u01 20G
Slice 0, partition about 2G Slice 1, partition about 4G Slice 3, partition about 50G, maybe you link it to /u01 Slice 4, partition about 2G Slice 5, partition about 10G if you plan to install apps here Slice 6, partition about 2G Slice 7, partition optional, standard it's /home Depending on how you configure /export, size could be around
find . -name dfctowdk\*.zip | while read file; do pkzip25 -extract -translate=unix -> 2.2.4 mount command on AIX: =========================== Typical examples: # mount -o soft 10.32.66.75:/data/nim /mnt # mount -o soft abcsrv:/data/nim /mnt # mount -o soft n580l03:/data/nim /mnt Note 1: ------mount [ -f ] [ -n Node ] [ -o Options ] [ -p ] [ -r ] [ -v VfsName ] [ -t Type | [ Device | Node:Directory ] Directory | all | -a ] [-V [generic_options] special_mount_points If you specify only the Directory parameter, the mount command takes it to be the name of the directory or file on which a file system, directory, or file is usually mounted (as defined in the /etc/filesystems file). The mount command looks up the associated device, directory, or file and mounts it. This is the most convenient way of using the mount command, because it does not require you to remember what is normally mounted on a directory or file. You can also specify only the device. In this case, the command obtains the mount point from the /etc/filesystems file. The /etc/filesystems file should include a stanza for each mountable file system, directory, or file. This stanza should specify at least the name of the file system and either the device on which it resides or the directory name. If the stanza includes a mount attribute, the mount command uses the associated values. It recognizes five values for the mount attributes: automatic, true, false, removable, and readonly. The mount all command causes all file systems with the mount=true attribute to be mounted in their normal places. This command is typically used during system initialization, and the corresponding mounts are referred to as
automatic mounts. Example mount command on AIX: ----------------------------$ mount node mounted options -------- ----------------------------/dev/hd4 rw,log=/dev/hd8 /dev/hd2 rw,log=/dev/hd8 /dev/hd9var rw,log=/dev/hd8 /dev/hd3 rw,log=/dev/hd8 /dev/hd1 rw,log=/dev/hd8 /proc /dev/hd10opt rw,log=/dev/hd8 /dev/fslv00 rw,log=/dev/hd8 /dev/fslv01 rw,log=/dev/hd8 /dev/fslv02 rw,log=/dev/hd8 /dev/oralv rw,log=/dev/hd8 /dev/db2lv rw,log=/dev/loglv00 /dev/fslv03 rw,log=/dev/hd8 /dev/homepeter rw,log=/dev/hd8 /dev/bmclv rw,log=/dev/hd8 /dev/u01 rw,log=/dev/loglv01 /dev/u02 rw,log=/dev/loglv01 /dev/u05 rw,log=/dev/loglv01 /dev/u03 rw,log=/dev/loglv01 /dev/backuo rw,log=/dev/loglv02 /dev/u02back rw,log=/dev/loglv03 /dev/u01back rw,log=/dev/loglv03 /dev/u05back rw,log=/dev/loglv03
mounted over
vfs
date
---------------
------ ------------
/
jfs2
Jun 06 17:15
/usr
jfs2
Jun 06 17:15
/var
jfs2
Jun 06 17:15
/tmp
jfs2
Jun 06 17:15
/home
jfs2
Jun 06 17:16
/proc /opt
procfs Jun 06 17:16 rw jfs2 Jun 06 17:16
/XmRec
jfs2
Jun 06 17:16
/tmp/m2
jfs2
Jun 06 17:16
/software
jfs2
Jun 06 17:16
/opt/app/oracle
jfs2
Jun 06 17:25
/db2_database
jfs2
Jun 06 19:54
/bmc_home
jfs2
Jun 07 12:11
/home/peter
jfs2
Jun 13 18:42
/bcict/stage
jfs2
Jun 15 15:21
/u01
jfs2
Jun 22 00:22
/u02
jfs2
Jun 22 00:22
/u05
jfs2
Jun 22 00:22
/u03
jfs2
Jun 22 00:22
/backup_ora
jfs2
Jun 22 00:22
/u02back
jfs2
Jun 22 00:22
/u01back
jfs2
Jun 22 00:22
/u05back
jfs2
Jun 22 00:22
/dev/u04back rw,log=/dev/loglv03 /dev/u03back rw,log=/dev/loglv03 /dev/u04 rw,log=/dev/loglv01
/u04back
jfs2
Jun 22 00:22
/u03back
jfs2
Jun 22 00:22
/u04
jfs2
Jun 22 10:25
Example /etc/filesystems file: /var: dev vfs log mount check type vol free
= = = = = = = =
/dev/hd9var jfs2 /dev/hd8 automatic false bootfs /var false
dev vfs log mount check vol free
= = = = = = =
/dev/hd3 jfs2 /dev/hd8 automatic false /tmp false
dev vfs log mount check vol free
= = = = = = =
/dev/hd10opt jfs2 /dev/hd8 true true /opt false
/tmp:
/opt:
Example of the relation of Logigal Volumes and mountpoints: /dev/lv01 /dev/lv02 /dev/lv03 /dev/lv04 /dev/lv00
= = = = =
/u01 /u02 /u03 /data /spl
2.2.5 Some other commands related to mounts: =========================================== fsstat command: ---------------
On some unixes, the fsstat command is available. It provides filesystem statitstics. It can take a lot of switches, thus be sure to check the man pages. On Solaris, the following example shows the statistics for each file operation for "/" (using the -f option): $ fsstat -f / Mountpoint: / operation #ops open 8.54K close 9.8K read 43.6K write 1.57K ioctl 2.06K setfl 4 getattr 40.3K setattr 38 access 9.19K lookup 203K create 595 remove 56 link 0 rename 9 mkdir 19 rmdir 0 readdir 2.02K symlink 4 readlink 8.31K fsync 199 inactive 2.96K fid 0 rwlock 47.2K rwunlock 47.2K seek 29.1K cmp 42.9K frlock 4.45K space 8 realvp 3.25K getpage 104K putpage 2.69K map 13.2K addmap 34.4K delmap 33.4K poll 287 dump 0 pathconf 54 pageio 0 dumpctl 0 dispose 23.8K getsecattr 697 setsecattr 0 shrlock 0 vnevent 0
bytes 65.9M 2.99M
2.27M
fuser command: -------------AIX: Purpose Identifies processes using a file or file structure. Syntax fuser [ -c | -d | -f ] [ -k ] [ -u ] [ -x ] [ -V ]File ... Description The fuser command lists the process numbers of local processes that use the local or remote files specified by the File parameter. For block special devices, the command lists the processes that use any file on that device. Flags -c Reports on any open files in the file system containing File. -d Implies the use of the -c and -x flags. Reports on any open files which haved been unlinked from the file system (deleted from the parent directory). When used in conjunction with the -V flag, it also reports the inode number and size of the deleted file. -f Reports on open instances of File only. -k Sends the SIGKILL signal to each local process. Only the root user can kill a process of another user. -u Provides the login name for local processes in parentheses after the process number. -V Provides verbose output. -x Used in conjunction with -c or -f, reports on executable and loadable objects in addition to the standard fuser output. To list the process numbers of local processes using the /etc/passwd file, enter: # fuser /etc/passwd To list the process numbers and user login names of processes using the /etc/filesystems file, enter: # fuser -u /etc/filesystems To terminate all of the processes using a given file system, enter: #fuser -k -x -u /dev/hd1 -OR#fuser -kxuc /home Either command lists the process number and user name, and then terminates each process that is using the /dev/hd1 (/home) file system. Only the root user can terminate processes that belong to another user. You might want to use this command if you are trying to unmount the /dev/hd1 file system and a process that is accessing the /dev/hd1 file system prevents this.
To list all processes that are using a file which has been deleted from a given file system, enter: # fuser -d /usr
Examples on linux distro's: - To kill all processes accessing the file system /home in any way. # fuser -km /home - invokes something if no other process is using /dev/ttyS1. if fuser -s /dev/ttyS1; then :; else something; fi - shows all processes at the (local) TELNET port. # fuser telnet/tcp A similar command is the lsof command. 2.2.6 Starting and stopping NFS: ================================ Short note on stopping and starting NFS. See other sections for more detail. On all unixes, a number of daemons should be running in order for NFS to be functional, like for example the rpc.* processes, biod, nfsd and others. Once nfs is running, and in order to actually "share" or "export" your filesystem on your server, so remote clients are able to mount the nfs mount, in most cases you should edit the "/etc/exports" file. See other sections in this document (search on exportfs) on how to accomplish this. -- AIX: The following subsystems are part of the nfs group: nfsd, biod, rpc.lockd, rpc.statd, and rpc.mountd. The nfs subsystem (group) is under control of the "resource controller", so starting and stopping nfs is actually easy # startsrc -g nfs # stopsrc -g nfs Or use smitty. -- Redhat Linux: # /sbin/service nfs restart # /sbin/service nfs start # /sbin/service nfs stop
-- On some other Linux distros # /etc/init.d/nfs start # /etc/init.d/nfs stop # /etc/init.d/nfs restart -- Solaris: If the nfs daemons aren't running, then you will need to run: # /etc/init.d/nfs.server start -- HP-UX: Issue the following command on the NFS server to start all the necessary NFS processes (HP): # /sbin/init.d/nfs.server start Or if your machine is only a client: # cd /sbin/init.d # ./nfs.client start
=========================================== 3. Change ownership file/dir, adding users: =========================================== 3.1 Changing ownership: ----------------------chown -R user[:group] file/dir chown -R user[.group] file/dir
(SVR4) (bsd)
(-R recursive dirs) Examples: chown -R oracle:oinstall chown -R oracle:oinstall chown -R oracle:oinstall chown -R oracle:oinstall
/opt/u01 /opt/u02 /opt/u03 /opt/u04
-R means all subdirs also. chown rjanssen file.txt rjanssen. # # # # # # #
groupadd dba useradd oracle mkdir /usr/oracle mkdir /usr/oracle/9.0 chown -R oracle:dba /usr/oracle touch /etc/oratab chown oracle:dba /etc/oratab
- Give permissions as owner to user
Note: Not owner message: ----------------------->>> Solaris: it is possible to turn the chown command on or off (i.e., allow it to be used or disallow its use) on a system by altering the /etc/system file. The /etc/system file, along with the files in /etc/default should be thought of a "system policy files" -- files that allow the systems administrator to determine such things as whether root can login over the network, whether su commands are logged, and whether a regular user can change ownership of his own files. On a system disallowing a user to change ownership of his files (this is now the default), the value of rstchown is set to 1. Think of this as saying "restrict chown is set to TRUE". You might see a line like this in /etc/system (or no rstchown value at all): set rstchown=1 On a system allowing chown by regular users, this value will be set to 0 as shown here: set rstchown=0 Whenever the /etc/system rebooted for the changes Since there is no daemon there is no process that a hangup (HUP) to effect
file is changed, the system will have to be to take effect. process associated with commands such a chown, one could send the change in policy "on the fly".
Why might system administrators restrict access to the chown command? For a system on which disk quotas are enforced, they might not want to allow files to be "assigned" by one user to another user's quota. More importantly, for a system on which accountability is deemed important, system administrators will want to know who created each file on a system - whether to track down a potential system abuse or simply to ask if a file that is occupying space in a shared directory or in /tmp can be removed. When a system disallows use of the chown command, you can expect to see dialog like this: % chown wallace myfile chown: xyz: Not owner Though it would be possible to disallow "chowning" of files by changing permissions on /usr/bin/chown, such a change would not slow down most Unix users. They would simple copy the /usr/bin/chown file to their own directory and make their copy executable. Designed to be extensible, Unix will happily comply. Making the change in the /etc/system file blocks any chown operation from taking effect, regardless of where the executable is stored, who owns it,
and what it is called. If usage of chown is restricted in /etc/system, only the superuser can change ownership of files.
3.2 Add a user in Solaris: -------------------------Examples: # useradd -u 3000 -g other -d /export/home/tempusr -m -s /bin/ksh -c "temporary user" tempusr # useradd -u 1002 -g dba -d /export/home/avdsel -m -s /bin/ksh -c "Albert van der Sel" avdsel # useradd -u 1001 -g oinstall -G dba -d /export/home/oraclown -m -s /bin/ksh -c "Oracle owner" oraclown # useradd -u 1005 -g oinstall -G dba -d /export/home/brighta -m -s /bin/ksh -c "Bright Alley" brighta useradd -u 300 -g staff -G staff -d /home/emc -m -s /usr/bin/ksh -c "EMC user" emc a password cannot be specified using the useradd command. Use passwd to give the user a password: # passwd tempusr UID must be unique and is typically a number between 100 and 60002 GID is a number between 0 and 60002 Or use the graphical "admintool" or smc, the solaris management console. -- Profiles a user can use to set the environment: 1. Korn Shell ksh: -----------------When the POSIX or Korn Shell is your login shell, it looks for these following files and executes them, if they exist: /etc/profile This default system file is executed by the shell program and sets up default environment variables. .profile If this file exists in your home directory, it is executed next at login. At any time-this includes login time-the POSIX or Korn Shell is invoked, it looks for the file referenced by the following shell variable, and executes it, if it exists: ENV
When you invoke the shell, it looks for a shell variable called ENV which is usually set in your .profile. ENV is evaluated and if it is set to an existing file, that file is executed. By convention, ENV is usually set to .kshrc but may be set to any file name. These files provide the means for customizing the shell environment to fit your needs. 2. Bourne Shell sh: ------------------it looks for these following files and executes them, if they exist: /etc/profile .profle in the home directory, for example "/home/user1/.profile" 3.3 Add a user in AIX: ---------------------You can also use the useradd command, just as in Solaris. Or use the native "mkuser" command. # mkuser albert The mkuser command does not create password information for a user. It initializes the password field with an * (asterisk). Later, this field is set with the passwd or pwdadm command. New accounts are disabled until the passwd or pwdadm commands are used to add authentication information to the /etc/security/passwd file. You can use the Users application in Web-based System Manager to change user characteristics. You could also use the System Management Interface Tool (SMIT) "smit mkuser" fast path to run this command. The /usr/lib/security/mkuser.default file contains the default attributes for new users. This file is an ASCII file that contains user stanzas. These stanzas have attribute default values for users created by the mkuser command. Each attribute has the Attribute=Value form. If an attribute has a value of $USER, the mkuser command substitutes the name of the user. The end of each attribute pair and stanza is marked by a new-line character. There are two stanzas, user and admin, that can contain all defined attributes except the id and admin attributes. The mkuser command generates a unique id attribute. The admin attribute depends on whether the -a flag is used with the mkuser command.
A typical user stanza looks like the following: user: pgroup = staff groups = staff shell = /usr/bin/ksh home = /home/$USER auth1 = SYSTEM # mkuser [ -de | -sr ] [-attr Attributes=Value [ Attribute=Value... ] ] Name # mkuser [ -R load_module ] [ -a ] [ Attribute=Value ... ] Name
To create the davis user account with the default values in the /usr/lib/security/mkuser.default file, type: # mkuser davis To create the davis account with davis as an administrator, type: # mkuser -a davis Only the root user or users with the UserAdmin authorization can create davis as an administrative user. To create the davis user account and set the su attribute to a value of false, type: # mkuser su=false davis To create the davis user account that is identified and authenticated through the LDAP load module, type: # mkuser -R LDAP davis To add davis to the groups finance and accounting, enter: chuser groups=finance,accounting davis -- Add a user with the smit utility: -- --------------------------------Start SMIT by entering smit From the Main Menu, make the following selections: -Security and Users -Users -Add a User to the System The utility displays a form for adding new user information. Use the and keys to move through the form. Do not use until you are finished and ready to exit the screen. Fill in the appropriate fields of the Create User form (as listed in Create User Form) and press . The utility exits the form and creates the new user.
-- Using SMIT to Create a Group: -- ----------------------------Use the following procedure to create a group. Start SMIT by entering the following command: smit The utility displays the Main Menu. From the Main Menu, make the following selections: -Security and Users -Users -Add a Group to the System The utility displays a form for adding new group information. Type the group name in the Group Name field and press . The group name must be eight characters or less. The utility creates the new group, automatically assigns the next available GID, and exits the form Primary Authentication method of system: ---------------------------------------To check whether root has a primary authentication method of SYSTEM, use the following command: # lsuser -a auth1 root If needed, change the value by using # chuser auth1=SYSTEM root 3.4 Add a user in HP-UX: ------------------------- Example 1: Add user john to the system with all of the default attributes. # useradd john Add the user john to the system with a UID of 222 and a primary group of staff. # useradd -u 222 -g staff john -- Example 2: => => => => =>
Add a user called guestuser as per following requirements Primary group member of guests Secondary group member of www and accounting Shell must be /usr/bin/bash3 Home directory must be /home/guestuser
# useradd -g guests -G www,accounting -d /home/guests -s /home/guestuser/ -m guestuser # passwd guestuser
3.5 Add a user in Linux Redhat: ------------------------------You can use tools like useradd or groupadd to create new users and groups from the shell prompt. But an easier way to manage users and groups is through the graphical application, User Manager. Users are described in the /etc/passwd file Groups are stored on Red Hat Linux in the /etc/group file. Or invoke the Gnome Linuxconf GUI Tool by typing "linuxconf". In Red Hat Linux, linuxconf is found in the /bin directory.
================================ 4. Change filemode, permissions: ================================ Permissions are given to: u = user g = group o = other/world a = all file/directory permissions (or also called "filemodes") are: r = read w = write x = execute special modes are: X = sets execute if already set (this one is particularly sexy, look below) s = set setuid/setgid bit t = set sticky bit
Examples: --------readable by all, everyone % chmod a+r essay.001 to remove read write and execute permissions on the file biglist for the group and others
% chmod go-rwx biglist make executable: % chmod +x mycommand set mode: % chmod 644 filename rwxrwxrwx=777 rw-rw-rw-=666 rw-r--r--=644 corresponds to umask 022 r-xr-xr-x=555 rwxrwxr-x=775 1 = execute 2 = write 4 = read note that the total is 7 execute and read are: 1+4=5 read and write are: 2+4=6 read, write and exec: 1+2+4=7 and so on directories must always be executable... so a file with, say 640, means, the owner can read and write (4+2=6), the group can read (4) and everyone else has no permission to use the file (0). chmod -R a+X . This command would set the executable bit (for all users) of all directories and executables below the current directory that presently have an execute bit set. Very helpful when you want to set all your binary files executable for everyone other than you without having to set the executable bit of all your conf files, for instance. *wink* chmod -R g+w . This command would set all the contents below the current directory writable by your current group. chmod -R go-rwx This command would remove permissions for group and world users without changing the bits for the file owner. Now you don't have to worry that 'find . -type f -exec chmod 600 {}\;' will change your binary files non-executable. Further, you don't need to run an additional command to chmod your directories. chmod u+s /usr/bin/run_me_setuid This command would set the setuid bit of the file. It's simply easier than remembering which number to use when wanting to setuid/setgid, IMHO.
======================== 5. About the sticky bit: ======================== - This info is valid for most Unix OS including Solaris and AIX: ---------------------------------------------------------------A 't' or 'T' as the last character of the "ls -l" mode characters indicates that the "sticky" (save text image) bit is set. See ls(1) for an explanation the distinction between 't' and 'T'. The sticky bit has a different meaning, depending on the type of file it is set on... sticky bit on directories ------------------------[From chmod(2)] If the mode bit S_ISVTX (sticky bit) is set on a directory, files inside the directory may be renamed or removed only by the owner of the file, the owner of the directory, or the superuser (even if the modes of the directory would otherwise allow such an operation). [Example] drwxrwxrwt
104 bin
bin
14336 Jun
7 00:59 /tmp
Only root is permitted to turn the sticky bit on or off. In addition the sticky bit applies to anyone who accesses the file. The syntax for setting the sticky bit on a dir /foo directory is as follows: chmod +t /foo sticky bit on regular files --------------------------[From chmod(2)] If an executable file is prepared for sharing, mode bit S_ISVTX prevents the system from abandoning the swap-space image of the program-text portion of the file when its last user terminates. Then, when the next user of the file executes it, the text need not be read from the file system but can simply be swapped in, thus saving time. [From HP-UX Kernel Tuning and Performance Guide] Local paging. When applications are located remotely, set the "sticky bit" on the applications binaries, using the chmod +t command. This tells the system to page the text to the local disk. Otherwise, it is "retrieved" across the network. Of course, this would only apply when there is actual paging occurring. More recently, there is a kernel parameter, page_text_to_local, which when set to 1, will tell the kernel to page all NFS executable text pages to local swap space.
[Example] -r-xr-xr-t /usr/bin/vi
6 bin
bin
24111111111664 Nov 14
2000
Solaris: -------The sticky bit on a directory is a permission bit that protects files within that directory. If the directory has the sticky bit set, only the owner of the file, the owner of the directory, or root can delete the file. The sticky bit prevents a user from deleting other users' files from public directories, such as uucppublic: castle% ls -l /var/spool/uucppublic drwxrwxrwt 2 uucp uucp castle%
512 Sep 10 18:06 uucppublic
When you set up a public directory on a TMPFS temporary file system, make sure that you set the sticky bit manually. You can set sticky bit permissions by using the chmod command to assign the octal value 1 as the first number in a series of four octal values. Use the following steps to set the sticky bit on a directory: 1. If you are not the owner of the file or directory, become superuser. 2. Type chmod <1nnn> and press Return. 3. Type ls -l and press Return to verify that the permissions of the file have changed. The following example sets the sticky bit permission on the pubdir directory: castle% chmod 1777 pubdir castle% ls -l pubdir drwxrwxrwt 2 winsor staff castle%
512 Jul 15 21:23 pubdir
================ 6. About SETUID: ================ Each process has three user ID's: the real user ID (ruid) the effective user ID (euid) and the saved user ID (suid) The real user ID identifies the owner of the process, the effective uid is used in most access control decisions, and the saved uid stores a previous user ID so that it can be restored later.
Similar, a process has three group ID's. When a process is created by fork, it inherits the three uid's from the parent process. When a process executes a new file by exec..., it keeps its three uid's unless the set-user-ID bit of the new file is set, in which case the effective uid and saved uid are assigned the user ID of the owner of the new file. When setuid (set-user identification) permission is set on an executable file, a process that runs this file is granted access based on the owner of the file (usually root), rather than the user who created the process. This permission enables a user to access files and directories that are normally available only to the owner. The setuid permission is shown as an s in the file permissions. For example, the setuid permission on the passwd command enables a user to change passwords, assuming the permissions of the root ID are the following: castle% ls -l /usr/bin/passwd -r-sr-sr-x 3 root sys castle%
96796 Jul 15 21:23 /usr/bin/passwd
You setuid permissions by using the chmod command to assign the octal value 4 as the first number in a series of four octal values. Use the following steps to setuid permissions: 1. If you are superuser. 2. Type chmod 3. Type ls -l permissions of
not the owner of the file or directory, become <4nnn> and press Return. and press Return to verify that the the file have changed.
The following example sets setuid permission on the myprog file: #chmod 4555 myprog -r-sr-xr-x 1 winsor #
staff
12796 Jul 15 21:23 myprog
The setgid (set-group identification) permission is similar to setuid, except that the effective group ID for the process is changed to the group owner of the file and a user is granted access based on permissions granted to that group. The /usr/bin/mail program has setgid permissions: castle% ls -l /usr/bin/mail -r-x-s-x 1 bin mail castle%
64376 Jul 15 21:27 /usr/bin/mail
When setgid permission is applied to a directory, files subsequently created in the directory belong to the group
the directory belongs to, not to the group the creating process belongs to. Any user who has write permission in the directory can create a file there; however, the file does not belong to the group of the user, but instead belongs to the group of the directory. You can set setgid permissions by using the chmod command to assign the octal value 2 as the first number in a series of four octal values. Use the following steps to set setgid permissions: 1. If you are not the owner of the file or directory, become superuser. 2. Type chmod <2nnn> and press Return. 3. Type ls -l and press Return to verify that the permissions of the file have changed. The following example sets setuid permission on the myprog2 file: #chmod 2551 myprog2 #ls -l myprog2 -r-xr-s-x 1 winsor #
staff
26876 Jul 15 21:23 myprog2
========================= 7. Find command examples: ========================= Introduction The find command allows the Unix user to process a set of files and/or directories in a file subtree. You can specify the following: where to search (pathname) what type of file to search for (-type: directories, data files, links) how to process the files (-exec: run a process against a selected file) the name of the file(s) (-name) perform logical operations on selections (-o and -a) Search for file with a specific name in a set of files (-name) EXAMPLES -------# find . -name "rc.conf" -print This command will search in the current directory and all sub directories for a file named rc.conf. Note: The -print option will print out the path of any file that is found with that name. In general -print wil print out the path of any file that meets the find criteria. # find . -name "rc.conf" -exec chmod o+r '{}' \;
This command will search in the current directory and all sub directories. All files named rc.conf will be processed by the chmod -o+r command. The argument '{}' inserts each found file into the chmod command line. The \; argument indicates the exec command line has ended. The end results of this command is all rc.conf files have the other permissions set to read access (if the operator is the owner of the file). How to find text in a set of files: ----------------------------------# find . -exec grep "www.athabasca" '{}' \; -print This command will search in the current directory and all sub directories. All files that contain the string will have their path printed to standard output. # find .
-exec grep "CI_ADJ_TYPE" {} \; -print
This command search all subdirs all files to find text CI_ADJ_TYPE How to find files of certain size: ---------------------------------# find / -xdev -size +2048 -ls | sort -r +6 # find . -xdev -size +2048 -ls | sort -r +6 This command will find all files in the root directory larger than 1 MB. How to find files between dates: -------------------------------thread 1: --------olddate="200407010001" newdat="200407312359" touch -t $olddate ./tmpoldfile touch -t $newdat ./tmpnewfile find /path/to/directory -type f ./tmpnewfile
-newer a ./tmpoldfile ! -newer
a
the "-newer a " means access time, you can use "-newer m " for modify time thread 2: --------Touch 2 files, start_date and stop_date, like this: $ touch -t 200603290000.00 start_date $ touch -t 200603290030.00 stop_date
Ok, start_date is 03/29/06 midnight, stop_date is 03/29/06 30 minutes after midnight. You might want to do a ls -al to check. On to find, you can find -newer and then ! -newer, like this: $ find /dir -newer start_date ! -newer stop_date -print Combine that with ls -l, you get: $ find /dir -newer start_date ! -newer stop_date -print0 | xargs -0 ls -l (Or you can try -exec to execute ls -l. I am not sure of the format, so you have to muck around a little bit) HTH . thread 3: --------ls -lrtR | awk '{print $6$7"\t"$9}'|grep Nov thread 4: --------1) between 2 dates (say 15 Aug 08 to 31 Aug 08) touch -t 20080150000 /tmp/start touch -t 20080831000 /tmp/finish find / -size +10k -newer /tmp/start -a ! -newer /tmp/finish 2) later than a specified date (say 25 Aug 08) touch -t 20080250000 /tmp/ref find / -size +10k ! -newer /tmp/ref
Other examples: --------------# find . -name file -print # find / -name $1 -exec ls -l {} \; # find / -user nep -exec ls -l {} \; >nepfiles.txt In English: search from the root directory for any files owned by nep and execute an ls -l on the file when any are found. Capture all output in nepfiles.txt. # find $HOME -name \*.txt -print In order to protect the asterisk from being expanded by the shell, it is necessary to use a backslash to escape the asterisk as in: # find / -atime +30 -print This prints files that have not been accessed in the last 30 days
# find / -atime +100 -size +500000c -print The find search criteria can be combined. This command will locate and list all files that were last accessed more than 100 days ago, and whose size exceeds 500,000 bytes. # find /opt/bene/process/logs -name 'ALBRACHT*' \; # # # # # # # # # # # # # #
-mtime +90 -exec rm {}
find /example /new/example -exec grep -l 'Where are you' {} \; find / \( -name a.out -o -name '*.o' \) -atime +7 -exec rm {} \; find . -name '*.trc' -mtime +3 -exec rm {} \; find / -fsonly hfs -print cd /; find . ! -path ./Disk -only -print | cpio -pdxm /Disk cd /; find . -path ./Disk -prune -o -print | cpio -pdxm /Disk cd /; find . -xdev -print | cpio -pdm /Disk find -type f -print | xargs chmod 444 find -type d -print | xargs chmod 555 find . -atime +1 -name '*' -exec rm -f {} \; find /tmp -atime +1 -name '*' -exec rm -f {} \; find /usr/tmp -atime +1 -name '*' -exec rm -f {} \; find / -name core -exec rm -f {} \; find . -name "*.dbf" -mtime -2 -exec ls {} \;
* Search and list all files from current directory and down for the string ABC: find ./ -name "*" -exec grep -H ABC {} \; find ./ -type f -print | xargs grep -H "ABC" /dev/null egrep -r ABC * * Find all files of a given type from current directory on down: find ./ -name "*.conf" -print * Find all user files larger than 5Mb: find /home -size +5000000c -print * Find all files owned by a user (defined by user id number. see /etc/passwd) on the system: (could take a very long time) find / -user 501 -print * Find all files created or updated in the last five minutes: (Great for finding effects of make install) find / -cmin -5 * Find all users in group 20 and change them to group 102: (execute as root) find / -group 20 -exec chown :102 {} \; * Find all suid and setgid executables: find / \( -perm -4000 -o -perm -2000 \) -type f -exec ls -ldb {} \; find / -type f -perm +6000 -ls Example: -------cd /database/oradata/pegacc/archive archdir=`pwd` if [ $archdir=="/database/oradata/pegacc/archive" ] then find . -name "*.dbf" -mtime +5 -exec rm {} \; else
echo "error in onderhoud PEGACC archives" >> /opt/app/oracle/admin/log/archmaint.log fi Example: -------The following example shows how to find files larger than 400 blocks in the current directory: # find . -size +400 -print REAL COOL EXAMPLE: -----------------This example could even help in recovery of a file: In some rare cases a strangely-named file will show itself in your directory and appear to be un-removable with the rm command. Here is will the use of ls -li and find with its -inum [inode] primary does the job. Let's say that ls -l shows your irremovable as -rw-------
1 smith
smith
0 Feb
1 09:22 ?*?*P
Type: ls -li to get the index node, or inode. 153805 -rw-------
1 smith
smith
0 Feb
1 09:22 ?*?^P
The inode for this file is 153805. Use find -inum [inode] to make sure that the file is correctly identified. % find -inum 153805 -print ./?*?*P Here, we see that it is. Then used the -exec functionality to do the remove. . % find . -inum 153805 -print -exec /bin/rm {} \; Note that if this strangely named file were not of zero-length, it might contain accidentally misplaced and wanted data. Then you might want to determine what kind of data the file contains and move the file to some temporary directory for further investigation, for example: % find . -inum 153805 -print -exec /bin/mv {} unknown.file \; Will rename the file to unknown.file, so you can easily inspect it.
COOL EXAMPLE: Using find and cpio to create really good backups: ---------------------------------------------------------------Suppose you have a lot of subdirs and files in "/dir1/dira" Now you want to copy, or backup, this to "/dir2/dirb" And not only just the files and subdirs, BUT ALSO all filemodes (permissions), ownership information, acl's etc.. Then DO NOT USE "cp -R" or something similar. Instead use "find" in combination with the "cpio" backup command. # cd /dir1/dira # find . | cpio -pvdm /dir2/dirb Note: difference betweeen mtime and atime: -----------------------------------------In using the find command where you want to delete files older than a certain date, you can use commands like find . -name "*.log" -mtime +30 -exec rm {} \; or find . -name "*.dbf" -atime +30 -exec rm {} \; Why should you choose, or not choose, between atime and mtime? It is important to distinguish between a file or directory's change time (ctime), access time (atime), and modify time (mtime). ctime -- In UNIX, it is not possible to tell the actual creation time of a file. The ctime--change time-is the time when changes were made to the file or directory's inode (owner, permissions, etc.). The ctime is also updated when the contents of a file change. It is needed by the dump command to determine if the file needs to be backed up. You can view the ctime with the ls -lc command. atime -- The atime--access time--is the time when the data of a file was last accessed. Displaying the contents of a file or executing a shell script will update a file's atime, for example. mtime -- The mtime--modify time--is the time when the actual contents of a file was last modified. This is the time displayed in a long directoring listing (ls -l). Thats why backup utilities use the mtime when performing incremental backups: When the utility reads the data for a file that is to be included in a backup, it does not affect the file's modification time, but it does affect the file's access time.
So for most practical reasons, if you want to delete logfiles (or other files) older than a certain date, its best to use the mtime attribute. How to make those times visible? "ls -l" "ls -lc" "ls -lm"
shows atime shows ctime shows mtime
"istat filename" will show all three. pago-am1:/usr/local/bb>istat bb18b3.tar.gz Inode 20 on device 10/9 File Protection: rw-r--r-Owner: 100(bb) Group: 100(bb) Link count: 1 Length 427247 bytes Last updated: Last modified: Last accessed:
Tue Aug 14 11:01:46 2001 Thu Jun 21 07:36:32 2001 Thu Nov 01 20:38:46 2001
=================== 7. Crontab command: =================== Cron is uded to schedule or run periodically all sorts of executable programs or shell scripts, like backupruns, housekeeping jobs etc.. The crond daemon makes it all happen. Who has access to cron, is on most unixes determined by the "cron.allow" and "cron.deny" files. Every allowed user, can have it's own "crontab" file. The crontab of root, is typically used for system administrative jobs. On most unixes the relevant files can be found in: /var/spool/cron/crontabs or /var/adm/cron or /etc/cron.d For example, on Solaris the /var/adm/cron/cron.allow and /var/adm/cron/cron.deny files control which users can use the crontab command. Most common usage: - if you just want a listing: crontab -l - if you want to edit and change: crontab -e crontab [ -e | -l | -r | -v | File ]
-e: edit, submit
-r remove, -l list
A crontab file contains entries for each cron job. Entries are separated by newline characters. Each crontab file entry contains six fields separated by spaces or tabs in the following form: minute
hour
day_of_month
month
0
0
*
8
weekday *
command /u/harry/bin/maintenance
Notes: -----Note 1: start and stop cron: ----------------------------- Solaris and some other unixes: The proper way to stop and restart cron are: # /etc/init.d/cron stop # /etc/init.d/cron start In Solaris 10 you could use the following command as well: # svcadm refresh cron # svcadm restart cron -- Other way to restart cron: In most unixes, cron is started by init and there is a record in the /etc/initab file which makes that happen. Check if your system has indeed a record of cron in the inittab file. The type of start should be "respawn", which means that should the superuser do a "kill -9 crond", the cron daemon is simply restarted again. Again, preferrably, there should be a stop and start script to restart cron. Especially on AIX, there is no true way to restart cron in a neat way. Not via the Recourse Control startscr command, or script, a standard method is available. Just kill crond and it will be restarted. -- On many linux distros: to restart the cron daemon, you could do either a "service crond restart" or a "service crond reload". Note 2: -------
Create a cronjobs file You can do this on your local computer in Notepad or you can create the file directly on your Virtual Server using your favorite UNIX text editor (pico, vi, etc). Your file should contain the following entries: MAILTO="[email protected] " 0 1 1 1-12/3 * /usr/local/bin/vnukelog This will run the command "/usr/local/bin/vnukelog" (which clears all of your log files) at 1 AM on the first day of the first month of every quarter, or January, April, July, and October (1-12/3). Obviously, you will need to substitute a valid e-mail address in the place of "[email protected] ". If you have created this file on your local computer, FTP the file up to your Virtual Server and store it in your home directory under the name "cronjobs" (you can actually use any name you would like). Register your cronjobs file with the system After you have created your cronjobs file (and have uploaded it to your Virtual Server if applicable), you need to Telnet to your server and register the file with the cron system daemon. To do this, simply type: crontab cronjobs Or if you used a name other than "cronjobs", substitute the name you selected for the occurrence of "cronjobs" above. Note 3: ------# use /bin/sh to run commands, no matter what /etc/passwd says SHELL=/bin/sh # mail any output to `paul', no matter whose crontab this is MAILTO=paul # # run five minutes after midnight, every day 5 6-18 * * * /opt/app/oracle/admin/scripts/grepora.sh # run at 2:15pm on the first of every month -- output mailed to paul 15 14 1 * * $HOME/bin/monthly # run at 10 pm on weekdays, annoy Joe 0 22 * * 1-5 mail -s "It's 10pm" joe%Joe,%%Where are your kids?% 23 0-23/2 * * * echo "run 23 minutes after midn, 2am, 4am ..., everyday" 5 4 * * sun echo "run at 5 after 4 every sunday" 2>&1 means: It means that standard error is redirected along with standard output. Standard error could be redirected to a different file, like
ls > toto.txt 2> error.txt If your shell is csh or tcsh, you would redirect standard output and standard error like this lt >& toto.txt Csh or tcsh cannot redirect standard error separately. Note 4: ------thread Q: > > > >
Isn't there a way to refresh cron to pick up changes made using crontab -e? I made the changes but the specified jobs did not run. I'm thinking I need to refresh cron to pick up the changes. Is this true? Thanks.
A: Crontab -e should do that for you, that's the whole point of using it rather than editing the file yourself. Why do you think the job didn't run? Post the crontab entry and the script. Give details of the version of Tru64 and the patch level. Then perhaps we can help you to figure out the real cause of the problem. Hope this helps A: I have seen the following problem when editing the cron file for another user: crontab -e idxxxxxx This changed the control file, when I verified with crontab -l the contents was correctly shown, but the cron daemon did not execute the new contents. To solve the problem, I needed to follow the following commands: su - idxxxxxx crontab -l |crontab This seems to work ... since then I prefer the following su - idxxxxxx crontab -e which seems to work also ... Note 5: -------
On AIX it is observed, that if the "daemon=" attribute of a user is set to be false, this user cannot use crontab, even if the account is placed in cron.allow. You need to set the attribute to "daemon=true". * daemon system * false.
Defines whether the user can execute programs using the resource controller (SRC). Possible values: true or
Note 6: ------If you want to quick test the crontab of a user: su - user and put the following in the crontab of that user: * * * * *
date >/tmp/elog
After checking the /tmp/elog file, which will rapidly fills with dates, don't forget to remove the crontab entry shown above. Note 7: the at and atq commands: -------------------------------On many unix systems the scheduling "at" command and "atq" commands are available. With "at", you can schedule commands, and with "atq" you can view all your, or other users, scheduled tasks. atq- Display the jobs queued to run at specified times For example, on Solaris: The at command is used to schedule jobs for execution at a later time. Unlike crontab, which schedules a job to happen at regular intervals, a job submitted with at executes once, at the designated time. To submit an at job, type at followed by the time that you would like the program to execute. You'll see the at> prompt displayed and it's here that you enter the at commands. When you are finished entering the at command, press control-d to exit the at prompt and submit the job as shown in the following example: # at 07:45am today at> who > /tmp/log at> job 912687240.a at Thu Jun 6 07:14:00
When you submit an at job, it is assigned a job identification number, which becomes its filename along with the .a extension. The file is stored in the /var/spool/cron/atjobs directory. In much the same way as it schedules crontab jobs, the cron daemon controls the scheduling of at files.
=========================== 8. Job control, background: =========================== To put a sort job (or other job) in background: # sort < foo > bar & To show jobs: # jobs To show processes: # ps # ps -ef | grep ora Job in foreground -> background: Ctrl-Z (suspend) #bg or bg jobID Job in background -> foreground: # fg %jobid Stop a process: # kill -9 3535
(3535 is the pid, process id)
Stop a background process you may try this: # kill -QUIT 3421
-- Kill all processes of a specific users: -- --------------------------------------To kill all processes of a specific user, enter: # ps -u [user-id] -o pid | grep -v PID | xargs kill -9 Another way: Use who to check out your current users and their terminals. Kill all processes related to a specific terminal: # fuser -k /dev/pts[#] Yet another method: Su to the user-id you wish to kill all processes of and enter: # su - [user-id] -c kill -9 -1 Or su - to that userid, and use the killall command, which is available on most unix'es, like for example AIX. # killall
So in order to kill all processes of a user: # kill -9 -1
# not on all unixes
or # killall
# not on all unixes
The nohup command: -----------------When working with the UNIX operating system, there will be times when you will want to run commands that are immune to log outs or unplanned login session terminations. This is especially true for UNIX system administrators. The UNIX command for handling this job is the nohup (no hangup) command. Normally when you log out, or your session terminates unexpectedly, the system will kill all processes you have started. Starting a command with nohup counters this by arranging for all stopped, running, and background jobs to ignore the SIGHUP signal. The syntax for nohup is: nohup command [arguments] You may optionally add an ampersand to the end of the command line to run the job in the background: nohup command [arguments] & If you do not redirect output from a process kicked off with nohup, both standard output (stdout) and standard error (stderr) are sent to a file named nohup.out. This file will be created in $HOME (your home directory) if it cannot be created in the working directory. Real-time monitoring of what is being written to nohup.out can be accomplished with the "tail -f nohup.out" command. Although the nohup command is extremely valuable to UNIX system administrators, it is also a must-know tool for others who run lengthy or critical processes on UNIX systems The nohup command runs the command specified by the Command parameter and any related Arg parameters, ignoring all hangup (SIGHUP) signals. Use the nohup command to run programs in the background after logging off. To run a nohup command in the background, add an & (ampersand) to the end of the command. Whether or not the nohup command output is redirected to a terminal, the output is appended to the nohup.out file in the current directory. If the nohup.out file is not writable in the current directory, the output is redirected
to the $HOME/nohup.out file. If neither file can be created nor opened for appending, the command specified by the Command parameter is not invoked. If the standard error is a terminal, all output written by the named command to its standard error is redirected to the same file descriptor as the standard output. To run a command in the background after you log off, enter: $ nohup find / -print & After you enter this command, the following is displayed: 670 $ Sending output to nohup.out The process ID number changes to that of the background process started by & (ampersand). The message Sending output to nohup.out informs you that the output from the find / -print command is in the nohup.out file. You can log off after you see these messages, even if the find command is still running. Example of ps -ef on a AIX5 system: [LP 1]root@ol16u209:ps -ef UID PID PPID C STIME TTY TIME CMD root 1 0 0 Oct 17 - 0:00 /etc/init root 4198 1 0 Oct 17 - 0:00 /usr/lib/errdemon root 5808 1 0 Oct 17 - 1:15 /usr/sbin/syncd 60 oracle 6880 1 0 10:27:26 - 0:00 ora_lgwr_SPLDEV1 root 6966 1 0 Oct 17 - 0:00 /usr/ccs/bin/shlap root 7942 43364 0 Oct 17 - 0:00 sendmail: accepting connections alberts 9036 9864 0 20:41:49 - 0:00 sshd: alberts@pts/0 root 9864 44426 0 20:40:21 - 0:00 sshd: alberts [priv] root 27272 36280 1 20:48:03 pts/0 0:00 ps -ef oracle 27856 1 0 10:27:26 - 0:01 ora_smon_SPLDEV1 oracle 31738 1 0 10:27:26 - 0:00 ora_dbw0_SPLDEV1 oracle 31756 1 0 10:27:26 - 0:00 ora_reco_SPLDEV1 alberts 32542 9036 0 20:41:49 pts/0 0:00 -ksh maestro 33480 34394 0 05:59:45 - 0:00 /prj/maestro/maestro/bin/batchman -parm 32000 root 34232 33480 0 05:59:45 - 0:00 /prj/maestro/maestro/bin/jobman maestro 34394 45436 0 05:59:45 - 0:00 /prj/maestro/maestro/bin/mailman -parm 32000 -- 2002 OL16U209 CONMAN UNIX 6. root 34708 1 0 13:55:51 lft0 0:00 /usr/sbin/getty /dev/console oracle 35364 1 0 10:27:26 - 0:01 ora_cjq0_SPLDEV1 oracle 35660 1 0 10:27:26 - 0:04 ora_pmon_SPLDEV1 root 36280 32542 0 20:45:06 pts/0 0:00 -ksh root 36382 43364 0 Oct 17 - 0:00 /usr/sbin/rsct/bin/IBM.ServiceRMd root 36642 43364 0 Oct 17 - 0:00 /usr/sbin/rsct/bin/IBM.CSMAgentRMd root 36912 43364 0 Oct 17 - 0:03 /usr/opt/ifor/bin/i4lmd -l /var/ifor/logdb -n clwts root 37186 43364 0 Oct 17 - 0:00 /etc/ncs/llbd
root 37434 43364 0 Oct 17 - 0:17 /usr/opt/ifor/bin/i4llmd -b -n wcclwts -l /var/ifor/llmlg root 37738 37434 0 Oct 17 - 0:00 /usr/opt/ifor/bin/i4llmd -b -n wcclwts -l /var/ifor/llmlg root 37946 1 0 Oct 17 - 0:00 /opt/hitachi/HNTRLib2/bin/hntr2mon -d oracle 38194 1 0 Oct 17 - 0:00 /prj/oracle/product/9.2.0.3/bin/tnslsnr LISTENER -inherit root 38468 43364 0 Oct 17 - 0:00 /usr/sbin/rsct/bin/IBM.AuditRMd root 38716 1 0 Oct 17 - 0:00 /usr/bin/itesmdem itesrv.ini /etc/IMNSearch/search/ imnadm 39220 1 0 Oct 17 - 0:00 /usr/IMNSearch/httpdlite/httpdlite -r /etc/IMNSearch/httpdlite/httpdlite.con root 39504 36912 0 Oct 17 - 0:00 /usr/opt/ifor/bin/i4lmd -l /var/ifor/logdb -n clwts root 39738 43364 0 Oct 17 - 0:01 /usr/DynamicLinkManager/bin/dlmmgr root 40512 43364 0 Oct 17 - 0:01 /usr/sbin/rsct/bin/rmcd -r root 40784 43364 0 Oct 17 - 0:00 /usr/sbin/rsct/bin/IBM.ERrmd root 41062 1 0 Oct 17 - 0:00 /usr/sbin/cron was 41306 1 0 Oct 17 - 2:10 /prj/was/java/bin/java -Xmx256m -Dwas.status.socket=32776 -Xms50m -Xbootclas oracle 42400 1 0 10:27:26 - 0:02 ora_ckpt_SPLDEV1 root 42838 1 0 Oct 17 - 0:00 /usr/sbin/uprintfd root 43226 43364 0 Oct 17 - 0:00 /usr/sbin/nfsd 3891 root 43364 1 0 Oct 17 - 0:00 /usr/sbin/srcmstr root 43920 43364 0 Oct 17 - 0:00 /usr/sbin/aixmibd root 44426 43364 0 Oct 17 - 0:00 /usr/sbin/sshd -D root 44668 43364 0 Oct 17 - 0:00 /usr/sbin/portmap root 44942 43364 0 Oct 17 - 0:00 /usr/sbin/snmpd root 45176 43364 0 Oct 17 - 0:00 /usr/sbin/snmpmibd maestro 45436 1 0 Oct 17 - 0:00 /prj/maestro/maestro/bin/netman root 45722 43364 0 Oct 17 - 0:00 /usr/sbin/inetd root 45940 43364 0 Oct 17 - 0:00 /usr/sbin/muxatmd root 46472 43364 0 Oct 17 - 0:00 /usr/sbin/hostmibd root 46780 43364 0 Oct 17 - 0:00 /etc/ncs/glbd root 46980 43364 0 Oct 17 - 0:00 /usr/sbin/qdaemon root 47294 1 0 Oct 17 - 0:00 /usr/local/sbin/syslogng -f /usr/local/etc/syslog-ng.conf root 47484 43364 0 Oct 17 - 0:00 /usr/sbin/rpc.lockd daemon 48014 43364 0 Oct 17 - 0:00 /usr/sbin/rpc.statd root 48256 43364 0 Oct 17 - 0:00 /usr/sbin/rpc.mountd root 48774 43364 0 Oct 17 - 0:00 /usr/sbin/biod 6 root 49058 43364 0 Oct 17 - 0:00 /usr/sbin/writesrv [LP 1]root@ol16u209: Another example of ps -ef on a AIX5 system: # ps -ef UID root
PID 1
PPID 0
C 0
STIME Jan 23
TTY -
TIME CMD 0:33 /etc/init
root 69706 1 0 Jan 23 - 0:00 /usr/lib/errdemon root 81940 1 0 Jan 23 - 0:00 /usr/sbin/srcmstr root 86120 1 2 Jan 23 - 236:39 /usr/sbin/syncd 60 root 98414 1 0 Jan 23 - 0:00 /usr/ccs/bin/shlap64 root 114802 81940 0 Jan 23 - 0:32 /usr/sbin/rsct/bin/IBM.CSMAgentRMd root 135366 81940 0 Jan 23 - 0:00 /usr/sbin/sshd -D root 139446 81940 0 Jan 23 - 0:07 /usr/sbin/rsct/bin/rmcd -r root 143438 1 0 Jan 23 - 0:00 /usr/sbin/uprintfd root 147694 1 0 Jan 23 - 0:26 /usr/sbin/cron root 155736 1 0 Jan 23 - 0:00 /usr/local/sbin/syslog-ng -f /usr/local/etc/syslog-ng.conf root 163996 81940 0 Jan 23 - 0:00 /usr/sbin/rsct/bin/IBM.ERrmd root 180226 81940 0 Jan 23 - 0:00 /usr/sbin/rsct/bin/IBM.ServiceRMd root 184406 81940 0 Jan 23 - 0:00 /usr/sbin/qdaemon root 200806 1 0 Jan 23 - 0:08 /opt/hitachi/HNTRLib2/bin/hntr2mon -d root 204906 81940 0 Jan 23 - 0:00 /usr/sbin/rsct/bin/IBM.AuditRMd root 217200 1 0 Jan 23 - 0:00 ./mflm_manager root 221298 81940 0 Jan 23 - 1:41 /usr/DynamicLinkManager/bin/dlmmgr root 614618 1 0 Apr 03 lft0 0:00 -ksh reserve 1364024 1548410 0 07:10:10 pts/0 0:00 -ksh root 1405140 1626318 1 08:01:38 pts/0 0:00 ps -ef root 1511556 614618 2 07:45:52 lft0 0:41 tar -cf /dev/rmt1.1 /spl reserve 1548410 1613896 0 07:10:10 - 0:00 sshd: reserve@pts/0 root 1613896 135366 0 07:10:01 - 0:00 sshd: reserve [priv] root 1626318 1364024 1 07:19:13 pts/0 0:00 -ksh
Some more examples: # nohup somecommand & sleep 1; tail -f preferred-name # nohup make bzImage & # tail -f nohup.out # nohup make modules 1> modules.out 2> modules.err & # tail -f modules.out
========================================== 9. Backup commands, TAR, and Zipped files: ========================================== For SOLARIS as well as AIX, and many other unix'es, the following commands can be used: tar, cpio, dd, gzip/gunzip, compress/uncompress, backup and restore.
Very important: If you will backup to tape, make sure you know what is your "rewinding" class and "nonrewinding" class of your tapedevice. 9.1 tar: Short for "Tape Archiver": =================================== Some examples should explain the usage of "tar" to create backups, or to create easy to transport .tar files. Create a backup to tape device 0hc of file sys01.dbf # tar -cvf /dev/rmt/0hc /u01/oradata/sys01.dbf # tar -rvf /dev/rmt/0hc /u02/oradata/data_01.dbf -c -r -x -v -t
create append extract verbose list
Extract the contents of example.tar and display the files as they are extracted. # tar -xvf example.tar Create a tar file named backup.tar from the contents of the directory /home/ftp/pub # tar -cf backup.tar /home/ftp/pub list contents of example.tar to the screen # tar -tvf example.tar to restore the file /home/bcalkins/.profile from the archive: - First we do a backup: # tar -cvf /dev/rmt/0 /home/bcalkins - And later we do a restore: # tar -xcf /dev/rmt/0 /home/bcalkins/.profile If you use an absolute path, you can only restore in "a like" destination directory. If you use a relative path, you can restore in any directory. In this case, use tar with a relative pathname, for example if you want to backup /home/bcalkins change to that directory and use # tar -cvf backup_oracle_201105.tar ./* To extract the directory conv: # tar -xvf /dev/rmt0 /u02/oradata/conv Example: --------
mt -f /dev/rmt1 rewind mt -f /dev/rmt1.1 fsf 6 tar -xvf /dev/rmt1.1 /data/download/expdemo.zip
Most common errors messages with tar: -------------------------------------- 0511-169: A directory checksum error on media: MediaName not equal to Number Possible Causes From the command line, you issued the tar command to extract files from an archive that was not created with the tar command. -- 0511-193: An error occurred while reading from the media Possible Causes You issued the tar command to read an archive from a tape device that has a different block size than when the archive was created. Solution: # chdev -l rmt0 -a block_size=0 -- File too large:
Extra note of tar command on AIX: --------------------------------If you need to backup multiple large mountpoints to a large tape, you might think you can use something like: tar tar tar tar tar tar tar tar tar tar tar
-cvf -rvf -rvf -rvf -rvf -rvf -rvf -rvf -rvf -rvf -rvf
/dev/rmt1 /dev/rmt1 /dev/rmt1 /dev/rmt1 /dev/rmt1 /dev/rmt1 /dev/rmt1 /dev/rmt1 /dev/rmt1 /dev/rmt1 /dev/rmt1
/spl /prj /opt /usr /data /backups /u01/oradata /u02/oradata /u03/oradata /u04/oradata /u05/oradata
Actually on AIX this is not OK. The tape will rewind after each tar command, effectively you will end up with ONLY the last backupstatement.
You should use the non-rewinding class instead, like for example: tar -cf /dev/rmt1.1 /spl tar -cf /dev/rmt1.1 /apps tar -cf /dev/rmt1.1 /prj tar -cf /dev/rmt1.1 /software tar -cf /dev/rmt1.1 /opt tar -cf /dev/rmt1.1 /usr tar -cf /dev/rmt1.1 /data tar -cf /dev/rmt1.1 /backups #tar -cf /dev/rmt1.1 /u01/oradata #tar -cf /dev/rmt1.1 /u02/oradata #tar -cf /dev/rmt1.1 /u03/oradata #tar -cf /dev/rmt1.1 /u04/oradata #tar -cf /dev/rmt1.1 /u05/oradata Use this table to decide on which class to use: The following table shows the names of the rmt special files and their characteristics. Special_File Rewind_on_Close Retension_on_Open Density_Setting /dev/rmt*Yes No #1 /dev/rmt*.1No No #1 /dev/rmt*.2Yes Yes #1 /dev/rmt*.3No Yes #1 /dev/rmt*.4Yes No #2 /dev/rmt*.5No No #2 /dev/rmt*.6Yes Yes #2 /dev/rmt*.7No Yes #2
To restore an item from a logical tape, use commands as in the following example: mt -f /dev/rmt1 rewind mt -f /dev/rmt1.1 fsf 2 of block 3. mt -f /dev/rmt1.1 fsf 7 of block 8.
in order to put the pointer to the beginning in order to put the pointer to the beginning
Now you can use a command like for example: tar -xvf /dev/rmt1.1 /backups/oradb/sqlnet.log Another example: mt -f /dev/rmt1 rewind mt -f /dev/rmt1.1 fsf 8 tar -xvf /dev/rmt1.1 /u01/oradata/spltrain/temp01.dbf Tapedrives on Solaris: ----------------------
Tape dvices on Solaris are named like /dev/rmt/0 or /dev/rmt/1 The default is /dev/rmt0. This also configured in the "/kernel/drv/st.conf" file. If you need to add support for a tape device, you need to modify this file. First tape device name: /dev/rmt/0 Second tape device name: /dev/rmt/1 You can also add special character letter to specify density using following format /dev/rmt/ZX Z is tape drive number such as 0,1..n X can be any one of following (as supported by your device, read the manual of your tape device & controller to see if all of them supported or not): l - Low density m - Medium density h - High density u - Ultra density c - Compressed density n - No rewinding For example to specify the first, drive with high-density with no rewinding use device /dev/rmt/0hn. First drive, rewinding /dev/rmt/0 First drive, nonrewinding /dev/rmt/0n Second drive, rewinding /dev/rmt/1 Second drive, nonrewinding /dev/rmt/1n
Example Backupscript on AIX: ---------------------------#!/usr/bin/ksh # # # #
BACKUP-SCRIPT SPL SERVER PSERIES 550 DIT IS DE PRIMAIRE BACKUP, NAAR DE TAPEROBOT RMT1. OPMERKING: ER LOOPT NAAST DEZE BACKUP, OOK NOG EEN BACKUP VAN DE /backup DISK NAAR DE INTERNE TAPEDRIVE RMT0.
# OMDAT WE NOG NIET GEHEEL IN BEELD HEBBEN OF WE VOORAF DE BACKUP APPLICATIES MOETEN # STOPZETTEN, IS DIT SCRIPT NOG IN REVISIE.
# VERSIE: 0.1 # DATUM : 27-12-2005 # DOEL VAN HET SCRIPT: # - STOPPEN VAN DE APPLICATIES # - VERVOLGENS BACKUP NAAR TAPE # - STARTEN VAN DE APPLICATIES # CONTROLEER VOORAF OF DE TAPELIBRARY GELADEN IS VIA "/opt/backupscripts/load_lib.sh" BACKUPLOG=/opt/backupscripts/backup_to_rmt1.log export BACKUPLOG DAYNAME=`date +%a`;export DAYNAME DAYNO=`date +%d`;export DAYNO ######################################## # 1. REGISTRATIE STARTTIJD IN EEN LOG # ######################################## echo "-----------------" >> ${BACKUPLOG} echo "Start Backup 550:" >> ${BACKUPLOG} date >> ${BACKUPLOG} ######################################## # 2. STOPPEN APPLICATIES # ######################################## #STOPPEN VAN ALLE ORACLE DATABASES su - oracle -c "/opt/backupscripts/stop_oracle.sh" sleep 30 #STOPPEN VAN WEBSPHERE cd /prj/was/bin ./stopServer.sh server1 -username admin01 -password vga88nt sleep 30 #SHUTDOWN ETM instances: su - cissys -c '/spl/SPLDEV1/bin/splenviron.sh stop"' sleep 2 su - cissys -c '/spl/SPLDEV2/bin/splenviron.sh stop"' sleep 2 su - cissys -c '/spl/SPLCONF/bin/splenviron.sh stop"' sleep 2 su - cissys -c '/spl/SPLPLAY/bin/splenviron.sh stop"' sleep 2 su - cissys -c '/spl/SPLTST3/bin/splenviron.sh stop"' sleep 2
-e SPLDEV1 -c "spl.sh -t -e SPLDEV2 -c "spl.sh -t -e SPLCONF -c "spl.sh -t -e SPLPLAY -c "spl.sh -t -e SPLTST3 -c "spl.sh -t
su - cissys stop"' sleep 2 su - cissys stop"' sleep 2 su - cissys stop"' sleep 2 su - cissys stop"' sleep 2 su - cissys stop"' sleep 2
-c '/spl/SPLTST1/bin/splenviron.sh -e SPLTST1 -c "spl.sh -t -c '/spl/SPLTST2/bin/splenviron.sh -e SPLTST2 -c "spl.sh -t -c '/spl/SPLDEVP/bin/splenviron.sh -e SPLDEVP -c "spl.sh -t -c '/spl/SPLPACK/bin/splenviron.sh -e SPLPACK -c "spl.sh -t -c '/spl/SPLDEVT/bin/splenviron.sh -e SPLDEVT -c "spl.sh -t
#STOPPEN SSH DEMON stopsrc -s sshd sleep 2 date >> /opt/backupscripts/running.log who >> /opt/backupscripts/running.log ######################################## # 3. BACKUP COMMANDS # ######################################## case $DAYNAME in Tue) tapeutil -f /dev/smc0 tapeutil -f /dev/smc0 move ;; Wed) tapeutil -f /dev/smc0 tapeutil -f /dev/smc0 move ;; Thu) tapeutil -f /dev/smc0 tapeutil -f /dev/smc0 move ;; Fri) tapeutil -f /dev/smc0 tapeutil -f /dev/smc0 move ;; Sat) tapeutil -f /dev/smc0 tapeutil -f /dev/smc0 move ;; Mon) tapeutil -f /dev/smc0 tapeutil -f /dev/smc0 move ;; esac
move 256 4116 4101 256 move 256 4117 4100 256 move 256 4118 4099 256 move 256 4119 4098 256 move 256 4120 4097 256 move 256 4121 4096 256
sleep 50 echo "Starten van de backup zelf" >> ${BACKUPLOG} mt -f /dev/rmt1 rewind tar -cf /dev/rmt1.1 /spl
tar tar tar tar tar tar tar tar tar tar tar tar tar tar tar tar tar
-cf -cf -cf -cf -cf -cf -cf -cf -cf -cf -cf -cf -cf -cf -cf -cf -cf
/dev/rmt1.1 /dev/rmt1.1 /dev/rmt1.1 /dev/rmt1.1 /dev/rmt1.1 /dev/rmt1.1 /dev/rmt1.1 /dev/rmt1.1 /dev/rmt1.1 /dev/rmt1.1 /dev/rmt1.1 /dev/rmt1.1 /dev/rmt1.1 /dev/rmt1.1 /dev/rmt1.1 /dev/rmt1.1 /dev/rmt1.1
/apps /prj /software /opt /usr /data /backups /u01/oradata /u02/oradata /u03/oradata /u04/oradata /u05/oradata /u06/oradata /u07/oradata /u08/oradata /home /backups3
sleep 10 # TIJDELIJKE ACTIE date >> /opt/backupscripts/running.log ps -ef | grep pmon >> /opt/backupscripts/running.log ps -ef | grep BBL >> /opt/backupscripts/running.log ps -ef | grep was >> /opt/backupscripts/running.log who >> /opt/backupscripts/running.log defragfs /prj # EIND TIJDELIJKE ACTIE ######################################## # 4. STARTEN APPLICATIES # ######################################## #STARTEN SSH DEMON startsrc -s sshd sleep 2 #STARTEN VAN ALLE ORACLE DATABASES su - oracle -c "/opt/backupscripts/start_oracle.sh" sleep 30 #STARTEN ETM instances: su - cissys -c '/spl/SPLDEV1/bin/splenviron.sh start"' sleep 2 su - cissys -c '/spl/SPLDEV2/bin/splenviron.sh start"' sleep 2 su - cissys -c '/spl/SPLCONF/bin/splenviron.sh start"' sleep 2 su - cissys -c '/spl/SPLPLAY/bin/splenviron.sh start"' sleep 2
-e SPLDEV1 -c "spl.sh -t -e SPLDEV2 -c "spl.sh -t -e SPLCONF -c "spl.sh -t -e SPLPLAY -c "spl.sh -t
su - cissys start"' sleep 2 su - cissys start"' sleep 2 su - cissys start"' sleep 2 su - cissys start"' sleep 2 su - cissys start"' sleep 2 su - cissys start"' sleep 2
-c '/spl/SPLTST3/bin/splenviron.sh -e SPLTST3 -c "spl.sh -t -c '/spl/SPLTST1/bin/splenviron.sh -e SPLTST1 -c "spl.sh -t -c '/spl/SPLTST2/bin/splenviron.sh -e SPLTST2 -c "spl.sh -t -c '/spl/SPLDEVP/bin/splenviron.sh -e SPLDEVP -c "spl.sh -t -c '/spl/SPLPACK/bin/splenviron.sh -e SPLPACK -c "spl.sh -t -c '/spl/SPLDEVT/bin/splenviron.sh -e SPLDEVT -c "spl.sh -t
#STARTEN VAN WEBSPHERE cd /prj/was/bin ./startServer.sh server1 -username admin01 -password vga88nt sleep 30 ######################################## # 5. REGISTRATIE EINDTIJD IN EEN LOG # ######################################## #Laten we het tapenummer en einddtijd registreren in de log: tapeutil -f /dev/smc0 inventory | head -88 | tail -2 echo "Einde backup 550:" >> ${BACKUPLOG} date >> ${BACKUPLOG}
Some examples about day vars: ----------------------------DAYNAME=`date +%a`;export DAYNAME echo $DAYNAME Thu DAYNO=`date +%d`;export DAYNO echo $DAYNO 29 weekday=`date +%a%A`; export weekday echo $weekday ThuThursday weekday=`date +%a-%A`
>> ${BACKUPLOG}
echo $weekday Thu-Thursday %a %A %b %B %c
Displays the locale's abbreviated weekday name. Displays the locale's full weekday name. Displays the locale's abbreviated month name. Displays the locale's full month name.
Displays the locale's appropriate date and time representation. This is the default. %C Displays the first two digits of the four-digit year as a decimal number (00-99). A year is divided by 100 and truncated to an integer. %d Displays the day of the month as a decimal number (01-31). In a two-digit field, a 0 is used as leading space fill. %D Displays the date in the format equivalent to %m/%d/%y. %e Displays the day of the month as a decimal number (1-31). In a two-digit field, a blank space is used as leading space fill.
9.2 compress and uncompress: ============================ # compress -v bigfile.exe Would compress bigfile.exe and rename that file to bigfile.exe.Z. # uncompress *.Z would uncompress the files *.Z 9.3 gzip: ========= To compress a file using gzip, execute the following command: # gzip filename.tar This will become filename.tar.gz To decompress: # gzip -d filename.tar.gz # gunzip filename.tar.gz # gzip -d users.dbf.gz 9.4 bzip2: ==========
#bzip2 filename.tar This will become filename.tar.bz2 9.5 dd: ======= Solaris: -------# dd if= of= to duplicate a tape: # dd if=/dev/rmt/0 of=/dev/rmt/1 to clone a disk with the same geometry: # dd if=/dev/rdsk/c0t1d0s2 of=/dev/rdsk/c0t4d0s2 bs=128 AIX: ---same command syntax apply to IBM AIX. Here is an AIX pSeries machine with floppydrive example: clone a diskette: # dd if=/dev/fd0 of=/tmp/ddcopy # dd if=/tmp/ddcopy of=/dev/fd0 Note: On Linux distros the device associated to the floppy drive is also /dev/fd0 9.6 cpio: ========= solaris: -------cpio copy-out: cpio -o copy_in : cpio -i pass : cpio -p # # # #
cd /var/bigspace cpio -idmv Linux9i_Disk1.cpio.gz cpio -idmv Linux9i_Disk2.cpio.gz cpio -idmv Linux9i_Disk3.cpio.gz
#
cpio -idmv < 9204_solaris_release.cpio
# cd /work # ls -R | cpio -ocB > /dev/rmt/0
# cd /work # cpio -icvdB < /dev/rmt/0 d c v c
will create directories as needed will create header information in ascii format for portability verbose character heading in file
AIX: ---AIX uses the same syntax. Usually, you should use the following command: # cpio -idmv < filename.cpio Copying directories with cpio: -----------------------------cpio is very good in cloning directories, or making backups, because it copies files and directories inclusive their ownership and permissions. Example: -------Just cd to the directory that you want to clone and use a command similar to the following examples. # find . -print | cpio -pdl /u/disk11/jdoe/fiber # find . -print | cpio -pdm /a/dev # find . -print | cpio -pdl /home/jim/newdir # find . -print | cpio -pdmv /backups2/CONV2-0212 # find . -print | cpio -pdmv /backups2/SPLcobAS40 # find . -print | cpio -pdmv /backups2/SPLcobAS40sp2 # find . -print | cpio -pdmv /backups2/runtime/SPLTST2 The p in the flags, stands for pass-through cd /spl/SPLDEV1 find . -print | cpio -pdmv /spl/SPLDEVT find . -print | cpio -pdmv /backups2/data # find . -print | cpio -pdmv /data/documentum/dmadmin/backup_1008/dba_cluster # find . -print | cpio -pdmv /data/documentum/dmadmin/backup_1008/dmw_et3 # find . -print | cpio -pdmv /data/documentum/dmadmin/backup_1008/dmw_et # find . -print | cpio -pdmv /data/documentum/dmadmin/backup_1508/dmw_eu find . -print | cpio -pdmv /data/emcdctm/home2
find . -print | cpio -pdmv /data/documentum/dmadmin/backup_1809/dmw_et find . -print | cpio -pdmv /data/documentum/dmadmin/backup_1809/dmw_et3
find find find find
. . . .
-print -print -print -print
| | | |
cpio cpio cpio cpio
-pdmv -pdmv -pdmv -pdmv
/data/documentum/dmadmin/appl/l13appl /data/documentum/dmadmin/appl/l14appl /data/documentum/dmadmin/backup_3110/dmw_et /appl/emcdctm/dba_save_311007
Example: -------Use cpio copy-pass to copy a directory structure to another location: # find path -depth -print | cpio -pamVd /new/parent/dir Example: -------Become superuser or assume an equivalent role. Change to the appropriate directory. # cd filesystem1 Copy the directory tree from filesystem1 to filesystem2 by using a combination of the find and cpio commands. # find . -print -depth | cpio -pdm filesystem2 Example: -------Copying directories Both cpio and tar may be used to copy directories while preserving ownership, permissions, and directory structure. cpio example: cd fromdir find . | cpio -pdumv todir tar example: cd fromdir; tar cf - . | (cd todir; tar xfp -) tar example over a compressed ssh tunnel: tar cvf - fromdir | gzip -9c | ssh user@host 'cd todir; gzip -cd | tar xpf -' Errors: -------
Errors sometimes found with cpio: cpio: 0511-903 cpio: 0511-904 1.Try using with -c option: cpio -imdcv < filename.cpio
9.7 the pax command: ==================== Same for AIX and SOLARIS. The pax utility supports several archive formats, including tar and cpio. The syntax for the pax command is as follows: pax -r: Read mode .when -r is specified, pax extracts the filenames and directories found in the archive. The archive is read from disk or tape. If an extracted file is a directory, the hierarchy is extracted as well. The extracted files are created relative to the current directory. None: List mode. When neither -r or -w is specified, pax displays the filenames and directories found in the archive file. The list is written to standard output. -w: Write mode. If you want to create an archive, you use -w. Pax writes the contents of the file to the standard output in an archive format specified by the -x option. -rw: Copy mode. When both -r and -w are specified, pax copies the specified files to the destination directory. most important options: -a = append to the end of an existing archive -b = block size, multiple of 512 bytes -c = you can specify filepatterns -f = specifies the pathname of the input or output archive -p = aemo a does not preserve file access time e preserve everything: user id, group id, filemode bits, etc.. m does not preserve file modification times o preserve uid and gid P preserve filemode bits -x = specifies the archive format.
Examples: To copy current directory contents to tape, use -w mode and -f # pax -w -f /dev/rmt0 To list a verbose table of contents stored on tape rmt0, use None mode and f # pax -v -f /dev/rmt0 9.8 pkzip25: ============ PKZIP Usage: Usage: pkzip25 [command] [options] zipfile [@list] [files...] Examples: View .ZIP file contents: pkzip25 zipfile Create a .ZIP file: pkzip25 -add zipfile file(s)... Extract files from .ZIP: pkzip25 -extract zipfile These are only basic examples of PKZIP's capability About "-extract" switch: extract extract files from a .ZIP file. Its a configurable switch. -- all - all files in .ZIP file -- freshen - only files in the .ZIP file that exist in the target directory and that are "newer" than those files will be extracted -- update - files in the .ZIP file which already exist in the target directory and that are "newer" than those files as well as files that are "not" in the target directory will be extracted default = all Example: # pkzip25 -ext=up save.zip
9.9 SOLARIS: ufsdump and ufsrestore: ==================================== level 0 is an full backup, 1-9 are incremental backups Examples: ---------
# ufsdump 0ucf /dev/rmt/0 /users # ufsdump 0ucf sparc1:/dev/rmt/0 /export/home # ufsrestore f /dev/rmt/0 filename # ufsrestore rf sparc1:/dev/rmt/0 filename 9.10 AIX: mksysb: ================ The mksysb command creates an installable image of the rootvg. This is synonym to say that mksysb creates a backup of the operating system (that is, the root volume group). You can use this backup to reinstall a system to its original state after it has been corrupted. If you create the backup on tape, the tape is bootable and includes the installation programs needed to install from the backup. To generate a system backup and create an /image.data file (generated by the mkszfile command) to a tape device named /dev/rmt0, type: # mksysb -i /dev/rmt0 To generate a system backup and create an /image.data file with map files (generated by the mkszfile command) to a tape device named /dev/rmt1, type: # mksysb -m /dev/rmt1 To generate a system backup with a new /image.data file, but exclude the files in directory /home/user1/tmp, create the file "/etc/exclude.rootvg" containing the line /home/user1/tmp/, and type: # mksysb -i -e /dev/rmt1 This command will backup the /home/user1/tmp directory but not the files it contains. To generate a system backup file named /mksysb_images/node1 and a new /image.data file for that image, type: # mksysb -i /userimage/node1 There will be four images on the mksysb tape, and the fourth image will contain ONLY rootvg JFS or JFS2 mounted file systems. The target tape drive must be local to create a bootable tape. The following is a description of mksysb's four images. +---------------------------------------------------------+ | Bosboot | Mkinsttape | Dummy TOC | rootvg | | Image | Image | Image | data | |-----------+--------------+-------------+----------------| |<----------- Block size 512 ----------->| Blksz defined | | | by the device |
+---------------------------------------------------------+
Special notes: -------------Note 1: mksysb problem ---------------------Question: I'm attempting to restore a mksysb tape to a system that only has 18GB of drive space available for the Rootvg. Does the mksysb try to restore these mirrored LVs, or does it just make one copy? If it is trying to rebuild the mirror, is there a way that I can get around that? Answer: I had this same problem and received a successful resolution. I place those same tasks here: 1) Create a new image.data file, run mkszfile file. 2) Change the image.data as follows: a) cd / b) vi image.data c) In each lv_data stanza of this file, change the values of the copies line by one-half (i.e. copies = 2, change to copies = 1) Also, change the number of Physical Volumes "hdisk0 hdisk1" to "hdisk0". d) Save this file. 3) Create another mksysb from the command line that will utilize the newly edited image.data file by the command: mksysb /dev/rmt0 (Do not use smit and do not run with the -i flag, both will generate a new image.data file 4) Use this new mksysb to restore your system on other box without mirroring. Note 2: How to restore specific files from a mksysb tape: --------------------------------------------------------$ tctl fsf 3 $ restore -xvf /dev/rmt0.1 ./your/file/name For example, if you need to get the vi command back, put the mksysb tape in the tape drive (in this case, /dev/rmt0) and do the following: cd /
# get to the root directory
tctl -f /dev/rmt0 rewind
# rewind the tape
tctl -f /dev/rmt0.1 fsf 3 rewind
# move the tape to the third file, no
restore -xqf /dev/rmt0.1 -s 1 ./usr/bin/vi no rewind
# extract the vi binary,
Further explanation why you must use the fsf 3 (fast forward skip file 3): The format of the tape is as follows: 1. A BOS boot image 2. A BOS install image 3. A dummy Table Of Contents 4. The system backup of the rootvg So if you just need to restore some files, first forward the tape pointer to position 3, counting from 0. Note 3: How to restore specific files from a mksysb FILE -------------------------------------------------------See also note 2 view: restore -Tvqf [mksysb file] To restore: restore -xvqf [mksysb file] [file name] Note 4: How to restore a directory from a mksysb FILE -----------------------------------------------------Simply using the restore command. restore -xvdf ./your/directory The dot at the front of the path is important. The "-d" flag indicates that this is a directory and everything in it should be restored. If you omit that, you'll restore an empty directory. The directory will be restored underneath whatever directory you're in. So if you're in your home directory it might create: /home/azhou/your/directory. With a mksysb image on disk you don't have any positioning to do, like with a tape. Note 5: Performing a mksysb migration with CD installation ----------------------------------------------------------
You can perform a mksysb migration with a CD installation of AIXr 5.3 Step 1. Prepare your system for installation: Prepare for migrating to the AIX 5.3 BOS by completing the following steps: - Insert the AIX Volume 1 CD into the CD-ROM device. - Shut down the target system. If your machine is currently running, power it off by following these steps: Log in as the root user. Type shutdown -F. If your system does not automatically power off, place the power switch in the Off (0) position. Attention: You must not turn on the system unit until instructed to do so. - Turn on all attached external devices. External devices include the following: Terminals CD-ROM drives DVD-ROM drives Tape drives Monitors External disk drives Turning on the external devices first is necessary so that the system unit can identify each peripheral device during the startup (boot) process. - If your MKSYSB_MIGRATION_DEVICE is a tape, insert the tape for the mksysb in the tape drive. If your MKSYSB_MIGRATION_DEVICE is a CD or DVD, and there is an additional CD or DVD drive on the system (other than the one being used to boot AIX), insert the mksysb CD or DVD in the drive to avoid being prompted to swap medias. - Insert your customized bosinst.data supplemental diskette in the diskette drive. If the system does not have a diskette drive, use the network installation method for mksysb migration. Step 2. Boot from your installation media: The following steps migrate your current version of the operating system to AIX 5.3. If you are using an ASCII console that was not defined in your previous system, you must define it. For more information about defining ASCII consoles, see Step 3. Setting up an ASCII terminal. Turn the system unit power switch from Off (0) to On (|).
When the system beeps twice, press F5 on the keyboard (or 5 on an ASCII terminal). If you have a graphics display, you will see the keyboard icon on the screen when the beeps occur. If you have an ASCII terminal (also called a tty terminal), you will see the word "keyboard" when the beeps occur. Note: If your system does not boot using the F5 key (or the 5 key on an ASCII terminal), refer to your hardware documentation for information about how to boot your system from an AIX product CD. The system begins booting from the installation media. The mksysb migration installation proceeds as an unattended installation (non-prompted) unless the MKSYSB_MIGRATION_DEVICE is the same CD or DVD drive as the one being used to boot and install the system. In this case, the user is prompted to switch the product CD for the mksysb CD or DVD(s) to restore the image.data and the /etc/filesystems file. After this happens the user is prompted to reinsert the product media and the installation continues. When it is time to restore the mksysb image, the same procedure repeats. The BOS menus do not currently support mksysb migration, so they cannot be loaded. In a traditional migration, if there are errors that can be fixed by prompting the user for information through the menus, the BOS menus are loaded. If such errors or problems are encountered during mksysb migration, the installation asserts and an error stating that the migration cannot continue displays. Depending on the error that caused the assertion, information specific to the error might be displayed. If the installation asserts, the LED shows "088". Note 6: create a mksysb tape MANUALLY ------------------------------------THIS NOTE DESCRIBES NOT A SUPPORTED METHOD, AND IS NOT CHECKED.. Here we do not mean the "mksysb -i /dev/rmtx" method, but...: Question: I have to clone a standalone 6H1 equipped with a 4mm tape, from another 6H1 which is node of an SP and which does not own a tape ! The consequence is that my source mksysb is a file that is recorded in /spdata/sys1/install/aixxxx/images How will I copy this file to a tape to create the correct mksysb tape that could be used to restore on my target machine ? Answer: using the following method in the case the two server are in the same AIX level and kernel type (32/64 bits, jfs or jfs2)
- the both servers must communicate over an IP network and have .rhosts file documented (for using rsh) cp /var/adm/ras/bosinst.data /bosinst.data mkszfile copy these files (bosinst.data and image.data) under "/" on the remote system on the server: tctl -f /dev/rmt0 status if the block size is not 512: # chdev -l /dev/rmt0 -a block_size=512 tctl -f /dev/rmt0 rewind bosboot -a -d /dev/rmt0.1 (create the boot image on the first file of mksysb) mkinsttape /dev/rmt0.1 (create the second file on the mksysb with image.data, bosinst.data, and oher files like drivers and commands) echo " Dummy tape TOC" | dd of=/dev/rmt0.1 conv=sync bs=512 > /dev/null 2>&1 (create the third file "dummy toc") create a named pipe: mknod /tmp/pipe p and run the mksysb as this: dd if=/tmp/pipe | rsh "server_hostname" dd of=/dev/rmt0.1 & mksysb /tmp/pipe this last command create the fourth file with "rootvg" in backup/restore format Note 7: Creating a root volume group backup on CD or DVD with the ISO9660 format ------------------------------------------------------------------------------Follow this procedure to create a root volume group backup on CD or DVD with the ISO9660 format. You can use Web-based System Manager or SMIT to create a root volume group backup on CD or DVD with the ISO9660 format, as follows: Use the Web-based System Manager Backup and Restore application and select System backup wizard method.
This method lets you create bootable or non-bootable backups on CD-R, DVD-R, or DVD-RAM media. OR To create a backup to CD, use the smit mkcd fast path. To create a backup to DVD, use the smit mkdvd fast path and select ISO9660 (CD format). The following procedure shows you how to use SMIT to create a system backup to CD. (The SMIT procedure for creating a system backup to an ISO9660 DVD is similar to the CD procedure.) Type the smit mkcd fast path. The system asks whether you are using an existing mksysb image. Type the name of the CD-R device. (This can be left blank if the Create the CD now? field is set to no.) If you are creating a mksysb image, select yes or no for the mksysb creation options, Create map files? and Exclude files?. Verify the selections, or change as appropriate. The mkcd command always calls the mksysb command with the flags to extend /tmp. You can specify an existing image.data file or supply a user-defined image.data file. See step 16. Enter the file system in which to store the mksysb image. This can be a file system that you created in the rootvg, in another volume group, or in NFS-mounted file systems with read-write access. If this field is left blank, the mkcd command creates the file system, if the file system does not exist, and removes it when the command completes. Enter the file systems in which to store the CD or DVD file structure and final CD or DVD images. These can be file systems you created in the rootvg, in another volume group, or in NFS-mounted file systems. If these fields are left blank, the mkcd command creates these file systems, and removes them when the command completes, unless you specify differently in later steps in this procedure. If you did not enter any information in the file systems' fields, you can select to have the mkcd command either create these file systems in the rootvg, or in another volume group. If the default of rootvg is chosen and a mksysb image is being created, the mkcd command adds the file systems to the exclude file and calls the mksysb command with the -e exclude files option. In the Do you want the CD or DVD to be bootable? field, select yes to have a boot image created on the CD or DVD. If you select no, you must boot from a product CD at the same version.release.maintenance level, and then select to install the system backup from the system backup CD. If you change the Remove final images after creating CD? field to no, the file system for the CD images
(that you specified earlier in this procedure) remains after the CD has been recorded. If you change the Create the CD now? field to no, the file system for the CD images (that you specified earlier in this procedure) remains. The settings that you selected in this procedure remain valid, but the CD is not created at this time. If you intend to use an Install bundle file, type the full path name to the bundle file. The mkcd command copies the file into the CD file system. You must have the bundle file already specified in the BUNDLES field, either in the bosinst.data file of the mksysb image or in a userspecified bosinst.data file. When this option is used to have the bundle file placed on the CD, the location in the BUNDLES field of the bosinst.data file must be as follows: /../usr/sys/inst.data/user_bundles/bundle_file_name To place additional packages on the CD or DVD, enter the name of the file that contains the packages list in the File with list of packages to copy to CD field. The format of this file is one package name per line. If you are planning to install one or more bundles after the mksysb image is restored, follow the directions in the previous step to specify the bundle file. You can then use this option to have packages listed in the bundle available on the CD. If this option is used, you must also specify the location of installation images in the next step. Enter the location of installation images that are to be copied to the CD file system (if any) in the Location of packages to copy to CD field. This field is required if additional packages are to be placed on the CD (see the previous step). The location can be a directory or CD device. You can specify the full path name to a customization script in the Customization script field. If given, the mkcd command copies the script to the CD file system. You must have the CUSTOMIZATION_FILE field already set in the bosinst.data file in the mksysb image or else use a userspecified bosinst.data file with the CUSTOMIZATION_FILE field set. The mkcd command copies this file to the RAM file system. Therefore, the path in the CUSTOMIZATION_FILE field must be as follows: /../filename You can use your own bosinst.data file, rather than the one in the mksysb image, by typing the full path name of your bosinst.data file in the User supplied bosinst.data file field. To turn on debugging for the mkcd command, set Debug output? to yes. The debug output goes to the smit.log. You can use your own image.data file, rather than the image.data file in the mksysb image, by typing the full path name of your image.data file for the User supplied image.data file field.
Note 8: 0301-150 bosboot: Invalid or no boot device specified! -------------------------------------------------------------== Technote: APAR status Closed as program error. Error description On a system, that does not have tape support installed, running mkszfile will show the following error: 0301-150 bosboot: Invalid or no boot device specified. Local fix Install device support for scsi tape devices. Problem summary Error message when creating backup if devices.scsi.tape.rte not installed even if the system does not have a tape drive. Problem conclusion Redirect message to /dev/null. Temporary fix Ignore message. Comments APAR information APAR number IY52551 IY95261 Reported component name AIX 5L POWER V5 Reported component ID 5765E6200 Reported release 520 Status CLOSED PER PE NoPE HIPER NoHIPER Submitted date 2004-01-12 Closed date 2004-01-12 Last modified date 2004-02-27 == Technote: APAR status Closed as program error. Error description If /dev/ipldevice is missing, mksfile will show the bosboot usage statement. 0301-150 bosboot: Invalid or no boot device specified!
Local fix Problem summary If /dev/ipldevice is missing, mksfile will show the bosboot usage statement. 0301-150 bosboot: Invalid or no boot device specified! Problem conclusion Do not run bosboot against /dev/ipldevice. Temporary fix Comments APAR information APAR number IY95261 Reported component name AIX 5.3 Reported component ID 5765G0300 Reported release 530 Status CLOSED PER PE NoPE HIPER NoHIPER Submitted date 2007-02-22 Closed date 2007-02-22 Last modified date 2007-06-06 APAR is sysrouted FROM one or more of the following: APAR is sysrouted TO one or more of the following: Publications Referenced Fix information Fixed component name AIX 5.3 Fixed component ID 5765G0300 == thread: Q: > > Someone out there knows the fix for this one; if you get a moment, would you > mind giving me the fix? > > > # mksysb -i /dev/rmt0 > > /dev/ipldevice not found > A: The ipldevice file is probably deleted from your /dev directory, or point to wrong
entry. The '/dev/ipldevice' file is (re)created in boot time 2nd phase. For additional information look into /sbin/rc.boot script... The ipldevice entry type is hardlink. Usually point to /dev/rhdiskN, assuming that boot device is hdiskN. Check your system and you should got similar ... find /dev -links 2 -ls .... 8305 0 crw------- 2 root system 14, 1 Feb 20 2005 /dev/rhdisk0 8305 0 crw------- 2 root system 14, 1 Feb 20 2005 /dev/ipldevice ... (The first cloumn of the output is the inode number) So, you can recreate the wrong, or missing ipdevice file. 'bootinfo -b' says the physical boot device name. For exapmle: ln -f /dev/rhdisk0 /dev/ipldevice I hope this will solve your bosboot problem. Q: I was installing Atape driver and noticed bosboot failure when installp calls bosboot with /dev/ipldevice. Messages below: 0503-409 installp: bosboot verification starting... 0503-497 installp: An error occurred during bosboot verification processing. Inspection of /dev showed no ipldevice file I was able to easily recreate the /dev/ipldevice using ln /dev/rhdisk0 /dev/ipldevice then successfully install the Atape driver software. After reboot /dev/ipldevice is missing again???. Environment is p5 520 AIX 5.3 ML1 mirrored internal drives hdisk0 and hdisk1 in rootvg I have 5.3 ML2 (but have not applied yet) I don't see any APAR's in ML2 regarding /dev/ipldevice problems. A: Are you using EMC disk? There is a known problem with the later Powerpath versions where the powerpath startup script removes the /dev/ipldevice file if there is more than one device listed in the bootlist. A: Yes, running EMC PowerPath 4.3 for AIX, with EMC Clariion CX600 Fibre disks attached to SAN. I always boot from, and mirror the OS on IBM
internal disks. We order 4 internal IBM drives. Two for primary OS and mirror, the other two for alt_disk and mirrors. Thanks for the tip. I will investigate at EMC Powerlink site for fix. I know PowerPath 4.4 for AIX is out, but still pretty new. A: ipldevice is a link to the rawdevice (rhdisk0 , not hdisk0) -----Original Message----From: IBM AIX Discussion List [mailto:[email protected] ] On Behalf Of Robert Miller Sent: Wednesday, April 07, 2004 6:13 PM To: [email protected] Subject: Re: 64 Bit Kernel It may be one of those odd IBMisms where they want to call something a certain name so they put it in as a link to the actual critter... Looking on my box, the /dev/ipldevice has the same device major and minor numbers as hdisk0 - tho it is interesting that ipldevice is a character device, where a drive is usually a block device: mybox:rmiller$ ls crw------- 2 root mybox:rmiller$ ls brw------- 1 root
-l /dev/ipl* system 23, 0 Jan 15 2002 /dev/ipldevice -l /dev/hdisk0 system 23, 0 Sep 13 2002 /dev/hdisk0
A: > Hi, > > > >
AIX 5.3 I have a machine where /dev/ipldevice doesn't exit I can reboot it safely ? How I can I re-create it ?
> Thanks in advance I did this today, and there is probably a more accepted way. I made a hard link from my rhdiskX device to /dev/ipldevice. If your boot device is /dev/hdisk0, then the command line would be as follows: ln /dev/rhdisk0 /dev/ipldevice Again, there is probably a more acceptable way to achieve this, but it worked for me.
== thread: how to recover from an invalid or no boot device error in AIX Description When running the command "bosboot -ad /dev/ipldevice" in IBM AIX, you get the following error: 0301-150 bosboot: Invalid or no boot device specified! A device specified with the bosboot -d command is not valid. The bosboot command was unable to finish processing because it could not locate the required boot device. The installp command calls the bosboot command with /dev/ipldevice. If this error does occur, it is probably because /dev/ipldevice does not exist. /dev/ipldevice is a link to the boot disk. To determine if the link to the boot device is missing or incorrect : 1) Verify the link exists: # ls -l /dev/ipldevice ls: 0653-341 The file /dev/ipldevice does not exist. 2) In this case, it does not exist. To identify the boot disk, enter "lslv -m hd5". The boot disk name displays. # lslv -m hd5 hd5:N/A LP PP1 PV1 PP2 PV2 PP3 PV3 0001 0001 hdisk4 0001 hdisk1 In this example the boot disk name is hdisk4 and hdisk1. 3) Create a link between the boot device indicated and the /dev/ipldevice file. Enter: # ln /dev/boot_device_name /dev/ipldevice (An example of boot_device_name is rhdisk0.) In my case, I ran: # ln /dev/rhdisk4 /dev/ipldevice 4) Now run the bosboot command again: # bosboot -ad /dev/ipldevice Example lslv -m hd5; ln /dev/rhdisk4 /dev/ipldevice; bosboot -ad /dev/ipldevice Q: p595 LPAR no longer sees SAN boot disk
Hello, we have a wierd and urgent problem, with a few of our p595 LPARs running AIX 5.3. The LPARs ran AIX 5.3 TL 7 and booted off EMC SAN disks, using EMC Powerpath. Every boot we run "pprootdev on" and "pprootdev fix". We can issue "bosboot -a" and we can reboot the machines. Now, on two occasions, right after the update to AIX 5.3 Technology Level 9, Service Pack 3 the system fails to reboot. When starting the partition to the SMS menu you can see the correct (rootvg) devices being scanned, but the devices are NOT listed as a possible boot device. When booting off a NIM server and trying to restore an mksysb, the entire rootvg (all disks in it) is invisible and cannot be selected. Some user-defined volume groups are also "missing". When giving the partition access to a 'new' EMC disk, this new disk shows up and can be used to restore the mksysb. When that is complete, the original disks show up properly (using lspv etc.) and seem perfectly alright. Anyone ran into this same problem? Any idea's, suggestions, fixes?? A: Being able to see a device but being unable to access the data area sounds like a SCSI disk reservation problem.
Note 9: Other mksysb errors on AIX 5.3: --------------------------------------It turns out, that on AIX 5.3, on certain ML/TL levels (below TL 6), an mksysb error turns up, if you have other volume groups defined other than rootvg, while there is NO filesystem created on those Volume groups. Solution: create a filesystem, even only a "test" or "dummy" filesystem, on those VG's. >> thread 1: Q: Hi can't find any information about "backup structure of volume group, vios". included service: "savevgstruct vgname" working with errors: # lsvg rootvg
vg_dev datavg_dbs # /usr/ios/cli/ioscli savevgstruct vg_dev Creating information file for volume group vg_dev.. Some error messages may contain invalid information for the Virtual I/O Server environment. cat: 0652-050 Cannot open /tmp/vgdata/vg_dev/fs_data_tmp. # ls -al /tmp/vgdata/vg_dev/ total 16 drwxr-xr-x 2 root staff 256 Apr 02 08:38 . drwxrwxr-x 5 root system 256 Apr 02 08:20 .. -rw-r--r-- 1 root staff 2002 Apr 02 08:35 filesystems -rw-r--r-- 1 root staff 1537 Apr 02 08:35 vg_dev.data # oslevel -r 5300-05 # df -k | grep tmp /dev/hd3 1310720 1309000 1% 42 1% /tmp A: I had this issue as well with VIO 1.3. I called IBM support about it and it is a known issue. The APAR is IY87935. The fix will not be released until AIX 5.3 TL 6, which is due out in June. It occurs when you run savevgstruct on a user defined volume group that contains volumes where at least one does not have a filesystem defined on it. The workaround is to define a filesystem on every volume in the user defined volume group. >> thread 2: IBM APAR Note: http://www-1.ibm.com/support/docview.wss?uid=isg1IY87935 IY87935: MKVGDATA/SAVEVG CAN FAIL APAR status Closed as program error. Error description The mkvgdata command when executed on a volume group that does not have any mounted filesystems: # savevg -f /home/vgbackup -i vg00 Creating information file for volume group vg00..cat: 0652-050 Cannot open /tmp/vgdata/vg00/fs_data_tmp. /usr/bin/savevg 33 : BACKUPSHRINKSIZE = 16 + FSSHRINKSIZE : 0403-009 The specified number is not valid for this command.
Local fix Problem summary The mkvgdata command when executed on a volume group that does not have any mounted filesystems: # savevg -f /home/vgbackup -i vg00 Creating information file for volume group vg00..cat: 0652-050 Cannot open /tmp/vgdata/vg00/fs_data_tmp. /usr/bin/savevg 33 : BACKUPSHRINKSIZE = 16 + FSSHRINKSIZE : 0403-009 The specified number is not valid for this command. Problem conclusion Check variable. Temporary fix Comments APAR information APAR number IY87935 Reported component name AIX 5.3 Reported component ID 5765G0300 Reported release 530 Status CLOSED PER PE NoPE HIPER NoHIPER Submitted date 2006-08-09 Closed date 2006-08-09 Last modified date 2006-08-09
9.11 AIX: the backup and restore commands: -----------------------------------------The backup command creates copies of your files on a backup medium, such as a magnetic tape or diskette. But you can also backup to diskspace. The copies are in one of the two backup formats: - Specific files and directories, backed up by name using the -i flag. - Entire file system backed up by i-node, not using the -i flag, but instead using the Level and FileSystem parameters. Unless you specify another backupmedia with the -f parameter, the backup command automatically writes its output to /dev/rfd0 which is the diskette drive. (1) Backing up the user directory "userdirectory": # cd /userdirectory
# find . -depth | backup -i -f /dev/rmt0 -print
# or use find .
(2) Incremental backups: You can create full and incremental backups of filesystems as well, as shown in the following example. When the -u flag is used with the backup command, the system will do an incremental backup according to the -level number specified. For example, a level 5 backup will only back up the data that has changed after the level 4 was made. Levels can range from 0 to 9. Example; On Sunday: # backup -0 -uf /dev/rmt0 /data On Monday: # backup -1 -uf /dev/rmt0 /data .. .. On Saturday: # backup -6 -uf /dev/rmt0 /data Due to the -u parameter, information about the backups is written to the /etc/dumpdates file. To backup the / (root) file system, enter: # backup -0 -u -f /dev/rmt0 / Note that we do noy use the -i flag, but instead backup an entire fs "/". Other examples: --------------To backup all the files and subdirectories in current directory using relative pathnames, use # find . -print | backup -if /dev/rmt0 To backup the files /bosinst.data and /signature to the diskette, use # ls ./bosinst.dat ./signature | backup -iqv How to restore a file: ---------------------Suppose we want to restore the /etc/host file, because its missing. # tctl -f /dev/rmt0 rewind # - rewind tape # restore -x -d -v -q -s4 -f /dev/rmt0.1 ./etc/hosts Another example: # restore -qvxf /dev/rmt0.1 "./etc/passwd" # restore -s4 -qTvf /dev/rmt0.1 mksysb tape
Restore /etc/passwd file Lists contents of a
More on checking and perform the restore command: ------------------------------------------------Check (or list) the backup can be done by using a command similar to the following example: # restore –Tqf /save/backup_ddmmyy.backup 1>/dev/null 2>&1;echo $? Because the echo commands returns $?, that would mean that 0 as a result means that the listing the backup is succesfull. If you do not use the echo, the above command just shows the complete listing of the backup contents. It does not do the actual restore, it just shows a listing. If you actually want to restore the backup, use the following command: restore –xdvqf /save/backup_ddmmyy.backup 1>/dev/null 2>&1;echo $?
9.12 AIX: savevg and restvg: ---------------------------To backup, or clone, a VG, you can use the - mksysb command for the rootvg - savevg command for other user VG's To backup a user Volume Group (VG, see also sections 30 and 31) you can use savevg to backup a VG and restvg to restore a VG. # lsvg rootvg uservg
# - shows a list of online VG's
# savevg -if /dev/rmt0 uservg
# - now backup the uservg
9.13 AIX: tctl: --------------Purpose Gives subcommands to a streaming tape device. Syntax tctl [ -f Device ] [ eof | weof | fsf | bsf | fsr | bsr | rewind | offline | rewoffl | erase | retension | reset | status ] [ Count ] tctl [ -n ] [
-b BlockSize ] [ -f Device ] [ -B ] { read | write }
-p BufferSize ] [
-v ] [
Description The tctl command gives subcommands to a streaming tape device. If you do not specify the Device variable
with the -f flag, the TAPE environment variable is used. If the environment variable does not exist, the tctl command uses the /dev/rmt0.1 device. (When the tctl command gives the status subcommand, the default device is /dev/rmt0.) The Device variable must specify a raw (not block) tape device. The Count parameter specifies the number of end-of-file markers, number of file marks, or number of records. If the Count parameter is not specified, the default count is 1. Examples To rewind the rmt1 tape device, enter: tctl -f /dev/rmt1 rewind To move forward two file marks on the default tape device, enter: tctl fsf 2 To write two end-of-file markers on the tape in /dev/rmt0.6, enter: tctl -f /dev/rmt0.6 weof 2 To read a tape device formatted in 80-byte blocks and put the result in a file, enter: tctl -b 80 read > file To read variable-length records from a tape device formatted in 80-byte blocks and put the result in a file, enter: tctl -b 80 -n read > file To write variable-length records to a tape device using a buffer size of 1024 byes, enter: cat file | tctl -b 1024 -n -f/dev/rmt1 write To write to a tape device in 512-byte blocks and use a 5120-byte buffer for standard input, enter: cat file | tctl -v -f /dev/rmt1 -p 5120 -b 512 write Note: The only valid block sizes for quarter-inch (QIC) tape drives are 0 and 512. To write over one of several backups on an 8 mm tape, position the tape at the start of the backup file and issue these commands: tctl bsf 1 tctl
eof 1
9.14 AIX mt command: -------------------Purpose Gives subcommands to streaming tape device. Syntax mt [ -f TapeName ] Subcommand [ Count ]
Description The mt command gives subcommands to a streaming tape device. If you do not specify the -f flag with the TapeName parameter, the TAPE environment variable is used. If the environment variable does not exist, the mt command uses the /dev/rmt0.1 device. The TapeName parameter must be a raw (not block) tape device. You can specify more than one operation with the Count parameter. Subcommands eof, weof Writes the number of end-of-file markers specified by the Count parameter at the current position on the tape. fsf Moves the tape forward the number of files specified by the Count parameter and positions it to the beginning of the next file. bsf Moves the tape backwards the number of files specified by the Count parameter and positions it to the beginning of the last file skipped. If using the bsf subcommand would cause the tape head to move back past the beginning of the tape, then the tape will be rewound, and the mt command will return EIO. fsr Moves the tape forward the number of records specified by the Count parameter. bsr Moves the tape backwards the number of records specified by the Count parameter. rewoff1, rewind Rewinds the tape. The Count parameter is ignored. status Prints status information about the specified tape device. The output of the status command may change in future implementations Examples To rewind the rmt1 tape device, enter: mt -f /dev/rmt1 rewind To move forward two files on the default tape device, enter: mt fsf 2 To write two end-of-file markers on the tape in the /dev/rmt0.6 file, enter: mt -f /dev/rmt0.6 weof 2 9.14 AIX tapeutil command: -------------------------tapeutil -f - A program which came with the tape library to control it's working. Called without arguments gives a menu. Is useful for doing things like moving tapes from the slot to the drive. e.g. $ tapeutil -f /dev/smc0 move -s 10 -d 23
which moves the tape in slot 10 to the drive (obviously, this will depend on your own individual tape library, may I suggest the manual?). The fileset you need to install for 'tapeutil' command is: Atape.driver 7.1.5.0. Example: -------We are using 3583 automated tape library for backups.for tapeutil command u need to have a file atape.sys on ur system.to identify the positioning of tape drives and source just type tapeutil it will give u a number of options.choose element information to identify the source and tape drive numbers. In our case the tape drives numbers are 256 and 257 and the source number to insert the tape is 16. we usually give the following commands to load and move the tape. Loading Tape:tapeutil -f /dev/smc0 move -s 16 -d 256 (to insert the tape in tapedrive 1,where 16 is source and 256 is destination) to take the backup:find filesystem1 filesystem2 | backup -iqvf /dev/rmt1 ((filessystem name without mount point slash)) after taking the backup and unloading tape:tapeutil -f /dev/rmt1 unload tapeutil -f /dev/smc0 move -s 256 -d 16 (first unload the tape then move it to source destination) this might help u to use the taputil command in taking backup. Example: -------In order to move tapes in and out of the Library here is what I do. First I unload the tape with the command #tapeutil -f /dev/rmtx unload Where x is 0,1,2,3... then I move the tape from external slot (16) using the media changer, not the tape drive. #tapeutil -f /dev/smcx move 256 16 The above command moves the tape in your first tape drive (256) to the external slot. Note that you can also move from the internal slots to the external slot or the tape drive.
To move the tape back from the external slot, I just switch 256 and 16 parameters. Example: -------The code I use to list the I/O station slots is: /usr/bin/tapeutil -f /dev/smc0 inventory | grep -p Station | egrep 'Station|Volume' | awk '{ if($1 =3D=3D "Import/Export") ioslot=3D$4; if($1 =3D=3D "Volume") { if(NF =3D=3D 4) volser=3D$4; else volser=3D"-open-"; print ioslot, volser; }}' The tapeutil command to move a tape is: /usr/bin/tapeutil -f /dev/smc0 move For example:
/usr/bin/tapeutil -f /dev/smc0 move 773 1037
You can get the slot numbers, and volsers in them, with the command: /usr/bin/tapeutil -f /dev/smc0 inventory To find an open slot just look for a slot with a blank "Volume Tag". One little hitch, however. If a tape is currently mounted, the "tapeut=il inventory" command will show a slot as open ("Volume Tag" is blank), but TSM will have it reserved for= the mounted tape. So what I did in my script is to check the TSM device configuration file for each ope= n slot that I find and if that slot number appears in it then I skip that slot and go on to the next one. Example: -------#!/bin/ksh DEVICE=$1 HOST=$2 TAPE=$3 case $TAPE in 2) tapeutil -f tapeutil ;; 3) tapeutil -f tapeutil ;; 4) tapeutil -f tapeutil ;;
/dev/smc0 move 23 10 -f /dev/smc0 move 11 23 /dev/smc0 move 23 11 -f /dev/smc0 move 12 23 /dev/smc0 move 23 12 -f /dev/smc0 move 13 23
5) tapeutil -f /dev/smc0 move 23 13 tapeutil -f /dev/smc0 move 14 23 ;; esac Example: -------tapeutil tapeutil tapeutil tapeutil tapeutil
-f -f -f -f -f
/dev/rmt1 /dev/smc0 /dev/smc0 /dev/smc0 /dev/smc0
unload move 257 16 move -s 256 -d 16 move 257 1025 move 16 257
tapeutil -f /dev/smc0 exchange 34 16 40 tapeutil -f /dev/smc0 inventory | more tctl -f/dev/rmt0 rewoffl tapeutil -f/dev/smc0 elementinfo tapeutil -f /dev/scm0 inventory
Example: -------tapeutil -f /dev/rmt1 unload sleep 20 DAYNO=`date +%d`;export DAYNO case $DAYNO in 01) tapeutil -f /dev/smc0 move 23 10 tapeutil -f /dev/smc0 move 11 23 ;; 02) tapeutil -f /dev/smc0 move 23 10 tapeutil -f /dev/smc0 move 11 23 ;; 03) tapeutil -f /dev/smc0 move 23 10 tapeutil -f /dev/smc0 move 11 23 ;; 04) tapeutil -f /dev/smc0 move 23 10 tapeutil -f /dev/smc0 move 11 23 ;; 05) tapeutil -f /dev/smc0 move 23 10 tapeutil -f /dev/smc0 move 11 23 ;; 06) tapeutil -f /dev/smc0 move 23 10 tapeutil -f /dev/smc0 move 11 23 ;; 07) tapeutil -f /dev/smc0 move 23 10 tapeutil -f /dev/smc0 move 11 23 ;; esac Example: --------
tapeutil -f /dev/rmt1 unload sleep 20 DAYNAME=`date +%a`;export DAYNAME case $DAYNAME in Sun) tapeutil -f /dev/smc0 tapeutil -f /dev/smc0 move ;; Mon) tapeutil -f /dev/smc0 tapeutil -f /dev/smc0 move ;; Tue) tapeutil -f /dev/smc0 tapeutil -f /dev/smc0 move ;; Wed) tapeutil -f /dev/smc0 tapeutil -f /dev/smc0 move ;; Thu) tapeutil -f /dev/smc0 tapeutil -f /dev/smc0 move ;; Fri) tapeutil -f /dev/smc0 tapeutil -f /dev/smc0 move ;; Sat) tapeutil -f /dev/smc0 tapeutil -f /dev/smc0 move ;; esac
move 256 4098 4099 256 move 256 4099 4100 256 move 256 4100 4113 256 move 256 4113 4114 256 move 256 4114 4109 256 move 256 4109 4124 256 move 256 4124 4110 256
tapeutil -f /dev/smc0 move 256 4098 tapeutil -f /dev/smc0 move 4099 256 Example: --------
tapeutil sleep 10 tapeutil sleep 10 tapeutil sleep 10 tapeutil sleep 10 tapeutil sleep 10 tapeutil sleep 10
-f /dev/smc0 move 16 4096 -f /dev/smc0 move 17 4097 -f /dev/smc0 move 18 4098 -f /dev/smc0 move 19 4099 -f /dev/smc0 move 20 4100 -f /dev/smc0 move 21 4101
Example: -------mt -f /dev/rmt1 rewind mt -f /dev/rmt1.1 fsf 6 tar -xvf /dev/rmt1.1 /data/download/expdemo.zip SPL bld
About Ts3310: ------------Abstract Configuration Information for IBM TS3310 (IBM TotalStorage 3576) Content IBM TS3310 (IBM TotalStorage 3576) Drive Addresses Storage Slot Addresses Changer Address Entry/Exit Slot Address 256-261 4096-4223 1 16-21 Notes: 1. Barcodes are required. Without a barcode label, a volume will show as unknown media. 2. ELEMent=AUTODetect in the DEFINE/UPDATE DRIVE command is supported. 3. Device identification and firmware used during validation Library ID: IBM 3576-MTL --- Firmware: 0.62 4. The IBM device driver is required. The IBM device drivers are available at ftp://ftp.software.ibm.com/storage/devdrvr. 5. The library is available with IBM LTO Generation 3 drives. 6. For more information on IBM TS3310, see TS3310 Tape Library.
Example: -------First, list the tape device names: lsdev -Cc tape Assume it returns smc0 for the library, and rmt0 and rmt1 for the tape drives, and all devices are Available. Next, take an inventory of the library. tapeutil -f /dev/smc0 inventory | more Assume the inventory returns two drives with element numbers 256 and 257 and shows a tape stored in slot 1025. Then, start moving the tape to each drive in turn, and verify which device name it is associated with by running tctl or mt rewoffl. If it returns without error, the device name matches the element number. Move the tape from the tape slot to the first drive: tapeutil -f /dev/smc0 move 1025 256 tctl -f/dev/rmt0 rewoffl
If the command returns with no errors, then element # 256 matches device name /dev/rmt0. Move the tape to the next drive tapeutil -f /dev/smc0 move 256 257 tctl -f/dev/rmt1 rewoffl If the command returns with no errors, then element # 257 matches device name /dev/rmt1 Move the tape back to the storage slot it came from: tapeutil -f /dev/smc0 move 257 1025 If at any point, the tctl command returns with errors, then try another device name until it returns without errors. NOTE: the 'rewoffl' flag on tctl simply rewinds and ejects the tape from the drive. 9.15 Recover from AIX OS failure: --------------------------------Recover from OS failure. Contents: 1. How to view the bootlist: 2. How to change the bootlist: 3. How to make a device bootable: 4. How to make a backup of the OS: 5. Shutdown a pSeries AIX system in the most secure way: 6. How to restore specific files from a mksysb tape: 7. Recovery of rootvg 1. How to view the bootlist: At boottime, once the POST is completed, the system will search the boot list for a bootable image. The system will attempt to boot from the first entry in the bootlist. Its always a good idea to see what the OS thinks are the bootable devices and the order of what the OS thinks it should use. Use the bootlist command to view the order: # bootlist -m normal -o
As the first item returned, you will see hdisk0, the bootable harddisk. If you need to check the bootlist in "service mode", for example if you want to boot from tape to restore the rootvg, use # bootlist -m service -o
2. How to change the bootlist: The bootlist, in normal operations, can be changed using the same command as used in section 1, for example # bootlist -m normal hdisk0 cd0 This command makes sure the hdisk0 is the first device used to boot the system. If you want to change the bootlist for the system in service mode, you can change the list in order to use rmt0 if you need to restore the rootvg. # bootlist -m service rmt0
3. How to make a device bootable: To make a device bootable, use the bosboot command: # bosboot -ad /dev/ipldevice So, if hdisk0 must be bootable, or you want to be sure its bootable, use # bosboot -ad /dev/hdisk0
4. How to make a backup of the OS: The mksysb command creates an installable image of the rootvg. This is synonym to say that mksysb creates
a backup of the operating system (that is, the root volume group). You can use this backup to reinstall a system to its original state after it has been corrupted. If you create the backup on tape, the tape is bootable and includes the installation programs needed to install from the backup. To generate a system backup and create an /image.data file (generated by the mkszfile command) to a tape device named /dev/rmt0, type: # mksysb -i /dev/rmt0 If a backup tape was created with the -e switch, like in: # mksysb -i -e /dev/rmt0 then a number of directories are NOT included in the backup. These exclusions are listed in the "/etc/exclude.rootvg" file. The mksysb command should be used regularly. It must certainly be done after installing apps or devices. In normal conditions, the OS does not change, and a bootable tape should be created at some frequency.
5. Shutdown a pSeries AIX system in the most secure way: 1. Shut down all applications in a controlled way. 2. Make sure no users are on the system. 3. Use the shutdown command: shutdown -r
to reboot the system
shutdown -m
to reboot in maintenance mode
6. How to restore specific files from a mksysb tape: $ tctl fsf 3
$ restore -xvf /dev/rmt0.1 ./your/file/name For example, if you need to get the vi command back, put the mksysb tape in the tape drive (in this case, /dev/rmt0) and do the following: cd /
# get to the root directory
tctl -f /dev/rmt0 rewind
# rewind the tape
tctl -f /dev/rmt0.1 fsf 3 # move the tape to the third file, no rewind restore -xqf /dev/rmt0.1 -s 1 ./usr/bin/vi # extract the vi binary, no rewind Further explanation why you must use the fsf 3 (fast forward skip file 3): The format of the tape is as follows: 1. A BOS boot image 2. A BOS install image 3. A dummy Table Of Contents 4. The system backup of the rootvg So if you just need to restore some files, first forward the tape pointer to position 3, counting from 0.
7. Recovery of rootvg 7.1 Check if the system can boot from tape: # bootinfo -e If a 1 is returned, the system can boot from tape, if a 0 is returned a boot from tape is not supported. 7.2 Recover the rootvg: One possible method is the following: 1. Check whether the tape is in front of the disk with the bootlist command: # bootlist -m normal -o 2. Insert the mksysb tape 3. Power on the machine. The system will boot from the tape. 4. The Installation and Maintenance Menu will be displayed.
Welcome to Base Operating System Installation and Maintenance
Type the number of your choice and press Enter. >>>.
Choice is indicated by
>>> 1 Start Install Now with Default Settings 2 Change/Show Installation Settings and Install 3 Start Maintenance Mode for System Recovery Type 3 and press enter to start maintenance mode. The next screen you should see is :Maintenance Type the number of your choice and press Enter. >>> 1 2 3 4
Access a Root Volume Group Copy a System Dump to Removable Media Access Advanced Maintenance Functions Install from a System Backup
>>> Choice [1]: Type 4 and press enter to install from a system backup. The next screen you should see is :Choose Tape Drive Type the number of the tape drive containing the system backup to be installed and press Enter. Tape Drive >>> 1 tape/scsi/ost
Path Name /dev/rmt0
>>> Choice [1]: Type the number that corresponds to the tape drive that the mysysb tape is in and press enter. The next screen you should see is :Welcome to Base Operating System Installation and Maintenance Type the number of your choice and press Enter. >>>.
Choice is indicated by
>>> 1 Start Install Now with Default Settings 2 Change/Show Installation Settings and Install 3 Start Maintenance Mode for System Recovery +----------------------------------------------------
88 Help ? /dev/rmt0 99 Previous Menu
|Select 1 or 2 to install from tape device | |
>>> Choice [1]: You can now follow your normal mksysb restore procedures. 9.16 HP-UX make_net_recovery: ----------------------------There are two ways you can recover from a tape with make_net_recovery. The method you choose depends on your needs. - Use make_medialif This method is useful when you want to create a totally self-contained recovery tape. The tape will be bootable and will contain everything needed to recover your system, including the archive of your system. During recovery, no access to an Ignite-UX server is needed. Using make_medialif is described beginning on "Create a Bootable Archive Tape via the Network" and also on the Ignite-UX server in the file: /opt/ignite/share/doc/makenetrec.txt - Use make_boot_tape This method is useful when you do not have the ability to boot the target machine via the network, but are still able to access the Ignite-UX server via the network for your archive and configuration data. This could happen if your machine does not support network boot or if the target machine is not on the same subnet as the Ignite-UX server. In these cases, use make_boot_tape to create a bootable tape with just enough information to boot and connect with the Ignite-UX server. The configuration files and archive are then retrieved from the Ignite-UX server. See the make_boot_tape(1M) manpage for details. -- make_boot_tape: make_boot_tape(1M) make_boot_tape(1M) NAME
make_boot_tape - make a bootable tape to connect to an Ignite-UX server
SYNOPSIS /opt/ignite/bin/make_boot_tape [-d device-file-for-tape] [-f configfile] [-t tmpdir] [-v] /opt/ignite/bin/make_boot_tape [-d device-file-for-tape] [-g gateway]
[-m netmask] [-t tmpdir] [-v] DESCRIPTION The tape created by make_boot_tape is a bootable tape that contains just enough information to boot the system and then connect to the Ignite-UX server where the tape was created. Once the target system has connected with the Ignite-UX server, it can be installed or recovered using Ignite-UX. The tape is not a fully self-contained install tape; an Ignite-UX server must also be present. The configuration information and software to be installed on the target machine reside on the Ignite-UX server, not on the tape. If you need to build a fully self-contained recovery tape, see make_recovery(1m) or make_media_lif(1m). make_boot_tape is used in situations when you have target machines that cannot boot via the network from the Ignite-UX server. This happens either because the machine does not support booting from the
network or because it is not on the same subnet as the Ignite-UX server. In this case, booting from a tape generated by make_boot_tape means you do not need to set up a boot helper system. A tape created by make_boot_tape can be used to kick off a normal Ignite-UX installation. It can also be used to recover from recovery configurations saved on the Ignite-UX server. There is no "target-specific" information on the boot tape. Only information about the Ignite-UX server is placed on the tape. Thus,
it is possible to initiate an installation of any target machine
from
the same boot tape provided that the same Ignite-UX server is
used.
Likewise, the target machine can be installed with any operating system configuration that is available on the Ignite-UX server. Typically, the make_boot_tape command is run from the Ignite-UX
server
that you wish to connect with when booting from the tape later on. A key file that contains configuration information is called INSTALLFS. This file exists on the Ignite-UX server at /opt/ignite/boot/INSTALLFS and is also present on the tape created by make_boot_tape. See instl_adm(4) for details on the configuration file syntax. Unless the -f option is used, the configuration information already present in the INSTALLFS file is used on the tape as well. The make_boot_tape command will never alter the INSTALLFS file on the
the
Ignite-UX server; it will only change the copy that is placed on tape.
Examples: --------Create a boot tape on the default tape drive (/dev/rmt/0m). # make_boot_tape Create a boot tape on a specified (non-default) tape drive. Create a DDS1 device file for the tape drive first. Show as much information about the tape creation as is possible. ioscan -fC tape # to get the hardware path mksf -v -H -b DDS1 -n -a make_boot_tape -d /dev/ -v Create a boot tape and replace the configuration information contained in the INSTALLFS file. Use the /tmp directory for all temporary files instead of the default /var/tmp. # instl_adm -d > tmp_config_file ## edit tmp_config_file as appropriate # make_boot_tape -f tmp_config_file -t /tmp Create a boot tape and specify a different gateway IP address. Set from
the netmask value as well. All other configuration information is what is already in /opt/ignite/boot/INSTALLFS. # make_boot_tape -g 15.23.34.123 -m 255.255.248.0
9.17 /etc/dumpdates ------------------On some unixes the /etc/dumpdates file exists, for example, Solaris. Purpose of the /etc/dumpdates File The ufsdump command, when used with the -u option, maintains and updates the /etc/dumpdates file. Each line in the /etc/dumpdates file shows the following information: The file system backed up The dump level of the last backup The day, date, and time of the backup For example:
# cat /etc/dumpdates /dev/rdsk/c0t0d0s0 /dev/rdsk/c0t0d0s7 /dev/rdsk/c0t0d0s7
0 Wed Jul 28 16:13:52 2004 0 Thu Jul 29 10:36:13 2004 9 Thu Jul 29 10:37:12 2004
When you do an incremental backup, the ufsdump command checks the /etc/dumpdates file to find the date of the most recent backup of the next lower dump level. Then, this command copies to the media all files that were modified since the date of that lower-level backup. After the backup is complete, a new information line, which describes the backup you just completed, replaces the information line for the previous backup at that level. Use the /etc/dumpdates file to verify that backups are being done. This verification is particularly important if you are having equipment problems. If a backup cannot be completed because of equipment failure, the backup is not recorded in the /etc/dumpdates file. If you need to restore an entire disk, check the /etc/dumpdates file for a list of the most recent dates and levels of backups so that you can determine which tapes you need to restore the entire file system. 9.18 UFS snapshot on Solaris ---------------------------UFS Snapshots Overview The Solaris release includes the fssnap command for backing up file systems while the file system is mounted. You can use the fssnap command to create a read-only snapshot of a file system. A snapshot is a file system's temporary image that is intended for backup operations. When the fssnap command is run, it creates a virtual device and a backing-store file. You can back up the virtual device, which looks and acts like a real device, with any of the existing Solaris backup commands. The backing-store file is a bitmap file that contains copies of presnapshot data that has been modified since the snapshot was taken. Why Use UFS Snapshots? The UFS snapshots feature enables you to keep the file system mounted and the system in multiuser mode during backups. Previously, you were advised to bring the system to single-user mode to keep the file system inactive when you used the ufsdump command to perform backups. You can also use additional Solaris backup commands, such as tar and cpio, to back up a UFS snapshot for more reliable backups. The fssnap command gives administrators of nonenterprise-level systems the power of enterprise-level tools, such as Sun StorEdgeT Instant Image, without the large storage demands.
The UFS snapshots feature is similar to the Instant Image product. Although UFS snapshots can make copies of large file systems, Instant Image is better suited for enterprise-level systems. UFS snapshots is better suited for smaller systems. Instant Image allocates space equal to the size of the entire file system that is being captured. However, the backing-store file that is created by UFS snapshots occupies only as much disk space as needed. Example of how to use it: # fssnap -F ufs -o bs=/backing-store-file /file-system Obviously, the backing-store file must reside on a different file system than the file system that is being captured using UFS snapshots. The following example shows how to create a snapshot of the /usr file system. The backing-store file is /scratch/usr.back.file. The virtual device is /dev/fssnap/1. # fssnap -F ufs -o bs=/scratch/usr.back.file /usr /dev/fssnap/1 You can display the current snapshots on the system by using the fssnap -i option. If you specify a file system, you see detailed information about that snapshot. If you don't specify a file system, you see information about all of the current UFS snapshots and their corresponding virtual devices. List all current snapshots: For example: # /usr/lib/fs/ufs/fssnap -i Snapshot number Block Device Raw Device Mount point Device state Backing store path Backing store size Maximum backing store size Snapshot create time Copy-on-write granularity Snapshot number Block Device Raw Device Mount point Device state Backing store path Backing store size Maximum backing store size Snapshot create time Copy-on-write granularity
: : : : : : : : : : : : : : : : : : : :
0 /dev/fssnap/0 /dev/rfssnap/0 /usr idle /var/tmp/snapshot3 256 KB Unlimited Wed Oct 08 10:38:25 2003 32 KB 1 /dev/fssnap/1 /dev/rfssnap/1 / idle /tmp/bs.home 448 KB Unlimited Wed Oct 08 10:39:29 2003 32 KB
19.19 Recovery of the root filesystem on Solaris: ================================================= Note 1: -----Restoring the root (/) File System -- To restore the / (root) file system, boot from the Solaris CD-ROM and then run ufsrestore. If / (root), /usr, or the /var file system is unusable because of some type of corruption the system will not boot. The following procedure demonstrates how to restore the / (root) file system which is assumed to be on boot disk c0t0d0s0. 1. Insert the Solaris 8 Software CD 1, and boot the CD-ROM with the single-user mode option. ok boot cdrom -s 2. Create the new file system structure. # newfs /dev/rdsk/c0t0d0s0 3. Mount the file system to an empty mount point directory, /a and change to that directory. # mount /dev/dsk/c0t0d0s0 /a # cd /a 4. Restore the / (root) file system from its backup tape. # ufsrestore rf /dev/rmt/0 Note - Remember to always restore a file system starting with the level 0 backup tape and continuing with the next lowest level tape up through the highest level tape. 5. Remove the restoresymtable file. # rm restoresymtable 6. Install the bootblk in sectors 1-15 of the boot disk. Change to the directory containing the bootblk, and run the installboot command. # cd /usr/platform/`uname -m`/lib/fs/ufs # installboot bootblk /dev/rdsk/c0t0d0s0 7. Unmount the new file system.
# cd / # umount /a 8. Use the fsck command to check the restored file system. # fsck /dev/rdsk/c0t0d0s0 9. Reboot the system. # init 6 10. Perform a full backup of the file system. For example: # ufsdump 0uf /dev/rmt/0 /dev/rdsk/c0t0d0s0 Note - Always back up the newly created file system, as ufsrestore repositions the files and changes the inode allocation. Restoring the /usr and /var File Systems -- To restore the /usr and /var file systems repeat the steps described above, except step 6. This step is required only when restoring the (/) root file system. To restore a regular file system, (for example, /export/home, or /opt) back to disk, repeat the steps described above, except steps 1, 6, and 9. Example # # # # # # # # #
newfs /dev/rdsk/c#t#d#s# mount /dev/dsk/c#t#d#s# /mnt cd /mnt ufsrestore rf /dev/rmt/# rm restoresymtable cd / umount /mnt fsck /dev/rdsk/c#t#d#s# ufsdump 0uf /dev/rmt/# /dev/rdsk/c#t#d#s#
Note 2: -------
============= 10. uuencode: ============= Unix to Unix Encoding. A method for converting files from Binary to ASCII so that they can be sent across the Internet via e-mail. Encode binary file (to uuencoded ASCII file)
uuencode file remotefile uudecode file Example: Encode binary file uuencode example example.en Decode encoded file uudecode example.en
uuencode converts a binary file into an encoded representation that can be sent using mail(1) . It encodes the contents of source-file, or the standard input if no source-file argument is given. The decode_pathname argument is required. The decode_pathname is included in the encoded file's header as the name of the file into which uudecode is to place the binary (decoded) data. uuencode also includes the permission modes of source-file, (except setuid , setgid, and sticky-bits), so that decode_pathname is recreated with those same permission modes. example: The following example packages up a source tree, compresses it, uuencodes it and mails it to a user on another system. When uudecode is run on the target system, the file ``src_tree.tar.Z'' will be created which may then be uncompressed and extracted into the original tree. # tar cf - src_tree | compress | uuencode src_tree.tar.Z | mail sys1! sys2!user example: uuencode > | | note: here, file_a is encoded and a new file named uufile is produced | | when you decode file uufile a file named file_b is produced | # uuencode dipl.doc dipl.doc >dipl.uu Hier wird die Datei dipl.doc (z.B. ein WinWord-Dokument) in die Datei dipl.uu umgewandelt. Dabei legen wir fest, dasz die Datei nach dem Decodieren wieder dipl.doc heiszen soll. example: uuencode long_name.tar.Z arc.trz > arc.uue 11. grep command: ================= # grep Sally people # grep "Sally Smith" people
# grep -v "^$" people.old > people # grep -v "^ *$" people.old > people # grep "S.* D.*" people.old > people
# deletes all blank lines
12. sort command: ================= sort files by size, largest first... # ls -al | sort +4 -r | more # # # # # #
sort sort sort sort sort sort
+1 -2 people +2b people +2n +1 people +1 -2 *people > everybody -u +1 hardpeople softpeople > everybody -t: +5 /etc/passw
# -u=unique # -t field sep.
cp /etc/hosts /etc/hosts.`date +%o%b%d` 13. SED: ======== Can be used to replace a character sting with a different string. # sed s/string/newstring file #sed s/Smith/White/ people.old > people #sed "s/Sally Smith/Sally White/" people.old > people Note: depending on your shell and system, in most cases, you might need to enclose s/string/newstring by a " or a '. you can also use a regular expression, for instance we can put a left margin of 5 spaces on the people file # sed "s/^/ /" people.old > people # sed "s/[0-9]*$//" people.old > people # sed -e "s/^V^M//" filename > outputfilename
(remove numbers)
The character after the s is the delimiter. It is conventionally a slash, because this is what ed, more, and vi use. It can be anything you want, however. If you want to change a pathname that contains a slash - say /usr/local/bin to /common/bin you could use the backslash to quote the slash: sed 's/\/usr\/local\/bin/\/common\/bin/' new or use _ as a delimter sed 's_/usr/local/bin_/common/bin_' new Example:
-------Suppose the file cdc_LEG.sql contains the following: spool Publisher.06.PublisherDefineChangeTable.tdba_cdc.cdc_LEG.log ; connect / as sysdba ; grant all on rm_live.LEG to tdba_cdc ; prompt User: tdba_cdc ; connect tdba_cdc ; begin dbms_cdc_publish.create_change_table ( owner => 'tdba_cdc' , change_table_name => 'cdc_LEG' , change_set_name => 'BODI_CDC_SET' , source_schema => 'rm_live' , source_table => 'LEG' , column_type_list => ' IDFLT NUMBER(9) , IDLEG NUMBER(9) , LEGDATE DATE , IDLEGDATA NUMBER(9) , CANCELLED CHAR(1) , IDWORKSET NUMBER(9) , IDTEXTTTS NUMBER(9) , IDSEGMENTDATACOMBINE NUMBER(9) ' , capture_values => 'both' , source_colmap => 'y' , target_colmap => 'y' , options_string => 'tablespace tdba_cdc' ) ; end ; / grant select on tdba_cdc.cdc_LEG to bodi_cdc ; Now we want to replace the "connect tdba_cdc" by "connect tdba_cdc/tdba_cdc" Try: #sed 's!connect tdba_cdc!connect tdba_cdc/tdba_cdc!' cdc_LEG.sql > cdc_LEG.txt #sed 's/playroca/accproca!' cdc_LEG.sql > cdc_LEG.txt gives: spool Publisher.06.PublisherDefineChangeTable.tdba_cdc.cdc_LEG.log ; connect / as sysdba ; grant all on rm_live.LEG to tdba_cdc ; prompt User: tdba_cdc ; connect tdba_cdc/tdba_cdc ; begin dbms_cdc_publish.create_change_table
( owner => 'tdba_cdc' , change_table_name => 'cdc_LEG' , change_set_name => 'BODI_CDC_SET' , source_schema => 'rm_live' , source_table => 'LEG' , column_type_list => ' IDFLT NUMBER(9) , IDLEG NUMBER(9) , LEGDATE DATE , IDLEGDATA NUMBER(9) , CANCELLED CHAR(1) , IDWORKSET NUMBER(9) , IDTEXTTTS NUMBER(9) , IDSEGMENTDATACOMBINE NUMBER(9) ' , capture_values => 'both' , source_colmap => 'y' , target_colmap => 'y' , options_string => 'tablespace tdba_cdc' ) ; end ; / grant select on tdba_cdc.cdc_LEG to bodi_cdc If you have a lot of those files, use something like for file in `ls` do sed 's!connect tdba_cdc!connect tdba_cdc/tdba_cdc!' $file > $file.sql done for file in `ls` do echo $file done for file in `ls` do echo "connect / as sysdba;" >> $file done for file in `ls` do sed 's!quit!;!' $file > $file.sql done Other example: -------------If you want sed to remove a space at either side of a field, like Albert van der Sel , Antapex.org, 5 , 20 you could use: sed 's/[ ]*,[ ]*/,/g' or sed -e 's/[ ]*,[ ]*/,/g' -e 's/^[ ]*//' -e 's/[ ]*$//' file1 > file2 sed -e 's#\(00/00/0000\)[, ][, ]*$#\1,,,,,,,,,,,,,,,,,#g' file
Most common error: Message sed: 0602-404 Function __ cannot be parsed. If you were trying to use the sed "substitute" command, e.g. s/a/b/, you may have forgotton the trailing delimiter.
14. AWK: ======== When lines containing `foo' are found, they are printed, because `print $0' means print the current line: # awk '/foo/ { print $0 }' BBS-list looks for all files in the ls listing that matches Nov and it prints the total of bytes: # ls -l | awk '$5 == "Nov" { sum += $4 } END { print sum }' only print the lines containing Smith from file people: # awk /Smith/ people # # # #
awk awk awk awk
'/gold/' coins.txt '/gold/ {print $0}' coins.txt '/gold/ {print $5,$6,$7,$8}' coins.txt '{if ($3 < 1980) print $3, " ",$5,$6,$7,$8}' coins.txt
# awk '/Smith/ {print $1 "-" $3}' people # ls -l /home | awk '{total += $5}; END {print total}' # ls -lR /home | awk '{total += $5}; END {print total}' Example: -------Suppose you have a text file with lines much longer than, for example, 72 characters, and you want to have a file with lines with a maximum of 72 chars, then you might use awk in the following way: -- Shell file r13.sh: #!/bin/bash DIR=/cygdrive/c/exports FILE=result24.txt awk -f r13.awk ${DIR}/${FILE} > ${DIR}/${FILE}.new -- r13.awk BEGIN { maxlength=72 }
{
l=length(); if (l > 72) { i=(l/72) for (j=0; j
}
15. tr command: =============== Used for translating characters in a file. tr works on standard input, so if you want to take input from a file you have to redirect standard input so that it comes from that file. Suppose we want to replace all characters in the range a-z by the characters A-Z # tr "[a-z]" "[A-Z]" < people squeeze muliple occurences osf a character (e.g. a space) in one # tr -s " " people.old > people remove blank lines: # tr -s "\012" < people.old > people to remove the evil microsoft carriage return. # tr -d '\015' < original.file > new.file # cat filename1 | tr -d "^V^M" > newfile #! /bin/sh # # recursive dark side repair technique # eliminates spaces in file names from current directory down # useful for supporting systems where clueless vendors promote NT # for name in `find . -depth -print` do na=`echo "$name" | tr ' ' '_'` if [ "$na" != "$name" ] then echo "$name" fi done note:
> I have finally competed setting up the samba server and setup the share > between NT and Samba server. > > However, when I open a unix text file in Windows NT using notepad, i see > many funny characters and the text file is not in order (Just like when I > ftp the unix text file out into NT in binary format) ...I think this has to > be something to do with whether the file transfer is in Binary format or > ASCII ... Is there a parameter to set for this ? I have checked the > documents ... but couldn't find anything on this ... > This is a FAQ, but it brief, it's like this. Unix uses a single newline character to end a line ("\n"), while DOS/Win/NT use a carriage-return/newline pair ("\r\n"). FTP in ASCII mode translates these for you. FTP in binary mode, or other forms of file transfer, such as Samba, leave the file unaltered. Doing so would be extremely dangerous, as there's no clear way to isolate which files should be translated You can get Windows editors that understand Unix line-end conventions (Ultra Edit is one), or you can use DOS line endings on the files, which will then look odd from the Unix side. You can stop using notepad, and use Wordpad instead, which will deal appropriately with Unix line endings. You can convert a DOS format text file to Unix with this:tr -d '\r' < dosfile.txt > unixfile.txt The best solution to this seems to be using a Windows editor that can handle working with Unix line endings. HTH Mike. Note: There are two ways of moving to a new line...carriage return, which is chr(13), and new line which is chr(10). In windows you're supposed to use a sequence of a carriage return followed by a new line. For example, in VB you can use Wrap$=Chr$(13)+Chr$(10) which creates a wrap character. 16. cut and paste: ================== cutting columns:
# cut -c17, 18, 19 people # cut -c17- people > phones # cut -c1-16 people > names cutting fields: #cut -d" " -f1,2 people > names
# -d field seperator
paste: # paste -d" " firstname lastname phones > people
17. mknod: ========== mknod creates a FIFO (named pipe), character special file, or block special file with the specified name. A special file is a triple (boolean, integer, integer) stored in the filesystem. The boolean chooses between character special file and block special file. The two integers are the major and minor device number. Thus, a special file takes almost no place on disk, and is used only for communication with the operating system, not for data storage. Often special files refer to hardware devices (disk, tape, tty, printer) or to operating system services (/dev/null, /dev/random). Block special files usually are disk-like devices (where data can be accessed given a block number, and e.g. it is meaningful to have a block cache). All other devices are character special files. (Long ago the distinction was a different one: I/O to a character special file would be unbuffered, to a block special file buffered.) The mknod command is what creates files of this type. The argument following name specifies the type of file to make: p b c
for a FIFO for a block (buffered) special file for a character (unbuffered) special file
When making a block or character special file, the major and minor device numbers must be given after the file type (in decimal, or in octal with leading 0; the GNU version also allows hexadecimal with leading 0x). By default, the mode of created files is 0666 (`a/rw') minus the bits set in the umask. In /dev we find logical devices, created by the mknod command.
# mknod /dev/kbd c 11 0 # mknod /dev/sunmouse c 10 6 # mknod /dev/fb0 c 29 0 create a pipe in /dev called 'rworldlp' # mknod /dev/rworldlp p; chmod a+rw /dev/rworldlp If one cannot afford to buy extra disk space one can run the export and compress utilities simultaneously. This will prevent the need to get enough space for both the export file AND the compressed export file. Eg: # Make a pipe mknod expdat.dmp p # or mkfifo pipe # Start compress sucking on the pipe in background compress < expdat.dmp > expdat.dmp.Z & # Wait a second or two before kicking off the export sleep 5 # Start the export exp scott/tiger file=expdat.dmp Create a compressed export on the fly. # create a named pipe mknod exp.pipe p # read the pipe - output to zip file in the background gzip < exp.pipe > scott.exp.gz & # feed the pipe exp userid=scott/tiger file=exp.pipe ...
Extended Example: ----------------# Load the cron environment . ~/cronjobs/.profile.cron ################################################################## compareVersionDBMS 10.2.0.1.0 10.2.0.2.0 10.2.0.3.0 ################################################################## wantedSchemas="" wantedDatabase="" wantedInputDir="" ################################################################## if [ $# -ne 0 ] then function showSyntaxParam { ( Comment "\t[-db=] [-dir=] [schema=]" )
} for param in $* do echo ${param} \ | awk 'BEGIN {FS="="}{print $1,$2}' \ | read flag value if [ "${flag}" != "-h" ] && [ "${value}" = "" ] then Error "Empty value for ${flag}" showSyntaxSystemParam showSyntaxParam else case ${flag} in -schema) wantedSchemas=${value} ;; -dir) wantedInputDir=${value} ;; *) checkSystemParam ${param} || showSyntaxParam ;; esac fi done
fi ################################################################## BlankLine Comment "Selected options are:" Comment " Database : -db=${wantedDatabase}" Comment " Schema's : -schema=${wantedSchemas}" Comment " Input directory : -dir=${wantedInputDir}" Line ################################################################## if [ ${continue} = true ] then if [ "${wantedDatabase}" = "" ] then BlankLine Error "No database specified" BlankLine else echo ${wantedDatabase} \ | grep -i prod \ | wc -l \ | read prod if [ ${prod} -ne 0 ] then BlankLine Error "Production environment not allowed!!" BlankLine else moveLogFile ${wantedDatabase} fi fi # if [ "${wantedInputDir}" = "" ] then BlankLine Error "No input directory specified" BlankLine else if [ ! -d ${wantedInputDir} ]
then BlankLine Error "Input directory ${wantedInputDir} doesn't exists" BlankLine fi
fi # if [ "${wantedSchemas}" = "" ] then BlankLine Error "No schema's to load" BlankLine fi
fi ################################################################## wantedSchemas=`echo ${wantedSchemas} | sed 's/,/ /g'` if [ ${continue} = true ] then for schema in ${wantedSchemas} do impPipeFile=${wantedInputDir}/${schema}.${currentUser}.load.pipe impLogFile=${wantedInputDir}/${schema}.${currentUser}.load.log impCompressFile=${wantedInputDir}/${schema}.data.Z # if [ ${continue} = true ] then Message "Check file permissions" if [ ! -w ${wantedInputDir} ] then Error "Unable to write in ${wantedInputDir}" fi fi # if [ ${continue} = true ] then rm ${impPipeFile} 2> /dev/null rm ${impLogFile} 2> /dev/null fi # if [ ${continue} = true ] then Message "Load schema ${schema} into database ${wantedDatabase}" fi # if [ ${continue} = true ] then Message "Create pipe for load" CmdCapture "mknod ${impPipeFile} p" fi # if [ ${continue} = true ] then if [ ! -f ${impCompressFile} ] then BlankLine Error "File not found: ${impCompressFile}" BlankLine
fi fi # if [ ${continue} = true ] then Message "Start uncompression into background" uncompress -c < ${impCompressFile} > ${impPipeFile} & # Message "Start import" imp \"sys/change_on_install as sysdba\" file=${impPipeFile} log=$ {impLogFile} full=y statistics=always >/dev/null 2>/dev/null # Message "Output of import" CmdCapture "cat ${impLogFile}" # Message "Allowed warnings are:" Comment " IMP-00017 IMP-00041 IMP-00003 ORA-14063 ORA-14048 ORA02270" cat ${impLogFile} \ | egrep '^ORA-|^ERROR|^IMP-' \ | egrep -v 'IMP-00017|IMP-00041|IMP-00003|ORA-14063|ORA-14048| ORA-02270' \ | wc -l \ | read count if [ ${count} -ne 0 ] then Error "Problem with import !!" else Message "Import succesful" fi fi # rm ${impPipeFile} 2> /dev/null if [ ${continue} = true ] then Line fi done fi ################################################################## finish ##################################################################
18. Links: ========== A symbolic link is a pointer or an alias to another file. The command # ln -s fromfile /other/directory/tolink makes the file fromfile appear to exist at /other/directory/tolink simultaneously.
The file is not copied, it merely appears to be a part of the file tree in two places. Symbolic links can be made to both files and directories. The usage of the link command is. %ln -s ActualFilename LinkFileName Where -s indicates a symbolic link. ActualFilename is the name of the file which is to be linked to, and LinkFileName is the name by which the file should be known. You should use full paths in the command. Example: -------Suppose we have the file "mvdat" in: /opt/myprog So if we take a look there albert@starboss:/opt/myprog $ ls -al total 8 drwxr-x--drwxrwxrwx -r-xr-xr-x
2 root system 3 root system 1 albert beab_krn
256 Apr 21 10:12 . 4096 Apr 21 09:59 .. 9544 Apr 21 10:12 mvdat
Now we want a symbolic link of that file in "/apps/myapps/bin", as if the file also exists at that place. In fact we only make a link there. We can do that in the following way: albert@starboss:/opt/myprog $ ln -s mvdat /apps/myapps/bin/mvdat Other examples: --------------This example shows copying three files from a directory into the current working directory. [2]%cp ~team/IntroProgs/MoreUltimateAnswer/more* [3]%ls -l more* -rw-rw-r-1 mrblobby mrblobby 632 Sep 21 18:12 moreultimateanswer.adb -rw-rw-r-1 mrblobby mrblobby 1218 Sep 21 18:19 moreultimatepack.adb -rw-rw-r-1 mrblobby mrblobby 784 Sep 21 18:16 moreultimatepack.ads The three files take a total of 2634 bytes. The equivalent ln commands would be:
[2]%ln -s ~team/IntroProgs/MoreUltimateAnswer/moreultimateanswer.adb . [3]%ln -s ~team/IntroProgs/MoreUltimateAnswer/moreultimatepack.adb . [4]%ln -s ~team/IntroProgs/MoreUltimateAnswer/moreultimatepack.adb . [5]%ls -l lrwxrwxrwx 1 mrblobby mrblobby 35 Sep 22 08:50 moreultimateanswer.adb -> /users/team/IntroProgs/MorUltimateAnswer/moreultim ateanswer.adb lrwxrwxrwx 1 mrblobby mrblobby 37 Sep 22 08:49 moreultimatepack.adb -> /users/team/IntroProgs/MorUltimateAnswer/moreultim atepack.adb lrwxrwxrwx 1 mrblobby mrblobby 37 Sep 22 08:50 moreultimatepack.ads -> /users/team/IntroProgs/MorUltimateAnswer/moreultim atepack.ads The ln utility creates a new directory entry (linked file) which has the same modes as the original file. It is useful for maintaining multiple copies of a file in many places at once without using up storage for the copies; instead, a link ``points'' to the original copy. There are two types of links; hard links and symbolic links. How a link points to a file is one of the differences between a hard and symbolic link. By default, ln makes ``hard'' links. A hard link to a file is indistinguishable from the original directory entry; any changes to a file are effectively independent of the name used to reference the file. Hard links may not normally refer to directories and may not span file systems. A symbolic link contains the name of the file to which it is linked. The referenced file is used when an open(2) operation is performed on the link. A stat(2) on a symbolic link will return the linked-to file; an lstat(2) must be done to obtain information about the link. The readlink(2) call may be used to read the contents of a symbolic link. Symbolic links may span file systems, refer to directories, and refer to non-existent files.
19. Relink van Oracle: ====================== info: showrev -p pkginfo -i relink: mk -f $ORACLE_HOME/rdbms/lib/ins_rdbms.mk install mk -f $ORACLE_HOME/svrmgr/lib/ins_svrmgr.mk install mk -f $ORACLE_HOME/network/lib/ins_network.mk install
20. trace: ==========
20.1 truss: ----------A quick one: The "truss" tool on many unixes: Here is a quick one to trace a shell script, or executable program: using "truss". The "truss" tool is available on many unix platforms. It has many options, but a very usefull command to trace the system calls that a script or program does is: $ truss -o /tmp/myprg.log myprg In this example, truss will log in the file "/tmp/myprg.log" while it traces the program "myprg". Ofcourse, you can choose another path and logfile to trace to. The upper command is quite good for tracing a shell script, or program, that starts up, does some work, and then terminates. If an error occurs during runtime, it's likely that you find some pointers in the logfile that truss made for you. This tool has so many options, for example, you can focus your trace on a certain library etc.. Anyway, even the upper example of truss can already be very helpfull. So, for example, if you find in the log that truss has produced, the error "EACCES" which is "errno 13 = Permission denied", that would really be helpfull. Obviously, your shell script or
program tries to access a certain object, to which it has insufficient permisions, and thus may fail. Be warned though, that some errno's might be found multiple times, while it's actually not something to worry about. For example "ENOENT= No such file or directory" might be found quite often. Here, your script or program seems to be unable to find a file or directory. Well, if it's related to the $PATH environment variable, it could be quite reasonable. Your shell will search your $PATH from beginning, to the end, until the object has been found. Thus, it's quite possible that some ENOENT errors occurred.
20.2 truss on Solaris: ---------------------truss -aef -o /tmp/trace svrmgrl To trace what a Unix process is doing enter: truss -rall -wall -p truss -p $ lsnrctl dbsnmp_start NOTE: The "truss" command works on SUN and Sequent. Use "tusc" on HPUX, "strace" on Linux, "trace" on SCO Unix or call your system administrator to find the equivalent command on your system. Monitor your Unix system: Solaris: Truss is used to trace the system/library calls (not user calls) and signals made/received by a new or existing process. It sends the output to stderr. NOTE: Trussing a process throttles that process to your display speed. Use -wall and -rall sparingly. Truss usage truss truss -a -e -f
-a -a
-e -e
-f -f
-rall -rall
-wall -wall
-p
Show arguments passed to the exec system calls Show environment variables passed to the exec system calls Show forked processes (they will have a different pid: in column 1) -rall Show all read data (default is 32 bytes) -wall Show all written data (default is 32 bytes) -p Hook to an existing process (must be owner or root) Specify a program to run
Truss examples # truss -rall -wall -f -p # truss -rall -wall lsnrctl start # truss -aef lsnrctl dbsnmp_start 20.2 syscalls command on AIX: ----------------------------1. syscalls Command Purpose Provides system call tracing and counting for specific processes and the system. Syntax To Create or Destroy Buffer: syscalls [ [ -enable bytes ]| -disable
]
To Print System Call Counts: syscalls -c To Print System Call Events or Start Tracing: syscalls [ -o filename ] [ -t ] { [ [ -p pid ] -start | -stop program }
] | -x
Description The syscalls (system call tracing) command, captures system call entry and exit events by individual processes or all processes on the system. The syscalls command can also maintain counts for all system calls made over long periods of time. Notes: System call events are logged in a shared-memory trace buffer. The same shared memory identifier may be used by other processes resulting in a collision. In such circumstances, the -enable flag needs to be issued. The syscalls command does not use the trace daemon. The system crashes if ipcrm -M sharedmemid is run after syscalls has been run. Run stem -shmkill instead of running ipcrm -M to remove the shared memory segment. Flags -c Prints a summary of system call counts for all processes. The counters are not reset. -disable Destroys the system call buffer and disables system call tracing and counting. -enable bytes Creates the system call trace buffer. If this flag is not used, the syscalls command creates a buffer of the default size of 819,200 bytes. Use this flag if events are not being logged in the buffer. This is the result of a collision with another process using the same shared memory buffer ID.
-o filename
Prints output to filename rather than standard out.
-p pid When used with the -start flag, only events for processes with this pid will be logged in the syscalls buffer. When used with the -stop option, syscalls filters the data in the buffer and only prints output for this pid. -start Resets the trace buffer pointer. This option enables the buffer if it does not exist and resets the counters to zero. -stop Stops the logging of system call events and prints the contents of the buffer. -t Prints the time associated with each system call event alongside the event. -x program Runs program while logging events for only that process. The buffer is enabled if needed. Security Access Control: You must be root or a member of the perf group to run this command. Examples To collect system calls for a particular program, enter: syscalls -x /bin/ps Output similar to the following appears: PID TTY TIME CMD 19841 pts/4 0:01 /bin/ksh 23715 pts/4 0:00 syscalls -x /bin/ps 30720 pts/4 0:00 /bin/ps 34972 pts/4 0:01 ksh PID System Call 30720 .kfork Exit , return=0 Call preceded tracing. 30720 .getpid () = 30720 30720 .sigaction (2, 2ff7eba8, 2ff7ebbc) = 0 30720 .sigaction (3, 2ff7eba8, 2ff7ebcc) = 0 30720 .sigprocmask (0, 2ff7ebac, 2ff7ebdc) = 0 30720 .sigaction (20, 2ff7eba8, 2ff7ebe8) = 0 30720 .kfork () = 31233 30720 .kwaitpid (2ff7ebfc, 31233, 0, 0) = 31233 30720 .sigaction (2, 2ff7ebbc, 0) = 0 30720 .sigaction (3, 2ff7ebcc, 0) = 0 30720 .sigaction (20, 2ff7ebe8, 0) = 0 30720 .sigprocmask (2, 2ff7ebdc, 0) = 0 30720 .getuidx (4) = 0 30720 .getuidx (2) = 0 30720 .getuidx (1) = 0 30720 .getgidx (4) = 0 30720 .getgidx (2) = 0 30720 .getgidx (1) = 0 30720 ._load NoFormat, (0x2ff7ef54, 0x0, 0x0, 0x2ff7ff58) = 537227760
30720 30720
.sbrk .getpid
(65536) = 537235456 () = 30720
To produce a count of system calls made by all processes, enter: syscalls -start followed by entering: syscalls -c Output similar to the following appears: System Call Counts for all processes 5041 .lseek 4950 .kreadv 744 .sigaction 366 .close 338 .sbrk 190 .kioctl 120 .getuidx 116 .kwritev 108 .kfcntl 105 .getgidx 95 .kwaitpid 92 .gettimer 92 .select 70 .getpid 70 .sigprocmask 52 .execve 51 ._exit 51 .kfork 35 .open 35 ._load 33 .pipe 33 .incinterval 28 .sigreturn 27 .access 16 .brk 15 .times 15 .privcheck 15 .gettimerid 10 .statx 9 .STEM_R10string 4 .sysconfig 3 .P2counters_accum 3 .shmget 3 .shmat 2 .setpgid 2 .shmctl 2 .kioctl 1 .Patch_Demux_Addr_2 1 .Patch_Demux_Addr_High 1 .STEM_R3R4string 1 .shmdt 1 .Stem_KEX_copy_demux_entry 1 .STEM_R3R4string 1 .Patch_Demux_Addr_1 1 .pause 1 .accessx Files /usr/bin/syscalls Contains the syscalls command.
20.3 truss command on AIX: -------------------------AIX 5.1,5.2,5.3 The truss command is also available for SVR4 UNIX-based environments. This command is useful for tracing system calls in one or more processes. In AIX 5.2, all base system call parameter types are now recognized. In AIX 5.1, only about 40 system calls were recognized. Truss is a /proc based debugging tool that executes and traces a command, or traces an existing process. It prints names of all system calls made with their arguments and return code. System call parameters are displayed symbolically. It prints information about all signals received by a process. The AIX 5.2 version supports library calls tracing. For each call, it prints parameters and return codes. It can also trace a subset of libraries and a subset of routines in a given library. The timestamps on each line are also supported. In AIX 5.2, truss is packaged with bos.sysmgt.serv_aid, which is installable from the AIX base installation media. See the command reference for details and examples, or use the information below. A good and simple way to use truss is using a command like shown in section 20.1: # truss - o path_to_logfile executable_to_truss For example: # truss -o /tmp/myprg.log myprg The upper command is quite good for tracing a shell script, or program, that starts up, does some work, and then terminates. If an error occurs during runtime, it's likely that you find some pointers in the logfile that truss made for you. Further notes: -a Displays the parameter strings that are passed in each executed system call. # truss -a
sleep
execve("/usr/bin/sleep", 0x2FF22980, 0x2FF22988) argv: sleep
argc: 1
sbrk(0x00000000) = 0x200007A4 sbrk(0x00010010) = 0x200007B0 getuidx(4) = 0 . . __loadx(0x01000080, 0x2FF1E790, 0x00003E80, 0x2FF22720, 0x00000000) = 0xD0077130 access("/usr/lib/nls/msg/en_US/sleep.cat", 0) = 0 _getpid() = 31196 open("/usr/lib/nls/msg/en_US/sleep.cat", O_RDONLY) = 3 kioctl(3, 22528, 0x00000000, 0x00000000) Err#25 ENOTTY kfcntl(3, F_SETFD, 0x00000001) = 0 kioctl(3, 22528, 0x00000000, 0x00000000) Err#25 ENOTTY kread(3, "\0\001 £\001\001 I S O 8".., 4096) = 123 lseek(3, 0, 1) = 123 lseek(3, 0, 1) = 123 lseek(3, 0, 1) = 123 _getpid() = 31196 lseek(3, 0, 1) = 123 Usage: sleep Seconds kwrite(2, " U s a g e : s l e e p".., 21) = 21 kfcntl(1, F_GETFL, 0x00000000) = 2 kfcntl(2, F_GETFL, 0x00000000) = 2 _exit(2) -c Counts traced system calls, faults, and signals rather than displaying trace results line by line. A summary report is produced after the traced command terminates or when truss is interrupted. If the -f flag is also used, the counts include all traced Syscalls, Faults, and Signals for child processes. # truss -c ls syscall execve __loadx _exit close kwrite lseek setpid getuidx getdirent kioctl open statx getgidx sbrk access kfcntl sys totals: usr time: elapsed:
.00
seconds .00 17 .00 .00 .00 .00 .00 .00 .00 .00 .00 .00 .00 .00 .00 .00 ---.01 .00 .01
calls 1 1 2 5 1 1 19 3 3 1 2 18 4 1 6 --85
errors
--0
More truss examples: -------------------truss -o /tmp/tst -p 307214 root@zd93l14:/tmp#cat tst = 0 = 0 = 0 = 0 = 0 0x43548E38) 0x434C3E38) 0x4343FE38) 0x433BBE38) 0x432B2E38) 0x4322EE38) 0x431AAE38) 0x42F99E38) 0x4301DE38) 0x42E90E38) 0x42E0CE38) 0x43337E38) 0x42F14E38) = 0 thread_tsleep(0, 0xF033153C, 0x00000000, 0x42D03E38) _nsleep(0x4128B8E0, 0x4128B958) = 0 _nsleep(0x4128B8E0, 0x4128B958) _nsleep(0x4128B8E0, 0x4128B958) _nsleep(0x4128B8E0, 0x4128B958) _nsleep(0x4128B8E0, 0x4128B958) thread_tsleep(0, 0xF033159C, 0x00000000, thread_tsleep(0, 0xF0331594, 0x00000000, thread_tsleep(0, 0xF033158C, 0x00000000, thread_tsleep(0, 0xF0331584, 0x00000000, thread_tsleep(0, 0xF0331574, 0x00000000, thread_tsleep(0, 0xF033156C, 0x00000000, thread_tsleep(0, 0xF0331564, 0x00000000, thread_tsleep(0, 0xF0331554, 0x00000000, thread_tsleep(0, 0xF033154C, 0x00000000, thread_tsleep(0, 0xF0331534, 0x00000000, thread_tsleep(0, 0xF033152C, 0x00000000, thread_tsleep(0, 0xF033157C, 0x00000000, thread_tsleep(0, 0xF0331544, 0x00000000,
= = = = = = = = = = = = =
0 0 0 0 0 0 0 0 0 0 0 0 0
= 0
20.4 man pages for truss AIX: ----------------------------Purpose Traces a process's system calls, dynamically loaded user level function calls, received signals, and incurred machine faults. Syntax truss [ -f] [ -c] [ -a] [ -l ] [ -d ] [ -D ] [ -e] [ -i] [ { -t | -x} [!] Syscall [...] ] [ -s [!] Signal [...] ] [ { -m }[!] Fault [...]] [ { -r | -w} [!] FileDescriptor [...] ] [ { -u } [!]LibraryName [...]:: [!]FunctionName [ ... ] ] [ -o Outfile] {Command| -p pid [. . .]} Description The truss command executes a specified command, or attaches to listed process IDs, and produces a trace of the system calls, received signals, and machine faults a process incurs. Each line of the trace output reports either the Fault
or Signal name, or the Syscall name with parameters and return values. The subroutines defined in system libraries are not necessarily the exact system calls made to the kernel. The truss command does not report these subroutines, but rather, the underlying system calls they make. When possible, system call parameters are displayed symbolically using definitions from relevant system header files. For path name pointer parameters, truss displays the string being pointed to. By default, undefined system calls are displayed with their name, all eight possible argments and the return value in hexadecimal format. When the -o flag is used with truss, or if standard error is redirected to a non-terminal file, truss ignores the hangup, interrupt, and signals processes. This facilitates the tracing of interactive programs which catch interrupt and quit signals from the terminal. If the trace output remains directed to the terminal, or if existing processes are traced (using the -p flag), then truss responds to hangup, interrupt, and quit signals by releasing all traced processes and exiting. This enables the user to terminate excessive trace output and to release previously existing processes. Released processes continue to function normally. Flags -a Displays the parameter strings which are passed in each executed system call. -c Counts traced system calls, faults, and signals rather than displaying trace results line by line. A summary report is produced after the traced command terminates or when truss is interrupted. If the -f flag is also used, the counts include all traced Syscalls, Faults, and Signals for child processes. -d A timestamp will be included with each line of output. Time displayed is in seconds relative to the beginning of the trace. The first line of the trace output will show the base time from which the individual time stamps are measured. By default timestamps are not displayed. -D Delta time is displayed on each line of output. The delta time represents the
elapsed time for the LWP that incurred the event since the last reported event incurred by that thread. By default delta times are not displayed. -e Displays the environment strings which are passed in each executed system call. -f Follows all children created by the fork system call and includes their signals, faults, and system calls in the trace output. Normally, only the first-level command or process is traced. When the -f flag is specified, the process id is included with each line of trace output to show which process executed the system call or received the signal. -i Keeps interruptible sleeping system calls from being displayed. Certain system calls on terminal devices or pipes, such as open and kread, can sleep for indefinite periods and are interruptible. Normally, truss reports such sleeping system calls if they remain asleep for more than one second. The system call is then reported a second time when it completes. The -i flag causes such system calls to be reported only once, upon completion. -l Display the id (thread id) of the responsible LWP process along with truss output. By default LWP id is not displayed in the output. -m [!]Fault Traces the machine faults in the process. Machine faults to trace must be separated from each other by a comma. Faults may be specified by name or number (see the sys/procfs.h header file). If the list begins with the "!" symbol, the specified faults are excluded from being traced and are not displayed with the trace output. The default is -mall -m!fltpage. -o Outfile Designates the file to be used for the trace output. By default, the output goes to standard error. -p Interprets the parameters to truss as a list of process ids for an existing process rather than as a command to be executed. truss takes control of each process and begins tracing it, provided that the user id and group id of the process match those of the user or that the user is a privileged user. -r [!] FileDescriptor Displays the full contents of the I/O buffer for each read
on any of the specified file descriptors. The output is formatted 32 bytes per line and shows each byte either as an ASCII character (preceded by one blank) or as a two-character C language escape sequence for control characters, such as horizontal tab (\t) and newline (\n). If ASCII interpretation is not possible, the byte is shown in two-character hexadecimal representation. The first 16 bytes of the I/O buffer for each traced read are shown, even in the absence of the -r flag. The default is -r!all. -s [!] Signal Permits listing Signals to trace or exclude. Those signals specified in a list (separated by a comma) are traced. The trace output reports the receipt of each specified signal even if the signal is being ignored, but not blocked, by the process. Blocked signals are not received until the process releases them. Signals may be specified by name or number (see sys/signal.h). If the list begins with the "!" symbol, the listed signals are excluded from being displayed with the trace output. The default is -s all. -t [!] Syscall Includes or excludes system calls from the trace process. System calls to be traced must be specified in a list and separated by commas. If the list begins with an "!" symbol, the specified system calls are excluded from the trace output. The default is -tall. -u [!] [LibraryName [...]::[!]FunctionName [...] ] Traces dynamically loaded user level function calls from user libraries. The LibraryName is a comma-separated list of library names. The FunctionName is a comma-separated list of function names. In both cases the names can include name-matching metacharacters *, ?, [] with the same meanings as interpreted by the shell but as applied to the library/function name spaces, and not to files. A leading ! on either list specifies an exclusion list of names of libraries or functions not to be traced. Excluding a library excludes all functions in that library. Any function list following a library exclusion list is ignored. Multiple -u options may be specified and they are honored left-toright. By default no library/function calls are traced.
-w [!] FileDescriptor Displays the contents of the I/O buffer for each write on any of the listed file descriptors (see -r). The default is -w!all. -x [!] Syscall Displays data from the specified parameters of traced sytem calls in raw format, usually hexadecimal, rather than symbolically. The default is -x!all. Examples 1. To produce a trace of the find command on the terminal, type: truss find . -print >find.out 2. To trace the lseek, close, statx, and open system calls, type: truss -t lseek,close,statx,open find . -print > find.out 3. To display thread id along with regular output for find command, enter: truss -l find . -print >find.out 4. To display timestamps along with regular output for find command, enter: truss -d find . -print >find.out 5. To display delta times along with regular output for find command, enter: truss -D find . -print >find.out 6. To trace the malloc() function call and exclude the strlen() function call in the libc.a library while running the ls command, enter: truss -u libc.a::malloc,!strlen ls 7. To trace all function calls in the libc.a library with names starting with "m" while running the ls command, enter: truss -u libc.a::m*,!strlen ls 8. To trace all function calls from the library libcurses.a and exclude calls from libc.a while running executable foo, enter: truss -u libcurses.a,!libc.a::* foo 9. To trace the refresh() function call from libcurses.a and the malloc() function call from libc.a while running the executable foo, enter: truss -u libc.a::malloc -u libcurses.a::refresh foo
20.5 Note: How to trace an AIX machine: ---------------------------------------
The trace facility and commands are provided as part of the Software Trace Service Aids fileset named bos.sysmgt.trace. To see if this fileset is installed, use the following command: # lslpp -l | grep bos.sysmgt.trace Taking a trace: --------------The events traced are referenced by hook identifiers. Each hook ID uniquely refers to a particular activity that can be traced. When tracing, you can select the hook IDs of interest and exclude others that are not relevant to your problem. A trace hook ID is a 3 digit hexidecimal number that identifies an event being traced. Trace hook IDs are defined in the "/usr/include/sys/trchkid.h" file. The currently defined trace hook IDs can be listed using the trcrpt command: # trcrpt -j | sort | pg 001 002 003 004 005 006 .. ..
TRACE ON TRACE OFF TRACE HEADER TRACEID IS ZERO LOGFILE WRAPAROUND TRACEBUFFER WRAPAROUND
The trace daemon configures a trace session and starts the collection of system events. The data collected by the trace function is recorded in the trace log. A report from the trace log can be generated with the trcrpt command. When invoked with the -a, -x, or -X flags, the trace daemon is run asynchronously (i.e. as a background task). Otherwise, it is run interactively and prompts you for subcommands. Some trace examples: # trace -adf -C all -r PURR -o trace.raw # trace -Jfop fact proc procd filephys filepfsv filepvl filepvld locks -A786578 -Pp -a # trace -Jfop fact proc procd filephys filepfsv filepvl filepvld locks -Pp -a
# trace -Jfop fact proc procd filephys filepfsv filepvl filepvld locks -Pp -a Some trcrpt examples: Examples 1 2
To format the trace log file and print the result, enter: trcrpt | qprt To send a trace report to the /tmp/newfile file, enter:
trcrpt -o /tmp/newfile To display process IDs and exec path names in the trace report, enter: 3
4
trcrpt pid=on,exec=on -O /tmp/newfile To create trace ID histogram data, enter:
5
trcrpt -O hist=on To produce a list of all event groups, enter: trcrpt -G The format of this report is shown under the trcevgrp
command. 6 To generate back-to-back LMT reports from the common and rare buffers, specify: trcrpt -M all 7 If, in the above example, the LMT files reside at /tmp/mydir, and we want the LMT traces to be merged, specify: trcrpt -m -M all:/tmp/mydir 8 To merge the system trace with the scdisk.hdisk0 component trace, specify: trcrpt -m -l scdisk.hdisk0 /var/adm/ras/trcfile 9 To merge LMT with the system trace while not eliminating duplicate events, specify: trcrpt -O removedups=off -m -M all /var/adm/ras/trcfile 10 To merge all component traces in /tmp/mydir with the LMT traces in the default LMT directory while showing the source file for each trace event, specify: trcrpt -O filename=on -m -M all /tmp/mydir Note: This is equivalent to: trcrpt -O filename=on -m -M all -l all:/tmp/mydir Note: If the traces are from a 64-bit kernel, duplicate entries will be removed. However, on the 32-bit kernel, duplicate entries will not be removed since we do not know the CPU IDs of the entries in the components traces.
Another example of the usage of trace: ------------------------------------->> Obtaining a Sample Trace File Trace data accumulates rapidly. We want to bracket the data collection as closely around the area of interest as possible. One technique for doing this is to issue several commands on the same command line. For example: $ trace -a -k "20e,20f" -o ./trcraw ; cp ../bin/track /tmp/junk ; trcstop captures the execution of the cp command. We have used two features of the trace command. The -k "20e,20f" option suppresses the collection of events from the lockl and unlockl functions. These calls are numerous and add volume to the report without adding understanding at the level we're interested in. The -o ./trc_raw option causes the raw trace output file to be written in our local directory. Note: This example is more educational if the input file is not already cached in system memory. Choose as the source file any file that is about 50KB and has not been touched recently. >> Formatting the Sample Trace We use the following form of the trcrpt command for our report: $ trcrpt -O "exec=on,pid=on" trcraw > /tmp/cp.rpt This reports both the fully qualified name of the file that is execed and the process ID that is assigned to it. A quick look at the report file shows us that there are numerous VMM page assign and delete events in the trace, like the following sequence: 1B1 ksh 8525 VMM page delete: V.S=00 00.150E ppage=1F7F
0.003109888
0.162816
1B0 ksh 8525 0.003141376 VMM page assign: V.S=00 00.2F33 ppage=1F7F delete_in_progress process_private working_ storage
0.031488
delete_in_progress proce ss_private working_storage
We are not interested in this level of VMM activity detail at the moment, so we reformat the trace with:
$ trcrpt -k "1b0,1b1" -O "exec=on,pid=on" trcraw > cp.rpt2 The -k "1b0,1b1" option suppresses the unwanted VMM events in the formatted output. It saves us from having to retrace the workload to suppress unwanted events. We could have used the -k function of trcrpt instead of that of the trace command to suppress the lockl and unlockl events, if we had believed that we might need to look at the lock activity at some point. If we had been interested in only a small set of events, we could have specified -d "hookid1,hookid2" to produce a report with only those events. Since the hook ID is the left-most column of the report, you can quickly compile a list of hooks to include or exclude. A comprehensive list of Trace hook IDs is defined in /usr/include/sys/trchkid.h. >> Reading a Trace Report The header of the trace report tells you when and where the trace was taken, as well as the command that was used to produce it: Fri Nov 19 12:12:49 1993 System: AIX ptool Node: 3 Machine: 000168281000 Internet Address: 00000000 0.0.0.0 trace -ak 20e 20f -o -o ./trc_raw The body of the report, if displayed in a small enough font, looks as follows: ID PROCESS NAME PID SYSCALL KERNEL INTERRUPT 101 ksh 8525 kfork 101 ksh 7214 execve 134 cp 7214 exec cp ../bin/trk/junk
ELAPSED_SEC
DELTA_MSEC
0.005833472
0.107008
0.012820224
0.031744
0.014451456
0.030464
APPL
In cp.rpt you can see the following phenomena: The fork, exec, and page fault activities of the cp process The opening of the input file for reading and the creation of the /tmp/junk file The successive read/write system calls to accomplish the copy The process cp becoming blocked while waiting for I/O completion, and the wait process being dispatched How logical-volume requests are translated to physical-volume requests The files are mapped rather than buffered in traditional kernel buffers, and the read accesses cause page faults that must be resolved by the Virtual Memory Manager. The Virtual Memory Manager senses sequential access and begins to prefetch the file pages.
The size of the prefetch becomes larger as sequential access continues. When possible, the disk device driver coalesces multiple file requests into one I/O request to the drive. The trace output looks a little overwhelming at first. This is a good example to use as a learning aid. If you can discern the activities described, you are well on your way to being able to use the trace facility to diagnose system-performance problems. >> Filtering of the Trace Report The full detail of the trace data may not be required. You can choose specific events of interest to be shown. For example, it is sometimes useful to find the number of times a certain event occurred. To answer the question "How many opens occurred in the copy example?" first find the event ID for the open system call. This can be done as follows: $ trcrpt -j | grep -i open You should be able to see that event ID 15b is the open event. Now, process the data from the copy example as follows: $ trcrpt -d 15b -O "exec=on" trc_raw The report is written to standard output, and you can determine the number of open subroutines that occurred. If you want to see only the open subroutines that were performed by the cp process, run the report command again using the following: $ trcrpt -d 15b -p cp -O "exec=on" trc_raw $ trcrpt -o /tmp/newfile
A Wrapper around trace: ----------------------Simple instructions for using the AIX trace facility >> Five aix commands are used: -trace -trcon -trcoff -trcstop -trcrpt These are described in AIX Commands Reference, Volume 5, but hopefully you won't have to dig into that. Scripts to download
I've provided wrappers for the trace and trcrpt commands since there are various command-line parameters to specify. -atrace -atrcrpt >> Contents atrace: # To change from the default trace file, set TRCFILE to # the name of the raw trace file name here; this should # match the name of the raw trace file in atrcrpt. # Don't do this on AIX 4.3.3 ML 10, where you'll need # to use the default trace file, /usr/adm/ras/trcfile #TRCFILE="-o /tmp/raw" # trace categories not to collect IGNORE_VMM="1b0,1b1,1b2,1b3,1b5,1b7,1b8,1b9,1ba,1bb,1bc,1bd,1be" IGNORE_LOCK=20e,20f IGNORE_PCI=2e6,2e7,2e8 IGNORE_SCSI=221,223 IGNORE_OTHER=100,10b,116,119,11f,180,234,254,2dc,402,405,469,7ff IGNORE="$IGNORE_VMM,$IGNORE_LOCK,$IGNORE_PCI,$IGNORE_SCSI,$IGNORE_LVM, $IGNORE_OTHER" trace -a -d -k $IGNORE $TRCFILE >> Contents atrcrpt: # # # # # #
To change from the default trace file, set TRCFILE to the name of the raw trace file name here; this should match the name of the raw trace file in atrace. Don't do this on AIX 4.3.3 ML 10, where you'll need to use the default trace file, /usr/adm/ras/trcfile TRCFILE=/tmp/raw
# edit formatted trace file name here FMTFILE=/tmp/fmt trcrpt -O pid=on,tid=on,timestamp=1 $TRCFILE >$FMTFILE Setup instructions edit atrace and atrcrpt and ensure that names of files for raw and formatted trace are appropriate Please see the comments in the scripts about 4.3.3 ML 10 being broken for trcrpt, such that the default file name needs to be used. You may find that specifying non-default filenames does not have the desired effect. make atrace and atrcrpt executable via chmod Data collection ./atrace (this is my wrapper for the trace command) trcon (at this point we're collecting the trace; wait for a bit of time to
trace whatever the failure is) trcoff trcstop ./atrcrpt (this is my wrapper for formatting the report) After running atrcrpt, the formatted report will be in file /tmp/fmt. Sample section of formatted trace Note that failing system calls generally show "error Esomething" in the race, as highlighted below. The second column is the process id and the third column is the thread id. Once you see something of interest in the trace, you may want to use grep to pull out all records for that process id, since in general the trace is interleaved with the activity of all the processes in the system. 101 14690 19239 statx LR = D0174110 107 14690 19239 lookuppn: /usr/HTTPServer/htdocs/en_US/manual/ibm/index.htmlxxxxxxxxxxx 107 14690 19239 lookuppn: file not found 104 14690 19239 return from statx. error ENOENT [79 usec] 101 14690 19239 statx LR = D0174110 107 14690 19239 lookuppn: /usr/HTTPServer/htdocs/en_US/manual/ibm 104 14690 19239 return from statx [36 usec] Note about an AIX trace on Websphere: ------------------------------------In addition to the WebSpherer MQ trace, WebSphere MQ for AIXr users can use the standard AIX system trace. AIX system tracing is a two-step process: >> Gathering the data >> Formatting the results WebSphere MQ uses two trace hook identifiers: X'30D' This event is recorded by WebSphere MQ on entry to or exit from a subroutine. X'30E' This event is recorded by WebSphere MQ to trace data such as that being sent or received across a communications network. Trace provides detailed execution tracing to help you to analyze problems. IBMr service support personnel might ask for a problem to be re-created with trace enabled. The files produced by trace can be very large so it is important to qualify a trace, where possible. For example, you can optionally qualify a trace by time and by component. There are two ways to run trace: >> Interactively.
The following sequence of commands runs an interactive trace on the program myprog and ends the trace. trace -j30D,30E -o trace.file ->!myprog ->q >> Asynchronously. The following sequence of commands runs an asynchronous trace on the program myprog and ends the trace. trace -a -j30D,30E -o trace.file myprog trcstop You can format the trace file with the command: trcrpt -t /usr/mqm/lib/amqtrc.fmt trace.file > report.file report.file is the name of the file where you want to put the formatted trace output.
20.6 Nice example: Tracing with truss on AIX: --------------------------------------------Application tracing displays the calls that an application makes to external libraries and the kernel. These calls give the application access to the network, the file system, and the display. By watching the calls and their results, you can get some idea of what the application "expects", which can lead to a solution. Each UNIXr system provides its own commands for tracing. This article introduces you to truss, which Solaris and AIXr support. On Linuxr, you perform tracing with the strace command. Although the command-line parameters might be slightly different, application tracing on other UNIX flavors might go by the names ptrace, ktrace, trace, and tusc. >> A classic file permissions problem One class of problems that plagues systems administrators is file permissions. An application likely has to open certain files to do its work. If the open operation fails, the application should let the administrator know. However, developers often forget to check the result of functions or, to add to the confusion, perform the check, but don't adequately handle the error. For example, here's the output of an application that's failing to open: $ ./openapp This should never happen!
After running the fictitious openapp application, I received the unhelpful (and false) error message, This should never happen!. This is a perfect time to introduce truss. Listing 1 shows the same application run under the truss command, which shows all the function calls that this program made to outside libraries. Listing 1. Openapp run under truss $ truss ./openapp execve("openapp", 0xFFBFFDEC, 0xFFBFFDF4) argc = 1 getcwd("/export/home/sean", 1015) = 0 stat("/export/home/sean/openapp", 0xFFBFFBC8) = 0 open("/var/ld/ld.config", O_RDONLY) Err#2 ENOENT stat("/opt/csw/lib/libc.so.1", 0xFFBFF6F8) Err#2 ENOENT stat("/lib/libc.so.1", 0xFFBFF6F8) = 0 resolvepath("/lib/libc.so.1", "/lib/libc.so.1", 1023) = 14 open("/lib/libc.so.1", O_RDONLY) = 3 memcntl(0xFF280000, 139692, MC_ADVISE, MADV_WILLNEED, 0, 0) = 0 close(3) = 0 getcontext(0xFFBFF8C0) getrlimit(RLIMIT_STACK, 0xFFBFF8A0) = 0 getpid() = 7895 [7894] setustack(0xFF3A2088) open("/etc/configfile", O_RDONLY) Err#13 EACCES [file_dac_read] ioctl(1, TCGETA, 0xFFBFEF14) = 0
fstat64(1, 0xFFBFEE30) = 0 stat("/platform/SUNW,Sun-Blade-100/lib/libc_psr.so.1", 0xFFBFEAB0) = 0 open("/platform/SUNW,Sun-Blade-100/lib/libc_psr.so.1", O_RDONLY) = 3 close(3) = 0 This should never happen! write(1, " T h i s s h o u l d ".., 26) = 26 _exit(3)
Each line of the output represents a function call that the application made along with the return value, if applicable. (You don't need to know each function call, but for more information, you can call up the man page for the function, such as with the command man open.) To find the call that is potentially causing the problem, it's often easiest to start at the end (or as close as possible to where the problems start). For example, you know that the application outputs This should never happen!, which appears near the end of the output. Chances are that if you find this message and work your way up through the truss command output, you'll come across the problem.
Scrolling up from the error message, notice the line beginning with open("/etc/configfile"..., which not only looks relevant but also seems to return an error of Err#13 EACCES. Looking at the man page for the open() function (with man open), it's evident that the purpose of the function is to open a file -- in this case, /etc/configfile -- and that a return value of EACCES means that the problem is related to permissions. Sure enough, a look at /etc/configfile shows that the user doesn't have permissions to read the file. A quick chmod later, and the application is running properly. The output of Listing 1 shows two other calls, open() and stat(), that return an error. Many of the calls toward the beginning of the application, including the other two errors, are added by the operating system as it runs the application. Only experience will tell when the errors are benign and when they aren't. In this case, the two errors and the three lines that follow them are trying to find the location of libc.so.1, which they eventually do. You'll see more about shared library problems later.
>> The application doesn't start Sometimes, an application fails to start properly; but rather than exiting, it just hangs. This behavior is often a symptom of contention for a resource (such as two processes competing for a file lock), or the application is looking for something that is not coming back. This latter class of problems could be almost anything, such as a name lookup that's taking a long time to resolve, or a file that should be found in a certain spot but isn't there. In any case, watching the application under truss should reveal the culprit. While the first code example showed an obvious link between the system call causing the problem and the file, the example you're about to see requires a bit more sleuthing. Listing 2 shows a misbehaving application called Getlock run under truss. Listing 2. Getlock run under truss $ truss ./getlock execve("getlock", 0xFFBFFDFC, 0xFFBFFE04) argc = 1 getcwd("/export/home/sean", 1015) = 0 resolvepath("/export/home/sean/getlock", "/export/home/sean/getlock", 1023) = 25 resolvepath("/usr/lib/ld.so.1", "/lib/ld.so.1", 1023) = 12 stat("/export/home/sean/getlock", 0xFFBFFBD8) = 0 open("/var/ld/ld.config", O_RDONLY) Err#2 ENOENT stat("/opt/csw/lib/libc.so.1", 0xFFBFF708) Err#2 ENOENT stat("/lib/libc.so.1", 0xFFBFF708) = 0
resolvepath("/lib/libc.so.1", "/lib/libc.so.1", open("/lib/libc.so.1", O_RDONLY) close(3) getcontext(0xFFBFF8D0) getrlimit(RLIMIT_STACK, 0xFFBFF8B0) getpid() setustack(0xFF3A2088) open("/tmp/lockfile", O_WRONLY|O_CREAT, 0755) getpid() fcntl(3, F_SETLKW, 0xFFBFFD60) (sleeping...)
1023) = 14 = 3 = 0 = 0 = 10715 [10714] = 3 = 10715 [10714]
The final call, fcntl(), is marked as sleeping, because the function is blocking. This means that the function is waiting for something to happen, and the kernel has put the process to sleep until the event occurs. To determine what the event is, you must look at fcntl(). The man page for fcntl() (man fcntl) describes the function simply as "file control" on Solaris and "manipulate file descriptor" on Linux. In all cases, fcntl() requires a file descriptor, which is an integer describing a file the process has opened, a command that specifies the action to be taken on the file descriptor, and finally any arguments required for the specific function. In the example in Listing 2, the file descriptor is 3, and the command is F_SETLKW. (The 0xFFBFFD60 is a pointer to a data structure, which doesn't concern us now.) Digging further, the man page states that F_SETLKW opens a lock on the file and waits until the lock can be obtained. From the first example involving the open() system call, you saw that a successful call returns a file descriptor. In the truss output of Listing 2, there are two cases in which the result of open() returns 3. Because file descriptors are reused after they are closed, the relevant open() is the one just above fcntl(), which is for /tmp/lockfile. A utility like lsof lists any processes holding open a file. Failing that, you could trace through /proc to find the process with the open file. However, as is usually the case, a file is locked for a good reason, such as limiting the number of instances of the application or configuring the application to run in a user-specific directory. >> Attaching to a running process Sometimes, an application is already running when a problem occurs. Being able to run an already-running process under truss would be helpful. For example, notice that in the output of the Top application, a certain process has been consuming 95 percent of the CPU for quite some time, as shown in Listing 3.
Listing 3. Top output showing a CPU-intensive process PID USERNAME LWP PRI NICE SIZE 11063 sean 1 0 0 1872K
RES STATE 952K run
TIME CPU COMMAND 87.9H 94.68% udpsend
The -p option to truss allows the owner of the process, or root, to attach to a running process and view the system call activity. The process id (PID) is required. In the example shown in Listing 3, the PID is 11063. Listing 4 shows the system call activity of the application in question. Listing 4. truss output after attaching to a running process $ truss -p 11063: sendto(3, sendto(3, sendto(3, sendto(3, sendto(3, sendto(3, sendto(3, sendto(3, . repeats
" a " a " a " a " a " a " a " a ...
b b b b b b b b
c", c", c", c", c", c", c", c",
3, 3, 3, 3, 3, 3, 3, 3,
0, 0, 0, 0, 0, 0, 0, 0,
0xFFBFFD58, 0xFFBFFD58, 0xFFBFFD58, 0xFFBFFD58, 0xFFBFFD58, 0xFFBFFD58, 0xFFBFFD58, 0xFFBFFD58,
16) 16) 16) 16) 16) 16) 16) 16)
= = = = = = = =
3 3 3 3 3 3 3 3
The sendto() function's man page (man sendto) shows that this function is used to send a message from a socket -- typically, a network connection. The output of truss shows the file descriptor (the first 3) and the data being sent (abc). Indeed, capturing a sample of network traffic with the snoop or tcpdump tool shows a large amount of traffic being directed to a particular host, which is likely not the result of a properly behaving application. Note that truss was not able to show the creation of file descriptor 3, because you had attached after the descriptor was created. This is one limitation of attaching to a running process and the reason why you should gather other information using a tool, such as a packet analyzer before jumping to conclusions. This example might seem somewhat contrived (and technically it was, because I wrote the udpsend application to demonstrate how to use truss), but it is based on a real situation. I was investigating a process running on a UNIX-based appliance that had a CPU-bound process. Tracing the application showed the same packet activity. Tracing with a network analyzer showed the packets were being directed to a host on the Internet. After escalating with the vendor, I determined that the problem was their application failing to perform proper error checking on a binary configuration file. The file had somehow become corrupted. As a result, the application interpreted
the file incorrectly and repeatedly hammered a random IP address with User Datagram Protocol (UDP) datagrams. After I replaced the file, the process behaved as expected.
>> Filtering output After a while, you'll get the knack of what to look for. While it's possible to use the grep command to go through the output, it's easier to configure truss to focus only on certain calls. This practice is common if you're trying to determine how an application works, such as which configuration files the application is using. In this case, the open() and stat() system calls point to any files the application is trying to open. You use open() to open a file, but you use stat() to find information about a file. Often, an application looks for a file with a series of stat() calls, and then opens the file it wants. For truss, you add filtering system calls with the -t option. For strace under Linux, you use -e. In either case, you pass a comma-separated list of system calls to be shown on the command line. By prefixing the list with the exclamation mark (!), the given calls are filtered out of the output. Listing 5 shows a fictitious application looking for a configuration file. Listing 5. truss output filtered to show only stat() and open() functions $ truss -tstat,open ./app stat("/export/home/sean/app", 0xFFBFFBD0) = 0 open("/var/ld/ld.config", O_RDONLY) Err#2 ENOENT stat("/opt/csw/lib/libc.so.1", 0xFFBFF700) Err#2 ENOENT stat("/lib/libc.so.1", 0xFFBFF700) = 0 open("/lib/libc.so.1", O_RDONLY) = 3 stat("/export/home/sean/.config", 0xFFBFFCF0) Err#2 ENOENT stat("/etc/app/configfile", 0xFFBFFCF0) Err#2 ENOENT stat("/etc/configfile", 0xFFBFFCF0) = 0 open("/etc/configfile", O_RDONLY) = 3 The final four lines are the key here. The stat() function for /export/home/sean/.config results in ENOENT, which means that the file wasn't found. The code then tries /etc/app/configfile before it finds the correct information in /etc/configfile. The significance of first checking in the user's home directory is that you can override the configuration by user.
>> Final thoughts
Whether your operating system uses truss, strace, trace, or something else, the ability to peer into an application's behavior is a powerful tool for problem solving. The methodology can be summed up as follows: Describe the problem. Trace the application. Start at the spot at which the problem occurs and work backward through the system calls to identify the problem. Use the man pages for help on interpreting the system calls. Correct the behavior and test. Tracing application behavior is a powerful troubleshooting tool, because you're observing the system calls that the application makes to the operating system. When the usual problem-solving methods fail, turn to application tracing.
20.7. snap command on AIX: -------------------------The snap command gathers system configuration information and compresses the information into a pax file. The information gathered with the snap command may be required to identify and resolve system problems. In normal conditions, the command "snap -gc" should be sufficient. The pax file will be stored in /tmp/ibmsupt # snap -gc create the following file: /tmp/ibmsupt/snap.pax.Z Further info: snap Command Purpose Gathers system configuration information. Syntax snap [ -a ] [ -A ] [ -b ] [ -B ] [ -c ] [ -C ] [ -D ] [ -f ] [ -g ] [ -G ] [ -i ] [ -k ] [ -l ] [ -L ][ -n ] [ -N ] [ -p ] [ -r ] [ -R ] [ -s ] [ -S ] [ -t ] [ -T Filename ] [ -w ] [ -o OutputDevice ] [ -d Dir ] [ -v Component ] [ -O FileSplitSize ] [ -P Files ] [ script1 script2 ... | All | file:filepath ]
snap [ -a ] [ -A ] [ -b ] [ -B ] [ -c ] [ -C ] [ -D ] [ -f ] [ -g ] [ -G ] [ -i ] [ -k ] [ -l ] [ -L ][ -n ] [ -N ] [ -p ] [ -r ] [ -R ] [ -s ] [ -S ] [ -t ] [ -T Filename ] [ -o OutputDevice ] [ -d Dir ] [ -v Component ] [ -O FileSplitSize ] [ -P Files ] [ script1 script2 ... | All | file:filepath ] snap -e [ -m Nodelist ] [ -d Dir ] Description The snap command gathers system configuration information and compresses the information into a pax file. The file may then be written to a device such as tape or DVD, or transmitted to a remote system. The information gathered with the snap command might be required to identify and resolve system problems. Note: Root user authority is required to execute the snap command. Use the snap -o /dev/cd0 command to copy the compressed image to DVD. Use the snap -o /dev/rmt0 command to copy the image to tape. Use the snap -o /dev/rfd0 command to copy the compressed image to diskette. Use the snap -o /dev/rmt0 command to copy the image to tape. Approximately 8MB of temporary disk space is required to collect all system information, including contents of the error log. If you do not gather all system information with the snap -a command, less disk space may be required (depending on the options selected). Note: If you intend to use a tape to send a snap image to IBM(R) for software support, the tape must be one of the following formats: * 8mm, 2.3 Gb capacity * 8mm, 5.0 Gb capacity * 4mm, 4.0 Gb capacity Using other formats prevents or delays IBM software support from being able to examine the contents. The snap -g command gathers general system information, including the following: * Error report * Copy of the customized Object Data Manager (ODM) database * Trace file * User environment * Amount of physical memory and paging space * Device and attribute information * Security user information The output of the snap -g command is written to the /tmp/ibmsupt/general/general.snap file. The snap command checks for available space in the /tmp/ibmsupt directory, the default directory for snap command output. You can write the output to another directory by using the -d flag. If there is not enough space to hold the snap command output, you must expand the file system.
Each execution of the snap command appends information to previously created files. Use the -r flag to remove previously gathered and saved information. Flags: -a Gathers all system configuration information. This option requires approximately 8MB of temporary disk space. -A Gathers asynchronous (TTY) information. -b Gathers SSA information. -B Bypasses collection of SSA adapter dumps. The -B flag only works when the -b flag is also specified; otherwise, the -B flag is ignored. -c Creates a compressed pax image (snap.pax.Z file) of all files in the /tmp/ibmsupt directory tree or other named output directory. Note: Information not gathered with this option should be copied to the snap directory tree before using the -c flag. If a test case is needed to demonstrate the system problem, copy the test case to the /tmp/ibmsupt/testcase directory before compressing the pax file. -C Retrieves all the files in the fwdump_dir directory. The files are placed in the "general" subdirectory. The -C snap option behaves the same as -P*. -D Gathers dump and /unix information. The primary dump device is used. Notes: 1 If bosboot -k was used to specify the running kernel to be other than /unix, the incorrect kernel is gathered. Make sure that /unix is , or is linked to, the kernel in use when the dump was taken. 2 If the dump file is copied to the host machine, the snap command does not collect the dump image in the /tmp/ibmsupt/dump directory. Instead, it creates a link in the dump directory to the actual dump image. -d AbsolutePath Identifies the optional snap command output directory (/tmp/ibmsupt is the default). You must specify the absolute path. -e Gathers HACMP(TM) specific information. Note: HACMP specific data is collected from all nodes belonging to the cluster . This flag cannot be used with any other flags except -m and -d. -f Gathers file system information. -g Gathers the output of the lslpp -hac command, which is required to recreate exact operating system environments. Writes output to the /tmp/ibmsupt/general/lslpp.hBc file. Also collects general system information and writes the output to the /tmp/ibmsupt/general/general.snap file.
-G
Includes predefined Object Data Manager (ODM) files in general information collected with the -g flag. -i Gathers installation debug vital product data (VPD) information.
strace example on Linux: -----------------------One main trace utility on most Linux distro's, is the "strace" command. You can use it with many parameters, but the "-o outputfile" is very important, in order to save the output to a file. Use it like: # strace -o logfile Because strace will show you the systemcalls and signals, you can use it to reveal whether a program cannot find a file, or does not have permissions to read (or write to) a file. In such a case, a program might fail. Example: Suppose we have a file called "/etc/security.conf". Now we run a utility to read the file (like cat, pg, more, less etc..) as a normal user, which user does not have permissions to read the file. Let's trace that event to a logfile, and see what we can discover. $ strace -o strace_example.log less /etc/security.conf A trace file can get pretty long, but you should just browse it and be alert on what seems to be an error reported. So, if we take a look in the logfile "strace_example.log" .. .. open("/etc/security.conf", O_RDONLY|O_LARGEFILE) = -1 EACCES (Permission denied) write(2, "/etc/security.conf: Permission denied\n", 32) = 32 .. .. We can clearly see, that our program failed due to lack of read permission.
============= 21. Logfiles:
============= 21.1 Solaris: ============= Unix message files record all system problems like disk errors, swap errors, NFS problems, etc. Monitor the following files on your system to detect system problems: tail -f /var/adm/syslog tail -f /var/adm/messages tail -f /var/log/syslog You can also use the dmesg command. Messages are recorded by the syslogd demon. Diagnostics can be done from the OK prompt after a reboot, like probescsci, show-devs, show-disks, test memory etc.. You can also use SunVTS tool to run diagnostics. SunVTS is Suns's Validation Test package. System dumps: You can manage system dumps by using the dumpadm command. Userlogins are recorded in /var/adm/utmpx Solaris 8,9 does not use wtmp or utmp Logfiles: --------/var/adm/messages The syslogd daemon logs its findings into this file /var/adm/lastlog This file holds the most recent login time for each user of the system /var/adm/utmpx This database file commands such as who, write, login. terminal and login are stored, and if information.
contains user access and accounting information for The utmpx file is where information such as the time you use the who command, it will retrieve that
/var/adm/wtmpx This file contains the history of user access and accounting information, for the utmpx database. The "last" command will use this file, to show you the historical login and logout info, since the last reboot. /var/adm/sulog This file shows you which users has used the su command, to switch to another user.
/var/adm/acct If accounting is enabled, accounting information is recorded in that file. /var/adm/loginlog If it is important for you to track whether users are trying to log in to your user accounts, you can create a /var/adm/loginlog file with read and write permissions for root only. After you create the loginlog file, all failed login activity is written to this file automatically after five failed attempts. The five-try limit avoids recording failed attempts that are the result of typographical errors. The loginlog file contains one entry for each failed attempt. Each entry contains the user's login name, tty device, and time of the attempt.
AIX: ---Periodical the following files have to be decreased in size. You can use cat /dev/null command Example: cat /dev/null >/var/adm/sulog /var/adm/sulog /var/adm/cron/log /var/adm/wtmp /etc/security/failedlogin Notes about the errorlog, thats the file /var/adm/ras/errlog. Do NOT use cat /dev/null to clear the errorlog. Use instead the following procedure: # /usr/lib/errstop (stop the error daemon) move the errlog file # /usr/lib/errstart (start the error daemon)
errdemon: --------On most UNIX systems, information and errors from system events and processes are managed by the syslog daemon (syslogd); depending on settings in the configuration file /etc/syslog.conf, messages are passed from the operating system, daemons, and applications to the console, to log files, or to nowhere at all. AIX includes the syslog daemon, and it is used in the same way that other UNIX-based operating systems use it. In addition to syslog, though, AIX also contains another facility for the management of hardware, operating system,
and application messages and errors. This facility, while simple in its operation, provides unique and valuable insight into the health and happiness of an AIX system. The AIX error logging facility components are part of the bos.rte and the bos.sysmgt.serv_aid packages, both of which are automatically placed on the system as part of the base operating system installation. Unlike the syslog daemon, which performs no logging at all in its default configuration as shipped, the error logging facility requires no configuration before it can provide useful information about the system. The errdemon is started during system initialization and continuously monitors the special file /dev/error for new entries sent by either the kernel or by applications. The label of each new entry is checked against the contents of the Error Record Template Repository, and if a match is found, additional information about the system environment or hardware status is added, before the entry is posted to the error log. The actual file in which error entries are stored is configurable; the default is /var/adm/ras/errlog. That file is in a binary format and so should never be truncated or zeroed out manually. The errlog file is a circular log, storing as many entries as can fit within its defined size. A memory buffer is set by the errdemon process, and newly arrived entries are put into the buffer before they are written to the log to minimize the possibility of a lost entry. The name and size of the error log file and the size of the memory buffer may be viewed with the errdemon command: [aixhost:root:/] # /usr/lib/errdemon -l Error Log Attributes -------------------------------------------Log File /var/adm/ras/errlog Log Size 1048576 bytes Memory Buffer Size 8192 bytes The parameters displayed may be changed by running the errdemon command with other flags, documented in the errdemon man page. The default sizes and values have always been sufficient on our systems, so I've never had reason to change them. Due to use of a circular log file, it is not necessary (or even possible) to rotate the error log. Without intervention, errors will remain in the log indefinitely, or until the log fills up with new entries. As shipped, however, the crontab for the root user contains two entries that are executed daily, removing hardware errors that are older than 90 days, and all other errors that are older than 30 days.
0 11 0 12
* *
* *
* /usr/bin/errclear -d S,O 30 * /usr/bin/errclear -d H 90
The errdemon deamon constantly checks the /dev/error special file, and when new data is written, the deamon conducts a series of operations. - To determine the path to your system's error logfile, run the command: # /usr/lib/errdemon -l Error Log Attributes Log File /var/adm/ras/errlog Log Size 1048576 bytes Memory 8192 bytes - To change the maximum size of the error log file, enter: # /usr/lib/errdemon -s 200000 You can generate the error reports using smitty or through the errpt command. # smitty errpt gives you a dialog screen where you can select types of information. # errpt -a # errpt - d H # errpt -a|pg Produces a detailed report for each entry in the error log # errpt -aN hdisk1 Displays an error log for ALL errors occurred on this drive. If more than a few errors occur within a 24 hour period, execute the CERTIFY process under DIAGNOSTICS to determine if a PV is becoming marginal. If you use the errpt without any options, it generates a summary report. If used with the -a option, a detailed report is created. You can also display errors of a particular class, for example for the Hardware class. Examples using errpt: --------------------To display a complete summary report, enter: errpt To display a complete detailed report, enter: errpt -a To display a detailed report of all errors logged for the error identifier E19E094F, enter: errpt -a -j E19E094F
To display a detailed report of all errors logged in the past 24 hours, enter: errpt -a -s mmddhhmmyy where the mmddhhmmyy string equals the current month, day, hour, minute, and year, minus 24 hours. To list error-record templates for which logging is turned off for any error-log entries, enter: errpt -t -F log=0 To view all entries from the alternate error-log file /var/adm/ras/errlog.alternate, enter: errpt -i /var/adm/ras/errlog.alternate To view all hardware entries from the alternate error-log file /var/adm/ras/errlog.alternate, enter: errpt -i /var/adm/ras/errlog.alternate -d H To display a detailed report of all errors logged for the error label ERRLOG_ON, enter: errpt -a -J ERRLOG_ON To display a detailed report of all errors and group duplicate errors, enter: errpt -aD To display a detailed report of all errors logged for the error labels DISK_ERR1 and DISK_ERR2 during the month of August, enter: errpt -a -J DISK_ERR1,DISK_ERR2 -s 0801000004 -e 0831235904" errclear: Deletes entries in the error log Example: errclear 0 (Truncates the errlog to 0 bytes) Example errorreport: -------------------Example 1: ---------P550:/home/reserve $ errpt IDENTIFIER TIMESTAMP 0EC00096 0130224507 0EC00096 0130224007 0EC00096 0130224007 0EC00096 0130223507 F7DDA124 0130223507 MISSING 52715FA5 0130223507 STATUS AREA CAD234BE 0130223507 CLOSING
T P P P P U
C U U U U H
RESOURCE_NAME SYSPFS SYSPFS SYSPFS SYSPFS LVDD
DESCRIPTION STORAGE SUBSYSTEM FAILURE STORAGE SUBSYSTEM FAILURE STORAGE SUBSYSTEM FAILURE STORAGE SUBSYSTEM FAILURE PHYSICAL VOLUME DECLARED
U H LVDD
FAILED TO WRITE VOLUME GROUP
U H LVDD
QUORUM LOST, VOLUME GROUP
613E5F38 0130223507 613E5F38 0130223507 613E5F38 0130223507 0873CF9F 0130191907 0EC00096 0130162407 51E537B5 0130161807 291D64C3 0130161807 291D64C3 0130161807 BFE4C025 0130161807 51E537B5 0130161707 291D64C3 0130161707 291D64C3 0130161707 51E537B5 0130161707 291D64C3 0130161707 291D64C3 0130161707 BFE4C025 0130161607 BFE4C025 0130161407 BFE4C025 0130161307 BFE4C025 0130161307 BFE4C025 0130161207 BFE4C025 0130161207 0EC00096 0130161207 BFE4C025 0130161107 D2A1B43E 0130161107 D2A1B43E 0130161107 CD546B25 0130161107 CD546B25 0130161107 1ED0A744 0130161107 CD546B25 0130161107 D2A1B43E 0130161107 1ED0A744 0130161107 F7DDA124 0130161107 MISSING 52715FA5 0130161107 STATUS AREA CAD234BE 0130161107 CLOSING 613E5F38 0130161107 EAA3D429 0130161107 613E5F38 0130161107 613E5F38 0130161107 41BF2110 0130161107 613E5F38 0130161107 CAD234BE 0130161107 CLOSING F7DDA124 0130161107 MISSING 41BF2110 0130161107 613E5F38 0130161107 6472E03B 0130161107 FEC31570 0130161107 C14C511C 0130161107 BFE4C025 0130161107 FE2DEE00 0130144307 IN THE NET FE2DEE00 0130143207 IN THE NET
P P P T P P I I P P I I P I I P P P P P P P P P P I I P I P P U
H H H S U H H H H H H H H H H H H H H H H U H U U O O U O U U H
LVDD LVDD LVDD pts/4 SYSPFS sysplanar0 sysplanar0 sysplanar0 sysplanar0 sysplanar0 sysplanar0 sysplanar0 sysplanar0 sysplanar0 sysplanar0 sysplanar0 sysplanar0 sysplanar0 sysplanar0 sysplanar0 sysplanar0 SYSPFS sysplanar0 SYSPFS SYSPFS SYSPFS SYSPFS SYSPFS SYSPFS SYSPFS SYSPFS LVDD
I/O ERROR DETECTED BY LVM I/O ERROR DETECTED BY LVM I/O ERROR DETECTED BY LVM TTYHOG OVER-RUN STORAGE SUBSYSTEM FAILURE platform_dump saved to file platform_dump indicator event platform_dump indicator event UNDETERMINED ERROR platform_dump saved to file platform_dump indicator event platform_dump indicator event platform_dump saved to file platform_dump indicator event platform_dump indicator event UNDETERMINED ERROR UNDETERMINED ERROR UNDETERMINED ERROR UNDETERMINED ERROR UNDETERMINED ERROR UNDETERMINED ERROR STORAGE SUBSYSTEM FAILURE UNDETERMINED ERROR FILE SYSTEM CORRUPTION FILE SYSTEM CORRUPTION FILE SYSTEM RECOVERY REQUIRED FILE SYSTEM RECOVERY REQUIRED FILE SYSTEM LOGGING SUSPENDED FILE SYSTEM RECOVERY REQUIRED FILE SYSTEM CORRUPTION FILE SYSTEM LOGGING SUSPENDED PHYSICAL VOLUME DECLARED
U H LVDD
FAILED TO WRITE VOLUME GROUP
U H LVDD
QUORUM LOST, VOLUME GROUP
P U P P U P U
I/O ERROR DETECTED BY LVM PHYSICAL PARTITION MARKED STALE I/O ERROR DETECTED BY LVM I/O ERROR DETECTED BY LVM MIRROR WRITE CACHE WRITE FAILED I/O ERROR DETECTED BY LVM QUORUM LOST, VOLUME GROUP
H S H H H H H
LVDD LVDD LVDD LVDD LVDD LVDD LVDD
U H LVDD
PHYSICAL VOLUME DECLARED
U P P P T P P
MIRROR WRITE CACHE WRITE FAILED I/O ERROR DETECTED BY LVM EEH permanent error for adapter UNDETERMINED ERROR ADAPTER ERROR UNDETERMINED ERROR DUPLICATE IP ADDRESS DETECTED
H H H H H H S
LVDD LVDD sysplanar0 sisscsia2 scsi5 sysplanar0 SYSXAIXIF
P S SYSXAIXIF
DUPLICATE IP ADDRESS DETECTED
B6048838 0129100507 P S SYSPROC TERMINATED B6048838 0129100307 P S SYSPROC TERMINATED
SOFTWARE PROGRAM ABNORMALLY SOFTWARE PROGRAM ABNORMALLY
You might create a script called alert.sh and call it from your .profile #!/usr/bin/ksh cd ~ rm -rf /root/alert.log echo "Important alerts in errorlog: " >> /root/alert.log errpt | grep -i STORAGE >> /root/alert.log errpt | grep -i QUORUM >> /root/alert.log errpt | grep -i ADAPTER >> /root/alert.log errpt | grep -i VOLUME >> /root/alert.log errpt | grep -i PHYSICAL >> /root/alert.log errpt | grep -i STALE >> /root/alert.log errpt | grep -i DISK >> /root/alert.log errpt | grep -i LVM >> /root/alert.log errpt | grep -i LVD >> /root/alert.log errpt | grep -i UNABLE >> /root/alert.log errpt | grep -i USER >> /root/alert.log errpt | grep -i CORRUPT >> /root/alert.log cat /root/alert.log if [ `cat alert.log|wc -l` -eq 1 ] then echo "No critical errors found." fi echo " " echo "Filesystems that might need attention, e.g. %used:" df -k |awk '{print $4,$7}' |grep -v "Filesystem"|grep -v tmp /tmp/tmp.txt cat /tmp/tmp.txt | sort -n | tail -3
>
Example 2: ---------IDENTIFIER 173C787F adapter 90D3329C AE3E3FAD AE3E3FAD AE3E3FAD AE3E3FAD AE3E3FAD AE3E3FAD AE3E3FAD C1348779 C1348779 C1348779
TIMESTAMP T C RESOURCE_NAME 0710072007 I S topsvcs
DESCRIPTION Possible malfunction on local
0710072007 0710064907 0710064907 0710064907 0710064907 0710064907 0710064907 0710064907 0710061107 0710061107 0710061107
NIM read/write error FSCK FOUND ERRORS FSCK FOUND ERRORS FSCK FOUND ERRORS FSCK FOUND ERRORS FSCK FOUND ERRORS FSCK FOUND ERRORS FSCK FOUND ERRORS LOG I/O ERROR LOG I/O ERROR LOG I/O ERROR
P I I I I I I I I I I
S O O O O O O O O O O
topsvcs SYSJ2 SYSJ2 SYSJ2 SYSJ2 SYSJ2 SYSJ2 SYSJ2 SYSJ2 SYSJ2 SYSJ2
EAA3D429
0710061007 U S LVDD
IDENTIFIER TIMESTAMP T C RESOURCE_NAME 12337A8D 0723152107 T S DR_KER_MEM for DR rem
PHYSICAL PARTITION MARKED STALE DESCRIPTION Affected memory not available
Some notes on disk related errors: ---------------------------------DISK_ERR4 is bad block relocation. Not a serious error. DISK_ERR2 is a hardware error as opposed to a media or corrected read error on disk. This could be serious. EAA3D429
0121151108 U S LVDD
PHYSICAL PARTITION MARKED STALE
Note 1: ------thread 1: Q: Has anyone seen these errors before? We're running 6239 fc cards on a CX600. AIX level is 52-03 with the latest patches for devices.pci.df1000f7 as well. I didn't know that these adapters still used devices.pci.df1000f7 as part of their device driver set, but aparently they do. We're mostly seeing ERR4s on bootup and occassionaly throughout the day. They're TEMP but should I be concerned about this? Any help would be greatly appreciated! LABEL: SC_DISK_ERR4 IDENTIFIER: DCB47997 A: DISK_ERR_4 are simply bad-block relocation errors. They are quite normal. However, I heard that if you get more than 8 in an 8-hour period, you should get the disk replaced as it is showing signs of impending failure. thread 2: Q:
> Has anyone corrected this issue? SC_DISK_ERR2 with EMC Powerpath = > filesets listed below? I am using a CX-500.=20 > A: got those errors before using a CX700 and it turned out to be a firmware problem on the fibre adapter, model 6259. EMC recommended the 92X1 firmware and to find out IBM found problems with timeouts to the drives and recommended going back a level to 81X1. A: We have the same problem as well. EMC say its a firmware error on the FC adapters A: This is how to fix these errors, downgrading firware is not recommended. Correcting SCSI_DISK_ERR2's in the AIX Errpt Log - Navisphere Failover Wizard 1. In the Navisphere main screen, select tools and then click the Failover Setup Wizard. Click next to continue. 2. From the drop-down list select the host server you wish to modify and click next 3. Highlight the CX-500 and click next 4. Under the specify settings box be sure to select 1 for the failover setting and disable for array commpath. Click next to process. 5. The next screen is the opportunity to review your selections (host, failover mode and array commpath); click next to commit 6. The following screen displays a warning message to alert you are committing these changes. Click yes to process. 7. Next login to the AIX command prompt as root and perform the following commands to complete stopping the SCSI_DISK_ERR2. a. lsdev -Cc disk | grep LUNZ (Filter for disks with LUNZ in the description) b. rmdev -dl hdisk(#)'s (Note the disks and remove them from the ODM) c. errclear 0 (Clear the AIX system error log) d. cfgmgr -v (Attempt to re-add the LUNZ disks) e. lsdev -Cc disk | grep LUNZ (Double check to make sure the LUNZ disk does not add itself back to the system after the cfgmgr command) f. errpt -a
(Monitor the AIX error log to insure the SCSI_DISK_ERR2's are gone) Task Complete... E87EF1BE 0512150008 P O dumpcheck The largest dump device is too small. ----------------------------------------------------------------------------HACMP error: -----------LABEL: IDENTIFIER: Date/Time: Sequence Number: Machine Id: Node Id: Class: Type: Resource Name: Resource Class: Resource Type: Location:
LVM_GS_RLEAVE AB59ABFF Tue May 26 09:05:36 ZOM 2009 1149 00CC696E4C00 wijting U UNKN LIBLVM NONE NONE
Description Remote node Concurrent Volume Group failure detected Probable Causes Remote node Concurrent Volume Group forced offline Failure Causes Remote node left VGSA/VGDA groups due to failure Recommended Actions Examine error log on identified remote node Detail Data Remote Node Name vleet Volume Group ID 00CC 94EE 0000 4C00 0000 0111 4FE8 8651 MAJOR/MINOR DEVICE NUMBER 0045 0000 SENSE DATA 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
Problems with errpt:
-------------------Invalid log, or other problems thread 1: Q: Hello ... the 'errpt' Command tells me: 0315-180 logread: UNEXPECTED EOF 0315-171 Unable to process the error log file /var/adm/ras/errlog. 0315-132 The supplied error log is not valid: /var/adm/ras/errlog. # ls -l /var/adm/ras/errlog -rw-r--r-- 1 root system 0 Jun 14 17:31 /var/adm/ras/errlog How can I fix this problem? A: /usr/lib/errstop
# stop logging
rm /var/adm/ras/errlog
# get rid of that log.
/usr/lib/errdemon log.
# restart the daemon, creating a new error
Some err identifiers that can sometimes be hard to trace to their true sources: ======================================================================= ======== Take a look at those errpt entries: ------------------------------------------------------------------------ERRPT ENTRY 1: -------------LABEL: IDENTIFIER: Date/Time: Sequence Number: Machine Id: Node Id: Class: Type: Resource Name:
CORE_DUMP C69F5C9B Thu Jan 15 02:00:45 MET 2009 999 00CC94EE4C00 srv1 S PERM SYSPROC
Description SOFTWARE PROGRAM ABNORMALLY TERMINATED Probable Causes SOFTWARE PROGRAM User Causes USER GENERATED SIGNAL Recommended Actions CORRECT THEN RETRY Failure Causes SOFTWARE PROGRAM Recommended Actions RERUN THE APPLICATION PROGRAM IF PROBLEM PERSISTS THEN DO THE FOLLOWING CONTACT APPROPRIATE SERVICE REPRESENTATIVE Detail Data SIGNAL NUMBER 11 USER'S PROCESS ID: 1298680 FILE SYSTEM SERIAL NUMBER 57 INODE NUMBER 37134 CORE FILE NAME /var/core/core.1298680.15010044 PROGRAM NAME BS_sear STACK EXECUTION DISABLED 0 COME FROM ADDRESS REGISTER PROCESSOR ID hw_fru_id: 1 hw_cpu_id: 9 ADDITIONAL INFORMATION ?? ?? Unable to generate symptom string. (or as another example of the last lines, where you can see the "program name") PROGRAM NAME opmn STACK EXECUTION DISABLED 0 COME FROM ADDRESS REGISTER
PROCESSOR ID hw_fru_id: 0 hw_cpu_id: 2 ADDITIONAL INFORMATION strlen 0 pmStrdup 14 Symptom Data REPORTABLE 1 INTERNAL ERROR 0 SYMPTOM CODE PCSS/SPI2 FLDS/opmn SIG/11 FLDS/strlen VALU/0 FLDS/pmStrdup
------------------------------------------------------------------------POSSIBLE EXPLANATION: ===================== http://publib.boulder.ibm.com/infocenter/systems/index.jsp? topic=/com.ibm.aix.security/doc/security/stack_exec_disable.htm AIXr has enabled the stack execution disable (SED) mechanism to disable the execution of code on a stack and select data areas of a process. By disabling the execution and then terminating, an infringing program, the attacker is prevented from gaining root user privileges through a buffer overflow attack. While this feature does not stop buffer overflows, it provides protection by disabling the execution of attacks on buffers that have been overflowed. Beginning with the POWER4T family of processors, you can use a pagelevel execution enable and/or disable feature for the memory. The AIX SED mechanism uses this underlying hardware support for implementing a no-execution feature on select memory areas. Once this feature is enabled, the operating system checks and flags various files during the executable programs. It then alerts the operating system memory manager and the process managers that the SED is enabled for the process being created. The select memory areas are marked for no-execution. If any execution occurs on these marked areas, the hardware raises an exception flag and the operating system stops the corresponding process. The exception and application termination details are captured through the AIX error log events. SED is implemented mainly through the sedmgr command. The sedmgr command permits control
of the systemwide SED mode of operation as well as setting the executable file based SED flags. SED modes and monitoring The stack execution disable (SED) mechanism in AIXr is implemented through systemwide mode flags, as well as individual executable file-based header flags. While systemwide flags control the systemwide operation of the SED, file level flags indicate how files should be treated in SED. The buffer overflow protection (BOP) mechanism provides for four systemwide modes of operation: -- off The SED mechanism is turned off and no process is marked for SED protection. --select Only a select set of files are enabled and monitored for SED protection. The select set of files are chosen by reviewing the SED related flags in the executable program binary headers. The executable program header enables SED related flags to request to be included in the select mode. -- setidfiles Permits you to enable SED, not only for the files requesting such a mechanism, but all the important setuid and setgid system files. In this mode, the operating system not only provides SED for the files with the request SED flag set, but also enables SED for the executable files with the following characteristics (except the files marked for exempt in their file headers): .SETUID files owned by root .SETGID files with primary group as system or security -- all All executable programs loaded on the system are SED protected except for the files requesting an exemption from SED mode. Exemption related flags are part of the executable program headers. The SED feature on AIX also provides the ability to monitor instead of stopping the process when an exception happens. This systemwide control permits a system administrator to check for breakdowns and issues in the system environment by monitoring it before the SED is deployed in the production systems. The sedmgr command provides an option that permits you to enable SED to monitor files instead of stopping the processes when exceptions occur. The system administrator can evaluate whether an executable program is doing any legitimate stack execution. This setting works in conjunction with the systemwide mode set using the -c option. When the monitor mode is turned on, the system permits the process to continue operating even if an SED-related exception occurs. Instead of stopping the process,
the operating monitoring is the operating exception per
system logs the exception in the AIX error log. If SED off, system stops any process that violates and raises an SED facility.
Any changes to the SED mode systemwide flags requires that you restart the system for the changes to take effect. All of these types of events are audited. note: ===== Today, I found out why my LDAP installation is giving me core dump on a certain LPAR but not on the other LPAR, although both LPARs are "identically" built. If we look closely in the error log... ....CORE FILE NAME/ldap/filesets/corePROGRAM NAMEldapcfgSTACK EXECUTION DISABLED1COME FROM ADDRESS REGISTER.....we can see the "STACK EXECUTION DISABLED" under the program name. What this means is that the AIX system had the Stack Execution Disable protection turned ON. To confirm if its really turned ON, # sedmgrStack Execution Disable (SED) mode: allSED configured in kernel: all To change it, # sedmgr -m selectSystem wide SED has been set successfully. It is effective at 64 bit kernel boot time. Run sedmgr again (this step is not necessary), # sedmgrStack Execution Disable (SED) mode: selectSED configured in kernel: all We need to reboot the server for the change to take effect. # sedmgrStack Execution Disable (SED) mode: selectSED configured in kernel: select Now I can configure my LDAP with the mksecldap command flawlessly. Labels: AIX note: =====
------------------------------------------------------------------------ERRPT ENTRY 2: -------------LABEL: IDENTIFIER: Date/Time: Sequence Number: Machine Id: Node Id: Class: Type: Resource Name:
SRC E18E984F Fri Jan 16 09:31:33 MET 2009 1513 00C503AC4C00 heilbot S PERM SRC
Description SOFTWARE PROGRAM ERROR Probable Causes APPLICATION PROGRAM Failure Causes SOFTWARE PROGRAM Recommended Actions PERFORM PROBLEM RECOVERY PROCEDURES Detail Data SYMPTOM CODE 0 SOFTWARE ERROR CODE -9053 ERROR CODE 2 DETECTING MODULE 'tellsrc.c'@line:'87' FAILING MODULE Duplicates Number of duplicates 3 Time of first duplicate Fri Jan 16 09:31:18 MET 2009 Time of last duplicate Fri Jan 16 09:31:33 MET 2009 POSSIBLE EXPLANATIONS: ====================== In entry 2, we see the identifier E18E984F, and "SOFTWARE ERROR CODE -9053", and "Detecting module tellsrc.c@line:87". tellsrc.c'@line:'87'
http://www-01.ibm.com/support/docview.wss?uid=isg1IZ03064 IZ03064: VARYONVG -C FAILS WITH "GSCHILD:CANNOT REGISTER WITH DRIVER APPLIES TO AIX 5300-07 APAR status Closed as program error. Error description "varyonvg -c" fails to varyon concurrent volume group and reports the following error message: tellclvmd: request failed rc = -9014 [UNKNOWN rc] 0516-1334 varyonvg: The command /usr/sbin/tellclvmd returned an error. errpt logs following entry: LABEL: IDENTIFIER: Class: Type: Resource Name:
SRC E18E984F S PERM SRC
Description SOFTWARE PROGRAM ERROR Probable Causes APPLICATION PROGRAM Failure Causes SOFTWARE PROGRAM Recommended Actions PERFORM PROBLEM RECOVERY PROCEDURES Detail Data SYMPTOM CODE 0 SOFTWARE ERROR CODE -9053 ERROR CODE 74 DETECTING MODULE 'srcmstr.c'@line:'529' FAILING MODULE Local fix This problem occurs when multiple "varyonvg -nc" commands are performed together. By serializing these commands, this can be avoided. Problem summary Multiple varyonvg -c processes will all create threads in the gsclvmd daemon. With certain timing, these threads can interfere with eachothers global variables and possibly cause varyonvg to fail.
Problem conclusion Privatize variables so mutliple vgs coming online can't interfere with eachother. Temporary fix Comments 5200-10 - use AIX APAR IZ05735 5300-06 - use AIX APAR IZ02334 5300-07 - use AIX APAR IZ03064 APAR information APAR number IZ03064 Reported component name AIX 5.3 Reported component ID 5765G0300 Reported release 530 Status CLOSED PER PE NoPE HIPER NoHIPER Submitted date 2007-08-14 Closed date 2007-09-04 Last modified date 2007-12-06 APAR is sysrouted FROM one or more of the following: APAR is sysrouted TO one or more of the following: Publications Referenced Fix information Fixed component name AIX 5.3 Fixed component ID 5765G0300 error INTRPPC_ERR: -----------------LABEL: IDENTIFIER: Date/Time: Sequence Number: Machine Id: Node Id: Class: Type: Resource Name: Resource Class: Resource Type: Location:
INTRPPC_ERR 853015D6 Sun Mar 22 00:27:49 MET 2009 1515 00C503AC4C00 starboss H UNKN sysplanar0 planar sysplanar_rspc
Description UNDETERMINED ERROR Probable Causes SYSTEM I/O BUS SOFTWARE PROGRAM ADAPTER DEVICE
Recommended Actions PERFORM PROBLEM DETERMINATION PROCEDURES Detail Data BUS NUMBER 9001 00C0 INTERRUPT LEVEL 0009 0001 Number of Occurrences 0000 0001 Possible explanations: ---------------------thread 1: IY58847: INTRPPC_ERR ERRORS IN ERROR LOGS A fix is available Download fix packs
APAR status Closed as program error. Error description INTRPPC_ERR errors were observed in the error log while customer ran a testcase as mentioned in the defect. Local fix Problem summary INTRPPC_ERR errors were observed in the error log while customer ran a testcase, which brings up and down the phxentdd interface in a infinite loop. A ping is executed using the ip address associated with this interface. Problem conclusion A simple code change to ignore the interrupts while driver is in closing state. Temporary fix Comments APAR information APAR number IY58847 Reported component name AIX 5L FOR POWE Reported component ID 5765E6100 Reported release 510 Status CLOSED PER PE NoPE HIPER NoHIPER Submitted date 2004-07-13 Closed date 2004-07-13 Last modified date 2004-10-29 APAR is sysrouted FROM one or more of the following:
APAR is sysrouted TO one or more of the following: Publications Referenced Fix information Fixed component name AIX 5L FOR POWE Fixed component ID 5765E6100 Applicable component levels R510 PSY U477721 UP04/10/29 I 1000 thread 2: > > I've recently started getting INTRPCC_ERR's on an old (but important!) > > aix 4.3 box. They dont seem to correspond to anything else and the > > box seems to be working normally. I found a way to lookup the BUS > > NUMBER via odmget -q value= CuAt, but that didn't return anything > > for me. Also looking for the interrupt number via lsresource didn't > > give any matches either. And diag/Advanced Diagnostics/Problem > > Determination didn't find any trouble. > > > Any other suggestions on how to track this down? > > > Thanks, > > > LABEL: INTRPPC_ERR > > IDENTIFIER: DADF69E4 > > > Date/Time: Wed Jul 11 08:51:41 > > Sequence Number: 735309 > > Machine Id: 000247824C00 > > Node Id: scully > > Class: H > > Type: UNKN > > Resource Name: SYSINTR > > Resource Class: NONE > > Resource Type: NONE > > Location: NONE > > > Description > > UNDETERMINED ERROR > > > Probable Causes > > SYSTEM I/O BUS > > SOFTWARE PROGRAM > > ADAPTER > > DEVICE > > > Recommended Actions > > PERFORM PROBLEM DETERMINATION PROCEDURES > > > Detail Data
> > > > > > >
> > > >
BUS NUMBER 0000 00C0 INTERRUPT LEVEL 0000 0005
convert the bus number from hex, then look for that value in `ls -l / dev`
.... but it is more than likely a device driver issue rather than the device itself. thread 3: Here's how to map the error information to a specific adapter. Let's do that first. >Detail Data >BUS NUMBER >0000 00C0 >INTERRUPT LEVEL >0000 0005 Example: Detail Data BUS NUMBER 0000 00C0 INTERRUPT LEVEL 0000 0003 lsresource -al pci0 | grep 0x000000C0 --> O pci0 0x8d5c_5 0x000000c0 - 0x000000df lsresource -al pci0 | grep 3 | grep bus_intr_lvl --> N sa1 bus_intr_lvl 3 Note: lsresource command example: --------------------------------selalbe@starboss:/home/beab_krn/selalbe $ lsresource -al pci0 TYPE DEVICE ATTRIBUTE S G CURRENT B pci0 0xda40_1 0x0000000080080000 - 0x00000000800bffff B pci0 0xdfa8_1 0x0000000080000000 - 0x000000008003ffff B ent0 busmem 0x0000000080120000 - 0x000000008013ffff B ent0 rom_mem 0x00000000800c0000 - 0x00000000800fffff B ent1 busmem 0x0000000080100000 - 0x000000008011ffff B ent1 rom_mem 0x0000000080040000 - 0x000000008007ffff O pci0 0xda40_0 0x00000000000df800 - 0x00000000000df83f O pci0 0xdfa8_0 0x00000000000dfc00 - 0x00000000000dfc3f I ent0 busintr 249 (A1) I ent1 busintr 250 (A1)
------------------------------------------------------------------------5561971C ECH_CANNOT_SET_CLBK in errpt. Most of the time, this error can be ignored errpt command shows: ECH_CANNOT_SET_CLBK for etherchannel virtual i/o network adapters. Technote (FAQ) Question Why is errpt showing: ECH_CANNOT_SET_CLBK error for an etherchannel that is configured in Network Interface Backup mode using virtual i/o network adapters.
Answer It is indeed true that ECH_CANNOT_SET_CLBK error entry is expected and can be ignored. This error is logged because the virtual ethernet does not support an ioctl currently. This was the comment by the developers on this error: "This just means that currently virtual Ethernets do not support the code to detect when their link status is down. Since virtual Ethernets are not physical and hardly if ever experience a "link down" event, this is just an informative message and can be safely ignored. Most importantly is to verify that the Virtual i/o adapter etherchannel has a "internet address to ping" to verify this: #lsattr -El entX (where entX is the etherchannel) if it is not set, it can be done using smitty: #smitty etherchannel select entX select option: internet address to ping. ------------------------------------------------------------------------ERRPT entry: ============
LABEL: IDENTIFIER: Date/Time: Sequence Number: Machine Id: Node Id: Class: Type: Resource Name: Resource Class: Resource Type: Location:
TS_LATEHB_PE 3C81E43F Sat Mar 28 16:30:18 MET 2009 1149 00CC94EE4C00 vleet U PERF topsvcs NONE NONE
Description Late in sending heartbeat Probable Causes Heavy CPU load Severe physical memory shortage Heavy I/O activities Failure Causes Daemon can not get required system resource Recommended Actions Reduce the system load Detail Data DETECTING MODULE rsct,bootstrp.C,1.213,4835 ERROR ID 6zESUw.88Yn7/UKN0Nr9GF0................... REFERENCE CODE A heartbeat is late by the following number of seconds 86
diag command: ------------Whenever a hardware problem occurs in AIX, use the diag command to diagnose the problem. The diag command is the starting point to run a wide choice of tasks and service aids. Most of the tasks/service aids are platform specific. To run diagnostics on the scdisk0 device, without questions, enter: # diag -d scdisk0 -c
System dumps: ------------A system dump is created when the system has an unexpected system halt or system failure. In AIX 5L the default dump device is /dev/hd6, which is also the default paging device. You can use the sysdumpdev command to manage system crash dumps. The sysdumpdev command changes the primary or secondary dump device designation in a system that is running. The primary and secondary dump devices are designated in a system configuration object. The new device designations are in effect until the sysdumpdev command is run again, or the system is restarted. If no flags are used with the sysdumpdev command, the dump devices defined in the SWservAt ODM object class are used. The default primary dump device is /dev/hd6. The default secondary dump device is /dev/sysdumpnull. Examples To display current dump device settings, enter: sysdumpdev -l To designate logical volume hd7 as the primary dump device, enter: sysdumpdev -p /dev/hd7 To designate tape device rmt0 as the secondary dump device, enter: sysdumpdev -s /dev/rmt0 To display information from the previous dump invocation, enter: sysdumpdev -L To permanently change the database object for the primary dump device to /dev/newdisk1, enter: sysdumpdev -P -p /dev/newdisk1 To determine if a new system dump exists, enter: sysdumpdev -z If a system dump has occurred recently, output similar to the following will appear: 4537344 /dev/hd7 To designate remote dump file /var/adm/ras/systemdump on host mercury for a primary dump device, enter: sysdumpdev -p mercury:/var/adm/ras/systemdump A : (colon) must be inserted between the host name and the file name. To specify the directory that a dump is copied to after a system crash, if the dump device is /dev/hd6, enter: sysdumpdev -d /tmp/dump
This attempts to copy the dump from /dev/hd6 to /tmp/dump after a system crash. If there is an error during the copy, the system continues to boot and the dump is lost. To specify the directory that a dump is copied to after a system crash, if the dump device is /dev/hd6, enter: sysdumpdev -D /tmp/dump This attempts to copy the dump from /dev/hd6 to the /tmp/dump directory after a crash. If the copy fails, you are prompted with a menu that allows you to copy the dump manually to some external media. Starting a system dump: ----------------------If you have the Software Service Aids Package installed, you have access to the sysdumpstart command. You can start the system dump by entering: # sysdumpstart -p You can also use: # smit dump Notes regarding system dumps: ----------------------------note 1: ------The_Nail wrote: > I handle several AIX 5.1 servers and some of them warns me (via errpt) > about a lack of disk space for the dumpcheck ressource. > Here is a copy of the message : > > > > > > > > > > > > > > >
Description The copy directory is too small. Recommended Actions Increase the size of that file system. Detail Data File system name /var/adm/ras Current free space in kb 7636 Current estimated dump size in kb 207872
> I guess /dev/hd6 is not big enough to contain a system dump. So how > can i change that?
The error message tells you something else. Read it, and you will understand! > How can i configure a secondary susdump space in case the primary > would be unavailable? sysdumpdev -s /dev/whatever > What does "copy directory /var/adm/ras" mean? That's where the crash dump will be put when you reboot after the crash. /dev/hd6 will be needed for other purposes (paging space), so you cannot keep your system dump there. And that file system is too small to contain the dump, that's the meaning of the error message. You have two options: - increase the /var file system (it should have ample free space anyway). - change the dump directory to something where you have more space: sysdumpdev -D /something/in/rootvg/with/free/space Yours, Laurenz Albe Note 2: ------Suppose you find the following error: $ errpt IDENTIFIER TIMESTAMP T C RESOURCE_NAME F89FB899 0822150005 P O dumpcheck
DESCRIPTION The copy directory is too small
This message is the result of a dump device check. You can fix this by increasing the size of your dump device. If you are using the default dump device (/dev/hd6) then increase your paging size or go to smit dump and "select System Dump Compression". Myself, I don't like to use the default dump device so I create a sysdumplv and make sure I have enough space. To check space needed go to smit dump and select "Show Estimated Dump Size" this will give you an idea about the size needed.
The copy directory is whatever sysdumpdev says it is. Run sysdumpdev and you will get something like #sysdumpdev primary /dev/hd6 secondary /dev/sysdumpnull copy directory /var/adm/ras forced copy flag TRUE always allow dump FALSE dump compression ON # sysdumpdev -e 0453-041 Estimated dump size in bytes: 57881395 Divide this number by 1024. This is the free space that is needed in your copy directory. Compare it to a df -k or divide this number by 512. This is the free space that is needed in your copy directory. Compare it to a df
Note 2: ------Suppose you find the following error: selalbe@wijting:/home/beab_krn/selalbe $ errpt IDENTIFIER TIMESTAMP T C RESOURCE_NAME DESCRIPTION E87EF1BE 0309150009 P O dumpcheck The largest dump device is too small. thread: do sysdumpdev -l you should see both primary and secondary dump devices from this you need to ensure that these are big enough to hold a system dump so type sysdumpdev -e to get an estimate on the dump size and resize your dump devices accordingly. Try to increase these above the value you have if it is a new system allow for growth of the system and give it plenty of space if possible thread:
IZ05158: POSSIBLE SYSTEM CRASH AFTER PCI BUS ERROR AFFECTING FC ADAPTER APPLIES TO AIX 5300-06
A fix is available Obtain fix for this APAR
APAR status Closed as program error. Error description A system crash can occur if a Fibre Channel adapter suffers PCI bus errors around the time of an adapter reset. Adapter resets are performed at adapter configuration time or as recovery from certain severe errors. Adapter resets are not performed as a result of normal SAN or storage communications failures. End device or SAN problems would not expose this problem. PCI bus errors are usually accompanied by errors with label "PCI_RECOVERABLE_ERR" in the AIX error log. Systems logging those errors may be susceptible to this problem. The stack traceback of the crash as seen in kdb will be similar to the following, and will include efc_finish_read_rev_mb() calling efc_read_reg(): (2)> f pvthread+800000 STACK: [03CE5678]efc_read_reg+000048 () [03CFF738]efc_finish_read_rev_mb+000290 () [03CF1154]efc_adap_post_trb+00049C () [00031F3C]clock+00017C () [000DF250]i_softmod+00027C () [000DE928].finish_interrupt+000024 () Local fix Respond promptly to any PCI_RECOVERABLE_ERR errors that appear in the AIX error log. It will not be possible to respond quickly enough to prevent a crash associated with a particular error, but addressing the underlying problem causing repeated PCI errors will lessen the risk to the environment. Problem summary A system crash can occur if a Fibre Channel adapter suffers PCI bus errors around the time of an adapter reset. Problem conclusion The handling of possible Errorneous value read around EEH event introduced in the relevant code path. Temporary fix Comments 5300-06 - use AIX APAR IZ05158 5300-07 - use AIX APAR IZ25395 5300-08 - use AIX APAR IZ24657 5300-09 - use AIX APAR IZ19062 6100-00 - use AIX APAR IZ23541
6100-01 - use AIX APAR IZ19769 6100-02 - use AIX APAR IZ19384 APAR information APAR number IZ05158 Reported component name AIX 5.3 Reported component ID 5765G0300 Reported release 530 Status CLOSED PER PE NoPE HIPER NoHIPER Submitted date 2007-09-21 Closed date 2008-03-18 Last modified date 2008-11-17 APAR is sysrouted FROM one or more of the following: APAR is sysrouted TO one or more of the following: IZ19062 IZ19384 IZ19769 IZ23541 IZ24657 IZ25395 Publications Referenced Fix information Fixed component name AIX 5.3 Fixed component ID 5765G0300 Applicable component levels R530 PSY U816768 UP08/07/24 I 1000 PTF to Fileset Mapping U816768 devices.pci.df1000f7.com 5.3.0.68
22. Diagnostic output: ====================== 0:Standard input
1: Standard output
2: Diagnostic output
redirect diag. outp. to file # cat somefile nofile 2>errfile # cat somefile nofile > outfile 2>errfile redirect diag. outp. to same place as standard outp. # cat firsthalf secondhalf > composite 2>1& 23. DOS2UNIX: ============= If you want to convert a ascii PC file to unix, you can use many tools like tr etc..
# tr -d '\r' < original.file > new.file # tr -d '\015' < original.file > new.file Or scripts like: #!/bin/sh perl -p -i -e 'BEGIN { print "Converting DOS to UNIX.\n" ; } END { print "Done.\n" ; } s/\r\n$/\n/' $* perl -p -i.bak -e 's/^\r+//;s/\r+$//;s/\r/\n/gs' file Or, on many unixes You can use the utility " ^M Just type: dos2unix
dos2unix " to remove the [RETURN]
dos2unix [ -ascii ] [ -iso ] [ -7 ] originalfile convertedfile -ascii Removes extra carriage returns and converts end of file characters in DOS format text files to conform to SunOS requirements. -iso This is the default. It converts characters in the DOS extended character set to the corresponding ISO standard characters. -7 Convert 8 bit DOS graphics characters to 7 bit space characters so that SunOS can read the file. #!/bin/sh # a script if test -f then tr rm mv fi
to strip carriage returns from DOS text files $1 -d '\r' <$1 >$.tmp $1 $.tmp $1
# tr -d '\015' < original.file > new.file or this works well on most shells: # tr -d '\r' < original.file > new.file Note: Other formats on AIX: --------------------------1. nvdmetoa command: How to convert EBCDIC files to ASCII: On your AIX system, the tool nvdmetoa might be present. Examples:
nvdmetoa
>AIXver3.dat
Converts an EBCDIC file taken off an AS400 and converts to an ASCII file for the pSeries or RS/6000 nvdmetoa 132
>AIXver3.txt
Converts an EBCDIC file with a record length of 132 characters to an ASCII file with 132 bytes per line PLUS 1 byte for the linefeed character. 2. od command: The od command translate a file into other formats, like for example hexadecimal format. To translate a file into several formats at once, enter: # od -t cx a.out > a.xcd This command writes the contents of the a.out file, in hexadecimal format (x) and character format (c), into the a.xcd file.
24. Secure shell connections: ============================= ssh: ==== What is Open Secure Shell? Open Secure Shell (OpenSSH) is an open source version of the SSH protocol suite of network connectivity tools. The tools provide shell functions that are authenticated and encrypted. A shell is a command language interpreter that reads input from a command line string, stdin or a file. Why use OpenSSH? When you're running over unsecure public networks like the Internet, you can use the SSH command suite instead of the unsecure commands telnet, ftp, and r-commands. OpenSSH delivers code that communicates using SSH1 and SSH2 protocols. What's the difference? The SSH2 protocol is a re-write of SSH1. SSH2 contains separate, layered protocols, but SSH1 is one large set of code. SSH2 supports both RSA & DSA keys, but SSH1 supports only RSA, and SSH2 uses a strong crypto integrity check, where SSH1 uses a CRC-32 check. The Internet Engineering Task Force (IETF) maintains the secure shell standards.
Example 1:
---------Go to a terminal on your local Unix system (Solaris, Linux, Mac OS X, etc.) and type the following command: ssh -l username acme.gatech.edu Replace "username" with your Prism ID. If this is your first time connecting to acme, you will see a warning similar to this: The authenticity of host 'acme.gatech.edu (130.207.165.23)' can't be established. DSA key fingerprint is 72:ce:63:c5:86:3a:cb:8c:cb:43:6c:da:00:0d:4c:1f. Are you sure you want to continue connecting (yes/no)? Type the word "yes" and hit . You should see the following warning: Warning: Permanently added 'acme.gatech.edu,130.207.165.23' (DSA) to the list of known hosts. Next, you will be prompted for your password. Type your password and hit . Example 2: ---------A secure shell 'terminal': # ssh -l oracle 193.172.126.193 # ssh [email protected] pscp: ===== Example to Copy a file to a remote unix server: # pscp c:\documents\foo.txt [email protected] :/tmp/foo To receive (a) file(s) from a remote server: pscp [options] [user@]host:source target So to copy the file /etc/hosts from the server example.com as user fred to the file c:\temp\example-hosts.txt, you would type: pscp [email protected] :/etc/hosts c:\temp\example-hosts.txt To send (a) file(s) to a remote server: pscp [options] source [source...] [user@]host:target
So to copy the local file c:\documents\foo.txt to the server example.com as user fred to the file /tmp/foo you would type: pscp c:\documents\foo.txt [email protected] :/tmp/foo You can use wildcards to transfer multiple files in either direction, like this: pscp c:\documents\*.doc [email protected] :docfiles pscp [email protected] :source/*.c c:\source Example of scripts using pscp with parameters; -----------------------------------@echo off REM Script om via pscp.exe een bestand van een UNIX systeem te copi %ren naar het werkstation. Echo Copy bestand van unix naar werkstation SET SET SET SET
/P /P /P /P
systemname=Geef volledige systeemnaam: remotefile=Geef UNIX path+filename: localfile=Geef local filename: username=Geef username:
echo pscp.exe %username%@%systemname%:%remotefile% %localfile% pscp.exe %username%@%systemname%:%remotefile% %localfile% echo bestand %remotefile% gecopieerd naar %localfile% pause -----------------------------------@echo off REM Script om via pscp.exe een bestand naar een UNIX systeem te copi %ren van het werkstation. Echo Copy bestand van werkstation naar unix SET SET SET SET
/P /P /P /P
systemname=Geef volledige systeemnaam: localfile=Geef local filename: remotefile=Geef UNIX path+filename: username=Geef username:
echo pscp.exe %localfile% %username%@%systemname%:%remotefile% pscp.exe %localfile% %username%@%systemname%:%remotefile% echo bestand %localfile% gecopieerd naar %remotefile% pause -----------------------------------scp: ====
Scp is a utility which allows files to be copied between machines. Scp is an updated version of an older utility named Rcp. It works the same, except that information (including the password used to log in) is encrypted. Also, if you have set up your .shosts file to allow you to ssh between machines without using a password as described in help on setting up your .shosts file, you will be able to scp files between machines without entering your password. Either the source or the destination may be on the remote machine; i.e., you may copy files or directories into the account on the remote system OR copy them from the account on the remote system into the account you are logged into. Example: # scp conv1.tar.gz [email protected] :/backups/520backups/splenvs # scp conv2.tar.gz [email protected] :/backups/520backups/splenvs Example: # scp myfile xyz@sdcc7:myfile Example: To copy a directory, use the -r (recursive) option. # scp -r mydir xyz@sdcc7:mydir Example: cd /oradata/arc /usr/local/bin/scp *.arc
SPRAT:/oradata/arc
Example: While logged into xyz on sdcc7, copy file "letter" into file "application" in remote account abc on sdcc3: % scp letter abc@sdcc3:application While logged into abc on sdcc3, copy file "foo" from remote account xyz on sdcc7 into filename "bar" in abc: % scp xyz@sdcc7:foo bar
passwordless using ssh/scp between two (or more) hosts: ======================================================= 1. decide which useraccount to use (on all hosts), and logon to the local host with that account 2. Generate a public/private key pair on the local machine ssh-keygen -t dsa dsa=protocol 2)
(or 'ssh-keygen -t rsa')
In response, you should see:
(rsa=protocol 1;
Generating public/private dsa key pair Enter file in which to save the key ... Press Enter to accept this. In response, you should see: Enter passphrase (empty for no passphrase): You don't need a passphrase, so press Enter twice. In response, you should see: Your identification has been saved in ... Your public key has been saved in ... 3. Note the name and location of the public key just generated. It always ends in .pub. 4. Change the example chmod In effect, access to the
permissions of the generated .pub file to 600, for 600 id_dsa.pub (or 700). make sure that no group, or everyone (world), has any file.
5. Copy the public key just generated to all of your remote boxes. You can use scp or FTP or whatever to make the copy. if you are logging in as a user, for example, albert, you should copy it to "/home/albert/.ssh/authorized_keys". But (!) first check whether that file already exists. If the file already exists and contains text, you need to append the contents of your public key file to what already is there. That should do the job. Now you can use statements like, for example albert@hosta:/tmp$> scp testfile albert@hostb:/tmp without being prompted for a password. If you want to do the same for scp from hostb to hosta, perform the same steps again, but now ofcourse with the serverroles reversed. Notes: 1. If it doesn't work, try changing the authorized_keys file name to authorized_keys2, or ask your system administrator what file name is ssh actually using. 2. The name of the target server must have been registered in the "known_hosts" file in the .ssh directory.
This can be done with a regular (with password) ssh connection, and accepting the host "as known". 3. SSH protocol 2 is assumed in this procedure (it uses dsa keys). If your ssh configuration not uses this as a default, you may have to force it with the -2 option of the ssh and scp.
ssh on AIX: =========== After you download the OpenSSL package, you can install OpenSSL and OpenSSH. Install the OpenSSL RPM package using the geninstall command: # geninstall -d/dev/cd0 R:openssl-0.9.6m Output similar to the following displays: SUCCESSES --------openssl-0.9.6m-3 Install the OpenSSH installp packages using the geninstall command: # geninstall -I"Y" -d/dev/cd0 I:openssh.base Use the Y flag to accept the OpenSSH license agreement after you have reviewed the license agreement. (Note: we have seen this line as well: # geninstall -Y -d/dev/cd0 I:openssh.base) Output similar to the following displays: Installation Summary -------------------Name Level Part Event Result -----------------------------------------------------------------------------openssh.base.client 3.8.0.5200 USR APPLY SUCCESS openssh.base.server 3.8.0.5200 USR APPLY SUCCESS openssh.base.client 3.8.0.5200 ROOT APPLY SUCCESS openssh.base.server 3.8.0.5200 ROOT APPLY SUCCESS You can also use the SMIT install_software fast path to install OpenSSL and OpenSSH. The following OpenSSH binary files are installed as a result of the preceding procedure: scp File copy program similar to rcp
sftp Program similar to FTP that works over SSH1 and SSH2 protocol sftp-server SFTP server subsystem (started automatically by sshd daemon) ssh Similar to the rlogin and rsh client programs ssh-add Tool that adds keys to ssh-agent ssh-agent An agent that can store private keys ssh-keygen Key generation tool ssh-keyscan Utility for gathering public host keys from a number of hosts ssh-keysign Utility for host-based authentication ssh-rand-helper A program used by OpenSSH to gather random numbers. It is used only on AIX 5.1 installations. sshd Daemon that permits you to log in The following general information covers OpenSSH: The /etc/ssh directory contains the sshd daemon and the configuration files for the ssh client command. The /usr/openssh directory contains the readme file and the original OpenSSH open-source license text file. This directory also contains the ssh protocol and Kerberos license text. The sshd daemon is under AIX SRC control. You can start, stop, and view the status of the daemon by issuing the following commands: startsrc -s sshd stopsrc -s sshd lssrc -s sshd
OR startsrc -g ssh OR stopsrc -g ssh OR lssrc -s ssh
(group)
Automatic startup of sshd on boot: ---------------------------------For example, on AIX create the following script "Sssh" in /etc/rc.d/rc2.d root@zd110l14:/etc/rc.d/rc2.d#cat Ssshd #!/bin/ksh ################################################## # name: Ssshd # purpose: script that will start or stop the sshd daemon. ################################################## case "$1" in start ) startsrc -g ssh ;; stop ) stopsrc -g ssh ;; * ) echo "Usage: $0 (start | stop)" exit 1
esac
25. Pipelining and Redirecting: =============================== CONCEPT: UNIX allows you to connect processes, by letting the standard output of one process feed into the standard input of another process. That mechanism is called a pipe. Connecting simple processes in a pipeline allows you to perform complex tasks without writing complex programs. EXAMPLE: Using the more command, and a pipe, send the contents of your .profile and .shrc files to the screen by typing cat .profile .shrc | more to the shell. EXERCISE: How could you use head and tail in a pipeline to display lines 25 through 75 of a file? ANSWER: The command cat file | head -75 | tail -50 would work. The cat command feeds the file into the pipeline. The head command gets the first 75 lines of the file, and passes them down the pipeline to tail. The tail command then filters out all but the last 50 lines of the input it received from head. It is important to note that in the above example, tail never sees the original file, but only sees the part of the file that was passed to it by the head command. It is easy for beginners to confuse the usage of the input/output redirection symbols < and >, with the usage of the pipe. Remember that input/output redirection connects processes with files, while the pipe connects processes with other processes. Grep The grep utility is one of the most useful filters in UNIX. Grep searches line-by-line for a specified pattern, and outputs any line that matches the pattern. The basic syntax for the grep command is grep [-options] pattern [file]. If the file argument is omitted, grep will read from standard input. It is always best to enclose the pattern within single quotes, to prevent the shell from misinterpreting the command. The grep utility recognizes a variety of patterns, and the pattern specification syntax was taken from the
vi editor. Here are some of the characters you can use to build grep expressions: The carat (^) matches the beginning of a line. The dollar sign ($) matches the end of a line. The period (.) matches any single character. The asterisk (*) matches zero or more occurrences of the previous character. The expression [a-b] matches any characters that are lexically between a and b. EXAMPLE: Type the command grep 'jon' /etc/passwd to search the /etc/passwd file for any lines containing the string "jon". EXAMPLE: Type the command grep '^jon' /etc/passwd to see the lines in /etc/passwd that begin with the character string "jon". EXERCISE:List all the files in the /tmp directory owned by the user root. EXPLANATION: The command ls -l /tmp | grep 'root' would show all processes with the word "root" somewhere in the line. That doesn't necessarily mean that all the process would be owned by root, but using the grep filter can cut the down the number of processes you will have to look at. Redirecting: -----------CONCEPT: Every program you run from the shell opens three files: Standard input, standard output, and standard error. The files provide the primary means of communications between the programs, and exist for as long as the process runs. The standard input file provides a way to send data to a process. As a default, the standard input is read from the terminal keyboard. The standard output provides a means for the program to output data. As a default, the standard output goes to the terminal display screen. The standard error is where the program reports any errors encountered during execution. By default, the standard error goes to the terminal display.
CONCEPT: A program can be told where to look for input and where to send output, using input/output redirection. UNIX uses the "less than" and "greater than" special characters (< and >) to signify input and output redirection, respectively. Redirecting input Using the "less-than" sign with a file name like this: < file1 in a shell command instructs the shell to read input from a file called "file1" instead of from the keyboard. EXAMPLE:Use standard input redirection to send the contents of the file /etc/passwd to the more command: more < /etc/passwd Many UNIX commands that will accept a file name as a command line argument, will also accept input from standard input if no file is given on the command line. EXAMPLE: To see the first ten lines of the /etc/passwd file, the command: head /etc/passwd will work just the same as the command: head < /etc/passwd Redirecting output Using the "greater-than" sign with a file name like this: > file2 causes the shell to place the output from the command in a file called "file2" instead of on the screen. If the file "file2" already exists, the old version will be overwritten. EXAMPLE: Type the command ls /tmp > ~/ls.out to redirect the output of the ls command into a file called "ls.out" in your home directory. Remember that the tilde (~) is UNIX shorthand for your home directory. In this command, the ls command will list the contents of the /tmp directory. Use two "greater-than" signs to append to an existing file. For example: >> file2 causes the shell to append the output from a command to the end of a file called "file2". If the file "file2" does not already exist, it will be created.
EXAMPLE: In this example, I list the contents of the /tmp directory, and put it in a file called myls. Then, I list the contents of the /etc directory, and append it to the file myls: ls /tmp > myls ls /etc >> myls Redirecting error Redirecting standard error is a bit trickier, depending on the kind of shell you're using (there's more than one flavor of shell program!). In the POSIX shell and ksh, redirect the standard error with the symbol "2>". EXAMPLE: Sort the /etc/passwd file, place the results in a file called foo, and trap any errors in a file called err with the command: sort < /etc/passwd > foo 2> err
=========================== 27. UNIX DEVICES and mknod: =========================== 27.1 Note 1: ============ the files in the /dev directory are a little different you may be used to in other operating systems. The very first thing to understand is that these files drivers for the devices. Drivers are in the kernel itself (/unix etc..), and the files in /dev contain anything at all: they are just pointers to where the driver code can be kernel. There is nothing more to it than that. These aren't programs, they aren't drivers, pointers.
from anything are NOT the do not actually found in the they are just
That also means that if the device file points at code that isn't in the kernel, it obviously is not going to work. Existence of a device file does not necessarily mean that the device code is in the kernel, and creating a device file (with mknod) does NOT create kernel code. Unix actually even shows you what the pointer is. When you do a long listing of a file in /dev, you may have noticed that there are two numbers where the file size should be: brw-rw-rw-
2 bin
bin
2, 64 Dec
8 20:41 fd0
That "2,64" is a pointer into the kernel. I'll explain more about this in a minute, but first look at some more files: brw-rw-rwbrw-rw-rwbrw-rw-rwbrw-rw-rwbrw-rw-rwbrw-rw-rw-
2 2 2 1 2 3
bin bin bin bin bin bin
bin bin bin bin bin bin
2, 2, 2, 2, 2, 2,
64 48 60 16 44 36
Dec Sep Feb Sep Sep Sep
8 15 12 15 15 15
20:41 16:13 10:45 16:13 16:13 16:13
fd0 fd0135ds15 fd0135ds18 fd0135ds21 fd0135ds36 fd0135ds9
A different kind of device would have a different major number. For example, here are the serial com ports: crw-rw-rwcrw-rw-rwcrw-rw-rwcrw-r--r--
1 1 1 1
bin root root uucp
bin root sys sys
5,128 5, 0 5,136 5, 8
Feb 14 05:35 tty1A Dec 9 13:13 tty1a Nov 25 07:28 tty2A Nov 25 07:16 tty2a
Notice the "b" and the "c" as the first characters in the mode of the file. It designates whether we have a block "b", or a character "c" device. Notice that each of these files shares the "5" part of the pointer, but that the other number is different. The "5" means that the device is a serial port, and the other number tells exactly which com port you are referring to. In Unix parlance, the 5 is the "major number" and the other is the "minor number". These numbers get created with a "mknod" command. For example, you could type "mknod /dev/myfloppy b 2 60" and then "/dev/myfloppy" would point to the same driver code that /dev/fd0135ds18 points to, and it would work exactly the same. This also means that if you accidentally removed /dev/fd0135ds18, you could instantly recreate it with "mknod". But if you didn't know that the magic numbers were "2,60", how could you find out? It turns out that it's not hard. First, have a look at "man idmknod". The idmknod command wipes out all non-required devices, and then recreates them. Sounds scary, but this gets called every time you answer "Y" to that "Rebuild Kernel environment?" question that follows relinking. Actually, on 5.0.4 and on, the existing /dev files don't get wiped out; the command simply recreates whatever it has to. idmknod requires several arguments, and you'd need to get them right to have success. You could make it easier by simply relinking a new kernel and answering "Y" to the "Rebuild" question, but that's using a fire hose to
put out a candle. A less dramatic method would be to look at the files that idmknod uses to recreate the device nodes. These are found in /etc/conf/node.d In this case, the file you want would be "fd". A quick look at part of that shows: fd fd fd fd fd fd fd
fd0 fd0135ds36 fd0135ds21 fd0135ds18 fd0135ds15 fd0135ds9 fd048
b b b b b b b
64 44 16 60 48 36 4
bin bin bin bin bin bin bin
bin bin bin bin bin bin bin
666 666 666 666 666 666 666
This gives you *almost* everything you need to know about the device nodes in the "fd" class. The only thing it doesn't tell you is the major number, but you can get that just by doing an "l" of any other fd entry: brw-rw-rw-
1 bin
bin
2, 60 Feb
5 09:45 fd096ds18
this shows you that the major number is "2". Armed with these two pieces of information, you can now do mknod chown chgrp chmod
/dev/fd0135ds18 b 2 60 bin /dev/fd0135ds18 bin /dev/fd0135ds18 666 /dev/fd0135ds18
If you examined the node file closely, you would also notice that /dev/rfd0135ds18 and /dev/fd0135ds18 differ only in that the "r" version is a "c" or character device and the other is "b" or block. If you had already known that, you wouldn't have even had to look at the node file; you'd simply have looked at an "l" of the /dev/rfd0135ds18 and recreated the block version appropriately. There are other fascinating things that can be learned from the node files. For example, fd096ds18 is also minor number 60, and can be used in the same way with identical results. In other words, if you z'd out (were momentarily innattentive, not CTRL-Z in a job control shell) and dd'd an image to /dev/fd096ds18, it would write to your hd floppy without incident. If you have a SCSI tape drive, notice what happens when you set it to be the "default" tape drive. It creates device files that have different names (rct0, etc.) but that have the same major and minor numbers. Knowing that it's easy to recreate missing device files also means that you can sometimes capture the output of programs that write directly to a device. For example, suppose some application prints directly to /dev/lp
but you need to capture this to a file. In most situations, you can simply "rm /dev/lp" (after carefully noting its current ownership, permissions and, of course, major/minor numbers), and then "touch /dev/lp" to create an ordinary file. You'll need to chmod it for appropriate permissions, and then run your app. Unless the app has tried to do ioctl calls on the device, the output will be there for your use. This can be particularly useful for examining control characters that the app is sending. What's the Difference? One question that comes up fairly often is "what's the difference between a block and a character device and when should I use one rather than the other?". To answer that question fully is hard, but I'm going to try to at least get you started here. The real difference lies in what the kernel does when a device file is accessed for reading or writing. If the device is a block device, the kernel gives the driver the address of a kernel buffer that the driver will use as the source or destination for data. Note that the address is a "kernel" address; that's important because that buffer will be cached by the kernel. If the device is raw , then the address it will use is in the user space of the process that is using the device. A block device is something you could make a filesystem on (a disk). You can move forward and backward, from the beginning of a block device to its end, and then back to the beginning again. If you ask to read a block that the kernel has buffered, then you get data from the buffer. If you ask for a block that has not yet been buffered, the kernel reads that block (and probably a few more following it) into the buffer cache. If you write to a block device, it goes to the buffer cache (eventually to the device, of course). A raw (or character) device is often something that doesn't have a beginning or end; it just gives a stream of characters that you read. A serial port is an excellent example- however, it is not at all unusual to have character (raw) drivers for things that do have a beginning and an end- a tape drive, for example. And many times there are BOTH character and block devices for the same physical device- disks, for example. Nor does using a raw device absolutely mean that you can't move forward and back, from beginning to end- you can move wherever you want with a tape or /dev/rfd0. And that's where the differences get confusing. It seems pretty reasonable that you'd use the block device to mount a disk. But which do you use for format? For fsck? For mkfs? Well, if you try to format /dev/fd0135ds18, you'll be told that it is not a formattable device. Does that make any sense? Well, the format process involves sequential access- it starts at the beginning and just keeps on going, so it seems to make sense that it wouldn't use the block device. But you can run "mkfs" on either
the block or character device; it doesn't seem to care. The same is true for fsck. But although that's true for those programs on SCO OSR5, it isn't necessarily going to be true on some other UNIX, and the "required" device may make sense to whover wrote the program, but it may not make sense to you. You'd use a block device when you want to take advantage of the caching provided by the kernel. You'd use the raw device when you don't, or for ioctl operations like "tape status" or "stty -a". 27.2 Note 2: ============ One of the unique things about Unix as an operating system is that regards everything as a file. Files can be divided into three categories; ordinary or plain files, directories, and special or device files. Directories in Unix are properly known as directory files. They are a special type of file that holds a list of the other files they contain. Ordinary or plain files in Unix are not all text files. They may also contain ASCII text, binary data, and program input or output. Executable binaries (programs) are also files, as are commands. When a user enters a command, the associated file is retrieved and executed. This is an important feature and contributes to the flexibility of Unix. Special files are also known as device files. In Unix all physical devices are accessed via device files; they are what programs use to communicate with hardware. Files hold information on location, type, and access mode for a specific device. There are two types of device files; character and block, as well as two modes of access. - Block device files are used to access block device I/O. Block devices do buffered I/O, meaning that the the data is collected in a buffer until a full block can be transfered. - Character device files are associated with character or raw device access. They are used for unbuffered data transfers to and from a device. Rather than transferring data in blocks the data is transfered character by character. One transfer can consist of multiple characters. So what about a device that could be accessed in character or block mode? How many device files would it have? One. Two. There are no such devices.
Some devices, such as disk partitions, may be accessed in block or character mode. Because each device file corresponds to a single access mode, physical devices that have more than one access mode will have more than one device file. Device files are found in the /dev directory. Each device is assigned a major and minor device number. The major device number identifies the type of device, i.e. all SCSI devices would have the same number as would all the keyboards. The minor device number identifies a specific device, i.e. the keyboard attached to this workstation. Device files are created using the mknod command. The form for this command is: mknod device-name type major minor device-name is the name of the device file type is either "c" for character or "b" for block major is the major device number minor is the minor device number The major and minor device numbers are indexed to device switches. There are two types of device switches; c devsw for character devices and bdevsw for block devices. These switches are kernel structures that hold the names of all the control routines for a device and tell the kernel which driver module to execute. Device switches are actually tables that look something like this: 0 keyboard 1 SCSIbus 2 tty 3 disk Using the ls command in the /dev directory will show entries that look like: brw-r----- 1 root sys 1, 0 Aug 31 16:01 /dev/sd1a The "b" before the permissions indicates that this is a block device file. When a user enters /dev/sd1a the kernel sees the file opening, realizes that it's major device number 1, and calls up the SCSIbus function to handle it.
==================== 28. Solaris devices: ==================== Devices are described in three ways in the Solaris environment, using three distinct naming conventions: the physical device name, the instance name, and the logical device name. Solaris stores the entries for physical devices under the /devices directory, and the logical device entries behind the /dev directory.
- A "physical device name" represents the full pathname of the device. Physical device files are found in the /devices directory and have a naming convention like the following example: /devices/sbus@1,f8000000/esp@0,40000/sd@3,0:a Each device has a unique name representing both the type of device and the location of that device in the system-addressing structure called the "device tree". The OpenBoot firmware builds the device tree for all devices from information gathered at POST. The device tree is loaded in memory and is used by the kernel during boot to identify all configured devices. A device pathname is a series of node names separated by slashes. Each device has the following form: driver-name@unit-address:device-arguments - The "instance name" represents the kernel's abbreviated name for every possible device on the system. For example, sd0 and sd1 represents the instance names of two SCSI disk devices. Instance names are mapped in the /etc/path_to_inst file, and are displayed by using the commands dmesg, sysdef, and prtconf - The "Logical device names" are used with most Solaris file system commands to refer to devices. Logical device files in the /dev directory are symbolically linked to physical device files in the /devices directory. Logical device names are used to access disk devices in the following circumstances: - adding a new disk to the system and partitioning the disk - moving a disk from one system to another - accessing or mounting a file system residing on a local disk - backing up a local file system - repairing a file system Logical devices are organized in subdirs under the /dev directory by their device types /dev/dsk block interface to disk devices /dev/rdsk raw or character interface to disk devices. In commands, you mostly use raw logical devices, like for example # newfs /dev/rdsk/c0t3d0s7 /dev/rmt tape devices /dev/term serial line devices etc.. Logical device files have a major and minor number that indicate device drivers, hardware addresses, and other characteristics.
Furthermore, a device filename must follow a specific naming convention. A logical device name for a disk drive has the following format: /dev/[r]dsk/cxtxdxsx where cx refers to the SCSI controller number, tx to the SCSI bus target number, dx to the disk number (always 0 except on storage arrays) and sx to the slice or partition number.
=========================== 29. filesystems in Solaris: =========================== 29.1 A few traditional filesystem commands: =========================================== The UFS filesystem has always been the most popular fs on Solaris. Ofcourse, when the newer ZFS filesystem became available, it has been rapidly adopted. We will frst take a look at a few classical commands, that you would typically use on a UFS filesystem. Ofcourse, many "listing commands" like for example, df (to show what's used and what is free space), can be used on ZFS as well. But creating an fs on ZFS goes absolutly different from what you can find in section 29.1 Checks on the filesystems in Solaris: ------------------------------------1. used space etc.. # df -k, df -h etc.. # du -ks /home/fred Shows only a summary of the disk usage of the /home/fred subdirectory (measured in kilobytes). # du -ks /home/fred/* Shows a summary of the disk usage of each subdirectory of /home/fred (measured in kilobytes). # du -s /home/fred Shows a total summary of /home/fred # du -sg /data Shows a total summary of /data in GB
This command shows the diskusage of /dirname in GB # du -g /dirname 2. examining the disklabel # prtvtoc /dev/rdisk/c0t3d0s2 3. format just by itself shows the disks # format # format -> specify disk -> choose partition -> choose print to get the partition table 4. Display information about SCSI devices # cfgadm -al or, from the PROM, commands like probe-scsi What is the CDROM device in Solaris: ------------------------------------- pointer 1. If you have a CD put in the drive, and it was automounted, simply use the "df" command to view your filesystems: # df -k
or df -h
-- pointer 2. From the output of the command # iostat -En you could figure out what logical device name your CDROM has. -- pointer 3. Solaris uses the same naming conventions as used with hardisks, for example the CDROM in the following command # mount -r -F hsfs /dev/dsk/c0t6d0s2 /cdrom means that in this case, the CDROM device is "/dev/dsk/c0t6d0s2" Normally, a CD is automounted on "/cdrom" or "/cdrom/cdrom0" The simplest way to mount CDROM on Solaris is use vold daemon. The vold daemon in Solaris manages the CD-ROM device and automatically performs the mounting similar to how Windows manages CDROMs (but not as transparent or reliable). If CD is detected in drive its should be automatically mounted to the /cdrom/cdrom0 directory. Recovering disk partition information in Solaris:
------------------------------------------------Use the fmthard command to write the backup VTOC information back to the disk. The following example uses the fmthard command to recover a corrupt label on a disk named /dev/rdisk/c0t3d0s1. The backup VTOC information is in a file named c0t3d0 in the /vtoc directory. # fmthard -s /vtoc/c0t3d0s0 /dev/rdsk/c0t3d0s2 Remember that the format of /dev/(r)dsk/cWtXdYsZ means: W X Y Z
is is is is
the the the the
controller number, SCSI target number, logical unit number (LUN, almost always 0), slice or partition number
Make a new filesystem in Solaris: --------------------------------To create a UFS filesystem on a formatted disk that already has been divided into slices you need to know the raw device filename of the slice that will contain the filesystem. Example: # newfs /dev/rdsk/c0t3d0s7 defaults on UFS on Solaris: blocksize 8192 fragmentsize 1024 one inode for each 2K of diskspace FSCK in Solaris: ---------------If you just want to determine the state of a filesystem, whether it needs checking, you can use the fsck command while the fs is mounted. Example: # fsck -m /dev/rdsk/c0t0d0s6 The state flag in the superblock of the filesystem you specify is checked to see whether the filesystem is clean or requires checking. If you ommit the device argument, all the filesystems listed in /etc/vfstab with a fsck pass value greater than 0 are checked. Adding a disk in Solaris 2.6, 2.7, 8, 9: ----------------------------------------
In case you have just build in a new disk, its probably best, to first use the probe-scsi command from the OK prompt: ok probe-scsi .. Target 3 Unit 0 Disk ..
Seagate ST446452W
0001
Next, do a reconfiguration reboot, with the "boot -r" command: ok boot -r Specifying the -r flag when booting, tells Solaris to reconfigure itself by scanning for new hardware. Once the system is up, check the output for "dmesg" to find kernel messages relating to the new disk. You probably find complaints telling you stuff as "corrupt label wrong magic number" etc.. That's good, because we now know that the kernel is aware of this new disk. In this example, our disk is SCSI target 3, so we can refer to the whole disks as /dev/rdsk/c0t3d0s2 # slice 2, or partition 2, s2 refers to the whole disk Remember that the format of /dev/(r)dsk/cWtXdYsZ means: W X Y Z
is is is is
the the the the
controller number, SCSI target number, logical unit number (LUN, almost always 0), slice or partition number
We now use the format program to partition the disk, and afterwards create filesystems. # format /dev/rdsk/c0t3d0s2 (.. output..) FORMAT MENU: format>label Ready to label disk, continue? y format>partition PARTITION MENU: partition> Once you have created and sized the partitions, you can get a list with the "partition>print" command.
Now, for example, you can create a filesystem like in the following command: # newfs /dev/rdsk/c0t3d0s0 devfsadm: --------As from Solaris 8: devfsadm(1M) maintains the /dev and /devices namespaces. It replaces the previous suite of devfs administration tools including drvconfig(1M) , disks(1M) , tapes(1M) , ports(1M) , audlinks(1M) , and devlinks(1M) . The default operation is to attempt to load every driver in the system and attach to all possible device instances. devfsadm then creates device special files in /devices and logical links in /dev . In other words, the devfsadm command is used to dynamically reconfigure system device tables without having to reboot the system. Examples: # devfsadm -i sd # devfsadm -c tape In the first example, devfsadm configures only those devices supported by the sd driver. In the second example, devfsadm configures only tape devices.
29.2 Notes on filesystems on Solaris: ===================================== There are at least 4 different types of filesystems you can use with Solaris 10 (except for zfs, for the older Solaris 8 and 9 versions). These are: -- UFS The traditional filesystem for Solaris systems. UFS is old technology but it is a stable and fast filesystem. Sun has continuously tuned and improved the code over the years. Solaris 10 (and older ofcouse) can only boot from a UFS root filesystem. In the future, ZFS boot will be available, as it already is in OpenSolaris. But for now, every Solaris system must have at least one UFS filesystem. Note: This "boot-statement" was true at the time of writing. Maybe you read this way after that time, and maybe Solaris can now boot from zfs or other filesystem. -- ZFS
We will talk a bit on ZFS in section 29.3 -- VxFS The Veritas filesystem and volume manager have their roots in a faulttolerant proprietary minicomputer built by Veritas in the 1980s. They have been available for Solaris since at least 1993 and have been ported to AIX and Linux. They are integrated into HP-UX and SCO UNIX, and Veritas Volume Manager code has been used (and extensively modified) in Tru64 UNIX and even in Windows. VxFS has never been part of Solaris but, when UFS was the only option, it was a popular addition. VxVM and VxFS are tightly integrated. Through vxassist, one may shrink and grow filesystems and their underlying volumes with minimal trouble. VxFS can run in single instance mode or in a parallel access/cluster file system mode. This latter mode allows for multiple servers (also known as cluster nodes) to simultaneously access the same file system. When run in this mode, VxFS is referred to as VERITAS Cluster File System. Cluster File System provides cache coherency and POSIX compliance across nodes, so that data changes are atomically seen by all cluster nodes simultaneously. Because Cluster File System shares the same binaries and same on-disk layout as single instance VxFS, moving between cluster and single instance mode is straightforward. -- SAM and QFS QFS is Sun's cluster filesystem, meaning that the same filesystem may be simultaneously mounted by multiple systems. SAM is a hierarchical storage manager; it allows a set of disks to be used as a cache for a tape library. SAM and QFS are designed to work together, but each may be used separately. -- PCFS It's even possible to use the DOS FAT filesystem. -- HSFS Ofcourse, the CDROM HSFS can be used. Maybe the following list will show you what can be used in Solaris: Filesystem UFS Solaris ZFS VxFS QFS pcfs hsfs tmpfs
Type Device Regular Disk
Description Unix Fast filesystem; default in
Regular Regular Regular Regular Regular Regular
The new Regular FS in Solaris 10 Veritas filesystem QFS filesystem from LSC Inc. MSDOS FAT and FAT32 filesystem High Sierra filesystem (CDROM) Uses memory and swap
Disk Disk Disk Disk Disk Memory
nfs Pseudo cachefs Pseudo another NFS filesystem autofs Pseudo other filesystems specfs Pseudo procfs Pseudo processes sockfs Pseudo fifofs Pseudo
Network filesystem
Network filesystem Uses a local disk as cache for
filesystem
Uses a dynamic layout to mount
Device drivers filesystem for the /dev devices Kernel /proc filesystem representing Network Files
Filesystem of socket connections FIFO filesystem
If we look at the regular disk based filesystems, the following can be said on the "allocation format": Filesystem UFS VxFS QFS ZFS
Allocation format Block, allocator tries to allocate sequential blocks Extent based Extent based Extent based
29.3 Some notes on the ZFS filesystem. Solaris 10 ================================================= >>> ZFS Pooled Storage: ----------------------ZFS uses the concept of storage pools to manage physical storage. Historically, file systems were constructed on top of a single physical device. To address multiple devices and provide for data redundancy, the concept of a "logical volume manager", LVM, was introduced to provide for Volume Groups, and Logical Volumes (which could span multiple disks), and then add a filesystem on such a Logical Volume. This design added another layer of complexity and ultimately prevented certain file system advances, because the file system had no control over the physical placement of data on the virtualized volumes. ZFS eliminates the volume management altogether. Instead of forcing you to create virtualized volumes, ZFS aggregates devices into a storage pool. The storage pool describes the physical characteristics of the storage (device layout, data redundancy, and so on,) and acts as an arbitrary data store from which file systems can be created. File systems are no longer constrained to individual devices, allowing them to share space with all file systems in the pool. You no longer need to predetermine the size of a file system, as file systems grow automatically within the space allocated to the storage pool. When new storage is added, all file systems within the pool can immediately use the additional space without additional work. In many ways,
the storage pool acts as a virtual memory system. When a memory DIMM is added to a system, the operating system doesn't force you to invoke some commands to configure the memory and assign it to individual processes. All processes on the system automatically use the additional memory. Everything you hate about managing file systems and volumes is gone: you don't have to use format, and create slices/partitions, use newfs, mount, edit /etc/vfstab, fsck, growfs, metadb, metainit, etc. Meet your new best friends: zpool and zfs. ZFS is easy, so let's get on with it! It's time to create your first pool: # zpool create tank c1t2d0 You now have a single-disk storage pool named tank, with a single file system mounted at /tank. There is nothing else to do. Yes, its really true: The new ZFS file system, tank, can use as much of the disk space as needed, and is automatically mounted at /tank. You can determine if your pool was successfully created by using the zpool list command. # zpool list NAME tank
SIZE 80G
USED 137K
AVAIL 80G
CAP 0%
HEALTH ONLINE
ALTROOT -
Suppose we create a file in /tank and want to see how things looks like: # mkfile 100m /tank/foo # df -h /tank Filesystem size used avail capacity Mounted on tank 80G 100M 80G 1% /tank If you want mirrored storage for mail and home directories, that's easy too: Create the pool: # zpool create tank mirror c1t2d0 c2t2d0 Now lets try to create the "/var/mail" file system: # zfs create tank/mail # zfs set mountpoint=/var/mail tank/mail Create home directories, and mount them all in /export/home/: # zfs create tank/home # zfs set mountpoint=/export/home tank/home
At this point, we have "/export/home" present. Now you could even do this: # zfs create tank/home/ahrens ZFS file systems are hierarchical: each one inherits properties from above. In this example, the mountpoint property is inherited as a pathname prefix. That is, tank/home/ahrens is automatically mounted at /export/home/ahrens because tank/home is mounted at /export/home. You don't have to specify the mountpoint for each individual user - you just tell ZFS the pattern. >>> Commit and Rollback semantics: ---------------------------------ZFS uses a commit and rollback mechanism, to ensure that all data is written completely, and if not, everything is rolled back. You probably know that with former filesystems, that you could choose - for a filesystem without journaling (logging) - or indeed use journaling (or logging). Now you have a third option: using a transactional filesystem, like zfs. ZFS is a transactional file system, which means that the file system state is always consistent on disk. Traditional file systems (with no logging) overwrite data in place, which means that if the machine loses power, for example, between the time a data block is allocated and when it is linked into a directory, the file system will be left in an inconsistent state. Historically, this problem was solved through the use of the fsck command. This command was responsible for going through and verifying file system state, making an attempt to repair any inconsistencies in the process. This problem sometimes caused great pain to administrators and was never guaranteed to fix all possible problems. More recently, file systems have introduced the concept of journaling. The journaling process records action in a separate journal, which can then be replayed safely if a system crash occurs. This process introduces unnecessary overhead, because the data needs to be written twice, and often results in a new set of problems, such as when the journal can't be replayed properly. With a transactional file system, data is managed using copy on write semantics. Data is never overwritten, and any sequence of operations is either entirely committed or entirely ignored. This mechanism means that the file system can never be corrupted through accidental loss of power or a system crash. So, no need for a fsck equivalent exists. While the most recently written pieces of data might be lost, the file system itself will always be consistent. In addition, synchronous data (written using the O_DSYNC flag) is always guaranteed to be written before returning, so it is never lost.
>>> Unparalleled Scalability: ----------------------------ZFS has been designed from the ground up to be a very scalable file system. The file system itself is 128-bit, allowing for 256 quadrillion zettabytes of storage. All metadata is allocated dynamically, so no need exists to pre-allocate inodes or otherwise limit the scalability of the file system when it is first created. All the algorithms have been written with scalability in mind. Directories can have up to 248 (256 trillion) entries, and no limit exists on the number of file systems or number of files that can be contained within a file system. >>> Some more examples: ------------------------ To give user ahrens a 10G quota: # zfs set quota=10g tank/home/ahrens -- To give user bonwick a 100G reservation (membership has its privileges): # zfs set reservation=100g tank/home/bonwick -- To automatically NFS-export all home directories read/write: # zfs set sharenfs=rw tank/home -- To scrub all disks and verify the integrity of all data in the pool: # zpool scrub tank -- To replace a flaky disk: # zpool replace tank c2t2d0 c4t1d0 -- To add more space: # zpool add tank mirror c5t1d0 c6t1d0 -- To move your pool from SPARC machine 'sparky' to AMD machine 'amdy': [on sparky] # zpool export tank Physically move your disks from sparky to amdy. [on amdy] # zpool import tank -- Determining if Problems Exist in a ZFS Storage Pool
The easiest way to determine if any known problems exist on the system is to use the "zpool status x" command. This command describes only pools exhibiting problems. If no bad pools exist on the system, then the command displays a simple message, as follows: # zpool status -x all pools are healthy Without the x flag, the command displays the complete status for all pools (or the requested pool, if specified on the command line), even if the pools are otherwise healthy. -- Understanding zpool status Output The complete zpool status output looks similar to the following: # zpool pool: state: status: in a
status tank tank DEGRADED One or more devices has been taken offline by the administrator. Sufficient replicas exist for the pool to continue functioning
degraded state. action: Online the device using 'zpool online' or replace the device with 'zpool replace'. scrub: none requested config: NAME tank mirror c1t0d0 c1t1d0
STATE DEGRADED DEGRADED ONLINE OFFLINE
READ WRITE CKSUM 0 0 0 0 0 0 0 0 0 0 0 0
errors: No known data errors 29.4 Some examples on VxFS: =========================== See section 29.2 for a general description about the filesystems you can use on Solaris. Example 1: ---------# mkfs -F vxfs /dev/vx/rdsk/testdg/msvol1 200m version 4 layout 409600 sectors, 204800 blocks of size 1024, log size 1024 blocks unlimited inodes, largefiles not supported 204800 data blocks, 203656 free data blocks 7 allocation units of 32768 blocks, 32768 data blocks last allocation unit has 8192 data blocks
Example 2: ---------We are going to show how to create a mirroring volume and a stripping volume on Veritas Storage Foundation. on Solaris 10. The first step is to check quantity of disks you have available on the server. A simple way to check this on solaris is using format utility: bash-3.00# format Searching for disks.done AVAILABLE DISK SELECTIONS: 0. c1t0d0 /pci@0,0/pci15ad,1976@10/sd@0,0 1. c1t1d0 /pci@0,0/pci15ad,1976@10/sd@1,0 2. c1t2d0 /pci@0,0/pci15ad,1976@10/sd@2,0 3. c1t3d0 /pci@0,0/pci15ad,1976@10/sd@3,0 Also, you can check disks available to Veritas Storage Foundation using vxdisk command: bash-3.00# vxdisk -o alldgs list DEVICE TYPE DISK GROUP STATUS c1t0d0s2 c1t1d0s2 c1t2d0s2 c1t3d0s2
auto:none auto:none auto:none auto:none
-
-
online online online online
invalid invalid invalid invalid
You can see above that there are 4 disks on the server that are available to Veritas but they have not yet been initialized by Veritas (invalid status). To use a disk on Veritas SF you need to initialize this using Veritas utilities. NOTE: If you are going to use a disk on Veritas, pay attention that you should give this whole disk to Veritas. Disk will be formatted and you will lose all data in the disk when you are allocating a disk to Veritas Storage. In this example the only disk that is in use for O.S Solaris is the first one. (c1t0d0s2). We can use those 3 others disks to add on Veritas Storage.
Caution: If for a mistake we add the first disk (c1t0d0s2) to Veritas Storage, it will format the disk and erase Solaris info. We need to pay attention to get the right disks. Let's start allocating (initializing) those 3 disks to solaris: # vxdisksetup -i c1t1d0 # # vxdisksetup -i c1t2d0 # vxdisksetup -i c1t3d0 We have those 3 disks initialized on Veritas, then the next step is to create a Disk Group. >>> Disk Group Disk Group is a collection of disks. Disk Group is very useful for management and isolation purpose. Lets create a DG using only the fist disk initialized on Veritas (c1t1d0). We are using DG1 for the name of Disk Group. # vxdg init DG1 c1t1d0 Check if
DG1 was created successfully:
# vxdg list NAME STATE ID DG1 enabled,cds 1218633322.13.vrt2 Also, check if the disk is properly assigned to DG1: # vxdisk -o alldgs list DEVICE TYPE DISK GROUP STATUS c1t0d0s2 c1t1d0s2 c1t2d0s2 c1t3d0s2
auto:none - - online invalid auto:cdsdisk c1t1d0 DG1 online auto:cdsdisk - - online auto:cdsdisk - - online
Let's add more 2 disks to DG1: # vxdg -g DG1 adddisk c1t2d0s2 c1t3d0s2 Check if the disks are properly assigned to DG1: # vxdisk -o alldgs list DEVICE TYPE DISK GROUP STATUS c1t0d0s2 auto:none - - online invalid c1t1d0s2 auto:cdsdisk c1t1d0 DG1 online
c1t2d0s2 auto:cdsdisk c1t2d0 DG1 online c1t3d0s2 auto:cdsdisk c1t3d0 DG1 online At this point we have added 3 disks into Disk Group DG1. Next step we will create 2 different volumes in the DG1. >>> Volumes A volume is a virtual storage that is used as an physical disk. Volume can be composed by many disks and have many layouts. In this example, we are going to create two Volumes: Volume VolS - Stripping layout using c1t1d0 and c1t2d0 disks (RAID 0). Volume VolM - Mirroring layout using c1t2d0 and c1t3d0 (RAID 1). -- To create a Stripping Volume VolS (Size=10m): # vxassist -g DG1 make VolS 10m layout=stripe c1t1d0s2 c1t2d0s2 To check if volume VolS was created successfully: # vxprint -g DG1 TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 dg DG1 DG1 - - - - - dm c1t1d0 c1t1d0s2 - 159488 - - - dm c1t2d0s2 c1t2d0s2 - 159488 - - - dm c1t3d0s2 c1t3d0s2 - 159488 - - - v VolS fsgen ENABLED 20480 - ACTIVE - pl VolS-01 VolS ENABLED 20480 - ACTIVE - sd c1t1d0-01 VolS-01 ENABLED 10240 0 - - sd c1t2d0s2-01 VolS-01 ENABLED 10240 0 - - -- To create a Mirroring Volume VolM (Size=10m): # vxassist -g DG1 make VolM 10m layout=mirror c1t2d0s2 c1t3d0s2 To check if Volume VolM was created successfully: # vxprint -g DG1 TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 dg DG1 DG1 - - - - - dm c1t1d0 c1t1d0s2 - 159488 - - - dm c1t2d0s2 c1t2d0s2 - 159488 - - - dm c1t3d0s2 c1t3d0s2 - 159488 - - - -
v VolM fsgen ENABLED 20480 - ACTIVE - pl VolM-01 VolM ENABLED 20480 - ACTIVE - sd c1t3d0s2-01 VolM-01 ENABLED 20480 0 - - pl VolM-02 VolM ENABLED 20480 - ACTIVE - sd c1t2d0s2-02 VolM-02 ENABLED 20480 0 - - v VolS fsgen ENABLED 20480 - ACTIVE - pl VolS-01 VolS ENABLED 20480 - ACTIVE - sd c1t1d0-01 VolS-01 ENABLED 10240 0 - - sd c1t2d0s2-01 VolS-01 ENABLED 10240 0 - - Note: You can see above that both Volumes were created successfully. Also, you can note the difference between stripping and mirroring volume layouts. VolM is using two different Plex in differente disks. This means that if you lose one disk (Plex) you still have the data in the other disk (other Plex). It is the main configuration of Mirroring Volumes. VolS is using only one Plex divided in 2 disks. This means that the data will be split in those 2 disks. If you lose one disk you would lose the whole Plex, therefore you would lose the data. This is the main configuration of Stripping Volumes. It does not provide data protection but it is very useful for performance for purpose. Also, you can add those 2 layouts in only one layout that provide data protection and better performance. It is the case of RAID 0 + 1 or RAID 1 + 0. In the next step we will create 2 different Filesystem using those 2 Volumes. >>> Filesystem In this example we will create two filesystem: - Filesystem fsS will use VolS. It will be mounted at /stripe mount point. - Filesystem fsM will use VolM. It will be mounted at /mirror mount point. To create a VxFS filesystem: # mkfs -F vxfs /dev/vx/rdsk/DG1/VolS version 7 layout 20480 sectors, 10240 blocks of size 1024, log size 1024 blocks largefiles supported # mkfs -F vxfs /dev/vx/rdsk/DG1/VolM version 7 layout
20480 sectors, 10240 blocks of size 1024, log size 1024 blocks largefiles supported To mount a VxFS filesystem: # mount -F vxfs /dev/vx/dsk/DG1/VolS /stripe/ # mount -F vxfs /dev/vx/dsk/DG1/VolM /mirror/ Now there are 2 filesystems configured and you can use it at Solaris Mount Point level. Any data written in /stripe directory will be written in the stripping VolS volume. Any data written in /mirror directory will be written in the mirroring VolM volume. Example 3: ---------Rather than mess with vxmake you can employ vxassist to do all the dirty work. If you have any amount of experience with vxassist you'll know that the more information you can supply to vxassist the better the end product will be. I'm going to use vxassist to build a stripe-pro volume from four disks and I want the volume to be 1G in size: # vxassist -g testdg make stripeprovol 1g layout=stripe-mirror \ testdg01 testdg02 testdg03 testdg04 Pretty kool, huh? Quick, efficient, and poorly named; everything you love about vxassist. I can then go a bit further and explore my sizing options to see how much I can grow my new volume if I need to: # vxassist -g testdg maxgrow stripeprovol Volume stripeprovol can be extended by 282050560 to 284147712 (138744Mb) See? Just like a normal volume. Now comes the beauty part. When you look at that seemingly unmanageable mess of objects above does it really make you want to tear it apart and work on it like you might other "normal" volumes? Probably not. And you'd be wise to feel that way, there are just too many places to get confused or make a mistake when real data is involved. What if you could get back to a more normal point of view? Luckily you can, check this out: # vxassist -g testdg convert stripeprovol layout=mirror-stripe Veritas terminology:
In a "typical" RAID0+1 volume configuration, we take several disks and then create a stripe across thoughs disks (the RAID0 part). Then once complete we do this again on a separate set of disks, and then attach that new stripe to the first creating a mirror (the +1 part). We then have a RAID0+1 volume thats ready to have a filesystem put on it. The point of interest with this setup is that we're actually mirroring a complete stripe (and therefore ALL the disks in that stripe) to another stripe (and therefore ALL of it's disks). The problem here is that if for some reason we need to re-sync the volume we'd need to re-sync a full stripe to a full stripe (very timely) which is a nearly tragic proposition if your talking about 50G+. A far more efficient setup would be to mirror each disk to each disk... in other words, to mirror a bunch of disks on a one-to-one basis, and then build a stripe on top of these mirrors. In this case if we need to re-sync due to a disk failure we can simply sync the failed disk to its mirror, instead of the full stripe. This is the power of RAID1+0; the difference between mirroring the stripes (0+1) and stripping the mirrors (1+0). If the terms seem to confuse you, try this for size: RAID0 Striping (VxVM says: stripe) RAID1 Mirroring (VxVM says: mirror) RAID0+1 Striping plus Mirroring (VxVM says: mirror-stripe) Think this: Striped disks, then mirror the stripes RAID1+0 Mirroring plus Striping (VxVM says: stripe-mirror) (Veritas Marketing Dept says: StripePro Think this: Mirrored disks, then stripe on top of the mirrors Concat+Mirror Concatenation plus Mirroring (VxVM says: mirror) Same as RAID1 Mirror+Concat Mirroring plus Concatenation (VxVM says: concat-mirror) (Veritas Marketing Dept says: ConcatPro) Think this: Concatenation on top of mirrored disks. Veritas Default diskgroup: rootdg Default rootdg disk group. Block Device Node /dev/vx/dsk/volume_name Raw Device Node /dev/vx/rdsk/volume_name Other DiskGroups Block Device Node /dev/vx/dsk/diskgroup_name/volume_name Raw Device Node /dev/vx/rdsk/diskgroup_name/volume_name Example 4: ---------Some more examples: Create Veritas layout on a disk: vxdisksetup -i c1t10d0 Create a disk group on a new disk: vxdg init =c1t10d0
Add disk to an existing disk group: vxdg -g adddisk =c2t0d0 replace addisk with rmdisk to remove a disk Set up a preferred reading plex, this can be useful if we have a sparse plex (plex in RAM): vxvol -g rdpol prefer instead of prefer we can have round or sdeet View configuration: vxprint -th List disks: vxdisk list vxdisk -o alldgs list (shows deported disks) Adding disks while solaris is running: drvconfig (This probes scsi - Solaris) disks (Creates links in /dev - Solaris) prtvtoc (View the vtoc - Solaris) vxdctl enable (Rescan for disks - Veritas) vxdisk list (Shows the disk in error as they are not initalized jet) vxdisksetup (init the disks) To encapsulate use: vxencap -g Export a disk group: vxdg deport vxdg -h deport to export to another host Import a disk group: vxdg import vxdg -C to clear hostid of old host (When failing over in DR situation) vxdg -fC to clear hostid of old host and forcing diskgroup online Destroy a disk group: vxdg destroy Evacuate data from a disk: vxevac -g Create a volume on a diskgroup: vxassist -g make layou=stripe ncols=number of colums stripeunit=size Create a veritas filesystem on this volume: mkfs -F vxfs /dev/vx/rdsk// Delete a volume same as creatiuon but replace make with remove Resize a filesystem: vxresize -g -F
If Veritas is ever causing you problems, do the following: Touch /etc/vx/reconfig.d/state.d/install-db edit /etc/system and modify /etc/vfstab to disable VRTS to start up and access the old root partitions vxassist make martin 100m makes a volume called martin using any disk vxassist make martin 100m disk10 makes a volume called martin using disk10 vxassist make martin 100m layout=stripe disk07 disk08 creates a 100mb striped volume called martin using disks7 and 8 vxassist mirror martin disk05 disk06 uses disks5 and 6 ro make a mirror on volume called martin vxassist make martin 50m layout=mirror makes a 50Mb mirror using any 2 disks vxassist make martin 50m layout=mirror disk05 disk06 makes a 50mb mirror using disks 5 and 6 vxassist make martin 50m layout=mirror,stripe disk05 disk06 disk07 disk08 makes a 50Mb stripe using disks5 and 6 mirrored across 7 and 8 vxassist make martin 50m layout=mirror,stripe,log disk05 disk06 disk07 disk08 makes a 50Mb stripe using disks5 and 6 mirrored across 7 and 8 and uses a log subdisk vxassist make martin 100m layout=raid5 makes a 100m raid5 volume /usr/sbin/vxedit -g rootdg rename disk12 disk09 to rename disk12 to disk09 in the rootdg vxedit rm disk10 to remove a greyed out or obsolete disk in this case disk10 or to remove a disk from a diskgroup vxdisk list - to list all disks under vmcontrol vxdisk clearimport c#t#d#s# to allow a disk to be imported after a server crash vxdg -g razadg rmdisk test to remove a disk called test from a dg called razadg vxdg -g razadg adddisk test=c1t3d3 to add disk c1t3d3 to a dg called razadg calling the disk test, use vxdisk list to determine what disks are free :)
vxedit -g rootdg set spare=on disk09 sets disk09 in the rootdg as a hotspare. vxmirror rootdisk disk01 mirrors all the volumes on the root disk to disk01 vxassist -g rootdg mirror vol01 disk03 mirrors vol01 (in rootdg) to disk03 vxassist mirror martin will mirror the volume martin to make a mirror manually try /usr/sbin/vxmake -g rootdg sd disk03-01 dm_name=disk03 dm_offset=0 len=81920 to create a subdisk on disk03 callin the subdisk disk03-01 the len 81920 is 81920sectors x 512bytes =40M vxmake plex martin-02 sd=disk03-01 creates a plex called martin-02 using subdisk disk03-01 vxplex att martin martin-02 attaches the plex martin-02 to volume martin to list all volumes on your primary boot disk enter vxprint -t -v -e 'aslist.aslist.sd_disk="boot_disk_name"' vxsd mv disk03-01 disk05-01 moves the contents of subdisk disk03-01 to disk05-01 then moves subdisk disk05-01 into the plex where subdisk disk03-01 once lived, leaving disk03-01 to your mercy :) to make a subdisk vxmake sd disk02-02 disk02,0,8000 this would create a subdisk called disk02-02 at the start of disk02 and would be 8000blocks (4000k) long. if you wanted to create another subdisk on this disk the offset would be 8000 as this is where the next free space would be onthe disk so... vxmake sd disk02-02 disk02,8000,8000 would create another 8000block subdisk. vxdisk rm c#t#d#s2 to remove a disk so it's out of vm control vxdiskadd c#t#d#
to add bring a new disk under vm control or you can try... vxdisksetup -i c#t#d# vxvol -g dg volname stop this stops a volume vxedit -rf rm martin removes a volume called martin and plex(es) and subdisks though vxprint -ht volume
================ 30. AIX devices: ================ In AIX 5.x, the device configuration information is stored in the ODM repository. The corresponding files are in /etc/objrepos /usr/lib/objrepos /usr/share/lib/objrepos There are 2 sections in ODM: - predefined: all of the devices in principle supported by the OS - customized: all devices already configured in the system Every device in ODM has a unique definition that is provided by 3 attributes: 1. Type 2. Class 3. Subclass Information thats stored in the ODM: -
PdDv,PdAt, PdCn : Predefined device information CuDv, CuAt, CuDep : Customized device information lpp, inventory : Software vital product data smit menu's Error log, alog, and dump information System Resource Controller: SRCsubsys, SRCsubsrv NIM: nim_attr, nim_object, nim_pdattr
There are commands, representing an interface to ODM, so you can add, retrieve, drop and change objects. The following commands can be used with ODM: odmadd,
odmdrop, odmshow, odmdelete, odmcreate, odmchange Examples: # odmget -q "type LIKE lv*" PdDv # odmget -q name=hdisk0 CuAt Logical devices and physical devices: ------------------------------------AIX includes both logical devices and physical devices in the ODM device configuration database. Logical devices include Volume Groups, Logical Volumes, network interfaces and so on. Physical devices are adapters, modems etc.. Most devices are selfconfiguring devices, only serial devices (modems, printers) are not selfconfigurable. The command that configures devices is "cfgmgr", the "configuration manager". When run, it compares the information from the device with the predefined section in ODM. If it finds a match, then it creates the entries in the customized section in ODM. The configuration manager runs every time the system is restarted. If you have installed an adapter for example, and you have put the software in a directory like /usr/sys/inst.images, you can call cfgmgr to install device drivers as well with # cfgmgr -i /usr/sys/inst.images $$ 09-08-00-1,0 u5971-t1-l1-l0 Device information: ------------------The most important AIX command to show device info is "lsdev". This command queries the ODM, so we can use it to locate the customized or the predifined devices. The main commands in AIX to get device information are: - lsdev : queries ODM - lsattr : gets specific configuration attributes of a device
- lscfg device
: gets vendor name, serial number, type, model etc.. of the
lsdev also shows the status of a device as Available (that is configured) or as Defined (that is predefined). lsdev examples: --------------If you need to see disk or other devices, defined or available, you can use the lsdev command as in the following examples: # lsdev -Cc tape rmt0 Available 10-60-00-5,0 # lsdev -Cc disk hdisk0 Available hdisk1 Available hdisk2 Available hdisk3 Available hdisk4 Available
SCSI 8mm Tape Drive
20-60-00-8,0 20-60-00-9,0 20-60-00-10,0 20-60-00-11,0 20-60-00-13,0
16 16 16 16 16
Bit Bit Bit Bit Bit
LVD LVD LVD LVD LVD
SCSI SCSI SCSI SCSI SCSI
Disk Disk Disk Disk Disk
Drive Drive Drive Drive Drive
Note: -C queries the Customized section of ODM, -P queries the Predefined section of ODM. Example if some of the disks are on a SAN (through FC adapters): # lsdev -Cc disk hdisk0 Available Virtual SCSI Disk Drive hdisk1 Available Virtual SCSI Disk Drive hdisk2 Available 02-08-02 SAN Volume Controller MPIO Device FC adapter) hdisk3 Available 02-08-02 SAN Volume Controller MPIO Device FC adapter) # lsattr -El hdisk2 PCM PCM/friend/sddpcm True PR_key_value none Reserve Key algorithm load_balance Algorithm dist_err_pcnt 0 Distributed Error Percentage dist_tw_width 50 Distributed Error Sample Time hcheck_interval 20 Health Check Interval hcheck_mode nonactive Health Check Mode location Location Label lun_id 0x0 Logical Unit Number ID
(through (through
PCM True True True True True True True False
lun_reset_spt yes Support SCSI LUN reset True max_transfer 0x40000 Maximum TRANSFER Size True node_name 0x50050768010029c8 FC Node Name False pvid 00cb5b9e66cc16470000000000000000 Physical volume identifier False q_type simple Queuing TYPE True qfull_dly 20 delay in seconds for SCSI TASK SET FULL True queue_depth 20 Queue DEPTH True reserve_policy no_reserve Reserve Policy True rw_timeout 60 READ/WRITE time out value True scbsy_dly 20 delay in seconds for SCSI BUSY True scsi_id 0x611013 SCSI ID False start_timeout 180 START unit time out value True unique_id 33213600507680190014E30000000000001E204214503IBMfcp Device Unique Identification False ww_name 0x50050768014029c8 FC World Wide Name False
lsdev [ -C ][ -c Class ] [ -s Subclass ] [ -t Type ] [ -f File ] [ -F Format | -r ColumnName ] [ -h ] [ -H ] [ -l { Name | - } ] [ -p Parent ] [ -S State ] lsdev -P [ -c Class ] [ -s Subclass ] [ -t Type ] [ -f File ] [ -F Format | -r ColumnName ] [ -h ] [ -H ] Remark: For local attached SCSI devices, the general format of the LOCATION code "AB-CD-EF-GH" is actually "AB-CD-EF-G,H" , the first three sections are the same and for the GH section, the G is de SCSI ID and the H is the LUN. For adapters, only the AB-CD is mentioned in the location code. A location code is a representation of the path to the device, from drawer, slot, connector and port. - For an adapter it is sufficient to have the codes of the drawer and slot to identify the adapter. The location code of an adapter takes the form of AB-CD. - Other devices needs more specification, like a specific disk on a specific SCSI bus.
For other devices the format is AB-CD-EF-GH. The AB-CD part then indicates the adapter the device is connected on. - For SCSI devices we have a location code like AB-CD-EF-S,L where the S,L fields identifies the SCSI ID and LUN of the device. To lists all devices in the Predefined object class with column headers, use # lsdev -P -H To list the adapters that are in the Available state in the Customized Devices object class, use # lsdev -C -c adapter -S lsattr examples: ---------------This command gets the current attributes (-E flag) for a tape drive: # lsattr -El rmt0 mode yes block_size 1024 extfm no ret no .. ..
Use DEVICE BUFFERS during writes Block size (0=variable length) Use EXTENDED file marks RETENSION on tape change or reset
True True True True
(Ofcourse, the equivalent for the above command is for example # lsattr -l rmt0 -E ) To list the default values for that tape device (-D flag), use # lsattr -l -D rmt0 This command gets the attributes for a network adapter: # lsattr -E -l ent1 busmem 0x3cfec00 busintr 7 .. ..
Bus memory address Bus interrupt level
False False
To list only a certain attribute (-a flag), use the command as in the following example: # lsattr -l -E scsi0 -a bus_intr_lvl bus_intr_lvl 14 Bus interrupt level False # lsattr -El tty0 -a speed speed 9600 BAUD rate true You must specify one of the following flags with the lsattr command: -D Displays default values.
-E Displays effective values (valid only for customized devices specified with the -l flag). -F Format Specifies the user-defined format. -R Displays the range of legal values. -a Displays for that attribute lscfg examples: --------------Example 1: This command gets the Vital Product Data for the tape drive rmt0: # lscfg -vl rmt0 Manufacturer...............EXABYTE Machine Type and Model.....IBM-20GB Device Specific(Z1)........38zA Serial Number..............60089837 .. .. -l Name Displays device information for the named device. -p Displays the platform-specific device information. This flag only applies to AIX 4.2.1 or later. -v Displays the VPD found in the Customized VPD object class. Also, on AIX 4.2.1 or later, displays platform specific VPD when used with the -p flag. -s Displays the device description on a separate line from the name and location. # lscfg -vp | grep -p 'Platform Firmware:' # lscfg -vp | grep -p Platform sample output: Platform Firmware: ROM Level.(alterable).......3R040602 Version.....................RS6K System Info Specific.(YL)...U1.18-P1-H2/Y2 Physical Location: U1.18-P1-H2/Y2 The ROM Level denotes the firmware/microcode level Platform Firmware: ROM Level ............. RH020930 Version ................RS6K .. Example 2: The following command shows details about the Fiber Channel cards:
# lscfg -vl fcs*
(fcs0 for example, is the parent of fsci0)
Adding a device: ---------------Adding a device with cfmgr: --------------------------To add a device you can run cfgmgr, or shutdown the system, attach the new device and boot the system. There are also many smitty screens to accomplish the task of adding a new device. Adding a device with mkdev: --------------------------Also the mkdev command can be used as in the following example: # mkdev -c tape -s scsi -t scsd -p scsi0 -w 5,0 where -c Class of the device -s Subclass of the device -t Type of the device. This is a specific attribute for the device -p The parent adapter of the device. You have to specify the logical name. -w You have to know the SCSI ID that you are goiing to assign to the new device. If it's non SCSI, you have to know the port number on the adapter. -a Specifies the device attribute-value pair The mkdev command also creates the ODM entries for the device and loads the device driver. The following command configures a new disk and ensures that it is available as a physical volume. This example adds a 2.2GB disk with a scsi ID of 6 and a LUN of 0 to the scsi3 SCSI bus. # mkdev -c disk -s scsi -t 2200mb -p scsi3 -w 6,0 -a pv=yes This example adds a terminal: # mkdev -c tty -t tty -s rd232 -p sa1 -w 0 -a login=enable -a term=ibm3151 tty0 Available Changing a device with chdev: -----------------------------
Suppose you have just added a new disk. Suppose the cfgmgr has run and detected the disk. Now you run # lspv hdisk1 none OR hdisk1 0005264d2
none none
The first field identifies the system-assigned name of the disk. The second field displays the "physical volume id" PVID. If that is not shown, you can use chdev: # chdev -l hdisk2 -a pv=yes Removing a device with rmdev: ----------------------------Examples: # lsdev -Cc tape rmt0 Available 10-60-00-5,0 # rmdev -l rmt0 name rmt0 Defined
SCSI 8mm Tape Drive # -l indicates using the logical device
The status have shifted from Available to Defined. # lsdev -Cc tape rmt0 Defined 10-60-00-5,0
SCSI 8mm Tape Drive
If you really want to remove it from the system, use the -d flag as well # rmdev -l rmt0 -d To unconfigure the childeren of PCI bus pci1 and all devices under them, while retaining their device definition in the Customized Devices Object Class. # rmdev -p pci1 rmt0 Defined hdisk1 Defined scsi1 Defined ent0 Defined
The special device sys0: -----------------------In AIX 5.x we have a special device named sys0 that is used to manage some kernel parameters. The way to change these values is by using smitty, the chdev command or WSM.
Example. To change the maxusersprocesses parameter, you can for example use the Web-based System Manager. You can also use the chdev command: #chdev -l sys0 -a maxuproc=50 sys0 changed Note: In Solaris, to change kernel parameters, you have to edit /etc/system. Device drivers: --------------Device drivers are located in /usr/lib/drivers directory.
============================ 31. filesystem commands AIX: ============================ 31.1 The Logical Volume Manager LVM: ==================================== In AIX, it's common to use a Logical Volume Manager LVM to cross the boundaries posed by traditional disk management. Traditionally, a filesystem was on a single disk or on a single partition. Changing a partionion size was a difficult task. With a LVM, we can create logical volumes which can span several disks. The LVM has been a feature of the AIX operating system since version 3, and it is installed automatically with the Operating System. LVM commands in AIX: -------------------mkvg (or the mkvg4vp command in case of SAN vpath disks. See section 31.3) cplv rmlv mklvcopy extendvg reducevg getlvcb lspv lslv lsvg mirrorvg chpv migratepv
exportvg, importvg varyonvg, varyoffvg And related commands: mkdev chdev rmdev lsdev Volume group: ------------What a physical disk is, or a physical volume is, is evident. When you add a physical volume to a volume group, the physical volume is partitioned into contiguous equal-sized units of space called "physical partitions". A physical partition is the smallest unit of storage space allocation and is a contiguous space on a physical volume. The physical volume must now become part of a volume group. The disk must be in a available state and must have a "physical volume id" assigned to it. A volume group (VG) is an entity consisting of 1 to 32 physical volumes (of varying sizes and types). A "Big volume group" kan scale up to 128 devices. You create a volume group with the "mkvg" command. You add a physical volume to an existing volume group with the "extendvg" command, you make use of the changed size of a physical volume with the "chvg" command, and remove a physical volume from a volume group with the "reducevg" command. Some of the other commands that you use on volume groups include: list (lsvg), remove (exportvg), install (importvg), reorganize (reorgvg), synchronize (syncvg), make available for use (varyonvg), and make unavailable for use (varyoffvg). To create a VG, using local disks, use the "mkvg" command: mkvg -y -s Typical example: mkvg -y oravg -s 64 hdisk3 hdisk4 mkvg -y appsvg -s 32 hdisk2 mkvg -y datavg -s 64 hdisk3 mkvg -y appsvg -s 32 hdisk3 mkvg -y datavg -s 32 hdisk2 mkvg -y vge1corrap01 -s 64 hdisk2 In case you use the socalled SDD subsystem with vpath SAN storage, you should use the "mkvg4vp" command,
which works similar (same flags) as the mkvg command.
Types of VG's: ============== There are 3 kinds of VG's: - Normal VG (AIX 5L) - Big VG (AIX 5L) - Scalable VG (as from AIX 5.3) Normal VG: ---------Number of disks 1 2 4 8 16 32
Max number of partitions/disk 32512 16256 8128 4064 2032 1016
Big VG: ------Number of disks 1 2 4 8 16 32 64 128
Max number of partitions/disk 130048 65024 32512 16256 8128 4064 2032 1016
VG Type Max PV's Max LV's Max PP's per VG --------------------------------------------------------------Normal 32 256 32512 Big 128 512 130048 Scalable 1024 4096 2097152 Physical Partition: =================== You can change the NUMBER of PPs in a VG, but you cannot change the SIZE of PPs afterwards. Defaults: - 4 MB partition size. It can be a multiple of that amount. The Max size is 1024 MB - The default is 1016 PPs per disk. You can increase the number of PPs in powers of 2 per PV, but the number of maximum disks per VG is decreased. #disks
max # of PPs / disk
32 16 8 4 2 1
1016 2032 4064 8128 16256 32512
In the case of a set of "normal" internal disks of, for example, 30G or 70G or so, common partition sizes are 64M or 128M. Logical Partition: -----------------A LP maps to (at least) one PP, and is actually the smallest unit of allocatable space. Logical Volume: --------------Consists of LPs in a VG. A LV consists of LPs from actual PPs from one or more disks. |-----| |LP1 | ---> |-----| |LP2 | ---> |-----| |.. | |.. | |.. | |-----| |LPn | ---> |-----| |LPn+1| ---> |-----| Logical Volume
| | | | |
----| PP1 | ----| PP2 | ----| hdisk 1 (Physical Volume 1)
|---- | |PPn | |---- | |PPn+1| |---- | hdisk2 (Physical Volume 2)
So, a VG is a collection of related PVs, but you know that actually LVs are created in the VG. For the applications, the LVs are the entities they work with. In AIX, a filesystem like "/data", corresponds to a LV. lspv Command -----------Purpose: Displays information about a physical volume within a volume group. lspv [ -L ] [ -l | -p | -M ] [ -n DescriptorPhysicalVolume] [ -v VolumeGroupID] PhysicalVolume
-p: lists range, state, region, LV names, type and mount points # lspv # lspv hdisk3 # lspv -p hdisk3 # lspv hdisk0 hdisk1
00453267554 00465249766
rootvg rootvg
# lspv hdisk23 PHYSICAL VOLUME: hdisk23 VOLUME GROUP: PV IDENTIFIER: 00ccf45d564cfec0 VG IDENTIFIER 00ccf45d00004c0000000104564d2386 PV STATE: active STALE PARTITIONS: 0 ALLOCATABLE: PP SIZE: 256 megabyte(s) LOGICAL VOLUMES: TOTAL PPs: 947 (242432 megabytes) VG DESCRIPTORS: FREE PPs: 247 (63232 megabytes) HOT SPARE: USED PPs: 700 (179200 megabytes) FREE DISTRIBUTION: 00..00..00..57..190 USED DISTRIBUTION: 190..189..189..132..00 # lspv -p hdisk23: PP RANGE POINT 1-22 23-190 191-379 380-568 569-600 601-700 701-757 758-947
hdisk23
# lspv -p hdisk0: PP RANGE POINT 1-1 2-48 49-51 52-52 53-108 109-116 117-215 216-216 217-217 218-222 223-320 .. ..
hdisk0
oravg
yes 3 1 no
STATE
REGION
LV NAME
TYPE
MOUNT
used used used used used used free free
outer edge outer edge outer middle center inner middle inner middle inner middle inner edge
u01 u02 u01 u01 u02 u03
jfs2 jfs2 jfs2 jfs2 jfs2 jfs2
/u01 /u02 /u01 /u01 /u02 /u03
STATE
REGION
LV NAME
TYPE
MOUNT
used free used used used used used used used used used
outer edge outer edge outer edge outer edge outer edge outer middle outer middel center center center center
hd5
boot
N/A
hd9var hd2 hd6 hd6 hd2 hd8 hd4 hd2 hd4
jfs jfs paging paging jfs jfslog jfs jfs jfs
/var /usr N/A N/A /usr N/A / /usr /
Note that in this example the Logical Volumes corresponds to the filesystems in the following way: hd4= /, hd5=boot, hd6=paging, hd2=/usr, hd3=/tmp, hd9var=/var lslv Command -----------Purpose: Displays information about a logical volume. To Display Logical Volume Information lslv [ -L ] [ -l| -m ] [ -nPhysicalVolume ] LogicalVolume To Display Logical Volume Allocation Map lslv [ -L ] [ -nPhysicalVolume ] -pPhysicalVolume [ LogicalVolume ] # lslv -l lv06 lv06:/backups PV hdisk3
COPIES 512:000:000
IN BAND 100%
DISTRIBUTION 000:218:218:076:000
# lslv lv06 LOGICAL VOLUME: lv06 VOLUME GROUP: backupvg LV IDENTIFIER: 00c8132e00004c0000000106ef70cec2.2 PERMISSION: read/write VG STATE: active/complete LV STATE: opened/syncd TYPE: jfs WRITE VERIFY: off MAX LPs: 512 PP SIZE: 64 megabyte(s) COPIES: 1 SCHED POLICY: parallel LPs: 512 PPs: 512 STALE PPs: 0 BB POLICY: relocatable INTER-POLICY: minimum RELOCATABLE: yes INTRA-POLICY: middle UPPER BOUND: 32 MOUNT POINT: /backups LABEL: /backups MIRROR WRITE CONSISTENCY: on/ACTIVE EACH LP COPY ON A SEPARATE PV ?: yes Serialize IO ?: NO # lslv FREE 1-10 FREE 11-20 FREE 21-30 FREE 31-40 FREE 41-50 FREE 51-60
-p hdisk3 FREE FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE 61-70 FREE 71-80 FREE 81-90 .. ..
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
FREE
Also, you can list LVs per VG by running, for example: # lsvg -l backupvg backupvg: LV NAME POINT loglv02 lv06 # lsvg -l splvg splvg: LV NAME POINT loglv01 lv04 lv00 lv07
TYPE
LPs
PPs
PVs
LV STATE
MOUNT
jfslog jfs
1 512
1 512
1 1
open/syncd open/syncd
N/A /backups
TYPE
LPs
PPs
PVs
LV STATE
MOUNT
jfslog jfs jfs jfs
1 240 384 256
1 240 384 256
1 1 1 1
open/syncd open/syncd open/syncd open/syncd
N/A /data /spl /apps
For a complete storage system, this could yield in for example: -redovg: LV NAME POINT redo1lv redo2lv loglv03 -db2vg: LV NAME POINT db2lv /db2_database loglv00 -oravg: LV NAME POINT u01 u02 u03 logfs -rootvg: LV NAME POINT hd5 hd6 hd8 hd4
TYPE
LPs
PPs
PVs
LV STATE
MOUNT
jfs2 jfs2 jfs2log
42 1401 1
42 1401 1
3 3 1
open/syncd open/syncd open/syncd
/u05 /u04 N/A
TYPE
LPs
PPs
PVs
LV STATE
MOUNT
jfs2
600
600
2
open/syncd
jfs2log
1
1
1
open/syncd
N/A
TYPE
LPs
PPs
PVs
LV STATE
MOUNT
jfs2 jfs2 jfs2 jfs2log
800 400 200 2
800 400 200 2
2 2 2 1
open/syncd open/syncd open/syncd open/syncd
/u01 /u02 /u03 N/A
TYPE
LPs
PPs
PVs
LV STATE
MOUNT
boot paging jfs2log jfs2
1 36 1 8
2 72 2 16
2 2 2 3
closed/syncd open/syncd open/syncd open/syncd
N/A N/A N/A /
hd2 hd9var hd3 hd1 hd10opt fslv00 fslv01 paging00 sysdump1 oralv /opt/app/oracle fslv03
jfs2 jfs2 jfs2 jfs2 jfs2 jfs2 jfs2 paging sysdump jfs2
24 9 11 10 2 1 2 32 80 100
48 18 22 20 4 2 4 32 80 100
2 3 3 2 2 2 3 1 1 1
open/syncd open/syncd open/syncd open/syncd open/syncd open/syncd open/syncd open/syncd open/syncd open/syncd
/usr /var /tmp /home /opt /XmRec /tmp/m2 N/A N/A
jfs2
63
63
2
open/syncd
/bmc_home
And you can list the LVs by PV by running # lspv -l hdiskn lsvg Command: -------------o -p VG_name -l VG_name
Shows only the active volume groups. Shows all the PVs that belong to the vg_name Shows all the LVs that belong to the vg_name
Examples: # lsvg rootvg informixvg oravg # lsvg -o rootvg oravg # lsvg oravg VOLUME GROUP: oravg 00ccf45d00004c0000000104564d2386 VG STATE: active VG PERMISSION: read/write megabytes) MAX LVs: 256 megabytes) LVs: 4 megabytes) OPEN LVs: 4 TOTAL PVs: 2 STALE PVs: 0 ACTIVE PVs: 2 MAX PPs per PV: 1016 LTG size: 128 kilobyte(s) HOT SPARE: no # lsvg -p informixvg informixvg
VG IDENTIFIER: PP SIZE: TOTAL PPs:
256 megabyte(s) 1894 (484864
FREE PPs:
492 (125952
USED PPs:
1402 (358912
QUORUM: VG DESCRIPTORS: STALE PPs: AUTO ON: MAX PVs: AUTO SYNC: BB POLICY:
2 3 0 yes 32 no relocatable
PV_NAME PV STATE hdisk3 active 109..28..108..108..109 hdisk4 active 109..13..108..108..109
FREE PPs 462
542
447
# lsvg -l rootvg LV NAME TYPE POINT hd5 boot hd6 paging hd8 jfslog hd4 jfs hd2 jfs hd9var jfs hd3 jfs paging00 paging .. ..
TOTAL PPs 542
FREE DISTRIBUTION
LPs
PPs
PVs
LV STATE
MOUNT
1 24 1 4 76 4 6 20
1 24 1 4 76 4 6 20
1 1 1 1 1 1 1 1
closed/syncd open/syncd open/syncd open/synced open/synced open/synced open/synced open/synced
N/A N/A N/A / /usr /var /tmp N/A
Suppose we have 70GB disk=70000MB 1016 partitions=> 63 MB per PP extendvg command: ----------------extendvg VGName hdiskNumber # extendvg newvg hdisk23 How to Add a Disk to a Volume Group? extendvg
VolumeGroupName
hdisk0 hdisk1 ... hdiskn
reducevg command: ----------------To remove a PV from a VG: # reducevg myvg hdisk23 To remove a VG: Suppose we have a VG informixvg with 2 PV, hdisk3 and hdisk4: # reducevg -d informixvg hdisk4 When you delete the last disk from the VG, the VG is also removed. # reducevg -d informix hdisk3 varyonvg and varyoffvg commands: --------------------------------
When you activate a VG for use, all its resident filesystems are mounted by default if they have the flag mount=true in the /etc/filesystems file. # varyonvg apachevg # varyoffvg apachevg To use this command, you must be sure that none of the logical volumes are opened, that is, in use. mkvg command: ------------You can create a new VG by using "smitty mkvg" or by using the mkvg command. Use the following command, where s "partition_size" sets the number of megabytes in each physical partition where the partition_size is expressed in units of megabytes from 1 through 1024. The size variable must be equal to a power of 2 (for example 1, 2, 4, 8). The default value is 4. mkvg -y -s As with physical volumes, volume groups can be created and removed and their characteristics can be modified. Before a new volume group can be added to the system, one or more physical volumes not used in other volume groups, and in an available state, must exist on the system. The following example shows the use of the mkvg command to create a volume group myvg using the physical volumes hdisk1 and hdisk5. # mkvg -y myvg -d 10 -s 8 hdisk1 hdisk5 # mkvg -y oravg -d 10 -s 64 hdisk1
mklv command: ------------To create a LV, you can use the smitty command "smitty mklv" or just use the mklv command by itself. The mklv command creates a new logical volume within the VolumeGroup. For example, all file systems must be on separate logical volumes. The mklv command allocates the number of logical partitions
to the new logical volume. If you specify one or more physical volumes with the PhysicalVolume parameter, only those physical volumes are available for allocating physical partitions; otherwise, all the physical volumes within the volume group are available. The default settings provide the most commonly used characteristics, but use flags to tailor the logical volume to the requirements of your system. Once a logical volume is created, its characteristics can be changed with the chlv command. When you create a LV, you also specify the number of LP's, and how a LP maps to PP's. Later, you can create one filesystem per LV. Examples The following example creates a LV "lv05" on the VG "splvg", with two copies (2 PPs) of each LP. In this case, we are mirroring a LP to two PP's. Also, 200 PP's are specified. If a PP is 128 MB is size, the total amount of space of one "mirror" is 25600 MB. # mklv -y lv05 -c 2 splvg 200 The following example shows the use of mklv command to create a new LV newlv in the rootvg and it will have 10 LP's and each LP consists of 2 physical partitions. # mklv -y newlv -c 2 rootvg 10 To make a logical volume in volume group vg02 with one logical partition and a total of two copies of the data, enter: # mklv -c 2 vg02 1 To make a logical volume in volume group vg03 with nine logical partitions and a total of three copies spread across a maximum of two physical volumes, and whose allocation policy is not strict, enter: # mklv -c 3 -u 2 -s n vg03 9 To make a logical volume in vg04 with five logical partitions allocated across the center sections of the physical volumes when possible, with no bad-block relocation, and whose type is paging, enter: # mklv -a c -t paging -b n vg04 5 To make a logical volume in vg03 with 15 logical partitions chosen from physical volumes hdisk5, hdisk6, and hdisk9, enter: # mklv vg03 15 hdisk5 hdisk6 hdisk9
To make a striped logical volume in vg05 with a stripe size of 64K across 3 physical volumes and 12 logical partitions, enter: # mklv -u 3 -S 64K vg05 12 To make a striped logical volume in vg05 with a stripe size of 8K across hdisk1, hdisk2, and hdisk3 and 12 logical partitions, enter: # mklv -S 8K vg05 12 hdisk1 hdisk2 hdisk3 The following example uses a "map file /tmp/mymap1" which list which PPs are to be used in creating a LV: # mklv -t jfs -y lv06 -m /tmp/mymap1 rootvg 10 The setting Strict=y means that each copy of the LP is placed on a different PV. The setting Strict=n means that copies are not restricted to different PVs. The default is strict. # mklv -y lv13 -c 2 failovervg 150 # crfs -v jfs -d lv13 -m /backups2 -a bf=true Another simple example using local disks: # mkvg -y appsvg -s 32 hdisk2 # mkvg -y datavg -s 32 hdisk3 # mklv -y testlv -c 1 appsvg 10 # mklv -y backuplv -c 1 datavg 10 # crfs -v jfs -d testlv -m /test -a bf=true # crfs -v jfs -d backuplv -m /backup -a bf=true mklv mklv crfs crfs
-y -y -v -v
testlv1 -c 1 appsvg 10 testlv2 -c 1 datavg 10 jfs -d testlv1 -m /test1 -a bf=true jfs -d testlv2 -m /test2 -a bf=true
mklv mklv crfs crfs
-y -y -v -v
testlv1 -c 1 vgp0corddap01 10 testlv2 -c 1 vgp0corddad01 10 jfs -d testlv1 -m /test1 -a bf=true jfs -d testlv2 -m /test2 -a bf=true
rmlv command: ------------# rmlv newlv Warning, all data on logical volume newlv will be destroyed. rmlv: Do you wish to continue? y(es) n(o) y #
extendlv command: ----------------The following example shows the use of the extentlv command to add 3 more LP's to the LP newlv: # extendlv newlv 3 cplv command: ------------The following command copies the contents of LV oldlv to a new LV called newlv: # cplv -v myvg -y newlv oldlv To copy to an existing LV: # cplv -e existinglv oldlv Purpose Copies the contents of a logical volume to a new logical volume. Syntax To Copy to a New Logical Volume cplv [ -vg VolumeGroup ] [ -lv NewLogicalVolume | -prefix Prefix ] SourceLogicalVolume To Copy to an Existing Logical Volume cplv [ -f ] SourceLogicalVolume DestinationLogicalVolume cplv -e DestinationLogicalVolume [-f] SourceLogicalVolume -e: specifies that the DestinationLogicalVolume already exists. -f: no user confirmation -y: specifies the name to use for the NewLogicalVolume, instead of a system generated name. Description Attention: Do not copy from a larger logical volume containing data to a smaller one. Doing so results in a corrupted file system because some data is not copied. The cplv command copies the contents of SourceLogicalVolume to a new or existing logical volume. The SourceLogicalVolume parameter can be a logical volume name or a logical volume ID. The cplv command creates a new logical volume with a system-generated name by using the default syntax. The system-generated name is displayed. Note: The cplv command can not copy logical volumes which are in the open state, including logical volumes that are being used as backing devices for virtual storage. Flags
-f Copies to an existing logical volume without requesting user confirmation. -lv NewLogicalVolume Specifies the name to use, in place of a systemgenerated name, for the new logical volume. Logical volume names must be unique systemwide names, and can range from 1 to 15 characters. -prefix Prefix Specifies a prefix to use in building a system-generated name for the new logical volume. The prefix must be less than or equal to 13 characters. A name cannot be a name already used by another device. -vg VolumeGroup Specifies the volume group where the new logical volume resides. If this is not specified, the new logical volume resides in the same volume group as the SourceLogicalVolume. Examples To copy the contents of logical volume fslv03 to a new logical volume, type: # cplv fslv03 The new logical volume is created, placed in the same volume group as fslv03, and named by the system. To copy the contents of logical volume fslv03 to a new logical volume in volume group vg02, type: #cplv -vg vg02 fslv03 The new logical volume is created, named, and added to volume group vg02. #To copy the contents of logical volume lv02 to a smaller, existing logical volume, lvtest, without requiring user confirmation, type: cplv -f lv02 lvtest Errors: ------0516-746 cplv: Destination logical volume must have type set to copy chlv -t copy lvprj ======================================================================= === CASES of usage of cplv command: CASE 1: ------TITLE
: Procedure for moving a filesystem between disks that are in different volume groups using the cplv command. OS LEVEL : AIX 4.x
DATE VERSION
: 25/11/99 : 1.0
--------------------------------------------------------------------------In the following example, an RS6000 has 1 one disk with rootvg on, and has just had a second disk installed. The second disk needs a volume group creating on it and a data filesystem transferring to the new disk. Ensure that you have a full system backup befor you start. lspv hdisk0 hdisk1
00009922faf79f0d None
rootvg None
df -k Filesystem /dev/hd4 /dev/hd2 /dev/hd9var /dev/hd3 /dev/hd1 /dev/lv00 /dev/ftplv /dev/lv01
1024-blocks 8192 380928 32768 28672 53248 200704 102400 114688
Free %Used 1228 86% 40984 90% 20952 37% 1644 95% 51284 4% 110324 46% 94528 8% 58240 50%
Iused %Iused Mounted on 1647 41% / 11014 12% /usr 236 3% /var 166 3% /tmp 95 1% /home 1869 4% /home/john 32 1% /home/ftp 59 1% /usr2
In this example the /usr2 filesystem needs to be moved to the new disk drive, freeing up space in the root volume group. 1, Create a data volume group on the new disk (hdisk1), the command below will create a volume group called datavg on hdisk1 with a PP size of 32 Meg:mkvg -s 32 -y datavg hdisk1 2, Create a jfslog logical volume on the new volume group :mklv -y datalog -t jfslog datavg 1 3, Initialise the jfslog :logform /dev/datalog logform: destroy /dev/datalog (y)?y 4, Umount the filesystem that is being copied :umount /usr2
5, Copy the /usr2 logical volume (lv01) to a new logical volume (lv11) on the new volume group :cplv -y lv11 -v datavg lv01 cplv: Logical volume lv01 successfully copied to lv11 . 6, Change the /usr2 filesystem to use the new (/dev/lv11) logical volume and not the old (/dev/lv01) logical volume :chfs -a dev=/dev/lv11 /usr2 7, Change the /usr2 filesystem to use the jfslog on the new volume group (/dev/datalog) :chfs -a log=/dev/datalog /usr2 8, Mount the filesystem :mount /usr2 df -k Filesystem /dev/hd4 /dev/hd2 /dev/hd9var /dev/hd3 /dev/hd1 /dev/lv00 /dev/ftplv /dev/lv11
1024-blocks 8192 380928 32768 28672 53248 200704 102400 114688
Free %Used 1220 86% 40984 90% 20952 37% 1644 95% 51284 4% 110324 46% 94528 8% 58240 50%
Iused %Iused Mounted on 1649 41% / 11014 12% /usr 236 3% /var 166 3% /tmp 95 1% /home 1869 4% /home/john 32 1% /home/ftp 59 1% /usr2
9, Once the filesystem has been checked out, the old logical volume can be removed :rmfs /dev/lv01 Warning, all data contained on logical volume lv01 will be destroyed. rmlv: Do you wish to continue? y(es) n(o)? y rmlv: Logical volume lv01 is removed. If you wish to copy further filesystems repeat parts 4 to 9. ======================================================================= === CASE 2: ------Doel: -----
Een "move" van het /prj filesystem (met Websphere in /prj/was) op rootvg, naar een nieuw (groter en beter) volume group "wasvg". Het huidige /prj op rootvg, correspondeerd met de LV "prjlv". De nieuw te maken /prj op wasvg, correspondeerd met de LV "lvprj". ROOTVG WASVG --------------------------|/usr (hd2) | | | |.. | | | |/prj (prjlv)|----------->|/prj (lvprj) | |.. | | | -------------------------hdisk0,hdisk1 hdisk12,hdisk13 opm: /prj bevat "/prj/was", en dat is Websphere. Hier maken we geen gebruik van een backup tape. Gebruik het cplv command umount /prj chfs -m /prj_old /prj + mkvg -y wasvg -d 10 -s 128 hdisk12 hdisk13
-- maak VG aan
+ mklv -y lvprj -c 2 wasvg 400
-- maak LV aan
+ mklv -y waslog -t jfslog wasvg 1
-- maak een jfslog
+ logform /dev/waslog
-- init de log
cplv -e lvprj prjlv chfs -a dev=/dev/lvprj /prj_old
--
chfs -a log=/dev/waslog /prj_old chfs -m /prj /prj_old mount /prj ======================================================================= === migratepv command: -----------------Use the following command to move PPs from hdisk1 to hdisk6 and hdisk7 (all PVs must be in 1 VG) # migratepv hdisk1 hdisk6 hdisk7 Use the following command to move PPs in LV lv02 from hdisk1 to hdisk6 # migratepv -l lv02 hdisk1 hdisk6
chvg command: ------------This example multiplies by 2 the number of PPs: # chvg -t2 datavg chpv command: ------------The chpv command changes the state of the physical volume in a volume group by setting allocation permission to either allow or not allow allocation and by setting the availability to either available or removed. This command can also be used to clear the boot record for the given physical volume. Characteristics for a physical volume remain in effect unless explicitly changed with the corresponding flag. Examples To close physical volume hdisk03, enter: # chpv -v r hdisk03 The physical volume is closed to logical input and output until the -v a flag is used. To open physical volume hdisk03, enter: # chpv -v a hdisk03 The physical volume is now open for logical input and output. To stop the allocation of physical partitions to physical volume hdisk03, enter: # chpv -a n hdisk03 No physical partitions can be allocated until the -a y flag is used. To clear the boot record of a physical volume hdisk3, enter: # chpv -c hdisk3
How to synchronize stale partitions in a VG?: --------------------------------------------the syncvg command: syncvg Command Purpose Synchronizes logical volume copies that are not current. Syntax
syncvg [ -f ] [ -i ] [ -H ] [ -P NumParallelLps ] { -l | -p | -v } Name ... Description The syncvg command synchronizes the physical partitions, which are copies of the original physical partition, that are not current. The syncvg command can be used with logical volumes, physical volumes, or volume groups, with the Name parameter representing the logical volume name, physical volume name, or volume group name. The synchronization process can be time consuming, depending on the hardware characteristics and the amount of data. When the -f flag is used, a good physical copy is chosen and propagated to all other copies of the logical partition, whether or not they are stale. Using this flag is necessary in cases where the logical volume does not have the mirror write consistency recovery. Unless disabled, the copies within a volume group are synchronized automatically when the volume group is activated by the varyonvg command. Note: For the sycnvg command to be successful, at least one good copy of the logical volume should be accessible, and the physical volumes that contains this copy should be in ACTIVE state. If the -f option is used, the above condition applies to all mirror copies. If the -P option is not specified, syncvg will check for the NUM_PARALLEL_LPS environment variable. The value of NUM_PARALLEL_LPS will be used to set the number of logical partitions to be synchronized in parallel. Examples To synchronize the copies on physical volumes hdisk04 and hdisk05, enter: # syncvg -p hdisk04 hdisk05 To synchronize the copies on volume groups vg04 and vg05, enter: # syncvg -v vg04 vg05
How to Mirror a Logical Volume? : -------------------------------mklvcopy LogicalVolumeName Numberofcopies syncvg VolumeGroupName To add a copy for LV lv01 on disk hdisk7: # mklvcopy lv01 2 hdisk7
Identifying hotspots: lvmstat command: -------------------------------------The lvmstat command display statistics values since the previous lvmstat command. # lvmstat -v rootvg -e # lvmstat -v rootvg -C # lvmstat -v rootvg Logical Volume hd8 paging01 .. ..
iocnt 4 0
KB_read 0 0
KB_wrtn 0 0
Kbps 0.00 0.00
31.2 Mirroring a VG: ==================== LVM provide a disk mirroring facility at the LV level. Mirroring is the association of 2 or 3 PP's with each LP in a LV. Use the "mklv", or the "mklvcopy", or the "mirrorvg" command. The mklv command allows you to select one or two additional copies for each logical volume. example: To make a logical volume in volume group vg03 with nine logical partitions and a total of three copies spread across a maximum of two physical volumes, and whose allocation policy is not strict, enter: mklv -c 3 -u 2 -s n vg03 9 Mirroring can also be added to an existing LV using the mklvcopy command. The mirrorvg command mirrors all the LV's on a given VG. Examples: - To triply mirror a VG, run # mirrorvg -c 3 myvg - To get default mirroring of the rootvg, run # mirrorvg rootvg # # #
To replace a failed disk in a mirrored VG, run unmirrorvg workvg hdisk7 reducevg workvg hdisk7 rmdev -l hdisk7 -d
Now replace the failed disk with a new one and name it hdisk7 # extendvg workvg hdisk7
# mirrorvg workvg mirrorvg command: ----------------mirrorvg Command Purpose Mirrors all the logical volumes that exist on a given volume group. This command only applies to AIX 4.2.1 or later. Syntax mirrorvg [ -S | -s ] [ -Q ] [ -c Copies] [ -m ] VolumeGroup [ PhysicalVolume ... ] Description The mirrorvg command takes all the logical volumes on a given volume group and mirrors those logical volumes. This same functionality may also be accomplished manually if you execute the mklvcopy command for each individual logical volume in a volume group. As with mklvcopy, the target physical drives to be mirrored with data must already be members of the volume group. To add disks to a volume group, run the extendvg command. By default, mirrorvg attempts to mirror the logical volumes onto any of the disks in a volume group. If you wish to control which drives are used for mirroring, you must include the list of disks in the input parameters, PhysicalVolume. Mirror strictness is enforced. Additionally, mirrorvg mirrors the logical volumes, using the default settings of the logical volume being mirrored. If you wish to violate mirror strictness or affect the policy by which the mirror is created, you must execute the mirroring of all logical volumes manually with the mklvcopy command. When mirrorvg is executed, the default behavior of the command requires that the synchronization of the mirrors must complete before the command returns to the user. If you wish to avoid the delay, use the -S or -s option. Additionally, the default value of 2 copies is always used. To specify a value other than 2, use the -c option. Note: To use this command, you must either have root user authority or be a member of the system group. Attention: The mirrorvg command may take a significant amount of time before completing because
of complex error checking, the amount of logical volumes to mirror in a volume group, and the time is takes to synchronize the new mirrored logical volumes. You can use the Volumes application in Web-based System Manager (wsm) to change volume characteristics. You could also use the System Management Interface Tool (SMIT) smit mirrorvg fast path to run this command. Flags -c Copies Specifies the minimum number of copies that each logical volume must have after the mirrorvg command has finished executing. It may be possible, through the independent use of mklvcopy, that some logical volumes may have more than the minimum number specified after the mirrorvg command has executed. Minimum value is 2 and 3 is the maximum value. A value of 1 is ignored. -m exact map Allows mirroring of logical volumes in the exact physical partition order that the original copy is ordered. This option requires you to specify a PhysicalVolume(s) where the exact map copy should be placed. If the space is insufficient for an exact mapping, then the command will fail. You should add new drives or pick a different set of drives that will satisfy an exact logical volume mapping of the entire volume group. The designated disks must be equal to or exceed the size of the drives which are to be exactly mirrored, regardless of if the entire disk is used. Also, if any logical volume to be mirrored is already mirrored, this command will fail. -Q Quorum Keep By default in mirrorvg, when a volume group's contents becomes mirrored, volume group quorum is disabled. If the user wishes to keep the volume group quorum requirement after mirroring is complete, this option should be used in the command. For later quorum changes, refer to the chvg command. -S Background Sync Returns the mirrorvg command immediately and starts a background syncvg of the volume group. With this option, it is not obvious when the mirrors have completely finished their synchronization. However, as portions of the mirrors become synchronized, they are immediately used by the operating system in mirror usage. -s Disable Sync Returns the mirrorvg command immediately without performing any type of mirror synchronization. If this option is used, the mirror may exist for a logical volume but is not used by the operating system until it has been synchronized with the syncvg command. The following is a description of rootvg:
- rootvg mirroring When the rootvg mirroring has completed, you must perform three additional tasks: bosboot, bootlist, and reboot. The bosboot command is required to customize the bootrec of the newly mirrored drive. The bootlist command needs to be performed to instruct the system which disk and order you prefer the mirrored boot process to start. Finally, the default of this command is for Quorum to be turned off. For this to take effect on a rootvg volume group, the system must be rebooted. - non-rootvg mirroring When this volume group has been mirrored, the default command causes Quorum to deactivated. The user must close all open logical volumes, execute varyoffvg and then varyonvg on the volume group for the system to understand that quorum is or is not needed for the volume group. If you do not revaryon the volume group, mirror will still work correctly. However, any quorum changes will not have taken effect. rootvg and non-rootvg mirroring The system dump devices, primary and secondary, should not be mirrored. In some systems, the paging device and the dump device are the same device. However, most users want the paging device mirrored. When mirrorvg detects that a dump device and the paging device are the same, the logical volume will be mirrored automatically. If mirrorvg detects that the dump and paging device are different logical volumes, the paging device is automatically mirrored, but the dump logical volume is not. The dump device can be queried and modified with the sysdumpdev command. Remark: ------Run bosboot to initialize all boot records and devices by executing the following command: bosboot -a -d /dev/hdisk? hdisk? is the first hdisk listed under the PV heading after the command lslv -l hd5 has executed. Secondary, you need to understant that the mirroring under AIX it's at the logical volume level. The mirrorvg command is a hight level command that use "mklvcopy" command. So, all LV created before runing the mirrorvg command are keep synchronised, but if you add a new LV after runing mirrorvg, you need to mirror it manualy using "mklvcopy" . Remark: ------lresynclv
Mirroring the rootvg: --------------------Method 1: --------Howto mirror an AIX rootvg The following steps will guide you trough the mirroring of an AIX rootvg. This info is valid for AIX 4.3.3, AIX 5.1, AIX 5.2 and AIX 5.3. Make sure you have an empty disk, in this example its hdisk1 Add the disk to the vg via # extendvg rootvg hdisk1 Mirror the vg via: # mirrorvg -s rootvg Now synchronize the new copies you created: # syncvg -v rootvg As we want to be able to boot from different disks, we need to use bosboot: # bosboot -a As hd5 is mirrored there is no need to do it for each disk. Now, update the bootlist: # bootlist -m normal hdisk1 hdisk0 # bootlist -m service hdisk1 hdisk0 When mirrorvg is executed, the default behavior of the command requires that the synchronization of the mirrors must complete before the command returns to the user. If you wish to avoid the delay, use the -S or -s option. Additionally, the default value of 2 copies is always used. To specify a value other than 2, use the -c option. Method 2: -------------------------------------------------------------------------------------# Add the new disk, say its hdisk5, to rootvg extendvg rootvg hdisk5
# If you use one mirror disk, be sure that a quorum is not required for varyon: chvg -Qn rootvg # Add the mirrors for all rootvg LV's: mklvcopy mklvcopy mklvcopy mklvcopy mklvcopy mklvcopy mklvcopy mklvcopy mklvcopy mklvcopy
hd1 2 hdisk5 hd2 2 hdisk5 hd3 2 hdisk5 hd4 2 hdisk5 hd5 2 hdisk5 hd6 2 hdisk5 hd8 2 hdisk5 hd9var 2 hdisk5 hd10opt 2 hdisk5 prjlv 2 hdisk5
#If you have other LV's in your rootvg, be sure to create copies for them as well !! ----------------------------------------------------------------------------# lspv -l hdisk0 hd5 prjlv hd6 fwdump /var/adm/ras/platform hd8 hd4 hd2 hd9var hd3 hd1 hd10opt
1 256 59 5
1 256 59 5
01..00..00..00..00 108..44..38..50..16 00..59..00..00..00 00..05..00..00..00
N/A /prj N/A
1 26 45 10 22 8 24
1 26 45 10 22 8 24
00..00..01..00..00 00..00..02..24..00 00..00..37..08..00 00..00..02..08..00 00..00..04..10..08 00..00..08..00..00 00..00..16..08..00
N/A / /usr /var /tmp /home /opt
Method 3: --------In the following example, an RS6000 has 3 disks, 2 of which have the AIX filesystems mirrored on. The boolist contains both hdisk0 and hdisk1. There are no other logical volumes in rootvg other than the AIX system logical volumes. hdisk0 has failed and need replacing, both hdisk0 and hdisk1 are in "Hot Swap" carriers and therefore the machine does not need shutting down. lspv hdisk0 hdisk1 hdisk2
00522d5f22e3b29d 00522d5f90e66fd2 00522df586d454c3
rootvg rootvg datavg
lsvg -l rootvg rootvg: LV NAME POINT hd6 hd5 hd8 hd4 hd2 hd9var hd3 hd1
TYPE
LPs
PPs
PVs
LV STATE
MOUNT
paging boot jfslog jfs jfs jfs jfs jfs
4 1 1 1 12 1 2 1
8 2 2 2 24 2 4 2
2 2 2 2 2 2 2 2
open/syncd closed/syncd open/syncd open/syncd open/syncd open/syncd open/syncd open/syncd
N/A N/A N/A / /usr /var /tmp /home
1, Reduce the logical volume copies from both disks to hdisk1 only :rmlvcopy rmlvcopy rmlvcopy rmlvcopy rmlvcopy rmlvcopy rmlvcopy rmlvcopy
hd6 1 hdisk0 hd5 1 hdisk0 hd8 1 hdisk0 hd4 1 hdisk0 hd2 1 hdisk0 hd9var 1 hdisk0 hd3 1 hdisk0 hd1 1 hdisk0
2, Check that no logical volumes are left on hdisk0 :lspv -p hdisk0 hdisk0: PP RANGE 1-101 102-201 202-301 302-401 402-501
STATE free free free free free
REGION outer edge outer middle center inner middle inner edge
LV ID
TYPE
MOUNT POINT
3, Remove the volume group from hdisk0 reducevg -df rootvg hdisk0 4, Recreate the boot logical volume on hdisk1, and reset bootlist:bosboot -a -d /dev/hdisk1 bootlist -m normal rmt0 cd0 hdisk1 5, Check that everything has been removed from hdisk0 :lspv hdisk0 hdisk1 hdisk2
00522d5f22e3b29d 00522d5f90e66fd2 00522df586d454c3
6, Delete hdisk0 :-
None rootvg datavg
rmdev -l hdisk0 -d 7, Remove the failed hard drive and replace with a new hard drive. 8, Configure the new disk drive :cfgmgr 9, Check new hard drive is present :lspv 10, Include the new hdisk in root volume group :extendvg rootvg hdisk?
(where hdisk? is the new hard disk)
11, Re-create the mirror :mirrorvg rootvg hdisk?
(where hdisk? is the new hard disk)
12, Syncronise the mirror :syncvg -v rootvg 13, Reset the bootlist :bootlist -m normal rmt0 cd0 hdisk0 hdisk1 14, Turn off Quorum checking on rootvg :chvg -Q n rootvg Method 4: --------Howto mirror an AIX rootvg The following steps will guide you trough the mirroring of an AIX rootvg. This info is valid for AIX 4.3.3, AIX 5.1, AIX 5.2 and AIX 5.3. Make sure you have an empty disk, in this example its hdisk1 Add the disk to the vg via "extendvg rootvg hdisk1 Mirror the vg via: "mirrorvg rootvg" Adapt the bootlist to add the current disk, the system will then fail to hdisk1 is hdisk0 fails during startup do bootlist -o -m normal this will list currently 1 disk, in this exmaple hdisk0 do bootlist -m normal hdisk0 hdisk1 Run a bosboot on both new disks, this will install all software needed for boot on the disk bosboot -ad hdisk0 bosboot -ad hdisk1 Method 5:
--------Although the steps to mirror volume groups between HP and AIX are incredibly similar, there are enough differences to send me through hoops if/when I ever have to do that. Therefore, the following checklist: 1. Mirror the logical volumes: If you don't care what disks the lvs get mirrored to, execute mirrorvg rootvg Otherwise: for lv in $(lsvg -l rootvg | grep -i open/syncd | \ grep -v dumplv | awk '{print $1}') do mklvcopy ${lv} 1 ${disk} done 2. Change the quorum checking if you did not use mirrorvg: chvg -Q n rootvg 3. Run bosboot on the new drive to copy boot files to it: bosboot ${disk} 4. Update the bootlist with the new drive: bootlist -m normal hdisk0 hdisk1 5. Reboot the system to enable the new quorum checking parameter Method 6: --------Audience: System Administrators Date: September 25, 2002 Mirroring "rootvg" protects the operating system from a disk failure. Mirroring "rootvg" requires a couple extra steps compared to other volume groups. The mirrored rootvg disk must be bootable *and* in the bootlist. Otherwise, if the primary disk fails, you'll continue to run, but you won't be able to reboot. In brief, the procedure to mirror rootvg on hdisk0 to hdisk1 is
1. Add hdisk1 to rootvg: extendvg rootvg hdisk1 2. Mirror rootvg to hdisk1: mirrorvg rootvg hdisk1 (or smitty mirrorvg) 3. Create boot images on hdisk1: bosboot -ad /dev/hdisk1 4. Add hdisk1 to the bootlist: bootlist -m normal hdisk0 hdisk1 5. Reboot to disable quorum checking on rootvg. The mirrorvg turns off quorum by default, but the system needs to be rebooted for it to take effect. For more information, and a comprehensive procedure see the man page for mirrorvg and
Example using mklvcopy: ----------------------mklvcopy [ -a Position ] [ -e Range ] [ -k ] [ -m MapFile ] [ -s Strict ] [ -u UpperBound ] LogicalVolume Copies [ PhysicalVolume... ] Add a copy of LV "lv01" on disk hdisk7: # mklvcopy lv01 2 hdisk7 The mklvcopy command increases the number of copies in each logical partition in LogicalVolume. This is accomplished by increasing the total number of physical partitions for each logical partition to the number represented by Copies. The LogicalVolume parameter can be a logical volume name or logical volume ID. You can request that the physical partitions for the new copies be allocated on specific physical volumes (within the volume group) with the PhysicalVolume parameter; otherwise, all the physical volumes within the volume group are available for allocation. The logical volume modified with this command uses the Copies parameter as its new copy characteristic. The data in the new copies are not synchronized until one of the following occurs: the -k option is used, the volume group is activated by the varyonvg command, or the volume group or logical volume is synchronized explicitly by the syncvg command. Individual logical partitions are always updated as they are written to.
The default allocation policy is to use minimum numbering of physical volumes per logical volume copy, to place the physical partitions belong to a copy as contiguously as possible, and then to place the physical partitions in the desired region specified by the -a flag. Also, by default, each copy of a logical partition is placed on a separate physical volume.
Using smitty: ------------# smit mklv or # smit mklvcopy Using "smit mklv" you can create a new LV and at the same time tell the system to create a mirror (2 or 3 copies) of each LP and which PV's are involved. Using "smit mklvcopy" you can add mirrors to an existing LV.
31.3 Filesystems in AIX: ======================== After a VG is created, you can create filesystems. You can use smitty or the crfs and mkfs command. File systems are confined to a single logical volume. The journaled file system (JFS) and the enhanced journaled file system (JFS2) are built into the base operating system. Both file system types link their file and directory data to the structure used by the AIX Logical Volume Manager for storage and retrieval. A difference is that JFS2 is designed to accommodate a 64-bit kernel and larger files. Run lsfs -v jfs2 to determine if your system uses JFS2 file systems. This command returns no output if it finds only standard file systems. crfs: ----crfs -v VfsType { -g -m MountPoint [ no } ] [ -p {ro | rw [ -a Attribute=
VolumeGroup | -d Device } [ -l LogPartitions ] -n NodeName ] [ -u MountGroup ] [ -A { yes | } ] Value ... ] [ -t { yes | no } ]
The crfs command creates a file system on a logical volume within a previously created volume group. A new logical volume is created for the file system unless the name of an existing logical volume is specified using the -d. An entry for the file system is put into the /etc/filesystems file. crfs -v jfs -g(vg) -m(mount point) -a size=(size Will create a logical volume on the volume group system on the logical volume. All at the size stated. Will /etc/filesystems and will create the mount point not exist.
of fs) -A yes and create the file add entry into directory if it does
- To make a JFS on the rootvg volume group with nondefault fragment size and nondefault nbpi, enter: # crfs -v jfs -g rootvg -m /test -a size=32768 -a frag=512 -a nbpi=1024 This command creates the /test file system on the rootvg volume group with a fragment size of 512 bytes, a number of bytes per i-node (nbpi) ratio of 1024, and an initial size of 16MB (512 * 32768). - To make a JFS on the rootvg volume group with nondefault fragment size and nondefault nbpi, enter: # crfs -v jfs -g rootvg -m /test -a size=16M -a frag=512 -a nbpi=1024 This command creates the /test file system on the rootvg volume group with a fragment size of 512 bytes, a number of bytes per i-node (nbpi) ratio of 1024, and an initial size of 16MB. - To create a JFS2 file system which can support NFS4 ACLs, type: # crfs -v jfs2 -g rootvg -m /test -a size=1G -a ea=v2 - This command creates the /test JFS2 file system on the rootvg volume group with an initial size of 1 gigabyte. The file system will store extended attributes using the v2 format. # crfs -v jfs -g backupvg -m /backups -a size=32G -a bf=true # crfs -v jfs -g oravg -m /filetransfer -a size=4G -a bf=true Extended example: ----------------The following command creates a JFS filesystem on a previously created LV "lv05". In this example, suppose the LV was created in the following way: # mklv -y lv05 -c 2 splvg 200 In this case, it is clear that we mirror each LP to 2 PP's (because of the -c 2).
Now to create a filesystem on lv05, we can use the command # crfs -v jfs -d lv05 -m /spl -a bf=true Note that we did not mentioned the size of the filesystem. This is because we use a previously defined LV with a known size. Notes: 1. The option -a bf=true allows large files [ > 2Gb]; 2. Specifying -m / (like for example "/data") will create the entry in /etc/filesystems for you Some more examples: ------------------Commands to create VG's: mkvg oravg -d 10 -s 128 hdisk2 hdisk4 mkvg splvg -d 10 -s 128 hdisk3 hdisk5 mkvg softwvg -d 10 -s 128 hdisk6 mkvg backupvg -d 10 -s 128 hdisk7 Set of Create Logical Volume and Filesystem commands: # crfs -v jfs -g -m -a size=xG -a bf=true or # mklv -y -c 2 No_Of_PPs # crfs -v jfs -d -m -a bf=true # # # #
mklv crfs mklv crfs
-y -v -y -v
lv05 -c 2 splvg 300 jfs -d lv05 -m /spl -a bf=true lv06 -c 2 splvg 100 jfs -d lv06 -m /u04 -a bf=true
# # # # # #
mklv mklv mklv crfs crfs crfs
-y -y -y -v -v -v
lv02 -c 2 oravg 200 lv03 -c 2 oravg 200 lv04 -c 2 oravg 200 jfs -d lv02 -m /u01 -a bf=true jfs -d lv03 -m /u02 -a bf=true jfs -d lv04 -m /u03 -a bf=true
# # # # #
crfs crfs crfs crfs crfs
-v -v -v -v -v
jfs jfs jfs jfs jfs
mkfs: -----
-g -g -g -g -g
backupvg -m /backups -a size=33G -a bf=true backupvg -m /data -a size=33G -a bf=true softwvg -m /apps -a size=16G -a bf=true softwvg -m /software -a size=33G -a bf=true softwvg -m /u05 -a size=12G -a bf=true
The mkfs command makes a new file system on a specified device. The mkfs command initializes the volume label, file system label, and startup block. The Device parameter specifies a block device name, raw device name, or file system name. If the parameter specifies a file system name, the mkfs command uses this name to obtain the following parameters from the applicable stanza in the /etc/filesystems file, unless these parameters are entered with the mkfs command. - To specify the volume and file system name for a new file system, type: # mkfs -lworks -vvol001 /dev/hd3 This command creates an empty file system on the /dev/hd3 device, giving it the volume serial number vol001 and file system name works. The new file system occupies the entire device. The file system has a default fragment size (4096 bytes) and a default nbpi ratio (4096). - To create a file system with nondefault attributes, type: # mkfs -s 8192 -o nbpi=2048,frag=512 /dev/lv01 This command creates an empty 4 MB file system on the /dev/lv01 device with 512-byte fragments and 1 i-node for each 2048 bytes. -To create a large file enabled file system, type: # mkfs -V jfs -o nbpi=131072,bf=true,ag=64 /dev/lv01 This creates a large file enabled JFS file system with an allocation group size of 64 megabytes and 1 inode for every 131072 bytes of disk. The size of the file system will be the size of the logical volume lv01. - To create a file system with nondefault attributes, type: # mkfs -s 4M -o nbpi=2048, frag=512 /dev/lv01 This command creates an empty 4 MB file system on the /dev/lv01 device with 512-byte fragments and one i-node for each 2048 bytes. - To create a JFS2 file system which can support NFS4 ACLs, type: # mkfs -V jfs2 -o ea=v2 /dev/lv01 This command creates an empty file system on the /dev/lv01 device with v2 format for extended attributes. chfs command: ------------- Example 1: How do I change the size of a filesystem?
To increase /usr filesystem size by 1000000 512-byte blocks, type: # chfs -a size=+1000000 /usr - Example 2: To split off a copy of a mirrored file system and mount it read-only for use as an online backup, enter: # chfs -a splitcopy=/backup -a copy=2 /testfs This mount a read-only copy of /testfs at /backup. - Example 3: To change the mount point of a file system, enter: # chfs -m /test2 /test This command changes the mount point of a file system from /test to /test2. - Eaxample 4: # chfs -a size=+20G /data/udb/eidwha2/eddwha2/DATA03 - Example 5: chfs -a size=+5M /opt
would do it this way: 1) chfs -m old_filename new_filename 2) umount old_filename 3) mount new_filename To stop or kill access to a fs, use: fuser -xuc /scratch
lsfs command: ------------Displays the characteristics of file systems. Syntax lsfs [ -q ] [ -c | -l ] [ -a | -v VfsType | -u MountGroup| [FileSystem...] ] Description The lsfs command displays characteristics of file systems, such as mount points, automatic mounts, permissions, and file system size. The FileSystem parameter reports on a specific file system. The following subsets can be queried for a listing of characteristics: All file systems
All file systems of a certain mount group All file systems of a certain virtual file system type One or more individual file systems The lsfs command displays additional Journaled File System (JFS) or Enhanced Journaled File System (JFS2) characteristics if the -q flag is specified. To show all file systems in the /etc/filesystems file, enter: #lsfs To show all file systems of vfs type jfs, enter: #lsfs -v jfs To show the file system size, the fragment size, the compression algorithm (if any), and the number of bytes per i-node as recorded in the superblock of the root file system, enter: #lsfs -q /
31.4 SAN connection via SDD, and related commands: ================================================== If you use advanced storage on AIX, the workings on disks and volume groups are a bit different from the traditional ways, using local disks, as described above. You can use SDD or SDDPCM Multipath IO. This section describes SDD. See section 31.5 for SDDPCM. Overview of the Subsystem device driver: ---------------------------------------The IBM System Storage Multipath Device Driver SDD provides multipath configuration environment support for a host system that is attached to storage devices. It provides: -Enhanced data availability -Automatic path failover and recovery to an alternate path -Dynamic load balancing of multiple paths -Concurrent microcode upgrade. The IBM System Storage Multipath Subsystem Device Driver Path Control Module SDDPCM provides AIX MPIO support. Its a loadable module. During the configuration of supported devices, SDDPCM is loaded and becomes part of the AIX MPIO Fibre Channel protocol device driver. The AIX MPIO-capable device driver with the SDDPCM module provides the same functions that SDD provides. Note that before attempting to exploit the Virtual shared disk support for the Subsystem device driver, you must read IBM Subsystem Device Driver Installation and User's Guide.
An SDD implementation is available for AIX, Solaris, HP-UX, some Linux distro's, Windows 200x. An impression about the architecture on AIX can be seen in the following figure: ------------------------------| Host System | | ------------- | | |FC 0 | | FC 1| | | ------------- | ------------------------------| | | | ---------------------------------ESS | --------------| | |port 0| |port 1| | | -------- \ /-------| | | \ / | | | | \/ | | | | / \ | | | -----------/ \---------- | | |Cluster 1| |Cluster 2|| | ---------------------| | | | | | | | | | | | | | | | | | | | | | O--|--|--|-------| | | | | | lun0| | | | | | | | O--|--|---------| | | | | lun1| | | | | | O--|-----------| | | | lun2| | | | O--------------| | | lun3 | --------------------------------DPO (Data Path Optimizer) was renamed by IBM a couple years ago- and became SDD (Subsystem Device Driver). When redundant paths are configured to ESS logical units, and the SDD is installed and configured, the AIX(R) lspv command shows multiple hdisks as well as a new construct called a vpath. The hdisks and vpaths represent the same logical unit. You will need to use the lsvpcfg command to get more information. Each SDD vpath device represents a unique physical device on the storage server. Each physical device is presented to the operating system as an operating system disk device. So, essentially, a vpath device acts like a disk. You will see later on that a hdisk is actually a "path" to a LUN, that can be reached either by fscsi0 or fscsi1. Also you will see that a vpath represents the LUN.
SDD does not support multipathing to a bootdevice. Support for VIO: ---------------Starting from SDD version 1.6.2.0, a unique ID attribute is added to SDD vpath devices, in order to support AIX5.3 VIO future features. AIX device configure methods have been changed in both AIX52 TL8 and AIX53 TL4 for this support. Examples: --------For example, after issuing lspv, you see output similar to this: # lspv hdisk0 hdisk1 hdisk18 hdisk19 hdisk20 hdisk21 hdisk22 hdisk23 hdisk24 vpath0 vpath1 vpath2 vpath3
000047690001d59d 000047694d8ce8b6 000047694caaba22 000047694caadf9a none none 000047694cab2963 none none none none 000047694cab0b35 000047694cab1d27
rootvg None None None None None None None None None None gpfs1scsivg gpfs1scsivg
After issuing lsvpcfg, you see output similar to this: # lsvpcfg vpath0 (Avail ) 502FCA01 = hdisk18 (Avail pv ) vpath1 (Avail ) 503FCA01 = hdisk19 (Avail pv ) vpath2 (Avail pv gpfs1scsivg) 407FCA01 = hdisk20 (Avail ) hdisk24 (Avail ) The examples above illustrate some important points: - vpath0 consists of a single path (hdisk18) and therefore will not provide failover protection. Also, hdisk18 is defined to AIX as a physical volume (pv flag) and has a PVID, as you can see from the output of the lspv command. Likewise for vpath1. - vpath2 has two paths (hdisk20 and hdisk24) and has a volume group defined on it. Notice that with the lspv command, hdisk20 and hdisk24 look like newly installed disks with no PVIDs. The lsvpcfg command had to be used to determine that hdisk20 and hdisk24 make up vpath2, which has a PVID.
Warning: so be very carefull not to use a hdisk for a "local" VG, if its already used for a vpath. Other Example: -------------# lspv hdisk0 active hdisk1 active -hdisk2 -hdisk3 vpath0 active vpath1 active -hdisk4 vpath2 active -hdisk5 -hdisk6 -hdisk7
00c49e8c8053fe86
rootvg
00c49e8c841a74d5
rootvg
none none 00c49e8c94c02c15
None None datavg
00c49e8c94c050d4
appsvg
none 00c49e8c2806dc22
None appsvg
none none none
None None None
# lsvpcfg vpath0 (Avail pv datavg) 75BAFX1006C = hdisk2 (Avail ) hdisk5 (Avail ) vpath1 (Avail pv appsvg) 75BAFX1017B = hdisk3 (Avail ) hdisk6 (Avail ) vpath2 (Avail pv appsvg) 75BAFX10329 = hdisk4 (Avail ) hdisk7 (Avail ) # datapath query adapter Active Adapters :2 Adpt# Active 0 3 1 3
Name
State
Mode
Select
Errors
Paths
fscsi0
NORMAL
ACTIVE
12611291
0
3
fscsi1
NORMAL
ACTIVE
13375287
0
3
# datapath query device Total Devices : 3 DEV#: 0 DEVICE NAME: vpath0 TYPE: 2107900 POLICY: Optimized # this is vpath0 SERIAL: 75BAFX1006C ======================================================================= ===
Path# Errors 0 0 1 0
Adapter/Hard Disk
State
Mode
Select
fscsi0/hdisk2
OPEN
NORMAL
12561763
fscsi1/hdisk5
OPEN
NORMAL
13324883
DEV#: 1 DEVICE NAME: vpath1 TYPE: 2107900 POLICY: Optimized SERIAL: 75BAFX1017B ======================================================================= === Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk3 OPEN NORMAL 28024 0 1 fscsi1/hdisk6 OPEN NORMAL 28847 0 DEV#: 2 DEVICE NAME: vpath2 TYPE: 2107900 POLICY: Optimized SERIAL: 75BAFX10329 ======================================================================= === Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk4 OPEN NORMAL 21672 0 1 fscsi1/hdisk7 OPEN NORMAL 21712 0 # lsattr -El vpath0 active_hdisk hdisk2/75BAFX1006C/fscsi0 False active_hdisk hdisk5/75BAFX1006C/fscsi1 False policy df True pvid 00c49e8c94c02c150000000000000000 identifier False serial_number 75BAFX1006C False # lsdev -Cc adapter ent0 Available ent1 Available fcs0 Available fcs1 Available sa0 Available sisscsia0 Available
10/100/1000 Base-TX 10/100/1000 Base-TX FC Adapter FC Adapter LPAR Virtual Serial 03-08 PCI-X Ultra320 SCSI
# lsattr -El fcs0 bus_intr_lvl 131193 False
04-08 06-08 05-08 07-08
Bus interrupt level
Active hdisk Active hdisk Scheduling Policy Physical volume LUN serial number
PCI-X Adapter (14106902) PCI-X Adapter (14106902) Adapter Adapter
bus_io_addr False bus_mem_addr False init_link True intr_priority False lg_term_dma True max_xfer_size True num_cmd_elems adapter True pref_alpa True sw_fc_class True
0xcfc00
Bus I/O address
0xc0040000 Bus memory address al
INIT Link flags
3
Interrupt priority
0x800000
Long term DMA
0x100000
Maximum Transfer Size
200
Maximum number of COMMANDS to queue to the
0x1
Preferred AL_PA
2
FC Class for Fabric
# lscfg -lv fcs0 fcs0
U7879.001.DQDKCPR-P1-C2-T1
FC Adapter
Part Number.................03N6441 EC Level....................A Serial Number...............1D54508045 Manufacturer................001D Feature Code................280B FRU Number.................. 03N6441 Device Specific.(ZM)........3 Network Address.............10000000C94F91CD ROS Level and ID............0288193D Device Specific.(Z0)........1001206D Device Specific.(Z1)........00000000 Device Specific.(Z2)........00000000 Device Specific.(Z3)........03000909 Device Specific.(Z4)........FF801412 Device Specific.(Z5)........0288193D Device Specific.(Z6)........0683193D Device Specific.(Z7)........0783193D Device Specific.(Z8)........20000000C94F91CD Device Specific.(Z9)........TS1.90X13 Device Specific.(ZA)........T1D1.90X13 Device Specific.(ZB)........T2D1.90X13 Device Specific.(YL)........U7879.001.DQDKCPR-P1-C2-T1 # lsdev -Cc adapter -F 'name parent' ent0 pci4 ent1 pci6 fcs0 pci5 fcs1 pci7 sa0 sisscsia0 pci3 # lsdev -Cc disk -F 'name location'
hdisk0 hdisk1 hdisk2 hdisk3 hdisk4 hdisk5 hdisk6 hdisk7 vpath0 vpath1 vpath2
03-08-00-3,0 03-08-00-5,0 05-08-01 ------------------------>| 05-08-01 ------------------------>| 05-08-01 ------------------------>| 07-08-01 | 07-08-01 | 07-08-01 | | | | | | # lsdev -Cc driver -F 'name location' | dpo | fcnet0 05-08-02 | fcnet1 07-08-02 | fscsi0 05-08-01 <------------------------fscsi1 07-08-01 iscsi0 scsi0 03-08-00 Please note that, for example, from the above output, that fsci0 can be "linked" to hdisk2, hdisk3 and hdisk4, due to the location code. You can compare that to the output of "datapath query device". Also interesting can be the following: # lsdev -C | grep fc fcnet0 Defined Device fcnet1 Defined Device fcs0 Available fcs1 Available
05-08-02
Fibre Channel Network Protocol
07-08-02
Fibre Channel Network Protocol
05-08 07-08
FC Adapter FC Adapter
# lsdev -C | grep fsc fscsi0 Available 05-08-01 Device fscsi1 Available 07-08-01 Device
FC SCSI I/O Controller Protocol FC SCSI I/O Controller Protocol
From this, you can see that fcs0 is the "parent" of the child "fsci0". # lsattr -D -l fscsi0 attach none dyntrk no fc_err_recov delayed_fail scsi_id sw_fc_class 3
How this adapter is CONNECTED False Dynamic Tracking of FC Devices True FC Fabric Event Error RECOVERY Policy True Adapter SCSI ID False FC Class for Fabric True
# lsattr -D -l fcs0 bus_intr_lvl Bus interrupt level Fals bus_io_addr 0x00010000 Bus I/O address Fals
e e
bus_mem_addr Fals init_link True intr_priority Fals lg_term_dma True max_xfer_size True num_cmd_elems adapter True pref_alpa True sw_fc_class True
0x01000000 Bus memory address
e
al
INIT Link flags
3
Interrupt priority
0x800000
Long term DMA
0x100000
Maximum Transfer Size
200
Maximum number of COMMANDS to queue to the
0x1
Preferred AL_PA
2
FC Class for Fabric
e
# datapath query essmap Disk Path P Location adapter LUN SN Type Size LSS Vol Rank C/A S Connection port RaidMode ----------- ----------- ----------------------------------- ----- ---- ----------- ----------vpath0 hdisk2 05-08-01[FC] fscsi0 75BAFX1006C IBM 2107900 107.5GB 0 108 fff2 02 Y R1-B3-H3-ZC 232 RAID5 vpath0 hdisk5 07-08-01[FC] fscsi1 75BAFX1006C IBM 2107900 107.5GB 0 108 fff2 02 Y R1-B3-H3-ZA 230 RAID5 vpath1 hdisk3 05-08-01[FC] fscsi0 75BAFX1017B IBM 2107900 14.3GB 1 123 fff1 0b Y R1-B3-H3-ZC 232 RAID5 vpath1 hdisk6 07-08-01[FC] fscsi1 75BAFX1017B IBM 2107900 14.3GB 1 123 fff1 0b Y R1-B3-H3-ZA 230 RAID5 vpath2 hdisk4 05-08-01[FC] fscsi0 75BAFX10329 IBM 2107900 14.3GB 3 41 ffe1 08 Y R1-B3-H3-ZC 232 RAID5 vpath2 hdisk7 07-08-01[FC] fscsi1 75BAFX10329 IBM 2107900 14.3GB 3 41 ffe1 08 Y R1-B3-H3-ZA 230 RAID5 From this you can see that a hdisk is actually a "path" to a LUN, that can be reached either by fscsi0 or fscsi1. Also you can see that a vpath represents the LUN. # datapath query adaptstats Adapter #: 0 ============= Maximum I/O: 23 SECTOR: 5128 Adapter #: 1 ============= Maximum
Total Read
Total Write
Active Read
Active Write
9595892
4371836
0
0
176489389
138699019
0
0
Total Read
Total Write
Active Read
Active Write
I/O: 24 SECTOR: 5128
10238891
4523508
0
0
188677891
143739157
0
0
# datapath query portmap BAY-3(B3) ESSID DISK H1 H2 H3 H4 ABCD ABCD ABCD ABCD BAY-7(B7) H1
H2
ABCD ABCD 75BAFX1 ---- ---75BAFX1 ---- ---75BAFX1 ---- ---Y O N PD
= = = = =
H3
H4
ABCD ABCD vpath0 Y-Y- ---vpath1 Y-Y- ---vpath2 Y-Y- ----
BAY-1(B1) BAY-4(B4) H1 H2 H3 H4 H1 H2 H3 H4 ABCD ABCD ABCD ABCD ABCD ABCD ABCD ABCD BAY-5(B5) BAY-8(B8) H1 H2 H3 H4 H1 H2 H3 H4 ABCD ABCD ABCD ABCD ABCD ABCD ABCD ABCD ---- ---- ---- ------- ---- ---- ------- ---- ---- ------- ---- ---- ------- ---- ---- ------- ---- ---- ----
online/open online/closed offline path not configured path down
BAY-2(B2) H1
H2
H3
H4
ABCD ABCD ABCD ABCD BAY-6(B6) H1
H2
H3
H4
ABCD ABCD ABCD ABCD ---- ---- ---- ------- ---- ---- ------- ---- ---- ----
y = (alternate path) online/open o = (alternate path) online/closed n = (alternate path) offline
Note: 2105 devices' essid has 5 digits, while 1750/2107 device's essid has 7 digits. # datapath query wwpn Adapter Name PortWWN fscsi0 10000000C94F91CD fscsi1 10000000C94F9923
If you need to force the Subsystem Device Driver (SDD), or equivalent driver, to rescan and map the new devices, use the following command at the system prompt: # /usr/sbin/cfgvpath Procedure to make a new lun available to AIX: ---------------------------------------------Allocate the new lun on the SAN -Run "cfgmgr" -Verify the new vpath/hdisk by running "lsvpcfg"
There should be a new vpath and it should be available with no volume group - if not, rerun cfgmgr Create Volume groups with vpaths: --------------------------------You should use the mkvg4vp command to create Volume Groups. Example: # mkvg4vp -B -t 32 -s 4 -y DB01_RECOV_VG1 vpath4 vpath10 By default, VG's can accommodate up to 255 LV's and 32 PV's. If the -B flag is used on the mkvg or mkvg4vp command, the resulting VG will support up to 512 LV's and 128 PV's. The -s flag, as usual, designates the Partition size.
SDD software on AIX: -------------------Starting with SDD 1.6.1.0, the SDD package for AIX53 is devices.sdd.53.rte and requires AIX53E with APAR IY76997. Starting with SDD 1.6.2.0, the SDD package for AIX52 is devices.sdd.52.rte and requires AIX52M with APAR IY76997. See also in this document: IBM Flash Alert: SDD 1.6.2.0 requires minimum AIX code levels; possible 0514-035 error The SDD installation package installs a number of new commands, like datapath, chgvpath, lsvpcfg etc.. Before installing SDD, you should check firmware levels, and AIX APAR requirements. See the following sites: -- scsi and ESS, and Fiber: www-1.ibm.com/servers/storage/support/ www-1.ibm.com/servers/eserver/support/unixservers/index.html -- AIX APAR: www-03.ibm.com/servers/eserver/support/unixservers/aixfixes.html or, www.ibm.com/servers/eserver/support/pseries/aixfixes.html or, www14.software.ibm.com/webapp/set2/sas/f/genunix3/aixfixes.html
31.5 SAN connections with SDDPCM MPIO: ====================================== We have seen the SDD connections in section 31.4. This section covers some of the SDDPCM MPIO SAN connections. There are some different commands with this type of connections to SAN storage. The use of SDD or SDDPCM gives the AIX host the ability to access multiple paths to a single LUN within an ESS or SAN. This ability to access a single LUN on multiple paths allows for a higher degree of data availability in the event of a path failure. Data can continue to be accessed within the ESS as long as there is at least one available path. Without one of these installed, you will lose access to the LUN in the event of a path failure. If you have "sdd" installed use the datapath command, and with sddpcm use the pcmpath command. Just as the commands shown in section 31.4, just replace datapath with pcmpath, like
# pcmpath query device DEV#: 2 DEVICE NAME: hdisk2 TYPE: 2107900 ALGORITHM: Load Balance SERIAL: 75065711100 ======================================================================= === Path# Adapter/Path Name State Mode Select Errors 0 fscsi0/path0 OPEN NORMAL 1240 0 1 fscsi0/path1 OPEN NORMAL 1313 0 2 fscsi0/path2 OPEN NORMAL 1297 0 3 fscsi0/path3 OPEN NORMAL 1294 0 DEV#: 3 DEVICE NAME: hdisk3 TYPE: 2107900 ALGORITHM: Load Balance SERIAL: 75065711101 ======================================================================= === Path# Adapter/Path Name State Mode Select Errors 0 fscsi0/path0 CLOSE NORMAL 0 0 1 fscsi0/path1 CLOSE NORMAL 0 0 2 fscsi0/path2 CLOSE NORMAL 0 0
0
3
fscsi0/path3
CLOSE
NORMAL
0
DEV#: 4 DEVICE NAME: hdisk4 TYPE: 1750500 ALGORITHM: Load Balance SERIAL: 13AAGXA1101 ======================================================================= === Path# Adapter/Path Name State Mode Select Errors 0* fscsi0/path0 OPEN NORMAL 12 0 1 fscsi0/path1 OPEN NORMAL 3787 0 2* fscsi1/path2 OPEN NORMAL 17 0 3 fscsi1/path3 OPEN NORMAL 3822 0 # pcmpath query essmap Some possible errors with pcmpath: root@zd110l04:/root#pcmpath query device Kernel extension sdduserke was not loaded. Errno=8. Please verify SDDPCM device configuration. On a system with SDDPCM, you will see the SDDPCM server daemon, "pcmsrv", running. This process checks available paths and does other checks and monitoring. The process is under control of the resource controller, like for example starting and stopping it goes with # stopsrc -s pcmsrv # startsrc -s pcmsrv The process is started on boot from inittab: # cat /etc/inittab | grep pcmsrv srv:2:wait:/usr/bin/startsrc -s pcmsrv > /dev/null 2>&1
Notes on SDD and SDDPCM: ======================== Note 1: ------thread
Q +A: > > > > >
I've been reading IBM web sites and PDF manuals and still can't decide on exactly how to upgrade my AIX 4.3.3 machine to AIX 5.2 and have my ESS SDD vpath disks visible and working when I'm done. Has someone done this? Can you comment on my proposed method here?
Yes, I've done this. > What I think I need to do is this: > > 1. Do the migration installation from 4.3.3 to 5. Question: Do I need to > do anything to my ESS disks BEFORE migrating? Unmount? Vary off volume > groups? Export volume groups? Yes to all of the above, prior to upgrade. Uninstall SDD software. > 2. After the migration, and reboot, I understand that the ESS disks will > not "be there", since the migration does not upgrade the SDD (subsystem > device driver) does NOT get upgraded. Question: Is this true? Yes, the datapath devices will be gone because you deleted the SDD software; IIRC, that is part of the un-install process. After your upgrade, install SDD just like the first time. This will get you your hdisks and vpaths back, though not necessarily with the same numbers; have a 'lsvpcfg' from before your upgrade to cross-reference your new setup to. 'importvg' the VG(s) one at a time, using one of the hdisk's which constitute the vpath, then run 'hd2vp' on the VG. That will convert the VG back to using the vpath's. Note: IIRC, If I Recall/Remember Correctly > > 3. Vary off all ESS volume groups, if I shouldn't have done this back in > step 1. > > 4. Remove all the "datapath devices", via: rmdev -dl dpo -R > > 5. Uninstall the 4.3 version of the SDD. > > 6. Install the 5.2 version of the SDD. > > 7. Install the latest PTF of the 5.2 SDD, that they call version > 1.5.1.3. > > 8. Reboot. > >
> If you can tell me how to make this procedure more nearly correct, I'd > greatly appreciate it. Note 2: ------thread Q + A: > > I need a quick refresher here. SAN- attached > ESS storage. SDD is installed. volume groups on > the fly, or does HA need to be done this and I > can't quite remember if I have the help.
I've got a HACMP (4.4) cluster with Can I add volumes to one of these down? It's been awhile since I have to jump through any hoops. Thanks for
Should be relatively easy with no downtime required. 1) acquire the new disks on primary node (where the VG is in service) with: cfgmgr -Svl fcs0 - repeat this for all fcs adapters in system 2) convert hdisks to vpaths, note use the smit screens for this because the commands have changed from version to version. 3) add vpaths to VG with: extendvg4vp vgname vpath# 4) create LVs/filesystems on the vpaths. 5) break VG/scsi locks so that other systems can see the disks with: varyonvg -b -u vgname 6) perform steps 1 & 2 for all failover nodes in the cluster. 7) refresh the VG definitions on all the failover nodes with: importvg -L vgname vpath# 8) reestablish disk locks on service node with: varyonvg vgname 9) add new filesystems to HA configuration. 10) synchronise HA resources to the cluster. Note 3: ------From IBM Doc SC30-4131-00: hd2vp and vp2hd SDD provides two conversion scripts, hd2vp and vp2hd. The hd2vp script converts a volume group from supported storage device hdisks to SDD vpath devices, and the vp2hd script converts a volume group from SDD vpath devices to supported storage device hdisks.
Use the vp2hd program when you want to configure your applications back to original supported storage device hdisks, or when you want to remove SDD from your AIX host system. The syntax for these conversion scripts is as follows: hd2vp vgname vp2hd vgname vgname Specifies the volume group name to be converted. Note 4: ------thread Q: Hi There, I want to add a vpath to running hacmp cluster with HACMP 5.1 on AIX 5.2 with Rotating Resource Group. If anyone has done it before then can provide a step by step procedure for this. Do i need to stop and start HACMP for this? A: On Vg active node : #extendvg4vp vg00 vpath10 vpath11 #smitty chfs ( Increase the f/s as required ) #varyonvg -bu vg00 ( this is to un-lock the vg) On Secondary node where vg is not active : # cfgmgr -vl fscsi0 ( fscsi1 and fcs0 and fcs1 ) Found new vpaths # chdev -l vpath10 -a pv=yes ( for vpath11 also ) # lsvg vg00|grep path ( just note down any one vpath which is from this o/p-for e.g vpath0 ) # importvg vg00 vpath0 Once its fine...go to Primary Node # varyonvg vg00 ( Locking the VG ) Regards Note 5: ------> HI, > Is there a way to know dependencies between devices. > For example, > hdisk2 is attached to fscsi0 which in turn is attached to fcs0
> I have found nothing in lsdev's man > Do I have to look in the odm directly > I need this in order to improve a script This is a good question and the lsdev man page should be burned in front of the building where they develop and document AIX in Austin, TX, for not answering it for you. After all, you bothered to read the damn thing; why didn't it tell you? $ /usr/sbin/lsdev -Cc adapter -F 'name parent' ppa0 isa0 sa0 isa0 sa1 isa0 sa2 isa0 siokma0 isa0 fda0 isa0 scsi0 pci0 ent0 pci0 cxpa0 pci0 ent1 pci0 mga0 pci1 ent2 pci1 scsi1 pci2 sioka0 siokma0 sioma0 siokma0 ent3 pci0 There's also the lsparent command. Regards, Actually, I have the same question as Frederic and you have not quite answered it. Sure, lsdev can tell you that "hdisk5" is matched to "fcs0" . . . but what tells you that "fcs0" in turn matches to "fscsi0"? And if "hdisk126" matches to adapter "fchan1", how do I determine what that matches to? I've checked all of the various lsxxxx commands but can't find this bit of info. ONCE AGAIN the answer pops up just moments after announcing to the world that "there's no way to do that" and "I've looked everywhere and tried everything". Herewith the output from the necessary commands, with extraneous lines removed: # lsdev -C -c disk -F 'name location' hdisk0 11-08-00-2,0 hdisk1 11-08-00-4,0 hdisk2 3A-08-01 hdisk3 3A-08-01 hdisk4 27-08-01 hdisk5 27-08-01 # lsdev -C -c driver -F 'name location' fscsi0 27-08-01
fscsi1 3A-08-01 # lsdev -C -c adapter -F 'name location' scsi0 11-08 scsi1 11-09 fcs0 27-08 mg20 2D-08 fcs1 3A-08 # Obviously it is a simply matter to match disk to adapter to driver by the location of each object. After that I can easily sprintf(pathname, "/dev/%s", driver); fp = open(pathname, O_RDONLY | O_NDELAY); ioctl(fp, SCIOINQU, &info); to get the scsi inquiry buffer. Note 6: ------thread Q: where to fidnd a guide for the adapter (described blinkging/lighting)
all its states, LED
Adapter is cabled by SAN guys, they double checked it and when I run: rmdev -Rl fcs0 cfgmgr -l fcs0 lsattr -El fscsi0 -l attach I don't see "switch" but "none". thx in advance. A: Did you check SAN Switch Zoning? Regards, Do something like: rmdev -Rdl fscsi0 rmdev -dl fcnet0 rmdev -l fcs0 cfgmgr -l fcs0 rmdev -Rdl fscsi0 rmdev -Rdl fscsi1
rmdev -l fcs1 This way, the FC adapter re-negociates an FC fabric logon. HTH, I had already done something similiar but it didn't helped: # lsslot -c slot|grep fcs0 U787B.001.DNWFFM5-P1-C4 Logical I/O Slot pci4 fcs0 # rmdev -dl pci4 -R fcnet0 deleted fscsi0 deleted fcs0 deleted pci4 deleted # cfgmgr Method error (/usr/lib/methods/cfgefscsi -l fscsi0 ): 0514-061 Cannot find a child device. # lsattr -El fscsi0 -a attach attach none How this adapter is CONNECTED False the second FC is connected ok: # lsattr -El fscsi1 -a attach attach switch How this adapter is CONNECTED False # thx anyway, I will ask my SAN team to check cables once more. Note 7: ------thread hdisk and vpath correspondance for IBM SAN (shark) Description Correspondance between phsical disks: 4 hdisk = 1 vpath = 1 physical disk To remove all vpaths run the command: # rmdev -dl dpo -R To remove all fibre channel disks (2 cards in this example): # rmdev -dl fscsi0 -R # rmdev -dl fscsi1 -R To recreate the hdisks run the command: # cfgmgr -vl fcs0 # cfgmgr -vl fcs1 To recreate the vpaths run the command:
# cfallvpath To delete a device run this command: # rmdev -l fcs1 -d Example rmdev -dl dpo -R ; rmdev -dl fscsi0 -R ; cfgmgr -vl fcs0 ; cfallvpath Note 8: ------Technote (FAQ) Problem When non-root AIX users issue SDD datapath commands, the "No device file found" message results. Cause AIX SDD does not distinguish between file not found and invalid permissions. Solution Login as the root user or "su" to root user and re-execute command in order to obtain the desired SDD datapath command output. Note 9: ------(thread ibm site) Question: Hi, I have an AIX 5.3 server running with 2 FCs. One on a DS8300 and one on a DS4300. On the server, i have a filesystems that is mounted and active (hdisks are from the DS8300). I can access it fine, write, delete etc... Yet, when i do a "datapath query adapter" i get the following : # datapath query adapter Active Adapters :1 Adpt# Name State Mode Select Errors Paths Active 0 fscsi0 NORMAL ACTIVE 4111177 0 32 0 I would expect to see my 32 paths Active. I checked another server that has a similar configuration (though it only has 1 FC) and i can see 32 Paths, 32 Active... Is it because of the other FC being connected to a DS4300?
Answer: Hi. The reason is that the vpaths are not part of a varied on volume group. If you do a 'datapath query device' you should find all the paths will be state=closed. If the vpaths are being used by a volume group, do a varyonvg xxxx. Then display the datapath and the paths should be active. Question: Hi. THanks, but as i mentionned in my original post, the VG is varied on and the FS is mounted. I ran the datapath command after i i varyonvg bkpvg and mount /backup. I then dumped a DB within the FS, deleted and everything else works...yet datapath query adapter shows no Active paths...weird... Question: Hi. What version of SDD? What does 'datapath query device' say? Answer: Version of SDD is 1.6.0.5 And a datapath query device shows : ... DEV#: 14 DEVICE NAME: vpath14 TYPE: 2107900 POLICY: Optimized SERIAL: 75AYYV111B7 ======================================================================= ==== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk40 CLOSE NORMAL 147989 0 1 fscsi0/hdisk23 CLOSE NORMAL 0 0 DEV#: 15 DEVICE NAME: vpath15 TYPE: 2107900 POLICY: Optimized SERIAL: 75AYYV111B8 ======================================================================= ==== Path# Adapter/Hard Disk State Mode Select Errors 0 fscsi0/hdisk41 CLOSE NORMAL 155256 0 1 fscsi0/hdisk24 CLOSE NORMAL 0 0 yet, as i mentionned, my FS /backup is mounted and accessible... Note 10:
-------thread Q: Hi All, I am having problems on a p570 on which there are 3 HBA cards. 2 of the HBAs are connected via a SAN switch to an ESS 800. It appears only one of the "paths" to the ESS 800 is working As I only have one set of view of the disks on the ESS. Running cfgmgr on the adapter gives the following error. I have tried removing fscsi0 then unconfiguring fcs0, Then reconfiguring fcs0 but I still get the same error. Any ideas? Is there some command/utility I can run to verify The state of ths HBA? Thank you. bash-3.00# cfgmgr -l fcs0 Method error (/usr/lib/methods/cfgefscsi -l fscsi0 ): 0514-061 Cannot find a child device. bash-3.00# 0514-061 Cannot find a child device A: HI I have had the same problem using HDS SAN devices. AT that time I did not have the corect version off the device driver for the fiber cards in P570. For aix 5.2 devices.pci.df1000fa >= 5.2.0.40 For aix 5.3 devices.pci.df1000f7 >= 5.3.0.10 /HGA Note 11: -------Greetings: The "0514-061 Cannot find a child device" is common when the FC card is either not attached to a FC device, or if it is attached, then I would look at the polarity of the cable ie. (tx -> rx and rx -> tx) NOT (tx -> tx and rx -> rx)
cfgmgr is attempting to configure the FC device it is connected to (child device) but is unable to see it. In this context, device would be some sort of FC endpoint, not just a switch or director. I would make sure the FC card has connectivity to a FC device, not just the fabric and re-run cfgmgr. -=Patrick="Vincent D'Antonio, III" on 02/19/2003 01:51:24 PM Please respond to IBM AIX Discussion List To: [email protected] cc: (bcc: Patrick Bigelbach/DSS) Subject Re: Cannot cfgmgr on a new FC Put in your OS cd in the cdrom drive and run: cfgmgr -vi /dev/cd0 this should load any filesets you need for the adapter if they are not already there. You should the adapter in lsdev -Cc adapter | grep fs. HTH Vince -----Original Message----From: IBM AIX Discussion List [mailto:[email protected] ] On Behalf Of Calderon, Linda Sent: Wednesday, February 19, 2003 10:12 AM To: [email protected] Subject: Cannot cfgmgr on a new FC I am trying to connect a new HBA on a P660 to a switch for a SAN. This HBA has not been used previously, newly cabled etc. I issued the following commands and receive the following errors: * rmdev -Rdl fsc1 0514-519 The following device was not found in the customized device configuration database: name 'fcs1' * cfgmgr 0514-061 Cannot find a child device Looking for ideas as to root cause.
Note 12: -------thread Q: Hi All AIXers, I am trying to add some vpath to Current Volume Group (which is on vpath)and i am getting this error Method Error (/usr/lib/methods/chgvpath): 0514-047 Cannot access a device 0516-1182 extendvg open failure on vpath3 0516-792 extendvg: Unable to estend a Volume Group Do anybody have any idea about this error. I never seen this error before. Thanks A: James, If you're adding a vpath to a volume group that has other vpaths, you will need to use extendvg4vp instead of extendvg. Hope this helps! Note 13: -------On Vg active node : #extendvg4vp vg00 vpath10 vpath11 #smitty chfs ( Increase the f/s as required ) #varyonvg -bu vg00 ( this is to un-lock the vg) On Secondary node where vg is not active : # cfgmgr -vl fscsi0 ( fscsi1 and fcs0 and fcs1 ) Found new vpaths # chdev -l vpath10 -a pv=yes ( for vpath11 also ) # lsvg vg00|grep path ( just note down any one vpath which is from this o/p-for e.g vpath0 ) # importvg vg00 vpath0 Once its fine...go to Primary Node # varyonvg vg00 ( Locking the VG )
Regards Note 14: -------thread How to add a a new PV into an existing concurrent mounted VG. The PMR action plan suggests: -
stop of the resource group varyoffvg dummyvg varyonvg -nc dummyvg extendvg4vp dummyvg vpath0 start of the resource group
as a backup action - restart of the cluster - extendvg4vp dummyvg vpath0 - start of the resource group After a spech with the Country IBM referent we modify the action plan in: - stop of the cluster - varyoffvg dummyvg - varyonvg dummyvg dummyvg should remain Enhanced Concurrent Capable, but I mount it in normal mode to do the extentions - extendvg4vp dummyvg vpath0 - importvg -L dummyvg disk on the other node of the cluster - varyoffvg dummyvg - cluster verification & syncro - start of the cluster Anyway before applying original one, but with works, with someothers with others return the
the modified action plan I try to follow the unpredictable return codes. With some vpaths halfworks (update the VGDA, but not the odm), original error.
In my opinion there is an high probability that the cause is in gsclvmd... So, a bit disappointed, I applied the modified plan. All works and the extendvg4vp enlarged the dummyvg... My machines are too downlevel and very full of lacks :-( After that my curiosity pulls me to try the next step: mirrorvg -s -c 2 dummyvg vpath0 vpath1 0516-1509 : VGDA corruption: physical partition info for this LV is invalid. 0516-842 : Unable to make logical partition copies for logical volume.
0516-1199 mirrorvg: Failed to create logical partition copies for logical volume dummylv. 0516-1200 mirrorvg: Failed to mirror the volume group Now, IBM support is working for analyze this new issue...... Regards. Note 15: cfgmgr method errors: -----------------------------1: == APAR status Closed as program error. Error description Users of the 64bit kernel may observe an error when cfgmgr is invoked at runtime in the cfgsisscsi or cfgsisioa config methods. Following is an example: # cfgmgr Method error (/usr/lib/methods/cfgsisscsi -l sisscsia0 ): 0514-061 Cannot find a child device. The error occurs in the cfgsisscsi or cfgsisioa routines which automatically update the microcode on the adapter if it is found to be at a level lower than the minimum supported microcode level. If the adapter was previously unconfigured, the adapter will remain in the Defined state. A system reboot should make it Available. APAR information APAR number IY48873 Reported component name AIX 5L POWER V5 Reported component ID 5765E6200 Reported release 520 Status CLOSED PER PE NoPE HIPER NoHIPER Submitted date 2003-09-19 Closed date 2003-09-19 Last modified date 2003-10-24 Note 16: cfgmgr method errors: -----------------------------Q: cfgmgr error-- devices are reported twice Asked by kuntal_acharyy... on 11/28/2005 6:15:00 AM
I have an IBM DS4400 with two EXP 700s expansion units connected to a pSeries 650 with AIX 5.1.I have created two logical drives in the storage unit.When i run "cfgmgr" to recognise the new raw physical volume each disk is reported twice. hdisk4 hdisk5 hdisk6 hdisk7
Available Available Available Available
1n-08-01 1n-08-01 11-08-01 11-08-01
1742 1742 1742 1742
(700) (700) (700) (700)
Disk Disk Disk Disk
Array Array Array Array
Device Device Device Device
There is an error message while running cfgmgr: Method error (/etc/methods/cfgfdar -l dar0 ): 0514-002 Cannot initialize the ODM. cfgmgr: 0514-621 WARNING: The following device packages are required for device support but are not currently installed. devices.scsi What may have cause the problem ? How ca I solve this problem? Any advice is truly welcome. A: hi, I had met the same problem just as yours. 3 LPARs(AIX 5300-02) on a p570 connect FastT600(Ds4300) with 2 HBA cards each, using SAN fibre switch. 2 of the LPARs reported hdisk twice, and 1 of them reported normally. And I found that the HBA cards on the normal one are in the PCI Slots belong to different BUSs, and the HBA cards on unnormal ones are in the same BUSs. Then I changed HBA cards to different BUSs' slots, deleted all the dar dac and HBA cards in the system, and cfgmgr at last. The problem got solved. I guess there must be some thing wrong with the BUS design. Some one told me that he solved the problem by install the last patch (AIX 5300-03). So my advice is that you should chang the HBA cards to differet slots, clear the system and cfgmgr. Or maybe update your AIX with the last patch. Just try and tell me the result. Good luck! Note 17: cfgmgr method errors: [email protected] (Ed) wrote in message news:... > I deleted a scsi device from my 4.3.3 configuration with the following > command: > rmdev -l scsi2 -dR
> > > > > > > > > > > > > > > > > > > >
The device is a dual channel ultra scsi 3 card. I deleted it to try to resolve some performance problems with a drawer connected to the device. Incidentally, scsi3 which is the other side of the dual channel card, is working fine. When I try to reconfigure the device with: cfgmgr -v -lscsi2 I get the following error: Method error (/usr/lib/methods/cfgncr_scsi -l scsi2 ): 0514-034 The following attributes do not have valid values: Any thoughts on how to fix it? For the timebeing I can't reboot the machine. Would a reboot be able to resolve the problem if there is no other solution? Thanks! -- Ed
#>> Ed, what you probably should do is run the cfgmgr comand without the device name behind it. Because you deleted the scsi device with the options -dR you also removed any child devices. try this: cfgmgr -v Note 18: cfgmgr method errors: -----------------------------Q: Hi... Does someone know what to do with an SDD driver which can't detect vpaths from an ESS F20 but hdisks are already available on AIX? showvpath, cfgvpath, datapath query commands don't display or found anything By the way, rebooting the system didn't help I accept any suggestions. Regards Luis A. Rojas A: Thank you all for your suggestions
I solve the problem using the hd2vp command which converts the logical hdisk to its related vpath. And Wal? !.. vpaths suddenly were recognized by cfgvpath command. I don't know why this happened, but, everything is OK now. To those people with similar problems, please check these following commands: dpovgfix, hd2vp, vp2hd Best Regards
Note 19: fget_config: --------------------how to show the current state and volume (hdisk) ownership in a IBM DS4000 Description The fget_config command shows the current state and volume (hdisk) ownership. To display controllers and hdisks that are associated with a specified DS4000 (dar): # fget_config To display the state of each controller in a DS4000 array, and the current path that is being used for I/O for each hdisk: # fget_config -A Example fget_config -A Note 20: -------Q: dpovgfix, hd2vp, vp2hd Asked by RandallGoff on 1/23/2007 9:38:00 AM What filesets do dpovgfix, hd2vp and vp2hd belong to. I installed my sdd driver and can see everything but can't find these commands. A: They are part of your SDD drivers. You probably installed the devices.xxx filesets. Did you also install the host attachment script... the ibm2105 filesets?
Note 21: -------thread Q: Hi I have several AIX LPARS running on SVC controlled disks. Right now i have SDD SW 1.6.1.2. After configuration i have some vpath devices that can be managed using the datapath command. Now in a recent training of SVC i was asked to install the new SDDPCM driver in order to get some of the benefits of this SW driver. SDDPCM does not use the concept of vpath anymore, instead a hdisk device object is created. This object has definitions and attributes in ODM files. Recently i had to change a faulty HBA under SDD drivers. I was able to: 1- datapath query device: in order to check hdisk devices belonging to the faulty adaptr. 2- datapath query adapter: in order to check the faulty adapter. 3- datapath set adapter XX offline: in order to put the faulty HAB offline. 4- datapath remove adapter XX 5- Used the diag Hot Plug option to remove the PCI-x HBA and install a new one. Configured the system and modified the corresponden zone. How to do the same with SDDPCM even when there's no concept of vpath anymore. Thanks in advanced A: Hello , You can do the same with sddpcm , either using the MPIO commands or smitty screens , smitty devices ---> MPIO devices there you can list paths , remove paths , adapters. IN the SDD user guide there is a complete section describing what you can do , but same functions you use for the vpath , you can use for sddpcm. Here is the link for the latest user guide http://www-1.ibm.com/support/docview.wss?rsP3&con text=ST52G7&dc=DA490&dc=DA4A30&dc=DA480&dc=D700&dc =DA410&dc=DA4A20&dc=DA460&dc=DA470&dc=DA400&uid=ss g1 S7000303&loc=en_US&cs=utf-8&lang=en Note 22:
-------thread Q: Greetings: Has anyone encountered the 0516-1182 ( mkvg: Open Failure on vpath ) or 0516-826 ( mkvg: Unable to create volume group ) errors while trying to create a new volume group ? I attempted to create a new volume group using a couple of newly added vpath devices and received those errors. Any help will be greatly appreciated. Thanks in advance. Jay. A: Hi If using vpath devices then you can confirm that you can open any given device by running: datapath query device and confirm there's no error in the HBA communications. Also you can review the errpt reports in order to look for VPATH OPEN messages. You can also use the lquerypr command in order to check for SCSI reservations in the SAN box previously set by another host (in case of a cluster). Hope this helps Example lquerypr output # lquerypr -Vh /dev/hdisk12 connection type: fscsi1 open dev: /dev/hdisk12 Attempt to read reservation key... Attempt to read registration keys... Read Keys parameter Generation : 52 Additional Length: 32 Key0 : c8ca9d09 Key1 : c8ca9d09 Key2 : c8cabd09
Key3 : c8cabd09 Reserve Key provided by current host = c8cabd09 Not reserved.
Note 23: -------thread Q: All, I'm in the process of preparing for our upcoming disaster recovery exercise which is happening in a few weeks. Our plan is to create one big volume group, instead of a bunch of little ones like we have in our production environment, to try and save some time. My question is, is there a way to script using a for/next loop to assign each hdisk/vpath when creating a new volume group instead of going into smit and assigning them one by one by hand? The hdisks will be sequential and will probably be over a hundred in number so you can imagine how tedious this will be. Also, this will need to be bigvg enabled. Any of you scripters out there have any suggestions? Thanks for your help in advance! A: Create the VG >mkvg -B -y datavg vpathN Extend it for i in `lspv | grep vpath | grep None | awk '{print #1}'` do extendvg datavg $i done That would assign all unused vpaths to the VG. BTW Use the vpath and not the hdisk. You could add a count into it to limit the number of disks you assign. Note 24: -------thread
Q: Is anyone aware of a problem if i do a cfgmgr -vl dp0 and once the vpaths are made it shows as vpathxx none None and then i add the vpath to VG #extendvg VGname vpathxx Does this create a problem ? A: it sound like the vpath is showing correctly after cfgmgr so thats OK. But you need to use extendvg4vp and not just extendvg Do a 'smitty vg' and choose 'Add a Data Path Volume to a Volume Group' Once its added to a VG then it will show more info in lspv
Note 25: cfgmgr Method error (/usr/sbin/fcppcmmap > /etc/essmap.out): --------------------------------------------------------------------Method error (/usr/sbin/fcppcmmap > /etc/essmap.out): 0514-001 System error:
Note 26: mkpath, lspath commands: --------------------------------Examples mkpath: --To define and configure an already defined path between scsi0 and the hdisk1 device at SCSI ID 5 and LUN 0 (i.e., connection 5,0), enter: # mkpath -l hdisk1 -p scsi0 -w 5,0 The system displays a message similar to the following: path available --To configure an already defined path from 'fscsi0' to fiber channel disk 'hdisk1', the command would be: # mkpath -l hdisk1 -p fscsi0 The message would look similar to:
path available --To only add to the Customized Paths object class a path definition between scsi0 and the hdisk1 disk device at SCSI ID 5 and LUN 0, enter: # mkpath -d -l hdisk1 -p scsi0 -w 5,0 The system displays a message similar to the following: path defined Examples lspath: lspath displays information about paths to an MultiPath I/O (MPIO) capable device. Examples of displaying path status: -- To display the status of all paths to hdisk1 with column headers, enter: # lspath -H -l hdisk1 The system will display a message similar to the following: status device parent enabled hdisk1 scsi0 disabled hdisk1 scsi1 missing hdisk1 scsi2 -- To display, without column headers, the set of paths whose operational status is disabled, enter: # lspath -s disabled The system will display a message similar to the following: disabled hdisk1 scsi1 disabled hdisk2 scsi1 disabled hdisk23 scsi8 disabled hdisk25 scsi8 --To display the set of paths whose operational status is failed, enter: # lspath -s failed The system will display a message similar to the following: failed hdisk1 scsi1 failed hdisk2 scsi1 failed hdisk23 scsi8 failed hdisk25 scsi8 -- To display in a user-specified format, without column headers, the set of paths to hdisk1 whose path status is available enter: # lspath -l hdisk1 -s available -F"connection:parent:path_status:status" The system will display a message similar to the following: 5,0:scsi0:available:enabled 6,0:scsi1:available:disabled
Note that this output shows both the path status and the operational status of the device. The path status simply indicates whether the path is configured or not. The operational status indicates how the path is being used with respect to path selection processing in the device driver. Only paths with a path status of available also have an operational status. If a path is not currently configured into the device driver, it does not have an operational status. Examples of displaying path attributes: --If the target device is a SCSI disk, to display all attributes for the path to parent scsi0 at connection 5,0, use the command: # lspath -AHE -l hdisk10 -p scsi0 -w "5,0" The system will display a message similar to the following: attribute value description user_settable weight 1 Order of path failover selection true Note 26: About FastT and DS Storage: -----------------------------------IBM TotalStorager FAStT has been renamed IBM TotalStorage DS4000 series DS4100 formerly FAStT100 DS4300 formerly FAStT600 DS4300 Turbo formerly FAStT600 Turbo DS4400 formerly FAStT700 DS4500 formerly FAStT900 Note 27: from GPFS FAQ: ----------------------Q20: What's the difference between using an ESS with or without SDD or SDDPCM installed on the host? A20: The use of SDD or SDDPCM gives the AIX host the ability to access multiple paths to a single LUN within an ESS. This ability to access a single LUN on multiple paths allows for a higher degree of data availability in the event of a path failure. Data can continue to be accessed within the ESS as long as there is at least one available path. Without one of these installed, you will lose access to the LUN in the event of a path failure. However, your choice of whether to use SDD or SDDPCM impacts your ability to use single-node quourm:
Single-node quorum is not supported if SDD is installed. Single-node quorum is support if SDDPCM is installed. To determine the GPFS disk support guidelines for SDD and SDDPCM for your cluster type, see Q3: What disk support guidelines must be followed when running GPFS in an sp cluster type? Q6: What disk support guidelines must be followed when running GPFS in an rpd cluster type? Q9:What are the disk support guidelines that must be followed when running GPFS in an hacmp cluster type Note 28: changing attributes of a fcs0 device: ---------------------------------------------Examples: # chdev -l fscsi0 -a fc_err_recov=fast_fail # chdev -l fscsi0 -a dyntrk=yes Display attributes: # lsattr -El fscsi0 attach dyntrk fc_err_recov scsi_id sw_fc_class
switch no fast_fail 0x741113 3
How this adapter is CONNECTED False Dynamic Tracking of FC Devices True FC Fabric Event Error RECOVERY Policy True Adapter SCSI ID False FC Class for Fabric True
Note 29: Flash alerts: ---------------------IBM Flash Alert on AIX migration with vpaths: --------------------------------------------http://www-1.ibm.com/support/docview.wss? rs=540&context=ST52G7&uid=ssg1S1002295&loc=en_US&cs=utf-8&lang=en All hdisks and vpath devices must be removed from host system before upgrading to SDD host attachment script 32.6.100.21 and above. All MPIO hdisks must be removed from host system before upgrading to SDDPCM host attachment script 33.6.100.9. Flash (Alert) Abstract When upgrading from SDDPCM host attachment script devices.fcp.disk.ibm2105.mpio.rte version 33.6.100.8 or below to 33.6.100.9, all SDDPCM MPIO hdisks must be removed from the AIX host system before the upgrade.
When upgrading from SDD host attachment script ibm2105.rte version 32.6.100.18 or below to 32.6.100.21 or later, all AIX hdisks and SDD vpath devices must be removed from the AIX host system before the upgrade. Content Please note that this document contains the following sections: Problem description, symptoms, and information SDD/host attachment upgrade procedures Recovery procedures should the ODM become corrupted Recovery procedures should the associations become corrupted Procedures for upgrading if rootvg is on an ESS disk - Problem description, symptoms, and information: Starting with SDDPCM host attachment script devices.fcp.disk.ibm2105.mpio.rte version 33.6.100.9 and SDD host attachment script ibm2105.rte version 32.6.100.21, ESS FCP devices are configured as "IBM MPIO FC 2105" for MPIO devices, and "IBM FC 2105" for ESS devices. This information can be seen in the "lsdev -Cc disk" output. Prior to these host attachment script versions, ESS FCP devices were configured as "IBM MPIO FC 2105XXX" for MPIO devices and "IBM FC 2105XXX" for ESS devices, where 'XXX' is the ESS device module, such as F20 or 800. If a host system is upgraded without removing all of the hdisks first, then the AIX host system ODM will be corrupted. Additionally, if all he hdisks are removed without removing all SDD vpath devices, then the associations between an SDD vpath device and its hdisks may be corrupted because the hdisk's device minor number may change after reconfiguration. The ODM corruption may look something like the following in the "lsdev -Cc disk" output: # lsdev -Cc disk lsdev: 0514-521 Cannot find information in the predefined configuration database for the customized device hdisk1. lsdev: 0514-521 Cannot find information in the predefined configuration database for the customized device hdisk2. lsdev: 0514-521 Cannot find information in the predefined configuration database for the customized device hdisk3. lsdev: 0514-521 Cannot find information in the predefined configuration database for the customized device hdisk4. lsdev: 0514-521 Cannot find information in the predefined configuration database for the customized device hdisk5. lsdev: 0514-521 Cannot find information in the predefined configuration database for the customized device hdisk6. lsdev: 0514-521 Cannot find information in the predefined configuration database for the customized device hdisk7. lsdev: 0514-521 Cannot find information in the predefined configuration database for the customized device hdisk8. hdisk0 Available 10-60-00-8,0 16 Bit SCSI Disk Drive hdisk1 Available 20-60-01 N/A
device device device device device device device device
hdisk2 hdisk3 hdisk4 hdisk5 hdisk6 hdisk7 hdisk8
Available Available Available Available Available Available Available
20-60-01 20-60-01 20-60-01 20-60-01 20-60-01 20-60-01 20-60-01
N/A N/A N/A N/A N/A N/A N/A
- SDD/host attachment upgrade procedures: In order to prevent ODM corruption and vpath/hdisk association corruption, all hdisks and SDD vpath devices must be removed prior to the upgrade. The following procedure should be used when you want to upgrade: -
AIX OS only* Host attachment + AIX OS* SDD + AIX OS* Host attachment + SDD Host attachment only SDD + Host attachment + AIX OS*
* Upgrading the AIX OS will always require you to install the SDD which corresponds to the new AIX OS level. To upgrade SDD only, follow the procedure in the SDD User's Guide. 1. Ensure rootvg is on local scsi disks. If this is not possible, see "Procedures for upgrading if rootvg is on an ESS disk" below. 2. Stop all applications running on SDD Volume Groups/File Systems. 3. Unmount all File Systems of SDD volume group. 4. Varyoff all SDD volume groups. 5. If upgrading OS, save output of lspv command to remember pvids of VGs. 6. If upgrading OS, export volume groups with exportvg. 7. Remove SDD vpath devices with rmdev command. 8. Remove 2105 hdisk devices with rmdev command. 9. If upgrading OS, run 'stopsrc -s sddsrv' to stop sddsrv daemon. 10. If upgrading OS, uninstall SDD. 11. If required, upgrade ibm2105.rte. The recommended version is 32.6.100.18 if support for ESS model 750 is not needed. Version 32.6.100.21 is required to support ESS model 750. 12. If upgrading OS, migrate AIX OS level. 13. If OS upgraded, boot to new AIX level with no disk groups online except rootvg, which is on local scsi disks. /* reboot will automatically start at the end of migration */ 14. If OS upgraded, install SDD for the new OS level. Otherwise, if required, upgrade SDD. 15. If OS not upgraded, configure hdisks with the 'cfgmgr -vl fcsX' command. 16. Configure SDD vpath devices by running 'cfallvpath'. 17. If OS upgraded, use lspv command to find out one physical volume which has a pvid matching the previous SDD VG's pv.
Example: =================================================== Previous lspv output (from step 4): hdisk0 000bc67da3945d3c None hdisk1 000bc67d531c699f rootvg active hdisk2 none None hdisk3 none None hdisk4 none None hdisk5 none None hdisk6 none None hdisk7 none None hdisk8 none None hdisk9 none None hdisk10 none None hdisk11 none None hdisk12 none None hdisk13 none None hdisk14 none None hdisk15 none None hdisk16 none None hdisk17 none None hdisk18 none None hdisk19 none None hdisk20 none None hdisk21 none None vpath0 000bc67d318fb8ea SDDVG0 vpath1 000bc67d318fde50 SDDVG1 vpath2 000bc67d318ffbb0 SDDVG2 vpath3 000bc67d319018f3 SDDVG3 vpath4 000bc67d319035b2 SDDVG4 Current lspv output (from this step): hdisk0 000bc67da3945d3c None hdisk1 000bc67d531c699f rootvg active hdisk2 000bc67d318fb8ea None hdisk3 000bc67d318fde50 None hdisk4 000bc67d318ffbb0 None hdisk5 000bc67d319018f3 None hdisk6 000bc67d319035b2 None hdisk7 000bc67d318fb8ea None hdisk8 000bc67d318fde50 None hdisk9 000bc67d318ffbb0 None hdisk10 000bc67d319018f3 None hdisk11 000bc67d319035b2 None hdisk12 000bc67d318fb8ea None hdisk13 000bc67d318fde50 None hdisk14 000bc67d318ffbb0 None hdisk15 000bc67d319018f3 None hdisk16 000bc67d319035b2 None hdisk17 000bc67d318fb8ea None hdisk18 000bc67d318fde50 None hdisk19 000bc67d318ffbb0 None hdisk20 000bc67d319018f3 None hdisk21 000bc67d319035b2 None vpath0 none None vpath1 none None vpath2 none None vpath3 none None
vpath4 none None In this case, hdisk2, hdisk7, hdisk12, and hdisk17 from the current lspv output has the pvid which matches the pvid of SDDVG0 from the previous lspv output. So, use either hdisk2, hdisk7, hdisk12, or hdisk17 to import the volume group with the name SDDVG0 18. Run hd2vp on all SDD volume groups. 19. Vary on all SDD volume groups. 20. Mount all file system back. - Recovery procedures should the ODM become corrupted: If the host system's ODM is already corrupted as a result of upgrading without removing the hdisks, please contact IBM Customer Support at 1-800-IBM-SERV to request a script to fix the corrupted ODM. - Recovery procedures should the associations become corrupted: If vpath/hdisk association corruption has occurred because hdisks were removed without removing SDD vpath devices, all SDD vpath devices must be removed and reconfigured in order to correct this corrupted association. - Procedures for upgrading if rootvg is on an ESS disk: If rootvg is on an ESS device and cannot be moved to local scsi disks, all hdisks cannot be removed prior to the upgrade. In this case, the following procedure should be used to upgrade the SDD host attachment script to version 32.6.100.21 or later: . Contact IBM Customer Support at 1-800-IBM-SERV to request a script to fix the corrupted ODM referenced above. . Without removing ESS hdisks, use smitty to upgrade the SDD host attachment script on the host system. . Immediately run the script to fix the corrupted ODM on the host system. . Run bosboot on the host system. . Reboot the host system so that the hdisks can be configured with the new ODM attributes. . Return to the "SDD/host attachment upgrade procedures" above and follow the appropriate upgrade steps now that the SDD host attachment script upgrade is complete. This issue only occurs when upgrading to devices.fcp.disk.ibm2105.mpio.rte version 33.6.100.9 and SDD host attachment script ibm2105.rte version 32.6.100.21 and above.
IBM Flash Alert: SDD 1.6.2.0 requires minimum AIX code levels; possible 0514-035 error: -------------------------------------------------------------------------------------Flash (Alert) Abstract SDD 1.6.2.0 requires minimum AIX code levels. Not upgrading to correct AIX version and level can result in 0514-035 error when attempting removal of dpo or vpath device Content Starting from SDD version 1.6.2.0, a unique ID attribute is added to SDD vpath devices, in order to support AIX5.3 VIO future features. AIX device configure methods have been changed in both AIX52 TL8 and AIX53 TL4 for this support. Following are the requirements for this version of SDD with: AIX5.2 and AIX5.3: AIX52 TL8 & above with PTF U804193 (IY76991) AIX53 TL4 & above with PTF U804397 (IY76997) Please view 1.6.2.0 readme for further details If upgraded to SDD 1.6.2.0 and above without first upgrading AIX to the levels listed above the following error will be experienced when attempting to remove any vpath devices using the: # rmdev -dl dpo -R or the # rmdev -dl vpathX command. Method error (/usr/lib/methods/ucfgdevice): 0514-035 Cannot perform the requested function because of missing predefined information in the device configuration database. Solution: 1) Upgrade AIX to correct level and ptf, or 2) Contact SDD support at 1-800-IBM-SERV for steps to clean up ODM to allow for downgrading the SDD level from 1.6.2.0, if unable to upgrade AIX to a newer technology level.
Note 30: -------Suppose the following happens: # rmdev -dRl fcs0
fcnet0 deleted fscsi0 deleted fcs0 deleted # cfgmgr Method error (/usr/lib/methods/cfgefscsi -l fscsi0 ): 0514-061 Cannot find a child device. root@n5114l02:/root# adapter checked with several commands connection with san seems impossible. root@n5114l02:/root#lsattr -El fscsi0 attach none How this adapter is CONNECTED False dyntrk no Dynamic Tracking of FC Devices True fc_err_recov delayed_fail FC Fabric Event Error RECOVERY Policy True scsi_id Adapter SCSI ID False sw_fc_class 3 FC Class for Fabric True Note 31: -------IY83872: AFTER CHVG -T, VG IS IN INCONSISTENT STATE A fix is available Obtain fix for this APAR
APAR status Closed as program error. Error description #--------------------------------------------------chvg -t renumber pvs that have pv numbers greater than maxpvs with the new factor. chvg -t is only updating the new pv_num in lvmrec and not updating the VGDA. chvg -t leaves the vg is inconsistent state and any changes to vg may get unpredictable results like a system crash. Local fix Problem summary #--------------------------------------------------chvg -t renumber pvs that have pv numbers greater than maxpvs with the new factor. chvg -t is only updating the new pv_num in lvmrec and not updating the VGDA. chvg -t leaves the vg is inconsistent state and any changes to vg may get unpredictable results like a system crash. Problem conclusion Fix chvg -t to update the VGDA with the new pv number. Add a check in hd_kextendlv to make sure that the pvol we are trying to access is not null. Temporary fix Comments APAR information APAR number IY83872 Reported component name AIX 5.3
Reported component ID 5765G0300 Reported release 530 Status CLOSED PER PE NoPE HIPER NoHIPER Submitted date 2006-04-11 Closed date 2006-04-11 Last modified date 2006-05-03 APAR is sysrouted FROM one or more of the following: APAR is sysrouted TO one or more of the following: Publications Referenced Fix information Fixed component name AIX 5.3 Fixed component ID 5765G0300 Applicable component levels R530 PSY U805071 UP06/05/03 I 1000
Note 32: ======== ESB-2008.0267 -- [AIX] -- AIX Logical Volume Manager buffer overflow ------------------------------------------------------------------------------Date: 14 March 2008 AusCERT Reference #: ESB-2008.0267 Click here for printable version Click here for PGP verifiable version -----BEGIN PGP SIGNED MESSAGE----Hash: SHA1 ======================================================================= ==== AUSCERT External Security Bulletin Redistribution ESB-2008.0267 -- [AIX] AIX Logical Volume Manager buffer overflow 14 March 2008 ======================================================================= ==== AusCERT Security Bulletin Summary ---------------------------------
Product:
AIX 5.2 AIX 5.3 IBM AIX Root Compromise Existing Account
Publisher: Operating System: Impact: Access:
Original Bulletin: http://www14.software.ibm.com/webapp/set2/subscriptions/pqvcmjd? mode=18&ID=4169 - --------------------------BEGIN INCLUDED TEXT-------------------- -----BEGIN PGP SIGNED MESSAGE----Hash: SHA1 IBM SECURITY ADVISORY First Issued: Tue Jan 22 14:02:18 CST 2008 | Updated: Tue Mar 11 12:55:14 CDT 2008 | IZ10828 availablity updated ======================================================================= ======== VULNERABILITY SUMMARY VULNERABILITY:
AIX Logical Volume Manager buffer overflow
PLATFORMS:
AIX 5.2, 5.3
SOLUTION:
Apply the fix or workaround as described below.
THREAT:
A local attacker may execute arbitrary code with root privileges.
CERT VU Number: n/a CVE Number: n/a ======================================================================= ======== DETAILED INFORMATION I. OVERVIEW The AIX Logical Volume Manager provides a suite of utilities for AIX logical volume management features and functions. The primary fileset for the AIX Logical Volume Manager is 'bos.rte.lvm'. In addition, AIX provides another suite of utilities for concurrent logical volume management across multiple hosts. The primary fileset for the AIX Concurrent Logical Volume Manager is 'bos.clvm.enh'. Several imporant commands provided by these filesets for performing various logical volume management tasks have been identified as containing buffer overflow vulnerabilities. II. DESCRIPTION Buffer overflow vulnerabilities exist in the 'bos.rte.lvm' and 'bos.clvm.enh' fileset commands listed below. A local attacker
may execute arbitrary code with root privileges because the commands are setuid root. The local attacker must be a member of the 'system' group to execute these commands. The following 'bos.rte.lvm' commands are vulnerable: /usr/sbin/lchangevg /usr/sbin/ldeletepv /usr/sbin/putlvodm /usr/sbin/lvaryoffvg /usr/sbin/lvgenminor The following 'bos.clvm.enh' command is vulnerable: /usr/sbin/tellclvmd III. IMPACT The successful exploitation of this vulnerability allows a non-privileged user to execute code with root privileges. IV. PLATFORM VULNERABILITY ASSESSMENT To determine if your system is vulnerable, execute the following command: lslpp -L bos.rte.lvm bos.clvm.enh The following fileset levels are vulnerable: AIX Fileset Lower Level Upper Level -----------------------------------------------bos.rte.lvm 5.2.0.0 5.2.0.107 bos.rte.lvm 5.3.0.0 5.3.0.61 bos.clvm.enh 5.2.0.0 5.2.0.105 bos.clvm.enh 5.3.0.0 5.3.0.60 V. SOLUTIONS A. APARS IBM provides the following fixes:
|
AIX Level APAR number Availability ----------------------------------------------------5.2.0 IZ00559 (available now) 5.2.0 IZ10828 05/07/2008 5.3.0 IY98331 (available now) 5.3.0 IY98340 (available now) 5.3.0 IY99537 (available now) Subscribe to the APARs here: http://www.ibm.com/support/docview.wss?uid=isg1IZ00559 http://www.ibm.com/support/docview.wss?uid=isg1IZ10828 http://www.ibm.com/support/docview.wss?uid=isg1IY98331 http://www.ibm.com/support/docview.wss?uid=isg1IY98340
http://www.ibm.com/support/docview.wss?uid=isg1IY99537 By subscribing, you will receive periodic email alerting you to the status of the APAR, and a link to download the fix once it becomes available. B. FIXES Fixes are available. from:
The fixes can be downloaded via ftp
ftp://aix.software.ibm.com/aix/efixes/security/lvm_ifix.tar The link above is to a tar file containing this signed advisory, fix packages, and PGP signatures for each package. The fixes below include prerequisite checking. This will enforce the correct mapping between the fixes and AIX Technology Levels.
--
AIX Fileset AIX Level Fix and Interim Fix ---------------------------------------------------------------
bos.lvm.rte IZ10828_08.071212.epkg.Z bos.lvm.rte IZ00559_8a.071212.epkg.Z bos.clvm.enh IZ00559_8b.071212.epkg.Z
5200-08
bos.lvm.rte IZ10828_09.071212.epkg.Z bos.lvm.rte IZ00559_9a.071211.epkg.Z bos.clvm.enh IZ00559_9b.071211.epkg.Z
5200-09
bos.lvm.rte IZ10828_10.071212.epkg.Z bos.lvm.rte bos.clvm.enh bos.clvm.enh.5.2.0.107.U
5200-10
bos.lvm.rte IY98331_05.071212.epkg.Z bos.lvm.rte IY99537_05.071212.epkg.Z bos.lvm.rte IY98340_5a.071211.epkg.Z bos.clvm.enh IY98340_5b.071211.epkg.Z
5300-05
5200-08 5200-08
5200-09 5200-09
5200-10 5200-10
bos.rte.lvm.5.2.0.107.U
5300-05 5300-05
bos.lvm.rte bos.clvm.enh
5300-05 5300-06 5300-06
To extract the fixes from the tar file: tar xvf lvm_ifix.tar
bos.rte.lvm.5.3.0.63.U bos.clvm.enh.5.3.0.61.U
cd lvm_ifix Verify you have retrieved the fixes intact: The checksums below were generated using the "sum", "cksum", "csum -h MD5" (md5sum), and "csum -h SHA1" (sha1sum) commands and are as follows: sum filename -----------------------------------14660 17 IY98331_05.071212.epkg.Z 26095 9 IY98340_5a.071211.epkg.Z 40761 8 IY98340_5b.071211.epkg.Z 10885 16 IY99537_05.071212.epkg.Z 24909 10 IZ00559_8a.071212.epkg.Z 64769 9 IZ00559_8b.071212.epkg.Z 65110 10 IZ00559_9a.071211.epkg.Z 25389 9 IZ00559_9b.071211.epkg.Z 26812 26 IZ10828_08.071212.epkg.Z 55064 26 IZ10828_09.071212.epkg.Z 55484 26 IZ10828_10.071212.epkg.Z 03885 157 bos.clvm.enh.5.2.0.107.U 30581 128 bos.clvm.enh.5.3.0.61.U 48971 1989 bos.rte.lvm.5.2.0.107.U 64179 2603 bos.rte.lvm.5.3.0.63.U cksum filename ------------------------------------------3121912357 16875 IY98331_05.071212.epkg.Z 107751313 9190 IY98340_5a.071211.epkg.Z 1129637178 7735 IY98340_5b.071211.epkg.Z 4019303479 16201 IY99537_05.071212.epkg.Z 1791374386 9289 IZ00559_8a.071212.epkg.Z 3287090389 8299 IZ00559_8b.071212.epkg.Z 565672617 9294 IZ00559_9a.071211.epkg.Z 257555679 8302 IZ00559_9b.071211.epkg.Z 3930477686 26525 IZ10828_08.071212.epkg.Z 1199269029 26533 IZ10828_09.071212.epkg.Z 358657844 26480 IZ10828_10.071212.epkg.Z 3753492719 160768 bos.clvm.enh.5.2.0.107.U 4180839749 131072 bos.clvm.enh.5.3.0.61.U 3765659627 2036736 bos.rte.lvm.5.2.0.107.U 3338925192 2665472 bos.rte.lvm.5.3.0.63.U csum -h MD5 (md5sum) filename ---------------------------------------------------------73bcf7604dd13f26a7500e45468ff5f7 IY98331_05.071212.epkg.Z 5f32179fc2156bb6e29e775aa7bff623 IY98340_5a.071211.epkg.Z 7c47e56cadabcba0a105ffa7fc1d40fc IY98340_5b.071211.epkg.Z ef3e4512c3b55091893ce733c707e1a2 IY99537_05.071212.epkg.Z db04be33e56169b6a8e8fd747e6948da IZ00559_8a.071212.epkg.Z 553f31ccf6a265333938d81eeae6dabc IZ00559_8b.071212.epkg.Z 2921b9d2a3dbd84591d60fddf0663798 IZ00559_9a.071211.epkg.Z 93ce34dec8f4fa9681a2c7c86be065fc IZ00559_9b.071211.epkg.Z e6b0a4a91ba197de0005bd800f06ba4e IZ10828_08.071212.epkg.Z 602a8c777cc27e51c3d3dbfa8ebd69be IZ10828_09.071212.epkg.Z b84a5cae03921d30675e522da29da1aa IZ10828_10.071212.epkg.Z
2aa4b9b43ca55f74b0fac6be7bc48b66 844e1f2ef9d388d2ddd8cf3ef6251f06 0c73aa8f0211c400455feaa6fb8a95c4 1b5a08eabe984d957db9a145e2a4fd06
---
bos.clvm.enh.5.2.0.107.U bos.clvm.enh.5.3.0.61.U bos.rte.lvm.5.2.0.107.U bos.rte.lvm.5.3.0.63.U
csum -h SHA1 (sha1sum) filename ---------------------------------------------------------------
d9929214a4d85b986fb2e06c9b265c768c7178a9 IY98331_05.071212.epkg.Z 0f5fbcdfbbbf505366dad160c8dec1c1ce75285e IY98340_5a.071211.epkg.Z cf2cda3b8d19b73d06b69eeec7e4bae192bec689 IY98340_5b.071211.epkg.Z 9d8727b5733bc34b8daba267b82864ef17b7156f IY99537_05.071212.epkg.Z e7a366956ae7a08deb93cbd52bbbbf451d0f5565 IZ00559_8a.071212.epkg.Z 1898733cdf6098e4f54ec36132a03ebbe0682a7e IZ00559_8b.071212.epkg.Z f68c458c817f99730b193ecbd02ae24b9e51cc67 IZ00559_9a.071211.epkg.Z 185954838c439a3c7f8e5b769aa6cc7d31123b59 IZ00559_9b.071211.epkg.Z 6244138dc98f3fd16928b2bbcba3c5b4734e9942 IZ10828_08.071212.epkg.Z 98bfaf44ba4bc6eba452ea074e276b8e87b41c9d IZ10828_09.071212.epkg.Z 2a9c0dd75bc79eba153d0a4e966d930151121d45 IZ10828_10.071212.epkg.Z 96706ec5afd792852350d433d1bf8d8981b67336 bos.clvm.enh.5.2.0.107.U 91f6d3a4d9ffd15d258f4bda51594dbce7011d8a bos.clvm.enh.5.3.0.61.U 4589a5bca998f437aac5c3bc2c222eaa51490dab bos.rte.lvm.5.2.0.107.U 3449afd795c24594c7a0c496f225c7148b4071ab
bos.rte.lvm.5.3.0.63.U
To verify the sums, use the text of this advisory as input to csum, md5sum, or sha1sum. For example: csum -h SHA1 -i Advisory.asc md5sum -c Advisory.asc sha1sum -c Advisory.asc These sums should match exactly. The PGP signatures in the tar file and on this advisory can also be used to verify the integrity of the fixes. If the sums or signatures cannot be confirmed, contact IBM AIX Security at [email protected] and describe the discrepancy. C. FIX AND INTERIM FIX INSTALLATION IMPORTANT: If possible, it is recommended that a mksysb backup of the system be created. Verify it is both bootable and readable before proceeding.
To preview a fix installation: installp -a -d . -p all To install a fix package: installp -a -d . -X all Interim fixes have had limited functional and regression testing but not the full regression testing that takes place for Service Packs; thus, IBM does not warrant the fully correct functionality of an interim fix. Interim fix management documentation can be found at: http://www14.software.ibm.com/webapp/set2/sas/f/aix.efixmgmt/ho me.html To preview an interim fix installation: the
emgr -e ipkg_name -p
# where ipkg_name is the name of # interim fix package being
previewed.
To install an interim fix package: emgr -e ipkg_name -X the
# where ipkg_name is the name of # interim fix package being
installed. VI. WORKAROUNDS There are two workarounds available. A. OPTION 1 Change the permissions of these commands to remove the setuid bit using the following commands: chmod chmod chmod chmod chmod chmod
500 500 500 500 500 500
/usr/sbin/lchangevg /usr/sbin/ldeletepv /usr/sbin/putlvodm /usr/sbin/lvaryoffvg /usr/sbin/lvgenminor /usr/sbin/tellclvmd
NOTE: chmod will disable functionality of these commands for all users except root. B. OPTION 2 (AIX 6.1, AIX 5.3 TL6 and TL7) Use the File Permissions Manager (fpm) command to manage setuid and setgid programs. fpm documentation can be found in the AIX 6 Security Redbook
at: http://www.redbooks.ibm.com/abstracts/sg247430.html An fpm level of high will remove the setuid bit from the affected commands. For example: fpm -l high -p fpm -l high
# to preview changes # to execute changes
NOTE: Please review the documentation before execution. fpm will disable functionality of multiple commands for all users except root. VII. OBTAINING FIXES AIX security related fixes can be downloaded from: ftp://aix.software.ibm.com/aix/efixes/security AIX fixes can be downloaded from: s/aix
http://www.ibm.com/eserver/support/fixes/fixcentral/main/pserie
NOTE: Affected customers are urged to upgrade to the latest applicable Technology Level and Service Pack. VIII. CONTACT INFORMATION If you would like to receive AIX Security Advisories via email, please visit: http://www14.software.ibm.com/webapp/set2/subscriptions/pqvcmjd Comments regarding the content of this announcement can be directed to: [email protected] To request the PGP public key that can be used to communicate securely with the AIX Security Team you can either: A. Send an email with "get key" in the subject line to: [email protected] B. Download the key from a PGP Public Key Server. The key ID is: 0xA6A36CCC Please contact your local IBM AIX support center for any assistance. eServer is a trademark of International Business Machines Corporation. IBM, AIX and pSeries are registered trademarks of International Business Machines Corporation. All other trademarks
are property of their respective holders. IX. ACKNOWLEDGMENTS IBM discovered and fixed this vulnerability as part of its commitment to secure the AIX operating system.
31.6 Other filesystem commands: =============================== df command: ----------df Command Purpose Reports information about space on file systems. This document describes the AIXr df command as well as the System V version of df. Syntax df [ [ -P ] | [ -I | -M | -i | -t | -v ] ] [ -k ] [ -m ] [ -g ] [ -s ] [FileSystem ... | File... ] Description The df command displays information about total space and available space on a file system. The FileSystem parameter specifies the name of the device on which the file system resides, the directory on which the file system is mounted, or the relative path name of a file system. The File parameter specifies a file or a directory that is not a mount point. If the File parameter is specified, the df command displays information for the file system on which the file or directory resides. If you do not specify the FileSystem or File parameter, the df command displays information for all currently mounted file systems. File system statistics are displayed in units of 512-byte blocks by default. The df command gets file system space statistics from the statfs system call. However, specifying the -s flag gets the statistics from the virtual file system (VFS) specific file system helper. If you do not specify arguments with the -s flag and the helper fails to get the statistics, the statfs system call statistics are used. Under certain exceptional conditions, such as when a file system is being modified while the df command is running, the statistics displayed by the df command might not be accurate.
Note: Some remote file systems, such as the Network File System (NFS), do not provide all the information that the df command needs. The df command prints blanks for statistics that the server does not provide. flags: -g Displays statistics in units of GB blocks. The output values for the file system statistics would be in floating point numbers as value of each unit in bytes is significantly high. -i Displays the number of free and used i-nodes for the file system; this output is the default when the specified file system is mounted. -I Displays information on the total number of blocks, the used space, the free space, the percentage of used space, and the mount point for the file system. -k Displays statistics in units of 1024-byte blocks. -m Displays statistics in units of MB blocks. The output values for the file system statistics would be in floating point numbers as value of each unit in bytes is significantly high. -M Displays the mount point information for the file system in the second column. -P Displays information on the file system in POSIX portable format. -s Gets file system statistics from the VFS specific file system helper instead of the statfs system call. Any arguments given when using the -s flag must be a JFS or Enhanced JFS filesystem mount point or device. The filesystem must also be listed in /etc/filesystems. -t Includes figures for total allocated space in the output. -v Displays all information for the specified file system. examples: To display information about all mounted file systems, enter: df If your system has the /, /usr, /site, and /usr/venus file systems mounted, the output from the df command resembles the following: Filesystem 512-blocks Free /dev/hd0 19368 9976 /dev/hd1 24212 4808 /dev/hd2 9744 9352 /dev/hd3 3868 3856
%Used 48% 80% 4% 0%
Iused 4714 5031 1900 986
%Iused 5% 19% 4% 0%
Mounted on / /usr /site /usr/venus
To display information about /test file system in 1024-byte blocks, enter: df -k /test Filesystem 1024 blocks Free %Used Iused %Iused Mounted on /dev/lv11 16384 15824 4% 18 1% /tmp/ravi1 This displays the file system statistics in 1024-byte disk blocks. To display information about /test file system in MB blocks, enter:
df -m /test Filesystem MB blocks Free %Used Iused %Iused Mounted on /dev/lv11 16.00 15.46 4% 18 1% /tmp/ravi1 This displays file system statistics in MB disk blocks rounded off to nearest 2nd decimal digit. To display information about the /test file system in GB blocks, enter: df -g /test Filesystem GB blocks Free %Used Iused %Iused Mounted on /dev/lv11 0.02 0.02 0% 18 1% /tmp/ravi1 This displays file system statistics in GB disk blocks rounded off to nearest 2nd decimal digit. To display available space on the file system in which your current directory resides, enter: cd/ df . The output from this command resembles the following: Device 512-blocks /dev/hd4 19368
free 9976
%used 48%
iused 4714
%iused 5%
Mounted on /
The defragfs command: --------------------defragfs Command Purpose Increases a file system's contiguous free space. Syntax defragfs [ -q | -r | -s] { Device | FileSystem } Description The defragfs command increases a file system's contiguous free space by reorganizing allocations to be contiguous rather than scattered across the disk. The file system to be defragmented can be specified with the Device variable, which is the path name of the logical volume (for example, /dev/hd4). It can also be specified with the FileSystem variable, which is the mount point in the /etc/filesystems file. The defragfs command is intended for fragmented and compressed file systems. However, you can use the defragfs command to increase contiguous free space in nonfragmented file systems. You must mount the file system read-write for this command to run successfully. Using the -q flag, the -r flag or the -s flag generates a fragmentation report. These flags do not alter the file system.
The defragfs command is slow against a JFS2 file system with a snapshot due to the amount of data that must be copied into snapshot storage object. The defragfs command issues a warning message if there are snapshots. The snapshot command can be used to delete the snapshots and then used again to create a new snapshot after the defragfs command completes. Flags -q Reports the current state of the file system. -r Reports the current state of the file system and the state that would result if the defragfs command is run without either the -q, -r or -s flag. -s Reports the fragmentation in the file system. This option causes defragfs to pass through meta data in the file system which may result in degraded performance. Output On a JFS filesystem, the definitions for the messages reported by the defragfs command are as follows: Number of free fragments The number of free fragments in the file system. Number of allocated fragments The number of allocated fragments in the file system. Number of free spaces shorter than a block The number of free spaces within the file system that are shorter than a block. A free space is a set of contiguous fragments that are not allocated. Number of free fragments in short free spaces The total number of fragments in all the short free spaces. A short free space is one that is shorter than a block. Number of fragments moved The total number of fragments moved. Number of logical blocks moved The total number of logical blocks moved. Number of allocation attempts The number of times free fragments were reallocated. Number of exact matches The number of times the fragments that are moved would fit exactly in some free space. Total number of fragments The total number of fragments in the file system. Number of fragments that may be migrated The number of fragments that may be moved during defragmentation. FileSystem filesystem is n percent fragmented Shows to what extent the file system is fragmented in percentage. On a JFS2 filesystem the definitions for the messages reported by the defragfs command are as follows: Total allocation groups The number of allocation groups in the file system. Allocation groups divide the space on a file system into chunks. Allocation groups allow JFS2 resource allocation policies to use well known methods for achieving good I/O performance.
Allocation groups defragmented The number of allocation groups that were defragmented. Allocation groups skipped - entirely free The number of allocation groups that were skipped because they were entirely free. Allocation groups skipped - too few free blocks The number of allocation groups that were skipped because there were too few free blocks in them for reallocation. Allocation groups skipped - contains a large contiguous free space The number of allocation groups that were skipped because they contained a large contiguous free space which is not worth defragmenting. Allocation groups are candidates for defragmenting The number of allocation groups that are fit for defragmenting. Average number of free runs in candidate allocation groups The average number of free runs per allocation group, for allocation groups that are found fit for defragmentation. A free run is a contiguous set of blocks which are not allocated. Total number of blocks The total number of blocks in the file system. Number of blocks that may be migrated The number of blocks that may be moved during defragmentation. FileSystem filesystem is n percent fragmented Shows to what extent the file system is fragmented in percentage. Examples: To defragment the /data1 file system located on the /dev/lv00 logical volume, enter: defragfs /data1 To defragment the /data1 file system by specifying its mount point, enter: defragfs /data1 To generate a report on the /data1 file system that indicates its current status as well as its status after being defragmented, enter: defragfs -r /data1 To generate a report on the fragmentation in the /data1 file system, enter: defragfs -s /data1 The fsck command: ----------------Purpose Checks file system consistency and interactively repairs the file system. Syntax fsck [ -n ] [ -p ] [ -y ] [ -dBlockNumber ] [ -f ] [ -ii-NodeNumber ] [ -o Options ] [ -tFile ] [ -V VfsName ] [ FileSystem1 - FileSystem2 ... ]
Description Attention: Always run the fsck command on file systems after a system malfunction. Corrective actions may result in some loss of data. The default action for each consistency correction is to wait for the operator to enter yes or no. If you do not have write permission for an affected file system, the fsck command defaults to a no response in spite of your actual response. Notes: The fsck command does not make corrections to a mounted file system. The fsck command can be run on a mounted file system for reasons other than repairs. However, inaccurate error messages may be returned when the file system is mounted. The fsck command checks and interactively repairs inconsistent file systems. You should run this command before mounting any file system. You must be able to read the device file on which the file system resides (for example, the /dev/hd0 device). Normally, the file system is consistent, and the fsck command merely reports on the number of files, used blocks, and free blocks in the file system. If the file system is inconsistent, the fsck command displays information about the inconsistencies found and prompts you for permission to repair them. The fsck command is conservative in its repair efforts and tries to avoid actions that might result in the loss of valid data. In certain cases, however, the fsck command recommends the destruction of a damaged file. If you do not allow the fsck command to perform the necessary repairs, an inconsistent file system may result. Mounting an inconsistent file system may result in a system crash. If a JFS2 file system has snapshots, the fsck command will attempt to preserve them. If this action fails, the snapshots cannot be guaranteed to contain all of the before-images from the snapped file system. The fsck command will delete the snapshots and the snapshot logical volumes. If you do not specify a file system with the FileSystem parameter, the fsck command checks all file systems listed in the /etc/filesystems file for which the check attribute is set to True. You can enable this type of checking by adding a line in the stanza, as follows: check=true You can also perform checks on multiple file systems by grouping the file systems in the /etc/filesystems file. To do so, change the check attribute in the /etc/filesystems file as follows: check=Number The Number parameter tells the fsck command which group contains a particular file system.
File systems that use a common log device should be placed in the same group. File systems are checked, one at a time, in group order, and then in the order that they are listed in the /etc/filesystems file. All check=true file systems are in group 1. The fsck command attempts to check the root file system before any other file system regardless of the order specified on the command line or in the /etc/filesystems file. The fsck command checks for the following inconsistencies: -Blocks or fragments allocated to multiple files. -i-nodes containing block or fragment numbers that overlap. -i-nodes containing block or fragment numbers out of range. -Discrepancies between the number of directory references to a file and the link count of the file. -Illegally allocated blocks or fragments. -i-nodes containing block or fragment numbers that are marked free in the disk map. -i-nodes containing corrupt block or fragment numbers. -A fragment that is not the last disk address in an i-node. This check does not apply to compressed file systems. -Files larger than 32KB containing a fragment. This check does not apply to compressed file systems. -Size checks: Incorrect number of blocks. Directory size not a multiple of 512 bytes. These checks do not apply to compressed file systems. -Directory checks: Directory entry containing an i-node number marked free in the i-node map. i-node number out of range. Dot (.) link missing or not pointing to itself. Dot dot (..) link missing or not pointing to the parent directory. Files that are not referenced or directories that are not reachable. -Inconsistent disk map. -Inconsistent i-node map. -Orphaned files and directories (those that cannot be reached) are, if you allow it, reconnected by placing them in the lost+found subdirectory in the root directory of the file system. The name assigned is the i-node number. If you do not allow the fsck command to reattach an orphaned file, it requests permission to destroy the file. In addition to its messages, the fsck command records the outcome of its checks and repairs through its exit value. This exit value can be any sum of the following conditions: 0 All checked file systems are now okay. 2 The fsck command was interrupted before it could complete checks or repairs. 4 The fsck command changed the file system; the user must restart the system immediately. 8 The file system contains unrepaired damage. When the system is booted from a disk, the boot process explicitly runs the fsck command,
specified with the -f and -p flags on the /, /usr, /var, and /tmp file systems. If the fsck command is unsuccessful on any of these file systems, the system does not boot. Booting from removable media and performing maintenance work will then be required before such a system will boot. If the fsck command successfully runs on /, /usr, /var, and /tmp, normal system initialization continues. During normal system initialization, the fsck command specified with the -f and -p flags runs from the /etc/rc file. This command sequence checks all file systems in which the check attribute is set to True (check=true). If the fsck command executed from the /etc/rc file is unable to guarantee the consistency of any file system, system initialization continues. However, the mount of any inconsistent file systems may fail. A mount failure may cause incomplete system initialization. Note: By default, the /, /usr, /var, and /tmp file systems have the check attribute set to False (check=false) in their /etc/filesystem stanzas. The attribute is set to False for the following reasons: The boot process explicitly runs the fsck command on the /, /usr, /var, and /tmp file systems. The /, /usr, /var, and /tmp file systems are mounted when the /etc/rc file is executed. The fsck command will not modify a mounted file system. Furthermore, the fsck command run on a mounted file system produces unreliable results. You can use the File Systems application in Web-based System Manager (wsm) to change file system characteristics. You could also use the System Management Interface Tool (SMIT) smit fsck fast path to run this command. Flags -dBlockNumber Searches for references to a specified disk block. Whenever the fsck command encounters a file that contains a specified block, it displays the i-node number and all path names that refer to it. For JFS2 filesystems, the i-node numbers referencing the specified block will be displayed but not their path names." -f Performs a fast check. Under normal circumstances, the only file systems likely to be affected by halting the system without shutting down properly are those that are mounted when the system stops. The -f flag prompts the fsck command not to check file systems that were unmounted successfully. The fsck command determines this by inspecting the s_fmod flag in the file system superblock. This flag is set whenever a file system is mounted and cleared when it is unmounted successfully. If a file system is unmounted successfully, it is unlikely to have any problems. Because most file systems
are unmounted successfully, not checking those file systems can reduce the checking time. -ii-NodeNumber Searches for references to a specified i-node. Whenever the fsck command encounters a directory reference to a specified i-node, it displays the full path name of the reference. -n Assumes a no response to all questions asked by the fsck command; does not open the specified file system for writing. -o Options Passes comma-separated options to the fsck command. The following options are currently supported for JFS (these options are obsolete for newer file systems and can be ignored): mountable Causes the fsck command to exit with success, returning a value of 0, if the file system in question is mountable (clean). If the file system is not mountable, the fsck command exits returning with a value of 8. mytype Causes the fsck command to exit with success (0) if the file system in question is of the same type as either specified in the /etc/filesystems file or by the -V flag on the command line. Otherwise, 8 is returned. For example, fsck -o mytype -V jfs / exits with a value of 0 if / (the root file system) is a journaled file system. -p Does not display messages about minor problems but fixes them automatically. This flag does not grant the wholesale license that the -y flag does and is useful for performing automatic checks when the system is started normally. You should use this flag as part of the system startup procedures, whenever the system is being run automatically. If the primary superblock is corrupt, the secondary superblock is verified and copied to the primary superblock. -tFile Specifies a File parameter as a scratch file on a file system other than the one being checked, if the fsck command cannot obtain enough memory to keep its tables. If you do not specify the -t flag and the fsck command needs a scratch file, it prompts you for the name of the scratch file. However, if you have specified the -p flag, the fsck command is unsuccessful. If the scratch file is not a special file, it is removed when the fsck command ends. -V VfsName Uses the description of the virtual file system specified by the VFSName variable for the file system instead of using the /etc/filesystems file to determine the description. If the -V VfsName flag is not specified on the command line, the /etc/filesystems file is checked and the vfs=Attribute of the matching stanza is assumed to be the correct file system type. -y Assumes a yes response to all questions asked by the fsck command. This flag lets the fsck command take any action it considers necessary. Use this flag only on severely damaged file systems. Examples To check all the default file systems, enter: fsck This command checks all the file systems marked check=true in the /etc/filesystems file.
This form of the fsck command asks you for permission before making any changes to a file system. To fix minor problems with the default file systems automatically, enter: fsck -p To check a specific file system, enter: fsck /dev/hd1 This command checks the unmounted file system located on the /dev/hd1 device.
31.6 DESCRIPTOR AREA'S: ----------------------- 1. VOLUME GROUP DESCRIPTOR AREA, VGDA Global to the VG: The VGDA, located at the beginning of each physical volume, contains information that describes all the LV's and all the PV's that belong to the VG of which that PV is a member. The VGDA makes a VG selfdescribing. An AIX System can read the VGDA on a disk, and from that, can determine what PV's and LV's are part of this VG. There are one or two copies per disk. - 2. VOLUME GROUP STATUS AREA, VGSA Tracks the state of mirrorred copies. The VGSA contains state information about physical partitions and physical volumes. For example, the VGSA knows if one PV in a VG is unavailable. Each PV has at least one VGDA/VGSA. The number of VGDA's contained on a single disk varies according to the number of disks in the VG. - 3. LOGICAL VOLUME CONTROL BLOCK, LVCB Contains LV attributes (policies, number of copies). The LVCB is located at the start of every LV. It contains information about the logical volume. You can however, use the mklv command with the -T option, to request that the LVCB will not be stored in the beginning of the LV. With Scalable VG's, LVCM info is no longer stored in the first user block of any LV. All relevant LVCM info is kept in the VGDA.
31.7 The lqueryvg command:
-------------------------The lqueryvg command reads the VGDA from a specified disk in a VG. Example: # lqueryvg -p hdisk1 -At # lqueryvg -Atp hdisk0 -p: which PV -A: show all available information -t: show descriptive tags Example: #lqueryvg -Atp hdisk0 Max LVs: 256 PP Size: 25 Free PPs: 468 LV count: 20 PV count: 2 Total VGDAs: 3 Conc Allowed: 0 MAX PPs per PV 1016 MAX PVs: 32 Conc Autovaryo 0 Varied on Conc 0 Logical: 00c665ed00004c0000000112b7408848.1 hd5 1 00c665ed00004c0000000112b7408848.2 hd6 1 00c665ed00004c0000000112b7408848.3 hd8 1 00c665ed00004c0000000112b7408848.4 hd4 1 00c665ed00004c0000000112b7408848.5 hd2 1 00c665ed00004c0000000112b7408848.6 hd9var 1 00c665ed00004c0000000112b7408848.7 hd3 1 00c665ed00004c0000000112b7408848.8 hd1 1 00c665ed00004c0000000112b7408848.9 hd10opt 1 00c665ed00004c0000000112b7408848.10 hd7 1 00c665ed00004c0000000112b7408848.11 hd7x 1 00c665ed00004c0000000112b7408848.12 beheerlv 1 00c665ed00004c0000000112b7408848.13 varperflv 1 00c665ed00004c0000000112b7408848.14 loglv00 1 00c665ed00004c0000000112b7408848.15 db2_server_v8 1 00c665ed00004c0000000112b7408848.16 db2_var_v8 1 00c665ed00004c0000000112b7408848.17 db2_admin_v8 1 00c665ed00004c0000000112b7408848.18 db2_adminlog_v8 1 00c665ed00004c0000000112b7408848.19 db2_dasscr_v8 1 00c665ed00004c0000000112b7408848.20 db2_Fixpak10 1 Physical: 00c665edb74079bc 2 0 00c665edb7f2987a 1 0 Total PPs: 1022 LTG size: 128 HOT SPARE: 0 AUTO SYNC: 0 VG PERMISSION: 0 SNAPSHOT VG: 0 IS_PRIMARY VG: 0 PSNFSTPP: 4352
VARYON MODE: VG Type: Max PPs:
0 0 32512
31.8 The lquerypv command: -------------------------------How do I find out what the maximum supported logical track group (LTG) size of my hard disk? You can use the lquerypv command with the -M flag. The output gives the LTG size in KB. For instance, the LTG size for hdisk0 in the following example is 256 KB. /usr/sbin/lquerypv -M hdisk0 256 -----run lquerypv -h core 6b0 to find the executable (probably man, but man may have called something else in the background) then run dbx path_/to_/executable core and run the subcommand dbx> where and paste the stack output, should be able to find it from there. also paste the level of fileset you are on for the executable lslpp -w /path_/to_/executable -> this will give fileset_name lslpp -l fileset_name ------Wie l,sst sich ein Storage Lock auf einer SAN-Disk brechen? Endlich die ersehnte SAN-Disk bekommen und dann das, es l,sst sich keine Volume Group darauf anlegen. # mkvg -f vpath100 gibt einen I/O Error. Was tun? H"chstwahrscheinlich befindet sich noch ein Lock auf der SAN-Disk. Dies l,sst sich mit dem Befehl
# lquerypv -ch /dev/vpath100 aufbrechen und die Volume Group kann angelegt werden. ------# lquerypv -h /dev/hdisk9 80 10 00000080 00001155 583CD4B0 00000000 00000000 # lquerypv 00000000 00000010 00000020 00000030 00000040 00000050 00000060 00000070 00000080 00000090 000000A0 000000B0 000000C0 000000D0 000000E0 000000F0
-h /dev/hdisk1 C9C2D4C1 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00C665ED B7F2987A 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
|...UX<..........|
|................| |................| |................| |................| |................| |................| |................| |................| |..e....z........| |................| |................| |................| |................| |................| |................| |................|
# lquerypv -h /dev/hdisk0 80 10 root@zd93l12:/root#lquerypv -h /dev/hdisk0 80 10 00000080 00C665ED B74079BC 00000000 00000000 |..e..@y.........|
31.9 The getlvcb command: ------------------------The LVCB stores attributes of a LV. The getlvcb command reads the LVCB of a specified LV. Displays a formatted output of the data in the LVCB of a LV. Example: # getlvcb -At hd2 # getlvcb -TA hd3 Displays the information held in the LVCB of LV hd3. 31.10 The putlvcb command: -------------------------Writes the control block information (only the specified fields) into block 0 of a logical volume (LVCB).
# putlvcb -t jfs lvdata writes the LV type jfs to the LVCB of LV lvdata.
32. Some Filesystem related errors in AIX: ==========================================
32.1 The root / Filesystem is full: =================================== Dealing with a 100% full root (/) filesystem in AIX Number one - DON'T Re-boot. Do a chfs -a size=+1 / (enter). increased by one physical partition.
The root filesystem will be
If the box is re-booted, shutdown, or crashes do the following: Load the AIX Installation CD #1 and type shutdown -Fr. Upon re-boot press F1 to enter the Systems Management Services (SMS) Menu. Click on the Multi-Boot icon. The bootlist needs to be changed so that CD0 is the first boot device. Shutdown and re-boot. Press F1 and enter. Press 1 and enter. Select Maintenance Mode option (3?). Select Access a Root Volume Group. Select the option that does NOT mount the filesystems. At the prompt, type mount /dev/hd4 (this is where the root filesystem lives) /mnt At the prompt type mount /dev/hd2 /usr Type df and enter.
Note filesystem sizes.
Now, chfs -a size=+1 / Type: Type:
df and enter. sync
Note that the filesystem / is larger.
You need to change your bootlist to boot off of hdisk0: Type: bootlist -m normal hdisk0 hdisk1 rmt0 cd0 and enter. Type: shutdown -Fr.
the system will re-boot and should come back online in it's proper state.
32.2 Fixing ODM problems on a VG which is not the rootvg: ========================================================= In the following examle, the VG is called "myvg" consisting of the Physical Volume hdisk3. 1. Unmount all filesystems in that VG first, otherwise you cannot varyoff the VG. Then varyoff the VG. # varyoffvg myvg 2. Now remove the complete information of that VG from ODM. The VGDA and LVCB on the actual disks are NOT touched by the exportvg command. # exportvg myvg 3. Now import the VG and create new ODM objects associated with that VG: # importvg -y myvg hdisk3 You only need to specify one intact PV of the VG in the above command. Any disk in the VG will have a VGDA which contains all neccessary information. The importvg command reads the VGDA and LVCB on that disk and creates completely new ODM entries.
32.3 Fixing ODM problems on the rootvg: ======================================= rvgrecover: ----------You can try to use the "rvgrecover" shell script. The rootvg cannot be varied off, like an ordinary VG, so the solution from the former section cannot be used. But the script "rvgrecover" issues a series of odmdelete statements, just like exportvg does. At the end of the script, an importvg is done. The importvg command, reads the VGDA and LVCB from the boot disk, resulting in new ODM entries. The rvgrecover script has the following contents: Reinitializing the rootvg Volume Group To reinitialize the rootvg volume group, copy the shell script to /bin/rvgrecover and run the following to make that file executable:
chmod +x /bin/rvgrecover Then run: /bin/rvgrecover Use the following shell script to reinitialize the ODM entries for the rootvg volume group: PV=/dev/ipldevice # PV=hdisk0 VG=rootvg cp /etc/objrepos/CuAt /etc/objrepos/CuAt.$$ cp /etc/objrepos/CuDep /etc/objrepos/CuDep.$$ cp /etc/objrepos/CuDv /etc/objrepos/CuDv.$$ cp /etc/objrepos/CuDvDr /etc/objrepos/CuDvDr.$$ lqueryvg -Lp $PV | awk '{ print $2 }' | while read LVname; do odmdelete -q "name = $LVname" -o CuAt odmdelete -q "name = $LVname" -o CuDv odmdelete -q "value3 = $LVname" -o CuDvDr done odmdelete -q "name = $VG" -o CuAt odmdelete -q "parent = $VG" -o CuDv odmdelete -q "name = $VG" -o CuDv odmdelete -q "name = $VG" -o CuDep odmdelete -q "dependency = $VG" -o CuDep odmdelete -q "value1 = 10" -o CuDvDr odmdelete -q "value3 = $VG" -o CuDvDr importvg -y $VG $PV # ignore lvaryoffvg errors varyonvg $VG
redefinevg: ----------redefinevg Command Purpose Redefines the set of physical volumes of the given volume group in the device configuration database. Syntax redefinevg { -d Device | -i Vgid } VolumeGroup Description During normal operations the device configuration database remains consistent with the Logical Volume Manager (LVM) information in the reserved area on the physical volumes. If inconsistencies occur between the device configuration database and the LVM, the redefinevg command determines which physical volumes belong to the specified volume group and re-enters this information in the device configuration database. The redefinevg command checks for inconsistencies by reading the reserved areas of all the configured physical volumes attached to the system.
Note: To use this command, you must either have root user authority or be a member of the system group. Flags -d Device The volume group ID, Vgid, is read from the specified physical volume device. You can specify the Vgid of any physical volume belonging to the volume group that you are redefining. -i Vgid The volume group identification number of the volume group to be redefined. Example To redefine rootvg physical volumes in the Device Configuration Database, enter a command similar to the following: # redefinevg -d hdisk0 rootvg synclvodm: ---------synclvodm Command Purpose Synchronizes or rebuilds the logical volume control block, the device configuration database, and the volume group descriptor areas on the physical volumes. Syntax synclvodm [ -v ] VolumeGroup [ LogicalVolume ... ] Description During normal operations, the device configuration database remains consistent with the logical volume manager information in the logical volume control blocks and the volume group descriptor areas on the physical volumes. If for some reason the device configuration database is not consistent with Logical Volume Manager information, the synclvodm command can be used to resynchronize the database. The volume group must be active for the resynchronization to occur (see varyonvg). If logical volume names are specified, only the information related to those logical volumes is updated. Attention: Do not remove the /dev entries for volume groups or logical volumes. Do not change the device configuration database entries for volume groups or logical volumes using the object data manager. Note: To use this command, you must either have root user authority or be a member of the system group. Flags -v verbose
Example To synchronize the device configuration database with the logical volume manager information for rootvg, enter the following: synclvodm rootvg
32.4 How to Replace a Disk?: ============================ 1. Short version for normal VG (not rootvg) and the disk is working: -------------------------------------------------------------------extendvg VolumeGroupName hdiskY migratepv hdiskX hdiskY reducevg -d VolumeGroupName hdiskX 2. More Detail: --------------2.1 The disk is mirrored: ------------------------1. Remove all copies from the disk: # unmirrorvg vg_name hdiskX 2. Remove disk from VG: # reducevg vg_name hdiskX 3. Remove disk from ODM: # rmdev -l hdiskX -d 4. Add new disk to the system. 5. Add the new disk to the VG: # extendvg vg_name hdiskY 6. Create new copies: # mirrorvg vg_name # syncvg vg_name 2.2 The disk was not mirrored, or you want to replace a working disk: --------------------------------------------------------------------1. Add the new disk to the system. 2. Add the disk to the VG: # extendvg vg_name hdiskY 3. Migrate old disk to new disk: # migratepv hdiskX hdiskY
4. Remove old disk from VG: # reducevg vg_name hdiskX 5. Remove old disk from ODM: # rmdev -l hdiskX -d 2.3 Replace the disk in the rootvg: ----------------------------------1. Add the new disk to the system. 2. Add the disk to the VG: # extendvg rootvg hdiskY 3. The diskX contains hd5? If so: # # # #
migratepv -l hd5 hdiskX hdiskY bosboot -ad /dev/hdiskY chpv -c hdiskX bootlist -m normal hdiskY
If hdiskX contains the primary dump device, you must deactivate it: # sysdumpdev -p /dev/sysdumpnull 4. Migrate old disk to new disk: # migratepv hdiskX hdiskY If the primary dump device has been deactivated, activate it again # sysdumpdev -p /dev/hdX 5. Remove old disk from VG: # reducevg rootvg hdiskX 6. Remove old disk from ODM: # rmdev -l hdiskX -d
32.5 Filesystem errors: =======================
32.5.1 ksh: Invalid file system control data detected: ====================================================== Note 1: ------Q: Anybody recognize this? This directory seems to be missing the ".", I can't
umount, can't remove the directory, can't copy a good directory over it, etc. spiderman# cd probes spiderman# pwd /opt/diagnostics/probes spiderman# ls -la ls: 0653-341 The file . does not exist. spiderman# cd .. spiderman# ls -la probes ls: probes: Invalid file system control data detected. total 0 spiderman# spiderman# fuser /opt /opt: spiderman# umount /opt umount: 0506-349 Cannot unmount /dev/hd10opt: The requested resource is busy. spiderman# umount /dev/hd10opt umount: 0506-349 Cannot unmount /dev/hd10opt: The requested resource is busy. spiderman# fsck /opt ** Checking /dev/hd10opt (/opt) MOUNTED FILE SYSTEM; WRITING SUPPRESSED; Checking a mounted filesystem does not produce dependable results. ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames DIRECTORY CORRUPTED (NOT FIXED) DIRECTORY CORRUPTED (NOT FIXED) Directory /diagnostics/probes, '.' entry is missing. (NOT FIXED) Directory /diagnostics/probes, '..' entry is missing. (NOT FIXED) ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts link count directory I@98 owner=bin mode$0755 sizeQ2 mtime=May 13 14:54 2005 count 3 should be 2 (NOT ADJUSTED) link count directory I@99 owner=bin mode$0755 size24 mtime=Jan 10 13:45 2005 count 2 should be 1 (NOT ADJUSTED) Unreferenced file IA06 owner=bin mode0555 sizee56 mtime=Jul 07 14:25 2004 (NOT RECONNECTED) Unreferenced file IA06 (NOT CLEARED) Unreferenced file IA07 owner=bin mode0555 size)12 mtime=Jul 07 14:25 2004 (NOT RECONNECTED) etc.... A: Some good news here. Yes, your directory is hosed, but the important things is that all a directory is a repository for storing inode numbers and associated (human readable) file names. Since fsck is so nicely
generating all of those now currently inaccessible inode numbers, a find command can be used to move them into a new directory. Once the old directory is empty, you can (hopefully) rm -r it. Here's what you need to do. a) Get all the inode numbers generated from your fsck b) put them into a variable (e.g. lost_inodes="4099 4106....etc." c) Make a target directory for the lost inodes to be moved into: mkdir /tmp/recovery d) cd into your problem File System: cd /opt d) Run a loop using find: for i in ${lost_inodes} do find . -inum ${i} mv * /tmp/recovery \; echo "Moved and recovered inode # ${i}" done That should do it. Let me know if it works ok! BTW, the new "file name" should be the inode number of the file. You will have to rename the files as needed. Note 2: IY94101: J2_DMAP_CORRUPT ERROR REPORT AFTER SHRINKING JFS2 FILESYSTEM ---------------------------------------------------------------------------http://www-1.ibm.com/support/docview.wss?uid=isg1IY94101 IY94101: J2_DMAP_CORRUPT ERROR REPORT AFTER SHRINKING JFS2 FILESYSTEM APAR status Closed as program error. Error description After shrinking a filesystem, J2_DMAP_CORRUPT reports appear in the error report and some file creates/writes fail with "Invalid file system control data detected". Local fix Problem summary Problem conclusion Temporary fix Comments APAR information APAR number IY94101 Reported component name AIX 5.3 Reported component ID 5765G0300 Reported release 530 Status CLOSED PER PE NoPE HIPER NoHIPER Submitted date 2007-01-26 Closed date 2007-01-29 Last modified date 2007-05-25
APAR is sysrouted FROM one or more of the following: APAR is sysrouted TO one or more of the following: Publications Referenced Fix information Fixed component name AIX 5.3 Fixed component ID 5765G0300
Note 3: ------Q: Since applying ML7 for AIX 5.1 I have been getting file corruption error messages on a particular filesystem and the only way to fix it is to umount the filesystem and fsck it. I thought it might be a hardware problem but now it is also happening on another machine I put the ML7 on and it is happening to the same filesystem (one machine is a test server of the other). The only unique thing about the filesystem is that it is not in rootvg and it is large -1281228 1024-blocks. Has anyone heard of this? Below is the error I am getting: LABEL: JFS_META_CORRUPTION IDENTIFIER: 684A365B Date/Time: Tue Apr 26 13:45:26 EDT Sequence Number: 2023 Machine Id: 0000F11F4C00 Node Id: XX00 Class: U Type: UNKN Resource Name: SYSPFS Resource Class: NONE Resource Type: NONE Location: NONE VPD: Description FILE SYSTEM CORRUPTION Probable Causes INVALID FILE SYSTEM CONTROL DATA Recommended Actions PERFORM FULL FILE SYSTEM RECOVERY USING FSCK UTILITY OBTAIN
DUMP
CHECK ERROR LOG FOR ADDITIONAL RELATED ENTRIES
Failure Causes ADAPTER HARDWARE OR MICROCODE DISK DRIVE HARDWARE OR MICROCODE SOFTWARE PROGRAM STORAGE CABLE LOOSE, DEFECTIVE, OR UNTERMINATED Recommended Actions CHECK CABLES AND THEIR CONNECTIONS INSTALL LATEST ADAPTER AND DRIVE MICROCODE INSTALL LATEST STORAGE DEVICE DRIVERS IF PROBLEM PERSISTS, CONTACT APPROPRIATE SERVICE REPRESENTATIVE Detail Data FILE NAME xix_lookup.c LINE NO. 300 MAJOR/MINOR DEVICE NUMBER 0026 0006 ADDITIONAL INFORMATION 4A46 5345 426E 8C46 0000 000E 0000 001D 0003 0610 0000 0000 0000 0000 0000 0002 164D A330 0001 86D3 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 -------------------------------------------------------------------------LABEL: JFS_FSCK_REQUIRED IDENTIFIER: CD546B25 Date/Time: Tue Apr 26 13:45:26 EDT Sequence Number: 2022 Machine Id: 0000F11F4C00 Node Id: XX00 Class: O Type: INFO Resource Name: SYSPFS Description FILE SYSTEM RECOVERY REQUIRED Recommended Actions PERFORM FULL FILE SYSTEM RECOVERY USING FSCK UTILITY
Detail Data MAJOR/MINOR DEVICE NUMBER 0026 0006 FILE SYSTEM DEVICE AND MOUNT POINT /dev/lv04, /opt/egate Note 3: ------Q: How can I remove a bizarre, irremovable file from a directory? I've tried every way of using /bin/rm and nothing works." A: In some rare cases a strangely-named file will show itself in your directory and appear to be un-removable with the rm command. Here is will the use of ls -li and find with its -inum [inode] primary does the job. Let's say that ls -l shows your irremovable as -rw-------
1 smith
smith
0 Feb
1 09:22 ?*?*P
Type: ls -li to get the index node, or inode. 153805 -rw-------
1 smith
smith
0 Feb
1 09:22 ?*?^P
The inode for this file is 153805. Use find -inum [inode] to make sure that the file is correctly identified. % find -inum 153805 -print ./?*?*P Here, we see that it is. Then used the -exec functionality to do the remove. . % find . -inum 153805 -print -exec /bin/rm {} \; Note that if this strangely named file were not of zero-length, it might contain accidentally misplaced and wanted data. Then you might want to determine what kind of data the file contains and move the file to some temporary directory for further investigation, for example: % find . -inum 153805 -print -exec /bin/mv {} unknown.file \; Will rename the file to unknown.file, so you can easily inspect it.
Another way to remove strangely-named files is to use "ls -q" or "cat -v" to show the special characters, and then use shell's globbing mechanism to delete the file. $ ls -????*'? $ ls | cat -v -^B^C?^?*' $ rm ./-'^B'* $ ls
-- achieved by typing control-V control-B
the argument given to rm is a judicious selection of glob wildcards (*'s) and sufficient control characters to uniquely identify the file. The leading "./" is useful when the file begins with a hyphen. These binary name files are caused by: * accidental cut-and-pastes to shell prompts - especially when you paste something of the form: "junk > garbage" because the shell creates the file "garbage" before trying to execute the command "junk" * filesystem corruption (in which case touching the filesystem any more can really stuff things up) If you discover that you have two files of the same name, one of the files probably has a bizarre (and unprintable) character in its name. Most probably, this unprintable character is a backspace. For example: $ ls filename filename $ ls -q filename fl?ilename $ ls | cat -v filename fl^Hilename
32.5.2 More on Filesystem errors (1): ===================================== Note 1: ------Q: Hi all, I have a error message complaining about filesystem being full.
but df does not sure any filesystem being full. The error report gives me the major/minor number: 0027/0004 I went to /dev dir, and searched for the numbers, but it turns out to be ptyp4. Why is that? What does this mean? Any suggestion? A: Those numbers are reported in hex, the actual major/minor #'s are 39 and 4 A: Convert the errpt #'s to hex. The use ls -l to find them. Note 2: ------Q: Hi, I get a error concerning a filesystem. Now I have 2 questions: - What is the way to find out which filesystems is concerned? - What can I do? Because all fs have unused space. I cannot find any fs with 100% in use. LABEL: J2_FS_FULL IDENTIFIER: CED6B4B5 Date/Time: Mon Dec 27 12:49:35 NFT Sequence Number: 3420 Machine Id: 00599DDD4C00 Node Id: srvdms0 Class: O Type: INFO Resource Name: SYSJ2 Description UNABLE TO ALLOCATE SPACE IN FILE SYSTEM Probable Causes FILE SYSTEM FULL Recommended Actions INCREASE THE SIZE OF THE ASSOCIATED FILE SYSTEM REMOVE UNNECESSARY DATA FROM FILE SYSTEM USE FUSER UTILITY TO LOCATE UNLINKED FILES STILL REFERENCED Detail Data JFS2 MAJOR/MINOR DEVICE NUMBER 002B 000B A: 002b is 2*16+11 -->43
ls -l /dev|grep 43, 000b is 11 --> look for 43, 11 Date: Wed, 29 Dec 2004 11:06:27 +0000 To: [email protected] Q: Subject Re: error concerning filesystem [Virus checked] Hi Holger, A small query...how did you arrive at this figure of 43 from the error code. The decimal value of B is 11 but I could not understand the 2*16.. can you please exp this.... A: The major/minor numbers (002B 000B) are in hex: hex abcd = a*16^3+b*16^2+c*16^1+d therefore hex 002B=0*16^3+0*16^2+2*16^1+11=2*16+11 Note 3: AIX superblock issues: ------------------------------- Hint 1 for AIX: -- --------------thread: Use this command in case the superblock is corrupted. This will restore the BACKUP COPY of the superblock to the CURRENT copy. # dd count=1 bs=4k skip=31 seek=1 if=/dev/hd4 of=/dev/hd4 # fsck /dev/hd4 2>&1 | tee /tmp/fsck.errors Note: fuser Identifies processes using a file or file system # fuser -u /dev/hd3 Sample output: /dev/hd3: 2964(root) 6615c(root) 8465(casado) 11290(bonner) -- Hint 2 for AIX: -- ---------------
http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp? topic=/com.ibm.aix.howtos/doc/howto/HT_baseadmn_badmagnumber.htm Fixing a corrupted magic number in the file system superblock If the superblock of a file system is damaged, the file system cannot be accessed. You can fix a corrupted magic number in the file system superblock. Most damage to the superblock cannot be repaired. The following procedure describes how to repair a superblock in a JFS file system when the problem is caused by a corrupted magic number. If the primary superblock is corrupted in a JFS2 file system, use the fsck command to automatically copy the secondary superblock and repair the primary superblock. In the following scenario, assume /home/myfs is a JFS file system on the physical volume /dev/lv02. The information in this how-to was tested using AIXr 5.2. If you are using a different version or level of AIX, the results you obtain might vary significantly. 1. Unmount the /home/myfs file system, which you suspect might be damaged, using the following command: # umount /home/myfs 2. To confirm damage to the file system, run the fsck command against the file system. For example: # fsck -p /dev/lv02 If the problem is damage to the superblock, the fsck command returns one of the following messages: fsck: Not an AIXV5 file system OR Not a recognized filesystem type 3. With root authority, use the od command to display the superblock for the file system, as shown in the following example: # od -x -N 64 /dev/lv02 +0x1000 Where the -x flag displays output in hexadecimal format and the -N flag instructs the system to format no more than 64 input bytes from the offset parameter (+), which specifies the point in the file where the file output begins. The following is an example output: 0001000 0001010 0001020 0001030
1234 0001 3300 0000
0234 8000 0000 0001
0000 1000 000a 0000
0000 0000 0003 0200
0000 2f6c 0100 0000
4000 7633 0000 2000
0000 0000 2f28 0000
000a 6c76 0383 0000
0001040 In the preceding output, note the corrupted magic value at 0x1000 (1234 0234). If all defaults were taken when the file system was created, the magic number should be 0x43218765. If any defaults were overridden, the magic number should be 0x65872143. 4. Use the od command to check the secondary superblock for a correct magic number. An example command and its output follows: # od -x -N 64 /dev/lv02 +0x1f000 001f000 001f010 001f020 001f030 001f040
6587 0001 3300 0000
2143 8000 0000 0001
0000 1000 000a 0000
0000 0000 0003 0200
0000 2f6c 0100 0000
4000 7633 0000 2000
0000 0000 2f28 0000
000a 6c76 0383 0000
Note the correct magic value at 0x1f000. 5. Copy the secondary superblock to the primary superblock. An example command and output follows: # dd count=1 bs=4k skip=31 seek=1 if=/dev/lv02 of=/dev/lv02 dd: 1+0 records in. dd: 1+0 records out. Use the fsck command to clean up inconsistent files caused by using the secondary superblock. For example: # fsck /dev/lv02 2>&1 | tee /tmp/fsck.errs For more information The fsck and od command descriptions in AIX 5L Version 5.3 Commands Reference, Volume 4 AIX Logical Volume Manager from A to Z: Introduction and Concepts, an IBM Redbook AIX Logical Volume Manager from A to Z: Troubleshooting and Commands, an IBM Redbook "Boot Problems" in Problem Solving and Troubleshooting in AIX 5L, an IBM Redbook
Note 4: Linux superblock issues: -------------------------------1. DAMAGED SUPERBLOCK
If a filesystem check fails and returns the error message "Damaged Superblock" you're lost . . . . . . . or not ? Well, not really, the damaged "superblock" can be restored from a backup. There are several backups stored on the harddisk. But let me first have a go at explaining what a "superblock"is. A superblock is located at position 0 of every partition, contains vital information about the filesystem and is needed at a fielsystem check. The information stored in the superblock are about what sort of fiesystem is used, the I-Node counts, block counts, free blocks and I-Nodes, the numer of times the filesystem was mounted, date of the last filesystem check and the first I-Node where / is located. Thus, a damaged superblock means that the filesystem check will fail. Our luck is that there are backups of the superblock located on several positions and we can restore them with a simple command. The usual ( and only ) positions are: 8193, 32768, 98304, 163840, 229376 and 294912. ( 8193 in many cases only on older systems, 32768 is the most current position for the first backup ) You can check this out and have a lot more info about a particular partition you have on your HD by: CODE # dumpe2fs /dev/hda5 You will see that the primary superblock is located at position 0, and the first backup on position 32768. O.K. let's get serious now, suppose you get a "Damaged Superblock" error message at filesystem check ( after a power failure ) and you get a root-prompt in a recovery console, then you give the command: CODE # e2fsck -b 32768 /dev/hda5 don't try this on a mounted filesystem It will then check the filesystem with the information stored in that backup superblock and if the check was successful it will restore the backup to position 0. Now imagine the backup at position 32768 was damaged too . . . then you just try again with the backup stored at position 98304, and 163840, and 229376 etc. etc. until you find an undamaged backup ( there are five backups so if at least one of those five is okay it's bingo ! )
So next time don't panic . . just get the paper where you printed out this Tip and give the magic command CODE # e2fsck -b 32768 /dev/hda5
32.6 Undelete programs: ======================= Note 1: AIX and JFS ------------------/ *********************************************************************** ****** * rsb.c - Read Super Block. Allows a jfs superblock to be dumped, inode * table to be listed or specific inodes data pointers to be chased and * dumped to standard out (undelete). * * Phil Gibbs - Trinem Consulting ([email protected] ) ********************************************************************** ******/ #include #include #include #include #include #include #include #include #define FOUR_MB #define THIRTY_TWO_KB extern extern extern extern
(1024*1024*4) (1024*32)
int optind; int Optopt; int Opterr; char *optarg;
void PrintSep() { int k=80;
}
while (k) { putchar('-'); k--; } putchar('\n');
char *UserName(uid_t uid) 33333{
char replystr[10]; struct passwd *res; res=getpwuid(uid); if (res->pw_name[0]) { return res->pw_name; } else { sprintf(replystr,"%d",uid); return replystr; } } char *GroupName(gid_t gid) { struct group *res; res=getgrgid(gid); return res->gr_name; } ulong NumberOfInodes(struct superblock *sb) { ulong MaxInodes; ulong TotalFrags; if (sb->s_version==fsv3pvers) { TotalFrags=(sb->s_fsize*512)/sb->s_fragsize; MaxInodes=(TotalFrags/sb->s_agsize)*sb->s_iagsize; } else { MaxInodes=(sb->s_fsize*512)/sb->s_bsize; } return MaxInodes; } void AnalyseSuperBlock(struct superblock *sb) { ulong TotalFrags; PrintSep(); printf("SuperBlock Details:\n-------------------\n"); printf("File system size: %ld x 512 bytes (%ld Mb)\n", sb->s_fsize, (sb->s_fsize*512)/(1024*1024)); printf("Block size: %d bytes\n",sb->s_bsize); printf("Flags: "); switch (sb->s_fmod) { case (char)FM_CLEAN: break; case (char)FM_MOUNT:
printf("mounted "); break; case (char)FM_MDIRTY: printf("mounted dirty "); break; case (char)FM_LOGREDO: printf("log redo failed "); break; default: printf("Unknown flag "); break;
} if (sb->s_ronly) printf("(read-only)"); printf("\n"); printf("Last SB update at: %s",ctime(&(sb->s_time))); printf("Version: %s\n", sb->s_version?"1 - fsv3pvers":"0 - fsv3vers"); printf("\n"); if (sb->s_version==fsv3pvers) { TotalFrags=(sb->s_fsize*512)/sb->s_fragsize; printf("Fragment size: %5d ",sb>s_fragsize); printf("inodes per alloc: %8d\n",sb->s_iagsize); printf("Frags per alloc: %5d ",sb->s_agsize); printf("Total Fragments: %8d\n",TotalFrags); printf("Total Alloc Grps: %5d ", TotalFrags/sb->s_agsize); printf("Max inodes: %8ld\n",NumberOfInodes(sb)); } else { printf("Total Alloc Grps: %5d ", (sb->s_fsize*512)/sb->s_agsize); printf("inodes per alloc: %8d\n",sb->s_agsize); printf("Max inodes: %8ld\n",NumberOfInodes(sb)); } PrintSep(); } void ReadInode( FILE *in, ulong StartInum, struct dinode *inode, ulong InodesPerAllocBlock, ulong AllocBlockSize) { off_t SeekPoint; long BlockNumber; int OffsetInBlock; static struct dinode I_NODES[PAGESIZE/DILENGTH]; ulong AllocBlock; ulong inum; static off_t LastSeekPoint=-1; AllocBlock=(StartInum/InodesPerAllocBlock); BlockNumber=(StartInum-(AllocBlock*InodesPerAllocBlock))/ (PAGESIZE/DILENGTH);
}
OffsetInBlock=(StartInum-(AllocBlock*InodesPerAllocBlock))(BlockNumber*(PAGESIZE/DILENGTH)); SeekPoint=(AllocBlock)? (BlockNumber*PAGESIZE)+(AllocBlock*AllocBlockSize): (BlockNumber*PAGESIZE)+(INODES_B*PAGESIZE); if (SeekPoint!=LastSeekPoint) { sync(); fseek(in,SeekPoint,SEEK_SET); fread(I_NODES,PAGESIZE,1,in); LastSeekPoint=SeekPoint; } *inode=I_NODES[OffsetInBlock];
void DumpInodeContents( FILE ulong ulong ulong ulong { struct dinode ulong char ulong int int ulong int
long inode, *in, InodesPerAllocBlock, AllocBlockSize, Mask, Multiplier) DiskInode; SeekPoint; Buffer[4096]; FileSize; k; BytesToRead; *DiskPointers; NumPtrs;
ReadInode(
in, inode, &DiskInode, InodesPerAllocBlock, AllocBlockSize); FileSize=DiskInode.di_size; if (FileSize>FOUR_MB) { /* Double indirect mapping */ } else if (FileSize>THIRTY_TWO_KB) { /* Indirect mapping */ SeekPoint=DiskInode.di_rindirect & Mask; SeekPoint=SeekPoint*Multiplier; DiskPointers=(ulong *)malloc(1024*sizeof(ulong)); fseek(in,SeekPoint,SEEK_SET); fread(DiskPointers,1024*sizeof(ulong),1,in); NumPtrs=1024; } else { /* Direct Mapping */ DiskPointers=&(DiskInode.di_rdaddr[0]);
}
NumPtrs=8;
for (k=0;k<=NumPtrs && FileSize;k++) { SeekPoint=(DiskPointers[k] & Mask); SeekPoint=SeekPoint*Multiplier; BytesToRead=(FileSize>sizeof(Buffer))? sizeof(Buffer):FileSize; fseek(in,SeekPoint,SEEK_SET); fread(Buffer,BytesToRead,1,in); FileSize=FileSize-BytesToRead; write(1,Buffer,BytesToRead); } } void DumpInodeList(
{
FILE ulong ulong ulong
long struct dinode struct tm
*in, MaxInodes, InodesPerAllocBlock, AllocBlockSize) inode; DiskInode; *TimeStruct;
printf(" Inode Links User Group Size ModDate\n"); printf("-------- ----- -------- -------- --------------\n"); for (inode=0;inode<=MaxInodes;inode++) { ReadInode( in, inode, &DiskInode, InodesPerAllocBlock, AllocBlockSize); if (DiskInode.di_mtime) { TimeStruct=localtime((long *)&DiskInode.di_mtime); printf("%8d %5d %8s %8s %8d %02d/%02d/%4d\n", inode, DiskInode.di_nlink, UserName(DiskInode.di_uid), GroupName(DiskInode.di_gid), DiskInode.di_size, TimeStruct->tm_mday, TimeStruct->tm_mon, TimeStruct->tm_year+1900); } } } void ExitWithUsageMessage() {
fprintf(stderr,"USAGE: rsb [-i inode] [-d] [-s] \n"); exit(1); } main(int argc,char **argv) { FILE struct superblock short long struct dinode ulong ulong ulong ulong ulong int int int
*in; SuperBlock; Valid; inode=0; DiskInode; AllocBlockSize; InodesPerAllocBlock; MaxInodes; Mask; Multiplier; option; DumpSuperBlockFlag=0; DumpFlag=0;
while ((option=getopt(argc,argv,"i:ds")) != EOF) { switch(option) { case 'i': /* Inode specified */ inode=atol(optarg); break; case 'd': /* Dump flag */ DumpFlag=1; break; case 's': /* List Superblock flag */ DumpSuperBlockFlag=1; break; default: break; } } if (strlen(argv[optind])) in=fopen(argv[optind],"r"); else ExitWithUsageMessage(); if (in) {
fseek(in,SUPER_B*PAGESIZE,SEEK_SET); fread(&SuperBlock,sizeof(SuperBlock),1,in); switch (SuperBlock.s_version) { case fsv3pvers: Valid=! strncmp(SuperBlock.s_magic,fsv3pmagic,4); InodesPerAllocBlock=SuperBlock.s_iagsize; AllocBlockSize=
SuperBlock.s_fragsize*SuperBlock.s_agsize; Multiplier=SuperBlock.s_fragsize; Mask=0x3ffffff; break; case fsv3vers: Valid=! strncmp(SuperBlock.s_magic,fsv3magic,4); InodesPerAllocBlock=SuperBlock.s_agsize; AllocBlockSize=SuperBlock.s_agsize*PAGESIZE; Multiplier=SuperBlock.s_bsize; Mask=0xfffffff; break; default: Valid=0; break; } if (Valid) { if (DumpSuperBlockFlag==1) { AnalyseSuperBlock(&SuperBlock); } MaxInodes=NumberOfInodes(&SuperBlock); if (DumpFlag==1) { if (inode) DumpInodeContents(inode,in,InodesPerAllocBlock,AllocBlockSize,Mask,Mult iplier); else DumpInodeList(in,MaxInodes,InodesPerAllocBlock,AllocBlockSize); } } else { fprintf(stderr,"Superblock - bad magic number\n"); exit(1); } } else { fprintf(stderr,"couldn't open "); perror(argv[optind]); exit(1); } }
Note 2: Undelete a text file on most unixes (no garantee): ----------------------------------------------------------
Works mainly on Linux Distro's Using grep (traditional UNIX way) to recover files Use following grep syntax: # grep -b 'search-text' /dev/partition > file.txt OR # grep -a -B[size before] -A[size after] `text' /dev/[your_partition] > file.txt Where, -i : Ignore case distinctions in both the PATTERN and the input files i.e. match both uppercase and lowercase character. -a : Process a binary file as if it were text -B Print number lines/size of leading context before matching lines. -A: Print number lines/size of trailing context after matching lines. To recover text file starting with "nixCraft" word on /dev/sda1 you can try following command: # grep -i -a -B10 -A100 'nixCraft' /dev/sda1 > file.txt Next use vi to see file.txt. This method is ONLY useful if deleted file is text file. If you are using ext2 file system, try out recover command. . Note 3: ------For AIX there are undelete tools: http://www.compunix.com/ Note 4: lsof and Linux: ----------------------Bring back deleted files with lsof By Michael Stutz on November 16, 2006 (8:00:00 AM) Briefly, a file as it appears somewhere on a Linux filesystem is actually just a link to an inode, which contains all of the file's properties, such as permissions and ownership, as well as the addresses of the data blocks where the file's content is stored on disk. When you rm a file, you're removing the link that points to its inode, but not the inode itself; other processes (such as your audio player) might still have it open. It's only after they're through and all links are removed that an inode and the data blocks it pointed to are made available for writing. This delay is your key to a quick and happy recovery: if a process still has the file open, the data's there somewhere, even though according to the directory listing the file already appears to be gone. This is where the Linux process pseudo-filesystem, the /proc directory, comes into play. Every process on
the system has a directory here with its name on it, inside of which lies many things -including an fd ("file descriptor") subdirectory containing links to all files that the process has open. Even if a file has been removed from the filesystem, a copy of the data will be right here: /proc/process id/fd/file descriptor To know where to go, you need to get the id of the process that has the file open, and the file descriptor. These you get with lsof, whose name means "list open files." (It actually does a whole lot more than this and is so useful that almost every system has it installed. If yours isn't one of them, you can grab the latest version straight from its author.) Once you get that information from lsof, you can just copy the data out of /proc and call it a day. This whole thing is best demonstrated with a live example. First, create a text file that you can delete and then bring back: $ man lsof | col -b > myfile Then have a look at the contents of the file that you just created: $ less myfile You should see a plaintext version of lsof's huge man page looking out at you, courtesy of less. Now press Ctrl-Z to suspend less. Back at a shell prompt make sure your file is still there: $ ls -l myfile -rw-r--r-- 1 jimbo jimbo 114383 Oct 31 16:14 myfile $ stat myfile File: `myfile' Size: 114383 Blocks: 232 IO Block: 4096 regular file Device: 341h/833d Inode: 1276722 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1010/ jimbo) Gid: ( 1010/ jimbo) Access: 2006-10-31 16:15:08.423715488 -0400 Modify: 2006-10-31 16:14:52.684417746 -0400 Change: 2006-10-31 16:14:52.684417746 -0400 Yup, it's there all right. OK, go ahead and oops it: $ rm myfile $ ls -l myfile ls: myfile: No such file or directory $ stat myfile stat: cannot stat `myfile': No such file or directory $ It's gone.
At this point, you must not allow the process still using the file to exit, because once that happens, the file will really be gone and your troubles will intensify. Your background less process in this walkthrough isn't going anywhere (unless you kill the process or exit the shell), but if this were a video or sound file that you were playing, the first thing to do at the point where you realize you deleted the file would be to immediately pause the application playback, or otherwise freeze the process, so that it doesn't eventually stop playing the file and exit. Now to bring the file back. First see what lsof has to say about it: $ lsof | grep myfile less 4158 jimbo 4r REG 3,65 114383 1276722 /home/jimbo/myfile (deleted) The first column gives you the name of the command associated with the process, the second column is the process id, and the number in the fourth column is the file descriptor (the "r" means that it's a regular file). Now you know that process 4158 still has the file open, and you know the file descriptor, 4. That's everything you have to know to copy it out of /proc. You might think that using the -a flag with cp is the right thing to do here, since you're restoring the file -but it's actually important that you don't do that. Otherwise, instead of copying the literal data contained in the file, you'll be copying a now-broken symbolic link to the file as it once was listed in its original directory: $ ls -l /proc/4158/fd/4 lr-x------ 1 jimbo jimbo 64 Oct 31 16:18 /proc/4158/fd/4 -> /home/jimbo/myfile (deleted) $ cp -a /proc/4158/fd/4 myfile.wrong $ ls -l myfile.wrong lrwxr-xr-x 1 jimbo jimbo 24 Oct 31 16:22 myfile.wrong -> /home/jimbo/myfile (deleted) $ file myfile.wrong myfile.wrong: broken symbolic link to `/home/jimbo/myfile (deleted)' $ file /proc/4158/fd/4 /proc/4158/fd/4: broken symbolic link to `/home/jimbo/myfile (deleted)' So instead of all that, just a plain old cp will do the trick: $ cp /proc/4158/fd/4 myfile.saved And finally, verify that you've done good: $ ls -l myfile.saved -rw-r--r-- 1 jimbo jimbo 114383 Oct 31 16:25 myfile.saved $ man lsof | col -b > myfile.new $ cmp myfile.saved myfile.new No complaints from cmp -- your restoration is the real deal. Incidentally, there are a lot of useful things you can do with lsof in addition to rescuing lost files.
32.7 Some notes about disks on x86 systems: MBR and Partition Bootsector: ======================================================================= == The following applies to PC's and x86 based Servers. There are two sectors on the disk that are critical to starting the computer: - Master Boot Record - Partition Boot Sector The MBR is created when you create the first partition on the harddisk. The location is always cylinder 0, head 0 and sector 1. The MBR contains the Partition Table for the disk and a small amount of executable code. On x86 machines, this executable code examines the Partition Table and identifies the system partition. The code then finds the system partition's starting location on the disk, and loads an copy of its Partition Boot Sector into memory. If you would take a look at the MBR, you would find: The first 446 bytes in the sector is the MBR. After that, you would see the Partition Table, a 64 byte structure. Each table entry is 16 bytes long, the first byte being the Boot Indicator field. This tells the code which partition is bootable. The Partition Boot Sector, has its own "layout" depending on the type of system. 32.8 How to get LUN ID's: ========================= # lscfg -vl hdiskx # lsattr -El hdiskx ZD110L05 600507680190014DC000000000000304 ZD110L08 600507680190014DC000000000000305 ZD111L05 600507680190014DC000000000000306 ZD111L08 600507680190014DC000000000000307
############################# 33. Filesystems in Linux: ############################# 33.1 Disks: =========== Linux on x86 systems, have the following (storage) devices: -- Entire harddisks are listed as devices without numbers, such as "/dev/hda" or "/dev/sda". - IDE: /dev/hda /dev/hdb /dev/hdc /dev/hdd - SCSI: /dev/sda etc..
is is is is
the the the the
primary IDE master drive, primary IDE slave drive, secondary IDE master, secondary IDE slave,
is the first SCSI interface and 1st device id number
-- Partitions on a disk are referred to with a number such as /dev/hda1 Floppydrive: /dev/fd0 # mount -t auto /dev/fd0 /mnt/floppy # mount -t vfat /dev/fd0 /mnt/floppy # mount /dev/fd0 /mnt/floppy Zipdrive: # insmod ppa # load the module # mount -t vfat /dev/sda /mnt/zip 33.2 Filesystems: ================= Linux supports a huge number of filesystems, including FAT, JFS, NTFS etc.. But the most common are ext2 and ext3. For the "native" filesystems, we take a look at the following FS's: - ReiserFS A journaled filesystem
- Ext2 The most popular filesystem for years. But it does not use a log/jounal, so gradually it becomes less important. - Ext3 Very related to Ext2, but this one supports journaling. An Ext2 filesystem can easily be upgraded to Ext3. 33.3 Adding a disk in Linux: ============================ Suppose you have SCSI card on with a disk is attached. The disk as a whole would be refferred to as "/dev/sda" and the first partition would be referred to as "/dev/sda1". But we have a new disk here. If you cannot find the device files /dev/sda in /dev, you might create it with the /dev/MAKEDEV script: # cd /dev # ./MAKEDEV sda The disk is now ready to be partitioned. In this example, we plan to create 3 partitions, including a swap partition. # fdisk /dev/sda The number of cylinders for this disk is set to .. (.. more output..) Command: The fdisk program is interactive; pressing m displays a list of all its commands. Command: new Command action e extended p primary partition (1-4): 1 (.. more output..) Command: print Device /dev/sda1
Boot
Start 1
End 255
Blocks 2048256
Id 83
System Linux
So we have created our first partition. We now create the swap partition: Command: new Command action e extended p primary partition (1-4): 2 (.. more output..) Command: type Partition number (1-4): 2 Hex code: 82 # which is a Linix swap partition
Changed system type of partition 2 to 82 (Linux swap) The third partition can be created in a similar way. We now would like to see a listing of our partitions Command: print Device /dev/sda1 /dev/sda2 /dev/sda3
Boot
Start 1 256 512
End 255 511 5721
Blocks 2048256 2056320 41849325
Id 83 82 83
System Linux Swap Linux
Now, save the label to the disk: Command: write (.. more output..) Ofcourse, we now would like to create the filesystems and the swap. If you want to use the Ext2 filesystem on partition one, use the following command: # mke2fs /dev/sda1 2048256
( or # mkfs -t ext2 -b 4096 /dev/sda1 )
Lets check the filesystem with fsck: # fsck -f /dev/sda1 A new filesystem can be mounted as soon as the mount point is created. # mkdir /bkroot # mount /dev/sda1 /bkroot Lets now create the swap space: # mkswap -c /dev/sda2 2056320 and activate it using the command: # swapon /dev/sda2 See also section 34.3 for administering swap space on Linux.
33.4 Notes about Linux and LVM: ============================== Note 1: ======= -What is RAID and -Initial setup of -Initial setup of -Handling a Drive -Common Glitches
LVM a RAID-5 array LVM on top of RAID Failure
-Other Useful Resources -Expanding an Array/Filesytem -------------------------------------------------------------------------------What is RAID and LVM RAID is usually defined as Redundant Array of Inexpensive disks. It is normally used to spread data among several physical hard drives with enough redundancy that should any drive fail the data will still be intact. Once created a RAID array appears to be one device which can be used pretty much like a regular partition. There are several kinds of RAID but I will only refer to the two most common here. The first is RAID-1 which is also known as mirroring. With RAID-1 it's basically done with two essentially identical drives, each with a complete set of data. The second, the one I will mostly refer to in this guide is RAID-5 which is set up using three or more drives with the data spread in a way that any one drive failing will not result in data loss. The Red Hat website has a great overview of the RAID Levels. There is one limitation with Linux Software RAID that a /boot parition can only reside on a RAID-1 array. Linux supports both several hardware RAID devices but also software RAID which allows you to use any IDE or SCSI drives as the physical devices. In all cases I'll refer to software RAID. LVM stands for Logical Volume Manager and is a way of grouping drives and/or partition in a way where instead of dealing with hard and fast physical partitions the data is managed in a virtual basis where the virtual partitions can be resized. The Red Hat website has a great overview of the Logical Volume Manager. There is one limitation that a LVM cannot be used for the /boot. ------------------------------------------------------------------------------Initial set of a RAID-5 array I recommend you experiment with setting up and managing RAID and LVM systems before using it on an important filesystem. One way I was able to do it was to take old hard drive and create a bunch of partitions on it (8 or so should be enough) and try combining them into RAID arrays. In my testing I created two RAID-5 arrays each with 3 partitions. You can then manually fail and hot remove the partitions from the array and then add them back to see how the recovery process works. You'll get a warning
about the partitions sharing a physical disc but you can ignore that since it's only for experimentation. In my case I have two systems with RAID arrays, one with two 73G SCSI drives running RAID-1 (mirroring) and my other test system is configured with three 120G IDE drives running RAID-5. In most cases I will refer to my RAID-5 configuration as that will be more typical. I have an extra IDE controller in my system to allow me to support the use of more than 4 IDE devices which caused a very odd drive assignment. The order doesn't seem to bother the Linux kernel so it doesn't bother me. My basic configuration is as follows: hda hdb hde hdf hdg
120G drive 120G drive 60G boot drive not on RAID array 120G drive CD-ROM drive
The first step is to create the physical partitions on each drive that will be part of the RAID array. In my case I want to use each 120G drive in the array in it's entirety. All the drives are partitioned identically so for example, this is how hda is partitioned: Disk /dev/hda: 120.0 GB, 120034123776 bytes 16 heads, 63 sectors/track, 232581 cylinders Units = cylinders of 1008 * 512 = 516096 bytes Device Boot /dev/hda1 * autodetect
Start 1
End 232581
Blocks 117220792+
Id fd
System Linux raid
So now with all three drives with a partitioned with id fd Linux raid autodetect you can go ahead and combine the paritions into a RAID array: # /sbin/mdadm --create --verbose /dev/md0 --level=5 --raid-devices=3 \ /dev/hdb1 /dev/hda1 /dev/hdf1 Wow, that was easy. That created a special device /dev/md0 which can be used instead of a physical parition. You can check on the status of that RAID array with the mdadm command: # /sbin/mdadm --detail /dev/md0 Version : 00.90.01 Creation Time : Wed May 11 20:00:18 2005 Raid Level : raid5 Array Size : 234436352 (223.58 GiB 240.06 GB) Device Size : 117218176 (111.79 GiB 120.03 GB) Raid Devices : 3 Total Devices : 3 Preferred Minor : 0 Persistence : Superblock is persistent Update Time : Fri Jun 10 04:13:11 2005
Active Working Failed Spare
State Devices Devices Devices Devices
: : : : :
clean 3 3 0 0
Layout : left-symmetric Chunk Size : 64K UUID : 36161bdd:a9018a79:60e0757a:e27bb7ca Events : 0.10670 Number 0 1 2
Major 3 3 33
Minor 1 65 65
RaidDevice 0 1 2
State active sync active sync active sync
/dev/hda1 /dev/hdb1 /dev/hdf1
The important lines to see are the State line which should say clean otherwise there might be a problem. At the bottom you should make sure that the State column always says active sync which says each device is actively in the array. You could potentially have a spare device that's on-hand should any drive should fail. If you have a spare you'll see it listed as such here. One thing you'll see above if you're paying attention is the fact that the size of the array is 240G but I have three 120G drives as part of the array. That's because the extra space is used as extra parity data that is needed to survive the failure of one of the drives. ------------------------------------------------------------------------------- Initial set of LVM on top of RAID Now that we have /dev/md0 device you can create a Logical Volume on top of it. Why would you want to do that? If I were to build an ext3 filesystem on top of the RAID device and someday wanted to increase it's capacity I wouldn't be able to do that without backing up the data, building a new RAID array and restoring my data. Using LVM allows me to expand (or contract) the size of the filesystem without disturbing the existing data. Anyway, here are the steps to then add this RAID array to the LVM system. The first command pvcreate will "initialize a disk or parition for use by LVM". The second command vgcreate will then create the Volume Group, in my case I called it lvm-raid: # pvcreate /dev/md0 # vgcreate lvm-raid /dev/md0 The default value large RAID array. to specify the -s size. The default
for the physical extent size can be too low for a In those cases you'll need option with a larger than default physical extent is only 4MB as of the
version in Fedora Core 5. For example, to successfully create a 550G RAID array a size of 2G works well: # vgcreate -s 2G Ok, you've created a blank receptacle but now you have to tell how many Physical Extents from the physical device (/dev/md0 in this case) will be allocated to this Volume Group. In my case I wanted all the data from /dev/md0 to be allocated to this Volume Group. If later I wanted to add additional space I would create a new RAID array and add that physical device to this Volume Group. To find out how many PEs are available to me use the vgdisplay command to find out how many are available and now I can create a Logical Volume using all (or some) of the space in the Volume Group. In my case I call the Logical Volume lvm0. # vgdisplay lvm-raid . . Free PE / Size
57235 / 223.57 GB
# lvcreate -l 57235 lvm-raid -n lvm0 In the end you will have a device you can use very much like a plain 'ol parition called /dev/lvm-raid/lvm0. You can now check on the status of the Logical Volume with the lvdisplay command. The device can then be used to to create a filesystem on. # lvdisplay /dev/lvm-raid/lvm0 --- Logical volume --LV Name /dev/lvm-raid/lvm0 VG Name lvm-raid LV UUID FFX673-dGlX-tsEL-6UXl-1hLs-6b3Y-rkO9O2 LV Write Access read/write LV Status available # open 1 LV Size 223.57 GB Current LE 57235 Segments 1 Allocation inherit Read ahead sectors 0 Block device 253:2 # mkfs.ext3 /dev/lvm-raid/lvm0 . . # mount /dev/lvm-raid/lvm0 /mnt # df -h /mnt Filesystem Size Used Avail Use% Mounted on /dev/mapper/lvm--raid-lvm0 224G 93M 224G 1% /mnt
------------------------------------------------------------------------------- Handling a Drive Failure As everything eventually does break (some sooner than others) a drive in the array will fail. It is a very good idea to run smartd on all drives in your array (and probably ALL drives period) to be notified of a failure or a pending failure as soon as possible. You can also manually fail a partition, meaning to take it out of the RAID array, with the following command: # /sbin/mdadm /dev/md0 -f /dev/hdb1 mdadm: set /dev/hdb1 faulty in /dev/md0 Once the system has determined a drive has failed or is otherwise missing (you can shut down and pull out a drive and reboot to similate a drive failure or use the command to manually fail a drive above it will show something like this in mdadm: # /sbin/mdadm --detail /dev/md0 Update Time : Wed Jun 15 11:30:59 2005 State : clean, degraded Active Devices : 2 Working Devices : 2 Failed Devices : 1 Spare Devices : 0 . . Number Major Minor RaidDevice State 0 3 1 0 active sync 1 0 0 removed 2 33 65 2 active sync
/dev/hda1 /dev/hdf1
You'll notice in this case I had /dev/hdb fail. I replaced it with a new drive with the same capacity and was able to add it back to the array. The first step is to partition the new drive just like when first creating the array. Then you can simply add the partition back to the array and watch the status as the data is rebuilt onto the newly replace drive. # /sbin/mdadm /dev/md0 -a /dev/hdb1 # /sbin/mdadm --detail /dev/md0 Update Time : Wed Jun 15 12:11:23 2005 State : clean, degraded, recovering Active Devices : 2 Working Devices : 3 Failed Devices : 0 Spare Devices : 1 Layout : left-symmetric Chunk Size : 64K Rebuild Status : 2% complete . .
During the rebuild process the system performance may be somewhat impacted but the data should remain in-tact. ------------------------------------------------------------------------------- Expanding an Array/Filesytem The answer to how to expand a RAID-5 array is very simple: You can't. I'm used to working with a NetApp Filer where you plug in a drive, type a simple command and that drive was added to the existing RAID array, no muss, no fuss. While you can't add space to a RAID-5 array directly in Linux you CAN add space to an existing Logical Volume and then expand the ext3 filesytem on top of it. That's the main reason you want to run LVM on top of RAID. Before you start it's probably a good idea to back up your data just in case something goes wrong. Assuming you want your data to be protected from a drive failing you'll need to create another RAID array per the instructions above. In my case I called it /dev/md1 so after partitioning I can create the array: # /sbin/mdadm --create --verbose /dev/md1 --level=5 --raid-devices=3 \ /dev/hde1 /dev/hdg1 /dev/hdh1 # /sbin/mdadm --detail /dev/md1 The next couple steps will add the space from the new RAID array to the space available to be used by Logical Volumes. You then check to see how many Physical Extents you have and add them to the Logical Volume you're using. Remember that since you can have multiple Logical Volumes on top of a physical RAID array you need to do this extra step. # vgextend lvm-raid /dev/md1 # vgdisplay lvm-raid . . . Alloc PE / Size 57235 / 223.57 GB Free PE / Size 57235 / 223.57 GB # lvextend -l 57235 lvm-raid -n lvm0 There, you now have a much larger Logical Volume which is using space on two separate RAID arrays. You're not done yet, you now have to extend your filesystem to make use of all that new space. Fortunately this is easy on FC4 and RHEL4 since there is a command to expand a ext3 filesytem without even unmounting it! Be patient, expanding the file system takes a while. # lvdisplay /dev/lvm-raid/lvm0 . . LV Size 447.14 GB .
# df /raid-array Filesystem 1K-blocks Used /dev/mapper/lvm--raid-lvm0 230755476 40901348 # ext2online /dev/lvm-raid1/lvm0 447g Get yourself a sandwich # df /raid-array Filesystem 1K-blocks Used /dev/mapper/lvm--raid-lvm0 461510952 40901348
Available Use% Mounted on 178132400
19% /raid-array
Available Use% Mounted on 40887876
9% /raid-array
Congrats, you now have more space. Now go fill it with something.
Note 2: ======= Creating a LVM in Linux I am sure anybody who have used windows (2000 and above) have come across the term dynamic disks. Linux/Unix also have its own dynamic disk management called LVM. What is an LVM ? LVM stands for Logical Disk Manager which is the fundamental way to manage UNIX/Linux storage systems in a scalable manner. An LVM abstracts disk devices into pools of storage space called Volume Groups. These volume groups are in turn subdivided into virtual disks called Logical Volumes. The logical volumes may be used just like regular disks with filesystem created on them and mounted in the Unix/Linux filesystem tree. The logical volumes can span multiple disks. Even though a lot of companies have implemented their own LVM's for *nixes, the one created by Open Software Foundation (OSF) was integrated into many Unix systems which serves as a base for the Linux implementation of LVM. Note: Sun Solaris ships with LVM from Veritas which is substantially different from the OSF implementation. Benefits of Logical Volume Management LVM created in conjunction with RAID can provide fault tolerance coupled with scalability and easy disk management. Create a logical volume and filesystem which spans multiple disks. By creating virtual pools of space, an administrator can create dozens of small filesystems for different projects and add space to them as needed without (much) disruption. When a project ends, he can remove the space a nd put it back into the pool of free space.
Note : Before you move to implement LVM's in linux, make sure your kernel is 2.4 and above. Or else you will have to recompile your kernel from source to include support for LVM. LVM Creation To create a LVM, we follow a three step process. Step One : We need to select the physical storage resources that are going to be used for LVM. Typically, these are standard partitions but can also be Linux software RAID volumes that we've created. In LVM terminology, these storage resources are called "physical volumes" (eg: /dev/hda1, /dev/hda2 ... etc). Our first step in setting up LVM involves properly initializing these partitions so that they can be recognized by the LVM system. This involves setting the correct partition type (usually using the fdisk command, and entering the type of partition as 'Linux LVM' - 0x8e ) if we're adding a physical partition; and then running the pvcreate command. # pvcreate /dev/hda1 /dev/hda2 /dev/hda3 # pvscan The above step creates a physical volume from 3 partitions which I want to initialize for inclusion in a volume group. Step Two : Creating a volume group. You can think of a volume group as a pool of storage that consists of one or more physical volumes. While LVM is running, we can add physical volumes to the volume group or even remove them. First initialize the /etc/lvmtab and /etc/lvmtab.d files by running the following command: # vgscan Now you can create a volume group and assign one or more physical volumes to the volume group. # vgcreate my_vol_grp /dev/hda1 /dev/hda2 Behind the scenes, the LVM system allocates storage in equal-sized "chunks", called extents. We can specify the particular extent size to use at volume group creation time. The size of an extent defaults to 4Mb, which is perfect for most uses.You can use the -s flag to change the size of the extent. The extent affects the minimum size of changes which can be made to a logical volume in the volume group, and the maximum size of logical and physical volumes in the volume group. A logical volume can contain at most 65534 extents, so the default extent size (4 MB) limits the volume to about 256 GB; a size of 1 TB would require
extents of atleast 16 MB. So to accomodate a 1 TB size, the above command can be rewriten as : # vgcreate -s 16M my_vol_grp /dev/hda1 /dev/hda2 You can check the result of your work at this stage by entering the command: # vgdisplay This command displays the total physical extends in a volume group, size of each extent, the allocated size and so on. Step Three : This step involves the creation of one or more "logical volumes" using our volume group storage pool. The logical volumes are created from volume groups, and may have arbitary names. The size of the new volume may be requested in either extents (-l switch) or in KB, MB, GB or TB ( -L switch) rounding up to whole extents. # lvcreate -l 50 -n my_logical_vol my_vol_grp The above command allocates 50 extents of space in my_vol_grp to the newly created my_logical_vol. The -n switch specifies the name of the logical volume we are creating. Now you can check if you got the desired results by using the command : # lvdisplay which shows the information of your newly created logical volume. Once a logical volume is created, we can go ahead and put a filesystem on it, mount it, and start using the volume to store our files. For creating a filesystem, we do the following: # mke2fs -j /dev/my_vol_grp/my_logical_vol The -j signifies journaling support for the ext3 filesystem we are creating. Mount the newly created file system : # mount /dev/my_vol_grp/my_logical_vol /data Also do not forget to append the corresponding line in the /etc/fstab file: #File: /etc/fstab /dev/my_vol_grp/my_logical_vol /data ext3 defaults 0 0 Now you can start using the newly created logical volume accessable at /data mount point. Next : Resizing Logical Volumes Some more on Linux LVM commands:
Linux vgcreate command: ======================= Linux / Unix Command: vgcreate Command Library NAME vgcreate - create a volume group SYNOPSIS vgcreate [-A|--autobackup {y|n}] [-d|--debug] [-h|--help] [-l|-maxlogicalvolumes MaxLogicalVolumes] [-p|--maxphysicalvolumes MaxPhysicalVolumes] [-s|--physicalextentsize PhysicalExtentSize[kKmMgGtT]] [-v|--verbose] [--version] VolumeGroupName PhysicalVolumePath [PhysicalVolumePath...] DESCRIPTION vgcreate creates a new volume group called VolumeGroupName using the block special device PhysicalVolumePath previously configured for LVM with pvcreate(8). OPTIONS -A, --autobackup {y|n} Controls automatic backup of VG metadata after the change (see vgcfgbackup(8)). Default is yes. -d, --debug Enables additional debugging output (if compiled with DEBUG). -h, --help Print a usage message on standard output and exit successfully. -l, --maxlogicalvolumes MaxLogicalVolumes Sets the maximum possible logical volume count. More logical volumes can't be created in this volume group. Absolute maximum is 256. -p, --maxphysicalvolumes MaxPhysicalVolumes Sets the maximum possible physical volume count. More physical volumes can't be included in this volume group. Absolute maximum is 256. -s, --physicalextentsize PhysicalExtentSize[kKmMgGtT] Sets the physical extent size on physical volumes of this volume group. A size suffix (k for kilobytes up to t for terabytes) is optional, megabytes is the default if no suffix is present. Values can be from 8 KB to 16 GB in powers of 2. The default of 4 MB causes maximum LV sizes of ~256GB because as many as ~64k extents are supported per LV. In case larger maximum LV sizes are needed (later), you need to set the PE size to a larger value as well. Later changes of the PE size in an existing VG are not supported. -v, --verbose Display verbose runtime information about vgcreate's activities. --version Display tool and IOP version and exit successfully. EXAMPLES
To create a volume group named test_vg using physical volumes /dev/hdk1, /dev/hdl1, and /dev/hdm1 with default physical extent size of 4MB: # vgcreate test_vg /dev/sd[k-m]1 To create a volume group named test_vg using physical volumes /dev/hdk1, and /dev/hdl1 with default physical extent size of 4MB: # vgcreate test_vg /dev/sdk1 /dev/sdl1 NOTE: If you are using devfs it is essential to use the full devfs name of the device rather than the symlinked name in /dev. so: the above could be # vgcreate test_vg /dev/scsi/host1/bus0/target[1-3]/lun0/part1
Linux vgextend command: ======================= Linux / Unix Command: vgextend Command Library NAME vgextend - add physical volumes to a volume group SYNOPSIS vgextend [-A|--autobackup{y|n}] [-d|--debug] [-h|--help] [-v|--verbose] VolumeGroupName PhysicalVolumePath [PhysicalVolumePath...] DESCRIPTION vgextend allows you to add one or more initialized physical volumes ( see pvcreate(8) ) to an existing volume group to extend it in size. OPTIONS -A, --autobackup y/n Controls automatic backup of VG metadata after the change ( see vgcfgbackup(8) ). Default is yes. -d, --debug Enables additional debugging output (if compiled with DEBUG). -h, --help Print a usage message on standard output and exit successfully. -v, --verbose Gives verbose runtime information about lvextend's activities. Examples # vgextend vg00 /dev/sda4 /dev/sdn1
tries to extend the existing volume group "vg00" by the new physical volumes (see pvcreate(8) ) "/dev/sdn1" and /dev/sda4". Linux pvcreate command: ======================= Linux / Unix Command: pvcreate Command Library NAME pvcreate - initialize a disk or partition for use by LVM SYNOPSIS pvcreate [-d|--debug] [-f[f]|--force [--force]] [-y|--yes] [-h|--help] [-v|--verbose] [-V|--version] PhysicalVolume [PhysicalVolume...] DESCRIPTION pvcreate initializes PhysicalVolume for later use by the Logical Volume Manager (LVM). Each PhysicalVolume can be a disk partition, whole disk, meta device, or loopback file. For DOS disk partitions, the partition id must be set to 0x8e using fdisk(8), cfdisk(8), or a equivalent. For whole disk devices only the partition table must be erased, which will effectively destroy all data on that disk. This can be done by zeroing the first sector with: # dd if=/dev/zero of=PhysicalVolume bs=512 count=1 Continue with vgcreate(8) to create a new volume group on PhysicalVolume, or vgextend(8) to add PhysicalVolume to an existing volume group. OPTIONS -d, --debug Enables additional debugging output (if compiled with DEBUG). -f, --force Force the creation without any confirmation. You can not recreate (reinitialize) a physical volume belonging to an existing volume group. In an emergency you can override this behaviour with -ff. In no case case can you initialize an active physical volume with this command. -s, --size Overrides the size of the physical volume which is normally retrieved. Useful in rare case where this value is wrong. More useful to fake large physical volumes of up to 2 Terabyes - 1 Kilobyte on smaller devices for testing purposes only where no real access to data in created logical volumes is needed. If you wish to create the supported maximum, use "pvcreate -s 2147483647k PhysicalVolume [PhysicalVolume ...]". All other LVM tools will use this size with the exception of lvmdiskscan(8)
-y, --yes Answer yes to all questions. -h, --help Print a usage message on standard output and exit successfully. -v, --verbose Gives verbose runtime information about pvcreate's activities. -V, --version Print the version number on standard output and exit successfully. Example Initialize partition #4 on the third SCSI disk and the entire fifth SCSI disk for later use by LVM: # pvcreate /dev/sdc4 /dev/sde
33.5 Installing a Cluster filesystem on Linux: ============================================== Suppose, in this example, we have 2 Linux nodes, and we want to create a scsi attached shared disksystem. We plan to use OCFS2 as the Clustered FileSystem. First, we partition the disks to raw volumes. This example uses /dev/sdb (an empty SCSI disk with no existing partitions) to create a single partition for the entire disk (36 GB). We will do this for all disks. Ex: # fdisk /dev/sdb Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. The number of cylinders for this disk is set to 4427. There is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with: 1) software that runs at boot time (e.g., old versions of LILO) 2) booting and partitioning software from other OSs (e.g., DOS FDISK, OS/2 FDISK) Command (m for help): p Disk /dev/sdb: 255 heads, 63 sectors, 4427 cylinders Units = cylinders of 16065 * 512 bytes Device Boot Start End Blocks Id System Command (m for help): n
Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-4427, default 1): Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-4427, default 4427): Using default value 4427 Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. WARNING: If you have created or modified any DOS 6.x partitions, please see the fdisk manual page for additional information. Syncing disks. Now verify the new partition: Ex: # fdisk -l /dev/sdb Disk /dev/sdb: 36.4 GB, 36420075008 bytes 255 heads, 63 sectors/track, 4427 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Device Boot /dev/sdb1 *
Start 1
End 4427
Blocks 35559846
Id 83
System Linux
Repeat the above steps for each disk to be partitioned. Disk partitioning should be done from one node only. When finished partitioning, run the 'partprobe' command as root on each of the remaining cluster nodes in order to assure that the new partitions are configured. Ex: # partprobe Oracle Cluster File System (OCFS) Release 2 ------------------------------------------OCFS2 is a general-purpose cluster file system that can be used to store Oracle Clusterware files, Oracle RAC database files, Oracle software, or any other types of files normally stored on a standard filesystem such as ext3. This is a significant change from OCFS Release 1, which only supported Oracle Clusterware files and Oracle RAC database files. Obtain OCFS2 OCFS2 is available free of charge from Oracle as a set of three RPMs: a kernel module, support tools, and a console.
There are different kernel module RPMs for each supported Linux kernel so be sure to get the OCFS2 kernel module for your Linux kernel. OCFS2 kernel modules may be downloaded from http://oss.oracle.com/projects/ocfs2/files/ and the tools and console may be downloaded from http://oss.oracle.com/projects/ocfs2-tools/files/. To determine the kernel-specific module that you need, use uname -r. # uname -r 2.6.9-22.ELsmp For this example I downloaded: ocfs2console-1.0.3-1.i386.rpm ocfs2-tools-1.0.3-1.i386.rpm ocfs2-2.6.9-22.ELsmp-1.0.7-1.i686.rpm >>> Install OCFS2 as root on each cluster node # rpm -ivh ocfs2console-1.0.3-1.i386.rpm \ ocfs2-tools-1.0.3-1.i386.rpm \ ocfs2-2.6.9-22.ELsmp-1.0.7-1.i686.rpm Preparing... [100%] 1:ocfs2-tools [ 33%] 2:ocfs2console [ 67%] 3:ocfs2-2.6.9-22.ELsmp [100%] Configure OCFS2
########################################### ########################################### ########################################### ###########################################
Run ocfs2console as root: # ocfs2console Now a Graphical interface will appear: Select Cluster ? Configure Nodes Click on Add and enter the Name and IP Address of each node in the cluster Once all of the nodes have been added, click on Cluster --> Propagate Configuration. This will copy the OCFS2 configuration file to each node in the cluster. You may be prompted for root passwords as ocfs2console uses ssh to propagate the configuration file. Leave the OCFS2 console by clicking on File --> Quit. It is possible to format and mount the OCFS2 partitions using the ocfs2console GUI; however, this guide will use the command line utilities. >>> Enable OCFS2 to start at system boot: As root, execute the following command on each cluster node to allow the OCFS2 cluster stack to load at boot time:
/etc/init.d/o2cb enable Ex: # /etc/init.d/o2cb enable Writing O2CB configuration: OK Loading module "configfs": OK Mounting configfs filesystem at /config: OK Loading module "ocfs2_nodemanager": OK Loading module "ocfs2_dlm": OK Loading module "ocfs2_dlmfs": OK Mounting ocfs2_dlmfs filesystem at /dlm: OK Starting cluster ocfs2: OK >>> Create a mount point for the OCFS filesystem As root on each of the cluster nodes, create the mount point directory for the OCFS2 filesystem Ex: # mkdir /u03
>>> Create the OCFS2 filesystem on the unused disk partition: The example below creates an OCFS2 filesystem on the partition with a volume label of "/u03" (-L /u03), a (-b 4K) and a cluster size of 32K (-C 32K) with 4 node slots OCFS2 Users Guide for more information on mkfs.ocfs2 options.
unused /dev/sdc1 block size of 4K (-N 4). See the command line
Ex: # mkfs.ocfs2 -b 4K -C 32K -N 4 -L /u03 /dev/sdc1 mkfs.ocfs2 1.0.3 Filesystem label=/u03 Block size=4096 (bits=12) Cluster size=32768 (bits=15) Volume size=36413280256 (1111245 clusters) (8889960 blocks) 35 cluster groups (tail covers 14541 clusters, rest cover 32256 clusters) Journal size=33554432 Initial number of node slots: 4 Creating bitmaps: done Initializing superblock: done Writing system files: done Writing superblock: done Writing lost+found: done mkfs.ocfs2 successful >>> Mount the OCFS2 filesystem:
Since this filesystem will contain the Oracle Clusterware files and Oracle RAC database files, we must ensure that all I/O to these files uses direct I/O (O_DIRECT). Use the "datavolume" option whenever mounting the OCFS2 filesystem to enable direct I/O. Failure to do this can lead to data loss in the event of system failure. Ex: # mount -t ocfs2 -L /u03 -o datavolume /u03 Notice that the mount command uses the filesystem label (-L u03) used during the creation of the filesystem. This is a handy way to refer to the filesystem without having to remember the device name. To verify that the OCFS2 filesystem is mounted, issue the mount command or run df: # mount -t ocfs2 /dev/sdc1 on /u03 type ocfs2 (rw,_netdev,datavolume) # df /u03 Filesystem /dev/sdc1
1K-blocks 35559840
Used Available Use% Mounted on 138432 35421408 1% /u03
The OCFS2 filesystem can now be mounted on the other cluster nodes. To automatically mount the OCFS2 filesystem at system boot, add a line similar to the one below to /etc/fstab on each cluster node: LABEL=/u03 /u03 ocfs2 _netdev,datavolume,nointr 0 0 Create the directories for shared files CRS files mkdir /u03/oracrs chown oracle:oinstall /u03/oracrs chmod 775 /u03/oracrs Database files mkdir /u03/oradata chown oracle:oinstall /u03/oradata chmod 775 /u03/oradata
34. SWAP space: =============== 34.1 Solaris: -------------- View swap space: -- ---------------The /usr/sbib/swap utility provides a method of adding, deleting, and monitoring the system swap areas used by the memory manager.
# swap -l The -l option can be used to list swap space. The system displays information like: swapfile dev swaplo blocks free /dev/dsk/c0t0d0s3 136,3 16 302384 302384 path : the pathname for the swaparea. In this example the pathname is swapfile. dev : the major/minor device number is in decimal if it's a block special device; zeroes otherwise swaplo: the offset in 512 byte blocks where usable swapspace begins blocks: size in 512 byte blocks. The swaplen value can be adjusted as a kernel parameter. free : free 512 byte blocks. The swap -l command does not include physical memory in it's calculation of swap space. # swap -s The -s option can be used to list a summary of the system's virtual swap space. total: 31760k bytes allocated + 5952k reserved = 37712k used, 202928k available These numbers are in 1024 byte blocks. -- Add swap area's: -- ---------------There are 2 methods available for adding more swap to your system. (1) create a secondary swap partition: (2) create a swapfile in an existing UFS file system (1) Creating a secondary swap partition requires additional unused diskspace. You must use the format coommand to create a new partition and filesystem on a disk. Suppose we have the /data directory currently on slice 5 and is 200MB in size. - free up the /data directory (save the contents to another location ) - unmount /dev/dsk/c0t0d0s5 - use format: Enter partition id tag (unassigned): swap Enter partition permission flags (wm): wu Enter new starting cil(3400): return Enter partition size: return Then label the disk as follows Partition> la Ready to label disk? y - Run the newfs command on that partition to create a fresh filesystem on slice 5 newfs /dev/rdsk/c0t0d0s5 - Make an entry to the /etc/vfstab file - Run the swapadd script to add the swap to your system as follows:
/sbin/swapadd - verify that the swap has been added with swap -l (2) The other method to add more swap space is to use the mkfile and swap commands to designate a part of an existing UFS filesystem as a supplementary swap area. You can use it as a temporary solution, or as a solution for longer duration as well, but a swap file is just another file in the filesystem, so you cannot unmount that filesystem while the swapfile is in use. The following steps enable you to add more swap space without repartitioning a disk. - As root, use df -k to locate a suitable filesystem. Suppose /data looks allright for this purpose - Use the mkfile command to add a 50MB swapfile named swapfile in the /data partition. mkfile 50m /data/swapfile - use ls -l /data to verify that the file has been created. Notice that the sticky bit has automatically been set. - Activate the swaparea with the swap command as follows: /usr/sbin/swap -a /data/swapfile - verify that the swap has been added with swap -l The system responds something like this: swapfile /dev/dsk/c0t0d0s3 /data/swapfile
dev 136,3 -
swaplo 16 16
blocks 302384 102384
free 302384 102384
If this will be a permanent swaparea, add an entry for the swapfile in the vfstab file. /data/swapfile - - swap - no -- Removing a swapfile: -- -------------------As root use the swap -d command to remove a swaparea is follows swap -d /dev/dsk/c0t0d0s5 swap -d /data/swapfile
for a swap partition for a swapfile
Use the swap -l command to verify that the swaparea is gone. Edit the /etc/vfstab file and delete the entry for the swapfile if neccessary. In case of a swapfile, just remove the file with rm /data/swapfile -- Creating a Temporary File System: -- ---------------------------------
Create a directory which will serve as the mount point for the TMPFS file system. There is no command such as newfs to create a TMPFS file system before mounting it. The TMPFS file system actually gets created in RAM when you execute the mount command and specify a filesystem type of TMPFS. The following example creates a new directory /export/data and mounts a TMPFS filesystem, limiting it to 25MB. mount -F tmpfs -o size=25m swap /export/data
34.2 AIX: --------The installation creates a default paging logical volume, hd6, on drive hdisk0, also referred as primary paging space. The reports from the "vmstat" and "topas" commands indicate the amount of paging space I/O that is taking place. Showing paging space: --------------------The lsps -a command provides a snapshot of the current utilization of each of the paging spaces on the system, while the lsps -s command provides a summary of the total active paging space and its current utilization. # lsps -a Page Space Physical Volume Active Auto Type paging00 hdisk1 yes lv hd6 hdisk1 yes lv
Volume Group
Size
%Used
rootvg
80MB
1
yes
rootvg
256MB
1
yes
The /etc/swapspaces file specifies the paging-space devices that are activated by the swapon -a command. A pagingspace is added to this file when its created by the mkps -a command, and removed from the file when rmps is used. You can also try: # pstat -s Managing Paging space: ---------------------The following commands are used to manage paging space:
chps lsps pstat -s mkps rmps swapon swapoff
: : : : : : :
changes the attributes of a paging space displays the characteristics of a paging space displays the characteristics of a paging space creates an additional paging space removes an inactive paging space activates a paging space deactivates one or more paging spaces
Managing Paging behaviour: -------------------------Note 1: ------There are several page space allocation policies available in AIX. -
Deferred Page Space Allocation (DPSA) Late Page Space Allocation (LPSA) Early Page Space Allocation (EPSA) Deferred page space allocation
The deferred page space allocation policy is the default policy in AIX. Late page space allocation LPSA The AIX operating system provides a way to enable the late page space allocation policy, which means that the disk block for a paging space page is only allocated when the corresponding inmemory page is touched. Early page space allocation EPSA If you want to ensure that a process will not be killed due to low paging conditions, this process can preallocate paging space by using the early page space allocation policy. Choosing between LPSA and DPSA with the vmo command: Using the "vmo -o defps" command enables turning the deferred page space allocation, or DPSA, on or off in order to preserve the late page space allocation policy, or LPSA. Paging space and virtual memory The vmstat command (avm column), ps command (SIZE, SZ), and other utilities report the amount of virtual memory actually accessed because with DPSA, the paging space might not get touched. Note 2: ------High paging space during online backup on AIX Technote (FAQ)
Question During an online backup, you might see a high paging space usage, which will not be released even after online backup completion in DB2® Universal Database™ (DB2 UDB) Version 8. This problem does not occur during an offline backup. Cause Paging space usage increases during online database backups on AIX® 5.2 and 5.3. This is an expected behavior from ML4 of AIX 5L™ 5.2 and ML1 of AIX 5L 5.3 onwards. During an online database backup operation, file pages are loaded into memory by AIX in order for the backup processes to read them. If DB2 UDB runs out of memory, AIX has to free memory to fit additional file pages into RAM. It does this by writing DB2 UDB shared memory segments out to paging space. When the backup completes, these pages in paging space are not released because they are still in use by the other DB2 UDB processes. They will only be freed when the database is deactivated. Answer To free up paging space without stopping the database, use the AIX tuning parameter lru_file_repage. It affects Virtual Memory Manager (VMM) page replacement. By setting this parameter to 0, you force the system to only free file pages when you run out of memory and to not write working pages out to paging space. This will stop paging use from increasing. To set this parameter to zero, use vmo command. For example: vmo -o lru_file_repage=0 This parameter was introduced in ML4 of AIX5L 5.2 and ML1 of AIX5L 5.3. The default value is 1. Note 3: ------Warning: this is a trick. trick I have found to "reset" paging is to increase page space by 1 PP and then decrease it by 1 PP. Decreasing the size of paging space causes the S to create a new page space, copy everything to the new space, delete the old recreate it at the new size. You'll need enough free disk space to create a new page space. Note 4:
------The VM kernel parameters minperm% and maxperm% affect the use of physical memory that can be used for file system caching and govern when computational pages of memory get paged (swapped) to paging space. If these values have been changed recently, that could explain the results that you describe. When the (dynamic) value of numperm% drops below minperm%, it will cause the paging of computational pages to page space. It would be interesting to know if the minperm% and maxperm% values were changed and, if so, what the former and current values are. Note 5: -------
Show paging space usage: # lsps -a # lsps -s Increase paging space: # chps -s 32 hd6
32x32MB
where we increased the size of hd6 with 30 LP's. Reducing paging space: # chps -d 1 hd6
where we decreased the size of hd6 with 1 LP. mkps: ----To Add a Logical Volume for Additional Paging Space mkps [ -a ] [ -n ] [ -t lv ] -s LogicalPartitions VolumeGroup [ PhysicalVolume ] To create a paging space in volume group myvg that has four logical partitions and is activated immediately and at all subsequent system restarts, enter: # mkps
-a
-n
-s 4 myvg
To create a paging space in rootvg on hdisk0
# mkps -a -n -s 30 rootvg hdisk0 rmps: ----Before AIX 5L: Active paging spaces cannot be removed. It must first be made inactive. Use the chps command so the paging space is not used on the next restart. After reboot, the paging space is inactive and can be removed with the rmps command. AIX 51 or later: Use the swapoff command to dynamically deactive the paging space, then use the rmps command. # swapoff /dev/paging03 # rmps paging03 chps: ----As from AIX 5L you can use the chps -d command, to decrease the size of a paging space, without having to deactive it, then reboot, then remove, and then recreate it with a smaller size. Decrease it with a number of LP's like: # chps -d 2 paging03 chps -a {y|n} paging00 : specifies that the paging space paging00 is active (y) or inactive (n) at subsequent system restarts. chps -s 10 paging02 : adds ten LPs to paging02 without rebooting. chps -d 5 paging01 : removes five LPs from paging01 without rebooting. chps -d 50 hd6 : removes fifty LPs from hd6 without rebooting. List the active paging spaces: -----------------------------# lsps -a
or lsps -s
# pg /etc/swapspaces hd6: dev=/dev/hd6 paging00 dev=/dev/paging00
Note on paging on AIX: ---------------------If the amount of paging space is less than the amount of real memory in the system, it's possible the system will run out of paging space before real memory. This is because AIX performs early allocation of page space.
When a page is referenced, real memory and paging space blocks are allocated. If there are less paging space blocks then real memory pages, paging space will be exhaused before all of real memory is consumed. Early allocation algorithm The second operating system's paging-space-slot-allocation method is intended for use in installations where this situation is likely, or where the cost of failure to complete is intolerably high. Aptly called early allocation, this algorithm causes the appropriate number of paging-space slots to be allocated at the time the virtual-memory address range is allocated, for example, with the malloc() subroutine. If there are not enough paging-space slots to support the malloc() subroutine, an error code is set. The early-allocation algorithm is invoked as follows: # export PSALLOC=early This example causes all future programs to be executed in the environment to use early allocation. The currently executing shell is not affected. Early allocation is of interest to the performance analyst mainly because of its paging-space size implications. If early allocation is turned on for those programs, paging-space requirements can increase many times. Whereas the normal recommendation for paging-space size is at least twice the size of the system's real memory, the recommendation for systems that use PSALLOC=early is at least four times the real memory size. Actually, this is just a starting point. Analyze the virtual storage requirements of your workload and allocate paging spaces to accommodate them. As an example, at one time, the AIXwindows server required 250 MB of paging space when run with early allocation. When using PSALLOC=early, the user should set a handler for the following SIGSEGV signal by pre-allocating and setting the memory as a stack using the sigaltstack function. Even though PSALLOC=early is specified, when there is not enough paging space and a program attempts to expand the stack, the program may receive the SIGSEGV signal. Deferred allocation algorithm The third operating system's paging-space-slot-allocation method is the default beginning with AIX 4.3.2 Deferred Page Space Allocation (DPSA) policy delays allocation of paging space until it is necessary to page out the page, which results in no wasted paging space allocation. This method can save huge amounts of paging space, which means disk space. Best to use Deffered. On some systems, paging space might not ever be needed even if all the pages accessed have been touched. This situation is most common on systems with very large amount of RAM. However, this may result in overcommitment
of paging space in cases where more virtual memory than available RAM is accessed. To disable DPSA and preserve the Late Page Space Allocation policy, run the following command: # vmo -o defps=0 To activate DPSA, run the following command: # vmo -o defps=1 In general, system performance can be improved by DPSA, because the overhead of allocating page space after page faults is avoided the. Paging space devices need less disk space if DPSA is used
34.3 Linux: ------------ Check the swapspace: # cat /proc/meminfo # cat /proc/swaps # /sbin/swapon -s -- Creating swap space using a partition Create a partition of the proper size using fdisk. Format the partition, for example # mkswap -c /dev/hda4 Enable the swap, for example # swapon /dev/hd4 If you want the swap space enabled after boot, include the appropriate entry into /etc/fstab, for example /dev/hda4 swap swap defaults 0 0 If you need to disable the swap, you can do it with # swapoff /dev/hda4 -- Creating swap space using a swapfile Create a file with the size of your swapfile # dd if=/dev/zero of=/swapfile bs=1024 count=8192 Setup the file with the command # mkswap /swapfile 8192
Enable the swap with the command # swapon /swapfile When you are done using the swapfile, you can turn it off and remove with # swapoff /swapfile # rm /swapfile 34.4: Note about swap: ---------------------Page replacement in Linux 2.4 memory management Rik van Riel Conectiva Inc. [email protected] , http://www.surriel.com/ Abstract While the virtual memory management in Linux 2.2 has decent performance for many workloads, it suffers from a number of problems. The first part of this paper contains a description of how the Linux 2.2 VMM works and an analysis of why it has bad behaviour in some situations. The way in which a lot of this behaviour has been fixed in the Linux 2.4 kernel is described in the second part of the paper. Due to Linux 2.4 being in a code freeze period while these improvements were implemented, only known-good solutions have been integrated. A lot of the ideas used are derived from principles used in other operating systems, mostly because we have certainty that they work and a good understanding of why, making them suitable for integration into the Linux codebase during a code freeze. --Linux 2.2 memory management The memory management in the Linux 2.2 kernel seems to be focussed on simplicity and low overhead. While this works pretty well in practice for most systems, it has some weak points left and simply falls apart under some scenarios. Memory in Linux is unified, that is all the physical memory is on the same free list and can be allocated to any of the following memory pools on demand. Most of these pools can grow and shrink on demand. Typically most of a system's memory will be allocated to the data pages of processes and the page and buffer caches. The slab cache: this is the kernel's dynamically allocated heap storage. This memory is unswappable, but once all objects within one (usually page-sized) area are unused, that area can be reclaimed. The page cache: this cache is used to cache file data for both mmap() and read() and is indexed by (inode, index) pairs.
No dirty data exists in this cache; whenever a program writes to a page, the dirty data is copied to the buffer cache, from where the data is written back to disk. The buffer cache: this cache is indexed by (block device, block number) tuples and is used to cache raw disk devices, inodes, directories and other filesystem metadata. It is also used to perform disk IO on behalf of the page cache and the other caches. For disk reads the pagecache bypasses this cache and for network filesystems it isn't used at all. The inode cache: this cache resides in the slab cache and contains information about cached files in the system. Linux 2.2 cannot shrink this cache, but because of its limited size it does need to reclaim individual entries. The dentry cache: this cache contains directory and name information in a filesystem-independent way and is used to lookup files and directories. This cache is dynamically grown and shrunk on demand. SYSV shared memory: the memory pool containing the SYSV shared memory segments is managed pretty much like the page cache, but has its own infrastructure for doing things. Process mapped virtual memory: this memory is administrated in the process page tables. Processes can have page cache or SYSV shared memory segments mapped, in which case those pages are managed in both the page tables and the data structures used for respectively the page cache or the shared memory code. --Linux 2.2 page replacement The page replacement of Linux 2.2 works as follows. When free memory drops below a certain threshold, the pageout daemon (kswapd) is woken up. The pageout daemon should usually be able to keep enough free memory, but if it isn't, user programs will end up calling the pageout code itself. The main pageout loop is in the function try_to_free_pages, which starts by freeing unused slabs from the kernel memory pool. After that, it calls the following functions in a loop, asking each of them to scan a small part of their part of memory until enough memory has been freed. shrink_mmap is a classical clock algorithm, which loops over all physical pages, clearing referenced bits, queueing old dirty pages pages for IO and freeing old clean pages. The main disadvantage it has compared to a clock algorithm, however, is that it isn't able to free pages which are in use by a program or a shared memory segment. Those pages need to be unmapped by swap_out first. shm_swap scans the SYSV shared memory segments, swapping out those pages that haven't been referenced recently
and which aren't mapped into any process. swap_out scans the virtual memory of all processes in the system, unmapping pages which haven't been referenced recently, starting swapout IO and placing those pages in the page cache. shrink_dcache_memory recaims entries from the VFS name cache. This is not directly reusable memory, but as soon as a whole page of these entries gets unused we can reclaim that page. Some balancing between these memory freeing function is achieved by calling them in a loop, starting of by asking each of these functions to scan a little bit of their memory, as each of these funnctions accepts a priority argument which tells them how big a percentage of their memory to scan. If not enough memory is freed in the first loop, the priority is increased and the functions are called again. The idea behind this scheme is that when one memory pool is heavily used, it will not give up its resources lightly and we'll automatically fall through to one of the other memory pools. However, this scheme relies on each of the memory pools to react in a similar way to the priority argument under different load conditions. This doesn't work out in practice because the memory pools just have fundamentally different properties to begin with. --Problems with the Linux 2.2 page replacement Balancing between evicting pages from the file cache, evicting unused process pages and evicting pages from shm segments. If memory pressure is "just right" shrink_mmap is always successful in freeing cache pages and a process which has been idle for a day is still in memory. This can even happen on a system with a fairly busy filesystem cache, but only with the right phase of moon. Simple NRU[Note] replacement cannot accurately identify the working set versus incidentally accessed pages and can lead to extra page faults. This doesn't hurt noticably for most workloads, but it makes a big difference in some workloads and can be fixed easily, mostly since the LFU replacement used in older Linux kernels is known to work. Due to the simple clock algorithm in shrink_mmap, sometimes clean, accessed pages can get evicted before dirty, old pages. With a relatively small file cache that mostly consists of dirty data, eg unpacking a tarball, it is possible for the dirty pages to evict the (clean) metadata buffers that are needed to write the dirty data to disk. A few other corner cases with amusing variations on this theme are bound to exist. The system reacts badly to variable VM load or to load spikes after a period of no VM activity. Since kswapd, the pageout daemon, only scans when the system is low on memory, the system can end up in a state where some pages have referenced bits from the last 5 seconds, while other pages have referenced bits from 20 minutes ago. This means that on a load spike the system has no clue which are the right pages to evict from memory, this can lead to a swapping storm, where the wrong pages are evicted and almost immediately afterwards faulted back in, leading to the pageout of another random page, etc...
Under very heavy loads, NRU replacement of pages simply doesn't cut it. More careful and better balanced pageout eviction and flushing is called for. With the fragility of the Linux 2.2 pageout framework this goal doesn't really seem achievable. The facts that shrink_mmap is a simple clock algorithm and relies on other functions to make process-mapped pages freeable makes it fairly unpredictable. Add to that the balancing loop in try_to_free_pages and you get a VM subsystem which is extremely sensitive to minute changes in the code and a fragile beast at its best when it comes to maintenance or (shudder) tweaking. --Changes in Linux 2.4 For Linux 2.4 a substantial development effort has gone into things like making the VM subsystem fully fine-grained for SMP systems and supporting machines with more than 1GB of RAM. Changes to the pageout code were done only in the last phase of development and are, because of that, somewhat conservative in nature and only employ known-good methods to deal with the problems that happened in the page replacement of the Linux 2.2 kernel. Before we get to the page replacement changes, however, first a short overview of the other changes in the 2.4 VM: More fine-grained SMP locking. The scalability of the VM subsystem has improved a lot for workloads where multiple CPUs are reading or writing the same file simultaneously; for example web or ftp server workloads. This has no real influence on the page replacement code. Unification of the buffer cache and the page cache. While in Linux 2.2 the page cache used the buffer cache to write back its data, needing an extra copy of the data and doubling memory requirements for some write loads, in Linux 2.4 dirty page cache pages are simply added in both the buffer and the page cache. The system does disk IO directly to and from the page cache page. That the buffer cache is still maintained separately for filesystem metadata and the caching of raw block devices. Note that the cache was already unified for reads in Linux 2.2, Linux 2.4 just completes the unification. Support for systems with up to 64GB of RAM (on x86). The Linux kernel previously had all physical memory directly mapped in the kernel's virtual address space, which limited the amount of supported memory to slightly under 1GB. For Linux 2.4 the kernel also supports additional memory (so called "high memory" or highmem), which can not be used for kernel data structures but only for page cache and user process memory. To do IO on these pages they are temporarily mapped into kernel virtual memory and the data is copied to or from a bounce buffer in "low memory". At the same time the memory zone for ISA DMA (0 - 16 MB physical address range) has also been split out into a separate page zone. This means larger x86 systems end up with 3 memory zones, which all need their free memory balanced so we can continue allocating kernel data structures and ISA DMA buffers. The memory zones logic is generalised enough to also work for NUMA systems. The SYSV shared memory code has been removed and replaced with a simple memory filesystem which uses the page cache for all its functions. It
supports both POSIX SHM and SYSV SHM semantics and can also be used as a swappable memory filesystem (tmpfs). Since the changes to the page replacement code took place after all these changes and in the (one and a half year long) code freeze period of the Linux 2.4 kernel, the changes have been kept fairly conservative. On the other hand, we have tried to fix as many of the Linux 2.2 page replacement problems as possible. Here is a short overview of the page replacement changes: they'll be described in more detail below. Page aging, which was present in the Linux 1.2 and 2.0 kernels and in FreeBSD has been reintroduced into the VM. However, a few small changes have been made to avoid some artifacts of virtual page based aging. To avoid the eviction of "wrong" pages due to interactions from page aging and page flushing, the page aging and flushing has been separated. There are active and inactive page lists. Page flushing has been optimised to avoid too much interference by writeout IO on the more time-critical disk read IO. Controlled background page aging during periods of little or no VM activity in order to keep the system in a state where it can easily deal with load spikes. Streaming IO is detected; we do early eviction on the pages that have already been used and reward the IO stream with more agressive readahead. --Linux 2.4 page replacement changes in detail The development of the page replacement changes in Linux 2.4 has been influenced by two main factors. Firstly the bad behaviours of Linux 2.2 page replacement had to be fixed, using only known-good strategies because the development of Linux 2.4 had already entered the "code freeze" state. Secondly the page replacement had to be more predictable and easier to understand than Linux 2.2 because tuning the page replacement in Linux 2.2 was deserving of the proverbial label "subtle and quick to upset". This means that only VM ideas that are well understood and have little interactions with the rest of the system were integrated. Lots of ideas were taken from other freely available operating systems and literature. --Page aging Page aging was the first easy step in making the bad border-case behaviour from Linux 2.2 go away, it works reasonably well in Linux 1.2, Linux 2.0 and FreeBSD. Page aging allows us to make a much finer distinction between pages we want to keep in memory and pages we want to swap out than the NRU aging in Linux 2.2. Page aging in these OSes works as follows: for each physical page we keep a counter (called age in Linux, or act_count in FreeBSD) that indicates how desirable it is to keep this page in memory. When scanning through memory for pages to evict, we increase the page age (adding a constant) whenever we find that the page was accessed and we decrease the page age (substracting a constant) whenever we find that
the page wasn't accessed. When the page age (or act_count) reaches zero, the page is a candidate for eviction. However, in some situations the LFU[Note] page aging of Linux 2.0 is known to have too much CPU overhead and adjust to changes in system load too slowly. Furthermore, research[Smaragdis, Kaplan, Wilson] has shown that recency of access is a more important criteria for page replacement than frequency. These two problems are solved by doing exponential decline of the page age (divide by two instead of substracting a constant) whenever we find a page that wasn't accessed, resulting in page replacement which is closer to LRU[Note] than LFU. This reduces the CPU overhead of page aging drastically in some cases; however, no noticable change in swap behaviour has been observed. Another artifact comes from the virtual address scanning. In Linux 1.2 and 2.0 the system reduces the page age of a page whenever it sees that the page hasn't been accessed from the page table which it is currently scanning, completely ignoring the fact that the page could have been accessed from other page tables. This can put a severe penalty on heavily shared pages, for example the C library. This problem is fixed by simply not doing "downwards" aging from the virtual page scans, but only from the physical-page based scanning of the active list. If we encounter pages which are not referenced, present in the page tables but not on the active list, we simply follow the swapout path to add this page to the swap cache and the active list so we'll be able to lower the page age of this page and swap it out as soon as the page age reaches zero. --Multiple page lists The bad interactions between page aging and page flushing, where referenced clean pages were freed before old dirty pages, is fixed by keeping the pages which are candidates for eviction separated from the pages we want to keep in memory (page age zero vs. nonzero). We separate the pages out by putting them on various page lists and having separate algorithms deal with each list. Pages which are not (yet) candidate for eviction are in process page tables, on the active list or both. Page aging as described above happens on these pages, with the function refill_inactive() balancing between scanning the page tables and scanning the active list. When the page age on a page reaches zero, due to a combination of pageout scanning and the page not being actively used, the page is moved to the inactive_dirty list. Pages on this list are not mapped in the page tables of any process and are, or can become, reclaimable. Pages on this list are handled by the function page_launder(), which flushes the dirty pages to disk and moves the clean pages to the inactive_clean list. Unlike the active and inactive_dirty lists, the inactive_clean list isn't global but per memory zone. The pages on these lists can be immediately reused by the page allocation code and count as free pages. These pages can also still be faulted back into where it came from,
since the data is still there. In BSD this would be called the "cache" queue. --Dynamically sized inactive list Since we do page aging to select which pages to evict, having a very large statically sized inactive list (like FreeBSD has) doesn't seem to make much sense. In fact, it would cancel out some of the effects of doing the page aging in the first place: why spend much effort selecting which pages to evict[Dillon] when you keep as much as 33% of your swappable pages on the inactive list? Why do careful page aging when 33% of your pages end up as candidates for eviction at the same priority and you've effectively undone the aging for those 33% of pages which are candidates for eviction? On the other hand, having lots of inactive pages to choose from when doing page eviction means you have more chances of avoiding writeout IO or doing better IO clustering. It also gives you more of a "buffer" to deal with allocations due to page faults, etc. Both a large and a small target size for the inactive page list have their benefits. In Linux 2.4 we have chosen for a middle ground by letting the system dynamically vary the size of the inactive list depending on VM activity, with an artificial upper limit to make sure the system always preserves some aging information. Linux 2.4 keeps a floating average of the amount of pages evicted per second and sets the target for the inactive list and the free list combined to the free target plus this average number of page steals per second. Not only does this second give us enough time to do all kinds of page flushing optimisations, it also is small enough to keep page age distribution within the system intact, allowing us to make good choices on which pages to evict and which pages to keep. --Optimised page flushing Writing out pages from the inactive_dirty list as we encounter them can cause a system to totally destroy read performance because of the extra disk seeks done. A better solution is to delay writeout of dirty pages and let these dirty pages accumulate until we can do better IO clustering so that these pages can be written out to disk with less disk seeks and less interference with read performance. Due to the development of the page replacement changes happening in the code freeze, the system currently has a rather simple implementation of what's present in FreeBSD 4.2. As long as there are enough clean inactive pages around, we keep moving those to the inactive_clean list and never bother with syncing out the dirty pages. Note that this catches both clean pages and pages which have been written to disk by the update daemon (which commits filesystem data to disk periodically). This means that under loads where data is seldom written we can avoid writing out dirty inactive pages most of the time, giving us much better latencies in freeing pages and letting streaming reads continue without the disk head moving away to write out data all the time. Only under loads where lots of pages are being dirtied quickly does the system suffer a bit from syncing out dirty data irregularly.
Another alternative would have been the strategy used in FreeBSD 4.3, where dirty pages get to stay in the inactive list longer than clean pages but are synced out before the clean pages are exhausted. This strategy gives more consistent pageout IO in FreeBSD during heavy write loads. However, a big factor causing the irregularities in pageout writes using the simpler strategy above may well be caused because of the huge inactive list target in FreeBSD (33It is not at all clear what this more complicated strategy would do when used on the dynamically sized inactive list on Linux 2.4, because of this Linux 2.4 uses the better understood strategy of evicting clean inactive pages first and only after those are gone start syncing the dirty ones. --Background page aging On many systems the normal operating mode is that after a period of relative activity a sudden load spike comes in and the system has to deal with that as gracefully as possible. Linux 2.2 has the problem that, with the lack of an inactive page list, it is not clear at all which pages should be evicted when a sudden demand for memory kicks in. Linux 2.4 is better in this respect, with the reclaim candidates neatly separated out on the inactive list. However, the inactive list could have any random size the moment VM pressure drops off. We'd like get the system in a more predictable state while the VM pressure is low. In order to achieve this, Linux 2.4 does background scanning of the pages, trying to get a sane amount of pages on the inactive list, but without scanning agressively so only truly idle pages will end up on the inactive list and the scanning overhead stays small. --Drop behind Streaming IO doesn't just have readahead, but also its natural complement: drop behind. After the program doing the streaming IO is done with a page, we depress its priority heavily so it will be a prime candidate for eviction. Not only does this protect the working set of running processes from being quickly evicted by streaming IO, but it also prevents the streaming IO from competing with the pageouts and pageins of the other running processes, which reduces the number of disk seeks and allows the streaming IO to proceed at a faster speed. Currently readahead and drop-behind only work for read() and write(); mmap()ed files and swap-backed anonymous memory aren't supported yet. --Conclusions Since the Linux 2.4 kernel's VM subsystem is still being tuned heavily, it is too early to come with conclusive figures on performance. However, initial results seem to indicate that Linux 2.4 generally has better performance than Linux 2.2 on the same hardware. Reports from users indicate that performance on typical desktop machines has improved a lot, even though the tuning of the new VM has only just begun. Throughput figures for server machines seem to be better too, but that could also be attributed to the fact that the unification of the page cache and the buffer cache is complete. One big difference between the VM in Linux 2.4 and the VM in Linux 2.2 is that the new VM is far less sensitive to subtle changes. While in Linux 2.2 a subtle change in the page flushing logic could upset page replacement, in Linux 2.4 it is possible to tweak the various aspects
of the VM with predictable results and little to no side-effects in the rest of the VM. The solid performance and relative insensitivity to subtle changes in the environment can be taken as a sign that the Linux 2.4 VM is not just a set of simple fixes for the problems experienced in Linux 2.2, but also a good base for future development. Remaining issues The Linux 2.4 VM mainly contains easy to implement and obvious to verify solutions for some of the known problems Linux 2.2 suffers from. A number of issues are either too subtle to implement during the code freeze or will have too much impact on the code. The complete list of TODO items can be found on the Linux-MM page[Linux-MM]; here are the most important ones: Low memory deadlock prevention: with the arrival of journaling and delayed-allocation filesystems it is possible that the system will need to allocate memory in order to free memory; more precisely, to write out data so memory can become freeable. To remove the possibility for deadlock, we need to limit the number of outstanding transactions to a safe number, possibly letting each of the page flushing functions indicate how much memory it may need and doing bookkeeping of these values. Note that the same problem occurs with swap over network. Load control: no matter how good we can get the page replacement code, there will always be a point where the system ends up thrashing to death. Implementing a simple load control system, where processes get suspended in round-robin fashion when the paging load gets too high, can keep the system alive under heavy overload and allow the system to get enough work done to bring itself back to a sane state. RSS limits and guarantees: in some situations it is desirable to control the amount of physical memory a process can consume (the resident set size, or RSS). With the virtual address based page scanning of Linux' VM subsystem it is trivial to implement RSS ulimits and minimal RSS guarantees. Both help to protect processes under heavy load and allow the system administrator to better control the use of memory resources. VM balancing: in Linux 2.4, the balancing between the eviction of cache pages, swap-backed anonymous memory and the inode and dentry caches is essentially the same as in Linux 2.2. While this seems to work well for most cases there are some possible scenarios where a few of the caches push the other users out of memory, leading to suboptimal system performance. It may be worthwhile to look into improving the balancing algorithm to achieve better performance in "non-standard" situations. Unified readahead: currently readahead and drop-behind only works for read() and write(). Ideally they should work for mmap()ed files and anonymous memory too. Having the same set of algorithms for both read()/write(), mmap() and swap-backed anonymous memory will simplify the code and make performance improvements in the readahead and dropbehind code immediately available to all of the system.
AIX swap notes: --------------Note 1: ------Q: Hi All, I'm seeing an interesting paging behavior (paging out to paging space when I don't think it should) on our AIX 5.3 TL3CSP system. First the system particulars: AIX 5.3 TL3 with CSP HACMP v5.2 Oracle 10g 28GB memory 8GB paging space EMC LUNs for Oracle data. CIO used for Oracle data. Virtual memory tuned as such vmo -p -o maxclient%=50 vmo -p -o maxperm%=50 vmo -p -o 'lru_file_repage=0' vmo -p -o 'minperm%=3' So, given that configuration, it is my understanding that AIX, when under memory pressure, will steal memory from the file cache instead of paging process memory out to the paging space (lru_file_repage = 0). Now, this system works for the most part like I understand it should. Via nmon, I can watch it stealing memory from the FileSystemCache (numclient values decrease) when the box gets under memory pressure. However, every once in a while when under memory pressure, I can see that the system starts writing to the paging space when there is plenty of FileSystemCache available to steal from. Below is a snapshot from the nmon 'm'emory switch: nmon.jpg You can see here that I've got 1.7GB paged out, while numclient is at 21%. So, my question is, why does AIX page out when under memory pressure instead of stealing from the FileSystemCache memory like I want it to? A: Look at the Paging to/from the Paging Space - its zero. Once info is in the paging space its left there until the space is needed for something else. So at this point the server isn't actually paging. It Has paged in the past however.
Note 2: -----AIX will always try to use 100% of real memory--> AIX will use the amount of memory solicited by your processes. The remaining capacity will be used as filesystem cache. You can change the minimum and maximum amounts of memory used to cache files with vmtune (vmo for 5.2+), and it is advised to do so if your're running databases with data on raw devices (since the db engine usually has its own cache algorithm, and AIX can't cache data on raw devices). The values to modify are minperm, maxperm, minclient and maxpin (use at you own risk!!!). Paging space use will be very low: 5% is about right--> A paging space so little used seems to be oversized. In general, the paging space should be under 40%, and the size must be determined accordingly to the application running (i.e. 4X the physical memory size for oracle). In AIX 5L a paging space can be reduced without rebooting. Anyway, AIX always uses some paging space, even keeping copies of the data on memory and on disk, as a "predictive" paging. Look in topas for the values "comp mem" (proceses) and "non comp mem" (filesystem cache) to see the distribution of the memory usage. Nmon can show you the top proceses by memory usage, along with many other statistics. There are several tools which can give you a more detailed picture of how memory is being used. "svmon" is very comprehensive. Tools such as topas and nmon will also give you a bit more information. Note 3: ------Memory utilization on AIX systems typically runs around 100%. This is often a source of concern. However, high memory utilization in AIX does not imply the system is out of memory. By design, AIX leaves files it has accessed in memory.
This significantly improves performance when AIX reaccesses these files because they can be reread directly from memory, not disk*. When AIX needs memory, it discards files using a "least used" algorithm. This generates no I/O and has almost no performance impact under normal circumstances. Sustained paging activity is the best indication of low memory. Paging activity can be monitored using the "vmstat" command. If the "page-in" (PI) and "page-out" (PO) columns show non-zero values over "long" periods of time, then the system is short on memory. (All systems will show occasional paging, which is not a concern.) Memory requirements for applications can be empirically determined using the AIX "rmss"command. The "rmss" command is a test tool that dynamically reduces usable memory. The onset of paging indicates an application's minimum memory requirement. Finally, the "svmon" command can be used to list how much memory is used each process. The interpretation of the svmon output requires some expertise. See the AIX documentation for details.
================================================================== 35 Volume group, logical volumes, and filesystem commands in HPUX: ================================================================== 35.1 Filesystems in HPUX: ------------------------HFS : used at HP-UX < v. 10 VxFS: used at HP-UX >= v. 10 Ofcourse, CDFS (cdroms), and other filesystem types, are supported. HP-UX's implementation of a journaled file system, also known as JFS, is based on the version from VERITAS Software Inc. called VxFS. Up through the 10.0 release of HP-UX, HFS has been the only available locally mounted read/write file system. Beginning at 10.01, you also have the option of using VxFS. (Note, however, that VxFS cannot be used as the root file system.) As compared to HFS, VxFS allows much shorter recovery times in the event of system failure. It is also particularly useful in environments that require high performance or deal with large volumes of data. This is because the unit of file storage, called an extent, can be multiple blocks, allowing considerably faster I/O than with HFS. It also provides for minimal downtime by allowing online backup and administration - that is, unmounting the file system will not be necessary for
certain tasks. You may not want to configure VxFS, though, on a system with limited memory because VxFS memory requirements are considerably larger than that for HFS. Basic VxFS functionality is included with the HP-UX operating system software. Additional enhancements to VxFS are available as a separately orderable product called HP "OnlineJFS", product number B5117AA (Series 700) and B3928AA (Series 800). 35.2 How to create a filesystem in HP-UX: an outline. ------------------------------------------------------ Task 1. Estimate the Size Required for the Logical Volume -- Task 2. Determine If Sufficient Disk Space Is Available for the Logical Volume within Its Volume Group Use the vgdisplay command to calculate this information. vgdisplay will output data on one or more volume groups, including the physical extent size (under PE Size (Mbytes)) and the number of available physical extents (under Free PE). By multiplying these two figures together, you will get the number of megabytes available within the volume group. See vgdisplay(1M) for more information. -- Task 3. Add a Disk to a Volume Group If Necessary If there is not enough space within a volume group, you will need to add a disk to a volume group. To add a disk to an existing volume group, use pvcreate(1M) and vgextend(1M). You can also add a disk by creating a new volume group with pvcreate(1M) and vgcreate(1M). -- Task 4. Create the Logical Volume Use lvcreate to create a logical volume of a certain size in the above volume group. See lvcreate(1M) for details. Use lvcreate as in the following example: Create a logical volume of size 100 MB in volume group /dev/vg03: # lvcreate -L 100 /dev/vg03 -- Task 5. Create the New File System Create a file system using the newfs command. Note the use of the character device file. For example: # newfs -F hfs /dev/vg02/rlvol1 If you do not use the -F FStype option, by default, newfs creates a file system based on the content of your /etc/fstab file. If there is no entry for the file system in /etc/fstab, then the file system type
is determined from the file /etc/default/fs. For information on additional options, see newfs(1M). $ cat /etc/default/fs LOCAL=vxfs For HFS, you can explicitly specify that newfs create a file system that allows short file names or long file names by using either the -S or -L option. By default, these names will as short or long as those allowed by the root file system. Short file names are 14 characters maximum. Long file names allow up to 255 characters. Generally, you use long file names to gain flexibility in naming files. Also, files created on other systems that use long file names can be moved to your system without being renamed. When creating a VxFS file system, file names will automatically be long. After creating a filesystem, you need to mount it to make it accesible, for example like: -- Task 6. mount the new local file system: Choose an empty directory to serve as the mount point for the file system. Use the mkdir command to create the directory if it does not currently exist. For example, enter: # mkdir /test Mount the file system using the mount command. Use the block device file name that contains the file system. You will need to enter this name as an argument to the mount command. For example, enter # mount /dev/vg01/lvol1 /test Note: The newfs command is a "friendly" front-end to the mkfs command (see mkfs(1M)). The newfs command calculates the appropriate parameters and then builds the file system by invoking the mkfs command.
35.3 HP-UX LVM commands: ======================== -- vgdisplay: -- ---------Displays information about volume groups.
Examples: # vgdisplay # vgdisplay -v vgdatadir -- pvdisplay: -- ---------Display information about physical volumes within LVM volume group. EXAMPLES Display the status and characteristics of a physical volume: # pvdisplay /dev/dsk/c1t0d0 Display the status, characteristics, and allocation map of a physical volume: # pvdisplay -v /dev/dsk/c2t0d0 # pvdisplay /dev/dsk/c102t9d3 --- Physical volumes --PV Name PV Name VG Name PV Status Allocatable VGDA Cur LV PE Size (Mbytes) Total PE Free PE Allocated PE Stale PE IO Timeout (Seconds) Autoswitch
/dev/dsk/c43t9d3 /dev/dsk/c102t9d3 Alternate Link /dev/vgora_e1atlas_data available yes 2 2 4 1668 102 1566 0 default On
-- lvdisplay: -- ---------Displays information about logical volumes. Examples: # lvdisplay lvora_p0gencfg_apps # lvdisplay -v lvora_p0gencfg_apps # lvdisplay -v /dev/vg00/lvol2 # lvdisplay /dev/vgora_e0etea_data/lvora_e0etea_data --- Logical volumes --LV Name /dev/vgora_e0etea_data/lvora_e0etea_data VG Name /dev/vgora_e0etea_data LV Permission read/write LV Status available/syncd Mirror copies 1
Consistency Recovery Schedule LV Size (Mbytes) Current LE Allocated PE Stripes Stripe Size (Kbytes) Bad block Allocation IO Timeout (Seconds)
MWC parallel 17020 4255 8510 0 0 on strict default
-- vgchange: -- --------Set volume group availability. This command activates or deactivates one or more volume groups as specified by the -a option, namely y or n. Activate a volume group: # vgchange -a y /dev/vg03 Deactivate a volume group: # vgchange -a n /dev/vg03
-- vgcreate: -- --------/usr/sbin/vgcreate [-f] [-A autobackup] [-x extensibility] [-e max_pe] [-l max_lv] [-p max_pv] [-s pe_size] [-g pvg_name] vg_name pv_path ... The vgcreate command creates a new volume group. vg_name is a symbolic name for the volume group and must be used in all references to it. vg_name is the path to a directory entry under /dev that must contain a character special file named group. Except for the group entry, the vg_name directory should be empty. The vg_name directory and the group file have to be created by the user (see lvm(7)). vgcreate leaves the volume group in an active state. EXAMPLES 1. Create a volume group named /dev/vg00 containing two physical volumes with extent size set to 2 Mbytes. If directory /dev/vg00 exists with the character special file group, the volume group is created: # vgcreate -s 2 /dev/vg00 /dev/dsk/c1d0s2 /dev/dskc2d0s2 2. Create a volume group named /dev/vg01 that can contain a maximum of
three logical volumes, with extent size set to 8 Mbytes: # vgcreate -l 3 -s 8 /dev/vg01 /dev/dsk/c4d0s2 3. Create a volume group named /dev/vg00 and a physical volume group named PVG0 with two physical volumes: # vgcreate -g PVG0 /dev/vg00 /dev/dsk/c1d0s2 /dev/dsk/c2d0s2 3. Create a volume group named /dev/vg00 containing two physical volumes with extent size set to 2 MB, from scratch. First, create the directory /dev/vg00 with the character special file called group. mkdir /dev/vg00 mknod /dev/vg00/group c 64 0x030000 The minor number for the group file should be unique among all the volume groups on the system. It has the format 0xNN0000, where NN runs from 00 to ff. The maximum value of NN is controlled by the kernel tunable parameter maxvgs. Initialize the disks using pvcreate(1M). pvcreate /dev/rdsk/c1t0d0 pvcreate /dev/rdsk/c1t2d0 Create the volume group. vgcreate -s 2 /dev/vg00 /dev/dsk/c1t0d0 /dev/dsk/c1t2d0 Note About the "dsk" and "rdsk" notation: ----------------------------------------Physical volumes are identified by their device file names, for example /dev/dsk/cntndn /dev/rdsk/cntndn Note that each disk has a block device file and a character or raw device file, the latter identified by the r. Which name you use depends on what task you are doing with the disk. In the notation above, the first name represents the block device file while the second is the raw device file. -- Use a physical volume's raw device file for these two tasks only: -> When creating a physical volume. Here, you use the device file for the disk. For example, this might be /dev/rdsk/c3t2d0 if the disk were at card instance 3, target address 2, and device number 0.
(The absence of a section number beginning with s indicates you are referring to the entire disk.) -> When restoring your volume group configuration. For all other tasks, use the block device file. For example, when you add a physical volume to a volume group, you use the disk's block device file for the disk, such as /dev/dsk/c5t3d0.
-- vgextend: -- --------Extends a volume group by adding physical volumes to it. Examples: Add physical volumes /dev/dsk/c1d0s2 and /dev/dsk/c2d0s2 to volume group /dev/vg03: # vgextend /dev/vg03 /dev/dsk/c1d0s2 /dev/dsk/c2d0s2 # vgextend vg01 /dev/dsk/c0t4d0 -- pvcreate: -- --------Creates physical volume for use in a volume group. Examples: # pvcreate -f /dev/rdsk/c1d0s2 # ioscan -fnC disk # pvcreate -f /dev/rdsk/c0t1d0 -- lvcreate: -- --------Create logical volume in LVM volume group The lvcreate command creates a new logical volume within the volume group specified by vg_name. Up to 255 logical volumes can be created in one volume group SYNOPSIS /etc/lvcreate [-d schedule] {-l logical_extents_number | -L logical_volume_size} [-m mirror_copies] [-n lv_path] [-p permission] [-r relocate] [-s strict] [-C contiguous] [-M mirror_write_cache] [-c vol_group_name
Examples: Create a logical volume in volume group /dev/vg02: # lvcreate /dev/vg02 Create a logical volume in volume group /dev/vg03 with nonstrict allocation policy: # lvcreate -s n /dev/vg03 Create a logical volume of size 100 MB in volume group /dev/vg03: # lvcreate -L 100 /dev/vg03 Create a logical volume of size 90 MB striped across 3 disks with a stripe size of 64 KB: # lvcreate -L 90 -i 3 -I 64 /dev/vg03 -- fstyp: -- -----Determines file system type. SYNOPSIS /usr/sbin/fstyp [-v] special The fstyp command allows the user to determine the file system type of a mounted or unmounted file system. special represents a device special file (for example: /dev/dsk/c1t6d0). The file system type is determined by reading the superblock of supplied special file. If the superblock is read successfully, the command prints the file system type identifier on the standard output and exits with an exit status of 0. If the type of the file system cannot identified, the error message unknown_fstyp (no matches) is printed and the exit status is 1. status 2 is not currently returned, but is reserved for the situation where the file system matches than one file system type. Any other error will cause exit status 3 to be returned.
the
be Exit more
The file system type is determined by reading the superblock of the supplied special file. Examples: Find the type of the file system on a disk, /dev/dsk/c1t6d0: # fstyp /dev/dsk/c1t6d0 Find the type of the file system on a logical volume, /dev/vg00/lvol6: # fstyp /dev/vg00/lvol6
Find the file system type for a particular device file and also information about its super block: # fstyp -v /dev/dsk/c1t6d0
-- mkboot: -- ------mkboot is used to install or update boot programs on the specified device file. The position on device at which boot programs are installed depends on the disk layout of the device. mkboot examines device to discover the current layout and uses this as the default. If the disk is uninitialized, the default is LVM layout on PA-RISC and Whole Disk on Itanium(R)-based systems. The default can be overridden by the -l, -H, or -W options. Boot programs are stored in the boot area in Logical Interchange Format (LIF), which is similar to a file system. For a device to be bootable, the LIF volume on that device must contain at least the ISL (the initial system loader) and HPUX (the HP-UX bootstrap utility) LIF files. If, in addition, the device is an LVM physical volume, the LABEL file must be present (see lvlnboot(1M) ). For the VERITAS Volume Manager (VxVM) layout on the Itanium-based system architecture, the only relevant LIF file is the LABEL file. All other LIF files are ignored. VxVM uses the LABEL file when the system boots to determine the location of the root, stand, swap, and dump volumes. EXAMPLES Install default boot programs on the specified disk, treating it as an LVM disk: # mkboot -l /dev/dsk/c0t5d0 Use the existing layout, and install only SYSLIB and ODE files and preserve the EST file on the disk: # mkboot -i SYSLIB -i ODE -p EST /dev/rdsk/c0t5d0 Install only the SYSLIB file and retain the ODE file on the disk. Use the Whole Disk layout. Use the file /tmp/bootlf to get the boot programs rather than the default. (The -i ODE option will be ignored): # mkboot -b /tmp/bootlf -i SYSLIB -i ODE -p ODE -W /dev/rdsk/c0t5d0
Install EFI utilities to the EFI partition on an Itanium-based system, treating it as an LVM or VxVM disk: # mkboot -e -l /dev/dsk/c3t1d0 Create AUTO file with the string autofile command on a device. If the device is on an Itanium-based system, the file is created as /EFI/HPUX/AUTO in the EFI partition. If the device is on a PA-RISC system, the file is created as a LIF file in the boot area. # mkboot -a "autofile command" /dev/dsk/c2t0d0 -- bdf: -- ---Report number of free disk blocks. bdf prints out the amount of free disk space available on the specified filesystem (/dev/dsk/c0d0s0, for example) or on the file system in which the specified file ($HOME, for example) is contained. If no file system is specified, the free space on all of the normally mounted file systems is printed. The reported numbers are in kilobytes. Examples: # bdf oranh300:/home/se1223>bdf | more Filesystem kbytes used avail %used Mounted on /dev/vg00/lvol3 434176 165632 266504 38% / /dev/vg00/lvol1 298928 52272 216760 19% /stand /dev/vg00/lvol8 2097152 1584488 508928 76% /var /dev/vg00/lvol11 524288 2440 490421 0% /var/tmp /dev/vg00/lvucmd 81920 1208 75671 2% /var/opt/universal /dev/vg00/lvol9 1048576 791925 240664 77% /var/adm /dev/vg00/lvol10 2064384 47386 1890941 2% /var/adm/crash /dev/vg00/lvol7 1548288 1262792 283320 82% /usr /dev/vg00/vsaunixlv 311296 185096 118339 61% /usr/local/vsaunix /dev/vg00/lvol4 1867776 5264 1849784 0% /tmp /dev/vg00/lvol6 1187840 757456 427064 64% /opt /dev/vg00/lvol5 262144 34784 225632 13% /home /dev/vg00/lvbeheer 131072 79046 48833 62% /beheer /dev/vg00/lvbeheertmp 655360 65296 553190 11% /beheer/tmp /dev/vg00/lvbeheerlog 524288 99374 398407 20% /beheer/log /dev/vg00/lvbeheerhistlog .. .. # bdf /tmp
Filesystem /dev/vg00/lvol4
kbytes 1867776
used avail %used Mounted on 5264 1849784 0% /tmp
-- lvextend: -- --------Increase number of physical extents allocated to a logical volume. /etc/lvextend {-l logical_extents_number | -L logical_volume_size | -m mirror_copies} lv_path [physical_volume_path ... | physical_vol_group_name...] lvextend increases the number of mirrored copies or the size of the lv_path parameter. The change is determined according to which command options are specified. WARNINGS The -m option cannot be used on HP-IB devices. EXAMPLES - Increase the number of the logical extents of a logical volume to one hundred: # lvextend -l 100 /dev/vg01/lvol3 - Increase the logical volume size to 400 Mbytes: # lvextend -L 400 /dev/vg01/lvol4 Allocate two mirrors (that is, three copies) for each logical extent of a logical volume: # lvextend -m 2 /dev/vg01/lvol5
-- extendfs: -- --------Extend file system size. /etc/extendfs [-q] [-v] [-s size] special If the original hfs filesystem image created on special does not make use of all of the available space, extendfs can be used to increase the capacity of an hfs filesystem by updating the filesystem structure to include the extra space. The command-line parameter special specifies the character device special file of either a logical volume or a disk partition. If special refers to a mounted filesystem, special must be un-mounted before extendfs can be run (see mount(1M)). The root filesystem cannot be extended using the extendfs command
because the root filesystem is always mounted, and extendfs only works on unmounted filesystems. EXAMPLES To increase the capacity of a filesystem created on a logical volume, enter: # umount /dev/vg00/lvol1 # lvextend -L larger_size /dev/vg00/lvol1 # extendfs /dev/vg00/rlvol1 -- fsadm: -- -----EXAMPLES Convert a HFS file system from a nolargefiles file system to a largefiles file system: # fsadm -F hfs -o largefiles /dev/vg02/lvol1 Display HFS relevant file system statistics: # fsadm -F hfs /dev/vg02/lvol1 -- diskinfo: -- --------diskinfo - describe characteristics of a disk device SYNOPSIS /etc/diskinfo [-b|-v] character_devicefile DESCRIPTION diskinfo determines whether the character special file named by character_devicefile is associated with a SCSI, CS/80, or Subset/80 disk drive; if so, diskinfo summarizes the disk's characteristics. Example: # diskinfo /dev/rdsk/c31t1d3 SCSI describe of /dev/rdsk/c31t1d3: vendor: IBM product id: 2105800 type: direct access size: 13671904 Kbytes bytes per sector: 512
35.4 Notes and further examples: ================================ Examples: More on how to create a filesystem on HP-UX: -----------------------------------------------------Example 1: ---------Here we repeat the essentials of section 35.2: Task 1. Estimate the Size Required for the Logical Volume Task 2. Determine If Sufficient Disk Space Is Available for the Logical Volume within Its Volume Group Task 3. Add a Disk to a Volume Group If Necessary Task 4. Create the Logical Volume Use lvcreate to create a logical volume of a certain size in the above volume group. See lvcreate(1M) for details. Use lvcreate as in the following example: Create a logical volume of size 100 MB in volume group /dev/vg03: # lvcreate -L 100 /dev/vg03 -- Task 5. Create the New File System Create a file system using the newfs command. Note the use of the character device file. For example: # newfs -F hfs /dev/vg02/rlvol1 -- Task 6. mount the new local file system: Choose an empty directory to serve as the mount point for the file system. Use the mkdir command to create the directory if it does not currently exist. For example, enter: # mkdir /test Mount the file system using the mount command. Use the block device file name that contains the file system. You will need to enter this name as an argument to the mount command. For example, enter # mount /dev/vg01/lvol1 /test
Example 2: ---------This is an example of creating volume group vg01 & logical
volume/partion data. Prepare for logical volume creation: root:/> mkdir /dev/vg01 root:/> mknod /dev/vg01/group c 64 0x010000 root:/> pvcreate -f /dev/rdsk/c0t5d0 Physical volume "/dev/rdsk/c0t5d0" has been successfully created. root:/> vgcreate vg01 /dev/dsk/c0t5d0 Volume group "/dev/vg01" has been successfully created. Volume Group configuration for /dev/vg01 has been saved in /etc/lvmconf/vg01.conf root:/> vgdisplay -v vg01 root:/> lvcreate -L 100 -n data vg01 Logical volume "/dev/vg01/data" has been successfully created with character device "/dev/vg01/rdata". Create HFS file system root:/> newfs -F hfs /dev/vg01/rdata Create Journal or Veritas file system root:/> newfs -F vxfs /dev/vg02/rdata
Example 3: ---------To create a VxFS file system 12288 sectors in size on VxVM volume, enter: # mkfs -F vxfs /dev/vx/rdsk/diskgroup/volume 12288 To use mkfs to create a VxFS file system on /dev/rdsk/c0t6d0: # mkfs -F vxfs /dev/rdsk/c0t6d0 1024 To use mkfs to determine the command that was used to create the VxFS file system on /dev/rdsk/c0t6d0: # mkfs -F vxfs -m /dev/rdsk/c0t6d0 To create a VxFS file system on /dev/vgqa/lvol1, with a Version 4 disk layout and largefiles capability: # mkfs -F vxfs -o version=4,largefiles /dev/vgqa/lvol1 http://www.docs.hp.com/en/B2355-90672/index.html Example 4: ----------
Example: Creating a Logical Volume Using HP-UX Commands To create a logical volume: Select one or more disks. ioscan(1M) shows the disks attached to the system and their device file names. Initialize each disk as an LVM disk by using the pvcreate command. For example, enter # pvcreate /dev/rdsk/c0t0d0 Note that using pvcreate will result in the loss of any existing data currently on the physical volume. You use the character device file for the disk. Once a disk is initialized, it is called a physical volume. - Pool the physical volumes into a volume group. To complete this step: Create a directory for the volume group. For example: # mkdir /dev/vgnn Create a device file named group in the above directory with the mknod command. # mknod /dev/vgnn/group c 64 0xNN0000 The c following the device file name specifies that group is a character device file. The 64 is the major number for the group device file; it will always be 64. The 0xNN0000 is the minor number for the group file in hexadecimal. Note that each particular NN must be a unique number across all volume groups. For more information on mknod, see mknod(1M); for more information on major numbers and minor numbers, see Configuring HP-UX for Peripherals. Create the volume group specifying each physical volume to be included using vgcreate. For example: # vgcreate /dev/vgnn /dev/dsk/c0t0d0 Use the block device file to include each disk in your volume group. You can assign all the physical volumes to the volume group with one command. No physical volume can already be part of an existing volume group. Once you have created a volume group, you can now create a logical volume using lvcreate. For example: # lvcreate /dev/vgnn Using the above command creates the logical volume /dev/vgnn/lvoln with LVM automatically assigning
the n in lvoln. When LVM creates the logical volume, it creates the block and character device files and places them in the directory /dev/vgnn.
VxFS can, theoretically, support files up to two terabytes in size because file system structures are no longer in fixed locations (see Chapter 2 "Disk Layout"). The maximum size tested and supported on HP-UX 11.x systems is one terabyte. Large files are files larger than two gigabytes in size. NOTE: Be careful when enabling large file capability. Applications and utilities such as backup may experience problems if they are not aware of large files. Creating a File System with Large Files You can create a file system with large file capability by entering the following command: # mkfs -F vxfs -o largefiles special_device size Specifying largefiles sets the largefiles flag, which allows the file system to hold files up to one terabyte in size. Conversely, the default nolargefiles option clears the flag and limits files being created to a size of two gigabytes or less: # mkfs -F vxfs -o nolargefiles special_device size
Notes: -----Note 1: Create a System Mirror Disk: -----------------------------------This note describes how to configure LVM mirroring of a system disk. In this example the HP server is STSRV1, the primary boot device is SCSI=6 (/dev/dsk/c2t6d0) and the alternative mirrored bootdevice is SCSI=5 (/dev/dsk/c2t5d0). The following commands will do the trick: # # # # #
ioscan -fnC disk pvcreate -Bf /dev/rdsk/c2t5d0 mkboot -l /dev/rdsk/c2t5d0 mkboot -a "hpux -lq (;0)/stand/vmunix" /dev/rdsk/c2t5d0 vgextend /dev/vg00 /dev/dsk/c2t5d0
# for P in 1 2 3 4 5 6 7 8 9 10
> > > >
do lvextend -m 1 /dev/vg00/lvol$P /dev/dsk/c2t5d0 sleep 1 done
Note 2: Create a System Mirror Disk: -----------------------------------# ioscan -fnC disk Class I H/W Path Driver S/W State H/W Type Description ===================================================================== disk 0 0/0/1/1.2.0 sdisk CLAIMED DEVICE HP 73.4GMAN3735MC /dev/dsk/c1t2d0 /dev/rdsk/c1t2d0 disk 1 0/0/2/0.2.0 sdisk CLAIMED DEVICE HP 73.4GATLAS10K3_73_SCA /dev/dsk/c2t2d0 /dev/rdsk/c2t2d0 Note: c1t2d0 is the boot disk and c2t2d0 is the mirrored disk. 1) Initialize the disk and make it bootable pvcreate -B /dev/rdsk/c2t2d0 Note: the -B parameter tells pvcreate that this will be a bootable disk. 2) Add the physical volume to the volume group vgextend /dev/vg00 /dev/dsk/c2t2d0 3) Use mkboot to place the boot utilities in the boot area and add the AUTO file. mkboot /dev/dsk/c2t2d0 mkboot -a "hpux -lq" /dev/rdsk/c2t2d0 4) Use mkboot to update the AUTO file on the primary boot disk. mkboot -a "hpux -lq" /dev/rdsk/c1t2d0 5) Mirror the stand, lvextend lvextend lvextend
root -m 1 -m 1 -m 1
and swap logical volumes /dev/vg00/lvol1 /dev/vg00/lvol2 /dev/vg00/lvol3
Note: LVM will resynchronize the new mirror copies. Repeat the lvextend for lvextend -m lvextend -m lvextend -m lvextend -m lvextend -m
all other logical volumes on the boot mirror. 1 /dev/vg00/lvol4 1 /dev/vg00/lvol5 1 /dev/vg00/lvol6 1 /dev/vg00/lvol7 1 /dev/vg00/lvol8
6) Modify your alternate boot path to point to the mirror copy of the boot disk. Note: Use the Hardware path for your new boot disk. setboot -a 0/0/2/0.2.0
Note 3: Increase a filesystem in HP-UX: --------------------------------------Example 1: ---------In this example, you would need to increase the file system size of /var by 10 MB, which actually needs to be rounded up to 12 MB. Increase /var Follow these steps to increase the size limit of /var. - Determine if any space is available for the /dev/vg00: # /sbin/vgdisplay /dev/vg00 The Free PE indicates the number of 4 MB extents available, in this case 79 (equivalent to 316 MB). - Change to single user state: /sbin/shutdown This allows /var to be unmounted. - View mounted volumes: # /sbin/mount You see a display similar to the following: / on /dev/vg00/lvol1 defaults on Sat Mar 8 23:19:19 1997 /var on /dev/vg00/lvol7 defaults on Sat Mar 8 23:19:28 1997 # Determine which logical volume maps to /var. In this example, it is /dev/vg00/lvol7 - Unmount /var: # /sbin/umount /var This is required for the next step, because extendfs can only work on unmounted volumes. If you get a "device busy" error at this point, reboot the system and log on in single-user mode before continuing. - Extend the size of the logical volume: # /sbin/lvextend -L new_size_in_MB /dev/vg00/lvol7 For example, to make this volume 332 MB:
# /sbin/lvextend -L 332 /dev/vg00/lvol7 To extend the file system size to the logical volume size: # /sbin/extendfs /dev/vg00/rlvol7 Mount /var: # /sbin/mount /var Go back to the regular init state: init 3 or init 4, or reboot. Example 2: ---------To increase the capacity of a file system created on a logical volume, enter: # umount /dev/vg00/lvol1 # lvextend -L larger_size /dev/vg00/lvol1 # extendfs -F hfs /dev/vg00/rlvol1 -- For operation like mkfs or extendfs, you should use raw device interface. # mount /dev/vg00/lvol1 mount_directory Example 3: ---------> > Date: 12/14/99 > Document description: Extending /var, /usr, /tmp without Online JFS > Document id: KBRC00000204 > > > You may provide feedback on this document > > > Extending /var, /usr, /tmp without Online JFS DocId: KBRC00000204 Updated: > 12/14/99 1:14:29 PM > > PROBLEM > Since /var, /usr, /tmp (and sometimes /opt) are always in use by the > operating system, they cannot be unmounted with the umount command. In order > to extend these filesystems, the system must be in single user mode. > > RESOLUTION > This example will show how to extend /usr to 400MB without Online JFS > > > 1.. Backup the filesystem before extending > > > 2.. Display disk information on the logical volume >
> lvdisplay -v /dev/vg00/lvol4 | more > > > a.. Make sure this is enough Free PE's to increase this filesystem. > b.. Make sure that allocation is NOT strict/contiguous. > > > 3.. Reboot the machine > > shutdown -r now > > > 4.. When prompted, press "ESC" to interrupt the boot. > > > 5.. Boot from the primary device and invoke ISL interaction. > > bo pri isl > > NOTE: If prompted to interact with ISL, respond "y" > > > 6.. Boot into single user mode > > hpux -is > > NOTE:Nothing will be mounted. > > > 7.. Extend the logical volume that holds the filesystem. > > /sbin/lvextend -L 400 /dev/vg00/lvol4 > > > 8.. Extend the file system. > > /sbin/extendfs -F hfs /dev/vg00/rlvol4 > > NOTE: The use of the character device. > > > 9.. Ensure the filesystem now reports to be the new size > > bdf > > > 10.. Reboot the system to its normal running state. > > shutdown -r now > > > The only thing is that you have to have contiguous lvols to do that. The best way is to do an Ignite make_tape_recovery -i for vg00 and then resize it when you recreate it. If you have vg00 on a seperate disk then
it is real easy, the backup can run in the background, and the restore interactive will take about 2.5 hours for a 9GB root disk, you can make the lvols any size you want and it also puts it back in place in order so you save space. Example 4: ---------The right way to extend a file system with "OnLine jfs" is using the command "fsadm". For example, if you want to extend the fs /mk2/toto in the /dev/vgmk2/lvtoto in from 50Mbytes to 60 you must extend de logical volume # lvextend -L 60 /dev/vgmk2/lvtoto Now use fsadm ( I supose you have vxfs, if you are using hfs is not possible to increase on-line, or at least I don't know how ). # fsadm -F vxfs -b 61440 /mk2/toto You will have your fs increased on line ... be carefull if your fs is 100% occupied the comand fsadm will fail, you need some free space on the file system ( it depends on the fs type, size etc ..). In general, Online jfs should be increased in the following way: lvextend -L ???? /dev/vg??/lvol?? fsadm -F vxfs -b ????? / oranh300:/home/se1223>cat /etc/inittab | grep enab vxen::bootwait:/sbin/fs/vxfs/vxenablef -a Note 4: ------Extend OnlineJFS licenses on next D&ST servers: aavnh400 oranh503 oranh603 orazh500 orazh601 orazh602 commands are: swagentd -r swinstall -x mount_all_filesystems=false -x enforce_dependencies=true -s hpdepot.ao.nl.abnamro.com:/beheer/depot/OnlineJFS_License OnlineJFS swagentd -k
HP-UX errors: Error 23 filetable overflow:
-----------------------------------------Error: 23 is a infamous error, as shown in this thread: thread: Doc ID: Note:1018306.102 Problem Description: ==================== You are backing up your database and are getting the following errors: HP-UX Error 23: file table overflow RMAN-569 file not found LEM-00031 file not found LEM-00033 lempgfm couldn't open message file RMAN indicates that Recovery Manager is complete, however the database and the catalog are not resync'd. Problem Explanation: ==================== Recovery Manager cannot find or open the message file. Search Words: ============= Recovery Manager, LEM-33, LEM-31, RMAN-00569, message file, lempgfm, error 23, HPUX error 23, HP-UX error 23 Solution Description: ===================== You may need to increase the value of the unix kernel parameter 'nfile'. Solution Explanation: ===================== 'nfile' needs to have a value in the thousands for a database server. If this parameter is < 1000, increase it to something like 5000 or greater. If there is enough memory on your system, this parameter can be set to values > 30000.
35.5 Some important filesystem related kernel params: ===================================================== nfile: -----nfile defines the maximum number of files that can be open simultaneously, system-wide, at any given time. Acceptable Values: Minimum 14 Maximum Memory limited Default ((16*(Nproc+16+MaxUsers)/10)+32+2*(Npty+Nstrpty) Specify integer value or use integer formula expression. For more information, see Specifying Parameter Values.
Description nfile defines the maximum number files that can be open at any one time, system-wide. It is the number of slots in the file descriptor table. Be generous with this number because the required memory is minimal, and not having enough slots restricts system processing capacity. Related Parameters and System Factors The value used for nfile must be sufficient to service the number of users and processes allowed by the combination of nproc, maxusers, npty , and nstrpty. Every process uses at least three file descriptors per process (standard input, standard output, and standard error). Every process has two pipes per process (one per side), each of which requires a pty. Stream pipes also use s treams ptys which are limited by nstrpty.
35.6 HP-UX kernel parameters: ============================= Take especially notice of the parameters nfile, nflocks, ninodes, nprocs. They determine how many open files, open locks, simultaneous processes are possible *system-wide*. Too low values may result in HP-UX errors when dealing with larger databases, huge App Servers and the like. Entering Values: Use the kcweb web interface or the kmtune command to view and change values. kcweb is described in the kcweb(1M) manpage and in the program's help topics. You can run kcweb from the command line or from the System Administration Manager (SAM); see sam(1M). You run kmtune from the command line; see kmtune(1M) for details.
Accounting acctresume Resume accounting when free space on the file system where accounting log files reside rises above acctresume plus minfree percent of total usable file system size. Manpage: acctsuspend(5). Accounting acctsuspend
Suspend accounting when free space on the file system where accounting log files reside drops below acctsuspend plus minfree percent of total usable file system size. Manpage: acctsuspend(5). Asynchronous I/O aio_listio_max Maximum number of POSIX asynchronous I/O operations allowed in a single lio_listio() call. Manpage: aio_listio_max(5). Asynchronous I/O aio_max_ops System-wide maximum number of POSIX asynchronous I/O operations allowed at one time. Manpage: aio_max_ops(5). Asynchronous I/O aio_physmem_pct Maximum percentage of total system memory that can be locked for use in POSIX asynchronous I/O operations. Manpage: aio_physmem_pct(5). Asynchronous I/O aio_prio_delta_max Maximum priority offset (slowdown factor) allowed in a POSIX asynchronous I/O control block (aiocb). Manpage: aio_prio_delta_max(5). Memory Paging allocate_fs_swapmap Enable or disable preallocation of file system swap space when swapon() is called as opposed to allocating swap space when malloc() is called. Enabling allocation reduces risk of insufficient swap space and is used primarily where high availability is important. Manpage: allocate_fs_swapmap(5). Kernel Crash Dump alwaysdump Select which classes of system memory pages are to be dumped if a kernel panic occurs. Manpage: alwaysdump(5). Spinlock Pool bufcache_hash_locks Buffer-cache spinlock pool. NO MANPAGE. File System: Buffer bufpages Number of 4 KB pages in file system static buffer cache. Manpage: bufpages(5). Spinlock Pool chanq_hash_locks Channel queue spinlock pool. Manpage: chanq_hash_locks(5). IPC: Share core_addshmem_read Flag to include readable shared memory in a process core dump. Manpage: core_addshmem_read(5). IPC: Share core_addshmem_write
Flag to include read/write shared memory in a process core dump. Manpage: core_addshmem_write(5). Miscellaneous: Links create_fastlinks Create fast symbolic links using a newer, more efficient format to improve access speed by reducing disk block accesses during path name look-up sequences. Manpage: create_fastlinks(5). File System: Buffer dbc_max_pct Maximum percentage of memory for dynamic buffer cache. Manpage: dbc_max_pct(5). File System: Buffer dbc_min_pct Minimum percentage of memory for dynamic buffer cache. Manpage: dbc_min_pct(5). Miscellaneous: Disk I/O default_disk_ir Immediate reporting for disk writes; whether a write() returns immediately after the data is placed in the disk's write buffer or waits until the data is physically stored on the disk media. Manpage: default_disk_ir(5). File System: Buffer disksort_seconds Maximum wait time for disk requests. NO MANPAGE. Miscellaneous: Disk I/O dma32_pool_size Amount of memory to set aside for 32-bit DMA (bytes). Manpage: dma32_pool_size(5). Spinlock Pool dnlc_hash_locks Number of locks for directory cache synchronization. NO MANPAGE. Kernel Crash Dump dontdump Select which classes of system memory pages are not to be dumped if a kernel panic occurs. Manpage: dontdump(5). Miscellaneous: Clock dst Enable/disable daylight savings time. Manpage: timezone(5). Miscellaneous: IDS enable_idds Flag to enable the IDDS daemon, which gathers data for IDS/9000. Manpage: enable_idds(5). Miscellaneous: Memory eqmemsize Number of pages of memory to be reserved for equivalently mapped memory, used mostly for DMA transfers. Manpage: eqmemsize(5).
ProcessMgmt: Process executable_stack Allows or denies program execution on the stack. Manpage: executable_stack(5). File System: Write fs_async Enable/disable asynchronous writes of file system data structures to disk. Manpage: fs_async(5). Spinlock Pool ftable_hash_locks File table spinlock pool. NO MANPAGE. Spinlock Pool hdlpreg_hash_locks Set the size of the pregion spinlock pool. Manpage: hdlpreg_hash_locks(5). File System: Read hfs_max_ra_blocks The maximum number of read-ahead blocks that the kernel may have outstanding for a single HFS file system. Manpage: hfs_max_ra_blocks(5). File System: Read hfs_max_revra_blocks The maximum number of reverse read-ahead blocks that the kernel may have outstanding for a single HFS file system. Manpage: hfs_max_revra_blocks(5). File System: Read hfs_ra_per_disk The amount of HFS file system read-ahead per disk drive, in KB. Manpage: hfs_ra_per_disk(5). File System: Read hfs_revra_per_disk The amount of memory (in KB) for HFS reverse read-ahead operations, per disk drive. Manpage: hfs_revra_per_disk(5). File System: Read hp_hfs_mtra_enabled Enable or disable HFS multithreaded read-ahead. NO MANPAGE. Kernel Crash Dump initmodmax Maximum size of the dump table of dynamically loaded kernel modules. Manpage: initmodmax(5). Spinlock Pool io_ports_hash_locks I/O port spinlock pool. NO MANPAGE. Miscellaneous: Queue ksi_alloc_max Maximum number of system-wide queued signals that can be allocated. Manpage: ksi_alloc_max(5).
Miscellaneous: Queue ksi_send_max Maximum number of queued signals that a process can send and have pending at one or more receivers. Manpage: ksi_send_max(5). ProcessMgmt: Memory maxdsiz Maximum process data storage segment space that can be used for statics and strings, as well as dynamic data space allocated by sbrk() and malloc() (32-bit processes). Manpage: maxdsiz(5). ProcessMgmt: Memory maxdsiz_64bit Maximum process data storage segment space that can be used for statics and strings, as well as dynamic data space allocated by sbrk() and malloc() (64-bit processes). Manpage: maxdsiz(5). File System: Open/Lock maxfiles Soft limit on how many files a single process can have opened or locked at any given time. Manpage: maxfiles(5). File System: Open/Lock maxfiles_lim Hard limit on how many files a single process can have opened or locked at any given time. Manpage: maxfiles_lim(5). ProcessMgmt: Memory maxrsessiz Maximum size (in bytes) of the RSE stack for any user process on the IPF platform. Manpage: maxrsessiz(5). ProcessMgmt: Memory maxrsessiz_64bit Maximum size (in bytes) of the RSE stack for any user process on the IPF platform. Manpage: maxrsessiz(5). ProcessMgmt: Memory maxssiz Maximum dynamic storage segment (DSS) space used for stack space (32bit processes). Manpage: maxssiz(5). ProcessMgmt: Memory maxssiz_64bit Maximum dynamic storage segment (DSS) space used for stack space (64bit processes). Manpage: maxssiz(5). ProcessMgmt: Memory maxtsiz Maximum allowable process text segment size, used by unchanging executable-code (32-bit processes). Manpage: maxtsiz(5). ProcessMgmt: Memory maxtsiz_64bit Maximum allowable process text segment size, used by unchanging executable-code (64-bit processes). Manpage: maxtsiz(5).
ProcessMgmt: Process maxuprc Maximum number of processes that any single user can have running at the same time, including login shells, user interface processes, running programs and child processes, I/O processes, etc. If a user is using multiple, simultaneous logins under the same login name (user ID) as is common in X Window, CDE, or Motif environments, all processes are combined, even though they may belong to separate process groups. Processes that detach from their parent process group, where that is possible, are not counted after they detach (line printer spooler jobs, certain specialized applications, etc.). Manpage: maxuprc(5). Miscellaneous: Users maxusers Maximum number of users expected to be logged in on the system at one time; used by other system parameters to allocate system resources. Manpage: maxusers(5). File System: LVM maxvgs Maximum number of volume groups configured by the Logical Volume Manager on the system. Manpage: maxvgs(5). Accounting max_acct_file_size Maximum size of the accounting file. Manpage: max_acct_file_size(5). Asynchronous I/O max_async_ports System-wide maximum number of ports to the asynchronous disk I/O driver that processes can have open at any given time. Manpage: max_async_ports(5). Memory Paging max_mem_window Maximum number of group-private 32-bit shared memory windows. Manpage: max_mem_window(5). ProcessMgmt: Threads max_thread_proc Maximum number of threads that any single process can create and have running at the same time. Manpage: max_thread_proc(5). IPC: Message mesg Enable or disable IPC messages at system boot time. Manpage: mesg(5). Kernel Crash Dump modstrmax Maximum size, in bytes, of the savecrash kernel module table that contains module names and their locations in the file system. Manpage: modstrmax(5). IPC: Message msgmap Size of free-space resource map for allocating shared memory space for messages. Manpage: msgmap(5).
IPC: Message msgmax System-wide maximum size (in bytes) for individual messages. Manpage: msgmax(5). IPC: Message msgmnb Maximum combined size (in bytes) of all messages that can be queued simultaneously in a message queue. Manpage: msgmnb(5). IPC: Message msgmni Maximum number of message queues allowed on the system at any given time. Manpage: msgmni(5). IPC: Message msgseg Maximum number of message segments that can exist on the system. Manpage: msgseg(5). IPC: Message msgssz Message segment size in bytes. Manpage: msgssz(5). IPC: Message msgtql Maximum number of messages that can exist on the system at any given time. Manpage: msgtql(5). File System: Buffer nbuf System-wide number of static file system buffer and cache buffer headers. Manpage: nbuf(5). Miscellaneous: CD ncdnode Maximum number of entries in the vnode table and therefore the maximum number of open CD-ROM file system nodes that can be in memory. Manpage: ncdnode(5). Miscellaneous: Terminal nclist Maximum number of cblocks available for data transfers through tty and pty devices. Manpage: nclist(5). File System: Open/Lock ncsize Inode space needed for directory name lookup cache (DNLC). NO MANPAGE. File System: Open/Lock nfile Maximum number of files that can be open simultaneously on the system at any given time. Manpage: nfile(5). File System: Open/Lock nflocks
Maximum combined number of file locks that are available system-wide to all processes at one time. Manpage: nflocks(5). File System: Open/Lock ninode Maximum number of open inodes that can be in memory. Manpage: ninode(5). ProcessMgmt: Threads nkthread Maximum number of kernel threads allowed on the system at the same time. Manpage: nkthread(5). ProcessMgmt: Process nproc Defines the maximum number of processes that can be running simultaneously on the entire system, including remote execution processes initiated by other systems via remsh or other networking commands. Manpage: nproc(5). Miscellaneous: Terminal npty Maximum number of pseudo-tty entries allowed on the system at any one time. Manpage: npty(5). Streams NSTREVENT Maximum number of outstanding streams bufcalls that are allowed to exist at any given time on the system. This number should be equal to or greater than the maximum bufcalls that can be generated by the combined total modules pushed onto any given stream, and serves to limit run-away bufcalls. Manpage: nstrevent(5). Miscellaneous: Terminal nstrpty System-wide maximum number of streams-based pseudo-ttys that are allowed on the system. Manpage: nstrpty(5). Streams nstrpty System-wide maximum number of streams-based pseudo-ttys that are allowed on the system. Manpage: nstrpty(5). Streams NSTRPUSH Maximum number of streams modules that are allowed to exist in any single stream at any one time on the system. This provides a mechanism for preventing a software defect from attempting to push too many modules onto a stream, but it is not intended as adequate protection against malicious use of streams. Manpage: nstrpush(5). Streams NSTRSCHED Maximum number of streams scheduler daemons that are allowed to run at any given time on the system. This value is related to the number of processors installed in the system. Manpage: nstrsched(5).
Miscellaneous: Terminal nstrtel Number of telnet session device files that are available on the system. Manpage: nstrtel(5). Memory Paging nswapdev Maximum number of devices, system-wide, that can be used for device swap. Set to match actual system configuration. Manpage: nswapdev(5). Memory Paging nswapfs Maximum number of mounted file systems, system-wide, that can be used for file system swap. Set to match actual system configuration. Manpage: nswapfs(5). Miscellaneous: Memory nsysmap Number of entries in the kernel dynamic memory virtual address space resource map (32-bit processes). Manpage: nsysmap(5). Miscellaneous: Memory nsysmap64 Number of entries in the kernel dynamic memory virtual address space resource map (64-bit processes). Manpage: nsysmap(5). Miscellaneous: Disk I/O o_sync_is_o_dsync Specifies whether an open() or fcntl() with the O_SYNC flag set can be converted to the same call with the O_DSYNC flag instead. This controls whether the function can return before updating the file access. NO MANPAGE. ProcessMgmt: Memory pa_maxssiz_32bit Maximum size (in bytes) of the stack for a user process running under the PA-RISC emulator on IPF. Manpage: pa_maxssiz(5). ProcessMgmt: Memory pa_maxssiz_64bit Maximum size (in bytes) of the stack for a user process running under the PA-RISC emulator on IPF. Manpage: pa_maxssiz(5). Spinlock Pool pfdat_hash_locks Pfdat spinlock pool. Manpage: pfdat_hash_locks(5). Miscellaneous: Disk I/O physical_io_buffers Total buffers for physical I/O operations. Manpage: physical_io_buffers(5). Spinlock Pool region_hash_locks Process-region spinlock pool. Manpage: region_hash_locks(5). Memory Paging
remote_nfs_swap Enable or disable swap to mounted remote NFS file system. Used on cluster clients for swapping to NFS-mounted server file systems. Manpage: remote_nfs_swap(5). Miscellaneous: Schedule rtsched_numpri Number of distinct real-time interrupt scheduling priority levels are available on the system. Manpage: rtsched_numpri(5). Miscellaneous: Terminal scroll_lines Defines the number of lines that can be scrolled on the internal terminal emulator (ITE) system console. Manpage: scroll_lines(5). File System: SCSI scsi_maxphys Maximum record size for the SCSI I/O subsystem, in bytes. Manpage: scsi_maxphys(5). File System: SCSI scsi_max_qdepth Maximum number of SCSI commands queued up for SCSI devices. Manpage: scsi_max_qdepth(5). ProcessMgmt: Process secure_sid_scripts Controls whether setuid and setgid bits on scripts are honored. Manpage: secure_sid_scripts(5). IPC: Semaphore sema Enable or disable IPC semaphores at system boot time. Manpage: sema(5). IPC: Semaphore semaem Maximum value by which a semaphore can be changed in a semaphore "undo" operation. Manpage: semaem(5). IPC: Semaphore semmni Maximum number of sets of IPC semaphores allowed on the system at any one time. Manpage: semmni(5). IPC: Semaphore semmns Maximum number of individual IPC semaphores available to system users, system-wide. Manpage: semmns(5). IPC: Semaphore semmnu Maximum number of processes that can have undo operations pending on any given IPC semaphore on the system. Manpage: semmnu(5). IPC: Semaphore semmsl
Maximum number of individual System V IPC semaphores per semaphore identifier. Manpage: semmsl(5). IPC: Semaphore semume Maximum number of IPC semaphores that a given process can have undo operations pending on. Manpage: semume(5). IPC: Semaphore semvmx Maximum value any given IPC semaphore is allowed to reach (prevents undetected overflow conditions). Manpage: semvmx(5). Miscellaneous: Web sendfile_max The amount of buffer cache that can be used by the sendfile() system call on HP-UX web servers. Manpage: sendfile_max(5). IPC: Share shmem Enable or disable shared memory at system boot time. Manpage: shmem(5). IPC: Share shmmax Maximum allowable shared memory segment size (in bytes). Manpage: shmmax(5). IPC: Share shmmni Maximum number of shared memory segments allowed on the system at any given time. Manpage: shmmni(5). IPC: Share shmseg Maximum number of shared memory segments that can be attached simultaneously to any given process. Manpage: shmseg(5). Streams STRCTLSZ Maximum number of control bytes allowed in the control portion of any streams message on the system. Manpage: strctlsz(5). Streams streampipes Force all pipes to be streams-based. Manpage: streampipes(5). Streams STRMSGSZ Maximum number of bytes that can be placed in the data portion of any streams message on the system. Manpage: strmsgsz(5). File System: SCSI st_ats_enabled Flag whether to reserve a tape device on open. Manpage: st_ats_enabled(5). File System: SCSI
st_fail_overruns SCSI tape read resulting in data overrun causes failure. Manpage: st_fail_overruns(5). File System: SCSI st_large_recs Enable large record support for SCSI tape. Manpage: st_large_recs(5). Memory Paging swapmem_on Enable or disable pseudo-swap allocation. This allows systems with large installed memory to allocate memory space as well as disk swap space for virtual memory use instead of restricting availability to defined disk swap area. Manpage: swapmem_on(5). Memory Paging swchunk Amount of space allocated for each chunk of swap area. Chunks are allocated from device to device by the kernel. Changing this parameter requires extensive knowledge of system internals. Without such knowledge, do not change this parameter from the normal default value. Manpage: swchunk(5). Spinlock Pool sysv_hash_locks System V interprocess communication spinlock pool. Manpage: sysv_hash_locks(5). Miscellaneous: Network tcphashsz TCP hash table size, in bytes. Manpage: tcphashsz(5). ProcessMgmt: CPU timeslice Maximum time a process can use the CPU until it is made available to the next process having the same process execution priority. This feature also prevents runaway processes from causing system lock-up. Manpage: timeslice(5). Miscellaneous: Clock timezone The offset between the local time zone and Coordinated Universal Time (UTC), often called Greenwich Mean Time or GMT. Manpage: timezone(5). Miscellaneous: Memory unlockable_mem Amount of system memory to be reserved for system overhead and virtual memory management, that cannot be locked by user processes. Manpage: unlockable_mem(5). Spinlock Pool vnode_cd_hash_locks Vnode clean/dirty spinlock pool. NO MANPAGE. Spinlock Pool vnode_hash_locks Vnode spinlock pool. NO MANPAGE.
Memory Paging: Size vps_ceiling Maximum system-selected page size (in KB) if the user does not specify a page size. Manpage: vps_ceiling(5). Memory Paging: Size vps_chatr_ceiling Maximum page size a user can specify with the chatr command in a program. Manpage: vps_chatr_ceiling(5). Memory Paging: Size vps_pagesize Minimum user page size (in KB) if no page size is specified using chatr. Manpage: vps_pagesize(5). File System: Journaled vxfs_max_ra_kbytes Maximum amount of read-ahead data, in KB, that the kernel may have outstanding for a single VxFS file system. Manpage: vxfs_max_ra_kbytes(5). File System: Read vxfs_max_ra_kbytes Maximum amount of read-ahead data, in KB, that the kernel may have outstanding for a single VxFS file system. Manpage: vxfs_max_ra_kbytes(5). File System: Journaled vxfs_ra_per_disk Maximum amount of VxFS file system read-ahead per disk, in KB. Manpage: vxfs_ra_per_disk(5). File System: Read vxfs_ra_per_disk Maximum amount of VxFS file system read-ahead per disk, in KB. Manpage: vxfs_ra_per_disk(5). File System: Journaled vx_fancyra_enable Enable or disable VxFS file system read-ahead. NO MANPAGE. File System: Journaled vx_maxlink Number of subdirectories created within a directory. NO MANPAGE. File System: Journaled vx_ncsize Memory space reserved for VxFS directory path name cache. Manpage: vx_ncsize(5). File System: Journaled vx_ninode Number of entries in the VxFS inode table. NO MANPAGE
Some HP-UX troubleshooting tips: -------------------------------Where to get information about problems: dmesg --> provides a finite list of diagnostic messages /var/adm/syslog/syslog.log --> system log /opt/resmon/log/error.log --> /etc/shutdownlog --> shutdown information /etc/rc.log --> system startup log /var/tombstones/ts99 --> crash analysis file cstm - command line support tool manager mstm - menu based support tool manager --> to move to another portion of the screen, such as the drop down menu area Service Processor from a serial console he - help co - return to console mode (exits the program) sl - show log
Panic Reboots Check these files for clues: /var/tombstones/ts99 /etc/shutdownlog Bad disk 1. Check the syslog (/var/adm/syslog/syslog.log) looking for disk errors. 2. Check the ioscan (ioscan -fnC disk), looking for NO_HW rather than Claimed. 3. If diaglogd is running then check STM logs (/var/opt/resmon/log/event.log) 4. Check the volume group to see if the disk is listed and whether there is any problem with it's status (vgdisplay -v | more) 5. Check lvmtab to see if the disk is supposed to be in a volume group (strings /etc/lvmtab | more) Filesystem do not mount after a reboot 1. Reactivate the Volume Group --> vgchange -a y /dev/ 2. Remount the filesystems --> mount -a 3. If still no success then perform a filesystem check --> fsck /dev// 4. Remount the filesystems --> mount -a 5. Check to see if all the filesystems are there: a) bdf b) compare with /etc/fstab
Filesystem full du -kx / | sort -rn | more du -akx | sort -nr | more Shows directories on the local filesystem and how much space they are taking up NFS mount - Permission Denied 1. Check to see if the format of the /etc/exports file is correct on the server that is the nfs server. 2.
exportfs -av to export the filesystem
3. Check the /etc/fstab file on the client to make sure that it is correct 4. /usr/sbin/showmount -e being exported
on the client to show what is
5. To bypass the /etc/exports file execute the following on the nfs server: exportfs -i -o rw . NFS Server /etc/rc.config.d/nfsconf
--> NFS_SERVER=1
Verify the proper processes are running: /sbin/init.d/nfs.server stop The processes should NOT be running: # # # #
ps ps ps ps
-ef|grep -ef|grep -ef|grep -ef|grep
nfsd rpc.mountd rpc.lockd rpc.statd
/sbin/init.d/nfs.server start These processes should be running: # ps -ef|grep nfsd root 3444 1 0 10:39:12 root 3451 3444 0 10:39:12 root 3449 3444 0 10:39:12 root 3445 3444 0 10:39:12 # ps -ef|grep rpc.mountd root 3485 1 0 10:42:09 # ps -ef|grep rpc.lockd root 3459 1 0 10:39:12 # ps -ef|grep rpc.statd root 3453 1 0 10:39:12
? ? ? ?
0:00 0:00 0:00 0:00
/usr/sbin/nfsd /usr/sbin/nfsd /usr/sbin/nfsd /usr/sbin/nfsd
4 4 4 4
?
0:00 rpc.mountd
?
0:00 /usr/sbin/rpc.lockd
?
0:00 /usr/sbin/rpc.statd
To start a process if it is not running: # ps -ef|grep rpc.mountd # rpc.mountd or /usr/sbin/rpc.mountd # ps -ef|grep rpc.mountd root 3485 1 0 10:42:09 ?
0:00 rpc.mountd
/etc/inetd.conf needs to have the proper services active (not commented out) ## # WARNING: The rpc.mountd should now be started from a startup script. # Please enable the mountd startup script to start rpc.mountd. ## #rpc stream tcp nowait root /usr/sbin/rpc.rexd 100017 1 rpc.rexd rpc dgram udp wait root /usr/lib/netsvc/rstat/rpc.rstatd 100001 2-4 rpc.rstatd rpc dgram udp wait root /usr/lib/netsvc/rusers/rpc.rusersd 100002 1-2 rpc.rusersd rpc dgram udp wait root /usr/etc/rpc.mountd 100005 1 rpc.mountd -e rpc dgram udp wait root /usr/lib/netsvc/rwall/rpc.rwalld 100008 1 rpc.rwalld #rpc dgram udp wait root /usr/sbin/rpc.rquotad 100011 1 rpc.rquotad rpc dgram udp wait root /usr/lib/netsvc/spray/rpc.sprayd 100012 1 rpc.sprayd
NIC problems: The lanadmin utility provides NIC statistics The nettladmin utility provides packet trace information Replacing a Mirrored Root Disk: Replace the disk Hot swap can be performed while system is up Not hot swappable means the system must be brought down Reboot the system into single user mode shutdown -r 0, unless the system is powered off already, then power it back on interrupt the boot bo pri (or bo alt if the disk that was replaced was the primary boot disk) IPL>hpux -is -lq (;0)/stand/vmunix vgcfgrestore -n /dev/vg00 /dev/rdsk/c?t?d? vgsync /dev/vg00 mkboot /dev/rdsk/c?t?d? mkboot -a "hpux -lq (;0)/stand/vmunix" /dev/rdsk/c?t?d? shutdown -r 0 lvlnboot -v /dev/vg00 to verify that the disk is seen as bootable
Software Installation (swinstall, sd, etc) ERROR: "server::/tmp/omni_tmp/packet": You do not have the required permissions to perform this operation. Check permissions using the "swacl" command or see your system administrator for assistance. Or, to manage applications designed and packaged for nonprivileged mode, see the "run_as_superuser" option in the "sd" man page. WARNING: More information may be found in the daemon logfile on this target (default location is server:/var/adm/sw/swagentd.log). Bounce swagentd daemon:
/usr/sbin/swagentd -r
36. Some remarks about VI: ========================== Before you run vi: -----------------If you've connected to a central UCS computer to use vi, first tell that host about your communications software (e.g., NCSA Telnet). At IUB, your software will typically emulate a VT100 terminal. To find out what shell program you use, type: echo $SHELL Then if you use ksh, bash, or sh, type: TERM=vt100; export TERM If you use csh or tcsh, type: set term = vt100 You can automate this task by adding the appropriate command to your default command shell's configuration file. Using vi modes: --------------Vi has three "modes": edit, insert, and colon. - Edit mode (press Esc) Vi enters edit mode by default when it starts up. Edit mode allows you to move the cursor and edit the text buffer. - Insert mode (press i) Insert mode "drops" the cursor at a specific point in the buffer, allowing you to insert text. To enter insert mode, position the cursor where you want to place text and press i.
If you make a typing mistake, press ESC to return to edit mode and then reposition the cursor at the error, and press i to get back to insert mode. - Colon mode (press : with a command) You enter colon mode from edit mode by typing a colon followed by a command. Some useful commands are: :w :w newname :r :r oldname :q! :wq :e filename :e #
Write buffer to the current filename. Write buffer to file newname. Read the current filename into the buffer. Read the file oldname into the buffer. Quit vi without saving buffer. Write buffer to current filename and quit vi. Close current buffer and edit (open) filename. Close current buffer and edit (open) previous file.
Search and Replace: ------------------Replace: Same as with sed, Replace OLD with NEW: ESC, First occurrence on current line: Globally (all) on current line: Between two lines #,#: Every occurrence in file:
:s/OLD/NEW :s/OLD/NEW/g :#,#s/OLD/NEW/g :%s/OLD/NEW/g
The VI editor has two kinds of searches: string and character. For a string search, the / and ? commands are used. When you start these commands, the command just typed will be shown on the bottom line, where you type the particular string to look for. These two commands differ only in the direction where the search takes place. The / command searches forwards (downwards) in the file, while the ? command searches backwards (upwards) in the file. The n and N commands repeat the previous search command in the same or opposite direction, respectively. Some characters have special meanings to VI, so they must be preceded by a backslash (\) to be included as part of the search expression. Searching for Text: ------------------/text
Search forward (down) for text (text can include spaces and characters with special meanings.)
?text
Search backward (up) for text
n
Repeat last search in the same direction
N
Repeat last search in the opposite direction
fchar
Search forward for a charcter on current line
Fchar
Search backward for a character on current line
;
Repeat last character search in the same direction
%
Find matching ( ), { }, or [ ]
Some other: ----------Moving half screens Up or Down: Ctrl-U, Ctrl-D showing line numbers: :set number If you tire of the line numbers, enter the following command to turn them off: :set nonumber
36. ulimit: =========== limit, ulimit, unlimit - set or get limitations on the resources available to the current shell and its descendents. /usr/bin/ulimit Example 1:
system
Limiting the stack size
To limit the stack size to 512 kilobytes: example% ulimit -s 512 example% ulimit -a time(seconds) unlimited file(blocks) 100 data(kbytes) 523256 stack(kbytes) 512 coredump(blocks) 200 nofiles(descriptors) 64 memory(kbytes) unlimited ULIMIT - Sets the file size limit for the login. Units are disk blocks. Default is zero (no limit). Be sure to specify even numbers, as the ULIMIT variable accepts a number of 512-byte blocks. $ ulimit -a
# Display limits for your session under sh or ksh
$ limit # Display limits for your session under csh or tcsh $ ulimit -c SIZE_IN_BLOCKS # Limit core size under sh or ksh $ limit coredumpsize SIZE_IN_KB # Limit core size under csh or tcsh If you see a core file lying around, just type "file core" to get some details about it. Example: $ file core core:ELF-64 core file - PA-RISC 2.0 from 'sqlplus' - received SIGABRT Run the Unix process debugger to obtain more information about where and why the process abended. This information is normally requested by Oracle Support for in-depth analysis of the problem. Some example: Solaris: $ gdb $ORACLE_HOME/bin/sqlplus core bt # backtrace of all stack frames quit HP-UX, Solaris, etc: $ adb $ORACLE_HOME/bin/sqlplus core $c $q Sequent: $ debug -c core $ORACLE_HOME/bin/sqlplus debug> stack debug> quit AIX: Purpose Sets or reports user resource limits. Syntax ulimit [ -H ] [ -S ] [ -a ] [ -c ] [ -d ] [ [ -t ] [ Limit ]
-f ] [ -m ] [ -n ] [ -s ]
Description The ulimit command sets or reports user process resource limits, as defined in the /etc/security/limits file. This file contains these default limits: fsize = 2097151 core = 2097151 cpu = -1 data = 262144 rss = 65536 stack = 65536 nofiles = 2000 These values are used as default settings when a new user is added to the system. The values are set with the
mkuser command when the user is added to the system, or changed with the chuser command. Limits are categorized as either soft or hard. With the ulimit command, you can change your soft limits, up to the maximum set by the hard limits. You must have root user authority to change resource hard limits. Many systems do not contain one or more of these limits. The limit for a specified resource is set when the Limit parameter is specified. The value of the Limit parameter can be a number in the unit specified with each resource, or the value unlimited. To set the specific ulimit to unlimited, use the word unlimited Note: Setting the default limits in the /etc/security/limits file sets system wide limits, not just limits taken on by a user when that user is created. The current resource limit is printed when you omit the Limit parameter. The soft limit is printed unless you specify the -H flag. When you specify more than one resource, the limit name and unit is printed before the value. If no option is given, the -f flag is assumed. Since the ulimit command affects the current shell environment, it is provided as a shell regular built-in command. If this command is called in a separate command execution environment, it does not affect the file size limit of the caller's environment. This would be the case in the following examples: nohup ulimit -f 10000 env ulimit 10000 Once a hard limit has been decreased by a process, it cannot be increased without root privilege, even to revert to the original limit. For more information about user and system resource limits, refer to the getrlimit, setrlimit, or vlimit subroutine in AIX 5L Version 5.2 Technical Reference: Base Operating System and Extensions Volume 1. Flags -a Lists all of the current resource limits. -c Specifies the size of core dumps, in number of 512-byte blocks. -d Specifies the size of the data area, in number of K bytes. -f Sets the file size limit in blocks when the Limit parameter is used, or reports the file size limit if no parameter is specified. The -f flag is the default. -H Specifies that the hard limit for the given resource is set. If you have root user authority, you can increase the hard limit. Anyone can decrease it. -m Specifies the size of physical memory, in number of K bytes.
-n Specifies the limit on the number of file descriptors a process may have. -s Specifies the stack size, in number of K bytes. -S Specifies that the soft limit for the given resource is set. A soft limit can be increased up to the value of the hard limit. If neither the -H nor -S flags are specified, the limit applies to both. -t Specifies the number of seconds to be used by each process. You can check the current ulimit settings using the ulimit -a command, and at least the following three commands should be run, as the user account that will launch Java: ulimit -m unlimited ulimit -d unlimited ulimit -f unlimited
===================================== 37. RAM disks: ===================================== 37.1 AIX: ========= Example: -------# mkramdisk SIZE /dev/rramdiskxx # mkfs -V jfs /dev/ramdiskxx # mount -V jfs -o nointegrity /dev/ramdiskxx /whatever_mountpoint mkramdisk Command: -----------------Purpose Creates a RAM disk using a portion of RAM that is accessed through normal reads and writes. Syntax mkramdisk [ -u ] size[ M | G ] Description The mkramdisk command is shipped as part of bos.rte.filesystems, which allows the user to create a RAM disk. Upon successful execution of the mkramdisk command, a new RAM disk is created, a new entry added to /dev, the name of the new RAM disk is written to standard output, and the command exits with a value of 0.
If the creation of the RAM disk fails, the command prints an internalized error message, and the command will exit with a nonzero value. The size can be specified in terms of MB or GB. By default, it is in 512 byte blocks. A suffix of M will be used to specify size in megabytes and G to specify size in gigabytes. The names of the RAM disks are in the form of /dev/rramdiskx where x is the logical RAM disk number (0 through 63). The mkramdisk command also creates block special device entries (for example, /dev/ramdisk5) although use of the block device interface is discouraged because it adds overhead. The device special files in /dev are owned by root with a mode of 600. However, the mode, owner, and group ID can be changed using normal system commands. Up to 64 RAM disks can be created. Note: The size of a RAM disk cannot be changed after it is created. The mkramdisk command is responsible for generating a major number, loading the ram disk kernel extension, configuring the kernel extension, creating a ram disk, and creating the device special files in /dev. Once the device special files are created, they can be used just like any other device special files through normal open, read, write, and close system calls. RAM disks can be removed by using the rmramdisk command. RAM disks are also removed when the machine is rebooted. By default, RAM disk pages are pinned. Use the -u flag to create RAM disk pages that are not pinned. Flags -u Specifies that the ram disk that is created will not be pinned. By default, the ram disk will be pinned. Parameters size Indicates the amount of RAM (in 512 byte increments) to use for the new RAM disk. For example, typing: # mkramdisk 1 creates a RAM disk that uses 512 bytes of RAM. To create a RAM disk that uses approximately 20 MB of RAM, type: # mkramdisk 40000 Exit Status The following exit values are returned: 0 Successful completion.
>0 An error occurred. Examples: To create a new ram disk using a default 512-byte block size, and the size is 500 MBs (1048576 * 512), enter: # mkramdisk 1048576 /dev/rramdisk0 The /dev/rramdisk0 ramdisk is created. To create a new ramdisk with a size of 500 Megabytes, enter: # mkramdisk 500M /dev/rramdisk0 The /dev/rramdisk0 ramdisk is created. Note that the ramdisk has the same size as example 1 above. To create a new ram disk with a 2-Gigabyte size, enter: # mkramdisk 2G /dev/rramdisk0 To set up a RAM disk that is approximately 20 MB in size and create a JFS file system on that RAM disk, enter the following: # # # # #
mkramdisk 40000 ls -l /dev | grep ram mkfs -V jfs /dev/ramdiskx mkdir /ramdisk0 mount -V jfs -o nointegrity /dev/ramdiskx /ramdiskx
where x is the logical RAM disk number. Note: If using file system on a RAM disk, the RAM disk must be pinned. 37.2 Linux: =========== Redhat: It is very easy to use a ramdisk. First of all, the default installation of RedHat >= 6.0 comes with ramdisk support. All you have to do is format a ramdisk and then mount it to a directory. To find out all the ramdisks you have available, do a "ls -al /dev/ram*". This gives you the preset ramdisks available to your liking. These ramdisks don't actually grab memory until you use them somehow (like formatting them). Here is a very simple example of how to use a ramdisk. # create a mount point:
mkdir /tmp/ramdisk0 # create a filesystem: mke2fs /dev/ram0 # mount the ramdisk: mount /dev/ram0 /tmp/ramdisk0 Those three commands will make a directory for the ramdisk , format the ramdisk (create a filesystem), and mount the ramdisk to the directory "/tmp/ramdisk0". Now you can treat that directory as a pretend partition! Go ahead and use it like any other directory or as any other partition. If the formatting of the ramdisk faild then you might have no support for ramdisk compiled into the Kernel. The Kernel configuration option for ramdisk is CONFIG_BLK_DEV_RAM . The default size of the ramdisk is 4Mb=4096 blocks. You saw what ramdisk size you got while you were running mke2fs. mke2fs /dev/ram0 should have produced a message like this: mke2fs 1.14, 9-Jan-1999 for EXT2 FS 0.5b, 95/08/09 Linux ext2 filesystem format Filesystem label= 1024 inodes, 4096 blocks 204 blocks (4.98%) reserved for the super user First data block=1 Block size=1024 (log=0) Fragment size=1024 (log=0) 1 block group 8192 blocks per group, 8192 fragments per group 1024 inodes per group Running df -k /dev/ram0 tells you how much of that you can really use (The filesystem takes also some space): >df -k /dev/ram0 Filesystem 1k-blocks /dev/ram0 3963
Used Available Use% Mounted on 13 3746 0% /tmp/ramdisk0
What are some catches? Well, when the computer reboots, it gets wiped. Don't put any data there that isn't copied somewhere else. If you make changes to that directory, and you need to keep the changes, figure out some way to back them up. - Changing the size of the ramdisks To use a ram disk you either need to have ramdisk support compiled into the Kernel or you need to compile it as loadable module. The Kernel configuration option is CONFIG_BLK_DEV_RAM . Compiling the ramdisk a loadable module has the advantage that you can decide at load time what the size of your ramdisks should be. Okay, first the hard way. Add this line to your lilo.conf file: ramdisk_size=10000 (or ramdisk=10000 for old kernels)
and it will make the default ramdisks 10 megs after you type the "lilo" command and reboot the computer. Here is an example of my /etc/lilo.conf file. boot=/dev/hda map=/boot/map install=/boot/boot.b prompt timeout=50 image=/boot/vmlinuz label=linux root=/dev/hda2 read-only ramdisk_size=10000 Actually, I got a little over 9 megs of usable space as the filesystem takes also a little space. When you compile ramdisk support as loadable module then you can decide at load time what the size should be. This is done either with an option line in the /etc/conf.modules file: options rd rd_size=10000 or as a command line parameter to ismod: insmod rd rd_size=10000 Here is an example which shows how to use the module: Unmount the ramdisk mounted in the previous chapter, umount /tmp/ramdisk0 . Unload the module (it was automatically loaded in the previous chapter), rmmod rd Load the ramdisk module and set the size to 20Mb, insmod rd rd_size=20000 create a file system, mke2fs /dev/ram0 mount the ramdisk, mount /dev/ram0 /tmp/ramdisk0 - Example of how to use a RamDisk for a webserver. Okay, here is an example of how to use 3 ramdisks for a webserver. Let us say you are 99% confident that your default installation of Apache for RedHat 6.0 won't use more than 9 megs for its cgi-scripts, html, and icons. Here is how to install one. First, issue this command to move the real copy of the document root directory of your webserver to a different place. Also, make the directories to mount the ramdisks . mv /home/httpd/ /home/httpd_real mkdir /home/httpd mkdir /home/httpd/cgi-bin mkdir /home/httpd/html mkdir /home/httpd/icons Then, add these commands to the start procedure in your /etc/rc.d/init.d/httpd.init (or where ever the httpd gets started on your system): ### Make the ramdisk partitions /sbin/mkfs -t ext2 /dev/ram0
/sbin/mkfs -t ext2 /dev/ram1 /sbin/mkfs -t ext2 /dev/ram2 ### Mount the ramdisks to their appropriate places mount /dev/ram0 /home/httpd/cgi-bin mount /dev/ram1 /home/httpd/icons mount /dev/ram2 /home/httpd/html ### Copying real directory to ramdisks (the ### data on the ramdisks is lost after a reboot) tar -C /home/httpd_real -c . | tar -C /home/httpd -x ### After this you can start the web-server.
37.3 Solaris: ============= Note 1: ------Solaris 9 and higher: use the ramdiskadm command: Quick example: Example: Creating a 2MB Ramdisk Named mydisk # ramdiskadm -a mydisk 2m /dev/ramdisk/mydisk Example: Listing All Ramdisks # ramdiskadm Block Device /dev/ramdisk/miniroot /dev/ramdisk/certfs /dev/ramdisk/mydisk
Size 134217728 1048576 2097152
Removable No No Yes
-- The ramdiskadm command: NAME ramdiskadm- administer ramdisk pseudo device SYNOPSIS /usr/sbin/ramdiskadm -a name size [g | m | k | b] /usr/sbin/ramdiskadm -d name /usr/sbin/ramdiskadm DESCRIPTION The ramdiskadm command administers ramdisk(7D), the ramdisk driver. Use ramdiskadm to create a new named ramdisk device, delete an existing named ramdisk, or list information about exisiting ramdisks.
Ramdisks created using ramdiskadm are not persistent across reboots. OPTIONS The following options are supported: -a name size Create a ramdisk named name of size size and its corresponding block and character device nodes. name must be composed only of the characters a-z, A-Z, 0-9, _ (underbar), and - (hyphen), but it must not begin with a hyphen. It must be no more than 32 characters long. Ramdisk names must be unique. The size can be a decimal number, or, when prefixed with 0x, a hexadecimal number, and can specify the size in bytes (no suffix), 512-byte blocks (suffix b), kilobytes (suffix k), megabytes (suffix m) or gigabytes (suffix g). The size of the ramdisk actually created might be larger than that specified, depending on the hardware implementation. If the named ramdisk is successfully created, its block device path is printed on standard out. -d name Delete an existing ramdisk of the name name. This command succeeds only when the named ramdisk is not open. The associated memory is freed and the device nodes are removed. You can delete only ramdisks created using ramdiskadm. It is not possible to delete a ramdisk that was created during the boot process. Without options, ramdiskadm lists any existing ramdisks, their sizes (in decimal), and whether they can be removed by ramdiskadm (see the description of the -d option, above). Note 2: ------thread: In Solaris =< version 8, its a bit of a pain. This is what i asked: Is there anyone who could tell me how to make a ram disk in Solaris 8? I have a Sun Sparc Box running Solaris 8, and I want to use some of it's memory to mount a new file-system Thanks in advance, The solution:
As many mentioned i could use tmpfs, lik this: mkdir /ramdisk mount -F tmpfs -o size=500m swap /ramdisk However this is not a true ramdisk (it really uses VM, not RAM, and the size is an upper limit, not a reservation) This is what Solaris provides.
====================== 38. Software Packages: ====================== 38.1 Software Packages on Solaris: ================================== This section deals about software packages for Solaris. A software package is a collection of files and directories in a defined format. It describes a software application such as manual pages and line printer support. Solaris 8 has about 80 packages that total about 900MB. A Solaris software package is the standard way to deliver bundeld and unbundled software. Packages are administered by using the package administration commands, and are generally identified by a SUNWxxx naming convention. Software packages are grouped into software clusters, which are logical collections of software packages. Some clusters contain just 1 or 2 packages, while another may contain more packages. Installing Software Packages: ----------------------------Solaris provides the tools for adding and removing software from a system. You can use pkgadd command to install packages, and the pkgrm command to remove packages. There are also GUI tools to install and remove packages. Package files are delivered in package format and are unusable as they are delivered. The pkgadd command interprets the software package's control files, and then uncompresses and installs the product files onto the system's local disk. Although the pkgadd and pkgrm commands do not log their output to a standard location, they do keep track of the product that is installed or removed. The pkgadd and pkgrm commands store information about a package that has been installed
or removed in a software product database. By updating this database, the pkgadd and pkgrm commands keep a record of all software products installed on the system. -- pkgadd: -- ------pkgadd [-nv] [-a admin] [-d device] [[-M]-R root_path] [-r response] [V fs_file] [pkginst...] pkgadd -s spool [-d device] [pkginst...] -a admin Define an installation administration file, admin, to be used in place of the default administration file. The token none overrides the use of any admin file, and thus forces interaction with the user. Unless a full path name is given, pkgadd first looks in the current working directory for the administration file. If the specified administration file is not in the current working directory, pkgadd looks in the /var/sadm/install/admin directory for the administration file. -d device Install or copy a package from device. device can be a full path name to a directory or the identifiers for tape, floppy disk, or removable disk (for example, /var/tmp or /floppy/floppy_name ). It can also be a device alias (for example, /floppy/floppy0). pkgadd transfers the contents of a software package from the distribution medium or directory to install it onto the system. Used without the -d option, pkgadd looks in the default spool directory for the package (var/spool//pkg). Used with the -s option, it writes the package to a spool directory instead of installing it. In general you would pkgadd as follows: # pkgadd -a admin-file -d device-name pkgid Or just # pkgadd -d device-name pkgid -a admin-file (Optional) Specifies an administration file that the pkgadd command should consult during the installation. -d device-name Specifies the absolute path to the software packages. device-name can be the path to a device, a directory, or a spool directory.
If you do not specify the path where the package resides, the pkgadd command checks the default spool directory (/var/spool/pkg). If the package is not there, the package installation fails. pkgid (Optional) Is the name of one or more packages (separated by spaces) to be installed. If omitted, the pkgadd command installs all available packages. After installing a package, verify the install with # pkgchk -v pkgid Example 1: following example shows how install the SUNWpl5u package from a mounted Solaris 9 CD. The example also shows how to verify that the package files were installed properly. # pkgadd -d /cdrom/cdrom0/s0/Solaris_9/Product SUNWpl5u . Installation of was successful. # pkgchk -v SUNWpl5u /usr /usr/bin /usr/bin/perl /usr/perl5 /usr/perl5/5.00503 Example 2: # pkgadd -d /cdrom/cdrom0/s0/Solaris_2.6 Example 3: # pkgadd -d /tmp/signed_pppd The following packages are available: 1 SUNWpppd Solaris PPP Device Drivers (sparc) 11.10.0,REV=2003.05.08.12.24 Select package(s) you wish to process (or 'all' to process all packages). (default: all) [?,??,q]: all Enter keystore password: Example 4: # pkgadd -d http://install/signed-video.pkg ## Downloading... ..............25%..............50%..............75%..............100% ## Download Complete Example 5: # pkgadd -d . DISsci The command will create a new directory structure in /opt/DISsci Example 6:
Spooling the packages to a spool directory # pkgadd -d /cdrom/sol_8_sparc/s0/Solaris_8/Product -s /var/spool/pkg SUNWaudio Example 7: Installing Software Packages From a Remote Package Server If the packages you want to install are available from a remote system, you can manually mount the directory that contains the packages (in package format) and install packages on the local system. The following example shows how install software packages from a remote system. In this example, assume that the remote system named package-server has software packages in the /latest-packages directory. The mount command mounts the packages locally on /mnt, and the pkgadd command installs the SUNWpl5u package. # mount -F nfs -o ro package-server:/latest-packages /mnt # pkgadd -d /mnt SUNWpl5u . Installation of was successful.
Other package related commands: -------------------------------
pkgrm pkgchk pkginfo pkgask pkgparam
Displays a package parameter values. # pkgparam -d /cdrom/cdrom0/s0/Solaris_2.8/Product SUNWvolr SUNW_PKGTYPE The system responds with the location where the application will be stored. Using a Response File: ---------------------A response file contains your answers to specific questions that are asked by an interactive package. An interactive package includes a request script that asks you questions prior to package installation, such as whether or not optional pieces of the package should be installed. If prior to installation, you know that the package you want to install is an interactive package, and you want to store your answers to prevent user interaction during future installations of this package, you can use the pkgask command to save your response. Once you have stored your responses to the questions asked by the request script, you can use the pkgadd -r command
to install the package without user interaction. -- pkginfo -- ------# pkginfo system system system system . . etc..
SUNWaccr SUNWaccu SUNWadmap SUNWadmc
System System System System
Accounting, (Root) Accounting, (Usr) administration applications administration core libraries
Example-Displaying Detailed Information About Software Packages # pkginfo -l SUNWcar PKGINST: NAME: CATEGORY: ARCH: VERSION: BASEDIR: VENDOR: DESC: PSTAMP: INSTDATE: HOTLINE: STATUS: FILES:
SUNWcar Core Architecture, (Root) system sparc.sun4u 11.9.0,REV=2001.10.16.17.05 / Sun Microsystems, Inc. core software for a specific hardware platform group crash20011016171723 Nov 02 2001 08:53 Please contact your local service provider completely installed 111 installed pathnames 36 shared pathnames 40 directories 56 executables 17626 blocks used (approx)
# pkginfo -d /export/host1/packages -l SUNWman For the spool directory, you may use the token spool. -- pkgrm: -- -----Always use the pkgrm command to remove installed packages. Do not use the rm command, which will corrupt the system's record-keeping of installed packages. Examples: # pkgrm pkgid ... pkgid identifies the name of one or more packages (separated by spaces) to be removed. If omitted, pkgrm removes all available packages.
# pkgrm SUNWctu The following package is currently installed: SUNWctu Netra ct usr/platform links (64-bit) (sparc.sun4u) 11.9.0,REV=2001.07.24.15.53 Do you want to remove this package? y ## ## ## ##
Removing installed package instance Verifying package dependencies. Processing package information. Removing pathnames in class
This example shows how to remove a spooled package. # pkgrm -s /export/pkg SUNWdmfex.u The following package is currently spooled: SUNWdmfex.u Sun Davicom 10/100Mb Ethernet Driver (64-bit) (sparc.sun4u) 11.9.0,REV=2001.07.24.15.53 Do you want to remove this package? y Removing spooled package instance Some Graphical tools for installing packages: -------------------------------------------->>> admintool (Solaris 8,9 Not in Solaris 10) >>> Solaris Product Registry The Solaris Product Registry is a GUI tool that enables you to install and uninstall software packages. To startup the Solaris Product Registry to view, install or uninstall software, use the command /usr/bin/prodreg >>> Solaris Management Console (smc) Patch Manager The Solaris Management Console provides a new Patches Tool for managing patches. You can only use the Patches Tool to add patches to a system running the Solaris 9 or later release.
Installing Patches: ------------------#patchadd #patchrm
patchadd [-d] [-u] [-B backout_dir] [-C net_install_image| -R client_root_path| -S service] patch patchadd [-d] [-u] [-B backout_dir] [-C net_install_image| -R client_root_path| -S service] -M patch_dir| patch_id... | patch_dir patch_list patchadd [-C net_install_image| -R client_root_path| -S service] -p Examples: Example 1: Show the patches on your system: # showrev -p shows all patches applied to a system # patchadd -p same as above # pkgparam PATCHLIST shows all patches applied to the package identified by Example 2: # patchadd /var/spool/patch/104945-02 # patchadd -R /export/root/client1 /var/spool/patch/104945-02 # patchadd -M /var/spool/patch 104945-02 104946-02 102345-02 # patchadd -M /var/spool/patch patchlist # patchadd -M /var/spool/patch -R /export/root/client1 -B /export/backoutrepository 104945-02 104946-02 102345-02 The /var/sadm/install/contents file: -----------------------------------The /var/sadm/install/contents file is the file which Solaris uses to keep track of all the files installed on a system, and their corresponding packages. Every file installed on a Solaris OS using the pkgadd command has an entry in the database of installed files /var/sadm/install/contents. The contents is a textfile that contains one line per installed file.
38.2 Software Packages on AIX: ============================== Installing software, filesets, packages, lpp: --------------------------------------------Similar to Solaris, AIX5L also has a specific terminology related to installable software. There are 4 basic package concepts in AIX5L: fileset, package, LPP, and bundle. - Fileset: A fileset is the smallest individually installable unit. It's a collection of files that provide a specific function. For example, the "bos.net.tcp.client" is a fileset in the "bos.net" package. - Package:
A package contains a group of filesets with a common function, This is a single installable image, for example "bos.net". - LPP: This is a complete software product collection, including all the packages and filesets required. LPP's are separately orderable products that will run on the AIX operating system, for example BOS, DB2, CICS, ADSM and so on. -- AIX verifying correct installation: # lppchk # lppchk -v # lppchk -l
Fileset version consistency check File link verification
P521:/apps $lppchk -l lppchk: No link found from /etc/security/mkuser.sys to /usr/lib/security/mkuser.sys. lppchk: No link found from /etc/security/mkuser.default to /usr/lib/security/mkuser.default.
-- AIX installing maintenance levels and fixes: 1. download the fix from IBM website http://techsupport.services.ibm.com/server/support?view=pSeries 2. uncompress and untar the software archive 3. type smitty update_all Install a fix with instfix: --------------------------P521:/apps $instfix Usage: instfix [-T [-M platform]] [-s string] [ -k keyword | -f file ] [-d device] [-S] [-p | [-i [-c] [-q] [-t type] [-v] [-F]]] [-a] Function: Installs or queries filesets associated with keywords or fixes. -a Display the symptom text (can be combined with -i, -k, or -f). keyword and level),
-c Colon-separated output for use with -i. Output includes name, fileset name, required level, installed level, status, abstract.
Status values are < (down level), = (correct
+ (superseded), and ! (not installed). -d Input device (required for all but -i and -a).
-F Returns failure unless all filesets associated with the fix are installed. -f Input file containing keywords or fixes. Use '-' for standard input. The -T option produces a suitable input file format for -f. -i Use with -k or -f option to display whether specified fixes or keywords are installed. Installation is not attempted. If neither -k nor -f is specified, all known fixes are displayed. -k Install filesets for a keyword or fix. -M Use with -T option to display information for fixes present on the media that have to do with the platform specified. -p Use with -k or -f to print filesets associated with keywords. Installation is not attempted when -p is used. -q Quiet option for use with -i. If -c is specified, no heading is displayed. Otherwise, no output is displayed. -S Suppress multi-volume processing. -s Search for and display fixes on media containing a specified string. -T Display fix information for complete fixes present on the media. -t Use with -i option to limit search to a given type. Currently valid types are 'f' (fix) and 'p' (preventive maintenance). -v Verbose option for use with -i. Gives information about each fileset associated with a fix or keyword. to the environment provided.
Another option is to use the instfix command. Any fix can have a single fileset or multiple filesets that comprise that fix. Fix information is organized in the Table of Contents (TOC) on the installation media. After a fix is installed, fix information is kept on the system in a fix database. instfix [ -T ] [ -s String ] [ -S ] [ -k Keyword | -f File ] [ -p ] [ -d Device ] [ -i [ -c ] [ -q ] [ -t Type ] [ -v ] [ -F ] ] [ -a ] Examples: - If you want to install only a specific fix, use # instfix -k -d , for example # instfix -k IX75893 -d /dev/cd0 # instfix -k IX75893 -d . # instfix -k IY63533 -d . - To list fixes that are on a CD-ROM in /dev/cd0, enter # instfix -T -d /dev/cd0 IX75893 - To determine if for example APAR IX75893 is installed on the system, enter
# instfix -ik IX75893 Not all filesets for IX75893 were found. You will always be able to determine if an APAR is installed on your system using the command instfix -ivk APAR_NUMBER , whereas installed PTFs are not trackable. - How to determine if all filesets of a ML are installed? P521:/apps $instfix -i | grep ML All filesets for 5.2.0.0_AIX_ML All filesets for 5200-01_AIX_ML All filesets for 5200-02_AIX_ML All filesets for 5200-03_AIX_ML All filesets for 5200-04_AIX_ML All filesets for 5200-05_AIX_ML All filesets for 5200-06_AIX_ML All filesets for 5200-07_AIX_ML All filesets for 5200-08_AIX_ML All filesets for 5200-09_AIX_ML
were were were were were were were were were were
found. found. found. found. found. found. found. found. found. found.
The command "instfix -i | grep ML" is essentially the same as "instfix -i -tp". - To detect incomplete AIX maintaince levels: # instfix -i |grep ML Not all filesets for 4.3.1.0_AIX_ML were found. Not all filesets for 4.3.2.0_AIX_ML were found. All filesets for 4.3.1.0_AIX_ML were found. Not all filesets for 4.3.2.0_AIX_ML were found. Not all filesets for 4.3.3.0_AIX_ML were found. Not all filesets for 4330-02_AIX_ML were found. All filesets for 4320-02_AIX_ML were found. Not all filesets for 4330-03_AIX_ML were found. .. .. You can also use smitty: # smitty instfix Update Software by Fix (APAR) Type or select a value for the entry field. Press Enter AFTER making all desired changes. * INPUT device / directory for software +
The lslpp command: ------------------
[]
[Entry Fields]
Purpose Lists installed software products. Syntax lslpp { -d | -E | -f | -h | -i | -l | -L | -p } ] [ -a] [ -c] [ -J ] [ -q ] [ -I ] [ -O { [ r ] [ s ] [ u ] } ] [ [ FilesetName ... | FixID ... | all ] lslpp -w [ -c ] [ -q ] [ -O { [ r ] [ s ] [ u ] } ] [ FileName ... | all ] lslpp -L -c [ -v] lslpp -S [A|O] lslpp -e Description The lslpp command displays information about installed filesets or fileset updates. The FilesetName parameter is the name of a software product. The FixID (also known as PTF or program temporary fix ID) parameter specifies the identifier of an update to a formatted fileset. When only the -l (lowercase L) flag is entered, the lslpp command displays the latest installed level of the fileset specified for formatted filesets. The base level fileset is displayed for formatted filesets. When the -a flag is entered along with the -l flag, the lslpp command displays information about all installed filesets for the FilesetName specified. The -I (uppercase i) flag combined with the -l (lowercase L) flag specifies that the output from the lslpp command should be limited to base level filesets.
-a Displays additional ("all") information when combined with other flags. (Not valid with -f, only valid with -B when combined with -h) -B Permits PTF ID input. (Not valid with -L) -c Colon-separated output. (Includes all deinstallable levels of software if -Lc) -d Dependents (filesets for which this is a requisite). -E License Agreements. -S Lists Automatically and Optionally installed filesets. -e Lists all efixes on the system. -f Files that belong to this fileset. -h History information.
-I Limits listings to base level filesets (no updates displayed). -i Product Identification information (requested per fileset). -J Use list as the output format. (Valid with -l and -L) -L Lists fileset names, latest level, states, and descriptions. (Consolidates usr, root and share part information.) -l Lists fileset names, latest level, states, and descriptions. (Separates usr, root and share part information.) -O Data comes from [r] root and/or [s] share and/or [u] usr. (Not valid with -L) -p Requisites of installed filesets. -q Quiet (no column headers). -v Lists additional information from vendor database. (Valid with -Lc only) -w Lists the fileset that owns this file. One of the following mutually exclusive flags: d,f,h,i,L,l,p,w,E,S,e must be specified. P521:/apps $ To display information about installed filesets, you can use the lslpp command. If you need to check whether certain filesets have been installed, use the lslpp command as in the following example: # lslpp -h bos.adt.include bos.adt.l1b bos.adt.l1bm \ bos.net.ncs 1for_ls.compat 1for_ls.base In the above example, we check whether those filesets have been installed. lslpp options: -l: -h: -p: -d: -f: the -w:
Displays the name, level, state and description of the fileset. Displays the installation and update history for the fileset. Displays requisite information for the fileset. Displays dependent information for the fileset. Displays the filenames added to the system during installation of fileset. Lists the fileset that owns a file or files.
Examples: - To display the name, level of the bos.adt.include fileset, use zd57l09 # lslpp -l bos.adt.include Fileset Level State Description --------------------------------------------------------------------------Path: /usr/lib/objrepos bos.adt.include 5.2.0.95 COMMITTED Base Application Development
Include Files - To display all files in the inventory database which include vmstat, use # lslpp -w "*vmstat*" File Fileset Type --------------------------------------------------------------------------/usr/sbin/lvmstat bos.rte.lvm File /usr/share/man/info/EN_US/a_doc_lib/cmds/aixcmds6/vmstat.htm infocenter.man.EN_US.commands File /usr/share/man/info/EN_US/a_doc_lib/cmds/aixcmds3/lvmstat.htm infocenter.man.EN_US.commands File /usr/bin/vmstat bos.acct File /usr/bin/vmstat64 bos.acct File /usr/es/sbin/cluster/OEM/VxVM40/cllsvxvmstat cluster.es.server.utils File The same for trying to find out what contains the make command: # lslpp -w "*make*" /usr/bin/makedev /usr/ccs/bin/make /usr/bin/make Symlink /usr/bin/makekey Symlink /usr/ccs/bin/makekey
bos.txt.tfs bos.adt.base bos.adt.base
File File
bos.adt.base bos.adt.base
File
- To list the installation state for the most recent level of installed filesets for all of the bos.rte filesets, use # lslpp -l "bos.rte.*" # lslpp -l | grep bos.rte So, "lslpp -l" shows all of the filesets - To display the names of the files added to the system during installation of the bos.perf.perfstat fileset, use # lslpp -f "*perf*" - To check whether some certain filesets have been installed, like in the following example: # lslpp -h bos.adt.include bos.adt.lib bos.adt.l1bm \ bos.net.ncs 1for_ls.compat 1for_ls.base - To check you have the SDD driver on your system: # lslpp -L devices.sdd.* - To check the Java filesets on your system: # lslpp -l | grep Java
/root:>lslpp -l | grep Java Java131.rte.bin 1.3.1.16 Environment Java131.rte.lib 1.3.1.16 Environment
COMMITTED
Java Runtime
COMMITTED
Java Runtime Java-based build tool. JavaBeans(TM)
(EJB(TM)).
Javadocs Java(TM) technology-
based Web
Java(TM) technology-
based Web idebug.rte.hpj Runtime idebug.rte.jre Environment idebug.rte.olt.Java
9.2.5.0
COMMITTED
Javadocs High-Performance Java
9.2.5.0
COMMITTED
Java Runtime
9.2.5.0
COMMITTED
Object Level Trace Java
# lslpp -l | grep Java13_64 # lslpp -l | grep App
Application Server
Dynamic
WebSphere Application
Server.
for WebSphere
Application
the WebSphere
Application and X11.adt.bitmaps X11.adt.ext X11.adt.imake X11.adt.include X11.adt.lib X11.adt.motif X11.apps.aixterm Application X11.apps.clients Applications X11.apps.msmit Application
Application Profile, 5.2.0.0 5.2.0.30 5.2.0.0 5.2.0.10 5.2.0.40 5.2.0.0 5.2.0.30
COMMITTED COMMITTED COMMITTED COMMITTED COMMITTED COMMITTED COMMITTED
AIXwindows AIXwindows AIXwindows AIXwindows AIXwindows AIXwindows AIXwindows
5.2.0.0
COMMITTED
AIXwindows Client
5.2.0.50
COMMITTED
Applications AIXwindows msmit Configuration
Applications X11.apps.xdm Application X11.apps.xterm Application Application
Application Application Application Application Application Application aixterm
5.2.0.40
COMMITTED
Applications AIXwindows xdm
5.2.0.0
COMMITTED
AIXwindows xterm
5.2.0.0
COMMITTED
AIXwindows Client
X11.msg.en_US.apps.config Application bos.adt.base Development bos.adt.debug Development bos.adt.graphics Development bos.adt.include Development bos.adt.lib Development bos.adt.libm Development bos.adt.sccs Development bos.adt.syscalls Application bos.adt.utils Development bos.net.tcp.adt Toolkit
COMMITTED
AIXwindows Config
5.2.0.50
COMMITTED
Base Application
5.2.0.50
COMMITTED
Base Application
5.2.0.40
COMMITTED
Base Application
5.2.0.53
COMMITTED
Base Application
5.2.0.50
COMMITTED
Base Application
5.2.0.50
COMMITTED
Base Application
5.2.0.0
COMMITTED
SCCS Application
5.2.0.50
COMMITTED
System Calls
5.2.0.50
COMMITTED
Base Application
5.2.0.40
COMMITTED
TCP/IP Application
6.0.0.0 5.2.0.0
COMMITTED COMMITTED
Application Runtime C Set ++ Application Base Application
xlC.adt.include bos.adt.data Development
5.2.0.0
Removing a fix: --------------On AIX you can use either the installp -r command, or use the smitty reject fast path Smitty fastpaths: ------------------ AIX software maintenance: # smitty maintain_software From here you can commit or reject installed software. You can also copy the filesets from the installation media to a directory on disk. The default directory for doing this is /usr/sys/inst.images -- Install new software: # smitty install_update # smitty install_latest -- To commit software: # smitty install_commit -- To reject software:
# smitty install_reject -- To remove installed and commited software: # smitty install_remove -- To see what fixes are installed on your system: # smitty show_apar_stat -- To install individual fix: # smitty instfix or # smitty update_by_fix -- To install all filesets: # smitty update_all -- To view already installed software: # smitty list_installed
The AIX installp command: ------------------------installp Command Purpose Installs available software products in a compatible installation package. Syntax To Install with Apply Only or with Apply and Commit installp [ -a | -ac [ -N ] ] [ -eLogFile ] [ -V Number ] [ -dDevice ] [ -b ] [ -S ] [ -B ] [ -D ] [ -I ] [ -p ] [ -Q ] [ -q ] [ -v ] [ -X ] [ -F | -g ] [ -O { [ r ] [ s ] [ u ] } ] [ -tSaveDirectory ] [ -w ] [ -zBlockSize ] { FilesetName [ Level ]... | -f ListFile | all } To Commit Applied Updates installp -c [ -eLogFile ] [ -VNumber ] [ -b ] [ -g ] [ -p ] [ -v ] [ -X ] [ -O { [ r ] [ s ] [ u ] } ] [ -w ] { FilesetName [ Level ]... | -f ListFile | all } To Reject Applied Updates installp -r [ -eLogFile ] [ -VNumber ] [ -b ] [ -g ] [ -p ] [ -v ] [ -X ] [ -O { [ r ] [ s ] [ u ] } ] [ -w ] { FilesetName [ Level ]... | -f ListFile } To Deinstall (Remove) Installed Software installp -u [ -eLogFile ] [ -VNumber ] [ -b ] [ -g ] [ -p ] [ -v ] [ -X ] [ -O { [ r ] [ s ] [ u ] } ] [ -w ] { FilesetName [ Level ]... | -f ListFile } To Clean Up a Failed Installation: installp -C [ -b ] [ -eLogFile ] To List All Installable Software on Media installp { -l | -L } [ -eLogFile ] [ -d Device ] [ -B ] [ -I ] [ -q ] [ -zBlockSize ] [ -O { [ s ] [ u ] } ]
To List All Customer-Reported Problems Fixed with Software or Display All Supplemental Information installp { -A|-i } [ -eLogFile ] [ -dDevice ] [ -B ] [ -I ] [ -q ] [ -z BlockSize ] [ -O { [ s ] [ u ] } ] { FilesetName [ Level ]... | -f ListFile | all } To List Installed Updates That Are Applied But Not Committed installp -s [ -eLogFile ] [ -O { [ r ] [ s ] [ u ] } ] [ -w ] { FilesetName [ Level ]... | -fListFile | all } fileset is the lowest installable base unit. For example, bos.net.tcp.client 4.1.0.0 is a fileset. A fileset update is an update with a different fix ID or maintenance level. For example, bos.net.tcp.client 4.1.0.2 and bos.net.tcp.client 4.1.1.0 are both fileset updates for bos.net.tcp.client 4.1.0.0. When a base level (fileset) is installed on the system, it is automatically committed. You can remove a fileset regardless of the state (committed, broken, committed with applied updates, committed with committed updates, etc.). When a fileset update is applied to the system, the update is installed. The current version of that software, at the time of the installation, is saved in a special save directory on the disk so that later you can return to that version if desired. Once a new version of a software product has been applied to the system, that version becomes the currently active version of the software. Updates that have been applied to the system can be either committed or rejected at a later time. The installp -s command can be used to get a list of applied updates that can be committed or rejected. When updates are committed with the -c flag, the user is making a commitment to that version of the software product, and the saved files from all previous versions of the software product are removed from the system, thereby making it impossible to return to a previous version of the software product. Software can be committed at the time of installation by using the -ac flags. Note that committing already applied updates does not change the currently active version of a software product. It merely removes saved files for previous versions of the software product. Examples: To install all filesets within the bos.net software package in /usr/sys/inst.images directory in the applied state, enter # installp -avX -d/usr/sys/inst.images bos.net
To commit all updates, enter # installp -cgX all To list the software that is on your CDROM, enter # installp -L -d /dev/cd0 A record of the installp output can be found in the /var/adm/sw/installp.summary # cat /var/adm/sw/installp.summary Used to cleanup after a failed lpp install/update: # installp -C Commits all applied LPPs or PTFs: # installp -c -g -X all Lists the table of contents for the install/update media and saves it into a file named /tmp/toc.list # installp -q -d/dev/rmt1.1 -l > /tmp/toc.list Lists the lpps that have been applied but not yet committed or rejected: # installp -s [P521]root@ol116u106:installp -s 0503-459 installp: No filesets were found in the Software Vital Product Database in the APPLIED state.
The AIX geninstall command: --------------------------A generic installer that installs software products of various packaging formats. For example, installp, RPM, and ISMP. With the geninstall command, you can list and install packages from media that contains installation images packaged in any of the listed formats. The geninstall and gencopy commands recognize the non-installp installation formats and either call the appropriate installers or copy the images, respectively. Beginning in AIX 5L, you can not only install installp formatted packages, but also RPM and Install Shield Mutli-Platform (ISMP) formatted packages. Use the Webbased System Manager, SMIT, or the geninstall command to install and uninstall these types of packages. The geninstall command is designed to detect the format type of a specified package and run the appropriate install command.
Syntax geninstall -d Media [ -I installpFlags ] [ -E | -T ] [ -t ResponseFileLocation ] [-e LogFile] [ -p ] [ -F ] [ -Y ] [ -Z ] [ -D ] { -f File | Install_List ] | all} OR geninstall -u [-e LogFile] [ -E | -T ] [ -t ResponseFileLocation ] [ -D ] {-f File | Uninstall_List...} OR geninstall -L -d Media [-e LogFile] [ -D ] Description Accepts all current installp flags and passes them on to installp. Some flags (for example, -L) are overloaded to mean list all products on the media. Flags that don't make sense for ISMP packaged products are ignored. This allows programs (like NIM) to continue to always send in installp flags to geninstall, but only the flags that make sense are used. The geninstall command provides an easy way to see what modifications have been made to the configuration files listed in /etc/check_config.files. When these files have been changed during a geninstall installation or update operation, the differences between the old and new files will be recorded in the /var/adm/ras/config.diff. If /etc/check_config.files requests that the old file be saved, the old file can be found in the /var/adm/config directory. The /etc/check_config.files file can be edited and can be used to specify whether old configuration files that have been changed should be saved (indicated by s) or deleted (indicated by d), and has the following format: d /etc/inittab A summary of the geninstall command's install activity is kept at /var/adm/sw/geninstall.summary. This file contains colon-separated lists of filesets installed by installp and components installed by ISMP. This is used mainly to provide summary information for silent installs. Note: Refer to the README.ISMP file in the /usr/lpp/bos directory to learn more about ISMP-packaged installations and using response files. Examples: - To install all the products on a CD media that is in drive cd0, type:
# geninstall -d /dev/cd0 all If ISMP images are present on the media, a graphical interface is presented. Any installp or RPM images are installed without prompting, unless the installp images are spread out over multiple CDs. - If you using the geninstall command to install RPM or ISMP packages, use the prefix type to designate to the geninstall command the type of package you are installing. In AIX 5L, the package prefix types are the following: I: installp format R: RPM format J: ISMP format For example, to install the cdrecord RPM package and the bos.games installp package, type the following: # geninstall -d/dev/cd0 R:cdrecord I:bos.games The geninstall command detects that the cdrecord package is an RPM package type and runs the rpm command to install cdrecord. The geninstall command then detects that bos.games is an installp package type and runs the installp command to install bos.games. The process for uninstallation is similar to the installation process.
Fixdist: -------There is a tool named fixdist you can use to download fixes from IBM. Maintenance levels: =================== Notes: Note 1: ------Current versions of AIX5L are 5200-04, 05, 06, 07 04: V5.2 with the 5200-04 Recommended Maintenance Package APAR IY56722 plus APAR IY60347 £ 05: V5.2 with the 5200-05 Recommended Maintenance Package Note 2: Go from 5200-00 to 5200-05:
----------------------------------Use this package to update to 5200-05 (ML 05) an AIX 5.2.0 system whose current ML is 5200-00 (i.e. base level) or higher. (Nota: ML 05 notably brings the fileset bos.mp.5.2.0.54) AIX 5200-05 maintenance package: AIX 5200-05 maintenance package Recommended maintenance for AIX 5.2.0 This package, 5200-05, updates AIX 5.2 from base level (no maintenance level) to maintenance level 05 (5200-05). This package is a recommended maintenance package for AIX 5.2. IBM recommends that customers install the latest available maintenace package for their AIX release. To determine if AIX 5200-05 is already installed on your system, run the following command: oslevel -r General description This package contains code corrections for the AIX operating system and many related subsystems. Unless otherwise stated, this package is released for all languages. For additional information, refer to the Package information Download and install instructions Package Released Size (Bytes) Checksum 520005.tar.gz (See Note) 01/20/05 750,314,420 2116147779 Additional space needed to extract the filesets 1,034,141,696 Note: IBM recommends that you create a separate file system for /usr/sys/inst.images to prevent the expansion of the /usr file system. More information Click on the package name above. Put the package (a tar.gz file) in /usr/sys/inst.images Extract the filesets from the package. cd /usr/sys/inst.images gzip -d -c 520005.tar.gz | tar -xvf Back up your system. Install the package by creating a table of contents for install to use. Then update the install subsystem itself. Run SMIT to complete the installation. # inutoc /usr/sys/inst.images # installp -acgXd /usr/sys/inst.images bos.rte.install # smit update_all Reboot your system. This maintenance package replaces critical operating system code.
Installation Tips * You will need to be logged in as 'root' to perform the installation of this package. * Creating a system backup is recommended before starting the installation procedure. Refer to the mksysb command in the AIX 5.2 Commands Reference manual for additional information. * The latest AIX 5.2 installation hints and tips are available from the eServer Subscription Services web site at: https://techsupport.services.ibm.com/server/pseries.subscriptionSvcs These tips contain important information that should be reviewed before installing this update. Installation To install selected updates from this package, use the command: smit update_by_fix To install all updates from this package that apply to installed filesets on your system, use the command: smit update_all It is highly recommended that you apply all updates from this package. After successful installation, a system reboot is required for this update to take effect. Note 2: Go from 5200-04 to 5200-05: ----------------------------------AIX 5200(04)-05 maintenance package Recommended maintenance for AIX 5.2.0 This package, 5200(04)-05, updates AIX 5.2 from maintenance level 04 (5200-04) to maintenance level 05 (5200-05). This package is a recommended maintenance package for AIX 5.2. IBM recommends that customers install the latest available maintenace package for their AIX release. To determine if AIX 5200-05 is already installed on your system, run the following command: oslevel -r General description
This package many related this package refer to the
contains code corrections for the AIX operating system and subsystems. Unless otherwise stated, is released for all languages. For additional information, Package information
Download and install instructions Package Released Size (Bytes) Checksum 520405.tar.gz (See Note) 01/20/05 637,751,943 3712904912 Additional space needed to extract the filesets 856,494,080 Note: IBM recommends that you create a separate file system for /usr/sys/inst.images to prevent the expansion of the /usr file system. More information Click on the package name above. Put the package (a tar.gz file) in /usr/sys/inst.images Extract the filesets from the package. cd /usr/sys/inst.images gzip -d -c 520405.tar.gz | tar -xvf Back up your system. Install the package by creating a table of contents for install to use. Then update the install subsystem itself. Run SMIT to complete the installation. # inutoc /usr/sys/inst.images # installp -acgXd /usr/sys/inst.images bos.rte.install # smit update_all Reboot your system. This maintenance package replaces critical operating system code. Note 3: Go from 5200-05 to 5200-07: ----------------------------------Always run the inutoc command to ensure the installation subsystem will recognize the new fix packages you download. This command creates a new .toc file for the fix package. Run the inutoc command in the same directory where you downloaded the package filesets. For example, if you downloaded the filesets to /usr/sys/inst.images, run the following command: # inutoc /usr/sys/inst.images - For selected updates To install selected updates from this package, use the following command: # smit update_by_fix - For all updates
To install all updates from this package that apply to the installed filesets on your system, use the following command: # smit update_all It is highly recommended that you apply all updates from this package. Reboot the system. A reboot is required for this update to take effect. -First do the bos.rte.install # installp -acgYqXd /software/ML07 bos.rte.install # inutoc /software/ML07 # smitty update_all
Note 4: About the /usr/sys/inst.images fs: -----------------------------------------Create a LV # crfs -v jfs -a bf=true -dXXX##instlv -m/usr/sys/inst.images -Ayes -prw -tno -a nbpi=4096 -a ag=64 # mount /usr/sys/inst.images
Note 5: About the inutoc command: --------------------------------inutoc Command Purpose Creates a .toc file for directories that have backup format file install images. This command is used by the installp command and the install scripts. Syntax inutoc [ Directory ] Description The inutoc command creates the .toc file in Directory. If a .toc file already exists, it is recreated with new information. The default installation image Directory is /usr/sys/inst.images. The inutoc command adds table of contents entries in the .toc file for every installation image in Directory.
The installp command and the bffcreate command call this command automatically upon the creation or use of an installation image in a directory without a .toc file. Examples To create the .toc file for the /usr/sys/inst.images directory, enter: # inutoc To create a .toc file for the /tmp/images directory, enter: # inutoc /tmp/images Note 6: About the bffcreate command: -----------------------------------bffcreate Command Purpose Creates installation image files in backup format. Syntax bffcreate [ -q ] [ -S ] [ -U ] [ -v ] [ -X ] [ -d Device ] [ -t SaveDir ] [ -w Directory ] [ -M Platform ] { [ -l | -L ] | -c [ -s LogFile ] | Package [Level ] ... | -f ListFile | all } Description The bffcreate command creates an installation image file in backup file format (bff) to support software installation operations. The bffcreate command creates an installation image file from an installation image file on the specified installation media. Also, it automatically creates an installation image file from hyptertext images (such as those on the operating system documentation CD-ROMs). The installp command can use the newly created installation file to install software onto the system. The file is created in backup format and saved to the directory specified by SaveDir. The .toc file in the directory specified by the SaveDir parameter is updated to include an entry for the image file. The bffcreate command determines the bff name according to this information: Neutral Packages POWER-based platform
package.v.r.m.f.platform.installtype Packages package.v.r.m.f.installtype
Image Type Installation image for the POWER-based platform package.v.r.m.f.I Installation image for Neutral package.v.r.m.f.N.I 3.1 update for the POWER-based platform package.v.r.m.f.service#
Target bff Name
3.2 update for the POWER-based platform package.v.r.m.f.ptf 4.X** or later updates for the POWER-based platform package.part.v.r.m.f.U Update image for Neutral package.v.r.m.f.N.U ** 4.X or later updates contain one package only. In addition, AIX Version 4 and later updates do not contain ptf IDs. package = the name of the software package as described by the PackageName parameter v.r.m.f = version.release.modification.fix, the level associated with the software package. The PackageName is usually not the same as the fileset name. ptf = program temporary fix ID (also known as FixID) The installation image file name has the form Package.Level.I. The Package is the name of the software package, as described for the Package Name parameter. Level has the format of v.r.m.f, where v = version, r = release, m = modification, f = fix. The I extension means that the image is an installation image rather than an update image. Update image files containing an AIX 3.1 formatted update have a service number extension following the level. The Servicenum parameter can be up to 4 digits in length. One example is xlccmp.3.1.5.0.1234. Update image files containing an AIX 3.2 formatted update have a ptf extension following the level. One example is bosnet.3.2.0.0.U412345. AIX Version 4 and later update image file names begin with the fileset name, not the PackageName. They also have U extensions to indicate that they are indeed update image files, not installation images. One example of an update image file is bos.rte.install.4.3.2.0.U. The all keyword indicates that installation image files are created for every installable software package on the device. You can extract a single update image with the AIX Version 4 and later bffcreate command. Then you must specify the fileset name and the v.r.m.f. parameter. As in example 3 in the Examples section, the PackageName parameter must be the entire fileset name, bos.net.tcp.client, not just bos.net. Attention: Be careful when selecting the target directory for the extracted images, especially if that directory already contains installable images. If a fileset at a particular level exists as both an installation image and as an update image in the same directory, unexpected installation results can occur.
In cases like this, installp selects the image it finds first in the table of contents (.toc) file. The image it selects may not be the one you intended and unexpected requisite failures can result. As a rule of thumb, you should extract maintenance levels to clean directories. Examples To create an installation image file from the bos.net software package on the tape in the /dev/rmt0 tape drive and use /var/tmp as the working directory, type: # bffcreate -d /dev/rmt0.1 -w /var/tmp bos.net To create an installation image file from the package software package on the diskette in the /dev/rfd0 diskette drive and print the name of the installation image file without being prompted, type: # bffcreate -q -v package To create a single update image file from the bos.net.tcp.client software package on the CD in /dev/cd0, type: # bffcreate -d /dev/cd0 bos.net.tcp.client 4.2.2.1 To list the packages on the CD in /dev/cd0, type: # bffcreate -l -d /dev/cd0 To create installation and/or update images from a CD in /dev/cd0 by specifying a list of PackageNames and Levels in a ListFile called my MyListFile, type: # bffcreate -d /dev/cd0 -f MyListFile To create installation or update images of all software packages on the CD-ROM media for the current platform, type: # bffcreate -d /dev/cd0 all To list fileset information for the bos.games software package from a particular device, type: # bffcreate -d /usr/sys/inst.images/bos.games -l To list all the Neutral software packages on the CD-ROM media, type: # bffcreate -d /dev/cd0 -MN -l
38.3 Software Packages on Linux: ================================ 38.3.1 RPM packages on Linux (1): --------------------------------Note 1: ------First we show a few simple examples:
- Examples getting software info from your system: # rpm -q kernel kernel-2.4.7-10 # rpm -q glibc glibc-2.2.4-19.3 # rpm -q gcc gcc-2.96-98 Show everything: # rpm -qa - Examples installing rpm packages: # rpm -Uvh libpng-1.2.2-22.i386.rpm # rpm -Uvh gnome-libs-1.4.1.2.90-40.i386.rpm # rpm -Uvh oracleasm-support-2.0.0-1.i386.rpm \ oracleasm-lib-2.0.0-1.i386.rpm \ oracleasm-2.6.9-5.0.5-ELsmp-2.0.0-1.i686.rpm # rpm -Uvh /mnt/cdrom/RedHat/RPMS/tripwire*.rpm Note: the U switch really means starting an Upgrade, but if nothing is there, an installation will take place. Note 2: ------What is RPM? RPM is the RPM Package Manager. It is an open packaging system available for anyone to use. It allows users to take source code for new software and package it into source and binary form such that binaries can be easily installed and tracked and source can be rebuilt easily. It also maintains a database of all packages and their files that can be used for verifying packages and querying for information about files and/or packages. Red Hat, Inc. encourages other distribution vendors to take the time to look at RPM and use it for their own distributions. RPM is quite flexible and easy to use, though it provides the base for a very extensive system. It is also completely open and available, though we would appreciate bug reports and fixes. Permission is granted to use and distribute RPM royalty free under the GPL.
More complete documentation is available on RPM in the book by Ed Bailey, Maximum RPM. That book is available for download or purchase at www.redhat.com. RPM is a core component of many Linux distributions, such as Red Hat Enterprise Linux, the Fedora Project, SUSE Linux Enterprise, openSUSE, CentOS, Mandriva Linux, and many others. It is also used on many other operating systems as well, and the RPM format is part of the Linux Standard Base. Acquiring RPM The best way to get RPM is to install Red Hat Linux. If you don't want to do that, you can still get and use RPM. It can be acquired from ftp.redhat.com. RPM Requirements RPM itself should build on basically any Unix-like system. It has been built and used on Tru64 Unix, AIX, Solaris, SunOS, and basically all flavors of Linux. To build RPMs from source, you also need everything normally required to build a package, like gcc, make, etc. In its simplest form, RPM can be used to install packages: # rpm -i foobar-1.0-1.i386.rpm The next simplest command is to uninstall a package: # rpm -e foobar One of the more complex but highly useful commands allows you to install packages via FTP. If you are connected to the net and want to install a new package, all you need to do is specify the file with a valid URL, like so: # rpm -i ftp://ftp.redhat.com/pub/redhat/rh-2.0-beta/RPMS/foobar-1.01.i386.rpm Please note, that RPM will now query and/or install via FTP. While these are simple commands, rpm can be used in a multitude of ways. To see which options are available in your version of RPM, type: # rpm --help You can find more details on what those options do in the RPM man page, found by typing: # man rpm
RPM is a very useful tool and, as you can see, has several options. The best way to make sense of them is to look at some examples. I covered simple install/uninstall above, so here are some more examples: Let's say you delete some files by accident, but you aren't sure what you deleted. If you want to verify your entire system and see what might be missing, you would do: # rpm -Va Let's say you run across a file that you don't recognize. To find out which package owns it, you would do: # rpm -qf /usr/X11R6/bin/xjewel The output would be sometime like: xjewel-1.6-1 You find a new koules RPM, but you don't know what it is. To find out some information on it, do: # rpm -qpi koules-1.2-2.i386.rpm The output would be: Name : Colgate Version : Release : 11:59:12 1996 Install date: Group : 2.src.rpm Size : Summary : support Description :
koules
Distribution: Red Hat Linux
1.2 2
Vendor: Red Hat Software Build Date: Mon Sep 02
(none) Games
Build Host: porky.redhat.com Source RPM: koules-1.2-
614939 SVGAlib action game with multiplayer, network, and sound
This arcade-style game is novel in conception and excellent in execution. No shooting, no blood, no guts, no gore. The play is simple, but you still must develop skill to play. This version uses SVGAlib to run on a graphics console. Now you want to see what files the koules RPM installs. You would do: # rpm -qpl koules-1.2-2.i386.rpm The output is: /usr/doc/koules /usr/doc/koules/ANNOUNCE /usr/doc/koules/BUGS /usr/doc/koules/COMPILE.OS2 /usr/doc/koules/COPYING
/usr/doc/koules/Card /usr/doc/koules/ChangeLog /usr/doc/koules/INSTALLATION /usr/doc/koules/Icon.xpm /usr/doc/koules/Icon2.xpm /usr/doc/koules/Koules.FAQ /usr/doc/koules/Koules.xpm /usr/doc/koules/README /usr/doc/koules/TODO /usr/games/koules /usr/games/koules.svga /usr/games/koules.tcl /usr/man/man6/koules.svga.6 SYNOPSIS QUERYING AND VERIFYING PACKAGES: rpm {-q|--query} [select-options] [query-options] rpm {-V|--verify} [select-options] [verify-options] rpm --import PUBKEY ... rpm {-K|--checksig} [--nosignature] [--nodigest] PACKAGE_FILE ... INSTALLING, UPGRADING, AND REMOVING PACKAGES: rpm {-i|--install} [install-options] PACKAGE_FILE ... rpm {-U|--upgrade} [install-options] PACKAGE_FILE ... rpm {-F|--freshen} [install-options] PACKAGE_FILE ... rpm {-e|--erase} [--allmatches] [--nodeps] [--noscripts] [--notriggers] [--repackage] [--test] PACKAGE_NAME ... MISCELLANEOUS: rpm {--initdb|--rebuilddb} rpm {--addsign|--resign} PACKAGE_FILE ... rpm {--querytags|--showrc} rpm {--setperms|--setugids} PACKAGE_NAME ...
Note 3: ------NAME rpm - RPM Package Manager SYNOPSIS QUERYING AND VERIFYING PACKAGES: rpm {-q|--query} [select-options] [query-options] rpm {-V|--verify} [select-options] [verify-options] rpm --import PUBKEY ... rpm {-K|--checksig} [--nosignature] [--nodigest] PACKAGE_FILE ...
INSTALLING, UPGRADING, AND REMOVING PACKAGES: rpm {-i|--install} [install-options] PACKAGE_FILE ... rpm {-U|--upgrade} [install-options] PACKAGE_FILE ... rpm {-F|--freshen} [install-options] PACKAGE_FILE ... rpm {-e|--erase} [--allmatches] [--nodeps] [--noscripts] [--notriggers] [--repackage] [--test] PACKAGE_NAME ... MISCELLANEOUS: rpm {--initdb|--rebuilddb} rpm {--addsign|--resign} PACKAGE_FILE ... rpm {--querytags|--showrc} rpm {--setperms|--setugids} PACKAGE_NAME ... select-options [PACKAGE_NAME] [-a,--all] [-f,--file FILE] [-g,--group GROUP] {-p,--package PACKAGE_FILE] [--fileid MD5] [--hdrid SHA1] [--pkgid MD5] [--tid TID] [--querybynumber HDRNUM] [--triggeredby PACKAGE_NAME] [--whatprovides CAPABILITY] [--whatrequires CAPABILITY] query-options [--changelog] [-c,--configfiles] [-d,--docfiles] [--dump] [--filesbypkg] [-i,--info] [--last] [-l,--list] [--provides] [--qf,--queryformat QUERYFMT] [-R,--requires] [--scripts] [-s,--state] [--triggers,--triggerscripts] verify-options [--nodeps] [--nofiles] [--noscripts] [--nodigest] [--nosignature] [--nolinkto] [--nomd5] [--nosize] [--nouser] [--nogroup] [--nomtime] [--nomode] [--nordev] install-options [--aid] [--allfiles] [--badreloc] [--excludepath OLDPATH] [--excludedocs] [--force] [-h,--hash] [--ignoresize] [--ignorearch] [--ignoreos] [--includedocs] [--justdb] [--nodeps] [--nodigest] [--nosignature] [--nosuggest] [--noorder] [--noscripts] [--notriggers] [--oldpackage] [--percent] [--prefix NEWPATH] [--relocate OLDPATH=NEWPATH] [--repackage] [--replacefiles] [--replacepkgs] [--test] DESCRIPTION rpm is a powerful Package Manager, which can be used to build, install, query, verify, update, and erase
individual software packages. A package consists of an archive of files and meta-data used to install and erase the archive files. The meta-data includes helper scripts, file attributes, and descriptive information about the package. Packages come in two varieties: binary packages, used to encapsulate software to be installed, and source packages, containing the source code and recipe necessary to produce binary packages. One of the following basic modes must be selected: Query, Verify, Signature Check, Install/Upgrade/Freshen, Uninstall, Initialize Database, Rebuild Database, Resign, Add Signature, Set Owners/Groups, Show Querytags, and Show Configuration. GENERAL OPTIONS These options can be used in all the different modes. -?, --help Print a longer usage message then normal. --version Print a single line containing the version number of rpm being used. --quiet Print as little as possible - normally only error messages will be displayed. -v Print verbose information - normally routine progress messages will be displayed. -vv Print lots of ugly debugging information. --rcfile FILELIST Each of the files in the colon separated FILELIST is read sequentially by rpm for configuration information. Only the first file in the list must exist, and tildes will be expanded to the value of $HOME. The default FILELIST is /usr/lib/rpm/rpmrc:/usr/lib/rpm/redhat/rpmrc:~/.rpmrc. --pipe CMD Pipes the output of rpm to the command CMD. --dbpath DIRECTORY Use the database in DIRECTORY rathen than the default path /var/lib/rpm --root DIRECTORY Use the file system tree rooted at DIRECTORY for all operations. Note that this means the database within DIRECTORY will be used for dependency checks and any scriptlet(s) (e.g. %post if installing, or %prep if building, a package) will be run after a chroot(2) to DIRECTORY. INSTALL AND UPGRADE OPTIONS The general form of an rpm install command is rpm {-i|--install} [install-options] PACKAGE_FILE ... This installs a new package.
The general form of an rpm upgrade command is rpm {-U|--upgrade} [install-options] PACKAGE_FILE ... This upgrades or installs the package currently installed to a newer version. This is the same as install, except all other version(s) of the package are removed after the new package is installed. rpm {-F|--freshen} [install-options] PACKAGE_FILE ... This will upgrade packages, but only if an earlier version currently exists. The PACKAGE_FILE may be specified as an ftp or http URL, in which case the package will be downloaded before being installed. See FTP/HTTP OPTIONS for information on rpm's internal ftp and http client support. --aid Add suggested packages to the transaction set when needed. --allfiles Installs or upgrades all the missingok files in the package, regardless if they exist. --badreloc Used with --relocate, permit relocations on all file paths, not just those OLDPATH's included in the binary package relocation hint(s). --excludepath OLDPATH Don't install files whose name begins with OLDPATH. --excludedocs Don't install any files which are marked as documentation (which includes man pages and texinfo documents). --force Same as using --replacepkgs, --replacefiles, and --oldpackage. -h, --hash Print 50 hash marks as the package archive is unpacked. Use with -v|-verbose for a nicer display. --ignoresize Don't check mount file systems for sufficient disk space before installing this package. --ignorearch Allow installation or upgrading even if the architectures of the binary package and host don't match. --ignoreos Allow installation or upgrading even if the operating systems of the binary package and host don't match. --includedocs Install documentation files. This is the default behavior. --justdb Update only the database, not the filesystem. --nodigest Don't verify package or header digests when reading. --nosignature Don't verify package or header signatures when reading. --nodeps
Don't do a dependency check before installing or upgrading a package. --nosuggest Don't suggest package(s) that provide a missing dependency. --noorder Don't reorder the packages for an install. The list of packages would normally be reordered to satisfy dependancies. --noscripts --nopre --nopost --nopreun --nopostun Don't execute the scriptlet of the same name. The --noscripts option is equivalent to --nopre --nopost --nopreun --nopostun and turns off the execution of the corresponding %pre, %post, %preun, and %postun scriptlet(s). --notriggers --notriggerin --notriggerun --notriggerpostun Don't execute any trigger scriptlet of the named type. The --notriggers option is equivalent to --notriggerin --notriggerun --notriggerpostun and turns off execution of the corresponding %triggerin, %triggerun, and %triggerpostun scriptlet(s). --oldpackage Allow an upgrade to replace a newer package with an older one. --percent Print percentages as files are unpacked from the package archive. This is intended to make rpm easy to run from other tools. --prefix NEWPATH For relocateable binary packages, translate all file paths that start with the installation prefix in the package relocation hint(s) to NEWPATH. --relocate OLDPATH=NEWPATH For relocatable binary packages, translate all file paths that start with OLDPATH in the package relocation hint(s) to NEWPATH. This option can be used repeatedly if several OLDPATH's in the package are to be relocated. --repackage Re-package the files before erasing. The previously installed package will be named according to the macro %_repackage_name_fmt and will be created in the directory named by the macro %_repackage_dir (default value is /var/tmp). --replacefiles Install the packages even if they replace files from other, already installed, packages. --replacepkgs Install the packages even if some of them are already installed on this system. --test Do not install the package, simply check for and report potential conflicts.
ERASE OPTIONS The general form of an rpm erase command is rpm {-e|--erase} [--allmatches] [--nodeps] [--noscripts] [--notriggers] [--repackage] [--test] PACKAGE_NAME ... The following options may also be used: --allmatches Remove all versions of the package which match PACKAGE_NAME. Normally an error is issued if PACKAGE_NAME matches multiple packages. --nodeps Don't check dependencies before uninstalling the packages. --noscripts --nopreun --nopostun Don't execute the scriptlet of the same name. The --noscripts option during package erase is equivalent to --nopreun --nopostun and turns off the execution of the corresponding %preun, and %postun scriptlet(s). --notriggers --notriggerun --notriggerpostun Don't execute any trigger scriptlet of the named type. The --notriggers option is equivalent to --notriggerun --notriggerpostun and turns off execution of the corresponding %triggerun, and %triggerpostun scriptlet(s). --repackage Re-package the files before erasing. The previously installed package will be named according to the macro %_repackage_name_fmt and will be created in the directory named by the macro %_repackage_dir (default value is /var/tmp). --test Don't really uninstall anything, just go through the motions. Useful in conjunction with the -vv option for debugging. QUERY OPTIONS The general form of an rpm query command is rpm {-q|--query} [select-options] [query-options] You may specify the format that package information should be printed in. To do this, you use the --qf|--queryformat QUERYFMT
option, followed by the QUERYFMT format string. Query formats are modifed versions of the standard printf(3) formatting. The format is made up of static strings (which may include standard C character escapes for newlines, tabs, and other special characters) and printf(3) type formatters. As rpm already knows the type to print, the type specifier must be omitted however, and replaced by the name of the header tag to be printed, enclosed by {} characters. Tag names are case insesitive, and the leading RPMTAG_ portion of the tag name may be omitted as well. Alternate output formats may be requested by following the tag with :typetag. Currently, the following types are supported: :armor Wrap a public key in ASCII armor. :base64 Encode binary data using base64. :date Use strftime(3) "%c" format. :day Use strftime(3) "%a %b %d %Y" format. :depflags Format dependency flags. :fflags Format file flags. :hex Format in hexadecimal. :octal Format in octal. :perms Format file permissions. :shescape Escape single quotes for use in a script. :triggertype Display trigger suffix. For example, to print only the names of the packages queried, you could use %{NAME} as the format string. To print the packages name and distribution information in two columns, you could use %-30{NAME}% {DISTRIBUTION}. rpm will print a list of all of the tags it knows about when it is invoked with the --querytags argument. There are two subsets of options for querying: package selection, and information selection. PACKAGE SELECTION OPTIONS: PACKAGE_NAME Query installed package named PACKAGE_NAME. -a, --all Query all installed packages. -f, --file FILE Query package owning FILE. --fileid MD5 Query package that contains a given file identifier, i.e. the MD5 digest of the file contents. -g, --group GROUP
Query packages with the group of GROUP. --hdrid SHA1 Query package that contains a given header identifier, i.e. the SHA1 digest of the immutable header region. -p, --package PACKAGE_FILE Query an (uninstalled) package PACKAGE_FILE. The PACKAGE_FILE may be specified as an ftp or http style URL, in which case the package header will be downloaded and queried. See FTP/HTTP OPTIONS for information on rpm's internal ftp and http client support. The PACKAGE_FILE argument(s), if not a binary package, will be interpreted as an ASCII package manifest. Comments are permitted, starting with a '#', and each line of a package manifest file may include white space seperated glob expressions, including URL's with remote glob expressions, that will be expanded to paths that are substituted in place of the package manifest as additional PACKAGE_FILE arguments to the query. --pkgid MD5 Query package that contains a given package identifier, i.e. the MD5 digest of the combined header and payload contents. --querybynumber HDRNUM Query the HDRNUMth database entry directly; this is useful only for debugging. --specfile SPECFILE Parse and query SPECFILE as if it were a package. Although not all the information (e.g. file lists) is available, this type of query permits rpm to be used to extract information from spec files without having to write a specfile parser. --tid TID Query package(s) that have a given TID transaction identifier. A unix time stamp is currently used as a transaction identifier. All package(s) installed or erased within a single transaction have a common identifier. --triggeredby PACKAGE_NAME Query packages that are triggered by package(s) PACKAGE_NAME. --whatprovides CAPABILITY Query all packages that provide the CAPABILITY capability. --whatrequires CAPABILITY Query all packages that requires CAPABILITY for proper functioning. PACKAGE QUERY OPTIONS: --changelog Display change information for the package. -c, --configfiles List only configuration files (implies -l). -d, --docfiles List only documentation files (implies -l). --dump Dump file information as follows: path size mtime md5sum mode owner group isconfig isdoc rdev symlink This option must be used with at least one of -l, -c, -d. --filesbypkg List all the files in each selected package. -i, --info
Display package information, including name, version, and description. This uses the --queryformat if one was specified. --last Orders the package listing by install time such that the latest packages are at the top. -l, --list List files in package. --provides List capabilities this package provides. -R, --requires List packages on which this package depends. --scripts List the package specific scriptlet(s) that are used as part of the installation and uninstallation processes. -s, --state Display the states of files in the package (implies -l). The state of each file is one of normal, not installed, or replaced. --triggers, --triggerscripts Display the trigger scripts, if any, which are contained in the package. VERIFY OPTIONS The general form of an rpm verify command is rpm {-V|--verify} [select-options] [verify-options] Verifying a package compares information about the installed files in the package with information about the files taken from the package metadata stored in the rpm database. Among other things, verifying compares the size, MD5 sum, permissions, type, owner and group of each file. Any discrepencies are displayed. Files that were not installed from the package, for example, documentation files excluded on installation using the "--excludedocs" option, will be silently ignored. The package selection options are the same as for package querying (including package manifest files as arguments). Other options unique to verify mode are: --nodeps Don't verify dependencies of packages. --nodigest Don't verify package or header digests when reading. --nofiles Don't verify any attributes of package files. --noscripts Don't execute the %verifyscript scriptlet (if any). --nosignature Don't verify package or header signatures when reading. --nolinkto --nomd5 --nosize --nouser --nogroup --nomtime --nomode
--nordev Don't verify the corresponding file attribute. The format of the output is a string of 8 characters, a possible attribute marker: c %config configuration file. d %doc documentation file. g %ghost file (i.e. the file contents are not included in the package payload). l %license license file. r %readme readme file. from the package header, followed by the file name. Each of the 8 characters denotes the result of a comparison of attribute(s) of the file to the value of those attribute(s) recorded in the database. A single "." (period) means the test passed, while a single "?" (question mark) indicates the test could not be performed (e.g. file permissions prevent reading). Otherwise, the (mnemonically emBoldened) character denotes failure of the corresponding --verify test: S M 5 D L U G T
file Size differs Mode differs (includes permissions and file type) MD5 sum differs Device major/minor number mis-match readLink(2) path mis-match User ownership differs Group ownership differs mTime differs
DIGITAL SIGNATURE AND DIGEST VERIFICATION The general forms of rpm digital signature commands are
rpm --import PUBKEY ... rpm {--checksig} [--nosignature] [--nodigest] PACKAGE_FILE ... The --checksig option checks all the digests and signatures contained in PACKAGE_FILE to ensure the integrity and origin of the package. Note that signatures are now verified whenever a package is read, and --checksig is useful to verify all of the digests and signatures associated with a package. Digital signatures cannot be verified without a public key. An ascii armored public key can be added to the rpm database using --import. An imported public key is carried in a header, and key ring management is performed exactly like package management. For example, all currently imported public keys can be displayed by: rpm -qa gpg-pubkey*
Details about a specific public key, when imported, can be displayed by querying. Here's information about the Red Hat GPG/DSA key: rpm -qi gpg-pubkey-db42a60e Finally, public keys can be erased after importing just like packages. Here's how to remove the Red Hat GPG/DSA key rpm -e gpg-pubkey-db42a60e SIGNING A PACKAGE rpm --addsign|--resign PACKAGE_FILE ... Both of the --addsign and --resign options generate and insert new signatures for each package PACKAGE_FILE given, replacing any existing signatures. There are two options for historical reasons, there is no difference in behavior currently. USING GPG TO SIGN PACKAGES In order to sign packages using GPG, rpm must be configured to run GPG and be able to find a key ring with the appropriate keys. By default, rpm uses the same conventions as GPG to find key rings, namely the $GNUPGHOME environment variable. If your key rings are not located where GPG expects them to be, you will need to configure the macro %_gpg_path to be the location of the GPG key rings to use. For compatibility with older versions of GPG, PGP, and rpm, only V3 OpenPGP signature packets should be configured. Either DSA or RSA verification algorithms can be used, but DSA is preferred. If you want to be able to sign packages you create yourself, you also need to create your own public and secret key pair (see the GPG manual). You will also need to configure the rpm macros %_signature The signature type. Right now only gpg and pgp are supported. %_gpg_name The name of the "user" whose key you wish to use to sign your packages. For example, to be able to use GPG to sign packages as the user "John Doe " from the key rings located in /etc/rpm/.gpg using the executable /usr/bin/gpg you would include %_signature gpg %_gpg_path /etc/rpm/.gpg %_gpg_name John Doe %_gpgbin /usr/bin/gpg in a macro configuration file. Use /etc/rpm/macros for per-system configuration and ~/.rpmmacros for per-user configuration. REBUILD DATABASE OPTIONS The general form of an rpm rebuild database command is
rpm {--initdb|--rebuilddb} [-v] [--dbpath DIRECTORY] [--root DIRECTORY] Use --initdb to create a new database, use --rebuilddb to rebuild the database indices from the installed package headers. SHOWRC The command rpm --showrc shows the values rpm will use for all of the options are currently set in rpmrc and macros configuration file(s). FTP/HTTP OPTIONS rpm can act as an FTP and/or HTTP client so that packages can be queried or installed from the internet. Package files for install, upgrade, and query operations may be specified as an ftp or http style URL: ftp://USER:PASSWORD@HOST:PORT/path/to/package.rpm If the :PASSWORD portion is omitted, the password will be prompted for (once per user/hostname pair). If both the user and password are omitted, anonymous ftp is used. In all cases, passive (PASV) ftp transfers are performed. rpm allows the following options to be used with ftp URLs: --ftpproxy HOST The host HOST will be used as a proxy server for all ftp transfers, which allows users to ftp through firewall machines which use proxy systems. This option may also be specified by configuring the macro %_ftpproxy. --ftpport HOST The TCP PORT number to use for the ftp connection on the proxy ftp server instead of the default port. This option may also be specified by configuring the macro %_ftpport. rpm allows the following options to be used with http URLs: --httpproxy HOST The host HOST will be used as a proxy server for all http transfers. This option may also be specified by configuring the macro %_httpproxy. --httpport PORT The TCP PORT number to use for the http connection on the proxy http server instead of the default port. This option may also be specified by configuring the macro %_httpport. LEGACY ISSUES Executing rpmbuild The build modes of rpm are now resident in the /usr/bin/rpmbuild executable. Although legacy compatibility provided by the popt aliases below has been adequate, the compatibility is not perfect; hence build mode compatibility through popt aliases is being removed from rpm. Install the rpmbuild package, and see rpmbuild(8) for documentation of all the rpm build modes previously documented here in rpm(8).
Add the following lines to /etc/popt if you wish to continue invoking rpmbuild from the rpm command line: rpm rpm rpm rpm rpm rpm rpm rpm rpm rpm rpm rpm rpm rpm rpm rpm rpm rpm rpm rpm rpm
exec exec exec exec exec exec exec exec exec exec exec exec exec exec exec exec exec exec exec exec exec
--bp --bc --bi --bl --ba --bb --bs --tp --tc --ti --tl --ta --tb --ts --rebuild --recompile --clean --rmsource --rmspec --target --short-circuit
rpmb rpmb rpmb rpmb rpmb rpmb rpmb rpmb rpmb rpmb rpmb rpmb rpmb rpmb rpmb rpmb rpmb rpmb rpmb rpmb rpmb
-bp -bc -bi -bl -ba -bb -bs -tp -tc -ti -tl -ta -tb -ts --rebuild --recompile --clean --rmsource --rmspec --target --short-circuit
SEE ALSO popt(3), rpm2cpio(8), rpmbuild(8), http://www.rpm.org/ http://www.rpm.org/>
39. Simplified overview Kernel parameters Solaris, AIX, Linux: ============================================================== Throughout this document, you can find many other examples of settings. This section is only a simplified overview. 39.1 Solaris: ------------The "/etc/system" file: Available for Solaris Operating Environment, the /etc/system file contains definitions for kernel configuration limits such as the maximum number of users allowed on the system at a time, the maximum number of processes per user, and the inter-process communication (IPC) limits on size and number of resources. These limits are important because
they affect, for example, DB2, Oracle performance on a Solaris Operating Environment machine. Some examples: set set set set set set set set set .. ..
shmsys:shminfo_shmmax=4294967295 shmsys:shminfo_shmmin=1 shmsys:shminfo_shmmni=100 shmsys:shminfo_shmseg=10 semsys:seminfo_semmni=100 semsys:seminfo_semmsl=100 semsys:seminfo_semmns=2500 semsys:seminfo_semopm=100 semsys:seminfo_semvmx=32767
You can use, among others, the "ipcs" command and "adb" command to retrieve kernel parameters and mem info. Some remarks on Shared Memory and Semaphores: - Shared Memory Shared memory provides the fastest way for processes to pass large amounts of data to one another. As the name implies, shared memory refers to physical pages of memory that are shared by more than one process. Of particular interest is the "Intimate Shared Memory" facility, where the translation tables are shared as well as the memory. This enhances the effectiveness of the TLB (Translation Lookaside Buffer), which is a CPU-based cache of translation table information. Since the same information is used for several processes, available buffer space can be used much more efficiently. In addition, ISM-designated memory cannot be paged out, which can be used to keep frequently-used data and binaries in memory. Database applications are the heaviest users of shared memory. Vendor recommendations should be consulted when tuning the shared memory parameters. Solaris 10 only uses the shmmax and shmmni parameters. (Other parameters are set dynamically within the Solaris 10 IPC model.) shmmax (max-shm-memory in Solaris 10+): This is the maximum size of a shared memory segment (ie the largest value that can be used by shmget). Its theoretical maximum value is 4294967295 (4GB), but practical considerations usually limit it to less than this. There is no reason not to tune this value as high as possible, since no kernel resources are allocated based on this parameter. Solaris 10 sets shmmax to 1/4 physical memory by default, vs 512k for previous versions.
shmmin: This is the smallest possible shared memory segment size. The default is 1 byte; this parameter should probably not be tuned. shmmni (max-shm-ids in Solaris 10+): Maximum number of shared memory identifiers at any given time. This parameter is used by kernel memory allocation to determine how much size to put aside for shmid_ds structures. Each of these is 112 bytes and requires an additional 8 bytes for a mutex lock; if it is set too high, memory useage can be a problem. The maximum setting for this variable in Solaris 2.5.1 and 2.6 is 2147483648 (2GB), and the default is 100. For Solaris 10, the default is 128 and the maximum is MAXINT. shmseg: Maximum number of segments per process. It is usually set to shmmni, but it should always be less than 65535. Sun documentations suggests a maximum for this parameter of 32767 and a default of 8 for Solaris 2.5.1 and 2.6. - Semaphores Semaphores are a shareable resource that take on a non-negative integer value. They are manipulted by the P (wait) and V (signal) functions, which decrement and increment the semaphore, respectively. When a process needs a resource, a "wait" is issued and the semaphore is decremented. When the semaphore contains a value of zero, the resources are not available and the calling process spins or blocks (as appropriate) until resources are available. When a process releases a resource controlled by a semaphore, it increments the semaphore and the waiting processes are notified. Solaris 10 only uses the semmni, semmsl and semopm parameters. (Other parameters are dynamic within the Solaris 10 IPC model.) semmap: This sets the number of entries in the semaphore map. This should never be greater than semmni. If the number of semaphores per semaphore set used by the application is "n" then set semmap = ((semmni + n - 1)/n)+1 or more. Alternatively, we can set semmap to semmni x semmsl. An undersized semmap leads to "WARNING: rmfree map overflow" errors. The default setting is 10; the maximum for Solaris 2.6 is 2GB. The default for Solaris 9 was 25; Solaris 10 increased the default to 512. The limit is SHRT_MAX. semmni (max-sem-ids in Solaris 10+): Maximum number of systemwide semaphore sets. Each control structure consumes 84 bytes. For Solaris 2.5.1-9, the default setting is 10; for Solaris 10, the default setting is 128. The maximum is 65535 semmns: Maximum number of semaphores in the system. Each structure uses 16 bytes. This parameter should be set to semmni x semmsl. The default is 60; the maximum is 2GB. semmnu: Maximum number of undo structures in the system. This should be set to semmni so that each control structure
has an undo structure. The default is 30, the maximum is 2 GB. semmsl (max-sem-nsems in Solaris 10+): Maximum number of semaphores per semaphore set. The default is 25, the maximum is 65535. semopm (max-sem-ops in Solaris 10+): Maximum number of semaphore operations that can be performed in each semop call. The default in Solaris 2.5.1-9 is 10, the maximum is 2 GB. Solaris 10 increased the default to 512. semume: Maximum number of undo structures per process. This should be set to semopm times the number of processes that will be using semaphores at any one time. The default is 10; the maximum is 2 GB. semusz: Number of bytes required for semume undo structures. This should not be tuned; it is set to semume x (1 + sizeof(undo)). The default is 96; the maximum is 2 GB. semvmx: Maximum value of a semaphore. This should never exceed 32767 (default value) unless SEM_UNDO is never used. The default is 32767; the maximum is 65535. semaem: Maximum adjust-on-exit value. This should almost always be left alone. The default is 16384; the maximum is 32767.
39.2 Linux: ----------Kernel parameters used for system configuration are found in "/etc/sysctl.conf" and on a running system also in "/proc/sys/kernel", where you will find an individual file for each configuration parameter. Because these parameters have a direct effect on system performance and viability, you must have root access in order to modify them. Occasionally, a prerequisite to a package installation requires the modification of kernel parameters. Since each parameter file contains a single line of data consisting of either a text string or numeric values, it is often easy to modify a parameter by simply using the echo command: # echo 2048 > /proc/sys/kernel/msgmax The aforementioned command will set the value of the msgmax parameter to 2048. -- More on the proc File System: The Linux kernel has two primary functions: to control access to physical devices on the computer and to schedule when and how processes interact with these devices. The /proc/ directory contains a hierarchy of special files which represent the current state of the kernel - allowing applications and users to peer into the kernel's view of the system.
Within the /proc/ directory, one can find a wealth of information about the system hardware and any processes currently running. In addition, some of the files within the /proc/ directory tree can be manipulated by users and applications to communicate configuration changes to the kernel. Under Linux, all data are stored as files. Most users are familiar with the two primary types of files: text and binary. But the /proc/ directory contains another type of file called a virtual file. It is for this reason that /proc/ is often referred to as a virtual file system. These virtual files have unique qualities. Most of them are listed as zero bytes in size and yet when one is viewed, it can contain a large amount of information. In addition, most of the time and date settings on virtual files reflect the current time and date, indicative of the fact they constantly changing. Virtual files such as interrupts, /proc/meminfo, /proc/mounts, and /proc/partitions provide an up-to-the-moment glimpse of the system's hardware. Others, like /proc/filesystems and the /proc/sys/ directory provide system configuration information and interfaces. For organizational purposes, files containing information on a similar topic are grouped into virtual directories and sub-directories. For instance, /proc/ide/ contains information for all physical IDE devices. Likewise, process directories contain information about each running process on the system. By using the cat, more, or less commands on files within the /proc/ directory, you can immediately access an enormous amount of information about the system. For example, if you want to see what sort of CPU your computer has, type "cat /proc/cpuinfo" and you will see something similar to the following: processor vendor_id cpu family model model name stepping cpu MHz cache size fdiv_bug hlt_bug f00f_bug coma_bug fpu fpu_exception cpuid level wp flags k6_mtrr
: : : : : : : : : : : : : : : : :
0 AuthenticAMD 5 9 AMD-K6(tm) 3D+ Processor 1 400.919 256 KB no no no no yes yes 1 yes fpu vme de pse tsc msr mce cx8 pge mmx syscall 3dnow
bogomips
: 799.53
When viewing different virtual files in the /proc/ file system, you will notice some of the information is easily understandable while some is not human-readable. This is in part why utilities exist to pull data from virtual files and display it in a useful way. Some examples of such applications are lspci, apm, free, and top. As a general rule, most virtual files within the /proc/ directory are read only. However, some can be used to adjust settings in the kernel. This is especially true for files in the /proc/sys/ subdirectory. To change the value of a virtual file, use the echo command and a > symbol to redirect the new value to the file. For instance, to change your hostname on the fly, you can type: echo bob.subgenius.com > /proc/sys/kernel/hostname Other files act as binary or boolean switches. For instance, if you type cat /proc/sys/net/ipv4/ip_forward, you will see either a 0 or a 1. A 0 indicates the kernel is not forwarding network packets. By using the echo command to change the value of the ip_forward file to 1, you can immediately turn packet forwarding on. Another command used to alter settings in the /proc/sys/ subdirectory is /sbin/sysctl. -- sysctl: Linux also provides the sysctl command to modify kernel parameters at runtime. Sysctl uses parameter information stored in a file called /etc/sysctl.conf. If, for example, we wanted to change the value of the msgmax parameter as we did above, but this time using sysctl, the command would look like this: # sysctl -w kernel.msgmax=2048 - About the kernel: Finding the Kernel Locate the kernel image on your hard disk. It should be in the file /vmlinuz, or /vmlinux, or /boot/vmlinux In some installations, /vmlinuz is a soft link to the actual kernel, so you may need to track down the kernel by following the links. On Redhat 6.1 it is in "/boot/vmlinuz". To find the kernel being used look in "/etc/lilo.conf".
You can also type "uname -a" to see the kernel version. /proc/cmdline This file shows the parameters passed to the kernel at the time it is started. A sample /proc/cmdline file looks like this: ro root=/dev/hda2 This tell us the kernel is mounted read-only - signified by (ro) - off of the second partition on the first IDE device (/dev/hda2). - Kernel, memory tuning: Most about tuning memory en kernel params seem to do with the "/etc/sysctl.conf" file: In most distributions, the "/etc/sysctl.conf" determines the limits and/or behaviour of the kernel and memory. If you type "sysctl -a |more" you will see a long list of kernel parameters. You can use this sysctl program to modify these parameters, for example: # sysctl -w kernel.shmmax=100000000 # sysctl -w fs.file-max=65536 # echo "kernel.shmmax = 100000000" >> /etc/sysctl.conf Example configuration: setting kernel parameters before installing Oracle 10g: ----------------------------------------------------------------------------Most out of the box kernel parameters (of RHELS 3,4,5) are set correctly for Oracle except a few. You should have the following minimal configuration: net.ipv4.ip_local_port_range kernel.sem kernel.shmmni kernel.shmall kernel.shmmax fs.file-max
1024 65000 250 32000 100 4096 2097152 2147483648 65536
128
You can check the most important parameters using the following command: # /sbin/sysctl -a | egrep 'sem|shm|file-max|ip_local' net.ipv4.ip_local_port_range = 1024
65000
kernel.sem = 250 32000 100 kernel.shmmni = 4096 kernel.shmall = 2097152 kernel.shmmax = 2147483648 fs.file-max = 65536
128
If some value should be changed, you can change the "/etc/sysctl.conf" file and run the "/sbin/sysctl -p" command to change the value immediately. Every time the system boots, the init program runs the /etc/rc.d/rc.sysinit script. This script contains a command to execute sysctl using /etc/sysctl.conf to dictate the values passed to the kernel. Any values added to /etc/sysctl.conf will take effect each time the system boots.
Example configuration: from: Installing Oracle 91 on Linux ----------------------------------------------------------For Linux, use the ipcs command to obtain a list of the system's current shared memory segments and semaphore sets, and their identification numbers and owner. Perform the following steps to modify the kernel parameters by using the /proc file system. Log in as the root user. Change to the /proc/sys/kernel directory. Review the current semaphore parameter values in the sem file by using the cat or more utility. For example, using the cat utility, enter the following command: # cat sem The output lists, in order, the values for the SEMMSL, SEMMNS, SEMOPM, and SEMMNI parameters. The following example shows how the output appears: 250 32000 32 128 In the preceding output parameter, 32000 is the SEMMNS parameter, 32 is the value of the SEMMNI
example, 250 is the value of the SEMMSL value of the the value of the SEMOPM parameter, and 128 is parameter.
Modify the parameter values by using the following command syntax: # echo SEMMSL_value SEMMNS_value SEMOPM_value SEMMNI_value > sem Replace the parameter variables with the values for your system in the order that they are entered in the preceding example. For example:
# echo 100 32000 100 100 > sem Review the current shared memory parameters by using the cat or more utility. For example, using the cat utility, enter the following command: # cat shared_memory_parameter In the preceding example, the variable shared_memory_parameter is either the SHMMAX or SHMMNI parameter. The parameter name must be entered in lowercase letters. Modify the shared memory parameter by using the echo utility. For example, to modify the SHMMAX parameter, enter the following command: # echo 2147483648 > shmmax Modify the shared memory parameter by using the echo utility. For example, to modify the SHMMNI parameter, enter the following command: # echo 4096 > shmmni Modify the shared memory parameter by using the echo utility. For example, to modify the SHMALL parameter, enter the following command: # echo 2097152 > shmall Write a script to initialize these values during system startup, and include the script in your system init files. See Also: Your system vendor's documentation for more information on script files and init files. Set the File Handles by using ulimit -n and /proc/sys/fs/file-max. # echo 65536 > /proc/sys/fs/file-max ulimit -n 65536 Set the Sockets to /proc/sys/net/ipv4/ip_local_port_range # echo 1024 65000 > /proc/sys/net/ipv4/ip_local_port_change Set the Process limit by using ulimit -u. This will give you the number of processes per user. ulimit -u 16384 39.4 Linux modules: ------------------Modules on Linux (1):
--------------------- insmod, rmmod, lsmod lsmod: -----lsmod - list loaded modules. SYNOPSIS lsmod [-hV] DESCRIPTION lsmod shows information about all loaded modules. The format is name, size, use count, list of referring modules. The information displayed is identical to that available from "/proc/modules". If the module controls its own unloading via a can_unload routine then the user count displayed by lsmod is always -1, irrespective of the real use count. insmod: ------insmod - install loadable kernel module SYNOPSIS insmod [-fhkLmnpqrsSvVxXyYN] [-e persist_name] [-o module_name] [-O blob_name] [-P prefix] module [ symbol=value ... ] DESCRIPTION insmod installs a loadable module in the running kernel. insmod tries to link a module into the running kernel by resolving all symbols from the kernel's exported symbol table. If the module file name is given without directories or extension, insmod will search for the module in some common default directories. The environment variable MODPATH can be used to override this default. If a module configuration file such as /etc/modules.conf exists, it will override the paths defined in MODPATH. The environment variable MODULECONF can also be used to select a different configuration file from the default /etc/modules.conf (or /etc/conf.modules (deprecated)). This environment variable will override all the definitions above. When environment variable UNAME_MACHINE is set, modutils will use its value instead of the machine field from the uname() syscall. This is mainly of use when you are compiling 64 bit modules in 32 bit user space or vice versa, set UNAME_MACHINE to the type of the modules. Current modutils does not support full cross build mode for modules, it is limited to choosing between 32 and 64 bit versions of the host architecture.
rmmod: -----rmmod - unload loadable modules SYNOPSIS rmmod [ -aehrsvV ] module ... DESCRIPTION rmmod unloads loadable modules from the running kernel. rmmod tries to unload a set of modules from the kernel, with the restriction that they are not in use and that they are not referred to by other modules. If more than one module is named on the command line, the modules will be removed in the given order. This supports unloading of stacked modules. With the option '-r', a recursive removal of modules will be attempted. This means that if a top module in a stack is named on the command line, all modules that are used by this module will be removed as well, if possible.
More info about the mod commands: --------------------------------- Hardware Detection with the Help of hwinfo hwinfo can detect the hardware of your system and select the drivers needed to run this hardware. Get a small introduction to this command with hwinfo --help. If you, for example, need information about your SCSI devices, use the command hwinfo --scsi. All this information is also available in YaST in the hardware information module. - Handling Modules The following commands are available: insmod insmod loads the requested module after searching for it in a subdirectory of /lib/modules/. It is better, however, to use modprobe rather than insmod. rmmod Unloads the requested module. This is only possible if this module is no longer needed. For example, the isofs module cannot be unloaded while a CD is still mounted. depmod Creates the file modules.dep in /lib/modules/ that defines the dependencies of all the modules. This is necessary to ensure that all dependent modules are loaded with the selected ones. This file will be built after the system is started if it does not exist.
modprobe Loads or unloads a given module while taking into account dependencies of this module. This command is extremely powerful and can be used for a lot of things (e.g., probing all modules of a given type until one is successfully loaded). In contrast to insmod, modprobe checks /etc/modprobe.conf and therefore is the preferred method of loading modules. For detailed information about this topic, refer to the corresponding man page. lsmod Shows which modules are currently loaded as well as how many other modules are using them. Modules started by the kernel daemon are tagged with autoclean. This label denotes that these modules will automatically be removed once they reach their idle time limit. modinfo Shows module information. /etc/modprobe.conf The loading of modules is affected by the files /etc/modprobe.conf and /etc/modprobe.conf.local and the directory /etc/modprobe.d. See man modprobe.conf. Parameters for modules that access hardware directly must be entered in this file. Such modules may need system-specific options (e.g., CD-ROM driver or network driver). The parameters used here are described in the kernel sources. Install the package kernel-source and read the documentation in the directory /usr/src/linux/Documentation. Kmod - the Kernel Module Loader The kernel module loader is the most elegant way to use modules. Kmod performs background monitoring and makes sure the required modules are loaded by modprobe as soon as the respective functionality is needed in the kernel. To use Kmod, activate the option `Kernel module loader' (CONFIG_KMOD) in the kernel configuration. Kmod is not designed to unload modules automatically; in view of today's RAM capacities, the potential memory savings would be marginal. For reasons of performance, monolithic kernels may be more suitable for servers that are used for special tasks and need only a few drivers. modprobe.conf: -------------Example 1: # This file is autogenerated from /etc/modules.conf using generatemodprobe.conf command
alias eth1 sk98lin alias eth0 ipw2200 alias sound-slot-0 snd-hda-intel install scsi_hostadapter /sbin/modprobe ahci; /bin/true remove snd-hda-intel /sbin/modprobe -r snd-pcm-oss; /sbin/modprobe --first-time -r --ignore-remove snd-hda-intel install snd-hda-intel /sbin/modprobe --first-time --ignore-install sndhda-intel && { /sbin/modprobe snd-pcm-oss; /bin/true; } install usb-interface /sbin/modprobe uhci-hcd; /sbin/modprobe ehci-hcd; /bin/true #alias eth1 eth1394 alias ieee1394-controller ohci1394 alias net-pf-10 off #irda alias tty-ldisc-11 irtty alias char-major-161-* ircomm-tty # Para nsc 383 SIO: alias char-major-160-* nsc-ircc alias irda0 nsc-ircc options nsc-irc io=0x2f8 irq=3 dma=0 install nsc-ircc { /bin/setserial /dev/ttyS1 uart none; } ; /sbin/modprobe --first-time --ignore-install nsc-ircc #irda: 0x2f8, irq 3, dma 0 #lpt: 0x3f8, irq 7, dma 1 options parport_pc io=0x378 irq=7 dma=1 Example 2: alias ieee1394-controller ohci1394 alias eth0 eepro100 alias sound-slot-0 emu10k1 alias net-pf-10 off install snd-emu10k1 /sbin/modprobe --first-time --ignore-install sndemu10k1 && { /sbin/modprobe snd-pcm-oss; /bin/true; } install usb-interface /sbin/modprobe usb-uhci; /sbin/modprobe ehci-hcd; /bin/true remove snd-emu10k1 { /sbin/modprobe -r snd-pcm-oss; } ; /sbin/modprobe -r --first-time --ignore-remove snd-emu10k1 /etc/sysconfig: --------------Note 1: ------SuSEconfig and /etc/sysconfig The main configuration of SUSE LINUX can be made with the configuration files in /etc/sysconfig. Former versions of SUSE LINUX relied on /etc/rc.config for system configuration, but it became obsolete
in previous versions. /etc/rc.config is not created at installation time, as all system configuration is controlled by /etc/sysconfig. However, if /etc/rc.config exists at the time of a system update, it remains intact. The individual files in /etc/sysconfig are only read by the scripts to which they are relevant. This ensures that network settings, for instance, need to be parsed only by networkrelated scripts. Apart from that, there are many other system configuration files that are generated according to the settings in /etc/sysconfig. This task is performed by SuSEconfig. For example, if you change the network configuration, SuSEconfig is likely to make changes to the file /etc/host.conf as well, as this is one of the files relevant for the network configuration. If you change anything in these files manually, run SuSEconfig afterwards to make sure all the necessary changes are made in all the relevant places. If you change the configuration using the YaST sysconfig editor, all changes are applied automatically - YaST automatically starts SuSEconfig to update the configuration files as needed. This concept enables you to make basic changes to your configuration without needing to reboot the system. Because some changes are rather complex, some programs must be restarted for the changes to take effect. For instance, changes to the network configuration may require a restart of the network programs concerned. This can be achieved by entering the commands rcnetwork stop and rcnetwork start. Note 2: ------The Linux sysconfig directory The /etc/sysconfig directory is where many of the files that control the system configuration are stored. This section lists these files and many of the optional values in the files used to make system changes. To get complete information on these files read the file /usr/doc/initscripts-4.48/sysconfig.txt. /etc/sysconfig/clock Used to configure the system clock to Universal or local time and set some other clock parameters. An example file: UTC=false ARC=false Options: UTC - true means the clock is set to UTC time otherwise it is at local time
ARC - Set true on alpha stations only. It indicates the ARC console's 42-year time offset is in effect. If not set to true, the normal Unix epoch is assumed. ZONE="filename" - indicates the zonefile under the directory /usr/share/zoneinfo that the /etc/localtime file is a copy of. This may be set to: ZONE="US/Eastern" /etc/sysconfig/init This file is used to set some terminal characteristics and environment variables. A sample listing: # color => new RH6.0 bootup # verbose => old-style bootup # anything else => new style bootup without ANSI colors or positioning BOOTUP=color # column to start "[ OK ]" label in RES_COL=60 # terminal sequence to move to that column. You could change this # to something like "tput hpa ${RES_COL}" if your terminal supports it MOVE_TO_COL="echo -en \\033[${RES_COL}G" # terminal sequence to set color to a 'success' color (currently: green) SETCOLOR_SUCCESS="echo -en \\033[1;32m" # terminal sequence to set color to a 'failure' color (currently: red) SETCOLOR_FAILURE="echo -en \\033[1;31m" # terminal sequence to set color to a 'warning' color (currently: yellow) SETCOLOR_WARNING="echo -en \\033[1;33m" # terminal sequence to reset to the default color. SETCOLOR_NORMAL="echo -en \\033[0;39m" # default kernel loglevel on boot (syslog will reset this) LOGLEVEL=1 # Set to something other than 'no' to turn on magic sysrq keys... MAGIC_SYSRQ=no # Set to anything other than 'no' to allow hotkey interactive startup... PROMPT=yes Options: BOOTUP=bootupmode - Choices are color, or verbose. The choice color sets new boot display. The choice verbose sets old style display. Anything else sets a new display without ANSI formatting. LOGLEVEL=number - Sets the initial console logging level for the kernel. The default is 7. The values are: emergency, panic - System is unusable alert - Action must be taken immediately crit - Critical conditions err, error (depreciated) - Error conditions warning, warn (depreciated) - Warning conditions notice - Normal but significant conditions info - Informational message debug - Debug level message RES_COL=number - Screen column to start status labels at. The Default is 60. MOVE_TO_COL=command - A command to move the cursor to $RES_COL. SETCOLOR_SUCCESS=command - Set the color used to indicate success. SETCOLOR_FAILURE=command - Set the color used to indicate failure. SETCOLOR_WARNING=command - Set the color used to indicate warning. SETCOLOR_NORMAL=command - Set the color used tor normal color
MAGIC_SYSRQ=yes|no - Set to 'no' to disable the magic sysrq key. PROMPT=yes|no - Set to 'no' to disable the key check for interactive mode. /etc/sysconfig/keyboard Used to configure the keyboard. Used by the startup script /etc/rc.d/rc.sysinit. An example file: KEYTABLE="us" Options: KEYTABLE="keytable file" - The line [ KEYTABLE="/usr/lib/kbd/keytables/us.map" ] tells the system to use the file shown for keymapping. KEYBOARDTYPE=sun|pc - The selection, "sun", indicates attached on /dev/kbd is a sun keyboard. The selection "pc" indicates a PS/2 keyboard is on the ps/2 port. /etc/sysconfig/mouse This file is used to configure the mouse. An example file: FULLNAME="Generic - 2 Button Mouse (PS/2)" MOUSETYPE="ps/2" XEMU3="yes" XMOUSETYPE="PS/2" Options: MOUSETYPE=type - Choices are microsoft, mouseman, mousesystems, ps/2, msbm, logibm, atibm, logitech, mmseries, or mmhittab. XEMU3=yes|no - If yes, emulate three buttons, otherwise not. /etc/sysconfig/network Used to configure networking options. All IPX options default to off. An example file: NETWORKING=yes FORWARD_IPV4="yes" HOSTNAME="mdct-dev3" GATEWAY="10.1.0.25" GATEWAYDEV="eth0" Options: NETWORKING=yes|no - Sets network capabilities on or off. HOSTNAME="hostname". To work with old software, the /etc/HOSTNAME file should contain the same hostname. FORWARD_IPV4=yes|no - Turns the ability to perform IP forwarding on or off. Turn it on if you want to use the machine as a router. Turn it off to use it as a firewall or IP masquerading. DEFRAG_IPV4=yes|no - Set this to automatically defragment IPv4 packets. This is good for masquerading, and a bad idea otherwise. It defaults to 'no'. GATEWAY="gateway IP" GATEWAYDEV="gateway device" Possible values include eth0, eth1, or ppp0. NISDOMAIN="nis domain name" IPX=yes|no - Turn IPX ability on or off. IPXAUTOPRIMARY=on|off - Must not be yes or no.
IPXAUTOFRAME=on|off IPXINTERNALNETNUM="netnum" IPXINTERNALNODENUM="nodenum" /etc/sysconfig/static-routes Configures static routes on a network. Used to set up static routing. An example file: eth1 net 192.168.199.0 netmask 255.255.255.0 gw 192.168.199.1 eth0 net 10.1.0.0 netmask 255.255.0.0 gw 10.1.0.153 eth1 net 255.255.255.255 netmask 255.255.255.255 The syntax is: device net network netmask netmask gw gateway The device may be a device name such as eth0 which is used to have the route brought up and down as the device is brought up or down. The value can also be "any" to let the system calculate the correct devices at run time. /etc/sysconfig/routed Sets up dynamic routing policies. An example file: EXPORT_GATEWAY="no" SILENT="yes" Options: SILENT=yes|no EXPORT_GATEWAY=yes|no /etc/sysconfig/pcmcia Used to configure pcmcia network cards. An example file: PCMCIA=no PCIC= PCIC_OPTS= CORE_OPTS= Options: PCMCIA=yes|no PCIC=i82365|tcic PCIC_OPTS=socket driver (i82365 or tcic) timing parameters CORE_OPTS=pcmcia_core options CARDMGR_OPTS=cardmgr options /etc/sysconfig/amd Used to configure the auto mount daemon. An example file: ADIR=/.automount MOUNTPTS='/net /etc/amd.conf' AMDOPTS= Options: ADIR=/.automount (normally never changed) MOUNTPTS='/net /etc/amd.conf' (standard automount stuff) AMDOPTS= (extra options for AMD)
/etc/sysconfig/tape Used for backup tape device configuration. Options: DEV=/dev/nst0 - The tape device. Use the non-rewinding tape for these scripts. For SCSI tapes the device is /dev/nst#, where # is the number of the tape drive you want to use. If you only have one then use nst0. For IDE tapes the device is /dev/ht#. For floppy tape drives the device is /dev/ftape. ADMIN=root - The person to mail to if the backup fails for any reason SLEEP=5 - The time to sleep between tape operations. BLOCKSIZE=32768 - This worked fine for 8mm, then 4mm, and now DLT. An optimal setting is probably the amount of data your drive writes at one time. SHORTDATE=$(date +%y:%m:%d:%H:%M) - A short date string, used in backup log filenames. DAY=$(date +log-%y:%m:%d) - Used for the log file directory. DATE=$(date) - Date string, used in log files. LOGROOT=/var/log/backup - Root of the logging directory LIST=$LOGROOT/incremental-list - This is the file name the incremental backup will use to store the incremental list. It will be $LIST-{some number}. DOTCOUNT=$LOGROOT/.count - For counting as you go to know which incremental list to use. COUNTER=$LOGROOT/counter-file - For rewinding when done...might not use. BACKUPTAB=/etc/backuptab - The file in which we keep our list of backup(s) we want to make. /etc/sysconfig/sendmail An example file: DAEMON=yes QUEUE=1h Options: DAEMON=yes|no - yes implies -bd QUEUE=1h - Given to sendmail as -q$QUEUE. The -q option is not given to sendmail if /etc/sysconfig/sendmail exists and QUEUE is empty or undefined. /etc/sysconfig/i18n Controls the system font settings. The language variables are used in /etc/profile.d/lang.sh. An example i18n file: LANG="en_US" LC_ALL="en_US" LINGUAS="en_US" Options: LANG= set locale for all categories, can be any two letter ISO language code. LC_CTYPE= localedata configuration for classification and conversion of characters. LC_COLLATE= localedata configuration for collation (sort order) of strings. LC_MESSAGES= localedata configuration for translation of yes and no messages.
LC_NUMERIC= localedata configuration for non-monetary numeric data. LC_MONETARY= localedata configuration for monetary data. LC_TIME= localedata configuration for date and time. LC_ALL= localedata configuration overriding all of the above. LANGUAGE= can be a : separated list of ISO language codes. LINGUAS= can be a ' ' separated list of ISO language codes. SYSFONT= any font that is legal when used as /usr/bin/consolechars -f $SYSFONT ... (See console-tools package for consolechars command) UNIMAP= any SFM (screen font map, formerly called Unicode mapping table - see consolechars(8)) /usr/bin/consolechars -f $SYSFONT --sfm $UNIMAP SYSFONTACM= any ACM (application charset map - see consolechars(8)) /usr/bin/consolechars -f $SYSFONT --acm $SYSFONTACM The above is used by the /sbin/setsysfont command (which is run by rc.sysinit at boot time.)
/etc/sysconfig/network-scripts/ifup: /etc/sysconfig/network-scripts/ifdown: These are symbolic links to /sbin/ifup and /sbin/ifdown, respectively. These symlinks are here for legacy purposes only. They will probably be removed in future versions. These scripts take one argument normally: the name of the device (e.g. eth0). They are called with a second argument of "boot" during the boot sequence so that devices that are not meant to be brought up on boot (ONBOOT=no, see below) can be ignored at that time. /etc/sysconfig/network-scripts/network-functions This is not really a public file. Contains functions which the scripts use for bringing interfaces up and down. In particular, it contains most of the code for handling alternative interface configurations and interface change notification through netreport. /etc/sysconfig/network-scripts/ifcfg-interface /etc/sysconfig/network-scripts/ifcfg-interface-clone Defines an interface. An example file called ifcfg-eth0: DEVICE="eth0" IPADDR="10.1.0.153" NETMASK="255.255.0.0" ONBOOT="yes" BOOTPROTO="none" IPXNETNUM_802_2="" IPXPRIMARY_802_2="no" IPXACTIVE_802_2="no" IPXNETNUM_802_3="" IPXPRIMARY_802_3="no" IPXACTIVE_802_3="no" IPXNETNUM_ETHERII="" IPXPRIMARY_ETHERII="no" IPXACTIVE_ETHERII="no" IPXNETNUM_SNAP="" IPXPRIMARY_SNAP="no"
IPXACTIVE_SNAP="no" The /etc/sysconfig/network-scripts/ifcfg-interface-clone file only contains the parts of the definition that are different in a "clone" (or alternative) interface. For example, the network numbers might be different, but everything else might be the same, so only the network numbers would be in the clone file, but all the device information would be in the base ifcfg file. Base items in the above two files: NAME="friendly name for users to see" - Most important for PPP. Only used in front ends. DEVICE="name of physical device" IPADDR= NETMASK= GATEWAY= ONBOOT=yes|no USERCTL=yes|no BOOTPROTO=none|bootp|dhcp - If BOOTPROTO is not "none", then the only other item that must be set is the DEVICE item; all the rest will be determined by the boot protocol. No "dummy" entries need to be created. Base items being deprecated: NETWORK="will be calculated automatically with ifcalc" BROADCAST="will be calculated automatically with ifcalc" Ethernet-only items: {IPXNETNUM,IPXPRIMARY,IPXACTIVE}_{802_2,802_3,ETHERII,SNAP} configuration matrix for IPX. Only used if IPX is active. Managed from /etc/sysconfig/network-scripts/ifup-ipx PPP/SLIP items: PERSIST=yes|no MODEMPORT=device - An example device is /dev/modem. LINESPEED=speed - An example speed is 115200. DEFABORT=yes|no - Tells netcfg whether or not to put default abort strings in when creating/editing the chat script and/or dip script for this interface. PPP-specific items WVDIALSECT="list of sections from wvdial.conf to use" - If this variable is set, then the chat script (if it exists) is ignored, and wvdial is used to open the PPP connection. PEERDNS=yes|no - Modify /etc/resolv.conf if peer uses msdns extension. DEFROUTE=yes|no - Set this interface as default route? ESCAPECHARS=yes|no -Simplified interface here doesn't let people specify which characters to escape; almost everyone can use asyncmap 00000000 anyway, and they can set PPPOPTIONS to asyncmap foobar if they want to set options perfectly). HARDFLOWCTL=yes|no - Yes implies "modem crtscts" options. PPPOPTIONS="arbitrary option string" - It is placed last on the command line, so it can override other options like asyncmap that were specified differently. PAPNAME="name $PAPNAME" - On pppd command line. Note that the "remotename" option is always specified as the logical ppp device name, like "ppp0" (which might perhaps be the physical device ppp1 if some other ppp device was brought up earlier...), which makes it easy to manage pap/chap files -- name/password pairs are associated with the logical ppp device name so that they can be managed together. REMIP="remote ip address" - Normally unspecified.
MTU= MRU= DISCONNECTTIMEOUT="number of seconds" The current default is 5. This is the time to wait before re-establishing the connection after a successfully-connected session terminates before attempting to establish a new connection. RETRYTIMEOUT="number of seconds" - The current default is 60. This is the time to wait before re-attempting to establish a connection after a previous attempt fails. /etc/sysconfig/network-scripts/chat-interface - This is the chat script for PPP or SLIP connection intended to establish the connection. For SLIP devices, a DIP script is written from the chat script; for PPP devices, the chat script is used directly. /etc/sysconfig/network-scripts/dip-interface A write-only script created from the chat script by netcfg. Do not modify this. In the future, this file may disappear by default and created on-the-fly from the chat script if it does not exist. /etc/sysconfig/network-scripts/ifup-post Called when any network device EXCEPT a SLIP device comes up. Calls /etc/sysconfig/network-scripts/ifup-routes to bring up static routes that depend on that device. Calls /etc/sysconfig/network-scripts/ifupaliases to bring up aliases for that device. Sets the hostname if it is not already set and a hostname can be found for the IP for that device. Sends SIGIO to any programs that have requested notification of network events. It could be extended to fix up nameservice configuration, call arbitrary scripts, etc, as needed. /etc/sysconfig/network-scripts/ifup-routes Set up static routes for a device. An example file: #!/bin/sh # adds static routes which go through device $1 if [ "$1" = "" ]; then echo "usage: $0 " exit 1 fi if [ ! -f /etc/sysconfig/static-routes ]; then exit 0 fi #note the trailing space in the grep gets rid of aliases grep "^$1 " /etc/sysconfig/static-routes | while read device args; do /sbin/route add -$args $device done /etc/sysconfig/network-scripts/ifup-aliases Bring up aliases for a device.
/etc/sysconfig/network-scripts/ifdhcpc-done Called by dhcpcd once dhcp configuration is complete; sets up /etc/resolv.conf from the version dhcpcd dropped in /etc/dhcpc/resolv.conf Note 3: ------Red Hat Linux 8.0: The Official Red Hat Linux Reference Guide Prev Chapter 3. Boot Process, Init, and Shutdown Next ------------------------------------------------------------------------------The /etc/sysconfig/ Directory The following information outlines some of the files found in the /etc/sysconfig/ directory, their function, and their contents. This information is not intended to be complete, as many of these files have a variety of options that are only used in very specific or rare circumstances. The /usr/share/doc/initscripts-/sysconfig.txt file contains a more authoritative listing of the files found in the /etc/sysconfig directory and the configuration options available. Files in the /etc/sysconfig/ Directory The following files are normally found in the /etc/sysconfig/ directory: amd apmd arpwatch authconfig cipe clock desktop dhcpd firstboot gpm harddisks hwconf i18n identd init ipchains iptables irda keyboard kudzu mouse named netdump network ntpd pcmcia radvd
rawdevices redhat-config-users redhat-logviewer samba sendmail soundcard squid tux ups vncservers xinetd It is possible that your system may be missing a few of them if the corresponding program that would need that file is not installed. Next, we will take a look at each one. /etc/sysconfig/amd The /etc/sysconfig/amd file contains various parameters used by amd allowing for the automounting and automatic unmounting of file systems. /etc/sysconfig/apmd The /etc/sysconfig/apmd file is used by apmd as a configuration for what things to start/stop/change on suspend or resume. It is set up to turn on or off apmd during startup, depending on whether your hardware supports Advanced Power Management (APM) or if you choose not to use it. apm is a monitoring daemon that works with power management code within the Linux kernel. It can alert you to a low battery if you are using Red Hat Linux on a laptop, among other things. /etc/sysconfig/arpwatch The /etc/sysconfig/arpwatch file is used to pass arguments to the arpwatch daemon at boot time. The arpwatch daemon maintains a table of Ethernet MAC addresses and their IP address pairings. For more information about what parameters you can use in this file, type man arpwatch. By default, this file sets the owner of the arpwatch process to the user pcap. /etc/sysconfig/authconfig The /etc/sysconfig/authconfig file sets the kind of authorization to be used on the host. It contains one or more of the following lines: USEMD5=, where is one of the following: yes - MD5 is used for authentication. no - MD5 is not used for authentication. USEKERBEROS=, where is one of the following: yes - Kerberos is used for authentication. no - Kerberos is not used for authentication.
USELDAPAUTH=, where is one of the following: yes - LDAP is used for authentication. no - LDAP is not used for authentication. /etc/sysconfig/clock The /etc/sysconfig/clock file controls the interpretation of values read from the system hardware clock. The correct values are: UTC=, where is one of the following boolean values: true or yes - Indicates that the hardware clock is set to Universal Time. false or no - Indicates that the hardware clock is set to local time. ARC=, where is the following: true or yes - Indicates the ARC console's 42-year time offset is in effect. This setting is only for ARC- or AlphaBIOS-based Alpha systems. Any other value indicates that the normal UNIX epoch is in use. SRM=, where is the following: true or yes - Indicates the SRM console's 1900 epoch is in effect. This setting is only for SRM-based Alpha systems. Any other value indicates that the normal UNIX epoch is in use. ZONE= - Indicates the timezone file under /usr/share/zoneinfo that /etc/localtime is a copy of, such as: ZONE="America/New York" Earlier releases of Red Hat Linux used the following values (which are deprecated): CLOCKMODE=, where is one of the following: GMT - Indicates that the clock is set to Universal Time (Greenwich Mean Time). ARC - Indicates the ARC console's 42-year time offset is in effect (for Alpha-based systems only). /etc/sysconfig/desktop The /etc/sysconfig/desktop file specifies the desktop manager to be run, such as: DESKTOP="GNOME" /etc/sysconfig/dhcpd
The /etc/sysconfig/dhcpd file is used to pass arguments to the dhcpd daemon at boot time. The dhcpd daemon implements the Dynamic Host Configuration Protocol (DHCP) and the Internet Bootstrap Protocol (BOOTP). DHCP and BOOTP assign hostnames to machines on the network. For more information about what parameters you can use in this file, type man dhcpd. /etc/sysconfig/firstboot Beginning with Red Hat Linux 8.0, the first time you boot the system, the /sbin/init program calls the etc/rc.d/init.d/firstboot script. This allows the user to install additional applications and documentation before the boot process completes. The /etc/sysconfig/firstboot file tells the firstboot command not to run on subsequent reboots. If you want firstboot to run the next time you boot the system, simply remove /etc/sysconfig/firstboot and execute chkconfig --level 5 firstboot on. /etc/sysconfig/gpm The /etc/sysconfig/gpm file is used to pass arguments to the gpm daemon at boot time. The gpm daemon is the mouse server which allows mouse acceleration and middle-click pasting. For more information about what parameters you can use in this file, type man gpm. By default, it sets the mouse device to /dev/mouse. /etc/sysconfig/harddisks The /etc/sysconfig/harddisks file allows you to tune your hard drive(s). You can also use / etc/sysconfig/hardiskhd[a-h], to configure parameters for specific drives. Warning Do not make changes to this file lightly. If you change the default values stored here, you could corrupt all of the data on your hard drive(s). The /etc/sysconfig/harddisks file may contain the following: USE_DMA=1, where setting this to 1 enables DMA. However, with some chipsets and hard drive combinations, DMA can cause data corruption. Check with your hard drive documentation or manufacturer before enabling this. Multiple_IO=16, where a setting of 16 allows for multiple sectors per I/O interrupt. When enabled, this feature reduces operating system overhead by 30-50%. Use with caution. EIDE_32BIT=3 enables (E)IDE 32-bit I/O support to an interface card. LOOKAHEAD=1 enables drive read-lookahead. EXTRA_PARAMS= specifies where extra parameters can be added.
/etc/sysconfig/hwconf The /etc/sysconfig/hwconf file lists all the hardware that kudzu detected on your system, as well as the drivers used, vendor ID and device ID information. The kudzu program detects and configures new and/or changed hardware on a system. The /etc/sysconfig/hwconf file is not meant to be manually edited. If you do edit it, devices could suddenly show up as being added or removed. /etc/sysconfig/i18n The /etc/sysconfig/i18n file sets the default language, such as: LANG="en_US" /etc/sysconfig/identd The /etc/sysconfig/identd file is used to pass arguments to the identd daemon at boot time. The identd daemon returns the username of processes with open TCP/IP connections. Some services on the network, such as FTP and IRC servers, will complain and cause slow responses if identd is not running. But in general, identd is not a required service, so if security is a concern, you should not run it. For more information about what parameters you can use in this file, type man identd. By default, the file contains no parameters. /etc/sysconfig/init The /etc/sysconfig/init file controls how the system will appear and function during the boot process. The following values may be used: BOOTUP=, where is one of the following: BOOTUP=color means the standard color boot display, where the success or failure of devices and services starting up is shown in different colors. BOOTUP=verbose means an old style display, which provides more information than purely a message of success or failure. Anything else means a new display, but without ANSI-formatting. RES_COL=, where is the number of the column of the screen to start status labels. Defaults to 60. MOVE_TO_COL=, where moves the cursor to the value in the RES_COL line. Defaults to ANSI sequences output by echo -e. SETCOLOR_SUCCESS=, where sets the color to a color indicating success. Defaults to ANSI sequences output by echo -e, setting the color to green.
SETCOLOR_FAILURE=, where sets the color to a color indicating failure. Defaults to ANSI sequences output by echo -e, setting the color to red. SETCOLOR_WARNING=, where sets the color to a color indicating warning. Defaults to ANSI sequences output by echo -e, setting the color to yellow. SETCOLOR_NORMAL=, where sets the color to 'normal'. Defaults to ANSI sequences output by echo -e. LOGLEVEL=, where sets the initial console logging level for the kernel. The default is 7; 8 means everything (including debugging); 1 means nothing except kernel panics. syslogd will override this once it starts. PROMPT=, where is one of the following boolean values: yes - Enables the key check for interactive mode. no - Disables the key check for interactive mode. /etc/sysconfig/ipchains The /etc/sysconfig/ipchains file contains information used by the kernel to set up ipchains packet filtering rules at boot time or whenever the service is started. This file is modified by typing the command /sbin/service ipchains save when valid ipchains rules are in place. You should not manually edit this file. Instead, use the /sbin/ipchains command to configure the necessary packet filtering rules and then save the rules to this file using /sbin/service ipchains save. Use of ipchains to set up firewall rules is not recommended as it is deprecated and may disappear from future releases of Red Hat Linux. If you need a firewall, you should use iptables instead. /etc/sysconfig/iptables Like /etc/sysconfig/ipchains, the /etc/sysconfig/iptables file stores information used by the kernel to set up packet filtering services at boot time or whenever the service is started. You should not modify this file by hand unless you are familiar with how to construct iptables rules. The simplest way to add rules is to use the /usr/sbin/lokkit command or the gnome-lokkit graphical application to create your firewall. Using these applications will automatically edit this file at the end of the process. If you wish, you can manually create rules using /sbin/iptables and then type /sbin/service iptables save to add the rules to the /etc/sysconfig/iptables file. Once this file exists, any firewall rules saved there will persist through a system reboot or a service restart. For more information on iptables see Chapter 13.
/etc/sysconfig/irda The /etc/sysconfig/irda file controls how infrared devices on your system are configured at startup. The following values may be used: IRDA=, where is one of the following boolean values: yes - irattach will be run, which periodically checks to see if anything is trying to connect to the infrared port, such as another notebook computer trying to make a network connection. For infrared devices to work on your system, this line must be set to yes. no - irattach will not be run, preventing infrared device communication. DEVICE=, where is the device (usually a serial port) that handles infrared connections. DONGLE=, where specifies the type of dongle being used for infrared communication. This setting exists for people who use serial dongles rather than real infrared ports. A dongle is a device that is attached to a traditional serial port to communicate via infrared. This line is commented out by default because notebooks with real infrared ports are far more common than computers with add-on dongles. DISCOVERY=, where is one of the following boolean values:d yes - Starts irattach in discovery mode, meaning it actively checks for other infrared devices. This needs to be turned on for the machine to be actively looking for an infrared connection (meaning the peer that does not initiate the connection). no - Does not start irattach in discovery mode. /etc/sysconfig/keyboard The /etc/sysconfig/keyboard file controls the behavior of the keyboard. The following values may be used: KEYBOARDTYPE=sun|pc, which is used on SPARCs only. sun means a Sun keyboard is attached on /dev/kbd, and pc means a PS/2 keyboard connected to a PS/2 port. KEYTABLE=, where is the name of a keytable file. For example: KEYTABLE="us". The files that can be used as keytables start in /lib/kbd/keymaps/i386 and branch into different keyboard layouts from there, all labeled .kmap.gz. The first file found beneath /lib/kbd/keymaps/i386that matches the KEYTABLE setting is used. /etc/sysconfig/kudzu The /etc/sysconfig/kuzdu allows you to specify a safe probe of your system's hardware by kudzu at boot time. A safe probe is one that disables serial port probing. SAFE=, where is one of the following:
yes - kuzdu does a safe probe. no - kuzdu does a normal probe. /etc/sysconfig/mouse The /etc/sysconfig/mouse file is used to specify information about the available mouse. The following values may be used: FULLNAME=, where refers to the full name of the kind of mouse being used. MOUSETYPE=, where is one of the following: microsoft - A MicrosoftT mouse. mouseman - A MouseManT mouse. mousesystems - A Mouse SystemsT mouse. ps/2 - A PS/2 mouse. msbm - A MicrosoftT bus mouse. logibm - A LogitechT bus mouse. atibm - An ATIT bus mouse. logitech - A LogitechT mouse. mmseries - An older MouseManT mouse. mmhittab - An mmhittab mouse. XEMU3=, where is one of the following boolean values: yes - The mouse only has two buttons, but three mouse buttons should be emulated. no - The mouse already has three buttons.
XMOUSETYPE=, where refers to the kind of mouse used when X is running. The options here are the same as the MOUSETYPE setting in this same file. DEVICE=, where is the mouse device. In addition, /dev/mouse is a symbolic link that points to the actual mouse device. /etc/sysconfig/named The /etc/sysconfig/named file is used to pass arguments to the named daemon at boot time. The named daemon is a Domain Name System (DNS) server which implements the Berkeley Internet Name Domain (BIND)
version 9 distribution. This server maintains a table of which hostnames are associated with IP addresses on the network. Currently, only the following values may be used: ROOTDIR="", where refers to the full directory path of a configured chroot environment under which named will run. This chroot environment must first be configured. Type info chroot for more information on how to do this. OPTIONS="", where any option listed in the man page for named except -t. In place of -t, use the ROOTDIR line above instead. For more information about what parameters you can use in this file, type man named. For detailed information on how to configure a BIND DNS server, see Chapter 16. By default, the file contains no parameters. /etc/sysconfig/netdump The /etc/sysconfig/netdump file is the configuration file for the /etc/init.d/netdump service. The netdump service sends both oops data and memory dumps over the network. In general, netdump is not a required service, so you should only run it if you absolutely need to. For more information about what parameters you can use in this file, type man netdump. /etc/sysconfig/network The /etc/sysconfig/network file is used to specify information about the desired network configuration. The following values may be used: NETWORKING=, where is one of the following boolean values: yes - Networking should be configured. no - Networking should not be configured. HOSTNAME=, where should be the Fully Qualified Domain Name (FQDN), such as hostname.domain.com, but can be whatever hostname you want. Note For compatibility with older software that people might install (such as trn), the /etc/HOSTNAME file should contain the same value as here. GATEWAY=, where is the IP address of the network's gateway. GATEWAYDEV=, where is the gateway device, such as eth0. NISDOMAIN=, where is the NIS domain name. /etc/sysconfig/ntpd The /etc/sysconfig/ntpd file is used to pass arguments to the ntpd daemon at boot time. The ntpd daemon sets and maintains the system clock to synchronize with an Internet standard time server. It implements version 4 of the Network Time Protocol (NTP). For more
information about what parameters you can use in this file, point a browser at the following file: /usr/share/doc/ntp-/ntpd.htm (where is the version number of ntpd). By default, this file sets the owner of the ntpd process to the user ntp. /etc/sysconfig/pcmcia The /etc/sysconfig/pcmcia file is used to specify PCMCIA configuration information. The following values may be used: PCMCIA=, where is one of the following: yes - PCMCIA support should be enabled. no - PCMCIA support should not be enabled. PCIC=, where is one of the following: i82365 - The computer has an i82365-style PCMCIA socket chipset. tcic - The computer has a tcic-style PCMCIA socket chipset. PCIC_OPTS=, where is the socket driver (i82365 or tcic) timing parameters. CORE_OPTS=, where is the list of pcmcia_core options. CARDMGR_OPTS=, where is the list of options for the PCMCIA cardmgr (such as -q for quiet mode; -m to look for loadable kernel modules in the specified directory, and so on). Read the cardmgr man page for more information. /etc/sysconfig/radvd The /etc/sysconfig/radvd file is used to pass arguments to the radvd daemon at boot time. The radvd daemon listens to for router requests and sends router advertisements for the IP version 6 protocol. This service allows hosts on a network to dynamically change their default routers based on these router advertisements. For more information about what parameters you can use in this file, type man radvd. By default, this file sets the owner of the radvd process to the user radvd. /etc/sysconfig/rawdevices The /etc/sysconfig/rawdevices file is used to configure raw device bindings, such as: /dev/raw/raw1 /dev/sda1 /dev/raw/raw2 8 5
/etc/sysconfig/redhat-config-users The /etc/sysconfig/redhat-config-users file is the configuration file for the graphical application, User Manager. Under Red Hat Linux 8.0 this file is used to filter out system users such as root, daemon, or lp. This file is edited by the Preferences => Filter system users and groups pull-down menu in the User Manager application and should not be edited by hand. For more information on using this application, see the
chapter called User and Group Configuration in the Official Red Hat Linux Customization Guide. /etc/sysconfig/redhat-logviewer The /etc/sysconfig/redhat-logviewer file is the configuration file for the graphical, interactive log viewing application, Log Viewer. This file is edited by the Edit => Preferences pull-down menu in the Log Viewer application and should not be edited by hand. For more information on using this application, see the chapter called Log Files in the Official Red Hat Linux Customization Guide. /etc/sysconfig/samba The /etc/sysconfig/samba file is used to pass arguments to the smbd and the nmbd daemons at boot time. The smbd daemon offers file sharing connectivity for Windows clients on the network. The nmbd daemon offers NetBIOS over IP naming services. For more information about what parameters you can use in this file, type man smbd. By default, this file sets smbd and nmbd to run in daemon mode. /etc/sysconfig/sendmail The /etc/sysconfig/sendmail file allows messages to be sent to one or more recipients, routing the message over whatever networks are necessary. The file sets the default values for the Sendmail application to run. Its default values are to run as a background daemon, and to check its queue once an hour in case something has backed up. The following values may be used: DAEMON=, where is one of the following boolean values: yes - Sendmail should be configured to listen to port 25 for incoming mail. yes implies the use of Sendmail's -bd options. no - Sendmail should not be configured to listen to port 25 for incoming mail. QUEUE=1h which is given to Sendmail as -q$QUEUE. The -q option is not given to Sendmail if /etc/sysconfig/sendmail exists and QUEUE is empty or undefined. /etc/sysconfig/soundcard The /etc/sysconfig/soundcard file is generated by sndconfig and should not be modified. The sole use of this file is to determine what card entry in the menu to pop up by default the next time sndconfig is run. Sound card configuration information is located in the /etc/modules.conf file. It may contain the following: CARDTYPE=, where is set to, for example, SB16 for a Soundblaster 16 sound card. /etc/sysconfig/squid The /etc/sysconfig/squid file is used to pass arguments to the squid daemon at boot time. The squid daemon is a proxy caching server for Web client applications. For more information on configuring a squid proxy
server, use a Web browser to open the /usr/share/doc/squid-/ directory (replace with the squid version number installed on your system). By default, this file sets squid top start in daemon mode and sets the amount of time before it shuts itself down. /etc/sysconfig/tux The /etc/sysconfig/tux file is the configuration file for the Red Hat Content Accelerator (formerly known as TUX), the kernel-based web server. For more information on configuring the Red Hat Content Accelerator, use a Web browser to open the /usr/share/doc/tux/tux/index.html (replace with the version number of TUX installed on your system). The parameters available for this file are listed in /usr/share/doc/tux-/tux/parameters.html. /etc/sysconfig/ups The /etc/sysconfig/ups file is used to specify information about any Uninterruptible Power Supplies (UPS) connected to your system. A UPS can be very valuable for a Red Hat Linux system because it gives you time to correctly shut down the system in the case of power interruption. The following values may be used: SERVER=, where is one of the following: yes - A UPS device is connected to your system. no - A UPS device is not connected to your system. MODEL=, where must be one of the following or set to NONE if no UPS is connected to the system: apcsmart - For a APC SmartUPST or similar device. fentonups - For a Fenton UPST. optiups - For an OPTI-UPST device. bestups - For a Best PowerT UPS. genericups - For a generic brand UPS. ups-trust425+625 - For a TrustT UPS. DEVICE=, where specifies where the UPS is connected, such as /dev/ttyS0. OPTIONS=, where is a special command that needs to be passed to the UPS. /etc/sysconfig/vncservers The /etc/sysconfig/vncservers file configures the way the Virtual Network Computing (VNC) server starts up. VNC is a remote display system which allows you to view a desktop environment not only on the machine where it is running but across different networks on a variety of architectures. It may contain the following:
VNCSERVERS=, where is set to something like "1:fred", to indicate that a VNC server should be started for user fred on display : 1. User fred must have set a VNC password using vncpasswd before attempting to connect to the remote VNC server. Note that when you use a VNC server, your communication with it is unencrypted, and so it should not be used on an untrusted network. For specific instructions concerning the use of SSH to secure the VNC communication, please read the information found at http://www.uk.research.att.com/vnc/sshvnc.html. To find out more about SSH, see Chapter 9 or Official Red Hat Linux Customization Guide. /etc/sysconfig/xinetd The /etc/sysconfig/xinetd file is used to pass arguments to the xinetd daemon at boot time. The xinetd daemon starts programs that provide Internet services when a request to the port for that service is received. For more information about what parameters you can use in this file, type man xinetd. For more information on the xinetd service, see the Section called Access Control Using xinetd in Chapter 8. Directories in the /etc/sysconfig/ Directory The following directories are normally found in /etc/sysconfig/ and a basic description of what they contain: apm-scripts - This contains the Red Hat APM suspend/resume script. You should not edit this file directly. If you need customization, simple create a file called /etc/sysconfig/apm-scripts/apmcontinue and it will be called at the end of the script. Also, you can control the script by editing /etc/sysconfig/apmd. cbq - This directory contains the configuration files needed to do Class Based Queuing for bandwidth management on network interfaces. networking - This directory is used by the Network Administration Tool (redhat-config-network) and its contents should not be edited manually. For more information about configuring network interfaces using the Network Administration Tool, see the chapter called Network Configuration in the Official Red Hat Linux Customization Guide. network-scripts - This directory contains the following network-related configuration files: Network configuration files for each configured network interface, such as ifcfg-eth0 for the eth0 Ethernet interface. Scripts used to bring up and down network interfaces, such as ifup and ifdown. Scripts used to bring up and down ISDN interfaces, such as ifup-isdn and ifdown-isdn Various shared network function scripts which should not be edited directly.
For more information on the network-scripts directory, see Chapter 12 rhn - This directory contains the configuration files and GPG keys for the Red Hat Network. No files in this directory should be edited by hand. For more information on the Red Hat Network, see the Red Hat Network website at the following URL: https://rhn.redhat.com.
39.5 More on AIX kernel parameters: ----------------------------------Througout this document, you can find many AIX kernel parameter statements. Most commands are related to retrieving or changing attributes on the sys0 object. Please see section 9.2 for a complete description. For example, take a look at the following example: maxuproc: Values: Display: Change:
Specifies the maximum number of processes per user ID. Default: 40; Range: 1 to 131072 lsattr -E -l sys0 -a maxuproc chdev -l sys0 -a maxuproc=NewValue Change takes effect immediately and is preserved over boot. If value is reduced, then it goes into effect only after a system boot. Diagnosis: Users cannot fork any additional processes. Tuning: This is a safeguard to prevent users from creating too many processes. Kernel Tunable Parameters Following are kernel parameters, grouped into the following sections: -Scheduler and Memory Load Control Tunable Parameters -Virtual Memory Manager Tunable Parameters -Synchronous I/O Tunable Parameters -Asynchronous I/O Tunable Parameters -Disk and Disk Adapter Tunable Parameters -Interprocess Communication Tunable Parameters -Scheduler and Memory Load Control Tunable Parameters -Most of the scheduler and memory load control tunable parameters are fully described in the schedo man page. -The following are a few other related parameters:
40. NFS: ======== On Solaris: -----------
NFS uses a number of deamons to handle its services. These services are initialized at startup from the "/etc/init.d/nfs.server" and "/etc/init.d/nfs.client" startup scripts. nfsd: handles filesystem exporting and file access from remote systems mountd: handles mount requests from nfs clients. provides also info about which filesystems are mounted by which clients. use the showmount command to view this information. lockd: runs on nfs server and nfs clients and provides locking services statd: runs on nfs server and nfs clients and provides crash and recovery functions for lockd rpcbind: facilitates the initial connection between client and server nfslogd: provides logging On AIX: ------To start the NFS daemons for each system, whether client or Server, you can use either # smitty mknfs # mknfs -N (or -B or -I) The mknfs command configures the system to rum the NFS daemons. The command also adds an entry to the /etc/inittab file, so that the /etc/rc.nsf file is executed on system restart. mknfs flags: -B: adds an entry to the inittab and it also executes /etc/rc.nsf to start the daemons now. -I: adds an entry to the inittab to execute rc.nfs at system restart. -N: executes rc.nfs now to start the daemons. The NFS daemons can be started individually or all at once. To start individual daemons, you can use the System Resource Controller: # startsrc -s daemon, like e.g. # startsrc -s nfsd To start the complete nfs system: (good command) # startsrc -g nfs Exporting NFS directories: To export filesystems using smitty, follow this procedure:
1. Verify that NFS is already running using the command "lssrc -g nfs". The output should indicate that the nfsd and rpc.mountd daemons are active. # lssrc -g nfs Subsystem biod nfsd rpc.mountd rpc.statd rpc.lockd
Group nfs nfs nfs nfs nfs
PID 1234 5678 9101 1213 1516
Status active active active active active
2. To export the dirctory use either # smitty mknfsexp or # mknfsexp or # edit the /etc/exports file, like for example vi /etc/exports /home1 /home2 etc..
41. NETWORK COMMANDS AND FILES: =============================== 41.1 SOLARIS: ============= ifconfig: --------ifconfig enables or disables a network interface, sets its IP address, subnet mask, and sets various other options. syntax: ifconfig interface address options .. up Examples: # ifconfig -a Displays the systems IP address and mac address. # ifconfig en0 128.138.240.1 netmask 255.255.255 up # ifconfig lo0 127.0.0.1 up # ifconfig en0 128.138.243.151 netmask 255.255.255.192 broadcast 128.138.243.191 up An identifier as en0 identifies the network interface to which the command applies.
Some common names are ie0, le0, le1, en0, we0, qe0, hme0, eth0, lan0, lo0 Under Solaris, network interfaces must be attached with "ifconfig interface plumb" before they become configurable. rpcinfo: -------This utility can list all registered RPC services running on a system, for example # rpcinfo -p 192.168.1.21 You can also unregister an rpc service using the -d option, for example #rpcinfo -d sprayd 1 which would stop spayd route: -----The route command defines static routes. Syntax: route [-f] add/delete destination gateway [hop-count] # route add default gateway_ipaddress files: ------ /etc/hostname.interface The file contains the hostname or IP address associated with the networkinterface. Suppose the system is called system1 and the interface is le0 then the file would be "hostname.le0" and contains the entry "system1". - /etc/nodename The file should contain one entry: the hostname of the local machine. - /etc/defaultdomain The file is present if the network uses a name service. The file should contain one entry: the fully qualified Domain name of the administrative domain to which the local host belongs. - /etc/inet/hosts or /etc/hosts This is the well known local hosts file, which resolves names to IP addresses. The /etc/hosts is a symbolic link to /etc/inet/hosts.
- /etc/defaultrouter This file should contain an entry for each router directly connected to the network. - /etc/inetd.conf The inetd deamon runs on behalf of other networkservices. It starts the appropriate server process when a request for that service is received. The /etc/inetd.conf file lists the services that inetd is to provide - /etc/services This file lists the well known ports. - /etc/hosts.equiv This file contains a list of trusted hosts for a remote system, one per line. It has the following structure: system1 system2 user_a If the user attemps to login remotely by using rlogin from one of the hosts listed in this file, the system allows the user to login without a password. ~/.rhosts This file is the user equivalent of /etc/hosts.equiv file. This is normally regarded as a security hole. This file could be found in a user home directory. It could contain the name of a remote host that want, for example, copy files to this host. - /etc/resolv.conf Create or edit /etc/resolv.conf Here you tell it three things: What domain we're in Specify any additional search domains What the nameservers are (it will use them in the order you put them in the file) When you're done it should look something like this: # cat resolv.conf domain yourdomain.com search yourdomain.com search client1.com nameserver 192.168.0.9 nameserver 192.168.0.11
41.2 AIX: =========
41.2.1 Network initialization at boot: -----------------------------------At IPL time, the init process will run the /etc/rc.tcpip after starting the SRC. This is so because in /etc/inittab the following record is present: rctcpip:23456789:wait:/etc/rc.tcpip > /dev/console 2>&1 # Start TCP/IP daemons The /etc/rc.tcpip file is a shell script that uses SRC commands to initialize selected deamons. It can also be executed at any time from the command line. These deamons are: inetd (started by default),gated,routed,named,timed,rwhod There are also deamons specific to the bos or to other applications that can be started through the rc.tcpip file. These deamons are lpd, portmap, sendmail, syslogd (started by default) The subsystems started from rc.tcpip can be stopped and restarted using the stopsrc and startsrc commands. Example: # stopsrc -s inetd To configure tcp/ip use the command # mktcpip or use smitty # # # #
smitty smitty smitty smitty
mktcpip (only for the first time) tcpip inet OR smitty chgenet (for configuring the network interface) configtcp (many advanced options)
or use the Web-based System manager. Smitty uses a number of screens to guide you through the process, As an example of the command, take a look at the following example: # mktcpip -h server1 -a 10.10.10.5 -m 255.255.255.0 -i en0 \ -n 10.10.10.254 -d abc.xyz.nl -g 10.10.10.254 -s -C -A no If you need to further configure your network, use # smitty configtcp 41.2.2 resolving hostnames and /etc/netsvc.conf: -----------------------------------------
The default order in resolving host names is: - BIND/DNS (named) - Network Information Service (NIS) - Local /etc/hosts file The default order can be overwritten by creating the configuration file, /etc/netsvc.conf and specifying the desired order. Both the default and /etc/netsvc.conf can be overwritten with the environment variable NSORDER. You can override the order by creating the /etc/netsvc.conf file with an entry. If /etc/netsvc.conf does not exist, it will be just like you have the following entry: hosts = bind,nis,local You can override the order by changing the NSORDER environment variable. If it is not set, it will be just like you have issued the command: export NSORDER=bind,nis,local the /etc/resolv.conf file: -------------------------If you use name services, you can provide the minimal information needed through the mktcpip command. Typically, the "/etc/resolv.conf" file stores your domain name and name server ip addresses. The mktcpip command creates or updates the /etc/resolv.conf file for you. 41.2.3 Adapter: --------------When an adapter is added to the system, a logical device is created in the ODM, for example Ethernet adapters as follows: # lsdev -Cc adapter | grep ent ent0 Available 10-80 IBM PCI Ethernet Adapter (22100020) ent1 Available 20-60 Gigabit Ethernet-SX PCI Adapter (14100401) So you will have an adapter, and a corresponding interface, like for example The Adapter is : ent0 Then the interface is: en0 To list all interfaces on the system, use: # lsdev -Cc if en0 Defined 10-80
Standard Ethernet Network Interface
en1 et0 et1 lo0
Defined 20-60 Defined 10-80 Defined 20-60 Available
Standard Ethernet Network Interface IEEE 802.3 Ethernet Network INterface IEEE 802.3 Ethernet Network INterface Loopback Network INterface
A corresponding network interface will allow tcpip to use the adapter. Most of the time, we will deal with auto-detectable adapters, but in some cases an interface might need to be created manually with # smitty inet or smitty mkinet To change or view attributes like duplex settings, use # smitty chgenet more info: An Ethernet can have 2 interfaces: Standard ethernet (enX) or IEEE 802.3 (etX). X is the same number in the entX adapter name, like for example ent0 and en0. Only one of these interfaces can be using TCPIP at a time. The adapter ent0 can have en0 and et0 interfaces. An ATM adapter (atmX) can have only one atm interface (atX). For example ATM adapter atm0 has an at0 interface. 41.2.4 Other stuff: ------------------iptrace: -------The iptrace command can be used to record the packets that are exchanged on an interface to and from a remote host. This is like a Solaris snoop facility. Examples 1. To start the iptrace daemon with the System Resource Controller (SRC), enter: startsrc -s iptrace -a "/tmp/nettrace" To stop the iptrace daemon with SRC enter the following: stopsrc -s iptrace 2. To record packets coming in and going out to any host on every interface, enter the command in the following format: iptrace /tmp/nettrace The recorded packets are received on and sent from the local host. All
packet flow between the local host and all other hosts on any interface is recorded. The trace information is placed into the /tmp/nettrace file.
3. To record packets received on an interface from a specific remote host, enter the command in the following format: iptrace - i en0 -p telnet -s airmail /tmp/telnet.trace The packets to be remote hostairmail, over into the /tmp/telnet.trace 4. To record packets host, enter the command
recorded are received on the en0 interface, from the telnet port. The trace information is placed file. coming in and going out from a specific remote in the following format:
iptrace -i en0 -s airmail -b /tmp/telnet.trace The packets to be recorded are received on the en0 interface, from remote hostairmail. The trace information is placed into the /tmp/telnet.trace file.
Adding routes: -------------Use smitty mkroute or use the route add command, like for example: # route add -net 192.168.1 -netmask 255.255.255.0 9.3.1.124 Changing the IP Address: -----------------------You can check the interfaces whether they have IP addresses asigned to them with # ifconfig -a # ifconfig Changing the IP adress: # smitty mktcpip # smitty chinet or use the ifconfig command, like for example: # ifconfig # ifconfig interface # ifconfig interface # ifconfig # ifconfig address