LDAP • ldap_ssl_client_init failed! rc == 113, failureReasonCode == 202

# mksecldap -c -h [server] -a"cn=proxyusers,ou=clients,ou=ldap" -p [password] -d [...] -n 636 -k /opt/IBM/ldap/vault/clientkey.kdb -w [password] -A 
GSKit8.gskcrypt32.ppc.rte
GSKit8.gskcrypt64.ppc.rte
GSKit8.gskssl32.ppc.rte
GSKit8.gskssl64.ppc.rte
gskak.rte
gsksa.rte
gskta.rte
ldap_ssl_client_init failed! rc == 113, failureReasonCode == 202
Keyring file open error
Client presetup check failed.

To Fix:

1. Verify /opt/IBM/ldap/vault/ for the .kdb encryption key. It's probably missing.

HMC • HSCL040E Partition FullSystem cannot be activated with profile all_resources, which is a full system profile.

HSCL040E Partition FullSystem cannot be activated with profile
all_resources, which is a full system profile. It is not possible to have a full system partition on managed system Server-9119-FHB-SN89XXX56 because all of the processors in the system cannot be allocated to a single partition since the processor limit for
partitions is set to 32 processors.



To Fix:

1. Go to HMC Web GUI.

Properties => Advanced => Processor Performance => Next SPPL => Maximum

2. Increase the Maximum value and then the Frame should be powered off and then powered on.


Error Reference:

HSCL040E

LVM • 0516-1736 varyonvg: Unable to determine if the volume group contains an active paging space.

# varyonvg t1vg
PV Status: hdisk45 00068a3afbdd6cda PVACTIVE 
hdisk69 00068a3ahhb868d2 PVACTIVE 
hdisk74 00068a3a20bgdd82 PVACTIVE 
0516-1736 varyonvg: Unable to determine if the volume group contains an
active paging space.

Problem related to a lack of synchronization between VGDA, LVCB and
ODM. 

To Fix:

# synclvodm -vP [vg_name] 

# varyonvg [vg_name]

ISD • Connection authorization failure occurred. Reason: User ID revoked. ERRORCODE=-4214, SQLSTATE=28000

Start of IBM Systems Director Server failed with "Connection authorization failure occurred. Reason: User ID revoked. ERRORCODE=-4214, SQLSTATE=28000" error message.

To fix:

1. Stop Director Server

# smstop

2. Run changePassword to get the password changed
   If you are letting Director manage the passwords
   Run <director-root>/bin/changePassword.sh

# /opt/ibm/director/bin/changePassword.sh

For additional information see:

changePassword tool
http://pic.dhe.ibm.com/infocenter/director/pubs/topic/com.ibm.director.cli.helps.doc/fqm0_r_cli_changepassword.html

3. Stop the ISD managed DB2:

# /opt/ibm/director/bin/smdb stop

4. Start the ISD managed DB2:

# /opt/ibm/director/bin/smdb start

6. Verify the connection:

# /opt/ibm/director/bin/smdbcli.sh connect

5. Start Director Server

# smstart

HACMP • migcheck471: cl_connect() error, nodename="hostname", rc=-1

migcheck471: cl_connect() error, nodename="hostname", rc=-1

cldare: A communication error prevents obtaining the VRMF from remote nodes.
A DARE cannot be run until this is corrected. Please ensure clcomd is running.

To Fix:

1. Check /usr/es/sbin/cluster/etc/rhosts permissions. It must have chmod 600.

# ls -l /usr/es/sbin/cluster/etc/rhosts

-rw-------    1 root     system            1 Feb 27 2009 

2. Check if all boot and service IP addresses are included
in the /usr/es/sbin/cluster/etc/rhosts.

3. If there are missing IPs. Add it and restart clcomd service.

# stopsrc -s clcomdES
# startsrc -s clcomdES

4. If there's still a problem, remove odmcache.


# cd /var/hacmp/odmcache
# rm -r nodname  

* The directory "/var/hacmp/odmcache" is where a copy of HA ODM is kept.


5. Make sure all IP addresses are correct within "/etc/hosts".


6. Configure "/etc/netsvc.conf" with the correct string.

hosts=local4,bind4


7. Verify network interfaces for incorrect IP address.

8.  Make sure that you did not configure persistent IP addresses through AIX. This must be done within HACMP. If you have it configured, it need to be deleted.

6. Run a Discover HACMP-related Information from Configured Nodes.

7. Run "Extended Verification and Synchronization" using the automatically correct errors option.

8. Reboot both nodes

IVM • New install of IVM using serial cable not show login prompt

ENVIRONMENT:
------------

POWER SYSTEM WITHOUT HMC
+ IVM 2.2.x

PROBLEM DESCRIPTION:
--------------------

I have a Power System without HMC and I using serial cable to perform a new install of IVM, but first reboot after install has finished the console stop with a message like this below and I'm not able login in to the system:

"Starting The LWI Nonstop Profile..."

Even if I reinstall IVM the same problem always happens.

ACTION TAKEN:
-------------

First of all you need boot a IVM in maintenance mode using option "Access this Volume Group and start a shell".

Note: For more details for how to boot your lpars in  maintenance mode I recomend read this follow link below:

Booting AIX into Maintenance Mode
http://www-01.ibm.com/support/docview.wss?uid=isg3T1013056

Now chage 'STTY attributes for RUN time' and 'STTY attributes for LOGIN' parameters of vty0 device adding 'clocal' property:

# clear
# export TERM=vt100
# smit chtty -> Change / Show Characteristics of a TTY -> vty0
---------------------------------------------------------------------
                  Change / Show Characteristics of a TTY

Type or select values in entry fields.
Press Enter AFTER making all desired changes.
  
[TOP]                      [Entry Fields]
  TTY                                                 vty0
  TTY type                                            tty
  TTY interface                                       vcon
  Description                                         Asynchronous Terminal
  Status                                              Available
  Location
  Parent adapter                                      vsa0
  PORT number                                        [0]
  Enable LOGIN                                        disable
  TIME before advancing to next port setting         [0]
  TERMINAL type                                      [vt220]
  FLOW CONTROL to be used                            [xon]
  STTY attributes for RUN time                       <k,echoctl,echoke,imaxbel,iexten,clocal] <- Add 'clocal'
  STTY attributes for LOGIN                          [hupcl,cread,echoe,cs8,clocal] <- Add 'clocal'
[...]
---------------------------------------------------------------------

# sync;sync;sync;reboot

After reboot you are able to login into system without any problem.

PureFlex/AIX • How to prepare a USB Mass Storage device to make the first install of VIO Server on PureFlex System

First of all you will need use another AIX machine to prepare the usb mass storage device before you plug in to a PureFlex Power node.

Be sure to have the correct fileset(s) installed before inserting your USB stick. The following file-sets are required to access a USB stick:

# lslpp -L devices.usbif.08025002.rte devices.common.IBM.usb.rte
Fileset                      Level  State  Type  Description (Uninstaller)
----------------------------------------------------------------------------
devices.common.IBM.usb.rte
                          6.1.8.15    C     F    USB System Software
devices.usbif.08025002.rte
                          6.1.8.15    A     F    USB Mass Storage Device
                                                 Software

Insert the USB stick and run cfgmgr to discover it, this command will create the a new USB Mass Storage device:

# lsdev -p usb0
usbms0          Available 2.3         USB Mass Storage

Note: For AIX 5.3, the first USB device is shown as flashdrive0.

At this point you can use the flash drive like a tape device and use the backup/restore, tar, cpio and dd commands to write to the flash drive.

---------------------------------------------------------------------------------------------------------------
Note: For more information about usb mass storage device supported by PureFlex System and Power System, please read this two links below:

AIX and USB memory sticks
https://www.ibm.com/developerworks/aix/library/au-flashdrive

Supported USB devices
http://pic.dhe.ibm.com/infocenter/flexsys/information/topic/com.ibm.acc.7895.doc/usb.html
---------------------------------------------------------------------------------------------------------------
 
Mount the VIOS base DVD and copy the VIOS mksysb image from the DVD (in /usr/sys/inst.images) to your AIX machine.

If using VIOS 1.5 or higher media, the mksysb file may be split into two parts. To combine these two parts and copy them to hard disk, run the following:

# cat /mnt/usr/sys/inst.images/mksysb_image /mnt/usr/sys/inst.images/mksysb_image2 > /export/mksysb/mksysb_image

Note: You can substitute any path you would like to save the combined mksysb image, for ‘/export//mksysb/mksysb_image'.

If using VIOS 2.2.0.0 installation media or later, there is now a 3rd mksysb image in /usr/sys/inst.images directory on volume 2. You need to copy the two mksysb images from /usr/sys/inst.images on volume 1 of the installation media to hard disk. Then mount volume 2 of the installation media and copy the third mksysb image from the /usr/sys/inst.images directory to hard disk. Once you have done this, you can use the 'cat' command to combine all 3 into one file for the mksysb resource on the AIX machine

# cat /export/mksysb/vios2.2/mksysb_image /export/mksysb/vios2.2/mksysb_image2 /export/mksysb/vios2.2/mksysb_image3 > /export/mksysb/nim_vios2.2mksysb
Note: You can substitute any path you would like to save the combined mksysb image, for ‘/export/mksysb/vios2.2/'.

Use mkdvd function of AIX to make a ISO file based on the mksysb of VIO collected of installation DVD:

# smit mkdvd
---------------------------------------------------------------------
                 Back Up This System to ISO9660 DVD

Type or select values in entry fields.
Press Enter AFTER making all desired changes.

                            [Entry Fields]
  DVD-R or DVD-RAM Device                            []
* Location of existing mksysb image                  [/export/mksysb/nim_vios2.2mksysb] <- Location of VIO mksysb
 
  File system to store DVD file structure            []
     (If blank, the file system
       will be created for you.)
 
  File system to store final DVD images              []
     (If blank, the file system
       will be created for you.)
 
  If file systems are being created:
    Volume Group for created file systems            [rootvg]
 
  Advanced Customization Options:
  Do you want the DVD to be bootable?                 yes
  Remove final images after creating DVD?             no <- Change this option of 'yes' to 'no'
  Create the DVD now?                                 no <- Change this option of 'yes' to 'no'
  Install bundle file                                []
  File with list of packages to copy to DVD          []
  Location of packages to copy to DVD                []
  Customization script                               []
  User supplied bosinst.data file                    []
  Debug output?                                       no
  User supplied image.data file                      []
---------------------------------------------------------------------

When this process finish a ISO file will be generated on /mkcd/cd_fs filesystem like this example bellow:

# ls -l /mkcd/cd_fs
total 7100352
-rw-r--r--    1 root     system   3635378176 Oct 03 14:13 cd_image_27066408

"Burn" this new ISO file in to the usb mass storage device using dd command:

# dd if=/mkcd/cd_fs/cd_image_27066408 of=/dev/usbms0 bs=4k

Test the usb mass storage device with mount command and list his content:

# mount -rv cdrfs /dev/usbms0 /mnt
# ls -l /mnt
total 80
-r--r--r--    1 root     system           24 Oct 03 11:11 OSLEVEL
dr-xr-xr-x    3 root     system         2048 Oct 03 11:02 RPMS
-r--r--r--    1 root     system         5576 Oct 04 2012  bosinst.data
dr-xr-xr-x    3 root     system         2048 Oct 03 11:03 etc
-r--r--r--    1 root     system        11252 Oct 04 2012  image.data
dr-xr-xr-x    3 root     system         2048 Oct 03 11:02 installp
dr-xr-xr-x    3 root     system         2048 Oct 03 11:02 ismp
-r--r--r--    1 root     system           52 Oct 03 11:11 mkcd.data
dr-xr-xr-x    3 root     system         2048 Oct 03 11:03 ppc
dr-xr-xr-x    4 root     system         2048 Oct 03 11:03 root
dr-xr-xr-x    3 root     system         2048 Oct 03 11:03 sbin
dr-xr-xr-x    3 root     system         2048 Oct 03 11:02 udi
dr-xr-xr-x    9 root     system         2048 Oct 03 11:03 usr

You are now ready to plug in this usb mass storage device on the usb port on front of PureFlex System Power node and perform a new installation of VIO on him.

NIM • 0042-175 c_installp: An unexpected result was returned by the "/usr/lib/instl/sm_inst" command: echo: No space left on device

0042-175 c_installp: An unexpected result was returned by the "/usr/lib/instl/sm_inst" command:
echo: No space left on device

This error can happen during filesets update using NIM.

To Fix:

1. Verify /usr for space left. 

2. If there's not enough free space, please increase it.

eg.

# chfs /usr -a size=+1G

AIX • rpmQuery: rpmdbOpen() failed

# oslevel -s
cannot get shared lock on database
rpmQuery: rpmdbOpen() failed
6100-07-01-1141

To Fix:

1. Verify if there's any rpm process active.

# ps -ef | grep rpm 

2. If there's "rpm" process, kill them all.

# kill -9 [process number] [...]

3. Rebuild RPM DB.

# rpm --rebuilddb

HMC/DLPAR • HSCL2931 DLPAR ADD MEMORY RESOURCE FAILED: 0931-016 There are no dynamically reconfigurable LMBs available.

HSCL2931 DLPAR ADD MEMORY RESOURCE FAILED:

0931-016 There are no dynamically reconfigurable LMBs available.

1. Reboot HMC and try again.

or

2. Execute on HMC

:~> rsthwres -r mem -m [managed_system]

Reference:

HSCL2931

TSA • 2632-044 The domain cannot be created due to the following errors that were detected while harvesting information from the target nodes

# mkrpdomain tsadomain tsanode1 tsanode2
2632-044 The domain cannot be created due to the following errors that were detected while harvesting information from the target nodes:
tsanode2: 2610-441 Permission is denied to access the resource class specified in this command.
Network Identity UNAUTHENT requires ‘s’ permission for the resource class IBM.PeerDomain on node tsanode2.

This kind of error occurs when the command “preprpnode” is not executed on all nodes.

To Fix:

# preprpnode [node1] [node2] […]

TSA • 2621-076 The move action is rejected because the target group is only allowed to run on a single node. rgreq: 2622-009 An unexpected RMC error occurred.The RMC return code was 1. & 2622-463 request action on resource group failed with return code “1”.

# rgreq -o move -n ossaixnim tsarg
(runact-api) 2621-076 The move action is rejected because the target group is only allowed to run on a single node. 
rgreq: 2622-009 An unexpected RMC error occurred.The RMC return code was 1.
rgreq: 2622-463 request action on resource group failed with return code “1”.

 This error occurs when there are missing nodes into the “AllowedNode” attribute withing a TSA resource group.

To verify:

# lsrg -g tsarg
Displaying Resource Group information:
For Resource Group “tsarg”.

Resource Group 1:
Name = tsarg
MemberLocation = Collocated
Priority = 0
AllowedNode = tsanode1
NominalState = Online
ExcludedList = {}
Subscription = {}
Owner =
Description =
InfoLink =
Requests = {[0,0,Operator,{0}„,0e02ba3ec694dea24 05249acab 034430]}
Force = 0
ActivePeerDomain = tsadomain
OpState = Online
TopGroup = tsarg
ConfigValidity =
TopGroupNominalState = Online

To Fix:

change “AllowedNode” to ALL

1. stop RG on all nodes

2. chrg -e ALL tsarg 

3. lsrg -g tsarg

Displaying Resource Group information:
For Resource Group “tsarg”.

Resource Group 1:
Name = tsarg
MemberLocation = Collocated
Priority = 0
AllowedNode = ALL
NominalState = Offline
ExcludedList = {}
Subscription = {}
Owner =
Description =
InfoLink =
Requests = {[0,1,Operator,{0}„,0e02ba3ec694dea24 05249ae92 086a8b]}
Force = 0
ActivePeerDomain = tsadomain
OpState = Offline
TopGroup = tsarg
ConfigValidity =
TopGroupNominalState = Offline

TSA • 2610-415 Cannot execute the command. The resource manager IBM.GblResRM is not available.

# mkrsrc -f template.out IBM.ServiceIP2610-415 Cannot execute the command. The resource manager IBM.GblResRM is not available.

Verify if the TSA domain is active. If it’s not, start it.

To verify:

# lsrpdomain -l 

eg.

# lsrpdomain -l

Name = tsadomain
OpState = Offline
RSCTActiveVersion = 3.1.4.4
MixedVersions = No
TSPort = 12347
GSPort = 12348

To Fix:

# startrpdomain [domain_name]

TSA • 2610-377 A command argument specified the node name [hostname], but this is not a configured node name.

# mkrsrc -f template.out IBM.ServiceIP2610-377 A command argument specified the node name aixtest1, but this is not a configured node name.

To fix this you must perform this initial configuration:

On all nodes, set and export the environment variable
CT_MANAGEMENT_SCOPE to 2 (peer domain scope) for all users.

export CT_MANAGEMENT_SCOPE=2

or

Add the line below to /etc/environment.

CT_MANAGEMENT_SCOPE=2

FSM 1.2.X or Higher • DB2SET processing complete, rc = 4, SQLCODE = 4 SQL6048N A communication error occurred during START or STOP DATABASE MANAGER processing.

Environemnt

FSM 1.2.X or higher

 

Problem Description

 

Command “smstart” failure within FSM. This kind of error most of times can happen after a hardware problem, power outage, etc.

Error

USERID@fsm:~> smstart

DB2SET processing complete, rc = 4, SQLCODE = 4 SQL6048N A communication error occurred during START or STOP DATABASE MANAGER processing.

Starting IBM Director…The starting process may take a while. Please use smstatus to check if the server is active.

 

Action Taken

 

First off, you will need the ‘pesh’ password to solve this problem, so you will need to call IBM Support and ask how to obtain this password.

IBM will request the output of the “lsconfig” command.

 

# lsconfig -v

vpd=*FC ????????

*N2 Tue Jul 30 09:58:18 BRT 2013

*DS FSM

*TM -[795501M]-

*SE 1024CEB

*MN IBM

*OS Embedded Operating Systems

*NA Unknown

*RM 1.2.1

*UUID 682890CB-28AF-499E-A9E7-1341E0E93A73

*UVMID 657f:782b:a541:f9fe

 

With the pesh password provided by IBM Support you have to login with ‘pe’ user through SSH client (like Putty) and run these commands to gain root access over FSM;

 

pe@fsm1:~> pesh <UVMID>

Password: <- imput pesh password

pe@fsm1:~> su - root

Password: <- imput password of ‘pe’ user

fsm1:~ #

 

Verify if there are only the monitor daemon of DB2 (db2fmcd) running on FSM;

 

# ps -ef |grep -i DB2

root     11997     1  0 Jul26 ?        00:00:14 /opt/ibm/director/db2/bin/db2fmcd

root     22637 22029  0 10:01 pts/1    00:00:00 grep -i DB2

 

Check the output of db2_all command.

# db2_all “echo OK”

ssh: Could not resolve hostname PURESYSTEMFSM: Name or service not known

 

This output indicates that FSM has a problem to resolve the DB2 instance hostname. The hostname can be found in the db2nodes.cfg and the same hostname will be missing from /etc/hosts file. 

# cat /home/db2inst1/sqllib/db2nodes.cfg

0 PURESYSTEMFSM 0

 

# ping PURESYSTEMFSM

ping: unknown host PURESYSTEMFSM

 

# cat /etc/hosts

127.0.0.1 localhost.localdomain localhost

::1 localhost ipv6-localhost ipv6-loopback

fe00::0 ipv6-localnet

ff00::0 ipv6-mcastprefix

ff02::1 ipv6-allnodes

ff02::2 ipv6-allrouters

ff02::3 ipv6-allhosts

 

After confirming that we have a resolution problem and adding 'PURESYSTEMFSM' hostname to /etc/hosts, the db2stop/db2start should work just fine.

 

# cat /etc/hosts

127.0.0.1 localhost.localdomain localhost

<IP ADDRESS> PURESYSTEMFSM

::1 localhost ipv6-localhost ipv6-loopback

fe00::0 ipv6-localnet

ff00::0 ipv6-mcastprefix

ff02::1 ipv6-allnodes

ff02::2 ipv6-allrouters

ff02::3 ipv6-allhosts

 

# db2stop

07/30/2013 10:14:53     0   0   SQL1032N  No start database manager command was issued.

SQL1032N  No start database manager command was issued.  SQLSTATE=57019

 

# db2start

07/30/2013 10:15:09     0   0   SQL1063N  DB2START processing was successful.

SQL1063N  DB2START processing was successful.

 

# ps -ef |grep -i DB2

root     11301 22029  0 10:14 pts/1    00:00:00 su - db2inst1

db2inst1 11303 11301  0 10:14 pts/1    00:00:00 -bash

root     11997     1  0 Jul26 ?        00:00:14 /opt/ibm/director/db2/bin/db2fmcd

root     13087     1  0 10:15 pts/1    00:00:00 db2wdog 0                                      

db2inst1 13089 13087  0 10:15 pts/1    00:00:00 db2sysc 0                                     

root     13090 13089  0 10:15 pts/1    00:00:00 db2ckpwd 0                                     

root     13091 13089  0 10:15 pts/1    00:00:00 db2ckpwd 0                                     

root     13092 13089  0 10:15 pts/1    00:00:00 db2ckpwd 0                                     

db2inst1 13096 13089  0 10:15 pts/1    00:00:00 db2vend (PD Vendor Process - 1)                                                                                           

db2inst1 13120 13087  0 10:15 pts/1    00:00:00 db2acd 0 ,0,0,0,1,0,0,0000,1,0,8a6740,14,1e014,2,0,1,11fc0,0x210000000,0x210000000,1600000,7800a,2,15001b

db2inst1 14693 11303  0 10:16 pts/1    00:00:00 ps -ef

db2inst1 14694 11303  0 10:16 pts/1    00:00:00 grep db2

 

# tail -n 10 /home/db2inst1/sqllib/db2dump/db2diag.log

2013-07-30-10.15.09.094936-180 E58894787E306       LEVEL: Event

PID     : 13085                TID  : 140737352849152PROC : db2star2

INSTANCE: db2inst1             NODE : 000

FUNCTION: DB2 UDB, base sys utilities, DB2StartMain, probe:911

MESSAGE : ADM7513W  Database manager has started.

START   : DB2 DBM

 

# smstart

Starting IBM Director…The starting process may take a while. Please use smstatus to check if the server is active.

# smstatus

Inactive

# smstatus

Starting

# smstatus

Active

[Link Reference]

How to solve db2start error SQL6048N

 

QL6048N communication error while starting the instance which has Logical MPP(Logical Partitioning) configured