Sunday, 16 September 2012

Oracle executable file permission

BRBACKUP fails with the following error messages, for example:
BR0051I BRBACKUP 7.00 (20)
BR0055I Start of database backup: bddzbuxf.ant 2009-11-10 10.12.35
BR0280I Time stamp 2009-11-10 10.12.36
BR0301W SQL error -1031 at location BrLicCheck-7
ORA-01031: insufficient privileges
BR0301W SQL error -942 at location BrbDbLogOpen-1
ORA-00942: table or view does not exist
BR0324W Insertion of database log header failed
BR0280I Time stamp 2009-11-10 10.12.37
BR0301E SQL error -1031 at location BrCntrlCopy-1
ORA-01031: insufficient privileges
BR0320E Copying of control file to ... failed
BR0314E Collection of information on database files failed
BR0056I End of database backup: bddzbuxf.ant 2009-11-10 10.12.37
BR0280I Time stamp 2009-11-10 10.12.37
BR0054I BRBACKUP terminated with errors

select grantee,granted_role from dba_role_privs where grantee='OPS$BIPADM';
select grantee,granted_role from dba_role_privs where grantee='OPS$BIQADM';
SQL> connect /
ERROR:
ORA-27140: attach to post/wait facility failed

SQL> connect /as sysdba
ERROR:
ORA-09925: Unable to create audit trail file
Linux-x86_64 Error: 13: Permission denied
Additional information: 9925
ORA-09925: Unable to create audit trail file
Linux-x86_64 Error: 13: Permission denied
Additional information: 9925
Note 400241 - Problems with ops$ or sapr3 connect to Oracle
Note 50088 - Creating OPS$ users on Windows NT/Oracle
https://forums.oracle.com/forums/thread.jspa?threadID=384361

Number range error after client copy

Problem:
--------
This message "xpress document "Update was terminated" received from author
" is coming while trying to post any FI documents.
Solution
----------
1: Check the details in SM13 transaction.
You can check reason for update termination in SM13.
2:  Most common cause for this error relates to the document number range
Please refer
http://help.sap.com/saphelp_nw04/helpdata/en/95/3d5540b8cdcd01e10000000a155106/content.htm

Restated the SAP application and the nubmer range buffer was cleaned.and the error did not came

SXPG_COMMAND_EXECUTE failed for BRBACKUP

My online backup has been scheduled in DB13
when its reach the backup time its showins error.

Status : Scheduled failed
JOB LOG:
30.11.2012 17:26:56 Job started
30.11.2012 17:26:56 Step 001 started (program RSDBAJOB, variant &0000000000672, user ID BASIS)
30.11.2012 17:26:56 No application server found on database host - rsh/gateway will be used
30.11.2012 17:26:56 Execute logical command BRBACKUP On host PRODORADB
30.11.2012 17:26:56 Parameters:-u / -jid ALLOG20121130172655 -c force -t online -m all -p initPDR.sap -a -c force -p initPRD
30.11.2012 17:26:56 offsite.sap -cds
30.11.2012 17:27:28 SXPG_STEP_XPG_START: is_local_host: rc = 403
30.11.2012 17:27:28 SXPG_STEP_XPG_START: host = ECCDB
30.11.2012 17:27:28 SXPG_STEP_XPG_START: is_local_r3_host: rc = 802
30.11.2012 17:27:28 SXPG_STEP_XPG_START: RFC_TCPIP_CONNECTION_OPEN: rc = 1003
30.11.2012 17:27:28 SXPG_STEP_COMMAND_START: SXPG_STEP_XPG_START returned: 1.003
30.11.2012 17:27:28 SXPG_COMMAND_EXECUTE(LONG)
30.11.2012 17:27:28 = 20121130172728
30.11.2012 17:27:28 COMMANDNAME = BRBACKUP
30.11.2012 17:27:28 ADDITIONAL_PARAMETERS =
30.11.2012 17:27:28 -u / -jid ALLOG20121130172655 -c force -t online -
30.11.2012 17:27:28 m all -p initPRD.sap -a -c force -p initIRP
30.11.2012 17:27:28 PRD.sap -cds
30.11.2012 17:27:28 LONG_PARAMS
30.11.2012 17:27:28 OPERATINGSYSTEM = ANYOS
30.11.2012 17:27:28 TARGETSYSTEM = PRODORADB
30.11.2012 17:27:28 DESTINATION
30.11.2012 17:27:28 SY-SUBRC = 1003
30.11.2012 17:27:28 SXPG_COMMAND_EXECUTE failed for BRBACKUP - Reason: program_start_error: For More Information, See SYS
30.11.2012 17:27:28 Job cancelled after system exception ERROR_MESSAGE

Solution:
--------
Refer to note 446172 : Note 446172 - SXPG_COMMAND_EXECUTE (program_start_error) in DB13
Note 34219 - DB13: CPIC error CM_RESOURCE_FAILURE_RETRY
Note 853610 - Configuring transaction DB13C for Oracle
http://help.sap.com/saphelp_nw70/helpdata/en/c4/3a7fbb505211d189550000e829fbbd/content.htm

The gw/remsh = rsh in rz11
SAP application server : ECCCI
Database server : ECCDB
From ECCCI I execute rsh ECCDB and it was giving error.
I activate rsh in ECCDB and execute rsh ECCDB date and it was successful.
RFC : SAPXPG_DBDEST_ECCDB was giving error
After activating rsh this rfc was successful.

Stop SAP does not work

When I executed the command stopsap r3 It did not stop the SAP application.
It did not kill the sapstart and dw work process.

Solution:
In the dir /usr/sap/ND1/DVBMGE17/work kill.sap was there but no entry was there.
Generally it stores the PID of sapstart
Then I did : ps -ef | grep sapstart
get the PID
and add this in kill.sap
echo "kill -2 <PID of sapstart>" > kill.sap
and it successfully stopped.
Actually during stopsap it searches for kill.sap and do the entry in it

Monday, 3 September 2012

Upgrade in Distributed system

During upgrade to EHP5 on ECC landscape the system did not come up in downtime phase after the kernel switch and repository import to main instance from shadow instance.

This problem occurs generally in distributed system i.e ASCS instance and Central instances are on different hosts.The kernel switch occurs in /usr/sap/<SID>/SYS/exe/run to 720 version and also in /usr/sap/<SID>/DV*/exe by the kernel file sapcpe.

But that did not happen in ASCS instance . So I have to mannualy copy all the execucutables from
/usr/sap/<SID>/SYS/exe/run  to /usr/sap/<SID>/ASCS00/exe.

And then execute the SAPup again where it was stopped and it worked fine.

Manual checks before starting Technical Downtime


Database and IS specific preparation
--------------------------------------------
  • Deactivaion of cluster failover during the technical downtime
  • Free memory space avalable for the database
  • rrequired memory space avalable in F.S
  • Automatic restart of SAP system is deactivated
  • Dattabase statistics is updated

SAP system side preparation.
------------------------------------
  • All functional that is bussiness-critiacal, background jobs are complete and planned.
  • Outstanding updates are processed.
  • Setting up a system messsage (SM02) with the inforamtion that system is unavaliable
  • No SAP object locked by repairs
  • consistence check (missing table and indexes ) successful (DB02)
  • Background jobs of class C configurred for CI
  • RDDIMPDP background job configurred for CI
  • Batch process configured for CI
  • Locking of SAP users
  • No active users logged into the system
  • No open or aborted updates in system(SM13)
  • BW delta queues processed (SMQ1)
  • General : interface gueues cleaned (LBWG)
  • Monitoring tools ( application server and CI) stoipped during downtime.
  • Isolation of CI (stopping the application servers)
  • Take a complete offline database backup( better if you take F.S backup of /oracle directoty)
  • Consisttent backup of installation directory,of the kernel directory and SAP profiles avaliable as they will be overwritten during downtime. 

Administration of shadow instance

In SAPehpi a shadow instance is created that runs in parrallel to live instance while you implement the modification adjustment (SPDD) and then start the activation.The last activity of shadow instance is the removal of activation error.

There are situation in which you need to intervene the installation routine like starting,stopping,locking and unlocking of shadow instance.

Starting the shadow instance
-----------------------------------
Go to installtion directory /EHPI/ABAP/BIN. Use the command ./SAPehpi startshd start the shadow instance.

Stopping the shadow instance
-----------------------------------
Go to installtion directory /EHPI/ABAP/BIN. Use the command ./SAPehpi stopshd stop the shadow instance.Note that shadow instance only runs between the phases STARTSHDI_FIRST and STOPSHDI_LAST.Logging on to the shadow instance is only for modification adjustment and removing activation error.

Unlocking the shadow instance
--------------------------------------
A manual unlocking of the shadow insttance is required for removing the activation error.
cd /EHPI/abap/bin
./SAPehpi unlockshd <SAPSID>

Locking the shadow instance
-----------------------------------
Manual locking of shadow instance is imperative after the manual unlocking before you continue the installation routine.
cd /EHPI/abap/bin
./SAPehpi lockshd <SAPSID>

Unlocking the system and the development environment
--------------------------------------------------------------------
In exceptional cases( for example support cases) you must unlock the instnce and undo a transport lock, if refquired . To do this call the following commands

tp unlocksys <SAPSID> pf=<path to transport profile >/tpprofile
tp unlock_eu <SAPSID> pf=<path to transport profile >/tpprofile

Use this command only if the support team explicitily request you do so.
The unauthorised import of transport after an official lockins of the development env by EHPI means that you can bring your system to an incostitent state, you may lose your support claim.

Locking the system and the development environment
-----------------------------------------------------------------
Always lock your system again before you continue the installation routine using the command

tp locksys <SAPSID> pf=<path to transport profile >/tpprofile
tp lock_eu <SAPSID> pf=<path to transport profile >/tpprofile

Tuesday, 1 May 2012

SAP system is not started during the downtime phase MAIN_NEW

Problem:
------------

We are upgrading to ECC6.0 EHP5 from ECC 6.0 EHP3
I have completed the pre-processing stage successfully. Then I started
the Downtime phase.But after some time in the pahase
MAIN_NEWBAS/STARTSAP_NBAS it gives the below error :
CURRENTPHASE MAIN_NEWBAS/STARTSAP_NBAS
...started at 20120212103622
..times self: 0.030000/0.030000 usr/sys, children 7.170000/3.630000
usr/sys.
..finished at 20120212103654 with status NO SYSTEM START.
# Error message set: 'Process 'startsap' exited with 1,
see '/sapcd/EHP5_Upg/EHPI/abap/log/SAPehpi.ECO' for details
System start failed'
...begin dialogue at 20120212103654

Analysis:
----------------

I checked the log in the work directory and error logs are as follows
sendbox:jepadm 121> pwd
/usr/sap/JEP/ASCS00/work
sendbox:jepadm 122> cat sapstartsrv.log
---------------------------------------------------
trc file: "sapstartsrv.log", trc level: 0, release: "700"
---------------------------------------------------
pid 368814
Sun Feb 12 11:11:13 2012
No halib defined => HA support disabled
Initializing SAPControl Webservice
SapSSLInit failed => https support disabled
Starting WebService thread
Webservice thread started, listening on port 50013
sendbox:jepadm 123>

sendbox:jepadm 81> startsap r3
Checking JEP Database
Database is running
-------------------------------------------
Starting Startup Agent sapstartsrv
OK
Instance Service on host sendbox started
-------------------------------------------
starting SAP Instance ASCS00
Startup-Log is written to /home/jepadm/startsap_ASCS00.log
-------------------------------------------
/usr/sap/JEP/ASCS00/exe/sapcontrol -prot NI_HTTP -nr 00 -function Start
startup of Instance failed
See /home/jepadm/startsap_ASCS00.log for details

Solution:
------------

The folloiwng error has occured:
----------------------
EXECUTING /usr/sap/JEP/DVEBMGS01/exe/sapcontrol
(/usr/sap/JEP/DVEBMGS01/exe/sapcontrol) -prot NI_HTTP -host sendbox -nr
01 -function StopWait 300 10
12.02.2012 09:01:14
Stop
FAIL: HTTP error, HTTP/1.1 401 Unauthorized
Process with ID 508234 terminated with status 1
BLOCKED SIGNALS: ''

Kindly check these things:
1) is there sapstartsrv process runnin with the profile O.S instance 01
on the server sendbox?
2) If the sapstartsrv is running it should create a stream file in the
/tmp called ".sapstream5<instance number>13", in your case:
/tmp/.sapstream50113
take a look if this file has the right permition, something like that:
srwx 1 <sap>adm sapsys .sapstream50113
What you can do to solve this problem is:
a) you can try restart the sapstartsrv for isntance 01;
(you need make sure that the stream file will be created)
b) you can manually start the instance 01.
Please be informed that sapstartsrv offers 2 authentication models. One
is http basic authentication which requires you to provide user and
password. Additionally it offers a so called trusted connect. On Unix
this is based on a Unix domain socket created by sapstartsrv at startup
(/tmp/.sapstream5??13). Unix startsap / stopsap script tries to use
this trusted connect. When running locally as <sid>adm it should be ableto connect via the Unix domain socket. However if it cannot connect thisway it falls back to http basic authencation. Since startsap/stopsap
never provide user/password to sapcontrol this fallback will not work.
If tursted connect doesn't work one of the following might have
happened:

1) sapstarstrv was not able to create /tmp/.sapstreamXXX, e.g. because
it already exist and is owned by a different user.
2) Someone / something deleted /tmp/.sapstreamXXX after sapstartsrv
had been started (e.g. some automatic cleanup in /tmp ?).
3) sapstartsrv or startsap/stopsap are not executed with the
same <sid>adm user.
Please check these and please also review following notes
1250195 - No UDS available when configuring service/hostname
(please try to set service/hostname as per the note)
927637 - Web service authentication in sapstartsrv as of Release 7.00
877795 - Problems w/ sapstartsrv as of Release 7.00 & 6.40 patch 169
(Please check the 6th point)

Then I compare with a working as follows and rename the below files
cd /tmp
mv .sapstream50113 .sapstream50113.orig
mv .sapstream50013 .sapstream50013.orig
========================================================================================
---------------------------------------------------
trc file: "sapstartsrv.log", trc level: 0, release: "720"
---------------------------------------------------
pid        409892
[Thr 01] Sun Feb 12 18:01:13 2012
No halib defined => HA support disabled
CCMS agent initialization: return code 0.
CCMS agent start: return code 0.
Initializing SAPControl Webservice
Starting AutoRestart thread
AutoRestart thread started
SapSSLInit failed => https support disabled
Starting WebService thread
Webservice thread started, listening on port 50013

srwx------    1 400      sapsys            0 Feb 01 09:45 .sapstream50013
srwx------    1 410      sapsys            0 Feb 01 09:45 /tmp/.sapstream50113
srwx------    1 400      sapsys            0 Feb 01 09:45 /tmp/.sapstream50013

==============================================================================
# cat sapstartsrv.log
---------------------------------------------------
trc file: "sapstartsrv.log", trc level: 0, release: "700"
---------------------------------------------------
pid        2752654
Mon Jan 23 06:28:32 2012
No halib defined => HA support disabled
Initializing SAPControl Webservice
SapSSLInit failed => https support disabled
Starting WebService thread
Webservice thread started, listening on port 50013
Trusted http connect via Unix domain socket '/tmp/.sapstream50013' enabled.

# ls -ltr /tmp/.sapstream50013
srwx------    1 jepadm   sapsys            0 Jan 23 06:28 /tmp/.sapstream50013
# ls -ltr /tmp/.sapstream*
srwx------    1 jepadm   sapsys            0 Jan 23 06:28 /tmp/.sapstream50013
srwx------    1 jepadm   sapsys            0 Jan 23 06:29 /tmp/.sapstream50113

===============================================================================================
and started by sapstart r3 and it got started