Scenario
While uploading the RPD the server was not restarting and clicking on the coreapplication on the em was throwing up error regarding ADF related issues. To resolve the issue we had restarted the whole box and from then the weblogic and opmnctl were not coming up.
Reason
The disk space was on the server was full which was the reason for the servers to come up. Due to restarting the server because of full disk space corrupted all the files. In order to bring the services up we would have to delete/replace all the corrupted files in order for all the services to come up and running.
Solution
Solution
Part – I – Resolving the weblogic issue
In order for the weblogic to come up these are the following steps that were taken.
1. CWallet.sso was deleted
The CWallet.sso file which has all the encrypted data is to be deleted or taken a backup on the following location mentioned in the screenshot
2. Replicas.prop file has to be deleted.
The file replica.prop has to be deleted which is in the following location
$DOMAIN/servers/AdminServer/data/ldap/conf/replicas.prop
This file was null when we had this issue. Once this file was deleted and the weblogic.sh was run this file had an entry. Make sure that there is this entry in the file
replica.num=0
3. Changing the username and password in the boot properties.
Sometimes we would have to reset the username and password of weblogic in the boot properties file. When you traverse into the following location
$DOMIAN\bifoundation_domain\servers\AdminServer\security
In the boot properties file edit the username and password and once the services are running it will get encrypted back.
Once these 3 steps were performed the weblogic was up and running and the status changed from FORCE SHUT DOWN.
Part – II – Resolving the opmnctl services issues
After the weblogic was up and running as usual tried to start the opmnctl services and none of the services were running and was showing all sorts of errors while running. For this we created a system.user credential key in em
1. Following a Oracle Forum we created a system users in em under the following location
Weblogic -->Expand-->bi_foundation_domain-->Right Click-->Security-->Credential
Create a map oracle.bi.system and a key system.user
Create a credential key scheduler.schema with the information provided below
Create a map oracle.biapps.system and a key (Not required but do it if the services do not come up)
2. While adding the first credential key the BI Server and the BI Presentation Services were up and running and the while doing the second step the Cluster and Schdeuler were up and running.
Different errors which were faced
1. In the saw log in Presentation Services
State: HY000. Code: 10058. [NQODBC] [SQL_STATE: HY000] [nQSError: 10058] A general error has occurred.
[nQSError: 12002] Socket communication error at call=Connect: (Number=107) Transport endpoint is not connected
[nQSError: 12010] Communication error connecting to remote end point: address = obalcdcqvm001; port = 9706.
[nQSError: 12008] Unable to connect to port 9706 on machine obalcdcqvm001. (HY000)[[
2. In the Cluster logs
SUGEnterpriseCredentialRepository not initialized
[2015-08-26T06:18:37.000-04:00] [OracleBIClusterControllerComponent] [ERROR:1] [] [] [ecid: ] [tid: c65c4720] [71002] The Cluster Controller is terminating due to errors in the cluster configuration.
SUGEnterpriseCredentialRepository not initialized
Please note that when this issue was raised with Oracle the above mentioned error shows up when there was an error if there was a disk space issue .
Oracle SR Information - 3-11255809271