Trouble shooting Websphere Poral 6.1 default Database Crash ( Derby database)

Recently we had issue with Websphere Portal Startup when portal crashed because of free disk space issues.

When we try to access portal URL from browser

HTTP 404 when accessing the WebSphere Portal URL from a browser , Initialization of one or more services failed

In SystemOut.log we see that  Derby database connection failed for the first time and after restart of the server we see the Couple of exceptions like

Embed SQL Exception , 'The conglomerate (7,777) requested does not exist' message

and like

SQL Exception , 'The conglomerate (15,424) requested does not exist' message ( When it tries to access the PROT_RES table in JCR schema and WPS_TASK scheduler table in RELEASE database)

Trouble Shooting Steps we followed

1. Stop the Portal server if it's still running.
2. Delete the two lock files, db.lck and dbex.lck, in <wp_profile>/PortalServer/derby/wpsdb.
3. Delete the "tranlog" for WebSphere_Portal, under <wp_profile>/tranlog/<cell>/<node>/WebSphere_Portal.
4. Restart the Portal server.
5. If the Portal server startup still has problems, remove all the files in <wp_profile>/PortalServer/derby/wpsdb/log and then restart the Portal server.
6. If all of the above fails and you have a backup of the "wpsdb" database, you may attempt to archive the current database in another location outside of the WebSphere directory structure and put the backup in place.
NOTE: Because of the potential database corruptions documented Its recommend periodic backup of your Derby databases


But the above steps didn't Solve our problem, Raised PMR but that didn't help .

We tried to check the above tables (PROT_RES and WPS_TASKSCH) manually by connecting Derby DB but we got the same error like above when we fire Select queries . Realized that both tables corrupt(crashed).

We restored to wpsDB (we had wpsDB backup and deleted the old one replaced it).
NOTE: Recommended a periodic backup of your Derby databases

No comments:

Post a Comment