OpManager database connection lost and not being reestablished - MS SQL Server
Hi,
We have found a number of occurrences recently where OpManager looses database connectivity to the MSSQL server that holds its database.
The wrapper.log fills with errors saying that "Connection to the Database is lost." and after about 6 hours the log starts to fill with "OutOfMemory" errors.
e.g.
INFO | jvm 1 | 2015/09/06 18:01:31 | ##########################################################
INFO | jvm 1 | 2015/09/06 18:01:31 | Connection to the Database is lost. Trying to reconnect to the Database
INFO | jvm 1 | 2015/09/06 18:01:31 | Please Wait ...
INFO | jvm 1 | 2015/09/06 18:02:08 | Reconnected to the database
INFO | jvm 1 | 2015/09/06 18:02:08 | ##########################################################
...
INFO | jvm 1 | 2015/09/07 00:15:43 | ##########################################################
INFO | jvm 1 | 2015/09/07 00:15:45 | Connection to the Database is lost. Trying to reconnect to the Database
INFO | jvm 1 | 2015/09/07 00:15:45 | Please Wait ...
INFO | jvm 1 | 2015/09/07 00:15:45 | ##########################################################
INFO | jvm 1 | 2015/09/07 00:17:11 | java.lang.OutOfMemoryError: Java heap space
INFO | jvm 1 | 2015/09/07 00:17:11 | Dumping heap to java_pid3256.hprof ...
INFO | jvm 1 | 2015/09/07 00:17:29 | Heap dump file created [457918522 bytes in 18.795 secs]
INFO | jvm 1 | 2015/09/07 00:18:45 | OutOfMemoryError: Throws UnCaughtException
INFO | jvm 1 | 2015/09/07 00:18:46 | OutOfMemoryError: Throws UnCaughtException
etc.
I believe our scheduled backups of the MS SQL server may be causing the dropout as this is happening at the same time each Sunday at 6pm but my real concern is that OpManager isn't reestablishing the database connection after it's lost.
Restarting the services has apparently worked to fix this problem in the past but I was hoping for a configuration fix rather than scheduling a restart of one of our most important services every Sunday evening.
Any feedback would be greatly appreciated.
We have found a number of occurrences recently where OpManager looses database connectivity to the MSSQL server that holds its database.
The wrapper.log fills with errors saying that "Connection to the Database is lost." and after about 6 hours the log starts to fill with "OutOfMemory" errors.
e.g.
INFO | jvm 1 | 2015/09/06 18:01:31 | ##########################################################
INFO | jvm 1 | 2015/09/06 18:01:31 | Connection to the Database is lost. Trying to reconnect to the Database
INFO | jvm 1 | 2015/09/06 18:01:31 | Please Wait ...
INFO | jvm 1 | 2015/09/06 18:02:08 | Reconnected to the database
INFO | jvm 1 | 2015/09/06 18:02:08 | ##########################################################
...
INFO | jvm 1 | 2015/09/07 00:15:43 | ##########################################################
INFO | jvm 1 | 2015/09/07 00:15:45 | Connection to the Database is lost. Trying to reconnect to the Database
INFO | jvm 1 | 2015/09/07 00:15:45 | Please Wait ...
INFO | jvm 1 | 2015/09/07 00:15:45 | ##########################################################
INFO | jvm 1 | 2015/09/07 00:17:11 | java.lang.OutOfMemoryError: Java heap space
INFO | jvm 1 | 2015/09/07 00:17:11 | Dumping heap to java_pid3256.hprof ...
INFO | jvm 1 | 2015/09/07 00:17:29 | Heap dump file created [457918522 bytes in 18.795 secs]
INFO | jvm 1 | 2015/09/07 00:18:45 | OutOfMemoryError: Throws UnCaughtException
INFO | jvm 1 | 2015/09/07 00:18:46 | OutOfMemoryError: Throws UnCaughtException
etc.
I believe our scheduled backups of the MS SQL server may be causing the dropout as this is happening at the same time each Sunday at 6pm but my real concern is that OpManager isn't reestablishing the database connection after it's lost.
Restarting the services has apparently worked to fix this problem in the past but I was hoping for a configuration fix rather than scheduling a restart of one of our most important services every Sunday evening.
Any feedback would be greatly appreciated.