This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 196112 - Memory / performance issues w/ db
Summary: Memory / performance issues w/ db
Status: RESOLVED INCOMPLETE
Alias: None
Product: db
Classification: Unclassified
Component: Code (show other bugs)
Version: 7.0
Hardware: PC Linux
: P3 normal (vote)
Assignee: Libor Fischmeistr
URL:
Keywords:
: 195392 (view as bug list)
Depends on:
Blocks:
 
Reported: 2011-02-28 16:11 UTC by _ tboudreau
Modified: 2015-03-22 09:29 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description _ tboudreau 2011-02-28 16:11:46 UTC
I routinely have connections to both embedded memory H2 and MySQL open in NetBeans.

I can be running NB for several days, but once I start doing database work in the IDE, within a few hours, I have to restart because of heap size.

My assumption would be that not all JDBC resources are being properly closed.

I did a quick scan of the database modules, and it looks like this could easily be true - there are some methods which are passed a ResultSet which is never explicitly closed, and things like that (maybe it's closed later somewhere else, but it's hard to tell).

I've profiled and fixed similar issues in my own app code, and one main root cause is:
 - MySQL's ConnectorJ driver does very naive static caching of unclosed PreparedStatements - they will live forever
 - H2 uses more statics than I'd like

So you end up with a lot of byte[]s on the heap which belong to dead JDBC objects.

How I would fix this globally:
 - Write wrappers for all JDBC resources (it's easy, just write implementations of Statement, PreparedStatement, CallableStatement, ResultSet and DataBaseMetadata which delegate to an inner instance)
 - Each wrapper should keep a weak reference to the object which requested it, and use a background thread and a ReferenceQueue for those references, and set a flag when close is called;  in "debug mode" it should keep a stack trace from its creation
 - When the "creator object" is garbage collected,
   - In "debug mode", log the creation stack trace if it was not closed
   - Close the underlying object and discard it
 - When close() is called
   - Close the underlying object and discard it

This means that the "creator object" must live as long as the JDBC object is needed (and no longer).

I have some code for a wrapper library which does almost this which I'd happily donate somewhere.
Comment 1 Jiri Rechtacek 2011-03-04 10:40:17 UTC
*** Bug 195392 has been marked as a duplicate of this bug. ***
Comment 2 matthias42 2015-03-21 22:33:42 UTC
Closing this as INCOMPLETE. Of course it is possible to wrap every JDBC object into another wrapper, but I don't see it as a solution, but only as another possible point of failure.

There are memory problems in db code of netbeans, but many (I'd bet most of them) are not caused by not properly closing/handling jdbc resources. You see a slow down in netbeans - I'd like to see a heapdump + threaddump of the problem.

Please feel free to reopen this bug.
Comment 3 _ tboudreau 2015-03-22 02:47:43 UTC
If you read the detailed description I gave, two examples where this is a problem are clearly identified.  And not closing statements and result sets is plainly a bug - it is not using these objects in the way that is clearly outlined in the JDBC spec - a heap dump wouldn't tell you anything that isn't obvious by glancing at the code.

Reopening. "I don't feel like reading the code" does not make a bug incomplete.

Simple diagnostic: Use a connection pool such as BoneCP that features unclosed resource tracking, and it will tell you when an unclosed JDBC resource gets garbage collection.
Comment 4 matthias42 2015-03-22 09:29:45 UTC
You asume(!) jdbc resources are not properly closed. You claim unclosed JDBC objects hold resources that don't show up on heapdump but flood your heap. I seriously doubt that.

I had another quick glance at the code - the statements are closed correctly (either try-with-resource of closed in finally block), ResultSets are closed - in an error condition they might end up unclosed, but then you'd have seen error messages.

I understand your idea but as said it introduces another complexity layer without a proven problem.

Just for the record - please keep you accusations for yourself - I'm far from reluctant to dig through the code, but without a lead it's just pointless.