This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 227678 - entity classes from database hangs when "retrieving the keys" from data source
Summary: entity classes from database hangs when "retrieving the keys" from data source
Status: RESOLVED FIXED
Alias: None
Product: db
Classification: Unclassified
Component: DB schema (show other bugs)
Version: 8.0.2
Hardware: PC Windows 7 x64
: P3 normal (vote)
Assignee: matthias42
URL:
Keywords:
: 229425 246034 (view as bug list)
Depends on:
Blocks:
 
Reported: 2013-03-19 20:33 UTC by pmb2300
Modified: 2015-03-21 13:07 UTC (History)
7 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
This is where it stops (68.99 KB, image/png)
2013-03-19 20:33 UTC, pmb2300
Details
threaddump during hang while in entity classes from database wizard (14.17 KB, text/plain)
2013-03-20 15:16 UTC, pmb2300
Details
thread dump during hang (47.19 KB, application/msword)
2013-03-21 22:00 UTC, pmb2300
Details
Thread dump when the wizard is stuck (35.02 KB, text/plain)
2015-03-12 20:12 UTC, broot02_86
Details
Testbuild of fix (610.93 KB, application/x-java)
2015-03-12 21:18 UTC, matthias42
Details
Image of tested fix (24.28 KB, image/png)
2015-03-12 23:57 UTC, broot02_86
Details

Note You need to log in before you can comment on or make changes to this bug.
Description pmb2300 2013-03-19 20:33:26 UTC
Created attachment 132803 [details]
This is where it stops

Inside Netbeans 7.3:
Steps: New File > Persistence > Entity Classes from Database > Data Source: jdbc/appName

Retrieval of the Tables and Views begin, gets to 55% and then hangs.  It freezes on the same view OR the very next view. 

I've tried on multiple machines - same result.

Is there a limit on the size of the input data - in the wizard?  Or, maybe a problem with the database - itself?

thanks
Comment 1 David Konecny 2013-03-19 22:38:26 UTC
Could you please attach IDE threaddump? (see http://wiki.netbeans.org/GenerateThreadDump for more info) Thanks.
Comment 2 Sergey Petrov 2013-03-20 05:15:13 UTC
except thread dump, please try to generate db schema as it may be schema generation issue during retrieve keys.
Comment 3 Sergey Petrov 2013-03-20 09:50:02 UTC
please reopen with details requested above.
also, is it freeze? can you close the progress by cross or cancel?
how big is your schema? can you share sql script to create the schema?
Comment 4 pmb2300 2013-03-20 15:16:44 UTC
Created attachment 132850 [details]
threaddump during hang while in entity classes from database wizard

I performed the threaddump while running the IDE and the Entity classes from database wizard - oi ran it once the IDE hung.  It hung at the same spot as pictured in the previous attachment
Comment 5 Sergey Petrov 2013-03-20 15:26:20 UTC
reopen for evaluation.
Comment 6 pmb2300 2013-03-20 15:29:25 UTC
(In reply to comment #3)
> please reopen with details requested above.
> also, is it freeze? can you close the progress by cross or cancel?
> how big is your schema? can you share sql script to create the schema?

I cannot close by cross or cancel - only task manager
Comment 7 pmb2300 2013-03-20 15:32:11 UTC
...this thing keeps saying "mid air collision..."
Comment 8 pmb2300 2013-03-20 15:32:15 UTC
...this thing keeps saying "mid air collision..."
Comment 9 Sergey Petrov 2013-03-20 15:33:25 UTC
I will look more carefully tomorrow, but are you sure it's complete thread dump? I see no usual header and may be a bit smaller the usual.
Comment 10 pmb2300 2013-03-20 15:35:16 UTC
mid-air collision detedted - keeps coming up....
Comment 11 pmb2300 2013-03-20 16:02:21 UTC
reopened
Comment 12 pmb2300 2013-03-20 16:05:17 UTC
Bugzilla won't let me add comments OR it won't show me the updated page...what am I doing wrong?
Comment 13 Sergey Petrov 2013-03-20 18:13:43 UTC
it was already reopened by me, also you can always click first on issue link on top to get fresh state.
if you will have some time it's nice if you'll try to recreate thread dump, even 2 full dumps sometimes better then one. you one seems to be copied from comand line may as also contain some unusual newlines, can you forward output to some file?
Comment 14 Sergey Petrov 2013-03-21 04:51:47 UTC
Have you tried Persistence->generate DB Schema wizard also? With all tables selected for generation.
Comment 15 Sergey Petrov 2013-03-21 09:22:16 UTC
attached thread dump contain no .persistence.* classes, and no .db.* classes. please re-dump.
Comment 16 Sergey Petrov 2013-03-21 11:29:05 UTC
also I'm not sure it's P1 if it's reproducible for some specific schema only. but move to incomplete once again before gtting new thread dump and if schema generation works or not.
Comment 17 David Konecny 2013-03-21 19:40:36 UTC
The attach thread dump is not complete. Please try to get the IDE into the state when it looks like it hangs and then trigger tread dump several times. And wait between each thread dump 5-10 seconds. That allows us to look at what's happening at the code level. Thanks.

(In reply to comment #10)
> mid-air collision detedted - keeps coming up....

This should happen only when two people are submitting issue changes at the same time. Happens very rarely. Try reload the page fully before responding and do not use Back button.
Comment 18 pmb2300 2013-03-21 21:58:57 UTC
I think it is here:


at org.netbeans.modules.dbschema.jdbcimpl.ViewDependency.getMSSQLServerPKTable(ViewDependency.java:149)



but, how to fix, I dunno..... :-/
Comment 19 pmb2300 2013-03-21 22:00:14 UTC
Created attachment 132923 [details]
thread dump during hang
Comment 20 pmb2300 2013-03-21 22:00:55 UTC
I'll add another...I'm creating the schema now...but, it seems to be hanging too....
Comment 21 Sergey Petrov 2013-03-22 05:55:08 UTC
ok, need to move to db area as persistence just use it here
Comment 22 pmb2300 2013-03-22 14:25:13 UTC
"ok, need to move to db area as persistence just use it here" - what does this mean?

Also, while generating the schema - netbeans hung, again...
see attachment: threadForSchema

also, I can add an atachment! bugzilla say "You already used the form to file attachment 132850 [details]."
Comment 23 Sergey Petrov 2013-03-22 14:56:00 UTC
I suppose it's occasional resolve-incomplete, feel free to reopen.
Comment 24 Sergey Petrov 2013-03-22 16:46:34 UTC
additional comments (from email)"All I need to do is: add entity classes from a database - using the wizard.  This is for a web app for Union Bank.  The connection to our server is successful, creating a "test page" with SQL fields is successful (as described in the affable bean tutorial), the glassfish server pings the pool successfully.  Everything appears to be fine however, when I select the data source in the add entity classes from databases - wizard - it hangs on v_applicationStatusRootSumm (which is a Microsoft view that is built in SQL Server 2008 R2).  I've run the thread dump (see attached) and I've run the SQL "generate schema" function on the IDE.  Also, I've tried other Schemas and - they completed successfully - in the wizard.  On top of that, we created a new schema, on a different server, and placed the tables, views, etc - that are needed, on that db/schema and the wizard will run but, it will hang, it will finish but - it will not populate the list on the left side of the wizard - Available tables."

isn't populated tables may be another issue, not sure.
Comment 25 pmb2300 2013-04-24 17:12:07 UTC
I am still experiencing the same issue.  I have tried work-arounds but, to no avail.  I need to delve into this farther and am considering opening a ticket with Microsoft - to have them check our DB however, I need to exhaust the Application side first
Comment 26 Jaroslav Havlin 2013-04-25 06:15:47 UTC
I haven't been able to reproduce the bug yet. I'll investigate it and let you know soon.
Comment 27 Jaroslav Havlin 2013-04-26 13:21:12 UTC
I still cannot reproduce.

The table v_AdvertisementStatusRootSumm is not probably part of Microsoft SQL Server (I couldn't find it there), but seems to be part of Microsoft Systems Management Server. But I don't have access to this software.

To fix the bug, I would need to have an example of view for which the "entity classes from database" fails.

Could you please check structure of the view, try to guess what could be wrong with it, and create a simple view (and related tables) that causes the same problem?
Thank you.
Comment 28 Jaroslav Havlin 2013-04-26 14:10:42 UTC
The stacktrace shows that the thread is runnable and the JDBC driver is reading some data. Maybe that the amount of data is huge and it takes a lot of time to process them.

You could try to run the operation again, and create several stacktraces, with interval e.g. one minute, to check that the thread is still in the same method.


The last SQL statement that was called is "{call sp_depends(?)}". The parameter is probably a view that v_AdvertisementStatusRootSumm depends on (directly or transitively).

You can also run "call sp_depends('v_AdvertisementStatusRootSumm')" manually, check what dependencies of type "view" it returns, and then call "sp_depends('returned_view')" for all those dependencies, and continue recursively until some of the calls get frozen (then we would know which view causes the problem and also that the problem is not on NetBeans' side).
Comment 29 Jaroslav Havlin 2013-05-07 05:36:44 UTC
*** Bug 229425 has been marked as a duplicate of this bug. ***
Comment 30 patbai1 2013-05-08 20:00:07 UTC
When we try to add entity classes from the database. To our project, where does the Wizard pull the data from?  I ask because, when we run the wizard, we are not getting any query records on the SQL side (Ie inside the SQL server manager).
Comment 31 Jaroslav Havlin 2013-05-14 14:58:13 UTC
(In reply to comment #30)
> When we try to add entity classes from the database. To our project, where does
> the Wizard pull the data from?  I ask because, when we run the wizard, we are
> not getting any query records on the SQL side (Ie inside the SQL server
> manager).
The wizard pulls data from the database, using the JDBC driver. The last query should be "call sp_depends(?)".

You can see the relevant source code here: Method getMSSQLServerPKTable in
http://hg.netbeans.org/core-main/file/530c37a18b46/dbschema/src/org/netbeans/modules/dbschema/jdbcimpl/ViewDependency.java
Comment 32 Jaroslav Havlin 2013-05-31 07:46:56 UTC
>> The stacktrace shows that the thread is runnable and the JDBC driver is
>> reading some data. Maybe the amount of data is huge and it takes a lot of 
>> time to process them.
Hard to reproduce, and not sure that the problem can be fixed in NetBeans. Decreasing priority. Further investigation is needed. A reproducible test case would be very helpful.
Comment 33 nidhalbt 2014-06-20 15:33:48 UTC
I also had the same problem mentioned.
In my case I started the derby database and it worked.
Comment 34 Edworld 2014-11-18 18:49:43 UTC
According to the code, there is a potential infinite loop: the method getMSSQLServerPKTable() calls itself whenever a view have a dependency relationship with another view.

If a view V1 depends on a column of another view V2, then MSSQLServer's stored procedure "sp_depends" reports the following:
EXEC sp_depends v1 --> v2, view
EXEC sp_depends v2 --> v1, view

After the diagnostic, the bug corretion will be trivial: it is enough to avoid calling getMSSQLServerPKTable() twice for the same view.
Comment 35 matthias42 2014-11-18 19:16:43 UTC
(In reply to Edworld from comment #34)
> According to the code, there is a potential infinite loop: the method
> getMSSQLServerPKTable() calls itself whenever a view have a dependency
> relationship with another view.
> 
> If a view V1 depends on a column of another view V2, then MSSQLServer's
> stored procedure "sp_depends" reports the following:
> EXEC sp_depends v1 --> v2, view
> EXEC sp_depends v2 --> v1, view
> 
> After the diagnostic, the bug corretion will be trivial: it is enough to
> avoid calling getMSSQLServerPKTable() twice for the same view.

You are right - in current netbeans recursion is already limited (whether the limiit is to large is discussable). 

Please see bug 239181 for the corresponding bug.
Comment 36 Sergey Petrov 2014-11-19 11:49:18 UTC
I suppose if the reason is clear and there is supposed way to fix it, it may have sense to rise issue priority to at least p3, as p4 are rarely in view, and was set when there was much less information
Comment 37 andrzejo 2015-01-03 17:56:49 UTC
Just vanted to to vote for this one, too...
Comment 38 broot02_86 2015-03-12 18:05:26 UTC
same issue here
Comment 39 matthias42 2015-03-12 19:09:26 UTC
To the new commenters: It is good to know it is reproducible, but it is bad, that no further information is supplied.

Please at least provide a thread dump (see: http://wiki.netbeans.org/GenerateThreadDump), as this should clearly show if the stack was overrun. Please also attach the message.log (http://wiki.netbeans.org/FaqLogMessagesFile).

The fix i talked about in comment 35 was merged over a year ago is part of netbeans 8.
Comment 40 broot02_86 2015-03-12 20:12:23 UTC
Created attachment 152584 [details]
Thread dump when the wizard is stuck
Comment 41 broot02_86 2015-03-12 20:13:44 UTC
I attached thread dump, I am running Netbeans 8.0.2 as well.
Comment 42 matthias42 2015-03-12 21:18:10 UTC
Created attachment 152586 [details]
Testbuild of fix

(In reply to broot02_86 from comment #41)
> I attached thread dump, I am running Netbeans 8.0.2 as well.

Thank you! That really helped. It looks like the recursion level is indeed too high - relevant is this entry in the stacktrace:

org.netbeans.modules.dbschema.jdbcimpl.ViewDependency.getMSSQLServerPKTable(ViewDependency.java:169)

or to be more precise the nearly 50 repetitions. Then the DB server does not answer anymore.

I reviewed the fix from bug 239181 again (at that time it looked like the easier solution). Obvisually I was not totally correct.

I now made a different approach (was thought about in the past and was suggested by edword). Basicly: Track which view was already analysed for this round and if so, log the cyclic dependency and go on.

I attached a rebuild of the dbschema module. I would be great if you could test it. Beware: Make sure you have a backup copy of the files you modify!

1. grab the attached testbuild file
2. Backup <netbeans-directory>/java/modules/org-netbeans-modules-dbschema.jar
3. Overwrite the file from point 2 with the new version downloaded in point 1
4. Delete the cache directory of netbeans (http://wiki.netbeans.org/FaqWhatIsUserdir)
5. Do your test again

The expected result: The process works and you will find an entry like this in the IDE Log (View -> IDE Log):

WARNING [org.netbeans.modules.dbschema.jdbcimpl.ViewDependency]: Cyclic dependency detected in view definition: XY

The output is buffered, so it can take a few seconds before that message is visible. I verified, that the new approach still fixes the problem in bug 239181.
Comment 43 broot02_86 2015-03-12 23:57:44 UTC
Created attachment 152587 [details]
Image of tested fix

WARNING [org.netbeans.modules.dbschema.jdbcimpl.ViewDependency]: Cyclic dependency detected in view definition: viewNWIReceivedCheckDate

Also in IDE Log.
Comment 44 broot02_86 2015-03-12 23:58:55 UTC
Tested the new jar, and the fix seems to have solved this issue, works like a charm now, let me know if you need more details.
Comment 45 matthias42 2015-03-13 20:31:47 UTC
Thank you very much! This really helped. I pushed the fix into core-main:

http://hg.netbeans.org/core-main/rev/47aad7fecf75

which ensures, that it will be in the next netbeans release. In the next few days a netbeans nightly build is created - a message is then appended to this bug.

Please test that build if the bug is really fixed. This is necessary because I did some more changes (warnings, clean-ups) and I want to be save that this does not break it again.

I verified already, that bug 239181 is still fixed - if you also find the bug fixed in the nightly please change the status of this bug from FIXED to VERIFIED. Thanks again!
Comment 46 Quality Engineering 2015-03-16 02:11:24 UTC
Integrated into 'main-silver', will be available in build *201503160001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/47aad7fecf75
User: Matthias Blaesing <matthias42@netbeans.org>
Log: #227678: Change cycle detection in view dependency analyser, don't limit recursion level, but record visited children
Comment 47 matthias42 2015-03-21 13:07:18 UTC
*** Bug 246034 has been marked as a duplicate of this bug. ***