Bug 218837 - services/servers tab hangs
services/servers tab hangs
Status: VERIFIED FIXED
Product: serverplugins
Classification: Unclassified
Component: GlassFish
7.2
PC Windows 7 x64
: P1 (vote)
: 7.3
Assigned To: TomasKraus
issues@serverplugins
72patch-candidate
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-09-22 21:11 UTC by emiddio
Modified: 2012-10-24 14:04 UTC (History)
4 users (show)

See Also:
Issue Type: DEFECT
:


Attachments
thread dump textfile (41.22 KB, text/plain)
2012-09-23 22:36 UTC, emiddio
Details
7.3 nightly thread dump with deadlock (102.66 KB, text/plain)
2012-09-26 14:56 UTC, emiddio
Details

Note You need to log in before you can comment on or make changes to this bug.
Description emiddio 2012-09-22 21:11:52 UTC

    
Comment 1 emiddio 2012-09-22 21:21:21 UTC
when select services/servers tab it just hangs with please wait and the circle cursor forever.

Product Version: NetBeans IDE 7.2 (Build 201207171143)
Java: 1.7.0_04; Java HotSpot(TM) 64-Bit Server VM 23.0-b21
System: Windows 7 version 6.1 running on amd64; Cp1252; en_US (nb)
User directory: C:\Users\emiddio\AppData\Roaming\NetBeans\7.2
Cache directory: C:\Users\emiddio\AppData\Local\NetBeans\Cache\7.2

8GB ram quad core amd phenomII

no issues with 7.2rc or earlier releases.

i import setting from other installations of netbeans.

i do have Sun app server 8.1 8.2, gfv2ur2, gfv301, ... installed along with
netbeans 5, 6.9.1, 7.0.1, 7.1.2 on this win 7 box.

when nb7.2 hangs it appears to exit but the netbeans*64 process continues to run, have to task manager kill it.

if i new where and how to -- i could manually remove the gf-server instances known to netbeans 7.2 prior to gfv3 to see if that has something to do with the problem.

tried installing the gfv1,2 support plugin but doesnt help; does no help if i start netbeans with jdk1.6.0_26 - same issue also.

also have various tomcats installed.

i have lots of configured info i want to import into netbeans - i do not want to have to start with no import settings.

suggestions welcome.
Comment 2 Petr Jiricka 2012-09-23 20:25:40 UTC
Hi, do you have the thread dump while NetBeans is stuck? I am afraid without the thread dump we can't investigate. Thanks.
Comment 3 emiddio 2012-09-23 20:41:03 UTC
tell me how to produce a thread dump and i will get one for you.

what about my question -- how can i manually removed servers known to netbeans 
by editing or deleting configuration information in the filesystem manually ?
Comment 4 Petr Jiricka 2012-09-23 20:51:15 UTC
Regarding the thread dump: http://wiki.netbeans.org/GenerateThreadDump

Regarding your question - I must say I am not sure.
Comment 5 emiddio 2012-09-23 22:36:42 UTC
Created attachment 124776 [details]
thread dump textfile
Comment 6 TomasKraus 2012-09-24 09:48:47 UTC
I'm not using NetBeans on Windows but it should be similar to Linux.
I have following directory structure in mu home directory:
.netbeans/7.1.2/config/GlassFishEE6/Instances

In your case 7.1.2 should be 7.2 somewhere where windows stores application config files for users.

You'll see following files in that folder:
-rw-r--r-- 1 kratz kratz    0 2012-09-13 11:40 instance
-rw-r--r-- 1 kratz kratz    0 2012-09-13 12:22 instance_1
-rw-r--r-- 1 kratz kratz 2547 2012-09-13 12:22 .nbattrs

Instance and Instance_1 is just some kind of 'notification' that .nbattrs contains XML element with the same name.

In .nbattrs you'll see:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE attributes PUBLIC "-//NetBeans//DTD DefaultAttributes 1.0//EN" "http://www.netbeans.org/dtds/attributes-1_0.dtd">
<attributes version="1.0">
    <fileobject name="instance">
        <attr name="adminPort" stringvalue="4848"/>
        <attr name="derbyStartOn" stringvalue="true"/>
        <attr name="displayName" stringvalue="GlassFish Server 3.1.2.2"/>
        ...
    </fileobject>
    <fileobject name="instance_1">
        <attr name="adminPort" stringvalue="4848"/>
        <attr name="derbyStartOn" stringvalue="true"/>
        <attr name="displayName" stringvalue="GlassFish Server 3.1.2"/>
        ...
    </fileobject>
</attributes>

Make sure netbeans are not running. Find fileobject you want to delete and remove it from .nbattrs. Also remove 0 size file with the same name as in removed name attribute in removed fileobject element. E.g. you are removing <fileobject name="instance_1"> so also file instance_1 shall be removed.
Comment 7 TomasKraus 2012-09-24 12:52:03 UTC
I see a deadlock there related to GlassfishInstanceProvider.getInstance() method.

Please can you check this issue with last 7.3 nightly build from http://bits.netbeans.org/dev/nightly/latest/ ?

I see that in one of the affected callchain:
        at org.netbeans.modules.j2ee.deployment.impl.ServerRegistry.getServerInstance(ServerRegistry.java:317)
        - waiting to lock <0x00000000fc4247e8> (a org.netbeans.modules.j2ee.deployment.impl.ServerRegistry)
        at org.netbeans.modules.j2ee.deployment.plugins.api.InstanceProperties.getInstanceProperties(InstanceProperties.java:155)
        at org.netbeans.modules.glassfish.javaee.JavaEEServerModuleFactory.createModule(JavaEEServerModuleFactory.java:117)
        at org.netbeans.modules.glassfish.common.GlassfishInstance.updateFactories(GlassfishInstance.java:1124)
        - locked <0x00000000fc435498> (a org.openide.util.lookup.ProxyLookup$R)
        at org.netbeans.modules.glassfish.common.GlassfishInstance.updateModuleSupport(GlassfishInstance.java:312)
        at org.netbeans.modules.glassfish.common.GlassfishInstanceProvider.loadServerInstances(GlassfishInstanceProvider.java:490)
        at org.netbeans.modules.glassfish.common.GlassfishInstanceProvider.init(GlassfishInstanceProvider.java:426)
        - locked <0x00000000fc4327a0> (a java.util.Collections$SynchronizedMap)
        at org.netbeans.modules.glassfish.common.GlassfishInstanceProvider.getEe6(GlassfishInstanceProvider.java:133)

I already moved init() method outside lock in getEe6() method and this deadlock condition should be resolved in 7.3.
Comment 8 emiddio 2012-09-25 18:22:10 UTC
i checked with last night nightly build; still hangs. did not have time to get another thread dump.

is the previous thread dump good enough ?
Comment 9 TomasKraus 2012-09-25 22:37:33 UTC
Thread dump from 7.3 nightly would be very helpful.
Comment 10 emiddio 2012-09-26 14:56:23 UTC
Created attachment 124951 [details]
7.3 nightly thread dump with deadlock
Comment 11 emiddio 2012-09-30 20:35:50 UTC
changing priority; no feedback on this issue in around a week. This is a deadlock issue.
Comment 12 TomasKraus 2012-10-01 09:43:26 UTC
The problem seems to be in calling loadServerInstances() code in synchronized (instanceMap) block.
The same locking is coming from other side trough ServerRegistry and lassfishInstanceProvider.getInstance().

I can't do much with get call() on synchronized map so the only possible fix goes trough loadServerInstances() call from init().

I moved 
        for (GlassfishInstance gi : instanceMap.values()) {
            GlassfishInstance.updateModuleSupport(gi);
        }
from loadServerInstances() to init() to get it outside synchronized (instanceMap) block.
Comment 13 Petr Hejl 2012-10-01 09:52:15 UTC
BTW last time I saw the code there was some hacky check for server registry mismatch. I would guess that is hack/workaround for some faulty behavior which should be fixed (prehaps in memory IP already did that, don't know). See Hk2DeploymentFactory:165. Without this check there would be no deadlock afaik.
Comment 14 TomasKraus 2012-10-01 12:42:36 UTC
Yes, that's true. On the other hand instances map locks like the one in init() shall not cover more than local data structure handling.
In GlassFish plugin there are so many complicated things done inside synchronized blocks.
Comment 15 TomasKraus 2012-10-01 12:57:46 UTC
changeset:   234604:16a05ed5da31
summary:     #218837 - Deadlock removal
Comment 16 Jiri Skrivanek 2012-10-01 14:38:29 UTC
There is an exception in tests and similar when you expand the Servers node.

http://bertram-tst.netbeans.org:8080/job/web-main/8714/testReport/org.netbeans.core.validation/ValidateLayerConsistencyTest/testInstantiateAllInstances/
Comment 17 TomasKraus 2012-10-01 14:49:40 UTC
Looks like I have bug in automatic GlassFish Tooling SDK library merge into web-main.

changeset:   234755:31d3033fe85f
summary:     #218837 - Fixed GlassFish Tooling SDK library path

This is hot-fix for now. I have to fix related Hudson job.
Comment 18 Quality Engineering 2012-10-02 02:25:17 UTC
Integrated into 'main-golden', will be available in build *201210020001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/16a05ed5da31
User: Tomas Kraus <TomasKraus@netbeans.org>
Log: #218837 - Deadlock removal
Comment 19 emiddio 2012-10-02 15:28:37 UTC
issue resolved in todays download for me.
thanks
Comment 20 Jiri Skrivanek 2012-10-24 14:04:43 UTC
Verified. Reporter claims it works in recent daily builds. I was not able reproduce it.


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo