This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 198319 - Redeploy to GlassFish 3.1 often hangs
Summary: Redeploy to GlassFish 3.1 often hangs
Status: RESOLVED WONTFIX
Alias: None
Product: serverplugins
Classification: Unclassified
Component: GlassFish (show other bugs)
Version: 7.0
Hardware: Macintosh (x86) Mac OS X
: P3 normal (vote)
Assignee: Vince Kraemer
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-05-04 15:59 UTC by rweaver
Modified: 2011-05-15 14:17 UTC (History)
2 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Screen shot of deploy after minutes of activity (107.04 KB, image/png)
2011-05-04 16:02 UTC, rweaver
Details
Task information from glassfish deploy / not restarting (87.90 KB, text/rtf)
2011-05-04 16:09 UTC, rweaver
Details
another screen shot (79.69 KB, image/png)
2011-05-04 16:11 UTC, rweaver
Details
Activity monitor (247.99 KB, image/png)
2011-05-04 16:12 UTC, rweaver
Details
Activity monitor (194.16 KB, image/png)
2011-05-04 16:13 UTC, rweaver
Details
scanning (31.12 KB, image/png)
2011-05-04 16:13 UTC, rweaver
Details
GF start failure (55.89 KB, image/png)
2011-05-04 16:14 UTC, rweaver
Details
NetBeans Thread dump (30.34 KB, application/octet-stream)
2011-05-09 05:39 UTC, rweaver
Details
GF thread dump (61.51 KB, application/octet-stream)
2011-05-09 05:41 UTC, rweaver
Details
Server log from development server on CentOS (38.57 KB, application/octet-stream)
2011-05-13 17:48 UTC, rweaver
Details

Note You need to log in before you can comment on or make changes to this bug.
Description rweaver 2011-05-04 15:59:59 UTC
Product Version = NetBeans IDE 7.0 (Build 201104080000)
Operating System = Mac OS X version 10.6.7 running on x86_64
Java; VM; Vendor = 1.6.0_24
Runtime = Java HotSpot(TM) 64-Bit Server VM 19.1-b02-334

When working with current JavaEE 6 project, see the application hang for long periods of time. Memory usage is extremely high, and I often have to kill the Glassfish java executable by "force quit" and restart ide just to test small changes.
Comment 1 rweaver 2011-05-04 16:02:03 UTC
Added screen shot of IDE hang on deploy, note that it sometimes it only gets as far as displaying the mojarra message.
Comment 2 rweaver 2011-05-04 16:02:09 UTC
Created attachment 108109 [details]
Screen shot of deploy after minutes of activity
Comment 3 rweaver 2011-05-04 16:09:28 UTC
Created attachment 108110 [details]
Task information from glassfish deploy / not restarting

Some additional information from looking at running tasks and listening ports.
Comment 4 rweaver 2011-05-04 16:11:17 UTC
Created attachment 108111 [details]
another screen shot
Comment 5 rweaver 2011-05-04 16:12:06 UTC
Created attachment 108112 [details]
Activity monitor
Comment 6 rweaver 2011-05-04 16:13:04 UTC
Created attachment 108113 [details]
Activity monitor
Comment 7 rweaver 2011-05-04 16:13:48 UTC
Created attachment 108114 [details]
scanning
Comment 8 rweaver 2011-05-04 16:14:33 UTC
Created attachment 108115 [details]
GF start failure

This will often happen when I try to exit NetBeans
Comment 9 Vince Kraemer 2011-05-04 17:10:37 UTC
It sounds like this issue is pretty 'context dependent' but we don't have enough info about your situation to replicate the issue that you are seeing.

A description of the 'initial state' and step by step instruction to replicate the issue would be useful.

a thread dump of the NetBeans process and the GlassFish server process would also be useful.

Some info about the scale of the OmnisTools application and the prime-showcase-1.0.0-SNAPSHOT may be helpful also.
Comment 10 rweaver 2011-05-04 17:34:27 UTC
It's pretty simple, I deploy the app, and starts up OK (with the exception of the "reference problems" dialog box, which displays most of the time (there's another bug open on that one, has to do with the GF restlib getting included dynamically).

At any rate the app is extremely simple: a MySQL database, created Hibernate entity beans, generated REST from entity beans, and generated JSF from entity beans.

Very basic CRUD.

This has been happening for some time on Mac, haven't spent enough time on any other platform to know if it is a problem.

I'd be glad to give you a thread dump if you give me instructions. That was the intent of dumping the task information and screen shots of Activity monitor.
Comment 11 rweaver 2011-05-04 17:39:43 UTC
May or may not be related to 198158 - http://netbeans.org/bugzilla/show_bug.cgi?id=198158
Comment 12 Jiri Skrivanek 2011-05-05 07:20:43 UTC
Instructions how to generate thread dump are here http://wiki.netbeans.org/GenerateThreadDump#VisualVM.
Comment 13 rweaver 2011-05-08 16:17:17 UTC
Redeploy failed again with out of memory condition (heap) on GF.

Exit Netbeans and "ghost" GlassFish is still running, had to kill using Activity Monitor (was using lots of CPU).
Comment 14 rweaver 2011-05-09 05:39:52 UTC
Created attachment 108181 [details]
NetBeans Thread dump

SEVERE: Exception in thread "RMI RenewClean-[10.0.1.2:8686]" 
SEVERE: Exception in thread "connector-timer-proxy" 
SEVERE: java.lang.OutOfMemoryError: Java heap space
	at java.io.BufferedOutputStream.<init>(BufferedOutputStream.java:59)
	at java.io.BufferedOutputStream.<init>(BufferedOutputStream.java:42)
	at sun.rmi.transport.tcp.TCPConnection.getOutputStream(TCPConnection.java:66)
	at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:202)
	at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
	at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:322)
	at sun.rmi.transport.DGCImpl_Stub.dirty(Unknown Source)
	at sun.rmi.transport.DGCClient$EndpointEntry.makeDirtyCall(DGCClient.java:342)
	at sun.rmi.transport.DGCClient$EndpointEntry.access$1600(DGCClient.java:153)
	at sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:555)
	at java.lang.Thread.run(Thread.java:680)
Comment 15 rweaver 2011-05-09 05:41:18 UTC
Created attachment 108182 [details]
GF thread dump

Thread dump from GF at same heap space error time.
Comment 16 David Konecny 2011-05-09 21:29:33 UTC
Sounds like a GF issue. Passing to Vince to evaluate.
Comment 17 David Konecny 2011-05-09 21:39:02 UTC
This comment
  http://netbeans.org/bugzilla/show_bug.cgi?id=198158#c9
*might* be related to this issue, for example:

[#|2011-05-08T08:53:20.321-0700|WARNING|glassfish3.1|org.eclipse.persistence.session.file:/Users/newadminaccount/NetBeansProjects/OmnisTools/build/web/WEB-INF/classes/_OmnisToolsPU.weaver|_ThreadID=17;_ThreadName=Thread-2;|Weaver encountered an exception while trying to weave class com/omnistools/Company.  The exception was: java.lang.OutOfMemoryError: Java heap space|#]
Comment 18 rweaver 2011-05-13 17:48:10 UTC
Created attachment 108278 [details]
Server log from development server on CentOS

Same issue happens on Linux server running CentOS when doing "redeploy" through admin port:

[#|2011-05-13T17:37:35.824+0000|INFO|glassfish3.1|org.eclipse.persistence.session.file:/usr/local/glassfish3/glassfish/domains/domain1/applications/OmnisTools/WEB-INF/classes/_OmnisToolsPU|_ThreadID=30;_ThreadName=Thread-1;|file:/usr/local/glassfish3/glassfish/domains/domain1/applications/OmnisTools/WEB-INF/classes/_OmnisToolsPU logout successful|#]

[#|2011-05-13T17:40:05.007+0000|SEVERE|glassfish3.1|javax.enterprise.system.std.com.sun.enterprise.server.logging|_ThreadID=63;_ThreadName=Thread-1;|Exception in thread "AutoDeployer" |#]

[#|2011-05-13T17:40:09.650+0000|SEVERE|glassfish3.1|javax.enterprise.system.tools.admin.org.glassfish.deployment.admin|_ThreadID=84;_ThreadName=Thread-1;|Exception while visiting com/sun/xml/ws/message/AttachmentUnmarshallerImpl.class of size 1719
java.lang.OutOfMemoryError: Java heap space
|#]
Comment 19 rweaver 2011-05-13 17:58:17 UTC
[root@Omnis-Slice-2 ~]# service glassfish stop
CLI306 Warning - server is not running.
Command stop-domain executed successfully.
[root@Omnis-Slice-2 ~]# service glassfish start
There is a process already using the admin port 4848 -- it probably is another instance of a GlassFish server.
Command start-domain failed.
[root@Omnis-Slice-2 ~]# service glassfish stop
It appears that server [localhost:4848] does not accept secure connections. Retry with --secure=false.
CLI306 Warning - server is not running.
Command stop-domain executed successfully.
[root@Omnis-Slice-2 ~]# ps -ef | grep java
root      9495     1  0 May09 ?        00:00:10 /usr/java/jdk1.6.0_24/bin/java -Djava.library.path=/usr/local/glassfish3/glassfish/lib -cp /usr/local/glassfish3/glassfish/modules/admin-cli.jar:/usr/local/glassfish3/glassfish/modules/glassfish-extra-jre-packages.jar:/usr/local/glassfish3/glassfish/modules/glassfish-registration.jar:/usr/local/glassfish3/glassfish/modules/glassfish-ee-api.jar:/usr/local/glassfish3/glassfish/modules/glassfish-corba-orbgeneric.jar:/usr/local/glassfish3/glassfish/modules/glassfish.jar:/usr/local/glassfish3/glassfish/modules/glassfish-oracle-jdbc-driver-packages.jar:/usr/local/glassfish3/glassfish/modules/glassfish-corba-omgapi.jar:/usr/local/glassfish3/glassfish/modules/common-util.jar:/usr/local/glassfish3/glassfish/modules/glassfish-api.jar:/usr/local/glassfish3/glassfish/modules/glassfish-mbeanserver.jar:/usr/local/glassfish3/glassfish/modules/glassfish-corba-internal-api.jar:/usr/local/glassfish3/glassfish/modules/glassfish-naming.jar:/usr/local/glassfish3/javadb/lib/derby.jar:/usr/local/glassfish3/javadb/lib/derbytools.jar:/usr/local/glassfish3/javadb/lib/derbynet.jar:/usr/local/glassfish3/javadb/lib/derbyclient.jar com.sun.enterprise.admin.cli.optional.DerbyControl start 0.0.0.0 1527 true /usr/local/glassfish3/glassfish/databases
root      9577     1  0 May09 ?        00:29:44 /usr/java/jdk1.6.0_24/bin/java -cp /usr/local/glassfish3/glassfish/modules/glassfish.jar -XX:+UnlockDiagnosticVMOptions -XX:MaxPermSize=512m -XX:NewRatio=2 -Xmx256m -server -javaagent:/usr/local/glassfish3/glassfish/lib/monitor/btrace-agent.jar=unsafe=true,noServer=true -Dosgi.shell.telnet.maxconn=1 -Djdbc.drivers=org.apache.derby.jdbc.ClientDriver -Dfelix.fileinstall.disableConfigSave=false -Dfelix.fileinstall.dir=/usr/local/glassfish3/glassfish/modules/autostart/ -Djavax.net.ssl.keyStore=/usr/local/glassfish3/glassfish/domains/domain1/config/keystore.jks -Dosgi.shell.telnet.port=6666 -Djava.security.policy=/usr/local/glassfish3/glassfish/domains/domain1/config/server.policy -Dfelix.fileinstall.log.level=2 -Dfelix.fileinstall.poll=5000 -Dcom.sun.aas.instanceRoot=/usr/local/glassfish3/glassfish/domains/domain1 -Dosgi.shell.telnet.ip=127.0.0.1 -Dcom.sun.enterprise.config.config_environment_factory_class=com.sun.enterprise.config.serverbeans.AppserverConfigEnvironmentFactory -Djava.endorsed.dirs=/usr/local/glassfish3/glassfish/modules/endorsed:/usr/local/glassfish3/glassfish/lib/endorsed -Dcom.sun.aas.installRoot=/usr/local/glassfish3/glassfish -Djava.ext.dirs=/usr/java/jdk1.6.0_24/lib/ext:/usr/java/jdk1.6.0_24/jre/lib/ext:/usr/local/glassfish3/glassfish/domains/domain1/lib/ext -Dfelix.fileinstall.bundles.startTransient=true -Dfelix.fileinstall.bundles.new.start=true -Djavax.net.ssl.trustStore=/usr/local/glassfish3/glassfish/domains/domain1/config/cacerts.jks -Dorg.glassfish.additionalOSGiBundlesToStart=org.apache.felix.shell,org.apache.felix.gogo.runtime,org.apache.felix.gogo.shell,org.apache.felix.gogo.command -Dcom.sun.enterprise.security.httpsOutboundKeyAlias=s1as -DANTLR_USE_DIRECT_CLASS_LOADING=true -Djava.security.auth.login.config=/usr/local/glassfish3/glassfish/domains/domain1/config/login.conf -Dgosh.args=--nointeractive -Djava.library.path=/usr/local/glassfish3/glassfish/lib:/usr/java/jdk1.6.0_24/jre/lib/amd64/server:/usr/java/jdk1.6.0_24/jre/lib/amd64:/usr/java/jdk1.6.0_24/lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib com.sun.enterprise.glassfish.bootstrap.ASMain -domainname domain1 -asadmin-args --host,,,localhost,,,--port,,,4848,,,--secure=false,,,--terse=false,,,--echo=false,,,--interactive=true,,,start-domain,,,--verbose=false,,,--debug=false,,,--domaindir,,,/usr/local/glassfish3/glassfish/domains,,,domain1 -instancename server -verbose false -debug false -asadmin-classpath /usr/local/glassfish3/glassfish/modules/admin-cli.jar -asadmin-classname com.sun.enterprise.admin.cli.AsadminMain -upgrade false -type DAS -domaindir /usr/local/glassfish3/glassfish/domains/domain1 -read-stdin true
root     17965 17871  0 17:56 pts/0    00:00:00 grep java
[root@Omnis-Slice-2 ~]# 
[root@Omnis-Slice-2 ~]# kill -9 9495 9577
[root@Omnis-Slice-2 ~]# ps -ef | grep java
root     17967 17871  0 17:56 pts/0    00:00:00 grep java
[root@Omnis-Slice-2 ~]# service glassfish start
Waiting for domain1 to start ...........
Successfully started the domain : domain1
domain  Location: /usr/local/glassfish3/glassfish/domains/domain1
Log File: /usr/local/glassfish3/glassfish/domains/domain1/logs/server.log
Admin Port: 4848
Command start-domain executed successfully.
[root@Omnis-Slice-2 ~]#
Comment 20 Vince Kraemer 2011-05-13 20:02:26 UTC
(In reply to comment #18)
> Created an attachment (id=108278) [details]
> Server log from development server on CentOS
> 
> Same issue happens on Linux server running CentOS when doing "redeploy" through
> admin port:

What do you mean in the phrase 'when doing "redeploy" through admin port'?

Did you do something in NB (Which UI object was 'selected'? What menu item did you select?)

Did you do something at the command-line, like use asadmin or copy a war file into the directory glassfish3/glassfish/domains/domain1/autodeploy?

I have tried to replicate the issue that you are reporting, but I haven't had much luck yet. Getting detailed, step-by-step instructions on the actions that you take to run into this would be really helpful.  Getting info about the scale of the DB that is being used by OmnisTools would be really helpful, too.
Comment 21 rweaver 2011-05-15 00:53:07 UTC
Sorry, poor wording, meant redeploy via admin web page (:4848). Upload through the web interface using WAR file.

Just isolating that this isn't NB, and does appear to be GF only. Not doing anything special other than "redeploy" (or "run" from NetBeans).

I'm thinking that GF doesn't like running in low memory conditions - VPS is relatively small, and my Mac is 2Gb, so it gets pretty loaded with NB and GF running.

This is happening for me on both my local Mac and remote CentOS VPS (hosted at Slice Host).
Comment 22 Vince Kraemer 2011-05-15 14:16:19 UTC
Opened http://java.net/jira/browse/GLASSFISH-16649.

Closing this as wontfix, since the fix will be in the server code not NetBeans
Comment 23 Vince Kraemer 2011-05-15 14:17:51 UTC
rweaver: please start to watch the glassfish issue, since the assigned engineer wil probably need more info to replicate the issue and develop a fix.