This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 168109 - Long (>30 min) hang moving files from one project to another
Summary: Long (>30 min) hang moving files from one project to another
Status: RESOLVED WORKSFORME
Alias: None
Product: versioncontrol
Classification: Unclassified
Component: Mercurial (show other bugs)
Version: 6.x
Hardware: All All
: P3 blocker (vote)
Assignee: Marian Petras
URL:
Keywords: PERFORMANCE
Depends on:
Blocks:
 
Reported: 2009-07-04 02:19 UTC by _ tboudreau
Modified: 2009-09-21 16:42 UTC (History)
3 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Thread dump (52.13 KB, text/plain)
2009-07-04 02:20 UTC, _ tboudreau
Details
log (12.66 KB, application/x-gzip)
2009-07-04 02:20 UTC, _ tboudreau
Details
thread-dump with unwrapped lines (51.15 KB, text/plain)
2009-07-07 13:43 UTC, Marian Petras
Details
patch representing changes made in step 3) in the corrected instructions (31.75 KB, patch)
2009-07-08 20:33 UTC, Marian Petras
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description _ tboudreau 2009-07-04 02:19:43 UTC
Take maven project A, under Hg control.  Take class A.A.java and extract a superclass A.A1.java.  A already is committed
in Hg; A.A1 is a new file.

Create new project B, under the same Hg root, using the Maven Quickstart archetype.  Build it once.

Right click Libraries in A and add a dependency on B using the Open Projects tab.

Delete the existing packages in B, create a new B.pkg.  Cut files A.A1 and a few dependent ones A.B.java, A.C.java, A.D
through E.properties.

Refactor-Paste them into B.pkg.

Refactoring dialog comes up;  click Refactor.  AWT thread has now been hung for over 30 minutes and counting (I haven't
quite given up hope yet).

Thread dump and log attached.

Seems like some nasty interaction between Maven support and Hg support actually trying to move files and keep their
metadata intact.

Hmm, some interesting stuff was just logged:
WARNING [org.netbeans.modules.mercurial]: command: [hg, rename, -A, --repository, H:\work\hgwicket,
--cwd, H:\work\hgwicket, complete-quickmodels-skeleton-app\src\main\java\com\timboudreau\appskeleton
\ExpiredUserPruneThread.java, base-quickmodels-application\src\main\java\com\timboudreau\dbapp\base\
ExpiredUserPruneThread.java]
WARNING [org.netbeans.modules.mercurial]: output: [abort: working directory of H:\work\hgwicket: tim
ed out waiting for lock held by satan:3124]
INFO [org.netbeans.modules.mercurial]: RefreshTask called refreshAll H:\work\hgwicket\complete-quick
models-skeleton-app\target\wicket-quickmodels-application-base-1.0-SNAPSHOT\WEB-INF\classes\com\timb
oudreau
INFO [org.netbeans.modules.mercurial]: RefreshTask called refreshAll H:\work\hgwicket\complete-quick
models-skeleton-app\target\wicket-quickmodels-application-base-1.0-SNAPSHOT\WEB-INF\classes\com\timb
oudreau\appskeleton
INFO [org.netbeans.modules.mercurial]: RefreshTask called refreshAll H:\work\hgwicket\complete-quick
models-skeleton-app\src\main\java\com\timboudreau\appskeleton\signin
INFO [org.netbeans.modules.mercurial]: RefreshTask called refreshAll H:\work\hgwicket\complete-quick
models-skeleton-app\target\wicket-quickmodels-application-base-1.0-SNAPSHOT\WEB-INF\classes\com\timb
oudreau\appskeleton\signin
INFO [org.netbeans.modules.mercurial]: RefreshTask called refreshAll H:\work\hgwicket\base-quickmode
ls-application\src\main\java\com\timboudreau\dbapp
Comment 1 _ tboudreau 2009-07-04 02:20:11 UTC
Created attachment 84352 [details]
Thread dump
Comment 2 _ tboudreau 2009-07-04 02:20:53 UTC
Created attachment 84353 [details]
log
Comment 3 _ tboudreau 2009-07-04 05:31:57 UTC
Gave up after 45 minutes and killed the IDE.  On restart, the files were in fact refactored correctly - no idea if the
hg metadata was preserved, but it's a small loss.

I've got to say:
"timed out waiting for lock held by satan:3124"

is cute (I had to call my windows machine *something*).

Interestingly, on the command line, when I try
hg commit
now I get 
waiting for lock on working directory of H:\work\hgwicket held by 'satan:3124'

Probably windows-file-locking related?  Not sure if it's the IDE, or some long-dead hg process that's causing the
problem.  No, shutting down the IDE doens't help...
Comment 4 Jesse Glick 2009-07-06 17:02:33 UTC
Mercurial module is clearly what is consuming time here. Refactoring module also appears to be doing entirely
unnecessary work in EQ. maven.j2ee.web.CopyOnSave is doing something which it probably should not be doing, since I do
not see anything in Tim's description about using a war-packaging project.
Comment 5 _ tboudreau 2009-07-06 19:21:51 UTC
Actually, it was a Maven web project packaged as a war file.  http://timboudreau.com/hg/projects/wicket/rev/d71357219e98
Comment 6 Milos Kleint 2009-07-07 12:48:18 UTC
reassigning to mercurial for evaluation, nothing in the thread dump can be directly attributed to maven support as far
as I can tell.
Comment 7 Marian Petras 2009-07-07 13:43:20 UTC
Created attachment 84438 [details]
thread-dump with unwrapped lines
Comment 8 Marian Petras 2009-07-07 19:38:18 UTC
There are three threads involved in the delay:

   "MercurialRefresh"
   "OpenIDE-request-processor-93"
   "AWT-EventQueue-1"

Thread "OpenIDE-request-processor-93" is doing some "hg remove" operations and using mutex write access for them
(BaseFileObj.java:598). The other two threads are waiting until it leaves the section guarded by the write access mutex.
Comment 9 Marian Petras 2009-07-07 19:41:55 UTC
Although it is quite clear from the thread-dump that there are two threads waiting for another thread to finish, it is
not clear how long that thread spends in the section guarded by the mutex. It is also not clear whether there is some
progress or not. If there is no progress, it is not clear what blocks the progress, but it is possible it is the file
lock that is pending.
Comment 10 Marian Petras 2009-07-07 20:01:11 UTC
I have tried to reproduce the bug (on Windows XP) but failed. Could you answer the following questions, please?

Regarding the first paragraph of the bug description, sentence "A already is committed in Hg; A.A1 is a new file.":
What exactly was the sequence?

     a) create new A.A, commit A.A, extract super class A.A1, commit A.A again
       -OR-
     b) create new A.A, commit A.A, extract super class A.A1
       -OR-
     c) create new A.A, extract a super class, commit A.A

Regarding the fourth paragraph:
Where do these files come from, what is their content and what is their Hg status?

     A.B.java, A.C.java, A.D through E.properties

What is disk H:? Is it a local disk or it is a mapped network location?

What version of Mercurial were you using (hg --version)?

You experienced this bug when you were running with JDK 7-ea.
Are you able to reproduce this bug with a stable JDK (which one)?

What is the version of NetBeans that you were running?
According to the log file, the product version is "NetBeans IDE Dev (Build 020117)".
Is it a dev. version of NB 6.7? Or dev. version of NB 6.8? Or some build from 17 January 2002?
Comment 11 _ tboudreau 2009-07-08 02:25:01 UTC
I have tried to reproduce the bug (on Windows XP) but failed. Could you answer the following questions, please?

Regarding the first paragraph of the bug description, sentence "A already is committed in Hg; A.A1 is a new file.":
What exactly was the sequence?

     a) create new A.A, commit A.A, extract super class A.A1, commit A.A again
       -OR-
     b) create new A.A, commit A.A, extract super class A.A1
       -OR-
     c) create new A.A, extract a super class, commit A.A

TB: b) is the scenario.  A was an existing project already committed and pushed in hg.  Project B was a new project
under the same hg root as A.

Regarding the fourth paragraph:
Where do these files come from, what is their content and what is their Hg status?

     A.B.java, A.C.java, A.D through E.properties

TB: A.B.java and A.C.java were package private dependencies of A, which became dependencies of the super-interface once
I extracted it, so they needed to be moved along with A1.java

What is disk H:? Is it a local disk or it is a mapped network location?

What version of Mercurial were you using (hg --version)?

TB:  Mercurial Distributed SCM (version 4c030ada58d2) - built from source.  Due to a Windows-specific bug in the current
release of Mercurial, I cannot use the released version, I need a build that contains the fix.


You experienced this bug when you were running with JDK 7-ea.
Are you able to reproduce this bug with a stable JDK (which one)?

TB:  I can try this when I have time.

What is the version of NetBeans that you were running?
According to the log file, the product version is "NetBeans IDE Dev (Build 020117)".
Is it a dev. version of NB 6.7? Or dev. version of NB 6.8? Or some build from 17 January 2002?

TB:  My computer's system clock had been reset when I built it.  It is a build from probably about 11 June, 2009 - I
built it just after returning home, the Wednesday after JavaOne.
Comment 12 _ tboudreau 2009-07-08 04:30:36 UTC
If you want to specifically try to recreate it, do the following:
hg clone http://timboudreau.com/hg/projects/wicket
(don't worry, it's small - they're all maven projects - if the server's down, let me know - the power has been out here
a few times today)

Open the projects voting-app and base-quickmodels-application.

What I had done was factor 
base-quickmodels-application\src\main\java\com\timboudreau\dbapp\base\BaseDbApplication.java
out of 
hgwicket\voting-app\src\main\java\com\timboudreau\voteit\VotingApplication.java - I used extract superclass to extract
the methods you see in BaseDbApplication, into a class in the same package.  Then I created the new project and
cut/pasted BaseDbApplication.java, AppSession.java, LostPassword.properties, Welcome.properties and Warning.properties
into the new application.

If you want to replicate exactly what I did, just do an hg clone only up to d71357219e98, or from an hour earlier than 
http://timboudreau.com/hg/projects/wicket/rev/d71357219e98 EST.

BTW, one question I didn't answer:  H: is a local drive - not only that, it's a 3x250GB 10K RPM drive raid-0 array - it
has insanely good throughput.

I think your suspicion of windows file locking is probably correct.

This could also be avoided if the window system didn't fire a selected node change when a modal dialog was shown (since
the user cannot access any main menu or toolbar actions while a modal dialog is present anyway) - I'll file that separately.
Comment 13 _ tboudreau 2009-07-08 04:36:45 UTC
General issue w/ dialogs and selection updates filed as issue 168260
Comment 14 Marian Petras 2009-07-08 18:54:14 UTC
Corrected instructions (I hope):

If you want to specifically try to recreate it, do the following:

1) hg clone -r d91b6a49d6ce http://timboudreau.com/hg/projects/wicket

    (Don't worry, it's small - they're all maven projects - if the server's down, let me know - the power has been out
    here a few times today.)

2) Open project "complete-quickmodels-skeleton-app".

3) Factor

      complete-quickmodels-skeleton-app/src/main/java/com/timboudreau/appskeleton/BaseDbApplication.java

   out of

      complete-quickmodels-skeleton-app/src/main/java/com/timboudreau/appskeleton/WicketApplication.java

   I used refactoring "extract superclass" to extract the methods you see in BaseDbApplication,
   into a class in the same package.

4) Then I created new project "base-quickmodels-application" and cut/pasted files BaseDbApplication.java,
   AppSession.java, LostPassword.properties, Welcome.properties and Warning.properties into it,
   to package "com.timboudreau.dbapp.base".
Comment 15 Marian Petras 2009-07-08 20:33:08 UTC
Created attachment 84512 [details]
patch representing changes made in step 3) in the corrected instructions
Comment 16 Marian Petras 2009-07-08 20:35:37 UTC
Step 3) can now be done quickly by importing the attached patch:

    hg import --no-commit step1.diff

Or an alternative way:

    hg import step1.diff
    hg rollback
Comment 17 _ tboudreau 2009-07-09 22:09:05 UTC
Do you mean this as a workaround, or was something in NetBeans actually fixed?
Comment 18 Marian Petras 2009-07-10 16:17:29 UTC
It is neither a workaround nor I did anything in the NetBeans code. It is just a note for a developer fixing this bug
(me or anyone else) such that they can try to reproduce the bug more easily.
Comment 19 _ tboudreau 2009-07-10 16:59:43 UTC
So you *were* able to reproduce it?
Comment 20 Marian Petras 2009-07-10 17:07:39 UTC
No, I was able to *try to reproduce* it. But during that try, HgCommand.doRemove() was not called at all and the
refactoring passed in a few seconds.
Comment 21 Marian Petras 2009-09-21 16:42:48 UTC
Apparently, there was a problem with operating system's file locks.

I have never seen the issue and I am not able to reproduce it. The logs do not seem to provide enough information for
finding possible scenarios that would lead to the IDE hang.

Marked as WORKSFORME. If you are able to reproduce the issue with a stable JDK, or if a similar hang occured multiple
times (with a stable JDK), please reopen this bug (or file a new one) and provide as much information as possible.