Please use the Apache issue tracking system for new NetBeans issues (https://issues.apache.org/jira/projects/NETBEANS0/issues) !!
Bug 195288 - Running NetBeans 7.0 causes file operations in temporary folder to fail at random
Running NetBeans 7.0 causes file operations in temporary folder to fail at ra...
Status: RESOLVED DUPLICATE of bug 197927
Product: platform
Classification: Unclassified
Component: Filesystems
7.0
PC Windows Vista
: P2 with 2 votes (vote)
: 7.0.1
Assigned To: Jaroslav Tulach
issues@platform
:
: 198392 (view as bug list)
Depends on:
Blocks: 26230
  Show dependency treegraph
 
Reported: 2011-02-10 12:54 UTC by cschlichtherle
Modified: 2011-06-01 09:23 UTC (History)
7 users (show)

See Also:
Issue Type: DEFECT
:


Attachments
~/.netbeans/7.0beta2/var/log/messages.log (61.97 KB, text/plain)
2011-02-21 10:06 UTC, cschlichtherle
Details
Patch (824 bytes, patch)
2011-03-09 11:23 UTC, Yulia Novozhilova
Details | Diff
log (28.84 KB, text/plain)
2011-04-07 15:08 UTC, Yulia Novozhilova
Details
log with o.o.filesystems.DeepListener.level=FINE (59.47 KB, application/octet-stream)
2011-04-08 11:23 UTC, Tomas Danek
Details

Note You need to log in before you can comment on or make changes to this bug.
Description cschlichtherle 2011-02-10 12:54:11 UTC
Product Version = NetBeans IDE 7.0 Beta (Build 201011152355)
Operating System = Windows Vista version 6.0 running on x86
Java; VM; Vendor = 1.6.0_23
Runtime = Java HotSpot(TM) Client VM 19.0-b09

In a more complex Java project, I am creating, reading, writing and deleting files and directories in the temporary folder during unit testing.
These tests run fine on NetBeans 6.9.1, but as soon as I am running NetBeans 7.0 Beta, creating or deleting files or directories in the temporary folder fail at random.
This applies even if I run the tests from within NetBeans 6.9.1 and no project is opened in NetBeans 7.0 Beta.
So just running the process for NetBeans 7.0 Beta causes my file operations to fail at random - very strange!
Comment 1 Yulia Novozhilova 2011-02-11 11:03:05 UTC
Could you please attach the log file here (.netbeans/(_NB_version_)/var/log/messages.log) right after failure happened.
Comment 2 cschlichtherle 2011-02-21 10:06:24 UTC
Created attachment 106237 [details]
~/.netbeans/7.0beta2/var/log/messages.log

Here's the log after building the project with NetBeans 7.0 Beta 2.
Comment 3 Yulia Novozhilova 2011-03-09 10:30:20 UTC
Hi, sorry for a delay in respond. Thank you for the log but it looks fine.
Do I understand correctly that you have several unit test that perform some file operations and fail randomly when Beta 7.0 is running? Could you please attach an example of that test and java file that is tested? I have to be able to reproduce your issue to understand what is wrong and be able to help. It may be a java issue or whatever.
I'm downgrading this bug to P3 since it seems a quite rare case.
Comment 4 Yulia Novozhilova 2011-03-09 11:23:50 UTC
Created attachment 106850 [details]
Patch
Comment 5 cschlichtherle 2011-03-09 11:25:06 UTC
Yes, the problem is that accessing files in the temporary directory randomly fails with NetBeans 7.0 Beta 2 on Windows. To reproduce the issue, please use "hg clone https://hg.java.net/hg/truezip~v7", open the project in NetBeans 7.0 Beta 2 and run "mvn clean install" alias 'Clean and Build' from the project's context menu. You will then see various unit tests fail. The failing tests are all subclasses of the integration test class "de.schlichtherle.truezip.file.TFileTestSuite". When analyzing the logs and debugging any of these tests, you will notice that various java.io.File methods fail randomly when operating on files in the temporary directory.
Comment 6 cschlichtherle 2011-03-30 02:15:10 UTC
This bug is still existing in NetBeans 7.0 RC 1. Are you working on this?

When I look at the exceptions thrown, then they appear randomly, but their root cause is always a FileNotFoundException because of denied access. Here's an example stack trace (where "Zugriff verweigert" means "access denied"):

	... 34 more
Caused by: java.io.FileNotFoundException: C:\Users\christian\AppData\Local\Temp\tzp3295769303022183395.tar (Zugriff verweigert)
	at java.io.FileOutputStream.open(Native Method)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:179)
	at de.schlichtherle.truezip.fs.file.FileOutputSocket$1OutputStream.<init>(FileOutputSocket.java:97)
	at de.schlichtherle.truezip.fs.file.FileOutputSocket.newOutputStream(FileOutputSocket.java:143)
	at de.schlichtherle.truezip.socket.IOSocket.copy(IOSocket.java:105)
	at de.schlichtherle.truezip.fs.file.FileOutputSocket$1OutputStream.close(FileOutputSocket.java:114)
	at org.apache.tools.tar.TarBuffer.close(TarBuffer.java:450)
	at org.apache.tools.tar.TarOutputStream.close(TarOutputStream.java:147)
	at de.schlichtherle.truezip.fs.archive.FsMultiplexedArchiveOutputShop.close(FsMultiplexedArchiveOutputShop.java:294)
	at de.schlichtherle.truezip.socket.ConcurrentOutputShop.close(ConcurrentOutputShop.java:165)
	at de.schlichtherle.truezip.fs.archive.FsDefaultArchiveController.commitSync(FsDefaultArchiveController.java:544)
	... 46 more

The access is denied to a temporary file which has just been created before using File.createTempFile(*). You can reproduce this issue by cloning the Mecurial repository https://hg.java.net/hg/truezip~v7 and running the command "Clean and Build" alias "mvn clean install" from the project directory.
Comment 7 cschlichtherle 2011-04-06 14:16:51 UTC
Just to state my point: This is a show-stopper, i.e. NetBeans 7.0 can't be used for unit testing the TrueZIP project.
Comment 8 Marian Mirilovic 2011-04-06 15:54:36 UTC
Tonda, could your team evaluate this ASAP ? Thanks in advance.
Comment 9 Tomas Danek 2011-04-07 10:01:51 UTC
it seems to be really windows specific, cannot reproduce on my mac.
Comment 10 Tomas Danek 2011-04-07 10:49:05 UTC
i was not able to reproduce even on my xp:
- clone mentioned project
- invoke clean and build on top level maven project
- everything is built and tests pass 100%

tried even running tests separately, still no luck. We might try on Vista...
Comment 11 Yulia Novozhilova 2011-04-07 11:05:39 UTC
Also not reproducible on my Win XP. Will try to find Vista.
Comment 12 cschlichtherle 2011-04-07 11:09:04 UTC
It sure is Windows specific because Windows is much more restrictive on accessing files concurrently.

Is there a background thread in NB 7.0 RC 1 which monitors the temporary file folder? I presume this might cause the OS to consider it being concurrent access to the same file and prevent it?!
Comment 13 Yulia Novozhilova 2011-04-07 15:07:05 UTC
Managed to reproduce on Vista. Log is attached.
Comment 14 Yulia Novozhilova 2011-04-07 15:08:13 UTC
Created attachment 107588 [details]
log
Comment 15 cschlichtherle 2011-04-07 15:44:37 UTC
I looked at your log. It's exactly the showing the problem: When you run the build several times, it randomly fails in different sub-classes of de.schlichtherle.truezip.file.TFileTestSuite. This super class runs various I/O tests which use a lot of temporary files. These temporary files randomly disappear or get inaccessible with NB 7.0 RC 1 on Vista with JDK 1.6.0_24. This does NOT happen with NB 6.9.1, so I suppose there must be some interaction with the temporary file folder and NB 7.0 RC 1.
Comment 16 Jesse Glick 2011-04-07 16:06:58 UTC
If you can reproduce, try running with -J-Dorg.netbeans.modules.masterfs.watcher.disable=false and see if that fixes the issue. If so, block bug #26230 and add jtulach to CC; you will need to add some logging to org.openide.filesystems.DeepListener.init to record the File path and the call stack.

Otherwise I do not have any suggestions other than to try to find some tool which would tell you what Java call stack is responsible for opening the files in the temp dir. Logging in org.netbeans.modules.masterfs.filebasedfs.utils.FileChangedManager.checkRead might suffice.

I doubt this has anything to do with the JUnit module, but need some kind of logging to diagnose the actual culprit. (Has anyone managed to reproduce on XP yet? If so, I can try to help.)
Comment 17 cschlichtherle 2011-04-07 17:38:11 UTC
I've added -J-Dorg.netbeans.modules.masterfs.watcher.disable=false to netbeans_default_options in netbeans.conf, but the issue still persists. :-(
Comment 18 Jesse Glick 2011-04-07 18:02:13 UTC
Sorry, I meant =true not =false (you want to try disabling it).
Comment 19 cschlichtherle 2011-04-07 19:18:49 UTC
Bingo!

Adding -J-Dorg.netbeans.modules.masterfs.watcher.disable=true to netbeans_default_options in netbeans.conf of NetBeans 7.0 RC 1 makes the unit tests run flawlessly like it used to be in NetBeans 6.9.1.

I have added 26230 to the block list and jtulach AT netbeans.org . I don't understand what else to do, but steps for repeating are explained above and a log is attached, too. Please tell me if you expect something more.

Thank you very much for looking into this so far.
Comment 20 Jesse Glick 2011-04-07 20:37:56 UTC
I added some diagnostics in core-main #bf55960efcb2. (An automated annotation will appear in this issue when a post-7.0 dev build with this patch is published.) -J-Dorg.openide.filesystems.DeepListener.level=FINE enables the diagnostics to appear in console/log. In Ubuntu, opened the TrueZIP root project and clicked Test Project; all 399 tests passed. That was expected (problem known to occur just on Windows) but I thought that if something was mistakenly listening to ${java.io.tmpdir} it would probably occur on all platforms and only pose a visible problem due to file locks on Windows. But there was no such event.
Comment 21 Jesse Glick 2011-04-07 20:51:49 UTC
Failed to reproduce any problem on XP (inside VBox). Might be specific to Vista? Or involve some interaction with virus checkers, Vista's security system, etc.?
Comment 22 cschlichtherle 2011-04-07 22:11:07 UTC
The issue persists even if the virus checker (Microsoft Security Essentials) is switched off.

Unfortunately I don't have Windows 7 available. It would be nice to know if the issue is present with this OS, too.
Comment 23 cschlichtherle 2011-04-08 06:15:20 UTC
To help estimating the priority of this issue: This issue may affect ANY THIRD PARTY PROCESS running - not just JVM processes. Since almost any application uses temporary files, I think this is really important.
Comment 24 cschlichtherle 2011-04-08 06:49:48 UTC
E.g. when I run NB 6.9.1 and NB 7.0 RC 1 concurrently with -J-Dorg.netbeans.modules.masterfs.watcher.disable=false then the unit tests fail even when I run them in NB 6.9.1!!!
Comment 25 Marian Mirilovic 2011-04-08 08:25:43 UTC
Ok, we will still work on this issue, but do not keep it as stopper for NB 7.0.
Comment 26 Tomas Danek 2011-04-08 08:31:17 UTC
(In reply to comment #21)
> Failed to reproduce any problem on XP (inside VBox). Might be specific to
> Vista? Or involve some interaction with virus checkers, Vista's security
> system, etc.?

I'll try to reproduce on vista in dev build with added logging.
Comment 27 Quality Engineering 2011-04-08 08:53:22 UTC
Integrated into 'main-golden', will be available in build *201104080400* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/bf55960efcb2
User: Jesse Glick <jglick@netbeans.org>
Log: Diagnostics for #195288.
Comment 28 cschlichtherle 2011-04-08 08:55:39 UTC
(In reply to comment #25)
> Ok, we will still work on this issue, but do not keep it as stopper for NB 7.0.

Then you'll risk negatively affecting any other third party process. I can imagine this could seriously damage the public perception of NB 7.0.
Comment 29 Tomas Danek 2011-04-08 11:21:32 UTC
finally reproduced on Windows Vista. Attaching log with FS logging enabled.
Comment 30 Tomas Danek 2011-04-08 11:23:24 UTC
Created attachment 107603 [details]
log with o.o.filesystems.DeepListener.level=FINE
Comment 31 Jesse Glick 2011-04-08 21:44:47 UTC
That log file looks similar to mine on Linux and XP; there is no indication that a recursive listener is being attached to the temp dir. So I have no clue what might be going on. Maybe some weird global side effect of making those Win32 calls on Vista has some effect on access rules? This does not sound like the usual Windows problem of the IDE holding a file open and thus preventing other processes from writing to it or deleting it; that would not produce "access denied" errors as far as I can recall, and you would expect it to happen on XP as well as Vista.
Comment 32 cschlichtherle 2011-04-26 21:54:24 UTC
This bug still applies to NB 7.0.
Comment 33 Jesse Glick 2011-05-06 14:53:58 UTC
*** Bug 198392 has been marked as a duplicate of this bug. ***
Comment 34 cschlichtherle 2011-05-06 15:39:14 UTC
If the information in the duplicate is correct, then we know now that the issue applies to Windows 7, too, not just Windows Vista.
Comment 35 velodiver 2011-05-06 15:41:48 UTC
I filed "the duplicate".  Yes, I am running Windows 7 (x64).
Comment 36 bushychris 2011-05-13 20:38:52 UTC
I have observed this issue on both Windows Vista x64 and Windows 7 x64.  Using the suggested workaround (-J-Dorg.netbeans.modules.masterfs.watcher.disable=true) was successful in resolving the issue on the Vista system.  However, this workaround does not seem to work on my Windows 7 system.
Comment 37 superole2 2011-05-30 08:36:42 UTC
(In reply to comment #23)
> To help estimating the priority of this issue: This issue may affect ANY THIRD
> PARTY PROCESS running - not just JVM processes. Since almost any application
> uses temporary files, I think this is really important.

...and it does not only apply to temporary files either. It seems to apply to all files relevant to open projects in NB7.
Comment 38 Antonin Nebuzelsky 2011-05-30 10:45:46 UTC
Reassigning to Filesystems for evaluation.
Comment 39 Jaroslav Tulach 2011-06-01 09:23:49 UTC

*** This bug has been marked as a duplicate of bug 197927 ***


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2014, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo