Could you please attach the log file here (.netbeans/(_NB_version_)/var/log/messages.log) right after failure happened.

Created attachment 106237 [details]
~/.netbeans/7.0beta2/var/log/messages.log

Here's the log after building the project with NetBeans 7.0 Beta 2.

Hi, sorry for a delay in respond. Thank you for the log but it looks fine.
Do I understand correctly that you have several unit test that perform some file operations and fail randomly when Beta 7.0 is running? Could you please attach an example of that test and java file that is tested? I have to be able to reproduce your issue to understand what is wrong and be able to help. It may be a java issue or whatever.
I'm downgrading this bug to P3 since it seems a quite rare case.

Created attachment 106850 [details]
Patch

Yes, the problem is that accessing files in the temporary directory randomly fails with NetBeans 7.0 Beta 2 on Windows. To reproduce the issue, please use "hg clone https://hg.java.net/hg/truezip~v7", open the project in NetBeans 7.0 Beta 2 and run "mvn clean install" alias 'Clean and Build' from the project's context menu. You will then see various unit tests fail. The failing tests are all subclasses of the integration test class "de.schlichtherle.truezip.file.TFileTestSuite". When analyzing the logs and debugging any of these tests, you will notice that various java.io.File methods fail randomly when operating on files in the temporary directory.

This bug is still existing in NetBeans 7.0 RC 1. Are you working on this?

When I look at the exceptions thrown, then they appear randomly, but their root cause is always a FileNotFoundException because of denied access. Here's an example stack trace (where "Zugriff verweigert" means "access denied"):

	... 34 more
Caused by: java.io.FileNotFoundException: C:\Users\christian\AppData\Local\Temp\tzp3295769303022183395.tar (Zugriff verweigert)
	at java.io.FileOutputStream.open(Native Method)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:179)
	at de.schlichtherle.truezip.fs.file.FileOutputSocket$1OutputStream.<init>(FileOutputSocket.java:97)
	at de.schlichtherle.truezip.fs.file.FileOutputSocket.newOutputStream(FileOutputSocket.java:143)
	at de.schlichtherle.truezip.socket.IOSocket.copy(IOSocket.java:105)
	at de.schlichtherle.truezip.fs.file.FileOutputSocket$1OutputStream.close(FileOutputSocket.java:114)
	at org.apache.tools.tar.TarBuffer.close(TarBuffer.java:450)
	at org.apache.tools.tar.TarOutputStream.close(TarOutputStream.java:147)
	at de.schlichtherle.truezip.fs.archive.FsMultiplexedArchiveOutputShop.close(FsMultiplexedArchiveOutputShop.java:294)
	at de.schlichtherle.truezip.socket.ConcurrentOutputShop.close(ConcurrentOutputShop.java:165)
	at de.schlichtherle.truezip.fs.archive.FsDefaultArchiveController.commitSync(FsDefaultArchiveController.java:544)
	... 46 more

The access is denied to a temporary file which has just been created before using File.createTempFile(*). You can reproduce this issue by cloning the Mecurial repository https://hg.java.net/hg/truezip~v7 and running the command "Clean and Build" alias "mvn clean install" from the project directory.

Just to state my point: This is a show-stopper, i.e. NetBeans 7.0 can't be used for unit testing the TrueZIP project.

Tonda, could your team evaluate this ASAP ? Thanks in advance.

it seems to be really windows specific, cannot reproduce on my mac.

i was not able to reproduce even on my xp:
- clone mentioned project
- invoke clean and build on top level maven project
- everything is built and tests pass 100%

tried even running tests separately, still no luck. We might try on Vista...

Also not reproducible on my Win XP. Will try to find Vista.

It sure is Windows specific because Windows is much more restrictive on accessing files concurrently.

Is there a background thread in NB 7.0 RC 1 which monitors the temporary file folder? I presume this might cause the OS to consider it being concurrent access to the same file and prevent it?!

Managed to reproduce on Vista. Log is attached.

Created attachment 107588 [details]
log

I looked at your log. It's exactly the showing the problem: When you run the build several times, it randomly fails in different sub-classes of de.schlichtherle.truezip.file.TFileTestSuite. This super class runs various I/O tests which use a lot of temporary files. These temporary files randomly disappear or get inaccessible with NB 7.0 RC 1 on Vista with JDK 1.6.0_24. This does NOT happen with NB 6.9.1, so I suppose there must be some interaction with the temporary file folder and NB 7.0 RC 1.

If you can reproduce, try running with -J-Dorg.netbeans.modules.masterfs.watcher.disable=false and see if that fixes the issue. If so, block bug #26230 and add jtulach to CC; you will need to add some logging to org.openide.filesystems.DeepListener.init to record the File path and the call stack.

Otherwise I do not have any suggestions other than to try to find some tool which would tell you what Java call stack is responsible for opening the files in the temp dir. Logging in org.netbeans.modules.masterfs.filebasedfs.utils.FileChangedManager.checkRead might suffice.

I doubt this has anything to do with the JUnit module, but need some kind of logging to diagnose the actual culprit. (Has anyone managed to reproduce on XP yet? If so, I can try to help.)

I've added -J-Dorg.netbeans.modules.masterfs.watcher.disable=false to netbeans_default_options in netbeans.conf, but the issue still persists. :-(

Sorry, I meant =true not =false (you want to try disabling it).

Bingo!

Adding -J-Dorg.netbeans.modules.masterfs.watcher.disable=true to netbeans_default_options in netbeans.conf of NetBeans 7.0 RC 1 makes the unit tests run flawlessly like it used to be in NetBeans 6.9.1.

I have added 26230 to the block list and jtulach AT netbeans.org . I don't understand what else to do, but steps for repeating are explained above and a log is attached, too. Please tell me if you expect something more.

Thank you very much for looking into this so far.

I added some diagnostics in core-main #bf55960efcb2. (An automated annotation will appear in this issue when a post-7.0 dev build with this patch is published.) -J-Dorg.openide.filesystems.DeepListener.level=FINE enables the diagnostics to appear in console/log. In Ubuntu, opened the TrueZIP root project and clicked Test Project; all 399 tests passed. That was expected (problem known to occur just on Windows) but I thought that if something was mistakenly listening to ${java.io.tmpdir} it would probably occur on all platforms and only pose a visible problem due to file locks on Windows. But there was no such event.

Failed to reproduce any problem on XP (inside VBox). Might be specific to Vista? Or involve some interaction with virus checkers, Vista's security system, etc.?

The issue persists even if the virus checker (Microsoft Security Essentials) is switched off.

Unfortunately I don't have Windows 7 available. It would be nice to know if the issue is present with this OS, too.

To help estimating the priority of this issue: This issue may affect ANY THIRD PARTY PROCESS running - not just JVM processes. Since almost any application uses temporary files, I think this is really important.

E.g. when I run NB 6.9.1 and NB 7.0 RC 1 concurrently with -J-Dorg.netbeans.modules.masterfs.watcher.disable=false then the unit tests fail even when I run them in NB 6.9.1!!!

Ok, we will still work on this issue, but do not keep it as stopper for NB 7.0.

(In reply to comment #21)
> Failed to reproduce any problem on XP (inside VBox). Might be specific to
> Vista? Or involve some interaction with virus checkers, Vista's security
> system, etc.?

I'll try to reproduce on vista in dev build with added logging.

Integrated into 'main-golden', will be available in build *201104080400* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/bf55960efcb2
User: Jesse Glick <jglick@netbeans.org>
Log: Diagnostics for #195288.

(In reply to comment #25)
> Ok, we will still work on this issue, but do not keep it as stopper for NB 7.0.

Then you'll risk negatively affecting any other third party process. I can imagine this could seriously damage the public perception of NB 7.0.

finally reproduced on Windows Vista. Attaching log with FS logging enabled.

Created attachment 107603 [details]
log with o.o.filesystems.DeepListener.level=FINE

That log file looks similar to mine on Linux and XP; there is no indication that a recursive listener is being attached to the temp dir. So I have no clue what might be going on. Maybe some weird global side effect of making those Win32 calls on Vista has some effect on access rules? This does not sound like the usual Windows problem of the IDE holding a file open and thus preventing other processes from writing to it or deleting it; that would not produce "access denied" errors as far as I can recall, and you would expect it to happen on XP as well as Vista.

This bug still applies to NB 7.0.

*** Bug 198392 has been marked as a duplicate of this bug. ***

If the information in the duplicate is correct, then we know now that the issue applies to Windows 7, too, not just Windows Vista.

I filed "the duplicate".  Yes, I am running Windows 7 (x64).

I have observed this issue on both Windows Vista x64 and Windows 7 x64.  Using the suggested workaround (-J-Dorg.netbeans.modules.masterfs.watcher.disable=true) was successful in resolving the issue on the Vista system.  However, this workaround does not seem to work on my Windows 7 system.

(In reply to comment #23)
> To help estimating the priority of this issue: This issue may affect ANY THIRD
> PARTY PROCESS running - not just JVM processes. Since almost any application
> uses temporary files, I think this is really important.

...and it does not only apply to temporary files either. It seems to apply to all files relevant to open projects in NB7.

Reassigning to Filesystems for evaluation.

*** This bug has been marked as a duplicate of bug 197927 ***