Pete has been having troubles with commit validation on his machine; it consistently fails with OOMEs. I have been
looking at his log file, which is full (~83 Mb!) of stuff like


WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file
C:\work\main\C:\work\main\ide.kit\build\test\qa-functional\work\userdir0\config\Modules failed: java.io.IOException: The
filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file
C:\work\main\C:\work\main\ide.kit\build\test\qa-functional\work\userdir0\config\Modules failed: java.io.IOException: The
filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file
C:\work\main\C:\work\main\ide.kit\build\test\qa-functional\work\userdir0\config\Modules failed: java.io.IOException: The
filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file
C:\work\main\C:\work\main\ide.kit\build\test\qa-functional\work\userdir0\config\Modules failed: java.io.IOException: The
filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file
C:\work\main\C:\work\main\ide.kit\build\test\qa-functional\work\userdir0\config\Modules failed: java.io.IOException: The
filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file
C:\work\main\C:\work\main\ide.kit\build\test\qa-functional\work\userdir0\config\Modules failed: java.io.IOException: The
filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file
C:\work\main\C:\work\main\ide.kit\build\test\qa-functional\work\userdir0\config failed: java.io.IOException: The
filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file
C:\work\main\C:\work\main\ide.kit\build\test\qa-functional\work\userdir0\config failed: java.io.IOException: The
filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file
C:\work\main\C:\work\main\ide.kit\build\test\qa-functional\work\userdir0 failed: java.io.IOException: The filename,
directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file
C:\work\main\C:\work\main\ide.kit\build\test\qa-functional\work\userdir0 failed: java.io.IOException: The filename,
directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file
C:\work\main\C:\work\main\ide.kit\build\test\qa-functional\work failed: java.io.IOException: The filename, directory
name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file
C:\work\main\C:\work\main\ide.kit\build\test\qa-functional\work failed: java.io.IOException: The filename, directory
name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file
C:\work\main\C:\work\main\ide.kit\build\test\qa-functional failed: java.io.IOException: The filename, directory name, or
volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file
C:\work\main\C:\work\main\ide.kit\build\test\qa-functional failed: java.io.IOException: The filename, directory name, or
volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file C:\work\main\C:\work\main\ide.kit\build\test
failed: java.io.IOException: The filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file C:\work\main\C:\work\main\ide.kit\build\test
failed: java.io.IOException: The filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file C:\work\main\C:\work\main\ide.kit\build failed:
java.io.IOException: The filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file C:\work\main\C:\work\main\ide.kit\build failed:
java.io.IOException: The filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file C:\work\main\C:\work\main\ide.kit failed:
java.io.IOException: The filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file C:\work\main\C:\work\main\ide.kit failed:
java.io.IOException: The filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file C:\work\main\C:\work\main failed:
java.io.IOException: The filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file C:\work\main\C:\work\main failed:
java.io.IOException: The filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file C:\work\main\C:\work failed: java.io.IOException:
The filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file C:\work\main\C:\work failed: java.io.IOException:
The filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file C:\work\main\C: failed: java.io.IOException: The
filename, directory name, or volume label syntax is incorrect
WARNING [org.openide.filesystems.FileUtil]: getCanonicalFile() on file C:\work\main\C: failed: java.io.IOException: The
filename, directory name, or volume label syntax is incorrect



(Not obvious from these messages what is triggering this. Currently you cannot get a stack trace to see who the caller
is that is creating this funny path name; I will put in a patch to make it possible for the future.)


Now we get to the fun part:

java.lang.OutOfMemoryError: Java heap space
	at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:95)
	at java.io.PrintStream.write(PrintStream.java:412)
	at org.apache.tools.ant.util.TeeOutputStream.write(TeeOutputStream.java:82)
	at java.io.PrintStream.write(PrintStream.java:412)
	at sun.nio.cs.StreamEncoder$CharsetSE.writeBytes(StreamEncoder.java:336)
	at sun.nio.cs.StreamEncoder$CharsetSE.implWrite(StreamEncoder.java:395)
	at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:136)
	at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:146)
	at java.io.OutputStreamWriter.write(OutputStreamWriter.java:204)
	at java.io.Writer.write(Writer.java:126)
	at java.util.logging.StreamHandler.publish(StreamHandler.java:192)
	at org.netbeans.core.startup.TopLogging$NonClose.publish(TopLogging.java:428)
	at java.util.logging.Logger.log(Logger.java:452)
	at java.util.logging.Logger.doLog(Logger.java:474)
	at java.util.logging.Logger.log(Logger.java:497)
	at java.util.logging.Logger.warning(Logger.java:1000)
	at org.openide.filesystems.FileUtil.normalizeFileOnWindows(FileUtil.java:1440)
	at org.openide.filesystems.FileUtil.normalizeFile(FileUtil.java:1344)
	at org.netbeans.modules.masterfs.filebasedfs.FileBasedURLMapper.getFileObjects(FileBasedURLMapper.java:118)
	at org.netbeans.modules.masterfs.MasterURLMapper.getFileObjects(MasterURLMapper.java:61)
	at org.openide.filesystems.URLMapper.findFileObject(URLMapper.java:213)
	at org.openide.filesystems.FileUtil.toFileObject(FileUtil.java:673)
	at org.netbeans.modules.versioning.VersioningAnnotationProvider.refreshAnnotations(VersioningAnnotationProvider.java:245)
	at org.netbeans.modules.versioning.VersioningManager.propertyChange(VersioningManager.java:299)
	at java.beans.PropertyChangeSupport.firePropertyChange(PropertyChangeSupport.java:333)
	at java.beans.PropertyChangeSupport.firePropertyChange(PropertyChangeSupport.java:270)
	at org.netbeans.modules.versioning.spi.VersioningSystem.fireStatusChanged(VersioningSystem.java:218)
	at org.netbeans.modules.versioning.spi.VersioningSystem.fireStatusChanged(VersioningSystem.java:236)
	at org.netbeans.modules.mercurial.MercurialVCS.propertyChange(MercurialVCS.java:124)
	at java.beans.PropertyChangeSupport.firePropertyChange(PropertyChangeSupport.java:333)
	at java.beans.PropertyChangeSupport.firePropertyChange(PropertyChangeSupport.java:270)
	at org.netbeans.modules.mercurial.FileStatusCache.fireFileStatusChanged(FileStatusCache.java:866)
	at org.netbeans.modules.mercurial.FileStatusCache.getScannedFiles(FileStatusCache.java:462)
	at org.netbeans.modules.mercurial.FileStatusCache.refreshFileStatus(FileStatusCache.java:478)
[catch] at org.netbeans.modules.mercurial.FileStatusCache.refreshFileStatus(FileStatusCache.java:470)

Clearly the OOME is a secondary effect of the huge pile of file canonicalization warnings.

I am pretty sure this is all due to the Mercurial support module. In particular, HgCommand.getDirStatusWithFlags has
this suspicious code:

filePath = new StringBuffer(repository.getAbsolutePath()).append(File.separatorChar);
...
filePath.append(...truncated status line...)
...
...new File(filePath.toString())...

Now it looks to me like

C:\work\main\C:\work\main\ide.kit\build\test\qa-functional\work\userdir0\config\Modules

(which is indeed a bogus path that FileUtil.normalizeFile ought to complain about) is "C:\work\main" (repository) + "\"
(File.sC) + "C:\work\main\ide.kit\..." (some other stuff). This might happen if 'hg status' with some flags output
absolute paths instead of repository-relative paths. I don't know of any way in which that would happen, either in Hg
1.0.1 or 0.9.5 (Pete runs 0.9.5 currently), but it does not seem unlikely. Anyway, the code should probably be doing
something like this:

new File(repository, ...truncated status line...)

which is safer (either a relative or absolute path will be accepted as the second arg) and simpler.

I could also imagine this problem not arising on Unix; a bogus path such as

/home/pete/main//home/pete/main/ide.kit/...

will be normalized down to

/home/pete/main/ide.kit/...

without complaint.