This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 246438 - Wrong error badges
Summary: Wrong error badges
Status: RESOLVED FIXED
Alias: None
Product: platform
Classification: Unclassified
Component: JDK Problems (show other bugs)
Version: 8.0
Hardware: PC Linux
: P1 normal with 1 vote (vote)
Assignee: Tomas Zezula
URL:
Keywords:
Depends on: 246374
Blocks: 246439
  Show dependency tree
 
Reported: 2014-08-13 13:49 UTC by ulfzibis
Modified: 2015-05-03 17:28 UTC (History)
5 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Patch (954 bytes, patch)
2014-08-20 15:18 UTC, Tomas Zezula
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description ulfzibis 2014-08-13 13:49:05 UTC
[ JDK VERSION : 1.7.0_67 ]

STEPS:
   1) Steps from bug 246374.
   2) In project B change reference to test/ of project A.

ACTUAL:
   1) No error badge, even file compiles with errors.
   2) Error badge appears correctly.

EXPECTED:
   Error badges should always correctly indicate, if file is compileable.
Comment 1 ulfzibis 2014-08-13 13:58:56 UTC
Correction: [ JDK VERSION : 1.7.0_55 ]
Comment 2 Tomas Zezula 2014-08-20 11:02:12 UTC
There are 2 cache folders for the same root which is wrong.
Probably some problem in SourceForBinaryQuery translation.
Comment 3 Tomas Zezula 2014-08-20 11:59:40 UTC
The problem is with encoding of the U-umlaut in the KomPruefStand
Once it's encoded as single char (252) and second time it's encoded as 2 characters (117,776). The URIs do not equal.
The File.toURI().toURL().toString().charAt(42) returns 252.
I am not sure it it's generic problem or if it's Mac OS specific.
Evaluation how the U-umlaut -> (117,776) is created.
Comment 4 Tomas Zezula 2014-08-20 12:58:19 UTC
Now comes the funny part:
final File f = new File("KomPrüfStand");
f.toURI().toURL().toString().charAt(42) --> 252
f.toPath().toUri().toURL().toString().charAt(42) --> 117

The same file can produce both versions of URLs (URIs).
Comment 5 Tomas Zezula 2014-08-20 13:19:54 UTC
There is an inconsistency in the URLs returned by FileUtil.urlForArchiveOrDir() and
FileObject.getURL().
FileUtil.urlForArchiveOrDir() as all other using Utilities.toURI() are returning URLs with 117 while FileObject.getURL() returns URL with 252.
Such URLs (URIs) are not equal.
I can solve this case in cache folder by translating the 252 to (117,776) however the same problem exists on other places.
Comment 6 Tomas Zezula 2014-08-20 14:50:34 UTC
I narrowed the problem a bit.
The correct encoding of 'ü' is 252 as it's produced by FileObjects or by normalised File.
The wrong encoding of ü (117,776) is created by Utilities.toURI(File f) which returns the URL with (117,776). Unfortunately it's produced by Path.toUri().
I can workaround the problem by passing a subclass of File which throws InvalidPathException from toPath() and the Utilities.toURI(File f) falls back to File.toURI() which works fine.
However it seems that using NIO Path API is quite dangerous as it brings non normalised  paths into IDE and should be disabled before it's fixed.
Comment 7 Tomas Zezula 2014-08-20 15:18:52 UTC
Created attachment 148795 [details]
Patch

Patch which disables use of NIO Path
Comment 8 Tomas Zezula 2014-08-20 15:50:21 UTC
I've tested it on old Windows XP with JDK 1.7 and it seems that Windows XP with JDK 1.7 has no this problem.

On Ubuntu with JDK 8 the same problem as on Mac, seems a problem in UNIX implementation of NIO FileSystem.

Here is the simple test application demonstrating the problem:

import java.io.File;
import java.io.IOException;
import java.net.MalformedURLException;
public class Test {
    public static void main(String[] args) throws MalformedURLException, IOException {
        final File f = new File ("Frühling").getCanonicalFile();   //NOI18N
        f.mkdir();
        final String io = f.toURI().toURL().toString();
        final String nio = f.toPath().toUri().toURL().toString();
        final boolean res = io.equals(nio);
        System.out.println(res);
        System.out.println(io);
        System.out.println(nio);
    }
}
Comment 9 Tomas Zezula 2014-08-20 15:51:31 UTC
Reassign as Utilities is a part of utilities API not FS API.
Comment 10 Jaroslav Tulach 2014-08-21 13:01:03 UTC
I would say it is a JDK bug and should be reported to their bug tracking system. I don't see it as urgent to require a workaround on the NetBeans side.
Comment 11 Tomas Zezula 2014-08-21 13:13:05 UTC
Workaround is to use ue rather than u-umlaut in folder name.
Comment 12 ulfzibis 2014-09-09 14:12:43 UTC
(In reply to Jaroslav Tulach from comment #10)
> I would say it is a JDK bug and should be reported to their bug tracking
> system. I don't see it as urgent to require a workaround on the NetBeans
> side.

Did you file a bug against JDK already? Which is the bug ID?
Comment 13 Antonin Nebuzelsky 2015-02-09 20:16:59 UTC
Tomasi, thanks for the detailed evaluation!

JDK bug:
https://bugs.openjdk.java.net/browse/JDK-8072808
Comment 14 Tomas Zezula 2015-03-04 10:35:57 UTC
Sorry Tondo,
But we cannot ignore this. The JDK issue is P4 and even if fixed it will take a time unless it gets to customers.
This issue causes several other IDE problems like wrong error badges, editor features does not work, project system does not work or even OOM errors as it disables caches.
It is a cause of lots of such a reports, some of them are easy identified as the reported states that he has non ISO-Latin char in path (https://netbeans.org/bugzilla/show_bug.cgi?id=250167) but others are hard or impossible to find and duplicate.
Also by ignoring this we say that we support just ISO-Latin chars.

The importance of canonical URI (URL) is discussed in issue: https://netbeans.org/bugzilla/show_bug.cgi?id=214131.

The problem is that we have 3 ways how to get URIs (producing unequals URIs).

1st) Path.toURI() used by BaseUtilities.toURI()
2nd) File.toURI() used by BaseUtilities.toURI() and other places.
3rd) MasterFS FileBaseURLMapper custom URI creation from File.getAbsolutePath() - performance optimalisation. Seems that it's consistent with File.toURI().

So the problem is inconsistence among 1) and 2).
Comment 15 Tomas Zezula 2015-03-04 10:41:25 UTC
I suggest to disable the Path.toUri() in BaseUtilities unless it's fixed.
The disabling can be done dynamically depending on URI equality, so when the JDK bug is fixed, it will be reenabled. Seems to works fine on UNIX, I cannot test on Windows.
It's platform specific (JNI code in UnixNativeDispatcher.c and UnixFileSystem_md.c).
Anyway nothing is worse than current state when we get "random" issues.
Comment 16 Tomas Zezula 2015-03-04 10:49:30 UTC
Fixed jet-main 0901f6d2f827
Comment 17 Alexander Simon 2015-03-05 10:32:29 UTC
Wrong fix, please redo it.
FileUtil.normalizeFile throw exception.
For example:
Malformed input or input contains unmappable chacraters: k??
java.nio.file.InvalidPathException: Malformed input or input contains unmappable chacraters: k??
	at sun.nio.fs.UnixPath.encode(UnixPath.java:147)
	at sun.nio.fs.UnixPath.<init>(UnixPath.java:71)
	at sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:281)
	at java.io.File.toPath(File.java:2186)
	at org.openide.util.BaseUtilities.pathToURISupported(BaseUtilities.java:1709)
	at org.openide.util.BaseUtilities.toURI(BaseUtilities.java:1636)
	at org.openide.filesystems.FileUtil.normalizeFileOnUnixAlike(FileUtil.java:1679)
	at org.openide.filesystems.FileUtil.normalizeFileImpl(FileUtil.java:1666)
	at org.openide.filesystems.FileUtil.normalizeFileCached(FileUtil.java:1643)
	at org.openide.filesystems.FileUtil.normalizeFile(FileUtil.java:1625)
	at org.openide.filesystems.FileUtil.toFileObject(FileUtil.java:868)
	at org.openide.filesystems.FileUtil.getDiskFileSystemFor(FileUtil.java:166)
	at org.openide.filesystems.FileUtil.refreshFor(FileUtil.java:186)
Comment 18 Tomas Zezula 2015-03-05 14:14:06 UTC
Which OS?
Comment 19 Tomas Zezula 2015-03-05 14:21:01 UTC
Seems that the File cannot be mapped into Path.
In this case I can only fallback to disable Path.
What OS you are running on?
Comment 20 Alexander Simon 2015-03-05 14:54:08 UTC
(In reply to Tomas Zezula from comment #19)
> Seems that the File cannot be mapped into Path.
> In this case I can only fallback to disable Path.
> What OS you are running on?

#uname -a
SunOS beta 5.11 11.2 i86pc i386 i86pc
Comment 21 Alexander Simon 2015-03-05 17:08:57 UTC
Tomas, for me it is a P0, because I cannot start IDE.
See log:
tryme:
     [exec] Exception in thread "main" java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: k??
     [exec] 	at sun.nio.fs.UnixPath.encode(UnixPath.java:147)
     [exec] 	at sun.nio.fs.UnixPath.<init>(UnixPath.java:71)
     [exec] 	at sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:281)
     [exec] 	at java.io.File.toPath(File.java:2234)
     [exec] 	at org.openide.util.BaseUtilities.pathToURISupported(BaseUtilities.java:1709)
     [exec] 	at org.openide.util.BaseUtilities.toURI(BaseUtilities.java:1636)
     [exec] 	at org.netbeans.JarClassLoader$JarSource.toURI(JarClassLoader.java:504)
     [exec] 	at org.netbeans.JarClassLoader$JarSource.<init>(JarClassLoader.java:475)
     [exec] 	at org.netbeans.JarClassLoader$Source.create(JarClassLoader.java:424)
     [exec] 	at org.netbeans.JarClassLoader.<init>(JarClassLoader.java:169)
     [exec] 	at org.netbeans.JarClassLoader.<init>(JarClassLoader.java:153)
     [exec] 	at org.netbeans.MainImpl$BootClassLoader.<init>(MainImpl.java:272)
     [exec] 	at org.netbeans.MainImpl.execute(MainImpl.java:178)
     [exec] 	at org.netbeans.MainImpl.main(MainImpl.java:85)
     [exec] 	at org.netbeans.Main.main(Main.java:83)
Comment 22 Tomas Zezula 2015-03-05 17:14:48 UTC
I will push fix in several minutes.
Comment 23 Tomas Zezula 2015-03-05 18:37:01 UTC
Fixed jet-main 86f9a1f7209f
Comment 24 Tomas Zezula 2015-03-05 20:17:36 UTC
The exception was caused by wrong encoding causing that the 'special' chars like umlaut cannot be encoded. On such systems it's better to disable Path as Files like 'küñ' becomes 'k??' (on disk the file name is correct).
It helps to set correct encoding but to make the situation more complicated the behaviour is different on JDK 7 and JDK 8. On JDK 7 the Charset.defaultCharset is used (file.encoding) on JDK 8 the Util.jnuEncoding is used (sun.jnu.encoding). So it's good to set both if Path should works correctly.

I was able to reproduce the problem on Ubuntu Server:
Linux 3.13.0-43-generic #72-Ubuntu SMP x86_64 x86_64 x86_64 GNU/Linux

where the encodings were:

file.encoding=US-ASCII
sun.jnu.encoding=ANSI_X3.4-1968
Comment 25 Quality Engineering 2015-03-06 04:11:41 UTC
Integrated into 'main-silver', will be available in build *201503060001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/0901f6d2f827
User: Tomas Zezula <tzezula@netbeans.org>
Log: #246438:Wrong error badges
Comment 26 Quality Engineering 2015-03-07 08:41:18 UTC
Integrated into 'main-silver', will be available in build *201503070001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/86f9a1f7209f
User: Tomas Zezula <tzezula@netbeans.org>
Log: #246438:Wrong error badges
Comment 27 ulfzibis 2015-04-17 13:56:46 UTC
Hi,

is it possible to have this as a hotfix for NB 8.0.2 ?
Comment 28 Tomas Zezula 2015-04-17 14:29:49 UTC
Marian, Can we include this into NB 8.0.2?
Thanks
Comment 29 ulfzibis 2015-04-17 16:46:52 UTC
Thanks, it also affects bug 246374.