This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 253549 - Processors that use lambdas are completely broken in the IDE
Summary: Processors that use lambdas are completely broken in the IDE
Status: RESOLVED FIXED
Alias: None
Product: platform
Classification: Unclassified
Component: JDK Problems (show other bugs)
Version: 8.1
Hardware: PC Windows 8
: P1 normal with 1 vote (vote)
Assignee: Tomas Zezula
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-07-17 16:23 UTC by misterm
Modified: 2015-08-01 02:01 UTC (History)
5 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
IDE log (409.52 KB, text/plain)
2015-07-17 16:23 UTC, misterm
Details
Processor (1.93 KB, text/plain)
2015-07-21 15:41 UTC, misterm
Details
Test Case (160.84 KB, application/octet-stream)
2015-07-22 11:28 UTC, Tomas Zezula
Details
nb-javac diff file (4.26 KB, patch)
2015-07-24 05:53 UTC, Tomas Zezula
Details | Diff
nb patch file (2.39 KB, patch)
2015-07-24 05:53 UTC, Tomas Zezula
Details | Diff
nb-javac diff file (6.87 KB, patch)
2015-07-24 12:38 UTC, Tomas Zezula
Details | Diff
nb-javac diff file (7.97 KB, patch)
2015-07-27 09:59 UTC, Tomas Zezula
Details | Diff
nb-javac diff file (8.30 KB, patch)
2015-07-27 11:21 UTC, Tomas Zezula
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description misterm 2015-07-17 16:23:13 UTC
If a project uses annotation processors that were written using lambdas, exceptions like this are thrown:

INFO [com.sun.tools.javac.processing.JavacProcessingEnvironment]: Annotation processing error:
java.lang.invoke.LambdaConversionException: Type mismatch in captured lambda parameter 0: expecting interface javax.annotation.processing.RoundEnvironment, found interface javax.annotation.processing.RoundEnvironment
        at java.lang.invoke.AbstractValidatingLambdaMetafactory.validateMetafactoryArgs(AbstractValidatingLambdaMetafactory.java:256)
        at java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:303)
        at java.lang.invoke.CallSite.makeSite(CallSite.java:302)
Caused: java.lang.BootstrapMethodError: call site initialization exception
        at java.lang.invoke.CallSite.makeSite(CallSite.java:341)
        at java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307)
        at java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297)
        at br.com.tecsinapse.processors.PossiblyBrokenStatelessProcessor.doProcess(PossiblyBrokenStatelessProcessor.java:26)
        at br.com.tecsinapse.processors.BaseProcessor.process(BaseProcessor.java:19)
[catch] at com.sun.tools.javac.processing.JavacProcessingEnvironment.callProcessor(JavacProcessingEnvironment.java:823)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment.access$400(JavacProcessingEnvironment.java:91)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment$DiscoveredProcessors$ProcessorStateIterator.runContributingProcs(JavacProcessingEnvironment.java:640)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.run(JavacProcessingEnvironment.java:1047)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessing(JavacProcessingEnvironment.java:1145)
        at com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler.java:1236)
        at com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler.java:1123)
        at com.sun.tools.javac.api.JavacTaskImpl.enter(JavacTaskImpl.java:375)
        at com.sun.tools.javac.api.JavacTaskImpl.enterTrees(JavacTaskImpl.java:418)
        at org.netbeans.modules.java.source.indexing.MultiPassCompileWorker.compile(MultiPassCompileWorker.java:211)
        at org.netbeans.modules.java.source.indexing.JavaCustomIndexer.index(JavaCustomIndexer.java:251)
        at org.netbeans.modules.parsing.spi.indexing.Indexable$MyAccessor$2.run(Indexable.java:161)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.runIndexer(RepositoryUpdater.java:296)
        at org.netbeans.modules.parsing.spi.indexing.Indexable$MyAccessor.index(Indexable.java:159)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work.doIndex(RepositoryUpdater.java:2745)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work.access$400(RepositoryUpdater.java:2149)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work$1.run(RepositoryUpdater.java:2631)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work$1.run(RepositoryUpdater.java:2629)
        at org.netbeans.modules.parsing.impl.indexing.errors.TaskCache.refreshTransaction(TaskCache.java:564)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work.index(RepositoryUpdater.java:2629)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work$3.call(RepositoryUpdater.java:3282)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work$3.call(RepositoryUpdater.java:3248)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$3.run(RepositoryUpdater.java:2122)
        at org.openide.util.lookup.Lookups.executeWith(Lookups.java:304)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.runInContext(RepositoryUpdater.java:2118)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.runInContext(RepositoryUpdater.java:2099)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.access$1100(RepositoryUpdater.java:157)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work.scanFiles(RepositoryUpdater.java:3248)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$FileListWork.getDone(RepositoryUpdater.java:3767)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work.doTheWork(RepositoryUpdater.java:3402)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Task._run(RepositoryUpdater.java:6166)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Task.access$4300(RepositoryUpdater.java:5813)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Task$3$1.run(RepositoryUpdater.java:6084)
        at org.openide.util.lookup.Lookups.executeWith(Lookups.java:304)
        at org.netbeans.modules.parsing.impl.RunWhenScanFinishedSupport.performScan(RunWhenScanFinishedSupport.java:106)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Task$3.call(RepositoryUpdater.java:6080)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Task$3.call(RepositoryUpdater.java:6076)
        at org.netbeans.modules.masterfs.filebasedfs.utils.FileChangedManager.priorityIO(FileChangedManager.java:176)
        at org.netbeans.modules.masterfs.providers.ProvidedExtensions.priorityIO(ProvidedExtensions.java:360)
        at org.netbeans.modules.parsing.nb.DataObjectEnvFactory.runPriorityIO(DataObjectEnvFactory.java:141)
        at org.netbeans.modules.parsing.impl.Utilities.runPriorityIO(Utilities.java:88)
        at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Task.run(RepositoryUpdater.java:6076)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1443)
        at org.netbeans.modules.openide.util.GlobalLookup.execute(GlobalLookup.java:68)
        at org.openide.util.lookup.Lookups.executeWith(Lookups.java:303)
        at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:2058)

The worst part is that if any of these processors generates sources, they are not added to the editor classpath, unfortunately.

Product Version = NetBeans IDE Dev (Build 20150716-7a93174cf75e)
Operating System = Windows 8 version 6.2 running on amd64
Java; VM; Vendor = 1.8.0_40
Runtime = Java HotSpot(TM) 64-Bit Server VM 25.40-b25
Comment 1 misterm 2015-07-17 16:23:19 UTC
Created attachment 154700 [details]
IDE log
Comment 2 Dusan Balek 2015-07-20 09:32:49 UTC
Unfortunately, I cannot reproduce the problem. Could you please create a simple test case and attach it to the issue? Thanks.
Comment 3 misterm 2015-07-21 15:41:57 UTC
Created attachment 154772 [details]
Processor

Simply add this processor to its own project. You will need to expand the imported static constants to "javax.ejb.Stateless" and "javax.inject.Inject".
Include the processor FQN in META-INF/services/javax.annotation.processing.Processor . 

Then create a new Java project with any Java files annotated with @Stateless and the processor jar in its classpath.
Comment 4 Tomas Zezula 2015-07-22 11:28:54 UTC
Created attachment 154794 [details]
Test Case

Test Netbeans Project demonstrating bug in SystemDictionary::find_method_handle_type(Symbol*,KlassHandle,Thread*) which does not use the caller class loader when the signature is materialized on boot class path. Such a resolution does not work with inverse class loader.
Comment 5 Tomas Zezula 2015-07-22 11:32:27 UTC
The SystemDictionary::find_method_handle_type(Symbol*,KlassHandle,Thread*) does not use the caller class loader when the signature is materialized on boot class path. Such a resolution does not work with inverse class loader. See: http://hg.openjdk.java.net/jdk8/jdk8/hotspot/file/87ee5ee27509/src/share/vm/classfile/systemDictionary.cpp#l2412
Comment 6 Tomas Zezula 2015-07-22 11:50:07 UTC
The test can be executed by:

export JAVA_HOME=<path_to_jdk8>; ant run

The test failure:
Exception in thread "main" java.lang.BootstrapMethodError: call site initialization exception
at java.lang.invoke.CallSite.makeSite(CallSite.java:341)
at java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307)
at java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297)
at indyclzlkp.Action.run(Action.java:15)
at indyclzlkp.Main.main(Main.java:22)
Caused by: java.lang.invoke.LambdaConversionException: Invalid receiver type class javax.lang.model.element.ElementKind; not a subtype of implementation type class javax.lang.model.element.ElementKind
at java.lang.invoke.AbstractValidatingLambdaMetafactory.validateMetafactoryArgs(AbstractValidatingLambdaMetafactory.java:233)
at java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:303)
at java.lang.invoke.CallSite.makeSite(CallSite.java:302)
Comment 7 Tomas Zezula 2015-07-24 05:53:11 UTC
Created attachment 154834 [details]
nb-javac diff file
Comment 8 Tomas Zezula 2015-07-24 05:53:36 UTC
Created attachment 154835 [details]
nb patch file
Comment 9 Tomas Zezula 2015-07-24 06:00:10 UTC
Added a patch for javac and nb which tries to replace using byte code patching calls to java.lang.invoke.LambdaMetafactory to custom one which corrects wrong MethodTypes creed by the SystemDictionary::find_method_handle_type(Symbol*,KlassHandle,Thread*).
Unfortunately the wrong klassOop referencing class on BCP is taken from "somewhere".

As seen from:
java.lang.invoke.WrongMethodTypeException: MethodHandle(RoundEnvironment)Supplier should be of type (RoundEnvironment)Supplier
	at java.lang.invoke.CallSite.wrongTargetType(CallSite.java:194)
	at java.lang.invoke.CallSite.makeSite(CallSite.java:335)
Caused: java.lang.BootstrapMethodError: call site initialization exception
	at java.lang.invoke.CallSite.makeSite(CallSite.java:341)
	at java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:307)
	at java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:297)
	at ap.Ap.process(Ap.java:24)
Comment 10 Tomas Zezula 2015-07-24 11:12:02 UTC
The patch problem is that CallSite.makeSite verifies compatibility of the MethodType type and created CallSite. The CallSite created by the patched LambdaMetaFactory has correct type but the expected type is wrong.
Possible solution may be not to create a new MethodType but to rewrite the sysDict created one by reflection. I will try.
Comment 11 Tomas Zezula 2015-07-24 12:38:59 UTC
Created attachment 154854 [details]
nb-javac diff file
Comment 12 Tomas Zezula 2015-07-27 09:59:33 UTC
Created attachment 154940 [details]
nb-javac diff file
Comment 13 Tomas Zezula 2015-07-27 10:00:06 UTC
Doing the byte code patching only on affected JDKs.
Comment 14 Tomas Zezula 2015-07-27 11:21:43 UTC
Created attachment 154944 [details]
nb-javac diff file

Sure it cannot work without sneaky throw.
Comment 15 Jan Lahoda 2015-07-27 11:32:42 UTC
Seems reasonable to me.
Comment 16 Tomas Zezula 2015-07-27 12:37:21 UTC
Fixed: nb-javac fcc3354424b2
Comment 17 Tomas Zezula 2015-07-27 12:48:12 UTC
Fixed jet-main db96a3afb14a.
JDK 8 < u51 are workarounding the hotspot problem by byte code patching.
JDK 8 >= u51 - the problem is already fixed in hotspot.
Comment 18 Quality Engineering 2015-07-28 01:20:11 UTC
Integrated into 'main-silver', will be available in build *201507280002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/db96a3afb14a
User: Tomas Zezula <tzezula@netbeans.org>
Log: #253549:Processors that use lambdas are completely broken in the IDE
Comment 19 Tomas Zezula 2015-07-28 09:30:53 UTC
I've fixed the problem but the test annotation processor was very simple.
Please, can someone verify the fix? If the verification comes today I will be able to integrate the fix to beta.
Thanks
Comment 20 Vladimir Riha 2015-07-28 12:47:56 UTC
I can only verify this problem is no longer reproducible, but I cannot say if it could broke something or not. This is not my area, I don't know much about it.


Product Version: NetBeans IDE Dev (Build Trunk-Build-1303-on-20150728)
Java: 1.8.0_45; Java HotSpot(TM) 64-Bit Server VM 25.45-b02
Runtime: Java(TM) SE Runtime Environment 1.8.0_45-b14
System: Linux version 3.16.0-30-generic running on amd64; UTF-8; en_US (nb)
Comment 21 Tomas Zezula 2015-07-28 17:34:06 UTC
Integrated into release81_beta d6594b2a7e68
Comment 22 misterm 2015-07-29 15:44:58 UTC
Still reproducible:

INFO [com.sun.tools.javac.processing.JavacProcessingEnvironment]: Annotation pro
cessing error:
java.lang.invoke.LambdaConversionException: Type mismatch in captured lambda par
ameter 0: expecting interface javax.annotation.processing.RoundEnvironment, foun
d interface javax.annotation.processing.RoundEnvironment
        at java.lang.invoke.AbstractValidatingLambdaMetafactory.validateMetafact
oryArgs(AbstractValidatingLambdaMetafactory.java:256)
        at java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java
:303)
        at sun.reflect.GeneratedMethodAccessor112.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at com.sun.tools.hc.LambdaMetafactory.metafactory(LambdaMetafactory.java
:81)
        at java.lang.invoke.CallSite.makeSite(CallSite.java:302)
Caused: java.lang.BootstrapMethodError: call site initialization exception
        at java.lang.invoke.CallSite.makeSite(CallSite.java:341)
        at java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNat
ives.java:307)
        at java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives
.java:297)
        at br.com.tecsinapse.processors.PossiblyBrokenStatelessProcessor.doProce
ss(PossiblyBrokenStatelessProcessor.java:38)
        at br.com.tecsinapse.processors.BaseProcessor.process(BaseProcessor.java
:19)
[catch] at com.sun.tools.javac.processing.JavacProcessingEnvironment.callProcess
or(JavacProcessingEnvironment.java:823)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment.access$400(
JavacProcessingEnvironment.java:91)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment$DiscoveredP
rocessors$ProcessorStateIterator.runContributingProcs(JavacProcessingEnvironment
.java:640)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment$Round.run(J
avacProcessingEnvironment.java:1047)
        at com.sun.tools.javac.processing.JavacProcessingEnvironment.doProcessin
g(JavacProcessingEnvironment.java:1145)
        at com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler
.java:1236)
        at com.sun.tools.javac.main.JavaCompiler.processAnnotations(JavaCompiler
.java:1123)
        at com.sun.tools.javac.api.JavacTaskImpl.enter(JavacTaskImpl.java:375)
        at com.sun.tools.javac.api.JavacTaskImpl.enter(JavacTaskImpl.java:310)
        at org.netbeans.modules.java.source.parsing.JavacParser.moveToPhase(Java
cParser.java:630)
        at org.netbeans.modules.java.source.parsing.JavacParser.getResult(JavacP
arser.java:496)
        at org.netbeans.modules.java.source.parsing.JavacParser.getResult(JavacP
arser.java:162)
        at org.netbeans.modules.parsing.impl.TaskProcessor.callGetResult(TaskPro
cessor.java:631)
        at org.netbeans.modules.parsing.impl.SourceCache.getResult(SourceCache.j
ava:262)
        at org.netbeans.modules.parsing.impl.TaskProcessor$RequestPerformer.run(
TaskProcessor.java:798)
        at org.openide.util.lookup.Lookups.executeWith(Lookups.java:304)
        at org.netbeans.modules.parsing.impl.TaskProcessor$RequestPerformer.exec
ute(TaskProcessor.java:725)
        at org.netbeans.modules.parsing.impl.TaskProcessor$CompilationJob.run(Ta
skProcessor.java:686)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:51
1)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1443
)
        at org.netbeans.modules.openide.util.GlobalLookup.execute(GlobalLookup.j
ava:68)
        at org.openide.util.lookup.Lookups.executeWith(Lookups.java:303)
        at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java
:2058)

I use main-silver, but:

C:\projetos\main-silver>hg log --rev "ancestors(.) and db96a3afb14a"
revisÒo:      289616:db96a3afb14a
pai:          289477:0ca817cb6fe1
usußrio:      Tomas Zezula <tzezula@netbeans.org>
data:         Mon Jul 27 14:46:26 2015 +0200
sumßrio:      #253549:Processors that use lambdas are completely broken in the IDE

My checkout contains your change :-S
Comment 23 Tomas Zezula 2015-07-29 15:58:03 UTC
The processor is the one from the attachement (https://netbeans.org/bugzilla/attachment.cgi?id=154772)?
If so I will add the logging options.
Also can you try JDK 1.8.0 u51?
Thanks
Comment 24 misterm 2015-07-29 16:04:25 UTC
Yes, it is that processor.

About trying JDK 8u51, I'm not so sure. I'm about to go on vacations and I need to get my work finished before and installing a new JDK might mess up something... I'll try, but can't promise though, unfortunately.
Comment 25 Tomas Zezula 2015-07-29 16:34:35 UTC
OK, no problem I will add logging and let you know.
Comment 26 Tomas Zezula 2015-07-30 13:00:32 UTC
It's not enough to patch invokedType but the implMethod MethodHandle needs to be patched as well. The funny past is that when the implMethod is broken the invokeType is fine. So anything can be broken in invokedynamic + lambda on JDK 8 < u51 with inversed CL, the only certain thing is that never all params are valid. :-)
Comment 28 Tomas Zezula 2015-07-30 14:25:05 UTC
Fixed jet-main: http://hg.netbeans.org/jet-main/rev/46de5752faae
Comment 29 Quality Engineering 2015-07-31 01:51:46 UTC
Integrated into 'main-silver', will be available in build *201507310002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/46de5752faae
User: Tomas Zezula <tzezula@netbeans.org>
Log: #253549:Processors that use lambdas are completely broken in the IDE
Comment 30 Quality Engineering 2015-08-01 02:01:43 UTC
Integrated into 'main-silver', will be available in build *201508010002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/78ac0e59785a
User: Tomas Zezula <tzezula@netbeans.org>
Log: #253549:Processors that use lambdas are completely broken in the IDE