This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
| Summary: | Ecmascript 6 (harmony) syntax support | ||
|---|---|---|---|
| Product: | javascript | Reporter: | garaboncias |
| Component: | Editor | Assignee: | Petr Pisl <ppisl> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | Afterster, aladio, anebuzelsky, chrizzly, dab1818, dhaowoods, dusty, dylanv, FMA_, hmichel, jkovalsky, JVerstrynge, karex, luislobo, Marcel_K, mediacept, mehiel, mossroy, M_C_02, NukemBy, pangalz, pgbakker, phejl, ppisl, raliclo, ricktw, scamo, terje7601, viggonavarsete, vriha, zimmi, Zoff |
| Priority: | P1 | ||
| Version: | 8.1 | ||
| Hardware: | All | ||
| OS: | All | ||
| Issue Type: | ENHANCEMENT | Exception Reporter: | |
| Bug Depends on: | 258657, 258682, 258683, 257203, 257204, 257338, 257341, 257342, 257390, 257451, 257589, 258600, 258603, 258643, 258646, 258647, 258654, 258656, 258658, 258659, 258664, 258678, 258679, 258680, 258681, 258830, 258857, 258858, 262490, 262491 | ||
| Bug Blocks: | |||
| Attachments: |
generator-gulp-angular
gulp-angular project malarkey spec file index module file Result of performance test from NukemBy Template String Problem Warning arrow function |
||
|
Description
garaboncias
2014-02-26 23:30:21 UTC
This is enhancment for the next version. any news for this? Unfortunately, there is no plan yet. This depends on the new parser and lexer. I'm going to look at this. *** Bug 245527 has been marked as a duplicate of this bug. *** New syntax is extensively used by ember-cli, and almost every js file in ember-cli projects being marked by NetBeans with red exclamation signs. It will be nice to have ability to just turn off syntax checking for JS files. Can't find this option in menus. Any news on this? With all that traceur support it seems like more than "enhancement". +1 +1 +1 +1 +1 ECMA 6 enters mainstream. Netbeans support is a must or developers will be forced to change their IDE :( +1 We have started using ES6 at work and for a while we might manage to use Netbeans, but without any auto completion and code formatting it will soon be to hard to work in Netbeans and we will be forced to switch IDE. Same regarding Babel and Typescript, Support really needed or else we will be forced to develop our own solution to keep working with Netbeans. +1 +1 According to WiKi http://wiki.netbeans.org/BugPriorityGuidelines this one should become P2 (more than 10 duplicates + votes). seems like it attacks P1 with 20 votes :) Thanks for the reminder. +1 *** Bug 251748 has been marked as a duplicate of this bug. *** +1. Would love to see this implemented. Or at least a way to disable syntax checking in JS files, so I can use new keywords without errors. +1 +1 ES6 specifications has been finalized and now accepted as standard: http://www.ecma-international.org/news/index.html +1 +1 Trying to switch to Idea Ultimate (with ES6, TypeScript, Python and what-else) is SUCH A PAIN... :/ On the other hand it is an investment, one editor for every possible language would be very convenient... If NetBeans had a paid version with many languages, it would be nice, but I guess there is no room on the market for it :( Two days later, in pain and despair (I've been using Netbeans exclusively for 11 years now) I can confirm, it's (the switch) doable. Idea is full of quirks, it's like 10x more complicated, for no god reason, I guess. Still, it seems to be the most NetBeans-like IDE so far for me (Eclipse is totally different). If you have to use ES6 now, Idea Ultimate can be a good harbor until NetBeans brings it in. +1 Very interesting trolling going on here. Not interested in changing my IDE though. For the same reason I prefer Firefox over Chrome. Is there any advancements in this? I would gladly test a daily build with at least limited ES6 support denbo +1 :) @denbo
> Not interested in changing my IDE though.
Yeah, especially when you have to code/support new stuff in ES6. At least my solution actually WORKS and it's as NetBeans-like as one can possibly find (they have even "NetBeans" keymap as an option, built-in, if someone prefers).
Would love to hear better solution (yes, I know, I can write ES6 plugin on my own, of course). NetBeans daily, last week I checked, had nothing new about ES6 and there is not even a mention about ES6 in NewAndNoteworthyNB81 page.
Go ahead and call me a troll if you like, I don't really care.
Agreeing with @witoldsz. Actively using ES6 and switched to Webstorm several months ago. It's a well-polished IDE, but I much prefer NetBeans and will probably switch back, once ES6 and node.js debugging are available. A good javascript ide is https://c9.io/ , but it's JS only +1 from me. I'll have to move to Webstorm (for now at least) Same to me, still waiting for either typescript or ES6 to be supported by netbeans. For now Atom or Webstorm are the alternatives I found, even if they do not substitute NB. Hi all, as a temporary workaround, does this new feature in NetBeans IDE 8.1 help in any way: https://blogs.oracle.com/geertjan/entry/future_proof_netbeans_ide_filter I.e., at least you will not see red error marks everywhere in your Ecmascript 6 code. +1 the bigger project, the harder to navigate without "Navigate" window and typehinting Node4 is out and by default it uses ES6. Is there a timeline for the support in NB? A small start has been made on ECMAScript 6 support. All those interested in this issue are invited to join the project and contribute, via suggesting features, trying out the current status, and providing code: https://github.com/GeertjanWielenga/TypeScript https://blogs.oracle.com/geertjan/entry/ecmascript_typescript_and_netbeans_ide Please report this bug as needed for the upcoming 8.1 release: https://netbeans.org/community/netcat/ca_survey_81.html since node v4 uses ES6 by default there is no point to claim to support it without supporting the syntax Paweł, please don't change milestones and versions arbitrarily as this will certainly not bring the feature to the version you set unless you already have it implemented locally and want to contribute today. Thanks for your understanding and cooperation. Oh boy, I was so happy that netbeans made this decision... too bad There is a temporary workaround. Not fantastic, but it works. I use grunt watch with eslint for grunt. It checks for errors as I type my code. I am using the 8.1 beta feature disabling code parsing within Netbeans. I fully understand that the support of ECMAScript 6 is needed. I'm working on the ECMAScript 6 support now at hg.netbeans.org/web-main branch ecma6_dev. Currently I work on the parser, syntax coloring is done. Soon I will need volunteers, who will test it. So speak up, if you are interested in. This is a way how to reach good quality of the support in reasonable time. I'm in, let me now when there is something to test :-D I would like to do manually testing too :). I can try :) I'm currently working on a es6 project and should be able to start testing right away. Just some simple instructions on how to set it up would be needed. Regards. I'm going to write up instruction here. So far is done just the coloring lexer. As I wrote, I'm working on the parser, which is the based stone for almost every feature. I need probably 2 or 3 weeks to get it in the reasonable and usable shape. I will put updates here too. I would like to test. This is my priority now. I'm work on the EcmaScript 6 parser and something is already done. Work is in progress and I hope that there will be soon something for testing. Thanks for your patience. Everlaw just made an annoucement about their new plugin for TypeScript support. After a very brief tests it seems quite complete feature-wise and could be used for real work already. See http://blog.everlaw.com/2015/10/06/open-source-typescript-netbeans/#comment-5284 and https://github.com/Everlaw/nbts. Also it seems that https://github.com/GeertjanWielenga/TypeScript will be deleted in order to join effort with Everlaw. @Petr Pisl: any news on the plugin? Even a basic one would be better than nothing. Thanks! I'm working on this every day now. Currently the lexer is done, parser almost and I'm integrating the parser in the editor. The development is done in web-main repository on branch ecma6_dev. It can be built from sources, but the current state is far from production quality. Anyway if someone really want it to try it in this state, I will write up steps, how you can do it. Just speak up :). Thanks I can't promise that I'll have time and will be capable to build The Netbeans O_o But I can give it a try on some of my node projects, at least next week we are finishing deployment of our node.js project and will have some time to play with ES6. But don't count on me much - now we are working 24/7 fixing bugs sometimes all nights long :( I can help with testing as I am learning angular2 using typescript and I'll be using JAX-RS as backend. So I can use netbeans with ecmascript 6 support. But I won't be able to build netbeans myself. @Petr Pisl: can you post instructions? I'll definitely help you on testing. I work everyday with NetBeans and I'm starting ES6 code right now. I have asked for the public build for my branch and here is: http://deadlock.netbeans.org/job/ECMAScript6/ The first build takes something between 5 - 8 hours. On the build page should be then a link to the last build, where you can download a zip file. This zip file are NetBeans that can be used on every platform. Just download the zip file on your computer, unzip to a location. It can be started from terminal window like: /yourlocation/NetBeans/bin/netbeans.sh --userdir /path/to/your/new/userdir I strongly suggest to use the --userdir option, to work with new userdir. Remember that these builds are under development and many things in javascript are broken and probably is not usable for real development. The --userdir option helps separate your "real" development build with these testing builds and you will not mix setting from this untested builds with your production environment. Let me know, if it's ok for you. Thanks Build failed. My fault. I set up wrong JDK :(. Now the build is build, but as you can see there is more than 1 000 tests are failing. Anyway the new parser is here. You can play with it. get this exception On class file java.lang.IllegalArgumentException: Token id must not be null. Fix lexer org.netbeans.modules.javascript2.editor.lexer.JsLexer@80b484d All classes that extends other classes is not really working. @jockeeriksson thanks for trying it. I have entered new issue #257203. I suggest to create for every case/issue/bug/enhancement (like this) new issue in bugzilla to track it. If we will write all in this issue, is will be mess. This issue should depend on the new one, so we will know that someone enter issue that is connected with the ES6 support rewrite and basically is blocking this one. I know, it can look like unnecessary, but let's keep it clear. Also I use for these issues whiteboard ecma6. (In reply to jockeeriksson from comment #61) > All classes that extends other classes is not really working. I have to admit that it's not implemented yet. Generally in such case like this, you can fill new enhancement with simple example and what you expect that NB should do. I will implement as soon as possible. It will speed up the development and help me concentrate on the user important things. OK will do. Ok need a new version with the token fix to be able to continue testing, all other problems I can live with. And thanks for all your hard work on this feature. Ok tested the latest and it gets better and better I think everybody who has stated that they are willing to help testing should jump in now. A problem I see is that Options -> HTML/JS -> Node.js still tries to download sources from version 0.12.7 while it should download v5.3.0 (the one I'm currently using) Status: The problem with downloading Node.js version should be fixed from the build #11 at http://deadlock.netbeans.org/job/ECMAScript6/. NB downloaded the latest ECMAScript 5 version of Node.js sources to be able parse them. Now when there is some support of ECMAScript 6 NB downloads the correct version of sources and there is not a limitation. Currently I don't add a new ECMAScript 6 features, but I'm trying get the editor in the same shape before replacing the parser. There are still more then 850 failing tests from 2 500 tests. Formatting, json support, some frameworks support are main areas that do not work correctly now. If you have a problem with parsing ES 6 code, please report it. I usually try to fix the parser issues immediately. The same applay if you run in an issue that it makes the editor unusable. I'm sorry, all the fixes that I have mentioned in the comment above will be available in the build #12. Thanks Petr for the work in progress!
I've tried build 11 but I've several problems, the biggest one is that NB does not ever finishes the "Background scanning of projects" and keeps my six cores at 100% indefinitely (ok, after ten minutes I killed the java processes) and hence renders the IDE unusable.
The other problem is code reformatting, a simple source like this:
====
import Ember from 'ember';
export default Ember.Component.extend({
name: 'Alex',
cm: Ember.inject.service('cm'),
});
====
gets scrambled: the "export" line (or whatever there is after the import line) gets indented wrongly
@dusty: I have found one cycle in parsing. The fix will be in the build #14. Could you try it? I would like to know, whether your scanning is finished. If not could you try to switch off the RequireJs support. Regarding the formatting, you are right. The formatter is broken now. You could install the JSBeuatify plugin as a temporary solution. I'm testing build 15 and it works much better, thanks Petr! But I'm continuously getting this error: java.util.ConcurrentModificationException at java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:711) at java.util.LinkedHashMap$LinkedValueIterator.next(LinkedHashMap.java:739) at org.netbeans.modules.javascript2.editor.model.impl.ModelVisitor6.createVarLetConst(ModelVisitor6.java:880) at org.netbeans.modules.javascript2.editor.model.impl.ModelVisitor6.visitLexicalBinding(ModelVisitor6.java:431) at org.netbeans.modules.javascript2.editor.model.impl.ModelVisitor6.visitLexicalBinding(ModelVisitor6.java:73) at org.netbeans.modules.javascript2.editor.parser6.ECMAScript6Parser$LexicalBindingContext.accept(ECMAScript6Parser.java:4753) at org.netbeans.modules.javascript2.editor.model.impl.ModelVisitor6.visitLexicalDeclaration(ModelVisitor6.java:442) at org.netbeans.modules.javascript2.editor.model.impl.ModelVisitor6.visitLexicalDeclaration(ModelVisitor6.java:73) at org.netbeans.modules.javascript2.editor.parser6.ECMAScript6Parser$LexicalDeclarationContext.accept(ECMAScript6Parser.java:4580) at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:70) at org.netbeans.modules.javascript2.editor.parser6.ECMAScript6BaseVisitor.visitDeclaration(ECMAScript6BaseVisitor.java:401) at org.netbeans.modules.javascript2.editor.parser6.ECMAScript6Parser$DeclarationContext.accept(ECMAScript6Parser.java:4265) at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:70) at org.netbeans.modules.javascript2.editor.parser6.ECMAScript6BaseVisitor.visitStatementList(ECMAScript6BaseVisitor.java:429) at org.netbeans.modules.javascript2.editor.parser6.ECMAScript6Parser$StatementListContext.accept(ECMAScript6Parser.java:4500) at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:70) at org.netbeans.modules.javascript2.editor.parser6.ECMAScript6BaseVisitor.visitProgramItem(ECMAScript6BaseVisitor.java:947) at org.netbeans.modules.javascript2.editor.parser6.ECMAScript6Parser$ProgramItemContext.accept(ECMAScript6Parser.java:9261) at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:70) at org.netbeans.modules.javascript2.editor.parser6.ECMAScript6BaseVisitor.visitProgram(ECMAScript6BaseVisitor.java:940) at org.netbeans.modules.javascript2.editor.parser6.ECMAScript6Parser$ProgramContext.accept(ECMAScript6Parser.java:9196) at org.netbeans.modules.javascript2.editor.model.Model.getModelVisitor(Model.java:182) at org.netbeans.modules.javascript2.editor.model.Model.getGlobalObject(Model.java:506) at org.netbeans.modules.javascript2.editor.index.JsIndexer.index(JsIndexer.java:105) at org.netbeans.modules.parsing.spi.indexing.Indexable$MyAccessor$3.run(Indexable.java:248) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.runIndexer(RepositoryUpdater.java:296) at org.netbeans.modules.parsing.spi.indexing.Indexable$MyAccessor.index(Indexable.java:246) [catch] at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work$2.run(RepositoryUpdater.java:3210) at org.netbeans.modules.parsing.impl.TaskProcessor.callUserTask(TaskProcessor.java:609) at org.netbeans.modules.parsing.api.ParserManager$UserTaskAction.run(ParserManager.java:153) at org.netbeans.modules.parsing.api.ParserManager$UserTaskAction.run(ParserManager.java:137) at org.netbeans.modules.parsing.impl.TaskProcessor$2.call(TaskProcessor.java:204) at org.netbeans.modules.parsing.impl.TaskProcessor$2.call(TaskProcessor.java:201) at org.netbeans.modules.masterfs.filebasedfs.utils.FileChangedManager.priorityIO(FileChangedManager.java:176) at org.netbeans.modules.masterfs.providers.ProvidedExtensions.priorityIO(ProvidedExtensions.java:360) at org.netbeans.modules.parsing.nb.DataObjectEnvFactory.runPriorityIO(DataObjectEnvFactory.java:141) at org.netbeans.modules.parsing.impl.Utilities.runPriorityIO(Utilities.java:88) at org.netbeans.modules.parsing.impl.TaskProcessor.runUserTask(TaskProcessor.java:201) at org.netbeans.modules.parsing.api.ParserManager.parse(ParserManager.java:104) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work.indexEmbedding(RepositoryUpdater.java:3144) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work.doIndex(RepositoryUpdater.java:2865) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work.access$500(RepositoryUpdater.java:2157) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work$1.run(RepositoryUpdater.java:2639) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work$1.run(RepositoryUpdater.java:2637) at org.netbeans.modules.parsing.impl.indexing.errors.TaskCache.refreshTransaction(TaskCache.java:565) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work.index(RepositoryUpdater.java:2637) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$AbstractRootsWork$4.call(RepositoryUpdater.java:5705) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$AbstractRootsWork$4.call(RepositoryUpdater.java:5613) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$4.run(RepositoryUpdater.java:2130) at org.openide.util.lookup.Lookups.executeWith(Lookups.java:304) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.runInContext(RepositoryUpdater.java:2126) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.runInContext(RepositoryUpdater.java:2107) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater.access$1200(RepositoryUpdater.java:157) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$AbstractRootsWork.scanSource(RepositoryUpdater.java:5740) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$AbstractRootsWork.scanSources(RepositoryUpdater.java:5410) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$RootsWork.getDone(RepositoryUpdater.java:5028) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Work.doTheWork(RepositoryUpdater.java:3410) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Task._run(RepositoryUpdater.java:6174) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Task.access$4300(RepositoryUpdater.java:5825) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Task$2$1.run(RepositoryUpdater.java:6090) at org.openide.util.lookup.Lookups.executeWith(Lookups.java:304) at org.netbeans.modules.parsing.impl.RunWhenScanFinishedSupport.performScan(RunWhenScanFinishedSupport.java:106) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Task$2.call(RepositoryUpdater.java:6086) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Task$2.call(RepositoryUpdater.java:6082) at org.netbeans.modules.masterfs.filebasedfs.utils.FileChangedManager.priorityIO(FileChangedManager.java:176) at org.netbeans.modules.masterfs.providers.ProvidedExtensions.priorityIO(ProvidedExtensions.java:360) at org.netbeans.modules.parsing.nb.DataObjectEnvFactory.runPriorityIO(DataObjectEnvFactory.java:141) at org.netbeans.modules.parsing.impl.Utilities.runPriorityIO(Utilities.java:88) at org.netbeans.modules.parsing.impl.indexing.RepositoryUpdater$Task.run(RepositoryUpdater.java:6082) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1443) at org.netbeans.modules.openide.util.GlobalLookup.execute(GlobalLookup.java:68) at org.openide.util.lookup.Lookups.executeWith(Lookups.java:303) at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:2058) Another problem is that in the bower project properties page I get the error "Unable to list the installed Bower packages" I'm using node v5.4.0 installed via nvm The java.util.ConcurrentModificationException should be fixed in the build #16. That exception did go away, but the background scanning of the projects still goes in busy loop and I've to kill -9 the IDE Using nodejs 5.4.0 via nvm I forgot to mention that I used build 17 Dusty could this be your problem? I run in to this issue and I could not open my project because I have a very large node_modules folder. https://netbeans.org/bugzilla/show_bug.cgi?id=238709 No, the problem is in the parsing code because when all the cpus are at 100%, there is no I/O activity and netbeans stops being responsive. After a while I've to kill -9 it. Dusty is right. I have find a big performance issue in the parser. I'm already working on this. It will take some time. Would like to test the ES6 support to, I can use a nightly build for this or do I have to build it by my own? @ChrisLE : There is a continual build at http://deadlock.netbeans.org/job/ECMAScript6/ Use the yellow ones is ok. But as I mentioned. There is a performance problem now, which I'm solving. It's usable for project that contains small amount of js files. Unfortunately parsing big bunch of files (a libraries) will take a lot of time. I tested build 22 and I can confirm it still has the 100% cpu loop problem even without downloading the node sources I used the yo generator-gulp-angular and said I want use ES6 with babel. When the project opens, there is an error hint in the project window for the file malarkey.directive.spec.js. See my screenshot, to see what happens in there. I don't know why. I did change anything. Created attachment 158106 [details]
generator-gulp-angular
And here: http://pastebin.com/YjTKVrPM. First, the variables like config, routerConfig and runBlock shows warnings that the global variable is not declared. Second, when I ctrl + click on config, nothing happens, when I ctrl + click on routerConfig or runBlock, every is fine and NB navigates to the file. You can test this with a HTML5 project when you install the generator-gulp-angular and than yo gulp-angular and ES6 Babel support. (In reply to ChrisLE from comment #85) > Created attachment 158106 [details] > generator-gulp-angular Could you just add the file here? Created attachment 158107 [details]
gulp-angular project
Created attachment 158108 [details]
malarkey spec file
The spec file with the errors inside.
Created attachment 158109 [details]
index module file
The file the global variable warning and the problem with the config navigate to.
I added the 2 files with the problems and the complete project without node_modules and bower_components. I used a build using fe92d26a9bb5 (ecma6_dev) with jdk8.
After starting (ant tryme) I created an HTML5Application from the Initializr template. I kept the selection of "create bower/gulp/package.json" files.
The project was created, NetBeans say "background scanning..." it quickly consumed the whole heap until an OOME occured.
On the next start I tried to stop the background scanning before the IDE was unusable. Dialog said "too early to stop".
[exec] java.lang.OutOfMemoryError: GC overhead limit exceeded
[exec] at java.lang.Long.valueOf(Long.java:840)
[exec] at sun.awt.X11.XToolkit.windowToXWindow(XToolkit.java:369)
[exec] at sun.awt.X11.XToolkit.dispatchEvent(XToolkit.java:486)
[exec] [catch] at sun.awt.X11.XToolkit.run(XToolkit.java:616)
[exec] at sun.awt.X11.XToolkit.run(XToolkit.java:532)
[exec] at java.lang.Thread.run(Thread.java:745)
[exec] WARNING [org.netbeans.core.TimableEventQueue]: no snapshot taken
[exec] SEVERE [global]
[exec] java.lang.OutOfMemoryError: GC overhead limit exceeded
I am currently trying to upload the logfile and heap dump through the exception reporter.
Build 38 still freezes with 100% cpu time and gives a stack overflow while processing node sources: http://statistics.netbeans.org/analytics/exception.do?id=807753 I'm sorry, you have to be patient. I'm very hard working on the performance of parser. Is not finished yet. There is some progress, but still the numbers are not ok for me. I will inform you, when the parser will not suffer with the performance issue. More details if you are interested: The parser is generated with Antlr 4. I have wrote the grammar (you can find the grammar here: http://hg.netbeans.org/web-main/file/ef6230e35c66/javascript2.editor/tools/ECMAScript6.g4), but it appears that the generated parser has big issue with performance. Last week we parsed the prototype.js file almost one minute at first time, the second parsing took 7 seconds. Today after some grammar optimalization, we are able to parse the prototype.js file at the first round about 10s and second time around 860ms. But still the first parsing, when the ATN are filled, is too long and the second parsing should be around 400 ms to get the same performance, like the old parser in NB 8.1 If there is an expert for Antlr4 and could give us an advice or help us with the performance... I will really appreciate it. No worry Petr, I try regularly the latest versions to give you some feedback: I know that the problem is hard. I thought you could be interested on the recursive loop stack overflow. I'm sorry I've no experience no antlr, but wouldn't be best if you try to develop the grammar as a new branch for https://github.com/antlr/grammars-v4/tree/master/ecmascript ? Maybe you can get some help from the developers of the old ES5 grammar. @dusty: I have measured the grammar which you pointed out (the grammar for ECMAScript 5), but the numbers are even worst then we have now. Thanks for evaluating it Petr :) I think that more than one person will be happy to know that there is a ES6 grammar in progress, and maybe you could create a pull request with an entry pointing to the public netbeans repository with it. Just for the record, build #45 still locks netbeans with the need to kill the process I know. I'm solving only the performance issue. I will let you know, when it will be fixed. Could someone fix the build on jenkins so we can test the performance improvements. Jenkins has been restarted. Please give it a try now. Latest build is still #46, failed. Is there any new release to test? I'm really desperate for ES6 support :) Guys, I have to apologise. The parser based on Antlr doesn't suffer from performance problems like a month ago, but the memory consumption is still high. We run in problems in ANTLR 4 runtime itself and we are working with antlr authors to solving these issues. I can not guess what time it will take. In parallel I have taken a close source parser and currently I'm trying to work with this one. Unfortunately it's close source and we have to go through licensing process, where I don't have any doubts how it long it can take. When I get the permission to put the parser to our hg, then there will be available build and hopefully with a few implemented ECMA 6 features. Sorry Dusty that you can't make it work. This is what I had to do to make it work reasonable well. Download the module that is attached to this bug https://netbeans.org/bugzilla/show_bug.cgi?id=238709 and install it. Add a .nbignore file to any folder that contains javascript that is not your source code, this would be node_modules, lib folders and dist folders if you have any. Disable warnings about global variables and change the coloring to match normal variables. Hope that helps someone that want's to try out the branch. Hello Petr, how's going? @Dusty: Still working on the rewriting editor to be able used new parser and I'm waiting to the approval to publish parser. Any news on this? Thank you I'm working on rewriting the support to use new parser. But still waiting for the licence approval to as I wrote before. Now it looks promising. So, technically it looks promising, we are just waiting for license approval? Basically yes. Still a lot of work has to be done. Have you seen this? http://esprima.org/ it is BSD licensed and runs on nashorn (JavaScript parsing in JavaScript... don't know if that works for NetBeans integration) From a blog: http://techshangrila.blogspot.de/2015_10_01_archive.html ANTLR 4 makes use of 1 minute, while Esprima only makes use of 100 milliseconds. Might be worth a shot? In addition to that I found this hint regarding antlr performance improvement: https://groups.google.com/d/msg/antlr-discussion/AUN7BLSDIzo/AdZZSbrBQ7gJ which seems like nobody merged to the antlr4 grammar yet. (As far as I could tell from the antlr4 grammars commit history for ecmascript grammar) (In reply to everflux from comment #111) > Have you seen this? http://esprima.org/ it is BSD licensed and runs on > nashorn (JavaScript parsing in JavaScript... don't know if that works for > NetBeans integration) I have tried to use Esprima. I run in the performance problems as well. Esprima is fast, if you run it in a browser like Chrome. From java, you have to run it as a script through ScriptEngine API and convert the javascript objects to java. It's not as fast as we need it. > > From a blog: http://techshangrila.blogspot.de/2015_10_01_archive.html > ANTLR 4 makes use of 1 minute, > while > Esprima only makes use of 100 milliseconds. In the blog is mentioned grammar, which is ECMAScript 5 grammar, not ECMAScript 6. I have played with this grammar as well. The performance is worst than performance of our grammar. I can provide some numbers. I have a test folder with 20 files (prototype.js, common.js, backbone.js ...). The times of parsing are: Esprima: 8 635ms, 2 991ms Antlr4: 12 869ms, 2 831ms Hand written java parser: 178ms, 66ms The first number represents average time of the first run, when the engines has to be initialized. The second number is a number of the second run, when the files in the tested folder are parsed again. When ANTLR4 is correctly initialized, then the performance of our grammar is the same as performance of Esprima called from java. On the other hand the java hand written parser is much much faster. The good news is that the java parser could be opensourced very soon and it's on good way. At this moment I can publish a link of a build with this parser. I hope that something workable can be put out soon, the project I'm working on uses ES6 everywhere and is driving me mad :) Sorry that esprima didn't work out as expected, but I am glad to hear that the other parser could be open source and fast enough! Regarding esprima I found this blog post http://ariya.ofilabs.com/2014/03/nashorn-the-new-rhino-on-the-block.html which gave me the impression that esprima would be fast enough once nashorn/jvm has optimized/JITted the code. I am really sorry this caused wasted time. My "2 cents" regarding general technologies that may apply to JS vs. Java. Firstly, i'd like to mention performance of Nashorn vs. Rhino ... ------------------------------------------------------------------ In my project we extensively use JS in Java for compiling of LESS files (script being executed is taken from lesscss.org and slightly adapted). Roughly half year ago we switched from Rhino to Nashorn as hosting engine - artificial performance tests in JMH showed that Nashorn is roughly 20-30% quicker after 20th cycle, however warmup was really long and first iterations were 10 times slower. Assuming that final performance under Nashorn should be better - we switched to it. But ... real-life usage was not that encouraging, it was hard to measure, but the feeling was that with Nashorn performance was 20-30% worse. After that few new releases of Rhino came out - the last one at February (See http://mvnrepository.com/artifact/org.mozilla/rhino & https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Rhino/Download_Rhino). We implemented double support - product can optionally run with Nashorn and Rhino. During self-build huge number of LESS files is compiled and current result is below (executed with gradle with '--parallel' option): Rhino 03:42 Nashorn 04:42 Out of this time roughly 2 minutes is taken by compilation of Java sources, so execution time of JS-only part is 1:42 (Rhino) vs. 2:42 (Hashorn) - or 60% longer. Release notes for Rhino mention many changes to support ES6 features. So ... if there is some JS needed to be run in Java I would recommend trying Rhino nowadays too, even if it is not that feature-reach and easy to use as Nashorn. Using Google Closure Compiler as JS parser ... ------------------------------------------------------------------ Another part of functionality in my product is compiling/minifying JS files and extracting various metadata from them. Major part is executed via Google Closure Compiler - it generates both 'minified JS as text' and AST. We are using that AST to generate needed metadata. More info is here: https://github.com/google/closure-compiler/wiki. In recent releases they declare support for ES6 syntax - more details is here https://github.com/google/closure-compiler/wiki/Releases. I've done some perf test - generation of AST for lessc-rhino-1.7.5.js (9383 lines) takes roughly 300ms at first call and 200ms in consequtive calls. You can take that 'reference' file from here and compare against your implementation https://github.com/less/less.js/blob/v1.7.5/dist/lessc-rhino-1.7.5.js I can provide some code samples for quick start if you are interested. @NukemBy: Thanks for your comment. I appreciate this discussion. Rhino we used three or four years ago. Unfortunately the Rhino licence is not compatible for NetBeans anymore. Don't ask me why, it's company lawyers decision and I'm not lawyer, so I can not answer in correct way. So we had to remove all libraries under MPL (Mozilla Public License) from NetBeans and replace Rhino with Nashorn. From this point of view Rhino is not acceptable. Regarding using Google Closure Compiler I need to look at this and evaluate. During editing the source code is in 90% unparsable. We usually improve the parser with better error strategy to get as much info about the source as is possible. The linked file lessc-rhino-1.7.5.js has only 450 lines. Is it the right "reference" file? You wrote about 9838 lines. Or I missed something? Oops, yes - there are 2 files less-rhino-1.7.5.js lessc-rhino-1.7.5.js The first one is long, the last one is short. Here is the long one https://github.com/less/less.js/blob/v1.7.5/dist/less-rhino-1.7.5.js I'm not sure how GCC is resistant to errors - it definitely generates list of errors and warnings, but I did not care beyond that - i.e. in our case we need only healthy files. @NukemBy: I have tested the time of parsing of the longer file (with 9383 lines) and the first parsing takes around 166ms and the second round takes about 65ms. But we can not compare with your numbers, because we have different machines. I can sand you my performance test program and command how to run it, if you want. I have i5-3470 CPU, which has 4 hardware cores. I don't believe you are using some kind of 8-12-core Xeon to baseline difference between 200ms and 65ms by purely hardware reasons. Anyway - 65ms per 9383 lines is really good result to not look onto alternative libraries. Out of curiosity ... - you can compare performance of various CPUs here - http://www.notebookcheck.net/%E2%80%A6rs-Benchmarklist.2436.0.html (list can be filtered/restricted) - the one closer to beginning of list is more performant. I also usually look onto "SuperPI 1M*" as refpoint for single-threaded performance and "wPrime 32" as refpoint for multi-threaded performance. - i moved GCC related perftest code out to JMH test and you can get it here https://github.com/NukemBy/jsparse-test You can add usage of your library side-by-side and check the difference in equal environment. Usage as usual: - mvn install - java -jar target/benchmarks.jar There are 2 tests - ES5-only and ES5 source parsed in ES6 mode (because i did not find rather large JS file in ES6 syntax for tests). My results: Warm-up --------- test.jsparse.MyBenchmark.testGccES6 # Warmup Iteration 1: 594.035 ms/op # Warmup Iteration 2: 275.605 ms/op # Warmup Iteration 3: 265.514 ms/op # Warmup Iteration 4: 264.900 ms/op # Warmup Iteration 5: 263.011 ms/op test.jsparse.MyBenchmark.testGccJS5 # Warmup Iteration 1: 389.972 ms/op # Warmup Iteration 2: 194.451 ms/op # Warmup Iteration 3: 191.395 ms/op # Warmup Iteration 4: 189.125 ms/op # Warmup Iteration 5: 183.466 ms/op Final: Benchmark Mode Cnt Score Error Units MyBenchmark.testGccES6 avgt 5 266.540 ? 11.220 ms/op MyBenchmark.testGccJS5 avgt 5 186.691 ? 15.148 ms/op Created attachment 158785 [details]
Result of performance test from NukemBy
@NukemBy: I have run your test on my machine and this is the result.
When is this going to be ready? I certainly don't wanna dig through all those many comments. Thanks! Hi petr, how's going? any good news? :) (In reply to FMA_ from comment #121) > When is this going to be ready? > I certainly don't wanna dig through all those many comments. Please read https://netbeans.org/bugzilla/show_bug.cgi?id=242387#c103 With the latest compiling dev build, the parser results in high CPU usage, rendering the IDE unusable. And please skim through comments next time, if this feature is so important to you. I had to do so, too. (In reply to dusty from comment #122) > Hi petr, how's going? any good news? :) The process of open sorcing the parser is almost done. Now I need a permission from our layers, which is another process. Hopefully it will be finished soon. Thanks for the update petr, is a pre-release available before the layers do their part? (In reply to dusty from comment #125) > is a pre-release available before the layers do their part? Unfortunately I can not do that, because we can not put the parser in Mercurial and we can not build / distribute it. So far I build it just on my machine. When a build will be available, I put here a link. Thanks for the understanding and patience. Implemented generator support. Issue #258600. Support for different way how to initilized object literals: computed property names, shorthand property and method names. See issue #258603 All these new functionalities give more pain to not be able to be used by us... Please provide a test version ASAP ;-) I fully understand:(. I'm these mainly tracking to remember what is already done. For me this situation is not good as well, because nobody test it :( *** Bug 258710 has been marked as a duplicate of this bug. *** https://github.com/Benvie/JavaScriptNext.tmLanguage Here are some references for ES6 syntax support. Any progress or news about this issue? I'm currently using the latest build, would love to see better support in the parser for template strings and support for object properties that uses reserved words. The main things of the support are done. Unfortunately we are still waiting for the license approval. We are expecting it soon, but the exact date I don't know. There are implemented both support for template strings and for object properties. Even though the performance of the ANTLR based parser is not good, would it make sense to test with that parser to check out the other features? (In reply to everflux from comment #135) > Even though the performance of the ANTLR based parser is not good, would it > make sense to test with that parser to check out the other features? No, I don't think so. When we will get the license for Nashorn/Grall parser, then I basically close the branch with ANTLR parser. The development is done in different branch ecma6-truffel. If some one is willing to test the build, write me personally. (In reply to Petr Pisl from comment #136) > (In reply to everflux from comment #135) > > Even though the performance of the ANTLR based parser is not good, would it > > make sense to test with that parser to check out the other features? > > No, I don't think so. When we will get the license for Nashorn/Grall parser, > then I basically close the branch with ANTLR parser. The development is done > in different branch ecma6-truffel. > > If some one is willing to test the build, write me personally. I want to test the build :) Would be great if there would be at least a way to build from source. I have a hg mirror but it fails to download the parser artifact (of course). I hope the license issues are finally solved and we can have a public build... :) Guys, good news. The license issue was solved! We are merging the branch to the trunk and preparing first public build with the new ECMA6 functionality. Hopefully, it will be available later today or tomorrow. I'm going to put there link, when it will be finished. Thanks for your patience. Great news! Thanks for your work Pietr :-) The first publicly available build with ECMA 6 Features is available at continual build machine here: http://deadlock.netbeans.org/job/javascript2/. It's the build with #2270 and all build with higher number. Only zip file is available there. Just download, unzip somewhere and run it. The best way how to run it, is from a commandline with --userdir swith to specify new userdir. This build has not been tested yet. When all the changes will be propagated to the main repository, then the build will be available as daily build with installers. (In reply to Petr Pisl from comment #142) > The first publicly available build with ECMA 6 Features is available at > continual build machine here: http://deadlock.netbeans.org/job/javascript2/. > It's the build with #2270 and all build with higher number. Only zip file > is available there. Just download, unzip somewhere and run it. The best way > how to run it, is from a commandline with --userdir swith to specify new > userdir. This build has not been tested yet. > > When all the changes will be propagated to the main repository, then the > build will be available as daily build with installers. I just opened a JavaScript ES6 project with this build, worked like a charm. Thanks :) I'm having an exception when I try to debug a NodeJS app: http://statistics.netbeans.org/analytics/exception.do?id=821747 (In reply to pangalz from comment #144) > I'm having an exception when I try to debug a NodeJS app: > > http://statistics.netbeans.org/analytics/exception.do?id=821747 Could you create a new issue for this and assign to the javascript -> debugger category? We should not solve here the particular problems, it will be mess. Thanks (In reply to Petr Pisl from comment #145) > (In reply to pangalz from comment #144) > > I'm having an exception when I try to debug a NodeJS app: > > > > http://statistics.netbeans.org/analytics/exception.do?id=821747 > > Could you create a new issue for this and assign to the javascript -> > debugger category? We should not solve here the particular problems, it will > be mess. Thanks Created a new ticket to handle that https://netbeans.org/bugzilla/show_bug.cgi?id=262433 @Petr, I am using the nightly build of 15th june. Seems like ecmascript 6 changes has been integrated in main build. But I can't get the code completion for even 'const', 'let' and 'class' keywords. Should I file a bug or is the code completion yet to be implemented ? (In reply to Rahul.khandelwal from comment #147) > @Petr, I am using the nightly build of 15th june. > Seems like ecmascript 6 changes has been integrated in main build. > But I can't get the code completion for even 'const', 'let' and 'class' > keywords. > Should I file a bug or is the code completion yet to be implemented ? I'm not sure it really gets to the daily build. Please try with zip build from http://deadlock.netbeans.org/job/javascript2. And if possible try also the next daily build (tomorrow). (In reply to Petr Hejl from comment #148) > (In reply to Rahul.khandelwal from comment #147) > > @Petr, I am using the nightly build of 15th june. > > Seems like ecmascript 6 changes has been integrated in main build. > > But I can't get the code completion for even 'const', 'let' and 'class' > > keywords. > > Should I file a bug or is the code completion yet to be implemented ? > > I'm not sure it really gets to the daily build. Please try with zip build > from http://deadlock.netbeans.org/job/javascript2. And if possible try also > the next daily build (tomorrow). I'll try the next daily build and if it doesn't work then I'll also try zip build from http://deadlock.netbeans.org/job/javascript2. (In reply to Rahul.khandelwal from comment #147) > @Petr, I am using the nightly build of 15th june. > Seems like ecmascript 6 changes has been integrated in main build. > But I can't get the code completion for even 'const', 'let' and 'class' > keywords. > Should I file a bug or is the code completion yet to be implemented ? You are right, they are really missing. I have entered new issue #262448. Thanks I tried the latest nightly build and it's very cool thx for the hard work.
I have to report 2 things:
1. I got a warning if I use the new template string after the end of ``. Please see my screenshot (Template String Problem)
2. I use an arrow function to create a service:
var myService = () => {}; and after this, I add myService to the
.service method:
.service('myService', myService); The parser give me an warning that the global variable myService is not defined and var myService give me the hint that this is unused. Please see my screenshot (Warning arrow function)
Cheers
Created attachment 160081 [details]
Template String Problem
Problem with the template string
Created attachment 160082 [details]
Warning arrow function
Problem with the arrow function.
(In reply to ChrisLE from comment #151) > I tried the latest nightly build and it's very cool thx for the hard work. > I have to report 2 things: > > 1. I got a warning if I use the new template string after the end of ``. > Please see my screenshot (Template String Problem) > > 2. I use an arrow function to create a service: > var myService = () => {}; and after this, I add myService to the > .service method: > > .service('myService', myService); The parser give me an warning that the > global variable myService is not defined and var myService give me the hint > that this is unused. Please see my screenshot (Warning arrow function) > > > Cheers Thanks for the report. We will look into this. But pretty please do not report more and more bugs to this issue. File a separate issues for particular problems so it is better trackable. PetrH. is right. This issue is already huge and nobody is able to read it from begining:). I have filed new issues for the problems found by Chris: issue #262468 and issue #262469 Sure I will create new tickets if I found smth new. Thx. Hi Guys I'm using the nightly build of 15th June too but when I try put ` or other special character the editor makes a scroll down Great Job! @argordmel@netbeans.org Petr has literally just asked if new issues can be logged elsewhere for ECMA 6 (I know the thread is long so you may not have read that). @Petr, is it not possible to close this issue to get people to start new issues (just an idea)? I'm closing this task as done. The basic support is in trunk and if you find a particular problem, please file a new issue. Thanks a lot for your help. *** Bug 238942 has been marked as a duplicate of this bug. *** |