This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
The feature would allow to set breakpoints inside the sources of the projects that have generated source maps. A source map provides a way of mapping code within a compressed (JS optimizers, shrinkers, obfuscators) or compiled (GWT, Coffiescript, Dart) back to it’s original position in a source file. This means that – with the help of IDE you can easily debug your applications even after your assets have been optimized and without any plugins. The Chrome and Firefox developer tools both ship with built-in support for source maps. How it is done by InteliJ: http://confluence.jetbrains.com/display/RUBYDEV/Debugging+CoffeeScript+Code
Support for source maps was added into node.js debugger: http://hg.netbeans.org/web-main/rev/de8fa03575e1 changeset: 283817:de8fa03575e1 Source maps support in the client-side debugger might be added as well.
Integrated into 'main-silver', will be available in build *201502242039* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-silver/rev/de8fa03575e1 User: mentlicher@netbeans.org Log: #242780: Use source maps when debugging node.js.
I looked a bit into supporting source maps with the chrome connector. At first I thought there was 2 options: 1. make netbeans converts a file breakpoint to a minified file breakpoint. Hopefully the code written for v8 can be reused, especially the SourceMapsTranslator 2. send the breakpoint as is to chrome and let chrome handle the mapping between minified and actual source. It turns out that this second option is not possible because source maps are not supported when in remote debugging. See: https://code.google.com/p/chromium/issues/detail?id=435924 I will now look into the 1st option.
Created attachment 157975 [details] First draft of source maps support with webkit Here is a first draft of source maps support with webkit. Work remains to be done but I would like to make sure I am heading in the correct direction first. To debug I used this pet project: https://github.com/avandecreme/netbeans-sourcemaps-test Running 'npm install' and 'grunt default' will generate the source maps. With the patch applied one should be able to set breakpoints in fibonacci.js, fizzbuzz.js and sort.js. Those files are compiled into concat.js (for fibonacci.js and fizzbuzz.js) and uglify.js (for sort.js). However, one thing is not working correctly: the breakpoints need to be set after the first execution of the file it resides in (in practice, that means run index.html, set breakpoints, refresh index.html). This is because I rely on the webkit "scriptParsed" event to load the mapping between source and compiled files. I don't think there is a reliable way to use that event and making the breakpoints work directly so I am considering to parse each project file and check if the last line has //# sourceMappingURL=... to discover the mappings before the debugger start. Any feedback welcome!
Created attachment 157976 [details] First draft of source maps support with webkit (with added file)
avandecreme, thanks for your patch! Can you please fill out a Contributor Agreement? http://wiki.netbeans.org/NetBeansUserFAQ#Contributor_Agreement Thanks!
I just sent it by email.
You're right with JSLineBreakpoint.setLine() not matching getLineNumber(). The line numbers set/get should be 1-based contrary to 0-based lines used by Line object. I've fixed that by changeset: 294846:fb2aad77bb49
Integrated into 'main-silver', will be available in build *201601140002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-silver/rev/fb2aad77bb49 User: mentlicher@netbeans.org Log: JSLineBreakpoint.setLine(int) should accept 1-based line number for consistency (as mentioned in a patch to issue #242780).
My OCA has just been accepted.
@Martin, any other comment on the code? Should I parse all the projects file to find the mapping instead of relying on the "script parsed" event? (see my comment with the initial patch)
I'm working on it these days, I have your patch applied and made a bunch of related changes. I'm currently in the process of testing... Thanks a lot for your patch, it helps a lot. I will integrate it soon...
And I'll provide comments, after I have it fully working. :-) I'm using two maps to find efficient inverse translations and currently have similar problems with late breakpoints from scriptParsed() events.
Excellent! I am pretty excited to see source maps supported by netbeans. :)
Martin, it looks like your previous comments have been removed. Is it intentional? If needed, I have them in my mailbox. I can copy them back here. In the mean time, I started to test the latest version of web-main a bit and it seems to be working. I have some comments and questions on the implementation though. So let me know if you are still working on it or not. Thanks again for working on that!
The removal of comments was not intentional, it's a result of replication problems between Bugzilla databases. I hope the comments will be added back... I'm still working on it, I started to write an indexer that collects source maps in the project. Feel free to provide any comments or questions you may have.
The Monday's comments are unfortunately really lost. There's my commit log and comments: 25.1.2016 15:52 UTC Antoine, your patch with modifications is integrated as changeset: 294831:777f30f99645 http://hg.netbeans.org/web-main/rev/777f30f99645 Thanks! Comments: I did not apply changes in Mapping class, the line is not necessary after we use a map for inverse translation. I also did not change the MappingTokenizer, since we do not need the line information. In the SourceMap class, I've added inverseMappings map. The source map does not have to be an invertible function, therefore we register the inverse mapping always for the first value. We sort the mapping according the columns for fast lookup based on binary search. I used your getSource() method. In MiscEditorUtil and it's usages I've added the column number from your patch. I did not add scriptParsed() to Debugger.Listener, since there are 14 implementations of Debugger.Listener and all would have to be changed. I've introduced Debugger.ScriptsListener instead. In LineBreakpointUtils I've applied your patch. I'ved moved SourceMapsTranslator to web.common module, and I've introduced Location class, that encapsulates each side of the mapping, so that queries for file, line and column do not have to repeat the searching. In WebKitBreakpointManager I mostly followed your patch, added a validity check into resubmit() and remember the breakpoint's compiled location to know the file for backward translation. I also wrote a test for SourceMap and SourceMapsTranslator. It seems to work fine, but breakpoints are missed for the first time as you mentioned. I do not think this is a big deal for web development, since after a refresh all breakpoint get hit. But not to confuse users, we should have some scanning that finds and parses all source maps before breakpoints are submitted. I'm about to start to write such a scanner. There is yet another problem - NetBeans IDE does not allow to submit breakpoints into unknown files and thus if user's source files are not JavaScript files, but e.g. TypeScript files, they'll not be able to submit breakpoints into these files. We know the source files from source maps, thus we need a mechanism that would allow us to dynamically enable adding breakpoints to specific files. I'll take this as a separate issue later (but still for the next version). And I'd like to add translation of names as well.
>We sort the mapping according the columns for fast lookup based on binary search. The mappings are stored in a LinkedList making the binary search inefficient. Also, in the binary search, is int i = (i1 + i2) >>> 1; equivalent to int i = (i1 + i2) / 2; ? Any reason to use prefer >>> 1 over / 2 ? Other than that the patch looks good to me. I will try to use it as often as possible in my js developments to try to catch any bug. Adding the mapping of variable names would be great indeed!
> The mappings are stored in a LinkedList making the binary search inefficient. Thanks for the catch. Yes, you're right, I've changed it to ArrayList instead: http://hg.netbeans.org/web-main/rev/d9ae3f6ae007 > Any reason to use prefer >>> 1 over / 2 ? a + b can overflow, when the sum is greater than Integer.MAX_VALUE (for non-negative a and b). Therefore (a+b)/2 can produce bad results. E.g.: (Integer.MAX_VALUE + Integer.MAX_VALUE) / 2 = -1 Fortunately, ints are signed, thus if a sum of two non-negative ints overflow, it still fits into the "unsigned" int. And the unsigned shift operator >>> then produces the correct result.
Thanks for the explanation. :)
Integrated into 'main-silver', will be available in build *201601290002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-silver/rev/d9ae3f6ae007 User: mentlicher@netbeans.org Log: #242780: LinkedList is inefficient in binary search, use ArrayList instead.
Scanning of project sources implemented to find source maps: changeset: 295027:52e66243d911 http://hg.netbeans.org/web-main/rev/52e66243d911 I wanted not to cause any overhead when there are no breakpoints, therefore the scan is run only when there is some breakpoint to be set. Since we scan for the source map files, we expect them to have a "map" file name extension. This seems to be a common convention.
Integrated into 'main-silver', will be available in build *201602030002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-silver/rev/52e66243d911 User: mentlicher@netbeans.org Log: #242780: Pre-scan project sources to find and register all source maps before we add breakpoints.
Variable names and frame names are translated according to source maps: changeset: 295041:6f360122949f http://hg.netbeans.org/web-main/rev/6f360122949f
Integrated into 'main-silver', will be available in build *201602040002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-silver/rev/6f360122949f User: mentlicher@netbeans.org Log: #242780: Translate names of variables and frames according to source maps.
The source maps are improved for node.js debugging as well: changeset: 295080:2f686e5df782 http://hg.netbeans.org/web-main/rev/2f686e5df782 It looks like this is finally fixed, please verify that it works with your projects. To be able to submit breakpoints into e.g. TypeScript or CoffeeScript files, there is issue #257857.
Integrated into 'main-silver', will be available in build *201602060002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-silver/rev/2f686e5df782 User: mentlicher@netbeans.org Log: #242780: The source maps support improved for node.js.
>It looks like this is finally fixed, please verify that it works with your projects. Thanks! I am now using the nightly build. I will let you know if I find anything.
I just got this NPE: java.lang.NullPointerException at org.netbeans.modules.web.common.sourcemap.SourceMapsTranslator$Location.getColumn(SourceMapsTranslator.java:180) at org.netbeans.modules.web.common.sourcemap.SourceMapsTranslatorImpl.getSourceLocation(SourceMapsTranslatorImpl.java:200) at org.netbeans.modules.web.common.sourcemap.SourceMapsTranslatorImpl.getSourceLocation(SourceMapsTranslatorImpl.java:186) at org.netbeans.modules.web.javascript.debugger.MiscEditorUtil$SourceMapsTranslatorDelegate.getSourceLocation(MiscEditorUtil.java:735) at org.netbeans.modules.javascript2.debug.NamesTranslator$1.run(NamesTranslator.java:164) at org.netbeans.modules.parsing.impl.TaskProcessor.callUserTask(TaskProcessor.java:609) at org.netbeans.modules.parsing.api.ParserManager$UserTaskAction.run(ParserManager.java:153) at org.netbeans.modules.parsing.api.ParserManager$UserTaskAction.run(ParserManager.java:137) at org.netbeans.modules.parsing.impl.TaskProcessor$2.call(TaskProcessor.java:204) at org.netbeans.modules.parsing.impl.TaskProcessor$2.call(TaskProcessor.java:201) at org.netbeans.modules.masterfs.filebasedfs.utils.FileChangedManager.priorityIO(FileChangedManager.java:176) at org.netbeans.modules.masterfs.providers.ProvidedExtensions.priorityIO(ProvidedExtensions.java:360) at org.netbeans.modules.parsing.nb.DataObjectEnvFactory.runPriorityIO(DataObjectEnvFactory.java:141) at org.netbeans.modules.parsing.impl.Utilities.runPriorityIO(Utilities.java:88) at org.netbeans.modules.parsing.impl.TaskProcessor.runUserTask(TaskProcessor.java:201) Caused: org.netbeans.modules.parsing.spi.ParseException at org.netbeans.modules.parsing.impl.TaskProcessor.runUserTask(TaskProcessor.java:209) at org.netbeans.modules.parsing.api.ParserManager.parse(ParserManager.java:104) [catch] at org.netbeans.modules.javascript2.debug.NamesTranslator.registerVarTranslations(NamesTranslator.java:151) at org.netbeans.modules.javascript2.debug.NamesTranslator.reverseTranslate(NamesTranslator.java:135) at org.netbeans.modules.web.javascript.debugger.eval.EvaluatorServiceImpl.evaluateExpression(EvaluatorServiceImpl.java:103) at org.netbeans.modules.web.javascript.debugger.eval.Evaluator.evaluateExpression(Evaluator.java:69) at org.netbeans.modules.web.javascript.debugger.annotation.ToolTipAnnotation.evaluate(ToolTipAnnotation.java:114) at org.netbeans.modules.web.javascript.debugger.annotation.ToolTipAnnotation.evaluate(ToolTipAnnotation.java:95) at org.netbeans.modules.javascript2.debug.tooltip.AbstractJSToolTipAnnotation.evaluate(AbstractJSToolTipAnnotation.java:194) at org.netbeans.modules.javascript2.debug.tooltip.AbstractJSToolTipAnnotation.access$000(AbstractJSToolTipAnnotation.java:80) at org.netbeans.modules.javascript2.debug.tooltip.AbstractJSToolTipAnnotation$1.run(AbstractJSToolTipAnnotation.java:135) at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1443) at org.netbeans.modules.openide.util.GlobalLookup.execute(GlobalLookup.java:68) at org.openide.util.lookup.Lookups.executeWith(Lookups.java:303) at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:2058) I can investigate it later on. Just let me know. Also, the name mapping doesn't seem to work for me (I am using the latest nightly build). But I might be doing something wrong with the source maps.
The NPE should be fixed by changeset: 295156:8ae3ada735e8 http://hg.netbeans.org/web-main/rev/8ae3ada735e8 It's likely that this exception broke the names mapping. If names mapping still won't work for you after this fix, please provide a sample test case on which I could investigate it.
Integrated into 'main-silver', will be available in build *201602110002* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-silver/rev/8ae3ada735e8 User: mentlicher@netbeans.org Log: #242780: Prevent from NPE when column is not defined.
I got another exception: java.lang.NullPointerException at org.netbeans.modules.javascript2.debug.NamesTranslator$2.run(NamesTranslator.java:189) at org.netbeans.modules.parsing.impl.TaskProcessor.callUserTask(TaskProcessor.java:609) at org.netbeans.modules.parsing.api.ParserManager$UserTaskAction.run(ParserManager.java:153) at org.netbeans.modules.parsing.api.ParserManager$UserTaskAction.run(ParserManager.java:137) at org.netbeans.modules.parsing.impl.TaskProcessor$2.call(TaskProcessor.java:204) at org.netbeans.modules.parsing.impl.TaskProcessor$2.call(TaskProcessor.java:201) at org.netbeans.modules.masterfs.filebasedfs.utils.FileChangedManager.priorityIO(FileChangedManager.java:176) at org.netbeans.modules.masterfs.providers.ProvidedExtensions.priorityIO(ProvidedExtensions.java:360) at org.netbeans.modules.parsing.nb.DataObjectEnvFactory.runPriorityIO(DataObjectEnvFactory.java:141) at org.netbeans.modules.parsing.impl.Utilities.runPriorityIO(Utilities.java:88) at org.netbeans.modules.parsing.impl.TaskProcessor.runUserTask(TaskProcessor.java:201) Caused: org.netbeans.modules.parsing.spi.ParseException at org.netbeans.modules.parsing.impl.TaskProcessor.runUserTask(TaskProcessor.java:209) at org.netbeans.modules.parsing.api.ParserManager.parse(ParserManager.java:104) [catch] at org.netbeans.modules.javascript2.debug.NamesTranslator.translateDeclarationNodeName(NamesTranslator.java:183) at org.netbeans.modules.web.javascript.debugger.callstack.CallStackModel.getDisplayName(CallStackModel.java:202) at org.netbeans.spi.viewmodel.Models$DelegatingNodeModel.getDisplayName(Models.java:3054) at org.netbeans.spi.viewmodel.Models$CompoundModel.getDisplayName(Models.java:4491) at org.netbeans.modules.viewmodel.TreeModelNode.setModelDisplayName(TreeModelNode.java:843) at org.netbeans.modules.viewmodel.TreeModelNode.getDisplayName(TreeModelNode.java:893) at org.openide.explorer.view.VisualizerNode.getDisplayName(VisualizerNode.java:228) at org.openide.explorer.view.VisualizerNode.toString(VisualizerNode.java:529) at org.openide.awt.HtmlRendererImpl.configureFrom(HtmlRendererImpl.java:197) at org.openide.awt.HtmlRendererImpl.getTableCellRendererComponent(HtmlRendererImpl.java:131) at sun.reflect.GeneratedMethodAccessor85.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.netbeans.swing.outline.DefaultOutlineCellRenderer$HtmlRenderer$Renderer.getTableCellRendererComponent(DefaultOutlineCellRenderer.java:605) at org.netbeans.swing.outline.DefaultOutlineCellRenderer.getTableCellRendererComponent(DefaultOutlineCellRenderer.java:277) at org.openide.explorer.view.OutlineView$OutlineViewOutline$TranslatedTableCellRenderer.getTableCellRendererComponent(OutlineView.java:2308) at org.netbeans.modules.viewmodel.DelegatingCellRenderer.getTableCellRendererComponent(DelegatingCellRenderer.java:91) at org.openide.explorer.view.OutlineView$OutlineViewOutline$TranslatedTableCellRenderer.getTableCellRendererComponent(OutlineView.java:2308) at javax.swing.JTable.prepareRenderer(JTable.java:5723) at javax.swing.plaf.basic.BasicTableUI.paintCell(BasicTableUI.java:2114) at javax.swing.plaf.basic.BasicTableUI.paintCells(BasicTableUI.java:2016) at javax.swing.plaf.basic.BasicTableUI.paint(BasicTableUI.java:1812) at javax.swing.plaf.ComponentUI.update(ComponentUI.java:161) at javax.swing.JComponent.paintComponent(JComponent.java:780) at javax.swing.JComponent.paint(JComponent.java:1056) at javax.swing.JComponent.paintChildren(JComponent.java:889) at javax.swing.JComponent.paint(JComponent.java:1065) at javax.swing.JViewport.paint(JViewport.java:728) at javax.swing.JComponent.paintChildren(JComponent.java:889) at javax.swing.JComponent.paint(JComponent.java:1065) at javax.swing.JComponent.paintChildren(JComponent.java:889) at javax.swing.JComponent.paint(JComponent.java:1065) at javax.swing.JComponent.paintChildren(JComponent.java:889) at javax.swing.JComponent.paint(JComponent.java:1065) at javax.swing.JComponent.paintChildren(JComponent.java:889) at javax.swing.JComponent.paint(JComponent.java:1065) at javax.swing.JComponent.paintChildren(JComponent.java:889) at javax.swing.JComponent.paint(JComponent.java:1065) at javax.swing.JComponent.paintChildren(JComponent.java:889) at javax.swing.JComponent.paint(JComponent.java:1065) at org.netbeans.swing.tabcontrol.TabbedContainer.paint(TabbedContainer.java:994) at javax.swing.JComponent.paintChildren(JComponent.java:889) at javax.swing.JComponent.paint(JComponent.java:1065) at javax.swing.JComponent.paintToOffscreen(JComponent.java:5210) at javax.swing.RepaintManager$PaintManager.paintDoubleBuffered(RepaintManager.java:1579) at javax.swing.RepaintManager$PaintManager.paint(RepaintManager.java:1502) at javax.swing.BufferStrategyPaintManager.paint(BufferStrategyPaintManager.java:306) at javax.swing.RepaintManager.paint(RepaintManager.java:1272) at javax.swing.JComponent._paintImmediately(JComponent.java:5158) at javax.swing.JComponent.paintImmediately(JComponent.java:4969) at javax.swing.JComponent.paintImmediately(JComponent.java:4950) at javax.swing.RepaintManager$4.run(RepaintManager.java:831) at javax.swing.RepaintManager$4.run(RepaintManager.java:814) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76) at javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:814) at javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:789) at javax.swing.RepaintManager.prePaintDirtyRegions(RepaintManager.java:738) at javax.swing.RepaintManager.access$1200(RepaintManager.java:64) at javax.swing.RepaintManager$ProcessingRunnable.run(RepaintManager.java:1732) at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311) at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756) at java.awt.EventQueue.access$500(EventQueue.java:97) at java.awt.EventQueue$3.run(EventQueue.java:709) at java.awt.EventQueue$3.run(EventQueue.java:703) at java.security.AccessController.doPrivileged(Native Method) at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76) at java.awt.EventQueue.dispatchEvent(EventQueue.java:726) at org.netbeans.core.TimableEventQueue.dispatchEvent(TimableEventQueue.java:159) at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201) at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116) at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93) at java.awt.EventDispatchThread.run(EventDispatchThread.java:82) Also, displaying the call stack is very slow (hang the IDE for a while).