This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 268778 - ModelUtils.getVariables can be faster
Summary: ModelUtils.getVariables can be faster
Status: RESOLVED FIXED
Alias: None
Product: javascript
Classification: Unclassified
Component: Editor (show other bugs)
Version: Dev
Hardware: PC Linux
: P3 normal (vote)
Assignee: Petr Pisl
URL:
Keywords: PERFORMANCE
Depends on:
Blocks: 268745
  Show dependency tree
 
Reported: 2016-11-01 22:33 UTC by Petr Pisl
Modified: 2016-12-06 07:16 UTC (History)
0 users

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Petr Pisl 2016-11-01 22:33:42 UTC
This is a part of suggested performance improvements by NukemBy in issue #268745.

Report from the user:
ModelUtils.getVariables() is the second top-most CPU consuming function. 

All this is happening in this code:

    src\javascript2.model\src\org\netbeans\modules\javascript2\model\api\ModelUtils.java

    public static Collection<? extends JsObject> getVariables(DeclarationScope inScope) {
        HashMap<String, JsObject> result = new HashMap<String, JsObject>();
        while (inScope != null) {
            for (JsObject object : ((JsObject)inScope).getProperties().values()) {
                if (!result.containsKey(object.getName()) && object.getModifiers().contains(Modifier.PRIVATE)) {
                    result.put(object.getName(), object);
                }
            }
            if (inScope instanceof JsFunction) {
                for (JsObject object : ((JsFunction)inScope).getParameters()) {
                    if (!result.containsKey(object.getName())) {
                        result.put(object.getName(), object);
                    }
                }
            }
            for (JsObject object : ((JsObject)inScope).getProperties().values()) {
                if (!result.containsKey(object.getName())) {
                    result.put(object.getName(), object);
                }
            }
            if (inScope.getParentScope() != null && !result.containsKey(((JsObject)inScope).getName())) {
                result.put(((JsObject)inScope).getName(), (JsObject)inScope);
            }
            inScope = inScope.getParentScope();
        }
        return result.values();
    }

and consequtive searches like this

        Collection<? extends JsObject> variables = ModelUtils.getVariables(modelBuilder.getCurrentDeclarationScope());
        for (JsObject variable : variables) {
            if (variable.getName().equals(name.getName())) {
  ---->         alreadyThere = variable;
                break;
            }
        }

Not saying that in the first excerpt HashMap.putIfAbsent() better be used, all this looks like upward searching of a variable in JavaScript scopes, but, per each search, with additional allocation of plenty of temporary objects and raw scan of resulting temp collections afterwards ... this looks weird to me, because everything is ready for much quicker way.

So, I've replaced such code with following

    public static JsObject getScopeVariable(DeclarationScope inScope, String name) {
        for( DeclarationScope curScope = inScope; curScope != null; curScope = curScope.getParentScope()) {
            JsObject prop = ((JsObject)curScope).getProperty(name);
            
            if( prop != null && prop.getModifiers().contains(Modifier.PRIVATE) )
                return prop;
            
            if( curScope instanceof JsFunction ) {
                JsObject param = ((JsFunction)curScope).getParameter(name);
                if( param != null )
                    return param;
            }
            
            if( prop != null )
                return prop;
            
            if( name.equals(((JsObject)inScope).getName()) )
                return (JsObject)inScope;
        }
        
        return null;
    }

    ...
    
  ---->  alreadyThere = ModelUtils.getCurrentScopeVariable(modelBuilder, name.getName());

... no temp object, whole operation is a matter of few hashmap lookup - what is very quick.
 
Above mentioned performance hotspots disappeared in self-profiler - CPU usage of parent function 'ModelVisitor.createJsObject()' dropped from 2900ms to 240ms (12x times, see attached screenshot).

The perf-test i'm conducting is following
 1. Start another instance of NB with separate workspace (--userdir XXX --cachedir XXX)
 2. Open single MAVEN project including few huge JS files - attached    
 3. Save that into separate project group, e.g. "JS Perf Test"
 4. Switch to 'None' project group and close netbeans
 5. Goto 'workspace\cache' and remove 'index' folder
 6. Start NetBeans and then start Self-Profiler
 7. Switch to project group "JS Perf Test"
 8. Wait until background scan is finished and CPU usage goes to 0
 9. Stop self-profiler and check collected metrics
 

With these changes applied unit tests in 'javascript2.model' - All OK. 

I've also made two similar changes in 'javascript2.editor' - but it is hard to say if something get broken there. My initial set of sources has 33 of 2885 tests failing without my changes, and 37 - with them. I suspect that 4 tests fail because of similar optimization in this function (i'm not sure how 'isAnonymous()' coexist with 'getProperty(name)'):

    javascript2.model\src\org\netbeans\modules\javascript2\model\api\Model.java

    public JsObject getVariableAtOffset(String name, int offset) {
        for( DeclarationScope scope = ModelUtils.getDeclarationScope(this, offset);
                scope != null;
                scope = scope.getParentScope() ) {
            JsObject prop = ((JsObject)scope).getProperty(name);
            if (prop != null && !prop.isAnonymous()) {
                return prop;
            }

            if (scope instanceof JsFunction) {
                JsObject param = ((JsFunction)scope).getParameter(name);
                if (param != null) {
                    return param;
                }
            }
        }
        return null;
    }
    
    public Collection<? extends JsObject> getVariables(int offset) {
        List<JsObject> result = new ArrayList<>();
        DeclarationScope scope = ModelUtils.getDeclarationScope(this, offset);
        while (scope != null) {
            for (JsObject object : ((JsObject)scope).getProperties().values()) {
                if (!object.isAnonymous()) {
                    result.add(object);
                }
            }
            if (scope instanceof JsFunction) {
                for (JsObject object : ((JsFunction)scope).getParameters()) {
                    result.add(object);
                }
            }
            scope = scope.getParentScope();
        }
        return result;
    }
    
.. to be used in this context

    javascript2.editor\src\org\netbeans\modules\javascript2\editor\JsCodeCompletion.java

        // new client code
        Model requestModel = Model.getModel(request.result, false);
        JsObject variable = ModelUtils.getVariableAtOffset(requestModel, functionName, request.anchor);
        if (variable != null && variable.getJSKind().isFunction()) {
            // do we know the type of the argument?
            JsFunction function = (JsFunction)variable;
            Collection<? extends JsObject> parameters = function.getParameters();
            for (JsObject parameter: parameters) {
                if (!parameter.getAssignments().isEmpty()) {
                    possibleTypes.addAll(parameter.getAssignments());
                }
            }
        }

        // old client code
        Collection<? extends JsObject> variables = ModelUtils.getVariables(Model.getModel(request.result, false), request.anchor);
        for (JsObject variable : variables) {
            if (variable.getName().equals(functionName) && variable.getJSKind().isFunction()) {
                // do we now the tape of the argument?
                JsFunction function = (JsFunction)variable;
                Collection<? extends JsObject> parameters = function.getParameters();
                for (JsObject parameter: parameters) {
                    if (!parameter.getAssignments().isEmpty()) {
                        possibleTypes.addAll(parameter.getAssignments());
                    }
                }
                break;
            }
        }

.. so ... i'm not sure what is happening here but have two doubts:

1. why code inside 'getVariables(int offset)' is different from 'getVariables(DeclarationScope inScope)'
    - both retrive variables from some scope - but result should be different? 
      Strange, may it happen that forwarding call to existing getVariables(scope) is more correct?

        public Collection<? extends JsObject> getVariables(int offset) {
            DeclarationScope scope = ModelUtils.getDeclarationScope(this, offset);
    -->     return ModelUtils.getVariables(scope); <--
        }
    
    
2. failing unit tests 
    - it is hard to tell what is happening there because of too many tests and too many failing tests - is it problem in unit tests or problen in new code? .
Comment 1 Petr Pisl 2016-11-01 22:43:13 UTC
I have tried your test project and look at the numbers. On my machine ModelUtils.getVariables() consumed 1.299 ms and ModelVisitor.createObject takes 523ms.

After rewriting the code and adding ModelUtils.getScopeVariable(), ModelVisitor.createObject() takes 43ms and ModelUtils.getScopeVariable() takes about 53ms. 

I have just implemented the first part. I will continue with getVariableAtOffset() later. 

Thanks.
Comment 2 Petr Pisl 2016-11-14 13:01:12 UTC
Because the luck of time, I have pushed the first part. Marking as fixed. I will return to the second part in separate issue.
Comment 4 Quality Engineering 2016-11-16 02:51:42 UTC
Integrated into 'main-silver', will be available in build *201611160001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/42483ea70e70
User: Petr Pisl <ppisl@netbeans.org>
Log: #268778 - ModelUtils.getVariables can be faster
Comment 5 Petr Pisl 2016-12-06 07:16:06 UTC
Transplanted to the releases/release82.