This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 262141

Summary: [PHP7] Context sensitive lexer & Uniform variable syntax
Product: php Reporter: m6w6
Component: EditorAssignee: junichi11
Status: RESOLVED FIXED    
Severity: normal CC: Itako, joepsyko, junichi11, mbukovsky, neufena, sirpilan, sviatopolk, tmysik
Priority: P3    
Version: 8.2   
Hardware: All   
OS: All   
Issue Type: DEFECT Exception Reporter:
Attachments: Screenshot of syntax errors
screenshot for context sensitive lexer
proposed patch for context sensitive lexer
screenshot for uniform variable syntax
proposed patch for context sensitive lexer v2
screenshot for Uniform Variable Syntax

Description m6w6 2016-05-18 15:53:18 UTC
Created attachment 159802 [details]
Screenshot of syntax errors

Many parts of the "Context sensitive lexer"[1] and the "Uniform variable syntax"[2] RFC seem not implemented yet, see screenshot.



Examples:

<?php

class Example
{
	const GOTO = 1;

	function __construct(Example $list = null) {
		if ($list) {
			$this->foreach($list);
		}
	}

	function apply(callable $function) {
		foreach ([self::GOTO] as $key => $val) {
			$function($key, $val);
		}
	}

	function foreach(Example $list) {
		$list->apply(function() {
			printf("So contrived!\n");
		});
		return $list;
	}

	function as() : Example {
		return (clone $this)->foreach($this);
	}

}

(clone (new Example(new Example)))->as();

?>

[1] https://wiki.php.net/rfc/context_sensitive_lexer
[2] https://wiki.php.net/rfc/uniform_variable_syntax
Comment 1 Tomas Mysik 2016-05-19 08:30:03 UTC
Unfortunately, due to limitations of the NetBeans PHP support internals, this issue will very likely not be fixed in NB 8.2, sorry. Context sensitive lexer is definitely out of scope, some (the most basic) cases of Uniform variable syntax are supported and we are working hard to try to support more such cases but as I wrote, it is not easy at all.

Thanks for understanding and your report.
Comment 2 Tomas Mysik 2016-10-11 11:18:41 UTC
*** Bug 268463 has been marked as a duplicate of this bug. ***
Comment 3 junichi11 2016-10-21 14:23:22 UTC
First, I'll try Context sensitive lexer.

Thanks.
Comment 4 Tomas Mysik 2016-10-27 10:10:14 UTC
*** Bug 268708 has been marked as a duplicate of this bug. ***
Comment 5 junichi11 2016-11-03 02:14:31 UTC
Created attachment 162736 [details]
screenshot for context sensitive lexer
Comment 6 junichi11 2016-11-03 02:21:53 UTC
Created attachment 162737 [details]
proposed patch for context sensitive lexer
Comment 7 junichi11 2016-11-03 03:03:58 UTC
Created attachment 162738 [details]
screenshot for uniform variable syntax

Now, I'm fixing syntax errors for uniform variable syntax.

The following case is not resolved yet:
[$object1, $object2][0]->property;

I think that basically, variables($variable, $this->foo, Foo::$bar, $arr['test'], getStr() e.t.c.) and clone expression(clone $object) are used within "()". 

e.g.
($variable->getFoo())::foo();
(clone new MyClass())->foo();

Are there other cases using "()"? Could you provide them if you know them?

BTW, I want to fix other issues before I fix this issue. To fix this issue takes much time. So, I'll try this later.

Thanks for understanding.
Comment 8 Tomas Mysik 2016-11-03 06:40:55 UTC
Great job, Junichi!
Comment 9 m6w6 2016-11-03 06:58:54 UTC
> Are there other cases using "()"? Could you provide them if you know them?

(function($s){var_dump($s);})("hi!");
Comment 10 Tomas Mysik 2016-11-03 14:08:52 UTC
Once this issue is fixed, we should improve our signature files (they contain methods like _clone() etc. to avoid syntax errors).

Thanks.
Comment 11 Tomas Mysik 2016-11-03 14:09:25 UTC
(In reply to Tomas Mysik from comment #10)
> Once this issue is fixed, we should improve our signature files (they
> contain methods like _clone() etc. to avoid syntax errors).

I forgot to add - I will do that :)

Thanks.
Comment 12 junichi11 2016-11-04 02:04:26 UTC
(In reply to m6w6 from comment #9)
> > Are there other cases using "()"? Could you provide them if you know them?
> 
> (function($s){var_dump($s);})("hi!");

Thanks for it! It' was fixed in the issue 259111[1].

[1] https://netbeans.org/bugzilla/show_bug.cgi?id=259111

(In reply to Tomas Mysik from comment #11)
> (In reply to Tomas Mysik from comment #10)
> > Once this issue is fixed, we should improve our signature files (they
> > contain methods like _clone() etc. to avoid syntax errors).
> 
> I forgot to add - I will do that :)

Thanks!
Comment 13 junichi11 2017-03-28 23:09:29 UTC
*** Bug 270092 has been marked as a duplicate of this bug. ***
Comment 14 junichi11 2017-04-21 00:31:57 UTC
I'm sorry to be late. I'll work on this issue again. First of all, I'll commit changes for Context Sensitive Lexer part. We have to verify the patch once before that because the parser has some changes for PHP7.1 support.

Thanks.
Comment 15 junichi11 2017-05-03 03:53:54 UTC
Created attachment 164229 [details]
proposed patch for context sensitive lexer v2

The first patch has a problem in this case(const CONST = [1,2], GOTO = 2;). So I fixed it.

I moved exsting testfiles for CC(from php.editor/test/unit/data/testfiles/completion/lib/php70/* to php.editor/test/unit/data/testfiles/completion/lib/php70/base/*)

I run all unit tests once. Then I'll push my changes today or tomorrow.

Thanks.
Comment 16 junichi11 2017-05-04 02:02:43 UTC
Fixed context sensitive lexer part.

Please try to test it if possible. Probably, it will be available in a dev build tomorrow or later.

http://hg.netbeans.org/web-main/rev/63e15ba37eaa

Thanks.
Comment 17 Tomas Mysik 2017-05-04 04:39:05 UTC
Great job, Junichi, as always! BTW feel free to add it to NB 9 N&N wiki page [1] ;)

Thanks a lot!
[1] http://wiki.netbeans.org/NewAndNoteworthyNB9
Comment 18 Quality Engineering 2017-05-05 01:46:24 UTC
Integrated into 'main-silver', will be available in build *201705050001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/63e15ba37eaa
User: Junichi Yamamoto <junichi11@netbeans.org>
Log: #262141 - [PHP7] Context sensitive lexer & Uniform variable syntax

Context sensitive lexer part
Comment 19 junichi11 2017-05-11 01:35:48 UTC
*** Bug 269380 has been marked as a duplicate of this bug. ***
Comment 20 junichi11 2017-05-11 01:57:53 UTC
Created attachment 164293 [details]
screenshot for Uniform Variable Syntax

Current progress for Uniform Variable Syntax:

I fixed the parser and existing unit tests.

Although the parser may possibly have wrong fixes(it seems that there is no error at the moment), we can fix missing cases for Uniform Variable Syntax.

Missing cases:
// (...) expression
($a)['0'];
($a)();
(($a)['0'])();
(Example::bar());
(Example::$bar)['test'];
(Example::$bar)['baz']();
(getString()){0};
(($example->arr())[0])[9];
($example->create())->test();
($example->create())::bar();

// nested ::
$foo::$bar::$baz;
$foo::$bar::$baz::$test;
$foo::$bar::$baz->test();
$foo::$bar::$baz();
$foo::$bar::baz();

// clone expression ((...) expression)
class CloneExpression
{
    public function create() {
        $test = (clone $this->test1())->test2();
        return (clone $this)->test1();
    }

    public function test1() {
    }

    public function test2() {
    }

}
(clone (new CloneExpression()))->create();

// array access with {}
getStr(){0};


Now, we have to fix some features(e.g. CC, GotoDeclaration). Probably, I'll just look at simple cases for them.

Thanks.
Comment 21 junichi11 2017-05-11 02:12:02 UTC
(In reply to junichi11 from comment #20)
> Missing cases:
The following case also is fixed:
[$obj1, $obj2][0]->prop;
Comment 22 Tomas Mysik 2017-05-11 04:17:29 UTC
Nice job, Junichi. Thanks!
Comment 23 junichi11 2017-05-19 03:06:16 UTC
Fixed Uniform variable syntax part.
Could someone verify it when it is available in a development build?

http://hg.netbeans.org/web-main/rev/48ca30d3fcc5

Unfortunately, some features(CC, GotoDeclaration, MarkOccurrences) don't work with (...) expression yet. This problem should be submitted as a new issue. To resolve it, VariousUtils.getSemiType() has to be improved.

Thanks.
Comment 24 Tomas Mysik 2017-05-19 05:07:50 UTC
Junichi, thanks a lot!
Comment 25 junichi11 2017-05-19 05:09:52 UTC
You are welcome :)
Comment 26 Quality Engineering 2017-05-24 19:39:12 UTC
Integrated into 'main-silver', will be available in build *201705241652* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/48ca30d3fcc5
User: Junichi Yamamoto <junichi11@netbeans.org>
Log: #262141 - [PHP7] Context sensitive lexer & Uniform variable syntax

Uniform variable syntax part. Some features(CC, GotoDeclaration, MarkOccurrences) don't work with (...) expression yet.
Comment 27 junichi11 2017-07-20 22:40:23 UTC
*** Bug 271124 has been marked as a duplicate of this bug. ***
Comment 28 junichi11 2017-07-27 02:17:34 UTC
*** Bug 271164 has been marked as a duplicate of this bug. ***
Comment 29 instead 2017-11-29 18:53:09 UTC
This is still (again) an issue. Netbeans 8.2. Using as method name keyword like foreach, print, list cause error.
Comment 30 junichi11 2017-11-29 22:46:54 UTC
(In reply to instead from comment #29)
> This is still (again) an issue. Netbeans 8.2. Using as method name keyword
> like foreach, print, list cause error.

Please don't reopen it. This was already fixed. Please try to use a dev build[1].

[1] http://bits.netbeans.org/download/trunk/nightly/latest/