This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 262141 - [PHP7] Context sensitive lexer & Uniform variable syntax
Summary: [PHP7] Context sensitive lexer & Uniform variable syntax
Status: RESOLVED FIXED
Alias: None
Product: php
Classification: Unclassified
Component: Editor (show other bugs)
Version: 8.2
Hardware: All All
: P3 normal with 1 vote (vote)
Assignee: junichi11
URL:
Keywords:
: 268463 268708 269380 270092 271124 271164 (view as bug list)
Depends on:
Blocks:
 
Reported: 2016-05-18 15:53 UTC by m6w6
Modified: 2017-11-29 22:46 UTC (History)
8 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Screenshot of syntax errors (98.59 KB, image/png)
2016-05-18 15:53 UTC, m6w6
Details
screenshot for context sensitive lexer (17.01 KB, image/png)
2016-11-03 02:14 UTC, junichi11
Details
proposed patch for context sensitive lexer (4.44 MB, patch)
2016-11-03 02:21 UTC, junichi11
Details | Diff
screenshot for uniform variable syntax (61.10 KB, image/png)
2016-11-03 03:03 UTC, junichi11
Details
proposed patch for context sensitive lexer v2 (4.65 MB, patch)
2017-05-03 03:53 UTC, junichi11
Details | Diff
screenshot for Uniform Variable Syntax (50.24 KB, image/png)
2017-05-11 01:57 UTC, junichi11
Details

Note You need to log in before you can comment on or make changes to this bug.
Description m6w6 2016-05-18 15:53:18 UTC
Created attachment 159802 [details]
Screenshot of syntax errors

Many parts of the "Context sensitive lexer"[1] and the "Uniform variable syntax"[2] RFC seem not implemented yet, see screenshot.



Examples:

<?php

class Example
{
	const GOTO = 1;

	function __construct(Example $list = null) {
		if ($list) {
			$this->foreach($list);
		}
	}

	function apply(callable $function) {
		foreach ([self::GOTO] as $key => $val) {
			$function($key, $val);
		}
	}

	function foreach(Example $list) {
		$list->apply(function() {
			printf("So contrived!\n");
		});
		return $list;
	}

	function as() : Example {
		return (clone $this)->foreach($this);
	}

}

(clone (new Example(new Example)))->as();

?>

[1] https://wiki.php.net/rfc/context_sensitive_lexer
[2] https://wiki.php.net/rfc/uniform_variable_syntax
Comment 1 Tomas Mysik 2016-05-19 08:30:03 UTC
Unfortunately, due to limitations of the NetBeans PHP support internals, this issue will very likely not be fixed in NB 8.2, sorry. Context sensitive lexer is definitely out of scope, some (the most basic) cases of Uniform variable syntax are supported and we are working hard to try to support more such cases but as I wrote, it is not easy at all.

Thanks for understanding and your report.
Comment 2 Tomas Mysik 2016-10-11 11:18:41 UTC
*** Bug 268463 has been marked as a duplicate of this bug. ***
Comment 3 junichi11 2016-10-21 14:23:22 UTC
First, I'll try Context sensitive lexer.

Thanks.
Comment 4 Tomas Mysik 2016-10-27 10:10:14 UTC
*** Bug 268708 has been marked as a duplicate of this bug. ***
Comment 5 junichi11 2016-11-03 02:14:31 UTC
Created attachment 162736 [details]
screenshot for context sensitive lexer
Comment 6 junichi11 2016-11-03 02:21:53 UTC
Created attachment 162737 [details]
proposed patch for context sensitive lexer
Comment 7 junichi11 2016-11-03 03:03:58 UTC
Created attachment 162738 [details]
screenshot for uniform variable syntax

Now, I'm fixing syntax errors for uniform variable syntax.

The following case is not resolved yet:
[$object1, $object2][0]->property;

I think that basically, variables($variable, $this->foo, Foo::$bar, $arr['test'], getStr() e.t.c.) and clone expression(clone $object) are used within "()". 

e.g.
($variable->getFoo())::foo();
(clone new MyClass())->foo();

Are there other cases using "()"? Could you provide them if you know them?

BTW, I want to fix other issues before I fix this issue. To fix this issue takes much time. So, I'll try this later.

Thanks for understanding.
Comment 8 Tomas Mysik 2016-11-03 06:40:55 UTC
Great job, Junichi!
Comment 9 m6w6 2016-11-03 06:58:54 UTC
> Are there other cases using "()"? Could you provide them if you know them?

(function($s){var_dump($s);})("hi!");
Comment 10 Tomas Mysik 2016-11-03 14:08:52 UTC
Once this issue is fixed, we should improve our signature files (they contain methods like _clone() etc. to avoid syntax errors).

Thanks.
Comment 11 Tomas Mysik 2016-11-03 14:09:25 UTC
(In reply to Tomas Mysik from comment #10)
> Once this issue is fixed, we should improve our signature files (they
> contain methods like _clone() etc. to avoid syntax errors).

I forgot to add - I will do that :)

Thanks.
Comment 12 junichi11 2016-11-04 02:04:26 UTC
(In reply to m6w6 from comment #9)
> > Are there other cases using "()"? Could you provide them if you know them?
> 
> (function($s){var_dump($s);})("hi!");

Thanks for it! It' was fixed in the issue 259111[1].

[1] https://netbeans.org/bugzilla/show_bug.cgi?id=259111

(In reply to Tomas Mysik from comment #11)
> (In reply to Tomas Mysik from comment #10)
> > Once this issue is fixed, we should improve our signature files (they
> > contain methods like _clone() etc. to avoid syntax errors).
> 
> I forgot to add - I will do that :)

Thanks!
Comment 13 junichi11 2017-03-28 23:09:29 UTC
*** Bug 270092 has been marked as a duplicate of this bug. ***
Comment 14 junichi11 2017-04-21 00:31:57 UTC
I'm sorry to be late. I'll work on this issue again. First of all, I'll commit changes for Context Sensitive Lexer part. We have to verify the patch once before that because the parser has some changes for PHP7.1 support.

Thanks.
Comment 15 junichi11 2017-05-03 03:53:54 UTC
Created attachment 164229 [details]
proposed patch for context sensitive lexer v2

The first patch has a problem in this case(const CONST = [1,2], GOTO = 2;). So I fixed it.

I moved exsting testfiles for CC(from php.editor/test/unit/data/testfiles/completion/lib/php70/* to php.editor/test/unit/data/testfiles/completion/lib/php70/base/*)

I run all unit tests once. Then I'll push my changes today or tomorrow.

Thanks.
Comment 16 junichi11 2017-05-04 02:02:43 UTC
Fixed context sensitive lexer part.

Please try to test it if possible. Probably, it will be available in a dev build tomorrow or later.

http://hg.netbeans.org/web-main/rev/63e15ba37eaa

Thanks.
Comment 17 Tomas Mysik 2017-05-04 04:39:05 UTC
Great job, Junichi, as always! BTW feel free to add it to NB 9 N&N wiki page [1] ;)

Thanks a lot!
[1] http://wiki.netbeans.org/NewAndNoteworthyNB9
Comment 18 Quality Engineering 2017-05-05 01:46:24 UTC
Integrated into 'main-silver', will be available in build *201705050001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/63e15ba37eaa
User: Junichi Yamamoto <junichi11@netbeans.org>
Log: #262141 - [PHP7] Context sensitive lexer & Uniform variable syntax

Context sensitive lexer part
Comment 19 junichi11 2017-05-11 01:35:48 UTC
*** Bug 269380 has been marked as a duplicate of this bug. ***
Comment 20 junichi11 2017-05-11 01:57:53 UTC
Created attachment 164293 [details]
screenshot for Uniform Variable Syntax

Current progress for Uniform Variable Syntax:

I fixed the parser and existing unit tests.

Although the parser may possibly have wrong fixes(it seems that there is no error at the moment), we can fix missing cases for Uniform Variable Syntax.

Missing cases:
// (...) expression
($a)['0'];
($a)();
(($a)['0'])();
(Example::bar());
(Example::$bar)['test'];
(Example::$bar)['baz']();
(getString()){0};
(($example->arr())[0])[9];
($example->create())->test();
($example->create())::bar();

// nested ::
$foo::$bar::$baz;
$foo::$bar::$baz::$test;
$foo::$bar::$baz->test();
$foo::$bar::$baz();
$foo::$bar::baz();

// clone expression ((...) expression)
class CloneExpression
{
    public function create() {
        $test = (clone $this->test1())->test2();
        return (clone $this)->test1();
    }

    public function test1() {
    }

    public function test2() {
    }

}
(clone (new CloneExpression()))->create();

// array access with {}
getStr(){0};


Now, we have to fix some features(e.g. CC, GotoDeclaration). Probably, I'll just look at simple cases for them.

Thanks.
Comment 21 junichi11 2017-05-11 02:12:02 UTC
(In reply to junichi11 from comment #20)
> Missing cases:
The following case also is fixed:
[$obj1, $obj2][0]->prop;
Comment 22 Tomas Mysik 2017-05-11 04:17:29 UTC
Nice job, Junichi. Thanks!
Comment 23 junichi11 2017-05-19 03:06:16 UTC
Fixed Uniform variable syntax part.
Could someone verify it when it is available in a development build?

http://hg.netbeans.org/web-main/rev/48ca30d3fcc5

Unfortunately, some features(CC, GotoDeclaration, MarkOccurrences) don't work with (...) expression yet. This problem should be submitted as a new issue. To resolve it, VariousUtils.getSemiType() has to be improved.

Thanks.
Comment 24 Tomas Mysik 2017-05-19 05:07:50 UTC
Junichi, thanks a lot!
Comment 25 junichi11 2017-05-19 05:09:52 UTC
You are welcome :)
Comment 26 Quality Engineering 2017-05-24 19:39:12 UTC
Integrated into 'main-silver', will be available in build *201705241652* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/48ca30d3fcc5
User: Junichi Yamamoto <junichi11@netbeans.org>
Log: #262141 - [PHP7] Context sensitive lexer & Uniform variable syntax

Uniform variable syntax part. Some features(CC, GotoDeclaration, MarkOccurrences) don't work with (...) expression yet.
Comment 27 junichi11 2017-07-20 22:40:23 UTC
*** Bug 271124 has been marked as a duplicate of this bug. ***
Comment 28 junichi11 2017-07-27 02:17:34 UTC
*** Bug 271164 has been marked as a duplicate of this bug. ***
Comment 29 instead 2017-11-29 18:53:09 UTC
This is still (again) an issue. Netbeans 8.2. Using as method name keyword like foreach, print, list cause error.
Comment 30 junichi11 2017-11-29 22:46:54 UTC
(In reply to instead from comment #29)
> This is still (again) an issue. Netbeans 8.2. Using as method name keyword
> like foreach, print, list cause error.

Please don't reopen it. This was already fixed. Please try to use a dev build[1].

[1] http://bits.netbeans.org/download/trunk/nightly/latest/