This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 235280

Summary:	Netbeans fails to parse TAP report if no line is reported in diagnostic
Product:	php	Reporter:	jubianchi
Component:	Atoum	Assignee:	Tomas Mysik <tmysik>
Status:	RESOLVED FIXED
Severity:	normal	CC:	mmirilovic, obrejla, vriha
Priority:	P1	Keywords:	7.4_HR_FIX
Version:	7.4
Hardware:	All
OS:	All
Issue Type:	DEFECT	Exception Reporter:

Description jubianchi 2013-08-29 22:01:13 UTC

I just opened a PR on the atoum repository to improve the TAP report in case of uncomplete test method (when a die, exit or PHP Fatal ends the test): https://github.com/atoum/atoum/pull/253

In this case, atoum is not able to report the line where the fail happened and Netbeans seems to not be able to parse the TAP report in this case. Here is an example TAP output produced when this happen (I got this from the output window in Netbeans):

-------------------------------------------
1..2
not ok 1 - tests\units\netbeans\sample\Calculator::testDivide()
# Fatal error : Call to undefined method netbeans\sample\Calculator::divide()
# /Users/jubianchi/NetBeansProjects/nb-atoum-blog/tests/units/netbeans/sample/Calculator.php
not ok 2 - tests\units\netbeans\sample\Calculator::testAdd()
# Fatal error : Call to undefined method netbeans\sample\Calculator::add()
# /Users/jubianchi/NetBeansProjects/nb-atoum-blog/tests/units/netbeans/sample/Calculator.php
------------------------------------------- 

As you can see here, the two test cases produce a PHP Fatal error and the second diagnostic line only conatains the file path where the error happened. In this case, Netbeans reports everything as OK, saying "Both tests passed." in the test results window.

If I edit my source code so that there is only one fatal error, then Netbeans only parses the line where we have both the file path and line number:

-------------------------------------------
1..2
not ok 1 - tests\units\netbeans\sample\Calculator::testAdd()
# null is not an integer
# /Users/jubianchi/NetBeansProjects/nb-atoum-blog/tests/units/netbeans/sample/Calculator.php:13
not ok 2 - tests\units\netbeans\sample\Calculator::testDivide()
# Fatal error : Call to undefined method netbeans\sample\Calculator::divide()
# /Users/jubianchi/NetBeansProjects/nb-atoum-blog/tests/units/netbeans/sample/Calculator.php
-------------------------------------------

Now, in the test results window, I get the following result : "1 test passed, 1 tests failed"
Here, I should get something like "2 tests failed"

I think the expression used to parse the diagnostic line should be a bit more permissive and allow missing line number as atoum won't be able to find them in such cases.

Comment 1 Tomas Mysik 2013-08-30 06:06:41 UTC

Thanks for reporting, writing parser without any specification of the input format is always funny :)

Comment 2 Tomas Mysik 2013-08-30 07:09:18 UTC

You have some older build, in the latest dev build the tests failed with ABORTED status. However, I have improved it a bit:

http://hg.netbeans.org/web-main/rev/35791eff71ec

Please, verify using at least this build [1] (simply download the ZIP file from there once the build finishes).

Thanks.
[1] http://bertram.netbeans.org:8080/job/PHP-build/7005/

Comment 3 jubianchi 2013-09-02 19:38:11 UTC

Just tested with the Netbeans build from here : http://bertram.netbeans.org:8080/job/PHP-build/7019/

Unfortunately, I'm still getting a bug but it's a bit different, here is the output from atoum (I checked out master to test without my PR, so I get empty diagnostic line):

-------------------------------------------
1..2
not ok 1 - tests\units\netbeans\sample\Calculator::testDivide()
# 
# /Users/jubianchi/NetBeansProjects/nb-atoum-blog/tests/units/netbeans/sample/Calculator.php
ok 2
# tests\units\netbeans\sample\Calculator::testAdd()
-------------------------------------------

With this output from atoum, Netbeans reports "No tests executed"
When I check out my PR which fixes the empty diagnostic line, everything looks good.

So the filepath with no line are parsed correctly but the bug still appears when we have empty diagnostic line (which should no happen with the fix on atoum).

Comment 4 jubianchi 2013-09-02 19:46:12 UTC

Sorry for the flood but I wanted to add some more infos on this bug.
The fact that Netbeans can't parse filepath with no line number is a bit disturbing as it reports tests success when there are fails:

-------------------------------------------
1..2
not ok 1 - tests\units\netbeans\sample\Calculator::testDivide()
# Fatal error : Call to undefined method netbeans\sample\Calculator::divide()
# /Users/jubianchi/NetBeansProjects/nb-atoum-blog/tests/units/netbeans/sample/Calculator.php
not ok 2 - tests\units\netbeans\sample\Calculator::testAdd()
# Fatal error : Call to undefined method netbeans\sample\Calculator::add()
# /Users/jubianchi/NetBeansProjects/nb-atoum-blog/tests/units/netbeans/sample/Calculator.php
-------------------------------------------

With this output, I get "Both tests passed".
To reproduce this bug, you can add a unit test on a method which does not exists, or simply follow this: https://gist.github.com/jubianchi/3ef229b2a370fec609df/8931c25542f4b9d4d45c84d485cc12d6d045a49d#file-netbeans_atoum-md
When you get the first unit test ready, simple launch it and you'll get the bug.

Comment 5 Tomas Mysik 2013-09-03 07:07:17 UTC

Unfortunately, too late for 7.4. Will be fixed in the next version of NetBeans (or maybe in some patch of 7.4 if there will be any).

Thanks.

Comment 6 Tomas Mysik 2013-09-03 08:04:26 UTC

(In reply to jubianchi from comment #4)
> -------------------------------------------
> 1..2
> not ok 1 - tests\units\netbeans\sample\Calculator::testDivide()
> # Fatal error : Call to undefined method netbeans\sample\Calculator::divide()
> #
> /Users/jubianchi/NetBeansProjects/nb-atoum-blog/tests/units/netbeans/sample/
> Calculator.php
> not ok 2 - tests\units\netbeans\sample\Calculator::testAdd()
> # Fatal error : Call to undefined method netbeans\sample\Calculator::add()
> #
> /Users/jubianchi/NetBeansProjects/nb-atoum-blog/tests/units/netbeans/sample/
> Calculator.php
> -------------------------------------------

Weird, I have a unit test for it (it is the same as in the initial description of this issue). I will try to reproduce it following your steps...

Thanks.

Comment 7 Tomas Mysik 2013-09-03 08:17:32 UTC

Another issue found:

1..1
not ok 1 - tests\units\netbeans\sample\Calculator::testAdd()
# E_USER_ERROR : Tested class 'netbeans\sample\Calculator' does not exist for test class 'tests\units\netbeans\sample\Calculator'
# /home/gapon/NetBeansProjects/atoum-issue/tests/units/netbeans/sample/Calculator.php:


NetBeans says it is passing. Investigating...

Comment 8 Tomas Mysik 2013-09-03 08:20:37 UTC

(In reply to Tomas Mysik from comment #7)
> #
> /home/gapon/NetBeansProjects/atoum-issue/tests/units/netbeans/sample/
> Calculator.php:

BTW please notice - no line number but ":" is there... I will handle it as well.

Thanks.

Comment 9 jubianchi 2013-09-03 08:57:53 UTC

(In reply to Tomas Mysik from comment #8)
> (In reply to Tomas Mysik from comment #7)
> > #
> > /home/gapon/NetBeansProjects/atoum-issue/tests/units/netbeans/sample/
> > Calculator.php:
> 
> BTW please notice - no line number but ":" is there... I will handle it as
> well.
> 
> Thanks.

This issue should be resolved here : https://github.com/atoum/atoum/pull/253

You can test this by running the following commands after having your dependencies installed:

----------
cd vendor/atoum/atoum
git remote add jubianchi https://github.com/jubianchi/atoum.git
git fetch jubianchi
git checkout jubianchi/master
----------

After that, when running your test in Netbeans, the ':' should disappear if there is no line number.

Comment 10 Tomas Mysik 2013-09-03 09:06:57 UTC

And what about the empty message? Will it be fixed on the atoum's side as well? Or is a valid situation? I can easily handle it on NetBeans's side.

Thanks.

Comment 11 Tomas Mysik 2013-09-03 09:08:19 UTC

BTW the problem that ABORTED test is not shown is not in the PHP area but in the common testing UI - reported as issue #235443.

Thanks.

Comment 12 jubianchi 2013-09-03 09:15:47 UTC

(In reply to Tomas Mysik from comment #10)
> And what about the empty message? Will it be fixed on the atoum's side as
> well? Or is a valid situation? I can easily handle it on NetBeans's side.
> 
> Thanks.

The empty message is also fixed in the previously linker PR ;)

Comment 13 Tomas Mysik 2013-09-03 11:11:06 UTC

It should be fixed now - we use ERROR instead of ABORTED status so it is shown properly in the UI ;) Also, empty message is not a problem anymore. Please, test it, using PHP build 7023 [1] or newer.

Láďo, please consider this issue as a stopper for 7.4 - the fix is very simple (3 lines changed) and there are unit tests for this area.

http://hg.netbeans.org/web-main/rev/b8c2e852cc41

Thanks.
[1] http://bertram.netbeans.org:8080/job/PHP-build/7023/

Comment 14 Vladimir Riha 2013-09-03 12:34:46 UTC

Agree, fix is quite simple and it would be good to have it in 7.4. I'll verify it once it gets to regular trunk.

Adding keyword and raising to P1 per HR process instructions

Comment 15 jubianchi 2013-09-03 17:24:03 UTC

Works for me with:
* empty diagnostic message
* diagnostic message
* filepath without line number

Thanks a lot!

Comment 16 Quality Engineering 2013-09-04 01:44:16 UTC

Integrated into 'main-silver', will be available in build *201309040001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)

Changeset: http://hg.netbeans.org/main-silver/rev/b8c2e852cc41
User: Tomas Mysik <tmysik@netbeans.org>
Log: #235280 cont'd - Netbeans fails to parse TAP report if no line is reported in diagnostic

Comment 17 Tomas Mysik 2013-09-04 11:31:18 UTC

Adding Ondra to CC so he can review the fix. Thanks.

Comment 18 Ondrej Brejla 2013-09-04 11:33:12 UTC

Fix seems to be ok for me.

Comment 19 Vladimir Riha 2013-09-04 11:38:55 UTC

Looks OK to me as well, verified in trunk. Please continue with integration to release74 as described in [1]. Thank you


[1] http://wiki.netbeans.org/NetBeans_74_HighResistance

Product Version: NetBeans IDE Dev (Build 201309040001)
Java: 1.7.0_40; Java HotSpot(TM) Client VM 24.0-b55
Runtime: Java(TM) SE Runtime Environment 1.7.0_40-b39
System: Linux version 3.2.0-48-generic-pae running on i386; UTF-8; en_US (nb)

Comment 20 Tomas Mysik 2013-09-05 05:11:02 UTC

Transplanted to the release74 branch in the releases reporitory.

Thanks.

http://hg.netbeans.org/releases/rev/700c1942d6be