This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 201654 - Support more test directories
Summary: Support more test directories
Status: VERIFIED FIXED
Alias: None
Product: php
Classification: Unclassified
Component: Project (show other bugs)
Version: 7.3
Hardware: All All
: P2 normal with 7 votes (vote)
Assignee: Tomas Mysik
URL:
Keywords:
: 206258 (view as bug list)
Depends on:
Blocks:
 
Reported: 2011-09-05 22:20 UTC by pulzaraider
Modified: 2013-10-23 13:13 UTC (History)
5 users (show)

See Also:
Issue Type: ENHANCEMENT
Exception Reporter:


Attachments
Symfony 2.0.1 PHP basic project (5.17 MB, application/octet-stream)
2011-09-05 22:20 UTC, pulzaraider
Details

Note You need to log in before you can comment on or make changes to this bug.
Description pulzaraider 2011-09-05 22:20:05 UTC
Created attachment 110411 [details]
Symfony 2.0.1 PHP basic project

This ehancement can be associated with Symfony 2 support.

Problem:
Symfony 2 is the future of PHP frameworks and i think it will be the most popular framework. It consists of separate bundles. Each bundle has its own configuration, own classes, css, img, js... and also tests.
Tests of each bundle are stored in "Tests" directory.

The problem is that this differs from how Netbeans search tests now. Symfony has no global directory in which are tests situated.

Now (v. 7.0.1) Netbeans can run own testsuites, XML configuration, custom bootstrap files what is really great and running all Symfony tests can be done even now. But single tests in Symfony 2 (or other framework that has tests in separate directory structure) can not run (with "Test" button) and also Code Coverage feature doesn't work.

Goal:
In Netbeans project configuration should be some input box to specify the name (or regex for name) of directory in which tests are stored. 
If this directory is specified (let's say, the name of the dir will be "Tests"), Netbeans should search for test in this order:
1. search for test in global "FileTest.php" directory (this is current feature of Netbeans)
2. If file is stored in this directory structure /a/b/c/File.php:
- Netbeans will search for test file FileTest.php in directory /a/b/c/Tests. 
- If nothing was found - search for test in dir /a/b/Tests/c (full path: /a/b/Tests/c/FileTest.php)
If nothing was found - search for test in dir /a/Tests/b/c (full path: /a/Tests/b/c/FileTest.php) and so on till the main project directory is reached.

I don't know how Code Coverage is developt in Netbeans now and if it's using phpUnit generated xml or own functions, but if Netbeans will find the correct test file the Code Coverage should have also work.

Important note: Test files can extened not only PHPUnit_Framework_TestCase class. For example in Symfony2 there are common functional tests like this:

class DemoControllerTest extends WebTestCase
{
...
}

And finally class WebTestCase extends PHPUnit_Framework_TestCase. So when Netbeans will search for test of some file it has to keep in mind that test can extends some middle class(es) which are extended at the end from PHPUnit_Framework_TestCase.

I'm including basic symfony2 project with netbeans configuration. Tests can be found in "Tests" directories in "vendor" or "src" dirs.
Comment 1 pulzaraider 2011-10-19 05:50:56 UTC
Or instead of algorithm I described would be better to implement testsuite locations in phpunit xml configuration

For Symfony 2 phpunit.xml.dist looks like this:
<?xml version="1.0" encoding="UTF-8"?>

<!-- http://www.phpunit.de/manual/current/en/appendixes.configuration.html -->
<phpunit
<!-- ... -->

    <testsuites>
        <testsuite name="Project Test Suite">
            <directory>../src/*/*Bundle/Tests</directory>
            <directory>../src/*/Bundle/*Bundle/Tests</directory>
        </testsuite>
    </testsuites>

    <filter>
        <whitelist>
            <directory>../src</directory>
            <exclude>
                <directory>../src/*/*Bundle/Resources</directory>
                <directory>../src/*/*Bundle/Tests</directory>
                <directory>../src/*/Bundle/*Bundle/Resources</directory>
                <directory>../src/*/Bundle/*Bundle/Tests</directory>
            </exclude>
        </whitelist>
    </filter>

</phpunit>
Comment 2 Tomas Mysik 2013-02-15 08:37:13 UTC
*** Bug 206258 has been marked as a duplicate of this bug. ***
Comment 3 soyel 2013-04-22 21:39:08 UTC
There's a workaround to make netbeans and phpunit works with Symfony2 tests but it is not really practical.

You need to create your own Test Suite as described here : http://forum.symfony-project.org/viewtopic.php?f=23&t=33543

The php test suite file is basically a duplication of the provided app/phpunit.xml.dist file which does not work in netbeans (it keeps asking for a "test directory" even though the phpunit.xml.dist describe the test directories structure).

There is still some issues with that workaround:

- It can not run test from base class / file.
- Code Coverage will not work.
- You can only run all tests (which can be really time consuming).
- You need to create your own test suite every time you start a new project.

This functionality is really lacking for a proper Symfony2 support.
Comment 4 Tomas Mysik 2013-07-24 08:15:42 UTC
More test roots could be added. What should work:

- running all tests (action Run Tests on Project node)
- running individual test files
- navigate to test/source file

But there are several problems here:

- Create Test
How to decide in which test root create a new test file? The only option right now seems to be to show a dialog and let user select proper test root. Please notice that the PHPUnit support does not know wnything about Symfony etc. project so it cannot select the proper test root easily. Perhaps it could preselect the "closest" test root.

- PHPUnit properties
PHPUnit properties (in Project Properties like Bootstrap etc.) cannot be set per test root; in other words, there will be possible to set only one bootstrap etc. per project.

Anything else you can think of, please? Thanks.
Comment 5 Tomas Mysik 2013-09-18 07:26:57 UTC
Done. Milane, verify it please. Thanks.
Comment 6 AdamJessop 2013-10-21 14:37:53 UTC
Did this not make it into the 7.4 release? Any ideas if this would be available as a hotfix?
Comment 7 Tomas Mysik 2013-10-22 06:51:59 UTC
(In reply to AdamJessop from comment #6)
> Did this not make it into the 7.4 release?

This is part of NetBeans after NB 7.4.

> Any ideas if this would be
> available as a hotfix?

Definitely not, the change is too complex, sorry. But feel free to use development version of NetBeans but please note that it can be broken.

Thanks.
Comment 8 Tomas Mysik 2013-10-23 11:49:00 UTC
(In reply to Tomas Mysik from comment #5)
> Done. Milane, verify it please. Thanks.

Milane, Láďo, please verify this feature also on Windows.

Thanks.
Comment 9 mmolda 2013-10-23 12:12:04 UTC
Tomasi, I have created a new project with multiple test directories and I tested all three scenarios.

1. Running all tests (Right click on Project node->Test) - all tests executed OK
2. Running individual test files from different test roots - tests within selected file were executed OK
3. Navigate to Test/Source file (from different source files into different test roots) - works OK

Verified. Thanks.

Product Version: NetBeans IDE Dev (Build 201310230001)
Java: 1.7.0_45; Java HotSpot(TM) Client VM 24.45-b08
Runtime: Java(TM) SE Runtime Environment 1.7.0_45-b18
System: Windows 7 version 6.1 running on x86; Cp1250; en_US (nb)
Comment 10 mmolda 2013-10-23 12:26:03 UTC
I also tested generating tests in different directories by switching their order in Project Properties->Testing.

I have tested PHPUnit, Atoum and Nette/Tester. Everything works fine. Great work :)
Comment 11 Tomas Mysik 2013-10-23 13:13:00 UTC
Thanks a lot for verification, Milane! :)