This might be more of an issue with the IDE's cvs support than with its jsp compiling--I'm not clear on that yet.

I'm not so familiar with the IDE's cvs functionality, nor do I know of a "practice" cvs repository I could commit on as
a test.

bakarirum: as a start, could you attach your project? Thanks.

I've verified that the problem at least does not occur on a newly created web project (i.e., with no cvs involved). That
is, I created the web project, selected project properties > Categories > Build > Compiling > Test compile all JSP files
during builds, saved the project, exited the IDE, and executed ant dist and ant compile-jsps from the command line
without a problem.

I get the same problem whether I am using CVS or SVN.  If I create a brand new project, I only get the problem sporadically.

Ok.  Here are some more specific reproduction steps.

   Create NB Web Application project.

   Check the "Test compile all JSP files during builds" under project
   properties-->Categories-->Build-->Compiling

   Import and commit to CVS or SVN through IDE.

   Use command-line CVS or SVN and check out project (into a different folder).

   Type "ANT DIST" or "ANT COMPILE-JSPS" from command-line.

The key is to create the project in NetBeans and check flag and then import into a repository (which I have learned it
does not matter which source control you use).  Then checkout the project into another directory that is not associated
with your NetBeans projects using the command-line tool.  Then when you try to run ant to build the project, it no
longer works because it gives "NoClassDefFoundError" as below.

bakarirum: are you able to attach your project (the one that fails)?

Created attachment 74762 [details]
Test Case - Should be able to unzip and try to run ant on it.

Please let me know if you need anything else.

I can resolve the problem by doing the following:

- Open the project in netbeans 6.5.
- Right-click on the project and do resolve missing server problem, selecting a glassfish installation.
- attempt ant dist or ant compile-jsps, etc. outside netbeans at the command line.

The problem is that the ant scripts need to know where your appserver installation is. Marking invalid--bakarirum,
please reopen if this does not resolve the problem.

You have defeated the whole purpose of a "headless" build.  The goal is to take the project away from NetBeans and
compile it from the command-line using ant.  When I originally created the project my app server was Tomcat.  Therefore,
when you pull up the application in NetBeans, OF COURSE, you will get a missing server problem.  So no, this is not a
solution nor work around.

Thank you.

I want to make another point.  If you have a project created in NetBeans and you go the directory for that project
(where NetBeans accesses it), then the build works fine.  The problems stems from using a source controlled project that
is a NetBeans project and that project is checked out into a different directory and is never pulled into NetBeans. 
This is because there is the "private" folder which is not imported into source control.  Thus when you try to build
your project using Ant away from the IDE, it fails.  So I will assume that "mbohm" tried to run the attached test case
from the command-line and got the error.  Then he opened that project in NetBeans and noticed the "missing server"
problem and resolve that.  But when he did that, he also created the "private" directory structure as this project is
now under NetBeans' control.  Thus, the problem I am referring to goes away.  Can you verify/deny this, mbohm?

bakarirum, when I marked the issue invalid, I allowed for the possibility that I did not fully understand your
requirements. That is why I wrote "please reopen if this does not resolve the problem."

You are correct that when I opened the project in netbeans 6.5, the IDE created the private directory structure. In this
way, various build properties needed by the build scripts were established.

Another way to populate those build properties is to create a properties file containing the needed properties and build
like so:

ant -Duser.properties.file=<path_to_property_file> dist

In fact, the private.properties actually contains a property named user.properties.file, establishing the location of a
build.properties. For instance, on my system the value is C:\\Documents and
Settings\\mbohm\\.netbeans\\6.5\\build.properties

The individual developers on your team could create a user properties file that is unique to their system containing
needed properties.

For instance, I can do the following successfully on the TempWeb project:

- Manually change line 388 of build-impl.xml to the following. (It appears this is necessary because the project was
created on an earlier version of the IDE.)
            <classpath path="${java.home}/../lib/tools.jar:${jspctask.classpath}:${jspcompilation.classpath}"/>

- Create a user.properties file that is a sibling of build.xml, like so. (There are references to jars under C:/Program
Files/NetBeans 6.5, but these could be located elsewhere on your system.)

j2ee.platform.classpath=C:\\programs\\tomcat\\lib\\annotations-api.jar:C:\\programs\\tomcat\\lib\\catalina-ant.jar:C:\\programs\\tomcat\\lib\\catalina-ha.jar:C:\\programs\\tomcat\\lib\\catalina-tribes.jar:C:\\programs\\tomcat\\lib\\catalina.jar:C:\\programs\\tomcat\\lib\\derbyclient.jar:C:\\programs\\tomcat\\lib\\el-api.jar:C:\\programs\\tomcat\\lib\\jasper-el.jar:C:\\programs\\tomcat\\lib\\jasper.jar:C:\\programs\\tomcat\\lib\\jsp-api.jar:C:\\programs\\tomcat\\lib\\servlet-api.jar:C:\\programs\\tomcat\\lib\\tomcat-coyote.jar:C:\\programs\\tomcat\\lib\\tomcat-dbcp.jar:C:\\programs\\tomcat\\lib\\tomcat-i18n-es.jar:C:\\programs\\tomcat\\lib\\tomcat-i18n-fr.jar:C:\\programs\\tomcat\\lib\\tomcat-i18n-ja.jar:C:\\programs\\tomcat\\bin\\tomcat-juli.jar

jspc.classpath=C:\\Program Files\\NetBeans 6.5\\java2\\ant\\lib\\ant.jar:${j2ee.platform.classpath}
jspctask.classpath=C:\\Program Files\\NetBeans 6.5\\enterprise5\\ant\\extra\\jspcompile.jar

- At the command line:
ant clean
ant -Duser.properties.file=user.properties dist

Let me know if this is helpful.

I am not trying to argue.  I am just trying to provide the exact steps to reproduce the problem.  Maintaining a separate
properties file for each project defeats the purpose.  If NetBeans is maintaining the build files, then we do not want
to maintain something else just to get it to build outside of NetBeans.  If that is the case, then we might as well use
Free Form Projects.

We are using NetBeans 6.0.  The reference to the jspctask.classpath property to a jspcompile.jar file is not in my install.

So, why create a build structure that should run under Ant that does not run under Ant unless it is told how?  Something
is not computing here.

Mr/Mrs.  Mbohm:

   I was able to take my ant code (which I am calling from java using the ant library and not from the command-line) and
add all the properties from the %netbeansuserdir%/build.properties file to the project.  When I did this, the problem
with compile-jsps target went away.  OK, so you can call this a workaround if you want.  However, this is not a "good"
solution.  If I want to take that project to a whole new operating system and/or computer and try to build it, it would
fail because the other location does not have a "NetBeans USER DIR" in which I can point to/use.  So my goal with this
request is to make ALL the builds autonomous regardless of project settings.  We know that by doing this we cannot
depend on the "classpath" of the project because the "private" folder is not under source control.  However, we are
handling that manually by adding our own classpath by reading the dependent projects/libraries in the project
properties.  Please reassign if that is needed, but I feel this is a defect in the NetBeans build file.  From my
experience (which is not the most experience), build files should be autonomous.  Thanks for your time on this request.

Hi, my first name is Matt. :-)

As you know, the purpose of the private folder, and the netbeans userdir, is to specify paths and other settings that
are specific to the local machine. (The private folder for such settings at the project level and the userdir for
settings at the IDE level.) Some of the jars needed by the build scripts come with the IDE, and others come with the
application server. The build scripts expect that the installation location of these will vary from machine to machine.

If you want to do away with machine-specific metadata, you will have to ensure somehow that the needed jars (and similar
resources) are always in some predictable place on the local machine, and adjust the build scripts accordingly. Because
you likely do not want to include all those jars in the project itself, you would likely have to standardize across your
development team the locations of the application server installation, the IDE installation, and the netbeans userdir.

Let me know if this makes sense.

I do have one more comment to make.

My goal is allow a NetBeans project's build file run outside the IDE.  I understand that third party libraries and
project dependencies, etc cannot be handled.  But I am not worried about the classpath for building the project.  I am
concerned with the fact that the project does not even find the needed properties to run properly.  In the example, you
have to either have the project in a NetBeans project space or you have to include a proprietary properties file
(build.properties) from %netbeansuserdir%.  There is no way for you to build a generic build script that includes all
the dependent projects and libraries so that the project's build is autonomous.  The classpath should be left up the
user of the project.

> I am concerned with the fact that the project does not even find the needed properties to run properly.
Indeed, the build scripts need certain machine-specific settings, located in private.properties and build.properties. To
my knowledge, there is no getting around the need for machine-specific settings (except putting all dependencies in the
project itself). If you don't like the machine-specific settings being located in private.properties and
build.properties, I suppose you could move the ones needed for the targets you use into a user.properties. Perhaps you
are looking for a template for such a user.properties that would include only mandatory properties for particular targets?

>There is no way for you to build a generic build script that includes all the dependent projects and libraries so that
the project's build is autonomous.
Do you mean you are considering putting all the dependencies in the project so that you can do away with
machine-specific settings? In any case, you can, of course, customize the build scripts as needed--though I do
understand that you would ideally prefer an out-of-the-box solution. I'm just not clear on what solution you want.
Perhaps you could customize your project accordingly and attach the result for consideration for a new feature or
enhancement.

I would simply like to create a web project (with zero dependencies) and copy it to a machine without NetBeans and run
the build scripts.  That is what I want to do.  The dependencies (project, machine specific, etc) should not affect
anything.  Those items should be (which is the current way) left up to the user (me) to make sure are set up properly
and are available for a remote build, etc.  The problem I am identifying is a flaw in the build logic.  The build logic
requires a NetBeans only specific class (org/netbeans/modules/web/project/ant/JspC) to compile a JSP file.  This class
reference is added properly to the build sequence when the user.properties from %netbeansuserdir% is included.  I am not
asking for any reconfiguring of the way machine specific settings and/or project dependencies are handled.  I simply
believe the class reference property in the user.properties file that causes the compile-jsps target to work properly
needs to be included in the default build-impl.xml.  Therefore a basic web application with the "Test compile all JSP
files during builds" flag is set will compile without the need to have NetBeans installed.

I guess this might lead to the simple question of why use the NetBeans specific class of
(org/netbeans/modules/web/project/ant/JspC) instead of the standard (Java or Ant) JSP compiler?

>"I simply
believe the class reference property in the user.properties file that causes the compile-jsps target to work properly
needs to be included in the default build-impl.xml."

I presume you mean the jspctask.classpath property defined in build.properties. Here is the property as defined in
build.properties on my system:

jspctask.classpath=C:\\Program Files\\NetBeans 6.5\\enterprise5\\ant\\extra\\jspcompile.jar

If we were to define that property in build-impl.xml instead of build.properties, what would the value of the property
be? The location of the jspcompile.jar is machine-specific. The only way we could define it in a way that was not
machine-specific would be to include jspcompile.jar in the project (i.e., every web project).

>I guess this might lead to the simple question of why use the NetBeans specific class of
(org/netbeans/modules/web/project/ant/JspC) instead of the standard (Java or Ant) JSP compiler?

org.netbeans.modules.web.project.ant.JspC extends org.apache.jasper.JspC. It tweaks the arguments supplied to it, sets
some properties, and invokes the parent logic. So besides jspcompile.jar, a jar containing org.apache.jasper.JspC, such
as jasper.jar located in the lib folder of tomcat 6, is also needed. Again, the location of jasper.jar is
machine-specific; it is located wherever the application server is installed. We could only define that jar's location
in a way that was not machine-specific if we included the jasper.jar (or a different jar containing
org.apache.jasper.JspC) in the project (i.e., every web project).

bakarirum: am I still somehow misunderstanding?

>org.netbeans.modules.web.project.ant.JspC extends org.apache.jasper.JspC. It tweaks the arguments supplied to it, sets
>some properties, and invokes the parent logic. 

Couldn't this be done using antcall (to set the properties) and/or a macro definition wrapping Tomcat's JspC task?

>So besides jspcompile.jar, a jar containing org.apache.jasper.JspC, such
>as jasper.jar located in the lib folder of tomcat 6, is also needed. Again, the location of jasper.jar is
>machine-specific; it is located wherever the application server is installed. We could only define that jar's location
>in a way that was not machine-specific if we included the jasper.jar (or a different jar containing
>org.apache.jasper.JspC) in the project (i.e., every web project).

Couldn't it work more like the copyLibs task does? i.e. in -init-taskdefs check for jspctask.classpath and if not set
fail a message along the lines of

The jspctask.classpath property is not set up.
This property must point to 
jasper.jar file which is part
of the Tomcat installation and is usually located at 
<tomcat_installation>/lib/whatever folder.
Either open the project in the IDE and make sure Tomcat plugin is
installed or setup the property manually. For example like this:
 ant -Djspctask.classpath=a/path/to/jasper.jar

Much better than a NoClassDefFoundError.

This last suggestion should be suffice.  If we are told what is missing and how to resolve it, then it would be a better
resolution.  I understand that "compiling" jsps is dependent upon the JSPC compiler, but the build should not be
dependent upon a NetBeans only class.  So updating the error message and allowing the use of just the Jasper compiler
would be enough.

Please reevaluate

I can reproduce using the following steps:

1. Create a web project, while creating check "Use Dedicated Folder for Storing Libraries" and "Use Dedicated Folder for
Storing Server JAR files".
2. Select "Test compile all JSP files during builds"
3. Delete nbproject/private directory
4. On the command line, run "ant dist"

The problem is that enterprise6/ant/extra/jspcompile.jar from the NB distribution is not found (not only jasper.jar from
the server). The solution should be the same as with copylibs.jar, i.e. create a library with jspcompile.jar under the
shared libraries. 

This may not be enough though, we'll see what else will need to be done. Reassigning to web/project for now.

Here is temporary workaround for the usecase Petr Jiricka gave (this workaround needs to be performed once per each
"sharable libraries folder"):

#1) create folder <your_shared_lib_folder>/jspc

#2) copy <your_nb_installation>/enterprise6/ant/extra/jspcompile.jar to <your_shared_lib_folder>/jspc/ (NOTE: depending
on version of NB you are running the base directory can have different version number, eg. enterprise5 instead of 6; the
same applies below for java3 directory)

#3) copy <your_nb_installation>/enterprise6/modules/ext/glassfish-jspparser-2.0.jar to <your_shared_lib_folder>/jspc/

#4) copy <your_nb_installation>/java3/ant/lib/ant.jar to <your_shared_lib_folder>/jspc/

#5) add following records to <your_shared_lib_folder>/nblibraries.properties and update property on the last line to
point to your glassfish server library property (property which should be defined in
<your_shared_lib_folder>/nblibraries.properties depending on GF server you are running):

  jspctask.classpath=\
    ${base}/jspc/jspcompile.jar
  jspcompilation.classpath=\
    ${base}/jspc/glassfish-jspparser-2.0.jar:\
    ${base}/jspc/ant.jar:\
    ${libs.GlassFish_XXXX.classpath}

That should enable JSP compilation on headless server.

The work around is great and everything, but it is just that; a work around.  It does not correct the flaw of the build
script.  I worked around this problem a long time ago, so a work around is not the solution.

re. workaround - I plan to fix this issue for final release of NB68. Workaround is here just for record for other people
who might have the same problem.

So I guess it hasn't been fixed, as the bug is still open and NB 6.8 is already out?

It is vitally important to allow "headless" builds, because how are we supposed to work with Eclipse users in a team otherwise? Another example would be the project.properties file, which is auto-generated by Netbeans and thus "untouchable" by Eclipse users. But how are they supposed to add additional dependencies? Any NB-generated files must have a "hook" in build.xml, so non-NB users can work with NB-users.

We have not upgrade to NB6.8 so I have not tested to see if this is corrected.  Since it is still an open issue can Matthew, Petr, or David give an update to this issue?

Thank you.

No, this issue has not been fixed yet. Sorry for any inconvenience.

Fixed as 3b043cf14b0e

Two new libraries are introduced: JSP Compiler and JSP Compilation. JSP compiler library is used to compile JSP files into Java. JSP Compilation library is required on classpath in order to successfully compile Java classes generated from JSP. When JSP Compilation is enabled in project properties the project will use these two libraries to perform JSP compilation. If project is sharable then these two libraries will be automatically copied into libraries folder to support JSP compilation in headless mode.

I did thorough testing myself but I would like to ask QA and originator of the issue to test it when the build is available. Thanks!

Integrated into 'main-golden', will be available in build *201003250201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/3b043cf14b0e
User: David Konecny <dkonecny@netbeans.org>
Log: #154275 -  Problem with NetBeans Headless Builds

the exact problem is still occurring to me in both NetBeans 6.9.1 and 7.0 beta

(In reply to comment #31)
> the exact problem is still occurring to me in both NetBeans 6.9.1 and 7.0 beta

Did you try removing/renaming the ~/.netbeans directory on your system before installing a current version of netbeans.

(In reply to comment #31)
> the exact problem is still occurring to me in both NetBeans 6.9.1 and 7.0 beta

Could you please provide information on how to reproduce the problem? What exactly have you done? Is the problem reproducible on fresh new project? The more info the better. Thanks.

phill84 has not provided requested info. re-closing the issue.