This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 142760 - Startup regression: org.netbeans.modules.php.project.PhpMimeResolver
Summary: Startup regression: org.netbeans.modules.php.project.PhpMimeResolver
Status: VERIFIED FIXED
Alias: None
Product: platform
Classification: Unclassified
Component: Filesystems (show other bugs)
Version: 6.x
Hardware: All All
: P2 blocker (vote)
Assignee: Jiri Skrivanek
URL: http://wiki.netbeans.org/FitnessViaWh...
Keywords: API_REVIEW_FAST, PERFORMANCE, REGRESSION, TEST
: 118977 (view as bug list)
Depends on:
Blocks: 199927
  Show dependency tree
 
Reported: 2008-08-04 09:40 UTC by Alexander Kouznetsov
Modified: 2012-01-12 00:39 UTC (History)
9 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Stacktraces (9.82 KB, text/plain)
2008-08-04 09:41 UTC, Alexander Kouznetsov
Details
Changes in openide.filesystems. (42.07 KB, text/plain)
2008-11-03 18:33 UTC, Jiri Skrivanek
Details
Changes in php.project, cnd, ruby, languages.sh. (58.58 KB, text/plain)
2008-11-03 18:34 UTC, Jiri Skrivanek
Details
Changes in openide.filesystems - version2 (69.38 KB, text/plain)
2008-11-10 15:40 UTC, Jiri Skrivanek
Details
Changes in php.project, cnd, ruby, languages.sh - version2. (68.35 KB, text/plain)
2008-11-10 15:41 UTC, Jiri Skrivanek
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Kouznetsov 2008-08-04 09:40:22 UTC
Performance test reports there is a NetBeans startup regression which is caused by the loading of the following classes:

org.netbeans.modules.php.project.PhpMimeResolver
org.netbeans.modules.php.project.PhpMimeResolver$MutableInteger
org.netbeans.modules.php.project.PhpMimeResolver$Sign

Please don't load these classes during NetBeans startup.

Additional comment from jtulach: "I gave them an exception. Maybe report bug to fix this for post-6.5."
Comment 1 Alexander Kouznetsov 2008-08-04 09:41:02 UTC
Created attachment 66450 [details]
Stacktraces
Comment 2 Tomas Mysik 2008-08-04 11:12:55 UTC
We will look at it after NB 6.5.
Comment 3 rmatous 2008-08-18 16:02:12 UTC
P4 - we need these classes now, keep it just as a reminder - will be evaluated later
Comment 4 rmatous 2008-10-16 15:30:44 UTC
There is no way how to achieve this functionality by declarative mime-resolver. Reassigning  to core for deeper eval. to
come up with any idea.
Comment 5 Jiri Skrivanek 2008-11-03 18:32:45 UTC
I would like to ask for review of my changes in declarative MIME resolvers. I added two new elements to MIME resolver
DTD. Element 'name' is intended for file name checking (issue 118977). For example this 

    <name name="makefile" substring="true"/>

matches files like makefile, Makefile, gnumakefile, etc.
Element 'pattern' inspired by PhpMimeResolver is intended for checking string pattern in file content. For example this

    <pattern value="&lt;HTML&gt;" range="255" ignorecase="true">
        <pattern value="&lt;?php" range="4000"/>
    </pattern>

matches files with '<HTML>' in first 255 bytes and '<?php' in first 4000 bytes (checked only if outer condition is
fulfilled).
I also added a logic that <ext name=""/> matches files without extension. Using these new elements I rewrote procedural
ShellScriptResolver, RubyMimeResolver, CndSniffyMIMEResolver and PhpMimeResolver. Only CndMIMEResolver is remaining in
standard distribution. It is possible to rewrite it but there is some UI in Options dialog connected to it.
I will attach separate diff for openide.filesystems and separate for other modules (php.project, cnd, ruby,
languages.sh). Comments welcome. Thanks.
Comment 6 Jiri Skrivanek 2008-11-03 18:33:29 UTC
Created attachment 73152 [details]
Changes in openide.filesystems.
Comment 7 Jiri Skrivanek 2008-11-03 18:34:45 UTC
Created attachment 73153 [details]
Changes in php.project, cnd, ruby, languages.sh.
Comment 8 Jaroslav Tulach 2008-11-04 08:11:45 UTC
Nice change.

Y01 Too little documentation. I really like your description in this issue, however most of the people will only read 
javadoc and for them you have just two lines in apichanges. That is not enough, imho. Is not there a special page 
about mime resolving that needs to be updated?

Y02 Versioning. Update the version number and make all modules that use the new functionality request this version of 
org.openide.filesystems

Y03 Right now one can match name as substring, would not it be better to offer match by regexp?

Y04 The "#!" needs to be at beginning. Then there can be spaces and then "/bin/sh". The proposed change checks for 
different situation. Maybe one could specify "^#! */bin/sh" regexp?

Y05 The php resolver reads 255 and then 4000 bytes. That is 4255, which is more than 4192. How much bytes are read by 
the infrastructure? If 4192, then this resolver causes two disk touches and that is not good, imho.
Comment 9 Jiri Skrivanek 2008-11-10 15:39:39 UTC
I will attach new patches. Except below mentioned changes I fixed functionality of the <exit/> element. Now it escapes
resolving if conditions are fulfilled. No next <file> element is checked.

> Y01 Too little documentation.

I added topic to wiki (http://wiki.netbeans.org/DevFaqMIMEResolver) and updated existing documentation at
org.openide.filesystems.doc-files. Examples are in comments of elements in DTD documentation (linked from wiki).

> Y02 Versioning

Done.

> Y03 Right now one can match name as substring, would not it be better to offer match by regexp?

Regex is time expensive. At the time I was not able to rewrite only two hopefully corner cases in ruby. Files with
"#!/bin/rubystuff/bin/ksh" and "#! /usr/b\nin/ruby" header are mistakenly resolved as ruby files. Tor could comment
whether it is a blocker for removal of procedural RubyMimeResolver.

> Y04 Maybe one could specify "^#! */bin/sh" regexp?

It is more or less fulfilled by the following construct:

        <pattern value="#!" range="2">
            <pattern value="/bin/bash" range="40"/>
        </pattern>


> Y05 The php resolver reads 255 and then 4000 bytes. That is 4255, which is more than 4192.

A range attribute of the pattern element always means number of bytes from the beginning. I checked it and there was a
mistake in MIMESupport.CachedInputStream which I fixed. So now only 4000 bytes are read if there are two embedded
pattern elements. I also added an assertion into MIMEResolverImpl (if a resolver wants to read more that 4000 bytes).
Comment 10 Jiri Skrivanek 2008-11-10 15:40:47 UTC
Created attachment 73582 [details]
Changes in openide.filesystems - version2
Comment 11 Jiri Skrivanek 2008-11-10 15:41:18 UTC
Created attachment 73583 [details]
Changes in php.project, cnd, ruby, languages.sh - version2.
Comment 12 Vladimir Voskresensky 2008-11-13 01:00:19 UTC
Seems, I don't see makefile declarative resolver in your changes, only in examples.

+ do not see where /cnd/src/org/netbeans/modules/cnd/resources/mime-resolver-content-based.xml
catch lines like:
#![any number of spaces]/bin/bash
in fact, this is responsibility of sh module, so you can review shells declared by cnd and remove it from cnd completely
by adding into sh module (cnd declared shell resolvers, only because sh module didn't)
Comment 13 Vladimir Voskresensky 2008-11-13 01:09:22 UTC
I can comment on CndMIMEResolver.
As I understand you handle declarative resolvers and fill some internal infrastructure with it.
We need:
- programmatic way to get all extensions associated with interested mime-type
- programmatic way to add new extension for interested mime-type + this must be persisted between sessions
In this case we can easily stay with our UI and will remember in own preferences default extension.

Btw, having "default extension" attribute provided by your infrastructure would be useful for us as well. Default
extension is used when new file of correspondent mime-type is created (try to create new C++ file)
Comment 14 Jiri Skrivanek 2008-11-13 10:35:32 UTC
> VV: Seems, I don't see makefile declarative resolver in your changes, only in examples.

It is in CndMIMEResolver and I didn't touch it at all.

> VV: #![any number of spaces]/bin/bash

Yes, it is covered by nested <pattern> element.

> VV: you can review shells declared by cnd and remove it from cnd completely.

OK, I will move shell script resolvers to languages.sh module.

> VV: - programmatic way to get all extensions associated with interested mime-type

Theoretically possible. But I doubt it is necessary to offer to choose extension while creating a new file. It could
offer default file name like newmain.cpp with possibility to edit it.

> VV: - programmatic way to add new extension for interested mime-type.

To add a new extension is possible in Tools > Options > Miscellaneous > Files. I don't think there should be two ways of
doing the same thing.

> VV: Btw, having "default extension" attribute provided by your infrastructure would be useful for us as well.

It doesn't make sense for MIME resolvers to define something like "default extension". It should be handled in cnd if
you need it.
Comment 15 Jiri Skrivanek 2008-11-14 09:50:17 UTC
I will integrate proposed changes. I filed issue 153204 and issue 153202 to separatelly track changes necessary for
replacing CndMIMEResolver.
Comment 16 Jiri Skrivanek 2008-11-18 09:48:50 UTC
Fixed.
http://hg.netbeans.org/core-main/rev/1c98e8c87f81
Comment 17 Jesse Glick 2008-11-18 16:20:54 UTC
[JG01] You cannot add new elements to an existing DTD. You need to make a separate DTD with a new public ID, e.g.
"-//NetBeans//DTD MIME Resolver 1.1//EN", and use this public ID on the resolvers you are changing. The new DTD needs to
be registered into the system catalog (currently in o.n.core/src/org/netbeans/core/resources/mf-layer.xml, though
perhaps these entries would better be moved to openide.filesystems), and should probably also be published at

http://www.netbeans.org/source/browse/www/www/dtds/

to match the documented remote system ID.
Comment 18 Quality Engineering 2008-11-19 16:43:51 UTC
Integrated into 'main-golden', will be available in build *200811191401* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/1c98e8c87f81
User: Jiri Skrivanek <jskrivanek@netbeans.org>
Log: #142760 - Added new elements to declarative MIME resolver DTD. The name element to check file names and the pattern element to check file content. Procedural resolvers ShellScriptResolver, RubyMimeResolver, CndSniffyMIMEResolver and PhpMimeResolver were rewritten using these new features.
Comment 19 Jiri Skrivanek 2008-11-20 15:04:03 UTC
> [JG01]
Fixed.
http://hg.netbeans.org/core-main/rev/a769d343214c
Comment 20 Quality Engineering 2008-11-22 04:46:27 UTC
Integrated into 'main-golden', will be available in build *200811220201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/a769d343214c
User: Jiri Skrivanek <jskrivanek@netbeans.org>
Log: #142760 - MIME resolver DTD version increased.
Comment 21 Alexander Kouznetsov 2008-11-25 09:33:22 UTC
Verified with 081125
Comment 22 Jesse Glick 2012-01-12 00:34:36 UTC
*** Bug 118977 has been marked as a duplicate of this bug. ***