This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 208394 - convert existing DataLoaders and Factories to @DataObject.Registration
Summary: convert existing DataLoaders and Factories to @DataObject.Registration
Status: RESOLVED FIXED
Alias: None
Product: apisupport
Classification: Unclassified
Component: Refactoring (show other bugs)
Version: 7.2
Hardware: All All
: P3 normal (vote)
Assignee: Jesse Glick
URL: http://wiki.netbeans.org/DeclarativeR...
Keywords:
Depends on: 207960 207219 208670
Blocks: 210377 210388 210382 210391 211017 211019
  Show dependency tree
 
Reported: 2012-02-14 14:24 UTC by skygo
Modified: 2012-04-11 14:19 UTC (History)
1 user (show)

See Also:
Issue Type: TASK
Exception Reporter:


Attachments
DataObject Hinter (6.70 KB, patch)
2012-02-15 13:31 UTC, skygo
Details | Diff
DataObject Hinter (19.74 KB, patch)
2012-02-17 17:31 UTC, skygo
Details | Diff
DataObject Hinter (26.21 KB, patch)
2012-02-22 23:48 UTC, skygo
Details | Diff
Variations around layers.txt generate xml (123.77 KB, patch)
2012-02-29 23:49 UTC, skygo
Details | Diff
DataObject Hinter (24.28 KB, patch)
2012-03-03 20:13 UTC, skygo
Details | Diff
image module convert (9.84 KB, patch)
2012-03-03 20:28 UTC, skygo
Details | Diff
Variations around layers.txt generate xml (124.68 KB, patch)
2012-03-09 23:52 UTC, skygo
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description skygo 2012-02-14 14:24:25 UTC
This task is a following of issue #207219. This task can be affected to me.

Before beeing handled the DataObject.Registrations (plural form) should be merged with the main.


Basic Tips from Jesse:
-Yes please use @Messages in newly generated code.
-For updating existing code with Bundle.properties entries just refer to the
existing key. A separate hint exists to move the key into @Messages if desired.

LastBuild layer info is available at [1]. I create a wiki page [2] to help tracking things (human made)


[EB01]
Cosmetic or coding rules :p.
@DataObject.Registration vs @Registration

For me @DataObject.Registration is more readable than @Registration. Want to know what is preferable to use.
Hinter is doing with @Registration but I can rewrite to @DataObject.Registration

[EB02]
I can also look a to Actions under Loaders Folder.



---------
[1]
http://deadlock.netbeans.org/hudson/job/nbms-and-javadoc/lastSuccessfulBuild/artifact/nbbuild/build/generated/layers.txt

[2]
http://wiki.netbeans.org/DeclarativeRegistrationUsingAnnotationsDataObjectMigration
Comment 1 Jesse Glick 2012-02-14 16:43:30 UTC
For patches involving hints (apisupport.refactoring) please use apisupport/project in BZ.
Comment 2 Jesse Glick 2012-02-14 17:19:50 UTC
Is there a running patch for just the hint part? If so, please attach it here.
Comment 3 skygo 2012-02-14 17:55:22 UTC
I was working at the same time on the DataObject.Registrations 
and the Hinter compatible with DataObject.Registrations because without it it was not possible to convert the image module. 

As annotation was not yet in use, I keep the version 7.36 for the  DataObject.Registrations 

I try to go to subfolder but diff make difference for all the implementation I was doing this is why I put all in one in the following attachement for issue #207219

http://netbeans.org/bugzilla/attachment.cgi?id=115551
Comment 4 Jesse Glick 2012-02-14 20:08:46 UTC
(In reply to comment #3)
> I try to go to subfolder but diff make difference for all the implementation I
> was doing

A Mercurial patch is always against the whole repo, but using MQ you can keep a stack of separate yet interdependent patches.
Comment 5 skygo 2012-02-15 13:31:46 UTC
Created attachment 115753 [details]
DataObject Hinter

Hinter for DataObject Layer refactoring using @DataObject.Registration(s) issue #207219

Thanks Jesse :p
Comment 6 Jesse Glick 2012-02-15 19:21:01 UTC
Besides referring to DataObject.Registrations which does not yet exist (can I think just omit this clause in annotationsAvailable), the hint does not work at all. It tries to add an annotation to DataLoaderPool.factory; it needs to read the dataObjectClass instead (i.e. the parameter to findAndModifyDeclaration is wrong).

You also need to verify that the .instance file is really what you expected: instanceCreate=method:org.openide.loaders.DataLoaderPool.factory, mimeType matches what you inferred from the path, and all other attributes are ones you explicitly handle (position, iconBase, displayName).

And you need to read the shadows and separators in Loaders/$mimeType/Actions/ and create @ActionReference's to match.

Tested on o.apache.tools.ant.module/src/org/apache/tools/ant/module/resources/AntModuleLayer.xml as an example.
Comment 7 skygo 2012-02-15 22:18:18 UTC
(In reply to comment #6)

I will rewrite the hinter to check more case and extend this scope  (I was testing on DataLoader subclass but not DataObject.Factory subclass). 

It will take some times to me before reposting a more complete (serious) hinter.

> 
> And you need to read the shadows and separators in Loaders/$mimeType/Actions/
> and create @ActionReference's to match.

Thanks to make it clear to me. I will look for that as part of the hinter. 

I have a related question to this, should the hinter allow refactoring of all the actions in one time or one by one?


Note for me (Eric); actionsContext() in DataLoader :
$mimeType is not always related to the factory mimeType ( "Loaders/image/png-gif-jpeg-bmp/Actions/";)
Comment 8 Jesse Glick 2012-02-15 23:52:44 UTC
(In reply to comment #7)
> I was testing on DataLoader subclass but not DataObject.Factory

Ah, I see. Yes, the hint should handle either variant.

> should the hinter allow refactoring of all
> the actions in one time or one by one?

I would say all at once - if you agree to convert the loader registration then any associated actions should be converted at the same time.
Comment 9 skygo 2012-02-16 22:34:55 UTC
> I would say all at once - if you agree to convert the loader registration then
> any associated actions should be converted at the same time.


I totally agree to the refactoring of actions all at once. I would nuance only the total conversion of loader registration at once.

I would prefer the following two steps which can be done in any order:
 
 #1 hint for Loader/mime/factories/file
    ! DataLoader and DataObject.Factory check +all parameter 
    ( if "vintage" SystemFileSystem.localizingBundle present or not handled parameters then fallback hints to tell user what is wrong)

 #2 hint for Loader/dummymime/actions folder 
    ! look for DataLoader and DataObject.Factory in the Classpath.SOURCE (AFAIK current module source) then propose a hint per file found to allow user to choose where he wants actions to be registred.


It's for me a way to deals with actions when "mimetype" differs.
Comment 10 skygo 2012-02-17 17:31:23 UTC
Created attachment 115893 [details]
DataObject Hinter


-Compatible with DataObject.Factory subclass and DataObject
-Actions are refactored with maybe a too complex way or a not safe way to process

note:
shadow-hidden in  not processed me (Eric) 
should now be not able to do multiple DataObjectRegistrations until in 207219 accepted.
Comment 11 Jesse Glick 2012-02-20 20:40:18 UTC
(In reply to comment #9)
> I would prefer the following two steps which can be done in any order:

So long as the resulting registrations are in fact independent; see my comments in bug #207219 comment #15, not responded to yet as far as I can tell.

(JG11 probably needs to be fixed in the New File wizard, too.)

> It's for me a way to deals with actions when "mimetype" differs.

I guess this can be useful for cases like the image loader (right?).

> if "vintage" SystemFileSystem.localizingBundle present

This is already handled by a separate hint, no need to worry about it (except to not enable your hint when this unknown attribute is present).

(In reply to comment #10)
> multiple DataObjectRegistrations until [] #207219 [is] accepted

That issue is marked FIXED so it is off the radar; if you have more changes to do like @Registrations probably you need to file separate blocking issues.
Comment 12 Jesse Glick 2012-02-20 21:21:44 UTC
Still too buggy to commit and clean up the details later; have not managed to get it to work fully correctly on even one module:

1. On o.apache.tools.ant.module, incorrectly deals with RunTargetAction, already registered on the loader from an @ActionRegistration in the same module.

2. On the html module, screws up in various ways. Processes both loader definitions correctly. For the actions, in text/html it leaves behind html-project-separator-3.instance for no clear reason, and puts the action refs on XhtmlLoader rather than HtmlLoader as it should have; the remaining Actions folder in the layer now shows an error that the hint cannot find the associated loader; the Actions folder for text/xhtml is not processed at all, and the hint cannot find its loader.

3. On the Actions folder in the layer for cnd.source, I got four hints all with the same label: "Convert registration to Java annotation". I picked the first one and then accepted all subsequent hints, which seems to have worked (placed action refs more or less arbitrarily on CDataLoader). Similar on j2ee.ddloaders.

4. Tried to use it on the actions folder in languages.yaml but got a somewhat cryptic message about GsfDataLoader, I guess due to use of @LanguageRegistration on YamlLanguage.

5. Trying on the xsl module, the actions folder in text/xml got dumped inappropriately on XSLDataLoader. I could translate the XSLDataLoader registration, but trying to process its annotations got me the "No visible dataObject [sic]" error.

Also the new getFileSystem method is unnecessary since ctx.file() already gives you access, either via file.getFileSystem(), or by just calling getParent() one or more times.

And note that calling fileSystem.getRoot().getChildren(true) and then checking each for getPath().startsWith(LOADERS_FOLDER) is wasteful; rather look up fileSystem.findResource(LOADERS_FOLDER) and if not null then iterate it.
Comment 13 skygo 2012-02-20 23:45:57 UTC
Thanks for feedback,

>This is already handled by a separate hint, no need to worry about it (except
>to not enable your hint when this unknown attribute is present). 

I would prefer a hint just telling the user "something may be available if you change this old bundle usage".


comment #12

I was having shadow-hidden extension with ant module and I was not able to process further investigation. My process is not the best I think. I try with only two modules and stop at the first blocking.

For all point I will look when back with my dev computer.


Just a few comments 

2. 
3. For Actions folder:

my basic idea was to ad more info in tip text to have something like this:
"Convert registration to Java annotation in HtmlLoader" 
"Convert registration to Java annotation in XhtmlLoader" 

this allow to replace action registration on a chosen loader restricted to module (for now).

  j2eedlloaders conduct me to a complex concern with the factories. No position for a same mimetype. May be useful to hint on "unsafe" position( may also be cross module).   
<file name="web-DDWeb25DataLoader.instance"/>
<file name="ejb-EjbJarDataLoader.instance"/>
<file name="ejb-EjbJar30DataLoader.instance"/>
<file name="client-ClientDataLoader.instance"/>
<file name="app-EarDataLoader.instance"/>
 

4. I do a separation on wikipage because I was not sure who should take care on language registration.


I will also take care of coding practice you mention.
Comment 14 Jesse Glick 2012-02-21 20:33:59 UTC
(In reply to comment #13)
> I would prefer a hint just telling the user "something may be available if you
> change this old bundle usage".

All other hints expect a particular set of file attributes and just show "unrecognized attribute" or similar if something else is encountered; the hint to switch to displayName/iconBase from SystemFileSystem.localizingBundle/icon seems to appear first anyway.

> I was having shadow-hidden extension with ant module and I was not able to
> process further investigation.

The problem has something to do with META-INF/generated-layer.xml containing the *.shadow for the action defined in the same module; probably need to explicitly ignore entries from the generated layer. LayerHints.Prov.run never passes these to hinters, but your hint is going out and looking for _other_ layer entries which is the problem.

You can try comparing (Utilities.compareObjects) file.getAttribute("layers") with the same on the possible sibling entry. Hinter.Context.delete will not work perfectly - will leave behind an empty folder in layer.xml - but not a big deal.

> my basic idea was to ad more info in tip text to have something like this:
> "Convert registration to Java annotation in HtmlLoader" 
> "Convert registration to Java annotation in XhtmlLoader" 
> 
> this allow to replace action registration on a chosen loader restricted to
> module (for now).

Makes sense, but the hints should be limited to those loaders which actually bind to the same MIME type - except in the case that the "MIME type" used for the action path is some synthetic string not used on any loader (as in CND).

> j2eedlloaders conduct me to a complex concern with the factories. No position
> for a same mimetype.

Sorry, do not understand.

> I was not sure who should take care on language registration.

About GSF/CSL, you mean. Just skip these for now; this system is deprecated anyway.
Comment 15 Jesse Glick 2012-02-22 13:37:12 UTC
Linking to wiki page with summary. Odd organization; division by clusters would be more meaningful.

For actually converting registrations, can do some directly myself (platform, ide, java, apisupport clusters); other clusters probably need to be filed as patches in appropriate components. In particular, enterprise cluster has a lot of registrations, mostly for config files, which would get tested as part of the web-main Hudson job; and cnd cluster also has its own testing system.
Comment 16 skygo 2012-02-22 23:48:45 UTC
Created attachment 116041 [details]
DataObject Hinter


better text in hinter 
 (try to restric to proper mimetype)

take care of existing registred actions
less bug in separator check
merge @actionreferences if one allready exist
Comment 17 skygo 2012-02-23 00:10:55 UTC
(In reply to comment #15)
> Linking to wiki page with summary. Odd organization; division by clusters would
> be more meaningful.
> 

Done but cluster may not be in proper order


>Current Hinter.

Was able to use with ant and xsl. Not perfect yet :D but less bug.


for the j2eeddloader stuff. I anticipate a potential issue.

>>Current layer:
#1Loaders/text/x-dd-application5.0/Factories/EarDataLoader.instance
#4Loaders/text/x-dd-application5.0/Factories/ClientDataLoader.instance

#2Loaders/text/x-dd-application6.0/Factories/EarDataLoader.instance
#3Loaders/text/x-dd-application6.0/Factories/ClientDataLoader.instance


>> Potential revwrite with annotation according to number 

@DORs(
@DOR("text/x-dd-application5.0"),
@DOR("text/x-dd-application6.0"),
)
class EarDataLoader ....

@DORs(
@DOR("text/x-dd-application6.0"),
@DOR("text/x-dd-application5.0"),
)
class ClientDataLoader ....

>> This may conduct to an different ordering of loader due to lack of position attribute; Is this critical?

If yes I will modify hinter to set position or propose thinks to user.
Comment 18 skygo 2012-02-27 18:19:36 UTC
Seems that #208670 will be integrated soon so I will be able to refine Hinter.

[EB03]
This is a borderline remarks to make life easier in converting to annotation. 
  
1. 
I tried to do a xml version of the generated/layers.txt.
 -short tag name and attribute name reduce output size. 
 -id for module to factorize and reduce output size.

2.
I associated xslt intended to be included in xml (client side processing) 
 reproduce a htmlized output of the current layers.txt and enable a jquery filtering and some colorization.

3.
I need now to generate XML for clusters

Final goal is to be able to generate wiki formated text to do copy paste of more up to date information in the wikipage.

>>But I don't know if I can propose that because:<<

(Xml + Xsl + css . jquery) is about the same size as (layers.txt)
=> having the 2 at same time is too much space
=> layers.txt may be input of other build process
=> grep, awk on unix system works often better with text
Comment 19 skygo 2012-02-29 23:49:20 UTC
Created attachment 116222 [details]
Variations around layers.txt generate xml

This is a work in progress on a layers report in xml. Currently , it hooks in a ugly way the current LayerIndex.

If it's possible to have a feedback on the resulting web page.
If usefull I will make it cleaner.
# use Jquery and quicksearch
# not ready yet for wiki 

(only work with IE + firefox on hard drive, if on web server then google chrome work security reason).
Comment 20 skygo 2012-03-03 20:13:12 UTC
Created attachment 116316 [details]
DataObject Hinter

Now support both DO.Registrations and DO.Registration
Comment 21 skygo 2012-03-03 20:28:07 UTC
Created attachment 116317 [details]
image module convert

This patch is for image. 

It's more for having feedback on how to do convertion better for other cluster.

---
@Registration(displayName = ".....", mimeType = "image/jpeg")}
ext.addMimeType("image/jpeg");  

is it fine to introduce private static String to hold the mimetype ?

---
also adding the @override to the dataobject or loader to make it the more up to date we can ?
Comment 22 skygo 2012-03-09 23:52:06 UTC
Created attachment 116530 [details]
Variations around layers.txt generate xml
Comment 23 Jesse Glick 2012-03-21 17:47:57 UTC
(In reply to comment #22)
> Created attachment 116530 [details]
> Variations around layers.txt generate xml

More or less off topic for this issue. Anyway we intentionally produce simply-formatted text files for nbbuild/build/generated/ - compact, easy to grep (*), and produce readable diffs sent to api-changes@netbeans.org (**).


(*) For example:

ant index-layer-paths && egrep 'Loaders/.+/.+/Factories/[^ ]' nbbuild/build/generated/layers.txt | fgrep -v '@ '


(**) For which we want every line to either be self-describing, or to be nicely categorized by unindented lines beginning with an all-caps keyword like "MODULE": diff -u --show-function-line='^[A-Z][A-Z]'
Comment 24 skygo 2012-03-21 18:02:02 UTC
ok thanks for feedback on the layers file
(sorry but grep on windows is something hard)
I will only keep it in my local repository for the automatic generation of the wikipage.
Comment 25 Jesse Glick 2012-03-21 19:16:40 UTC
Applied as core-main #4add40c00611.

Some problems remain (mainly I18N violations) which I will try to address myself since it is too time-consuming and cumbersome to do a back-and-forth review: core-main #fb77dbd938fd.

Applied myself to image module to see it in action, plus various manual cleanups of the result to follow good code style (esp. using constants): core-main #adc9ddbaebd7

Also in o.apache.tools.ant.module: core-main #8e6995e77b92
Comment 26 Jesse Glick 2012-03-21 19:23:53 UTC
Conversions of other modules can be done at any time; probably best filed as issues for the owners of those areas. Do not expect every assignee to be responsive, though, since this is not a high priority; maybe file an issue first offering to supply a patch, but delay doing the actual work until the assignee agrees to review & apply.
Comment 27 skygo 2012-03-23 22:26:32 UTC
Ok will setup this in one week.

To Jesse
I also may help on the cluster you manage depending on schedule you have in mind.
Comment 28 Quality Engineering 2012-03-24 11:00:29 UTC
Integrated into 'main-golden', will be available in build *201203240400* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/fb77dbd938fd
User: Jesse Glick <jglick@netbeans.org>
Log: #208394 cont'd: warnings, I18N.
Comment 29 Quality Engineering 2012-03-25 09:45:16 UTC
Integrated into 'main-golden', will be available in build *201203250401* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main-golden/rev/fb77dbd938fd
User: Jesse Glick <jglick@netbeans.org>
Log: #208394 cont'd: warnings, I18N.