This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Summary: | I18N - Change scope of l10n.list file | ||
---|---|---|---|
Product: | www | Reporter: | rbalada <rbalada> |
Component: | Builds & Repositories | Assignee: | rbalada <rbalada> |
Status: | RESOLVED FIXED | ||
Severity: | blocker | CC: | jchalupa, jkovalsky, kfrank, mzlamal, rnovak |
Priority: | P1 | Keywords: | I18N |
Version: | 6.x | ||
Hardware: | All | ||
OS: | All | ||
Issue Type: | ENHANCEMENT | Exception Reporter: | |
Bug Depends on: | |||
Bug Blocks: | 71609, 93080 | ||
Attachments: |
Patch for nbbuild/antsrc/org/netbeans/nbbuild/L10nTask.java
Commit log |
Description
rbalada
2007-04-15 11:42:09 UTC
Created attachment 40926 [details]
Patch for nbbuild/antsrc/org/netbeans/nbbuild/L10nTask.java
is this still viewed as enhancement only and at p3 ? I am guessing that means it does not need to be done for nb6 ? Its getting close to time in nb6 when l10n kits will be needed which means that if there are changes to l10n.lists layout and approach, it needs to happen soon as to getting feedback from developers, being implemented and being tested out, and how it might imact building of ml builds or langpacks. that is, l10n team is requesting existence of l10n kits even earlier than the normal date for when the kits need to be accurate and complete, which is normally at HR date - which could also impact when this might need to be done. ken.frank@sun.com Changing priority to P1, issue type to FEATURE and milestone to 6.0M10. I have got majority of the work done. Preliminary estimates are that first kits could be available on Wednesday or Thursday this week (6/6-6/7). Because this change is major and nontrivial, I'll let the build run in dual mode for some time to verify the kits are same. Marking STARTED Thanks for starting this. How many l10n kits will there be and what will be granularity of them ? I know its not a direct topic of this issue but it will be helpful to still have some separate kits, even though its all one product. Perhaps at granularity of the downloadable components seen on installer page ? ken.frank@sun.com Could you note some rules for developers about this new approach to l10n.list so they can start to rewrite/reorganize their current l10n.list files per module ? Then this could be placed on wiki and shared with dev teams, nb-tech, etc ? One question - since uc modules will be localized separately from product modules and on a different schedule, can/should instructions state that files of stable uc modules not be put into any l10n.list ? There is separate discussion about need for separate kit for stable uc modules, and a separate l10n.list.uc is not the solution that is acceptable, so a subteam is discussing how dev can indicate about stable uc files to translate. But having the uc files in the product kits can result in some problems. ken.frank@sun.com Ken, basic introduction of the change. Now we have one l10n.list file per each CVS module. After this change we would have one global l10n.list and bunch of small l10n.list files for local exceptions The change of scope is actually scope from several points of view. Current solution: We have a list of toplevel CVS modules. Each CVS module has got one (and only one) l10n.list file in the root. Number of CVS modules exactly matches number of l10n.list files. Each l10n.list file holds wildcard masks for all necessary submodules within CVS module's directory structure. New solution: List of submodules matching exact build configuration of NB IDE (or component/pack/UC list). There's new global nb_all/nbbuild/l10n-global.list file which holds generic masks covering 80-90% localizable files within these (sub)modules. In case that some submodule needs exception, you put *new* l10n.list file under NB IDE module's structure, at exactly the same level where it has it's 'nbproject' directory. For example for autoupdate/services module the nbproject directory is 'autoupdate/services/nbproject' and the local override l10n.list would be 'autoupdate/services/l10n.list' file. Such a file is used only at time when we need L10Nkit for the 'autoupdate/services' module. This particular module is specific by having 'libsrc' subdirectory, which contains some localizable stuff. In this case this local overriding l10n.list file may contain something like: <!-- sample autoupdate/services/l10n.list file --> read global ${l10n-module}/libsrc/**/Bundle.properties ${l10n-module}/libsrc/**/resources/*.gif ${l10n-module}/src/**/*.png <!-- end of sample --> Let me read it for you. The line "read global" is a command to load generic masks from 'nb_all/nbbuild/l10n-global.list'. Loading global generic masks is optional, mainly when you have very little amount of exceptions an generic masks are safe to use. Lines starting '${l10n-module}'. This property reference would be evaluated at runtime and would be replaced with 'autoupdate/services' string. As a result we get masks which are valid from the nb_all root. Why we do all of these? Let me summarize pluses first: - unlimited number of L10N kit configurations based on build content configuration system allowing us to have extra kits for IDE, VWP, SOA, UML, Update Center etc. - developer cares only about modules in his charter, his change almost cannot affect others - developer cares only when global generic masks don't fit module's localization needs - if the global l10n.list would be written properly, then majority of modules won't need specific local overriding l10n.list file Minuses: - in case you put local overriding l10n.list file somewhere in upper level module and the file would have been written too liberally, then the kit may include data also for deeper level submodules (i.e. autoupdate/l10n.list vs. autoupdate/services/l10n.list - any general minuses when using global mask and local overrides Rudolf, thanks for the info/spec; some followup questions/comments: - seems dev would need info to know if/when they would need to create/maintain an override file I guess just some explanation of what typical kinds of file names/types is used by l10n-global.list file could be provided to them. - what about the excludes syntax - this is used by various modules usually on specific files, sometimes with wildcards - would this be a case for the override kind of l10n.list file ? does it allow exclude syntax as do the current l10n.lists ? - is this correct - the override file goes at the nbproject dir level, but there might be >1 such file for a given top level cvs dir, since there might be >1 nbproject dirs under that module ? - from example about sample override l10n.list file ${l10n-module}/libsrc/**/Bundle.properties ${l10n-module}/libsrc/**/resources/*.gif ${l10n-module}/src/**/*.png --> but shouldn't these patterns for Bundle.properties and png and gif be part of the global file so that dev would not need to use these ? - as to pluses about having extra kits be easier to do - it sounds good, but for UC kit; how does the build system/RE know what parts of what top level cvs modules have files only meant for uc kit ? (and uc files should not be in the other kits) seems that only dev knows about that ? Thanks - Ken Hi Ken, here go my comments > - seems dev would need info to know if/when they would > need to create/maintain an override file > I guess just some explanation of what typical kinds of file > names/types is used by l10n-global.list file could be provided to them. I'll document it at wiki.netbeans.org. > - what about the excludes syntax ... I made no changes to syntax. However I did some bugfixing affecting syntax and hopefully leading to intended behavior. Now it is possible to use '#' character for commenting out lines at any position in the l10n.list file, or the reference "read global" is now correctly interpreted each time it is referred. It's also possible to run the l10nkit build on Windows box and it will produce same result as on unix box. > - is this correct - the override file goes at the nbproject dir level, > but there might be >1 such file for a given top level cvs dir, > since there might be >1 nbproject dirs under that module ? First of all there's no recursive search for another l10n.list file. It's always interpreted locally in current context/directory. The l10n-global.list has been written in such a way that it does not allow easily descent into submodule (with one exception where there's a clash between modules core and core/javahelp, causing a need for local override in core) > - from example about sample override l10n.list file > > ${l10n-module}/libsrc/**/Bundle.properties > ${l10n-module}/libsrc/**/resources/*.gif > ${l10n-module}/src/**/*.png > > --> but shouldn't these patterns for Bundle.properties > and png and gif be part of the global file so that dev > would not need to use these ? This is actually tricky question :-). The global file must represent just reasonable majority of cases, not all possible cases. We have to find a balance between less than accurate and more than accurate kit (in terms of included data compared to localization need). In this case the 'libsrc' subdirectory exists in about 6-8 modules (out of ~280 modules/=codenamebases/ constructing NB IDE). Regarding the pattern "src/**/*.png" the global file contains less liberal pattern "src/**/resources/**/*.png". Adding all *.png files is not always good option (not all of them are need for localization process. I decided to not put 'libsrc' into global pattern file. It must not be the best decision, but helps keep the global file short and not too much liberal. > - as to pluses about having extra kits be easier to do - > it sounds good, but for UC kit; how does the > build system/RE know what parts of what top level > cvs modules have files only meant for uc kit ? > (and uc files should not be in the other kits) > > seems that only dev knows about that ? Build Engineering always gets list of modules for Update Center enough in advance. This allows us to build those modules and stage these modules on Staging Update Center as well as start preparing L10N kit for the list of modules. When we get the list of modules, we process it down to submodules so as a result the list reads something like: <!-- sample list --> ant/docs,ant/freeform/samples,apisupport/apidocs,ide/projectimport <!-- end of sample list --> This list is exactly build configuration in a form which is used for building module binaries as well for L10Bkit build. This configuration is then put into nbbuild/build.properties file. This property file can hold number of configurations. The number is limited just by our ability to maintain those configurations. For example there's configuration called 'daily-alpha-nbms'. This configuration holds list of modules which should be daily refreshed on so called "Development Update Center". Building l10nkit for this configuration is simple. Just run "ant -Dmoduleconfig=daily-alpha-nbms l10n-kit" in the nbbuild module. If you're not sure you've got all necessary sources in your working directory just add "checkout " target before the "l10n-kit" string and you should get all sources first. Status update: I've committed all necessary changes to run L10Nkit build in dual mode. Starting tomorrow morning we are going to have two 'same' L10N kits on our internal storage. Please note that those kits are not 100% same, because current l10n.list files do not match product content. At this moment I reached a level where all the differences look reasonable to me and I would need help with shooting offending differences. More info tomorrow. comments to Rudolf's comments in next to last section below start here and start with lines that start with &&& Hi Ken, here go my comments > - seems dev would need info to know if/when they would > need to create/maintain an override file > I guess just some explanation of what typical kinds of file > names/types is used by l10n-global.list file could be provided to them. I'll document it at wiki.netbeans.org. > - what about the excludes syntax ... I made no changes to syntax. However I did some bugfixing affecting syntax and hopefully leading to intended behavior. Now it is possible to use '#' character for commenting out lines at any position in the l10n.list file, or the reference "read global" is now correctly interpreted each time it is referred. It's also possible to run the l10nkit build on Windows box and it will produce same result as on unix box. > - is this correct - the override file goes at the nbproject dir level, > but there might be >1 such file for a given top level cvs dir, > since there might be >1 nbproject dirs under that module ? First of all there's no recursive search for another l10n.list file. It's always interpreted locally in current context/directory. The l10n-global.list has been written in such a way that it does not allow easily descent into submodule (with one exception where there's a clash between modules core and core/javahelp, causing a need for local override in core) &&& suggest documenting this part also for developers > - from example about sample override l10n.list file > > ${l10n-module}/libsrc/**/Bundle.properties > ${l10n-module}/libsrc/**/resources/*.gif > ${l10n-module}/src/**/*.png > > --> but shouldn't these patterns for Bundle.properties > and png and gif be part of the global file so that dev > would not need to use these ? This is actually tricky question :-). The global file must represent just reasonable majority of cases, not all possible cases. We have to find a balance between less than accurate and more than accurate kit (in terms of included data compared to localization need). In this case the 'libsrc' subdirectory exists in about 6-8 modules (out of ~280 modules/=codenamebases/ constructing NB IDE). Regarding the pattern "src/**/*.png" the global file contains less liberal pattern "src/**/resources/**/*.png". Adding all *.png files is not always good option (not all of them are need for localization process. I decided to not put 'libsrc' into global pattern file. It must not be the best decision, but helps keep the global file short and not too much liberal. &&& I agree, it is a tricky situation. As to png or filenames in general, I thought that the global file would just be a list of filenames/suffixes so that all of them in the cvs src would be obtained. But I guess its not so easy since cvs src has test files, and other non product files, which might have files with that suffix. As to image files, usually they are in javahelp and all are usually in l10n.list files from before; as to others, if they are not included, I guess it can be part of dev review of global l10n.list and rules about them needing to do override file. And I think its impt for them to do really the same review of src as they did before when they created their own l10n.list -- to compare the global list and see which dirs they still need to either add files to or exclude them. I think it depends on complexity of global list - can you add here or send me a copy of it ? > - as to pluses about having extra kits be easier to do - > it sounds good, but for UC kit; how does the > build system/RE know what parts of what top level > cvs modules have files only meant for uc kit ? > (and uc files should not be in the other kits) > > seems that only dev knows about that ? Build Engineering always gets list of modules for Update Center enough in advance. This allows us to build those modules and stage these modules on Staging Update Center as well as start preparing L10N kit for the list of modules. When we get the list of modules, we process it down to submodules so as a result the list reads something like: <!-- sample list --> ant/docs,ant/freeform/samples,apisupport/apidocs,ide/projectimport <!-- end of sample list --> This list is exactly build configuration in a form which is used for building module binaries as well for L10Bkit build. This configuration is then put into nbbuild/build.properties file. This property file can hold number of configurations. The number is limited just by our ability to maintain those configurations. For example there's configuration called 'daily-alpha-nbms'. This configuration holds list of modules which should be daily refreshed on so called "Development Update Center". Building l10nkit for this configuration is simple. Just run "ant -Dmoduleconfig=daily-alpha-nbms l10n-kit" in the nbbuild module. If you're not sure you've got all necessary sources in your working directory just add "checkout " target before the "l10n-kit" string and you should get all sources first. &&& that's good news as it seems it wil solve the need for separate uc kits without needing separate uc l10n.list but actually the kit that is needed is for stable uc modules only; not all uc modules. and what is on stable can change from time to time. will this solution be able to provide a kit for that ? But am assuming even for this case dev will need to review the global list and see if any exeption files are needed ? I would recommend **/*src/**/Bundle.properties to catch libsrc and similar dir names. Created attachment 43634 [details]
Commit log
Major infrastructure pushed into production whats the plan to communicate to dev about these changes and the parts they need to do ? (special exclude/include lists). Even though there is new process and global list, dev still owns it. I don't think there is a category that such a task can be filed on this - what's your suggestion on how to get this on dev plate and tracked (the dev part) ? since there is a new i18n readiness milestone (if you and dev approve) that has early l10n kit by beta 2 - but this means not just the already going kits you are publishing but that dev has done at least some work on their lists -- but it does not have to be 100% accurate. whereas the usual l10n kit complete milestone means that kit is 100% accurate and that all modules are covered by RE scripts as to being included in kits and that stable uc modules are not in product kit product kit has no stable uc or other uc only module files (these last 2 are part of uc kit issue71609 --> should another issue be filed as to having kit include all nb6 tarfiles (they dont need to be correct, just that all the tars are there -- or is this complete as per this issue ? ken.frank@sun.com |