This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
Please see link below for docs: http://wiki.netbeans.org/DLightToolkit
I am not sure about category/subcategory.
[Y1] Why don't you expose the generic interfaces (DataCollector, DataStorage, Indicator, Visualizer, DataProvider) as a public API? Their definitions seem to be independent on D-Light or dtrace at all. If we get the API right we might be able to unify data extraction/collection/visualization in D-Light, NB Profiler and any other 3rd party tool (eg. BTrace). [Y2] Is H2 database really necessary? Wouldn't JavaDB do?
Here are the answers: 1) We are definitely plan to expose our API, but It seems to me it is too early yo do this in this very first D-Light Toolkit release. 2) H2 database is really lightweight and now we are in the process of legal clearance to be able to use it in NetBeans. We have a version which uses JavaDB but H2 is more appropriate for us in the terms of performance and SQL extenstions it supports.
Are you aware of the database policy at http://sac.eng.sun.com/cgi-bin/bp.cgi?NAME=dbpolicy.bp If you want to use H2, you need to send an email to db_tech@sun.com and discuss an exemption. But before you do that: have you talked with the Java DB team about your issues and see if we can get them resolved? The fewer databases we are shipping with and trying to use, the better. Have you considered the longer-term issues of support and sustaining? The Java DB team is part of Sun and can give your needs priority. How are you going to fix bugs/issues in a timely manner in the H2 code base? The Java DB people to contact are Masood Mortazavi, the Java DB manager, and Rick Hillegas, the tech lead for Java DB.
David, thanks for your comment, I was not aware about db policy. As for JavaDB: 1. We have an implementation with JavaDB and for the first time before we will get decision from database technology group we will use JavaDb implementation in D-Light toolkit 2. We had some performance measurement and on the base of it we have chosen H2 database. 3. H2 database has some special features which give us some benefits: Java functions can be used in SELECT queries for example. 4. I will contact JavaDB team as you suggested
Team, Do you have any other comments/questions about D-Light Toolkit? Can we moving forward with this review? As I mentioned in my previous comment we will put into the NetBeans version of D-Light Toolkit which uses JavaDB before we have solved all problems with H2 database. Thanks, Maria
I was expecting to find answer to my questions at http://wiki.netbeans.org/DLightToolkitDesign, but some of them were not answered. Can you please modify the/a wiki page to include discussion of following topics? Y01 I have not found any information about the # of new modules, their target clusters, their dependencies. Y02 What external libraries will this project add to NetBeans? Y03 Will the project include native libraries? Y04 Can you provide expected list of public/friend packages, please? Y05 I do not understand the dependency on H2 database. I'd like to see explicit "go" to include it from all involved parties before you integrate. Btw. I would expect "before work" and "during implementation" questions from following list http://openide.netbeans.org/tutorial/questions.html to be answered where applicable.
Jaroslav, I will add info requested on the wiki and will generate arch.xml files Thanks, Maria
Jaroslav, I just wanted to say that according to technical review steps described here http://openide.netbeans.org/tutorial/review-steps.html and requesting for fast review I was sure that I do not need to answer Architecture Questions. Was I wrong? Thanks, Maria
Updated info with answers and modules info on Wiki http://wiki.netbeans.org/DLightToolkitDesign#section-DLightToolkitDesign-AnsweringQuestions
OK, thanks for the information. I'll pick the most important item to comment: Re. Y01 "target cluster will be IDE". I am not pleased to find out that we want to put 11 new, solaris dependant modules, into ideX cluster. I guess the reason is "no better handy cluster to use which would be shared between php, and cnd". But that is not good reason, because it also has significant drawbacks: 1. the ideX cluster is part of "small IDE" (java se, ruby) 2. it is infrastructure cluster (no really big UI) 3. it is included in Linux distributions (needs to be completely buildable from sources) 4. the ideX cluster is enabled by default in Ergonomics IDE mode (needs to be lightweight) Due to these problems I have strong feeling that it is not the right place for D-Light modules. At most, create one module with API and put it into the ideX cluster. Other parts of the system can use it to communicate with each other. But the D-Light UI, functionality and implementation shall be in some other cluster (cnd, its own). I understand that this is hard to discuss over email or in IZ. That is why I am suggesting you (I changed the review keyword to API_REVIEW as Step 3 from http://openide.netbeans.org/tutorial/review-steps.html advices) to schedule standard review, invite few reviewers and together agree what to do. I am ready to participate in such discussion in January 2009.
Please see my comments: >>Y01 "target cluster will be IDE". I am not pleased to find out that we want to put 11 new, solaris dependant >>modules, into ideX cluster. I guess the reason is "no better handy cluster to use which would be shared between php, >>and cnd". But that is not good reason, because it also has significant drawbacks: >>1. the ideX cluster is part of "small IDE" (java se, ruby) IDE cluster is selected as NetBeans Technical Council had suggested it. What can you suggest? >>2. it is infrastructure cluster (no really big UI) D-Light Toolkit doesn't contain UI visible to user >>3. it is included in Linux distributions (needs to be completely buildable from sources) It will be built from the sources (native part is really small) . We have some support for Linux in D-Light Toolkit. >>4. the ideX cluster is enabled by default in Ergonomics IDE mode (needs to be lightweight) >>Due to these problems I have strong feeling that it is not the right place for D-Light modules. At most, create one >>module with API and put it into the ideX cluster. Other parts of the system can use it to communicate with each other. I've got your point, it looks like D-Light Core module joined with NativeExecutionSupport can be such module. But where to put all other parts? DtraceDataCollector, CLIODataCollector, implementation of DataStorage? >>>>But the D-Light UI, functionality and implementation shall be in some other cluster (cnd, its own). Again, D-Light Toolkit doesn't have visible UI for the users. It has only some ready-to-use implementation which can be used by other modules which will build observability tool on the top of D-Light Toolkit (cnd, php). What do you mean by functionality and implementation here? D-Light Toolkit provide default implementation for DtraceDataCollector, default implementation of DataStorage, some implementation of DataProvider, target and run management. Do you think it should be moved out D-Light Toolkit? I understood your point not to put all these in ideX cluster, but which cluster will we put all these parts which will be used by developers which will build there tools on the base of D-Light Toolkit? I will schedule standard review at January 2009.
I would like to add that we will have remote support in D-Light Toolkit (which is listed in the features list)
See my comments: >>>>4. the ideX cluster is enabled by default in Ergonomics IDE mode (needs to be lightweight) >>>>Due to these problems I have strong feeling that it is not the right place for D-Light modules. At most, create one >>>>module with API and put it into the ideX cluster. Other parts of the system can use it to communicate with each other. >> >> I've got your point, it looks like D-Light Core module joined with NativeExecutionSupport can be such module. >>But where to put all other parts? DtraceDataCollector, CLIODataCollector, implementation of DataStorage? Why wouldn't you use a separate cluster for the purely DTrace related functionality? >>>>>>But the D-Light UI, functionality and implementation shall be in some other cluster (cnd, its own). >> >>Again, D-Light Toolkit doesn't have visible UI for the users. It has only some ready-to-use implementation which can >>be According to the wiki page visualizers are going to be included in the product - that sounds like a UI to me. >>used by other modules which will build observability tool on the top of D-Light Toolkit (cnd, php). >>What do you mean by functionality and implementation here? D-Light Toolkit provide default implementation for >>DtraceDataCollector, default implementation of DataStorage, some implementation of DataProvider, target and run >>management. Do you think it should be moved out D-Light Toolkit? I understood your point not to put all these in ideX Well, the implementation should definitely stay in the D-Light Toolkit. But the API, or abstract interfaces definitions, are not tightly bound to the D-Light or DTrace (at least it seems to be so). They should be taken out of the D-Light Toolkit and form a base for a more generic performance data collection and processing API. And this lightweight API would definitely belong to the IDE cluster. From my point of view there is no need to increase the clutter in the performance tool area in NetBeans - we have NB Profiler, VisualVM and now we are going to have the third monitoring, observation and performance management tool - of course each of the with it's own infrastructure that does the same in slight variation. I feel that this might be the time to unify the approaches to wisely use our resources. If you'd be interested I'm ready to participate in the review process as well. >>cluster, but which cluster will we put all these parts which will be used by developers which will build there tools >>on >>the base of D-Light Toolkit?
I think we would prefer to create own cluster for D-Light Toolkit and not to put anything for now in IDE or any other existing cluster. It would be really great to unify approaches for existing profiling tools in NetBeans but we have really limited time frame before NetBeans 7.0 release. So I would suggest to create separate cluster for D-Light Toolkit at least for NetBeans 7.0
Y11 Having new dedicated cluster for D-Light functionality seems reasonable for 7.0 timeframe Y12 It is desirable to have some "profiler ui" abstract module in ideX cluster shared between profilerX and D-Light: For example we want one "Attach Profiler" action, don't we?
Do you have any information from your conversations with the Java DB team? Were they able to resolve your issues with Java DB?
about "profile ui" and "Attach Profiler" action. For Gizmo (C/C++) D-Light will be running when Run Project action is invoked, and I believe the same will be done for Photon(Php). So there is no plan to use "Attach profiler" action in both projects which will use D-Light Toolkit (at least for NetBeans 7.0). There is functionality in D-Light Toolkit which allows to attach collectors to the process. It is supposed that all UI actions such as "Attach Profiler" are implemented outside D-Light Toolkit.
David, about JavaDB: We had email conversation with Rick Hillegas. We have informed him that our use case assumes many INSERTs that should be executed as fast as possible, and occasional SELECTs that are not so time critical. Experiments showed that in this scenario H2 is around 2 times faster than Java DB. This is the main reason for choosing H2 not JavaDB. He had advised us try to tune JavaDB: To use PreparedStatement with ? parameters (we did it) and using the following links: 1) Olav Sandsta's Java One talk: http://developers.sun.com/learning/javaoneonline/j1sessn.jsp?sessn=TS-45170&yr=2007&track=3 2) The "Performance tips and tricks" section of the Derby Tuning Guide: http://db.apache.org/derby/docs/10.4/tuning/ We will try to tune JavaDB but if we will not achieve results which are acceptable for us we would like to use H2 database as performance is really critical here. Thanks, Maria
[YA1] "D-Light will be running when Run Project action is invoked, and I believe the same will be done for Photon(Php)." - does this mean that there will be no possibility to run a project (C++, PHP) without D-Light?
Well, actually I think I was not quite right here, I mean for C/C++ projects you are right, Gizmo (C/C++ observability tool built on the top of D-Light) will be turned on by default and will be ran every time project is run and that's also the reason performance is really critical for D-Light Toolkit as it should be able to gather information with low overhead (as Gizmo requires). For PHP I think I was wrong, there is no any User View at this moment for Photon, but I think it could be some button similar to "Attach Profiler" for Java profiler. So it would be really nice to have this "profiler ui" in ideX cluster but the point is D-Light Toolkit itself is not supposed to use "Attach Profiler" action, Photon is.
Integrated into 'main-golden', will be available in build *200901200201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/7dce80431f8e User: Vladimir Kvashin <vkvashin@netbeans.org> Log: #155478 D-Light
Link http://wiki.netbeans.org/DLightToolkitDesign updated, Architecture Questions and javadoc added (see in the bottom of page) for the modules we are planning to have friends packages opened in. D-Light toolkit sources are in NB repository (but not built yet)
Please look at the new documentation and javadoc along with arch.html provided and send your comments
Any comments? Any questions?
I guess the most interesting aspect from my point of view is to make sure the modules are not part of ideX cluster. If that is satisfied and only php and cnd clusters depend on this one, I think I am fine. I guess you have my Go. I am not sure if everyone else is OK with current state. Either people shall put their Go here, or to finish the review and allow you to move on, I suggest to reuse Thursday's tech council slot (http://wiki.netbeans.org/TechCouncilMeetingAgenda). Please let other reviewers know and either express their Go here or let's meet on Thursday.
To Jaroslav: Thanks To yardus: Jarda, could you look at the documentes updated and give your Go/No Go or require API review meeting. To vkvashin: Vladimir, please say your Go/No Go
I still feel a bit sorry for the fact that we will have yet another APIs for the same task set of collecting performance data. But given that the DLight will go to a separate cluster and basically doesn't expose any public APIs I must give it a GO
We discussed this API several times in person, and finally all my requirements are met. So I give it a GO.
Hi, we(me and T.Mysik) were asked to join this review process and comment from the PHP perspective, say GO/No GO or require API review meeting. [R1] Are you going to go through separate API review once again for Photon(Php)? [R2] I'm missing more info about Photon design,architecture, User View, UI spec, schedule? Actually, if you plan one separate API review for Photon(Php), then you have my GO
We will have separate API review for Photon. This API review is about D-Light Toolkit, not Photon Photon is supposed to be the part of PHP module and be built on the top of D-Light Toolkit we are reviewing here. So, I can think we have GO, right?
Yes
Now we have 4 Go and I planning to close this IZ and add D-Light Toolkit to NB build in separate D-Light cluster.
Good. Just two notes: What will be the name of the cluster? dlight1? Second, your review says that all the modules in this cluster are supposed to be libraries, so do not forget to make them autoloads.
Thanks to all for your time! to Jaroslav: 1) name will be dlight1 2) ok, all modules will be autoloads
Integrated into 'main-golden', will be available in build *200902270313* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/c2f9fb7b3059 User: Michal Zlamal <mzlamal@netbeans.org> Log: #155478 Addition of dlight cluster
Changeset: http://hg.netbeans.org/main/rev/c2f9fb7b3059