This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 57798 - Permit userdir to be split into preferences vs. cache
Summary: Permit userdir to be split into preferences vs. cache
Status: RESOLVED FIXED
Alias: None
Product: platform
Classification: Unclassified
Component: Launchers&CLI (show other bugs)
Version: 4.x
Hardware: All All
: P2 blocker with 5 votes (vote)
Assignee: Jesse Glick
URL:
Keywords: API, API_REVIEW_FAST, PLAN
: 85229 115670 (view as bug list)
Depends on: 201805 201350
Blocks: 196075
  Show dependency tree
 
Reported: 2005-04-12 22:04 UTC by ulfzibis
Modified: 2012-02-09 10:34 UTC (History)
11 users (show)

See Also:
Issue Type: ENHANCEMENT
Exception Reporter:


Attachments
Running patch ('hg pdi var_cache_57798' in core-main to refresh) (86.15 KB, patch)
2011-07-28 17:13 UTC, Jesse Glick
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description ulfzibis 2005-04-12 22:04:31 UTC
[ JDK VERSION : 1.5.0_02 ]

Ideally, the application data should be stored in the 'Application Data' subfolder. All programs in Windows put there data there.
The advantage is, that only 1 folder must be backuped each day.

see file: ${NETBEANS_HOME}\netbeans-4.1\etc\netbeans.conf

bad:
netbeans_default_userdir="${HOME}/.netbeans/4.1"

better:
netbeans_default_userdir="${APPDATA}/netbeans/4.1"
or:
netbeans_default_userdir="${APPDATA}/Java/netbeans/4.1"
Comment 1 Jan Chalupa 2005-04-21 17:40:50 UTC

*** This issue has been marked as a duplicate of 49813 ***
Comment 2 ulfzibis 2005-04-27 16:32:58 UTC
Move netbeans_default_userdir to APPDATA.

I reopen this issue, because the mentioned duplicate originally had a different
target and so has a misleading summary (no hint to the APPDATA part).

The userdir at Windows is not directly designated for application software usage.

I prefer (reasons see Issue 58381):
netbeans_default_userdir="${APPDATA}/Java/netbeans/4.1"

Note: JRE's settings folder is "${APPDATA}/Sun/Java/Deployment"

> jchalupa (Issue 49813):
> I tend to agree regarding the APPDATA part. Ideally, the application data should
> be stored in the 'Application Data' subfolder. The reason why NetBeans doesn't
> use it is that Java APIs don't provide a reliable way to get that folder. A
> possible workaround would be to use native code and Win32 APIs to get the name
> of the APPDATA folder. We can consider that for future releases.

For solution have a look at JRE's installer. ;-)
Comment 3 Jan Chalupa 2005-04-27 17:50:35 UTC
I don't see a precedent for *Java* in the suggested ${APPDATA}/Java/netbeans/4.1
folder name. IMO, the convention seems to be the vendor name under the
${APPDATA} folder. So, in the case of NetBeans, it would be:

  ${APPDATA}/NetBeans/4.1/

Anyway, this is not going to be implemented for 4.1. Too late.
Comment 4 ulfzibis 2005-04-27 19:33:54 UTC
Isn't the vendor *Sun* ?
Comment 5 Jan Chalupa 2005-04-28 09:32:30 UTC
No. NetBeans is an open source project, although Sun is the biggest contributor,
of course.
Comment 6 ulfzibis 2005-05-03 10:20:48 UTC
@jchalupa
I agree with you. Thanks for discussion.
I was thinking of Issue 58381 (All Java-apps in ../Java/..-path) when proposing
the Java-part.

So in following the convention, JRE should be in "Program Files\Sun\JRExx".
I more like the convention proposed in Issue 58381: All Java apps in "Program
Files\Java\...".
Comment 7 ulfzibis 2005-10-17 13:55:32 UTC
So in following the convention, JRE should be in "Program Files\Sun\JRExx", but
it is in "Program Files\Java\JRExx".

IMO, the convention seems to be the "vendor name\product name" under the
${APPDATA} folder. So, in the case of NetBeans, it would be:

  ${APPDATA}/NetBeans/IDE/4.1/
Comment 8 _ rkubacki 2006-08-08 09:51:28 UTC
Deplyoment in Sun's JDK uses
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/shellcc/platform/shell/reference/functions/shgetspecialfolderpath.asp
with CSIDL_APPDATA param. Probably we can pass this information from launcher. 

Need to check what is the impact on settings import from earlier version.
Comment 9 _ rkubacki 2006-10-02 13:47:38 UTC
*** Issue 85229 has been marked as a duplicate of this issue. ***
Comment 10 Tim Lebedkov 2006-10-02 20:32:15 UTC
please also consider using
c:\documents and settings\local settings\application data\Company\Product
for settings that cannot be moved to another computer (from #85229)
Comment 11 dlipin 2007-09-27 09:02:35 UTC
not the installer RFE but the platform`s one.
Comment 12 Chiana 2011-01-19 07:05:23 UTC
There is a technical aspect to doing this change; Networked computers with "roaming profiles" a.k.a. Domain Computers.

First, when you have roaming profiles the content of the user directory is copied into (or rather synchronized with) the server when you logout, one need not much of brainpower to realize that netbeans cache takes a long time to process causing long login/logout time.
Second, the cache itself (rather the "var" directory) is local data belonging to the machine installation and thus should not be stored in the profile, some configuration settings however should be stored in the profile (like open projects that resides on networked drives or inside the profile itself)

This means you cannot make the whole .netbeans directory local or roaming, it needs to be split into one part that is stored within the profile and one that is stored outside.

Some say "this works on Linux", but then again, how many are there using windows contra Linux? (No I'm NOT going to respond to any l*x../..w*s comment's on this, so don't make any, this is a fact that should be known to everyone)

Because of this I think this change should be made, not just for the current status but also for future changes to both l*x and w*s
Comment 13 ulfzibis 2011-01-19 11:37:24 UTC
(In reply to comment #12)
> There is a technical aspect to doing this change; Networked computers with
> "roaming profiles" a.k.a. Domain Computers.
You are right, thanks for that aspect.

BTW: Mozilla Firefox has that splitting:
Profile -> %APPDATA%\Mozilla\Firefox\...
Cache   -> %HOMEPATH%\Locale Settings\Application Data\Mozilla\Firefox\...
Comment 14 Antonin Nebuzelsky 2011-01-24 12:20:47 UTC
Changing the summary to reflect the point made above.
Comment 15 ulfzibis 2011-01-24 13:19:20 UTC
I guess on iX the two parts would be reasonable too, and on Windows in:
Profile   -> %APPDATA%\Netbeans\IDE xxx\...
Cache/Var -> %HOMEPATH%\Locale Settings\Application Data\Netbeans\IDE xxx\...

For the application I would prefer:
%PogramFiles%\NetBeans\IDE xxx\ or %PogramFiles%\Oracle\NetBeans IDE xxx\
instead:
%PogramFiles%\NetBeans xxx\

So there would only be 1 top folder for Netbeans in the PogramFiles tree, similar, as it is in the start menu tree.
Comment 16 Sc0tTyXL 2011-01-26 08:46:38 UTC
(In reply to comment #15)
> I guess on iX the two parts would be reasonable too, and on Windows in:
> Profile   -> %APPDATA%\Netbeans\IDE xxx\...
> Cache/Var -> %HOMEPATH%\Locale Settings\Application Data\Netbeans\IDE xxx\...
> 
> For the application I would prefer:
> %PogramFiles%\NetBeans\IDE xxx\ or %PogramFiles%\Oracle\NetBeans IDE xxx\
> instead:
> %PogramFiles%\NetBeans xxx\
> 
> So there would only be 1 top folder for Netbeans in the PogramFiles tree,
> similar, as it is in the start menu tree.

I would prefer %ProgramFiles%\Netbeans IDE\%Version%

i now have like 15 Netbeans folders ( nightly's and a few stables )

Also cache should be local only, settings i'm not sure if i want that romaing or not but it has to be put into %appdata%\Netbeans IDE\%Version%
Comment 17 ulfzibis 2011-01-26 13:33:42 UTC
(In reply to comment #16)
> I would prefer %ProgramFiles%\Netbeans IDE\%Version%
The standard pattern on Windows is %ProgramFiles%\%Vendor%\%Product%\
I'm not sure about the right vendor of NetBeans (Oracle?).
But keep in mind, there could be more products from NetBeans besides the IDE, e.g. NetBeans Platform.
I'm 50/50 for both suggestions.

> i now have like 15 Netbeans folders ( nightly's and a few stables )
Same to me, it's ugly!

For the JDK part please comment on thread:
http://mail.openjdk.java.net/pipermail/nio-discuss/2011-January/000501.html
Comment 18 nigjo_iqn 2011-02-26 18:24:19 UTC
(In reply to comment #15)
> I guess on iX the two parts would be reasonable too, and on Windows in:
> Profile   -> %APPDATA%\Netbeans\IDE xxx\...
> Cache/Var -> %HOMEPATH%\Locale Settings\Application Data\Netbeans\IDE xxx\...

Wherever the "cache" (or better "var") Directory will be, it will be a lot work to change the behavior of many modules. I did a quick search through the sources and many modules do similar things like this to get their "cache" folder:

  private File getStorageRootFile() {
    String userDir = System.getProperty("netbeans.user"); // NOI18N
      return new File(new File(userDir, "var"), "bugtracking"); // NOI18N
  }

[NI01] I suggest a new system property value "netbeans.user.var" to point to the "local settings" and change all occurrences like the above to use this new one. As a fallback for installations which do not set "netbeans.user.var" in the launcher this property should be set somewhere early in the startup process.
Comment 19 ulfzibis 2011-02-26 19:43:56 UTC
I've filed a bug:
You can monitor this bug on the Java Bug Database at
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7022730.

It may take a day or two before your bug shows up in this external database.
Comment 20 ulfzibis 2011-04-13 15:11:48 UTC
(In reply to comment #2)
> Move netbeans_default_userdir to APPDATA.
>
> jchalupa (Issue 49813):
> I tend to agree regarding the APPDATA part. Ideally, the application data should
> be stored in the 'Application Data' subfolder. The reason why NetBeans doesn't
> use it is that Java APIs don't provide a reliable way to get that folder. A
> possible workaround would be to use native code and Win32 APIs to get the name
> of the APPDATA folder. We can consider that for future releases.

I found out, there is a way from the JRE:
    System.getEnv("APPDATA");
Comment 21 Jesse Glick 2011-07-27 21:43:45 UTC
Leaving bug #196075 for the move of valuable files to APPDATA, this issue can be considered the problem of splitting var\ or var\cache\ to "Local Settings".
Comment 22 Jesse Glick 2011-07-27 22:34:00 UTC
Using var_cache_57798 pbranch in core-main.
Comment 23 Jesse Glick 2011-07-28 17:06:46 UTC
For simplicity, treating this issue just as the _capability_ of using a separate cache directory, with --cachedir to select one. API request for new org.openide.modules.Places. Bug #196075 should track changing the _default_ on Windows to use Application Data for the configuration part and Local Settings for the cache part; not sure if this can/should be done in the netbeans.exe launcher, or only in the installer. There may be appropriate defaults for other operating systems as well but I am not sure.
Comment 24 Jesse Glick 2011-07-28 17:13:43 UTC
Created attachment 109680 [details]
Running patch ('hg pdi var_cache_57798' in core-main to refresh)
Comment 25 nigjo_iqn 2011-07-28 18:38:48 UTC
(In reply to comment #23)
> API request for new org.openide.modules.Places.
It looks good to me on a first quick peek.

> Bug #196075 should track changing the _default_ on
> Windows to use Application Data for the configuration part and Local Settings
> for the cache part; not sure if this can/should be done in the netbeans.exe
> launcher, or only in the installer. There may be appropriate defaults for other
> operating systems as well but I am not sure.
The launcher can access the windows registry where all these paths are stored. This is already be done to get different folder locations. (via constant "REG_SHELL_FOLDERS_KEY")
%APPDATA% key-entry is "AppData"
you can use the key entry "Local AppData" for local settings.

[NI02] In my current Win7 installation this comment is inside the registry key:
"Use the SHGetFolderPath or SHGetKnownFolderPath function instead". 
SHGetKnownFolderPath (WinVista+): http://msdn.microsoft.com/en-us/library/bb762188%28v=vs.85%29.aspx
SHGetFolderPath (WinXP and below): http://msdn.microsoft.com/en-us/library/bb762181%28v=vs.85%29.aspx

[NI03] Shouldn't the whole "var" folder be moved to "local settings"?
Comment 26 Jesse Glick 2011-07-29 12:11:26 UTC
NI02 and related Windows-specific comments - thanks; please use bug #196075 for this though.


NI03 - I think not. The great majority of the large files that would typically overload network drives, or be unwanted in a backup, are already in the cache/ subdir: parser indices, Maven repository indices, downloaded Auto Update catalogs, SVN offline caches, and so on.

What remains in var/ is not that large on average and is typically not transparently recreated like a real cache would be:

- log files (well you hope these do not get too large, anyway!)

- attributes.xml (references to absolute file paths which might be shared across drives, and may include user preferences specific to files or to projects which decline to keep NB-specific metadata in versioned sources)

- autoprojects.properties for http://wiki.netbeans.org/AutomaticProjects (currently a mix of cache and preferences, but generally small anyway)

- filehistory (potentially valuable for backup; anyway some sort of history trimming probably needs to be introduced here, e.g. delete history prior to a SCM commit)

- bugtracking (not clear to me if this is cache or not, but that is up to devs of that module to decide)

- a few miscellaneous flag or lock files too small to bother with
Comment 27 Jesse Glick 2011-08-04 19:54:57 UTC
core-main #dc00bb155bce
Comment 28 Quality Engineering 2011-08-05 14:39:32 UTC
Integrated into 'main-golden'
Changeset: http://hg.netbeans.org/main-golden/rev/dc00bb155bce
User: Jesse Glick <jglick@netbeans.org>
Log: #57798: permit var/cache/ to be stored separately from the userdir.
Comment 29 Vladimir Voskresensky 2011-08-05 15:30:16 UTC
I have a comment about change done in 
cnd.repository/src/org/netbeans/modules/cnd/repository/disk/StorageAllocator.java

 97 prefix = reduceString(prefix);
 98
-108 path = getCachePath() + File.separator + prefix + File.separator; // NOI18N
-109
-110 File pathFile = new File (path);
+ 99 File pathFile = new File(diskRepository, prefix);
+100
+101 path = pathFile + File.separator; 

what about line 101 as
path = pathFile.getAbsolutePath() + File.separator; 

?
Comment 30 Jesse Glick 2011-08-05 15:43:19 UTC
(In reply to comment #29)
> I have a comment about change done in cnd.repository/...

Yes, please look over your module's functionality; I tried to leave behavior untouched (when --cachedir not specified) but I do not know how to test things like this.

> path = pathFile + File.separator; 
> 
> what about
> path = pathFile.getAbsolutePath() + File.separator; 

Sure. Would only make a difference in case pathFile were relative, which I suppose could happen if you passed -J-Dcnd.repository.cache.path=../cndcache.
Comment 31 Vladimir Voskresensky 2011-08-05 18:02:21 UTC
(In reply to comment #30)
> (In reply to comment #29)
> > I have a comment about change done in cnd.repository/...
> 
> Yes, please look over your module's functionality; I tried to leave behavior
> untouched (when --cachedir not specified) but I do not know how to test things
> like this.
> 
> > path = pathFile + File.separator; 
> > 
> > what about
> > path = pathFile.getAbsolutePath() + File.separator; 
> 
> Sure. Would only make a difference in case pathFile were relative, 
pathFile is an instance of java.io.File and it's not very nice 'a+b' expression where toString() is used indirectly
Comment 32 Jesse Glick 2011-08-05 18:14:07 UTC
(In reply to comment #31)
> it's not very nice 'a+b' expression where toString() is used indirectly

As you like.
Comment 33 ulfzibis 2011-08-09 19:17:52 UTC
(In reply to comment #31)
> and it's not very nice 'a+b' expression where toString() is used indirectly
Under the hood, always StringBuilder.append() is used.
Comment 34 Jaroslav Tulach 2011-08-25 07:02:15 UTC
Looking once again at the created API, I believe it offers its users something that they are supposed to never use. The getters are OK, but the setters are only for privileged users (e.g. core.startup) or for unit tests. Nobody else should ever use them.

Y01 Remove setters from Places API

I can see two ways to do it. Either rely on System properties (as has been the case so far), which is in my opinion OK in tests. Or Lookup.getDefault().lookup(Places.class) implemented in core.startup with additional protected abstract findCacheDir() & findUserDir() to call from the getters.

Sorry for reopening existing issue, I could/can create new bug, if desired.
Comment 35 Jesse Glick 2011-08-25 11:25:01 UTC
(In reply to comment #34)
> Nobody else should ever use them.

So, do not use them. Is there any evidence that anyone would "accidentally" call these methods? Existing code could already call System.setProperty("netbeans.user", ...) but that was never a threat.

A setter with Javadoc explaining its intended scope of usage (and friendly to JPDA breakpoints, logging, Find Usages, etc.) is if anything safer than a system property.

(By the way I initially prototyped throwing SecurityException when an attempt was made to change the value once set. This did not work, as usages from TestCase.setUp need to modify an existing value in the same JVM.)

> Lookup.getDefault().lookup(Places.class) implemented in core.startup

Possible, though this would be more cumbersome for tests; see several test usages in the patch.

> I could/can create new bug, if desired.

That would be clearer.
Comment 36 Jiri Rechtacek 2012-02-09 10:34:03 UTC
*** Bug 115670 has been marked as a duplicate of this bug. ***