Bug 185135 - Warn about, or offer to increase, file descriptor limit
Warn about, or offer to increase, file descriptor limit
Status: NEW
Product: installer
Classification: Unclassified
Component: NBI
6.x
PC Linux
: P1 (vote)
: 7.1
Assigned To: Libor Fischmeistr
issues@installer
: PERFORMANCE
Depends on: 197248
Blocks: 26230 183696 187130 190713 191678 197966
  Show dependency treegraph
 
Reported: 2010-04-27 19:01 UTC by Jesse Glick
Modified: 2014-02-10 14:14 UTC (History)
6 users (show)

See Also:
Issue Type: ENHANCEMENT
:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jesse Glick 2010-04-27 19:01:43 UTC
Rather than hacks like those in bug #183696, I think it would be better for the NB (IDE) launcher on Linux to display a warning or just refuse to run unless the file handle limit `ulimit -n` were set reasonably high (say, at least 4000). Running with a low ulimit is not only bad for class loading performance, but you run the risk of running out of descriptors during other operations - classpath scanning, or even from other unrelated applications.

The launcher could even offer to increase it for you, using e.g. gksudo to modify /etc/security/limits.conf or whatever the most appropriate config file is for common distributions. (Not sure if you need to reboot or log in again after making such a change.) This could also/instead be done by the installer.
Comment 1 Jaroslav Tulach 2010-04-28 05:58:22 UTC
Personal observation: I have not yet managed to increase the limit myself, on my computer (and I tried). I believe that any recommendation without some automatic action is just going to be ignored by 95% of users.

Anyway putting such warning in nbexec is unreasonable, as the warning applies only to NetBeans IDE, not other NetBeans Platform applications.

Also I am afraid that showing such warning on start is going to hurt initial start experience. Rather than that, the check and also fix shall be made part of installation procedure. That is the time the user expects changes to system configuration. Up to Ondřej to decide. 

I'd like to point out that we have a wiki page talking about the topic:
http://wiki.netbeans.org/FaqScanningAndIndexingPerformanceHints#File_Descriptor_Limit_On_UNIX
Comment 2 Jesse Glick 2010-04-28 14:32:28 UTC
(In reply to comment #1)
> I have not yet managed to increase the limit myself, on my computer

Which distribution? limits.conf works on Ubuntu. I have heard that some other distributions use a different file for this purpose.

(Are the limits on Mac OS X, Solaris 10, and OpenSolaris set reasonably high by default?)

> putting such warning in nbexec is unreasonable, as the warning applies
> only to NetBeans IDE, not other NetBeans Platform applications.

Right, which is why I wrote "NB (IDE) launcher", i.e. not nbexec.

> the check and also fix shall be made part of installation procedure

Note that this will not help users of ZIP distributions or source builds, including many NetCAT participants. That is why I suggest at least a warning to console in the launcher.

> http://wiki.netbeans.org/FaqScanningAndIndexingPerformanceHints#File_Descriptor_Limit_On_UNIX

Which is fine, but a new user would know to look for that unless they were told about it - they will experience poor performance and assume that this is just normal for NetBeans, never realizing there is a simple fix.
Comment 3 Jesse Glick 2010-10-26 16:37:10 UTC
Should also try to increase /etc/sysctl.conf#fs.inotify.max_user_watches now that we do native listening.
Comment 4 Jesse Glick 2011-02-26 00:54:29 UTC
I think we need to fix this. Attempting to use the IDE when your ulimit is too low to actually open all module JARs causes a lot of random-looking performance problems and potentially errors too. If you were not prompted to increase your ulimit, it is unlikely you would ever guess this is all that needs to be done.
Comment 5 Antonin Nebuzelsky 2011-06-08 13:50:02 UTC
Planned to be looked at for 7.1.
Comment 6 Yulia Novozhilova 2011-06-22 15:33:45 UTC
There are several approaches now:

(1) Check "ulimit -n" in installer and if it is too low, put the warning message with a description and a link to wiki page [*] into the last installer panel.

[*] http://wiki.netbeans.org/FaqOutOfFileDescriptors

"+" Easy to fix and don't overload installer UI too much
"-" Additional configuration operations are required from the user (though it is a usual situation for the active Unix users)

QUESTION: what is "too low" value for "ulimit -n"? Most distributions have 1024 by default. It means the warning will be shown to everyone who haven't change default value (the majority). 

(2) Check "ulimit -n" in installer and if it is too low(?) and the user is authorized for changing /etc files, suggest to change it. Then request sudo password and correct /etc/security/limits.conf file (for Linux) as suggested at [*] and [**]. 

[**]  http://wiki.netbeans.org/FaqScanningAndIndexingPerformanceHints#File_Descriptor_Limit_On_UNIX

"+" No additional configuration operations are required after NetBeans installation
"-" Installer UI will be overloaded with the new panels or dialogs. It is important since it looks like the majority of the Linux users have default value for "ulimit -n" = 1024, so all of them will be suggested to increase the one.
"-" limits.conf settings typically takes effect when the user logs in, so log out/log in will be required to get the advantage of this change
"-" very system dependent, will need a lot of testing on different distributions
"-" Suggestion to change system level settings in installer for IDE may look strange for inexperienced Unix user

(3) Use the approach (1) to let the user know about possible performance issues and how to fix them on a system level. Also try to increase a soft limit locally ("ulimit -n 4096") for the current session in the IDE launcher.  

"+" looks like the most harmonious way
"-" the most Ubuntu distributions are misconfigured by default with a hard limit identical to the soft limit = 1024. So it is impossible to increase the limit without sudo. But this bug is already fixed for Ubuntu Natty, see [***]

[***] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/663090

I would prefer (3). Any comments?
Comment 7 Jesse Glick 2011-06-27 19:57:22 UTC
(In reply to comment #6)
> Most distributions have 1024
> by default. It means the warning will be shown to everyone who haven't change
> default value (the majority).

Right... and most Linux users _should_ be incrementing at least the "hard" part of this ulimit. Not only NetBeans can be broken by an artificially low limit.

> "-" limits.conf settings typically takes effect when the user logs in, so log
> out/log in will be required to get the advantage of this change

True, you would need to ask the user to log in again after installation.

> "-" the most Ubuntu distributions are misconfigured by default with a hard
> limit identical to the soft limit = 1024. So it is impossible to increase the
> limit without sudo. But this bug is already fixed for Ubuntu Natty

That is great for people running Natty once that is released, but not helpful before then, and even then not helpful for people using other distributions or LTS versions of Ubuntu.

That said, it would be useful immediately to run ulimit -S -n 4000 (or whatever) in the IDE launcher, and just issue a warning if that command fails (with special advice in case ulimit -H -n reports a low hard limit). This will at least address the problem for those people whose hard limit is high enough but whose soft limit is not.
Comment 8 Jiri Rechtacek 2012-10-07 12:58:36 UTC
Assigned to new owner.
Comment 9 Svata Dedic 2012-11-12 09:24:08 UTC
As I am browsing exception reports for background scan cancel, sometimes I see  errno: 28 messages logged by org.netbeans.modules.masterfs.watcher.Watcher, which could indicate low descriptor limit. I'll attach the exception reports to this defect.


By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo