This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
My project's build script (from scons) happens to make symbolic links pointing up the tree, for example: /home/sirikata/workspace/sirikata/externals/cxxtest/build_tools/SCons/test/default_env/cxxtest -> ../../../../ This causes Eclipse to hang in the "Scanning projects" state. Because it has no status when doing this, the only way to debug was by running strace. I saw that eclipse was just walking down the tree indefinitely (path names getting longer and longer), and it doesn't seem to have a reasonable max path length. I'm not sure how to solve the problem, but this, combined with the inability to abort Scanning Projects, was making eclipse unusable. Even with a short max path length, this directory happened to have several of these symlinks, so it would have to go through every permutation of them which would still take a lot of time and memory. Luckily, I happened to be able to remove all the offending symbolic links by doing some trickery like moving the recursive directories outside of the project root. But it would be nice not to need to use hacks like that. I also noticed several UI hangs (1 second response time at best, sometimes the whole UI would freeze for 5 seconds)... and I also encountered many deadlocks that required a killall -9 java when trying to debug this issue.
Reassigning.
Your description refers to Eclipse; did you mean NetBeans? I can't do anything without steps to reproduce the problem from scratch (including all necessary files, either in a ZIP or SCM checkout URL): what you did, what you saw, what you expected to see instead.
Whoops, I made kind of an embarrassing mistake--yes I meant NetBeans. Eclipse's indexer is able to handle this, so I've been using it instead for now. Anyway, the best way to reproduce this is to do something like: mkdir test cd test ln -s . recursivelink1 ln -s . recursivelink2 ln -s . recursivelink3 ln -s . recursivelink4 ln -s . recursivelink5 ln -s . recursivelink6 If you would like to see a project with this problem, there is a git repository that comes with the symbolic links already in place. git clone git://github.com/danielrh/cxxtest.git git fetch origin cd cxxtest git checkout ef8efc At this point, if you do an "ls -lR | less", you should notice a few weird symlinks--that should trigger the problem if you open the directory as a project inside of netbeans. I suspect if there is only one or two recursive links, you might pretty quickly reach the maximum path length. In my case I had at least 6 of these symlinks, and it's possible that the problem only happens when the amount it spans is large enough (it would probably go up exponentially, i.e. 6^maxpathlength).. Also, I don't believe Ant is the correct target--would it be possible to change that? In my case I was using a C++ project and hadn't yet built a makefile for it, so it's clearly part of the core IDE itself, in whatever produces the "Scanning projects" status bar.
"if you open the directory as a project inside of netbeans" - there is no NB project metadata that I can see in this directory, so it seems we are still missing complete, self-contained, minimal steps to reproduce the problem. Your issue may be specific to a particular project type or project setup but it is hard for me to guess.
reassigning to code model based on 'hang in the "Scanning projects" state'.
pathorn, could you create please sample project, tar it, gzip it and attach to IZ? Thanks, Vladimir
Created attachment 93128 [details] Larger testcase that includes enough files to replicate the bug report Attaching a tar file with that entire cxxtest repository I mentioned in my post, including the faulty symbolic links. Upon opening this project, it should open the project relatively quickly. Then, it will index the sourcecode correctly (and you should be able to ctrl-click on the "print" and "std::cout" functions). The problem is that the IDE will take 100% CPU, you will always see Scanning projects... in the status, and the IDE will respond very sluggishly every few seconds. In order to see the symbolic links being processed, run: strace -ff netbeans After the IDE gets into the scanning projects.. phase, you should see the console spew out 1000 character pathnames returning "ELOOP (Too many levels of symbolic links)" Thanks for looking at this.
Created attachment 93129 [details] Attempt at a small testcase. Triggers a different but related bug. This testcase was my first attempt at manually replicating the conditions that caused the Scanning Projects... problem; however it seems to trigger a different, much more severe set of bugs, even though there are fewer files here, and it even causes the IDE to freeze in many cases. Here's what happens: I see it say "Task scanning..." Unlike the large testcase, this status remains forever, and the only way to abort is by clicking the "X" button, which does work. At this point, it will say "Opening projects". The project never opens, and clicking on the + on the leftbar Projects tab will show "Initializing project..." If you click on the Files tab there, it shows all the files, however, selecting any of them with the mouse (not even opening one) will hang the IDE. it is possible to open files by clicking on the toplevel helloworld entry, and using arrow keys and enter to open source files, or also by specifying them as command line arguments. In addition, the project will not always index properly if you open source files; though this worked once or twice, so again the indexing is not affected by this bug. Because the project is not initialized, I cannot build or do anything else useful with the project. There are several other cases here that cause a hang--happened once when I clicked on the menu bar which caused the whole X server to hang due to a mouse grab. Since this only seems to happen for "small" projects (less than 10 source files or so?) this is not as big a bug to solve, but it is likely related to a bunch of threads waiting for the Initialization of the project, and the project initialization pending on the infinite scan. Perhaps this testcase should go into a different bug, but I have a feeling that some race condition determines whether the project can get opened before it gets stuck scanning files, or not. I hope this helps you debug the problem.
*** Bug 189004 has been marked as a duplicate of this bug. ***
yes, this is a very serious and easily encountered problem. all it takes is one symlink pointing to a parent of the current directory. currently it will recurse the directories to 37 levels after which it gives up. So the loop does eventually terminate, but if you have a lot of symlinks it could take a very long time. Meanwhile it is generating hundreds of megabytes of cache files as it keeps re-parsing the same file over and over. if you look in Bug 189004 which has now been marked as a dup of this one, you will see a further discussion of ways to solve this problem.
fixing, change set: http://hg.netbeans.org/cnd-main?cmd=changeset;node=aeaf323b322b
Integrated into 'main-golden', will be available in build *201008250001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main/rev/aeaf323b322b User: Alexander Simon <alexvsimon@netbeans.org> Log: fixing BZ#178180 Netbeans hangs if any directory contains a recursive symbolic link
The issue still exists in Netbeans 6.9.1 under windows and ubuntu on the latest nightly build (1/24/2011) 201101230001 (log is from windows) WARNING [org.netbeans.modules.masterfs.watcher.Watcher]: Cannot add filesystem watch for \<myproject>\libs\npmlibs\.npm\chainsaw\0.0.5\dependson\traverse@0.2.3\dependents\chainsaw@0.0.5\dependson\traverse@0.2.3\dependents\chainsaw@0.0.5\dependents\binary@0.1.0\dependson\put@0.0.1\dependents\binary@0.1.0\dependson\put@0.0.1\dependents\binary@0.1.0\dependson\put@0.0.1\dependents\binary@0.1.0\dependson\put@0.0.1\dependents\binary@0.1.0\dependson\put@0.0.1\package\lib Those x@n.n.n files are symlinks back in the root /libs/npmlibs/.npm/ folder I let it run all night and when I got back into office it was still scanning at that path. The proper fix to fully solve all issues with recursive paths and scanning is to change the scanning system to do the following: First of all, honor the ignored directories setting of the project.... I have /npmlibs/libs on the ignored directories setting yet it still scans it... Second, start a single thread and traverse the file paths of the project (thats not ignored), and follow the path of the ABSOLUTE paths. As directories are entered, add their ABSOLUTE path to an array. Then before you process a directory (and also before adding it to the array), check if the entry already exists in the array. If it does, that means it was processed already and this is just a recursive symlink. Walk the entire tree and add every absolute path to an array to build a list of files to index. Third, you then spawn multiple threads to pop entries off of this list to index 1 at a time (with thread safety of course). The problem here is that netbeans is not checking absolute path and seeing that this specific directory has already been indexed. The above solution will fix that.
bumping priority since this essentially disables the IDE from functioning if your project has a recursive symlink... I had to switch to plain file editor because Netbeans will not function fully while its scanning files.
(In reply to comment #14) > bumping priority since this essentially disables the IDE from functioning if > your project has a recursive symlink... > > I had to switch to plain file editor because Netbeans will not function fully > while its scanning files. Are you talking about C++ scanning?
Please provide threads dump.
Sorry thought higher = more important on priority... fixing priority. I'm experiencing it in PHP projects The issue is extremely trivial to replicate on your own mkdir test cd test touch test.js ln -s ./ test file > new project > PHP with Existing Sources > choose new test folder you made behold unending scanning
and from ide log: WARNING [org.netbeans.modules.masterfs.watcher.Watcher]: Cannot add filesystem watch for \\dev\dennis\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\test\nbproject\private I've posted the solution to fix this above.
I cannot reproduce the case as is described by Aikar in the latest build, but it can be reproduced with the same steps, only the symlink has to be define as ln -s ../test/ test2 Problem with PHP project duplicates issue #189004. Briefly there is explained that we shouldn't use getCanonicalPath due to the performance and that we should wait for JDK 7, which should handle this situation. So I'm closing this issue and reopening issue #189004, that describe the same problem as has Aikar. IMHO the fix that is done in CND is not right and can caused big slowdown of the IDE.
Note my issue does not relate to a syntax error... it relates more so to this tickets name, netbeans scanning the project dir doesnt ever stop and hangs it up breaking all functionality.
This issue is not fixed. I still encounter this in both 6.9.1 as well as 7.0 RC1. We have a symlink that points at "." (the directory it's in) to support some legacy pathing in our system. While the paths are queued to be addressed so we can do away with the ugly symlinking, this bug prevents us from using Netbeans (or most other IDEs out there).
Which project do you use? The recursive symlink are not fixed and will not be fixed until issue #189004 will not be resolved. The issue #189004 can be resolved with using jdk 1.7. So we should wait for this JDK. If you want you can inclrease the priority of the issue #189004.
(In reply to comment #22) > The recursive symlink are not fixed and will not be fixed until issue #189004 > will not be resolved. The issue #189004 can be resolved with using jdk 1.7. So > we should wait for this JDK. If you want you can increase the priority of the > issue #189004. Ok, I increased the priority of issue 189004, and we'll do our best to deliver the fix into NB 7.0.1. We won't fix it into NB 7.0 (it's too late for fixing and verification potential regressions).
The issue #189004 is now fixed in trunk. Now it should work. Please could you test it on your projects? Feel free to reopen, if it still hangs.
The issue still stands in 7.0.1. I have a directory in my project which contains a symlink to the directory itself. If i include it in my project it hangs. If I don't include it, it works fine. Other than this symlink, there are just some JS files in the directory.
Hickeroar, could you describe the path and how the link is done? Is a symlink? Which OS do you use? Thanks.
Adding Tomas to cc.
Hickeroar, could you provide please example of your directory structure? And which os do you use?
Waiting for the answer from Hickeroar. Marking as incomplete, please reopen when you provide the info. Thanks.