This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 183059 - [69cat] Checking for external changes is "heavy"
Summary: [69cat] Checking for external changes is "heavy"
Status: RESOLVED FIXED
Alias: None
Product: platform
Classification: Unclassified
Component: Filesystems (show other bugs)
Version: 6.x
Hardware: PC Windows XP
: P2 normal (vote)
Assignee: Jaroslav Tulach
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-03-29 11:24 UTC by stefan79
Modified: 2010-04-20 11:06 UTC (History)
5 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Scanning projects and checking for external changes runs together. (13.17 KB, image/jpeg)
2010-04-08 07:00 UTC, stefan79
Details
Logfile (636.69 KB, application/octet-stream)
2010-04-08 07:02 UTC, stefan79
Details
Please have a look at the Logfile. (1.18 MB, application/octet-stream)
2010-04-08 14:27 UTC, stefan79
Details
A Snapshot while the lastest hours. (1.23 MB, application/octet-stream)
2010-04-08 14:30 UTC, stefan79
Details
Log-File (3.28 MB, text/plain)
2010-04-20 11:06 UTC, stefan79
Details

Note You need to log in before you can comment on or make changes to this bug.
Description stefan79 2010-03-29 11:24:50 UTC
[ BUILD # : 201003290201 ]
[ JDK VERSION : 1.6.* ]

This task runs very often is also a very heavy task.
That means, that the IDE isn´t usable while this task runs.

Is there any option to log the reason for running this task?
Comment 1 Jan Lahoda 2010-03-29 12:00:20 UTC
Changeset:
http://hg.netbeans.org/core-main/rev/7c5f5b8074fe
Comment 2 Jaroslav Tulach 2010-03-31 08:07:40 UTC
The task is started because you switched to the IDE. You can cancel it, if you want.

How long does the check take? A minute?
Comment 3 stefan79 2010-03-31 12:47:32 UTC
(In reply to comment #2)
> The task is started because you switched to the IDE. You can cancel it, if you
> want.
The task also starts sometimes when I switch inside of NetBeans.
 
> How long does the check take? A minute?
Sometimes some seconds, otherwise some minutes.

Can you make a Log-Message, with the following details:
-) reason for starting
-) duration
Comment 4 Jaroslav Tulach 2010-04-01 14:15:23 UTC
I'll suspend the refresh when other I/O activity is detected.
Comment 5 Jaroslav Tulach 2010-04-01 21:01:03 UTC
core-main#8939454932dc
Comment 6 Quality Engineering 2010-04-03 05:33:15 UTC
Integrated into 'main-golden', will be available in build *201004030201* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress)
Changeset: http://hg.netbeans.org/main/rev/8939454932dc
User: Jaroslav Tulach <jtulach@netbeans.org>
Log: #183059: Checking for external changes is now lightweight.
The masterfs tries to measure disk I/O activity and performs the refresh only, if the activity is low.
The activity is the # of disk touches in last 100ms, half of the # of disk touches in previous 100ms, and so on.
There is a method idleIO(int, Runnable) which is excluded from the I/O statistics, but which is slowed down if the I/O load is higher than expected.
Comment 7 stefan79 2010-04-08 07:00:34 UTC
Created attachment 96879 [details]
Scanning projects and checking for external changes runs together.

So I don´t think, that the always works.
Comment 8 stefan79 2010-04-08 07:02:07 UTC
Created attachment 96880 [details]
Logfile
Comment 9 stefan79 2010-04-08 14:27:12 UTC
Created attachment 96898 [details]
Please have a look at the Logfile.

I think the "Checking for external changes" runs to often.
Comment 10 stefan79 2010-04-08 14:30:06 UTC
Created attachment 96899 [details]
A Snapshot while the lastest hours.
Comment 11 stefan79 2010-04-08 14:31:29 UTC
I´ve changed to reopened, because the main problem isn´t fixed.
Comment 12 Jaroslav Tulach 2010-04-08 19:15:49 UTC
I want to understand what is the "main problem". The issue mentioned few times that the check "heavy". I believe I made it "lightweight". The question is how we will measure that. My idea is to compare the up-to-date check of 6.8 with current version, I believe that the "checking for external" changes will not influence the parsing. If not, reopen, otherwise, I suggest to leave as fixed.

Saying that the check runs too often, is something completely else. Unrelated to being "heavy".
Comment 13 stefan79 2010-04-09 06:36:22 UTC
I think you have ignored my uploaded attachments.

-) http://netbeans.org/bugzilla/attachment.cgi?id=96879:
>The masterfs tries to measure disk I/O activity and performs 
> the refresh only, if the activity is low.
When Scanning projects is running, there´s always much I/O.

-) http://netbeans.org/bugzilla/attachment.cgi?id=96898:
I don´t know which task, this is in the Snapshot, but I´m sure there are many invocations.

And so I´m back at my third comment
(In reply to comment #3)
> Can you make a Log-Message, with the following details:
> -) reason for starting
> -) duration
But I think, the reason for starting is more interested.
Comment 14 Jaroslav Tulach 2010-04-12 09:27:17 UTC
Hello Stefan, the parsing API experts will examine your logs today. If they find something suspicious they create new bugs. 

Meanwhile I'd be interested in knowing whether the initial up-to-date check is now on par with 6.8. If so, then my change 8939454932dc achieved its goal and made the "check for external changes" really lightweight. Could you compare  your 6.8 and trunk IDEs and measure how long it takes for them to finish the initial up-to-date check?
Comment 15 stefan79 2010-04-13 06:31:46 UTC
I can´t make the Snapshot now, because I get always an OutOfMemoryError (Bug #183847).
I´ll test this in the next days.
Comment 16 Petr Jiricka 2010-04-16 12:39:32 UTC
Bug 183847 is now fixed, so it should be possible to verify the fix.
Comment 17 stefan79 2010-04-16 15:35:30 UTC
Current state:

-) OutOfMemoryError is still alive (-> 183888).
-) Check for external ... runs very often and also while other tasks are running (examples: while Scanning Projects and while a CVS-Diff on a folder with many files)

So again:
Please make a log-message:
-) Starting: Checking for external changes startet at yyyy-MM-dd HH:mm:ssss, Reason: XXXX
-) Ending: Checking for external changes finished at yyyy-MM-dd HH:mm:ssss, duration= XXX seconds
Comment 18 Jaroslav Tulach 2010-04-16 18:36:06 UTC
-J-Dorg.netbeans.core.ui.focus.level=FINE as soon as bug 184219 propagates to trunk. But remember this bug is about "Check for external changes" and so is the log.

I am still interested in knowing the after start comparition between 6.8 scanning/up-to-date check time and 6.9 scanning/up-to-date check time.
Comment 19 stefan79 2010-04-20 11:06:31 UTC
Created attachment 97700 [details]
Log-File

(In reply to comment #18)
> I am still interested in knowing the after start comparition between 6.8
> scanning/up-to-date check time and 6.9 scanning/up-to-date check time.
Me too, but the OutOfMemoryError is still alive (-> #183888).


Today I made a test (duration was 2h 40min):

Statistic of this time:
-) 78 x "Window deactivated, preparing refresh task"
-) 78 x Refresh done in XXX ms
  MIN=28390 ms
  MAX=282859 ms
  SUM=3422029 ms -> 57 min
-) 78 x SystemFileSystem refresh done XXX ms
  MIN=454 ms
  MAX=2438 ms
  SUM=52128 ms -> 52 seconds

Conclusion: 35% of the time the "Checking for external changes" - task is running.