This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.
This was initially filed as a P2 issue 246115 against C/C++ remote file system. But investigation showed that the root cause lays in general infrastructure. Or at least the issue is quite the same with masterfs if underlaying file system is slow. To reproduce: open a file on a slow file system (via vpn, or from Europe to US, etc). Make a change, save, make another change and save prior than hourglass cursor is changed to normal one. After that recent changes are lost.
More info on how to reproduce. As a host my file resides I used a host that is located in US; I'm in St-Petersburg, ping to this host is about 200ms. On Solaris, this host shared directories are mounted automatically via /net/full_host_name/... Then I used "tail -f ${my_file}" command on the remote host. It prints the last lines of a file and updates its output each time the file is updated. Then, at last file line, I added "1", pressed Ctrl-S, then added "2", pressed Ctrl-S, then "3", Ctrl-S, "4", Ctrl-S, etc. File stuck at "123", while in editor I saw 12345678. "Save all" button was grayed, although file tab title was bold. Subsequent attempts to save the file had no effect. When trying to close the file, a question "Save/Discard/Cancel". I chose "Cancel", but the file wasn't save and wasn't close and the tab title is still bold.
This can also be reproduced via setting breakpoint(s) in debugger.
(In reply to Vladimir Kvashin from comment #0) > This was initially filed as a P2 issue 246115 against C/C++ Sorry, I mixed issue number, I meant issue 245417 against CND
It is easy to reproduce this with breakpoint set on line FileObj.java:130 - in FileObj.getOutputStream right after MUT_EXCL_SUPPORT mutex is taken. 1) Set breakpoint 2) open any file 3) make 1-st modification, save file - the breakpoint is hit. Don't press "Continue", let this thread hang (as if it hanged with slow operation) 4) make 2-nd modification, save file 5) make 3-rd modification, save file 6) release the thread that hit breakpoint Two "Cannot get exclusive access to" error message boxes appear, "Save all" button is disabled, editor tab is bold, you can not save the file any more, recent changes are lost.
> "Save all" button is disabled, editor tab is bold, > you can not save the file any more, > recent changes are lost. It seems that there are two ways to get to this situation (related code is in method CloneableEditorSupport.saveDocument()): 1) - The First save action is invoked, locks the file and writes into the stream. - The second save action with updated content is invoked, but the file is still locked, so the "Cannot get exclusive access to" is thrown (action fails). - The first save action completes, data object is marked as not-modified, but the document still contains updated unsaved content. 2) - The save action at first writes the content to MemoryOutputStream and then writes the data from memory info the file. If two or more save actions are invoked, each of them reads current document data into memory. If they later need to wait for some monitor, they can continue in different order and the older file content can be saved after the newer one.
Jaroslav, yes, that's what I saw in debugger (either 1-st or 2-nd way, depending on where I stopped).
I'm mostly concerned with dependent issue 245417. I work on remote C/C++ development; if a connection is slow and a file is large, saving it to remote host takes long, so this bug is very visible on slow connections. I know in person several users who are very unhappy because of that. And if for ordinary user we can recommend to move to faster file system, this does not work for remote. It is slow by definition. If it was possible to somehow disable editing of a file while the file is being saved, and if it was possible to do this conditionally (via file attribute? or an SPI?) - this would solve the issue (for remote, in my opinion). Sure, the ideal scenario is to just fix this without disabling editing.
Another solution is to serialize the two SaveActions into single thread, so they run one by one. This has been already reported as bug 218621.
I agree with either, it's up to you guys to decide. We need this fix in 8.0.1 very much.
Created attachment 148589 [details] Wrong content saved simulating test
*** Bug 245417 has been marked as a duplicate of this bug. ***
Created attachment 148591 [details] Possible fix, please verify if it works for you
Waiting for pre-verification.
(In reply to Jaroslav Tulach from comment #13) > Waiting for pre-verification. Fix does not work. Steps in comment 2 are easy reproduced. I used host with "normalization" time about 1 second. I.e.: WARNING [org.openide.filesystems.FileUtil]: FileUtil.normalizeFile(<net file>) took 1,106 ms. Ping to host is about 220 ms.
> Fix does not work. That is surprising. > Steps in comment 2 are easy reproduced. I see no steps in comment 2. Just a sentence "This can also be reproduced via setting breakpoint(s) in debugger."
(In reply to Jaroslav Tulach from comment #15) > > Fix does not work. > > That is surprising. > > > Steps in comment 2 are easy reproduced. > > I see no steps in comment 2. Just a sentence "This can also be reproduced > via setting breakpoint(s) in debugger." sorry, steps in comment #1: Then, at last file line, I added "1", pressed Ctrl-S, then added "2", pressed Ctrl-S, then "3", Ctrl-S, "4", Ctrl-S, etc. File stuck at "123", while in editor I saw 12345678.
You must be doing something wrong. The patch works. Have you applied it correctly before testing? What I did. I've added Thread.sleep(1000) at FileObj.java:130 and then I typed 1, Ctrl-S, 2, Ctrl-S, 3...., Ctrl-S, 9, Ctrl-S. The file on disk correctly contains 123456789 at the end. I polish my fix and integrate into default branch.
changeset: 7d16d2b0603d parent: 703ecc6b7c50 user: Jaroslav Tulach <jtulach@netbeans.org> date: Wed Aug 13 10:47:14 2014 +0200 summary: #246127: Prevent concurrent saves, don't start the next one before the prior one finishes.
(In reply to Jaroslav Tulach from comment #18) > changeset: 7d16d2b0603d > parent: 703ecc6b7c50 > user: Jaroslav Tulach <jtulach@netbeans.org> > date: Wed Aug 13 10:47:14 2014 +0200 > summary: #246127: Prevent concurrent saves, don't start the next one > before the prior one finishes. I check the fix 7d16d2b0603d. It does not work. My steps is: - add to favorites far far auto mounted host - create new empty file with name "main.c" - open file in editor. - start type "1" Ctrl+S "2" Ctrl+S ... undo Ctrl+S "1" Ctrl+S "2" Ctrl+S after series of 3 - 10 Ctrl+S I have a describes state: - Editor header is bold, icon save is disabled. - Editor cannot be closed without data lost - real file content does not equal editor content
Didn't you once promise to write a test to simulate your problem? I wrote a test myself, but apparently you are facing different problem than I can imagine. Can you improve my test to start to fail? That would help. The other possible way to help is to generate log files. There is "org.openide.text.CloneableEditorSupport" logger, turn it on and let see if something shows in the output.
Created attachment 148695 [details] IDE log with -Dorg.openide.text.CloneableEditorSupport.level=FINEST
Field DataObject.modif is not volatile, but it is accessed from different threads. I tried: private volatile boolean modif = false; It does not help.
Created attachment 148697 [details] two loggers Probably DO & CES loggers can help
It seems main problem is in implementation of Savable.save in CppEditorSupport: public void save() throws IOException { CppEditorSupport.this.saveDocument(); CppEditorSupport.this.getDataObject().setModified(false); } Commenting second line in the method resolves the problem (with volatile modif). Probably such savable pattern is used in other data objects.
(In reply to Alexander Simon from comment #21) > Created attachment 148695 [details] > IDE log with -Dorg.openide.text.CloneableEditorSupport.level=FINEST OK, so the problem is not that the stream would accidentally override each other, but rather that the document is considered unmodified. See FINE [org.openide.text.CloneableEditorSupport]: main.c: saveDocument() started. FINE [org.openide.text.CloneableEditorSupport]: main.c No save performed because cesEnv().isModified() == false however I don't see your attempts to type something into the document. There is probably no logging for that. Would you identify them manually in the log?
(In reply to Jaroslav Tulach from comment #25) > OK, so the problem is not that the stream would accidentally override each > other, but rather that the document is considered unmodified. See > > FINE [org.openide.text.CloneableEditorSupport]: main.c: saveDocument() > started. > FINE [org.openide.text.CloneableEditorSupport]: main.c No save performed > because cesEnv().isModified() == false > > however I don't see your attempts to type something into the document. There > is probably no logging for that. Would you identify them manually in the log? Be sure that "saveDocument()" called after typing char in editor. I hope that my comment #24 finds the problem?
> Probably such savable pattern is used in other data objects. Really? Actually, this is the question we should always start with when getting report from CND: Have you seen the problem with any other file type?
(In reply to Jaroslav Tulach from comment #27) > > Probably such savable pattern is used in other data objects. > > Really? Actually, this is the question we should always start with when > getting report from CND: Have you seen the problem with any other file type? See XMLDataObject.XMLEditorSupport.Save.
OK. So we know there is a bug in CND and XML. Why assigning the issue to platform? Honestly, I see no reason doing so. There is SimpleES in openide.loaders, but if that one does not suffer from the problem, then issues need to be raised against implementations that do suffer from the problem. There is little platform can do, imho.
(In reply to Jaroslav Tulach from comment #29) > OK. So we know there is a bug in CND and XML. Why assigning the issue to > platform? Honestly, I see no reason doing so. There is SimpleES in > openide.loaders, but if that one does not suffer from the problem, then > issues need to be raised against implementations that do suffer from the > problem. There is little platform can do, imho. C/C++ data object were fixed in change set: http://hg.netbeans.org/cnd-main/rev/940335c470e7 Reassign back to platform. Still there is no proof that DataObject.modif is visible from different threads. The fix 7d16d2b0603d + 940335c470e7 + volatile modif works for C/C++ files.
Integrated into 'main-silver', will be available in build *201408190001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-silver/rev/940335c470e7 User: Alexander Simon <alexvsimon@netbeans.org> Log: fixing Bug #246127 Last edited data loss after saving a file on a slow file system - fixed C/C++ data objects
By the way the bug #206513 should fix C/C++ data object (but did not fix).
Jarda, It's me who was going to write a test and didn't to that - sorry. I just decided that a description of how to reproduce via setting breakpoints in debugger is even more convenient (for me, it is). I'm sorry for that. I also apologize for the CND bug - calling getDataObject().setModified(false). Anyhow it's fixed now. However I'd like to note that such approach presented - apart from CND - in *many* editor supports in many modules; then some of these places were fixed by a single commit Miloslav Metelka in 2013. But some were not. This incorrect pattern still present in J2MEDataObject (mobility.editor) PageFlowElement (web.jsf.navigation) SourceMultiViewElement (websvc.design) JaxWsJavaEditorSupport (websvc.design) XmlMultiViewEditorSupport (xml.multiview) Fixed by Miloslav Metelka in 2013 in GsfDataObject (csl.api) TplEditorSupport (php.smarty) WadlEditorSupport (websvc.rest.wadl.design) TextEditorSupport (xml) I tested your and Alexender's fixes. Them both in combination work perfectly, thank you! Alexander's fix without your one seem to fix user data loss, but seveal error messages "Can not write to locked file" appear. Jarda, I have two questions to you: 1) I see that changes weren't propagate from ergonomics (push-ergonomics is blue, but last ran right before you pushed the fix). Whom should I ask to propagate them? 2) Would you mind making DataObject.modif volatile? I don't have any specific evidence, but it is used in open API and is for sure used from different threads without any sync, which is definitely not correct. If you don't mind, then I can make this change and push it into cnd-main.
(In reply to Vladimir Kvashin from comment #33) > 2) Would you mind making DataObject.modif volatile?... Sorry, I din't notice dependency on issue 246546. However I'd like to restate the question instead of removing it. We'll have to create a patch soon after 8.01 (this issue is one of the reasons for that). Your fix for the issue 246546 does not seem simple and safe enough to include it in the patch. I propose that we just add "volatile" in the patch and leave your fix in trunk. Are you OK with that?
> include it in the patch. I propose that we just add "volatile" in the patch > and leave your fix in trunk. Are you OK with that? +1 please add volatile to modif field of DataObject on release801 branch. btw. I started the push-ergonomics job few minutes ago.
> > This incorrect pattern still present in > J2MEDataObject (mobility.editor) > PageFlowElement (web.jsf.navigation) > SourceMultiViewElement (websvc.design) > JaxWsJavaEditorSupport (websvc.design) > XmlMultiViewEditorSupport (xml.multiview) AFAIK XmlMultiViewEditorSupport was fixed in Jan 2013 as part of issue #206513. I've fixed remaining ones (there was one more): http://hg.netbeans.org/jet-main/rev/10fcc672489b
(In reply to Miloslav Metelka from comment #36) > AFAIK XmlMultiViewEditorSupport was fixed in Jan 2013 as part of issue > #206513. I could be mistaken, sorry. Thank you for fixing the rest!
Integrated into 'main-silver', will be available in build *201408220001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-silver/rev/7d16d2b0603d User: Jaroslav Tulach <jtulach@netbeans.org> Log: #246127: Prevent concurrent saves, don't start the next one before the prior one finishes.
All fixes are now in cnd-main. Everything seems to work perfect.
Integrated into 'main-silver', will be available in build *201408230001* on http://bits.netbeans.org/dev/nightly/ (upload may still be in progress) Changeset: http://hg.netbeans.org/main-silver/rev/10fcc672489b User: Miloslav Metelka <mmetelka@netbeans.org> Log: #246127 - Last edited data loss after saving a file on a slow file system - fixed remaining editor supports to not call DataObject.setModified(false).
Just as a confirmation: all 3 change sets transplanted into release801 branch of hg.netbeans.org/releases repository: changeset: 287714:5d3210f5e2b8 branch: release801_fixes user: Miloslav Metelka <mmetelka@netbeans.org> date: Thu Aug 21 15:36:00 2014 +0200 summary: #246127 - Last edited data loss after saving a file on a slow file system - fixed remaining editor supports to not call DataObject.setModified(false). changeset: 287713:c4d0962b5064 branch: release801_fixes user: Jaroslav Tulach <jtulach@netbeans.org> date: Wed Aug 13 10:47:14 2014 +0200 summary: #246127: Prevent concurrent saves, don't start the next one before the prior one finishes. changeset: 287704:08b43da10782 branch: release801_fixes user: Alexander Simon <alexvsimon@netbeans.org> date: Mon Aug 18 17:41:43 2014 +0400 summary: fixing Bug #246127 Last edited data loss after saving a file on a slow file system