This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 271768 - Concurrent file modification bugs using refactor/rename on a property
Summary: Concurrent file modification bugs using refactor/rename on a property
Status: NEW
Alias: None
Product: java
Classification: Unclassified
Component: Refactoring (show other bugs)
Version: 8.2
Hardware: PC Linux
: P3 normal (vote)
Assignee: Svata Dedic
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-11-08 13:22 UTC by hans_adler
Modified: 2017-11-10 11:00 UTC (History)
0 users

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description hans_adler 2017-11-08 13:22:59 UTC
I am the guy who does the large-scale refactorings on an Enterprise Java project that has a long history and about 140,000 lines of code (60,000 LOC Java, 80,000 LOC JSF). For at least the last four years I have been using workarounds for two special cases of the following problem. Unfortunately, I could not reproduce it in a small example project.


TYPICAL SYMPTOMS

Using refactor/rename on a field to change its name to a new one with a different length, and selecting "Rename Getters and Setters" (to rename an entire property), there is a high probability that in some of the files using the property more than once, the resulting code is garbled. Suppose we start with the following:

class A {
  int x;
  public int getX() {
    return x;
  }
  public void setX(int x) {
    this.x = x;
  }
}

class B {
  void doSomething() {
    int i = getX();
    i = i + getX();
    i = i + getX();
    setX(i);
  }
}

Trying to rename A.x to a.yz, we are likely to end up with something like the following:

class A {
  int yz;
  public int getYz() {
    return yz;
  }
  public void setYz(int yz) {
    this.yz = yz;
  }
}

class B {
  void doSomething() {
    int i = getYz();
    i = i +getYzX();
    i = i getYztX();
 setYzetX(i);
  }
}

This seems to indicate that between one thread deciding which part of the file to overwrite with which text and it actually doing this, another thread also operates on the same file.


WORKAROUND

In Java code this problem usually results in syntax errors. They can be handled by the following method:

- Search all uages of the getter. When doing this right away, the IDE finds even usages which would exist after a correct refactoring, but don't exist because the code doesn't even make sense!
- Repeat for the setter.
- In the usage lists, the broken files have a red exclamation mark icon.
- In broken files with only few references, fix them manually.
- In broken files with many references, revert to the pre-refactoring version and do the replacement manually using search/replace.

In JSF code (.xhtml) the errors are not reported, though this may be due to my settings. (In our project, NetBeans incorrectly reports a large number of errors in valid JSF code.) Here I have to update all references manually before the refactoring.


SPECIAL CASES

(1) In the first few years, this problem occurred only in the following special case: We had JSF pages (.xhtml) which somehow (not through sym links) were included in the project more than once, so that a full text search over the entire project found each of them twice. The second copies were shown grayed out at the end. When we finally found out what caused this and fixed it, the bug disappeared.

(2) More recently, the bug has reappeared even though the duplicate file inclusion issue seems to remain fixed. It now happens in Java code as well as in JSF code. For example, I just got two garbled files after renaming a property that had 43 usages in 8 files (including 2 files with only one usage each).


SPECULATION ON THE CAUSE

Apparently, refactorings first operate on a structured internal representation of the code, then update the code itself accordingly. Apparently, under certain circumstances (size of the project? size of the refactoring?) this updating is done in more than one thread, causing occasional concurrency issues.

(1) My initial special case was probably caused by NetBeans operating on an internal representation of the file rather than the file itself, which in my case was duplicated for some files, rather than the files themselves. As a result, more than one thread got the task of operating on that file, and if there was any locking, it was also on the internal representations, so that concurrent access wasn't prevented.

(2) I would speculate that the more recent problem is caused by a buggy locking method or an incorrect method for avoiding locking. It also seems possible that the renaming of the setter and the renaming of the getter are started concurrently when they should really be started consecutively.

As there have been many reports of distinct but similar problems in the past, it might be a good idea to implement strict locking on the basis of canonical representations of the file paths. Otherwise you will probably keep introducing and fixing similar bugs.
Comment 1 hans_adler 2017-11-08 22:26:17 UTC
Another speculation on the cause of (2): If some code has been converted to the Stream API recently, maybe something is now done concurrently that shouldn't be.
Comment 2 hans_adler 2017-11-10 10:16:16 UTC
After writing this report I have started using a new method for remaining properties: renaming the getter seperately, then the setter, then the field. I just learned that the bug can appear in JSF code even when renaming only a getter.
Comment 3 hans_adler 2017-11-10 10:50:37 UTC
Here is some more detail. When I just renamed a getter, leading to some field references being garbled in a JSF file, I got an extra version in the file's history. After repairing the file (so that the final result of the refactoring is now displayed in history), file history starts like this:

- Today 11:09:13 AM
- Today 11:09:12 AM
- Yesterday ...

The two "Today" versions on the list were created by the refactoring. Today 11:09:12 AM has the first usage of the property updated, the other three untouched. Today 11:09:13 AM does not change the first usage, but garbles the file as follows on the remaining usages:

The second, third and fourth usages are each written to the position that would be correct if the previous usages had not shifted the text. As my refactoring replaced getVeranstaltung by getLv in a class called Klausur, we get roughly the following picture. Before the refactoring:

- {klausur.veranstaltung.kurzname}
- {klausur.veranstaltung.kurzname}
- {klausur.veranstaltung.semester.startdatum}">
- {klausur.veranstaltung.semester.name}

First refactoring step (11:09:12):

- {klausur.lv.kurzname}
- {klausur.veranstaltung.kurzname}
- {klausur.veranstaltung.semester.startdatum}">
- {klausur.veranstaltung.semester.name}

Second refactoring step (11:09:13):

- {klausur.lv.kurzname}
- {klausur.veranstaltulv
- {klausur.veranstaltung.semesterlv>
- {klausur.veranstaltung.semester.name} ... lv

The second, third and fourth instances of "lv" each correctly replace 13 characters, but rather than replacing "veranstaltung", they replace strings that are 11, 22 and 33 characters further into the file, respectively.
Comment 4 hans_adler 2017-11-10 11:00:43 UTC
During the same refactoring described in the previous post, another file with three usages was also garbled, but differently. Again there were two versions. In the 11:09:12 version, the first usage was left untouched, the second usage had the correct replacement, and the third usage was shifted to the right by 22(!) characters as described above. Then the 11:09:13 version correctly replaced the first usage.