This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 237355 - `nbproject/qt-Debug.mk': Permission denied issue
Summary: `nbproject/qt-Debug.mk': Permission denied issue
Status: CLOSED FIXED
Alias: None
Product: cnd
Classification: Unclassified
Component: Project (show other bugs)
Version: 7.4
Hardware: PC Windows 7
: P2 normal (vote)
Assignee: igor_nikiforov
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-10-19 18:48 UTC by brodovv
Modified: 2013-11-13 08:25 UTC (History)
2 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Adds more support for Qt5, including qt-Debug.mk locking (8.70 KB, patch)
2013-10-25 23:52 UTC, adipose
Details | Diff
1 of 2 jar files with patch integrated. (986.54 KB, application/octet-stream)
2013-10-25 23:56 UTC, adipose
Details
2 of 2 jar files with patch integrated. (4.06 MB, application/x-xpinstall)
2013-10-25 23:57 UTC, adipose
Details
Patch for qt-Debug.mk locking (2.25 KB, patch)
2013-10-29 20:33 UTC, adipose
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description brodovv 2013-10-19 18:48:21 UTC
Product Version = NetBeans IDE 7.4 (Build 201310111528)
Operating System = Windows 7 version 6.1 running on amd64
Java; VM; Vendor = 1.7.0_11
Runtime = Java HotSpot(TM) 64-Bit Server VM 23.6-b04

"/C/msys/bin/make.exe" -f nbproject/Makefile-Debug.mk QMAKE=/D/Qt/4.8.5/bin/qmake.exe SUBPROJECTS= .build-conf
make.exe[1]: Entering directory `/d/Users/brodovv/BTSync/Applications/UltraMARS'
/D/Qt/4.8.5/bin/qmake.exe VPATH=. -spec win32-g++ -o qttmp-Debug.mk nbproject/qt-Debug.pro
mv -f qttmp-Debug.mk nbproject/qt-Debug.mk
mv: cannot move `qttmp-Debug.mk' to `nbproject/qt-Debug.mk': Permission denied
make.exe[1]: *** [nbproject/qt-Debug.mk] Error 1
make.exe[1]: Leaving directory `/d/Users/brodovv/BTSync/Applications/UltraMARS'
make: *** [.build-impl] Error 2

When delete file manually build works perfectly but only once. On second run the same issue arises.
Comment 1 brodovv 2013-10-19 19:06:29 UTC
When exit and start ide again build also successes. On second try to build is causing the same issue.
Comment 2 adipose 2013-10-25 23:52:45 UTC
Created attachment 141571 [details]
Adds more support for Qt5, including qt-Debug.mk locking

This patch contains two things:

1. Adds support for the "printsupport" qmake option checkbox.
2. Works around locking of .mk files in QtInfoProvider.java

I have not identified the precise workflow, so my patch could be better.  But, I observed that getQtAdditionalMacros opens (and locks) the qt-Debug.mk file at approximately the same time that the build script is initiated.  The build script needs to regenerate qt-Debug.mk, but cannot due to getQtAdditionalMacros locking it at the same time.

My solution was to cp the .mk file to a ".nb" file during the make command, and ignore any errors in that cp command.  Then instead of reading the ".mk", getQtAdditionalMacros reads the ".nb" file.  Thus, the only file to get locked by Netbeans will be the .nb file, and if the script fails to create the .nb file, there is not much harm (compared to failing to build!).

Hope this helps.
Comment 3 adipose 2013-10-25 23:56:29 UTC
Created attachment 141572 [details]
1 of 2 jar files with patch integrated.
Comment 4 adipose 2013-10-25 23:57:44 UTC
Created attachment 141573 [details]
2 of 2 jar files with patch integrated.
Comment 5 brodovv 2013-10-26 08:44:38 UTC
Hi 
It seems to be working. Thanks for fast turnaround and fix. 

Regards,
 Vladimir.
Comment 6 Vladimir Voskresensky 2013-10-26 16:15:24 UTC
Adipose, thanks a lot for your contribution. Igor, please, review and integrate.
Comment 7 adipose 2013-10-27 03:56:42 UTC
Since netbeans only is looking for the CXXFLAGS, instead of copying the .mk file, you could grep CXXFLAGS, to optimize a bit.
Comment 8 adipose 2013-10-27 03:57:35 UTC
(In reply to brodovv from comment #5)
> Hi 
> It seems to be working. Thanks for fast turnaround and fix. 
> 
> Regards,
>  Vladimir.

Welcome.  I suffered from the same bug myself so I fixed it.  Very frustrating to have to restart the IDE to build :)
Comment 9 igor_nikiforov 2013-10-28 13:53:38 UTC
Thanks a lot for you patch!

I'm suggesting a little bit different fix: in previous implementation input stream was not closed properly, which could result in described error on Windows.

My fix is here:
  http://hg.netbeans.org/cnd-main/rev/f8c04096315a

Unfortunately, I cannot reproduce this issue: I've tried it on both Windows XP and Windows 7. So it's hard for me to verify either your or my fixes.

It would be great if you try the nightly with my fix and let me know if it works or not.
Comment 10 adipose 2013-10-28 20:23:27 UTC
(In reply to igor_nikiforov from comment #9)
> Thanks a lot for you patch!
> 
> I'm suggesting a little bit different fix: in previous implementation input
> stream was not closed properly, which could result in described error on
> Windows.
> 
> My fix is here:
>   http://hg.netbeans.org/cnd-main/rev/f8c04096315a
> 
> Unfortunately, I cannot reproduce this issue: I've tried it on both Windows
> XP and Windows 7. So it's hard for me to verify either your or my fixes.
> 
> It would be great if you try the nightly with my fix and let me know if it
> works or not.

Igor,

1. Link did not work for me.
2. I analayzed the code carefully and determined that the inputstream does get closed by the "aslines()" macro.  It reads all lines from the file and puts them in the list.  The "hasnext() will be called repeatedly, and closes the inputstream as soon as it fails to read the line.  I confirmed that the bug persisted even though this behavior should theoretically close the file.  So, I don't think this was the issue.
3. I am running Windows 8.1, but I don't know if that is the issue.  It may be the speed of my disk (Crucial CT960M500) or processor (i7-4770) as well.
4. I will try out dev. build, but the printersupport patch would be nice to get integrated.
Comment 11 adipose 2013-10-28 20:32:08 UTC
(In reply to igor_nikiforov from comment #9)
> Thanks a lot for you patch!
> 
> I'm suggesting a little bit different fix: in previous implementation input
> stream was not closed properly, which could result in described error on
> Windows.
> 
> My fix is here:
>   http://hg.netbeans.org/cnd-main/rev/f8c04096315a
> 
> Unfortunately, I cannot reproduce this issue: I've tried it on both Windows
> XP and Windows 7. So it's hard for me to verify either your or my fixes.
> 
> It would be great if you try the nightly with my fix and let me know if it
> works or not.

Igor, I tried latest dev build and the issue is still there.
Comment 12 rob.johnson 2013-10-29 08:40:30 UTC
(In reply to adipose from comment #10)


> 3. I am running Windows 8.1, but I don't know if that is the issue.  It may
> be the speed of my disk (Crucial CT960M500) or processor (i7-4770) as well.

Hi,

I have been experiencing the same problem on Widnows 7 Professional SP1 (64 bit).  I tried the nightly build (29/10/13) and still experienced the issue.  Let me know if you need more info or I can test anything else.

Thanks for the support,

Rob.
Comment 13 igor_nikiforov 2013-10-29 09:32:47 UTC
There are some infrastructure troubles on NetBeans side, so the latest nightly do not contain my fix.

This issue will be automatically updated by robot when integration happen.

Meanwhile I try to reproduce the issue myself: QA reports that they have seen this issue, but on semi-large Qt projects. It looks like it is not possible to reproduce it on Hello Qt World.
Comment 14 igor_nikiforov 2013-10-29 09:40:49 UTC
(reply )(In reply to adipose from comment #10)
> 1. Link did not work for me.

Do you mean that you have import this patch and tried you test case?

> 2. I analayzed the code carefully and determined that the inputstream does
> get closed by the "aslines()" macro.  It reads all lines from the file and
> puts them in the list.  The "hasnext() will be called repeatedly, and closes
> the inputstream as soon as it fails to read the line.  I confirmed that the
> bug persisted even though this behavior should theoretically close the file.
> So, I don't think this was the issue.

The problem is that it is closed at the end of iteration, not when value return happens. And what happen if iteration won't be finished?

> 3. I am running Windows 8.1, but I don't know if that is the issue.  It may
> be the speed of my disk (Crucial CT960M500) or processor (i7-4770) as well.
> 4. I will try out dev. build, but the printersupport patch would be nice to
> get integrated.

Could you, please, file another enchantment for printersupport. It doesn't connected with this issue and we are trying not to mix changes, related to different issues with some changesets.
Comment 15 igor_nikiforov 2013-10-29 09:46:09 UTC
> 4. I will try out dev. build, but the printersupport patch would be nice to
> get integrated.

Forget to answer one question about printsupport. Usually we asking people to use additional options (Project Options -> Build -> Qt -> Custom Definitions) if they want to enable some additional qt modules. Does this works for you?

The problem with printsupport, is that it is Qt 5 only (may be I'm wrong?) and we are trying to keep UI compatible with Qt4 as well.
Comment 16 adipose 2013-10-29 16:50:39 UTC
(In reply to igor_nikiforov from comment #14)
> (reply )(In reply to adipose from comment #10)
> > 1. Link did not work for me.
> 
> Do you mean that you have import this patch and tried you test case?

I clicked on the link and it never loaded.

> 
> > 2. I analayzed the code carefully and determined that the inputstream does
> > get closed by the "aslines()" macro.  It reads all lines from the file and
> > puts them in the list.  The "hasnext() will be called repeatedly, and closes
> > the inputstream as soon as it fails to read the line.  I confirmed that the
> > bug persisted even though this behavior should theoretically close the file.
> > So, I don't think this was the issue.
> 
> The problem is that it is closed at the end of iteration, not when value
> return happens. And what happen if iteration won't be finished?

I thought this as well.  But if you read the "aslines()" carefully, you will see that it actually iterates through the whole file and returns a list of strings (which results in the file being closed).  Then in qtinfoprovider it loops through the aslines() returned list.

The loop in qtinfoprovider can terminate early, but it has no effect on the file being closed.  The file is closed before that loop even begins (as soon as aslines() returns, the file is already closed).

> 
> > 3. I am running Windows 8.1, but I don't know if that is the issue.  It may
> > be the speed of my disk (Crucial CT960M500) or processor (i7-4770) as well.
> > 4. I will try out dev. build, but the printersupport patch would be nice to
> > get integrated.
> 
> Could you, please, file another enchantment for printersupport. It doesn't
> connected with this issue and we are trying not to mix changes, related to
> different issues with some changesets.

Sure, I think it is harmless for QT4 because the user will not need to check it in QT4.  Perhaps it can say "PrintSupport (QT5 only)".  Or do more work and have it detect QT version.
Comment 17 adipose 2013-10-29 20:33:03 UTC
Created attachment 141659 [details]
Patch for qt-Debug.mk locking
Comment 18 adipose 2013-10-29 20:47:46 UTC
Igor, I was able to see your patch now (it would not load before, on multiple attempts).  But, I don't think it will solve the issue per my other comment.

The file is successfully closed (by asLines()), it just happens to be open at the same time as the build script.

If you follow the convoluted function calls below, you can see that as it slurps the file, it does call "hasNext", which closes the file when it reaches the end.  Kind of sloppy, but it seems to work (maybe with Exceptions it would fail).  So, by the time asLines() returns, there is no open file anymore.

    public List<String> asLines(final String encoding) throws IOException {
        return new FileObjectLines(encoding, this);
    }

    public FileObjectLines(String encoding, FileObject fo) throws IOException {
        super();
        this.fo = fo;
        this.ready = new FileObjectLineIterator(this, encoding);
    }


    public FileObjectLineIterator(FileObjectLines lines, String encoding) throws IOException {
        this.lines = lines;
        this.encoding = encoding;
        initReader();
    }


    private void initReader() throws UnsupportedEncodingException, IOException {
        InputStream is;
        if (lines.fo.getSize() < 64 * 1024) {
            if (buffer == null) {
                buffer = lines.fo.asBytes();
            }
            is = new ByteArrayInputStream(buffer);
        } else {
            is = lines.fo.getInputStream();
        }
        this.reader = new BufferedReader(new InputStreamReader(is, encoding));
        this.index = 0;
        while (this.index < index) {
            next();
        }
    }

    public synchronized String next() {
        if (!hasNext()) {
            throw new NoSuchElementException();
        }
        String l = line;
        line = null;
        return l;
    }

    public synchronized boolean hasNext() {
        if (line == null) {
            try {
                line = reader.readLine();
                if (line == null) {
                    reader.close();
                }
                index++;
            } catch (IOException ex) {
                Exceptions.printStackTrace(ex);
            }
        }
        return line != null;
    }
Comment 19 adipose 2013-10-29 20:53:27 UTC
>         while (this.index < index) {
>             next();
>         }

Hmm, this looks suspicious, actually.  Does this actually work?
Comment 20 adipose 2013-10-29 23:01:14 UTC
Igor, looks like I was wrong, it doesn't slurp the file.  There appears to be some dead code at the end of the initreader().

I tried your bufferedreader edition and it seems to fix the issue.

Thanks!
Comment 21 igor_nikiforov 2013-10-30 07:47:38 UTC
Thanks for investigation!

The build is not happened yet, but if you still see the issue after it happens please reopen the bug.

And thank you for filling separate issue for printsupport!
Comment 22 soldatov 2013-10-30 16:54:03 UTC
Verified in NetBeans IDE 7.4 (Build cnd-build-10629-on-20131030). In that build I can't reproduce "permission denied" issue.

>System Info: 
  Product Version         = NetBeans IDE 7.4 (Build cnd-build-10629-on-20131030) (#ab5e89548d97)
  Operating System        = Windows 7 version 6.1 running on amd64
  Java; VM; Vendor        = 1.8.0-ea; Java HotSpot(TM) 64-Bit Server VM 25.0-b53; Oracle Corporation
  Runtime                 = Java(TM) SE Runtime Environment 1.8.0-ea-b111
  Java Home               = C:\Program Files\Java\jdk1.8.0\jre
  System Locale; Encoding = ru_RU (nb); Cp1251
Comment 23 rob.johnson 2013-10-31 10:06:09 UTC
(In reply to soldatov from comment #22)

I still see the issue in the nightly build:

Product Version: NetBeans IDE Dev (Build 201310310001)
Java: 1.7.0_25; Java HotSpot(TM) 64-Bit Server VM 23.25-b01
Runtime: Java(TM) SE Runtime Environment 1.7.0_25-b16
System: Windows 7 version 6.1 running on amd64; Cp1252; en_GB (nb)
User directory: C:\Users\RJohnson\AppData\Roaming\NetBeans\dev
Cache directory: C:\Users\RJohnson\AppData\Local\NetBeans\Cache\dev
Comment 24 igor_nikiforov 2013-10-31 10:31:44 UTC
(In reply to rob.johnson from comment #23)
> (In reply to soldatov from comment #22)
> 
> I still see the issue in the nightly build:
> 
> Product Version: NetBeans IDE Dev (Build 201310310001)
> Java: 1.7.0_25; Java HotSpot(TM) 64-Bit Server VM 23.25-b01
> Runtime: Java(TM) SE Runtime Environment 1.7.0_25-b16
> System: Windows 7 version 6.1 running on amd64; Cp1252; en_GB (nb)
> User directory: C:\Users\RJohnson\AppData\Roaming\NetBeans\dev
> Cache directory: C:\Users\RJohnson\AppData\Local\NetBeans\Cache\dev

Valera Soldatov is talking about internal build verification. AFAICS, changes are not in nightly yet.
Comment 25 brodovv 2013-11-03 13:21:08 UTC
The original issue is fixed in:

Product Version: NetBeans IDE Dev (Build 201311020744)
Java: 1.7.0_11; Java HotSpot(TM) 64-Bit Server VM 23.6-b04
Runtime: Java(TM) SE Runtime Environment 1.7.0_11-b21
System: Windows 7 version 6.1 running on amd64; Cp1251; en_US (nb)
Comment 26 Quality Engineering 2013-11-11 20:13:54 UTC
Integrated into 'releases/release74', will be available in build *201311111738* or newer. Wait for official and publicly available build.

Changeset: http://hg.netbeans.org/releases/rev/85e93c98ff1b
User: igor_nikiforov <igor_nikiforov@netbeans.org>
Log: Bug 237355 - `nbproject/qt-Debug.mk': Permission denied issue - fix.
(transplanted from f84a5012e6d27ddd3ab759467864b40f89da24e7)
Comment 27 soldatov 2013-11-12 17:48:17 UTC
verified in NetBeans IDE 7.4 (Build 201311111738)