Bug 191321

Those files are the for years. I don't know what was changed recently, which caused this problem. They are not external binaries, they are build from sources in netbeans repository. This is the same case as with nbexex.dll, nbexec.exe, app.exe, pre7_app.exe and other binary files in netbeans repository.

(In reply to comment #1)
> I don't know what was changed recently, which caused this problem.

You modified the files recently. The changesets adding the files initially were already in all repos so did not trigger the hook.

> They are not external binaries, they are build from
> sources in netbeans repository.

Nonetheless compiled executables would better not be stored in the repo.

> This is the same case as with nbexec.dll [...]
> and other binary files in netbeans repository.

These should probably be fixed too.

This problem was introduced by the following commit:

http://hg.netbeans.org/nb-hooks/rev/9bc05eed1e59

Looking at the forbid_external.py, the easiest way is to remove dll from the list of forbidden extensions. Since .exe, .so, .sl, .jnilib are not there, I don't understand why we should block files with dll extensions.

Those other extensions should likely be added. It doesn't matter much what is done in the short term to get this one changeset through.

(In reply to comment #4)
> Those other extensions should likely be added. It doesn't matter much what is
> done in the short term to get this one changeset through.

I am not talking about this one changeset. In fact I do not see a reason, why these files should be treated as external binaries. They are not external, their size is small and they are changed very sporadically. I understand that having binary files in repository is not very nice, but the real problem here is that we don't have the infrastructure to build them as part of the ordinary build process.

(In reply to comment #5)
> They are not external

Doesn't really matter.

> their size is small and they are changed very sporadically

So not a high priority.

> we don't have the infrastructure to build them as part of the ordinary
> build process.

Of course, and we probably never will, unless someone ports gcc to Java and provides open-source copies of every relevant header file. (Fortunately we use JNA for most native integration in the IDE, which avoids the issue.)

The profiler DLLs are handled manually on demand. It means, pretxnchangegroup.binary hook is switched off for a commit of DLLs into a specific single repo and it is switched on back after a push-to-* job successfully distribute DLLs to that team repo. Repositories are processed one by one.

push-to-core-main (IIRC) was just broken by this recently. Anyway any solution which requires manually disabling hooks temporarily is not acceptable. Either the binary files should be uploaded using ExternalBinaries probably in a single ZIP, or *.dll & *.so should be removed from the blacklist (probably bad), or the hook should permit files under, say, 64Kb.

Using ExternalBinaries is probably best since lib.profiler/release/lib/deployed has 2.3Mb of binary content overall. I could make this change if agreed.

(In reply to comment #8)
> push-to-core-main (IIRC) was just broken by this recently. Anyway any solution
> which requires manually disabling hooks temporarily is not acceptable. Either
> the binary files should be uploaded using ExternalBinaries probably in a single
> ZIP, or *.dll & *.so should be removed from the blacklist (probably bad), or
> the hook should permit files under, say, 64Kb.
> 
> Using ExternalBinaries is probably best since lib.profiler/release/lib/deployed
> has 2.3Mb of binary content overall. I could make this change if agreed.

As I wrote previously, the easiest way is to remove dll from the list of the forbidden extensions. I understand the need to block zips and jars, but how many problem were prevented by blocking dlls? I would like to see the fix for platform and netbeans launcher binaries first. Even if this is fixed in netbeans, we still have to verify that moving profiler binaries to external binaries does not break VisualVM, integration of Java VisualVM into JDK and integration of VisualVM into Linux distributions.  They cannot use external binaries and need to be completely build-able from sources.

(In reply to comment #9)
> Even if this is fixed in
> netbeans, we still have to verify that moving profiler binaries to external
> binaries does not break VisualVM, integration of Java VisualVM into JDK and
> integration of VisualVM into Linux distributions.  They cannot use external
> binaries and need to be completely build-able from sources.

http://hg.netbeans.org/binaries/ is effectively a part of sources, merely kept outside the Hg repo for technical SCM reasons (would not be true if using Subversion, for example). When we create a source ZIP the necessary binaries are included by the build process, so they do not need to be downloaded.

http://hg.netbeans.org/nb-hooks/rev/7ba514fff57c