Bug 195839

Hi all,
I also noticed very strange behavior. Some pages are reported to be missing from the webdav location even though they were the re before and also my sync script report they are still in place.

eg: http://platform.netbeans.org/screenshots.html

From the page above, there is strange breadcrumbs stating that 

netbeans.org> projects > qa > Website 

Which is obviously wrong! And that leads to the fact that page seems to be missing as it is not in qa module.

Please fix it ASAP.

Thanks

I have experienced this problem too a few times randomly and have seen this reported by various users about a variety of pages on platform.netbeans.org.

See also #195912 (Missing Content on netbeans.org Web site), which may be a related issue.

Marco, any updates/comments on this ?

*** Bug 195826 has been marked as a duplicate of this bug. ***

*** Bug 195912 has been marked as a duplicate of this bug. ***

Redirects seem to be completely broken. Is someone working on this, since it is a P1? I went to edu.netbeans.org and ended up on the Profiler page, some other random pages, and then somehow ended up on edu.netbeans.org in the end. Very weird.

The not so good news is that it looks more and more like an Apache problem:-(

Some background:
The handling of <project>.<basedomain> (eg. project => edu, basedomain => netbeans.org ==> edu.netbeans.org) is pretty complicated. It needs to decide between three different ways to serve the right content:
* Default project home ==> forward to http://<basedomain>/projects/<project>/
  That's the default when the project admins don't do anything special.
* Let Apache serve files from a directory where the project admins uploaded
  content. That's our `website' feature.
* Merge the uploaded content with the normal site header/footer/sidebar...
  That's what we call `website with navigation'.

In order to do that, we have some mod_rewrite magic in Apache. That interfaces with an external script to make (cached) webservice calls to the backend to decide how to handle the requests for a particular project.

I instrumented that external helper script to log what it sees as requests from Apache and what it decides to hand back based on it's cache or remote calls. At that point, everything looks consistent.

So my current guess is, that somehow the Apache threads stumble over each other when trying to talk to that external RewriteMap script:-(

Unfortunately, I don't know more at this point:-(

Since the problem is the handling of <project>.<basedomain>, you should move the content of those sites on to the www.netbeans.org main site. Create subfolders within netbeans.org for 'edu', 'platform', etc, and put the content in there and then kill all those other weird domains.

OK, I filed a bug @ apache:-( https://issues.apache.org/bugzilla/show_bug.cgi?id=50916

I'm also trying a patch right now @netbeans.org. The problem is that the bug happens only after a long runtime:-( So I don't know if my patch will actually improve the situation:-(

Last I checked, one of the two NB-apaches showed the problem after running for 2.5 days.

Please comment when you see problems!!

Thanks,
-- Marco

I filed issue #195912, which was later marked as a duplicate of this issue.  I am not convinced it's the same root cause, but at any rate, I am still observing most of the problems I reported last month.

A few of the URLs from my bug report are fixed, but several still show error messages instead of content:

   http://openide.netbeans.org/versioning-policy.html

   http://core.netbeans.org/proposals/threading/

   http://netbeans.org/projects/jellytools/docs.html

I was trying to update some documentation and found other similarly broken pages, including:

   http://xtest.netbeans.org/NbJUnit/NbJUnit-overview.html

That last one seems to expose another problem, namely that five of the items under "Project Links" are in Japanese.  I am not Japanese, do not read Japanese and have never set my browser or operating system's locale to Japanese.  

I suppose this is related to the error "The file /web/projects/ja/NbJUnit/NbJUnit-overview.html appears to be missing" (which contains 'ja' for the Japanese locale), but you will see from the URL that nothing there indicates I requested a page in Japanese.  The HTTP Accept-Language header from my request had a value of "en-us,en;q=0.5".

All other pages on the NetBeans site, even the three broken ones I listed above, show the "Project Links" in English as I'd expect.

Yes, my patch did not really help:-(

The problem seems to be that an Apache thread disappears after sending a request [line] to a helper program. So all the following exchanges are off by one.

So far I was not able to find out what happens to that thread:-( Apache does not log any error conditions and somehow the mutexes held by that thread get released.

The problem shows up after two++ days of running in a production setup. So any additional tests are very slow:-(

Sorry, but that's all I know so far.

Here's another example of a page with missing content (though no Japanese text):

   http://ui.netbeans.org/docs/ui/code_folding/cf_uispec.html

Does it help for me to provide further examples? If so, I will update the issue as I find them.  If it doesn't add anything useful for you, let me know and I will stop posting them.

(In reply to comment #11)
>    http://openide.netbeans.org/versioning-policy.html

This gives me a redirect to wiki.netbeans.org (as intended).

(In reply to comment #14)
>>    http://openide.netbeans.org/versioning-policy.html
> This gives me a redirect to wiki.netbeans.org (as intended).

I just tried it again, and it still gives me the same message:

   The file /web/projects/ja/versioning-policy.html appears to be missing.

   To add content to your website, use webdavs.

And the "Project Links" are still in Japanese.

What is interesting is that I am now on a different computer (home instead of Work) running a different OS (Linux instead of Windows) than when I originally reported the problem.

I know it sounds crazy, so I'll attach a screenshot as evidence.

Created attachment 107142 [details]
screenshot illustrating error message and Japanese characters in sidebar

I face the same issue with exception of different redirection. I get redirected to "qa" module instead of Tom's "ja" module. My error message is this:

The file /web/projects/qa/versioning-policy.html appears to be missing.

New patched Apache is running @ netbeans.org. Let's see how this goes.

It looks like my last Apache patch fixed this. It's not ideal but a workaround for now:-(

Yes, I agree. This really seems to be fixed. Thanks Marco!

I also agree.  I have not seen this problem in a few weeks -- thanks for fixing it.