Bug 195839

Summary: Strange redirects between project homepages
Product: www Reporter: Jiri Kovalsky <jkovalsky>
Component: AdminAssignee: Marco Walther <marcow>
Status: VERIFIED FIXED QA Contact: Admins <izadmins>
Priority: P1 CC: jglick, mmirilovic, pjiricka, tomwheeler, vkraemer, zolta
Version: 7.0   
Target Milestone: 7.0   
Hardware: PC   
OS: Linux   
Whiteboard:
Issue Type: DEFECT Exception Report:
Bug Depends on:    
Bug Blocks: 181761    
Attachments: screenshot illustrating error message and Japanese characters in sidebar

Description Jiri Kovalsky 2011-02-23 10:05:53 UTC
We are experiencing weird behaviour of netbeans.org project homepages. It is very difficult to get to the desired project, because for example qa.netbeans.org was redirected to cnd.netbeans.org, dreamteam.netbeans.org or db.netbeans.org but never to the correct site!

Was there another deployment of the netbeans.org instance recently? Marco, please help! Thanks a lot.
Comment 1 Jan Pirek 2011-02-23 13:31:38 UTC
Hi all,
I also noticed very strange behavior. Some pages are reported to be missing from the webdav location even though they were the re before and also my sync script report they are still in place.

eg: http://platform.netbeans.org/screenshots.html

From the page above, there is strange breadcrumbs stating that 

netbeans.org> projects > qa > Website 

Which is obviously wrong! And that leads to the fact that page seems to be missing as it is not in qa module.

Please fix it ASAP.

Thanks
Comment 2 Geertjan Wielenga 2011-02-23 13:43:22 UTC
I have experienced this problem too a few times randomly and have seen this reported by various users about a variety of pages on platform.netbeans.org.
Comment 3 tomwheeler 2011-02-23 22:22:51 UTC
See also #195912 (Missing Content on netbeans.org Web site), which may be a related issue.
Comment 4 Marian Mirilovic 2011-02-24 07:14:34 UTC
Marco, any updates/comments on this ?
Comment 5 Jan Pirek 2011-02-24 09:59:00 UTC
*** Bug 195826 has been marked as a duplicate of this bug. ***
Comment 6 Jan Pirek 2011-02-24 10:26:56 UTC
*** Bug 195912 has been marked as a duplicate of this bug. ***
Comment 7 Geertjan Wielenga 2011-02-27 12:28:56 UTC
Redirects seem to be completely broken. Is someone working on this, since it is a P1? I went to edu.netbeans.org and ended up on the Profiler page, some other random pages, and then somehow ended up on edu.netbeans.org in the end. Very weird.
Comment 8 Marco Walther 2011-02-27 18:57:05 UTC
The not so good news is that it looks more and more like an Apache problem:-(

Some background:
The handling of <project>.<basedomain> (eg. project => edu, basedomain => netbeans.org ==> edu.netbeans.org) is pretty complicated. It needs to decide between three different ways to serve the right content:
* Default project home ==> forward to http://<basedomain>/projects/<project>/
  That's the default when the project admins don't do anything special.
* Let Apache serve files from a directory where the project admins uploaded
  content. That's our `website' feature.
* Merge the uploaded content with the normal site header/footer/sidebar...
  That's what we call `website with navigation'.

In order to do that, we have some mod_rewrite magic in Apache. That interfaces with an external script to make (cached) webservice calls to the backend to decide how to handle the requests for a particular project.

I instrumented that external helper script to log what it sees as requests from Apache and what it decides to hand back based on it's cache or remote calls. At that point, everything looks consistent.

So my current guess is, that somehow the Apache threads stumble over each other when trying to talk to that external RewriteMap script:-(

Unfortunately, I don't know more at this point:-(
Comment 9 Geertjan Wielenga 2011-02-27 21:36:14 UTC
Since the problem is the handling of <project>.<basedomain>, you should move the content of those sites on to the www.netbeans.org main site. Create subfolders within netbeans.org for 'edu', 'platform', etc, and put the content in there and then kill all those other weird domains.
Comment 10 Marco Walther 2011-03-12 00:22:59 UTC
OK, I filed a bug @ apache:-( https://issues.apache.org/bugzilla/show_bug.cgi?id=50916

I'm also trying a patch right now @netbeans.org. The problem is that the bug happens only after a long runtime:-( So I don't know if my patch will actually improve the situation:-(

Last I checked, one of the two NB-apaches showed the problem after running for 2.5 days.

Please comment when you see problems!!

Thanks,
-- Marco
Comment 11 tomwheeler 2011-03-17 21:50:53 UTC
I filed issue #195912, which was later marked as a duplicate of this issue.  I am not convinced it's the same root cause, but at any rate, I am still observing most of the problems I reported last month.

A few of the URLs from my bug report are fixed, but several still show error messages instead of content:

   http://openide.netbeans.org/versioning-policy.html

   http://core.netbeans.org/proposals/threading/

   http://netbeans.org/projects/jellytools/docs.html

I was trying to update some documentation and found other similarly broken pages, including:

   http://xtest.netbeans.org/NbJUnit/NbJUnit-overview.html

That last one seems to expose another problem, namely that five of the items under "Project Links" are in Japanese.  I am not Japanese, do not read Japanese and have never set my browser or operating system's locale to Japanese.  

I suppose this is related to the error "The file /web/projects/ja/NbJUnit/NbJUnit-overview.html appears to be missing" (which contains 'ja' for the Japanese locale), but you will see from the URL that nothing there indicates I requested a page in Japanese.  The HTTP Accept-Language header from my request had a value of "en-us,en;q=0.5".

All other pages on the NetBeans site, even the three broken ones I listed above, show the "Project Links" in English as I'd expect.
Comment 12 Marco Walther 2011-03-17 22:33:08 UTC
Yes, my patch did not really help:-(

The problem seems to be that an Apache thread disappears after sending a request [line] to a helper program. So all the following exchanges are off by one.

So far I was not able to find out what happens to that thread:-( Apache does not log any error conditions and somehow the mutexes held by that thread get released.

The problem shows up after two++ days of running in a production setup. So any additional tests are very slow:-(

Sorry, but that's all I know so far.
Comment 13 tomwheeler 2011-03-18 20:38:14 UTC
Here's another example of a page with missing content (though no Japanese text):

   http://ui.netbeans.org/docs/ui/code_folding/cf_uispec.html

Does it help for me to provide further examples? If so, I will update the issue as I find them.  If it doesn't add anything useful for you, let me know and I will stop posting them.
Comment 14 Jesse Glick 2011-03-18 23:42:25 UTC
(In reply to comment #11)
>    http://openide.netbeans.org/versioning-policy.html

This gives me a redirect to wiki.netbeans.org (as intended).
Comment 15 tomwheeler 2011-03-21 00:38:00 UTC
(In reply to comment #14)
>>    http://openide.netbeans.org/versioning-policy.html
> This gives me a redirect to wiki.netbeans.org (as intended).

I just tried it again, and it still gives me the same message:

   The file /web/projects/ja/versioning-policy.html appears to be missing.

   To add content to your website, use webdavs.

And the "Project Links" are still in Japanese.

What is interesting is that I am now on a different computer (home instead of Work) running a different OS (Linux instead of Windows) than when I originally reported the problem.

I know it sounds crazy, so I'll attach a screenshot as evidence.
Comment 16 tomwheeler 2011-03-21 00:39:18 UTC
Created attachment 107142 [details]
screenshot illustrating error message and Japanese characters in sidebar
Comment 17 Jiri Kovalsky 2011-03-21 12:24:57 UTC
I face the same issue with exception of different redirection. I get redirected to "qa" module instead of Tom's "ja" module. My error message is this:

The file /web/projects/qa/versioning-policy.html appears to be missing.
Comment 18 Marco Walther 2011-03-25 14:39:29 UTC
New patched Apache is running @ netbeans.org. Let's see how this goes.
Comment 19 Marco Walther 2011-05-12 17:55:10 UTC
It looks like my last Apache patch fixed this. It's not ideal but a workaround for now:-(
Comment 20 Jiri Kovalsky 2011-05-13 09:21:20 UTC
Yes, I agree. This really seems to be fixed. Thanks Marco!
Comment 21 tomwheeler 2011-05-16 15:54:27 UTC
I also agree.  I have not seen this problem in a few weeks -- thanks for fixing it.
By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo