This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 129258

Summary: netbeans.org banned access to java applications
Product: obsolete Reporter: Dan Kolar <dkolar>
Component: collabnetAssignee: support <support>
Status: RESOLVED INVALID    
Severity: blocker CC: blaha, jcatchpoole, jhorvath, jlahoda, jtulach, mmirilovic, rbalada, rnovak, saubrecht, sustaining
Priority: P1    
Version: 6.x   
Hardware: All   
OS: All   
Issue Type: DEFECT Exception Reporter:
Attachments: patched jar that doesn't ignore serverside caches

Description Dan Kolar 2008-03-05 10:48:08 UTC
Since 03-04-2008 cca 12PM GMT netbeans.org banned access for our java application we use for synchronizing data in
issuezilla with our local database ( which was created just for minimizing traffic to netbeans.org ).
This heavily influences our infrastucture, so we need access again.
Comment 1 Marian Mirilovic 2008-03-05 11:29:54 UTC
Jack, please look at this. It's a stopper for us with request to fix this ASAP. 

We are using this way for :
- since NB 6.0 (in IDE) by reporting new issues into IZ
- our dashboard 
- all our bugs statistics

For more details look at issue 129249.
Comment 2 jcatchpoole 2008-03-05 13:14:04 UTC
I'll call Collab.
Comment 3 Unknown 2008-03-05 13:38:34 UTC
Jack,

I have updated https://sun-netbeans.extranet.collab.net/issues/show_bug.cgi?id=100 that demos.xml is being requested by
multiple ip address & it was increasing by 50 hits/second.Though we know these are valid requests, we have blocked the
access to prevent site outage.

We would like to know why there is a sudden increase from the users side to demos.xml.

Regards,
Ramya
Support Operations
Comment 4 jcatchpoole 2008-03-05 13:46:30 UTC
Thanks for the update.

cc'ing Standa - anyone have any ideas why this would be ?
Comment 5 Marian Mirilovic 2008-03-05 14:17:00 UTC
Ramaya,

When did you encounter problem with increased numbers of requests for the first time? 
You said it's being requested by multiple ips, so do you have a list of ips ordered by number of requests? 

> Though we know these are valid requests, we have blocked the access to prevent site outage.

Ok, so all of them are valid, why do you still hold them blocked ? We are approaching NB 6.1 Beta release, it means
users would very probably report new issues. If you will still block those ips, we'll lost a lot of data as well as make
our users disappointed. This is blocker for us, we need feedback from users, that's the reason we are doing Beta
releases (scheduled for tomorrow).

BTW: the link you provided is hidden for most of us. 
Comment 6 jcatchpoole 2008-03-05 14:26:54 UTC
> why do you still hold them blocked

They are probably still blocked to prevent the whole site going down.  I guess this is/was the cause of the outage we
saw yesterday, and the performance problems we have been seeing since last Friday.  If the IPs/clients were unblocked,
nb.org would probably go down completely quite quickly.  Ramya, is that correct ?

Ramya, as Marian suggested a list of IPs with #requests would be useful, perhaps also the full httpd log extracts, so we
can see as much as possible about the clients.
Comment 7 jcatchpoole 2008-03-05 14:31:52 UTC
Ramya it seems right now that demos.xml is zapped somehow, AND all Java user agents are blocked, is that correct ?

Is demos.xml the only file causing problems ?

If yes, what about leaving demos.xml zapped, but unblocking Java clients ?  There are several other applications
impacted by this which are (probably) not related to the trouble, but are currently broken bcs Java clients are blocked.
 We can live without demos.xml for a few days, but the other Java apps critically need access to nb.org.
Comment 8 Stanislav Aubrecht 2008-03-05 15:46:40 UTC
just a wild guess - i'm using javax.swing.ImageIcon(URL) to create the demo image, which in turn calls
Toolkit.getImage(URL). maybe some non-sun java implementation doesn't cache this result and tries to fetch the image
from the url on each repaint...?
Comment 9 Unknown 2008-03-05 15:53:52 UTC
Jack,

I am currently discussing on unblocking the java clients with the internal team though I am afraid doing that might
affect the site performance.

Regards,
Ramya
Support Operations
Comment 10 jcatchpoole 2008-03-05 15:59:51 UTC
Can you explain why it might affect performance ?  Pls see my questions in desc 8.
Comment 11 jcatchpoole 2008-03-05 16:40:00 UTC
*** Issue 129317 has been marked as a duplicate of this issue. ***
Comment 12 jcatchpoole 2008-03-05 16:58:15 UTC
This appears to have started in the last day or 2.  Looking at raw logfiles, number of hits to demos.xml went through
the roof :

zgrep -c demos.xml ./2008_03/access_2008030*
./2008_03/access_20080301.gz:35600
./2008_03/access_20080302.gz:23663
./2008_03/access_20080303.gz:29254
./2008_03/access_20080304.gz:1432530

I'll keep looking at the logfiles, but is anyone aware of any change that happened on Monday that might cause this ?  Eg
the dev build on that day ?  I know 2 new bundles were released on sun.com yesterday (or Mon?), anything odd about them ?
Comment 13 Stanislav Aubrecht 2008-03-05 17:09:28 UTC
btw, there's invalid character in the demos.xml feed. there's '&' on line 21 in <title> which is not allowed
Comment 14 Stanislav Aubrecht 2008-03-05 17:10:25 UTC
and that broken item was published on the day the high traffic started...
Comment 15 jcatchpoole 2008-03-05 17:21:07 UTC
Hmm, timing seems about right, someone committed that at 2008-03-03 17:58:26+0000.  Good catch.  I fixed it, I wonder if
that was the cause ?
Comment 16 Stanislav Aubrecht 2008-03-05 17:42:11 UTC
yes, that was the cause. the feed reader didn't handle this gracefully and went into an infinite loop (now fixed:
ff81b5775c93).
so it should be alright to fix the feed and enable java clients for netbeans.org again
Comment 17 jcatchpoole 2008-03-05 18:03:17 UTC
Thanks Standa.  I've asked Collab to unblock Java clients.
Comment 18 Stanislav Aubrecht 2008-03-05 18:35:31 UTC
we should keep watching for broken content in the feeds. otherwise the situation will repeat, the fix went to current
main repo only
Comment 19 Unknown 2008-03-05 18:44:27 UTC
Jack,

The Java agents have been unblocked. Kindly keep a close watch & get back to us,if you face an issue.

Thanks,
Ramya
Comment 20 jcatchpoole 2008-03-05 18:47:55 UTC
Many thanks, but I think you're in a better position to watch if anything goes out of control.  The first we will know
about it is if the site goes down or starts performing badly, but I hope you can monitor the server directly and see if
# requests or load is spiking.

Thanks for your help with this.
Comment 21 jcatchpoole 2008-03-05 18:48:53 UTC
Standa : yes, I'll set up some validation so that bad XML will never make it to the live www site.  Thanks.
Comment 22 _ rkusterer 2008-03-05 18:49:14 UTC
The feeds in the IDE startscreen work now. 
Is there a list of characters that we cannot use in the feed? It would be good to know for the future.
Comment 23 Unknown 2008-03-07 12:31:57 UTC
We are closing the issue from CollabNet's end,as the java applications were unblocked to access nb.org.

Regards,
Ramya
Comment 24 Stanislav Aubrecht 2008-03-13 18:46:11 UTC
Created attachment 58343 [details]
patched jar that doesn't ignore serverside caches
Comment 25 pslechta 2008-03-14 14:32:25 UTC
This fix was backported to release601_fixes branch.

Checking in RSSFeed.java;
/cvs/ide/welcome/src/org/netbeans/modules/welcome/content/Attic/RSSFeed.java,v <--  RSSFeed.java
new revision: 1.36.8.1; previous revision: 1.36
done

Checking in RSSFeed.java;
/cvs/ide/welcome/src/org/netbeans/modules/welcome/content/Attic/RSSFeed.java,v <--  RSSFeed.java
new revision: 1.36.8.2; previous revision: 1.36.8.1
done

Checking in CombinationRSSFeed.java;
/cvs/ide/welcome/src/org/netbeans/modules/welcome/content/Attic/CombinationRSSFeed.java,v  <--  CombinationRSSFeed.java
new revision: 1.10.10.1; previous revision: 1.10
done

Checking in DemoPanel.java;
/cvs/ide/welcome/src/org/netbeans/modules/welcome/ui/Attic/DemoPanel.java,v  <--  DemoPanel.java
new revision: 1.7.8.1; previous revision: 1.7
done
Comment 26 Dan Kolar 2008-04-30 15:24:36 UTC
v.
Comment 27 Marian Mirilovic 2009-11-08 02:38:22 UTC
We recently moved out from Collabnet's infrastructure