This Bugzilla instance is a read-only archive of historic NetBeans bug reports. To report a bug in NetBeans please follow the project's instructions for reporting issues.

Bug 187139 - [69cat] Very poor spam protection for @netbeans.org mailing lists
Summary: [69cat] Very poor spam protection for @netbeans.org mailing lists
Status: REOPENED
Alias: None
Product: www
Classification: Unclassified
Component: Admin (show other bugs)
Version: 6.x
Hardware: All All
: P1 normal with 2 votes (vote)
Assignee: Jan Pirek
URL:
Keywords:
: 210204 217612 (view as bug list)
Depends on:
Blocks:
 
Reported: 2010-06-03 12:51 UTC by Jiri Kovalsky
Modified: 2013-04-10 11:55 UTC (History)
10 users (show)

See Also:
Issue Type: DEFECT
Exception Reporter:


Attachments
Moderation requests in September 2009 (28.29 KB, image/png)
2010-06-03 12:51 UTC, Jiri Kovalsky
Details
Moderation requests in January 2010 (31.45 KB, image/png)
2010-06-03 12:51 UTC, Jiri Kovalsky
Details
Moderation requests in May 2010 (27.08 KB, image/png)
2010-06-03 12:52 UTC, Jiri Kovalsky
Details
Moderation requests in the first decade of June 2010 (21.60 KB, image/png)
2010-06-10 13:56 UTC, Jiri Kovalsky
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jiri Kovalsky 2010-06-03 12:51:15 UTC
Created attachment 99786 [details]
Moderation requests in September 2009

Since mid November 2009 the average load of moderation requests dramatically increased due to migration from CollabNet to Kenai infrastructure. I analyzed data from September 2009, January 2010 and May 2010 and found out that this increase is obviously caused by worse spam filtering.

   Please look at the attached charts that prove my hypothesis. While before the migration it was necessary to reject only ~14.1 spam messages daily, it is now 4 times more!

Month             Average # spam    Maximum # spam     Average # valid e-mails
===============================================================================
September 2009         ~14.1             26                    ~32.6
January 2010           ~48.5            105                    ~31.9
May 2010               ~60.6            182 (!)                ~36.4

   Due to this unfortunate trend we had to hire two more volunteers as mailing list moderators! So, can you please get rid of current spam solution and use something else for example SpamAssassin [1]?

[1] http://spamassassin.apache.org

Thanks a lot!
Jiri Kovalsky
Comment 1 Jiri Kovalsky 2010-06-03 12:51:47 UTC
Created attachment 99787 [details]
Moderation requests in January 2010
Comment 2 Jiri Kovalsky 2010-06-03 12:52:14 UTC
Created attachment 99788 [details]
Moderation requests in May 2010
Comment 3 Marco Walther 2010-06-03 15:10:52 UTC
You might be surprised to learn that we use SpamAssassin from day one.
Comment 4 Marco Walther 2010-06-03 18:47:42 UTC
OK, I updated the SpamAssassin throw-out limit from 6.9 to now 6.2, let's see if that makes difference.

I have to be careful because we don't have anybody who wants to fish false-positives out of the 1000+ messages/hour we throw out already. And that's after the 3000+ messages/hour which we reject right away.

I'm also working on updating some SpamAssassin helpers but that needs extra help from the network people. And I don't know when that will happen.

So, please update this bug with the stats for the next week and we can look after that.

Thanks,
-- Marco
Comment 5 Jiri Kovalsky 2010-06-04 10:31:53 UTC
I will do Marco! BTW, isn't it possible to teach SpamAssassin on the spam messages that went through? We have thousands of these in our moderator's e-mail account. Let me know if you are interested in this.

And thanks for enlightening me - I am surprised indeed. :-)
Comment 6 Jiri Kovalsky 2010-06-10 13:56:46 UTC
Created attachment 99981 [details]
Moderation requests in the first decade of June 2010

Month             Average # spam    Maximum # spam     Average # valid e-mails
===============================================================================
6/1 - 6/10            ~59.4              84                    ~48.4

My interpretation is that number of spam is same :-( but maximum is lower :-) while more valid e-mails goes through! :-) I think it's still early for deeper evaluation. Let's wait for end of June.

Does anybody else have other opinion?
Comment 7 Marian Mirilovic 2011-11-16 14:02:50 UTC
Jiri, is this issue still valid ?
Comment 8 Jiri Kovalsky 2011-11-16 14:06:59 UTC
I don't know. I turned to our mailing lists moderators [1] and will update this issue once I know more.

[1] http://netbeans.org/projects/www/lists/moderators/archive/2011-11/message/0
Comment 9 smokindoug 2011-11-17 00:51:26 UTC
I have moderated for over a year and things were poor to start, but then got a fair amount better. I think I probably deal with 50% less spam now than at the worst periods.

I think "vacation" spam was dealt with better this summer versus last summer for example.

It is difficult to quantify, but moderating has been easier the last several months.
Comment 10 Antonin Nebuzelsky 2011-11-21 09:17:31 UTC
(In reply to comment #7)
> Jiri, is this issue still valid ?

I started receiving tons (~50 a day) of spam a week ago through various netbeans.org mailing lists (commits, issues and other lists).

A misconfiguration of spam filters after the site upgrade?
Comment 11 Jiri Kovalsky 2011-11-21 09:42:33 UTC
I also experience way more spam these days than it used to be a week or two ago. Honzo, can you please file appropriate issue in Kenai's JIRA system? It really looks like a misconfiguration after the recent netbeans.org upgrade. Thanks!
Comment 12 Jan Pirek 2011-11-21 15:57:19 UTC
I started kenai ticket for this

http://kenai.com/jira/browse/KENAI-3121

honza
Comment 13 Jan Pirek 2011-11-23 07:14:48 UTC
Fixed by Marco from the kenai team. His message: Should be back to `normal'.
Comment 14 _ proxity 2012-09-05 14:10:42 UTC
During the last weeks I'm receiving a lot of spam over my @netbeans.org 
address. Most of them use Content-Type: text/plain; charset="koi8-r"
Could you please prevent this? In fact I'm convinced that it's not 
necessary that @netbeans.org addresses should be reachable from the "outside". 
I mean it's surely enough if the bug tracking system and the mailing lists 
can use it.
Comment 15 Marian Mirilovic 2013-04-10 11:47:20 UTC
*** Bug 217612 has been marked as a duplicate of this bug. ***
Comment 16 Marian Mirilovic 2013-04-10 11:54:18 UTC
Current stats show that percentage of SPAMs crossed 85% of emails from moderated mailing lists, moderators needs to filter-out incredible percentage of emails : 

( Spam, Moderation requests, Month)
30%   1401   September2009
72%   4463   August2010
87%   3779   March2013

I do not think it's acceptable in any kind of community, to cover job of SPAM filters by human bodies !
Comment 17 Marian Mirilovic 2013-04-10 11:55:00 UTC
*** Bug 210204 has been marked as a duplicate of this bug. ***