You are not logged in.

#1 2015-09-07 06:31:47

escada
Moderator
Registered: 2011-08-13
Posts: 1,753

Forum update [was: the forum is flooded by Spam. What can I do ?]

Hallo,

the past few days the "Questions and Answers" forum is flooded by spam messages. Is there anything I can do/help to get rid of them ?

regards

Offline

#2 2015-09-07 12:42:49

bgirardot
Member
From: Zurich
Registered: 2014-08-13
Posts: 16
Website

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

One thing the forum admins could do (of which I volunteer to be one to help delete spam) is remove the 60 second forced time window for reporting spam. I want to help these forums out by reporting the 3 or 4 spam messages in the forums I care about, forcing it to take 3 or 4 minutes instead of the 30 seconds it should take is not helping.

Offline

#3 2015-09-07 15:47:20

Tordanik
Moderator
From: Germany
Registered: 2008-06-17
Posts: 2,765
Website

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

The spam is getting worse. sad If I can do anything to help, I'd happily volunteer.


OSM in 3D: OSM2World

Offline

#4 2015-09-07 19:43:41

Lambertus
Inactive
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

Yes, the spam is getting worse unfortunately. I'm trying to get modifications installed that can help against this, but the forum software is already modified so new modifications possibly conflict.

Reporting these korean spammers is not necessary as they reveal themselves to me in one glance. In fact i'd rather not have too many reports because i need to acknoledge them one by one, adding even more work.

Edit:
Just tried a few usernames and ip addresses of known spammers with the StopForumSpam service. None of them were marked as spammer. So SFS is of no use with this spam run.


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#5 2015-09-12 09:56:49

escada
Moderator
Registered: 2011-08-13
Posts: 1,753

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

FYI, Lambertus made me moderator to help with this. I think I removed 50-60 spammers in the last 3 days. Feels like a game, you see one popping up somewhere, hit it, and another one pops up smile

No seriously, I do this in the morning, which seems to be the moment they all had the chance to write their messages.

Offline

#6 2015-09-12 11:29:09

escada
Moderator
Registered: 2011-08-13
Posts: 1,753

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

[Posted in wrong thread]

Last edited by escada (2015-09-13 20:18:11)

Offline

#7 2015-09-13 17:47:39

Tordanik
Moderator
From: Germany
Registered: 2008-06-17
Posts: 2,765
Website

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

Just for the record, I also got my moderator status as a result of the spam wave. I have removed dozens of spammers since then – not sure how many exactly, I've lost count.


OSM in 3D: OSM2World

Offline

#8 2015-09-13 18:19:41

Lambertus
Inactive
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

Thanks, looks like we all remove a lot of them. There is noticeably less spam when visiting the forums, so that's great!


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#9 2015-09-14 13:02:44

Lambertus
Inactive
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

I've hacked the login page with a second username field and a honeypot field. Hopefully this will elude the spammers for a while.


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#10 2015-09-14 13:37:07

escada
Moderator
Registered: 2011-08-13
Posts: 1,753

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

Fingers crossed.

Thanks for taking the time to come up with a solution

Offline

#11 2015-09-14 19:52:23

Lambertus
Inactive
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

Well, this time the implementation delay wasn't so much lack of time but lack of moments with relative clear thinking to offer the possible solution after two years of sleep-deprivated nights cool


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#12 2015-09-15 04:53:37

escada
Moderator
Registered: 2011-08-13
Posts: 1,753

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

Only 6 spammers this morning

Offline

#13 2015-09-15 09:14:58

Lambertus
Inactive
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

Yeah, that didn't take them long to work around. If this doesn't stop soon we'll have to implement heavier measures. Next step in fighting the spam is to add post moderation for new users so that new posts only appear on the forum when a moderator approves it. Lot's of work but almost guaranteed no spam anymore.


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#14 2015-09-25 08:54:35

Lambertus
Inactive
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

A heads up:
With FluxBB 1.5.8 the forum can automatically promote users from e.g. a New Member group to a Member group (currently all registered forum users are Members except the moderators and admin). When we combine this function with a post moderation system then (obvious) spam would not become visible on the forum, significantly reducing the incentive for spammers to attempt posting. Moderators would only have to approve posts from New Member and after (say) five approved posts the New Member would automatically upgrade to full Member whose posts are not actively moderated.

This week I've successfully tested the upgrade from Fluxbb 1.4.8 (current version) to 1.5.8 (latest version). Some modifications to the current forum software will initially be lost but can be added again after the upgrade to 1.5.8. The first obvious modification to re-implement would be the remote user authentication against the main OSM website API. When this is up-and-running the post moderation mod will be installed in the following days.

Sounds good?

Only issue I see with post-moderation is the "Users: *" forums and foreign languages, especially Cyrillic and Asian languages (i.e. non-Latin character sets). But we'll see how it goes.

As it stands now I'll start upgrading this evening, depending on real-world necessities (i.e. household and baby stuff that needs to be done). smile


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#15 2015-09-25 18:44:06

Lambertus
Inactive
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

Forum software upgrade has finished. Next is apply the various modifications...

.. fixed various counters that were out of sync (post count, topic counts, etc) and removed orphaned last_post references and such. Fixed a bug in a FluxBB plugin as well during this process.

.. forum registration is disabled so only existing members can login. Will take a while before new OSM users will be able to login again because I've decided that  the post moderation should be implemented first.

.. new New Member group created with restricted permissions (e.g. unable to send emails as the spammers were using the forum email system to send email spam).

.. about half way through adding the post moderation modification. Will continue tomorrow when time permits. New users won't be able to join the forum for now unfortunately.

Last edited by Lambertus (2015-09-25 21:24:45)


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#16 2015-09-25 21:32:39

XAN_ua
Member
Registered: 2012-10-20
Posts: 603

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

The main issue so far - fixed width of forum. 22" @ 1680x1050 looks not that good.

Offline

#17 2015-09-26 06:14:31

wambacher
Member
From: Schlangenbad/Wambach, Germany
Registered: 2009-12-16
Posts: 16,736
Website

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

Great job, Lambertus smile

one minor problem: Clicking on a link in a post does not open a new page like before.

Regards

walter

Offline

#18 2015-09-26 08:46:58

wmyrda
Member
Registered: 2014-07-07
Posts: 947

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

I am using number of features in my squid - proxy server - to enhance anonymity a little bit

reply_header_access Via deny all
reply_header_access X-Forwarded-For deny all
reply_header_access From deny all
reply_header_access Server deny all
reply_header_access WWW-Authenticate deny all
reply_header_access Link deny all
reply_header_access X-Cache-Lookup deny all
reply_header_access X-Squid-Error deny all
reply_header_access X-Cache deny all
reply_header_access Referer deny all
reply_header_access User-Agent deny all

So far I have had no problems using any of these features on any of the web pages I use until upgrade of this very forum. Now using "reply_header_access Referer deny all" returns following error while posting:

Bad HTTP_REFERER. You were referred to this page from an unauthorized source. If the problem persists please make sure that 'Base URL' is correctly set in Admin/Options and that you are visiting the forum by navigating to that URL. More information regarding the referrer check can be found in the FluxBB documentation.

I have disabled that line in squid to post this message, but i would hate it to be the last post as I am not going to turn it of just for one forum, sorry. Besides does that enhance security of the forum anyway?

Last edited by wmyrda (2015-09-26 08:47:37)

Offline

#19 2015-09-26 11:11:19

Jojo4u
Member
Registered: 2014-08-03
Posts: 1,090

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

I have Refcontrol as a Firefox Plugin for years, never had a problem with it until now! Setting is "forge", it sends the root of site (e.g. forum.openstreetmap.org). Same problem "HTTP_REFERER". Workaround is to add an exception, but is this security feature really neccessary?

https://addons.mozilla.org/en-US/firefo … efcontrol/

Offline

#20 2015-09-26 11:25:02

Lambertus
Inactive
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

XAN_ua wrote:

The main issue so far - fixed width of forum. 22" @ 1680x1050 looks not that good.

Does choosing one of the other styles (via your Profile page) help?

wambacher wrote:

one minor problem: Clicking on a link in a post does not open a new page like before.

Yes, that's one of the modifications I need to re-apply.
Edit: Should be fixed now

Re: Bad HTTP_REFERER
This seems to be a design flaw in FluxBB. I'm not a security expert, so I hope someone more knowledgeable can say if this modification would help? I.e changing from checking a token instead of the HTTP_REFERER?

Last edited by Lambertus (2015-09-26 11:44:41)


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#21 2015-09-26 13:55:54

Jojo4u
Member
Registered: 2014-08-03
Posts: 1,090

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

Lambertus wrote:

Re: Bad HTTP_REFERER
This seems to be a design flaw in FluxBB. I'm not a security expert, so I hope someone more knowledgeable can say if this modification would help? I.e changing from checking a token instead of the HTTP_REFERER?

Failing test post with Refcontrol on:
Referer header of POST request is: "http://forum.openstreetmap.org/"

Successful test post with Refcontrol off:
Referer header of POST request is: "http://forum.openstreetmap.org/post.php … &tid=36900"

I see that this is a quite crude security measure by FluxBB, the patch might help here if it applies cleanly. If there is no other easy solution I can life with adding an exception in Refcontrol

Offline

#22 2015-09-27 19:31:23

Lambertus
Inactive
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

Post moderation is implemented now. Existing users should have no trouble posting anywhere, but there are several options at various places so it might need some tinkering to get all settings correct. Please report problems if you encounter one.


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#23 2015-09-27 19:33:57

SomeoneElse
Member
Registered: 2010-10-13
Posts: 1,578

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

@Lambertus - thanks for all your efforts (and to the moderators too).  Hope you get some sleep now smile

Offline

#24 2015-09-27 19:44:28

Lambertus
Inactive
From: Apeldoorn (NL)
Registered: 2007-03-17
Posts: 3,269
Website

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

You're welcome.

Now, in order of importancy, the following mods:
- OSM API authentication so that new users can enter the forum (new registrations are disabled for now) -> Finished.
- fix the 'bad referrer' problem by implementing a CSRF synchronizer token pattern. Thanks Jojo4u for the feedback.
- Easy BBCode & quick quote


Mapping tools: Garmin GPSmap 60CSx, Giant Terrago 2002

Offline

#25 2015-09-28 08:56:55

marczoutendijk
Member
From: Vught
Registered: 2012-03-04
Posts: 2,390
Website

Re: Forum update [was: the forum is flooded by Spam. What can I do ?]

When editing/creating messages, I don't see the handy buttons to format my text. BBCode is on.


--
There is only one place where you can connect a high voltage line to a river: on osm! (but not in JOSM cool )

Offline

Board footer

Powered by FluxBB