:: Home :: Get Hosted :: PPM FAQ :: Forum FAQ :: Privacy Policy :: Search :: Memberlist :: Usergroups :: Register :: Profile :: Log in to check your private messages :: Log in ::
Do you want to advertise at Project Perfect Mod. Find out how to do it HERE.

The time now is Fri Apr 03, 2020 1:13 am
All times are UTC + 0
 Forum index » PPM Zone » PPM Discussion Zone
PPM slow?
Moderators: Global Moderators
Post new topic   Reply to topic Page 2 of 2 [78 Posts] View previous topic :: View next topic
Goto page: Previous 1, 2
Author Message
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Sun Sep 22, 2019 3:05 pm    Post subject: Reply with quote

He is not living in China at the moment.

Back to top
View user's profile Send private message Visit poster's website Skype Account
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Sun Sep 29, 2019 3:41 pm    Post subject: Reply with quote

And the chinese bot has adapted to my filter today.... so I had to implement another filter. It should speed up things again for a while.

Back to top
View user's profile Send private message Visit poster's website Skype Account
TAK02
General


Joined: 28 Jun 2015
Location: It was Damascus.

PostPosted: Sun Sep 29, 2019 5:15 pm    Post subject: Reply with quote

Banshee wrote:
And the chinese bot has adapted to my filter today....

The age of machines has begun... we're doomed.

Back to top
View user's profile Send private message Send e-mail Visit poster's website ModDB Profile ID Skype Account
suffle19
Light Infantry


Joined: 27 Jun 2019

PostPosted: Sun Sep 29, 2019 5:19 pm    Post subject: Reply with quote

MasterHaosis wrote:


Im also have this sometimes

Back to top
View user's profile Send private message
Deformat
Defense Minister


Joined: 17 Sep 2007
Location: Bucharest,Romania Posts:29240348568

PostPosted: Sun Sep 29, 2019 5:21 pm    Post subject: Reply with quote

I might get a lot of hate for this suggestion, but why not ban all Chinese IPs for about a couple days? Maybe it will go away like this.

I don't have anything against Chinese people, it's just that this might be a much easier solution to the current problem.

Back to top
View user's profile Send private message
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Sun Sep 29, 2019 7:28 pm    Post subject: Reply with quote

There are many and many reasons why I won't follow your suggestion, Deformat.

I won't get into details here about all of them, but our problem here is not the Chinese users. It's rather someone or an organization that seems to use infrastructure from China in a completely abusive way to spam undesired advertisement here. However, their abusive means reached a level where it overloads this place, preventing the supposing customers of these undesired advertisements from reading them. Which is kinda bizarre. But it is still very toxic to PPM and it must be stopped at all costs.

Anyway, your suggestion, if applied, wouldn't stop this abusive organization, since they've started to use cloud services from other countries near China to do this kind of service.

Back to top
View user's profile Send private message Visit poster's website Skype Account
MasterHaosis
Commander


Joined: 01 Nov 2010
Location: Serbia

PostPosted: Mon Oct 28, 2019 10:06 am    Post subject: Reply with quote

Service Temporarily Unavailable

The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

Additionally, a 503 Service Temporarily Unavailable error was encountered while trying to use an ErrorDocument to handle the request.


In moment when I just uploaded SHP
_________________
OrangeNero wrote:
Me picking a fight? I know I am the greatest user alive here so everyone starts reading topics with my posts first and ignores the ones before and after.

Back to top
View user's profile Send private message
PussyPus
Commander


Joined: 14 Jul 2015
Location: Underground Posts: N/A

PostPosted: Mon Oct 28, 2019 3:13 pm    Post subject: Reply with quote

Banshee wrote:
There are many and many reasons why I won't follow your suggestion, Deformat.

I won't get into details here about all of them, but our problem here is not the Chinese users. It's rather someone or an organization that seems to use infrastructure from China in a completely abusive way to spam undesired advertisement here. However, their abusive means reached a level where it overloads this place, preventing the supposing customers of these undesired advertisements from reading them. Which is kinda bizarre. But it is still very toxic to PPM and it must be stopped at all costs.

Anyway, your suggestion, if applied, wouldn't stop this abusive organization, since they've started to use cloud services from other countries near China to do this kind of service.


MasterHaosis wrote:
Service Temporarily Unavailable

The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

Additionally, a 503 Service Temporarily Unavailable error was encountered while trying to use an ErrorDocument to handle the request.


In moment when I just uploaded SHP

Can't we just use 5th Generation technologies to prevent all this?
_________________
If you are a MetalHead (Heavy Metal Fan) and don't want to be a metalhead, Just remove your metal ball from your head. �:p .

Back to top
View user's profile Send private message ModDB Profile ID Skype Account
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Tue Oct 29, 2019 2:29 am    Post subject: Reply with quote

It has nothing to do with 5g, 4g (LTE), 3g or whatever. The service temporarily unavailable happens because you get a time out (response is of the server is too slow). On MasterHaosis's case, all the sites in the server were slow or not responding at all. I don't know what caused it. However, in the chinese bot net case, the problem is that PPM was trying to bypass the limit of MySQL queries allowed by the server and, for this reason, it was not getting a response. Once I've blocked the bot net, we've been far below our MySQL queries limit, which normalized the situation.

Back to top
View user's profile Send private message Visit poster's website Skype Account
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Fri Jan 10, 2020 5:08 am    Post subject: Reply with quote

Update: After many observations about the behavior of this forum, comparison with other forums (like Revora), and some thoughts on what could be making this place get much more SQL queries than all other forums on the server, I had an idea that could make the situation much better and seriously reduce the MySQL queries. And I've implemented that idea. It won't cause any damage to anyone's experience and the speedup was already noticed after some minutes testing.

How's the forum running now for you (after this post was done, please, so pay attention to its data and time)?

Back to top
View user's profile Send private message Visit poster's website Skype Account
TAK02
General


Joined: 28 Jun 2015
Location: It was Damascus.

PostPosted: Fri Jan 10, 2020 7:21 am    Post subject: Reply with quote

503s or server irresponsiveness and/or slowmess has been popping up more, recently. Nothing a quick refresh won't fix tho, unless it's just slow.
_________________
One and only developer of C&C S. True Supremacy.
Youtube: C&C Supremacy Mod | Skype: TAK02 Supreme CnC Mods
m7 wrote:
I tend to release things I create so that assets are never lost to hard drive problems, accidental deletion, or me having to pretend to care about rippers taking things from my project when it is done. #Tongue
I once ninja'd MigEater #Tongue
Back to top
View user's profile Send private message Send e-mail Visit poster's website ModDB Profile ID Skype Account
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Fri Jan 10, 2020 3:41 pm    Post subject: Reply with quote

Yea, the fact that I've added these friendly URLs made several bots to be more agresssive to crawl them, which is why the forums are being overloaded more often and also what inspired me to think about other ways to improve performance here. I think it is a temporary side effect of the friendly URLs feature. At some point, things should get calmer again.

Back to top
View user's profile Send private message Visit poster's website Skype Account
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Fri Jan 10, 2020 8:18 pm    Post subject: Reply with quote

Update: Today I've just figured out... why do bots need to know forum statistics? Cutting down forum stats for them seems to speed up the forum considerably.

Back to top
View user's profile Send private message Visit poster's website Skype Account
MasterHaosis
Commander


Joined: 01 Nov 2010
Location: Serbia

PostPosted: Wed Jan 15, 2020 11:00 pm    Post subject: Reply with quote

From this evening and so far I experienced blank page, and very slow site. Something is wrong.
_________________
OrangeNero wrote:
Me picking a fight? I know I am the greatest user alive here so everyone starts reading topics with my posts first and ignores the ones before and after.

Back to top
View user's profile Send private message
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Thu Jan 16, 2020 2:13 am    Post subject: Reply with quote

Blank page was a stupid mistake in the code that I've done and due to lack of testing (lack of attention, mostly), this one lasted long. But blank pages are really my fault. Regarding the slow site, more than 90% of the traffic of the site are bots. However, there are ones in China that are more aggressive and may severely slow down the site starting from 6:40 am GMT. I don't know when it ends because I'm sleeping after that. Sometimes between 19:00 and 21:00 GMT, it also lags a bit. But I haven't figured out the reason yet.

Back to top
View user's profile Send private message Visit poster's website Skype Account
MasterHaosis
Commander


Joined: 01 Nov 2010
Location: Serbia

PostPosted: Thu Jan 16, 2020 3:48 am    Post subject: Reply with quote

What do you think that is solution for those bots? And why do they chose this site that much as target?
_________________
OrangeNero wrote:
Me picking a fight? I know I am the greatest user alive here so everyone starts reading topics with my posts first and ignores the ones before and after.

Back to top
View user's profile Send private message
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Thu Jan 16, 2020 6:35 am    Post subject: Reply with quote

They crawl every site. The thing is that PPM has over 500 thousand topics and millions of links derived from it. So, they have clouds with thousands or million computers and PPM has only one old computer to serve them all... and serve other sites as well, including more popular ones like Revora. Some companies like Google do respect the fact that certain sites can't serve all those bots at once and it tries to respect robots.txt. Other services like Semrush, Mj12Bot, magpie crawler, those non-identified chinese services, and some others have no respect at all for our limited abilities to serve their clouds.


So, our solution is separated in few steps:

1) Friendly urls (that actually makes the problem worse, since it increases the number of indexable pages).

2) Creating a sitemap with desirable indexable links.

3) Prevent bots from indexing bad links: Block sections that won't be interesting to index from bots with robots.txt, with rel=nofollow, meta robots noindex, internal control on those php files to prevent these bots from accessing these places and cuting unnecessary features from bots.

4) Forcing redirect of repetitive links that we don't want to be indexed to those who we want to be indexed, so it deletes those links and indexes only the ones from the sitemap.

Step 1 deteriorates performance. The steps 3 and 4 should improved it a lot and reduce the intensity of bot invasion. It should also improve PPM's performance on search engines.

At the moment, step 1 has been done, although I am planning to eventually add topic/forum pages as an additional command.

Step 2 is finished, although it will be incremented as topic/forum pages gets implemented.

Step 3 is nearly finished or at least it has a considerable progress.

Step 4 wasn't done yet.

Last edited by Banshee on Mon Jan 20, 2020 3:04 am; edited 1 time in total

Back to top
View user's profile Send private message Visit poster's website Skype Account
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Sun Jan 19, 2020 7:41 am    Post subject: Reply with quote

Update: It seems I am pretty much done with all 4 steps mentioned on the post above for both the site and forums. PPM runs faster in the weekends, because the chinese botnet doesn't seem to work on weekends. Only on weekdays, don't ask me why.

There might be few tweaks that I'll still do to improve how search engines index forum pages and one minor tweak to improve filtering of certain botnets. But we are really close to the finished state of this project and hopefully the performance from PPM should be better now (although the real tests only happens during the weekdays... specially at 6:40 am GMT).

Back to top
View user's profile Send private message Visit poster's website Skype Account
MasterHaosis
Commander


Joined: 01 Nov 2010
Location: Serbia

PostPosted: Sun Jan 19, 2020 10:41 pm    Post subject: Reply with quote

Seems faster now! (....for now) Very Happy
_________________
OrangeNero wrote:
Me picking a fight? I know I am the greatest user alive here so everyone starts reading topics with my posts first and ignores the ones before and after.

Back to top
View user's profile Send private message
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Mon Jan 20, 2020 3:05 am    Post subject: Reply with quote

It is faster indeed, but we can't celebrate it yet. It seems that the evil chinese botnet of doom only works on weekdays. So, I'll only celebrate if it continues to be stable as soon as it handles its invasion properly.

Back to top
View user's profile Send private message Visit poster's website Skype Account
Lin Kuei Ominae
Seth


Joined: 16 Aug 2006
Location: Germany

PostPosted: Mon Jan 20, 2020 3:32 pm    Post subject: Reply with quote

in the Tutorials Factory i can't reach page 2 or later anymore.
https://ppmforums.com/forum-30/tutorials-factory/
it always shows page 1 even if the friendly url now includes a page-2 text when clicking on a later page or clicking on the "next" button
https://ppmforums.com/forum-30-page-2/tutorials-factory/?ppage=50&sort=lastpost&order=DESC
_________________
SHP Artist of Twisted Insurrection:  Nod buildings

Public SHPs
X-Mech Calendar (28 Mechs for GDI and Nod)
5 GDI, 5 Nod, 1 Mutant, 1 Scrin unit, 1 GDI building

Tools
Image Shaper______TMP Shop______C&C Executable Modifier

Back to top
View user's profile Send private message
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Mon Jan 20, 2020 6:18 pm    Post subject: Reply with quote

That's a good bug report, LKO. I did edit the way traditional forums read its topics but I forgot to tweak the ones with hashtags. But, don't worry. This bug is fixed now.

Back to top
View user's profile Send private message Visit poster's website Skype Account
E1 Elite
Commander


Joined: 28 May 2013

PostPosted: Tue Jan 21, 2020 12:33 am    Post subject: Reply with quote

Also check if the downloads from ppmsite.com are being served after the human/bot check.

Back to top
View user's profile Send private message
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Tue Jan 21, 2020 12:54 am    Post subject: Reply with quote

I've tested with TS Manual, and it worked fine (using PPM's left menu -> Tiberian Sun). Do you have any specific link that is no longer working? Please, cite samples, so I can work specifically on them or figure out what's going on.

Back to top
View user's profile Send private message Visit poster's website Skype Account
E1 Elite
Commander


Joined: 28 May 2013

PostPosted: Tue Jan 21, 2020 1:09 am    Post subject: Reply with quote

Zip files like
https://www.ppmsite.com/download/cnc-ealayer3-asset-generator.zip/
https://www.ppmsite.com/download/fs101.zip/

After the bot check it reloads the same page, didn't give the download.

Back to top
View user's profile Send private message
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Tue Jan 21, 2020 1:34 am    Post subject: Reply with quote

As far as I've tried, logged in with my account, it works fine here.

However, using a private window (logged out), I've managed to reproduce what happened to you.

Anyway, I've made some tweaks and on my latest attempts, I could download these files using captcha. Check it out now and tell me the results, please.

Back to top
View user's profile Send private message Visit poster's website Skype Account
E1 Elite
Commander


Joined: 28 May 2013

PostPosted: Tue Jan 21, 2020 1:52 am    Post subject: Reply with quote

Working now, gives downloads.

Back to top
View user's profile Send private message
Banshee
Supreme Banshee


Joined: 15 Aug 2002
Location: Brazil

PostPosted: Tue Jan 21, 2020 2:31 am    Post subject: Reply with quote

Excellent! I'm glad it works Smile

If you guys find any other problem, please, report it so I can fix it asap.

Back to top
View user's profile Send private message Visit poster's website Skype Account
Display posts from previous:   
Post new topic   Reply to topic Page 2 of 2 [78 Posts] Goto page: Previous 1, 2
View previous topic :: View next topic
 Forum index » PPM Zone » PPM Discussion Zone
Jump to:  
Share on TwitterShare on FacebookShare on Google+Share on DiggShare on RedditShare on PInterestShare on Del.icio.usShare on Stumble Upon
Quick Reply
Username:


If you are visually impaired or cannot otherwise answer the challenges below please contact the Administrator for help.


Write only two of the following words separated by a sharp: Brotherhood, unity, peace! 

 
You cannot post new topics in this forum
You can reply to topics in this forum
You can edit your posts in this forum
You can delete your posts in this forum
You can vote in polls in this forum
You can attach files in this forum
You can download files in this forum


Powered by phpBB © phpBB Group

[ Time: 0.4421s ][ Queries: 10 (0.0208s) ][ Debug on ]