Forum very slow

[Arpangel]

Been extremely slow last couple of days, pages taking about 30 secs to load, plus lots of 504 Target Not Responding notifications.

[MOF]

Agreed, same here, 504 error happened only when I was trying to open the Home page.

[Hugh Robjohns]

I had 504 notices earlier this morning, but it was fine yesterday and over the weekend for me, and it's running fine again now....

But, as has been discussed previously, the SOS site is a popular target for AI bot scraping and it does cause us problems now and again, despite the team's strenuous efforts to prevent and resolve issues.

[ManFromGlass]

I come back later and it’s generally fine.
Isn’t there a reverse digital time bomb anti scraper virus containing the full works of Shakespeare in every known language on earth and beyond that can be planted on a secret page here?

[Folderol]

I come back later and it’s generally fine.
Isn’t there a reverse digital time bomb anti scraper virus containing the full works of Shakespeare in every known language on earth and beyond that can be planted on a secret page here?

I don't know the details, but I've heard there is a method of sending bots down the rabbit hole.

[ef37a]

I come back later and it’s generally fine.
Isn’t there a reverse digital time bomb anti scraper virus containing the full works of Shakespeare in every known language on earth and beyond that can be planted on a secret page here?

I don't know the details, but I've heard there is a method of sending bots down the rabbit hole.

Hmm, this sounds rather like retaliating against noise nuisance by bombarding the offenders with more noise? Never an advisable course of action.

Then, the owners of the AI bots are likely far more powerful and vastly better resourced than SoS? Not someone you want to pick a fight with!

I guess we can only hope they will be done scraping at some point or less agressive means will be found to defeat them?

Dave.

[MOF]

Is the Captcha system a possibility?

[Forum Admin]

We already use Captcha but it's not a great tool... these AI companies are clued-up.

Apologies everyone, yet again SOS site and the knowledge buried inside this SOS Forum is being scraped hourly by Bots. Our WebDev does block them but it is an endless, thankless task.

We are exploring options.

Your continued patience when 504 timeout messages appear is much appreciated.

Ian G.

[merlyn]

I guess we can only hope they will be done scraping at some point ...

Nope. @Folderol may have been thinking of Cloudflare's 'AI labyrinth'.

https://developers.cloudflare.com/bots/ ... ight-mode/

[Folderol]

I guess we can only hope they will be done scraping at some point ...

Nope. @Folderol may have been thinking of Cloudflare's 'AI labyrinth'.

https://developers.cloudflare.com/bots/ ... ight-mode/

Yes. That's the one

[ConcertinaChap]

According to New Scientist Wikipedia is suffering so much from AI bots scraping that they are beginning to regard it as an existential threat.

CC

[Hugh Robjohns]

I think they're right....

[OneWorld]

According to New Scientist Wikipedia is suffering so much from AI bots scraping that they are beginning to regard it as an existential threat.

CC

This to me appears to be about the only tangible aspect of AI that has had any impact and I can’t say I’ve seen any obvious benefit.

[Drew Stephenson]

I think we have to remember there are lots of different types of AI. Don't dump everything in with the LLMs and generative stuff.

[Forum Admin]

Yep, a large % of corporates are using automated AI Chat Bots for all of their customer-facing "customer service" enquiries... at least, the initial wave until it loops round to the fork that says "pass to a human to resolve".

[Drew Stephenson]

Deleted as I thought it would sit better in music business. My bad!

[ef37a]

Been painfully slow for me all week, spinning blobs. I am with BT and use Firefox. Anything known?

Dave.

[Forum Admin]

Been painfully slow for me all week, spinning blobs. I am with BT and use Firefox. Anything known?

Dave.

I use Virgin Business broadband 1Gbps connection and have found it OK, from Cambridge area, but a couple of our team reported slowness.

Broadband speed varies daily and hourly all across the globe.

@Dave - is it working OK for you today?

Ian G

[Hugh Robjohns]

No speed or access issues here (Plusnet).

[zenguitar]

Been running slow for me most of the week. Now it's threatening to get quicker.

Andy

[ef37a]

Been painfully slow for me all week, spinning blobs. I am with BT and use Firefox. Anything known?

Dave.

I use Virgin Business broadband 1Gbps connection and have found it OK, from Cambridge area, but a couple of our team reported slowness.

Broadband speed varies daily and hourly all across the globe.

@Dave - is it working OK for you today?

Ian G

No Ian but "funny"! Getting to your reply from inbox, practically instant. Go back to Arp's post about mixers, blobs and or just stuck.

My internet speed does not seem to vary much at all, just checked with Ookla, 138 down, 47 up and those figures are almost always within 10% in fact upload rarely changes a unit from 47. something.

Dave.

[Forum Admin]

According to New Scientist, Wikipedia is suffering so much from AI bots scraping that they are beginning to regard it as an existential threat.

CC

Indeed the adoption by Google of AI Overviews and by other Search Engines has impacted already in the USA (where AI Overviews were first trialled) and will have an effect on highly-regarded, authoritative and well-ranked content sources like SOS.

This screenshot from Cloudflare's blog highlights the problem.