Jump to content

ICE CROW

Recommended Posts

Unfortunately, one side effect of using CloudFlare is that all the board traffic comes through CloudFlare, so at the socket level every single connection seems to come from CloudFlare's various server farms rather than actual locations. While CloudFlare does pass the originating IP as a special HTTP header field and our mutant lighttpd variant can read that field and restore the IP to that session in order to pass along proper information to the board (for banning purposes and logs and such not), by that point you've already incurred the overhead of accepting the connection. And once you've reached that stage, per-IP traffic shaping gives minimal performance gain; the big performance benefit is in never opening the socket to begin with.

But beyond that... no, the worst of the traffic is really not Chinese botnets. There have been some botnet waves in the past — Ran posted about one a while ago — but the heavy load at the beginning of the week is sadly all real people. There's just that many Google searches after an episode airs, and since the Westeros.org wiki turns up as one of the top results — if not the top hit —for GoT Google searches, we get an immense amount of traffic hitting the site as people watch the episode — sometimes time shifted a day or so on their TiVo/DVR — and hit the net to Google various character names, events, houses, or whatever else they want to look up while watching.

To put this in perspective, the site had 12Mbit/sec worth of sustained traffic primarily just from one particular Google search phrase after the penultimate episode of last season aired. (I'm pretty sure everyone can guess what search phrase that was.) That's a particularly pronounced example and things aren't usually quite that dramatic, but the traffic spikes around Sunday-Tuesday of each week are genuinely search-result traffic to the Wiki rather than botnets trying to harvest things.

Link to comment
Share on other sites

FYI, folks, I'm going to be taking things down for a couple of 5-10 minute spells today during slow bits at work; I'm going to try to do them in the smallest chunks possible, ideally a couple minutes at a time, but this'll let me get a couple of maintenance tweaks in place in preparation for adding Varnish once the post-episode traffic lets up a bit. So if you get a 500 Server Error very briefly, that's probably not an actual issue, that's me temporarily taking the back-end offline.

Link to comment
Share on other sites

The Seven willing, maintenance today is done; things have been tweaked and tuned in preparation for Varnish testing on Wednesday, and I should be done kicking things and causing 5 minute downtimes.

(Of course, watch something catch fire now that I've said that...)

Link to comment
Share on other sites

  • 2 weeks later...

So we've just recently had a fund raiser. Does that mean we can expect anything improvement any time soon because I'm still seeing these 503 Server Errors quite frequently.

Link to comment
Share on other sites

These are actually quite different errors. These relate to Varnish, and obviously since we've just put it in, there'll be some time needed to tweak everything to make sure it runs as smoothly as possible.

Moreover, you'll notice that we haven't had to turn off profiles and various other features as we have had to do in the past to keep up with traffic. Varnish may cough and choke from time to time and I'm sure that will improve as we get used to it, but on the whole the server is more responsive and more featureful since it has been put in place.

Link to comment
Share on other sites

I can't post in topic called "Is Jon handsome".

i always see cut off page. Like, I can't see one part of page. One part depict tower who is normally between westeros shield and "connect with us" message, and one part depicts what I should see normally. I can use this area where I write this message and buttons which helps writing, but I can't click anything else, incuding add reply button and A Dance with dragons redirection button.

Link to comment
Share on other sites

These are actually quite different errors. These relate to Varnish, and obviously since we've just put it in, there'll be some time needed to tweak everything to make sure it runs as smoothly as possible.

As Ran notes, I'm tweaking Varnish every day or two, trying to find the ideal configuration for this setup. If you *do* get a 503 error, could you make a note of the specific circumstances (particularly whether the 503 came back instantly or took too long, whether it was on the wiki or the forum, and so on)? That'll help me find the best configuration for Westeros' needs.

Thanks!

Link to comment
Share on other sites

So it has been a few weeks and no one every answered me when i asked last time, I understand in the grand scheme of things avatar pics is of small significance but i must ask again. Is it possible to get back the function of changing our avatars by URL?.

Link to comment
Share on other sites

Not right now. I think an available update may restore it but it'll be awhile before I can implement it.

If this is in response to my post thank you, just knowing whats going on is really all i wanted to know :).

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

Guest
This topic is now closed to further replies.
×
×
  • Create New...