Photo taken by Amira

I’m about to leave Sunny Miami Florida to return to frigid Montreal and am feeling particularly blessed to get to work and sometimes hang out with such wonderful people.

The WeMedia conference invited us and kindly sponsored some of our costs, but the real treasure has been sharing a house, food, drinks and stories with my coworkers. I’ve gushed about it in the past but its worth repeating: I couldn’t possibly do better in life than to have these people around me. That I get paid to spend time with them is a little miracle.

Shout outs to Eddie, Lova, Georgia, Solana, Jillian, Lokman, Rebekah, Ivan, Amira and Leonard. I fuckin’ love you guys.

Anyway, these days I usually just point people at WordPress.com, since they have great themes, the best software around and an easy way to take it to the next level.

So what’s left to do? The other half of giving someone a blog: Showing them how to use it!

Global Voices’ core mission, aside from reporting on what’s happening in blogs all over the world, is to get more blogs and bloggers going whenever possible. Writing is good for you, it helps you clarify your thoughts and hone your writing, and it also helps people find you and keep up with you online.

GV is having a drive to get people to teach someone else how much fun blogging can be, or alternately blogging’s little cousin, microblogging (which means services like Twitter or identi.ca similar to “status” on facebook but with more power and more fun. btw I’m almost always @jeremyclarke).

I signed the pledge to write about it and teach someone, but as the above list implies, my life is already pretty saturated with people I’ve taught to blog, so if you live in Montreal and want to learn how to blog (or want some advice about it too I guess) drop me a line and I can help you out sometime.

If you have someone in your life with problems getting their message out (or a business to promote, or any other problem that could be solved if they had a TV channel) then consider taking some time to show them how blogging could improve their life <3

Each translator chooses which posts they want to translate (we have too many in English for the other languages to translate everything) so usually we get 2 or 3 of the dozen or so active languages that translate any given post, and we show links at the top of the post to the translations (a mix of showing off and making it easy for people who’s first language isn’t English to find more convenient versions).

This is something wonderful for me to see: One post that the whole Lingua community got together and translated, resulting in a ridiculously long list that shows the truly amazing variety of letterforms, if nothing else, that Global Voices is printed in.

We use the ISO codes for languages (like wikipedia, i.e. fr.wikipedia.org) but in case you’re curious, the post is available in: French, Chinese Traditional, Chinese Simplified, Macedonian, German, Malagasy, Bangla, Swahili, Albanian, Spanish, Italian, Dutch, Russian, Serbian, Portuguese, Arabic, Polish, Indonesian, Japanese and Farsi.

That’s a mouthfull. I love my job.

When the Global Voices community saw the baby joke (pictured in the last post) they loved it so much they demanded a lolcat-based campaign and the above was born.

Just to keep it classy we also have a more subdued badge:

Donation page, All badges.

I think it will really rake in the dough, don’t you?

Note: This is obviously a joke. Global Voices would never resort to ridiculous, cute graphics as a branding tactic.

Lately GV has been pretty out of control and our server has been crashing way to regularly due to too many visitors or bots. I’ve been working to find all the little holes in the Apache/PHP/MySQL configurations that are causing the crashes when load gets high, but it’s impossible while you’re constantly putting out fires and restarting the servers manually.

I’ve been having frustrating fun with a tool called Monit that helps stop your server from completely crashing by watching it’s system stats and selectively restarting processes or executing whatever command you want. It installs pretty easily on Linux servers (I think it’s in both Yum for CentOS/RH and in apt for Debian/Ubuntu) and it uses text files similar to Apache to set up different status conditions and what to do. The configuration took me awhile to get right, but once the percentages were tuned based on watching it for awhile it has kept the server from crashing even once for more than a minute despite some record traffic related to our Mumbai coverage. If it weren’t for Monit I’d probably still be getting calls in the middle of the night saying the site was down.

You still need to find the bugs in your server configuration, or move to more powerful hardware (what we’re doing), but even if its annoying that the apache needs to be restarted every few minutes in order to not crash the server, its better than having it crash randomly when you’re not around. While you’re still tuning the system, you can have it email you based on certain conditions, so you can see how often a certain status is reached and determine whether a restart is necessary. The manual explains the functions pretty well and isn’t too long.

My advice if you’re setting it up

Setting the reset/exec levels

If you’re setting it up for the first time and you’re not having any problems at the moment you should be careful not to set the percentages too high, or the server might crash before it got that bad. I was giving a Memory/RAM max around 80%, but the remaining 20% didn’t seem to be enough to save the server, it was already to late. Here are my settings for our Apache webserver:

check system server1.globalvoicesonline.org
if loadavg (1min) > 5 for 2 cycles then exec "/etc/init.d/httpd restart"
if loadavg (1min) > 7 for 1 cycles then exec "/etc/init.d/httpd restart"
if memory usage > 65% for 3 cycles then exec "/etc/init.d/httpd restart"
if memory usage > 75% then exec "/etc/init.d/httpd restart"

The ‘exec’ action is running the apache restart command directly, which will clear out all appache processes and restart them, freeing up RAM temporarily. I’m also running two levels of Load checking, which will measure the strain on the CPU. Together these cover a lot of situations that result in crashes, and there are two versions of each, one for bad situations that have gone on for awhile (“for 3 cycles” i.e. 3 minutes) and one for terrible situations that are seen even once (“for 1 cycles”, which is actually unnecessary to write).

You can actually also set up monitoring of specific processes like Apache or other servers, but its been a lot buggier (thinks the program’s not running when it is) for me than the raw server statistics, so use at your own risk.

Alternate Email Formatting for Monit

The default email format template that comes with Monit is pretty hard to read to the point where it’s kind of maddening to recieve messages from it. Luckily they offer a custom mail formatting api so you can make one that makes sense for you. The pieces they give you are a bit limiting but I worked out one that is very short and clear and should even work okay as an sms:

set mail-format {
from: monit@yourserver.org
subject: [$ACTION] $EVENT on $SERVICE
message: $DESCRIPTION
- – - – - – - – - – - – - – - – - -
Action: [$ACTION] at $DATE from $HOST

–monit
}

Which sends you emails like:

[exec] Resource limit matched for server2.globalvoicesonline.org
’server.yourdomain.org’ mem usage of 71.9% matches resource limit [mem usage>65.0%]
- – - – - – - – - – - – - – - – - -
Action: [exec] at Fri, 28 Nov 2008 19:35:22 -0500 from server.yourdomain.org

–monit

Which I think is a lot better than the default. Any Monit users out there with a good format I’d love to see what else you’ve come up with.

click image for larger version

Thanks to Jules Rincon for her quick reflexes getting this photo of the tv for us all!

(graphic by Solana Larsen, click to see bigger version)

Most of those lists run on the Google Groups service, which like everything Big Gee does is simple, elegant and feature rich, epecially in how it presents archives on the web in case you don’t keep them in your mailbox. It lets us keep using the fairly-archaic but very effective email list structure while also giving us the same opportunities as say a web-based forum or Drupal community.

Google Groups SPAM and how to stop it
In the last few days a lot of our extra groups have started getting spammed by sham user accounts that join your group and post about puffy nipples and see-through tops, something that never happened before. It seems that Google uses a captcha (“type in the letters to prove your a human”) to stop such spam accounts that used to work, but now the spammers have managed to solve it using robots and thus create all the accounts they want. (here’s an article about it with an annoying ad before you can read it)

So far it seems that only open-membership groups are effected, so if your group is invite only or you moderate your posts you shouldn’t have too many real problems. That said, at Global Voices we’re discovering that some of our open groups used to feel closed, and need to be secured now that the spam has started.

If you run a google group, log into the admin interface by going to groups.google.com and choosing your list, then go to GROUP SETTINGS > ACCESS (access is a tab at the top) and check the following settings:

• Who can join? : If your group is small/exclusive enough then you might want to set this to “People can request an invitation to join”, that way you get an email before they join and you can check their posting history to see if they look like spammers. Each Google Groups user has a profile, and the spambots show their colors with the obviousness of their previous postings to other random groups.
• Who can post messages? : If you are only using your list to send messages out and don’t actually have disucussions, set this to “Managers only”. It should definitely be set at least to Members only.
• Message moderation: Choosing the “Messages from new members are moderated” option should help stop spammers. Unless you approve every new user account (time consuming and frustrating for users) the spambots will be able to create accounts, but in my experience so far they always post right away with some sex spam, so if you moderate just the new messages from each member you’ll avoid anyone else seeing the spam.
• Save Settings: Dont’ forget to save!

I’ll try to update this post if my advice turns out not to work. Any other advice about securing GGroups from this stuff without completely locking them down is very welcome!

I’ve heard only great things about past Wordcamps (they’re all over the place, there’s one in Toronto in October and a New York one on the same day!), and I’m deeply honored to be able to represent Global Voices and all the amazing people that make it a reality at the SF meeting. I can’t think of anywhere I’d rather be, let alone be the one speaking.

If anyone is in the area and into Wordpress you should check it out, if I know you and you’re from SF drop me a line, I’ll be in town for a few days.

And because i made this badge and love how it looks:

P.S. if you’re looking for some actual content check out Jim Kunstler’s Clusterfuck Nation, that’s what’s on my mind.