Very happy to share the news that Global Voices has fully implemented Amber, a project from Berkman Klein Center for Internet & Society at Harvard University that backs up every site we link to in case it ever goes down ???
It was a big project getting Amber to handle our 100k post archive, but the satisfaction of having a tool like this be part of my job is exactly why I cherish working for GV. It’s a bit of a love note to GV, The Internet Archive, and the whole endeavor of preserving the web both socially and technologically ?
Thanks to Carl Alexander, who works with me on the backend and programming side of Global Voices, together we came up with some great patches for the plugin that we shared back upstream on Github ???
GV is Preserving the Web in Amber and Why That's Great News
This is an open letter to Facebook because their feedback system is very limited (no room for detail) and I’d rather have a public place I can link to with full details. If you work for Facebook, please tell me who I can talk to that has the power to change infrastructure details like this. If you don’t work for Facebook but know a good place I can share this important information, please point me in the right direction.
EDIT 2018-02: Somehow this bug persists! Despite multiple attempts by Facebook to fix it, multiple sysadmins are still experiencing the problem of FB bots visiting with no user agent. Even worse, it seems that same bot has a new bug in it, where it sends dozens/hundreds of visits in a short period, effectively generating a DDOS attack with no UA accountability! Obviously this is totally unacceptable, and it’s time for Facebook to do whatever it takes to fix this bug. You can read my most recent ticket on the “Facebook for Developers: Support” bug tracker (and leave a comment saying how you believe this needs to be fixed) here: Facebook HTTP media fetcher has no User Agent
EDIT 2017-08: In the end I DID find the correct place to post a technical bug to Facebook, and created a ticket about this problem with the empty user agents. I got a decent reply from them where they accepted that the problem is real and stated their intention to fix it in February 2017. As of August 2017, they claim to have fixed the problem, but unfortunately I was able to find more examples of the problem in my logs after they closed the ticket as fixed. Hopefully it will start working soon, and I’ll update this again. Below is the original post, which assumes the problem persists.
Continue reading “Facebook needs to stop using empty user agents FFS”