This is an open letter to Facebook because their feedback system is very limited (no room for detail) and I’d rather have a public place I can link to with full details. If you work for Facebook, please tell me who I can talk to that has the power to change infrastructure details like this. If you don’t work for Facebook but know a good place I can share this important information, please point me in the right direction.
EDIT 2018-02: Somehow this bug persists! Despite multiple attempts by Facebook to fix it, multiple sysadmins are still experiencing the problem of FB bots visiting with no user agent. Even worse, it seems that same bot has a new bug in it, where it sends dozens/hundreds of visits in a short period, effectively generating a DDOS attack with no UA accountability! Obviously this is totally unacceptable, and it’s time for Facebook to do whatever it takes to fix this bug. You can read my most recent ticket on the “Facebook for Developers: Support” bug tracker (and leave a comment saying how you believe this needs to be fixed) here: Facebook HTTP media fetcher has no User Agent
EDIT 2017-08: In the end I DID find the correct place to post a technical bug to Facebook, and created a ticket about this problem with the empty user agents. I got a decent reply from them where they accepted that the problem is real and stated their intention to fix it in February 2017. As of August 2017, they claim to have fixed the problem, but unfortunately I was able to find more examples of the problem in my logs after they closed the ticket as fixed. Hopefully it will start working soon, and I’ll update this again. Below is the original post, which assumes the problem persists.
Continue reading “Facebook needs to stop using empty user agents FFS”