idea: a "no crawlers" option that followbots and search engines have to both check and respect
it can be applied both per-user and instance-wide