An Open Letter To Operators Of Redirection Sites
Dear Redirection Site Operator,
In our efforts to help identify unsolicited messages and related abuse, we have noticed the use of redirection sites in unsolicited messages. The effect is to "wash" the sites through redirectors presumably to defeat URI-matching techniques. In essence, abusers are using redirection sites as legitimate-looking camouflage to conceal their sites and bypass URI checks.
URI-checking mail filter programs such as SpamAssassin have been updated to filter out the redirected sites when a destination remains visible, for example as part of a path or in a CGI argument. However for those "opaque" redirector sites which hide, encode or key the destination so that it's not visible (after extraction or decoding) in the URI, the only option remaining for URI checkers is to follow the redirector through to see what site it leads to. Clearly this would be too resource-expensive for most mail filters, especially if a chain of multiple redirections were used.
Without a doubt abusers will figure out this loophole soon enough, and the abuse of redirectors in unsolicited messages will increase as a result.
A good solution to the issue of abusers washing their URIs through redirectors may be for the operators of redirector sites to deny services to sites listed on SURBLs. Several operators of redirection services are doing this currently, as you can see from the news below. Perhaps the worst solution would be to do nothing and let abuse of redirector services continue. This also increases the operating and abuse-handling costs of the redirector service, not to mention the costs to the community of unsolicited messages and related phishing and malware sites.
Therefore we appeal to operators of redirection sites to deny access to your services for SURBLed sites. This is easy to do via a DNS query, as explained in the Implementation Guidelines. You may be able to do these queries using existing code and scripts on our Links Page and in our best practice recommendations below.
Best Practice Recommendations
- Check links against SURBL lists using existing programs such as the below.
- In addition to checking links at creation time, also recheck periodically including when redirecting. But don't check on every redirection or too frequently. Rechecking once every 10 minutes may be often enough.
- Deny link creation from malicious IP addresses, for example using Spamhaus lists.
- Also check the resolved IP addresses of domain name and nameserver records against lists such as Spamhaus.
- Disallow redirection to other redirectors. In other words, don't shorten other shortener sites.
- Return an HTTP status code of 410 when an abused redirector has been removed. This makes it easier to determine that the abuse has been removed and no longer requires reporting. You can still monetize the page by providing a page that browsers can display as well.
- Register an abuse contact role account at abuse.net, which is widely used to find where to report abuse.
SURBL is now sending shortener abuse reports in ARF (RFC 5965) format.
Partial list of shorteners using SURBLs
Ask Bjørn Hansen of Metamark.net is now using SURBL data to deny services to abusers:
30 April 2004: Ask Bjørn Hansen of develooper.com is using SURBL data to block abused sites in the Metamark Shorten™ Service URI shortening and redirection service. This is the first use of SURBL data to prevent abuse of a redirection site that we've heard of! Great going! Ask explains his motivation as: "I mostly did it to make it less likely that I'll have to deal with abusers of the service manually. Hopefully the other redirection services will realize that benefit soon as well."
SnipURL is another redirection service that is using SURBLs in a similar way:
23 July 2004: SnipURL is now using SURBLs to deny abusers access to their URL shortening and redirection service.
And TinyURL is also using SURBLs:
17 November 2005: Kevin Gilbertson reports that he has been using SURBLs for "about a year now" to protect his popular redirection site TinyURL.com against abusers and phishers.
As is Notlong.com:
18 November 2005: Eric Hammond says that his Notlong.com redirection service "has been protected by SURBL since July 2004."
As is easyurl.net:
27 September 2006: Mark Jeftovic reports that they "are now checking destination URLs against [SURBLs] and refusing to shorten them via easyurl.net."
21 February 2008: Daniel Flandorfer says: "Our redirection service YATUC (yet another tiny url creator) uses SURBL data to check links added into the system. Every link will from now on be checked against SURBL and - if not passed - not be added into the system. [...] Additionally, we will periodically check all links in our system and if needed mark them so that they can't be used any longer. Today we already marked 858 links as spam !! We hope that we can help a bit to reduce the massive use of spam urls."
4 August 2008: Nathan Folkman has integrated SURBL checks into his bit.ly URL shortening service.
4 August 2008: Richard West says, "I consider [SURBL] an extremely valuable tool to deny redirection to the majority of spam sites and greatly decrease the amount of abuse we experience."
25 March 2009: David Harper says, "Delivr.com has integrated SURBL checks into its mobile-friendly sharing service."
As is safe.mn:
31 March 2011: "We are using SURBL in cort.as due to our main interest in security for our users. SPAM is one of the biggest problems for URL shorteners and we are strongly fighting spammer sites," says Raul Rivero, CTO of elpais.com.
Other shortener services not listed above are also using SURBL data. At this point using SURBL data has become nearly a standard practice against shortener abuse. Please see more references at: http://www.surbl.org/news.html.
Thank you for your attention and your hopeful consideration of using SURBL data to stop the abuse of your services.
SURBL Data Feed Request
SURBL Data Feeds offer higher performance for professional users through faster updates and resulting fresher data. Freshness matters since the threat behavior is often highly dynamic, so Data Feed users can expect higher detection rates and lower false negatives.
The main data set is available in different formats:
Rsync and DNS are typically used for mail filtering and RPZ for web filtering. High-volume systems and non-filter uses such as security research should use rsync.
For more information, please contact your SURBL reseller or see the references in Links.
Sign up for SURBL Data Feed Access.