SURBL

Locally mirror dnsbl zone files using rbldnsd with BIND on FreeBSD

UPDATE: FreeBSD 6.2 and later appear to have fully-integrated support for rbldnsd from the ports collection. So several of the steps below are not needed, and the configuration is more cleanly integrated. Start by installing rbldnsd from /usr/ports/dns/rbldnsd . The config file is: /usr/local/etc/rc.d/rbldnsd . To enable rbldnsd upon booting, add rbldnsd_enable="YES" to /etc/rc.conf

Here are some notes on setting up rbldnsd to run with BIND 8 or 9 under FreeBSD in order to locally cache and serve dnsbl zone files.

1. If needed, install rsync from the FreeBSD ports collection or using the package tarball. rbldnsd seems to be installed by default in FreeBSD 6.2 and later, but it is a good idea to update it to get the latest security patches.

    

2. Install rbldnsd from the FreeBSD ports collection or using the package tarball. The install should add the manual pages, put the binary in /usr/local/sbin/rbldnsd, create a sample startup script in /usr/local/etc/rc.d/rbldnsd.sh, etc. The install also creates an rbldns user and group, which rbldnsd can run as. We changed the ownership of the scripts and directories used by rbldnsd to user and group rbldns also.

   ---

   Note that the generic rbldnsd tarball does not install manual pages, or create directories or an rbldns user or group. All it does is create the rbldnsd binary after a "make configure" and a "make". When using the original rbldnsd tarball instead of the FreeBSD ports collection version, you will need to make the other changes by hand if you choose to follow them. Remember to add a startup script appropriate to your operating system so that rbldnsd starts when your system boots.

   ---

3. Modify the default sample rbldnsd startup script /usr/local/etc/rc.d/rbldnsd.sh to use appropriate flags such as:

   		rbldnsd_flags=${rbldnsd_flags:-"-u rbldns:rbldns 
       -p /var/run/rbldnsd.pid -r /usr/local/etc/rbldnsd -b 10.11.12.153/53 
       -t 900 multi.surbl.org:dnset:multi.surbl.org.rbldnsd"}
   
   

   where -u specifies the user and group to run as, -p is the pid file location, -r is the rbldnsd root directory, -b is the address and port to bind to, and the remaining arguments define the zones, their types and their files.

   ---

   Note that most SURBL applications and installations will probably use only the multi.surbl.org zone. All zones other than multi may go away someday. They're all obsolete since they're all included in multi.

   ---

4. If using BIND 8, then edit /etc/rc.conf to ifconfig a fake internal address at startup time on the loopback (or a real) interface for rbldnsd to run on:

   		ifconfig_lo0_alias0="inet 10.11.12.153 netmask 255.255.255.255"
   
   

   and ifconfig it manually now also:

   		ifconfig lo0 10.11.12.153 netmask 255.255.255.255 alias
   

   If using BIND 9, then rbldnsd can run on an existing address, including the loopback address 127.0.0.1.

   ---

   For BIND 9, substitute something like 127.0.0.1/650 for 10.11.12.153/53 in the examples below. No ifconfig should be needed with BIND 9 assuming you can specify some unused port on an existing address.

   ---

5. Also add the following to /etc/rc.conf so that rbldnsd starts at boot time, in conjunction with the startup script mentioned above:

   		# Start rbldnsd
     #  See: /usr/local/etc/rc.d/rbldnsd.sh  (startup script)
     #  And: /usr/local/etc/rbldnsd          (data directory)
     rbldnsd_enable="YES"
   

6. Contact the RBL operator to ask for rsync access to their zone files. For example, to apply for SURBL rsync access or Spamhaus datafeed access.

    

7. Create an rsync script as /usr/local/etc/rbldnsd/rsync-zone-files : VERY IMPORTANT: Make your script check to see if the previous rsync is still running. If the previous rsync is still running, then the cron job should not start another rsync. This is important to prevent a multiple rsync processes from starting up if there are any unusual delays. For example, here's a tcsh script to rsync the files from a crontab:

   		#!/bin/tcsh -f
   
   # rsync the zone file only if this program is not already running
   
   # make sure lock file exists first
   /usr/bin/touch /usr/local/etc/rbldnsd/lockfile
   
   if ( -z /usr/local/etc/rbldnsd/lockfile ) then
     echo "rsync is running" > /usr/local/etc/rbldnsd/lockfile
     /usr/local/bin/rsync -aq "server_name_here::surbl/multi.surbl.org.rbldnsd" /usr/local/etc/rbldnsd/
     echo -n "" > /usr/local/etc/rbldnsd/lockfile
   endif
   

   (It only rsyncs if the lockfile is empty.) Where rsync_server_here is replaced with the actual server name provided when rsync access is granted. Note that multi.surbl.org.rbldnsd is the only zone that should be used.

   Here's a more sophisticated bash script which Chris Zutler has put in the public domain that includes locking and will restart a stuck rsync process:

   		#!/bin/bash
   
   LOCK_DIR="/tmp"
   basename=$(basename $0)
   
   print_usage() {
           echo "Usage: ${basename} [-s sleep [-t timeout] "
           echo "Run COMMAND with simple file locking. "
           echo "    -s sleep"
           echo "        Sleep between 0 and the number of seconds specified before running command."
           echo "    -t timeout"
           echo "        Attempt to kill the old process if the lock is older than the specified number of seconds."
           echo
           exit
   }
   
   while [ "${1}" == "-s" ] || [ "${1}" == "-t" ]; do
           opt="${1}"
           shift
           if [ -z "$(expr "${1}" + 0)" ] || [ "${1}" -lt 0 ]; then
                   echo "Invalid number of seconds."
                   print_usage
           fi
           case "${opt}" in
                   "-s") sleep="${1}" ;;
                   "-t") timeout="${1}" ;;
           esac
           shift
   done
   
   if [ -z "$*" ]; then
           print_usage
   else
           cmd=$*
   fi
   
   lock="${LOCK_DIR}/$(basename ${1}).$(echo ${cmd} | md5sum | cut -d" " -f1).lock"
   
   if (set -o noclobber; echo -n > "${lock}") 2> /dev/null; then
           echo $$          >> "${lock}"
           echo $(date +%s) >> "${lock}"
           echo $cmd        >> "${lock}"
   
           if [ "${sleep}" -gt 0 ]; then
                   sleep $(expr ${RANDOM} % ${sleep})
           fi
   
           ${cmd}
   
           rm "${lock}"
   elif [ -n "${timeout}" ]; then
           read -d! lockpid locktime lockcmd < <(cat "${lock}" 2>/dev/null || echo NOFILE 0 0)
   
           if [ -n "$(expr "${locktime}" + 0)" ] && [ $(( $(date +%s)-${locktime} )) -gt "${timeout}" ]; then
                   # Try to kill grand children, children and then parent
                   pkill -P $(pgrep -d, -P ${lockpid}) > /dev/null 2>&1 || 
                   pkill -P ${lockpid}                 > /dev/null 2>&1 ||        
                   (kill ${lockpid}; rm "${lock}")     > /dev/null 2>&1
           fi
   fi
   

   Xia Qingran of SINA.com points out that FreeBSD has its own lockfile program called "lockf", and he uses it to prevent multiple rsyncs. -s means silent; -t0 means terminate at 0 seconds (immediately); see 'man lockf':

   		7/20 * * * * rbldns lockf -st0 /var/run/rsync_surbl.lock /usr/local/sbin/rsync_surbl.sh
   

    

8. Run the rsync script manually to test it, and see if it caused some fresh zone files to show up in the /usr/local/etc/rbldnsd directory. If the files aren't showing up, then it can be useful to leave off the "> /dev/null" at the end of the rsync script to show all the output for debugging purposes.

    

9. Install the rsync script in /etc/crontab or a root or rbldns crontab:

   		# rsync dnsbl zone files
     X,Y,Z *      *       *       *       rbldns  /usr/local/etc/rbldnsd/rsync-zone-files
   

   Notes:
   1. X, Y and Z are some minutes or an expression.
   2. Note that each dnsbl may have its own policies about when you should rsync, which you should follow.
   3. "rbldns" above sets the user which the job runs as, when using a FreeBSD root /etc/crontab. For a normal user crontab, that field does not exist.

    

10. Create an manual rbldnsd start and stop script in /usr/local/etc/rbldnsd/rbldnsd.sh (note that recent versions of rbldnsd automatically install an appropriate subr script in /usr/local/etc/rc.d, but it will need to be edited to start the specific zones you need to run locally):

    		#!/bin/sh
    #
    # /usr/local/etc/rbldnsd/rbldnsd.sh
    
    PATH=$PATH:/usr/bin:/usr/local/sbin
    
    case "$1" in
    'start')
            if [ -x /usr/local/sbin/rbldnsd ]
            then
                    /usr/local/sbin/rbldnsd 
                    -u rbldns:rbldns 
                    -r /usr/local/etc/rbldnsd 
                    -b 10.11.12.153/53 
                    -t 900s 
                    -c 1m 
                    -p /var/run/rbldnsd.pid 
                    multi.surbl.org:dnset:multi.surbl.org.rbldnsd 
            fi
            ;;
    
    'stop')
            if [ -r /var/run/rbldnsd.pid ]
            then
                    /bin/kill -9 `cat /var/run/rbldnsd.pid` > /dev/null
            fi
            ;;
    
    *)
            echo "Usage: $0 { start | stop }"
            exit 1
            ;;
    esac
    exit 0
    

    And use it to start rbldnsd manually now, noting any error messages. If things are happy it should look like this:

    		% /usr/local/etc/rbldnsd/rbldnsd.sh start
      rbldnsd: listening on 10.11.12.153/53
      rbldnsd: dnset:multi.surbl.org.rbldnsd: 20040518 070014: e/w=118583/0
      rbldnsd: zones reloaded, time 19.42e/18.0u sec
      rbldnsd: rbldnsd version 0.992 (7 Mar 2004) started (1 socket(s), 1 zone(s))
    
    

    ---

    Note that it's only necessary to stop and restart rbldnsd when adding or removing a zone entirely. Changes to existing zone files, for example those resulting from rsyncs, are picked up automatically by rbldnsd every 60 seconds. Be sure to use rsync -a or other flags which preserve original file timestamps, since rbldnsd uses file modification times to determine when a zone has changed.

    ---

11. Edit /etc/namedb/named.conf to tell BIND to forward queries for the dnsbl domains to the local IP address and/or port which rbldnsd is running on.

     

    • For BIND 9 simply specify the IP and port rbldnsd is using:

      				zone "multi.surbl.org" IN {
              type forward;
              forward first;
              forwarders {
                      127.0.0.1 port 650;
              };
      };
      

      Where "forward first" means try to forward to the other server first, but fall back to local resolution if the other server is not available.

       

    • In contrast, BIND 8 can only operate on port 53. So in order to tell it to forward responses for certain domains, first we need to tell it what specific local addresses BIND 8 itself should respond on:

      				options {
      
              // Let BIND use only these specific addresses
              // meaning that unlisted addresses such as 10.11.12.153:53
              // can be used by rbldnsd instead
              listen-on {
                      N.N.N.N;
                      127.0.0.1;
              };
      };
      

      Where N.N.N.N is the real address of the server, and where all other aliased IP addresses should also be listed here. (Note that blocks may also be specified in CIDR notation, such as N.N.N.N/29.) Note that we specifically don't list rbldnsd's local address so that BIND 8 doesn't try to respond on it. If we don't configure listen-on, then we get an error message like:

      				May 17 22:26:26 myserver named[38142]: bind(dfd=24, [10.11.12.153].53): Address already in use
        May 17 22:26:26 myserver named[38142]: deleting interface [10.11.12.153].53
      

      Which results from BIND trying to run on (port 53 of) the address that rbldnsd is already running on. So we need to set the listen-on option in BIND 8.

      Then tell BIND 8 where to forward queries for the dnsbl domains:

      				zone "multi.surbl.org" IN {
              type forward;
              forward first;
              forwarders {
                      10.11.12.153;
              };
      };
      

      (BIND 8 does not know anything about ports other than 53, so we can't specify a port, and we must use some other address to forward requests to rbldnsd.)

    ---

    Note: It can be quite useful to try name resolution using rbldnsd on its own address or port, before starting the BIND fowarding in the next step. That way you can check that rbldnsd itself is serving up the zones correctly, independently of BIND. For example, with BIND 8:

    			dig test.surbl.org.multi.surbl.org @10.11.12.153
    

    or for BIND 9 setups, assuming 127.0.0.1 address and port 650:

    			dig test.surbl.org.multi.surbl.org @127.0.0.1 -p 650
    

    ---

12. Reload BIND and confirm that it is serving up the dnsbl zones with the appropriate serial numbers, testpoints resolving, etc.