Locally mirror RBL zone files using rbldnsd with BIND on FreeBSD

Here are some notes on setting up rbldnsd to run with BIND 8 or 9 under FreeBSD in order to locally cache and serve RBL zone files.
  1. If needed, install rsync from the FreeBSD ports collection or using the package tarball. Typically rsync is installed by default, though it is a good idea to update it to get the latest security patches. Update: rbldnsd seems to be installed by default in FreeBSD 6.2.

  2. Install rbldnsd from the FreeBSD ports collection or using the package tarball. The install should add the manual pages, put the binary in /usr/local/sbin/rbldnsd, create a sample startup script in /usr/local/etc/rc.d/rbldnsd.sh, etc. The install also creates an rbldns user and group, which rbldnsd can run as. We changed the ownership of the scripts and directories used by rbldnsd to user and group rbldns also.

    Note that the generic rbldnsd tarball does not install manual pages, or create directories or an rbldns user or group. All it does is create the rbldnsd binary after a "make configure" and a "make". When using the original rbldnsd tarball instead of the FreeBSD ports collection version, you will need to make the other changes by hand if you choose to follow them. Remember to add a startup script appropriate to your operating system so that rbldnsd starts when your system boots.
  3. Modify the default sample rbldnsd startup script /usr/local/etc/rc.d/rbldnsd.sh to use appropriate flags such as:
      rbldnsd_flags=${rbldnsd_flags:-"-u rbldns:rbldns -p /var/run/rbldnsd.pid -r /usr/local/etc/rbldnsd -b 10.11.12.153/53 -t 900 multi.surbl.org:dnset:multi.surbl.org.rbldnsd list.dsbl.org:ip4set:list.dsbl.org.rbldns"}
    
    where -u specifies the user and group to run as, -p is the pid file location, -r is the rbldnsd root directory, -b is the address and port to bind to, and the remaining arguments define the zones, their types and their files.

    Note that most SURBL applications and installations will probably use only the multi.surbl.org zone. The main exception are those running public DNS servers for the surbl.org subdomains (i.e. publically serving the list data). Public name servers should serve all 6 zones, multi, sc, ws, be, ob, ab.
  4. If using BIND 8, then edit /etc/rc.conf to ifconfig a fake internal address at startup time on the loopback (or a real) interface for rbldnsd to run on:
      ifconfig_lo0_alias0="inet 10.11.12.153 netmask 255.255.255.255"
    
    and ifconfig it manually now also:
      ifconfig lo0 10.11.12.153 netmask 255.255.255.255 alias
    
    If using BIND 9, then rbldnsd can run on an existing address, including the loopback address 127.0.0.1.

    For BIND 9, substitute something like 127.0.0.1/650 for 10.11.12.153/53 in the examples below. No ifconfig should be needed with BIND 9 assuming you can specify some unused port on an existing address.
  5. Also add the following to /etc/rc.conf so that rbldnsd starts at boot time, in conjunction with the startup script mentioned above:
      # Start rbldnsd
      #  See: /usr/local/etc/rc.d/rbldnsd.sh  (startup script)
      #  And: /usr/local/etc/rbldnsd          (data directory)
      rbldnsd_enable="YES"
    
  6. Contact the RBLs to ask for rsync access to their zone files. For example, to apply for SURBL rsync access or Spamhaus sbl, xbl or sbl-xbl access.

  7. Create an rsync script as /usr/local/etc/rbldnsd/rsync-zone-files : VERY IMPORTANT: Make your script check to see if the previous rsync is still running. If the previous rsync is still running, then the cron job should not start another rsync. This is important to prevent a multiple rsync processes from starting up if there are any unusual delays. For example, here's a tcsh script to rsync the files from a crontab:
    #!/bin/tcsh
    
    # rsync the zone files only if this program is not already running
    
    if ( -z /usr/local/etc/rbldnsd/lockfile ) then
      echo "rsync is running" > /usr/local/etc/rbldnsd/lockfile
      /usr/local/bin/rsync -aq "server.name.here::surbl/*.rbldnsd" /usr/local/etc/rbldnsd/
      echo -n "" > /usr/local/etc/rbldnsd/lockfile
    endif
    
    (It only rsyncs if a lockfile is empty.) Where rsync.server.here is replaced with the actual server name provided when rsync access is granted.

    Here's a more sophisticated bash script which Chris Zutler of Habeas has put in the public domain that includes locking and will restart a stuck rsync process:

    #!/bin/bash
    
    LOCK_DIR="/tmp"
    basename=`basename $0`
    
    function print_usage {
        echo "Usage: $basename [-s sleep [-t timeout] "
        echo "Run COMMAND with simple file locking. "
        echo "    -s sleep"
        echo "        Sleep between 0 and the number of seconds specified
    before running command."
        echo "    -t timeout"
        echo "        Attempt to kill the old process if the lock is older
    than the specified number of seconds."
        echo
        exit
    }
    
    while [ "$1" == "-s" ] || [ "$1" == "-t" ]; do
        opt=$1
        shift
        if [ -z "`expr "$1" + 0`" ] || [ "$1" -lt 0 ]; then
            echo "Invalid number of seconds."
            print_usage
        fi
        case "$opt" in
            "-s") sleep=$1 ;;
            "-t") timeout=$1 ;;
        esac
        shift
    done
    
    if [ -z "$*" ]; then
        print_usage
    fi
    
    lock="${LOCK_DIR}/`basename $1`.`echo $cmd | md5sum - | cut -d" " -f1`.lock"
    
    if [ -e "$lock" ]; then
        if [ -n "$timeout" ]; then
            let diff="`date +%s`"-"`sed '2!d' "$lock"`"
            if [ "$diff" -gt "$timeout" ]; then
                pkill -P `sed '1!d' "$lock"`
            fi
        fi
        exit
    else
        echo $$ > "$lock"
        echo `date +%s` >> "$lock"
        echo $cmd >> "$lock"
        if [ -n "$sleep" ]; then sleep `expr $RANDOM % $sleep`; fi
        $cmd
        rm "$lock"
    fi
    

    Xia Qingran of SINA.com points out that FreeBSD has its own lockfile program called "lockf", and he uses it to prevent multiple rsyncs. -s means silent; -t0 means terminate at 0 seconds (immediately); see 'man lockf':

    7/20 * * * * rbldns lockf -st0 /var/run/rsync_surbl.lock /usr/local/sbin/rsync_surbl.sh
    

  8. Run the rsync script manually to test it, and see if it caused some fresh zone files to show up in the /usr/local/etc/rbldnsd directory. If the files aren't showing up, then it can be useful to leave off the "> /dev/null" at the end of the rsync script to show all the output for debugging purposes.

  9. Install the rsync script in /etc/crontab or a root or rbldns crontab:
      # rsync RBL zone files
      X,Y,Z *      *       *       *       rbldns  /usr/local/etc/rbldnsd/rsync-zone-files
    
    Notes:
    1. X, Y and Z are some random minutes, at most three times per hour, avoiding 0, 30, 20, 40, etc.
    2. Note that each RBL may have its own policies about when you should rsync, which you should follow.
    3. "rbldns" above sets the user which the job runs as, when using a FreeBSD /etc/crontab. For a normal user crontab, that field does not exist.

  10. Create an manual rbldnsd start and stop script in /usr/local/etc/rbldnsd/rbldnsd.sh (note that recent versions of rbldnsd automatically install an appropriate subr script in /usr/local/etc/rc.d, but it will need to be edited to start the specific zones you need to run locally):
    #!/bin/sh
    #
    # /usr/local/etc/rbldnsd/rbldnsd.sh
    
    PATH=$PATH:/usr/bin:/usr/local/sbin
    
    case "$1" in
    'start')
            if [ -x /usr/local/sbin/rbldnsd ]
            then
                    /usr/local/sbin/rbldnsd \
                    -u rbldns:rbldns \
                    -r /usr/local/etc/rbldnsd \
                    -b 10.11.12.153/53 \
                    -t 900s \
                    -c 1m \
                    -p /var/run/rbldnsd.pid \
                    multi.surbl.org:dnset:multi.surbl.org.rbldnsd \
                    list.dsbl.org:ip4set:rbldns-list.dsbl.org
            fi
            ;;
    
    'stop')
            if [ -r /var/run/rbldnsd.pid ]
            then
                    /bin/kill -9 `cat /var/run/rbldnsd.pid` > /dev/null
            fi
            ;;
    
    *)
            echo "Usage: $0 { start | stop }"
            exit 1
            ;;
    esac
    exit 0
    
    And use it to start rbldnsd manually now, noting any error messages. If things are happy it should look like this:
      % /usr/local/etc/rbldnsd/rbldnsd.sh start
      rbldnsd: listening on 10.11.12.153/53
      rbldnsd: dnset:multi.surbl.org.rbldnsd: 20040518 070014: e/w=118583/0
      rbldnsd: ip4set:list.dsbl.org.rbldnsd: 20040518 073634: e/w=3284690/0
      rbldnsd: zones reloaded, time 19.42e/18.0u sec
      rbldnsd: rbldnsd version 0.992 (7 Mar 2004) started (1 socket(s), 3 zone(s))
    

    Note that it's only necessary to stop and restart rbldnsd when adding or removing a zone entirely. Changes to existing zone files, for example those resulting from rsyncs, are picked up automatically by rbldnsd every 60 seconds. Be sure to use rsync -t or other flags which preserve original file timestamps, since rbldnsd uses file modification times to determine when a zone has changed.
  11. Edit /etc/named.conf to tell BIND to forward queries for the RBL domains to the local IP address and/or port which rbldnsd is running on.


    Note: It can be quite useful to try name resolution using rbldnsd on its own address or port, before starting the BIND fowarding in the next step. That way you can check that rbldnsd itself is serving up the zones correctly, independently of BIND. For example, with BIND 8:
      dig test.surbl.org.multi.surbl.org @10.11.12.153
    
    or for BIND 9 setups, assuming 127.0.0.1 address and port 650:
      dig test.surbl.org.multi.surbl.org @127.0.0.1 -p 650
    

  12. Reload BIND and confirm that it is serving up the RBL zones with the appropriate serial numbers, testpoints resolving, etc.

rbldnsd-bind-freebsd.html version 1.08 by Jeff Chan on 4/6/05