Faulbaer's Schlafmulde :: Aug 2010

djbdns + svn + cron = dns replication

I'm sure this has been done before - probably even pretty often - but since I couldn't find anything on how to make it work properly without spending too much time on it, I decided to write up what I did there.

I wanted to replicate my dns-database between several djbdns installations on different operating systems. When I first set this up, I went for ssh without password. there were some minor issues with that:

- no passphrase (must be something spiritual but I don't like that)
- without rsync involved: one-way only (as in master->slave, slave...)
- with rsync involved still not optimal to keep env, root etc. apart
- no versioning, no roll-back, no safety

on the plus side it's been super-easy to set it up, though.

since I recently bought into a new server plan and just a month before found and old wrap-pc that had been configured as openbsd-dns-server years ago, that was still working, I decided to put some more brain into the matter. these were the results I was looking for:

- it shouldn't matter where to change configuration
- shouldn't be needed to log into the dns server ever again
- versioning/backup/roll-back should be an option
- only parts of the directory should be replicated/distributed
- take into consideration that not every server is the same
- server should pull the updates
- encrypted communication
- authentication required for at least writing

the main trigger was going to be cron like before but I had to find something for allmy other needs. subversion was the solution.

I finally went for a subversion repository with different env-directories for each server as there are IP-address as well as the path to the root-directory configured and those can differ from server to server.

the svn is behind an apache2 just because it's already been there and I couldn't be bothered to make svnserver fly with ssl and authentication. apache works just fine. so we have svn behind apache/ssl and that part took my rusty brain two attempts and maybe an hour to get it working properly. I'm getting too old for this stuff.

the configuration of the servers didn't take much longer and finally writing a script for cron with all the necessary environment variables took another two hours. I bet every half-way routined administrator finishes this in less than an hour.

now let's talk source (and yes, I'm perfectly aware of there being room for improvement - so just let me know if you find something, will you?)

since djbdns/tinydns has become part of the even debian lenny distribution I think you will find the basic setup instructions for your favorite flavor of linux/unit real quick on any major search-engine. so let's dive into the specifics of my setup, will we?

first of all I'm kind of a control-freak when it comes to my servers. I didn't really like to have all domains on one data-file and decided to add a directory for configurations I'd later just "cat > data". also I prepared some files I was going to work with later.

cd /service/tinydns/root/
mkdir data.d
touch tosses.info.data
touch lsrv.net.data
touch make.sh
touch svnup.sh

I went on configuring my .data files like I'd do it for a regular tinydns data file. to assemble the data file into the root directory, I just hacked a little make.sh script like this:

vim make.sh

echo # This file will be generated by $DATAD/make.sh > $ROOT/data
cat $DATAD/*.data >> $ROOT/data
chmod +x make.sh
svn add make.sh
svn ci -m "build your data from data.d with make.sh"

my dnsservers were half-way done. next up had to be the subversion configuration at the svn-server as well as at the dns-servers.

at the subversion server I made a repository just for the dns servers. into that repository I made some directories and finally I added some basic auth ssl configuration to the apache setup:

mkdir -p /var/svn/repositories
chown www:www /var/svn/repositories
su www -
svn create /var/svn/repositories/dns
svn mkdir --parents /var/svn/repositories/dns/tinydns/root/data.d
svn mkdir --parents /var/svn/repositories/dns/tinydns/ns1/env
svn mkdir --parents /var/svn/repositories/dns/tinydns/ns2/env
svn mkdir --parents /var/svn/repositories/dns/tinydns/ns3/env

now I had the directories for data, data.cdb and the .data files as well as env directories for each of my dns servers.

the apache2 configuration was pretty simple as well. after enabling ssl and configuring the ports as well as the virtual hosts, I added the site configurations for my dns repository as follows as root:

vim /etc/apache2/sites_available/svn.tosses.info

  ServerName svn.tosses.info
  ServerAdmin svn@tosses.info
  SSLEngine on
  SSLCertificateFile /etc/apache2/ssl.crt
  SSLCertificateKeyFile /etc/apache2/ssl.key
  Alias /svn/ /var/svn/repositories/
  <Location /svn>
    DAV svn
    SVNParentPath /var/svn/repositories
  <Directory /var/svn/repositories/dns>
    AuthType Basic
    AuthName DNS
    AuthUserFile /etc/apache2/htpasswd.svn
    require valid-user
    order deny,allow
    deny from all
    satisfy any
  ErrorLog /var/log/svn.tosses.info-error.log
  CustomLog /var/log/svn.tosses.info-access.log common
htpasswd -c /etc/apache2/htpasswd.svn dns

after a graceful restart I could access the repository and see the directories I had made before. my work on the svn-server was done.

back on the (till then) main dns server I checked out the empty directories and started adding files to the svn:

cd /service/tinydns/env
svn co https://svn.tosses.info:443/svn/dns/ns1/env .
svn add IP ROOT
svn ci -m "IP and ROOT configuration for ns1 added"
cd /service/tinydns/root
svn co https://svn.tosses.info:443/svn/dns/root .
svn add data.d data.cdb
svn ci -m "general data.d directory and data.cdb database for all dns"

so far so good. for cron I still needed an svn update script including the necessary variables. those scripts needed to be executable and I had to add svnup.sh to the crontab:

cd /service/tinydns/root/data.d
vim svnup.sh

cd /service/tinydns/root
svn update
chmod +x svnup.sh
svn add svnup.sh
svn ci -m "now svnup.sh will update the svn"

crontab -e

* * * * * /service/tinydns/data.d/svnup.sh > /dev/null

for the other two servers all I had to do was to repeat the checkout of their own env directories and in the root directory I needed to remove data, data.d and data.cdb before a checkput could be successful. but those had to go anyway:

on ns2

cd /service/tinydns/env
svn co https://svn.tosses.info:443/svn/dns/ns2/env .
svn add IP ROOT
svn ci -m "IP and ROOT configuration for ns2 added"
cd /service/tinydns/root
rm -Rf data.d data data.cdb
svn co https://svn.tosses.info:443/svn/dns/root .

crontab -e

* * * * * /service/tinydns/data.d/svnup.sh > /dev/null

on ns3

cd /service/tinydns/env
svn co https://svn.tosses.info:443/svn/dns/ns3/env .
svn add IP ROOT
svn ci -m "IP and ROOT configuration for ns3 added"
cd /service/tinydns/root
rm -Rf data.d data data.cdb
svn co https://svn.tosses.info:443/svn/dns/root .

crontab -e

* * * * * /service/tinydns/data.d/svnup.sh > /dev/null

now every minute every server tries to update from the repository. sure, that's a lot and maybe I even go back to five minutes as checking interval.

the only thing that might become an issue is if I change files in data.d on several servers without immediate check-in. but there are enough ways to fix this in svn.

Faulbaer (I hope I haven't forgotten anything)tt

( tags :: , , , , )
[ 2010.08.23, 08:16 :: thema: /english/hacking :: link zum artikel :: 0 Comments ]

if in doubt ... use xen

it's done, I've decided and it's become a xen system on lenny. to sum it up real quick: ubuntu server: pain in the arse. kvm very much unfinished busines. vmbuilder hyped crap. virsh pretty much is like masochism.

if I hadn't spent all my energy on trying to get virsh and kvm running on ubuntu first and later on debian lenny - well - I might have given centos a try. but after two-and-a-half day I was just too exhausted for further experimenting and just worked with what I knew would work: debian lenny, xen and lvm. xm-create-image works like a charm and does what it promises. not half of it or only two thirds right and the rest is a mess - it just does what it can and that it does very well.

in fact I already started migrating services to the new box. the weblog is already there. I couldn't be happier. the really big thing is going to be the zimbra migration but I've done that before and it worked every time. so no bad expectations here.

I've decided to drop ejabberd since I'm the only user it's kind of an overkill. also I guess zimbra's jabber services should have improved by now - so I'm gonna try that.

I haven't yet decided if I'm going to upgrade the old server or use the old machine as backup system. will figure that out sooner or later - got enough time for thinking, now that the new system is running well and migration seems to be going on pretty easy, too.

firt blogpost on the new machine - hope it works - fingers crossed ;)

Faulbaer (permissions set properly? we'll see ...)

( tags :: , , )
[ 2010.08.19, 02:21 :: thema: /english/technology :: link zum artikel :: 0 Comments ]

kvm? I've had it.

I've really had it - I'm going back to xen - kvm may be the better, faster, more stable and more actively developed engine but to be honest, it's not just the engine, I need the tools around it to be working, too.

I've tried to make it work for one-and-a-half days now and that's kind of the longest time I've put into new software-product that is constantly failing, ever. kvm may be working like a charm but vmbuilder and virsh are nothing short of a pile of crap, designed to make a seasoned administrator cry.

vmbuilder cannot resolve dependencies and appears to be so much more beta than the straight-forward xen-tools I cannot begin to understand why I put so much effort into trying to make it work. I should have just stopped when it wouldn't build a 'hardy' guest. I should have stopped, when I saw the --part param would only accept four partitions - all of them turned out primary by the way. I should have stopped when I saw that --raw was just constantly ignored. I should have stopped, when it automatically added the wrong bridge device. But I tried and tried and tried - just to give virsh a go.

virsh is so overdeveloped bloat it makes me want to organise it's developers into an anonymous xml-java-addicts group. not only is it impossible to see where it's current configuration is stored - some parameters cannot even be changed without going into the virsh shell first. this poorly documented pile of shit wouldn't see the changes I made in qemu/networks/default.xml even after a reboot. that's why most tutorials work with the default settings for virbr0 - their authors just didn't know where to change this shit. Ididn't find even a single document describing where this had to be done - not one! attaching a physical drive always ended up in crashing the guest to never be started again because virsh actually messes up it's own xml config! so you need to edit the machine to make it work again. then it starts but hey - for how long?

and don't get me started on the idiotic qemu drive images! could someone explain to me why the xml file sais the image was going to be mounted as hba/ide but really it is going to be mounted as scsi sda? what sense does that make? is there a way to handle this properly via lvm? no, there isn't. because all of your lvm volumes will end up to be represented as drives, not as partitions which would make plenty more sense to me.

I'm through with this shit - I'm going back to lenny with xen - I cannot even say it's been nice while it lasted because it hasn't.

Faulbaer (fsck that shit - I'm outta here!)

( tags :: , , )
[ 2010.08.14, 11:38 :: thema: /english/anger :: link zum artikel :: 0 Comments ]
mario cart wii

25C3, aperture, apple, apps, arbeit, arbyte, bef, berlin, beta, bett, bonn, canon, ccc, chaos communications congress, debian, deutschland, diy, dns, do it yourself, drobo, em, em 2008, embedded, english, essen, fail, flagge, food, fotografie, fussball, german, hacking, internet, ipad, iphone, job, joy, kochen, koeln, konsum, kueche, kvm, london, love, mac mini, macbook air, meinblog, mobile, mobile phone, o2, palm, palm pre, photography, piraten, pre, raid, rant, ranz, saegen, schwimmen, server, smartphone, spielzeug, sport, teuer, translation, uk, updates, vertrag, virtualization, waschen, wochenende, wohnen, workout, wucher, xen, zimbra

angelos widmung johls signatur Malte Fukami nonocat by nonograffix bob
Politiker-Stopp - Diese Seite ist gesch├╝tzt vor Internet-Ausdruckern.
Erdstrahlenfreie Webseite mit Hochbürder Zertifikat