sovereign

h1. Introduction

Sovereign is a set of "Ansible":http://ansibleworks.com playbooks that you can use to build and maintain your own "personal cloud":http://www.urbandictionary.com/define.php?term=clown%20computing (I know I know). It's based entirely on open source software, so you're in control.

If you've never used Ansible before, you a) are in for a treat and b) might find these playbooks useful to learn from, since they show off a fair bit of what the tool can do.

h2. Background/Motivations

I had been a paying Google Apps customer for personal and corporate use since the service was in beta. Until several weeks ago, that is. I was about to set up another Google Apps account for a new project when I stopped to consider what I would be funding with my USD $50 per user per year:

# A "seriously questionable privacy track record":https://en.wikipedia.org/wiki/Criticism_of_Google#Privacy.
# A "dwindling commitment to open standards":https://www.eff.org/deeplinks/2013/05/google-abandons-open-standards-instant-messaging.
# A "lack of long-term commitment to products":http://www.quora.com/Google-Products/What-are-all-the-Google-products-that-have-been-shut-down.
# Development of Google+: a cynical and "unimaginative Facebook ripoff":http://gigaom.com/2012/03/15/google-plus-the-problem-isnt-design-its-a-lack-of-demand/ that's "intruding into progressively more Google products":http://bits.blogs.nytimes.com/2012/03/06/google-defending-google-plus-shares-usage-numbers/?_r=0.

To each her/his own, but personally I saw little reason to continue participating in the Google ecosystem. It had been years since I last ran my own server for email and such, but it's only gotten cheaper and easier to do so. Plus, none of the commercial alternatives I looked at provided all the services I was looking for.

Rather than writing up a long and hard-to-follow set of instructions, I decided to share my server setup in a format that you can more or less just clone, configure, and run. Ansible seemed like the most appropriate way to do that: it's simple, straightforward, and easy to pick up.

I've been using this setup for about a month now and it's been great. It's also replaced a couple of non-Google services I used, saving me money and making me feel like I've got a little more privacy.

The backbone of this was inspired by "this post by Drew Crawford":http://sealedabstract.com/code/nsa-proof-your-e-mail-in-2-hours/. Unlike him, my goal is not "NSA-proofing" my email, just providing a reasonable alternative to Google Apps that isn't wildly insecure. My view is that if the NSA or any other motivated party really wants to pwn me, they're gonna, simple as that, no matter where I host my email.

h2. Services Provided

What do you get if you point this thing at a VPS? All kinds of good stuff!

* "IMAP":https://en.wikipedia.org/wiki/Internet_Message_Access_Protocol over SSL via "Dovecot":http://dovecot.org/, complete with full text search provided by "Solr":https://lucene.apache.org/solr/.
* "SMTP":https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol over SSL via Postfix, including a nice set of "DNSBLs":https://en.wikipedia.org/wiki/DNSBL to discard spam before it ever hits your filters.
* Webmail via "Roundcube":http://www.roundcube.net/.
* Virtual domains for your email, backed by "MySQL":https://www.mysql.com/.
* Secure on-disk storage for email and more via "EncFS":http://www.arg0.net/encfs.
* Spam fighting via "DSPAM":http://dspam.sourceforge.net/ and "Postgrey":http://postgrey.schweikert.ch/.
* Mail server verification via "OpenDKIM":http://www.opendkim.org/, so folks know you're legit.
* "CalDAV":https://en.wikipedia.org/wiki/CalDAV and "CardDAV":https://en.wikipedia.org/wiki/CardDAV to keep your calendars and contacts in sync, via "ownCloud":http://owncloud.org/.
* Your own private "Dropbox":https://www.dropbox.com/, also via "ownCloud":http://owncloud.org/.
* Your own VPN server via "OpenVPN":http://openvpn.net/index.php/open-source.html.
* An IRC bouncer via "ZNC":http://wiki.znc.in/ZNC.
* "Monit":http://mmonit.com/monit/ to keep everything running smoothly (and alert you when it's not).
* Web hosting (ex: for your blog) via "Apache":https://www.apache.org/.
* Firewall management via "ferm":http://ferm.foo-projects.org/.
* Intrusion prevention via "fail2ban":http://www.fail2ban.org/ and rootkit detection via "rkhunter":http://rkhunter.sourceforge.net.
* SSH configuration preventing root login and insecure password authentication
* Nightly backups to "Tarsnap":https://www.tarsnap.com/.
* A bunch of nice-to-have tools like "mosh":http://mosh.mit.edu and "htop":http://htop.sourceforge.net that make life with a server a little easier.

No setup is perfect, but the general idea is to provide a bunch of useful services while being reasonably secure and low-maintenance. Set it up, SSH in every couple weeks, but mostly forget about it.

Don't want one or more of the above services? Comment out the relevant role in @site.yml@. Or get more granular and comment out the associated @include:@ directive in one of the playbooks.

h1. Usage

h2. What You'll Need

# A VPS (or bare-metal server if you wanna ball hard). My VPS is hosted at "Linode":http://www.linode.com/?r=45405878277aa04ee1f1d21394285da6b43f963b. You'll probably want at least 512 MB of RAM between Apache, Solr, and MySQL. Mine has 1024.
# "Debian 7":http://www.debian.org/News/2013/20130504 or an equivalent Linux distribution. (You can use whatever distro you want, but deviating from Debian will require more tweaks to the playbooks. See Ansible's different "packaging":http://www.ansibleworks.com/docs/modules.html#packaging modules.)
# A wildcard SSL certificate. I bought one. You could self-sign if you wanna save money.
# A "Tarsnap":http://www.tarsnap.com account with some credit in it. You could comment this out if you want to use a different backup service. I pay for backups at Linode in addition to the Tarsnap nightlies because you can never be too sure.

h2. Installation

h3. 1. Get a wildcard SSL certificate

Create a private key and a certificate signing request (CSR):

bc. openssl req -nodes -newkey rsa:2048 -keyout roles/common/files/wildcard_private.key -out mycert.csr

Purchase a wildcard cert from a certificate authority, such as "Positive SSL":https://positivessl.com or "AlphaSSL":https://www.alphassl.com. You will provide them with the contents of your CSR, and in return they will give you your signed public certificate. Place the certificate in @roles/common/files/wildcard_public_cert.crt@.

Download your certificate authority's combined cert to @roles/common/files/wildcard_ca.pem@. You can also download the intermediate and root certificates separately and concatenate them together in that order.

Lastly, test your certificates using the @security@ program on Mac OS X:

bc. security verify-cert -L -p ssl -s example.com -c roles/common/files/wildcard_public_cert.crt -c roles/common/files/wildcard_ca.pem
...certificate verification successful.

h3. 2. Get a Tarsnap machine key

If you haven't already, "download and install tarsnap":https://www.tarsnap.com/download.html, or use @brew install tarsnap@ if you use "Homebrew":http://brew.sh.

Create a new machine key for your server:

bc. tarsnap-keygen --keyfile roles/common/files/root_tarsnap.key --user me@example.com --machine example.com

h3. 3. Prep the server

For goodness sake, change the root password:

bc. passwd

Create a user account for Ansible to do its thing through:

bc. useradd deploy
passwd deploy
mkdir /home/deploy

Authorize your ssh key if you want passwordless ssh login (optional):

bc. mkdir /home/deploy/.ssh
chmod 700 /home/deploy/.ssh
nano /home/deploy/.ssh/authorized_keys
chmod 400 /home/deploy/.ssh/authorized_keys
chown deploy:deploy /home/deploy -R

This account should be set up for passwordless sudo. Use @visudo@ and add this line:

bc. deploy  ALL=(ALL) NOPASSWD: ALL

h3. 4. Run the ansible scripts

First, make sure you've "got Ansible installed":http://ansibleworks.com/docs/gettingstarted.html#getting-ansible.

Now, the time-consuming part: grep through the files for the string @TODO@ and replace as necessary. You'll probably want to check out all the files in the respective @vars/@ sub-directories in each playbook directory.

To run the whole dang thing:

bc. ansible-playbook -i ./hosts site.yml

To run just one or more piece, use tags. I try to tag all my includes for easy isolated development. For example, to focus in on your firewall setup:

bc. ansible-playbook -i ./hosts --tags=ferm site.yml

You might find that it fails at one point or another. This is probably because something needs to be done manually, usually because there's no good way of automating it. Fortunately, all the tasks are clearly named so you should be able to find out where it stopped. I've tried to add comments where manual intervention is necessary. OpenVPN in particular requires a bunch of manual command line stuff to get running.

h3. 5. Set up DNS

If you've just bought a new domain name, point it at "Linode's DNS Manager":https://library.linode.com/dns-manager or similar. Most VPS services (and even some domain registrars) offer a managed DNS service that you can use for this at no charge. If you're using an existing domain that's already managed elsewhere, you can probably just modify a few records.

Create an @A@ record for @example.com@ as well as @mail.example.com@ which points to your server IP. Create an @MX@ record for @example.com@ which assigns @mail.example.com@ as the domain's mail server.

Set up SPF and reverse DNS "as per the inspirational post":http://sealedabstract.com/code/nsa-proof-your-e-mail-in-2-hours/. Make sure to validate that it's all working, for example by sending an email to <a href="mailto:check-auth@verifier.port25.com">check-auth@verifier.port25.com</a> and reviewing the report that will be emailed back to you.

h3. 6. Miscellaneous Configuration

Sign in to the ZNC web interface and set things up to your liking.


h2. How I Use It

First, I moved all my email off Google with "larch":https://github.com/rgrove/larch/. It worked like a charm. Calendars and contacts were even easier: just export and then import the standard formats with your clients of choice; no issues with Calendar.app and Contacts.app.

I use this setup from my Mac like this:

* I read email in "Airmail":https://itunes.apple.com/us/app/airmail/id573171375?mt=12.
* I manage my calendar and contacts via the Apple-provided Calendar.app and Contacts.app. See "ownCloud's docs":http://doc.owncloud.com/server/5.0EE/user_manual/pim/index.html to get it set up.
* I connect to the VPS via "Viscosity":http://www.sparklabs.com/viscosity/. It has some dumb DNS bug right now so I have to point my machine to "OpenDNS":https://use.opendns.com/ in order to resolve names. Despite that, it's better than the "alternative":https://code.google.com/p/tunnelblick/.
* I connect to the IRC bouncer with "Textual":http://www.codeux.com/textual/.
* I run the "ownCloud sync client":https://owncloud.com/download for Dropbox-like file sync.
* I manage my blog and other sites with "Jekyll":http://jekyllrb.com/ locally, then push the resulting builds up to the server via "rsync":https://rsync.samba.org/ over SSH.

... and from my iPhone like this:

* I read email in the Apple-provided Mail app and check it quickly in "Triage":http://www.triage.cc/.
* I manage my calendar and contacts with the built-in apps. Boring, effective. See the "ownCloud docs":http://doc.owncloud.com/server/5.0EE/user_manual/pim/index.html for setup instructions.
* I access files stored in my ownCloud instance via "their app":https://itunes.apple.com/us/app/owncloud/id543672169?mt=8.
* I connect to my IRC bouncer with "Palaver":https://itunes.apple.com/us/app/id538073623?mt=8.

h1. Contributing

If you improve one of the provided playbooks or add an exciting new one, send a pull request. Everyone benefits.

h2. License

Original content is "GPLv3":http://gplv3.fsf.org, same as Ansible. All files and templates based on third-party software should be considered under their respective licenses.