Yearly Archives: 2018

Easy Pi-Hole and Stubby on Orange Pi Zero & Raspberry Pi 3

Skip to the install guide if you just want to know how to set up your Pi easily ;) Otherwise, read on for a little background.


I’ve been deep in DNS land of late. At work I’m working on DNS Stats and helping QA/release/document a packet capture tool for DNS stats. I even, just today, automated a complete Pi-Hole install to have a reliable dev environment for DNS stats. At the shop, I’ve set up the same encrypted DNS + Pi-Hole + LXD + Quad9 as I have at home. It’s all DNS all the time here is what I’m trying to say ;)

I’ve yet to find the magic sauce to compile Stubby on the Orange Pi Zero board though. It’s so cheap ($20 shipped), has a built in Ethernet, and is just so dang cute! So, I was looking around at stubby posts and Linux posts and Ubuntu posts and found this great write up on Ubuntu 18.04 and stubby and it said,

Stubby is in Ubuntu 18.04 repository

This was awesome! This means my previous trickery of having to compile stubby from source on Ubuntu wasn’t needed! However, the revelations about easy DNS set up and encryption were only just getting started.

The next one I found was that the 4.0 release of Pi-Hole from early August, had a new feature: custom ports can be used for upstream servers. Wham! This double awesome! Now, in the GUI of Pi-Hole, you could safely add a the IP of stubby and specify a random port to use! But we’re done yet, no sir, two more revelations to go.  Hold on.

The penultimate revelation was BOTH the Orange Pi Zero AND the Raspberry Pi 3 b had a release of Ubuntu 18.04 for them. This means that you not only don’t have to compile stubby for your x86 LXD environment, but you don’t have to do it for ARM SoC setups either! Yay!

The final revelation dates back to a long, LONG time ago, and I’m just late to the party.  I’m talking proto-internet long time ago.  The legend Jon Postel decided that not only would IPv4 have a reserved IP address of for localhost, but in RFC 790 in 1981, he said it would actually be a /8, so you get just over 16 million localhost IPs just for your bad ass self. This means you don’t actually need the new port specifying feature of Pi-Hole – you can just set up Stubby on port 53 on and Pi-Hole on port 53 on Ugh – this makes it so much easier – if I only was more a network guy!

Now that my rambling background on my recent revelations is done, let’s get to the technical write up.  Though, honestly, this part will be pretty short and sweet.

Installation Guide

This guide assumes you’ve downloaded and installed Ubuntu 18.04 for either your Orange Pi Zero or Raspberry Pi 3 B. Note that both the official download page of both Orange Pi and Raspberry Pi Foundation, do not list 18.04 options. It also assumes you’re running everything as root. The instructions are identical for both boards:

  1. Ensure you’re up to date:
    apt-get update&&apt-get upgrade
  2. Install Stubby: apt-get install stubby
  3. Edit /etc/stubby/stubby.yml so that it’s listening on
  4. Restart Stubby: systemctl restart stubby
  5. Install Pi-Hole. Use what ever upstream DNS server you want when prompted, we’re going to override it with Stubby:
    curl -sSL | bash

    – See Troubleshooting below if you get stuck on “Time until retry:” or “DNS resolution is not available” when installing Pi-Hole

Pi-Hole DNS settings
  1. Log into your new Pi-Hole at YOUR_PI_IP/admin and go to Settings -> DNS. Uncheck any DNS servers and enter a Custom 1 (IPv4) of

Coming full circle, the Reddit thread I cited in my original write up, now has a comment that Ubuntu 18.04 has a Stubby package.


If you want to use Quad9 (and I think you should ;), in step 3, while you’re in stubby.yml, comment out all the other servers in upstream_recursive_servers: and un-comment Quad9 so it looks like this::

  - address_data:
    tls_auth_name: ""
  - address_data: 2620:fe::fe
  tls_auth_name: "

Full disclosure, I work for PCH which sponsors Quad9.


A few things I found while researching this post that might help you:

  • The login on the Raspberry Pi is Ubuntu with password is Ubuntu. The login on the Orange Pi Zero is root and password is 1234. Check out my SSH Bootstrap trick as well.
  • The Orange Pi Zero didn’t get an IP via DHCP the first boot. A reboot solved that.
  • The Pi-Hole script gave me a headache when installing. Near the end of the install it said, “Starting DNS service” and then was waiting to retry. I found a post on the Pi-Hole boards that solved it perfectly. To work around this, edit /etc/init.d/pihole-FTL so that this line:

    su -s /bin/sh -c "/usr/bin/pihole-FTL" "$FTLUSER"

    is replaced by this line:


    After that, be sure to reload your init script with:

    systemctl daemon-reload

    Finally you should be able to complete your install just by restarting Pi-Hole:

    systemctl restart pihole-FTL

  • Even though I followed step 4, during one my tests stubby was still blocking port 53 on If that happens, restart stubby:

    systemctl restart stubby

  • At any point you can test that stubby or pi-hole is working. These are good to intersperse with each install and configuration change:

    dig @ +short # stubby
    dig @ +short # pi-hole

Bootstrap SSH on Ubuntu Core with out Ubuntu SSO credentials

I was playing around with Snaps and I wanted to try out Ubuntu Core as well. I still had some of those  Orange Pi Zero boards laying around and when you go to the Core Download page – there’s an option for Orange Pi right there – sweet! I downloaded the .img file, wrote it to my microsd card with dd, slapped it in my Orange Pi Zero, found the new IP in my DHCP server and off I went to SSH in.

Then I saw this step on install docs:

you will be asked to enter your Ubuntu SSO credentials

– Core Install Docs

Whhhaaat? Oh, I see, Core’s whole shtick is that it’s secure by default. They say, “Secure by default – Automatic updates ensure that critical security issues are addressed in the field, even if a device is unattended.”.  Cool, I can get behind that. IoT needs some thought leaders in IoT security. However…I still just want to SSH in and poke around a bit – I don’t want to have to set up an account at Ubuntu. 

Then I thought, “What if I just create a .ssh directory in /root/ and put my public key in the authorized keys file?” Assuming you’re on an Ubuntu system, logged in as mrjones and just stuck in your microSD card with the core image on it, that’d look like this:

cd /media/mrjones/writable/system-data/root/
mkdir .ssh
chmod 700 .ssh/
vim .ssh/authorized_keys 
chmod 700 .ssh/authorized_keys

After unmounting the card, inserting it into and rebooting my Pi, I SSHed as root and it just worked – that’s awesome! Now you know how to do it as well!  In fact, this likely will work with the Raspbian, Armbian and Ubuntu images for all kinds of Pi boards as well.

Stay tuned for my next post where I’ll massively simplify my stubby and pi-hole how to!


Teaching Kids About Busy Signals with a PBX on LXD for $15

Linksys RTP300 Circa 2005

The other day the kids and I were at our local, awesome Goodwill store. I guess they’re all awesome, eh? I was looking over the pile of small electronics while the kids picked out a shirt (“any shirt you want!!”), and stumbled across a device with Ethernet jacks (RJ45) and phone jacks (RJ11). It’s a router with two ATA ports – cool! It was only $5 bucks!! A quick internet search suggests that it may be locked to a provider like Vonage, but that there’s lots of folks unlocking it. Given this was Goodwill, there were was a plethora of POTs devices handy – two more phones at just $5/ea – no problem!

Now that I had the whole kit home, it was time to see what the ATA was up to. It’s a Linksys RTP300 – and the voice section was definitely locked down :( But that means it’s time to start hacking! I dug up the site I initially found before buying it – luckily all the downloads still work! The post is 8 years old, forever in ‘net time, so I was impressed.

I’m pretty pleased with myself because I fully understood everything and was able to improve the process because of this. Mainly:

Using inspector tool to make the ping input field longer
  • Being a web developer I already had a web server running. This is much easier to use instead of setting up a new TFTP server as they suggested.
  • Being a web developer (still) I was kinda in awe that I’d already forgotten how amazeballs Firebug was, and how I take for granted that modern browsers have such good developer tools built in.
  • Knowing all the linux commands used (wget, chmod, cd & dd) I was able to explore the device a bit
  • I figured out that the guide wanted you to download into /var/tmp which didn’t exist on my device, but /var worked
  • They have you download some *.img, but running it through strings and checking the size, it appeared to match the 3.1.24 version. Completing the install confirmed this. However, md5sum outputs don’t match.
That, sir, is an odd response to a ping ;)

After all this I had a generic, if not quite dated, ATA that was ready to talk to a VoIP provider. Time to install Asterisk! This is where LXD comes in. I already had my home server running Stubby and Pi Hole – time to add another. After spinning up an Ubuntu 16.04 box, hitting a snag, then spinning up an 18.04 box, a quick apt-get install asterisk and I was ready to go!

I created 4 extensions, one for each of us, by following some simple guides I’d found (edit /etc/asterisk/sip.conf, /etc/asterisk/extensions.conf and a reload). I plunked in the LXD Asterisk IP, extension and password and BAM! both lines registered easy peasy on the ATA.

With two hard phones set up (just plug ’em in to the ATA), I set up me and the wife using the built in client on our phones. Now we all had a phone and could call each other. See the gallery below for the visual story!

On a personal note, this was pleasing for a two reasons:

  1. I saved 3 small electronic devices from the landfill.  More and more I’m trying to be conscientious about buying less or buying used.
  2. My kids got to look up at me and say, “it’s making a weird sound!?” I explained to them that it was a “busy signal”.  See, cell phones, office phones and 99% of home phones don’t do this any more. They have either call waiting or voice mail, or both ; )

Deploying a PGP SKS server on Ubuntu 18.04


While the venerable Matt Rude has a great write up on compiling the latest SKS server for Ubuntu 18.04, using the one that ships with Ubuntu 18.04 is considerably easier. Given I just did this for work, this blog post has already been 99% written – handy!

The prerequisite for this guide is that you have root on an Ubuntu 18.04 server with a static, public IP address.


This is out of scope for this guide, but you should secure your server. I recommend only allowing SSH with public keys, no passwords. If possible, only allow SSH from your subnet, or from behind a VPN/firewall. Only open the ports to the internet for the keyserver (11371) and SSH (22) if you must. Keep your software up to date – consider turning on automatic updates for security releases.

Initial install

Given that there’s a native Ubuntu SKS package, this one command gets you a LOT. All your config files are put in place, your user is created, a cronjob is added and there’s no compiling from source. While compiling is nice for knowing exactly what code you’re runnig, it’s slower and more laborious to get set up.

Run this as root:

apt-get update
apt-get -y install sks gnupg wget apache2 xz-utils

Configure Apache

Apache functions as a reverse HTTP proxy for the SKS HTTP interface, and so binds to the SKS HTTP port (11371) only on the public IP addresses.

Create the file /etc/apache2/sites-available/100-keyserver.conf with the following contents. Be sure to update the SERVER_IP_HERE to be the real IP:

# Configuration for SKS reverse proxy #

Listen SERVER_IP_HERE:11371
<VirtualHost *:11371>
    <Proxy *>
            Order deny,allow
            Allow from all
    ProxyPass /
    ProxyPassReverse /
    ProxyVia On
    SetEnv proxy-nokeepalive 1

Run the following commands as root to set up apache with the new vhost and enable the right config. The last command should yield no errors:

a2enmod proxy
a2enmod proxy_http
a2enmod proxy_balancer
a2enmod lbmethod_byrequests
systemctl restart apache2
systemctl status apache2

Configure SKS

As everything was prepped for you with the apt-get install sks call from above, you only have to define server_contact: and hostname:in the /etc/sks/sksconf file. The contact should be your PGP key ID. Mine is 0xA105C2764BF2C4CB, for example. The hostname is the FQDN for your server.

Initialize the SKS Database

Run the following commands to download a recent dump of the SKS database, decompress it, update permissions and import it. This puts less of a load on other SKS servers as we won’t need to synchronize much when we come online.

When running, select the ”normalbuild” option. This loads all keys into the database, unlike the ”fastbuild” option, which uses the key dump files as a basis over which to operate, but loads the keys much faster. The bunzip2 and load process will take **several hours to complete**.

Run this as root:

mkdir -p /var/lib/sks/dump/
cd /var/lib/sks/dump/
wget -e robots=off -r --no-parent
mv*pgp .
cd /usr/lib/sks/
chown -R debian-sks:debian-sks /var/lib/sks
systemctl enable sks

Note: The source of the key dump for our installation is However, a key dump may be obtained from any up-to-date SKS key server, since each is a mirror of all the others. Key dumps from public key servers are listed on sks-keyserver’s wiki on GitHub. If required, any of the sources listed on this page may be used to obtain the key dump.

If you need to rebuild the database from a fresh download for some reason, be sure to fully delete the DB file with rm -rf /var/lib/sks/DB first.

Membership File

For your keyserver to be known by others and have it’s keys be up to date via synchronizatoin, you need to contact the keyserver operators so they they list you in their membership file. That way when you list them, the servers will not not error out.

The format of /var/lib/sks/membership is simply KEYSERVER_URL PORT

After you create it, ensure it can be read by SKS:

chown -R debian-sks:debian-sks /etc/sks/memberships

Start and Test

Start your server with systemctl start sks. That’s it! You’re done.

Monitor the log files for any problems during startup and testing: tail -f /var/log/syslog. As well, check the output of systemctl status sks.

To test, open a browser window to http://YOUR_SERVER:11371. Search for a few different keys to verify that key information is being retrieved correctly.

Check the SKS stats page to verify the number of keys loaded: http://YOUR_SERVER:11371/pks/lookup?op=stats

If you have peered with other servers, verify that it is showing up properly in the pool.


Support the EFF and others in the face of the Tax Cuts and Jobs Act

I’ll keep this short and sweet: When I read the headline, “EFF sues to kill FOSTA, calling it ‘unconstitutional Internet censorship law'”, I realized how proud I am to support the Electronic Frontier Foundation (EFF) in the form of annual donations. They are an awesome group doing amazing things. Further, my financial support will not wane now that the Tax Cuts and Jobs Act has been passed.

With the direction the current administration is going with human rights and health care, and with the Supreme Court switching up a justice, now, more than ever, is the time to support the causes you believe in. Do you what you can!


SYN Shop Class: SSH Keys with free VMs for members

As part of putting some good hardware to use, I taught a class on how to use SSH Keys the other week. As this was the first time I taught this class, it took a good long while to do the prep for it. I figured it’d take 6 hours (3 sessions @ 2hr), but I think I ended up putting in closer to 15 hours. There’s still room for improvement!

The slides are available (PDF), as is the video (below). I hope to redo this class so that it’s a generic “learn about SSH Keys” which then could be open to the general public. That said, I’m really glad to see we’re up to 16 containers and the load average is only at 0.37 right now ;)

PS – I know, I know, those are not VMs, they’re actually containers!


USB-C accessories I like (and some I don’t)

After buying USB-C power adapter that didn’t work out, I’d been on the hunt for a replacement. After getting a Chromebook for video chats and music (and curiosity :) which has USB-C charging, I had an extra need for cheap, portable chargers and other simple USB-C accessories.

All these devices were tested on either my ASUS Chromebook Flip C101PA-DB02 or my Dell XPS 13 9350 or both.

What I really wanted was to plug a single cord into my laptop and have it work with my external webcam, external speakers, external 4k monitor @60Hz, external wireless headset and USB3 gigabit NIC. In the end, for stability, I ended up going with two cables: one for power and DisplayPort and one USB3.0 for my USB3.0 hub with everything else. My laptop not crashing makes me be totally OK with the two cables ;)

First up is a good, small travel charger with USB-C and USBA ports. It delivers the full 45W so it’ll work nicely with your MacBook and Dell XPS 13. It’ll work with your MacBook Pro, but not at the full rate as the stock charger. It’s the Yojock YKJ USB-C PD Charger with 45W:

After that, if you want a charger that’ll work at home (read: no folding plug), then this charger with attached cable is quite small and cheap. It’s the Nekteck 45W USB-C charger. It only has USB-C and, beware, my XPS 13 9350 says it’s only delivering 40W, but I’ve had no problems with it keeping my laptop charged even when it’s powering an external 4k monitor, running 4 containers, a full IDE and twenty odd tabs in FireFox.

Then, in the non-power-adapter accessories:

These items I don’t recommend as they didn’t work as advertised with my ubuntu laptop;

To be clear, I am gathering the referral kickbacks from Amazon on these links, but I do use these products and wasn’t paid for the positive review.

If you have questions, feel free to contact me or ask in the comments!


Hardware disabling the mic on WyzeCam v2

I’m testing out using two of these cute, cheap security cameras:

It’s the v2 of the WyzeCam and it’s only $25 shipped.

While they’re easy to set up, there doesn’t seem to be a way to turn off the mics on them. Here’s the config screen for a camera on the android app as of version 1.2.76 on Apr 26, 2018:

So what’s a hacker to do? Open it up and remove the mic, of course ;)

2019/01/10 Update – At this point in the post, I should let you know that we’re about to permanently remove your camera microphone. As well, there’s a chance you can rip off the antenna cable, effectively bricking your camera.  Only one reader has reported any problems, but as with all DIY modifications, proceed at your own risk and happy hacking!

2018/05/08 Update! – If you just yank off the mic there’s a LOT less disassembly. I had two more cams come through, so I made a video of the whole thing. 10 minutes or less! Otherwise, read on below for old, solder method.

Old Solder method – I was too impatient to take all the step by step photos, but the only tricky part was getting the two plastic plates off after the initial two outer screws. My advise: it takes more force than you think!

You have to fully disassemble the camera to be able to solder out the mic. There were 5 or 6 screws, two ribbon cables and an antenna, speaker and camera power lead that all needed to be taken apart.

Here’s a before and after picture:

After re-assembling both cameras everything still works – w00t! Here’s to hardwired audio privacy, as it should be.


Stubby + Pi-Hole + Quad9 + LXD


For some time now, really since last November, I’ve wanted to do two things: Encrypt all my DNS traffic leaving my house LAN and run an instance of Pi-Hole to reduce ads spamming my browser (and running cryptocurrency mining software ;). As well, even if you’re connecting to a web server over HTTPS, your DNS lookups are still in the clear free to monitor, monetize or possible mangle. I thought I wanted to do this on a single, small single board computer. I even bought a few of these sweet Orange Pi Zero boards to run it on!

However, in the end I realized I already had a dual core/quad thread i3 NUC running my ownCloud instance, and I could leverage that to run some of my favorite software: LXD! See my previous write up on setting up LXD so that your containers get an IP accessible on your LAN.


Once you have LXD set up, then we can create two containers with two static IPs on your LAN, one to run the awesome ad-blocking Pi-Hole and anther to run Stubby which we’ll use to run an encrypted DNS tunnel to Quad9. Quad9 is the semi-new DNS service that does not log, is very fast and is anycasted to over 100 locations world wide. Full disclosure, I work for PCH which sponsors Quad9.

To make provisioning Ubuntu instances easier in LXD, I have this bash script which I’ve cobbled together from various sources:

# Create LXD container
if [ $# -eq 0 ]
echo " myvm member"
echo "    or    "
echo " myvm default"
###### VARS
version: 1
- type: physical
name: eth0
- type: static
ipv4: true
address: $IP_4
gateway: $GATEWAY
control: auto
- type: nameserver
lxc launch ubuntu: $NAME -c"$NC" -c user.user-data="$UC" -p $PROFILE

My LAN has a subnet of, so I used these two calls to make my two LXD instances: pihole default stubby default

Now my (only 2 (for now ;)) LXD containers are running and ready with static IPs:

# lxc list -c n,s,4
|  NAME  |  STATE  |         IPV4         |
| pihole | RUNNING | (eth0) |
| stubby | RUNNING | (eth0) |


The Pi-Hole folks have made the install awesomely easy. You just run this one liner after connecting to your Pi-Hole container:

curl -sSL | bash

Of course, you shouldn’t just trust random commands you copy and paste from the Internet, so you can git clone from their repo as well. During the install, you can choose what ever DNS provider you want as we’re going to change it. Otherwise, accept all the defaults. Do notice the admin password at the end – you’ll need this!

After the installer is done, you should be able to to your Pi-Hole IP to log into the admin web GUI. For me this is at After you’re logged in, go to the DNS settings page. For me this is

Once here, change “Interface listening behavior” to “Listen on all interfaces, permit all origins”. As well, change “Upstream DNS Servers” to be only the IP of your stubby container. For me this is

Now let’s go set up stubby at!


I’ve found a number of guides that show you how to install both Pi-Hole and Stubby on the same box, often a Raspberry Pi. While convenient to not have more hardware on your network, they also have to hack Pi-Hole to see Stubby. The reason is that Pi-Hole can’t talk to anything but port 53 on which ever IP your specify. The hack is then to get dnsmasq that Pi-Hole uses to talk directly to Stubby. The Pi-Hole GUI cannot see this change and it may get overwritten on Pi-Hole upgrades. The fact that I could have 10 or 20 containers and I’d still run the same amount of hardware coupled with the fact that the config (see Pi-Hole section above) is all done cleanly and upgrade safely in the GUI, made me use two discrete containers.

This Stubby install guide below is 99.99% not mine, so please thank redditor SphericalRedundancy and his comment about setting up Stubby on Ubuntu. Thanks SphericalRedundancy! The changes I made is to move another comment about running ldconfig as a key component of the install and remove Pi-Hole specific changes. Oh, the other change is that I want to run Stubby on port 53 which is a privileged port, so I run Stubby as root instead of creating a new user (stubby).

Install build & run time dependencies

sudo apt install -y build-essential libssl-dev libtool m4 autoconf
sudo apt install -y libev4 libyaml-dev libidn11 libuv1 libevent-core-2.0.5 

Build Stubby

git clone
cd getdns
git checkout develop
git submodule update --init
libtoolize -ci
autoreconf -fi
mkdir build
cd build
../configure --prefix=/usr/local --without-libidn --without-libidn2 --enable-stub-only --with-ssl --with-stubby
sudo make install 

Configure stubby.yml

cd ../stubby
cp stubby.yml.example stubby.yml
sed -i.bak '/  -,/  -  0::1/{/  -  0::1/ s/.*/  -\
  -  0::2@2053/; t; d}' stubby.yml
sudo /usr/bin/install -Dm644 stubby.yml /etc/stubby.yml 

Configure stubby.service

cd systemd
echo ' ' > ./stubby.service
sed -i '$i [Unit]' ./stubby.service
sed -i '$i Description=stubby DNS resolver' ./stubby.service
sed -i '$i' ./stubby.service
sed -i '$i' ./stubby.service
sed -i '$i [Service]' ./stubby.service
sed -i '$i ExecStart=/usr/local/bin/stubby -C /etc/stubby.yml' ./stubby.service
sed -i '$i Restart=on-abort' ./stubby.service
sed -i '$i User=root' ./stubby.service
sed -i '$i [Install]' ./stubby.service
sed -i '$i' ./stubby.service 

Install stubby service

sudo /usr/bin/install -Dm644 stubby.conf /usr/lib/tmpfiles.d/stubby.conf
sudo /usr/bin/install -Dm644 stubby.service /lib/systemd/system/stubby.service 

Edit host file

sudo sed -i '/' /etc/hosts
sudo sed -i '/0::2/d' /etc/hosts
sudo sed -i '$i     Stubby' /etc/hosts
sudo sed -i '$i 0::2          Stubby-v6' /etc/hosts 

Add path to libgetdns library & running ldconfig

sudo sed -i '$i LD_LIBRARY_PATH=/lib:/usr/lib:/usr/local/lib' /etc/environment
sudo /sbin/ldconfig -v

Enable and run stubby service

sudo systemctl enable stubby
sudo systemctl start stubby 

Clean up

cd ../../../
rm -rf ./getdns/ 


Before we configure Stubby to use Quad9’s DNS over TLS service, let’s test everything to make sure it’s working. From the Stubby container, let’s make sure Stubby is working by running sudo systemctl status stubby. My output looks like this:

● stubby.service - stubby DNS resolver
   Loaded: loaded (/lib/systemd/system/stubby.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2018-04-20 17:32:27 UTC; 2 days ago
 Main PID: 14731 (stubby)
    Tasks: 1
   Memory: 2.8M
      CPU: 13.951s
   CGroup: /system.slice/stubby.service
           └─14731 /usr/local/bin/stubby -C /etc/stubby.yml

Apr 20 17:32:27 stubby systemd[1]: Started stubby DNS resolver.
Apr 20 17:32:27 stubby stubby[14731]: [17:32:27.969574] STUBBY: Read config from file /etc/stubby.yml
Apr 20 17:32:27 stubby stubby[14731]: [17:32:27.970478] STUBBY: DNSSEC Validation is OFF
Apr 20 17:32:27 stubby stubby[14731]: [17:32:27.970498] STUBBY: Transport list is:
Apr 20 17:32:27 stubby stubby[14731]: [17:32:27.970506] STUBBY:   - TLS
Apr 20 17:32:27 stubby stubby[14731]: [17:32:27.970515] STUBBY: Privacy Usage Profile is Strict (Authentication required)
Apr 20 17:32:27 stubby stubby[14731]: [17:32:27.970522] STUBBY: (NOTE a Strict Profile only applies when TLS is the ONLY 
Apr 20 17:32:27 stubby stubby[14731]: [17:32:27.970529] STUBBY: Starting DAEMON....

All good! Now let’s ensure DNS lookups are working using the default DNS servers with a dig call, again while logged into your Stubby instance:

dig @ -p 2053 +short

Sweet! So if we use port 2053 on the localhost IP of, Stubby is listening and relaying the DNS lookups. All good.

Now edit /etc/stubby.yml so this section:

  -  0::2@2053

Looks like this:

  -  0::2@2053

Again, swap out the 192 IP with what ever IP you’re using for your Stubby instance. Now find *all* the sections after upstream_recursive_servers: in that same file and ensure that only Quad9 is there:

  - address_data:
    tls_auth_name: ""

Removing all comments and empty lines, my complete stubby.yml file looks like this (thanks to this handy grep call: egrep -v '#' /etc/stubby.yml |grep -v -e '^$'):

tls_query_padding_blocksize: 128
edns_client_subnet_private : 1
idle_timeout: 10000
  -  0::2@2053
round_robin_upstreams: 1
  - address_data:
    tls_auth_name: ""

Now let’s restart Stubby and make sure it still works with another dig command (again, use your Stubby IP where the 192 IP is), but this time we don’t specify a port so the default 53 is used:

sudo systemctl restart stubby 
dig @ +short

All good! Finally, we already configured the Pi-Hole instance to use .21 (but you use your IP!) so we can test the full round trip by running a dig call against it which will, int turn, use the Stubby proxy:

sudo systemctl restart stubby 
dig @ +short

The final step is to configure your router to hand out as the DNS server. Each router is different, but for me, I run pfSense, so I changed it there under “ServicesDHCP -> ServerLAN” and then on the page under “DNS servers” I entered just my Pi-Hole IP (you enter yours though, not mine!) of

Now you can enjoy ad blocked, unlogged, encrypted DNS on your home LAN – w00t!



Be sure to read my follow up post about trekking in Nepal after the conference!


This is post closely mirrors my trip 2 years ago to attend SANOG 27: a conference followed by some awesome tourism. Like before I was luck to have my work send me to APRICOT 2018 and was again humbled to have my talk selected for presenting.

APRICOT is the “Asia Pacific Regional Internet Conference on Operational Technologies”. It’s more or less a larger SANOG meeting encompassing a greater regional area so more folks come. However, a lot of the usual suspects were in attendance (if I can get away saying that having only been to two conferences in Kathmandu ;).

Talks and Workshops

Note: All links below have full YouTube streams to watch and decks to download. Be sure to click through if you’re interested in knowing more about any of them!

I attended a slew of talks, but some of highlights for me were:

  • Daniel Griggs of NZRS gave a fun talk, “Accidental Data Analytics” that was eerily similar to mine and presented immediately before me.
  • Yasunari Momoi and Kota Kanbe’s talk “Vuls & VulsRepo” was…just pure awesome. Especially Kota’s half of the talk. He’s inspired me to do a pull request for how to deploy Vuls on Ubuntu server.
  • I missed the first half, but I very much enjoyed Dima Bekerman’s workshop “Brace Yourselves: DDoS is Coming“.
  • My own co-worker Gaël Hernandez’s talk was of course great to see, “Building and operating a global DNS content delivery anycast network“. It’s always a fun twist to see a presentation intended for outsiders on something that you’re so deeply inside of.
  • While it was plagued with some unfortunate overwhelming of the Hypervisor used for the workshop, Nick Hilliard’s, “IXP Manager Tutorial – Part 2” was a good introduction to introduction to IXP Manager, which I only knew about in concept, not practice.
  • Artyom Gavrichenkov’s talk, “DNS Survival Guide” was a fun history of DNS including up to date comparison of which resolvers are fastest (BIND isn’t even on the list ;). As well, his lightening talk on Feb 28th titled “Memcached amplification DDoS: a 2018 threat” was ridiculously prescient: literally the very next day the largest recorded DDoS hit the ‘net. This was a 1.1Tbps (yes, capital “T” for “Terabit”) attack hit GitHub on March 1st using the Mecached amplification.


As I mentioned earlier, I was lucky enough to be selected to speak at APRICOT this year. Unlike my security based talk last trip, I was speaking about DNS and Stats this time. Specifically, my talk was title, “Visualizing a global DNS network with open source tools“. While the final, customer facing product I’m talking about (ccTLD DNS Stats) isn’t live yet (it’s in internal Beta), all the software I talk about is ready for prime time. Well…DNSAuth could use some love, but the other three are prime time ready, for sure ;) Those are:

  • DNSAuth – PCH Go App for cooking pcaps to TSDB faster
  • MapTable – PCH JS app for rendering SVG maps and tables
  • C3.js – Client side, real time charts
  • InfluxDB – TSDB to archive data

Networking and co-workers

It was a real treat to go to Nepal to attend APRICOT. Like all good conferences, you meet a lot of people that you can network with. I had a lot of fun conversations! Best thing of all about going to APRICOT was that I got to chat with all my co-workers, including two I’d never met before who came all the way from Dublin and Johannesburg. Awesome!