Yearly Archives: 2022

Keys-To-The-Tunnel 1.1.0 released

3 minutes, 14 seconds

Hey hey! I’d been meaning to consolidate some of my VMs I’m paying for and semi-accidentally deleted the place where I was hosting my work’s instance of Keys-To-The-Tunnel (KTTT). With this VM deletion, I decided to leverage my recent dabblings with Caddy and do a big refactor of KTTT by replacing Apache with Caddy.

As a reminder, KTTT is an easy way for an organization which both develops web apps and uses GitHub to share their local dev app on the internet via a publicly accessible URL all protected with solid TLS and SSH encryption.

Let’s dive into what the update means and why I did it!

Parenthetical aside about TLS for local Android development

Originally I asked if potential KTTT users :

ever need to test an Android application against a web server such that you need a valid TLS certificate?

Not so good idea on my original KTTT post

This is actually a bad idea if all you need is a valid TLS cert for Android testing. The reason is that you’ll literally be holding your phone in your hand which is less than a foot from the computer hosting your app you want to test against. By introducing KTTT into the mix, you send your traffic thousands of miles/kilometers (you pick which) and back just to get a TLS cert. Crazy times!

A much better approach is to use something like local-ip.co . If you want an easy way to keep your traffic entirely local, you can use nginx-local-ip with a one line docker compose call to set up everything you need to locally run a local-ip.co TLS. It’s a really sweet set up! (I’m biased because I help author some of it ;)

KTTT is still a great idea if you need to share your local dev environment though!

With that out of the way, back to KTTT updates…

What’s new with KTTT

For an end user of KTTT who just wants to share their app, it’s now WAY easier to figure out which SSH command to run and which URL to share. This is thanks to the handy web app which walks you through three easy questions:

  1. What is your GitHub Username?
  2. What port is your app running on locally?
  3. Is your local app using http or https?

This is MUCH better than wading through a list of random ports and other GitHub usernames which you only cared about your username and port. Here’s what the updated web app looks like in action (15 second video):

For the administrators of KTTT, you’ll note that KTTT now uses Caddy instead of Apache. While there’s nothing wrong with Apache, Caddy is a simpler take on the needs of an app like KTTT that requires a bunch of small reverse proxies. Caddy is 7 years old and came to being in the world of Docker, containers and micro-services. Where as Apache is 27 years old and came being near the birth of world wide web.

Ironically, a key feature of Caddy, the ability to automatically provision and renew TLS certs, is NOT being used. Instead, the opportunity to use a wildcard TLS cert came up via acme-dns.io and I took it.

That all said, it’s a joy to use Caddy because I can create a simple JSON file with four lines to define a reverse proxy:

mrjones-plip.awesome-tunnel.plip.com {
   tls /etc/certs/fullchain.pem /etc/certs/privkey.pem
   reverse_proxy 127.0.0.1:3089
}

Love it!

Putting it all together

The web app above gives a pretty good idea of the improvements, but since I added a demo video of the whole KTTT experience on GitHub, may as well post it here in case you’re curious (44 second video):

Other odds and ends in 1.1.0

There’s also a bunch of other fixes and improvments I made while in there. Here’s the notes from the 1.1.0 release:

  • Replace Apache with Caddy
  • Add mini web app to help devs figure which URL and SSH command to use
  • Unify mutli SNI certs to one wildcard with acme-dns.io (still using Let’s Encrypt though) per #1
  • Don’t overwrite existing user’s ports every time you run setup per #3
  • Don’t regenerate TLS certs every time you run setup per #3
  • Don’t rewrite vhosts in web server every time you run setup per #3
  • Update MOTD on login per #4
  • Try my hand at being an artist and create a KTTT logo

My first from-scratch 3D printed design (that is actually useful)

2 minutes, 24 seconds

I’ve had a 3d printer for a about 8 months now. It’s been fun to download STLs from the internet and print them. We’ve done articulated snakes and planetary gear prints. It is really amazing to print something in one go that you can move and spin around. Amazeballs!

The kids have printed up little castles and and little knickknacks the designed in TinkerCAD. This has been fun, but I was always bothered that I didn’t know a “real” design program that was also open source. Really that left FreeCAD. I did a tutorial or two on-line which were pretty great. I understood “parametric” and “fully constrained” finally \o/. But, it was time consuming to learn, and I was impatient. I printed up a laptop holder I found online. Time went on.

Recently I got some new mini servers (I’ll save the setup and install story for another blog post) and I wanted to mount them to the wall. I knew this would be pretty trivial to design a custom bracket if I’d learned FreeCAD well enough, but I still hadn’t.

I just decided to go for it, use TinkerCAD even though it was proprietary and online. Not ideal, but I got what I wanted! I made a two left and right brackets that held two mini servers against a plywood board. Perfect!

The point of this blog post is to let other folks know that they should just go for it – don’t let the tools slow you down. Once you see the power of creating something from nothing, and a physical something (not just software), you’ll see how empowering and inspiring it is!

I will say, you won’t get it right on your first try. I already knew this from some earlier prototyping I’d done, so I was ready for the 4 or 5 prints that failed before the final one succeeded. Don’t get flustered when, even after measuring twice before you cut print, that it’s still wrong!

So, here’s what I made:

What you see here are two “L” type brackets that each can hold the corners of two mini servers. Note the hole through the cross brace that allows you to screw in the top screw (the first draft totally lacked this and it was nigh impossible to mount). These are of course rotated up in how you’d mount them, and in a terrible position to print.

While these held the servers pretty well in place, there was still a chance they could tip forward off the wall, so I made a toolless top bracket to match. The nice thing is that you can screw this in as one piece, and the slide off the bracket from the screw mount. Some very light sanding was needed to ensure the two pieces slide together with just the right of amount of friction (showing two of them side by side, but only used one):

And here they are in situ:

If you have little servers and you want the STL, here’s the file and here’s the TinkerCAD link (which may go away b/c I’m not a fan of cloud services).

I hope if you’re thinking about designing your first part from scratch, you go for it!!

Legba the Net-tracker

4 minutes, 15 seconds

Intro

I’d been meaning to learn how to write an app using something more than CSV files, but less than MariaDB, to store files – I’m thinking SQLite of course! Then along came the desire to have a simple way to track when a computer was on a network as a proxy for kids’ daily screen time. After all, the network is the computer, right?

While there’s so very many ways to solve detecting if a computer is online (more on this later), I thought it’d be fun to write a simple app that could correlate multiple IPs to a single person, and then give a histogram of minutes per day per person. Given this is just a proxy for screen time, it’s fine if it doesn’t have alerting, password protection or even a way to prevent going over the allotted time per day. The goal will be for any interested parties to see how long a device has been on for the current day. It’s then up to the family to have a discussion about what it means to go over your daily allotment.

Ok, let’s do this! We have a requirement to track computers being online and to write and read the results to a SQLite DB. I’ve been groovin’ on learning Python, so let’s double down and use that. I did some Wikipedia exploring and read about Papa Legba, and thought it made a mighty fine sounding name. Finally, after some nudging from a friend, we’ll package it up in Docker so it’s easy to try out and host in an isolated container.

Ping FTW

The first step to using Legba, is to define a list of users and which IPs they’ll be on. Very likely the best way to do this is to either use static IPs on your LAN clients, or have your DHCP server set the same IPs per MAC every time.

Then you’ll create a conf.py file copied from the conf.example.py file and fill it out. Here we see Jon and Habib have one IP each, where as Mohamed has 2:

trackme = {
    'Jon': ["192.168.1.82"],
    'Habib': ["192.168.1.12"],
    'Mohamed, ': ["192.168.1.240", "192.168.1.17"]
}

The code to track if a device is achieved via the subprocess module via a ping() function with just two lines that send a single ICMP packet:

# thanks https://stackoverflow.com/a/10402323
def ping(host):
    """ Ping a host on the network. Returns boolean """
    command = ["ping", "-c", "1", "-w1", host]
    return subprocess.run(args=command, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL).returncode == 0

Back in the main() function, we then read in config, loop over each person and try and ping() each of their IPs. If we see them online, we write to the DB via record(). It ended up, just as I’d hoped, that Python’s SQLite libraries are robust and it’s just 6 lines to insert a row:

sql = ''' INSERT INTO status(name,state,date) VALUES(?,?,?) '''
cur = sqlite.cursor()
activity = (name, state, datetime.now())
cur.execute(sql, activity)
sqlite.commit()

return cur.lastrowid

Just before the end of the loop we call probably the most complex function of the lot output_stats_html(). This function is responsible for reading the day’s active users, getting each users activity by hour, the total for the day and finally output static HTML as well as a static JSON file that will get called via AJAX so the stats will auto-refresh.

At the end of the loop we sleep for 60 seconds. In theory if you had hundreds (thousands?!) of IPs to track and they were on connections with >500ms latency, it would take way longer than 60 seconds. Legba will not scale to this level. It’s currently been comfortably tested with 5-10 devices on a LAN where each device has ~20ms of latency.

A histogram is worth a 1000 words

After you’ve done a bit of a git clone with a lil pip3 install and fleshed out your own config.py and done a little systemd love, you’ll have some sweet sweet histograms! (Some keen eyed readers may note this histogram looks familiar ;)

It’s interesting to note that mobile devices, as seen withe “Adnon Cell”, are effectively on all the time. In this sense, Legba is not much use to track a cell phone. Meanwhile, Bobby Table’s desktop, Adnon’s Laptop and Chang’s Nintendo Switch all work as expected (NB – I didn’t actually test with a Switch).

Existing Solutions

I’ve been running this for solution for just about 4 months now. It’s been a great way for our family to have an open discussion about what it means to spend too much time on the computer and it’s been rock solid. Checking ls and select count(*) from status; I see my DB is 23MB and has 487,069 rows.

Given the simplicity of this app, could this DB and rows be easily stored and retrieved elsewhere? When I wrote the app, I didn’t care – I just wanted to write it for the fun of writing it! However, I was listening to episode 171 of Late Night Linux and they mentioned how utilitarian Telegraf is. It struck me that, indeed, if you had a Telegraf, InfluxDB and Grafana (aka “the TIG stack”) already set up, it would be pretty trivial to capture these same stats. I would do this by setting up a centralized instance of Telegraf and either use the built in Ping plugin, or possibly the more extensible [[inputs.exec]] input type. With the latter, you could even re-use parts of Legba to pretty trivially input the data to InfluxDB. Then, it would be equally trivial, to slice up the ping counts per hour, per user and have a slick dashboard. Just food for thought!

Otherwise, I hope some else than me gives Legba a try!