Category Archives: PHP

Migrating FreshRSS from Ubuntu to Alpine Linux

Leave a reply

5 minutes, 22 seconds

Intro

For some time I’ve had all my home lab systems running on LXD. For me at least, LXD predated Docker and so I’ve stuck with it. The containers are a bit more pet like and less cattle like – but that’s OK. They’re there for me to learn with – plus I can totes do docker in LXD inception if I wanna! (I’ll figure my way over to Incus some other month)

So years and years ago I found out bout FreshRSS as a great way to self host an RSS reader. This meant I didn’t have to install any apps on my phone, my feeds could stay synced and since I already had a VPN setup, it was trivial to access while on the road. Lovin’ it! I’d set this up around late 2018.

Fast forward to yesterday, I flippantly upgraded FresshRSS to the latest, as I occasionally do, and the site started having a fatal error, as it often does after I upgrade ;) I pulled up the error logs on the FressRSS LXD container running Apache and MariaDB and immediately saw some sort of unexpected { character in function blah blah on line 1-oh-smang-thirty error. Huh, what’s that about?

Turns out in the latest dev release of FreshRSS, they’d formally removed support for PHP 7.x and required PHP >8.0. Specifically, this was because they’re using Union Types in their function declarations. This is cool stuff! you can see the int|string values in the sample function here:

// Declaring a function with Union Type
function getMixedValue(int|string $value): int|string {
   return $value;
}

This is no problem! I’ll just update PH….P…. oh yeah – I’m on hella old Ubuntu 18…so then I’ll just find some third party apt repo and add that and then…. Hrrmm…might be more trouble than it’s worth. That’s cool! I’ll just deploy a new Ubuntu container on 24.04. Oh, well, that’s gonna take a chunk of disk space – I’ve been using a bit of Alpine to build small Docker images at work, what about a pet Alpine in LXD? They have an image – why not!

Preparing the leave the old system

Before we sudo rm -rf / on the old box (er, container), let’s get our data outa there. We need to first make a dump of the database. We’re root so we can just zing right through any permissions with a one liner. Next up we can zip up our old files into one big ol’ honkin zip file. That looks like this:

mysqldump freshrss > ~/freshrss.sql
cd /var/www/localhost/htdocs/
zip -r ~/freshrss.zip .

Finally, we can generate an SSH key for this root user to easily copy to the new container – I knowingly didn’t add a password because I’m about to delete the container and we’re all friends here:

ssh-keygen -t ed25519 
cat /root/.ssh/id_ed25519.pub

Ok – I’ll hold on to that pub key for the “One last trip to the old digs” section below.

Bless the new hotness

Here’s how to bloop out an Alpine container in LXD:

lxc launch images:alpine/3.20 freshrss
lxc shell freshrss

That’s it! You’re now sitting as root on the new instance. Let’s install the base packages including Apache, MariaDB, OpenSSH and PHP with all it’s libraries:

apk add \
   mariadb mariadb-client openssh \
   apache2 apache2-http2 php83 \
   php83-cli php83-apache2 php83-session php83-curl \
   php83-gmp php83-intl php83-mbstring php83-sqlite3 \
   php83-xml php83-zip php83-ctype php83-fileinfo \
   php83-dom php83-pdo

Now let’s ensure the three services start at boot and then we can start Apache and OpenSSH (MariaDB will have to wait):

rc-update add apache2
rc-update add sshd
rc-update add mariadb
rc-service apache2 start
rc-service sshd start

As well, that pub key you got from the old server? Let’s add that in on the new server:

mkdir ~/.ssh
echo "ssh-ed25519 AAAAC3Nz-SNIP-NplQ3 root@freshrss-old" > ~/.ssh/authorized_keys
chmod 700 ~/.ssh/
chmod 600 ~/.ssh/authorized_keys

One last trip to the old digs

As final hurrah at the old server, now that our SSH key is on the new server, let’s copy over the zip archive and the SQL dump. Be sure to replace 192.168.68.217 with your real IP!

scp ~/freshrss.* 192.168.68.217:

Go SQL!

Now that we have our server with all the software installed and all the data copied over, we just need to pull together all the correct configs. First, let’s run setup for MariaDB and then harden it. Note that it’s called mysql… , but that’s just for backwards compatibility:

rc-service mariadb setup
rc-service mariadb start
mysql_secure_installation

That last command will ask questions – default answers are all good! And the initial password is empty, so you can just hit return when prompted for the current password. Maybe check out passphraseme if you need a password generation tool? Let’s add the database, user and perms now. Be sure to not use password as your password though!

echo "CREATE USER 'freshrss'@'localhost' IDENTIFIED BY 'password';" | mysql
echo "GRANT ALL PRIVILEGES ON *.* TO 'freshrss'@'localhost' WITH GRANT OPTIO" | mysql
echo "GRANT ALL PRIVILEGES ON *.* TO 'freshrss'@'localhost' WITH GRANT OPTION;" | mysql

Now we can load up the SQL and move all the PHP files to their correct home. Again, we’re root so no SQL password, and again, this is actually MariaDB, not MySQL: 

mysql  freshrss < freshrss.sql 
unzip freshrss.zip 
mv * /var/www/localhost/htdocs/.
mv .* /var/www/localhost/htdocs/.

Go Apache!

Apache just needs four updates – easy!

Edit /etc/apache2/httpd.conf with your favorite editor. Find these three lines and uncomment them – they won’t be next to each other:

#LoadModule rewrite_module modules/mod_rewrite.so
#LoadModule session_module modules/mod_session.so
#LoadModule remoteip_module modules/mod_remoteip.so

Now find the one line where DocumentRoot is set and change it to this value and add two more lines. One to allow encoded slashes and one to set the server name. Be sure to use the IP address or FQDN of the server – don’t use rss.plip.com! 

DocumentRoot "/var/www/localhost/htdocs/p"
AllowEncodedSlashes On
ServerName rss.plip.com:80

Now that apache has been configured, let’s restart it so all the settings are loaded:

rc-service apache2 restart

Conclusion

The old FreshRSS install should now be running on your new Alpine based container – congrats! This has been a fun adventure to appreciate how Alpine works as compared to Ubuntu. This really came down to two main differences:

  • systemd vs OpenRC – Ubuntu has used systemd for sometime now and the primary interface to that is systemctl. Alpine on the other hand uses OpenRC which you interface with rc-update and rc-service. Alpine picked this up from when it split off from Gentoo.
  • apt vs apk – Package management is slightly different! I found this to be an inconsequential change.

There’s plenty of guides out there that do the same as this one. Heck, you’re likely better of just using a pre-built docker image (though the top results were pinned to PHP7)! However, I wanted to document this for myself and hopefully I’ll save someone a bunch of little trips off to this wiki or that FAQ to understand how to migrate off of Ubuntu to Alpine.

Cheers!

Happy Hacker Halloween!

1 Reply

2 minutes, 12 seconds

Last year I wanted to be a “Hacker” and code up a solution to show near by access points and nearby phones. I failed. However, I did a good job of brushing up on what I needed to do over the past year and so this year I was a hacker for reals. Here I am in the final get up:

hacker.jones.jpg

Let’s break it down! Here’s the hardware list (affiliate links to Amazon):

My final build out looked like this:

IMG_20191031_160240.jpg

A quick write up of the software is:

  1. install latest Rasbpian on your MicrSD card
  2. Install latest YANPIWS in /var/www/html/YANPIWS
  3. Install howmanypeoplearearound as the pi user. Ensure it’s path is /home/pi/.local/bin/howmanypeoplearearound
  4. Ensure you have all the libs for YANPIWS python scripts installed so you can talk to the BME280
  5. Hook up the BME280 to the right 4 pins on the Pi using the jumper cables
  6. Hook up the monitor to the Pi’s HDMI and the External WiFi adapter a USB port
  7. Have the Pi boot into kiosk mode with a browser that points to http://127.0.0.1 by following this awesome guide on pimylife.com. Note that you’ll only use the one URL and have no while loop in the kiosk bash script.
  8. Install Apache and PHP with sudo apt install apache2 php
  9. In /var/www/html/ put all of the files I just published on this gist. Basically it’s a small web app to show the data we’re collecting as well as some bash scripts that get run in cron.
  10. Install a bunch of cron jobs that gather the data as the pi user. This will use wlan0 (built in) to look for nearby access points using the venerable iw command. It will use wlan1 (USB adapter) to look for phones and such in monitor mode using howmanypeoplearearound. Finally, it will get the temp and humidity using the python script from YANPIWS. You may need to make /var/www/html writable by pi user to make this work.

It’s not my finest code, but if everything worked correctly, the Pi will boot up every time and show something like this:

IMG_20191031_190702.jpg

As you can see it got cold tonight on our walk – by the time we got home at 8pm it was 45. Happy Hacker Halloween!

tableMaker: Open Source PHP MySQL CRUD GUI library (Updated!)

Leave a reply

1 minute, 56 seconds

I have to admit, one of the utter joys of my job is that they encourage me to open source software I write at my day job.  After looking high and low for a PHP framework or library to do a basic MySQL CRUD GUI, I gave up.  While phpMyAdmin is the longstanding champion for full featured DB administration, it’s way to complicated for an end user looking to just add a row right quick.  There seems to be an amazing project called CrudKit (great name!), but it has this one, massive blocking “feature”:

prevents usage in MVC frameworks
– commit 047807d01f

This is, literally, what I was trying to do. I spent a some time seeing how hard it would be to contribute to CrudKit to get the feature I wanted working.  I ultimately decided that a bespoke solution would more quickly achieve my desired goals.  That said, if you do want a stand alone app, do check out CrudKit.

While I suspect it could use some rewrites to not have silly-long arrays passed as arguments, I’m quite happy with my results: tableMaker. This guy takes this PHP:

$tm = new tableManager(DB_SERVER, DB_USER, DB_PASS, DATABASE, TABLE);
$rowsArray = $tm->getRowsFromTable();
print $tm->getHtmlFromRows($rowsArray, "/edit?table={$tm->table}&id=");

And turns it into this HTML:

If you want to render a fully functional edit form with dynamic client side error handling and table sensitive validation rules, just run this PHP:

$row = $tm->getRowFromTable($_GET['id']);
print $tm->getAddEditHtml($row, 'edit', "/save?table={$tm->table}");

Which will output this responsive, nice looking HTML:

Two big features of tableMaker are it’s simplicity and it’s security*. Yes table maker can do whiz bang client side sorting, but it also can output tidy, HTML compliant tables.  Yes, we can make your browser download 100k+ of web fonts just to render an “X” when you have an error in your form, but it can also do with out all that noise – implementers choice!  Security wise, tableMaker abstracts away all the complexity while ensuring there’s simply no way for you to expose yourself to a SQL injection attack. (* We need some nonces).

Along they way in making this, I see all the cool kids are using Composer.  I’ve earmarked this guy for my next project!

I’m really happy to have been paid to write this library; I’d be even happier if some else on started using it! I’d about piss my pants with glee if some opened a PR ;)

Update 3/18/2017 – Issue #3 on tableManager has been closed – CSRF protection in place!  Go Nonce, go!

MapTableMaker: An open source, simple-to-use, high resolution SVG map tool

Leave a reply

0 minutes, 45 seconds

After helping release MapTable, I knew I wanted to make it more accessible for non-programmer types. While my post about using MapTable in WordPress was a start at this, you still had to have be fairly comfortable with code.

No longer! As of today, you can now use a stand alone web site to generate high resolution choropleth maps: enter MapTableMaker! This is an web application that let’s you enter CSV values for each country and quickly generate a choropleth map of the values. It allows you to specify both positive integers which will render in blue and negative ones that will show in red. It uses a percentile scale to distance the colors so they’ll be more easy to discern from each other. The resulting map is high resolution and print-ready. Here’s an example:

Of course, if you’d like run your own instance of this, MapTableMaker is fully open source (MIT). Check out the GitHub page for MapTableMaker for details on how to get your own server set up!

Detecting IPv6 with JS and PHP

Leave a reply

0 minutes, 36 seconds

A friend of mine wanted a way to know if visitors to her site were coming from an IPv6 address. To do this, you would run some PHP to output a JS variable with global scope:

// thanks to:
// https://plugins.svn.wordpress.org/ipv6detector
$ip =  $_SERVER['REMOTE_ADDR'];
if (substr_count($ip, ":") > 0 && substr_count($ip, ".") == 0) {
        $js = 'var is6 = true';
} else {
        $js = 'var is6 = false';
}
print "<script>$js</script>";

And then to give the user a sense of excitement, animate showing them if they are v6 or v4. You’ll need jQuery for this, of course:

<div id="ipwhich" style="font-size:2em">?</div>
<script>
var showMe = 'v4!';
if(is6){
        var showMe = 'v6!';     
}
$("#ipwhich").delay(1000).hide().delay(1000).html(showMe).fadeIn();
</script>
</div>

I did up a live demo: give it a try!

How to test PHPs memory_limit setting

4 Replies

0 minutes, 42 seconds

So, we all know that in PHP, you configure it with a php.ini file. And in there, you can set the amount of RAM a script can use with the memory_limitsetting (remember this is “M” not “MB”!).  And if you get this error:

PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 234881025 bytes)

Then you can increase the memory_limit to be larger (don’t forget to restart apache!). However, what if you want a script to hit that limit to see how your error logs and such are set up?  I had more fun than I thought I would writing a textbook solution to a textbook problem.  Here it is in it’s 4 line glory:

$str = 'memory!';
$i = 1;
while ($i++ != 100) $str .= $str  ;
print "done!";

When you run this you should see an error as this will exceed 128M of memory. If not, so salt to taste ($i++ != 200) if you run with a higher memory_limit setting!

All HTTPS all the time, With HSTS to boot

4 Replies

1 minute, 54 seconds

I’ve been brushing up on my web security best practices recently.  OWASP is a great resource for this!  One of their recommended best practices is to use HTTP Strict Transport Security (HSTS).  This involves redirecting traffic from unencrypted HTTP to HTTPS.  However to ensure that no future Man in the Middle attacks happen with the redirect, it’s best to tell the browser to always go directly to HTTPS regardless of the protocol.  This, in a nutshell is the HSTS solution.

I’ve updated plip.com and blog.plip.com to be served over exclusively over HTTPS.  This is thanks to a *.plip.com wildcard certificate from Global Sign. After setting up Apache to use the certs on the SSL vhosts, I then needed to redirect all traffic away from HTTP.  For plip.com, this was a simple Apache rule in the HTTP vhost:

# send everything to HTTPS
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}

And then for the blog.plip.com, iThemes had this codex entry about a simple plugin to rewrite HTTP to HTTPS, following the second option on their page.  They do caution that this plugin might have performance drawbacks as you’re parsing every post on the fly.  You can fix this if you’re running a caching system, like W3 Total Cache, which I am! W3TC recommends you fix slow HTTPS calls by enabling caching of HTTPS: Go to Performance -> Page Cache and check “Cache SSL (https) requests.” Easy peasy!

Now to add the HSTS to the HTTP header.  For plip.com this is easy as I have a single PHP header file for the entire site. I just added this line:

header('Strict-Transport-Security: max-age=31536000');

For the blog, I extended the simple iThemes plugin by adding these lines:

add_action( 'send_headers', 'add_header_hsts');
function add_header_hsts(){
        header('Strict-Transport-Security: max-age=31536000');
}

Special thanks to the WordPress Codex on how to set headers as well as a random post over at Hakre on WordPress on how to format the HTTP header in PHP for HSTS.

Plip.com has absolutely zero affect on the big players, and the EFF would never care about giving me a report, but I’m scoring 4 out of 5 on EFFs encrypt the web report:

  1. Plip doesn’t have a data center, but all connections for administration are encrypted.
  2. Plip now, of course, supports HTTPS
  3. Plip now supports HSTS
  4. Plip does not support Forward Secrecy
  5. As Plip uses Google Apps, it supports STARTTLS

Looking at what it takes to set up my ciphers, I’m still gonna shoot for getting a perfect 5 of 5!

How to spam this blog

Leave a reply

1 minute, 39 seconds

As a follow up to last week’s post (How to comment on this blog), this week I bring you the results of the no-captcha test.

After much spam slipping through reCAPTCHA, I decided to nix a captcha all together. Originally I thought that just requiring a field via javascript and doing no server side checking would work. This was silly of me, of course. The spammers, having the source code of WordPress, would just blindly submit a comment to any post, bypassing any client side JS checks I had in place.

The fix was to create a field that was not known to spammers like the reCAPTCHA is. Further, if it is appended via javascript, then it is even harder to automate. I wrote the simple-math plugin (have a copy!) and implemented it as follow:

  • Turn off reCAPTCHA
  • Add a field via javascript
  • Ask a simple math question, validated in client side JS
  • Only validate that the field exists, not that the math is right, on the server side

The jury is, and I’m fully vindicated. Here’s the stats:

Hits Comment
Attempts		 Comment
Succeses		 Attempts
per
Visit		 Defense
Success
Rate
Feb 6th-12th 1191 57 17 4.79% 70.18%
Feb 12 11pm – Feb 13 10am 58 20 13 34.48% 35.00%
Feb 13th-Feb18th 1204 132 0 10.96% 100.00%

#spamstats td, #spamstats th {padding:4px;margin:5px}
#spamstats td {text-align:center;}
#spamstats tr:hover {background:#ccc}

The important thing to note is twofold. The first is that the average number of raw hits (excluding me, yahoo and google) was the same week to week. Further, the number of attempts went up 200% of which 100% were thwarted (Defense Success Rate). Again, I suspect this is all possible because it’s not easy, nor worth while (it’s OK, plip isn’t a big blog, I know…sniff) to automate spamming against one off solutions like mine.

I should note that I used the free version of Splunk to garner the ad hoc stats for this post. As I was hemming and hawing on whether to count cookies or IPs or hits, it wasn’t worth while to use the old school command line style stats. Splunk scoffs at this level of stats and reporting. Really, it’s above it, but will happily crank out what you ask for it with ease. Here’s a purty graph:

Caveat Emptor: I work at Splunk.

How to comment on this blog

Leave a reply

1 minute, 20 seconds

It seems that reCAPTCHA is a victim of its own success. Y’all know I’m a huge, huge fan. However, recently the spammers have started to submit comments, successfully getting past the reCAPTCHA . I suspect this is a mechanical turk or some such tomfoolery. Of course the comments don’t get approved, but they’re still a bother to have to delete.

Our friend over at hanskellner.com ( guess which friend?) also has the same problem with submitted span. This makes it clear that reCAPTCHA is being targeted (well, not clear, but it’s better than n=1!). However, he found a solution to stop the spammers. He added a static math question to his comment form. That is, it’s always “what is 5 + 6”, never any other question. Funny enough, his spam stopped all together. He still has his reCAPTCHA giong, but now it’s a two factor anti-spam.

I posit that the reCAPTCHA code is easy enough to programmatically detect, but some random math question isn’t, so it breaks the spam scripts. Let’s test this theory, shall we? I’ve just written a word press plug-in called simple-math. Using a simple to hack, all client side javascript there’s now an easy to solve math problem on the comment form. It is random, choosing two numbers between 0 and 9. I haven’t tested it too broadly, but you’re welcome to a copy.

I’ll let it run for a week and see how it goes and report back.

Feb 13th Update: I fought the law, and law won! Spammers got past round one of simple math. I’ve updated it to now check for the existence of the field on post, but still, no checking for a right answer on the server. As well, the field is created via javascript. Spammers, back to you for round 2.