• A Linux SysAdmins Guide to Mischief

    This book is designed to give Sysadmins some entertainment in the form of winding users up (in a non-destructive manner). The book contains tricks ranging from puerile to annoying, all designed for those administering Linux workstations to play on those using Linux workstations!

    All are achievable remotely, allowing the Sysadmin to sit back and listen for the sounds of frustration safely out of range of flying staplers. Intended as an exercise in humour, it should provide Sysadmins with some entertainment. Don't expect it to beat a Friday afternoon read of Bootnotes and Odd's n Sod's though!

    It's a pretty short book, hence the low price.

     

    2021: You can access the book here.

  • An easier method of recovering deleted files on Linux

    A little while ago I posted some Documentation on how to Recover Deleted Inodes on an Ext2 filesystem. After a self-inflicted disaster last night, I found an eaier way to recover deleted files. It also has the benefit of being largely filesystem agnostic, there have been reports that it doesn't work with ReiserFS but otherwise it can recover deleted files from almost any filesystem on any medium.

    WARNING: The Steps Given in this Article are potentially Dangerous, Proceed at your own risk and read the Article in full before starting

  • Archiving a large backup across multiple discs on Linux

     

    Hopefully, we all back up our data, but what should we do once our data won't fit on our chosen media?

     

    We have two options (as we obviously don't want to delete our data!)

    • Use a different backup medium
    • Split the backup across multiple volumes

    Sometimes the former just isn't appropriate, as much because of the cost of harddrives vs Optical Media (i.e. CD's/DVD's).

    This short tutorial will explain how to create a single backup archive, and then split it across multiple CD's/DVD's.

     

  • Automatically Mounting Secondary Encrypted Disk/Partition at Boot

    There are a wide variety of use-cases for disk encryption, and the idea of automatically mounting an encrypted disk/partition without user intervention is an anathema to many of those - anyone who can take physical possession of your system will have the disk auto-mount for them.

    However, there is a very simple use-case which benefits from being able to automount a second encrypted disk.

    If you're storing data unencrypted on a drive and it fails, you're now potentially left with something of an issue, particularly if you intend to RMA it (return it under warranty) - could the drive be fixed, allowing someone else to pull that data off the drive (bearing in mind the manufacturer may fix the drive and sell as refurbished)?

    Similarly, when you need to expand your storage, you hit a similar conundrum - do you trust disk wipes sufficiently to be willing to sell/pass the disk on (a particular concern with SSDs where data may previously have been written to a now bad block, so won't be overwritten by your wipe), or do you feel you have to physically destroy the disk, un-necessarily generating e-waste.

    Using Full Disk Encryption (FDE) addresses both of these situations - the manufacturer might fix the disk, but without the key the data's just random bytes, similarly, for whoever buys your disk off ebay.

    But, FDE can quickly become a major inconvenience at boot - your system will stop booting and ask you to provide the decryption passphrase. That's particularly problematic if you're talking about a headless system like a NAS, where you want things to come up working following a power cycle.

    It's possible (trivial even) to configure so that the system uses a key stored on another disk (like your root filesystem, or if you prefer, a USB flash drive) so that the partition is automagically mounted.

    This documentation details how to set up ecryptfs on a disk (or partition) and add it to /etc/fstab so that it automatically mounts at boot

    All commands are run as root, so use sudo -i/su

     

  • Bow Before Me

  • I got woken up earlier than normal this morning.

    As a result, I spent longer than I should've creating this and the derived gif:

  • Building a DNS over TLS (DoT) server

    WARNING: This article is outdated and has been superseded by Configuring Unbound for Downstream DoT.

    Since Unbound 1.6.7 there's a better way than the one described here

    I previously posted some documentation on how to build a DNS over HTTPS (DoH) Server running Pi-Hole and/or Unbound.

    There's another standard available, however - RFC 7858 DNS over TLS (DoT)

    DoT isn't as censorship resistant as DoH (as it's easier to block), but does provide you with additional privacy. It also has the advantage of being natively supported in Android Pie (9), so can be used to regain control of your queries without needing to run a dedicated app link Intra, with all the issues that might entail.

    In this documentation we're going to trivially build and place queries against a DoT server.

  • Building a HLS Muxing Raspberry Pi Cluster

    It was quite a long time ago now that I started HLS-Stream-Creator, and I've previously released an example of automating HLS-Stream-Creator so that it receives and processes workloads.

    I never really expected that I'd actually have much practical use for HLS Stream Creator when I created it (I created it as a means to learning about HLS in advance of a 2nd interview), particularly as I wasn't generating/publishing any video at the time.

    Over time, though, that's changed and my needs have grown from occasionally invoking the script to wanting the convenience of having a dedicated muxing pool so that I can simply submit a video and have it come out in HTTP Live Streaming (HLS) format (particularly useful now that I have videos.bentasker.co.uk).

    Although I'm not going to focus on the Command and Control aspect (at it's heart it's a simple REST API) in any depth, this documentation will detail the process I followed in order to have 3 Raspberry Pi's PXE boot and run HLS-Stream-Creator as a service in order to receive arbitrary videos, calculate the output bitrates and then generate HLS output.

    It's as much an opportunity to document the process I used to PXE boot Raspberry Pi's with a NFS root filesystem.

  • Building a Tor Hidden Service From Scratch - Part 1 - Design and Setup

    Despite some fairly negative media attention, not every Tor Hidden Service is (or needs to be) a hotbed of immorality. Some exist in order to allow those in restrictive countries to access things we might take for granted (like Christian materials).

    Whilst I can't condone immoral activities, Tor is a tool, and any tool can be used or misused

    This is part one in a detailed walk through of the considerations and design steps that may need to be made when setting up a new Tor Hidden Service.

    The steps provided are intended to take security/privacy seriously, but won't defend against a wealthy state-backed attacker.

    How much of it you'll need to implement will obviously depend on your own circumstances, and in some cases there may be additional steps you need to take

  • Building and running your own DNS-over-HTTPS Server

    There's been a fair bit of controversy over DNS-over-HTTPS (DoH) vs DNS-over-TLS (DoT), and some of those arguments still rage on.

    But, DoH isn't currently going anywhere, and Firefox has directly implemented support (though it calls them Trusted Recursive Resolvers or TRR for short).

    Although DoH offers some fairly serious advantages when out and about (preventing blocking or tampering of DNS lookups by network operators), when left with default configuration it does currently come with some new privacy concerns of it's own. Do you really want all your DNS queries going via Cloudflare? Do you want them to be able to (roughly) tell when your mobile device is home, and when it's out and about (and potentially, also your employer - if they own the netblock)? The same questions of course go if you use Google's DNS too.

    That, however, is addressable by running your own DNS-over-HTTPS server. This also has advantages if you're trying to do split-horizon DNS on your LAN, so I'll discuss that later too.

    The primary purpose of this documentation is to detail how to set up your own DoH server on Linux. The main block of this documentation is concerned with getting a NGinx fronted DoH server backed by Unbound up and running, but will also discuss the steps needed to add Pi-Hole into the mix.

    Unless otherwise noted, all commands are run as root

  • CentOS: Using NGinx to serve static files and Apache for dynamic

    Apache is a great web-server, but it has a pretty heavy memory footprint. It can get quite restrictive quite quickly, especially if you're on a system will limited resources (given how many people now run on a VPS, and the poor disk IO of these systems it's all the more important - swapping is slow).

    The way around it, is to configure your system to use NGinx as a reverse-proxy. Depending how many virtualhosts you have, you can make the changes almost completely transparently within about 10 minutes.

  • Checking for Outdated Joomla Extensions on your server

    When you're managing Joomla sites it's reasonably easy to keep track of updates, especially if you use something like Watchful to help you. When you're running a server and only managing some (or none) of those sites, it becomes a little more difficult (especially on a busy shared hosting server).

    It's quite easy to shrug and say 'Not my site, not my problem', but the simple fact is that it is. The second someone manages to compromise one of the sites you host, they're going to try and find a way to run arbitrary code, once they've done that they'll try to run an auto-rooter. If they succeed, it's game over for everyone you host!

    The extension that always comes to mind, is the Joomla Content Editor (JCE) as they had a nasty vulnerability involving spoofed GIFs some time back. You'd hope that everyone would have updated by now, but there still seem to be a lot of sites running versions older than 2.1.1!

    In this post, we'll be creating a script designed to automatically check every one of the sites you host for a version of JCE older than the latest. Adjusting it to check other extensions is easy, so long as that extension has an update stream.

  • Configuring NGinx to act as a Reverse Proxy for PHPMyAdmin

    In a previous post, I detailed how to Use NGinx to serve static files and Apache for dynamic as well as the minor tweaks you need to make to have it work nicely with Joomla.

    One thing I didn't cover, though, is setting up PHPMyAdmin. This documentation isn't going to go into the detail of installing and configuring PHPMyAdmin as there's plenty of that available elsewhere on the web. What we will discuss, though, is the NGinx configuration changes you need to make to have the connection reverse proxied to Apache.

    These steps only really apply if you've gone for a system-wide installation of PMA. If you've unpacked into a web-accessible directory then you probably don't need to make any changes!

  • Configuring Postfix to automatically forward mail for one address to another

    There seem to be a number of people searching for how to do this, and from what I can see there's very little quick and easy documentation on the net. You've got a server, hosting a website (for example) for example.com.

    You want the server to accept mail for example.com but to automatically pass the mail onto a different address.

    Assuming you're running Postfix, it's as simple as the steps below

  • Configuring Postfix to block outgoing mail to all but one domain

    This is so simple to do, but I have to look it up every time I need it (not something that comes up regularly!);

    When configuring a development server, you may find you have a need to ensure that emails will not be sent to any domain except those you explicitly permit (for example if you're using real-world data to do some testing, do you want to send all those users irrelevant emails?).

    This documentation details how to configure Postfix on a Linux server to disregard any mail sent to domains that are not explicitly permitted.

  • Configuring Unbound for Downstream DoT

    Quite some time ago, I wrote some documentation on how to stand up a DNS-over-TLS server using a Nginx reverse streams proxy to terminate the SSL.

    However, since then (in fact, back in 1.6.7) Unbound released support for directly terminating TLS connections.

    This documentation details the (simple) config changes necessary to configure Unbound to service DNS over TLS (RFC 7858) queries.

     

  • Copying a Linux Kernel From One System to Another

    There may be occasions where, for testing purposes, you want to copy a kernel from one machine to another.

    There are some fairly self-explanatory caveats:

    • The donor and target system must be running on the same architecture
    • The target machine shouldn't have any (important) hardware that's unsupported by your donor kernel

    Obviously, you'll ideally want to make sure that the hardware is as close to identical as possible (otherwise your testing may be invalid) so the above should be considered a minimum

  • Creating a virtual Network Interface in CentOS 6

    Sometimes you need to assign more than one IP to a server, even if it only has one NIC. To do so, you create a virtual (or aliased) interface, attached to the physical NIC.

    This documentation details how to do this in CentOS5 / CentOS 6 (this also applies to CentOS7 if you're not using Network Manager).

  • Creating a Virtual Network Interface in Debian

    There are times when you might want to assign more than one IP to a system, even if it only has a single physical NIC. This documentation details how to create a virtual network interface (known as aliasing) under Debian (see here for how to alias in Centos 6).

  • Creating an IPv6 Tunnel on Linux

    RIPE, the European internet registry has started heavily rationing IPv4 addresses, meaning that the day of IPv6 only connections is fast approaching. BT don't yet support IPv6 on their connections, but I need to be able to use IPv6 to help ensure that servers are correctly set up to handle IPv6 only traffic.

    So, I need to create an IPv6 over IPv4 tunnel.

    This documentation details the steps to do this using Helium Electric's (free) tunnelbroker service

  • Darkleech Apache attacks on the rise, but is it really that hard to detect?

    Reports of CDorked.A infections are still on the rise by the looks of things. The attack is reported as 'hard-to-detect', but this should only be true for the more naive sysadmins out there.

    Whilst it's true that CDorked changes nothing on disk, except the HTTPD binary, this change alone should be triggering alerts. On a production server, you should be storing checksums of known good files and comparing these regularly to see if anything's changed.

    As some obviously aren't following this basic step, in this post we'll look at what you need to do to at least be made aware if CDorked gets onto your system - it'd be nice to be able to do a post on avoiding it, but the attack vector is still unknown!