• Building a HLS Muxing Raspberry Pi Cluster

    It was quite a long time ago now that I started HLS-Stream-Creator, and I've previously released an example of automating HLS-Stream-Creator so that it receives and processes workloads.

    I never really expected that I'd actually have much practical use for HLS Stream Creator when I created it (I created it as a means to learning about HLS in advance of a 2nd interview), particularly as I wasn't generating/publishing any video at the time.

    Over time, though, that's changed and my needs have grown from occasionally invoking the script to wanting the convenience of having a dedicated muxing pool so that I can simply submit a video and have it come out in HTTP Live Streaming (HLS) format (particularly useful now that I have videos.bentasker.co.uk).

    Although I'm not going to focus on the Command and Control aspect (at it's heart it's a simple REST API) in any depth, this documentation will detail the process I followed in order to have 3 Raspberry Pi's PXE boot and run HLS-Stream-Creator as a service in order to receive arbitrary videos, calculate the output bitrates and then generate HLS output.

    It's as much an opportunity to document the process I used to PXE boot Raspberry Pi's with a NFS root filesystem.

  • Building a Raspberry Pi Based Music Kiosk

    I used to use Google's Play Music to host and play our music collection.

    However, years ago, I got annoyed with Google's lacklustre approach to shared collections, and odd approach to VMs. So, our collection migrated into a self-hosted copy of Subsonic.

    Other than a few minor frustrations, I've never looked back.

    I buy my music through whatever music service I want, download it onto the NFS share and Subsonic picks up on it following the next library scan - we can then stream it to our phones (using DSub), to the TV (via a Kodi plugin) or to a desktop (generally, using Jamstash). In the kitchen, I tend to use a bluetooth speaker with the tablet that I use to look up recipes.

    However, we're planning on repurposing a room into a puzzle and playroom, so I wanted to put some dedicated music playback in there.

    Sonos devices have Subsonic support, but (IMO) that's a lot of money for something that's not great quality, and potentially has an arbitrarily shortened lifetime.

    So, I decided to build something myself using a Raspberry Pi, a touchscreen and Chromium in kiosk mode. To keep things simple, I've used the audio out jack on the Pi, but if over time I find the quality isn't what I hope, it should just be a case of connecting a USB soundcard in to resolve it.

    There's no reason you shouldn't be able to follow almost exactly the same steps if you're using Ampache or even Google Play Music as your music source.

  • Creating a Virtual Network Interface in Debian

    There are times when you might want to assign more than one IP to a system, even if it only has a single physical NIC. This documentation details how to create a virtual network interface (known as aliasing) under Debian (see here for how to alias in Centos 6).

  • HLS Muxing Pi Cluster

  • Video of a 3 pi cluster (the one on the left is doing something else) processing muxing jobs, transcoding video to x264 and then muxing down to HLS for output. Setup process detailed here

  • Implementing Secure Password Storage with PHPCredlocker and a Raspberry Pi

    Password storage can be a sensitive business, but no matter whether you're using PHPCredlocker or KeePassX, dedicated hardware is best. The more isolated your password storage solution, the less likely it is that unauthorised access can be obtained.

    Of course, dedicated hardware can quickly become expensive. Whilst it might be ideal in terms of security, who can afford to Colo a server just to store their passwords? A VPS is a trade-off - anyone with access to the hypervisor could potentially grab your encryption keys from memory (or the back-end storage).

    To try and reduce the cost, whilst maintaining the security ideal of having dedicated hardware, I set out to get PHPCredlocker running on a Raspberry Pi.

    This documentation details how to build the system, a Raspberry Pi Model B+ was used, but the B should be fine too

  • Kernel Modules missing on Rasberry Pi

    This, almost certainly, was a mess of my own making, but as I didn't find any answers with web searches I thought it was worth documenting for anyone else who sets a similar time bomb for themselves.

    I've got some Raspberry Pi's which use NFS for their root partition. They used to be PXE booted, but at some point starting failing to boot so some time back I put a SD card back in for the /boot partition.

    This, I suspect, was probably my undoing.

    The Pi's have been working fine since, but I wanted to install Docker onto one of them. Although it installed, Docker failed to start, logging the following

    Oct 09 22:45:43 redim-4-search-pi dockerd[3534]: failed to start daemon: Error initializing network controller: error obtaining controller instance: failed to
    Oct 09 22:45:43 redim-4-search-pi dockerd[3534]: modprobe: FATAL: Module ip_tables not found in directory /lib/modules/4.19.75-v8+
    Oct 09 22:45:43 redim-4-search-pi dockerd[3534]: iptables v1.6.0: can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
    Oct 09 22:45:43 redim-4-search-pi dockerd[3534]: Perhaps iptables or your kernel needs to be upgraded.
    Oct 09 22:45:43 redim-4-search-pi dockerd[3534]:  (exit status 3)
    Oct 09 22:45:43 redim-4-search-pi systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE

    On examination, there is no modules directory for the kernel version I'm currently running

    root@redim-4-search-pi:~# uname -r
    root@redim-4-search-pi:~# ls /lib/modules/
    4.19.66+  4.19.66-v7+

    This post details the steps I took to resolve this issue

  • NTPD Refusing to accept time from GPSD

    One of the (minor) drawbacks of the Raspberry Pi is the lack of a hardware clock. Normally, you'd work around this by configuring a good pool of NTP servers to connect to. What do you do though, if you can't guarantee there will be an Internet connection available when needed?

    The solution is obvious, so obvious that many have already done it - use the time provided by a cheap GPS dongle. The gpsd daemon helpfully pushes the time to Shared Memory Segments (SHM) so it's a simple adjustment to the NTP configuration file to have NTPD pull the time from the dongle.

    Except, it seems on Raspbian, it isn't quite so simple. You've followed all the instructions (simple as they are) but are still seeing an entry like this

    # ntpq -p
         remote           refid      st t when poll reach   delay   offset  jitter
    SHM(0)          .GPS.            0 l    -   16    0    0.000    0.000   0.001

    No matter what you try, reach stays at 0.

    Frustrating, and there's very little to give you a guide. This post will tell you what the issue is, as well as how to go about finding it should it re-occur

  • Testing Raspberry Pi Images with Qemu

    Although changing the OS on a Raspberry Pi is quick and easy (especially if you have a spare SD card), there are times when you might want to test a system first, or simply tinker without needing a spare Pi.

    This documentation details how to use Qemu to run a RaspberryPi image.

  • Usurping the BTHomeHub with a Raspberry Pi: Part 1

    I used to run a nice pfSense box as my router, unfortunately power bills being what they are, I reverted back to using a BTHomeHub. Unfortunately, the BTHomeHub isn't particularly good - the Wifi signal sucks, it's DNS server seems to daydream and it occasionally forgets that it should be assigning some devices the same IP every time (or more precisely, will give their IP away if they are not currently present).

    We could, of course, replace the HomeHub with something a bit more up market, but where's the fun in that? In this post, we'll be starting down the route of using a Raspberry Pi to usurp some of the power the BTHomeHub currently holds over the LAN. Eventually, the HH will be acting as nothing but a dumb internet gateway, doing a little bit of NAT and not much else.

  • Usurping the BTHomeHub with a Raspberry Pi: Part 2 - DNS, DHCP and NTP

    In Part One, we configured our RaspberryPi to act as a Wireless access point and bridged the wireless and wired interfaces so that WLAN client's were easily accessible from the LAN.

    As part of that setup, we configured a DHCP server, however we haven't yet made it the DHCP server for the LAN - our tired old BTHomeHub is still the authoritative server for the network.

    In this part, we'll be reconfiguring our DHCP server so that it takes responsibility for the entire LAN, configuring DNS services, and making our Pi the LANs central NTP (Network Time Protocol) Server

    Step by step, we'll be configuring our Raspberry Pi to take over nearly all of the duties performed by the BTHomeHub.

  • Usurping the BTHomeHub with a Raspberry Pi: Part 3 - Routing, Remote Administration and Utilities

    In Part One we configured a RaspberryPi to act as a Wireless Access point, providing DHCP services to wireless clients. In Part Two we then configured our Pi to provide DHCP, DNS and NTP services to the entire LAN.

    In this part, we'll be taking some more responsibility away from the BTHomeHub, as well as configuring a few conveniences, such as Remote administration and useful utilities, including

    • Wake On Lan
    • Network Troubleshooting Tools
    • Dynamic DNS Update Client (No-Ip.com)


  • Usurping the BTHomeHub with a Raspberry Pi: Part 4 - Using a VPN to Tunnel Connections to Specific IPs

    Content Filtering is becoming increasingly popular amongst Politicians, ISPs and generally clueless do-gooders. The problem  is, whatever you think of their motives, it's generally poorly implemented and interferes with the end-users browsing experience, even when it's not supposed to (the image to the right appeared with filtering off! - click to enlarge).

    As we've been Usurping the BTHomeHub with a Raspberry Pi, we're going to take a brief break to implement some useful functionality that the HomeHub didn't provide.

    In this Part, we're going to configure our Raspberry Pi to connect to an OpenVPN server and route some of our traffic over the tunnel - depending on the destination IP (i.e. Split tunnelling). This will allow us to easily bypass the troublesome content filtering, whilst not un-necessarily introducing any latency to any connection that is (for the time being at least) unaffected by the filters.

    Note: We'll be manually specifying the connections that are routed via VPN, so that we can 'whitelist' mistakes such as the EFF and Wikipedia, whilst still being 'protected' against other filtered pages.

    Unless otherwise stated, all commands need to be run as root

  • Usurping the BTHomeHub with a Raspberry Pi: Part 5 - Inbound OpenVPN

    In Part 4 we configured our Raspberry Pi router to maintain a number of OpenVPN tunnels and to route through them selectively. Now we'll look at the steps needed to allow connection to our LAN via OpenVPN. Although helpful, as the HomeHub doesn't provide VPN connectivity, this stage doesn't really count as Usurping the BTHomeHub.

    The steps are almost completely identical to those performed when Installing Open VPN on Debian. We're going to have to NAT connections though, as the HomeHub is a little stupid and we can't add static routes to it (so if we're connected to the VPN and accessing the Internet, it won't know where to route the response packets).

    What we'll do, though, is only NAT if the connection isn't to something on the LAN.