• Building a Tor Hidden Service CDN

    Last year I started experimenting with the idea of building a Hidden Service CDN.

    People often complain that Tor is slow, though my domain sharding adjustments to the bentasker.co.uk onion have proven fairly effective in addressing page load times.

    On the clearnet, the aim traditionally, is to try and direct the user to an edge-node close to them. That's obviously not possible for a Tor Hidden service to do (and even if it were, the users circuit might still take packets half-way across the globe). So, the primary aim is instead to spread load and introduce some redundancy.

    One option for spreading load is to have a load balancer run Tor and then spread requests across the back-end. That, however, does nothing for redundancy if the load-balancer (or it's link) fails.

    The main aim was to see what could be achieved in terms of scaling out a high traffic service. Raw data and more detailed analysis of the results can be seen here. Honestly speaking, It's not the most disciplined or structured research I've ever done, but the necessary information should all be there.

    This document is essentially a high-level write up along with some additional observations

  • Transcoding files ready for HTTP Live Streaming on Linux

    HTTP Live Streaming (HLS) is an IETF draft standard created by Apple Inc. It's use is pretty widespread, although it was primarily designed to allow video to be easily delivered to iOS devices it works well with a wide range of clients. Later versions of Android support it, as do players like VLC.

    Unfortunately, a lot of the tools for creating (and testing) HLS streams are created and released by Apple. Unless you develop for iOS devices, you probably lack a developer login!

    It's actually pretty easy to set up at a basic level. In this documentation we'll be looking at what HLS is, and how to prepare video for transmission using HLS.