Building a Tor Hidden Service From Scratch - Part 1 - Design and Setup

Despite some fairly negative media attention, not every Tor Hidden Service is (or needs to be) a hotbed of immorality. Some exist in order to allow those in restrictive countries to access things we might take for granted (like Christian materials).

Whilst I can't condone immoral activities, Tor is a tool, and any tool can be used or misused

This is part one in a detailed walk through of the considerations and design steps that may need to be made when setting up a new Tor Hidden Service.

The steps provided are intended to take security/privacy seriously, but won't defend against a wealthy state-backed attacker.

How much of it you'll need to implement will obviously depend on your own circumstances, and in some cases there may be additional steps you need to take

Introduction

This section will cover the basics of setting up a hidden service, including

  • Decisions you need to make regarding hosting and hidden service operation
  • Controlling/Securing SSH access via Clearnet address
  • Basic Opsec changes to the server
  • Installing and Configuring Tor
  • Enabling a Tor Hidden Service for routine SSH access
  • Configuring an SSH client to connect to that hidden service

 

Hosting Theory

Hosting: VM or Hardware?

Price is an obvious factor in the decision, but leaving that aside.

Each bring their own benefit, using VM's makes it easier to isolate the server from the wider network but you also have to completely trust the integrity of the host hardware.

Where possible, the preferred solution would be to have full control of the host hardware and then run hidden services within their own VM. An additional VM is then used as a firewall, with two NIC's - one on the RFC1918 subnet (used by all the other VM's) and one bridged to the Host.

One risk to be aware of if running multiple hidden services (regardless of VM usage or otherwise) - if the server goes down, then so will _all_ of the hidden services. Over time, an adversary could correlate downtime in order to show that the services are all hosted on the same machine.

One thing you'll likely notice as we progress is that consideration always needs to be given to the fact that you might get compromised at some point. Whilst it's important to minimise the likelihood of that, it's no less important to ensure that we attempt to mitigate the effects of it (for example by compartmentalizing services).

 

Common Deployments

It's obviously very difficult to say what configurations/systems are preferred by Hidden Service operators, but anecdotal evidence would suggest that OpenVZ slices are incredibly popular.

It's not entirely clear why, as using an OpenVZ slice (on someone else's hardware) brings it's own set of risks, but the monthly cost is generally lower, and there are at least some OpenVZ hosts who are willing to accept payment in anonymous forms (such as BTC).

Because OpenVZ is effectively containerisation, the overhead of running a 'VM' is lower, so you can squeeze more slices onto a single server than you could traditional VMs.

For the HS operator though, it carries some additional risks:

  1. Your root filesystem is a directory structure on the host system. The host can easily use tools such as grep and find to explore your filesystem. You don't have the option of HD encryption to protect against this
  2. You have little to no control over what kernel modules are loaded
  3. Escaping from containerisation is, generally, easier than escaping full virtualisation, so there's a (small) risk that another user on that system may also gain access to your VM

 

HTTP or HTTPS

Connecting via Tor gives you encryption 'for free', but some thought should still be given to whether HTTPS is a better option. There are some definite drawbacks, but also some benefits to using HTTPS.

Consider the following (simplified) deployment

  Client -> Tor HS (ServerA) -> HTTP(S) Server (ServerB)

The Client's connection is encrypted until it reaches the hidden service on ServerA, however the connection is then proxied, in the clear to ServerB. There may be a number of reasons why you'd want to configure a hidden service in this way, the most obvious being the following

  Client -> Tor HS (ServerA) -> Load Balancer
				  -> HTTP(S) Server (ServerB)
				  -> HTTP(S) Server (ServerC)

In this deployment, there has to be some trust of the network between ServerA, the load balancer and Servers B&C. The client isn't even aware of the transmission in cleartext, so some would argue that you're unfairly putting their data at risk.

The Snowden leaks showed that the NSA had noted the point at which Google terminated SSL connections (and transmitted in the clear within their network) so it is worth considering if Tor is running on a different server to the services host. For more info regarding the Snowden leak, search for the phrase "SSL added and removed here"

Downsides:

  • If you use a self-signed certificate, users will see a certificate warning before they can access the site
  • If you want a CA signed certificate (once you find a willing CA) you as the service provider will likely need to sacrifice some anonymity in order to be issued the certificate

When generating a self-signed certificate, there are also a couple of risks associated - so we'll cover doing that as a precaution

 

Practice vs Live

When working on a live server, you want to minimise the amount of information available for an attacker to use if they manage to compromise a server. The difficulty is, that whilst you want to practice that level of OpSec, it's also largely incompatible with looking back to understand where and why you made a mistake. To work around that, on a *practice server* it's wise to ensure you always to the following as soon as you've logged in

  screen -L

This will write a log of your activities (and program output) to ~/screenlog.0 Don't use screen sessions on a live server!

 

Getting Set Up

Before Tor is even installed, there are a few things which need to be considered

 

Initial access to the server

SSH must be via Tor - a single direct connection could be logged and would forever associate your originating IP with that server.

 

Locking down SSH

Options:

  • Key based auth only
  • SSH access limited to a 'trusted' source IP

If key based auth is used, the public key needs to be inspected to make sure it doesn't tie back to you (2019: for someone with resources/time, this can be done without compromising your box). A new key should be generated and then specified in your client's SSH config file (using IdentityFile).

However, ideally SSH needs to be firewalled off to prevent against Server key comparison based attacks (See Firewall config basics below for an example).

This is less of an issue, however, if you are configuring a system which isn't directly reachable from the internet (such as a VM on a class C network)

If firewall rules are used, the 'trusted' server needs to be one that isn't directly associated with you. It could be an anonymous VPN endpoint, another virtual machine, or potentially a specific Tor exit node.

Each carries it's own risks, using a source only accessible to you reduces the likelihood of key comparison attacks, but may make it easier to identify the server administrator (you).

Permitting publicly accessible sources makes it harder to identify you, but may make it easier to identify your server as the host of a given hidden service.

One other option, once you've achieved the initial access to the server is to configure a VPN (e.g. OpenVPN) on the server and then limit non-tor SSH access to addresses within the VPN address range. You can then connect to the VPN via Tor (hiding your true source IP) without exposing SSH to other users. Although this is a valid option longer-term, you still need to be able to limit SSH access in the meantime.

Routine access will be via a hidden service dedicated for SSH use, but you need some sort of fallback access (for example, if you need to restart Tor). If a serial console is available, that's also a viable option instead of SSH.

If SSH is enabled on the server, you should also consider disabling root logins via SSH - though the various SSH hardening methods fall outside the scope of this document.

 

Firewall Config Basics

In principle, no service that's made available via a Tor hidden service should be available to the clearnet. If a service is available on both, an attacker could compare information gleaned from both to prove that a given server hosts a given hidden service.

For example, if a hidden service providing SSH returns a server RSA fingerprint of A1234B, an NMap of the entire IPv4 address space could be used to identify whether any 'real' servers also return that fingerprint.

Traffic hitting a Hidden Service will always have an origination IP of 127.0.0.1 so adding an ALLOW rule for the loopback device should be sufficient to ensure connectivity.

Unless there's a specific case for doing otherwise (e.g. SSH), daemon's should be configured to bind to the loopback device only. This is done to mitigate any mistakes which may be made within the firewall configuration - external connections will fail

As a minimum, you'll want to do the following

    # Create a new chain for SSH
    iptables -N SSH

Replace the following source IP with your trusted host/source range.

    iptables -I SSH -s 192.168.1.3 -j ACCEPT
    iptables -A SSH -j DROP

These rules ensure that existing connections are given a pass, and allow all loopback connections

    iptables -I INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
    iptables -I INPUT -i lo -j ACCEPT

We then need to make the SSH rules apply - your connection will remain intact if the IP you authorised in the SSH chain was incorrect, just don't disconnect it

    iptables -A INPUT -p tcp --dport 22 -j SSH

Try and SSH to the server from an authorised IP. If you cannot connect you need to remove the rule (if your current connection drops for some reason you won't be able to reconnect without restarting the server) Once happy that the rules are appropriate, save them

    service iptables save

 

Protecting your History

By default, most shells will log your command history (for example BASH logs to ~/.bash_history). To help minimise the information available to an adversary who has compromised your server, you should do the following Ensure the file will never contain content

  rm ~/.bash_history
  ln -s /dev/null ~/.bash_history

Effectively disable any other history logging:

  echo "alias exit='kill -9 \$\$'" >> /etc/profile

The second change ensures that running 'exit' will send a SIGKILL to the parent process rather than running a true exit. Any commands set to run on logout (such as writing a history file) will not execute.

 

Hidden Service Layout Design

Some thought needs to go into how many Hidden Service descriptors you want to publish. Services that are not related to each other should always use different .onion addresses.

For example, assuming you wanted to make the following available via Tor Hidden services

  • HTTP
  • HTTPS
  • SSH
  • FTP (for users of the HTTP/HTTPS site)

Where the SSH access is only provided for your administration purposes, you'd want 2 hidden services, one containing SSH and the other the 'advertised' services, e.g.

  • HTTP, HTTPS, FTP - 123.onion
  • SSH - 456.onion

This helps to defend against two different threats

  • Tor based adversaries trying to compromise your hidden service
  • Adversaries trying to identify the clearnet address of the hosting for that hidden service

In the SSH section above, an example of correlating HS and Clearnet Server fingerprint's was given. Using a different .onion means that should such an attack occur, without further compromise of your systems, that adversary would only be able to show that the physical server is hosting 456.onion.

If the HTTP(S) descriptor also provided the SSH service, that adversary would now have identified you as the hosting platform for 123.onion

 

Preparation Specifics

So far, we've simply designed or made minor changes. Before we can install and configure Tor we need to make some specific changes to invalidate any information that an adversary might already have captured about our server (such as SSH RSA fingerprint)

To begin with, we need to ensure that the SSH restrictions are in place, so ensure you've got a firewall rule restricting access to port 22 to a trusted source IP (or have serial console access etc).

Our next step is to make sure that any keys already collected by our unnamed adversary won't match those of the hidden service when published

    ssh-keygen -q -f /etc/ssh/ssh_host_key -N '' -t rsa1
    ssh-keygen -f /etc/ssh/ssh_host_rsa_key -N '' -t rsa
    ssh-keygen -f /etc/ssh/ssh_host_dsa_key -N '' -t dsa
    ssh-keygen -f /etc/ssh/ssh_host_ecdsa_key -N '' -t ecdsa -b 521

The next time you SSH to the server from the client you are currently using, you will see a warning that the server key has changed.

This provides a good means to ensure that the changes were successful, so without disconnecting your current session, try to start a new SSH session (from the same client) with the server.

We also want to ensure that SSH won't leak information whenever we try to connect. The first thing we need to address is SSH's tendency to perform a reverse DNS (PTR) lookup whenever a client attempts to connect

    echo "UseDNS no" >> /etc/ssh/sshd_config

Now we need to make our changes take effect

    service sshd restart

 

Installing Tor

Always use the Tor provided repos - where distros provide their own Tor builds, they tend to fall out of date which may expose you to known weaknesses. Assuming a CentOS 6 server, do the following Create /etc/yum.repos.d/tor.repo and enter the following content into it

[tor]
name=Tor repo
enabled=1
baseurl=https://deb.torproject.org/torproject.org/rpm/el/6/$basearch/
gpgcheck=1
gpgkey=https://deb.torproject.org/torproject.org/rpm/RPM-GPG-KEY-torproject.org.asc

[tor-source]
name=Tor source repo
enabled=1
autorefresh=0
baseurl=https://deb.torproject.org/torproject.org/rpm/el/6/SRPMS
gpgcheck=1
gpgkey=https://deb.torproject.org/torproject.org/rpm/RPM-GPG-KEY-torproject.org.asc

Then run

    yum install tor -y
    service tor stop
    chkconfig tor off

The installer, by default, will start the Tor service (which is, a little irritating). We don't want Tor running until we've configured it, so we've stopped the service and removed Tor from init (just in case the system reboots whilst we're working).

 

Configuring Tor

Don't do this until you have restricted SSH access using iptables!

By default, the Tor configuration file contains a lot of entries, some we need, some we don't. You should never run a system as both a relay and a hidden service host as an adversary will easily be able to correlate any HS downtime with the relay information published in the Tor atlas (and onionoo etc).

The deployment that you'll most likely want, is to have Tor configured both as a client and publishing Hidden Services (we'll look at why you need the former shortly).

So to begin with, enter the following into /etc/tor/torrc (I usually blank the file before beginning)

    # Run in the background
    RunAsDaemon 1

    # Use the default DataDirectory
    DataDirectory /var/lib/tor

    # Client Config
    TransPort 9040
    TransListenAddress 127.0.0.1
    DNSPort 127.0.0.1:54

    # Configure the Socks proxy (useful for troubleshooting)
    SocksPort 127.0.0.1:8080
    SocksPolicy accept 127.0.0.1
    SocksPolicy accept 192.168.1.1 # Replace this with the server's IPv4 address
    SocksPolicy reject *

At this point, we've effectively configured Tor to act as a client and enabled the Socks proxy (allowing us to use a tool like tsocks or torify to force wget requests and the like over Tor)

The next thing to do, is to add some Hidden services - each needs to have it's own directory within the data directory (Tor will create the dir if it doesn't exist).

So in /etc/tor/torrc we might do the following

    HiddenServiceDir /var/lib/tor/ssh_hs/
    HiddenServicePort 22 127.0.0.1:22

Where the syntax of HiddenServicePort is

    HiddenServicePort [Listen to Port] [IP/Port to proxy connections to]

If we start Tor now, it'll generate a private key for each, a hash of which will give us the hostname

    service tor start
    cat /var/lib/tor/ssh_hs/hostname

Note: If at this point you receive 'Permission Denied', you may want to look at reconfiguring SELinux to work with Tor

If you want to back up the private key (if so, do it securely, that key is all that's required for someone to publish to the same address) it can be found in /var/lib/tor/ssh_hs/private_key

There are additional options that you might consider setting in torrc, see 'Other Considerations' below.

Within a few minutes, from a Tor connected client, you should be able to SSH to that hostname. Our next step is to configure our SSH client to make connecting a little easier

We can also now allow Tor to start at boot

  chkconfig tor on

 

Client SSH Config

This section assumes you'll be initiating SSH connections from a Linux based system - and it'll obviously need to have Tor installed and running. This section assumes Tor is configured (as by default) to expose a SOCKS proxy on port 9050

You'll also want Netcat (nc) to be installed

First, we generate a new keypair to use when authenticating with the server (you want to make sure you use a different key for every SSH hidden service - if one or more gets compromised, you don't want a comparison of authorized_keys to be able to show there's a common administrator)

    ssh-keygen -b 4096 -t rsa -C "myKey" -f ~/.ssh/sshhs1.rsa

You should set a password for the key, but it is technically optional A breakdown of the arguments used is

    -b 4096 - Keylength should be 4096 bits (4K)
    -t rsa  - Generate a RSA keypair (Assuming your server supports Eliptic
	      Curve, you could use ecdsa but there's no current security 
	      benefit)
    -C "myKey" - Set the comment to be 'myKey'. Normally this would be
		 $USER@$HOSTNAME which we want to avoid
    -f - Output file

Next you want to authorise that key, so (we'll look at what the arguments mean shortly)

    ssh-copy-id -o VerifyHostKeyDNS=no -o User=user -o CheckHostIP=no\
    -o ProxyCommand="nc -X 5 -x localhost:9050 %h %p" \
    -i ~/.ssh/sshhs1.rsa domain

Where user is the username you're using to SSH onto the server, and domain is your .onion

Once your key is copied across, you should be able to connect with the following

    ssh -o VerifyHostKeyDNS=no -o User=user -o CheckHostIP=no\
    -o IdentitiesOnly=yes \
    -o ProxyCommand="nc -X 5 -x localhost:9050 %h %p" \
    -i ~/.ssh/sshhs1.rsa domain

Whilst functional, this is obviously a nightmare to type and introduces a huge scope for mistakes. So the next thing to do is to populate your SSH configuration file with these settings Open ~/.ssh/config for editing and insert the following

    Host myOnion
      Hostname domain # This should be your .onion
      User user # Whatever username you connect with
      IdentityFile ~/.ssh/sshhs1.rsa
      ProxyCommand nc -X 5 -x localhost:9050 %h %p
      VerifyHostKeyDNS no
      CheckHostIP no
      IdentitiesOnly yes

The name specified for 'Host' is nothing more than a nickname to pass to SSH (so can be the full .onion if desired). Once you've saved the conf file, running

    ssh myOnion

will be the equivalent of the commands given above.

The arguments we're passing to SSH are

IdentityFile - 	 Specifies the key to use when authenticating

ProxyCommand - 	 Establish a connection to a proxy (using the specified command)
		 in order to then try and establish the SSH session. In this
		 case that means route over Tor

VerifyHostKeyDNS - By setting this to No, we're ensuring our SSH client will
		   _NOT_ make a DNS request to try and verify the received
		   host key. 

		   Failing to set this potentially gives you a major source 
		   of information leakage whenever you	attempt to connect to
		   the server.

CheckHostIP - 	 Don't perform a DNS lookup of the hostname (the Tor proxy will
		 do that for us and we don't want an observable DNS request 
		 going over the clearnet)

IdentitiesOnly - By default, SSH will offer up any keys it knows about to try
		 and authenticate. This is inefficient and introduces a (small)
		 risk that a compromise of the server could lead to you being 
		 identified. 

		 Setting IdentitiesOnly to yes tells SSH to only ever use the 
		 keys listed in IdentityFile.

 

Things to Remember

You should now be set to SSH via your hidden service address, and having copied a key to the server can safely disable Password authentication with SSH if you wish.

As was noted at the beginning, you need to ensure that any changes you make do not prevent you from gaining access to the server via alternative means. At times, you may need to restart the Tor client, which will cause your SSH session to drop, if for any reason Tor does not come back up, you need to have the means to connect and correct that.

Routine connections to the server should be made via the Hidden Service address to minimise the potential for an adversary to identify you as the administrator of the system. Using a hidden service (rather than transitting via a Tor exit node) also reduces an adversary's ability to identify the times at which an administrator connects via SSH (reducing the likelihood of Timezone identification etc).

 

Tor - Other Considerations

The configuration used for Tor above is fairly basic, and there may be other options that you want to consider introducing. For example, in the configuration above, the tor daemon will be permitted to use all CPU cores if needed. If for some reason, you wish to limit usage to a specific number of cores, you can add the following to torrc

    NumCPUs 2

On CentOS, Tor will automatically drop root privileges and run as user _tor. This is not always the case on other distributions, so you may need to create a user (say _tor) and include

    User _tor

If you have multiple NICs and want to ensure that Tor only uses one, you can tell Tor which IP to bind to, so in the following example

  • eth0 192.168.1.2
  • eth1 192.168.1.3

To ensure Tor only ever uses eth0 you'd enter the following in torrc

    OutboundBindAddress 192.168.1.2

If you're particularly concerned about Tor writing connection information to disk, you can set the following to reduce the likelihood of this occurring

    AvoidDiskWrites 1
    DisableAllSwap 1

This will minimise writes where possible and attempt to prevent memory from being paged to disk.

You can view the options you tor client supports by running

    tor --list-torrc-options

The Tor Project publish a manual on what each of these mean at https://www.torproject.org/docs/tor-manual.html.en

 

Conclusion

You should now have a server configured for SSH administration via a Tor hidden service, and a basic design of how many other hidden services you will need to set up.

The means by which you'll be anonymously administering the server should also now be in place with steps taken to minimise information leakage from both the client and server end.

 

Part 2 - HTTP and HTTPS Hidden Services