Understanding Password Storage

I occasionally receive emails from people who have come across PHPCredlocker, and the question is usually the same - "Why are you storing passwords using reversible encryption?". Most emails are polite, some not so much, but they all have one thing in common - assuming that a commonly stated fact applies to all scenarios, and failing to apply a bit of simple logic that would tell them the answer - because that's the only way the system would work.

In this post, we'll be briefly looking at some of the ways in which you can store credentials, and which of them are appropriate to use (and when), in the context of building an application (web or otherwise).

I actually wrote a white-paper on this a good few years ago, but things have moved on considerably since then - Rainbow tables are no longer used, and it's now possible to cheaply use cloud services with GPUs available.

One thing that hasn't changed, however, is that you should always work on the basis that one day, somehow, an attacker will manage to get a copy of your database. At that point, how you've stored the credentials becomes very important (users re-use passwords, so it may not just be your system that gets compromised if the credentials are recovered).

 

Storing Passwords in Plaintext

In two words..... just don't.

There's a lot of legacy software out there which may be storing credentials in plaintext, but unfortunately some also continue to build software that does this.

The reasons it's a bad idea should be obvious, but to re-iterate: if someone gets a copy of your database, they've got all the credentials they need with no extra effort.

 

MD5 /  SHA1

This method entails generating an cryptographic hash of the user's password and then storing that in your database, and can take two forms - salted or unsalted. When you salt the password, you add a (known) random string to the password before generating the hash. Whenever a user tries to log in, you add that same salt to their submission before comparing the hashes.

Salting was considered a defence against Rainbow tables (essentially tables containing pre-generated hashes of possible passwords) as the same password will have different hashes if the salts differ.

Unsalted hashes remain vulnerable to Rainbow tables and therefore should be considered little (or in the case of MD5, no) better than storing in Plaintext.

However, in today's world, Salted SHA1/MD5 should be considered an unacceptable password storage mechanism for any new software. Both were designed to generate hashes quickly and inexpensively (in processing terms), and as a result an attacker can easily try to brute force the attacks (Using HashCat a Windows 7 machine with 1 GPU can attempt 2495,000,000 SHA1 hashes per second).

There's plenty of software still using Salted SHA1 or MD5 (and sadly a lot using unsalted), but the world is slowly moving on.

 

BCrypt

BCrypt is just one example of a range of hashing mechanisms designed with credential storage in mind. The mechanism requires more computational power to generate the hash, which in turn means that an attacker can make fewer attempts per second on the same hardware.

When creating your application, you can set the 'cost' of the hash, which directly controls how much processing is required to generate the hash. In a previous post, I found that setting the BCrypt cost to 11 led to hash generation taking 17x longer than a SHA1 hash of the same password.

As a minor bonus, there's no longer a need to store a separate salt with BCrypt either

Joomla has made strides in the right direction, having moved from Salted SHA1 to BCrypt.

If you're building or updating an application, then BCrypt or a similar mechanism is the method you want to be using to store authentication tokens.

 

Reversible Encryption

We now need to address the mis-understanding that leads to me receiving emails;

As a general rule of thumb, you should not use reversible encryption to store passwords in your database (i.e. you should be using a one-way hash such as those mentioned above). The principle being that if an attacker also gets hold of crypto key (which your application will need to (en|de)crypt the passwords then you are no better off than if you'd stored the passwords in plaintext.

This advice, however, doesn't apply to all storage of passwords. It largely relates to authentication tokens (i.e. your stored version of the password the user uses to log into your system).

A Password repository such as PHPCredLocker, has to use reversible encryption because you as the user want to be able to retrieve the plaintext version of any passwords you've added. It's not like you can authenticate with a system by submitting a hash of the correct password, after all.

The actual authentication tokens within CredLocker are stored using BCrypt (though the hash is then also encrypted using reversible encryption).

So, if you're designing an authentication system, no, you should never rely on reversible encryption to store/secure credentials.

However, it is appropriate to use it if there's a need to retrieve the plaintext version of the password at a later date (and 'I forgot my password' should never, ever be considered a suitable justification for this)

 

Conclusion

The way in which credentials should be handled has changed quite dramatically over the years, and not all developers have kept up with the times.

However, whenever considering password storage you should also apply a little logic to any solution and assess whether it's appropriate. Don't rely on 'accepted wisdom' to be a one-size-fits-all solution, and do your best to consider future developments.

The days of using Rainbow tables, and even dictionary attacks are gone. Attackers use known password lists and masks to make hash-cracking more efficient and once they've got database, your only defence is to make sure that cracking the hashes will require enough effort that it's just not worth it (the threshold for which will of course raise depending on the profile/size of your site, and the attackers personal feelings towards you).

It's an ongoing game of cat and mouse, and the best thing you can do is to make sure you stay ahead of as many other mice as possible.