Jay Taylor's notes

back to listing index

Building account systems – Mike’s blog

[web search]
Original source (blog.plan99.net)
Tags: security best-practices blog.plan99.net
Clipped on: 2017-08-04

Image (Asset 1/15) alt=

If you aren’t quite ready to rely 100% on third party ID providers, at least do everyone a favour and don’t make users pick a password at all.

This isn’t as stupid as it sounds. You are already asking the user for their email address. The very first feature you will add to your login system after going live is forgotten password recovery, which will work by sending the user a clickable link via an email. Therefore anyone who can read your user’s email can log in as them anyway and your own site password adds no extra security.

Instead, skip the first step and go straight to the second — your login system can be as simple as emailing the user a link that sets a login cookie when clicked. Medium.com is an example of a site that does this.

This approach works as long as every device where the user might log in to your service has an email client. This is true for desktops, laptops, phones and tablets. It is not true for games consoles or TVs, but you probably aren’t targeting them yet. If you are it’s better to use a Bluetooth style pairing process anyway, as these devices don’t have convenient keyboards.

There have been suggestions in the past that users can be confused by the lack of a password entry box. But the modern Google sign-in experience starts by asking the user for their email address only, so it’s unlikely users are confused by this any more … and the benefits are huge.

This approach has an additional benefit: some users have phone numbers but not email addresses. This is especially true in developing countries, so if this is a possible target market for your website you may eventually want to support users who can only log in by receiving a code to their phone. Such accounts won’t have passwords at all, so if you assumed all users do have passwords it will require you to go back and add lots of special cases to security-sensitive code paths (this can easily lead to fatal mistakes).

4. Don’t use secret questions

If you simply must use passwords — perhaps you don’t want to try explaining to your boss why you did things differently — at the very least don’t let the user recover using secret questions and answers.

  • The answers to secret questions are often trivially guessed. Users find it incredibly hard to think of questions that only they could answer and nobody else.
  • Pre-supplied questions make the guessing problem worse.
  • Pre-supplied questions often have cultural bias that makes them useless for many users (e.g. “what was your high school’s mascot?”)
  • Some savvy users realise they can’t think of a hard-to-guess answer so just use it as a second password field, meaning they then can’t recover when they forget.
  • There is a long list of high profile celebrity and VIP hacks that worked by abusing password recovery flows. You don’t want this to be you.

Google had severe problems with secret questions. A couple of my old colleagues published research on it that’s worth reading or watching (video below)

Image (Asset 2/15) alt=
All these CAPTCHAs are uselessly weak

The thing to understand about CAPTCHAs is they’re only useful for imposing a very basic throttle on automated attacks. They will not protect your account system against bulk registrations. Other than account security I also spent some years working on Google signup abuse. We routinely saw spammers solve tens of millions of our hardest CAPTCHAs. There are professional CAPTCHA solving firms like DeathByCaptcha that use a mix of OCR and human solving. Ordinary CAPTCHAs block blind people from signing up, which may be a problem, but speech recognition based CAPTCHAs are either trivially solved by computers or unsolvable by humans.

CAPTCHAs are most useful for blocking password brute forcing attempts. Brute forces may require hundreds of thousands or millions of attempts against an account to find the right password. A simple way to stop them without annoying users is to start throwing CAPTCHAs if the user has had some recent failed login attempts. Even easy CAPTCHAs are enough to throw a small delay into a bot loop.

CAPTCHAs are much less useful for stopping bulk account registration. Building systems to detect and stop that is a whole other ball game; one I spent several years playing. To get a sense of how hard it can be, go to buyaccs.com and observe the huge variation in prices charged by underground account sellers. The higher prices are caused by better defence systems. Unless you’re one of the Big 5 you won’t be able to beat the work we did on account signup security — it’s just one more reason to outsource login to the major players.

If you still want to use CAPTCHAs, use reCAPTCHA and make sure your CAPTCHA is bound appropriately to avoid replay attacks. Don’t try to roll your own or use a kit you found on GitHub. Such CAPTCHAs are invariably solved by modern OCR and will accomplish nothing except reducing the success rate of customer signup.

6. Outsource 2-factor authentication

Image (Asset 3/15) alt=

Two-factor auth is a pretty common feature these days. Again, doing this well is hard, expensive and you do not want to implement it yourself.

  • SMS is unreliable, especially in some countries. Recovery codes will occasionally just not show up. You will eventually want to implement phone calls with speech synthesis as a result because phone calls are much more reliable, but now you need multi-lingual speech synthesis engines.
  • Doing lots of SMSs or phone calls is extremely expensive, even if you can negotiate good bulk discounts.
  • People lose access to their phone numbers all the time. If you rely on email addresses your password recovery flow can be pretty easy, but once solid 2FA is introduced your password recovery becomes the weakest point in the system. If you don’t upgrade it attackers will simply go around it. If you block it, then you will discover that …
  • 2FA can be abused by attackers who add it to accounts they phished or hacked. This is to prevent the real users from stealing the account back whilst the malicious activity is performed.
  • Phone numbers are vulnerable to porting attacks, so the trend is towards asking users to set up mobile apps or security keys. Implementing those is even more work, and of course both of those can be lost too, so you will ultimately still need some customer support flow to help them recover.
  • As you’re figuring out, 2FA adds a lot of manual customer support work because you can no longer just push users towards email or secret Q/A based recovery. That’s expensive.

Some of these problems are fundamental, but most of them are solved already by the big players who will pay the phone bills and customer support people on your behalf, for free!

Still, if you don’t want to use them, there are startups that will solve small parts of the 2FA puzzle for you.

7. Don’t force password changes

Troy already covered this just fine so I won’t repeat it here, except to say that this is really important. Don’t require users to change passwords just because it’s been a while.

  • Some users won’t make it through the process and you will bleed users.
  • Some users will be smarter than you and use tricks like changing their password (once, twice, three times) and then immediately changing it back to their old password, meaning you will end up wanting to store a history of recent passwords to prevent this behaviour. But I bet your first implementation won’t do this.
  • It doesn’t improve security anyway.

8. Don’t expire sessions

Yet another best practice that isn’t. It’s tempting to set your session cookies to expire. Sometimes people think that this improves security for the same reason they think expiring people’s passwords improves security.

  • Attackers tend to perform malicious activities immediately, so expirations don’t help much.
  • Session expiry trains users that random unexpected password prompts are normal, which makes them incredibly easy to phish.
  • Sessions that expire randomly create an explosion of bugs that your developers will waste large amounts of time on. Most parts of your website will not be written to handle the case of sessions expiring half way through an action, so you’ll have to go back and fix them, assuming you even notice the problems at all. Expiry tends to surface as user reports of random flakiness which are hard to track down.

9. Remember sign-out

Getting sign-out wrong is remarkably common in immature account systems. It sounds superficially easy but the most obvious ways to do it have flaws.

  • Simply deleting the session cookie is fine as a convenience to the user, but means you can’t recover from XSS. Once an XSS is found you may wish to invalidate possibly stolen session cookies, but if sign-out is just “ask the browser to delete the cookie” then you can’t do it.
  • Adding timestamps to session cookies and then setting a “last sign-out time” requires every action to check against the accounts database to discover if the user’s session is too old. This can slow things down, meaning developers will be tempted to optimise it out (it doesn’t seem dangerous to do so after all). But then if they remove the check for an endpoint of interest to attackers, you’ve still got the problem in step one. Additionally, this means signing out of one browser or device signs the user out of all of them, which isn’t expected behaviour.

The right way to do this is keeping a list of invalidated session cookies with in-memory caching. But for most companies, there’s a less costly approach which is good enough: have the user’s sign-out link be just a way to clear the session cookie and nothing more, then make session cookies expire but be automatically and silently replaced every 5 minutes or so. The act of replacing an expired session cookie consults the database to see if the administrators have forced a logout. If the user is presenting an expired cookie then they are required to log in again. This recognises that cleaning up after cookies may have been stolen is a relatively rare event.

10. Separate account emails from marketing mail

Image (Asset 4/15) alt=

The obvious way to send password recovery links, signup verifications etc is simply from your company’s main email server. Unfortunately, some people in your company are trying to build a “relationship” with the user by sending them commercial mails they don’t want.

Even if users agreed to receive these during account signup, many of them don’t want them anymore and some will solve this by reporting them as spam. This is an expedient solution for the savvy user who has noticed that simply clicking “Report spam” a couple of times makes the emails go away, without any mental effort expended on finding tiny light-grey-on-white-6pt-font unsubscribe links or … gasp … writing email filters.

Unfortunately, this entirely normal behaviour will start to degrade the reputation of your mail email domain. Mail from your account system may start going into the user’s spam folder. We’ve all seen warnings on signup or password recovery flows telling us to check our spam folders — that’s why.

One way to solve this is by buying a separate top-level domain to send your account mails from and making sure to configure DKIM. But then some users will notice the mismatch and report your email as phishing instead. The best solution is to send your marketing emails from a different DKIM domain, but that will likely involve picking a fight with your product folks. Still … the moment you chose to roll your own you accepted this pain, remember?

11. Keep your password database well protected

If you have passwords, you have a database attackers want (and frequently get). They don’t care about your company directly, they just want the passwords so they can try them at higher value targets. Yet data breaches are embarrassing and can lead to big penalties even if the direct impact on your customers is low. A database of OAuth tokens is of far less value to attackers and thus you’re much less likely to be attacked.


There is far more I could write about account systems. Defending your site against malicious account hacking/signup is an entire book all by itself. I can’t write that book but you can watch this video of a talk I gave in 2012 instead, if you’re curious.

But it’s fair to say the task is deceptively large. That’s why I keep recommending you bite the bullet and outsource your account management to the big boys. Fiddling with CAPTCHAs is not your core business. Writing design documents for “log out” is not your core business. Diagnosing why you’re bleeding users who forgot their password is not your core business. Diagnosing why SMS message delivery to Peru isn’t reliable isn’t your core business. Every dollar you spend on these things is a dollar your competitors who use “Sign in with …” are spending on their core business.

So throw out your password database, and don’t look back.

  • Image (Asset 5/15) alt=