Have I Tried Enough Weird Stuff?

I was testing a piece of software that collected a person’s addresses for shipping within the United States. My developer had tried zip codes in the direct vicinity of our office in Manhattan, which all started with 1. I tried the zip codes for my hometown in New Jersey and the college I attended in Maine, both of which started with 0. Together we determined that the zip codes (and other address fields) needed to be stored differently so leading zeros would not be cut off. But it got me thinking: what other things might occur that were outside the direct experience of me and my developer?

So I asked the internet.

That’s when I first came across Falsehoods programmers believe about addresses. We were constrained to collecting American shipping addresses, so things like “are the odd street numbers all on the same side?” weren’t our concern. But plenty of them were. Was our form going to allow people whose shipping address was any of these?

  • a post office box
  • outside one of the fifty states (Washington D.C., Puerto Rico, Guam, etc.)
  • on an American military base
  • a fractional number

As I tested inputs on other applications, I kept wondering if I was only thinking of things I already knew about, or if the problem space was bigger than I could conceive. I’ve come across a few lists that I love to review with my developers before they start building an input field (or an API parameter) so we can agree on what kind of validation we’re going to do.

The Test Heuristics Cheat Sheet provides a great jumping-off point specific inputs for text fields on the first page and different ways to try inputting them on the second page.

The Big List of Naughty Strings collects different kinds of characters (languages with non-Roman characters, emojis, Javascript that might trigger script injection, etc.) in one place so I don’t have to search for each of these cases individually. I usually copy-paste the ones we’ve agreed we want to support from here. [Note: I recommend bookmarking this repository so you’re not accidentally getting NSFW results after searching “naughty strings.”]

Searching for “Falsehoods programmers believe about {input type}” is my go-to for more specific types of inputs. There’s a list of a bunch of them, but these are some of my favorites:

I encourage you to keep asking “have I tried enough weird stuff?” and deciding together with your developers what constitutes “weird.”

Thanks to Trish Khoo and Anne-Marie Charrett for the impetus to publish this, and Joep Schuurkes for pointing out that my headline falls under Betteridge’s law.

Originally published on Medium.