how to measure a problem

When I evaluate new companies — be it to join, to advise, to invest in — I primarily asses two areas:

  1. The people, and

  2. The problem.

This isn’t too different from how a VC firm or investor might evaluate a new startup, but it is distinct in one way: Most commonly, standard practice is to evaluate the people and the product.

Instead, I choose to focus on the problem. I view the problem as the root the product, and the product as the side effect of the problem.

Products, certainly, are easier to measure. And, in fact, measuring a product in some of the standard ways — such as daily active users, churn rate, etc. — is often a direct reflection of the problem, a sort of after-effect, delayed way of measuring a problem.

But I believe problems are more important to measure for a few reasons:

  • First, it can occasionally be easy, these days, to pay for users or to force a semblance of product market fit. This leads to distorted perception of the product. By looking at a problem before the product (solution), you have a bit more teeth to see through the signal to noise ratio.

  • Furthermore, evaluating problems allow you to envision a world of many solutions, as opposed to the one solution (problem), and force you to think through why one might be the best option over others. When startups “pivot,” they often are changing their solution (product) to the same core problem. (1) So, why not evaluate the core problem from the get-go?

  • Finally, you can think a bit bigger when focused on the problem: How could certain solutions (products) lead to flywheel effects? How could this same problem apply elsewhere, perhaps to a different subset of users or industries? (footnote) Are there variations of this problem that exist? This allows you to not silo yourself into thinking solely about one product, or interface, or solution to the problem, but rather the problem in and of itself — and again, envision a better world where this problem is solved, as opposed to envisioning the solution itself.

Below is a version of the framework I use for evaluating problems.

The Basics

  • Size of the problem. How many people does it affect, and how deeply? How frequently does the problem occur? This is related to market size, but it’s bigger than just that. (2) You should be measuring both frequency and size of the problem, and viewing it as a ratio. To measure this problem effectively, you’ll need both extensive market research and user research, as opposed to strictly market research often needed for addressable market and penetration.

  • Hacks, or existing solutions to the problem. This is similar to competition, but it is not to be confused with simply competitors, as it is bigger than that. The existing solution that solves your problem might be another young startup, or a large corporation you’re wanting to disrupt, but it also might be a paperclip or a low-tech solution workaround that works just fine. The bread machine and pill slicer. Sometimes this can be a beautiful entry into the market to solve problems people didn’t even know they had, but also that means they might not pay for it. You’ll notice that this can easily get into the world of solutionizing, so be careful! But is the problem a true problem that people are facing, or is it no longer a problem because they have existing solutions to remedy the problem?

  • Riskiest risks, or confidence interval. What are the biggest hurdles in this problem? Is it a problem in a highly regulated industry, in an industry that doesn’t welcome new players, in an industry where the cost of solving the problem would be too high? What scares you most about this problem? Evaluate it, test those hypotheses right away. If there aren’t many, you’re good. Alternatively… see point three below. These might end up being benefits.

Problem Differentiators

  • Exponentiality of a problem. Does this problem serve as a wedge into future markets? I think of this as new version of the Hooked framework. The classic example is Amazon starting as a bookstore, and using this as a hook to acquire a universal set of online shoppers, before using a variety of growth tactics to cleverly expand into additional product categories.

  • Expandability of a problem. Can this problems apply to other similar or parallel problems out there, perhaps in another industry? Some of my favorite (more frivolous, more serendipitous, and less strategic, for sure) examples of this in history are the frisbee, first sold as pie containers, and play-doh, first sold as a wall cleaner.

  • Unique market hooks creating an unfair advantage. Are there regulations in a market that enforce stagnant supply or demand?

Ethics and Mission

  • Potential for future harm. Dig into the uncomfortable: Is there any possible way that solving this problem could actively cause harm? This does not always mean that the problem should be left unsolved, but it does certainly mean that those considerations need to be addressed from the beginning. I believe that anticipating harm beforehand is something the Tech industry at large has failed to do consistently in the past, and something that is incredibly valuable to do.

  • Mission alignment. Does solving this problem align with my core values? Do I see this problem as more important or urgent, from a moral and ethical standpoint, than other problems out there?

I’m constantly iterating on the best ways to measure a problem, and believe that in and of itself is one of the most interesting problems out there.

Next time you’re evaluating a company to join, a company to found, or a new product or large feature to build, take a stab at using this framework (or apply it retroactively to products or companies out there) and let me know what you think is missing, or what did and didn’t work.




  1. One example of this is Netflix. They began with a mail ordering product, delivering DVDs to people’s mailboxes. Later, they shifted to the all-digital product we know today. But both pre-pivot and post-pivot, Netflix focused on the user’s core problem of wanting smoother access to newly released media. ↩︎

  2. This is also related to the theory of choosing frequently recurring problems that are smaller (i.e. Uber) or less frequently recurring problems that are bigger (i.e. Airbnb). ↩︎