Ask HN: Tools or mental model for answering subjective questions quantitatively?

5 points by jlelonm 6 years ago | 4 comments

Hello HN!

I'm interested in tricks, tools, or mental models to help answer subjective questions with accuracy and precision (and eliminate biases).

For example, how do you turn the following questions into insightful, actionable data?

- How much does this hurt?

- How much do you like this product?

- What are you feeling right now?

Backstory:

I'm 28 and I developed tinnitus out of nowhere a few months ago (I don't go to concerts, I don't sit in server rooms, etc).

It's "mild" to "moderate" right now, I'm terrified of it getting "worse" as I age. I'd like to track it over time (months, years) to see if it gets quantitatively worse.

The issue: it's really hard to

1) control for all environmental variables (sleep, exercise, diet, etc), and

2) answer the question "how bad is it right now?" with consistency (i.e. however bad it is today, I will also give that same response when it is also that bad n days from now)

At first, I recorded (twice daily) my answer to the question:

Scale 1-10 how bad is it right now?

But then I realized - what if it's just not there at some point? I could do mental gymnastics and normalize it so that 1 corresponds to not there but it seemed easier to add 0 to make it 0-10.

But then, when the question became

Scale 0-10 how bad is it right now?

I realized I had no visceral, high-fidelity understanding of what a 3 was, nor a 4, etc.

So I shrank it to 0-5

(I think I'm sacrificing granularity for precision?)

Now, it's easier to assign a number to a subjective experience, but it got me thinking:

This is interesting from a customer / product research perspective and from a scientific research prospective. How do you phrase survey questions to maximize accuracy and precision? Is there a tradeoff? I'd love resource recommendations for studying this idea.

sethammons 6 years ago |

For tinnitus specifically, you could make simulator to play audio of what your tinnitus sounds like. The you adjust the volume/decibels until it matches the level you are experiencing in your head. Now you can track actual measurements to see the relative loudness of your tinnitus over time. Just a thought.

jlelonm 6 years ago | |

Ooh! That's a really good idea, thanks!

tucaz 6 years ago |

I have pretty much no knowledge about statistics and surveys but it is my understanding that it doesn’t matter how much you try to quantify a subjective question it will always be a “snapshot” of people’s feelings without any objective way to verify it.

It also seems like this is not a problem and people will still use this to make decisions. Example: politic polls, president approval ratings, etc.

These are all subjective measurements that are somewhat actionable based on “feelings”.

For the pain scale it doesn’t seem like it makes sense to rate it at 1-10 rating and I think about that every time I pay a visit to the doctors office and they ask about that. My guess is that they are not after the objective number, but the subjective information they can get from the number you give + how you behave.

kleer001 6 years ago |

From what I understand the pain scale is used only relatively on a per patient basis. So a 5 for one person is their 5, but not a 5 for someone else.