Logistic Regression from Bayes’ Theorem(countbayesie.com) |
Logistic Regression from Bayes’ Theorem(countbayesie.com) |
In reality the temperature is required to be a certain exact value within a range.
If you start adding data points for very high temperatures, then your linear model will probably end up ignoring the temperature.
You would need a non-linear transformation of the temperature that can capture that the further away from the sweet spot the worse the coffee is.
You can relate to NNs for free since it is a linear layer with sigmoid activation.
You can stress it is linear in that your decision boundary is linear.
I don't like how capitalized letters are not random variables but are observations.
You can give some examples of what conditional PDFs P(H=1 | D ) look like and what you can model. In your case if the ideal temp for coffee is 190F and +/- 10 or more and the coffee is bad then you hope that (temp - 190)^2 is a feature input.
Congrats on the book deal!
Let p_k be the probability of being in class k. We assume log p_k = f_k(x) + C(x) where x is the feature vector and C(x) is normalisation to make the probabilities sum to 1.
Equivalently, p_k is proportional to exp(f_k(x)), so p_k = exp(f_k(x)) / sum_j exp(f_j(x)).
Because of the normalisation we may assume without loss of generality that f_0(x) = 0. Then if we have 2 classes and f_1(x) is linear, we get logistic regression.
1. Is D a binary random variable? If so, what exactly does it mean to say beta*D + beta_0 is an approximation for log odds? Doesn't this formula only take on 2 possible values?
2. Could you provide intuition for why a linear function of D would be a good approximation for the log odds mentioned?
As for your second point, there is no prior reason in this case why a linear function of D would be a good approximation. Indeed in the current case, we would probably prefer to at least write beta1D+beta2D^2+beta0 which is still linear in a transformation of D.
That being said, however, there is a notion on why a linear function may be a good approach. If you are interested in the direction of change around the averages values of the variables involved, then a linear function gives you such a "linear approximation" of the slope. This of course quickly breaks down if the function is not really linear, and in particular, it breaks down if you are interested in predicting an observation that is not "average". But often, one may be interested in such very qualitative statements as: on average, the coffee is improved by fresher beans - yes or no? In that case, such a linear model may give an answer.
Note that the above is absolutely not formally correct.
Finally, logistic regression can also be motivated differently.
- It arises from minimizing certain entropy losses in Machine Learning
- One may assume that the binary variable we observe is really just based on a "latent" variable (here something like coffee quality), which is determined by such a linear model
- finally, in economics and reinforcement learning, we assume agents make one decision (here whether the coffee is good or bad) by judging the inputs plus some random "error" or "taste" parameter which is has extrem value distribution. Since only the differences between these utilities matter (cf. odd ratios), and the actual values are meaningless, the logistic regression also arises.
D is a vector of input data. If it's more than a single number, then beta D needs to be interpreted as a dot product.
There's no guarantee that in any specific case a linear function of D will be a good approximation to the log odds. (In the present instance, where D is the temperature, it won't be -- there'll be a narrow range of good temperatures and the further away you get from that range, the worse the coffee is likely to be.)
But a linear approximation is at least simple and log odds (unlike e.g. probability) at least can take any value from -oo to +oo. Sometimes you get lucky.
The example is probably not the very best, but P(D) may make more sense if you think of the following:
If D equals the amount of coffee I put in the grinder, then D has a certain random component. Sometimes I put in more, sometimes less - even though I aim at a specific level. This is why it is important to have a concept of P(D) in Bayes' equation. The one case where I inadvertently put in a lot of coffee should not be used for "strong evidence" - is the idea here.
Sorry about that.
That "Drop the trailing s to form possessives" exception needs to die.
At least that's how I've learned it as a non-native speaker.
Don't take it too literally - P(..) here is not some well defined function, it's effectively just part of the name, as a convention for naming probabilities. I find it confusing too.
As the article points out, that set is for all intents and purposes infinite in this case, but this doesn't matter, as you can sidestep it by comparing to complementary hypotheses (which makes P(D) cancel out). This is all covered in the article.
The only maths worth reading up on to understand this article is a basic introduction to Bayes theorem - the wikipedia page is quite decent.
You kinda have to expect a student to use the examples you give.
There’s really no upside to using that example.
Instead, why not simplify it down to D=Freshness of Coffee Beans ?
In my opinion it is just better to keep the s as that matches how people will pronounce it and regularization of the language is better. Plus the misuse of the rule makes it even worse.
Not that I don't sympathize. I think that "it's" should be the possessive form of 'it', not a contraction of 'it is'. Maybe if we band together we can force-feed both our changes at once to the rest of the world?