Introduction to Logistic Regression
Logistic Regression
Introduction
Logistic regression is a variation of ordinary regression which is used when the dependent (response) variable is a dichotomous variable (i. e. it takes only two values, which usually represent the occurrence or non-occurrence of some outcome event, usually coded as 0 or 1) and the independent (input) variables are continuous, categorical, or both. For instance, in a medical study, the patient survives or dies.
Unlike ordinary linear regression, logistic regression does not assume that the relationship between the independent variables and the dependent variable is a linear one. Nor does it assume that the dependent variable or the error terms are distributed normally.
The form of the model is
http://pnu-club.com/imported/mising.jpg
where p is the probability that Y=1 and X1, X2,.. .,Xk are the independent variables (predictors). b0 , b1, b2, .... bk are known as the regression coefficients, which have to be estimated from the data. Logistic regression estimates the probability of a certain event occurring.
Logistic regression thus forms a predictor variable (log (p/(1-p)) which is a linear combination of the explanatory variables. The values of this predictor variable are then transformed into probabilities by a logistic function. Such a function has the shape of an S. On the horizontal axis we have the values of the predictor variable, and on the vertical axis we have the probabilities.
Logistic regression also produces Odds Ratios (O.R.) associated with each predictor value. The "odds" of an event is defined as the probability of the outcome event occurring divided by the probability of the event not occurring. In general, the "odds ratio" is one set of odds divided by another. The odds ratio for a predictor is defined as the relative amount by which the odds of the outcome increase (O.R. greater than 1.0) or decrease (O.R. less than 1.0) when the value of the predictor variable is increased by 1.0 units. In other words, (odds for PV+1)/(odds for PV) where PV is the value of the predictor variable.
