1.1 What Is Econometrics?
Definition: Econometrics is the application of statistical and mathematical methods to economic data for the purpose of testing hypotheses, forecasting, and informing policy decisions. It bridges the gap between economic theory and empirical observation.
The Econometric Process:
Economic Theory → Mathematical Model → Econometric Model
↓ ↓ ↓
Data Collection → Estimation → Hypothesis Testing
↓ ↓ ↓
Forecasting → Policy Recommendation → Evaluation
| Concept | Meaning | Example |
|---|
| Economic Model | Theoretical representation | Supply = Demand |
| Econometric Model | Empirical counterpart with stochastic error | Q = α + βP + ε |
| Estimation | Using data to quantify relationships | β̂ = -1.2 |
| Inference | Drawing conclusions about populations from samples | p < 0.05 |
1.2 The Linear Regression Model
The Population Model:
Yᵢ = β₀ + β₁X₁ᵢ + β₂X₂ᵢ + ... + βₖXₖᵢ + εᵢ
Where:
- Yᵢ = Dependent variable (response, outcome) for observation i
- Xⱼᵢ = Independent variables (regressors, predictors) for observation i
- βⱼ = Population parameters (unknown, to be estimated)
- εᵢ = Error term (unobserved factors affecting Y)
The Sample Model (Estimated):
Ŷᵢ = b₀ + b₁X₁ᵢ + b₂X₂ᵢ + ... + bₖXₖᵢ
Where bⱼ are the estimated coefficients from the sample.
Matrix Notation:
Y = Xβ + ε
| Term | Symbol | Definition |
|---|
| Dependent Variable | Y | The variable being explained |
| Independent Variable | X | The variables doing the explaining |
| Coefficient | βⱼ | The effect of Xⱼ on Y, holding others constant |
| Intercept | β₀ | The expected value of Y when all X = 0 |
| Error Term | εᵢ | The difference between actual and predicted Y |
| Residual | eᵢ | Estimated error: eᵢ = Yᵢ − Ŷᵢ |
1.3 The Ordinary Least Squares (OLS) Estimator
The OLS Principle:
Minimize: Σ(Yᵢ − Ŷᵢ)² = Σ(Yᵢ − b₀ − b₁X₁ᵢ − ... − bₖXₖᵢ)²
Simple Regression (k=1):
b₁ = Σ(Xᵢ − X̄)(Yᵢ − Ȳ) / Σ(Xᵢ − X̄)²
b₀ = Ȳ − b₁X̄
Multiple Regression (k>1):
b = (X'X)⁻¹X'Y
| Property | Description | Expression |
|---|
| Unbiased | On average, correct | E(b) = β |
| Consistent | Correct in large samples | plim(b) = β |
| Efficient | Minimum variance among unbiased estimators | Var(b) ≤ Var(b̃) |
| BLUE | Best Linear Unbiased Estimator | Gauss-Markov Theorem |
Gauss-Markov Theorem:
Under the CLRM assumptions (1-5), OLS is the Best Linear Unbiased Estimator (BLUE) — it has the minimum variance among all linear unbiased estimators.
1.4 The Classical Linear Regression Model (CLRM) Assumptions
| # | Assumption | Description | Violation → |
|---|
| 1 | Linearity | Model is linear in parameters | Non-linear models needed |
| 2 | Zero Mean Error | E(εᵢ) = 0 for all i | Biased intercept |
| 3 | No Autocorrelation | Cov(εᵢ, εⱼ) = 0 for i≠j | Autocorrelation → DW test |
| 4 | Homoskedasticity | Var(εᵢ) = σ² (constant) | Heteroskedasticity → BP test |
| 5 | No Perfect Collinearity | No X is a linear combination of others | Singular matrix → VIF |
| 6 | Normality | εᵢ ~ N(0, σ²) | Non-normal errors → JB test |
Consequences of Violations:
| Violation | Problem | Detection | Solution |
|---|
| Heteroskedasticity | Inefficient SEs, invalid inference | Breusch-Pagan, White | Robust SEs, WLS |
| Autocorrelation | Inefficient SEs, invalid inference | Durbin-Watson | Newey-West HAC |
| Multicollinearity | Imprecise estimates, high SEs | VIF | Drop variables, Ridge |
| Non-normality | Invalid inference in small samples | Jarque-Bera | Large n, Bootstrap |
| Endogeneity | Biased and inconsistent estimates | Hausman test | IV/2SLS (Chapter 2) |
1.5 Goodness of Fit Measures
R-Squared (Coefficient of Determination):
R² = SSR / SST = 1 − SSE / SST
- Interpretation: The proportion of variation in Y explained by the model
- Range: 0 to 1 (higher = better fit)
- ⚠️ Warning: R² increases with more variables (even irrelevant ones!)
Adjusted R-Squared:
R̄² = 1 − (1 − R²) × (n-1)/(n-k)
- Penalizes adding irrelevant variables
- Preferred for model comparison
Components of Variation:
SST = SSR + SSE
| Component | Formula | Meaning |
|---|
| SST | Σ(Yᵢ − Ȳ)² | Total variation in Y |
| SSR | Σ(Ŷᵢ − Ȳ)² | Variation explained by the model |
| SSE | Σ(Yᵢ − Ŷᵢ)² | Unexplained variation |
| Measure | Formula | Use |
|---|
| MSE | SSE/(n-k) | Mean Squared Error (error variance estimate) |
| RMSE | √MSE | Root Mean Squared Error (in Y units) |
| σ̂² | SSE/(n-k) | Estimated error variance |
1.6 Hypothesis Testing
Individual Coefficient Tests (t-test):
H₀: βⱼ = βⱼ₀ (often βⱼ = 0)
H₁: βⱼ ≠ βⱼ₀
t = (bⱼ − βⱼ₀) / SE(bⱼ)
Decision Rules:
- Critical Value Method: Reject H₀ if |t| > t_critical(α/2, n-k)
- p-Value Method: Reject H₀ if p-value < α
Joint Significance Test (F-test):
H₀: β₁ = β₂ = ... = βₖ = 0
H₁: At least one βⱼ ≠ 0
F = (SSR/k) / (SSE/(n-k-1))
Significance Stars Guide:
| p-value | Interpretation | Symbol |
|---|
| p < 0.01 | Strong evidence against H₀ | *** |
| p < 0.05 | Moderate evidence against H₀ | ** |
| p < 0.10 | Weak evidence against H₀ | * |
| p ≥ 0.10 | No evidence against H₀ | — |
Confidence Intervals:
P(bⱼ − t_critical×SE ≤ βⱼ ≤ bⱼ + t_critical×SE) = 1−α
1.7 Diagnostic Tests Reference
VIF — Variance Inflation Factor (Multicollinearity)
VIFⱼ = 1/(1 − Rⱼ²)
| VIF | Severity | Action |
|---|
| 1-5 | None | Proceed with caution |
| 5-10 | Moderate | Consider dropping variables |
| >10 | Severe | Drop variables or use Ridge regression |
Breusch-Pagan (BP) Test — Heteroskedasticity
Procedure:
1. Run OLS and obtain residuals eᵢ
2. Regress eᵢ² on all X variables
3. Compute LM = n × R²_aux
| p-value | Decision |
|---|
| p < 0.01 | STRONG heteroskedasticity |
| p < 0.05 | Heteroskedasticity detected |
| p ≥ 0.10 | Homoskedasticity (OK) |
Durbin-Watson (DW) Test — Autocorrelation
DW = Σ(eᵢ − eᵢ₋₁)² / Σeᵢ²
| DW Range | Meaning | Action |
|---|
| 1.5 - 2.5 | No autocorrelation | Proceed |
| < 1.5 | Positive autocorrelation | Use HAC SEs |
| > 2.5 | Negative autocorrelation | Use HAC SEs |
⚠️ Important: Only meaningful for time-ordered data!
Jarque-Bera (JB) Test — Normality
JB = n[Skew²/6 + (Kurtosis−3)²/24]
| p-value | Decision |
|---|
| p < 0.01 | STRONG non-normality |
| p < 0.05 | Non-normality detected |
| p ≥ 0.10 | Normal (OK) |
Newey-West HAC — Robust Standard Errors
- What it does: Corrects SEs for autocorrelation AND heteroskedasticity
- When to use: When DW or BP tests are significant
- Formula: Bandwidth m = ⌊4(n/100)^(2/9)⌋
1.8 Model Selection and Specification
AIC — Akaike Information Criterion
AIC = n·ln(SSE/n) + 2k
- Use: Compare models with different numbers of variables
- Rule: Lower AIC = better model
- Penalty: 2k (moderate penalty for parameters)
- Best for: Prediction (asymptotically efficient)
BIC — Bayesian Information Criterion
BIC = n·ln(SSE/n) + k·ln(n)
- Use: Model selection with larger penalty for parameters
- Rule: Lower BIC = better model
- Penalty: k·ln(n) (heavier penalty)
- Best for: Model identification (consistent)
| Aspect | AIC | BIC |
|---|
| Penalty per parameter | 2 | ln(n) |
| Asymptotic property | Efficient | Consistent |
| Best for | Prediction | Model identification |
| Large n | May overfit | More parsimonious |
Stepwise Selection Methods:
| Method | Starting Point | Process | Use Case |
|---|
| Forward | Intercept only | Add variables one by one | Few candidate variables |
| Backward | All variables | Remove variables one by one | Many candidate variables |
| Stepwise | Both | Forward + Backward combination | General search |
1.9 Prediction and Forecasting
Point Prediction:
Ŷ₀ = X₀'b
Prediction Interval (95%):
Ŷ₀ ± t_critical × SE_pred
SE_pred = σ̂ × √(1 + X₀'(X'X)⁻¹X₀)
Confidence Interval for Mean Response:
Ŷ₀ ± t_critical × SE_mean
SE_mean = σ̂ × √(X₀'(X'X)⁻¹X₀)
| Interval Type | Width | Covers |
|---|
| Prediction | Wider | A new observation |
| Confidence | Narrower | The mean response |
1.10 Advanced Topics Preview
Instrumental Variables (IV/2SLS)
- When: When X is endogenous (correlated with error term)
- Why: OLS is biased and inconsistent when X is endogenous
- Solution: Use instruments Z that are:
- Relevant: Correlated with X
- Exogenous: Uncorrelated with ε
- Two-Stage Process:
- Stage 1: X = Zπ + v → Get X̂
- Stage 2: Y = X̂β + ε → Get consistent β
- Covered in detail: Chapter 2
Panel Data Models:
| Model | When to Use | Key Feature |
|---|
| Fixed Effects (FE) | Entity effects correlated with X | Entity-specific intercepts |
| Random Effects (RE) | Entity effects uncorrelated with X | GLS estimation |
| Hausman Test | Test FE vs RE | If p < 0.05 → FE |
Binary Choice Models:
| Model | Distribution | Link Function |
|---|
| Logit | Logistic | log(p/(1-p)) = Xβ |
| Probit | Normal | Φ⁻¹(p) = Xβ |
Difference-in-Differences (DiD):
Y = β₀ + β₁·Treat + β₂·Post + β₃·(Treat×Post) + ε
- β₃ = Causal effect of treatment
- Requires: Parallel trends assumption
1.11 Practical Guidance
Common Mistakes to Avoid:
| Mistake | Why It's Wrong | What To Do Instead |
|---|
| Adding irrelevant variables | Increases R² artificially, overfits | Use AIC/BIC or economic theory |
| Ignoring diagnostics | Invalid inference, biased SEs | Always run VIF, BP, DW, JB tests |
| Overinterpreting R² | R² doesn't imply causality | Focus on coefficients and significance |
| Running DW on cross-sections | Meaningless result | Use BP for heteroskedasticity |
| Treating stepwise as confirmatory | Data-driven, may not replicate | Validate with economic theory |
Best Practice Checklist:
- Check economic theory makes sense
- Ensure data quality (no missing values)
- Run OLS and examine coefficients
- Check VIF for multicollinearity
- Check BP for heteroskedasticity
- Check DW (if time series) for autocorrelation
- Check JB for normality
- Use HAC SEs if needed
- Compare AIC/BIC for model selection
- Interpret coefficients in context
- Report results with significance stars
Reporting Results:
| Variable | Coef | Std Error | t-stat | p-value | Sig. |
|---|
| Rainfall | 45.67 | 12.34 | 3.70 | 0.002 | *** |
| Area | -0.57 | 0.12 | -4.60 | 0.000 | *** |
| Intercept | 124,567 | 45,678 | 2.73 | 0.016 | ** |
R² = 0.9877 | Adj R² = 0.9857 | F = 456.79 (p = 0.0000)
N = 18 | k = 3 | MSE = 97,942
1.12 Key Concepts Glossary
Adjusted R² — R² penalized for number of predictors
AIC — Akaike Information Criterion
Autocorrelation — Correlation of errors across observations
BIC — Bayesian Information Criterion
BLUE — Best Linear Unbiased Estimator
CLRM — Classical Linear Regression Model
Coefficient — Effect of Xⱼ on Y
Consistent — Estimates converge to true value as n→∞
df — Degrees of freedom (n-k)
Efficient — Minimum variance among unbiased estimators
Error Term — Difference between actual and expected Y
F-statistic — Test for joint significance of coefficients
Heteroskedasticity — Non-constant error variance
Homoskedasticity — Constant error variance
Intercept — Expected Y when all X=0
k — Number of parameters (including intercept)
MSE — Mean Squared Error
Multicollinearity — High correlation among predictors
n — Number of observations
OLS — Ordinary Least Squares
p-value — Probability of observing test statistic if H₀ is true
R² — Coefficient of determination
Residual — Yᵢ − Ŷᵢ
SSE — Sum of Squared Errors
SSR — Sum of Squared Regression
SST — Sum of Squares Total
t-statistic — Test for individual coefficient = 0
Unbiased — Expected value equals true value
VIF — Variance Inflation Factor
1.13 Quick Reference Cards
OLS Formula Summary:
Simple Regression:
b₁ = Σ(Xᵢ−X̄)(Yᵢ−Ȳ) / Σ(Xᵢ−X̄)²
b₀ = Ȳ − b₁X̄
Multiple Regression:
b = (X'X)⁻¹X'Y
Residuals:
eᵢ = Yᵢ − Ŷᵢ
Ŷᵢ = Xᵢ'b
Variance-Covariance Matrix:
Var(b) = σ²(X'X)⁻¹
Hypothesis Testing Summary:
Individual Coefficient (t-test):
t = (bⱼ − βⱼ₀) / SE(bⱼ)
Critical value: t(α/2, n-k)
Joint Significance (F-test):
F = (SSR/k) / (SSE/(n-k-1))
Critical value: F(α, k, n-k-1)
Confidence Interval:
bⱼ ± t_critical × SE(bⱼ)
Diagnostic Test Summary:
VIF = 1/(1 − Rⱼ²)
BP = n × R²_aux
DW = Σ(eᵢ−eᵢ₋₁)² / Σeᵢ²
JB = n[Skew²/6 + (Kurtosis−3)²/24]
HAC = Newey-West estimator
1.14 Chapter Summary
Key Takeaways:
- OLS is the workhorse of econometrics — understand its assumptions and properties
- Always run diagnostics — VIF, BP, DW, JB tests are essential
- R² is not the goal — Economic significance > Statistical significance
- Model selection matters — Use AIC/BIC and economic theory
- Know your data — Cross-section vs Time Series vs Panel
- Diagnostic tests are your friends — They tell you when OLS fails
- Advanced methods build on OLS — Learn the basics first!
Checklist Before Proceeding:
- Understand OLS and its assumptions
- Can interpret coefficients and significance
- Know the diagnostic tests and when to use them
- Understand the difference between AIC and BIC
- Know what to do when assumptions are violated
- Ready to learn Instrumental Variables (Chapter 2)
1.15 What's Next?
📘 Chapter 2: Instrumental Variables (IV/2SLS)
In the next chapter, we will cover:
- The Endogeneity Problem — What is it? When does it occur?
- Instrumental Variables — What makes a good instrument?
- Two-Stage Least Squares (2SLS) — How it works
- Testing for Endogeneity — Hausman-Wu test
- Testing Instruments — Relevance and validity
- Overidentification Tests — Sargan-Hansen J-test
- Weak Instruments — F-statistic diagnostics
- Applications — Real-world examples