This is a very simple sample problem from ISI MStat PSB 2018 Problem 9. It is mainly based on estimation of ordinary least square estimates and Likelihood estimates of regression parameters. Try it!
Suppose \((y_i,x_i)\) satisfies the regression model,
\( y_i= \alpha + \beta x_i + \epsilon_i \) for \(i=1,2,....,n.\)
where \({ x_i : 1 \le i \le n }\) are fixed constants and \({ \epsilon_i : 1 \le i \le n}\) are i.i.d. \(N(0, \sigma^2)\) errors, where \(\alpha, \beta \) and \(\sigma^2 (>0)\) are unknown parameters.
(a) Let \(\tilde{\alpha}\) denote the least squares estimate of \(\alpha\) obtained assuming \(\beta=5\). Find the mean squared error (MSE) of \(\tilde{\alpha}\) in terms of model parameters.
(b) Obtain the maximum likelihood estimator of this MSE.
Normal Distribution
Ordinary Least Square Estimates
Maximum Likelihood Estimates
These problem is simple enough,
for the given model, \( y_i= \alpha + \beta x_i + \epsilon_i \) for \( i=1,....,n\).
The scenario is even simpler here since, it is given that \(\beta=5\) , so our model reduces to,
\(y_i= \alpha + 5x_i + \epsilon_i \), where \( \epsilon_i \sim N(0, \sigma^2)\) and \(\epsilon_i \)'s are i.i.d.
now we know that the Ordinary Least Square (OLS) estimate of \(\alpha\) is
\( \tilde{\alpha} = \bar{y} - \tilde{\beta}\bar{x} \) (How ??) where \(\tilde{\beta}\) is the (generally) the OLS estimate of \(\beta\), but here \(\beta=5\) is known, so,
\(\tilde{\alpha}= \bar{y} - 5\bar{x} \) again,
\(E(\tilde{\alpha})=E( \bar{y}-5\bar{x})=alpha-(\beta-5)\bar{x}\), hence \( \tilde{\alpha} \) is a biased estimator for \(\alpha\) with \(Bias_{\alpha}(\tilde{\alpha})= (\beta-5)\bar{x}\).
So, the Mean Squared Error, MSE of \(\tilde{\alpha}\) is,
\(MSE_{\alpha}(\tilde{\alpha})= E(\tilde{\alpha} - \alpha)^2=Var(\tilde{\alpha}) \) + \({Bias^2}_{\alpha}(\tilde{\alpha}) \)
\(= frac{\sigma^2}{n}+ \bar{x}^2(\beta-5)^2 \)
[ as, it follows clearly from the model, \( y_i \sim N( \alpha +\beta x_i , \sigma^2)\) and \(x_i\)'s are non-stochastic ] .
(b) the last part follows directly from the, the note I provided at the end of part (a),
that is, \(y_i \sim N( \alpha + \beta x_i , \sigma^2 ) \) and we have to find the Maximum Likelihood Estimator of \(\sigma^2\) and \(\beta\) and then use the inavriant property of MLE. ( in the MSE obtained in (a)). In leave it as an Exercise !! Finish it Yourself !
Suppose you don't know the value of \(\beta\) even, What will be the MSE of \(\tilde{\alpha}\) in that case ?
Also, find the OLS estimate of \(\beta\) and you already have done it for \(\alpha\), so now find the MLEs of all \(\alpha\) and \(\beta\). Are the OLS estimates are identical to the MLEs you obtained ? Which assumption induces this coincidence ?? What do you think !!
Publishing original scientific research in a peer-reviewed journal is an achievement many scholars work toward for years. Achieving this feat early in one's academic journey requires exceptional intellectual rigor, creativity, and persistence. Aaditya Punatar and Muskan Sadana have reached this remarkable milestone, successfully publishing their collaborative research paper in the prestigious Journal of Emerging Investigators […]
Her project on quantum-accelerated Steiner trees for 15-minute cities earned a place among the finalists at the world's largest pre-college STEM competition. Prishaa Shrimali of Marymount School of New York, was selected as a Grand Finalist at the 2026 Regeneron International Science and Engineering Fair (ISEF) in Phoenix, Arizona. Her project was Optimizing Urban Accessibility: […]
In 2026, the following Cheenta students have been successful for Indian Statistical Institute's M.Stat Entrance. They ranked within the first 50 in the entire country in these entrances. I.S.I. M.Stat Entrance
In 2026, the following Cheenta students have been successful for Indian Statistical Institute's B.Stat Entrance and Chennai Mathematical Institute's B.Sc. Math Entrance. They ranked within the first 200 in the entire country in these entrances. Most of these students attended the problem solving workshops regularly, which happen 5 days every week. CMI B.Sc. Math Entrance […]
