Theory of ridge regression estimators with applications. (2016)

• Record Type: Book
• Further Information: Note: A.K.Md. Ehsanes Saleh, Golam Kibria.
• Authors: Saleh, A. K. Md. Ehsanes; Kibria, Golam
• Contents: List of Figures xvii List of Tables xxi Preface xxvii Acronyms xxxi List of Symbols xxxiii 1 Introduction to Ridge Regression 1 1.1 Introduction 1 1.1.1 Multicollinearity Problem 3 1.2 Ridge Regression Estimator: Ridge Notion 5 1.3 LSE vs. RRE 6 1.4 Estimation of Ridge parameter 7 1.5 Preliminary-test and Stein-type Ridge Estimators 7 1.6 High dimension Setting 9 1.7 Notes and References 11 1.8 Organization of the Book 12 2 Location and Simple Linear Models 15 2.1 Introduction 15 2.2 Location Model 16 2.2.1 Location Model: Estimation 16 2.2.2 Shrinkage Estimation of Location 17 2.2.3 Ridge Regression Type Estimation of Location Parameter 18 2.2.4 LASSO for Location Parameter 19 2.2.5 Bias and MSE expression for the LASSO of Location Parameter 20 2.2.6 Preliminary Test Estimator, Bias and MSE 24 2.2.7 Stein-type Estimation of Location Prameter 24 2.2.8 Comparison of LSE, PTE, Ridge, SE, and LASSO 25 2.3 Simple Linear Model 27 2.3.1 Estimation of the Intercept and Slope Parameters 27 2.3.2 Test For Slope Parameter 28 2.3.3 PTE of the Intercept and Slope Parameters 28 2.3.4 Comparison of Bias and MSE Functions 30 2.3.5 Alternative PT Estimator 32 2.3.6 Optimum Level of Significance of Preliminary Test 33 2.3.7 Ridge-type Estimation of Intercept and Slope 34 2.3.8 LASSO Estimation of Intercept and Slope 36 2.4 Summary and Concluding Remarks 40 2.5 Problems 40 3 ANOVA Model 43 3.1 Introduction 43 3.2 Model, Estimation and Tests 44 3.4 Comparison of Estimators 52 3.4.1 Comparison of LSE with RLSE 52 3.4.2 Comparison of LSE with PTE 52 3.4.3 Comparison of LSE with SE and PRSE 53 3.4.4 Comparison of LSE and RLSE with RRE 56 3.4.5 Comparison of RRE with PTE, SE and PRSE 56 3.4.6 Comparison of LASSO with LSE and RLSE 59 3.4.7 Comparison of LASSO with PTE, SE and PRSE 59 3.4.8 Comparison of LASSO with RRE 60 3.5 Application 60 3.6 Efficiency in terms of unweighted L2-Risk 63 3.7 Summary and Concluding Remarks 64 3.8 Problems 78 4 Seemingly Unrelated Simple Linear Models 81 4.1 Model, Estimation and Test of Hypothesis 82 4.1.1 LSE of θ and β 82 4.1.2 Penalty Estimation of β and θ 82 4.1.3 PTE and Stein Type Estimators of β and θ 83 4.2 Bias and MSE expressions of the estimators 84 4.3 Comparison of Estimators 88 4.3.1 Comparison of LSE with RLSE 88 4.3.2 Comparison of LSE with PTE 88 4.3.3 Comparison of LSE with SE and PRSE 89 4.3.4 Comparison of LSE and RLSE with RRE 91 4.3.5 Comparison of RRE with PTE, SE and PRSE 91 4.3.6 Comparison of LASSO with RRE 93 4.3.7 Comparison of LASSO with LSE and RLSE 93 4.3.8 Comparison of LASSO with PTE, SE and PRSE 95 4.4 Efficiency in terms of unweighted L2-Risk 95 4.4.1 Efficiency for β 97 4.4.2 Efficiency for θ 98 4.5 Summary and Concluding Remarks 99 4.6 Problems 110 5 Multiple Regression 111 5.1 Introduction 111 5.2 Linear Model and the Estimators 112 5.2.1 Penalty estimators 113 5.2.2 Shrinkage estimators 116 5.3 Bias and Weighted L2-Risks of Estimators 117 5.3.1 Hard Threshold Estimator 117 5.3.2 Modified LASSO 118 5.3.3 Multivariate normal decision theory and oracles for diagonal linear projection 119 5.3.4 Ridge Regression Estimator 121 5.3.5 Shrinkage Estimators 122 5.4 Comparison of Estimators 122 5.4.1 Comparison of LSE with RLSE 123 5.4.2 Comparison of LSE with PTE 123 5.4.3 Comparison of LSE with SE and PRSE 124 5.4.4 Comparison of LSE and RLSE with RRE 125 5.4.5 Comparison of RRE with PTE, SE and PRSE 126 5.4.6 Comparison of modified LASSO with LSE and RLSE 127 5.4.7 Comparison of modified LASSO with PTE, SE and PRSE129 5.4.8 Comparison of MLASSO with RRE 130 5.5 Efficiency in terms of unweighted L2-Risk 130 5.6 Summary and Concluding Remarks 131 5.7 Problems 143 6 Ridge Regression in Theory and Applications 145 6.1 Multiple Linear Model Specification 145 6.1.1 Estimation of Regression Parameters 146 6.1.2 Test of Hypothesis for the Coefficients Vector 148 6.2 Ridge Regression Estimators (RRE) 148 6.3 Bias, MSE and L2-risk of Ridge Regression Estimator 149 6.4 Determination of the Tuning Parameters 153 6.5 Ridge Trace 154 6.6 Degrees of Freedom of RRE 155 6.7 Generalized Ridge Regression Estimators 157 6.8 LASSO and Adaptive Ridge Regression Estimators 158 6.9 Optimization Algorithm 160 6.9.1 Prostate Cancer Data 162 6.10 Estimation of Regression Parameters for Low-dimensional Models162 6.10.1 BLUE and Ridge Regression Estimators 163 6.10.2 Bias and L2-Risk Expressions of Estimators 164 6.10.3 Comparison of the Estimators 167 6.10.4 Asymptotic Results of RRE 168 6.11 Summary and Concluding Remarks 170 6.12 Problems 170 7 Partially Linear Regression Models 173 7.1 Introduction 173 7.2 Partial Linear Model and Estimation 174 7.3 Ridge Estimators of Regression Parameter 177 7.4 Biases and L2-Risks of Shrinkage Estimators 179 7.5 Numerical Analysis 180 7.5.1 Example: Housing Prices Data 184 7.6 High dimensional PLM 191 7.6.1 Example: Riboflavin Data 194 7.7 Summary and Concluding Remarks 196 7.8 Problems 197 8 Logistic Regression Model 199 8.1 Introduction 199 8.1.1 Penalty estimators 201 8.1.2 Shrinkage estimators 203 8.1.3 Results on penalty estimators 203 8.1.4 Results on PTE and Stein-type estimators 204 8.1.5 Results on penalty estimators 206 8.2 Asymptotic Distributional L2-risk Efficiency Expressions of the Estimators 207 8.2.1 MLASSO vs MLE 208 8.2.2 MLASSO vs RMLE 208 8.2.3 Comparison of MLASSO vs PTE 209 8.2.4 PT and MLE 209 8.2.5 Comparison of MLASSO vs SE 210 8.2.6 Comparison of MLASSO vs PRSE 211 8.2.7 Ridge vs MLE 212 8.2.8 Comparison of Ridge vs PTE 213 8.2.9 Comparison of Ridge vs SE 214 8.2.10 Comparison of Ridge vs PRSE 214 8.2.11 Comparison of PTE vs SE and PRSE 215 8.2.12 Numerical comparison among the estimators 215 8.3 Summary and Concluding Remarks 218 8.4 Problems 221 9 Regression Models with Autoregressive errors 223 9.1 Introduction 223 9.1.1 Penalty Estimators 225 9.1.2 Shrinkage Estimators 227 9.1.3 Results on Penalty Estimators 227 9.1.4 Results on PTE and Stein-Type Estimators 228 9.1.5 Results on penalty estimators 231 9.2 Asymptotic Distributional L2-Risk Efficiency Comparison 232 9.2.1 Comparison of GLSE with RGLSE 232 9.2.2 Comparison of GLSE with PTE 233 9.2.3 Comparison of LSE with SE and PRSE 234 9.2.4 Comparison of LSE and RLSE with RRE 235 9.2.5 Comparison of RRE with PTE, SE and PRSE 236 9.2.6 Comparison of modified LASSO with GLSE and RGLSE237 9.2.7 Comparison of MLASSO with PTE, SE and PRSE 238 9.2.8 Comparison of MLASSO with RRE 238 9.3 Examlpe: Sea Level Rise at Key West, Florida 239 9.3.1 Estimation of the Model Parameters 239 9.3.2 Relative Efficiency 242 9.4 Summary and Concluding Remarks 250
• Edition: 1st
• Publisher Details: Hoboken, New Jersey : John Wiley & Sons, Inc
• Publication Date: 2016
• Extent: 1 online resource
• Subjects: 519.536; Ridge regression (Statistics); Estimation theory; Regression analysis
• Languages: English
• ISBNs: 9781118644508; 9781118644522
• Related ISBNs: 9781118644614
• Notes: Note: Description based on CIP data; item not viewed.
• Access Rights: Legal Deposit; Only available on premises controlled by the deposit library and to one user at any one time; The Legal Deposit Libraries (Non-Print Works) Regulations (UK).
• Access Usage: Restricted: Printing from this resource is governed by The Legal Deposit Libraries (Non-Print Works) Regulations (UK) and UK copyright law currently in force.