Package 'liureg' reference manual

Title:	Liu Regression with Liu Biasing Parameters and Statistics
Description:	Linear Liu regression coefficient's estimation and testing with different Liu related measures such as MSE, R-squared etc. REFERENCES i. Akdeniz and Kaciranlar (1995) <doi:10.1080/03610929508831585> ii. Druilhet and Mom (2008) <doi:10.1016/j.jmva.2006.06.011> iii. Imdadullah, Aslam, and Saima (2017) iv. Liu (1993) <doi:10.1080/03610929308831027> v. Liu (2001) <doi:10.1016/j.jspi.2010.05.030>.
Authors:	Imdad Ullah Muhammad [aut, cre] , Aslam Muhammad [aut, ctb]
Maintainer:	Imdad Ullah Muhammad <[email protected]>
License:	GPL-2 \| GPL-3
Version:	1.1.2
Built:	2025-01-24 05:24:30 UTC
Source:	https://github.com/cran/liureg

Liu Regression and Estimator

Description

R package for fitting linear Liu Regression and Estimator which is proposed by Liu (1993) <doi:10.1080/03610929308831027>.

Details

This package contains functions for fitting Liu regression models, including function for computation of different Liu related statistics (such as MSE, Var-Cov matrix, R-squared), estimation of biasing parameter from different researchers, testing of Liu coefficients, model selection criteria, residuals, predicted values and fitted values. The package also includes function for plotting of Liu coefficients and different Liu statistics for selection of optimal value of biasing parameter $d$ .

For a complete list of functions, use library(help="liureg").

Author(s)

Muahmmad Imdad Ullah, Muhammad Aslam

Computation of Liu Biasing Parameter $d$

Description

The dest function computes different Liu biasing parameters available in the literature, proposed by different researchers such as given in Liu (1993) <doi:10.1080/03610929308831027>, Liu (2011) <doi:10.1016/j.jspi.2010.05.030>, and Ozkale and Kaciranlar (2007) <doi:10.1080/03610920601126522>.

Usage

dest(object, ...)
## S3 method for class 'liu'
dest(object, ...)
## S3 method for class 'dliu'
print(x, ...)
dest(object, ...)
## S3 method for class 'liu'
dest(object, ...)
## S3 method for class 'dliu'
print(x, ...)

Arguments

`object`	An object of class "liu".
`x`	An object of class "dliu" for the `print.dest.dliu`.
`...`	Not presently used in this implementation.

Details

The dest function computes different biasing parameter for the Liu regression. All these methods are already available in the literature proposed by various authors.

Value

This function returns the list of following estimators of Liu parameter, available in the literature.

`dopt`	By Liu (1993): $\frac{\sum_{j=1}^p \left[\frac{\alpha_j^2-\sigma^2}{(\lambda_j+1)^2}\right]}{\sum_{j=1}^p\left[\frac{\sigma^2+\lambda_j\times\alpha_j^2}{\lambda_j(\lambda_j+1)^2}\right]}$
`dILE`	By Liu, (2011): $\frac{\sum_{j=1}^n \left(\frac{\widetilde{e}_i}{1-g_{ii}}\left(\frac{\widetilde{e}_i}{1-h_{1-ii}}-\frac{\hat{e}_i}{1-h_{ii}}\right)\right)} {\sum_{i=1}^n \left(\frac{\widetilde{e}_i}{1-g_{ii}}-\frac{\hat{e}_i}{1-h_{ii}}\right)^2}$ , where, $\hat{e}=y_i-x'_i(X'X-x_i x'_i)^{-1}(X'y-x_iy_i)$ , $\widetilde{e}=y_i-x'_i(X'X+I_p-x_i x'_i)^{-1}(X'y-x_iy_i)$ , $G=X(X'X+I_p)^{-1}X'$ and $H\cong X(X'X)^{-1}X'$ .
`dmm`	Liu, (1993): $(1-\hat{\sigma}^2) \left[\frac{\sum_{j=1}^p\left[\frac{1}{\lambda_j(\lambda_j+1)}\right]}{\sum_{j=1}^p\left[\frac{\hat{\alpha}_j^2}{(\lambda_j+1)^2}\right]}\right]$
`dcl`	By Liu, (1993): $(1-\hat{\sigma}^2)\left[\frac{\sum_{j=1}^p\left[\frac{1}{(\lambda_j+1)}\right]}{\sum_{j=1}^p \left[\frac{\lambda_j\hat{\alpha}_j^2}{(\lambda_j+1)^2}\right]}\right]$ .
`GCV`	GCV criterion for selection of optimal $d$ , that is, $GCV=\frac{SSR_d}{(n-1-trace(H_d))}$ , where $SSR_d$ is residuals sum of squares from Liu regression at certain value of $d$ and $trace(H_d)$ is trace of hat matrix from Liu.
`dGCV`	returns value of $d$ at which GCV is minimum.

Author(s)

Muhammad Imdad Ullah, Muhammad Aslam

References

Akdeniz, F. and Kaciranlar, S. (1995). On the Almost Unbiased Generalized Liu Estimators and Unbiased Estimation of the Bias and MSE. Communications in Statistics-Theory and Methods, 24, 1789–1897. http://doi.org/10.1080/03610929508831585.

Imdad, M. U. (2017). Addressing Linear Regression Models with Correlated Regressors: Some Package Development in R (Doctoral Thesis, Department of Statistics, Bahauddin Zakariya University, Multan, Pakistan).

Imdadullah, M., Aslam, M., and Altaf, S. (2017). liureg: A comprehensive R Package for the Liu Estimation of Linear Regression Model with Collinear Regressors. The R Journal, 9 (2), 232–247.

Liu, K. (1993). A new Class of Biased Estimate in Linear Regression. Journal of Statistical Planning and Inference, 141, 189–196. http://doi.org/10.1080/03610929308831027.

Liu, X-Q. (2011). Improved Liu Estimator in a Linear Regression Model. Journal of Statistical Planning and Inference,141, 189–196. https://doi.org/10.1016/j.jspi.2010.05.030.

Ozkale, R. M. and Kaciranlar, S. (2007). A Prediction-Oriented Criterion for Choosing the Biasing Parameter in Liu Estimation. Commincations in Statistics-Theory and Methods, 36(10): 1889–1903. http://doi.org/10.1080/03610920601126522.

Examples

mod<-liu(y ~ ., data = as.data.frame(Hald), d = seq(-5, 5, 0.1))
dest(mod)
## Vector of GCV values for each d
dest(mod)$GCV
mod<-liu(y ~ ., data = as.data.frame(Hald), d = seq(-5, 5, 0.1))
dest(mod)
## Vector of GCV values for each d
dest(mod)$GCV

Portland Cement benchmark of Hald(1952)

Description

Heat evolved during setting of 13 cement mixtures of four basic ingredients. Each ingredient percentage appears to be rounded down to a full integer. The sum of the four mixture percentages varies from a maximum of 99% to a minimum of 95%. If all four regressor X-variables always summed to 100%, the centered X-matrix would then be of rank only 3. Thus, the regression of heat on four X-percentages is ill-conditioned, with an approximate rank deficiency of MCAL = 1.

Usage

data(Hald)data(Hald)

Format

A data frame with 13 observations on the following 5 variables.

X1: p3ca: Integer percentage of 3CaO.Al2O3 in the mixture.
X2: p3cs: Integer percentage of 3CaO.SiO2 in the mixture.
X3: p4caf: Integer percentage of 4CaO.Al2O3.Fe2O3 in the mixture.
X4: p2cs: Integer percentage of 2CaO.SiO2 in the mixture.
y: hear: Heat (cals/gm) evolved in setting, recorded to nearest tenth.

Details

The (lmridge) Hald data are identical to the (MASS) cement data except for variable names.

Source

Woods, H., Steinour, H.H. and Starke, H.R. (1932). Effect of Composition of Portland Cement on Heat Evolved During Hardening. Industrial Engineering and Chemistry 24: 1207–1214.

References

Hald, A. (1952). Statistical Theory with Engineering Applications.(page 647.) New York; Wiley.

Liu Regression: Hat Matrix

Description

The hatl function computes hat matrix of Liu regression (Liu (1993) <doi:10.1080/03610929308831027>).

Usage

hatl(object, ...)
## S3 method for class 'liu'
hatl(object, ...)
hatl(object, ...)
## S3 method for class 'liu'
hatl(object, ...)

Arguments

`object`	An object of class "liu".
`...`	Not presently used in this implementation.

Details

Hat matrix for scalar or vector values of biasing parameter $d$ provided as argument to liu function. It is used to compute degrees of freedom for given $d$ , and error degree of freedom etc. The hat matrix can be computed using formula $X(X'X+I_p)^{-1}(X'X+dI_p)(X'X)^{-1}X'$ .

Value

Returns a list of matrix for each biasing parameter $d$ :

hatl

A list of hat matrix for each biasing parameter $d$ .

Note

The hat matrix is not idempotent because it is not projection matrix, therefore it is called quasi-projection matrix.

Author(s)

Muhammad Imdad Ullah, Muhammad Aslam

References

Imdadullah, M., Aslam, M., and Altaf, S. (2017). liureg: A comprehensive R Package for the Liu Estimation of Linear Regression Model with Collinear Regressors. The R Journal, 9 (2), 232–247.

Liu, K. (1993). A new Class of Biased Estimate in Linear Regression. Journal of Statistical Planning and Inference, 141, 189–196. http://doi.org/10.1080/03610929308831027.

Examples

mod<-liu(y ~ . , data = as.data.frame(Hald), d = c(-5, -1, 0.2, 0.3))
## Hat matrix for each biasing parameter
hatl(mod)

## Hat matrix for second biasing parameter i.e. d = -1
hatl(mod)[[2]]

## Diagonal element of hat matrix for second biasing parameter
diag(hatl(mod)[[2]])
mod<-liu(y ~ . , data = as.data.frame(Hald), d = c(-5, -1, 0.2, 0.3))
## Hat matrix for each biasing parameter
hatl(mod)

## Hat matrix for second biasing parameter i.e. d = -1
hatl(mod)[[2]]

## Diagonal element of hat matrix for second biasing parameter
diag(hatl(mod)[[2]])

Model Selection Criteria for Liu Regression

Description

The infoliu.liu computes model selection criteria (AIC and BIC).

Usage

infoliu(object, ...)
## S3 method for class 'liu'
infoliu(object, ...)
infoliu(object, ...)
## S3 method for class 'liu'
infoliu(object, ...)

Arguments

`object`	An object of class "liu".
`...`	Not presently used in this implementation.

Details

Model information selection criteria are common way of selecting among model while balancing the competing goals of fit and parsimony. The model selection criteria AIC and BIC are computed by quantifying degree of freedom df in the Liu regression model, using formula $trace(H_d)$ , where $H_d$ is hat matrix from Liu regression. Model selection criteria may be helpful for selecting optimal value of biasing parameter $d$ .

Value

It returns a matrix of information criteria, AIC and BIC for each biasing parameter $d$ . Column of matrix indicates model selection criteria AIC and BIC, respectively, while rows indicate value of biasing parameter $d$ for which model selection criteria are computed.

Author(s)

Muhammad Imdad Ullah, Muhammad Aslam

References

Akaike, H. (1974). A new look at the Statistical Model Identification. IEEE Transaction on Automatic Control, 9(6), 716–723. https://doi.org/10.1109/TAC.1974.1100705.

Imdadullah, M., Aslam, M., and Altaf, S. (2017). liureg: A comprehensive R Package for the Liu Estimation of Linear Regression Model with Collinear Regressors. The R Journal, 9 (2), 232–247.

Liu, K. (1993). A new Class of Biased Estimate in Linear Regression. Journal of Statistical Planning and Inference, 141, 189–196. http://doi.org/10.1080/03610929308831027.

Schwarz, G. (1978). Estimating the Dimension of a Model. Annals of Statistics, 6(2), 461–464. https://projecteuclid.org/euclid.aos/1176344136.

Examples

mod<-liu(y~X1+X2+X3+X4, data = as.data.frame(Hald), d = seq(-5, 5, 0.1), scaling = "centered")

infoliu(mod)

## AIC values
infoliu(mod)[,1]

## BIC values
infoliu(mod)[,2]
mod<-liu(y~X1+X2+X3+X4, data = as.data.frame(Hald), d = seq(-5, 5, 0.1), scaling = "centered")

infoliu(mod)

## AIC values
infoliu(mod)[,1]

## BIC values
infoliu(mod)[,2]

Liu Regression and Estimator

Description

Fits a linear Liu regression model after scaling regressors and returns an object of class "liu" (by calling liuest function), designed to be used in plotting method, testing of Liu coefficients and for computation of different Liu related statistics. The Liu biasing parameter $d$ can be a scalar or a vector. This new biased estimator was first proposed by Liu (1993) <doi:10.1080/03610929308831027>.

Usage

liu(formula, data, d = 1, scaling=c("centered", "sc", "scaled"), ...)
liuest(formula, data, d=1, scaling=c("centered", "sc", "scaled"), ...)
## Default S3 method:
liu(formula, data, d = 1, scaling=c("centered", "sc", "scaled"), ...)
## S3 method for class 'liu'
coef(object, ...)
## S3 method for class 'liu'
print(x, digits = max(5,getOption("digits") - 5), ...)
## S3 method for class 'liu'
fitted(object, ...)
liu(formula, data, d = 1, scaling=c("centered", "sc", "scaled"), ...)
liuest(formula, data, d=1, scaling=c("centered", "sc", "scaled"), ...)
## Default S3 method:
liu(formula, data, d = 1, scaling=c("centered", "sc", "scaled"), ...)
## S3 method for class 'liu'
coef(object, ...)
## S3 method for class 'liu'
print(x, digits = max(5,getOption("digits") - 5), ...)
## S3 method for class 'liu'
fitted(object, ...)

Arguments

`formula`	Standard R formula expression, that is, a symbolic representation of the model to be fitted and has form `response~predictors`. For further details, see `formula`.
`data`	An optional data frame containing the variables in the model. If not found in data, the variables are taken from `environment(formula)`, typically the environment from which `liu` or `liuest` is called.
`d`	Liu biasing parameter (may be a vector).
`scaling`	The method to be used to scale the predictors. The scaling option `"sc"` scales the predictors to correlation form, such that the correlation matrix has unit diagonal elements. `"scaled"` option standardizes the predictors to have zero mean and unit variance. `"centered"` option centers the predictors.
`object`	A liu object, typically generated by a call to `liu` for `fitted.liu`, `predict.liu`, `vcov.liu`, `residuals.liu`, `infocr.liu`, `coef.liu`, `summary.liu` and `press.liu` functions.
`x`	An object of class `liu` (for the `hatl.liu`, `lstats.liu`, `dest.liu`, `summary.liu`, `print.liu`, `print.summary.liu`, `print.dest`, `print.lstats`, and `plot.liu`, `plot.biasliu` and `plot.infoliu`.
`digits`	Minimum number of significant digits to be used.
`...`	Additional arguments to be passed to or from other methods.

Details

liu or liuest function fits in Liu regression after scaling the regressors and centering the response. The liu is default a function that calls liuest for computation of Liu coefficients and returns an object of class "liu" designed to be used in plotting method, testing of Liu coefficients and for computation of different Liu related statistics. If intercept is present in the model, its coefficient is not penalized. However, intercept is estimated from the relation $y=\overline{y}-\beta \overline{X}$ . print.liu tries to be smart about formatting of Liu coefficients.

Value

liu function returns an object of class "liu" after calling list of named objects from liuest function:

`coef`	A named vector of fitted coefficients.
`call`	The matched call.
`Inter`	Was an intercept included?
`scaling`	The scaling method used.
`mf`	Actual data used.
`y`	The centered response variable.
`xs`	The scaled matrix of predictors.
`xm`	The vector of means of the predictors.
`terms`	The `terms` object used.
`xscale`	Square root of sum of squared deviation from mean regarding the scaling option used in `liu` or `liuest` function as argument.
`lfit`	The fitted value of Liu regression for given biasing parameter $d$ .
`d`	The Liu regression biasing parameter $d$ which can be scalar or a vector.

Note

The function at the current form cannot handle missing values. The user has to take prior action with missing values before using this function.

Author(s)

Muhammad Imdad Ullah, Muhammad Aslam

References

Imdadullah, M., Aslam, M., and Altaf, S. (2017). liureg: A comprehensive R Package for the Liu Estimation of Linear Regression Model with Collinear Regressors. The R Journal, 9 (2), 232–247.

Liu, K. (1993). A new Class of Biased Estimate in Linear Regression. Journal of Statistical Planning and Inference, 141, 189–196. http://doi.org/10.1080/03610929308831027.

Examples

data(Hald)
mod<-liu(y~., data = as.data.frame(Hald), d = seq(0, 0.1, 0.01), scaling = "centered")
## Scaled Coefficients
mod$coef

## Re-Scaled Coefficients
coef(mod)

## Liu fitted values
fitted(mod)

## Liu predited values
predict(mod)

## Liu Residuals
residuals(mod)

## Liu trace
plot(mod)

## Liu Var-Cov matrix
vcov(mod)

## Liu biasing parameters by researchers
dest(mod)

## Liu related statistics
lstats(mod)

## list of objects from liuest function
liuest(y~., data = as.data.frame(Hald), d = seq(0, 0.1, 0.01), scaling = "centered")
data(Hald)
mod<-liu(y~., data = as.data.frame(Hald), d = seq(0, 0.1, 0.01), scaling = "centered")
## Scaled Coefficients
mod$coef

## Re-Scaled Coefficients
coef(mod)

## Liu fitted values
fitted(mod)

## Liu predited values
predict(mod)

## Liu Residuals
residuals(mod)

## Liu trace
plot(mod)

## Liu Var-Cov matrix
vcov(mod)

## Liu biasing parameters by researchers
dest(mod)

## Liu related statistics
lstats(mod)

## list of objects from liuest function
liuest(y~., data = as.data.frame(Hald), d = seq(0, 0.1, 0.01), scaling = "centered")

Liu Regression Statistics

Description

The lstats function computes the Liu regression related statistics such as variance, estimated squared bias, MSE, R-squared and adjusted R-squared etc. These statistics are computed by following Liu (1993) <doi:10.1080/03610929308831027>; Akdeniz, F. and Kaciranlar, S. (1995) <doi:10.1080/03610929508831585>; Cule, E. and De Iorioa, M. (2012); Hastie, T. and Tibshirani, R. (1990); and Mallows (1973) <doi:10.2307/1267380>.

Usage

lstats(object, ...)
## S3 method for class 'liu'
lstats(object, ...)
## S3 method for class 'lstats'
print(x, ...)
lstats(object, ...)
## S3 method for class 'liu'
lstats(object, ...)
## S3 method for class 'lstats'
print(x, ...)

Arguments

`object`	An object of class "liu".
`x`	An object of class "liu" for `print.lstats.liu`.
`...`	Not presently used in this implementation.

Details

The lstats function computes the Liu regression related statistics which may help in selecting appropriate optimal value of biasing parameter $d$ . If value of $d$ is one then these statistics are equivalent to the relevant OLS statistics.

Value

`lEDF`	Residual effective degrees of freedom for given biasing parameter $d$ from Hastie and Tibshirani (1990), i.e., $n-trace(2H_d)-H_d t(H_d)$ .
`lsigma2`	Computation of $\hat{\sigma}^2$ from Liu regression.
`Cl`	Mallows $C_p$ like statistics for given biasing parameter $d$
`var`	Variance of Liu regression for given biasing parameter $d$ .
`bias2`	Estimated squared bias of Liu regression for given biasing parameter $d$ .
`mse`	Total MSE value for given biasing parameter $d$ .
`Fv`	F-statistics value for testing of the significance of the Liu regression estimator computed for given biasing parameter $d$ .
`R2`	R-squared for given biasing parameter $d$ .
`adjR2`	Adjusted R-squared for given biasing parameter $d$ .
`minmse`	Minimum MSE value for a certain value of biasing parameter $d$ .
`SSER`	Sum of squares of error from Liu regression for each biasing parameter $d$ .

Author(s)

Muhammad Imdad Ullah, Muhammad Aslam

References

Cule, E. and De lorioa, M. (2012). A semi-Automated method to guide the choice of ridge parameter in ridge regression. arXiv:1205.0686v1[stat.AP]. https://arxiv.org/abs/1205.0686v1.

Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models. Chapman \& Hall.

Imdadullah, M., Aslam, M., and Altaf, S. (2017). liureg: A comprehensive R Package for the Liu Estimation of Linear Regression Model with Collinear Regressors. The R Journal, 9 (2), 232–247.

Liu, K. (1993). A new Class of Biased Estimate in Linear Regression. Journal of Statistical Planning and Inference, 141, 189–196. http://doi.org/10.1080/03610929308831027.

Mallows, C. L. (1973). Some Comments on Cp. Technometrics, 15: 661–675. http://doi.org/10.2307/1267380.

Examples

mod<-liu(y~., data = as.data.frame(Hald), d = seq(-5, 5, 0.1), scaling = "centered")

lstats(mod)
mod<-liu(y~., data = as.data.frame(Hald), d = seq(-5, 5, 0.1), scaling = "centered")

lstats(mod)

Bias Variance and MSE Trade-off Plot

Description

Trade-off between bias, variance and MSE of the Liu regression against vector or scalar value of biasing parameter $d$ .

Usage

## S3 method for class 'biasliu'
plot(x, abline = TRUE, ...)## S3 method for class 'biasliu'
plot(x, abline = TRUE, ...)

Arguments

`x`	An object of "liu".
`abline`	Horizontal and vertical lines show the minimum value of the Liu MSE at certain value of biasing parameter $d$ .
`...`	No presently used in this implementation.

Details

The effect of multicollinearity on the coefficients can be identified using different graphical display. One of them is plot of bias, variance and MSE. Addition of biasing parameter $d$ lead to a substantial impact on variance and MSE of Liu regression estimates. Therefore, a trade-off is made between bias and variance to have an acceptable MSE. The plot.biasliu can be helpful for selection of optimal value of biasing parameter $d$ .

Value

Nothing returned

Author(s)

Muhammad Imdad Ullah, Muhammad Aslam

References

Imdadullah, M., Aslam, M., and Altaf, S. (2017). liureg: A comprehensive R Package for the Liu Estimation of Linear Regression Model with Collinear Regressors. The R Journal, 9 (2), 232–247.

Kalivas, J. H., and Palmer, J. (2014). Characterizing multivariate calibration tradeoff (bias, variance, selectivity, and sensitivity) to select model tuning parameters. Journal of Chemometrics, 28(5), 347–357. https://doi.org/10.1002/cem.2555.

Examples

mod<-liu(y~., as.data.frame(Hald), d = seq(-5, 5, 0.1))

## for indication of vertical line (biasing parameter d) and
## horizontal line (minimum Liu MSE value corresponding to vertical line)

plot.biasliu(mod)

## without horizontal and vertical line
plot.biasliu(mod, abline = FALSE)
mod<-liu(y~., as.data.frame(Hald), d = seq(-5, 5, 0.1))

## for indication of vertical line (biasing parameter d) and
## horizontal line (minimum Liu MSE value corresponding to vertical line)

plot.biasliu(mod)

## without horizontal and vertical line
plot.biasliu(mod, abline = FALSE)

Model Selection Criteria Plots

Description

Plot of the Liu AIC and BIC model selection criteria against Liu degrees of freedom.

Usage

## S3 method for class 'infoliu'
plot(x, abline = TRUE, ...)## S3 method for class 'infoliu'
plot(x, abline = TRUE, ...)

Arguments

`x`	An object of class "liu".
`abline`	Vertical line to show minimum value of Liu MSE at certain of Liu degrees of freedom.
`...`	Not presently used in this implementation.

Details

Plot of the Liu AIC and BIC against the Liu degree of freedom (sum of diagonal elements of the Liu Hat matrix). A vertical line represents the minimum Liu MSE at certain value of the Liu degree of freedom.

Value

Nothing returned

Author(s)

Muhammad Imdad Ullah, Muhammad Aslam

References

Akaike, H. (1974). A new look at the Statistical Model Identification. IEEE Transaction on Automatic Control, 9(6), 716–723. https://doi.org/10.1109/TAC.1974.1100705.

Imdadullah, M., Aslam, M., and Altaf, S. (2017). liureg: A comprehensive R Package for the Liu Estimation of Linear Regression Model with Collinear Regressors. The R Journal, 9 (2), 232–247.

Schwarz, G. (1978). Estimating the Dimension of a Model. Annals of Statistics, 6(2), 461–464. https://projecteuclid.org/euclid.aos/1176344136.

Examples

mod<- liu(y~., as.data.frame(Hald), d = seq(-5, 5, 0.1))
## for indication of minimum MSE at Liu df (as vertical line)
plot.infoliu(mod)

## without vertical line
plot.infoliu(mod, abline = FALSE)
mod<- liu(y~., as.data.frame(Hald), d = seq(-5, 5, 0.1))
## for indication of minimum MSE at Liu df (as vertical line)
plot.infoliu(mod)

## without vertical line
plot.infoliu(mod, abline = FALSE)

Plot of the Liu Coefficients

Description

Plot of the Liu coefficient for scalar or vector values of biasing parameter $d$ .

Usage

## S3 method for class 'liu'
plot(x, abline = TRUE, ...)## S3 method for class 'liu'
plot(x, abline = TRUE, ...)

Arguments

`x`	An object of class "liu".
`abline`	Vertical line to minimum value of MSE at certain value of biasing parameter $d$ used as argument in `liu` function.
`...`	Not presently used in this implementation.

Details

Graphical way of judging optimal value of biasing parameter $d$ . The biasing parameter may be selected for acceptable (or minimum) value of MSE.

Value

Nothing returned

Author(s)

Muhammad Imdad Ullah, Muhammad Aslam

References

Imdadullah, M., Aslam, M., and Altaf, S. (2017). liureg: A comprehensive R Package for the Liu Estimation of Linear Regression Model with Collinear Regressors. The R Journal, 9 (2), 232–247.

Liu, K. (1993). A new Class of Biased Estimate in Linear Regression. Journal of Statistical Planning and Inference, 141, 189–196. http://doi.org/10.1080/03610929308831027.

Examples

mod<-liu(y~., data = as.data.frame(Hald), d = seq(-5, 5, 0.4))
plot(mod)
plot(mod, abline = TRUE)
mod<-liu(y~., data = as.data.frame(Hald), d = seq(-5, 5, 0.4))
plot(mod)
plot(mod, abline = TRUE)

Predict Method for Liu Regression Model Fits

Description

Predicted values based on the Liu regression model for scalar or vector values of biasing parameter $d$ .

Usage

## S3 method for class 'liu'
predict(object, newdata, na.action = na.pass, terms = NULL, ...)## S3 method for class 'liu'
predict(object, newdata, na.action = na.pass, terms = NULL, ...)

Arguments

`object`	An object of class "liu".
`newdata`	An optional data frame in which to look for variables with which to predict.
`na.action`	Function determine what should be done with missing values in `newdata`. The default is to predict `NA`.
`terms`	The `terms` object used.
`...`	Not presently used in this implementation.

Details

The predict.liu function predicted values, obtained by evaluating the regression function in the frame newdata which defaults to model.frame (object). If newdata is omitted the predictions are based on the data used for the fit. In that case how cases with missing values in the original fit are handled is determined by the na.action argument of that fit. If na.action=na.omit omitted cases will not appear in the predictions, whereas if na.action=na.exclude they will appear (in predictions), with value NA.

Value

predict.liu produces a vector of prediction or a matrix of predictions for scalar or vector values of biasing parameter $d$ .

Note

Variables are first looked for in newdata and then are searched for in the usual way (which will include the environment of the formula used in the fit). A warning will be given if the variables found are not of the same length as those in the newdata if it was supplied.

Author(s)

Muhammad Imdad Ullah, Muhammad Aslam

Examples

mod <- liu(y~., data = as.data.frame(Hald), d = seq(-5, 5, 0.1))
predict(mod)

predict(mod, newdata=as.data.frame(Hald[1:5, -1]))
mod <- liu(y~., data = as.data.frame(Hald), d = seq(-5, 5, 0.1))
predict(mod)

predict(mod, newdata=as.data.frame(Hald[1:5, -1]))

Predicted Residual Sum of Squares

Description

The press.liu function computes predicted residual sum of squares (PRESS), computed from by following Ozkale and Kaciranlar (2007) <doi:10.1080/03610920601126522>.

Usage

press(object, predr = FALSE, ...)
## S3 method for class 'liu'
press(object, predr = FALSE, ...)
press(object, predr = FALSE, ...)
## S3 method for class 'liu'
press(object, predr = FALSE, ...)

Arguments

`object`	An object of class "liu".
`predr`	If `TRUE` then predicted residuals are returned.
`...`	Not presently used in this implementation.

Details

For all of the n leave-one-out predicted residual sum of squares is calculated by fitting full regression model. PRESS is computed by using, $\sum (\hat{e}_{d(i)})^2$ or $\sum \left[\frac{\hat{e}_{di}}{1-h_{1-ii}}-\frac{e_i}{(1-h_{1-ii})(1-h_{ii})}(h_{1-ii}-\widetilde{H}_{d-ii})\right]^2$ , where $h_{ii}=X(X'X)^{-1} X'$ 's ith diagonal element, $h_{1-ii}=X(X'X+I)^{-1}X'$ 's ith diagonal element and $\hat{e}_{di}$ is the ith residual at specific value of $d$ .

Value

The press.liu produces a vector of PRESS for scalar or vector values of biasing parameter $d$ . If argument predr is TRUE then predicted residuals are returned instead of predicted residual sum of squares.

Author(s)

Muhammad Imdad Ullah, Muhammad Aslam

References

Allen, D. M. (1971). Mean Square Error of Prediction as a Criterion for Selecting Variables. Technometrics, 13, 469-475. http://www.jstor.org/stable/1267161.

Allen, D. M. (1974). The Relationship between Variable Selection and Data Augmentation and Method for Prediction. Technometrics, 16, 125-127. http://www.jstor.org/stable/1267500.

Imdadullah, M., Aslam, M., and Altaf, S. (2017). liureg: A comprehensive R Package for the Liu Estimation of Linear Regression Model with Collinear Regressors. The R Journal, 9 (2), 232–247.

Liu, K. (1993). A new Class of Biased Estimate in Linear Regression. Journal of Statistical Planning and Inference, 141, 189–196. http://doi.org/10.1080/03610929308831027.

Examples

mod<-liu(y~., data = as.data.frame(Hald), d = seq(-5, 5, 0.1))
## PRESS
press(mod)

## Predicted residual
press(mod, predr = TRUE)
mod<-liu(y~., data = as.data.frame(Hald), d = seq(-5, 5, 0.1))
## PRESS
press(mod)

## Predicted residual
press(mod, predr = TRUE)

Liu Regression Residuals

Description

The residuals function computes the Liu residuals for scalar or vector value of biasing parameter $d$ .

Usage

## S3 method for class 'liu'
residuals(object, ...)## S3 method for class 'liu'
residuals(object, ...)

Arguments

`object`	An object of class "liu".
`...`	Not presently used in this implementation.

Details

The generic functions residuals can be used to compute residuals of object of class "liu" from function liu.

Value

Returns a vector or a matrix of the Liu residuals for scalar or vector value biasing parameter $d$ provided as argument to liu function.

Author(s)

Muhammad Imdad Ullah, Muhammad Aslam

References

Liu, K. (1993). A new Class of Biased Estimate in Linear Regression. Journal of Statistical Planning and Inference, 141, 189–196. http://doi.org/10.1080/03610929308831027.

Examples

mod<-liu(y~., data = as.data.frame(Hald), d = seq(-2, 2, 0.1))

residuals(mod)
mod<-liu(y~., data = as.data.frame(Hald), d = seq(-2, 2, 0.1))

residuals(mod)

Summarizing Linear Liu Regression Fits

Description

The summary method for class "liu" for scalar or vector biasing parameter $d$ .

Usage

## S3 method for class 'liu'
summary(object, ...)
## S3 method for class 'summary.liu'
print(x, digits = max(4, getOption("digits") - 3),
           signif.stars = getOption("show.signif.stars"), ...)
## S3 method for class 'liu'
summary(object, ...)
## S3 method for class 'summary.liu'
print(x, digits = max(4, getOption("digits") - 3),
           signif.stars = getOption("show.signif.stars"), ...)

Arguments

`object`	An "liu" object, typically generated by a call to `liu`.
`x`	An object of class `summary.liu` for the `print.summary.liu`.
`signif.stars`	logical: if `TRUE`, p-values are additionally encoded visually as `significance starts` in order to help scanning of long coefficient tables. It default to the `show.signif.stars` slot of `options`.
`digits`	The number of significant digits to use when printing.
`...`	Not presently used in this implementation.

Details

print.summary.liu tries to be smart about formatting the coefficients, standard errors etc. and additionally gives 'significance stars' if signif.stars is TRUE.

Value

The function summary computes and returns a list of summary statistics of the fitted linear Liu regression model for scalar or vector value biasing parameter $d$ given as argument in liu function.

`coefficients`	A $p \times 5$ matrix with columns for the scaled estimated, descaled estimated coefficients, scaled standard error, scaled t-statistics, and corresponding p-value (two-tailed). The Intercept term is computed by the relation $\hat{\beta}_{od}=\overline{y}-\sum_{j=1}^{p}\overline{X}_j \hat{\beta}_{jd}$ . The standard error of intercept term is computed as, $SE(\hat{\beta}_{0d})=\sqrt{Var(\overline{y}) +\overline{X}_j^2 diag[Cov(\hat{\beta}_{jd})]}$ .
`stats`	Liu related statistics of R-squared, adjusted R-squared, F-statistics for testing of coefficients, AIC and BIC values for given biasing parameter $d$ .
`rmse1`	Minimum MSE value for given biasing parameter $d$ .
`rmse2`	Value of $d$ at which MSE is minimum.

Author(s)

Muhammad Imdad Ullah, Muhammad Aslam

References

Aslam, M. (2014). Using Heteroscedasticity-Consistent Standard Errors for the Linear Regression Model with Correlated Regressors. Communication in Statistics-Simulation and Computation, 43, 2353–2373. http://doi.org/10.1080/03610918.2012.750354.

Cule, E. and De lorio, M. (2012). A semi-Automatic method to guide the choice of ridge parameter in ridge regression. arXiv:1205.0686v1 [stat.AP]. https://arxiv.org/abs/1205.0686v1.

Halawa, A. And El-Bassiouni, M. (2000). Tests of Regression Coefficients Under Ridge Regression Models. Journal of Statistical Computation and Simulation, 65, 341–356. https://www.tandfonline.com/doi/abs/10.1080/00949650008812006.

Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models. Chapman & Hall.

Imdadullah, M., Aslam, M., and Altaf, S. (2017). liureg: A comprehensive R Package for the Liu Estimation of Linear Regression Model with Collinear Regressors. The R Journal, 9 (2), 232–247.

Examples

mod <- liu(y~., as.data.frame(Hald), d = c(-2, -1.47218, 0, 0.5, 1) )
summary(mod)
## coefficients for first biasing parameter
summary(mod)$summaries[[1]]$coefficients
summary(mod)$summaries[[1]][[1]]
## Liu related statistics from summary function
summary(mod)$summaries[[1]]$stats
mod <- liu(y~., as.data.frame(Hald), d = c(-2, -1.47218, 0, 0.5, 1) )
summary(mod)
## coefficients for first biasing parameter
summary(mod)$summaries[[1]]$coefficients
summary(mod)$summaries[[1]][[1]]
## Liu related statistics from summary function
summary(mod)$summaries[[1]]$stats

Variance-Covariance Matrix for Fitted Liu Model

Description

The vcov function computes the variance-covariance matrix for the estimates of linear Liu regression model Liu (1993) <doi:10.1080/03610929308831027>.

Usage

## S3 method for class 'liu'
vcov(object, ...)## S3 method for class 'liu'
vcov(object, ...)

Arguments

`object`	For `VCOV` method, an object of class "liu", i.e., a fitted model.
`...`	Not presently used in this implementation.

Details

The vcov function computes variance-covariance matrix for scalar or vector value of biasing parameter $d$ provided as argument to liu function.

Value

A list of matrix of estimated covariances in the linear Liu regression model for scalar or vector biasing parameter dd is produced. Each list element has row and column names corresponding to the parameter names given by the coef(mod). List items are named correspond to values of biasing parameter $d$ .

Note

Covariance will be without intercept term, as intercept term is not penalized in the Liu regression.

Author(s)

Muhammad Imdad Ullah, Muhammad Aslam

References

Imdadullah, M., Aslam, M., and Altaf, S. (2017). liureg: A comprehensive R Package for the Liu Estimation of Linear Regression Model with Collinear Regressors. The R Journal, 9 (2), 232–247.

Kaciranlar, S., Sakalhoglu, S., Akdeniz, F., Styan, G., and Werner, H. (1999). A new Biased Estimator in Linear Regression and a Detailed Analysis of the Widely Analysed Dataset on Portland Cement. Sankhya: The Indian Journal of Statistics, Series B, 61 (B3), 443–459.

Examples

data(Hald)
mod<- liu(y~., data = as.data.frame(Hald), scaling = "centered", d = seq(-2, 2, 0.1) )

vcov(mod)
data(Hald)
mod<- liu(y~., data = as.data.frame(Hald), scaling = "centered", d = seq(-2, 2, 0.1) )

vcov(mod)

Package 'liureg'

Help Index

Liu Regression and Estimator

Description

Details

Author(s)

Computation of Liu Biasing Parameter ddd

Description

Usage

Arguments

Details

Value

Author(s)

References

See Also

Examples

Portland Cement benchmark of Hald(1952)

Description

Usage

Format

Details

Source

References

Liu Regression: Hat Matrix

Description

Usage

Arguments

Details

Value

Note

Author(s)

References

See Also

Examples

Model Selection Criteria for Liu Regression

Description

Usage

Arguments

Details

Value

Author(s)

References

See Also

Examples

Liu Regression and Estimator

Description

Usage

Arguments

Details

Value

Note

Author(s)

References

See Also

Examples

Liu Regression Statistics

Description

Usage

Arguments

Details

Value

Author(s)

References

See Also

Examples

Bias Variance and MSE Trade-off Plot

Description

Usage

Arguments

Details

Value

Author(s)

References

See Also

Examples

Model Selection Criteria Plots

Description

Usage

Arguments

Details

Computation of Liu Biasing Parameter $d$