Evaluate Variable Selection Performance

This function evaluates the performance of variable selection methods using the output from select_variables(). It computes the True Discovery Rate (TDR) and False Discovery Rate (FDR) for each selection method based on known non-zero and zero parameters in the model.

Usage

evaluate_selection(selected)

Arguments

selected

A list output from select_variables(), containing:

• mle: Variables selected by the MLE method (if applicable).

• abr_horseshoe: Variables selected by the ABR Horseshoe method (if applicable).

• abr_ridge: Variables selected by the ABR Ridge method (if applicable).

• abr_lasso: Variables selected by the ABR Lasso method (if applicable).

• parameters: The original model parameters, where non-zero values indicate true effects.

• methods: The methods used for selection.

Value

A list containing:

• tdr: A list of True Discovery Rates (TDR) for each method.

• fdr: A list of False Discovery Rates (FDR) for each method.

Details

The evaluation is performed by comparing the selected variables against the known non-zero and zero parameters:

• True Discovery Rate (TDR): The proportion of correctly identified non-zero parameters.

• False Discovery Rate (FDR): The proportion of incorrectly identified zero parameters.

For each method in selected$methods, the function calculates:

• TDR = (number of correctly identified non-zero variables) / (total number of non-zero variables)

• FDR = (number of incorrectly identified zero variables) / (total number of zero variables)

The results are summarized and printed for user review.

Note

This function assumes the parameters field in selected contains the true model parameters, where non-zero values represent true effects and zero values represent noise or irrelevant effects.

See also

select_variables() for selecting variables based on significance criteria.

Examples

# Example selected variables from select_variables
selected <- list(
  mle = c("send_z1", "difference_z2"),
  abr_horseshoe = c("send_z1"),
  abr_ridge = c("send_z1", "difference_z2", "inertia"),
  abr_lasso = c("send_z1"),
  parameters = list(
    send_z1 = 0.5,
    difference_z2 = 0.2,
    inertia = 0,
    reciprocity = 0
  ),
  methods = c("mle", "abr_horseshoe", "abr_ridge", "abr_lasso")
)

# Evaluate selection performance
performance <- evaluate_selection(selected)
#> Selection Performance:
#>    MLE:
#>      TDR:  1 
#>      FDR:  0 
#>    ABR Horseshoe:
#>      TDR:  0.5 
#>      FDR:  0 
#>    ABR Ridge:
#>      TDR:  1 
#>      FDR:  0.5 
#>    ABR Lasso:
#>      TDR:  0.5 
#>      FDR:  0 

# View results
print(performance$tdr)  # True Discovery Rates
#> $mle
#> [1] 1
#> 
#> $abr_horseshoe
#> [1] 0.5
#> 
#> $abr_ridge
#> [1] 1
#> 
#> $abr_lasso
#> [1] 0.5
#> 
print(performance$fdr)  # False Discovery Rates
#> $mle
#> [1] 0
#> 
#> $abr_horseshoe
#> [1] 0
#> 
#> $abr_ridge
#> [1] 0.5
#> 
#> $abr_lasso
#> [1] 0
#>