Simple Heuristics

Author

A.E. Rodriguez

Gerd Gigerenzer’s fast-and-frugal heuristics are simple, efficient mental shortcuts (rules of thumb) that help people make quick, good-enough decisions with limited information, time, and computational power. This is radically different from the more conventional belief that complex calculations always yield better results.

Giverenzer’s “adaptive toolbox” identifies and relies on evolved psychological capacities to exploit environmental structures.

“Context matters:” and it matters most when we confront un-kind envirornments with bounded rationality and where speed and simplicity are the focus. Under these, commonplace, circumstances, Gigerenzer’s heuristics outperform complex models.

Examples of these adaptive toolbox heuristics are “Take-the-Best.” In take-the-best an analyst stops at the first useful clue. The “Recognition Heuristic” where presented with two options where the analyst recognizes only one of the options then you choose the recognized one. Fast-and-Frugal Trees, are simple, sequential decision trees where questions are asked one question at a time. The sequence ends when a decision can be made. The “Tallying Heuristic” is another simple decision-making strategy where you count the number of features favoring one option over another. Each one of the features is ascribed equal importance. After the tallying is complete the analyst chooses the one with the most “votes.” Put differently, this is a weighted linear average where the weights are equal to 1. It is appealing because it ignores the routine trade-offs that naturally creep in when we appraise different baskets of features.

Modeling

FFTrees and OneR are both R packages designed for creating simple, interpretable classification models, but they differ significantly in their methodology, structure, and complexity. FFTrees creates fast-and-frugal trees—hierarchical, non-compensatory, sequential decision trees. OneR creates a “One Rule” model—the simplest possible rule-based model that selects the single best predictor variable to classify data. 

Methodology

This R script compares the FFTrees and OneR classification algorithms using the built-in breastcancer dataset. It evaluates both models based on accuracy and provides visualizations for each.

Generate synthetic data

Imbalanced sample

options(digits =3, scipen = 9999)
remove(list =ls())
graphics.off()

setwd("C:/Users/arodriguez/Dropbox/RPractice/tools")
suppressPackageStartupMessages({
  suppressWarnings({
    library(tidyverse)
    library(ROSE)
    library(rsample)
    library(modeldata)
    library(probably)
    library(e1071)
    library(yardstick)
    library(pROC)
    library(rms)
    library(FFTrees)
library(OneR)
library(caret)
  })
})

    n = 1500
              
    fraud = rbinom(n, 1, prob = 0.04)  # 4% fraud prevalence 
              
              x1 = rnorm(n, mean = 0 + 2*fraud, sd = 1)
              x2 = rnorm(n, mean = 0 - 1*fraud, sd = 1.5)
              x3 = rbinom(n, 1, prob = 0.2 + 0.5*fraud)
              
              data <- data.frame(fraud = fraud, 
                                   x1 = x1, 
                                   x2 = x2, 
                                   x3 = x3)

              
              data$myfraud = c(data$fraud == 1)
              data$fraud <- as.factor(data$fraud)
              data$myfraud <- as.factor(data$myfraud)
              
# Split into training (70%) and testing (30%) sets
set.seed(42)              
split = initial_split(data, prop = 0.70) 
train_data <- training(split)
test_data <- testing(split)  

head(data)
  fraud     x1     x2 x3 myfraud
1     0 -1.734  1.207  0   FALSE
2     0  1.002 -0.824  0   FALSE
3     0  0.330  1.161  1   FALSE
4     0 -0.953 -0.252  0   FALSE
5     0  0.155  2.322  0   FALSE
6     0 -0.648  1.032  1   FALSE

Fit the FFTrees Model

FFTrees creates fast-and-frugal decision trees

fft_model <- FFTrees(formula = myfraud ~ 
                       x1 + x2 + x3, 
                     data = train_data,
                     data.test = test_data)
! Converted criterion to logical (by 'myfraud == TRUE') in 'train' data.
! Converted criterion to logical (by 'myfraud == TRUE') in 'test' data.
✔ Created an FFTrees object.
  Ranking 3 cues:  ■■■■■■■■■■■■■■■■■■■■■             67% | ETA:  0s
  Ranking 3 cues:  ■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■  100% | ETA:  0s
✔ Ranked 3 cues (optimizing 'bacc').
✔ Created 4 FFTs with 'ifan' algorithm (chasing 'bacc').
✔ Defined 4 FFTs.
✔ Applied 4 FFTs to 'train' data.
✔ Ranked 4 FFTs by 'train' data.
✔ Applied 4 FFTs to 'test' data.
✔ Expressed 4 FFTs in words.

Fit the OneR Model

OneR finds the single best predictor for classification

oner_model <- OneR(fraud ~ x1 + x2 + x3, data = train_data)

Generate Predictions on Test Data

FFTrees predictions (using the best training tree by default)

fft_pred <- predict(fft_model, newdata = test_data |> select(-fraud))
✔ Applied 4 FFTs to 'test' data.
✔ Generated predictions for tree 1.
# OneR predictions
oner_pred <- predict(oner_model, 
                     test_data |> dplyr::select(-myfraud) )
oner_pred = factor(oner_pred, levels = c("0","1"))

Compare Performance

fft_cm  <- confusionMatrix(as.factor(fft_pred), 
                           as.factor(test_data$myfraud))

oner_cm <- confusionMatrix(as.factor(oner_pred), 
                           as.factor(test_data$fraud))

cat("--- Performance Comparison ---\n")
--- Performance Comparison ---
cat("FFTrees Accuracy:", round(fft_cm$overall['Accuracy'], 4), "\n")
FFTrees Accuracy: 0.902 
cat("OneR Accuracy:   ", round(oner_cm$overall['Accuracy'], 4), "\n\n")
OneR Accuracy:    0.967 
cat("--- Confusion Matrix ---\n")
--- Confusion Matrix ---
cat("FFTrees CM:","\n\n")
FFTrees CM: 
fft_cm$table
          Reference
Prediction FALSE TRUE
     FALSE   389    4
     TRUE     40   17
cat("\n")
cat("OneR CM:", "\n\n")
OneR CM: 
oner_cm$table
          Reference
Prediction   0   1
         0 426  13
         1   2   8
cat("FFTrees Recall:")
FFTrees Recall:
yardstick::recall_vec(
  as.factor(test_data$myfraud),
  as.factor(fft_pred), 
)
[1] 0.907
cat("OneR Recall:")
OneR Recall:
yardstick::recall_vec(
  as.factor(test_data$fraud),
  as.factor(oner_pred), 
)
[1] 0.995

Extract Recall (Sensitivity) from the confusion matrices

fft_recall   <- fft_cm$byClass['Sensitivity']
oner_recall  <- oner_cm$byClass['Sensitivity']

cat("--- Recall (Sensitivity) Comparison ---\n")
--- Recall (Sensitivity) Comparison ---
cat("FFTrees Recall: ", round(fft_recall, 4), "\n")
FFTrees Recall:  0.907 
cat("OneR Recall:    ", round(oner_recall, 4), "\n")
OneR Recall:     0.995 
# Alternative: Calculate manually if needed
# Recall = TP / (TP + FN)
fft_table <- fft_cm$table

fft_manual_recall <- fft_table[2,2] / (fft_table[2,2] + fft_table[1,1]) # Assumes 1st level is 'positive'

round(fft_manual_recall, 6)
[1] 0.0419

Visualize the Models

Plot the best FFTree

plot(fft_model, main = "FFTrees: Fraud Detection")

Print the OneR rule

print(oner_model)

Call:
OneR.formula(formula = fraud ~ x1 + x2 + x3, data = train_data)

Rules:
If x1 = (-3.54,-2]    then fraud = 0
If x1 = (-2,-0.469]   then fraud = 0
If x1 = (-0.469,1.06] then fraud = 0
If x1 = (1.06,2.59]   then fraud = 0
If x1 = (2.59,4.13]   then fraud = 1

Accuracy:
1019 of 1050 instances classified correctly (97%)
summary(oner_model)

Call:
OneR.formula(formula = fraud ~ x1 + x2 + x3, data = train_data)

Rules:
If x1 = (-3.54,-2]    then fraud = 0
If x1 = (-2,-0.469]   then fraud = 0
If x1 = (-0.469,1.06] then fraud = 0
If x1 = (1.06,2.59]   then fraud = 0
If x1 = (2.59,4.13]   then fraud = 1

Accuracy:
1019 of 1050 instances classified correctly (97%)

Contingency table:
     x1
fraud (-3.54,-2] (-2,-0.469] (-0.469,1.06] (1.06,2.59] (2.59,4.13]  Sum
  0         * 28       * 304         * 542       * 132           5 1011
  1            0           0            10          16        * 13   39
  Sum         28         304           552         148          18 1050
---
Maximum in each column: '*'

Pearson's Chi-squared test:
X-squared = 300, df = 4, p-value <0.0000000000000002

Key Considerations for Recall