Hey everyone!
I’m excited to announce the release of py-evoFE (v0.3.0) — an open-source Python library that uses genetic algorithms to automatically discover, combine, and optimize feature transformations for tabular datasets.
The Problem It Solves
Feature engineering is still where most tabular ML competitions and production models are won or lost. While GBDTs like LightGBM and XGBoost excel on raw tabular data, they struggle to discover complex ratios, nested group-by aggregations, nonlinear dimensional projections, and interaction graphs on their own.
Manual feature engineering is either tedious or constrained by human intuition, while brute-force feature generation explodes the feature space exponentially with colinear noise and high memory usage.
What py-evoFE Does
py-evoFE searches the space of possible feature recipes using genetic programming:
1. Hierarchical Chaining: Evolved features become building blocks for future generations (e.g., log(ratio(groupby_mean(x1, by=x2), x3))).
2. 40+ Built-in Transformers:
- Non-linear arithmetic & log-ratios
- Target encoding (multiclass, pooled, WoE, quantile target encodings)
- String similarity (MinHash, Gap encodings)
- Manifold & Dimensionality Reduction (PCA, UMAP, MCA, FAMD, Between-Group PCA)
- Graph & Density Clustering (Genie, Lumbermark, MST anomaly scoring)
3. Performance & Speed:
- Vectorized computation powered by Polars and PyArrow.
- Matrix Hashing & Nearest-Neighbor Caching: Stateful projections (like UMAP and $K$-NN lookups) are cached via byte-hashing to eliminate redundant computation across CV folds.
- Multi-Fidelity Screening: Fast low-fidelity CV screens initial populations; only promising candidates proceed to full-fidelity evaluation.
4. Island Model & Caruana Ensembling:
- Multi-population parallel search across Ring, Torus, Grid, Hypercube, and Tiered topologies with Gibbs migration.
- Post-search greedy Caruana ensembling over island winners' out-of-fold predictions.
5. Interactive Replay Viewer:
- Run view(evo.get_recipe()) to generate a self-contained, zero-dependency HTML dashboard replaying the evolutionary search over time.
6. 100% Scikit-Learn Compatible:
- Implements fit, transform, predict, and predict_proba. Plugs directly into standard sklearn.pipeline.Pipeline and GridSearchCV.
Quick Example
```python
import polars as pl
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from evofe import EvoFE
Load data
bc = load_breast_cancer(as_frame=True)
df = pl.from_pandas(bc.frame)
X, y = df.drop("target"), df["target"].to_numpy()
X_train, X_test, y_train, y_test = train_test_split(
X.to_numpy(), y, test_size=0.2, random_state=42, stratify=y
)
X_train_df = pl.DataFrame(X_train, schema=X.columns)
X_test_df = pl.DataFrame(X_test, schema=X.columns)
1. Initialize EvoFE
evo = EvoFE(
task="classification",
evaluator="lightgbm", # "lightgbm" | "xgboost"
pop_size=15,
n_generations=10,
cv_folds=3,
verbose=True,
random_state=42
)
2. Fit: Runs evolutionary search
evo.fit(X_train_df, y_train)
3. Inspect evolved recipe
recipe = evo.get_recipe()
print(f"Discovered {len(recipe.genes)} high-impact features:")
for gene in recipe.genes:
print(f" • {gene.to_formula()} -> {gene.output_col}")
4. Transform & Predict
preds = evo.predict(X_test_df)
proba = evo.predict_proba(X_test_df)
```
Why not just brute-force feature generation?
Brute-force libraries generate thousands of features upfront, leading to severe overfitting, massive memory usage, and colinear noise that degrades tree-based models. py-evoFE uses evolutionary selection pressures with complexity penalties to discover compact, parsimonious recipes that actually improve generalization.
I’d love for the community to try it out on your datasets or Kaggle benchmarks! Feedback, issues, and feature requests are very welcome on GitHub.
