Expanding Protein Fitness Landscape Benchmarks for Real-World Machine Learning Applications

Overview

FLIP2 significantly expands the original FLIP benchmark with new datasets that better represent the diversity of protein engineering applications. While FLIP focused on thermostability, binding, and viral capsid viability, FLIP2 introduces datasets covering enzymatic activity, protein-protein interactions, and optogenetics applications.

The benchmark includes 16 distinct train-test splits across 5 categories that simulate realistic protein engineering scenarios: generalization to more mutations (number splits), to mutations at different sequence positions (position splits), to unseen mutations (mutation splits), to higher fitness variants (fitness splits), and to different wild-type proteins (wild-type splits).

The FLIP2 Datasets

FLIP2 includes seven datasets covering diverse protein functions and engineering challenges. Each dataset contains experimentally measured fitness values for hundreds to hundreds of thousands of protein variants.

Split Types

FLIP2 includes five categories of splits that test different types of generalization relevant to protein engineering:

Baseline Performance

We evaluated zero-shot protein language model likelihood scores, ridge regression baselines, and fine-tuned pLMs on all FLIP2 splits. Performance is measured using Spearman rank correlation and normalized discounted cumulative gain (NDCG) on held-out test sets. These baselines confirmed that FLIP2 splits are more challenging than random splits with the same number of training examples.

Data Formats

FLIP2 datasets are provided in two formats: CSV for regression/classification tasks and FASTA for sequence-based tasks. All data files are gzip-compressed (.csv.gz and .fasta.gz) for efficient storage and transfer.

CSV Format (Gzipped)

CSV files (*.csv.gz) contain four columns:

• sequence: Amino acid sequence of the variant
• target: Measured fitness value (continuous or binned)
• set: Data split assignment ("train", "test", or "validation")
• validation: Boolean indicating if the sequence is in the validation set

Example:

sequence,target,set,validation
MKTAYIAKQRQISFVKSHFSRQLEERLGLIEVQAPIL,0.5123,train,False
MKTAYIAKQRQISFVKSHFSRQLEERLGLIKVQAPIL,-0.2341,test,False
MKTAYIAKQRQISFVKSHFSRQLEERLGLIEVQAPIL,0.6789,train,True

FASTA Format (Gzipped)

FASTA files (*.fasta.gz) include metadata in the header line:

• Header format: >[ID] TARGET=[value] SET=[train/test/validation] VALIDATION=[True/False]
• Sequence follows on subsequent lines

Example:

>Q5LL55 TARGET=0.0 SET=train VALIDATION=False
MKTAYIAKQRQISFVKSHFSRQLEERLGLIEVQAPIL
>H9L4N9 TARGET=1.0 SET=test VALIDATION=False
MKTAYIAKQRQISFVKSHFSRQLEERLGLIKVQAPIL

Loading Data

Example Python code to load FLIP2 data (gzipped files):

import pandas as pd

# Load gzipped CSV format (pandas handles .gz automatically)
df = pd.read_csv('assets/splits/amylase/one_to_many.csv.gz')

# Split into train/test
train_df = df[df['set'] == 'train']
test_df = df[df['set'] == 'test']
validation_df = df[df['validation'] == True]

# Alternative: Load FASTA format
from Bio import SeqIO
import gzip

with gzip.open('assets/splits/bind/one_vs_many.fasta.gz', 'rt') as f:
    for record in SeqIO.parse(f, 'fasta'):
        # Parse metadata from header
        metadata = dict(item.split('=') for item in record.description.split()[1:])
        sequence = str(record.seq)
        target = float(metadata['TARGET'])
        split = metadata['SET']

Download Data

All FLIP2 datasets and splits are available for download. All data files are gzip-compressed for efficient transfer. Each dataset includes a README with complete provenance and attribution information. All data is licensed under CC-BY 4.0 or MIT (see individual README files).

Additional Resources

• GitHub Repository - Code for data processing, baselines, and evaluation
• FLIP2 Manuscript - Full paper with detailed methods and results
• Original FLIP Paper - Original 2021 FLIP benchmark paper

Legacy FLIP Datasets

The original FLIP benchmark (2021) included datasets for AAV capsid viability, GB1 domain stability/binding, and thermostability across multiple protein families. These datasets remain available for continuity and comparison.

Citation

If you use FLIP2 in your research, please cite:

@article{didi2025flip2,
  title={FLIP2: Expanding Protein Fitness Landscape Benchmarks for Real-World Machine Learning Applications},
  author={Didi, Kieran and Alamdari, Sarah and Lu, Alex X. and Wittmann, Bruce and Johnston, Kadina E. and Amini, Ava A. and Madani, Ali and Czeneszew, Maya and Dallago, Christian and Yang, Kevin K.},
  journal={},
  year={2026}
}

Please also cite the original FLIP paper:

@article{dallago2021flip,
  title={FLIP: Benchmark tasks in fitness landscape inference for proteins},
  author={Dallago, Christian and Mou, Jody and Johnston, Kadina E and Wittmann, Bruce J and Bhattacharya, Nicholas and Goldman, Samuel and Madani, Ali and Yang, Kevin K},
  journal={Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
  year={2021}
}

Individual Dataset Citations: Each dataset has specific attribution requirements. Please see the README file in each dataset directory for proper citations and licenses.