Neural Latents Benchmark

Logo

A Benchmark for Models of Neural Data


About
Datasets
NLB'21 Challenge Info
NLB'21 Codepack
NLB'21 EvalAI

Overview

Advances in neural recording present increasing opportunities to study neural activity in unprecedented detail. Latent variable models (LVMs) are promising tools for analyzing this rich activity across diverse neural systems and behaviors, as LVMs do not depend on known relationships between the activity and external experimental variables. However, progress in latent variable modeling is currently impeded by a lack of standardization, resulting in methods being developed and compared in an ad hoc manner. To coordinate these modeling efforts, we introduce the Neural Latents Benchmark (NLB). In the first benchmark suite, NLB 2021, participating models are evaluated on 7 datasets of neural spiking activity spanning 4 tasks and brain areas. While the benchmark will be available indefinitely, the challenge will close January 7, 2022. To get started with the challenge, follow the links on the left.

FAQ

How do I submit a model to the benchmark?

We are hosting our challenge on EvalAI, a platform for evaluating machine learning models. On the platform, you can choose to make private or public submissions to any or all of the individual datasets.

Can I view the leaderboard without submitting?

Yes, the full leaderboard will be available on EvalAI, and EvalAI is also synced with Papers With Code. Model open-sourcing is encouraged and thus may be available through the leaderboard.

Is there a deadline?

The benchmark and its leaderboard can be submitted to indefinitely on EvalAI as a resource for the community. However, the winners of the challenge will be determined from the leaderboard on January 7th, 2022. Prizes for the challenge will be distributed to the winner shortly thereafter.

Is NLB one benchmark or many benchmarks?

NLB aims to regularly organize benchmark suites, a collection of tasks, datasets, and metrics around a theme in neural latent variable modeling. For example, NLB’21 will emphasize general population modeling.

Contact

The Neural Latents Benchmark is being led by the Systems Neural Engineering Lab in collaboration with labs across several universities. General inquiries should be directed to [Dr. Pandarinath] at chethan [at] gatech [dot] edu.