Mind Reading 101

For research purposes, the cortex is mapped onto a standardized surface called fsaverage5: a reference template with 20,484 vertices, each one a point on the cortical sheet that can be tracked across subjects. When neuroscientists talk about “a region of the brain doing X,” they usually mean the average activity at some collection of these vertices, across many people, doing X.

In 2011, Yeo and colleagues at Harvard published a reproducible parcellation of this surface[1]. They didn't decide what the cortex should look like; they let the cortex's resting activity reveal its own organization. That map is now one of the most reproduced findings in cognitive neuroscience.

Ability tests like the MSCEIT score people on emotion problems where the “correct” answer is whatever the majority of experts say.[2] Consensus-based, which is to say circular. Self-report questionnaires (EQ-i, TEIQue) ask people what they think they're good at — measuring reputation, not capacity, and overlapping heavily with personality and IQ.[3] Informant ratings (ESCI) ask the people around someone — measuring social legibility, which is real but not the same thing.

After thirty-five years and dozens of meta-analyses, the numbers don't agree, and the field cannot say what its central object is.

Marcus's father just died. What should you say to him?

American workplace: something direct — I'm so sorry, let me know if there's anything I can do. Japanese workplace: silence and a formal condolence card; direct emotional language would be intrusive. A stoic colleague: ask Marcus about a work project — distraction as kindness. A therapist: ask Marcus what was coming up for him.

All four are defensible. Any “correct” answer a test assigns smuggles in a cultural norm, a professional norm, or an individual preference.

In 2023, EQ-Bench began scoring large language models on emotional reasoning. The benchmark correlated 0.97 with MMLU, the standard general-knowledge benchmark.[4] Whatever EQ-Bench was measuring was almost identical to general reasoning. The field had been measuring general cognitive ability with emotional wallpaper.

That sentence has stood for eight years. It still stands.

Individual brains differ enormously in their fine-grained activity. The response patterns across cortex turn out to be surprisingly stable. MindReader doesn't measure your brain — it predicts the response of the average human cortex across the 720+ subjects in TRIBEv2's training set.[5] When we say "the cortex did X," we mean: averaged across those people, the cortex did X.

For content evaluation this is exactly the right unit of analysis. What someone making a video, ad, or script actually wants to know isn't how will this one person respond? — it's how will humans on average respond? A focus group is one (broken) way to ask. The cortex, simulated at the population level, is another.

For most of the history of cognitive neuroscience, the way you studied a brain was by giving it a tightly controlled task. Identify the face. Remember the word. Solve the puzzle. Beautiful science — reproducible, falsifiable. It also bore almost no resemblance to how brains actually spend their time.

Hasson's paradigm — “naturalistic stimuli” — is the methodological bridge between the lab task and the real world. Without it, the seven cortical systems MindReader exposes would only be defined by how they respond to flashcards. With it, we can see them respond to content as it is actually consumed. The entire training-data regime MindReader inherits is downstream of this shift.

In 2021, Schrimpf and colleagues at MIT showed that the internal representations of large language models — GPT-2, BERT, others — predicted human brain activity in language regions, with surprising accuracy.[7] The same statistical structure that made language models work showed up in cortex.

In 2022, Goldstein and colleagues at Princeton went further: LLMs and human brains share computational principles for language — predicting the next word, integrating context, building meaning across time.[8] Not just similar outputs. Similar processing.

That same year, Caucheteux and King in Paris formalized “brain score”: a metric for how brain-like a model's representations are.[9] A way to benchmark not what models can do, but how closely they resemble what cortex does on the same input.

In 2023, Tang and the Huth lab at UT Austin demonstrated semantic decoding from fMRI: continuous reconstruction of language from brain recordings, using a language model as the decoding bridge.[10] Not telepathy — but proof that the cortex-to-model mapping was real, bidirectional, and useful.

Each paper added one move. By 2024 the open question wasn't whether encoding models could predict brain response — it was whether the technique could extend beyond text, to images, audio, and video.

Video is the hard case because audio, visuals, and language arrive simultaneously and interact — change the narration over the same image and the cortex responds differently. TRIBEv2 handles all three streams together: three frozen feature extractors (LLaMA 3.2-3B, V-JEPA2, Wav2Vec-BERT) process each stream; a temporal transformer fuses them; a subject-specific projection maps the fused signal onto cortex.

Output, per input video: 20,484 cortical points × 1Hz of predicted BOLD response. One number per cortical point per second.

Honest disclosure: this is predicted response, not measured. Out-of-distribution stimuli — pure music, abstract animation, content unlike the training set — produce noisier predictions.