đź“„ Papers
Core references and recent advances in sample-efficient learning • 🕸️ View Graph
🌍 World Models & Model-Based RL
DreamerV3: Mastering Diverse Domains through World Models Core
Third generation Dreamer. World model + actor-critic trained from replayed experience. Key baseline in HICA.
Director: Deep Hierarchical Planning from Pixels Core
Hierarchical extension of Dreamer with worker-manager architecture. Key comparison model in HICA.
Clockwork VAE (CWVAE): Temporal Abstraction in Sequences Core
Hierarchical extension of RSSM with multiple temporal scales. Inspiration for HICA architecture.
S4WM: State Space World Models Core
State-space models for world modeling. Alternative architecture to RNN-based approaches.
MuZero: Mastering Games by Planning with Learned Models Core
Planning with learned world model. Mastered Atari, Go, Chess, Shogi without knowing rules.
World Models Core
The foundational paper. Learn world model in latent space, train controller entirely in imagination.
Dreamer 4: Training Agents Inside of Scalable World Models New
First agent to get diamonds in Minecraft from offline data. Scalable imagination training.
Accelerating MBRL with State-Space World Models New
10Ă— faster world model training, 4Ă— faster overall MBRL.
THICK: Learning Hierarchical World Models with Adaptive Temporal Abstractions New
Temporal Hierarchies from Invariant Context Kernels. Lower level forms invariant contexts via discrete latent dynamics, higher level predicts context changes.
🔄 Complementary Learning Systems
Why There Are Complementary Learning Systems in the Hippocampus and Neocortex Core
Foundational CLS theory paper. Explains hippocampal fast learning + neocortical slow consolidation.
What Learning Systems Do Intelligent Agents Need? Core
Modern update to CLS theory. Framework for AI systems inspired by hippocampus-neocortex.
FSC-Net: Fast-Slow Consolidation Networks New
Dual-network: fast learner + slow learner inspired by memory consolidation.
Semantic and Episodic Memories in Predictive Coding New
Semantic memory in neocortex, episodic memory resembles hippocampal CA3.
ComS2T: Complementary Spatiotemporal Learning New
Stable neocortex for consolidation + dynamic hippocampus for new knowledge.
🏗️ Hierarchical Reinforcement Learning
Neural History Compressor Core
Pioneering hierarchical sequence learning. Higher levels receive only unpredictable (surprising) inputs—automatic temporal abstraction through compression.
A Clockwork RNN Core
Hidden layer partitioned into modules running at different clock rates (1, 2, 4, 8... steps). Architectural inductive bias for multi-timescale temporal processing.
Phased LSTM: Accelerating Recurrent Network Training Core
Adds learned time gate with oscillating open/close periods. Only updates cell state during open phases—handles irregular/event-driven sequences and long time lags efficiently.
Hierarchical Multiscale Recurrent Neural Networks Core
Learns to discover latent hierarchical structure via boundary detection. Higher layers update only at detected boundaries—adaptive temporal abstraction without predefined timescales.
Recent Advances in Hierarchical Reinforcement Learning Core
Foundational survey on HRL. Temporal abstraction and options framework.
Introduction to Reinforcement Learning Core
The RL textbook. Foundation for all modern RL research.
Emergent Temporal Abstractions in Autoregressive Models New
"Internal RL" enables hierarchical RL within foundation models.
Exploring the Limits of Hierarchical World Models in RL New
Analyzes HMBRL: combining MBRL sample efficiency with HRL abstraction. Comprehensive study of hierarchical model-based RL.
🧠Spatial Memory & Navigation
Memory Maze: A Benchmark for Long-Term Memory Core
Evaluation platform for HICA. Tests rapid long-term memory acquisition.
Clone Structured Causal Graph (CSCG) Core
Hidden Markov Model approach. Efficiently learns simplified action spaces.
Grid Cells in the Entorhinal Cortex Core
Biological spatial representation. Inspiration for cognitive maps in AI.
The Hippocampus as a Predictive Map Core
Seminal HICA paper. Successor representation in hippocampus for predictive coding of future states.
The Tolman-Eichenbaum Machine Core
Unifying space and relational memory. Transformer-like attention in hippocampal formation.
Improving Generalization: The Successor Representation Core
The root of Successor Representation. Foundation for predictive map theories.
🏛️ Predictive Coding & Cortical Theory
A Brief History of Intelligence Core
Traces 5 evolutionary breakthroughs in intelligence (steering, reinforcing, simulating, mentalizing, speaking) and connects them to modern AI.
On Intelligence Core
Foundational book proposing the Memory-Prediction Framework. Intelligence = ability to predict the future.
Canonical Microcircuits for Predictive Coding Core
Predictive coding in cortical columns. Foundation for hierarchical brain models.
Hierarchy or Heterarchy: Long-Range Cortical Theory New
Cortical columns learn structured world models for prediction.
Brain-inspired Intelligence via Predictive Coding New
Neural Generative Coding (NGC) for cortical functionality.
🤖 Robot Learning & Sample Efficiency
Towards Sample Efficient Reinforcement Learning Core
Defines the sample efficiency challenge. Key motivation for HICA research.
DayDreamer: World Models for Physical Robot Learning Core
Dreamer applied to real robots. Quadruped and wheeled robot navigation.
EmbodiSwap for Zero-Shot Robot Imitation New
Human→robot swap. 82% zero-shot success rate.
Annotation-Free One-Shot Imitation New
Multi-step tasks from single demonstration.
🎯 Exploration & Intrinsic Motivation
Intrinsically Motivated Reinforcement Learning Core
Foundation of curiosity-driven learning. Internal reward signals.
Planning to Explore via Self-Supervised World Models Core
Plan2Explore: curiosity via world model disagreement.
VIME: Variational Information Maximizing Exploration Core
Prior/posterior belief comparison for exploration. Used in HICA.
🎮 Game AI Benchmarks
AlphaGo: Mastering Go without Human Knowledge Core
Self-play without human data. Demonstrates power of MCTS + neural networks.
AlphaStar: Grandmaster Level in StarCraft II Core
Complex real-time strategy. Example of sample inefficiency problem.
🕸️ Paper Relationship Graph
Click nodes for details • Drag to move • Scroll to zoom