The conventional “Godmother of AI” narrative credits Li primarily with ImageNet (2009). That dataset was decisive — it proved data abundance and structural annotation, not isolated algorithmic genius, were the binding constraint on visual intelligence. But the deeper legacy is the human and institutional lattice through which that data-centric, spatially-grounded ethos propagated.

During Andrej Karpathy’s PhD (2011–2015) under Li at the Stanford Vision Lab, the pair produced foundational work on large-scale video classification with CNNs (2014) and deep visual-semantic alignments (2015). These papers moved the field from static 2D classification toward spatio-temporal reasoning and multimodal grounding — the exact nexus that now defines world models. Karpathy and Li also co-designed and taught CS231n, the first deep learning course at Stanford, which scaled from 150 to hundreds of students and enforced a grueling, from-scratch understanding of architectural determinism.

That lattice extended outward: Olga Russakovsky carried the data-curation ethos into Princeton and AI4ALL; Justin Johnson advanced neural rendering; Yunzhu Li (postdoc under Fei-Fei) now leads PointWorld at Columbia, demonstrating that universal 3D point-flow representations outperform embodiment-specific control for in-the-wild manipulation. Karpathy himself took the spatio-temporal intuition into Tesla’s data engine for 4D trajectory prediction and later backed Simile AI (with Li) to simulate human behavioral dynamics at scale.

Li’s consistent thesis across ImageNet, CS231n, and the 2014–2015 papers is that an algorithm is only a lens; the resolution of the resulting intelligence is determined by the structural fidelity of the data it processes. This is not a scaling claim. It is an architectural one: the substrate must encode the right invariances (spatial, temporal, semantic) before any optimizer can discover useful representations.

The same logic now governs the shift from language-model abstraction to spatial intelligence. Language models absorb the statistical structure of human thought. Spatial systems must absorb the physics of space and time — how light falls on occluded surfaces, how objects respond to force, how state persists outside the camera frustum.

In “A Functional Taxonomy of World Models,” Li and the World Labs team impose mathematical clarity on the overloaded term by anchoring it in the classic POMDP agent–world loop (Sutton & Barto tradition). The three functional projections are:

The structural thesis is unforgiving: a system that cannot simulate the physical, geometric, and dynamical constraints of a state space is not a world model. It is a shadow generator. A planner trained inside a shadow generator will fail when it touches reality.

Li positions simulation as the bridge between the visual beauty of renderers and the action space of planners. It is the contract that enforces conservation laws, collision responses, and object permanence — the objective reality against which any agent’s policy must be tested. Current industry skew (tens of billions into video generation + humanoid demos) has created a structural under-investment in this layer. The result is brittle planners and hallucinated physics.

NVIDIA’s Omniverse and Cosmos efforts, World Labs’ Marble, and academic work like PointWorld all converge on the same recognition: controllable, physics-annotated 3D generation and hybrid neural-analytic simulation engines are the highest-leverage infrastructure for closing the sim-to-real gap at scale.

World Labs (Li co-founder/CEO, >$230M raised, >$1B valuation) is the commercial vehicle for this thesis. Marble, their first public artifact, is a multimodal-prompted generative world model that deliberately collapses the renderer–simulator boundary. It accepts text, image, video, or “Chisel Mode” geometric primitives and outputs both Gaussian splats (photorealistic visual substrate) and aligned triangle collision meshes (physical substrate) — the exact dual representation required by Isaac Sim / MuJoCo pipelines.

This is “described datasets” replacing hand-authored curated environments. It enables essentially infinite domain randomization while preserving metric accuracy and rigid-body dynamics. The remaining frontiers (self-intersections, long-horizon scale consistency, multi-physics cost) are acknowledged research problems, not marketing claims.

Capital has flowed overwhelmingly to visually impressive renderers and charismatic planner demos. Li’s taxonomy reveals this as a misallocation. The durable economic moat for physical AI lies in the simulator layer — whoever controls high-fidelity, editable, physics-grounded world models will dictate the speed and safety at which reliable planners can be trained and deployed. Founders and allocators who treat simulation fidelity as first-class infrastructure capture the value of the entire downstream ecosystem.

1. Cease conflating visual fidelity with structural fidelity. A planner trained only on renderer outputs learns statistical heuristics, not Newtonian laws. Export meshes, not just pixels.
2. Treat simulation as the critical path past data scarcity. The physical world is too slow and dangerous to label at the volume required. Synthetic, physics-annotated “described datasets” are the only mathematically viable scaling path.
3. Embrace universal state-action representations. Embodiment-specific control schemes limit generalization. 3D point flows and shared spatial substrates (as in PointWorld) enable one simulator to serve multiple morphologies.
4. Capitalize on the missing middle. The highest-leverage opportunity for sovereign builders is the unglamorous tooling, ingestion pipelines, and hybrid engines that improve simulator latency, multi-physics cost, and physical accuracy guarantees.