Unlocking the Secrets of Learning: What AI Can Learn from Babies

Soul Machines
3 min readApr 5, 2024


As the rapid progression of technology continues, we find ourselves faced with a critical challenge — how do we create artificial intelligence (AI) systems that can learn, reason, and develop common sense, yet remain steerable and safe? And as AI becomes more embedded in our daily interactions, how we interact with it becomes increasingly important.

This is no easy feat, but the key to unlocking the secrets of advanced, adaptable AI may lie in studying the most remarkable learning machines of all — human infants.

In a captivating presentation on Objective-Driven AI, Yann LeCun, VP & Chief of AI Scientist at Meta, shared insights that challenge us to look to the innate learning abilities of babies as a blueprint for the next generation of AI. From the moment they enter the world, infants embark on a remarkable journey of cognitive development, gradually building an understanding of the physical and social environments around them.

Objective-Driven AI presentation by Yann LeCun, VP & Chief of AI Scientist, Meta
Figure 1: Objective-Driven AI presentation by Yann LeCun, VP & Chief of AI Scientist, Meta, 3/28/24. Making AI systems that can learn, remember, reason, plan, have common sense, yet are steerable and safe.

Now imagine an AI that we can teach naturally — in the way that we do with children — not through brute-force algorithms or narrow specialization, or crunching big data — but through interaction : Infants’ learning is both self-guided, driven by infants’ own curiosity and experiences and equally also interactive, driven by targeted real-time input from caregivers

Through a groundbreaking research project from Dr Mark Sagar, CSO and Co-founder of Soul Machines, called Baby X, you can see exactly how this remarkable approach to interactive AI becomes possible.

Figure 2. Baby X learns through multimodal shared interaction with the real world and the digital world — by Dr. Mark Sagar. CSO and Co-founder Soul Machines.

Through multimodal perception — integrating sights, sounds, touch, and social interaction — Baby X creates a feedback loop with the caregiver that facilitates human-like interaction and learning.

These early building blocks of learning ground what is learnt through direct experience.

It is this holistic, experience-driven approach to understanding the world through interaction — and social interaction that holds the key to creating AI systems that interact and cooperate with us like in the intuitive ways we can with other people.

Of course, replicating the nuanced and dynamic learning process of human infants within artificial constructs is no simple task. Ethical considerations and stringent safeguards must be at the forefront, ensuring that as we imbue machines with greater autonomy and reasoning capabilities, we do so in a way that is responsible, transparent, and aligned with human values.

Yet, the potential rewards are immense.

Today, Soul Machines bridges the gap between human and machine learning through its pioneering Biological AI technology R&D. This technology extends the world of generative AI by grounding it in experience — providing it with multimodal context to produce more relevant outcomes, and creates a uniquely engaging and empathetic system you can interact with, just like you would a real person.

Creating a more human-like and truly interactive user experience with an embodied agent is the real differentiator in Biological AI and Soul Machines offers a glimpse of what we are developing in R&D, through Soul Machines Studio. Within Soul Machines Studio, you can create Digital People™ that are embodied agents that you can create and configure yourself. They can interact with personality in a human-like way.

The future is now.