Inworld AI, a company that aims to develop a platform for users to create AI-driven virtual characters that can be used to populate virtual worlds, today announced that it has raised $ 7 million in seed funding. .

In an exclusive interview, Inworld co-founder and CEO Ilya Gelfenbeyn explained that “Inworld AI is a platform to build, essentially brains for virtual characters” to populate virtual environments, including metaverse worlds. , VR and AR. “What we are providing is a set of tools that allows developers to add brains and create these characters for the world, for different types of environments.”

To successfully create immersive characters, Inworld AI attempts to mimic human cognitive abilities by leveraging a blend of AI technologies such as natural language understanding and processing, optical character recognition, reinforcement learning and conversational AI to develop sophisticated virtual characters – characters who can even answer questions and carry on conversations.

Inworld AI does not develop a solution for designing visual avatars, but rather aims to create an AI development platform that allows companies that produce digital avatars and virtual characters to add more advanced communication to their designs. visual.

The end goal of the platform is to offer a platform that visual avatar providers and organizations can use to develop “characters who can interact naturally with a wide and completely open dialogue,” Gelfenbeyn said. . Although speaking is only the tip of the iceberg in terms of the communication skills of these AI characters.

As Gelfenbeyn notes, “Inworld characters should not be limited to speech, but be able to interact with many modalities that humans use, such as facial gestures, body language, emotions, as well as physical interactions. “

Improving the metaverse experience with AI brains

“We are structuring our technology stack based on the human brain. We have three main components: perception, cognition and behavior. Perception focuses on the contribution and understanding of the environment and other agents, using senses such as audio and visual, ”said Gelfenbeyn.

To enable virtual characters to perceive the environment both audibly and visually, the organization uses a complex mix of text-to-speech, rule engines, natural language understanding, OCR, and event triggers.

The next element is cognition. “Cognition is about the internal states of the character, such as memory, emotions, personality, goals and background,” he said. Here, Inworld AI will use natural language processing, emotion recognition, reinforcement learning, and goal-oriented conversational AI to enhance the cognitive abilities of virtual characters.

Finally, “behavior relates to the character’s output or interactions, such as speech, body language and movement”. Technologies such as advanced generative language models, reinforcement learning and personalized voice and emotion synthesis, ”allow virtual characters to reproduce human gestures and behaviors.

Together, these three components provide a solid framework for developers to create virtual characters capable of responding in detail to natural language, perceiving the digital environment, and delivering meaningful interactions to users.

Investors include Kleiner Perkins, CRV and Meta. The launch of Inworld AI is timely, with publicity for the Metaverse at an all-time high following the rebranding from Facebook to Meta, and decision makers keen to identify the solutions available to engage with customers in the Metaverse.

As Izhar Harmony, General Partner of CRV explained, “The team is growing rapidly, so it’s an exciting time for people interested in virtual reality, games and virtual worlds to team up and join. company, so that they can be at the forefront of this rapidly growing space.

New kid in the neighborhood

Inworld AI enters the highly competitive AI development and machine learning space and competes with established vendors like Open AI and Google AI, which allow you to build machine learning models, but Inworld AI fills the gap. a unique gap in the market, as it provides a highly specialized solution to develop conversational AI for AI-driven virtual characters, rather than generic machine learning models.

At the same time, the AI ​​solutions developed by Inworld AI will enable the creation of virtual characters that extends far beyond the complexity of AI-driven avatars like Pandora Bots and Soul Machines.

“Many existing companies have solutions that provide limited responses to script triggers and dialogue. In fact, our team has built one of the largest providers of these services (API.ai, acquired by Google and now known as Google Dialogflow), so we are very familiar with their capabilities, ”said Gelfenbeyn.

“Other companies are starting to experiment with new technologies (such as large language models) but we believe these parts, while essential, only provide one piece of the battery needed to really bring characters to life,” did he declare.

In other words, these solutions have only scratched the surface of human-AI interactions, and Inworld AI’s approach to replicating human cognition is designed to create much smarter virtual entities. While Inworld AI’s mission to create AI brains for virtual characters is ambitious, the team’s AI development pedigree speaks for itself.

The founders of Inworld AI include a host of experts such as Gelfenbeyn who was previously the CEO of API.ai, CTO Michael Ermolenko who led the development of machine learning at API.ai and the The Dialogflow NLU / AI team at Google, and Product Manager Kylan Gibbs, who previously led the product for Applied Generative Language Models at DeepMind.

With this experienced team, the organization is in a strong position to set the standard for interactive virtual characters. After all, “the widespread success of the metaverse and other immersive applications depends on how enveloping these experiences can be,” said Ilya Fushman, investment partner at Kleiner Perkins.

“Inworld AI is building the engine that enables businesses to deliver this exciting depth of experience and engage users. With the team’s experience in providing developers with the tools they need to build AI-powered applications, we are excited to help the company build the future of immersive experiences, ”explained Fushman.

Virtual characters are the key to immersion

As the metaverse boom begins to gain momentum, Inworld AI also has a unique role to play in providing vendors with a set of tools they can use to create sophisticated virtual characters and create more compelling digital experiences. for users. The level of immersion offered by these experiences will determine whether the metaverse lives or dies.

The types of experiences developers can use Inworld AI to create are diverse. As Gelfenbeyn explained, “Immersive realities continue to accelerate, with an increasingly diverse and fascinating ecosystem of worlds and use cases. “

“Virtual spaces such as Meta’s Horizon Worlds, Roblox, Fortnite and others that offer unique experiences and allow users to exist in other worlds will also continue to see rapid demand from companies, offering everything, from games to story content to new business applications. ” noted.

While Gelfenbeyn noted that the technology is simply aimed at enabling vendors to create a “native population” for the digital world to deliver realistic experiences, the metaverse is also becoming a new channel that technical decision-makers can use to engage with customers on the go. the future.

While the full and immersive realities with sophisticated virtual characters are far away, the Inworld AI team’s knowledge of conversational AI will undoubtedly allow other vendors to come one step closer to creating dynamic, virtually populated digital worlds. and interactive.