Scaling AI chatbot creation with Chatfuel

Originally posted at nebius.com

Overview

For SaaS companies, delivering an exceptional customer experience is paramount, as it sets the foundation for trust and satisfaction.

Using Nebius AI Studio to leverage a cascade of Llama-405B models along with a custom SDK, leading AI-powered customer engagement automation platform Chatfuel achieved significant efficiency gains with much better response quality and interaction speed for its AI chatbot agents.

Founded in 2015 by two Y Combinator alumni, Chatfuel’s no-code AI-powered platform is designed to automate communication and enhance customer engagement for service businesses.

Partnering with Meta from 2016, the company reaches thousands of clients, integrating its software directly with the tech giant’s messaging platforms and streamlining interactions on WhatsApp, Facebook Messenger, Instagram and websites.

Chatfuel’s AI agents are pre-trained on billions of messages and free up human reps to focus on growth and creativity. Previously, Chatfuel relied solely on GPT-4 models for its chatbot agents. However, it needed more reliable and high-performing models to optimize onboarding processes and enhance the agents' capabilities. This required infrastructure capable of handling demanding training and testing processes.

Through our partnership, Chatfuel leveraged a cascade of state-of-the-art Llama-405B models using Nebius’ infrastructure to reach exceptional chatbot performance. The Nebius team developed this cascade and custom SDK, which allowed Chatfuel to achieve significant efficiency gains with much better response quality and interaction speed.

Balancing cost, performance and speed

Launching the cascade of models into production took just one month using Nebius AI Studio. The Nebius team played a crucial role in developing and testing multiple model versions, helping Chatfuel identify the best balance between performance, cost and speed. This collaboration ensured that Chatfuel’s new models met the strict requirements necessary for real-time chatbot performance and scalability.

Chatfuel utilised Nebius AI Studio for serving the Llama 405B models, allowing them to accomplish various tasks (constrained text generation, text classification, etc.) with state-of-the-art quality. Additionally, Chatfuel used Nebius’s Compute Cloud to host the components of the solutions and evaluate their quality.

Adopting the Llama-405b model also improved the quality of parsing Chatfuel’s clients’ free-form descriptions of the agents they desire and reduced the costs associated with the LLM. Furthermore, it improved the performance over OpenAI’s GPT-4o by 24% in the task of routing the user’s input to the correct agent.

Data handling

Previously, creating effective AI-driven chatbots required large amounts of data to fine-tune the models. However, much of the data Chatfuel works with initially lacked proper labeling, making it challenging to achieve high-quality results.

The Nebius team supported in optimizing the training process, enabling Chatfuel to maintain excellent results even with smaller, well-curated data samples. This approach not only improved the quality of Chatfuel’s solutions, but also saved significant costs and resources, allowing them to deliver powerful chatbot solutions both more efficiently and effectively.

Model evaluation

Chatfuel deploys a meticulous three-step process for evaluating its chatbot models and ensuring optimal results.

1. Nebius team evaluation: The Nebius team conducts an initial assessment using advanced metrics and basic human evaluation to verify the technical performance and reliability of the models.

2. Chatfuel product team review: Chatfuel’s product team evaluates the models within the context of real-world use cases, ensuring they align with product goals and customer needs.

3. Feedback from loyal customers: Finally, they involve their loyal customers to test the models in practical scenarios, gathering their insights to fine-tune and enhance the experience further.

Custom SDK

For SaaS companies, delivering an exceptional customer experience is paramount, as it sets the foundation for trust and satisfaction. Poor customer service ends up costing companies billions of dollars in revenue each year.

Recognizing this, the Nebius team developed a custom SDK that streamlined the integration process, enabling Chatfuel’s engineering team to implement the entire solution into their product in just three days.

This efficiency allowed Chatfuel to focus on refining the customer journey and ensuring seamless performance right from the start.

Chatfuel’s highlights of working with Nebius

• Stable infrastructure: Nebius provides a highly reliable and robust platform to support AI operations.

• Expert guidance: The Nebius team offers exceptional support, aiding in critical AI product decision-making processes.

• Enhanced business outcomes: Collaboration with Nebius has significantly boosted product funnel performance and improved client retention.

Because of its collaboration with Nebius, Chatfuel can now create diverse AI chatbots swiftly, enhancing customer onboarding processes. The company has achieved significant AI efficiency gains with much better response quality, interaction speed, and established efficient routing between different chatbot agents.

Pioneering AI-powered customer engagement solutions

Because of Chatfuel’s partnership with Nebius, the company’s customers are now creating diverse agents for different chatbot tasks in minutes, pioneering AI-powered customer engagement solutions.

Going forward, Chatfuel will continue to utilize Nebius AI Studio and the custom-developed SDK to create more personalized experiences for each customer. This includes better re-engagement through advanced chatbot capabilities as well as improved training and retraining processes for chatbot agents.