Theia AI Sneak Preview: Choose your own LLM

Most commercial AI-driven tools and IDEs provide you with a preset LLM that you cannot influence as a user or adopter. Usually, this is even combined with a mandatory subscription model and sometimes, the LLMs actually used underneath are even unknown black boxes.

In this article we provide a sneak preview about Theia AI, a flexible AI framework for building AI-assistance for tools and IDEs. We will highlight one of the core capabilities: freedom in choosing any LLM. This freedom applies to tool builders that use Theia AI to create AI-augmented tools and IDEs. However, as we show at the example of the terminal assistant in the Theia IDE, tool providers can decide to even let their users choose their favorite LLM!

Theia AI is an open and flexible technology that enables developers and companies to build tailored AI-enhanced custom tools and IDEs. Theia AI significantly simplifies this task by taking care of base features such as LLM access, a customizable chat view, prompt templating and much more, and lets tool developers focus on engineering prompts for their use cases and integrate them seamlessly in Theia’s editors and views, as well as in the tool provider’s custom editors and views. Theia AI is part of the Theia Platform and is ready to be adopted by tool builders wanting to be in full control over their AI solutions. Learn more about the vision of Theia AI.

Theia IDE is a modern and open IDE built on the Theia platform. With version 1.54 Theia IDE will integrate experimental AI support based on Theia AI to showcase AI-powered functionalities in a highly customizable, transparent and open setting. Learn more about the Theia IDE.

Before we dive into the ability to choose an LLM, let’s look at the example feature. Theia AI does not only provide a chat window, it allows to integrate AI assistance into custom views and editors. The Theia IDE, built on Theia AI, provides a terminal assistant that can be invoked directly in the terminal view and provides assistance in generating terminal commands (see screenshot below).

By default, the Theia IDE uses the latest models from OpenAI for these kinds of features. However, Theia AI supports arbitrary AI models via its “LLM providers” API. Therefore, tool builders can very easily plugin any LLM of their choice, also a self-hosted LLM service or a local LLM. Moreover, if the tool provider allows this, the users themselves can actually easily configure the used LLM for every single AI agent among the available options. This is even configurable on a fine-grained basis, i.e. you can select the used LLM per use case.

In the Theia IDE, we provide the user with this maximum freedom. As an example, the user can very easily select which OpenAI model to use, e.g. to switch to the very latest and typically cheapest version (see screenshot below).

Even more interesting, you can also integrate other providers or connect Theia AI to your own hosted models. This includes models that run locally on the user’s machine. For Theia IDE, there is experimental support to integrate LLamafile, which lets you distribute and run LLMs from a single file on your machine. In the following screenshot, we use the terminal agent again, but this time using a locally running Llama 3.1 7b. Please note that using local LLMs might take significantly more time, especially when running on machines without any GPU support, but you are completely independent of any provider or internet connection.

As mentioned before, Theia AI allows to connect any type of LLMs very easily via “LLM providers”. This is especially interesting for tool builders, who can choose whatever LLM they want to integrate into their custom tool or IDE, may it be a proprietary one, a local one, or a self-hosted solution.

We will show more showcases demonstrating the core principles of Theia AI within the next few days, see also:

← Previous: Transparent Code Completion

Stay tuned and follow us on Twitter.

If you want to sponsor the project or use Theia AI to create your own AI solution, please get in contact with us. In particular, we are also looking for LLM providers who want to make their language models available via Theia AI.

EclipseSource is at the forefront of technological innovation, ready to guide and support your AI initiatives based on Theia AI or any other technology. Our comprehensive AI integration services provide the specialized know-how necessary to develop customized, AI-enhanced solutions that elevate your tools and IDEs. Explore how we can assist in integrating AI into your tools with our AI technology services. Reach out to begin your AI integration project with us.