Theia AI Sneak Preview: Create your own AI assistance!

Do you want to augment your tool or IDE with AI assistance? In this article, we show an example of how easy it is to create custom AI agents with Theia AI, a fully open AI framework for building AI-assistance for tools and IDEs. We will highlight one of the core principles, Tailorability and Extensibility for Tool Builders at the example of an agent for finding arbitrary commands in the Theia IDE.

Theia AI is an open and flexible technology that enables developers and companies to build tailored AI-enhanced custom tools and IDEs. Theia AI significantly simplifies this task by taking care of base features such as LLM access, a customizable chat view, prompt templating and much more, and lets tool developers focus on engineering prompts for their use cases and integrate them seamlessly in Theia’s editors and views, as well as in the tool provider’s custom editors and views. Theia AI is part of the Theia Platform and is ready to be adopted by tool builders wanting to be in full control over their AI solutions. Learn more about the vision of Theia AI.

Theia IDE is a modern and open IDE built on the Theia platform. With version 1.54 Theia IDE will integrate experimental AI support based on Theia AI to showcase AI-powered functionalities in a highly customizable, transparent and open setting. Learn more about the Theia IDE.

AI is just at the very beginning of being integrated in tools and IDEs. Once an idea for an AI use case is created, the next step is to evaluate its feasibility and quickly iterate on prompt strategies, UX concepts and user interaction and maybe even the evaluation of different LLMs. To make this process as efficient as possible, tool builders should ideally not worry about basic concerns such as LLM communication or creating yet another chat UI. With Theia AI, you have a powerful and solid basis to quickly implement and iterate on your custom AI solution, seamlessly integrated in your custom tool or IDE.

Let’s create a new agent for the Theia IDE as an example! This new agent will assist users in identifying (and later executing) arbitrary commands in the Theia IDE, such as opening the settings or showing the toolbar. It will be integrated in the default chat and therefore be a so-called chat agent.

Let’s start simple and define our first ChatAgent. Let’s define the system prompt template for our agent first:

export class CommandChatAgentSystemPromptTemplate = {
   id = 'command-system';
   template = `Always respond with: “I am the command agent”`
}

Next we create the actual agent. By inheriting from the standard ChatAgent (provided in Theia AI), you will only need to define some base properties, such as an ID and a description. Furthermore, we register our prompt template, so that the user can review and even adapt it in the UI. Finally, we return our prompt template in the given function getSystemMessage, just by loading it from the prompt service.

export class CommandChatAgent extends AbstractChatAgent {


   id: string = 'Command';
   name: string = 'Command';
   description: string = `Command Agent Description`;
   variables: string[] = [];
   promptTemplates: PromptTemplate[] = [new CommandChatAgentSystemPromptTemplate()];


   protected async getSystemMessage(): Promise<SystemMessage | undefined> {
       const systemPrompt = await this.promptService.getPrompt('command-system');
       return SystemMessage.fromResolvedPromptTemplate(systemPrompt);
   }
}

Theia AI is modular, so we can just register the agent along with the existing agents via dependency injection, no forking is required. The simple agent above will already be available in the default chat, you can directly “talk” to it via typing “@command”. In the current state, it will of course only respond with “I am the command agent”. What’s most interesting is how the agent tells the LLM which commands are actually available. For this purpose, we add a variable to our prompt (see below).

You are a service that helps users find commands to execute in an IDE.
You reply with the command id specifying which command to execute and its arguments, if any


Here are the known Theia commands:


Begin List:
{{command-ids}}
End List
```md

The variable command-ids is not yet existing, so it won’t be resolved yet. We could now make it a global variable (available to all agents) or consider it to be agent-specific. In the second, simpler case, we could just manually resolve the variable when returning the system prompt (see below). If we register the variable globally, Theia AI would do the resolvement for us. 
In the following code, we just retrieve all commands using the Theia API and resolve the variable in the system prompt with this list. While this is a simple example, it shows how to easily connect an AI assistant to tool-specific APIs and concepts. This enables very powerful but yet very easy to create integration for tool builders, specifically also in domain-specific tools with custom APIs.

```ts
       const knownCommands: string[] = [];
       for (const command of this.commandRegistry.getAllCommands()) {
           knownCommands.push(`${command.id}: ${command.label}`);
       }
       const systemPrompt = await this.promptService.getPrompt('command-chat-agent-system-prompt-template', {
           'command-ids': knownCommands.join('\n')
       });

As soon as we register this new chat agent in Theia AI, users can now ask questions and get responses about Theia command ids they can execute in their Theia-based tool. Of course it would be nicer, if the user would not need to manually copy the id. In Theia AI, we can actually achieve this with a few lines of code that we will demonstrate in the next sneak preview article, so stay tuned!

Please note that, we are also working on adding support for dynamic “Custom Agents”, i.e. agents that can be easily defined at runtime and directly by the user. Many chat agents essentially consist of a custom system prompt and the usage of variables and tool functions. As prompts are modifiable in Theia AI, the user will be able to create new agents on the fly, e.g. directly in the Theia IDE. As you imagine, the possibilities to spontaneously support custom workflows with this capability are endless, we are looking forward to it!

We will show more showcases demonstrating the core principles of Theia AI within the next few days, see also:

← Previous: Transparent variables and tool functions   |   Next: Custom Part Renderer and actionable responses →

Stay tuned and follow us on Twitter.

If you want to sponsor the project or use Theia AI to create your own AI solution, please get in contact with us. In particular, we are also looking for LLM providers who want to make their language models available via Theia AI.

EclipseSource is at the forefront of technological innovation, ready to guide and support your AI initiatives based on Theia AI or any other technology. Our comprehensive AI integration services provide the specialized know-how necessary to develop customized, AI-enhanced solutions that elevate your tools and IDEs. Explore how we can assist in integrating AI into your tools with our AI technology services. Reach out to begin your AI integration project with us.