Introducing Theia AI: Build AI-Enhanced Tools with Ease!

There is a growing demand for powerful tools and IDEs that seamlessly integrate AI capabilities across all industries. However, creating such AI-powered tools from scratch involves significant effort. Even aside from developing reliable prompts, augmenting them with the adequate context, and potentially fine-tuning AI models, integrating LLMs into existing tools, handling data interactions, and ensuring a smooth user experience can be daunting. For many tool builders, this means investing valuable time and resources into foundational work, instead of focusing on the unique AI features that make their tools stand out and that really help boosting the productivity of their tool users.

This is where Theia AI comes in. Designed specifically for tool builders, Theia AI simplifies the process of developing AI-powered tools and IDEs by providing a robust framework including the required infrastructure, user interfaces, reusable components and patterns that proved to be useful in practice. Theia AI takes care of the foundational AI integration work, allowing you to concentrate on what matters most: delivering innovative, domain-specific solutions.

Theia AI is part of the Theia Platform and available in the recent 1.54 version. Despite being in a beta version, Theia AI is already very successfully used by tool builders and our customer projects to create innovative AI solutions. It is crafted based on our real-world experience from building innovative AI solutions. With further adoption and feedback, we will further streamline the APIs and extract reusable AI integration functionality from the community into the generic framework.

The Effort Behind AI Tools – and How Theia AI Makes It Easier

Building AI-driven tools is no small task. From managing communication between the tool and the large language model (LLM) to ensuring smooth data handling and crafting custom AI interactions, there’s a lot to consider. Many of these challenges are universal across different tools, but until now, they often had to be addressed from scratch in every project.

Theia AI addresses these challenges head-on by offering a set of reusable features that drastically reduce the development effort:

1. LLM Access and Communication: Instead of spending time setting up the basic infrastructure for communicating with large language models, Theia AI provides out-of-the-box LLM integration and makes the LLMs registered in the tool available through a straightforward, yet powerful API. This API not only simplifies making use of streaming support to process content from the LLM as it arrives incrementally or parse it according to specific rules (e.g. to filter chain-of-thought-content or detect code blocks), but it also supports more complex configurations, as well as handling function calls from the LLM or requesting and processing structured output (i.e. JSON following a specific schema) and more. This means you can easily plug in your preferred AI model—whether it’s OpenAI, a self-hosted LLM, or even a locally running model—without the hassle of building these connections and LLM content processing from scratch, and access them from your tool-specific AI agents via a straightforward API (see below). Theia AIs flexibility easily allows you to even use multiple LLMs for different tasks or optionally even make the LLM selection a user-configurable preference.

2. Agent Creation, Configurability and Extensibility: A common pattern for building AI support is to create so-called agents that are responsible for one or more specific user tasks and manage the interaction between the user, the tool and the underlying LLMs. With Theia AI, building custom AI agents for specific tasks becomes a straightforward process. The framework includes reusable components for creating and configuring agents, managing variable inputs and integrating tool-specific functionality. Furthermore, you can easily manage and evolve the granularity of agents based on your use cases, embed them into a testing environment and even let agents dynamically communicate with each other. This infrastructure allows you to focus on building actual AI capabilities that provide value to your users rather than dealing with integration and generic architecture concerns.

3. Prompt Management and Template Registry: Streamlining AI prompts is key to delivering accurate and context-aware AI responses. When applying popular prompting strategies such as n-shot prompting or chain of thought, most tool integrations of AI will have to deal with multiple prompt templates and augment them with use-case-specific context information. Theia AI provides a powerful prompt management system that allows you to store, manage, and even dynamically adjust prompts at runtime. This is particularly useful when iterating on AI interactions or adjusting the AI’s behavior based on user feedback. Plus, these prompt templates can be exposed to testers or users, enabling a collaborative approach to optimizing your AI interactions.

4. Data Interaction and Control: One of the complex challenges of AI integration is managing how the AI interacts with your tool’s state and its data. This is typically achieved by augmenting prompts with variables as well as using “tool functions” that allow the LLM to actively retrieve information or do manipulations. By providing dedicated registries and a native API for variables and function calling, Theia AI makes defining and implementing such variables and function call capabilities a simple task. At the same time, Theia AI gives you full control over these interactions, with reusable features for managing data access and functions. This allows controlling LLM interactions with your tool without worrying about building custom data and interaction pipelines from scratch.

5. Customizable Chat and User Interfaces: Every AI-driven tool needs a user interface for interacting with the AI, whether it’s a chat window or an embedded suggestion system. Theia AI includes an interactive, fully customizable chat view that can be adapted to fit your tool’s unique requirements. You no longer need to design and build these UIs from scratch, let alone taking care of managing the underlying conversations. The generic chat view can be extended with custom UI elements (e.g. buttons, context menus, links), while still allowing you to benefit from existing integrations, like an inline code widget, markdown rendering, progress indicators, etc. Theia AI also provides corresponding APIs to integrate custom UI editors and widgets, may it be textual editors, such as code completion, graphical diagrams or any other representation like tables and forms.

Theia AI offers dedicated extension points to hook up your domain-specific implementations, such as agents, tool call handlers, tool context providers, UI renderers for certain types of responses in the chat and LLM providers.

High Level Architecture of Theia AI

Any component in the tool can invoke agents and therefore get equipped with AI capabilities. In addition, there are a variety of settings to configure which agents are to be used in which context, and tools that help you diagnose issues, such as inspecting LLM communication directly in the tool, or tracing the entire flows of an agent, including its obtained information and processed tool calls.

Stay in Full Control of Your AI Integration

Theia AI not only reduces development effort tremendously but also ensures that you as a tool builder have full control over every aspect of your AI integration. With its extensible architecture, you can tailor how the AI interacts with your tool to suit your unique use case, whether it’s in a domain-specific tool or a more general-purpose IDE. This flexibility extends across the following three major dimensions:

1. Control over the User Interface (UI)

Theia AI allows tool builders to design a fully customized AI experience. Whether you need a chat interface, inline code suggestions, or a completely different UI interaction model, Theia AI provides the flexibility to adapt. You can embed AI-driven elements into various parts of the tool, such as terminals, editors, or graphical interfaces, all while controlling how AI suggestions, interactions, and outputs are displayed. The fully customizable chat view, along with dedicated APIs for integrating custom UI elements, ensures that the AI fits seamlessly into your tool’s design without the need for building UIs from scratch.

2. Control over Tool Data and the Entire Prompt Chain

An AI’s effectiveness depends on its ability to access relevant context, such as the tool state or the workspace. Theia AI gives you complete control over how context data from your tool is managed and provided to the LLM. With its prompt management system, Theia AI allows tool builders to create customizable prompts that integrate variables and even tool-specific functions. This allows the AI to dynamically retrieve specific data, such as selected text, files, or tool outputs, and adjust its responses based on the current project context. You can also fine-tune the entire prompt chain, adding relevant context to each interaction and ensuring the AI operates with full situational awareness. Finally, tool functions can enable AI to actively perform tasks like retrieving a file list or even modifying a workspace.

3. Control over the LLM (Provided Service, Self-Hosted, Local)

Theia AI’s flexibility extends to your choice of LLM provider, whether it’s a cloud-based service like OpenAI, a self-hosted model, or a local LLM running within your environment. You can easily plug in different LLMs for various tasks and even allow end users to configure which LLM they want to use for specific operations. This means you’re not locked into a single vendor or service, and you have the freedom to choose the most suitable model for your application, whether for privacy reasons, performance, or specific domain requirements and business model constraints. Theia AI supports integrations with OpenAI, VLLM, Ollama, and LlamaFile out of the box, but can also be extended to work with additional LLM providers as needed.

Flexibility for End Users

If desired, tool builders can even extend this flexibility to their users. With Theia AI, you can expose configuration options that allow users to customize AI behavior, such as selecting which LLM to use, adjusting prompts, or modifying data interactions. This level of transparency and user empowerment can be particularly useful in scenarios where end-user control over AI interactions is a key requirement or in early phases of a project to to involve testers or end users in the refinement process.

Open Architecture and Governance

In summary, Theia AI is built as an extensible and adaptable framework integrated into the Eclipse Theia platform, designed for building AI-powered tools and IDEs tailored to various domains beyond just code editing. It does not restrict you in terms of what use case you want to enable and how you want to design the AI integration into your custom tools. Theia AI allows tool builders to fully customize the platform through its extensible architecture without needing to fork the code. Therefore, you can clearly separate your own proprietary code from the framework. Theia AI is fully open source and supports a commercial-friendly and open governance model, ensuring independence from any single vendor. Embedded in an active and welcoming open-source community, adopting Theia AI also fosters true collaboration and ensures you benefit from continuous innovations landing in Theia AI.

If you want to learn more about how Theia AI empowers tool builders with full control over their AI solutions, please refer to our previous article about the vision of Theia AI

Theia AI in action

Enough of the theoretical introduction—let’s see Theia AI in action! Theia AI empowers tool builders by offering flexible, reusable components for building AI-powered features into their custom tools and IDEs. Below are a few real-world examples of how these capabilities can be applied. We will mostly use the AI support in the Theia IDE as an example, as it is [publicly available](link to other blog post).

The Standard Chat: AI Interaction with Custom Response Renderers

One of the core features of Theia AI is the reusable and customizable chat interface. For example, in the Theia IDE, the Command Chat Agent helps users interact with the IDE through natural language. If you ask for specific actions, such as toggling the toolbar or opening settings, the agent provides executable command suggestions directly in the chat. This interaction is enhanced with response renderers—UI elements like buttons that allow users to trigger actions directly from the chat window. These renderers make the chat highly interactive, bridging the gap between user commands and tool execution. As a tool builder you can of course easily define your own chat agents and response renderers to show arbitrary interactive replies in the chat view, such as trees, drop-down selections, and more.

Chat Embedded into Another UI Element

Theia AI also allows embedding AI-driven chat into various parts of the user interface. Consider the Terminal Assistant in the Theia IDE. This AI agent is integrated directly into the terminal view, providing command suggestions based on the current environment and the output of recent commands. When you open the terminal and start typing, the assistant offers relevant suggestions based on your actions, such as build commands or directory navigation. This shows how AI-driven functionality can enhance not only chat interactions but also the productivity of arbitrary other components and widgets within a tool.

A Feature Without Any Chat: AI-Powered Inline Code Completion

Not all AI interactions require a chat interface. The Inline Code Completion feature, built with Theia AI, showcases how AI can be applied more directly in a tool. As you write code, the AI continuously provides inline suggestions that go beyond traditional static code completion. These suggestions are context-aware and adapt to your coding style, helping to boost productivity. This feature exemplifies how Theia AI can be integrated into any UX workflow without disrupting the user experience.

Domain-Specific AI Integration: Graphical Editor with Theia AI

Theia AI is not limited to code development tools. An example for a domain-specific tool is an AI integration into a custom diagram editor developed with the Graphical Language Server Protocol (GLSP). Here, Theia AI powers graphical modeling tools by providing real-time AI-driven insights and recommendations. For instance, users working on diagrams can receive AI-based suggestions for adding new nodes or adjusting configurations, enhancing both the efficiency and accuracy of complex domain-specific workflows.

Conclusion

Theia AI offers a powerful way for tool builders to harness the tremendous potential of AI for their users without getting bogged down by the complexities of integration. Whether you’re creating a new AI-powered tool or enhancing an existing solution, Theia AI’s reusable components and flexible architecture make it easier than ever to bring your vision to life. The Theia AI framework is already being successfully applied in various AI-driven tools. If you’re looking to build an AI-driven coding editor or create a domain-specific AI-powered tool that doesn’t focus on code, check out Theia AI today and get in contact with us.

EclipseSource is at the forefront of technological innovation, ready to guide and support your AI initiatives based on Theia AI or any other technology. Our comprehensive AI integration services provide the specialized know-how necessary to develop customized, AI-enhanced solutions that elevate your tools and IDEs. Explore how we can assist in integrating AI into your tools with our AI technology services. Reach out to begin your AI integration project with us.

We will soon publish more articles about Theia AI, so stay tuned and follow us on Twitter.