Partner – Moderne – NPI EA (cat=Spring Boot)
announcement - icon

Refactor Java code safely — and automatically — with OpenRewrite.

Refactoring big codebases by hand is slow, risky, and easy to put off. That’s where OpenRewrite comes in. The open-source framework for large-scale, automated code transformations helps teams modernize safely and consistently.

Each month, the creators and maintainers of OpenRewrite at Moderne run live, hands-on training sessions — one for newcomers and one for experienced users. You’ll see how recipes work, how to apply them across projects, and how to modernize code with confidence.

Join the next session, bring your questions, and learn how to automate the kind of work that usually eats your sprint time.

Course – Black Friday 2026 – NPI EA – main – before_post_content
announcement - icon

Yes, we're now running our Black Friday Sale. All Courses are 33% off until 1st December, 2026 at 17:00 Romania time:

>> EXPLORE ACCESS NOW

Course – Black Friday 2026 – NPI – main – before_post_content
announcement - icon

Yes, we're now running our Black Friday Sale. All Courses are 33% off until 1st December, 2026 at 17:00 Romania time:

>> EXPLORE ACCESS NOW

1. Overview

Large language models (LLMs) have emerged as a key component of modern applications. While the most capable models are proprietary and hosted in a cloud environment. Accessing these models requires an internet connection and sending our application data to external providers.

For applications that process privacy-sensitive information or need to operate offline, these requirements may not be acceptable. As a result, there is a growing demand for integrating local LLMs into applications.

In this tutorial, we’ll learn how to integrate Spring AI with LM Studio and configure Spring AI to communicate with locally hosted chat and embedding models.

2. LM Studio

LM Studio is an application that can download, manage, and host LLMs and embedding models in our local environment. Its graphical interface allows us to browse and download models from public repositories with ease:

lm studio select model

Besides model management, it provides an API server that exposes the loaded models via REST-based APIs. This allows AI frameworks such as Spring AI to interact with local models.

There are other applications, such as Ollama, providing similar capabilities, but with different focuses:

LM Studio Ollama
User Interface Graphical desktop application Primarily command-line interface (CLI)
Model Discovery Browse and download models from the GUI Download models via CLI, without an interface to browse models.
Supported APIs OpenAI-compatible and Anthropic-compatible Native API, OpenAI-compatible, and Anthropic-compatible
Best fit Non-technical users without CLI experience, model experimentation Technical users, application integration

In general, LM Studio offers a more user-friendly experience for exploring and experimenting with local models, while Ollama is optimized for command-line workflows.

3. Maven Dependency

LM Studio provides both OpenAI-compatible and Anthropic-compatible API endpoints.

To integrate with Spring AI, we could either pick the Spring AI OpenAI dependency or the Spring AI Anthropic dependency. However, Spring AI Anthropic does not support embedding models.

Therefore, we’ll adopt the Spring AI OpenAI dependency in this tutorial:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-openai</artifactId>
    <version>2.0.0</version>
</dependency>

Note that Spring AI 2.0.0 is built on Spring Framework 7 and is intended for Spring Boot 4.

4. Configuration

Let’s configure our application.yml to adopt the local models in LM Studio:

spring:
  ai:
    openai:
      api-key: dummy
      base-url: http://localhost:1234/v1
      chat:
        model: google/gemma-4-e4b
      embedding:
        model: text-embedding-embeddinggemma-300m

There are several properties worth mentioning:

api-key: We don’t really need a real API key when connecting to the local model. Thus, we just need to fill in an arbitrary value.

base-url: We could find the base URL when we start the local server in LM Studio. If we use the Spring AI OpenAI dependency, remember to append /v1 to the base URL.

model: We only need to specify the model when multiple models are loaded in LM Studio. Otherwise, we can omit this model configuration.

5. Sample Endpoints

Now, let’s create two endpoints: One for the chat model and one for the embedding model.

The controller basically delegates the work to 2 services that work with the models:

@RequestMapping("/lm-studio")
public class LmStudioController {
    private final ChatService chatService;
    private final EmbeddingService embeddingService;

    public LmStudioController(ChatService chatService, EmbeddingService embeddingService) {
        this.chatService = chatService;
        this.embeddingService = embeddingService;
    }

    @PostMapping("/chat")
    @ResponseStatus(HttpStatus.OK)
    public String chat(@RequestBody String prompt) {
        return chatService.chat(prompt);
    }

    @PostMapping("/embeddings")
    @ResponseStatus(HttpStatus.OK)
    public EmbeddingResponse getEmbeddings(@RequestBody String text) {
        return embeddingService.getEmbeddings(text);
    }
}

The ChatService uses the ChatClient to send a prompt to the configured chat model:

@Service
public class ChatService {
    private final ChatClient chatClient;

    public ChatService(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    public String chat(String prompt) {
        return chatClient.prompt()
          .user(userMessage -> userMessage.text(prompt))
          .call()
          .content();
    }
}

Likewise, the EmbeddingService sends the text to the configured embedding model:

@Service
public class EmbeddingService {
    private final EmbeddingModel embeddingModel;

    public EmbeddingService(EmbeddingModel embeddingModel) {
        this.embeddingModel = embeddingModel;
    }

    public EmbeddingResponse getEmbeddings(String text) {
        EmbeddingRequest request = new EmbeddingRequest(List.of(text), null);
        return embeddingModel.call(request);
    }
}

6. Test Run

Before starting our Spring Boot application, make sure the LM Studio local server is running with both the chat model and embedding model loaded:

LM studio server

Now, let’s verify both the chat and embedding endpoints by initiating HTTP requests via curl.

First, we send a request to the chat endpoint:

curl -X POST http://localhost:8080/lm-studio/chat 
  -d "Tell me who you are"

The request is processed by the chat model currently loaded in LM Studio. In this example, we’ve loaded Google’s Gemma 4 model, so the endpoint returns the following response:

I am Gemma 4, a Large Language Model developed by Google DeepMind.

Next, we use the chat model response as the request payload for the embedding endpoint:

curl -X POST http://localhost:8080/lm-studio/embeddings 
  -d "I am Gemma 4, a Large Language Model developed by Google DeepMind."

The endpoint returns the embedding response. Since the full response is large, we’ll only show the model name and the first few elements:

{
  "metadata": {
    "model": "text-embedding-embeddinggemma-300m:2"
  },
  "result": {
    "index": 0,
    "output": [
      -0.07699633,
      -0.030751443,
      0.019093497,
      ...
    ]
  }
}

These results confirm that Spring AI successfully invokes the locally hosted chat model and the embedding model via LM Studio’s OpenAI-compatible API.

7. Conclusion

In this article, we learned how to integrate Spring AI with LM Studio to use a locally hosted chat model and embedding model. We configured Spring AI to work with the open-source models via LM Studio’s OpenAI-compatible API.

LM Studio is a perfect choice for running AI models without relying on a cloud provider. This approach keeps your data within the local environment and enables it to operate without an internet connection. These make it ideal for privacy-sensitive applications.

The code backing this article is available on GitHub. Once you're logged in as a Baeldung Pro Member, start learning and coding on the project.
Partner – Moderne – NPI EA (tag=Refactoring)
announcement - icon

Modern Java teams move fast — but codebases don’t always keep up. Frameworks change, dependencies drift, and tech debt builds until it starts to drag on delivery. OpenRewrite was built to fix that: an open-source refactoring engine that automates repetitive code changes while keeping developer intent intact.

The monthly training series, led by the creators and maintainers of OpenRewrite at Moderne, walks through real-world migrations and modernization patterns. Whether you’re new to recipes or ready to write your own, you’ll learn practical ways to refactor safely and at scale.

If you’ve ever wished refactoring felt as natural — and as fast — as writing code, this is a good place to start.

Course – Black Friday 2026 – NPI EA – main – after_post_content

announcement - icon

Yes, we're now running our Black Friday Sale. All Courses are 33% off until 1st December, 2026 at 17:00 Romania time:

>> EXPLORE ACCESS NOW

Course – Black Friday 2026 – NPI – main – after_post_content

announcement - icon

Yes, we're now running our Black Friday Sale. All Courses are 33% off until 1st December, 2026 at 17:00 Romania time:

>> EXPLORE ACCESS NOW