Large Language Models (LLMs) are increasingly used not only to generate content, but also to evaluate the output produced by other models. This technique is commonly known as LLM-as-a-Judge. Spring AI’s Advisor API provides a convenient place to implement this pattern. In particular, Recursive Advisors allow an advisor to execute the remaining advisor chain multiple times. This makes it possible to generate an answer, evaluate it, provide feedback, and ask the model to try again when the answer does not meet the required quality threshold.
1. Overview
An LLM-as-a-Judge architecture typically consists of two logical roles: a Generator, which produces an answer for the user’s request, and a Judge, which evaluates the generated answer against predefined criteria such as correctness, relevance, completeness, and clarity. In a Spring AI implementation, a Recursive Advisor coordinates this process by first passing the user request to the Generator LLM and then sending the generated response to the Judge LLM for evaluation. If the judge approves the response, it is returned to the caller; if the judge rejects it, the judge’s feedback is added to the prompt and the Generator is given another opportunity to produce an improved answer, which is then evaluated again. This generate-evaluate-improve cycle continues until the response passes the evaluation or a configured maximum number of attempts is reached. Bounding the retry count is important because it prevents an indefinite evaluation loop and helps control latency, token consumption, and API costs.
2. Understanding Recursive Advisors in Spring AI
Spring AI Advisors provide a mechanism for intercepting and enhancing AI requests and responses as they move through the application, allowing developers to apply additional behavior before a request reaches the chat model and after a response is generated. A standard advisor generally processes a request, passes it to the next component in the advisor chain, and then optionally processes the returned response. A Recursive Advisor extends this concept by allowing the downstream portion of the advisor chain to be executed multiple times when necessary, while avoiding the repeated execution of advisors that have already run before it. This makes recursive advisors particularly useful for workflows that require iteration, such as LLM evaluation and correction, structured-output validation, prompt refinement, and iterative AI processing. In an LLM-as-a-Judge implementation, the recursive advisor coordinates the evaluation cycle by sending the original request to a generator LLM, passing the generated answer to a judge LLM, and checking whether the answer satisfies the required quality criteria. If the judge approves the answer, it is returned to the application; if the answer is rejected, the judge’s feedback is incorporated into the request so the generator can produce an improved response, which is then evaluated again. This process continues until the judge approves the response or a configured maximum number of attempts is reached, providing a controlled way to improve AI-generated answers while preventing unlimited retries, excessive token consumption, and unnecessary latency.
2.1 Why Recursive Advisors Fit LLM-as-a-Judge
Recursive Advisors are a natural fit for the LLM-as-a-Judge pattern because the workflow requires the same generation and evaluation process to be repeated until an acceptable response is produced. Instead of placing retry and evaluation logic directly inside controllers or service classes, a recursive advisor encapsulates this behavior within the Spring AI advisor chain. It can generate a response, send it to the Judge LLM for evaluation, incorporate the judge’s feedback when the response is rejected, and execute the downstream chain again with the improved instructions. This keeps the application’s business logic clean while making the evaluation mechanism reusable across different AI requests. Recursive Advisors also allow the number of attempts to be bounded, helping control latency, token consumption, and API costs. As a result, they provide a simple and structured approach for implementing iterative generation, evaluation, and correction workflows in Spring AI.
3. Spring Boot Code Example
3.1 Maven Configuration
The first step is to configure the required dependencies for the Spring Boot application. The following Maven configuration includes Spring Web for exposing REST APIs and the Spring AI OpenAI starter for integrating the application with an OpenAI chat model.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>4.0.7</version>
<relativePath/>
</parent>
<groupId>com.example</groupId>
<artifactId>llm-judge</artifactId>
<version>0.0.1-SNAPSHOT</version>
<properties>
<java.version>21</java.version>
<spring-ai.version>2.0.0</spring-ai.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-bom</artifactId>
<version>${spring-ai.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-starter-model-openai</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
This pom.xml defines a Java 21 application using Spring Boot 4.0.7 and Spring AI 2.0.0. The spring-ai-bom manages compatible versions of Spring AI dependencies so that individual Spring AI modules do not need their own version declarations. The spring-boot-starter-web dependency provides the web and REST capabilities required to expose the LLM-as-a-Judge functionality through an HTTP endpoint, while spring-ai-starter-model-openai provides the Spring AI integration needed to configure and communicate with OpenAI chat models. Finally, the spring-boot-maven-plugin allows the application to be packaged and executed as a standard Spring Boot application.
3.2 Application Configuration
Next, configure the OpenAI connection and chat model settings in the Spring Boot application.properties file. The API key is supplied through an environment variable, while the model and temperature settings control how the LLM generates responses.
spring.application.name=llm-judge
spring.ai.openai.api-key=${OPENAI_API_KEY}
spring.ai.openai.chat.options.model=gpt-4.1-mini
spring.ai.openai.chat.options.temperature=0.2
This configuration names the Spring Boot application llm-judge and instructs Spring AI to obtain the OpenAI API key from the OPENAI_API_KEY environment variable instead of storing sensitive credentials directly in the source code. The spring.ai.openai.chat.options.model property selects gpt-4.1-mini as the chat model used by the application, while the temperature is set to 0.2 to favor more consistent and focused responses. A lower temperature is particularly useful in an LLM-as-a-Judge workflow because both generation and evaluation benefit from predictable behavior rather than highly creative or variable responses.
Note: To obtain an OpenAI API key, sign in to the OpenAI Platform, open the API key settings, and create a new secret key. Copy and store the key securely when it is created, then set it as the OPENAI_API_KEY environment variable used by this application. Do not commit API keys to source control or include them directly in application.properties. API access and billing are managed through the OpenAI Platform and are separate from a ChatGPT subscription.
3.3 Spring Boot Application
Create the main Spring Boot application class to serve as the entry point for the LLM-as-a-Judge application. It starts the Spring application context and enables Spring Boot’s automatic configuration and component scanning.
package com.example.llmjudge;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@SpringBootApplication
public class LlmJudgeApplication {
public static void main(String[] args) {
SpringApplication.run(LlmJudgeApplication.class, args);
}
}
The LlmJudgeApplication class is the standard bootstrap class for the application. The @SpringBootApplication annotation combines Spring Boot configuration, auto-configuration, and component scanning, allowing Spring to automatically discover components such as controllers, configuration classes, and AI-related beans within the application package. The main() method calls SpringApplication.run() to initialize the Spring application context, create the configured beans, start the embedded web server, and make the application ready to accept requests.
3.4 Judge Result
Create a simple Java record to represent the structured result returned by the Judge LLM. It captures whether the generated answer is accepted, its quality score, and any feedback that can be used to improve a rejected response.
package com.example.llmjudge;
public record JudgeResult(
boolean approved,
int score,
String feedback) {
}
The JudgeResult record provides a small and strongly typed representation of the LLM evaluation result. The approved field indicates whether the generated response satisfies the required quality criteria, while score stores the numerical rating assigned by the judge. The feedback field contains concise improvement instructions when the response does not meet the required standard. Using a Java record keeps the model simple and immutable while allowing Spring AI to map the Judge LLM’s structured response directly into a Java object that can be used by the recursive advisor.
3.5 Creating the Recursive LLM Judge Advisor
The LlmJudgeAdvisor contains the core LLM-as-a-Judge workflow. It executes the generator, evaluates the generated answer with a separate judge client, and retries the downstream advisor chain with improvement feedback when the response does not meet the required quality threshold.
package com.example.llmjudge;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.ChatClientRequest;
import org.springframework.ai.chat.client.ChatClientResponse;
import org.springframework.ai.chat.client.advisor.api.CallAdvisor;
import org.springframework.ai.chat.client.advisor.api.CallAdvisorChain;
import org.springframework.core.Ordered;
public class LlmJudgeAdvisor implements CallAdvisor {
private final ChatClient judgeClient;
private final int maxAttempts;
public LlmJudgeAdvisor(ChatClient judgeClient, int maxAttempts) {
this.judgeClient = judgeClient;
this.maxAttempts = maxAttempts;
}
@Override
public ChatClientResponse adviseCall(ChatClientRequest request, CallAdvisorChain callAdvisorChain) {
CallAdvisorChain recursiveChain = callAdvisorChain.copy(this);
ChatClientRequest currentRequest = request;
ChatClientResponse response = null;
for (int attempt = 1; attempt <= maxAttempts; attempt++) {
response = recursiveChain.nextCall(currentRequest);
String answer = response.chatResponse().getResult().getOutput().getText();
JudgeResult result = judge(answer);
System.out.printf("Attempt %d - score=%d, approved=%s%n", attempt, result.score(), result.approved());
if (result.approved()) {
return response;
}
if (attempt == maxAttempts) {
return response;
}
String improvementInstruction = """
Your previous answer was evaluated by a quality judge.
Judge score: %d/10
Judge feedback:
%s
Rewrite the answer and correct the issues identified by the judge. Return only the improved answer.
"""
.formatted(result.score(), result.feedback());
currentRequest = currentRequest.mutate()
.prompt(currentRequest.prompt().augmentUserMessage(improvementInstruction))
.build();
}
return response;
}
private JudgeResult judge(String answer) {
return judgeClient.prompt()
.system("""
You are a strict evaluator of AI-generated answers.
Evaluate the answer using these criteria:
1. Correctness
2. Relevance
3. Completeness
4. Clarity
Give a score between 1 and 10.
Set approved=true only when the score is 8 or higher.
When the answer is rejected, provide concise, actionable feedback explaining what must improve.
""")
.user("""
Evaluate the following answer:
----------------
%s
----------------
""".formatted(answer))
.call()
.entity(JudgeResult.class, spec -> spec.schemaValidation());
}
@Override
public String getName() {
return "LlmJudgeAdvisor";
}
@Override
public int getOrder() {
return Ordered.HIGHEST_PRECEDENCE + 100;
}
}
The LlmJudgeAdvisor implements CallAdvisor and receives a dedicated judgeClient together with a configurable maxAttempts value. Inside adviseCall(), it creates a recursive copy of the downstream advisor chain and uses that chain to generate an initial response. The generated text is then passed to the judge() method, where the Judge LLM evaluates it for correctness, relevance, completeness, and clarity and returns a structured JudgeResult. If the response is approved, it is returned immediately; otherwise, the judge’s score and feedback are added to the current prompt as an improvement instruction and the downstream chain is executed again. This process continues until the answer is approved or the maximum number of attempts is reached. The getName() and getOrder() methods identify the advisor and control its position in the advisor chain, while the bounded retry count prevents endless evaluation cycles and unnecessary token usage.
3.6 Configuring the Generator and Judge
Next, configure two separate ChatClient instances for the generator and judge roles. Both clients can use the same underlying ChatModel, while their separate configurations ensure that generation and evaluation remain logically independent.
package com.example.llmjudge;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class AiConfiguration {
@Bean
ChatClient judgeClient(ChatModel chatModel) {
return ChatClient.builder(chatModel).build();
}
@Bean
ChatClient generatorClient(ChatModel chatModel, ChatClient judgeClient) {
LlmJudgeAdvisor judgeAdvisor = new LlmJudgeAdvisor(judgeClient, 3);
return ChatClient.builder(chatModel)
.defaultSystem("""
You are a technical assistant.
Answer questions accurately and clearly.
Prefer concise explanations and practical examples.
""")
.defaultAdvisors(judgeAdvisor)
.build();
}
}
The AiConfiguration class defines the two ChatClient beans required by the LLM-as-a-Judge workflow. The judgeClient is a simple client responsible only for evaluating generated answers and deliberately does not register the recursive advisor, preventing the judge’s own requests from entering the evaluation loop. The generatorClient is responsible for producing answers and is configured with a default system instruction that encourages accurate, concise, and practical responses. It also registers an instance of LlmJudgeAdvisor with a maximum of three attempts, meaning responses generated through this client are automatically evaluated by the judge and may be regenerated based on the judge’s feedback. Although both clients use the same ChatModel in this example to keep the configuration simple, they perform clearly separated generator and evaluator responsibilities.
3.7 REST Controller
Finally, expose the LLM-as-a-Judge functionality through a simple REST endpoint. The controller accepts a user’s question and sends it to the configured generator client, while the recursive advisor automatically handles evaluation and retries behind the scenes.
package com.example.llmjudge;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class AnswerController {
private final ChatClient generatorClient;
public AnswerController(ChatClient generatorClient) {
this.generatorClient = generatorClient;
}
@GetMapping("/ask")
public String ask(@RequestParam String question) {
return generatorClient.prompt()
.user(question)
.call()
.content();
}
}
The AnswerController exposes a GET endpoint at /ask and receives the user’s input through the question request parameter. The configured generatorClient is injected through constructor injection and is used to create a prompt containing the user’s question, execute the model request, and return the final response content. Because LlmJudgeAdvisor is already registered as a default advisor on the generator client, the controller does not need to contain any evaluation or retry logic. The generated answer is automatically passed through the LLM-as-a-Judge workflow, where it can be evaluated and improved when necessary before the final approved response is returned to the API caller.
3.8 Running the Application
After configuring the project, set the OpenAI API key as an environment variable so that Spring AI can authenticate with the OpenAI service without exposing the credential in the application source code.
export OPENAI_API_KEY="your-api-key"
This command stores the OpenAI API key in the current terminal session under the OPENAI_API_KEY environment variable. The value is automatically picked up by the spring.ai.openai.api-key=${OPENAI_API_KEY} property defined earlier in application.properties.
Once the API key is available, start the Spring Boot application from the project directory using Maven.
mvn spring-boot:run
The Maven command compiles the application, resolves the required Spring Boot and Spring AI dependencies, initializes the configured ChatClient beans, and starts the embedded web server. By default, the REST endpoint becomes available on port 8080. With the application running, send a request to the /ask endpoint and provide the question as a query parameter.
curl "http://localhost:8080/ask?question=Explain%20why%20Java%20String%20is%20immutable"
This request sends the question to the generator LLM through the generatorClient. The recursive judge advisor then evaluates the generated response, retries the generation with feedback when necessary, and finally returns the approved or best available response to the caller after the configured maximum number of attempts.
The first generated answer receives a score below the required threshold, causing the recursive advisor to request an improved response. The application console shows the following evaluation attempts:
Attempt 1 - score=6, approved=false Attempt 2 - score=9, approved=true
The first response receives a score of 6/10 and is rejected by the Judge LLM. The advisor adds the judge’s feedback to the prompt and asks the generator to produce an improved answer. The second response receives a score of 9/10, satisfies the configured approval threshold, and is returned to the caller.
The HTTP response returned by the /ask endpoint is:
Java String is immutable because its value cannot be changed after the object is created. Operations such as concat() or replace() create a new String rather than modifying the existing instance. Immutability makes Strings safe for sharing, supports the String pool, improves thread safety, and allows hash codes to be cached efficiently. This is especially useful because Strings are commonly used as keys in collections such as HashMap.
The rejected response triggers another generation attempt using the judge’s feedback. Once the response meets the approval threshold, the recursive evaluation cycle ends and the final response is returned to the client.
4. Conclusion
Recursive Advisors provide a clean way to build iterative AI workflows in Spring AI. Instead of manually coordinating a generation loop throughout the application’s service layer, the evaluation process can be encapsulated inside an advisor. The LLM-as-a-Judge implementation shown here follows a straightforward flow: generate an answer, evaluate it, attach judge feedback when necessary, and execute the downstream advisor chain again. The critical Spring AI feature enabling the design is CallAdvisorChain.copy(this), which allows the advisor to repeat only the remaining portion of the advisor chain. For production systems, additional features such as metrics, separate judge models, token-budget limits and specialized evaluation rubrics can be introduced later. For many applications, however, a bounded retry count, a structured judge result and a focused recursive advisor are enough to create a practical LLM-as-a-Judge implementation without over-engineering the solution.
