Large Language Models (LLMs) are transforming various fields, including software development. Their ability to understand and generate text (and other data types) enables code suggestion, correction, and even generation from textual prompts. This article explores the JLama library, a Java-based solution for integrating LLMs into the Java ecosystem. JLama offers flexibility, usable as a command-line interface (CLI) or as a dependency in your projects (e.g., via pom.xml). We'll demonstrate its functionality by integrating it with a Spring Boot application.

Prerequisites and Highlights

JLama requires Java 20 or higher due to its use of the Java Vector API. Existing LangChain users can integrate it with JLama, leveraging LangChain's tools for simplified LLM interaction.

This example project features two endpoints interacting with LLMs through prompts:

• A JLama-only endpoint.
• A LangChain and JLama combined endpoint.

Project Implementation

JLama Endpoint

This endpoint directly utilizes JLama to generate responses based on user prompts.

@PostMapping("/jlama") // Endpoint for JLama chat functionality
public ResponseEntity chatJlama(@RequestBody ChatPromptRequest request) {
    PromptContext context;
    if (abstractModel.promptSupport().isPresent()) {
        context = abstractModel.promptSupport()
                .get()
                .builder()
                .addSystemMessage("You are a helpful chatbot providing concise answers.")
                .addUserMessage(request.prompt())
                .build();
    } else {
        context = PromptContext.of(request.prompt());
    }

    System.out.println("Prompt: "   context.getPrompt()   "\n");
    Generator.Response response = abstractModel
            .generate(UUID.randomUUID(), context, 0.0f, 256, (s, f) -> {});
    System.out.println(response.responseText);

    return ResponseEntity.ok(new ChatPromptResponse(response.responseText));
}

The desired model is defined. If not locally available, it's automatically downloaded to the specified directory. The prompt context is created, and JLama generates the response.

// Defining the model and directory for downloading (if needed) from Hugging Face
String model = "tjake/Llama-3.2-1B-Instruct-JQ4";
String workingDirectory = "./models";

// Downloading (if necessary) or retrieving the model locally
File localModelPath = new Downloader(workingDirectory, model).huggingFaceModel();

// Loading the model
ModelSupport.loadModel(localModelPath, DType.F32, DType.I8);

LangChain and JLama Endpoint

This endpoint uses LangChain, reducing the code required for JLama interaction.

@PostMapping("/langchain")
public ResponseEntity

LangChain simplifies implementation by defining the model and parameters directly within the builder.

Links and References

This project was inspired by Professor Isidro's presentation at SouJava. [Link to presentation (replace with actual link if available)]

Useful documentation:

• JLama on GitHub [Link to JLama GitHub (replace with actual link)]
• LangChain [Link to LangChain documentation (replace with actual link)]

Conclusion

JLama and LangChain provide a powerful way to integrate LLMs into Java applications. This article demonstrated how to configure and use these tools with Spring Boot to create efficient textual prompt processing endpoints.

Have you worked with LLMs in Java projects? Share your experiences and insights in the comments!