While LangChain4j 0.34 introduced my new Google AI Gemini module, a new 0.35.0 version is already here today, with some more cool stuff for Gemini and Google Cloud!

Let’s have a look at what’s in store!

Gemini 1.5 Pro 002 and Gemini 1.5 Flash 002

This week, Google announced the release of the new versions of the Google 1.5 models:

• google-1.5-pro-002
• google-1.5-flash-002

Of course, both models are supported by LangChain4j! The Google AI Gemini module also supports the gemini-1.5-flash-8b-exp-0924 8-billion parameter model.

A new version of LangChain4j, the super powerful LLM toolbox for Java developers, was released today. In 0.34.0, a new Gemini model has been added. This time, this is not the Gemini flavor from Google Cloud Vertex AI, but the Google AI variant.

It was a frequently requested feature by LangChain4j users, so I took a stab at developing a new chat model for it, during my summer vacation break.

Gemini, show me the code!

Let’s dive into some code examples to see it in action!

As I work a lot with Large Language Models (LLMs), I often have to deal with flaky test cases, because LLMs are not always consistent and deterministic in their responses. Thus, sometimes, a test passes maybe a few times in a row, but then, once in a while, it fails.

Maybe some prompt tweaks will make the test pass more consistently, lowering the temperature too, or using techniques like few-shot prompting will help the model better understand what it has to do. But in some circumenstances, you can’t find ways around those weird failures, and the sole solution I found was to make a test retryable.

In this article, we’ll explore another great task where Large Language Models shine: entity and data extraction. LLMs are really useful beyond just mere chatbots (even smart ones using Retrieval Augmented Generation).

Let me tell you a little story of a handy application we could build, for wannabe Instagram influencers!

Great Instagram hashtags, thanks to LLMs

When posting Instagram pictures, I often struggle with finding the right hashtags to engage with the community. Large Language Models are pretty creative, and they’ve certainly seen a bunch of Instagram pictures with their descriptions.

Generative AI use cases are usually about running large language models somewhere in the cloud. However, with the advent of smaller models and open models, you can run them locally on your machine, with projects like llama.cpp or Ollama.

And what about in the browser? With MediaPipe and TensorFlow.js, you can train and run small neural networks for tons of fun and useful tasks (like recognising hand movements through the webcam of your computer), and it’s also possible to run Gemma 2B and even 7B models.

In a rencent article, we talked about text classification using Gemini and LangChain4j.

A typical example of text classification is the case of sentiment analysis.

In my LangChain4j-powered Gemini workshop, I used this use case to illustrate the classification problem:

ChatLanguageModel model = VertexAiGeminiChatModel.builder()
    .project(System.getenv("PROJECT_ID"))
    .location(System.getenv("LOCATION"))
    .modelName("gemini-1.5-flash-001")
    .maxOutputTokens(10)
    .maxRetries(3)
    .build();

PromptTemplate promptTemplate = PromptTemplate.from("""
    Analyze the sentiment of the text below.
    Respond only with one word to describe the sentiment.

    INPUT: This is fantastic news!
    OUTPUT: POSITIVE

    INPUT: Pi is roughly equal to 3.14
    OUTPUT: NEUTRAL

    INPUT: I really disliked the pizza. Who would use pineapples as a pizza topping?
    OUTPUT: NEGATIVE

    INPUT: {{text}}
    OUTPUT:
    """);

Prompt prompt = promptTemplate.apply(
    Map.of("text", "I love strawberries!"));

Response<AiMessage> response = model.generate(prompt.toUserMessage());

System.out.println(response.content().text());

I used a PromptTemplate to craft the prompt, with a {{text}} placeholder value to analyze the sentiment of that particular text.

Certain models like Gemini are multimodal. This means that they accept more than just text as input. Some models support text and images, but Gemini goes further and also supports audio, video, and PDF files. So you can mix and match text prompts and different multimedia files or PDF documents.

Until LangChain4j 0.32, the models could only support text and images, but since my PR got merged into the newly released 0.33 version, you can use all those files with the LangChain4j Gemini module!

Generative AI has potential applications far beyond chatbots and Retrieval Augmented Generation. For example, a nice use case is: text classification.

I had the chance of meeting some customers and prospects who had the need for triaging incoming requests, or for labeling existing data. In the first case, a government entity was tasked with routing citizen requests to access undisclosed information to the right governmental service that could grant or reject that access. In the second case, a company needed to sort out tons of existing internal documents that were not properly organized, and they wanted to quickly start better structuring this trove of information, by labelling each of these docs into different categories.

LangChain4j 0.32.0 was released yesterday, including my pull request with the support for lots of new Gemini features:

• JSON output mode, to force Gemini to reply using JSON, without any markup,
• JSON schema, to control and constrain the JSON output to comply with a schema,
• Response grounding with Google Search web results and with private data in Vertex AI datastores,
• Easier debugging, thanks to new builder methods to log requests and responses,
• Function calling mode (none, automatic, or a subset of functions),
• Safety settings to catch harmful prompts and responses.

Let’s explore those new features together, thanks to some code examples! And at the end of the article, if you make it through, you’ll also discover 2 extra bonus points.

Large Language Models, available through easy to use APIs, bring powerful machine learning tools in the hands of developers. Although Python is usually seen as the lingua franca of everything ML, with LLM APIs and LLM orchestration frameworks, complex tasks become easier to implement for enterprise developers.

Abstract

Large language models (LLMs) are a powerful new technology that can be used for a variety of tasks, including generating text, translating languages, and writing different kinds of creative content. However, LLMs can be difficult to use, especially for developers who are not proficient in Python, the lingua franca for AI. So what about us Java developers? How can we make use of Generative AI?

Read more...