It’s always interesting to know how the sausage is made, don’t you think? That’s why, a while ago, I looked at embedding model tokenization, and I implemented a little visualization to see the tokens in a colorful manner. Yet, I was still curious to see how Gemini would tokenize text…

Both LangChain4j Gemini modules (from Vertex AI and from Google AI Labs) can count the tokens included in a piece of text. However, both do so by calling a REST API endpoint method called countTokens. This is not ideal, as it requires a network hop to get the token counts, thus adding undesired extra latency. Wouldn’t it be nicer if we could count tokens locally instead?

Every year, in October, takes place the Inktober challenge: every day of the month, you have to do a drawing representing the word of the day. The list of prompts this year is the following:

I participated to some of the daily challenges the past few years, but I never did all of them. But this year, for the fun, I thought I could ask Google’s Imagen 3 image model to draw for me! (Or at least to draw something I could try to reproduce.)

When integrating an LLM into your applicaton to extend it and make it smarter, it’s important to be aware of the pitfalls and best practices you need to follow to avoid some common problems and integrate them successfully. This article will guide you through some key best practices that I’ve come across.

Understanding the Challenges of Implementing LLMs in Real-World Applications

One of the first challenges is that LLMs are constantly being improved. This means that the model you start using could change under the hood, and suddenly your application doesn’t work as it did before. Your prompts might need adjustments to work with the newer version, or worse, they might even lead to unintended results!

Prelude

As I’m focusing a lot on Generative AI, I’m curious about how things work under the hood, to better understand what I’m using in my gen-ai powered projects. A topic I’d like to focus on more is: vector embeddings, to explain more clearly what they are, how they are calculated, and what you can do with them.

A colleague of mine, André, was showing me a cool experiment he’s been working on, to help people prepare an interview, with the help of an AI, to shape the structure of the resulting final article to write.

Read more...

This week LangChain4j, the LLM orchestration framework for Java developers, released version 0.26.1, which contains my first significant contribution to the open source project: support for the Imagen image generation model.

Imagen is a text-to-image diffusion model that was announced last year. And it recently upgraded to Imagen v2, with even higher quality graphics generation. As I was curious to integrate it in some of my generative AI projects, I thought that would be a great first contribution to LangChain4j.

A promising feature of the Gemini large language model released recently by Google DeepMind, is the support for function calls. It’s a way to supplement the model, by letting it know an external functions or APIs can be called. So you’re not limited by the knowledge cut-off of the model: instead, in the flow of the conversation with the model, you can pass a list of functions the model will know are available to get the information it needs, to complete the generation of its answer.

Hot on the heels of the release of Gemini, I’d like to share a couple of resources I created to get your hands on large language models, using LangChain4J, and the PaLM 2 model. Later on, I’ll also share with you articles and codelabs that take advantage of Gemini, of course.

The PaLM 2 model supports 2 modes:

• text generation,
• and chat.

In the 2 codelabs, you’ll need to have created an account on Google Cloud, and created a project. The codelabs will guide you through the steps to setup the environment, and show you how to use the Google Cloud built-in shell and code editor, to develop in the cloud.

Google announced today the availability of Gemini, its latest and more powerful Large Language Model. Gemini is multimodal, which means it’s able to consume not only text, but also images or videos.

I had the pleasure of working on the Java samples and help with the Java SDK, with wonderful engineer colleagues, and I’d like to share some examples of what you can do with Gemini, using Java!

First of all, you’ll need to have an account on Google Cloud and created a project. The Vertex AI API should be enabled, to be able to access the Generative AI services, and in particular the Gemini large language model. Be sure to check out the instructions.

Inspired my by super boss Richard Seroter with his regular daily reading list, I decided to record and share my tech watch, every week (or so). I always take notes of interesting articles I read for my own curiosity and to remember them when I need those references later on. But also to share them with Les Cast Codeurs podcast! So I hope it’ll be interesting to my readers too!

• LLMs Demand Observability-Driven Development
  A great tribune from Charity Majors on the importance of observability-driven development, in the wake of large language models. Developing LLM based solutions is typically not something you can do with a classical test-driven approach, as you only really get proper test data when you have it coming from production usage. Furthermore, LLMs are pretty much unpredictable and underterministic. But with observability in place, you can better understand why there’s latency in some scenarios, why the LLM came to certain solutions, and this will help you improve as your learn along the way.

  Read more...

As I started my journey with Generative AI and Large Language Models, I’ve been overwhelmed with the omnipresence of Python. Tons of resources are available with Python front and center. However, I’m a Java developer (with a penchant for Apache Groovy, of course). So what is there for me to create cool new Generative AI projects?

When I built my first experiment with the PaLM API, using the integration within the Google Cloud’s Vertex AI offering, I called the available REST API, from my Micronaut application. I used Micronaut’s built-in mechanism to marshal / unmarshal the REST API constructs to proper classes. Pretty straightfoward.