• Some friends shared this article from Uwe Friedrichsen, tilted back to the future, that talks about this feeling of “déjà-vu”, this impression that in IT we keep on reinventing the wheel. With references to mainframes, Uwe compared CICS to Lambda function scheduling, JCL to step functions, mainframe software development environments to the trendy platform engineering. There are two things I like about this article. First of all, it rings a bell with me, as we’ve seen the pendulum swing as we keep reinventing some patterns or rediscovering certain best practices, sometimes favoring an approach one day, and coming back to another approach the next day. But secondly, Uwe referenced Gunter Dueck who talked about spirals rather than a pendulum. I’ve had that same analogy in mind for years: rather than swinging on one side to the next and back, I always had this impression that we’re circling and spiraling, but each time, even when passing on the same side, we’ve learned something along the way, and we’re getting closer to an optimum, with a slightly different view angle, and hopefully with a better view and more modern practices. Last week at FooConf #2 in Helsinki, I was just talking with my friend Venkat Subramaniam about this spiral visualisation, and I’m glad to see I’m not the only one thinking that IT is spiraling rather than swinging like a pendulum.

  Read more...

• The State of AI report is pretty interesting to read (even if long!). Among the major sections: research, industry, but also politics, safety, and some predictions. You’ll find an executive summary in one slide, on slide #8.
  
  On #22, emergent capabilities of LLMs is covered and mentions Stanford’s research that talks about the importance of more linear and continuous measures as otherwise capabilities sound like they emerge out of the blue.
  
  On #23, they talk about the context length of LLMs being the new parameter count, as models try to have bigger context windows.
  
  However, on slide #24, they also talk about researchers who showed that in long context windows the content provided in the middle is more ignored by LLMs compared to content at the beginning or end of the window.
  So be sure to put the important bits first or last, but not lost in the middle.
  
  Slide #26 speaks about smaller models trained with smaller curated datasets and can rival 50x bigger models.
  
  Slide #28 wonders if we’re running out of human-generated data, and thus, if we’re going to have our LLMs trained on… LLM generated data!

  Read more...

• Stop Using char in Java. And Code Points
  It’s a can of worms, when you start messing with chars, code points, and you’re likely going to get it wrong in the end. As much as possible, stay away from chars and code points, and instead, use as much as possible the String methods like indexOf() / substring(), and some regex when you really need to find grapheme clusters.

• Paul King shared his presentations on Why use Groovy in 2023 and an update on the Groovy 5 roadmapIt’s interesting to see how and where Groovy goes beyond what is offered by Java, sometimes thanks to its dynamic nature, sometimes because of its compile-time transformation capabilities. When Groovy adopts the latest Java features, there’s always a twist to make things even groovier in Groovy!

  Read more...

• Generative AI exists because of the transformer
  I confess I rarely read the Financial Times, but they have a really neat articles with animations on how large language models work, thanks to the transformer neural network architecture, an architecture invented by Google in 2017. They talk about text vector embeddings, how the self-attention makes LLM understand the relationship between words and the surrounding context, and also doesn’t forget to mention hallucinations, how “grounding” and RLHF (Reinforcement Learning with Human Feedback) can help mitigate them to some extent.

  Read more...

Inspired my by super boss Richard Seroter with his regular daily reading list, I decided to record and share my tech watch, every week (or so). I always take notes of interesting articles I read for my own curiosity and to remember them when I need those references later on. But also to share them with Les Cast Codeurs podcast! So I hope it’ll be interesting to my readers too!

• LLMs Demand Observability-Driven Development
  A great tribune from Charity Majors on the importance of observability-driven development, in the wake of large language models. Developing LLM based solutions is typically not something you can do with a classical test-driven approach, as you only really get proper test data when you have it coming from production usage. Furthermore, LLMs are pretty much unpredictable and underterministic. But with observability in place, you can better understand why there’s latency in some scenarios, why the LLM came to certain solutions, and this will help you improve as your learn along the way.

  Read more...