• The State of AI report is pretty interesting to read (even if long!). Among the major sections: research, industry, but also politics, safety, and some predictions. You’ll find an executive summary in one slide, on slide #8.
  
  On #22, emergent capabilities of LLMs is covered and mentions Stanford’s research that talks about the importance of more linear and continuous measures as otherwise capabilities sound like they emerge out of the blue.
  
  On #23, they talk about the context length of LLMs being the new parameter count, as models try to have bigger context windows.
  
  However, on slide #24, they also talk about researchers who showed that in long context windows the content provided in the middle is more ignored by LLMs compared to content at the beginning or end of the window.
  So be sure to put the important bits first or last, but not lost in the middle.
  
  Slide #26 speaks about smaller models trained with smaller curated datasets and can rival 50x bigger models.
  
  Slide #28 wonders if we’re running out of human-generated data, and thus, if we’re going to have our LLMs trained on… LLM generated data!

  Read more...

When developing new projects on my laptop, I often run some commands over and over again. Regardless of how far you’ve gone with your CI/CD pipelines, running commands locally without resorting to becoming a bash ninja can be pretty easy with… just!

just is a handy way to save and run project-specific commands

It’s a command-line tool that lets you define some commands to run (called recipes), in the form of a Makefile-inspired syntax. It even allows you to define dependencies between the various tasks of your justfile. It runs across all environments (Mac, Linux, Windows), and is quick to install. It loads .env files in which you can define variables specific to your project (other developers can have the same justfile but have variables specific for their projects)