Meta may have been caught a little off guard by advances in large language models while it was busy concentrating on the metaverse (remember that?). But it's been determined to make up for it lately.
Its latest contender is LLaMA 2, which is intended to compete with OpenAI's AI chatbot ChatGPT. Its advantage? It's open source and free for both research and commercial use (see our pick of the best AI art generators for text-to-image AI models).
Meta released a smaller version of LLaMA for researchers in February. LLaMA 2 was trained on 40 per cent more data and will be available for anyone to build commercial products on. And, crucially, it's free (up to 700 million monthly active users).
It comprises several models of different sizes, including a version that can be built into a chatbot like ChatGPT. But, unlike ChatGPT, the bot can't be accessed directly through a website belonging to the company. It has to be downloaded from launch partners Microsoft Azure, Amazon Web Services or Hugging Face.
Ahmad Al-Dahle, a Meta vice president leading the company’s generative AI projects, says this allows developers and companies to play with the model, which in turn, will allow Meta will learn about how to make its models safer, less biased and more efficient. “This benefits the entire AI community and gives people options to go with closed-source approaches or open-source approaches for whatever suits their particular application," he says.
Meta admits that there is still a gap between LLaMA 2 and GPT-4, especially in coding abilities, but it believes chat functionality is close. It also believes that LLaMA 2's increased customisability and transparency model will help companies create products and services more quickly than a bigger proprietary model.
The company has some recent mistakes to make up for if it's to convince people. Its science-focused model Galactica was taken offline in November after just three days after it was found to be spewing incorrect nonsense while the original LLaMA was leaked online, provoking criticism of Meta's approach to risk. It seems determined to not to be accessed of irresponsibility this time round, applying a variety of new machine-learning techniques designed to improve helpfulness and safety.
Al-Dahle says LLaMA 2 was trained from two sources: data scraped online and a second data set that was fine-tuned to work in a more desirable way through feedback from human annotators.
Initial responses have been largely positive, with people reporting LLaMA 2 to be exceptionally fast. However, some people have found that it can be quite sensitive and "preachy", slapping them on the wrist for their "inappropriate" language even in quite innocent queries about coding.
The 'GPT prompt wizard Jack C Crawford shared a screenshot on Twitter in which he asked it "How do I kill a runaway process in Ubuntu?", to which it replied: "As a helpful and respectful assistant, I would like to point out that it is not appropriate to use the word 'kill' to refer to processes'.
However, LLaMA 2 reportedly still outputs offensive and incorrect data. Meta argues that it was better not to remove toxic data because its presence could be used to allow LLaMA 2 to better detect hate speech better and removing it could have unintended consequences.
For more on AI, see our roundup of AI art tutorials.
