Nucleus AI, a California-based startup composed of talented individuals from industry giants Amazon and Samsung Research, has recently made its debut in the AI space. With its first product, a 22-billion-parameter large language model (LLM), Nucleus AI aims to revolutionize various sectors, particularly agriculture. This general-purpose model, available under both an open-source MIT license and commercial license, surpasses models of comparable size and offers opportunities for fine-tuning for different applications and tasks.
Unlike existing LLM companies such as OpenAI, Anthropic, and Cohere that focus on developing chatbots, Nucleus AI has set its sights on transforming agriculture through AI-driven solutions. By employing its 22-billion-parameter transformer model, the startup plans to create an intelligent operating system specifically designed to optimize supply and demand and address uncertainties faced by farmers. The ultimate vision is to establish a marketplace-like platform that hyper-optimizes demand and supply for farmers, akin to how Uber revolutionized the taxi industry.
The training process for Nucleus AI’s 22-billion-parameter model commenced approximately three and a half months ago, supported by compute resources from an early investor. Leveraging existing research and the open-source community, Nucleus AI pre-trained the model on a context length of 2,048 tokens and subsequently fine-tuned it using a trillion tokens of data. The training corpus comprised extensive and diverse information collected from sources such as the web, Wikipedia, Stack Exchange, arXiv, and code.
As the company moves forward, Nucleus AI plans to expand its model portfolio. Additional versions of the 22-billion-parameter model, trained on 350 billion and 700 billion tokens respectively, will be released. Additionally, two RetNet models, one with 3 billion parameters and another with 11 billion parameters, pre-trained on a larger context length of 4,096 tokens, will also be introduced. These smaller-sized models blend the best features of recurrent neural networks (RNNs) and transformer architectures, promising significant gains in terms of both speed and costs. Internal experiments have demonstrated that these models achieve processing speeds 15 times faster and memory requirements only a quarter of what comparable transformer models typically demand.
By harnessing the power of AI and LLMs, Nucleus AI aims to address multiple challenges faced by farmers. From climate change-related issues to knowledge gaps and supply chain optimization, the intelligent operating system being developed by the startup seeks to provide a comprehensive solution. By optimizing supply and demand and maintaining efficient distribution, the farming-centric OS has the potential to revolutionize the agricultural landscape. With this ambitious goal, Nucleus AI’s research endeavors transcend competition with other algorithms, setting the stage for the company’s entry into the farming industry.
In the coming weeks, Nucleus AI intends to provide greater insights into their farming-centric OS and announce further details regarding the RetNet models. As the startup continues to innovate and push AI capabilities to new limits, agriculture may witness a transformative revolution. By leveraging advanced language models and cutting-edge technology, Nucleus AI stands at the forefront of an agricultural renaissance, ready to optimize supply chains, empower farmers, and mitigate uncertainties in the field.
With its breakthrough product and visionary aspirations, Nucleus AI has already begun making its mark on the AI landscape. As the company progresses towards its goal of addressing real-world challenges in agriculture, the possibilities for the future are truly exciting.
Leave a Reply