30th Edition: 3/8/24

Anthropic's new Claude 3.0 LLM family, LLM inference routing emerging as a trend, AI2's $200M GPU cache, and MSFT's 70% more efficient LLM architecture...

Greetings from Japan! Let's get started…

While general purpose LLM companies (Mistral, Meta, OpenAI etc.) are continue their arms race toward human-level performance, developers' switching costs between LLMs are asymptotically approaching zero. This is creating an entirely new inference-layer market, as Foundation Capital's Feb 2024 report outlines. I highly recommend giving this a read (thanks to Ian for sending). Let me know what you think!

New businesses acting as the "single pane of glass" are emerging, using reinforcement learning to route queries to the highest-graded LLM for that specific prompt. Similar to Mixture of Experts (MoE - discussed in my newsletter here) architecture within single LLMs, this routing phenomenon is effectively MoE cross-model. One startup building this is Inovia-backed Not Diamond. Crazy.

Not going to write about the Elon/Sam Altman conflict.

PNW AI News:

• Microsoft Research just published a new paper: “The Era of 1-bit LLMs”, introducing a new LLM variant breakthrough called BitNet b1.58 that’s just 1-bit in size, where each parameter is ternary {-1, 0 , 1}

  • 70% more energy-efficient in most outputs

  • Attaching the parameter transform proof below, if anyone interested in the math

• AI2 Incubator secured $200M worth of AI compute resources (primarily NVIDIA H100's) for portfolio companies, up to $1M/company

• Anthropic is expanding is workforce footprint in Seattle with 20+ open positions, deepening ties with investor Amazon

• Microsoft Sr software engineer just escalated concerns to the FTC regarding Copilot Designer, claiming MSFT is not doing enough to prevent the AI from generating harmful content with the image generation model DALL-E

• Amazon made a $53M investment into Glacier, an AI startup working to improve recycling efficiency and reduce waste

Key AI Product Updates:

• Nvidia's upcoming B100 GPU, with anticipated 1,000-watt power consumption (42% higher than predecessor), raises concerns about energy use and cooling.

  • AI-driven Energy consumption surge becoming evident. Room for new energy-efficient server centers isolated from traditional electric utility grid

• Anthropic has introduced Claude 3, a family of AI models with varying capabilities that surpass GPT-4 in most benchmark evaluations.

  • Users can choose between Haiku, Sonnet, Opus, or a MoE of all 3, which have different balances of intelligence and cost

• Ebay is building its own LLM trained on its e-commerce data, with plans to double its GPU capacity over the next year

• An Oxford study in Ghana revealed that students using Rori, an AI-powered chatbot math tutor, achieved significantly higher math scores compared to just lessons.

Key AI Business/Investment Updates:

• The US Army Research Lab is reportedly testing commercial AI chatbots like GPT-4 as battlefield planning assistants in war game simulations

• AI chatbots from TurboTax and H&R Block were shown to give misleading or incorrect answers almost 50% of the time, according to a Washington Post study

• Palantir won a $178M contract from the U.S. Army to deliver 10 prototypes of the TITAN vehicle, which provides  soldiers with advanced battlefield intelligence using AI

• Alibaba led a $600M investment into Chinese AI startup Minimax at a $2.5B valuation, its second major AI deal in 2024

• Overjet raised a $53M Series C round from March Capital, General Catalyst, and other to advance its dental AI automation platform

PNW AI/ML Fundings:

• PreemptiveAI (Seattle, WA) raised $6.4M from Inspired Capital, AI2, Precursor Ventures, Meridian Street Capital, and more

  • Predictive medical model that maps human physiology/pathology in real-time via biomedical signals from smartphones - TFTD $6.4M

  • CEO: Jamien McCullum

• Enzzo (Seattle, WA) raised $3M in Seed funding led by Unlock VP, joined by the joint investing partnership between PSL Ventures and Mayfield Fund

  • Using AI in the hardware development process to autogenerate requirements, review, refine, and approve designs with ease - TFTD $3M

  • CEO: Ford Davidson