E39: 5/10/24
MSFT in-houses a new LLM to combat over-dependence on OpenAI models, the “benchmark overfit” controversy virally unfolds, Randy Travis releases an entirely AI-generated song (more controversy !), etc.
Hey everyone, here’s this week’s update! FYI, I’m likely going to missing out on next week’s edition, since I will likely be without service on next Thu-Fri in Morocco.
Also, see the end of this article; It’s been a ~year since I initially outlined Vector Databases to the FUSE team, so I’m reattaching my old essay on them for a re-read… VDBs are so much larger just 12 months later.
PNW AI News:
Microsoft is planning to invest $3.3B in Wisconsin-based AI data centers funded in slight by Biden's CHIPS Act, expected to create 2.5K jobs
Microsoft launched a top-secret genAI service for U.S. intelligence agencies like the NSA, CIA, which uses an isolated GPT-4 model to securely analyze sensitive info
NVIDIA invests in Seattle-based Carbon Robotics, a computer vision-enabled weed zapping agricultural robot, citing it as 'one of the furthest along vertical applications of AI'
Microsoft is building an in-house LLM dubbed MAI-1, with 500B parametersand co-developed by DeepMind founder Mustafa Suleyman, but is still expected to fall short of GPT-4 performance
AI2 spinout Lexion, an AI-enabled contract lifecycle mgmt startup, was acquired by DocuSign for $165M
Key AI Product/Research Updates:
In response to a MAJOR outcry against LLMs overfitting to perform against test benchmarks (as detailed in last newsletter), LMsys has gone viral for ELO-based benchmarking that analyzes model performance across language, query complexity, modality, etc.
Also in response to benchmark controversy, OpenAI introduced Model Spec, framework that outlines their approach to shaping the behavior of its models
Also an effort to re-establish trustworthiness, amidst controversies like GDPR complaints, lawsuits from NYT and other news agencies, and StackOverflow users deleting their data after its OpenAI partnership announcement (read more below)
Google DeepMind and Isomorphic Labs just introduced AlphaFold 3, an updated version of their groundbreaking AI model that can predict the structure of proteins, DNA, and other molecules with extreme accuracy
50% accuracy improvement on inferring proteins' response to certain drug exposures, incorporating learnings from LlaMa-3 and Sora
Apple unveiled its new line of iPads, which feature custom M4 chip that enables advanced AI capabilities run locally
Hugging Face launched LeRobot, an open-source code library featuring tools, pre-trained models, and a community-driven effort aimed at democratizing robotics
A new model called DeepSeek V2 has taken over Mistral's 8x22B mixture of experts (MoE) LLM in efficiency with >160 expert models at HALF the cost
Snowflake Arctic was the last MoE model record holder at 128 exprts
The LLM race to the bottom is now leaking into MoE models too
Nvidia researchers just introduced DrEureka, an AI system that uses LLM agents to automate the process of training robot skills in simulations and transferring them to the real world
Competitive to Seattle-based, Khosla-backed Scaled Foundations
Key AI Business/Investment Updates:
Nashville is unpheavel after country music star Randy Travis just released his first new song in over a decade, using AI to recreate his voice years after a stroke left him unable to speak or sing
Intense debates sparked about AI's place in music, arts, digital licensing
UK-based Abound raised a $800M Series B for its AI-powered lending platform that analyzes bank transaction data for personalized loan assessments
OpenAI's COO, Brad Lightcap, has been quoted calling today's OpenAI models "laughably bad", insinuating that GPT-5 will be able to support complex tasks, seamless teamwork-like interactions, and a total shift towards voice interfaces and multimodality
OpenAI is exploring the ability to "responsibly provide the ability to generate NSFW content" and "speak profanity" upon request in its model suite
Stack Overflow and OpenAI announce a new API partnership to strengthen the performance of LLMs, with OverflowAPI access to all OpenAI model users
PNW AI/ML Fundings:
Carbon Robotics (Seattle, WA) raised an undisclosed post-Series C round from NVIDIA's Nventures
An agricultural robotics company that focuses on developing AI and computer-vision enabled solutions for ag, starting with their "Laserweeder" - TFTD $85M
CEO: Paul Mikesell
FUSE portfolio company
Predict.law (Seattle, WA) raised an undisclosed Pre-Seed round from Mudita Venture Partners
A predictive AI tool intended to improve prediction of legal case outcomes - TFTD unknown
CEO: Patrick Wilburn
PNW AI/Startup Events:
Tried this for a couple weeks, axing it. Stay tuned for an AI version from Parsa who's using a multi-agent to scrape Discord, Reddit, Techstars, LinkedIn etc. to automate this :)
Vector Databases:
Math background
Through our Roboto investment, we have become intimately aware of the challenges of wrangling unstructured data. The commoditization of sensory data input and innovation in foundational LLMs has spurred the proliferation of process automation products that can be trained more easily, and on more complex logic. To train these AI/ML models, code programs assign point value function weights autoregressively to optimize and identify ground truth relationships. These autoregressive neural network loops take 20 computational steps per weight assignment iteration, on average. Existing AI infrastructure has reasonably streamlined this computing process for simple throughput data fields. For semi-structured/unstructured, complex data, this computation becomes burdensome. When these function weights have an increasing number of dimensions due to multimodal input data, they are computing in vector datapoints rather than point, static data.
In practice problem
In Roboto’s multimodal drone engineering data example, these ‘dimensions’ are Lidar, motor actuation, camera input, etc. However, these AI process automation products use ‘dimensions’ of fields being pulled from SaaS apps like ERPs, CRMs, etc. Modern innovation in AI render LLMs capable of semantically searching and analyzing business multimodal, unstructured vector datasets. Unfortunately, since this synthesis of unstructured data is computationally intensive to organize, the disparate data sources can become rigorous to connect to for AI models leveraging them. Vector database platforms are specifically architected to efficiently query and logically tabulate vector datasets, so they’re an effective solution for reducing computational intensively of AI models comparing logic from a higher degree of inputs. Since generative AI can now query the entire internet and several file formats, generative AI models have accelerated the need for solution to this efficiency bottleneck.
Implications
These AI/ML models are already being strained in latency and response time by viral consumer/business adoption. The models often incur significant cloud computing costs for the organizations operating them, creating another significant barrier to continued disruption by the AI category. Accordingly, investors have come to the belief that the company that significantly solves LLM computational efficiency limitations will be a massive outcome. Recent adoption of vector databases by customers like OpenAI and many AI-first software companies with their own proprietary models has proven these DBs commercial viability to significantly streamline the backend of these LLMs, hence why they’re receiving such significant investor attention.
Companies to watch
Pinecone raised $100M Series B at $750M valuation led by Andreesen Horowitz, joined by Iconiq Growth and existing investors Menlo Ventures & Wing Venture Capital
Tiger Global led their A and had to sit their pro rata out because no reserves allocated haha
Chroma raised $18M Seed round led by Quiet Capital, joined by Sam Altman and the founders of Motherduck (Seattle Madrona co), Notion, MongoDB, Scale, Jasper, etc.
Zilliz (Milvus’ open source vector DB) raised $60M in Series B led by Saudi Aramco’s CVC Prosperity7 Ventures, Temasek, HillHouse, and several Chinese investors
Qdrant raised $8M Seed round from Unusual Ventures, IBM Ventures, and a Cloudera co-founder
Weviate raised $16M Series A led by NEA, joined by Cortical Ventures and Boost VC
Read more
Pinecone’s description of Vector DBs
List of all major Vector DB platforms
Other sources
PitchBook, CB Insights