Overview

• Foundation model competition has intensified dramatically in 2024, with OpenAI's API market share dropping from 90% to 60%, while smaller models (70B parameters) are now only two points behind state-of-the-art performance, and model costs have decreased by 80-85%.

• The AI startup landscape is evolving beyond "GPT wrappers" toward four key opportunity areas: service automation for previously expensive tasks, enhanced information interaction, democratization of creative and technical skills, and transformation of traditionally challenging markets like healthcare and legal services.

• Despite concerns about an "AI bubble," funding appears more rational in 2024, with successful AI startups demonstrating they can achieve high revenue-to-employee ratios with relatively modest initial investment rather than requiring massive capital infusions.

• Companies building AI products should strategize based on the assumption that foundation models will continue to rapidly improve, focusing on unique value creation through novel capabilities, better products, or innovative business models rather than competing directly with foundation models.

• The development of AI agents remains in early stages, with significant uncertainty about required infrastructure primitives, while multimodal AI applications are currently more relevant in consumer spaces than enterprises, where most data remains text-based and structured.

Content: State of AI Startups and Top Themes for 2024

Conference Context

• Latent Space Live mini-conference at NeurIPS 2024 in Vancouver
• Keynote by Sarah Guo (Conviction founder) and Pranav Reddy (Conviction partner)
• Sarah Guo previously worked at Greylock for a decade
• Conviction fund focuses on AI infrastructure, foundation models, and applications
• Speakers view AI as the "greatest technical and economic opportunity" in their careers

Foundation Model Landscape Evolution

• Much closer competition compared to 2023 when OpenAI dominated unchallenged
• OpenAI's API market share dropped from 90% to 60% in less than a year
• Surprising developments:

• Open source models becoming increasingly competitive in areas like math, instruction following, and adversarial robustness
• Smaller models showing strong performance:

Model Economics and Capabilities

• OpenAI model API costs have dropped by 80-85% in the last year
• Creating large volumes of data now costs only a few thousand dollars
• New modalities developing across different domains:

• Debate continues about limits of model scaling with diminishing returns from simply increasing model size
• New scaling paradigms emerging, such as test-time compute scaling

AI and Startup Landscape

• Current AI capabilities work best in well-defined domains like math, physics, and software engineering
• Contrary to "AI bubble" narrative, funding appears more rational in 2024
• Foundation Model Labs raising significant funds ($30-40 billion), but overall startup funding seems more measured

Emerging AI Startup Trends

2. Enhanced Search and Information Interaction - Text-based modalities showing significant effectiveness - Potential for evolving beyond text to more engaging information formats

3. Democratization of Skills - AI enabling broader access to creative and technical capabilities - Surprising latent demand for creative tools (e.g., Midjourney) - Users often different from traditional target markets

4. Traditionally Challenging Markets - Legal, healthcare, defense, education becoming more viable with AI - Potential to dramatically reduce service costs - Changing market buying patterns - Enabling new business models

Value Creation and Competition

• Contrary to the "GPT wrapper" narrative, significant potential exists for innovation at the application layer
• AI ecosystem offers multiple model choices, price competition, open source options
• Startups can compete with incumbents by:

• Potential for elastic demand in services, particularly software development
• Current technological landscape favors startups due to rapid changes in software and data
• Large incumbent companies struggle to pivot quickly in the new paradigm

Challenges for AI Startups

• Data availability is critical - many incumbent companies lack detailed data needed for advanced AI applications
• Increased computational costs and focus on gross margins
• Product development requires rethinking traditional workflows and handling non-deterministic AI outputs
• Infrastructure management becoming crucial again, moving beyond simple cloud solutions
• Uncertain business models with potential for outcomes-based pricing
• Rapid growth can be misleading - companies face challenges in:

Investment Perspectives

• Speakers argue against assumption that all AI companies require massive upfront capital
• Advocate for smaller, more efficient fund sizes and company growth models
• Investment philosophy emphasizes "no GPU before product market fit"
• Some AI companies can be profitable or break-even with relatively low initial investment
• Many portfolio companies achieving high revenue-to-employee ratios
• Second-time founders increasingly focused on maintaining lean, efficient organizations
• Launched "Embed" program to engage with broader set of companies while providing network and guidance

Enterprise Multimodality Insights

• Currently limited enterprise demand for multimodal AI applications
• Most enterprise data is text-based and structured (SQL, databases)
• Multimodal capabilities more developed in consumer/prosumer spaces
• Future enterprise workflows may capture and utilize more video/audio data
• Companies like Highlight exploring on-screen and audio capture technologies

AI Intelligence Pricing and Adoption

• Large language models becoming increasingly affordable
• Demand for AI intelligence is elastic - cheaper models drive more complex use cases
• Foundation model providers offering more tokens at lower costs
• Inference capacity remains a significant technical challenge
• Compute requirements and inference capacity are ongoing constraints
• Test time compute scaling exponentially on a logarithmic scale

Strategic Considerations for AI Companies

• Companies should be excited, not anxious, when new AI models emerge
• Key strategy is to predict how foundation models will improve in core capabilities
• Build companies assuming models will become increasingly sophisticated
• Some companies (like early copywriting services) risk being quickly obsoleted by advanced AI models
• Risk of underestimating AI capabilities rather than overestimating them
• Potential for new hardware platforms driven by AI usage patterns

Agent Infrastructure and Development

• Current state of agent development is very early, with only a few working agents
• Significant uncertainty about infrastructure primitives needed for agents in production
• Potential useful infrastructure components include:

• Successful agent frameworks likely to emerge from companies solving their own internal challenges

Consumer AI Company Development

• Researchers were first to recognize AI's potential capabilities
• Young entrepreneurs were early adopters due to lower opportunity costs
• Experienced product professionals now starting to enter the AI startup space
• "Diffusion of innovation curve" applies to both customers and entrepreneurs
• Current portfolio heavily represented by researchers pushing technical boundaries
• More consumer AI companies expected to emerge in the next few years
• Building intuition for AI product interfaces takes time
• Consumer AI company development is a "matter of time" and will continue to expand as more diverse talent enters the space