Overview
* The AI Engineer role has emerged as a distinct discipline focused on applying foundation models to quickly ship products, with a "fire, ready, aim" approach that contrasts with traditional ML engineering's more methodical processes.
* AI engineering requires a unique skill blend of product thinking, basic ML knowledge, and engineering capabilities, sitting between hardcore ML engineers and front-end developers on the spectrum from "data-constrained" to "product-constrained" work.
* Vertical AI startups are outperforming horizontal ones by targeting specific domains with proprietary data and high-margin markets, making them less vulnerable to disruption from large AI companies.
* The AI landscape is experiencing rapid commodification with annual 5-10x cost reductions for comparable performance, while capabilities in inference speed, context length, and multimodality continue to expand dramatically.
* Organizations should initially purchase rather than build AI tooling infrastructure, focusing on evaluation platforms and monitoring tools while leveraging community-solved problems before developing custom solutions.
Content
The Emergence of the AI Engineer Role
* The conversation begins with a discussion about the emergence of the "AI Engineer" role, one year after Sean Wang's original essay on the topic. * This new engineering discipline focuses on applying AI technologies, particularly foundation models, which are transforming software engineering. * Key characteristics of AI Engineering include: - More focus on the "zero-to-one" phase compared to traditional ML Engineering - Emphasis on rapid iteration and market feedback - Requirement to quickly ship products and learn from market responses * Sean advocates for a "fire, ready-aim approach instead of a ready-aim-fire approach" to AI development. * AI has dramatically reduced development time: tasks that took 5 years in 2013 now require "API docs and a spare afternoon in 2023." * The AI Engineer role sits between hardcore ML engineers and front-end developers on a spectrum from "data-constrained" to "product-constrained."
Shifting Landscape of AI Talent and Engineering
* The AI landscape is evolving rapidly: - AI work is increasingly moving from internal company teams to between companies - The cost of creating AI models is rising, and model labs are becoming more closed - AI talent is being outsourced more frequently due to foundation models * Specialist engineers who stay up-to-date with the rapidly evolving AI stack have an advantage. * Many AI tasks that previously required years of research can now be accomplished in hours using APIs. * Traditional qualifications like PhDs are becoming less critical in the new AI landscape.
Distinguishing ML Engineers from AI Engineers
* ML engineers and AI engineers have fundamentally different approaches and skill sets: - ML engineers focus on one-to-end problems with large datasets, optimizing specific models - AI engineers focus on zero-to-one phases, are more product-oriented and full-stack * AI Engineers typically: - Have more front-end skills - Are closer to product thinking - Use tools like JavaScript more frequently - Focus on making models into useful products * The suggested team ratio is approximately 4 AI engineers to 1 ML engineer. * There's a lower barrier to entry for AI engineers compared to ML engineers. * Success in one engineering domain (ML vs AI) doesn't necessarily translate to the other.
Non-Engineers in AI Development
* Product managers and domain experts are becoming increasingly crucial in AI product development. * They provide critical customer insights and product direction that cannot be replaced by AI engineering. * Product managers can now more directly participate in creating product artifacts through prompts. * The ideal AI engineer embodies: - Product thinking - Basic ML knowledge - Sufficient engineering skills * About 50% of engineers are still AI skeptics (e.g., not using tools like GitHub Copilot). * AI development is shifting towards more direct collaboration between product managers, domain experts, and AI engineers.
Current Status and Evolution of the AI Engineer Role
* AI Engineering is currently perceived as a low-status role with a low barrier to entry. * The role lacks a clear, standardized definition but is being adopted by companies like MasterCard and OpenAI. * There's no established skill ladder yet for AI engineers. * OpenAI requires 5-7 years of ML engineering experience for some AI engineering roles. * Being early in the field has advantages: - Easier to keep up with evolving techniques and knowledge - Opportunity to establish foundational understanding * However, early entry doesn't guarantee success, as demonstrated by AutoGPT's rapid rise and subsequent decline. * "AI engineer" is emerging as the preferred title over alternatives like "LLM engineer."
The AI World Fair Conference
* The conference is expanding from a single-track to a nine-track event this year. * New tracks include: - Multimodality - Evals and ops - AI in Fortune 500 - VPs of AI leadership track - Traditional tracks like RAG and code generation * Targeting 2,000 attendees (currently at 1,500). * Last year's event had 500 in-person attendees, 20,000 online attendees, and 150,000 asynchronous views for top talks. * The conference aims to create networking opportunities beyond just talks and bring together diverse perspectives in AI engineering. * There's a shift to include enterprise perspectives beyond startup-focused discussions.
Strategic Approach to AI Product Development
* Sean advocates for moving fast and challenging traditional development timelines. * Recommends a "fire, ready, aim" approach instead of "ready, aim, fire." * Emphasizes rapid deployment and iterative improvement based on market feedback. * Most companies can experiment with generative AI without excessive caution.
Vertical vs. Horizontal AI Startups
* Vertical startups are performing better and seeing more market success. * Vertical startups have advantages: - More proprietary data - Target high-margin, price-insensitive markets - Solve specific domain problems - Less likely to be disrupted by large AI companies like OpenAI * Promising vertical AI startup examples include: - Harvey AI (legal tech) - MidJourney (creative market) - Construction-focused AI startup - Perplexity (positioned as an "anti-Google" alternative) - Developer tooling (e.g., Cursor, Co-pilot) - Real estate AI (interior staging) - Medical research AI - Financial research AI (e.g., Bright Wave) - Summarization tools * Vertical-specific products can more easily achieve product-market fit (PMF).
AI Tooling and Procurement Advice
* Different categories of AI tooling include: - OpenAI API management - AI product tooling - Internal productivity tooling * Developer tools like Copilot, Cursor, and SourceGraph are becoming baseline adoption. * Recommended to initially buy tools from the community rather than building in-house. * Benefits of buying include: - Leveraging community-solved problems - Moving faster - Understanding specific needs before potentially building custom solutions * Suggested initial purchased tools include evaluation platforms and API observability and monitoring tools.
AI Capabilities and Competitive Landscape
* Emerging concept of "virtual employees" that can perform specific job tasks. * AI capabilities are not directly comparable to humans - AIs are already superhuman in some dimensions while weaker in others. * Four critical competitive domains for AI companies: - Fighting over data - Fighting over GPU resources - Competing between generalized vs. domain-specific models - Competing in retrieval-augmented generation (RAG) and operations * The "Sour Lesson" suggests humans and AI learn/develop differently.
AI Research and Development Trends
* Research directions ranked in order of importance: 1. Long inference 2. Synthetic data 3. Alternative architectures 4. Mixture of experts and model merging 5. Online outlets * "Moore's law of AI" suggests significant annual improvements: - MMLU performance cost decreases 5-10x yearly - GPT-4 level model costs dropped from $20 to $2 per million tokens - Costs expected to continue trending towards zero * Emerging AI capabilities include: - Increasing inference speed (targeting 5,000 tokens per second) - Expanding context length (from 4,000 to 1 million tokens) - Growing multimodal capabilities - Commodification of intelligence
Temperature and Creativity in AI
* Contrasting "temperature zero" vs "temperature two" use cases: - Temperature zero: Locking down models for specific retrieval tasks - Temperature two: Embracing hallucination and creativity as a feature * Viewing AI-generated unexpected ideas as valuable. * Combining AI models as "conjecture machines" with testing mechanisms. * Goal of generating new knowledge through high-temperature, non-deterministic modes.
Closing Information
* Conference details available at ai.engineer domain. * Discount code: iagency for last-minute tickets. * Speaker contact: Sean on Twitter (@Swix). * Podcast: latent.space * Host: Reza Habib * Podcast: High Agency * Additional details at humanloop.com/podcast