“Mapping the future of *truly* open source models” to talk about

Latent Space: The AI Engineer Podcast

How AI is eating Finance — with Mike Conover of Brightwave

Overview

  • Brightwave is an AI startup for financial services that functions as a thought partner for finance professionals, helping them generate comprehensive analyses, answer complex market questions, and process information across filings and transcripts for clients ranging from small RIAs to large hedge funds.
  • The company employs a sophisticated systems of systems approach to AI, recognizing that large context windows alone don't solve document comprehension challenges, instead focusing on specialized subsystems, semantic document processing, and knowledge graphs with unprecedented granularity.
  • To address factual accuracy and hallucinations, Brightwave implements multiple verification strategies including model voting, entailment assessment, and human annotation benchmarks, while maintaining strict data privacy controls essential in regulated financial environments.
  • Rather than replacing traditional financial modeling, Brightwave aims to augment human decision-making by automating idea generation, identifying noteworthy patterns across complex economic relationships, and generating investment theses while acknowledging that humans remain essential for strategy validation.

Content

Background and Career

  • Mike Conover has an extensive research background in large-scale data analysis:
- PhD research focused on propaganda and misinformation networks - Worked at LinkedIn analyzing economic graph and job transitions - Conducted research demonstrating how job transition data could predict S&P 500 market cap changes

  • Conover views organizations and markets as complex systems:
- Sees digital trace data as a way to observe systemic interactions humans can't normally perceive - Compares organizational decision-making to social insect behavior - Believes language models are "compressions" of the world at their training time, inherently encoding societal structures and beliefs

Brightwave Overview

  • Brightwave is Conover's new startup:
- Raised $6 million seed round with investors including Decibel, Point72, and Moonfire Ventures - Focus on financial services and asset management - Goal is to use AI to identify market insights humans might miss

  • Co-founder Brandon Katara serves as CTO with a unique background:
- Experience at a federally regulated derivatives exchange - Served as tech lead for semantic search at Workday - Holds an early deep learning patent from 2018

  • Team building strategy balances three critical elements:
- AI technical capabilities - Systems engineering expertise - Deep domain-specific knowledge

Product Capabilities and Customer Base

  • Brightwave functions as an AI partner for finance professionals with capabilities including:
- Generating comprehensive financial analysis - Answering complex industry/market questions - Analyzing supply chains and market trends - Rapid information processing across filings and transcripts

  • Customer base ranges from $500 million to tens of billions in assets under management:
- Small owner-operated RIAs - Crossover hedge funds - Investor relations teams - Corporate strategy professionals

  • Key value proposition:
- Quickly generate actionable financial insights - Help professionals prepare for client interactions - Provide comprehensive market understanding

Technical Approach and Challenges

  • Context window limitations:
- Initially believed large context windows would enable comprehensive content synthesis - Empirically, models struggle with high-quality, deep insights across very large documents - Models tend to have a characteristic output length (around 1200 tokens) - As context window increases, probability of generating an end token becomes higher

  • Information processing approach:
- Advocates for a "systems of systems" approach with specialized subsystems - Emphasizes creating subsystems that perform specific tasks well - Recognizes challenges in decomposing large documents into atomic reasoning units

  • Document processing strategies:
- Effective processing requires domain expertise in understanding document structure - Chunking should focus on semantic intent, not just sliding window approaches - Selective retrieval based on document semantics is important

Factuality and Hallucination Challenges

  • Addressing factual accuracy through multiple approaches:
- Multiple model generations and comparison - Treating verification as a voting problem - Training models to assess entailment - Implementing product features for information verification

  • Evaluation methodology:
- Human annotators remain the "reference standard" for evaluating generative AI outputs - LLM supervision and heuristics used for initial quality assessment - Creating small, high-quality domain expert annotation corpora - Comparing automated evaluations against human benchmarks

  • Data quality considerations:
- Creating reproducible evaluation rubrics is difficult, even for skilled human annotators - The Google search quality guidelines demonstrate the complexity of defining "high quality" - Methodological approaches and data generation are key competitive advantages

User Interaction and Personalization

  • Views AI as a "thought partner" that complements human knowledge:
- Observes users' "revealed preferences" rather than requiring explicit descriptions - System suggests natural next questions to guide investigation - Personalization helps develop implicit representations of user beliefs - User interaction reveals what they care about and find important

  • Data privacy and confidence:
- Background in regulated financial environments informs approach to data management - Strict controls on data access and sensitive information - Systems designed to combine public and private data securely

RAG and Knowledge Graph Approach

  • Retrieval Augmented Generation (RAG) implementation:
- Uses composable prompts that adapt based on document semantics - Focuses on understanding document origin and trustworthiness - Propagates metadata and context through AI inference processes - Differentiates between high and low-quality information sources

  • Knowledge graph development:
- Creating highly detailed knowledge graphs to extract structured information - Goal of unprecedented granularity (e.g., tracking specific congressional testimonies) - Single-pass information extraction for reasoning and inference

  • Fine-tuning vs. RAG perspective:
- Skeptical about imbuing LLMs with new information through fine-tuning - Views fine-tuning as a way to "differentiate" models into specific behavioral modes - RAG's key benefit is "grounded reasoning" - forcing models to attend to specific facts

Financial Analysis Approach

  • Bright Wave's approach to financial analysis:
- Does not aim to create Excel spreadsheets - Focuses on quantitative reasoning with high-quality data retrieval - Prioritizes a "partner in thought" modality allowing dialogue and flexibility

  • Critique of traditional financial modeling:
- Excel spreadsheets are non-fault tolerant - Small errors can invalidate entire models - Current financial modeling is often very personal and context-specific

  • AI's potential in investment analysis:
- Can automate idea generation processes - Parse documents and find second-order derivative investment opportunities - Identify noteworthy fact patterns - Generate investment theses across complex economic relationships

Future Outlook

  • Different investment domains have different statistical distributions:
- Public equities: Bell curve/normal distribution - Venture capital: Power law distribution, making AI application more challenging

  • Human-AI collaboration remains essential:
- Humans critical for assessing strategy coherence - Synthesizing knowledge and making final investment decisions - Validating AI-generated insights

  • Model training and industry trends:
- Private evaluation corpuses crucial for measuring model performance - Recent models showing similar behaviors due to overlapping training data - Future innovation may focus on instruction tuning and fine-tuning - Lower barriers to experimenting with model fine-tuning enabling more research

  • Brightwave is hiring across multiple technical domains:
- AI engineering - Classical machine learning - Systems engineering - Distributed systems - Front-end engineering and design - Philosophy emphasizes finding exceptional people and adapting roles to fit individual talents

More from Latent Space: The AI Engineer Podcast

Explore all episode briefs from this podcast

View All Episodes →

Listen smarter with PodBrief

Get AI-powered briefs for all your favorite podcasts, plus a daily feed that keeps you informed.

Download on the App Store