It was startling to read this and see where I am and my organization are amidst Ai adoption.

Enterprises can achieve AI value through small-scale efforts on three levels: boosting individual productivity, incorporating AI into defined tasks, and automating production processes. (source)

If I had to gauge where I am in this, it’s at boosting individual productivity, then showing others how to do the same. It’s not an easy thing to get a grasp on because it means re-thinking and being meta cognitive about your work…and those insights are a slow drip rather than a fast flow…but AI tools are getting better, and the speed of the drops increasing ever so slowly (more my limitations than AI).

Scaling AI Results: Strategies from MIT Sloan Management Review

I. Introduction: The Reality of Enterprise AI Transformation

“When MIT Sloan senior lecturers sought examples of enterprises that had achieved major transformations using generative AI, they didn’t find any.”

• Current state: No major enterprise transformations achieved through generative AI yet
• Key finding: Smart leaders achieve big value through small-scale, systematic AI efforts
• Focus: Practical strategies for achieving real results from AI investments

II. Three-Level Framework for AI Value Creation

“Smart leaders get big value from small-scale AI efforts deployed through a measured, systematic approach.”

A. Level 1: Individual Productivity Enhancement

1. Create safe environments for employee experimentation
2. Common use cases:

• Inbox management and email optimization
• Meeting transcription and summarization
• Calendar optimization
• Briefing preparation

1. Cross-cultural communication support

• Adapting tone and cultural norms in business writing
• Example: Europeans addressing American audiences

B. Level 2: Task and Role Integration

1. Developer support functions:

• Code writing assistance
• Data analysis
• Documentation creation

1. Customer-facing applications:

• AI agents for sales representatives
• Call center quick answer systems

1. Design team capabilities:

• Proposal generation from minimal text input
• Real-time visualization during client meetings

C. Level 3: Production and Operational Automation

1. Marketing automation:

• Complete campaign creation beyond just content

1. Enterprise software integration:

• Supply chain management automation
• Workforce skills gap identification
• Conversational AI interfaces for all processes

III. Vanguard Group Case Study: Proven ROI

“Vanguard Group estimates that its ROI from AI is close to $500 million.”

A. Successful AI Implementations

1. Call center AI agents:

• Draw answers from internal content
• Faster issue resolution

1. Personalized adviser summaries:

• Autogenerated market perspectives
• Enhanced client communication

1. Programming productivity gains:

• 25% improvement in productivity
• 15% reduction in system development lifecycle

1. Financial analysis tools:

• LLM analysis of earnings calls
• Dividend cut signal detection

B. Implementation Philosophy

1. Measured rollout approach:

• Dozens of pilots in testing
• Scale deployment only after “kinks worked out”

1. Continuous monitoring:

• AI model performance tracking
• Utilization metrics

1. Training investment:

• 50% employee completion rate for Vanguard AI Academy

IV. Understanding LLM Capabilities and Limitations

“Knowing how large language models work is a necessary foundation for making sound business decisions.”

A. Key Technical Insights

1. Live data access requirements:

• Models can answer post-training questions only with live data access

1. Document upload limitations:

• No guarantee of limiting responses to uploaded documents
• Models reference similar training set documents

1. Context window considerations:

• Large windows can hold entire books
• Too much irrelevant information hurts performance
• Models focus on beginning and end of prompts

B. Hallucination Management

1. Cannot be eliminated entirely
2. Mitigation strategies:

• Use second model for verification
• Focus on structured tasks
• Employ easily validated data formats

V. Governance Model: Decentralized Rule-Making

“Executives should erect guardrails, but individual teams should define the rules for AI use.”

A. Current Problem

1. Only 47% of professionals say AI policies reflect work realities
2. Different departments use AI differently
3. “Judgment is local” principle

B. Recommended Approach

1. Executive responsibilities:

• Define privacy, security, IP, and ethics policies
• Build AI platforms
• Provide training programs
• Set guardrails, not detailed rules

1. Team leader responsibilities:

• Create specific practices from broad policies
• Determine where, when, and how to implement AI
• Ensure rules match day-to-day work realities

C. Benefits of Decentralization

1. Avoid back-channel AI use that creates risk
2. Prevent ignoring of company AI investments
3. “Decentralization is not abdication”

VI. Strategic Implementation Principles

“To make progress on strategy, leaders need to balance immediate action with long-term thinking — to ‘build the scaffolding.'”

A. Building the Foundation

1. Balance immediate action with long-term thinking
2. Create organizational scaffolding for AI
3. Align AI efforts with core business capabilities

B. Avoiding Common Pitfalls

1. Pilot projects staying on sidelines
2. Misalignment with business capabilities
3. Top-down rule creation that doesn’t reflect reality

VII. Next Steps and Action Items

A. For Executive Leadership

1. Establish privacy, security, and ethics guardrails
2. Invest in AI platform infrastructure
3. Create comprehensive training programs (following Vanguard’s 50% completion model)
4. Monitor overall ROI and strategic alignment

B. For Department/Team Leaders

1. Translate corporate guardrails into specific team practices
2. Identify high-value use cases within three-level framework
3. Document and share successful implementations
4. Provide feedback on policy effectiveness

C. For Individual Contributors

1. Experiment within safe environments for productivity gains
2. Complete available AI training programs
3. Share use cases and learnings with teams
4. Focus on measurable productivity improvements

D. For IT/Technology Teams

1. Build and maintain AI platforms
2. Monitor model performance and utilization metrics
3. Support pilot programs with measured rollout approach
4. Ensure technical infrastructure supports all three levels of implementation