At Videocrawl https://www.videocrawl.dev/, we’ve been exploring how AI assistants can enhance our development process. Recently, we built a text synchronization feature for our video player using Claude as our AI pair programmer. The feature highlights transcript text as a video plays, but the journey to get there revealed both the strengths and limitations of AI-assisted development.
The Initial Approach
We presented Claude with our requirements: synchronize transcript text with video playback, highlight the current text, and auto-scroll to keep it visible. Claude quickly generated a wireframe showing how the feature would look and proposed an initial implementation.
The first solution used custom HTML spans with direct styling to highlight words. While technically sound, this approach had a critical flaw: it broke our existing markdown rendering system. The highlighting was being applied at the DOM level after markdown processing, causing formatting inconsistencies.
As the developer, I had to intervene: “This breaks our markdown formatting. Can we use markdown bold tags instead of custom styling?”
Claude immediately pivoted to a new approach using markdown bold syntax (word), which preserved our existing formatting system. This was our first insight: AI needs guidance on system context that isn’t obvious from the code alone.
Tackling Unexpected Edge Cases
When testing the implementation, I noticed two significant issues that Claude’s code didn’t address:
First, when the same word appeared multiple times in a transcript, the highlighting would jump erratically between occurrences. Claude had implemented a simple text-matching approach that highlighted the first instance of each word rather than considering position in the transcript.
“What will happen if the transcript section has the same word multiple times? It will always pick the first occurrence, causing highlighting to jump backward and forward,” I pointed out.
Claude acknowledged the oversight and redesigned the algorithm to use positional indexing rather than text matching, ensuring the correct word was highlighted based on timing rather than content.
Refining the User Experience
The second issue was with auto-scrolling. Claude’s initial implementation was overly aggressive, immediately scrolling back to the highlighted text whenever the user tried to scroll manually.
I challenged Claude: “We’re unable to scroll down. Each time we do, we automatically go back up. We still want to give users the ability to scroll up and down.”
Claude then incorporated user scroll detection with a cooldown period, allowing manual scrolling to temporarily override the auto-scroll feature. This balance maintained the synchronization while respecting user control.
From Functional to Polished
Once the core functionality worked, I pushed for enhancements: “Rather than just bolding one word, can we bold a range—5 words before and 4 after the selected index? This will give better intuition.”
Claude redesigned the highlighting system to create a “sliding window” effect, with special emphasis on the current word and subtle highlighting on surrounding words for context. This transformed the feature from merely functional to genuinely intuitive.
The Collaborative Process
What made this development process unique was the iterative feedback loop. Claude provided working code quickly, while I contributed system knowledge and user experience insights. When Claude went down suboptimal paths, specific feedback redirected it toward better solutions.
This collaboration highlighted how AI can accelerate development while demonstrating that developer guidance remains essential, particularly for integrating new features into existing systems and creating polished user experiences.
Feature Demo
Discover more from Shekhar Gulati
Subscribe to get the latest posts sent to your email.
