EIDR Modernizes Deduplication Efforts

For more than 15 years, the Entertainment ID Registry (EIDR) has used rules-based deduplication to help ensure content uniqueness by comparing new asset metadata against existing records.

As the scale and complexity of the content ecosystem have grown, EIDR has introduced a next-generation deduplication capability that incorporates machine learning to continuously improve performance over time.

“It’s still early, but we’re encouraged by what we’re seeing so far,” said Hollie Choi, Managing Director of EIDR. “As we continue to make decisions, the system learns and becomes more effective over time. It’s a positive step in how the Registry continues to evolve.”

The approach introduces a human-in-the-loop model, combining machine learning with expert review to handle edge cases and ensure accuracy.

“Moving to a human-in-the-loop system gives us the best of both worlds,” said Richard Kroon, EIDR Technical Director. “Machine learning handles the majority of matches, while human expertise remains critical for the most complex scenarios.”

The enhanced capability is supported by an AI-native master data management platform from Tamr, designed specifically for large-scale data mastering and entity resolution.

“We’re not just applying a general-purpose AI model to this problem,” Kroon added. “The system is purpose-built for metadata deduplication, and it does that one thing very well.”

As content volumes continue to grow across streaming, FAST, and global distribution workflows, EIDR’s investment in adaptive, learning-based technology ensures the registry can evolve alongside the industry it serves.