Home > Research > Publications & Outputs > DreamFactory

Electronic data

  • 2408.11788v1

    Final published version, 7.03 MB, PDF document

Links

View graph of relations

DreamFactory: Pioneering Multi-Scene Long Video Generation with a Multi-Agent Framework

Research output: Working paperPreprint

Published
  • Zhifei Xie
  • Daniel Tang
  • Dingwei Tan
  • Jacques Klein
  • Tegawend F. Bissyand
  • Saad Ezzini
Close
Publication date21/08/2024
PublisherArxiv
<mark>Original language</mark>English

Abstract

Current video generation models excel at creating short, realistic clips, but struggle with longer, multi-scene videos. We introduce \texttt{DreamFactory}, an LLM-based framework that tackles this challenge. \texttt{DreamFactory} leverages multi-agent collaboration principles and a Key Frames Iteration Design Method to ensure consistency and style across long videos. It utilizes Chain of Thought (COT) to address uncertainties inherent in large language models. \texttt{DreamFactory} generates long, stylistically coherent, and complex videos. Evaluating these long-form videos presents a challenge. We propose novel metrics such as Cross-Scene Face Distance Score and Cross-Scene Style Consistency Score. To further research in this area, we contribute the Multi-Scene Videos Dataset containing over 150 human-rated videos.

Bibliographic note

13 pages, 8 figures