MULTIMODAL GENERATIVE AI
Distributed video diffusion training at Moonvalley
Large-scale distributed model training, alongside research on visual quality and temporal consistency.
The challenge
Train large generative models across distributed compute while maintaining visual quality, temporal consistency, and data provenance.
My contribution
Built parallel training code with model sharding and tensor and data parallelism. Also led R&D on rendered-text fidelity and motion-aware placement, and developed preference-optimization workflows for 70B-parameter LLM/VLM assistants.
The outcome
Trained large models using distributed infrastructure and developed synthetic-data pipelines spanning hundreds of millions of licensed video–text pairs.
Areas of work
- Distributed training
- Tensor & data parallelism