Original video
Original Chinese speech · 18.5 seconds.
DuraS2ST · English translation
AI-generated English speech with the original video.
Chain-of-Thought and Reinforcement Learning for Duration-Aligned Speech-to-Speech Translation
Official repository of the EMNLP 2026 Main Conference paper.
DuraS2ST explicitly plans target wording and phonetic length before generating translated speech, balancing translation fidelity with duration consistency.
[2026-08-21] 🎉 DuraS2ST is accepted to the EMNLP 2026 Main Conference (15.4% acceptance rate for Main Conference papers).
Compare the original clips with speech translated by DuraS2ST. The video and its timeline are preserved.
Original video
Original Chinese speech · 18.5 seconds.
DuraS2ST · English translation
AI-generated English speech with the original video.
Original video
Original English speech · 16.7 seconds.
DuraS2ST · Chinese translation
AI-generated Chinese speech with the original video.
Original video
Original Chinese speech · 14.6 seconds.
DuraS2ST · English translation
AI-generated English speech with the original video.
Original video
Original English speech · 29.4 seconds.
DuraS2ST · Chinese translation
AI-generated Chinese speech with the original video.
Original video
Chinese dialect source clip from the Wikitongues archive.
DuraS2ST · English translation
English speech mixed with a lowered background track.
Original video
Chinese dialect source clip from the Wikitongues archive.
DuraS2ST · English translation
English speech mixed with a lowered background track.
Each example shows the source, the model's duration-planning chain of thought, and the corresponding translated speech.
Example 1
Source
both works mentioned above are full of romanticism
Duration-planning CoT
DuraS2ST output
上述两部作品都充满浪漫主义色彩。
Example 2
Source
he is also a professional butcher
Duration-planning CoT
DuraS2ST output
他也是一名专业屠夫。
Example 3
Source
bill wyatt australian basketball player
Duration-planning CoT
DuraS2ST output
比尔·怀亚特,澳大利亚篮球运动员。
Example 4
Source
he is now head coach of rostov
Duration-planning CoT
DuraS2ST output
他现在是罗斯托夫的主教练。
Example 5
Source
computing in parallel
Duration-planning CoT
DuraS2ST output
并行计算。
The reasoning, translated text, and generated waveform in each example come from the same DuraS2ST prediction.
Example 1
Source
样条理论已成为函数逼近的有力工具。
Duration-planning CoT
DuraS2ST output
spline theory has become a powerful tool for function approximation
Example 2
Source
这个拱门作为白金汉宫的北门。
Duration-planning CoT
DuraS2ST output
this arch is the north gate of buckingham palace
Example 3
Source
斯里兰卡的总统是国家元首。
Duration-planning CoT
DuraS2ST output
the president of sri lanka is the head of state
Example 4
Source
没看过这么弱的天才
Duration-planning CoT
DuraS2ST output
never seen such a weak genius
Example 5
Source
未出嫁的女子戴圆形的耳环。
Duration-planning CoT
DuraS2ST output
unmarried women wear round earrings