DeepRapper: Neural Rap Era with Rhyme and Rhythm Modeling

Because of the advanced rhyme construction of rap, normal rhyming fashions are usually not appropriate for rap era. Because of the lack of datasets with rap beat-lyric alignment, no rhythmic modeling technique for rap has been created earlier than.

A current examine on arXiv.org proposes a transformer-based rap era mannequin for each rhymes and rhythms.

Picture credit score: Pxhere, free licence

Firstly, an information mining pipeline is developed to create rap datasets with aligned rhythmic beats. With a purpose to generate rap lyrics with rhyme constraint, an autoregressive language mannequin is created. Beat info is modeled by inserting a beat token apart from the corresponding phrase.

The mannequin is pre-trained utilizing non-rap songs with aligned beats and pure lyrics. Then, it’s fine-tuned on the rap songs with aligned beats. Goal and subjective evaluations affirm that the mannequin generates high-quality raps with good rhymes and rhythms.

Rap era, which goals to provide lyrics and corresponding singing beats, must mannequin each rhymes and rhythms. Earlier works for rap era targeted on rhyming lyrics however ignored rhythmic beats, that are essential for rap efficiency. On this paper, we develop DeepRapper, a Transformer-based rap era system that may mannequin each rhymes and rhythms. Since there is no such thing as a accessible rap dataset with rhythmic beats, we develop an information mining pipeline to gather a large-scale rap dataset, which incorporates numerous rap songs with aligned lyrics and rhythmic beats. Second, we design a Transformer-based autoregressive language mannequin which rigorously fashions rhymes and rhythms. Particularly, we generate lyrics within the reverse order with rhyme illustration and constraint for rhyme enhancement and insert a beat image into lyrics for rhythm/beat modeling. To our data, DeepRapper is the primary system to generate rap with each rhymes and rhythms. Each goal and subjective evaluations display that DeepRapper generates inventive and high-quality raps with rhymes and rhythms. Code will probably be launched on GitHub.

Analysis paper: Xue, L., “DeepRapper: Neural Rap Era with Rhyme and Rhythm Modeling”, 2021. Hyperlink: https://arxiv.org/abs/2107.01875






Source link