Results for the SE task

Here you can find the results submitted to the REVERB challenge for the SE task.
For the results for the ASR task, please see Result(ASR) tab.

Note that the figures concerning the objective evaluation do not display properly with some browsers, e.g. Internet Explorer. Please use Firefox, Google Chrome or any other HTML5 capable browser.

• Results for the subjective evaluation
• Results of the web-based MUSHRA test
• Results for the objective evaluation
• Cepstrum distance (CD)
• Log likelihood ratio (LLR)
• Frequency-weighted segmental SNR (FWSegSNR)
• Speech-to-reverberation modulation energy ratio (SRMR)
• Perceptual Evaluation of Speech Quality (PESQ)

Results of the web-based MUSHRA test

To evaluate subjective quality of the SE participants' systems, we carried out a web-based MUSHRA test. The subjective quality was evaluated in terms of the following 2 metrics:

• Perceived amount of reverberation (Used adjectives: Very large/Large/Mid./Small/Very small)
• Overall quality (Used adjectives: Bad/Poor/Fair/Good/Excellent)

Please refer to the online document for more details of the test. The test reverberation conditions include the following 4 conditions: SimData Room2 near, SimData Room2 far, RealData near, RealData far. Finally, please note that the tests were carried out separately for the 1ch, 2ch and 8ch systems, and separately for each reverberation condition. Therefore, it is not recommended to directly compare the results/numbers across 1ch, 2ch and 8ch systems, and across reverberation conditions. In the following results, all the mean scores were plotted along with the associated 95% confidence interval as recommended in the ITU document.

MUSHRA score for the 1ch systems

MUSHRA score for the 2ch systems

MUSHRA score for the 8ch systems

Cepstrum distance (CD) The lower, the better.

Please use the following check boxes to select the systems to be displayed.

Log likelihood ratio (LLR) The lower, the better.

Please use the following check boxes to select the systems to be displayed.

Frequency-weighted segmental SNR (FWSegSNR) The higher, the better.

Please use the following check boxes to select the systems to be displayed.

Speech-to-reverberation modulation energy ratio (SRMR) The higher, the better.

Please use the following check boxes to select the systems to be displayed.

Percepsutal Evaluation of Speech Quality (PESQ) The higher, the better.

Please use the following check boxes to select the systems to be displayed.