We used several tests, all of which involved coding an audio extract, decoding it using the programs under test, and checking the results. To complicate matters :-) we used several encoders, as there may be a complex encoder/decoder compatibility issue, which we aim to discover.
First, we checked that each decoder would fully decode an mp3 file. Some did, whilst some clipped the last few samples from the file. These samples are usually silence (as each music track fades out at the end), but they may not be, especially on a live album, so it's worth choosing a decoder that decodes the whole file! The results of this test are here.
VBR stands for variable bit rate. In such files, the bit-rate is varied continuously, depending on the complexity of the music at that point. This can reduce the average bit rate required, but not all decoders support files of this type. Some decoders are fine until the bitrate drops below 128kbps. The results of this test are here.
Many decoders have a bug that causes audible problems when decoding a 100Hz tone. This bug may also cause audible problems with real music signals, but it is very obvious with a simple tone. This bug is strongly decoder AND encoder dependent. The results of this test are here.
Beyond these simple tests, a comparison of decoded waveforms shows that decoders can make more subtle mistakes. Some simply ignore all information above 16kHz (even though it is stored in the mp3 file), others reproduce high frequencies incorrectly, others produce low frequency problems in addition to the 100Hz bug, and still others alter the overall level of the whole signal, or just one stereo channel of it. The results of this test are here.
There are small (1-bit!) differences between the outputs of the good 16-bit decoders here. To see if any of these differences could be defined as correct or incorrect, we coded a signal that was only 1-bit in amplitude. Decoding this signal revealed how accurate the decoders were at the lowest level. The results of this test are here.
A few decoders claim to offer 24-bit accuracy. This may also offer better sound quality to users of 16-bit soundcards. We tested the accuracy of these 24-bit decoders, and compared them with the 16-bit competition. The results of this test are here.
ID3v2 tags are often added at the start of mp3 files to give information about the song's title and artist. Unfortunately, some decoders cannot handle this new feature, and refuse to decode tagged mp3s. The results of this test are here.
Occasionally mp3 files can become damaged. In this test we deliberately altered some of the data within an mp3, and attempted to decode it. Some decoders refuse to play corrupt files, though others can recover the audio signal intact. The results of this test are here.
A rarely used mp3 option is "free format." In such files, the bitrate can be any integer value between 8 and 640kbps. Very few decoders can handle such files. The results of this test are here.
The results of all these tests are summarised in one table. Pick a decoder that passes all the essential tests, and has the optional features that you require. The overall results are here.
Copyright 2000 David J M Robinson. All Rights reserved. You may not re-publish any information or content from this site without the authors' express permission.