Incoherent return values from the RS decoder

[fblomqvi]

The man page for the Reed-Solomon encoding and decoding functions states:

The decode_ functions return a count of corrected symbols,or -1 if the block was uncorrectible.

This doesn't specify how how "erased" symbols are accounted for in the returned count. There are two logical options:

1. The number of erasures is always included in the count of corrected symbols.
2. Only erasure positions where there actually was a symbol error are included in the count.

We run the following code to check what happens:

#include <stdio.h>
#include <stdlib.h>
#include "fec.h"

#define NN 7

int main()
{
  void* rs = init_rs_char(3, 0xb, 1, 1, 3, 0);
  if(rs == NULL){
    printf("init_rs_char failed!\n");
    return -1;
  }

  unsigned char data[NN] = {1,2,3,4};
  int erasures[NN] = {2};

  encode_rs_char(rs, data, data + 4);
  int count = decode_rs_char(rs, data, erasures, 1);
  printf("Decoder fixed %d errors\n", count);
  printf("Error locations:");
  for(int i = 0; i < count; i++)
    printf(" %d", erasures[i]);
  printf("\n");

  data[0] = 2;
  count = decode_rs_char(rs, data, erasures, 1);
  printf("Decoder fixed %d errors\n", count);
  printf("Error locations:");
  for(int i = 0; i < count; i++)
    printf(" %d", erasures[i]);
  printf("\n");

  free_rs_char(rs);
  return 0;
}

The output is:

Decoder fixed 0 errors
Error locations:
Decoder fixed 2 errors
Error locations: 0 2

We see that erased symbols are not included in the count if the word to be decoded is a codeword (option 2). On the other hand, if the word to be decoded contains errors, then the erased symbols are included in the count even if the erased symbol was correct (option 1). Thus the decoder does not comply with either of the logical options.

The same issue applies to the returned list of corrected symbol positions, i.e. the one returned in eras_pos.