I was experimenting with Avro C, a library from the Apache project used to encode/decode information whose Java implementation that is commonly used to enode data being stored in Kafka, a log based data distribution system. Avro C is fairly difficult to use. In order to get it to encode my data (a simple log file), I had to climb into the code and enhance the diagnostics. I’ve just published the enhanced library on GitHub: https://github.com/jimbelton/avro-c-packaging
Here’s a summary of the changes:
- In datafile.c, all errors from avro_[*_]write functions were being treated as buffer overflows, and the low level error messages (the real problems) overwritten. I changed the buffer overflow messages to prefix the lower level messages.
- In datum.c, added detailed errors on failure of avro_record_get, including the set of valid field names.
- In datum_validate.c, added detailed errors on failure of avro_schema_datum_validate.
- In datum_write.c, changed avro_write_data to take advantage of the change to datum_valididate.c.
- In st.c, added a new function st_keys_as_string that extracts a comma separated list of the keys in an st_table.
Feel free to use this version if you like.