Avro C Diagnostics

avro-rotaI was experimenting with Avro C, a library from the Apache project used to encode/decode information whose Java implementation that is commonly used to enode data being stored in Kafka, a log based data distribution system. Avro C is fairly difficult to use. In order to get it to encode my data (a simple log file), I had to climb into the code and enhance the diagnostics. I’ve just published the enhanced library on GitHub: https://github.com/jimbelton/avro-c-packaging

Here’s a summary of the changes:

  • In datafile.c, all errors from avro_[*_]write functions were being treated as buffer overflows, and the low level error messages (the real problems) overwritten. I changed the buffer overflow messages to prefix the lower level messages.
  • In datum.c, added detailed errors on failure of avro_record_get, including the set of valid field names.
  • In datum_validate.c, added detailed errors on failure of avro_schema_datum_validate.
  • In datum_write.c, changed avro_write_data to take advantage of the change to datum_valididate.c.
  • In st.c, added a new function st_keys_as_string that extracts a comma separated list of the keys in an st_table.

Feel free to use this version if you like.

Advertisements

About jimbelton

I'm a software developer, and a writer of both fiction and non-fiction, and I blog about movies, books, and philosophy. My interest in religious philosophy and the search for the truth inspires much of my writing.
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s