JSON: comments, order & numpy

Natively, JSON files do not support comments. Douglas Crockford removed them early on to prevent parsing directives, and the json standard is frozen now so they'll never return.

Douglas Crockford is, of course, a pretty smart guy, so I'm not going to argue that decision. But like many other people, I want comments anyway. There's a package commentjson, but unfortunately it does not support Python 3.

There are several other json tricks I like, so I made my own package *drum roll*: json-tricks (docs). It has 3 features:

  1. Reading commented json.
  2. Preserving dictionary key order.
  3. Storing and reading numpy arrays.

While you can use a list for ordered structures, sometimes you want an ordering, but still want to access the data by key (in O(1)). For example, a list of possible operations to apply, from which some might be preferred, but you'd like to check for supported ones by name quickly.

Numpy array encoding and decoding it also obvious: if your arrays aren't too large, it's much more convenient to store all data in one json file rather than store separate npy files, even if it's just for caching. I used list representation with some meta data (as opposed to b64 encoding or something) so it's human readable. Or programmer readable at least.

If you're thinking that'll take too much space, not to worry! Gzip compression is built in (set compression to something like 6).


Topics: #coding #python #imadethis


No comments yet

You need to be logged in to comment.