To encode to JSON without extraneous whitespace, you set the OSNOWHITESPACE flag in the context like so:

rtxCtxtSetFlag (pctxt, OSNOWHITESPACE);

For C++, you can obtain the context from the encode buffer:

OSCTXT* pctxt = encodeBuffer.getCtxtPtr();
rtxCtxtSetFlag (pctxt, OSNOWHITESPACE);