In working through the kinks of the encoding proposal in #66; a nice issue that came up was terminating a repeating list of values that start with a length (like the field names in an object). It became clear that "Not a Length" or a "Null Length" value was needed in addition to the values used for indicating the size of the value that followed after the first byte if it was needed.
@lightmare was instrumental in discussing that part; the rest of us had earlier hashed through that regardless of the decision, any change made to how length is encoded would be a breaking change.
I really liked the proposal overall and it could be part of creating a single unified Integer encoding for the UBJ types (in place of the multiple type codes used today). So I coded up a working reference in C# and posted it as a gist: https://gist.github.com/MikeFair/a3c6bd817ff716ce090f
To briefly redescribe the format, values less than 240, null, and the value 255 are encoded in a single byte. All other values are encoded using the first byte as a type descriptor and the fewest number of bytes that will represent their value; the type descriptor is also used to describe whether the encoding is in big or little endian format.
The code in the gist provides a UBJUInt class for sending/receiving Unsigned Integer values as a new encoding for UBJ. This is in addition to using the encoding description as a < LENGTH > whenever a < LENGTH > is required by the spec. The code should fairly easily translate into other languages.
Some highlights:
- a nullable ulong "Value" property for the caller to read and update
- "ReadFromBinaryStream" and "ReadFromByteArray" as a constructor or updater to the value
- "GetBytes", "FillBytes", and "WriteToBinaryWriter" routines
- construct or update directly from a provided ulong value
- Supports endian types: Big, Little, MachineLocal, and MatchRemote
Setting the endian type to Big, Little, or MachineLocal will always encode the bytes as the described endian type. MatchRemote will use MachineLocal until it has been updated from a message; after which it will encode the value as Big or Little depending on the encoding read in. The intention is that the parser/encoder would only need to create one instance of the class and then reuse that instance via the routines provided to have it encode/decode the values as needed.
Please take a look and tell me what you think.
With some additional work, this class could be extended to handle Big and Little Endian variants of the "String of Bytes" type by receiving/providing a byte[] directly from/to the caller... A similar class could also be written for arbitrary decimal, float, and double values. All in all, this could become a single UBJ type code character to handle all of the Unsigned Integer Value types.
What prevented encoding Signed values into the class as well were the Signed Single Bytes; there is simply no way using only a single byte to encode both signed and unsigned variants of the same value.
I was thinking UBJ could use the preceding type code to mean Unsigned and Signed values:
- "=" would mean Unsigned Values
- "+" would mean Signed Values
The type code would determine how the bits are interpreted.
Instead of trying to coerce signedness for a single byte into the format; using the UBJ type code would provide the foreknowledge required to set the class instance to read or encode the data as either signed or unsigned. Nothing about the encoded bits on the wire would need to change; the class would provide the features and settings to allow sending/retrieving the value as either a Signed or Unsigned value.
Thanks
In working through the kinks of the encoding proposal in #66; a nice issue that came up was terminating a repeating list of values that start with a length (like the field names in an object). It became clear that "Not a Length" or a "Null Length" value was needed in addition to the values used for indicating the size of the value that followed after the first byte if it was needed.
@lightmare was instrumental in discussing that part; the rest of us had earlier hashed through that regardless of the decision, any change made to how length is encoded would be a breaking change.
I really liked the proposal overall and it could be part of creating a single unified Integer encoding for the UBJ types (in place of the multiple type codes used today). So I coded up a working reference in C# and posted it as a gist: https://gist.github.com/MikeFair/a3c6bd817ff716ce090f
To briefly redescribe the format, values less than 240, null, and the value 255 are encoded in a single byte. All other values are encoded using the first byte as a type descriptor and the fewest number of bytes that will represent their value; the type descriptor is also used to describe whether the encoding is in big or little endian format.
The code in the gist provides a UBJUInt class for sending/receiving Unsigned Integer values as a new encoding for UBJ. This is in addition to using the encoding description as a < LENGTH > whenever a < LENGTH > is required by the spec. The code should fairly easily translate into other languages.
Some highlights:
Setting the endian type to Big, Little, or MachineLocal will always encode the bytes as the described endian type. MatchRemote will use MachineLocal until it has been updated from a message; after which it will encode the value as Big or Little depending on the encoding read in. The intention is that the parser/encoder would only need to create one instance of the class and then reuse that instance via the routines provided to have it encode/decode the values as needed.
Please take a look and tell me what you think.
With some additional work, this class could be extended to handle Big and Little Endian variants of the "String of Bytes" type by receiving/providing a byte[] directly from/to the caller... A similar class could also be written for arbitrary decimal, float, and double values. All in all, this could become a single UBJ type code character to handle all of the Unsigned Integer Value types.
What prevented encoding Signed values into the class as well were the Signed Single Bytes; there is simply no way using only a single byte to encode both signed and unsigned variants of the same value.
I was thinking UBJ could use the preceding type code to mean Unsigned and Signed values:
The type code would determine how the bits are interpreted.
Instead of trying to coerce signedness for a single byte into the format; using the UBJ type code would provide the foreknowledge required to set the class instance to read or encode the data as either signed or unsigned. Nothing about the encoded bits on the wire would need to change; the class would provide the features and settings to allow sending/retrieving the value as either a Signed or Unsigned value.
Thanks