Proposing a new function like fl_utf8len* with buffer bounds check #1590

Since I've noticed the frequent problem that fl_utf8len*() calculates the expected length but cannot check whether the current buffer can hold that number of bytes, I suggest implementing another function that can do this, something like:

int fl_utf8len_bounded(const char* p, const char* end) {
  if (p >= end) return 0;          // end of buffer
  const int len = fl_utf8len(*p);  // expected length
  if (len < 1) return 1;           // invalid byte, can be skipped
  if (p + len <= end) return len;  // OK, fits in the buffer
  return 1;                        // OK, but would exceed the buffer
}

Function name yet to be defined, and also return values, but I think it's good as it stands:

• 0: end of buffer
• 1: invalid byte or ASCII
• >1: valid UTF-8, fits in the buffer

This is just an idea, comments appreciated. We'd have to check whether and where such a function would be useful.

I proposed this function here, and @ManoloFLTK proposed the function name and a small modification in a comment which is also included in the code above. Thanks for that suggestion.

@MatthiasWM replied here:

I do not like it. The length of the utf8 sequence doesn't change, just because the string may or may not end before the character ends. What would you return? Another error code?
fl_utf8decode and fl_utf8fwd should be enough to iterate over a UTF-8 string. We could verify that those functions don't access memory illegally if the string is malformed, and that fl_utf8len is only used where it is safe to use.

I replied:

My proposed function would return 0 for end of buffer and > 0 like fl_utf8len1() for UTF-8 sequences that fit in the given buffer. The caller would have to check for 0 explicitly.

@erco77 proposed in another comment yet another function that also checks for valid continuation bytes. For more information please see his comment.

All mentioned functions don't check for Unicode combination characters like accents and extra code points used for emojis etc. but this should be done by the caller.