How to use Unicode Dataset Generator
- List the categories (or "all").
- Add random mixed strings if you want more volume.
- Press Generate and compare the length columns with what your system stores.
Unicode Dataset Generator features
- 18 categories: Latin diacritics, Arabic, Devanagari, CJK, Cyrillic, Greek, Hebrew, Thai, emoji, combining marks, NFC/NFD pairs, surrogate pairs, bidi controls, zero-width, homoglyphs, mixed scripts, case mapping and fullwidth forms
- Per row: escaped form, code points, UTF-16 units, UTF-8 bytes, graphemes and NFC = NFD flag
- Extra random strings combining samples from the chosen categories
- Counts of multi-unit graphemes in the summary
- Seedable; JSON, CSV, SQL, XML, JSON Lines or table output
Unicode Dataset Generator example
Family emoji
Input:
Categories: emojiOutput:
text,codePoints,utf16Units,utf8Bytes,graphemes
👨👩👧👦,7,11,25,1Frequently asked questions about Unicode Dataset Generator
What does each row tell me?
The text, its escaped form, the number of code points, UTF-16 units, UTF-8 bytes and user-perceived graphemes, and whether NFC and NFD forms are identical — everything you need to spot length and normalisation bugs.
Which scripts and cases are covered?
Latin with diacritics, Arabic, Devanagari, CJK, Cyrillic, Greek, Hebrew, Thai, emoji and ZWJ sequences, combining marks, NFC/NFD pairs, surrogate pairs, bidi controls, zero-width characters, homoglyphs, mixed scripts, case-mapping edge cases and fullwidth forms.
Can I generate random mixed strings too?
Yes — set how many extra strings to build from random samples of the selected categories, and how many samples each should combine.