fix: improve number parsing, add number_nd_tokens

add dictionary overrides
2023-05-06 17:27:51 +02:00 · 2023-05-06 17:27:51 +02:00 · 19781eab36
commit 19781eab36
parent 97492780c6
13 changed files with 33097 additions and 35712 deletions
--- a/notes/dictionary.md
+++ b/notes/dictionary.md
@ -30,3 +30,5 @@ build a dictionary.
 - Examples: "1.4M views"
 - There is an exception for the value 0 ("no views") and in some languages for the value
  1 (pt: "Um vídeo")
+- Special case: Language "gu", "જોવાયાની સંખ્યા" = "no views", contains no unique tokens
+  to parse