4 Commits

Include renames

Author	SHA1	Message	Date
gazhonsepaskwa	c36c1550d7	tokenizer_v3	2025-01-17 17:04:38 +01:00
Nathan Lebrun	9a7c1d0dd4	tokenizer v2	2025-01-17 08:29:12 +01:00
Nathan Lebrun	687dd72dba	tokenisation	2025-01-15 16:29:53 +01:00
Nathan Lebrun	8902b8f141	added the folder and makefile structure for the tests and made some tests	2025-01-14 19:04:06 +01:00