Fixed infinite loop bug in tokenization

Modifying how model tokenization is handled

bugfix

remove deprecation

Attempt to fix infinite loop

updating with arxiv and proper refactoring

adding requirements for proper install

uploading experimental continuous model honing code

updating and bugfixing ai generation

modifying token length

added error handling

update the ai generation bit

adding error workaround in build dataset

Adding better model resillience

Add multi-model support