Merge pull request #21 from ando-masaki/gpt-5-fix

Merge pull request #2 from ando-masaki/ando-masaki-patch-1

Update tokenizer.go

Merge pull request #18 from ando-masaki/add-o3-and-o4-mini-fix

Update tokenizer.go

Revert "small performance improvement"

small performance improvement

Remove defer, also small improvement using strings for bpe

🚀 feat: added support for O3-mini model

🐛 problem with regexp generated regexp code dependencies

Merge pull request #13 from amalucelli/count

Refactored code to remove duplication

feat: add count method for tokens

Merge pull request #11 from terwey/regexp2cg

Added a go generate command for the compiled regular expression