Back to search
Perseids Tokenization Service
Stars
0
Forks
0
Watchers
0
Open Issues
4
Overall repository health assessment
No package.json found
This might not be a Node.js project
1
commits
handling 02bc modifier apostrophe as breaktext in french and also removing indent because it caused artificial spacing in the resulting xml
ebea057View on GitHubadding support for tags parameter to specify custom token tags
46f1ed9View on GitHubremoving insertion of line numbers and breaking tokenization up into groups of new line separated text to avoid eXist stack overflow
af598c8View on GitHubinitial version of xquery code for tokenization service - derived from Alpheios markwords.xquery
abfaa85View on GitHub