Add model weights to Core NuGet package and publish to GitHub Packages

fix: cap perplexity context to MaxSequenceLength

Remove time limits on Azure DevOps pipelines (set job timeoutInMinutes: 0)

Use self-hosted pool for main pipeline

Use local agent for benchmark pipeline

Fix benchmark pipeline runtime and report formatting

Fix benchmark pipeline deploy job

Add Azure DevOps pipeline definitions

Added tokenized data.

Merge pull request #26 from sharpninja/copilot/replace-stub-training-with-full-epochs

Merge branch 'main' into copilot/replace-stub-training-with-full-epochs

Complete PLAN-ROADMAP-001

Merge remote-tracking branch 'origin/main' into copilot/replace-stub-training-with-full-epochs

Merge pull request #27 from sharpninja/copilot/add-wikitext-2-validation-set

Polish corpus script usage docs