node-llama-cpp

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level

aibindingscataicmakecmake-jscuda

MIT License

Created on Aug 12, 2023

Updated on Apr 4, 2026

Stars

2.0k

Forks

180

Watchers

2.0k

Open Issues

22

Repository Health Score

🧡

69/100

Fair

Overall repository health assessment

Score Breakdown

Activity

Active development - updated this week

30/30

100%

Issues Analytics

49

Total Issues

All time

16

Open

33% of total

33

Closed

Recent Commits

feat(minor): customize `postinstall` behavior (#582)

Gilad S.•2 weeks ago

57bea3dView on GitHub

feat: automatic checkpoints for models that need it (#573)

Gilad S.•3 weeks ago

c641959View on GitHub

fix(`resolveModelFile`): deduplicate concurrent downloads (#570)

Gracie Gould•3 weeks ago

cc105b9View on GitHub

fix: correct Vulkan URL casing in documentation links (#568)

mcwindy•1 month ago

5a44506View on GitHub

fix: Electron template (#566)

Gilad S.•1 month ago

8931402View on GitHub

feat(`getLlama`): `build: "autoAttempt"` (#564)

Gilad S.•1 month ago

dda5adeView on GitHub

fix: macOS 14 prebuilt binaries (#559)

Gilad S.•1 month ago

6faa5aeView on GitHub

fix: export missing types (#557)

Gilad S.•1 month ago

498711cView on GitHub

feat: Exclude Top Choices (XTC) (#553)

Gilad S.•1 month ago

57e8c22View on GitHub

fix: adapt to `llama.cpp` changes (#547)

Gilad S.•2 months ago

4baa480View on GitHub

fix: bin test log level config (#546)

Gilad S.•2 months ago

1997b4eView on GitHub

fix: duplicate backend library files (#541)

Gilad S.•2 months ago

f5123bfView on GitHub

feat(`LlamaCompletion`): `stopOnAbortSignal` (#538)

Gilad S.•2 months ago

734693dView on GitHub

docs: fix cmake dependencies link (#534)

Gilad S.•3 months ago

7e467ccView on GitHub

test: exclude irrelevant tests (#533)

Gilad S.•3 months ago

f78a78eView on GitHub

View all commits

embedding

function-calling

gguf

gpu

grammar

json-schema

llama

llama-cpp

llm

metal

nodejs

prebuilt-binaries

self-hosted

vulkan

Community

1,982 stars, 180 forks

4/30

13%

Documentation

Has description, license

15/20

75%

Maintenance

1.1% issue ratio

20/20

100%

Health score is calculated based on activity, community engagement, documentation quality, and maintenance practices

Languages

TypeScript

91.3%

C++

4.5%

CSS

1.7%

Vue

0.8%

JavaScript

0.7%

CMake

0.7%

Shell

0.2%

HTML

0.0%

C

0.0%

Dependencies90 total

28

Production

48

Development

Production Dependencies (28)

@huggingface/jinja^0.5.6

async-retry^1.3.3

bytes^3.1.2

chalk^5.6.2

Top Contributors

1

giladgd

User

229

commits

2

ido-pluto

User

7

commits

3

scenaristeur

User

1

commits

4

graciegould

User

1

commits

5

yhg8423

User

1

commits

6

JonHolman

User

1

commits

7

jhstatewide

User

1

commits

8

vamshi9666

User

1

commits

9

aratahikaru0

User

1

commits

10

mcwindy

User

1

commits

Languages

TypeScript

91.3%

C++

4.5%

CSS

1.7%

Vue

0.8%

JavaScript

0.7%

CMake

0.7%

Shell

0.2%

HTML

0.0%

C

0.0%

Issues Analytics

49

Total Issues

All time

16

Open

33% of total

33

Closed

Dependencies90 total

28

Production

48

Development

Production Dependencies (28)

@huggingface/jinja^0.5.6

async-retry^1.3.3

bytes^3.1.2

chalk^5.6.2

chmodrp

^1.0.2

cmake-js^8.0.0

cross-spawn^7.0.6

env-var^7.5.0

filenamify^6.0.0

fs-extra^11.3.4

ignore^7.0.4

ipull^3.9.5

is-unicode-supported^2.1.0

lifecycle-utils^3.1.1

log-symbols^7.0.1

nanoid^5.1.6

node-addon-api^8.6.0

ora^9.3.0

pretty-ms^9.3.0

proper-lockfile^4.1.2

semver^7.7.1

simple-git^3.33.0

slice-ansi^8.0.0

stdout-update^4.0.1

strip-ansi^7.2.0

validate-npm-package-name^7.0.2

which^6.0.1

yargs^17.7.2

Development Dependencies (48)

@commitlint/cli^20.4.1

@commitlint/config-conventional^20.4.1

@eslint/compat^2.0.2

@fontsource/inter^5.2.8

@nolebase/vitepress-plugin-git-changelog^2.18.2

@nolebase/vitepress-plugin-og-image^2.18.2

@resvg/resvg-js^2.6.2

@semantic-release/exec^7.1.0

@semantic-release/github^12.0.6

@semantic-release/npm^13.1.5

@shikijs/vitepress-twoslash^3.22.0

@stylistic/eslint-plugin^5.8.0

@types/async-retry^1.4.9

@types/bytes^3.1.5

@types/cross-spawn^6.0.6

@types/fs-extra^11.0.4

@types/node^20.19.33

@types/proper-lockfile^4.1.4

@types/semver^7.7.1

@types/validate-npm-package-name^4.0.2

@types/which^3.0.4

@types/yargs^17.0.33

@vitest/coverage-v8^4.1.0

@vitest/ui^4.1.0

electron^40.4.1

eslint^9.39.2

eslint-import-resolver-typescript^4.4.4

eslint-plugin-import^2.32.0

eslint-plugin-jsdoc^62.5.5

eslint-plugin-n^17.24.0

feed^5.2.0

husky^9.1.7

rehype^13.0.2

rimraf^6.1.3

semantic-release^25.0.3

sharp^0.34.5

tslib^2.8.1

typedoc^0.28.17

typedoc-plugin-markdown^4.10.0

typedoc-plugin-mdn-links^5.1.1

typedoc-vitepress-theme^1.1.2

typescript^5.9.3

typescript-eslint^8.56.0

vite-node^6.0.0

vitepress^1.6.4

vitepress-plugin-llms^1.11.0

vitest^4.1.0

zx^8.8.5

Peer Dependencies (1)

typescript>=5.0.0

node-llama-cppv0.1.0

67% of total

6d

Avg Close Time

Fast response ✅

Issues Activity: Last 6 months

Top Labels

Hottest Issues

1

#556 bug: Windows on ARM not running at full possible speed

bugrequires triage

18

open

2

#549 bug: Failed to create context on M5

bugrequires triage

16

closed

3

#560 bug: Jetson Orin compile + install

bugrequires triage

13

open

4

#554 Segmentation fault on CPU fallback when Vulkan prebuilt binary is incompatible (Linux x64, no GPU)

10

open

5

#577 bug: version >=v3.15.0 not support CUDA v13.1 & v13.2

bugreleased

9

closed

Top Contributors

1

giladgd

User

229

commits

2

ido-pluto

User

7

commits

3

scenaristeur

User

1

commits

4

graciegould

User

1

commits

5

yhg8423

User

1

commits

6

JonHolman

User

1

commits

7

jhstatewide

User

1

commits

8

vamshi9666

User

1

commits

9

aratahikaru0

User

1

commits

10

mcwindy

User

1

commits

Recent Commits

feat(minor): customize `postinstall` behavior (#582)

Gilad S.•2 weeks ago

57bea3dView on GitHub

feat: automatic checkpoints for models that need it (#573)

Gilad S.•3 weeks ago

c641959View on GitHub

fix(`resolveModelFile`): deduplicate concurrent downloads (#570)

Gracie Gould•3 weeks ago

cc105b9View on GitHub

fix: correct Vulkan URL casing in documentation links (#568)

mcwindy•1 month ago

5a44506View on GitHub

fix: Electron template (#566)

Gilad S.•1 month ago

8931402View on GitHub

feat(`getLlama`): `build: "autoAttempt"` (#564)

Gilad S.•1 month ago

dda5adeView on GitHub

fix: macOS 14 prebuilt binaries (#559)

Gilad S.•1 month ago

6faa5aeView on GitHub

fix: export missing types (#557)

Gilad S.•1 month ago

498711cView on GitHub

feat: Exclude Top Choices (XTC) (#553)

Gilad S.•1 month ago

57e8c22View on GitHub

fix: adapt to `llama.cpp` changes (#547)

Gilad S.•2 months ago

4baa480View on GitHub

fix: bin test log level config (#546)

Gilad S.•2 months ago

1997b4eView on GitHub

fix: duplicate backend library files (#541)

Gilad S.•2 months ago

f5123bfView on GitHub

feat(`LlamaCompletion`): `stopOnAbortSignal` (#538)

Gilad S.•2 months ago

734693dView on GitHub

docs: fix cmake dependencies link (#534)

Gilad S.•3 months ago

7e467ccView on GitHub

test: exclude irrelevant tests (#533)

Gilad S.•3 months ago

f78a78eView on GitHub

View all commits

GitHub Explorer

node-llama-cpp

Score Breakdown

Production Dependencies (28)

Production Dependencies (28)

Development Dependencies (48)

Peer Dependencies (1)

Issues Activity: Last 6 months

Top Labels

Hottest Issues