nutrition: pre-download HuggingFace models at build time
All checks were successful
CI / update (push) Successful in 4m26s
All checks were successful
CI / update (push) Successful in 4m26s
The deployment server couldn't fetch transformer models at runtime due to restricted network access and permission errors writing to node_modules. Add a prebuild script to download models during build and document TRANSFORMERS_CACHE env var for configuring a shared writable cache path.
This commit is contained in:
17
scripts/download-models.ts
Normal file
17
scripts/download-models.ts
Normal file
@@ -0,0 +1,17 @@
|
||||
/**
|
||||
* Pre-downloads HuggingFace transformer models so they're cached for runtime.
|
||||
* Run with: pnpm exec vite-node scripts/download-models.ts
|
||||
*/
|
||||
import { pipeline } from '@huggingface/transformers';
|
||||
|
||||
const MODELS = [
|
||||
'Xenova/all-MiniLM-L6-v2',
|
||||
'Xenova/multilingual-e5-small',
|
||||
];
|
||||
|
||||
for (const name of MODELS) {
|
||||
console.log(`Downloading ${name}...`);
|
||||
const p = await pipeline('feature-extraction', name, { dtype: 'q8' });
|
||||
await p.dispose();
|
||||
console.log(` done`);
|
||||
}
|
||||
Reference in New Issue
Block a user