implement content-hash based image cache invalidation
Add content-based hashing to recipe images for proper cache invalidation while maintaining graceful degradation through dual file storage. Changes: - Add imageHash utility with SHA-256 content hashing (8-char) - Update Recipe model to store hashed filenames in images[0].mediapath - Modify image upload endpoint to save both hashed and unhashed versions - Update frontend components to use images[0].mediapath with fallback - Add migration endpoint to hash existing images (production-only) - Update image delete/rename endpoints to handle both file versions Images are now stored as: - recipe.a1b2c3d4.webp (hashed, cached forever) - recipe.webp (unhashed, graceful degradation fallback) Database stores hashed filename for cache busting, while unhashed version remains on disk for backward compatibility and manual uploads.
This commit is contained in:
255
MIGRATION_IMAGE_HASHES.md
Normal file
255
MIGRATION_IMAGE_HASHES.md
Normal file
@@ -0,0 +1,255 @@
|
||||
# Image Hash Migration Guide
|
||||
|
||||
This guide explains how to migrate existing images to use content-based hashing for cache invalidation.
|
||||
|
||||
## Overview
|
||||
|
||||
The new system stores images with content-based hashes for proper cache invalidation:
|
||||
- **Database**: `images[0].mediapath = "maccaroni.a1b2c3d4.webp"`
|
||||
- **Disk (hashed)**: `maccaroni.a1b2c3d4.webp` - cached forever (immutable)
|
||||
- **Disk (unhashed)**: `maccaroni.webp` - fallback for graceful degradation
|
||||
|
||||
## What This Does
|
||||
|
||||
The migration will:
|
||||
|
||||
1. **Find all recipes** in the database
|
||||
2. **Check each recipe's images**:
|
||||
- If `images[0].mediapath` already has a hash → skip
|
||||
- If image file doesn't exist on disk → skip
|
||||
- Otherwise → generate hash and create hashed copy
|
||||
3. **Generate content hash** from the full-size image (8-char SHA-256)
|
||||
4. **Copy files** (keeps originals!) in all three directories:
|
||||
- `/var/lib/www/static/rezepte/full/`
|
||||
- `/var/lib/www/static/rezepte/thumb/`
|
||||
- `/var/lib/www/static/rezepte/placeholder/`
|
||||
5. **Update database** with new hashed filename in `images[0].mediapath`
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Must be logged in as admin
|
||||
- Only runs in production (when `IMAGE_DIR=/var/lib/www/static`)
|
||||
- Requires confirmation token to prevent accidental runs
|
||||
- Backup your database before running (recommended)
|
||||
|
||||
## Step 1: Deploy Code Changes
|
||||
|
||||
Deploy the updated codebase to production. The changes include:
|
||||
- Image upload endpoint now saves both hashed and unhashed versions
|
||||
- Frontend components use `images[0].mediapath` for image URLs
|
||||
- New migration endpoint at `/api/admin/migrate-image-hashes`
|
||||
|
||||
## Step 2: Update Nginx Configuration
|
||||
|
||||
Add this to your nginx site configuration for `bocken.org`:
|
||||
|
||||
```nginx
|
||||
location /static/rezepte/ {
|
||||
root /var/lib/www;
|
||||
|
||||
# Cache hashed files forever (they have content hash in filename)
|
||||
location ~ /static/rezepte/(thumb|placeholder|full)/[^/]+\.[a-f0-9]{8}\.webp$ {
|
||||
add_header Cache-Control "public, max-age=31536000, immutable";
|
||||
}
|
||||
|
||||
# Cache unhashed files with revalidation (fallback for manual uploads)
|
||||
location ~ /static/rezepte/(thumb|placeholder|full)/[^/]+\.webp$ {
|
||||
add_header Cache-Control "public, max-age=3600, must-revalidate";
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Reload nginx:
|
||||
```bash
|
||||
sudo nginx -t && sudo nginx -s reload
|
||||
```
|
||||
|
||||
## Step 3: Run Migration
|
||||
|
||||
### Option 1: Using curl (Recommended)
|
||||
|
||||
```bash
|
||||
# Get your session cookie from browser DevTools
|
||||
# In Chrome/Firefox: F12 → Network tab → Click any request → Headers → Copy Cookie value
|
||||
|
||||
curl -X POST https://bocken.org/api/admin/migrate-image-hashes \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Cookie: YOUR_SESSION_COOKIE_HERE" \
|
||||
-d '{"confirm": "MIGRATE_IMAGES"}'
|
||||
```
|
||||
|
||||
### Option 2: Using the Shell Script
|
||||
|
||||
```bash
|
||||
cd /path/to/homepage
|
||||
|
||||
# Save your session cookie to a file (from browser DevTools)
|
||||
echo "your-session-cookie-value" > .prod-session-cookie
|
||||
|
||||
# Make script executable
|
||||
chmod +x scripts/migrate-image-hashes.sh
|
||||
|
||||
# Run migration
|
||||
./scripts/migrate-image-hashes.sh
|
||||
```
|
||||
|
||||
### Option 3: Using Browser (Postman, Insomnia, etc.)
|
||||
|
||||
1. Make sure you're logged in to bocken.org in your browser
|
||||
2. Send POST request to: `https://bocken.org/api/admin/migrate-image-hashes`
|
||||
3. Headers:
|
||||
```
|
||||
Content-Type: application/json
|
||||
```
|
||||
4. Body (JSON):
|
||||
```json
|
||||
{
|
||||
"confirm": "MIGRATE_IMAGES"
|
||||
}
|
||||
```
|
||||
|
||||
## Response Format
|
||||
|
||||
Success response:
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"message": "Migration complete. Migrated 42 of 100 recipes.",
|
||||
"total": 100,
|
||||
"migrated": 42,
|
||||
"skipped": 58,
|
||||
"errors": [],
|
||||
"details": [
|
||||
{
|
||||
"shortName": "maccaroni",
|
||||
"status": "migrated",
|
||||
"unhashedFilename": "maccaroni.webp",
|
||||
"hashedFilename": "maccaroni.a1b2c3d4.webp",
|
||||
"hash": "a1b2c3d4",
|
||||
"filesCopied": 3,
|
||||
"note": "Both hashed and unhashed versions saved for graceful degradation"
|
||||
},
|
||||
{
|
||||
"shortName": "pizza",
|
||||
"status": "skipped",
|
||||
"reason": "already hashed",
|
||||
"filename": "pizza.e5f6g7h8.webp"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## What Gets Skipped
|
||||
|
||||
The migration will skip recipes where:
|
||||
- `images[0].mediapath` already contains a hash pattern (`.[a-f0-9]{8}.webp`)
|
||||
- Image file doesn't exist on disk
|
||||
- Recipe has no images array
|
||||
|
||||
## After Migration
|
||||
|
||||
### Verification
|
||||
|
||||
1. Check a few recipe pages - images should load correctly
|
||||
2. Check browser DevTools → Network tab:
|
||||
- Hashed images should have `Cache-Control: max-age=31536000, immutable`
|
||||
- Unhashed images should have `Cache-Control: max-age=3600, must-revalidate`
|
||||
|
||||
### New Uploads
|
||||
|
||||
All new image uploads will automatically:
|
||||
- Generate content hash
|
||||
- Save both hashed and unhashed versions
|
||||
- Store hashed filename in database
|
||||
- Work immediately with proper cache invalidation
|
||||
|
||||
### Manual Uploads
|
||||
|
||||
If you manually upload an image:
|
||||
1. Drop `recipe-name.webp` in all three folders (full, thumb, placeholder)
|
||||
2. It will work immediately (graceful degradation)
|
||||
3. Next time the recipe is edited and image re-uploaded, it will get a hash
|
||||
|
||||
## Rollback (If Needed)
|
||||
|
||||
If something goes wrong:
|
||||
|
||||
1. **Database rollback**: Restore from backup taken before migration
|
||||
2. **Files**: The original unhashed files are still on disk - no data loss
|
||||
3. **Remove hashed files** (optional):
|
||||
```bash
|
||||
cd /var/lib/www/static/rezepte
|
||||
find . -name '*.[a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9][a-f0-9].webp' -delete
|
||||
```
|
||||
|
||||
## Safety Features
|
||||
|
||||
1. ✅ **Production-only**: Won't run unless `IMAGE_DIR=/var/lib/www/static`
|
||||
2. ✅ **Confirmation token**: Requires `{"confirm": "MIGRATE_IMAGES"}` in request body
|
||||
3. ✅ **Authentication**: Requires logged-in user
|
||||
4. ✅ **Non-destructive**: Copies files (keeps originals)
|
||||
5. ✅ **Skip already migrated**: Won't re-process files that already have hashes
|
||||
6. ✅ **Detailed logging**: Returns detailed report of what was changed
|
||||
|
||||
## Technical Details
|
||||
|
||||
### Hash Generation
|
||||
|
||||
- Uses SHA-256 of image file content
|
||||
- First 8 hex characters used (4 billion combinations)
|
||||
- Same image = same hash (deterministic)
|
||||
- Different image = different hash (cache invalidation)
|
||||
|
||||
### File Structure
|
||||
|
||||
```
|
||||
/var/lib/www/static/rezepte/
|
||||
├── full/
|
||||
│ ├── maccaroni.webp ← Unhashed (fallback)
|
||||
│ ├── maccaroni.a1b2c3d4.webp ← Hashed (cache busting)
|
||||
│ └── ...
|
||||
├── thumb/
|
||||
│ ├── maccaroni.webp
|
||||
│ ├── maccaroni.a1b2c3d4.webp
|
||||
│ └── ...
|
||||
└── placeholder/
|
||||
├── maccaroni.webp
|
||||
├── maccaroni.a1b2c3d4.webp
|
||||
└── ...
|
||||
```
|
||||
|
||||
### Database Schema
|
||||
|
||||
```typescript
|
||||
images: [{
|
||||
mediapath: "maccaroni.a1b2c3d4.webp", // Full filename with hash
|
||||
alt: "Maccaroni and cheese",
|
||||
caption: "Delicious comfort food"
|
||||
}]
|
||||
```
|
||||
|
||||
### Frontend Usage
|
||||
|
||||
```svelte
|
||||
<!-- Card.svelte -->
|
||||
<script>
|
||||
// Uses images[0].mediapath (with hash)
|
||||
// Falls back to short_name.webp if missing
|
||||
const img_name = $derived(
|
||||
recipe.images?.[0]?.mediapath ||
|
||||
`${recipe.short_name}.webp`
|
||||
);
|
||||
</script>
|
||||
|
||||
<img src="https://bocken.org/static/rezepte/thumb/{img_name}" />
|
||||
```
|
||||
|
||||
## Questions?
|
||||
|
||||
If you encounter issues:
|
||||
1. Check nginx error logs: `sudo tail -f /var/log/nginx/error.log`
|
||||
2. Check application logs for the migration endpoint
|
||||
3. Verify file permissions on `/var/lib/www/static/rezepte/`
|
||||
4. Ensure database connection is working
|
||||
|
||||
The migration is designed to be safe and non-destructive. Original files are never deleted, only copied.
|
||||
Reference in New Issue
Block a user