mirror of
https://github.com/kyantech/Palmr.git
synced 2025-10-23 06:11:58 +00:00
docs: add migration guide for file expiration feature
Co-authored-by: danielalves96 <62755605+danielalves96@users.noreply.github.com>
This commit is contained in:
388
apps/server/MIGRATION_FILE_EXPIRATION.md
Normal file
388
apps/server/MIGRATION_FILE_EXPIRATION.md
Normal file
@@ -0,0 +1,388 @@
|
|||||||
|
# File Expiration Feature - Migration Guide
|
||||||
|
|
||||||
|
This guide helps you migrate to the new file expiration feature introduced in Palmr v3.2.5-beta.
|
||||||
|
|
||||||
|
## What's New
|
||||||
|
|
||||||
|
The file expiration feature allows files to have an optional expiration date. When files expire, they can be automatically deleted by a maintenance script, helping with:
|
||||||
|
|
||||||
|
- **Security**: Reducing risk of confidential data exposure
|
||||||
|
- **Storage Management**: Automatically freeing up server space
|
||||||
|
- **Convenience**: Eliminating the need for manual file deletion
|
||||||
|
- **Legal Compliance**: Facilitating adherence to data retention regulations (e.g., GDPR)
|
||||||
|
|
||||||
|
## Database Changes
|
||||||
|
|
||||||
|
A new optional `expiration` field has been added to the `File` model:
|
||||||
|
|
||||||
|
```prisma
|
||||||
|
model File {
|
||||||
|
// ... existing fields
|
||||||
|
expiration DateTime? // NEW: Optional expiration date
|
||||||
|
// ... existing fields
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Migration Steps
|
||||||
|
|
||||||
|
### 1. Backup Your Database
|
||||||
|
|
||||||
|
Before running the migration, **always backup your database**:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# For SQLite (default)
|
||||||
|
cp apps/server/prisma/palmr.db apps/server/prisma/palmr.db.backup
|
||||||
|
|
||||||
|
# Or use the built-in backup command if available
|
||||||
|
pnpm db:backup
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Run the Migration
|
||||||
|
|
||||||
|
The migration will automatically run when you start the server, or you can run it manually:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd apps/server
|
||||||
|
pnpm prisma migrate deploy
|
||||||
|
```
|
||||||
|
|
||||||
|
This adds the `expiration` column to the `files` table. **All existing files will have `null` expiration (never expire).**
|
||||||
|
|
||||||
|
### 3. Verify the Migration
|
||||||
|
|
||||||
|
Check that the migration was successful:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd apps/server
|
||||||
|
pnpm prisma studio
|
||||||
|
```
|
||||||
|
|
||||||
|
Look at the `files` table and verify the new `expiration` column exists.
|
||||||
|
|
||||||
|
## API Changes
|
||||||
|
|
||||||
|
### File Registration (Upload)
|
||||||
|
|
||||||
|
**Before:**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"name": "document.pdf",
|
||||||
|
"description": "My document",
|
||||||
|
"extension": "pdf",
|
||||||
|
"size": 1024000,
|
||||||
|
"objectName": "user123/document.pdf"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**After (optional expiration):**
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"name": "document.pdf",
|
||||||
|
"description": "My document",
|
||||||
|
"extension": "pdf",
|
||||||
|
"size": 1024000,
|
||||||
|
"objectName": "user123/document.pdf",
|
||||||
|
"expiration": "2025-12-31T23:59:59.000Z"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The `expiration` field is **optional** - omitting it or setting it to `null` means the file never expires.
|
||||||
|
|
||||||
|
### File Update
|
||||||
|
|
||||||
|
You can now update a file's expiration date:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
PATCH /files/:id
|
||||||
|
Content-Type: application/json
|
||||||
|
|
||||||
|
{
|
||||||
|
"expiration": "2026-01-31T23:59:59.000Z"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
To remove expiration:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"expiration": null
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### File Listing
|
||||||
|
|
||||||
|
File list responses now include the `expiration` field:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"files": [
|
||||||
|
{
|
||||||
|
"id": "file123",
|
||||||
|
"name": "document.pdf",
|
||||||
|
// ... other fields
|
||||||
|
"expiration": "2025-12-31T23:59:59.000Z",
|
||||||
|
"createdAt": "2025-10-21T10:00:00.000Z",
|
||||||
|
"updatedAt": "2025-10-21T10:00:00.000Z"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Setting Up Automatic Cleanup
|
||||||
|
|
||||||
|
The file expiration feature includes a maintenance script that automatically deletes expired files.
|
||||||
|
|
||||||
|
### Manual Execution
|
||||||
|
|
||||||
|
**Dry-run mode** (preview what would be deleted):
|
||||||
|
```bash
|
||||||
|
cd apps/server
|
||||||
|
pnpm cleanup:expired-files
|
||||||
|
```
|
||||||
|
|
||||||
|
**Confirm mode** (actually delete):
|
||||||
|
```bash
|
||||||
|
cd apps/server
|
||||||
|
pnpm cleanup:expired-files:confirm
|
||||||
|
```
|
||||||
|
|
||||||
|
### Automated Scheduling
|
||||||
|
|
||||||
|
#### Option 1: Cron Job (Recommended for Linux/Unix)
|
||||||
|
|
||||||
|
Add to crontab to run daily at 2 AM:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
crontab -e
|
||||||
|
```
|
||||||
|
|
||||||
|
Add this line:
|
||||||
|
```
|
||||||
|
0 2 * * * cd /path/to/Palmr/apps/server && /usr/bin/pnpm cleanup:expired-files:confirm >> /var/log/palmr-cleanup.log 2>&1
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Option 2: Systemd Timer (Linux)
|
||||||
|
|
||||||
|
Create `/etc/systemd/system/palmr-cleanup.service`:
|
||||||
|
```ini
|
||||||
|
[Unit]
|
||||||
|
Description=Palmr Expired Files Cleanup
|
||||||
|
After=network.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
User=palmr
|
||||||
|
WorkingDirectory=/path/to/Palmr/apps/server
|
||||||
|
ExecStart=/usr/bin/pnpm cleanup:expired-files:confirm
|
||||||
|
StandardOutput=journal
|
||||||
|
StandardError=journal
|
||||||
|
```
|
||||||
|
|
||||||
|
Create `/etc/systemd/system/palmr-cleanup.timer`:
|
||||||
|
```ini
|
||||||
|
[Unit]
|
||||||
|
Description=Daily Palmr Cleanup
|
||||||
|
Requires=palmr-cleanup.service
|
||||||
|
|
||||||
|
[Timer]
|
||||||
|
OnCalendar=daily
|
||||||
|
OnCalendar=02:00
|
||||||
|
Persistent=true
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=timers.target
|
||||||
|
```
|
||||||
|
|
||||||
|
Enable:
|
||||||
|
```bash
|
||||||
|
sudo systemctl enable palmr-cleanup.timer
|
||||||
|
sudo systemctl start palmr-cleanup.timer
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Option 3: Docker Compose
|
||||||
|
|
||||||
|
Add a scheduled service to your `docker-compose.yml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
palmr-cleanup:
|
||||||
|
image: palmr:latest
|
||||||
|
command: sh -c "while true; do sleep 86400; pnpm cleanup:expired-files:confirm; done"
|
||||||
|
environment:
|
||||||
|
- DATABASE_URL=file:/data/palmr.db
|
||||||
|
volumes:
|
||||||
|
- ./data:/data
|
||||||
|
- ./uploads:/uploads
|
||||||
|
restart: unless-stopped
|
||||||
|
```
|
||||||
|
|
||||||
|
Or use an external scheduler with a one-shot container:
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
palmr-cleanup:
|
||||||
|
image: palmr:latest
|
||||||
|
command: pnpm cleanup:expired-files:confirm
|
||||||
|
environment:
|
||||||
|
- DATABASE_URL=file:/data/palmr.db
|
||||||
|
volumes:
|
||||||
|
- ./data:/data
|
||||||
|
- ./uploads:/uploads
|
||||||
|
restart: "no"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Backward Compatibility
|
||||||
|
|
||||||
|
This feature is **fully backward compatible**:
|
||||||
|
|
||||||
|
- Existing files automatically have `expiration = null` (never expire)
|
||||||
|
- The `expiration` field is optional in all API endpoints
|
||||||
|
- No changes required to existing client code
|
||||||
|
- Files without expiration dates continue to work exactly as before
|
||||||
|
|
||||||
|
## Client Implementation Examples
|
||||||
|
|
||||||
|
### JavaScript/TypeScript
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Upload file with expiration
|
||||||
|
const uploadWithExpiration = async (file: File) => {
|
||||||
|
// Set expiration to 30 days from now
|
||||||
|
const expiration = new Date();
|
||||||
|
expiration.setDate(expiration.getDate() + 30);
|
||||||
|
|
||||||
|
const response = await fetch('/api/files', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({
|
||||||
|
name: file.name,
|
||||||
|
extension: file.name.split('.').pop(),
|
||||||
|
size: file.size,
|
||||||
|
objectName: `user/${Date.now()}-${file.name}`,
|
||||||
|
expiration: expiration.toISOString(),
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
|
||||||
|
return response.json();
|
||||||
|
};
|
||||||
|
|
||||||
|
// Update file expiration
|
||||||
|
const updateExpiration = async (fileId: string, days: number) => {
|
||||||
|
const expiration = new Date();
|
||||||
|
expiration.setDate(expiration.getDate() + days);
|
||||||
|
|
||||||
|
const response = await fetch(`/api/files/${fileId}`, {
|
||||||
|
method: 'PATCH',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({
|
||||||
|
expiration: expiration.toISOString(),
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
|
||||||
|
return response.json();
|
||||||
|
};
|
||||||
|
|
||||||
|
// Remove expiration (make file permanent)
|
||||||
|
const removExpiration = async (fileId: string) => {
|
||||||
|
const response = await fetch(`/api/files/${fileId}`, {
|
||||||
|
method: 'PATCH',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({
|
||||||
|
expiration: null,
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
|
||||||
|
return response.json();
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
### Python
|
||||||
|
|
||||||
|
```python
|
||||||
|
from datetime import datetime, timedelta
|
||||||
|
import requests
|
||||||
|
|
||||||
|
# Upload file with expiration
|
||||||
|
def upload_with_expiration(file_data):
|
||||||
|
expiration = datetime.utcnow() + timedelta(days=30)
|
||||||
|
|
||||||
|
response = requests.post('http://localhost:3333/files', json={
|
||||||
|
'name': file_data['name'],
|
||||||
|
'extension': file_data['extension'],
|
||||||
|
'size': file_data['size'],
|
||||||
|
'objectName': file_data['objectName'],
|
||||||
|
'expiration': expiration.isoformat() + 'Z'
|
||||||
|
})
|
||||||
|
|
||||||
|
return response.json()
|
||||||
|
|
||||||
|
# Update expiration
|
||||||
|
def update_expiration(file_id, days):
|
||||||
|
expiration = datetime.utcnow() + timedelta(days=days)
|
||||||
|
|
||||||
|
response = requests.patch(f'http://localhost:3333/files/{file_id}', json={
|
||||||
|
'expiration': expiration.isoformat() + 'Z'
|
||||||
|
})
|
||||||
|
|
||||||
|
return response.json()
|
||||||
|
```
|
||||||
|
|
||||||
|
## Best Practices
|
||||||
|
|
||||||
|
1. **Start with dry-run**: Always test the cleanup script in dry-run mode first
|
||||||
|
2. **Monitor logs**: Keep track of what files are being deleted
|
||||||
|
3. **User notifications**: Consider notifying users before their files expire
|
||||||
|
4. **Grace period**: Set expiration dates with a buffer for important files
|
||||||
|
5. **Backup strategy**: Maintain backups before enabling automatic deletion
|
||||||
|
6. **Documentation**: Document your expiration policies for users
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Migration Fails
|
||||||
|
|
||||||
|
If the migration fails:
|
||||||
|
|
||||||
|
1. Check database connectivity
|
||||||
|
2. Ensure you have write permissions
|
||||||
|
3. Verify the database file isn't locked
|
||||||
|
4. Try running `pnpm prisma migrate reset` (WARNING: this will delete all data)
|
||||||
|
|
||||||
|
### Cleanup Script Not Deleting Files
|
||||||
|
|
||||||
|
1. Verify files have expiration dates set and are in the past
|
||||||
|
2. Check script is running with `--confirm` flag
|
||||||
|
3. Review logs for specific errors
|
||||||
|
3. Ensure script has permissions to delete from storage
|
||||||
|
|
||||||
|
### Need to Rollback
|
||||||
|
|
||||||
|
If you need to rollback the migration:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd apps/server
|
||||||
|
|
||||||
|
# View migration history
|
||||||
|
pnpm prisma migrate status
|
||||||
|
|
||||||
|
# Rollback (requires manual SQL for production)
|
||||||
|
# SQLite example:
|
||||||
|
sqlite3 prisma/palmr.db "ALTER TABLE files DROP COLUMN expiration;"
|
||||||
|
```
|
||||||
|
|
||||||
|
Note: Prisma doesn't support automatic rollback. You must manually reverse the migration or restore from backup.
|
||||||
|
|
||||||
|
## Support
|
||||||
|
|
||||||
|
For issues or questions:
|
||||||
|
|
||||||
|
- Create an issue on GitHub
|
||||||
|
- Check the documentation at https://palmr.kyantech.com.br
|
||||||
|
- Review the scripts README at `apps/server/src/scripts/README.md`
|
||||||
|
|
||||||
|
## Changelog
|
||||||
|
|
||||||
|
### Version 3.2.5-beta
|
||||||
|
|
||||||
|
- Added optional `expiration` field to File model
|
||||||
|
- Created `cleanup-expired-files` maintenance script
|
||||||
|
- Updated File DTOs to support expiration in create/update operations
|
||||||
|
- Added API documentation for expiration field
|
||||||
|
- Created comprehensive documentation for setup and usage
|
Reference in New Issue
Block a user