---
title: Cleanup Orphan Files
icon: Trash2
---

This guide provides detailed instructions on how to identify and remove orphan file records from your Palmr database. Orphan files are database entries that reference files that no longer exist in the storage system, typically resulting from failed uploads or interrupted transfers.

## When and why to use this tool

The orphan file cleanup script is designed to maintain database integrity by removing stale file records. Consider using this tool if:

- Users are experiencing "File not found" errors when attempting to download files that appear in the UI
- You've identified failed uploads that left incomplete database records
- You're performing routine database maintenance
- You've migrated storage systems and need to verify file consistency
- You need to free up quota space occupied by phantom file records

> **Note:** This script only removes **database records** for files that don't exist in storage. It does not delete physical files. Files that exist in storage will remain untouched.

## How the cleanup works

Palmr provides a maintenance script that scans all file records in the database and verifies their existence in the storage system (either filesystem or S3). The script operates in two modes:

- **Dry-run mode (default):** Identifies orphan files and displays what would be deleted without making any changes
- **Confirmation mode:** Actually removes the orphan database records after explicit confirmation

The script maintains safety by:
- Checking file existence before marking as orphan
- Providing detailed statistics and file listings
- Requiring explicit `--confirm` flag to delete records
- Working with both filesystem and S3 storage providers
- Preserving all files that exist in storage

## Understanding orphan files

### What are orphan files?

Orphan files occur when:

1. **Failed chunked uploads:** A large file upload starts, creates a database record, but the upload fails before completion
2. **Interrupted transfers:** Network issues or server restarts interrupt file transfers mid-process
3. **Manual deletions:** Files are manually deleted from storage without removing the database record
4. **Storage migrations:** Files are moved or lost during storage system changes

### Why they cause problems

When orphan records exist in the database:
- Users see files in the UI that cannot be downloaded
- Download attempts result in "ENOENT: no such file or directory" errors
- Storage quota calculations become inaccurate
- The system returns 500 errors instead of proper 404 responses (in older versions)

### Renamed files with suffixes

Files with duplicate names are automatically renamed with suffixes (e.g., `file (1).png`, `file (2).png`). Sometimes the upload fails after the database record is created but before the physical file is saved, creating an orphan record with a suffix.

**Example:**
```
Database record: photo (1).png → objectName: user123/1758805195682-Rjn9at692HdR.png
Physical file: Does not exist ❌
```

## Step-by-step instructions

### 1. Access the server environment

**For Docker installations:**

```bash
docker exec -it <container_name> /bin/sh
cd /app/palmr-app
```

**For bare-metal installations:**

```bash
cd /path/to/palmr/apps/server
```

### 2. Run the cleanup script in dry-run mode

First, run the script without the `--confirm` flag to see what would be deleted:

```bash
pnpm cleanup:orphan-files
```

This will:
- Scan all file records in the database
- Check if each file exists in storage
- Display a summary of orphan files
- Show what would be deleted (without actually deleting)

### 3. Review the output

The script will provide detailed information about orphan files:

```text
Starting orphan file cleanup...
Storage mode: Filesystem
Found 7 files in database
❌ Orphan: photo(1).png (cmddjchw80000gmiimqnxga2g/1758805195682-Rjn9at692HdR.png)
❌ Orphan: document.pdf (cmddjchw80000gmiimqnxga2g/1758803757558-JQxlvF816UVo.pdf)

📊 Summary:
  Total files in DB: 7
  ✅ Files with storage: 5
  ❌ Orphan files: 2

🗑️  Orphan files to be deleted:
  - photo(1).png (0.76 MB) - cmddjchw80000gmiimqnxga2g/1758805195682-Rjn9at692HdR.png
  - document.pdf (2.45 MB) - cmddjchw80000gmiimqnxga2g/1758803757558-JQxlvF816UVo.pdf

⚠️  Dry run mode. To actually delete orphan records, run with --confirm flag:
   pnpm cleanup:orphan-files:confirm
```

### 4. Confirm and execute the cleanup

If you're satisfied with the results and want to proceed with the deletion:

```bash
pnpm cleanup:orphan-files:confirm
```

This will remove the orphan database records and display a confirmation:

```text
🗑️  Deleting orphan file records...
  ✓ Deleted: photo(1).png
  ✓ Deleted: document.pdf

✅ Cleanup complete!
   Deleted 2 orphan file records
```

## Example session

Below is a complete example of running the cleanup script:

```bash
$ pnpm cleanup:orphan-files

> palmr-api@3.2.3-beta cleanup:orphan-files
> tsx src/scripts/cleanup-orphan-files.ts

Starting orphan file cleanup...
Storage mode: Filesystem
Found 15 files in database
❌ Orphan: video.mp4 (user123/1758803869037-1WhtnrQioeFQ.mp4)
❌ Orphan: image(1).png (user123/1758805195682-Rjn9at692HdR.png)
❌ Orphan: image(2).png (user123/1758803757558-JQxlvF816UVo.png)

📊 Summary:
  Total files in DB: 15
  ✅ Files with storage: 12
  ❌ Orphan files: 3

🗑️  Orphan files to be deleted:
  - video.mp4 (97.09 MB) - user123/1758803869037-1WhtnrQioeFQ.mp4
  - image(1).png (0.01 MB) - user123/1758805195682-Rjn9at692HdR.png
  - image(2).png (0.76 MB) - user123/1758803757558-JQxlvF816UVo.png

⚠️  Dry run mode. To actually delete orphan records, run with --confirm flag:
   pnpm cleanup:orphan-files:confirm

$ pnpm cleanup:orphan-files:confirm

> palmr-api@3.2.3-beta cleanup:orphan-files:confirm
> tsx src/scripts/cleanup-orphan-files.ts --confirm

Starting orphan file cleanup...
Storage mode: Filesystem
Found 15 files in database
❌ Orphan: video.mp4 (user123/1758803869037-1WhtnrQioeFQ.mp4)
❌ Orphan: image(1).png (user123/1758805195682-Rjn9at692HdR.png)
❌ Orphan: image(2).png (user123/1758803757558-JQxlvF816UVo.png)

📊 Summary:
  Total files in DB: 15
  ✅ Files with storage: 12
  ❌ Orphan files: 3

🗑️  Orphan files to be deleted:
  - video.mp4 (97.09 MB) - user123/1758803869037-1WhtnrQioeFQ.mp4
  - image(1).png (0.01 MB) - user123/1758805195682-Rjn9at692HdR.png
  - image(2).png (0.76 MB) - user123/1758803757558-JQxlvF816UVo.png

🗑️  Deleting orphan file records...
  ✓ Deleted: video.mp4
  ✓ Deleted: image(1).png
  ✓ Deleted: image(2).png

✅ Cleanup complete!
   Deleted 3 orphan file records

Script completed successfully
```

## Troubleshooting common issues

### No orphan files found

```text
📊 Summary:
  Total files in DB: 10
  ✅ Files with storage: 10
  ❌ Orphan files: 0

✨ No orphan files found!
```

**This is good!** It means your database is in sync with your storage system.

### Script cannot connect to database

If you see database connection errors:

1. Verify the database file exists:
   ```bash
   ls -la prisma/palmr.db
   ```

2. Check database permissions:
   ```bash
   chmod 644 prisma/palmr.db
   ```

3. Ensure you're in the correct directory:
   ```bash
   pwd  # Should show .../palmr/apps/server
   ```

### Storage provider errors

For **S3 storage:**
- Verify your S3 credentials are configured correctly
- Check that the bucket is accessible
- Ensure network connectivity to S3

For **Filesystem storage:**
- Verify the uploads directory exists and is readable
- Check file system permissions
- Ensure sufficient disk space

### Script fails to delete records

If deletion fails for specific files:
- Check database locks (close other connections)
- Verify you have write permissions to the database
- Review the error message for specific details

## Understanding the output

### File statistics

The script provides several key metrics:

- **Total files in DB:** All file records in your database
- **Files with storage:** Records where the physical file exists
- **Orphan files:** Records where the physical file is missing

### File information

For each orphan file, you'll see:

- **Name:** Display name in the UI
- **Size:** File size as recorded in the database
- **Object name:** Internal storage path

Example: `photo(1).png (0.76 MB) - user123/1758805195682-Rjn9at692HdR.png`

## Prevention and best practices

### Prevent orphan files from occurring

1. **Monitor upload failures:** Check server logs for upload errors
2. **Stable network:** Ensure reliable network connectivity for large uploads
3. **Adequate resources:** Provide sufficient disk space and memory
4. **Regular maintenance:** Run this script periodically as part of maintenance

### When to run cleanup

Consider running the cleanup script:

- **Monthly:** As part of routine database maintenance
- **After incidents:** Following server crashes or storage issues
- **Before migrations:** Before moving to new storage systems
- **When users report errors:** If download failures are reported

### Safe cleanup practices

1. **Always run dry-run first:** Review what will be deleted before confirming
2. **Backup your database:** Create a backup before running with `--confirm`
3. **Check during low usage:** Run during off-peak hours to minimize disruption
4. **Document the cleanup:** Keep records of when and why cleanup was performed
5. **Verify after cleanup:** Check that file counts match expectations

## Technical details

### How files are stored

When files are uploaded to Palmr:

1. Frontend generates a safe object name using random identifiers
2. Backend creates the final `objectName` as: `${userId}/${timestamp}-${randomId}.${extension}`
3. If a duplicate name exists, the **display name** gets a suffix, but `objectName` remains unique
4. Physical file is stored using `objectName`, display name is stored separately in database

### Storage providers

The script works with both storage providers:

- **FilesystemStorageProvider:** Uses `fs.promises.access()` to check file existence
- **S3StorageProvider:** Uses `HeadObjectCommand` to verify objects in S3 bucket

### Database schema

Files table structure:
```typescript
{
  name: string           // Display name (can have suffixes like "file (1).png")
  objectName: string     // Physical storage path (always unique)
  size: bigint          // File size in bytes
  extension: string     // File extension
  userId: string        // Owner of the file
  folderId: string?     // Parent folder (null for root)
}
```

## Related improvements

### Download validation (v3.2.3-beta+)

Starting from version 3.2.3-beta, Palmr includes enhanced download validation:

- Files are checked for existence **before** attempting download
- Returns proper 404 error if file is missing (instead of 500)
- Provides helpful error message to users

This prevents errors when trying to download orphan files that haven't been cleaned up yet.

## Security considerations

- **Read-only by default:** Dry-run mode is safe and doesn't modify data
- **Explicit confirmation:** Requires `--confirm` flag to delete records
- **No file deletion:** Only removes database records, never deletes physical files
- **Audit trail:** All actions are logged to console
- **Permission-based:** Only users with server access can run the script

> **Important:** This script does not delete physical files from storage. It only removes database records for files that don't exist. This is intentional to prevent accidental data loss.

## FAQ

**Q: Will this delete my files?**  
A: No. The script only removes database records for files that are already missing from storage. Physical files are never deleted.

**Q: Can I undo the cleanup?**  
A: No. Once orphan records are deleted, they cannot be recovered. Always run dry-run mode first and backup your database.

**Q: Why do orphan files have suffixes like (1), (2)?**  
A: When duplicate files are uploaded, Palmr renames them with suffixes. If the upload fails after creating the database record, an orphan with a suffix remains.

**Q: How often should I run this script?**  
A: Monthly maintenance is usually sufficient. Run more frequently if you experience many upload failures.

**Q: Does this work with S3 storage?**  
A: Yes! The script automatically detects your storage provider (filesystem or S3) and works with both.

**Q: What if I have thousands of orphan files?**  
A: The script handles large numbers efficiently. Consider running during off-peak hours for very large cleanups.

**Q: Can this fix "File not found" errors?**  
A: Yes, if the errors are caused by orphan database records. The script removes those records, preventing future errors.
