mirror of
https://github.com/kyantech/Palmr.git
synced 2025-10-23 06:11:58 +00:00
refactor: update translation management system to enhance key management and remove auto-translation
- Revised the translation management system to focus on key management, automating synchronization and validation of internationalization files. - Removed the auto-translation functionality, emphasizing the need for manual translation by native speakers or professionals. - Updated documentation to reflect changes in the workflow and best practices for managing translations. - Adjusted related scripts and commands in package.json to align with the new translation process.
This commit is contained in:
Daniel Luiz Alves
@@ -6,7 +6,7 @@ icon: Languages
|
||||
import { Tab, Tabs } from "fumadocs-ui/components/tabs";
|
||||
import { Callout } from "fumadocs-ui/components/callout";
|
||||
|
||||
Palmr includes a comprehensive translation management system that automates synchronization, validation, and translation of the application's internationalization files.
|
||||
Palmr includes a comprehensive translation key management system that automates synchronization and validation of the application's internationalization files.
|
||||
|
||||
## Overview
|
||||
|
||||
@@ -14,7 +14,7 @@ The translation management system consists of Python scripts that help maintain
|
||||
|
||||
- **Synchronization**: Automatically add missing translation keys
|
||||
- **Validation**: Check translation status and completeness
|
||||
- **Auto-translation**: Use Google Translate API for initial translations
|
||||
- **Key Management**: Manage translation keys structure and consistency
|
||||
- **Reporting**: Generate detailed translation reports
|
||||
|
||||
## Quick Start
|
||||
@@ -31,10 +31,7 @@ The translation management system consists of Python scripts that help maintain
|
||||
# Synchronize missing keys
|
||||
pnpm run translations:sync
|
||||
|
||||
# Auto-translate [TO_TRANSLATE] strings
|
||||
pnpm run translations:translate
|
||||
|
||||
# Dry run mode (test without changes)
|
||||
# Test workflow without changes (dry run)
|
||||
pnpm run translations:dry-run
|
||||
|
||||
# Show help
|
||||
@@ -52,7 +49,7 @@ The translation management system consists of Python scripts that help maintain
|
||||
# Individual commands
|
||||
python3 run_translations.py check
|
||||
python3 run_translations.py sync
|
||||
python3 run_translations.py translate
|
||||
python3 run_translations.py all --dry-run
|
||||
python3 run_translations.py help
|
||||
```
|
||||
|
||||
@@ -64,11 +61,10 @@ The translation management system consists of Python scripts that help maintain
|
||||
### Main Commands (npm/pnpm)
|
||||
|
||||
| Command | Description |
|
||||
| --------------------------------- | ------------------------------------------- |
|
||||
| `pnpm run translations` | Complete workflow: sync + translate + check |
|
||||
| ------------------------------- | ----------------------------------------- |
|
||||
| `pnpm run translations` | Complete workflow: sync + check |
|
||||
| `pnpm run translations:check` | Check translation status and completeness |
|
||||
| `pnpm run translations:sync` | Synchronize missing keys from en-US.json |
|
||||
| `pnpm run translations:translate` | Auto-translate [TO_TRANSLATE] strings |
|
||||
| `pnpm run translations:dry-run` | Test workflow without making changes |
|
||||
| `pnpm run translations:help` | Show detailed help and examples |
|
||||
|
||||
@@ -80,13 +76,13 @@ When you add new text to the application:
|
||||
|
||||
1. **Add to English**: Update `apps/web/messages/en-US.json` with your new keys
|
||||
2. **Sync translations**: Run `pnpm run translations:sync` to add missing keys to all languages
|
||||
3. **Auto-translate**: Run `pnpm run translations:translate` for automatic translations
|
||||
4. **Review translations**: **Mandatory step** - Check all auto-generated translations for accuracy
|
||||
5. **Test in UI**: Verify translations work correctly in the application interface
|
||||
3. **Manual translation**: Translate strings marked with `[TO_TRANSLATE]` manually
|
||||
4. **Test in UI**: Verify translations work correctly in the application interface
|
||||
|
||||
<Callout type="important">
|
||||
**Never skip step 4**: Auto-generated translations must be reviewed before
|
||||
committing to production. They are a starting point, not a final solution.
|
||||
**Manual translation required**: All strings marked with `[TO_TRANSLATE]` must
|
||||
be manually translated by native speakers or professional translators for
|
||||
accuracy.
|
||||
</Callout>
|
||||
|
||||
### 2. Checking Translation Status
|
||||
@@ -110,7 +106,7 @@ The report shows:
|
||||
|
||||
### 3. Manual Translation Process
|
||||
|
||||
For critical strings or when automatic translation isn't sufficient:
|
||||
For all translation strings:
|
||||
|
||||
1. **Find untranslated strings**: Look for `[TO_TRANSLATE] Original text` in language files
|
||||
2. **Replace with translation**: Remove the prefix and add proper translation
|
||||
@@ -130,13 +126,13 @@ apps/web/
|
||||
├── run_translations.py # Main wrapper
|
||||
├── sync_translations.py # Synchronization
|
||||
├── check_translations.py # Status checking
|
||||
└── translate_missing.py # Auto-translation
|
||||
└── clean_translations.py # Cleanup utilities
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Python 3.6 or higher** - Required for running the translation scripts
|
||||
- **No external dependencies** - Scripts use only Python standard libraries (googletrans auto-installs when needed)
|
||||
- **No external dependencies** - Scripts use only Python standard libraries
|
||||
- **UTF-8 support** - Ensure your terminal supports UTF-8 for proper display of translations
|
||||
|
||||
## Script Details
|
||||
@@ -149,8 +145,7 @@ The main script provides a unified interface for all translation operations:
|
||||
|
||||
- `check` - Check translation status and generate reports
|
||||
- `sync` - Synchronize missing keys from reference language
|
||||
- `translate` - Automatically translate marked strings
|
||||
- `all` - Run complete workflow (sync + translate + check)
|
||||
- `all` - Run complete workflow (sync + check)
|
||||
- `help` - Show detailed help with examples
|
||||
|
||||
#### How it Works
|
||||
@@ -197,27 +192,6 @@ Provides comprehensive translation analysis:
|
||||
- **Quality insights**: Identifies potential translation issues
|
||||
- **Export friendly**: Output can be redirected to files for reports
|
||||
|
||||
### Auto-Translation Script (`translate_missing.py`)
|
||||
|
||||
Automated translation using Google Translate API:
|
||||
|
||||
#### Process
|
||||
|
||||
1. **Dependency check**: Auto-installs `googletrans` if needed
|
||||
2. **Scan for work**: Finds all `[TO_TRANSLATE]` prefixed strings
|
||||
3. **Language mapping**: Maps file names to Google Translate language codes
|
||||
4. **Batch translation**: Processes strings with rate limiting
|
||||
5. **Update files**: Replaces marked strings with translations
|
||||
6. **Error handling**: Retries failed translations, reports results
|
||||
|
||||
#### Safety Features
|
||||
|
||||
- **Rate limiting**: Configurable delay between requests
|
||||
- **Retry logic**: Multiple attempts for failed translations
|
||||
- **Dry run mode**: Preview changes without modifications
|
||||
- **Language skipping**: Exclude specific languages from processing
|
||||
- **Progress tracking**: Real-time status of translation progress
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
### Custom Parameters
|
||||
@@ -240,22 +214,6 @@ python3 scripts/run_translations.py sync --messages-dir /path/to/messages
|
||||
python3 scripts/run_translations.py sync --dry-run
|
||||
```
|
||||
|
||||
#### Translation Parameters (`translate`)
|
||||
|
||||
```bash
|
||||
# Custom delay between translation requests (avoid rate limiting)
|
||||
python3 scripts/run_translations.py translate --delay 2.0
|
||||
|
||||
# Skip specific languages from translation
|
||||
python3 scripts/run_translations.py translate --skip-languages pt-BR.json fr-FR.json
|
||||
|
||||
# Dry run - see what would be translated
|
||||
python3 scripts/run_translations.py translate --dry-run
|
||||
|
||||
# Specify custom messages directory
|
||||
python3 scripts/run_translations.py translate --messages-dir /path/to/messages
|
||||
```
|
||||
|
||||
#### Check Parameters (`check`)
|
||||
|
||||
```bash
|
||||
@@ -269,57 +227,45 @@ python3 scripts/run_translations.py check --messages-dir /path/to/messages
|
||||
### Parameter Reference
|
||||
|
||||
| Parameter | Commands | Description |
|
||||
| ------------------------ | -------------------------- | --------------------------------------------- |
|
||||
| `--dry-run` | `sync`, `translate`, `all` | Preview changes without modifying files |
|
||||
| ------------------------ | --------------- | --------------------------------------------- |
|
||||
| `--dry-run` | `sync`, `all` | Preview changes without modifying files |
|
||||
| `--messages-dir` | All | Custom directory containing translation files |
|
||||
| `--reference` | `sync`, `check` | Reference file to use (default: en-US.json) |
|
||||
| `--no-mark-untranslated` | `sync` | Don't add [TO_TRANSLATE] prefix to new keys |
|
||||
| `--delay` | `translate` | Delay in seconds between translation requests |
|
||||
| `--skip-languages` | `translate` | List of language files to skip |
|
||||
|
||||
### Dry Run Mode
|
||||
|
||||
Always test changes first:
|
||||
|
||||
```bash
|
||||
# Test complete workflow
|
||||
# Test complete workflow (recommended)
|
||||
pnpm run translations:dry-run
|
||||
|
||||
# Test individual commands
|
||||
# Alternative: Direct Python commands
|
||||
python3 scripts/run_translations.py all --dry-run
|
||||
python3 scripts/run_translations.py sync --dry-run
|
||||
python3 scripts/run_translations.py translate --dry-run
|
||||
```
|
||||
|
||||
### Common Use Cases
|
||||
|
||||
#### Scenario 1: Careful Translation with Review
|
||||
#### Scenario 1: Adding Keys Without Marking for Translation
|
||||
|
||||
```bash
|
||||
# 1. Sync new keys without auto-marking for translation
|
||||
# Sync new keys without auto-marking for translation
|
||||
python3 scripts/run_translations.py sync --no-mark-untranslated
|
||||
|
||||
# 2. Manually mark specific keys that need translation
|
||||
# Manually mark specific keys that need translation
|
||||
# Edit files to add [TO_TRANSLATE] prefix where needed
|
||||
|
||||
# 3. Translate only marked strings with slower rate
|
||||
python3 scripts/run_translations.py translate --delay 2.0
|
||||
```
|
||||
|
||||
#### Scenario 2: Skip Already Reviewed Languages
|
||||
|
||||
```bash
|
||||
# Skip languages that were already manually reviewed
|
||||
python3 scripts/run_translations.py translate --skip-languages pt-BR.json es-ES.json
|
||||
```
|
||||
|
||||
#### Scenario 3: Custom Project Structure
|
||||
#### Scenario 2: Custom Project Structure
|
||||
|
||||
```bash
|
||||
# Work with translations in different directory
|
||||
python3 scripts/run_translations.py all --messages-dir ../custom-translations
|
||||
```
|
||||
|
||||
#### Scenario 4: Quality Assurance
|
||||
#### Scenario 3: Quality Assurance
|
||||
|
||||
```bash
|
||||
# Use different language as reference for comparison
|
||||
@@ -352,15 +298,15 @@ Translation files use nested JSON structure:
|
||||
}
|
||||
```
|
||||
|
||||
## Automatic Translation
|
||||
## Manual Translation
|
||||
|
||||
<Callout type="warning">
|
||||
**Review Required**: Automatic translations are provided for convenience but
|
||||
**must be reviewed** before production use. They serve as a starting point,
|
||||
not a final solution.
|
||||
**Professional translation recommended**: For production applications,
|
||||
consider using professional translation services or native speakers to ensure
|
||||
high-quality, culturally appropriate translations.
|
||||
</Callout>
|
||||
|
||||
The system uses Google Translate (free API) to automatically translate strings marked with `[TO_TRANSLATE]`:
|
||||
The system marks strings that need translation with the `[TO_TRANSLATE]` prefix:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -368,7 +314,7 @@ The system uses Google Translate (free API) to automatically translate strings m
|
||||
}
|
||||
```
|
||||
|
||||
After auto-translation:
|
||||
After manual translation:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -378,14 +324,15 @@ After auto-translation:
|
||||
|
||||
### Translation Review Process
|
||||
|
||||
1. **Generate**: Use `pnpm run translations:translate` to auto-translate
|
||||
2. **Review**: Check each translation for:
|
||||
1. **Identify**: Use `pnpm run translations:check` to find untranslated strings
|
||||
2. **Translate**: Replace `[TO_TRANSLATE]` strings with proper translations
|
||||
3. **Review**: Check each translation for:
|
||||
- **Context accuracy**: Does it make sense in the UI?
|
||||
- **Technical terms**: Are they correctly translated?
|
||||
- **Tone consistency**: Matches the application's voice?
|
||||
- **Cultural appropriateness**: Suitable for target audience?
|
||||
3. **Test**: Verify translations in the actual application interface
|
||||
4. **Document**: Note any translation decisions for future reference
|
||||
4. **Test**: Verify translations in the actual application interface
|
||||
5. **Document**: Note any translation decisions for future reference
|
||||
|
||||
### Common Review Points
|
||||
|
||||
@@ -405,15 +352,15 @@ After auto-translation:
|
||||
### Translation Workflow for Development
|
||||
|
||||
1. **English First**: Add all new text to `apps/web/messages/en-US.json`
|
||||
2. **Auto-generate**: Use scripts to generate translations for other languages
|
||||
3. **Review Required**: All auto-generated translations must be reviewed before production use
|
||||
2. **Sync keys**: Use scripts to generate key structure for other languages
|
||||
3. **Manual translation**: All strings must be manually translated for accuracy
|
||||
4. **Quality Check**: Run translation validation before merging PRs
|
||||
|
||||
### Why English as Parent Language?
|
||||
|
||||
- **Consistency**: Ensures all languages have the same keys structure
|
||||
- **Reference**: English serves as the source of truth for meaning
|
||||
- **Auto-translation**: Scripts use English as source for automatic translation
|
||||
- **Key management**: Scripts use English as source for key synchronization
|
||||
- **Documentation**: Most technical documentation is in English
|
||||
|
||||
## Best Practices
|
||||
@@ -422,8 +369,8 @@ After auto-translation:
|
||||
|
||||
1. **Always use English as reference**: Add new keys to `en-US.json` first - never add keys directly to other languages
|
||||
2. **Use semantic key names**: `dashboard.welcome` instead of `text1`
|
||||
3. **Test translations**: Run `pnpm run translations:dry-run` before committing
|
||||
4. **Review auto-translations**: All generated translations must be reviewed before production
|
||||
3. **Test key sync**: Run `pnpm run translations:dry-run` before committing
|
||||
4. **Coordinate with translators**: Ensure translation team is aware of new keys
|
||||
5. **Maintain consistency**: Use existing patterns for similar UI elements
|
||||
|
||||
### For Translators
|
||||
@@ -432,6 +379,7 @@ After auto-translation:
|
||||
2. **Check identical strings**: May need localization even if identical to English
|
||||
3. **Use proper formatting**: Maintain HTML tags and placeholders
|
||||
4. **Test in context**: Verify translations work in the actual UI
|
||||
5. **Maintain glossary**: Keep consistent terminology across translations
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
@@ -439,17 +387,9 @@ After auto-translation:
|
||||
|
||||
**Python not found**: Ensure Python 3.6+ is installed and in PATH
|
||||
|
||||
**googletrans errors**: The system auto-installs dependencies, but you can manually install:
|
||||
**Permission errors**: Ensure write permissions to message files
|
||||
|
||||
```bash
|
||||
pip install googletrans==4.0.0rc1
|
||||
```
|
||||
|
||||
**Rate limiting**: Increase delay between requests:
|
||||
|
||||
```bash
|
||||
python3 scripts/run_translations.py translate --delay 2.0
|
||||
```
|
||||
**Encoding issues**: Ensure your terminal supports UTF-8
|
||||
|
||||
### Getting Help
|
||||
|
||||
@@ -491,28 +431,6 @@ LANGUAGE COMPLETENESS STRINGS UNTRANSLATED POSSIBLE MATCHES
|
||||
================================================================================
|
||||
```
|
||||
|
||||
### Auto-Translation Progress
|
||||
|
||||
```
|
||||
🌍 Translating 3 languages...
|
||||
⏱️ Delay between requests: 1.0s
|
||||
|
||||
[1/3] 🌐 Language: PT
|
||||
🔍 Processing: pt-BR.json
|
||||
📝 Found 12 strings to translate
|
||||
📍 (1/12) Translating: dashboard.welcome
|
||||
✅ "Welcome to Palmr" → "Bem-vindo ao Palmr"
|
||||
💾 File saved with 12 translations
|
||||
|
||||
📊 FINAL SUMMARY
|
||||
================================================================================
|
||||
✅ Translations performed:
|
||||
• 34 successes
|
||||
• 2 failures
|
||||
• 36 total processed
|
||||
• Success rate: 94.4%
|
||||
```
|
||||
|
||||
## Contributing
|
||||
|
||||
When contributing translations:
|
||||
@@ -520,6 +438,6 @@ When contributing translations:
|
||||
1. **Follow the workflow**: Use the provided scripts for consistency
|
||||
2. **Test thoroughly**: Run complete checks before submitting
|
||||
3. **Document changes**: Note any significant translation decisions
|
||||
4. **Maintain quality**: Review auto-translations for accuracy
|
||||
4. **Maintain quality**: Ensure manual translations are accurate and appropriate
|
||||
|
||||
The translation management system ensures consistency and makes it easy to maintain high-quality localization across all supported languages.
|
||||
|
||||
@@ -24,9 +24,8 @@
|
||||
"translations": "python3 scripts/run_translations.py all",
|
||||
"translations:check": "python3 scripts/run_translations.py check",
|
||||
"translations:sync": "python3 scripts/run_translations.py sync",
|
||||
"translations:translate": "python3 scripts/run_translations.py translate",
|
||||
"translations:help": "python3 scripts/run_translations.py help",
|
||||
"translations:dry-run": "python3 scripts/run_translations.py all --dry-run"
|
||||
"translations:dry-run": "python3 scripts/run_translations.py all --dry-run",
|
||||
"translations:help": "python3 scripts/run_translations.py help"
|
||||
},
|
||||
"dependencies": {
|
||||
"@hookform/resolvers": "^5.0.1",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Main script to run all Palmr translation operations.
|
||||
Main script to run Palmr translation management operations.
|
||||
Makes it easy to run scripts without remembering specific names.
|
||||
"""
|
||||
|
||||
@@ -17,25 +17,65 @@ def run_command(script_name: str, args: list) -> int:
|
||||
return subprocess.run(cmd).returncode
|
||||
|
||||
|
||||
def filter_args_for_script(script_name: str, args: list) -> list:
|
||||
"""Filter arguments based on what each script accepts."""
|
||||
|
||||
# Arguments that check_translations.py accepts
|
||||
check_args = ['--messages-dir', '--reference']
|
||||
|
||||
# Arguments that sync_translations.py accepts
|
||||
sync_args = ['--messages-dir', '--reference', '--no-mark-untranslated', '--dry-run']
|
||||
|
||||
if script_name == 'check_translations.py':
|
||||
filtered = []
|
||||
skip_next = False
|
||||
for i, arg in enumerate(args):
|
||||
if skip_next:
|
||||
skip_next = False
|
||||
continue
|
||||
if arg in check_args:
|
||||
filtered.append(arg)
|
||||
# Add the value for the argument if it exists
|
||||
if i + 1 < len(args) and not args[i + 1].startswith('--'):
|
||||
filtered.append(args[i + 1])
|
||||
skip_next = True
|
||||
return filtered
|
||||
|
||||
elif script_name == 'sync_translations.py':
|
||||
filtered = []
|
||||
skip_next = False
|
||||
for i, arg in enumerate(args):
|
||||
if skip_next:
|
||||
skip_next = False
|
||||
continue
|
||||
if arg in sync_args:
|
||||
filtered.append(arg)
|
||||
# Add the value for the argument if it exists
|
||||
if i + 1 < len(args) and not args[i + 1].startswith('--'):
|
||||
filtered.append(args[i + 1])
|
||||
skip_next = True
|
||||
return filtered
|
||||
|
||||
return args
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description='Main script to manage Palmr translations',
|
||||
epilog='Examples:\n'
|
||||
' python3 run_translations.py check\n'
|
||||
' python3 run_translations.py sync --dry-run\n'
|
||||
' python3 run_translations.py translate --delay 2.0\n'
|
||||
' python3 run_translations.py all # Complete workflow\n',
|
||||
' python3 run_translations.py all --dry-run\n',
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter
|
||||
)
|
||||
|
||||
parser.add_argument(
|
||||
'command',
|
||||
choices=['check', 'sync', 'translate', 'all', 'help'],
|
||||
choices=['check', 'sync', 'all', 'help'],
|
||||
help='Command to execute:\n'
|
||||
'check - Check translation status\n'
|
||||
'sync - Synchronize missing keys\n'
|
||||
'translate - Automatically translate strings\n'
|
||||
'all - Run complete workflow\n'
|
||||
'all - Run complete workflow (sync + check)\n'
|
||||
'help - Show detailed help'
|
||||
)
|
||||
|
||||
@@ -57,13 +97,7 @@ def main():
|
||||
print(" python3 run_translations.py sync --dry-run")
|
||||
print(" python3 run_translations.py sync --no-mark-untranslated")
|
||||
print()
|
||||
print("🌐 translate - Automatically translate")
|
||||
print(" python3 run_translations.py translate")
|
||||
print(" python3 run_translations.py translate --dry-run")
|
||||
print(" python3 run_translations.py translate --delay 2.0")
|
||||
print(" python3 run_translations.py translate --skip-languages pt-BR.json")
|
||||
print()
|
||||
print("⚡ all - Complete workflow (sync + translate)")
|
||||
print("⚡ all - Complete workflow (sync + check)")
|
||||
print(" python3 run_translations.py all")
|
||||
print(" python3 run_translations.py all --dry-run")
|
||||
print()
|
||||
@@ -72,22 +106,21 @@ def main():
|
||||
print(" apps/web/messages/ - Translation files")
|
||||
print()
|
||||
print("💡 TIPS:")
|
||||
print("• Use --dry-run on any command to test")
|
||||
print("• Use --dry-run on sync or all commands to test")
|
||||
print("• Use --help on any command for specific options")
|
||||
print("• Read https://docs.palmr.dev/docs/3.0-beta/translation-management for complete documentation")
|
||||
print("• Manually translate strings marked with [TO_TRANSLATE]")
|
||||
print("• Read documentation for complete translation guidelines")
|
||||
return 0
|
||||
|
||||
elif args.command == 'check':
|
||||
print("🔍 Checking translation status...")
|
||||
return run_command('check_translations.py', remaining_args)
|
||||
filtered_args = filter_args_for_script('check_translations.py', remaining_args)
|
||||
return run_command('check_translations.py', filtered_args)
|
||||
|
||||
elif args.command == 'sync':
|
||||
print("🔄 Synchronizing translation keys...")
|
||||
return run_command('sync_translations.py', remaining_args)
|
||||
|
||||
elif args.command == 'translate':
|
||||
print("🌐 Automatically translating strings...")
|
||||
return run_command('translate_missing.py', remaining_args)
|
||||
filtered_args = filter_args_for_script('sync_translations.py', remaining_args)
|
||||
return run_command('sync_translations.py', filtered_args)
|
||||
|
||||
elif args.command == 'all':
|
||||
print("⚡ Running complete translation workflow...")
|
||||
@@ -98,7 +131,8 @@ def main():
|
||||
|
||||
# 1. Initial check
|
||||
print("1️⃣ Checking initial status...")
|
||||
result = run_command('check_translations.py', remaining_args)
|
||||
check_args = filter_args_for_script('check_translations.py', remaining_args)
|
||||
result = run_command('check_translations.py', check_args)
|
||||
if result != 0:
|
||||
print("❌ Error in initial check")
|
||||
return result
|
||||
@@ -107,26 +141,18 @@ def main():
|
||||
|
||||
# 2. Sync
|
||||
print("2️⃣ Synchronizing missing keys...")
|
||||
result = run_command('sync_translations.py', remaining_args)
|
||||
sync_args = filter_args_for_script('sync_translations.py', remaining_args)
|
||||
result = run_command('sync_translations.py', sync_args)
|
||||
if result != 0:
|
||||
print("❌ Error in synchronization")
|
||||
return result
|
||||
|
||||
if not is_dry_run:
|
||||
print("\n" + "="*50)
|
||||
|
||||
# 3. Translate
|
||||
print("3️⃣ Automatically translating strings...")
|
||||
result = run_command('translate_missing.py', remaining_args)
|
||||
if result != 0:
|
||||
print("❌ Error in translation")
|
||||
return result
|
||||
|
||||
print("\n" + "="*50)
|
||||
|
||||
# 4. Final check
|
||||
print("4️⃣ Final check...")
|
||||
result = run_command('check_translations.py', remaining_args)
|
||||
# 3. Final check
|
||||
print("3️⃣ Final check...")
|
||||
check_args = filter_args_for_script('check_translations.py', remaining_args)
|
||||
result = run_command('check_translations.py', check_args)
|
||||
if result != 0:
|
||||
print("❌ Error in final check")
|
||||
return result
|
||||
@@ -134,6 +160,8 @@ def main():
|
||||
print("\n🎉 Complete workflow executed successfully!")
|
||||
if is_dry_run:
|
||||
print("💡 Run without --dry-run to apply changes")
|
||||
else:
|
||||
print("💡 Review strings marked with [TO_TRANSLATE] and translate them manually")
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
@@ -1,342 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Script to automatically translate strings marked with [TO_TRANSLATE]
|
||||
using free Google Translate.
|
||||
"""
|
||||
|
||||
import json
|
||||
import time
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, List, Tuple, Optional
|
||||
import argparse
|
||||
import sys
|
||||
|
||||
# Language code mapping from file names to Google Translate codes
|
||||
LANGUAGE_MAPPING = {
|
||||
'pt-BR.json': 'pt', # Portuguese (Brazil) -> Portuguese
|
||||
'es-ES.json': 'es', # Spanish (Spain) -> Spanish
|
||||
'fr-FR.json': 'fr', # French (France) -> French
|
||||
'de-DE.json': 'de', # German -> German
|
||||
'it-IT.json': 'it', # Italian -> Italian
|
||||
'ru-RU.json': 'ru', # Russian -> Russian
|
||||
'ja-JP.json': 'ja', # Japanese -> Japanese
|
||||
'ko-KR.json': 'ko', # Korean -> Korean
|
||||
'zh-CN.json': 'zh-cn', # Chinese (Simplified) -> Simplified Chinese
|
||||
'ar-SA.json': 'ar', # Arabic -> Arabic
|
||||
'hi-IN.json': 'hi', # Hindi -> Hindi
|
||||
'nl-NL.json': 'nl', # Dutch -> Dutch
|
||||
'tr-TR.json': 'tr', # Turkish -> Turkish
|
||||
'pl-PL.json': 'pl', # Polish -> Polish
|
||||
}
|
||||
|
||||
# Prefix to identify untranslated strings
|
||||
TO_TRANSLATE_PREFIX = '[TO_TRANSLATE] '
|
||||
|
||||
|
||||
def load_json_file(file_path: Path) -> Dict[str, Any]:
|
||||
"""Load a JSON file."""
|
||||
try:
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
return json.load(f)
|
||||
except Exception as e:
|
||||
print(f"❌ Error loading {file_path}: {e}")
|
||||
return {}
|
||||
|
||||
|
||||
def save_json_file(file_path: Path, data: Dict[str, Any], indent: int = 2) -> bool:
|
||||
"""Save a JSON file with consistent formatting."""
|
||||
try:
|
||||
with open(file_path, 'w', encoding='utf-8') as f:
|
||||
json.dump(data, f, ensure_ascii=False, indent=indent, separators=(',', ': '))
|
||||
f.write('\n') # Add newline at the end
|
||||
return True
|
||||
except Exception as e:
|
||||
print(f"❌ Error saving {file_path}: {e}")
|
||||
return False
|
||||
|
||||
|
||||
def get_nested_value(data: Dict[str, Any], key_path: str) -> Any:
|
||||
"""Get a nested value using a key with dots as separator."""
|
||||
keys = key_path.split('.')
|
||||
current = data
|
||||
|
||||
for key in keys:
|
||||
if isinstance(current, dict) and key in current:
|
||||
current = current[key]
|
||||
else:
|
||||
return None
|
||||
|
||||
return current
|
||||
|
||||
|
||||
def set_nested_value(data: Dict[str, Any], key_path: str, value: Any) -> None:
|
||||
"""Set a nested value using a key with dots as separator."""
|
||||
keys = key_path.split('.')
|
||||
current = data
|
||||
|
||||
# Navigate to the second-to-last level, creating dictionaries as needed
|
||||
for key in keys[:-1]:
|
||||
if key not in current:
|
||||
current[key] = {}
|
||||
elif not isinstance(current[key], dict):
|
||||
current[key] = {}
|
||||
current = current[key]
|
||||
|
||||
# Set the value at the last level
|
||||
current[keys[-1]] = value
|
||||
|
||||
|
||||
def find_untranslated_strings(data: Dict[str, Any], prefix: str = '') -> List[Tuple[str, str]]:
|
||||
"""Find all strings marked with [TO_TRANSLATE] recursively."""
|
||||
untranslated = []
|
||||
|
||||
for key, value in data.items():
|
||||
current_key = f"{prefix}.{key}" if prefix else key
|
||||
|
||||
if isinstance(value, str) and value.startswith(TO_TRANSLATE_PREFIX):
|
||||
# Remove prefix to get original text
|
||||
original_text = value[len(TO_TRANSLATE_PREFIX):].strip()
|
||||
untranslated.append((current_key, original_text))
|
||||
elif isinstance(value, dict):
|
||||
untranslated.extend(find_untranslated_strings(value, current_key))
|
||||
|
||||
return untranslated
|
||||
|
||||
|
||||
def install_googletrans():
|
||||
"""Install the googletrans library if not available."""
|
||||
try:
|
||||
import googletrans
|
||||
return True
|
||||
except ImportError:
|
||||
print("📦 'googletrans' library not found. Attempting to install...")
|
||||
import subprocess
|
||||
try:
|
||||
subprocess.check_call([sys.executable, "-m", "pip", "install", "googletrans==4.0.0rc1"])
|
||||
print("✅ googletrans installed successfully!")
|
||||
return True
|
||||
except subprocess.CalledProcessError:
|
||||
print("❌ Failed to install googletrans. Install manually with:")
|
||||
print("pip install googletrans==4.0.0rc1")
|
||||
return False
|
||||
|
||||
|
||||
def translate_text(text: str, target_language: str, max_retries: int = 3) -> Optional[str]:
|
||||
"""Translate text using free Google Translate."""
|
||||
try:
|
||||
from googletrans import Translator
|
||||
|
||||
translator = Translator()
|
||||
|
||||
for attempt in range(max_retries):
|
||||
try:
|
||||
# Translate from English to target language
|
||||
result = translator.translate(text, src='en', dest=target_language)
|
||||
|
||||
if result and result.text:
|
||||
return result.text.strip()
|
||||
|
||||
except Exception as e:
|
||||
if attempt < max_retries - 1:
|
||||
print(f" ⚠️ Attempt {attempt + 1} failed: {str(e)[:50]}... Retrying in 2s...")
|
||||
time.sleep(2)
|
||||
else:
|
||||
print(f" ❌ Failed after {max_retries} attempts: {str(e)[:50]}...")
|
||||
|
||||
return None
|
||||
|
||||
except ImportError:
|
||||
print("❌ googletrans library not available")
|
||||
return None
|
||||
|
||||
|
||||
def translate_file(file_path: Path, target_language: str, dry_run: bool = False,
|
||||
delay_between_requests: float = 1.0) -> Tuple[int, int, int]:
|
||||
"""
|
||||
Translate all [TO_TRANSLATE] strings in a file.
|
||||
Returns: (total_found, successful_translations, failed_translations)
|
||||
"""
|
||||
print(f"🔍 Processing: {file_path.name}")
|
||||
|
||||
# Load file
|
||||
data = load_json_file(file_path)
|
||||
if not data:
|
||||
return 0, 0, 0
|
||||
|
||||
# Find untranslated strings
|
||||
untranslated_strings = find_untranslated_strings(data)
|
||||
|
||||
if not untranslated_strings:
|
||||
print(f" ✅ No strings to translate")
|
||||
return 0, 0, 0
|
||||
|
||||
print(f" 📝 Found {len(untranslated_strings)} strings to translate")
|
||||
|
||||
if dry_run:
|
||||
print(f" 🔍 [DRY RUN] Strings that would be translated:")
|
||||
for key, text in untranslated_strings[:3]:
|
||||
print(f" - {key}: \"{text[:50]}{'...' if len(text) > 50 else ''}\"")
|
||||
if len(untranslated_strings) > 3:
|
||||
print(f" ... and {len(untranslated_strings) - 3} more")
|
||||
return len(untranslated_strings), 0, 0
|
||||
|
||||
# Translate each string
|
||||
successful = 0
|
||||
failed = 0
|
||||
updated_data = data.copy()
|
||||
|
||||
for i, (key_path, original_text) in enumerate(untranslated_strings, 1):
|
||||
print(f" 📍 ({i}/{len(untranslated_strings)}) Translating: {key_path}")
|
||||
|
||||
# Translate text
|
||||
translated_text = translate_text(original_text, target_language)
|
||||
|
||||
if translated_text and translated_text != original_text:
|
||||
# Update in dictionary
|
||||
set_nested_value(updated_data, key_path, translated_text)
|
||||
successful += 1
|
||||
print(f" ✅ \"{original_text[:30]}...\" → \"{translated_text[:30]}...\"")
|
||||
else:
|
||||
failed += 1
|
||||
print(f" ❌ Translation failed")
|
||||
|
||||
# Delay between requests to avoid rate limiting
|
||||
if i < len(untranslated_strings): # Don't wait after the last one
|
||||
time.sleep(delay_between_requests)
|
||||
|
||||
# Save updated file
|
||||
if successful > 0:
|
||||
if save_json_file(file_path, updated_data):
|
||||
print(f" 💾 File saved with {successful} translations")
|
||||
else:
|
||||
print(f" ❌ Error saving file")
|
||||
failed += successful # Count as failure if couldn't save
|
||||
successful = 0
|
||||
|
||||
return len(untranslated_strings), successful, failed
|
||||
|
||||
|
||||
def translate_all_files(messages_dir: Path, delay_between_requests: float = 1.0,
|
||||
dry_run: bool = False, skip_languages: List[str] = None) -> None:
|
||||
"""Translate all language files that have [TO_TRANSLATE] strings."""
|
||||
|
||||
if not install_googletrans():
|
||||
return
|
||||
|
||||
skip_languages = skip_languages or []
|
||||
|
||||
# Find language JSON files
|
||||
language_files = []
|
||||
for file_name, lang_code in LANGUAGE_MAPPING.items():
|
||||
file_path = messages_dir / file_name
|
||||
if file_path.exists() and file_name not in skip_languages:
|
||||
language_files.append((file_path, lang_code))
|
||||
|
||||
if not language_files:
|
||||
print("❌ No language files found")
|
||||
return
|
||||
|
||||
print(f"🌍 Translating {len(language_files)} languages...")
|
||||
print(f"⏱️ Delay between requests: {delay_between_requests}s")
|
||||
if dry_run:
|
||||
print("🔍 DRY RUN MODE - No changes will be made")
|
||||
print("-" * 60)
|
||||
|
||||
total_found = 0
|
||||
total_successful = 0
|
||||
total_failed = 0
|
||||
|
||||
for i, (file_path, lang_code) in enumerate(language_files, 1):
|
||||
print(f"\n[{i}/{len(language_files)}] 🌐 Language: {lang_code.upper()}")
|
||||
|
||||
found, successful, failed = translate_file(
|
||||
file_path, lang_code, dry_run, delay_between_requests
|
||||
)
|
||||
|
||||
total_found += found
|
||||
total_successful += successful
|
||||
total_failed += failed
|
||||
|
||||
# Pause between files (except the last one)
|
||||
if i < len(language_files) and not dry_run:
|
||||
print(f" ⏸️ Pausing {delay_between_requests * 2}s before next language...")
|
||||
time.sleep(delay_between_requests * 2)
|
||||
|
||||
# Final summary
|
||||
print("\n" + "=" * 60)
|
||||
print("📊 FINAL SUMMARY")
|
||||
print("=" * 60)
|
||||
|
||||
if dry_run:
|
||||
print(f"🔍 DRY RUN MODE:")
|
||||
print(f" • {total_found} strings would be translated")
|
||||
else:
|
||||
print(f"✅ Translations performed:")
|
||||
print(f" • {total_successful} successes")
|
||||
print(f" • {total_failed} failures")
|
||||
print(f" • {total_found} total processed")
|
||||
|
||||
if total_successful > 0:
|
||||
success_rate = (total_successful / total_found) * 100
|
||||
print(f" • Success rate: {success_rate:.1f}%")
|
||||
|
||||
print("\n💡 TIPS:")
|
||||
print("• Run 'python3 check_translations.py' to verify results")
|
||||
print("• Strings that failed translation keep the [TO_TRANSLATE] prefix")
|
||||
print("• Consider reviewing automatic translations to ensure quality")
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description='Automatically translate strings marked with [TO_TRANSLATE]'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--messages-dir',
|
||||
type=Path,
|
||||
default=Path(__file__).parent.parent / 'messages',
|
||||
help='Directory containing message files (default: ../messages)'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--dry-run',
|
||||
action='store_true',
|
||||
help='Only show what would be translated without making changes'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--delay',
|
||||
type=float,
|
||||
default=1.0,
|
||||
help='Delay in seconds between translation requests (default: 1.0)'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--skip-languages',
|
||||
nargs='*',
|
||||
default=[],
|
||||
help='List of languages to skip (ex: pt-BR.json fr-FR.json)'
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if not args.messages_dir.exists():
|
||||
print(f"❌ Directory not found: {args.messages_dir}")
|
||||
return 1
|
||||
|
||||
print(f"📁 Directory: {args.messages_dir}")
|
||||
print(f"🔍 Dry run: {args.dry_run}")
|
||||
print(f"⏱️ Delay: {args.delay}s")
|
||||
if args.skip_languages:
|
||||
print(f"⏭️ Skipping: {', '.join(args.skip_languages)}")
|
||||
print("-" * 60)
|
||||
|
||||
translate_all_files(
|
||||
messages_dir=args.messages_dir,
|
||||
delay_between_requests=args.delay,
|
||||
dry_run=args.dry_run,
|
||||
skip_languages=args.skip_languages
|
||||
)
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
exit(main())
|
||||
@@ -23,7 +23,7 @@ docker buildx build \
|
||||
--no-cache \
|
||||
-t kyantech/palmr:latest \
|
||||
-t kyantech/palmr:$TAG \
|
||||
--load \
|
||||
--push \
|
||||
.
|
||||
|
||||
if [ $? -eq 0 ]; then
|
||||
|
||||
Reference in New Issue
Block a user