Create docker/readme.md
Browse files- docker/readme.md +316 -0
docker/readme.md
ADDED
@@ -0,0 +1,316 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# BackgroundFX Pro Docker Deployment
|
2 |
+
|
3 |
+
This directory contains Docker configurations for deploying BackgroundFX Pro in various environments.
|
4 |
+
|
5 |
+
## Quick Start
|
6 |
+
|
7 |
+
### Prerequisites
|
8 |
+
|
9 |
+
- Docker 20.10+ installed
|
10 |
+
- Docker Compose 2.0+ installed
|
11 |
+
- NVIDIA Docker runtime (for GPU support)
|
12 |
+
- At least 16GB RAM
|
13 |
+
- 20GB+ free disk space
|
14 |
+
|
15 |
+
### GPU Deployment (Recommended)
|
16 |
+
|
17 |
+
```bash
|
18 |
+
# Build and run with GPU support
|
19 |
+
make build
|
20 |
+
make run
|
21 |
+
|
22 |
+
# Or using docker-compose directly
|
23 |
+
docker-compose up -d backgroundfx-gpu
|
24 |
+
```
|
25 |
+
|
26 |
+
Access the application at http://localhost:7860
|
27 |
+
|
28 |
+
### CPU-Only Deployment
|
29 |
+
|
30 |
+
```bash
|
31 |
+
# Build and run CPU version
|
32 |
+
make build-cpu
|
33 |
+
make run-cpu
|
34 |
+
|
35 |
+
# Or using docker-compose
|
36 |
+
docker-compose --profile cpu up -d backgroundfx-cpu
|
37 |
+
```
|
38 |
+
|
39 |
+
Access at http://localhost:7861
|
40 |
+
|
41 |
+
## Available Docker Images
|
42 |
+
|
43 |
+
### 1. GPU Image (Dockerfile)
|
44 |
+
- **Base**: nvidia/cuda:12.1.0-cudnn8-runtime-ubuntu20.04
|
45 |
+
- **Features**: Full GPU acceleration, all models supported
|
46 |
+
- **Use case**: Production deployment with NVIDIA GPU
|
47 |
+
|
48 |
+
### 2. Production Image (Dockerfile.prod)
|
49 |
+
- **Type**: Multi-stage optimized build
|
50 |
+
- **Features**: Minimal size, pre-compiled Python, security hardening
|
51 |
+
- **Use case**: Production deployment with high performance requirements
|
52 |
+
|
53 |
+
### 3. CPU Image (Dockerfile.cpu)
|
54 |
+
- **Base**: python:3.10-slim
|
55 |
+
- **Features**: CPU-optimized, smaller footprint
|
56 |
+
- **Use case**: Development, testing, or CPU-only servers
|
57 |
+
|
58 |
+
## Docker Compose Services
|
59 |
+
|
60 |
+
### Core Services
|
61 |
+
|
62 |
+
- **backgroundfx-gpu**: Main application with GPU support
|
63 |
+
- **backgroundfx-cpu**: CPU-only variant
|
64 |
+
- **redis**: Cache and job queue
|
65 |
+
- **nginx**: Reverse proxy (production profile)
|
66 |
+
|
67 |
+
### Support Services
|
68 |
+
|
69 |
+
- **model-downloader**: Pre-download models (setup profile)
|
70 |
+
- **prometheus**: Metrics collection (monitoring profile)
|
71 |
+
- **grafana**: Metrics visualization (monitoring profile)
|
72 |
+
|
73 |
+
## Configuration
|
74 |
+
|
75 |
+
### Environment Variables
|
76 |
+
|
77 |
+
Copy the example environment file and customize:
|
78 |
+
|
79 |
+
```bash
|
80 |
+
cp docker/.env.example docker/.env
|
81 |
+
```
|
82 |
+
|
83 |
+
Key settings:
|
84 |
+
- `DEVICE`: auto, cuda, or cpu
|
85 |
+
- `MODEL_CACHE_DIR`: Model storage location
|
86 |
+
- `MAX_MEMORY_GB`: Memory limit
|
87 |
+
- `QUALITY_PRESET`: low, medium, high, ultra
|
88 |
+
|
89 |
+
### Volumes
|
90 |
+
|
91 |
+
- `model-cache`: Cached ML models (~2-5GB)
|
92 |
+
- `uploads`: Uploaded files
|
93 |
+
- `outputs`: Processed results
|
94 |
+
- `redis-data`: Redis persistence
|
95 |
+
|
96 |
+
## Makefile Commands
|
97 |
+
|
98 |
+
```bash
|
99 |
+
# Building
|
100 |
+
make build # Build GPU image
|
101 |
+
make build-cpu # Build CPU image
|
102 |
+
make build-all # Build all variants
|
103 |
+
make build-nocache # Build without cache
|
104 |
+
|
105 |
+
# Running
|
106 |
+
make run # Run GPU version
|
107 |
+
make run-cpu # Run CPU version
|
108 |
+
make run-dev # Development mode
|
109 |
+
make run-prod # Production with monitoring
|
110 |
+
|
111 |
+
# Management
|
112 |
+
make stop # Stop containers
|
113 |
+
make restart # Restart containers
|
114 |
+
make clean # Clean up
|
115 |
+
make logs # View logs
|
116 |
+
make shell # Container shell
|
117 |
+
|
118 |
+
# Models
|
119 |
+
make download-models # Download all models
|
120 |
+
make list-models # List available models
|
121 |
+
|
122 |
+
# Monitoring
|
123 |
+
make status # Container status
|
124 |
+
make stats # Resource usage
|
125 |
+
make health # Health checks
|
126 |
+
```
|
127 |
+
|
128 |
+
## Production Deployment
|
129 |
+
|
130 |
+
### 1. Build Production Image
|
131 |
+
|
132 |
+
```bash
|
133 |
+
make build-prod
|
134 |
+
```
|
135 |
+
|
136 |
+
### 2. Configure Environment
|
137 |
+
|
138 |
+
Edit `docker/.env` with production settings:
|
139 |
+
- Set secure `AUTH_SECRET_KEY`
|
140 |
+
- Configure `REDIS_PASSWORD`
|
141 |
+
- Adjust resource limits
|
142 |
+
- Enable authentication if needed
|
143 |
+
|
144 |
+
### 3. Deploy with Monitoring
|
145 |
+
|
146 |
+
```bash
|
147 |
+
make run-prod
|
148 |
+
```
|
149 |
+
|
150 |
+
This starts:
|
151 |
+
- Main application (port 7860)
|
152 |
+
- API server (port 8000)
|
153 |
+
- Nginx proxy (ports 80/443)
|
154 |
+
- Redis cache
|
155 |
+
- Prometheus + Grafana monitoring
|
156 |
+
|
157 |
+
### 4. SSL/TLS Setup
|
158 |
+
|
159 |
+
Place certificates in `nginx/ssl/`:
|
160 |
+
- `nginx/ssl/cert.pem`
|
161 |
+
- `nginx/ssl/key.pem`
|
162 |
+
|
163 |
+
### 5. Scaling
|
164 |
+
|
165 |
+
For horizontal scaling:
|
166 |
+
|
167 |
+
```bash
|
168 |
+
docker-compose up -d --scale backgroundfx-gpu=3
|
169 |
+
```
|
170 |
+
|
171 |
+
## GPU Support
|
172 |
+
|
173 |
+
### Check GPU Availability
|
174 |
+
|
175 |
+
```bash
|
176 |
+
make gpu-check
|
177 |
+
# Or
|
178 |
+
docker run --rm --gpus all nvidia/cuda:12.1.0-base-ubuntu20.04 nvidia-smi
|
179 |
+
```
|
180 |
+
|
181 |
+
### Install NVIDIA Docker Runtime
|
182 |
+
|
183 |
+
Ubuntu/Debian:
|
184 |
+
```bash
|
185 |
+
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
|
186 |
+
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
|
187 |
+
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
|
188 |
+
sudo apt-get update && sudo apt-get install -y nvidia-docker2
|
189 |
+
sudo systemctl restart docker
|
190 |
+
```
|
191 |
+
|
192 |
+
## Troubleshooting
|
193 |
+
|
194 |
+
### Out of Memory
|
195 |
+
|
196 |
+
Adjust memory limits in `docker-compose.yml`:
|
197 |
+
```yaml
|
198 |
+
deploy:
|
199 |
+
resources:
|
200 |
+
limits:
|
201 |
+
memory: 8G # Reduce if needed
|
202 |
+
```
|
203 |
+
|
204 |
+
### Models Not Loading
|
205 |
+
|
206 |
+
Pre-download models:
|
207 |
+
```bash
|
208 |
+
make download-models
|
209 |
+
```
|
210 |
+
|
211 |
+
### Permission Issues
|
212 |
+
|
213 |
+
Fix ownership:
|
214 |
+
```bash
|
215 |
+
docker-compose exec -u root backgroundfx-gpu chown -R appuser:appuser /app
|
216 |
+
```
|
217 |
+
|
218 |
+
### Slow Processing
|
219 |
+
|
220 |
+
- Ensure GPU is detected: `make gpu-check`
|
221 |
+
- Check resource usage: `make stats`
|
222 |
+
- Adjust quality preset in `.env`
|
223 |
+
|
224 |
+
## Development
|
225 |
+
|
226 |
+
### Local Development with Docker
|
227 |
+
|
228 |
+
```bash
|
229 |
+
# Mount local code for live reload
|
230 |
+
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up
|
231 |
+
```
|
232 |
+
|
233 |
+
### Running Tests
|
234 |
+
|
235 |
+
```bash
|
236 |
+
make test
|
237 |
+
```
|
238 |
+
|
239 |
+
### Building Custom Images
|
240 |
+
|
241 |
+
```bash
|
242 |
+
# With custom registry
|
243 |
+
make push REGISTRY=myregistry.com VERSION=1.0.0
|
244 |
+
```
|
245 |
+
|
246 |
+
## Monitoring
|
247 |
+
|
248 |
+
### Grafana Dashboard
|
249 |
+
|
250 |
+
Access at http://localhost:3000 (admin/admin)
|
251 |
+
|
252 |
+
Pre-configured dashboards:
|
253 |
+
- Container metrics
|
254 |
+
- GPU utilization
|
255 |
+
- Processing statistics
|
256 |
+
- Error rates
|
257 |
+
|
258 |
+
### Prometheus Metrics
|
259 |
+
|
260 |
+
Access at http://localhost:9090
|
261 |
+
|
262 |
+
Available metrics:
|
263 |
+
- `processing_time_seconds`
|
264 |
+
- `frames_processed_total`
|
265 |
+
- `model_load_time_seconds`
|
266 |
+
- `memory_usage_bytes`
|
267 |
+
|
268 |
+
## Backup and Restore
|
269 |
+
|
270 |
+
### Backup Volumes
|
271 |
+
|
272 |
+
```bash
|
273 |
+
make backup
|
274 |
+
```
|
275 |
+
|
276 |
+
Creates timestamped backups in `./backups/`
|
277 |
+
|
278 |
+
### Restore from Backup
|
279 |
+
|
280 |
+
```bash
|
281 |
+
make restore BACKUP_FILE=models-20240101-120000.tar.gz
|
282 |
+
```
|
283 |
+
|
284 |
+
## Security Considerations
|
285 |
+
|
286 |
+
1. **Change default passwords** in production
|
287 |
+
2. **Enable authentication** via AUTH_ENABLED=true
|
288 |
+
3. **Configure CORS** appropriately
|
289 |
+
4. **Use SSL/TLS** in production
|
290 |
+
5. **Limit exposed ports** using firewall rules
|
291 |
+
6. **Regular security updates**: `docker pull` base images
|
292 |
+
|
293 |
+
## Performance Optimization
|
294 |
+
|
295 |
+
### GPU Optimization
|
296 |
+
- Use TensorRT models when available
|
297 |
+
- Enable mixed precision with FP16
|
298 |
+
- Adjust batch size based on GPU memory
|
299 |
+
|
300 |
+
### CPU Optimization
|
301 |
+
- Use quantized models
|
302 |
+
- Enable OpenMP threading
|
303 |
+
- Adjust worker count based on cores
|
304 |
+
|
305 |
+
### Memory Optimization
|
306 |
+
- Enable swap for large videos
|
307 |
+
- Use frame skipping for preview
|
308 |
+
- Implement progressive processing
|
309 |
+
|
310 |
+
## Support
|
311 |
+
|
312 |
+
For issues or questions:
|
313 |
+
1. Check logs: `make logs`
|
314 |
+
2. Verify health: `make health`
|
315 |
+
3. Review configuration: `docker-compose config`
|
316 |
+
4. Check system requirements: `make gpu-check`
|