Video transcoding pipeline
- Segment
- Media infra
- Role
- Built with one collaborator (15 of 16 commits mine)
- Period
- 11.2024 – 01.2025
An event-driven video transcoding service on AWS. When a video is uploaded to S3, a message lands in a queue, a worker picks it up and starts a container that transcodes the video with FFmpeg and writes the result to an output bucket.
I designed it for cost: the first version launched each job on ECS Fargate, and the second runs the container directly on the EC2 host that reads the queue, which cut infrastructure spend by 60% compared with ECS.
Key features
- Upload-triggered processing. S3 upload events flow into SQS, and a worker picks up one job at a time, skipping S3's setup test events.
- Two ways to run jobs. Each job can run as an ECS Fargate task or as a Docker container on the worker's own EC2 host. The EC2 option is the cheaper default.
- FFmpeg transcoder container. It downloads the source video, transcodes it to H.264 and AAC, uploads the result, and cleans up after itself.
- Automated image builds. GitHub Actions builds the transcoder image on every push, publishes it to the container registry, and verifies the push.
- GPU batch compression. Scripts compress whole folders of videos on an NVIDIA GPU (HEVC NVENC), keeping original timestamps and metadata.
Tech stack
- Backend
- TypeScript on Bun (worker), Node.js (transcoder), FFmpeg, fluent-ffmpeg, dockerode
- Cloud
- AWS SQS, S3, EC2, ECS Fargate, ECR, AWS SDK v3
- Infra
- Docker, Docker Hub, GitHub Actions
- Other
- Bash, NVIDIA NVENC