Process images and media after an Amazon S3 upload
Take a presigned Amazon S3 upload through EventBridge and SQS to Lambda, Fargate, or AWS Batch workers, write derived assets to a separate bucket, and serve them through CloudFront.
Official AWS sources reviewed 2026-08-29.
Architecture flow
Accept the upload: The client PUTs the original to an originals bucket with a presigned URL, so bytes never pass through the API tier.
Emit the event: S3 sends object-created events to EventBridge, or straight to a queue when no content routing is needed.
Buffer and control concurrency: A queue absorbs upload bursts, caps worker concurrency, and gives retries and a dead-letter queue a place to live.
Transform: Lambda handles bounded transforms; ECS on Fargate or AWS Batch takes long, large, or dependency-heavy jobs.
Analyze when it earns its place: Rekognition adds smart cropping, moderation, and labels when the product needs them.
Store and serve: Derived assets land in a separate bucket and CloudFront serves them with a cache key you control.
Text alternative: A client uploads an original file to an Amazon S3 originals bucket using a presigned URL. S3 publishes the object-created event to Amazon EventBridge, which routes it to an Amazon SQS queue. Workers on AWS Lambda for short transforms, or Amazon ECS on AWS Fargate and AWS Batch for heavy jobs, read the queue, optionally call Amazon Rekognition, and write derived assets to a separate Amazon S3 bucket that Amazon CloudFront serves.
How the services connect
Keep originals and derived assets in different buckets. A worker that writes its output back into the bucket that triggered it creates a recursive invocation loop, and separate buckets also let you apply different lifecycle, access, and cache rules to source material and to output.
Direct S3-to-Lambda invocation is enough when the transform is short, the burst is modest, and a retry is harmless. Put a queue in the path when you need burst absorption, a concurrency ceiling that protects a downstream database or API, controlled retries, or a dead-letter queue that keeps the failed work instead of dropping it.
Choose the worker by the job, not by preference. Lambda fits transforms that finish inside its timeout and memory. ECS on Fargate suits long-running or native-dependency-heavy processing that should stay warm. AWS Batch fits queue-driven bulk jobs and backfills where throughput matters more than latency.
Eager generation on upload gives predictable read latency and a fixed set of variants, and it makes adding a variant later a backfill job over the whole corpus. On-demand transformation behind CloudFront generates only what is requested, at the cost of a slower first request and a cache key you have to constrain. AWS ships the on-demand path as Dynamic Image Transformation for Amazon CloudFront, the current name for Serverless Image Handler.
Make the transform idempotent with a deterministic output key derived from the source version ID and the transform parameters. Duplicate events then overwrite identical bytes instead of producing duplicate work with different names, and a replay after an outage is safe.
Constrain the accepted transform parameters to a fixed list. An open resize API lets anyone fill the derived bucket with arbitrary variants and pay for the compute that made them.
Tradeoffs and caveats
A queue between S3 and the worker buys backpressure, retry control, and a dead-letter queue at the cost of one more hop and a small delay.
Eager generation makes reads fast and predictable; on-demand generation makes storage small and new variants free, and moves the cost into the cache-miss path.
Rekognition, Fargate, Batch, CloudFront requests, and derived-asset storage each bill separately, and pre-generating variants nobody requests is the most common wasted spend in this pattern.
Decision points
Direct Lambda invocation versus an SQS buffer
Option
Choose when
Retry behavior
Cost of the choice
S3 event notification straight to Lambda
Transforms are short, traffic is smooth, and a lost retry is tolerable.
Lambda's asynchronous retries, then a Lambda dead-letter or on-failure destination.
Fewest moving parts, least control over concurrency and backpressure.
S3 or EventBridge to SQS to a Lambda worker
Bursts are spiky, or a downstream store needs a concurrency ceiling.
Visibility-timeout redelivery, partial batch responses, and an SQS dead-letter queue with redrive.
One more hop, a visibility timeout to tune, and queue request charges.
S3 or EventBridge to SQS to ECS or Batch workers
Jobs are long, large, or need native dependencies and GPUs.
Your worker controls acknowledgement; the queue redelivers what is not deleted.
Cluster or job-queue operations and slower cold-start behavior.
Eager generation versus on-demand transformation
Approach
First read latency
Adding a new variant
Main risk
Eager generation on upload
Fast: the variant already exists.
A backfill job across every existing object.
Paying storage and compute for variants nobody requests.
On-demand behind CloudFront
Slow on the first request, cached afterwards.
Free: the next request generates it.
Unbounded parameters inflating the cache and the derived bucket unless the accepted set is fixed.
Hybrid: eager for the primary size, on-demand for the rest
Fast for the common path.
Backfill only the primary size.
Two code paths that must produce identical output for the same inputs.
How companies use this
Outcomes below are attributed to their sources, not independently measured. Sources reviewed 2026-08-29.
LEVELS, a social network with payments, accepted user photos from modern phones that could exceed 8000 by 5000 pixels and several megabytes, and needed smaller variants to cut bandwidth and speed up in-app delivery.
Before
The team considered resizing on the API servers and resizing as a background job on EC2. Toptal reports that API-server processing exceeded two seconds per image and that ImageMagick used up to 1.5 GB of memory, which would have meant over-provisioning instances for upload spikes and paying through the cool-down after each one.
After
Uploads land in S3, an SNS topic triggers two Lambda functions, one producing six image variations and one making a backup copy, across three buckets for originals, processed images, and backups. CloudFront serves the originals immediately by path prefix while the variants appear a few seconds later.
Service bundle
Amazon S3, Amazon SNS, AWS Lambda, and Amazon CloudFront. No queue sits in the path; SNS provides the fan-out.
Disclosed scale
Toptal reports processing roughly 300,000 images totalling more than 8 GB for $27.50, which the post works out to under $0.0001 per image.
Failure modes
The design accepts a window where the original is served and the optimized variants do not exist yet. The post also reports that adding a new variant later meant reprocessing every existing image.
Reported outcome
Toptal reports that the serverless design removed the scaling problem for stochastic upload spikes and cost less than holding EC2 capacity for the peak.
What generalizes
Serving the original from the CDN immediately while variants generate asynchronously, and treating a new variant as a corpus-wide backfill, apply to any eager-generation pipeline.
What does not generalize
The cost figures are one workload at one point in time and are not a benchmark. The post predates Dynamic Image Transformation for Amazon CloudFront, so an on-demand design today has a supported path this one did not have. Its SNS fan-out also gives less concurrency control than a queue would.
Delivery semantics and failure handling
S3 event notifications are at-least-once and unordered, so the same upload can trigger two transforms and a delete event can arrive before the create it follows.
Derive the output key from the source version ID and the transform parameters; duplicate work then writes identical bytes to the same key.
Never write derived assets into the bucket that triggers the function, and if you must, scope the trigger to a prefix that the output does not use.
Return partial batch failures from a queue-driven Lambda so one bad object does not send an entire batch back for redelivery.
Give the queue a dead-letter queue and treat poison objects, such as truncated or hostile files, as an expected class of message rather than an incident.
Cap concurrency at the event source when the worker calls Rekognition or a database that cannot absorb the burst.
Operational signals to watch
Queue depth and ApproximateAgeOfOldestMessage, which is the signal that says work is falling behind.
Dead-letter queue count, and a sample of what lands there.
Lambda errors, throttles, duration against the timeout, and concurrent executions.
Time from object creation to derived-asset availability, measured per variant.
CloudFront cache hit rate and origin error rate for the derived bucket.
Rekognition throttling and error rates when analysis is in the path.
Cost drivers and quota pressure
Compute per transform: Lambda memory and duration, or Fargate and Batch task time for heavy jobs.
Storage and requests for every derived variant, which multiplies with the number of variants you generate eagerly.
CloudFront requests and data transfer, plus origin requests on every cache miss.
Rekognition per-image charges when moderation or smart cropping runs on everything rather than on the subset that needs it.
Quota pressure: Lambda account concurrency, Rekognition transactions per second, and the S3 request rate per prefix on hot key patterns.
AWS Periodic Table recommendations
Start with S3 to Lambda, and add a queue when you can name the pressure it relieves: burst absorption, a concurrency ceiling, retry control, or a dead-letter queue.
Keep originals immutable and treat every derived asset as reproducible; a corpus you can regenerate turns a bad transform into a backfill instead of data loss.
Fix the accepted variant list in code and reject anything else at the edge.
Version the transform, so a change in output can be rolled forward deliberately rather than discovered through inconsistent cached assets.