Pure CPU speed differences for parallel processing tests on the same hardware (tested using threads running ) were observable with Ubuntu vs Amazon Linux when I tested, but Amazon Linux was only ~4% faster. Much later edit: I never got around to properly testing all of the isolated components that could've affected this, but as per discussion in the thread, it seems clear that the actual source of the huge difference was different ImageMagick builds with different options in the distro packages. But for this project, clearly, that extra setup work is worth it. Ubuntu's generally a lot easier to get workloads up and running on. Of course, this all comes with a bunch of pains in the ass due to Amazon Linux not having the same package availability, having to build things from source by hand, etc. I'm not a Linux kernel guy and I have no idea exactly what AWS has done here (it must have something to do with thread context switching) but, holy crap.

Speed on Amazon Linux 2: ~180,000 images per hour Speed on Ubuntu 20.04: ~30,000 images per hour Speed on Windows Server 2019: ~70,000 images per hour

Task: Run ImageMagick in parallel (restrict each instance of ImageMagick to one thread and run many of them at once) to do a set of transformations (resizing, watermarking, compression quality adjustment, etc) for online publishing on large (20k - 60k per task) quantities of jpeg files.Īfter porting the Windows orchestration program that does this to run on Linux, I did some speed testing on c5ad.16xlarge EC2 instances with 64 processing threads and a representative input set (with I/O to a local NVME SSD). Just thought I'd share an interesting result from something I'm working on right now.

Ec2 linux imagemagick