The AWS Deep Learning AMIs for Ubuntu and Amazon Linux now come pre-installed and fully configured with Horovod, a popular open source distributed training framework to scale TensorFlow training on multiple GPUs.

Source: Amazon Web Services News AWS Deep Learning AMIs Now Include Horovod for Faster Multi-GPU TensorFlow Training on Amazon EC2 P3 Instances