Deploy a Shiny App on AWS

code
R
Reproduce
Author

Zhenglei Gao

Published

December 3, 2024

Update on 17-07-2025

Modified based on answers from MGA.

Setting up a Shiny app on an AWS EC2 instance using an AWS account is a great way to deploy a scalable and accessible R-based web application. Below, I’ll outline a step-by-step approach to achieve this, considering best practices for security, scalability, and cost-efficiency.

Troubleshooting

In the begining, I got timeout error from SSH, this is because my AWS account miss the transit gateway to connect to company intranet. I do have private subnet and public subnet, in principle, it is possible to exposure my app to general public from the internet.

Approach 1: Simple Setup (Single EC2 Instance with Shiny Server)

This approach is ideal for small to medium-sized apps or for testing purposes. It involves setting up a single EC2 instance with Shiny Server to host your app.

Steps:

  1. Launch an EC2 Instance
    • Log in to the AWS Management Console.
    • Navigate to the EC2 Dashboard and click “Launch Instance.”
    • Choose an Amazon Machine Image (AMI): Select a Linux-based AMI like “Ubuntu Server 20.04 LTS” (free tier eligible if available).
    • Instance Type: Start with a t2.micro (free tier) or t3.small for better performance, depending on your app’s requirements.
    • Configure Security Group: Allow inbound traffic on ports 22 (SSH), 80 (HTTP), and 3838 (default Shiny Server port). Restrict access to specific IP ranges if needed for security.
    • Storage: Default storage (8-30 GB) is usually sufficient for a basic setup.
    • Key Pair: Create or use an existing key pair to SSH into the instance.
    • Launch the instance and note its public IP or DNS.
  2. Connect to the EC2 Instance

    • Use SSH to connect to your instance using the key pair and public IP:

      ssh -i your-key-pair.pem ubuntu@<public-ip-address>
  3. Install R and Shiny Server

    • Update the package list:

      sudo apt update

    • Install R:

      sudo apt install r-base

    • Install necessary R packages for Shiny:

      sudo su - -c "R -e \"install.packages('shiny', repos='https://cran.rstudio.com/')\""

    • Install Shiny Server (open-source version):

      sudo apt install gdebi-core
wget https://download3.rstudio.org/ubuntu-18.04/x86_64/shiny-server-1.5.21.1012-amd64.deb
sudo gdebi shiny-server-1.5.21.1012-amd64.deb

    • Verify Shiny Server is running:

      sudo systemctl status shiny-server
  4. Deploy Your Shiny App

    • Copy your Shiny app files to the instance using scp or a tool like FileZilla:

      scp -i your-key-pair.pem app.R ubuntu@<public-ip-address>:/srv/shiny-server/

    • Ensure the app directory or file has the correct permissions:

      sudo chown -R shiny:shiny /srv/shiny-server/

    • Access your app via a browser at http://<public-ip-address>:3838/ or the specific app path if customized.

  5. Optional: Set Up a Custom Domain (Route 53)
    • If your company uses AWS Route 53 for DNS management, map a domain or subdomain to the EC2 instance’s public IP or Elastic IP.
    • Request an Elastic IP from AWS and associate it with your instance to avoid IP changes on reboot.
  6. Security Considerations

    • Restrict SSH access to specific IPs in the security group.

    • Use HTTPS by setting up a reverse proxy with Nginx or Apache and obtaining an SSL certificate via AWS Certificate Manager or Let’s Encrypt.

    • Regularly update the OS and software:

      sudo apt upgrade

Approach 2: Scalable Setup (EC2 with Auto Scaling and Load Balancer)

This approach is suitable for production environments or apps with high traffic. It uses an Application Load Balancer (ALB) and Auto Scaling to manage multiple EC2 instances running Shiny Server.

Steps:

  1. Create an AMI for Shiny Server
    • Follow the steps in Approach 1 to set up a single EC2 instance with Shiny Server and your app.
    • Once configured, create a custom AMI from this instance:
      • In the EC2 Dashboard, select the instance, click “Actions” > “Image and Templates” > “Create Image.”
      • Use this AMI for launching additional instances.
  2. Set Up an Auto Scaling Group
    • Navigate to the EC2 Dashboard > “Auto Scaling Groups.”
    • Create a new Auto Scaling Group using the custom AMI.
    • Define launch templates with the desired instance type and configuration.
    • Set scaling policies based on CPU usage or request count to automatically add/remove instances.
  3. Configure an Application Load Balancer (ALB)
    • In the EC2 Dashboard, go to “Load Balancers” and create an ALB.
    • Configure the ALB to listen on port 80 (HTTP) or 443 (HTTPS) and forward traffic to port 3838 on the target instances.
    • Register the Auto Scaling Group as a target group for the ALB.
    • Optionally, enable sticky sessions if your Shiny app requires session persistence.
  4. Route Traffic via Route 53
    • Create an Alias record in Route 53 pointing to the ALB’s DNS name.
    • Use HTTPS by integrating an SSL certificate from AWS Certificate Manager with the ALB.
  5. Monitoring and Logging
    • Enable CloudWatch monitoring for the EC2 instances and ALB to track performance metrics.
    • Set up CloudWatch Alarms to notify you of high CPU usage or other anomalies.
  6. Security Considerations
    • Use IAM roles for EC2 instances instead of storing credentials manually.
    • Place instances in a private subnet within a VPC, allowing only the ALB to access them from the public internet.
    • Regularly patch instances using AWS Systems Manager Patch Manager.

Approach 3: Containerized Setup (EC2 with Docker)

If your company encourages containerization or you prefer a more portable setup, use Docker to run Shiny apps on EC2.

Steps:

  1. Launch an EC2 Instance
    • Follow the same steps as in Approach 1 to launch an Ubuntu-based EC2 instance.
  2. Install Docker

    • Update the instance and install Docker:

      sudo apt update
sudo apt install docker.io
sudo systemctl start docker
sudo systemctl enable docker
sudo usermod -aG docker ubuntu
  3. Pull or Build a Shiny Docker Image

    • Use an existing Shiny Docker image from Docker Hub:

      docker pull rocker/shiny

    • Alternatively, create a custom Dockerfile for your app:

      FROM rocker/shiny:latest
COPY . /srv/shiny-server/
EXPOSE 3838
CMD ["/usr/bin/shiny-server"]

    • Build and run the container:

      docker build -t my-shiny-app .
docker run -p 3838:3838 my-shiny-app
  4. Access the App
    • Access the app via http://<public-ip-address>:3838/.
  5. Scaling and Management
    • Use Docker Compose or AWS ECS (Elastic Container Service) for managing multiple containers or scaling.
    • If using ECS, deploy your Docker image to an ECS cluster with an ALB for load balancing.
  6. Security Considerations
    • Secure Docker by restricting access to the Docker daemon.
    • Use AWS ECR (Elastic Container Registry) to store and manage your Docker images securely.

Additional Notes and Best Practices

  • Cost Management: Monitor costs using AWS Cost Explorer. Use Reserved Instances or Savings Plans for long-term deployments. Shut down unused instances during off-hours if possible.
  • Backup and Recovery: Regularly snapshot your EC2 instance or use AWS Backup for data protection.
  • Company Policies: Adhere to your company’s AWS usage policies, such as tagging resources (e.g., project name, owner) for tracking.
  • Alternative Services: If managing EC2 instances seems complex, consider AWS services like Elastic Beanstalk for simplified deployment or AWS Fargate with ECS for serverless container management. Additionally, RStudio’s hosted solutions like ShinyApps.io might be an option if approved by your company, though they are not on AWS directly.
  • Networking: If your app needs to connect to internal company resources (e.g., databases), set up the EC2 instance in a VPC with appropriate subnets and route tables.

Which Approach to Choose?

  • Approach 1 (Simple Setup): Best for quick deployment, testing, or low-traffic apps. Minimal setup time but limited scalability.
  • Approach 2 (Scalable Setup): Best for production apps with variable traffic. Higher setup complexity but offers reliability and scalability.
  • Approach 3 (Containerized Setup): Best if your team uses Docker or plans to adopt container orchestration. Offers portability and consistency across environments.

References

When I have a Linux Computer

  1. Remotely log on to that computer.
  2. Install R, RStudio, Shinyserver.
  3. Put the app package there.
  4. Configure the ports.

Steps I take to host the app on AWS

1. Create a new server in AWS

Follow the guidance, we use EC2 service. First we do the configuration for EC2 instance as my server. - AMI(Amazon Machine Image): Ubuntu Server 18.04 LTS. - Instance type: t2.micro (CPU with one 1, 1 GB of RAM, basic internet connection, you can use this type for free) - Storage size: default is 8 GB, up to 30 GB is free. - Create a key pair to protect the access to the server, which is kept on my local computer.

Access the server with SSH under windows via SSH client PuTTY, or from Ubuntu system. Note that pem.key is the file that was saved as the key pair.

Important

ssh: connect to host 10.69.31.123 port 22: Connection timed out

Warning

Troubleshooting: 1. You may not be able to connect to this instance as ports 22 may need to be open. 2. Connect to your instance using its Private IP:

I tried to Connect to Your Instances Without Requiring Public Ipv4 Address Using EC2 Instance Connect Endpoint following the steps in this guide.

  1. Associate IAM role to the instance endpoint. I created an IAM role “CDMdev” and modify the IAM role on the instance under Security menu.
  2. Find the ip-prefix for my region.
    {
      "ip_prefix": "15.230.221.0/24",
      "region": "us-east-1",
      "service": "AMAZON",
      "network_border_group": "us-east-1"
    },
  1. Create or edit a security group for the instance.(In my case it is “launch-wizard-2”).
    • Use hostname -I to get the IP address of the instance or under other linux systems.
    • For the endpoint connection, open all traffic from the IP prefix above.
    • I specified port 22 for any IP using 0.0.0/0, which might not be secure enough.

This is a perfect situation for AWS Virtual Private Cloud. Put the internal instances in private subnets, and the public-facing instances in public subnets.

  1. Under VPC (Virtual Private Cloud), create endpoint with relevance security group and subnet (I don’t know which is relevant).

I can connect to EC2 using Endpoint Connect after the steps above.

However, I still cannot connect to it using SSH from my RStudio session terminal under same AWS account. Also I realized all the sudo commands that I am familiar with under Ubuntu does not work. I realized I have created an instance using Amazon Linux 2023.6.20250115.

[ec2-user@ip-10-69-31-123 ~]$ cat /etc/os-release
NAME="Amazon Linux"
VERSION="2023"
ID="amzn"
ID_LIKE="fedora"
VERSION_ID="2023"
PLATFORM_ID="platform:al2023"
PRETTY_NAME="Amazon Linux 2023.6.20250115"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2023"
HOME_URL="https://aws.amazon.com/linux/amazon-linux-2023/"
DOCUMENTATION_URL="https://docs.aws.amazon.com/linux/"
SUPPORT_URL="https://aws.amazon.com/premiumsupport/"
BUG_REPORT_URL="https://github.com/amazonlinux/amazon-linux-2023"
VENDOR_NAME="AWS"
VENDOR_URL="https://aws.amazon.com/"
SUPPORT_END="2028-03-15"

So I started from scratch to create a new instance with Ubuntu 24.04 LTS. Anyway, after a while, I can connect using Amazon console with endpoint connect. However, SSH does not work.

I can edit the security group inbound rules to allow any or specific IPv4 or IPv6 address to connect to the instance. But this inbound rule(s) will disappear after I refresh the page.

I also tried to initiate an instance with public IPv4 address, but the same SSH issue and instance connect issue pesists.

Warning

Instance is not in public subnet Associated subnet subnet-00fc65bbe108b4ef5 (SC-039060251298-pp-qv5chh3ykxxz2-PrivateSubnet2) is not a public subnet. To use EC2 Instance Connect, your instance must be in a public subnet. To make the subnet a public subnet, add a route in the subnet route table to an internet gateway.

I don’t know how to add a route in the subnet route table to an internet gateway. The reference is here

Using SSH, I see

Warning

You may not be able to connect to this instance as ports 22 may need to be open in order to be accessible. The current associated security groups don’t have ports 22 open.

ssh -i "CDM_EC2.pem" ubuntu@ec2-54-86-11-86.compute-1.amazonaws.com

I checked the security group and the inbound rules, and I see that my original setting that port 22 is open for all SSH traffic disappeared again. Edited it, tried again, Same time out error.

Important

ssh -i “CDM_EC2.pem” ubuntu@ec2-54-86-11-86.compute-1.amazonaws.com ssh: connect to host ec2-54-86-11-86.compute-1.amazonaws.com port 22: Connection timed out

2. Install R and R shiny on the server

  1. Add the CRAN repo by
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9
sudo add-apt-repository 'deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/'

Or use the following commands to add the repo for R 4.0:

# update indices
sudo apt update -qq
# install two helper packages we need
sudo apt install --no-install-recommends software-properties-common dirmngr
# add the signing key (by Michael Rutter) for these repos
# To verify key, run gpg --show-keys /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc 
# Fingerprint: E298A3A825C0D65DFD57CBB651716619E084DAB9
wget -qO- https://cloud.r-project.org/bin/linux/ubuntu/marutter_pubkey.asc | sudo tee -a /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc
# add the R 4.0 repo from CRAN -- adjust 'focal' to 'groovy' or 'bionic' as needed
sudo add-apt-repository "deb https://cloud.r-project.org/bin/linux/ubuntu $(lsb_release -cs)-cran40/"
  1. Install R

I followed the guidance from the RStudio website.

sudo apt update
sudo apt install r-base r-base-dev
sudo apt install --no-install-recommends r-base
  1. Install shiny package
sudo R
> install.packages("shiny")
  1. Install shinyserver

Modify the version as needed. I used the guide from posit

sudo apt-get install gdebi-core
wget https://download3.rstudio.org/ubuntu-18.04/x86_64/shiny-server-1.5.22.1017-amd64.deb
sudo gdebi shiny-server-1.5.22.1017-amd64.deb

3. Prepare all the necessary app files.

Tip

Github serves up an html page that includes the file specified along with context and operations you can perform on it. Tools like wget and curl will need github to send the raw file rather than an html wrapper. Change the blob in the link to raw to get the raw file.

An example is:

## note the raw replace blob in the https link.
wget -q --trust-server-names https://github.com/Zhenglei-BCS/Share/raw/master/cdm-1.2.0-Linux.tar.gz -O file.tar.gz

When installing packages, it can happen that last package installation has interrupted abnormally, then you need to remove those lock files first, for example, in R:

unlink("/home/me/src/Rlibs/00LOCK-Rcpp", recursive = TRUE)

Also for Ubuntu, some libraries (libssl-dev, libcurl4-openssl-dev, libxml2-dev,libfontconfig1-dev, libharfbuzz-dev libfribidi-dev, libfreetype6-dev libpng-dev libtiff5-dev libjpeg-dev ) are needed for the installation of some packages like “devtools”. For example, when installing “openssl” package, I got the following error:

Configuration failed because openssl was not found. Try installing: * deb: libssl-dev (Debian, Ubuntu, etc) * rpm: openssl-devel (Fedora, CentOS, RHEL) * csw: libssl_dev (Solaris) * brew: openssl (Mac OSX)

Configuration failed because openssl was not found. Try installing: Configuration failed because libcurl was not found. Try installing: * deb: libcurl4-openssl-dev (Debian, Ubuntu, etc) * rpm: libcurl-devel (Fedora, CentOS, RHEL)

To avoid switching between R session and the bash command, I used this

R -e "install.packages('remotes')"

4. Deploy the app on the server

Copied from the guide^1

From EC2 dashborad, select your instance. In the bottom half of your screen, find the line Security groups and click on the link. In the new window, again the bottom half of the screen, click on Inbound, the 2nd tab. Notice that only the port 22 is open. Click on Edit, and the popup that opens, enter the following settings:

EC2 dashboard \(\rightarrow\) go to column IPv4 Public IP.

3.121.42.9:3838

Open the port 3838 on the firewall

AWS uses port 22 by default for SSH access. All other ports are blocked

4. Optional steps

  • get a domain name
  • install nginx, generate TLS certificates, and secure you app with HTTPS.
  • protect the app with a password

Special Steps

The main issue I encounter in a corporate environment is that I need to use a VPN to connect to the AWS server. Everything is then tunneled and the usual steps to connect to the server do not work. Port 22 is usually blocked for external connections due to security considerations. Therefore we need a Jumpbox (called also Bastion Host) to connect to the server.We jump through is to access the internal VPC bastion hosts.

Using the Bastion SSH Tunnel, I can test my deployed application before making it publicly accessible via a proxy (Akana or Ocelot).

  1. set up Vault on my computer.
  2. get a SSH key signed by Vault before you can SSH through the central bastions.

I am using Git BASH on Windows, and I have the following in my ~/.ssh/config file:

use echo "$http_proxy" to check if the proxy is set up correctly when I am in the office and have VPN activated. You must set the environment variable for each new shell/console. In bash, you can ensure that it is always set by adding it to your ~/.bashrc or ~/.bash_profile file. Other shells have similar mechanisms to do this.

Other Tips

To find the public IP address from Ubuntu terminal, use the following command:

curl https://ipecho.net/plain ; echo

or

 wget -qO- https://ipecho.net/plain ; echo