Deploy a Shiny App on AWS

code
Stats
R
Reproduce
Author

Zhenglei Gao

Published

December 3, 2024

References

When I have a Linux Computer

  1. Remotely log on to that computer.
  2. Install R, RStudio, Shinyserver.
  3. Put the app package there.
  4. Configure the ports.

Steps I take to host the app on AWS

1. Create a new server in AWS

Follow the guidance, we use EC2 service. First we do the configuration for EC2 instance as my server. - AMI(Amazon Machine Image): Ubuntu Server 18.04 LTS. - Instance type: t2.micro (CPU with one 1, 1 GB of RAM, basic internet connection, you can use this type for free) - Storage size: default is 8 GB, up to 30 GB is free. - Create a key pair to protect the access to the server, which is kept on my local computer.

Access the server with SSH under windows via SSH client PuTTY, or from Ubuntu system. Note that pem.key is the file that was saved as the key pair.

Important

ssh: connect to host 10.69.31.123 port 22: Connection timed out

Warning

Troubleshooting: 1. You may not be able to connect to this instance as ports 22 may need to be open. 2. Connect to your instance using its Private IP:

I tried to Connect to Your Instances Without Requiring Public Ipv4 Address Using EC2 Instance Connect Endpoint following the steps in this guide.

  1. Associate IAM role to the instance endpoint. I created an IAM role “CDMdev” and modify the IAM role on the instance under Security menu.
  2. Find the ip-prefix for my region.
    {
      "ip_prefix": "15.230.221.0/24",
      "region": "us-east-1",
      "service": "AMAZON",
      "network_border_group": "us-east-1"
    },
  1. Create or edit a security group for the instance.(In my case it is “launch-wizard-2”).
    • Use hostname -I to get the IP address of the instance or under other linux systems.
    • For the endpoint connection, open all traffic from the IP prefix above.
    • I specified port 22 for any IP using 0.0.0/0, which might not be secure enough.

This is a perfect situation for AWS Virtual Private Cloud. Put the internal instances in private subnets, and the public-facing instances in public subnets.

  1. Under VPC (Virtual Private Cloud), create endpoint with relevance security group and subnet (I don’t know which is relevant).

I can connect to EC2 using Endpoint Connect after the steps above.

However, I still cannot connect to it using SSH from my RStudio session terminal under same AWS account. Also I realized all the sudo commands that I am familiar with under Ubuntu does not work. I realized I have created an instance using Amazon Linux 2023.6.20250115.

[ec2-user@ip-10-69-31-123 ~]$ cat /etc/os-release
NAME="Amazon Linux"
VERSION="2023"
ID="amzn"
ID_LIKE="fedora"
VERSION_ID="2023"
PLATFORM_ID="platform:al2023"
PRETTY_NAME="Amazon Linux 2023.6.20250115"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2023"
HOME_URL="https://aws.amazon.com/linux/amazon-linux-2023/"
DOCUMENTATION_URL="https://docs.aws.amazon.com/linux/"
SUPPORT_URL="https://aws.amazon.com/premiumsupport/"
BUG_REPORT_URL="https://github.com/amazonlinux/amazon-linux-2023"
VENDOR_NAME="AWS"
VENDOR_URL="https://aws.amazon.com/"
SUPPORT_END="2028-03-15"

So I started from scratch to create a new instance with Ubuntu 24.04 LTS. Anyway, after a while, I can connect using Amazon console with endpoint connect. However, SSH does not work.

I can edit the security group inbound rules to allow any or specific IPv4 or IPv6 address to connect to the instance. But this inbound rule(s) will disappear after I refresh the page.

I also tried to initiate an instance with public IPv4 address, but the same SSH issue and instance connect issue pesists.

Warning

Instance is not in public subnet Associated subnet subnet-00fc65bbe108b4ef5 (SC-039060251298-pp-qv5chh3ykxxz2-PrivateSubnet2) is not a public subnet. To use EC2 Instance Connect, your instance must be in a public subnet. To make the subnet a public subnet, add a route in the subnet route table to an internet gateway.

I don’t know how to add a route in the subnet route table to an internet gateway. The reference is here

Using SSH, I see

Warning

You may not be able to connect to this instance as ports 22 may need to be open in order to be accessible. The current associated security groups don’t have ports 22 open.

ssh -i "CDM_EC2.pem" ubuntu@ec2-54-86-11-86.compute-1.amazonaws.com

I checked the security group and the inbound rules, and I see that my original setting that port 22 is open for all SSH traffic disappeared again. Edited it, tried again, Same time out error.

Important

ssh -i “CDM_EC2.pem” ubuntu@ec2-54-86-11-86.compute-1.amazonaws.com ssh: connect to host ec2-54-86-11-86.compute-1.amazonaws.com port 22: Connection timed out

2. Install R and R shiny on the server

  1. Add the CRAN repo by
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E298A3A825C0D65DFD57CBB651716619E084DAB9
sudo add-apt-repository 'deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran35/'

Or use the following commands to add the repo for R 4.0:

# update indices
sudo apt update -qq
# install two helper packages we need
sudo apt install --no-install-recommends software-properties-common dirmngr
# add the signing key (by Michael Rutter) for these repos
# To verify key, run gpg --show-keys /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc 
# Fingerprint: E298A3A825C0D65DFD57CBB651716619E084DAB9
wget -qO- https://cloud.r-project.org/bin/linux/ubuntu/marutter_pubkey.asc | sudo tee -a /etc/apt/trusted.gpg.d/cran_ubuntu_key.asc
# add the R 4.0 repo from CRAN -- adjust 'focal' to 'groovy' or 'bionic' as needed
sudo add-apt-repository "deb https://cloud.r-project.org/bin/linux/ubuntu $(lsb_release -cs)-cran40/"
  1. Install R

I followed the guidance from the RStudio website.

sudo apt update
sudo apt install r-base r-base-dev
sudo apt install --no-install-recommends r-base
  1. Install shiny package
sudo R
> install.packages("shiny")
  1. Install shinyserver

Modify the version as needed. I used the guide from posit

sudo apt-get install gdebi-core
wget https://download3.rstudio.org/ubuntu-18.04/x86_64/shiny-server-1.5.22.1017-amd64.deb
sudo gdebi shiny-server-1.5.22.1017-amd64.deb

3. Prepare all the necessary app files.

Tip

Github serves up an html page that includes the file specified along with context and operations you can perform on it. Tools like wget and curl will need github to send the raw file rather than an html wrapper. Change the blob in the link to raw to get the raw file.

An example is:

## note the raw replace blob in the https link.
wget -q --trust-server-names https://github.com/Zhenglei-BCS/Share/raw/master/cdm-1.2.0-Linux.tar.gz -O file.tar.gz

When installing packages, it can happen that last package installation has interrupted abnormally, then you need to remove those lock files first, for example, in R:

unlink("/home/me/src/Rlibs/00LOCK-Rcpp", recursive = TRUE)

Also for Ubuntu, some libraries (libssl-dev, libcurl4-openssl-dev, libxml2-dev,libfontconfig1-dev, libharfbuzz-dev libfribidi-dev, libfreetype6-dev libpng-dev libtiff5-dev libjpeg-dev ) are needed for the installation of some packages like “devtools”. For example, when installing “openssl” package, I got the following error:

Configuration failed because openssl was not found. Try installing: * deb: libssl-dev (Debian, Ubuntu, etc) * rpm: openssl-devel (Fedora, CentOS, RHEL) * csw: libssl_dev (Solaris) * brew: openssl (Mac OSX)

Configuration failed because openssl was not found. Try installing: Configuration failed because libcurl was not found. Try installing: * deb: libcurl4-openssl-dev (Debian, Ubuntu, etc) * rpm: libcurl-devel (Fedora, CentOS, RHEL)

To avoid switching between R session and the bash command, I used this

R -e "install.packages('remotes')"

4. Deploy the app on the server

Copied from the guide^1

From EC2 dashborad, select your instance. In the bottom half of your screen, find the line Security groups and click on the link. In the new window, again the bottom half of the screen, click on Inbound, the 2nd tab. Notice that only the port 22 is open. Click on Edit, and the popup that opens, enter the following settings:

EC2 dashboard \(\rightarrow\) go to column IPv4 Public IP.

3.121.42.9:3838

Open the port 3838 on the firewall

AWS uses port 22 by default for SSH access. All other ports are blocked

4. Optional steps

  • get a domain name
  • install nginx, generate TLS certificates, and secure you app with HTTPS.
  • protect the app with a password

Special Steps

The main issue I encounter in a corporate environment is that I need to use a VPN to connect to the AWS server. Everything is then tunneled and the usual steps to connect to the server do not work. Port 22 is usually blocked for external connections due to security considerations. Therefore we need a Jumpbox (called also Bastion Host) to connect to the server.We jump through is to access the internal VPC bastion hosts.

Using the Bastion SSH Tunnel, I can test my deployed application before making it publicly accessible via a proxy (Akana or Ocelot).

  1. set up Vault on my computer.
  2. get a SSH key signed by Vault before you can SSH through the central bastions.

I am using Git BASH on Windows, and I have the following in my ~/.ssh/config file:

use echo "$http_proxy" to check if the proxy is set up correctly when I am in the office and have VPN activated. You must set the environment variable for each new shell/console. In bash, you can ensure that it is always set by adding it to your ~/.bashrc or ~/.bash_profile file. Other shells have similar mechanisms to do this.

Other Tips

To find the public IP address from Ubuntu terminal, use the following command:

curl https://ipecho.net/plain ; echo

or

 wget -qO- https://ipecho.net/plain ; echo