From set up to deployment

Author: Young Ha Suh

Last updated: 6/27/2024

Setup and requirements

R and RStudio

R is a free, open-source programming language for statistical computing and data visualization. In the case of our app and collection data, it will be the main way we write the code needed to generate the application. Instructions for downloading can be found here or here.

We will use R through RStudio, an integrated development environment (IDE) for R. Think of it as a more visual, user-friendly software for coding in R. It includes a console, syntax-highlighting editor that supports direct code execution, and tools for plotting, history, debugging, and workspace management. Download here.

RStudio basics

While I won’t cover all the basics of RStudio here, it’s important to note certain components of the user interface. You can find more information here.

• The Source Pane is where you can view and edit various code-related files (.R, .rmd, .txt). If there is more than one file open, you will see tabs that you can select through.
• The Console is where you can execute code.
• The Environments are a set of tabs but the one that you’ll use the most is the first Environment tab which lists currently saved R objects such as tables or models.
• The Output Pane also shows multiple tabs.
  • Files lists all the files in the folder or R project directory.
  • Plots displays various outputs such as plots or html content.
  • Packages shows currently installed R packages and which ones are loaded in the current session. You can search or install/update packages here.
  • Help displays package documentation and vignettes, if you need to remember how a function works, for instance.

Installing packages

Once R and Rstudio is downloaded, we will also need to install a few packages installed to run the script. Packages are extensions of R that contain code and documentation, often even sample data, that can be installed and used for various purposes. For instance:

install.packages("tidyverse") 
install.packages("leaflet")
install.packages(“urbnmapr”)

*note, urbnmapr might not be available for download. In that case, use the following to download the package through devtools.

Packages need to be installed only** once. After installation, you will need to **load them to the current session. If you close RStudio and start it up again, you can skip the installation and just load the packages, or libraries. Note that the library function does not require quotation marks.

# load libraries
library(tidyverse)
library(leaflet)
library(urbnmapr)

*Any lines that start with a # become comments – notes to yourself/others that won’t run in the console. It is good practice to use comments throughout your script to make sure your code is easily readable and anyone (including your future self) can know what certain lines of code does.

Shiny

Shiny is a package that makes it easy to build interactive web apps straight from R. To install and load the shiny package in R, open an R session and run

install.packages("shiny") 
library(shiny)

*More information about how to get started with shiny here.

Using GitHub

Recommended reading: Chapter 1 in Jenny Bryan’s Happy Git with R

Additional reading: Excuse me, do you have a moment to talk about version control?

We will be using GitHub as a way to store and share the necessary scripts and files needed to create the application. GitHub is a developer platform that allows developers (you!) to store, manage, and share code using Git, which is the software that provides version control.

Create a GitHub account and install git.

Why do we need version control?

Version control simply means that the system records any changes to a file or set of files so you can recall certain changes. Basically, it is a detailed ‘track changes’ function for any files or scripts that you may edit but might want the backup copy of. In this instance, Git tracks changes of your files then GitHub stores them online by syncing to the cloud. Not only is it useful to have these backup copies, but it is also useful in recording who makes the different changes, allowing for more efficient collaborations since you can roll-back on previous versions or contribute to others’ work.

Configure GitHub

Create a GitHub account and remember to note the username, email address, and the password.

Go back to Rstudio, then install the necessary packages. We will use the “usethis” package.

# install packages
install.packages("usethis")
library(usethis)

# set up git on your system
use_git_config(user.name = "myname123", user.email = "myname123@example.com")

Now we will confirm that RStudio and GitHub are communicating.

In RStudio, select the logo on the top right (mine says test but yours is probably blank) then select “New Project”. Or, you can go to File > New Project…

*Also, don’t worry if your RStudio looks different than my screenshots! You can change the appearance by going to Tools > Global Options > Appearance and select different Editor themes.

Once you click New Project, you will get a pop up with different options. Select Version Control.

Since we are using Git, select Git.

The fact that you can see these options means that Git/GitHub/RStudio are working!

You will also see the options for Git on two spots, the top ribbon as well as the right pane next to Environment.

Using GitHub for version control

There are 4 steps involved with syncing your local repository to the online one.

Basically, what happens on your local device does not automatically sync with the version stored online (GitHub). So in order to reflect any changes you made on your local machine need to get “pushed” to the cloud server OR any changes made on the cloud server needs to be “pulled” to your local machine to make sure the two are matched. This feature is essential if you are collaborating with other users that have their own local machines, but you can also view yourself (past or future) as one of these users in which Git/GitHub can keep track of any changes that are made.

You always start with pulling to sync your local repository with what is in the cloud. This is particularly important if you are working on multiple devices (e.g., personal vs. work computers) or if there were any updates that happened to the remote repository. You can pull by clicking the down arrow in the Git tab on the top right corner.

Git will then tell you if there were any changes. The log message will provide some information on what changed.

If everything is up to date, it will say so. This is where you always want to start.

Now you are ready to write or change any of the script. Whenever you edit something on your files, RStudio will tell you so by highlighting and adding an asterisk to the file that has the unsaved changes. In this example, I made changes to the server.R file and haven’t saved yet.

Once you save, then the asterisk disappears and the color returns.

Once these changes are saved, your Git will indicate so as well by listing any changed and saved files. Here, I modified both the server.R and ui.R files.

Depending on what changes you make to the local repository, Git will identify the type of change by using the following symbols.

Next, we will stage, or select the files we want to sync to the remote repo. Usually, you’d want to stage every edit you make but it may not always be applicable. For instance, really large files may take a while to sync and you may want to do it directly on GitHub. Simply click the boxes to indicate which files you want to stage.

Once we select the files, we will commit them to the local Git with the commit button

A new “Review Changes” window will pop up. There are 3 key panels here.

1. The top left panel lists all the files you are staging. If you want, you can skip the previous staging step and select the files you want to stage here.
2. You want to write a short but comprehensive message here that explains what changes you’ve made you’re committing. Try to be more descriptive/specific than the example I provided! These notes will show on your GitHub page!
3. The bottom half displays all the changes that have been made since the previous version. The old or deleted lines are in red and the new, modified lines are in green. This is a good way to review your work. You can check the changes for each staged file if you select it in the first panel.

Basically, on this panel we are committing any changes we have made to be sent to the remote repo. This panel is fairly useful and you can do all the 4 steps here (pull - stage - commit - push).

Once you are done, hit the Commit button.

Git will then commit your changes to the remote repo. Again, it will describe what changes have been made. You can now close this window.

Last, we are finally ready to push! Once you commit, the “Review Changes” window should now be blank. Click on the up arrow that says Push.

Depending on the size of the commit, R will process this command. Once you get the following message “HEAD -> main”, you’re good to go!

You can check any changes by going to the GitHub repo. This shows that the two files - ui.R and server.R - indeed were modified just a few minutes ago. You can also see what I wrote for the commits (and why it would’ve been useful to write something more descriptive).

Here comes the power/usefulness of GitHub. You can click on the Commits to see what changes have been made.

The Commits page will show you all the changes that have been made and when.

You can click on each commit to see further details. If I click on the most recent commit (“Updated script”), GitHub will highlight all the changes.

For the instance of this app, any changes that I make will be pushed to the original remote repository. Those changes can be pulled to individual cloned repositories so the changes will reflect on different users’ repositories. While users can push their own changes to their own remote repositories, because they are working on clones (or forked repos), they cannot push their changes onto the original repository. Well, there are ways to push that I can either accept or reject, but we won’t get into that just yet.

GitHub has many other cool features. For instance, you can see who contributes which line of code for each file.

You can also create Projects that will help you keep track of your progress and collaborations!

Merging conflicts

If the remote and local repositories are not synced (i.e., not at the same starting point), this will result in a conflict. You will likely receive an error message on where the conflict is and you will need to fix it to make sure the repositories are syncing properly. See more here: https://happygitwithr.com/push-rejected.html

Merging conflicts can be a PITA so I recommend to always PULL PULL PULL before you start working! And if you are working with a collaborator, communicate when you are working to prevent merging conflicts.

Now we have the basics, we can clone the repository I made to your local computer.

Cloning the repository

We will start by cloning my repository that has all the necessary script and files. Go to my collectionsdata repository.

To clone this repository on your own computer, select the green <>Code button.

Use the clipboard function to copy the url in your clipboard.

Now paste this copied url into RStudio.

Remember when you created a new project and selected version control? Paste the copied url into the first line “Repository URL:” The Project directory name and subdirectory will autofill. Feel free to edit those as needed. Especially pay attention to where the project will be saved using the “Browse” button. Check the “Open in new session” to keep your current project open.

Once you hit “Create Project”, you will have essentially copied the same project I have shared on Github to your personal device. Take a minute to browse all the files in the directory (bottom right pane). Select the following R files: “app.R”, “server.R”, and “ui.R’ which will then show on your source pane.

Getting started in R

The entire script for the app is available in app.R. I will walk you through each part of the code.

To run or execute each line of code, you can do it in 3 ways:

1) run the script with CTRL + Enter at each line 2) highlight all lines of code you want to run then hit CTRL + enter, or 3) copy and paste the line in the Console then hit enter.

First, load all the libraries after you install them once initially (e.g., install.packages("here")). Highlight the following script then use ctrl+enter to run then. In the console (bottom window), you can see the code run.

library(here)
library(shiny)
library(ggplot2)
library(tidyverse)
library(urbnmapr)
library(usmap)
library(leaflet)
library(sf)

Check if the working directory matches the folder that you saved the shiny app to. The file extension should match. If not, manually set the working directory with the setwd() function. Alternatively, you can go to Session > Set Working Directory > To Project Directory OR Choose Directory.

getwd()
#or
setwd("C:/Users/young/Documents/collectionsdata")

You should get something like [1] “C:/Users/young/Documents/collectionsdata”.

Next, we will load the data sets to run the script. For this example, use the following LACM data set that I’ve uploaded in GitHub. We will load this data into our working environment with the following.

data <- read.csv(“data_2023.csv”)

The first step is to make your data become suitable for the written code, as in, we need to make sure the formatting and fields match. But before we do that, let’s load up the sample data first to see what we are working with.

names(data)
names(data)[2] <- "catalog" 

head(data)

Alternatively, you can click on the object data in the Environment pane and it will load the data file on your source pane. But a few noteworthy points are:

• There are 124096 entries in this database
• There are 30 fields (columns) but the first one (“X”) can be ignored

Note that this data was exported from our EMu database as a csv format using Crystal Reports.

Testing your app

First, split app.R into two files: ui.R and server.R. The ui.R file will have all the components of the shinyUI function and the server.R file will have all the components of shinyServer(). Remember that in the app.R file, the ui and server were saved as individual objects in the environment, but for it to become a published app, we need to have them saved as two different files.

See the following screenshot. Click the “Run App” button next to the green arrow. It will basically use the ui.R and server.R files to generate the application.

Alternatively, type the runApp() command on your console. Make sure that the shiny package is already loaded.

runApp()

Once that is done, there will be a popup window with the application! This window works as it would on a web browser. Explore what it can do by selecting a species and clicking through the tabs.

Publishing the app

Read:

https://shiny.posit.co/r/getstarted/shiny-basics/lesson7/

https://shiny.posit.co/r/articles/share/shinyapps/

Congratulations on your first application! Note that this app only runs on your system and is not publicly available. So, to allow anyone to use this application, we will share our shiny application as a webpage. Your users can navigate to your app through the internet with a web browser. They will find your app fully rendered, up to date, and ready to go.

We are going to use shinyapps.io, Posit’s hosting service for Shiny apps.

Using shinyapps.io

First, sign up and create an account.

You can then access your Dashboard to see what apps are deployed. You can see my main application “lacm_birds” running online.

Connecting with RStudio

Once your account is created, we need to configure the rsconnect package to use your account and the automatically generated token. In RStudio, install and load the rsconnect package.

install.packages('rsconnect')
library(rsconnect)

Return to your account on shinyapps.io and retrieve your token by clicking your avatar and selecting “Tokens”.

Now you can configure the rsconnect package to your account.

Select the “Show” button on your token page.

A pop-up window will show and give you the option to copy the command to your clipboard. Copy this script with CTRL + C then paste it into the command line of RStudio and run.

Deploying your app

Method 1:

Use the deployApp command from the rsconnect package.

library(rsconnect)
deployApp()

The console will start running, starting with a message saying “Preparing for deployment. This may take a few minutes. Once it is done, you will see a message saying “Deployment complete” and “Successfully deployed to https:// …..

Method 2:

From RStudio, launch your app using the runApp() or clicking on the Run App icon. Once the app is launched in a separate window, select the Publish button.

Once deployment is complete, your browser should automatically open. See the example here.

Alternatively, you can select the url produced in the console.

I usually have this link as a bookmark on my laptop and even phone so I can browse the collections database quickly and anywhere!

Updating your app

Say you wanted to add a feature or you noticed a bug in your application. You can easily make those changes in your R scripts then re-run the app to see if those changes have been made or errors have been fixed. Once you do, you will need to re-publish your app so that those changes will reflect on the server that it is hosted on.

In RStudio, open the app. This time, you will see a “Republish” button. Click this and the following window will appear. Select the necessary components for the app – usually the data file, server.R, ui.R, and any dependencies (in this case, there is a logo image) then select “Publish”.

R will get busy and the “Deploy” tab on the console will automatically start running. Give it a few minutes to fully run and the app should automatically launch on your browser once it is finished.

Make sure to check if the new app is running smoothly with either the new changes or the errors fixed.

Troubleshooting

If you ever have problems launching your application, it can be helpful to see the message logs to understand what is causing the error. Some errors will be apparent during the deployment stage and any error messages will appear on the Deploy tab on the terminal. Alternatively, you can use the command rsconnect::showLogs() to show all the log messages of a deployed application.

You can also go to the shinyapps.io dashboard and look at the logs there. Click the app you want to check (without clicking on the “open in new tab” icon).

It will show you some basic information about your app.

Select the Logs tab and it can also reveal what errors might be causing the issue.

Hosting multiple apps

You can host up to 5 applications on shinyapps.io with the free version. Here, you can see that I have 2 applications for my account.

To avoid confusion, I set up separate R projects and separate shinyapps.io tokens. You can create a new token by selecting “Add Token” on the “Tokens” page on your shinyapps.io page. Then you work through the same steps of using rsconnect to configure your new R project with shinyapps.io.

Here is another app I made to compare two collections: LACM and CUMV. It is fairly similar to the one I made before but it contains LACM and CUMV data on just 4 species. I plan to build on this and combine more species and collections.

To be continued In the next post, I will explain how to edit the script to fit your data as well as workflows that can be adopted for your collections.

Example of application when you enter/select “Tyto alba” in the Species field.

Species summary tab

Provides an overview on how specimens are available by specimen type/nature based on species. Once you start typing in the blank field it should list options that you can select from. If you cannot find a species, it is likely that there are no specimens of that kind or you are searching with an updated scientific name that is different from the database.

1. Specimen count by specimen type Lists all the different specimen types for the input species. Description is written out in depth based on the specimen nature codes and their descriptions in EMu.

2. Specimen count by year The count of specimens for each year based only on study skins and skeletons; rest are ignored for visualization purposes.

3. Specimen count by month The count of specimens by the month they were collected.

4. Specimen count by state A map showing the number of specimens collected in each state. Given the nature of the locality data, the map is only limited to the United States. Higher values are in yellow while lower values are in dark purple. States where we have no specimens are grayed out. Note, this map is generated with only georeferenced data.

5. Specimen count by county A map showing the number of specimens collected in each county of California. Given the nature of the locality data, the map is only limited to California. Higher values are in yellow while lower values are in dark purple. Counties where we have no specimens are grayed out. Note, this map is generated with only georeferenced data.

6. Global specimen distribution An interactive map (Feaflet) showing the global distribution of specimen localities. Adjust the zoom using the +/- sign on the top left corner of the map. Note, this map is generated with only georeferenced data.

7. Weights by sex Simple box plots of each specimen’s weight based on sex (male, female, unknown). Raw data points are scattered over the box plots.

8. Detailed counts by year Last is a long table including the counts of specimens by year and counts by year and type.

Example of application when you enter the catalog number “12345” in the LACM field.

LACM lookup

For looking up specimens based on LACM number. It will provide information on species, collection date, collector, and locality. There are a few instances where you will enter a number but have no results; this is likely that there is missing information on the specimen. This is a feature that will be fixed in the next update.