2018-01-11-ireland Sofware Carpentry Workshop: NOTES.md - R lesson

NOTES.md - R lesson

These notes are for the tutors on the two-part R Software Carpentry course, taught 11-12th January 2018 at NUI Galway.

THINGS TO REMEMBER

To clear a console environment in R:

rm(list=ls())

Start the slides

TITLE

• Programming in R: Part One

ETHERPAD

• Please use the Etherpad for the course

LEARNING OBJECTIVES

• We’re being QUITE AMBITIOUS, and covering a few things, but we HAVE TWO SESSIONS, so should be OK
• We’ll be covering the fundamentals of R and RStudio - how to GET AROUND R AND WHAT RStudio CAN DO
• You’ll come away with fundamental knowledge applicable to PROGRAMMING IN ANY LANGUAGE
• We’ll be learning some of the best practices for writing and organising your code (FOLLOWING ON FROM GIT) and for reproducible computing
• Finally, we’ll be going through how to use these skills to be MORE EFFECTIVE WITH DATA ANALYSIS in R

SECTION 01: Introduction to R and RStudio

SLIDE: Learning Objectives

• We’re going to talk in general terms about WHAT R and RSTUDIO are.
• We’re also going to discuss WHAT MAKES THEM DIFFERENT FROM SOME OTHER TOOLS you might already use for similar tasks, like Excel

SLIDE: What is R?

• R is a PROGRAMMING LANGUAGE, and the SOFTWARE that runs programs written in that language.
• R is AVAILABLE ON ALL PLATFORMS

• This can sometimes be confusing - IF AT ANY POINT I AM UNCLEAR, PLEASE ASK!

• HAS ANYONE IN THE ROOM USED R BEFORE? - GREEN STICKY
  • If anyone has used R, please could you be available to help one of your neighbours who has not, if they have any questions. Look around you - if there’s someone nearby with a green sticky, say ‘hi’!
• WHY USE OR TEACH R?
  • R is FREE, and very WIDELY-USED across a range of disciplines.
  • There are MANY USEFUL PACKAGES or data analysis and statistics
  • It has excellent GRAPHICS CAPABILITIES
  • There is an international, friendly COMMUNITY across a range of disciplines, so it’s easy to find local and online support

SLIDE: But I already know Excel

• I’m NOT HERE TO CRITICISE EXCEL. Excel is brilliant at what it’s meant to do. It’s POWERFUL and INTUITIVE.

• But R HAS MANY ADVANTAGES FOR REPRODUCIBLE AND COMPLEX ANALYSIS

• A key thing for reproducibility is that it SEPARATES DATA FROM ANALYSIS.
  • In R, when you do anything to the original data, that original data remains unmodified (unless you overwrite the file).
  • POINT OF GOOD PRACTICE: RAW DATA SHOULD BE READ-ONLY
  • In Excel however it’s easy to overwrite data with copy-and-paste (and many bad things have happened as a result) - see Mike Croucher’s talk for examples.
• Because YOUR ANALYSIS IN R IS A PROGRAM, every step is written down explicitly, and is transparent and understandable by someone else.
  • In Excel, there is no clear record of where you moved your mouse, or what you copied and pasted, and it’s not immediately obvious how your formulas work.
• R code is EASY TO SHARE AND ADAPT, and to apply again to a different or a modified input dataset. It’s easy to publish the analyses via online resources, such as GitHub.
• R code can also be RUN ON EXTREMELY LARGE DATASETS, quickly. That’s much harder in Excel.

SLIDE: What is RStudio?

• RStudio is an INTEGRATED DEVELOPMENT ENVIRONMENT - which is to say it’s a very powerful tool for writing and using R and programs in the R language.
  • It’s available on ALL PLATFORMS
  • It’s available as a webserver, too.
• On the left is a screenshot WHILE I WAS WRITING THIS PRESENTATION IN RSTUDIO on a Mac

• On the right is the Windows version, with an EXAMPLE ANALYSIS AND VISUALISATION

• You can use it to INTERACT WITH R DIRECTLY TO EXPERIMENT WITH DATA
• It has a CODE/SCRIPT EDITOR FOR WRITING PROGRAMS
• It has tools for VISUALISING DATA
• It has built-in GIT INTEGRATION FOR MANAGING YOUR PROJECTS

## SECTION 02: Getting To Know RStudio

SLIDE: Learning Objectives

• We need to get some FAMILIARITY WITH OUR WORKING ENVIRONMENT: the RStudio IDE
• Develop some BEST PRACTICES FOR PROJECT MANAGEMENT in general, and in RStudio
• We’ll BUILD UPON THE GIT LESSON because RStudio integrates naturally with git to keep all your analysis and code under version control.
• We’ll introduce R syntax, and SEE HOW R REPRESENTS DATA and how to PROGRAM IN R
• If we have time we’ll deal with MANAGING SOME OF THE MANY USEFUL PACKAGES available in R

SLIDE: RStudio overview - Interactive Demo

• START RStudio (click icon/go into start menu and select RStudio/etc.)
  • CHECK EVERYONE CAN START RSTUDIO

• REMIND PEOPLE THEY CAN USE RED/GREEN STICKIES AT ANY TIME
• You should see THREE PANELS
  • Interactive R console: you can type here and get instant feedback
  • Environment/History window
  • Files/Plots/Packages/Help/Viewer: interacting with files on the computer, and viewing help and some output
• We’re going to use R in the interactive console to get used to some of the features of the language, and RStudio. DEMO CODE: ASK PEOPLE TO TYPE ALONG
  • THE RIGHT ANGLED BRACKET IS A PROMPT: R EXPECTS INPUT
  • TYPE THE CALCULATION, THEN PRESS RETURN

> 1 + 100
[1] 101
> 30 / 3
[1] 10

• RESULT IS INDICATED WITH A NUMBER [1] this indicates the line with output in it
• If you type an INCOMPLETE COMMAND, R will wait for you to complete it
  • DEMO CODE

> 1 +
+ 

• The PROMPT CHANGES TO + WHEN R EXPECTS MORE INPUT
• You can either complete the line, or use Esc (Ctrl-C) to exit

> 1 +
+ 6
[1] 7
> 1 + 
+ 

> 

• R obeys the usual PRECEDENCE OPERATIONS ( (, **/^, /, *, +, -)
  • DEMO CODE: NOTE SPACES AROUND OPERATORS

> 3 + 5 * 2
[1] 13
> (3 + 5) * 2
[1] 16
> 3 + 5 * 2 ^ 2
[1] 23
> 3 + 5 * (2 ^ 2)
[1] 23

• ARROW KEYS RECOVER OLD COMMANDS
• THE HISTORY TAB SHOWS ALL COMMANDS USED
• R will report in SCIENTIFIC NOTATION
  • CHECK THAT EVERYONE KNOWS WHAT SCIENTIFIC NOTATION IS

> 2 / 1000
[1] 0.002
> 2 / 10000
[1] 2e-04
> 5e3
[1] 5000

• R has many STANDARD MATHEMATICAL FUNCTIONS
• FUNCTION SYNTAX
  • type the function name
  • open parentheses
  • type input value
  • close parentheses
  • press return
  • DEMO CODE

> sin(1)
[1] 0.841471
> log(1)
[1] 0
> log10(10)
[1] 1
> log(10)
[1] 2.302585

• How do we learn more about a function, or the difference between log() and log10()?
• USE R BUILT-IN HELP
  • Type ? then the function name
  • Scroll to the bottom of the page to find example code

> ?log

• This brings up help in the HELP WINDOW
• You can also use the SEARCH BOX at the top of the help window (try sin())

• If you’re not sure about spelling, the editor has AUTOCOMPLETION which will suggest all possible endings for something you type (try log) - USE TAB TO SEE AUTOCOMPLETIONS

• We can do COMPARISONS in R
  • Comparisons return TRUE or FALSE. DEMO CODE
  • NOTE: when comparing numbers, it’s better to use all.equal() (machine numeric tolerance) ASK IF THERE’S ANYONE FROM MATHS/PHYSICS

> 1 == 1
[1] TRUE
> 1 != 2
[1] TRUE
> 1 < 2
[1] TRUE
> 1 <= 1
[1] TRUE
> 1 > 0
[1] TRUE
> 1 >= -9
[1] TRUE
> all.equal(1.0, 1.0)
[1] TRUE
> all.equal(1.0, 1.1)
[1] "Mean relative difference: 0.1"
> all.equal(pi-1e-7, pi)
[1] "Mean relative difference: 3.183099e-08"
> all.equal(pi-1e-8, pi)
[1] TRUE
> pi-1e-8 == pi
[1] FALSE

SLIDE: Challenge 01

SLIDE: Variables

• VARIABLES are critical to programming in general, and also to working in R
• Variables are like NAMED BOXES
  • Like a box, they HOLD THINGS
  • When we make reference to a box’s name, we MEAN THE THING THE BOX CONTAINS
• Here, the box is called Name, and it contains the word Samia
• When we refer to the box, we call it Name, and we might ask questions like:
  • “What is the length of Name?”, meaning “What is the length of the word in the box called Name?” (answer: 5)

SLIDE: Variables - Interactive Demo

• This is a very important concept, so we’re going to go through some practical examples, to reinforce it
  • In R, variables are assigned with the ASSIGNMENT OPERATOR <-
  • We will assign the value 1/40 to the variable x
  • DEMO CODE

> x <- 1 / 40

• At first, nothing seems to happen, but we can see that the variable x now exists, and contains the value 0.025 - a DECIMAL APPROXMATION of the fraction 1/40

> x
[1] 0.025

• CLICK ON THE ENVIRONMENT WINDOW
  • You should see that x is defined, there

• The Environment window in RStudio tells you the name and content of every variable currently active in your R session.

• This VARIABLE CAN BE USED ANYWHERE THAT EXPECTS A NUMBER
  • such as an argument to a function

> log(x)
[1] -3.688879
> sin(x)
[1] 0.0249974
> x + x
[1] 0.05
> 2 * x
[1] 0.05
> x ^ 2
[1] 0.000625

• We can REASSIGN VALUES TO VARIABLES
  • MONITOR THE VALUE OF x IN THE ENVIRONMENT WINDOW
  • We can also assign a variable to itself, to modify the variable

> x <- 100
> x <- x + 5

• We can assign ANY KIND OF VALUE to a variable
  • Including STRINGS - i.e. sets of characters

> name <- "Samia"
> name
[1] "Samia"

• But BE CAREFUL - R is not always intuitive
  • Strings in particular may not work the way you expect

> length(name)
[1] 1
> nchar(name)
[1] 5

SLIDE: Functions

• You’ve now used some functions. These are like “canned scripts”
• Functions have THREE MAIN PURPOSES
  • They ENCAPSULATE COMPLEX TASKS - you don’t need to worry about how a sine is calculated, just that you give the function a value, and it tells you what the sine is.
  • They MAKE CODE MORE READABLE - by dividing code into logical operations, represented by short names, the code is easier to read
  • They also MAKE CODE MORE REUSABLE - you don’t need to write the routine for finding a square root every time you want one, you just need to call the sqrt() function
• ALL THE FUNCTIONS YOU’VE SEEN ARE BUILT-IN, the so-called base functions

• OTHER FUNCTIONS FOR SPECIFIC TASKS CAN BE BROUGHT IN, THROUGH libraries

• Functions usually TAKE ARGUMENTS (input), e.g. sqrt(4) - the 4 is an argument
• Functions often RETURN values (output), e.g. sqrt(4) returns the value 2

SLIDE: Getting Help for Functions

• DEMO IN CONSOLE

> args(lm)
function (formula, data, subset, weights, na.action, method = "qr", 
    model = TRUE, x = FALSE, y = FALSE, qr = TRUE, singular.ok = TRUE, 
    contrasts = NULL, offset, ...) 
NULL
> ?sqrt
> help(sqrt)
> ??sqrt
> help.search("sqrt")
> help.search("categorical")
> vignette(two-table)
Error in vignette(two - table) : object 'two' not found
> vignette("two-table")

SLIDE: Removing Variables

• To remove ONE OR MORE SPECIFIED VARIABLES, use rm()
  • ls() IS A FUNCTION THAT LISTS VARIABLES (like the Environment tab)
  • DEMO CODE

> x <- 1
> y <- 2
> z <- 3
> ls()
[1] "x" "y" "z"
> rm(x)
> ls()
[1] "y" "z"
> rm(y, z)
> ls()
character(0)

SLIDE: Challenge 02

Solution:

mass <- 47.5 This will give a value of 47.5 for the variable mass

age <- 122 This will give a value of 122 for the variable age

mass <- mass * 2.3 This will multiply the existing value of 47.5 by 2.3 to give a new value of 109.25 to the variable mass.

age <- age - 20 This will subtract 20 from the existing value of 122 to give a new value of 102 to the variable age.

SLIDE: Good Variable Names

• Good variable names SHOULD HELP YOU DESCRIBE WHAT’S GOING ON
  • Helpful FOR YOU AND FOR OTHERS
• It’s better to avoid using names that already exist

• Use a CONSISTENT NAMING STYLE

• Rules for VARIABLE NAMES DIFFER BETWEEN LANGUAGES
• IN R
  • You can only use LETTERS, NUMBERS, UNDERSCORES, AND PERIODS
  • You can’t start variable names with a number
  • You can’t use whitespace in a variable name

SLIDE: Good Project Management Practices

• There is NO SINGLE GOOD WAY TO ORGANISE A PROJECT
  • It’s important to find something that WORKS FOR YOU
  • But it’s ALSO IMPORTANT THAT IT WORKS FOR COLLABORATORS
  • Some PRINCIPLES MAKE THINGS EASIER FOR EVERYONE
• Using a SINGLE WORKING DIRECTORY PER PROJECT/ANALYSIS
  • NAME IT AFTER THE PROJECT
  • EASY TO PACKAGE UP the whole directory and move it around, or share it - OR PUBLISH ALONGSIDE YOUR PAPER
  • NO NEED TO HUNT AROUND THE WHOLE DISK to find relevant or important files.
  • You can use RELATIVE PATHS that will always work, so long as you work within the project directory
• Treat your RAW DATA AS READ-ONLY
  • Establishes PROVENANCE and ENABLES REANALYSIS
  • Keep in SEPARATE SUBFOLDER
• CLEAN THE DATA PROGRAMMATICALLY (part of the analysis chain)
  • Remove/fill in null values, etc. - whatever is appropriate
  • KEEP CLEANED DATA SEPARATE FROM RAW - like food hygiene
• GENERATED OUTPUT IS DISPOSABLE
  • This means ANYTHING GENERATED AUTOMATICALLY BY YOUR CODE/ANALYSIS
  • If a file can be generated by scripts/code in your project, no need to put it under version control

SLIDE: Example Directory Structure

• This is ONE WAY TO STRUCTURE YOUR WORKING DIRECTORY
  • It’s a good starting point, but something else might be more appropriate for your own work
• WORKING DIR/ is the root directory of the project.
  • Everything related to the project (subdirectories of data, scripts and figures; git files; configuration files; notes to yourself; whatever)
• data/ is a subdirectory for storing data
  • raw data only, or raw and intermediate data? YOUR DECISION
  • data/raw, data/intermediate - USE SUBFOLDERS WHEN SENSIBLE
• data_output/ could be a place to write the analysis output (.csv files etc.)
• documents/ is a place where notes, drafts, and explanatory text could be stored
• fig_output/ could be a place to write graphical output of the analysis (keep separate from tables)

• scripts might be where you would choose to keep executable code that automates your analysis

• The important thing is that the structure is SELF-EXPLANATORY WHERE POSSIBLE

SLIDE: Project Management in RStudio

• RStudio TRIES TO BE HELPFUL and provides the ‘Project’ concept
  • Keeps ALL PROJECT FILES IN A SINGLE DIRECTORY
  • INTEGRATES WITH GIT
  • Enables switching between projects within RStudio
  • Keeps project histories

• INTERACTIVE DEMO

• CREATE PROJECT
• Click File -> New Project
  • Options for how we want to create a project: brand new in a new working directory; turn an existing directory into a project; or checkout a project from GitHub or some other repository
• Click New Directory
  • Options for various things we can do in RStudio. Here we want New Project
• Click New Project
  • We are asked for a directory name. ENTER swc-r-lesson
  • We are asked for a parent directory. PUT YOURS ON THE DESKTOP; STUDENTS CAN CHOOSE ANYWHERE SENSIBLE
  • SELECT Create a git repository - this will create and initialise a git repository, just for this project
• Click Create Project

• YOU SHOULD SEE AN EMPTY-ISH RSTUDIO WINDOW

• INSPECT PROJECT ENVIRONMENT
• First, NOTE THE WINDOWS: editor; environment; files
• EDITOR is empty
• ENVIRONMENT is empty
• FILES shows
  • CURRENT WORKING DIRECTORY (see breadcrumb trail)
  • TWO FILES: *.Rproj - information about your project; .gitignore - your project’s .gitignore file (remember the git lesson?)
• CLICK ON GIT TAB
  • We see that the files are not yet added to the project (status is ?)
  • CHECK BOXES TO STAGE FILES
  • CLICK ON DIFF TO SEE CHANGES (note colours, lines, etc.)
  • CLICK ON COMMIT TO COMMIT CHANGES
  • WRITE COMMIT MESSAGE (add .gitignore and R project files to repo)
  • CLICK COMMIT (explain message)
• CREATE DIRECTORIES IN PROJECT
• Create directoris called scripts and data
  • Click on New Folder
  • Enter directory name (scripts)
  • Note that the directory now exists in the Files tab (but not in the git tab, as the directory is empty)
  • Do the same for data/
• NOTE THAT WE WILL NOW POPULATE THE DIRECTORY

SLIDE: Working in RStudio

• RStudio offers SEVERAL WAYS TO WRITE CODE
  • We’ll not see all of them today
  • You’ve seen DIRECT INTERACTION IN THE CONSOLE (entering variables)
  • RStudio also has an editor for writing scripts, notebooks, markdown documents, and Shiny applications (EXPLAIN BRIEFLY)
  • It can also be used to write plain text

• INTERACTIVE DEMO OF R SCRIPT

• Click on File -> New File -> Text File. NOTE THAT THE EDITOR WINDOW OPENS
• Enter the following text, and EXPLAIN CSV
  • plain text file
  • one row per line
  • column entries separated by commas
  • first row is header data
  • NEEDS A BLANK LINE AT THE END
  • DATA DESCRIBES CATS

coat,weight,likes_string
calico,2.1,1
black,5.0,0
tabby,3.2,1

• SAVE THE FILE AS data/feline_data.csv
  • Click on disk icon
  • Navigate to data/ subdirectory
  • Enter filename feline_data.csv
• NOTE CHANGES IN GIT TAB
  • The data directory appears!
  • Click on Staged and THE FILE APPEARS
  • Click Commit and add a commit string, e.g. “add cat dataset”

• CLOSE THE EDITOR FOR THAT FILE

• Click on File -> New File -> R Script.
• EXPLAIN COMMENTS while entering the code below
  • COMMENTS ANNOTATE YOUR CODE: reminders for you, and information for others
• EXPLAIN read.csv()
  • read.csv() is a FUNCTION that reads data from a CSV-FORMAT FILE into a variable in R

# Script for exploring data structures

# Load cat data as a dataframe
cats <- read.csv(file = "data/feline_data.csv")

• SAVE THE SCRIPT
  • Click on File -> Save
  • Navigate to the scripts/ subdirectory
  • Enter filename data_structures (EXTENSION IS AUTOMATICALLY APPLIED) NOTE CHANGES IN GIT TAB
  • The scripts directory appears!
  • Click on Staged and the file appears
  • Click on Commit and add a commit string, e.g. “add data structures script”
• DO YOU SEE THE VARIABLE IN THE ENVIRONMENT?
  • NO - because the code hasn’t been executed, only written.
• RUN THE SCRIPT
  • Click on Source and NOTE THIS RUNS THE WHOLE SCRIPT
• Go to the Environment tab
  • NOTE THE DATA WAS LOADED IN THE VARIABLE cats
  • Note that there is a description of the data (3 obs. of 3 variables)
  • CLICK ON THE VARIABLE AND NOTE THAT THE TABLE IS NOW VISIBLE - this is helpful
  • YOU CANNOT EDIT THE DATA IN THIS TABLE - you can sort and filter, but not modify the data. This ENFORCES GOOD PRACTICE (compare to Excel).

SLIDE: 03. A First Analysis in RStudio

SLIDE: Learning Objectives

• We’re going to cover some ways of loading data into an R project/analysis
• We’ll EXPLORE SOME FUNCTIONS FOR SUMMARISING DATA
• Sometimes we want to use only a portion of our data, and we’ll see some WAYS OF INDEXING DATASETS
• We’ll also look at some ways to PLOT DATA IN R, using the built-in base graphics

SLIDE: Our Task

• We’ve got some medical data relating to a new treatment for arthritis
• There are some measurements of patient inflammation, taken over a period of days post-treatment for each patient

• We’ve been ASKED TO PRODUCE A SUMMARY AND SOME GRAPHS

• DOWNLOAD THE FILE FROM THE LINK TO data/
• EXPLAIN THE LINK IS ON THE ETHERPAD PAGE (no need to type!)
• DEMO THIS
  • NOTE: A new directory is created within data, called data. THIS IS UNTIDY, SO LET’S CLEAN
  • COPY ALL FILES BEGINNING WITH inflammation TO THE PARENT FOLDER
  • THEN DELETE THE SUBFOLDER AND ZIP FILE
  • ADD THE DATA FILES TO THE REPO (can shift-click, here)
• CHECK EVERYONE’S READY TO PROCEED

SLIDE: Loading Data - Interactive Demo

• We’ve already created some cat data manually, but THIS IS UNUSUAL - most data comes in the form of plain text files

START DEMO

• INSPECT DATA IN FILES WINDOW
  • Click on filename, and select View File
  • Note: THERE IS NO HEADER and THERE ARE NO ROW NAMES
  • Ask: IS THIS WELL-FORMATTED DATA?
  • I happen to know that there is one row per patient, and the columns are days, in turn, post-treatment, and measurements are inflammation levels
• WHAT IS THE DATA TYPE
  • Tabular, with EACH COLUMN SEPARATED BY A COMMA, so CSV
  • IN THE CONSOLE use read.csv() to read the data in
  • Note: IF WE DON’T ASSIGN THE RESULT TO A VARIABLE WE JUST SEE THE DATA
• CREATE A NEW SCRIPT
  • Click the triangle next to the new document icon
  • Add the code and SAVE AS scripts/inflammation (RStudio adds the extension)
  • See that the file appears in Files and Git windows

# Preliminary analysis of inflammation in arthritis patients

# Load data (no headers, CSV)
data <- read.csv(file = "data/inflammation-01.csv", header = FALSE)

• INSPECT THE DATA
  • Source the script
  • Check the Environment window: 60 observations (patients) of 40 variables (days)
  • CLICK ON data
  • COLUMN HEADERS ARE PROVIDED: Vn for variable n
  • dim() - dimensions of data: rows X columns
  • length() - number of columns in the table
  • ncol() - number of columns in the table
  • nrow() - number of rows in the table

> head(data, n = 2)
  V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13 V14 V15 V16 V17 V18 V19 V20 V21 V22 V23 V24 V25 V26
1  0  0  1  3  1  2  4  7  8   3   3   3  10   5   7   4   7   7  12  18   6  13  11  11   7   7
2  0  1  2  1  2  1  3  2  2   6  10  11   5   9   4   4   7  16   8   6  18   4  12   5  12   7
  V27 V28 V29 V30 V31 V32 V33 V34 V35 V36 V37 V38 V39 V40
1   4   6   8   8   4   4   5   7   3   4   2   3   0   0
2  11   5  11   3   3   5   4   4   5   5   1   1   0   1
> dim(data)
[1] 60 40
> length(data)
[1] 40
> ncol(data)
[1] 40
> nrow(data)
[1] 60

SLIDE: Challenge 03

SOLUTION

read.csv(file='file.csv', sep=';', dec=',')

SLIDE: Indexing Data

• We can refer to an element in our dataset by indexing it
  • We LOCATE A SINGLE ELEMENT as [row, column] in square brackets

> ncol(data)
[1] 40
> data[1,1]
[1] 0
> data[50,1]
[1] 0
> data[50,20]
[1] 16
> data[30,20]
[1] 16

• To get a RANGE OF VALUES, use the : separator to mean ‘to’:

> data[1:4, 1:4]   # rows 1 to 4; columns 1 to 4
  V1 V2 V3 V4
1  0  0  1  3
2  0  1  2  1
3  0  1  1  3
4  0  0  2  0
> data[30:32, 20:22]
   V20 V21 V22
30  16  14  15
31  16  13   7
32   9  19  15

• To get a WHOLE ROW OR COLUMN, leave that entry blank

> data[5,]
  V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13 V14 V15 V16 V17 V18 V19 V20 V21 V22 V23 V24 V25 V26
5  0  1  1  3  3  1  3  5  2   4   4   7   6   5   3  10   8  10   6  17   9  14   9   7  13   9
  V27 V28 V29 V30 V31 V32 V33 V34 V35 V36 V37 V38 V39 V40
5  12   6   7   7   9   6   3   2   2   4   2   0   1   1
> data[,16]
 [1]  4  4 15  8 10 15 13  9 11  6  3  8 12  3  5 10 11  4 11 13 15  5 14 13  4  9 13  6  7  6 14
[32]  3 15  4 15 11  7 10 15  6  5  6 15 11 15  6 11 15 14  4 10 15 11  6 13  8  4 13 12  9

SLIDE: Summary Functions - Interactive Demo

• R was designed for data analysis, so has many built-in functions for analysing and describing data
  • TALK THROUGH CODE IN CONSOLE

> max(data)
[1] 20
> max(data[2,])
[1] 18
> max(data[,7])
[1] 6
> min(data[,7])
[1] 1
> mean(data[,7])
[1] 3.8
> median(data[,7])
[1] 4
> sd(data[,7])
[1] 1.725187

SLIDE: Challenge 04

> animal <- c('m', 'o', 'n', 'k', 'e', 'y')
> animal[1:3]
[1] "m" "o" "n"
> animal[4:6]
[1] "k" "e" "y"
> animal[3:1]
[1] "n" "o" "m"
> animal[-1]
[1] "o" "n" "k" "e" "y"
> animal[-4]
[1] "m" "o" "n" "e" "y"
> animal[-1:-4]
[1] "e" "y"
> animal[-1:4]
Error in animal[-1:4] : only 0's may be mixed with negative subscripts

SLIDE: Repetitive Calculations - Interactive Demo

• We might want to CALCULATE MEAN INFLAMMATION FOR EACH PATIENT, but doing it the way we’ve just seen is tedious and slow.
• Happily COMPUTERS WERE INVENTED TO SAVE US THE HASSLE
• We could automate this task in any of several ways available in R
  • What we’d like to do is APPLY A FUNCTION TO EACH ROW
  • R has an apply() function exactly for this
  • IN THE CONSOLE

> apply(X = data, MARGIN = 1, FUN = mean)
 [1] 5.450 5.425 6.100 5.900 5.550 6.225 5.975 6.650 6.625 6.525 6.775 5.800 6.225 5.750 5.225
[16] 6.300 6.550 5.700 5.850 6.550 5.775 5.825 6.175 6.100 5.800 6.425 6.050 6.025 6.175 6.550
[31] 6.175 6.350 6.725 6.125 7.075 5.725 5.925 6.150 6.075 5.750 5.975 5.725 6.300 5.900 6.750
[46] 5.925 7.225 6.150 5.950 6.275 5.700 6.100 6.825 5.975 6.725 5.700 6.250 6.400 7.050 5.900

• IN OUR ANALYSIS SCRIPT we want to assign these values to a variable, and ALSO CALCULATE AVERAGE BY DAY
  • So long as we provide arguments in the correct order, WE DON’T NEED TO PROVIDE ARGUMENT NAMES - true for most R functions

# Calculate average inflammation by patient and day
avg_inflammation_patient <- apply(X = data, MARGIN = 1, FUN = mean)
avg_inflammation_day <- apply(data, 2, mean)

• RUN THE LINES
  • Note that the values appear in the Environment tab
• Like many common operations, there’s an R function that’s a shortcut
  • IN THE CONSOLE

> rowMeans(data)
 [1] 5.450 5.425 6.100 5.900 5.550 6.225 5.975 6.650 6.625 6.525 6.775 5.800 6.225 5.750 5.225
[16] 6.300 6.550 5.700 5.850 6.550 5.775 5.825 6.175 6.100 5.800 6.425 6.050 6.025 6.175 6.550
[31] 6.175 6.350 6.725 6.125 7.075 5.725 5.925 6.150 6.075 5.750 5.975 5.725 6.300 5.900 6.750
[46] 5.925 7.225 6.150 5.950 6.275 5.700 6.100 6.825 5.975 6.725 5.700 6.250 6.400 7.050 5.900
> colMeans(data)
        V1         V2         V3         V4         V5         V6         V7         V8         V9 
 0.0000000  0.4500000  1.1166667  1.7500000  2.4333333  3.1500000  3.8000000  3.8833333  5.2333333 
       V10        V11        V12        V13        V14        V15        V16        V17        V18 
 5.5166667  5.9500000  5.9000000  8.3500000  7.7333333  8.3666667  9.5000000  9.5833333 10.6333333 
       V19        V20        V21        V22        V23        V24        V25        V26        V27 
11.5666667 12.3500000 13.2500000 11.9666667 11.0333333 10.1666667 10.0000000  8.6666667  9.1500000 
       V28        V29        V30        V31        V32        V33        V34        V35        V36 
 7.2500000  7.3333333  6.5833333  6.0666667  5.9500000  5.1166667  3.6000000  3.3000000  3.5666667 
       V37        V38        V39        V40 
 2.4833333  1.5000000  1.1333333  0.5666667 

SLIDE: Base Graphics

• We’re doing all this work to try to GAIN INSIGHT INTO OUR DATA

• VISUALISATION IS A KEY ROUTE TO INSIGHT

• R has many graphics packages - some of which produce extremely beautiful images, or are tailored to a specific problem domain
• The built-in graphics are known as base graphics
• They may not be as pretty, or as immediately suited for all circumstances as some other packages, but they are still very powerful

SLIDE: Plotting - Interactive Demo

• IN THE SCRIPT
  • R’s plot() FUNCTION IS GENERAL AND WORKS FOR MANY KINDS OF DATA
  • RUN EACH LINE IN TURN
  • NOTE WHERE PLOTS SHOW (Plot window)
  • Opportunity to note: VARIABLES HELP READABILITY
  • USE ARROW BUTTONS to cycle through plots
  • COMMIT CHANGES WHEN DONE

# Plot data summaries
# Average inflammation by patient
plot(avg_inflammation_patient)

# Average inflammation per day
plot(avg_inflammation_day)

# Maximum inflammation per day
max_inflammation_day <- apply(data, 2, max)
plot(max_inflammation_day)

# Minimum inflammation per day
plot(apply(data, 2, min))

# Show a historgram of average patient inflammation
hist(avg_inflammation_patient)

• THE hist() FUNCTION PLOTS A HISTOGRAM OF INPUT DATA FREQUENCY/COUNT
  • The choice of bin sizes/breaks could be improved
  • We need to PROVIDE THE BOUNDARIES BETWEEN BINS
  • IN THE CONSOLE

hist(avg_inflammation_patient, breaks=c(5, 6, 7, 8))

• We’d have to TYPE IN A LOT OF NUMBERS to get smaller breaks, which is SLOW
  • The seq() function generates a sequence of numbers for us

> seq(5, 8)
[1] 5 6 7 8
> hist(avg_inflammation_patient, breaks=seq(5, 8))

• We can SET THE INTERVAL OF THE SEQUENCE

> seq(5, 8, by=0.2)
 [1] 5.0 5.2 5.4 5.6 5.8 6.0 6.2 6.4 6.6 6.8 7.0 7.2 7.4 7.6 7.8 8.0
> hist(avg_inflammation_patient, breaks=seq(5, 8, by=0.2))

• IN THE SCRIPT
  • Add a line with the histogram for average patient inflammation

# Show a historgram of average patient inflammation
hist(avg_inflammation_patient, breaks=seq(5, 8, by=0.2))

• IN THE SCRIPT
  • Demonstrate changing the input file
  • CHANGING FILENAME IN A SCRIPT IS QUICKER THAN RETYPING ALL THE COMMANDS

SLIDE: Challenge 05

# Plot standard deviation by day
plot(apply(data, 2, sd))

SLIDE: 04. Data Structures in R

SLIDE: Learning Objectives

• In this section, you’ll be learning about the data types in R: WHAT DATA IS
• You’ll also be learning about the data structures: WHAT DATA IS BUILT INTO - HOW IT IS ARRANGED
• And you’ll also learn how to find out what type/structure a particular piece of data has
• Putting it together, you’ll see how R’s data types and structures relate to the types of data that you work with, yourself.

SLIDE: Data Types and Structures in R

• R is MOSTLY USED FOR DATA ANALYSIS
• R is set up with key, core data types designed to help you work with your own data
• A lot of the time, R focuses on tabular data (like our cat example)

• INTERACTIVE DEMO

• SWITCH TO THE CONSOLE (Establish that cats is available as a variable)
• If you type cats, you get a nice tabular representation of your data

> cats
    coat weight likes_string
1 calico    2.1            1
2  black    5.0            0
3  tabby    3.2            1

• THINK ABOUT THE DATA TYPES Are they all the same?
  • NO coat is text; weight is some real value (in kg or pounds, maybe), and likes_string looks like it should be TRUE/FALSE
  • DOES IT MAKE SENSE TO WORK WITH THEM AS IF THEY’RE THE SAME THING? (No)
• EXTRACT A COLUMN FROM A TABLE
  • Use $ notation in the console
  • NOTE THE AUTOCOMPLETION

> cats$weight
[1] 2.1 5.0 3.2

• WHAT DID R RETURN?
  • A vector (1D ordered collection) of numbers
• *WE CAN OPERATE ON THESE *VECTORS **
  • Vectors are an important concept, and R is largely built so that operations on vectors are central to data analysis.

> cats$weight + 2
[1] 4.1 7.0 5.2

• WHAT ABOUT OTHER COLUMNS?

> cats$coat
[1] calico black  tabby 
Levels: black calico tabby

• WHAT DID R RETURN?
  • A vector of levels
  • We’ll talk about these in more detail shortly, but the key point is that R DOESN’T THINK THEY’RE ONLY WORDS - it THINKS THEY’RE NAMED CATEGORIES OF OBJECT. R is assuming that you mean to import data
  • We can operate on this vector, too (EXPLAIN paste())

> paste("My cat is", cats$coat)
[1] "My cat is calico" "My cat is black"  "My cat is tabby" 

• WHAT HAPPENS NEXT?

> cats$weight + cats$coat
[1] NA NA NA
Warning message:
In Ops.factor(cats$weight, cats$coat) : ‘+’ not meaningful for factors

• You probably already realised that wasn’t going to work, because adding “calico” to “2.1” is nonsense.
• THESE DATA TYPES ARE NOT COMPATIBLE for addition
• R’s data types reflect the ways in which data is expected to interact
• UNDERSTANDING HOW YOUR DATA MAP TO R’s DATA TYPES IS KEY
  • It’s very important to understand how R sees your data (you want R to see your data the same way you do)
  • Many problems in R come down to incompatibilities between data and data types.

SLIDE: What Data Types Do You Expect?

• ASK THE STUDENTS
  • What data types would you expect to see?
  • What data types do you think you would WANT OR NEED, from your own experience?
• SPEND A COUPLE OF MINUTES ON THIS
  • The difference between a data type and a data structure

SLIDE: Data Types in R

• R’s data types are atomic: they are FUNDAMENTAL AND EVERYTHING ELSE IS BUILT UP FROM THEM, like matter is built up from atoms
  • In particular, all the data structures are built up from data types
• There are only FIVE DATA TYPES in R (though one is split into two…)
  • logical: Boolean, True/False (also 1/0)
  • numeric: anything that’s a number on the number line; two types of number are supported: integer and double (real)
  • complex: complex numbers, defined on the 2D plane
  • character: text data - readable symbols
  • raw: binary data (we’ll not be dealing with this)
• LET’S LEARN A BIT MORE ABOUT THEM IN THE DEMO
• ENTER DEFINITIONS INTO THE SCRIPT
  • Covering the major data types

# Some variables of several data types
truth <- TRUE
lie <- FALSE
i <- 3L
d <- 3.0
c <- 3 + 0i
txt <- "TRUE"

• EXECUTE THE VARIABLE DEFINITIONS
  • Select the definition lines
  • Click on Run
  • OBSERVE THAT THE LINES ARE RUN IN THE CONSOLE
  • OBSERVE THAT THE VALUES ARE DEFINED IN THE ENVIRONMENT
  • Note the difference between Data and Values in the environment
  • Note that the script is updated in the Git tab: commit the change
• USE typeof() TO FIND THE TYPE OF A VARIABLE

> typeof(i)
[1] "integer"
> typeof(c)
[1] "complex"
> typeof(d)
[1] "double"

• TO TEST IF A DATA ITEM HAS A TYPE, USE is.<type>()

> is.numeric(3)
[1] TRUE
> is.numeric(d)
[1] TRUE
> is.double(i)
[1] FALSE
> is.integer(d)
[1] FALSE
> is.numeric(txt)
[1] FALSE
> is.character(txt)
[1] TRUE
> is.character(truth)
[1] FALSE
> is.logical(truth)
[1] TRUE

• THE INTEGER, COMPLEX, AND DOUBLE ARE EQUAL even if they’re not the same data type
  • numbers are comparable, regardless of data type

> i == c
[1] TRUE
> i == d
[1] TRUE
> d == c
[1] TRUE

• INTEGER, COMPLEX AND DOUBLE ARE NOT ALL numeric though

> is.numeric(i)
[1] TRUE
> is.numeric(c)
[1] FALSE

SLIDE: Challenge 06

• Let the students work for a couple of minutes, then demonstrate.

SLIDE: FIVE COMMON R DATA STRUCTURES

• These are perhaps the five data types you’ll come across most often in R
• We’ll deal with them through examples

• INTERACTIVE DEMO IN SCRIPT

• VECTORS
  • These are the MOST COMMON DATA STRUCTURE
  • Vectors can contain ONLY A SINGLE DATA TYPE (atomic vectors)
  • ADD CODE TO SCRIPT then use Run to run in console
  • To create a vector USE THE c() FUNCTION (c() is combine; use ?c)
  • First we define an ATOMIC VECTOR OF NUMBERS - each element is an integer

# Define an integer vector
x <- c(10, 12, 45, 33)

• We can use some R functions to find out more about this variable
  • RUN CODE IN CONSOLE

> length(x)
[1] 4
> typeof(x)
[1] "double"
> str(x)
 num [1:4] 10 12 45 33

• The str() function REPORTS THE STRUCTURE OF A VARIABLE
  • Here, num means ‘numeric’; [1:4] means there are four elements; the elements are listed
  • NOTE THAT THIS INFORMATION IS IN THE ENVIRONMENT TAB
• DEFINE A SECOND VECTOR IN THE SCRIPT

# Define a vector
xx <- c(1, 2, 'a')

• In the Environment tab, you can see THIS IS A CHARACTER VECTOR

> length(xx)
[1] 3
> typeof(xx)
[1] "character"
> str(xx)
 chr [1:3] "1" "2" "a"

• IS THE TYPE OF THE VECTOR WHAT YOU EXPECTED?
  • This is one of the things that trips people up with R - they think their data is of one type, but R thinks it makes more sense to have it as another type

SLIDE: Challenge 07

• Let the students work for a couple of minutes, then demonstrate.

SLIDE: Coercion

• Coercion is what happens when you COVERT ONE DATA TYPE INTO ANOTHER
• If R thinks it needs to, it will COERCE DATA IMPLICITLY without telling you
• There is a set order for coercion
  • logical can be coerced to integer, but integer cannot be coerced to logical
  • That’s because integer can describe all logical values, but not vice versa
  • Everything can be represented as a character, so that’s the fallback position for R
• IF THERE’S A FORMATTING PROBLEM IN YOUR DATA, R MIGHT CONVERT THE TYPE TO COPE
  • R will choose the simplest data type that can represent all items in the vector
• INTERACTIVE DEMO IN CONSOLE More useful things to do with vectors
• You can (usually) COERCE VECTORS MANUALLY with as.<type>()

> as.character(x)
[1] "10" "12" "45" "33"
> as.complex(x)
[1] 10+0i 12+0i 45+0i 33+0i
> as.logical(x)
[1] TRUE TRUE TRUE TRUE
> xx
[1] "1" "2" "a"
> as.numeric(xx)
[1]  1  2 NA
Warning message:
NAs introduced by coercion 
> as.logical(xx)
[1] NA NA NA

• You can generate NUMBER SEQUENCES as vectors
  • The seq() function returns a vector
  • As does the : operator

> seq(10)
 [1]  1  2  3  4  5  6  7  8  9 10
> seq(1, 10)
 [1]  1  2  3  4  5  6  7  8  9 10
> seq(35, 40, by=0.5)
 [1] 35.0 35.5 36.0 36.5 37.0 37.5 38.0 38.5 39.0 39.5 40.0
> 1:10
 [1]  1  2  3  4  5  6  7  8  9 10
> 5:8
[1] 5 6 7 8 

• You can APPEND ELEMENTS TO A VECTOR WITH c()

> x
[1] 10 12 45 33
> c(x, 57)
[1] 10 12 45 33 57
> x
[1] 10 12 45 33
> x <- c(x, 57)
> x
[1] 10 12 45 33 57

• There are useful FUNCTIONS TO GET INFORMATION ABOUT A VECTOR

> x <- 0:10
> tail(x)
[1]  5  6  7  8  9 10
> head(x)
[1] 0 1 2 3 4 5
> head(x, n=2)
[1] 0 1

• You can GIVE VECTOR ELEMENTS NAMES
  • They’re then referred to as named vectors

> x <- 1:4
> names(x)
NULL
> str(x)
 int [1:4] 1 2 3 4
> names(x) <- c("a", "b", "c", "d")
> x
a b c d 
1 2 3 4 
> str(x)
 Named int [1:4] 1 2 3 4
 - attr(*, "names")= chr [1:4] "a" "b" "c" "d"

SLIDE: Factors

• In general DATA COMES AS ONE OF TWO TYPES
  • Quantitative data represents measurable values. These are usually either CONTINUOUS (real values like a height in centimetres) or a COUNT (like number of beans in a tin).
  • Categorical data representing DISCRETE GROUPS, which can be UNORDERED (like “types of computer”; “educational establishments”) or ORDERED (like floors of a building, or grades in school)
• THIS DISTINCTION IS CRITICAL IN MANY STATISTICAL/ANALYTICAL METHODS

• R WAS MADE FOR STATISTICS so has a special way of dealing with the difference

• FACTORS ARE SPECIAL VECTORS REPRESENTING CATEGORICAL DATA
  • Factors are stored as VECTORS OF LABELLED INTEGERS
  • Factors CANNOT BE TREATED AS TEXT
• CREATE FACTOR IN SCRIPT
  • We create a FACTOR WITH THREE ELEMENTS
  • Run the line
  • Commit the change

# Create a factor with three elements
> f <- factor(c("no", "yes", "no"))

• INSPECT THE FACTOR IN THE CONSOLE
  • When we look at the structure of the vector, it reports TWO LEVELS: "yes" and "no"
  • It also reports a list of values: 1 2 1
  • There is a mapping "no" -> 1 and "yes" -> 2
  • The VECTOR STORES INTEGERS 1 and 2, BUT THESE ARE LABELLED "no" and "yes"

> length(f)
[1] 3
> str(f)
 Factor w/ 2 levels "no","yes": 1 2 1
> levels(f)
[1] "no"  "yes"
> f
[1] no  yes no 
Levels: no yes

• IN OUR cats DATA THE COAT WAS STORED AS A FACTOR
• DEMO IN CONSOLE
  • The class() function IDENTIFIES DATA STRUCTURES
  • NOTE THAT BY DEFAULT FACTORS ARE NUMBERED IN ALPHABETICAL ORDER OF LABEL

> cats$coat
[1] calico black  tabby 
Levels: black calico tabby
> class(cats$coat)
[1] "factor"
> str(cats$coat)
 Factor w/ 3 levels "black","calico",..: 2 1 3

SLIDE: Challenge 08

> f <- factor(c("case", "control", "case", "control", "case"))
> str(f)
 Factor w/ 2 levels "case","control": 1 2 1 2 1
> f <- factor(c("case", "control", "case", "control", "case"), levels=c("control", "case"))
> str(f)
 Factor w/ 2 levels "control","case": 2 1 2 1 2

SLIDE: Matrices

• Matrices may be the MOST IMPORTANT DATA STRUCTURE IN NUMERICAL ANALYSIS, central to pretty much any statistical operation
• THEY GET THEIR OWN SPECIAL DATA STRUCTURE IN R

• They are 2D vectors (so contain atomic values)

• CREATE MATRICES IN SCRIPT
  • Run the lines when done
  • Commit the changes

# Create matrix of zeroes
m1 <- matrix(0, ncol = 6, nrow = 3)

# Create matrix of numbers 1 and 2
m2 <- matrix(c(1, 2), ncol = 3, nrow = 4)

• INSPECT THE RESULTS IN THE CONSOLE
  • ncol and nrow define the size of the matrix
  • providing a single value as the first argument fills the matrix with that value
  • The length() of a matrix IS THE TOTAL NUMBER OF ELEMENTS

> class(m1)
[1] "matrix"
> m1
     [,1] [,2] [,3] [,4] [,5] [,6]
[1,]    0    0    0    0    0    0
[2,]    0    0    0    0    0    0
[3,]    0    0    0    0    0    0
> str(m1)
 num [1:3, 1:6] 0 0 0 0 0 0 0 0 0 0 ...
 > length(m1)
[1] 18

• PROVIDING TWO VALUES TO THE FIRST ARGUMENT fills the matrix, too
  • So long as the LENGTH OF THE MATRIX IS A MULTIPLE OF THE INPUT LENGTH
  • We can INDEX AND SLICE just like before

> m2
     [,1] [,2] [,3] [,4]
[1,]    1    2    1    2
[2,]    2    1    2    1
[3,]    1    2    1    2
> m2[1, ]
[1] 1 2 1 2
> m2[2:3, 3:4]
     [,1] [,2]
[1,]    2    1
[2,]    1    2

SLIDE: Challenge 09 (5min)

> m <- matrix(1:50, nrow = 5, ncol = 10)
> m
     [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,]    1    6   11   16   21   26   31   36   41    46
[2,]    2    7   12   17   22   27   32   37   42    47
[3,]    3    8   13   18   23   28   33   38   43    48
[4,]    4    9   14   19   24   29   34   39   44    49
[5,]    5   10   15   20   25   30   35   40   45    50
> ?matrix
> m <- matrix(1:50, nrow = 5, ncol = 10, byrow = TRUE)
> m
     [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,]    1    2    3    4    5    6    7    8    9    10
[2,]   11   12   13   14   15   16   17   18   19    20
[3,]   21   22   23   24   25   26   27   28   29    30
[4,]   31   32   33   34   35   36   37   38   39    40
[5,]   41   42   43   44   45   46   47   48   49    50

SLIDE: Lists

• lists are like vectors, EXCEPT THEY CAN HOLD ANY DATA TYPE

• CREATE NEW LIST IN SCRIPT

  • Run from script
  • Commit to repo

# Create a list
l <- list(1, 'a', TRUE, matrix(0, nrow = 2, ncol = 2), f)

# Create a named list
l_named <- list(a = "SWC", b = 1:4)

• INSPECT THE LISTS IN THE CONSOLE
  • The elements are identified with DOUBLE SQUARE BRACKETS [[n]]
  • We use this LIKE ANY OTHER INDEX

> class(l)
[1] "list"
> class(l_named)
[1] "list"
> str(l)
List of 5
 $ : num 1
 $ : chr "a"
 $ : logi TRUE
 $ : num [1:2, 1:2] 0 0 0 0
 $ : Factor w/ 2 levels "no","yes": 1 2 1
> str(l_named)
List of 2
 $ a: chr "SWC"
 $ b: int [1:4] 1 2 3 4
> l
[[1]]
[1] 1

[[2]]
[1] "a"

[[3]]
[1] TRUE

[[4]]
     [,1] [,2]
[1,]    0    0
[2,]    0    0

[[5]]
[1] no  yes no 
Levels: no yes
> l[[4]][1,1]
[1] 0

• THE NAMED LIST IS SLIGHTLY DIFFERENT
  • CAN STILL INDEX
  • But can also NOW USE NAMES with $
  • INDICES CAN CHANGE IF DATA IS MODIFIED - NAMES ARE MORE ROBUST
  • NAMES CAN ALSO BE MORE DESCRIPTIVE (HELPS UNDERSTANDING)

> l_named
$a
[1] "SWC"

$b
[1] 1 2 3 4
> l_named[[1]]
[1] "SWC"
> l_named[[2]]
[1] 1 2 3 4
> l_named$a
[1] "SWC"
> l_named$b
[1] 1 2 3 4
> names(l_named)
[1] "a" "b"

SLIDE: Logical Indexing

• We’ve seen INDEXING and NAMES as ways to get elements from variables SO LONG AS WE KNOW WHICH ELEMENTS WE WANT
• LOGICAL INDEXING allows us to SPECIFY CONDITIONS FOR THE DATA WE WANT TO RECOVER
  • For instance, we might want ALL VALUES OVER A THRESHOLD or ALL NAMES STARTING WITH ‘S’
• DEMO IN SCRIPT (data_structures.R)
  • We create a vector of values as an example
  • We make a MASK OF TRUE/FALSE (i.e. logical) values
  • Run the lines

# Create a vector for logical indexing
v <- c(5.4, 6.2, 7.1, 4.8, 7.5)
mask <- c(TRUE, FALSE, TRUE, FALSE, TRUE)

• DEMO IN CONSOLE
• Now, when we USE THE MASK AS AN INDEX we only get THE ELEMENTS WHERE THE MASK IS TRUE

> v
[1] 5.4 6.2 7.1 4.8 7.5
> v[mask]
[1] 5.4 7.1 7.5

• COMPARATORS IN R RETURN VECTORS OF TRUE/FALSE VALUES
  • These can be USED AS LOGICAL MASKS fOR DATA
  • Comparators CAN BE COMBINED

> v
[1] 5.4 6.2 7.1 4.8 7.5
> v < 7
[1]  TRUE  TRUE FALSE  TRUE FALSE
> v[v < 7]
[1] 5.4 6.2 4.8
> v < 7
[1]  TRUE  TRUE FALSE  TRUE FALSE
> v > 5 & v < 7
[1]  TRUE  TRUE FALSE FALSE FALSE
> v[v > 5 & v < 7]
[1] 5.4 6.2
> v > 5 | v < 7
[1] TRUE TRUE TRUE TRUE TRUE
> v[v > 5 | v < 7]
[1] 5.4 6.2 7.1 4.8 7.5

• COMMIT CHANGES

SLIDE: 05. Dataframes

• Dataframes are probably the most important thing you will ever learn about, in R.
• Almost everything in R, on a practical day-to-day basis, involves dataframes

SLIDE: Learning Objectives

• After this section YOU WILL UNDERSTAND WHAT A DATAFRAME IS AND HOW IT IS BUILT UP FROM R DATA STRUCTURES YOU ALREADY KNOW
• You’ll also know HOW TO ACCESS ANY PART OF A DATA FRAME, INCLUDING CONDITIONAL ACCESS
• We’ll also see how to read data in to, and write it out from, a data frame.

SLIDE: Let’s look at a data.frame

• The cats data is a small data.frame
• DEMO IN CONSOLE
  • NOTE: TABULAR
  • NOTE: 9 elements but length 3?
  • Try list indexes… IT’S A LIST
  • Try names()… IT’S A NAMED LIST
  • What are the classes of each list element? THEY’RE VECTORS

> class(cats)
[1] "data.frame"
> cats
    coat weight likes_string
1 calico    2.1            1
2  black    5.0            0
3  tabby    3.2            1
> length(cats)
[1] 3
> cats[[1]]
[1] calico black  tabby 
Levels: black calico tabby
> typeof(cats)
[1] "list"
> names(cats)
[1] "coat"         "weight"       "likes_string"
> class(cats$coat)
[1] "factor"
> class(cats$weight)
[1] "numeric"
> class(cats$likes_string)
[1] "integer"

SLIDE: What is a data.frame

• This structure is THE MOST IMPORTANT THING IN R
  • It’s the standard structure for storing any king of tabular, ‘rectangular’ data
• We’ve seen that it’s a NAMED LIST where each element is a VECTOR
  • All the vectors have the same length
• This is VERY SIMILAR TO A SPREADSHEET, but it’s more finicky:
  • We require every element in a column to be the same data type
  • We need all the columns to be the same length
  • In spreadsheets, neither of these conditions are enforced
  • This makes R a bit more data-safe

SLIDE: Creating a data.frame

• DEMO IN SCRIPT
  • Run when done
  • Commit to repo

# Create a data frame
df <- data.frame(a=c(1,2,3), b=c('eeny', 'meeny', 'miney'),
                 c=c(TRUE, FALSE, TRUE))

• DEMO IN CONSOLE
  • !!!!STRINGS ARE INTERPRETED AS FACTORS!!!!
  • The summary() function SUMMARISES PROPERTIES OF EACH COLUMN
  • The summary depends on the column type

> str(df)
'data.frame':	3 obs. of  3 variables:
 $ a: num  1 2 3
 $ b: Factor w/ 3 levels "eeny","meeny",..: 1 2 3
 $ c: logi  TRUE FALSE TRUE
> df$c
[1]  TRUE FALSE  TRUE
> length(df)
[1] 3
> dim(df)
[1] 3 3
> summary(df)
       a           b         c          
 Min.   :1.0   eeny :1   Mode :logical  
 1st Qu.:1.5   meeny:1   FALSE:1        
 Median :2.0   miney:1   TRUE :2        
 Mean   :2.0                            
 3rd Qu.:2.5                            
 Max.   :3.0