The standard way to read text files into R is to use the read.table() command. However, many users struggle with time delays when loading large data sets. An alternative command that offers significant speed improvements is fread(), or fast read, which can found in the data.table package. The following code loads a tab delimited file with a million elements and reveals that fread() reduces load time by almost 99%, as confirmed by the benchmark performance stats at left. The function is still under development, but it is available for download and doesn’t suffer from stability issues. Instead, expect argument structure and command syntax to change over time.
# Create text file with 1e6 elements
obs <- rnorm(1e6, mean = 100, sd = 15)
obs.mat <- matrix(obs, nrow = 100)
write.table(obs.mat, "/Users/bxhorn/Downloads/mydata.txt", sep="\t")
# benchmark times
test1 <- system.time(read.table("/Users/bxhorn/Downloads/mydata.txt", sep="\t"))
test2 <- system.time(fread("/Users/bxhorn/Downloads/mydata.txt", sep="\t"))
Additional information on data.table package and the fread() function can be obtained in the package vignette.