There are many different R packages out there that allow you to collect tweets, one of the more common ones being twitteR. I have used this package for awhile now and found it very useful.
In the example below we will review how to collect tweets and export them to a CSV file. It is important to export these because Twitter only allows you to go back about a week and they limit the number of tweets you can pull at a given time. Collecting these in CSV format will allow you to create a more comprehensive dataset for future use.
First step is to set your working directory and import the libraries. In our case, the only library we need is twitteR.
You will then need to go onto the developer section of Twitter to get authentication credentials. The link above gives thorough instructions on how to get these.
Submit the credentials with the setup_twitter_oauth function. You will need to manually select 1: Yes in the console to accept this.
Let’s see what people have been tweeting about the stock market. Using the keyword #dowjones we can get a list of tweets on this topic. As you can see we are limiting this to the last 8 days and specifying that we wan to retrieve tweets in English only.
Highlighting the twitter_out dataset we created, we can see an example of the output.
The only criteria we added were language and date. There are many more specifications you can add including geo-location. Let’s say that we want to narrow this down and only look at tweets that are coming from withing 10 miles of the New York Stock Exchange. After looking up the exact latitude and longitude, the code below would then give us the result.
We will need to transform our dataset into a data frame using the lapply function and finally use the write.csv function to export out data as a CSV.
The CSV output can be seen below. There is much detail here that can be used for future analysis.
