Visualization of Tweets on Google Maps

Visualization of Tweets on Google Maps

Folks,

In this blog we will learn how to visualize tweets on Google Maps using R!

Twitter App required to get started with this. If you don’t have Twitter Application, go to Twitter Developer link and sign in with your credentials, after that go to twitter Apps & click on “Create New App”button for creating new application.

1.png

Once you’ve done this, make a note of your Keys & Access Token.

  • Consumer Key (API Key)
  • Consumer Secret (API Secret)
  • Access Token
  • Access Token Secret

R Packages required: 

install.packages(“ggmap”): It allows us to access maps from the Google Maps API.

ggmap package has a function get_map that can download maps from Google Maps API.

install.packages(“twitteR“): It provides an interface to the Twitter web API.

twitteR package has searchTwitter function that can search tweets based on a supplied search string.


Extracting Maps: 

Extraction of India map using get_map using below coordinates. Coordinates extracted using this Blog .

bounds.png

R Commands: 

7.png

I have selected the maptype = “terrain“, other maptype eg. satellite & roadmap.

Here is the output of ggmap(india_map).

8.png


Extracting Tweets: 

Twitter OAuth Setup:

2.png

Extracting 500 tweets having tag @narendramodi within a 1000 km radius of the given latitude/longitude.

20.593684° N, 78.96288° E is of India Location: Extracted using this Blog.

3.png

Here is the output of View(my_tweets_df). 500 Observation & 16 Variable.

4

5

Excluding rows from data frame where longitude/latitude=NA & taking only last two columns (unique data).

6.png

Here is the final output of View(my_tweets_df). 206 Observation & 2 Variable.

1111


Mapping of Tweets & Google Map: 

Converting both columns of my_tweets_df as numeric.

11

Mapping Map with my_tweets_df .

9.png

Here is the output of plot.

Seems like more tweets (having tag @narendramodi) are coming from Mumbai & Delhi.

10.png


Thanks!

Happy Learning!

Text Analysis – Facebook Post Comments

Text Analysis – Facebook Post Comments

Folks,

In this blog we will learn how to analyze the comments of Public Facebook post using Facebook Graph API Explorer & R!

Facebook developer account required to get started with this Facebook Graph API .

If you don’t have Facebook developer account, you can upgrade your personal Facebook account to a Facebook Developer account from here this link.

After registering as Facebook Developer, go to “Tools & Support”->”Graph API Explorer”

To explore Graph API – Token & Permissions are required, so just click on the “Get Token”.

1.png

~ Courtesy Facebook Developer

As public profiles included by default in permissions, just click on “Get Access Token”.

4.png

Below is my access token, which will expire after some time. As shown in token info.

5.png
Graph Explorer

Now we have token,  let’s explore now.


Extracting Comments from the Public Facebook Post.

First thing you required is the Post Id of the post. See below steps to get the post Id.

Suppose below is the post, we want to analyze. Click on the Post Date Time. See below highlighted box. ~ Post Courtesy Facebook

3.png

Copy below Id. This is the post Id.

2.png

Go to the Graph Explorer.

Type “Post_id/comments” in below box & click on Submit.

6.pngYou can also give limit for number of comments to return, like this

Post_id/comments?limit=”.

My Input:

7.png

Output: Below are post comments in the JSON format.

If you want more comments click on “next” for next page of comments.

7.png

Click on “Get Code” to get the cURL code. Copy this URL, we will use this URL in R.

8.pngText Analysis in R 

R Packages required: 

install.packages(“RCurl “): It allows us to compose general HTTP requests and provides convenient functions to fetch data.

install.packages(“rjson”): It allows us to converts JSON object into R objects and vice-versa.

install.packages(“tm”):  A Mining Package for text mining applications within R. It offers a number of transformations that ease the tedium of cleaning data.

R Commands:

111

url used in above image is copied from cURL code from Graph Explorer.

Output: So the first page give me 25 comments only. We will analyze here only 1st page i.e. 25 comments only.

output

Cleaning & Analyzing Data:

Creating corpus & removing extra spaces, special characters & other unwanted things.

cleaning.png

Creating Term Document Matrix:

1111.png

Here is the 760 extracted words with frequency.

1112.png

Creating Wordcloud: 

install.packages(“wordcloud”): For plotting a word cloud

lastt.png

In this Word Cloud we are taking only 100 words with minimum frequency of 2.

Output:

lasttttttt.png

Graph API Reference ~Facebook Developer  For more details, please read this Link.

Feedback and suggestions are most welcome. If you have any feedback, suggestions or questions please comment.


Thanks!

Happy Learning!

Exploring Instagram API using R!

Folks,

In this blog we will explore the basics of Instagram API using R.

“instaR” Package in R: Provides an interface to the Instagram API , which allows R users to access public users’ profile data.

Install “instaR” package from CRAN : install.packages(“instaR”)

Install “RCurl” package from CRAN : install.packages(“RCurl”)


Step 1: Registering an Application with Instagram.

If you already have an account with Instagram, go to Instagram Developer and register.

1.png

Click “Register your application” button. After you register as a Instagram developer, you can go to  a Manage clients & register a new client.

Note down your App Name, Client ID & Client Secret

2


Step 2: Create OAuth token to Instagram R session.

instaOAuth creates an OAuth access token that enables R to make authenticated calls to the Instagram API.

instaOAuth( client_id, client_secret, scope = “basic”)

Scope is related to the access permissions. Read Login Permissions (Scopes) Link

my_app_client_id  <- “”
my_app_client_secret <- “”

code 1

Now run the  instaOAuth command in R. See below format.

code 3

For setting the ‘redirect_uri’. Go to Manage Clients.

Copy and paste http://localhost:1410/ into ‘redirect_uri’ on Instagram App Settings.

2.png

After setting the redirect uri press enter on R command window. Automatic browser will open below page & ask for the Authorization.

r 1

After Authorizing the app. Below message will come.

r 2

See below message “Authentication Complete”. Now the connection is done with the Instagram API & R.

comp.PNG

Extracting the Token from OAuth. Just type below command.

code 5

The token can be saved as a file in disk to be re-used in future sessions.

saving auth

Now we have token = my_access_token. So here we goes with some of the basic end points of Instagram API.

Instagram End Points Links: Here


1) Getting the information from Instagram for token owner.

End Points: https://api.instagram.com/v1/users/self/?access_token=ACCESS-TOKEN

Using R

json1.PNG

Output:

11111111111.png

Using Postman API Client

End Points: https://api.instagram.com/v1/users/self/?access_token=ACCESS-TOKEN

Output: JSON/XML

pstman.png

2) Get the most recent media published on Instagram by token owner.

https://api.instagram.com/v1/users/self/media/recent/?access_token=ACCESS-TOKEN

Using Postman API Client

Output:

json3.png

Actual Image:

sunburn.png


Checking the details of any public image.

Hit this endpoint: https://api.instagram.com/oembed/?url=YOUR URL

See below gif for explanation. I’m using postman client for API testing.

output_wlPtEf.gif

You can get/post API request in your own way! Explore more End Points here: Link

PS: In Sandbox mode we cannot use extended permissions. Read Login Permissions (Scopes) here Link


Thanks!

Happy Learning!

Mining Facebook Data Using R & Facebook API!

Mining Facebook Data Using R & Facebook API!

Folks,

In this blog we will learn the basics of extracting Facebook data using R & Facebook API.

Rfacebook Package: Provides an interface to the Facebook API.

Rfacebook package in R provides functions that allow R to access Facebook’s API to get information about posts, comments, likes, group that mention specific keywords & much more.

Install “Rfacebook” package from CRAN : install.packages(“Rfacebook”) 


Step 1: Registering an Application with Facebook.

If you already have an account with Facebook, go to FacebookDeveloper and register.

Click “Register Now” button. After you register as a Facebook developer, you can register a new application.

Register a new application

From  FacebookDeveloper click on Apps at the top of the page to go to the application dashboard.

Click the fb-create-new-app-button button near the top. Once you are done with the verification process, your application is created. Note down the App Id & App Secret.

1
Application Dashboard

Site URL on Facebook App Settings: http://localhost:1410 

6.PNG
App Settings

Step 2: Create OAuth token to Facebook R session.

fbOAuth creates a long-lived OAuth access token that enables R to make authenticated calls to the Facebook API.

fbOAuth(app_id=””,app_secret=”YourAppSecret”)

4

5

Saving “my_oauth”as a file to be re-used in future sessions, which can be used as token in functions.

7.PNG

Now the connection is done with the Facebook API & R. So here we goes with some of the basic functions.

If you are getting below error while calling function, please go the bottom of this blog post for fix.

Error in callAPI(query, token) :
An active access token must be used to query information about the current user.

function getLikes

getLikes(user, n = n , token): Extract list of liked pages of a Facebook user with page id.

Arguments: user: user name/ID , n: Number of liked pages to return for user.

Use below command in R to get likes.

8.PNG

Here is the my result data set “my_likes”. With three variables: Id, names & website of pages.

view.PNG

9.PNG
Pages Liked

function getPage 

getPage(page , token, n = n): Extract list of posts from a public Facebook page.

Arguments: page: Page ID or page name, n : Number of posts to return for page.

Example: Extracting 10 posts of Facebook Page “Narendra Modi”. 

Facebook Page Id of this page you can get from this link: http://findmyfbid.com/ . See below the gif.

output_LJA4Du.gif

After getting the page id use below command in R to get posts.

getpage.PNG

Here is my result data set “getpagedata”. With 10 Observation & 10 Variables. Variables like Post with likes_count, share_count & comments_count etc.

padess.PNG

1a

1b


function search_groups

searchGroup(“text”,token, n = n): Find any group with its privacy status & Facebook ID.

Arguments: text: text string, n : Number of groups to return.

Use below command in R to search groups.

groups.PNG

Here is my result data set “search_groups”. With many observation & 3 Variables.

11.png
Groups


function getGroup

getGroup(ID, token, n = n): Extract list of posts from a public Facebook page. Whose privacy is open.

Arguments: ID: Group ID , n: Number of posts to return for group.

Example: We have to extract 10 posts from 7th group “Web Scraping and Data mining” present in above image groups.

Use below command in R to get groups.

12

Here is my result data set. With 10 Observation & 3 Variables.

postttttttttt

12a


function searchPages

searchPages(, token, n = n): It Search pages that having a string/keyword.

Arguments: string: any string , n: Number of pages to return

Example: We have to search 10 pages that mention a string “Sports”.

Use below command in R to search pages.

sports.PNG

Here is my result data set with 10 Observation & 16 Variables.

s1

s2


function updateStatus

updateStatus(“text”, token)

Arguments: text: any string , token

Use below command in R to update Facebook status.

status1.PNG

Result:

status2


 

Fix for Error in callAPI:

If you are getting below error, please follow below steps to fix this issue.

Error in callAPI(query, token) :
An active access token must be used to query information about the current user.

Run Below Function First :-

fbOAuth <- function(app_id, app_secret, extended_permissions=FALSE, legacy_permissions=FALSE, scope=NULL)
{
  ## getting callback URL
  full_url <- oauth_callback()
  full_url <- gsub("(.*localhost:[0-9]{1,5}/).*", x=full_url, replacement="\\1")
  message <- paste("Copy and paste into Site URL on Facebook App Settings:",
                   full_url, "\nWhen done, press any key to continue...")
  ## prompting user to introduce callback URL in app page
  invisible(readline(message))
  ## a simplified version of the example in httr package
  facebook <- oauth_endpoint(
    authorize = "https://www.facebook.com/dialog/oauth",
    access = "https://graph.facebook.com/oauth/access_token") 
  myapp <- oauth_app("facebook", app_id, app_secret)
  if (is.null(scope)) {
    if (extended_permissions==TRUE){
      scope <- c("user_birthday", "user_hometown", "user_location", "user_relationships",
                 "publish_actions","user_status","user_likes")
    }
    else { scope <- c("public_profile", "user_friends")}
  
    if (legacy_permissions==TRUE) {
      scope <- c(scope, "read_stream")
    }
  }

  if (packageVersion('httr') < "1.2"){
    stop("Rfacebook requires httr version 1.2.0 or greater")
  }

  ## with early httr versions
  if (packageVersion('httr') <= "0.2"){
    facebook_token <- oauth2.0_token(facebook, myapp,
                                     scope=scope)
    fb_oauth <- sign_oauth2.0(facebook_token$access_token)
    if (GET("https://graph.facebook.com/me", config=fb_oauth)$status==200){
      message("Authentication successful.")
    }
  }

  ## less early httr versions
  if (packageVersion('httr') > "0.2" & packageVersion('httr') <= "0.6.1"){
    fb_oauth <- oauth2.0_token(facebook, myapp,
                               scope=scope, cache=FALSE) 
    if (GET("https://graph.facebook.com/me", config(token=fb_oauth))$status==200){
      message("Authentication successful.")
    } 
  }

  ## httr version from 0.6 to 1.1
  if (packageVersion('httr') > "0.6.1" & packageVersion('httr') < "1.2"){
    Sys.setenv("HTTR_SERVER_PORT" = "1410/")
    fb_oauth <- oauth2.0_token(facebook, myapp,
                               scope=scope, cache=FALSE) 
    if (GET("https://graph.facebook.com/me", config(token=fb_oauth))$status==200){
      message("Authentication successful.")
    } 
  }

  ## httr version after 1.2
  if (packageVersion('httr') >= "1.2"){
    fb_oauth <- oauth2.0_token(facebook, myapp,
                               scope=scope, cache=FALSE) 
    if (GET("https://graph.facebook.com/me", config(token=fb_oauth))$status==200){
      message("Authentication successful.")
    } 
  }

  ## identifying API version of token
  error <- tryCatch(callAPI('https://graph.facebook.com/pablobarbera', fb_oauth),
                    error = function(e) e)
  if (inherits(error, 'error')){
    class(fb_oauth)[4] <- 'v2'
  }
  if (!inherits(error, 'error')){
    class(fb_oauth)[4] <- 'v1'
  }

  return(fb_oauth)
}

111

script.png

Fix Credit goes to (www.listendata.com/) Visit this link for more information.


Thanks!

Happy Learning!