Scheduling R Script using Windows Task Scheduler

Scheduling R Script using Windows Task Scheduler

Folks,

In this blog we will learn how to schedule R Script using Windows Task Scheduler!


Suppose here is the R Script which we want to schedule.

Output of this script: Create a log file of name “R_Scripts_Logs_<Time stamp>.TXT” & having text “Script successfully invoked by scheduler at <Time stamp>” .

1

Create a batch file having below commands:

@echo off
“Location of R.exe” CMD BATCH “Location of R Script”

Example: 

2.png


Steps for scheduling R Script using Windows Task Scheduler:

Open Window Task Scheduler & Create Basic Task

3

Step 1: Provide your task name & description.step-1

Step 2: Provide trigger timings details as per your requirement.

step-2

step-3

Step 3: In Action:- provide the location of the batch file of the script. After that save the task.

step-5step-6

Task has been scheduled.

step8

At 22:05, task triggered the batch file automatically.

Here is the output of that script:

outut

When you run R script using CMD Batch, one ROUT file also generated automatically having details of script commands & time elapsed of script.

Below is ROUT fie of the script my R Script.R

output


Thanks!

Happy Learning! Your feedback would be appreciated!

Advertisements

Visualization of Tweets on Google Maps

Visualization of Tweets on Google Maps

Folks,

In this blog we will learn how to visualize tweets on Google Maps using R!

Twitter App required to get started with this. If you don’t have Twitter Application, go to Twitter Developer link and sign in with your credentials, after that go to twitter Apps & click on “Create New App”button for creating new application.

1.png

Once you’ve done this, make a note of your Keys & Access Token.

  • Consumer Key (API Key)
  • Consumer Secret (API Secret)
  • Access Token
  • Access Token Secret

R Packages required: 

install.packages(“ggmap”): It allows us to access maps from the Google Maps API.

ggmap package has a function get_map that can download maps from Google Maps API.

install.packages(“twitteR“): It provides an interface to the Twitter web API.

twitteR package has searchTwitter function that can search tweets based on a supplied search string.


Extracting Maps: 

Extraction of India map using get_map using below coordinates. Coordinates extracted using this Blog .

bounds.png

R Commands: 

7.png

I have selected the maptype = “terrain“, other maptype eg. satellite & roadmap.

Here is the output of ggmap(india_map).

8.png


Extracting Tweets: 

Twitter OAuth Setup:

2.png

Extracting 500 tweets having tag @narendramodi within a 1000 km radius of the given latitude/longitude.

20.593684° N, 78.96288° E is of India Location: Extracted using this Blog.

3.png

Here is the output of View(my_tweets_df). 500 Observation & 16 Variable.

4

5

Excluding rows from data frame where longitude/latitude=NA & taking only last two columns (unique data).

6.png

Here is the final output of View(my_tweets_df). 206 Observation & 2 Variable.

1111


Mapping of Tweets & Google Map: 

Converting both columns of my_tweets_df as numeric.

11

Mapping Map with my_tweets_df .

9.png

Here is the output of plot.

Seems like more tweets (having tag @narendramodi) are coming from Mumbai & Delhi.

10.png


Thanks!

Happy Learning!

Text Analysis – Facebook Post Comments

Text Analysis – Facebook Post Comments

Folks,

In this blog we will learn how to analyze the comments of Public Facebook post using Facebook Graph API Explorer & R!

Facebook developer account required to get started with this Facebook Graph API .

If you don’t have Facebook developer account, you can upgrade your personal Facebook account to a Facebook Developer account from here this link.

After registering as Facebook Developer, go to “Tools & Support”->”Graph API Explorer”

To explore Graph API – Token & Permissions are required, so just click on the “Get Token”.

1.png

~ Courtesy Facebook Developer

As public profiles included by default in permissions, just click on “Get Access Token”.

4.png

Below is my access token, which will expire after some time. As shown in token info.

5.png
Graph Explorer

Now we have token,  let’s explore now.


Extracting Comments from the Public Facebook Post.

First thing you required is the Post Id of the post. See below steps to get the post Id.

Suppose below is the post, we want to analyze. Click on the Post Date Time. See below highlighted box. ~ Post Courtesy Facebook

3.png

Copy below Id. This is the post Id.

2.png

Go to the Graph Explorer.

Type “Post_id/comments” in below box & click on Submit.

6.pngYou can also give limit for number of comments to return, like this

Post_id/comments?limit=”.

My Input:

7.png

Output: Below are post comments in the JSON format.

If you want more comments click on “next” for next page of comments.

7.png

Click on “Get Code” to get the cURL code. Copy this URL, we will use this URL in R.

8.pngText Analysis in R 

R Packages required: 

install.packages(“RCurl “): It allows us to compose general HTTP requests and provides convenient functions to fetch data.

install.packages(“rjson”): It allows us to converts JSON object into R objects and vice-versa.

install.packages(“tm”):  A Mining Package for text mining applications within R. It offers a number of transformations that ease the tedium of cleaning data.

R Commands:

111

url used in above image is copied from cURL code from Graph Explorer.

Output: So the first page give me 25 comments only. We will analyze here only 1st page i.e. 25 comments only.

output

Cleaning & Analyzing Data:

Creating corpus & removing extra spaces, special characters & other unwanted things.

cleaning.png

Creating Term Document Matrix:

1111.png

Here is the 760 extracted words with frequency.

1112.png

Creating Wordcloud: 

install.packages(“wordcloud”): For plotting a word cloud

lastt.png

In this Word Cloud we are taking only 100 words with minimum frequency of 2.

Output:

lasttttttt.png

Graph API Reference ~Facebook Developer  For more details, please read this Link.

Feedback and suggestions are most welcome. If you have any feedback, suggestions or questions please comment.


Thanks!

Happy Learning!

Geocoding in R!

Folks,

In this blog we will explore the basics of Geocoding using R,  using Google Maps Geocoding API & R!

Geocoding: Process of transforming a description of a location (address, name of a place, or coordinates) to a location on the earth’s surface.

Forward Geocoding: Converting address/name of a place into geographic coordinates.

Reverse Geocoding: Converting geographic coordinates into readable address.

R Package required: 

install.packages(“RCurl ”): It allows us to compose general HTTP requests and provides convenient functions to fetch data.

install.packages(“rjson”): It allows us to converts JSON object into R objects and vice-versa.


Forward Geocoding using R

Google Maps Geocoding API:

11

Where output may be either of JSON/XML & parameters = address or components.

Example for JSON:

https://maps.googleapis.com/maps/api/geocode/json?address=India+Gate,+New+Delhi,+India&components=country:IN

Where address = “India Gate, New Delhi, India”, components = Country “IN” for India.

Component Country: country name or a two letter country code. For more components like locality & postal code etc. details read this link.

R Commands:

url<- “”

jdata<- getURL(url) # Fetching JSON data.

rdata<- fromJSON(jdata) # Convert a JSON object into an R object.

Output:

1

Geographic coordinates present in  rdata results.

2

Extracting geographic coordinates from the rdata.

3.png

PS: These are approx geographic coordinates, as it depends on the address provided in URL.


Reverse Geocoding using R

Google Maps Reverse Geocoding API:

12

Where output may be either of JSON/XML. Parameters:  ,

R Commands:

url<- “”

jdata<- getURL(url) # Fetching JSON data.

rdata<- fromJSON(jdata) # Convert a JSON object into an R object.

For Example taking latitude & longitude 28.632455, 77.217218 – Courtesy: Google Maps

4.png

Output:

7.png

5

For more details please read this google link.


Thanks!

Happy Learning!

Accessing Database (MS SQL Server/Oracle/MySQL) from R!

Folks,

In this blog we will learn how to access Oracle, MySQL & SQL Server databases from R!


Accessing Oracle Database using RJDBC Package

Below is my database hosted on AWS Cloud. See here how to setup Oracle Db on cloud (free tier).

11

R Packages required:-  install.packages(“RJDBC”)

Also download Oracle JDBC Drivers from here Link. After downloading, copy file location.

location.png

Commands:


driver <- JDBC("oracle.jdbc.driver.OracleDriver"," <paste JDBC Driver file location>")

dbConnect(driver , "jdbc:oracle:thin:@Host:Port:ServiceName or SID", "<username>", "<password>")

code1

R users have few more choices to access their Oracle Database: Using RODBC and ROracle packages.


MySQL Database

Here is my database hosted on AWS Cloud. See here how to setup MySQL Db on cloud (free tier).

1

a) Accessing MySQl Database using ODBC:

R Packages required:-  install.packages(“RODBC”)

For Windows: Go to start, search ODBC. Follow below steps of setup.

1

Click on “Add”.

2

Select MySQL ODBC Driver. If not present, then download using this link.

3

Provide your server details, username & password. Test your connection. Note down Data Source name.

4

Commands:


my_conn <- odbcConnect("<ODBC Data Source Name >")

sqlQuery(my_conn, paste("<Query>"))

code

b) Accessing MySQL DB using RMySQL Package:

R Packages required:-  install.packages(“RMySQL”)

Commands:


my_conn <- dbConnect(MySQL(), user="<username>", password="<password>", database= "<Database name>", host="<your host>")

result<- dbSendQuery(my_conn, "<Query>")

dbFetch(result)

code 2.png


Accessing SQL Server Database using ODBC

Here is my SQL Server database hosted locally.

server.png

a) Accessing SQL Server using ODBC:

For Windows: Go to start, search ODBC.

1

Follow below steps to setup ODBC datasource.

output_KUL4JC.gif

Commands:


my_conn <- odbcConnect("<Data Source Name>", uid="<username>", pwd="<password>")

sqlQuery(my_conn, "<Query>")

code2.png

b) Accessing SQL Server DB using RJDBC Package:

Suppose SQL Server database hosted on cloud. We can use below commands to connect.

R Packages required:-  install.packages(“RJDBC”) & Download SQL JDBC Drivers file from internet.


driver <- JDBC("com.microsoft.sqlserver.jdbc.SQLServerDriver",
"<location of SQL JDBC Drivers file on computer>")

conn <- dbConnect(driver , "jdbc:sqlserver://<serverName>", "<userID>", "<password>")

result <- dbGetQuery(conn, "<Query>")


Thanks!

Happy Learning!

Mining Facebook Data Using R & Facebook API!

Mining Facebook Data Using R & Facebook API!

Folks,

In this blog we will learn the basics of extracting Facebook data using R & Facebook API.

Rfacebook Package: Provides an interface to the Facebook API.

Rfacebook package in R provides functions that allow R to access Facebook’s API to get information about posts, comments, likes, group that mention specific keywords & much more.

Install “Rfacebook” package from CRAN : install.packages(“Rfacebook”) 


Step 1: Registering an Application with Facebook.

If you already have an account with Facebook, go to FacebookDeveloper and register.

Click “Register Now” button. After you register as a Facebook developer, you can register a new application.

Register a new application

From  FacebookDeveloper click on Apps at the top of the page to go to the application dashboard.

Click the fb-create-new-app-button button near the top. Once you are done with the verification process, your application is created. Note down the App Id & App Secret.

1
Application Dashboard

Site URL on Facebook App Settings: http://localhost:1410 

6.PNG
App Settings

Step 2: Create OAuth token to Facebook R session.

fbOAuth creates a long-lived OAuth access token that enables R to make authenticated calls to the Facebook API.

fbOAuth(app_id=””,app_secret=”YourAppSecret”)

4

5

Saving “my_oauth”as a file to be re-used in future sessions, which can be used as token in functions.

7.PNG

Now the connection is done with the Facebook API & R. So here we goes with some of the basic functions.

If you are getting below error while calling function, please go the bottom of this blog post for fix.

Error in callAPI(query, token) :
An active access token must be used to query information about the current user.

function getLikes

getLikes(user, n = n , token): Extract list of liked pages of a Facebook user with page id.

Arguments: user: user name/ID , n: Number of liked pages to return for user.

Use below command in R to get likes.

8.PNG

Here is the my result data set “my_likes”. With three variables: Id, names & website of pages.

view.PNG

9.PNG
Pages Liked

function getPage 

getPage(page , token, n = n): Extract list of posts from a public Facebook page.

Arguments: page: Page ID or page name, n : Number of posts to return for page.

Example: Extracting 10 posts of Facebook Page “Narendra Modi”. 

Facebook Page Id of this page you can get from this link: http://findmyfbid.com/ . See below the gif.

output_LJA4Du.gif

After getting the page id use below command in R to get posts.

getpage.PNG

Here is my result data set “getpagedata”. With 10 Observation & 10 Variables. Variables like Post with likes_count, share_count & comments_count etc.

padess.PNG

1a

1b


function search_groups

searchGroup(“text”,token, n = n): Find any group with its privacy status & Facebook ID.

Arguments: text: text string, n : Number of groups to return.

Use below command in R to search groups.

groups.PNG

Here is my result data set “search_groups”. With many observation & 3 Variables.

11.png
Groups


function getGroup

getGroup(ID, token, n = n): Extract list of posts from a public Facebook page. Whose privacy is open.

Arguments: ID: Group ID , n: Number of posts to return for group.

Example: We have to extract 10 posts from 7th group “Web Scraping and Data mining” present in above image groups.

Use below command in R to get groups.

12

Here is my result data set. With 10 Observation & 3 Variables.

postttttttttt

12a


function searchPages

searchPages(, token, n = n): It Search pages that having a string/keyword.

Arguments: string: any string , n: Number of pages to return

Example: We have to search 10 pages that mention a string “Sports”.

Use below command in R to search pages.

sports.PNG

Here is my result data set with 10 Observation & 16 Variables.

s1

s2


function updateStatus

updateStatus(“text”, token)

Arguments: text: any string , token

Use below command in R to update Facebook status.

status1.PNG

Result:

status2


 

Fix for Error in callAPI:

If you are getting below error, please follow below steps to fix this issue.

Error in callAPI(query, token) :
An active access token must be used to query information about the current user.

Run Below Function First :-

fbOAuth <- function(app_id, app_secret, extended_permissions=FALSE, legacy_permissions=FALSE, scope=NULL)
{
  ## getting callback URL
  full_url <- oauth_callback()
  full_url <- gsub("(.*localhost:[0-9]{1,5}/).*", x=full_url, replacement="\\1")
  message <- paste("Copy and paste into Site URL on Facebook App Settings:",
                   full_url, "\nWhen done, press any key to continue...")
  ## prompting user to introduce callback URL in app page
  invisible(readline(message))
  ## a simplified version of the example in httr package
  facebook <- oauth_endpoint(
    authorize = "https://www.facebook.com/dialog/oauth",
    access = "https://graph.facebook.com/oauth/access_token") 
  myapp <- oauth_app("facebook", app_id, app_secret)
  if (is.null(scope)) {
    if (extended_permissions==TRUE){
      scope <- c("user_birthday", "user_hometown", "user_location", "user_relationships",
                 "publish_actions","user_status","user_likes")
    }
    else { scope <- c("public_profile", "user_friends")}
  
    if (legacy_permissions==TRUE) {
      scope <- c(scope, "read_stream")
    }
  }

  if (packageVersion('httr') < "1.2"){
    stop("Rfacebook requires httr version 1.2.0 or greater")
  }

  ## with early httr versions
  if (packageVersion('httr') <= "0.2"){
    facebook_token <- oauth2.0_token(facebook, myapp,
                                     scope=scope)
    fb_oauth <- sign_oauth2.0(facebook_token$access_token)
    if (GET("https://graph.facebook.com/me", config=fb_oauth)$status==200){
      message("Authentication successful.")
    }
  }

  ## less early httr versions
  if (packageVersion('httr') > "0.2" & packageVersion('httr') <= "0.6.1"){
    fb_oauth <- oauth2.0_token(facebook, myapp,
                               scope=scope, cache=FALSE) 
    if (GET("https://graph.facebook.com/me", config(token=fb_oauth))$status==200){
      message("Authentication successful.")
    } 
  }

  ## httr version from 0.6 to 1.1
  if (packageVersion('httr') > "0.6.1" & packageVersion('httr') < "1.2"){
    Sys.setenv("HTTR_SERVER_PORT" = "1410/")
    fb_oauth <- oauth2.0_token(facebook, myapp,
                               scope=scope, cache=FALSE) 
    if (GET("https://graph.facebook.com/me", config(token=fb_oauth))$status==200){
      message("Authentication successful.")
    } 
  }

  ## httr version after 1.2
  if (packageVersion('httr') >= "1.2"){
    fb_oauth <- oauth2.0_token(facebook, myapp,
                               scope=scope, cache=FALSE) 
    if (GET("https://graph.facebook.com/me", config(token=fb_oauth))$status==200){
      message("Authentication successful.")
    } 
  }

  ## identifying API version of token
  error <- tryCatch(callAPI('https://graph.facebook.com/pablobarbera', fb_oauth),
                    error = function(e) e)
  if (inherits(error, 'error')){
    class(fb_oauth)[4] <- 'v2'
  }
  if (!inherits(error, 'error')){
    class(fb_oauth)[4] <- 'v1'
  }

  return(fb_oauth)
}

111

script.png

Fix Credit goes to (www.listendata.com/) Visit this link for more information.


Thanks!

Happy Learning!