How I implemented googleSignIn in R (shiny) and lived
(This article was first published on Roman Luštrik not Inc.  R, and kindly contributed to Rbloggers)
Known user identity when building shiny apps can sometimes come really handy. While you can implement your own user login, for instance using cookies, you can also use some of the services which authenticate a user for you, such as Google. This way, you don’t have to handle cookies or passwords, just a small part of bureaucracy in your database.
Enter googleAuthR. You can use the Google Authentication system, which enables you to call its APIs (e.g. see here), but sometimes a user login is just enough. This is where the accompanying module googleSignIn comes into play. Below is how I was able to implement this same name module with four lines of code, one line being loading the aforementioned library.
If you would be kind enough to scroll to the googleSignIn part of the page, you will notice code for a shiny app that (almost) just works. Read on to see what I mean by “almost”.
The demo app has all the key components, it loads libraries, prepares options("client_id"), creates sign in button in ui (googleSignInUI("demo")), does the auth dance for you in the server part (shiny::callModule(googleSignIn, "demo")) when this button is pressed and, as gratis, displays the name, email and your google image. I run the app using runApp(port = 1221)!
When I press the Sign in button, I get – bleh. Doesn’t work.
What is going on? The google console parameters check out.
Let me save you hours of clicking by pointing out this little bit from the documentation from the googleAuth module:
… then make sure if you launch your app locally to change the ip address from 127.0.0.1 to localhost in your browser (Google doesn’t accept ip addresses).
In other words, in your web browser, just change http://127.0.0.1:1221 to http://localhost:1221. The page should reload on its own. Click the sign in button et voilà.
var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));To leave a comment for the author, please follow the link and comment on their blog: Roman Luštrik not Inc.  R. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
R and Python: Using reticulate to get the best of both worlds
(This article was first published on rbloggers – STATWORX, and kindly contributed to Rbloggers)
It’s March 15th and that means it’s World Sleep Day (WSD). Don’t snooze off just yet! We’re about to check out a package that can make using R and Python a dream. It’s called reticulate and we’ll use it to train a Support Vector Machine for a simple classification task.
What’s WSD got do with data science?I discovered WSD on the train to STATWORX HQ in Frankfurt. If you’ve travelled by train before, you know that the only thing worse than disgruntled staff is unconscious staff. It hasn’t happened to me yet, but hey let’s face it: the personnel shortage of Deutsche Bahn isn’t getting better any time soon.
This brings me to the topic of today’s blog post: the sleeping habits of railroad workers. For this, I pulled a dataset from the US Federal Railroad Administration (FRA). The FRA conducted a survey on the work and sleep
schedules of its train dispatchers. You can find it here.
As the title suggests, we’re going to use both R and Python to predict whether a dispatcher was diagnosed with a sleeping disorder. Before we get started, a
warning to all R and Python purists out there: the example below is a bit contrived. Everything we’re about to do can be done entirely in either one of the languages.
As data scientists, we ideally have a firm grasp on both R and Python. After all, as my colleague Fran will discuss in an upcoming post, both have unique strengths that we can use to our advantage. Personally, I love R for the tidyverse and ggplot2, Python for its unified machine learning (ML) API scikitlearn.
Alright, point taken you might say, but it’s a hassle to switch back and forth. It’s not worth it, and until a few years ago you would have been right! However, nowadays there are some cool ways to get the best of both worlds. If you’re coming from the R community look no further than reticulate!
The reticulate package gives you a set of tools to use both R and Python interactively within an R session. Say you’re working in Python and need a specialized statistical model from an R package – or you’re working in R and want to access Python’s ML capabilities. Well, you’ve come to the right place. This package is a godsend for tasks where it’s required, or simply more convenient, to use both languages as part of your workflow.
Now, there are different ways to use R and Python interactively and I encourage you to check reticulate’s github site to see which one suits you best. In this post, we’re going through a simple example of how to use Python modules within an R Notebook (i.e. Markdown document).
How to get started?If you’re working on a Mac, Python comes preinstalled on your system. Irrespective of the machine you’re on though, I recommended downloading the Anaconda distribution, so you have everything you need. One more note: you need RStudio’s newest preview version 1.2 for this to work. That’s it!
Let’s open an R Notebook, insert an R chunk and (install and) load the reticulate library. Immediately after loading reticulate, use the use_python() command with the appropriate path. Placing it later in the script causes problems for some people. I specified my path as follows (note: yours may differ):
library(tidyverse) library(recipes) library(reticulate) use_python("/anaconda3/bin/python")We’re good to go. Let’s read in the data and perform some transformations with dplyr. This is mostly recoding work. As you can see in the select command, we pick a handful of variables like sex, age, caffeine consumption, health and stress to predict whether a railroad dispatcher was diagnosed with a sleeping disorder.
1) Read in the data data < readxl::read_xls("sleep.xls") sleep < data %>% select( Diagnosed_Sleep_disorder, Age_Group, Sex, Total_years_dispatcher, Total_years_present_job, Marital_Status, Childrendependents, Children_under_2_yrs, Caff_Beverages, Sick_Days_in_last_year, Health_status, Avg_Work_Hrs_Week, FRA_report, Phys_Drained, Mentally_Drained, Alert_at_Work, Job_Security ) %>% rename_all(tolower) %>% mutate_if(is.character, as.numeric) %>% mutate_at(vars(diagnosed_sleep_disorder, sex, caff_beverages, fra_report), ~ (.  2)) %>% mutate_at(vars(marital_status), ~ (.  1)) %>% drop_na()Now that we have the variables we want, it’s time to get the data into the right shape. Here’s one more reason to love R: the recipes package. If you’re not familiar with it, check it out. You may find its workflow a bit peculiar at first, but once you get used to it, it makes data preparation a breeze.
What we’re doing here is straightforward. First, we split the data into a training and test set. Next, we specify a data preparation recipe, which consists of three steps: one hot encoding factor predictors, standardising numeric predictors and downsampling the data. One hot encoding and standardising ensure that the Support Vector Machine algorithm works properly. Downsampling is a countermeasure against the class imbalance in our dataset.
2) Prepare the data numeric_variables < c( "total_years_dispatcher", "total_years_present_job", "childrendependents", "children_under_2_yrs", "sick_days_in_last_year", "avg_work_hrs_week" ) factor_variables < setdiff(colnames(sleep), numeric_variables) sleep < mutate_at(sleep, vars(factor_variables), as.factor) set.seed(2019) index < sample(1:nrow(sleep), floor(nrow(sleep) * .75)) sleep_train < sleep[index, ] sleep_test < sleep[index, ] recipe_formula < recipes::recipe(diagnosed_sleep_disorder ~ ., sleep_train) recipe_steps < recipe_formula %>% recipes::step_dummy(factor_variables, all_outcomes(), one_hot = TRUE) %>% recipes::step_downsample(diagnosed_sleep_disorder) %>% recipes::step_center(numeric_variables) %>% recipes::step_scale(numeric_variables) recipe_prep < recipes::prep(recipe_steps, sleep_train, retain = TRUE) training_data < juice(recipe_prep) testing_data < bake(recipe_prep, sleep_test)Now comes the part where Python shines: its unified ML library scikitlearn. Let’s go ahead and import the Support Vector Machine (SVM) classifier as well as some other modules to tune and evaluate our model.
SVM is a supervised learning algorithm. It works by finding a hyperplane in an Ndimensional space, which separates two (or more) classes as cleanly as possible. More technically, SVM maximizes the margin or the distance between the separating hyperplane and the closest data points. This is why SVM is also called a maximum margin estimator.
SVM is mostly used for classification, but it can do regression too. The upside is that it works with highdimensional data and different kernel functions, meaning it can flexibly adapt to different types of data. Its downside is that computation becomes costly with large datasets and that it reacts sensitively to hyperparameters. Still, for some applications SVM performs quite competitively.
Combining SVM with kernels allows us to project our data into a higherdimensional space. The point of this is to make the classes better separable. In our example here, we’ll use a simple linear and a radial basis function kernel. The latter can map the predictor space into infinite dimensions.
3) Train the model import numpy as np import pandas as pd import seaborn as sns import matplotlib.pyplot as plt from sklearn import svm from sklearn.model_selection import GridSearchCV, cross_val_score from sklearn.metrics import classification_report, confusion_matrix, accuracy_score y_train = r.training_data['diagnosed_sleep_disorder'] X_train = r.training_data.drop('diagnosed_sleep_disorder', axis = 1) y_test = r.testing_data['diagnosed_sleep_disorder'] X_test = r.testing_data.drop('diagnosed_sleep_disorder', axis = 1) clf = svm.SVC(kernel = 'linear') clf.fit(X_train, y_train) y_pred = clf.predict(X_test) print(confusion_matrix(y_test, y_pred)) print(classification_report(y_test, y_pred)) clf = svm.SVC(kernel = 'RBF') clf.fit(X_train, y_train) y_pred = clf.predict(X_test) print(confusion_matrix(y_test, y_pred)) print(classification_report(y_test, y_pred))We can tune an SVM with the help of two parameters: C and gamma. C is a regularisation parameter that represents how much error we’re willing to tolerate. The higher C, the stricter we are, the more exactly SVM will try to fit the data by picking a hyperplane with smaller margins. As a consequence, higher C’s carry a higher risk of overfitting. For the radial basis function kernel, we can also tune gamma which determines the size of the kernel and with it the decision boundary. The higher gamma, the closer the decision boundary is to single training examples. A higher gamma can lead to a better model, yet likewise increases the risk of overfitting.
4) Tune the model param_grid = [{'C': [0.01, 0.1, 1, 10, 100], 'gamma': [0.001, 0.01, 0.1, 1, 10], 'kernel': ['RBF', 'linear']}] grid = GridSearchCV(svm.SVC(), param_grid, cv = 5, scoring = 'balanced_accuracy') grid.fit(X_train, y_train) print(grid.best_params_)From the crossvalidation procedure, it appears that a gamma of 0.1 and a C of 0.01 are optimal in combination with a radial basis function kernel. So, let’s train our model with this kernel and the hyperparameter values.
5) Evaluate the accuracy of the model clf = grid.best_estimator_ y_pred = clf.predict(X_test) print('Confusion Matrix:\n\n', confusion_matrix(y_test, y_pred)) print('\nClassification Report:\n\n', classification_report(y_test, y_pred)) print('\nTraining Set Accuracy: {:.2f}%'.format(clf.score(X_train, y_train))) print('\nTest Set Accuracy: {:.2f}%'.format(clf.score(X_test, y_test))) conf_mat = confusion_matrix(y_test, y_pred) sns.heatmap(conf_mat, square = True, annot = True, fmt = 'g', cbar = False, cmap = 'viridis') plt.xlabel('predicted') plt.ylabel('observed') plt.show()In this case, we achieve a training set accuracy of 93 per cent and a test set accuracy of 74 per cent. This suggests that some overfitting has occurred. To achieve a higher accuracy (or better: sensitivity/recall), we could experiment with different kernels and/or hyperparameter values. But this I’d leave up to you. With reticulate, you now have a tool to get the best of both R and Python.
About the author Manuel Tilgner ABOUT USSTATWORX
is a consulting company for data science, statistics, machine learning and artificial intelligence located in Frankfurt, Zurich and Vienna. Sign up for our NEWSLETTER and receive reads and treats from the world of data science and AI.
Sign Up Now!
.button {backgroundcolor: #0085af;}
Der Beitrag R and Python: Using reticulate to get the best of both worlds erschien zuerst auf STATWORX.
var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));To leave a comment for the author, please follow the link and comment on their blog: rbloggers – STATWORX. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
Software Dependencies and Risk
(This article was first published on R – WinVector Blog, and kindly contributed to Rbloggers)
Dirk Eddelbuettel just shared an important point on software and analyses: dependencies are hard to manage risks.
If your software or research depends on many complex and changing packages, you have no way to establish your work is correct. This is because to establish the correctness of your work, you would need to also establish the correctness of all of the dependencies. This is worse than having nonreproducible research, as your work may have in fact been wrong even the first time.
Low dependencies and low complexity dependencies can also be wrong, but in this case there at least exists the possibility of checking things or running down and fixing issues.
This one reason we at WinVector LLC have been working on lowdependency R packages for data analysis. We don’t intend on controlling the whole analysis stack (that would be unethical), but we do intend to be in good position to fix things for our partners and clients. The bulk of our system’s utility comes from external systems such as R itself, the data.table package, and Rcpp. So we must (and hopefully do) give credit and thanks.
Also, not all dependencies are equal. So we have had to avoid some popular packages with unstable APIs (a history of breaking changes) and high historic error rates (a history of complexity and adding features over fixing things).
Again, dependencies are but one measure of quality and at best an approximation. But let’s take a look at some of our packages through this lens.
And almost to make the point our package where we relaxed the above discipline right now has CRANflagged issue (“significant warnings”), that we can not fix as the issue is in fact from one of the dependencies.
The issue is likely from ggplot2, which itself is likely picking up issues and errors from dplyr, tibble, and rlang (a few of ggplot2‘s dependencies that currently have detected, yet unfixed issues on CRAN). And these packages are likely picking up issues from their direct and indirect dependencies.
Now these issues are probably not serious, as if they were there would be a great panic motivating teams to fix them (this is a neat example of survivorship bias, visible acute problems attract enough attention to be fixed quickly but often subtle chronic issues can live a long time). But the point is: we have no lever to fix them on our end.
var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));To leave a comment for the author, please follow the link and comment on their blog: R – WinVector Blog. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
The Clash of the Titans in Test and ODI cricket
(This article was first published on R – Giga thoughts …, and kindly contributed to Rbloggers)
Who looks outside, dreams; who looks inside, awakes.
Show me a sane man and I will cure him for you.
We’re made of star stuff. We are a way for the cosmos to know itself.
If you want to make an apple pie from scratch, you must first create the universe.
The biggest nag in the collective psyche of cricketing fraternity these days, is whether Virat Kohli has surpassed Sachin Tendulkar. This question has been troubling cricket lovers the world over and particularly in India, for quite a while. This nagging question has only grown stronger with Kohli’s 41st ODI century and with Michael Vaughan bestowing the GOAT title to Virat Kohli for ODI cricket. Hence, I decided to do my bit in addressing this, by doing analysis of Kohli’s and Tendulkar’s performance in ODI cricket. I also wanted to address the the best among the cricketing idols of India in Test cricket, namely Sunil Gavaskar, Sachin Tendulkar and Virat Kohli. Hence this post has 2 parts
 Analysis of Tendulkar, Gavaskar and Kohli in Test cricket
 Analysis of Tendulkar and Kohli in ODIs
In this post, I analyze the performances of these titans in Test and ODI cricket using my R package cricketr. While some may feel that comparisons are not possible as these batsmen are from different eras. To some extent this is true. I would give some leeway to Gavaskar as he had to bat in a prehelmet era. But with Tendulkar and Kohli a fair and objective comparison is possible. There were preeminient bowlers in the times of Tendulkar as there are now.
From the analysis below, it can be seen that Tendulkar to of everybody else in Test cricket. However it must be noted that Tendulkar’s performance deteriorated towards the end of his career. Such was not the case with Gavaskar. Kohli has some catching up to do and he still has a lot of Test cricket in him.
In ODI Kohli can be seen to pulling ahead of Tendulkar in several aspects.
My R package cricketr can be installed directly from CRAN and you can use it analyze cricketers.
This package uses the statistics info available in ESPN Cricinfo Statsguru. The current version of this package supports all formats of the game including Test, ODI and Twenty20 versions.
You should be able to install the package from GitHub and use the many functions available in the package. Please mindful of the ESPN Cricinfo Terms of Use
Take a look at my short video tutorial on my R package cricketr on Youtube – R package cricketr – A short tutorial
Do check out my interactive Shiny app implementation using the cricketr package – Sixer – R package cricketr’s new Shiny avatar
Note 1: If you would like to do a similar analysis for a different set of batsman and bowlers, you can clone/download my skeleton cricketr templatefrom Github (which is the R Markdown file I have used for the analysis below).
Note 2: I sprinkle the charts with my observations. Feel free to look at them more closely and come to your conclusions.
Important note: Do check out the python avatar of cricketr, ‘cricpy’ in my post Introducing cricpy:A python package to analyze performances of cricketers
 This post is also available at Rpubs Clash of the Titans
 You can download this in PDF format at Clash of the Titans
 You can download this R Markdown file from Github at Clash of titans
It can be seen that Tendulkar and Gavaskar has been bowled more often than Kohli. Also Kohli does not have as many sixes in Test cricket as Tendulkar and Gavaskar
par(mfrow=c(1,3)) par(mar=c(4,4,2,2)) batsman4s("./tendulkar.csv","Tendulkar") batsman6s("./tendulkar.csv","Tendulkar") batsmanDismissals("./tendulkar.csv","Tendulkar") dev.off() 4b. More analyses for Kohli par(mfrow=c(1,3)) par(mar=c(4,4,2,2)) batsman4s("./kohli.csv","Kohli") batsman6s("./kohli.csv","Kohli") batsmanDismissals("./kohli.csv","Kohli") dev.off() 4c More analyses for Gavaskar par(mfrow=c(1,3)) par(mar=c(4,4,2,2)) batsman4s("./gavaskar.csv","Gavaskar") batsman6s("./gavaskar.csv","Gavaskar") batsmanDismissals("./gavaskar.csv","Gavaskar") dev.off() 5 Performance of batsmen on different grounds par(mar=c(4,4,2,2)) batsmanAvgRunsGround("./tendulkar.csv","Tendulkar") batsmanAvgRunsGround("./kohli.csv","Kohli") batsmanAvgRunsGround("./gavaskar.csv","Gavaskar")a
 Tendulkar averages 50 against the following countries – Australia, Bangladesh, England, Sri Lanka, West Indies and Zimbabwe
 Kohli average almost 50 against all the nations he has played – Australia, Bangladesh, England, New Zealand, Sri Lanka and West Indies
 Gavaskar averages 50 against Australia, Pakistan, West Indies, Sri Lanka
This is required for the next 2 function calls
tendulkarsp < getPlayerDataSp(35320,tdir=".",tfile="tendulkarsp.csv",ttype="batting") kohlisp < getPlayerDataSp(253802,tdir=".",tfile="kohlisp.csv",ttype="batting") gavaskarsp < getPlayerDataSp(28794,tdir=".",tfile="gavaskarsp.csv",ttype="batting") #dev.off() 8 Get contribution of batsmen in matches won and lostKohli contribution has had an equal contribution in won and lost matches. Tendulkar’s runs seem to have not helped in winning as much as only 50% of matches he has played have been won
par(mfrow=c(2,2)) par(mar=c(4,4,2,2)) batsmanContributionWonLost("tendulkarsp.csv","Tendulkar") batsmanContributionWonLost("./kohlisp.csv","Kohli") batsmanContributionWonLost("./gavaskarsp.csv","Gavaskar")a
9 Performance of batsmen at home and overseasThe boxplots show that Kohli performs better overseas than at home. The 3rd quartile is higher, though the median seems to lower overseas. For Tendulkar the performance is similar both ways. Gavaskar’s median runs scored overseas is higher.
par(mfrow=c(2,2)) par(mar=c(4,4,2,2)) batsmanPerfHomeAway("tendulkarsp.csv","Tendulkar") batsmanPerfHomeAway("./kohlisp.csv","Kohli") batsmanPerfHomeAway("./gavaskarsp.csv","Gavaskar") 10. Moving average of runsGavaskar’s moving average was very good at the time of his retirement. Kohli seems to be going very strong. Tendulkar’s performance shows signs of deterioration around the time of his retirement.
par(mfrow=c(2,2)) par(mar=c(4,4,2,2)) batsmanMovingAverage("./tendulkar.csv","Tendulkar") batsmanMovingAverage("./kohli.csv","Kohli") batsmanMovingAverage("./gavaskar.csv","Gavaskar") #dev.off() 11 Boxplot and histogram of runsKohli has a marginally higher average (50.69) than Tendulkar (48.65) while Gavaskar 46. The median runs are same for Tendulkar and Kohli at 32
par(mfrow=c(2,2)) par(mar=c(4,4,2,2)) batsmanPerfBoxHist("./tendulkar.csv","Sachin Tendulkar") batsmanPerfBoxHist("./kohli.csv","Kohli") batsmanPerfBoxHist("./gavaskar.csv","Gavaskar") 12 Cumulative average Runs for batsmenLooking at the cumulative average runs we can see a gradual drop in the cumulative average for Tendulkar while Kohli and Gavaskar’s performance seems to be getting better
par(mfrow=c(2,2)) par(mar=c(4,4,2,2)) batsmanCumulativeAverageRuns("./tendulkar.csv","Tendulkar") batsmanCumulativeAverageRuns("./kohli.csv","Kohli") batsmanCumulativeAverageRuns("./gavaskar.csv","Gavaskar") 13. Cumulative average strike rate of batsmenTendulkar’s strike rate is better than Kohli and Gavaskar
par(mfrow=c(2,2)) par(mar=c(4,4,2,2)) batsmanCumulativeStrikeRate("./tendulkar.csv","Tendulkar") batsmanCumulativeStrikeRate("./kohli.csv","Kohli") batsmanCumulativeStrikeRate("./gavaskar.csv","Gavaskar") 14 Performance forecast of batsmenThe forecasted performance for Kohli and Gavaskar is higher than that of Tendulkar
par(mfrow=c(2,2)) par(mar=c(4,4,2,2)) batsmanPerfForecast("./tendulkar.csv","Sachin Tendulkar") batsmanPerfForecast("./kohli.csv","Kohli") batsmanPerfForecast("./gavaskar.csv","Gavaskar") #dev.off() 15. Relative strike rate of batsmen par(mar=c(4,4,2,2)) frames < list("./tendulkar.csv","./kohli.csv","gavaskar.csv") names < list("Tendulkar","Kohli","Gavaskar") relativeBatsmanSR(frames,names) #dev.off() 16. Relative Runs frequency of batsmen par(mar=c(4,4,2,2)) frames < list("./tendulkar.csv","./kohli.csv","gavaskar.csv") names < list("Tendulkar","Kohli","Gavaskar") relativeRunsFreqPerf(frames,names) #dev.off() 17. Relative cumulative average runs of batsmenTendulkar leads the way here, but it can be seem Kohli catching up.
par(mar=c(4,4,2,2)) frames < list("./tendulkar.csv","./kohli.csv","gavaskar.csv") names < list("Tendulkar","Kohli","Gavaskar") relativeBatsmanCumulativeAvgRuns(frames,names) #dev.off() 18. Relative cumulative average strike rateTendulkar has better strike rate than the other two.
par(mar=c(4,4,2,2)) frames < list("./tendulkar.csv","./kohli.csv","gavaskar.csv") names < list("Tendulkar","Kohli","Gavaskar") relativeBatsmanCumulativeStrikeRate(frames,names) #dev.off() 19. Check batsman in formAs in the moving average and performance forecast and cumulative average runs, Kohli and Gavaskar are inform while Tendulkar was outofform towards the end.
checkBatsmanInForm("./tendulkar.csv","Sachin Tendulkar") ## [1] "**************************** Form status of Sachin Tendulkar **************************** \n\n Population size: 294 Mean of population: 50.48 \n Sample size: 33 Mean of sample: 32.42 SD of sample: 29.8 \n\n Null hypothesis H0 : Sachin Tendulkar 's sample average is within 95% confidence interval of population average\n Alternative hypothesis Ha : Sachin Tendulkar 's sample average is below the 95% confidence interval of population average\n\n Sachin Tendulkar 's Form Status: OutofForm because the p value: 0.000713 is less than alpha= 0.05 \n *******************************************************************************************\n\n" checkBatsmanInForm("./kohli.csv","Kohli") ## [1] "**************************** Form status of Kohli ****************************\n\n Population size: 117 Mean of population: 50.35 \n Sample size: 13 Mean of sample: 53.77 SD of sample: 46.15 \n\n Null hypothesis H0 : Kohli 's sample average is within 95% confidence interval of population average\n Alternative hypothesis Ha : Kohli 's sample average is below the 95% confidence interval of population average\n\n Kohli 's Form Status: InForm because the p value: 0.603244 is greater than alpha= 0.05 \n *******************************************************************************************\n\n" checkBatsmanInForm("./gavaskar.csv","Gavaskar") ## [1] "**************************** Form status of Gavaskar ****************************\n\n Population size: 125 Mean of population: 44.67 \n Sample size: 14 Mean of sample: 57.86 SD of sample: 58.55 \n\n Null hypothesis H0 : Gavaskar 's sample average is within 95% confidence interval of population average\n Alternative hypothesis Ha : Gavaskar 's sample average is below the 95% confidence interval of population average\n\n Gavaskar 's Form Status: InForm because the p value: 0.793276 is greater than alpha= 0.05 \n *******************************************************************************************\n\n" #dev.off() 20. Performance 3DA 3D regression plane is fitted between the the Balls faced, Minutes at crease and Runs scored
par(mfrow=c(2,2)) par(mar=c(4,4,2,2)) battingPerf3d("./tendulkar.csv","Sachin Tendulkar") battingPerf3d("./kohli.csv","Kohli") battingPerf3d("./gavaskar.csv","Gavaskar") #dev.off() 20. Runs likelihoodThis functions computes the KMeans and determines the runs the batsmen are likely to score.
par(mar=c(4,4,2,2)) batsmanRunsLikelihood("./tendulkar.csv","Tendulkar") ## Summary of Tendulkar 's runs scoring likelihood ## ************************************************** ## ## There is a 16.51 % likelihood that Tendulkar will make 139 Runs in 251 balls over 353 Minutes ## There is a 25.08 % likelihood that Tendulkar will make 66 Runs in 122 balls over 167 Minutes ## There is a 58.41 % likelihood that Tendulkar will make 16 Runs in 31 balls over 44 Minutes batsmanRunsLikelihood("./kohli.csv","Kohli") ## Summary of Kohli 's runs scoring likelihood ## ************************************************** ## ## There is a 20 % likelihood that Kohli will make 143 Runs in 232 balls over 330 Minutes ## There is a 33.85 % likelihood that Kohli will make 51 Runs in 92 balls over 127 Minutes ## There is a 46.15 % likelihood that Kohli will make 11 Runs in 24 balls over 31 Minutes batsmanRunsLikelihood("./gavaskar.csv","Gavaskar") ## Summary of Gavaskar 's runs scoring likelihood ## ************************************************** ## ## There is a 33.81 % likelihood that Gavaskar will make 69 Runs in 159 balls over 214 Minutes ## There is a 8.63 % likelihood that Gavaskar will make 172 Runs in 364 balls over 506 Minutes ## There is a 57.55 % likelihood that Gavaskar will make 13 Runs in 35 balls over 48 Minutes 21. Predict runs for a random combination of Balls faced and runs scored BF < seq( 10, 400,length=15) Mins < seq(30,600,length=15) newDF < data.frame(BF,Mins) tendulkar < batsmanRunsPredict("./tendulkar.csv","Tendulkar",newdataframe=newDF) kohli < batsmanRunsPredict("./kohli.csv","Kohli",newdataframe=newDF) gavaskar < batsmanRunsPredict("./gavaskar.csv","Gavaskar",newdataframe=newDF) batsmen <cbind(round(tendulkar$Runs),round(kohli$Runs),round(gavaskar$Runs)) colnames(batsmen) < c("Tendulkar","Kohli","Gavaskar") newDF < data.frame(round(newDF$BF),round(newDF$Mins)) colnames(newDF) < c("BallsFaced","MinsAtCrease") predictedRuns < cbind(newDF,batsmen) predictedRuns ## BallsFaced MinsAtCrease Tendulkar Kohli Gavaskar ## 1 10 30 7 6 4 ## 2 38 71 23 24 17 ## 3 66 111 39 42 30 ## 4 94 152 54 60 43 ## 5 121 193 70 78 56 ## 6 149 234 86 96 69 ## 7 177 274 102 114 82 ## 8 205 315 118 132 95 ## 9 233 356 134 150 108 ## 10 261 396 150 168 121 ## 11 289 437 165 186 134 ## 12 316 478 181 204 147 ## 13 344 519 197 222 160 ## 14 372 559 213 240 173 ## 15 400 600 229 258 186 #dev.off() Key findings Kohli has a marginally higher average than Tendulkar
 Tendulkar has the best strike rate of all the 3.
 The cumulative average runs and the performance forecast for Kohli and Gavaskar show an improving trend, while Tendulkar’s numbers deteriorate towards the end of his career
 Kohli is fast catching up Tendulkar on cumulative average runs vs innings in career.
The functions below get the ODI data for Tendulkar and Kohli as CSV files so that the analyses can be done
22 Get player data for ODIs tendulkarOD < getPlayerDataOD(35320,dir=".",file="tendulkarOD.csv",type="batting") kohliOD < getPlayerDataOD(253802,dir=".",file="kohliOD.csv",type="batting") #dev.off() 23a Basic performance of Tendulkar in ODI par(mfrow=c(3,2)) par(mar=c(4,4,2,2)) batsmanRunsFreqPerf("./tendulkarOD.csv","Tendulkar") batsmanRunsRanges("./tendulkarOD.csv","Tendulkar") batsman4s("./tendulkarOD.csv","Tendulkar") batsman6s("./tendulkarOD.csv","Tendulkar") batsmanScoringRateODTT("./tendulkarOD.csv","Tendulkar") #dev.off() 23b. Basic performance of Kohli in ODI par(mfrow=c(3,2)) par(mar=c(4,4,2,2)) batsmanRunsFreqPerf("./kohliOD.csv","Kohli") batsmanRunsRanges("./kohliOD.csv","Kohli") batsman4s("./kohliOD.csv","Kohli") batsman6s("./kohliOD.csv","Kohli") batsmanScoringRateODTT("./kohliOD.csv","Kohli") #dev.off() 24. Performance forecast in ODIsKohli’s forecasted runs are much higher than Tendulkar’s in ODIs
par(mar=c(4,4,2,2)) batsmanPerfForecast("./tendulkarOD.csv","Tendulkar") batsmanPerfForecast("./kohliOD.csv","Kohli") 25. Batting performanceA 3D regression plane is fitted between Balls faced, Minutes at crease and Runs scored.
par(mar=c(4,4,2,2)) battingPerf3d("./tendulkarOD.csv","Tendulkar") battingPerf3d("./kohliOD.csv","Kohli") 26. Predicting runs scored for the ODI batsmenKohli will score runs than Tendulkar for the same minutes at crease and balls faced.
BF < seq( 10, 200,length=10) Mins < seq(30,220,length=10) newDF < data.frame(BF,Mins) tendulkarDF < batsmanRunsPredict("./tendulkarOD.csv","Tendulkar",newdataframe=newDF) kohliDF < batsmanRunsPredict("./kohliOD.csv","Kohli",newdataframe=newDF) batsmen <cbind(round(tendulkarDF$Runs),round(kohliDF$Runs)) colnames(batsmen) < c("Tendulkar","Kohli") newDF < data.frame(round(newDF$BF),round(newDF$Mins)) colnames(newDF) < c("BallsFaced","MinsAtCrease") predictedRuns < cbind(newDF,batsmen) predictedRuns ## BallsFaced MinsAtCrease Tendulkar Kohli ## 1 10 30 7 8 ## 2 31 51 26 28 ## 3 52 72 45 48 ## 4 73 93 64 68 ## 5 94 114 83 88 ## 6 116 136 102 108 ## 7 137 157 121 128 ## 8 158 178 140 149 ## 9 179 199 159 169 ## 10 200 220 178 189 27. Runs likelihood for the ODI batsmenTendulkar has clusters around 13, 53 and 111 runs while Kohli has clusters around 13, 63,116. So it more likely that Kohli will tend to score higher
par(mar=c(4,4,2,2)) batsmanRunsLikelihood("./tendulkarOD.csv","Tendulkar") ## Summary of Tendulkar 's runs scoring likelihood ## ************************************************** ## ## There is a 18.09 % likelihood that Tendulkar will make 111 Runs in 118 balls over 172 Minutes ## There is a 28.39 % likelihood that Tendulkar will make 53 Runs in 63 balls over 95 Minutes ## There is a 53.52 % likelihood that Tendulkar will make 13 Runs in 18 balls over 27 Minutes batsmanRunsLikelihood("./kohliOD.csv","Kohli") ## Summary of Kohli 's runs scoring likelihood ## ************************************************** ## ## There is a 31.41 % likelihood that Kohli will make 63 Runs in 69 balls over 97 Minutes ## There is a 49.74 % likelihood that Kohli will make 13 Runs in 18 balls over 24 Minutes ## There is a 18.85 % likelihood that Kohli will make 116 Runs in 113 balls over 163 Minutes 28. Runs in different venues for the ODI batsmen par(mar=c(4,4,2,2)) batsmanAvgRunsGround("./tendulkarOD.csv","Tendulkar") batsmanAvgRunsGround("./kohliOD.csv","Kohli") 28. Runs against different opposition for the ODI batsmenTendulkar’s has 50+ average against Bermuda, Kenya and Namibia. While Kohli has a 50+ average against New Zealand, West Indies, South Africa, Zimbabwe and Bangladesh
par(mar=c(4,4,2,2)) batsmanAvgRunsOpposition("./tendulkarOD.csv","Tendulkar") batsmanAvgRunsOpposition("./kohliOD.csv","Kohli") 29. Moving average of runs for the ODI batsmenTendulkar’s moving average shows an improvement (50+) towards the end of his career, but Kohli shows a marked increase 60+ currently
par(mar=c(4,4,2,2)) batsmanMovingAverage("./tendulkarOD.csv","Tendulkar") batsmanMovingAverage("./kohliOD.csv","Kohli") 30. Cumulative average runs of ODI batsmenTendulkar plateaus at 40+ while Kohli’s cumulative average runs goes up and up!!!
par(mar=c(4,4,2,2)) batsmanCumulativeAverageRuns("./tendulkarOD.csv","Tendulkar") batsmanCumulativeAverageRuns("./kohliOD.csv","Kohli") 31 Cumulative strike rate of ODI batsmen par(mar=c(4,4,2,2)) batsmanCumulativeStrikeRate("./tendulkarOD.csv","Tendulkar") batsmanCumulativeStrikeRate("./kohliOD.csv","Kohli") 32. Relative batsmen strike rate par(mar=c(4,4,2,2)) frames < list("./tendulkarOD.csv","./kohliOD.csv") names < list("Tendulkar","Kohli") relativeBatsmanSRODTT(frames,names) #dev.off() 33. Relative Run Frequency percentages par(mar=c(4,4,2,2)) frames < list("./tendulkarOD.csv","./kohliOD.csv") names < list("Tendulkar","Kohli") relativeRunsFreqPerfODTT(frames,names) #dev.off() 34. Relative cumulative average runs of ODI batsmenKohli breaks away from Tendulkar in cumulative average runs after 100 innings
par(mar=c(4,4,2,2)) frames < list("./tendulkarOD.csv","./kohliOD.csv") names < list("Tendulkar","Kohli") relativeBatsmanCumulativeAvgRuns(frames,names) #dev.off() 35. Relative cumulative strike rate of ODI batsmenThis seems to be tussle with Kohli having an edge till about 40 innings and then from 40+ to 180 innings Tendulkar leads. Kohli just seems to be edging forward.
par(mar=c(4,4,2,2)) frames < list("./tendulkarOD.csv","./kohliOD.csv") names < list("Tendulkar","Kohli") relativeBatsmanCumulativeStrikeRate(frames,names) #dev.off() 36. Batsmen 4s and 6s par(mar=c(4,4,2,2)) frames < list("./tendulkarOD.csv","./kohliOD.csv") names < list("Tendulkar","Kohli") batsman4s6s(frames,names) ## Tendulkar Kohli ## Runs(1s,2s,3s) 66.29 69.67 ## 4s 29.65 25.90 ## 6s 4.06 4.43 #dev.off() 37. Check ODI batsmen form par(mar=c(4,4,2,2)) checkBatsmanInForm("./tendulkar.csv","Tendulkar") ## [1] "**************************** Form status of Tendulkar ******** ********************\n\n Population size: 294 Mean of population: 50.48 \n Sample size: 33 Mean of sample: 32.42 SD of sample: 29.8 \n\n Null hypothesis H0 : Tendulkar 's sample average is within 95% confidence interval of population average\n Alternative hypothesis Ha : Tendulkar 's sample average is below the 95% confidence interval of population average\n\n Tendulkar 's Form Status: OutofForm because the p value: 0.000713 is less than alpha= 0.05 \n *******************************************************************************************\n\n" checkBatsmanInForm("./kohli.csv","Kohli") ## [1] "**************************** Form status of Kohli *********** *****************\n\n Population size: 117 Mean of population: 50.35 \n Sample size: 13 Mean of sample: 53.77 SD of sample: 46.15 \n\n Null hypothesis H0 : Kohli 's sample average is within 95% confidence interval of population average\n Alternative hypothesis Ha : Kohli 's sample average is below the 95% confidence interval of population average\n\n Kohli 's Form Status: InForm because the p value: 0.603244 is greater than alpha= 0.05 \n *******************************************************************************************\n\n" #dev.off() Key Findings Kohli has a better performance against oppositions like West Indies, South Africa and New Zealand
 Kohli breaks away from Tendulkar in cumulative average runs
 Tendulkar has been leading the strike rate rate but Kohli in recent times seems to be breaking loose.
Check out some other players with my R package cricketr
Also see
 My book ‘Practical Machine Learning in R and Python: Third edition’ on Amazon
 A primer on Qubits, Quantum gates and Quantum Operations
 Deblurring revisited with Wiener filter using OpenCV
 Deep Learning from first principles in Python, R and Octave – Part 4
 The Many Faces of Latency
 Fun simulation of a Chain in Android
 Presentation on Wireless Technologies – Part 1
 yorkr crashes the IPL party ! – Part 1
To see all posts click Index of posts
var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));To leave a comment for the author, please follow the link and comment on their blog: R – Giga thoughts …. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
#20: Dependencies. Now with badges!
(This article was first published on Thinking inside the box , and kindly contributed to Rbloggers)
Welcome to post number twenty in the randomly redundant R rant series of posts, or R4 for short. It has been a little quiet since the previous post last June as we’ve been busy with other things but a few posts (or ideas at least) are queued.
Dependencies. We wrote about this a good year ago in post #17 which was (in part) tickled by the experience of installing one package … and getting a boatload of others pulled in. The topic and question of dependencies has seen a few posts over the year, and I won’t be able to do them all justice. Josh and I have been added a few links to the tinyverse.org page. The (currently) last one by Russ Cox titled Our Software Dependency Problem is particularly trenchant.
And just this week the topic came up in two different, and unrelated posts. First, in What I don’t like in you repo, Oleg Kovalov lists a brief but decent number of items by which a repository can be evaluated. And one is about [b]loated dependencies where he nails it with a quick When I see dozens of deps in the lock file, the first question which comes to my mind is: so, am I ready to fix any failures inside any of them? This is pretty close to what we have been saying around the tidyverse.
Second, in Beware the data science pin factory, Eric Colson brings an equation. Quoting from footnote 2: […] the number of relationships (r) grows as a function number of members (n) per this equation: r = (n^2n) / 2. Granted, this was about human coordination and ideal team size. But let’s just run with it: For n=10, we get r=9 which is not so bad. For n=20, it is r=38. And for n=30 we are at r=87. You get the idea. “BigOhNsquared”.
More dependencies means more edges between more nodes. Which eventually means more breakage.
Which gets us to announcement embedded in this post. A few months ago, in what still seems like a genuinely extraclever weekend hack in an initial 100 or so lines, Edwin de Jonge put together a remarkable repo on GitLab. It combines Docker / Rocker via hourly cron jobs with deployment at netlify … giving us badges which visualize the direct as well as recursive dependencies of a package. All in about 100 lines, fully automated, autonomously running and deployed via CDN. Amazing work, for which we really need to praise him! So a big thanks to Edwin.
With these CRAN Dependency Badges being available, I have been adding them to my repos at GitHub over the last few months. As two quick examples you can see
 Rcpp
 RcppArmadillo
to get the idea. RcppArmadillo (or RcppEigen or many other packages) will always have one: Rcpp. But many widelyused packages such as data.table also get by with a count of zero. It is worth showing this – and the badge does just that! And I even sent a PR to the badger package: if you’re into this, you can have a badge made for your via badger::badge_depdencies(pkgname).
Otherwise, more details at Edwin’s repo and of course his actual tinyverse.netlify.com site hosting the badges. It’s easy as all other badges: reference the CRAN package, get a badge.
So if you buy into the idea that lightweight is the right weight then join us and show it via the dependency badges!
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive reaggregation in thirdparty forprofit settings.
var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));
To leave a comment for the author, please follow the link and comment on their blog: Thinking inside the box . Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
sparklyr 1.0: Apache Arrow, XGBoost, Broom and TFRecords
(This article was first published on RStudio Blog, and kindly contributed to Rbloggers)
With much excitement built over the past three years, we are thrilled to share that sparklyr 1.0 is now available on CRAN!
The sparklyr package provides an R interface to Apache Spark. It supports dplyr, MLlib, streaming, extensions and many other features; however, this particular release enables the following new features:
 Arrow enables faster and larger data transfers between Spark and R.
 XGBoost enables training gradient boosting models over distributed datasets.
 Broom converts Spark’s models into tidy formats that you know and love.
 TFRecords writes TensorFlow records from Spark to support deep learning workflows.
This release also brings support for Spark 2.4, the ability to collect and copy in batches, increased Livy performance, and many more improvements listed in the sparklyr NEWS file. You can install sparklyr 1.0 from CRAN as follows:
install.packages("sparklyr") ArrowApache Arrow is a crosslanguage development platform for inmemory data, you can read more about this in the Arrow and beyond blog post. In sparklyr 1.0, we are embracing Arrow as an efficient bridge between R and Spark, conceptually:
In practice, this means faster data transfers and support for larger datasets; specifically, this improves collect(), copy_to() and spark_apply(). The following benchmarks make use of the bench package to measure performance with and without arrow.
We will first benchmark copy_to() over a dataframe with 1M and 10M rows. Notice that, with the default memory configuration, copy_to() can’t handle 10M rows while arrow can.
library(sparklyr) sc < spark_connect(master = "local") bench::press(rows = c(10^6, 10^7), { bench::mark( arrow_on = { library(arrow) sparklyr_df << copy_to(sc, data.frame(y = 1:rows), overwrite = T) }, arrow_off = if (rows <= 10^6) { if ("arrow" %in% .packages()) detach("package:arrow") sparklyr_df << copy_to(sc, data.frame(y = 1:rows), overwrite = T) } else NULL, iterations = 4, check = FALSE) })Next, we will benchmark collect() over 10M and 50M records; collecting 50M+ records is only possible with arrow.
bench::press(rows = c(10^7, 5 * 10^7), { bench::mark( arrow_on = { library(arrow) collected < sdf_len(sc, rows) %>% collect() }, arrow_off = if (rows <= 10^7) { if ("arrow" %in% .packages()) detach("package:arrow") collected < sdf_len(sc, rows) %>% collect() } else NULL, iterations = 4, check = FALSE) })Last but not least, spark_apply() over 100K and 1M rows shows the most significant improvements. A 40x speedup when running R on Spark, additional details are available in the Arrow project post.
bench::press(rows = c(10^5, 10^6), { bench::mark( arrow_on = { library(arrow) sdf_len(sc, rows) %>% spark_apply(~ .x / 2) %>% dplyr::count() %>% collect }, arrow_off = if (rows <= 10^5) { if ("arrow" %in% .packages()) detach("package:arrow") sdf_len(sc, rows) %>% spark_apply(~ .x / 2) %>% dplyr::count() %>% collect } else NULL, iterations = 4, check = FALSE) })To use arrow, you will first have to install the Apache Arrow runtime followed by installing the R arrow package, additional instructions are available under spark.rstudio.com/guides/arrow.
XGBoostsparkxgb is a new sparklyr extension that can be used to train XGBoost models in Spark. sparkxgb is available on CRAN and can be installed as follows:
install.packages("sparkxgb")We can then use xgboost_classifier() to train and ml_predict() to predict over large datasets with ease:
library(sparkxgb) library(sparklyr) library(dplyr) sc < spark_connect(master = "local") iris < copy_to(sc, iris) xgb_model < xgboost_classifier(iris, Species ~ ., num_class = 3, num_round = 50, max_depth = 4) xgb_model %>% ml_predict(iris) %>% select(Species, predicted_label, starts_with("probability_")) %>% glimpse() #> Observations: ?? #> Variables: 5 #> Database: spark_connection #> $ Species "setosa", "setosa", "setosa", "setosa", "… #> $ predicted_label "setosa", "setosa", "setosa", "setosa", "… #> $ probability_versicolor 0.003566429, 0.003564076, 0.003566429, 0.… #> $ probability_virginica 0.001423170, 0.002082058, 0.001423170, 0.… #> $ probability_setosa 0.9950104, 0.9943539, 0.9950104, 0.995010…You can read more about sparkxgb under its README file. Note that Windows is currently unsupported.
BroomWhile support for broom in Spark through sparklyr has been under development for quite some time, this release marks the completion of all modeling functions. For instance, we can now augment using an ALS model with ease:
movies < data.frame(user = c(1, 2, 0, 1, 2, 0), item = c(1, 1, 1, 2, 2, 0), rating = c(3, 1, 2, 4, 5, 4)) copy_to(sc, movies) %>% ml_als(rating ~ user + item) %>% augment() # Source: spark<?> [?? x 4] user item rating .prediction 1 2 2 5 4.86 2 1 2 4 3.98 3 0 0 4 3.88 4 2 1 1 1.08 5 0 1 2 2.00 6 1 1 3 2.80 TFRecordssparktf is a new sparklyr extension allowing you to write TensorFlow records in Spark. This can be used to preprocess large amounts of data before processing them in GPU instances with Keras or TensorFlow. sparktf is now available on CRAN and can be installed as follows:
install.packages("sparktf")You can simply preprocess data in Spark and write it as TensorFlow records using spark_write_tf():
library(sparktf) library(sparklyr) sc < spark_connect(master = "local") copy_to(sc, iris) %>% ft_string_indexer_model( "Species", "label", labels = c("setosa", "versicolor", "virginica") ) %>% spark_write_tfrecord(path = "tfrecord")You can then use TensorFlow and Keras from R to load this recordset and train deep learning models; for instance, using tfrecord_dataset(). Please read the sparktf README for more details.
Moar?When connecting to Spark running in YARN, RStudio’s connection pane can now launch YARN’s web application.
We also made it possible to copy and collect larger datasets by using callbacks. For instance, you can collect data incrementally in batches of 100K rows; this is configurable through the sparklyr.collect.batch setting. The following example collects 300K rows using batches and prints the total records collected; in practice, you save and load from disk.
sdf_len(sc, 3 * 10^5) %>% collect( callback = ~message("(", .y, ") Collecting ", nrow(.x), " rows.") ) (1) Collecting 100000 rows. (2) Collecting 100000 rows. (3) Collecting 100000 rows.For Livy connections, performance is improved when setting the spark_version parameter in livy_config(), this allows sparklyr to start a connection using JARs instead of loading sources.
In addition, extensions are now also supported in Livy. For example, you can run pagerank with Livy and graphframes as follows:
library(graphframes) library(sparklyr) livy_service_start() sc < spark_connect(master = "local", method = "livy", version = "2.4.0") gf_pagerank(gf_friends(sc), tol = 0.01, reset_probability = 0.15) GraphFrame Vertices: Database: spark_connection $ id "f", "g", "a", "e", "d", "b", "c" $ name "Fanny", "Gabby", "Alice", "Esther", "David", "Bob", "Charlie" $ age 36, 60, 34, 32, 29, 36, 30 $ pagerank 0.3283607, 0.1799821, 0.4491063, 0.3708523, 0.3283607, 2.6555078, 2.6878300 Edges: Database: spark_connection $ src "a", "b", "e", "e", "c", "a", "f", "d" $ dst "b", "c", "f", "d", "b", "e", "c", "a" $ relationship "friend", "follow", "follow", "friend", "follow", "friend", "follow", "friend" $ weight 0.5, 1.0, 0.5, 0.5, 1.0, 0.5, 1.0, 1.0The sparklyr NEWS contains a complete list of changes and features for this release. To catch up on previously released features, you can read the blog posts that got us here:
 sparklyr 0.9: Streams and Kubernetes.
 sparklyr 0.8: Production pipelines and graphs.
 sparklyr 0.7: Spark Pipelines and Machine Learning.
 sparklyr 0.6: Distributed R and external sources.
 sparklyr 0.5: Livy and dplyr improvements.
 sparklyr 0.4: R interface for Apache Spark.
We hope you enjoy this exciting release!
var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));To leave a comment for the author, please follow the link and comment on their blog: RStudio Blog. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
EARL London abstract submission deadline – 31 March
(This article was first published on RBlog – Mango Solutions, and kindly contributed to Rbloggers)
“EARL is a great Conference: it gives the opportunity of connecting with other companies that are using R for data analysis and getting inspired by their stories.”We have been delighted to already receive a host of interesting and brilliant abstracts for EARL London 2019, and there’s still time to submit yours!
The Enterprise Applications of the R Langauge Conference is all about sharing and inspiring others with your work using R. Our audience come from all sorts of different industries and experience levels, so whatever you’re doing with R in enterprise, EARL is the perfect platform to share it.
If you’re still wondering if you’d like to submit or not, take a look at our top 10 reasons why you should.
You have until 31 March to submit – we look forward to receiving yours soon!
We are also looking for 10minute lightning talks on ‘data for good’. Please use the same submission form for this style of talk.
var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));
To leave a comment for the author, please follow the link and comment on their blog: RBlog – Mango Solutions. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
How to select a seed for simulation or randomization
(This article was first published on The Princess of Science, and kindly contributed to Rbloggers)
If you need to generate a randomization list for a clinical trial, do some simulations or perhaps perform a huge bootstrap analysis, you need a way to draw random numbers. Putting many pieces of paper in a hat and drawing them is possible in theory, but you will probably be using a computer for doing this. The computer, however, does not generate random numbers. It generates pseudo random numbers. They look and feel almost like real random numbers, but they are not random. Each number in the sequence is calculated from its predecessor, so the sequence has to begin somewhere; it begins in the seed – the first number in the sequence.
Knowing the seed is a good idea. It enables reproducing the analysis, the simulation or the randomization list. If you run a clinical trial, reproducibility is crucial. You must know at the end of the trial which patient was randomized to each treatment; otherwise you will throw all your data to the garbage. During the years I worked at Teva Pharmaceuticals, we took every possible safety measure: We burnt the randomization lists, the randomization SAS code and the randomization seed on a CD and kept it in a fireproof safe. We also kept all this information in analog media. Yes, we printed the lists, the SAS code and the seed on paper, and these were also kept in the safe.
Using the same seed every time is not a good idea. If you use the same seed every time, you get the same sequence of pseudorandom numbers every time, and therefore your numbers are not pseudorandom anymore. Selecting a different seed every time is good practice.
How do you select the seed? Taking a number out of your head is still not a good idea. Our heads are biased. Like passwords, people tend to use some seeds more often than other possible seeds. Don’t be surprised if you see codes with seeds like 123, 999 or 31415.
The best practice is to choose a random seed, but this creates a magic circle. You can still turn to your old hat and some pieces of paper, but there is a simple alternative: generate the seed from the computer clock.
This is how you do it in R by using the Sys.time() function: Get the system time, convert it to an integer, and you’re done. In practice, I take only the last 5 digits of that integer. And off course, I keep the chosen seed on record. I also save the Sys.time value and its integer value, just in case. Finally, I hardcode the seed I get. This is the R code:
> # convert the time into a numeric variable > initial_seed=as.integer(initial_seed) > print (initial_seed) [1] 1552576418 > # take the last five digits f the initial seed > the_seed=initial_seed %% 100000 > print(the_seed) # 76151 [1] 76418 > set.seed(76418) > # do your simulation > print(rnorm(3)) [1] 0.1255811 1.1614262 0.8534025 > # reproduce your simulation > set.seed(76418) > print(rnorm(3)) [1] 0.1255811 1.1614262 0.8534025var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));
To leave a comment for the author, please follow the link and comment on their blog: The Princess of Science. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
Collecting Content Security Policy Violation Reports in S3 (‘Effortlessly’/’Freely’)
(This article was first published on R – rud.is, and kindly contributed to Rbloggers)
In the previous post I tried to explain what Content Security Policies (CSPs) are and how to work with them in R. In case you didn’t RTFPost the TLDR is that CSPs give you control over what can be loaded along with your web content and can optionally be configured to generate a violation report for any attempt to violate the policy you create. While you don’t need to specify a report URI you really should since at the very least you’ll know if you errantly missed a given host, wildcard, or path. You’ll also know when there’s been malicious or just plain skeezy activity going on with thirdparties and your content (which is part of the whole point of CSPs).
There’s an “R” category tag on this post (so it’s hitting Rbloggers, et al) since it’s part of an unnumbered series on working with CSPs in R and the next posts will show how to analyze the JSONformatted reports that are generated. But, to analyze such reports you kinda need a way to get them first. So, we’re going to setup a “serverless” workflow in Amazon AWS to shove CSP reports into a wellorganized structure in S3 from which we’ll be able to access, ingest, and analyze them.
Sure, there are services out there who will (legit for free) let you forward violation reports to them but if you can do this for “free” on your own and not give data out to a thirdparty to make money or ostensibly dogooder reputation from I can’t fathom an argument for just giving up control.
Note that all you need is an an internetaccessible HTTPS endpoint that can take an HTTP POST request with a JSON payload and then store that somewhere, so if you want to, say, use the plumber package to handle these requests without resorting to AWS, then by all means do so! (And, blog about it!)
AWS “Serverless” CSP Report Workflow PrerequisitesYou’re obviously going to need an Amazon AWS account and will also need the AWS Command Line Interface tools installed plus an IAM user that has permissions to use CloudFormation. AWS has been around a while now so yetanotherhowto on signing up for AWS, installing the CLI tools and generating an IAM user would be, atbest, redundant. Amazon has decent intro resources and, honestly, it’s 2019 and having some familiarity with how to work with at least one cloud provider is pretty much a necessary skillset at this point depending on what part of “tech” you’re in. If you’re new to AWS then follow the links in this paragraph, run through some basics and jump back to enter the four commands you’ll need to run to bootstrap your CSP collection setup.
Bootstrapping an S3 CSP Collector in AWSWe’re going to use this CloudFormation workflow to bootstrap the CSP collection process and you should skim the yaml file to see what’s going on. Said yaml is “infrastructure as code”, meaning it’s a series of configuration directives to generate AWS services for you (i.e. no pointingandclicking) and, perhaps more importantly, destroy them for you if you no longer want to keep this active.
The CF Output directive will be the URI you’re going to use in the reporturi/reportto CSP directives and is something we’ll be querying for at the end of the setup process.
The first set of resources are AWS Glue templates which would enable wiring up the CSP report results into AWS Athena. Glue is a nice ETL framework but it’s kinda expensive if set in active mode (Amazon calls it ‘crawler’ mode) so this CloudFormation recipe only created the Glue template but does not activate it. This section can (as the repo author notes) be deleted but it does no harm and costs nothing extra so leaving it in is fine as well.
The next bit sets up an AWS Firehose configuration which is a silly sounding name for setting up a workflow for where to store “streaming” data. This “firehose” config is just going to setup a path for an S3 bucket and then setup the necessary permissions associated with said bucket. This is where we’re going to pull data from in the next post.
The aforementioned “firehose” can take streaming data from all kinds of input sources and our data source is going to be a POSTed JSON HTTP interaction from a browser so we need to have something that listens for these POST requests and wire that up to the “firehose”. For that we need an API gateway and that’s what the penultimate section sets up for us. It instructs AWS to setup an API endpoint to listen for POST requests, tells it the data type (JSON) it will be handling and then tells it what AWS Lambda to call, which is in the last section.
Said lambda code is in the repo’s index.js and is a short Node.js script to postprocess the CSP report JSON into something slightly more usable in a data analysis context (the folks who made the violation report clearly did not have data science folks in mind when creating the structure given the liberal use if  in field names).
If the above sounds supercomplex just go get CSP reports, you’re notwrong. We trade off the cost and tedium of selfhosting and securing a standaloneyetsimple JSON POST handling server for a moderately complex workflow that involves multiple types of moving parts in AWS. The downside is having to gain a more than casual familiarity with AWS components. The plus side is that this is pretty much free unless your site is wildly popular and either constantly under XSS attack or your CSP policy is woefully misconfigured.
“‘Free’, you say?!” Yep. Free. (OK, “mostly” free)
 AWS API Gateway: 1,000,000 HTTP REST API calls (our POST reqs that call the lambda code) per month are free
 AWS Lambda (the index.js runner which sends data to the “firehose”): 1,000,000 free requests per month and 400,000 seconds of compute time per month (the index.js takes ~1s to run)
 AWS Firehose (the bit that shoves data into S3): first 500 TB/month is $0.029 USD
 AWS S3: First 50 TB / month is $0.023 per GB (the CSP JSON POSTs gzip’d are usually <1K each) + some superfractional (of a penny) costs for PUTting data into S3 and copying data from S3.
A wellcrafted CSP and a typical site should end up costing you way less than $1.00 USD/month and you can monitor it all via the console or with alerts (change your region, if needed). Plus, you can destroy it at any time with one command (we haven’t built it yet so we’ll see this in a bit).
Launching the BootstrapAs the repo says, do:
$ git clone git@github.com:michaelbanfield/serverlesscspreportto.git # get the repo $ cd serverlesscspreportto # go to the dir $ aws s3 mb s3://someuniqueanddecentbucketnametoholdthelambdacode/ # pick a good name that you'll recognize $ aws cloudformation package \ # generate the build template templatefile template.yaml \ s3bucket \ outputtemplatefile packagedtemplate.yaml $ aws cloudformation deploy \ # launch the build templatefile /path/to/packagedtemplate.yaml \ stackname CSPReporter \ capabilities CAPABILITY_IAMIt’ll take a minute or two and when it is done just do:
$ aws cloudformation describestacks \ query "Stacks[0].Outputs[0].OutputValue" \ output text \ stackname CSPReporterTo get the URL you’ll use in the reporting directives.
To get rid of all these created resources you can go into the console and do it or just do
$ aws cloudformation deletestack stackname CSPReporterTo see the bucket that was created for the CSP reports just do:
$ aws s3 ls  grep firehose FINIf you’re experienced with AWS that was likely not a big deal. If you’re new or inexperienced with AWS this is not a bad way to get some experience with a “serverless” API setup since it’s cheap, easy to delete and touches on a number of key components within AWS.
You can browse through the AWS console to see all of what was created and eventually tweak the CF yaml to bend it to your own will.
Next time we’ll dive in to CSP violation report analysis with R.
REMINDER to — regardless of the source (whether it’s me, RStudio, spiffy R package authors, or big names like AWS/Microsoft/etc.) — always at least spot check the code you’re about to install or execute. Everyone needs to start developing and honing a zerotrust mindset when it comes to even installing apps from app stores on your phones/tablets let alone allowing random R, C[++], Python, Go, Rust, Haskel, … code to execute on your laptops and servers. This is one reason I went through the sections in the YAML and deliberately linked to the index.js. Not knowing what the code does can lead to unfortunate situations down the line.
NOTE: If you have an alternative Terraform configuration for this drop a note in the comments since TF is a bit more “modern” and less AWScentric “infrastructure as code” framework. Also, if you’ve done this with Azure or other providers, also drop a note in the comments since it may be of use to folks who aren’t interested in using AWS. Finally, if you do make a plumber server for this, also drop a note to a post with how you did it and perhaps discuss the costs & headaches involved.
var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));To leave a comment for the author, please follow the link and comment on their blog: R – rud.is. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
Pi day quiz
(This article was first published on The Princess of Science, and kindly contributed to Rbloggers)
Today is the Pi Day – an annual celebration of the mathematical constant Pi. It is observed every year on March 14, since Pi can be approximated by 3.14, and the this date is written as 3/14 in the month/day format.
To celebrate this day, here is a short quiz about Pi. You can find all the answers on the web, but it is more fun to try answering while depending only on personal knowledge. Let’s go!
1) In the ancient world, many cultures derived approximations to ?. Which ancient culture had the best approximation?
a. Egypt
b. Babylon
c. Hebrews
d. India
2) Which of these fractions is the best approximation Pi?
a. 2549491779/811528438
b. 22/7
c. 3927/1250
d. 864/275
3) Who popularized the use of the Greek letter Pi to represent the ratio of a circle’s circumference to its diameter?
a. Carl Friedrich Gauss
b. Leonard Euler
c. Pierre Simon Laplace
d. Issac Newton
4) The problem of squaring the circle does not have a solution because Pi is
a. An algebraic number
b. A rational number
c. A transcendental number
d. An irrational number
5) Who prove that the problem of squaring the circle does not have a solution?
a. Carl Friedrich Gauss
b. Adrien Marie Legendre
c. Ferdinand von Lindman
d. Evariste Galois
6) Pi plays an important role in Statistics because
a. The value of Pi can be approximated by dropping needles on the floor
b. The sample size calculation formula contains Pi
c. The probability density function of the Normal distribution contains Pi
d. Pi is the maximal value of Euler’s population density curve
7) Who was born on Pi day?
a. Johann Strauss
b. Wolfgang Amadeus Mozart
c. Johann Sebastian Bach
d. Georges Bizet
8) How many decimal digits of Pi were calculated up to date?
a. ~ 206 billion
b. ~ 50 quintillion
c. ~ 22.5 trillion
d. ~ 352 million
9) The first known rigorous algorithm for calculating the value of Pi was devised by
a. Shankara Variyar
b. Liu Hui
c. Archimedes
d. Ibn alHaytham
10) Which of the following series does not converge to Pi?
Good luck! I will post the answers next week.
var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));To leave a comment for the author, please follow the link and comment on their blog: The Princess of Science. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
Simulating the Six Nations 2019 Rugby Tournament in R: Final Round Update
(This article was first published on R – stats on the cloud, and kindly contributed to Rbloggers)
In an earlier post I blogged how I had made a Monte Carlo simulation model of the Six Nations Rugby Tournament. With the final round of the tournament approaching this Saturday, I decided to do a quick update.
Who can win at this stage?
Wales, England, or Ireland can still win. Scotland, France and Italy do not have enough points at this stage to win. Quite a good article from the London Evening Standard explains the detail. The current league table is below.
Who is playing who in the final round?
What is the simulation model based upon?
A random sample from a probability mass function for tries, conversions and penalties, which is combined with a pwin for each team, calculated based on the RugbyPass Index for both home and away teams. If you want to know more, feel free to look at my previous post (linked above) or the R script (linked at the bottom).
What does the simulated league table look after the final round?
Running a simulation for the final three games, and adding these results on to the actual points each team has achieved after round 4, we get the distribution of league points shown below.
Apologies: a box plot can be a bit odd for discrete data such as this. Please forgive me! If I had the time I would reform this into something like a stacked histogram which would be more accurate
It should be noted that, whilst the ‘standard’ scoring scheme applies for these final matches, i.e.
 4pt for a win, 2pt for a draw, 0pt for a loss.
 plus 1 bonus pt for scoring 4 tries or more, regardless of win/lose/draw.
 plus 1 bonus pt if a team has lost but by 7 game points or less.
…there are also 3 additional points awarded if a team wins the ‘Grand Slam’ (wins all of their matches). The candidate for this is Wales only. They have so far won every match, and if they win their final match they get these extra points to ensure they win the tournament.
This rule avoids the situations where a team could lose one match but obtain maximum bonus points in the other, finishing up with more points overall than a team that has won every match but never obtained any bonus points.
So then, what are the final standings likely to look like?
After having run a simulation of the final round, the results are below.
Wales are “firm favourites” to win the tournament. England have a “reasonable chance”. Ireland retain an “outside chance”.
How does all of this compare to expectations before the start of the tournament?
Ireland, England, and Wales were predicted to be in close contention. Wales have outperformed the prediction (mainly due to beating England). England have outperformed the prediction (mainly due to beating Ireland, and due to amassing a lot of bonus points). Ireland have under performed against the prediction (mainly due to losing to England, and then narrowly missing out on bonus points: scored only 3 tries against Scotland; lost to England by only 12 game pt).
Scotland beat Italy with a bonus point victory, but they have only managed to pick up one bonus point in their other games. Picking up points against England in their final match will be tough. So they will be likely to under perform. France will likely beat Italy and perform roughly as expected. Italy are looking firm against the prediction of finishing bottom again this year (however imho they could be a team to watch in their final match, as they’ll be playing a presently disorientated France, at home in Rome).
It has been an interesting journey for me simulating sports tournaments over the past few months. Monte Carlo approaches can help you see the wood from the trees in complex situations, which has applications not just in sport but in industry as well.
Maybe this has inspired you to have a go yourself? If so, the code for this blog post is available via Git here. Although if you wish to have a play or to adopt the code, the original version is much cleaner, available here. Good luck!
var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));To leave a comment for the author, please follow the link and comment on their blog: R – stats on the cloud. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
Create interactive ggplot2 graphs with plotly
(This article was first published on Little Miss Data, and kindly contributed to Rbloggers)
One of the main reasons that I fell in love with R is for ggplot2. As Jennifer Thompson so eloquently put it:
“I *used* R before ggplot, but I never loved until then”
— Jennifer Thompson
As someone very interested in storytelling, ggplot2 is easily my data visualization tool of choice. It is like the Swiss army knife for data visualization. One of my favorite features is the ability to pack a graph chockfull of dimensions. This ability is incredibly handy during the data exploration phases. However, sometimes I find myself wanting to look at trends without all the noise. Specifically, I often want to look at very dense scatterplots for outliers. Ggplot2 is great at this, but when we’ve isolated the points we want to understand, we can’t easily examine all possible dimensions right in the static charts.
Enter plotly. The plotly package and ggploty function do an excellent job at taking our high quality ggplot2 graphs and making them interactive.
During this tutorial, we are going to explore the median reported wages of creative occupations within the city of Austin for 2016 and 2017. This data set was sourced from the Austin open data portal. The AustinGO2.0 team is amazing for continually making new and exciting local data sets available for us to explore!
Install and Load Packages
Before we get rolling, we need to install and load the necessary packages.
##### Install and load packages install.packages("plotly") install.packages("tidyverse") install.packages("htmlwidgets") library(plotly) library(tidyverse) library(htmlwidgets)Load The Data Set
To make things more reproducible, I have download the data set from the Austin open data portal and uploaded it to my github repo.
##### Import data creatives < read.csv(file="https://raw.githubusercontent.com/lgellis/MiscTutorial/master/ggplotly/Median_Earnings_of_Creative_Sector_Occupations__CLL.B.1.csv", header=TRUE, sep=",", stringsAsFactors = FALSE) dim(creatives) summary(creatives) str(creatives) head(creatives) Transform the dataWe are performing some simple transformations to prep the data for an easy scatterplot. First, I renamed the columns and then I created a new column to calculate the 2016 to 2017 year over year percent changes in median wage per occupation.
##### Data processing #Rename the columns and create a new column for year over year change creatives < creatives %>% rename(Median_2016 = X2016.Median.hourly.earnings, Median_2017 = X2017.Median.hourly.earnings) %>% rowwise() %>% mutate(Percent_Improvement = round((Median_2016Median_2017)/Median_2016*100,2)) Create the scatterplotI created a simple ggplot2 scatterplot of the occupations showing the 2017 median wage vs year over year percent improvement. I added a simple horizontal line to mark zero on the plot. This allows us to more easily digest the year over year changes in median wage. We then take one final step and input our ggplot2 scatterplot into the ggplotly function. This creates an interactive graph!
##### Create a scatterPlot scatterPlot < creatives %>% ggplot(aes(x = Median_2017, y = Percent_Improvement)) + geom_point(alpha=0.7, colour = "#51A0D5") + labs(x = "Median Occupation Hourly Wage in 2017", y = "% Improvement Year over Year (2016 to 2017)", title = "Austin Creative Occupations Median Hourly Wage") + geom_hline(yintercept=0, linetype="dashed", color = "#2C528C", size=0.5) + theme_classic() ggplotly(scatterPlot)Add the Labels
The above graph is great because we’ve successfully used plotly to make the ggplot2 scatterplot interactive. However, the mouseover data doesn’t have everything we want and it’s not very nicely formatted. Let’s fix this!
Shout out to Corinne Leopold on my team, who found a much more efficient way of assigning labels in the plots than the solution I was previously using. We simply add the mouseover details through the ggplot aesthetic text property. We then assign it to the tooltip in the ggplotly function.
##### Adding text scatterPlot < creatives %>% ggplot(aes(x = Median_2017, y = Percent_Improvement, text = paste( "Occupation: ", Occupation, "\n", "2017: ", Median_2017, "\n", "2016: ", Median_2016, "\n", "% Improvement Year over Year: ", Percent_Improvement, "\n", sep = "" ))) + geom_point(alpha=0.7, colour = "#51A0D5") + labs(x = "Median Occupation Hourly Wage in 2017", y = "% Improvement Year over Year (2016 to 2017)", title = "Austin Creative Occupations Median Hourly Wage") + geom_hline(yintercept=0, linetype="dashed", color = "#2C528C", size=0.5) + theme_classic() ggplotly(scatterPlot, tooltip = "text")
Save the Data Locally
Now we have the interactive scatterplot and it’s time to share it around! Start by saving locally.
##### Save it locally htmlwidgets::saveWidget(as_widget(p), "OccupationWages.html")Save the Data on plot.ly
You may also want to share your interactive graph by sending folks a link. A super easy way of doing this is just to upload it to your plotly account. Simply make an account, create your API keys and then run the code below with your username and password.
##### Save it on ploty #Create a plotly account: https://plot.ly/ #Create API Keys: https://plot.ly/settings/api Sys.setenv("plotly_username"="USERNAME") Sys.setenv("plotly_api_key"="PASSWORD") #2) Save to online file api_create(scatterPlot, filename = "Occupation Wages")After running the code, the URL for your hosted plotly chart will pop up and voila, you have a hosted plotly graph! You can send it off to friends and colleagues or you can embed it into a website like I did!
THANK YOU
Thank you for exploring how we can use the plotly package to make our ggplots interactive. Please comment below if you enjoyed this blog, have questions, or would like to see something different in the future. Note that the full code is available on my github repo. Due to some great suggestions on twitter by Christin Zasada and Alison Hill, I even included a nicely formatted .md file with the code and output displayed for easier reproducibility.
If you have trouble downloading the files or cloning the repo from github, please go to the main page of the repo and select “Clone or Download” and then “Download Zip”. Alternatively or you can execute the following R commands to download the whole repo through R (an awesome suggestion by Itamar Caspi)
install.packages("usethis") library(usethis) use_course("https://github.com/lgellis/MiscTutorial/archive/master.zip") var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));To leave a comment for the author, please follow the link and comment on their blog: Little Miss Data. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
On the poor performance of classifiers in insurance models
(This article was first published on Renglish – Freakonometrics, and kindly contributed to Rbloggers)
Each time we have a case study in my actuarial courses (with real data), students are surprised to have hard time getting a “good” model, and they are always surprised to have a low AUC, when trying to model the probability to claim a loss, to die, to fraud, etc. And each time, I keep saying, “yes, I know, and that’s what we expect because there a lot of ‘randomness’ in insurance”. To be more specific, I decided to run some simulations, and to compute AUCs to see what’s going on. And because I don’t want to waste time fitting models, we will assume that we have each time a perfect model. So I want to show that the upper bound of the AUC is actually quite low ! So it’s not a modeling issue, it is a fondamental issue in insurance !
By ‘perfect model’ I mean the following : denotes the heterogeneity factor, because people are different. We would love to get . Unfortunately, is unobservable ! So we use covariates (like the age of the driver of the car in motor insurance, or of the policyholder in life insurance, etc). Thus, we have data ‘s and we use them to train a model, in order to approximate . And then, we check if our model is good (or not) using the ROC curve, obtained from confusion matrices, comparing ‘s and ‘s where when exceeds a given threshold. Here, I will not try to construct models. I will predict each time the true underlying probability exceeds a threshold ! The point is that it’s possible to claim a loss () even if the probability is 3% (and most of the time ), and to not claim one () even if the probability is 97% (and most of the time ). That’s the idea with randomness, right ?
So, here denote the probabilities to claim a loss, to die, to fraud, etc. There is heterogeneity here, and this heterogenity can be small, or large. Consider the graph below, to illustrate,
In both cases, there is, on average, 25% chance to claim a loss. But on the left, there is more heterogeneity, more dispersion. To illustrate, I used the arrow, which is a classical 90% interval : 90% of the individuals have a probability to claim a loss in that interval. (here 10%40%), 5% are below 10% (low risk), and 5% are above 40% (high risk). Later on, we will say that we have 25% on average, with a dispersion of 30% (40% minus 10%). On the right, it’s more 25% on average, with a dispersion of of 15%. What I call dispersion is the difference between the 95% and the 5% quantiles.
Consider now some dataset, with Bernoulli variables , drawn with those probabilities . Then, let us assume that we are able to get a perfect model : I do not estimate a model based on some covariates, here, I assume that I know perfectly the probability (which is true, because I did generate those data). More specifically, to generate a vector of probabilities, here I use a Beta distribution with a given mean, and a given variance (to capture the heterogeneity I mentioned above)
1 2 3 a=m*(m*(1m)/v1) b=(1m)*(m*(1m)/v1) p=rbeta(n,a,b)from those probabilities, I generate occurences of claims, or deaths,
1 Y=rbinom(n,size = 1,prob = p)Then, I compute the AUC of my “perfect” model,
1 auc.tmp=performance(prediction(p,Y),"auc")And then, I will generate many samples, to compute the average value of the AUC. And actually, we can do that for many values of the mean and the variance of the Beta distribution. Here is the code
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 library(ROCR) n=1000 ns=200 ab_beta = function(m,inter){ a=uniroot(function(a) qbeta(.95,a,a/ma)qbeta(.05,a,a/ma)inter, interval=c(.0000001,1000000))$root b=a/ma return(c(a,b)) } Sim_AUC_mean_inter=function(m=.5,i=.05){ V_auc=rep(NA,ns) b=1 essai = try(ab<ab_beta(m,i),TRUE) if(inherits(essai,what="tryerror")) a=1 if(!inherits(essai,what="tryerror")){ a=ab[1] b=ab[2] } if((a>=0)&(b>=0)){ for(s in 1:ns){ p=rbeta(n,a,b) Y=rbinom(n,size = 1,prob = p) auc.tmp=performance(prediction(p,Y),"auc") V_auc[s]=as.numeric(auc.tmp@y.values)} L=list(moy_beta=m, var_beat=v, q05=qbeta(.05,a,b), q95=qbeta(.95,a,b), moy_AUC=mean(V_auc), sd_AUC=sd(V_auc), q05_AUC=quantile(V_auc,.05), q95_AUC=quantile(V_auc,.95)) return(L)} if((a<0)(b<0)){return(list(moy_AUC=NA))}} Vm=seq(.025,.975,by=.025) Vi=seq(.01,.5,by=.01) V=outer(X = Vm,Y = Vi, Vectorize(function(x,y) Sim_AUC_mean_inter(x,y)$moy_AUC)) library("RColorBrewer") image(Vm,Vi,V, xlab="Probability (Average)", ylab="Dispersion (Q95Q5)", col= colorRampPalette(brewer.pal(n = 9, name = "YlGn"))(101)) contour(Vm,Vi,V,add=TRUE,lwd=2)
On the xaxis, we have the average probability to claim a loss. Of course, there is a symmetry here. And on the yaxis, we have the dispersion : the lower, the less heterogeneity in the portfolio. For instance, with a 30% chance to claim a loss on average, and 20% dispersion (meaning that in the portfolio, 90% of the insured have between 20% and 40% chance to claim a loss, or 15% and 35% chance), we have on average a 60% AUC. With a perfect model ! So with only a few covariates, having 55% should be great !
My point here is that with a low dispersion, we cannot expect to have a great AUC (again, even with a perfect model). In motor insurance, from my experience, 90% of the insured are between 3% chance and 20% chance to claim a loss ! That’s less than 20% dispersion ! and in that case, even if the (average) probability is rather small, it is very difficult to expect an AUC above 60% or 65% !
var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));To leave a comment for the author, please follow the link and comment on their blog: Renglish – Freakonometrics. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
RStudio 1.2 Preview: Jobs
(This article was first published on RStudio Blog, and kindly contributed to Rbloggers)
When you run an R script in RStudio today, the R console waits for it to complete, and you can’t do much with RStudio until the script is finished running. When your R scripts take a long time to run, it can be difficult to get much done in RStudio while they do, unless you’re willing to juggle multiple instances of RStudio.
In RStudio 1.2, we’re introducing two new features to keep you productive while your code’s working: local jobs and remote jobs. You can use these to run your scripts in the background while you continue to use the IDE.
Local jobsA “local job” is an R script that runs in a separate, dedicated R session. You can run any R script in a separate session by pulling down the Source menu and choosing Source as Local Job.
This will give you some options for running your job.
By default, the job will run in a clean R session, and its temporary workspace will be discarded when the job is complete. This is the fastest and safest configuration, good for reproducible scripts that have no side effects.
However, if you want to feed data from your current R session into the job, or have the job return data to your current R session, change the dialog options as follows:
Run job with copy of global environment: If ticked, this option saves your global environment and loads it into the job’s R session before it runs. This is useful because it will allow your job to see all the same variables you can see in the IDE. Note that this can be slow if you have large objects in your environment.
Copy job results: By default, the temporary workspace in which the job runs is not saved. If you’d like to import data from your job back into your R session, you have a couple of choices:
Global environment: This places all the R objects your job creates back in your R session’s global environment. Use this option with caution! The objects created by the job will overwrite, without a warning, any objects that have the same name in your environment.
Results object: This places all the R objects your job creates into a new environment named yourscript_results.
LifetimeLocal jobs run as noninteractive child R processes of your main R process, which means that they will be shut down if R is. While your R session is running jobs:
 You will be warned if you attempt to close the window while jobs are still running (on RStudio Desktop)
 Your R session will not be suspended (on RStudio Server)
While local jobs are running, a progress bar will appear in the R console summarizing the progress of all running jobs.
Detailed progressThe progress bar RStudio shows for your job represents the execution of each toplevel statement in your R script. If you want a little more insight into which part of the script is currently running, you can use RStudio’s code sections feature. Add a section marker like this to your R script:
# Apply the model When your job reaches that line in your script, the name of the section will appear on the progress bar.
You can also emit output using the usual R mechanisms, such as print, message, and cat. This output appears in the Jobs pane when you select your job.
ScriptingYou can script the creation of jobs using the rstudioapi package method jobRunScript; it has options which correspond to each dialog option above. This makes it possible to automate and orchestrate more complicated sets of background tasks.
Note however that the IDE’s background job runner is generally designed for oneoff, interactive script runs. If you are writing R code and need to run a subtask asynchronously in a background R session, we recommend using the new callr package instead.
Remote (Launcher) jobsOn RStudio Server Pro, you also have the option of running your R script on your company’s compute infrastructure, using the new Job Launcher. To do this, select:
When launching a job, you’ll have the opportunity to specify how you want to run it, depending of course on the configuration the compute infrastructure exposes to RStudio Server. This can include settings like resource constraints as well as configuration parameters like which Docker image to use.
Monitoring launcher jobsUnlike local jobs, launcher jobs are independent from the R session. You can safely quit your R session without affecting any launcher jobs you may have started from it. Once you have started a job, you can see its status in the Launcher tab, which shows all your jobs (not just those launched from the current session).
You can also monitor the status and progress of your launcher jobs on your RStudio dashboard:
Showing task progressRStudio’s new Jobs pane can show more than just the progress of background jobs. It can also be scripted from R packages (and R code) to show status, progress, and output for any longrunning task.
If you’d like to show progress and/or output from a task using the jobs UI, refer to the rstudioapi documentation for details; start with addJob, which creates a new job in the UI and returns a handle you can use to update the UI as the job progresses.
WrapupWe hope RStudio’s new Jobs functionality helps streamline your workflow and get the most out of your hardware, especially if you often work with R scripts that take time to execute. Try out the new functionality in the RStudio 1.2 Preview Release (stable release coming very soon), and let us know what you think on the community forum!
var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));To leave a comment for the author, please follow the link and comment on their blog: RStudio Blog. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
Zotero hacks: unlimited synced storage and its smooth use with rmarkdown
(This article was first published on Ilya Kashnitsky, and kindly contributed to Rbloggers)
Here is a bit refreshed translation of my 2015 blog post, initially published on Russian blog platform habr.com. The post shows how to organize a personal academic library of unlimited size for free. This is a funny case of a self written manual which I came back to multiple times myself and many many more times referred my friends to it, even nonRussian speakers who had to use Google Translator and infer the rest from screenshots. Finally, I decided to translate it adding some basic information on how to use Zotero with rmarkdown.
A brief (and hopefully unnecessary for you) intro of bibliographic managersBibliographic manager is a life saver in everyday academic life. I suffer almost physical pain just thinking about colleagues who for some reason never started using one – all those excel spreadsheets with favorite citations, messy folders with PDFs, constant hours lost for the joykilling task of manual reference list formatting. Once you start using a reference manager this all becomes a happily forgotten nightmare.
I tend to think of bibliographic metadata as LEGO.
For each paper (book chapter / preprint / R package) we have a number of metadata pieces – title, authors, date published, etc. These are the LEGO blocks. For different bibliographic styles we just need to reshuffle those blocks inserting various commas, semicolons, and quotation marks.
Bibliographic manager keeps track of all the LEGO blocks and knows (learns easily) how to compose proper citation styles out of them. All we need is to download a specific journal’s citation style. There are more than six thousand bibliographic styles! [This is my #1 argument against the conspiracy ideas of some centralized power that rules our world .)]
Why Zotero?There are dozens of bibliographic managers out there (see a comparative table). Some of them are free, the others require paid subscriptions. Probably, the most popular two are Zotero and Mendeley. Both are free to use and make money by offering cloud storage to sync PDFs of the papers. Yet, both give some limited storage for free – Zotero gives 300MB, and Mendeley gives 2GB.
Why do I choose and recommend Zotero then? Because it’s fairly easy to setup Zotero so that the free 300MB are only used to sync metadata (which in practice means almost infinite storage), and the PDFs are synced separately using a cloud solution of one’s choice (I use Google Drive). It’s the main setup hack that I’m showing in this blog post. There is no similar hack for Mendeley, and with them at some point one is bound to pay for extra storage.
Another consideration in favor of Zotero is that it’s an opensource program with strong community and outspoken commitment to stay free forever, while Mendeley is an Elsevier forprofit product. Academic community knows a lot about Elsevier in particular and forprofit products in general. Here the story of Academia.edu is very indicative. Have a look at this Forbes piece. As a careerlong decision I’m confident in choosing Zotero. And the project keeps developing nicely – just look at the recent Zotero blog entries on the new features such as Google Docs integration, Unpaywall integration and a new web service for quick citations.
Finally, an example of how strong Zotero community is. Once I figured out there the style repository does not have a style for Demographic Research, one of the best journals in demography. I’ve opened a request on Zotero forum and in two days the style was created.
Prerequisites
Download and install Zotero. It’s crossplatform and works smoothly with various systems, even when the same database is sycned in parallel on machines with different operation systems. I’ve used win+linux and win+mac – no sync problems ever.

From the same download page go to install Zotero Connector, a browser extension that helps to retrieve bibliographic metadata.

Create an account on Zotero website. It will be used later on to sync the database of bibliographic metadata.

Download and install the two plugins we’ll need – ZotFile (organizes the database of PDFs) and Better BibTeX (exports the library to .bib, we’ll use it later with rmarkdown). The plugins for Zotero are .xpi archives. To install the plugins open Zotero and click Tools > Addons. A separate window for Addons manager will popup.
There we need to click the options gear button and select Install Addon From File option. Finally navigate to the .xpi file and install. Zotero will ask to restart, please do.
We are ready to go through the setup stepbystep.
Zotero preferencesFirst, let’s walk though Zotero Preferences. To edit them click Edit > Preferences. A window with several tabs pops up.
General. I only uncheck the option to create automatic web page snapshots which I find not so useful compared with all the cluttering effect of all those multiple small files needed to replicate an html page locally.
Sync. Here we need to specify the account details to sync our database. It is important to uncheck the option of fulltext sync otherwise the 300MB storage will quickly get filled. We’ll have the solution for full text a bit later.
Search. Defines the database for internal search engine. Defaults are reasonable.
Export. Choose the style for quick export using Shift+Ctrl+C shortcut.
Cite. Citation styles stored locally. One nice feature here is the Get additional styles link which brings an integrated selection from the whole Zotero Styles Database. Styles can also be installed from local .csl files, for that press the + button. Don’t miss the Word Processors subtab. There we can get the plugins that integrate Zotero to Microsoft Word and Libre Office.
Advanced. Here we are most interested in the subtab Files and Folders. This is the most important step to separate the storage of metadata and files.
The first path should lead to a directory which stores the fulltext PDFs, I call it zoterolibrary. This directory should be somewhere in the part of the local file system that is synced. In my case it’s the directory named ikashnitsky, which I sync with Google Drive. The second path leads to the system files of Zotero, I call it zoterosystem. This directory should be placed somewhere in the nonsynced part of the local file system. It will be updated by the native Zotero sync, and it’s better if those system files are not overwritten by any external sync software.
Better BibTeX. This tab appears after we install the Better BibTeX extension. The extension is needed to export the whole bibliographic library (or some of its parts) as a plain .bib text file. This step is needed to use Zotero in RStudio while writing academic papers with rmarkdown.
The most important option here is to define the rules for creating citation keys. There are almost infinite number of ways one can define these keys (check the manual). My personal choice is [auth:lower][year][journal:lower:abbr], which means that a key consists of the first author’s name, publication year, and the first letters abbreviation of the journal’s title, everything in lower case. Thus the key for my most recent paper published in Tijdschrift voor economische en sociale geografie is kashnitsky2019tveesg.
ZotFile PreferencesNext we need to setup ZotFile. This extension helps to rename PDFs according to predefined rules and store them in a hierarchical database with meaningful names of the subdirectories. To open the setup window click Tools > ZotFile Preferences. Again, the window has several tabs.
General. Here we define two paths. The first is the default location of the files downloaded by your browser. This option tells ZotFile where to look for the PDFs to process when you import a paper from the publisher’s website (recall that earlier we installed Zotero Connector). The second path leads to the local directory created for the fulltext PDFs, the one that I named zoterolibrary and which is synced with an external cloud solution of our choice.
To navigate easier in this database of PDFs check the option Use subfolder defined by. Here again we have a wide choice of the ways to define the rules to name the subdirectories. Click the info icon to learn the options. I choose to simply have a separate folder for each first author.
Tablet Settings. Apparently, this menu allows to setup an exchange of PDFs with a tablet. I’ve never used it, thus omit.
Renaming Rules. Here it’s important to make sure that ZotFile is responsible for renaming. Then we define how to rename the PDFs based on the bibliographic metadata available. Again, here we have many many options. My choice is {%a_}{%y_}{%t} which yields file names like kashnitsky_2018_russian_periphery_is_dying_in_movement.pdf (again an example for my recent paper in GeoJournal).
Advanced Settings. I only checked the option to replace all the nonstandard symbols with plain ASCII.
A very important note on ZotFile!.
If you parse the metadata manually from a PDF, make sure to rename the file using ZotFile. For that rightclick the metadata record Manage Attachments > Rename Attachments. This action explicitly tells to use ZotFile for renaming and will move the renamed PDF to a proper subdirectory. The attachment in Zotero should not look like a PDF file…
… but rather should be a link to the renamed file.
In these screenshot I also show the location of the actual PDFs in both cases (rightclick the metadata record Show File). As you can see, in the first case the PDF is located in a meaninglessly named folder somewhere in the zoterosystem directory. In contrast, the renamed by ZotFile PDF is located in a properly named subdirectory in zoterolibrary. Thus, in the latter case the PDF is synced to my Google Drive and can be accessed from anywhere.
More importantly, when I need to restore my whole database of academic papers on another machine, I just need to go through these steps. As long as the system metadata data base is synced by Zotero and I provide Zotero the link to a PDFs storage, it will recognize all the relative paths to the files, and the whole library is restored. This setup also makes it possible to have the same continuously synced library on multiple machines. The hack is in ZotFile which adds a file path line to the metadata of the papers.
As long as I keep the settings unchanged, everything will be synced fine across multiple devices. In the end, I enjoy the unlimited storage of my PDFs with the very nice and reliable native sync of metadata form Zotero.
Final remark on Zotero. Feel free to clean from time to time all the clutter from zoterosystem/storage.
Use Zotero library in RStudio with rmarkdownZotero has a very nice builtin integration with Microsoft Word and Libre Office. A bit of magic is needed if one wants to use it with LaTeX or (like me) with rmarkdown. The magic part is the Better BibTeX plugin, which we’ve installed and set up earlier.
Better BibTeX offers an easy way to export bibliographic records from Zotero as plain .bib text and keep the file updated once the records are changed. Just rightclick on the collection in Zotero and choose Export Collection.
Then in the next window choose to export as Better BibTeX and check the option to Keep updated.
The output .bib file should be placed in the directory from which we are going to knit the .rmd file. The name of the .bib is specified in YAML header of the .rmd. Here is an example from my running project with \@jm_aburto.
Note that the exact YAML functions may vary depending on the rmarkdown template package. In this case I’m using bookdown, which also allows to specify the desired bibliographic style, .csl file should also be copied to the knit directory.
Then, everything is ready to use the citation keys to generate citations throughout the text. For details on this syntax, it’s better to refer to the relevant chapter of @xieyihui’s book on bookdown.
The final hint here is to use citr package, which brings an easy and interactive way to select citations from the .bib file. Once the package in installed, an RStudio addin Insert citation appears which executes the citr:::insert_citation() command (you can assign a shortkey to the addin). This function brings a shiny app to select a citation interactively. More details in the github repo.
Happy paper writing with Zotero and RStudio! var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));To leave a comment for the author, please follow the link and comment on their blog: Ilya Kashnitsky. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
Visually explore Probability Distributions with vistributions
(This article was first published on Rsquared Academy Blog, and kindly contributed to Rbloggers)
We are happy to introduce the vistributions package, a set of tools for
visually exploring probability distributions.
vistributions includes a shiny app which can be launched using
vdist_launch_app()or try the live version here.
Read on to learn more about the features of vistributions, or see the
vistributions website for
detailed documentation on using the package.
The common vdist_ prefix will trigger autocomplete, allowing you to see all
vistributions functions:
In exploring statistical distributions, we focus on the following:
 what influences the shape of a distribution
 calculate probability from a given quantile
 calculate quantiles out of given probability
for the following distributions:
 Normal
 Binomial
 Chi Square
 F
 t
Visualize how changes in mean and standard deviation affect the shape of the
normal distribution.
 mean: mean of the normal distribution
 sd: standard deviation of the normal distribution
 probs: a probability value
 mean: mean of the normal distribution
 sd: standard deviation of the normal distribution
 type: lower/upper tail
Suppose X, the grade on a exam, is normally distributed with mean 60
and standard deviation 3. The teacher wants to give 10% of the class an A.
What should be the cutoff to determine who gets an A?
The teacher wants to give lower 15% of the class a D. What cutoff should the
teacher use to determine who gets an D?
The teacher wants to give middle 50% of the class a B. What cutoff should the
teacher use to determine who gets an B?
 perc: a quantile value
 mean: mean of the normal distribution
 sd: standard deviation of the normal distribution
 type: lower/upper/both tail
Let X be the IQ of a randomly selected student of a school. Assume X ~ N(90, 4).
What is the probability that a randomly selected student has an IQ below 80?
What is the probability that a randomly selected student has an IQ above 100?
vdist_normal_prob(100, mean = 90, sd = 4, type = 'upper')What is the probability that a randomly selected student has an IQ
between 85 and 100?
Visualize how changes in number of trials and the probability of success affect
the shape of the binomial distribution.
 p: a single aggregated probability of multiple trials
 n: the number of trials
 tp: the probability of success in a trial
 type: lower/upper tail
 p: probability of success
 n: the number of trials
 s: number of success in a trial
 type: lower/upper/interval/exact tail
Assume twentypercent (20%) of Magemill have no health insurance. Randomly
sample n = 12 Magemillians. Let X denote the number in the sample with no
health insurance. What is the probability that exactly 4 of the 15 sampled
have no health insurance?
What is the probability that at most one of those sampled has no health
insurance?
What is the probability that more than seven have no health insurance?
vdist_binom_prob(12, 0.2, 8, 'upper')What is the probability that fewer than 5 have no health insurance?
vdist_binom_prob(12, 0.2, c(0, 4), 'interval') Chi Square Distribution Distribution ShapeVisualize how changes in degrees of freedom affect the shape of the chi square
distribution.
 probs: a probability value
 df: degrees of freedom
 type: lower/upper tail
Let X be a chisquare random variable with 8 degrees of freedom. What is the
upper fifth percentile?
What is the tenth percentile?
vdist_chisquare_perc(0.10, 8, 'lower') Probability Calculate probability from a given quantile. Input perc: a quantile value
 df: degrees of freedom
 type: lower/upper tail
What is the probability that a chisquare random variable with 12 degrees of
freedom is greater than 8.79?
What is the probability that a chisquare random variable with 12 degrees of
freedom is greater than 8.62?
Visualize how changes in degrees of freedom affect the shape of the F
distribution.
 probs: a probability value
 num_df: nmerator degrees of freedom
 den_df: denominator degrees of freedom
 type: lower/upper tail
Let X be an F random variable with 4 numerator degrees of freedom and 5
denominator degrees of freedom. What is the upper twenth percentile?
What is the 35th percentile?
vdist_f_perc(0.35, 4, 5, 'lower') Probabilities Calculate probability from a given quantile. Input perc: a quantile value
 num_df: nmerator degrees of freedom
 den_df: denominator degrees of freedom
 type: lower/upper tail
What is the probability that an F random variable with 4 numerator degrees of
freedom and 5 denominator degrees of freedom is greater than 3.89?
What is the probability that an F random variable with 4 numerator degrees of
freedom and 5 denominator degrees of freedom is less than 2.63?
Visualize how degrees of freedom affect the shape of t distribution.
vdist_t_plot(df = 8) Percentiles Calculate quantiles out of given probability Input probs: a probability value
 df: degrees of freedom
 type: lower/upper/both tail
What is the upper fifteenth percentile?
vdist_t_perc(0.15, 8, 'upper')What is the eleventh percentile?
vdist_t_perc(0.11, 8, 'lower')What is the area of the curve that has 95% of the t values?
vdist_t_perc(0.8, 8, 'both') Probabilities Calculate probability from a given quantile. Input perc: a quantile value
 df: degrees of freedom
 type: lower/upper/interval/both tail
Let T follow a tdistribution with r = 6 df.
What is the probability that the value of T is less than 2?
vdist_t_prob(2, 6, 'lower')What is the probability that the value of T is greater than 2?
vdist_t_prob(2, 6, 'upper')What is the probability that the value of T is between 2 and 2?
vdist_t_prob(2, 6, 'both')What is the probability that the absolute value of T is greater than 2?
vdist_t_prob(2, 6, 'interval') Learning MoreThe vistributions website includes
comprehensive documentation on using the package, including the following
article that gives a brief introduction to vistributions:
vistributions has been on CRAN for a few months now while we were fixing bugs and
making the API stable. All feedback is welcome. Issues (bugs and feature
requests) can be posted to github tracker.
For help with code or other related questions, feel free to reach out to us
at pkgs@rsquaredacademy.com.
To leave a comment for the author, please follow the link and comment on their blog: Rsquared Academy Blog. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
Parsnipping Fama French
(This article was first published on R Views, and kindly contributed to Rbloggers)
Today, we will continue our exploration of developments in the world of tidy models, and we will stick with our usual Fama French modeling flow to do so. For new readers who want get familiar with Fama French before diving into this post, see here where we covered importing and wrangling the data, here where we covered rolling models and visualization, and here where we covered managing many models. If you’re into Shiny, this flexdashboard might be of interest, as well.
Let’s get to it.
First, we need our data and, as usual, we’ll import data for daily prices of five ETFs, convert them to returns (have a look here for a refresher on that code flow), then import the five Fama French factor data and join it to our five ETF returns data. Here’s the code to make that happen (this code was covered in detail in this post:
symbols < c("SPY", "EFA", "IJS", "EEM", "AGG") # The prices object will hold our daily price data. prices < getSymbols(symbols, src = 'yahoo', from = "20121231", to = "20171231", auto.assign = TRUE, warnings = FALSE) %>% map(~Ad(get(.))) %>% reduce(merge) %>% `colnames<`(symbols) asset_returns_long < prices %>% tk_tbl(preserve_index = TRUE, rename_index = "date") %>% gather(asset, prices, date) %>% group_by(asset) %>% mutate(daily_returns = (log(prices)  log(lag(prices)))) %>% na.omit() factors_data_address < "http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/ftp/Global_5_Factors_Daily_CSV.zip" factors_csv_name < "Global_5_Factors_Daily.csv" temp < tempfile() download.file( # location of file to be downloaded factors_data_address, # where we want R to store that file temp, quiet = TRUE) Global_5_Factors < read_csv(unz(temp, factors_csv_name), skip = 6 ) %>% rename(date = X1, MKT = `MktRF`) %>% mutate(date = ymd(parse_date_time(date, "%Y%m%d")))%>% mutate_if(is.numeric, funs(. / 100)) %>% select(RF) data_joined_tidy < asset_returns_long %>% left_join(Global_5_Factors, by = "date") %>% na.omit()For today, let’s work with just the SPY data by filtering our data set by asset.
spy_2013_2017 < data_joined_tidy %>% filter(asset == "SPY")Next, we resample this five years’ worth of data into smaller subsets of training and testing sets. This is frequently done by kfold cross validation (see here for an example), where random samples are taken from the data, but since we are working with time series, we will use a timeaware technique. The rsample package has a function for exactly this purpose, the rolling_origin() function. We covered this process extensively in this previous post. Here’s the code to make it happen.
rolling_origin_spy_2013_2017 < rolling_origin( data = spy_2013_2017, initial = 100, assess = 1, cumulative = FALSE ) rolling_origin_spy_2013_2017 %>% dim() [1] 1159 2We now have a data object called rolling_origin_spy_2013_2017 that holds 1159 splits of data. Each split consists of an analysis data set with 100 days of return and factor data, and an assessment data set with one day of return and factor data.
Now, we can start using that collection of data splits to fit a model on the assessment data, and then test our model on the assessment data. That means it’s time to introduce a relatively new addition to the R tool chain, the parsnip package.
parsnip is a unified model interface that allows us to create a model specification, set an analytic engine, and then fit a model. It’s a ‘unified’ interface in the sense that we can use the same scaffolding but insert different models, or different engines, or different modes. Let’s see how that works with linear regression.
Recall that in the previous post, we piped our data into a linear model like so:
analysis(rolling_origin_spy_2013_2017$splits[[1]]) %>% do(model = lm(daily_returns ~ MKT + SMB + HML + RMW + CMA, data = .)) %>% tidy(model) # A tibble: 6 x 6 # Groups: asset [1] asset term estimate std.error statistic p.value 1 SPY (Intercept) 0.000579 0.000338 1.71 8.98e 2 2 SPY MKT 0.909 0.0739 12.3 2.79e21 3 SPY SMB 0.495 0.112 4.43 2.52e 5 4 SPY HML 0.609 0.208 2.92 4.38e 3 5 SPY RMW 0.591 0.259 2.28 2.47e 2 6 SPY CMA 0.395 0.206 1.92 5.81e 2Now, we will pipe into the parsnip scaffolding, which will allow us to quickly change to a different model and specification further down in the code.
Since we are running a linear regression, we first create a specification with linear_reg(), then set the engine with set_engine("lm"), and finally fit the model with fit(five_factor_model, data = one of our splits)
lm_model < linear_reg() %>% set_engine("lm") %>% fit(daily_returns ~ MKT + SMB + HML + RMW + CMA, data = analysis(rolling_origin_spy_2013_2017$splits[[1]])) lm_model parsnip model object Call: stats::lm(formula = formula, data = data) Coefficients: (Intercept) MKT SMB HML RMW 0.0005794 0.9086303 0.4951297 0.6085088 0.5910375 CMA 0.3954515Now that we’ve fit the model on our test set, let’s see how well it predicted the test set. We can use the predict() function and pass it the results of our parnsip code flow, along with the assessment split.
assessment(rolling_origin_spy_2013_2017$splits[[1]]) %>% select(returns) %>% bind_cols(predict(lm_model, new_data = assessment(rolling_origin_spy_2013_2017$splits[[1]]))) # A tibble: 1 x 3 # Groups: asset [1] asset returns .pred 1 SPY 148. 0.00737That worked well, but now let’s head to a more complex model and use the ranger package as an engine for a random forest analysis.
To set up the ranger random forest model in parsnip, we first use rand_forest(mode = "regression", mtry = 3, trees = 100) to create the specification, set_engine("ranger") to set the engine as the ranger package, and fit(daily_returns ~ MKT + SMB + HML + RMW + CMA ~ , data = analysis(rolling_origin_spy_2013_2017$splits[[1]]) to fit the fivefactor Fama French model to the 100day sample in our first split.
# Need to load the packages to be used as the random forest engine library(ranger) rand_forest(mode = "regression", mtry = 3, trees = 100) %>% set_engine("ranger") %>% fit(daily_returns ~ MKT + SMB + HML + RMW + CMA, data = analysis(rolling_origin_spy_2013_2017$splits[[1]])) parsnip model object Ranger result Call: ranger::ranger(formula = formula, data = data, mtry = ~3, num.trees = ~100, num.threads = 1, verbose = FALSE, seed = sample.int(10^5, 1)) Type: Regression Number of trees: 100 Sample size: 100 Number of independent variables: 5 Mtry: 3 Target node size: 5 Variable importance mode: none Splitrule: variance OOB prediction error (MSE): 1.514654e05 R squared (OOB): 0.6880896Notice that ranger gives us an OOB prediction error (MSE) value as part of its return. parsnip returns to us what the underlying engine returns.
Now, let’s apply that random forest regression to all 1159 of our splits (recall that each split consists of 100 days of training data and one day of test data), so we can get an average RMSE. Warning: this will consume some resources on your machine and some time in your day.
To apply that model to our entire data set, we create a function that takes one split, passes it to our parsnip enabled model, and then uses the predict function to attempt to predict our assessment split. The function also allows us to specify the number of trees and the number of variables randomly sampled at each tree split, which is set with the mtry argument.
ranger_rf_regress < function(mtry = 3, trees = 5, split){ analysis_set_rf < analysis(split) model < rand_forest(mtry = mtry, trees = trees) %>% set_engine("ranger") %>% fit(daily_returns ~ MKT + SMB + HML + RMW + CMA, data = analysis_set_rf) assessment_set_rf < assessment(split) assessment_set_rf %>% select(date, daily_returns) %>% mutate(.pred = unlist(predict(model, new_data = assessment_set_rf))) %>% select(date, daily_returns, .pred) }Now we want to pass it our object of 1159 splits, rolling_origin_spy_2013_2017$splits, and we want the function to iterate over each split. For that we turn to map_df() from the purrr package, which allows us to iterate over the data object and return a data frame. map_df() takes the data as an argument and our function as an argument.
ranger_results < map_df(.x = rolling_origin_spy_2013_2017$splits, ~ranger_rf_regress(mtry = 3, trees = 100, split = .x))Here are the results. We now have 1159 predictions.
ranger_results %>% head() # A tibble: 6 x 4 # Groups: asset [1] asset date daily_returns .pred 1 SPY 20130528 0.00597 0.00583 2 SPY 20130529 0.00652 0.00403 3 SPY 20130530 0.00369 0.00658 4 SPY 20130531 0.0145 0.0114 5 SPY 20130603 0.00549 0.00119 6 SPY 20130604 0.00482 0.00202Notice how the date of each prediction is included since we included it in the select() call in our function. That will come in handy for charting later.
Now, we can use the rmse() function from yardstick to calculate the root meansquared error each of our predictions (our test sets had only one observation in them because we were testing on one month, so the RMSE is not a complex calculation here, but it would be the same code pattern if we had a larger test set). We can then find the average RMSE by calling summarise(avg_rmse = mean(.estimate)).
library(yardstick) ranger_results %>% group_by(date) %>% rmse(daily_returns, .pred) %>% summarise(avg_rmse = mean(.estimate)) # A tibble: 1 x 1 avg_rmse 1 0.00253We have the average RMSE; let’s see if the RMSE were stable over time, first with ggplot.
ranger_results %>% group_by(date) %>% rmse(daily_returns, .pred) %>% ggplot(aes(x = date, y = .estimate)) + geom_point(color = "cornflowerblue") + labs(y = "rmse", x = "", title = "RMSE over time via Ranger RF")And with highcharter.
ranger_results %>% group_by(date) %>% rmse(daily_returns, .pred) %>% hchart(., hcaes(x = date, y = .estimate), type = "point") %>% hc_title(text = "RMSE over time via Ranger RF") %>% hc_yAxis(title = list(text = "RMSE")){"x":{"hc_opts":{"title":{"text":"RMSE over time via Ranger RF"},"yAxis":{"title":{"text":"RMSE"},"type":"linear"},"credits":{"enabled":false},"exporting":{"enabled":false},"plotOptions":{"series":{"label":{"enabled":false},"turboThreshold":0,"showInLegend":false},"treemap":{"layoutAlgorithm":"squarified"},"scatter":{"marker":{"symbol":"circle"}}},"series":[{"group":"group","data":[{"date":"20130528",".metric":"rmse",".estimator":"standard",".estimate":0.000142752717804637,"x":1369699200000,"y":0.000142752717804637},{"date":"20130529",".metric":"rmse",".estimator":"standard",".estimate":0.00248144590239636,"x":1369785600000,"y":0.00248144590239636},{"date":"20130530",".metric":"rmse",".estimator":"standard",".estimate":0.00289139762321627,"x":1369872000000,"y":0.00289139762321627},{"date":"20130531",".metric":"rmse",".estimator":"standard",".estimate":0.00309703642561825,"x":1369958400000,"y":0.00309703642561825},{"date":"20130603",".metric":"rmse",".estimator":"standard",".estimate":0.00430144561084509,"x":1370217600000,"y":0.00430144561084509},{"date":"20130604",".metric":"rmse",".estimator":"standard",".estimate":0.00684300611337766,"x":1370304000000,"y":0.00684300611337766},{"date":"20130605",".metric":"rmse",".estimator":"standard",".estimate":0.00291214523819374,"x":1370390400000,"y":0.00291214523819374},{"date":"20130606",".metric":"rmse",".estimator":"standard",".estimate":0.00279000747041504,"x":1370476800000,"y":0.00279000747041504},{"date":"20130607",".metric":"rmse",".estimator":"standard",".estimate":0.0041295175155098,"x":1370563200000,"y":0.0041295175155098},{"date":"20130610",".metric":"rmse",".estimator":"standard",".estimate":0.00123362056195843,"x":1370822400000,"y":0.00123362056195843},{"date":"20130611",".metric":"rmse",".estimator":"standard",".estimate":0.000260231607059154,"x":1370908800000,"y":0.000260231607059154},{"date":"20130612",".metric":"rmse",".estimator":"standard",".estimate":0.00439769098328566,"x":1370995200000,"y":0.00439769098328566},{"date":"20130613",".metric":"rmse",".estimator":"standard",".estimate":0.00862505756614801,"x":1371081600000,"y":0.00862505756614801},{"date":"20130614",".metric":"rmse",".estimator":"standard",".estimate":0.00627871135416806,"x":1371168000000,"y":0.00627871135416806},{"date":"20130617",".metric":"rmse",".estimator":"standard",".estimate":0.00139361772830066,"x":1371427200000,"y":0.00139361772830066},{"date":"20130618",".metric":"rmse",".estimator":"standard",".estimate":0.00129999403832123,"x":1371513600000,"y":0.00129999403832123},{"date":"20130619",".metric":"rmse",".estimator":"standard",".estimate":0.00549970257328453,"x":1371600000000,"y":0.00549970257328453},{"date":"20130620",".metric":"rmse",".estimator":"standard",".estimate":0.010067877171231,"x":1371686400000,"y":0.010067877171231},{"date":"20130621",".metric":"rmse",".estimator":"standard",".estimate":0.00757847732215951,"x":1371772800000,"y":0.00757847732215951},{"date":"20130624",".metric":"rmse",".estimator":"standard",".estimate":0.00344560021960934,"x":1372032000000,"y":0.00344560021960934},{"date":"20130625",".metric":"rmse",".estimator":"standard",".estimate":0.00111555171287498,"x":1372118400000,"y":0.00111555171287498},{"date":"20130626",".metric":"rmse",".estimator":"standard",".estimate":0.00210720426205369,"x":1372204800000,"y":0.00210720426205369},{"date":"20130627",".metric":"rmse",".estimator":"standard",".estimate":0.00190601194984189,"x":1372291200000,"y":0.00190601194984189},{"date":"20130628",".metric":"rmse",".estimator":"standard",".estimate":0.00323689404809277,"x":1372377600000,"y":0.00323689404809277},{"date":"20130701",".metric":"rmse",".estimator":"standard",".estimate":0.0015246248388892,"x":1372636800000,"y":0.0015246248388892},{"date":"20130702",".metric":"rmse",".estimator":"standard",".estimate":0.00027955683504102,"x":1372723200000,"y":0.00027955683504102},{"date":"20130703",".metric":"rmse",".estimator":"standard",".estimate":0.00238816133375573,"x":1372809600000,"y":0.00238816133375573},{"date":"20130705",".metric":"rmse",".estimator":"standard",".estimate":0.00990653992577938,"x":1372982400000,"y":0.00990653992577938},{"date":"20130708",".metric":"rmse",".estimator":"standard",".estimate":0.0027759555185396,"x":1373241600000,"y":0.0027759555185396},{"date":"20130709",".metric":"rmse",".estimator":"standard",".estimate":0.00227547985103656,"x":1373328000000,"y":0.00227547985103656},{"date":"20130710",".metric":"rmse",".estimator":"standard",".estimate":0.00340354669550348,"x":1373414400000,"y":0.00340354669550348},{"date":"20130711",".metric":"rmse",".estimator":"standard",".estimate":0.00785244615911465,"x":1373500800000,"y":0.00785244615911465},{"date":"20130712",".metric":"rmse",".estimator":"standard",".estimate":0.00172836514222019,"x":1373587200000,"y":0.00172836514222019},{"date":"20130715",".metric":"rmse",".estimator":"standard",".estimate":0.000141612792237647,"x":1373846400000,"y":0.000141612792237647},{"date":"20130716",".metric":"rmse",".estimator":"standard",".estimate":0.00327137485857389,"x":1373932800000,"y":0.00327137485857389},{"date":"20130717",".metric":"rmse",".estimator":"standard",".estimate":0.00176641332984882,"x":1374019200000,"y":0.00176641332984882},{"date":"20130718",".metric":"rmse",".estimator":"standard",".estimate":0.00154772618154754,"x":1374105600000,"y":0.00154772618154754},{"date":"20130719",".metric":"rmse",".estimator":"standard",".estimate":0.00269690078023818,"x":1374192000000,"y":0.00269690078023818},{"date":"20130722",".metric":"rmse",".estimator":"standard",".estimate":0.00136133061201296,"x":1374451200000,"y":0.00136133061201296},{"date":"20130723",".metric":"rmse",".estimator":"standard",".estimate":0.00308872127573454,"x":1374537600000,"y":0.00308872127573454},{"date":"20130724",".metric":"rmse",".estimator":"standard",".estimate":0.00195016821410643,"x":1374624000000,"y":0.00195016821410643},{"date":"20130725",".metric":"rmse",".estimator":"standard",".estimate":0.000918117461552164,"x":1374710400000,"y":0.000918117461552164},{"date":"20130726",".metric":"rmse",".estimator":"standard",".estimate":0.00392416647401538,"x":1374796800000,"y":0.00392416647401538},{"date":"20130729",".metric":"rmse",".estimator":"standard",".estimate":0.00186499539974162,"x":1375056000000,"y":0.00186499539974162},{"date":"20130730",".metric":"rmse",".estimator":"standard",".estimate":0.000704951304307538,"x":1375142400000,"y":0.000704951304307538},{"date":"20130731",".metric":"rmse",".estimator":"standard",".estimate":0.000291087547639628,"x":1375228800000,"y":0.000291087547639628},{"date":"20130801",".metric":"rmse",".estimator":"standard",".estimate":0.00156555762355721,"x":1375315200000,"y":0.00156555762355721},{"date":"20130802",".metric":"rmse",".estimator":"standard",".estimate":0.00342466009642512,"x":1375401600000,"y":0.00342466009642512},{"date":"20130805",".metric":"rmse",".estimator":"standard",".estimate":0.00141234538646521,"x":1375660800000,"y":0.00141234538646521},{"date":"20130806",".metric":"rmse",".estimator":"standard",".estimate":0.00453039357456201,"x":1375747200000,"y":0.00453039357456201},{"date":"20130807",".metric":"rmse",".estimator":"standard",".estimate":0.000453949337123345,"x":1375833600000,"y":0.000453949337123345},{"date":"20130808",".metric":"rmse",".estimator":"standard",".estimate":0.00319715187637578,"x":1375920000000,"y":0.00319715187637578},{"date":"20130809",".metric":"rmse",".estimator":"standard",".estimate":0.00421636388783291,"x":1376006400000,"y":0.00421636388783291},{"date":"20130812",".metric":"rmse",".estimator":"standard",".estimate":0.000582989994890629,"x":1376265600000,"y":0.000582989994890629},{"date":"20130813",".metric":"rmse",".estimator":"standard",".estimate":0.00248043179424258,"x":1376352000000,"y":0.00248043179424258},{"date":"20130814",".metric":"rmse",".estimator":"standard",".estimate":0.00309464529048978,"x":1376438400000,"y":0.00309464529048978},{"date":"20130815",".metric":"rmse",".estimator":"standard",".estimate":0.00409494503279893,"x":1376524800000,"y":0.00409494503279893},{"date":"20130816",".metric":"rmse",".estimator":"standard",".estimate":0.00258176487593892,"x":1376611200000,"y":0.00258176487593892},{"date":"20130819",".metric":"rmse",".estimator":"standard",".estimate":0.00117265718611906,"x":1376870400000,"y":0.00117265718611906},{"date":"20130820",".metric":"rmse",".estimator":"standard",".estimate":0.00602758728915038,"x":1376956800000,"y":0.00602758728915038},{"date":"20130821",".metric":"rmse",".estimator":"standard",".estimate":0.000830939445518749,"x":1377043200000,"y":0.000830939445518749},{"date":"20130822",".metric":"rmse",".estimator":"standard",".estimate":0.00168966716232213,"x":1377129600000,"y":0.00168966716232213},{"date":"20130823",".metric":"rmse",".estimator":"standard",".estimate":0.00281591810324628,"x":1377216000000,"y":0.00281591810324628},{"date":"20130826",".metric":"rmse",".estimator":"standard",".estimate":0.00187026973058945,"x":1377475200000,"y":0.00187026973058945},{"date":"20130827",".metric":"rmse",".estimator":"standard",".estimate":0.00291060928304423,"x":1377561600000,"y":0.00291060928304423},{"date":"20130828",".metric":"rmse",".estimator":"standard",".estimate":0.00626027940262622,"x":1377648000000,"y":0.00626027940262622},{"date":"20130829",".metric":"rmse",".estimator":"standard",".estimate":0.00117709798349316,"x":1377734400000,"y":0.00117709798349316},{"date":"20130830",".metric":"rmse",".estimator":"standard",".estimate":0.00270900069061262,"x":1377820800000,"y":0.00270900069061262},{"date":"20130903",".metric":"rmse",".estimator":"standard",".estimate":0.000892207820183886,"x":1378166400000,"y":0.000892207820183886},{"date":"20130904",".metric":"rmse",".estimator":"standard",".estimate":0.00439848539962431,"x":1378252800000,"y":0.00439848539962431},{"date":"20130905",".metric":"rmse",".estimator":"standard",".estimate":0.000749332096425049,"x":1378339200000,"y":0.000749332096425049},{"date":"20130906",".metric":"rmse",".estimator":"standard",".estimate":0.00404723911777309,"x":1378425600000,"y":0.00404723911777309},{"date":"20130909",".metric":"rmse",".estimator":"standard",".estimate":0.00470142591028648,"x":1378684800000,"y":0.00470142591028648},{"date":"20130910",".metric":"rmse",".estimator":"standard",".estimate":0.000234752371109743,"x":1378771200000,"y":0.000234752371109743},{"date":"20130911",".metric":"rmse",".estimator":"standard",".estimate":0.00285950641404687,"x":1378857600000,"y":0.00285950641404687},{"date":"20130912",".metric":"rmse",".estimator":"standard",".estimate":0.000247670846161255,"x":1378944000000,"y":0.000247670846161255},{"date":"20130913",".metric":"rmse",".estimator":"standard",".estimate":0.0015043158443795,"x":1379030400000,"y":0.0015043158443795},{"date":"20130916",".metric":"rmse",".estimator":"standard",".estimate":0.00115698479726509,"x":1379289600000,"y":0.00115698479726509},{"date":"20130917",".metric":"rmse",".estimator":"standard",".estimate":0.00281728468879361,"x":1379376000000,"y":0.00281728468879361},{"date":"20130918",".metric":"rmse",".estimator":"standard",".estimate":0.00340383993451582,"x":1379462400000,"y":0.00340383993451582},{"date":"20130919",".metric":"rmse",".estimator":"standard",".estimate":0.00572689019624601,"x":1379548800000,"y":0.00572689019624601},{"date":"20130920",".metric":"rmse",".estimator":"standard",".estimate":0.00253135671003318,"x":1379635200000,"y":0.00253135671003318},{"date":"20130923",".metric":"rmse",".estimator":"standard",".estimate":0.000983885632568376,"x":1379894400000,"y":0.000983885632568376},{"date":"20130924",".metric":"rmse",".estimator":"standard",".estimate":0.000394017946609534,"x":1379980800000,"y":0.000394017946609534},{"date":"20130925",".metric":"rmse",".estimator":"standard",".estimate":0.00276821422041246,"x":1380067200000,"y":0.00276821422041246},{"date":"20130926",".metric":"rmse",".estimator":"standard",".estimate":0.00245223915591046,"x":1380153600000,"y":0.00245223915591046},{"date":"20130927",".metric":"rmse",".estimator":"standard",".estimate":0.00167896835475828,"x":1380240000000,"y":0.00167896835475828},{"date":"20130930",".metric":"rmse",".estimator":"standard",".estimate":0.00302581645549956,"x":1380499200000,"y":0.00302581645549956},{"date":"20131001",".metric":"rmse",".estimator":"standard",".estimate":0.00133532966948982,"x":1380585600000,"y":0.00133532966948982},{"date":"20131002",".metric":"rmse",".estimator":"standard",".estimate":0.000404804026220111,"x":1380672000000,"y":0.000404804026220111},{"date":"20131003",".metric":"rmse",".estimator":"standard",".estimate":0.00614434885800635,"x":1380758400000,"y":0.00614434885800635},{"date":"20131004",".metric":"rmse",".estimator":"standard",".estimate":0.00561509975346613,"x":1380844800000,"y":0.00561509975346613},{"date":"20131007",".metric":"rmse",".estimator":"standard",".estimate":0.00423365708880972,"x":1381104000000,"y":0.00423365708880972},{"date":"20131008",".metric":"rmse",".estimator":"standard",".estimate":0.000778471899940642,"x":1381190400000,"y":0.000778471899940642},{"date":"20131009",".metric":"rmse",".estimator":"standard",".estimate":0.00320215175070333,"x":1381276800000,"y":0.00320215175070333},{"date":"20131010",".metric":"rmse",".estimator":"standard",".estimate":0.0106384753612411,"x":1381363200000,"y":0.0106384753612411},{"date":"20131011",".metric":"rmse",".estimator":"standard",".estimate":0.00171777076192885,"x":1381449600000,"y":0.00171777076192885},{"date":"20131014",".metric":"rmse",".estimator":"standard",".estimate":0.0014617944493349,"x":1381708800000,"y":0.0014617944493349},{"date":"20131015",".metric":"rmse",".estimator":"standard",".estimate":0.00437650246614441,"x":1381795200000,"y":0.00437650246614441},{"date":"20131016",".metric":"rmse",".estimator":"standard",".estimate":0.00283682416435875,"x":1381881600000,"y":0.00283682416435875},{"date":"20131017",".metric":"rmse",".estimator":"standard",".estimate":0.000458134362367081,"x":1381968000000,"y":0.000458134362367081},{"date":"20131018",".metric":"rmse",".estimator":"standard",".estimate":0.000488303630816335,"x":1382054400000,"y":0.000488303630816335},{"date":"20131021",".metric":"rmse",".estimator":"standard",".estimate":0.000887043567693242,"x":1382313600000,"y":0.000887043567693242},{"date":"20131022",".metric":"rmse",".estimator":"standard",".estimate":0.00115608253109249,"x":1382400000000,"y":0.00115608253109249},{"date":"20131023",".metric":"rmse",".estimator":"standard",".estimate":0.00235818360568705,"x":1382486400000,"y":0.00235818360568705},{"date":"20131024",".metric":"rmse",".estimator":"standard",".estimate":0.00106066455567277,"x":1382572800000,"y":0.00106066455567277},{"date":"20131025",".metric":"rmse",".estimator":"standard",".estimate":0.00660056619588438,"x":1382659200000,"y":0.00660056619588438},{"date":"20131028",".metric":"rmse",".estimator":"standard",".estimate":0.00253966857103552,"x":1382918400000,"y":0.00253966857103552},{"date":"20131029",".metric":"rmse",".estimator":"standard",".estimate":0.000708643111683541,"x":1383004800000,"y":0.000708643111683541},{"date":"20131030",".metric":"rmse",".estimator":"standard",".estimate":0.00459062122339914,"x":1383091200000,"y":0.00459062122339914},{"date":"20131031",".metric":"rmse",".estimator":"standard",".estimate":0.00242759731143618,"x":1383177600000,"y":0.00242759731143618},{"date":"20131101",".metric":"rmse",".estimator":"standard",".estimate":0.00119101679630836,"x":1383264000000,"y":0.00119101679630836},{"date":"20131104",".metric":"rmse",".estimator":"standard",".estimate":0.00158715544003218,"x":1383523200000,"y":0.00158715544003218},{"date":"20131105",".metric":"rmse",".estimator":"standard",".estimate":0.000707140242553759,"x":1383609600000,"y":0.000707140242553759},{"date":"20131106",".metric":"rmse",".estimator":"standard",".estimate":0.000869667678226866,"x":1383696000000,"y":0.000869667678226866},{"date":"20131107",".metric":"rmse",".estimator":"standard",".estimate":3.41997301053047e05,"x":1383782400000,"y":3.41997301053047e05},{"date":"20131108",".metric":"rmse",".estimator":"standard",".estimate":0.00774212491252219,"x":1383868800000,"y":0.00774212491252219},{"date":"20131111",".metric":"rmse",".estimator":"standard",".estimate":0.00383346982479243,"x":1384128000000,"y":0.00383346982479243},{"date":"20131112",".metric":"rmse",".estimator":"standard",".estimate":0.000542045432367102,"x":1384214400000,"y":0.000542045432367102},{"date":"20131113",".metric":"rmse",".estimator":"standard",".estimate":0.00477472845551608,"x":1384300800000,"y":0.00477472845551608},{"date":"20131114",".metric":"rmse",".estimator":"standard",".estimate":7.97778727704276e05,"x":1384387200000,"y":7.97778727704276e05},{"date":"20131115",".metric":"rmse",".estimator":"standard",".estimate":0.000100208301392964,"x":1384473600000,"y":0.000100208301392964},{"date":"20131118",".metric":"rmse",".estimator":"standard",".estimate":0.00305915006399908,"x":1384732800000,"y":0.00305915006399908},{"date":"20131119",".metric":"rmse",".estimator":"standard",".estimate":0.00178020971267786,"x":1384819200000,"y":0.00178020971267786},{"date":"20131120",".metric":"rmse",".estimator":"standard",".estimate":0.00019925260780281,"x":1384905600000,"y":0.00019925260780281},{"date":"20131121",".metric":"rmse",".estimator":"standard",".estimate":0.00406160223566766,"x":1384992000000,"y":0.00406160223566766},{"date":"20131122",".metric":"rmse",".estimator":"standard",".estimate":0.000723442153834573,"x":1385078400000,"y":0.000723442153834573},{"date":"20131125",".metric":"rmse",".estimator":"standard",".estimate":0.000328744423454753,"x":1385337600000,"y":0.000328744423454753},{"date":"20131126",".metric":"rmse",".estimator":"standard",".estimate":0.00265982636503166,"x":1385424000000,"y":0.00265982636503166},{"date":"20131127",".metric":"rmse",".estimator":"standard",".estimate":0.00128358907179459,"x":1385510400000,"y":0.00128358907179459},{"date":"20131129",".metric":"rmse",".estimator":"standard",".estimate":0.00199764651921265,"x":1385683200000,"y":0.00199764651921265},{"date":"20131202",".metric":"rmse",".estimator":"standard",".estimate":0.00206032785364643,"x":1385942400000,"y":0.00206032785364643},{"date":"20131203",".metric":"rmse",".estimator":"standard",".estimate":0.000479576111824037,"x":1386028800000,"y":0.000479576111824037},{"date":"20131204",".metric":"rmse",".estimator":"standard",".estimate":0.00218374080985527,"x":1386115200000,"y":0.00218374080985527},{"date":"20131205",".metric":"rmse",".estimator":"standard",".estimate":0.000704660423616735,"x":1386201600000,"y":0.000704660423616735},{"date":"20131206",".metric":"rmse",".estimator":"standard",".estimate":0.000797892320329585,"x":1386288000000,"y":0.000797892320329585},{"date":"20131209",".metric":"rmse",".estimator":"standard",".estimate":0.000848881324378698,"x":1386547200000,"y":0.000848881324378698},{"date":"20131210",".metric":"rmse",".estimator":"standard",".estimate":0.000121244794397763,"x":1386633600000,"y":0.000121244794397763},{"date":"20131211",".metric":"rmse",".estimator":"standard",".estimate":0.00157056881431764,"x":1386720000000,"y":0.00157056881431764},{"date":"20131212",".metric":"rmse",".estimator":"standard",".estimate":0.000762878609040502,"x":1386806400000,"y":0.000762878609040502},{"date":"20131213",".metric":"rmse",".estimator":"standard",".estimate":0.000262529678308959,"x":1386892800000,"y":0.000262529678308959},{"date":"20131216",".metric":"rmse",".estimator":"standard",".estimate":0.000656239523109479,"x":1387152000000,"y":0.000656239523109479},{"date":"20131217",".metric":"rmse",".estimator":"standard",".estimate":0.000472490575181525,"x":1387238400000,"y":0.000472490575181525},{"date":"20131218",".metric":"rmse",".estimator":"standard",".estimate":0.00585553726209109,"x":1387324800000,"y":0.00585553726209109},{"date":"20131219",".metric":"rmse",".estimator":"standard",".estimate":0.00640125999372666,"x":1387411200000,"y":0.00640125999372666},{"date":"20131220",".metric":"rmse",".estimator":"standard",".estimate":0.000492841717857647,"x":1387497600000,"y":0.000492841717857647},{"date":"20131223",".metric":"rmse",".estimator":"standard",".estimate":0.00010882949525809,"x":1387756800000,"y":0.00010882949525809},{"date":"20131224",".metric":"rmse",".estimator":"standard",".estimate":0.000850285446368634,"x":1387843200000,"y":0.000850285446368634},{"date":"20131226",".metric":"rmse",".estimator":"standard",".estimate":0.00229071411562669,"x":1388016000000,"y":0.00229071411562669},{"date":"20131227",".metric":"rmse",".estimator":"standard",".estimate":0.00361979126537698,"x":1388102400000,"y":0.00361979126537698},{"date":"20131230",".metric":"rmse",".estimator":"standard",".estimate":0.00330600930550152,"x":1388361600000,"y":0.00330600930550152},{"date":"20131231",".metric":"rmse",".estimator":"standard",".estimate":0.00151186957064735,"x":1388448000000,"y":0.00151186957064735},{"date":"20140102",".metric":"rmse",".estimator":"standard",".estimate":0.00148733526212147,"x":1388620800000,"y":0.00148733526212147},{"date":"20140103",".metric":"rmse",".estimator":"standard",".estimate":0.000818930753694061,"x":1388707200000,"y":0.000818930753694061},{"date":"20140106",".metric":"rmse",".estimator":"standard",".estimate":0.00125900224553879,"x":1388966400000,"y":0.00125900224553879},{"date":"20140107",".metric":"rmse",".estimator":"standard",".estimate":0.00253206351383783,"x":1389052800000,"y":0.00253206351383783},{"date":"20140108",".metric":"rmse",".estimator":"standard",".estimate":0.00446864942376773,"x":1389139200000,"y":0.00446864942376773},{"date":"20140109",".metric":"rmse",".estimator":"standard",".estimate":0.000571434050674663,"x":1389225600000,"y":0.000571434050674663},{"date":"20140110",".metric":"rmse",".estimator":"standard",".estimate":0.0011876363523322,"x":1389312000000,"y":0.0011876363523322},{"date":"20140113",".metric":"rmse",".estimator":"standard",".estimate":0.00920096136928017,"x":1389571200000,"y":0.00920096136928017},{"date":"20140114",".metric":"rmse",".estimator":"standard",".estimate":0.00415825572747359,"x":1389657600000,"y":0.00415825572747359},{"date":"20140115",".metric":"rmse",".estimator":"standard",".estimate":0.00151620167562148,"x":1389744000000,"y":0.00151620167562148},{"date":"20140116",".metric":"rmse",".estimator":"standard",".estimate":0.000422788504755945,"x":1389830400000,"y":0.000422788504755945},{"date":"20140117",".metric":"rmse",".estimator":"standard",".estimate":0.00169462330330027,"x":1389916800000,"y":0.00169462330330027},{"date":"20140121",".metric":"rmse",".estimator":"standard",".estimate":0.000125594486346043,"x":1390262400000,"y":0.000125594486346043},{"date":"20140122",".metric":"rmse",".estimator":"standard",".estimate":0.00333538903158534,"x":1390348800000,"y":0.00333538903158534},{"date":"20140123",".metric":"rmse",".estimator":"standard",".estimate":0.00280448815225807,"x":1390435200000,"y":0.00280448815225807},{"date":"20140124",".metric":"rmse",".estimator":"standard",".estimate":0.0114439499820774,"x":1390521600000,"y":0.0114439499820774},{"date":"20140127",".metric":"rmse",".estimator":"standard",".estimate":0.000493179949841391,"x":1390780800000,"y":0.000493179949841391},{"date":"20140128",".metric":"rmse",".estimator":"standard",".estimate":0.00126244587670463,"x":1390867200000,"y":0.00126244587670463},{"date":"20140129",".metric":"rmse",".estimator":"standard",".estimate":0.00510795794284275,"x":1390953600000,"y":0.00510795794284275},{"date":"20140130",".metric":"rmse",".estimator":"standard",".estimate":0.00201308629063307,"x":1391040000000,"y":0.00201308629063307},{"date":"20140131",".metric":"rmse",".estimator":"standard",".estimate":0.000231244217711399,"x":1391126400000,"y":0.000231244217711399},{"date":"20140203",".metric":"rmse",".estimator":"standard",".estimate":0.0115634039089308,"x":1391385600000,"y":0.0115634039089308},{"date":"20140204",".metric":"rmse",".estimator":"standard",".estimate":0.00406486240510587,"x":1391472000000,"y":0.00406486240510587},{"date":"20140205",".metric":"rmse",".estimator":"standard",".estimate":0.0047391731187252,"x":1391558400000,"y":0.0047391731187252},{"date":"20140206",".metric":"rmse",".estimator":"standard",".estimate":0.00546190892666902,"x":1391644800000,"y":0.00546190892666902},{"date":"20140207",".metric":"rmse",".estimator":"standard",".estimate":0.00260291637432031,"x":1391731200000,"y":0.00260291637432031},{"date":"20140210",".metric":"rmse",".estimator":"standard",".estimate":0.00129593547982691,"x":1391990400000,"y":0.00129593547982691},{"date":"20140211",".metric":"rmse",".estimator":"standard",".estimate":0.000580525462518246,"x":1392076800000,"y":0.000580525462518246},{"date":"20140212",".metric":"rmse",".estimator":"standard",".estimate":0.00246574780588963,"x":1392163200000,"y":0.00246574780588963},{"date":"20140213",".metric":"rmse",".estimator":"standard",".estimate":0.00105519704792631,"x":1392249600000,"y":0.00105519704792631},{"date":"20140214",".metric":"rmse",".estimator":"standard",".estimate":0.00236393646847481,"x":1392336000000,"y":0.00236393646847481},{"date":"20140218",".metric":"rmse",".estimator":"standard",".estimate":0.00361555311954361,"x":1392681600000,"y":0.00361555311954361},{"date":"20140219",".metric":"rmse",".estimator":"standard",".estimate":0.00202325532716078,"x":1392768000000,"y":0.00202325532716078},{"date":"20140220",".metric":"rmse",".estimator":"standard",".estimate":0.00381074086602401,"x":1392854400000,"y":0.00381074086602401},{"date":"20140221",".metric":"rmse",".estimator":"standard",".estimate":0.00231301823550689,"x":1392940800000,"y":0.00231301823550689},{"date":"20140224",".metric":"rmse",".estimator":"standard",".estimate":3.89640725640683e05,"x":1393200000000,"y":3.89640725640683e05},{"date":"20140225",".metric":"rmse",".estimator":"standard",".estimate":0.000476087731678889,"x":1393286400000,"y":0.000476087731678889},{"date":"20140226",".metric":"rmse",".estimator":"standard",".estimate":0.00243442029091147,"x":1393372800000,"y":0.00243442029091147},{"date":"20140227",".metric":"rmse",".estimator":"standard",".estimate":0.00178820676495497,"x":1393459200000,"y":0.00178820676495497},{"date":"20140228",".metric":"rmse",".estimator":"standard",".estimate":1.08820158253868e05,"x":1393545600000,"y":1.08820158253868e05},{"date":"20140303",".metric":"rmse",".estimator":"standard",".estimate":0.0013276177154797,"x":1393804800000,"y":0.0013276177154797},{"date":"20140304",".metric":"rmse",".estimator":"standard",".estimate":0.00643113277773242,"x":1393891200000,"y":0.00643113277773242},{"date":"20140305",".metric":"rmse",".estimator":"standard",".estimate":0.00016254202236877,"x":1393977600000,"y":0.00016254202236877},{"date":"20140306",".metric":"rmse",".estimator":"standard",".estimate":0.00230278019254107,"x":1394064000000,"y":0.00230278019254107},{"date":"20140307",".metric":"rmse",".estimator":"standard",".estimate":0.00350509104329834,"x":1394150400000,"y":0.00350509104329834},{"date":"20140310",".metric":"rmse",".estimator":"standard",".estimate":0.00237167859360072,"x":1394409600000,"y":0.00237167859360072},{"date":"20140311",".metric":"rmse",".estimator":"standard",".estimate":0.000504901343541518,"x":1394496000000,"y":0.000504901343541518},{"date":"20140312",".metric":"rmse",".estimator":"standard",".estimate":0.00233872525911163,"x":1394582400000,"y":0.00233872525911163},{"date":"20140313",".metric":"rmse",".estimator":"standard",".estimate":0.00128850236022677,"x":1394668800000,"y":0.00128850236022677},{"date":"20140314",".metric":"rmse",".estimator":"standard",".estimate":0.00138860484329246,"x":1394755200000,"y":0.00138860484329246},{"date":"20140317",".metric":"rmse",".estimator":"standard",".estimate":0.000348630426874252,"x":1395014400000,"y":0.000348630426874252},{"date":"20140318",".metric":"rmse",".estimator":"standard",".estimate":0.00171305593536281,"x":1395100800000,"y":0.00171305593536281},{"date":"20140319",".metric":"rmse",".estimator":"standard",".estimate":0.00103422560544747,"x":1395187200000,"y":0.00103422560544747},{"date":"20140320",".metric":"rmse",".estimator":"standard",".estimate":0.00244948500910164,"x":1395273600000,"y":0.00244948500910164},{"date":"20140321",".metric":"rmse",".estimator":"standard",".estimate":0.00339788499753144,"x":1395360000000,"y":0.00339788499753144},{"date":"20140324",".metric":"rmse",".estimator":"standard",".estimate":0.00130675471112893,"x":1395619200000,"y":0.00130675471112893},{"date":"20140325",".metric":"rmse",".estimator":"standard",".estimate":0.00213675190692013,"x":1395705600000,"y":0.00213675190692013},{"date":"20140326",".metric":"rmse",".estimator":"standard",".estimate":0.00466893619476399,"x":1395792000000,"y":0.00466893619476399},{"date":"20140327",".metric":"rmse",".estimator":"standard",".estimate":0.00165591295288144,"x":1395878400000,"y":0.00165591295288144},{"date":"20140328",".metric":"rmse",".estimator":"standard",".estimate":0.0022503861609131,"x":1395964800000,"y":0.0022503861609131},{"date":"20140331",".metric":"rmse",".estimator":"standard",".estimate":0.00144421796473591,"x":1396224000000,"y":0.00144421796473591},{"date":"20140401",".metric":"rmse",".estimator":"standard",".estimate":0.000293571879672249,"x":1396310400000,"y":0.000293571879672249},{"date":"20140402",".metric":"rmse",".estimator":"standard",".estimate":0.000921277632247967,"x":1396396800000,"y":0.000921277632247967},{"date":"20140403",".metric":"rmse",".estimator":"standard",".estimate":0.0010656539825164,"x":1396483200000,"y":0.0010656539825164},{"date":"20140404",".metric":"rmse",".estimator":"standard",".estimate":0.00618316301989382,"x":1396569600000,"y":0.00618316301989382},{"date":"20140407",".metric":"rmse",".estimator":"standard",".estimate":0.00410145020592395,"x":1396828800000,"y":0.00410145020592395},{"date":"20140408",".metric":"rmse",".estimator":"standard",".estimate":0.000744359375515803,"x":1396915200000,"y":0.000744359375515803},{"date":"20140409",".metric":"rmse",".estimator":"standard",".estimate":0.00371180214554768,"x":1397001600000,"y":0.00371180214554768},{"date":"20140410",".metric":"rmse",".estimator":"standard",".estimate":0.00955218910528874,"x":1397088000000,"y":0.00955218910528874},{"date":"20140411",".metric":"rmse",".estimator":"standard",".estimate":0.000757652870459674,"x":1397174400000,"y":0.000757652870459674},{"date":"20140414",".metric":"rmse",".estimator":"standard",".estimate":0.00231523494562687,"x":1397433600000,"y":0.00231523494562687},{"date":"20140415",".metric":"rmse",".estimator":"standard",".estimate":0.00502091291754025,"x":1397520000000,"y":0.00502091291754025},{"date":"20140416",".metric":"rmse",".estimator":"standard",".estimate":0.00027389125520713,"x":1397606400000,"y":0.00027389125520713},{"date":"20140417",".metric":"rmse",".estimator":"standard",".estimate":0.00043409550268547,"x":1397692800000,"y":0.00043409550268547},{"date":"20140421",".metric":"rmse",".estimator":"standard",".estimate":0.000808809667989553,"x":1398038400000,"y":0.000808809667989553},{"date":"20140422",".metric":"rmse",".estimator":"standard",".estimate":0.0013365075844831,"x":1398124800000,"y":0.0013365075844831},{"date":"20140423",".metric":"rmse",".estimator":"standard",".estimate":0.000424163460882393,"x":1398211200000,"y":0.000424163460882393},{"date":"20140424",".metric":"rmse",".estimator":"standard",".estimate":0.00150007335746622,"x":1398297600000,"y":0.00150007335746622},{"date":"20140425",".metric":"rmse",".estimator":"standard",".estimate":0.00156559003191409,"x":1398384000000,"y":0.00156559003191409},{"date":"20140428",".metric":"rmse",".estimator":"standard",".estimate":0.0028638148816032,"x":1398643200000,"y":0.0028638148816032},{"date":"20140429",".metric":"rmse",".estimator":"standard",".estimate":0.00170800478467133,"x":1398729600000,"y":0.00170800478467133},{"date":"20140430",".metric":"rmse",".estimator":"standard",".estimate":0.00105875020917615,"x":1398816000000,"y":0.00105875020917615},{"date":"20140501",".metric":"rmse",".estimator":"standard",".estimate":0.00389061926590377,"x":1398902400000,"y":0.00389061926590377},{"date":"20140502",".metric":"rmse",".estimator":"standard",".estimate":0.000875771905077209,"x":1398988800000,"y":0.000875771905077209},{"date":"20140505",".metric":"rmse",".estimator":"standard",".estimate":0.000302082593616106,"x":1399248000000,"y":0.000302082593616106},{"date":"20140506",".metric":"rmse",".estimator":"standard",".estimate":0.00377581232614828,"x":1399334400000,"y":0.00377581232614828},{"date":"20140507",".metric":"rmse",".estimator":"standard",".estimate":0.00432694148901447,"x":1399420800000,"y":0.00432694148901447},{"date":"20140508",".metric":"rmse",".estimator":"standard",".estimate":0.00541476252741813,"x":1399507200000,"y":0.00541476252741813},{"date":"20140509",".metric":"rmse",".estimator":"standard",".estimate":0.00130666361160008,"x":1399593600000,"y":0.00130666361160008},{"date":"20140512",".metric":"rmse",".estimator":"standard",".estimate":0.00106690615564849,"x":1399852800000,"y":0.00106690615564849},{"date":"20140513",".metric":"rmse",".estimator":"standard",".estimate":0.00352556126676091,"x":1399939200000,"y":0.00352556126676091},{"date":"20140514",".metric":"rmse",".estimator":"standard",".estimate":0.00139230203117092,"x":1400025600000,"y":0.00139230203117092},{"date":"20140515",".metric":"rmse",".estimator":"standard",".estimate":0.00263900248643558,"x":1400112000000,"y":0.00263900248643558},{"date":"20140516",".metric":"rmse",".estimator":"standard",".estimate":0.00282314343433784,"x":1400198400000,"y":0.00282314343433784},{"date":"20140519",".metric":"rmse",".estimator":"standard",".estimate":0.000290963538759818,"x":1400457600000,"y":0.000290963538759818},{"date":"20140520",".metric":"rmse",".estimator":"standard",".estimate":0.00121348207373204,"x":1400544000000,"y":0.00121348207373204},{"date":"20140521",".metric":"rmse",".estimator":"standard",".estimate":0.00356217827640926,"x":1400630400000,"y":0.00356217827640926},{"date":"20140522",".metric":"rmse",".estimator":"standard",".estimate":0.000775913446580488,"x":1400716800000,"y":0.000775913446580488},{"date":"20140523",".metric":"rmse",".estimator":"standard",".estimate":0.00240507902895481,"x":1400803200000,"y":0.00240507902895481},{"date":"20140527",".metric":"rmse",".estimator":"standard",".estimate":0.00359690871245207,"x":1401148800000,"y":0.00359690871245207},{"date":"20140528",".metric":"rmse",".estimator":"standard",".estimate":0.000446458811152914,"x":1401235200000,"y":0.000446458811152914},{"date":"20140529",".metric":"rmse",".estimator":"standard",".estimate":0.00134881889143732,"x":1401321600000,"y":0.00134881889143732},{"date":"20140530",".metric":"rmse",".estimator":"standard",".estimate":4.37586477533163e05,"x":1401408000000,"y":4.37586477533163e05},{"date":"20140602",".metric":"rmse",".estimator":"standard",".estimate":6.62895970695884e05,"x":1401667200000,"y":6.62895970695884e05},{"date":"20140603",".metric":"rmse",".estimator":"standard",".estimate":0.00124667880122449,"x":1401753600000,"y":0.00124667880122449},{"date":"20140604",".metric":"rmse",".estimator":"standard",".estimate":0.000486938615441381,"x":1401840000000,"y":0.000486938615441381},{"date":"20140605",".metric":"rmse",".estimator":"standard",".estimate":0.000855713483865092,"x":1401926400000,"y":0.000855713483865092},{"date":"20140606",".metric":"rmse",".estimator":"standard",".estimate":0.00196057510424582,"x":1402012800000,"y":0.00196057510424582},{"date":"20140609",".metric":"rmse",".estimator":"standard",".estimate":0.000917943445872296,"x":1402272000000,"y":0.000917943445872296},{"date":"20140610",".metric":"rmse",".estimator":"standard",".estimate":0.00176967951524932,"x":1402358400000,"y":0.00176967951524932},{"date":"20140611",".metric":"rmse",".estimator":"standard",".estimate":0.00282423404314993,"x":1402444800000,"y":0.00282423404314993},{"date":"20140612",".metric":"rmse",".estimator":"standard",".estimate":0.00483068423246947,"x":1402531200000,"y":0.00483068423246947},{"date":"20140613",".metric":"rmse",".estimator":"standard",".estimate":0.0019474720610407,"x":1402617600000,"y":0.0019474720610407},{"date":"20140616",".metric":"rmse",".estimator":"standard",".estimate":0.000259047311752497,"x":1402876800000,"y":0.000259047311752497},{"date":"20140617",".metric":"rmse",".estimator":"standard",".estimate":0.000874624755127248,"x":1402963200000,"y":0.000874624755127248},{"date":"20140618",".metric":"rmse",".estimator":"standard",".estimate":0.00162835712315407,"x":1403049600000,"y":0.00162835712315407},{"date":"20140619",".metric":"rmse",".estimator":"standard",".estimate":0.00283043809754556,"x":1403136000000,"y":0.00283043809754556},{"date":"20140620",".metric":"rmse",".estimator":"standard",".estimate":0.000730178279525006,"x":1403222400000,"y":0.000730178279525006},{"date":"20140623",".metric":"rmse",".estimator":"standard",".estimate":0.000137424624938349,"x":1403481600000,"y":0.000137424624938349},{"date":"20140624",".metric":"rmse",".estimator":"standard",".estimate":0.000547485209905004,"x":1403568000000,"y":0.000547485209905004},{"date":"20140625",".metric":"rmse",".estimator":"standard",".estimate":0.00327809240197013,"x":1403654400000,"y":0.00327809240197013},{"date":"20140626",".metric":"rmse",".estimator":"standard",".estimate":0.00152343341502415,"x":1403740800000,"y":0.00152343341502415},{"date":"20140627",".metric":"rmse",".estimator":"standard",".estimate":0.00135515261171915,"x":1403827200000,"y":0.00135515261171915},{"date":"20140630",".metric":"rmse",".estimator":"standard",".estimate":0.00160195578294905,"x":1404086400000,"y":0.00160195578294905},{"date":"20140701",".metric":"rmse",".estimator":"standard",".estimate":0.000233602083369974,"x":1404172800000,"y":0.000233602083369974},{"date":"20140702",".metric":"rmse",".estimator":"standard",".estimate":0.00174962461912607,"x":1404259200000,"y":0.00174962461912607},{"date":"20140703",".metric":"rmse",".estimator":"standard",".estimate":0.000319944598172364,"x":1404345600000,"y":0.000319944598172364},{"date":"20140707",".metric":"rmse",".estimator":"standard",".estimate":0.00365706027220958,"x":1404691200000,"y":0.00365706027220958},{"date":"20140708",".metric":"rmse",".estimator":"standard",".estimate":0.0061451999312466,"x":1404777600000,"y":0.0061451999312466},{"date":"20140709",".metric":"rmse",".estimator":"standard",".estimate":0.0008333097331233,"x":1404864000000,"y":0.0008333097331233},{"date":"20140710",".metric":"rmse",".estimator":"standard",".estimate":0.000942594891875229,"x":1404950400000,"y":0.000942594891875229},{"date":"20140711",".metric":"rmse",".estimator":"standard",".estimate":0.000882323784889272,"x":1405036800000,"y":0.000882323784889272},{"date":"20140714",".metric":"rmse",".estimator":"standard",".estimate":0.00115369579766836,"x":1405296000000,"y":0.00115369579766836},{"date":"20140715",".metric":"rmse",".estimator":"standard",".estimate":0.00258036734617897,"x":1405382400000,"y":0.00258036734617897},{"date":"20140716",".metric":"rmse",".estimator":"standard",".estimate":0.000210557689839008,"x":1405468800000,"y":0.000210557689839008},{"date":"20140717",".metric":"rmse",".estimator":"standard",".estimate":0.00307865459624111,"x":1405555200000,"y":0.00307865459624111},{"date":"20140718",".metric":"rmse",".estimator":"standard",".estimate":0.00600957581655225,"x":1405641600000,"y":0.00600957581655225},{"date":"20140721",".metric":"rmse",".estimator":"standard",".estimate":0.000141034509552021,"x":1405900800000,"y":0.000141034509552021},{"date":"20140722",".metric":"rmse",".estimator":"standard",".estimate":0.00157316429465122,"x":1405987200000,"y":0.00157316429465122},{"date":"20140723",".metric":"rmse",".estimator":"standard",".estimate":5.79389393633141e05,"x":1406073600000,"y":5.79389393633141e05},{"date":"20140724",".metric":"rmse",".estimator":"standard",".estimate":0.00298640090271562,"x":1406160000000,"y":0.00298640090271562},{"date":"20140725",".metric":"rmse",".estimator":"standard",".estimate":0.000846890598854637,"x":1406246400000,"y":0.000846890598854637},{"date":"20140728",".metric":"rmse",".estimator":"standard",".estimate":0.000486268231837349,"x":1406505600000,"y":0.000486268231837349},{"date":"20140729",".metric":"rmse",".estimator":"standard",".estimate":0.00108729136628571,"x":1406592000000,"y":0.00108729136628571},{"date":"20140730",".metric":"rmse",".estimator":"standard",".estimate":0.00240110003602132,"x":1406678400000,"y":0.00240110003602132},{"date":"20140731",".metric":"rmse",".estimator":"standard",".estimate":0.00886385990974978,"x":1406764800000,"y":0.00886385990974978},{"date":"20140801",".metric":"rmse",".estimator":"standard",".estimate":0.00130950662628627,"x":1406851200000,"y":0.00130950662628627},{"date":"20140804",".metric":"rmse",".estimator":"standard",".estimate":0.00414144612909378,"x":1407110400000,"y":0.00414144612909378},{"date":"20140805",".metric":"rmse",".estimator":"standard",".estimate":0.00371691255412742,"x":1407196800000,"y":0.00371691255412742},{"date":"20140806",".metric":"rmse",".estimator":"standard",".estimate":0.00349694714381284,"x":1407283200000,"y":0.00349694714381284},{"date":"20140807",".metric":"rmse",".estimator":"standard",".estimate":0.00103639681438946,"x":1407369600000,"y":0.00103639681438946},{"date":"20140808",".metric":"rmse",".estimator":"standard",".estimate":0.00819731088788718,"x":1407456000000,"y":0.00819731088788718},{"date":"20140811",".metric":"rmse",".estimator":"standard",".estimate":0.00320991418289113,"x":1407715200000,"y":0.00320991418289113},{"date":"20140812",".metric":"rmse",".estimator":"standard",".estimate":0.00183064761761298,"x":1407801600000,"y":0.00183064761761298},{"date":"20140813",".metric":"rmse",".estimator":"standard",".estimate":0.000796963386906422,"x":1407888000000,"y":0.000796963386906422},{"date":"20140814",".metric":"rmse",".estimator":"standard",".estimate":0.000597107731675361,"x":1407974400000,"y":0.000597107731675361},{"date":"20140815",".metric":"rmse",".estimator":"standard",".estimate":0.000147625995064866,"x":1408060800000,"y":0.000147625995064866},{"date":"20140818",".metric":"rmse",".estimator":"standard",".estimate":0.00206935067639825,"x":1408320000000,"y":0.00206935067639825},{"date":"20140819",".metric":"rmse",".estimator":"standard",".estimate":0.000960516267774979,"x":1408406400000,"y":0.000960516267774979},{"date":"20140820",".metric":"rmse",".estimator":"standard",".estimate":0.00164247712046299,"x":1408492800000,"y":0.00164247712046299},{"date":"20140821",".metric":"rmse",".estimator":"standard",".estimate":0.00137361399566395,"x":1408579200000,"y":0.00137361399566395},{"date":"20140822",".metric":"rmse",".estimator":"standard",".estimate":0.00203472779931931,"x":1408665600000,"y":0.00203472779931931},{"date":"20140825",".metric":"rmse",".estimator":"standard",".estimate":0.00119471810723019,"x":1408924800000,"y":0.00119471810723019},{"date":"20140826",".metric":"rmse",".estimator":"standard",".estimate":0.00177017943673564,"x":1409011200000,"y":0.00177017943673564},{"date":"20140827",".metric":"rmse",".estimator":"standard",".estimate":0.00167361871840368,"x":1409097600000,"y":0.00167361871840368},{"date":"20140828",".metric":"rmse",".estimator":"standard",".estimate":0.00415390822110768,"x":1409184000000,"y":0.00415390822110768},{"date":"20140829",".metric":"rmse",".estimator":"standard",".estimate":0.000553509892759291,"x":1409270400000,"y":0.000553509892759291},{"date":"20140902",".metric":"rmse",".estimator":"standard",".estimate":0.0028580303501736,"x":1409616000000,"y":0.0028580303501736},{"date":"20140903",".metric":"rmse",".estimator":"standard",".estimate":0.00441100910006302,"x":1409702400000,"y":0.00441100910006302},{"date":"20140904",".metric":"rmse",".estimator":"standard",".estimate":0.000231429680361579,"x":1409788800000,"y":0.000231429680361579},{"date":"20140905",".metric":"rmse",".estimator":"standard",".estimate":0.00276346914628592,"x":1409875200000,"y":0.00276346914628592},{"date":"20140908",".metric":"rmse",".estimator":"standard",".estimate":0.00034802932604793,"x":1410134400000,"y":0.00034802932604793},{"date":"20140909",".metric":"rmse",".estimator":"standard",".estimate":0.00233804117778962,"x":1410220800000,"y":0.00233804117778962},{"date":"20140910",".metric":"rmse",".estimator":"standard",".estimate":0.00178648151267642,"x":1410307200000,"y":0.00178648151267642},{"date":"20140911",".metric":"rmse",".estimator":"standard",".estimate":0.000721554836218572,"x":1410393600000,"y":0.000721554836218572},{"date":"20140912",".metric":"rmse",".estimator":"standard",".estimate":0.00455760000316874,"x":1410480000000,"y":0.00455760000316874},{"date":"20140915",".metric":"rmse",".estimator":"standard",".estimate":0.00350688868934713,"x":1410739200000,"y":0.00350688868934713},{"date":"20140916",".metric":"rmse",".estimator":"standard",".estimate":0.00422725297543705,"x":1410825600000,"y":0.00422725297543705},{"date":"20140917",".metric":"rmse",".estimator":"standard",".estimate":0.00281860847561002,"x":1410912000000,"y":0.00281860847561002},{"date":"20140918",".metric":"rmse",".estimator":"standard",".estimate":0.00147408592681244,"x":1410998400000,"y":0.00147408592681244},{"date":"20140919",".metric":"rmse",".estimator":"standard",".estimate":0.00215655991310362,"x":1411084800000,"y":0.00215655991310362},{"date":"20140922",".metric":"rmse",".estimator":"standard",".estimate":0.00218257063330211,"x":1411344000000,"y":0.00218257063330211},{"date":"20140923",".metric":"rmse",".estimator":"standard",".estimate":0.00060948260244335,"x":1411430400000,"y":0.00060948260244335},{"date":"20140924",".metric":"rmse",".estimator":"standard",".estimate":0.00167606922803859,"x":1411516800000,"y":0.00167606922803859},{"date":"20140925",".metric":"rmse",".estimator":"standard",".estimate":0.00536744976052472,"x":1411603200000,"y":0.00536744976052472},{"date":"20140926",".metric":"rmse",".estimator":"standard",".estimate":0.000564206344653154,"x":1411689600000,"y":0.000564206344653154},{"date":"20140929",".metric":"rmse",".estimator":"standard",".estimate":0.00229672644043357,"x":1411948800000,"y":0.00229672644043357},{"date":"20140930",".metric":"rmse",".estimator":"standard",".estimate":0.00120139847585075,"x":1412035200000,"y":0.00120139847585075},{"date":"20141001",".metric":"rmse",".estimator":"standard",".estimate":0.00214985386840889,"x":1412121600000,"y":0.00214985386840889},{"date":"20141002",".metric":"rmse",".estimator":"standard",".estimate":0.0070625174291384,"x":1412208000000,"y":0.0070625174291384},{"date":"20141003",".metric":"rmse",".estimator":"standard",".estimate":0.0039993124787582,"x":1412294400000,"y":0.0039993124787582},{"date":"20141006",".metric":"rmse",".estimator":"standard",".estimate":0.00406570964175735,"x":1412553600000,"y":0.00406570964175735},{"date":"20141007",".metric":"rmse",".estimator":"standard",".estimate":0.00608770260280821,"x":1412640000000,"y":0.00608770260280821},{"date":"20141008",".metric":"rmse",".estimator":"standard",".estimate":0.0100893179621323,"x":1412726400000,"y":0.0100893179621323},{"date":"20141009",".metric":"rmse",".estimator":"standard",".estimate":0.0093044512515634,"x":1412812800000,"y":0.0093044512515634},{"date":"20141010",".metric":"rmse",".estimator":"standard",".estimate":0.0012772359736668,"x":1412899200000,"y":0.0012772359736668},{"date":"20141013",".metric":"rmse",".estimator":"standard",".estimate":0.0104442600151737,"x":1413158400000,"y":0.0104442600151737},{"date":"20141014",".metric":"rmse",".estimator":"standard",".estimate":0.00466686830959002,"x":1413244800000,"y":0.00466686830959002},{"date":"20141015",".metric":"rmse",".estimator":"standard",".estimate":0.00125758742696745,"x":1413331200000,"y":0.00125758742696745},{"date":"20141016",".metric":"rmse",".estimator":"standard",".estimate":0.000746949848383797,"x":1413417600000,"y":0.000746949848383797},{"date":"20141017",".metric":"rmse",".estimator":"standard",".estimate":0.000446330061162016,"x":1413504000000,"y":0.000446330061162016},{"date":"20141020",".metric":"rmse",".estimator":"standard",".estimate":0.00724374124158148,"x":1413763200000,"y":0.00724374124158148},{"date":"20141021",".metric":"rmse",".estimator":"standard",".estimate":0.00945699685916905,"x":1413849600000,"y":0.00945699685916905},{"date":"20141022",".metric":"rmse",".estimator":"standard",".estimate":0.00226758765277367,"x":1413936000000,"y":0.00226758765277367},{"date":"20141023",".metric":"rmse",".estimator":"standard",".estimate":0.00162200882440421,"x":1414022400000,"y":0.00162200882440421},{"date":"20141024",".metric":"rmse",".estimator":"standard",".estimate":0.00245971519926177,"x":1414108800000,"y":0.00245971519926177},{"date":"20141027",".metric":"rmse",".estimator":"standard",".estimate":0.000111437078117051,"x":1414368000000,"y":0.000111437078117051},{"date":"20141028",".metric":"rmse",".estimator":"standard",".estimate":0.00464060315517396,"x":1414454400000,"y":0.00464060315517396},{"date":"20141029",".metric":"rmse",".estimator":"standard",".estimate":0.000994038213760407,"x":1414540800000,"y":0.000994038213760407},{"date":"20141030",".metric":"rmse",".estimator":"standard",".estimate":8.24153890278114e05,"x":1414627200000,"y":8.24153890278114e05},{"date":"20141031",".metric":"rmse",".estimator":"standard",".estimate":0.00136078990878973,"x":1414713600000,"y":0.00136078990878973},{"date":"20141103",".metric":"rmse",".estimator":"standard",".estimate":0.00476872564334884,"x":1414972800000,"y":0.00476872564334884},{"date":"20141104",".metric":"rmse",".estimator":"standard",".estimate":0.00194937552489031,"x":1415059200000,"y":0.00194937552489031},{"date":"20141105",".metric":"rmse",".estimator":"standard",".estimate":0.000904004054702433,"x":1415145600000,"y":0.000904004054702433},{"date":"20141106",".metric":"rmse",".estimator":"standard",".estimate":0.00408408385962174,"x":1415232000000,"y":0.00408408385962174},{"date":"20141107",".metric":"rmse",".estimator":"standard",".estimate":0.000264872917642302,"x":1415318400000,"y":0.000264872917642302},{"date":"20141110",".metric":"rmse",".estimator":"standard",".estimate":0.00185908943461171,"x":1415577600000,"y":0.00185908943461171},{"date":"20141111",".metric":"rmse",".estimator":"standard",".estimate":0.00501504118188602,"x":1415664000000,"y":0.00501504118188602},{"date":"20141112",".metric":"rmse",".estimator":"standard",".estimate":0.00265893779930789,"x":1415750400000,"y":0.00265893779930789},{"date":"20141113",".metric":"rmse",".estimator":"standard",".estimate":0.00216546413051731,"x":1415836800000,"y":0.00216546413051731},{"date":"20141114",".metric":"rmse",".estimator":"standard",".estimate":0.00246891237286566,"x":1415923200000,"y":0.00246891237286566},{"date":"20141117",".metric":"rmse",".estimator":"standard",".estimate":9.41372941566312e05,"x":1416182400000,"y":9.41372941566312e05},{"date":"20141118",".metric":"rmse",".estimator":"standard",".estimate":0.00254598014396055,"x":1416268800000,"y":0.00254598014396055},{"date":"20141119",".metric":"rmse",".estimator":"standard",".estimate":2.7174938674092e05,"x":1416355200000,"y":2.7174938674092e05},{"date":"20141120",".metric":"rmse",".estimator":"standard",".estimate":0.00229655959266466,"x":1416441600000,"y":0.00229655959266466},{"date":"20141121",".metric":"rmse",".estimator":"standard",".estimate":0.00189549887810964,"x":1416528000000,"y":0.00189549887810964},{"date":"20141124",".metric":"rmse",".estimator":"standard",".estimate":0.000260583677516683,"x":1416787200000,"y":0.000260583677516683},{"date":"20141125",".metric":"rmse",".estimator":"standard",".estimate":0.00203921965649436,"x":1416873600000,"y":0.00203921965649436},{"date":"20141126",".metric":"rmse",".estimator":"standard",".estimate":3.96736941195731e06,"x":1416960000000,"y":3.96736941195731e06},{"date":"20141128",".metric":"rmse",".estimator":"standard",".estimate":0.000991448203063747,"x":1417132800000,"y":0.000991448203063747},{"date":"20141201",".metric":"rmse",".estimator":"standard",".estimate":0.00315909482979975,"x":1417392000000,"y":0.00315909482979975},{"date":"20141202",".metric":"rmse",".estimator":"standard",".estimate":0.00360565837034548,"x":1417478400000,"y":0.00360565837034548},{"date":"20141203",".metric":"rmse",".estimator":"standard",".estimate":0.000671187015248425,"x":1417564800000,"y":0.000671187015248425},{"date":"20141204",".metric":"rmse",".estimator":"standard",".estimate":0.000529501904159875,"x":1417651200000,"y":0.000529501904159875},{"date":"20141205",".metric":"rmse",".estimator":"standard",".estimate":0.00175912866840544,"x":1417737600000,"y":0.00175912866840544},{"date":"20141208",".metric":"rmse",".estimator":"standard",".estimate":0.000873646129684066,"x":1417996800000,"y":0.000873646129684066},{"date":"20141209",".metric":"rmse",".estimator":"standard",".estimate":0.0023342024229448,"x":1418083200000,"y":0.0023342024229448},{"date":"20141210",".metric":"rmse",".estimator":"standard",".estimate":0.00317063922796318,"x":1418169600000,"y":0.00317063922796318},{"date":"20141211",".metric":"rmse",".estimator":"standard",".estimate":0.00423979642772988,"x":1418256000000,"y":0.00423979642772988},{"date":"20141212",".metric":"rmse",".estimator":"standard",".estimate":0.0040882751945936,"x":1418342400000,"y":0.0040882751945936},{"date":"20141215",".metric":"rmse",".estimator":"standard",".estimate":0.00777943296248288,"x":1418601600000,"y":0.00777943296248288},{"date":"20141216",".metric":"rmse",".estimator":"standard",".estimate":0.00768636089900666,"x":1418688000000,"y":0.00768636089900666},{"date":"20141217",".metric":"rmse",".estimator":"standard",".estimate":0.0124775738986566,"x":1418774400000,"y":0.0124775738986566},{"date":"20141218",".metric":"rmse",".estimator":"standard",".estimate":0.0107834377209642,"x":1418860800000,"y":0.0107834377209642},{"date":"20141219",".metric":"rmse",".estimator":"standard",".estimate":0.00153144229868774,"x":1418947200000,"y":0.00153144229868774},{"date":"20141222",".metric":"rmse",".estimator":"standard",".estimate":0.000261767913493968,"x":1419206400000,"y":0.000261767913493968},{"date":"20141223",".metric":"rmse",".estimator":"standard",".estimate":0.00281002510430981,"x":1419292800000,"y":0.00281002510430981},{"date":"20141224",".metric":"rmse",".estimator":"standard",".estimate":0.000985711761268521,"x":1419379200000,"y":0.000985711761268521},{"date":"20141226",".metric":"rmse",".estimator":"standard",".estimate":0.00225208514274718,"x":1419552000000,"y":0.00225208514274718},{"date":"20141229",".metric":"rmse",".estimator":"standard",".estimate":0.00115263765734909,"x":1419811200000,"y":0.00115263765734909},{"date":"20141230",".metric":"rmse",".estimator":"standard",".estimate":0.00179371322443149,"x":1419897600000,"y":0.00179371322443149},{"date":"20141231",".metric":"rmse",".estimator":"standard",".estimate":0.00668050594834836,"x":1419984000000,"y":0.00668050594834836},{"date":"20150102",".metric":"rmse",".estimator":"standard",".estimate":0.00395677051677915,"x":1420156800000,"y":0.00395677051677915},{"date":"20150105",".metric":"rmse",".estimator":"standard",".estimate":0.00362346472773006,"x":1420416000000,"y":0.00362346472773006},{"date":"20150106",".metric":"rmse",".estimator":"standard",".estimate":5.35915047082565e05,"x":1420502400000,"y":5.35915047082565e05},{"date":"20150107",".metric":"rmse",".estimator":"standard",".estimate":0.00447041821256274,"x":1420588800000,"y":0.00447041821256274},{"date":"20150108",".metric":"rmse",".estimator":"standard",".estimate":0.000670507797887281,"x":1420675200000,"y":0.000670507797887281},{"date":"20150109",".metric":"rmse",".estimator":"standard",".estimate":0.00232322268857359,"x":1420761600000,"y":0.00232322268857359},{"date":"20150112",".metric":"rmse",".estimator":"standard",".estimate":0.0055324501544945,"x":1421020800000,"y":0.0055324501544945},{"date":"20150113",".metric":"rmse",".estimator":"standard",".estimate":0.00411443172418601,"x":1421107200000,"y":0.00411443172418601},{"date":"20150114",".metric":"rmse",".estimator":"standard",".estimate":0.00182992367814795,"x":1421193600000,"y":0.00182992367814795},{"date":"20150115",".metric":"rmse",".estimator":"standard",".estimate":0.00684786734873696,"x":1421280000000,"y":0.00684786734873696},{"date":"20150116",".metric":"rmse",".estimator":"standard",".estimate":0.00517728592452819,"x":1421366400000,"y":0.00517728592452819},{"date":"20150120",".metric":"rmse",".estimator":"standard",".estimate":0.00322047343366349,"x":1421712000000,"y":0.00322047343366349},{"date":"20150121",".metric":"rmse",".estimator":"standard",".estimate":0.00474144338964312,"x":1421798400000,"y":0.00474144338964312},{"date":"20150122",".metric":"rmse",".estimator":"standard",".estimate":0.00536853999227699,"x":1421884800000,"y":0.00536853999227699},{"date":"20150123",".metric":"rmse",".estimator":"standard",".estimate":0.00431737006355696,"x":1421971200000,"y":0.00431737006355696},{"date":"20150126",".metric":"rmse",".estimator":"standard",".estimate":0.00127446112543595,"x":1422230400000,"y":0.00127446112543595},{"date":"20150127",".metric":"rmse",".estimator":"standard",".estimate":0.0069018612083931,"x":1422316800000,"y":0.0069018612083931},{"date":"20150128",".metric":"rmse",".estimator":"standard",".estimate":0.00249742664827329,"x":1422403200000,"y":0.00249742664827329},{"date":"20150129",".metric":"rmse",".estimator":"standard",".estimate":0.00233421320349214,"x":1422489600000,"y":0.00233421320349214},{"date":"20150130",".metric":"rmse",".estimator":"standard",".estimate":0.00480908772662906,"x":1422576000000,"y":0.00480908772662906},{"date":"20150202",".metric":"rmse",".estimator":"standard",".estimate":0.00242542796840435,"x":1422835200000,"y":0.00242542796840435},{"date":"20150203",".metric":"rmse",".estimator":"standard",".estimate":0.00261338329411186,"x":1422921600000,"y":0.00261338329411186},{"date":"20150204",".metric":"rmse",".estimator":"standard",".estimate":0.00198146309507166,"x":1423008000000,"y":0.00198146309507166},{"date":"20150205",".metric":"rmse",".estimator":"standard",".estimate":0.00013164875324184,"x":1423094400000,"y":0.00013164875324184},{"date":"20150206",".metric":"rmse",".estimator":"standard",".estimate":0.000544066084412995,"x":1423180800000,"y":0.000544066084412995},{"date":"20150209",".metric":"rmse",".estimator":"standard",".estimate":0.000947089446149543,"x":1423440000000,"y":0.000947089446149543},{"date":"20150210",".metric":"rmse",".estimator":"standard",".estimate":0.000175757503755772,"x":1423526400000,"y":0.000175757503755772},{"date":"20150211",".metric":"rmse",".estimator":"standard",".estimate":0.000338261761013436,"x":1423612800000,"y":0.000338261761013436},{"date":"20150212",".metric":"rmse",".estimator":"standard",".estimate":0.00100328702089229,"x":1423699200000,"y":0.00100328702089229},{"date":"20150213",".metric":"rmse",".estimator":"standard",".estimate":0.00179271505344779,"x":1423785600000,"y":0.00179271505344779},{"date":"20150217",".metric":"rmse",".estimator":"standard",".estimate":0.00230582211235819,"x":1424131200000,"y":0.00230582211235819},{"date":"20150218",".metric":"rmse",".estimator":"standard",".estimate":0.00327919423348622,"x":1424217600000,"y":0.00327919423348622},{"date":"20150219",".metric":"rmse",".estimator":"standard",".estimate":0.00242318352913266,"x":1424304000000,"y":0.00242318352913266},{"date":"20150220",".metric":"rmse",".estimator":"standard",".estimate":0.000964435546642736,"x":1424390400000,"y":0.000964435546642736},{"date":"20150223",".metric":"rmse",".estimator":"standard",".estimate":0.00117093313511718,"x":1424649600000,"y":0.00117093313511718},{"date":"20150224",".metric":"rmse",".estimator":"standard",".estimate":0.000347530115280883,"x":1424736000000,"y":0.000347530115280883},{"date":"20150225",".metric":"rmse",".estimator":"standard",".estimate":0.00156198286358544,"x":1424822400000,"y":0.00156198286358544},{"date":"20150226",".metric":"rmse",".estimator":"standard",".estimate":0.000101984604233385,"x":1424908800000,"y":0.000101984604233385},{"date":"20150227",".metric":"rmse",".estimator":"standard",".estimate":0.00182582223293412,"x":1424995200000,"y":0.00182582223293412},{"date":"20150302",".metric":"rmse",".estimator":"standard",".estimate":0.00288086391998691,"x":1425254400000,"y":0.00288086391998691},{"date":"20150303",".metric":"rmse",".estimator":"standard",".estimate":0.00169682218549016,"x":1425340800000,"y":0.00169682218549016},{"date":"20150304",".metric":"rmse",".estimator":"standard",".estimate":0.00299797546408927,"x":1425427200000,"y":0.00299797546408927},{"date":"20150305",".metric":"rmse",".estimator":"standard",".estimate":0.000705709812426563,"x":1425513600000,"y":0.000705709812426563},{"date":"20150306",".metric":"rmse",".estimator":"standard",".estimate":0.00674386588619811,"x":1425600000000,"y":0.00674386588619811},{"date":"20150309",".metric":"rmse",".estimator":"standard",".estimate":0.00442197188878407,"x":1425859200000,"y":0.00442197188878407},{"date":"20150310",".metric":"rmse",".estimator":"standard",".estimate":0.00315569341433053,"x":1425945600000,"y":0.00315569341433053},{"date":"20150311",".metric":"rmse",".estimator":"standard",".estimate":0.000474564201651918,"x":1426032000000,"y":0.000474564201651918},{"date":"20150312",".metric":"rmse",".estimator":"standard",".estimate":0.0031755201986522,"x":1426118400000,"y":0.0031755201986522},{"date":"20150313",".metric":"rmse",".estimator":"standard",".estimate":0.000442805972450956,"x":1426204800000,"y":0.000442805972450956},{"date":"20150316",".metric":"rmse",".estimator":"standard",".estimate":0.00145573918699198,"x":1426464000000,"y":0.00145573918699198},{"date":"20150317",".metric":"rmse",".estimator":"standard",".estimate":0.0013937081708804,"x":1426550400000,"y":0.0013937081708804},{"date":"20150318",".metric":"rmse",".estimator":"standard",".estimate":0.000270695042425027,"x":1426636800000,"y":0.000270695042425027},{"date":"20150319",".metric":"rmse",".estimator":"standard",".estimate":0.00270669488153355,"x":1426723200000,"y":0.00270669488153355},{"date":"20150320",".metric":"rmse",".estimator":"standard",".estimate":0.000153771958484089,"x":1426809600000,"y":0.000153771958484089},{"date":"20150323",".metric":"rmse",".estimator":"standard",".estimate":0.00096664476767556,"x":1427068800000,"y":0.00096664476767556},{"date":"20150324",".metric":"rmse",".estimator":"standard",".estimate":0.00280287339960634,"x":1427155200000,"y":0.00280287339960634},{"date":"20150325",".metric":"rmse",".estimator":"standard",".estimate":0.00574731265276986,"x":1427241600000,"y":0.00574731265276986},{"date":"20150326",".metric":"rmse",".estimator":"standard",".estimate":2.69665922843373e05,"x":1427328000000,"y":2.69665922843373e05},{"date":"20150327",".metric":"rmse",".estimator":"standard",".estimate":0.00124996438604877,"x":1427414400000,"y":0.00124996438604877},{"date":"20150330",".metric":"rmse",".estimator":"standard",".estimate":0.000369263023341724,"x":1427673600000,"y":0.000369263023341724},{"date":"20150331",".metric":"rmse",".estimator":"standard",".estimate":0.00333011690004239,"x":1427760000000,"y":0.00333011690004239},{"date":"20150401",".metric":"rmse",".estimator":"standard",".estimate":0.000800734527692391,"x":1427846400000,"y":0.000800734527692391},{"date":"20150402",".metric":"rmse",".estimator":"standard",".estimate":0.00211113388134838,"x":1427932800000,"y":0.00211113388134838},{"date":"20150406",".metric":"rmse",".estimator":"standard",".estimate":0.00420722444446628,"x":1428278400000,"y":0.00420722444446628},{"date":"20150407",".metric":"rmse",".estimator":"standard",".estimate":0.00468779581382743,"x":1428364800000,"y":0.00468779581382743},{"date":"20150408",".metric":"rmse",".estimator":"standard",".estimate":0.00202548638905039,"x":1428451200000,"y":0.00202548638905039},{"date":"20150409",".metric":"rmse",".estimator":"standard",".estimate":0.00271888967659028,"x":1428537600000,"y":0.00271888967659028},{"date":"20150410",".metric":"rmse",".estimator":"standard",".estimate":0.00254943812031821,"x":1428624000000,"y":0.00254943812031821},{"date":"20150413",".metric":"rmse",".estimator":"standard",".estimate":0.000723113907213286,"x":1428883200000,"y":0.000723113907213286},{"date":"20150414",".metric":"rmse",".estimator":"standard",".estimate":0.000824328027385011,"x":1428969600000,"y":0.000824328027385011},{"date":"20150415",".metric":"rmse",".estimator":"standard",".estimate":0.000319171035078047,"x":1429056000000,"y":0.000319171035078047},{"date":"20150416",".metric":"rmse",".estimator":"standard",".estimate":0.000237303743723273,"x":1429142400000,"y":0.000237303743723273},{"date":"20150417",".metric":"rmse",".estimator":"standard",".estimate":0.00142556356502989,"x":1429228800000,"y":0.00142556356502989},{"date":"20150420",".metric":"rmse",".estimator":"standard",".estimate":0.00120494144999053,"x":1429488000000,"y":0.00120494144999053},{"date":"20150421",".metric":"rmse",".estimator":"standard",".estimate":0.00243306536885689,"x":1429574400000,"y":0.00243306536885689},{"date":"20150422",".metric":"rmse",".estimator":"standard",".estimate":0.000642918523424196,"x":1429660800000,"y":0.000642918523424196},{"date":"20150423",".metric":"rmse",".estimator":"standard",".estimate":0.000847782712402617,"x":1429747200000,"y":0.000847782712402617},{"date":"20150424",".metric":"rmse",".estimator":"standard",".estimate":0.0013466046179942,"x":1429833600000,"y":0.0013466046179942},{"date":"20150427",".metric":"rmse",".estimator":"standard",".estimate":0.00654832948070044,"x":1430092800000,"y":0.00654832948070044},{"date":"20150428",".metric":"rmse",".estimator":"standard",".estimate":0.00678489140048155,"x":1430179200000,"y":0.00678489140048155},{"date":"20150429",".metric":"rmse",".estimator":"standard",".estimate":0.000777663114918571,"x":1430265600000,"y":0.000777663114918571},{"date":"20150430",".metric":"rmse",".estimator":"standard",".estimate":0.00371967268152809,"x":1430352000000,"y":0.00371967268152809},{"date":"20150501",".metric":"rmse",".estimator":"standard",".estimate":0.00346225141118423,"x":1430438400000,"y":0.00346225141118423},{"date":"20150504",".metric":"rmse",".estimator":"standard",".estimate":0.000107364674304658,"x":1430697600000,"y":0.000107364674304658},{"date":"20150505",".metric":"rmse",".estimator":"standard",".estimate":3.08138475900062e05,"x":1430784000000,"y":3.08138475900062e05},{"date":"20150506",".metric":"rmse",".estimator":"standard",".estimate":0.00293495531587135,"x":1430870400000,"y":0.00293495531587135},{"date":"20150507",".metric":"rmse",".estimator":"standard",".estimate":0.00373552826403472,"x":1430956800000,"y":0.00373552826403472},{"date":"20150508",".metric":"rmse",".estimator":"standard",".estimate":0.000568415536384584,"x":1431043200000,"y":0.000568415536384584},{"date":"20150511",".metric":"rmse",".estimator":"standard",".estimate":6.1954636984678e05,"x":1431302400000,"y":6.1954636984678e05},{"date":"20150512",".metric":"rmse",".estimator":"standard",".estimate":0.00197403080935961,"x":1431388800000,"y":0.00197403080935961},{"date":"20150513",".metric":"rmse",".estimator":"standard",".estimate":0.00216174101677157,"x":1431475200000,"y":0.00216174101677157},{"date":"20150514",".metric":"rmse",".estimator":"standard",".estimate":0.00137483653164649,"x":1431561600000,"y":0.00137483653164649},{"date":"20150515",".metric":"rmse",".estimator":"standard",".estimate":0.000270038671279347,"x":1431648000000,"y":0.000270038671279347},{"date":"20150518",".metric":"rmse",".estimator":"standard",".estimate":0.00382533238668263,"x":1431907200000,"y":0.00382533238668263},{"date":"20150519",".metric":"rmse",".estimator":"standard",".estimate":6.50218287763947e05,"x":1431993600000,"y":6.50218287763947e05},{"date":"20150520",".metric":"rmse",".estimator":"standard",".estimate":0.000243958145563688,"x":1432080000000,"y":0.000243958145563688},{"date":"20150521",".metric":"rmse",".estimator":"standard",".estimate":0.00213951116460373,"x":1432166400000,"y":0.00213951116460373},{"date":"20150522",".metric":"rmse",".estimator":"standard",".estimate":0.00162751922109553,"x":1432252800000,"y":0.00162751922109553},{"date":"20150526",".metric":"rmse",".estimator":"standard",".estimate":2.74205103022677e05,"x":1432598400000,"y":2.74205103022677e05},{"date":"20150527",".metric":"rmse",".estimator":"standard",".estimate":0.00146570580680937,"x":1432684800000,"y":0.00146570580680937},{"date":"20150528",".metric":"rmse",".estimator":"standard",".estimate":0.00114926777927846,"x":1432771200000,"y":0.00114926777927846},{"date":"20150529",".metric":"rmse",".estimator":"standard",".estimate":0.000488777570115981,"x":1432857600000,"y":0.000488777570115981},{"date":"20150601",".metric":"rmse",".estimator":"standard",".estimate":0.00407131652498733,"x":1433116800000,"y":0.00407131652498733},{"date":"20150602",".metric":"rmse",".estimator":"standard",".estimate":0.00204226604847051,"x":1433203200000,"y":0.00204226604847051},{"date":"20150603",".metric":"rmse",".estimator":"standard",".estimate":0.000700663281221754,"x":1433289600000,"y":0.000700663281221754},{"date":"20150604",".metric":"rmse",".estimator":"standard",".estimate":0.00205612474736232,"x":1433376000000,"y":0.00205612474736232},{"date":"20150605",".metric":"rmse",".estimator":"standard",".estimate":0.00295680947839454,"x":1433462400000,"y":0.00295680947839454},{"date":"20150608",".metric":"rmse",".estimator":"standard",".estimate":0.00157716240117475,"x":1433721600000,"y":0.00157716240117475},{"date":"20150609",".metric":"rmse",".estimator":"standard",".estimate":0.000509103297587919,"x":1433808000000,"y":0.000509103297587919},{"date":"20150610",".metric":"rmse",".estimator":"standard",".estimate":0.00144361579136164,"x":1433894400000,"y":0.00144361579136164},{"date":"20150611",".metric":"rmse",".estimator":"standard",".estimate":0.000743861394920468,"x":1433980800000,"y":0.000743861394920468},{"date":"20150612",".metric":"rmse",".estimator":"standard",".estimate":0.00270145927928372,"x":1434067200000,"y":0.00270145927928372},{"date":"20150615",".metric":"rmse",".estimator":"standard",".estimate":0.000926839353652406,"x":1434326400000,"y":0.000926839353652406},{"date":"20150616",".metric":"rmse",".estimator":"standard",".estimate":0.000378560473414271,"x":1434412800000,"y":0.000378560473414271},{"date":"20150617",".metric":"rmse",".estimator":"standard",".estimate":0.000606422038276323,"x":1434499200000,"y":0.000606422038276323},{"date":"20150618",".metric":"rmse",".estimator":"standard",".estimate":0.00235093666152922,"x":1434585600000,"y":0.00235093666152922},{"date":"20150619",".metric":"rmse",".estimator":"standard",".estimate":0.0011336978173662,"x":1434672000000,"y":0.0011336978173662},{"date":"20150622",".metric":"rmse",".estimator":"standard",".estimate":0.0068783144157268,"x":1434931200000,"y":0.0068783144157268},{"date":"20150623",".metric":"rmse",".estimator":"standard",".estimate":0.00123664431681601,"x":1435017600000,"y":0.00123664431681601},{"date":"20150624",".metric":"rmse",".estimator":"standard",".estimate":0.00284828561798717,"x":1435104000000,"y":0.00284828561798717},{"date":"20150625",".metric":"rmse",".estimator":"standard",".estimate":0.000491312260733792,"x":1435190400000,"y":0.000491312260733792},{"date":"20150626",".metric":"rmse",".estimator":"standard",".estimate":0.00280120919641011,"x":1435276800000,"y":0.00280120919641011},{"date":"20150629",".metric":"rmse",".estimator":"standard",".estimate":0.00963998417169128,"x":1435536000000,"y":0.00963998417169128},{"date":"20150630",".metric":"rmse",".estimator":"standard",".estimate":0.00614788463974238,"x":1435622400000,"y":0.00614788463974238},{"date":"20150701",".metric":"rmse",".estimator":"standard",".estimate":0.000406467434416981,"x":1435708800000,"y":0.000406467434416981},{"date":"20150702",".metric":"rmse",".estimator":"standard",".estimate":0.000811571940198679,"x":1435795200000,"y":0.000811571940198679},{"date":"20150706",".metric":"rmse",".estimator":"standard",".estimate":0.00509080325444627,"x":1436140800000,"y":0.00509080325444627},{"date":"20150707",".metric":"rmse",".estimator":"standard",".estimate":0.00807239426729424,"x":1436227200000,"y":0.00807239426729424},{"date":"20150708",".metric":"rmse",".estimator":"standard",".estimate":0.00774965792847944,"x":1436313600000,"y":0.00774965792847944},{"date":"20150709",".metric":"rmse",".estimator":"standard",".estimate":0.00304060787414298,"x":1436400000000,"y":0.00304060787414298},{"date":"20150710",".metric":"rmse",".estimator":"standard",".estimate":0.00179563015071285,"x":1436486400000,"y":0.00179563015071285},{"date":"20150713",".metric":"rmse",".estimator":"standard",".estimate":0.00465201680262755,"x":1436745600000,"y":0.00465201680262755},{"date":"20150714",".metric":"rmse",".estimator":"standard",".estimate":0.00164811486400728,"x":1436832000000,"y":0.00164811486400728},{"date":"20150715",".metric":"rmse",".estimator":"standard",".estimate":0.00156774204135862,"x":1436918400000,"y":0.00156774204135862},{"date":"20150716",".metric":"rmse",".estimator":"standard",".estimate":0.000354375284437844,"x":1437004800000,"y":0.000354375284437844},{"date":"20150717",".metric":"rmse",".estimator":"standard",".estimate":0.00131872471606728,"x":1437091200000,"y":0.00131872471606728},{"date":"20150720",".metric":"rmse",".estimator":"standard",".estimate":0.0015726815682986,"x":1437350400000,"y":0.0015726815682986},{"date":"20150721",".metric":"rmse",".estimator":"standard",".estimate":0.000525162727281741,"x":1437436800000,"y":0.000525162727281741},{"date":"20150722",".metric":"rmse",".estimator":"standard",".estimate":0.00139037651660623,"x":1437523200000,"y":0.00139037651660623},{"date":"20150723",".metric":"rmse",".estimator":"standard",".estimate":0.00581183024543379,"x":1437609600000,"y":0.00581183024543379},{"date":"20150724",".metric":"rmse",".estimator":"standard",".estimate":0.00241219761518981,"x":1437696000000,"y":0.00241219761518981},{"date":"20150727",".metric":"rmse",".estimator":"standard",".estimate":0.000763883990557287,"x":1437955200000,"y":0.000763883990557287},{"date":"20150728",".metric":"rmse",".estimator":"standard",".estimate":0.00299993049672719,"x":1438041600000,"y":0.00299993049672719},{"date":"20150729",".metric":"rmse",".estimator":"standard",".estimate":0.00220400241253419,"x":1438128000000,"y":0.00220400241253419},{"date":"20150730",".metric":"rmse",".estimator":"standard",".estimate":0.000885402065001293,"x":1438214400000,"y":0.000885402065001293},{"date":"20150731",".metric":"rmse",".estimator":"standard",".estimate":0.00447994683444991,"x":1438300800000,"y":0.00447994683444991},{"date":"20150803",".metric":"rmse",".estimator":"standard",".estimate":0.00304606040265679,"x":1438560000000,"y":0.00304606040265679},{"date":"20150804",".metric":"rmse",".estimator":"standard",".estimate":8.54531067407526e05,"x":1438646400000,"y":8.54531067407526e05},{"date":"20150805",".metric":"rmse",".estimator":"standard",".estimate":0.003746877821574,"x":1438732800000,"y":0.003746877821574},{"date":"20150806",".metric":"rmse",".estimator":"standard",".estimate":0.00206132552144418,"x":1438819200000,"y":0.00206132552144418},{"date":"20150807",".metric":"rmse",".estimator":"standard",".estimate":0.00078137333722023,"x":1438905600000,"y":0.00078137333722023},{"date":"20150810",".metric":"rmse",".estimator":"standard",".estimate":0.00629187034087075,"x":1439164800000,"y":0.00629187034087075},{"date":"20150811",".metric":"rmse",".estimator":"standard",".estimate":0.00137798053706101,"x":1439251200000,"y":0.00137798053706101},{"date":"20150812",".metric":"rmse",".estimator":"standard",".estimate":0.00576804908886543,"x":1439337600000,"y":0.00576804908886543},{"date":"20150813",".metric":"rmse",".estimator":"standard",".estimate":0.00311120713321267,"x":1439424000000,"y":0.00311120713321267},{"date":"20150814",".metric":"rmse",".estimator":"standard",".estimate":0.00355661003054919,"x":1439510400000,"y":0.00355661003054919},{"date":"20150817",".metric":"rmse",".estimator":"standard",".estimate":0.00330562134101764,"x":1439769600000,"y":0.00330562134101764},{"date":"20150818",".metric":"rmse",".estimator":"standard",".estimate":0.00200251053844045,"x":1439856000000,"y":0.00200251053844045},{"date":"20150819",".metric":"rmse",".estimator":"standard",".estimate":0.00276658191142795,"x":1439942400000,"y":0.00276658191142795},{"date":"20150820",".metric":"rmse",".estimator":"standard",".estimate":0.00556411265509235,"x":1440028800000,"y":0.00556411265509235},{"date":"20150821",".metric":"rmse",".estimator":"standard",".estimate":0.0171847059316796,"x":1440115200000,"y":0.0171847059316796},{"date":"20150824",".metric":"rmse",".estimator":"standard",".estimate":0.0246145137540276,"x":1440374400000,"y":0.0246145137540276},{"date":"20150825",".metric":"rmse",".estimator":"standard",".estimate":0.0145503163878593,"x":1440460800000,"y":0.0145503163878593},{"date":"20150826",".metric":"rmse",".estimator":"standard",".estimate":0.0297846625793239,"x":1440547200000,"y":0.0297846625793239},{"date":"20150827",".metric":"rmse",".estimator":"standard",".estimate":0.0112745785635245,"x":1440633600000,"y":0.0112745785635245},{"date":"20150828",".metric":"rmse",".estimator":"standard",".estimate":0.000652500025708383,"x":1440720000000,"y":0.000652500025708383},{"date":"20150831",".metric":"rmse",".estimator":"standard",".estimate":0.00412055162031694,"x":1440979200000,"y":0.00412055162031694},{"date":"20150901",".metric":"rmse",".estimator":"standard",".estimate":0.00604123138552932,"x":1441065600000,"y":0.00604123138552932},{"date":"20150902",".metric":"rmse",".estimator":"standard",".estimate":0.00311910901748417,"x":1441152000000,"y":0.00311910901748417},{"date":"20150903",".metric":"rmse",".estimator":"standard",".estimate":0.00217445455632374,"x":1441238400000,"y":0.00217445455632374},{"date":"20150904",".metric":"rmse",".estimator":"standard",".estimate":0.00041940110598717,"x":1441324800000,"y":0.00041940110598717},{"date":"20150908",".metric":"rmse",".estimator":"standard",".estimate":0.00639249519682666,"x":1441670400000,"y":0.00639249519682666},{"date":"20150909",".metric":"rmse",".estimator":"standard",".estimate":0.0140990892360165,"x":1441756800000,"y":0.0140990892360165},{"date":"20150910",".metric":"rmse",".estimator":"standard",".estimate":0.00623370926371633,"x":1441843200000,"y":0.00623370926371633},{"date":"20150911",".metric":"rmse",".estimator":"standard",".estimate":0.00602317572435243,"x":1441929600000,"y":0.00602317572435243},{"date":"20150914",".metric":"rmse",".estimator":"standard",".estimate":0.00139710374113922,"x":1442188800000,"y":0.00139710374113922},{"date":"20150915",".metric":"rmse",".estimator":"standard",".estimate":0.00587095100687109,"x":1442275200000,"y":0.00587095100687109},{"date":"20150916",".metric":"rmse",".estimator":"standard",".estimate":0.00238667462773552,"x":1442361600000,"y":0.00238667462773552},{"date":"20150917",".metric":"rmse",".estimator":"standard",".estimate":0.00286420083260528,"x":1442448000000,"y":0.00286420083260528},{"date":"20150918",".metric":"rmse",".estimator":"standard",".estimate":0.00040238550366976,"x":1442534400000,"y":0.00040238550366976},{"date":"20150921",".metric":"rmse",".estimator":"standard",".estimate":0.00312927932426035,"x":1442793600000,"y":0.00312927932426035},{"date":"20150922",".metric":"rmse",".estimator":"standard",".estimate":0.00535824984915497,"x":1442880000000,"y":0.00535824984915497},{"date":"20150923",".metric":"rmse",".estimator":"standard",".estimate":0.000822491154298188,"x":1442966400000,"y":0.000822491154298188},{"date":"20150924",".metric":"rmse",".estimator":"standard",".estimate":0.00224123623473526,"x":1443052800000,"y":0.00224123623473526},{"date":"20150925",".metric":"rmse",".estimator":"standard",".estimate":0.0138344691619113,"x":1443139200000,"y":0.0138344691619113},{"date":"20150928",".metric":"rmse",".estimator":"standard",".estimate":0.00357444326627458,"x":1443398400000,"y":0.00357444326627458},{"date":"20150929",".metric":"rmse",".estimator":"standard",".estimate":0.00524808800446163,"x":1443484800000,"y":0.00524808800446163},{"date":"20150930",".metric":"rmse",".estimator":"standard",".estimate":0.00191601260547396,"x":1443571200000,"y":0.00191601260547396},{"date":"20151001",".metric":"rmse",".estimator":"standard",".estimate":0.00161335278422444,"x":1443657600000,"y":0.00161335278422444},{"date":"20151002",".metric":"rmse",".estimator":"standard",".estimate":0.00401563431263661,"x":1443744000000,"y":0.00401563431263661},{"date":"20151005",".metric":"rmse",".estimator":"standard",".estimate":0.00368765685413026,"x":1444003200000,"y":0.00368765685413026},{"date":"20151006",".metric":"rmse",".estimator":"standard",".estimate":0.00296462790166401,"x":1444089600000,"y":0.00296462790166401},{"date":"20151007",".metric":"rmse",".estimator":"standard",".estimate":0.00356094316941773,"x":1444176000000,"y":0.00356094316941773},{"date":"20151008",".metric":"rmse",".estimator":"standard",".estimate":0.00349268333631326,"x":1444262400000,"y":0.00349268333631326},{"date":"20151009",".metric":"rmse",".estimator":"standard",".estimate":0.00411732838803702,"x":1444348800000,"y":0.00411732838803702},{"date":"20151012",".metric":"rmse",".estimator":"standard",".estimate":0.000457143179134392,"x":1444608000000,"y":0.000457143179134392},{"date":"20151013",".metric":"rmse",".estimator":"standard",".estimate":0.00175946600828844,"x":1444694400000,"y":0.00175946600828844},{"date":"20151014",".metric":"rmse",".estimator":"standard",".estimate":0.000761371757338223,"x":1444780800000,"y":0.000761371757338223},{"date":"20151015",".metric":"rmse",".estimator":"standard",".estimate":0.00483450335566792,"x":1444867200000,"y":0.00483450335566792},{"date":"20151016",".metric":"rmse",".estimator":"standard",".estimate":0.0009009931278971,"x":1444953600000,"y":0.0009009931278971},{"date":"20151019",".metric":"rmse",".estimator":"standard",".estimate":0.00161812225629585,"x":1445212800000,"y":0.00161812225629585},{"date":"20151020",".metric":"rmse",".estimator":"standard",".estimate":0.000847963004616699,"x":1445299200000,"y":0.000847963004616699},{"date":"20151021",".metric":"rmse",".estimator":"standard",".estimate":0.00449553817830094,"x":1445385600000,"y":0.00449553817830094},{"date":"20151022",".metric":"rmse",".estimator":"standard",".estimate":0.00412720015707421,"x":1445472000000,"y":0.00412720015707421},{"date":"20151023",".metric":"rmse",".estimator":"standard",".estimate":0.0075915547846696,"x":1445558400000,"y":0.0075915547846696},{"date":"20151026",".metric":"rmse",".estimator":"standard",".estimate":0.00413753895445339,"x":1445817600000,"y":0.00413753895445339},{"date":"20151027",".metric":"rmse",".estimator":"standard",".estimate":0.00186023814882855,"x":1445904000000,"y":0.00186023814882855},{"date":"20151028",".metric":"rmse",".estimator":"standard",".estimate":0.0027230691101985,"x":1445990400000,"y":0.0027230691101985},{"date":"20151029",".metric":"rmse",".estimator":"standard",".estimate":0.00127307043453564,"x":1446076800000,"y":0.00127307043453564},{"date":"20151030",".metric":"rmse",".estimator":"standard",".estimate":0.00208697975205613,"x":1446163200000,"y":0.00208697975205613},{"date":"20151102",".metric":"rmse",".estimator":"standard",".estimate":0.0094658082655116,"x":1446422400000,"y":0.0094658082655116},{"date":"20151103",".metric":"rmse",".estimator":"standard",".estimate":0.00177553400072022,"x":1446508800000,"y":0.00177553400072022},{"date":"20151104",".metric":"rmse",".estimator":"standard",".estimate":0.00151539269324002,"x":1446595200000,"y":0.00151539269324002},{"date":"20151105",".metric":"rmse",".estimator":"standard",".estimate":0.000186268602158622,"x":1446681600000,"y":0.000186268602158622},{"date":"20151106",".metric":"rmse",".estimator":"standard",".estimate":0.00348294025067781,"x":1446768000000,"y":0.00348294025067781},{"date":"20151109",".metric":"rmse",".estimator":"standard",".estimate":0.00392325383365242,"x":1447027200000,"y":0.00392325383365242},{"date":"20151110",".metric":"rmse",".estimator":"standard",".estimate":0.00302523584044596,"x":1447113600000,"y":0.00302523584044596},{"date":"20151111",".metric":"rmse",".estimator":"standard",".estimate":0.00407952531120488,"x":1447200000000,"y":0.00407952531120488},{"date":"20151112",".metric":"rmse",".estimator":"standard",".estimate":0.00640934431579107,"x":1447286400000,"y":0.00640934431579107},{"date":"20151113",".metric":"rmse",".estimator":"standard",".estimate":0.00061421375038837,"x":1447372800000,"y":0.00061421375038837},{"date":"20151116",".metric":"rmse",".estimator":"standard",".estimate":0.00881911536493924,"x":1447632000000,"y":0.00881911536493924},{"date":"20151117",".metric":"rmse",".estimator":"standard",".estimate":0.00409082751160914,"x":1447718400000,"y":0.00409082751160914},{"date":"20151118",".metric":"rmse",".estimator":"standard",".estimate":0.00373578110429996,"x":1447804800000,"y":0.00373578110429996},{"date":"20151119",".metric":"rmse",".estimator":"standard",".estimate":0.000940209145386004,"x":1447891200000,"y":0.000940209145386004},{"date":"20151120",".metric":"rmse",".estimator":"standard",".estimate":0.00409924729863848,"x":1447977600000,"y":0.00409924729863848},{"date":"20151123",".metric":"rmse",".estimator":"standard",".estimate":0.00077736955312196,"x":1448236800000,"y":0.00077736955312196},{"date":"20151124",".metric":"rmse",".estimator":"standard",".estimate":0.00725924885664988,"x":1448323200000,"y":0.00725924885664988},{"date":"20151125",".metric":"rmse",".estimator":"standard",".estimate":0.0023727074013492,"x":1448409600000,"y":0.0023727074013492},{"date":"20151127",".metric":"rmse",".estimator":"standard",".estimate":0.00285667605279557,"x":1448582400000,"y":0.00285667605279557},{"date":"20151130",".metric":"rmse",".estimator":"standard",".estimate":0.000540441757668088,"x":1448841600000,"y":0.000540441757668088},{"date":"20151201",".metric":"rmse",".estimator":"standard",".estimate":0.00281707396194064,"x":1448928000000,"y":0.00281707396194064},{"date":"20151202",".metric":"rmse",".estimator":"standard",".estimate":0.00380555182760584,"x":1449014400000,"y":0.00380555182760584},{"date":"20151203",".metric":"rmse",".estimator":"standard",".estimate":0.00368604681807069,"x":1449100800000,"y":0.00368604681807069},{"date":"20151204",".metric":"rmse",".estimator":"standard",".estimate":0.0157260417998821,"x":1449187200000,"y":0.0157260417998821},{"date":"20151207",".metric":"rmse",".estimator":"standard",".estimate":0.00259219326445448,"x":1449446400000,"y":0.00259219326445448},{"date":"20151208",".metric":"rmse",".estimator":"standard",".estimate":0.000502732151715991,"x":1449532800000,"y":0.000502732151715991},{"date":"20151209",".metric":"rmse",".estimator":"standard",".estimate":0.00507356066681386,"x":1449619200000,"y":0.00507356066681386},{"date":"20151210",".metric":"rmse",".estimator":"standard",".estimate":0.00478650841022692,"x":1449705600000,"y":0.00478650841022692},{"date":"20151211",".metric":"rmse",".estimator":"standard",".estimate":0.00378942098803433,"x":1449792000000,"y":0.00378942098803433},{"date":"20151214",".metric":"rmse",".estimator":"standard",".estimate":0.0051286889839488,"x":1450051200000,"y":0.0051286889839488},{"date":"20151215",".metric":"rmse",".estimator":"standard",".estimate":0.00572619680990926,"x":1450137600000,"y":0.00572619680990926},{"date":"20151216",".metric":"rmse",".estimator":"standard",".estimate":0.000756598335299976,"x":1450224000000,"y":0.000756598335299976},{"date":"20151217",".metric":"rmse",".estimator":"standard",".estimate":0.00643824801210792,"x":1450310400000,"y":0.00643824801210792},{"date":"20151218",".metric":"rmse",".estimator":"standard",".estimate":0.00495687121937044,"x":1450396800000,"y":0.00495687121937044},{"date":"20151221",".metric":"rmse",".estimator":"standard",".estimate":0.00887843066546218,"x":1450656000000,"y":0.00887843066546218},{"date":"20151222",".metric":"rmse",".estimator":"standard",".estimate":0.00236204083051885,"x":1450742400000,"y":0.00236204083051885},{"date":"20151223",".metric":"rmse",".estimator":"standard",".estimate":0.000914800671941883,"x":1450828800000,"y":0.000914800671941883},{"date":"20151224",".metric":"rmse",".estimator":"standard",".estimate":0.0026669781862874,"x":1450915200000,"y":0.0026669781862874},{"date":"20151228",".metric":"rmse",".estimator":"standard",".estimate":0.00239132223435226,"x":1451260800000,"y":0.00239132223435226},{"date":"20151229",".metric":"rmse",".estimator":"standard",".estimate":0.000306643195085988,"x":1451347200000,"y":0.000306643195085988},{"date":"20151230",".metric":"rmse",".estimator":"standard",".estimate":0.000523259974480772,"x":1451433600000,"y":0.000523259974480772},{"date":"20151231",".metric":"rmse",".estimator":"standard",".estimate":0.000265027554290816,"x":1451520000000,"y":0.000265027554290816},{"date":"20160104",".metric":"rmse",".estimator":"standard",".estimate":0.00435473108535608,"x":1451865600000,"y":0.00435473108535608},{"date":"20160105",".metric":"rmse",".estimator":"standard",".estimate":4.99137262643245e05,"x":1451952000000,"y":4.99137262643245e05},{"date":"20160106",".metric":"rmse",".estimator":"standard",".estimate":4.8622227364398e05,"x":1452038400000,"y":4.8622227364398e05},{"date":"20160107",".metric":"rmse",".estimator":"standard",".estimate":0.0076402684003439,"x":1452124800000,"y":0.0076402684003439},{"date":"20160108",".metric":"rmse",".estimator":"standard",".estimate":9.21799517826765e05,"x":1452211200000,"y":9.21799517826765e05},{"date":"20160111",".metric":"rmse",".estimator":"standard",".estimate":0.0016457241875072,"x":1452470400000,"y":0.0016457241875072},{"date":"20160112",".metric":"rmse",".estimator":"standard",".estimate":0.000877015684916291,"x":1452556800000,"y":0.000877015684916291},{"date":"20160113",".metric":"rmse",".estimator":"standard",".estimate":0.00943443643821319,"x":1452643200000,"y":0.00943443643821319},{"date":"20160114",".metric":"rmse",".estimator":"standard",".estimate":0.009204029031713,"x":1452729600000,"y":0.009204029031713},{"date":"20160115",".metric":"rmse",".estimator":"standard",".estimate":0.000353175248814303,"x":1452816000000,"y":0.000353175248814303},{"date":"20160119",".metric":"rmse",".estimator":"standard",".estimate":0.00629909289057516,"x":1453161600000,"y":0.00629909289057516},{"date":"20160120",".metric":"rmse",".estimator":"standard",".estimate":0.00191760098925365,"x":1453248000000,"y":0.00191760098925365},{"date":"20160121",".metric":"rmse",".estimator":"standard",".estimate":0.00439061618074424,"x":1453334400000,"y":0.00439061618074424},{"date":"20160122",".metric":"rmse",".estimator":"standard",".estimate":0.010413951735192,"x":1453420800000,"y":0.010413951735192},{"date":"20160125",".metric":"rmse",".estimator":"standard",".estimate":0.00251449135223171,"x":1453680000000,"y":0.00251449135223171},{"date":"20160126",".metric":"rmse",".estimator":"standard",".estimate":0.0028650925539264,"x":1453766400000,"y":0.0028650925539264},{"date":"20160127",".metric":"rmse",".estimator":"standard",".estimate":0.00461631957115544,"x":1453852800000,"y":0.00461631957115544},{"date":"20160128",".metric":"rmse",".estimator":"standard",".estimate":0.00577257442460362,"x":1453939200000,"y":0.00577257442460362},{"date":"20160129",".metric":"rmse",".estimator":"standard",".estimate":0.00670028948029786,"x":1454025600000,"y":0.00670028948029786},{"date":"20160201",".metric":"rmse",".estimator":"standard",".estimate":0.00394007287995396,"x":1454284800000,"y":0.00394007287995396},{"date":"20160202",".metric":"rmse",".estimator":"standard",".estimate":0.00150439975495101,"x":1454371200000,"y":0.00150439975495101},{"date":"20160203",".metric":"rmse",".estimator":"standard",".estimate":0.00606677272423137,"x":1454457600000,"y":0.00606677272423137},{"date":"20160204",".metric":"rmse",".estimator":"standard",".estimate":0.00635159670254089,"x":1454544000000,"y":0.00635159670254089},{"date":"20160205",".metric":"rmse",".estimator":"standard",".estimate":0.00992725209016555,"x":1454630400000,"y":0.00992725209016555},{"date":"20160208",".metric":"rmse",".estimator":"standard",".estimate":0.00758043240968828,"x":1454889600000,"y":0.00758043240968828},{"date":"20160209",".metric":"rmse",".estimator":"standard",".estimate":0.00466694271667664,"x":1454976000000,"y":0.00466694271667664},{"date":"20160210",".metric":"rmse",".estimator":"standard",".estimate":0.00654589752122079,"x":1455062400000,"y":0.00654589752122079},{"date":"20160211",".metric":"rmse",".estimator":"standard",".estimate":0.00322855895854392,"x":1455148800000,"y":0.00322855895854392},{"date":"20160212",".metric":"rmse",".estimator":"standard",".estimate":0.00958575719258743,"x":1455235200000,"y":0.00958575719258743},{"date":"20160216",".metric":"rmse",".estimator":"standard",".estimate":0.00389361272050569,"x":1455580800000,"y":0.00389361272050569},{"date":"20160217",".metric":"rmse",".estimator":"standard",".estimate":0.000787449185594025,"x":1455667200000,"y":0.000787449185594025},{"date":"20160218",".metric":"rmse",".estimator":"standard",".estimate":0.000357318694135214,"x":1455753600000,"y":0.000357318694135214},{"date":"20160219",".metric":"rmse",".estimator":"standard",".estimate":0.000768228836225628,"x":1455840000000,"y":0.000768228836225628},{"date":"20160222",".metric":"rmse",".estimator":"standard",".estimate":0.000887784879040737,"x":1456099200000,"y":0.000887784879040737},{"date":"20160223",".metric":"rmse",".estimator":"standard",".estimate":0.00282139674783032,"x":1456185600000,"y":0.00282139674783032},{"date":"20160224",".metric":"rmse",".estimator":"standard",".estimate":0.0107872487574005,"x":1456272000000,"y":0.0107872487574005},{"date":"20160225",".metric":"rmse",".estimator":"standard",".estimate":0.00135768760736578,"x":1456358400000,"y":0.00135768760736578},{"date":"20160226",".metric":"rmse",".estimator":"standard",".estimate":0.00315134214895253,"x":1456444800000,"y":0.00315134214895253},{"date":"20160229",".metric":"rmse",".estimator":"standard",".estimate":0.00309953861889532,"x":1456704000000,"y":0.00309953861889532},{"date":"20160301",".metric":"rmse",".estimator":"standard",".estimate":0.00596851063581625,"x":1456790400000,"y":0.00596851063581625},{"date":"20160302",".metric":"rmse",".estimator":"standard",".estimate":0.002590271725395,"x":1456876800000,"y":0.002590271725395},{"date":"20160303",".metric":"rmse",".estimator":"standard",".estimate":0.000895392770388821,"x":1456963200000,"y":0.000895392770388821},{"date":"20160304",".metric":"rmse",".estimator":"standard",".estimate":0.000695556331745784,"x":1457049600000,"y":0.000695556331745784},{"date":"20160307",".metric":"rmse",".estimator":"standard",".estimate":0.00419991571814754,"x":1457308800000,"y":0.00419991571814754},{"date":"20160308",".metric":"rmse",".estimator":"standard",".estimate":0.00218963388792789,"x":1457395200000,"y":0.00218963388792789},{"date":"20160309",".metric":"rmse",".estimator":"standard",".estimate":0.00173384458370314,"x":1457481600000,"y":0.00173384458370314},{"date":"20160310",".metric":"rmse",".estimator":"standard",".estimate":0.00205103302636883,"x":1457568000000,"y":0.00205103302636883},{"date":"20160311",".metric":"rmse",".estimator":"standard",".estimate":0.000199092245679135,"x":1457654400000,"y":0.000199092245679135},{"date":"20160314",".metric":"rmse",".estimator":"standard",".estimate":0.00103645659262195,"x":1457913600000,"y":0.00103645659262195},{"date":"20160315",".metric":"rmse",".estimator":"standard",".estimate":0.00200455073617329,"x":1458000000000,"y":0.00200455073617329},{"date":"20160316",".metric":"rmse",".estimator":"standard",".estimate":0.000996575530414657,"x":1458086400000,"y":0.000996575530414657},{"date":"20160317",".metric":"rmse",".estimator":"standard",".estimate":0.00316782341795077,"x":1458172800000,"y":0.00316782341795077},{"date":"20160318",".metric":"rmse",".estimator":"standard",".estimate":0.000948794057591311,"x":1458259200000,"y":0.000948794057591311},{"date":"20160321",".metric":"rmse",".estimator":"standard",".estimate":0.00146085276874177,"x":1458518400000,"y":0.00146085276874177},{"date":"20160322",".metric":"rmse",".estimator":"standard",".estimate":0.000253473046446184,"x":1458604800000,"y":0.000253473046446184},{"date":"20160323",".metric":"rmse",".estimator":"standard",".estimate":0.000394696354946654,"x":1458691200000,"y":0.000394696354946654},{"date":"20160324",".metric":"rmse",".estimator":"standard",".estimate":0.00434327783930099,"x":1458777600000,"y":0.00434327783930099},{"date":"20160328",".metric":"rmse",".estimator":"standard",".estimate":0.00254485496312226,"x":1459123200000,"y":0.00254485496312226},{"date":"20160329",".metric":"rmse",".estimator":"standard",".estimate":0.00293945789754692,"x":1459209600000,"y":0.00293945789754692},{"date":"20160330",".metric":"rmse",".estimator":"standard",".estimate":0.00292839427870702,"x":1459296000000,"y":0.00292839427870702},{"date":"20160331",".metric":"rmse",".estimator":"standard",".estimate":0.000726623417067855,"x":1459382400000,"y":0.000726623417067855},{"date":"20160401",".metric":"rmse",".estimator":"standard",".estimate":0.00599720890133593,"x":1459468800000,"y":0.00599720890133593},{"date":"20160404",".metric":"rmse",".estimator":"standard",".estimate":0.00460142348446367,"x":1459728000000,"y":0.00460142348446367},{"date":"20160405",".metric":"rmse",".estimator":"standard",".estimate":0.00368981274482765,"x":1459814400000,"y":0.00368981274482765},{"date":"20160406",".metric":"rmse",".estimator":"standard",".estimate":0.00259717641563854,"x":1459900800000,"y":0.00259717641563854},{"date":"20160407",".metric":"rmse",".estimator":"standard",".estimate":0.00154438576727709,"x":1459987200000,"y":0.00154438576727709},{"date":"20160408",".metric":"rmse",".estimator":"standard",".estimate":0.00391666299607268,"x":1460073600000,"y":0.00391666299607268},{"date":"20160411",".metric":"rmse",".estimator":"standard",".estimate":0.00134671830652272,"x":1460332800000,"y":0.00134671830652272},{"date":"20160412",".metric":"rmse",".estimator":"standard",".estimate":0.00141967032713428,"x":1460419200000,"y":0.00141967032713428},{"date":"20160413",".metric":"rmse",".estimator":"standard",".estimate":0.00357924958125467,"x":1460505600000,"y":0.00357924958125467},{"date":"20160414",".metric":"rmse",".estimator":"standard",".estimate":0.000998301985721335,"x":1460592000000,"y":0.000998301985721335},{"date":"20160415",".metric":"rmse",".estimator":"standard",".estimate":0.00205469281685017,"x":1460678400000,"y":0.00205469281685017},{"date":"20160418",".metric":"rmse",".estimator":"standard",".estimate":0.00301784509946262,"x":1460937600000,"y":0.00301784509946262},{"date":"20160419",".metric":"rmse",".estimator":"standard",".estimate":0.00718699191439488,"x":1461024000000,"y":0.00718699191439488},{"date":"20160420",".metric":"rmse",".estimator":"standard",".estimate":0.00106201157201088,"x":1461110400000,"y":0.00106201157201088},{"date":"20160421",".metric":"rmse",".estimator":"standard",".estimate":0.00592184849459788,"x":1461196800000,"y":0.00592184849459788},{"date":"20160422",".metric":"rmse",".estimator":"standard",".estimate":0.00177747332129503,"x":1461283200000,"y":0.00177747332129503},{"date":"20160425",".metric":"rmse",".estimator":"standard",".estimate":0.000890597896476351,"x":1461542400000,"y":0.000890597896476351},{"date":"20160426",".metric":"rmse",".estimator":"standard",".estimate":0.00109709822713204,"x":1461628800000,"y":0.00109709822713204},{"date":"20160427",".metric":"rmse",".estimator":"standard",".estimate":0.00118245146927174,"x":1461715200000,"y":0.00118245146927174},{"date":"20160428",".metric":"rmse",".estimator":"standard",".estimate":0.00745966954471345,"x":1461801600000,"y":0.00745966954471345},{"date":"20160429",".metric":"rmse",".estimator":"standard",".estimate":0.000640248171194056,"x":1461888000000,"y":0.000640248171194056},{"date":"20160502",".metric":"rmse",".estimator":"standard",".estimate":0.00329397997581203,"x":1462147200000,"y":0.00329397997581203},{"date":"20160503",".metric":"rmse",".estimator":"standard",".estimate":0.00165645482482795,"x":1462233600000,"y":0.00165645482482795},{"date":"20160504",".metric":"rmse",".estimator":"standard",".estimate":0.00402444273380057,"x":1462320000000,"y":0.00402444273380057},{"date":"20160505",".metric":"rmse",".estimator":"standard",".estimate":0.00185719119331311,"x":1462406400000,"y":0.00185719119331311},{"date":"20160506",".metric":"rmse",".estimator":"standard",".estimate":0.0037362581696451,"x":1462492800000,"y":0.0037362581696451},{"date":"20160509",".metric":"rmse",".estimator":"standard",".estimate":0.00215024100261312,"x":1462752000000,"y":0.00215024100261312},{"date":"20160510",".metric":"rmse",".estimator":"standard",".estimate":0.000153526860089999,"x":1462838400000,"y":0.000153526860089999},{"date":"20160511",".metric":"rmse",".estimator":"standard",".estimate":0.00572802100333414,"x":1462924800000,"y":0.00572802100333414},{"date":"20160512",".metric":"rmse",".estimator":"standard",".estimate":0.000890313377164864,"x":1463011200000,"y":0.000890313377164864},{"date":"20160513",".metric":"rmse",".estimator":"standard",".estimate":0.00170487435991207,"x":1463097600000,"y":0.00170487435991207},{"date":"20160516",".metric":"rmse",".estimator":"standard",".estimate":0.00298076176335253,"x":1463356800000,"y":0.00298076176335253},{"date":"20160517",".metric":"rmse",".estimator":"standard",".estimate":0.00418253200971225,"x":1463443200000,"y":0.00418253200971225},{"date":"20160518",".metric":"rmse",".estimator":"standard",".estimate":0.00309503739952598,"x":1463529600000,"y":0.00309503739952598},{"date":"20160519",".metric":"rmse",".estimator":"standard",".estimate":0.00038459289575377,"x":1463616000000,"y":0.00038459289575377},{"date":"20160520",".metric":"rmse",".estimator":"standard",".estimate":0.0013834793689818,"x":1463702400000,"y":0.0013834793689818},{"date":"20160523",".metric":"rmse",".estimator":"standard",".estimate":0.00208638939394112,"x":1463961600000,"y":0.00208638939394112},{"date":"20160524",".metric":"rmse",".estimator":"standard",".estimate":0.00205138137682025,"x":1464048000000,"y":0.00205138137682025},{"date":"20160525",".metric":"rmse",".estimator":"standard",".estimate":0.00208963123522787,"x":1464134400000,"y":0.00208963123522787},{"date":"20160526",".metric":"rmse",".estimator":"standard",".estimate":0.00251970710245937,"x":1464220800000,"y":0.00251970710245937},{"date":"20160527",".metric":"rmse",".estimator":"standard",".estimate":0.00326808685249009,"x":1464307200000,"y":0.00326808685249009},{"date":"20160531",".metric":"rmse",".estimator":"standard",".estimate":0.0014940017301349,"x":1464652800000,"y":0.0014940017301349},{"date":"20160601",".metric":"rmse",".estimator":"standard",".estimate":0.00406429020421808,"x":1464739200000,"y":0.00406429020421808},{"date":"20160602",".metric":"rmse",".estimator":"standard",".estimate":0.00315603697795239,"x":1464825600000,"y":0.00315603697795239},{"date":"20160603",".metric":"rmse",".estimator":"standard",".estimate":0.00225870371101233,"x":1464912000000,"y":0.00225870371101233},{"date":"20160606",".metric":"rmse",".estimator":"standard",".estimate":0.00174967185587845,"x":1465171200000,"y":0.00174967185587845},{"date":"20160607",".metric":"rmse",".estimator":"standard",".estimate":0.00126028131051133,"x":1465257600000,"y":0.00126028131051133},{"date":"20160608",".metric":"rmse",".estimator":"standard",".estimate":0.00171757609224733,"x":1465344000000,"y":0.00171757609224733},{"date":"20160609",".metric":"rmse",".estimator":"standard",".estimate":0.00417535879558533,"x":1465430400000,"y":0.00417535879558533},{"date":"20160610",".metric":"rmse",".estimator":"standard",".estimate":0.000213703812772883,"x":1465516800000,"y":0.000213703812772883},{"date":"20160613",".metric":"rmse",".estimator":"standard",".estimate":0.00050605259871416,"x":1465776000000,"y":0.00050605259871416},{"date":"20160614",".metric":"rmse",".estimator":"standard",".estimate":0.00456610427707012,"x":1465862400000,"y":0.00456610427707012},{"date":"20160615",".metric":"rmse",".estimator":"standard",".estimate":0.00420693469354562,"x":1465948800000,"y":0.00420693469354562},{"date":"20160616",".metric":"rmse",".estimator":"standard",".estimate":0.000952615275277162,"x":1466035200000,"y":0.000952615275277162},{"date":"20160617",".metric":"rmse",".estimator":"standard",".estimate":0.00545205205103526,"x":1466121600000,"y":0.00545205205103526},{"date":"20160620",".metric":"rmse",".estimator":"standard",".estimate":0.00264430798564621,"x":1466380800000,"y":0.00264430798564621},{"date":"20160621",".metric":"rmse",".estimator":"standard",".estimate":0.00162799812729609,"x":1466467200000,"y":0.00162799812729609},{"date":"20160622",".metric":"rmse",".estimator":"standard",".estimate":0.00459564363340417,"x":1466553600000,"y":0.00459564363340417},{"date":"20160623",".metric":"rmse",".estimator":"standard",".estimate":0.00126125717424231,"x":1466640000000,"y":0.00126125717424231},{"date":"20160624",".metric":"rmse",".estimator":"standard",".estimate":0.0218964449621362,"x":1466726400000,"y":0.0218964449621362},{"date":"20160627",".metric":"rmse",".estimator":"standard",".estimate":0.00521052872799053,"x":1466985600000,"y":0.00521052872799053},{"date":"20160628",".metric":"rmse",".estimator":"standard",".estimate":0.00316642968791388,"x":1467072000000,"y":0.00316642968791388},{"date":"20160629",".metric":"rmse",".estimator":"standard",".estimate":0.00620234930872215,"x":1467158400000,"y":0.00620234930872215},{"date":"20160630",".metric":"rmse",".estimator":"standard",".estimate":0.00408141724618029,"x":1467244800000,"y":0.00408141724618029},{"date":"20160701",".metric":"rmse",".estimator":"standard",".estimate":0.00160529542707455,"x":1467331200000,"y":0.00160529542707455},{"date":"20160705",".metric":"rmse",".estimator":"standard",".estimate":0.000420495638959605,"x":1467676800000,"y":0.000420495638959605},{"date":"20160706",".metric":"rmse",".estimator":"standard",".estimate":0.00569225319450741,"x":1467763200000,"y":0.00569225319450741},{"date":"20160707",".metric":"rmse",".estimator":"standard",".estimate":0.000777195035299231,"x":1467849600000,"y":0.000777195035299231},{"date":"20160708",".metric":"rmse",".estimator":"standard",".estimate":0.00773284183498263,"x":1467936000000,"y":0.00773284183498263},{"date":"20160711",".metric":"rmse",".estimator":"standard",".estimate":0.00225238666352208,"x":1468195200000,"y":0.00225238666352208},{"date":"20160712",".metric":"rmse",".estimator":"standard",".estimate":0.00202290859257446,"x":1468281600000,"y":0.00202290859257446},{"date":"20160713",".metric":"rmse",".estimator":"standard",".estimate":0.000245693389992942,"x":1468368000000,"y":0.000245693389992942},{"date":"20160714",".metric":"rmse",".estimator":"standard",".estimate":0.00152066896876954,"x":1468454400000,"y":0.00152066896876954},{"date":"20160715",".metric":"rmse",".estimator":"standard",".estimate":0.00078254577677485,"x":1468540800000,"y":0.00078254577677485},{"date":"20160718",".metric":"rmse",".estimator":"standard",".estimate":0.00020540843385228,"x":1468800000000,"y":0.00020540843385228},{"date":"20160719",".metric":"rmse",".estimator":"standard",".estimate":0.000109756098568486,"x":1468886400000,"y":0.000109756098568486},{"date":"20160720",".metric":"rmse",".estimator":"standard",".estimate":0.000459558969443395,"x":1468972800000,"y":0.000459558969443395},{"date":"20160721",".metric":"rmse",".estimator":"standard",".estimate":0.0025596046571624,"x":1469059200000,"y":0.0025596046571624},{"date":"20160722",".metric":"rmse",".estimator":"standard",".estimate":0.00159646668902795,"x":1469145600000,"y":0.00159646668902795},{"date":"20160725",".metric":"rmse",".estimator":"standard",".estimate":0.000752919943341165,"x":1469404800000,"y":0.000752919943341165},{"date":"20160726",".metric":"rmse",".estimator":"standard",".estimate":0.000342669890539401,"x":1469491200000,"y":0.000342669890539401},{"date":"20160727",".metric":"rmse",".estimator":"standard",".estimate":0.00160685797048586,"x":1469577600000,"y":0.00160685797048586},{"date":"20160728",".metric":"rmse",".estimator":"standard",".estimate":1.32340893194056e05,"x":1469664000000,"y":1.32340893194056e05},{"date":"20160729",".metric":"rmse",".estimator":"standard",".estimate":0.0053179992417029,"x":1469750400000,"y":0.0053179992417029},{"date":"20160801",".metric":"rmse",".estimator":"standard",".estimate":0.00251111239187018,"x":1470009600000,"y":0.00251111239187018},{"date":"20160802",".metric":"rmse",".estimator":"standard",".estimate":0.00120298550294066,"x":1470096000000,"y":0.00120298550294066},{"date":"20160803",".metric":"rmse",".estimator":"standard",".estimate":0.00215962728514518,"x":1470182400000,"y":0.00215962728514518},{"date":"20160804",".metric":"rmse",".estimator":"standard",".estimate":0.000265387922839076,"x":1470268800000,"y":0.000265387922839076},{"date":"20160805",".metric":"rmse",".estimator":"standard",".estimate":0.00330781285050998,"x":1470355200000,"y":0.00330781285050998},{"date":"20160808",".metric":"rmse",".estimator":"standard",".estimate":0.00111069887988652,"x":1470614400000,"y":0.00111069887988652},{"date":"20160809",".metric":"rmse",".estimator":"standard",".estimate":0.00205034315014647,"x":1470700800000,"y":0.00205034315014647},{"date":"20160810",".metric":"rmse",".estimator":"standard",".estimate":0.00268895250401861,"x":1470787200000,"y":0.00268895250401861},{"date":"20160811",".metric":"rmse",".estimator":"standard",".estimate":0.000442165938877304,"x":1470873600000,"y":0.000442165938877304},{"date":"20160812",".metric":"rmse",".estimator":"standard",".estimate":0.00125028667314022,"x":1470960000000,"y":0.00125028667314022},{"date":"20160815",".metric":"rmse",".estimator":"standard",".estimate":0.00276551418131063,"x":1471219200000,"y":0.00276551418131063},{"date":"20160816",".metric":"rmse",".estimator":"standard",".estimate":0.000826884946007651,"x":1471305600000,"y":0.000826884946007651},{"date":"20160817",".metric":"rmse",".estimator":"standard",".estimate":0.000134187926020832,"x":1471392000000,"y":0.000134187926020832},{"date":"20160818",".metric":"rmse",".estimator":"standard",".estimate":0.00232007960670281,"x":1471478400000,"y":0.00232007960670281},{"date":"20160819",".metric":"rmse",".estimator":"standard",".estimate":0.00296264890988167,"x":1471564800000,"y":0.00296264890988167},{"date":"20160822",".metric":"rmse",".estimator":"standard",".estimate":0.000720129925662719,"x":1471824000000,"y":0.000720129925662719},{"date":"20160823",".metric":"rmse",".estimator":"standard",".estimate":0.000507613820433002,"x":1471910400000,"y":0.000507613820433002},{"date":"20160824",".metric":"rmse",".estimator":"standard",".estimate":0.00217257880341865,"x":1471996800000,"y":0.00217257880341865},{"date":"20160825",".metric":"rmse",".estimator":"standard",".estimate":0.000613887044415466,"x":1472083200000,"y":0.000613887044415466},{"date":"20160826",".metric":"rmse",".estimator":"standard",".estimate":0.00178061008027771,"x":1472169600000,"y":0.00178061008027771},{"date":"20160829",".metric":"rmse",".estimator":"standard",".estimate":0.00116055813386945,"x":1472428800000,"y":0.00116055813386945},{"date":"20160830",".metric":"rmse",".estimator":"standard",".estimate":0.00292582593184027,"x":1472515200000,"y":0.00292582593184027},{"date":"20160831",".metric":"rmse",".estimator":"standard",".estimate":0.00197813491112788,"x":1472601600000,"y":0.00197813491112788},{"date":"20160901",".metric":"rmse",".estimator":"standard",".estimate":0.00104498304856213,"x":1472688000000,"y":0.00104498304856213},{"date":"20160902",".metric":"rmse",".estimator":"standard",".estimate":0.00263560444221595,"x":1472774400000,"y":0.00263560444221595},{"date":"20160906",".metric":"rmse",".estimator":"standard",".estimate":0.000738604008634933,"x":1473120000000,"y":0.000738604008634933},{"date":"20160907",".metric":"rmse",".estimator":"standard",".estimate":0.000751260854430958,"x":1473206400000,"y":0.000751260854430958},{"date":"20160908",".metric":"rmse",".estimator":"standard",".estimate":0.00136297845744431,"x":1473292800000,"y":0.00136297845744431},{"date":"20160909",".metric":"rmse",".estimator":"standard",".estimate":0.0171051525717296,"x":1473379200000,"y":0.0171051525717296},{"date":"20160912",".metric":"rmse",".estimator":"standard",".estimate":0.00876376658570854,"x":1473638400000,"y":0.00876376658570854},{"date":"20160913",".metric":"rmse",".estimator":"standard",".estimate":0.00740929065252387,"x":1473724800000,"y":0.00740929065252387},{"date":"20160914",".metric":"rmse",".estimator":"standard",".estimate":0.000285211158969392,"x":1473811200000,"y":0.000285211158969392},{"date":"20160915",".metric":"rmse",".estimator":"standard",".estimate":0.0041449808882293,"x":1473897600000,"y":0.0041449808882293},{"date":"20160916",".metric":"rmse",".estimator":"standard",".estimate":0.00137146529713595,"x":1473984000000,"y":0.00137146529713595},{"date":"20160919",".metric":"rmse",".estimator":"standard",".estimate":0.00151473219216688,"x":1474243200000,"y":0.00151473219216688},{"date":"20160920",".metric":"rmse",".estimator":"standard",".estimate":0.00279413362661776,"x":1474329600000,"y":0.00279413362661776},{"date":"20160921",".metric":"rmse",".estimator":"standard",".estimate":0.0041009394467729,"x":1474416000000,"y":0.0041009394467729},{"date":"20160922",".metric":"rmse",".estimator":"standard",".estimate":0.000115329292514692,"x":1474502400000,"y":0.000115329292514692},{"date":"20160923",".metric":"rmse",".estimator":"standard",".estimate":0.00122090649588156,"x":1474588800000,"y":0.00122090649588156},{"date":"20160926",".metric":"rmse",".estimator":"standard",".estimate":0.00314035666753001,"x":1474848000000,"y":0.00314035666753001},{"date":"20160927",".metric":"rmse",".estimator":"standard",".estimate":0.00317780922371701,"x":1474934400000,"y":0.00317780922371701},{"date":"20160928",".metric":"rmse",".estimator":"standard",".estimate":0.00482269019241134,"x":1475020800000,"y":0.00482269019241134},{"date":"20160929",".metric":"rmse",".estimator":"standard",".estimate":0.000353841625704364,"x":1475107200000,"y":0.000353841625704364},{"date":"20160930",".metric":"rmse",".estimator":"standard",".estimate":0.00314513811300813,"x":1475193600000,"y":0.00314513811300813},{"date":"20161003",".metric":"rmse",".estimator":"standard",".estimate":0.00135476344691113,"x":1475452800000,"y":0.00135476344691113},{"date":"20161004",".metric":"rmse",".estimator":"standard",".estimate":0.00550845521949076,"x":1475539200000,"y":0.00550845521949076},{"date":"20161005",".metric":"rmse",".estimator":"standard",".estimate":0.00354310101259846,"x":1475625600000,"y":0.00354310101259846},{"date":"20161006",".metric":"rmse",".estimator":"standard",".estimate":0.00113420274953901,"x":1475712000000,"y":0.00113420274953901},{"date":"20161007",".metric":"rmse",".estimator":"standard",".estimate":0.000533430818523165,"x":1475798400000,"y":0.000533430818523165},{"date":"20161010",".metric":"rmse",".estimator":"standard",".estimate":0.000994234577704675,"x":1476057600000,"y":0.000994234577704675},{"date":"20161011",".metric":"rmse",".estimator":"standard",".estimate":0.00544168235562282,"x":1476144000000,"y":0.00544168235562282},{"date":"20161012",".metric":"rmse",".estimator":"standard",".estimate":0.000794636237756211,"x":1476230400000,"y":0.000794636237756211},{"date":"20161013",".metric":"rmse",".estimator":"standard",".estimate":0.00141791000631537,"x":1476316800000,"y":0.00141791000631537},{"date":"20161014",".metric":"rmse",".estimator":"standard",".estimate":0.00167771170140043,"x":1476403200000,"y":0.00167771170140043},{"date":"20161017",".metric":"rmse",".estimator":"standard",".estimate":0.00128606775744151,"x":1476662400000,"y":0.00128606775744151},{"date":"20161018",".metric":"rmse",".estimator":"standard",".estimate":0.000696321255851035,"x":1476748800000,"y":0.000696321255851035},{"date":"20161019",".metric":"rmse",".estimator":"standard",".estimate":0.00106988912544378,"x":1476835200000,"y":0.00106988912544378},{"date":"20161020",".metric":"rmse",".estimator":"standard",".estimate":0.00195601674382502,"x":1476921600000,"y":0.00195601674382502},{"date":"20161021",".metric":"rmse",".estimator":"standard",".estimate":0.00151595275158728,"x":1477008000000,"y":0.00151595275158728},{"date":"20161024",".metric":"rmse",".estimator":"standard",".estimate":0.000254057182805282,"x":1477267200000,"y":0.000254057182805282},{"date":"20161025",".metric":"rmse",".estimator":"standard",".estimate":0.00269233868707742,"x":1477353600000,"y":0.00269233868707742},{"date":"20161026",".metric":"rmse",".estimator":"standard",".estimate":0.00250671120842745,"x":1477440000000,"y":0.00250671120842745},{"date":"20161027",".metric":"rmse",".estimator":"standard",".estimate":0.00295175598622102,"x":1477526400000,"y":0.00295175598622102},{"date":"20161028",".metric":"rmse",".estimator":"standard",".estimate":0.00292179672571274,"x":1477612800000,"y":0.00292179672571274},{"date":"20161031",".metric":"rmse",".estimator":"standard",".estimate":0.0021935037024321,"x":1477872000000,"y":0.0021935037024321},{"date":"20161101",".metric":"rmse",".estimator":"standard",".estimate":0.00434584529735663,"x":1477958400000,"y":0.00434584529735663},{"date":"20161102",".metric":"rmse",".estimator":"standard",".estimate":0.00188345490020019,"x":1478044800000,"y":0.00188345490020019},{"date":"20161103",".metric":"rmse",".estimator":"standard",".estimate":0.00288298366891027,"x":1478131200000,"y":0.00288298366891027},{"date":"20161104",".metric":"rmse",".estimator":"standard",".estimate":0.00432576156597594,"x":1478217600000,"y":0.00432576156597594},{"date":"20161107",".metric":"rmse",".estimator":"standard",".estimate":0.00750457832153818,"x":1478476800000,"y":0.00750457832153818},{"date":"20161108",".metric":"rmse",".estimator":"standard",".estimate":0.00143570899145871,"x":1478563200000,"y":0.00143570899145871},{"date":"20161109",".metric":"rmse",".estimator":"standard",".estimate":0.00872335368942581,"x":1478649600000,"y":0.00872335368942581},{"date":"20161110",".metric":"rmse",".estimator":"standard",".estimate":0.00130036512165486,"x":1478736000000,"y":0.00130036512165486},{"date":"20161111",".metric":"rmse",".estimator":"standard",".estimate":0.000188960934723348,"x":1478822400000,"y":0.000188960934723348},{"date":"20161114",".metric":"rmse",".estimator":"standard",".estimate":0.00374913684516115,"x":1479081600000,"y":0.00374913684516115},{"date":"20161115",".metric":"rmse",".estimator":"standard",".estimate":0.00218357763184909,"x":1479168000000,"y":0.00218357763184909},{"date":"20161116",".metric":"rmse",".estimator":"standard",".estimate":0.00238196498526625,"x":1479254400000,"y":0.00238196498526625},{"date":"20161117",".metric":"rmse",".estimator":"standard",".estimate":0.000938810312724679,"x":1479340800000,"y":0.000938810312724679},{"date":"20161118",".metric":"rmse",".estimator":"standard",".estimate":0.00130263392187033,"x":1479427200000,"y":0.00130263392187033},{"date":"20161121",".metric":"rmse",".estimator":"standard",".estimate":0.00114916938760177,"x":1479686400000,"y":0.00114916938760177},{"date":"20161122",".metric":"rmse",".estimator":"standard",".estimate":0.000501342327058653,"x":1479772800000,"y":0.000501342327058653},{"date":"20161123",".metric":"rmse",".estimator":"standard",".estimate":0.00190004576015157,"x":1479859200000,"y":0.00190004576015157},{"date":"20161125",".metric":"rmse",".estimator":"standard",".estimate":0.0012117898747769,"x":1480032000000,"y":0.0012117898747769},{"date":"20161128",".metric":"rmse",".estimator":"standard",".estimate":0.00142795890448438,"x":1480291200000,"y":0.00142795890448438},{"date":"20161129",".metric":"rmse",".estimator":"standard",".estimate":0.00268304157491472,"x":1480377600000,"y":0.00268304157491472},{"date":"20161130",".metric":"rmse",".estimator":"standard",".estimate":0.00148940303325588,"x":1480464000000,"y":0.00148940303325588},{"date":"20161201",".metric":"rmse",".estimator":"standard",".estimate":0.000437561464797262,"x":1480550400000,"y":0.000437561464797262},{"date":"20161202",".metric":"rmse",".estimator":"standard",".estimate":0.00181584516009029,"x":1480636800000,"y":0.00181584516009029},{"date":"20161205",".metric":"rmse",".estimator":"standard",".estimate":0.00350600101325258,"x":1480896000000,"y":0.00350600101325258},{"date":"20161206",".metric":"rmse",".estimator":"standard",".estimate":0.000751234918225076,"x":1480982400000,"y":0.000751234918225076},{"date":"20161207",".metric":"rmse",".estimator":"standard",".estimate":0.00288827459285574,"x":1481068800000,"y":0.00288827459285574},{"date":"20161208",".metric":"rmse",".estimator":"standard",".estimate":0.000465255667712638,"x":1481155200000,"y":0.000465255667712638},{"date":"20161209",".metric":"rmse",".estimator":"standard",".estimate":0.0041919487983297,"x":1481241600000,"y":0.0041919487983297},{"date":"20161212",".metric":"rmse",".estimator":"standard",".estimate":0.00207827225774742,"x":1481500800000,"y":0.00207827225774742},{"date":"20161213",".metric":"rmse",".estimator":"standard",".estimate":0.00397264757248993,"x":1481587200000,"y":0.00397264757248993},{"date":"20161214",".metric":"rmse",".estimator":"standard",".estimate":0.0022687501699622,"x":1481673600000,"y":0.0022687501699622},{"date":"20161215",".metric":"rmse",".estimator":"standard",".estimate":0.00397321228206072,"x":1481760000000,"y":0.00397321228206072},{"date":"20161216",".metric":"rmse",".estimator":"standard",".estimate":0.00106538653048893,"x":1481846400000,"y":0.00106538653048893},{"date":"20161219",".metric":"rmse",".estimator":"standard",".estimate":0.00352588010170244,"x":1482105600000,"y":0.00352588010170244},{"date":"20161220",".metric":"rmse",".estimator":"standard",".estimate":0.00167888204009601,"x":1482192000000,"y":0.00167888204009601},{"date":"20161221",".metric":"rmse",".estimator":"standard",".estimate":0.00175087061937348,"x":1482278400000,"y":0.00175087061937348},{"date":"20161222",".metric":"rmse",".estimator":"standard",".estimate":0.00185851169646374,"x":1482364800000,"y":0.00185851169646374},{"date":"20161223",".metric":"rmse",".estimator":"standard",".estimate":0.00175558003853289,"x":1482451200000,"y":0.00175558003853289},{"date":"20161227",".metric":"rmse",".estimator":"standard",".estimate":0.00195891830748184,"x":1482796800000,"y":0.00195891830748184},{"date":"20161228",".metric":"rmse",".estimator":"standard",".estimate":0.00396659321091997,"x":1482883200000,"y":0.00396659321091997},{"date":"20161229",".metric":"rmse",".estimator":"standard",".estimate":0.000725011466779303,"x":1482969600000,"y":0.000725011466779303},{"date":"20161230",".metric":"rmse",".estimator":"standard",".estimate":0.00176939826781894,"x":1483056000000,"y":0.00176939826781894},{"date":"20170103",".metric":"rmse",".estimator":"standard",".estimate":0.00220068192788672,"x":1483401600000,"y":0.00220068192788672},{"date":"20170104",".metric":"rmse",".estimator":"standard",".estimate":0.000642315768222612,"x":1483488000000,"y":0.000642315768222612},{"date":"20170105",".metric":"rmse",".estimator":"standard",".estimate":0.00561006432182169,"x":1483574400000,"y":0.00561006432182169},{"date":"20170106",".metric":"rmse",".estimator":"standard",".estimate":0.00157459366482963,"x":1483660800000,"y":0.00157459366482963},{"date":"20170109",".metric":"rmse",".estimator":"standard",".estimate":0.00192663683645288,"x":1483920000000,"y":0.00192663683645288},{"date":"20170110",".metric":"rmse",".estimator":"standard",".estimate":0.000578662280016658,"x":1484006400000,"y":0.000578662280016658},{"date":"20170111",".metric":"rmse",".estimator":"standard",".estimate":0.000800513789868231,"x":1484092800000,"y":0.000800513789868231},{"date":"20170112",".metric":"rmse",".estimator":"standard",".estimate":0.000500465543224869,"x":1484179200000,"y":0.000500465543224869},{"date":"20170113",".metric":"rmse",".estimator":"standard",".estimate":0.00103834851978848,"x":1484265600000,"y":0.00103834851978848},{"date":"20170117",".metric":"rmse",".estimator":"standard",".estimate":0.00383031801301423,"x":1484611200000,"y":0.00383031801301423},{"date":"20170118",".metric":"rmse",".estimator":"standard",".estimate":0.00286167031512328,"x":1484697600000,"y":0.00286167031512328},{"date":"20170119",".metric":"rmse",".estimator":"standard",".estimate":0.00328813540835988,"x":1484784000000,"y":0.00328813540835988},{"date":"20170120",".metric":"rmse",".estimator":"standard",".estimate":0.000876948327263606,"x":1484870400000,"y":0.000876948327263606},{"date":"20170123",".metric":"rmse",".estimator":"standard",".estimate":0.00156888230108249,"x":1485129600000,"y":0.00156888230108249},{"date":"20170124",".metric":"rmse",".estimator":"standard",".estimate":0.0032166393991214,"x":1485216000000,"y":0.0032166393991214},{"date":"20170125",".metric":"rmse",".estimator":"standard",".estimate":0.00217879048330384,"x":1485302400000,"y":0.00217879048330384},{"date":"20170126",".metric":"rmse",".estimator":"standard",".estimate":0.00208204732072695,"x":1485388800000,"y":0.00208204732072695},{"date":"20170127",".metric":"rmse",".estimator":"standard",".estimate":0.000599960726891122,"x":1485475200000,"y":0.000599960726891122},{"date":"20170130",".metric":"rmse",".estimator":"standard",".estimate":0.000950789978650351,"x":1485734400000,"y":0.000950789978650351},{"date":"20170131",".metric":"rmse",".estimator":"standard",".estimate":0.000507809258160204,"x":1485820800000,"y":0.000507809258160204},{"date":"20170201",".metric":"rmse",".estimator":"standard",".estimate":0.00110790893937926,"x":1485907200000,"y":0.00110790893937926},{"date":"20170202",".metric":"rmse",".estimator":"standard",".estimate":0.00220632295079328,"x":1485993600000,"y":0.00220632295079328},{"date":"20170203",".metric":"rmse",".estimator":"standard",".estimate":0.00190173837291212,"x":1486080000000,"y":0.00190173837291212},{"date":"20170206",".metric":"rmse",".estimator":"standard",".estimate":0.000283413885439519,"x":1486339200000,"y":0.000283413885439519},{"date":"20170207",".metric":"rmse",".estimator":"standard",".estimate":0.000110672502792694,"x":1486425600000,"y":0.000110672502792694},{"date":"20170208",".metric":"rmse",".estimator":"standard",".estimate":0.00105880448341185,"x":1486512000000,"y":0.00105880448341185},{"date":"20170209",".metric":"rmse",".estimator":"standard",".estimate":0.00166694677845329,"x":1486598400000,"y":0.00166694677845329},{"date":"20170210",".metric":"rmse",".estimator":"standard",".estimate":0.000220772382538859,"x":1486684800000,"y":0.000220772382538859},{"date":"20170213",".metric":"rmse",".estimator":"standard",".estimate":0.000249209801303179,"x":1486944000000,"y":0.000249209801303179},{"date":"20170214",".metric":"rmse",".estimator":"standard",".estimate":0.00431022211578958,"x":1487030400000,"y":0.00431022211578958},{"date":"20170215",".metric":"rmse",".estimator":"standard",".estimate":0.00103493868767278,"x":1487116800000,"y":0.00103493868767278},{"date":"20170216",".metric":"rmse",".estimator":"standard",".estimate":0.00205249723613406,"x":1487203200000,"y":0.00205249723613406},{"date":"20170217",".metric":"rmse",".estimator":"standard",".estimate":0.00350548252229491,"x":1487289600000,"y":0.00350548252229491},{"date":"20170221",".metric":"rmse",".estimator":"standard",".estimate":0.00119927435630245,"x":1487635200000,"y":0.00119927435630245},{"date":"20170222",".metric":"rmse",".estimator":"standard",".estimate":0.00130648950199573,"x":1487721600000,"y":0.00130648950199573},{"date":"20170223",".metric":"rmse",".estimator":"standard",".estimate":0.000390169797896131,"x":1487808000000,"y":0.000390169797896131},{"date":"20170224",".metric":"rmse",".estimator":"standard",".estimate":0.00299172731009862,"x":1487894400000,"y":0.00299172731009862},{"date":"20170227",".metric":"rmse",".estimator":"standard",".estimate":0.00296701477459117,"x":1488153600000,"y":0.00296701477459117},{"date":"20170228",".metric":"rmse",".estimator":"standard",".estimate":0.00207270660961641,"x":1488240000000,"y":0.00207270660961641},{"date":"20170301",".metric":"rmse",".estimator":"standard",".estimate":0.00365719805722696,"x":1488326400000,"y":0.00365719805722696},{"date":"20170302",".metric":"rmse",".estimator":"standard",".estimate":0.00223678026754732,"x":1488412800000,"y":0.00223678026754732},{"date":"20170303",".metric":"rmse",".estimator":"standard",".estimate":0.00435131557483537,"x":1488499200000,"y":0.00435131557483537},{"date":"20170306",".metric":"rmse",".estimator":"standard",".estimate":0.00028169879948786,"x":1488758400000,"y":0.00028169879948786},{"date":"20170307",".metric":"rmse",".estimator":"standard",".estimate":0.00113239477279367,"x":1488844800000,"y":0.00113239477279367},{"date":"20170308",".metric":"rmse",".estimator":"standard",".estimate":0.00086881789580495,"x":1488931200000,"y":0.00086881789580495},{"date":"20170309",".metric":"rmse",".estimator":"standard",".estimate":0.000789334630958708,"x":1489017600000,"y":0.000789334630958708},{"date":"20170310",".metric":"rmse",".estimator":"standard",".estimate":0.00118177999223406,"x":1489104000000,"y":0.00118177999223406},{"date":"20170313",".metric":"rmse",".estimator":"standard",".estimate":0.000374107274594283,"x":1489363200000,"y":0.000374107274594283},{"date":"20170314",".metric":"rmse",".estimator":"standard",".estimate":0.000978798772025025,"x":1489449600000,"y":0.000978798772025025},{"date":"20170315",".metric":"rmse",".estimator":"standard",".estimate":0.00267634984239615,"x":1489536000000,"y":0.00267634984239615},{"date":"20170316",".metric":"rmse",".estimator":"standard",".estimate":0.00548569083226419,"x":1489622400000,"y":0.00548569083226419},{"date":"20170317",".metric":"rmse",".estimator":"standard",".estimate":0.000924198492296146,"x":1489708800000,"y":0.000924198492296146},{"date":"20170320",".metric":"rmse",".estimator":"standard",".estimate":0.00049963095661782,"x":1489968000000,"y":0.00049963095661782},{"date":"20170321",".metric":"rmse",".estimator":"standard",".estimate":0.00733028421635421,"x":1490054400000,"y":0.00733028421635421},{"date":"20170322",".metric":"rmse",".estimator":"standard",".estimate":0.00131465365401076,"x":1490140800000,"y":0.00131465365401076},{"date":"20170323",".metric":"rmse",".estimator":"standard",".estimate":0.00131953505018048,"x":1490227200000,"y":0.00131953505018048},{"date":"20170324",".metric":"rmse",".estimator":"standard",".estimate":0.00138218647588729,"x":1490313600000,"y":0.00138218647588729},{"date":"20170327",".metric":"rmse",".estimator":"standard",".estimate":0.00191462925483187,"x":1490572800000,"y":0.00191462925483187},{"date":"20170328",".metric":"rmse",".estimator":"standard",".estimate":0.00273949165867805,"x":1490659200000,"y":0.00273949165867805},{"date":"20170329",".metric":"rmse",".estimator":"standard",".estimate":0.000822054788003299,"x":1490745600000,"y":0.000822054788003299},{"date":"20170330",".metric":"rmse",".estimator":"standard",".estimate":0.00322639817543668,"x":1490832000000,"y":0.00322639817543668},{"date":"20170331",".metric":"rmse",".estimator":"standard",".estimate":0.00068188422327266,"x":1490918400000,"y":0.00068188422327266},{"date":"20170403",".metric":"rmse",".estimator":"standard",".estimate":0.000544672589532932,"x":1491177600000,"y":0.000544672589532932},{"date":"20170404",".metric":"rmse",".estimator":"standard",".estimate":0.000854233252933708,"x":1491264000000,"y":0.000854233252933708},{"date":"20170405",".metric":"rmse",".estimator":"standard",".estimate":0.00100477045596371,"x":1491350400000,"y":0.00100477045596371},{"date":"20170406",".metric":"rmse",".estimator":"standard",".estimate":0.00215866438382077,"x":1491436800000,"y":0.00215866438382077},{"date":"20170407",".metric":"rmse",".estimator":"standard",".estimate":0.000865894966884134,"x":1491523200000,"y":0.000865894966884134},{"date":"20170410",".metric":"rmse",".estimator":"standard",".estimate":0.00118363053690387,"x":1491782400000,"y":0.00118363053690387},{"date":"20170411",".metric":"rmse",".estimator":"standard",".estimate":0.000494117441836051,"x":1491868800000,"y":0.000494117441836051},{"date":"20170412",".metric":"rmse",".estimator":"standard",".estimate":0.00380490458823101,"x":1491955200000,"y":0.00380490458823101},{"date":"20170413",".metric":"rmse",".estimator":"standard",".estimate":0.000117661808128848,"x":1492041600000,"y":0.000117661808128848},{"date":"20170417",".metric":"rmse",".estimator":"standard",".estimate":0.0049480019472521,"x":1492387200000,"y":0.0049480019472521},{"date":"20170418",".metric":"rmse",".estimator":"standard",".estimate":0.00106821779106287,"x":1492473600000,"y":0.00106821779106287},{"date":"20170419",".metric":"rmse",".estimator":"standard",".estimate":0.000438994283468146,"x":1492560000000,"y":0.000438994283468146},{"date":"20170420",".metric":"rmse",".estimator":"standard",".estimate":0.00253151193669525,"x":1492646400000,"y":0.00253151193669525},{"date":"20170421",".metric":"rmse",".estimator":"standard",".estimate":0.00166868348772587,"x":1492732800000,"y":0.00166868348772587},{"date":"20170424",".metric":"rmse",".estimator":"standard",".estimate":9.66476184357384e06,"x":1492992000000,"y":9.66476184357384e06},{"date":"20170425",".metric":"rmse",".estimator":"standard",".estimate":0.000600998687300889,"x":1493078400000,"y":0.000600998687300889},{"date":"20170426",".metric":"rmse",".estimator":"standard",".estimate":0.000771834275634839,"x":1493164800000,"y":0.000771834275634839},{"date":"20170427",".metric":"rmse",".estimator":"standard",".estimate":0.0032970150530321,"x":1493251200000,"y":0.0032970150530321},{"date":"20170428",".metric":"rmse",".estimator":"standard",".estimate":0.000474000982886702,"x":1493337600000,"y":0.000474000982886702},{"date":"20170501",".metric":"rmse",".estimator":"standard",".estimate":0.00209574437391007,"x":1493596800000,"y":0.00209574437391007},{"date":"20170502",".metric":"rmse",".estimator":"standard",".estimate":0.00316517891181679,"x":1493683200000,"y":0.00316517891181679},{"date":"20170503",".metric":"rmse",".estimator":"standard",".estimate":0.000206302215708535,"x":1493769600000,"y":0.000206302215708535},{"date":"20170504",".metric":"rmse",".estimator":"standard",".estimate":0.00210164345440232,"x":1493856000000,"y":0.00210164345440232},{"date":"20170505",".metric":"rmse",".estimator":"standard",".estimate":0.00154657108301496,"x":1493942400000,"y":0.00154657108301496},{"date":"20170508",".metric":"rmse",".estimator":"standard",".estimate":0.0017768859329387,"x":1494201600000,"y":0.0017768859329387},{"date":"20170509",".metric":"rmse",".estimator":"standard",".estimate":0.000473959293859324,"x":1494288000000,"y":0.000473959293859324},{"date":"20170510",".metric":"rmse",".estimator":"standard",".estimate":0.000750032628056844,"x":1494374400000,"y":0.000750032628056844},{"date":"20170511",".metric":"rmse",".estimator":"standard",".estimate":0.00156257968148124,"x":1494460800000,"y":0.00156257968148124},{"date":"20170512",".metric":"rmse",".estimator":"standard",".estimate":0.00211340142295421,"x":1494547200000,"y":0.00211340142295421},{"date":"20170515",".metric":"rmse",".estimator":"standard",".estimate":0.000190435030948505,"x":1494806400000,"y":0.000190435030948505},{"date":"20170516",".metric":"rmse",".estimator":"standard",".estimate":0.00220134627792209,"x":1494892800000,"y":0.00220134627792209},{"date":"20170517",".metric":"rmse",".estimator":"standard",".estimate":0.00920513582953843,"x":1494979200000,"y":0.00920513582953843},{"date":"20170518",".metric":"rmse",".estimator":"standard",".estimate":0.00406630081956308,"x":1495065600000,"y":0.00406630081956308},{"date":"20170519",".metric":"rmse",".estimator":"standard",".estimate":6.73730041831069e05,"x":1495152000000,"y":6.73730041831069e05},{"date":"20170522",".metric":"rmse",".estimator":"standard",".estimate":0.00395531164381844,"x":1495411200000,"y":0.00395531164381844},{"date":"20170523",".metric":"rmse",".estimator":"standard",".estimate":0.00194058527039616,"x":1495497600000,"y":0.00194058527039616},{"date":"20170524",".metric":"rmse",".estimator":"standard",".estimate":0.00164017589020188,"x":1495584000000,"y":0.00164017589020188},{"date":"20170525",".metric":"rmse",".estimator":"standard",".estimate":0.00473730155666312,"x":1495670400000,"y":0.00473730155666312},{"date":"20170526",".metric":"rmse",".estimator":"standard",".estimate":0.00184065102288655,"x":1495756800000,"y":0.00184065102288655},{"date":"20170530",".metric":"rmse",".estimator":"standard",".estimate":0.00250601739829797,"x":1496102400000,"y":0.00250601739829797},{"date":"20170531",".metric":"rmse",".estimator":"standard",".estimate":0.000531071324415682,"x":1496188800000,"y":0.000531071324415682},{"date":"20170601",".metric":"rmse",".estimator":"standard",".estimate":0.00315499368568156,"x":1496275200000,"y":0.00315499368568156},{"date":"20170602",".metric":"rmse",".estimator":"standard",".estimate":0.00149827419496544,"x":1496361600000,"y":0.00149827419496544},{"date":"20170605",".metric":"rmse",".estimator":"standard",".estimate":0.000224562299631777,"x":1496620800000,"y":0.000224562299631777},{"date":"20170606",".metric":"rmse",".estimator":"standard",".estimate":0.00490647578662081,"x":1496707200000,"y":0.00490647578662081},{"date":"20170607",".metric":"rmse",".estimator":"standard",".estimate":0.00181778717567682,"x":1496793600000,"y":0.00181778717567682},{"date":"20170608",".metric":"rmse",".estimator":"standard",".estimate":0.00150359110204811,"x":1496880000000,"y":0.00150359110204811},{"date":"20170609",".metric":"rmse",".estimator":"standard",".estimate":0.000589126233841853,"x":1496966400000,"y":0.000589126233841853},{"date":"20170612",".metric":"rmse",".estimator":"standard",".estimate":0.000785267452530611,"x":1497225600000,"y":0.000785267452530611},{"date":"20170613",".metric":"rmse",".estimator":"standard",".estimate":0.00032663222694752,"x":1497312000000,"y":0.00032663222694752},{"date":"20170614",".metric":"rmse",".estimator":"standard",".estimate":0.00193962443631849,"x":1497398400000,"y":0.00193962443631849},{"date":"20170615",".metric":"rmse",".estimator":"standard",".estimate":0.00178480116194402,"x":1497484800000,"y":0.00178480116194402},{"date":"20170616",".metric":"rmse",".estimator":"standard",".estimate":0.00227838045911216,"x":1497571200000,"y":0.00227838045911216},{"date":"20170619",".metric":"rmse",".estimator":"standard",".estimate":0.00281397691092402,"x":1497830400000,"y":0.00281397691092402},{"date":"20170620",".metric":"rmse",".estimator":"standard",".estimate":0.00276198862616263,"x":1497916800000,"y":0.00276198862616263},{"date":"20170621",".metric":"rmse",".estimator":"standard",".estimate":0.000192884361855274,"x":1498003200000,"y":0.000192884361855274},{"date":"20170622",".metric":"rmse",".estimator":"standard",".estimate":0.000259720774694152,"x":1498089600000,"y":0.000259720774694152},{"date":"20170623",".metric":"rmse",".estimator":"standard",".estimate":0.000557558234526353,"x":1498176000000,"y":0.000557558234526353},{"date":"20170626",".metric":"rmse",".estimator":"standard",".estimate":0.000126300272981984,"x":1498435200000,"y":0.000126300272981984},{"date":"20170627",".metric":"rmse",".estimator":"standard",".estimate":0.00561618579486439,"x":1498521600000,"y":0.00561618579486439},{"date":"20170628",".metric":"rmse",".estimator":"standard",".estimate":0.0020236818136823,"x":1498608000000,"y":0.0020236818136823},{"date":"20170629",".metric":"rmse",".estimator":"standard",".estimate":0.00479603402518207,"x":1498694400000,"y":0.00479603402518207},{"date":"20170630",".metric":"rmse",".estimator":"standard",".estimate":0.0039170890474173,"x":1498780800000,"y":0.0039170890474173},{"date":"20170703",".metric":"rmse",".estimator":"standard",".estimate":0.00330775801804971,"x":1499040000000,"y":0.00330775801804971},{"date":"20170705",".metric":"rmse",".estimator":"standard",".estimate":0.000923877973580159,"x":1499212800000,"y":0.000923877973580159},{"date":"20170706",".metric":"rmse",".estimator":"standard",".estimate":0.00455581805067294,"x":1499299200000,"y":0.00455581805067294},{"date":"20170707",".metric":"rmse",".estimator":"standard",".estimate":0.00548688562762336,"x":1499385600000,"y":0.00548688562762336},{"date":"20170710",".metric":"rmse",".estimator":"standard",".estimate":0.000429896269046332,"x":1499644800000,"y":0.000429896269046332},{"date":"20170711",".metric":"rmse",".estimator":"standard",".estimate":0.00160472256570825,"x":1499731200000,"y":0.00160472256570825},{"date":"20170712",".metric":"rmse",".estimator":"standard",".estimate":0.00036455733410835,"x":1499817600000,"y":0.00036455733410835},{"date":"20170713",".metric":"rmse",".estimator":"standard",".estimate":0.00122078887115348,"x":1499904000000,"y":0.00122078887115348},{"date":"20170714",".metric":"rmse",".estimator":"standard",".estimate":0.00027744181369957,"x":1499990400000,"y":0.00027744181369957},{"date":"20170717",".metric":"rmse",".estimator":"standard",".estimate":0.000130179465291443,"x":1500249600000,"y":0.000130179465291443},{"date":"20170718",".metric":"rmse",".estimator":"standard",".estimate":0.000304545761030702,"x":1500336000000,"y":0.000304545761030702},{"date":"20170719",".metric":"rmse",".estimator":"standard",".estimate":0.000460089510473261,"x":1500422400000,"y":0.000460089510473261},{"date":"20170720",".metric":"rmse",".estimator":"standard",".estimate":0.000350007543751075,"x":1500508800000,"y":0.000350007543751075},{"date":"20170721",".metric":"rmse",".estimator":"standard",".estimate":0.00110942433359531,"x":1500595200000,"y":0.00110942433359531},{"date":"20170724",".metric":"rmse",".estimator":"standard",".estimate":0.000574869765522386,"x":1500854400000,"y":0.000574869765522386},{"date":"20170725",".metric":"rmse",".estimator":"standard",".estimate":0.00222648739942126,"x":1500940800000,"y":0.00222648739942126},{"date":"20170726",".metric":"rmse",".estimator":"standard",".estimate":0.000724593735381417,"x":1501027200000,"y":0.000724593735381417},{"date":"20170727",".metric":"rmse",".estimator":"standard",".estimate":0.000590437231668881,"x":1501113600000,"y":0.000590437231668881},{"date":"20170728",".metric":"rmse",".estimator":"standard",".estimate":0.000418764452709116,"x":1501200000000,"y":0.000418764452709116},{"date":"20170731",".metric":"rmse",".estimator":"standard",".estimate":0.00112373658911328,"x":1501459200000,"y":0.00112373658911328},{"date":"20170801",".metric":"rmse",".estimator":"standard",".estimate":0.00102780149433163,"x":1501545600000,"y":0.00102780149433163},{"date":"20170802",".metric":"rmse",".estimator":"standard",".estimate":0.0002863885935313,"x":1501632000000,"y":0.0002863885935313},{"date":"20170803",".metric":"rmse",".estimator":"standard",".estimate":0.00216380496530155,"x":1501718400000,"y":0.00216380496530155},{"date":"20170804",".metric":"rmse",".estimator":"standard",".estimate":0.0020390862850439,"x":1501804800000,"y":0.0020390862850439},{"date":"20170807",".metric":"rmse",".estimator":"standard",".estimate":0.00144022400144353,"x":1502064000000,"y":0.00144022400144353},{"date":"20170808",".metric":"rmse",".estimator":"standard",".estimate":0.00178102609248742,"x":1502150400000,"y":0.00178102609248742},{"date":"20170809",".metric":"rmse",".estimator":"standard",".estimate":0.00173423072868356,"x":1502236800000,"y":0.00173423072868356},{"date":"20170810",".metric":"rmse",".estimator":"standard",".estimate":0.00585601391382766,"x":1502323200000,"y":0.00585601391382766},{"date":"20170811",".metric":"rmse",".estimator":"standard",".estimate":0.0025707399242942,"x":1502409600000,"y":0.0025707399242942},{"date":"20170814",".metric":"rmse",".estimator":"standard",".estimate":0.00278565297348008,"x":1502668800000,"y":0.00278565297348008},{"date":"20170815",".metric":"rmse",".estimator":"standard",".estimate":0.000598661458612812,"x":1502755200000,"y":0.000598661458612812},{"date":"20170816",".metric":"rmse",".estimator":"standard",".estimate":0.00305110556814486,"x":1502841600000,"y":0.00305110556814486},{"date":"20170817",".metric":"rmse",".estimator":"standard",".estimate":0.00575431425787176,"x":1502928000000,"y":0.00575431425787176},{"date":"20170818",".metric":"rmse",".estimator":"standard",".estimate":0.000562189503631827,"x":1503014400000,"y":0.000562189503631827},{"date":"20170821",".metric":"rmse",".estimator":"standard",".estimate":0.00136023893386477,"x":1503273600000,"y":0.00136023893386477},{"date":"20170822",".metric":"rmse",".estimator":"standard",".estimate":0.00249350862483187,"x":1503360000000,"y":0.00249350862483187},{"date":"20170823",".metric":"rmse",".estimator":"standard",".estimate":0.00208495474240162,"x":1503446400000,"y":0.00208495474240162},{"date":"20170824",".metric":"rmse",".estimator":"standard",".estimate":0.00147841572396225,"x":1503532800000,"y":0.00147841572396225},{"date":"20170825",".metric":"rmse",".estimator":"standard",".estimate":0.00202125503968285,"x":1503619200000,"y":0.00202125503968285},{"date":"20170828",".metric":"rmse",".estimator":"standard",".estimate":0.000131985749159282,"x":1503878400000,"y":0.000131985749159282},{"date":"20170829",".metric":"rmse",".estimator":"standard",".estimate":0.00391999455458316,"x":1503964800000,"y":0.00391999455458316},{"date":"20170830",".metric":"rmse",".estimator":"standard",".estimate":0.00178521604081493,"x":1504051200000,"y":0.00178521604081493},{"date":"20170831",".metric":"rmse",".estimator":"standard",".estimate":0.000244439546468598,"x":1504137600000,"y":0.000244439546468598},{"date":"20170901",".metric":"rmse",".estimator":"standard",".estimate":0.000955465062412176,"x":1504224000000,"y":0.000955465062412176},{"date":"20170905",".metric":"rmse",".estimator":"standard",".estimate":0.00506874671983177,"x":1504569600000,"y":0.00506874671983177},{"date":"20170906",".metric":"rmse",".estimator":"standard",".estimate":0.00244980141200023,"x":1504656000000,"y":0.00244980141200023},{"date":"20170907",".metric":"rmse",".estimator":"standard",".estimate":0.00299945537967095,"x":1504742400000,"y":0.00299945537967095},{"date":"20170908",".metric":"rmse",".estimator":"standard",".estimate":0.00124741212445197,"x":1504828800000,"y":0.00124741212445197},{"date":"20170911",".metric":"rmse",".estimator":"standard",".estimate":0.00173984561777132,"x":1505088000000,"y":0.00173984561777132},{"date":"20170912",".metric":"rmse",".estimator":"standard",".estimate":0.00208563767344366,"x":1505174400000,"y":0.00208563767344366},{"date":"20170913",".metric":"rmse",".estimator":"standard",".estimate":0.00208755873836036,"x":1505260800000,"y":0.00208755873836036},{"date":"20170914",".metric":"rmse",".estimator":"standard",".estimate":5.35492512113534e05,"x":1505347200000,"y":5.35492512113534e05},{"date":"20170915",".metric":"rmse",".estimator":"standard",".estimate":0.000942207369800087,"x":1505433600000,"y":0.000942207369800087},{"date":"20170918",".metric":"rmse",".estimator":"standard",".estimate":0.00143626119319263,"x":1505692800000,"y":0.00143626119319263},{"date":"20170919",".metric":"rmse",".estimator":"standard",".estimate":0.000323798894153753,"x":1505779200000,"y":0.000323798894153753},{"date":"20170920",".metric":"rmse",".estimator":"standard",".estimate":0.00155049944364725,"x":1505865600000,"y":0.00155049944364725},{"date":"20170921",".metric":"rmse",".estimator":"standard",".estimate":0.00176376363798185,"x":1505952000000,"y":0.00176376363798185},{"date":"20170922",".metric":"rmse",".estimator":"standard",".estimate":0.000415792730767708,"x":1506038400000,"y":0.000415792730767708},{"date":"20170925",".metric":"rmse",".estimator":"standard",".estimate":0.00254524556426304,"x":1506297600000,"y":0.00254524556426304},{"date":"20170926",".metric":"rmse",".estimator":"standard",".estimate":0.00111846677080901,"x":1506384000000,"y":0.00111846677080901},{"date":"20170927",".metric":"rmse",".estimator":"standard",".estimate":0.00236947032196308,"x":1506470400000,"y":0.00236947032196308},{"date":"20170928",".metric":"rmse",".estimator":"standard",".estimate":0.0013698886857053,"x":1506556800000,"y":0.0013698886857053},{"date":"20170929",".metric":"rmse",".estimator":"standard",".estimate":0.00129241622887146,"x":1506643200000,"y":0.00129241622887146},{"date":"20171002",".metric":"rmse",".estimator":"standard",".estimate":0.00351452183004398,"x":1506902400000,"y":0.00351452183004398},{"date":"20171003",".metric":"rmse",".estimator":"standard",".estimate":0.00021945426661678,"x":1506988800000,"y":0.00021945426661678},{"date":"20171004",".metric":"rmse",".estimator":"standard",".estimate":0.000979514740696812,"x":1507075200000,"y":0.000979514740696812},{"date":"20171005",".metric":"rmse",".estimator":"standard",".estimate":0.00169205866343378,"x":1507161600000,"y":0.00169205866343378},{"date":"20171006",".metric":"rmse",".estimator":"standard",".estimate":0.00103678310109957,"x":1507248000000,"y":0.00103678310109957},{"date":"20171009",".metric":"rmse",".estimator":"standard",".estimate":0.00264705975475339,"x":1507507200000,"y":0.00264705975475339},{"date":"20171010",".metric":"rmse",".estimator":"standard",".estimate":0.00119099648284331,"x":1507593600000,"y":0.00119099648284331},{"date":"20171011",".metric":"rmse",".estimator":"standard",".estimate":0.000569358577152982,"x":1507680000000,"y":0.000569358577152982},{"date":"20171012",".metric":"rmse",".estimator":"standard",".estimate":0.00132402101517672,"x":1507766400000,"y":0.00132402101517672},{"date":"20171013",".metric":"rmse",".estimator":"standard",".estimate":0.000432152619367148,"x":1507852800000,"y":0.000432152619367148},{"date":"20171016",".metric":"rmse",".estimator":"standard",".estimate":0.00121290820282521,"x":1508112000000,"y":0.00121290820282521},{"date":"20171017",".metric":"rmse",".estimator":"standard",".estimate":0.000636049496482721,"x":1508198400000,"y":0.000636049496482721},{"date":"20171018",".metric":"rmse",".estimator":"standard",".estimate":0.000316197071631185,"x":1508284800000,"y":0.000316197071631185},{"date":"20171019",".metric":"rmse",".estimator":"standard",".estimate":0.000301573819444778,"x":1508371200000,"y":0.000301573819444778},{"date":"20171020",".metric":"rmse",".estimator":"standard",".estimate":0.00406596387415966,"x":1508457600000,"y":0.00406596387415966},{"date":"20171023",".metric":"rmse",".estimator":"standard",".estimate":0.00355686147299521,"x":1508716800000,"y":0.00355686147299521},{"date":"20171024",".metric":"rmse",".estimator":"standard",".estimate":0.00136140929244226,"x":1508803200000,"y":0.00136140929244226},{"date":"20171025",".metric":"rmse",".estimator":"standard",".estimate":0.00165174014030913,"x":1508889600000,"y":0.00165174014030913},{"date":"20171026",".metric":"rmse",".estimator":"standard",".estimate":0.00100238682889144,"x":1508976000000,"y":0.00100238682889144},{"date":"20171027",".metric":"rmse",".estimator":"standard",".estimate":0.00141794327172736,"x":1509062400000,"y":0.00141794327172736},{"date":"20171030",".metric":"rmse",".estimator":"standard",".estimate":0.00324552277266412,"x":1509321600000,"y":0.00324552277266412},{"date":"20171031",".metric":"rmse",".estimator":"standard",".estimate":0.00087199540983986,"x":1509408000000,"y":0.00087199540983986},{"date":"20171101",".metric":"rmse",".estimator":"standard",".estimate":0.00187456571156308,"x":1509494400000,"y":0.00187456571156308},{"date":"20171102",".metric":"rmse",".estimator":"standard",".estimate":0.000879943303785382,"x":1509580800000,"y":0.000879943303785382},{"date":"20171103",".metric":"rmse",".estimator":"standard",".estimate":0.000436694115392702,"x":1509667200000,"y":0.000436694115392702},{"date":"20171106",".metric":"rmse",".estimator":"standard",".estimate":0.00136881117908498,"x":1509926400000,"y":0.00136881117908498},{"date":"20171107",".metric":"rmse",".estimator":"standard",".estimate":0.00241278001359499,"x":1510012800000,"y":0.00241278001359499},{"date":"20171108",".metric":"rmse",".estimator":"standard",".estimate":0.00046799600464614,"x":1510099200000,"y":0.00046799600464614},{"date":"20171109",".metric":"rmse",".estimator":"standard",".estimate":0.000464882341365728,"x":1510185600000,"y":0.000464882341365728},{"date":"20171110",".metric":"rmse",".estimator":"standard",".estimate":0.000435266335629412,"x":1510272000000,"y":0.000435266335629412},{"date":"20171113",".metric":"rmse",".estimator":"standard",".estimate":0.000779059815034458,"x":1510531200000,"y":0.000779059815034458},{"date":"20171114",".metric":"rmse",".estimator":"standard",".estimate":0.00242723716408704,"x":1510617600000,"y":0.00242723716408704},{"date":"20171115",".metric":"rmse",".estimator":"standard",".estimate":0.000942307511846099,"x":1510704000000,"y":0.000942307511846099},{"date":"20171116",".metric":"rmse",".estimator":"standard",".estimate":0.00322323632467401,"x":1510790400000,"y":0.00322323632467401},{"date":"20171117",".metric":"rmse",".estimator":"standard",".estimate":0.00143858562566698,"x":1510876800000,"y":0.00143858562566698},{"date":"20171120",".metric":"rmse",".estimator":"standard",".estimate":0.00148140095216353,"x":1511136000000,"y":0.00148140095216353},{"date":"20171121",".metric":"rmse",".estimator":"standard",".estimate":0.000342712528964379,"x":1511222400000,"y":0.000342712528964379},{"date":"20171122",".metric":"rmse",".estimator":"standard",".estimate":0.00129852636681049,"x":1511308800000,"y":0.00129852636681049},{"date":"20171124",".metric":"rmse",".estimator":"standard",".estimate":0.000523981551319308,"x":1511481600000,"y":0.000523981551319308},{"date":"20171127",".metric":"rmse",".estimator":"standard",".estimate":0.000109118405659181,"x":1511740800000,"y":0.000109118405659181},{"date":"20171128",".metric":"rmse",".estimator":"standard",".estimate":0.00425595935550045,"x":1511827200000,"y":0.00425595935550045},{"date":"20171129",".metric":"rmse",".estimator":"standard",".estimate":0.000812053876236767,"x":1511913600000,"y":0.000812053876236767},{"date":"20171130",".metric":"rmse",".estimator":"standard",".estimate":0.00567167634956679,"x":1512000000000,"y":0.00567167634956679},{"date":"20171201",".metric":"rmse",".estimator":"standard",".estimate":0.00131648503522855,"x":1512086400000,"y":0.00131648503522855},{"date":"20171204",".metric":"rmse",".estimator":"standard",".estimate":0.000202800762239206,"x":1512345600000,"y":0.000202800762239206},{"date":"20171205",".metric":"rmse",".estimator":"standard",".estimate":0.000220439983437353,"x":1512432000000,"y":0.000220439983437353},{"date":"20171206",".metric":"rmse",".estimator":"standard",".estimate":0.000334522204095748,"x":1512518400000,"y":0.000334522204095748},{"date":"20171207",".metric":"rmse",".estimator":"standard",".estimate":0.00118911941047771,"x":1512604800000,"y":0.00118911941047771},{"date":"20171208",".metric":"rmse",".estimator":"standard",".estimate":0.00239770292034793,"x":1512691200000,"y":0.00239770292034793},{"date":"20171211",".metric":"rmse",".estimator":"standard",".estimate":0.000404106695356842,"x":1512950400000,"y":0.000404106695356842},{"date":"20171212",".metric":"rmse",".estimator":"standard",".estimate":0.000860339700663732,"x":1513036800000,"y":0.000860339700663732},{"date":"20171213",".metric":"rmse",".estimator":"standard",".estimate":0.000816992792503522,"x":1513123200000,"y":0.000816992792503522},{"date":"20171214",".metric":"rmse",".estimator":"standard",".estimate":0.000475246264804703,"x":1513209600000,"y":0.000475246264804703},{"date":"20171215",".metric":"rmse",".estimator":"standard",".estimate":0.00510855365225753,"x":1513296000000,"y":0.00510855365225753},{"date":"20171218",".metric":"rmse",".estimator":"standard",".estimate":0.000761697975464818,"x":1513555200000,"y":0.000761697975464818},{"date":"20171219",".metric":"rmse",".estimator":"standard",".estimate":0.00324328997442093,"x":1513641600000,"y":0.00324328997442093},{"date":"20171220",".metric":"rmse",".estimator":"standard",".estimate":0.00139695394569652,"x":1513728000000,"y":0.00139695394569652},{"date":"20171221",".metric":"rmse",".estimator":"standard",".estimate":5.61810734120722e05,"x":1513814400000,"y":5.61810734120722e05},{"date":"20171222",".metric":"rmse",".estimator":"standard",".estimate":0.00135185904279269,"x":1513900800000,"y":0.00135185904279269},{"date":"20171226",".metric":"rmse",".estimator":"standard",".estimate":0.000284135320262809,"x":1514246400000,"y":0.000284135320262809},{"date":"20171227",".metric":"rmse",".estimator":"standard",".estimate":0.000798608648818393,"x":1514332800000,"y":0.000798608648818393},{"date":"20171228",".metric":"rmse",".estimator":"standard",".estimate":0.000531259849385336,"x":1514419200000,"y":0.000531259849385336},{"date":"20171229",".metric":"rmse",".estimator":"standard",".estimate":0.00227136567770183,"x":1514505600000,"y":0.00227136567770183}],"type":"scatter"}],"xAxis":{"type":"datetime","title":{"text":"date"},"categories":null}},"theme":{"chart":{"backgroundColor":"transparent"}},"conf_opts":{"global":{"Date":null,"VMLRadialGradientURL":"http =//code.highcharts.com/list(version)/gfx/vmlradialgradient.png","canvasToolsURL":"http =//code.highcharts.com/list(version)/modules/canvastools.js","getTimezoneOffset":null,"timezoneOffset":0,"useUTC":true},"lang":{"contextButtonTitle":"Chart context menu","decimalPoint":".","downloadJPEG":"Download JPEG image","downloadPDF":"Download PDF document","downloadPNG":"Download PNG image","downloadSVG":"Download SVG vector image","drillUpText":"Back to {series.name}","invalidDate":null,"loading":"Loading...","months":["January","February","March","April","May","June","July","August","September","October","November","December"],"noData":"No data to display","numericSymbols":["k","M","G","T","P","E"],"printChart":"Print chart","resetZoom":"Reset zoom","resetZoomTitle":"Reset zoom level 1:1","shortMonths":["Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"],"thousandsSep":" ","weekdays":["Sunday","Monday","Tuesday","Wednesday","Thursday","Friday","Saturday"]}},"type":"chart","fonts":[],"debug":false},"evals":[],"jsHooks":[]}
It looks like our RMSE is relatively stable, except for a period in mid to late 2015.
The amazing power of parsnip is how efficiently we can toggle to another random forest engine. Let’s suppose we wished to use the randomForest package instead of ranger. Here’s how we could reconfigure our previous work to use a different engine.
First, we’ll load up the randomForest package, because we need to load the package in order to use it as our engine. Then, we make one tweak to the original ranger_rf_regress function, by changing set_engine("ranger") to set_engine("randomForest"). That’s all, and we’re now running a random forest model using a different package.
library(randomForest) randomForest_rf_regress < function(mtry = 3, trees = 5, split){ analysis_set_rf < analysis(split) model < rand_forest(mtry = mtry, trees = trees) %>% set_engine("randomForest") %>% fit(daily_returns ~ MKT + SMB + HML + RMW + CMA, data = analysis_set_rf) assessment_set_rf < assessment(split) assessment_set_rf %>% select(date, daily_returns) %>% mutate(.pred = unlist(predict(model, new_data = assessment_set_rf))) %>% select(date, daily_returns, .pred) }We now have a new function called randomForest_rf_regress() that uses randomForest as the engine for our model and can use the same code scaffolding to run that model on our 1159 splits.
randomForest_results < map_df(.x = rolling_origin_spy_2013_2017$splits, ~randomForest_rf_regress(mtry = 3, trees = 100, split = .x)) randomForest_results %>% head() # A tibble: 6 x 4 # Groups: asset [1] asset date daily_returns .pred 1 SPY 20130528 0.00597 0.00609 2 SPY 20130529 0.00652 0.00438 3 SPY 20130530 0.00369 0.00597 4 SPY 20130531 0.0145 0.00987 5 SPY 20130603 0.00549 0.00134 6 SPY 20130604 0.00482 0.00118And we can use the same yardstick code to extract the RMSE.
randomForest_results %>% group_by(date) %>% rmse(daily_returns, .pred) %>% summarise(avg_rmse = mean(.estimate)) # A tibble: 1 x 1 avg_rmse 1 0.00252There’s a lot more to explore in the parsnip package and the tidymodels collection. See you next time when we’ll get into some classification!
Wait: shameless book plug for those who read to the end: if you like this sort of thing, check out my new book Reproducible Finance with R!
_____='https://rviews.rstudio.com/2019/03/14/parsnippingfamafrench/';
var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));To leave a comment for the author, please follow the link and comment on their blog: R Views. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
Unit Tests in R
(This article was first published on R – WinVector Blog, and kindly contributed to Rbloggers)
I am collecting here some notes on testing in R.
There seems to be a general (false) impression among non Rcore developers that to run tests, R package developers need a test management system such as RUnit or testthat. And a further false impression that testthat is the only R test management system. This is in fact not true, as R itself has a capable testing facility in "R CMD check" (a command triggering R checks from outside of any given integrated development environment).
By a combination of skimming the Rmanuals ( https://cran.rproject.org/manuals.html ) and running a few experiments I came up with a description of how Rtesting actually works. And I have adapted the available tools to fit my current preferred workflow. This may not be your preferred workflow, but I have and give my reasons below.
A glimpse of the R test ecosystem
 During "R CMD check", R runs all .R files (and .r files) in the tests directory. It counts tests as failures if the test raises an exception (for example calls stop()), or if the text output does not match what is already in a .Rout.save file in the same directory.
 The contents of the tests directory are written into sourcedistribution packages, but not written into binarydistribution packages.
 The contents of the inst directory are copied into the rootlevel of package distributions.
 RUnit (released June 2004) itself collects test suites from directories and then runs them, recording user assertions in a JUnitinspired report. The idea is that once you have a bunch of tests you really want to track them some way.
 testthat (released November 2009) selfdescribes as integrating into a workflow. It runs tests found in the tests/testthat subdirectory (directory found relative to the package source, not relative to an installed package) and tracks user assertions. The related devtools/usethis package both writes a canonical test controlling file into the tests directory (allowing testthat to be triggered by "R CMD check"), and can also directly run tests.
 unitizer (released April 2017) bases its tests on comparisons of objects, rather than comparing text or requiring user assertions. It also aids in producing and updating reference objects.
 tinytest (prerelease) decouples the ideas of test failures from exceptions.
There are many reasons for testing, and different ways that tests are used. Much confusion stems from a failure to separate the different motivations behind testing. Some categories of tests include:
 Acceptance unit tests. These are tests that must succeed for a package to be considered usable. Failing these tests can cause the package to be rejected by CRAN or by endusers.
 Weak integration tests. Integration tests are different than unit tests, but there is some overlap. For packages that are tightly coupled a wrong version of one package can cause a related package to fail. In this case it make a lot of sense to expose the tests to the users, so they can check the compatibility of their installed package suites.
 Tests that represent development goals. These tests can be from testdriven development, or reproducible errors incorporated from submitted issues. These tests may be in a failing state for some time. They are more private to the package developer and should not be distributed to CRAN or to the end users.
Confusion between these (and additional) categories of use, or assuming there is only one use of tests, are the sources of many arguments over proper testing procedures and/or appropriate test systems. In fact, it is useful to discuss the currently available test systems in R in light of (at least) the testing scenarios we’ve just described:
The "R CMD check" mechanism seems optimized to support case 1. RUnit directly supports case 3; the wrapr adapter for RUnit (which we will discuss below) is designed to support cases 1 and 2. testthat::test_check() supports case 1, and testthat::test_dir() supports case 3. unitizer and tinytest seem to emphasize cases 3 and 1.
Observations (based on above) R package developers do not need to use a test system such as RUnit or testthat to run tests. The data.table package is a great example of this: a core package running thousands of tests, without needing an external testing package.
 If you wish to allow endusers to run tests for binary distributed packages, the package developer must place them somewhere other than in tests. My suggestion is put them in inst/unit_tests, which will get installed at the toplevel of packages and is findable with the system.file() command.
 Package developers need the ability to run tests from both their sources (which RUnit and testthat both supply) and also from installed copies of their package (which RUnit supplies, as RUnit is path oriented rather than package oriented, and testthat supplies through the test_dir() command).
 The same package may be distributed to users either in binary or source fashion. A user may receive a binary package from CRAN if they are a nonUnix using a current (or nearcurrent) version of R. They will receive a source version if they are running an obsolete version of R, or if the package has not yet been built by CRAN for their version of R. Because tests in the tests directory are present in source versions of packages and not in binary versions of packages this means the user may or may not get tests. This seems like needless variation. Any needless variation is a possible source of confusion and errors. In my opinion the user should never get tests, or always get tests. Since there is no way to strip CRAN acceptance tests out after CRAN submission, I suggest the user always get tests.
 Tests need to be in a canonical place that can be found both by the user and by the test runner. Relative paths can cause problems, so to reliably run tests we need an anchor point other that the R‘s current working directory. One such anchor point is the installed package directory structure, which can be searched with system.file(). This argues for tests being part of a the package distribution. (Another way is to find paths relative to test runner source code, though the solutions can be problematic, possibly the "getSrcDirectory(function(){})" solution is a good fit.)
 RUnit doesn’t seem to suggest a canonical testlauncher file: a "do things exactly this way, and this is how you signal an error to R CMD check" file. This isn’t a big deal, as it is just a few lines of code: scan directories for tests, run tests, and trigger an externally observable failure if there are any test failures. But there is a huge difference between a task being 99% done and actually done. To this end I am now prototyping this complete testrunning advice here: https://github.com/WinVector/wrapr/blob/master/tests/package_test_runner.R.
 testthat is a popular testing facility as it is installed by devtools/usethis. However, note that devtools/usethis registers testthat as a "Suggests" package; so technically the test control file written by devtools/usethis is incorrect, as it has an unqualified "library(testthat)" in it. This means there is in fact no guarantee that testthat is going to be installed on the machine testing the package. Correct behavior would be to check for the presence of the testthat package and skip testing if not present. The Rcpp RUnit tests ( https://github.com/RcppCore/Rcpp/blob/master/tests/doRUnit.R ) are an example of correctly checking if the testrunner is present before loading the test runner package. Likely to avoid pain most test infrastructures have RUnit and testthat preloaded, but it is incorrect to count on this.
For your package think a bit on what you want from testing, instead of uncritically following popular procedures. As in all cases: how R actually works is described in the manuals (https://cran.rproject.org/manuals.html), and may not be what you heard on the street.
 If you want absolute minimal dependencies, do not use any test runner, but instead use R‘s builtin testing capabilities.
 It is a good development practice to minimize the unnecessary differences between the developer’s environment and the user’s environment. To my mind this is an argument for distributing your basic tests. RUnit makes distributing tests easy, testthat by default does not.
 Keep in mind endusers have different needs than package developers, so different testentry points may be better for each.
 If you want to emphasize test reporting you may want to consider unitizer or tinytest (when released).
 I repeat, in current R many uservisible failures are due to environment and undeclared simultaneous package version details. Many users like to test a package before investing time coding against it. Also, a failing userrun test is in fact a reproducible failure example which can be very valuable in filing issues. Having tests present allows a package developer to ask "have you tried the tests?"
 Consider: why should a user start work with a package without an easy "start by running the standard tests" introduction?
To conveniently provide test interfaces both to R CMD check and to endusers simultaneously, I now do the following.
 Add RUnit and wrapr to the package Suggests fields in the package DESCRIPTION.
 Place all tests in inst/unit_tests with files names of the form test_.*\\.R and zero argument testfunctions with names of the form test_.* (example here).
 To integrate with R CMD check: include code such as the following in the tests directory: tests (changing the package name to be that of your own package). Note for package developers using RStudio this also integrate the tests with the "Check" button in the Build Pane.
 To provide convenient user acceptance tests: add a function similar to run_sigr_tests() to your package. Then tell your users that to accept your package all they have to do is install it (plus any dependencies) and then, from R, run run_PKGNAME_tests(). Notice the example we give here is for the sigr package (not the wrapr package).
To leave a comment for the author, please follow the link and comment on their blog: R – WinVector Blog. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
Unpacking immigration collocations
(This article was first published on R – Gradient Metrics, and kindly contributed to Rbloggers)
As part of our road to detecting metaphors we got stuck on a simple problem: compound nouns.
If you take the sentence:
series of immigration policy changes
Series modifies changes in reference to immigration policy, which is a compound noun.
“Series of changes” is not what we would consider metaphorical usage, but our detector would label “series of immigration” as potentially metaphorical, given its strangeness. Identifying compound nouns, and identifying which specific word is being modified (and is thus the “target” concept in the metaphor), is critical to improving performance.
But, we realized we didn’t want to throw this extra information out. Enter collocations:
“a sequence of words or terms that cooccur more often than would be expected by chance”
Using the same corpus that we’ve been using (which contains news articles, social media posts, and TV transcripts), we calculated the most prominent collocations containing “immigrant”, “immigration”, and “migration”.
prefix suffix prefix frequency suffix frequency cooccurrence chain immigration 16271 24547 8548 undocumented immigrant 23145 127600 13645 migration crisis 24547 30606 1980 illegal immigrant 61791 127600 18759 comprehensive immigration 9550 261664 4930 migration visa 24547 23667 1041 immigration custom 261664 16135 6669 unaccompanied immigrant 7945 127600 1547 immigration reform 261664 42233 16839 testify immigrant 13444 127600 2288In graphical form, here’s what the information looks like:
The blue lines indicate the word is a prefix to the key source words (e.g. “chain migration”, “unaccompanied immigrant”), and the green lines indicate it is a suffix (e.g. “immigration policy”, “illegal reform”.)
What stands out to us is how little overlap there is in the collocations that overlap between the three words (except “illegal”, which is highly related to all three). This is especially surprising between “migration” and “immigration” which are both abstract nouns.
var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));To leave a comment for the author, please follow the link and comment on their blog: R – Gradient Metrics. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...
Speed Test: Sapply vs. Vectorization
(This article was first published on R – Open Source Automation, and kindly contributed to Rbloggers)
(adsbygoogle = window.adsbygoogle  []).push({ google_ad_client: "capub4184791493740497", enable_page_level_ads: true });
The apply functions in R are awesome (see this post for some lesser known apply functions). However, if you can use pure vectorization, then you’ll probably end up making your code run a lot faster than just depending upon functions like sapply and lapply. This is because apply functions like these still rely on looping through elements in a vector or list behind the scenes – one at a time. Vectorization, on the other hand, allows parallel operations under the hood – allowing much faster computation. This posts runs through a couple such examples involving string substitution and fuzzy matching.
String substitutionFor example, let’s create a vector that looks like this:
test1, test2, test3, test4, …, test1000000
with one million elements.
With sapply, the code to create this would look like:
start < proc.time() samples < sapply(1:1000000, function(num) paste0("test", num)) end < proc.time() print(end  start)As we can see, this takes over 4 1/2 seconds. However, if we generate the same vector using vectorization, we can get the job done in only 0.75 seconds!
start < proc.time() samples < paste0("test", 1:1000000) end < proc.time() print(end  start)Now, we can also use gsub to remove the substring test from every element in the vector, samples — also with vectorization:
start < proc.time() nums < gsub("test", "", samples) end < proc.time() print(end  start)This takes just over one second. In comparison, using sapply takes roughly eleven times longer!
start < proc.time() nums < sapply(samples, function(string) gsub("test", "", string)) end < proc.time() print(end  start) Fuzzy matchingVectorization can also be used to vastly speed up fuzzy matching (as described in this post). For example, let’s use the stringi package to randomly generate one million strings. We’ll then use the stringdist package to compare the word “programming” to each random string.
library(stringi) library(stringdist) set.seed(1) random_strings < stri_rand_strings(1000000, nchar("programming"),"[az]")Now, let’s try using sapply to calculate a string similarity score (using default parameters) between “programming” and each of the one million strings.
start < proc.time() results < sapply(random_strings, function(string) stringsim("programming", string)) end < proc.time() print(end  start)As we can see, this takes quite a while in computational terms – over 193 seconds. However, we can vastly speed this up using vectorization, rather than sapply.
start < proc.time() results < stringsim("programming", random_strings) end < proc.time() print(end  start)Above, we’re able to calculate the same similarity scores in…under one second! This is vastly better than the first approach and is made possible due to the parallel operations vectorization performs under the hood. To see the randomly generated word with the maximum similarity score to “programming”, we can just run the below line of code:
names(which.max(results))This returns the string “wrrgrrmmrnb”.
That’s it for this post! Click here to view other R posts of mine.
The post Speed Test: Sapply vs. Vectorization appeared first on Open Source Automation.
var vglnk = { key: '949efb41171ac6ec1bf7f206d57e90b8' }; (function(d, t) { var s = d.createElement(t); s.type = 'text/javascript'; s.async = true; s.src = '//cdn.viglink.com/api/vglnk.js'; var r = d.getElementsByTagName(t)[0]; r.parentNode.insertBefore(s, r); }(document, 'script'));To leave a comment for the author, please follow the link and comment on their blog: R – Open Source Automation. Rbloggers.com offers daily email updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...