R, rvest and web-harvesting

R, rvest and web-harvesting

/in , /by 

Data harvested from the web pages is a source of interesting information. Pulling data used to require quite a lot of resilience and misshapen Perl scripts struggling with messy sources of web pages. Today’s web pages more and more frequently comply meet the standards. There are also more and more civilized tools for parsing websites.

Read more
Canonical discriminant analyses and HE plots

Canonical discriminant analyses and HE plots

/in , /by 

Last week we wrote about multidimensional linear models. We discussed a case in which a k-dimensional vector of the dependent variables is related to a grouping variable. We look at matrices E and H in order to find out whether there is any relationship (see the previous blog).

Read more
HE plots

HE plots

/in , /by 

GPS helps the drivers to avoid traffic jams, yet in more advanced uses it allows for fleet management or remote drone strikes. It is just the same with visualization. Bars and dots can be used to present a set of several means but there are also more advanced uses…

Read more
Spark + R = SparkR

Spark + R = SparkR

/in , /by 

Spark wins more and more hearts. And no wonder, comments from different sources tell us about a significant speed up (by an order of magnitude) for analysis of big datasets. Well-developed system for caching objects in memory allows us to avoid torturing hard discs during iterative operations performed on same data.

Read more
Pretty heat maps

Pretty heat maps

/in , /by 

Do you know where Kamil Stoch earns most of his points in season 2013/2014? Some time ago I came across a pheatmaps package for R software which generates much nicer heat maps than the standard heatmap() function. This is why the package is named…

Read more
Is a simple linear regression able to knock spots off SVM and Random Forest?

Is a simple linear regression able to knock spots off SVM and Random Forest?

/in , /by 

A friend of mine took part in a project in which he had to perform future prediction of Y characteristic. The problem was that Y characteristic showed an increasing trend over time. For the purposes of this post let us assume that Y characteristic was energy demand or milk yield of cows or any other characteristic that with positive trend over time.

Read more