Articles by Ron Pearson (aka TheNoodleDoodler)

Classification Tree Models

April 13, 2013 | Ron Pearson (aka TheNoodleDoodler)

On March 26, I attended the Connecticut R Meetup in New Haven, which featured a talk by Illya Mowerman on decision trees in R.  I have gone to these Meetups before, and I have always found them to be interesting and informative.  Attendees range from those who are just starting to ... [Read more...]

Finding outliers in numerical data

February 16, 2013 | Ron Pearson (aka TheNoodleDoodler)

One of the topics emphasized in Exploring Data in Engineering, the Sciences and Medicine is the damage outliers can do to traditional data characterizations.  Consequently, one of the procedures to be included in the ExploringData package is FindOutliers, described in this post.  Given a vector of numeric values, this procedure ... [Read more...]

Data Science, Data Analysis, R and Python

December 15, 2012 | Ron Pearson (aka TheNoodleDoodler)

The October 2012 issue of Harvard Business Review prominently features the words “Getting Control of Big Data” on the cover, and the magazine includes these three related articles:“Big Data: The Management Revolution,” by Andrew McAfee and Erik Brynjolfsson, pages 61 – 68;“Data Scientist: The Sexiest Job of the 21st Century,” by Thomas ... [Read more...]

Characterizing a new dataset

October 27, 2012 | Ron Pearson (aka TheNoodleDoodler)

In my last post, I promised a further examination of the spacing measures I described there, and I still promise to do that, but I am changing the order of topics slightly.  So, instead of spacing measures, today’s post is about the DataframeSummary procedure to be included in the ... [Read more...]

Spacing measures: heterogeneity in numerical distributions

September 22, 2012 | Ron Pearson (aka TheNoodleDoodler)

Numerically-coded data sequences can exhibit a very wide range of distributional characteristics, including near-Gaussian (historically, the most popular working assumption), strongly asymmetric, light- or heavy-tailed, multi-modal, or discrete (e.g., count data).  In addition, numerically coded values can be effectively categorical, either ordered, or unordered.  A specific example that illustrates ... [Read more...]

Base versus grid graphics

July 21, 2012 | Ron Pearson (aka TheNoodleDoodler)

In a comment in response to my latest post, Robert Young took issue with my characterization of grid as an R graphics package. Perhaps grid is better described as a “graphics support package,” but my primary point – and the main point of this post – is that to generate the display ... [Read more...]

Classifying the UCI mushrooms

June 10, 2012 | Ron Pearson (aka TheNoodleDoodler)

In my last post, I considered the shifts in two interestingness measures as possible tools for selecting variables in classification problems.  Specifically, I considered the Gini and Shannon interestingness measures applied to the 22 categorical mushroom characteristics from the UCI mushroom dataset.  The proposed variable selection strategy was to compare these ... [Read more...]

Interestingness comparisons

May 19, 2012 | Ron Pearson (aka TheNoodleDoodler)

In three previous posts (April 3, 2011,  April 12, 2011,and May 21, 2011), I have discussed interestingness measures, which characterize the distributional heterogeneity of categorical variables.  Four specific measures are discussed in Chapter 3 of Exploring Data in Engineering, the Sciences and Medicine: the Bray measure, the Gini measure, the Shannon measure, and the Simpson measure.  ... [Read more...]

Gastwirth’s location estimator

March 3, 2012 | Ron Pearson (aka TheNoodleDoodler)

The problem of outliers – data points that are substantially inconsistent with the majority of the other points in a dataset – arises frequently in the analysis of numerical data.  The practical importance of outliers lies in the fact that even a few of these points can badly distort the results of ... [Read more...]

Cleaning time-series and other data streams

November 27, 2011 | Ron Pearson (aka TheNoodleDoodler)

The need to analyze time-series or other forms of streaming data arises frequently in many different application areas.  Examples include economic time-series like stock prices, exchange rates, or unemployment figures, biomedical data sequences like electrocardiograms or electroencephalograms, or industrial process operating data sequences like temperatures, pressures or concentrations.  As a ... [Read more...]

Is the “Long Tail” a Useless Concept?

September 28, 2011 | Ron Pearson (aka TheNoodleDoodler)

In response to my last post, “The Long Tail of the Pareto Distribution,” Neil Gunther had the following comment:            “Unfortunately, you've fallen into the trap of using the ‘long tail’ misnomer. If you think about it, it can't possibly be the length of the tail that sets distributions like Pareto ... [Read more...]
1 2

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)