[Read more...]

I occasionally see queries on various social media as to overfitting — what is it?, etc. I’ll post an example here. (I mentioned it at my talk the other night on our novel approach to missing values, but had a bug in the code. Here is the correct account.) The ... [Read more...]

[Read more...]

[Read more...]

I’m about to show you an R trick. Various readers may find it cool, useful and interesting, or stupid, useless and an evil deed undermining the sanctity of R’s functional programming nature (“All bow”). But I hope many of you will find the material here rather intriguing if ... [Read more...]

[Read more...]

You may be interested in my new arXiv paper, joint work with Xi Cheng, an undergraduate at UC Davis (now heading to Cornell for grad school); Bohdan Khomtchouk, a post doc in biology at Stanford; and Pete Mohanty,  a Science, Engineering & Education Fellow in statistics at Stanford. The paper is ... [Read more...]

Last week I gave one of the keynote addresses at R/Finance 2018 in Chicago. I considered it an honor and a pleasure to be there, both because of the stimulating intellectual exchange and the fine level of camaraderie and hospitality that prevailed. I mentioned at the start of my talk ... [Read more...]

[Read more...]

The above title was the title of my talk this evening at our Bay Area R Users Group. I had been asked to talk about my new book, and I presented four of the myths that are dispelled in the book. Hadley also gave an interesting talk, “An introduction to ... [Read more...]

My students, Vincent Yang and Harrison Nguyen, and I have developed a new data visualization package, cdparcoord, available now on CRAN. It can be viewed as an extension of the freqparcoord package written by a former grad student, Yingkang Xie and myself, which I have written about before in this ... [Read more...]

[Read more...]

On May 31, I made a post here about our R package revisit, which is designed to help remedy the reproducibility crisis in science. The intended user audience includes reviewers of research manuscripts submitted for publication, scientists who wish to confirm the results in a published paper, and explore alternate analyses, ... [Read more...]

In my talk at useR! earlier this month, I emphasized the fact that a major impediment to obtaining good speed from parallelizing an algorithm is systems overhead of various kinds, including: Contention for memory/network. Bandwidth limits — CPU/memory, CPU/network, CPU/GPU. Cache coherency problems. Contention for I/O ... [Read more...]

I gave a talk titled, “Parallel Computation in R:  What We Want, and How We (Might) Get It,” at last week’s useR! 2017 conference in Brussels. You can view my slides here, and I think the conference organizers said the videos would be placed online, not sure of that though. ... [Read more...]

Readers who found my posting last week on our revisit package of interest may wish to check the somewhat related package ropensci/rrrpkg, which Edzer Pemesma tweeted about today. [Read more...]

Several years ago, John Ionnidis jolted the scientific establishment with an article titled, “Why Most Published Research Findings Are False.” He had concerns about inattention to statistical power, multiple inference issues and so on. Most people had already been aware of all this, of course, but that conversation opened the ... [Read more...]

[Read more...]

A really nice function available in Python is walk(), which recursively descends a directory tree, calling a user-supplied function in each directory within the tree. It might be used, say, to count the number of files, or maybe to remove all small files and so on. I had students in ... [Read more...]

[Read more...]