There are a number of good open source projects for statistics and data mining, for example the software WEKA developed at the University of Waikato.
The description on their website states that:
Weka is a collection of machine learning algorithms for data mining tasks. The algorithms can either be applied directly to a dataset or called from your own Java code. Weka contains tools for data pre-processing, classification, regression, clustering, association rules, and visualization. It is also well-suited for developing new machine learning schemes.
The software is written in Java and available under the GNU General Public Licence. The website also provides access to data sets from the UCI Machine Learning website for use with WEKA.