####################################################################################################

Supplementary Material 2: Electronic appendix to the article:

Ordinal Forests

Roman Hornung*

* Institute for Medical Information Processing, Biometry and Epidemiology,
  University of Munich, Marchioninistr. 15, D-81377, Munich, Germany;
  for questions please contact:  hornung@ibe.med.uni-muenchen.de

####################################################################################################


Program and Platform:
#####################

- Program: R, version 3.3.0

- Used R packages:

     'ggplot2', version: 2.2.1  
     'gridExtra', version: 2.2.1
     'MASS', version: 7.3-47
     'mvtnorm', version: 1.0-6
     'ordinalForest', version: 2.1
     'plyr', version: 1.8.4
     'psych', version: 1.7.5 
     'ranger', version: 0.8.0 
     'ROCR', version: 1.0-7 
     'TH.data', version: 1.0-8

  NOTE: Packages listed above depend on others, which might have to be installed
        manually in case this is not performed automatically.

- Platforms: Linux (x86-64)  (for the conduction of the analyses)
             Windows 7 64-bit (for the evaluation of the results)


##### !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ######
#
# WARNING: 
#
# In many of the  R script the command
# "rm(list=ls());gc()" is used,
# WHICH DELETES ALL OBJECTS IN THE CURRENT WORKSPACE.
#
##### !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! ######


General information:
####################

- Paths of the form "./HornungOrdinalForests/..." are used in all
  R scripts, where "./" is the R working directory. Therefore, the
  folder "HornungOrdinalForests" this README file is contained in
  has to be put into the R working directory.

- The following subfolders are found in "HornungOrdinalForests":

  - "RealDataApplications": The R script "RealDataAnalysis.R"
    performs the repeated stratified cross-validation using the five 
    considered datasets.
    The R script "RealDataAnalysis_Evaluation.R" performs the evaluation 
    of the results of the real data analysis.
    The subfolder "data" contains the following: The R script 
    "DataPreprocessing.R" which performs the pre-processing of the five 
    considered datasets. Using the R script "RealDataDescriptionTable.R"
    the information provided with Table 1 can be retrieved from 
    the real datasets. Moreover, the raw versions of the real
    datasets are contained in this folder (except for the raw version
    of the dataset 'mammography' which is contained in the R package 
    'TH.data') and the processed versions of the datasets which
    are indicated by the substring "_dataset".

  - "Simulation": This subfolder contains all R scripts necessary
     to perform and evaluate all analysis involving simulated data:
    "Simulation.R" performs the large simulation study used to compare
    the three considered methods with respect to their prediction
    performance and the qualities of their variable importance measures.
    "Simulation_Evaluation.R" performs the evaluation of the simulation,
    that is, using this R script all results presented in section 3.2.2
    can be obtained. 
    "SimulationPerffunction.R" performs the simulation study
    that studies the performance of the performance functions. 
    "SimulationPerffunction_Evaluation.R" performs the evaluation of the
    latter simulation study.
    "SimulationSensitivityAnalysisHyperparameters.R" performs the
    simulation study using which the appropriateness of the default
    hyperparameter values and the robustness of the results
    obtained using ordinal forest with respect to changes in these default 
    hyperparameter values are investigated.
    "SimulationSensitivityAnalysisHyperparameters_Evaluation.R" performs
    the evaluation of the latter simulation study.
	
  - "Functions": This subfolder contains the R scripts "FunctionsRealDataAnalysis.R"
    and "FunctionsSimulation.R" that contain all functions used in the real data analysis
    and the simulations, respectively. These scripts are called by the R scripts
    that perform the corresponding analyses.
	
  - "Results": This folder contains Rda files containing the raw results
    of the analyses, which are produced by the corresponding R scripts
    performing these analyses. Moreover, a subfolder "Figures" is
    contained in "Results" that contains all figures produced in the analyses
    performed in the main paper and in Supplementary Material A.

	
  
Evaluation of the results:
##########################

- As indicated above, for the evaluation of the results it is not necessary to 
  re-perform the analyses:
  In the folders "RealDataApplications" and "Simulation" R files are provided
  using which all results presented in the main paper and in Supplementary
  Material A can be obtained. These files are designated by the substring
  "_Evaluation" (see above for details).

  

Reproducing the results:
########################

- For reproducing the results, the folder "HornungOrdinalForests"
  this README is contained in has to be put into the home directory 
  ("~/") of a Linux machine.

- An MPI environment is required.

- The R scripts "Simulation.R", "SimulationPerffunction.R", 
  "SimulationSensitivityAnalysisHyperparameters.R" (subfolder: "Simulation"), and 
  "RealDataAnalysis.R" (subfolder: "RealDataApplications") have to be excecuted 
  to perform all analyses presented in the main paper and in Supplementary Material A.
  See above for details on which of these scripts perform
  which analyses.
