Sections
 
You are here: Home » Development » API » API 1.2 - current version » Validation

Validation

— filed under: , ,

 

Related links

 
Examples: a how-to with curl calls  http://opentox.informatik.uni-freiburg.de/validation/examples
Background information: validation workflow  Validation and Reporting Overview and Data Flow
Technical details: statistics computed by the validation Validation statistics
Frequently asked questions
 Validation FAQ
Currently running validation web service
 http://opentox.informatik.uni-freiburg.de/validation
Algorithm and Dataset Examples to validate
 https://spreadsheets.google.com/spreadsheet/ccc?key=0AllBraSK8gV-dFgwUDJHaVJmQTlJRG1iTnJqNXBaZEE&hl=en_US&authkey=CJGPxMgB
 

 

Validation

 

Component description

A validation corresponds to the validation of a model on a test dataset. The results are stored in another dataset. Parameters with default values are optional.

REST operations

Description Method URI Parameters Result Status codes
Get all validations
 GET / [subjectid] List of validation URIs 200,404
Retrieves a validation representation GET /{id} [subjectid] Validation representation in one of the supported MIME types
 200,404
Validates a model on a test dataset
 POST /test_set_validation [subjectid]
model_uri
test_dataset_uri
test_target_dataset_uri (default = test_dataset_uri)
prediction_feature (default = dependent variable of model)
 Validation URI or Task URI
 200,400,404,500
Builds a model on a training dataset and validates it on a test dataset
 POST
 /training_test_valdiation algorithm_uri
prediction_feature
algorithm_params (string, default="")
training_dataset_uri
test_dataset_uri
test_target_dataset_uri (default = test_dataset_uri)
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
[subjectid]
 Validation URI or Task URI 200,400,404,500
Splits a dataset into training and test dataset according to a certain ratio, and performs a validation
 POST /training_test_split algorithm_uri
prediction_feature
algorithm_params (string, default="")
dataset_uri
split_ratio (float, default=0.66)
random_seed (integer, default=1)
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
[subjectid]
 Validation URI or Task URI 200,400,404,500
Performs a bootstrap validation
 POST /bootstrapping algorithm_uri
prediction_feature
dataset_params (string, default="")
dataset_uri
bootstrap_percentage (float, default=0.66)
random_seed (integer, default=1)
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
[subjectid]
 Validation URI or Task URI 200,400,404,500
Directly perform a validation by specifying test- and prediction dataset  POST  /validate_datasets prediction_feature
test_dataset_uri
test_target_dataset_uri (default = test_dataset_uri)
prediction_dataset_uri
predicted_feature (.i.e feature in prediction dataset)
[subjectid]		 Validation URI or Task URI 200,400,404,500
Deletes a validation. DELETE /{id} [subjectid] -
 200,404

Validation representation

  • Requested MIME type should be set in the requests "Accept" header, e.g. curl -X GET -H "Accept:application/xml" http://{server}/validation/{id}
  • RDF representation defined in opentox.owl
  • RDF Examples
  • subjectid (optional) parameter that contains the OpenSSO A&A token needed to access protected services.

Supported MIME types:

Mandatory:

  • application/rdf+xml (default)

Optional:

HTTP status codes

Interpretation Nr Name
Success 200 OK
Validation not found 404 Not Found
Illegal model/algorithm/dataset/algorithm params 400 Bad request
Validation/prediction error 500 Internal Server Error

  

Cross-Validation

Component description

Performs a k-fold cross-validation, which results in k validations. Parameters with default values are optional.

REST operations

Description Method URI Parameters Result Status codes
Get all cross-validations
 GET /crossvalidation [subjectid] List of crossvalidation URIs 200,404
Retrieves a cross-validation representation GET /crossvalidation/{id} [subjectid] Cross-Validation in one of the supported MIME types
 200,404
Returns all (k) validations that belong to a crossvalidation
 GET /crossvalidation/{id}/validations [subjectid] List of validation URIs
 200,404
Performs a k-fold cross-validation.
POST
 /crossvalidation algorithm_uri
prediction_feature
algorithm_params (string, default="")
num_folds (integer, default=10)
random_seed (integer, default=1)
stratified (boolean, default=true)
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
[subjectid]
 Cross-Validation URI or Task URI 200,400,404,500
Performs a leave-one-out cross-validation.
POST
 /crossvalidation/loo algorithm_uri
prediction_feature
algorithm_params (string, default="")
y_scramble (boolean, default=false)
y_scramble_seed (integer, default=1)
[subjectid]
 Cross-Validation URI or Task URI 200,400,404,500
Deletes a cross-validation. DELETE /crossvalidation/{id} [subjectid] -
 200,404

Cross-validation representation

  • Requested MIME type should be set in the requests "Accept" header, e.g. curl -X GET -H "Accept:application/xml" http://{server}/crossvalidation/{id}

Supported MIME types:

Mandatory:

  • application/rdf+xml (default)

Optional:

HTTP status codes

Interpretation Nr Name
Success 200 OK
Cross validation not found 404 Not Found
Illegal model/algorithm/dataset/algorithm params 400 Bad request
Validation/prediction error 500 Internal Server Error
 
Document Actions

Validation - Report

Component description

A report visualizes the (prediction) results of algorithms.

REST operations

Description Method URI Parameters Result Status codes
Get all report types
 GET /report [subjectid] List of available report types
 200,404
Get all reports for the particular report type
 GET /report/{report-type} [subjectid] List of available reports as URI
 200,404
Retrieves a report, format according to accept-header GET /report/{report-type}/{id} [subjectid] According to accept-header:
text/x-yaml, application/rdf+xml
-> report object with meta data (see rdf example below)

text/xml, text/html, application/pdf
-> report document in specified format
 200,404
Creates a report
 POST /report/{report-type} various params, see below
[subjectid]
 Report URI or Task URI 200,400,404,500
Deletes a report. DELETE /report/{report-type}/{id} [subjectid] -
 200,404
Available (validation-)report types




Create single validation report
(one model, one test dataset)
 POST
 /report/validation validation_uris (comma separated list of validation URIs)
[subjectid]
 Report URI or Task URI 200,400,404,500
Create cross-validation report
(crossvalidation with one algorithm and one dataset)
 POST
 /report/crossvalidation validation_uris (comma separated list of crossvalidation URIs)
[subjectid]
 Report URI or Task URI 200,400,404,500
Create report for comparing different prediction algorithms
(crossvalidations/validations with multiple algorithms and datasets)
 POST
 /report/algorithm_comparison validation_uris (comma separated list of crossvalidation URIs)
identifier (comma seperated list of string-ids, one for each crossvalidation-uri)
[subjectid]
 Report URI or Task URI 200,400,404,500

 

Output (file) format

Validation reports are returned in 2 different ways according to the accept-header

  • Accept header is text/xml, text/html, application/pdf: the actual report document is returned (the core document is an XML format according to the DocBook standard, which allows conversion to the other formats).
  • Accept header is application/rdf+xml or text/x-yaml: the report object is returned, containing meta-data. Example (in N3) below
@prefix : <http://www.opentox.org/api/1.1#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
    
<http://opentox.informatik.uni-freiburg.de/validation/report/validation/8>     
a :ValidationReport;
	:date "Wed Jun 09 13:23:28 +0200 2010"^^<http://www.w3.org/2001/XMLSchema#dateTime>;
        :reportAlgorithm <http://ambit.uni-plovdiv.bg:8080/ambit2/algorithm/LR>;
        :reportModel <http://ambit.uni-plovdiv.bg:8080/ambit2/model/359152>;
        :reportType "validation"^^<http://www.w3.org/2001/XMLSchema#string>;
        :reportValidation <http://opentox.informatik.uni-freiburg.de/validation/20> .


HTTP status codes

Interpretation Nr Name
Success 200 OK
Report type / report not found 404 Not Found
Illegal params 400 Bad request
Error creating the report
 500 Internal Server Error

 

Document Actions