Validate Your QSAR Model and Create a Report using Web Forms
In this tutorial you will learn how to employ the validation services and explore the reporting facilities, including generating QMRF reports, via web forms.
General Information / Requirements
Description of Activity:
In this tutorial we will demonstrate how to effectively use the validation and reporting web services behind the applications of ToxPredict and ToxCreate. Using web forms, we will contact the web services and validate a model or algorithm using a number of different approaches such as k-fold split, training-test-split, or bootstrapping. Furthermore, we will generate a QMRF report and visualize it using the QMRF Editor web start application. This tutorial is aimed at advanced users and developers, who want to look behind the scenes.
Datasets and Algorithms/Models:
Example 1 - Build and validate a regression model with training-test-set validation
In this use case, we will build a model with a training dataset, apply the model to predict the compounds in the test-dataset, and validate the results.
Step 1 - Log in as guest (user: guest, passowrd: guest) at http://opentox.informatik.uni-freiburg.de/validation/sign_in
We have Authentication & Authorization enabled on our services. The resources used in this tutorial are available to the user guest (password guest).
Step 2 - Fill out html form at http://opentox.informatik.uni-freiburg.de/validation/training_test_validation
We will use Multiple Linear Regression for prediction (the service is located in Athens, http://opentox.ntua.gr:8080/algorithm)
- algorithm uri: http://opentox.ntua.gr:8080/algorithm/mlr
The dataset is a well known permeability dataset (http://pubs.acs.org/doi/abs/10.1021/ci049884m). It includes 4 numerical chemical descriptors, and one measured numerical endpoint (caco-2 permeability).
The dataset is available at the Ambit web service (located in Sophia, http://apps.ideaconsult.net:8080/ambit2/dataset)
- training dataset uri: http://apps.ideaconsult.net:8080/ambit2/dataset/603157
- test dataset uri: http://apps.ideaconsult.net:8080/ambit2/dataset/603159
- prediction feature uri: http://apps.ideaconsult.net:8080/ambit2/feature/763038
Visit http://opentox.informatik.uni-freiburg.de/validation/training_test_validation and fill out the form with the above given links. Click Perform training-set-validation. This will load a task resource, like e.g.: http://opentox.informatik.uni-freiburg.de/task/12251. Reload this page (hitting key F5, or using the reload button of your browser) until the task is completed. The result of the task is given in the section Related links (e.g.: The task result: http://opentox.informatik.uni-freiburg.de/validation/1406).
Step 3 - Create validation report
Visiting your validation result directly with a browser (e.g. http://opentox.informatik.uni-freiburg.de/validation/1406) will give you the internal representation of the validation. You can create a report by clicking on the Search for corresponding report link (example link: http://opentox.informatik.uni-freiburg.de/validation/report/validation?validation=http://opentox.informatik.uni-freiburg.de/validation/1406)
A form is provided for creating a report. The parameter field is already filled out with the corresponding validation link, you just have to click the Create validation report button. This will forward you to another task page. Again, reload until the task is completed and follow the task-result link to the validation report.
Step 4 - Visit validation report
An example validation report is available here: http://opentox.informatik.uni-freiburg.de/validation/report/validation/147.
- The first section of the report is the Results section. It contains the most important validation results, like R² and Root mean squared error.
- The Plots provides the Actual vs Predicted values plot. This plot gives an intuitive overview on the predictions performance of your model.
- The All Results section gives more details on the validation, like validation runtime, or the variance of the actual and predicted values.
- Predictions lists all compounds, providing a picture, the actual target value and the predicted target value for each compound.
Example 2 - Evaluate classification with crossvalidation
Tutorial will be added soon....
Example crossvalidation report: http://opentox.informatik.uni-freiburg.de/validation/report/crossvalidation/5
Play around and make your own validations
You can follow the links to opentox-resources at the following link to play around with the validation service and make your own validations:https://docs.google.com/spreadsheet/pub?hl=en_US&hl=en_US&key=0AllBraSK8gV-dFgwUDJHaVJmQTlJRG1iTnJqNXBaZEE&single=true&gid=0&output=html
Recording of the online tutorialDownload the Recording of the Online Tutorial held 22 June 2011
API-DefinitionBefore we start, it is probably an advantage to have a window with the API definitions for the validation services open in a browser: so please open the following link in a browser, preferably Firefox: http://www.opentox.org/data/documents/development/validation/Validation/
Validation Examples using the commandline
The validation web services can be accessed using the command line tool curl (http://curl.haxx.se). The examples are available here: http://opentox.informatik.uni-freiburg.de/validation/examples
Please don't forget to provide your feedback after completing this exercise using the Evaluation form at https://www.surveymonkey.com/s/652XY7C