More docs

git-svn-id: https://dng.biozentrum.unibas.ch/svn/openstructure/trunk@2347 5a81b35b-ba03-0410-adc8-b2c5c5119f08

doc/conf/conf.py

@@ -16,8 +16,10 @@ import sys, os
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
-#sys.path.append(os.path.abspath('.'))
-
+sys.path.append(os.path.join(os.path.abspath('../..'), 
+                             'stage/lib/openstructure'))
+sys.path.append(os.path.join(os.path.abspath('../..'), 
+                             'stage/lib64/openstructure'))
 # -- General configuration -----------------------------------------------------
 
 # Add any Sphinx extension module names here, as strings. They can be extensions

modules/base/doc/base.rst

+:mod:`ost.settings` - Locate Files and Retrieve Preferences
+================================================================================
+
+.. automodule:: ost.settings
+  :synopsis: Helper Functions to Locate Files and Retrieve Preferences
+  :members:
+  
+  
+

modules/base/doc/generic.rst

+Storing Custom Data
+================================================================================
+
+Introduction
+--------------------------------------------------------------------------------
+
+It is often very convenient to store any arbitrary data inside an Entity. A few examples are: 
+
+  * calculated properties of atoms
+  * sequence conservation of a residue
+  * interaction energy of a substructure with its surrounding
+  * fit of a fragment inside an electron density map
+
+In OpenStructure this is supported by the use of generic properties. Most 
+building blocks are derived from :class:`GenericPropertyContainer`, meaning that 
+arbitrary key-value pairs can be stored in them. In essence, the following 
+classes support generic properties:
+
+  * :class:`~mol.EntityHandle` and :class:`~mol.EntityView`
+  * :class:`~mol.ChainHandle` and :class:`~mol.ChainView`
+  * :class:`~ResidueHandle` and :class:`~mol.ResidueView`
+  * :class:`~mol.AtomHandle` and :class:`~mol.AtomView`
+  * :class:`~mol.BondHandle`
+  * :class:`~seq.SequenceHandle` and :class:`~seq.AlignmentHandle`
+
+The view variants will reflect the generic properties of the handle variants.
+
+A generic property key is always a string, and a value can be one of string, float, int or bool. For each of these data types, methods to retrieve and store values are available both in Python and C++.
+
+Storing and Accessing Data
+--------------------------------------------------------------------------------
+
+All OpenStructure building blocks that are :class:`GenericPropContainers`, have 
+four different methods to store generic data, depending on the data type (i.e. 
+string, float, int or bool).
+
+To store a float value with the key 'myfloatprop' in all atoms of an entity:
+
+.. code-block:: python
+  
+  import math
+  for atom in entity.GetAtomList(): 
+    val=5*math.sin(0.4*atom.GetPos().GetX())
+    atom.SetFloatProp("myfloatprop", val)
+  
+If a GenericProp at a given level (i.e. atom, bond, residue, chain or entity) 
+already exists, it will be overwritten. To check if it exists, use:
+
+.. code-block:: python
+  
+  exists=atom.HasProp("myfloatprop")
+  print exists
+    
+To access the value of a generic property, we first check if the property exists
+and then access it, using the method suitable for the data type of the property. 
+For the previously set property "myfloatprop" of the data type real, at the atom 
+level:
+
+.. code-block:: python
+  
+  for atom in entity.GetAtomList(): 
+    if atom.HasProp("myfloatprop"):
+      print atom.GetFloatProp("myfloatprop")
+        
+When trying to access a property that has not been set, or one that has been 
+set, but at a different level, an error is thrown. The same is true when trying 
+to access a property of a different data type, e.g.:
+
+.. code-block:: python
+
+  # all of the following lines will throw errors
+  # error because the property does not exist 
+  print atom.GetFloatProp("unknownprop")
+  
+  # error because the property was set at another level
+  print entity.GetFloatProp("myfloatprop")
+  
+  # error because the data type of the property is different
+  print atom.GetStringProp("myfloatprop")
+      
+
+Use of Generic Properties in Queries
+--------------------------------------------------------------------------------
+
+The :doc:`query` can also be used for numeric generic properties (i.e. bool, 
+int, float), but the syntax is slightly different. To access any generic 
+properties, it needs to be specified that they are generic and at which level 
+they are defined. Therefore, all generic properties start with a 'g', followed 
+by an 'a', 'r' or 'c' for atom, residue or chain level respectively. For more 
+details see :doc:`query`. 
+
+
+API documentation
+--------------------------------------------------------------------------------
+
+.. class:: GenericPropertyContainer
+
+  .. method:: HasProp(key)
+  
+    checks existence of property. Returns true, if the the class contains a
+    property with the given name, false if not.
+  
+  .. method:: GetPropAsString(key)
+  
+    Returns the string representation of a property, or the empty String if 
+    the property addressed by key does not exist. Note that this is not the 
+    same as trying to get a generic float/int/bool property as a string type; 
+    the latter will result in a boost:get exception. Use this method to obtain 
+    a representation suitable for output.
+    
+  .. method:: GetStringProp(key)
+              GetStringProp(key, default_value)
+  
+    Get string property. The first signature raises a GenericPropError error if
+    the property does not exist, the second returns the default value.
+  
+  
+  .. method:: GetFloatProp(key)
+              GetFloatProp(key, default_value)
+   
+    Get float property. The first signature raises a GenericPropError error if
+    the property does not exist, the second returns the default value.
+
+
+  .. method:: GetIntProp(key)
+              GetIntProp(key, default_value)
+
+    Get int property. The first signature raises a GenericPropError error if
+    the property does not exist, the second returns the default value.
+
+  .. method:: GetBoolProp(key)
+              GetBoolProp(key, default_value)
+
+    Get bool property. The first signature raises a GenericPropError error if
+    the property does not exist, the second returns the default value.
+        
+  .. method:: ClearProps()
+    
+    Remove all generic properties
+  
+ 
+  .. method:: SetStringProp(key, value)
+    
+    Set string property, overriding an existing property with the same name
+    
+  .. method:: SetFloatProp(key, value)
+    
+    Set float property, overriding an existing property with the same name
+
+  .. method:: SetIntProp(key, value)
+  
+    Set int property, overriding an existing property with the same name
+  
+  .. method:: SetBoolProp(key, value)
+  
+    Set bool property, overriding a property with the same name
+

modules/base/pymod/settings.py

@@ -3,12 +3,13 @@ import __main__
  
 
 def GetValue(val_key,val_default=None,prefix='OST'):
-  '''
+  """
   Returns the value of the variable val_key if defined, otherwise returns the 
   default value provided by the user (if provided). Search order: 
-  1) environment variable called $prefix_$val_key 
-  2) variable called val_key in .dngrc file
-  '''
+  
+   * environment variable called $prefix_$val_key 
+   * variable called val_key in .ostrc file
+  """
   if prefix:
     env_var_name='%s_%s' % (prefix, val_key)
   else:
@@ -24,6 +25,12 @@ def GetValue(val_key,val_default=None,prefix='OST'):
       return val_default
 
 class FileNotFound(RuntimeError):
+  """
+  Raised when :func:`Locate` is unable to locate a file. The exception contains
+  detailed information on what was tried to locate the file, i.e. search paths, 
+  environment variables and also provides useful hints on how to let Locate know
+  where to find the file.
+  """
   def __init__(self, name, reason):
     self.name=name
     self.reason=reason
@@ -33,21 +40,23 @@ class FileNotFound(RuntimeError):
 def Locate(file_name, explicit_file_name=None, search_paths=[],
            env_name=None, search_system_paths=True):
   """
-  Helper function to locate files. To get the full name of 
-  an executable, let's say qmake, use
+  Helper function to locate files. To get the full name of an executable, let's 
+  say qmake, use
+  
+  .. code-block:: python
 
-  abs_qmake_path=Locate('qmake', env_name='QMAKE_EXECUTABLE')
+    abs_qmake_path=Locate('qmake', env_name='QMAKE_EXECUTABLE')
 
-  First the function checks if an environment variable with the
-  name QMAKE_EXECUTABLE is set. If so, the value of this variable
-  is returned. Next, each directory listed in search_paths is 
-  searched. If the executable could still not be found and
-  search_system_paths is set to True, the binary search paths are
-  searched.
+  First the function checks if an environment variable with the name 
+  QMAKE_EXECUTABLE is set. If so, the value of this variable is returned. Next, 
+  each directory listed in search_paths is searched. If the executable could 
+  still not be found and search_system_paths is set to True, the binary search 
+  paths are searched.
 
-  If the file could not be located, a FileNotFound exception will be raised
-  containing a detail description why Locate failed. The error message is
-  formatted in such a way that it can directly be presented to the user.
+  If the file could not be located, a :exc:`~ost.settings.FileNotFound` 
+  exception will be raised containing a detail description why Locate failed. The 
+  error message is formatted in such a way that it can directly be presented to 
+  the user.
   """
   if type(file_name) is str:
     file_names=[file_name]

modules/doc/external.rst

+Using External Programs within OpenStructure
+================================================================================
+
+Introduction
+--------------------------------------------------------------------------------
+
+It is often very useful to use external programs to do a specific task. In principle, this can be done by writing out files from OpenStructure and manually running an external program, however, for convenience, this can also be done directly from within OpenStructure using Python commands. 
+
+This tutorial will give you some hints how to do this for a new external program. The process basically consists of four steps:
+
+  * locate the executable of the external program
+  * prepare all necessary files
+  * execute the external program from python
+  * read in generated output
+
+
+Locating the Executable
+--------------------------------------------------------------------------------
+
+There is a helper function available to locate files, and especially executables: :func:`~ost.settings.Locate`. Using this, you can obtain the full path of an executable.
+
+As an example, we would like to obtain the full path of the msms executable (a program to calculate molecular surfaces):
+
+.. code-block:: python
+
+  from ost import settings
+  exe_path=settings.Locate('msms', search_paths=['/opt/app','/home/app'],
+              env_name='MSMS', search_system_paths=True)
+  print exe_path
+  
+The :func:`~ost.settings.Locate` command looks for the program with the name 
+`msms`. If env_name is set, it first looks if an environment variable with the 
+name `MSMS` is set. If not, all paths in search_paths are searched. If the 
+executable could still not be found and search_system_paths is set to True, the 
+binary search paths are searched. If the executable could not be found, a 
+:exc:`~ost.FileNotFound` exception is raised with a detailed description where 
+Locate was searching for the executable.
+    
+Prepare All Files
+--------------------------------------------------------------------------------
+
+The preparation of the necessary files is very dependent on the external program. Often it is useful to generate a temporary directory or file. For this, the python module tempfile is very handy.
+
+An example how to generate a temporary directory, open a file in this directory and write the position and radius of all atoms into this file is shown here:
+
+.. code-block:: python
+
+  import tempfile
+  import os
+  
+  # generate a temporary directory
+  tmp_dir_name=tempfile.mkdtemp()
+  print 'temporary directory:',tmp_dir_name
+  
+  # generate and open a file in the temp directory
+  tmp_file_name=os.path.join(tmp_dir_name,"entity")
+  tmp_file_handle=open(tmp_file_name, 'w')
+  print 'temporary file:',tmp_file_handle
+  
+  # write position and radius of all atoms to file
+  for a in entity.GetAtomList():
+    position=a.GetPos()
+    tmp_file_handle.write('%8.3f %8.3f %8.3f %4.2f\n' % (position[0],
+                          position[1], position[2], a.GetProp().radius))
+                          
+  # close the file
+  tmp_file_handle.close()
+
+Execute the External Program
+--------------------------------------------------------------------------------
+
+The external program can be executed from python using the python module subprocess.
+
+To run the external program msms from the above example, with the temporary file generated before, we can use the following:
+
+.. code-block:: python
+
+  import subprocess
+
+  # set the command to execute
+  command="%s -if %s -of %s" % (exe_path,
+            tmp_file_name, tmp_file_name)
+  print 'command:',command
+
+  # run the executable with the command
+  proc = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE)
+  stdout_value, stderr_value = proc.communicate()
+
+  #check for successful completion of msms
+  if proc.returncode!=0:
+    print "WARNING: msms error\n", stdout_value
+    raise subprocess.CalledProcessError(proc.returncode, command)
+
+  # print everything written to the command line (stdout)
+  print stdout_value
+    
+Read Generated Output
+--------------------------------------------------------------------------------
+
+The last step includes reading of generated files (like in the case of msms) and/or processing of the generated command line output.
+
+Here we first print the command line output and then load the generated msms surface and print the number of vertex points:
+
+.. code-block:: python
+
+  # print everything written to the command line (stdout)
+  print stdout_value
+  
+  # read msms surface from file
+  surface=io.LoadSurface(tmp_file_name, "msms")
+  print 'number of vertices:',len(surface.GetVertexIDList())

modules/doc/newmodule.rst

+Creating a New Module
+================================================================================
+
+OpenStructure can be extended by writing additional modules. A module will 
+usually consist of a set of C++ classes and methods, most of which will also be 
+exported to Python. It is also possible to write modules completely in Python.
+
+The build system of OpenStructure is quite simple. The main difference to other 
+projects is the use of a so-called stage  directory. The stage directory 
+replicates the normal layout of a standard Linux directory structure, with an 
+'include' directory for the headers, a 'lib' directory containing the shared 
+library files, a `bin` directory  for the executables and a 'share' directory 
+for the platform-independent data like icons, images and examples.
+
+OpenStructure uses `CMake <http://www.cmake.org>`_ to build the project. The 
+rules for the build-system are defined in `CMakeLists.txt` files. When running 
+`CMake <http://cmake.org>`_, the files are  compiled and copied into stage. The 
+real installation, if necessary, happens at a later stage. This is referred to 
+as staging of the files.
+
+If a new module is written following the guidelines in this page, it will be
+seamlessly included in the build system and will then be available form both
+the DNG python console and the OpenStructure command line as any other native 
+module.
+
+As a first step, a new directory structure must be created to accommodate the 
+new module.
+
+Directory Structure
+--------------------------------------------------------------------------------
+
+For the purpose of this example, let's assume we are creating a new module 
+called 'mod' (for 'modeling'). Let's create a directory named `mod` under the 
+'modules' directory in the OpenStructure development tree, and  populate it with 
+the three subdirectories `src`, `pymod`, and `tests`.  Then we add a 
+`CMakeLists.txt` file in the 'mod' directory, consisting of three lines:
+
+.. code-block:: bash
+
+  add_subdirectory(src)
+  add_subdirectory(pymod)
+  add_subdirectory(tests)
+  
+The Module Code
+--------------------------------------------------------------------------------
+
+In the `src` subdirectory we put the code that implements the functionality of 
+the new module, plus a `config.hh` header file.
+
+Here is a skeleton of one of the files in  the directory , `modeling_new_class.hh`:
+
+.. code-block:: cpp 
+
+  #ifndef OST_MOD_NEW_CLASS_H
+  #define OST_MOD_NEW_CLASS_H
+  
+  #include <ost/mod/module_config.hh>
+  
+  // All other necessary includes go here
+  
+  namespace ost { namespace mod {
+  
+  class DLLEXPORT_OST_MOD NewClass {
+   public:
+     void NewMethod();
+           
+    // All declarations of NewClass go here 
+  
+  };
+  
+  }} // namespaces
+  
+  #endif
+  
+And here is the skeleton of the corresponding `modeling_new_class.cc` file:
+
+.. code-block:: cpp
+
+  #include "modeling_new_class.hh"
+  
+  using namespace ost::mol;
+  using namespace ost::mod;
+  
+  // All other necessary includes and namespace directives
+  // go here
+  
+  void NewClass::NewMethod():
+  {
+    // Implementation     
+  }
+  
+  // Implementation code for NewClass goes here
+  
+Obviously, the `src` directory can contain many files, each implementing classes
+and functions that will end up in the module. In order to build and stage
+the module shared library, a `CMakeLists.txt` file needs to be written for the
+`src` directory:
+
+.. code-block:: bash
+
+  set(OST_MOD_SOURCES 
+  modeling_new_class.cc
+  // All other source files 
+  )
+  
+  set(OST_MOD_HEADERS
+  modeling_new_class.hh
+  // All other header files
+  )
+  
+   module(NAME mod SOURCES "${OST_MOD_SOURCES}"
+          HEADERS ${OST_MOD_HEADERS}
+          DEPENDS_ON mol mol_alg)
+  
+
+The line containing the `DEPENDS_ON` directive lists all the modules on which
+the new module depends (in this case :mod:`mol` and :mod:`mol.alg`).  The 
+`module` macro will take care of staging the headers, in this case into 
+`ost/mod` and compiling, linking and staging of a library with the  name 
+`libost_mod.so` (`libost_gmod.dylib` on MacOS X).
+
+.. note:: 
+
+  Due to a limitation in the built-int install command of CMake, for modules
+  that have their headers in several directories, it is required to group the
+  headers by directory, leading to a call of module like:
+
+.. code-block:: bash
+
+  module(NAME mol SOURCES atom_handle.cc impl/atom_impl.cc
+         HEADERS atom_impl.hh IN_DIR impl
+         atom_handle.hh)   
+
+The `module_config.hh` header is required for each module to setup the 
+environment on Windows: Each public class, method and function needs to marked 
+with `DLLEXPORT` or `DLLIMPORT` to teach the linker where to look for the 
+symbol. The correct use of either `DLLIMPORT` and `DLLEXPORT` is depending on 
+the context: While compiling a header file that is part of the same shared
+library, `DLLEXPORT` must be used. When compiling a header that is part of
+an external shared library, `DLLIMPORT` must be used. A typical module_config
+header looks like this:
+
+.. code-block:: cpp   
+
+  #ifndef OST_MOD_MODULE_CONFIG_HH
+  #define OST_MOD_MODULE_CONFIG_HH
+  
+  #include <ost/base.hh>
+  
+  #if defined(OST_MODULE_OST_MOD)
+  #  define DLLEXPORT_OST_MOD DLLEXPORT
+  #else
+  #  define DLLEXPORT_OST_MOD DLLIMPORT
+  #endif
+  #endif
+      
+The Testing Framework
+--------------------------------------------------------------------------------
+
+The `tests` directory contains code for unit tests. The code is compiled and 
+executed when one invokes compilation using the 'make check' command.  Tests are 
+run by means of the `Boost Unitests Library 
+<http://www.boost.org/doc/libs/1_37_0/libs/test/doc/html/index.html>`_, which is 
+used throughout OpenStructure. Before coding the test routines, the required 
+skeleton needs to be put in place.
+
+The main code is put into 'tests.cc', which will become the test executable:
+    
+.. code-block:: cpp
+
+  #include <boost/test/unit_test.hpp>
+  using boost::unit_test_framework::test_suite;
+  
+  #include "test_modeling.hh"
+  
+  test_suite*
+  unit_unit_test_suite( int argc, char * argv[] ) {
+    std::auto_ptr<test_suite> test(BOOST_TEST_SUITE( "Module Mod Test" ));
+  
+    test->add(CreateModelingTest()); 
+  
+    return test.release(); 
+  }
+  
+        
+The most relevant line adds the test suite for the new module to the global test 
+list. The test suite is created by the factory function CreateModelingTest(), 
+which is declared in the `test_modeling.hh` header file. 
+
+.. code-block:: cpp
+
+  #ifndef OST_MOD_TEST_MODELING_H
+  #define OST_MOD_TEST_MODELING_H
+  
+  #include <boost/test/unit_test.hpp>
+  using boost::unit_test_framework::test_suite;
+  
+  test_suite* CreateModelingTest();
+  
+  #endif
+      
+The definition of the factory function is found in the actual test code,
+which we put in `test_modeling.cc`. Here is a skeleton version of that file:
+
+.. code-block:: cpp
+
+  #include "test_modeling.hh"
+  
+  // additional include statements will go here
+  
+  namespace test_modeling {
+  
+    void test() 
+    {
+      // test code will go here
+    }
+  
+  }
+  
+  test_suite* CreateModelingTest()
+  {
+    using namespace test_modeling;
+    test_suite* ts=BOOST_TEST_SUITE("Modeling Test");
+     ts->add(BOOST_TEST_CASE(&test));
+  
+     return ts;
+  }
+  
+In this file, all the normal Boost Test Library macros and functions can be used. (For example `BOOST_CHECK`, `BOOST_FAIL`, etc.)
+
+Here is finally the build script skeleton that needs to be put into 
+`mod/tests/`:
+
+.. code-block:: bash
+
+  set(OST_MOD_UNIT_TESTS
+  tests.cc
+  test_modeling.cc
+  )
+  
+  ost_unittest(mod "${OST_MOD_UNIT_TESTS}")
+  target_link_libraries(ost_mol ost_mol_alg ost_mod)
+  
+In the last line the call to the 'target\_link\_libraries' function contains the 
+names of the modules on which the 'mod' unit test code depends (in this case, 
+the :mod:`mol` and :mod:`mol.alg` modules), in addition to the `mod` module 
+itself.
+
+The Python Wrapper
+--------------------------------------------------------------------------------
+
+Finally, the module API is exported to Python using the `Boost Python 
+Library <http://www.boost.org/doc/libs/1_37_0/libs/python/doc/index.html>`_.
+In `mod/pymod`, the wrapper code for the classes in the new module is put into a 
+file named `wrap\_mod.cc`:
+
+.. code-block:: cpp
+
+  #include <boost/python.hpp>
+  using namespace boost::python;
+  
+  #include <ost/mod/modeling_new_class.hh>
+  
+  using namespace ost::mol;
+  using namespace ost::mod;
+  
+  // All other necessary includes and namespace directives
+  // go here
+  
+  BOOST_PYTHON_MODULE(_mod)
+  {
+    class_<NewClass>("NewClass", init<>() )
+      .def("NewMethod", &NewClass::NewMethod)
+    ;
+  
+    // All other Boost Python code goes here    
+  }
+
+The `mod/pymod` directory must obviously contain a `CMakeLists.txt` file:
+
+.. code-block:: bash
+
+  set(OST_MOD_PYMOD_SOURCES
+  wrap_mod.cc
+  )
+  
+  pymod(NAME mod OUTPUT_DIR ost/mod 
+        CPP ${OST_MOD_PYMOD_SOURCES} PY __init__.py)
+  
+The directory should also contain an `__init.py__` file with the
+following content:
+
+.. code-block:: python
+  
+  from _mod import *
+    
+In case one wants to implement Python-only functionality for the new module, any
+number of function definitions can be added to the `__init.py__` file.
+
+That's it!. The next time the OpenStructure project is compiled, the new module 
+will be built and made available at both the C++ and the Python level.

modules/index.rst

@@ -4,11 +4,35 @@ OpenStructure documentation
 .. toctree::
   :maxdepth: 2
   
+Introduction
+--------------------------------------------------------------------------------  
+
+.. toctree::
+  :maxdepth: 2
+
   intro
   install.rst
+  
+Modules
+--------------------------------------------------------------------------------
+
+.. toctree::
+  :maxdepth: 2
+
+  base/generic
   img/base/img
   img/alg/alg
   geom/geom
   conop/conop
   mol/base/mol
   seq/base/seq
+
+Extending OpenStructure
+--------------------------------------------------------------------------------
+
+.. toctree::
+  :maxdepth: 2
+  
+  newmodule
+  external
+  
\ No newline at end of file