[cig-commits] r16378 - cs/portal/trunk/doc/admin
leif at geodynamics.org
leif at geodynamics.org
Wed Mar 3 19:39:04 PST 2010
Author: leif
Date: 2010-03-03 19:39:03 -0800 (Wed, 03 Mar 2010)
New Revision: 16378
Added:
cs/portal/trunk/doc/admin/move-daemon.rst
Modified:
cs/portal/trunk/doc/admin/deps.rst
cs/portal/trunk/doc/admin/index.rst
cs/portal/trunk/doc/admin/quick.rst
Log:
Wrote a new chapter describing how we moved the daemon from Crust to
the main server.
Modified: cs/portal/trunk/doc/admin/deps.rst
===================================================================
--- cs/portal/trunk/doc/admin/deps.rst 2010-03-03 20:45:07 UTC (rev 16377)
+++ cs/portal/trunk/doc/admin/deps.rst 2010-03-04 03:39:03 UTC (rev 16378)
@@ -14,7 +14,7 @@
- SQLite
* GMT
+ netCDF
-* `ImageMagic http://www.imagemagick.org/`_
+* `ImageMagic <http://www.imagemagick.org/>`_
Note that both Python v3.0 and Django v1.0 are dramatically different from prior versions (they are not backwards compatible). Consequently, the portal does not work with Python v3.0 or Django v1.0.
Modified: cs/portal/trunk/doc/admin/index.rst
===================================================================
--- cs/portal/trunk/doc/admin/index.rst 2010-03-03 20:45:07 UTC (rev 16377)
+++ cs/portal/trunk/doc/admin/index.rst 2010-03-04 03:39:03 UTC (rev 16378)
@@ -15,6 +15,7 @@
quick.rst
deps.rst
web-server.rst
+ move-daemon.rst
Indices and tables
Added: cs/portal/trunk/doc/admin/move-daemon.rst
===================================================================
--- cs/portal/trunk/doc/admin/move-daemon.rst (rev 0)
+++ cs/portal/trunk/doc/admin/move-daemon.rst 2010-03-04 03:39:03 UTC (rev 16378)
@@ -0,0 +1,185 @@
+
+*****************
+Moving the Daemon
+*****************
+
+This chapter describes how to move the heart of the portal backend -- the daemon -- from one machine to another.
+
+The daemon must be run on a host which is also running a web server. This is so that users can download simulation output files via HTTP. The daemon uses ``scp`` to copy the output files from the TeraGrid cluster; the daemon is configured to place them under ``~/public_html``, where they can be served by Apache.
+
+The daemon is flexible enough to run on a machine distinct from the one which hosts the main Django app. However, both the daemon and the Django app are typically run on the same server.
+
+The daemon is usually run under its own Unix account (``portal``) but this is not a requirement.
+
+
+Killing the Old Daemon
+**********************
+
+Before you begin, shut down the daemon on the old server, so that the portal's state does not change during the move::
+
+ $ ps aux | grep stackless
+ portal 15467 0.0 0.0 71704 15712 ? S 14:36 0:02 stackless daemon.py
+ portal 17587 0.0 0.0 5168 856 pts/0 S+ 17:46 0:00 grep stackless
+ $ kill -INT 15467
+ $
+
+Before you do this, make sure there are no active runs. One way to do this is to check the hidden ``list.py`` page in your web browser (i.e., ``geodynamics.org/portals/seismo/<secret>/runs/list.py``). If the last few entries have a ``status`` of either ``done`` or ``error``, then the portal is probably idle.
+
+
+Output Files
+************
+
+The output files for the seismo portal consume many gigabytes, so it is best to start copying them right away::
+
+ $ scp -r ~portal/public_html/output portal at www.geodynamics.org:/home/portal/public_html
+
+**Note:** The Django app has support for purging the output files of a run. However, it is not as simple as merely removing the files on disk. The corresponding ``ArchivedRun`` records in the database must also be removed so that the Django app knows that the run as been purged. A script to purge old runs could be written with little effort.
+
+
+Dependencies
+************
+
+The daemon.py script has the following dependencies:
+
+* `Stackless Python <http://www.stackless.com/>`_ for Python v2.x (but not Python v3.x).
+
+* `CIG Pythia 0.8.1.x <http://pypi.python.org/pypi/pythia/>`_.
+
+So the first step is to install Stackless on the destination server::
+
+ $ wget http://www.stackless.com/binaries/stackless-264-export.tar.bz2
+ $ tar xjf stackless-264-export.tar.bz2
+ $ cd stackless-2.6.4
+ $ ./configure --prefix=$HOME/opt/stackless
+ $ make && make install
+
+Next, create a symlink on your ``PATH`` so that you can easily run Stackless Python::
+
+ $ cd ~/bin
+ $ ln -s $HOME/opt/stackless/bin/python stackless
+
+Note how we name the symlink ``stackless``, so as not to conflict with the system ``python``.
+
+Install CIG Pythia under the same directory where you installed Stackless::
+
+ $ wget http://pypi.python.org/packages/source/p/pythia/pythia-0.8.1.10.tar.gz
+ $ tar xzf pythia-0.8.1.10.tar.gz
+ $ cd pythia-0.8.1.10
+ $ stackless setup.py install --prefix=$HOME/opt/stackless
+
+Note how we used ``stackless`` to build and install Pythia. This is especially important if the version of Stackless does not match the system ``python``.
+
+
+Authentication
+**************
+
+The daemon uses ``ssh`` and ``scp`` to connect to the remote TeraGrid cluster. This is acheived though a "passphrase-less" SSH key installed on the TeraGrid cluster.
+
+For security reasons, the SSH public key listed in ``~/.ssh/authorized_keys`` on the TeraGrid is restricted using ``from="geodynamics.org"``. You may have to update this so that the daemon can connect from its new host.
+
+Most importantly, you must copy the corresponding public/private key pair, ``~/.ssh/daemon_id_rsa.pub`` and ``~/.ssh/daemon_id_rsa``, from the daemon's old host to the new one.
+
+
+Installing daemon.py
+********************
+
+You are now ready to create a working directory for the daemon -- home of the ``daemon.py`` script, its configuration file, and its log file. You may copy ``daemon.py`` from the old host, or simply grab it from the Subversion server::
+
+ $ mkdir ~/daemon
+ $ cd ~/daemon
+ $ svn cat http://geodynamics.org/svn/cig/cs/portal/trunk/northridge/backend/daemon.py > daemon.py
+
+
+.. _config:
+
+Configuration
+*************
+
+Next, copy ``web-portal-daemon.cfg`` from the old host to the ``~/daemon`` directory you just created on the new host. You need change only a handful of lines in this file (if any): those corresponding to the *output root*, and those lines which mention the SSH private key.
+
+First, there is the output root::
+
+ [web-portal-daemon]
+ output-root-pathname = /home/portal/public_html/output/
+ output-root-url = http://crust.geodynamics.org/~portal/output/
+
+ #...
+
+The daemon sends ``output-root-url`` to the Django app (using HTTP POST) to identify the base URL from which output files may be downloaded. This needs to changed to reflect the daemon's new home (assuming the hostname or path is different)::
+
+ output-root-url = http://www.geodynamics.org/~portal/output/
+
+If you placed the output files in a different location, you will need to change ``output-root-pathname`` as well.
+
+Second, there are three lines which mention the passphrase-less SSH private key used for connecting to the TeraGrid cluster. If the path to this file is different on the new host, update the ``ssh`` and ``scp`` configuration lines accordingly.
+
+
+Starting the Daemon
+*******************
+
+Congratulations! You are now ready to start the daemon::
+
+ $ cd ~/daemon
+ $ nohup stackless daemon.py &
+
+**Important:** The daemon is not smart enough to disconnect from the terminal or go into the background by itself. So, both the ``nohup`` and the ampersand are critical to ensure that the daemon lives on after you logout.
+
+Two log files should have been created in the working directory: ``nohup.out`` and ``journal.log``. Use the following command to monitor the daemon's activity::
+
+ $ tail -f journal.log
+
+
+Cron
+====
+
+It is good idea to arrange for the daemon to start automatically at boot. On Linux -- or any system with Vixie Cron -- this is easily acheived using ``crontab``::
+
+ $ cat >crontab <<EOF
+ > PATH=$PATH
+ > PYTHONPATH=$PYTHONPATH
+ >
+ > @reboot cd /home/portal/daemon && nohup stackless daemon.py &
+ >
+ > EOF
+ $ crontab crontab
+ $
+
+
+**Important:** The daemon *must* be started from its special working directory. It uses the current working directory to find the ``web-portal-daemon.cfg`` configuration file.
+
+
+Output Files Revisited
+**********************
+
+Remember how you ran ``scp`` to copy the output files from the old host to the new one? Now is a good time to check on its progress.
+
+Once the copy is complete, there is one final task to perform.
+
+The old job records in the database still point to the web server on the daemon's old host. If the URL for the new host is different (and you therefore changed the ``output-root-url`` setting as described under :ref:`config`), you need to update the records in the database to point to the new server. This is achieved by running a simple Python script on the machine hosting the Django app::
+
+ #!/usr/bin/env python
+
+ import SeismoWebPortal.models as m
+
+ for j in m.Job.objects.all():
+ j.url = j.url.replace("crust", "www")
+ j.save()
+
+Obviously, you should customize this code as appropriate. Since this script accesses the database using Django, your enviroment must be configured to use Django, and you need to have write permission to the database. (Depending on your setup, the latter may involve becoming ``www-data`` using ``su``.)
+::
+
+ $ cd ~/www-data
+ $ . www-setup.sh
+ $ ./update-job-url.py
+
+If you don't have a copy of ``www-setup.sh``, use `debug-setup.sh <http://geodynamics.org/svn/cig/cs/portal/trunk/northridge/debug-setup.sh>`_ as a template.
+
+
+Testing
+*******
+
+To verify that the new setup works:
+
+* Visit the portal web site and start a new run. In a terminal window, use the ``tail -f journal.log`` command to monitor the daemon's progress as executes the new run.
+
+* Browse to an old run and try to download one of its output files. Take note of the output file's URL. This verfies that the old output files were copied to their new location, and that the database was updated accordingly (if necessary).
Modified: cs/portal/trunk/doc/admin/quick.rst
===================================================================
--- cs/portal/trunk/doc/admin/quick.rst 2010-03-03 20:45:07 UTC (rev 16377)
+++ cs/portal/trunk/doc/admin/quick.rst 2010-03-04 03:39:03 UTC (rev 16378)
@@ -18,7 +18,7 @@
* `Python <http://python.org/>`_ v2.5 [#sqlite]_ or later (but not Python v3.0).
* `Django <http://www.djangoproject.com/>`_ v0.96 (v1.0 and later will not work)
* `GMT <http://gmt.soest.hawaii.edu/>`_; and the `netCDF <http://www.unidata.ucar.edu/software/netcdf/>`_ package upon which GMT depends (these two are optional for the impatient)
-* `ImageMagic http://www.imagemagick.org/`_
+* `ImageMagic <http://www.imagemagick.org/>`_
See :ref:`web-app-dependencies` for more information.
More information about the CIG-COMMITS
mailing list