============================
WebCoso - web site generator
============================
:Id: SW/WebCoso
:CreationDate: 2009-08-18 11:26:45
:tags: - software
       - perl

The story
=========

Read `the pre-history <../sitemake/>`_ if you're interested.

The old system worked well enough, but I was getting tired of its
limitations:

* the input was XHTML with annotations (for languages), and writing
  XHTML by hand is a bit painful
* the structure was very rigid:

  - having more than two nesting levels of categories would have
    required heavy changes to the compiler
  - navigation would not have been not very comfortable, if I had
    added many categories

* these was no way to assign a document to more than one category
  (the things that nowadays are called "tags")

It was evidently the time to write (or find) something better.

The functions I wanted were:

- input in `reStructuredText`_
- multi-language support:

  * each document can exists in different languages
  * the user can go from one language to the other via links
  * the web server must choose the right version using `content
    negotiation`_

- presentation defined in XSLT_
- each document must be able to define a specific presentation
- support for tags
- reduce to a minimum the amount of writing to create a new document

After a few years of sporadic development, the result is WebCoso_

.. _reST:
.. _reStructuredText: http://docutils.sourceforge.net/rst.html
.. _`content negotiation`: http://httpd.apache.org/docs/2.2/content-negotiation.html
.. _XSLT: http://www.w3.org/TR/xslt.html
.. _WebCoso: http://www.thenautilus.net/cgit/WebCoso/

How does it work
================

At the `heart of the thing`_ is a set of declarations for
`Slay::Maker`_; these define two passes [#]_:

.. [#] asterisks stand for the language code

1) generation of XML:

   * files called ``document.*.rest.tt`` are processed by
     `Template Toolkit`_, producing files called
     ``document.*.rest.txt``
   * files called ``document.*.rest.txt`` are processed by
     `Text::Restructured`_ producing files called
     ``document.*.du.xml``
   * files called ``feed.*.tt`` are processed by `Template
     Toolkit`_ producing files called ``feed.*.xml``
   * from `reST`_ documents, the system extracts the tags with which
     each document is associated, and the dates of creation and last
     modification
   * documents in the ``tags`` directory are processed like the
     others, but only after the tag extraction

2) generation of (X)HTML and feeds:

   * files called ``document.*.du.xml`` are processed by
     `XML::LibXSLT`_ using the "stylesheet" ``du2html.xsl`` in the
     same directory, producing files called ``document.*.html`` which
     are stored in the destination directory
   * files called ``feed.*.xml`` are processed by `XML::LibXSLT`_
     using the "stylesheet" ``fillFeed.xsl`` in the same directory,
     producing files called ``feed.*.xml`` which are stored in the
     destination directory

Note that the modules that interface `Template Toolkit`_,
`Text::Restructured`_ and `XML::LibXSLT`_ to WebCoso_, export various
extension functions to allow access to the various data extracted
(tags, dates).

At this point all files have been generated, and a couple of rsync_
(one for the documents, one for images css etc.) will update the copy
on the web server.

Future developments
===================

Feed generation is sub-optimal: to get the full text of the documents
inside the feeds, each document is transformed multiple times; I
should probably add a third pass, specific for feeds, to use the
already transformed documents.

There are, surely, functions that are not exported to the templates,
and that one day I'll need: they're not hard to add.

.. _`heart of the thing`: http://www.thenautilus.net/cgit/WebCoso/tree/lib/WebCoso/Maker.pm
.. _`Slay::Maker`: http://search.cpan.org/dist/Slay-Maker/
.. _`Template Toolkit`: http://search.cpan.org/dist/Template-Toolkit/
.. _`Text::Restructured`: http://search.cpan.org/dist/Text-Restructured/
.. _`XML::LibXSLT`: http://search.cpan.org/dist/XML-LibXSLT/
.. _rsync: http://www.samba.org/rsync/