CartoWeb 3.5.0
Download Now!


CartoWeb
Advanced Geographical Information System for the Web

Automatic project deployment and management

Introduction

This page describes a mechanism that facilitates working with versioned CartoWeb(s) and deploying to remote environments.

Making your project deployable

To make your project deployable, configuration elements which depend on where the project is run must be handled specifically.

For instance, if your client.ini contains:

cartoclientBaseUrl = http://example1.com/cartoweb

The project will run fine on the server example1 in a specific path, but it won't run any more once deployed on the server example2.com or if it is deployed on another path on this server, because the cartoclientBaseUrl is not the same.

That's why a mechanism exists to handle this situation. This mechanism works by generating configuration files, and replaces server/path dependent configuration elements using variables.

.in files

Any configuration file that contains location-specific variables should be renamed to a file with a ".in" suffix, and all those server/path dependent variables must use template variables which will be replaced dynamically.

As an example, let's say you have the following client.ini file in your project:

cartoclientBaseUrl = http://example1.com/cartoweb
profile = production

You must rename it to client.ini.in and change its content to:

cartoclientBaseUrl = "@CARTOCLIENT_BASE_URL@"
profile = @PROFILE@

During deployment, the client.ini.in file will generate the client.ini file, replacing @CARTOCLIENT_BASE_URL@ and @PROFILE@ with the values contained in a project deployment file (see next paragraph) that is specific to the local path.

  • Note 1 : this mechanism can be used for mapfiles too; this comes particularly handy for PostGIS layers :

[...]
  CONNECTIONTYPE postgis
  CONNECTION "user=@DB_USER@ password=@DB_PASSWD@ dbname=@DB_NAME@ host=@DB_HOST@ port=@DB_PORT@"
[...]
  • Note 2 : only the .in file must be saved in CVS repository.

Project deployment files

The project deployment file is a configuration file which contains the values of the variables which have to be replaced during the generation of configuration files from the <.in> files. It is located in the "deployment" directory of your project, and its name contains the current server name and the directory where cartoweb is located.

To see the exact name of the project deployment file you have to use, see "The first fetch" section (for instance, if you want to deploy on hostname "example1.com" in directory "/home/alice/cartoweb3", the file name will be config_example1.com_home_alice_cartoweb3.properties in the deployment directory of your project).

A deployment configuration file contains the values of the variables, according to the following syntax:

PROFILE = production
CARTOCLIENT_BASE_URL = "http://example1.com/~alice/cartoweb3"

If you want to deploy the same project elsewhere, you'll need another deployment configuration file for this second location.

Configuration of the automatic fetch & deploy

Concepts

The fetch & deploy architecture can suppport a set of multiple cartowebs, possibly retrieved with different cvs versions, called cartoweb instances.

Each instance can contain several cartoweb projects

The directory layout is the following

  • Makefile: the file containing the deploy configuration
  • cartowebs : contains the instances; each instance contains a full-featured cartoweb and all projects which are attached to this instance.
  • deploy : contains the deploy scripts; do not modifiy these.
  • htdocs :

    for convenience, a symbolic link is created in this directory for each project, pointing to the htdocs of the instance containing the project. Thus, the webserver has to be given access to this directory, and everything will be available (of course, this can be tuned depending on the desired web server configuration).

Configuration

The Makefile contains the following definitions blocks:

instances

projects

hostnames

The sample file cartoweb3/scripts/deploy/Makefile.sample contains documentation on these different variables. It should be used as a starting point.

Fetching

The first fetch

Once you have your Makefile setup and your project available on CVS, you can run the initial instance fetch, so that you can see the deployment filename to use.

Type the command inside the directory containing the Makefile, by replacing YOUR_INSTANCE by the instance name you want to use:

make fetch_instance/YOUR_INSTANCE

This will display the following error:

Error message: Can't find project config file. It should be in one of the path (tried in order):

projects/example1/deployment/config_example1.com_home_alice_public_html_cartowebs_example1_cartoweb3.properties
projects/example1/deployment/config_example1.com_home_alice_public_html_cartowebs_fpds.properties
projects/example1/deployment/config_example1.com.properties
projects/example1/deployment/config.properties

This means that you have to create a file named "config_example1.com_home_alice_public_html_cartowebs_example1_cartoweb3.properties" in the deployment directory of your project containing server/path dependent variables. See the previous section for an example.

Fetching commands

Once this file is configured, the following commands can be used to manage deployment: These commands have to be used from the directory containing the Makefile:

make fetch_all_instances

  • Fetch all instances and projects

make fetch_instance/INSTANCE

  • Fetch the instance named INSTANCE with all its projects

make fetch_project/PROJECT

  • Fetch the project called PROJECT

Deploying

Here is a situation where you need an automatic deploy process. Say you have three servers :

  • Server 0

    • Development server
    • cartoweb cvs access + http access for archive fetching
    • project cvs access

      -> deployment to server 1

  • Server 1

    • Validation server
    • No cvs access

      -> deployment to server 2

  • Server 2

    • Production server

Makefile configuration

In the hostname part of the Makefile, you have to define :

server0_TARGET_HOST := server1
server1_TARGET_HOST := server2

Deploying commands

make deploy_all_instances

  • Deploys all instances to the next server

make deploy_instance/INSTANCE

  • Deploys instance named INSTANCE to the next server

In this configuration, you have to be on server1 to deploy to server2.

Example

Take the following file as a starting point:

http://cvs.camptocamp.com/cgi-bin/viewcvs.cgi/*checkout*/cartoweb3/scripts/deploy/Makefile.sample?rev=HEAD

 cvs -d :pserver:anonymous@dev.camptocamp.com:/var/lib/cvs/public co cartoweb3/scripts/deploy/Makefile.sample
 mv cartoweb3/scripts/deploy/Makefile.sample Makefile
 rm -rf cartoweb3/

Edit the Makefile, replacing the instance definition and project definitions with (if necessary change USERNAME to your username):

[...]

# ###################################
# Instances definitions

elk_REVISION := "-D 2020-01-26"

# This MUST contain all instance names
ALL_INSTANCES := elk

# Project definitions
# ###################################


default_CVSROOT := :pserver:USERNAME@source.c2c:/var/lib/cvs/projects/cw3

elk_INSTANCE := elk

foobar_INSTANCE := elk
foobar_CVSROOT := :pserver:bob@example.com./var/lib/cvs/

# This MUST contain all project names
ALL_PROJECTS := elk foobar

[...]

As you can see, several projects can be defined. If you want to add a new project, add at least the <project>_INSTANCE line and update the ALL_PROJECTS variable. Other variables can optionally be given, like the CVSROOT of the project, using the <project>_CVSROOT variable.

Now you should be able to type:

make fetch_instance/elk

(or make fetch_instance/elk NO_CONFIRM=1 to avoid confirmation message)

You may see the error message:

Error message: Can't find project config file. It should be in one of the path (tried in order):

projects/elk/deployment/config_sarge_home_sypasche_public_html_deploy_cartowebs_elk_cartoweb3.properties
projects/elk/deployment/config_sarge_home_sypasche_public_html_deploy_cartowebs_elk.properties
projects/elk/deployment/config_sarge.properties
projects/elk/deployment/config.properties

Now create the deployment configuration file:

cd cartowebs/elk/cartoweb3/projects/elk/deployment/
echo 'CARTOCLIENT_BASE_URL = "http://zuort.c2c/~sypasche/deploy/htdocs/elk"' > config_sarge_home_sypasche_public_html_deploy_cartowebs_elk_cartoweb3.properties
 cvs ci -m added config_sarge_home_sypasche_public_html_deploy_cartowebs_elk_cartoweb3.properties
cd ../../../../../..

And relaunch the deploy:

make fetch_instance/elk NO_CONFIRM=1

And it will automatically:

  • fetch deploy scripts
  • fetch cartoweb instance with the given revision
  • fetch project inside instance
  • launch cw3setup install scripts
  • create link into htdocs

Now the application should be available in the htdocs/elk directory.

To deploy to the next server with rsync:

First configure the hostname definition section correctly. Then you can type the following command:

make deploy_instance/elk

Deploy update

If you see the message: Warning: A new version of the deploy script is available. Press enter to continue, or control-c to abort so that you can update

You must first update the deploy version. See the Changes section below, and pick a date (or a CVS Tag). Then edit your Makefile to update the DEPLOY_REVISION variable. For instance:

DEPLOY_REVISION := -D 2020-02-20

Then you need to delete the deploy directory

rm -rf deploy

The script will automatically fetch the latest deploy directory again from CVS. You might then need to update your Makefile if you see the message mentionned on the next section.

Makefile update

If you recieve a message like Your Makefile version X is not compatible with the new deploy version Y, see the following migration guide, and update your Makefile and COMPAT_VERSION variable.

The COMPAT_VERSION variable can be found at the top of the Makefile:

COMPAT_VERSION := 0

Changes

2020-01-26: First version

Migration guide

Version 0 -> version 1 Migration:

Nothing yet

CartoWeb Wiki: AutomaticDeployment (last edited 2020-06-05 10:27:10 by OliverChristen)

© 2002-2007 CartoWeb by Camptocamp SA - Wiki powered by MoinMoin