CartoWeb 3.5.0
Download Now!


CartoWeb
Advanced Geographical Information System for the Web

CartoWeb Accounting

Accounting is available since CartoWeb from date 2020-02-23.

See documentation here: http://www.cartoweb.org/doc_head/docbook/xhtml/user.accounting.html

Patch to Store IP Addresses for CartoWeb Versions Before 2020-03-23

Available here:

Instructions for applying it:

cd cartoweb3
wget -O accounting_ip.diff 'http://www.cartoweb.org/cwiki/Accounting?action=AttachFile&do=get&target=accounting_ip.diff'
patch -p0    < accounting_ip.diff

What information is stored

log name

description

since version

General plugin version:1 (2020-02-23:0, 2020-03-23:1)

export_plugin

Name of the export plugin used, for instance exportHtml, exportPdf

0

browser_info

Information about the browser, like Java support, window size, number of colors, ...

0

mapid

Mapfile identifier, for instance test_main.test

0

time

Unix time when the request is made

0

ip

Ip address of client (works behind proxy / Security Proxy)

1

ua

User agent string of the client browser

0

sessid

Session identifier

0

direct_access

1 if direct access mode is enabled

0

request_id

Internal use

0

security.user

Current logged in user identifier

0

elapsed_time

Time taken for request on server

0

cache_id

Internal use for cache management

0

cache_hit

Internal use for cache management

0

ExportPdf plugin version:1 (2020-02-23:1)

format

Format name, for instance A4

1

resolution

Resolution, for example 72

1

Images plugin version:0 (2020-02-23:0)

mainmap.width

Mainmap width

0

mainmap.height

Mainmap height

0

Layers plugin (server) version:1 (2020-02-23:1)

layers

Coma sepearated list of selected layers

0

switch_id

The selected switch id

1

Layers plugin (client) version:0 (2020-02-23:0)

visible_layers

Coma sepearated list of visible layers (the selected layers witch are not out-of-scale)

0

Location plugin version:0 (2020-02-23:0)

bbox

Current bbox of request

0

scale

Scale of the map

0

Query plugin version:1 (2020-02-23:1)

results_count

Number of results found

1

results_table_count

Number of results for each layers, for instance POLYGON1=1;some_rectangles=4"

1

Importing the log files into the database

Use the manage_logs.py script from the scripts/accounting directory in CartoWeb.

Usage example:

1) First create the tables:

DSN="dbname=accounting user=my_user password=my_password" python manage_logs.py -gen-schema

2) Then, populate the tables with the log files:

DSN="dbname=accounting user=my_user password=my_password" python manage_logs.py -import /path/to/accounting

DSN contains the information for connecting to the database. In the database, three table will be created: stats, stats_client and stats_server. The resulting data will be stored in the stats table.

/path/to/accounting must be a directory where you can find the "project.mapid/*log" files. Inside of CartoWeb, it can be found in "cartoweb3/www-data/accounting".

You can have some basic statistics by invoking:

DSN="dbname=accounting user=my_user password=my_password" python manage_logs.py -raw_stats

Performance information:

  • on a P4 2.8Ghz, a 39697 lines log file is processed in 1m52. That means ~350 lines/seconds.

Random notes: getting the percentage of incorrect values (because of the cached entry present in a previous log file):

select (select count (1)::float from stats where images_mainmap_height is null) / (select count (1)::float from stats);

SQL commands for analysing stats

Logs from date A to date B : SELECT min (general_time), max (general_time) FROM stats;

Number of maps diplayed by month: SELECT extract(month from  general_time), count(*) FROM stats GROUP BY extract(month from  general_time);

Count maps in different mapsizes: SELECT images_mainmap_height, images_mainmap_width, COUNT(*) AS count FROM stats GROUP BY images_mainmap_height, images_mainmap_width HAVING count(*) > 100 ORDER BY images_mainmap_height ASC;

Count maps in different mapsize for january: SELECT images_mainmap_height, images_mainmap_width, COUNT(*) AS count FROM stats GROUP BY images_mainmap_height, images_mainmap_width, extract(month from general_time) HAVING count(*) > 100 AND extract(month from general_time) = 1 ORDER BY images_mainmap_height ASC;

Count maps in different mapsizes per month for 2007: SELECT extract(month from general_time), images_mainmap_height AS height, images_mainmap_width AS width, images_mainmap_height * images_mainmap_width AS pixels, COUNT(*) AS count FROM stats GROUP BY images_mainmap_height, images_mainmap_width, extract(month from general_time), extract(year from general_time) HAVING count(*) > 100 AND extract(year from general_time) = 2007 ORDER BY date_part, images_mainmap_height ASC;

Count maps and pixels per month for 2007: SELECT substr(date(general_time),0,8) AS month, images_mainmap_height AS height,images_mainmap_width AS width, images_mainmap_height * images_mainmap_width AS pixels_per_map , COUNT(*) AS count_maps , images_mainmap_height * images_mainmap_width * COUNT(*) AS total_pixels FROM stats GROUP BY images_mainmap_height, images_mainmap_width, month , extract(year from general_time) HAVING  count(*) > 10  AND extract(year from general_time) = 2007 ORDER BY month ASC, images_mainmap_height ASC, images_mainmap_width ASC;

Count maps and pixels per day for may 2007: SELECT extract(day from general_time) AS day, images_mainmap_height AS height, images_mainmap_width AS width, images_mainmap_height * images_mainmap_width AS pixels_per_map, COUNT(*) AS count_maps, images_mainmap_height * images_mainmap_width * COUNT(*) AS total_pixels FROM stats GROUP BY images_mainmap_height, images_mainmap_width, extract(day from general_time), extract(month from general_time), extract(year from general_time) HAVING count(*) > 10 AND extract(year from general_time) = 2007 AND extract(month from general_time) = 05 ORDER BY day ASC, images_mainmap_height ASC, images_mainmap_width ASC;

How many PDF exports? SELECT exportpdf_format, exportpdf_resolution, COUNT(*) FROM stats GROUP BY exportpdf_format, exportpdf_resolution;

How many pdf exports per month, size and resolution: SELECT extract(month from general_time) as month, exportpdf_format AS pdf_format, exportpdf_resolution AS pdf_resolution, COUNT(*) AS count_pdf FROM stats WHERE exportpdf_format is not null GROUP BY exportpdf_format, exportpdf_resolution, extract(month from general_time), extract(year from general_time) HAVING extract(year from general_time) = 2007 ORDER BY month ASC, exportpdf_format ASC, exportpdf_resolution ASC;

CartoWeb Wiki: Accounting (last edited 2020-01-29 09:11:58 by SylvainPasche)

© 2002-2007 CartoWeb by Camptocamp SA - Wiki powered by MoinMoin