Commit 2bfc15cf authored by Moritz Schott's avatar Moritz Schott
Browse files

add documentation

parent 28e85465
# OSM Element Vectorisation
This tool transforms single OSM elements into feature vectors for attribute investigation and machine learning.
# Comments
these queries would be much easier with a postgers_fdw
but that showed very bad performance running nested loops
with no server aggregation.
Also setting that up automatically is extra effort and creates other pitfalls.
## move to fs documentation
/*
* User localness has been discussed and analysed in multiple studies (e.g. Neis).
* The following implementation calculates the median distance between the target object and all changesets issued by its last editor .
* The further the object from the median the less local the user to the area of the object.
* Changesets have certain inaccuracies e.g. their bbox will extend to a large region if they edit relations or objects are
* 'diagonal'.
* An alternative would be the average location or the median distance to the corners of the changesets. This would penalise
* more for large changesets which are relatively rare but a sign of bad quality.
*
* Another issue is the fact that only the localness of the most recent editor of an objects is calculated.
*/
-- User localness queries are currently part of the main script
# Installation
## TL;DR
Run `sudo ./vectorisation_docker.sh repex`. _Something_ will happen!
## Setup
The package uses [poetry](https://python-poetry.org) as build tool and dependency management. Run `poetry install` to install the package (it is _not yet_ available on pypi) and `poetry run vectorise` to run the tools CLI. You can of course also import the package into your module and use the python API. Yet, the backend requires a certain layout.
We therefore provide a Docker workflow to get you started:
### Docker
Both [Docker](https://www.docker.com/) and [docker-compose](https://docs.docker.com/compose/) need to be installed. If you do not want to use Docker, you can extract the _import_ procedure from this [dockerfile](Docker/dockerfiles/Dockerfile.data) and the _installation_ procedure from this [dockerfile](Docker/dockerfiles/Dockerfile). In that case you must also set up your own backend database (e.g. like [here](Docker/dockerfiles/Dockerfile.postgis)).
The docker workflow has five modes:
- _repex_ for a minimum reproducible example
- _benchmark_ to run a standardised benchmark
- _full_ to create a complete backend. **NOTE**: this will take several hours and needs considerable resources!
- _custom_ to execute the tool against your own database with your custom parameters
- _run_ to run the tool against the provided database with your custom parameters
## Description
To use the workflow execute
This tool transforms single OSM elements into feature vectors for attribute investigation and machine learning. It does that by calculating indicators for the OSM elements. These characterise the OSM object, e.g. based on its edit history, properties of the OSM user who created or modified the object or by OSM objects in the surrounding of the OSM object. The indicators provide new data insights but can also serve as feature vectors/predictors/explanatory variables in machine learning applications - e.g. to estimate data quality.
`./vectorisation_docker.sh <mode> <args>`.
Calculating (quality) **indicators** for [OSM](https://www.openstreetmap.org) can be a complex task combining multiple sources and analyses platforms. We therefore introduce an automated workflow to make data analyses as easy as possible. By providing a minimum amount of configuration, any user will be able to download OSM data, calculate indicators and predict quality using a pretrained ML-model.
The `<args>` are only useful for modes _custom_ and _run_ and will be passed to the tools [CLI](cli-docu).
The workflow is a result of the [IDEAL-VGI](https://www.geog.uni-heidelberg.de/gis/ideal_en.html) project funded by [DFG](https://gepris.dfg.de/gepris/projekt/424966858). The project focuses on land-use and land-cover (LULC) data. The workflow is therefore specifically useful for LULC data, especially polygons. _Use with caution outside this domain!_ The workflow targets single OSM objects, yet the results can mostly be aggregated to a courser resolution. Not all indicators may though be suitable. The same applies for other objecttypes like Nodes and Ways where the indicator calculation may need adaption or rethinking.
## Backend
For information on installing and running the tool as well as the workflows' documentation see:
The tool uses a [postgres](https://www.postgresql.org/) - [postgis](https://postgis.net/) database as backend. [docker-compose.yaml](Docker/docker-compose.yaml) provides a basic setup but of course you should adapt it to your needs (user management, security etc.).
- [Workflow](doc/workflow.md)
- [Indicators](doc/indicators.md)
- [How to install and run](doc/installation_run.md)
- [Configuration](doc/config.md)
- [Data](doc/data.md)
- [Benchmarking](doc/benchmark.md)
# Contribution Guidelines
......
## Benchmark
A benchmark test was run on **856** objects around Eppingen (Baden-Württemberg, Germany: {"targetBbox": "8.823079,49.08726,8.998173,49.182279", "timestamp": "2020-05-04"}) with an average _1.8_ minior versions (tag changes or geometry changes):
![](benchmarking/benchmark_automatic_workflow.png)
### Benchmark howto
Run `./vectorisation_docker.sh benchmark`
<mxfile host="app.diagrams.net" modified="2021-07-12T09:55:42.647Z" agent="5.0 (Windows)" etag="FXqmj7HEwLy7gDAaK-18" version="14.8.5" type="device"><diagram id="2fjUBvsxHqN9l28NOZ2B" name="Page-1">7V1rc5u4Gv41mel+iEcgrh/bpM3OntNzOpvT3e1+8WBQbLYYOSAndn79kTBgEDKXGBDOpLudWkKA0Pu8V0mvruDNencXOZvVV+yh4EoF3u4K3l6pqqJZkP7DavaHGtMwDhXLyPfSRseKe/8FpZUgrd36HopLDQnGAfE35UoXhyFySanOiSL8XG72gIPyWzfOElUq7l0nqNb+6Xtkdai1dHCs/xX5y1X2ZgWkV9ZO1jitiFeOh58LVfDzFbyJMCaHX+vdDQrY4GXjcrjvy4mrecciFJI2N/yK7e/z+Y+fvz1t716AdmO9ROg6pU5M9tkHI49+f1rEEVnhJQ6d4POx9lOEt6GH2FMBLR3b/BvjDa1UaOU/iJB9SkxnSzCtWpF1kF6lHY72f7H7Z3pW/JE+Linc7kqlfVqqfnGGCCdaIlLzmeqhHfu2wo3pON0hvEb0PbRBhAKH+E9l2jsphJZ5u/zWb9inXVFBCnfNTGmdgl3JaJ89IsbbyEXpXUda0R+FbhyrEgp2oKb5Jqh5GKSaz2xLdaNvqpfo1ZU4iiKDOsOPsjIScxlamblyFZE94tDTCnN9jCJnX2i2YQ3imvcAMRMfaX94Yq+cq0oRxGeDo2eadyWVDvUyqUxOBXLtoQXq2pdJ23y3bbUC4Csw8hhG15Zmf9X+/GosX5Tw4bsdSdLVO5/8Vfj94yjoaeko2lkhk+yT0AjCIZwIbO0ybO161Cq5GSFsP4xASr/5yQm26ShcqUZAx/aT5z/Rn0v2c+344YdfsguLKKtXVvRWuPbD6qWshnaq8KAKssu4fV75BN1vnAQDz9TRKGP0wQ+CGxzgKLkXPujsP1ofkwj/RIUrRvKH3YFDUqg//KlD4BOKCNrVYia3AYXEBc9H7yGn56rgOUBwGmVn2R26gJKXpGrUlnaINgneVmzOSrH0eubmrZpy+7OZ++n5TvnvLfxnvkda9Cn4nTyRQK4mOWqPH4UrYk3SK2qEQzGM8doVNSqPAqNeJVTaN6Ase95QKKsjgUCFxBsnLOHPeNziRE847s9lgrRr9yCeP7KxXy4+JPKR/k+7AlRgHX+r6i/H+zN94jnEmaMdiRyX+Ligo+iHHF5e1kYF1XWl6tVaaMzU+Dwl1YNqUXgqgpa6xRhKt2gSSYyfwwA73nyxny8WeNeBxrY108Rk1meGfDJzQSSoyaayIZHKi3mMyHbTgbpgBoTEpfWKfOLCqbGwJY+48TZKnuKHy16FtTWDUyB0C0prAkqbQ1FasaSYfcdggGGYxXDANeVINoK1MYGk9A1FPh0DFJ1tFmbByuaopjmIYViNBXDSXuMZvaeoJv8eqNRbmA3tB7IYNdkIVXUeofpUAQoHAWhnXOkcThpCsBqsbd8UguW8a42fYOsvBCsGqCEFoLnnbI7hOo82oWPwos8YRvTxIZkG55pvrzWIysp31InK6t28RWhzkO5vzti9/uPv+8+re+v21rfm3zzj838eMy/9UqKBIvko/K5phPp56sIG7CnAOPMGbZTZAYmxnS/3c9cJ3C0jWydvwTZnttBhSKYrlGS6AugT8BF1Pron3UlURCEAqZaapdhFS425EmNbarCtpaZPQhRBzSyhSgdjCAp4UlCUpv9Gkh2bCG0i7KI4bpQb/ASlvE6Loxyduj+RkIhenhrNl8vJk2uiydGRyHrOi9hA5g+389+KDgUveogPMdRZ/Bg0oObcvpR/iz/aWTNkhYt4815+Lw9eBtRWSAQouDJvkiKThGCAFzVL5Hr7VKaOiTeBT4izCNDv6HGLYlKjH7+M1it3G83QDrlbgpL+8BQ4Ma5NhLKtWapTKyRkEKnUQjPBTP8Ll5zAX4b0d4Aejg87S8VyzqAuWH0EdYGGtXrQsEI3XPYGA2hUIroqvBrXUTDbOgr2II5CNcDFrSfUNbX8iL4Cb5n1n71Hbwi81bcfyEMx5ZmALI4xRwFaU0x1muFWhS6BOYXVCxofPZVv5NtD+6CdzPEzXTmZXXdXyP35B9VaHqP7wZ8Afb290Xcw1DR+l1py2syMW/PMxYwx5T2n3lObmIE9tTK18M2ywQ/0k9bbWC54xKzq0N1fFGUB+64brTiYarIO8hRzDc8ca+anbD10WeMIZ7CMSFWXOYoeIo4fXNQQyi4DZaaXSajUUFCGiAmJs4zfadpNUXBsCZQGRSHF6MKRH16WwGsTKmuuEtBL4+gFa5hQkqoP3ZUTLlF8tI77iVqNgH6pqp34a3RyzAauGolEVInwQ26f6bD3H0HUuMUHpmDxQb4goei/K4M58KrEWbokRFNYqN5ldbqazjxU4jTpUhNdfrSGT6ogP1qjno7HydG+25BEfibP20mO92lUGSZEUyAAlmdx3ki8aOFTxmuLzr7L72iXhH7AZjBB2ZjQ1TcI75XnS8K27PI7b42nKRQlnVJLmUmHs7fITBu88VD4HiWachlArewlqgdvQVNlT2Jg6lYETsMywomN5lsvtwx5cJEzpTBzKiHUE6+oy+rNFzjyUPS2pRE/8lJDbMX99i7jZuSNOvigpGPtfCHmFMaDClSKRZ8MO0MLsuQU7YKuMlaCJgGP+3YBr4kxW2dqmOWlwdAcZ/qnEz22of/4PUbRDaOLcBXsu+PQl3gC6brgTE3Svwfji+UZBU0OweQ3EiWx7Ce2fMshOJr7cbxF8UVBin7YKRJpbUg0DZ+tO9FCTFpS6lIo+V7uxbBXygsijVfKqGHzsw6wzYHLTmNk+xeKk5TZFFVpkpLPrtDbRofTCXPHERFbaiMwCdG0DanWyrT0E1vkdZgLWmB1WHTRB2xcihO2DaMP4CiA2/sgmvC0BcDpY76zLgGpDNxscEzS7dD/S3Z8dJncTtfDVfcmTGAPgsrlXhs1haIw8bic1GuvyLhb2DqlwFK6NrY8zroaaudUXbr2xrTu0g76qOv1RFIeoN2GQkeUpf1iTGBwfgIEQfdBwVrqJ/j0qu3/Lbt/jkRlgtOnRsLHVKESvOlLm742I7E+lKA9nZFYivMWOHHsP+wvKNtIFgPpm920d3Y7l93MNillR+W2NmdLhN5Hdg4fM+QTbnDFRsoYB8yUydjKLjnruJm2R1IU6CfanJ/Vnbvtm0vVqfCb/k9kGmw+IYs/u6Sn/eMmp10U0O+pBXW25EQUSLxdpB7aLNqGH1Ii+yh+/awMgFyYe0MtdQo4nxw6ma5KTy8STBw2Zh/UQxv64rRV3Qao6agzbgD90KOimeDojAHsHMozjPLG5cqAq23GW7vE8d5EiA44S9TYYcBBstEb1GO0HUhBF5T2r/T7j3xWTW5B6DNPsjxKbGPw9ArdAMhSnc1Tw5vxOsWeeAUphVlxoZVgrr4BPNM25DlOpCr5HKVhmzOjhiHVWcI8J9mRTU6c3oV/CYyXTzFkfCcIHEPReQ6D8Z0C+rO+XxkiVGGebqmN/d1zeFBraYa3Tas0jhmu8Ee+5ajpaocrlcPm+NmtgU8kFaYFljxV7vaQ2rUx4NgUWWxKOCKoOsOoG3X2drDohsUnMh9zK6IQ3qpIwEqGN9pRzeuj0EU1MAeKxra+gTrrhl1PV85Mcdlb8rEBpv+EDdmap7Zwo7R4P80qD40JDnG+5PWCRld2WbAGp9LEolLipoQCDWZr1EZeOjG0zLa5dTi6UZXZuaE8jswWzb6OaxRDPU8wms6bD3iIVIezT1pYygeNJ80yVrnFOXmgubNlrGaZ3blUogNYxjch/Kr+C8P9p/j+ASq/KX/fvQjXALwahUYRhTOgmG2ReMbkCHvksKea1XoUE/HVLP4UPPWVgLQ5ZFdOeexpysTIRrrlcWz82VVN7fnv6Ny+59OJhMyn9sB8hRhHJy46ySON2Z/VtidOjgN9E3ISFHKPaA19pQyBtidcNqOBFiPMTNpjc2oJrb5iD7EW/wc=</diagram></mxfile>
# Configuration File
Here you can see an example configuration that can be adapted to your needs with a detailed description thereafter. Configuration is done via a .json file. All parameters except `aoi` and `timestamp` (which can be set via the CLI) are optional. Yet, the chosen defaults by the CLI may not be suitable for your use case and the number of calculated [indicators](indicators.md) is reduced with every missing parameter. The [setup documentation](installation_run.md) provides an overview on how to run the tool via an automated setup procedure that will help you set up the necessary [data](data.md).
## Stub
```json
{
"aoi": "8.68340358163951,49.4291689071729,8.70084609911781,49.4414943595746",
"timestamp": "2020-05-04T00:00:00",
"limit_ids": "",
"geom_type": "polygon",
"target_database_conn": {
"user": "uname",
"password": "pw",
"host": "hostIP",
"port": 5432,
"dbname": "dbName"
},
"data_schema": "public",
"ohsomeURL": "https://api.ohsome.org/v1",
"ohsomeURLDownload": "",
"oshdb": {
"oshdb": "h2:/path/to/file.oshdb.mv.db",
"prefix": "",
"keytables": ""
},
"changesets": {
"user": "uname",
"password": "pw",
"host": "localhost",
"port": 5432,
"dbname": "dbName"
},
"notes": {
"user": "uname",
"password": "pw",
"host": "localhost",
"port": 5432,
"dbname": "dbName"
},
"osmcha_api_key": "",
"sentinelhub_id": "",
"stop_after": "export",
"comment": "",
"logLevel": "DEBUG"
}
```
## Description
- **aoi** (mandatory)
- The area of interest to be analysed. Choose one of the following notations:
- 'latitude\_minimum,longitude\_minimum,latitude\_maximum,longitude\_maximum'
- table name and ID like `schema.tablename-id` where the geometry is stored in the 'geom'-column and id is stored in the 'id'-column. This allows the use of arbitrary, user chosen geometries for the aoi from a predefined and populated table.
- _NOTE_: Only elements that lie completely inside the given bbox (including some buffer) are analysed and errors may occur where objects temporarily moved outside this bbox in their history.
- The tool can be run multiple times with different aois. The result will then contain the indicators from all aois within the same data\_schema (see below). The extracted OSM elements are internally linked to the first aoi they lie within, but that is not disclosed in the output.
- **timestamp** (mandatory)
- Timestamp (in ISO-8601 format) in relation to which the data will be analysed.
- **limit_ids** (optional; default: empty)
- OSM-IDs to which the extraction should be limited. This is useful if you want to only analyse some objects within the aoi. Positive IDs are interpreted as OSM-Way IDs, negative as OSM-Relation IDs. Choose one of the following formats or leave empty to query and analyse all data within the aoi:
- a JSON list of integers with IDs (e.g. [12376,-34289])
- a path to a csv file containing the ids as "/path/to/file.csv"
- **geom_type** (optional; default: "polygon")
- Geometry type for which indicators will be calculated. Possible values: `"point"`, `"line"`, `"polygon"`
- **target_database_conn** (optional; default: docker database)
- a JSON object containing the connection details to the analyses backend database (aka. target database)
- *user*
- database username with rwx rights
- *password*
- database user password
- *host*
- database host (e.g. localhost or url to remote database)
- *port*
- database server port
- *target_db_name*
- name of target database
- **data_schema** (optional; default: "osm_vectorisation")
- schema name for data storage (schema will be created if it does not exist)
- **ohsomeURL** (optional; default: https://api.ohsome.org/v1)
- URL of an ohsome REST api instance
- **ohsomeURLDownload** (optional; default: https://api.ohsome.org/v1)
- In case you have special access to a download url with longer timeout you may set it here
- **oshdb** (optional; default: empty)
- a JSON object containing the details to connect to an [oshdb](https://github.com/GIScience/oshdb)-file, database or cluster. See [oshdb-driver](https://gitlab.gistools.geog.uni-heidelberg.de/giscience/big-data/ohsome/helpers/oshdb-database-driver) for more details on how to define this. OSHDB is FOSS, so you can set up an instance by yourself. See the [data documentation](data.md) for more details.
- _oshdb_
- either "ignite:/path/to/ignite_config.xml" or "h2:/path/to/h2-file.mv.db"
- _prefix_ (optional; default: empty)
- prefix for storage caches or tables
- _keytables_ (optional; default: empty)
- [JDBC](https://docs.oracle.com/javase/tutorial/jdbc/basics/connecting.html) connection string
- **changesets** (optional; default: empty)
- JSON database connection object (see above) for a database containing [OSM changesets](https://wiki.openstreetmap.org/wiki/Changeset). See the [data documentation](data.md) for more details. This notion will run the queries via a data pull to the client as postgres foreign data wrappers have shown very bad performance on joins. In case the data is available within the same database use the same connection parameters as for `target_database_conn`. If available it is used to estimate the user experience/skill (Attention: time-consuming, see benchmarks) and other indicators.
- **notes** (optional; default: empty)
- JSON database connection object (see above) for a database containing [OSM Notes](https://wiki.openstreetmap.org/wiki/Notes). See the [data documentation](data.md) for more details. This notion will run the queries via a data pull to the client as postgres foreign data wrappers have shown very bad performance on joins. In case the data is available within the same database use the same connection parameters as for `target_database_conn`
- **osmcha_api_key** (optional; default: empty)
- [Mapbox OSMCha changeset analyser](https://osmcha.org) api key. It is automatically created on first login at [https://osmcha.org/](https://osmcha.org/) using your [OSM account](https://www.openstreetmap.org/login). You can find your key under _"Account Settings"_. Be sure to copy it *as-is* including the prepended __Token__ keyword and space.
- **sentinelhub_id** (optional; default: empty)
- your id gained from a [sentinelhub](https://www.sentinel-hub.com/) membership that will be used to calculate remote sensing indicators
- **stop_after** (optional; default: empty (the whole script will be run))
- after which step should the script be stopped (see [workflow image](workflow.md) for details)? Available values are
- setup
- data_extraction
- surrounding_extraction
- fs_calculation
- classification
- **comment** (optional; default empty)
- specify a custom note attached to the newly extracted data e.g., for marking objects
- **logLevel** (optional; default: DEBUG)
- level of logging. Logging is issued to a custom file in the .log directory. One of
- CRITICAL
- ERROR
- WARNING
- INFO
- DEBUG
### Former Configuration Possibilities
The following parameters were used as part of the configuration options in earlier versions of the tool. They are still documented, in case they become relevant again in the future:
- **osm\_tags\_tableName** (optional; default: metadata.osm_tags)
- `schema.table_name` of osm tags in format `key, value, clc_id` where `key` and `value` create a valid [OSM-Tag](https://wiki.openstreetmap.org/wiki/Tags) and `clc_id` is the [CORINE class id](https://land.copernicus.eu/pan-european/corine-land-cover), if available. If the table is empty or non-existent, it will be populated with a default set of tags (see [osm_tags file](../src/osm_element_vectorisation/resources/sql/setup/osm_tags.sql)).
- For custom tags you either have to assure the table exists in the requested format in your database or alter the [osm_tags file](../src/osm_element_vectorisation/resources/sql/setup/osm_tags.sql) if the table does not yet exist.
- **custom_data_table_name** (optional; default: osm_data)
- table into which objects within that given aoi and with the optional additional id-filter will be downloaded. If the table already contains data in the expected format, it will be further analysed
- **metadata_schema** (optional; default: metadata)
- schema name for metadata storage (schema will be created if not exists)
- **countries\_tableName** (optional; default: metadata.countries)
- any logical regions with the minimum fields ogc_fid, continent and wkb_geometry specified in the format `schema.table_name`. See the [data documentation](data.md) for more details.
- **popDensity\_tableName** (optional; default: metadata.popgrid)
- `schema.table_name` of population grid analogues to description above
- **fast_contribs** (optional; default: true)
- If _true_ will request contributions based on buffers. Some contributions that lie outside that area may be missing. If _false_ requests contributions based on bbox. If bbox is global, all contributions are included (see [https://github.com/GIScience/ohsome-api/issues/256](https://github.com/GIScience/ohsome-api/issues/256)) but this may take considerably more time.
# Data
The workflow makes use of external data wherever useful. The more datasets you provide, the more indicators can be calculated. The tool is resilient against any missing backend data, but the results will be very limited if datasets are missing. See the [indicators documentation](indicators.md), the [configuration](config.md) and the [setup documentation](installation_run.md) for more info on what these datasets are used for and how you can configure them in the tool.
The [import script](../Docker/setup/populate.sh) serves as a documentation of the data setup procedure. It will populate the backend database with all necessary datasets to produce meaningful results if run within the docker workflow (using `full` mode).
## Description
### Required Data
#### Ohsome
[Ohsome](https://ohsome.org/) is a framework for OSM history analyses. For basic functionality, the workflow uses the global public instance of the Ohsome REST-API. You may though specify a custom instance in the configuration.
### Optional Data
#### OSHDB
The [oshdb](https://github.com/GIScience/oshdb) provides an additional Java-API which is used to calculate more sophisticated user based indicators. OSHDB is FOSS so you can set up your own instance. Alternatively you can download an ohsdb-file [here](http://downloads.ohsome.org/) or contact the [oshdb-team](https://heigit.org/contact/) for a custom extraction or access to their cluster. The OSHDB instance will be used for user specific calculations. You therefore have to assure that all contributions (touched entities) by the users in your area are present in the extract you are using.
#### World borders
A vector file depicting world borders as polygons together with meta information. During development the [Natural Earth Dataset](https://www.naturalearthdata.com/downloads) was used. The dataset is expected to contain the fields ogc_fid, wkb_geometry and continent produced by uploading the file to the database with the following command:
```shell
ogr2ogr -f PostgreSQL PG:"user={user} password={password} dbname={dbname} host={host} port={port}" -lco SCHEMA=metadata /path/to/file.shp -nln countries -nlt PROMOTE_TO_MULTI -progress
```
#### Biomes
A vector file depicting global biome habitats. During development the [WWF ecoregions of the world](https://www.worldwildlife.org/publications/terrestrial-ecoregions-of-the-world) were used. The dataset is expected to contain the fields ogc_fid, wkb_geometry and biome produced by uploading the file to the database with the following command:
```shell
ogr2ogr -f PostgreSQL PG:"user={user} password={password} dbname={dbname} host={host} port={port}" -lco SCHEMA=metadata /path/to/file.shp -nln biomes -nlt PROMOTE_TO_MULTI -progress
```
#### Human Development Index (HDI)
A vector file depicting global HDI-regions. During development the [subnational HDI regions](https://globaldatalab.org/shdi/shapefiles/) from Radboud University were used. The dataset is expected to contain the fields ogc_fid, wkb_geometry and hdi produced by uploading the file to the database with the following command:
```shell
ogr2ogr -f PostgreSQL PG:"user={user} password={password} dbname={dbname} host={host} port={port}" -lco SCHEMA=metadata /path/to/file.shp -nln hdi -nlt PROMOTE_TO_MULTI -progress
```
The workflow expects the HDI value in the `hdi` column while the linked dataset uses a filed called `shdi`. Rename using the following command:
```shell
psql -h {host} -d {dbname} -U {user} -p {port} -c "ALTER TABLE metadata.hid RENAME COLUMN shdi TO hdi;"
```
#### Population density grid
A raster denoting population density in terms of inhabitants per km². During development, we used the [Global human settlement](https://cidportal.jrc.ec.europa.eu/ftp/jrc-opendata/GHSL/GHS_POP_MT_GLOBE_R2019A/GHS_POP_E2015_GLOBE_R2019A_54009_250/V1-0/GHS_POP_E2015_GLOBE_R2019A_54009_250_V1_0.zip) file distributed by the European Union. Any dataset depicting the same information should though work as long as it has the same projection (Mollweide). We propose the following commands:
```shell
raster2pgsql -N -200 -d -l 2,4,10 -e -C -M -I -t auto -s 54009 /path/to/file.tif metadata.popgrid > temp_statemetnsUploadPopgrid.sql
psql -v ON_ERROR_STOP=on -d {dbname} -U {username} -h {host} -p {port} -f temp_statemetnsUploadPopgrid.sql
```
#### Changesets
Some information is calculated based on changeset data. A table with userid and one row per changeset is the minimum requirement. We use the `osm_changeset` table from [ChangesetMD](https://github.com/ToeBee/ChangesetMD) setup in the public schema of a postgres database.
#### Notes
Analogous to Changesets, a Notes table created via [OSM Comments Praser](https://github.com/mapbox/osm-comments-parser) can be used.
# Indicators
The following tables provide information on the indicators calculated. Fore more information on the used [datasets](data.md) and how to [configure](config.md) them, see the respective documentation.
## Element Based
### Semantic
| Indicator Name | Description | Calculation | Logical Link to LULC Quality | Extrinsic Data Involved | Problems | Reference | Alternative approaches |
|---|---|---|---|---|---|---|---|
| relation | Is the OSM object a way or a relation? | relation? -> true:false | Relations in OSM are more complex and need higher skill to be mapped but also are more error prone. | | | | |
| primtag | Primary tag of the object. | Key and value of tags contained in the [MapFeatures list](https://wiki.openstreetmap.org/wiki/Map_features) | Some tags may be more difficult to map, ambiguous, less established or less documented than others. | | | | |
| corine\_class | A semantic aggregation of primary tags into the CORINE class level 2. | According to the [mapping](../src/osm_element_vectorisation/resources/sql/setup/osm_tags.sql) of the primary key to CORINE land use / land cover classes. | | | Large information overlap with *primtag*. | [CORINE class definition](https://land.copernicus.eu/user-corner/technical-library/corine-land-cover-nomenclature-guidelines/html) | |
### Geometric
| Indicator Name | Description | Calculation| Logical Link to LULC Quality | Extrinsic Data Involved | Problems | Reference | Alternative approaches |
|---|---|---|---|---|---|---|---|
| invalidity | Gravity of geometric invalidity. | The geometry is made valid using different approaches. The count of failed approaches is recorded. A value of 0 is assigned if the geometry was valid.| Invalid geometries describe bad data. | | | | |
| detail | How detailed is the object drawn? | perimeter(m)/n\_vertex as the average segment length | The more detailed an object the more effort and the higher the quality | | | [Mooney at al 2010](https://www.researchgate.net/publication/229009232_A_study_of_data_representation_of_natural_features_in_OpenStreetMap) | The sum of inner angles [https://osm.zz.de/dbview/?#landuseoverlap-bw](https://osm.zz.de/dbview/?#landuseoverlap-bw) |
| complexity | How complex is the geometry?| 0.8 * vibration amplitude * vibration frequency + 0.2 * deviation from convex hull. If multiple rings are present they are summed up. Vibration aplitude and frequency describe how often and strong 'notches' occur on the geometry. The result is a unitless complexity index. | | | Only defined for simple polygons. | [Brinkhoff et al 1995](https://www.researchgate.net/publication/221589831_Measuring_the_Complexity_of_Polygonal_Objects)|[https://ieeexplore.ieee.org/abstract/document/4014089](https://ieeexplore.ieee.org/abstract/document/4014089)|
| obj\_area | Size of the object. | area(sqm) | In combination with the region or continent, where the object is located, this might show oversized elements. The trend in OSM goes towards smaller elements because they can better describe local detail → the smaller the element the better (up to a certain point). | | | |May be combined with other indicators to normalise (e.g. size in relation to mean size for this tag (is it a large farmland (in this area ?) or in relation to similar objects).|
## Surrounding
| Indicator Name | Description | Calculation | Logical Link to LULC Quality | Extrinsic Data Involved | Problems | Reference | Alternative approaches |
|---|---|---|---|---|---|---|---|
| continent | The country or continent an element is located on. | Extract continent from closest natural earth feature.| Regions have different communities, physical geographic contexts etc. that influence quality. | [Natural Earth](https://www.naturalearthdata.com/) (OSM internal boundaries are too complex) | | | |
| pop5k | Population density in the objects surrounding. This can also be interpreted as the number of potential local editors of the object. | The mean pop-density within a 5km radius around the object is caclulated. | Findings are that rural areas often have lower quality; how many potential local mappers are available and what features are plausible given the population density? | [https://ghsl.jrc.ec.europa.eu/download.php?ds=pop](https://ghsl.jrc.ec.europa.eu/download.php?ds=pop); 1km resolution is preferred over 250m to have a small averaging effect and more interpretable results. | | **Add Neis and others**| |
| hdi | The (economic) situation of a region. | Human development index (hdi) value at object location. | The economic status influences mapping (leisure time, equipment etc.). | [https://globaldatalab.org/shdi/shapefiles/](https://globaldatalab.org/shdi/shapefiles/) | | **Add Neis and others**| |
| biome | The 'physical appearance' of the objects surrounding. | Biome class at the object location. | The biome influences the probability/validity of certain objects. | [https://www.worldwildlife.org/publications/terrestrial-ecoregions-of-the-world](https://www.worldwildlife.org/publications/terrestrial-ecoregions-of-the-world) | | | |
| remotesensing\_variance | How homogeneous is the object in remote sentsing imagery?| Sum of surface reflectance variances of optical Sentinel 2 bands. | homogeneous reflectance -> homogeneous area -> object well drawn (depending on object type) | [Sentinel-2](https://www.sentinel-hub.com/) | | | |
### OSM surrounding
| Indicator Name | Description | Calculation | Logical Link to LULC Quality | Extrinsic Data Involved | Problems | Reference | Alternative approaches |
|---|---|---|---|---|---|---|---|
| overlap | How much does the element overlap with elements in the surrounding. | Ratio of overlap at target timestamp with all valid *landuse*, *natural*, *waterway* polygons adding up each primary tag value separately and only considering overlaps <90%. | less overlap -> better geometry | | Sometimes overlap is allowed (e.g. leisure=garden, landuse=residential), sometimes even inevitable (mountain\_range). Overlap may also only be bad if it is partial (within or disjoint is good). | |At what time does the overlap count (currently or at creation)?|
| shared\_border | What part of the polygons border is shared with surrounding objects? | % of border shared | If the object is well placed other objects might be closely attached to it. As LULC is optimally space filling , a well mapped area would be represented by connected LULC objects. | | Surrounding in db is unioned by tag -> might create incorrectness. | |What about close alignment and parallel lines (other side of the road)?|
| surrounding\_mapped | How well is the area mapped in general? | % area mapped in surrounding (currently within buffer of 150m excluding geometry itself); values >1 indicate overlap within buffer. | Well mapped regions indicate good quality? | | Surrounding currently seen as 150m around element and therefore dependent on element size | |Considers only LULC elements, are others also relevant? |
| surrounding\_diversity | How many different primary tags (reperesenting Map Features or object types) relevant for LULC are present in the surrounding? | no of osm LULC primary tags within buffer of 150m | Depending on the primary tag diversity or the lack thereof. Diversity of mapped LULC objects might indicate high quality - if real world LULC is characterized by some diversity. Often some LULC classes are mapped first (e.g. landuse=forest) - if surrounding diversity is high this could indicate that the mapping has progressed beyond this point. | | High correlation with surrounding_mapped | |[shannon index](https://de.wikipedia.org/wiki/Shannon-Index) could be calculated.|
| surrounding\_obj\_dens | How many objects are there in general in the area? | no of nodes, ways and relations in the bbox (per area) | Well mapped areas have many objects (in relation to the population density etc.) | | Relations not working well due to limited capability of oshdb but they are not very important (only few and include turn restrictions, routes etc.) | | Each primary feature type and geometry type should be included separately (bulding area, highway length and poi count).|
| how\_many\_eyes | How many contributors were involved in this area? | mean unique users (contributors) per month since object creation per elements in buffered surrounding |Does this area even deserve the "crowdsourced" tag? | | | | ohsome api can currently only count number of users, could be implemented in java to give user id list for own personal user aggregation.|
| oqt\_mapping\_saturation | What is the estimated area mapping completeness?| weighted mean of POI, LULC, infrastructure and building saturation | Saturated areas may have better quality. Especially with respect to completeness of mapping in the region but also with respect to individual OSM objects as more effort by mappers can be put into improvement instead of creation of missing objects. |[OQT API](https://oqt.ohsome.org/) | very slow | **add Brückner et al** | |
## Temporal
| Indicator Name | Description | Calculation | Logical Link to LULC Quality | Extrinsic Data Involved | Problems | Reference | Alternative approaches |
|---|---|---|---|---|---|---|---|
| uptodateness |How up to date is the object| time in seconds since last minor OSHDB version (user touch of element or member) | Recently changed objects are potentially more up to date. | | Only because I changed one edge of a large relation does not mean I checked the whole object. | [https://is-osm-uptodate.frafra.eu](https://is-osm-uptodate.frafra.eu)| |
| object\_age | How old is the object? | time in seconds since creation of object | Old objects are either outdated and were mapped with superseded mapping practices or had much time to mature with many small improvements. | | | | |
| changes\_per\_year | How frequently has the object been changed/updated | n\_changes/object\_age | Objects that are regularly changed are well maintained. | | Not relevant for new objects | | |
## Mapper
| Indicator Name | Description | Calculation | Logical Link to LULC Quality | Extrinsic Data Involved | Problems | Reference | Alternative approaches |
|---|---|---|---|---|---|---|---|
| user\_mean\_exp | How experienced were the users that changed this object| weighted mean number of changesets of users that edited the object | More experience users may have a higher probability to make changes that improve quality. | | |**Add Schott et al and sources therin (Neis)**| Experience is not well defined. Alternatives are: time since first edit; number of edits... . Weighting is arbitrary. Most objects are at version 1 so creation may be most important. Others have created complex experience indicators: [https://onlinelibrary.wiley.com/doi/full/10.1111/tgis.12454](https://onlinelibrary.wiley.com/doi/full/10.1111/tgis.12454)|
| user\_localness | How local where the mapper(s) of the object? | median distance of changeset centroids to element centroid | Local users are more familiar with the region and should create better data. | [Changesets](https://wiki.openstreetmap.org/wiki/Changeset)| HOT users will be located in Africa; only the localness of the most recent editor of an objects is calculated. | **add Neis** | Alternatives described in the reference|
| user\_diversity | How specialised is the user|[shannon index](https://de.wikipedia.org/wiki/Shannon-Index) if edits made by user grouped by [tagGrouping.json](../src/osm_element_vectorisation/resources/misc/tagGrouping.json)| Diverse users may be general experts but specialized users may be experts on this topic.| | | **add Schott et al**| An alternative would be the average location or the median distance to the corners of the changesets. This would penalize more for large changesets which are relatively rare but a sign of bad quality.|
## Mapping process
| Indicator Name | Description | Calculation | Logical Link to LULC Quality | Extrinsic Data Involved | Problems | Reference | Alternative approaches |
|---|---|---|---|---|---|---|---|
| data\_source | What sources were given for the changes?|source tag from the object and the linked changesets | Certain sources may be more reliable | [Changesets](https://wiki.openstreetmap.org/wiki/Changeset) | | | |
| cs\_editor | Editor/editing software used.| editor tag from the linked changesets | Some editors might have bugs (e.g. [https://www.openstreetmap.org/user/stragu/diary/394199](https://www.openstreetmap.org/user/stragu/diary/394199)) or are not suited for certain editing| [Changesets](https://wiki.openstreetmap.org/wiki/Changeset)| |[Muttaqien et al 2018](https://onlinelibrary.wiley.com/doi/full/10.1111/tgis.12454) | |
| cs\_median\_area | Was the object edited by local edits or large scale edits? | median bbox-area of linked changesets | Changesets that span large areas hint towards bad quality, imports or unspecific changes.|[Changesets](https://wiki.openstreetmap.org/wiki/Changeset) | | | |
## Linting tools
| Indicator Name | Description | Calculation | Logical Link to LULC Quality | Extrinsic Data Involved | Problems | Reference | Alternative approaches |
|---|---|---|---|---|---|---|---|
| validator\_issues\_density |Density of issues reported by linting tools.| sum of errors reported by [osmose](../src/osm_element_vectorisation/resources/misc/osmose_parameters.json) and [keepright](keepright_checks.md) per element present object surrounding | Less errors indicate a well maintained region and good quality. | [osmose](http://osmose.openstreetmap.fr) and [keepright](https://keepright.at/)| | |Choice of selected error categories, e.g. [alternative tags for osmose](../src/osm_element_vectorisation/resources/misc/osmose_parameters_alternative.json) taken from [the website](http://osmose.openstreetmap.fr/api/0.3/tags).|
| n\_osm\_notes | How many osm notes are there?| notes per element in object surrounding | Many notes either show bad data or an active community. |[Notes](https://wiki.openstreetmap.org/wiki/Notes) | | [Seto et al 2020](https://www.mdpi.com/2220-9964/9/6/372) | |
| osm\_cha\_flags | What osmcha flags were added to the changesets? | list of flags | Many [flags](osmcha_reasons.md) are related to suspicious changesets or edits. |[osmcha](https://osmcha.org/). | | |
# Installation
## Setup
The package uses [poetry](https://python-poetry.org) as build tool and dependency management. Run `poetry install` to install the package (it is _not yet_ available on pypi) and
`poetry run vectorise`
to run the tools CLI. You can of course also import the package into your module and use the python API. Yet, the backend requires a certain layout.
We therefore provide a Docker workflow to get you started. If you do not want to use Docker, you can extract the _import_ procedure from this [Dockerfile](../Docker/dockerfiles/Dockerfile.data) and the _installation_ procedure and prerequisites from this [Dockerfile](../Docker/dockerfiles/Dockerfile). In that case you must also set up your own backend database (e.g. like [here](../Docker/dockerfiles/Dockerfile.postgis)).
The tool requires a [configuration file](config.md) to be passed as first positional argument but can additionally be configured through the CLI.
### Docker
Both [Docker](https://www.docker.com/) and [docker-compose](https://docs.docker.com/compose/) need to be installed.
The docker workflow has five modes:
- _repex_ for a minimum reproducible example
- _benchmark_ to run a standardised [benchmark](benchmark.md)
- _full_ to create a complete backend. **NOTE**: this will take several hours and needs considerable resources (see [datasets docu](data.md))!
- _custom_ to execute the tool against your own database with your custom parameters
- _run_ to run the tool against the provided database with your custom parameters
To use the workflow execute
`./vectorisation_docker.sh <mode> <args>`.
The `<args>` are only useful for modes _custom_ and _run_ and will be passed to the tools. Run `poetry run vectorise -h` for a documentation of the CLI.
## Backend
The tool uses a [postgres](https://www.postgresql.org/) - [postgis](https://postgis.net/) database as backend. [docker-compose.yaml](../Docker/docker-compose.yaml) provides a basic setup but of course you should adapt it to your needs (user management, security etc.).
## General Comment
This workflow aims at providing a minimum effort data analyses solution to the end user. Yet, we are aware that this 'minimum effort' may become reasonable effort, if the necessary infrastructure has to be set up for all analyses. This is though inevitable when respecting the [OSM usage policy](https://operations.osmfoundation.org/policies/api/).
# Keepright Checks
List of quality checks to be performed by Keepright. Source: [https://sourceforge.net/p/keepright/code/HEAD/tree/config/error_types.php](https://sourceforge.net/p/keepright/code/HEAD/tree/config/error_types.php). However, some checks mentioned here do not seem to be implemented (e.g. 260). See [https://sourceforge.net/p/keepright/code/HEAD/tree/checks](https://sourceforge.net/p/keepright/code/HEAD/tree/checks) for the source code of each quality check.
List of quality checks to be performed by Keepright. Source: [https://sourceforge.net/p/keepright/code/HEAD/tree/config/error_types.php](https://sourceforge.net/p/keepright/code/HEAD/tree/config/error_types.php). However, some checks mentioned here do not seem to be implemented (e.g. 260). See [https://sourceforge.net/p/keepright/code/HEAD/tree/checks](https://sourceforge.net/p/keepright/code/HEAD/tree/checks) for the source code of each quality check. Only a subset of these is currently used. See the respective source code.
- 10 - deleted items
- 20 - multiple nodes on the same spot
......
# OSM-Cha-Reasons
This is a list of osm-cha reasons for flagging changests extracted from [https://osmcha.org/filters](https://osmcha.org/filters) in a very cumbersome process. It is unknown if this list is complete, all we know is that the flag `no reason` was added by us. The grouping is also our own interpretation of the flags.
This is a list of osm-cha reasons for flagging changests extracted from [https://osmcha.org/filters](https://osmcha.org/filters) and the [API](https://osmcha.org/api-docs/)in a very cumbersome process. It is unknown if this list is complete, all we know is that the flag `no reason` was added by us. The grouping is also our own interpretation of the flags.
## Import
......
# Workflow Description
![](workflow_graphic/flowchart_automatic_workflow.png)
## DB Setup
First, a database set-up is performed. Thereby any necessary schemas, tables, functions and extensions are created and the backend integrity and usability is ensured. See [setup documentation](installation_run.md) for information how to prepare your backend.
## Data Extraction
Next, the requested OpenStreetMap data is extracted via the [ohsome-api](https://api.ohsome.org/). This means either the all elements within the given area of interest (aoi) or the defined elements within the aoi are downloaded. This step is necessary because many indicators need the OSM geometries as input. In addition to the target OSM data, all data within a buffer is equally downloaded for neighborhood analyses.
## (Quality) Indicator Calculation
Calculation of the intrinsic or semi-intrinsic quality [indicators](indicators.md) with the previously extracted as well as the external [data](data.md).
## Quality Estimation/Classification
This step is experimental. Three models have been trained that will predict the overall quality of the extracted elements as well as their geometric and semantic quality. This procedure uses the [R scripting language](https://www.r-project.org/).
- For the semantic quality the model predicts the _probability_ for the primary tag to be correct using a [random forest](https://cran.r-project.org/web/packages/randomForest/randomForest.pdf) classifier.
- For the geometric quality the model predicts the error of commission, meaning the relative areal over estimation. It uses a [quantile forest regression](https://cran.r-project.org/web/packages/quantregForest/quantregForest.pdf) using the median prediction.
- The overall quality of the object is equally predicted via a quantile forest regression. The overall quality is defined as `if (primary tag is correct) {geometrc quality} else {0}`. In addition to the median prediction, the other quartiles are also reported.
During quality prediction indicators are normalised automatically.
## Resulting Data
The tool populates the provided database and is resilient against multiple runs (it will ignore/reuse any existing data). Indicators are available in the table `indicators_view` in your defined data schema of the target database. It is linked to your specified data table via the column `ref_id`. The data can also be exported to files for easier access. Namely the geodata, the raw indicators, the normalised indicators and the prediction results are exported.
<mxfile host="app.diagrams.net" modified="2022-03-09T10:40:25.173Z" agent="5.0 (X11)" etag="-BoajGFkcmmxUm5nLL1W" version="16.5.6" type="device"><diagram id="71DWNRE2uBbWXSLaC3Lt" name="Page-1">7V1de6MoFP41uew8fkRjLpt2ptPddj62uzvdqz5EMWFrxEFs0v31CyqJCklMqjUfc5MqIsJ7Xg5wzoH2zKvZ4oaAaHqPPRj0DM1b9MzrnmEM+wb75QmvWYJlaVnChCAvS9JXCQ/oP5gnimwJ8mBcykgxDiiKyokuDkPo0lIaIATPy9l8HJS/GoEJlBIeXBDIqT+QR6dZqiNawdM/QzSZii/rWv5kBkTmPCGeAg/PC0nmx555RTCm2dVscQUDjp3AJXvv05qny4oRGNI6L3x6eCQXP4eXL+bvIfhCH+9p5F3kpbyAIMkbfA0oGIMYstQHSJMorzx9FYgQnIQe5IVqPXM0nyIKHyLg8qdzRgGWNqWzgN3p7NJHQXCFA0zSd00PQMd3WXpMCX6GhSe268Cxz57IzRJ1hITCRSEpb+YNxDNIySvLkj/tCxnknDP7+f18JUFdiGVakJ7IB3LSTJZFr3BlFzm0O8BsSDBLsEKP8S6/xYRO8QSHIPi4Sh2VgV/lucM4yuH+F1L6mncikFBcFsZaaGOcEBduqL+Z9zxAJpBuyGdl+XhbNgqKwABQ9FLuY42jbirJzVI+LigBLkU4PEJ2W/ahsbt/5Oy2arJ7cFDstiTUHxKSoojCCXtwClwfGIfGdfvIuT6oyXVTPyiyDyTYP0FAE5JOU1LOchbYYMZZm/2ylO8JCBDltbxlvcIFlHHU0K5A4Ca80kfZJRzn0LqEo+gSdsC+OvLQC/9ggCZh+sD+mfDZ7iiAPl3dsatJ/jd9aywSrnDoo0lCclFpnxATU56J1XRceDETeTiOo1r32vJbpPr1KOWUG4A4LtXZZSsbjigDNEhBLdRfCwBTucXcM0Ce2Ww/XN/KOALhClmR6+7KKRUs1yNAIWzo23LhkeKdwaiAefZmubQ1yTK065o87LLJc9U7WoEia5q3tsAwUNYiT8kU72iMF6Vce31JJS3zct/S1gFRSFxeXW3H582s0EvfPnFaBGiG6K0XHw0pdhO9mkI7M0I/I0ZQNIMx5ePWKVBip7qtuNGabjHOiEk4nj1RMInZzziAX8Cs1LJmKbV+vGhZoOYZCTSbQzx5uaX2idu+z09LtD8v1fsHSqr2qJXEkJw2lWJjU/vbafs76D/r7KgasexzTLxfdD1CutpnR9cpjmk74vpF1VapOjg7qkaYHD1V17Lu0OjVqb23E3p547DVNegJ6cK30+u0bOvKdeC10pj44cOH9XDtBjdLTr1ChQ9uKDh1eonUte5F7j/czb3oOy50le7FsWP1Le6vFJ621L8mPv8mb+Ow7GzsK5yNywC3orPRFImNexuH2x3w8RRE/NJNSPA6IsB9hnQ73GXZZCCLkD+TiyNA0d9CNOz6c2+LL74+yqZZgdlUwKxC2WwLZRF+WYD52yudHqXHXK+GA+oyuo4K3dbAlaMuH77fNYusBR2vr0LWMcambbeE7KBrZOVAS3cK3WeWdM2HHrigMIwRDuMjRHupfIWWMLpGWw6wFGgjn/1cfr3lbSb4BXFkTwDxzvkth1q6BALKI1Nihv0sjfk7epytzpktB1cKZgNvhkLE4Mii3gxtzOEFBMFT0ClWTYZbrSEvB1gK5CMcLcP2NI/r8TS0L91IcvzI207XyMsxlgL5rw/37PdPMDkFig+GXSsXOWByqcRTF/0poOzUpHN7KNdYKMLQu+R7xNhdamtAbhlJuED0sXD9DxfAByu/u17k8khvXsVNyGr/WLwpvMVvV6+ld+K9tTLYGtIttsVtDenO8m2N6C5IzNqggGoHfudf+IYRa9mSMHZl7dB3KkzI2p2/tSKDXJBVKahKqQwYqaCUVctmv2Fvl7xUlogmLBJ+ABc540bbyLeWEAciwL5ext0cDPcTYLWgYdV21LYA5eW4GPjyPsWmG8c5sZY6WdcTa9E31m0WyOym3NwcYODJEtB8gmfsD45nYpUZYlpaZaoNsdI36ttwpVdF1XAvMyu82Rh8qJYzu7L9qu76V2/N+mvIJoezULZWddfPvqNltSBr+M7KVrVZdJMCiMWmxolaCfzq7s3ttrT3Gy1a7O4KO0y+e4+vlqQ9gEc3QA+rGrYm5HZriNfY4HqKCnbQb0jBVguyqvsu21awshnnLAQ4rPqi9hVgtSBpYdq2ALfspy0POj5O674SroiP4A8u4nQDOd+XYVhRecdhWsodCL0kPetFuVV6Oa5lX9lx3Gymir3Khm3tBybPfoDnO1av0gWYfqZlgovgCZcRHDJVP+JanCERXOYPZsjzsu37kFWaG+rygSXiVEjJYY161jUvK6E4a1hTIQKDMikNRYSAqSl6V3tH2tSxrx3ySQhmbbuZUVPRvdOpNjXsTacBfP+wzqAwZTvRZRQFvAL3d/wnP/vMT0+ZWCnUbwQyjdrFcRMeiKfLiKYGlJCuVWdJnR8+ITrn5tGy7F9J68uqm0cYIp+Pd9ngwlR7EtBu1njbFxyty1O1/HhnecoGHn7sR7MuMd+HtjqU0hsMx9pGd0x9dI3K2q7zuDNTtr3chlFCs+nXt6J7/XrpXr9p2r3+PuBLXl/FhEkFfnvElk0ZCkWFlvKoOcm+rIajjArhKK3rrG5kadc0krQnS5WRRBLN7a6yLIZZnKjonK5F1z/22XO/9uz5sJYtfXn2/DWhvIcY2h/5rOvIZ8fGwZ07KzhQtJ/HL++tR1oHuvtpa1/llywo+NqH4KVh6mKVksT8dGpDw369YWT3o/a2LnD2Kf2Cr6wYpeimWci+BQOMGi8zwlEWRdpsseLgnTZG8ybG5kovMhTrE91U9KLlto/mu9EW7/DZdaPCUT8HyqLqcdSGatBTsWjQGom2rLTOikT7Uy9enSa9gX/VnAT6B0rU6lnSKheKkqhV/19zRN2yjGyQqO2Sqz39VyRee1+BHqKNTyp4oZg0XiwS7tkWS356QXDedPH8aLGndFrYeM3TotlrkGRGy2ZLf4YwIlwdPEGm55rHnU0TcQwrhbfremhAnToDq6RO9brjvt7EwH83tjV4A/Tx018Xj+bgNx18vlD8M5vzHfcrHTpa+iDflWESnRSk2+CdMssUa3NqyW5X/4opC+VZ/T8r8+P/</diagram></mxfile>
......@@ -295,7 +295,7 @@ def get_identifiers(
"""
metadata_schema = "metadata"
ttable = "data"
ttable = "osm_data"
itable = ttable + "_indicators"
return {
......
......@@ -125,7 +125,7 @@ def calc_diversity(settings: ActivatedSettings):
if uids:
uids = [str(uid[0]) for uid in uids]
outfile = settings.processing.outpath / "diversityOutput.json"
outfile = settings.processing.outpath / "temp_diversityOutput.json"
with path(misc, "tagGrouping.json") as filepath:
key_map_file = filepath
......
......@@ -45,6 +45,6 @@ case $1 in
docker-compose --project-directory . -f Docker/docker-compose.yaml down
;;
*)
echo Unknown mode, please select one of -repex-, -benchmark-, -full-, -custom- or -run-
echo Unknown mode, please select one of -repex-, -benchmark-, -full-, -custom-, -run- or -dev-
;;
esac
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment