Operations Guide

The operations guide describes how to install, configure and maintain django-eveuniverse.

Installation

To install django-eveuniverse into your Django project please follow the these steps:

Install from PyPI

You can install this app directly from PyPI:

pip install django-eveuniverse

Update settings

Next add eveuniverse to INSTALLED_APPS in your project’s settings file.

By default only the core models are automatically loaded on-demand. If you want to also include some of the additional models please add them to your settings.

See also

For an overview of all settings please see Settings.

Setup celery

This app uses Celery for loading large sets of data, e.g. with the load commands. Please make sure celery is setup and working for your Django project.

In addition you need to have a configuration for celery_once.

Note

Note that in case your installation is running with Alliance Auth, then you already have a celery once configuration and can skip this step.

Eve Universe comes with generic Django backend, which should work with most Django installations and which can be setup by adding this to your celery configuration file (e.g. celery.py):

celery_app.conf.ONCE = {
    "backend": "eveuniverse.backends.DjangoBackend",
    "settings": {},
}

Or you can use one of the backends provided with celery_once. See here for more information.

Note

Note on celery worker setup

For an efficient loading of large amounts of data from ESI we recommend a thread based setup of celery workers with at least 10 concurrent workers. (e.g. celery -A myapp worker --pool threads --concurrency 10 )

For example on our test system with 20 threads the loading of the complete Eve Online map (with the command: eveuniverse_load_data map) consisting of all regions, constellation and solar systems took only about 15 minutes.

Finalize installation

python manage.py migrate

Finally restart your Django instance so your changes become effective.

Updating

Hint

Before updating please always check the Change Log for any special instructions on updating or important changes that might affect your project.

To update your installation first install the new version:

pip install -U django-eveuniverse

Then run Django migrations:

python manage.py migrate

And finally restart your Django instance so your changes become effective.

Settings

Here is a list of available settings for this app. They can be configured by adding them to your local Django settings file.

Note

All settings are optional and the app will use the documented default settings if they are not used.

Important

Many settings will enable the automatic loading of related models. For example: if you enable Planets, all related planet object are automatically loaded when updating a solar system. This will significantly increase load times of objects, so we recommend to only enable additional models that are functionally needed.

The preferred approach is load related model on demand with the enabled_sections feature. Please see Load related models on-demand for more details.

Settings for eveuniverse.

EVEUNIVERSE_API_SDE_URL = 'https://sde.eve-o.tech/latest'

URL to a web site providing the SDE tables as JSON files.

EVEUNIVERSE_BULK_METHODS_BATCH_SIZE = 500

Technical parameter defining the maximum number of objects processed per run of Django batch methods, e.g. bulk_create and bulk_update.

EVEUNIVERSE_LOAD_ASTEROID_BELTS = False

When true will automatically load astroid belts with every solar system.

EVEUNIVERSE_LOAD_DOGMAS = False

When true will automatically load dogma, e.g. with every type.

EVEUNIVERSE_LOAD_GRAPHICS = False

When true will automatically load graphics with every type.

EVEUNIVERSE_LOAD_MARKET_GROUPS = False

When true will automatically load market groups with every type.

EVEUNIVERSE_LOAD_MOONS = False

When true will automatically load moons be with every planet.

EVEUNIVERSE_LOAD_PLANETS = False

When true will automatically load planets with every solar system.

EVEUNIVERSE_LOAD_STARGATES = False

When true will automatically load stargates with every solar system.

EVEUNIVERSE_LOAD_STARS = False

When true will automatically load stars with every solar system.

EVEUNIVERSE_LOAD_STATIONS = False

When true will automatically load stations be with every solar system.

EVEUNIVERSE_LOAD_TASKS_PRIORITY = 6

Priority of tasks for data loads. This priority should be below 5 to not interfere with normal task operation.

EVEUNIVERSE_LOAD_TYPE_MATERIALS = False

When true will automatically load type materials be with every type.

EVEUNIVERSE_REQUESTS_DEFAULT_TIMEOUT = 5

Default timeout for HTTP GET requests in seconds.

EVEUNIVERSE_TASKS_TIME_LIMIT = 7200

Global timeout for tasks in seconds to reduce task accumulation during outages.

EVEUNIVERSE_USE_EVESKINSERVER = True

When True a call to EveType.icon_url for a SKIN type will return a eveskinserver URL else it will return a generic SKIN icon.

Management commands

The following management commands are available:

eveuniverse_load_data

This command will load large sets of data from ESI into local database for selected topics. This is useful to optimize performance or when you want to provide the user with drop-down lists covering all values in a topic (e.g. ship types). Available topics are:

  • map: All regions, constellations and solar systems

  • ships: All ship types

  • structures: All structures types

  • types: All types

Here is how you can use this command (not including default Django arguments):

usage: manage.py eveuniverse_load_data [-h]
                        [--types-enabled-sections {dogmas,graphics,market_groups,type_materials,industry_activities} [{dogmas,graphics,market_groups,type_materials,industry_activities} ...]]
                        [--map-enabled-sections {dogmas,graphics,market_groups,type_materials,industry_activities} [{dogmas,graphics,market_groups,type_materials,industry_activities} ...]]
                        [--version] [-v {0,1,2,3}] [--settings SETTINGS]
                        [--pythonpath PYTHONPATH] [--traceback] [--no-color]
                        [--force-color] [--skip-checks]
                        {map,ships,structures,types} [{map,ships,structures,types} ...]

Load large sets of data from ESI into local database for selected topics

positional arguments:
  {map,ships,structures,types}
                        Topic(s) to load data for

options:
  -h, --help            show this help message and exit
  --types-enabled-sections {dogmas,graphics,market_groups,type_materials,industry_activities} [{dogmas,graphics,market_groups,type_materials,industry_activities} ...]
                        List of enabled sections for types, ships and structures topics
  --map-enabled-sections {dogmas,graphics,market_groups,type_materials,industry_activities} [{dogmas,graphics,market_groups,type_materials,industry_activities} ...]
                        List of enabled sections for map topic

eveuniverse_purge_all

This command will purge ALL data of your models.

eveuniverse_load_types

This command loads large sets of types as specified from ESI into the local database.

Here is how you can use this command (not including default Django arguments):

usage: manage.py eveuniverse_load_types [-h] [--category_id CATEGORY_ID]
                        [--category_id_with_dogma CATEGORY_ID_WITH_DOGMA]
                        [--group_id GROUP_ID] [--group_id_with_dogma GROUP_ID_WITH_DOGMA]
                        [--type_id TYPE_ID] [--type_id_with_dogma TYPE_ID_WITH_DOGMA]
                        [--disable_esi_check]
                        app_name

Loads large sets of types as specified from ESI into the local database. This is a helper command meant
to be called from other apps only.

positional arguments:
  app_name              Name of app this data is loaded for

options:
  -h, --help            show this help message and exit
  --category_id CATEGORY_ID
                        Eve category ID to be loaded excl. dogma
  --category_id_with_dogma CATEGORY_ID_WITH_DOGMA
                        Eve category ID to be loaded incl. dogma
  --group_id GROUP_ID   Eve group ID to be loaded excl. dogma
  --group_id_with_dogma GROUP_ID_WITH_DOGMA
                        Eve group ID to be loaded incl. dogma
  --type_id TYPE_ID     Eve type ID to be loaded excl. dogma
  --type_id_with_dogma TYPE_ID_WITH_DOGMA
                        Eve type ID to be loaded incl. dogma
  --disable_esi_check   Disables checking that ESI is online

Database tools

On some DBMS like MySQL it is not possible to reset the database and remove all eveuniverse tables with the standard “migrate zero” command. The reason is that eveuniverse is using composite primary keys and Django seams to have problems dealing with that correctly, when trying to roll back migrations.

As workaround you will need remove all tables with SQL commands. To make this easier we are providing a SQL script that contains all commands to drop the tables. The process for “migrating to zero” is then as follows:

  1. Run SQL script drop_tables.sql on your database

  2. Run python manage.py migrate eveuniverse zero --fake