Installation ############ This application is written in Python (API) and Typescript (client): - API: - Flask - `gpxpy `_ to parse gpx files - `staticmap `_ to generate a static map image from gpx coordinates - `python-forecast.io `_ to fetch weather data from `Dark Sky `__ (former forecast.io) - `dramatiq `_ for task queue - `Authlib `_ for OAuth 2.0 Authorization support - `Flask-Limiter `_ for API rate limits - Client: - Vue3/Vuex - `Leaflet `__ to display map - `Chart.js `__ to display charts with elevation and speed | Logo, some sports and weather icons are made by `Freepik `__ from `www.flaticon.com `__. | FitTrackee also uses icons from `Fork Awesome `__. Prerequisites ~~~~~~~~~~~~~ - mandatory - Python 3.7+ - PostgreSQL 11+ - optional - Redis for task queue (if email sending is enabled and for data export requests) and API rate limits - SMTP provider (if email sending is enabled) - API key from a `weather data provider `__ - `Poetry `__ (for installation from sources only) - `Yarn `__ (for development only) - Docker and Docker Compose (for development or evaluation purposes) .. note:: | If registration is enabled, it is recommended to set Redis and a SMTP provider for email sending and data export requests. .. note:: | The following steps describe an installation on Linux systems (tested on Debian and Arch). | On other OS, some issues can be encountered and adaptations may be necessary. Environment variables ~~~~~~~~~~~~~~~~~~~~~ .. warning:: | Since FitTrackee 0.4.0, ``Makefile.custom.config`` is replaced by ``.env`` The following environment variables are used by **FitTrackee** web application or the task processing library. They are not all mandatory depending on deployment method. .. envvar:: FLASK_APP | Name of the module to import at flask run. | ``FLASK_APP`` should contain ``$(PWD)/fittrackee/__main__.py`` with installation from sources, else ``fittrackee``. .. envvar:: HOST **FitTrackee** host. :default: 127.0.0.1 .. envvar:: PORT **FitTrackee** port. :default: 5000 .. envvar:: APP_SETTINGS **FitTrackee** configuration. :default: fittrackee.config.ProductionConfig .. envvar:: APP_SECRET_KEY **FitTrackee** secret key, must be initialized in production environment. .. warning:: Use a strong secret key. This key is used in JWT generation. .. envvar:: APP_WORKERS Number of workers spawned by **Gunicorn**. :default: 1 .. envvar:: APP_LOG .. versionadded:: 0.4.0 Path to log file .. envvar:: UPLOAD_FOLDER .. versionadded:: 0.4.0 **Absolute path** to the directory where `uploads` folder will be created. :default: `/fittrackee` .. danger:: | With installation from PyPI, the directory will be located in **virtualenv** directory if the variable is not initialized. .. envvar:: DATABASE_URL | Database URL with username and password, must be initialized in production environment. | For example in dev environment : ``postgresql://fittrackee:fittrackee@localhost:5432/fittrackee`` .. warning:: | Since `SQLAlchemy update (1.4+) `__, engine URL should begin with `postgresql://`. .. envvar:: DATABASE_DISABLE_POOLING .. versionadded:: 0.4.0 Disable pooling if needed (when starting application with **FitTrackee** entry point and not directly with **Gunicorn**), see `SqlAlchemy documentation `__. :default: false .. envvar:: UI_URL **FitTrackee** URL, needed for links in emails. .. envvar:: EMAIL_URL .. versionadded:: 0.3.0 Email URL with credentials, see `Emails `__. .. versionchanged:: 0.6.5 :default: empty string .. danger:: If the email URL is empty, email sending will be disabled. .. warning:: If the email URL is invalid, the application may not start. .. envvar:: SENDER_EMAIL .. versionadded:: 0.3.0 **FitTrackee** sender email address. .. envvar:: REDIS_URL .. versionadded:: 0.3.0 Redis instance used by **Dramatiq** and **Flask-Limiter**. :default: local Redis instance (``redis://``) .. envvar:: WORKERS_PROCESSES .. versionadded:: 0.3.0 Number of processes used by **Dramatiq**. .. envvar:: API_RATE_LIMITS .. versionadded:: 0.7.0 API rate limits, see `API rate limits `__. :default: `300 per 5 minutes` .. envvar:: TILE_SERVER_URL .. versionadded:: 0.4.0 | Tile server URL (with api key if needed), see `Map tile server `__. | Since **0.4.9**, it's also used to generate static maps (to keep default server, see `DEFAULT_STATICMAP `__) :default: `https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png` .. envvar:: STATICMAP_SUBDOMAINS .. versionadded:: 0.6.10 | Some tile servers require a subdomain, see `Map tile server `__. | For instance: "a,b,c" for OSM France. :default: empty string .. envvar:: MAP_ATTRIBUTION .. versionadded:: 0.4.0 Map attribution (if using another tile server), see `Map tile server `__. :default: `© OpenStreetMap contributors` .. envvar:: DEFAULT_STATICMAP .. versionadded:: 0.4.9 | If `True`, it keeps using default tile server to generate static maps (Komoot.de tile server). | Otherwise, it uses the tile server set in `TILE_SERVER_URL `__. .. versionchanged:: 0.6.10 | This variable is now case-insensitive. | If `False`, depending on tile server, `subdomains `__ may be mandatory. :default: False .. envvar:: WEATHER_API_KEY .. versionchanged:: 0.4.0 ⚠️ replaces ``WEATHER_API`` Weather API key (not mandatory), see ``WEATHER_API_PROVIDER``. .. envvar:: WEATHER_API_PROVIDER 🆕 .. versionadded:: 0.7.11 Provider for weather data (not mandatory), see `Weather data `__. .. envvar:: VUE_APP_API_URL **FitTrackee** API URL, only needed in dev environment. Emails ^^^^^^ .. versionadded:: 0.3.0 To send emails, a valid ``EMAIL_URL`` must be provided: - with an unencrypted SMTP server: ``smtp://username:password@smtp.example.com:25`` - with SSL: ``smtp://username:password@smtp.example.com:465/?ssl=True`` - with STARTTLS: ``smtp://username:password@smtp.example.com:587/?tls=True`` .. warning:: | - If the email URL is invalid, the application may not start. | - Sending emails with Office365 may not work if SMTP auth is disabled. .. versionchanged:: 0.5.3 | Credentials can be omitted: ``smtp://smtp.example.com:25``. | If ``:`` is omitted, the port defaults to 25. .. warning:: | Since 0.6.0, newly created accounts must be confirmed (an email with confirmation instructions is sent after registration). Emails sent by FitTrackee are: - account confirmation instructions - password reset request - email change (to old and new email adresses) - password change - notification when a data export archive is ready to download (*new in 0.7.13*) .. versionchanged:: 0.6.5 For single-user instance, it is possible to disable email sending with an empty ``EMAIL_URL`` (in this case, no need to start dramatiq workers). A `CLI `__ is available to activate account, modify email and password and handle data export requests. Map tile server ^^^^^^^^^^^^^^^ .. versionadded:: 0.4.0 Default tile server is now **OpenStreetMap**'s standard tile layer (if environment variables are not initialized). The tile server can be changed by updating ``TILE_SERVER_URL`` and ``MAP_ATTRIBUTION`` variables (`list of tile servers `__). To keep using **ThunderForest Outdoors**, the configuration is: - ``TILE_SERVER_URL=https://{s}.tile.thunderforest.com/outdoors/{z}/{x}/{y}.png?apikey=XXXX`` where **XXXX** is **ThunderForest** API key - ``MAP_ATTRIBUTION=© Thunderforest, © OpenStreetMap contributors`` .. note:: | Check the terms of service of tile provider for map attribution. .. versionchanged:: 0.6.10 Since the tile server can be used for static map generation, some servers require a subdomain. For instance, to set OSM France tile server, the expected values are: - ``TILE_SERVER_URL=https://{s}.tile.openstreetmap.fr/osmfr/{z}/{x}/{y}.png`` - ``MAP_ATTRIBUTION='fond de carte par OpenStreetMap France, sous licence CC BY-SA'`` - ``STATICMAP_SUBDOMAINS=a,b,c`` The subdomain will be chosen randomly. API rate limits ^^^^^^^^^^^^^^^ .. versionadded:: 0.7.0 | API rate limits are managed by `Flask-Limiter `_, based on IP with fixed window strategy. | To enable rate limits, **Redis** must be available. .. note:: | If no Redis instance is available for rate limits, FitTrackee can still start. | All endpoints are subject to rate limits, except endpoints serving assets. | Limits can be modified by setting the environment variable ``API_RATE_LIMITS`` (see `Flask-Limiter documentation for notation `_). | Rate limits must be separated by a comma, for instance: .. code-block:: export API_RATE_LIMITS="200 per day, 50 per hour" **Flask-Limiter** provides a `Command Line Interface `_ for maintenance and diagnostic purposes. .. code-block:: bash $ flask limiter Usage: flask limiter [OPTIONS] COMMAND [ARGS]... Flask-Limiter maintenance & utility commmands Options: --help Show this message and exit. Commands: clear Clear limits for a specific key config View the extension configuration limits Enumerate details about all routes with rate limits Weather data ^^^^^^^^^^^^ .. versionchanged:: 0.7.11 The following weather data providers are supported by **FitTrackee**: - `Dark Sky `__ (deprecated, will stop on March 31st, 2023) - `Visual Crossing `__ (**note**: historical data are provided on hourly period) To configure a weather provider, set the following environment variables: - ``WEATHER_API_PROVIDER``: ``darksky`` for **Dark Sky** or ``visualcrossing`` for **Visual Crossing** - ``WEATHER_API_KEY``: the key to the corresponding weather provider Installation ~~~~~~~~~~~~ .. warning:: | Note that FitTrackee is under heavy development, some features may be unstable. From PyPI ^^^^^^^^^ .. note:: | Recommended way on production. - Create and activate a virtualenv - Install **FitTrackee** with pip .. code-block:: bash $ pip install fittrackee - Create ``fittrackee`` database Example : .. code-block:: sql CREATE USER fittrackee WITH PASSWORD ''; CREATE SCHEMA fittrackee AUTHORIZATION fittrackee; CREATE DATABASE fittrackee OWNER fittrackee; .. note:: | see PostgreSQL `documentation `_ for schema and privileges. - Initialize environment variables, see `Environment variables `__ For instance, copy and update ``.env`` file from ``.env.example`` and source the file. .. code-block:: bash $ nano .env $ source .env - Initialize database schema .. code-block:: bash $ ftcli db upgrade - Start the application .. code-block:: bash $ fittrackee - Start task queue workers if email sending is enabled, with flask-dramatiq CLI: .. code-block:: bash $ flask worker --processes 2 .. note:: | To start application and workers with **systemd** service, see `Deployment `__ - Open http://localhost:5000 and register - To set admin rights to the newly created account, use the following command line: .. code:: bash $ ftcli users update --set-admin true .. note:: If the user account is inactive, it activates it. From sources ^^^^^^^^^^^^ .. warning:: | Since FitTrackee 0.2.1, Python packages installation needs Poetry. | To install it on ArchLinux: .. code-block:: bash $ yay poetry $ poetry --version Poetry 1.0.17 # optional $ poetry config virtualenvs.in-project true For other OS, see `Poetry Documentation `__ Dev environment """"""""""""""" - Clone this repo: .. code:: bash $ git clone https://github.com/SamR1/FitTrackee.git $ cd FitTrackee - Create **.env** from example and update it (see `Environment variables `__). - Install Python virtualenv, Vue and all related packages and initialize the database: .. code:: bash $ make install-dev $ make install-db - Start the server and the client: .. code:: bash $ make serve - Run dramatiq workers: .. code:: bash $ make run-workers - Open http://localhost:3000 and register - To set admin rights to the newly created account, use the following command line: .. code:: bash $ make user-set-admin USERNAME= .. note:: If the user account is inactive, it activates it. Production environment """""""""""""""""""""" .. warning:: | Note that FitTrackee is under heavy development, some features may be unstable. - Download the last release (for now, it is the release v0.7.13): .. code:: bash $ wget https://github.com/SamR1/FitTrackee/archive/v0.7.13.tar.gz $ tar -xzf v0.7.13.tar.gz $ mv FitTrackee-0.7.13 FitTrackee $ cd FitTrackee - Create **.env** from example and update it (see `Environment variables `__). - Install Python virtualenv and all related packages: .. code:: bash $ make install-python - Initialize the database (**after updating** ``db/create.sql`` **to change database credentials**): .. code:: bash $ make install-db - Start the server and dramatiq workers: .. code:: bash $ make run .. note:: If email sending is disabled: ``$ make run-server`` - Open http://localhost:5000 and register - To set admin rights to the newly created account, use the following command line: .. code:: bash $ make user-set-admin USERNAME= .. note:: If the user account is inactive, it activates it. Upgrade ~~~~~~~ .. warning:: | Before upgrading, make a backup of all data: | - database (with `pg_dump `__ for instance) | - upload directory (see `Environment variables `__) From PyPI ^^^^^^^^^ - Stop the application and activate the virtualenv - Upgrade with pip .. code-block:: bash $ pip install -U fittrackee - Update environment variables if needed and source environment variables file .. code-block:: bash $ nano .env $ source .env - Upgrade database if needed (see changelog for migrations): .. code-block:: bash $ ftcli db upgrade - Restart the application and task queue workers (if email sending is enabled). From sources ^^^^^^^^^^^^ Dev environment """"""""""""""" - Stop the application and pull the repository: .. code:: bash $ git pull - Update **.env** if needed (see `Environment variables `__). - Upgrade packages: .. code:: bash $ make install-dev - Upgrade database if needed (see changelog for migrations): .. code:: bash $ make upgrade-db - Restart the server: .. code:: bash $ make serve - Run dramatiq workers: .. code:: bash $ make run-workers Prod environment """""""""""""""" - Stop the application - Change to the directory where FitTrackee directory is located - Download the last release (for now, it is the release v0.7.13) and overwrite existing files: .. code:: bash $ wget https://github.com/SamR1/FitTrackee/archive/v0.7.13.tar.gz $ tar -xzf v0.7.13.tar.gz $ cp -R FitTrackee-0.7.13/* FitTrackee/ $ cd FitTrackee - Update **.env** if needed (see `Environment variables `__). - Upgrade packages: .. code:: bash $ make install-dev - Upgrade database if needed (see changelog for migrations): .. code:: bash $ make upgrade-db - Restart the server and dramatiq workers: .. code:: bash $ make run .. note:: If email sending is disabled: ``$ make run-server`` Deployment ~~~~~~~~~~ There are several ways to start **FitTrackee** web application and task queue library. One way is to use a **systemd** services and **Nginx** to proxy pass to **Gunicorn**. Examples (to update depending on your application configuration and given distribution): - for application: ``fittrackee.service`` .. code-block:: [Unit] Description=FitTrackee service After=network.target After=postgresql.service After=redis.service StartLimitIntervalSec=0 [Service] Type=simple Restart=always RestartSec=1 User= StandardOutput=syslog StandardError=syslog SyslogIdentifier=fittrackee Environment="APP_SECRET_KEY=" Environment="APP_LOG=" Environment="UPLOAD_FOLDER=" Environment="DATABASE_URL=" Environment="UI_URL=" Environment="EMAIL_URL=" Environment="SENDER_EMAIL=" Environment="REDIS_URL=" Environment="TILE_SERVER_URL=" Environment="STATICMAP_SUBDOMAINS=" Environment="MAP_ATTRIBUTION=" Environment="WEATHER_API_KEY=" WorkingDirectory=/home// ExecStart=/home///.venv/bin/gunicorn -b 127.0.0.1:5000 "fittrackee:create_app()" --error-logfile /home///gunicorn.log Restart=always [Install] WantedBy=multi-user.target .. note:: To handle large files, a higher value for `timeout `__ can be set. .. note:: More information on deployment with Gunicorn in its `documentation `__. - for task queue workers: ``fittrackee_workers.service`` .. code-block:: [Unit] Description=FitTrackee task queue service After=network.target After=postgresql.service After=redis.service StartLimitIntervalSec=0 [Service] Type=simple Restart=always RestartSec=1 User= StandardOutput=syslog StandardError=syslog SyslogIdentifier=fittrackee_workers Environment="FLASK_APP=fittrackee" Environment="APP_SECRET_KEY=" Environment="APP_LOG=" Environment="UPLOAD_FOLDER=" Environment="DATABASE_URL=" Environment="UI_URL=" Environment="EMAIL_URL=" Environment="SENDER_EMAIL=" Environment="REDIS_URL=" WorkingDirectory=/home// ExecStart=/home///.venv/bin/flask worker --processes Restart=always [Install] WantedBy=multi-user.target - **Nginx** configuration: .. code-block:: server { listen 443 ssl http2; server_name example.com; ssl_certificate fullchain.pem; ssl_certificate_key privkey.pem; ## this parameter controls how large of a file can be ## uploaded, and defaults to 1MB. If you change the FitTrackee ## settings to allow larger uploads, you'll need to change this ## setting by uncommenting the line below and setting the size limit ## you want. Set to "0" to prevent nginx from checking the ## request body size at all # client_max_body_size 1m; location / { proxy_pass http://127.0.0.1:5000; proxy_redirect default; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $server_name; proxy_set_header X-Forwarded-Proto $scheme; } } server { listen 80; server_name example.com; location / { return 301 https://example.com$request_uri; } } .. note:: If needed, update configuration to handle larger files (see `client_max_body_size `_). Docker ~~~~~~ Installation ^^^^^^^^^^^^ .. versionadded:: 0.4.4 For evaluation purposes, docker files are available, installing **FitTrackee** from **sources**. - To install **FitTrackee**: .. code-block:: bash $ git clone https://github.com/SamR1/FitTrackee.git $ cd FitTrackee $ cp .env.docker .env $ make docker-build - To initialise database: .. code-block:: bash $ make docker-init - Open http://localhost:5000 and register. Open http://localhost:8025 to access `MailHog interface `_ (email testing tool) - To set admin rights to the newly created account, use the following command line: .. code:: bash $ make docker-set-admin USERNAME= .. note:: If the user account is inactive, it activates it. - To stop **Fittrackee**: .. code-block:: bash $ make docker-stop - To start **Fittrackee** (application and dramatiq workers): .. code-block:: bash $ make docker-run-all - To run shell inside **Fittrackee** container: .. code-block:: bash $ make docker-shell Development ^^^^^^^^^^^ .. versionadded:: 0.5.0 - an additional step is needed to install `fittrackee_client` .. code-block:: bash $ make docker-build-client - to start **FitTrackee** with client dev tools: .. code-block:: bash $ make docker-serve-client Open http://localhost:3000 .. note:: Some environment variables need to be updated like `UI_URL` - to run lint or tests: .. code-block:: bash $ make docker-lint-client # run lint on javascript files $ make docker-test-client # run unit tests on Client $ make docker-lint-python # run type check and lint on python files $ make docker-test-python # run unit tests on API