Getting started

Installation

Before we can do anything, we need a Python 3.7+ distribution. We recommend Miniconda. If you don’t already have it, go ahead and install it.

If you intend to install from source, you will also need Git.

Now that the prerequisites are satisfied, the next order of business is to install Timeflux and its dependencies. To keep things nice and clean, we will do this in a new virtual environment. Depending on your system, open a shell or command prompt:

conda create --name timeflux python=3.9 pytables bottleneck
conda activate timeflux
pip install timeflux
pip install timeflux_example

If everything went well, Timeflux is now installed. Hooray!

Basic usage

Applications are self-described in YAML files. Running an app is easy:

timeflux app.yaml

Let’s try!

First, download a very simple app that we will use as an example:

curl -O https://raw.githubusercontent.com/timeflux/timeflux/master/examples/test.yaml

If the timeflux environment is not already activated, do it:

conda activate timeflux

You can now run the test app:

timeflux -d test.yaml

You should see and a bunch of random numbers every second. Hit Ctrl+C to stop.

Did you notice the -d flag in the command line? It’s a shorthand for --debug, and this what allowed us to see the messages in the console.

Command line options

There are only a few options, and you can list them with:

timeflux --help

This should print:

usage: timeflux [-h] [-v] [-d] [-E ENV_FILE] [-e ENV] app

positional arguments:
  app                   path to the YAML or JSON application file

optional arguments:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit
  -d, --debug           enable debug messages
  -E ENV_FILE, --env-file ENV_FILE
                        path to an environment file
  -e ENV, --env ENV     environment variables

Besides the -d flag we already discussed, two options are worth mentioning: -E or --env-file and -e or --env.

Environment

Storing an app configuration in the environment is a good practice. There are a few ways of doing this:

If a file named .env is found in the current directory or in any of its parent directories, it will be loaded. A .env file looks like this:

# A comment that will be ignored
FOO=bar
MEANING_OF_LIFE=42

As we saw earlier, you can also specify a custom path to an environment file with the --env-file option.

Another way of setting environment variables is with the -e option:

timeflux -e FOO="bar" -e MEANING_OF_LIFE=42 app.yaml

Finally, you can temporarily set environment variables for the duration of the session, directly from the console.

Windows:

set FOO "bar"

Linux, MacOS:

export FOO="bar"

The following environment variables are understood by Timeflux:

  • TIMEFLUX_LOG_LEVEL_CONSOLE – This is the level of details printed in the console. Possible values are DEBUG, INFO, WARNING, ERROR and CRITICAL. The default value is INFO. Running the timeflux command with the -d flag is the same as setting this variable to DEBUG.

  • TIMEFLUX_LOG_LEVEL_FILE – This is the logging level when the output of the application is written to a file. This variable accepts the same values as previously. The default value is DEBUG.

  • TIMEFLUX_LOG_FILE – If set to a valid path, Timeflux will write the application output to a log file. Standard format codes are accepted.

  • TIMEFLUX_SLEEP – When a graph has a rate of zero, it will run as fast as possible, but will result in a high CPU load. Setting this variable to a non-zero value can help mitigating this issue. Default is 0.

  • TIMEFLUX_HOOK_PRE – Name of a Python module that will be run before executing the app.

  • TIMEFLUX_HOOK_POST – Name of a Python module that will be run after executing the app.

Others variables may be used by specific nodes and plugins. Refer to the relevant documentation for details.

By combining environment variables and templating, you can add logic to your YAML files and build configurable applications.

Plugins

Timeflux is modular. The timeflux Python package contains the core features and the most essential nodes. Plugins are standard Python packages that provide one or several nodes. Officially supported plugins can be found on Timeflux GitHub page. Some plugins (especially those dealing with hardware) have special requirements. Please refer to each plugin repository for installation instructions.

Notable plugins include: