/
InfluxDB in FlexDeploy

InfluxDB in FlexDeploy

Owned by Karl Henselin
Last updated: Oct 11, 2023
5 min read

InfluxDB is an open-source time series database for the storage and retrieval of time series data. It is designed to store large volumes of time series data and quickly perform analysis of the stored data. Within FlexDeploy, these features are leveraged to provide end-users with Value Stream Metrics that are available on the Dashboard page and the Project Insights section in all the Projects.

InfluxDB System Requirements

  • 2 to 4 GB of available RAM for the system

  • 1+ CPU or vCPU Cores

  • Recommend 5GB of disk space for storage of InfluxDB data with a minimum of 500 IOPS for the storage device. This is a part of the total space recommended for the Application Directory here

  • At least 16384 available file descriptors (ulimit) on Linux machines

InfluxDB Directory Structure with FlexDeploy

The executables for the database and the command line client along with the associated configuration are stored in the influxdb directory in the <FLEXDEPLOY_HOME>

The files in the config directory:

  • config.yaml - Used to store customizations to the overall InfluxDB configuration. Details on available options can be found here

  • fdinflux.config - Stores details that will be used by FlexDeploy to connect to the InfluxDB instance like URL, Org, and Bucket. Can also be used to override the default value for the wait duration between writes.

The time series data for the InfluxDB instance is stored in <Application Directory>/vsm/data

The presence of the wal directory indicates that the Org has been successfully set up in InfluxDB and data is being written here.

InfluxDB Token Management

The influx auth command can be used to manage the API Tokens for use with FlexDeploy. Further information about the command can be found here.

  • The influx auth list command can be used to list all the API Tokens available for the instance.

  • The influx auth create command can be used to create a new API Token that can be used by FlexDeploy.
    FlexDeploy requires the created API Token to have the following permissions at the minimum.

    read-buckets read-orgs write-buckets write-orgs

  • The influx auth active -i <token_id> and the influx auth inactive -i <token_id> commands can be used to activate and deactivate API Tokens temporarily as required.

  • The influx auth delete -i <token_id> command can be used to delete any unused API Tokens.

The current API Token that you wish to use with FlexDeploy must be set as the value for the FD_INFLUXDB_ADMIN_TOKEN credential.

Retention Policy for Metrics/Insights

The retention policy for the time series data in InfluxDB is set to 2 years after the installation of FlexDeploy. This is separate from the Purge Settings available in FlexDeploy and only covers the duration of data available to be displayed in Metrics/Insights. Any data older than this is automatically removed by InfluxDB at regular intervals.

The retention period can be modified by using the below command if required.

Linux

# The id of the bucket can be obtained from the "influx bucket ls" command ${FLEXDEPLOY_HOME}/influxdb/influx bucket update --id <bucket_id> --retention <rentention_period>

 

Windows

# The id of the bucket can be obtained from the "influx bucket ls" command %FLEXDEPLOY_HOME%\influxdb\influx bucket update --id <bucket_id> --retention <rentention_period>

Common Troubleshooting

 

Influx Log is growing very fast

Check your open file limits. Influx opens a large number of files to complete its work. Usually the log messages are about this.

ulimit -Hn

will show the open file limit for the current user. Make sure you run it as the user who will be executing FlexDeploy and Influx.

If it is less that 65536, we recommend raising it.

sudo vi /etc/security/limits.conf

where oracle is the user logged into the box to run FlexDeploy:

oracle soft nofile 16384

oracle hard nofile 65536

Check if InfluxDB is currently running

The influx ping command can be used to check if InfluxDB is currently running. Influx starts from the StartFlexDeploy.sh script. Please always use the StartFlexDeploy.sh script to start FlexDeploy.

If you are running multiple influx servers on your box, or you are running on a different port, then your command will be influx ping --host ${INFLUX_DB_URL}

If InfluxDB is running, “OK“ is printed as the output with a 0 return code.

If InfluxDB is not currently running, the return code is 1 with an error along the following lines.

Error: Get "http://localhost:8086/health": dial tcp [::1]:8086: connect: connection refused

Resolving unauthorized access Issue

If influxdb is having a token authentication issue, you will see something like this:

Query execution failed - [HTTP_REQUEST_FAIL: Request failed: {"code":"unauthorized","message":"unauthorized access"}].

We will need to either read current token, or generate new token and then set the token value in FlexDeploy credential. Note that there is no need to restart FlexDeploy tomcat server.

Step 1.1 - Reading current tokens

You can find the value of any existing tokens with the following commands:

Linux / Docker

#alter the next 2 lines as needed. #If you are using docker, you would need to run docker exec -it <containername> sh and your working directory may be /dockervolume/application or /home/oracle/application FLEXDEPLOY_HOME=/u01/flexdeploy WORKING_DIRECTORY=${FLEXDEPLOY_HOME}/application cd ${FLEXDEPLOY_HOME}/apache-tomcat-flexdeploy/bin ./influxd-stop.sh ${FLEXDEPLOY_HOME}/influxdb/influxd recovery auth list --bolt-path ${WORKING_DIRECTORY}/vsm/data/influxd.bolt ./influxd-start.sh

Windows

#alter the next 2 lines as needed. #If you are using docker, your working directory may be /dockervolume/application or /home/oracle/application set FLEXDEPLOY_HOME=/u01/flexdeploy set WORKING_DIRECTORY=%FLEXDEPLOY_HOME%/application cd %FLEXDEPLOY_HOME%\apache-tomcat-flexdeploy\bin .\influxd-stop.ps1 %FLEXDEPLOY_HOME%\influxdb\influxd recovery auth list --bolt-path %WORKING_DIRECTORY%/vsm/data/influxd.bolt .\influxd-start.ps1

Your result may be something like:

2023-03-09T14:17:02.264054Z info Resources opened {"log_id": "0gTgZ_Dl000", "system": "bolt-kvstore", "path": "/u01/flexdeploy/application/vsm/data/influxd.bolt"} ID User Name User ID Description Token Permissions 0a0c1dafc9159000 flexuser 0a0c1daf96d59000 flexuser's Token kjnwlkjnalskdfjh3987sn [read:authorizations write:authorizations read:buckets write:buckets read:dashboards write:dashboards read:orgs write:orgs read:sources write:sources read:tasks write:tasks read:telegrafs write:telegrafs read:users write:users read:variables write:variables read:scrapers write:scrapers read:secrets write:secrets read:labels write:labels read:views write:views read:documents write:documents read:notificationRules write:notificationRules read:notificationEndpoints write:notificationEndpoints read:checks write:checks read:dbrp write:dbrp read:notebooks write:notebooks read:annotations write:annotations read:remotes write:remotes read:replications write:replications]

Token value is seen in Token column. If you want to use another token or did not find any token, use next step to generate a new token.

Step 1.2 - Generate a new token (if not found)

The following command will generate a new token for InfluxDB.

Linux

#alter the next 2 lines as needed. #If you are using docker, your working directory may be /dockervolume/application or /home/oracle/application FLEXDEPLOY_HOME=/u01/flexdeploy WORKING_DIRECTORY=${FLEXDEPLOY_HOME}/application cd ${FLEXDEPLOY_HOME}/apache-tomcat-flexdeploy/bin ./influxd-stop.sh ${FLEXDEPLOY_HOME}/influxdb/influxd recovery auth create-operator --bolt-path ${WORKING_DIRECTORY}/vsm/data/influxd.bolt --username flexuser --org flexdeploy ./influxd-start.sh

Windows

#alter the next 2 lines as needed. #If you are using docker, your working directory may be /dockervolume/application or /home/oracle/application set FLEXDEPLOY_HOME=/u01/flexdeploy set WORKING_DIRECTORY=%FLEXDEPLOY_HOME%/application cd %FLEXDEPLOY_HOME%\apache-tomcat-flexdeploy\bin .\influxd-stop.ps1 %FLEXDEPLOY_HOME%\influxdb\influxd recovery auth create-operator --bolt-path %WORKING_DIRECTORY%\vsm\data\influxd.bolt --username flexuser --org flexdeploy .\influxd-start.ps1

 

the output will contain something like:

flexuser's Recovery Token gCQtXdu_zIbztH-tOkccUx9o1HQZinoaKGivqIT2MAn5X8b-RjjED3RK7QRhRhxS0PqK_2AmZwHcKf08IE7SkA==

In this case, gCQtXdu_zIbztH-tOkccUx9o1HQZinoaKGivqIT2MAn5X8b-RjjED3RK7QRhRhxS0PqK_2AmZwHcKf08IE7SkA== is the token.

Step 2 - Update Token value in FlexDeploy

Launch FlexDeploy, go to Credentials page and look for FD_INFLUXDB_ADMIN_TOKEN credential. Then update it’s value to your token.

No data is shown in Dashboard / Project Insights soon after first upgrading FlexDeploy to 6.0+

After the first startup with InfluxDB, relevant data is migrated from the existing FlexDeploy database for use in the Dashboard / Project Insights. During the course of this migration(running in the background), the InfluxDB API is unavailable and no data is shown for the various available metrics. Data is automatically populated in the configured Dashboard after this migration is completed. If it is not showing up after a day, then consider reaching out to FlexDeploy support team.

Related content

The following macros are not currently supported in the footer:
  • style