Upgrade FlexDeploy - Tomcat

Owned by Hayden Mumm
Last updated: Dec 05, 2021 by Chandresh Patel

This page provides instructions for upgrading a FlexDeploy installation. These instructions may upgrade Tomcat to the version included in the distribution zip depending on current version of FlexDeploy. FlexDeploy packages Tomcat version 9.0.54.

Note that FlexDeploy requires Java 8 for Tomcat, if you are upgrading from older version you may have to upgrade Java as well.

It was also noticed that <JarScanner scanManifest="false"/> was missing in some older context.xml files, which caused java.io.FileNotFoundException: /opt/flexagon/tomcat/apache-tomcat-flexdeploy/lib/gdk_custom.jar (No such file or directory).

Make sure you have downloaded new version of FlexDeploy and unzipped it, so it can be accessed during upgrade process.

Upgrade Instructions

Step 1. Stop the Tomcat server. Use shutdown.sh on Linux and shutdown.bat on Windows.

Step 2. Backup the FlexDeploy schemas, Artifact Repository and apache-tomcat-flexdeploy folder. In case you need to back out FlexDeploy, you can use this backup data.

For back-out purposes, it is highly recommended that a database backup of the database be taken using standard tools like RMAN for Oracle. Another option would be to export the FD, FD_ADMIN, and FF schemas using data pump. The schemas can be exported using the following data pump command for Oracle as shown below. Follow standard procedures for PostgreSQL for database backup.

Recommendation for Oracle database backup

You will need to setup environment variables before running expdp. For example,

export ORACLE_HOME=/u01/app/oracle/product/12.1.0/dbhome_1
export ORACLE_SID=fd01

export statements above are for example only as values will be different for your installation. 

expdp system directory=DATA_PUMP_DIR dumpfile=fd_bkup.dmp schemas=FD,FF,FD_ADMIN

If this database is exclusively used for just FlexDeploy application, you can rely on Database backup and recovery processes (RMAN) 
instead of export data procedure.

It is recommended that a backup of the artifact repository file system be taken in case a back-out is required. The location of the artifact repository can be identified by looking at value of -Dflexagon.fd.repository.root in setenvoverride.sh/setenv.sh or setenvoverride.bat/setenv.bat file. This file will be in <FlexDeploy Home>/apache-tomcat-flexdeploy/bin/. Standard file backup procedures can be followed. See following example for unix only, which can be used to create a tar file backup of the entire directory.

Backup Artifact Repository

cd <artifact repository location>
tar –cvzf artifact.tar.gz <artifact repository location>

Step 3. Upgrade the FlexDeploy schemas

Locate upgrade scripts folder in distribution zip file. For example, /database/oracle/upgrade, /database/postgres/upgrade. This folder(s) includes a number of folders containing migration.sql, which upgrade the FlexDeploy schemas.

As part of FlexDeploy 5.4 upgrade, missing UK constraint was added for REL_METADATA_VALUE table. Please check that there are no duplicate rows by running following SQL prior to upgrade. If you find duplicates then contact us via support portal.

SELECT * FROM (SELECT REL_DEFINITION_ID, PIPELINE_METADATA_ID, COUNT(1) NUMROWS
FROM FD.REL_METADATA_VALUE
GROUP BY REL_DEFINITION_ID, PIPELINE_METADATA_ID)
WHERE NUMROWS>1;

If you are upgrading from version prior to FlexDeploy 5.2, run following SQL prior to running upgrade scripts and make sure no rows are returned. If there are any row returned, it is possible that migration may not work correctly, in which case contact us via support portal.

SELECT
    kd.property_key_name, k.*
FROM
    fd.property_key k,
    fd.property_key_definition kd,
    fd.property_value pv
WHERE
    k.property_definition_id = kd.property_definition_id
    AND pv.sequence_number = 0
    AND k.property_key_id = pv.property_key_id
    AND kd.is_encrypted = 'Y'
    AND k.is_encrypted = 'N'
    AND pv.property_value not in ('<NOT CONFIGURED>', '<OPTIONAL>');

FlexDeploy 5.1 migration includes updates to Unique Constraint for FF.DB_PROPERTIES_DATA. If you are upgrading from version prior to 5.1, then run following SQL before running upgrade scripts. If you have duplicates then it needs to be fixed up prior to starting migration. If there are duplicates, work with Flexagon support team to resolve prior to migration.

SELECT * FROM (SELECT DB_PROPERTIES_ID, SEQUENCE_NUMBER, COUNT(1) NUMROWS
FROM FF.DB_PROPERTIES_DATA 
GROUP BY DB_PROPERTIES_ID, SEQUENCE_NUMBER) 
WHERE NUMROWS>1;

You will need to execute one or more SQL script(s) from the version of FlexDeploy you currently have, up to the latest version. And they must be executed in that order.

See some examples below to help you understand database upgrade process.

Upgrading 5.3.0.0 to 5.5.0.0
v5.3.0.0-to-v5.3.0.2/migration.sql
v5.3.0.2-to-v5.3.0.3/migration.sql
v5.3.0.3-to-v5.3.0.4/migration.sql
v5.3.0.4-to-v5.4.0.0/migration.sql
v5.4.0.0-to-v5.4.0.1/migration.sql
v5.4.0.1-to-v5.4.0.3/migration.sql
v5.4.0.3-to-v5.4.0.4/migration.sql
v5.4.0.4-to-v5.5.0.0/migration.sql
Upgrading 5.4.0.0 to 5.5.0.0
v5.4.0.0-to-v5.4.0.1/migration.sql
v5.4.0.1-to-v5.4.0.3/migration.sql
v5.4.0.3-to-v5.4.0.4/migration.sql
v5.4.0.4-to-v5.5.0.0/migration.sq
Upgrading 5.4.0.2 to 5.5.0.0
v5.4.0.1-to-v5.4.0.3/migration.sql
v5.4.0.3-to-v5.4.0.4/migration.sql
v5.4.0.4-to-v5.5.0.0/migration.sql

Executing migration files

Once you have decided which migration scripts to execute, you need to run them against your database using database specific tools in proper sequence.

Oracle

These scripts should be executed as system, sys, or another user which has privileges to create objects in other schemas, performs grants, etc.  You can load and execute these scripts using other tools such as Toad or SQL Developer. Instructions below are for using the sqlplus client.

sqlplus

Oracle SQL*Plus is part of the Oracle Client, and may be executed on any host which has it installed. If the Oracle Client is not installed on the same host as where you are installing FlexDeploy, you can copy the scripts to another host which has it installed.

Make sure to spool output to a file, so it can be analyzed later if necessary.

To launch sqlplus:

export ORACLE_HOME=<your Oracle Home directory>
export ORACLE_SID=<the SID of your database>
spool migration5.2.0.out #(change for each script execution so logs are preserved)
$ORACLE_HOME/bin/sqlplus #(login as sys or system when prompted)
SQL> <<execute scripts as shown above>>

If running on another host other than the database server, you must update your tnsnames file and and launch sqlplus as follows:

export ORACLE_HOME=<your Oracle Home directory>
export ORACLE_SID=<the SID of your database>
spool migration5.2.0.out #(change for each script execution so logs are preserved)
$ORACLE_HOME/bin/sqlplus system@<tnsname>
SQL> <<execute scripts as shown above>>

For example,

PostgreSQL

These scripts should be executed as postgres or another super user which has privileges to create objects in other schemas, performs grants, etc. Change the bold text as needed in your environment.

Linux
  • Run FlexDeploy migration scripts. Type the password when asked. PostgresDDL.txt records the output of the script so that you can refer back to it later.
    • cd <Flexdeploy Unzip Folder>\database\postgres\upgrade\vx.y.0.z-v.a.b.0.c
    • /path/to/psql -U postgres -h hostname -p 5432 -d flexdeploy -a -b -e -f migration.sql > PostgresDDL.txt
Windows
  • Run FlexDeploy scripts. Type the password when asked. PostgresDDL.txt records the output of the script so that you can refer back to it later. 
    • cd <Unzip Folder>/database/postgres/upgrade/vx.y.0.z-v.a.b.0.c
    • C:\path\to\psql -U postgres -h hostname -p 5432 -d flexdeploy -a -b -e -f migration.sql > PostgresDDL.txt

Step 4. Take a backup and delete apache-tomcat-flexdeploy folder (this will be replaced by new copy). It is required to take backup since the old configuration files will need to be referenced. See example below for Linux. This folder was updated as part of 5.6.0.4, but you can follow this process for every upgrade. 

# Assuming FLEXDEPLOY_HOME environment variable is set, it should be parent folder of apache-tomcat-flexdeploy
# export FLEXDEPLOY_HOME=/u01/flexdeploy
cd $FLEXDEPLOY_HOME
# change version in backup folder as per version you are upgrading from
mv apache-tomcat-flexdeploy/ apache-tomcat-flexdeploy-5603/

Step 5. Copy apache-tomcat-flexdeploy folder from distribution zip.

Linux Example
# let's say download zip was extracted in /u01/software/flexdeploy
# export UNZIPPED_FLEXDEPLOY=/u01/software/flexdeploy
cd $FLEXDEPLOY_HOME
mkdir apache-tomcat-flexdeploy
# assuming download zip has been unzipped in UNZIPPED_FLEXDEPLOY folder. 
cd $UNZIPPED_FLEXDEPLOY/apache-tomcat-flexdeploy
cp -R * $FLEXDEPLOY_HOME/apache-tomcat-flexdeploy/
cd $FLEXDEPLOY_HOME/apache-tomcat-flexdeploy/
chmod +x ./bin/*.sh
cp $FLEXDEPLOY_HOME/apache-tomcat-flexdeploy-5603/libext/* $FLEXDEPLOY_HOME/apache-tomcat-flexdeploy/libext/

Note on additional extension jar files : If you had placed additional libraries in apache-tomcat-flexdeploy/lib folder or in the apache-tomcat-flexdeploy/libext folder, then you will need to copy them from backup folder into the new apache-tomcat-flexdeploy/libext folder. Note that all extension jars should be kept in libext folder. Examples might be JDBC driver (for Oracle drivers there is additional detail in step 6), CyberArk libraries for Tomcat connection pool provider etc.

Step 6. Copy the context.xml file from the backup folder into <FlexDeploy Home>/apache-tomcat-flexdeploy/conf.

Linux Example
cd $FLEXDEPLOY_HOME/apache-tomcat-flexdeploy-5603/conf/
cp context.xml ../../apache-tomcat-flexdeploy/conf/

If using Oracle database,

  • Download the Oracle JDBC 8 driver from https://www.oracle.com/database/technologies/appdev/jdbc-downloads.html. Click on the driver link matching your Oracle database version, and download the driver compatible with JDK8. For example, ojdbc8-full.tar.gz.
  • Remove ojdbc jar files like ojdbc*.jar or ojdbc*dms.jar from <FlexDeploy Home>/apache-tomcat-flexdeploy/lib.

  • Copy all jars from downloaded jdbc driver to <FlexDeploy Home>/apache-tomcat-flexdeploy/libext. You may have done this already as part of Step 5.

  • If using Oracle database and have not switched to using UCP conection pool, then see Implementing UCP connections with Tomcat and Oracle for more information. Look at type in context.xml, if it is oracle.ucp.jdbc.PoolDataSource, then you are already setup to use UCP connection pool.

Step 7. Edit the <FlexDeploy Home>/apache-tomcat-flexdeploy/conf/server.xml and modify the HTTP/HTTPS ports to match that of the original file which was backed up in step 4. If you had setup keystore details, copy that as well from backup file. DO NOT simply copy over the backup file as there may be other changes.

Step 8. Starting with FlexDeploy 5.6.0.1 changes will no longer be made in setenv.sh (or .bat).

If you were already using setenvoverride.sh, then copy that from backup folder (Step 4), otherwise any changes previously done for setenv.sh must be added to setenvoverride.sh (or .bat). For example, Artifact directory, application directory, and java_home changes will need to be transferred over from your old setenv.sh file. If your old file looks like this:

Your new setenvoverride.sh should look like this:

Step 9. Copy plugins from download zip for auto upload.

FlexDeploy download zip contains plugin jar files. You just need to 1) copy plugin jars from download zip to plugins folder in your installation (details explained below) and 2) all files will be automatically uploaded and 3) activated on the server startup.

  1. Plugin files to copy are in following location in download zip.
    • Tomcat Download - application/plugins
    • WebLogic Download - plugins

  2. Let's determine where to copy files in your installation now. Locate flexagon.fd.install.root folder for your FlexDeploy installation, which is FlexDeploy working directory. This is specified on the Java command line. For example, -Dflexagon.fd.install.root=/u01/flexdeploy/application.
    • For Tomcat, you can look for this information in setenv.bat or setenv.sh file depending whether using Windows or Unix.
    • For Weblogic, find it in the managed server startup settings.
       
  3. Create plugins folder if it does not exist under flexagon.fd.install.root folder. Copy files from download zip (#1 above) to this plugins folder.

     

Step 10 . Start tomcat by running the following script

<FlexDeploy Home>/apache-tomcat-flexdeploy/bin/startup.sh (startup.bat on Windows)

Step 11. Launch FlexDeploy in your browser - http://<hostname>:<port>/flexdeploy

Step 12. There is no need to activate plugins manually. FlexDeploy will automatically upload and activate new versions of plugins once they are copied to plugins folder. You will notice that plugins that you copied in Step 8 will eventually be uploaded automatically and folder will not have any jar files left. You can also look at Administration - Plugins in UI and see newer versions of plugins.

The following macros are not currently supported in the footer:
  • style