News
Abstract
A new version of the QRadar® application framework is available for development teams. Application developers can review and complete the steps outlined in this article to migrate your apps to application framework version 2.
Content
Overview
In response to the end of life announcement for CentOS 6 and Python 2, a new version of the QRadar® App Framework is developed. App Framework v2 uses a new app base image derived from Red Hat Universal Base Image and code executes in Python 3 (v3.6.8 or later). More specifically, the v2 app base image is the RHEL8 ubi-minimal
image plus a number of add-ons including Python 3, OpenSSL, systemd, and SQLite.
If you want to run on both v1 and v2 of the framework, you need to make available two different versions of your app. This guide shows how to migrate an app from v1 to v2 of the App Framework, and describes some of the new features in v2.
Notice: For a video discussion on application framework v1 to v2 migration, see: https://www.securitylearningacademy.com/enrol/index.php?id=5656.
Migrating your application
To migrate an app from v1 to v2 of the App Framework, complete the following steps:
1. Update manifest.json
By default, applications can continue to install with the CentOS 6 base image, unless you specify otherwise in your app’s
manifest.json
file. A new manifest field named image
identifies the base image (qradar-app-base
) and version (2.0.0
or later) that you want to use, as demonstrated in this example:
{
"name": "Hello World",
"description": "Simple example app",
"version": "1.0",
"image": "qradar-app-base:2.0.0",
"uuid":"558d7935-f00b-42da-a278-c82abdb12b34",
"areas": [{
"id": "QHelloWorld",
"text": "Hello World",
"description": "Hello World area",
"url": "index",
"required_capabilities": ["SEM"]
}]
}
Important: Use the same uuid in your manifest as your CentOS version of the app.If your manifest includes a
dependencies
entry for specifying the location of RPMs and Python packages inside your app, you must remove it, as it is no longer supported. For more details on dependency location, see step 5 Update app layout.environment_variables
In App Framework v1, the environment variable
PATHSTART
could be used to alter the container PATH
setting so that Python 2.7 was used instead of Python 2.6. In v2, as all processing is done with Python 3, PATHSTART
is no longer required for this purpose and, if not needed for any other PATH
customization, can be removed.2. Check qpylib usage
The new app base image includes qpylib preinstalled as a Python package. If your app has a qpylib directory containing an older version of qpylib source code, remove the directory and use the version in the base image instead. The source code for qpylib is now maintained in an open source repository on GitHub: https://github.com/IBM/qpylib.
Implementation changes in qpylib:
- Due to the introduction of new features in App Framework SDK v2, most of the sdk/live abstraction is removed.
- Most functions are the equivalent of their pre-GitHub counterparts, although there are some minor changes to the parameters of these functions:
get_root_path
,get_store_path
,REST
. - The encryption module
encdec.py
updates.- Encryption algorithms are strengthened.
- Support for backwards-compatible decryption of old secrets was added.
- Error-handling is changed, and all errors are reported as EncryptionError.
- New module
ariel.py
adds support for performing Ariel searches.
3. Convert Python 2 code to Python 3
The v2 app base image supports Python 3 only, so any Python 2 code in your app must be converted to Python 3. There is a porting tool available called2to3
, which helps with this process. For more details, see https://docs.python.org/3/howto/pyporting.html.4. Check qjslib usage
qjslib now supports usage as a module or as a browser script. Like qpylib, the source code for qjslib is now maintained in an open source repository on GitHub: https://github.com/IBM/qjslib. Instructions on how to use qjslib in your app can be found there.5. Update app layout
The v1 src_deps
directory is replaced with a directory named container
. Its contents are:
container/pip
: replaces src_deps/pips and includes Python package dependenciescontainer/rpm
: replaces src_deps/rpms and includes RPM dependenciescontainer/build
: replaces src_deps/init for scripts that can be executed at image build timecontainer/run
: replaces src_deps/init for scripts that can only be executed at container start timecontainer/clean
: holds cleanup.sh script to be run when the app container is shut downcontainer/service
: holds scripts that start app background processescontainer/conf
: holds app configuration filescontainer/conf/supervisord.d
: the contents of this directory are copied to/etc/supervisord.d/
and included as a part of the supervisord configuration.
To enable fast container startup, best practice dictates that as far as possible, all dependencies must be installed and all startup scripts must be run when the app image is built. Do not use container/run
unless you have to.
Also, the location of pip and rpm dependencies can no longer be overridden via the app manifest. You must use container/pip
and container/rpm
.
6. Check for location-dependent code
In the app container, all app-related files and directories are now located under /opt/app-root
rather than /
.
The subdirectories of /opt/app-root
include:
app
: the source code for your appcontainer
: as described in section 5store
: the directory for persisting app data, particularly log filesbin
andstartup.d
: these directories hold container plumbing scripts
The manifest.json
file now lives at the top level of the deployed app (in /opt/app-root
) rather than being relocated inside the app
directory as it was in v1.
If you have any source code that directly accesses app files or directories using a hardcoded path location, you may need to update it. Python-based apps are abstracted from these changes if they use qpylib utility functions to access items within an app.
7. Update app dependencies
The v2 app base image has a different operating system and a different version of Python from v1, so any RPM dependencies must be el8
versions compatible with Red Hat 8, and any Python packages must be compatible with Python 3.6.8 or later.
RPM example: nginx-1.11.13-1.el6.ngx.x86_64.rpm
from CentOS 6 might be replaced by nginx-1.17.8-1.el8.ngx.x86_64.rpm
.
Note that the package manager in the v2 base image is microdnf
. The yum
package manager is not installed.
Also, bear in mind that the v2 base image contains updated versions of software such as Flask and supervisor. Your app will need to be compatible with the new versions.
8. Check permissions and privileges
For security reasons, processes within the app container no longer execute as root but instead use a non-privileged user account. supervisord runs Flask-based apps as this user, and any other processes should run as this user if possible. The same non-privileged user owns all app-related files and directories inside the container.
If you need to run something as root, use the as_root
script provided in /opt/app-root/bin
.
New features in QRadar App Framework v2
App Framework SDK v2
SDK v2 supports the new app base image and Python 3. These are some of the SDK’s new features:
- Using Docker, you can now test your app locally by building an app image and running an app container, effectively using the same environment as when an app is deployed to a QRadar server.
- Root privileges are no longer required to run the SDK v2 install script. The SDK is installed in a sub-directory of the user’s home directory, and has its own Python 3 virtual environment. Any existing Python installation on the user’s machine is left untouched, as is any installation of SDK v1.
- SDK v2 actions are invoked using the command
qapp
rather than the v1 entry pointqradar_app_creator
. - The combination of different install locations and different entry points means that SDK v1 and v2 can coexist. This allows developers to maintain an older CentOS version of an app while working on a newer Red Hat UBI version.
- The updated SDK example app demonstrates best practice for writing a Flask-based app for App Framework v2.
Container cleanup script
If your app has a container/clean/cleanup.sh
script, that script will be executed when the app container is stopped. The script is triggered when either a SIGINT or SIGTERM signal is received. Having a cleanup script allows graceful shutdown of things like database connections. Without a cleanup script, the container process is forcefully killed after 10 seconds.
Supervisord configuration
If you place .conf
files in your app’s container/conf/supervisord.d
directory, those files will be copied to /etc/supervisord.d
in the container, and will be picked up thanks to the inclusion of this entry in /etc/supervisord.conf
:
[include]
files = /etc/supervisord.d/*.conf
Note: only .conf
files will be copied; subdirectories will be ignored.
Named service access
For apps that don’t use Flask and instead use, for example, nginx, if you add nginx as a named service on port 5000 then it will be accessible via the standard app base url /console/plugins/<app id>/app_proxy
as well as the named service url /console/plugins/<app id>/app_proxy:<service name>
.
Was this topic helpful?
Document Information
Modified date:
14 May 2021
UID
ibm16356287