QRadar: How to validate downloads from IBM Fix Central are trusted and code signed

How To

Summary

The files that you download from IBM Fix Central for IBM Security QRadar product are digitally signed. Administrators can use these instructions to verify the integrity of these files to ensure that they originated from IBM and not modified by external sources.

Objective

All files posted to IBM Fix Central for QRadar include the software download, signature file, and SHA256 sum. Administrators can download these files to verify that software was created by IBM development. The code signing utility includes a certificate bundle and verify_signature tool that can be used to validate the downloaded file and public signature posted on IBM Fix Central.

The following download types include a file and signature on IBM Fix Central:

Update Package
Interim Fix
ISO
Scripts

Environment

Administrators can use any Linux system or the QRadar appliance to confirm that the software and signature file in the Fix Central download was created by IBM with the verify_signature tool. You must download the correct tool version based on the software you want to test. The code signing tool can run on any QRadar software version, but you must match the downloaded SFS or ISO to the correct tool version.

Code signing file	Software to test	Internal version in help (-h) output
Codesigning-1.0.1.tgz	QRadar 7.5.0 Update Package 6 and later	1.2
Codesigning-1.0.0.tgz	QRadar 7.5.0 Update Package 5 and earlier OR QRadar 7.4.x all fix pack versions	1.1

Note: To view the internal version of your verify_signature tool, use the -h option.

For example,

If the Console is installed with QRadar 7.4.3 and you download QRadar 7.5.0 UP6 IF2, you must use Codesigning-1.0.1.tgz to validate the software.
If the Console is installed with 7.5.0 Update Package 2 and you download QRadar 7.5.0 Update Package 5, you must use Codesigning-1.0.0.tgz to validate the software.
If Console is installed with 7.5.0 Update Package 6 and you download QRadar 7.5.0 Update Package 6 Interim Fix 2, you must use Codesigning-1.0.1.tgz to validate the software.

Steps

How to validate QRadar downloads from IBM Fix Central

Download the Codesigning tool from IBM Fix Central for your software version:
1. For QRadar 7.5.0 UP6 and later: https://ibm.biz/codesigning101.
2. For QRadar 7.5.0 UP5 and earlier or QRadar 7.4.x: https://ibm.biz/codesigning100.
Download QRadar software and signature file from IBM Fix Central.

Note: Software posted before September 2021 might not include a signature file. IBM's policy is to include signature files as new software is posted to IBM Fix Central.
Copy the files to the QRadar Console.
Create a directory for the code signing utility, such as /store/codesigning.
```
mkdir /store/codesigning
```
Copy both files to the /store/codesigning directory on the QRadar Console or any Linux host that has Internet access.
To extract the code signing script, type:
```
tar -zxvf codesigning-{version}.tgz
```
To ensure the file can run, type:
```
chmod +x verify_signature.sh
```

Type the following command:

sh verify_signature.sh -s <signatureFilename> -f <fileSigned> -c <certificateBundleFile>

For example,

sh verify_signature.sh -s 750-QRADAR-QRSIEM-2021.6.6.20230519190832.sfs.sig 
-f 750-QRADAR-QRSIEM-2021.6.6.20230519190832.sfs -c certificate_bundle.pem

Wait for an output from the verify_signature.sh utility.
Note: Validation might take several minutes to complete as the utility remotely confirms that the signature matches the software and supplied certificate_bundle.

Results
If successful, an 'OK: The file signature verification succeeded' message is displayed. If the validate check fails, you might have an expired certificate_bundle, and incorrect tool version, or a network communication issue. The following sections include example output messages for different success and error messages.

Example outputs for successful verification and error messages

OK: The file signature verification succeeded.

The following output is an example of a successful check that the signature provided with the software matches IBM's public key. A successful test confirms that the software tested was developed and distributed by IBM. Administrators who want to take the extra step to then confirm the SHA256 sum matches can do so when required by your corporate security policies.

Signature file 750-QRADAR-QRSIEM-2021.6.6.20230519190832.sfs.sig provided.
Signed file 750-QRADAR-QRSIEM-2021.6.6.20230519190832.sfs provided.
Certificate file certificate_bundle.pem provided.
certificate_bundle.pem: OK
Verified OK
OK: Signature verified, signed by IBM
<html>
<body>
        CRL/CACERT Repository
</body>
</html>OCSP Request Data:
    Version: 1 (0x0)
    Requestor List:
        Certificate ID:
          Hash Algorithm: sha1
          Issuer Name Hash: 915DEAC5D1E15E49646B8A94E04E470958C9BB89
          Issuer Key Hash: 6837E0EBB63BF85F1186FBFE617B088865F44E42
          Serial Number: 057E39DCE23C785C0CBD079C82E8CEEB
OCSP Response Data:
    OCSP Response Status: successful (0x0)
    Response Type: Basic OCSP Response
    Version: 1 (0x0)
    Responder Id: 6837E0EBB63BF85F1186FBFE617B088865F44E42
    Produced At: Jul 13 13:07:25 2023 GMT
    Responses:
    Certificate ID:
      Hash Algorithm: sha1
      Issuer Name Hash: 915DEAC5D1E15E49646B8A94E04E470958C9BB89
      Issuer Key Hash: 6837E0EBB63BF85F1186FBFE617B088865F44E42
      Serial Number: 057E39DCE23C785C0CBD079C82E8CEEB
    Cert Status: good
    This Update: Jul 13 12:51:02 2023 GMT
    Next Update: Jul 20 12:06:02 2023 GMT

    Signature Algorithm: sha384WithRSAEncryption
         d4:d1:6c:19:84:1d:81:8d:4f:b3:c4:98:1d:27:40:10:f3:74:
         1f:98:a1:38:57:42:e2:71:9d:ad:f2:93:a2:15:22:c5:07:7c:
         3a:bc:e9:27:14:32:b0:07:7f:bc:d6:f6:c5:6f:31:4a:f0:8a:
         bc:a5:ce:14:45:39:f1:53:42:78:1d:13:19:fc:e6:3c:f8:74:
         df:be:fb:39:03:45:e9:a4:ba:e6:e4:a7:12:40:69:ae:a5:7e:
         e6:c4:c0:a3:fc:a9:79:c5:6e:db:fe:74:eb:ec:06:10:9b:5c:
         e6:32:de:29:13:aa:6e:fc:d8:92:c8:dd:80:ff:1d:76:3c:bb:
         4f:94:01:fb:52:82:29:71:6b:d2:eb:be:6f:5b:eb:c9:fc:0e:
         89:27:33:ea:f8:85:b5:27:08:40:07:40:d9:88:98:19:4b:23:
         d3:de:6c:71:85:6e:fe:08:b9:d2:04:30:5a:5b:ac:06:c0:1a:
         cb:90:5c:22:92:ad:80:85:82:56:fd:9f:83:40:e2:9f:8a:96:
         2d:62:7a:cd:4f:89:7e:ac:8f:a0:cf:9e:77:ab:a9:9e:9a:04:
         e9:3b:7e:33:8f:c9:be:b5:ef:fd:cd:32:36:29:bb:99:04:56:
         23:50:f0:46:90:23:0a:7a:7d:a2:57:68:0f:99:22:e1:38:ab:
         88:f1:07:d1:47:86:21:b6:23:e0:10:65:bc:3a:b1:91:d3:9d:
         9d:d9:cd:a8:b7:31:49:b2:65:09:46:6d:63:7e:d7:89:d9:1a:
         b9:3e:24:57:8c:77:d6:2e:39:26:73:c6:04:ca:ad:74:43:b2:
         18:df:c2:62:81:75:17:98:b5:bd:8a:24:25:a9:7c:bc:a9:10:
         00:fe:8d:15:00:ae:4d:d6:4d:8a:d2:00:ac:c7:c0:46:d2:6a:
         a0:b4:6b:61:e2:69:89:2d:fc:d6:3a:e6:01:dd:4a:8a:09:63:
         6f:d5:3c:3b:7b:17:c9:73:73:44:21:2c:84:a4:ad:6a:28:fd:
         a3:02:15:97:c2:26:e2:a5:0f:b6:0b:61:7a:20:57:65:99:06:
         bc:6c:61:be:84:c1:97:1f:7d:c7:81:e1:97:6f:3e:34:b0:3e:
         d4:c0:ae:03:2c:05:2f:15:93:fa:5b:07:1b:56:9c:1a:31:a9:
         e3:e1:51:d8:0b:f3:e7:01:99:28:01:ae:6c:f7:f3:d6:3f:00:
         1a:92:b4:ff:80:c0:06:80:da:9c:cb:9f:86:fd:83:43:79:90:
         95:1c:0b:36:5f:71:41:26:71:96:bd:d1:c8:a5:4d:c7:89:36:
         30:fd:bc:1c:e6:87:ea:a3:88:12:22:30:b1:8a:2a:3c:b3:9f:
         83:91:0f:bd:47:49:bc:6c
Response verify OK
cert-00: good
        This Update: Jul 13 12:51:02 2023 GMT
        Next Update: Jul 20 12:06:02 2023 GMT
OK: The file signature verification succeeded.

ERROR: Signature verify failed, signed by an unknown entity
When the utility identifies that software is signed by an unknown entity, do not install the file on QRadar.

What to do
- Download the latest code signing utility from IBM Fix Central and run the verify_signature utility a second time.
- Confirm that the download was retrieved from a reputable source, such as IBM Fix Central.
- If you continue to experience errors, contact QRadar Support to report this issue.
  Note: QRadar Support can verify on a lab appliance whether the software posted to IBM Fix Central is valid. If you open a case, make sure you include the download link for the files you are attempted to validate from IBM Fix Central and your verify_signature tool version.
Example output:
```
Signature file 750-QRADAR-QRSIEM-2021.6.6.20230519190832.sfs.sig provided. 
Signed file 750-QRADAR-QRSIEM-2021.6.6.20230519190832.sfs provided.
Certificate file certificate_bundle.pem provided.
certificate_bundle.pem: OK
Verification Failure
ERROR: Signature verify failed, signed by an unknown entity
Contact customer support or refer to the signing document at https://ibm.biz/qradarcodesigning.
```
ERROR: Failed to verify the file signature
If the utility fails to verify the signature, it is likely the signature file is corrupted or the tool version you are using needs to be updated to the latest version. Administrators who experience the error message, "Failed to verify file signature" can download the signature file from IBM Fix Central a second time and confirm whether the error is reproducible.

What to do
- Confirm that the signature file is not zero bytes.
- Download the latest code signing file from IBM Fix Central and run the verify_signature utility a second time.
- If you continue to experience errors, contact QRadar Support.
  Note: QRadar Support can verify on a lab appliance if the signature is posted to IBM Fix Central is valid. If you open a case, make sure you include the download link for the files you are attempting to validate from IBM Fix Central.
Example output:
```
Signature file 750-QRADAR-QRSIEM-2021.6.6.20230519190832.sfs.sig provided. 
Signed file 750-QRADAR-QRSIEM-2021.6.6.20230519190832.sfs provided.
Certificate file certificate_bundle.pem provided.
certificate_bundle.pem: OK
Verification Failure
ERROR: Failed to verify the file signature.
Contact customer support or refer to the signing document at https://ibm.biz/qradarcodesigning.
```
ERROR: The IBM public certificate validation check failed
If the certificate and signature do not match, administrators can download the QRadar software, and signature from IBM Fix Central or ensure they have the latest version of the codesigning.tgz bundle from IBM Fix Central. Software that does not pass validation is not trusted.

The verify_signature utility checks the following conditions:
- The certificate bundle valid and decodes properly.
  If the certificate bundle fails multiple times, download the codesigning-{version}.tgz to confirm you have the latest tool on your QRadar appliance. Optionally, you can manually test the certificates with the included instructions in the Additional information section of this technical note.
- The signature file is valid and loads properly.
  If the signature file does not match, the software is untrusted. Download the latest codesigning bundle file from IBM Fix Central to confirm the signature is not corrupted. If you continue to experience errors, contact QRadar Support.
- The modulus matches the signature and the certificate bundle.
  If the modulus does not match, the software is untrusted. Administrators can download the software again from IBM Fix Central or manually confirm that the certificates are valid with the included instructions in the Additional information section of this technical note.
- Example error of a bad certificate bundle:
```
Signature file 743_QRadar_FixPack2_2020.11.2.20210810221124.sfs.sig provided.
Signed file 743_QRadar_FixPack2_2020.11.2.20210810221124.sfs provided.
Certificate file certificate_bundle.pem provided.
unable to load certificates
140215464757136:error:0906D064:PEM routines:PEM_read_bio:bad base64 decode:pem_lib.c:829:
ERROR: The certificate bundle provided is not a trusted file.
```

Connection refused

If you receive a connection refused error, you can verify you have access to http://oscp.digicert.com.

Example error message when the appliance cannot verify signatures due to a network issue:

Signature file 202161_QRadar_patchupdate-2021.6.1.20210910160507.sfs.sig provided.
Signed file 202161_QRadar_patchupdate-2021.6.1.20210910160507.sfs provided.
Certificate file certificate_bundle.pem provided.
certificate_bundle.pem: OK
Verified OK
OK: The modulus of the public key and certificate are identical.
curl: (7) Failed connect to ocsp.digicert.com:80; Connection refused
ERROR: Unable to validate the certificate due to failure to connect to http://ocsp.digicert.com. 
Please check your network policy.

To confirm access

Use SSH to log in to the QRadar Console as the root user.

To confirm the Console and connect to DigiCert, type:

curl -Is http://ocsp.digicert.com

Example success output:

HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 1540
Cache-Control: public, max-age=300
Content-Type: application/ocsp-response
Date: Fri, 17 Sep 2021 13:46:19 GMT
Etag: "5f46cfe9-5"
Last-Modified: Wed, 26 Aug 2020 21:11:05 GMT
Server: ECS (dcb/7F3C)
X-Cache: HIT
Content-Length: 5

Results
If successful, a HTTP/1.1 200 OK is returned in the command line. If you experience any errors, contact your corporate firewall team to add an exception to http://ocsp.digicert.com.

Additional Information

Manually validating files are code signed

Administrators who want to manually run commands to confirm the output of the code signing tool can run the following commands to double check an output or confirm that files are code signed by IBM.

Using openssl, validate the certificates are trusted from IBM Fix Central. For example,

openssl verify -x509_strict -untrusted certificate_bundle.pem certificate_bundle.pem

The output of the file confirms the certificates are OK. For example

# openssl verify -x509_strict -untrusted certificate_bundle.pem certificate_bundle.pem
certificate_bundle.pem: OK

To verify certificate was generated by IBM, type:

openssl x509 -inform pem -in certificate_bundle.pem  -noout -subject -issuer -startdate -enddate

Output

subject= /C=US/ST=New York/L=Armonk/O=International Business Machines Corporation/CN=International Business Machines Corporation
issuer= /C=US/O=DigiCert, Inc./CN=DigiCert Trusted G4 Code Signing RSA4096 SHA384 2021 CA1
notBefore=Dec 11 00:00:00 2022 GMT
notAfter=Dec  9 23:59:59 2024 GMT

To confirm the certificate revocation, type:

To split the certs out to individual file, type:

csplit -s -f cert- "certificate_bundle.pem" '/-----BEGIN CERTIFICATE-----/' {*} --elide-empty-files

Check for ocsp revocation, type:

openssl ocsp -no_nonce -issuer cert-01 -cert cert-00 -VAfile cert-01 -text -url http://ocsp.digicert.com -respout ocsptest

Example success output: Response verify OK

OCSP Request Data:
    Version: 1 (0x0)
    Requestor List:
        Certificate ID:
          Hash Algorithm: sha1
          Issuer Name Hash: 915DEAC5D1E15E49646B8A94E04E470958C9BB89
          Issuer Key Hash: 6837E0EBB63BF85F1186FBFE617B088865F44E42
          Serial Number: 057E39DCE23C785C0CBD079C82E8CEEB
OCSP Response Data:
    OCSP Response Status: successful (0x0)
    Response Type: Basic OCSP Response
    Version: 1 (0x0)
    Responder Id: 6837E0EBB63BF85F1186FBFE617B088865F44E42
    Produced At: Jul 18 13:07:24 2023 GMT
    Responses:
    Certificate ID:
      Hash Algorithm: sha1
      Issuer Name Hash: 915DEAC5D1E15E49646B8A94E04E470958C9BB89
      Issuer Key Hash: 6837E0EBB63BF85F1186FBFE617B088865F44E42
      Serial Number: 057E39DCE23C785C0CBD079C82E8CEEB
    Cert Status: good
    This Update: Jul 18 12:51:01 2023 GMT
    Next Update: Jul 25 12:06:01 2023 GMT

    Signature Algorithm: sha384WithRSAEncryption
         33:23:55:9a:61:84:cf:f9:81:cc:bc:1e:5b:a5:d7:13:ac:4e:
         c5:bf:2c:a4:b0:ed:53:97:bf:e7:40:8f:f1:03:9d:7c:d4:49:
         a3:95:ca:6f:b0:ab:0a:98:d1:d8:96:9f:45:ee:45:41:71:dd:
         b4:e5:cc:21:f5:e4:07:d8:70:e2:7f:24:12:ff:54:d9:03:49:
         2d:86:56:7e:b4:8a:a9:4c:d0:44:8e:94:47:b5:f3:81:2c:a5:
         b3:44:2c:ec:0d:1e:e3:da:8d:46:15:7e:82:44:22:3c:19:52:
         ac:7c:f6:7c:2d:40:57:aa:6f:ab:a0:51:c9:ae:cd:ac:b5:6a:
         80:c6:18:c6:5a:bb:0d:93:4c:7b:76:c1:7c:59:23:0e:21:60:
         15:8a:35:60:3d:df:be:05:ab:ee:5b:45:e4:1d:fd:a5:d6:ca:
         a5:f7:f8:14:8f:29:94:c9:b9:9f:34:85:59:eb:6e:12:f0:3c:
         12:54:76:11:b8:55:7c:4d:fa:67:c6:f1:bd:17:2d:c5:d1:70:
         5b:1f:ea:39:b4:7b:86:79:01:03:e6:7f:d5:cc:3a:6a:00:a6:
         56:ef:7c:4d:23:4d:25:cc:5d:b0:5d:ac:4c:48:41:57:b4:b6:
         99:4a:1c:a2:8b:c5:be:66:86:57:81:58:b6:e3:3b:1b:0b:be:
         32:0e:8c:33:65:7e:d0:16:98:5f:24:b3:a0:2c:53:59:f3:d5:
         54:83:8f:7e:b9:a0:e7:4a:77:d2:77:07:ac:f9:2f:de:79:c7:
         41:68:6e:05:3b:e3:62:05:4e:96:52:d4:22:d2:6b:63:da:a8:
         af:f5:dd:5b:0b:71:5d:11:b7:e6:6b:6b:c0:5e:df:6a:fd:24:
         79:a4:34:06:cd:b9:b9:55:75:26:c6:99:15:0f:1f:85:eb:be:
         46:91:dd:a6:58:ba:74:ed:93:21:73:fc:9e:90:2a:25:57:cd:
         a4:30:03:3c:d6:da:6b:41:b9:57:f9:78:b0:77:16:bf:3c:0e:
         fb:ac:96:2c:fb:68:3b:e9:64:89:2e:cc:4f:e7:84:a0:ea:ae:
         aa:e9:0d:fa:1d:17:14:02:b0:44:7d:22:de:c7:1e:5f:67:f0:
         04:15:a9:6e:47:3f:18:c2:d9:91:74:17:83:ce:d5:5e:0e:0c:
         96:83:ff:18:52:83:b9:c4:c0:f3:4b:bf:1e:24:2d:11:15:c6:
         69:a2:8d:15:48:0c:8b:1b:d6:f4:14:a6:80:f1:8d:2b:a5:75:
         fb:00:4f:d0:3d:50:7e:61:72:5f:c7:fb:55:45:f0:55:69:77:
         87:19:dd:3f:c1:49:9a:bd:98:6c:ee:51:c9:a9:19:58:39:0b:
         c5:a2:b4:da:01:87:ff:7e
Response verify OK
cert-00: good
        This Update: Jul 18 12:51:01 2023 GMT
        Next Update: Jul 25 12:06:01 2023 GMT

To create a public key from certificate, type:

openssl x509 -pubkey -noout -in certificate_bundle.pem > public.pem

Verify the software against the signature file from IBM Fix Central.
```
openssl dgst -sha256 -verify public.pem -signature <filename.sig> <software.sfs>
```
The output confirms that the software is valid.

Note: An error might display during the certificate verification process when you do not have the DigiCert CA. If you do not have a DigiCert CA, you can add the DigiCert root CA to your CA path, or you can add it to the verify command with the -CAfile option. The root cert needed is "DigiCert Assured ID Root CA", which can be obtained and verified from https://www.digicert.com/kb/digicert-root-certificates.htm.

About signature files

Code signed files allow administrators to confirm that software was compiled by IBM Development teams. As part of IBM's ongoing security procedures, all software posted to IBM Fix Central must be code signed. This policy allows organizations to confirm that the software was developed by IBM and not an outside threat actor by publishing the public certificate PEM files.

Confirming sha256 sums for a download

SHA256 sums confirm that the files match a known output, which can inform administrators whether a file is altered or downloaded incorrectly. The purpose of a check of the sha256 sum is to confirm the integrity of the software against the value in the IBM sha256 file. QRadar Support typically advises users to validate the SHA256 sum before you install any software to validate the integrity of the download.

Procedure

Download the QRadar Software from IBM Fix Central.
Download the sha256 sum file.
Copy the software and sha256 sum to the QRadar Console.
To compare the sha256 file to the sfs file, type:
```
echo "$(cat file.sha256) file.sfs" | sha256sum -c
```
For example,
```
echo "$(cat 750-QRADAR-QRSIEM-2021.6.6.20230519190832.sha256) 750-QRADAR-QRSIEM-2021.6.6.20230519190832.sfs" | sha256sum -c
```
A successful check displays a status of OK when the sums of the files match.
```
750-QRADAR-QRSIEM-2021.6.6.20230519190832.sfs: OK
```
Results
If you receive an error that the sums do not match, download the file again from IBM Fix Central or compare the files manually.

I cannot validate my files are code signed

Confirm you have the latest version of the code signing utility from IBM Fix Central. The public key required to validate files provided by IBM can expire. If the public key is expired, you can download the codesigning bundle from IBM Fix Central. As the public keys expire, the bundle is replaced on IBM Fix Central by the IBM Security team.

Troubleshooting connection refused issues to Digicert

If you receive a connection refused error, you can verify you have access to http://oscp.digicert.com from your QRadar Console.

Procedure

Use SSH to log in to the QRadar Console as the root user.

To confirm the Console and connect to DigiCert, type:

curl -Is http://ocsp.digicert.com

Example success output:

HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 1540
Cache-Control: public, max-age=300
Content-Type: application/ocsp-response
Date: Fri, 17 Sep 2021 13:46:19 GMT
Etag: "5f46cfe9-5"
Last-Modified: Wed, 26 Aug 2020 21:11:05 GMT
Server: ECS (dcb/7F3C)
X-Cache: HIT
Content-Length: 5

Results
If successful, a HTTP/1.1 200 OK is returned in the command line. If you experience any errors, contact your firewall team to add an exception to http://ocsp.digicert.com.

Document Location

Worldwide

[{"Type":"SW","Line of Business":{"code":"LOB24","label":"Security Software"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Product":{"code":"SSBQAC","label":"IBM Security QRadar SIEM"},"ARM Category":[{"code":"a8m0z000000cwsyAAA","label":"Admin Tasks"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Version(s)"}]

Tips