Repository files navigation

This repository contains:

• pytest plugin that provides support for testing the Firebird engine. It uses new Pythondriver for Firebird (firebird-driver).
• tests for Firebird engine (directorytests)
• files needed by tests (directoriesdatabases,files,backups andconfigs)

Requirements: Python 3.8+, Firebird 3+

You should definitelly read theQA suite documentation!

1. Clone the git repository

2. If you don't havepipx tool installed, install it using::

   python -m pip install pipx

   Or you can use any other method listed atpipx website.

   Don't forget to run:

   pipx ensurepath

   once after installation to ensure that tools installed viapipx will be available onsearch path.

3. Install the plugin and required dependencies by running next command:

   pipx install --include-deps firebird-qa

4. Adjust Firebird server configuration.

   ONLY FOR MANUAL runs:

   Check content of$FB_HOME/databases.conf.

   Ensure that RemoteAccess is allowed forsecurity.db.Also, it is recommended to set number of buffers not less than 256 for it:

   security.db = $(dir_secDb)/security<suffix>.fdb{    RemoteAccess = true    DefaultDbCachePages = 256}

   This must be done only if you want to run some tests manually.Automated scenario for running tests will overwrite this fileand put there all needed data before every pytest session (using$QA_ROOT/files/qa-databases.conf as prototype for that purpose).

   1. $FB_HOME/firebird.conf:

      Firebird 3:

      # RequiredBugcheckAbort = 1ExternalFileAccess = FullAuthServer = Srp, Win_Sspi, Legacy_AuthUserManager = Srp, Legacy_UserManagerWireCrypt = EnabledFileSystemCacheThreshold = 99999KIpcName = xnet_fb3x_qaRemotePipeName = wnet_fb3x_qa# RecommendedDefaultDbCachePages = 10000MaxUnflushedWrites = -1MaxUnflushedWriteTime = -1# Needed for encryption-related tests.KeyHolderPlugin = fbSampleKeyHolder

      Firebird 4:

      # RequiredBugcheckAbort = 1ExternalFileAccess = FullAuthServer = Srp, Win_Sspi, Legacy_AuthUserManager = Srp, Legacy_UserManagerReadConsistency = 0WireCrypt = EnabledExtConnPoolSize = 10ExtConnPoolLifeTime = 10UseFileSystemCache = trueIpcName = xnet_fb4x_qaRemotePipeName = wnet_fb4x_qa# RecommendedDefaultDbCachePages = 10000MaxUnflushedWrites = -1MaxUnflushedWriteTime = -1# number of seconds after which statement execution will be automatically cancelled# by the engine# can be very useful if some test will hang or become work extremely slow:StatementTimeout = 7200# Needed for encryption-related tests:KeyHolderPlugin = fbSampleKeyHolder

      Firebird 5:

      currently all parameters from FB-4.x can be used, except 'RemotePipeName'because support of WNET protocol was removed from FB-5.x.It is recommended to assign value like 'xnet_fb5x_qa' to IpcName.

      [!NOTE]Proper values of some parameters strongly depends on ServerMode and amount ofavaliable RAM.

      • DefaultDbCachePages::On Classic/SuperClassic it must not be greater than 4K in real systems. For tests10K...20K is OK.On Super it can be increased so that size of page cache become 25%...33% ofphysical RAM.
      • TempCacheLimit::On Classic usually it must be limited because every connection uses its own memory areafor sort purposes. Nowadays may to use values about 256M ... 512M.On SuperClassic and Super common memory area is used to sorts, so this parameter can havevalues of dozen Gb. As first approximation, it can be set up to 33% of total RAM.

   2. Changes in OS environment variables:

      it is recommended to create variableFIREBIRD_TMP that will point to the pre-created directoryon some fast drive (e.g. SSD or RAM). This drive must have at least 30 Gb free space.Once this variable is defined, one may not specify parameter 'TempDirectories'.

   3. Required changes for running encryption-related tests::

      1. Change dir to$FB_HOME/examples/prebuilt/plugins/ and make copy of following files into$FB_HOME/plugins/ :

         1. Configs:

            fbSampleKeyHolder.conffbSampleDbCrypt.conf

         2. Binaries on Windows:

            fbSampleDbCrypt.dllfbSampleKeyHolder.dll

         3. Binaries on Linux:

            libfbSampleDbCrypt.solibfbSampleKeyHolder.so

            [!NOTE]These files missed in Firebird 3.x but one may to use such files from anyrecent FB 4.x snapshot.Config parameter KeyHolderPlugin currently is 'fbSampleKeyHolder'.This value must match to value of parameter 'ENCRYPTION_HOLDER' that isspecified in the file$(QA_ROOT)/files/test_config.ini (it contains several settings that are common for many tests).

      2. In$FB_HOME/plugins/fbSampleKeyHolder.conf:

         Auto = trueKeyRed=111KeyGreen = 119

      3. In$FB_HOME/plugins/fbSampleDbCrypt.conf:

         # Encure that Auto = false or just is commented out

      4. Restart Firebird and check that all set correct. Example for Linux:

         shell rm -f /var/tmp/tmp4test.fdb;create database 'localhost:/var/tmp/tmp4test.fdb' user sysdba password 'masterkey';

         Following must PASS:

         set echo on;set bail on;alter database encrypt with "fbSampleDbCrypt" key Red;shell sleep 2;alter database decrypt;shell sleep 2;alter database encrypt with "fbSampleDbCrypt" key Green;shell sleep 2;alter database decrypt;shell sleep 2;set echo off;set bail off;

         Following must FAIL with:

         -- Statement failed, SQLSTATE = 42000-- unsuccessful metadata update-- -ALTER DATABASE failed-- -Missing correct crypt key-- -Plugin fbSampleKeyHolder:-- -Crypt key NOSUCH not setset echo on;alter database encrypt with "fbSampleDbCrypt" key NoSuch;shell sleep 2;show database;quit;

      [!IMPORTANT]Ensure that EMPLOYEE database was not encrypted before with key/value that is unknown currently!Otherwise attempt to run ANY test will fail with:

        INTERNALERROR> firebird.driver.types.DatabaseError: Missing database encryption key for your attachment  INTERNALERROR> -Plugin fbSampleKeyHolder:  INTERNALERROR> -Crypt key <HERE_SOME_UNKNOWN_KEY> not set

   4. Additional issues about folder$(dir_sampleDb) ($FB_HOME/examples/empbuild/) and its subdirectories.

      1. There are many tests which supposes that this directory is avaliable for read/write access.Test suite (firebird-qa plugin for pytest) will re-create subdirectory with name 'qa' under$(dir_sample) forevery such test, so be sure that you have not any significant data in this folder.
      2. Firebird 4.x+ has ability to involve databases in replication schema. There are several tests which assumes thatsuch schema already was created (before pytest session) and there arte two databases in it (master and replica).It was decided to use directory with name:$(dir_sampleDb)/qa_replication/ for this purpoces. Two databases mustbe created in it:db_main.fdb anddb_repl.fdb, and one need to prepare them into replication beforehand.Also, one need to prepare two directories in THIS folder which will serve as replication journal and archive.All these actions are performed by batch scenarios which can be provided by IBSurgeon company by request.

5. Optional. Enable your OS to create dump files in case of FB crashes caused by tests::

   1. Windows:
      1. ParameterBugcheckAbort must always be set to 1, otherwise dumps will not be created.
      2. Run regedit, navigate to key:

         HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps\

         Create sub-key there with name: 'firebird.exe' (without single quotes).Add following parameters in the 'firebird.exe' key:

         DumpCount, type = DWORD, value: not less than 5;DumpFoler, type = REG_EXPAND_SZ, value = directory where you want dumps to be created;DumpType, type = DWORD, value = 2

      3. Following setting must present in the registry to disable any pop-up window when program crashes:key:HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Windows Error Reporting\parameter:'DontShowUI', type = DWORD, value: 2

   4.2. Linux:File/etc/security/limits.conf must have setting:* soft core unlimitedFile/etc/sysctl.conf must have parameterkernel.core_pattern that specifiesdirectory to store dumps and pattern for dumps name, e.g.:kernel.core_pattern=/var/tmp/core.%e.%t.%p

6. Cautions.

   1. Problems can occur on Windows if we launch two FB instances which uses the same major version ODS.Currently this relates to FB-4.x and FB-5.x: each of them tries to create file 'fb13_user_mapping'in%programdata%\firebird. This leads to conflict and attempt to connect to any DB using latter FB instanceissues "Error occurred during login, please check server firebird.log for details" and firebird.log will have:"Database is probably already opened by another engine instance in another Windows session".BE SURE THAT YOU DID NOT LAUNCH ANOTHER FIREBIRD INSTANCE THAT USES SAME ODS AS CURRENTLY TESTING.
   2. Be sure that directory specified by FIREBIRD_TMP variable actually exists and is accessible for 'firebird' account.
   3. Ensure that yourfirebird-driver.conf contains 'DEFAULT' section withencoding_errors = ignore.Otherwise outcome of some tests can be unpredictable if your OS has non-ascii system console

7. Use pytest to run tests.

   The plugin adds next options to pytest:

        Firebird server:         --server=SERVER       Server configuration name         --bin-dir=PATH        Path to directory with Firebird utilities         --protocol={xnet,inet,inet4,wnet}                               Network protocol used for database attachments         --runslow             Run slow tests         --save-output         Save test std[out|err] output to files         --skip-deselected={platform,version,any}                               SKIP tests instead deselection         --extend-xml          Extend XML JUnit report with additional information         --install-terminal    Use our own terminal reporter

   To run all tests (except slow ones) against local server use next command::

   pytest --server local ./tests

   [!NOTE]If plugin fails to determine the directory with Firebird utilities (isql, gbak etc.),use--bin-dir option to specify it.