The main test system in the MariaDB open source database project is the mariadb-test-run script (inherited from mysql-test-run). It is easy to run to and does not require you to compile any source code.

While writing MTR tests is relevant only for MariaDB developers, knowing how to run MTR is useful for any database administrator running MariaDB, as it is a quick way to validate that MariaDB can run correctly on your hardware and operating system version.

TL;DR for Debian/Ubuntu users

Run the full 6000+ test suite as current user in temporary directory with multiple workers in parallel and with detailed logging on failures, typically taking over 30 minutes on modern laptop:

Copy

apt install -y mariadb-test mariadb-backup mariadb-plugin-* patch gdb cd /usr/share/mysql/mysql-test export MTR_PRINT_CORE=detailed ./mtr --force --parallel=auto --vardir=$(mktemp -d) \ --skip-test-list=unstable-tests.amd64 --big-test

apt install -y mariadb-test mariadb-backup mariadb-plugin-* patch gdb cd /usr/share/mysql/mysql-test export MTR_PRINT_CORE=detailed ./mtr --force --parallel=auto --vardir=$(mktemp -d) \ --skip-test-list=unstable-tests.amd64 --big-test

For full details, read the whole article.

To avoid polluting your actual system with new packages, it is convenient to run MTR in a throwaway container. Start one with some RAM-memory-backed disk allocated with --shm-size=1G so running MTR with --mem later on is possible:

shell Copy

docker run --interactive --tty --rm --shm-size=1G debian:sid bash

docker run --interactive --tty --rm --shm-size=1G debian:sid bash

This example uses Docker, but the principle is the same with any Linux container tool, such as Podman.

Next, install the MariaDB test suite package. This will also pull in the MariaDB server and all the necessary dependencies. Additionally, also install the GNU Debugger (gdb) for automatic stack traces and patch.

shell Copy

apt update apt install -y mariadb-test mariadb-backup mariadb-plugin-* patch gdb

apt update apt install -y mariadb-test mariadb-backup mariadb-plugin-* patch gdb

The mariadb-test-run is not intended to be run as root and will skip some tests if run as root. Therefore, create a new user inside the container and grant it permissions to the test directory:

shell Copy

adduser --disabled-password mariadb-test-runner chown -R mariadb-test-runner /usr/share/mysql/mysql-test

adduser --disabled-password mariadb-test-runner chown -R mariadb-test-runner /usr/share/mysql/mysql-test

At minimum, the test runner user needs to be able to write to the path /usr/share/mysql/mysql-test/var (unless some other path is defined with --vardir and --tmpdir), but some tests also run patch to modify the test files on-the-fly, so just grant permissions to the whole test directory. As this is a throwaway container anyway, there is no need to be prudent with the permissions.

Next, switch to the test user and test directory and start the run with default settings:

Copy

$ su - mariadb-test-runner $ cd /usr/share/mysql/mysql-test $ ./mariadb-test-run Logging: ./mariadb-test-run VS config: vardir: /usr/share/mysql/mysql-test/var Creating var directory '/usr/share/mysql/mysql-test/var'... Checking supported features... MariaDB Version 10.11.2-MariaDB-1 - SSL connections supported - binaries built with wsrep patch Using suites: main-,archive-,atomic-,binlog-,binlog_encryption-,client-,csv-,compat/oracle-,compat/mssql-,compat/maxdb-,encryption-,federated-,funcs_1-,funcs_2-,gcol-,handler-,heap-,innodb-,innodb_fts-,innodb_gis-,innodb_i_s-,innodb_zip-,json-,maria-,mariabackup-,multi_source-,optimizer_unfixed_bugs-,parts-,perfschema-,plugins-,roles-,rpl-,stress-,sys_vars-,sql_sequence-,unit-,vcol-,versioning-,period-,sysschema-,disks,func_test,metadata_lock_info,query_response_time,sequence,sql_discovery,type_inet,type_uuid,user_variables Collecting tests... ... main.subselect_innodb 'innodb' w4 [ pass ] 3359 main.subselect_sj2 'innodb' w1 [ pass ] 2737 main.subselect_sj2_jcl6 'innodb' w3 [ pass ] 2933 main.parser_bug21114_innodb 'innodb' w8 [ pass ] 15364 innodb_gis.rtree_search 'innodb' w5 [ pass ] 52379 -------------------------------------------------------------------------- The servers were restarted 1736 times Spent 8527.863 of 1659 seconds executing testcases Completed: All 5131 tests were successful. 995 tests were skipped, 280 by the test itself.

$ su - mariadb-test-runner $ cd /usr/share/mysql/mysql-test $ ./mariadb-test-run Logging: ./mariadb-test-run VS config: vardir: /usr/share/mysql/mysql-test/var Creating var directory '/usr/share/mysql/mysql-test/var'... Checking supported features... MariaDB Version 10.11.2-MariaDB-1 - SSL connections supported - binaries built with wsrep patch Using suites: main-,archive-,atomic-,binlog-,binlog_encryption-,client-,csv-,compat/oracle-,compat/mssql-,compat/maxdb-,encryption-,federated-,funcs_1-,funcs_2-,gcol-,handler-,heap-,innodb-,innodb_fts-,innodb_gis-,innodb_i_s-,innodb_zip-,json-,maria-,mariabackup-,multi_source-,optimizer_unfixed_bugs-,parts-,perfschema-,plugins-,roles-,rpl-,stress-,sys_vars-,sql_sequence-,unit-,vcol-,versioning-,period-,sysschema-,disks,func_test,metadata_lock_info,query_response_time,sequence,sql_discovery,type_inet,type_uuid,user_variables Collecting tests... ... main.subselect_innodb 'innodb' w4 [ pass ] 3359 main.subselect_sj2 'innodb' w1 [ pass ] 2737 main.subselect_sj2_jcl6 'innodb' w3 [ pass ] 2933 main.parser_bug21114_innodb 'innodb' w8 [ pass ] 15364 innodb_gis.rtree_search 'innodb' w5 [ pass ] 52379 -------------------------------------------------------------------------- The servers were restarted 1736 times Spent 8527.863 of 1659 seconds executing testcases Completed: All 5131 tests were successful. 995 tests were skipped, 280 by the test itself.

If no test suite is selected, MTR will run about 6000 tests, which on my laptop takes about 30 minutes. If MTR is started with additional --big-test parameter, it will run additional tests that are resource intensive and consume, for example, a lot of RAM memory and take a long time to run, totalling to a test run that has 6100 tests and takes 43 minutes to complete (on my laptop). To only run big tests, use --big --big.

If there is a need to limit the scope, such as in build systems that want to validate that the built binary works without running all tests, typically --suite=main --skip-rpl is used. This results in about 1000 tests being run, which on my laptop takes about 3½ minutes.

Even when running without any limitations on what tests are run, many tests have code that makes them opt out automatically based on some condition being missing, and on my laptop about 1000 tests end up being skipped. Some examples:

Copy

main.connect2 [ skipped ] Requires debug build main.mysql_client_test_comp [ skipped ] No IPv6 main.connect-abstract [ skipped ] Need Linux main.grant_cache_ps_prot [ skipped ] Need ps-protocol main.fix_priv_tables [ skipped ] Test need MYSQL_FIX_PRIVILEGE_TABLES main.no-threads [ skipped ] Test requires: 'one_thread_per_connection' main.udf_skip_grants [ skipped ] Need udf example main.lowercase_mixed_tmpdir_innodb [ skipped ] Test requires: 'lowercase2' main.innodb_load_xa [ skipped ] Need InnoDB plugin mariabackup.alter_copy_excluded [ skipped ] No mariabackup binlog.binlog_expire_warnings [ skipped ] Test needs --big-test encryption.innodb-spatial-index [ skipped ] requires patch executable plugins.pam_cleartext [ skipped ] Not run as user owning auth_pam_tool_dir rpl.rpl_gtid_mdev4474 'innodb,row' [ skipped ] Neither MIXED nor STATEMENT binlog format

main.connect2 [ skipped ] Requires debug build main.mysql_client_test_comp [ skipped ] No IPv6 main.connect-abstract [ skipped ] Need Linux main.grant_cache_ps_prot [ skipped ] Need ps-protocol main.fix_priv_tables [ skipped ] Test need MYSQL_FIX_PRIVILEGE_TABLES main.no-threads [ skipped ] Test requires: 'one_thread_per_connection' main.udf_skip_grants [ skipped ] Need udf example main.lowercase_mixed_tmpdir_innodb [ skipped ] Test requires: 'lowercase2' main.innodb_load_xa [ skipped ] Need InnoDB plugin mariabackup.alter_copy_excluded [ skipped ] No mariabackup binlog.binlog_expire_warnings [ skipped ] Test needs --big-test encryption.innodb-spatial-index [ skipped ] requires patch executable plugins.pam_cleartext [ skipped ] Not run as user owning auth_pam_tool_dir rpl.rpl_gtid_mdev4474 'innodb,row' [ skipped ] Neither MIXED nor STATEMENT binlog format

If you are interested in one particular test, just give it as the last argument:

Copy

$ ./mariadb-test-run main.connect Logging: ./mariadb-test-run main.connect Creating var directory '/usr/share/mysql/mysql-test/var'... Checking supported features... MariaDB Version 10.11.2-MariaDB-1 - SSL connections supported - binaries built with wsrep patch Collecting tests... Installing system database... ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- main.connect [ pass ] 14206 -------------------------------------------------------------------------- The servers were restarted 0 times Spent 14.206 of 20 seconds executing testcases Completed: All 1 tests were successful.

$ ./mariadb-test-run main.connect Logging: ./mariadb-test-run main.connect Creating var directory '/usr/share/mysql/mysql-test/var'... Checking supported features... MariaDB Version 10.11.2-MariaDB-1 - SSL connections supported - binaries built with wsrep patch Collecting tests... Installing system database... ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- main.connect [ pass ] 14206 -------------------------------------------------------------------------- The servers were restarted 0 times Spent 14.206 of 20 seconds executing testcases Completed: All 1 tests were successful.

There are three parameters can greatly improve how quickly MTR runs. The primary one is --parallel=auto, which will run MTR in parallel with as many workers as there are CPUs (by default MTR runs with just one worker). On my laptop, going from one MTR worker to 8 in parallel reduced the total run time for the main suite from 17 to about 3 minutes.

Another parameter is --fast, which will make the MTR kill all server processes violently without waiting for them to gracefully shutdown. The test run restarts the MariaDB server hundreds of times, so saving half a second on every shutdown results in the main suite completing 20 seconds faster.

The parameter --mem instructs MTR to make the directory var/ a symbolic link to a subdirectory on the shared memory device (/dev/shm). This works if the container is started with --shm-size and has at least 350MB of space on the ramdisk. On my laptop, this further reduced the main test suite duration down to 2½ minutes.

When running a single test, one might use --verbose as an additional argument to see the commands that run in the test. It is also possible to define --verbose --verbose twice, but that makes the test run so verbose that is it unusable.

When running a suite of tests, you only want to have extra output visible if the test fails. If the server fails to start and a particular test doesn’t run at all, using --verbose-restart might be beneficial. If running the test in an automated system, one might want to save results in a JUnit-compatible XML file that can, for example, be rendered by GitLab CI.

This is the command that several CI systems run MTR with:

shell Copy

export MTR_PRINT_CORE=detailed eatmydata perl -I. ./mariadb-test-run \ --force --testcase-timeout=120 --suite-timeout=540 --retry=3 \ --verbose-restart --max-save-core=1 --max-save-datadir=1 \ --parallel=auto --skip-rpl --suite=main \ --xml-report=mariadb-test-run-junit.xml

export MTR_PRINT_CORE=detailed eatmydata perl -I. ./mariadb-test-run \ --force --testcase-timeout=120 --suite-timeout=540 --retry=3 \ --verbose-restart --max-save-core=1 --max-save-datadir=1 \ --parallel=auto --skip-rpl --suite=main \ --xml-report=mariadb-test-run-junit.xml

The command above also showcases use of eatmydata, which makes fsync and similar system calls to skip memory-to-disk guarantees, but in my testing with MTR, it didn’t affect speed.

The above summarizes everything one typically needs to know for running the mariadb-test-run.

For a DBA, there are also several other tools that ship with MariaDB, which can help with testing and validating ones own environment, such as mariadb-stress-test and mariadb-slap.

If you want to write a new test or fix an existing test, there are many more additional parameters to learn, such as --record and --gcov. The official contribution docs at mariadb.org list some of the most useful MTR parameters. The mariadb.com knowledge base article lists them all. As with all commands in Linux, there is also the main page for mariadb-test-run.

The structure of the tests is actually quite easy, and requires no C/C++ skills to write. Each test file (suffix .test) consists mainly of SQL code which is executed by mariadb-test-run (MTR), with the output compared (with diff) to the corresponding file with the expected output in text format (suffix .result).

In my development workflow, writing a test and running MTR might look something like this:

If you want to contribute to the MariaDB open source project, extending the test coverage is a great place to start. To scratch your own itch, think about a MariaDB bug you encountered while using it, and think if that can be reproduced as a test and submitted upstream so that it becomes part of the body of tests and thus easy to catch if it ever regresses again.

To learn more about writing mariadb-test-run tests, read the test case authoring guide at mariadb.org.

The main test system in the MariaDB open source database project is the mariadb-test-run script (inherited from mysql-test-run). It is easy to run to and does not require you to compile any source code.

While writing MTR tests is relevant only for MariaDB developers, knowing how to run MTR is useful for any database administrator running MariaDB, as it is a quick way to validate that MariaDB can run correctly on your hardware and operating system version.

TL;DR for Debian/Ubuntu users

Run the full 6000+ test suite as current user in temporary directory with multiple workers in parallel and with detailed logging on failures, typically taking over 30 minutes on modern laptop:

Copy

apt install -y mariadb-test mariadb-backup mariadb-plugin-* patch gdb cd /usr/share/mysql/mysql-test export MTR_PRINT_CORE=detailed ./mtr --force --parallel=auto --vardir=$(mktemp -d) \ --skip-test-list=unstable-tests.amd64 --big-test

apt install -y mariadb-test mariadb-backup mariadb-plugin-* patch gdb cd /usr/share/mysql/mysql-test export MTR_PRINT_CORE=detailed ./mtr --force --parallel=auto --vardir=$(mktemp -d) \ --skip-test-list=unstable-tests.amd64 --big-test

For full details, read the whole article.

To avoid polluting your actual system with new packages, it is convenient to run MTR in a throwaway container. Start one with some RAM-memory-backed disk allocated with --shm-size=1G so running MTR with --mem later on is possible:

shell Copy

docker run --interactive --tty --rm --shm-size=1G debian:sid bash

docker run --interactive --tty --rm --shm-size=1G debian:sid bash

This example uses Docker, but the principle is the same with any Linux container tool, such as Podman.

Next, install the MariaDB test suite package. This will also pull in the MariaDB server and all the necessary dependencies. Additionally, also install the GNU Debugger (gdb) for automatic stack traces and patch.

shell Copy

apt update apt install -y mariadb-test mariadb-backup mariadb-plugin-* patch gdb

apt update apt install -y mariadb-test mariadb-backup mariadb-plugin-* patch gdb

The mariadb-test-run is not intended to be run as root and will skip some tests if run as root. Therefore, create a new user inside the container and grant it permissions to the test directory:

shell Copy

adduser --disabled-password mariadb-test-runner chown -R mariadb-test-runner /usr/share/mysql/mysql-test

adduser --disabled-password mariadb-test-runner chown -R mariadb-test-runner /usr/share/mysql/mysql-test

At minimum, the test runner user needs to be able to write to the path /usr/share/mysql/mysql-test/var (unless some other path is defined with --vardir and --tmpdir), but some tests also run patch to modify the test files on-the-fly, so just grant permissions to the whole test directory. As this is a throwaway container anyway, there is no need to be prudent with the permissions.

Next, switch to the test user and test directory and start the run with default settings:

Copy

$ su - mariadb-test-runner $ cd /usr/share/mysql/mysql-test $ ./mariadb-test-run Logging: ./mariadb-test-run VS config: vardir: /usr/share/mysql/mysql-test/var Creating var directory '/usr/share/mysql/mysql-test/var'... Checking supported features... MariaDB Version 10.11.2-MariaDB-1 - SSL connections supported - binaries built with wsrep patch Using suites: main-,archive-,atomic-,binlog-,binlog_encryption-,client-,csv-,compat/oracle-,compat/mssql-,compat/maxdb-,encryption-,federated-,funcs_1-,funcs_2-,gcol-,handler-,heap-,innodb-,innodb_fts-,innodb_gis-,innodb_i_s-,innodb_zip-,json-,maria-,mariabackup-,multi_source-,optimizer_unfixed_bugs-,parts-,perfschema-,plugins-,roles-,rpl-,stress-,sys_vars-,sql_sequence-,unit-,vcol-,versioning-,period-,sysschema-,disks,func_test,metadata_lock_info,query_response_time,sequence,sql_discovery,type_inet,type_uuid,user_variables Collecting tests... ... main.subselect_innodb 'innodb' w4 [ pass ] 3359 main.subselect_sj2 'innodb' w1 [ pass ] 2737 main.subselect_sj2_jcl6 'innodb' w3 [ pass ] 2933 main.parser_bug21114_innodb 'innodb' w8 [ pass ] 15364 innodb_gis.rtree_search 'innodb' w5 [ pass ] 52379 -------------------------------------------------------------------------- The servers were restarted 1736 times Spent 8527.863 of 1659 seconds executing testcases Completed: All 5131 tests were successful. 995 tests were skipped, 280 by the test itself.

$ su - mariadb-test-runner $ cd /usr/share/mysql/mysql-test $ ./mariadb-test-run Logging: ./mariadb-test-run VS config: vardir: /usr/share/mysql/mysql-test/var Creating var directory '/usr/share/mysql/mysql-test/var'... Checking supported features... MariaDB Version 10.11.2-MariaDB-1 - SSL connections supported - binaries built with wsrep patch Using suites: main-,archive-,atomic-,binlog-,binlog_encryption-,client-,csv-,compat/oracle-,compat/mssql-,compat/maxdb-,encryption-,federated-,funcs_1-,funcs_2-,gcol-,handler-,heap-,innodb-,innodb_fts-,innodb_gis-,innodb_i_s-,innodb_zip-,json-,maria-,mariabackup-,multi_source-,optimizer_unfixed_bugs-,parts-,perfschema-,plugins-,roles-,rpl-,stress-,sys_vars-,sql_sequence-,unit-,vcol-,versioning-,period-,sysschema-,disks,func_test,metadata_lock_info,query_response_time,sequence,sql_discovery,type_inet,type_uuid,user_variables Collecting tests... ... main.subselect_innodb 'innodb' w4 [ pass ] 3359 main.subselect_sj2 'innodb' w1 [ pass ] 2737 main.subselect_sj2_jcl6 'innodb' w3 [ pass ] 2933 main.parser_bug21114_innodb 'innodb' w8 [ pass ] 15364 innodb_gis.rtree_search 'innodb' w5 [ pass ] 52379 -------------------------------------------------------------------------- The servers were restarted 1736 times Spent 8527.863 of 1659 seconds executing testcases Completed: All 5131 tests were successful. 995 tests were skipped, 280 by the test itself.

If no test suite is selected, MTR will run about 6000 tests, which on my laptop takes about 30 minutes. If MTR is started with additional --big-test parameter, it will run additional tests that are resource intensive and consume, for example, a lot of RAM memory and take a long time to run, totalling to a test run that has 6100 tests and takes 43 minutes to complete (on my laptop). To only run big tests, use --big --big.

If there is a need to limit the scope, such as in build systems that want to validate that the built binary works without running all tests, typically --suite=main --skip-rpl is used. This results in about 1000 tests being run, which on my laptop takes about 3½ minutes.

Even when running without any limitations on what tests are run, many tests have code that makes them opt out automatically based on some condition being missing, and on my laptop about 1000 tests end up being skipped. Some examples:

Copy

main.connect2 [ skipped ] Requires debug build main.mysql_client_test_comp [ skipped ] No IPv6 main.connect-abstract [ skipped ] Need Linux main.grant_cache_ps_prot [ skipped ] Need ps-protocol main.fix_priv_tables [ skipped ] Test need MYSQL_FIX_PRIVILEGE_TABLES main.no-threads [ skipped ] Test requires: 'one_thread_per_connection' main.udf_skip_grants [ skipped ] Need udf example main.lowercase_mixed_tmpdir_innodb [ skipped ] Test requires: 'lowercase2' main.innodb_load_xa [ skipped ] Need InnoDB plugin mariabackup.alter_copy_excluded [ skipped ] No mariabackup binlog.binlog_expire_warnings [ skipped ] Test needs --big-test encryption.innodb-spatial-index [ skipped ] requires patch executable plugins.pam_cleartext [ skipped ] Not run as user owning auth_pam_tool_dir rpl.rpl_gtid_mdev4474 'innodb,row' [ skipped ] Neither MIXED nor STATEMENT binlog format

main.connect2 [ skipped ] Requires debug build main.mysql_client_test_comp [ skipped ] No IPv6 main.connect-abstract [ skipped ] Need Linux main.grant_cache_ps_prot [ skipped ] Need ps-protocol main.fix_priv_tables [ skipped ] Test need MYSQL_FIX_PRIVILEGE_TABLES main.no-threads [ skipped ] Test requires: 'one_thread_per_connection' main.udf_skip_grants [ skipped ] Need udf example main.lowercase_mixed_tmpdir_innodb [ skipped ] Test requires: 'lowercase2' main.innodb_load_xa [ skipped ] Need InnoDB plugin mariabackup.alter_copy_excluded [ skipped ] No mariabackup binlog.binlog_expire_warnings [ skipped ] Test needs --big-test encryption.innodb-spatial-index [ skipped ] requires patch executable plugins.pam_cleartext [ skipped ] Not run as user owning auth_pam_tool_dir rpl.rpl_gtid_mdev4474 'innodb,row' [ skipped ] Neither MIXED nor STATEMENT binlog format

If you are interested in one particular test, just give it as the last argument:

Copy

$ ./mariadb-test-run main.connect Logging: ./mariadb-test-run main.connect Creating var directory '/usr/share/mysql/mysql-test/var'... Checking supported features... MariaDB Version 10.11.2-MariaDB-1 - SSL connections supported - binaries built with wsrep patch Collecting tests... Installing system database... ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- main.connect [ pass ] 14206 -------------------------------------------------------------------------- The servers were restarted 0 times Spent 14.206 of 20 seconds executing testcases Completed: All 1 tests were successful.

$ ./mariadb-test-run main.connect Logging: ./mariadb-test-run main.connect Creating var directory '/usr/share/mysql/mysql-test/var'... Checking supported features... MariaDB Version 10.11.2-MariaDB-1 - SSL connections supported - binaries built with wsrep patch Collecting tests... Installing system database... ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- main.connect [ pass ] 14206 -------------------------------------------------------------------------- The servers were restarted 0 times Spent 14.206 of 20 seconds executing testcases Completed: All 1 tests were successful.

There are three parameters can greatly improve how quickly MTR runs. The primary one is --parallel=auto, which will run MTR in parallel with as many workers as there are CPUs (by default MTR runs with just one worker). On my laptop, going from one MTR worker to 8 in parallel reduced the total run time for the main suite from 17 to about 3 minutes.

Another parameter is --fast, which will make the MTR kill all server processes violently without waiting for them to gracefully shutdown. The test run restarts the MariaDB server hundreds of times, so saving half a second on every shutdown results in the main suite completing 20 seconds faster.

The parameter --mem instructs MTR to make the directory var/ a symbolic link to a subdirectory on the shared memory device (/dev/shm). This works if the container is started with --shm-size and has at least 350MB of space on the ramdisk. On my laptop, this further reduced the main test suite duration down to 2½ minutes.

When running a single test, one might use --verbose as an additional argument to see the commands that run in the test. It is also possible to define --verbose --verbose twice, but that makes the test run so verbose that is it unusable.

When running a suite of tests, you only want to have extra output visible if the test fails. If the server fails to start and a particular test doesn’t run at all, using --verbose-restart might be beneficial. If running the test in an automated system, one might want to save results in a JUnit-compatible XML file that can, for example, be rendered by GitLab CI.

This is the command that several CI systems run MTR with:

shell Copy

export MTR_PRINT_CORE=detailed eatmydata perl -I. ./mariadb-test-run \ --force --testcase-timeout=120 --suite-timeout=540 --retry=3 \ --verbose-restart --max-save-core=1 --max-save-datadir=1 \ --parallel=auto --skip-rpl --suite=main \ --xml-report=mariadb-test-run-junit.xml

export MTR_PRINT_CORE=detailed eatmydata perl -I. ./mariadb-test-run \ --force --testcase-timeout=120 --suite-timeout=540 --retry=3 \ --verbose-restart --max-save-core=1 --max-save-datadir=1 \ --parallel=auto --skip-rpl --suite=main \ --xml-report=mariadb-test-run-junit.xml

The command above also showcases use of eatmydata, which makes fsync and similar system calls to skip memory-to-disk guarantees, but in my testing with MTR, it didn’t affect speed.

The above summarizes everything one typically needs to know for running the mariadb-test-run.

For a DBA, there are also several other tools that ship with MariaDB, which can help with testing and validating ones own environment, such as mariadb-stress-test and mariadb-slap.

If you want to write a new test or fix an existing test, there are many more additional parameters to learn, such as --record and --gcov. The official contribution docs at mariadb.org list some of the most useful MTR parameters. The mariadb.com knowledge base article lists them all. As with all commands in Linux, there is also the main page for mariadb-test-run.

The structure of the tests is actually quite easy, and requires no C/C++ skills to write. Each test file (suffix .test) consists mainly of SQL code which is executed by mariadb-test-run (MTR), with the output compared (with diff) to the corresponding file with the expected output in text format (suffix .result).

In my development workflow, writing a test and running MTR might look something like this:

If you want to contribute to the MariaDB open source project, extending the test coverage is a great place to start. To scratch your own itch, think about a MariaDB bug you encountered while using it, and think if that can be reproduced as a test and submitted upstream so that it becomes part of the body of tests and thus easy to catch if it ever regresses again.

To learn more about writing mariadb-test-run tests, read the test case authoring guide at mariadb.org.