Monday, October 5, 2009

Running Multiple Unit Tests

Qt Creator supports creating a unit test project but the Qt documentation only describes running a single test. To run multiple tests you need to modify your main function to create an instance of each test object and then call qExec on it. Easy enough, but when you start creating lots of tests it's easy to forget to add the necessary code. I decided to simplify this process and came up with a small header file that you can use to make running multiple tests a breeze.

The code for the header, AutoTest.h can be found below. To use it, simply:
  1. Add a #include "AutoTest.h" to all your QObject-derived test headers.
  2. Add DECLARE_TEST(YourTestClassName) below your class definition.
  3. Change your main.cpp to look like this:

    #include "AutoTest.h"

    TEST_MAIN

Easy. If you still need to supply your own main function (you might have other initialization code in here) then you can run all the tests with the following line of code:

AutoTest::run(argv, argc);

This function will return 0 if all the tests passed.

The source for AutoTest.h:

#ifndef AUTOTEST_H
#define AUTOTEST_H

#include <QTest>
#include <QList>
#include <QString>
#include <QSharedPointer>

namespace AutoTest
{
typedef QList<QObject*> TestList;

inline TestList& testList()
{
static TestList list;
return list;
}

inline bool findObject(QObject* object)
{
TestList& list = testList();
if (list.contains(object))
{
return true;
}
foreach (QObject* test, list)
{
if (test->objectName() == object->objectName())
{
return true;
}
}
return false;
}

inline void addTest(QObject* object)
{
TestList& list = testList();
if (!findObject(object))
{
list.append(object);
}
}

inline int run(int argc, char *argv[])
{
int ret = 0;

foreach (QObject* test, testList())
{
ret += QTest::qExec(test, argc, argv);
}

return ret;
}
}

template <class T>
class Test
{
public:
QSharedPointer<T> child;

Test(const QString& name) : child(new T)
{
child->setObjectName(name);
AutoTest::addTest(child.data());
}
};

#define DECLARE_TEST(className) static Test<className> t(#className);

#define TEST_MAIN \
int main(int argc, char *argv[]) \
{ \
return AutoTest::run(argc, argv); \
}

#endif // AUTOTEST_H

A typical test class header would look something like this:

#ifndef FOOTESTS_H
#define FOOTESTS_H

#include "AutoTest.h"

class FooTests : public QObject
{
Q_OBJECT
private slots:
void initTestCase();
void test1();
void test2();
void cleanupTestCase();
};

DECLARE_TEST(FooTests)

#endif // FOOTESTS_H

Apologies for the formatting - haven't worked out how to post decent looking code using blogger yet.

17 comments:

  1. I'm doing something similar to this using the post-link qmake step to run the unit test binary. BUT, is there a way to rebuild the unit test after the library (or thing that you're testing) has been built so that it is updated?

    ReplyDelete
  2. Yes, this is possible - I have mine setup to build and run the tests when the .lib the tests are for changes. I'll post details tomorrow.

    ReplyDelete
  3. This automatic testing sounds great, I'll try this for my projects!

    greets,
    3DH

    ReplyDelete
  4. Hello, this is what I'm finding!
    But, I got these errors for my two test classes:

    conflicting declaration 'Test(TestQString) t'
    't' has a previous declaration as `Test(TestArray) t'
    declaration of `Test(TestQString) t'
    conflicts with previous declaration `Test(TestArray) t'

    NOTE: "greater than" and "less than" marks are replaced by ) and ( due to editbox limitation.

    ReplyDelete
    Replies
    1. I'm seeing this same issue. Any resolution?

      Delete
    2. I think I found the issue. I couldn't #include the files in the main. But now I am facing other errors, such as:

      1) "invalid declaration of member template in local class"

      2) "local class 'class main(int, char**)::QTestClass1' shall not have static data member 'const QMetaObject main(int, char**)::QTestClass1::staticMetaObject'"

      3) "local class 'class main(int, char**)::QTestClass1' shall not have static data member 'const QMetaObjectExtraData main(int, char**)::QTestClass1::staticMetaObjectExtraData'"

      4) template argument for 'template class Test' uses local type 'main(int, char**)::QTestClass1'"

      5) "invalid type in declaration before '(' token"

      6) "initializer expression list treated as compound expression

      Note: I did add an enum to the DECLARE_TEST macro and the constructor of the Test class.

      Delete
    3. I had the above errors when using the macro DECLARE_TEST in my headers instead of my cpps.
      Another workaround may be to change the macro to: #define DECLARE_TEST(className) static Test t##className(#className);

      Delete
  5. DIFF ... give me an email address and I'll email you a sample Qt4 project that uses AutoTest.h - or send me some code.

    ReplyDelete
  6. Could you post a SSCCE? I.e. a complete qt test project that runs?

    ReplyDelete
  7. @Simpatico: It isn't possible to host files on here AFAIK but I'll see what I can do.

    ReplyDelete
  8. Great Work! Thx :)

    ReplyDelete
  9. I would like to say that you really made my day, it's wonderful when you just look around the web
    and find something like this, reminds me of that ''How to make a dinner for a romantic...'' by Elsa Thomas,
    you're a wonderful writer let me tell you!!! ñ_ñ

    James Maverick (maverickhunterjames@gmail.com)
    3453 Rardin Drive
    San Mateo, CA 94403
    Project Manager
    650-627-8033

    ReplyDelete
  10. Great Work! Just to be sure, is this code licensed under GPL?

    ReplyDelete
  11. Thanks very much, works great, only clarification I needed to debug was that you need to:
    # Add DECLARE_TEST(YourTestClassName) below your class DECLARATION (in the header) rather than below the definition (in the CPP file).

    ReplyDelete
  12. If you're using C++11 try this solution. http://edwrodrig.github.io/lessons/2014/05/05/testing-multiple-classes-using-c++11-lambdas-in-qt/

    ReplyDelete