So situation is following: We've got some early product, which has dozen of solutions. Most of them run different unit tests, some MSTest, some NUnit. You will find large amount of tests that test database, several rely on data. Quite simply they execute code to see if customer Andriy has orders with items A and B. I understand this is wrong, however i as stated the product was created by many people teams in various occasions.
Presently many teams are ongoing doing changes to projects and changes are addressed to various branches. Build conditions are in place in CI manner. Therefore we face problem with maintaining individuals tests. We spent a lot of time and efforts on keeping tests running. Because databases with test data are actually heavy we can't have separate database for every branch, but we are able to keep obvious sample databases (200Mb) for every of branches, however in this situation many tests will fail.
I already considered this and also have some preliminary plan like:
1) Eliminating data-dependent tests, by either getting rid of them or (if at all possible) spinning to place data and rollback soon after test performed.
2) Move all database-specific tests in separate projects - i.e. separating Integration tests from usual Unit Tests
3) Create sample database per each branch and run integration tests individually.
I would like to know what you think how the same situation were resolved in your projects. Have you separate integration tests and have you have data specific tests. Could it be suggested to possess data-dependent tests whatsoever? Any suggestions is going to be appreciated.
For the current project we have separated CI- from Unit-tests, exactly how you describe and it is working well. By moving to NHibernate we're free of charge an in-memory database (SQLite) to check data-dependent code that can't be easily mocked*, but we have some integration-type tests running from the test-server-local SQL Server. All individuals tests specify the MbUnit Rollback2-attribute and both place the information they require and query it within the same round, to ensure that the database does not get modified through the tests.
NUnit + SharpTestEx can be used for the system tests.
*Like the test that test collection cascading down mappings and sets, IEquatable which the legacy-table schema is properly planned.