Modify

#11845 closed Feature Requests (fixed)

Ability to generate the unique and stable test name for every data set in BOOST_DATA_TEST_CASE

Reported by: mikhail@… Owned by: Raffi Enficiaud
Milestone: Boost 1.62.0 Component: test
Version: Boost 1.59.0 Severity: Problem
Keywords: Cc:

Description

Ability to generate the unique and stable test name for every data item in BOOST_DATA_TEST_CASE.

In this example will be run 3 tests with name "T":

BOOST_DATA_TEST_CASE(TTT, std::vector<char const *>({"A", "B", "C"}), PPP)
{
}

There is no way to understand which parameter PPP was used for every test until test will be failed (our case is log_level::log_test_units, because log_level::log_successful_tests generate too many unnecessary messages).

Output for log_level::log_test_units:

Entering test case "TTT"
Leaving test case "TTT"; testing time: 1ms
Entering test case "TTT"
Leaving test case "TTT"; testing time: 1ms
Entering test case "TTT"
Leaving test case "TTT"; testing time: 1ms

Almost all continues integration tools can collect test statistics by its name. But test_unit_start()/test_unit_finish() from boost::unit_test::unit_test_log_formatter has no test data and has no access to the test context. It means that continues integration tools can receive only the information about test TTT which was run 3 times. It's not truth.

It's possible to inject test name customizer for m_tc_name here.

Attachments (0)

Change History (11)

comment:1 Changed 20 months ago by Raffi Enficiaud

Version: Boost 1.59.0Boost 1.61.0

comment:2 Changed 20 months ago by Raffi Enficiaud

Milestone: To Be DeterminedBoost 1.61.0
Version: Boost 1.61.0Boost 1.59.0

comment:3 Changed 18 months ago by Raffi Enficiaud

Milestone: Boost 1.61.0Boost 1.62.0

comment:4 Changed 13 months ago by Raffi Enficiaud

Each data test case now has its own name:

  • the name provided in
    BOOST_DATA_TEST_CASE(TTT, std::vector<char const *>({"A", "B", "C"}), PPP)
    
    defines a test suite TTT
  • each sample of the dataset is a test case that is a child of the previous test suite, with names in "_0", "_1", etc.

Currently on develop and master (rev a8fa08b9b78516b78458ecb8b005d520a0c173f7)

Last edited 13 months ago by Raffi Enficiaud (previous) (diff)

comment:5 Changed 13 months ago by Raffi Enficiaud

Owner: changed from Gennadiy Rozental to Raffi Enficiaud

comment:6 Changed 13 months ago by Raffi Enficiaud

Status: newassigned

comment:7 Changed 12 months ago by Raffi Enficiaud

Resolution: fixed
Status: assignedclosed

Released in 1.62

comment:8 Changed 12 months ago by mikhail@…

Milestone: Boost 1.62.0Boost 1.63.0
Resolution: fixed
Severity: ProblemRegression
Status: closedreopened
Type: Feature RequestsBugs
Version: Boost 1.59.0Boost 1.62.0

I was very upset when have seen the result. There is no more test name! I got numbers from 0 to 74 for my 7 data case tests in one test suite. There is more ability to understand which test was failed! Could you please return the test name back and keep numbers? But please make separate numeration for each data case test! This is very important!

comment:9 Changed 12 months ago by mikhail@…

Sorry for mistake: There is NO more ability to understand which test was failed!

comment:10 Changed 12 months ago by mikhail@…

Milestone: Boost 1.63.0Boost 1.62.0
Severity: RegressionProblem
Type: BugsFeature Requests
Version: Boost 1.62.0Boost 1.59.0

I'm very very sorry. I haven't seen that you create virtual test suite with test name. Unfortunately, I can't close the ticket. Could you please close it again?

comment:11 Changed 12 months ago by Raffi Enficiaud

Resolution: fixed
Status: reopenedclosed

Hi,

Next time I suggest you read the change log more carefully. Thanks.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Raffi Enficiaud.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.