continuous integration - How to fix failed tests count in teamcity for re-executed tests? -


we using teamcity selenium tests execution specflow+specrun, problem teamcity count re-executed tests.

for example if test failed first time re-executed 2 times more, in teamcity see 3 tests failed, 1 test.

also if first re-execution fail, other 2 success, reported in teamcity 2 failed, 1 passed, need reported 1 test has been passed.

it possible configure in teamcity using service messages or else?

updated:

based on answer can gather logs using powershell script , change build status using teamcity service messages:

$path = get-childitem %teamcity.build.checkoutdir%\projectfolder\bin\remote\testresults\specrun.log $file = get-content $path $total = $file | select-string "total:" $passed = $file | select-string "succeeded:" $failed = $file | select-string "failed:"  write-host $( "##teamcity[buildstatus text='tests {0}, {1}, {2}']" -f $total, $passed, $failed ) 

teamcity behaving expected. teamcity reports testing framework tells him. process is:

  • execute step run tests.
  • gather output logs.
  • extract stats: number of test executed, passed , failing.
  • fail build if tests fail if configured that.

you have configure testing framework (specflow+specrun in case) change way reports re-execution. you should configure specflow log re-executions in different way... if can.

in general should avoid re-execution of tests. tests should work @ first attempt. should prepare state if needed, test, , clean state.

the need of re-execution indicates dependency other tests. like:

  • test 001: search user foo1. (fails)
  • test 002: adds user foo1. (ok)
  • re-execution of test 001: search user foo1. (ok)

the correct way test001 not have dependency of test002, adds user foo1, test search , clean user foo1.


Comments

Popular posts from this blog

javascript - Laravel datatable invalid JSON response -

java - Exception in thread "main" org.springframework.context.ApplicationContextException: Unable to start embedded container; -

sql server 2008 - My Sql Code Get An Error Of Msg 245, Level 16, State 1, Line 1 Conversion failed when converting the varchar value '8:45 AM' to data type int -