continuous integration - How to fix failed tests count in teamcity for re-executed tests? -

-

we using teamcity selenium tests execution specflow+specrun, problem teamcity count re-executed tests.

for example if test failed first time re-executed 2 times more, in teamcity see 3 tests failed, 1 test.

also if first re-execution fail, other 2 success, reported in teamcity 2 failed, 1 passed, need reported 1 test has been passed.

it possible configure in teamcity using service messages or else?

updated:

based on answer can gather logs using powershell script , change build status using teamcity service messages:

$path = get-childitem %teamcity.build.checkoutdir%\projectfolder\bin\remote\testresults\specrun.log $file = get-content $path $total = $file | select-string "total:" $passed = $file | select-string "succeeded:" $failed = $file | select-string "failed:"  write-host $( "##teamcity[buildstatus text='tests {0}, {1}, {2}']" -f $total, $passed, $failed )

teamcity behaving expected. teamcity reports testing framework tells him. process is:

  • execute step run tests.
  • gather output logs.
  • extract stats: number of test executed, passed , failing.
  • fail build if tests fail if configured that.

you have configure testing framework (specflow+specrun in case) change way reports re-execution. you should configure specflow log re-executions in different way... if can.

in general should avoid re-execution of tests. tests should work @ first attempt. should prepare state if needed, test, , clean state.

the need of re-execution indicates dependency other tests. like:

  • test 001: search user foo1. (fails)
  • test 002: adds user foo1. (ok)
  • re-execution of test 001: search user foo1. (ok)

the correct way test001 not have dependency of test002, adds user foo1, test search , clean user foo1.


Comments

Post a Comment

Popular posts from this blog

wireshark - USB mapping with python -

-
while reading ctf write-ups came across script #!/usr/bin/env python import struct import image import dpkt init_x, init_y = 100, 400 def print_map(pcap, device): picture = image.new("rgb", (1200, 500), "white") pixels = picture.load() x, y = init_x, init_y ts, buf in pcap: device_id, = struct.unpack("b", buf[0x0b]) if device_id != device: continue data = struct.unpack("bbbb", buf[-4:]) status = data[0] x = x + data[1] y = y + data[2] if (status == 1): in range(-5, 5): j in range(-5, 5): pixels[x + , y + j] = (0, 0, 0, 0) else: pixels[x, y] = (255, 0, 0, 0) picture.save("riverside-map.png", "png") if __name__ == "__main__": f = open("usb.pcap", "rb") pcap = dpkt.pcap.reader(f) print_map(pcap, 5) f.close() and ...
Read more

c++ - nodejs socket.io closes connection before upgrading to websocket -

-
i'm trying write c++ client talk websocket server written using nodejs , socket.io. can't c++ client establish websocket connection; server drops connection right after client requests http connection upgraded websocket. i used tcpdump watch packets going , forth. when use websocket server browser, works great , tcpdump output shows packets going , forth. when try connect server using c++ client, websocket connection never gets established. output tcpdump shows upgrade request being sent server followed server dropping connection. i tried replicating sequence of events using nc . connected server (running on port 8080) , copy-pasted exact set of headers sent browser, captured tcpdump. duplicate of working headers, connection dropped. there no output on server end indicating why connection gets dropped. here headers sent: get /socket.io/?type=player&eio=3&transport=websocket&sid=7mdclht_8ctsd6f4aaac http/1.1 host: localhost:8080 connection: upgrade prag...
Read more

Deploying Qt Application on Android is really slow? -

-
as may know there 3 ways deploy qt application on android : use ministro service install qt deploy local qt libraries temporary directory bundle qt libraries in apk the first method takes 30 seconds , needs install apk . ministro . second takes 1 minute me ! , anytime try run program qt creator pushes qt libraries device. third 1 makes .apk file big , again takes 1 minute me. think situation that's not reasonable develop android application using qt. there way make deploying process faster? almost full year since op , things have not changed @ all. deployment of 7 mb apk takes on minute , half project compiles in 5 seconds. reason answering not problem got resolved, offer alternative solution. i've implemented "workaround" consisting of 2 applications work in tandem - 1 on pc , 1 on device - created compile files remotely, turned out faster alternative deployment well. on host create application launches compilation in separate process, when do...
Read more