Project

General

Profile

Task #1433

manual testing w/ TestLink

Added by Florian Effenberger almost 3 years ago. Updated 2 months ago.

Status:
In Progress
Priority:
Normal
Target version:
Team - Recurring
Start date:
Due date:
% Done:

0%

Estimated time:

Description

We regularly need to do manual testing for

  • alphas, betas, RCs
  • development snapshots
This involves:
  • keeping the test descriptions up to date
  • removing, updating and adapting tests where necessary
  • create runs adapted to each build
  • finding contributors to run the tests and report, growing the volunteer base
  • running a set of tests directly
  • verify the results of the tests run
  • enter issues in Bugzilla if not done by the tester
  • follow-up with the tester if needed

History

#1 Updated by Sophie Gautier almost 3 years ago

  • Description updated (diff)

#2 Updated by Florian Effenberger over 2 years ago

That's something I'd like to hand over to the upcoming QA role primarily, so let's revisit this when they are onboard

#3 Updated by Florian Effenberger over 1 year ago

With Xisco on board now, is there anything major time consuming on your plate left? Otherwise, I'd like to close/re-assign this ticket and have Xisco coordinate with Björn on it

#4 Updated by Sophie Gautier over 1 year ago

Florian Effenberger wrote:

With Xisco on board now, is there anything major time consuming on your plate left? Otherwise, I'd like to close/re-assign this ticket and have Xisco coordinate with Björn on it

I would like to continue to participate there. Currently, it's useless to go on with MozTrap as the version we used is not maintained for 4 years and need to be updated. There is an open source tool named Tarantula and I would like to give it a try http://www.testiatarantula.com/n but I didn't get an answer from Bjoern about it.
The reason I would like to go on with manual testing is because it's about community building together with QA.

#5 Updated by Florian Effenberger over 1 year ago

I would like to continue to participate there. Currently, it's useless
to go on with MozTrap as the version we used is not maintained for 4
years and need to be updated. There is an open source tool named
Tarantula and I would like to give it a try
http://www.testiatarantula.com/n but I didn't get an answer from Bjoern
about it.
The reason I would like to go on with manual testing is because it's
about community building together with QA.

Can you sync with Marina and Osvaldo (both are board oversight for QA)
to get their feedback on whether manual testing makes sense? I can't
judge - I think it does make sense, but I'd like to have some qualified
feedback on that. :-)

If the outcome is positive, we can look into factoring some time to get
Tarantula setup.

#6 Updated by Florian Effenberger over 1 year ago

  • Subject changed from MozTrap manual testing to manual testing
  • Description updated (diff)
  • Status changed from In Progress to Feedback

#7 Updated by Florian Effenberger over 1 year ago

Current plan is to use TestLink, feedback TBD

#8 Updated by Florian Effenberger over 1 year ago

  • Subject changed from manual testing to manual testing w/ TestLink
  • Status changed from Feedback to In Progress

#9 Updated by Florian Effenberger over 1 year ago

Current status is that Sophie and Xisco are working on setting things up, plan is to have it ready end-April for the next BHS
Sophie, correct me if that is wrong :-)

#10 Updated by Sophie Gautier over 1 year ago

Florian Effenberger wrote:

Current status is that Sophie and Xisco are working on setting things up, plan is to have it ready end-April for the next BHS
Sophie, correct me if that is wrong :-)

It's correct :)

#11 Updated by Florian Effenberger about 1 year ago

What's the current TestLink status?

#12 Updated by Sophie Gautier about 1 year ago

It is up and running. I've incorporated some feedback I get during the BHS, but as the alpha build was late, very few have run the tests. I've worked on the localization process (on FR only for the moment) and it's ready to be populated for the l10n teams who want to participate. I've also documented it on the wiki. I would like to discuss with Xisco on how to proceed further steps during the next QA meeting.

#13 Updated by Florian Effenberger about 1 year ago

Any update on TestLink? Are things up and running? How is the acceptance?
Anything missing or blocking production use?

#14 Updated by Sophie Gautier 11 months ago

I need to discuss it with Xisco during LibOCon. For the moment it is stalled, we need to define a process where automated tests are not duplicated in testlink and where manual tests are useful both for developers and l10n teams. Also the list of tests have to be updated on a regular basis.

#15 Updated by Florian Effenberger 11 months ago

Sophie Gautier wrote:

I need to discuss it with Xisco during LibOCon. For the moment it is stalled, we need to define a process where automated tests are not duplicated in testlink and where manual tests are useful both for developers and l10n teams. Also the list of tests have to be updated on a regular basis.

Is that a discussion we can kick off already earlier on, e.g. during one of the next QA calls and all-in team meetings?

#16 Updated by Florian Effenberger 10 months ago

Discussing with Xisco and Sophie pending to see how to move this forward

#17 Updated by Florian Effenberger 9 months ago

Was there any outcome at LibOCon on how to proceed?

#18 Updated by Sophie Gautier 8 months ago

Update on this:
Xisco has created a wiki page:
https://wiki.documentfoundation.org/ListUiTests
running a script he wrote:
https://gerrit.libreoffice.org/#/c/45496/
to be run from bin dir using ./list-uitest.py

by the end of the year, I'll update TestLink accordingly, remove duplicates with unittests, add tests for new functionnalities.

#19 Updated by Sophie Gautier 5 months ago

Update: Raal is writing automatic UI tests so we discussed (with him, Xisco and me) to not overlap between his work and TestLink.
I'll work on "everything related to mouse gestures (ie. insert a shape where drag&drop is needed). Besides, the uitests use the uno commands directly, meaning, it doesn't care if the menu/sidebar/toolbar entry exists or not, so the menus, sidebar and toolbars are not covered by the uitests. Manual tests are also good to tests the translations are ok and make sense and everything looks good in the UI. The uitests don't check the position of the objects in the dialogs."

To try to engage more people in both processes, during the last QA meeting, we have decided to add NeeUITest keyword to the bugs who need either manual or automatic UI tests. I tagged a first list, based on Raal input:
https://bugs.documentfoundation.org/buglist.cgi?keywords=needUITest&list_id=775877

Next step is to update wiki pages to get rid of MozTrap processes - remaining is mostly how to write UI tests on TestLink.

#20 Updated by Florian Effenberger 2 months ago

Can you give me a status update on where we stand with MozTrap, if anything is needed, any blockers or questions? ;-)

#21 Updated by Sophie Gautier 2 months ago

I set a test session for the last BHS but it has not been used, it seems nobody was interesting by running manual tests. I'll set another one for the next BHS on the beta.

#22 Updated by Florian Effenberger 2 months ago

If there's no progress, can you talk to Xisco and those in the board
with oversight on QA whether to keep this task?

Also available in: Atom PDF