QA Infrastructure

classic Classic list List threaded Threaded
24 messages Options
Next » 12
Thorsten Behrens Thorsten Behrens
Reply | Threaded
Open this post in threaded view
|

QA Infrastructure

Hi,

as we start to ramp up more infrastructure, I'd like to make
you think about what's crucially needed to do QA for LibreOffice
3.3.

First of all, this is what we currently have:

 * a LibreOffice technical list (I'd like to have devs & QA
   discussion there):
   http://lists.freedesktop.org/mailman/listinfo/libreoffice

 * a bugtracker - bugs.freedesktop.org for the while
   (use
    https://bugs.freedesktop.org/buglist.cgi?query_format=specific&order
=relevance+desc&bug_status=__open__&product=LibreOffice&content    to list all currently-filed LibreOffice bugs)

 * a wiki (well, soon ;))

 * the testtool (similar to OpenOffice.org)

I'd prefer if we do the 3.3 release in a somewhat lightweight
fashion, and add tools as we go (and decide that we need them) - I
know that the OpenOffice.org QA project has things like QATrack,
QUASTe, and TCM - but I wonder which of those pass the test of "we
really need it, and it's worth the effort to duplicate it/set it
up".

What do you think?

-- Thorsten

--
To unsubscribe, send an empty e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

Rainer Bielefeld Rainer Bielefeld
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

Thorsten Behrens schrieb:

>
>   * a bugtracker - bugs.freedesktop.org for the while
>     (use
>      https://bugs.freedesktop.org/buglist.cgi?query_format=ecific&order
> =levance+desc&bug_status=__open__&product=LibreOffice&content    to list all currently-filed LibreOffice bugs)
>


Hi,

a better working link is
<https://bugs.freedesktop.org/buglist.cgi?query_format=specific&order=relevance+desc&bug_status=__all__&product=LibreOffice&content=>

CU

Rainer
--
To unsubscribe, send an empty e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

Jean-Baptiste_Faure Jean-Baptiste_Faure
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

In reply to this post by Thorsten Behrens
 Le 07/10/2010 10:25, Thorsten Behrens a écrit :

> Hi,
>
> as we start to ramp up more infrastructure, I'd like to make
> you think about what's crucially needed to do QA for LibreOffice
> 3.3.
>
> First of all, this is what we currently have:
>
>  * a LibreOffice technical list (I'd like to have devs & QA
>    discussion there):
>    http://lists.freedesktop.org/mailman/listinfo/libreoffice
>
>  * a bugtracker - bugs.freedesktop.org for the while
>    (use
>     https://bugs.freedesktop.org/buglist.cgi?query_format=ecific&order
> =levance+desc&bug_status=__open__&product=LibreOffice&content    to list all currently-filed LibreOffice bugs)
>
>  * a wiki (well, soon ;))
>
>  * the testtool (similar to OpenOffice.org)
>
> I'd prefer if we do the 3.3 release in a somewhat lightweight
> fashion, and add tools as we go (and decide that we need them) - I
> know that the OpenOffice.org QA project has things like QATrack,
> QUASTe, and TCM - but I wonder which of those pass the test of "we
> really need it, and it's worth the effort to duplicate it/set it
> up".
>
> What do you think?

Hi Thorsten, all,

I think we need something like TCM to manage manual tests of localisation.
I know that Sophie is/was working on a new version of TCM.
She should have some good ideas in this matter.

Best regards

JBF

--
Seuls des formats ouverts peuvent assurer la pérennité de vos documents.

--
To unsubscribe, send an empty e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

Seuls des formats ouverts peuvent assurer la pérennité de vos documents.
Friedrich Strohmaier Friedrich Strohmaier
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

In reply to this post by Thorsten Behrens
Hi Thorsten, *,

Thorsten Behrens schrieb:

>as we start to ramp up more infrastructure, I'd like to make
>you think about what's crucially needed to do QA for LibreOffice
>3.3.

>First of all, this is what we currently have:

<stretched>

> * a LibreOffice technical list (I'd like to have devs & QA
>   discussion there):

So do I, but I'd suggest to put the patch conversation on a different
list - sort of [hidden email].

> * a bugtracker - bugs.freedesktop.org for the while (use

> * a wiki (well, soon ;))

> * the testtool (similar to OpenOffice.org)

</stretched>

+1

>I'd prefer if we do the 3.3 release in a somewhat lightweight
>fashion, and add tools as we go (and decide that we need them)

+1

>I know that the OpenOffice.org QA project has things like QATrack,
>QUASTe, and TCM - but I wonder which of those pass the test of "we
>really need it, and it's worth the effort to duplicate it/set it up".

>What do you think?

I'll put here a short draft of my dreams if I think of an efficient QA
:o))

I think of three states the software idally should pass:

1. The most recent developer build (nightly builds).

2. an "alive" release lets call it "LibO - fresh" which passed a quite
   short beta period for early adopters, experienced users, and "new
   features greedy" ones.

3. a "mature" release which has passed ~6 months "fresh" state.


First point doesn't need additional comments

2. comment:
"fresh" issues reported by "fresh"-users should get fixed as soon as
possible and end in nightly builds after an issue has been fixed.

3. comment:
"mature" issues can only be security issues and should also be aided by
nightly builds each time an issue has been fixed. Additionally security
fixes should be provided for a reasonable period in aspect of an office
environment.

The thought behind is:
Bugs are relevant not by possibility to happen but by appearence.

Therefore it might be a good idea to be as close as possible at users
*real* experience. We get the closer to it the closer our response is to
users anoyance. If we succeed doing this, we will get that users which
are greedy hunting bugs. ;o))

Because the *possibility* of a bug's existence doesn't matter much, I
think we shouldn't bother interested coworkers with boring and difficult
to learn tools blocking their machine by runnig automated tests which
never hit people's creativity to do strange things ;o)).

And last one:
The remaining dev time *after* fixing bugs is reserved for implementing
new features ;o))

In short -- draft -- not mature

Gruß/regards
--
Friedrich

Ansprechpartner / contact person for the "PrOOo-Box"
german language OpenOffice.org and more on CD/DVD
http://prooo-box.org 

--
To unsubscribe, send an empty e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

André Schnabel André Schnabel
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

In reply to this post by Thorsten Behrens
Hi Thorsten, *

Am 07.10.2010 10:25, schrieb Thorsten Behrens:

>
> I'd prefer if we do the 3.3 release in a somewhat lightweight
> fashion, and add tools as we go (and decide that we need them) - I
> know that the OpenOffice.org QA project has things like QATrack,
> QUASTe, and TCM - but I wonder which of those pass the test of "we
> really need it, and it's worth the effort to duplicate it/set it
> up".
>
> What do you think?

I think, we are quite good prepared for the start. What is missing are
some tools to collect if someone did test something at all. We might use
Wiki or mailinglist for this.

For all the tools (I'd say, we need something like TCM, QUASTE to
collect automated tests results and maybe QATrack), we should make sure
that we keep them simple so that many people are able to work with the
tools (there are many low hanging fruits for testers we should not make
it complicated). That said - tehre is unfortunately nothing "ready to
use" afaik.

André
--
To unsubscribe, send an empty e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

Per Eriksson Per Eriksson
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

In reply to this post by Thorsten Behrens

  Hi Thorsten,

Thorsten Behrens skrev 2010-10-07 10:25:

> Hi,
>
> as we start to ramp up more infrastructure, I'd like to make
> you think about what's crucially needed to do QA for LibreOffice
> 3.3.
>
> First of all, this is what we currently have:
>
>   * a LibreOffice technical list (I'd like to have devs&  QA
>     discussion there):
>     http://lists.freedesktop.org/mailman/listinfo/libreoffice
>
>   * a bugtracker - bugs.freedesktop.org for the while
>     (use
>      https://bugs.freedesktop.org/buglist.cgi?query_format=ecific&order
> =levance+desc&bug_status=__open__&product=LibreOffice&content    to list all currently-filed LibreOffice bugs)
>
>   * a wiki (well, soon ;))
>
>   * the testtool (similar to OpenOffice.org)
>
> I'd prefer if we do the 3.3 release in a somewhat lightweight
> fashion, and add tools as we go (and decide that we need them) - I
> know that the OpenOffice.org QA project has things like QATrack,
> QUASTe, and TCM - but I wonder which of those pass the test of "we
> really need it, and it's worth the effort to duplicate it/set it
> up".

This may not the most important thing for the first release, but in the
long run QATrack would serve as a dashboard for the different statuses
for different rc's for different languages. If we want to use it, I
would be happy to set it up for you. The prerequisite is a server to put
it on, which I could find out.

The other tools are great tools for the initial release, and also some
written tests as the qa starts?

Best

Per
--
To unsubscribe, send an empty e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

Cor Nouws Cor Nouws
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

In reply to this post by Jean-Baptiste_Faure
Jean-Baptiste Faure wrote (07-10-10 14:04)

> I think we need something like TCM to manage manual tests of localisation.
> I know that Sophie is/was working on a new version of TCM.
> She should have some good ideas in this matter.

Experience in Dutch NLC (where TCM is not used that much):

People joining find it annoying and difficult to understand that they
run into bugs that either are no showstoppers or already found before.

So first that means people need guidance, so they understand what is
expected.
But maybe also, that the system works as much as possible as an easy
helper app to do necessary checks.

Ciao - Cor

--
  - giving openoffice.org its foundation :: The Document Foundation -
  - ideas/remarks for the community council? See
    http://wiki.services.openoffice.org/wiki/Community_Council

--
To unsubscribe, send an empty e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

vuhung vuhung
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

In reply to this post by Friedrich Strohmaier
Hello,

On Thu, Oct 7, 2010 at 9:58 PM, Friedrich Strohmaier <
[hidden email]> wrote:

> Hi Thorsten, *,
>
> Thorsten Behrens schrieb:
>
> >as we start to ramp up more infrastructure, I'd like to make
> >you think about what's crucially needed to do QA for LibreOffice
> >3.3.
>
> >First of all, this is what we currently have:
>
> <stretched>
>
> > * a LibreOffice technical list (I'd like to have devs & QA
> >   discussion there):
>
> So do I, but I'd suggest to put the patch conversation on a different
> list - sort of [hidden email].
>
> > * a bugtracker - bugs.freedesktop.org for the while (use
>
> > * a wiki (well, soon ;))
>
+1


> > * the testtool (similar to OpenOffice.org)
>
> </stretched>
>
> +1
>
Do you mean VCLTestool - used for automated testing in LibO?

Do we need a (VCLTesttool-used) test server? and
Do we need a build server like Java Huson?

That would be a big help (for QA members)


> >I'd prefer if we do the 3.3 release in a somewhat lightweight
> >fashion, and add tools as we go (and decide that we need them)
>
> +1
>
+1


>
> >I know that the OpenOffice.org QA project has things like QATrack,
> >QUASTe, and TCM - but I wonder which of those pass the test of "we
> >really need it, and it's worth the effort to duplicate it/set it up".
>
> >What do you think?
>
> I'll put here a short draft of my dreams if I think of an efficient QA
> :o))
>
> I think of three states the software idally should pass:
>
> 1. The most recent developer build (nightly builds).
>
The build takes days, so I would like weekly buids.


> 2. an "alive" release lets call it "LibO - fresh" which passed a quite
>   short beta period for early adopters, experienced users, and "new
>   features greedy" ones.
>
I thought our developers would work on git/svn branches so we have lots of
builds
(like that way Linux kernel hackers do)

All the branches can combined as a "fresh" realease (.i.e. unstable)
Combined works on the branches those are targeted for official releases,
we will have RC1, RC2... versions.

This staging strategy will improve the quality of LibO - I hope.


> 3. a "mature" release which has passed ~6 months "fresh" state.
>
> Question: How do us determine LibO's realease cycle.
I want it to be as short as possible (6 months?)


--
Best Regards,
Nguyen Hung Vu [aka: NVH] ( in Vietnamese: Nguyễn Vũ Hưng
 )
vuhung16plus{remove}@gmail.dot.com
<vuhung16plus%7Bremove%[hidden email]>, YIM: vuhung16 , Skype:
vuhung16plus
A brief profile: http://www.hn.is.uec.ac.jp/~vuhung/Nguyen.Vu.Hung.html

--
To unsubscribe, send an empty e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

André Schnabel André Schnabel
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

In reply to this post by Per Eriksson
Hi,


Am 07.10.2010 19:10, schrieb Per Eriksson:
>
> This may not the most important thing for the first release, but in the
> long run QATrack would serve as a dashboard for the different statuses
> for different rc's for different languages. If we want to use it, I
> would be happy to set it up for you. The prerequisite is a server to put
> it on, which I could find out.

Just use the servers you already know for QATrack testing and productive
use. :)

(did you ever do a whois on www.documentfoundation.org? :) )

regards,

André
--
To unsubscribe, send an empty e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

Andrea Pescetti Andrea Pescetti
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

In reply to this post by Thorsten Behrens
Thorsten Behrens wrote:
> I'd prefer if we do the 3.3 release in a somewhat lightweight
> fashion

As I understood from an earlier message of yours, LibreOffice 3.3 will
be heavily based on OOo 3.3 and I thus agree with the "lightweight testing".

> the OpenOffice.org QA project has things like QATrack,
> QUASTe, and TCM - but I wonder which of those pass the test of "we
> really need it

I actually believe that these are fundamental tools to assess quality of
a build, and that insufficient testing could mean that LibreOffice is
doomed to be "the nicer but more buggy brother of OOo" forever in the
public opinion.

At least, I recommend that new tests are written for any LibreOffice
specific functionality, that we have a tool to track them (kind of TCM)
and a tool to track releases (QATrack being the natural candidate here).

Regards,
   Andrea Pescetti.
--
To unsubscribe, send an empty e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

Caio Tiago Oliveira Caio Tiago Oliveira
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

Andrea Pescetti, 07-10-2010 19:46:
> Thorsten Behrens wrote:
>> I'd prefer if we do the 3.3 release in a somewhat lightweight
>> fashion
>
> As I understood from an earlier message of yours, LibreOffice 3.3 will
> be heavily based on OOo 3.3 and I thus agree with the "lightweight
> testing".

+1

>> the OpenOffice.org QA project has things like QATrack,
>> QUASTe, and TCM - but I wonder which of those pass the test of "we
>> really need it
>
> I actually believe that these are fundamental tools to assess quality of
> a build, and that insufficient testing could mean that LibreOffice is
> doomed to be "the nicer but more buggy brother of OOo" forever in the
> public opinion.

Most of my friends already say OOo is too buggy. That is mainly due to
the longstanding bugs of the past and some old crashes, even though we
still with some annoying longstanding bugs.

That said, I strongly agreed with you here, that we should avoid
shipping with severe bugs.

VLC TestTool is a great tool to assure the quality of the branches and
releases. It helps avoiding regressions and makes the repetitive tasks
for us.

> At least, I recommend that new tests are written for any LibreOffice
> specific functionality, that we have a tool to track them (kind of TCM)
> and a tool to track releases (QATrack being the natural candidate here).

As far as I know, setting up QATrack for us is easy and Per said he only
needs a server. That's useful and easy to set up and use, so it's a must.

Oracle's TCM is closed. So we need to analyze alternatives.
Testmaster[1] looks nice, but without easy support to test case suits
among multiple languages.
Testopia[2], from Mozilla, is an bugzilla extension that adds test case
management capabilities to bugzilla. Most interesting, it also supports
automated test scripts associated with bugs and updating test status
automatically.
There are various other open source alternatives.
Testopia has some QUASTe-like features and some TCM-like features, both
integrated with the bug tracking system.

1 - http://testmaster.sourceforge.net/
2 - http://www.mozilla.org/projects/testopia/
--
To unsubscribe, send an empty e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

Thorsten Behrens Thorsten Behrens
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

Caio Tiago Oliveira wrote:

> >>the OpenOffice.org QA project has things like QATrack,
> >>QUASTe, and TCM - but I wonder which of those pass the test of "we
> >>really need it
> >
> >I actually believe that these are fundamental tools to assess quality of
> >a build, and that insufficient testing could mean that LibreOffice is
> >doomed to be "the nicer but more buggy brother of OOo" forever in the
> >public opinion.
>
> Most of my friends already say OOo is too buggy. That is mainly due
> to the longstanding bugs of the past and some old crashes, even
> though we still with some annoying longstanding bugs.
>
> That said, I strongly agreed with you here, that we should avoid
> shipping with severe bugs.
>
> VLC TestTool is a great tool to assure the quality of the branches
> and releases. It helps avoiding regressions and makes the repetitive
> tasks for us.
>
Hi Caio, all,

yeah, I think you nicely sum it up - so the tools mentioned are no
silver bullets, they don't per se give you a quality release. That's
why I asked what we'd _really_ need - it means striking a balance
between effort to set things up, or even recode them, providing QA
volunteers coming from OOo an effective work environment (there's a
certain amount of "but we're used to that tool!" mentality to cater
for) - and, I think that's the great opportunity - the chance to
honestly look at the process, make it lean, effective - and fun to
work with.

> As far as I know, setting up QATrack for us is easy and Per said he
> only needs a server. That's useful and easy to set up and use, so
> it's a must.
>
Cool - so I had folks from France and Brazil offering hardware, let
me see if we can connect you.

Cheers,

-- Thorsten

--
To unsubscribe, send an empty e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

Friedrich Strohmaier Friedrich Strohmaier
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

In reply to this post by vuhung
Hi Nguyen Vu Hung, *,

hope this is the correct appellation..

first of all: I'm not involved in Software development and neither in
QA *therefore* I'm answering here. :o))

Nguyen Vu Hung schrieb:
>On Thu, Oct 7, 2010 at 9:58 PM, Friedrich Strohmaier <
>[hidden email]> wrote:
>> Thorsten Behrens schrieb:
>> >as we start to ramp up more infrastructure, I'd like to make
>> >you think about what's crucially needed to do QA for LibreOffice
>> >3.3.

>> >First of all, this is what we currently have:

>> <stretched>

[..]

>> > * the testtool (similar to OpenOffice.org)

>> </stretched>


>Do you mean VCLTestool - used for automated testing in LibO?

>Do we need a (VCLTesttool-used) test server? and
>Do we need a build server like Java Huson?

>That would be a big help (for QA members)

Shure? Maybe it's a great tool for developers but if it is a great tool
for QA members isn't proven yet.

[..]

>> >I know that the OpenOffice.org QA project has things like QATrack,
>> >QUASTe, and TCM - but I wonder which of those pass the test of "we
>> >really need it, and it's worth the effort to duplicate it/set it
>> >up".

>> >What do you think?

>> I'll put here a short draft of my dreams if I think of an efficient
>> QA

>> I think of three states the software idally should pass:

>> 1. The most recent developer build (nightly builds).

>The build takes days, so I would like weekly buids.

Even if actually reality. What I *like* were nightly builds! remember:
It's a dream and I'm not a developer :o))

>> 2. an "alive" release lets call it "LibO - fresh" which passed a
>> quite short beta period for early adopters, experienced users, and
>> "new features greedy" ones.

>I thought our developers would work on git/svn branches so we have lots
>of builds (like that way Linux kernel hackers do)

I freely admit: I don't understand what You are talking about..

>All the branches can combined as a "fresh" realease (.i.e. unstable)
>Combined works on the branches those are targeted for official
> releases, we will have RC1, RC2... versions.

As far as I see those RCs are made to ensure the release which will last
long time (i.e. 6 months) to keep free of bugs. I can't see that anyone
succeded that except microsoft. ;o))

I'd prefer to face reality of the (outer microsoft) world and have the
RCs substituted by nightly builds with fixed bugs - even that ones
appearing *after* releasing the last RC.

>This staging strategy will improve the quality of LibO - I hope.

I am with You! :o))

>> 3. a "mature" release which has passed ~6 months "fresh" state.

>> Question: How do us determine LibO's realease cycle.
>I want it to be as short as possible (6 months?)

Fits that the requirements of an office environment? I'm not shure.


Gruß/regards
--
Friedrich

Ansprechpartner / contact person for the "PrOOo-Box"
german language "best Office Suite ever" and more on CD/DVD
http://prooo-box.org  -- footer changed on 2010-10-07

--
To unsubscribe, send an empty e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

Caio Tiago Oliveira Caio Tiago Oliveira
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

In reply to this post by Thorsten Behrens
On Fri, Oct 8, 2010 at 4:51 AM, Thorsten Behrens
<[hidden email]> wrote:

> Caio Tiago Oliveira wrote:
>> [snip]
>> VLC TestTool is a great tool to assure the quality of the branches
>> and releases. It helps avoiding regressions and makes the repetitive
>> tasks for us.
>>
> Hi Caio, all,
>
> yeah, I think you nicely sum it up - so the tools mentioned are no
> silver bullets, they don't per se give you a quality release. That's
> why I asked what we'd _really_ need - it means striking a balance
> between effort to set things up, or even recode them, providing QA
> volunteers coming from OOo an effective work environment (there's a
> certain amount of "but we're used to that tool!" mentality to cater
> for) - and, I think that's the great opportunity - the chance to
> honestly look at the process, make it lean, effective - and fun to
> work with.

One volunteer with a nice computer and lots of VMs would be able to
run at least the most important tests for each language and the
complete set for english builds.
That said, we don't have one.
Also, I don't know if there is someone here that knows how to write
the testtool scripts here contributing with this project.

Integrating without testing could be dangerous and for a few people
testing every branch (CWS) it would be boring. We can leave this
discussion for latter, but for now testtool is not one of the most
important things on QA, since we can test everything manually.

We can use the wiki and mailing lists instead of TCM, as already
suggested, but that can become a major headache in a near future,
since would be exponentially harder to manage more languages and more
tests.

>> As far as I know, setting up QATrack for us is easy and Per said he
>> only needs a server. That's useful and easy to set up and use, so
>> it's a must.
>>
> Cool - so I had folks from France and Brazil offering hardware, let
> me see if we can connect you.

Per Eriksson, what do you need on the server to run QATrack?
--
To unsubscribe, e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

Andrea Pescetti Andrea Pescetti
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

Caio Tiago Oliveira wrote:
> Per Eriksson, what do you need on the server to run QATrack?

I don'tspeak for Per, but until I maintained it QATrack was a rather
standard LAMP application and, as far as I could see,this is still the
case. But Andre' already clarified that qatrack.services.openoffice.org,
the current instance of QATrack, is on community-run servers and that a
fresh installation wouldn't be problematic.

Regards,
   Andrea Pescetti.
--
To unsubscribe, e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

André Schnabel André Schnabel
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

In reply to this post by Caio Tiago Oliveira
Hi,


Am 09.10.2010 05:00, schrieb Caio Tiago Oliveira:

> Also, I don't know if there is someone here that knows how to write
> the testtool scripts here contributing with this project.

The question is if we have someone willing to learn this. The
dosumentation for the scripts is quite good. With some knowledge about
OOo Basic UNO and LibO in general should be able to modifiy the scripts
within a week and write own scripts after a month or so.

>
> Integrating without testing could be dangerous and for a few people
> testing every branch (CWS) it would be boring. We can leave this
> discussion for latter, but for now testtool is not one of the most
> important things on QA, since we can test everything manually.

We should leave the most boring tasks to automation. But please let's
never ever fall in the trap that "automated testing will detect all
regressions" again. The effort to create, maintain and analyze the
results of complex tests scripts is (imho) in no relation to the benefit.

For the start it would be very helpful to adopt the release tests, so
that they do not fail at those points, where LibO has improved
functionality.

Regards,

André
--
To unsubscribe, e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

Thorsten Behrens Thorsten Behrens
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

In reply to this post by Caio Tiago Oliveira
Caio Tiago Oliveira wrote:
> Integrating without testing could be dangerous and for a few people
> testing every branch (CWS) it would be boring. We can leave this
> discussion for latter, but for now testtool is not one of the most
> important things on QA, since we can test everything manually.
>
Oh, there's currently no work on feature branches (that's what CWS
are) slated for integration into 3.3 - we're currently in the
progress of merging fixes and features from various contributors,
that have been around for a while and are considered stable.

That will continue until end of October, and we'll then branch off
3.3 and only fix (serious) bugs there.

Cheers,

-- Thorsten


--
To unsubscribe, e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

sophi sophi
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

In reply to this post by Thorsten Behrens
Hi Thorsten,

On Thu, Oct 7, 2010 at 10:25 AM, Thorsten Behrens
<[hidden email]> wrote:

> Hi,
>
> as we start to ramp up more infrastructure, I'd like to make
> you think about what's crucially needed to do QA for LibreOffice
> 3.3.
>
> First of all, this is what we currently have:
>
>  * a LibreOffice technical list (I'd like to have devs & QA
>   discussion there):
>   http://lists.freedesktop.org/mailman/listinfo/libreoffice
>
>  * a bugtracker - bugs.freedesktop.org for the while
>   (use
>    https://bugs.freedesktop.org/buglist.cgi?query_formatspecifi
c&order
> relevance+desc&bug_status__open__&productLibreOffice&content   �
�to list all currently-filed LibreOffice bugs)

>
>  * a wiki (well, soon ;))
>
>  * the testtool (similar to OpenOffice.org)
>
> I'd prefer if we do the 3.3 release in a somewhat lightweight
> fashion, and add tools as we go (and decide that we need them) - I
> know that the OpenOffice.org QA project has things like QATrack,
> QUASTe, and TCM - but I wonder which of those pass the test of "we
> really need it, and it's worth the effort to duplicate it/set it
> up".
>
> What do you think?

Sorry to jump so late in the discussion, I've been out of connexion
most of the week.
I agree with your proposal to join developers and QA people, it's
really important to have them working together.
Concerning the VCLTestTool, the most interesting part is if you can
compare between builds and languages like it is done on QUASTe. I
should ask also if some teams use the screenshot ability for
localization.
Concerning the TCM, I've worked on specifications for a new tool that
is being develop and will be integrated to QUASTe. The current
infrastructure for TCM is far too complicated and mix administrative
tasks and user tasks. Also, we need something that deal with plain
text and not html. I volunteer to work on this if you need me and also
to produce the test cases relative to the new features.
Also, we should not miss the power of the NLC on this testing tasks,
if we organize a simple and powerful workflow with a localization
process, I'm sure we will find a huge resource with the NLC projects.

HTH
Kind regards
Sophie

>

--
To unsubscribe, e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

Goran Rakic Goran Rakic
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

У суб, 09. 10 2010. у 19:26 +0200, Sophie Gautier пише:
> I should ask also if some teams use the screenshot ability for
> localization.

we do, we do, we do! :)

It should be possible to run the VCLTestTool automatically on the
nightly builds (or whatever other frequency is set).

Thinking about community I am convinced that with a proper call for
action we can find CPU power to help the buildbot and auto testing
effort.

Goran



--
To unsubscribe, e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

Per Eriksson Per Eriksson
Reply | Threaded
Open this post in threaded view
|

Re: QA Infrastructure

In reply to this post by sophi
  Hi Sophie,

Sophie Gautier skrev 2010-10-09 19:26:
> I volunteer to work on this if you need me and also
> to produce the test cases relative to the new features.
> Also, we should not miss the power of the NLC on this testing tasks,
> if we organize a simple and powerful workflow with a localization
> process, I'm sure we will find a huge resource with the NLC projects.

I am available to help if you need any on this side? What are the ideas
so far and how can the tooling become more effective?
:-)

Kind regards

Per

--
To unsubscribe, e-mail to [hidden email]
All messages you send to this list will be publicly archived and cannot be deleted.
List archives are available at http://www.documentfoundation.org/lists/discuss/

Next » 12