Implementing accessibility non-regression check tool

classic Classic list List threaded Threaded
31 messages Options
Next » 12
Samuel Thibault Samuel Thibault
Reply | Threaded
Open this post in threaded view
|

Implementing accessibility non-regression check tool

Hello,

This is a small mail to briefly present the work we (Hypra) will be
doing for accessibility in LibreOffice.

Basically, the idea is to design a tool which will check .ui files for
accessibility issues: missing relations between widgets and labels,
notably. The tool would just use libxml to parse the files and emit
warnings for the found issues.

Such a tool could be called by the LibreOffice build system so that
developers get the warnings along other compiler warnings, and treated
the same way. It could also be used in Continuous Integration reports.

Of course, there are a lot of existing issues, so we plan to add support
for suppression rules, so that when the tool invocation is integrated,
we also integrate an initial set of suppression rules which allows to
start with a zero-warning state, and then for a start developers will
try to stay without warning, and progressively fix existing issues and
their corresponding suppression rules.

One of the remaining questions we have (it's not blocking for our
immediate development, though) is whether this tool should be integrated
within libreoffice, or within glade. The latter would both allow more
widespread use of the tool by other projects, and make the maintenance
happen there, thus less work for LibreOffice :)

We can discuss about it at the ESC call today.

Samuel
_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice
Samuel Thibault Samuel Thibault
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

Samuel Thibault, on jeu. 18 janv. 2018 12:36:49 +0100, wrote:
> We can discuss about it at the ESC call today.

Ok, that mail didn't get to the list before the call, so it didn't get
into the agenda and I didn't manage to speak in. We can probably start
discussing what we can on this list, and discuss next week then.

Samuel
_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice
Toki Toki
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

In reply to this post by Samuel Thibault
On 01/18/2018 11:36 AM, Samuel Thibault wrote:

> Basically, the idea is to design a tool which will check .ui files for
> accessibility issues: missing relations between widgets and labels,
> notably. The tool would just use libxml to parse the files and emit
> warnings for the found issues.

Will this proposed tool mean that LibreOffice will be usable with JAWS,
despite all of the hard work on the part of the successors of Freedom
Scientific, to ensure otherwise?

jonathon


_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice

signature.asc (836 bytes) Download Attachment
Caolán McNamara Caolán McNamara
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

In reply to this post by Samuel Thibault
On Thu, 2018-01-18 at 12:36 +0100, Samuel Thibault wrote:
> Hello,
>
> This is a small mail to briefly present the work we (Hypra) will be
> doing for accessibility in LibreOffice.
>
> Basically, the idea is to design a tool which will check .ui files
> for accessibility issues: missing relations between widgets and
> labels, notably.

Sounds useful.

FWIW, there is a little script in LibreOffice as bin/lint-ui.py which
checks for various things. It doesn't do the above, but it does at
least demo some simple .ui parsing.

> One of the remaining questions we have (it's not blocking for our
> immediate development, though) is whether this tool should be
> integrated within libreoffice, or within glade. The latter would both
> allow more widespread use of the tool by other projects, and make the
> maintenance happen there, thus less work for LibreOffice :)

With glade would seem most natural, seeing as we just reuse the
gtkbuilder file format which gtk uses. But if ends up as a relatively
simple standalone script perhaps it just needs its own gitlab/github
toplevel project and LibreOffice can pull it down from there if it
wants to integrate it into our build
_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice
Samuel Thibault Samuel Thibault
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

Hello,

Caolán McNamara, on jeu. 25 janv. 2018 15:31:34 +0000, wrote:
> FWIW, there is a little script in LibreOffice as bin/lint-ui.py which
> checks for various things. It doesn't do the above, but it does at
> least demo some simple .ui parsing.

Thanks!

Is there a particular codestyle that we should follow?  I didn't find
particular codestyle in various python script in LO, probably because
the python syntax already imposes quite a lot, and thus enough to get
homoegenous code?

> > One of the remaining questions we have (it's not blocking for our
> > immediate development, though) is whether this tool should be
> > integrated within libreoffice, or within glade. The latter would both
> > allow more widespread use of the tool by other projects, and make the
> > maintenance happen there, thus less work for LibreOffice :)
>
> With glade would seem most natural, seeing as we just reuse the
> gtkbuilder file format which gtk uses. But if ends up as a relatively
> simple standalone script perhaps it just needs its own gitlab/github
> toplevel project and LibreOffice can pull it down from there if it
> wants to integrate it into our build

Ok, maybe we can start on our
https://git.hypra.fr/youpi/libreoffice-non-regressions repository for
now and see where to move it for better visibility and contributions
later.

Samuel
_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice
Miklos Vajna-4 Miklos Vajna-4
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

Hi Samuel,

On Thu, Jan 25, 2018 at 05:07:44PM +0100, Samuel Thibault <[hidden email]> wrote:
> Is there a particular codestyle that we should follow?  I didn't find
> particular codestyle in various python script in LO, probably because
> the python syntax already imposes quite a lot, and thus enough to get
> homoegenous code?

If you grep in commit messages, the pep8 tool is mentioned several
times. So if you want to follow some style guide for Python, that would
be it.

Regards,

Miklos

_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice

signature.asc (188 bytes) Download Attachment
Samuel Thibault-2 Samuel Thibault-2
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

Hello,

So there was a presentation about this tool at FOSDEM, the video
recording is already available on
https://fosdem.org/2018/schedule/event/ode_testing/
Basically, the tool (improved a bit since FOSDEM) is currently reporting
about 8000 warnings, i.e. 8 per file on average. I have attached the
evolution over time, we clearly see the migration to .ui files during
4.0 :) Looking at a few .ui files, there are some false positives, but
not so many, and they are usually based on semantic, so they wouldn't be
detectable anyway, one needs to mark them as such anyway. There are some
errors too (parsing error or missing targets), they are quite rare.

We have discussed with various people at FOSDEM about their feeling
on it, and thought how to proceed from there. Our goal is to achieve
zero-regression and fixing existing issues on the long run, while
avoiding to bother developers too hard. Our fears is that the tool might
produce too many false positives, that people need to be taught how to
fix the true positives, and that we don't want to do several a11y-fix
passes over all .ui files.

We thought about the following planning, step by step:

- Add to the build process error checking (only the hard errors such as
bogus target names). There are only a few existing issues, so we can fix
them alongside, and people won't introduce many, so making them errors
already shouldn't be bothering.

- Add to make check warning checking, one kind of warning at a time,
with suppression files alongside, so that the tool only displays "<n>
suppressed warnings" and new warnings introduced by developers from
there. These warnings would point to wiki pages explaining the ins and
outs of the issues and how to fix them. Introducing warnings one kind
of warning at a time should leave time to developers for learning the
accessibility rules progressively. It should also allow to observe how
well false positives are treated before enabling all warnings.

- When we get more and more confident that warnings are solid, we can
make them fatal (one kind at a time), to really enforce non-regression.

- At the same time, we would work on fixing issues raised by the tool on
some set of dialog boxes, to check that fixing them does provide good
accessibility, and to what extent we want to introduce more warnings to
reach good accessibility.

- At some point we'll get confident that we won't introduce other
big classes of warnings over hundreds of .ui files. That's the point
where we can say "ok, let's start fixing the existing issues over
all .ui files once for good". We can then run through .ui files one
by one, fixing the issues and removing the corresponding suppression
lines. These could be used as "easy hacks" entries, they are usually
just a few lines to fix.

The progression of all of this could be monitored with statistics
reported e.g. in the minutes of ESC calls.

What do people think about this plan?

Samuel

_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice

libreoffice.eps (32K) Download Attachment
Eike Rathke-2 Eike Rathke-2
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

Hi Samuel,

On Monday, 2018-02-12 15:30:59 +0100, Samuel Thibault wrote:

> [... a nice plan ...]

> What do people think about this plan?

Makes perfectly sense to me.

  Eike

--
LibreOffice Calc developer. Number formatter stricken i18n transpositionizer.
GPG key 0x6A6CD5B765632D3A - 2265 D7F3 A7B0 95CC 3918  630B 6A6C D5B7 6563 2D3A
Care about Free Software, support the FSFE https://fsfe.org/support/?erack

_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice

signature.asc (849 bytes) Download Attachment
Samuel Thibault-2 Samuel Thibault-2
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

Hello,

For now I have patched the build system to call the tool as the attached
patch does, does it look OK? (basically it's called along building the
image list in the UIC rule, I need to doublecheck the clean rule, but
the build rule works fine)

Samuel

_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice
Miklos Vajna-4 Miklos Vajna-4
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

Hi Samuel,

On Thu, Feb 15, 2018 at 06:58:42PM +0100, Samuel Thibault <[hidden email]> wrote:
> For now I have patched the build system to call the tool as the attached
> patch does, does it look OK? (basically it's called along building the
> image list in the UIC rule, I need to doublecheck the clean rule, but
> the build rule works fine)

Please submit patches to gerrit:
<https://wiki.documentfoundation.org/Development/gerrit>, it's much
easier to review anything there. :-)

Thanks,

Miklos

_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice

signature.asc (188 bytes) Download Attachment
Samuel Thibault-2 Samuel Thibault-2
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

Miklos Vajna, on ven. 16 févr. 2018 09:24:38 +0100, wrote:
> On Thu, Feb 15, 2018 at 06:58:42PM +0100, Samuel Thibault <[hidden email]> wrote:
> > For now I have patched the build system to call the tool as the attached
> > patch does, does it look OK? (basically it's called along building the
> > image list in the UIC rule, I need to doublecheck the clean rule, but
> > the build rule works fine)
>
> Please submit patches to gerrit:
> <https://wiki.documentfoundation.org/Development/gerrit>, it's much
> easier to review anything there. :-)

For reviews, sure, I'm here just asking whether the general approach is
fine, not an actual review of the implementation.

Samuel
_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice
sberg sberg
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

On 16.02.2018 09:55, Samuel Thibault wrote:

> Miklos Vajna, on ven. 16 févr. 2018 09:24:38 +0100, wrote:
>> On Thu, Feb 15, 2018 at 06:58:42PM +0100, Samuel Thibault <[hidden email]> wrote:
>>> For now I have patched the build system to call the tool as the attached
>>> patch does, does it look OK? (basically it's called along building the
>>> image list in the UIC rule, I need to doublecheck the clean rule, but
>>> the build rule works fine)
>>
>> Please submit patches to gerrit:
>> <https://wiki.documentfoundation.org/Development/gerrit>, it's much
>> easier to review anything there. :-)
>
> For reviews, sure, I'm here just asking whether the general approach is
> fine, not an actual review of the implementation.

...but even that can be easier to answer if people see a full, working
patch, which they can actually try out.  (It doesn't need to be
feature-complete, of course, just working well enough to demonstrate
what is going on.)
_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice
sberg sberg
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

In reply to this post by Samuel Thibault-2
On 16.02.2018 09:55, Samuel Thibault wrote:

> Miklos Vajna, on ven. 16 févr. 2018 09:24:38 +0100, wrote:
>> On Thu, Feb 15, 2018 at 06:58:42PM +0100, Samuel Thibault <[hidden email]> wrote:
>>> For now I have patched the build system to call the tool as the attached
>>> patch does, does it look OK? (basically it's called along building the
>>> image list in the UIC rule, I need to doublecheck the clean rule, but
>>> the build rule works fine)
>>
>> Please submit patches to gerrit:
>> <https://wiki.documentfoundation.org/Development/gerrit>, it's much
>> easier to review anything there. :-)
>
> For reviews, sure, I'm here just asking whether the general approach is
> fine, not an actual review of the implementation.

(Would have been useful if you'd reported back to this mail thread that
you'd filed <https://gerrit.libreoffice.org/#/c/49856/> "Integrate
initial version of gla11y tool in the build system".  The sad fact is
that there's so many gerrit changes/bugzilla issues/etc. that interested
parties may easily miss some.)
_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice
sberg sberg
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

In reply to this post by Samuel Thibault
I'm not sure basing this gla11y tool on lxml (<http://lxml.de/>) is a
good choice:  "The lxml XML toolkit is a Pythonic binding for the C
libraries libxml2 and libxslt." (<http://lxml.de/>)

Different LO builds have the choice/need to use either a system Python
or a locally-built one from external/python3.

Different LO builds have the choice/need to use either a system
libxml2/libxslt or a locally-built one from external/{libxml2,libxslt}.

Adding lxml into the mix, different LO builds will have the choice/need
to use either a system lxml or a locally-built one.  (Which
<https://gerrit.libreoffice.org/#/c/50115/3> "Build external lxml if not
provided by system" is going to add as external/lxml.)

That means external/lxml will need to:

* build against and run with either the system Python or the
locally-built one from external/python3,

* build against and run with either the system libxml2/libxslt or the
locally-built one from external/{libxml2,libxslt}.

That's four different ways how to build external/lxml, and slightly more
different ways (using the system lxml versus external/lxml) how to run
the gla11y tool in solenv/gbuild/UIConfig.mk.  And getting all those
ways to work, on the different platforms, will be hell.

Isn't there another option to make that gla11y tool process XML data,
one that better matches LO's needs?
_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice
Samuel Thibault-2 Samuel Thibault-2
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

Stephan Bergmann, on jeu. 22 févr. 2018 15:49:40 +0100, wrote:

> That means external/lxml will need to:
>
> * build against and run with either the system Python or the locally-built
> one from external/python3,
>
> * build against and run with either the system libxml2/libxslt or the
> locally-built one from external/{libxml2,libxslt}.
>
> That's four different ways how to build external/lxml, and slightly more
> different ways (using the system lxml versus external/lxml) how to run the
> gla11y tool in solenv/gbuild/UIConfig.mk.  And getting all those ways to
> work, on the different platforms, will be hell.

Well, python, libxml2 and libxslt are very-well-established projects
with a huge lot of reverse dependencies, and thus they have to take
backward compatibility into extremely careful consideration. I thus
don't fear too much problems building lxml against various versions of
them.

(and the current building issue reported in gerrit 50115 is merely that
the current LO python3 module doesn't install any header file or such)

> Isn't there another option to make that gla11y tool process XML data, one
> that better matches LO's needs?

Well, we can reimplement the world for sure.

More seriously, we can of course at least depend only on libxml2. Not
depending on a higher-level library, however, means to have to
reimplement all the tree browsing functions needed to reach the pieces
of .ui files.  And avoiding python means, writing all of this in C?
That's neither fun nor easy to extend for further .ui checking.  The
eventual script we plan to integrate is only about 300 lines of python.
I'm scared by the maintenance of the equivalent without using python and
lxml more than maintenance of building lxml.

Samuel
_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice
sberg sberg
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

On 22.02.2018 16:13, Samuel Thibault wrote:
> Stephan Bergmann, on jeu. 22 févr. 2018 15:49:40 +0100, wrote:
>> That's four different ways how to build external/lxml, and slightly more
>> different ways (using the system lxml versus external/lxml) how to run the
>> gla11y tool in solenv/gbuild/UIConfig.mk.  And getting all those ways to
>> work, on the different platforms, will be hell.
[...]
> (and the current building issue reported in gerrit 50115 is merely that
> the current LO python3 module doesn't install any header file or such)

...and that's only the start, I guess. :)  I have e.g. no idea
whether/how it's possible to tell that lxml setup.py which
libxml2/libxslt to use.  (And whether e.g. the
external/{libxml2,libxslt} case will require some extra scaffolding at
the calling site of gla11y in UIConfig.mk.)

Just be warned. ;)

>> Isn't there another option to make that gla11y tool process XML data, one
>> that better matches LO's needs?
>
> Well, we can reimplement the world for sure.
>
> More seriously, we can of course at least depend only on libxml2. Not
> depending on a higher-level library, however, means to have to
> reimplement all the tree browsing functions needed to reach the pieces
> of .ui files.  And avoiding python means, writing all of this in C?
> That's neither fun nor easy to extend for further .ui checking.  The
> eventual script we plan to integrate is only about 300 lines of python.
> I'm scared by the maintenance of the equivalent without using python and
> lxml more than maintenance of building lxml.

I was more hoping that there might be an established plain Python option
for XML processing?

_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice
Michael Stahl-2 Michael Stahl-2
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

On 22.02.2018 16:36, Stephan Bergmann wrote:
> I was more hoping that there might be an established plain Python option
> for XML processing?

FWIW, there is at least xml.etree.ElementTree and xml.dom.minidom in the
CPython bundled libs.

https://docs.python.org/3/library/xml.etree.elementtree.html
https://docs.python.org/3/library/xml.dom.minidom.html

_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice
Samuel Thibault-2 Samuel Thibault-2
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

Michael Stahl, on jeu. 22 févr. 2018 16:59:21 +0100, wrote:
> On 22.02.2018 16:36, Stephan Bergmann wrote:
> > I was more hoping that there might be an established plain Python option
> > for XML processing?
>
> FWIW, there is at least xml.etree.ElementTree and xml.dom.minidom in the
> CPython bundled libs.
>
> https://docs.python.org/3/library/xml.etree.elementtree.html
> https://docs.python.org/3/library/xml.dom.minidom.html

An issue with these is that they do not keep source line numbers of the
nodes.  That's really concerning for providing useful information to
the programmer.  We will be bothering them with new warnings that they
will have to learn about ; if we don't provide with line numbers, they
will get rightfully angry :) We could fallback to only providing the
class and id of the suspected widget. In some cases we have seen no id
defined, and thus no pointer to give to the programmer.

Samuel
_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice
Samuel Thibault-2 Samuel Thibault-2
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

In reply to this post by sberg
Stephan Bergmann, on jeu. 22 févr. 2018 16:36:44 +0100, wrote:
>  I have e.g. no idea whether/how
> it's possible to tell that lxml setup.py which libxml2/libxslt to use.

That was indeed a missing part in my patch, it's a mere setup.py option
which does work fine.

> (And whether e.g. the external/{libxml2,libxslt} case will require
> some extra scaffolding at the calling site of gla11y in UIConfig.mk.)

This is already handled with LD_LIBRARY_PATH on programs/

> Just be warned. ;)

Sure, I'm not saying that the plumbing is completely trivial :)

Samuel
_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice
Khaled Hosny Khaled Hosny
Reply | Threaded
Open this post in threaded view
|

Re: Implementing accessibility non-regression check tool

In reply to this post by Samuel Thibault-2
CONTENTS DELETED
The author has deleted this message.
Next » 12