build with code coverage

classic Classic list List threaded Threaded
25 messages Options
Next » 12
John Smith John Smith
Reply | Threaded
Open this post in threaded view
|

build with code coverage

Hi,

Im trying to build libreoffice with gcov/lcov code coverage. Im
following these steps :
https://wiki.documentfoundation.org/Development/Lcov

But im getting this error: http://pastebin.com/ugnMJsyd

I tried adding  '-lgcov' for ENVLINKFLAGS and LDFLAGS, but that doesnt help.


Any thoughts and pointers appreciated.


Regards,


John Smith.
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice
Miklos Vajna-2 Miklos Vajna-2
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

Hi,

On Sun, Jul 07, 2013 at 11:00:12PM +0200, John Smith <[hidden email]> wrote:
> Im trying to build libreoffice with gcov/lcov code coverage. Im
> following these steps :
> https://wiki.documentfoundation.org/Development/Lcov
>
> But im getting this error: http://pastebin.com/ugnMJsyd
>
> I tried adding  '-lgcov' for ENVLINKFLAGS and LDFLAGS, but that doesnt help.
>
> Any thoughts and pointers appreciated.

I'm regularly building part of the code with gcov options using this
script:

http://cgit.freedesktop.org/libreoffice/contrib/dev-tools/tree/lcov/coverage.sh

So, setting gb_GCOV does the right thing here. If you want to do that
for the whole build, try using:

----
make bootstrap fetch
make -j <num> -rs -f Makefile.gbuild gb_GCOV=YES
----

instead of plain "make".

HTH,

Miklos

_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

signature.asc (205 bytes) Download Attachment
John Smith John Smith
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

Hi,


Thanks, but ....


Looking at 'solenv/gbuild/platform/com_GCC_defs.mk', setting
gb_GCOV=YES seemingly sets the same options as I do (apart from '-O0',
but that doesnt really matter all that much) :


ifeq ($(strip $(gb_GCOV)),YES)
gb_CFLAGS_COMMON += -fprofile-arcs -ftest-coverage
gb_CXXFLAGS_COMMON += -fprofile-arcs -ftest-coverage
gb_LinkTarget_LDFLAGS += -fprofile-arcs -lgcov
gb_COMPILERDEFAULTOPTFLAGS := -O0


So, not surprisingly, doing this :


gb_GCOV=YES ./configure --disable-online-update --with-system-libs
--without-doxygen --with-system-mdds=no
gb_GCOV=YES make build-nocheck


Gives me the same result and the same errors. What 'make' target does
your script run ?



Regards,


John Smith


On Mon, Jul 8, 2013 at 9:35 AM, Miklos Vajna <[hidden email]> wrote:

> Hi,
>
> On Sun, Jul 07, 2013 at 11:00:12PM +0200, John Smith <[hidden email]> wrote:
>> Im trying to build libreoffice with gcov/lcov code coverage. Im
>> following these steps :
>> https://wiki.documentfoundation.org/Development/Lcov
>>
>> But im getting this error: http://pastebin.com/ugnMJsyd
>>
>> I tried adding  '-lgcov' for ENVLINKFLAGS and LDFLAGS, but that doesnt help.
>>
>> Any thoughts and pointers appreciated.
>
> I'm regularly building part of the code with gcov options using this
> script:
>
> http://cgit.freedesktop.org/libreoffice/contrib/dev-tools/tree/lcov/coverage.sh
>
> So, setting gb_GCOV does the right thing here. If you want to do that
> for the whole build, try using:
>
> ----
> make bootstrap fetch
> make -j <num> -rs -f Makefile.gbuild gb_GCOV=YES
> ----
>
> instead of plain "make".
>
> HTH,
>
> Miklos
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice
John Smith John Smith
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

Hi,


I tried running a more verbose 'make' : http://pastebin.ca/2420752

... And I dont get it. It looks like '-fprofile-arcs' and '-lgcov'
*are* being passed correctly on the cmd line, but *still* there is the
error...

Or am I simply misinterpreting the error, and has it got noting to do
with gcov ?



Regards,


John Smith
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice
Miklos Vajna-2 Miklos Vajna-2
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

In reply to this post by John Smith
On Mon, Jul 08, 2013 at 10:02:16AM +0200, John Smith <[hidden email]> wrote:

> So, not surprisingly, doing this :
>
>
> gb_GCOV=YES ./configure --disable-online-update --with-system-libs
> --without-doxygen --with-system-mdds=no
> gb_GCOV=YES make build-nocheck
>
>
> Gives me the same result and the same errors. What 'make' target does
> your script run ?
When building, no target, i.e. "all".

Just to be sure, have you tried using your current gcov/lcov on a small
example? To know your problem is specific to LO.

_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

signature.asc (205 bytes) Download Attachment
John Smith John Smith
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

On Mon, Jul 8, 2013 at 11:07 AM, Miklos Vajna <[hidden email]> wrote:
>
> Just to be sure, have you tried using your current gcov/lcov on a small
> example? To know your problem is specific to LO.
>
Thanks for the suggestion. I just build glusterfs using "./configure
LDFLAGS='-fprofile-arcs' CFLAGS='-fprofile-arcs -ftest-coverage'", and
that works as expected.

Also, the error I get when building libreoffice doesnt occur
immediately, but happens after its been happily building for a while
without problems.



Regards,


John Smith.
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice
Michael Stahl-2 Michael Stahl-2
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

In reply to this post by John Smith
On 08/07/13 10:37, John Smith wrote:

> Hi,
>
>
> I tried running a more verbose 'make' : http://pastebin.ca/2420752
>
> .... And I dont get it. It looks like '-fprofile-arcs' and '-lgcov'
> *are* being passed correctly on the cmd line, but *still* there is the
> error...
>
> Or am I simply misinterpreting the error, and has it got noting to do
> with gcov ?

it is likely caused by:

$(eval $(call gb_Library_add_ldflags,pyuno_wrapper,-nostdlib))

see the lengthy comment above the line in Library_pyuno_wrapper.mk
for why it is needed to avoid linking in libstdc++.

although actually nowadays (c007292ec3eedcf2b1ad673308fa42aad31a7333) we
could probably remove that line, since we now use gcc not g++ to link if
the library has only C objects.

... have just pushed 6121703620ed344b181a9f07a7262a664477a209 to revert
that, should fix your problem.

_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice
bfoman bfoman
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

In reply to this post by John Smith
Hi!
Is it possible to update http://dev-builds.libreoffice.org/lcov_reports/ at least once per branch to see progress (if any) ?
Thanks!
Best regards.
John Smith John Smith
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

On Mon, Jul 8, 2013 at 4:43 PM, bfo <[hidden email]> wrote:
> Hi!
> Is it possible to update http://dev-builds.libreoffice.org/lcov_reports/ at
> least once per branch to see progress (if any) ?
> Thanks!
> Best regards.
>
>
Yes, it was my intention to run it on master (not on all the branches
though, sorry). Ill post the results in that location if/when im done.
(building libreoffice takes ~ an entire day on my test setup). Ill
make some updates to the wiki page as well. For example, I think
setting ENVLINKFLAGS may not be required any longer.

If there is enough interest, perhaps you can use the wiki howto to
create a buildbot module that can run it as often as you want ?



Regards,


John Smith
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice
John Smith John Smith
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

In reply to this post by Michael Stahl-2
On Mon, Jul 8, 2013 at 12:21 PM, Michael Stahl <[hidden email]> wrote:

> On 08/07/13 10:37, John Smith wrote:
>> Hi,
>>
>>
>> I tried running a more verbose 'make' : http://pastebin.ca/2420752
>>
>> .... And I dont get it. It looks like '-fprofile-arcs' and '-lgcov'
>> *are* being passed correctly on the cmd line, but *still* there is the
>> error...
>>
>> Or am I simply misinterpreting the error, and has it got noting to do
>> with gcov ?
>
> it is likely caused by:
>
> $(eval $(call gb_Library_add_ldflags,pyuno_wrapper,-nostdlib))
>
> see the lengthy comment above the line in Library_pyuno_wrapper.mk
> for why it is needed to avoid linking in libstdc++.
>
> although actually nowadays (c007292ec3eedcf2b1ad673308fa42aad31a7333) we
> could probably remove that line, since we now use gcc not g++ to link if
> the library has only C objects.
>
> ... have just pushed 6121703620ed344b181a9f07a7262a664477a209 to revert
> that, should fix your problem.
>

Thank you very much. Ill update the sources and re-build (although
that may take a while). Il let you know how it goes. (If this is the
cause&solution, im curious why Miklos Vajna didnt seem to run into it
?)



Regards,


John Smith
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice
Miklos Vajna-2 Miklos Vajna-2
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

On Mon, Jul 08, 2013 at 04:54:24PM +0200, John Smith <[hidden email]> wrote:
> Thank you very much. Ill update the sources and re-build (although
> that may take a while). Il let you know how it goes. (If this is the
> cause&solution, im curious why Miklos Vajna didnt seem to run into it
> ?)

As mentioned above, I just build parts of LO with gcov, which is not
related to python, hence the difference.

_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice

signature.asc (205 bytes) Download Attachment
John Smith John Smith
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

In reply to this post by Michael Stahl-2
On Mon, Jul 8, 2013 at 12:21 PM, Michael Stahl <[hidden email]> wrote:

>
> it is likely caused by:
>
> $(eval $(call gb_Library_add_ldflags,pyuno_wrapper,-nostdlib))
>
> see the lengthy comment above the line in Library_pyuno_wrapper.mk
> for why it is needed to avoid linking in libstdc++.
>
> although actually nowadays (c007292ec3eedcf2b1ad673308fa42aad31a7333) we
> could probably remove that line, since we now use gcc not g++ to link if
> the library has only C objects.
>
> ... have just pushed 6121703620ed344b181a9f07a7262a664477a209 to revert
> that, should fix your problem.
>
Thanks, that does indeed fix the problem.



Regards,


John Smith
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice
John Smith John Smith
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

Hi,


Ive updated the lcov/gcov wiki page to more accurately reflect the
current situation :
https://wiki.documentfoundation.org/Development/Lcov

I also ran an lcov report on current master, and uploaded the results here:
http://dev-builds.libreoffice.org/lcov_reports/master~2013-07-08_14.53.51/


Regards,


John Smith
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice
bfoman bfoman
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

John Smith wrote
I also ran an lcov report on current master, and uploaded the results here:
http://dev-builds.libreoffice.org/lcov_reports/master~2013-07-08_14.53.51/
H!
Thanks a lot, hope you can run it and upload at least once after branching.
Anyone can comment the results vs master~2012-12-23_14.37.59? Are those comparable?
Best regards.
John Smith John Smith
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

On Tue, Jul 9, 2013 at 9:49 PM, bfo <[hidden email]> wrote:

> John Smith wrote
>> I also ran an lcov report on current master, and uploaded the results
>> here:
>> http://dev-builds.libreoffice.org/lcov_reports/master~2013-07-08_14.53.51/
>
> H!
> Thanks a lot, hope you can run it and upload at least once after branching.
> Anyone can comment the results vs master~2012-12-23_14.37.59? Are those
> comparable?
> Best regards.
>


Hi,


Remind me later on this list after your branch, and ill see what i can do.

Comparing earlier results, the amount of code lines 'hit' by the
tests/checks seems to have increased a lot, so I guess that is a good
thing.

But if you really want comparisons to have any siginificant meaning, I
guess some serious time needs to be invested in :

1.) improving and expanding the tests.
2.) running the lcov/gcov code coverage at regular intervals.

When posting on this list, I just get the impresssion that there just
isnt enough interest at this point to realize that right now. So I
guess that right now generating the reports is primailry for my
personal educational value, which is just fine by me; im learning as I
go.


Regards,


John Smith
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice
Michael Stahl-2 Michael Stahl-2
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

On 09/07/13 22:04, John Smith wrote:

> On Tue, Jul 9, 2013 at 9:49 PM, bfo <[hidden email]> wrote:
>> John Smith wrote
>>> I also ran an lcov report on current master, and uploaded the results
>>> here:
>>> http://dev-builds.libreoffice.org/lcov_reports/master~2013-07-08_14.53.51/
>>
>> H!
>> Thanks a lot, hope you can run it and upload at least once after branching.
>> Anyone can comment the results vs master~2012-12-23_14.37.59? Are those
>> comparable?
>
> Remind me later on this list after your branch, and ill see what i can do.
>
> Comparing earlier results, the amount of code lines 'hit' by the
> tests/checks seems to have increased a lot, so I guess that is a good
> thing.
>
> But if you really want comparisons to have any siginificant meaning, I
> guess some serious time needs to be invested in :
>
> 1.) improving and expanding the tests.
> 2.) running the lcov/gcov code coverage at regular intervals.
>
> When posting on this list, I just get the impresssion that there just
> isnt enough interest at this point to realize that right now. So I
> guess that right now generating the reports is primailry for my
> personal educational value, which is just fine by me; im learning as I
> go.

i'm afraid that right now we have so many obvious large gaps in test
coverage that using test coverage tools is a bit premature.  perhaps one
run per release cycle would be informative for general trends but likely
few developers have the time to look at the result in detail anyway (i
don't).


_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice
John Smith John Smith
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

On Tue, Jul 9, 2013 at 11:06 PM, Michael Stahl <[hidden email]> wrote:

> On 09/07/13 22:04, John Smith wrote:
>> On Tue, Jul 9, 2013 at 9:49 PM, bfo <[hidden email]> wrote:
>>> John Smith wrote
>>>> I also ran an lcov report on current master, and uploaded the results
>>>> here:
>>>> http://dev-builds.libreoffice.org/lcov_reports/master~2013-07-08_14.53.51/
>>>
>>> H!
>>> Thanks a lot, hope you can run it and upload at least once after branching.
>>> Anyone can comment the results vs master~2012-12-23_14.37.59? Are those
>>> comparable?
>>
>> Remind me later on this list after your branch, and ill see what i can do.
>>
>> Comparing earlier results, the amount of code lines 'hit' by the
>> tests/checks seems to have increased a lot, so I guess that is a good
>> thing.
>>
>> But if you really want comparisons to have any siginificant meaning, I
>> guess some serious time needs to be invested in :
>>
>> 1.) improving and expanding the tests.
>> 2.) running the lcov/gcov code coverage at regular intervals.
>>
>> When posting on this list, I just get the impresssion that there just
>> isnt enough interest at this point to realize that right now. So I
>> guess that right now generating the reports is primailry for my
>> personal educational value, which is just fine by me; im learning as I
>> go.
>
> i'm afraid that right now we have so many obvious large gaps in test
> coverage that using test coverage tools is a bit premature.  perhaps one
> run per release cycle would be informative for general trends but likely
> few developers have the time to look at the result in detail anyway (i
> don't).
>
Yes, of course this is what i meant to say.
;)



Regards,

John Smith.
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice
Bjoern Michaelsen Bjoern Michaelsen
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

Hi,

On Tue, Jul 09, 2013 at 11:11:29PM +0200, John Smith wrote:
> On Tue, Jul 9, 2013 at 11:06 PM, Michael Stahl <[hidden email]> wrote:
> > i'm afraid that right now we have so many obvious large gaps in test
> > coverage that using test coverage tools is a bit premature.  perhaps one
> > run per release cycle would be informative for general trends but likely
> > few developers have the time to look at the result in detail anyway (i
> > don't).
> >
> Yes, of course this is what i meant to say.
> ;)

I just filed an EasyHack on this:

 https://bugs.freedesktop.org/show_bug.cgi?id=66750

Best,


Bjoern
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice
John Smith John Smith
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

On Tue, Jul 9, 2013 at 11:30 PM, Bjoern Michaelsen
<[hidden email]> wrote:

> Hi,
>
>
> I just filed an EasyHack on this:
>
>  https://bugs.freedesktop.org/show_bug.cgi?id=66750
>
> Best,
>
>
> Bjoern

Far Out, Man!
:)
_______________________________________________
LibreOffice mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/libreoffice
bfoman bfoman
Reply | Threaded
Open this post in threaded view
|

Re: build with code coverage

In reply to this post by Michael Stahl-2
Michael Stahl-2 wrote
[...] but likely few developers have the time to look at the result in detail anyway (i
don't).
Hi!
Well, I think that this is not very known that such reports are generated, not mentioning the purpose of them. Unfortunately I see this as a reason that only few developers deliver new or fixed code with proper tests included.
This should be a part of a code review process. QA guys would be happy...
Anyway as TDF is getting more own Tinderbox build bot hardware maybe there is an idle timeframe (weekends?) when such important reports for overall quality of the code could be generated soon after branching.
Best regards.
Next » 12