Ccache depend mode

classic Classic list List threaded Threaded
4 messages Options
Luboš Luňák Luboš Luňák
Reply | Threaded
Open this post in threaded view
|

Ccache depend mode


 Hello,

 ccache 3.6 has introduced a new depend mode (CCACHE_DEPEND), in which ccache
never uses preprocessing (gcc -E), which speeds things up on ccache misses.  
Previously ccache used the output from gcc -E to find out all the headers the
source file depends on, in the depend mode it uses the output generated
by -MD/-MMD. With https://gerrit.libreoffice.org/#/c/71629/ I'd like to
enable this by default for LO, if supported by ccache
(with --enable-ccache=nodepend to disable the mode).

 The advantages:

- It's faster; especially with PCH enabled:

make slideshow.build clang +ccache+icecream (ccache depend):
non-pch nodepend:
164.42user 18.05system 0:26.82elapsed 680%CPU
non-pch depend:
144.61user 13.37system 0:23.11elapsed 683%CPU
pch nodepend:
142.33user 18.04system 0:25.90elapsed 619%CPU
pch depend:
116.64user 12.10system 0:21.51elapsed 598%CPU

 The disadvantages:

- Needs recent ccache. The advantage here is that since it's so new, if there
are problems with it, most people with not-new-enough ccache won't notice.

- There won't be any preprocessed hits. My current ccache statistics are
141256 direct hits and 7153 preprocessed hits, so this doesn't seem to matter
in practice.

- We use -MMD, which exludes system headers (or even our externals, since for
those we use -isystem too). This means that ccache could give incorrect hits
if those headers change. That may seem bad, but I think it's unlikely to
cause problems in practice, for several reasons:
 * System headers rarely change.
 * If they change, it's generally a binary compatible change.
 * If they change in an incompatible way, the sources usually need a change
too.
 * Since we use -MMD, our make dependencies ignore such changes as well.
This could be mitigated by switching to -MD, but that'd affect make
performance by blowing up the number of dependencies, which I think is not
worth it.

--
 Luboš Luňák
 [hidden email]
_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice
Michael Stahl-3 Michael Stahl-3
Reply | Threaded
Open this post in threaded view
|

Re: Ccache depend mode

On 07.05.19 11:32, Luboš Luňák wrote:

> - We use -MMD, which exludes ... headers (or even our externals, since for
> those we use -isystem too).

no, -isystem can't be used with bundled externals because it breaks
incremental builds.... with -I we get deps on the headers [to be
rewritten into deps on the UnpackedTarball target by concat-deps], and
since we don't copy the headers out of the UnpackedTarball dir
incremental builds tend to work fine, with oboslete .so.N files
littering instdir as the only problem typically.

>   * Since we use -MMD, our make dependencies ignore such changes as well.
> This could be mitigated by switching to -MD, but that'd affect make
> performance by blowing up the number of dependencies, which I think is not
> worth it.

agree; typically system header files only change in incompatible ways on
distro-upgrades so you know to "make clean" twice a year.
_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice
Rene Engelhard Rene Engelhard
Reply | Threaded
Open this post in threaded view
|

Re: Ccache depend mode

In reply to this post by Luboš Luňák
On Tue, May 07, 2019 at 11:32:12AM +0200, Luboš Luňák wrote:
> - We use -MMD, which exludes system headers (or even our externals, since for
> those we use -isystem too). This means that ccache could give incorrect hits
> if those headers change. That may seem bad, but I think it's unlikely to
> cause problems in practice, for several reasons:
>  * System headers rarely change.

They change all the time.

>  * If they change, it's generally a binary compatible change.

No? There's many versions of the same compatible header where the header
stays the same?

You seem to only think of people only developing on their stable, not
changing distro.

People often build stuff for their distros (as distro packagers, where
this stuff *does* change - and that also compatibly because said new update
affected someting else or a specific header of even the same header but
compatibly.

In development times, I upgrade my unstable chroot daily (and whatever changed
in unstable changes there, too. That includes system libs/headers and compilers
and whatever.)

Regards,

Rene
_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice
Luboš Luňák Luboš Luňák
Reply | Threaded
Open this post in threaded view
|

Re: Ccache depend mode

On Thursday 09 of May 2019, Rene Engelhard wrote:

> On Tue, May 07, 2019 at 11:32:12AM +0200, Luboš Luňák wrote:
> > - We use -MMD, which exludes system headers (or even our externals, since
> > for those we use -isystem too). This means that ccache could give
> > incorrect hits if those headers change. That may seem bad, but I think
> > it's unlikely to cause problems in practice, for several reasons:
> >  * System headers rarely change.
>
> They change all the time.
>
> >  * If they change, it's generally a binary compatible change.
>
> No? There's many versions of the same compatible header where the header
> stays the same?
>
> You seem to only think of people only developing on their stable, not
> changing distro.
>
> People often build stuff for their distros (as distro packagers, where
> this stuff *does* change - and that also compatibly because said new update
> affected someting else or a specific header of even the same header but
> compatibly.
>
> In development times, I upgrade my unstable chroot daily (and whatever
> changed in unstable changes there, too. That includes system libs/headers
> and compilers and whatever.)

 Fair enough, but does that actually change anything about what I said? Most
people do not develop on a rolling distro, so in the usual case still system
headers rarely change. And if your case is different, then the setup already
doesn't suit it perfectly. You already have missing make dependencies, and
now the same setup also needs --enable-ccache=nodepend.

--
 Luboš Luňák
 [hidden email]
_______________________________________________
LibreOffice mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/libreoffice