[tex-k] Speed of kpsewhich when querying the repertories of the TeX installation

[jfbu]

Hi 

this is interesting thanks, but let me again explicitely state that my only issue
is with querying what TEXMFHOME, TEXMFLOCAL, etc.. are

perhaps this is a no-issue for most people because those are already
set up as environment variables ?

as I will comment more in another mail, the issue is acute with Emacs/AUCTeX 11.88
which (until now where the maintainers on my prompting are starting to issue patches)
queries 9 times kpsewhich during launch. On a fast Mac OS X with TL2014 this meant
a 5 seconds loading time of Emacs /AUCTeX, entirely caused by these 9 calls to 
kpsewhich. Current development branch has already divided by 2 this to reduce
to 5 calls caching the earlier results. I am trying to convince the developers to 
arrange things to make only 1 call to kpsewhich simultaneously for all variables, as
is possible.

best

Jean-François


Le 1 mai 2015 à 01:21, Fabrice Popineau <fabrice.popineau at supelec.fr> a écrit :

> There is a way to speed up things.
> IIRC most of the time in a kpsewhich call is spent parsing texmf.cnf and reading ls-R files.
> Many years ago, I added some code to kpathsea so that internal hash tables be shared among process
> to speed up things when tex calls metapost which calls tex etc.
> Next Karel Skoupy wrote a kpathsea server exhibiting the same kind of speedup, but surely in a more flexible way.
> Maybe some of these ideas could be revived.
> 
> Another (simpler) option would be to allow kpsewhich to process several requests, if that can help Jean-François.
> 
> Fabrice
> 
> 2015-05-01 1:05 GMT+02:00 Norbert Preining <preining at logic.at>:
>>> If you run kpsewhich --debug=-1 --expand-path=\$TEXMFHOME,
>>> you will see what it's doing.
>> 
>> I am sure kpsewhich does a lot, but does one need to do such a lot
>> such to tell where is TEXMFLOCAL and TEXMFHOME and TEXMFDIST ?
> 
> Yes, one has to. Otherwise 20 years of expectation would be broken.
> 
> People regularly redefine TEXMFHOME and TEXMFLOCAL, and that might even happen on multiple levels (distribution makes first change, system admin another).
> 
> So besides:
> * reading the main texmf.cnf
> * searching for other texmf.cnf
> * evaluating their contents
> there is only hard coded path, and that is a no-go.
> 
> 
> Let us go back to your Makefile: how many var-values do younormally need? 2? 3? Evaluating them once and saving costs 300ms.
> 
> Evaluating a complex Makefile with rules is probably more time consuming.
> 
> We do this caching of vars in many of our scripts, and that is definitely not the bottle neck.
> 
> I would invest time in other, that is to say possible, optimizations.
> 
> Norbert
> ------------------------------------------------------------------------
> PREINING, Norbert                              http://www.preining.info
> JAIST, Japan                                 TeX Live & Debian Developer
> GPG: 0x860CDC13   fp: F7D8 A928 26E3 16A1 9FA0  ACF0 6CAC A448 860C DC13
> ------------------------------------------------------------------------
> 
> On May 1, 2015, at 6:51 AM, jfbu <jfbu at free.fr> wrote:
> 
>> 
>> Le 30 avr. 2015 à 23:42, Karl Berry <karl at freefriends.org> a écrit :
>> 
>>>   I may write a Makefile which will act after querying
>>>   the repertories locations. But one tenth of a second
>>>   for each kpsewhich call is a lot.
>>> 
>>> Save the result instead of making the same call thousands of times.  I
>>> don't agree that 100 ms is such a horrible time to do all those
>>> filesystem operations.  In fact, it seems rather quick to me.
>>> 
>>>   Could it be imagined to have some kpsewhichdir utility
>>> 
>>> 1) I fail to see how making a utility restricted to a single variable
>>> could speed anything up.  
>> 
>> 
>> I probably did not express myself precisely. I am not targeting
>> a single variable; I am targeting the variables describing the main
>> repertories of the TeXLive installation
>> 
>> 
>> 
>>> Surely what takes the time is looking for the
>>> config files in all the possible directories, and (more so) reading the
>>> ls-R file.  
>> one might need to do things to find packagefoo.sty via the 
>> parsing of the ls-R files or system calls to ls, but why would these things
>> be needed for the main installation repertories ???
>> 
>> Perhaps many distributions set up environment variables thus
>> the speed penalty is not obvious to these users.
>> 
>> 
>>> 
>>> 2) Regardless, the idea of a utility dedicated to a single variable
>>> sounds wrong in principle to me and I don't think it should be done.
>>> 
>>>   the environment variable $TEXMFHOME 
>>> 
>>> So far as I can see, if the envvar is set, its value is already (has
>>> always been) returned "instantly".  The lookups are only done if they
>>> need to be.
>>> 
>>> karl
>>> 
>> 
>> 
>> I have none such environment variable and do not see why I should
>> set some
>> 
>> Jean-François
> 
> 
> 
> -- 
> Fabrice Popineau
> -----------------------------
> SUPELEC
> Département Informatique
> 3, rue Joliot Curie
> 91192 Gif/Yvette Cedex
> Tel direct : +33 (0) 169851950
> Standard : +33 (0) 169851212
> ------------------------------
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://tug.org/pipermail/tex-k/attachments/20150501/91b50573/attachment.html>