[tex4ht] debugging tex4ht bootstrap

[Karl Berry]

Hi Deimi - this is fantastic!

Unfortunately, I'm about to disappear for a week (to TUG'16; back online
next weekend, July 30-31), but I'll review it carefully/commit after I
return.  A few quick replies for now.

    Literate sources are quite complex; not so easy to grasp.

Understatement of the year :(.

    How do I test if there are no side effects in the generated output files?

This is another major unresolved problem.  My current attempts along
those lines is the diff-all target and related at the bottom of the
Makefile, which attempts to diff an installed TeX Live against the
development files generated in lit/*, while ignoring comments and more.

There are, sadly, many differences.  They all need to be looked at and
resolved.  I have hopes that these latest fixes will reduce them
dramatically.

For now, I think the best that could be done is to copy the lit/*
directory to a backup, install your changes, rerun make, and then do a
diff between the backup and results.  If you see what I mean.

    I wonder if it is not a case to put it in a mainstream, not just for
    debugging.

I want to do that.  However, as long as the failed runs were required
for some documents to make it through to the end, it was not possible.

With your fixes, maybe we could try again.  But I also fear that doing
so now will introduce the same failure into workflows in the wild, which
also depend on the temporary results from failed runs.  I don't know.

(The scripts need a lot of attention in general.  Sigh.)

    "mktex4ht.cnf (and/or some other)"
    has two different recipes.

Right, which is why I didn't introduce it as a dependency.

    One option I can thing of is combining recipes. (though i am not
    quite sure if it is right thing to do): 

It would be much nicer if we could separate out the content so that the
mutual recursion was removed.  I.e., a new file that generates
mktex4ht.cnf separately -- run that first, then cond4ht, then tex4ht.
Except that doesn't look possible.  But the current setup just seems
fundamentally wrong.

I remain unclear as to why all those different \AddFile's exist in the
first place.  What does that number mean, and how does one choose what
number to use?  Argh.

    There are a lot of derived files from tex4ht-4ht.tex.
    Probably I would set them programatically rather than manually.

Definitely agreed.  The manual stuff was only because we had to get on
with fixing the real bugs, and introducing a complete derived list
started having problems.

    tex4ht-4ht.derived: tex4ht-4ht.fls
         grep -v $* $< | awk '/OUTPUT/ {print $$2}'  > $@

I hadn't thought of using recorder output, though.  Sounds good.  I'll
look at this when I get back.

    I cannot find the version identification in this file.

You're right, there's nothing there.  Don't worry about it :).  I'll
tweak it one way or another to be consistent later.

    Perhaps you have some other TODO items or things that do not work...

The ones mentioned are the ones that have been on my mind.  But it's
easy to test: do a clean svn checkout in a new directory, type make, and
see what error messages result, or what fails to get made (correctly).

One other thing, not about a clean checkout but about a clean
environment, is that it would be wonderful if *all* derived files were
written to a different directory, not ".".  Right now, just figuring out
what is a source file and what is a derived file is far from obvious.

There is a trace of my initial attempts with the wd= variable in the
Makefile; right now, the Java files get written there but nothing else.
(The Java files are another huge problem in and of themselves, like what
is all that code doing -- ofte nothing because Java on a given system
may or not even exist -- and shouldn't we do it a better way, but never
mind ...)  I suspect that, given the intermixed input and output, this
is next to impossible, but as long as we're here ...

It's also annoying to have to generate the HTML "documentation" for the
lit sources (with magical settings, as you noted) along with creating
the necessary derived files.  Those should really be separable.  But
this is even deeper into pipedream territory ...

See you all back here in a week ... --karl