[XeTeX] XeTeX interchar token mechanism vs. Mapping

[Aleksandar Zec]

Hello again,

Again, many thanks to Arthur for his explanation regarding XeTeX
interchar token mechanism.

The problem: XeTeX seems to take into account \XeTeXcharclass of a
particular character *before* Mapping is done. Please take a look at
the following example: the word "baino" should appear equally
non-hyphenated in both cases, however it doesn't, depending on whether
U+0301 is inserted directly or via Mapping.

To get it correctly hyphenated, you have to set \XeTeXcharclass for
'/', which is not as it should be.

------- test.map -------
LHSName "Testing XeTeX interchar token mechanism"
RHSName "UNICODE"
pass(Unicode)
U+002F U+0060 U+0060 <> U+0301 ; i.e. /`` -> combining acute
-------------------------

------- test2.tex -------
\documentclass{article}
\usepackage{polyglossia}
\setdefaultlanguage{greek}
\setmainfont[Mapping=test]{DejaVu Sans}

\XeTeXinterchartokenstate=1
\newXeTeXintercharclass\combiningchar
\XeTeXcharclass"0301=\combiningchar
\XeTeXinterchartoks 0 \combiningchar = {\penalty10000}

\begin{document}
  \setlength{\fboxrule}{0.2pt}
  \setlength{\fboxsep}{0.0cm}
  \fbox{
  \begin{minipage}{3cm}
    \hspace*{2.33cm}βαι/``νο κτλ.\\ % incorrect hyphenation
    \hspace*{2.33cm}βαι\char"0301 νο κτλ. % correct hyphenation
  \end{minipage}
  }
\end{document}
-------------------------

Thanks,
Alex