[tex-k] The nameless control sequence token (Was: Bug-report for the TeXbook: Not all non-primitive control-sequences are defined, ultimately, in terms of the primitive ones.)

[ud.usenetcorrespondence at web.de]

Doug McKenna wrote:

> I agree I was missing something, and as Heiko corrected, whatever is between \csname and \endcsname, even if empty, is equivalent to a \relax.
>
> But ... TeX's source code, contains the following:
>
> @<Basic printing...@>=
> procedure print_cs(@!p:integer); {prints a purported control sequence}
> begin if p<hash_base then {single character}
> if p>=single_base then
> if p=null_cs then
> begin print_esc("csname"); print_esc("endcsname"); print_char(" ");
> ...
>
> and similar tests elsewhere.
>
> In other words, the control sequence |null_cs| with no characters to its name is always output to the world as the two primitive CSes, "\csname\endcsname". So I guess the pertinent question is, can |null_cs|'s meaning be redefined? If not, isn't it "a primitive" CS?

Yes, it can be (re)defined.
By default it is not defined.
If via \let the meaning of a primitive control sequence is assigned, e.g.,
\expandafter\let\csname\endcsname=\hbox
, then it can be considered a "primitive" control sequence.

Let's summarize:

There are two ways known to me of obtaining the nameless control sequence token:

1. Having applied a single expansion step to \csname\endcsname .
   ( A side-effect of applying \csname...\endcsname is that control
     sequence tokens created this way - regardless the value of the
     \globaldefs-parameter - within the current scope get the same
     meaning as the \relax-primitive in case they are undefined at
     the time of creating them.
     You can test the "restrictedness" of \csname's "side effect
     assignment" to the current scope:

      \errorstopmode
      \show\undefined
      \show\UNDEFINED
      \begingroup
      \globaldefs=1
      % This assignment is global:
      \let\UNDEFINED=\relax
      % The "side effect assignment" of \csname is restricted to the
      % current scope despite \globaldefs having a positive value:
      \csname undefined\endcsname
      \show\undefined
      \show\UNDEFINED
      \endgroup
      \show\undefined
      \show\UNDEFINED
      \bye

   )

2. Assigning \endlinechar a value which does not denote the number
   of the code point of a character in the TeX engine's internal
   character encoding scheme, e.g., a negative value, and ending
   a line of .tex-input with a backslash/with a character whose
   category code is 0(escape).

The result of applying \string to the nameless control sequence token
is the same as if you did: \string\csname\string\endcsname
(The result of applying \string in turn depends on the value of
\escapechar.)

Applying \string to the nameless control sequence token yields the
same result as applying \string to the control word token which
you get by expanding \csname csname\string\endcsname\endcsname .
However, the nameless control sequence token and that
control word token are different tokens.

A meaning can be assigned to the nameless control sequence token via
\def / \let etc as usual.

(Unexpanded) Writing the nameless control sequence token to external file
is like writing the result of applying \string and a trailing space
character. With e-TeX-extensions you can use \detokenize for testing this.



% This example requires e-TeX extensions and a console
% where you see messages delivered by TeX and hit return
% for having TeX continue.
\errorstopmode
\newlinechar=`\^^J%
\message{%
  By default the nameless control sequence token is^^J%
  undefined - let's display the meaning of the^^J%
  nameless control sequence token, using way 2 for^^J%
  obtaining it:^^J%
}%
\begingroup
\endlinechar=-1 %
\message{\meaning\
}\immediate\read -1 to\scratchmacro
\endgroup
\message{%
  Now the nameless control sequence token gets defined,^^J%
  using way 1 for obtaining it.^^J%
}%
\expandafter\def\csname\endcsname{nameless control sequence}%
\message{%
  Let's display the meaning of the nameless^^J%
  control sequence token, using  way 2 for^^J%
  obtaining it:^^J%
}%
\begingroup
\endlinechar=-1 %
\message{\meaning\
}\immediate\read -1 to\scratchmacro
\endgroup
\message{^^J%
  Let's define another control sequence token.^^J%
}%
\expandafter\def\csname csname\string\endcsname\endcsname{%
  some other control sequence%
}%
\message{%
  Let's display the meaning of the other^^J%
  control sequence token:^^J%
}%
\message{% 
  \expandafter\meaning\csname csname\string\endcsname\endcsname
}\immediate\read -1 to\scratchmacro
\message{%
  Let's display the meaning of the nameless^^J%
  control sequence token, using  way 1 for^^J%
  obtaining it:^^J%
}%
\message{\expandafter\meaning\csname\endcsname
}\immediate\read -1 to\scratchmacro
\message{^^J%
  Let's display some stringifications:^^J%
}%
\message{%
  (\string\csname\string\endcsname)^^J%
}%
\message{%
  (\expandafter\string\csname\endcsname)^^J%
}%
\message{%
  (\expandafter\string\csname csname\string\endcsname\endcsname)^^J%
}%
\begingroup
\endlinechar=-1 %
\message{(\string\
  )%
}%
\endgroup
\immediate\read -1 to\scratchmacro
\message{^^J%
  Let's test unexpanded-writing by applying \string\detokenize:^^J%
}%
\message{%
  (\detokenize\expandafter{\csname\endcsname})^^J%
}%
\message{%
  (\detokenize\expandafter{\csname csname\string\endcsname\endcsname})^^J%
}%
\begingroup
\endlinechar=-1 %
\message{(\detokenize{\
  })^^J%
}%
\endgroup
\bye




Sincerely

Ulrich