The Tectonic Project
Envisioning a 21st-Century TeX Experience
I’m delighted and honored to be presenting today.
- Thank you!
- Much of the work on Tectonic has been “to the side” of the main TeX community
- I’d love to start changing that!
I come to TeX as a scientist, ~publisher, and developer.
I’m going to spend some time on my background to help motivate the vision behind the Tectonic project.
- Research astronomer (radio interferometry, stellar/planetary magnetism)
- Innovation Scientist of the CfA and AAS
- Director, AAS WorldWide Telescope project (“WWT”)
- Work with AAS Publishing group (ApJ, ApJL, …)
- Work with NASA ADS at CfA
- Moderator for astro-ph category on ArXiv.org (→ TeX Live, LaTeX)
- 20+ years in open-source software development
I’m preoccupied with scholarly communication.
There’s such a big gap between how we could be communicating and how we actually do.
Gap = opportunity.
In my view, that gap is almost entirely about the tools that people have available to them.
🤔 …
Many gap areas, but for me the overriding one is the creation of 21st-century documents. They are digital and interactive.
Specifically, they are HTML. This is the outcome of billions of dollars of annual investment.
HTML is used more and more widely. Like in these slides!
For a while, HTML couldn’t do precision typography.
Without that, PDF is unquestionably the best “target format” for digital technical documents. (… Right?)
For me, the release of pdf.js (July 2011) proved that this problem could be solved.
I wanted to see if I could get TeX-precise text into HTML.
That is, get equations etc. as text, not as included images.
First strategy: clean-room implementation in JavaScript (2014). “TeX: The Program is only 535 pages … how hard could it be?”
Pretty hard!
Underscored for me just how much value is embedded in the complete TeX ecosystem — in both narrow and wide senses.
Next, I looked into hacking (Xe)TeX itself …
… with the aim of prototyping and testing different approaches. XeTeX for Unicode and TrueType/OpenType font support.
The experience for a new developer looking to dive in and play with the codebase is ugly.
I concluded that I could not work with the code in the way that I wanted within the existing framework. I do not want to work without Git, GitHub, pull requests, public CI, etc.
I see reasons for why things are the way they are, but in my opinion there are tremendous opportunities that a different approach could capture.
If you can’t merge back, you’re forking, not branching.
Adopting different infrastructure means that there’s no realistic prospect for merging back into mainline (Xe)TeX( Live).
Forking is a big decision … but it can also be liberating.
Specifically: the TeX user experience (UX) often leaves something to be desired. Many aspects of this UX just can’t change and still be called “TeX”.
We see the outlines of a new effort: an experiment aiming to explore TeX for the 21st century.
The Tectonic Project is that experiment.
Launched in 2016, I like to frame the project in terms of its vision(s):
A world where …
… people can easily create digital technical documents that are beautiful and excellent
… document builds are reliable and reproducible
… TeX can go anywhere that it’s asked to
… using and learning TeX is fun, even delightful
… the Tectonic project has a nice logo 😉
Key to Tectonic’s strategy is a distinctive brand identity.
Tectonic is delivered as a self-contained executable.
Bundled traditional “engines” are xetex,
bibtex, and xdvipdfmx.
The xetex engine implementation is based on the C code
generated by web2c, with extensive manual refactoring.
To synchronize with new TeXLive releases, I have to manually import and review all of the code changes.
Outside of the “engines”, Tectonic is written in Rust.
Outside of the “engines”, Tectonic is written in Rust.
Rust is a “systems language”. Use it where you’d use C or C++. Please. It is great.
Rust’s packaging and compilation model is a great fit for a project like Tectonic.
Rust and TeX are similar languages in spirit, and Tectonic aims to emulate much of the Rust ethos:
- Ecosystem orientation — great tooling
- Welcoming atmosphere
- Experience-centered design
- Great error messages
Tectonic fetches files from “bundles” on the fly.
Implemented using a virtualized I/O subsystem.
Does not hit the network unless a new file is needed.
Bundle is basically a big tar file, identified with a hash of its contents.
https://github.com/tectonic-typesetting/tectonic-texlive-bundles
The bundle is the cornerstone of Tectonic’s approach to reproducible document builds.
Tectonic has a “document model” for describing builds.
Extremely cribbed from Rust’s Cargo.
Express build settings through configuration, not runtime options.
tectonic -X compile for “one-shot”
builds.
Current framework is sparse, but hoped to be a platform for lots of cool functionality in the future.
Early HTML work is finally landing.
The big question (to me): can we deliver precise typography?
If you can use browser-compatible fonts, CSS lets you position characters at will. But XeTeX outputs glyphs.
Tectonic deals with this by reversing the Unicode→glyph table and, when needed, emitting a “subsetted” font with custom mappings so that it can force the browser to use the specific glyph TeX wants:
Emphatically not trying to work on generic inputs, yet.
My strategy is to try to benchmark really great output, then make it “just work” more and more broadly.
“Chrome” is most of the effort:
Current chrome is called TDUX: the Tectonic Default User Experience.
I think there is an optimal layout for technical docs.
Tectonic prototyping is targeting XeTeX: The Program.
It’s the perfect test case! Long, rich internal structure, and useful.
Current work is called tt-weave.
It parses WEB code and emits it in
a prettified, C/Rust-like syntax.
It will only ever target (patched)
xetex.web as an input.
Let’s take a look!
click here for demo — opens in new window. Note that this is a 10+MB download. Keys “c”, “g” pull up modals.
I’m really excited to push this work forward.
This kind of document is really a text-heavy web application.
Lots of open research questions about how to design them. Most answers happen in HTML space, not TeX.
I’m really excited to push this work forward.
This kind of document is really a text-heavy web application.
Lots of open research questions about how to design them. Most answers happen in HTML space, not TeX.
After tt-weave comes Tectonic’s documentation.
Hopefully.
I find Divio’s “documentation system” to be compelling:
For the “Reference” component, I would love to start building a sort of “Tectonopedia”.
Tectonic needs all the help it can get!
There are so many areas where I’m an amateur at best:
- Fonts
- Non-Latin scripts
- Accessibility
- TeX internals
- Real typography
- Mobile design
- In-the-wild TeX workflows
- … and much, much more!
Thank you for your attention!
And my immense gratitude to the hundreds (thousands?) of people whose effort Tectonic is building upon. Especially Jonathan Kew, Khaled Hosny, Mark Wicks, and of course Donald Knuth, to name a few.
- Website: https://tectonic-typesetting.github.io/
- Code: https://github.com/tectonic-typesetting/
- Book: https://tectonic-typesetting.github.io/book/latest/
Peter K. G. Williams • pwilliams@cfa.harvard.edu • @pkgw • newton.cx/~peter/
HTML talk info: https://tinyurl.com/htmltalk • Design credits: Hakim El Hattab (“white” theme), Julieta Ulanovsky (Montserrat font), Christian Robertson (Roboto fonts) • Tech credits: git, reveal.js, d3.js, Firefox developer tools.