Shared publicly  - 
 
Textbooks to Hypertext. I  noticed (only) now that PlanetMath has wiki-fied and incorporated the entire HoTT book:

  http://planetmath.org/node/87534

That’s great, that they do this. This is the grand vision, that one day all textbook-like materal will be available in hypertext form. That’s what the visionaries of hypertext had thought it should be like, long before the internet actually came into being.

I have added the link to the body of nLab hypertext

 http://ncatlab.org/nlab/show/Homotopy+Type+Theory+--+Univalent+Foundations+of+Mathematics
11
3
Urs Schreiber's profile photoGershom B's profile photoRoss Schiff's profile photoDavid Roberts's profile photo
24 comments
 
This has been a project of +Raymond Puzio, who is a regular at our NYC HoTT reading group. He has been encouraging us to use it to collaborate as part of our discussions, but this far we haven't taken full advantage.
 
Thanks for the information. I find this very interesting. Can you tell me how the conversion is done, I mean from the book text to the hyperlinked PlanetMath pages? I am aware that the LaTeX source of the HoTT book is publically available, but it still takes some processing to get from there to a Wiki-fied version. I suppose  Raymond Puzio uses some automatic script for this? I'd be interested in knowing more about this.
 
It would be interesting to see how easy it is to port changes in this wiki back into the book git repository.

+Josef Urban explained to me that "compared nlab/Instiki Planetmath has a heuristic autolinker (which may obviously get things wrong sometimes). Also latexml might be a bit more mature than itex2MML." I wonder what +Andrew Stacey thinks about this.
 
Now if only they could make chapter 10 come after chapter 9.
 
Whoa! I knew the CC licence is good for something, but it's still see it working in action.
 
+Bas Spitters LaTeXML and itex2MML have different aims and so are not comparable.  itex2MML is definitely mature.  It deals with all the maths-to-MathML conversions on the nLab, the nForum, and the nCafé without batting an eyelid.

Of course, if anyone wants to read the HoTT version on the web or as an eBook, they can always use my fork as a starting point.  I haven't done anything on it for a while because there didn't seem to be any interest, beyond the odd call of "Do this for me" which I never find all that motivating.  In particular, none of the HoTT authors seemed all that bothered about it.

The same fork could be used to produce a version suitable for direct import to the nLab (or other instiki wiki), and +Urs Schreiber  ought to remember that I have a script for auto-linking to nLab pages.  So in principle it would be straightforward to get a preliminary version in the nLab (or an nLab-web).  It would need polishing, of course, but so would any conversion.

The other thing worth pointing out - indeed, shouting out - is that the planetmath page is using MathML.  This is brilliant.
 
+Andrew Stacey I was mainly wondering about the difference in technology. Thanks for the explanation!
 
+Bas Spitters In that case, let me add a little more detail.

LaTeXML is a program for entire conversion of LaTeX documents to XML, which can then be further converted to HTML or XHTML, possibly including MathML.  It is written in Perl.  As it tries to faithfully mimic TeX (which, I should add, is impossible), it is not suitable for running in a "live" environment and should only be used for one-shot conversions.

itex2MML is a program for converting a certain concise input syntax to MathML.  The input syntax is heavily influenced by LaTeX maths (more specifically, AMSLaTeX maths) but is not LaTeX.  In particular, \newcommand is not allowed.  It is written in C.  Because its input is precisely defined, it is suitable for running in a "live" environment.  Therefore, it can be used in a CMS (content management system) where the input format is not an XML variant, such as the nLab or nForum.

That is to say, if you want to do an initial conversion of a LaTeX document to some XML-based format, from which you will then go on to produce a nice webpage, use LaTeXML.  If you want to have a system where pages/posts can be easily revised afterwards, so with an input format a little more user-friendly than XML, then use a system with a fast concise converter like itex2MML.  So different programs for different purposes.

(I'd also like to promote my own project, which is a TeX-based converter, and it is that which I've used to produce the initial eBook version of HoTT.  Of the two, it's more in line with LaTeXML.)
 
+Andrew Stacey if I know one person who can in principle do a preliminary ascent of some mountain and then run into another person who just climbed it, then it seems excusable to ask the second how he did it.
And I would still like to know! How automatic it was, which lessons were learned. 

It would be excellent if the HoTT book were wiki-fied and incorporated into the nLab, too. The nLab has plenty of pages that would usefully cross-link with it, many more than PlanetMath.
 
I emailed Ray to ask him to chime in on this thread. Hopefully he'll swing by soon.
 
this is great! it looks as thought solutions to the exercises are also being done there. 
 
> That’s great, that they do this. This is the grand vision, that one day
> all textbook-like material will be available in hypertext form. That’s what
> the visionaries of hypertext had thought it should be like, long before the
> internet actually came into being.

We share you vision at PlanetMath and have been working on it for nearly
a decade now.  You can get some idea of our plans and progress from the following document:

http://metameso.org/~joe/docs/previews.pdf

Joe and I have been inspired by Ted Nelson and other pioneers of hypertext
and much of what we are working on has been informed by their ideas.

> Can you tell me how the conversion is done, I mean from the book text to
> the hyperlinked PlanetMath pages?

The process went something as follows:

* Use csplit to cut up the chapters into individual files, one per section.

* Use a custom script to split up the preamble into individual preambles for
  each section and exercise.

* Upload these files en masse to PlanetMath.

* Use the collections facility of Planetary to organize these pieces into a
coherent whole.

The NNexus component of the Planetary system automatically generated the links.

As it turns out, because the system is under development, things did not
work all that smoothly and a fair amount of elbow grease and
work-arounds were necessarry.  However, one of our development goals
for this year is to work on these shortcomings and get things to the point
where one can take the TeX source for a book and upload it to PlanetMath
automatically.  Once this is in place, we intend to reupload the HoTT book in an improved version which will be easy to keep synchronized with the master version in the Git repository.  In addition, we plan to upload mathematical classics from Project Gutenberg and various free textbooks then.

> As it tries to faithfully mimic TeX (which, I should add, is impossible), it
> is not suitable for running in a "live" environment and should only be
> used for one-shot  conversions.

The latter assertion is not correct.  PlanetMath currently is powered by Planetary, a program built atop the Drupal CMS which accepts input at LaTeX (not XML) and calls LaTeXML to generate displayed output.  This is
 a live environment in which one can edit pages in real time just as one would on a wiki or a blog.

> The nLab has plenty of pages that would usefully cross-link with it,
> many more than PlanetMath.

One of the things we have been thinking of doing is automatically generating links to nLab pages from PlanetMath.  To do that, we would want to know what is the proper way to obtain the requisite metadata from nLab.  To find that out as well as to discuss common interests and opportunities for mutually beneficial collaboration, were were
thinking of contacting you at some point.  Now that this discussion is happening, maybe this is as good as any such opportunity to introduce ourselves and plan to meet.  If any of you participating in this discussion thread is available this Saturday at 1:30 p.m. Eastern time (06:30:00 p.m. GMT), you are welcome to join us at our weekly PlanetMath conference call:

http://planetmath.org/node/88020
 
+Raymond Puzio , thanks a lot for dropping by and for providing this information!

That's nice to see some of the names of people behind the PM project and what their vision is. I suppose I could have found that by diging around more on the web, but it wasn't clear to me.

Yes, it seems quite natural that PlanetMath and the nLab should interact a bit more, there might be some good synergy effects to be found here. 

Concerning the conference call: I doubt that I'll be able make it, but if there is some default text-message venue to interact with the PlanetMath developers, that would be useful. 

For instance +Michael Shulman noticed...

(on the HoTT list here

https://groups.google.com/forum/#!topic/homotopytypetheory/tZMQEa7y35k

and on the nForum here

http://nforum.mathforge.org/discussion/5025/homotopy-type-theory-univalent-foundations-of-mathematics/?Focus=44415#Comment_44415 )

...some links in the PM version of the HoTT book which the automated system didn't get right.

Is there some default place where one can post comments like this and/or questions on PlanetMath entries? 
 
+Raymond Puzio Notice that I said "not suitable" and "should not", which are statements of opinion and therefore not assertions.  My reasons for holding those opinions are probably not worth examining here being more of a philosophical bent (well, with a mild technical part).  I would, though, not recommend using something like LaTeXML on a live site to anyone who didn't know what they were doing.  Clearly you do.  Most, I think, don't.

(At the risk of being a bit cheeky, what happens if I do `\def\a{\a}\a` on PlanetMath?)

Anyway, I don't want to argue.  I think it is great that PlanetMath is emitting MathML and can input lots of different formats.  I'm all for getting more maths in accessible ways on the web, however it is generated and however it is converted.

It also sounds like a great idea to get more interaction between the nSuite and PlanetMath.  I'm also not able to make that time.  However, if you'd like to join us on the nForum and explain what would be needed, we could discuss it and see if we can get something working.
 
+ Bas Spitters said

> It would be interesting to see how easy it is to port changes in this wiki
> back into  the book git repository.

A few years back, one of the things we did on PlanetMath was to take content from PlanetMath and combine it to make a hefty tome known as the "Free Encyclopaedia of Mathematics".  We haven't worked with it in a while, but, when we dust off the old code and adapt it to work with our new platform, it could just as readily be used to collect the pieces of the HoTT book from the PM site back into a single TeX file, which could then be posted to GitHub or wherever.

If one's goal were simply to port changes to this particular book back to the repository, however, ther would be a simpler way.  The way that the book repository is organized is that the TeX source for each chapter comes in a separate file whilst, on PlanetMath, each section is stored as an individual file.  Thus, all that needs to be done is to concatenate these small files into a larger file, name the result appropriately, and commit it to the repository.

+Steve Awodey said

> it looks as thought solutions to the exercises are also being done there. 

In the new version of the software running PlanetMath, one of the features added was a problem/solution section and one of the uses to which it has been put is the exercises in the HoTT book.  So far, all the exercises from the book have been posted and a handful of them have solutions.  As we discuss the exercises in the "recitation sections" of our HoTT NYC seminar and get solutions written up, they will appear on PlanetMath.  Hopefully,
during the course of this year, we will thus provide solutions to most of the remaining problems.

+ Urs Schreiber said

> if I know one person who can in principle do a preliminary ascent of
> some mountain and then run into another person who just climbed it,
> then it seems excusable to ask the second how he did it.

Carrying this imagery a step further, I would consider it a favorable outcome of such a situation if the three mountaineers were come together and combine their expertise to lay plans for a road across the mountain which the neighboring townspeople could then build and use.
 
+Andrew Stacey Sorry, I agree it would be interesting to have the HoTT book on the nlab! When I saw your experiments, I thought you were mainly looking at e-books.
 
+Bas Spitters My system was first devised to make it easier to write nLab pages. Ebooks were an afterthought.

+Raymond Puzio Sounds a good plan. I'll bring a shovel. I'm afraid I tend to be hard to pin down on times to "chat", but I know a good forum for asynchronous discussion ...
 
Hi all, glad to see this thread!

After this awesome but largely by-hand prototyping, Ray and I aim to get the book upload process fully automated sometime this year.  My hope is that PlanetMath can largely replace expensive paper textbooks.

... Wouldn't it be great if we could provide a web-viewable front-end that could be used during a book authoring process that takes place on Github (for example)?

The Saturday call at 1:30 p.m. Eastern time / 6:30p.m. GMT happens weekly, like an office hour, so do feel to drop by any week if you'd like!  I'm the coordinator for those meetings, so if you'd like to join, ping me, holtzermann17 on Skype and Gmail...

I'll likely want to join any other one-off meetings on topics of common interest (of which there are many discussed here).  BTW, here's a grant proposal on the theme of using PlanetMath as a book production platform https://meta.wikimedia.org/wiki/Grants:IEG/PlanetMath_Books_Project - it wasn't funded but could potentially be recycled into something new.
 
That's an excellent vision. I sure wish you succeed.

By the way, we once thought about doing this not for textbooks, but for research articles, that experiment is called "Publications of the nLab":

  http://ncatlab.org/publications/published/HomePage

Unfortunately is has become largely dormant.
 
BTW +Urs Schreiber, Ray mentioned that you found some problems w/ our autolinker.  Some issues with accuracy are currently being resolved, but if you'd like to report a couple representative problems, the best place to do that is here: https://github.com/dginev/nnexus/issues  (And that's likely to be useful information for us.)  Thanks!
Add a comment...