Thanks, everybody, for the discussion on running TeX distributed / in parallel. I am much educated about the state of the art. :-)
Summary ...
- There is plenty of optimization that normally can be done, without having to parallelize. If a ConTeXt run is taking a really long time, chances are that something is not being done according to the design.
- For most (current) purposes, documents are small enough and ConTeXt is fast enough that the effort to automate distribution of typesetting runs may not be worthwhile. On the other hand, the usage of TeX might expand to larger applications if greater throughput were available.
- However, as things stand now, one can always divide documents up by hand, typeset the parts independently, and stitch them back together using tools such as divert/undivert. One can even design a document with the spec that the canonical typesetting process is to typeset the sections independently; then the sections can never affect each other, except for explicitly added inter-section effects like page reference updates.
If you're not aware of MarkMail, it's a handy place to browse / search archives of mailing lists. This thread can be found at http://markmail.org/search/?q=ntg+context+distributed+parallel
On 12/17/2008 2:47 AM, Taco Hoekwater wrote:
> There are some interesting ideas in this discussion, but with > the current state of the code base all of this will be exceedingly > difficult (especially because of all the synchronisation issues). > > Unless someone wants to work on this idea him/herself (and that > would be great, there are not nearly enough people working on TeX > development!), you could remind me, say, two years from now?
Sure. Thank you for your interest.
I wasn't asking for someone to implement new features for this, though I would be happy to see it happen if it is worthwhile for the community.
As Dr Dobb's says, "Single core systems are history" (http://www.ddj.com/hpc-high-performance-computing/207100560). Software that can take advantage of multiple cores (or threads, or distributed nodes) will continue to scale. Of course some effort, and often some adjustment, is necessary to enable programs to effectively use parallelism.
I'll create a page at http://wiki.contextgarden.net/Parallel summarizing this discussion if that's OK.
Regards, Lars
--Huttarl 15:16, 17 December 2008 (CET)