How to index a book semi-automatically

You have finished your magnum opus, and your publisher wants an index.
He refuses to pay a professional indexer, and expects you to do it.
What do you do? Save this email for that eventuality.
These notes will assume you are using Word 97. Other word processors are similar.
You can index a book the hard way, or the easier way, or the easy way.

THE HARD WAY: MARK EACH WORD MANUALLY
- i.e. go through your document and mark every word which you want indexed.
- to mark a word, highlight it and click on Insert: Index and Tables...: Mark Entry...: Mark All
- or, use Alt-Shift X, and click on Mark All
- do that for every word which you want in the index
- if you are clever you will make a macro to bypass all those clicks, but you still have to pick out all the words.

AN EASIER WAY: USE AUTO-MARK
- i.e. make a concordance of words you want indexed and use Auto-Mark
- but making a concordance is not easy. You have to collect all the words yourself.
- the Word Help files tell you how to make a concordance, but it is hard work, so...

AN EASY WAY: USE A CONCORDANCE PROGRAM AND AUTO-MARK
- a concordance program collects all the words in your book
- you delete the words you don't want indexed, and tidy it up
- then use the resultant Concordance file for Auto-Mark
- it is a bit complicated, so I have laid out step-by-step instructions


STEP-BY-STEP INSTRUCTIONS
========================

INSTALL "SIMPLE CONCORDANCE PROGRAM" (SCP)
(This is a freeware program available from various sites, including Tyndale)
- download it from www.textworld.com/scp or from the Tyndale site
at http://www.tyndalehouse.co.uk/Download/SCP32x40.zip
- double-click on it to install it.
- you might like to read the Getting Started instructions.
- It is easy to use, but you probably have never used anything like it before.
- it is a really amazing program. If Young or Stong had one of these, they might still be young and strong
- if you hate reading software manuals, follow these instructions:

MAKE TEXT-ONLY VERSIONS OF THE FILE(S) YOU WANT TO INDEX
(SCP cannot read Word documents, so you have to make .txt versions)
- load the book file(s) into Word and save as text file(s)
- to do this, click on File: Save as. Then change the 'Save as type' to 'Text only' and save

MAKE A SCP FILE
(SCP uses a special format of file, which it can create from a text file)
- start up SCP and click on File: New
- in the right-hand box, highlight the folder in which you saved your chapters
- in the left-hand box, highlight the file(s) of your book and click on 'Add selected files'
- change the Title in the top-left from Project1 to something like Chap1 (or whatever you want)
- click on Save and after much working, it will offer to save the results with a name you choose.
(the program may appear to do nothing for a very long time, so be patient)
- click on OK to return to main SCP program

MAKE A CONCORDANCE LIST IN SCP
- load the .scp file which you just made by clicking on File: Open
(it may take a while to open if it is large)
- click on the tab 'Word List'
- change the order to 'Decreasing Frequency Order'
- change the layout to One Column
- remove the tick from 'Frequencies'
- click on Word List to produce the list
- save it by clicking on File: Save: It saves in .rtf format.

EDIT THE CONCORDANCE FILE
- open the SCP concordance file in Word, by clicking on File: Open
- change 'Types of file' to 'Rich Text Format'
- find the file you made (it is probably in the SCP folder)
- double-click on the file.
- remove all the words which you don't want to include
(ie most of the words at the top of the list)

MAKE A WORD CONCORDANCE TABLE
Make a Word concordance table
- the Word concordance needs to be a two column table
- highlight all the text by pressing Control-A
- click on Table: Convert text to table..'
- accept the default (1column, separated by paragraphs) and click OK
- click the cursor just outside the right-hand edge of the table
- click on Table: Select column, then click on Table: Insert columns
- save the file

REFINE YOUR WORD CONCORDANCE TABLE
- the words searched for are in the left hand column
- in the right hand column put the entry you want in the index
- eg 'paul', 'paul's' and 'pauline' might all have the index entry 'Paul'
- and 'davies' should be Davies, W.D.
(if you want more than one index, with Modern Authors separately, see below)
- this large task is easier if you sort the table alphabetically:
- click on Table: Select Table: then on Table: Sort: OK

USE THE WORD AUTO-MARK FUNCTION
- load the whole document to be indexed into Word
- put the cursor at the end, where you want the index
- click on Insert: Index and tables...: AutoMark...:
- when it asks you to find the AutoMark file:
- change 'Types of file' to 'Rich Text Format'
- find the file you made (it is probably in the SCP folder)
- double-click on the file
- wait while your text is automatically marked up
- click again on Insert: Index and tables...: and click on OK

YOU HAVE NOW CREATED AN INDEX
No doubt it is not as you would like it. You can delete it and do it again in another style (pick one from the Formats in the Index and tables box.
You can carry on editing the document, and re-index at any time by clicking on the index and pressing F9.

HOW TO MAKE MORE THAN ONE INDEX IN WORD:
(this is normally impossible. Word can only make one index per document)
- make a copy of the folder which has your document file(s)
- put the indexing concordance file in both folders
- edit the index files so one contains all names, and one has no names
- make an index on both documents, then copy and paste the second index
- OR, make an index concordance file from your Bibliography
- to make a separate Scripture Ref index, you will have to mark up the texts manually
Note: If you are still editing the book, make sure you know which one you edit, and re-copy it into the other folder before you re-index.

Update sent out Aug.2000
Several interesting emails from scholars contained significant nuggets which should be added to the previous email about indexing.
I won't mention names, in case I miss people out.

HOW TO INDEX ON A MAC
There is a Mac concordance program called Conc available from SIL
- http://www.sil.org/computing/catalog/conc.html
Apparently this works in a similar way to SCP. There are instructions at
http://www.sil.org/computing/conc/tutorial.html

HOW TO INDEX PUBLISHER'S PROOFS
If you are not supplying Camera Ready Copy, and you have to index your publisher's proofs, you can't simply supply an index of your files, because the page numbers will be wrong.
A rough work around is to produce a copy of your files with the same pagination as the proofs. You can make the pages match, either by making page sizes which are almost exactly the same as the proofs, or by forcing a New Page where-ever the proofs have a new page. The page layout will look terrible, but it doesn't matter, because you just want to produce an index. (Use Control-Enter to force a New Page).

CONSIDER EMPLOYING A PROFESSIONAL INDEXER
The DIY indexing which I described will always be inferior to that done by a professional. A concordance file only finds the words you have used, and doesn't list the concepts. Also, you need to edit the concordance file rigorously so that you link words with the concepts they represent. So, if you can afford it, employ a professional. You can find one at:
http://www.socind.demon.co.uk/ - an on-line version of Indexers Available 2000, searchable by subject (religion and theology currently has 22 entrants)
http://www.sfep.demon.co.uk/ - an on-line version of the SFEP Directory, currently being updated to 2000/2001. The 1999/2000 printed version has numerous entries under religion and theology: there are 36 entries in bold type that indicate specialists.

WARNINGS ABOUT FOOTNOTES ON WORD 97 & 98
There is a problem with footnotes on PC Word 97 and Mac Word 98. Sometimes a footnote may be split or moved onto the next page, even when there is room on the correct page.
For the Mac problems, see http://www.macfixit.com/ultimate/Forum2/HTML/001011.html
For Camera Ready Copy, it may be safer to print the file from PC Word 95 or Mac Word 5.1.
You will have to use 'Save As..' to save it in the older format, then re-open it in the older word processor.
Apparently this has been fixed in Word 2000, but you need to turn off the check box for tools\ options\compatibility\ layout footnotes like Word 97.
OR you can try the following fix:
Set the paragraph formatting of your body text to Exact line spacing. If you currently use Single line spacing, set it to around 115% of the point size. (The actual % to use depends on the font in use).

Update added Sept 2001
Simple Concordance program for the Mac.
A program called called Conc does roughly the same job as SCP on a PC.
I haven't used it personally, but you can read about it and download it from:
http://www.indiana.edu/~letrs/help-services/QuickGuides/about-conc.html

Update added Nov 2007
I've just used these instructions to index my latest book. They still work well!