17 December 2008

New Essential Research Tools

The key to research is taking notes and being able to find them again.
This used to be done on 5x3 library cards. Now it is best done on the web.
A former flatmate kept his PhD notes in two large trash sacks in a wardrobe.
To find a reference, he'd pour them on the floor, rummage, then stuff them back.
I don't think he ever completed. My system was better - I used a cardboard box.

With computers and the internet we can do much better.

Our aim should be maximum time for reading, thinking and creative writing,
with automatic facilities for searching, creating bibliographies and backups,
and minimum time for writing and filing notes on what you've read.
Utopia? Not any more.

With the following free software you can:
* access your work on any computer, and write it at the same time as a colleague
* never lose your work - automatically save to the internet every few seconds
* never ruin your work - previous versions of a document are always available
* add library catalog entries to your bibliography automatically, in your chosen style
* attach notes to a book or article which you can search and find later
* copy web pages or articles or documents which you can search later
* search videos from many sources (like YouTube) and save them (increasingly important!)
* copy pages from online books and save them as searchable documents
* save photocopies online and search them as though they were text documents

1) Zotero: Bibliography creation, note storage, document search, and more.
2) Google Docs: Word processing on the web with automatic archive and backup.
3) Evernote: Store photocopies and online books as searchable text.
4) FLV Converter: Search multiple sites for videos and save a copy.

1) Zotero: Bibliography creation, note storage, document search, and more.

Bibliography creation software has been around for some time in many forms,
eg. Endnote, Endnote Web, BibTex, Bookends and many others - and now Zotero.
They create footnotes and bibliography, in the style required by a publisher,
from a database of references; and they help you build up that database;
ie it copies a reference and turns it into any format you need.

Zotero is free, and in my opinion better - and perhaps Endnote agrees,
because they are suing Zotero for enabling Endnote users to migrate to Zotero.

What Zotero does for you:
* fills in a Bibliography database for you directly from a web library catalogue
- eg from TynCat.com or Library of Congress or even Amazon
* saves notes you write about a book, or other independent notes
* links to or saves copies of text documents, PDFs, and webpages
* organises references and documents by tags and collections
- things can go in more than one collection or have more than one tag
* exports references in over 500 styles of different publishers & journals
* also finds documents and references by searching for words used anywhere

In version 1.5 (expected soon) you will be able to sync your data online,
so your research material and references will be available where-ever you are.

Hints on using Zotero

* First install Firefox 3 then go to Zotero and click on "Download"
When Firefox re-starts, click on (bottom right) to open and close Zotero

* Watch the the Zotero tutorials then try a compliant catalog such as TynCat

* Click on in the address bar to add a single book, or to choose from multiple books
or click on on the Zotero bar to save a copy of the current web page.

* You can automatically download PDFs of articles in JSTOR etc (if you have access)
- in Preferences:General, tick "Automatically attach.."

* You can automatically index PDFs - in Preferences:Search, click on "Check for installer" for Xpdf. (this only works with PDFs containing text, and not images of pages - but see Evernote below)

* Add copies of your previous work so that it is indexed and searchable.

- save your document as PDF or as HTM (ie click on "Save as.." and set the Type as HTM or Webpage)
- click on then "Store copy of file" and click on your PDF or HTM file (Doc files aren't indexed yet).

* Chicago Styles are pre-installed. To add SBL style, click on Preferences: Export: Get additional styles.

2) Google Docs: Word processing on the web with automatic archive and backup.

Google Docs started life as Writely, which was bought up and developed as an online word processor to form part of Google's bid to rival Microsoft office.

Google Docs is superb for writing drafts, but you still need a 'proper' word processor for the final printout, because it does not have tools for the finer points of formatting.

Google Docs now includes all the features that most people use in a word processer:
- footnotes, spell check in multiple languages, tables, headers & footers, tables of contents, comments, bookmarks, images with text wrap, page numbers and basic styles.
- unicode right-to-left Hebrew works OK but isn't perfect. Accented Greek works well.

Google Docs lacks some wordprocessing features which are often neglected, but very useful:
- macros, tracking, custom tabs, borders, outlining, drawing tools, and pictures behind text

Some necessary formatting features are entirely missing, and are unlikely to be added:
- fonts (other than the few supplied), line-spacing, formatted footnotes and complex styles.
- printer settings and styles are limited, especially re footnotes and margins
Some very useful features in Google Docs are not found in normal word processors:
* copies are saved automatically online - not on your hard drive which will die one day
* older versions are saved continuously - so you can rescue something you deleted last month.
* two people can work on the same text simultaneously, or you can keep it open on two computers (click on "Share" and write in their email address to give someone shared access).
* you can publish to the web, or save as a Word document, or send as an email, in two clicks.
* you can work on it anywhere, on any online computer - even a hotel lobby computer.

What happens if the internet goes down? You can carry on writing, but you can't save, so copy+paste into Word (or equivalent) till your connection resumes.

You can make a local backup (File: Download) if you are feeling paranoid, but the internet copy is almost certainly more secure than your hard drive.

Hints for using Google Docs

* First open a GMail account and sign in. (free, and very useful)
At the top left of your mail page, click on "Documents", then on "New" and "Document"

* "Print settings" include margins, page numbers and hide comments.
Click on "Print.." to create a PDF file - though this doesn't show footnotes properly.
Use "Print as web page" to print with proper footnotes (though the Settings don't apply).

* For Greek or Hebrew use Unicode as in Word. I recommend the free Tyndale Unicode.
All the supplied fonts work on the web, but not when you Download as a Word file, so change the Greek and Hebrew in Word to Cardo (or another academic font)

* To resurrect old copies of a document, click on "File", "Revision history".
For very old copies, click on "Revisions 0-" in the bottom-right.

* Use Styles for titles and sub-headings, because these are needed for the Table of Contents, both in Google Docs and later when you Download the document to Word.

* To make text wrap round pictures, use "Insert" "Picture", choose the image file, then click on "More image options" and tick "Wrap text".

3) Evernote: Store photocopies and online books as searchable text.

Making notes is often more time consuming than reading, but without notes it can be impossible to find where you read something.

One solution is to scan what you read, then OCR it (ie put it through Optical Character Recognition software) so you have a searchable text.

But this is time-consuming, because OCR is imprecise (esp when the page has Greek or Hebrew!) so you need to spend time editing to make it readable.

Online books (eg Books.Google or Amazon) are even more difficult than real books because you can't print out the pages so you have to type all your notes.

The solution is EverNote - which automatically performs OCR on pages, so it can search your pages, but it always shows you the original copy of the page.
You can upload 40MB of scans per month and store them for ever, for free.
Screen-grabs from online books can be added and also searched.
Texts can be organised by folders and tags, by which you can limit a search.

Although it can't recognise Greek & Hebrew, it always shows the original scan, so you see real Greek - unlike a normal OCR where you just see gibberish Greek.

Now: Where did I read about "laws of nature" with regard to homosexuality? It was somewhere in Philo. I think it was when I researched Infanticide. Evernote found it: I searched for "law" within my photocopies about Infanticide:

Hints for using Evernote:
* Start by opening a free account at www.EverNote.com.
* Go to Download for a "desktop" version for your Mac or PC.
This isn't necessary but some features are much faster than the web version.

* Searches are fuzzy - I searched for "law" and it found "laws".

* OCR accuracy depends on the scan - 200 dpi (dots/inch) in greyscale work well.
300 dpi is better, but you can only save 40MB per month without paying.
(200 dpi greyscale lets you upload about 100 pages per month)

* Add scans on the web by clicking on "New" then "Attach file"
or email it (click on Settings to find the Incoming email address for your account)
or open the desktop version, click on "New Note", and drag the file(s) into the note.
(I find the last method the easiest)

* When the desktop version is running, it synchronises with the online documents.
Searching and displaying documents is almost instantaneous on this version.

* OCR is automatic, but not fast. So upload it then do something else.
OCR only works on bmp, jpg, gif & png - not on tiff or pdf (at present)
For PDF scans of books, extract the images with PDF2IMG and upload them as JPGs.

* To save a page from an online book, run the desktop version and press the "PrtSc" key
(near the top right on most keyboards - or Shift-Comm-4 on the Mac, I think)
then enlarge the box on the screen to select the grab, and press Enter
EverNote will OCR it so you can search the English (but not Greek etc)

For OCR on your desktop, use whatever software comes free with your scanner.
If you don't have any, the free software from http://www.simpleocr.com/ works well for books, but not for complex layouts with multiple columns, like magazines.
Don't pay for expensive software - 'better' software only improves the layout (ie pictures and correct formatting) without being much better at text accuracy.

4) Free FLV Converter: Search multiple sites for videos and save copies.

Videos are an increasingly important form of communication for scholarship.
You don't need special software to watch - just go to YouTube and similar sites.
Free FLV Converter searches 18 of the most important video sites at once, and saves videos onto your computer or iPod in a form you can project or even edit.

A search for "Lecture Bible Manuscripts" finds 288 videos (only 20 of them on YouTube), eg:
- Bart Erhman's Stanford Lecture - an academic lecture on his recent controvertial book
- Ahmed Deedat on "Muhammed in the Bible" (he finds Hebrew words sounding like the name)
- Pfander Films lively talk about the Qur'anic command to read the Bible, which implies the thousands of Bible MSS written before the Qur'an were 'uncorrupted'.
- Barthelemy on the Dead Sea Scrolls (in person! - in French)

Hints for using Free FLV Converter
* First get it and install it - I recommend Download.com where software is virus tested and searchable. (Here is something similar for Mac users).

* While installing, untick the usual offers of a toolbar. The Tyndale Toolbar has everything you need.

* Permanently turn off 'adult' sites, by clicking on "Site", "All adult" then "Remove adult websites".

* To save a video to your Desktop, highlight it and click "Download" (bottom left). Then wait a bit.

* The default format (ie AVI etc) is OK for playing and editing in most software

* For other settings, click on "Tube Downloader" (top row) then pick "Format" (iPod etc), pick quality in "Preset", and pick an "Output path", but don't change other settings unless you know what you are doing

You can project videos on a data projector in classes, or even in your room with a hand-sized projector linked to your laptop or iPod!

I expect that, like me, you are horrified by the poor content of most videos, but for good or ill, this is the communication medium of the present, so we should make sure that good material is added to the internet.

I've had a go at making a series of videos on my "Divorce & Remarriage" research. The sound quality is poor, but it is interesting, thanks to editing by PlaymoBible.
See, eg, my brief tour of some ancient manuscripts in Tyndale Library:


Tommy Keene said...

Dropbox should definitely be on this list. It combines perfectly with Zotero. I mentioned this on my blog, which includes a full guide to setting up zotero: http://www.nerdlets.org/2008/11/20/setting-up-zotero-link-by-link/

Dropbox automatically uploads, backs-up, and syncs files to the web, and then to any other computers you have Dropbox installed on. Put your Zotero directory in here and you're good to go. It's free: www.getdropbox.com

Also, as far as keep your data secure and backed-up, it's great to have an online solution (just in case of theft or fire). Mozy is quite nice for this. 2gb free, $5 /month for unlimited storage. mozy.com

Antonio B Duarte Jr said...
This comment has been removed by a blog administrator.
Unknown said...

thanks for the entry, much appreciated

thanks said...

Well done, it works for me.