What is this and who cares?

Screenshots

Known Problems

Getting Help


Download Agate Rule Tester

What is this and who cares?

The Agate Rule Tester is an attempt to lessen the pain involved when building agate translations for Digital Technology International's WireSpeed system.

'agate' is a term for a unit of measurement found primarily in newspaper publishing. Agate is approximately 5 1/2 points (1/14 inch or 1.814 mm). The very small typeface used for statistical, sports, and stock data in your newspaper is agate type.


A sample of sports agate from the paper

Most newspaper agate comes from a wire service (AP, UP, etc) and is in a format that helps with transmission of data and not necessarily readability. Lots of semicolons, special unprintable characters, tons of whitespace, etc delimit the file to denote line endings, table elements, and a whole host of other formatting options.

The process of agate translation is to turn this mess into what you see above, a nice, clean, readable output that can be printed in a newspaper. DTI's WireSpeed is the process that does this actual conversion (along with taking in the raw file and storing it in the database). It does this by using rulesets that are applied as the raw file is picked up for storage into the database. These rulesets are comprised of individual rules that can be relatively simple find and replacements (change all occurrences of 'Chevrolet' to 'Chevy') to ridiculously difficult ones that involve very long regular expressions.

The current process of building these rules is ugly. To test any changes to your ruleset, you must restart the entire WireSpeed process. WireSpeed is slow to restart and is a server process, meaning to restart it, you either need physical access to the machine or remote desktop installed on your rule builder's desktops. You also need continual access to a network as the rules are built in an application that is a management program for the entire database (SpeedManager).

Output is slow and tedious, especially when you are using regular expressions. Each iteration of testing can (and usually does) take minutes to complete as one needs to restart WireSpeed, load a test file for it to process, check the output stored in the database, note changes needed, make the changes in the SpeedManager interface, save, and repeat the entire process.

Agate Rule Tester bypasses a lot of these issues by firstly being a stand-alone application. You load the base ruleset, raw agate file, and save test InDesign documents all from the tester. Also, because it operates without the need for a database or SpeedManager, one could build rules offline and simply upload them at a later time.

The tester also includes accoutrements not present in DTIs present translation building workflow: the ability to turn off rules instead of just deleting them, real-time output shows what changes your rule just made, better error reporting when a regular expression fails, and the entire process is much faster. During initial testing of the application, some developers were able to complete rulesets in 20-30 minutes and with a single reload of WireSpeed instead of 2-8 hours and dozens of restarts, which is normal for the traditional approach.

You care about the tester because you're a DTI 5.5 user and you're attempting to either edit new translations or update existing ones. Or you are a Cocoa/Perl programmer and you're curious about seeing how one could incorporate the massively powerful perl regex engine into your Coca/Objective-C applications. Or you could be trying to test regular expressions on a text file and would like to see what the outcome would be (you just wouldn't use DT's mnemonics).

Screenshots

This is what the tester looks like when first launched.  Now we load a raw wire file and a simple base translation.  Adding a rule using regular expressions.  Notice that all the features found in SpeedManager are present in the tester.  A completed ruleset with all of DT's mnemonics turned on.  The finished output placed in InDesign. 

Known Problems

There are a couple of known issues with the tester. Most are trivial, but they are all worth mentioning.
  • The tester is usually right 95% of the time. There are some incredibly unique situations where that the tester and WireSpeed simply won't agree on how the output should look (especially with the more complicated regular expressions one can build). Of course, the tiebreaker is always what WireSpeed thinks is right.
  • The tester uses only perl at its core to do all the text manipulation one can do in a rule while I'm not positive what WireSpeed uses (I think it's perl in some cases and Java in others). This can cause some of the problems mentioned above.
  • The tester does not like hex values and only uses Unicode. WireSpeed will let you get away with hex. When in doubt, use the proper mnemonic to ensure your rule moves between the two correctly.
  • I'm still a little new to auto-saves and determining if a file is in a changed state since its last save, so the the app treats everything like an "open as". You'll need to tell it were to save eerytime you do so.
  • To load or start a new translation, you must quit the app. It sucks, I know, but there's a problem with the way the JTFileData object is being destroyed (or not being destroyed, as the case is).
  • The translated text window has issues displaying long lines broken by spaces. It wants to wrap the text even though there is ample room to display a non-wrapped version. This is especially true when there are a lot of tabs inline. This does not affect the viability of the output, however.
  • I have written a ART rule uploader and downloader, but I'm not publishing them as they can be particularly dangerous (especially the uploader). You are more than welcome to write your own, though. If I can discover a safe and reliable way to do uploads, I'll make it part of the application.
  • Files must be in a UTF-8 format otherwise the tester will complain that it's opening the file as ASCII and it'll prompty destroy any wide characters (emdash, thinspace, etc).

Getting Help

Help is available by posting your question on the SourceForge Forums for Agate Rule Tester. I will watch there for any questions on how to operate the tester and for suggestions on fixing/updating it.

It is important to note that I cannot assist with writing the actual translation itself. For that, you are on your own or at the mercy of DT Tech Support (say Hi to Russell for me).