Before I start, let me give you one piece of advice: Using mapSoN without any further tool installed that will allow you to filter and to redirect incoming mail into different folders will not make you happy. This kind of installation is nice to figure out whether mapSoN is useful to you, but you should install procmail or a similar program as soon as possible. Trust me.

Anyway, sendmail uses a simple mechanism to forward incoming local mail into application programs: A file named .forward, that must be located in your home directory. To activate mapSoN, all you have to do is to create that file and put the following line into it:

"|exec /usr/local/bin/mapson"

Apparently, on some systems the home directory must grant execute-permission to “other” for the sendmail to evaluate that file. So if you created the .forward file as shown above and still there's no sign of any mapSoN activity, execute chmod 711 $HOME and try again.

Another potential obstacle is that some sendmail installations use the restricted shell (smrsh) for the execution of the local mailer. This shell will not allow users to execute arbitrary commands in the .forward file. If your system uses smrsh, you must create a link from /usr/local/bin/mapson to /usr/adm/sm.bin/mapson in order to enable mapSoN. (The paths may vary from system to system, obviously.)

Most systems these days use procmail to deliver local mail. This means, that you can configure your local mailer by adding cryptic recipes in an entirely undocumented syntax to the file .procmailrc in your home directory:

ARGUMENT="$1"

# Have mapSoN accept anything that is a reply to a
# message of mine.
#
:0 w
* ^(In-Reply-To|References|Message-Id):.*example.org
| /usr/local/bin/mapson --accept

# Confirmation mails go into mapSoN.
#
:0
* ARGUMENT ?? [a-f0-9]…repeat 32 times…[a-f0-9]
| /usr/local/bin/mapson --cookie $ARGUMENT

# Forward the mail into mapSoN for approval unless
#   - it has an argument,
#   - is a bounce,
#   - comes from a mailing list, or
#   - is an automatically generated mail.
#
:0 w
* !ARGUMENT ?? ..*
* !^FROM_DAEMON
* !^Precedence: (list|bulk|junk)
* !^Auto-Submitted:
| /usr/local/bin/mapson

Don't panic, I know this recipe looks like hell, and to be perfectly host, it took me hours to get it working the way I wanted it. But all you have to do is to copy it into the .procmailrc file in your home directory … Be sure, though, to customize the parts marked replaceable for your system -- in particular the path to the mapSoN binary and the domain name of your e-mail address.

In case you want to know what this thing does, though, read on!

procmail has an incredibly useful feature that this recipe makes use of: The argument. If your address is, say, user@example.org, then you can receive e-mail under the address user+foo@example.org, too. You can append any string to your username by a plus sign as long as the result is still a valid e-mail address. procmail will still deliver these mails to user`s mailbox; the parameter is effectively ignored. But you can use that parameter to sort mail being sent to different addresses into different folders reliably!

If you subscribe to the mailing list “cat-lovers”, for example, you could subscribe the address user+cat-lovers@example.org instead of your ordinary address. That mail would still reach you, but with the recipe

:0:
* ARGUMENT ?? cat-lovers
/var/spool/mail/user2

you can easily sort the list's articles into a different folder!

In the recipe shown above, all mail that has such an argument will bypass mapSoN, and for a good reason: You don't want mapSoN to process mail that is delivered to you via a mailing list! It would be incredibly unpolite to request a confirmation from someone who posted to the mailing list and did not mail you at all -- at least not deliberately. To make matters worse, the challenge mail would not even reach the poster, but would be sent to the mailing list administrator because most mailing lists re-write the envelope of the mails delivered via them to that address.

So, to avoid all that mess: Subscribe under a user+something address and you won't have any problems. Plus: You can sort mail from different lists into different folders easily, if you want that.

Furthermore, any mail that just looks remotely as if it's not coming from a human sender will bypass mapSoN, too. This means that you'll receive a spam mail from time to time, but this is essential to avoid infinite mail looks, for instance. You don't want mapSoN to send a request for confirmation in response to a bounce mail that has been created because a former request for confirmation could not be delivered, etc.

Another nice thing is that the first rule in the recipe will make mapSoN accept any mail that looks as if it is a reply to an article of yours. If someone is replying to an e-mail or a news posting of yours, his mail reader will (hopefully) add the In-Reply-To header pointing to the message id of your article. And since message ids contain the hostname of the site that created the article, the first rule will recognize this and let the mail pass and add his address to the database.

The second rule will ensure that confirmation mails are processed correctly. In order to take advantage of that, you will have to edit your challenge template (see the section called “The Challenge File”) so that the From: line looks like this:

From: user+${MD5HASH}@example.com (Real Name's Anti-Spam-Tool)

In essence, this means that the person replying to the request will have the cookie put into procmail's argument automatically! If you don't want to use this, just don't -- mapSoN will find the cookies in the mail headers or body, too, but this approach is very error resistent. You wouldn't belive how many people are too dumb to understand “please reply and include that string in the mail: […]”.

In case you're wondering: The string “[0-9]” in the recipe must indeed be repeated exactly 32 times, because a cookie consists of 32 characters in the range of a to f or 0 to 9. Some regular expression libraries allow to shortcut this expression as [a-f0-9]{32}, but apparently the one shipped with procmail is not one of them. At least on my machines, I was not able to make that work.

One more general advice: Obviously the mapSoN-related recipes must be at the end of your .procmailrc file. Once mapSoN ran, the mail is processed and any recipes following below won't be invoked unless you do some heavy procmail magic. You have been warned.

Not everybody gets his e-mail delivered via SMTP, thus, not everybody can install mapSoN on the machine that actually accepts the incoming mail. If you use POP3 or IMAP for example, your mail will have been accepted by your mail server already and you just fetch it from there.

Luckily, you can still use mapSoN, but you'll have to use a tool like fetchmail. fetchmail will fetch the mails lying on your mail server via POP3, IMAP, or whatever and then invoke sendmail locally to actually deliver the mail to your mailbox. Hence, you can use the installation desribed in the section called “Using procmail”, too.

If you don't want to bother setting up sendmail -- and I could understand that --, tell fetchmail to call procmail as the delivery agent and you're fine. Use the following entry in your .fetchmailrc file:

poll mailserver.example.org mda "/usr/local/bin/procmail -d username"

Unfortunately, you cannot use procmail's argument feature in this setup, unless you can talk your e-mail provider into using procmail himself. If he does not, the user+foo username will yield an “unknown user” on his mail server otherwise.

libvarexp distinguishes variables into simple and complex expressions. A simple expression has the form “$NAME” and will basically only replace the variable in the text buffer with its contents. Complex expressions have the form “${NAME:operation1:operation2:…}” and may perform various operations on the variable's contents before inserting it into the text buffer.

Please note that due to the way simple expressions are parsed, it may not always be possible to use the simple-expression form even though you do not want to perform any operations. If your input text was “This is a $FOObar”, but the last “bar” part is meant to be a literal string, you'd have use “This is a ${FOO}bar”, because the parser will interpret any valid variable-name character following the dollar as part of the variable name; it will not recognize that “$FOO” would exist while “$FOObar” would not.

Also, libvarexp does not distinguish case in any way. For the library, “$FoObAr” and “$fOoBaR” are just strings -- whether they refer to the same variable or not is entirely up to the application that provides the callback used to resolve variables to their contents.

If you want to enter a text like “$foo” literally, you'll have to escape the “$” sign by prefacing it with a backslash: “\$foo”. Then libvarexp won't interpret this expression as a variable.

In addition to just inserting the variable's contents into the buffer, you can use various operations to modify its contents before the expression is expanded. Such operations are used by appending a colon plus the apropriate command character to the variable name in complex expression, for example: “${FOOBAR:l}”. Furthermore, you can chain any number of operations simply by appending another command to the last one: “${FOOBAR:l:u:l:u:…}”.

The supported operations are:

In addition to the variable expressions discussed in the previous sections, libvarexp can also be used to expand so called “quoted pairs” in the text. Quoted pairs are well-known from programming languages like C, for example. A quoted pair consists of the backslash followed by another character, for example: “\n”.

Any character can be quoted by a backslash; the terms “\=” or “\@”, for instance, are valid quoted pairs. But these quoted pairs don't have any special meaning to the library and will be expanded to the quoted character itself. There is a number of quoted pairs, though, that does have a special meaning and expands to some other value. The complete list is shown below. Please note that the name “quoted pair” is actually a bit inaccurate, because libvarexp supports some expressions that are no “pairs” in the sense that they consist of more than one quoted character. But the name “quoted pair” is very common for them anyway, so I stuck with it.

The quoted pairs supported by libvarexp are:

In addition to normal variables, libvarexp also supports arrays of variables. An array may only be accessed in a complex expression -- “$NAME[1]” is not correct syntax. Use “${NAME[1]}” instead. The reason for this limitation is that the brackets used to specify the index (“[” and “]”) have a different meaning in ordinary text; see the section called “Looping” for further discussion.

Which variables are arrays -- and which are not -- is entirely up to the application developer. In some applications, every variable may be accessed as both a normal variable and an array. In other applications, normal variables and arrays are different things. libvarexp does not dictate this. There exists the convention that accessing an array with a negative index, such as “${ARRAY[-1]}” should return the number of elements the array contains. But again, this is not a behavior required by libvarexp; different applications may behave differently here.

When specifying the index of the array's element you wish to access, you can use complete arithmetic expressions to calculate the entry. libvarexp supports the operands “+” (addition), “-” (subtractin), “*” (multiplication), “/” (division), and “+” (modulo).

These operations may be used on any signed integer. A valid expression is, for example: “${ARRAY[-12/4+5]}”. Please note that libvarexp follows the usual operator precedence. To group expressions explicitely, put brackets around them: “${ARRAY[-12/(2+4)]}”.

In any place you can write a number in such an expression, you can also use a simple or complex variable expression. If “$TWO” is “2”, the following expression would access the 5th entry in the “$FOO” array: “${FOO[10/$TWO}”.

Obviously, arithmetic in array indices would be quite pointless without a looping construct. libvarexp offers such a costruct, which can model both a “for” and a “while” loop. Let's start with the second version, which is slightly simpler.

If the index delimiters “[” and “[” are found in the text, the start a looping construct. An example would be “This is a test: [ $FOO ]”. What happens now is that all text between the loop-delimiters is repeated again and again until all variables found in the body of the loop say they're undefined for the current index. The current index starts counting at zero (0) and is increased with every interation of the loop. In the index-specifier of the variable, it is available as “#”.

Hence, if we assume that the variable “ARRAY[]” had three entries: “entry1”, “entry2”, and “entry3”, then the loop “[${ARRAY[i]}]” would expand to “entry1entry2entry3”. Once the conter reached index 4, “all” arrays in the loop's body are undefined.

That raises the question what the first example we presented, “This is a test: [ $FOO ]”, would expand to? The answer is: To the empty string! The loop would start expanding the body with index 0 and right at the very first iteration, all arrays in the body were empty -- that is, no array would have been expanded, because there weren't any arrays.

Thus, this form of looping only makes sense if you do specify arrays in the loop's body. If you do, though, you can do some weird things, like “[${ARRAY[#%2]}]”, which expands to “${ARRAY[0]}” for even numbers and to “${ARRAY[1]}” for odd numbers. But the expression has another property: It will never terminate, because the array-loopup will never fail, assuming that indices 0 and 1 are defined!

That is unfortunate but can't be helped, I'm afraid. Users of libvarexp may choose to disable looping for the users of their application to prevent the end-user from shooting himself in the foot with infinite loops, though. But if you want to use loops, you must know what you're doing. There ain't no such thing as a free lunch, right?

There is another form of the looping construct available, that resembles a “for” loop more closely. In this form, the start value, the step value and the stop value of the loop can be specified explicitely like this: “[$FOO]{start,step,stop}”. This loop will start to expand the body using index start, it will increase the current index in each iteration by step, and it will terminate when the current index is greater than stop. (Please note that “greater than” is concept that needs much thought if you use negative values here! There may be some infinite loops coming. You have been warned.)

If any of the first two values are omitted, the following defaults will be assumed: start = 0 and step = 1. If stop is omitted, the loop will terminate if none of the arrays in the loop's body is defined for the current index. Consequently, using the loop-limits “{,,}” is equivalent to not specifying any limits at all.

Since most users will not need the step parameter frequently, a shorter form “{start,stop}” is allowed, too.

By the way: Loops may be nested. :-)

To confuse the valued reader completely, let's look at this final example. Assume that the arrays “${FOO[]}” and “${BAR[]}” have the following values:

Then the expression:

would expand to:

Have fun!