Shell Programs

A shell program is simply a script consisting of one or more shell commands. Suppose we had a complex procedure consisting of a number of commands and pipelines:

extract -i '**Ursatz' inputfile | humsed '/X/d' \


| context -o Y -b Z > Ursatz

extract -i '**Urlinie' inputfile | humsed '/X/d' \

| context -o Y -b Z > Urlinie

assemble Ursatz Urlinie | rid -GLId | graph

In the above hypothetical script, we have processed an input file called inputfile. It may be that this is a procedure we would like to apply to several different files. Rather than typing the above command sequence for each file, an alternative is to place the above commands in a file. Let's assume that we put the above commands in a file called Schenker. In order to execute this file as a shell script, we need to assign execute permissions to the file. We can do this by invoking the UNIX chmod command.

chmod +x Schenker

The +x option causes chmod to add execute permissions to the file Schenker. Using chmod we can change modes related to executing a file, reading a file, and writing to a file. Possible mode changes include the following:

+x	add execute permission
-x	deny execute permission
+r	add read permission
-r	deny read permission
+w	add write permission
-w	deny write permission

Having added execute permissions to the file, we can now execute the shell script or program. This is done simply by typing the name of the file; in effect, the filename becomes a new command:

Schenker

Each time we type this command, our script will be executed anew. Notice that in our script, the final output has not been sent to a file. As a result, the output from our Schenker command will be sent to the screen (standard output). It is convenient not to specify an output file in the script since this is often something the user would like to specify. When typing our new command, we can use file-redirection to place the output in a user-specified file:

Schenker > outputfile

As currently written, our program can be applied only to an input file whose name is literally inputfile. If we wanted to, we could edit our script and update the name of the input filename every time we want to use the command. However, it would be more convenient to specify the input filename on the command line -- as we can do for other commands. For example, it would be convenient to be able to type commands such as the following:

Schenker opus118 > opus118.out

In order to allow such a possibility, we can use a predefined feature of the shell. Whenever the shell receives a command, each item of information on the command line is assigned to a shell variable. The first item on the command line is assigned to the variable $0 (normally, this is the command name). For example, in the above example, $0 is assigned the string value "Schenker." The variables $1, $2, $3, etc. are assigned to each successive item of information on the command line. So in the above example, $1 is assigned the string value "opus118."

These shell variables can be accessed within the shell script itself. We need to revise the script so that each occurrence of the input file is replaced by the variable $1:

extract -i '**Ursatz' $1 | humsed '/X/d' \


| context -o Y -b Z > Ursatz

extract -i '**Urlinie' $1 | humsed '/X/d' \

| context -o Y -b Z > Urlinie

assemble Ursatz Urlinie | rid -GLId | graph

This change means that our Schenker command can be applied to any user-specified input file -- simply by typing the filename in the command.

Flow of Control: The if Statement

Suppose we wanted our Schenker command to apply only to tonal works -- more specifically, to works with a known key. Before processing a work, we might want to have Schenker test for the presence of a tandem interpretation specifying the key.

Let's begin by using grep to search for a key tandem interpretation. An appropriate grep command would be:

grep '^\*[A-Ga-g][#-]*:' $1

Recall that we can assign the output of any command to a shell variable by placing the command within back-quotes or greves, i.e. `...`. Let's assign the key interpretation to the variable KEY:

KEY=`grep '^\*[A-Ga-g][#-]*:' $1`

If no key indicator is found by grep, then the variable KEY will be empty. We can test for this condition using the shell if statement.

KEY=`grep '^\*[A-Ga-g][#-]*:' $1`
if [ "$KEY" = "" ]
then

echo "Sorry, this input file has no key."
exit

fi

Notice that we use the dollars sign prior to the variable to mean the contents of variable KEY. The double quotation marks allow a string comparison. Our test is whether the variable $KEY is equivalent to the empty or null string "". If the test is true, then the commands following the then statement are executed. By convention, these commands are indented for clarity. In the above case, two commands are executed if the $KEY variable is empty. The echo command causes the quoted string to be output. The exit command causes the script to terminate. Notice the presence of the fi command (if backwards). This command simply indicates that the if-block has ended.

Of course, if there is a key designation, then it is appropriate to execute the rest of our Schenker script. The complete script would be as follows:

KEY=`grep '^\*[A-Ga-g][#-]*:' $1`
if [ "$KEY" = "" ]
then

echo "Sorry, this input file has no key." exit

else

extract -i '**Ursatz' $1 | humsed '/X/d' \


| context -o Y -b Z > Ursatz

extract -i '**Urlinie' $1 | humsed '/X/d' \

| context -o Y -b Z > Urlinie

assemble Ursatz Urlinie | rid -GLId | graph

fi

Notice the addition of the else statement. The else statement delineates the block of commands to be executed whenever the if condition fails -- that is, when the $KEY variable does not equal the null string. Once again, to make the script more readable, we indent the commands contained in the else-block.

The if command provides many other ways of testing some condition. For example, the shell provides ways to determine whether a file exists, and other features.

Flow of Control: The for Statement

In music research, a common task is to apply a particular process or script to a large number of score files. By way of illustration, suppose we wanted to know the maximum number of notes in any single folk melody in a collection of Czech folksongs. Suppose further that we are located in a directory containing a large number of Czech folksongs named czech01.krn, czech02.krn, czech03.krn, and so on.

We would like to run the census -k command on each file separately, but we'd prefer not to type the command for each score. The for statement provides a convenient way to do this. The following commands might be typed directly at the shell:

for J in czech*.krn
> do
> census -k $J | grep 'Number of notes:'
> done | sort -n

The pattern czech*.krn will be expanded to all of the files in the current directory that it matches. The variable J will take on each name in turn. The commands between do and done will be executed for each value of the variable $J. That is, initially $J will have the value czech01.krn. Having completed the do-done block of commands, the value of $J will become czech02.krn, and the do-done block will be repeated. This will continue until the value of $J has taken on all of the possible matches for czech*.krn.

The output might appear as follows:

Number of notes: 31
Number of notes: 32
Number of notes: 32
Number of notes: 34
Number of notes: 35
Number of notes: 39
Number of notes: 39
Number of notes: 40
Number of notes: 48
Number of notes: 48
Number of notes: 55
Number of notes: 78
etc.

Incidentally, the output from a for construction such as above can be piped to further commands, so we might identify the maximum number of notes in a Czech melody by piping the output through sort -n.

A Script for Identifying Transgressions of Voice-Leading

Shell programs can be of arbitrary complexity. Below is a shell program (dubbed leader) whose purpose is to identify all instances of betrayals of nine classic rules of voice-leading for a two-part input. A number of refinements have been added to the program -- including input file checking, and formatting of the output.

The program is invoked as follows:

leader

Reprise

In this chapter we have illustrated how to package complex Humdrum command scripts into shell programs. This allows us to create special-purpose commands. We learned that files can be transformed into executable scripts through the chmod command. We also learned how to pass parameters from the command line to the script, and how to assign and modify the contents of variables. In addition, we learned how to influence the flow of control using the if and for statements. Finally, we learned that multi-line scripts can be typed directly at the command line without creating a script file.

Shell scripts can be very brief or very long. It is possible to create scripts that carry out highly sophisticated processing such as searching for voice-leading transgressions. There are innumerable features to shell programming that have not been touched-on in this chapter. Several books are available that provide comprehensive tutorials for shell programming.

Locating Violations of the Rules of Voice-Leading

The traditional rules of voice-leading have formed a standard component of conservatory training for art musicians.

For illustration purposes, we'll apply some of the Humdrum tools to the problem of identifying betrayals of the classic rules of voice-leading in a **kern-encoded score. Note that our purpose here is not to legislate how to compose or arrange! We're simply using the traditional voice-leading rules as a way to introduce various pattern-searching techniques.

(1) Parts Out Of Range

RULE: "Avoid parts that are out of range."

The Humdrum census command provides a summary of various elementary features of any Humdrum input. With the -k option, census provides a summary of a further ten features pertaining to **kern format inputs. This includes the highest and lowest notes present.

census -k

Since we are interested in the highest and lowest notes for each individual part (rather than for the whole piece), we should extract each part before processing it with census.

extract -i '*soprano' > soprano.part
census -k soprano.part

On UNIX, a set of commands that sequentially process a given input can be joined together as a "pipeline". A pipeline feeds the output of one process to the input of another process. This means that we can simplify the above sequence of commands into a single pipeline:

extract -i '*soprano' | census -k

We could then repeat the pipeline for each voice present:

extract -i '*alto' | census -k
extract -i '*tenor' | census -k
extract -i '*bass' | census -k

If we wanted to get a little fancier, we could filter the output so that only the highest and lowest pitch information is output. The UNIX grep command will let us define a string for searching; egrep permits compound strings, such as the use of the OR bar (|):

extract -i '*soprano' | census -k | egrep 'Highest|Lowest'

(2) Augmented/Diminished Melodic Intervals

RULE: "Avoid parts that move by augmented or diminished intervals.

(3) Consecutive Fifths or Octaves

RULE: "Avoid consecutive fifths and octaves between any two parts.

Let's focus on identifying consecutive fifths -- since the process is the same for octaves.

Either the Humdrum patt or pattern commands can be used to find patterns that span more than one line or record. For this example, we'll use patt.

(4) Doubled Leading Tone

RULE: "Avoid doubling the leading-tone."

Pitches can be identified as "leading-tones" only when we have some idea of their key-related scale-degree. The Humdrum deg command translates pitch representations to a scale-degree representation where the numbers 1 to 7 represent tonic to leading-tone designations.

Notice that the score input must contain an explicit key indication (a special type of Humdrum tandem interpretation). For example, the key of G major is indicated through the presence of the following interpretation:

*G:

**kern

*b:

8r

8f#

8b

16.cc#

32dd

8a#

16.b

32cc#

8dd

*-

**deg

*b:

5

^1

^2

^3

v7

^1

^2

^3

*-

deg

Having translated the representation in this way, we need to search for any lines which contain two instances of the number `7' -- that is, two concurrent instances of the leading-tone.

Searching for the number `7' is easily done using the standard Unix grep (get regular expression) command:

deg | grep -n '7'

This will find and output all records that contain the number 7; the -n option means that the corresponding line number will also be output.

However, we want to find instances where two or more 7s occur on a single line. For this, we can use a slightly more complex regular expression

deg | grep -n '7.*7'

In the construction ".*" the period (.) means any character, and the asterisk means "zero or more instances of ..." Hence, the regular expression means "the number 7 followed by zero or more instances of any character, followed by the number 7". In short, this expression will match any record in which the number 7 occurs at least twice.

As in the case of our earlier search for augmented and diminished intervals, grep is insensitive to whether the matching character string is found in a data record, or whether it occurs in a Humdrum comment or interpretation. In order to avoid matching comments or interpretations, a further refinement to our regular expression is appropriate.

deg | egrep -n '^7.*7|^[^!*].*7.*7'

In this case, the regular expression says the following: "find any occurrence of the number 7 at the beginning of the line followed by zero or more characters followed by the number 7; or match any character at the beginning of the line -- other than an exclamation mark or an asterisk -- followed by zero or more characters, followed by the number 7, followed by zero or more characters, followed by another number 7.

Depending on the input, it is possible that Humdrum spines will be present that do not represent scale degree information. It is possible that these other kinds of data may also make use of the number 7 -- but not to represent the leading-tone. In other words, it is possible that a matching `7' has nothing to do with scale degrees. We can ensure that this doesn't happen by first ensuring that only scale-degree spines are present in the input to be searched.

(5) Avoid Unisons

RULE: "Avoid the sharing of pitches by two parts."

For this rule, let's assume that we also want to identify unisons that are spelled enharmonically (such as F-sharp and G-flat).

First, we need to translate the two parts into some absolute pitch representation -- such as frequency or semitones. This will ensure that enharmonically equivalent pitches have the same representation -- and so will facilitate comparison.

(6) Crossed Parts

RULE: "Avoid the crossing of parts."

Part-crossing occurs when a nominally higher voice uses a pitch that is lower than a nominally lower voice.

The relations "higher" and "lower" suggest the use of an arithmetic operator such as greater-than (>) or less-than (<). In brief, we will approach this question by translating the pitches to a numerical scale, and then use the general-purpose Unix awk command to test whether the nominally lower voice is truly lower.

(7) Parts Separated by Greater than an Octave

RULE: "Avoid intervals greater than an octave between the soprano and alto voices. Also avoid intervals greater than an octave between the alto and tenor voices."

Finding infringements of this voice-leading rule requires just a slight modification to our method for identifying the crossing of parts.

Having transformed the pitch input to a numerical form, we simply need to check whether the difference between the two semitone values is greater than 12 semitones.

The awk portion of our command is modified so that we are informed if the nominally higher voice is more than 12 semitones away from the other voice:

extract -i '*soprano,*alto' | semits -x | ditto -s = | awk '{if($0~/[^0-9\t-]/)next}{if($2-$1>12) print NR}'
extract -i '*alto,*tenor' | semits -x | ditto -s = | awk '{if($0~/[^0-9\t-]/)next}{if($2-$1>12) print NR}'

(8) Overlapped Parts

RULE: "Avoid the overlapping of parts, where the pitch in an ostensibly lower voice moves to a pitch higher than the previous pitch in an ostensibly higher voice; or where the pitch in an ostensibly higher voice moves to a pitch lower than the previous pitch in an ostensibly lower voice."

The following passage illustrates a violation of the part overlapping rule:

**pitch **pitch

C4 E4

F4 A4

E4 G4

*- *-

This rule is similar to the part-crossing rule, only we have to compare the current pitch in one part with the previous pitch in another part.

Rather than making a direct comparison, for the purpose of this tutorial, we will make a modification to our earlier part-crossing detector. In brief, we will extract one of the parts, shift the data tokens within that part, paste the two parts back together, and then check to determine whether the shifted part shows any "crossed parts" -- using our earlier command pipeline.

The following command pipe will shift the data tokens in a spine down one record. (The last data record will disappear.)

(9) Exposed Octaves

RULE: "When approaching an octave by similar motion, ensure that at least one of the parts moves by step."

Violations of the exposed octaves rules must meet three conditions. First, the two voices must be separated by an octave (or two octaves, or a unison, etc.). (This suggests that we use the hint command with the -c option in order to reduce compound intervals to their non-compound equivalents.) Second, the voices must be moving in the same direction. (The **deg representation may be suitable here, since it distinguishes notes according to whether they are approached from below ("^") or above ("v").) Third, both voices must be moving by leap (e.g. more than two semitones).

To address this problem, let's plan to create five different spines. The first spine will encode harmonic interval size so that all compound equivalents to a unison are represented by the string "P1".

The second spine will indicate whether the lower voice is ascending or descending ("^" or "v"). Similarly, the third spine will indicate whether the upper voices is ascending or descending.

The fourth spine will indicate whether the melodic motion for the lower voice is by leap ("leap"), and the fifth spine will indicate whether the melodic motion for the upper voice is by leap.

Examples of violations of the exposed octaves rule will appear as one of the following two situations:

(**hint **updown **updown **size **size)

P1 ^ ^ leap leap

P1 v v leap leap

In other words, our final test can be expressed using the following egrep command:

egrep -n 'P1.*^.*^.*leap.*leap|P1.*v.*v.*leap.*leap'

Now all we need to do is generate our five spines and assemble them in the proper order.

The first spine is easily generated using the hint command. Remember that the -c option means that all intervals an octave or greater will be represented by the within-octave equivalent.

extract -i '*alto,*tenor' | hint -c > spine1