FEI Version 3.2.0
Last update on October 25, 2001
Send documentation comments and corrections to MDMS or FEI Administrator
See Release Notes for list of recent changes to FEI. Please contact the FEI Administrator for upgrades.
Most FEI commands include an FEI file type. For the examples in this tutorial, we'll use the file type image1. If you've been given a different file type to use, substitute your file type for "image1" in the examples.
The commands you are to type at a shell prompt are presented in a bold type. For the examples, the symbol '%' denotes the shell or system environment prompt. Results returned by executing a command are displayed in a regular, fixed-width font.
The examples use a set of files labeled file1, file2, and so on. You can substitute any set of files you like in place of these. Hint: Before you start, create a /Test directory and copy a set of files into it. In the examples we'll be adding and getting files, and we don't want you to overwrite or destroy a valuable file.
Before you can use FEI you must have:
Installing the software and configuring an environment is covered in the Client Installation Guide. You can obtain authentication and access control credentials from your FEI Administrator. For the tutorial, you'll need the capability to add, replace, get and delete files on some FEI file type. Getting a Kerberos ticket is covered in the next section.
Try this command:
% feiList image1
Not authorized to connect to server to server "fei3"
More than likely, this error message is telling us that we forgot to get a ticket from Kerberos. That ticket is needed to authenticate us with the FEI server, so we'll need to get it before using FEI. It's easy to get a ticket. We just run a utility called feiKinit and supply a principal (or user) name and our Kerberos password. In the following example, our Kerberos principal name is bar.
% feiKinit bar
SunOS (milano)
Kerberos Initialization for "bar"
Password: <bar's Kerberos password>
%
Once we have a ticket we can begin to use FEI. The ticket is good for some number of hours, usually between 5 and 23, depending upon the discretion of the FEI Administrator. Once you have a ticket it's accessable to any local terminal session you start, so you can bring up several windows and run FEI commands in them without having to obtain a new ticket.
If you couldn't get a ticket, contact your FEI Administrator for assistance.
The command line utilities are described in this document and in the FEI Command Line Utilities Reference. This document introduces you to the FEI commands and the Reference gives additional information, especially on more detailed topics like date and time formats used with the utilities.
You can use these documents to whatever extent you need to, but after a short introduction, all the reference material you should need is contained in the command line utilities themselves. You'll find that from time to time you need a quick reference to three things:
In the remainder of this section, we show how to get this information using the command line utilities themselves.
To get the list of file types in your current FEI domain, type the command:
% feiFileTypes
Copyright 2001, The California Institute of Technology.
All rights reserved.
FEI API Version 3.2.0, August 1, 2001
File types in "feiDomain"
image1 image2
image3 image4
image5 image6
The first three lines show copyright and FEI version information. Following that is the information of interest. The first line shows the name of the FEI domain file were using. Normally there's only one, but in some larger systems there can be more. Following that are two columns displaying the names of the file types in the domain. You should have access to one or more of these. Anyway, if you forget the name of a file type, use this command to find it. Note: If this command didn't work, most likely your FEI environment is incorrect. See the FEI Client Installtion Guide for instructions on setting up the environment or contact your FEI Administrator for help.
Next we need to know what commands are available and what they do. Type the following command to get that information:
% feiHelp
feiFileTypes or feifiletypes
Lists the file types in your local FEI domain files.
feiAdd or feiadd
Add a file to FEI. Two modes: command line and interactive.
feiReplace or feireplace
Like Add, but will replace the file if it already exists in FEI.
feiAccept or feiaccept
Takes a list of file names from STDIN and adds or replaces them in FEI.
feiList or feilist
Lists the files in FEI. Optional file name expression and/or date-time.
feiGet or feiget
Gets files from FEI. Optional file name expression and/or date-time.
feiDisplay feidisplay
Writes the contents of a file to STDOUT.
feiListGet feilistget
Combines both list and get functions in an interactive mode.
feiNotoify feinotify
A subscription service that just displays the names of new files.
feiSubscribe or feisubscribe
A subscription service that delivers the file to you.
feiDelete feidelete
An interactive utility that lists files which you can delete.
feiRename or feirename
Rename a file within a file type.
feiFileMaker or feifilemaker
A testing tool. Creates test files and sends them to FEI.
feiMakeClean feimakeclaen
Deletes all files for a file type.
feicrc
Displays the crc of the file
== Type a command with no arguments to see its syntax.
For now, just notice that we have some quick, simple documentation at hand. We'll describe how to use the listed commands in the rest of this document.
The last line displayed by the FEI command, the one beginning with ==, says that we can get a command's syntax by just typing the command with no arguments. Let's try it:
% feiList
Copyright 2001, The California Institute of Technology.
All rights reserved.
FEI API Version 3.2.0, August 1, 2001
feiList <file type> ['<file name expression>']
{[on|before|after <date-time>]|[between <date-time1> and <date-time2>]
[long]}
The last line returned shows the syntax for the feiList command which returns a list of file names from FEI. Arguments in <angle backets> are names you supply. Any other word is a keyword. An argument in [square brackets] is optional. The list of arguments in {curly backets} are order independent, meaning that you can change their order on the command line and the command should still be able to make sense of them.
Most of the commands take a file type as the first argument. The second argument is a file name expression, which is sometimes optional. The rest of the arguments are keywords or keyword-value pairs that provide additional information to the command.
By the way, a file name expression is a file name that can take the form of a simple regular expression. If you've ever used the Unix 'ls' command or the DOS 'dir' command you're already familiar with the concept. Here are a few examples to show what we mean. The file name expression "file1" is just that file1. The expression "file*" is any file name starting with "file...", like file1, file.txt, fileName, and so on. The expression "[fF]ile" is file or File.
All of the FEI commands that can be run from a command window are listed in the FEI Command Line Reference. You might want to take a quick look at these now, although it's not necessary because we'll introduce each command as we use it.
Each FEI command is wrapped in its own little utility program so the commands appear to be an extension of your shell envorionment. For example:
% feiAdd image1 file1
% feiList image1
% feiGet image1 file1
Each FEI command starts with a word "fei" followed by the command name. The command name either contains all lower case letter or the first letter in command name in upper case followed by all lower case letter in the command name, feiList or feilist , for example; and each command is spelled out. If you don't like using the full names or using the convention of starting with the word "fei", create a set of aliases to make your own commands. We could use just the lower case letter "l" for feiList if we wanted:
% alias l feiList
% l
Just be careful not to use a command that's already defined. Many systems have a "delete" command for example, so that's a poor choice for the "feiDelete" command. (Separating FEI commands from other commands executed from a command window was the rational for starting each FEI command with a word fei.)
All FEI commands, with the exception of feiFileTypes, take a file type as the first argument. Most FEI commands have additional parameters. The syntax for each command is provided in the FEI Command Line Utilities Reference. You can also get a quick look by just typing the command with no arguments. Try this:
%feiList
Copyright 2001, The California Institute of Technology.
All rights reserved.
FEI API Version 3.2.0, August 1, 2001
feiList <file type> ['<file name expression>']
{[on|before|after <date-time>]|[between <date-time1> and <date-time2>]
[long]}
The last line returned shows the command's syntax.
To add a file, type something like this:
% feiAdd image1 file1
Adding to file type "image1":
1. Added "file1".
where "image1" is the file type and "file1" is the name of the file. "File1" must exists in the current directory.Now try to execute the same command again. % feiAdd image1 file1 and you will see a message "File already exists in FEI". This utility also allows the user to add a file from a directory other than the current directory by providing an absolute path name in front of the file name on the command line.
Let us create a subdirectory "Sub" under the current directory (Test) and copy "file1" from the current directory to the new subdirectory "Sub" as "fileOne". Steps explained above are performed as follows:
% mkdir Sub
% cp file1 /home/Test/Sub/fileOne
Now execute the feiAdd command by providing the absolute path name on the command line. Remember that you are still in the Test directory
% feiAdd image1 /home/Test/sub/fileOne
Adding to file type "image1":
1. Added "fileOne".
You can't add a file to FEI if it already exists, but you use two separate commands so you don't inadvertantly replace a file but you can replace it. when you didn't mean to do so. To replace "file1", we execute the command:
% feiReplace image1 file1
Replacing to file type "image1":
1. Replaced "file1".
In above command "file1" (on the server) is replaced by "file1" located in the current directory (Test). feiReplace allows the user to replace file from a directory other than the current directory by providing an absolute pathname followed by the file name. Let us replace "fileOne" which we added to the server in the previous step by the "fileOne" currently loacted in the "Sub" directory by issuing the following command:
% feiReplace image1 /home/Test/Sub/fileOne
Replacing to file type "image1":
1. Replaced "fileOne".
We can also add or replace sets of files in serveral ways. First, let's use a file name expression to add a set of files. We use the standard wild card character '*' to mean "any set of characters in a file name". As an example, let's add all of the files whose names are like "file..." (We'll use the feiReplace command in our example so existing file's are overwritten. You might want to be more cautious and use feiAdd in a real situation.):
% feiReplace image1 file*
Replacing to file type "image1":
1. Replaced "file1".
2. Replaced "file10".
3. Replaced "file11".
4. Replaced "file2".
5. Replaced "file25".
6. Replaced "file26".
7. Replaced "file27".
8. Replaced "file28".
9. Replaced "file29".
10. Replaced "file3".
11. Replaced "file30".
To add all of the files in a directory, just use the '*' star character for the file name expression.
% feiReplace image1 *
And if we want to add all of the files, but just after a specified time, like files just created today, we add a time following the command keyword after <date-time>. Time formats conform to CCSDS syntax, so for September 10, 2000 we use 2000-09-10. (Note: The 0 in front of the 9 in the month value, 09, is mandatory for CCSDS conformance.) Here's the command:
% feiReplace image1 * after 2000-09-10
Since we didn't include a time, a default value of midnight, or "00:00:00" was used. We can supply an explict time or 4 AM or 4 PM as follows:
% feiReplace image1 * after 2000-09-10T04:00
Replacing to file type "image1":
1. Replaced "file1".
. . .
% feiReplace image1 * after 2000-09-10T16:00
Replacing to file type "image1":
1. Replaced "file10".
. . .
We can add files in interactive mode which is useful when the names of the files we want to add don't fit a pattern. To enter interactive mode, don't include a file name expression. You can include other keyword parameters with interactive mode, so any of the following commands will work:
% feiReplace image1
% feiReplace image1 after 2000-09-10T04:00
% feiReplace image1 encrypt crc after 2000-09-10T04:00
Here's a simple example:
% feiReplace image1
Press the <enter> or <return> key to exit.
feiReplace: [<local file name>] <FEI file name>
feiReplace: file29
Replaced "file29".
feiReplace: file30 image30
Replaced "image30".
feiReplace: file4
Replaced "file4".
feiReplace:
%
Both feiAdd and feiReplace give a set of simple instructions when you enter interactive mode. They are:
Interactive mode is faster than repeatedly executing feiAdd or feiReplace with a file name on the command line because you remain connected to the FEI server while in interactive mode.
Notice that the commands echo the action taken before repompting. This is just some nice fuzzy feedback to let you know that what you asked to be done was done.
There's one last way to add files, and that's done with a different command named feiAccept. This command reads a list of files to add or replace from the stdin file stream, so you pipe a list to the command. Here's a simple example using the Unix "ls" command. First we list the file to make sure the list contains the file we want added:
% ls -1 file1*
file1
file10
file11
Now we just pipe that list to feiAccept:
% ls file1* | feiAccept image1 for replace
1. Replacing "file1".
2. Replacing "file10".
3. Replacing "file11".
By default, feiAccept adds files. We told it to replace files by adding the keywords "for replace" following the file type. How did we know to do that? We looked at the syntax either in the Reference Guide or by typing the command name alone:
% feiAccept
Copyright 2001, The California Institute of Technology. All rights reserved.
FEI API Version 3.2.0, August 1, 2001
feiAccept <file type> for <add|replace|gre|remove>
{[output <path>] [encrypt] [crc] [replace|version]}
We could use feiAccept to enter files by content using "grep" to get the list of file names:
% grep "Mars Pathfinder" file* | feiAccept image1
If we had a lot of files to enter and didn't want to use interactive mode with feiAdd or feiReplace, we could use "ls" to make a list of files, edit the list, and then send the list to feiAccept.
First we get the list of files and write the list to a file.
% ls -1 file3* > 3files
Next, we edit the file names in 3file so only file names ending in 3<n> that are even numbered are included in the list.
% vi 3files
And then we send the list to feiAccept
% feiAccept image1 for replace < 3files
1. Replacing "file30".
2. Replacing "file32".
3. Replacing "file34".
4. Replacing "file36".
5. Replacing "file38".
Now that we've added some files, let's see what we've put into FEI using the list command:
% feiList
Copyright 2001, The California Institute of Technology. All rights reserved.
FEI API Version 3.2.0, August 1, 2001
feiList <file type> ['<file name expression'>]
{[on|before|after <date-time>]|[between <date-time1> and <date-time2>]
[long]}
To get the list of all files for a file type, we just supply the file type as an argument:
% feiList image1
file2
file25
file26
file27
file28
file3
file31
. . .
If we include the "long" keyword, we get a listing that includes the file's FEI file modification date and the size of the file along with the file name.
% feiList image1 long
1. 2000-253T11:16:40.553, 87340, file2
2. 2000-253T11:16:40.773, 132087, file25
3. 2000-253T11:16:41.026, 104723, file26
4. 2000-253T11:16:41.273, 104574, file27
5. 2000-253T11:16:41.513, 153480, file28
6. 2000-253T11:16:42.023, 143407, file3
7. 2000-253T11:16:42.483, 348, file31
. . .
The file size is given in bytes. Notice the times sent back; they're in order. And, in fact, the listing for this command and the last one are in the same order. That means that feiList always organizes files by FEI modification time. Why not sort alphbetically by file name? Because, most files in FEI come from spacecraft where the significant ordering prameter is the time. While FEI is not using the spacecraft's clock, the sequence by time may still be of value.
We can include a file name expression in the command as well.
% feiList image 'file1*' long
1. 2000-253T11:44:32.770, 545932, file1
2. 2000-253T11:44:33.553, 6250349, file10
3. 2000-253T11:44:33.883, 454320, file11
Notice that the file name expression is enclosed in single quotes. That's done to prevent the local shell from expanding the expression into a set of file names - which may or may not work depending on what's in your local directory. We want the list generated by FEI and not by the local shell, so the single quotes are needed.
Rule: Whenever sending a file name expression to FEI, the expression must be enclosed by single quotes. We will see that this is true for the feiList command, the feiGet and feiListGet commands as well.
If we just want to list new files, say the ones that arrived today, we can include a date following the after keyword:
% feiList image1 long after 2000-09-10
1. 2000-253T11:57:46.963, 8, file30
2. 2000-253T11:57:47.283, 8, file32
3. 2000-253T11:57:47.593, 8, file34
4. 2000-253T11:57:47.913, 8, file36
5. 2000-253T11:57:48.223, 8, file38
Getting files works almost like listing files. For example:
% feiGet image1 '*'
Getting from file type "image1":
1. Got "file2"
2. Got "file25"
3. Got "file26"
4. Got "file27"
5. Got "file28"
6. Got "file3"
7. Got "file31"
. . .
Notice that we had to include a file name expression, '*', to get back all of the files, but we didn't have to do that with list. The reason is that feiGet has an interactive mode like feiAdd and feiReplace. Without the file name expression, feiGet enters that mode as we'll see below.
If we execute the last command again, you get a different result because the files we want to get already exists in the local directory, and FEI won't overwrite a file unless we give explicit instructions:
% feiGet image1 '*'
Skipping file 'file2', since it already exists.
Skipping file 'file25', since it already exists.
Skipping file 'file26', since it already exists.
Skipping file 'file27', since it already exists.
Skipping file 'file28', since it already exists.
Skipping file 'file3', since it already exists.
Skipping file 'file31', since it already exists.
. . .
FEI allows the user to get files from the server to a directory of the user choice. To get the files in a directory of the user choice the user is required to specify the absolute path name of the directory on the command line. Remember the directory "Sub" we created earliar. Let us get these files from the server to this directory by issuing this:
% feiGet image1 '*' output /home/Test/Sub
1. Got "file2"
2. Got "file25"
3. Got "file26"
4. Got "file27"
5. Got "file28"
6. Got "file3"
7. Got "file31"
. . .
If we still want the file, we have two choices: replace or make a version of the existing file. To make a version of a file, FEI takes on a unique number string to the end of the existing file. We'll show both options and then look at the name of the file generated when a version is created.
% feiGet image1 file1 replace
1. Got "file1"
% feiGet image1 file1 version
1. Got "file1"
% ls file1*
file1 file1.873925933
The pre existing file1 is now named file1.873925933.
feiGet, like feiAdd and feiReplace, has an interactive mode which is entered when no file name expression is specified for the command. In the following example, we continue to enter commands until we type <enter> or <return> at the prompt. Notice that we can use file name expressions and include <date-time> values at the prompt. If we want files replaced or versions made of exiting files, the keyword must appear on the command line, and it remains in effect for the entire session.
% feiGet image1 version
Press the <enter> or <return> key to exit.
feiGet: <file name expression>
{[output <path>] [on|before|after <date-time>]|
[between <date-time1> and <date-time2>]}
feiGet <image1>: image1
No Files found.
feiGet <image1>: file1
1. Got "file1"
feiGet <image1>: file3*
1. Got "file3"
2. Got "file31"
3. Got "file33"
4. Got "file35"
. . .
feiGet <image1>: file* after 2000-09-10
1. Got "file2"
2. Got "file25"
3. Got "file26"
4. Got "file27"
. . .
feiGet <image1>:
%
An advanced feature of feiGet allows us to get files whose names are piped to the utility or from a list of file names contained in a file. Normally, this mode is not used directly, so we defer a discussion of it.
We've looked at feiList, and we've looked at feiGet. The command feiListGet combines the two.
% feiListGet
Copyright 2001, The California Institute of Technology. All rights reserved.
FEI API Version 3.2.0, August 1, 2001
feiListGet <file type> ['<file name expression>']
{[output <path> [on|before|after <date-time>]
|[between <date-time1> and <date-time2>]
[encrypt] [crc] [replace|version] [long]}
In the following example we want to see all of todays files because we may decide to get some of them. We get a "long" listing so the FEI file modification date and file size are included in the Listing. We also include the keyword "replace" so that existing files are overwritten.
% feiListGet image1 '*' after 2000-09-10 replace long
Formated 'after': 2000-254T00:00:00
Press the <enter> or <return> key to skip file.
Type 'g' to get the file.
feiListGet/<image1>@Get " 2000-253T11:16:40.553, 7, file2":
feiListGet/<image1>@Get " 2000-253T11:16:40.773, 8, file25":
feiListGet/<image1>@Get " 2000-253T11:16:41.026, 8, file26": g
1. Got "file26".
feiListGet/<image1>@Get " 2000-253T11:16:41.273, 8, file27":
feiListGet/<image1>@Get " 2000-253T11:16:41.513, 8, file28":
feiListGet/<image1>@Get " 2000-253T11:16:42.023, 7, file3": g
1. Got "file3".
feiListGet/<image1>@Get " 2000-253T11:16:42.483, 8, file31":
feiListGet/<image1>@Get " 2000-253T11:16:42.973, 8, file33": ^C
%
The listing returns one line at a time as a prompt. If we want the file, we just type a 'g' next to the listing, and if we don't want the file we just press <end> or <return>. When the list is exhuasted, the program exists. In our example we exited early by sending the program a GETINT signal using the key combination of enter and <control-c>.
Just like feiGet, feiListGet allows the user to get the files from the server in a directory of the user choice. To get the files in the directory of the user choice , the user is allowed to specify the absolute path name of the directory on the command line in which to receives files from the server. Let us run this utility again. This time we will get the files from the server in the "Sub" directory by issuing the following command:
% feiListGet image1 output /home/Test/Sub '*' after 2000-09-10 long
Formated 'after': 2000-254T00:00:00
Press the <enter> or <return> key to skip file.
Type 'g' to get the file.
feiListGet/<image1>@Get " 2000-253T11:16:40.553, 7, file2":
feiListGet/<image1>@Get " 2000-253T11:16:40.773, 8, file25":
feiListGet/<image1>@Get " 2000-253T11:16:41.026, 8, file26": g
1. Got "file26".
feiListGet/<image1>@Get " 2000-253T11:16:41.273, 8, file27":
feiListGet/<image1>@Get " 2000-253T11:16:41.513, 8, file28":
feiListGet/<image1>@Get " 2000-253T11:16:42.023, 7, file3": g
1. Got "file3".
feiListGet/<image1>@Get " 2000-253T11:16:42.483, 8, file31":
feiListGet/<image1>@Get " 2000-253T11:16:42.973, 8, file33": ^C
%
Sometimes we'd like to see the contents of a file. For text files, this just means using the feiDisplay command:
% feiDisplay
Copyright 2001, The California Institute of Technology. All rights reserved.
FEI API Version 3.2.0, August 1, 2001
feiDisplay <file type> <file name> {[encrypt] [crc]}
Here we add the file face.text and then get it back and display its contents.
% feiAdd image1 face.text
Adding to file type "image1":
1. Added "face.text".
% feiDisplay image1 face.text
ooo%%%%%%%%%%%%oooo
oo%%%%%%%%%%%%%%%%%%%%%%%%o
oo%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%o o% %% o%
o % oo o%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%o %% %% %%o%
oo % % "% o%%%%%%%%% %%%%%%%%%%%%% %%%%%%%%%o %%%o%%o%
"%%%%%%o% o%%%%%%%%% %%%%%%%%%%% %%%%%%%%%%o %%%%%%%%
%%%%%%% %%%%%%%%%%% %%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%% %%%%%%%%%%%%%% """%%%
"%%%""""%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% "%%%
%%% o%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% "%%%o
o%%" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%o
%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%" "%%%%%%ooooo%%%%o
o%%%oooo%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% o%%%%%%%%%%%%%%%%%
%%%%%%%%"%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%""""""""
"""" %%%% "%%%%%%%%%%%%%%%%%%%%%%%%%%%%" o%%%
"%%%o """%%%%%%%%%%%%%%%%%%"%%" %%%
%%%o "%%""%%%%%%"""" o%%%
%%%%o oo o%%%"
"%%%%o o%%%%%%o"%%%%o o%%%%
"%%%%%oo ""%%%%o%%%%%o o%%%%""
""%%%%%oooo "%%%o%%%%%%%%%"""
""%%%%%%%oo %%%%%%%%%%
""""%%%%%%%%%%%
%%%%%%%%%%%%
%%%%%%%%%%"
"%%%""""
The program quickView reads an image from stdin and displays it if it knows the image type. We can pipe the output from feiDisplay to quickView to get a quick look at an image.
% feiDisplay image1 | quickView tiff
Two subscription modes are supported by FEI utilities: feiSubscribe and feiNotify. feiSubscribe sends us a copy of each new or updated file as it arrives in the file type to which we're subscribing. feiNotoify works like the feiList command, except it continually adds file information as new files arrive in the system. (The file's content is not transfered when using feiNotoify. That's how if differs from feiSubscribe.)
As an example, using subscription and notification, we're going to set-up a test subscription session. You can use the same procedure when you are doing your system test to see how everything plays together.
Our mock session requires that we use two FEI utilities designed just for testing: feiFileMaker and feiMakeClean. feiFileMaker automatically generates files at regular intervals and either adds or replaces them in FEI for the given file type. feiMakeClean deletes a large set of files in a file type with a single command. We'll use this to clean up the test file type before we run test sessions. Note: Use feiMakeClean carefully and only for file types that are used for testing! In an operational system you should only use feiDelete.
We can give feiMakeClean a file name expression, and since we're only going to be using files with names like "file...", we'll just clean up that type of file. Do this to begin with and then do it again whenever you finish a test subscription session:
% feiMakeClean image1 'file*'
1. Deleted file "file105".
2. Deleted file "file106".
3. Deleted file "file107".
. . .
145. Deleted file "file98".
146. Deleted file "file99".
147. Deleted file "file13".
Now we can use feiFileMaker to automatically generate files at regular intervals. The format of the command we'll use is:
% feiFileMaker image1 delay 5
This will generate a sequence of files with names like file1, file2, ... and send each newly created file to the FEI file type "image" with a delay of approximately 5 seconds between postings. The contents of the file is just the name of the file. This is an advantage in our case because we don't want to use a lot of space for our testing. It has another advantage as well. For small files, any time delays we see between the time the file reaches the FEI server and the time our subscription session gets a copy represents system overhead. During system testing it may be important to know how long it takes to send a file of a certain size and what part of that is overhead. (By the way, to send a larger file, pick one of your own, and have feiFileMaker use the content of that file for each test file it generates. The command is:
% feiFileMaker image1 delay 5 using myFile
Running a subscription session using feiSubscribe and feiNotify is easy. Just follow these steps:
Here's the output for the three processes from a sample session.
% feiFileMaker image1 for replace delay 5
1. replaced "file1", transaction took 0.000 seconds.
2. replaced "file2", transaction took 0.000 seconds.
3. replaced "file3", transaction took 0.000 seconds.
4. replaced "file4", transaction took 0.000 seconds.
^C
Normal shut down.
Operation completed.
% feiSubscribe image1
FEI Information on 2000-254T15:43:25
PID:2008. Subscribing to "image1" file type.
FEI Information on 2000-254T15:43:30
File type [image1]: received "file1" 7 bytes 2000-254T15:43:30.276
FEI Information on 2000-254T15:43:35
File type [image1]: received "file2" 7 bytes 2000-254T15:43:35.400
FEI Information on 2000-254T15:43:40
File type [image1]: received "file3" 7 bytes 2000-254T15:43:40.396
FEI Information on 2000-254T15:43:46
File type [image1]: received "file4" 7 bytes 2000-254T15:43:45.886
^C
FEI Information on 2000-254T15:44:10
Disconnected from FEI. Start shutting down.
FEI Information on 2000-254T15:44:10
Shutdown complete. Normal termination.
FEI allows the user to subscribe files from server to a directory of the user choice. To receives files in a directory of the user choice during the subscription the user is required to supply an absolute path name on the command line.
% feiNotoify image1
FEI Information on 2000-254T15:48:24
PID:2009. Listing files of type "image1".
1. "file1"
2. "file2"
3. "file3"
4. "file4"
^C
FEI Information on 2000-254T15:44:10
Disconnected from FEI. Start shutting down.
FEI Information on 2000-254T15:44:10
Shutdown complete. Normal termination.
Once the user halts the subscription session, the feiSubscribe utility generates the restart ("image1.subscribe" and its backup "image1.subscribe~") files. The loaction of the restart files and the files received from the server is determined by the absolute pathname supplied on the command line by the user at the time of the subscription. If the user had supplied the absolute path name at the time of the subscription, the feisubscribe utility transferred the incoming files from the server to the directory pointed by the pathname and also stored the restart files in the same directory. If the absolute pathname was not used by the user at the time of subscription the utility writes both incoming files ,from the server, and the restart files in the current directory.
The subscription session can be started from any directory as long as the user supplies the absolute pathname of the directory, on the command line, where the restart file is located. The example below will clear the points explained above. Let us start the subscription only this time use the absolute path name and run this command from the "Test" directory.
% feiSubscribe image1 output /home/Test/Sub
The files from the server will be transferred to the "Sub" directory. Let the subscription run for a while and then halts the subscription by using the combination of enter and <control-c>. The feisubscribe utility before quitting generates the "image1.subscribe" and its backup "image1.subscribe~" file in the /home/Test/Sub directory. If the user later on wishes to restart the subscription then the user is required to supply the same absolute path name because the feisubscribe utility is looking for the restart file and the restart files are in the "Sub" directory. The command below shows the correct syntax:
% feiSubscribe image1 /home/Test/Sub restart
Once you're comfortable running the test, remove the test files and run the test again with these changes:
% feiSubscribe image1 restart
We can use an options file to add more parameters to a subscription session. If we want an options file used, its name appears on the command line following the keyword using. This is true in the case where we are restarting a session as well as when starting a new session:
% feiSubscribe image1 using /home/jake/fei/options.image
% feiSubscribe image1 restart using /home/jake/fei/options.image
% feiNotify image1 using /home/jake/fei/options.image
% feiNotify image1 restart using /home/jake/fei/options.image
If the options file is not in the current directory, we must include the absolute path to the file.
A sample options file is included in the directory $FEI_ROOT/misc. You may want to get a copy of it and use it as a template when building your own options files. The file's content is reproduced below. For a complete description of each option see the Command Line Reference Document. In this section we'll just describe what the example options file does.
#########################################################################
#
# File
# options.image
#
# Function
# Sample Subscription Session options file. If you use this file as
# a template, make sure you change the path's and email addresses used
# as example values. Commands not used are commented out in this example.
#
# Option files can include comment lines beginning with the character
# '#', empty lines composed of white spaces, and command lines like
# those shown below. Each command must appear on a line by itself.
#
# Creator
# T. Huang, JPL
#
# Created
# October 25, 2000
#
# History
#
#########################################################################
######## File transfer options ########
#encrypt
#crc
######## File write options ########
# (use only one or neither)
#version
replace
######## Program to invoke ########
invoke perlScript $fileName
######## Message handling ########
logFile /home/jake/fei/logs/image.log
displayMessages
#limitMessages
bell
mailMessageTo jake@hardknocks.edu
######## Reporting activity ########
mailReportTo jake@hardknocks.edu
mailReportAt 08:30 12:00 18:00
Comment lines in a options file begin with the character '#'. In the example we used them both to document the file and to disable some valid options. In this way, we can use the same basic file for different circumstances.
There are five basic categories of options. All are shown in the example options file.
The email address used for reporting activity is different from the one used for reporting messages (see above). Very often messages are sent to operations while activity reports are sent to investigators.
When the program exits, a final report is mailed. Obviously, there's no time specified for this report; it's automatic.
With the feiDelete command, we can delete files in a file type. The syntax is:
% feiDelete
Copyright 2001, The California Institute of Technology. All rights reserved.
FEI API Version 3.2.0, August 1, 2001
Batch: feiDelete <using> <option file>
Option File Format (per line):
<file type> <file name>
Interactive: feiDelete <file type> ['<file name expression>']
so the command works for all files or just files matching an expression.
The command does not delete the files outright. Rather it lists the matching files one at a time at a prompt. If we want the file deleted, we type the leter 'd' at the prompt. Otherwise, we just press <return> and the file named is skipped. In the following example we want to delete all files like "file..." that have version numbers:
% feiDelete image1 'file*'
Press <enter> or <return> to skip a file. Type 'd' to delete the file.
Delete "file1.873925933": d
Deleting from file type "image1":
"file1.873925933" deleted
Delete "file1.873928995": d
Deleting from file type "image1":
"file1.873928995" deleted
Delete "file10.873928995": d
Deleting from file type "image1":
"file10.873928995" deleted
Delete "file105":
Delete "file106":
Delete "file11.873928995": d
Deleting from file type "image1":
"file11.873928995" deleted
Delete "file110":
%
To rename a file within a file type, use the feiRename command:
% feiRename
Copyright 2001, The California Institute of Technology. All rights reserved.
FEI API Version 3.2.0, August 1, 2001
Batch: feiRename <file type> <old file name> <new file name>
Batch: feiRename <using> <file name>
Option File Format (per line):
<file type> <old file name> <new file name>
Interactive: feiRename <file type>
It works with command line arguments or in interactive mode. In the following example - interactive mode - we rename some "image" files. We also show what happends for some error conditions.
% feiRename image1
Press the <enter> or <return> key to exit.
feiRename: <old file name> <new file name>
feiRename: file1 imagefile1
Message: File not listed in database
feiRename: file11 imagefile11
feiRename: file12 imagefile12
feiRename: file12 imagefile12
feiRename: file13 imagefile12
Message: File is already listed in database.
feiRename:
%
To compute and display the crc of a file, use the feicrc command:
% feicrc
Copyright 2001, The California Institute of Technology. All rights reserved.
FEI API Version 3.2.0, August 1, 2001
feicrc <file name>
This command does not take any file type as an argument as other fei command do.To run this command simply type this command and the file name,from the local directory, that you want to compute the crc vlaue:
% feicrc file1
ec0aaafcb8f65eefc94b0aabb3d7f968
feiCheckFiles is used to check the files. This utility allows the user to check if the files exists on the server and the local directory. The utility displays four differrent messages to the user. feiCheckFiles utility can be executed in two different Modes:
We will demonstrate the execution of feiCheckfiles utility in both modes by example.
Specifying Input list at the command Line
To check a files in "Specifying Input list at the command Line" mode you would type something like this:
% feiCheckFiles image File1
1. "File1"
The message comes back stating that file "File1" exists both at the server and at the user local directory. Let us rename the "File1" to "FILE1" at the user local directory by issuing:
% mv File1 FILE1
Rerun the feiCheckFiles command :
% feiCheckFiles image1 File1
1. "File1".
- No such file or directory.
Next step is to delete file "File1" from the server and rename "FILE1" back to File1. So that we can test the reverse of the previous step. Use feiDelete command to delete the file "File1" from the server under file type "image1.
% feiDelete image1 File1
The feiDelete utility will ask for confirmation. Go ahead and confirm 'y' to delete the file and confirm 'y'.
% mv File1 FILE1
Rerun the feicheckfiles again:
% feiCheckFiles image1 File1
No matching files found.
When the files is not on the server and on the local disk.
% rm File1
% feiCheckFiles image1
File1
No matching files found.
Specifying Input list in a file passed via the command line
The example below illustrate the " Specifying Input list in a file passed via the command line" mode for feiCheckfiles utility:
feiCheckFiles utility can be executed in a batch mode via the using option. To execute the feiCheckFiles utility in a batch mode the user is first required to generate a text files. The using features of the feiCheckFiles utility allows the user to check files on the FEI server in multiple filetypes. The record structure of the text file to be used with using option is shown below:
"<file type> ['<old file name>] {[since<date_time> [long]}"
Each record in the text file must start with the file type followed by the filename and option. If the "since" option is present then it must be followed by date-time. See FEI Command Line Reference for valid date-time format.The user is allowed to include as many records in the text file as he/she wishes. The user is also allowed to put comments any where in the text file by placing a "#" symbol in the first character of any record. The presence of "#" symbol(s) anywhere other than the first character in the record are considered valid character. For example if the file name expression contain a "#" symbol, the feiCheckFiles utility will consider the filename as valid filename and will process the record. Interactive processing embedded inside the text file is not permitted.
Let us assume that the following files "file1", "file2" under filetype "image1" and files "file1" and "file2", under filetype "image2" are currently stored on the FEI server and you wa nt to check these files from the server. If these files are not on the FEI server go ahead and add these files, under file t ype "image1" and image2", to the FEI server using feiAdd utility. So we build the text files as shown below. We will perform the following check: 1) Check that "file1" is on the FEI server under filetype "image1" and ver fify that the file "file1" exists both on the server and on the local directory. Before the test we will mak e sure that file "file1" exists on the local directory 2) Check that file "file3" under filetype "image2" is not on the FE I server and file "file3" exists on the local directory, 3) Check that file "file2" under filetype "image1" is on the FEI server an d file "file2" is not on the local disk and finally 4) check file "file4" under file type "image2" is not on the FEI server an d not on the local disk. Let us build the text file as shown below:
######################################################
# File Name: check.text
# This is a text file used to check files on the server
# and the local disk.
# Date generated 4/1/2000
# Generated by Gyan V. Ahluwalia
image1 file1
image2 file3
image1 file2
image1 file2
We will run feiCheckFiles utility in the " non verbose mode". As in dicated ealriar in the document this utility can also be run in the "verbose " mode. Remember you do not supply any keywo rd following the file name to run in the "verbose" mode.
% feiCheckfiles using check.text
The following output will be displayed by the utility:
Input File "delete.text"
Using option is used without verbose
Summary of Processing
________________________________________________________
______________________
This record is ignored "################################
#######################
This record is ignored "# File Name: check.text"
This record is ignored "# This is a text file used to check files on the server and the loal disk
This record is ignored "# Date generated 4/1/2000"
This record is ignored "# Generated by Gyan V. Ahluwalia"
1. "file1" found in the server and on the local disk.
Information - The file or filename expression "file3" is not found on the server and the local disk.
1. "file2" found in the server but not on the local disk.
Information - The file or filename expression "file4" is not found on the server and the local disk.
You can verify the result of running this utility.
% feiCheckfiles using check.text
There's another way to get files, and this can be done with the command feiAccept. This command reads a list of files to get from the stdin file stream. Basically, you pipe a list of files to the command. Here's a simple example using the Unix "ls" command. First we list the file to make sure the list contains the file we want to get:
% ls -1 file1*
file1
file10
file11
Now we just pipe that list to feiAccept and output the result to a directory called 'output_dir':
% ls file1* | feiAccept image1 for get output output_dir
1. Getting "file1".
2. Getting "file10".
3. Getting "file11".
We could use feiAccept to enter files by content using "grep" to get the list of file names:
% grep -l "Mars Pathfinder" file* | feiAccept image1 for get output output_dir
If we had a lot of files to enter and didn't want to use interactive mode with feiGet, we could use "ls" to make a list of files, edit the list, and then send the list to feiAccept.
First we get the list of files and write the list to a file.
% ls -1 file3* > 3files
Next, we edit the file names in 3file so only file names ending in 3<n> that are even numbered are included in the list.
% vi 3files
And then we send the list to feiAccept
% feiAccept image1 for get output output_dir < 3files
1. Replacing "file30".
2. Replacing "file32".
3. Replacing "file34".
4. Replacing "file36".
5. Replacing "file38".
You can obtain DS1 and Mars98 subscription related sample scripts and options files by putting your cursor over these links; hold the shift key while clicking your left mouse button and download these file to your local machine.
DS1 examples include:
Mars98 examples include:
With minor changes these scripts will probably work for all users on any project. Contact the FEI Administrator for your project (FEI Administrator) for information on automating these subscription even further using kerberos batch capabilities.
Subscriptions should be submitted a background processes (put an & at the end of the command line). If submitted to the foreground, the subscription process will terminate when you terminate the window (or log off the machine). If you submit a subscription to the foreground by accident (forget to put & at the end of the line), simply use <control-c> to kill the subscription and start it again with the &. Or you can use the UNIX kill command with the processes PID.
Our sample subscriptions included the "restart" parameter which will cause subscriptions to pick-up where they left off if the connection to the FEI server is interrupted.
Also with the sample script are a number of options files. The options used are very basic, users may wish to review the Command Line Utilities Reference for information on other options they may wish to use. If options are used, they must be contained in a file and the file name appears on the command line following the parameter using. This is true in the case where we are restarting a session as well as when starting a new session:
% feiSubscribe test using /usr/local/fei/options.test &
% feiSubscribe test restart using /usr/local/fei/options.test &
If the options file is not in the current directory, you must include the absolute path to the file.
Copyright © 2003 The California Institute of Technology