FEI Versions 3.2.0
Last update on October 25, 2001
Send questions, comments and corrections to MDMS
The FEI application program interface (API) is written in C++. It allows interaction with FEI servers and is used to:
The last item in this list has special importance because the API is multi-threaded (MT), allowing for concurrent operations in your program. You can queue an FEI command and return later to get the result from the result queue.
You don't have to know how to write threaded code to use the API; you just have to be aware of concurrent operations within your program. And, in fact, that's not even true. You can code an interface that doesn't take advantage of concurrancy. In that case, your program appears to execute in a single threaded manner.
FEI uses a transaction processing protocol. You queue a request - like get a file - to the request queue. Once FEI transmits the file to you, a profile object is placed in the result queue. You retrieve the profile from the result queue when you're ready. The profile contains information about the file acted upon - in our case, about the file we just got.
The API manages the request and result queues for you. You simply post commands to the request queue and extact profile objects from the result queue - one profile object for each file acted upon. The queues are what make concurrent processing possible within your program. FEI is reading from the request queue and writing to the result queue while you program is off doing other things. The queue mechanism also hides what's going on in FEI threads from the body of your program so you don't have to be concerned with MT programming.
Since the FEI API is written in C++ you must use new to create an object (unless its on the stack) and delete to deallocate memory used by the object once your done with it.
Objects created on the stack exists within the scope of their creation. For example:
{
Fei fei;
. . .
}
// The object is gone at this point.
Not only is the object gone once you leave the scope of its creation, but it has automatically disconnected from the FEI server if you were connected to one. (This is done when the objects destructor is called.)
If you create an FEI object using new, it exists until you delete it. For example:
Fei *fei = (Fei *)NULL;
{
fei = new Fei;
if (fei == (Fei*)NULL)
{
fprintf (stderr, "Can't create Fei object.\n");
exit (0);
}
}
. . .
delete fei;
If you haven't disconnected from the FEI server at the time you delete the object, the call to the object's destructor will do it for you.
There are other FEI classes that you'll also have to manage. The most important of these is called an FeiProfile. A profile contains information about a file and is queued to the result queue. Once you retrieve it, it's yours, so you have to delete it when your done using it. Also, you can save information contained in a profile, even after the profile is deallocated. If you retain a character string or a buffer associated with a profile object and later want to delete it, use the delete array syntax. The following example retains the file name in a profile object, then deletes the profile object and finally deletes the file name, which was created as a an array of characters.
FeiProfile *profile;
const char *fileName;
while ((profile = fei.result ()) != (FeiProfile *)NULL)
{
fileName = profile->getFileName (FEI_RETAIN);
delete profile;
delete [] fileName;
}
The FEI API is composed of four developer accessable classes:
You can't copy an Fei object. If you try, you'll get a compiler error. Why no copy? Because it makes no sense to have more than one object associated with the same FEI server connection. You can, however, have more than one pointer or reference to an Fei object.
Before you can transmit a file, you must connect to an FEI server. Authentication credentials must be sent to the server - a Kerberos ticket - and the server must confirm you access control rights. Also, once you're finished with a connection, you must disconnect from the FEI server. (If not done explicitly, disconnect is done for you when an Fei object is deleted.)
Here are a couple of examples. The first uses an Fei object on the stack and the second uses the heap. For our examples, we'll connect to a file type named "image". On the stack:
#include "Fei"
Fei fei;
const char *fileType = "image";
if (fei.connect (fileType) != FEI_OK)
exit (FEI_FATAL);
. . .
fei.disconnect ();
On the heap:
#include Fei
Fei *fei = new Fei;
if (fei == (Fei*)NULL)
{
fprintf (stderr, "Can't create Fei object.\n");
exit (FEI_FATAL);
}
const char *fileType = "image";
if (fei->connect (fileType) != FEI_OK)
exit (FEI_FATAL);
. . .
fei->disconnect ();
delete fei;
For the connection to succeed, your program must be run in an environment set up to handle FEI request. Make sure the following steps were followed before you try to connect to an FEI server:
After you disconnect, you can connect again to the same or to a different file type using the same Fei object. You can also test at any time to see if your Fei object is still connected to a server:
if (fei.isConnected () == true)
cout << "Connected." << endl;
else
cout << "Not connected." << endl;
isConnect () returns a boolean value which is either true or false.
This function is also useful for programs that must remain running even during network and server failures. Use isConnect to test for a connection if you receive an error from FEI, and then, try to connect later. For example:
while (1) // processing loop
{
. . .
if (fei.getStatus() == FEI_FATAL)
{
while (fei.isConnected() == false)
{
sleep (300); // sleep 5 minutes and see if things get better.
fei.connect (fileType);
}
}
. . .
}
There's one other way to connect to an FEI server that's used when you restart a subscription session. We'll discuss it when we cover subscribing to a file type. The only point to make now is that on subscription restart, a connection is established automatically for you. You don't use the functions discussed in this section.
Simple file transactions covers listing, adding, getting, replacing, deleting, and renaming files. The commands to add, get and replace files have two flavors: one reads files from and writes files to a file system on disk and the other one uses in memory buffers. We'll look at the ones that use the file system first.
All of these commands assume that you've created an Fei object and connected to an FEI server using a file type You don't include the file type again when you queue a transaction. All transactions are for the file type associated with the connection. Requested are sent to the request queue using a transaction request member function. Here are some examples:
const char *fileName = "image1";
const char *newFileName = "newImage1";
const char *localPath = "/home/jake/images";
int status;
status = fei.add (fileName, localPath);
status = fei.get (fileName, localPath);
status = fei.replace (fileName, localPath);
status = fei.remove (fileName);
status = fei.rename (fileName, newFileName);
Note: Developers on Microsoft systems would define a local path to be something like this:
const char *fileName = "c:\user\myself\products";
The file name is the name of a file in the file system. The local path either describes the directory where the file is found or where it should be placed once it's retrieved, depending upon the type of transactioin. You can specify the current working directory using "." as the value of the local path:
const char *localPath = ".";
Each member functions returns an FEI status value. If all went well, FEI_OK is returned, otherwise, something else, FEI_ERROR in most cases.
The functions we just described queue a transaction request. Since Fei is multi-threaded, the actual transfer takes place sometime in the future - hopefully not too far into the future. That means that these functions return immediately, before the transaction actually takes place. Once again, these functions queue a request. After that your program is free to go off and do other things, only comming back latter to see how things went. The status returned by these functions only tell you whether or not the transaction request was successfully queued.
To see the result of a request, you use the result () member function. There are several ways to code using result (). We'll start with the simplest.
If you want to ignore Fei's MT capabilities use code like this:
FeiProfile *profile;
if (fei.add ("myFile", "."))
{
while ((profile=fei.result ()) != (FeiProfile *)NULL)
{
if (profile->getStatus ())
cout << "Added file " << profile->getFileName() << endl;
delete profile;
}
}
A lot happened in this little piece of code, so we'll go over it point by point.
if (profile->getStatus())
result () is a blocking function. If you don't give it an argument, it blocks until the resultant profile is queued. So this code segment works just like a non-MT program. You make a request and wait for the outcome, testing a couple of status values.Now, we'll change the example to get a set of files using a file name expression, "file*". This returns all files for the given file type whose names match the expression. Here's the code again with the changes:
FeiProfile *profile;
if (fei.get ("file*", "."))
{
while ((profile = fei.result ()) != (FeiProfile *)NULL)
{
if (profile->getStatus ())
printf ("Got file %s\n", profile->getFileName ());
delete profile;
}
}
We now get back a set of profile records. (Think back to school for a moment and remember that a set is always something, even if its the empty set.) Suppose, our regular expression returned nothing, we have the empty set, which we represent by the NULL profile. So our code works. In the case where no files are returned, result () returns the NULL profile right away. If we do get back results, we don't know how may there are until we see the NULL profile object. Now inclosing result () in a while loop makes sense.
Here's the rule:
Every transaction returns a set of results represent by one profile object for each file involved in the transaction. When we have retrieved all of the profile objects, we are left with the empty set (only the NULL profile record in the queue), signalling the end of transaction results.
So far our examples work just like single-threaded programs. We ask for something and wait until we get it. Remember we said that result () was a blocking function? So far we've blocked for ever, but we can set the delay time to something shorter and pass it as the arugment to result.
long delay = 10; //seconds
fei.result (delay);
So now, let's just delay for 10 seconds waiting for our transaction to complete. result () looks in the result queue for a profile object, and continues to look for up to 10 seconds. If it finds a queued object before the 10 seconds expire, it returns immediately with the profile. Otherwise it returns at the end of 10 seconds. We can now go on and do other things. But we're not done with the transaction yet. We have to come back to result until the transaction is complete. When isn't the transaction complete? Consider these cases after waiting 10 seconds for a result, remembering that the transaction is being processed in its own thread.
Clearly we need to know when a transaction is complete when using a delay in a MT environment. And Fei has a member function that does that:
fei.transactions ()
This function returns the number of outstanding transactions - more than one may have been queued.
Let's rewrite the last example. This time in an MT context using a delay value and a transactions count. (Remember, the reason, we want to use the delay is so we can go off and do something else while were waiting for FEI, so we'll add a function to do something as well.
FeiProfile *profile;
long delay = 10; // seconds
int status;
if (status = fei.get ("file*", "."))
return (status);
while (fei.transactions () > 0)
{
while ((profile = fei.result (delay)) != (FeiProfile *)NULL)
{
if (profile->getStatus ())
cerr << "Got file " << profile->getFileName() << endl;
delete profile;
}
// Do something else each time the delay time expires until the
// transaction completes.
doSomething ();
}
Note: You only neet to code this way when result () includes a delay value. Without a delay value, result blocks until all the queued transactions complete.
You can use a delay value of 0, in which case result () returns immediately with or without a profile object for a file - but always with a final NULL profile object.
We'll look at one last MT situation that you might have to consider. Let's start with an example that adds 3 files. We want to know when each individual transaction completes, not the set of transactions as a whole. All that we said above still applies, but now we need to look at the transactions () value rather than just wait until it reaches 0.
fei.add ("file1", ".");
fei.add ("file2", ".");
fei.add ("file3", ".");
FeiProfile *profile;
long delay = 10; // seconds
int oldCount = fei.transactions ();
int count;
while ((count=fei.transactions()) > 0)
{
while ((profile=fei.result(delay)) != (FeiProfile *)NULL)
{
if (profile->getStatus ())
cout << "Got file " << profile->getFileName () << endl;
delete profile;
if (oldCount != count)
{
cout << "End of transaction " << count << endl;
oldCount = count;
}
// Do something else each time the delay time expires until the
// transactions complete.
doSomething ();
}
}
We queue three transaction requests and then save the number of requests in oldCount. In the transactions () loop we get the current count. As we retrieve results, transactions will complete and the count is decremented. We note this and set oldCount to the new value of count so we can detect the next change.
To send and receive the contents of a file in memory, we a void buffer and it size_t size. For add and replace operations, you create and delete the buffer, sending its address and size to FEI as transaction function arguments:
const char fileName = "myFile";
size_t size = 1048576; // 1 meg
void *buf = (void *) new char[size];
status = fei.add (fileName, buf, size);
status = fei.replace (filename, buf, size);
To recieve file content's in memory, the get function's localPath argument is given a special value: FEI_IN_MEMORY. The buffer holding the file's contents and the buffer's size are returned as part of the file's profile object and accessed with the FeiProfile member functions void *getAddress () and size_t getSize:
status = fei.get (fileName, FEI_IN_MEMORY);
void *buf = profile->getAddress ();
size_t bufSize = profile->getSize ();
Note: The file content buffer was created with new, so, if you delete it, use delete like this:
delete [] buf;
Note: The size of the buffer is the size of the file's content without any special trailing character like a zero byte or newline.
Here's an example where we add file "myFile" filled with a megabyte of "a"'s.
const char fileName = "myFile";
const size_t size = 1048576;
void *buf = (void *) new char[size];
FeiProfile *profile;
memset (buf, (int)'a', size);
if (fei.add (fileName, buf, size))
return;
while ((profile=fei.result()) != (FeiProfile*)NULL)
{
if (profile->getStatus ())
saveOnError (profile->getAddress(FEI_RETAIN), profile->getSize());
delete profile;
buf = (void *)NULL;
}
You've seen examples like this before; there's just one new thing to consider. Who controls the allocated memory buffer? Answer: Whoever has it at a particular time. And since we passed it to Fei with the add () command, Fei has it (or a reference to it) after that. So don't change its contents after adding it, which you could in an MT world. When the result of the add is returned in a profile, the profile contains the address of the buffer. If everything went well, you many want to delete the profile, which in turn deletes the space allocated for the buffer since it's part of the profile. If that's not what you want, and we don't in the example when there's an error, use the profile member function getAddress (FEI_RETAIN). Normally this function is used to return the address of the buffer, but we use it just to tell the profile object not to delete the file content buffer by passing FEI_RETAIN as the function's argument.
Why all this business of passing around the ownership of the buffer? It's an MT thing. If you have several transactions queued, it would not be easy to manage file content buffers because you don't know what ultimately happens to a transaction until the profile object turns up. And which buffer do you associate with which profile record in that case? So, the profile object returns the address to you - and will clean up for you unless you tell it not to.
Here's an example where we get a set of files, keeping their contents in memory:
FeiProfile *profile;
long delay = 10;
if (fei.get ("file*", FEI_IN_MEMORY))
return;
while (fei.transactions () > 0)
{
while ((profile=fei.result(delay)) != (FeiProfile*)NULL)
{
if (profile->getStatus ())
doSomethingWith (profile->getAddress (FEI_RETAIN), profile->getSize());
delete profile;
}
}
When you delete, rename or list files, the content of files are not transfered, but you still get a profile record for each file acted upon. We'll look at getting a list of files as an example. Coding for delete and rename is about the same.
Let's get all the file's whose names begin with "image..." and that have an FEI modification date later than July 4, 1997 at 5:00 PM, about the time we got an image from Mars Pathfinder.
FeiProfile *profile;
if (fei.list ("image...", ".", "4 Jul 1997 5:00PM"))
return;
while ((profile=fei.result()) != (FeiProfile*)NULL)
{
if (profile->getStatus ())
cout << profile->getFileName() << endl;
delete profile;
}
Note: The date used as a list argument is passed on to a database, so it's formatted to be included in a database SQL query. You could also use this form:
7/4/97 17:00
Since we're only getting data about files, FEI returns just a profile object. No file is transfered in this case. But the code looks just the same in both cases.
Note: If you want to see the list of new files entering FEI for a particular file type as they arrive, don't use list in a tight loop. It's just a waist of resources. Use liveList instead. We'll discuss this in the next section.
A subscription session can be implemented in one of three ways depending on your objective:
When using one of these functions, the Fei object is continually in use. You can't use it to execute other transactions. A subscription session last until the Fei object receives a disconnect () command.
Here are the transaction member functions used to implement a subscription session:
const char *localPath = "/home/jake/images";
const char *restartFileName = "image.restart";
status = fei.subscribe (localPath, restartFileName);
status = fei.notify (restartFileName);
status = fei.liveList (localPath, restartFileName);
To restart a subscription session Fei saves the current state of the session to a file, the name of which you provide to one of the subscription functions. (You don't have to supply this name. In that case restart is not performed for the session.) You can use any naming convention you like, but we prefer the syntax:
<file type>.restart
But be careful. If two processes use the same name and write the file to the same directory the information is obviously of no use and can't be used to restart the session. Also, Fei writes a new restart file each time a profile is recieved, so the content of the old file is lost. Caveat emptor!
One more thing about restart files; there's actually two copies of the file: one with the name you give it and another with a tilde (~) appended to the same name. Fei saves two state objects: the new current state and the state before that - called the backup restart state. The restart process examines the restart file to determine if it was corrupted; maybe your machine went down at the moment the file was being written. If that is the case, Fei attempts to use the backup restart file. In the unlikely event that that process also fails, you have to resynchronize the subscription session manually by going out and getting the files missed between two sessions.
Here's a code segment for creating the restart file name. It uses the file type supplied by the user and an environmental variable you defined to create the file name:
const char *delim = "/";
const char *extension = ".restart";
const char *fileType = argv[1]; // user supplied
char *path = getenv ("FEI_RESTART");
if (path)
{
size_t size = strlen (path) + strlen (fileType) +
strlen (delim) + strlen (extension) + 1;
restartFile = new char[size];
sprintf (restartFile, "%s%s%s%s", path, delim, fileType, extension);
}
You code the restart of the subscription session something like this:
Fei fei;
if (fei.restart (restartFile))
exit (Fei_FATAL);
You may already have an Fei object defined. The example shows its creation for one important reason: you don't explicitly connect to FEI during restart. That is handled for you. Fei restores the session state, all of it, when it restarts a session. That's why there's no connection; we don't want someone to inadvertantly use the wrong file type or use a connection that's already in service.
Two final notes on restart:
Subscription works like the basic functions except that you don't continue to queue transfer requests. You can return the files contents either to memory or to a file:
int status = fei.subscribe (FEI_IN_MEMORY, restartFile);
const char localPath = "/home/me/images/";
int status = fei.subscribe (localPath, restartFile);
You get the files in the usual way. We'll assume we're getting them back into memory because we want to display the contents as an image. (We're going to leave out some error checking that we've shown in previous examples to shorten the example.)
Fei fei;
if (restart)
{
restart (restartFileName (fileType));
}
else
{
fei.connect (fileType);
fei.subscribe (FEI_IN_MEMORY, restartFileName (fileType));
}
bool shutDown = false;
while (1)
{
if (shutDown) break;
displayImage (fei.result (0));
}
fei.disconnect ();
cleanUp (fei);
Here are a few things to notice:
Suppose your receiving images from a polar orbiting satelite and you only want those images that cover a specific part of Antarctic. The latitiude and longitude covered by each image is placed in a database table that you have access to. In this case you don't want all of the images, just those of interest to you which you can determine with a database query using the file type and file name of the image. This is a situation where notify () should be used.
Notify used to subordinate member functions:
FeiProfile *notifyProfile, accpetProfile;
fei.notify (FEI_IN_MEMORY, restartFile);
while ((notifyProfile=fei.result()) != (FeiProfile*)NULL)
{
fei.acceptTheFile (notifyProfile);
acceptProfile = fei.result ();
}
Notify serializes transfers until it receives an accept or reject notification for the last file sent. That means there is only one file in the notify queue at a time. For notify, each time the server delivers a profile record it waits for the client to respond with accept or reject and then delivers another profile if the client accepts. If this wasn't done, the result queue would have a mixture of notify and accept profiles and you'd have a difficult time determining the context of the next profile object.
Notice that only result can be used to block processing. Accept returns immediately after queue the request. In that sense acceptTheFile is another transfer function just like add, get or notify itself. The subtlty of notify is a consequence of Fei's MT flavor which always allows you to do something else while Fei functions in its own thread. With power goes responsibility.
Here's a more complete example illustrating the points just made:
Fei fei;
if (restart)
{
restart (restartFileName (fileType));
}
else
{
fei.connect (fileType);
fei.notify (FEI_IN_MEMORY, restartFileName (fileType));
}
FeiProfile *notifyProfile;
FeiProfile *acceptProfile = (FeiProfile *)NULL;
bool shutDown = false;
while (1)
{
if (shutDown) break;
if (notifyProfile = fei.result (0))
{
if (weWantIt (profile))
{
fei.acceptTheFile (profile);
acceptProfile = fei.result ();
}
else
fei.rejectTheFile (profile);
}
displayImage (acceptProfile); //displayImage deleted
acceptProfile = (FeiProfile *)NULL; //profile record sent to it.
}
fei.disconnect ();
cleanUp (fei);
This example is similar to the last so we'll just describe what's new.
The list () member function always causes a database search for targeted files. If we just want to create a list of files as they arrive, using list () is very inefficient. This is a stituation that calls for the use of liveList (). Here we just get a profile record for each file as it arrives. Later we can select form the list and get the set of files we want. For example:
Fei fei;
if (restart)
{
restart (restartFileName (fileType));
}
else
{
fei.connect (fileType);
fei.liveList (restartFileName (fileType));
}
bool shutDown = false;
while (1)
{
if (shutDown) break;
displayList (fei.result (0));
}
fei.disconnect ();
cleanUp (fei);
This is almost the same as the subscribe example. We won't comment on it further.
You can set one or more options to effect the way Fei handles files. The options come in two flavors: those that effect file transfer and those that effect the storage of files.
Options are set with the Fei member function setOptions (). The function returns the current options mask as type FeiOptions. The default option set is {skipFile}. You can reset to the default options set invoke setOptions () with no parameter.
replaceFile, skipFile and versionFile, are mutually exclusive. If you attempt to define more than one, the following precedence rule is applied when setting the mask value: skipFile : versionFile : replaceFile - with skipFile having the highest precedence.
setOptions () effects only only files transfered by transfers request made after it's execution. Any files being transfered or already in the result quest are not effected.
FeiOption oldMask = fei.setOptions (encrpyt | crc | replaceFile);
fei.setOptions (oldMask);
fei.setOptions ();
The FEI distribution comes with example programs.
Copyright © 2001 The California Institute of Technology