jpl.mipl.mdms.FileService.io
Class BufferedStreamIO

java.lang.Object
  jpl.mipl.mdms.FileService.io.BufferedStreamIO

public class BufferedStreamIO
extends Object

Purpose: Hides the complexity of "attaching" an output stream to an input stream for the purpose of file transfers and file transfers with message digest. This class has been revised to allow partial file transfer, which enables resume transfer in the case of network failure.

 Copyright 2008, California Institute of Technology. 
 ALL RIGHTS RESERVED. 
 U.S. Government Sponsorship acknowledge (NASA contract NAS7-918). 2008.
 

 ============================================================================
 Modification History :
 ----------------------
 
 Date              Who              What
 ----------------------------------------------------------------------------
 01/01/1999        MDMS             Initial Release
 07/08/2005        Nick             Initial documentation.
                                    Swapped "rwd" for "rw" for RandomAccFile 
                                    for server file writing.
 07/15/2005        Nick             Set raf.setLength() to occur after file 
                                    fully transfers to truncate, not before.
 06/01/2008        Nick             CRC placed in a read loop. 
 ============================================================================
 

Version:

$Id: BufferedStreamIO.java,v 1.40 2008/08/07 21:40:59 awt Exp $

Author:

Thomas Huang (Thomas.Huang@jpl.nasa.gov), Nicholas Toole (Nicholas.T.Toole@jpl.nasa.gov)

Constructor Summary

BufferedStreamIO(BufferedInputStream bis, BufferedOutputStream bos)
Constructor using default buffer size for buffered I/O streams.

BufferedStreamIO(BufferedInputStream bis, BufferedOutputStream bos, int bufSize)
Constructor with specified buffer size for buffered I/O streams.

BufferedStreamIO(BufferedInputStream bis, int inBufSize, BufferedOutputStream bos, int outBufSize)
Constructor with specified buffer sizes for each buffered stream object.

Method Summary

byte[]	genChecksum(byte[] buffer) Generate a checksum on a buffer.
byte[]	readAndVerifyBufferFromStream(byte[] buffer, long fileSize) Reads from the (buffered) input stream an writes it to file.
byte[]	readAndVerifyFileFromStream(String fileName, long fileSize) Reads from the (buffered) input stream and writes it to file.
byte[]	readAndVerifyFileFromStream(String fileName, long offset, long length) Reads from the (buffered) input stream and writes it to file with the specified offset.
void	readBufferFromStream(byte[] buffer, long fileSize) Reads from the (buffered) input stream and writes it to the specified byte buffer.
void	readBufferFromStream(byte[] buffer, long fileSize, boolean flushChecksum) Reads from the (buffered) input stream and writes it to the specified byte buffer.
protected byte[]	readChecksum() Reads the checksum from the input buffer and returns it as byte array.
void	readFileFromStream(OutputStream out, long offset, long length) Reads a block of binary data from stream and writes to output stream.
void	readFileFromStream(OutputStream out, long offset, long length, boolean checksum) Reads a block of binary data from stream and writes to output stream.
void	readFileFromStream(String fileName, long fileSize) Reads from the (buffered) input stream and writes it to file
void	readFileFromStream(String fileName, long offset, long length) Reads a block of binary data from stream and write to output file.
String	readLine() Get a line from input stream
MessagePkg	readMessage() Construct a message from a reply packet.
void	skipFile(long fileSize, boolean doChecksum) Method to flush an incoming file, and flush an incoming checksum, if required.
void	write(String s) Write string to output stream
byte[]	writeAndVerifyBufferToStream(byte[] buffer, long fileSize) Writes a file from memory to to the (buffered) output stream A message digest will be cloned and the reference calculatedChecksum updated.
byte[]	writeAndVerifyFileToStream(String fileName, long fileSize) Writes the input file onto buffered output stream.
void	writeAndVerifyFileToStream(String fileName, long fileSize, byte[] suppliedChecksum) Writes the input data buffer onto buffered output stream followed by the supplied checksum value.
byte[]	writeAndVerifyFileToStream(String fileName, long offset, long length) Writes the specified byte range within the input data file to buffered output stream and caclculates the checksum as the data is being transfered.
void	writeAndVerifyFileToStream(String fileName, long offset, long length, byte[] suppliedChecksum) Writes the data block within the input data file onto buffered output stream followed by the supplied checksum value.
void	writeFileToStream(byte[] buffer, long fileSize) Writes the input data buffer to buffered output stream.
void	writeFileToStream(String fileName, long fileSize) Reads a file and writes it to the (buffered) output stream.
void	writeFileToStream(String fileName, long offset, long length) Method to read a data block from an input file and writes it to the (buffered) output stream.
void	writeLine(String s) Write line to output stream
void	writeMessage(int errorNum, String s) Write a message to the command stream.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BufferedStreamIO

public BufferedStreamIO(BufferedInputStream bis,
                        BufferedOutputStream bos)

Constructor using default buffer size for buffered I/O streams.

Parameters:

bis - the BufferedInputStream object

bos - the BufferedOutputStream object

BufferedStreamIO

public BufferedStreamIO(BufferedInputStream bis,
                        BufferedOutputStream bos,
                        int bufSize)

Constructor with specified buffer size for buffered I/O streams.

Parameters:

bis - the BufferedInputStream object

bos - the BufferedOutputStream object

bufSize - input/output buffer size in unit of bytes

BufferedStreamIO

public BufferedStreamIO(BufferedInputStream bis,
                        int inBufSize,
                        BufferedOutputStream bos,
                        int outBufSize)

Constructor with specified buffer sizes for each buffered stream object.

Parameters:

bis - the BufferedInputStream object

inBufSize - input buffer size in unit of bytes

bos - the BufferedOutputStream object

outBufSize - output buffer size in unit of bytes

Method Detail

readFileFromStream

public void readFileFromStream(String fileName,
                               long fileSize)
                        throws IOException

Reads from the (buffered) input stream and writes it to file

Parameters:

fileName - absolute path of file to be written

fileSize - file size in bytes

Throws:

IOException - when file/stream I/O failed

readFileFromStream

public void readFileFromStream(String fileName,
                               long offset,
                               long length)
                        throws IOException

Reads a block of binary data from stream and write to output file. If offset is greater than zero, then the the binary data will be append to the existing file.

Parameters:

fileName - the output file name

offset - the offset byte

length - the lenght of byte stream

Throws:

IOException - when network/file IO failure

readFileFromStream

public void readFileFromStream(OutputStream out,
                               long offset,
                               long length)
                        throws IOException

Reads a block of binary data from stream and writes to output stream.

Parameters:

out - the output file stream, left open on return.

offset - the offset byte

length - the lenght of byte stream

Throws:

IOException - when network/file IO failure

readFileFromStream

public void readFileFromStream(OutputStream out,
                               long offset,
                               long length,
                               boolean checksum)
                        throws IOException

Reads a block of binary data from stream and writes to output stream.

Parameters:

out - the output file stream, left open on return.

offset - the offset byte

length - the lenght of byte stream

checksum - True if checksum follows file contents

Throws:

IOException - when network/file IO failure

readBufferFromStream

public void readBufferFromStream(byte[] buffer,
                                 long fileSize)
                          throws IOException

Reads from the (buffered) input stream and writes it to the specified byte buffer. The method limits the input file size to be less than or equal to Integer.MAX_VALUE (~2GB), since buffering such large file in memory is not recommended.

Parameters:

buffer - data buffer to store data read from input stream.

fileSize - file size in unit of bytes

Throws:

IOException - when network I/O error occurs

readBufferFromStream

public void readBufferFromStream(byte[] buffer,
                                 long fileSize,
                                 boolean flushChecksum)
                          throws IOException

Reads from the (buffered) input stream and writes it to the specified byte buffer. The method limits the input file size to be less than or equal to Integer.MAX_VALUE (~2GB), since buffering such large file in memory is not recommended.

Parameters:

buffer - data buffer to store data read from input stream.

fileSize - file size in unit of bytes

flushChecksum - Flag indicating if checksum needs to be flushed

Throws:

IOException - when network I/O error occurs

readChecksum

protected byte[] readChecksum()
                       throws IOException

Reads the checksum from the input buffer and returns it as byte array. The new byte array will be created with length equal to the digest length.

Returns:

Checksum byte array

Throws:

IOException - if IO error occurs

skipFile

public void skipFile(long fileSize,
                     boolean doChecksum)
              throws IOException

Method to flush an incoming file, and flush an incoming checksum, if required.

Parameters:

fileSize - Size of the file to be thrown away.

doChecksum - Do we need to flush the checksum too?

Throws:

IOException - when file I/O fail

readAndVerifyFileFromStream

public byte[] readAndVerifyFileFromStream(String fileName,
                                          long fileSize)
                                   throws IOException,
                                          VerifyException

Reads from the (buffered) input stream and writes it to file. A message digest will be cloned and the reference calculatedChecksum updated. See MessageDigest for methods to retrieve "checksum": toString() and digest().

Parameters:

fileName - absolute path of file to be written

fileSize - file size in unit of bytes

Returns:

checksum value in byte array

Throws:

IOException - when network/file I/O errors

VerifyException - when checksum unmatched

See Also:

MessageDigest

readAndVerifyFileFromStream

public byte[] readAndVerifyFileFromStream(String fileName,
                                          long offset,
                                          long length)
                                   throws IOException,
                                          VerifyException

Reads from the (buffered) input stream and writes it to file with the specified offset. A message digest will be cloned and the reference calculatedChecksum updated. If this is a partical file transfer, then the message digest will be first calculated from byte 0 to offset before resume transfer-time calculation of message digest. See MessageDigest for methods to retrieve "checksum": toString() and digest().

Parameters:

fileName - the name of the file

offset - the offset value to begin the transfer

length - the number of bytes to be transfered

Returns:

the checksum byte array

Throws:

IOException - when network/file IO failure

VerifyException - when checksum failed to verify.

readAndVerifyBufferFromStream

public byte[] readAndVerifyBufferFromStream(byte[] buffer,
                                            long fileSize)
                                     throws IOException,
                                            VerifyException

Reads from the (buffered) input stream an writes it to file. A message digest will be cloned and the reference calculatedChecksum updated. See MessageDigest for methods to retrieve "checksum": toString() and digest().

Parameters:

buffer - message digest byte array.

fileSize - file size

Returns:

the received, verified checksum

Throws:

IOException - when file I/O fail

VerifyException - when comparisons file fail

See Also:

MessageDigest

writeFileToStream

public void writeFileToStream(String fileName,
                              long fileSize)
                       throws IOException

Reads a file and writes it to the (buffered) output stream.

Parameters:

fileName - absolute path of file to be read

fileSize - file size in unit of bytes

Throws:

IOException - when network/file I/O error occurs

writeFileToStream

public void writeFileToStream(String fileName,
                              long offset,
                              long length)
                       throws IOException

Method to read a data block from an input file and writes it to the (buffered) output stream.

Parameters:

fileName - the file name.

offset - the offset set to begin the transfer

length - the number of bytes to be transfered

Throws:

IOException - when network/file IO failure.

genChecksum

public byte[] genChecksum(byte[] buffer)
                   throws IOException

Generate a checksum on a buffer. A message digest will be cloned and the reference calculatedChecksum updated. See MessageDigest for methods to retrieve "checksum": toString() and digest().

Parameters:

buffer - input message digest byte array.

Returns:

the checksum byte array.

Throws:

IOException - since the buffer was retrieved from network/file.

See Also:

MessageDigest

writeFileToStream

public void writeFileToStream(byte[] buffer,
                              long fileSize)
                       throws IOException

Writes the input data buffer to buffered output stream. Again, the method limits the data buffer to be less than Integer.MAX_VALUE (~2GB), since buffering such large file in memory is not recommended.

Parameters:

buffer - in-memory buffer contains the file.

fileSize - file size in unit of bytes.

Throws:

IOException - when general I/O failure

writeAndVerifyFileToStream

public void writeAndVerifyFileToStream(String fileName,
                                       long fileSize,
                                       byte[] suppliedChecksum)
                                throws IOException

Writes the input data buffer onto buffered output stream followed by the supplied checksum value.

Parameters:

fileName - absolute path of file to be written

fileSize - file size in unit of bytes

suppliedChecksum - the input checksum value to be sent

Throws:

IOException - when network/file I/O failure

See Also:

MessageDigest

writeAndVerifyFileToStream

public void writeAndVerifyFileToStream(String fileName,
                                       long offset,
                                       long length,
                                       byte[] suppliedChecksum)
                                throws IOException

Writes the data block within the input data file onto buffered output stream followed by the supplied checksum value.

Parameters:

fileName - the file name

offset - the offset to begin the transfer

length - number of bytes to be transfered

suppliedChecksum - the supplied checksum array.

Throws:

IOException - when network/file IO failure

writeAndVerifyFileToStream

public byte[] writeAndVerifyFileToStream(String fileName,
                                         long fileSize)
                                  throws IOException

Writes the input file onto buffered output stream. A message digest will be cloned and the reference calculatedChecksum updated. See MessageDigest for methods to retrieve digest: toString() and digest().

Parameters:

fileName - absolute path of file to be written

fileSize - file size in unit of bytes

Returns:

calculatedChecksum message digest byte array.

Throws:

IOException - when network/file I/O failure

See Also:

MessageDigest

writeAndVerifyFileToStream

public byte[] writeAndVerifyFileToStream(String fileName,
                                         long offset,
                                         long length)
                                  throws IOException

Writes the specified byte range within the input data file to buffered output stream and caclculates the checksum as the data is being transfered. This is a partical block, the the bytes 0-offset checksum value will be first calculated before continue on the reset of the transfer.

Parameters:

fileName - the file name

offset - the offset to begin the transfer

length - number of bytes to be transfered

Returns:

the checksum byte array

Throws:

IOException - when network/file IO failure

writeAndVerifyBufferToStream

public byte[] writeAndVerifyBufferToStream(byte[] buffer,
                                           long fileSize)
                                    throws IOException

Writes a file from memory to to the (buffered) output stream A message digest will be cloned and the reference calculatedChecksum updated. See MessageDigest for methods to retrieve digest: toString() and digest().

Parameters:

buffer - file in-memory buffer

fileSize - file size in bytes

Returns:

calculatedChecksum message digest byte array.

Throws:

IOException - when network/file I/O failures

See Also:

MessageDigest

readMessage

public MessagePkg readMessage()
                       throws IOException

Construct a message from a reply packet. See writeMessage() for reply packed format.

Returns:

a MessagePkg object that contains the message just read.

Throws:

IOException - when general I/O failure

readLine

public String readLine()
                throws IOException

Get a line from input stream

Returns:

an '\n' or '\r' delimited line

Throws:

IOException - when read failed

writeMessage

public void writeMessage(int errorNum,
                         String s)
                  throws IOException

Write a message to the command stream. A message consists of integer and a message. The format of the reply is " : ".

Parameters:

errorNum - the input error value.

s - the line to be written

Throws:

IOException - when write failed

writeLine

public void writeLine(String s)
               throws IOException

Write line to output stream

Parameters:

s - the line to be written

Throws:

IOException - when write fails

write

public void write(String s)
           throws IOException

Write string to output stream

Parameters:

s - the string to be written

Throws:

IOException - when write fails