I did not have time to write this out in Scala - this post will contain a few Java snippets. What counts is the concepts - Buffers, Streams, Channels and IO vs NIO vs NIO2. Also, Scala per se' does not natively support a "try with resources' concept - for this use scala-arm https://github.com/jsuereth/scala-arm/blob/master/README.md - ‘for' with resources: <- resource.managed()

Anyhow - enjoy:

I/O Basics

print( ) / println( )

java.io - byte / char stream-based I/O – produces + serializes objects / deserializes objects + consumes information over physical IO device: network connection / memory buffer / disk file

java.nio - buffer / channel-based I/O - complimentary abstraction - open a channel (connection) to an I/O device with a buffer to hold data, then perform IO data operations on the buffer. NIO1 was channel-based I/O. NIO2 supports stream-based I/O.

Using the NIO System - The most common I/O device is the disk file - all file channel operations are byte-based. Eg Open a file via a Path from Paths.get() and populate a ByteBuffer.

IO interfaces: Closeable, DataInput, DataOutput, Externalizable, FileFilter, FilenameFilter, Flushable, ObjectInput, ObjectInputValidation, objectOutput, ObjectStreamConstants, Serializable

IOException – subclasses: FileNotFoundException, SecurityException - requires a security manager eg for applets.

The IO/NIO Packages

• java.io - byte / char stream types

• java.nio – buffer types

• java.nio.channels – channels: open IO connections

• java.nio.channels.spi – service providers

• java.nio.charset – encoders, decoders for byte <--> char

• java.nio.charset.spi - service providers

• java.nio.file - files

• java.nio.file.attribute

• java.nio.file.spi

Buffers

Buffer Class - encapsulates current position (index of next R/W op), limit (index of 1 past the last), and capacity.

methods:

• array(), arrayOffset(), capacity(), clear(), flip(), hasArray(), hasRemaining(), isDirect(), isReadOnly(), limit(), mark(), position(), remaining(), reset(), rewind()

• various get() / put() methods

• allocate() - allocate a buffer manually

• wrap() - wrap an array inside a buffer

• slice() - create a subsequence

derived classes:

• for different types: ByteBuffer, CharBuffer, DoubleBuffer,FloatBuffer, IntBuffer, LongBuffer, ShortBuffer

• MappedByteBuffer – extends ByteBuffer - used to map a file to a buffer.

Streams

2 types of streams: low level byte (binary) and high level char (Unicode)

2 ways to close a stream - 1) explicitly call close() in finally, 2) try-with-resources (AutoCloseable)

Predefined Standard IO Streams

System.in / out

java.lang.System contains 3 predefined stream variables: in (InputStream), out, (PrintStream) and err.

These fields are declared as public, static, and final. may be redirected or wrapped within character-based streams.

Reading Console Input - wrap System.in

BufferedReader br = new BufferedReader(new InputStreamReader(System.in));

After this statement executes, br is a character-based stream that is linked to the console through System.in.

use readLine( ) to read strings:

    do {
      str = br.readLine();
    } while(!str.equals("\n"));

Writing Console Output - System.out.write can output chars – use PrintWriter.print() / println() for string output- System.out references PrintStream

    PrintWriter pw = new PrintWriter(System.out, true);
    pw.println("This is a string");

The Console Class

• System.console() - a Singleton convenience class over System.in / System.out - read from and write to the console. implements Flushable. input methods: readLine(), readPassword().

The Stream Classes

• 4 abstract classes in separate hierarchies.

• InputStream / OutputStream for byte streams - cannot work directly with Unicode characters.

• Reader / Writer for character streams.

• All implement AutoCloseable, Closeable. OutputStream/ Writer additionally implement Flushable. Writer implements Appendable.

• Stream Benefits - clean abstraction, composition of the filtered stream classes

The Byte Stream Classes

stream class names: <catagory><InputStream/OutputStream>

Byte Stream Categories

• Buffered - chunks

• ByteArray - vanilla

• Data – support for standard datatypes

• File

• Filter

• Object

• Piped

• Print – for console out – output only

• Pushback – supports one byte unget – input only

• Sequence – combine 2 streams sequently – input only

Input / Output Abstract Base Classes

• InputStream – methods: available(), close(), mark(), markSupported(), read(), reset(), skip()

• OutputStream – methods: close(), flush(), write()

derived classes overide read( ) / write( )

ByteArray Streams

• ByteArrayInputStream - ctor takes a byte[] as the stream source - not necessary to call close(). if mark() is not called, then reset() sets stream pointer to start of stream

    String tmp = "abcdefghijklmnopqrstuvwxyz";
    byte b[] = tmp.getBytes();
    ByteArrayInputStream input1 = new ByteArrayInputStream(b);

• ByteArrayOutputStream - uses a byte array as the destination - writeTo() convenience method to write the contents of f to test.txt.

ByteArrayOutputStream s = new ByteArrayOutputStream();

byte[] buf = "Hello Ruz".getBytes();

try {

s.write(buf);

...

try(FileOutputStream f = new FileOutputStream("copy.txt")){

s.writeTo(f);

Filter Streams

• FilterInputStream / FilterOutputStream - wrappers (via ctor) around underlying input or output streams that transparently provide some extended functionality

Buffered Streams

• BufferedInputStream / BufferedOutputStream. - size of the buffer is passed in ctor. Ballpark - 8192 bytes. Buffering supports moving backward in the stream of the available buffer. use mark() to remember location use reset() to return to it. Use flush() to write to stream. read() to read bytes from the file.

new BufferedInputStream(Files.newInputStream(Paths.get(“x.txt”)))

Pushback Streams

• PushbackInputStream - extends FilterInputStream with a memory buffer for multibyte operations (improving performance, supports skipping, marking, and resetting of the stream). use pushback() to allow a byte to be read and then returned to the stream --> “peek” . use unread() - pushes back the low-order byte as the next byte returned by a subsequent call to read(). side effect of invalidating mark() - use markSupported() to check the stream.

Sequenced Streams

• SequenceInputStream - use to concatenate multiple InputStreams. When the end of each file is reached, its associated stream is closed.

SequenceInputStream(Enumeration <? extends InputStream> streamEnum)

Print Streams

• PrintStream

• provides output capabilities for file handle System.out ( a PrintStream)

• implements Appendable, AutoCloseable, Closeable, Flushable

• print() / println() - leverages Object.toString() overrides.

• printf() - uses the Formatter class

• format()

Data Streams

• DataOutputStream / DataInputStream - write / read primitive data to / from a stream. implements DataOutput / DataInput interfaces - define methods that convert primitive values to or from a sequence of bytes --> easy serialization: convert values of a primitive type into a byte sequence and then writes it to the underlying stream: eg void writeDouble(double)

Random Access Files

• RandomAccessFile - not derived from InputStream or OutputStream. implements DataInput , DataOutput, which define the basic I/O methods.

• seek() - set the current position of the file pointer within the file

• setLength() - lengthen or shorten a file.

The Character Stream Classes

stream class names: <catagory><Reader/Writer>

Categories

• Buffered

• CharArray – vanilla

• File

• Filter

• InputStream / OutputStream – translators

• LineNumber – counts lines. Reader only

• Piped

• Print – for console out– Writer Only

• Pushback

• String

Reader / Writer Abstract base classes:

• Reader - abstract base. Methods: close(), mark(), markSupported(), read(), ready(), reset(), skip()

• Writer - abstract base. also implements Appendable. methods: append(), close(), flush(), write()

derived classes overide read( ) / write( )

File Streams

• FileReader

try ( FileReader fr = new FileReader("FileReaderDemo.java") ){
      while((i = fr.read()) != -1) System.out.print((char) i);

• FileWriter - will create the file before opening it for output when you create the object. creates a sample buffer of characters by first making a String and then using the getChars() method to extract the character array equivalent.

CharArray Streams

• CharArrayReader - uses a char[] as the source.

CharArrayReader(char array [ ], int start, int numChars)

• CharArrayWriter - uses a char[] as the destination.

Buffered Streams

• BufferedReader - improves performance by buffering input. Specify buffer size in ctor.

• BufferedWriter - buffers output.

Pushback Streams

• PushbackReader - allows one or more characters to be returned to the input stream. Unread() - returns characters to the invoking input stream.

Print Streams

• PrintWriter - a character-oriented version of PrintStream. works like printf().

Flushable Interface

• force buffered output to be written to the stream to which the object is attached. flush() causes buffered output to be physically written to the underlying device.

Try With Resources

• java.lang.AutoCloseable Interface - support for try-with-resources (note: resource declared in the try is implicitly final. can manage multiple resources seperated by semicolon.)

• java.io.Closable Interface– extends Autoclosable. automates closing a resource - close() closes the invoking object, releasing resources. implemented by stream classes. Automatically Closing a File via try-with-resources:

    try(FileInputStream fin = new FileInputStream(args[0])) {
      do {
        i = fin.read();
        if(i != -1) System.out.print((char) i);
      } while(i != -1);

    }catch(FileNotFoundException e) {
        System.out.println("File Not Found.");
    }catch(IOException e) {
        System.out.println("An I/O Error Occurred");
    }

Serialization

• Write object state to a byte stream --> for persistent storage or RMI. Considerations: objects relationships should be DAGs.

• Serializable interface – implementing class (& all of its subclasses) are serializable. Static fields and fields declared as transient opt out of serialization.

• Externalizable interface - Extensible serialization. Methods: readExternal(ObjectInput inStream), writeExternal(ObjectOutput outStream)

• ObjectOutput interface - extends DataOutput / AutoCloseable interfaces and supports object serialization. Methods: close(), flush(), write(), writeObject()

• ObjectOutputStream class - Extends OutputStream, mplements ObjectOutput. for writing objects to a stream.

• ObjectInput interface - extends DataInput , AutoCloseable

• ObjectInputStream - extends InputStream , implements ObjectInput

Channel Interface

represents open connection to I/O source / destination. extends Closeable, AutoCloseable.

obtain a channel by calling getChannel() on an object that supports channels:

• FileinputStream / FileOutputStream

• RandomAccessFile

• Socket / ServerSocket / DatagramSocket

• Files (via static SeekableByteChannel)

derived interfaces: FileChannel, SocketChannel, SeekableByteChannel

support various read() and write() methods

support additional channel access & control methods

FileChannel Class

get / set current position, transfer information between file channels, get size , lock the channel. provides a static method called open(), which opens a file and returns a channel to it. the map() method, which lets you map a file to a buffer.

Charsets and Selectors

• A charset - defines the way that bytes are mapped to characters.

• An encoder / decoder - encode / decode a char sequence into bytes / byte sequence into chars. defined in the java.nio.charset package.

• A selector - supports key-based, non-blocking, multiplexed I/O - enable you to perform I/O through multiple channels. defined in the java.nio.channels package. most applicable to socket-backed channels.

NIO Manual Channel File I/O

Manually Read from a File via a Channel

• Paths.get() – specify & open a Path to file. (defaults to RO)

• establish a channel to file: Files.newByteChannel() - returns a SeekableByteChannel interface object cast to FileChannel class ( implements AutoCloseable).

• allocate a buffer - used by the channel: wrap an existing array or ByteBuffer.allocate() to allocate dynamically.

• SeekableByteChannel.read(ByteBuffer buf) - fills buffer to capacity with data from the file. Sequential reads – call repeatedly. returns #bytes read or −1 at EOF (AutoCloseable uses this).

• load buffer with data from file - ByteBuffer.rewind() - reset to start --> read bytes by ByteBuffer.get(). bytes are cast to char so file can be displayed as text. (optionally create a char buffer to encodes bytes as they come in)

• streamlined sa single try with resources block calls Paths.get() and newByteChannel()

try (SeekableByteChannel fChan=Files.newByteChannel(Paths.get("x.txt"))){

ByteBuffer mbuf = ByteBuffer.allocate(128);

do {

int count = fChan.read(mbuf); //read from channel into buffer

if (count != -1) {

mbuf.rewind(); // so it can be read

for (int i=0; i < count; i++) {

System.out.print((char) mbuf.get());

}

}

}while (count != -1);

Manually Write to a File via a Channel

• specify StandardOpenOption.WRITE / CREATE.

• write data to buffer using ByteBuffer.put() - advances the current position.

• reset to start of buffer via rewind() before calling write().

• alternativelly call flip() instead of rewind() - sets value of current position to 0 and limit to previous current position.

for(int h=0; h<3; h++) {	// Write some bytes to the buffer.
  for(int i=0; i<26; i++)
    mBuf.put((byte)('A' + i));
  mBuf.rewind();			// Rewind the buffer so that it can be written.
  fChan.write(mBuf);		// Write the buffer to the output file.

NIO Automatic file IO via mapped buffer

Read from a file mapped buffer

• Cast retval of Files.newByteChannel() to FileChannel.

• map channel to a buffer: FileChannel.map() - returns MappedByteBuffer - extends ByteBuffer. Params:

• MapMode enum - values: READ_ONLY / READ_WRITE / PRIVATE (make private copy of file - changes to buffer do not affect underlying file).

• pos - location within file to begin mapping

• size - number of bytes to map

• read file from that buffer.

try (FileChannel fChan = (FileChannel)Files.newByteChannel(Paths.get("x.txt"))){

long fsize = fChan.size();

MappedByteBuffer mbuf = fChan.map(FileChannel.MapMode.READ_ONLY, 0, fsize);

for (int i=0; i< fsize; i++ ){ System.out.print((char) mbuf.get());}

Write to a file mapped buffer

• data written to buffer will automatically be written to the file. No explicit write operation is necessary.

• specify MapMode.READ_WRITE

File IO

File Byte Streams

• FileInputStream - ctor takes a path string or a File. opened for reading. to read a single byte, an array of bytes, and a subrange of an array of bytes. use available() to determine the number of bytes remaining and skip() to skip over unwanted bytes.

• FileOutputStream - write bytes to a file - will create a non-existent file before opening it

byte[] buf = s.getBytes();

try(FileOutputStream f = new FileOutputStream("blah.txt")){

f.write(buf, 1, buf.length-2);

– ctor takes file name

Members:

• close() - When done with a file, must close it

• read() - reads a single byte, returns –1 on EOF

• write() - To write to a file

open a file Path by calling Files.newInputStream() / newOutputStream()

Files

File class

• Does not operate on streams, deals directly with file properties and the file system – get permissions, datetime, get/set path

• NIO Path interface and Files class – a better alternative

    File f1 = new File("/blah/blah");

• query methods: getName(), getPath(), getAbsolutePath(),getParent(),exists() , canWrite() , canRead() , isDirectory() , isFile() , isAbsolute() , lastModified(), length() 

• 2 useful utility methods: renameTo(), delete()

• other methods: deleteOnExit(), getFreeSpace(), getTotalSPace(), getUsableSpace(), isHidden(), setLastModified(), setReadOnly()

• implements Comparable --> compareTo()

• toPath() - conversion to java.nio.file.Path

• list() - for directory files, lists children file names. Optional param FilenameFilter interface - limit the number of files returned – accept() is called once for each file in a list.

• listFiles() - return the file list as an array of File objects instead of strings.

• mkdir() , mkdirs()

The Files Class

• provides static methods to act upon a Path. open or create a file that has the specified path.

• Methods: copy(), createDirectory(), createFile(), delete(), exists(), isDirectory(), isExecutable(), isHidden(), isReadable(), isRegularFile(), isWritable(), move(), newByteChannel(), newDirectoryStream(), newInputStream(), newOutputStream(), notExists(), readAttributes(), size()

• take an argument of type OpenOption interface - describes how to open a file. It is implemented by the StandardOpenOption class - defines an enum with values: APPEND, CREATE, CREATE_NEW, DELETE_ON_CLOSE, DSYNC, READ, SPARSE, SYNC, TRUNCATE_EXISTING, WRITE

Files.copy(Paths.get("x.txt"), Paths.get("y.txt"), StandardCopyOption.REPLACE_EXISTING);

Paths

Path Interface

describes a file’s location. in java.nio.file package, extends interfaces: Watchable, Iterable<Path>, Comparable<Path>. convert a File instance into a Path instance by calling toPath(). Methods:

• getName(index) - obtain an element in a path.

• GetNameCount() - get number of elements in a path

• toString() a string representation of the entire path

• resolve() - relative path into an absolute path

• endsWith(), getFileName(), getName(), getNameCount(), getParent(), getRoot(), isAbsolute(), resolve(), startsWith(), toAbsolutePath(), toString()

The Paths Class

get() - obtain a Path

static Path get(String pathname, String … parts)

static Path get(URI uri)

The File Attribute Interfaces

Associated with a file is a set of attributes - hierarchy of interfaces defined in java.nio.file.attribute.

• BasicFileAttributes – methods: creationTime(), fileKey(), isDirectory(), isOther(), isRegularFile(), isSymbolicLink(), lastAccessTime(), lastModifiedTime(), size()

• From BasicFileAttributes two interfaces are derived: DosFileAttributes and PosixFileAttributes.

• DosFileAttributes – methods: isArchive(), isHidden(), isReadOnly(), isSystem()

• PosixFileAttributes (POSIX stands for Portable Operating System Interface.) - methods: group(), owner(), permissions()

to access a file’s attributes - call Files.readAttributes() static method or getFileAttributeView(): interfaces AttributeView, BasicFileAttributeView, DosFileAttributeView, and PosixFileAttributeView.

FileSystems

The FileSystem, FileSystems, and FileStore Classes

by using the newFileSystem() method defined by FileSystems, it is even possible to obtain a new file system. The FileStore class encapsulates the file storage system.

Watchable Interface

- an object that can be monitored for changes.

Using NIO for Stream-Based File I/O

NIO.2 - symbolic links, directory tree traversal, file metadata.

• Query a Path – Path methods: getName(), getParent(), toAbsolutePath().

• Query a File - Files methods: isExecutable(), isHidden(), isReadable(), isWritable(), exists(), readAttributes() - eg get BasicFileAttributes, PosixFileAttributes.

• List the Contents of a Directory - DirectoryStream<Path> implements Iterable<Path> - create using newDirectoryStream(Path), then use its iterator() method

• List a Directory Tree - Use Files.walkFileTree(Path root, FileVisitor<? extends Path> fv)

FileVisitor interface

• defines how the directory tree is traversed – methods: postVisitDirectory, preVisitDirectory, visitFile, visitFileFailed

• each method returns a FileVisitResult enum values: CONTINUE, SKIP_SIBLINGS, SKIP_SUBTREE, TERMINATE

• to continue traversing the directory and subdirectories, a method should return CONTINUE. For preVisitDirectory(), return SKIP_SIBLINGS to bypass the directory and its siblings and prevent postVisitDirectory() from being called. To bypass just the directory and subdirectories, return SKIP_SUBTREE. To stop the directory traversal, return TERMINATE.

• It is possible to watch a directory for changes by using java.nio.file.WatchService.