Scala IO is somewhat lacking at this point in time - often requiring a fallback to Java APIs.
I did not have time to write this out in Scala - this post will contain a few Java snippets. What counts is the concepts - Buffers, Streams, Channels and IO vs NIO vs NIO2. Also, Scala per se' does not natively support a "try with resources' concept - for this use scala-arm https://github.com/jsuereth/scala-arm/blob/master/README.md - ‘for' with resources: <- resource.managed()
I/O Basics
print(
) / println(
)
java.io
- byte / char
stream-based I/O
–
produces
+
serializes
objects
/
deserializes
objects + consumes
information over
physical IO
device:
network
connection / memory buffer / disk file
java.nio
- buffer
/ channel-based I/O
- complimentary
abstraction - open
a channel (connection)
to
an I/O device with
a buffer to hold data, then perform
IO data operations
on the buffer. NIO1
was channel-based I/O. NIO2
supports
stream-based I/O.
Using
the NIO System
- The
most common I/O device is the disk file - all file channel operations
are byte-based. Eg Open a file via a
Path
from Paths.get() and populate a ByteBuffer.
IO
interfaces:
Closeable,
DataInput, DataOutput, Externalizable, FileFilter, FilenameFilter,
Flushable, ObjectInput, ObjectInputValidation, objectOutput,
ObjectStreamConstants, Serializable
IOException
– subclasses:
FileNotFoundException,
SecurityException -
requires
a
security manager eg
for applets.
The IO/NIO
Packages
-
java.io -
byte
/ char stream
types
-
java.nio – buffer
types
-
java.nio.channels –
channels: open IO connections
-
java.nio.channels.spi –
service providers
-
java.nio.charset –
encoders, decoders for byte <--> char
-
java.nio.charset.spi -
service providers
-
java.nio.file - files
-
java.nio.file.attribute
-
java.nio.file.spi
Buffers
Buffer
Class - encapsulates
current position (index
of next R/W op), limit
(index of 1 past the
last), and capacity.
methods:
-
array(), arrayOffset(),
capacity(), clear(), flip(), hasArray(), hasRemaining(), isDirect(),
isReadOnly(), limit(), mark(), position(), remaining(), reset(),
rewind()
-
various
get() / put()
methods
-
allocate()
- allocate a
buffer manually
-
wrap()
- wrap an array inside a buffer
-
slice()
- create a
subsequence
derived
classes:
-
for
different types: ByteBuffer, CharBuffer,
DoubleBuffer,FloatBuffer, IntBuffer, LongBuffer, ShortBuffer
-
MappedByteBuffer
– extends ByteBuffer
- used to map a file to a buffer.
Streams
2
types of streams: low level byte (binary) and high level char
(Unicode)
2
ways
to close
a stream
- 1) explicitly call close() in finally, 2)
try-with-resources (AutoCloseable)
Predefined Standard
IO
Streams
System.in
/ out
java.lang.System
contains 3
predefined stream variables: in
(InputStream),
out,
(PrintStream)
and err.
These
fields are declared as public,
static,
and final.
may be redirected or
wrapped within
character-based streams.
Reading
Console Input - wrap
System.in
BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
After this statement executes,
br
is a character-based stream that is linked to the console through
System.in.
use
readLine(
) to
read strings:
do {
str = br.readLine();
} while(!str.equals("\n"));
Writing Console Output -
System.out.write
can output chars – use
PrintWriter.print()
/ println() for
string output-
System.out references
PrintStream
PrintWriter pw = new PrintWriter(System.out, true);
pw.println("This is a string");
The Console Class
The Stream Classes
-
4
abstract classes in
separate hierarchies.
-
InputStream
/ OutputStream
for byte streams - cannot work directly with Unicode characters.
-
Reader
/ Writer
for character streams.
-
All implement AutoCloseable,
Closeable.
OutputStream/
Writer
additionally
implement Flushable.
Writer
implements
Appendable.
The
Byte Stream Classes
stream
class names: <catagory><InputStream/OutputStream>
Byte Stream Categories
-
Buffered -
chunks
-
ByteArray -
vanilla
-
Data – support
for standard datatypes
-
File
-
Filter
-
Object
-
Piped
-
Print –
for console out – output only
-
Pushback –
supports one byte unget – input only
-
Sequence –
combine 2 streams sequently – input only
Input / Output Abstract Base Classes
-
InputStream –
methods:
available(), close(), mark(), markSupported(), read(), reset(),
skip()
-
OutputStream
– methods:
close(),
flush(), write()
derived
classes overide
read( ) / write( )
ByteArray Streams
String tmp = "abcdefghijklmnopqrstuvwxyz";
byte b[] = tmp.getBytes();
ByteArrayInputStream input1 = new ByteArrayInputStream(b);
ByteArrayOutputStream
s = new
ByteArrayOutputStream();
byte[]
buf = "Hello
Ruz".getBytes();
try
{
s.write(buf);
...
try(FileOutputStream
f = new
FileOutputStream("copy.txt")){
s.writeTo(f);
Filter Streams
Buffered Streams
new BufferedInputStream(Files.newInputStream(Paths.get(“x.txt”)))
Pushback Streams
-
PushbackInputStream
- extends FilterInputStream
with a
memory buffer for
multibyte
operations
(improving
performance, supports
skipping, marking, and resetting of the stream).
use pushback()
to allow a byte to be read and then returned to the stream -->
“peek” . use
unread() -
pushes
back the low-order byte as
the next byte returned by a subsequent call to
read(). side
effect of invalidating mark()
- use
markSupported()
to check the
stream.
Sequenced Streams
SequenceInputStream(Enumeration
<? extends InputStream> streamEnum)
Print Streams
-
provides output capabilities
for file
handle System.out
(
a PrintStream)
-
implements Appendable,
AutoCloseable,
Closeable,
Flushable
-
print()
/ println()
- leverages
Object.toString()
overrides.
-
printf()
- uses the Formatter
class
-
format()
Data Streams
Random Access Files
The
Character Stream Classes
stream
class names: <catagory><Reader/Writer>
Categories
-
Buffered
-
CharArray
– vanilla
-
File
-
Filter
-
InputStream /
OutputStream
– translators
-
LineNumber
– counts lines. Reader only
-
Piped
-
Print
–
for console out–
Writer Only
-
Pushback
-
String
Reader /
Writer Abstract base
classes:
-
Reader - abstract
base.
Methods: close(),
mark(), markSupported(), read(), ready(),
reset(), skip()
-
Writer - abstract
base.
also
implements Appendable. methods:
append(),
close(),
flush(), write()
derived
classes overide
read( ) / write( )
File Streams
try ( FileReader fr = new FileReader("FileReaderDemo.java") ){
while((i = fr.read()) != -1) System.out.print((char) i);
CharArray Streams
CharArrayReader(char
array
[ ], int start,
int numChars)
Buffered Streams
Pushback Streams
Print Streams
Flushable
Interface
Try With
Resources
-
java.lang.AutoCloseable
Interface
-
support for try-with-resources (note:
resource declared in the try
is implicitly final.
can manage multiple resources seperated
by semicolon.)
-
java.io.Closable
Interface–
extends Autoclosable.
automates closing a
resource - close()
closes the invoking object, releasing resources. implemented
by stream classes. Automatically
Closing a File via
try-with-resources:
try(FileInputStream fin = new FileInputStream(args[0])) {
do {
i = fin.read();
if(i != -1) System.out.print((char) i);
} while(i != -1);
}catch(FileNotFoundException e) {
System.out.println("File Not Found.");
}catch(IOException e) {
System.out.println("An I/O Error Occurred");
}
Serialization
-
Write object
state to
a byte stream -->
for
persistent storage or
RMI. Considerations:
objects relationships should
be DAGs.
-
Serializable interface
– implementing
class (& all
of its subclasses)
are serializable. Static
fields and fields
declared as transient
opt out of
serialization.
-
Externalizable
interface
- Extensible
serialization. Methods:
readExternal(ObjectInput
inStream),
writeExternal(ObjectOutput outStream)
-
ObjectOutput interface
- extends
DataOutput
/ AutoCloseable
interfaces and supports object serialization. Methods:
close(), flush(),
write(), writeObject()
-
ObjectOutputStream
class
-
Extends
OutputStream,
mplements ObjectOutput.
for writing objects to a stream.
-
ObjectInput interface
- extends
DataInput
, AutoCloseable
-
ObjectInputStream -
extends
InputStream
, implements ObjectInput
Channel Interface
represents
open connection to I/O source / destination. extends Closeable,
AutoCloseable.
obtain
a channel by calling getChannel()
on an object that supports channels:
-
FileinputStream /
FileOutputStream
-
RandomAccessFile
-
Socket / ServerSocket /
DatagramSocket
-
Files (via
static SeekableByteChannel)
derived
interfaces: FileChannel,
SocketChannel,
SeekableByteChannel
support
various read()
and write()
methods
support additional channel access & control methods
FileChannel
Class
get
/ set current position, transfer information between file channels,
get
size , lock the channel.
provides a static
method called open(),
which opens a file and returns a channel to it. the map()
method, which lets you map a file to a buffer.
Charsets and Selectors
-
A charset
- defines the way that bytes are mapped to characters.
-
An encoder / decoder - encode
/ decode
a char sequence
into bytes / byte
sequence into chars.
defined in the java.nio.charset
package.
-
A selector
- supports key-based, non-blocking, multiplexed I/O - enable you to
perform I/O through multiple channels. defined in the
java.nio.channels
package. most applicable to socket-backed channels.
NIO
Manual
Channel
File
I/O
Manually Read from a File via a Channel
-
Paths.get()
– specify & open
a Path
to file. (defaults
to RO)
-
establish a channel to file:
Files.newByteChannel()
- returns
a
SeekableByteChannel
interface object
cast to
FileChannel
class (
implements
AutoCloseable).
-
allocate a buffer - used by
the channel: wrap an existing array or ByteBuffer.allocate()
to allocate
dynamically.
-
SeekableByteChannel.read(ByteBuffer
buf)
-
fills buffer to
capacity with data from
the file. Sequential
reads – call
repeatedly. returns
#bytes
read or
−1 at EOF
(AutoCloseable
uses this).
-
load
buffer with data from file - ByteBuffer.rewind()
- reset to
start -->
read bytes
by ByteBuffer.get().
bytes
are cast to char
so file can be displayed as text. (optionally
create a char buffer
to
encodes bytes as they
come in)
-
streamlined sa
single try
with resources block
calls Paths.get()
and newByteChannel()
try
(SeekableByteChannel
fChan=Files.newByteChannel(Paths.get("x.txt"))){
ByteBuffer
mbuf = ByteBuffer.allocate(128);
do
{
int count = fChan.read(mbuf);
//read from channel into buffer
if
(count != -1) {
mbuf.rewind();
// so it can be read
for (int
i=0; i < count; i++) {
System.out.print((char)
mbuf.get());
}
}
}while
(count != -1);
Manually
Write
to a File via a Channel
-
specify
StandardOpenOption.WRITE
/ CREATE.
-
write data to buffer using
ByteBuffer.put()
- advances the
current position.
-
reset to start of buffer via
rewind()
before calling write().
-
alternativelly
call flip()
instead of rewind()
- sets value
of current position to 0
and limit to previous current position.
for(int h=0; h<3; h++) { // Write some bytes to the buffer.
for(int i=0; i<26; i++)
mBuf.put((byte)('A' + i));
mBuf.rewind(); // Rewind the buffer so that it can be written.
fChan.write(mBuf); // Write the buffer to the output file.
NIO Automatic
file IO via
mapped buffer
Read
from a file mapped buffer
try
(FileChannel fChan =
(FileChannel)Files.newByteChannel(Paths.get("x.txt"))){
long
fsize = fChan.size();
MappedByteBuffer
mbuf = fChan.map(FileChannel.MapMode.READ_ONLY,
0, fsize);
for
(int
i=0; i< fsize; i++ ){ System.out.print((char)
mbuf.get());}
Write to a file mapped buffer
File IO
File Byte Streams
-
FileInputStream
-
ctor
takes a path string or a File.
opened
for reading. to read a single byte, an array of bytes, and a
subrange of an array of bytes. use
available()
to
determine the number of bytes remaining and
skip()
to skip over unwanted bytes.
-
FileOutputStream
-
write
bytes to a file - will create a
non-existent
file before opening it
byte[]
buf = s.getBytes();
try(FileOutputStream
f = new
FileOutputStream("blah.txt")){
f.write(buf,
1, buf.length-2);
–
ctor
takes file name
Members:
-
close() - When
done with a file, must close it
-
read()
- reads a single byte, returns –1 on
EOF
-
write()
- To write to a file
open
a file Path by
calling Files.newInputStream()
/ newOutputStream()
Files
File
class
-
Does not
operate on streams,
deals directly with file properties
and the file system – get
permissions, datetime,
get/set path
-
NIO
Path
interface and Files
class – a better
alternative
File f1 = new File("/blah/blah");
-
query methods: getName(), getPath(), getAbsolutePath(),getParent(),exists() , canWrite() , canRead() , isDirectory() , isFile() , isAbsolute() , lastModified(), length()
-
2
useful utility methods: renameTo(),
delete()
-
other
methods: deleteOnExit(),
getFreeSpace(), getTotalSPace(), getUsableSpace(), isHidden(),
setLastModified(), setReadOnly()
-
implements Comparable
-->
compareTo()
-
toPath() - conversion
to
java.nio.file.Path
-
list()
- for
directory
files,
lists children file names. Optional param FilenameFilter
interface
-
limit the number of files returned – accept()
is called once
for each file in a list.
-
listFiles()
- return the
file list as an array of File
objects instead of strings.
-
mkdir()
, mkdirs()
The Files Class
-
provides
static
methods to act upon a
Path.
open or create a file that has the specified path.
-
Methods: copy(),
createDirectory(), createFile(), delete(), exists(), isDirectory(),
isExecutable(), isHidden(), isReadable(), isRegularFile(),
isWritable(), move(), newByteChannel(), newDirectoryStream(),
newInputStream(), newOutputStream(), notExists(), readAttributes(),
size()
-
take an argument of type
OpenOption
interface -
describes how to open a file. It is implemented by the
StandardOpenOption
class - defines an enum with
values: APPEND,
CREATE, CREATE_NEW, DELETE_ON_CLOSE, DSYNC, READ, SPARSE, SYNC,
TRUNCATE_EXISTING, WRITE
Files.copy(Paths.get("x.txt"),
Paths.get("y.txt"),
StandardCopyOption.REPLACE_EXISTING);
Paths
Path Interface
describes
a file’s location. in java.nio.file
package,
extends
interfaces: Watchable,
Iterable<Path>,
Comparable<Path>.
convert a File
instance into a Path
instance by calling toPath().
Methods:
-
getName(index)
- obtain an element in a path.
-
GetNameCount() -
get number
of elements in a path
-
toString()
a string representation of the entire path
-
resolve()
- relative path into an absolute path
-
endsWith(), getFileName(),
getName(), getNameCount(), getParent(), getRoot(), isAbsolute(),
resolve(), startsWith(), toAbsolutePath(), toString()
The Paths Class
get()
- obtain a Path
static
Path get(String pathname,
String … parts)
static
Path get(URI uri)
The File Attribute
Interfaces
Associated
with a file is a set of attributes - hierarchy of interfaces defined
in java.nio.file.attribute.
-
BasicFileAttributes –
methods:
creationTime(),
fileKey(), isDirectory(), isOther(), isRegularFile(),
isSymbolicLink(), lastAccessTime(), lastModifiedTime(), size()
-
From BasicFileAttributes
two interfaces are derived: DosFileAttributes
and PosixFileAttributes.
-
DosFileAttributes
– methods: isArchive(),
isHidden(), isReadOnly(), isSystem()
-
PosixFileAttributes
(POSIX stands for Portable
Operating System Interface.)
- methods: group(),
owner(), permissions()
to
access a file’s attributes - call Files.readAttributes()
static
method or
getFileAttributeView():
interfaces
AttributeView,
BasicFileAttributeView,
DosFileAttributeView,
and PosixFileAttributeView.
FileSystems
The FileSystem, FileSystems, and FileStore Classes
by
using the newFileSystem()
method defined by FileSystems,
it is even possible to obtain a new file system. The FileStore
class encapsulates the file storage system.
Watchable
Interface
-
an object that can be monitored for changes.
Using
NIO for Stream-Based File
I/O
NIO.2
- symbolic links, directory tree traversal, file
metadata.
-
Query
a Path –
Path methods:
getName(),
getParent(),
toAbsolutePath().
-
Query
a File - Files
methods:
isExecutable(),
isHidden(),
isReadable(),
isWritable(),
exists(),
readAttributes() - eg
get
BasicFileAttributes,
PosixFileAttributes.
-
List
the Contents of a Directory -
DirectoryStream<Path>
implements Iterable<Path>
-
create
using
newDirectoryStream(Path),
then
use
its
iterator()
method
-
List
a Directory Tree - Use
Files.walkFileTree(Path
root,
FileVisitor<? extends Path> fv)
FileVisitor
interface
-
defines
how the directory tree is traversed – methods: postVisitDirectory,
preVisitDirectory, visitFile, visitFileFailed
-
each method returns a
FileVisitResult
enum
values: CONTINUE,
SKIP_SIBLINGS,
SKIP_SUBTREE,
TERMINATE
-
to continue traversing the
directory and subdirectories, a method should return CONTINUE.
For preVisitDirectory(),
return SKIP_SIBLINGS
to bypass the directory and its siblings and prevent
postVisitDirectory()
from being called. To bypass just the directory and subdirectories,
return SKIP_SUBTREE.
To stop the directory traversal, return TERMINATE.
-
It is possible to watch a
directory for changes by using java.nio.file.WatchService.