Friday, October 16, 2009

Chapter 11: Service Layer Refactoring












Chapter 11: Service Layer Refactoring


In Chapter 10, “Programmer Tests: Using Transactions,” we added support for transactions to our application. It was a significant architectural change that we made after a substantial part of the application had been built. In this chapter, we will implement another architectural refactoring that has to do with reorganizing the packaging of the application.




The Problem


In Chapter 12, “Implementing a Web Client,” we will develop an ASP.NET Web client for the application. A typical enterprise application can have a variety of clients, and some of these clients provide interfaces for users to interact with the system; clients can also be in the form of other software systems consuming the services of the enterprise application. In both cases, the set of service operations provided by the enterprise application is conceptually shared among the clients. In the case of the application, we have a set of client-specific operations:




  • Find a recording by recordingId




  • Add a new review to an existing recording




  • Delete a review from an existing recording




These operations are exposed to the single client of the application via the Web service. This client is another software system, or software systems that are likely to be hosted on a variety of technology platforms, which will be consuming the services of the application. This is why we chose to use Web services as the technology to expose the application’s services.


Let’s review the application architecture from the packaging perspective. Figure 11-1 shows what it looks like:






Figure 11-1: Application packages

There are several packages in the application:





  • Data Access package This package is responsible for implementation of CRUD low-level operations for the underlying data in the data source. Data gateways (RecordingGateway, LabelGateway, and so on) are abstractions populating this package.


    This package also has TransactionManager and its supporting classes (Command and CommandExecutor), defined here to make data gateways transaction-capable.


    This package also has implemented application-specific transaction management and higher-level business operations. The Catalog class is another core abstraction in this package and it consolidates the CRUD operations into higher-level business operations.





  • Data Model package This package defines a typed data set that establishes the data-level contract between the data source and the rest of the application. This contract is pretty low level and is tightly coupled to the database schema of the underlying data source. Although ADO.NET and the typed data set give a database-vendor independent views of the data, conceptually the two schemas—the data set and the database—are very tightly coupled and must be maintained together.





  • Service Interface package To reduce the dependency between the data source’s schema and the rest of the application, we have defined a Recording Data Transfer Object (DTO)—an XML representation of the data as viewed by the application. Recording DTO is the application-level representation of the data in the database, and this representation is captured by the RecordingDto class. This application-level contract hides some of the database-related specifics; for example, the database schema normalizations are not directly exposed on the Recording DTO; Artist, Label, Genre, and Reviewer are not represented as individual entities and are flattened to be simple string attributes on the corresponding objects (Recording, Track, and Review). We have created a RecordingAssembler class that is responsible for mapping between the two representations of the recording data.


    This package also has CatalogService and DatabaseCatalogService classes that implement the functionality of retrieving the recording from the database and mapping it to the RecordingDto using the RecordingAssembler.


    The Service Interface package hosts the CatalogServiceInterface class that exposes the application’s operations as a Web service and adds mapping of the application exceptions into SoapFaults using the ExceptionMapper class.




The programmer tests are compiled into separate assemblies from the application code so that we can choose to deploy the code with or without the tests.




What’s Wrong?


Let’s take a closer look at the Service Interface package, which has classes that provide both a Web service-independent implementation of the application functionality and classes that are responsible for exposing and adapting this functionality via the Web service. The set of classes that are responsible for Web service–independent implementation of the application functionality includes CatalogService, DatabaseCatalogService, RecordingDto, and RecordingAssembler. This set of classes defines and implements a service-level contract between the application and its clients. We are adding a new client to the application, and this client needs to have access to this application-level contract. Unfortunately, however, this contract is packaged with the Web service into one assembly.


We have also been putting off some cleanup for the programmer tests. We still have StubCatalogService and StubCatalogServiceFixture classes in our service.interface.tests assembly. These two classes do not add to the test coverage, and they complicate the application code.





The Solution


We will start by removing the stub implementation of the CatalogService and the corresponding programmer test. This change is very simple, and we can simply remove the two classes: StubCatalogService and StubCatalogServiceFixture.


The first change allows us to simplify the implementation of the CatalogService class. Now that we have just one implementation of the data retrieval strategy, we don’t need to have the implementation span two classes: CatalogService and DatabaseCatalogService. These two classes can be merged into one (CatalogService), and we can get rid of the abstract methods and keep only the implementation. The following code shows what the new version of the CatalogService looks like:



using System;
using DataAccess;
using DataModel;

namespace ServiceInterface
{
public class CatalogService
{
public RecordingDto FindByRecordingId(long id)
{
RecordingDataSet dataSet = new RecordingDataSet();
RecordingDataSet.Recording recording =
Catalog.FindByRecordingId(dataSet, id);

if(recording == null) return null;

return RecordingAssembler.WriteDto(recording);
}

public ReviewDto AddReview(string reviewerName, string content,
int rating, long recordingId)
{
RecordingDataSet dataSet = new RecordingDataSet();
RecordingDataSet.Review review =
Catalog.AddReview(dataSet, reviewerName, content,
rating, recordingId);

return RecordingAssembler.WriteReview(review);
}

public void DeleteReview(long reviewId)
{
Catalog.DeleteReview(reviewId);
}
}
}


To do part of this refactoring, we had to change the users of the old CatalogService and DatabaseCatalogService classes.


The next step is to split the Service Interface package into two:




  • First, the package responsible for exposing the application’s operations using the Web services; we keep the name as the Service Interface package




  • Second, the package that actually defines the application services in a way that is independent of which client uses it; we call it the Service Layer




As part of this split, we need to restructure a few other packages. The mechanics of this refactoring are not complex. We will define a separate namespace, ServiceLayer, to host the classes of the Service Layer. We need to move the classes that define the application-level data contract and the classes that are responsible for the implementation of this contract into the ServiceLayer namespace. These classes are as follows:





  • RecordingDto generated from RecordingDto schema





  • RecordingAssembler





  • CatalogService




We will also move the related programmer test fixtures into this new namespace. The following test classes will be moved to this namespace:





  • CatalogServiceFixture (this class was called DatabaseCatalogServiceFixture before we rolled up the CatalogService and DatabaseCatalogService into one class: CatalogService)





  • DatabaseUpdateReviewFixture





  • InMemoryRecordingBuilder





  • RecordingAssemblerFixture, ReviewAssemblerFixture, and TrackAssemblerFixture




Modern refactoring tools support such refactoring, and they make it much easier to handle a task like this for much larger applications, in which thousands of classes might need to be moved. However, we do not have one of those tools, so we need to do this manually. When we get everything to compile again, we rerun all the tests, and they pass.


We are almost finished. We have one undesirable package dependency still present. The ExistingReviewMapper class from the ServiceInterface package needs to have access to the ExistingReviewException class defined in the DataAccess package. We want to break that dependency by moving the ExistingReviewException into a new package: ServerExceptions.



Figure 11-2 shows what our application architecture looks like after the extraction of the Service Layer.






Figure 11-2: Application architecture after ServiceLayer refactoring











Section 9.9. Axis










9.9. Axis


Although it's not usually associated with evil (although cursing is a different story), an axis is a node set starting at a particular node that is based on the relationship between the nodes in an XML document. The basic format for using an axis follows:


axis::context-node


Table 9-3 describes the properties of the various axes available in XPath.



Table 9-3. XPath Axes

Axis

Description

ancestor

Selects all nodes that are ancestors of the context node, farther up the document tree, in a direct line to the document root node. The resulting node set is in reverse document orderin other words, moving up the tree starting from the document's parent node.

ancestor-or-self

Selects the same nodes as the ancestor axis. However, it starts with the context node instead of the context node's parent.

attribute

Selects all the context node's attributes, if any.

child

Selects all the child nodes of the context node, excluding attributes and namespace nodes.

descendant

Recursively selects all children of the context node and their children until the end of each tree branch.

descendant-or-self

Selects the same nodes as the descendant axis, with the exception of starting with the context node.

following

Selects, in document order, all nodes at any level in the document tree that follow the context node.

following-sibling

Selects, in document order, all nodes at the same level and with the same parent node in the document tree that follow the context node.

namespace

Selects the namespace nodes that are in scope for the context node. If no namespace nodes are in scope, the namespace axis is empty.

parent

Selects the parent node of the context node. If the context node is the root node, the parent axis will be empty.

preceding

Selects all nodes, in reverse document order, excluding ancestor nodes, in the document tree that are before the context node.

preceding-sibling

Selects all nodes, in reverse document order, that are at the same level that have the same parent node as the context node.

self

Selects the context node.



The use of an axis is arguably the most formidable concept for developers new to XPath, who often have difficulty trying to visualize the results of using an axis. Fortunately, tools such as the XPath Evaluator in Altova's XMLSPY make it easier to see the results of specifying a particular axis. Starting with the original XML document from Listing 9-1, the following sections present examples of each of the various axes.




9.9.1. Ancestor Axis Example



XPath Statement

//book[3]/ancestor::*




Result Node Set

library




Explanation

Because the context node, the third book node, is a child of the root element, there is only a single ancestor.





9.9.2. ancestor-or-self Axis Example



XPath Statement

//book[3]/ancestor-or-self::*




Result Node Set

book
library




Explanation

In addition to the ancestor nodes, the ancestor-or-self axis returns the context node. Also, because the results are in reverse document order, the context node is the first node in the node set, followed by the parent node and so on up the tree.





9.9.3. attribute Axis Example



XPath Statement

//book[3]/attribute::*





Result Node Set

publisher




Explanation

Because the context node has only one attribute, it is the only attribute returned in the node set.





9.9.4. child Axis Example



XPath Statement

//book[3]/child::*




Result Node Set

series "The Lord of the Rings"
title "The Two Towers"
author "J.R.R. Tolkien"




Explanation

The resulting node set consists of the three child nodes of the context node. I have shown the contents of the individual nodes to distinguish these nodes from similar nodes with different contents.





9.9.5. descendant Axis Example



XPath Statement

//book[3]/descendant::*




Result Node Set

series "The Lord of the Rings"
title "The Two Towers"
author "J.R.R. Tolkien"





Explanation

The results shown here are identical to the results from the child axis. This is because of the structure of the XML document. For instance, if any of the child nodes shown here had children of their own, the descendant axis would have returned their children, and so on down the line in document order, whereas the child axis would not.





9.9.6. descendant-or-self Axis Example



XPath Statement

//book[3]/descendant-or-self::*




Result Node Set

book
series "The Lord of the Rings"
title "The Two Towers"
author "J.R.R. Tolkien"




Explanation

As with the descendant axis, all child nodes are returned recursively. However, instead of starting with the first child, the context node is the first node in the node set.





9.9.7. following Axis Example



XPath Statement

//book[3]/following::*




Result Node Set

book
series "The Lord of the Rings"
title "The Return of the King"
author "J.R.R. Tolkien"
book
series "Lord Darcy"
title "Too Many Magicians"
author "Randall Garrett"
book
series "Lord Darcy"
title "Murder and Magic"
author "Randall Garrett"
book
series "Lord Darcy"
title "The Napoli Express"
author "Randall Garrett"
book
series "Lord Darcy"
title "Lord Darcy Investigates"
author "Randall Garrett"




Explanation

The resulting node set for the following axis is always all the nodes that occur after the context node in document order.





9.9.8. following-sibling Axis Example



XPath Statement

//book[3]/following-sibling::*




Result Node Set

book
book
book
book
book




Explanation

These five book nodes retrieved using the following-sibling axis are the same nodes that were retrieved by the following axis. The only difference is that the following-sibling axis retrieves only those nodes on the same level as the context node and have the same parent as the context node.





9.9.9. namespace Axis Example



XPath Statement

//book[3]/namespace::*





Result Node Set

Empty node set




Explanation

Because no namespace was in scope on the context node, the resulting node set is empty. However, if one or more namespaces had been in scope, the resulting node set would have contained those in scope.





9.9.10. parent Axis Example



XPath Statement

//book[3]/parent::*




Result Node Set

library




Explanation

The resulting node set will always consist of either an empty node set or a single node. For example, the parent axis of the library element would have retrieved an empty node set.





9.9.11. preceding Axis Example



XPath Statement

//book[3]/preceding::*




Result Node Set

author "J.R.R. Tolkien"
title "The Fellowship of the Ring"
series "The Lord of the Rings"
book
author "Clifford D. Simak"
title "Way Station"
series
book





Explanation

The resulting node set of the preceding axis is made up of those nodes that occur in the XML document before the context node, in reverse document order.





9.9.12. preceding-sibling Axis Example



XPath Statement

//book[3]/preceding-sibling::*




Result Node Set

book
book




Explanation

These book nodes retrieved using the preceding-sibling axis are the same nodes that were retrieved by the preceding axis. However, the difference is that the preceding-sibling axis retrieves only those nodes that are on the same level as the context node and that have the same parent as the context node.





9.9.13. self Axis Example



XPath Statement

//book[3]/self::*




Result Node Set

book




Explanation

The self axis returns the context node; essentially, the result is the same as if the axis were omitted.














Showing It Off



[ Team LiB ]









Showing It Off


It was now the summer of 1978, and I was so proud of my system that I drove it around to a variety of wargamers' conventions, showing it off. Most of the gamers had not dreamed that such a thing was possible and were quite impressed. However, as with the older version of Tanktics on the IBM 1130, they found the game lacking in the richness that boardgames provided. My claim that Tanktics offered true blind play, just like the real world, fell on deaf ears. Nevertheless, there were a few wargamers who had purchased some of the early personal computers, primarily the Commodore PET, and they urged me to re-write Tanktics to run on their machines. I thought the problem over and decided to accept the challenge.






    [ Team LiB ]



    What's Covered in This Chapter










    What's Covered in This Chapter


    The focus of this book is more on technology and less on process. However, this chapter provides an overview of an agile software development process that you can easily apply to your project. I'm a big believer in having a bare-minimum process, even if it is a 1-page checklist of 10 or so items that serves as a memory jogger for things that need to be done as part of the process. (Note: I have included such a checklist in the appendixes.) This minimal process ensures that the project is run efficiently and at the same time is focused on customer satisfaction.


    In this chapter, we will accomplish the following:


    • Gain an understanding of what our sample application will do by looking at some business requirements.

    • Establish a simple software methodology based on Extreme Programming (XP) and Agile Modeling Driven Development (AMDD).

    • Develop some high-level artifacts such as a domain model, UI prototypes, high-level architecture, and more.

    • Create a simple release plan based on our user stories.


    Note



    It is important to realize that many of the artifacts shown in this chapter (release and iteration plans, for example) are more for demonstration purposes. However, this chapter is very relevant to the rest of the book because we will implement some of the functionality described in this chapter (for example, the Enter Hours and Timesheet List screens). In general, you can ignore particulars such as dates and estimates.


    Also, this chapter assumes that you have a basic understanding of software development processrelated concepts (use cases, for example). However, if my brief explanations on the various concepts in this chapter aren't sufficient, I recommend visiting www.agilemodeling.com for detailed explanations. In general, this website is loaded with information relevant to this chapter. Also, visit the extremeprogramming.org website for detailed information on the XP methodology.














    Thursday, October 15, 2009

    Section 1.2.  UNIX Architecture










    1.2. UNIX Architecture


    In a strict sense, an operating system can be defined as the software that controls the hardware resources of the computer and provides an environment under which programs can run. Generally, we call this software the kernel, since it is relatively small and resides at the core of the environment. Figure 1.1 shows a diagram of the UNIX System architecture.



    Figure 1.1. Architecture of the UNIX operating system







    The interface to the kernel is a layer of software called the system calls (the shaded portion in Figure 1.1). Libraries of common functions are built on top of the system call interface, but applications are free to use both. (We talk more about system calls and library functions in Section 1.11.) The shell is a special application that provides an interface for running other applications.


    In a broad sense, an operating system is the kernel and all the other software that makes a computer useful and gives the computer its personality. This other software includes system utilities, applications, shells, libraries of common functions, and so on.


    For example, Linux is the kernel used by the GNU operating system. Some people refer to this as the GNU/Linux operating system, but it is more commonly referred to as simply Linux. Although this usage may not be correct in a strict sense, it is understandable, given the dual meaning of the phrase operating system. (It also has the advantage of being more succinct.)










      15.3 Authorization











       < Day Day Up > 





      15.3 Authorization



      Authorization in a J2EE environment is based on the concept of roles. A user who has been granted at least one of the required roles is allowed to access a resource, such as a method in an enterprise bean. In the J2EE deployment descriptor, role names are associated with a set of method permissions and security constraints. The containers are responsible for mapping users and groups to these roles so that an authorization decision can be made when a resource is accessed. There are at least two ways to implement a role-based authorization engine: the role-permission model and the role-privilege model.



      The role-permission interpretation of the J2EE security model considers a role to be a set of permissions and uses the role name defined in the method-permission and security-constraint descriptors as the label to a set of permissions. A permission defines a resource�the enterprise bean to which a method-permission descriptor applies or a URI described in a security-constraint descriptor�and a set of actions�remote EJB methods or HTTP methods. For example, a Teller role may be associated with a set of permissions to invoke the getBalance() method on an AccountBean enterprise bean and to perform a GET invocation over HTTP to a /finance/account URI. If multiple method-permission descriptors refer to the same role, they are consolidated so that a single set of permissions is associated with the role name, likely within the scope of that application.



      Administrators define authorization policies for the roles in their application. This is done by associating subjects to a role, an operation that grants each subject the permissions associated with that role. This effectively grants the subject access to the enterprise bean methods and to the URIs permitted by that role.



      The method-permission table represents the association of a role to a set of permissions (see Table 3.1 on page 84). Based on the J2EE security model, a method can be accessed by a user who has at least one of the roles associated with the method. The roles associated with a method form the set of required roles to perform an action. The roles associated with a subject form the set of granted roles to that subject. A subject will be allowed to perform an action if the subject's granted roles contain at least one of the required roles to perform that action.



      An authorization table, or protection matrix, represents the association of a role to subjects (see Table 3.2 on page 84). In such a table, the role is defined as the security object, and users and groups are defined as security subjects.



      According to the J2EE security model, it is responsibility of the Application Assembler to associate actions on protected resources to sets of required roles. The Deployer refines and configures the policies specified by the Application Assembler when the application is installed into a WAS. Association of roles to subjects can be performed when the application gets installed in a WAS or at a later time, as part of security administration.



      An alternative approach to implement role-based authorization is to treat a role as a privilege attribute, typically a user group. For example, if a role name is Manager, a user group named Manager should be defined in the user registry. This model has advantages and disadvantages.



      • This approach has the advantage of maintaining only one table�the method-permission table�because the subject who is assigned a role is implicitly a user group that has the identical name. In the example given in Table 3.1 on page 84, user groups with the names Teller and Supervisor will be defined in the user registry.

      • This approach has the disadvantage that for every role defined in an application, a user group needs to be defined in the system. If two applications use the same role name, care should be taken to ensure that the users who are members of a user group identical to the defined role names are allowed to access both applications.

        Additionally, in order to manage users and groups in a user registry and associate them at will with user groups named identical to roles, the user registry would need to support nested groups. For example, if the Supervisor role should be assigned to the HRManagers user group, the HRManagers user group should be a subgroup of the Supervisor user group. This may be difficult to achieve in practice because not every user registry supports nested groups. Thus, unlike the role-permission model, the role-privilege model imposes a structure on the underlying user registry.



      Regardless of which model is used to achieve role-based authorization, the container runtime will need to make runtime authorization decisions each time an attempt is made to access a resource. For example, when the getBalance() method in the AccountBean enterprise bean is invoked, the container must make an authorization decision to verify whether the invocation is allowed. As the J2SE permission security model (see Chapter 8 on page 253) augmented by JAAS (see Chapter 9 on page 289) is rich enough to check permissions based on a subject, it is logical to use the J2SE permission model to perform access checks. Therefore, containers can invoke the java.lang.SecurityManager's checkPermission() method. This also implies that J2EE resources and methods will need to be modeled by different permission types. In Java 2, permissions are implemented as java.security.Permission objects (see Section 8.2 on page 258).



      The process of authorization starts from the time of application installation. When an application is installed, the deployment descriptor information that is stored with the application is read by the container deployment tools. The subject-to-role mapping is effectively managed using the tools provided by the container and/or security provider. Such tools must allow changing the permission-to-role mapping. Effectively, a set of policy management tools will help manage the security policies associated with authorization to access the various J2EE components. When a resource is accessed, the container will consult the authorization runtime provided by the security provider to verify whether the user accessing a resource can be allowed to perform the operation. The security provider may be the same vendor as the container provider or a different one. These steps are illustrated in Figure 15.3.



      Figure 15.3. Authorization in a J2EE Container




      The advantage of using the J2SE/JAAS Permission model is that a single security provider can perform both J2SE permission checks�for example, for accesses to the file system�and J2EE permission checks�for example, to execute EJB methods. Part of the design of authorization engines is to create java.security.Policy objects. Similar to how LoginModules can be used as an approach to allow multiple authentication mechanisms to be used by a container, one of many Policy implementations can be used by the container when making an access check on a resource.













         < Day Day Up > 



        Working with Binary (BLOB) Data











         < Day Day Up > 





        Working with Binary (BLOB) Data



        A common task I'm often asked about is how to read and write binary data�typically representing a graphical image�using the ADO.NET classes. As most ADO.NET articles and books don't seem to cover this important task, I'll use this chapter's first section to illustrate two distinct ways to work with stored binary data�one with the DataReader class and the other with the DataSet class. I'll also explore the use of "chunking" to facilitate working with very large amounts of binary data in an efficient manner. The section concludes with a demo application that allows you to view the images stored for a sample database and update the database with any other image stored in the file system.



        Using the DataReader Class



        Most managed providers define a data reader object that allows for reading a forward-only stream of rows from a specific data store type. For example, there is a data reader class for SQL Server (SqlDataReader), Oracle (OracleDataReader), OLE DB provider support (OleDbDataReader), and ODBC driver support (OdbcDataReader). Up to this point, I've ignored the data readers for the simple reason that�for the most part�they don't provide us MFC developers with anything that we don't already have, since we can pick from a plethora of native database access technologies. However, the data reader does make the task of reading and writing binary data very simple; hence its inclusion in this section.



        The first step in constructing a data reader object is to connect to a data store using one of the managed data connection objects (such as SqlConnection). Once that's done, you then construct a command object (such as SqlCommand) specifying the query to run against the data store. If the command will yield a result set, you can then call the command object's ExecuteReader method, which returns a data reader object.





        SqlConnection* conn =

        new SqlConnection(S"Server=localhost;"

        S"Database=NorthWind;"

        S"Integrated Security=true");

        SqlCommand* cmd =

        new SqlCommand(S"SELECT * FROM Employees", conn);



        conn->Open();



        SqlDataReader* reader = cmd->ExecuteReader();



        ...



        conn->Close(); // Don't call until finished using the data reader!



        Note that, unlike the disconnected nature of the DataSet, the data reader does not read all of the data in a result set into memory at once. Instead, the data reader is an object that keeps a database connection open and basically behaves like a connected, server-side, read-only cursor. It does this by reading only as much of the complete result set as necessary, thereby saving memory, especially in cases where you expect a large result set. As a result, the act of closing the connection also closes the data reader.



        Reading Binary Data with a Data Reader Object


        Once the data reader object has been constructed, call its Read method to advance to the result set's next record. Simply call this method successively until it returns a Boolean value of false, indicating that there are no more records to read.





        while (reader->Read())

        {

        // data accessed via overloaded Item indexer

        // Object* o1 = reader->Item[S"columnName"];

        // Object* o2 = reader->Item[columnIndex];

        }



        While most data can be obtained through the overloaded Item indexer (into an Object), the data reader classes also provide a number of data-type-specific methods such as GetBoolean, GetChars, GetDateTime, and GetGuid. The data-type-specific method we're most interested in here is the GetBytes method. The GetBytes method is used to read binary data and enables you to specify the binary data's column index, the buffer into which to read the data, the index of the buffer where writing should begin, and the amount of data to copy.





        public: virtual __int64 GetBytes(

        int columnIndex,

        __int64 dataIndex,

        unsigned char buffer __gc[],

        int bufferIndex,

        int bytesToCopy

        );



        Here's an example of allocating and then populating a Byte array with binary data from a data reader object:





        Byte image[] =

        __gc new Byte[Convert::ToInt32((reader->GetBytes(columnIndex,

        0, 0, 0,

        Int32::MaxValue

        )))];

        reader->GetBytes(columnIndex, 0, image, 0, image->Length);



        Note that the code snippet calls GetBytes twice. This is done because GetBytes can't be called to retrieve the data until a receiving buffer of the required length is allocated. Therefore, the first call is made with the buffer (third parameter) set to a null reference in order to determine the number of bytes to allocate. Once the image buffer has been allocated, the second call to GetBytes results in the population of the buffer.



        Most image classes are designed to read the graphic from a specified file in order to display the graphic. Therefore, you could write the image data to disk using the FileStream object covered in Chapter 3. Here's a snippet illustrating the writing of a Byte buffer to disk.





        FileStream* stream = new FileStream(destination,

        FileMode::Create,

        FileAccess::Write);

        stream->Write(image, 0, image->Length);

        stream->Close();



        Now that you've seen the individual steps involved in using the DataReader class to read a binary value from a database and, optionally, save it to a file, here's a generic function (GetPictureValue) that takes a SqlDataReader object, column index value, and destination file name. Using what you've just learned, this function first ensures that the column value is not null (GetBytes will throw an exception if the column is null), allocates a Byte buffer, reads the data into that buffer, and then saves the data to the specified destination file name.





        void GetPictureValue(

        SqlDataReader* reader,

        int columnIndex,

        String* destination)

        {

        #pragma push_macro("new")

        #undef new

        FileStream* stream;

        try

        {

        if (!reader->IsDBNull(columnIndex))

        {

        // Allocate a byte array

        Byte image[] =

        __gc new Byte[Convert::ToInt32((reader->GetBytes(columnIndex,

        0, 0, 0, Int32::MaxValue)))];



        // Read the binary data into the byte array

        reader->GetBytes(columnIndex, 0, image, 0, image->Length);



        // Open FileStream and write buffer to file.

        stream = new FileStream(destination,

        FileMode::Create,

        FileAccess::Write);

        stream->Write(image, 0, image->Length);

        Console::WriteLine(S"{0} written", destination);

        }

        else

        {

        Console::WriteLine(S"Image column is null");

        }

        }

        catch (Exception* e)

        {

        // Handle exception

        }

        __finally

        {

        stream->Close();

        }

        #pragma pop_macro("new")

        }



        You can now write something like the following where the code reads every photo in the Employees table and writes that photo out to a file named using the EmployeeID value. (The value 1 being passed to the GetPictureValue function refers to the second column�Photo�specified in the command object's constructor.)





        SqlConnection* conn =

        new SqlConnection(S"Server=localhost;"

        S"Database=NorthWind;"

        S"Integrated Security=true");

        SqlCommand* cmd =

        new SqlCommand(S"SELECT EmployeeID, Photo FROM Employees", conn);



        conn->Open();



        SqlDataReader* reader = cmd->ExecuteReader();

        while (reader->Read())

        {

        String* fileName = String::Format(S"{0}.jpg", reader->Item[0]);

        GetPictureValue(reader, 1, fileName);

        }



        conn->Close();



        Writing Binary Data with a Command and Parameter Object


        As the data reader objects are read-only, they can't be used to insert into or update a data store. Instead, you would use a command object. You can't insert the binary data into an SQL statement; however, each command object contains a collection of parameter objects that serve as placeholders in the query. Parameters are used in situations where either the data cannot be passed in the query (as in our case) or in situations where the data to be passed won't be resolved until after the command has been constructed. To specify that a command has a parameter, you simply use the special designation @parameterName in the query passed to the command object's constructor. In the following example, I'm specifying that once executed, the command object's parameter collection will include a parameter object named Photo that will contain the data to be used in this SQL UPDATE statement.





        SqlCommand* cmd = new SqlCommand(S"UPDATE Employees SET Photo=@Photo",

        conn);



        The next thing you would do is to construct the parameter object. Most managed providers define a parameter class that is specific to a given data store. For the SQL Server managed provider, this class is called SqlParameter. Here's an example of setting up an SqlParameter object and then adding it to the command object's parameter collection, where the constructor is being used to specify such values as parameter name, data type, data length, parameter direction (input, in this case, as we're setting a database value), and the data value.





        SqlParameter* param = new SqlParameter(S"@Photo",

        SqlDbType::VarBinary,

        image->Length,

        ParameterDirection::Input,

        false,

        0, 0, 0,

        DataRowVersion::Current,

        image);

        cmd->Parameters->Add(param);



        Once the parameters that you've specified for a given command have been constructed and added to the command object, you can then execute the command. As I presented a GetPictureValue in the previous section, here's a SetPictureValue function that illustrates how to use parameter objects for both the image data as well as the EmployeeID column value. This allows you to set up the connection and command one time and then SetPictureValue for each row whose image you wish to set.





        void SetPictureValue(SqlCommand* cmd, String* fileName, int id)

        {

        #pragma push_macro("new")

        #undef new

        FileStream* stream;

        try

        {

        // Read image into buffer.

        stream = new FileStream(fileName,

        FileMode::Open,

        FileAccess::Read);

        int size = Convert::ToInt32(stream->Length);

        Byte image[] = __gc new Byte[size];

        stream->Read(image, 0, size);



        // Create parameter object named @Photo for the image data.

        SqlParameter* paramPhoto =

        new SqlParameter(S"@Photo",

        SqlDbType::VarBinary,

        image->Length,

        ParameterDirection::Input,

        false,

        0, 0, 0,

        DataRowVersion::Current,

        image);

        cmd->Parameters->Add(paramPhoto);



        // Create parameter object named @ID for the passed employee id

        SqlParameter* paramID = new SqlParameter(S"@ID", __box(id));

        cmd->Parameters->Add(paramID);



        // Execute the query and report number of rows updated

        int rowsUpdated = Convert::ToInt32(cmd->ExecuteNonQuery());

        Console::WriteLine(S"{0} rows updated with specified image",

        __box(rowsUpdated));



        // Remove parameters when finished

        cmd->Parameters->Remove(paramPhoto);

        cmd->Parameters->Remove(paramID);

        }

        catch(Exception *e)

        {

        // Handle exception

        }

        __finally

        {

        stream->Close();

        }

        #pragma pop_macro("new")

        }



        Note that you could also have the client set up the command's parameters so that it's only done once and then have the SetPictureValue update the various parameter object members such as value and length. However, I like to minimize the work required by the client, and in the case of allocating an object such as the parameter object, the trade-off of performance vs. client work is such that I don't mind instantiating the parameter object each time. Obviously, this is reversed with regard to the connection because�depending on the environment�a connection may take some time to establish. Therefore, with this function, the client is responsible for creating the connection. In this simple test, I'm calling SetPictureValue for two employees. Also note the two parameters specified in the command object's constructor.





        SqlConnection* conn =

        new SqlConnection(S"Server=localhost;"

        S"Database=NorthWind;"

        S"Integrated Security=true");



        // Construct SQL command object specifying that a

        // parameter named @Photo will be used

        // (constructed and added to the command shortly).

        SqlCommand* cmd = new SqlCommand(S"UPDATE Employees "

        S"SET Photo=@Photo "

        S"WHERE EmployeeID=@ID",

        conn);



        // Open connection

        conn->Open();



        // Set two employee photos

        SetPictureValue(cmd, S"c:\\sample.jpg", 1);

        SetPictureValue(cmd, S"c:\\sample.jpg", 2);



        // Our work is done here. Close the connection.

        conn->Close();



        Using the DataSet Class



        In contrast to the DataReader class, the DataSet class is much easier to use�with the trade-off being much less control than the DataReader offers. As you already saw how to connect to a data source and fill a DataSet in Chapter 6, the following code assumes that you have a DataRow object containing a column (named Photo) that contains binary data. The following code is all you need in order to read a binary value from a DataRow object and then output that data to a file.





        // Assumes that you have a DataRow object called row



        // get photo from row

        Byte pictureData[] = (Byte[])(row->Item[S"Photo"]);



        // write data (image) to disk

        int size = pictureData->GetUpperBound(0);

        if (-1 < size)

        {

        String* fileName = S"Test.jpg";



        FileStream* stream = new FileStream(fileName,

        FileMode::OpenOrCreate,

        FileAccess::Write);

        stream->Write(pictureData, 0,size);

        stream->Close();

        }



        As you can see, all that is required is to simply cast the data returned from the Item property! The reason this is so much simpler is that the DataReader gives you much more control in reading data in terms of specifying data type, precision, size, scale, and so on. This paradigm is fine for the DataReader, since with it you read one column of data at a time for each row. However, the DataSet's internal DataTable object(s) is/are filled with data with a single call to the data adapter's Fill method, so that ADO.NET is obligated to look at the schema of the data store and download all the data at once, pursuant to the type of data. You can verify this by enumerating the column objects defined by each DataSet object's DataTable object, where you'll see that the schema information for that column was also downloaded and, ostensibly, used in determining how to read the data. Therefore, with the column data already downloaded and accessible via the Item property, moving the data into a variable is much easier, albeit at the cost of flexibility.



        That said, writing binary data is just as easy. The following code snippet first constructs a FileStream object encapsulating the file that will contain the image data for a given record. A Byte array is then allocated for the length of the file, and the data is read into that array. A DataRow object representing the record is then updated. As with any update to disconnected data, you then need to call the adapter's Update method to commit the changes to the data store.





        // read specified file

        FileStream* stream = new FileStream(S"Test.jpg",

        FileMode::Open,

        FileAccess::Read);

        Byte pictureData[] = new Byte[stream->Length];

        stream->Read(pictureData, 0, System::Convert::ToInt32(stream->

        Length));

        stream->Close();



        // Acquire row object

        row->Item[S"Photo"] = pictureData;





        DataReader vs. DataSet When Reading Binary Data



        As you've seen, the DataReader and command object provide a much finer level of control when reading and writing binary data. However, the DataSet is much easier to use and has the benefit of being usable in a disconnected environment. What you must keep in mind is that using the DataSet (and associated data adapter) results in all the data�corresponding to the query when you call the data adapter's Fill method�being retrieved at once. Therefore, in some situations a "blended" approach might work best.



        For example, let's say that you want to take advantage of the DataSet and design your system to be disconnected from the data source. However, one of the columns contains extremely large amounts of binary data that the user rarely needs (either for display or update). Instead of taking the performance hit of including such a large, rarely used column in the DataSet, you could exclude it from the DataSet. Then if the user wanted to view or update that column, you could use the DataReader and command objects to synchronously perform these tasks. This would give you the best of both worlds in terms of functionality and performance.




        Demo Application to Read and Write Image Data



        Assuming you've run the SQL Server script provided for the chapter's demo applications and imported the necessary data (as shown in the sidebar entitled "Creating the Sample Database for SQL Server or MSDE" at the beginning of this chapter), you should have a database named ExtendingMFCWithDotNet that includes a table called Contributors. This table defines four columns.





        • ID

          An IDENTITY column used to uniquely identify each row.



        • Name

          The full name of a contributor that is displayed in the list view.



        • Role

          The role of the contributor�such as Author, Co-Author and Tech Editor�that is also displayed in the list view.



        • Photo

          The image column that will contain a picture representing the contributor.



        As mentioned, my main focus in these ADO.NET chapters is on the disconnected side of things. Therefore, this demo will illustrate using the DataSet class to load the Contributors table, displaying the Name and Role values of each record in a list view. When a particular record is selected, the associated image data (Photo value) will be displayed on the dialog. As an added bonus, you'll also see how to display an image from memory using GDI+�as opposed to saving to a temporary file first. The demo will also allow the user to select�from the file system�a different image file for the record. The record can then be saved with this new image data.































        1. To get started, create an MFC dialog-based application called BLOBData and update the Project properties to support Managed Extensions.

        2. Open the stdafx.h file and add the following .NET support directives to the end of the file. You've seen most of these namespaces used in the previous chapters. The drawing namespaces are used in order to facilitate the displaying of the image.





          #using <mscorlib.dll>

          #using <system.dll>

          #using <system.data.dll>

          #using <system.xml.dll>

          #using <system.drawing.dll>

          #using <system.windows.forms.dll>



          using namespace System;

          using namespace System::Data;

          using namespace System::Data::SqlClient;

          using namespace System::Xml;

          using namespace System::Windows::Forms;

          using namespace System::IO;

          using namespace System::Drawing;

          using namespace System::Drawing::Drawing2D;



          #undef MessageBox

        3. Update the project's main dialog as shown in Figure 7-1. The control below the list view is a Picture control with its Type property set to Frame.



          Figure 7-1. The BLOBData demo application's main dialog


        4. Set the list view control's View property to Report and the picture control's Sunken property to True.

        5. Add the DDX value variables for this dialog as shown in Table 7-1.



          Table 7-1. DDX Variables for the BLOBData Demo

          Control

          Variable Type

          Variable Name

          List view control

          CListCtrl

          m_lstContributors

          Picture control

          CStatic

          m_wndPhoto

        6. Define the following ADO.NET member variables for the dialog class.





          class CBLOBDataDlg : public CDialog

          {

          ...

          protected:

          gcroot<DataSet*>dataset;

          gcroot<SqlDataAdapter*>adapter;

          gcroot<DataTable*>contributors;

          gcroot<SqlCommandBuilder*>commandBuilder;

          ...

        7. Add the following list control initialization code to the end of the dialog's OnInitDialog function.





          // All full row selection

          LONG lStyle =

          (LONG)m_lstContributors.SendMessage(LVM_

          GETEXTENDEDLISTVIEWSTYLE);

          lStyle |= LVS_EX_FULLROWSELECT;

          m_lstContributors.SendMessage(LVM_SETEXTENDEDLISTVIEWSTYLE,

          0, (LPARAM)lStyle);



          // Add columns to listview

          m_lstContributors.InsertColumn(0, _T("ID"));

          m_lstContributors.InsertColumn(1, _T("Name"));

          m_lstContributors.InsertColumn(2, _T("Role"));



          TCHAR buff[MAX_PATH];

          GetModuleFileName(NULL, buff, MAX_PATH);

          m_strWorkingDir = System::IO::Path::GetDirectoryName(buff);

        8. Add the following code to the end of the dialog's OnInitDialog function (just before the return statement) to initialize the ADO.NET objects. The first thing that is done is to make a connection to the sample database (ExtendingMFCWithDotNet). A SqlDataAdapter object (adapter) is then constructed with an SQL SELECT statement that retrieves all records from the Contributors table. Once the adapter has been constructed, its MissingSchemaAction property is set so that when the Fill method is called, primary key information will also be acquired. This is needed so that the DataRowCollection::Find method will allow us to search for records by primary key.



          An SqlCommandBuilder object (commandBuilder) is then instantiated, as this demo will also update the data source. After that, the connection is opened, and the SqlDataAdapter::Fill method is called, with the resulting DataTable object being named AllContributors. At this point, we have the data in memory, so the connection to the data store is closed.



          A DataTable object (contributors) is then allocated and points to the AllContributors DataTable created during the Fill method. Finally, a helper function (ReadAllRows) is called to read and display the downloaded records.





          #pragma push_macro("new")

          #undef new

          try

          {

          SqlConnection* conn =

          new SqlConnection(S"Server=localhost;"

          S"Database=ExtendingMFCWithDotNet;"

          S"Integrated Security=true;");



          adapter = new SqlDataAdapter(S"SELECT * FROM"

          "Contributors",

          conn);

          adapter->MissingSchemaAction =

          MissingSchemaAction::AddWithKey;



          commandBuilder = new SqlCommandBuilder(adapter);



          conn->Open();



          dataset = new DataSet();



          adapter->Fill(dataset, S"AllContributors");



          conn->Close(); // No longer needed



          DataTableCollection* tables = dataset->Tables;

          contributors = tables->Item[S"AllContributors"];



          ReadAllRows();

          }

          catch(Exception* e)

          {

          MessageBox::Show(e->Message, S".NET Exception Thrown",

          MessageBoxButtons::OK,

          MessageBoxIcon::Error);

          }

          #pragma pop_macro("new")

        9. At this point, let's implement the ReadAllRows methods. This function enumerates the contributors DataRowCollection member, inserting the ID, name, and role of each record into the list view. Note that the ID is shown in the first column so that it can later be used to search the DataRow record in the DataRowCollection. You could also simply store this value as item data if you didn't want to display it in a production system.





          void CBLOBDataDlg::ReadAllRows()

          {

          try

          {

          CWaitCursor wc;



          m_lstContributors.DeleteAllItems();



          DataRowCollection* rows = contributors->Rows;

          DataRow* row;



          String* id;

          String* name;

          String* role;

          for (int i = 0; i < rows->Count; i++)

          {

          row = rows->Item[i];



          id = row->Item[S"ID"]->ToString();

          name = row->Item[S"Name"]->ToString();

          role = row->Item[S"Role"]->ToString();



          int idx = m_lstContributors.InsertItem(i, (CString)id);

          m_lstContributors.SetItemText(idx, 1, (CString)name);

          m_lstContributors.SetItemText(idx, 2, (CString)role);

          }

          }

          catch(Exception* e)

          {

          MessageBox::Show(e->Message, S".NET Exception Thrown",

          MessageBoxButtons::OK,

          MessageBoxIcon::Error);

          }

          }

        10. Implement the following helper function that will return the index of the currently selected list view item.





          int CBLOBDataDlg::GetSelectedItem()

          {

          int iCurrSel = -1;



          POSITION pos =

          m_lstContributors.GetFirstSelectedItemPosition();

          if (pos)

          iCurrSel = m_lstContributors.GetNextSelectedItem(pos);



          return iCurrSel;

          }

        11. When the user clicks on a given list view item (representing a record in the Contributors table) we want that record's photo displayed. Therefore, implement the following handler for the list view's LVN_ITEMCHANGED message. After retrieving the record ID for the selected item (from the first column), the DataRowCollection::Find method is called to retrieve the DataRow object encapsulating that record. If the row contains data for the Photo column, a Byte array is then allocated and filled with that image data. This buffer is then passed to the DisplayImageFile function. If the Photo column is blank, then the DisplayImageFile is called with a value of NULL to indicate that the static control used to display the image should be erased (in case it's displaying the value from a previously selected record).





          void CBLOBDataDlg::OnLvnItemchangedList1(

          NMHDR *pNMHDR, LRESULT *pResult)

          {

          #pragma push_macro("new")

          #undef new

          try

          {

          CWaitCursor wc;



          LPNMLISTVIEW pNMLV = reinterpret_cast<LPNMLISTVIEW>

          (pNMHDR);



          int iCurrSel = GetSelectedItem();

          if (-1 < iCurrSel)

          {

          int id = atoi(m_lstContributors.GetItemText(iCurrSel,

          0));



          DataRow* row = contributors->Rows->Find(__box(id));

          if (row)

          {

          // get photo from row

          Byte pictureData[] = (Byte[])(row->Item[S"Photo"]);



          // display photo or erase photo based

          // on if we found image data

          int size = pictureData->GetUpperBound(0);

          if (-1 < size)

          {

          // display image

          DisplayImageFile(pictureData);

          }

          else

          {

          // Clear the photo display

          DisplayImageFile(NULL);

          }

          }

          }

          }

          catch(Exception* e)

          {

          MessageBox::Show(e->Message, S".NET Exception Thrown",

          MessageBoxButtons::OK,

          MessageBoxIcon::Error);

          }



          *pResult = 0;

          #pragma pop_macro("new")

          }

        12. Now for the DisplayImageFile function. This function uses the GDI+ Bitmap and Graphic objects in order to display an image file. If data is passed (indicating an image to display), the function first copies the image data buffer into a MemoryStream object, which is then passed to the Bitmap constructor. The device context of the static control is then used to draw the image. If a value of NULL is passed as the image data, then the static control's client area is erased.





          void CBLOBDataDlg::DisplayImageFile(Byte pictureData __gc[])

          {

          #pragma push_macro("new")

          #undef new



          // Forward-declare the following objects so they can be

          // released in the __finally statement

          System::IO::MemoryStream* dataStream = NULL;

          Bitmap* image = NULL;

          Graphics* graphics = NULL;

          CClientDC dc(&m_wndPhoto);



          try

          {

          CWaitCursor wc;



          if (NULL != pictureData)

          {

          // Wrap the pictureData in a Stream object so as to

          // avoid creating a temporary file on disk

          dataStream = new System::IO::MemoryStream(pictureData);



          // Create the GDI+ Bitmap and Graphic objects

          image = new Bitmap(dataStream);

          graphics = Graphics::FromHdc(dc.GetSafeHdc());



          // Erase static control's client area

          RECT clientRect;

          m_wndPhoto.GetClientRect(&clientRect);



          graphics->FillRectangle(SystemBrushes::Control,

          clientRect.left, clientRect.top,

          clientRect.right - clientRect.left,

          clientRect.bottom - clientRect.top);



          // Draw image on static control's client area

          graphics->DrawImage(image, 0, 0, image->Width,

          image->Height);

          }

          else // erase static control's client area

          {

          // Create GDI+ Graphics object from HDC

          graphics = Graphics::FromHdc(dc.GetSafeHdc());



          // Draw background

          RECT clientRect;

          m_wndPhoto.GetClientRect(&clientRect);



          graphics->FillRectangle(SystemBrushes::Control,

          clientRect.left, clientRect.top,

          clientRect.right - clientRect.left,

          clientRect.bottom - clientRect.top);

          }

          }

          catch(Exception* e)

          {

          MessageBox::Show(e->Message, S".NET Exception Thrown",

          MessageBoxButtons::OK,

          MessageBoxIcon::Error);

          }

          __finally

          {

          // Important! Clean up GDI+ and IO resources!

          if (graphics) graphics->Dispose();

          if (image) image->Dispose();

          if (dataStream) dataStream->Dispose();

          }

          #pragma pop_macro("new")

          }

        13. At this point, the application should display any photos stored for a given record. Now let's add the ability to change the photo so as to illustrate how to write image data. To do that, implement an event handler for the Set Photo...button. After displaying the File Open common dialog, the function opens the selected image file using a FileStream object. The file's data is read into a Byte array and, after locating the DataRow being updated (via the DataRowCollection::Find method), its Photo column is set to the Byte array. Finally, the DisplayImageFile presented in the previous step is called.





          void CBLOBDataDlg::OnBnClickedPhotoSet()

          {

          #pragma push_macro("new")

          #undef new

          FileStream* stream;

          try

          {

          CWaitCursor wc;



          int iCurrSel = GetSelectedItem();

          if (-1 < iCurrSel)

          {

          CFileDialog dlg(TRUE);

          dlg.m_pOFN->lpstrInitialDir = m_strWorkingDir;

          dlg.m_pOFN->lpstrTitle = _T("Open an image file");

          dlg.m_ofn.lpstrFilter = _T("Image"

          "Files\0*.bmp;*.gif;*.jpg;\0"

          "All Files (*.*)\0*.*\0\0");

          if (IDOK == dlg.DoModal())

          {

          // read specified file

          CString strFileName = dlg.GetPathName();

          stream = new FileStream(strFileName,

          FileMode::Open,

          FileAccess::Read);

          Byte pictureData[] = new Byte[stream->Length];

          stream->Read(pictureData, 0,

          System::Convert::ToInt32(stream->

          Length));



          // update DataTable

          int id = atoi(m_lstContributors.GetItemText(

          iCurrSel, 0));

          DataRow* row = contributors->Rows->Find(__box(id));

          if (row)

          {

          row->Item[S"Photo"] = pictureData;

          }



          // display image

          DisplayImageFile(pictureData);

          }

          }

          else MessageBox::Show(S"You must first select a

          contributor");

          }

          catch(Exception* e)

          {

          MessageBox::Show(e->Message, S".NET Exception Thrown",

          MessageBoxButtons::OK,

          MessageBoxIcon::Error);

          }

          __finally

          {

          stream->Close();

          }

          #pragma pop_macro("new")

          }

        14. The last step is to implement a handler for the Commit Data button so that, at any time, the user can commit the changes made in the application to the DataSet object's underlying data source. As you saw in the previous chapter, this is accomplished via calling the data adapter's Update function, and it will only work if a command builder has been set up to automatically generate the appropriate UpdateCommand.





          void CBLOBDataDlg::OnBnClickedCommitData()

          {

          try

          {

          CWaitCursor wc;



          adapter->Update(contributors);

          MessageBox::Show(S"Data successfully saved", S"Success",

          MessageBoxButtons::OK,

          MessageBoxIcon::Information);

          }

          catch(Exception* e)

          {

          MessageBox::Show(e->Message, S".NET Exception Thrown",

          MessageBoxButtons::OK,

          MessageBoxIcon::Error);

          }

          }



        At this point, you should have a fully functional application that allows you to both read and write image data to a SQL Server database. Running this application should provide results similar to those shown in Figure 7-2.



        Figure 7-2. The BLOBData demo application illustrates how to read and write image data.














           < Day Day Up >