BusinessRx Community

Dedicated to the advancement of software, technology and the people who devote their lives to it.

Welcome to BusinessRx Community Sign in | Join | Help
in Search

BusinessRx Reading List

These blog entries are written by industry experts and leaders. We consider this content to be a good read for any software developer or web technologist.

Browse by Tags

All Tags » Data » SQL Server » ASP.NET   (RSS)

  • LINQ to SQL (Part 6 - Retrieving Data Using Stored Procedures)

    Over the last few weeks I've been writing a series of blog posts that cover LINQ to SQL.  LINQ to SQL is a built-in O/RM (object relational mapper) that ships in the .NET Framework 3.5 release, and which enables you to model relational databases using .NET classes.  You can use LINQ expressions to query the database with them, as well as update/insert/delete data.

    Below are the first five parts of my LINQ to SQL series:

    In these previous LINQ to SQL blog posts I demonstrated how you could use LINQ query expressions to programmatically retrieve data from a database.

    In today's blog post I'll cover how you can also use database stored procedures (SPROCs) and user defined functions (UDFs) with your LINQ to SQL data model.  Today's blog post will specifically cover how to call SPROCs to query and retrieve data from the database.  In my next blog post in this series I'll then show how you can optionally also use SPROCs to update/insert/delete data from the database.

    To SPROC or not to SPROC?  That is the question....

    The question of whether to use Dynamic SQL generated by an ORM or instead use Stored Procedures when building a data layer is a topic that generates endless (very passionate) debate amongst developers, architects and DBAs.  A lot of people much smarter than me have written on this topic, so I won't rehash the arguments for and against each side here.

    The LINQ to SQL ORM that ships in .NET 3.5 is pretty flexible, and can be used to create data model classes whose object model can be independent of the underlying database schema, and which can encapsulate business logic and validation rules that work regardless of whether the data model is populated/persisted via dynamic SQL or via SPROCs.

    In my LINQ to SQL Part 3: Querying our Database post I discussed how you can write LINQ query expressions against a LINQ to SQL data model using code like below:

    When you write LINQ query expressions like this the LINQ to SQL ORM will execute the necessary dynamic SQL for you to retrieve Product objects that matches your query.

    As you'll learn in this post, you can also optionally map SPROCs in the database to your LINQ to SQL DataContext class, which allows you to alternatively retrieve the same Product objects by calling a stored procedure instead:

     

    This ability to use both dynamic SQL and SPROCs with a clean data model layer is pretty powerful, and provides a great deal of flexibility when working on projects.

    The Steps to Map and Call a SPROC using LINQ to SQL

    In my Part 2: Defining our Data Model Classes tutorial I discussed how to use the LINQ to SQL ORM designer to create a LINQ to SQL class model like below:

    Notice above how there are two panes on the LINQ to SQL ORM designer surface.  The left pane enables us to define data model classes that map to our database.  The right method pane allows us to optionally map SPROCs (and UDFs) to our LINQ to SQL DataContext object, which we can then use in-place of dynamic SQL to populate the data model objects.

    How to Map a SPROC to a LINQ to SQL DataContext

    To map SPROCs to our DataContext class, let's first go to the VS 2008 Server Explorer window and look at the SPROCs within our database:

    We can double click any of the SPROCs above to open and edit them.  For example, below is the "CustOrderHist" SPROC in Northwind:

    To map the above SPROC to our LINQ to SQL DataContext, we can drag/drop it from the Server Explorer onto our LINQ to SQL ORM designer.  This will automatically create a new method on our LINQ to SQL DataContext class like below:

    By default the method name created on the DataContext class will be the same as the SPROC name, and the return type of the method will be an automatically created type that follows the "[SprocName]Result" naming pattern. For example: the SPROC above would return a sequence of "CustOrderHistResult" objects.  We could optionally change the name of the method by selecting it in the designer and then use the property grid to rename it.

    How to Call our Newly Mapped SPROC

    Once we've done the steps above to map a SPROC onto our DataContext class, it is easy to use it to programmatically retrieve data.  All we need to-do is call the new method we mapped on our DataContext class to get back a sequence of strongly typed results from the SPROC:

    Calling the SPROC in VB:

    Calling the Sproc in C#:

    In addition to programming looping over the result like in the code samples above, I could also obviously bind the results to any UI control to display them.  For example, the below code databinds the result of our SPROC to a <asp:gridview> control:

    Which then displays the product history of our customer on a page like so:

    Mapping the Return Type of SPROC Methods to Data Model Classes

    In the "CustOrderHist" SPROC example above the stored procedure returned a sequence of product history results containing two columns of data: the ProductName of the product, and the Total Number of orders the customer has made for that product.  The LINQ to SQL designer automatically defined a new "CustOrderHistResult" class to represent this result.

    We could alternatively choose to have the return result of a SPROC map to an existing data model class we have already defined in the LINQ to SQL designer (for example: an existing Product or Order entity class). 

    For example, assume we have a "GetProductsByCategory" SPROC in our database that returns product information like so:

    Like before we can create a "GetProductsByCategory" method on our DataContext that calls this SPROC by dragging it onto our LINQ to SQL designer.  Rather than just dropping the SPROC anywhere on the designer, though, we'll instead drop the SPROC on top of the existing "Product" class in our data model designer:

    This gesture of dropping the SPROC onto the Product class tells the LINQ to SQL designer to have the "GetProductsByCategory" method return a sequence of "Product" objects as a return result:

     

    One of the cool things about having our SPROC return "Product" objects like above is that LINQ to SQL will automatically track the changes made to the returned Product objects just like it would Product objects returned via LINQ queries.  When we call the "SubmitChanges()" method on our DataContext, the changes we have made to these objects will automatically be saved back to the database.

    For example, we could write the code below to retrieve (using a SPROC) and change the price of all products within a specific Category to be 90% of their current value:

    When we call SubmitChanges() at the end it will transactionally update all of the product prices.  To understand more about how change tracking and the SubmitChanges() method work, as well as about how Validation Business Logic can be added to data model entities, please read my LINQ to SQL Part 4: Updating our Database tutorial. 

    In my next blog post in this LINQ to SQL series I'll also cover how you can replace the dynamic insert/update/delete SQL generated by the ORM with custom SPROCs that handle the database updates instead.  The nice thing is that the code above wouldn't change at all if I configured my DataContext to use SPROCs for updates - it would purely be a mapping layer change and the code written against my data model would be oblivious to it. 

    Handling SPROC Output Parameters

    LINQ to SQL maps "out" parameters in SPROCs as reference parameters (ref keyword), and for value types declares the parameter as nullable.

    For example, consider the below "GetCustomerDetails" SPROC which takes a CustomerID as an input parameter, and which returns the company name as an output parameter in addition to its order history as a query result:

    If we drag the above SPROC onto our "Order" class in the LINQ to SQL designer, we could then write the below code to call it:

    VB:

    C#:

    Notice in the code above how the SPROC helper method returns back a sequence of Order objects - but also then returns the CompanyName as an output parameter to the helper method.

    Handling Multiple Result Shapes from SPROCs

    When a stored procedure can return multiple result shapes, the return type of the SPROC method on the DataContext cannot be strongly typed to a single class shape.  For example, consider the SPROC below which returns either a product result or an order result depending on the input parameter:

    LINQ to SQL supports the ability to create SPROC helper methods that can return either a Product or Order shape by adding a partial "NorthwindDataContext" class to the project that defines a method (which in this case we'll call "VariablesShapeSample") that invokes the SPROC and returns an IMultipleResult object like so:

    VB:

    C#:

    Once this method is added into our project we can then call it and convert the result to be either a Product or Order sequence when we are using it:

    VB:

    C#:

    Supporting User Defined Functions (UDFs)

    In addition to SPROCS, LINQ to SQL also supports both scalar-valued and table-valued user defined functions (UDFs), as well as the in-line counterpart to both.  Once added to your DataContext as a method, you can use these UDF functions within your LINQ queries.

    For example, consider a simple scalar user defined function called "MyUpperFunction":

    We can drag/drop it from the Visual Studio Server Explorer onto our LINQ to SQL Designer to add it as a method on our DataContext:

    We can then use this UDF function inline within our LINQ expressions when writing queries against our LINQ to SQL data model (notice it is being used within the "where" clause below):

    VB:

    C#:

    If you use the LINQ to SQL Debug Visualizer that I blogged about here, you can see how LINQ to SQL transforms the above expression queries into raw SQL that execute the UDF inside the database at runtime:

    Summary

    LINQ to SQL supports the ability to call Stored Procedures and UDFs within the database and nicely integrate them into our data model.  In this blog post I demonstrated how you can use SPROCs to easily retrieve data and populate our data model classes.  In my next blog post in this series I'll cover how you can also use SPROCs to override the update/insert/delete logic when you SubmitChanges() on your DataContext to persist back to the database.

    Hope this helps,

    Scott

  • LINQ to SQL Debug Visualizer

    Probably the biggest programming model improvement being made in .NET 3.5 is the work being done to make querying data a first class programming concept.  We call this overall querying programming model "LINQ", which stands for .NET Language Integrated Query.  Developers can use LINQ with any data source, and built-in libraries are included with .NET 3.5 that enable LINQ support against Objects, XML, and Databases.

    Earlier this summer I started writing a multi-part blog series that discusses the built-in LINQ to SQL provider in .NET 3.5.  LINQ to SQL is an ORM (object relational mapping) implementation that allows you to model a relational database using .NET classes.  You can then query the database using LINQ, as well as update/insert/delete data from it.  LINQ to SQL fully supports transactions, views, and stored procedures.  It also provides an easy way to integrate data validation and business logic rules into your data model.

    You can learn more about LINQ to SQL by reading my posts below (more will be coming soon):

    Using the LINQ to SQL Debug Visualizer

    One of the nice development features that LINQ to SQL supports is the ability to use a "debug visualizer" to hover over a LINQ expression while in the VS 2008 debugger and inspect the raw SQL that the ORM will ultimately execute at runtime when evaluating the LINQ query expression.

    For example, assume we write the below LINQ query expression code against a set of data model classes:

    We could then use the VS 2008 debugger to hover over the "products" variable after the query expression has been assigned:

    And if we click the small magnifying glass in the expression above, we can launch the LINQ to SQL debug visualizer to inspect the raw SQL that the ORM will execute based on that LINQ query:

    If you click the "Execute" button, you can even test out the SQL query and see the raw returned results that will be returned from the database:

    This obviously makes it super easy to see precisely what SQL query logic LINQ to SQL ORM is doing for you. 

    You can learn even more about how all this works by reading the Part 3: Querying our Database segment in my LINQ to SQL series above.

    How to Install the LINQ to SQL Debug Visualizer

    The LINQ to SQL Debug Visualizer isn't built-in to VS 2008 - instead it is an add-in that you need to download to use.  You can download a copy of it here.

    The download contains both a binary .dll assembly version of the visualizer (within the \bin\debug directory below), as well as all of the source code for the visualizer:

    To install the LINQ to SQL debug visualizer, follow the below steps:

    1) Shutdown all running versions of Visual Studio 2008

    2) Copy the SqlServerQueryVisualizer.dll assembly from the \bin\debug\ directory in the .zip download above into your local \Program Files\Microsoft Visual Studio 9.0\Common7\Packages\Debugger\Visualizers\ directory:

    3) Start up Visual Studio 2008 again.  Now when you use the debugger with LINQ to SQL you should be able to hover over LINQ query expressions and inspect their raw SQL (no extra registration is required).

    Hope this helps,

    Scott

  • LINQ to SQL (Part 5 - Binding UI using the ASP:LinqDataSource Control)

    Over the last few weeks I've been writing a series of blog posts that cover LINQ to SQL.  LINQ to SQL is a built-in O/RM (object relational mapper) that ships in the .NET Framework 3.5 release, and which enables you to easily model relational databases using .NET classes.  You can use LINQ expressions to query the database with them, as well as update/insert/delete data.

    Below are the first four parts of my LINQ to SQL series:

    In these previous LINQ to SQL blog posts I focused on how you can programmatically use LINQ to SQL to easily query and update data within a database.

    In today's blog post I'll cover the new <asp:LinqDataSource> control that is shipping as part of ASP.NET in the upcoming .NET 3.5 release.  This control is a new datasource control for ASP.NET (like the ObjectDataSource and SQLDataSource controls that shipped with ASP.NET 2.0) which makes declaratively binding ASP.NET UI controls to LINQ to SQL data models super easy.

    Sample Application We'll be Building

    The simple data editing web application I'll walkthrough building in this tutorial is a basic data entry/manipulation front-end for products within a database:

    The application will support the following end-user features:

    1. Allow users to filter the products by category
    2. Allow users to sort the product listing by clicking on a column header (Name, Price, Units In Stock, etc)
    3. Allow users to skip/page over multiple product listings (10 products per page)
    4. Allow users to edit and update any of the product details in-line on the page
    5. Allow users to delete products from the list

    The web application will be implemented with a clean object-oriented data model built using the LINQ to SQL ORM.

    All of the business rules and business validation logic will be implemented in our data model tier - and not within the UI tier or in any of the UI pages.  This will ensure that: 1) a consistent set of business rules are used everywhere within the application, 2) we write less code and don't repeat ourselves, and 3) we can easily modify/adapt our business rules at a later date and not have to update them in dozens of different places across our application.

    We will also take advantage of the built-in paging/sorting support within LINQ to SQL to ensure that features like the product listing paging/sorting are performed not in the middle-tier, but rather in the database (meaning only 10 products are retrieved from the database at any given time - we are not retrieving thousands of rows and doing the sorting/paging within the web-server). 

    What is the <asp:LinqDataSource> control and how does it help?

    The <asp:LinqDataSource> control is an ASP.NET control that implements the DataSourceControl pattern introduced with ASP.NET 2.0.  It is similar to the ObjectDataSource and SqlDataSource controls in that it can be used to declaratively bind other ASP.NET controls on a page to a datasource.  Where it differs is that instead of binding directly to a database (like the SqlDataSource) or to a generic class (like the ObjectDataSource), the <asp:linqdatasource> is designed to bind against a LINQ enabled data model.

    One of the benefits of using the <asp:linqdatasource> control is that it leverages the flexibility that LINQ based ORMs provide.  You don't need to define custom query/insert/update/delete methods for the datasource to call - instead you can point the <asp:linqdatasource> control at your data model, identify what entity table you want it to work against, and then bind any ASP.NET UI control against the <asp:linqdatasource> and have them work with it.

    For example, to get a basic product listing UI on my page that works against Product entities within a LINQ to SQL data model, I could simply declare a <asp:linqdatasource> on my page that points to my LINQ to SQL datacontext class, and identify the entities (for example: Products) in the LINQ to SQL data model I want to bind against.  I could then point a GridView at it (by settings its DataSourceID property) to get a grid-like view of the Product content:

    Without having to-do anything else, I can run the page and have a listing of my Product data with built-in support for paging and sorting over the data.  I can add a edit/delete button on the Grid and automatically have update support as well.  I don't need to add any methods, map any parameters, or write any code for the <asp:LinqDataSource> to handle both these querying and updating scenarios - it can work against the LINQ to SQL data model we point it against and do these operations automatically.  When updates are made, the LINQ to SQL ORM will automatically ensure that all business rules and validation logic we've added (as partial methods) to the LINQ to SQL data model pass before persisting anything to the database.

    Important: The beauty of LINQ and LINQ to SQL is that it obviously isn't tied to being used only in UI scenarios - or with particular UI binding controls like the LinqDataSource.  As you've seen in my previous posts in this series, writing code using the LINQ to SQL ORM is extremely clean.  You can always write custom UI code to directly work against your LINQ to SQL data model if you prefer, or when you find a UI scenario that isn't particularly suited to using the <asp:linqdatasource>. 

    The below sections walkthrough using LINQ to SQL and the <asp:LinqDataSource> control to build the web application scenario I defined above.

    Step 1: Define our Data Model

    We'll begin working on the application by first defining the data model we'll use to represent our database. 

    I discussed how to create a LINQ to SQL data model using VS 2008's LINQ to SQL designer in Part 2 of this series.  Below is a screenshot of the data model classes I can quickly create using the LINQ to SQL designer to model the "Northwind" sample database:

    We'll revisit our data model in Step 5 of this tutorial below when we add some business validation rules to it.  But to begin with we'll just use the above data model as-is to build our UI.

    Step 2: Creating a Basic Product Listing

    We'll start our UI by creating an ASP.NET page with a <asp:gridview> control on it and use some CSS to style it:

    We could write code to programmatically bind our data model to the GridView (like I did in Part 3 of this series), or alternatively I could use the new <asp:linqdatasource> control to bind the GridView to our data model. 

    VS 2008 includes build-in designer support to make it easy to connect up our GridView (or any other ASP.NET server control) to LINQ data.  To bind our grid above to the data model we created earlier, we can switch into design-view, select the GridView, and then select the "New Data Source..." option within the "Choose Data Source:" drop-down:

    This will bring up a dialog box that lists the available datasource options to create.  Select the new "LINQ" option in the dialog box and name the resulting <asp:linqdatasource> control you want to create:

    The <asp:linqdatasource> designer will then display the available LINQ to SQL DataContext classes that your application can use (including those in class libraries that you are referencing):

    We'll want to select the data model we created with the LINQ to SQL designer earlier.  We'll then want to select the table within our data model that we want to be the primary entity for the <asp:linqdatasource> to bind against.  For this scenario we'll want to select the "Products" entity class we built.  We'll also want to select the "Advanced" button and enable updates and deletes for the datasource:

    When we click the "Finish" button above, VS 2008 will declare a <asp:linqdatasource> within our .aspx page, and update the <asp:gridview> to point to it (via its DataSourceID property).  It will also automatically provide column declarations in the Grid based on the schema of the Product entity we choose to bind against:

    We can then pull up the "smart task" context UI of the GridView and indicate that we want to enable paging, sorting, editing and deleting on it:

    We can then press F5 to run our application, and have a product listing page with full paging and sorting support (note the paging indexes at the bottom of the grid below):

    We can also select the "edit" or "delete" button on each row to update the data:

    If we flip into source view on the page, we'll see that the markup of the page contains the content below.  The <asp:linqdatasource> control points at the LINQ to SQL DataContext we created earlier, as well as the entity table we want to bind against.  The GridView then points at the <asp:linqdatasource> control (via its DataSourceID) and indicates which columns should be included in the grid, what their header text should be, as well as what sort expression to use when the column header is selected. 

    Now that we have the basics of our web UI working against our LINQ to SQL data-model, we can go ahead and further customize the UI and behavior.

    Step 3: Cleaning up our Columns

    Our GridView above has a lot of columns defined within it, and two of the column values (the SupplierID and the CategoryID) are currently foreign-key numbers -- which certainly isn't the ideal way to represent them to an end-user. 

    Removing Unnecessary Columns 

    We can start cleaning up our UI by deleting a few of the columns we don't need.  I can do this in source mode (simply nuke the <asp:boundfield> declarations) or in designer mode (just click on the column in the designer and choose the "Remove" task).  For example, we could remove the "QuantityPerUnit" column below and re-run our application to get this slightly cleaner UI:

    If you have used the <asp:ObjectDataSource> control before and explicitly passed update parameters to update methods (the default when using DataSet based TableAdapters) one of the things you know can be painful is that you have to change the method signatures of your TableAdapter's update methods when the parameters based by your UI are modified.  For example: if we deleted a column in our grid (like above), we'd end up having to modify our TableAdapter to support update methods without that parameter.

    One of the really nice things about the <asp:LinqDataSource> control is that you do not need to-do these types of changes.  Simply delete (or add) a column from your UI and re-run the application - no other changes are required.  This makes changing web UI built using the <asp:LinqDataSource> much easier, and enables much faster scenarios iterations within an application.

    Cleaning up the SupplierID and CategoryID Columns

    Currently we are displaying the foreign-key integer values in our GridView for the Supplier and Category of each Product: 

    While accurate from a data model perspective, it isn't very end-user friendly.  What I really want to-do is to display the CategoryName and SupplierName instead, and provide a drop-downlist while in Edit mode to enable end-users to easily associate the SupplierID and CategoryID values.

    I can change the GridView to display the Supplier Name and Category Name instead of the ID's by replacing the default <asp:BoundField> in our GridView with an <asp:TemplateField>.  Within this TemplateField I can add any content I want to customize the look of the column. 

    In the source code below I'm going to take advantage of the fact that each Product class in the LINQ to SQL data model we created has a Supplier and Category property on it. What this means is that I can easily databind their Supplier.CompanyName and Category.CategoryName sub-properties within our Grid:

     

    And now when I run the application I get the human readable Category and Supplier name values instead:

    To get drop-down list UI for the Supplier and Category columns while in Edit-Mode in the Grid, I will first add two additional <asp:LinqDataSource> controls to my page.  I will configure these to bind against the Categories and Suppliers within the LINQ to SQL data model we created earlier:

    I can then go back to the <asp:TemplateField> columns we added to our GridView earlier and customize their edit appearance (by specifying an EditItemTemplate).  We'll customize each column to have a dropdownlist control when in edit mode, where the available values in the dropdownlists are pulled from the categories and suppliers datasource controls above, and where we two-way databind the selected value to the Product's SupplierID and CategoryID foreign keys:

    And now when end-users click edit in the GridView, they are presented a drop-down list of all valid Supplier's to associate the product with:

    And when they hit save the Product is updated appropriately (the GridView will use the DropDownList's currently selected value to bind the SupplierID).

    Step 4: Filtering our Product Listing

    Rather than show all products within the database, we can update our UI to include a dropdownlist that allows the user to filter the products by a particular category. 

    Because we already added a <asp:LinqDataSource> control to the page earlier that references our Categories within our LINQ to SQL data model, all I need to-do to create a drop-downlist control at the top of the page that binds against this.  For example:

    When I run the page I'll now get a filter dropdownlist of all categories at the top of the page:

    My last step is to configure the GridView to only show those Products in the category the end-user selects from the dropdownlist.  The easiest way to-do this is by selecting the "Configure DataSource" option in the GridView smart task:

    This will bring me back to the <asp:LinqDataSource> control's design-time UI that we used at the very beginning of this tutorial.  I can select the "Where" button within this to add a binding filter to the datasource control.  I can add any number of filter expressions, and declaratively pull the values to filter by from a variety of places (for example: from the querystring, from form-values, from other controls on the page, etc):

    Above I'm going to choose to filter by the Products by their CategoryID value, and then retrieve this CategoryID from the DropDownList control we just created on our page:

    When we hit finish, the <asp:linqdatasource> control in our page will have been updated to reflect this filter clause like so:

    And when we now run the page the end-user will now be able to select from the available Categories in the filter drop-downlist and page, sort, edit and delete just the products in that category:

    The <asp:LinqDataSource> control will automatically apply the appropriate LINQ filter expression when working against our LINQ to SQL data model classes to ensure that only the required data is retrieved from the database (for example: in the Grid above only the 3 rows of Product data from the second page of Confection products will be retrieved from the database).

    You can optionally handle the Selecting event on the <asp:LinqDataSource> if you want to write a custom LINQ expression in code to completely customize the query instead.

    Step 5: Adding Business Validation Rules

    As I discussed in Part 4 of this LINQ to SQL series, when we define LINQ to SQL data models we will automatically have a default set of schema based validation constraints added to our data model classes.  This means that if I try and enter a null value for a required column, try and assign a string to an integer, or assign a foreign-key value to a row that doesn't exist, our LINQ to SQL data model will raise an error and ensure that our database integrity is maintained.

    Basic schema validation is only a first step, though, and is rarely enough for most real-world applications.  Typically we'll want/need to add additional business rules and application-level validation to our data model classes.  Thankfully LINQ to SQL makes adding these types of business validation rules easy (for details on the various validation approaches available, please read Part 4 of my LINQ to SQL series).

    Example Business Validation Rule Scenario

    For example, let's consider a basic business logic rule we might want to enforce.  Specifically, we want to ensure that a user of our application can't discontinue a product while we still have units on backorder for it:

    If a user tries to save the above row, we'll want to prevent this change from being persisted and throw an appropriate error telling the user how to fix it.

    Adding a Data Model Validation Rule

    The wrong place to add this type of business validation rule is in the UI layer of our application.  Adding it in the UI layer of our application will mean that the rule will be specific to only that one place, and will not be automatically enforced when we add another page to our application that also updates Products.  Distributing business rules/logic in our UI layer will also make life extremely painful as our application grows in size - since changes/updates to our business will necessitate making code changes all over the place.

    The right place to specify this type of business logic validation is instead in our LINQ to SQL data model classes that we defined earlier.  As I discussed in Part 4 of this series, all classes generated by the LINQ to SQL designer are defined as "partial" classes - which means that we can easily add additional methods/events/properties to them.  The LINQ to SQL data model classes automatically call validation methods that we can implement to enforce custom validation logic within them.

    For example, I could add a partial Product class to my project that implements the OnValidate() partial method that LINQ to SQL calls prior to persisting a Product entity.  Within this OnValidate() method I could add the following business rule to enforce that products can't have a Reorder Level if the product is discontinued:

    Once I add the above class into my LINQ to SQL project, the above business rule will be enforced anytime anyone uses my data model to try and modify the database.  This is true for both updating existing Products, as well as adding new Products into the database.

    Because the <asp:LinqDataSource> that we defined in our pages above works against our LINQ to SQL data model classes, all of its update/insert/delete logic will now have to pass the above validation check prior to the change being persisted.  We do not need to-do anything to our UI tier in order for this validation to occur - it will automatically be applied anywhere and everywhere our LINQ to SQL data model is used.

    Adding Nice Error Handling in our UI Tier

    By default if a user now uses our GridView UI to enter a non-valid UnitsOnOrder/Discontinued combination, our LINQ to SQL data model classes will raise an exception.  The <asp:LinqDataSource> will in turn catch this error and provides an event that users can use to handle it.  If no one handles the event then the GridView (or other) control bound to the <asp:LinqDataSource> will catch the error and provide an event for users to handle it.  If no one handles the error there then it will be passed up to the Page to handle, and if not there to the global Application_Error() event handler in the Global.asax file.  Developers can choose any place along this path to insert appropriate error handling logic to provide the right end-user experience.

    For the application we defined above, probably the best place to handle any update errors is by handling the RowUpdated event on our GridView.  This event will get fired every time an update is attempted on our datasource, and we can access the exception error details if the update event fails.  We can add the below code to check if an error occurs, and if so display an appropriate error message to the end-user:

    Notice above how we have not added any validation specific logic into our UI.  Instead, I am retrieving the validation error message string we raised in our business logic and am using it to display an appropriate message to the end-user (I am then displaying a more generic error message in the event of other failures). 

    Notice how I'm also indicating above that I want the GridView to stay in Edit mode when an error occurs - that way the user can avoid losing their changes, and can modify the values they entered and click "update" again to try and save them.  We can then add a <asp:literal> control with the "ErrorMessage" ID anywhere we want on our page to control where where we want the error message to be displayed:

    And now when we try and update the Product with an invalid value combination we'll see an error message indicating how to fix it:

    The nice thing about using this approach is that I can now add or change my data model's business rules and not have to modify any of my UI tier's code to have them pick up and honor the changes.  The validation rules, and corresponding error messages, can be written and centralized in one place in my data model and automatically applied everywhere.

    Summary

    The <asp:LinqDataSource> control provides an easy way to bind any ASP.NET UI control to a LINQ to SQL data model.  It enables UI controls to both retrieve data from a LINQ to SQL data model, as well as cleanly apply updates/inserts/deletes to it. 

    In our application above we used the LINQ to SQL ORM designer to create a clean, object oriented data model.  We then added three ASP.NET UI controls to our page (a GridView, a DropDownList, and a ErrorMessage Literal), and added three <asp:LinqDataSource> controls to bind Product, Category, and Supplier data from it:

    We then wrote 5 lines of business validation logic in our data model, and 11 lines of UI error handling logic. 

    The end result is a simple web application with custom UI that allows users to dynamically filter product data by category, efficiently sort and page over the product results, inline-edit the product data to save updates (providing our business rules pass), and delete products from the system (also providing our business rules allow it).

    In future posts in this series I'll cover more LINQ to SQL scenarios including optimistic concurrency, lazy and eager loading, table mapping inheritance, and custom SQL/SPROC usage.

    Next week I also plan to start a new series of blog posts that cover the new <asp:ListView> control - which is a new control that will ship with the ASP.NET release in .NET 3.5.  It provides total control over the markup generated for data scenarios (no tables, no spans, no inline styles...), while also delivering built-in support for paging, sorting, editing, and insertion scenarios.  For example, we could optionally use it to replace the default Grid layout look of our application above with a completely custom look and feel.  Best of all, I could replace it within the above page in my application and not have to change my Data Model, the <asp:linqdatasource> declaration, or my code-behind UI error handling logic at all.

    Hope this helps,

    Scott

  • LINQ to SQL (Part 4 - Updating our Database)

    Over the last few weeks I've been writing a series of blog posts that cover LINQ to SQL.  LINQ to SQL is a built-in O/RM (object relational mapper) that ships in the .NET Framework 3.5 release, and which enables you to easily model relational databases using .NET classes.  You can use LINQ expressions to query the database with them, as well as update/insert/delete data.

    Below are the first three parts of my LINQ to SQL series:

    In today's blog post I'll cover how we we can use the data model we created earlier, and use it to update, insert, and delete data.  I'll also show how we can cleanly integrate business rules and custom validation logic with our data model.

    Northwind Database Modeled using LINQ to SQL

    In Part 2 of this series I walked through how to create a LINQ to SQL class model using the LINQ to SQL designer that is built-into VS 2008.  Below is a class model created for the Northwind sample database and which I'll be using in this blog post:

     

    When we designed our data model using the LINQ to SQL data designer above we defined five data model classes: Product, Category, Customer, Order and OrderDetail.  The properties of each class map to the columns of a corresponding table in the database.  Each instance of a class entity represents a row within the database table.

    When we defined our data model, the LINQ to SQL designer also created a custom DataContext class that provides the main conduit by which we'll query our database and apply updates/changes.  In the example data model we defined above this class was named "NorthwindDataContext".  The NorthwindDataContext class has properties that represent each Table we modeled within the database (specifically: Products, Categories, Customers, Orders, OrderDetails).

    As I covered in Part 3 of this blog series, we can easily use LINQ syntax expressions to query and retrieve data from our database using this NorthwindDataContext class.  LINQ to SQL will then automatically translate these LINQ query expressions to the appropriate SQL code to execute at runtime.

    For example, I could write the below LINQ expression to retrieve a single Product object by searching on the Product name:

    I could then write the LINQ query expression below to retrieve all products from the database that haven't yet had an order placed for them, and which also cost more than $100:

    Note above how I am using the "OrderDetails" association for each product as part of the query to only retrieve those products that have not had any orders placed for them.

    Change Tracking and DataContext.SubmitChanges()

    When we perform queries and retrieve objects like the product instances above, LINQ to SQL will by default keep track of any changes or updates we later make to these objects.  We can make any number of queries and changes we want using a LINQ to SQL DataContext, and these changes will all be tracked together. 

    Note: LINQ to SQL change tracking happens on the consuming caller side - and not in the database.  This means that you are not consuming any database resources when using it, nor do you need to change/install anything in the database to enable it.

    After making the changes we want to the objects we've retrieved from LINQ to SQL, we can then optionally call the "SubmitChanges()" method on our DataContext to apply the changes back to the database.  This will cause LINQ to SQL to dynamically calculate and execute the appropriate SQL code to update the database.

    For example, I could write the below code to update the price and # of units in stock of the "Chai" product in the database:

    When I call northwind.SubmitChanges() above, LINQ to SQL will dynamically construct and execute a SQL "UPDATE" statement that will update the two product property values we modified above.

    I could then write the below code to loop over unpopular, expensive products and set the "ReorderLevel" property of them to zero:

    When I call northwind.SubmitChanges() above, LINQ to SQL will calculate and execute an appropriate set of UPDATE statements to modify the products who had their ReorderLevel property changed.

    Note that if a Product's property values weren't changed by the property assignments above, then the object would not be considered changed and LINQ to SQL would therefore not execute an update for that product back to the database.  For example - if the "Chai" product's unitprice was already $2 and the number of units in stock was 4, then calling SubmitChanges() would not cause any database update statements to execute.  Likewise, only those products in the second example whose ReorderLevel was not already 0 would be updated when the SubmitChanges() method was called.

    Insert and Delete Examples

    In addition to updating existing rows in the database, LINQ to SQL obviously also enables you to insert and delete data.  You can accomplish this by adding/removing data objects from the DataContext's table collections, and by then calling the SubmitChanges() method.  LINQ to SQL will keep track of these additions/removals, and automatically execute the appropriate SQL INSERT or DELETE statements when SubmitChanges() is invoked.

    Inserting a New Product

    I can add a new product to my database by creating a new "Product" class instance, setting its properties, and then by adding it to my DataContext's "Products" collection:

    When we call "SubmitChanges()" above a new row will be created in our products table.

    Deleting Products

    Just as I can express that I want to add a new Product to the database by adding a Product object into the DataContext's Products collection, I can likewise express that I want to delete a product from a database by removing it from the DataContext's Products collection:

    Note above how I'm retrieving a sequence of discontinued products that no one has ever ordered using a LINQ query, and then passing it to the RemoveAll() method on my DataContext's "Products" collection.  When we call "SubmitChanges()" above all of these Product rows will be deleted from our products table.

    Updates across Relationships

    What makes O/R mappers like LINQ to SQL extremely flexible is that they enable us to easily model cross-table relationships across our data model.  For example, I can model each Product to be in a Category, each Order to contain OrderDetails for line-items, associate each OrderDetail line-item with a Product, and have each Customer contain an associated set of Orders.  I covered how to construct and model these relationships in Part 2 of this blog series.

    LINQ to SQL enables me to take advantage of these relationships for both querying and updating my data. For example, I could write the below code to create a new Product and associate it with an existing "Beverages" category in my database like so:

    Note above how I'm adding the Product object into the Category's Products collection.  This will indicate that there is a relationship between the two objects, and cause LINQ to SQL to automatically maintain the foreign-key/primary key relationship between the two when I call "SubmitChanges()".

    For another example of how LINQ to SQL can help manage cross-table relationships for us and help clean up our code, let's look at an example below where I'm creating a new Order for an existing customer.  After setting the required ship date and freight costs for the order, I then create two order line-item objects that point to the products the customer is ordering.  I then associate the order with the customer, and update the database with all of the changes.

    As you can see, the programming model for performing all of this work is extremely clean and object oriented. 

    Transactions

    A transaction is a service provided by a database (or other resource manager) to guarantee that a series of individual actions occur atomically - meaning either they all succeed or they all don't, and if they don't then they are all automatically undone before anything else is allowed to happen.

    When you call SubmitChanges() on your DataContext, the updates will always be wrapped in a Transaction.  This means that your database will never be in an inconsistent state if you perform multiple changes - either all of the changes you've made on your DataContext are saved, or none of them are.

    If no transaction is already in scope, the LINQ to SQL DataContext object will automatically start a database transaction to guard updates when you call SubmitChanges(). Alternatively, LINQ to SQL also enables you to explicitly define and use your own TransactionScope object (a feature introduced in .NET 2.0).  This makes it easier to integrate LINQ to SQL code with existing data access code you already have.  It also means that you can enlist non-database resources into the transaction - for example: you could send off a MSMQ message, update the file-system (using the new transactional file-system support), etc - and scope all of these work items in the same transaction that you use to update your database with LINQ to SQL.

    Validation and Business Logic

    One of the important things developers need to think about when working with data is how to incorporate validation and business rule logic.  Thankfully LINQ to SQL supports a variety of ways for developers to cleanly integrate this with their data models. 

    LINQ to SQL enables you to add this validation logic once - and then have it be honored regardless of where/how the data model you've created is used.  This avoids you having to repeat logic in multiple places, and leads to a much more maintainable and clean data model. 

    Schema Validation Support

    When you define your data model classes using the LINQ to SQL designer in VS 2008, they will by default be annotated with some validation rules inferred from the schema of the tables in the database.

    The datatypes of the properties in the data model classes will match the datatypes of the database schema.  This means you will get compile errors if you attempt to assign a boolean to a decimal value, or if you attempt to implicitly convert numeric types incorrectly.

    If a column in the database is marked as being nullable, then the corresponding property in the data model class created by the LINQ to SQL designer will be a nullable type.  Columns not marked as nullable will automatically raise exceptions if you attempt to persist an instance with a null value.  LINQ to SQL will likewise ensure that identity/unique column values in the database are correctly honored.

    You can obviously use the LINQ to SQL designer to override these default schema driven validation settings if you want - but by default you get them automatically and don't have to take any additional steps to enable them.  LINQ to SQL also automatically handles escaping SQL values for you - so you don't need to worry about SQL injection attacks when using it.

    Custom Property Validation Support

    Schema driven datatype validation is useful as a first step, but usually isn't enough for real-world scenarios. 

    Consider for example a scenario with our Northwind database where we have a "Phone" property on the "Customer" class which is defined in the database as an nvarchar.  Developers using LINQ to SQL could write code like below to update it using a valid telephone number: 

    The challenge that we will run into with our application, however, is that the below code is also legal from a pure SQL schema perspective (because it is still a string even though it is not a valid phone number):

    To prevent bogus phone numbers from being added into our database, we can add a custom property validation rule to our Customer data model class.  Adding a rule to validate phone numbers using this feature is really easy.  All we need to-do is to add a new partial class to our project that defines the method below:

    The code above takes advantage of two characteristics of LINQ to SQL:

    1) All classes created by the LINQ to SQL designer are declared as "partial" classes - which means that developers can easily add additional methods, properties, and events to them (and have them live in separate files).  This makes it very easy to augment the data model classes and DataContext classes created by the LINQ to SQL designer with validation rules and additional custom helper methods that you define.  No configuration or code wire-up is required.

    2) LINQ to SQL exposes a number of custom extensibility points in its data model and DataContext classes that you can use to add validation logic before and after things take place.  Many of these extensibility points utilize a new language feature called "partial methods" that is being introduced with VB and C# in VS 2008 Beta2.  Wes Dyer from the C# team has a good explanation of how partial methods works in this blog post here.

    In my validation example above, I'm using the OnPhoneChanging partial method that is executed anytime someone programmatically sets the "Phone" property on a Customer object.  I can use this method to validate the input however I want (in this case I'm using a regular expression).  If everything passes successfully, I just return from the method and LINQ to SQL will assume that the value is valid.  If there are any issues with the value, I can raise an exception within the validation method - which will prevent the assignment from taking place.

    Custom Entity Object Validation Support

    Property level validation as used in the scenario above is very useful for validating individual properties on a data model class.  Sometimes, though, you want/need to validate multiple property values on an object against each other. 

    Consider for example a scenario with an Order object where you set both the "OrderDate" and the "RequiredDate" properties:

    The above code is legal from a pure SQL database perspective - even though it makes absolutely no sense for the required delivery date of the new order to be entered as yesterday. 

    The good news is that LINQ to SQL in Beta2 makes it easy for us to add custom entity level validation rules to guard against mistakes like this from happening.  We can add a partial class for our "Order" entity and implement the OnValidate() partial method that will be invoked prior to the entity's values being persisted into the database.  Within this validation method we can then access and validate all of the data model class properties:

    Within this validation method I can check any of the entity's property values (and even obtain read-only access to its associated objects), and raise an exception as needed if the values are incorrect.  Any exceptions raised from the OnValidate() method will abort any changes from being persisted in the database, and rollback all other changes in the transaction.

    Custom Entity Insert/Update/Delete Method Validation

    There are times when you want to add validation logic that is specific to insert, update or delete scenarios.  LINQ to SQL in Beta2 enables this by allowing you to add a partial class to extend your DataContext class and then implement partial methods to customize the Insert, Update and Delete logic for your data model entities.  These methods will be called automatically when you invoke SubmitChanges() on your DataContext.

    You can add appropriate validation logic within these methods - and if it passes then tell LINQ to SQL to continue with persisting the relevant changes to the database (by calling the DataContext's "ExecuteDynamicXYZ" method):

    What is nice about adding the above methods is that the appropriate ones are automatically invoked regardless of the scenario logic that caused the data objects to be created/updated/deleted.  For example, consider a simple scenario where we create a new Order and associate it with an existing Customer:

    When we call northwind.SubmitChanges() above, LINQ to SQL will determine that it needs to persist a new Order object, and our "InsertOrder" partial method will automatically be invoked. 

    Advanced: Looking at the Entire Change List for the Transaction

    There are times when adding validation logic can't be done purely by looking at individual insert/update/delete operations - and instead you want to be able to look at the entire change list of operations that are occurring for a transaction. 

    Starting with Beta2 of .NET 3.5, LINQ to SQL now enables you to get access to this change list by calling the public DataContext.GetChangeList() method.  This will return back a ChangeList object that exposes collections of each addition, removal and modification that has been made. 

    One approach you can optionally employ for advanced scenarios is to sub-class the DataContext class and override its SubmitChange() method.  You can then retrieve the ChangeList() for the update operation and perform any custom validation you want prior to executing it:

     

    The above scenario is a somewhat advanced one - but it is nice to know that you always have the ability to drop-down and take advantage of it if needed.

    Handling Simultaneous Changes with Optimistic Concurrency

    One of the things that developers need to think about in multi-user database systems is how to handle simultaneous updates of the same data in the database.  For example, assume two users retrieve a product object within an application, and one of the users changes the ReorderLevel to 0 while the other changes it to 1.  If both users then attempt to save the product back to the database, the developer needs to decide how to handle the change conflicts. 

    One approach is to just "let the last writer win" - which means that the first user's submitted value will be lost without the end-users realizing it.  This is usually considered a poor (and incorrect) application experience. 

    Another approach which LINQ to SQL supports is to use an optimistic concurrency model - where LINQ to SQL will automatically detect if the original values in the database have been updated by someone else prior to the new values being persisted.  LINQ to SQL can then provide a conflict list of changed values to the developer and enable them to either reconcile the differences or provide the end-user of the application with UI to indicate what they want to-do. 

    I'll cover how to use optimistic concurrency with LINQ to SQL in a future blog post.

    Using SPROCs or Custom SQL Logic for Insert/Update/Delete Scenarios

    One of the questions that developers (and especially DBAs) who are used to writing SPROCs with custom SQL usually ask when seeing LINQ to SQL for the first time is - "but how can I have complete control of the underlying SQL that is executed?" 

    The good news is that LINQ to SQL has a pretty flexible model that enables developers to override the dynamic SQL that is automatically executed by LINQ to SQL, and instead call custom insert, update, delete SPROCs that they (or a DBA) define themselves. 

    What is really nice is that you can start off by defining your data model and have LINQ to SQL automatically handle the insert, update, delete SQL logic for you.  You can then at a later point customize the data model to use your own custom SPROCs or SQL for updates - without having to change any of the application logic that is using your data model, nor would you have to change any of the validation or business rules logic supporting it (all of this stays the same).  This provides a lot of flexibility in how you build your application.

    I'll cover how to customize your data models to use SPROCs or custom SQL in a future blog post.

    Summary

    Hopefully the above post provides a good summary of how you can easily use LINQ to SQL to update your database, and cleanly integrate validation and business logic with your data models.  I think you'll find that LINQ to SQL can dramatically improve your productivity when working with data, and enable you to write extremely clean object-oriented data access code.

    In upcoming blog posts in this series I'll cover the new <asp:linqdatasource> control coming in .NET 3.5, and talk about how you can easily build data UI in ASP.NET that takes advantage of LINQ to SQL data models. I'll also cover some more specific LINQ to SQL programming concepts including optimistic concurrency, lazy and eager loading, table mapping inheritance, custom SQL/SPROC usage, and more.

    Hope this helps,

    Scott

  • New "Orcas" Language Feature: Extension Methods

    Last week I started the first in a series of blog posts I'll be making that cover some of the new VB and C# language features that are coming as part of the Visual Studio and .NET Framework "Orcas" release later this year. 

    My last blog post covered the new Automatic Properties, Object Initializer and Collection Initializer features.  If you haven't read my previous post yet, please read it here.  Today's blog post covers a much more significant new feature that is available with both VB and C#: Extension Methods.

    What are Extension Methods?

    Extension methods allow developers to add new methods to the public contract of an existing CLR type, without having to sub-class it or recompile the original type.  Extension Methods help blend the flexibility of "duck typing" support popular within dynamic languages today with the performance and compile-time validation of strongly-typed languages.

    Extension Methods enable a variety of useful scenarios, and help make possible the really powerful LINQ query framework that is being introduced with .NET as part of the "Orcas" release.

    Simple Extension Method Example:

    Ever wanted to check to see whether a string variable is a valid email address?  Today you'd probably implement this by calling a separate class (probably with a static method) to check to see whether the string is valid.  For example, something like:

    string email Request.QueryString["email"];

    if 
    ( EmailValidator.IsValid(email) ) {
       

    }

    Using the new "extension method" language feature in C# and VB, I can instead add a useful "IsValidEmailAddress()" method onto the string class itself, which returns whether the string instance is a valid string or not.  I can then re-write my code to be cleaner and more descriptive like so:

    string email Request.QueryString["email"];

    if 
    ( email.IsValidEmailAddress() ) {
       

    }

    How did we add this new IsValidEmailAddress() method to the existing string type?  We did it by defining a static class with a static method containing our "IsValidEmailAddress" extension method like below:

    public static class ScottGuExtensions
    {
        
    public static bool IsValidEmailAddress(this string s)
        {
            Regex regex 
    = new Regex(@"^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$");
            return 
    regex.IsMatch(s);
        
    }
    }

    Note how the static method above has a "this" keyword before the first parameter argument of type string.  This tells the compiler that this particular Extension Method should be added to objects of type "string".  Within the IsValidEmailAddress() method implementation I can then access all of the public properties/methods/events of the actual string instance that the method is being called on, and return true/false depending on whether it is a valid email or not.

    To add this specific Extension Method implementation to string instances within my code, I simply use a standard "using" statement to import the namespace containing the extension method implementation:

    using ScottGuExtensions;

    The compiler will then correctly resolve the IsValidEmailAddress() method on any string.  C# and VB in the public "Orcas" March CTP now provide full intellisense support for extension methods within the Visual Studio code-editor.  So when I hit the "." keyword on a string variable, my extension methods will now show up in the intellisense drop-downlist:

    The VB and C# compilers also naturally give you compile-time checking of all Extension Method usage - meaning you'll get a compile-time error if you mis-type or mis-use one.

    [Credit: Thanks to David Hayden for first coming up with the IsValidEmailAddress scenario I used above in a prior blog post of his from last year.]

    Extension Methods Scenarios Continued...

    Leveraging the new extension method feature to add methods to individual types opens up a number of useful extensibility scenarios for developers.  What makes Extension Methods really powerful, though, is their ability to be applied not just to individual types - but also to any parent base class or interface within the .NET Framework.  This enables developers to build a variety of rich, composable, framework extensions that can be used across the .NET Framework.

    For example, consider a scenario where I want an easy, descriptive, way to check whether an object is already included within a collection or array of objects.  I could define a simple .In(collection) extension method that I want to add to all objects within .NET to enable this.  I could implement this "In()" extension method within C# like so:

    Note above how I've declared the first parameter to the extension method to be "this object o".  This indicates that this extension method should applied to all types that derive from the base System.Object base type - which means I can now use it on every object in .NET. 

    The "In" method implementation above allows me to check to see whether a specific object is included within an IEnumerable sequence passed as an argument to the method.  Because all .NET collections and arrays implement the IEnumerable interface, I now have a useful and descriptive method for checking whether any .NET object belongs to any .NET collection or array.

    I could use then use this "In()" extension method to see whether a particular string is within an array of strings:

    I could use it to check to see whether a particular ASP.NET control is within a container control collection:

    I could even use it with scalar datatypes like integers:

    Note above how you can even use extension methods on base datatype values (like the integer value 42).  Because the CLR supports automatic boxing/unboxing of value-classes, extensions methods can be applied on numeric and other scalar datatypes directly. 

    As you can probably begin to see from the samples above, extension methods enable some really rich and descriptive extensibility scenarios.  When applied against common base classes and interfaces across .NET, they enable some really nice domain specific framework and composition scenarios. 

    Built-in System.Linq Extension Methods

    One of the built-in extension method libraries that we are shipping within .NET in the "Orcas" timeframe are a set of very powerful query extension method implementations that enable developers to easily query data.  These extension method implementations live under the new "System.Linq" namespace, and define standard query operator extension methods that can be used by any .NET developer to easily query XML, Relational Databases, .NET Objects, and/or any other type of data structure. 

    A few of the advantages of using the extension method extensibility model for this query support include:

    1) It enables a common query programming model and syntax that can be used across all types of data (databases, XML files, in-memory objects, web-services, etc).

    2) It is composable and allows developers to easily add new methods/operators into the query syntax. For example: we could use our custom "In()" method together with the standard "Where()" method defined by LINQ as part of a single query.  Our custom In() method will look just as natural as the "standard" methods supplied under the System.Linq namespace. 

    3) It is extensible and allows any type of data provider to be used with it.  For example: an existing ORM engine like NHibernate or LLBLGen could implement the LINQ standard query operators to enable LINQ queries against their existing ORM implementation and mapping engines.  This will enable developers to learn a common way to query data, and then apply the same skills against a wide variety of rich data store implementations.

    I'll be walking through LINQ much more over the next few weeks, but wanted to leave you with a few samples that show how to use a few of the built-in LINQ query extension methods with different types of data:

    Scenario 1: Using LINQ Extension Methods Against In-Memory .NET Objects

    Assume we have defined a class to represent a "Person" like so:

    I could then use the new object Initializer and collection Initializer features to create and populate a collection of "people" like so:

    I could then use the standard "Where()" extension method provided by System.Linq to retrieve a sequence of those "Person" objects within this collection whose FirstName starts with the letter "S" like so:

    The new p => syntax above is an example of a "Lambda expression", which is a more concise evolution of C# 2.0's anonymous method support, and enables us to easily express a query filter with an argument (in this case we are indicating that we only want to return a sequence of those Person objects where the firstname property starts with the letter "S").  The above query will then return 2 objects as part of the sequence (for Scott and Susanne).

     I could also write code that takes advantage of the new "Average" and "Max" extension methods provided by System.Linq to determine the average age of the people in my collection, as well as the age of the oldest person like so:

    Scenario 2: Using LINQ Extension Methods Against an XML File

    It is probably rare that you manually create a collection of hard-coded data in-memory.  More likely you'll retrieve the data either from an XML file, a database, or a web-service.

    Let's assume we have an XML file on disk that contains the data below:

    I could obviously use the existing System.Xml APIs today to either load this XML file into a DOM and access it, or use a low-level XmlReader API to manually parse it myself.  Alternatively, with "Orcas" I can now use the System.Xml.Linq implementation that supports the standard LINQ extension methods (aka "XLINQ") to more elegantly parse and process the XML. 

    The below code-sample shows how to use LINQ to retrieve all of the <person> XML Elements that have a <person> sub-node whose inner value starts with the letter "S":

    Note that it uses the exact same Where() extension method as with the in-memory object sample.  Right now it is returning a sequence of "XElement" elements, which is an un-typed XML node element.  I could alternatively re-write the query to "shape" the data that is returned instead by using LINQ's Select() extension method and provide a Lambda expression that uses the new object initializer syntax to populate the same "Person" class that we used with our first in-memory collection example:

    The above code does all the work necessary to open, parse and filter the XML in the "test.xml" file, and return back a strongly-typed sequence of Person objects.  No mapping or persistence file is necessary to map the values - instead I am expressing the shaping from XML->objects directly within the LINQ query above.

    I could also use the same Average() and Max() LINQ extension methods as before to calculate the average age of <person> elements within the XML file, as well as the maximum age like so:

    I do not have to manually parse the XML file.  Not only will XLINQ handle that for me, but it will parse the file using a low-level XMLReader and not have to create a DOM in order to evaluate the LINQ expression.  This means that it is lightening fast and doesn't allocate much memory.

    Scenario 3: Using LINQ Extension Methods Against a Database

    Let's assume we have a SQL database that contains a table called "People" that has the following database schema:

    I could use the new LINQ to SQL WYSIWYG ORM designer within Visual Studio to quickly create a "Person" class that maps to the database:

    I can then use the same LINQ Where() extension method I used previously with objects and XML to retrieve a sequence of strongly-typed "Person" objects from the database whose first name starts with the letter "S":

    Note how the query syntax is the same as with objects and XML. 

    I could then use the same LINQ Average() and Max() extension methods as before to retrieve the average and maximum age values from the database like so:

    You don't need to write any SQL code yourself to have the above code snippets work.  The LINQ to SQL object relational mapper provided with "Orcas" will handle retrieving, tracking and updating objects that map to your database schema and/or SPROCs.  You can simply use any LINQ extension method to filter and shape the results, and LINQ to SQL will execute the SQL code necessary to retrieve the data (note: the Average and Max extension methods above obviously don't return all the rows from the table - they instead use TSQL aggregate functions to compute the values in the database and just return a scalar result).

    Please watch this video I did in January to see how LINQ to SQL dramatically improves data productivity in "Orcas".  In the video you can also see the new LINQ to SQL WYSIWYG ORM designer in action, as well as see full intellisense provided in the code-editor when writing LINQ code against the data model.

    Summary

    Hopefully the above post gives you a basic understanding of how extension methods work, and some of the cool extensibility approaches you will be able to take with them.  As with any extensibility mechanism, I'd really caution about not going overboard creating new extension methods to begin with.  Just because you have a shiny new hammer doesn't mean that everything in the world has suddenly become a nail!  

    To get started trying out extension methods, I'd recommend first exploring the standard query operators provided within the System.Linq namespace in "Orcas".  These enable rich query support against any array, collection, XML stream, or relational database, and can dramatically improve your productivity when working with data.  I think you'll find they'll significantly reduce the amount of code you write within your applications, and allow you to write really clean and descriptive syntax.  They'll also enable you to get automatic intellisense and compile-time checking of query logic within your code.

    In the next few weeks I'll continue this series on new language features in "Orcas" and explore Anonymous Types and Type Inference, as well as talk more about Lambdas and other cool features.  I'll also obviously be talking a lot more about LINQ.

    Hope this helps,

    Scott

  • Free SQL Server Training Videos (and other good data tutorial pointers)

    Today on the www.asp.net site we posted a great new "How Do I?" video series focused on SQL Server 2005 Express (which you can download and use completely for free). 

    The series features 13 professional videos that start at the very beginning and then explore designing tables/schemas, using data types, creating SPROCs, using reporting services and the business intelligence engine, enabling full text search across your data, and more (I'm guessing even most devs with a lot of data experience might learn some new things from the reporting services and full text search videos):

    You can download or watch the the SQL Training videos online for free here.

    In case you missed my posts on uploading a SQL Express or SQL Server Database into a remote hosting environment, I also highly recommend going back and reading these two posts:

    Earlier this summer I also posted a popular blog post topic that provides 100s of sample database schemas you can download and use within your applications:

    And finally (and perhaps most importantly), make sure to check out Scott Mitchell's awesome ASP.NET Data Tutorials here (there are more than 50 fantastic tutorials all in VB and C# for common data scenarios in ASP.NET):

    Hope this helps,

    Scott

  • Tip/Trick: How to upload a .SQL file to a Hoster and Execute it to Deploy a SQL Database

    Last month I posted about the new (free) Database Publishing Wizard that is designed to make it much, much easier to upload and deploy SQL Express and SQL Server databases in a web hoster environment.

    In my first Database Publishing post, I walked through how you can use the Database Publishing Wizard to automatically generate a .SQL installation file that contains the script necessary to recreate your database schema (tables, views, sprocs, triggers, full-text catalogs, etc) and also populate your database with the same table row contents as your original database.  This provides a super easy way to put together a .SQL script that entirely automates replicating your database on another server:

    In my previous post I mentioned how hosters often provide an online SQL management tool that you can then use to open and run your newly generate .SQL file, or provide a SQL query tool that allows you to copy/paste the .SQL file's contents into a query execution window to run.  Unfortunately, though, not all hosters support tools like this.  And even with hosters that do support it you might run into cases where your generated .SQL file is so big that copying/pasting it into a text box isn't really viable (doing a form post of 200MB of content will typically time out!). 

    This blog post walks through an alternative way to deploy your .SQL files in a remote hosting environment, and which does not require your hoster to configure or install anything on the backend.

    Steps to Deploy a SQL Database to a Remote SQL Server without using an Admin Tool

    Below are the steps to take to deploy a local database to a remote hoster without requiring or using any admin tool:

    Step 1: Generate a .SQL File Containing your Schema and Data

    The first step to remotely deploy your database in a hosted environment is to generate a .SQL file that dumps the schema and content of your database.  Follow the steps in my first Database Publishing blog post to learn exactly how to generate a .SQL file for either a SQL Express or SQL Server database.

    Step 2: FTP up the .SQL File to your Remote Hoster

    Once you've generated your .SQL file, upload it to your remote hoster using FTP or some other file transfer mechanism.  Ideally you should copy this into a protected location where remote users can't easily get access to it (one suggestion: give it a random name and upload it into the /app_data folder which is typically protected by default).

    The benefit of uploading this file over FTP is that it won't force you to limit the size of the .SQL file.  It can be 100s of megabytes in size if necessary.

    Step 3: Download the RunSQL.aspx Utility Page

    Visit this page and download the RunSQL.aspx file linked on it. 

    The RunSQL.aspx file is an ASP.NET page that the SQL Server team put together that supports two arguments: 1) The name of a .SQL file, and 2) The connection string of a database.  When run the RunSQL.aspx page will open the .SQL file and iterate over each of its statements and execute them against the database (indicated via the connection string).  This will provision the database defined within the .SQL file to the remote database target.

    Step 4: Edit the RunSQL.aspx Utility Page

    Open and locally edit the RunSQL.aspx file and and configure the name of your .SQL file, as well as provide the connection-string your hoster gave you for the SQL database:

    Replace the <<YOUR_SCRIPTFILE>> marker and associated connection-string markers with the correct values for your hosted configuration.  Note that unless you know the fully qualified path of the .SQL file, you'll probably want to use ASP.NET's Server.MapPath(fileName) method to calculate the absolute path of the relative .SQL file path in your web application.  For example:

        // Filename of the T-SQL file you want to run
        
    string fileName Server.MapPath("personal.SQL");    
        
        
    // Connection string to the server you want to execute against
        
    string connectionString @"Server=server123;User ID=user123;Password=password123;Initial Catalog=MyDBName123";  
        
        
    // Timeout of batches (in seconds)
        
    int timeout 600;

    Step 5: Upload the RunSQL.aspx Utility Page to Your Remote Hoster

    Once you have finished updating the fileName and connectionString values above, upload the RunSQL.aspx file to your remote hoster (for example: using FTP). 

    For security reasons I recommend giving the file a random file-name when you upload it so that other users can't easily find and run it.

    Step 6: Hit the RunSQL.aspx Utility Page with a Browser

    Once uploaded, hit the remote RunSQL.aspx page via your browser.  This will cause the page on your remote server to parse the .SQL file, and execute all of its statements.  Since the .SQL file contains all of the database schema creation and population statements necessary to recreate your database, once the page finishes running you'll have an identical database deployed at your hoster:

    Step 7: Delete the RunSQL.aspx and .SQL Files

    Once you've finished running your .SQL script, delete both the RunSQL.aspx page and the .SQL file from your remote hoster server. 

    For security reasons you do not want anyone else to be able to remotely hit the RunSQL.aspx page (since it might recreate your database causing you to lose data).

    Step 9: Update the Web.Config file of your application to point to the hoster database

    The only final step remaining then is to update your web.config file's <connectionStrings> section to also point at your remote hoster's database connectionstring value.  Once you do this your app should work fine remotely.

    Hope this helps,

    Scott

    P.S. For more of my ASP.NET Tips, Tricks and Recipes, please visit this page.