BusinessRx Community

Dedicated to the advancement of software, technology and the people who devote their lives to it.

Welcome to BusinessRx Community Sign in | Join | Help
in Search

BusinessRx Reading List

These blog entries are written by industry experts and leaders. We consider this content to be a good read for any software developer or web technologist.

January 2008 - Posts

  • Versioning Databases – The Baseline

    Continuing from the last post (Three Rules for Database Work), I wanted to drill into some database versioning strategies that have worked well for me.

    Caveats and Considerations

    As a preface, let me say there are many different strategies that can work. I'm not presenting the one true way. My goal is to roll out database changes in a consistent, testable, reproducible manner. I'm used to working with largish databases that are packaged to install behind a customer's firewall. I don't want to use naïve update scripts that run for 6 hours. Concerns like this bias my thinking and strategies. Every application should manage database change, but you'll have to decide on the simplest strategy that works for your environment.

    The Baseline

    The first step in versioning a database is to generate a baseline schema. This is the starting point for versioning a database. After you've published the baseline to an authoritative source, any changes to the schema require a schema change script (a topic for the next post, because I have a feeling this is going to be long for a blog post).

    You are probably not going to baseline the database on day 1 of a project. I'd suggest letting the early design of the schema settle in a bit so you are not creating a huge number of change scripts. This might sound like I'm suggesting you need to do some big, up front schema design – but that's not true. You can get pretty far into an application these days with in-memory data, fakes, stubs, mocks, and unit tests. Once the model in your code starts to stabilize, you can start thinking of the schema required to persist all the data. If you are ORMing, you can even generate the first schema from your model.

    On the other hand, maybe your project and database have already been around a couple years. That's ok – you can baseline today (tomorrow at the latest), and manage change moving forward.

    But How?

    If you want to do things the hard way, then open a new file in a text editor and write all the SQL commands that will create every table, constraint, function, view, index, and every other object in your database. You'll also want to include commands that populate lookup tables with static data and include any bootstrap data needed by the application. Test the new script against a fresh database server and if successful, commit the file to source control. Consider your schema baselined!

    Nobody really does this step the hard way, though. Most of us use tools that we point to a database, and the tools generate one or more scripts for us. Some people like to generate everything into one big script file. Others like to generate one script file for every database object. SQL Server Management Studio provides both option when you elect to script a database. I've seen both approaches work, but the "one file per object" approach feels cumbersome on a day to day basis, and unwieldy if the number of objects grow into the thousands.

    Personally, I like to take a hybrid approach. I like to keep all of the SQL needed to create tables, constraints, defaults, and primary indexes in a single file. Any views, stored procedures, and functions are scripted one per file.

    If you go the multiple file approach, make sure to write a batch file, shell script, application, or some other form of automation that can automatically locate and run all of the script files required to install the database. Human intervention in this process is a step backwards.

    Also, many tools like to include CREATE DATABASE commands and/or include database names in the scripts they generate. You'll want to purge any reference to a hardcoded database name. You want the name to be configurable (a default name is fine), and you probably want to support multiple databases for your application on the same database instance (mostly for testing).

    Whichever approach you take (one single file, or multiple files), you now have scripts that can recreate the database schema on any developer machine, test machine, or production machine. Everyone uses the exact same schema. Congratulations! You've just increased the quality of your software because the database can be reliably reproduced in any environment.

    I Almost Forgot the Most Important Ingredient

    At some point in the future, the schema will have to change. Before you baseline the database you need to add a table to record these schema changes. The following table is the kind of table I'd use to track every change to a database.

    CREATE TABLE [dbo].[SchemaChanges](
       [ID] [int]
    IDENTITY(1,1) NOT NULL,
       [MajorReleaseNumber] [varchar](2)
    NOT NULL,
       [MinorReleaseNumber] [varchar](2)
    NOT NULL,
       [PointReleaseNumber] [varchar](4)
    NOT NULL,
       [ScriptName] [varchar](50)
    NOT NULL,
       [DateApplied] [datetime]
    NOT NULL,

        
    CONSTRAINT [PK_SchemaChangeLog]
            
    PRIMARY KEY CLUSTERED ([SchemaChangeID] ASC)
    )

    The first baseline schema script should, as the last step, officially install version 1.0 of the database:

    INSERT INTO [SchemaChangeLog]
           ([MajorReleaseNumber]
           ,[MinorReleaseNumber]
           ,[PointReleaseNumber]
           ,[ScriptName]
           ,[DateApplied])
    VALUES
           ('01'
           ,'00'
           ,'0000'
           ,'initial install'
           ,GETDATE())
  • Three Rules for Database Work

    Some developers love working with relational databases, and other developers can't stand to touch them. Either way - if your application uses a database, you have to treat the database with some respect. The database is as much a part of an application as the code and the models inside the software.

    Here are three rules I've learned to live by over the years of working with relational databases.

    1. Never use a shared database server for development work.

    Fossil!The convenience of a shared database is tempting. All developers point their workstations to a single database server where they can test and make schema changes. The shared server functions as an authoritative source for the database schema, and schema changes appear immediately to all team members. The shared database also serves as a central repository for test data.

    Like many conveniences in software development, a shared database is a tar pit waiting to fossilize a project. Developers overwrite each other's changes. The changes I make on the server break the code on your development machine. Remote development is slow and difficult.

    Avoid using a shared database at all costs, as they ultimately waste time and help produce bugs.

    2. Always Have a Single, Authoritative Source For Your Schema

    Ideally, this single source will be your source control repository (see rule #3). Consider the following conversation:

    Developer 1: It's time to push the app into testing. Do we copy the database from Jack's machine, or Jill's machine?

    Developer 2: Ummmmmmmm, I don't remember which one is up to date.

    Developer 1: We're screwed.

    Everyone should know where the official schema resides, and have a frictionless experience in getting a fresh database setup. I should be able to walk up to a computer, get the latest from source control, build, and run a simple tool to setup the database (in many scenarios, the build process can even setup a database if none exists, so the process is one step shorter).

    How you put your database into source control depends on your situation and preferences. Any decent O/R mapping tool should be able to create a database given the mappings you've defined in a project. You can also script out the database as a set of one or more files full of SQL DDL commands. I generally prefer to keep database views and programmatic features (including functions, triggers, and stored procedures) as separate files - but more on this in a later post.

    There are plenty of tools to help. Leon Bambrick has a long list (albeit one year old list) of tools and articles that can help, while Jeff Atwood gushes over the virtues of Visual Studio for Database Professionals.

    3. Always Version Your Database

    There are many ways to version databases, but the common goal is to propagate changes from development, to test, and ultimately to production in a controlled and consistent manner. A second goal is to have the ability to recreate a database at any point in time. This second goal is particularly important if you are shipping software to clients. If someone finds a bug in build 20070612.1 of your application, you must be able to recreate the application as it appeared in that build - database and all.

    In a future post, I'll describe an approach I've used for database versioning that has worked well for many years of commercial development.

    In the meantime, if you are looking for more database rule, then Adam Cogan and SSW maintain an excellent list.

  • VS 2008 Web Deployment Project Support Released

    This past Friday we released the final RTW (release to web) support for VS 2008 Web Deployment projects.  You can learn more about it and download it for free here

    Web Deployment projects can be used with either the "ASP.NET Web Site" or "ASP.NET Web Application Project" options built-into VS 2008, and provide a few additional build, packaging and deployment options for you to use.  You can read an old tutorial post of mine here to learn more about they work.

    The VS 2008 Web Deployment Project version supports all of the existing features provided by the VS 2005 web deployment download.  It also adds additional support for:

    • Easily migrating VS 2005 Web Deployment Projects to VS 2008 Web Deployment Projects
    • Replacing output only if web deployment builds succeed
    • IIS7 Support

    This RTW (release to web) version fixes bugs and adds some small features that people requested in the December CTP version we released last month.  Please make sure to run setup and uninstall any older version of the VS 2008 Web Deployment Project support you have installed before installing this final version.  VS 2008 Web Deployment Projects can be installed side-by-side with VS 2005 Web Deployment Projects - so there is no need to uninstall the VS 2005 version if you are still using it with older projects.

    More Deployment Features

    In my last link-listing post I pointed at the new Web Deployment Tool being released by the IIS team.  This tool works with both IIS6 and IIS7 and enables automated copy deployment, file synchronization, and migrating of applications onto web servers.  If you are looking for a great way to automate the deployment of your ASP.NET applications onto remote servers then this tool is definitely one to check out. 

    You can use VS 2008 Web Deployment Projects as a post-build step within your build environment to fix up last minute deployment settings - and then use the IIS Web Deployment tool to copy them remotely onto server machines.  Alternatively you can also use the IIS Web Deployment Tool to copy vanilla "ASP.NET Web Site" or "ASP.NET Web Application" projects to remote machines (no VS 2008 Web Deployment Project required).

    To learn more about the new IIS Web Deployment tool, read the walkthroughs at the bottom of this page (in particular the "Introduction to MS Deploy" one).  I will also be doing a blog post in the future that talks more about how to use it to automate your web server deployments.

    Hope this helps,

    Scott

  • Zune Update Depends On Windows Firewall Service

    I always feel a twinge of trepidation when I make a non-standard configuration change to one of my machines. For instance, when I turned off the Windows Firewall service on my desktop a few months ago, a voice in my head told me I'd one day be installing software that assumes the firewall service is running, and that software would fail with a mysterious error.

    Turns out that software is the Zune updater application.

    It took some trial and error (three failed update attempts with error code 0x80070643), but once I turned on Windows Firewall, my Zune update finally succeeded. I'd like to say I did a bunch of fancy sleuthing to track down the problem - but it was just blind luck and a hunch.

    The voice in my head now tells me I should leave the service running, even though I have the firewall turned off, because there is a lot more software just waiting to spit out cryptic error codes in the future...
  • Jan 24th Links: ASP.NET, ASP.NET AJAX, Visual Studio, .NET, IIS

    I just arrived back from my trip from Asia, and decided to celebrate (since I'm jet-lagged and can't sleep) with a new post in my link-listing series.  You can check out my ASP.NET Tips, Tricks and Tutorials page for links to popular articles I've done myself in the past.

    ASP.NET

    ASP.NET AJAX

    Visual Studio

    • Using Ctrl-Break to Stop VS Building: Steven Harman points out a cool tip/trick, which is that you can use the Ctrl-Break key within Visual Studio to kill the current compilation build.  A useful tip if you've accidentally kicked off a long build or get tired waiting for it to finish.

    .NET

    • Marshaling between Managed and Unmanaged Code: Yi Zhang and Xiaoying Guo from my team in Shanghai have written a great MSDN article that describes how to use the marshaling interop features of the CLR to call native code.  One of the tools they highlight is an awesome P/Invoke Interop Assistant application they built that makes it much, much easier to generate p/invoke interop signatures when calling native methods.  A must-have tool for anyone doing native/managed interop!

    • .NET Framework 3.5 Poster: Brad Abrams posts about the cool new .NET Framework 3.5 posters now available for download (now in multiple file formats).

    IIS

    • Microsoft Web Deployment Tool Technical Preview 1: Yesterday the IIS team posted the first preview of a new Microsoft Web Deployment tool.  This tool works with both IIS6 and IIS7 and enables automated deployment, synchronization, and migrating of applications on web servers.  If you are looking for a great way to automate the deployment of your ASP.NET applications then this tool is definitely one to check out.  To learn more, read the walkthroughs at the bottom of this page (in particular the "Introduction to MS Deploy" one).  This tool is awesome and should make automated deployment much easier.

    Hope this helps,

    Scott

  • Customer Expectations

    CFL light

    Recently, I received a package from my local power company - Alleghany Energy. The package contained two 13-watt carbon fluorescent lamps (CFLs). In the Unites States, various organizations are trying to encourage people to replace their incandescent light bulbs with CFLs to save energy. In fact, recent U.S. legislation will effectively prevent the sale of incandescent bulbs after the year 2014 (unless the bulbs somehow become a lot more efficient by then).

    Since the package didn't come with an invoice inside, I assumed my power company was giving away free bulbs. After all, Home Depot gave away 1 million bulbs during last year's Earth Day, so why shouldn't my utility company give away a few, too?

    I was wrong.

    It was quite a shock to Alleghany's customers when we found out we were being charged almost $12 for these two bulbs as part of the "Maryland Compact Fluorescent Light Energy Efficiency Program".

    Outrage ensued. The outrage wasn't so much over the cost of the bulbs as it was about unmet expectations. We expected free bulbs because we weren't told about the cost. Alleghany Energy was either being sneaky, or was incompetent at communicating with their customers. If the power company was upfront about their plans, the story might have a different ending. Instead, there were rumblings about class action lawsuits and legal charges for unsolicited merchandise.

    In the end, the power company decided to refund all charges and eat the 2.5 million dollars it cost to send light bulbs to 220,000 customers.

    Moral of the story – communicate with the customer. This is why contemporary software development processes like Agile and XP have such a focus on customer communication. Unmet expectations make for angry customers and bad business.

  • Tafiti and the Pain of Silverlight 1.0

    Tafiti

    Tafiti is a search visualization application built with Silverlight 1.0 and the Windows Live APIs. The Tafiti code is hosted on CodePlex under the Microsoft Public License (MS-PL), as announced by Marc Mercuri last month.

    What is amazing about Tafiti is the number of controls written in 3700 lines of JavaScript. Tafiti includes custom Button, Scrollbar, Hyperlink, TextEdit, StackPanel, FlowPanel, and Carousel controls. You can find the code inside the js folder of the Tafiti web project, inside a file by the name of controls.js.

    Tafiti highlights some of the pain inherent in Silverlight 1.0 development, too. Most notably the pain felt from a total absence of any infrastructure needed to build these reusable controls. Tafiti provides this infrastructure for inself. For example, a generate UniqueNames method:

    // Rewrite XAML to give the elements unique names.
    // This is necessary because all names have global scope in WPF/E.
    // Each %name% in the xaml string is replaced by %uniquename% and
    // names["name"] is set to "uniquename".

    Tafiti.generateUniqueNames = function (xaml, names) {
        xaml = xaml.replace(/%(\w+)%/g,
                            
    function (match, name) {
                                
    if (!names[name]) {
                                    names[name] =
    "Tafiti_" + Tafiti_uniqueID++;
                                }
                                
    return names[name];
                            });
        
    return xaml;
    }

    If Silverlight 1.0 did not have such a short self-life, it would be worthwhile to genericze the Tafiti controls into a JavaScript library, but when Silverlight 2.0 hits the world this year I wouldn't be surprised to see those 3700 lines of code replaced with 370 lines of code. Silverlight 2.0 will take all this pain away...

  • ASP.NET MVC and User Group Fun

    I did a couple user group presentations this month on the new ASP.NET MVC framework. This post is a follow-up with all the links I promised to publish.

    My first presentation of the year went to the Central Maryland Association of .NET Professionals in Columbia, MD. The turnout at the meeting was tremendous – I'd say well over 100 people were in attendance. It didn't hurt that G Andrew Duthie was giving away free copies of Visual Studio 2008 as part of the Microsoft Installfest, and the catered food was a lot better than the typical pizza-in-cardboard user group fare.

    Best of all, I walked out of the meeting with a new 8 GB Zune 2 (thanks to a winning raffle ticket). Many people tried to tell me that it's not fair for the speaker to win the grand prize raffle, but I pretended not to hear them and then slipped out quietly before the mob turned ugly.

    My second presentation went to the BaltoMSDN group in Hunt Valley, MD. Imagine going to a wine tasting and having a technical presentation break out. That's what BaltoMSDN is like, because they have their meetings in a private room at the Greystone Grill. Bacon wrapped sea scallops and Belgian beer on draft go well with code, I must say.

    The Links

  • Adding Up and Down Arrow Images to a Sortable GridView

    Prior to ASP.NET 2.0, I wrote an article on 4Guys showing how to add up and down arrow images to a sortable DataGrid's column headers to visually depict the column and direction the grid was sorted by (read the article; see a live demo). In particular, my approach required the page developer to add a bit of code to the ASP.NET page's code-behind class that would enumerate the grid's columns and remove the up/down arrow <img> tag for columns that were not currently being sorted, and would add the appropriate <img> tag to the column the data was sorted by.

    I recently updated this functionality for the GridView control, but rather than require that the page developer add code to the ASP.NET page, I instead created a custom server control that extended the GridView. This control automatically adds the up/down arrow image when the grid is sorted.

    To check out this code and a demo application, and to learn more about the inner workings of this custom server control, be sure to read my latest 4Guys article: Extending the GridView to Include Sort Arrows.

  • Pricing the Ultimate Developer Rig 7 Months Later

    It was about 7 months ago when Scott Hanselman and CodingHorror built the Ultimate Developer Rig.

    Let's see what the saving are for late adopters...

    Part

    Then

    Now

    Antec P182 Gun Metal Black 0.8mm cold rolled steel ATX Mid Tower Computer Case - Retail

    $154.99

    $139.99

    MSI P6N SLI Platinum LGA 775 NVIDIA nForce 650i SLI ATX Intel Motherboard - Retail

    $144.99

    $119.99*

    2 x MSI NX8600GTS-T2D256E-OC GeForce 8600GTS 256MB GDDR3 PCI Express x16 Over Clock Edition HDCP Video Card - Retail

    $337.98

    $239.98

    Western Digital Raptor WD1500ADFD 150GB 10,000 RPM Serial ATA150 Hard Drive - OEM

    $199.99

    $169.99

    Seagate Barracuda 7200.10 ST3500630AS 500GB 7200 RPM SATA 3.0Gb/s Hard Drive - OEM

    $120

    $120.00**

    Intel Core 2 Quad Q6600 Kentsfield 2.4GHz 2 x 4MB L2 Cache LGA 775 Quad-Core Processor - Retail

    $531.90

    $279.99

    Scythe SCMN-1100 100mm Sleeve CPU Cooler - Retail

    $32.99

    $32.99

    CORSAIR CMPSU-520HX ATX12V v2.2 and EPS12V 2.91 520W Power Supply - Retail

    $129.99

    $124.99

    Kingston ValueRAM 2GB (2 x 1GB) 240-Pin DDR2 SDRAM DDR2 800 (PC2 6400) Dual Channel Kit Desktop Memory Model KVR800D2N5K2/2G - OEM

    $216.98

    $89.98

    LITE-ON 20X DVD±R DVD Burner with 12X DVD-RAM write and LightScribe Technology Black E-IDE/ATAPI Model LH-20A1H-185 - OEM

    $33.99

    $30.99

    Totals

    $1903.80

    $1348.99

    There is enough headroom now to pick up an extra 2GB of RAM and still put out less than $1500. I call this configuration the Ultimate Developer Rig +2***.

      * Price includes mail-in rebate
     ** Drive is now deactivate, but comparable 500GB drives are still ~$120.00
    *** Not to be confused with the Ultimate Rod of Dexterity +2 by you D&D nerds.

  • A Few Interesting Windows Workflow Links

    Igor emailed me for my thoughts about his SMTP server built with Windows Workflow. A link for the source code is at the bottom of his post. As Igor points out, WF is well suited to solve more than just business problems, and I think his project is an excellent and practical demonstration of building state machines in WF.

    Matt Winkler is looking for feedback on refactoring workflows. Personally, I'd like to see some additional renaming support at the property and activity level that can auto-magically patch up declarative activity bindings.

    Many moons ago I wrote a post on unit testing custom activities in WF. I've stuck to this approach over the last year, and I'm still looking for something better. A couple new posts on the topic have come up recently. YooT – Unit Testing custom Workflow Activities. Damir Dobric – Unit Testing: How to create activity type from XOML. Andreas Erben's – Unit Testing: Simple approach to Activity testing and improving the XOML loader.

    Finally, Michael Scherotter has built a custom activity for Expression Encoder, and has a webcast to prove it. All we need now is a direct link to the source code, Michael...

  • My Working with Data Tutorials in Video Format

    One of the most frequent requests I receive regarding my Working with Data in ASP.NET tutorial series is to have a video-based version. Some people learn best by reading, others by watching. Unfortunately, I much prefer writing to narrating and producing videos. I created a 'test video' using Camtasia back in December 2006, and wasn't very pleased with my results: I talk to fast and I need to practice on lowering my voice about two octaves. So don't expect to see much in the way of video output from me.

    The good news is that Ravi Nangunoori enjoys creating educational videos and evangelizing ASP.NET, and has taken the time to create videos for the first 32 of my 75 data tutorials (with plans to complete videos for all 75). The videos are free (you do, however, have to register on the site) and available at www.DotNetVideos.NET.

    Thanks for your tireless work on creating these videos, Ravi!

  • The Trivial Things You Can Learn When Stress Testing an AJAX Application

    Internet Explorer has no problem creating an array of 65,536 elements.

    var array = [];
    for(var i = 0; i < 65535; i++)
    {
        array.push(i);
    }    
    array.push(i);

    alert(array.length);
    // says 65536

    However, IE does have a problem eval-ing an array of 65,536 elements.

    var buffer = '[0';
    for(var i = 0; i < 65535 / 15; i++)
    {
        buffer +=
    ',0,0,0,0,0,0,0,0,0,0,0,0,0,0,0';
    }
    buffer +=
    ']';

    var array = eval(buffer);    
    alert(array.length);
    // would be 65536,
                         // if it got this far..

    The code produces an out of memory error on the eval statement. The limit for eval appears to be 65,535 – a number that harkens back to the dark, dismal days of programming in a segmented memory architecture.

    Of course, if you came here looking for the upper limits of evalyou might be doing something wrong!

  • New Years Goals

    I know I am a little late, but I decided to lay out some New Years goals (goals not resolutions) for everyone to see.

    The obligatory weight loss, get into shape goal.
    I lost a ton of weight a couple years ago, but some of it has been slowly coming back. My goal is to drop 30 lbs and go biking, play tennis, or run three times a week. I am also doing the pushup/crunch challenge.

    Minimize and Simplify
    This has been all the rage lately, and for good reason. My goal this year is to simplify and minimize as much as I can, both in software development and in regular life. In software development that means focusing on getting projects done and getting results in the easiest and simplest way possible. In regular life that means getting rid of clutter and crap I don't need anymore and getting a little better organized.

    Stop Consulting
    I really want to completely stop consulting this year, I am working on saving money so I can quit and fully dedicate myself to a software idea. I believe I can get a working prototype with my money, and hopefully get some small investment to build it past there.

    Makes 22books a success
    22books started out a way to learn Rails, but I really think I can get it to the point where it helps replace some of the income from consulting. My immediate goal with the site is to get it into the top 100,000 with Alexa. I think that's a reasonable goal to start with.

    Grow The Lounge
    I am also counting on The Lounge to replace some of the income from consulting, but to do that I need to expand past just .NET advertising. I also plan on building The Lounge infrastructure to include some interesting new features and advertiser perks. My goals for this year is to start at least 2 more rooms and to get the .NET room to average 80% full.

    Re-Launch Windows Developer Power Tools
    The site for the book has been kind of languishing since we published it, but I have been paying someone to do some work to really make it more useful. My goal is to get this site re-launched in the next two months.

    Reset this blog
    In the next couple of weeks I am going to move this blog over to infozerk.com and convert it to Graffiti, when I do that I also want to really re-define it. Over the past year or so I have been in kind of auto-pilot, my goal is to write a high quality meaningful post at least once a week. I am also going to stop posting links or quick posts in an effort to increase the quality level. I am going to post links, pictures, music, and stuff like that over on my tumblr blog.

    I am going to have a busy year, especially considering that my wife is pregnant and due in June. :)

    -James

  • .NET Framework Library Source Code now available

    Last October I blogged about our plan to release the source code to the .NET Framework libraries, and enable debugging support of them with Visual Studio 2008.  Today I'm happy to announce that this is now available for everyone to use. Specifically, you can now browse and debug the source code for the following .NET Framework libraries:

    • .NET Base Class Libraries (including System, System.CodeDom, System.Collections, System.ComponentModel, System.Diagnostics, System.Drawing, System.Globalization, System.IO, System.Net, System.Reflection, System.Runtime, System.Security, System.Text, System.Threading, etc).
    • ASP.NET (System.Web, System.Web.Extensions)
    • Windows Forms (System.Windows.Forms)
    • Windows Presentation Foundation (System.Windows)
    • ADO.NET and XML (System.Data and System.Xml)

    We are in the process of adding additional framework libraries (including LINQ, WCF and Workflow) to the above list. I'll blog details on them as they become available in the weeks and months ahead.

    Enabling Reference Source Access in Visual Studio 2008

    Enabling .NET Framework source access within Visual Studio 2008 only takes a few minutes to setup. Shawn Burke has a detailed blog post that covers the exact steps on how to enable this in more depth here.

    If you run into problems or have questions setting it up, please post a question in the Reference Source Forum on MSDN here.

    Stepping into .NET Framework Library Source

    Once you follow the configuration steps in Shawn’s post above, you’ll be able to dynamically load the debug symbols for .NET Framework libraries and step into the source code. VS 2008 will download both the symbols and source files on demand from the MSDN reference servers as you debug throughout the framework code:

    Developer comments are included in the source files. Above you can see an example of one in the Dispose method for the Control base class.

    Sometimes you'll see comments that reference a past bug/tracking number in our bug/work-item tracking database that provides additional history about a particular code decision. For example, the comment above calls out that a particular field shouldn't be nulled to maintain backwards compatibility with an older release of the framework, and points to a backwards compatibility bug that was fixed because of this.

    Reference License

    The .NET Framework source is being released under a read-only reference license. When we announced that we were releasing the source back in October, some people had concerns about the potential impact of their viewing the source. To help clarify and address these concerns, we made a small change to the license to specifically call out that the license does not apply to users developing software for a non-Windows platform that has “the same or substantially the same features or functionality” as the .NET Framework. If the software you are developing is for Windows platforms, you can look at the code, even if that software has "the same or substantially the same features or functionality" as the .NET Framework.

    Summary

    We think that enabling source code access and debugger integration of the .NET Framework libraries is going to be really valuable for .NET developers. Being able to step through and review the source should provide much better insight into how the .NET Framework libraries are implemented, and in turn enable you to build better applications and make even better use of them.

    Hope this helps,

    Scott

  • Another Potential Gotcha When Creating a Website that Allows Users to Share Uploaded Files

    Many web applications I've worked on have had a requirement along the following lines:

    • A user should be able to upload a file from their computer, and other users of the site should be able to download and view these documents.

    The specifics differ from project to project. For some projects, only certain users can upload files. In others, only certain subsets of users can view the uploaded files. In some cases, only authenticated users can view the files. In others, anyone can upload any file and these files are viewable by everyone.

    When faced with such a task there are two design decisions that need to be made up front:

    • How are the uploads going to be stored?
    • How are the files going to be downloaded?

    The uploaded files can be stored directly in the database or on the web server's file system. I've used both approaches in several projects, and the two ways each have their own pros and cons. I discuss the benefits and demerits of storing binary data in the file system versus the database in the four Working with Binary Data tutorials available from my Working with Data in ASP.NET 2.0 tutorial series.

    If you choose to store files on the file system, then these files can be downloaded in one of two ways: by pointing users to the web-accessible folder the uploaded files reside in, or by creating an ASP.NET web page that takes in as a querystring parameter the name of the file to download and then streams the file to the visitor's browser. The second option is a must if you need to apply any server-side logic or actions before a file is downloaded. For example, if only certain users can download a file, then you would want to use the second option so that you can first ensure that the current user can view the requested file. Alternatively, you may want to log all file downloads or perform some other sort of bookkeeping, in which case having an ASP.NET page to process downloads is advantageous over sending the user directly to the file to download.

    In a recent project we wanted to allow all users to download any files. In short, we had a page where any authenticated user (company employees) could upload company wide documents (like PDF press releases and whatnot) and any visitor - authenticated or not - could download the documents. For this model, it seemed to make sense to just have the files uploaded to a web-accessible folder. Then, on the page that listed the documents, there would be a link pointing directly to the PDF file (or whatever), like http://intranet/files/OctoberProfits.pdf.

    Simple enough, eh? But can you see the security hole? What if a disgruntled employee created an ASP.NET page with inline code that deleted all of the files on the web server? He could then unleash his wrath by visiting http://intranet/files/DeleteItAll.aspx. Or, worse yet, the disgruntled employee might write an ASP.NET page that connects to the HR database and displays salary information. Whoops!

    In short, whenever you allow a user to upload a document that is saved to a web accessible folder, and then you allow others to visit that document directly through their browser, there exists the possibility that a nefarious user will upload a script file so that they can execute code when the file is visited through a broweser. The best way to prevent this is to configure IIS so as not to allow script execution on the folder (and subfolders) where files are uploaded. That is, set aside a single folder where all documents will be uploaded, and then configure IIS to disable script execution on that folder.

    In IIS 6, you can disable script execution by going into the IIS Manager and drilling down to the appropriate folder. Right-click the folder and choose Properties from the context menu. From the Directory tab, change the “Execute permissions” setting from “Scripts only” to “None”, as shown in the screen shot below.

    With this change in place, if someone uploads a script file (such as a .asp or .aspx file) and then they (or another visitor) visits it through their browser, IIS will return an HTTP 403.1 error - Forbidden: Execute access is denied. Security hole plugged!

  • Visiting China, South Korea and Japan the Next Two Weeks

    This Friday I'm leaving for a 10 day trip to Asia.  Traveling on business doesn't usually get me excited (I was on a plane ~70 times last year), but I am really looking forward to this trip as it will be my first trip to Asia. 

    I'll be visiting China (Beijing, Shanghai, Shenzhen), South Korea (Seoul), and Japan (Tokyo), and I will be presenting at events, meeting with customers, and visiting one of the development teams in my group that is based in China.  I'm also hoping to get a chance to see immerse myself a little in the countries.

    Below are some details on the presentations I'll be doing during the trip if you are interested in attending or learning more:

    • China (Beijing, January 13th)
    • South Korea (Coex Conference Center 310 on January 17th)
    • Japan (Izumi Garden Gallery in Roppongi, Tokyo on January 21st)

    I've been extremely fortunate the last year to have had some great people volunteer to translate my blog posts into other languages (including Chinese and Japanese).  Below are links to a few of the feeds if English isn't your first language:

    I'm hoping to meet Hongchao Wang (who has been translating my posts to Chinese) and Chica (who has been translating my posts into Japanese) on the trip and have the chance to thank them personally. :-)

    Thanks,

    Scott

  • Re-launching Infozerk.com with Graffiti

    If you went to infozerk.com for the last three years it was a pretty stale site that usually didn't represent me very well. I decided to finally remedy that situation. I want to eventually consolidate dotavery.com, this blog, and infozerk.com all together and this is the first step.

    I didn't want to spend alot of time on the site so I wanted to use some sort of content management system. My good buddy Jayme talked me into using Graffiti which he has been working on over at Telligent.

    I have been very impressed with Graffiti, it can take a little bit of time to get used to how things work, but once I figured it out it was easy to customize a theme to do exactly what I wanted. It also has the capability to handle this blog, which I will be moving over at some point. (I will miss SubText but it will be nice to have it all in one place) I also wanted something that I could use to replace the static pages on The Lounge, which is my next step with this site.

    This is one of the goals I set out for myself at the beginning of the year, look for some more posts about my other goals (one of which will be completely changing the focus of this blog)

    -James

  • Dynamic LINQ (Part 1: Using the LINQ Dynamic Query Library)

    LINQ (language integrated query) is one of the new features provided with VS 2008 and .NET 3.5.  LINQ makes the concept of querying data a first class programming concept in .NET, and enables you to efficiently express queries in your programming language of choice.

    One of the benefits of LINQ is that it enables you to write type-safe queries in VB and C#.  This means you get compile-time checking of your LINQ queries, and full intellisense and refactoring support over your code:

    While writing type-safe queries is great for most scenarios, there are cases where you want the flexibility to dynamically construct queries on the fly.  For example: you might want to provide business intelligence UI within your application that allows an end-user business analyst to use drop-downs to build and express their own custom queries/views on top of data. 

    Traditionally these types of dynamic query scenarios are often handled by concatenating strings together to construct dynamic SQL queries.  Recently a few people have sent me mail asking how to handle these types of scenarios using LINQ.  The below post describes how you can use a Dynamic Query Library provided by the LINQ team to dynamically construct LINQ queries.

    Downloading the LINQ Dynamic Query Library

    Included on the VS 2008 Samples download page are pointers to VB and C# sample packages that include a cool dynamic query LINQ helper library.  Direct pointers to the dynamic query library (and documentation about it) can be found below:

    Both the VB and C# DynamicQuery samples include a source implementation of a helper library that allows you to express LINQ queries using extension methods that take string arguments instead of type-safe language operators.  You can copy/paste either the C# or VB implementations of the DynamicQuery library into your own projects and then use it where appropriate to more dynamically construct LINQ queries based on end-user input.

    Simple Dynamic Query Library Example

    You can use the DynamicQuery library against any LINQ data provider (including LINQ to SQL, LINQ to Objects, LINQ to XML, LINQ to Entities, LINQ to SharePoint, LINQ to TerraServer, etc).  Instead of using language operators or type-safe lambda extension methods to construct your LINQ queries, the dynamic query library provides you with string based extension methods that you can pass any string expression into.

    For example, below is a standard type-safe LINQ to SQL VB query that retrieves data from a Northwind database and displays it in a ASP.NET GridView control:

    Using the LINQ DynamicQuery library I could re-write the above query expression instead like so:

     

    Notice how the conditional-where clause and sort-orderby clause now take string expressions instead of code expressions.  Because they are late-bound strings I can dynamically construct them.  For example: I could provide UI to an end-user business analyst using my application that enables them to construct queries on their own (including arbitrary conditional clauses).

    Dynamic Query Library Documentation

    Included with the above VB and C# Dynamic Query samples is some HTML documentation that describes how to use the Dynamic Query Library extension methods in more detail.  It is definitely worth looking at if you want to use the helper library in more depth:

     

    Download and Run a Dynamic Query Library Sample

    You can download and run basic VB and C# samples I've put together that demonstrate using the Dynamic LINQ library in an ASP.NET web-site that queries the Northwind sample database using LINQ to SQL:

    You can use either Visual Web Developer 2008 Express (which is free) or VS 2008 to open and run them.

    Other Approaches to Constructing Dynamic LINQ Queries

    Using the dynamic query library is pretty simple and easy to use, and is particularly useful in scenarios where queries are completely dynamic and you want to provide end user UI to help build them.

    In a future blog post I'll delve further into building dynamic LINQ queries, and discuss other approaches you can use to structure your code using type-safe predicate methods (Joseph and Ben Albahari, authors of the excellent C# 3.0 In a Nutshell book, have a good post on this already here). 

    Hope this helps,

    Scott

  • 22books

    22books is a site focused on finding, creating, and sharing book lists. 22books was written using Ruby on Rails and was launched in December of 2007.

  • January 2008 MSDN Toolbox Column

    James Avery stepped in and wrote a number of guest toolbox columns for MSDN Magazine. This column featured MbUnit, Rhino.Mocks, JetBrain's dotTrace, and Continuous Integration from Addison Wesley.

  • What is Visual Studio, OnDotNet.com

    James Avery wrote this article for O'Reilly's OnDotNet site providing a quick tour of Microsoft's Visual Studio.

  • What is .NET, OnDotNet.com

    What is .NET, or How I Learned to Stop Worrying and Love the Runtime was written by James Avery for O'Reilly's OnDotNet site. The article covers the fundamentals of .NET and what is means to developers.