I have been developing a strategy for deploying an ASP.Net Web Application (Visual Studio 2008, .NET 3.5) to staging servers, and to production.  In the process, I have encountered 3 products from Microsoft that almost sound synonymous to the untrained ear, but are very different and are designed for completely different purposes.  I wish that this post would have been written by Microsoft to emphasize the differences and pros/cons from their perspective.

But after spending some time researching and playing around with each, here is my summary:

Web Setup Projects

• Builds an MSI (installer) that will install and setup a web application on an IIS server.
• Included in Visual Studio 2008 out of the box.
• Accessed by: File > New Project, select “Other Project Types” > “Setup and Deployment” on the left side, then select “Web Setup Project” on the right side.

Web Deployment Projects

• Uses MS Build scripts to pre-compile and prepare a web application for deployment.
• Visual Studio “add-in” released by Microsoft.
• Downloaded free at: http://www.microsoft.com/downloads/details.aspx?FamilyId=0AA30AE8-C73B-4BDD-BB1B-FE697256C459&displaylang=en
• Accessed by: (after installing add-in) Right-click the web application project node and select “Add Web Deployment project…”
• Lets you specify different “build configurations” (besides Debug and Release) that enable you to have web.config replacement for your different builds.
• Pre-compiles aspx pages with two choices, allowing the .aspx (not the code behind) to be modified on the server, or even compiling in the data from the .aspx files.  In the second case, a “dummy” file is created with the aspx filename, but it is essentially blank–everything gets put into the dll.
• Merges multiple dll’s into a single one for easier deployment.

MSDeploy

• Written by the IIS team.
• Enables easy syncing of content and configuration settings between IIS servers.
• http://www.iis.net/?tabid=20111
• Installed as an addition to the IIS Manager on either the source or destination server.
• Easy to package up an existing virtual directory/application with all its content, settings, dependencies, etc.
• Easy to push out this package to many servers in a farm to have identical web sites.

My understanding is that Visual Studio 2010 is much more prepared to interact directly with MSDeploy than Visual Studio 2008 is (though I haven’t tried these features out there yet).  It seems they have taken many of the concepts of “Web Deployment Projects” and built them in standard to VS 2010, and used that type of methodology to produce the packages for MS Deploy.

Each time a new project is started, there is always a discussion (or there should be!) of what guidelines should be used in design, naming conventions, etc.

I typically like to defer to the standards given by the company that creates the language, and then define any differences. For the case of C# and .NET languages, this means WWMD (What Would Microsoft Do).

Here is a link to the Design Guidelines for Developing Class Libraries from msdn:

http://msdn.microsoft.com/en-us/library/ms229042.aspx

Subsections of this main topic are:

• Guidelines for Names
• Type Design Guidelines
• Member Design Guidelines
• Designing for Extensibility
• Design Guidelines for Exceptions
• Usage Guidelines

This is a pretty good specification, even getting down to capitalization schemes.  And between this and Code Complete, I have found that almost all questions that come up can be addressed.

I have written a command line application that, among other things, converts Word Documents to HTML using the Microsoft.Office.Interop.Word library.

I haven’t used it for a while, and recently had to re-run some conetnt and found this error:

“Word cannot start the convertor mswrd632″ occurring quite frequently.

Apparently Microsoft released a security update on 2009-12-08 that prevented some of the access 3rd party programs used to have.

You can re-enable this functionality by:

1. Open Regedit and browse to HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Applets
2. Create a key “Wordpad” if it is not already there (as a child of Applets)
3. Add a DWORD Value with a name of “AllowConversion” and a value of “1″

Here is an article on technet describing the issue:

http://blogs.technet.com/wordonenotesupport/archive/2009/12/10/when-opening-the-word-document-users-are-getting-error-word-cannot-start-the-convertor-mswrd632-wpc-or-cannot-load-word-for-windows-6-0-files.aspx

And here is the KB article from Microsoft support:

http://support.microsoft.com/kb/973904

*** Updated 2009-12-21 *** A comment was left suggesting that “Re-enabling the vulnerable converter that is designed to work with WordPad is probably not a good idea.” They give a couple other options that are worth considering before simply opening the door back up.  Please see the first comment below.

There are a few different ways to integrate Windows Active Directory (Domain) authentication with SVN. Specifically the two that seemed most promising were LDAP and SSPI, with SSPI looking the most straightforward, so that is the one that I will outline here.

Include dependent modules
First we need to include the apache modules we’ll be using.

1. Add to the “httpd\modules” directory, the following files:

• mod_auth_sspi.so
• mod_authz_svn.so

2. Add to the “httpd\conf\httpd.conf” file, the following lines:

• LoadModule sspi_auth_module modules/mod_auth_sspi.so
• LoadModule authz_svn_module modules/mod_authz_svn.so

Create svn access file
In the “httpd\conf”directory, create a new file called “svnaccess.conf”

Using active directory (windows domain) groups will be discussed later.

Add something similar to the following to this file:

[groups]
engineers = MYDOMAIN\bob, MYDOMAIN\patty, MYDOMAIN\steve
temps = MYDOMAIN\julie, MYDOMAIN\george

[/]
@engineers = rw
@temps = r

[/customer1/internal_work]
@temps = rw

Note that you can set different permissions for different groups or individuals at any folder level. Permissions will apply to subdirectories unless a more specific assignment is made. For instance, the @temps group has read (“r”) access to everything but has read-write (“rw”) access to the /customer1/internal_work directory.

Modify “Location” tag in httpd.conf file

In the “httpd/conf/httpd.conf” file, modify the Location tag to look similar to the following:

<Location /repos>
	DAV svn
	SVNPath c:\svn_repository

	AuthName "Subversion Authentication"
	AuthType SSPI
	SSPIAuth On
	SSPIAuthoritative On
	SSPIOfferBasic On
	SSPIDomain MYDOMAIN
	require valid-user

	AuthzSVNAccessFile "conf/svnaccess.conf"
</Location>

That tells this specific location/repository to use our SSPI authentication with the domain “MYDOMAIN”. The “require valid-user” makes sure they are authenticated with a user (any user) in the domain, and then the AuthzSVNAccessFile “conf/svnaccess.conf” says that the remaining authorization rules will be defined in the svnaccess.conf we setup earlier.

Using Active Directory (Windows Domain) Groups

I did quite a bit of searching around to find out how to use existing Active Directory (Windows Domain) groups in the svn access file. The best I could find was: http://www.thoughtspark.org/node/26 where they create a custom python script that is periodically run, which queries the Active directory and then modifies the svnaccess.conf file to add the group information. This is definitely a hack approach, but the best solution I could find.

Helpful links
The following links were the most helpful in figuring all this out.

http://geekswithblogs.net/flanakin/archive/2005/08/31/51743.aspx
http://blog.michaelcheng.idv.hk/2006/10/windows-domain-authentication-with.html
http://www.subversionary.org/sspidomainauth
http://www.thoughtspark.org/node/26
http://blogs.open.collab.net/svn/2009/03/subversion-with-apache-and-ldap-updated.html

When I first heard of Data Mining, I wondered what the difference was between it and simple indexing on one hand or statistics on the other. Was this just another buzz-word?

Since then, I’ve been been asked the same question by others.  This week, I came across an interesting article on the Data Miners Blog that did a good job of outlining some of the similarities and differences between Data Mining and Statistics.

In a nutshell, Data Mining is concerned with the process of obtaining meaningful information from raw data, some of which is not necessarily apparent to human observers. Statistical methods are some of the tools that can be used in this process, but there are also many other pieces of the puzzle.

I am going through the process of setting up SVN on Windows and later will be adding authentication via Active Directory. I plan to post about each step along the way, as a resource to myself for future work, and maybe it will help someone else as well.

Basic Installation and Setup Instructions

Environment:

• Subversion
• Apache
• Windows Server 2003
• No Authentication

1. Get the windows binaries
http://subversion.tigris.org/getting.html#windows

2. Run the install, make sure to check the box to include the apache modules.

Note:

• This install also brings a copy of Apache with it, so you don’t have to download and install it separately.
• The Apache service in the windows services list is: “CollabNet Subversion Apache”
• The install directory for Apache is (by default): C:\Program Files\CollabNet\Subversion Server\httpd

3. Open port 80 (or whatever port you chose) in Windows Firewall

4. Create the SVN repository

• The install will create a directory “C:\svn_repository” but it will not set it up as a repository
• From a command prompt run: “svnadmin create c:\svn_repository”
• This will add a handful of subdirectories and files

5. Ensure needed modules are present and “required” in Apache.

• This came automatically from the install for me, but if you’re using a separate Apache install you’ll need these
• In httpd\modules dir:
  • mod_dav.so
  • mod_dav_svn.so
• Corresponding “require” statements should exist in httpd/conf/httpd.conf
  • LoadModule dav_module modules/mod_dav.so
  • LoadModule dav_svn_module modules/mod_dav_svn.so

6: Add SVN “Location” information to the bottom of the httpd/conf/httpd.conf file.

<Location /repos>
DAV svn

SVNPath C:\svn_repository
</Location>

• Note that you want “SVNPath” Not “SVNParentPath” unless you are trying to have a parent folder with multiple repositories under it.
• SVNPath should point to the directory where you created the repository earlier
• /repos is what will be in your URL to request content, ie http://server/repos

Testing Your Setup

To test Apache, go to the default page in a browser:

http://server/

This should display a page that says “It Worked”

Once you know Apache is up, you can try to browse your your repository with a Tortoise SVN Repo browser.

1. Point it at: http://server/repos (where /repos is according to what you put in the httpd.conf file earlier)

This should allow you to browse (an empty repo).

2. In the repo browser, create a test folder.
3. Do a checkout of this folder to your local machine
4. Create a new text file in that folder on the hard drive.
5. Do an svn-add on that file
6. Do an svn-commit on that file
7. Open a new repo browser and browse into your new folder and ensure that the file exists.

Today after installing some new software and rebooting a VM, I ran a batch file to do a “net start” on a number of services and it failed with:

'net' is not recognized as an internal or external command, operable program or batch file


Apparently the install messed up my PATH environment variable, which was easy to fix. But it did take a little while to figure out what happened, so I thought I would mention it here.

The net.exe file is found in C:\Windows\System32 (if that’s where windows is installed).

In conducting research for computer science topics, I often find myself with huge CSV files that I open in Excel and try to make graphs. Often, I have way too many data points to graph. Sometimes, I do fancy things in my code to produce only a certain number of data points that are averages or maximums over a set, but other times I just want to pare down the points in Excel to make the graph.

The following shows how to use Excel formulas to get every nth value from a column.

Example:

• Data is in columns B and C, ranging from row 4 to row 10004
• Goal: put every 500th pair of values in Columns E and F, starting in row 4

Solution:

1. Put the value 500 in a cell to reference later, (example: C2).
2. In E4, put the following formula: “=OFFSET(B$4,(ROW()-4)*$C$2,0)”
3. Select E4 and use the fill box and drag to the right to fill F4
4. Select both E4 and F4 and use the fill box to drag down as far as you need

That’s it, you now have every 500th data point. Simply change the value in C2 to 100, to get every 100th instead, etc.

A little while back I wrote a blog post about how I wasn’t very excited about the news of Google’s new browser, Chrome. My feeling being that we have some pretty good browsers already, and I wished people would focus on making other improvements instead of reinventing the wheel, and trying to force all web pages to render well in yet another browser.

Now here we are a few months later, and I must say, the wheel was reinvented very well this time…And since installing Chrome, I very rarely use other browsers.

Here is what I really like about Chrome

1. Startup Speed. If I accidentally click Firefox, and start waiting, I usually click Chrome and I’m already off and into whatever page I needed long before Firefox finally opens.

2. One box for everything. I thought this would drive me crazy because I thought they would mix up addresses for searches and vice versa, but leave it to the folks and Google to get it right. I don’t think I’ve ever had a problem of them mixing it up in an unexpected way. On the other hand, now when I use Firefox, I constantly find myself putting searches in the address bar. It seems so archaic to have to have two separate boxes =).

3. Most visited startup page. I didn’t think this would really impress me that much either, but I was wrong. The nine pages it displays capture almost every page I visit on a recurring basis, and anything else starts with a search anyway.

4. Tab independence. The ability to move tabs from one browser to another or to it’s own window is a huge improvement over other browsers. Also to have other tabs unaffected when one dies is very nice.

At this point, the only reason why I use other browsers is:

1. Some pages don’t render right. These haven’t been very frequent, but it has happened.

2. Firebug.

This principle exists independent of any specific domain, and has come up recently in two completely different contexts.

First, I was having a discussion about some different civilizations and societies from history, and it was noted that the culture of the group seemed to almost always be established by the leader and his or her attitude. If a king was honorable, his subjects tended to follow his lead. If a leader was lazy or promiscuous, so went the folks.

Then a little later, I was watching a sports show, talking about the NCAA basketball tournament, and the commentators all agreed that the attitude and demeanor of the coach was the biggest element in shaping the team.

This principle absolutely holds true in business. Zappos is a great example of everyone at the company enjoying life, because that’s the attitude of the CEO. I have also observed that the best way to motivate a team, department or company, is simply the example of the leader. If they have a good work ethic, stay late to get tasks done, and are always excited, these same characteristics tend to find their way to the masses, as if by magic. On the other hand, nothing seems to keep people from pushing themselves, like a leader who is lazy.

While this is the natural flow, it can be altered. Even if your direct supervisor falls into the not-so-good category, you can make a conscious decision to break the chain. By setting the right example and having the right attitude, your group can attain success and set the standard for everyone else.