CI with MVC4 and Linux? No Problem

There are a lot of tools out there that we can use to automate our awesomeness. One of those easy/free/fun tools is a continuous integration (CI) server. Since we are building in Windows and deploying to Linux, we can go ahead and upgrade a CI server from 'important' to essential. There are various products out there that can accomplish this; I'm going with what I know and that is TeamCity from JetBrains.

On our Ubuntu box, after we finish the easy as candy install, we're going to want to install Mono. Check out this previous post on how to do that and set it up in your /etc/environment.

Now that Mono is rocking, lets hop over to our CI install. At the time of this post, the latest version of TeamCity is 7.0.2a. Looking at the code below, we're going to install Java, pull down the latest version of the TeamCity linux tar, decompress it, switch directories and fire it up :

sudo apt-get install openjdk-7-jre
wget - http://download.jetbrains.com/teamcity/TeamCity-7.0.2a.tar.gz
tar xfz TeamCity-7.0.2a.tar.gz
cd Teamcity
sudo ./bin/runAll.bat start

Pretty easy right? Lets check out our web interface:TeamcityWelcome

Yay! Everything works! You're also at a ubiquitous web interface which will likely give the Softie in you some solace. There are tons of resources online for setting up your project in TeamCity to check it out, build it, run tests, deploy, yada yada yada. If you are just getting started, I highly recommend Troy Hunt's post. You can also install Ruby on this server and have a wonderfully awesome auto-deployment method via Capistrano. If you are interested in deploying with Capistrano, let me know I'll try to barf out a quick post on it. This is a pretty short and to the point post, if it needs more content with regards to project checkouts and stuff, let me know and I can barf that out as well.

Running ASP.NET MVC4 on Ubuntu 12.04

The single most question I get regarding our Hello Postgres series over Tekpub is "How are you running your app on Linux?" As much as I enjoy sending volleys of fragmented explanations over emails to people, I decided it might be in everyone's best interest if I just blog it out. Technology moves fast... like super fast. So instead of showing you how to deploy an old and busted MVC3 app to a "has been" Ubuntu 11.10, we're going to be going with MVC4 and Ubuntu 12.04. We're so bleeding edge you can call us cutters!

I'm not going to show you how to set up and install your Ubuntu Server, as there is a ton of internet out there that can handle that for you and the installer itself is pretty hard to mess up. What I will cover is what to do after that last reboot where it is just you and Bash, together... at last.

SSH into your fancy new Ubuntu box with Putty and Run this:
sudo apt-get update && sudo apt-get -y install git-core curl python-software-properties

This is going to let us add the ppa(think of this as a package location) to install a current nginx:
sudo add-apt-repository ppa:nginx/stable
sudo apt-get update && sudo apt-get -y install nginx

Yes! We can now git things, we can curl things, and we can nginx things! Rawwrr!!! Let's kick some other processes to the curb... like that skank MySQL and that wet blanket Apache:
sudo update-rc.d mysql remove &&
sudo update-rc.d apache2 remove && mkdir mycoolapp

Now for the fun part, let's get Mono up and running. We really want the latest version of Mono, and like Nuget, not all packages in Ubuntu stay current. The fix for this in the mono world is to just grab this dudes install script and go have a drink because it takes a bit to finish:
mkdir mono-2.11
cd mono-2.11
wget --no-check-certificate https://github.com/nathanb/iws-snippets/raw/master/mono-install-scripts/ubuntu/install_mono-2.11.sh
chmod 755 install_mono-2.11.sh
./install_mono-2.11.sh

Now to check that we have the current version, we just run the following:
/opt/mono-2.11/bin/mono -V

Cool! We're current, but what we really need is to just be able to run "mono -V" anywhere we want. We can do that by modifying our /etc/environment file:
sudo nano /etc/environment
#then insert ":/opt/mono-2.11/bin" at the end of that path string. Log out, log in for it to take effect.

I don't now about you, but all this sys-admin crap is putting me to sleep and I want to code. Back on our Widows/Dev box, we're going to new up an empty MVC4 project (up to you to figure out how to get MVC4 going on your box, I used Web Platform Installer). After the project is new'ed up:

  • Add a controller
  • Add a view based on that controller
  • Hit F5 and make sure it works
  • Remove the EntityFramework.dll because we love our data and don't want to give our app a low self esteem
  • Once removed, hit F5 again

If everything is still running, then do a file system deploy to a local folder of your choice. Go check out what was just deployed, specifically the '/bin' folder. We want to remove the Web.Infrastructure.dll because Mono already has this built into its GAC. High Five Mono! With that out of the way, we are going to copy that deployment over to the 'mycoolapp' folder on our Linux server using WinsCP. While I don't have a WordPress plugin to read your mind, I hear you thinking "Rob, surely there are better ways to do this." and you are right. However, this post is long enough and that topic will likely go into its own post.

SSH back into your linux box, and 'cd mycoolapp'. "cd" into your "mycoolapp" directory, type in 'ls' to see the contents and make sure your app made it over. Once we see it, we can do a cassini like run of it by typeing out xsp4 and hitting enter. Try and hit your box on the port 8080 (xsp4 should tell you what it is running on) and see if you can see your app:

Yay! Our code works and we're so open source now that we can wear tight jeans at Starbucks and put Bon Iver on our ipod... just like those Rails wankers! As for nginx and running our site as a Mono service, we can just use some of the nice documentation over at the Mono site:

Nginx config:
http://www.mono-project.com/FastCGI_Nginx

Mono as a service:
http://yojimbo87.github.com/2010/03/14/mono-startup-script.html

Oklahoma City JavaScript Group

A new user group is starting in Oklahoma City and it is one I find
completely fascinating... a Javascript group! With the monumental surge in popularity of Javascript over the past year or so, this type of group is a can't miss.

Who should attend you might ask? All of us in the tech field because it is pretty much everywhere now. Even a database douche bag like myself is attending because I have finally accepted that Javascript is inescapable. Javascript is all over the web, going to be all over the desktop with Windows 8, appearing in the mobile markets, and starting to take over the back end with rad technologies like NodeJS.

I'm a complete noob with Javascript and I feel completely comfortable coming to this group because with as fast as this technology is moving, we're all noobs and need to learn from each other. This group will not waste your time because

Learning is cumulative.

As a speed freak, my primary interest lies in NodeJS, but I know for a fact that I will learn something to help me with Node from Vance talking about BDD testing with a testing framework called Jasmine. That is just how it works. He gives, I take, and down the road I give back. Even if you don't know or care a thing about Javascript, it can be a good networking opportunity with some of the people in our community that have a true passion for tech and are trying to push tech boundaries. If that doesn't interest you, you can lurk in the back and watch me ask stupid questions as I try and get the info to the highly firewall'ed learning center that is my brain... or just show up for a free lunch. You have options.

If you don't go or don't plan to go...no worries. I'll still love you (as a friend), but in our very fluid industry, dropping an hour a month of your time on something like this is an incredibly way to not go stagnant.

The first meeting is January 24th from 11:30-12:30 at the OKC CoCo. Come say hi and most importantly.... help me learn this stuff!

For more information, hit up http://okcjs.com/

Getting started with Postgres and MVC3

"Postgres? WTF man, you're a SQL Server DBA!" It's true, I am a SQL Server DBA but even I have to admit that the Postgres team is doing some wildy exciting stuff. With that in mind, lets take a quick look at just how quickly we can replace SQL Server with a free/open full bore Enterprise database system like Postgres.

One of the first things you will notice is Postgres is way easier to install than SQL Server. The SQL Server install is born from a truck stop romance between TFS and Sharepoint that someone found in a garbage bag in a dumpster and burned to a DVD. Postgres takes a different angle and opts for a 'less is more' type of approach. One of the big things you will want to pay attention to is the password you give the default 'postgres' account. Think of him as the SA account in SQL Server as he too wields a mighty big stick.

Once we finish the installer, lets open up pgAdmin III (henceforth known as pgAdmin). If you are accustomed to SSMS with SQL Server and always wished it went on a diet... then you might like pgAdmin. Be warned though, pgAdmin isn't SSMS on a diet... it is like SSMS with Anorexia. Once pgAdmin is opened and you are connected, then drill down to 'databases' and do a 'new database' and use the GUI to make your fancy new demo database, or just run the raw sql:

 
CREATE DATABASE "Demo"
  WITH ENCODING='UTF8'
       OWNER=postgres
       CONNECTION LIMIT=-1;
 

Next, lets hop in that database and throw up some tables/data:

 
CREATE TABLE "Colors"
(
	colorid serial NOT NULL,
	colorname character varying(50) NOT NULL,
	constraint pk_colorid PRIMARY KEY (colorid)
)
WITH (OIDS=false);
 
INSERT INTO "Colors" (colorname)
VALUES ('red');
INSERT INTO "Colors" (colorname)
VALUES ('blue');
SELECT * FROM "Colors";
 

So why the quotes around the table names you might be asking... well, SQL Server has a bit of Honey Badger approach to what case you assign names and Postgres treats it like a pretty big deal. Luckily, overcoming it isn't a big deal, we just double quote it. Now, if you don't have years of tsql or camelcasing drilled into your brain, you can easy just do everything in lower case and not fight the double quote dragon.
What else sticks out there... oh Data Types! You will have to adjust to some variance in data types, but in a good way. Postgres has a lavish assortment of data types for you to fall in love with and one of which is using 'serial' instead of the whole IDENTITY(1,1) you might be used to in SQL Server. There are also some really cool things called Sequences in Postgres which are blazing fast and give you the value of the ID before you insert it. DBAs like them because it can get you AppDevs away from fatty GUID primary keys.

Fire up a new MVC3 app and hop into the Nuget console. From there, we're going to add our Postgre provider with "Install-Package Npgsql". Now that we have our vital provider assemblies, lets go mess around in the web.config.

 
<configuration>
	<system.data>
		<DbProviderFactories>
			<add name="Npgsql Data Provider"
        invariant="Npgsql"
        support="FF"
        description=".Net Framework Data Provider for Postgresql Server"
        type="Npgsql.NpgsqlFactory, Npgsql, Version=2.0.11.0, Culture=neutral, PublicKeyToken=5d8b90d52f46fda7" />
		</DbProviderFactories>
	</system.data>
 
	<connectionStrings>
		<add name="pg" connectionString="Server=127.0.0.1;Port=5432;Database=Demo;User Id=postgres;Password=dontusethisaccount;CommandTimeout=20;" providerName="Npgsql" />
	</connectionStrings>
 

Yeah! We have our provider factory rocking and our connection string. Yes, I am using the postgres account in this demo and yes that is wrong/stupid, but I'm having so much fun playing with the Postgres engine that I decided to take a shortcut there for this entry.

So our color app is pretty quick and dirty. In keeping with that spirit, I'm going to use Massive as my ORM of choice. It already has a Postgres version ready to go. PetaPoco has some very nice Postgres options in it, but Massive is very quick/easy and the Expandos makes all the strong-typers uncomfortable and I love a good code-trolling.

In the Model folder of our app, I create a new class file called Massive and I copy/paste the Postgres version into it, and I also created a new class file called "Colors" that I pasted the following in:

 
	public class Color : DynamicModel
	{
		public Color() : base("pg", "Colors", "colorid") { }
 
	}
 

All wired up! In the home controller, we'll throw some code to play with our table in:

 
var table = new Color();
var colors = table.All();
ViewBag.colors = colors;
 

And then in our view:

 
@foreach (var color in ViewBag.colors)
{
	 @: @color.colorid @color.colorname
}
 

F5 and what do we get? BOOM!!! "Error 42P01: relation "colors" does not exist"
So remember when I started our Colors table with a capital C? This is where it can come back to bite you... and also one of the reasons I absolutely love Micro-ORMs. Fast, malleable, and so easy a DBA can use it. Let's hop into Massive and throw some quotes around our table access:

 
string sql = limit > 0 ? "SELECT TOP " + limit + " {0} FROM \"{1}\" " : "SELECT {0} FROM \"{1}\" ";
 

and Viola! It works! In pretty much the time it takes to install SQL Server, we're already rocking our stunning and highly valuable colors app!

Rock out with your Procmon out

If you are not a fan of Sysinternals tools then you clearly do not have your geek merit badge. While I have always kept a local copy in the past, there is now a cool and easy way to get to Sysinternals and it is via
Start + run + \\live.sysinternals.com\tools

If you are already a gigantic nerd like me and have various login scripts that run, it is way easy to simply map that share by throwing the following into your login scripts:

net use s: \\live.sysinternals.com\tools

Tekpub – Full Throttle with Rob Sullivan show notes

The question emails are starting to pile in (which is a good thing!) so this post is basically a way to centralize links/answers. In the show, I say a few times "we'll see this later" (like with SQL Server Profiler) and we never see it... that was mostly my fault. For some reason I kept thinking I would be able to squeeze 5 hours of content into 1 hour and was wrong. If there are aspects of SQL Server that you would like to see more of, as well as real world problems that you may have and would like to see solutions to, then let Tekpub know. We'll see if we can hack out real world solutions wrapped in a warm blanket of statistics and opinion.

Questions:
How do you get the stats to appear in the "messages" window?
Those stats come from the following commands:
SET STATISTICS IO ON;
SET STATISTICS TIME ON;
If you don't want to turn it on for every query/window, you can do what I do and set them to always be on in Management studio, you can do this by :
Tools -> Options -> Query Execution -> SQL Server -> Advanced
then check:
SET STATISTICS IO ON;
SET STATISTICS TIME ON;

Lock Pages in Memory, is that always good?
No, and perhaps I glossed over that section too much or technology changes too fast. While Lock Pages in Memory can be a wonderful setting, there are of course times when it can be pretty bad. You can read more about that here: http://sqlserverperformance.wordpress.com/2011/02/14/sql-server-and-the-lock-pages-in-memory-right-in-windows-server/

Why no filtered index on Posts?
In our first scenario, to get the front page up... we just went with the index that the engine suggested.. That got us up and running which is great, but there is room for improvement. To me, filtered indexes are more of a business rule decision and with enough time, if we saw that the load was heavy and predictable enough, we would have likely thrown a filtered index to satisfy the front page query.

What were you using for "Intellisense" in management studio? I am not extremely big on intellisense in general, however, I am kind of big on formatting and having a variety of administrative snippets at my finger tips. I use a 3rd party tool called Red-Gate SQL Prompt and have it moderately customized. Keep in mind, I really only use this type of product on data sets I am not very familiar with like the one in this demo. On datasets that I know pretty well, I find intellisense to really just get in the way and make me angry.

And the database? I am still working on a way to make the DB available. It is roughly 40GB raw, and 4GB compressed. If having the data is a big deal to you, hit me up on email and we can work something out (S3,FTP, yada yada yada)

Tech used in the series:
Visual Studio 2010 / MVC3 / EF 4.1
SQL Server 2008 R2 / Management Studio
Red Gate SQL Prompt "Intellisense" and "snippets" in Management Studio
SQL Sentry Plan Explorer The fun/effective way to view execution plans
EF Profiler The SQL Server friendly way to profile EF
CPU-Z A tried and true hardware enthusiasts way of checking hardware specs.

Source code: https://github.com/DataChomp/StackOverFaux

Massive and using multiple args:

So, in Rob Conery's example of using multiple args in a query

 
var tbl = new Products();
var products = tbl.All(where: "CategoryID = @0 AND UnitPrice > @1", orderBy: "ProductName", limit: 20, args: 5,20);

It makes the args appear to be this completely bad ass and intelligent command that knows exactly when and where to parse/apply the values. Well, in the real world versions of Massive the compiler doesn't take to kindly to that sort of syntax.

args: is a parameter of type object [] so the way I use multiple params is as follows:

 
object[] queryargs = {"Okapi", "online"};
var DBBasics = table.All(where: "WHERE ServerName=@0 and status = @1", args: queryargs );
 

and low and behold the magic happens.

Getting Started with Massive in MVC3 – Part 2

If you read over my first post on this you probably have some questions going through your mind.... I know I did... So lets ask them:

Rob, your queries make no sense to me because I have no idea about this 'drinks' table you're hitting.
I agree. When I start to play with something I tend to use very simple databases that I know for simplicity. From here on, I'll be using Adventure Works 2008 R2

All this stuff is dynamic and I don't know how to deal with it. I'm just as strongly typed as the next guy so this was an emotional hurdle. But we're not here for therapy so lets code using one of my favorite examples - Credit Cards!
Class:

public class CreditCard : DynamicModel
	{  public CreditCard()	: base("AW_Sales_ConnStr")
		{
			PrimaryKeyField = "CreditCardID";
		}
	}

ViewModel:

public class CreditCardViewModel
	{public IEnumerable&lt;Cards&gt;{ get; set; } }

Controller:

public ActionResult Index()
{
      var table = new CreditCard();
      var cards = table.All();
 
	var viewModel = new CreditCardViewModel
	{
          Cards = cards
         };
	return View(viewModel);
}

View:

@model Kona.Web.Models.CreditCardViewModel
@{ViewBag.Title = "Adventure Works Card Stealer";}
<table>
@foreach (var item in Model.Cards)
{
<tbody>
<tr>
<td>@item.CardType</td>
<td>@item.CardNumber</td>
<td>@item.ExpMonth</td>
<td>@item.ExpYear</td>
</tr>
 
}</tbody>
</table>
 

So 2 seconds after this works, I hope that you are feeling very uncomfortable about presenting Credit Card numbers or even pulling them. That is a good thing.... So since we have a special case where we care a whole bunch about not returning that data... lets change our controller:

var table = new CreditCard();
//var cards = table.All();
var cards = table.Query(@"SELECT CardType, ExpMonth ,ExpYear from CreditCard");

Sweet! it compiles, we're ready to check in and go grab a beer right? Heh, nope.... refresh your app:
'System.Dynamic.ExpandoObject' does not contain a definition for 'CardNumber'

So we update our view....

{
 
@item.CardType
@item.ExpMonth
@item.ExpYear
 
}

and all is well again.

Make no mistake about it... Massive will give you all the rope you want to easily swing around your apps data jungle... it will also give you all the rope you want to hang yourself if that is what you choose to do with it. Which is precisely why I like it. I feel like Entity Framework4 (And don't get me wrong, I like EF4) is a micromanaging boss that wants to check my every move. Massive is like the boss that gives me a task and trusts that I'll get it done.

Welcome to dynamic!

One last bit because this is getting lengthy:
Credit cards in AdventureWorks 2008 R2 are locked into a Sales schema and I can't figure out how make my class take "advantage" of that
This is a fun thing we DBA's like to do for the sole purpose of driving AppDev's nuts (with just some oh so minor security and organizational reasoning behind it). In our Credit Card model we can update to specify the TableName. Also, I use a special connection string:

public CreditCard() : base("AW_Sales_ConnStr")
		{
			TableName = "[Sales].[CreditCard]";
			PrimaryKeyField = "CreditCardID";
		}

That setting in the base() maps to my web.config:

add name="AW_Sales_ConnStr" connectionString="Data Source=localhost;Initial Catalog=AdventureWorks2008R2;User ID=aw_Kona_Sales;Password=Sales_Password;Application Name=KonaSales;" providerName="System.Data.SqlClient"

and in SQL Server this maps to the Sales Schema (key part being the specification of a Default_Schema:

CREATE USER [aw_Kona_Sales] FOR LOGIN [aw_Kona_Sales] WITH DEFAULT_SCHEMA=[Sales]
 

Last thing, I promise ... In my first post, I included how the commands were rendering out in the SQL Server. I didn't this time because this edition is more focused on a "getting things done in MVC" side of things.
Trust me, I am a DBA first ( and I imagine some of my code probably shows it) and I am still profiling every single thing I do with Massive looking for problems.

Getting Started with Massive in MVC3

Rob Conery has created a new ORM of sorts https://github.com/robconery/massive  and as a DBA, I must say I like what I see so far.

Getting started:  So some of you might be noob programmers like myself and after adding Massive to your MVC application via Nuget you become lost in space because it doesn't work.  The fix to this is rather easy,  just check the properties of the  Massive.cs file and make sure its build action is set to Compile... you might want to change the namespace as well depending on how picky you are with namespace juggling.

We DBA's are a particularly picky bunch and one of the things we hate about ORM's  is something called 'plan cache poisoning'.  Want to know why your DBA hated Linq to SQL until  .NET 4 came out?  One of the reasons is 'plan cache poisoning'  which I will call PCP for short.   When you make a query to the database SQL Server compiles the that query into a plan so that the next time you go to run  it... it doesn't have to compile a new plan... it just pulls from its cache.  When it comes to  linq-sql and EF before  .net 4....  they would pass a new string length depending upon the size of the string you are looking for.  So, if you have a table of contacts and want to pull someone up by first name...  it would compile a plan for 'Rob', a plan for 'Robert',  a plan for 'Mike' and so forth until you have exhausted all the different name lengths.    This sucks.   With Massive your  DBA won't wake up in cold sweats thinking about his beloved plan cache because of your ORM.

To make  basic call with Massive on a Drinks Table (DrinkID, DrinkName) we do this:

var table = new Drinks();<br/>
var drinks = table.All();

If you have done any Linq or EF you know that even a basic call like that is going to cause Linq or EF to create
the ugliest looking t-sql it can by breaking out into extent aliases God knows what else it likes to do to say 'hey look at me'
Does Massive have this problem? Nope, looking at profiler we get straight forward SQL -

SELECT * FROM DRINKS

Well, that's cool and all if you want to start barfing out tables into your app, but how often to get to exercise such negligence?
Rarely, so lets move to some conditions... lets look up a drink name:

var table = new Drinks();<br/>
var drinks = table.All(where: "WHERE DrinkName=@0", args: "Fire Rock Pale Ale");

Knowing EF, we're likely going to have 15 lines of aliases and subqueries to run this most complicated of queries..
In Massive, we get:

EXEC SP_EXECUTESQL N'SELECT * FROM Drinks WHERE DrinkName=@0',N'@0 nvarchar(4000)',@0=N'Fire Rock Pale Ale'

In addition to a simple to grok TSQL, we also start getting reusable query plans.

More examples to come as I start to peel back the covers a bit on this new tool.

Deploy MVC3 to Shared Hosting

Deploying ASP MVC3 to shared hosting isn't always sunshine and lollipops.   Though my beloved hosting company Arvixe is pretty cutting edge,  it didn't have (and I didn't request) that they hop on MVC3 or its beta.   Regardless, I went to deploy an MVC3 app and it blew up because of a variety of dependencies.   As discovered in early MVC2 work, you have to set some assemblies to 'copy local' to get it to work on shared hosting.

The list of assemblies to set to copy local = true is:

  • Microsoft.Web.Infrastructure
  • System.Web.Helpers
  • System.Web.Mvc
  • System.Web.Razor
  • System.Web.Webpages
  • System.Web.WebPages.Deployment
  • System.Web.Webpages.Razor

Drew Miller ( blog | twitter ) has a great breakdown on this : Click for Breakdown