Security and Code Analysis tools I use

Here are some of the tools I use for Security and Code analysis  (I bet you didn't see that coming from!)

WebConfig Analyzer - you can do a stand alone download and feed your webconfig into it

WireShark Use this to see what is going on on the network.

Fiddler - Great for https inspection.

Netsparker Use it to hit test sites and see if throws back anything useful.

BackTrack 4 Not sure what needs to be said here other than the best way to get a white hat, is to take a black hat and bleach it.

FXCop I tun this against my code when I want to feel stupid and see how many places I've goofed.  Things putting getters and setters on read only data. Doh!

Reflector Other peoples code and programs look pretty fun when uncompiled. Likewise, this is also good for making sure you didn't leave any sensitive information in your own binaries.

FireFox add-ons:

ViewState

FireBug

And then everything else that SnipeyHead (Blog | Twitter) uses:

http://www.snipe.net/2010/10/firefox-addons-xss-testing/

Pack the powder, light the wick!

There is going to be a strong disturbance in the force, Dallas Feb 21st - 25th when the SQLSkills team comes down for a Master Immersion class : http://www.sqlskills.com/Master1-Dallas-20110221.asp .

I am not going to spend a lot of time talking about why I want to attend a class taught by Paul Randal (blog | twitter) and Kimberly Tripp (blog | twitter). Their body of work in both the education and SQL Server stands second to no one, so trying to go on about this just belabors the point. Simply put, if you want to be among the best, then one should find the best and learn from them.

Now, let us graduate to the less obvious aspect of this post, which is why I would make the best use of this knowledge.  With technical debt, you sometimes have to write SQL checks that your brain can't cash. This class is like adding a few 0's to your brain balance.   As I take in the knowledge from this class, it is going to manifest itself into a trident to attack the following issues:

Personal: Let's be honest, keeping up with technology is an infinite and exhaustive grind.  Unfortunately, this can eventually create stress on one's work/life balance as the unending need to keep learning is often standing right in the way of the unending need to be with one's family. The quality and depth of this immersion event is an incredible opportunity to trade a few days in order to receive many years worth of nights. I personally think a good DBA is a "Jack of all trades, Master of one"... it is time I establish my mastery.

Professional: I currently work as the lone DBA for a virtual high school. This is simply the most fascinating job I have ever had for a variety of reasons. Having never had a DBA and more the point, no solid guidance in how to do things, the situations I come across can be truly migraine-inducing. The quicker I am able diagnose the correct path,  the more time I can spend on what really matters. Things like building analysis cubes which enable our teachers and staff to quickly identify where the system is breaking down with regards to the students' education. By doing this, it creates an opportunity for system efficiencies and satisfying end user experiences which are far more interesting and worthwhile than trying to continually identify where our SQL Servers are breaking down.

Community: As a founder of the Oklahoma City SQL Server Developers Group, what you teach me doesn't stop at my front lobe. You essentially essentially aid me in becoming Johnny SQL-Seed, spreading my knowledge to our community as I continue to give more talks to our group and region. In addition, my hope is that the networking from this class will enable us to find more speakers, as well as bring different points of views and technological diversity to our meetings. Without question, being involved in the SQL community is a lot of fun. However, the downside is that while I am organizing or speaking, I tend to miss out on a lot of content. This class is a chance for me to participate instead of provide, and I can't wait to be immersed in this baptism of SQL.

In closing, this is an incredible opportunity put out there by the SQLSkills team and I truly hope to INNER JOIN them in Dallas.

SQL Server Variety Hour

Thanks to everyone who came to the OKCSQL  last night and endured my presentation.  We covered a lot of cool stuff and since I didn't have any links in my slides, I figured I would link up the stuff we talked about:

WhoIsActive - http://sqlblog.com/blogs/adam_machanic/archive/2010/10/21/who-is-active-v10-00-dmv-monitoring-made-easy.aspx

SQLioSim- http://support.microsoft.com/kb/231619

SQL SourceControl- http://www.red-gate.com/products/sql-development/sql-source-control/

.Net Reflector- http://www.red-gate.com/products/dotnet-development/reflector/

LinqPad- http://linqpad.net/

EF Profiler- http://efprof.com/

RavenDB- http://ravendb.net/

Virtualization - http://www.vmware.com/

Stored Procedure Output Params and Return Values

Many times I see abuse of the Return Value in a stored procedure. What I mean by this is essentially using the Return Value to send back say an identity value or something like:

 
SET @TheID = SCOPE_IDENTITY();
RETURN @TheID
 

My personal thought on the matter is that you use the Return Values are for determining the execution health of the procedure you called, and that to get an ID value either from a record set or an OUTPUT parameter. This allows a separation of concerns and provides a way for an application to completely understand the intent of the procedure that executed. Lets jump to the code to see what I mean:

 
--drop table dbo.pizza
CREATE TABLE Pizza (pizzaid TINYINT IDENTITY(1,1), pizzasize VARCHAR(10), numberoftoppings TINYINT, pizzaname VARCHAR(40) )
 
CREATE PROCEDURE dbo.asp_InsertNewPizza
	(@PizzaSize VARCHAR(10),@NumberofToppings TINYINT,@pizzaName VARCHAR(40),@PizzaID TINYINT OUTPUT
AS
 
BEGIN TRY
INSERT INTO dbo.Pizza (pizzaSize, NumberOftoppings,pizzaname)
VALUES (@PizzaSize, @NumberofToppings, @pizzaName)
 
SELECT @PizzaID = SCOPE_IDENTITY();
RETURN 0;
END Try
BEGIN Catch
	RETURN -1
END Catch
GO
 
DECLARE @InsertedID TINYINT, @RunOK INT;
EXEC @RunOK = dbo.asp_InsertNewPizza @pizzaSize = 'Medium',@NumberOfToppings=2, @PizzaName='Medium Cheese and Sausage', @PizzaID=@InsertedID OUT
SELECT @RunOK, @InsertedID
 

So, when we run the above code, we get a 0 for the return value (@RunOK), and a 1 for the inserted ID. Had we received a -1 for the return value, we would know that something was jacked up and not to trust the value of the OUTPUT param. Of course, like many demo's, this is an incredibly simple demo just to show proof of concept. We could go a huge discourse about handling failures inside the procedure, application level transactions, or any number of scenarios that might result in a "Fine, in that case don't do the above" but all that does is take away from the general message of separation of concerns and being able to quickly convey what actually happened. Lets look at this slight alteration to our operation:

 
CREATE PROCEDURE dbo.asp_InsertNewPizza
	(@PizzaSize VARCHAR(10),@NumberofToppings TINYINT,@pizzaName VARCHAR(40),@PizzaID TINYINT OUTPUT
AS
 
IF EXISTS (SELECT pizzaID FROM dbo.Pizza WHERE pizzaname = @pizzaName)
BEGIN
	SET @PizzaID = NULL;
	RETURN 1;
END
ELSE
BEGIN TRY
	INSERT INTO dbo.Pizza (pizzaSize, NumberOftoppings,pizzaname)
	VALUES (@PizzaSize, @NumberofToppings, @pizzaName)
 
	SELECT @PizzaID = SCOPE_IDENTITY();
	RETURN 0;
END TRY
BEGIN CATCH
	RETURN -1
END CATCH
 
GO
 

Without catching the Return Values... if we get a NULL for the identity, how do we know if it is from an issue with inserting the values or an issue with a pizza name already existing? If we capture the ReturnValue and see that it is >= 0 we know that it wasn't from an error, but that a certain business rule had been met.

Well this is all good and well to us the database people, but what do we do if the app devs say it is too hard to won't help them. Here is just a basic call in C# with ADO.NET and how it could be used:

 
SqlCommand databaseCMD = new SqlCommand ("asp_InsertNewPizza", pizzaConn);
 
databaseCMD.CommandType = CommandType.StoredProcedure;
 
SqlParameter PizzaSize = databaseCMD.Parameters.Add ("@pizzaSize", SqlDbType.VarChar, 10);
PizzaSize.Direction = ParameterDirection.Input;
SqlParameter NumberOfToppings = databaseCMD.Parameters.Add ("@NumberOfToppings", SqlDbType.TinyInt);
NumberOfToppings.Direction = ParameterDirection.Input;
SqlParameter PizzaName = databaseCMD.Parameters.Add ("@PizzaName", SqlDbType.VarChar, 40);
PizzaName.Direction = ParameterDirection.Input;
 
SqlParameter PizzaID = databaseCMD.Parameters.Add ("@PizzaID", SqlDbType.TinyInt);
PizzaID.Direction = ParameterDirection.Output ;
SqlParameter ReturnValue = databaseCMD.Parameters.Add ("RetVal", SqlDbType.Int);
RetVal.Direction = ParameterDirection.ReturnValue;
 
int pizzaID;
pizzaConn.Open();
 
pizzaID =databaseCMD.ExecuteScalar ().ToString() ;
 
if (ReturnValue == 0){
ResponseWrite("The Pizza was added. The pizzaID is : " + pizzaID.ToString() );
ResponseWreite("Return Value: " + RetVal.Value);
}
else if{ ReturnValue == 1) {
ResponseWrite("Sorry friend, that pizza name already exists.");
ResponseWreite("Return Value: " + RetVal.Value);
}
else { FreakOut(); }
 

Virtual Victory – AWS gives out a year of usage

Amazon has just done something amazing... Amazingness... They are essentially doing a conditional year free of their services. While I am a huge fan of VMWare, what Amazon is offering is something completely unmatched by VCloud or MS Azure.  (Please see Craig's comment as this claim has the possibility of being debunked. Thanks Craig!)

So what can I do with this you might ask? Here are some ideas: (I am doing these off the top of my head so feel free to criticize them by suggesting a better ideas)

1. If you have never played with Linux/Ruby or anything else, here is a good/free way to to dabble without the 'will this mess up my system' fear. Microsoft is continuing their adoption of features from these platforms and it can only benefit you to see these ideas they are using from Ruby(gems) or Apache(modularization). The more you know, the more you grow.

2. NOSQL? What is a NOSQL? Want a simple way to learn about NOSQL? Then try out Amazon's simpleDB. They have various demos and such you can run and play with and start trying to see what all the hype is about. Remember fellow fearful DBA's, NOSQL stands for "Not Only SQL" not for No - SQL as in SQL Void.

3. Embrace the awesome of Amazon S3. Ever wonder why Dropbox/JungleDisk and various other companies can give you so much storage for so cheap? Amazon S3 is pretty much the answer to that question. I typically use the cheapest hosting plans possible and S3 as a CDN to smoke/mirror performance.

4. SQL Server testing - Microsoft provides trial software for pretty much all their projects. If you are curious about ... say playing with some feature of replication or maybe wondering about online indexing ... Fire up 1 or more instances of EC2 and actually try it. You no longer have an excuse to put it off.

5. .Net testing or running a service - With shared hosting, I don't have a way of keeping an ongoing on demand service per say... So why not have a micro instance where you can deploy services and such. For example, say you need to add SMS services to your app.... Sign up for Twilio, create a little service, throw it on your free AWS EC2 instance and poll every hour or so. Most shared hosting sites don't really support SQL Server SSIS... so why not have an AWS EC2 instance that fires up at night and starts some SSIS packages?

Now, some of this isn't without a learning curve, but we are in the software development industry... if we can't navigate learning curves or don't even pursue learning to begin with we add more nails to our career coffin as well as start becoming the problem.

Installing MongoDB on Ubuntu

First step to getting started with toying around with NoSQL is getting it running. Currently, MongoDB is my choice in the land of learning. To get it going on my VM's I do the following after my Ubuntu server is installed.

Navigate to: /etc/apt/sources.list
Then: sudo pico sources.list
In your sources, add the following line: deb http://downloads.mongodb.org/distros/ubuntu 10.4 10gen
Save/exit the file, then add the key: sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10

Once that is done, you're ready to update and install.
Update: sudo apt-get update
Install: sudo apt-get install mongodb-stable

Well now what?
If you need to configure MongoDB further, then navigate to: /etc/mongodb.conf
If you need to add/remove/mess with the database, it lives in: /var/lib/mongodb
Want the logs? They are here: /var/log/mongodb/ (.log extension)

Getting Started with SQL CLR Part 2

As you start playing with SQL CLR, you learn pretty quick that the built in namespaces can be a little handicapping. You can skip this by creating an 'unsafe' assembly in your database. In the following demo, we're going to load a 3rd party .library for XMPP messaging into our SQL Server instance, and use a stored procedure to send XMPP messages.

The XMPP library I'll be using is agsXMPP. As you'll see in my code below, I tend to just use a common shared library folder so I can house various libraries and it makes it easier from an organizational standpoint to cram them into the database. I use the following code to 'cram' the library into my instance so I'll be able to use it in a CLR procedure:

 
CREATE ASSEMBLY [agsXMPP]
FROM 'C:\Subversion\Shared Libs\agsxmpp\agsXMPP.dll'
WITH PERMISSION_SET = UNSAFE
GO
 

Once that is out of the way, we can roll our CLR proc, deploy and message to our hearts content (or until the network drops)

 
using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
 
using agsXMPP;
using agsXMPP.protocol.client;
// yes,  domains and names have been changed to protect the innocent servers.
public partial class StoredProcedures
{
	[Microsoft.SqlServer.Server.SqlProcedure]
	public static void clr_SendXMPP()
	{
		XmppClientConnection xmpp;
		xmpp = new XmppClientConnection();
		xmpp.AutoPresence = true;
 
		xmpp.AutoResolveConnectServer = true;
		xmpp.Port = 5222;
		xmpp.UseSSL = false;
		xmpp.Server = "xmpp.datachomp.com";
		xmpp.Username = "SQLAlert";
		xmpp.Password = "SqlClr";
 
		xmpp.Open();
 
		xmpp.OnLogin += delegate(object o) { xmpp.Send(new Message(new Jid("Gatir@xmpp.datachomp.com"), MessageType.chat, "This really should be a variable yes?")); };
	}
};
 

Getting Started with SQL CLR Part 1

Start: Anyone who has done scripting tasks in SSIS knows how valuable being able to leverage .NET can be when you are building data solutions. Based on a recent presentation, by Tim Mitchell, I was encouraged to extend .NET past SSIS and start exploring SQL CLR. Let's get started:

First, we need to enable CLR on our database:

sp_configure 'clr enabled', 1
GO
RECONFIGURE
GO

Next, we fire up Visual Studio and create a Database project. C# is my language of choice so I choose it:
CreateProject

After that, it will likely ask you for some DB credentials, much like creating a datasource when you are doing other .net applications. Like other .NET solutions, you can right click on your project, go to properties and change your connection on the 'database' tab. Once you get your project up, right click on add a new item, and select stored procedure. Give it a name bearing in mind that the name you give it will be what the procedure gets deployed as. Once created you will see that VS takes care of a lot of the plumbing for you:

 
using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
 
public partial class StoredProcedures
{
	[Microsoft.SqlServer.Server.SqlProcedure]
	public static void clr_Chomp()
	{
		// Put your code here
	}
};
 

Let's throw in some code:

 
	[Microsoft.SqlServer.Server.SqlProcedure]
	public static void clr_Chomp(out string Chomps)
	{
		Chomps = "Chomp Chomp Chomp Chomp";
	}
 

What did we add? We basically declared an output parameter (out string chomps), and then said that variable is just going to return the text 'Chomp' X 4.
Compile, and deploy. Gotcha - For those of us that use VS 2010 our default framework is 4.0 .... this won't jive in SQL Server so you will get a successful build, but a failed deployment. Follow the yellow brick errors and change your target framework down to 3.5 (if deploying to SQL Server 2008). Once that is done, it is time to see this guy in action!!!
Make sure you are in the proper database and run it:

 
DECLARE @words VARCHAR(25)
EXEC dbo.clr_Chomp @Chomps = @words OUTPUT
SELECT @words
 

and it should spit out some Chomps for you. If it worked, go reward yourself with some M&M's. If it didn't work for you then:

 
GOTO START;
 

In Part 2, I will begin leaving the playground of basic examples and start showing real practical implementations of SQLCLR.
In Part 3, I will likely look at performance and some other items to help round out the series.

DatabaseMail Time to clean house

So hopefully you have moved to databasemail and started enjoying its many benefits. Remember that cool benefit of it logging all the emails for you and such? Well that comes at a price, and the price is an ever growing MSDB. So, you should consider setting a retention date for your servers and create or add the following to your maintenance packages/jobs.

This script will clear out the mail items and the log based on the @Retention_Days variable:

 
DECLARE @Retention_days SMALLINT
	, @Delete_Date DATETIME
 
SET @Retention_Days = -90  -- 3 months
SET @Delete_Date = DATEADD(d, @Retention_Days, GETDATE())
 
EXECUTE msdb.dbo.sysmail_delete_mailitems_sp @sent_before = @Delete_Date
EXECUTE msdb.dbo.sysmail_delete_log_sp @logged_before = @Delete_Date
 

SSIS and the embedded delimiter fail

Simple CSV file with some embedded delimiters. No big deal in SQL 2000 DTS right? Well, it's a huge deal in SQL Server 2005 and SQL Server 2005 SSIS. Of course, it is a bit of a mystery while they keep removing functionality but that isn't the point of this post. The point of this post is to show a solution...

The solution comes by way of the Microsoft SQL Server Community Samples: Integration Services (yes, it is a long name for 'Look dudes! We are giving you a work around!' but it works non the less).

http://sqlsrvintegrationsrv.codeplex.com/

I tried the delimited file reader in a variety of scenarios and it passed with flying colors. How nice it is to be able do the same imports that were effortless in SQL 2000.