Thursday, November 11, 2010

Playing with SQL Azure; part 3

Well, I think I'm done with SQL Azure. Not because I'm finding any kind of major problems with it, just becausethere's not enough that's different between regular SQL Server and SQL Azure!

The last thing I did here was create some additional tables and stored procedures in my SQL Azure database. I inserted to these tables via BCP, that was also fairly straightforward once I figured out a few gotchas (naming for the user and server).

I thought there would be so much to learn, but honestly, for a database developer (as opposed to a DBA) it appears to be 99% the same as SQL Server. Now, if I were doing a full fledged pilot project, with stress tests, investigating privacy issues, size issues etc, it would be different. But for just playing around for my own interest, I'm all done, and it's been fun!

Thursday, November 4, 2010

Playing with SQL Azure; part 2

I'm working my way through the SQL Azure tutorial (the one I found most useful was here).

Ran into some surprises - you can't use the "Use DatabaseName" syntax to switch from one database to another (you have to open a connection in another database).

Also, it appears that you can't reference another database on the same server. For instance, when running the following:

select * from TestSylvia..test1

I got this error:

Msg 40515, Level 16, State 1, Line 1  

Reference to database and/or server name in 'TestSylvia..test1' is not supported in this version of SQL Server.

From some searching online, it looks like that is being worked on.

So, there's some thing that are not yet possible. But what really struck me is the fact that overall, working on SQL Azure is so similar to working with an on-premises MS SQL Server database. I look forward to exploring further. And writing up a micro blog post on SQL Azure really motivates me to explore further.




Thursday, October 28, 2010

Playing with SQL Azure; part 1

For my own amusement and edification I've decided to learn about cloud computing by playing with SQL Azure. Within the last year, I've moved my personal information into the cloud (from Palm Pilot to a Nexus One Android phone with Google Docs, Remember the Milk, Google Contacts, etc) and am curious to see how that same switch could happen with databases.

Also, I just read the book The Big Switch: Rewiring the World, from Edison to Google. It's basically about how there's a very strong analogy between the move to cloud computing today and what happened about 100 years ago, when factories, instead of generating their own electricity, converted to buying their power from electrical plants. Interesting stuff. And I do believe that the movement to cloud computing is just as inevitable as the movement to buying power from dedicated electrical plants.

For my first step, I've set up a SQL Azure account (had to provide a credit card number, but they say they won't charge anything for three months). I've set up a database, and will be experimenting with it (downloading tools to access it, etc). My first task will probably be to go through the tutorials for SQL Azure.


Friday, October 1, 2010

Recursive common table expression 'TableName' does not contain a top-level UNION ALL operator.

I've started using CTEs (Common Table Expressions) a lot. I find they can make my code quite a bit easier to read. But just now, working on a SQL statement with a CTE, it took me a while to figure out why I was getting this error message:

Msg 252 
Recursive common table expression [TableName] does not contain a top-level UNION ALL operator.


It was a long and complex sql statement with multiple CTEs, which obscured the real problem. The little I found online wasn't helping either. I ended up going step by step, and stripping out everything from the SQL statement. Finally I ended up with something similar to this, which returns the error:

;With table1 AS
(
Select field1 from table1 where field1 = 1
)
Select * from table1



At this point it was obvious—you can't name your CTE with the same name as a table it's referencing. So the answer was to rename my CTE to something like this:

;With table1_filtered AS
(
Select field1 from table1 where field1 = 1
)
Select * from table1



There's other conditions that can cause this error to occur as well, but I didn't see this one described online anywhere. If you have this problem as well, feel free to comment.

Thursday, September 16, 2010

Are you running out of space on your development SQL server?

On a couple of our development SQL Server machines, we've run out of space fairly often. Lots of databases are being created frequently by multiple different developers, sometimes production backups are restored, or sample data is loaded. Lots of databases are dropped, too. But I believe there may be some bug in the drop database command, such that sometimes, it drops the databases, but the mdf and ndf files are left there.

This is not something I've been able to reproduce, but I've run into it enough that I have a routine that goes like this:

1. Run out of space on the development server
2. Check on the drive that's low on space for files over a certain size - usually about 5o gigs. In our environment, these are almost always SQL Server database files.
3. Use the sp_msforeachdb stored procedure to check which, if any, of these files is associated with an existing database. If you have a lot of databases, this is LOTS easier than running sp_helpdb for each individual database

exec sp_msforeachdb
'
SELECT DatabaseName = ''?'', * FROM ?.dbo.sysfiles
where filename like ''%InsertFileNameHere%''
'

For those that are NOT associated with a current database, you can delete them and clear up some space. Be careful here, and do any double-checking you feel is necessary.

Thursday, December 31, 2009

Easy Error Trapping When Using xp_cmdshell

Error handling can be tough when using xp_cmdshell. Before I learned the trick that I go over below, I could usually figure out in my code if an error had occured when I ran a command via xp_cmdshell. However, getting details about the error, or any output at all, was tricky and could involve parsing out files.

Before you start, please be aware that xp_cmdshell will be executed under the same security context as the SQL Server service, and can be a security problem in some environments.

This code uses the insert/execute syntax. If you've never used this before, it's a good idea to learn it. Basically, instead of inserting data into a table the normal way, you can insert the results of an execute statement - in this case the xp_cmdshell statement, like below:

Insert into XPCmdShellOutput 
Execute master..xp_cmdshell 'bcp tempdb..Employee out c:\temp\Employee.txt -c'


So, below is an easy, straightforward way to use xp_cmdshell to bcp out a table, and also see the output from the command. The same principles can be used for any other use of xp_cmdshell, and not only bcp.

set nocount on
use tempdb

if object_id('tempdb..Employee') is not null drop table Employee
if object_id('tempdb..XPCmdShellOutput') is not null drop table XPCmdShellOutput

-- Create the table that we need to extract from
create table Employee (EmployeeName varchar(20))
insert into Employee values ('John')

-- This table will be used to gather the output of xp_cmdshell
create table XPCmdShellOutput (OutputLine varchar(1000))

-- Show the output of xp_cmdshell when the directory does not exist
Insert into XPCmdShellOutput
Execute master..xp_cmdshell 'bcp tempdb..Employee out c:\DirectoryDoesNotExist\Employee.txt -c'
select 'Error when directory does not exist' = OutputLine from XPCmdShellOutput
delete from XPCmdShellOutput

-- Show output of xp_cmdshell when the table to be exported does not exist
Insert into XPCmdShellOutput
Execute master..xp_cmdshell 'bcp tempdb..Employee1 out c:\temp\Employee.txt -c'
select 'Error when table does not exist' = OutputLine from XPCmdShellOutput
delete from XPCmdShellOutput

-- Finally, successfully export the table!
Insert into XPCmdShellOutput
Execute master..xp_cmdshell 'bcp tempdb..Employee out c:\temp\Employee.txt -c'
select 'Successfully export the table!' = OutputLine from XPCmdShellOutput


When you run the above code, this will be the output (note that you may need to modify the c:\temp directory in your environment, and you need create table permissions in the tempdb database):


Error when directory does not exist:

OutputLine
Password:
SQLState = S1000, NativeError = 0
Error = [Microsoft][ODBC SQL Server Driver]Unable to open BCP host data-file
NULL


Error when table does not exist:

OutputLine
Password:
SQLState = S0002, NativeError = 208
Error = [Microsoft][ODBC SQL Server Driver][SQL Server]Invalid object name 'tempdb..Employee1'.
NULL


Successfully export the table!

OutputLine
Password:
NULL
Starting copy...
NULL
1 rows copied.
Network packet size (bytes): 4096
Clock Time (ms.): total 1 Avg 1 (1000.00 rows per sec.)
NULL


The output of the xp_cmdshell was inserted into the XPCmdShellOutput table every time we ran it, because we used the insert/execute syntax. Then, we select from XPCmdShellOutput to show what the output actually was. The first section shows the error when the directory does not exist. The second shows the error message when table name to be exported is misspelled. And the third shows a successful export.

This is sample code, and simplified to make it easier to understand. In working code, you would check the XPCmdShellOutput table for the string "Error". If the error string exists, then obviously an error occurred, and the details would have been stored in the XPCmdShellOutput table. Note that when the export is successful, you can also extract other information from the XPCmdShellOutput table - for instance, how many rows were copied out, and how long the export took.

When using xp_cmdshell with bcp, keep in mind that it will NOT recognize any temporary tables that were created. If you need to use temporary tables, they must be global temporary tables, prefixed with ## instead of #.
 

Tuesday, June 23, 2009

Renaming a Column in a Temp Table in SQL Server 2005 - Yes, You Can!

For whatever weird reason, you may need to create a temp table in one step, and then rename one of the columns. Perhaps you're creating the temp table in one stored procedure, and modifying it in another. I'm not going to tell you that's a silly thing to do, or you should just create it with the right names in the first place - I saw those kind of responses online, and it's very unhelpful! Sometimes you just need to do this type of thing.

It's not a straightforward thing to do, though. When you just run the below script that calls sp_rename:

if object_id('tempdb..#Test123') is not null drop table #Test123
create table #Test123 (field1 int)
exec sp_rename '#Test123.Field1', 'Field2', 'COLUMN'

...you get this error: "Either the parameter @objname is ambiguous or the claimed @objtype (COLUMN) is wrong."

The key is to call sp_rename from the tempdb, like so:

if object_id('tempdb..#Test123') is not null drop table #Test123
create table #Test123 (field1 int)
exec tempdb..sp_rename '#Test123.Field1', 'Field2', 'COLUMN'
select * from #Test123

And - success! Note that you'll still have problems referencing the specific fieldname that was renamed. I solved this by doing a select into another temp table. Also, the procedure you're doing this in will probably have to do a recompile - that wasn't a problem for me, either. Overall, it was a better solution than the alternatives.