Automatized backup of MSSQL Databases using PowerShell

First of all I want to share with you some initial knowledge I gathered after jumping in the making of my first 40+ lines backup-script in this environment (MS,PowerShell,MSSql ..) and having some basic issues in the beginning -  testing my tiny script.
The default config of a powershell environment doesn’t let you run unsigned scripts..
The security settings built into Windows PowerShell include a so called the “execution policy;” – this policy determines how (or if) PowerShell runs scripts. By default, PowerShell’s execution policy is set to Restricted; that means that scripts – including those you write yourself – won’t run. I can tell you that Before I found this information it wasn’t a nice and motivating first step in my PowerShell “scripting career” . :)
After a short research at MS Technet I learned about this useful commands :

> Get-ExecutionPolicy ( – verify the settings for your execution – the default has a “AllSigned” value )
> Set-ExecutionPolicy  [ AllSigned | RemoteSigned | Unrestricted ]
> Get-Help About_Signing  ( – to learn more about “signing scripts” for PS )

RemoteSigned settings is the best choice if we want to run our scripts only on the local machine.
I choose for my scripting the folder  c:\scripts
If we want to test our script in the development process , the best way is to open a PowerShell command line running PowerShell.exe and run from there the scripts for catching the output, errors..etc :
……………………………………
PS C:\scripts> .\ mssql_backup.ps1
In the command line above we use “.\” notation for executing a PS1 file. ( PS1 being the file extension for Windows PowerShell scripts )

To do a SQL Server backup in SQL Server, we will need to use the SQL Server Management Objects, known as SMO, and specifically we need the SqlBackup method.
SMO is an object model for SQL Server and its configuration settings. SMO-based applications use .NET Framework languages to program against this in-memory object model, rather than sending Transact-SQL (T-SQL) commands to SQL Server to do so.
In SQL Server 2008 + , we will need to load Microsoft.SqlServer.SmoExtended assembly otherwise, we get the following error:
” Cannot find type [Microsoft.SqlServer.Management.Smo.Backup]: make sure the assembly containing this type is loaded. “

Here is the a commented version of the < mssql_backup.ps1 >  script :

To schedule the actions – recurring backup – on a daily or weekly basis you may want to run this script from a Windows Server TaskScheduler.

Because  we need to set PowerShell.exe as executable in the Action Pane of TaskScheduler with the Unrestricted or RemoteSigned execution enviroment policy and the script will be only an argument of this,  the best way is to create a batch-file and specify this as the “Program/script” of the TaskScheduler’s “Action Tab”   with the following content:
————————————————-  db_backup.cmd content: ———————————-—-
powershell.exe  -ExecutionPolicy RemoteSigned  -Noninteractive  -Noprofile  -command “& {C:\script\ mssql_backup.ps1 }”
——————————————————————————————————————————

Posted in System Admin / Scripting | 1 Comment

Scheduled FTP Sync & Maintenance Script

Our tester teams needed a solution for downloading large files (software builds,new releases, iso-images..)  in background from remote servers located in the USA.  Usually the testers starts their work (installing, testing)  after this software packages  are downloaded without errors.  In working hours they had a smaller bandwidth and many interruptions , so the manual download and verification of these large files was  a real wasting of time for us and for the tester guys too.

I wrote a little bash script to do this job for us:  mget.sh (inplemented in three steps).

1. Verifying the content of the remote ftp folder  &  Downloading the files if they exist.
2. Checking in a cycle if downloaded files properties matches the files on the FTP folder:
- if a particular file is corrupt because of a partial download, then it is downloaded again
- if the file size and type matches , then the remote file is deleted from the upload folder
3. Logging all the actions  made by the script (warnings too) in syslog format.

For the logging feature to work the mget.sh script must be invoked by root privileges. After you customize the script with the right variables ( hostname, remote user, password, logfiles, folders etc.)  you will need to make a new crontab entry for the root user to automatize this task In case we don’t want to allow or we don’t have root access to the Linux box, then make sure that the local download folder is accessible for read and write by our User (in this case jaky ) who will launch the mget.sh script.

Let’s say that we put our version of the script in jaky‘s home folder [ /home/jaky/mget.sh ] .

The entry in the  /etc/crontab

# m  h  dom mon dow    user      command  that will run @ 1:30 AM every day
30  1   *      *      *       root       /home/jaky/mget.sh

The logging is made in /var/log/mget.log file with date+time stamps for each operation.

You’ll find my commented version of the script here:

http://sftp.codespring.ro/~jaky/mget.sh

.. and a sample of the log file generated by the script:

http://sftp.codespring.ro/~jaky/mget.log

Have a nice day.. & Good Luck !

Posted in System Admin / Scripting | Leave a comment

The second most annoying Excel thing

I think we can say that Microsoft Office is a heavyweight product. By Office 2007 it is also a mature one, with hundreds of features. This post reflects the opinion of more colleagues, each one faced these problems on its own (so not just a single person is annoyed).

It is unbelievable (I say it again: unbelievable), that a program like MS Excel suffers from this childish problem: if you open more files, it opens them all into the same window, doing some tricks to appear as if more Excels are running. Maybe a consequence of this (or maybe THE most annoying thing) is that Excel can’t open two files having the same name. Note, that I’m talking about Excel 2007, which has been improved during tens of years.

Try this (I describe it as it is seen under WinXP): open one excel file, then another. You see two Excels in the taskbar. Begin to switch between them with Alt-Tab – there are three Excel icons (from which two brings up the same window).
Resize the first window to cover half of the screen, then switch to the second window. Guess what: it appears in the same half-covering window. This is when you know, that something stinks. One big issue is, that you can’t put the two windows side-by-side, only arrange them inside the one (real) Excel window.

Many people don’t realize this, probably they never needed to see two excel tables side-by-side. That becomes more annoying if you use two monitors set up as two desktops.

And the bug with the file names is definitely unacceptable. What, you can’t open two different files (from two different folders) if they happen to have the same name? How comes that Word 2007 has no such problem? I just can’t think of any reason.

UPDATE: a friend told me about the workaround for opening files into separate Excel windows. (I haven’t tested if this solves the “can’t open two files with the same name” issue, but there is a chance.)
The trick is, that one should not open the files by double clicking on them, but: launch Excel from the Start menu and open a file using Excel’s menu/toolbar. Then launch a new Excel from the Start menu, and open the second file using the second Excel’s menu/toolbar. Now you have two really distinct Excels running.

Posted in Uncategorized | Tagged | 1 Comment

How can a developer be fooled by SQL NULL?

The fact: NULL values in SQL have different rules than in programming languages.
The reality: these rules can be easily forgotten (or not even supposed to exist) by developers who don’t use SQL too often.

This was our case too (described below). Of course, once discovered, one will remember it (for a while), and if you google it, you will find plenty of examples and explanations. The only problem is that when you quickly write an SQL statement, you don’t even know that an alarm bell should ring.

So let’s say it out loud: in SQL the NULL value is not equal to any other value, not even to other NULL values. Everybody knows that NULL is not equal with an empty string, but NULL != NULL to be true, that is something scary (even if SQL folks have an explanation for it). And this causes complications.

Our query looked similar to this:

SELECT * FROM Containers WHERE ParentID NOT IN (SELECT ContainerID FROM Clients)

and we expected all containers whoose “parent” is not associated with any client. You can guess, ParentID is a nullable field (because it’s optional and it was designed so).
What we got was all containers who had parents and those parents were not asociated with any client. Those who had NULL for ParentID were not returned. A better query is:

SELECT * FROM Containers WHERE ParentID IS NULL OR ParentID NOT IN (SELECT ContainerID FROM Clients)

Note: the COALESCE function can also be used to test for some values including NULLs.
As we looked at the original query, our mind said: “clients are not asociated with NULL ContainerIDs, which means that NULL values are never returned by the inner SELECT, so the NOT IN condition is enough”. The random manual tests were also not suspicious, because of some unfortunate coincidences not discussed here.

More complicated situations exist in the wild – think of JOIN, GROUP BY, UNIQUE, etc. You can find help if you start searching.
The meaning of the story is: whenever you use a nullable column in a statement, always do special tests with NULL values.

Posted in Windows development | Tagged | Leave a comment

Will Industrial Applications Go Mobile?

Smartphones silently enter the realm of laptops and minicomputers. Having in mind that mobile phones are 100% a consumer device, a vividly debated question is whereas Smartphones can and will enter the enterprise and above all, the factory plant (?).

Posted in Android development | Leave a comment