Saturday, July 02, 2016

SQL Server Management Studio – July 2016 Release

The first monthly update release of SSMS following the release of SQL Server 2016 is now available. The update added the feature I personally was waiting for.  The following are few of the highlights that was mentioned during the announcement. 

  • Support for Azure SQL Data Warehouse in SSMS. This is good news indeed. 
  • Support for PowerShell script generation in the Always Encrypted wizard
  • Improved connection times to Azure SQL databases
  • New ‘Backup to URL’ dialog to support the creation of Azure storage credentials for SQL Server 2016 database backups
  • New Restore dialog to streamline restoring a SQL Server 2016 database backup from the Microsoft Azure storage service.

and some bug fixes too. I cannot wait to play with Azure SQL Data Warehouse using the new SSMS release

Monday, June 13, 2016

New Sample databases for SQL 2016

SQL 2016 is in market for just under two weeks and Microsoft released a new sample databases(WideWorldImporters and WideWorldImportersDW) Those sample database can be used to evaluate the new features of SQL 2016 such as
•    Polybase
•    Query store
•    In-Meomory OLTP
•    Updatable column store index (both clustered and non-clustered)
•    Native JSON support
•    Security (Dynamic data masking, row level security and Always encrypted)
•    Temporal database and tables

There are also few scripts and applicaiton that was released with new sample databases to test out the new features. Enjoy

Monday, June 06, 2016

Azure SQL Data Warehouse

Azure SQL Data Warehouse is the new SaaS offering from Microsoft. It is still in preview state and I had a chance to set up and play with it. It looks very promising.
So, what is Azure SQL Data Warehouse? Microsoft defines Azure SQL Data warehouse as follows: “Azure SQL Data Warehouse is a cloud-based, scale-out database capable of processing massive volumes of data - both relational and non-relational. Built on massively parallel processing (MPP) architecture, SQL Data Warehouse can handle your enterprise workload”.  FlyData [1] that compares Azure data warehouse vs Amazon Redshift defined it as “a distributed and enterprise level database capable of handling large amounts of relational and non-relational data”

Historically, those type of MPP are only available for enterprises that can afford to fork out a lot of money to buy appliance upfront. I believe what Microsoft done is great in trying to bring this powerful massively parallel processing architecture to all of us and now we are in position to investigate, experiment and perform cost benefit analysis and see if it can benefit the business under our domain. I believe the SaaS offering of the product will not discriminate against any size of companies whether small, medium or large. It is also a great opportunity for data scientists to experiment their modelling and avoid hassle of initial investment cost.   Whether this will be a game changer, we won’t say it for sure now but time will tell.
The abstraction and separation of compute from azure data warehouse is the one I personally like very much.  I believe the flexibility to dynamically deploy, grow, shrink and pause is indeed another feature.  In this article, I will look at two aspects of the product; architecture and pricing

Architecture
There are four layer that defines azure data warehouse.  There are disagreement if it is four or three or even two layers that defines Azure data warehouse architecture.  I wouldn't be drawn into the layering as they all talk about the same thing.   Those layers are DMS, storage, control and compute nodes.  The image below (source: Microsoft) gives you a visual architecture of Azure data warehouse
  1. Data Movement services is one of the layer that spans across both control node and compute nodes. It is a windows service that coordinates and/or manages data movement between different nodes.
  2. Control node is the entry point for all applications. It is like connecting to an instance of SQL server. The application that connects to ADW (Azure data warehouse) doesn’t know the underlying compute nodes nor will be able to connect to any of the compute nodes. The compute nodes are abstracted from our application. The control node is responsible in distributing our requests to underlying compute nodes
  3. Compute nodes are the engine of our computing power hence called compute nodes. When you issue a big query the first node you hit will be control node which will then distribute the query to compute node. Once the query run on compute nodes it is then the work of control nodes that will aggregate the results from all the compute node and return’s to our application.
  4. One of the great achievements of Azure data warehouse is the separation of dependence of storage layer from the compute node when scaling up. You should be able to increase your storage independent of your compute node.  The storage layer and compute nodes scales separately. The data will be stored in Azure Blob storage. The compute nodes are directly interacting with the storage layer when reading or writing data.

Charging
Charging is based on number of DWU. DWU (data warehouse unit) represents the amount of compute resources used at any given time.  The more DWU you use,  the faster your query runs as it will be executed on more distributed CPU and memory.  It will take some time and experience to understand the trade-off and/or optimum DWU to pricing to run your query to be completed within the time-frame you planned or anticipated.
 As per information I have got at current time, the charge is per minute. For instance, if you used 100 DWU for 30 minute you will be charged for 30 minutes instead of per hour pricing that is shown for compute capabilities. Currently, the minimum DWU you can configure from the portal is 100 DWU (with a cost of £0.43/hour) with increment of 100 up to a maximum of 2000 DWU (with a cost of £8.55/hour). If you pause you will not be charged.

 References
  1. https://azure.microsoft.com/en-gb/services/sql-data-warehouse/
  2. (https://azure.microsoft.com/en-gb/documentation/articles/sql-data-warehouse-get-started-provision/
  3. https://www.simple-talk.com/cloud/cloud-data/azure-sql-data-warehouse/



Thursday, May 05, 2016

SQL Server 2016 general availability

It is now official that SQL Server 2016 will be out on 1st of June 2016. As some of you may know Microsoft dropped the BI edition in it's 2016 release. Looks like there is now Enterprise, Standard, Express and Developer Edition. SQL Server developer edition is now a free edition providing the full feature set of SQL server Enterprise and can be used only for development and testing.

Friday, March 11, 2016

Microsoft SQL Server Announcement


  • Microsoft has indicated that support for SQL Server 2005 will end as of April 12th, 2016.

·         No access to critical security updates, opening the potential for business interruptions
·         Higher maintenance costs for maintaining legacy servers, firewalls, intrusion systems mean increased costs for the business
·         Compliance concerns – as support ends, organizations will likely fail to meet regulatory standards compliance 

  • With the introduction of SQL Server 2016, Microsoft will no longer offer the Business Intelligence edition. Beyond this change, they don’t anticipate any changes in the licensing model with the release of SQL Server 2016.

Tuesday, March 01, 2016

Instance wide script to add user (the 4th way)

My new script to add user to all databases in an instance published on http://www.sqlservercentral.com/ and can be found Instance wide script to add user (the 4th way). The script can be modified and can be used for various purpose.

Monday, February 15, 2016

SQL Server 2016 release date

Looking at the history of the last Microsoft releases, it looks like the release of SQL 2016 will be either March or April.. Hope my prediction is right:)

SQL Server Release_Date
SQL Server 2016 2016-04/03-??
SQL Server 2014 01/04/2014
SQL Server 2012 06/03/2012
SQL Server 2008 R2 21/04/2010
SQL Server 2008  2008-08-07
SQL Server 2005 07/11/2005

Thursday, February 11, 2016

Gartner positions Microsoft as a leader in BI and Analytics Platforms

For the first time, Microsoft is placed furthest in vision within the Leaders quadrant.
Gartner_MQ_BI_AP

Tuesday, December 29, 2015

sp_server_diagnostics

sp_server_diagnostics sp is used to capture quick  diagnostic data and health information about your SQL Server to detect potential issues. It is a must run sp that I start using recently as a quick check on your SQL server stack. You can even run this procedure when using DAC.   For more detailed info visit an article on www.mssqltips.com.

Tuesday, December 22, 2015

The new Community Tech Preview of SQL Server 2016, CTP3

I want to point out two features that is either added or improved with CTP3 release of SQL Server 2016.

New addition :  You will now be able to setup transnational replication from in premise to Azure SQL (Microsoft's cloud based version of SQL Server). this will for sure make it easier to migrate data to cloud with little or no downtime

 Improvement: Integration of R (Revolution Analytics)  package that will allow you to run R analytics in the database without pulling out data for analysis on a separate location or machine.  This preview also added the ability to execute R scripts via system stored procedure to query and pass data to R

Wednesday, December 16, 2015

Azure data warehouse architecture

SQL Data Warehouse Architecture
Diagram source : https://azure.microsoft.com/en-gb/documentation/articles/sql-data-warehouse-overview-what-is/

Azure SQL Data Warehouse is a distributed database capable of processing big data (both relational and non-relational). Microsoft claims that it is the first cloud based data warehouse that combines SQL capabilities with scalability at its core.



Clustered Columnstore Tables are now default on Azure SQL Data Warehouse platform.

For more info visit: https://azure.microsoft.com/en-us/updates/clustered-columnstore-tables-are-default-in-azure-sql-data-warehouse/

Thursday, October 01, 2015

SQL 2016 CTP 2.4 out now

The SQL server team at Microsoft announced availability of SQL Server 2016 CTP 2.4 yesterday. Main changes from previous CTP are:

1. Operational Analytics and In-Memory Data Warehouse
2. x86 server deprecation
3. SQL Server Smart Maintenance Plans
4. Tempdb Scalability improvement
5. SQL Server Integration Services (SSIS) improvements
6. SQL Server Reporting Services (SSRS) improvements


Tuesday, August 04, 2015

SQL Server 2016 CTP 2.2 is out

Microsoft released SQL Server 2016 CTP 2.2. A bit faster than I thought for this version to be out.  For detailed list of improvements on previous CTP and download go to  http://blogs.technet.com/b/dataplatforminsider/archive/2015/07/28/sql-server-2016-community-technology-preview-2-2-is-available.aspx

Tuesday, July 14, 2015

Native JSON support (SQL 2016)

The new release of SQL server has taken further steps and added native JSON support. JSON lately become one of important part of any developer’s toolkit. Recently it become clear that sites are start sharing data using JSON in addition to RSS feeds. JSON feeds can be loaded asynchronously much more easily than XML/RSS

JSON (JavaScript Object Notation) outputs human-readable collection of data that consists name/value pairs with individual values separated by comma. Objects are containers of one or more name/value pairs and are contained within curly brackets as shown below.


For instance it will store data as

var Product = {
        "ProductId" : "24",
        "ProductName" : "Lego Jurassic World",
        "Price" : "$30.00"
        };

Appending FOR JSON AUTO to a standard SELECT statement returns the result set in a JSON format. JSON arrays can contain multiple objects, and arrays are contained within square brackets.  Note here JSON is not treated as another data type in SQL server like XML.


Let us now create a table in SQL server and insert some data and output the result.

create table products (productId int primary key not null, productName nvarchar(100), Price money)
go
insert into products(productId, productName, Price)
values (1, 'Terraria Xbox 360', 12),
       (2, 'Batman Arkham Origins',6),
          (3, 'FIFA 16', 40)
go
select productId, ProductName, Price from [dbo].[Products]
FOR JSON PATH , ROOT ('PRODUCTS')


The result of the above select statement is.

{"PRODUCTS":[{"productId":1,"ProductName":"Terraria Xbox 360","Price":12.0000},
                 {"productId":2,"ProductName":"Batman Arkham Origins","Price":6.0000},
                 {"productId":3,"ProductName":"FIFA 16","Price":40.0000}]}


 A simple T-SQL construct with “FOR JSON PATH” outputs JSON output that can be feed to web site.

Thursday, July 02, 2015

Azure SQL Data Warehouse - New Cloud offering



Microsoft launched a new cloud based data warehousing services with 21 partners on 24th of June. Microsoft announced this offerings during build developer conference in Aril in San Francisco (http://venturebeat.com/2015/04/29/microsoft-announces-azure-sql-data-warehouse-and-azure-data-lake-in-preview/)


As usual Microsoft designed the SQL data warehouse for databases in the range of 5TB to 10TB. 
Well, that is way too small. As this a limited public preview it will definitely change when it comes operational. 

I personally believe this is indeed a very good move both for Microsoft and for businesses that cannot afford to buy the PDW appliance.

Polybase (Now in Editions of 2016 as new feature)



What is Polybase?  Polybase is a new technology that integrates PDW (SQL server Parallel Data warehouse) with Hadoop Distributed File System.  It used to only work with PDW and most small to medium enterprise that doesn’t have the appliance weren’t able benefit. 

Polybase allows users to access/query non-relational data in Hadoop, blobs, files, data from either on premise or on the cloud and run analytics and BI on the data from within SQL server. It also provides a concept of Data Lake where you query the data from where it is stored and once you complete your query leave it where it was. This concept will facilitate analysis on Big Data from its current location and reduce the costs associated in moving the data. The following diagram is taken from Microsoft white paper and shows the interaction that you can have with different data sources from within SQL server when using PolyBase feature.

 


This feature is now available with Standalone enterprise version of SQL Server 2016. That is really a game changer.  I would love to play with this feature soon and hopefully post on my blog here…