Pushing Data From Excel To Power BI Using Streaming Datasets

Very interesting Article!

Chris Webb's BI Blog

One Power BI feature that almost passed me by (because it was released in August while I was on holiday) was the ability to create streaming datasets in the Power BI web app and push data to them via the Power BI REST API. This blog post has the announcement:
https://powerbi.microsoft.com/en-us/blog/real-time-in-no-time-with-power-bi/
The documentation is here:
https://powerbi.microsoft.com/en-us/documentation/powerbi-service-real-time-streaming/
And Charles Sterling has an example of how to use it with Flow and PowerApps here:
https://blogs.msdn.microsoft.com/charles_sterling/2016/10/17/how-to-create-and-customize-applications-with-powerapps-with-laura-onu-webinar-1020-10am-pst/

However, when I played around with this I found there were a few things that were either confusing or not properly documented, so I thought it would be useful to give an example of how to use this functionality to automatically synch data from a table in Excel to Power BI using a Power Query query.

Creating the streaming dataset in Power BI

Imagine that you have a table called Sales in an Excel workbook on your…

View original post 1.143 altre parole

Columnstore Indexes SQL 2016

The columnstore index is the standard for storing and querying large data warehousing fact tables. It uses column-based data storage and query processing to achieve up to 10x query performance gains in your data warehouse over traditional row-oriented storage, and up to 10x data compression over the uncompressed data size. Beginning with SQL Server 2016, columnstore indexes enable operational analytics, the ability to run performant real-time analytics on a transactional workload.

These are key terms and concepts are associated with columnstore indexes.

columnstore

A columnstore is data that is logically organized as a table with rows and columns, and physically stored in a column-wise data format.

rowstore

A rowstore is data that is logically organized as a table with rows and columns, and then physically stored in a row-wise data format. This has been the traditional way to store relational table data. In SQL Server, rowstore refers to table where the underlying data storage format is a heap, a clustered index, or a memory-optimized table.

column segment

A column segment is a column of data from within the rowgroup.

Delete bitmap:

cancellations are logical, are erased when you recalculate

Delta rowgroups:

inserted rows that have yet to be compressed

Clustered Columnstore Index

They can even be created on tables in memory that does not. it’s handled as all other indices. Through the Delay feature is possible to force the delay after a certain time instead of reaching the number of rows. You can also Filter data and exclude  certain indices values (eg the deleted parts)

How to write it:

— Create a clustered columnstore index on disk-based table.
CREATE CLUSTERED COLUMNSTORE INDEX index_name
ON [database_name. [schema_name ] . | schema_name . ] table_name
[ WITH ( < with_option> [ ,…n ] ) ]
[ ON <on_option> ]
[ ; ]

— create a durable (data will be persisted) memory-optimized table
— two of the columns are indexed
CREATE TABLE dbo.ShoppingCart (
ShoppingCartId INT IDENTITY(1,1) PRIMARY KEY NONCLUSTERED,
UserId INT NOT NULL INDEX ix_UserId NONCLUSTERED HASH WITH (BUCKET_COUNT=1000000),
CreatedDate DATETIME2 NOT NULL,
TotalPrice MONEY

INDEX ShoppingCartId CLUSTERED COLUMNSTORE
) WITH (MEMORY_OPTIMIZED=ON)
GO

Here there are the principal differences between sql 2014 and sql 2016 :

Columnstore Index Feature SQL Server 2014 SQL Server 2016
Batch execution for multi-threaded queries yes yes
Batch execution for single-threaded queries yes
Archival compression option. yes yes
Snapshot isolation and read-committed snapshot isolation yes
Specify columnstore index when creating a table. yes
AlwaysOn supports columnstore indexes. yes yes
AlwaysOn readable secondary supports read-only nonclustered columnstore index yes yes
AlwaysOn readable secondary supports updateable columnstore indexes. yes
Read-only nonclustered columnstore index on heap or btree. yes yes*
Updateable nonclustered columnstore index on heap or btree yes
Additional btree indexes allowed on a heap or btree that has a nonclustered columnstore index. yes yes
Updateable clustered columnstore index. yes yes
Btree index on a clustered columnstore index. yes
Columnstore index on a memory-optimized table. yes
Nonclustered columnstore index definition supports using a filtered condition. yes
Compression delay option for columnstore indexes in CREATE TABLE and ALTER TABLE. yes