Business Intelligence Blogs

View blogs by industry experts on topics such as SSAS, SSIS, SSRS, Power BI, Performance Tuning, Azure, Big Data and much more! You can also sign up to post your own business intelligence blog.

«October 2015»

Data Warehouse from the Ground Up at SQL Saturday Orlando, FL on Oct. 10th

SQL Saturday #442SQL Saturday #442 is upon us and yours truly will be presenting in Orlando, Florida on October 10th alongside Mitchell Pearson (b|t). The session is scheduled at 10:35 AM and will last until 11:35 AM. I’m very excited to be presenting at SQL Saturday Orlando this year as it’ll be my first presenting this session in person and my first time speaking at SQL Saturday Orlando! If you haven’t registered yet for this event, you need to do that. This event will be top notch!

My session is called Designing a Data Warehouse from the Ground Up. What if you could approach any business process in your organization and quickly design an effective and optimal dimensional model using a standardized step-by-step method? In this session I’ll discuss the steps required to design a unified dimensional model that is optimized for reporting and follows widely accepted best practices. We’ll also discuss how the design of our dimensional model affects a SQL Server Analysis Services solution and how the choices we make during the data warehouse design phase can make or break our SSAS cubes. You may remember that I did this session a while back for Pragmatic Works via webinar. I’ll be doing the same session at SQL Saturday Orlando but on-prem! ;)

So get signed up for this event now! It’s only 11 days away!

Read more

Create Date Dimension with Fiscal and Time

Here are three scripts that create and Date and Time Dimension and can add the fiscal columns too. First run the Dim Date script first to create the DimDate table. Make sure you change the start date and end date on the script to your preference. Then run the add Fiscal Dates scripts to add the fiscal columns. Make sure you alter the Fiscal script to set the date offset amount. The comments in the script will help you with this.

This zip file contains three SQL scripts.

Create Dim Date

Create Dim Time

Add Fiscal Dates

These will create a Date Dimension table and allow you to run the add fiscal script to add the fiscal columns if you desire. The Create Dim Time will create a time dimension with every second of the day for those that need actual time analysis of your data.

Make sure you set the start date and end date in the create dim date script. Set the dateoffset in the fiscal script.

Download the script here:


Read more

SSIS–Um…. How To Preserve Umlauts from UTF-8 Flat File in Düssledorf? (or anywhere else for that matter…)

  • 3 May 2012
  • Author: Mike Milligan
  • Number of views: 7931

Recently I had the pleasure of pulling in a flat file encoded in UTF-8 format into a SQL Server 2008 table using SSIS.  I'd like to share my experience in the hopes that it will help someone else with a similar issue.  It might also help me in the event I come across the situation again and forget how to get past it.


The first indication of a problem was the following error message: 

The data conversion for column "Column 37" returned status value 4 and status text "Text was truncated or one or more characters had no match in the target code page."

The SSIS output also told me what row of data was causing the problem:

" on data row 1415."   (search for "on data row" if you need to find out what row your process failed.)

I verified that the column was wide enough so I opened up EditPadLite and loaded the offending file.  I pressed control G to go to a specific row and saw that the row had a column with the text: Düssledorf.  Without going into a bunch of detail of all the things I tried; but, didn't work, I will explain what finally allowed me to bring Düssledorf into the table with umlaut preserved.

These are the types of errors I was getting as I struggled to EDIT the components:

The column "Column 2" cannot be processed because more than one code page (65001 and 1252) are specified for it.

Codepage in output column 0 is 1252, but has to be 65001.

1.  EditPadLite: clicked Convert, Text encoding from menu.



This told me the file's encoding was Unicode UTF-8.



The next steps have to be followed exactly and in the correct order.  There is no editing about it.  If you already have ANY of these three objects they MUST be deleted and re-created from scratch:  Flat File connection manager, flat file source, OLE DB Destination.  Seriously, no joke.

Note:  Before you delete your Destination you may want to sort the columns by the destination column and take screenshots if your incoming columns are listed as column 1, column 2, etc. because you will have to remap them.

 1.  Create your flat file connection manager using code page 65001 (UTF-8) and on the advanced tab change the datatype property of any columns that contain umlauts to unicode string dt_wstr or dt_ntext for Unicode text stream.

 2.  Create your Flat file source component

 3.  Create your ole db destination component.  Change defaultCodePage property to 65001 and AlwaysUseDefaultCodePage to True.  Hook your source to destination and do your mappings.  (Your target column needs to be nvarchar or another unicode capable data type.)

 Sounds simple; but, believe me if one step is out of order you simply can't edit it and you will go out of your mind trying to.  You HAVE to do it in the exact order I have described.

More info

Best Fit in WideCharToMultiByte and System.Text.Encoding Should be Avoided.

BIDS 2008 R2 inserts phantom codepage(s) into SSIS components irrelevant to format/locales/codepages of data and software used (blocking the SSIS tasks

Flatfile Import: Persistent data conversion errors

Categories: Blogs
Rate this article:
No rating

Mike MilliganMike Milligan

Other posts by Mike Milligan

Please login or register to post comments.