Subscribe
Categories
Archives
- October 2022
- July 2020
- May 2020
- September 2019
- April 2019
- December 2017
- October 2017
- September 2017
- August 2017
- July 2017
- June 2017
- May 2017
- April 2017
- March 2017
- January 2017
- December 2016
- November 2016
- October 2016
- July 2016
- June 2016
- May 2016
- April 2016
- March 2016
- February 2016
- December 2015
- November 2015
- October 2015
- September 2015
- August 2015
- July 2015
- June 2015
- May 2015
- April 2015
- December 2014
- November 2014
- October 2014
- September 2014
- February 2014
- January 2014
- July 2013
- June 2013
- May 2013
- April 2013
- March 2013
- October 2012
- September 2012
- August 2012
- May 2012
- March 2012
- November 2011
- September 2011
- August 2011
- June 2011
- May 2011
- March 2011
- February 2011
- January 2011
- December 2010
- July 2010
- May 2010
- February 2010
- January 2010
- December 2009
Meta
Real Time Example for SSIS Transformations
Real Time Example for SSIS Transformations
Introduction:
List of 28 Transformations are as follows
S No
|
Transformation
|
Real Time Examples
|
1
|
Aggregate | Summing / Averaging a total of the products purchased by a customer online to produce the final amount. |
2
|
Audit | For audit purpose, when we need to audit the logs to send to DBA’s for weekly or monthly auditing. |
3
|
Character Map | For sending mails (do some manipulations) to the end users to do some formatting we can use this task. |
4
|
Conditional Split | Morning feeds which we get from different systems need to be transferred to different tables based on the feed which we get so we can use this task to do some condition check. |
5
|
Copy Column | Morning feeds which needs to be transferred to tables need to be scanned under for cleaning spaces, empty values etc then we can go with this task |
6
|
Data Conversion | Daily monitoring of the input files and data to have proper datatype before mapping it to the table then we can use this task. |
7
|
Data Mining Query | Evaluating the input data against the analysis model to get a proper set. |
8
|
Derived Column | Adding a title of courtesy (Mr., Mrs., Dr, etc) before the name and removing the trailing and ending spaces. |
9
|
Export Column | When we get the normal files/pdf files/image files from different systems and save it under a particular folder and map it to the table master |
10
|
Fuzzy Grouping | Matching the name of a customer with master and child table and use it to group and get the desired set |
11
|
Fuzzy Lookup | Matching the name of a customer with master and child table and use it to group and get the desired set |
12
|
Import Column | When we get the normal files/pdf files/image files from different systems and save it under a particular folder and map it to the table master |
13
|
Lookup | Employee table information saved in a master file and the region wise data available across the table which can be mapped and joined to perform a joined querying operation |
14
|
Merge | Combine data from multiple data source like master and child employee table and get result in single dataset. |
15
|
Merge Join | Combine data from multiple data source like master and child employee table and get result in single dataset. Can use any type of join like inner, outer, left , right etc |
16
|
Multicast | Similar to the conditional split but this splits across all the parts |
17
|
OLE DB Command | Used when we need to do updates to all the rows of a table like update If a message sent to the entire customer who have made a payment today. |
18
|
Percentage Sampling | Can be used in cases like the package should have access to only limited data. |
19
|
Pivot | When data fetched from the table and do some formatting to show in the front end we can use it. |
20
|
Row Count | Any point to log the count of the number of customers so we can get the count using this |
21
|
Row Sampling | Same as Percentage Sampling. |
22
|
Script Component | Used for places where we need to use framework specific assemblies. |
23
|
Slowly Changing Dimension | When we need to use some historic dimensions of data |
24
|
Sort | To make some sorting to get the desired result. Sorting like customer who made the highest payment in a particular day. |
25
|
Term Extraction | Used to get a data from a large set of data and get the extracted output in a formatted set. |
26
|
Term Lookup | Used to get a data from a large set of data and get the extracted output in a formatted set. |
27
|
Union All | Used to get data from different data sources and get in a single dimensional format. |
28
|
Unpivot | Restructuring the format of the data for normalizing the input prior to loading. |
Conclusion:
We have seen some real time examples for SSIS Transformations where we use the transformations, these are some of the real time usage which I came across.
How To Monitor Replication in SQL Server
How To Monitor Replication in SQL Server
This post takes you through the T-SQL script How to Monitor Replication in SQL Server. Usually we need to monitor replication periodically through out the day. I have designed a solution to monitor replication form T-SQL. It’ll send a mail to DBA team that replication is failing. Create the below two objects (Table and PROC) and create a job which calls the stored procedure for every 5 minutes.
This will save us a lot of time and effort for sure
Now here is the script to automate replication monitoring in sql server
Create a Table and a stored procedure. Please have a look at procedure code, all sections are provided with required comments.
Table Creation
/***** Create RepliMonitor Table *****/ CREATE TABLE dbo.repmonitor ( [ID] INT NOT NULL IDENTITY, [status] int null, warning int null , subscriber sysname null , subscriber_db sysname null , publisher_db sysname null , publication sysname null , publication_type int null , subtype int null , latency int null , latencythreshold int null , agentnotrunning int null , agentnotrunningthreshold int null , timetoexpiration int null , expirationthreshold int null , last_distsync datetime null , distribution_agentname sysname null , mergeagentname sysname null , mergesubscriptionfriendlyname sysname null , mergeagentlocation sysname null , mergeconnectiontype int null , mergePerformance int null , mergerunspeed float null , mergerunduration int null , monitorranking int null , distributionagentjobid binary(30) null , mergeagentjobid binary(30) null , distributionagentid int null , distributionagentprofileid int null , mergeagentid int null , mergeagentprofileid int null , logreaderagentname sysname null );
Stored Procedure Creation:
/*** Create the SPOC on Monitor Database ***/ /*** It requires the Publisher Name as parameter ***/ /*** Execute the Stored Procedure EXEC usp_replmonitor_Create @Publisher_Name = 'Repli_Publisher_01' @DBProfile = 'ProdMailPro', @Email = 'dbateam@company.com'; ***/ CREATE PROC usp_replmonitor_Create( @Publisher_Name VARCHAR(200), @DBProfile VARCHAR(100) = NULL, @Email VARCHAR(1000) ) AS BEGIN SET NOCOUNT ON DECLARE @Pub_Type TINYINT; DECLARE @C INT; DECLARE @ReplM TABLE ( [ID] INT NOT NULL IDENTITY, subscriber sysname null, publication sysname null, last_distsync datetime null ); IF (@DBProfile IS NULL) BEGIN SELECT @DBProfile = Name FROM msdb.dbo.sysmail_profile WHERE Profile_ID = 1; END /*** Clears the previous data ***/ TRUNCATE TABLE repmonitor; /*** For Transactional Replication ***/ SET @Pub_Type=0 INSERT INTO repmonitor EXEC [DSADistribution].[DBO].[sp_replmonitorhelpsubscription] @publisher = @Publisher_Name, @publication_type = @Pub_Type /*** For Snapshot Replication ***/ SET @Pub_Type=1 INSERT INTO repmonitor EXEC [DSADistribution].[DBO].[sp_replmonitorhelpsubscription] @publisher = @Publisher_Name, @publication_type = @Pub_Type /*** For Merge Replication ***/ SET @Pub_Type = 2 INSERT INTO repmonitor EXEC [DSADistribution].[DBO].[sp_replmonitorhelpsubscription] @publisher = @Publisher_Name, @publication_type = @Pub_Type /*** Check if any subscription is failing ***/ SET @C=0 SELECT @C=COUNT(1)FROM repmonitor WHERE Status = 6; IF(@C>0) BEGIN DECLARE @String varchar(8000); SET @String='Replication is Failing @ '+@Publisher_Name+' for below Subscribers'+CHAR(13)+CHAR(13); SET @String=@String+'Subscriber'+''+'Publication'+char(13); SELECT @String = @String + subscriber +' '+publication+char(13) FROM repmonitor WHERE Status=6; /*** Notify DBA Group ***/ EXEC msdb.dbo.sp_send_dbmail @profile_name = @DBProfile, @recipients =@Email, @body = @String, @subject ='Replication is Failing'; END END
Here is the script file: monitor_replication
We can schedule a job to execute this stored procedure.
Posted in High Availability, SQL Scripts
Tagged 2008, 2008R2, 2012, how to check replication status in sql server, how to check replication status in sql server 2005, How to monitor replication, how to monitor replication in sql server 2008, SQL Scripts, sql server replication snapshot index script, SQL Tips
4 Comments