Low Level SQL Server Architecture

Low Level SQL Server Architecture

 udayarumilli_sql_server_2012_ssrs

Here I would like to describe the process architecture whan a new request submitted to SQL Server.
I have submitted a Query to SQL Server from an Application and I got the reply as “data inserted successfully”. What are the overall processes worked inside?
At Client:
1. User enter data and click on submit
2. The client database library transforms the original request into a sequence of one or more Transact-SQL statements to be sent to SQL Server. These statements are encapsulated in one or more Tabular Data Stream (TDS) packets and passed to the database network library
3. The database network library uses the network library available in the client computer to repackage the TDS packets as network protocol packets.
4. The network protocol packets are sent to the server computer network library across the network
At Server:
5. The extracted TDS packets are sent to Open Data Services (ODS), where the original query is extracted.
6. ODS sends the query to the relational engine
7. A connection established to the relational engine and assigns a SID to the connection
At Relational Engine:
8. Check permissions and determines if the query can be executed by the user associated with     the request
9.  Query sends to Query Parser
  • It checks that the T-SQL is written correctly
  • Build a Parse Tree \ Sequence Tree
10. Parse Tree sends to Algebrizer
  • Verifies all the columns, objects and data types
  • Aggregate Binding (determines the location of aggregates such as GROUP BY, and MAX)
  • Builds aQuery Processor Tree in Binary Format
11. Query Processor Tree sends to Optimizer
  • Based on the query processor tree and Histogram (Statistics) builds an optimized execution plan
  • Stores the execution plan into cache and send it to the database engine
At Database Engine:
12. Database engine map a batch into different tasks
13. Each task associated with a process
14. Each process assigned with a Windows Thread or a Windows Fiber. The worker thread takes care of this.
15. The Thread/Fiber send to the execution queue and wait for the CPU time.
16. The Thread/Fiber identifies the table location where the data need to be stored
17. Go to the file header, checks the PFS, GAM and GSAM and go to the correct page
18. Verifies the page is not corrupted using Torn page Detection / Check SUM and writes the data
19. If require allocates new pages and stores data on it. Once the data is stored/updated/added in a page, it updates the below locations
  • PFS – Page Free Space
  • Page Header – Checksum / Torn Page Detection (Sector info)
  • BCM – Bulk Change MAP
  • DCM – Differential Change MAP
20. In this process the
  • Memory manager take care of allocating buffers, new pages etc,
  • Lock manager take care of allocating appropriate locks on the objects/pages and releasing them when task completed
  • Thread Scheduler: schedules the threads for CPU time
  • I/O manager: Establish memory bus for read/write operations from memory to disk and vice versa
  • Deadlock\Resource\Scheduler Monitor: Monitors the processes
21. Once the process is completed the result set is submitted to the relational engine and follow the same process for sending back the result set to client application.
22. The connection will be closed and the SID is removed.
This information I have collected from various articles and reading through books online.
If someone wants to add / update can suggest / assist me on this.
You can have a look at here for complete “Low Level SQL Server Architecture
Posted in Database Design, SQL Development, SQL Server DBA | Tagged , , , , , , , , | 2 Comments

Monitor CPU utilization, I/O Usage and Memory Usage in Sql Server

 Monitor CPU utilization, I/O Usage and Memory Usage in Sql Server

udayarumilli_cpu_memory_io_usage

CPU Usage, I/O Usage and Memory Usage of database

 

Database level / Database wise CPU, memory and I/O usage
As part of DBA’s daily checklist, we need to monitor few parameters of a database throughout the day. It includes CPU utilization, Memory utilization and I/O utilization. Here are the T-SQL scripts to monitor sql server instances database wise.

CPU Utilization:

WITH DB_CPU_Stats
AS
(SELECT DatabaseID,
DB_Name(DatabaseID)AS [DatabaseName],
SUM(total_worker_time)AS [CPU_Time(Ms)]
FROM sys.dm_exec_query_stats AS qs
CROSS APPLY(SELECT CONVERT(int, value)AS [DatabaseID]
 FROM sys.dm_exec_plan_attributes(qs.plan_handle)
 WHERE attribute =N'dbid')AS epa
GROUP BY DatabaseID)
SELECT ROW_NUMBER()OVER(ORDER BY [CPU_Time(Ms)] DESC)AS [row_num],
DatabaseName,
[CPU_Time(Ms)],
CAST([CPU_Time(Ms)] * 1.0 /SUM([CPU_Time(Ms)])
OVER()* 100.0 AS DECIMAL(5, 2))AS [CPUPercent]
FROM DB_CPU_Stats
WHERE DatabaseID > 4 -- system databases
AND DatabaseID <> 32767 -- ResourceDB
ORDER BY row_num
OPTION(RECOMPILE);

CPU Utilization History:

The query retrieves the SQL Server instance CPU usage from last 10 minutes.
/***** CPU Utilization history *****/
-- Get CPU Utilization History (SQL Server 2008 and above)
DECLARE @ts BIGINT
SELECT @ts =(SELECT cpu_ticks/(cpu_ticks/ms_ticks)
FROM sys.dm_os_sys_info);
SELECT TOP(10)SQLProcessUtilization AS [SQLServer_Process_CPU_Utilization],
SystemIdle AS [System_Idle_Process],
100 - SystemIdle - SQLProcessUtilization AS [Other_Process_CPU_Utilization],
DATEADD(ms,-1 *(@ts - [timestamp]),GETDATE())AS [Event_Time]
FROM (SELECT record.value('(./Record/@id)[1]','int')AS record_id,
record.value('(./Record/SchedulerMonitorEvent/SystemHealth/SystemIdle)[1]','int')AS [SystemIdle],
record.value('(./Record/SchedulerMonitorEvent/SystemHealth/ProcessUtilization)[1]','int')AS [SQLProcessUtilization],
[timestamp]
     FROM (SELECT[timestamp],
convert(xml, record) AS [record]
            FROM sys.dm_os_ring_buffers
            WHERE ring_buffer_type =N'RING_BUFFER_SCHEDULER_MONITOR'
AND record LIKE'%%')AS x
)AS y
ORDER BY record_id DESC;
--<strong> Get CPU Utilization History (SQL 2005 Only)</strong>
DECLARE  @ts_now BIGINT;
SELECT   @ts_now = cpu_ticks /CONVERT(float, cpu_ticks_in_ms)
FROM  sys.dm_os_sys_info
SELECT  TOP(10)SQLProcessUtilization AS [SQL_Server_Process_CPU_Utilization],
SystemIdle AS [System_Idle_Process],
100 - SystemIdle - SQLProcessUtilization AS [Other_Process_CPU_Utilization],
DATEADD(ms,-1 *(@ts_now - [timestamp]),GETDATE())AS [Event_Time]
FROM (SELECT  record.value('(./Record/@id)[1]','int')AS record_id,
record.value('(./Record/SchedulerMonitorEvent/SystemHealth/SystemIdle)[1]','int')AS [SystemIdle],
record.value('(./Record/SchedulerMonitorEvent/SystemHealth/ProcessUtilization)[1]','int')AS [SQLProcessUtilization],
[timestamp]
     FROM (SELECT [timestamp],
CONVERT(xml, record)AS [record]
           FROM sys.dm_os_ring_buffers
           WHERE ring_buffer_type =N'RING_BUFFER_SCHEDULER_MONITOR'
AND record LIKE'%%')AS x
          )AS y
ORDER BY record_id DESC;
SQLServer_Process_CPU_Utilization: Percentage of CPU utilizing by SQL Server instance
Sytem_Idle_Process:  Percentage of CPU is idle
Other_Process_CPU_Utilization: Percentage of CPU utilizing by all other processes other than SQL Server instance.  
Event_Time: Time when these details has been captured

I/O Usage by database:

/****** I/O Usage by database ****/
SELECT Name AS'Database Name'
,SUM(num_of_reads)AS'Number of Reads'
,SUM(num_of_writes)AS'Number of Writes'
FROM sys.dm_io_virtual_file_stats(NULL,NULL) I
INNER JOIN sys.databases D ON I.database_id = d.database_id
GROUP BY Name
ORDER BY 'Number of Reads'
DESC;

Memory usage by Database:

DECLARE @total_buffer INT;
SELECT @total_buffer = cntr_value
FROM sys.dm_os_performance_counters
WHERE RTRIM([object_name])LIKE'%Buffer Manager'
AND counter_name ='Total Pages';
;WITH src AS
(SELECT database_id,
db_buffer_pages =COUNT_BIG(*)
FROM sys.dm_os_buffer_descriptors
      --WHERE database_id BETWEEN 5 AND 32766
GROUP BY database_id
)
SELECT [db_name] = CASE [database_id] WHEN 32767 THEN 'Resource DB' ELSE DB_NAME([database_id])END,
db_buffer_pages,
db_buffer_MB = db_buffer_pages / 128,
db_buffer_percent =CONVERT(DECIMAL(6,3),
db_buffer_pages * 100.0 / @total_buffer)
FROM src
ORDER BY db_buffer_MB DESC;

Posted in Performance Tuning, SQL Server DBA | Tagged , , , , , , , , , , , , , | 1 Comment

How to control windows services from SQL Server?

How to control windows services from SQL Server?

For a SQL Server DBA there are scenarios where he/she needs to deal with Windows services from SQL Server when there is no direct access to Windows server and only have access to SQL Server.
Monitor SQL Server services using T-SQL
 
XP_SERVICECONTROL:
It helps DBA to know the status and control the windows services from SQL Server.

Syntax:  XP_SERVICECONTROL ,

Action:  There are total 5 actions can be performed on services.

 

  • Start: To start a service
  • Stop: To stop a service
  • Pause: To pause a service
  • Continue: To start a passed service
  • Querystats: To know the current status of a service

Service Name: Can be any windows service
Example:

— To know the status of SQL Server Agent
EXECMASTER..XP_SERVICECONTROL’QueryState’,’SQLSERVERAGENT’
GO
— Start a service – Postgresql 9.2 database service
EXECMASTER..XP_SERVICECONTROL’Start’,’postgresql-x64-9.2′
GO
— To know the status Distributed Transaction Coordinator
EXECMASTER..XP_SERVICECONTROL’QueryState’,’MSDTC’



— Stop SQL EXPRESS service
EXECMASTER..XP_SERVICECONTROL’Stop’,’MSSQL$SQLEXPRESS’
GO
WAITFORDELAY’00:00:05′—- 5 Second Delay
GO
— Get status SQL EXPRESS service
EXECMASTER..XP_SERVICECONTROL’QueryState’,’MSSQL$SQLEXPRESS’
GO
WAITFORDELAY’00:00:05′—- 5 Second Delay
GO
— Start SQL EXPRESS service
EXECMASTER..XP_SERVICECONTROL’Start’,’MSSQL$SQLEXPRESS’
GO

Note: It really helps a DBA in monitoring windows services and the process can be automated.

 

Posted in SQL Server DBA | Tagged , , , , , , , , | Leave a comment