Notebooks
S
Snowflake Scheduled Query Execution Report

Scheduled Query Execution Report

Scheduled_Query_Execution_Reportdata-sciencenotebookmachine-learningsnowflake-demo-notebooksdata-engineeringPythonsql
alph-notebooks/snowflake-demo-notebooks / Scheduled_Query_Execution_Report.ipynb

Scheduled Query Execution Report

A notebook to report on failed or long-running scheduled queries, providing insights into reliability issues.

Here's a breakdown of the steps:

  1. Retrieve Data
  2. Convert Table to a DataFrame
  3. Create an Interactive Slider Widget & Data Preparation
  4. Create a Heatmap for Visualizing Scheduled Query Execution

1. Retrieve Data

Firstly, we'll write an SQL query to retrieve the execution history for scheduled queries, along with their status, timing metrics, and execution status.

We're obtaining this from the snowflake.account_usage.task_history table.

[ ]

2. Convert Table to a DataFrame

Next, we'll convert the table to a Pandas DataFrame.

[ ]

3. Create an Interactive Slider Widget & Data Preparation

Here, we'll create an interactive slider for dynamically selecting the number of days to analyze. This would then trigger the filtering of the DataFrame to the specified number of days.

Next, we'll reshape the data by calculating the frequency count by hour and task name, which will subsequently be used for creating the heatmap in the next step.

[ ]

4. Create a Heatmap for Visualizing Scheduled Query Execution

Finally, a heatmap and summary statistics table are generated that will allow us to gain insights on the task name and state (e.g. SUCCEEDED, FAILED, SKIPPED).

[ ]

Want to learn more?