3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
EVAL1_SPC1
application UI
Details for Stage 221 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
2 s
Locality Level Summary:
Process local: 26
Input Size / Records:
5.8 MiB / 29428
Shuffle Write Size / Records:
3.2 MiB / 617988
Associated Job Ids:
129
DAG Visualization
digraph G { subgraph clusterstage_221 { label="Stage 221"; subgraph cluster2716 { label="Scan parquet "; 1080 [labelType="html" label="FileScanRDD [1080]<br>takeAsList at DataFrameCommonJob.java:145"]; 1081 [labelType="html" label="MapPartitionsRDD [1081]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2712 { label="WholeStageCodegen (1)"; 1082 [labelType="html" label="MapPartitionsRDD [1082]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2721 { label="Scan parquet "; 1083 [labelType="html" label="FileScanRDD [1083]<br>takeAsList at DataFrameCommonJob.java:145"]; 1084 [labelType="html" label="MapPartitionsRDD [1084]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2717 { label="WholeStageCodegen (2)"; 1085 [labelType="html" label="MapPartitionsRDD [1085]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2726 { label="Scan parquet "; 1086 [labelType="html" label="FileScanRDD [1086]<br>takeAsList at DataFrameCommonJob.java:145"]; 1087 [labelType="html" label="MapPartitionsRDD [1087]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2722 { label="WholeStageCodegen (3)"; 1088 [labelType="html" label="MapPartitionsRDD [1088]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2731 { label="Scan parquet "; 1089 [labelType="html" label="FileScanRDD [1089]<br>takeAsList at DataFrameCommonJob.java:145"]; 1090 [labelType="html" label="MapPartitionsRDD [1090]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2727 { label="WholeStageCodegen (4)"; 1091 [labelType="html" label="MapPartitionsRDD [1091]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2736 { label="Scan parquet "; 1092 [labelType="html" label="FileScanRDD [1092]<br>takeAsList at DataFrameCommonJob.java:145"]; 1093 [labelType="html" label="MapPartitionsRDD [1093]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2732 { label="WholeStageCodegen (5)"; 1094 [labelType="html" label="MapPartitionsRDD [1094]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2741 { label="Scan parquet "; 1095 [labelType="html" label="FileScanRDD [1095]<br>takeAsList at DataFrameCommonJob.java:145"]; 1096 [labelType="html" label="MapPartitionsRDD [1096]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2737 { label="WholeStageCodegen (6)"; 1097 [labelType="html" label="MapPartitionsRDD [1097]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2746 { label="Scan parquet "; 1098 [labelType="html" label="FileScanRDD [1098]<br>takeAsList at DataFrameCommonJob.java:145"]; 1099 [labelType="html" label="MapPartitionsRDD [1099]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2742 { label="WholeStageCodegen (7)"; 1100 [labelType="html" label="MapPartitionsRDD [1100]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2751 { label="Scan parquet "; 1101 [labelType="html" label="FileScanRDD [1101]<br>takeAsList at DataFrameCommonJob.java:145"]; 1102 [labelType="html" label="MapPartitionsRDD [1102]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2747 { label="WholeStageCodegen (8)"; 1103 [labelType="html" label="MapPartitionsRDD [1103]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2756 { label="Scan parquet "; 1104 [labelType="html" label="FileScanRDD [1104]<br>takeAsList at DataFrameCommonJob.java:145"]; 1105 [labelType="html" label="MapPartitionsRDD [1105]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2752 { label="WholeStageCodegen (9)"; 1106 [labelType="html" label="MapPartitionsRDD [1106]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2761 { label="Scan parquet "; 1107 [labelType="html" label="FileScanRDD [1107]<br>takeAsList at DataFrameCommonJob.java:145"]; 1108 [labelType="html" label="MapPartitionsRDD [1108]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2757 { label="WholeStageCodegen (10)"; 1109 [labelType="html" label="MapPartitionsRDD [1109]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2766 { label="Scan parquet "; 1110 [labelType="html" label="FileScanRDD [1110]<br>takeAsList at DataFrameCommonJob.java:145"]; 1111 [labelType="html" label="MapPartitionsRDD [1111]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2762 { label="WholeStageCodegen (11)"; 1112 [labelType="html" label="MapPartitionsRDD [1112]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2771 { label="Scan parquet "; 1113 [labelType="html" label="FileScanRDD [1113]<br>takeAsList at DataFrameCommonJob.java:145"]; 1114 [labelType="html" label="MapPartitionsRDD [1114]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2767 { label="WholeStageCodegen (12)"; 1115 [labelType="html" label="MapPartitionsRDD [1115]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2776 { label="Scan parquet "; 1116 [labelType="html" label="FileScanRDD [1116]<br>takeAsList at DataFrameCommonJob.java:145"]; 1117 [labelType="html" label="MapPartitionsRDD [1117]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2772 { label="WholeStageCodegen (13)"; 1118 [labelType="html" label="MapPartitionsRDD [1118]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2711 { label="Union"; 1119 [labelType="html" label="UnionRDD [1119]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2707 { label="WholeStageCodegen (14)"; 1120 [labelType="html" label="MapPartitionsRDD [1120]<br>takeAsList at DataFrameCommonJob.java:145"]; } subgraph cluster2706 { label="Exchange"; 1121 [labelType="html" label="MapPartitionsRDD [1121]<br>takeAsList at DataFrameCommonJob.java:145"]; } } 1080->1081; 1081->1082; 1083->1084; 1084->1085; 1086->1087; 1087->1088; 1089->1090; 1090->1091; 1092->1093; 1093->1094; 1095->1096; 1096->1097; 1098->1099; 1099->1100; 1101->1102; 1102->1103; 1104->1105; 1105->1106; 1107->1108; 1108->1109; 1110->1111; 1111->1112; 1113->1114; 1114->1115; 1116->1117; 1117->1118; 1082->1119; 1085->1119; 1088->1119; 1091->1119; 1094->1119; 1097->1119; 1100->1119; 1103->1119; 1106->1119; 1109->1119; 1112->1119; 1115->1119; 1118->1119; 1119->1120; 1120->1121; }
Event Timeline
Enable zooming
Tasks: 26. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time